2024-01-15T17:40:39.779 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-15T17:40:39.779 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100 --name rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} --verbose -- /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/orig.config.yaml 2024-01-15T17:40:39.821 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-15T17:40:39.898 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100 branch: rishabh-clone-stats description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} email: null first_in_suite: false job_id: '7517100' kernel: flavor: default kdb: true sha1: 2668949e0971849893a3fb7f177ce9032fc65780 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-clone-stats ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-clone-stats sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4427 sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-clone-stats suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi026.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKC4M7/gUHS0qgi/R2ONHIyBfj0g7m7E7n2NR4YtUuMPa7fYJ+XHdPpjCv5mpg7wZvsKv3pKjLthZdgU6MmnXW4= smithi124.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMnT11QoYwAOugPZL1E5SnA/BOcm9WyIhGutN3SBvKT0jssJcwBm3ezEJRkCO7YICWbWBHF3rH5E87M7LpTyhsQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestCloneProgress teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: cd45576300487d997e5a85abed65500b9f5d143b timestamp: 2024-01-15_17:35:13 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-15T17:40:39.898 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa; will attempt to use it 2024-01-15T17:40:39.899 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks 2024-01-15T17:40:39.899 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-15T17:40:39.900 INFO:teuthology.task.internal:Checking packages... 2024-01-15T17:40:39.924 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '95c90f413c4bb65e3e56909855f2308fd9bda05b' 2024-01-15T17:40:39.924 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-15T17:40:39.924 INFO:teuthology.packaging:ref: None 2024-01-15T17:40:39.924 INFO:teuthology.packaging:tag: None 2024-01-15T17:40:39.924 INFO:teuthology.packaging:branch: rishabh-clone-stats 2024-01-15T17:40:39.924 INFO:teuthology.packaging:sha1: 95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T17:40:39.924 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-clone-stats 2024-01-15T17:40:40.305 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7629.g95c90f41 2024-01-15T17:40:40.306 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-15T17:40:40.313 INFO:teuthology.task.internal:no buildpackages task found 2024-01-15T17:40:40.313 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-15T17:40:40.318 INFO:teuthology.task.internal:Saving configuration 2024-01-15T17:40:40.332 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-15T17:40:40.338 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-15T17:40:40.360 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-15 17:35:49.483362', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKC4M7/gUHS0qgi/R2ONHIyBfj0g7m7E7n2NR4YtUuMPa7fYJ+XHdPpjCv5mpg7wZvsKv3pKjLthZdgU6MmnXW4='} 2024-01-15T17:40:40.377 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-15 17:35:49.481638', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMnT11QoYwAOugPZL1E5SnA/BOcm9WyIhGutN3SBvKT0jssJcwBm3ezEJRkCO7YICWbWBHF3rH5E87M7LpTyhsQ='} 2024-01-15T17:40:40.377 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-15T17:40:40.382 INFO:teuthology.task.internal:roles: ubuntu@smithi026.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-01-15T17:40:40.382 INFO:teuthology.task.internal:roles: ubuntu@smithi124.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-01-15T17:40:40.382 INFO:teuthology.run_tasks:Running task console_log... 2024-01-15T17:40:40.456 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8c71747160>, signals=[15]) 2024-01-15T17:40:40.457 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-15T17:40:40.462 INFO:teuthology.task.internal:Opening connections... 2024-01-15T17:40:40.463 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:40:40.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:40:40.569 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:40:40.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:40:40.654 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-15T17:40:40.661 DEBUG:teuthology.orchestra.run.smithi026:> uname -m 2024-01-15T17:40:40.679 INFO:teuthology.orchestra.run.smithi026.stdout:x86_64 2024-01-15T17:40:40.679 DEBUG:teuthology.orchestra.run.smithi026:> cat /etc/os-release 2024-01-15T17:40:40.736 INFO:teuthology.orchestra.run.smithi026.stdout:NAME="CentOS Stream" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION="9" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:ID="centos" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:ID_LIKE="rhel fedora" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:VERSION_ID="9" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:PLATFORM_ID="platform:el9" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:ANSI_COLOR="0;31" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:LOGO="fedora-logo-icon" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:HOME_URL="https://centos.org/" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-15T17:40:40.737 INFO:teuthology.orchestra.run.smithi026.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-15T17:40:40.738 INFO:teuthology.lock.ops:Updating smithi026.front.sepia.ceph.com on lock server 2024-01-15T17:40:40.765 DEBUG:teuthology.orchestra.run.smithi124:> uname -m 2024-01-15T17:40:40.782 INFO:teuthology.orchestra.run.smithi124.stdout:x86_64 2024-01-15T17:40:40.782 DEBUG:teuthology.orchestra.run.smithi124:> cat /etc/os-release 2024-01-15T17:40:40.839 INFO:teuthology.orchestra.run.smithi124.stdout:NAME="CentOS Stream" 2024-01-15T17:40:40.839 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION="9" 2024-01-15T17:40:40.839 INFO:teuthology.orchestra.run.smithi124.stdout:ID="centos" 2024-01-15T17:40:40.839 INFO:teuthology.orchestra.run.smithi124.stdout:ID_LIKE="rhel fedora" 2024-01-15T17:40:40.839 INFO:teuthology.orchestra.run.smithi124.stdout:VERSION_ID="9" 2024-01-15T17:40:40.839 INFO:teuthology.orchestra.run.smithi124.stdout:PLATFORM_ID="platform:el9" 2024-01-15T17:40:40.839 INFO:teuthology.orchestra.run.smithi124.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-15T17:40:40.839 INFO:teuthology.orchestra.run.smithi124.stdout:ANSI_COLOR="0;31" 2024-01-15T17:40:40.839 INFO:teuthology.orchestra.run.smithi124.stdout:LOGO="fedora-logo-icon" 2024-01-15T17:40:40.840 INFO:teuthology.orchestra.run.smithi124.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-15T17:40:40.840 INFO:teuthology.orchestra.run.smithi124.stdout:HOME_URL="https://centos.org/" 2024-01-15T17:40:40.840 INFO:teuthology.orchestra.run.smithi124.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-15T17:40:40.840 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-15T17:40:40.840 INFO:teuthology.orchestra.run.smithi124.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-15T17:40:40.840 INFO:teuthology.lock.ops:Updating smithi124.front.sepia.ceph.com on lock server 2024-01-15T17:40:40.865 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-15T17:40:40.874 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-15T17:40:40.927 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-15T17:40:40.927 DEBUG:teuthology.orchestra.run.smithi026:> test '!' -e /home/ubuntu/cephtest 2024-01-15T17:40:40.929 DEBUG:teuthology.orchestra.run.smithi124:> test '!' -e /home/ubuntu/cephtest 2024-01-15T17:40:40.944 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-15T17:40:40.950 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-15T17:40:40.950 DEBUG:teuthology.orchestra.run.smithi026:> test -z $(ls -A /var/lib/ceph) 2024-01-15T17:40:40.987 DEBUG:teuthology.orchestra.run.smithi124:> test -z $(ls -A /var/lib/ceph) 2024-01-15T17:40:41.006 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-15T17:40:41.060 INFO:teuthology.run_tasks:Running task kernel... 2024-01-15T17:40:41.074 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'} 2024-01-15T17:40:41.074 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}}, timeout 300 2024-01-15T17:40:41.075 DEBUG:teuthology.orchestra.run.smithi026:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-15T17:40:41.075 DEBUG:teuthology.orchestra.run.smithi124:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-15T17:40:41.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:40:41.091 DEBUG:teuthology.task.kernel:sha1 for mon.b is 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:40:41.091 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "2668949e0971849893a3fb7f177ce9032fc65780"... 2024-01-15T17:40:41.091 DEBUG:teuthology.orchestra.run.smithi124:> uname -r 2024-01-15T17:40:41.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:40:41.092 DEBUG:teuthology.task.kernel:sha1 for mon.a is 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:40:41.092 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "2668949e0971849893a3fb7f177ce9032fc65780"... 2024-01-15T17:40:41.093 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-01-15T17:40:41.147 INFO:teuthology.orchestra.run.smithi124.stdout:5.14.0-361.el9.x86_64 2024-01-15T17:40:41.147 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:40:41.147 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-15T17:40:41.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:40:41.151 INFO:teuthology.orchestra.run.smithi026.stdout:5.14.0-361.el9.x86_64 2024-01-15T17:40:41.151 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:40:41.152 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-15T17:40:41.152 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:40:41.292 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum upgrade -y linux-firmware 2024-01-15T17:40:41.466 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum upgrade -y linux-firmware 2024-01-15T17:40:43.715 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - AppStream 9.8 MB/s | 6.4 MB 00:00 2024-01-15T17:40:43.795 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS-9 - AppStream 7.3 MB/s | 6.4 MB 00:00 2024-01-15T17:40:45.532 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - BaseOS 3.1 MB/s | 2.0 MB 00:00 2024-01-15T17:40:45.553 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS-9 - BaseOS 2.9 MB/s | 2.0 MB 00:00 2024-01-15T17:40:49.239 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 2.4 MB/s | 7.9 MB 00:03 2024-01-15T17:40:49.577 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 7.9 MB 00:03 2024-01-15T17:40:51.429 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - AppStream 24 MB/s | 18 MB 00:00 2024-01-15T17:40:55.690 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 3.5 MB/s | 18 MB 00:05 2024-01-15T17:40:58.804 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 5.9 MB 00:03 2024-01-15T17:41:00.148 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 5.9 MB 00:00 2024-01-15T17:41:01.623 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - Extras packages 8.7 kB/s | 15 kB 00:01 2024-01-15T17:41:01.682 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-01-15T17:41:06.591 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 20 MB 00:04 2024-01-15T17:41:08.609 INFO:teuthology.orchestra.run.smithi124.stdout:Extra Packages for Enterprise Linux 2.9 MB/s | 20 MB 00:06 2024-01-15T17:41:11.567 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-01-15T17:41:13.506 INFO:teuthology.orchestra.run.smithi124.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-01-15T17:41:13.908 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-01-15T17:41:13.917 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-01-15T17:41:13.917 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-01-15T17:41:13.917 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-01-15T17:41:13.917 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-01-15T17:41:13.917 INFO:teuthology.orchestra.run.smithi026.stdout: linux-firmware noarch 20240109-142.el9 CentOS-BaseOS 375 M 2024-01-15T17:41:13.918 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:41:13.918 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-01-15T17:41:13.918 INFO:teuthology.orchestra.run.smithi026.stdout:================================================================================ 2024-01-15T17:41:13.918 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 1 Package 2024-01-15T17:41:13.918 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:41:13.918 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 375 M 2024-01-15T17:41:13.918 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-01-15T17:41:15.885 INFO:teuthology.orchestra.run.smithi124.stdout:Dependencies resolved. 2024-01-15T17:41:15.894 INFO:teuthology.orchestra.run.smithi124.stdout:================================================================================ 2024-01-15T17:41:15.894 INFO:teuthology.orchestra.run.smithi124.stdout: Package Arch Version Repository Size 2024-01-15T17:41:15.894 INFO:teuthology.orchestra.run.smithi124.stdout:================================================================================ 2024-01-15T17:41:15.894 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrading: 2024-01-15T17:41:15.895 INFO:teuthology.orchestra.run.smithi124.stdout: linux-firmware noarch 20240109-142.el9 CentOS-BaseOS 375 M 2024-01-15T17:41:15.895 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:41:15.895 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction Summary 2024-01-15T17:41:15.895 INFO:teuthology.orchestra.run.smithi124.stdout:================================================================================ 2024-01-15T17:41:15.895 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrade 1 Package 2024-01-15T17:41:15.896 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:41:15.896 INFO:teuthology.orchestra.run.smithi124.stdout:Total download size: 375 M 2024-01-15T17:41:15.896 INFO:teuthology.orchestra.run.smithi124.stdout:Downloading Packages: 2024-01-15T17:41:27.307 INFO:teuthology.orchestra.run.smithi124.stdout:linux-firmware-20240109-142.el9.noarch.rpm 33 MB/s | 375 MB 00:11 2024-01-15T17:41:27.308 INFO:teuthology.orchestra.run.smithi124.stdout:-------------------------------------------------------------------------------- 2024-01-15T17:41:27.308 INFO:teuthology.orchestra.run.smithi124.stdout:Total 33 MB/s | 375 MB 00:11 2024-01-15T17:41:27.309 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction check 2024-01-15T17:41:27.481 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction check succeeded. 2024-01-15T17:41:27.481 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction test 2024-01-15T17:41:29.924 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction test succeeded. 2024-01-15T17:41:29.924 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction 2024-01-15T17:41:34.965 INFO:teuthology.orchestra.run.smithi026.stdout:linux-firmware-20240109-142.el9.noarch.rpm 18 MB/s | 375 MB 00:21 2024-01-15T17:41:34.966 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-01-15T17:41:34.967 INFO:teuthology.orchestra.run.smithi026.stdout:Total 18 MB/s | 375 MB 00:21 2024-01-15T17:41:34.967 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-01-15T17:41:35.069 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-01-15T17:41:35.069 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-01-15T17:41:36.135 INFO:teuthology.orchestra.run.smithi124.stdout: Preparing : 1/1 2024-01-15T17:41:36.419 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-15T17:41:36.515 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-15T17:41:37.598 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-01-15T17:41:37.599 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-01-15T17:41:37.629 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-01-15T17:41:37.629 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-15T17:41:38.020 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-15T17:41:38.020 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:41:38.020 INFO:teuthology.orchestra.run.smithi124.stdout:Upgraded: 2024-01-15T17:41:38.020 INFO:teuthology.orchestra.run.smithi124.stdout: linux-firmware-20240109-142.el9.noarch 2024-01-15T17:41:38.021 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:41:38.021 INFO:teuthology.orchestra.run.smithi124.stdout:Complete! 2024-01-15T17:41:38.312 INFO:teuthology.task.kernel:Downloading kernel 2668949e0971849893a3fb7f177ce9032fc65780 on mon.b... 2024-01-15T17:41:38.313 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:41:38.666 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/ 2024-01-15T17:41:38.667 DEBUG:teuthology.orchestra.run.smithi124:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_g2668949e0971-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/ --input-file=- 2024-01-15T17:41:38.670 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi124.front.sepia.ceph.com to complete... 2024-01-15T17:41:44.045 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-01-15T17:41:44.310 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-15T17:41:44.399 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-15T17:41:45.319 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-01-15T17:41:45.320 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-15T17:41:45.740 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-15T17:41:45.740 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:41:45.740 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-01-15T17:41:45.740 INFO:teuthology.orchestra.run.smithi026.stdout: linux-firmware-20240109-142.el9.noarch 2024-01-15T17:41:45.740 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:41:45.740 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-01-15T17:41:46.067 INFO:teuthology.task.kernel:Downloading kernel 2668949e0971849893a3fb7f177ce9032fc65780 on mon.a... 2024-01-15T17:41:46.068 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2668949e0971849893a3fb7f177ce9032fc65780 2024-01-15T17:41:46.342 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/ 2024-01-15T17:41:46.342 DEBUG:teuthology.orchestra.run.smithi026:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_g2668949e0971-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/ --input-file=- 2024-01-15T17:41:46.345 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi026.front.sepia.ceph.com to complete... 2024-01-15T17:42:41.314 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15 17:42:41 URL:https://2.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/kernel-6.7.0_g2668949e0971-1.x86_64.rpm [330211636/330211636] -> "/tmp/kernel.x86_64.rpm" [1] 2024-01-15T17:42:41.315 INFO:teuthology.orchestra.run.smithi124.stderr:FINISHED --2024-01-15 17:42:41-- 2024-01-15T17:42:41.315 INFO:teuthology.orchestra.run.smithi124.stderr:Total wall clock time: 1m 3s 2024-01-15T17:42:41.315 INFO:teuthology.orchestra.run.smithi124.stderr:Downloaded: 1 files, 315M in 1m 2s (5.07 MB/s) 2024-01-15T17:42:41.318 INFO:teuthology.task.kernel:Installing kernel 2668949e0971849893a3fb7f177ce9032fc65780 on mon.b... 2024-01-15T17:42:41.319 DEBUG:teuthology.orchestra.run.smithi124:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-01-15T17:42:43.206 INFO:teuthology.orchestra.run.smithi124.stdout:Verifying... ######################################## 2024-01-15T17:42:43.279 INFO:teuthology.orchestra.run.smithi124.stdout:Preparing... ######################################## 2024-01-15T17:42:43.358 INFO:teuthology.orchestra.run.smithi124.stdout:Updating / installing... 2024-01-15T17:42:50.675 INFO:teuthology.orchestra.run.smithi124.stdout:kernel-6.7.0_g2668949e0971-1 ######################################## 2024-01-15T17:42:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15 17:42:59 URL:https://2.chacra.ceph.com/r/kernel/testing/2668949e0971849893a3fb7f177ce9032fc65780/centos/9/flavors/default/x86_64/kernel-6.7.0_g2668949e0971-1.x86_64.rpm [330211636/330211636] -> "/tmp/kernel.x86_64.rpm" [1] 2024-01-15T17:42:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:FINISHED --2024-01-15 17:42:59-- 2024-01-15T17:42:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:Total wall clock time: 1m 13s 2024-01-15T17:42:59.693 INFO:teuthology.orchestra.run.smithi026.stderr:Downloaded: 1 files, 315M in 1m 13s (4.32 MB/s) 2024-01-15T17:42:59.696 INFO:teuthology.task.kernel:Installing kernel 2668949e0971849893a3fb7f177ce9032fc65780 on mon.a... 2024-01-15T17:42:59.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-01-15T17:43:01.695 INFO:teuthology.orchestra.run.smithi026.stdout:Verifying... ######################################## 2024-01-15T17:43:01.754 INFO:teuthology.orchestra.run.smithi026.stdout:Preparing... ######################################## 2024-01-15T17:43:01.838 INFO:teuthology.orchestra.run.smithi026.stdout:Updating / installing... 2024-01-15T17:43:04.721 INFO:teuthology.orchestra.run.smithi124.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-15T17:43:09.542 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-6.7.0_g2668949e0971-1 ######################################## 2024-01-15T17:43:23.896 INFO:teuthology.orchestra.run.smithi026.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-15T17:43:26.852 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi124.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-01-15T17:43:26.853 DEBUG:teuthology.orchestra.run.smithi124:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-01-15T17:43:26.942 INFO:teuthology.orchestra.run.smithi124.stdout:/boot/System.map-6.7.0-g2668949e0971 2024-01-15T17:43:26.942 INFO:teuthology.orchestra.run.smithi124.stdout:/boot/config-6.7.0-g2668949e0971 2024-01-15T17:43:26.942 INFO:teuthology.orchestra.run.smithi124.stdout:/boot/vmlinuz-6.7.0-g2668949e0971 2024-01-15T17:43:26.942 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-15T17:43:26.943 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32c-intel.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-15T17:43:26.944 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm.ko 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/ansi_cprng.ko 2024-01-15T17:43:26.945 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/anubis.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/arc4.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_memcpy.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_pq.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_tx.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_xor.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/raid6test.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/authenc.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/authencesn.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blake2b_generic.ko 2024-01-15T17:43:26.946 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_common.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_generic.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/camellia_generic.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast5_generic.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast6_generic.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast_common.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/ccm.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cmac.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crc32_generic.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_engine.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_user.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cts.ko 2024-01-15T17:43:26.947 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/des_generic.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/dh_generic.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/echainiv.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/essiv.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/fcrypt.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/khazad.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4hc.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/md4.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/michael_mic.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcbc.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcrypt.ko 2024-01-15T17:43:26.948 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/rmd160.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/seed.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/serpent_generic.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/tcrypt.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/tea.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_common.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_generic.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/vmac.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/wp512.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xcbc.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xor.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xxhash_generic.ko 2024-01-15T17:43:26.949 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_pad.ko 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/custom_method.ko 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/ec_sys.ko 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/platform_profile.ko 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbs.ko 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbshc.ko 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/video.ko 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/acard-ahci.ko 2024-01-15T17:43:26.950 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ahci_platform.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ata_generic.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/libahci_platform.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_acpi.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ali.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_amd.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_artop.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atiixp.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atp867x.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd640.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd64x.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cypress.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_efar.ko 2024-01-15T17:43:26.951 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt366.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt37x.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it8213.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it821x.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_jmicron.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_marvell.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_mpiix.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_netcell.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ninja32.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87410.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87415.ko 2024-01-15T17:43:26.952 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_oldpiix.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_opti.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_optidma.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pcmcia.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_piccolo.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_rdc.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sch.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_serverworks.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sil680.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sis.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sl82c105.ko 2024-01-15T17:43:26.953 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_triflex.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_via.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pdc_adma.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_inic162x.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_mv.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_nv.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_promise.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_qstor.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil24.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sis.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_svw.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sx4.ko 2024-01-15T17:43:26.954 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_uli.ko 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_via.ko 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_vsc.ko 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay/ks0108.ko 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap/regmap-i2c.ko 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap/regmap-mmio.ko 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma/bcma.ko 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block 2024-01-15T17:43:26.955 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe/aoe.ko 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/brd.ko 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd/drbd.ko 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/floppy.ko 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/loop.ko 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/nbd.ko 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk/null_blk.ko 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/pktcdvd.ko 2024-01-15T17:43:26.956 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/rbd.ko 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/virtio_blk.ko 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkfront.ko 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram/zram.ko 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hangcheck-timer.ko 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-15T17:43:26.957 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/via-rng.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/lp.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave/mwave.ko 2024-01-15T17:43:26.958 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ppdev.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tlclk.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_tis.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/uv_mmtimer.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/virtio_console.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-01-15T17:43:26.959 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/powernow-k8.ko 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp.ko 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common 2024-01-15T17:43:26.960 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-aes.ko 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-sha.ko 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca/dca.ko 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/devfreq 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw 2024-01-15T17:43:26.961 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/amd64_edac.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/e752x_edac.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/edac_mce_amd.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3000_edac.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3200_edac.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5100_edac.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5400_edac.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7300_edac.ko 2024-01-15T17:43:26.962 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7core_edac.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i82975x_edac.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/ie31200_edac.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/sb_edac.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/x38_edac.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/edd.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-ich.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-appleir.ko 2024-01-15T17:43:26.963 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-aureal.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-axff.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-dr.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elecom.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elo.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-emsff.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gaff.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gt683r.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gyration.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtekff.ko 2024-01-15T17:43:26.964 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-hyperv.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-icade.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-keytouch.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-kye.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lcpower.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-led.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lenovo.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-multitouch.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ortek.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-penmount.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-petalynx.ko 2024-01-15T17:43:26.965 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-picolcd.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-pl.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-plantronics.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-primax.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-rmi.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-common.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-15T17:43:26.966 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-saitek.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-samsung.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sjoy.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sony.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-speedlink.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-steelseries.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sunplus.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-15T17:43:26.967 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tivo.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tmff.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-topseed.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-twinhan.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-uclogic.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-waltop.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-wiimote.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-xinmo.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zpff.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zydacron.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/uhid.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/wacom.ko 2024-01-15T17:43:26.968 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_balloon.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_utils.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_vmbus.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru3.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7414.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7418.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adc128d818.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1025.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1026.ko 2024-01-15T17:43:26.969 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1029.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1031.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm9240.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ads7828.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7410.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7411.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7462.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7470.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7475.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7x10.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/amc6821.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/applesmc.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asb100.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asc7621.ko 2024-01-15T17:43:26.970 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/atxp1.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/coretemp.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dme1737.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds1621.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds620.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc1403.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc6w201.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71805f.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71882fg.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f75375s.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fam15h_power.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fschmd.ko 2024-01-15T17:43:26.971 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g760a.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g762.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl518sm.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl520sm.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5500_temp.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5k_amb.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmaem.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmpex.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina209.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina2xx.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/it87.ko 2024-01-15T17:43:26.972 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k10temp.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k8temp.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lineage-pem.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm63.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm73.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm75.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm77.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm78.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm80.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm83.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm85.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm87.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm90.ko 2024-01-15T17:43:26.973 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm92.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm93.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95234.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95241.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95245.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc2945.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4151.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4215.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4222.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4245.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4260.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4261.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max16065.ko 2024-01-15T17:43:26.974 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1619.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1668.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max197.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6639.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6650.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6697.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/mcp3021.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6683.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775-core.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct7802.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87360.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87427.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pcf8591.ko 2024-01-15T17:43:26.975 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-15T17:43:26.976 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/powr1220.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5627.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5636.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht15.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht21.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/shtc1.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sis5595.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47b397.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m1.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m192.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/thmc50.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp102.ko 2024-01-15T17:43:26.977 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp103.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp401.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp421.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via-cputemp.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via686a.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt1211.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt8231.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627ehf.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627hf.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83781d.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83791d.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83792d.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83793.ko 2024-01-15T17:43:26.978 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83795.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l785ts.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l786ng.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-15T17:43:26.979 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-15T17:43:26.980 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-dev.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-mux.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-smbus.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-stub.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_core.ko 2024-01-15T17:43:26.981 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-15T17:43:26.982 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport 2024-01-15T17:43:26.983 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/gameport.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/joydev.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/appletouch.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/bcm5974.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/cyapatp.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/sermouse.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-15T17:43:26.984 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/altera_ps2.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/arc_ps2.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/serio_raw.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/sparse-keymap.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-blinkm.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3530.ko 2024-01-15T17:43:26.985 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lp3944.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-ss4200.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache 2024-01-15T17:43:26.986 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache/bcache.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-bio-prison.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache-smq.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-crypt.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-delay.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-flakey.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-log-userspace.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-multipath.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-queue-length.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-raid.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-round-robin.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-service-time.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-switch.ko 2024-01-15T17:43:26.987 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-thin-pool.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-verity.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/faulty.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/linear.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/multipath.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid0.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid1.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid10.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid456.ko 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion 2024-01-15T17:43:26.988 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptbase.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptctl.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptfc.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptlan.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptsas.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptscsih.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptspi.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_ich.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_sch.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/mfd-core.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/sm501.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/viperboard.ko 2024-01-15T17:43:26.989 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/vx855.ko 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/wl1273-core.ko 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds9802als.ko 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds990x.ko 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/bh1770glc.ko 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710/cb710.ko 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/at24.ko 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-15T17:43:26.990 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/max6875.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/enclosure.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/hpilo.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29003.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29020.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-me.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-txe.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-gru 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-gru/gru.ko 2024-01-15T17:43:26.991 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xp.ko 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xpc.ko 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_7xx1.ko 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_core.ko 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tsl2550.ko 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_balloon.ko 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding/bonding.ko 2024-01-15T17:43:26.992 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dummy.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/eql.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-15T17:43:26.993 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd 2024-01-15T17:43:26.994 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom 2024-01-15T17:43:26.995 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-15T17:43:26.996 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-15T17:43:26.997 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dnet.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ethoc.ko 2024-01-15T17:43:26.998 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fealnx.ko 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb 2024-01-15T17:43:26.999 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/jme.ko 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox 2024-01-15T17:43:27.000 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-15T17:43:27.001 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-15T17:43:27.002 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rocker 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis 2024-01-15T17:43:27.003 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun 2024-01-15T17:43:27.004 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom 2024-01-15T17:43:27.005 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/geneve.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ifb.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvlan.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvtap.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-15T17:43:27.006 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mii.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/net_failover.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/netconsole.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/nlmon.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/amd.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-15T17:43:27.007 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm87xx.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/broadcom.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/cicada.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/davicom.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/dp83640.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/et1011c.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/icplus.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/lxt.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/marvell.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/micrel.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/national.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/phylink.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/qsemi.ko 2024-01-15T17:43:27.008 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/realtek.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/smsc.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/ste10Xp.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/vitesse.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_async.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_generic.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppoe.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppox.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pptp.ko 2024-01-15T17:43:27.009 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slhc.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slip.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/sungem_phy.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tap.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_random.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tun.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/veth.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/virtio_net.ko 2024-01-15T17:43:27.010 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan/vxlan.ko 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netfront.ko 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb/ntb.ko 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme-core.ko 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme.ko 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport.ko 2024-01-15T17:43:27.011 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_cs.ko 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_pc.ko 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_serial.ko 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pcie 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pcie/aer_inject.ko 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/xen-pcifront.ko 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/i82092.ko 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pd6729.ko 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-15T17:43:27.012 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acerhdf.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-15T17:43:27.013 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/hdaps.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel_ips.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-15T17:43:27.014 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi.ko 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps_parport.ko 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/pps_core.ko 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp.ko 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp_kvm.ko 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc 2024-01-15T17:43:27.015 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-em3027.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-15T17:43:27.016 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max6900.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-15T17:43:27.017 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-x1205.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-9xxx.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-sas.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-xxxx.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/BusLogic.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/a100u2w.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/advansys.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-15T17:43:27.018 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/am53c974.ko 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/atp870u.ko 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa/bfa.ko 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ch.ko 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor 2024-01-15T17:43:27.019 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dc395x.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dmx3191d.ko 2024-01-15T17:43:27.020 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esp_scsi.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic/fnic.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hpsa.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hptiop.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hv_storvsc.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/initio.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ipr.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ips.ko 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci 2024-01-15T17:43:27.021 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci/isci.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc/libfc.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas/libsas.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-15T17:43:27.022 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvumi.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pmcraid.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla1280.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/raid_class.ko 2024-01-15T17:43:27.023 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_debug.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ses.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/st.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/stex.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/virtio_scsi.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-15T17:43:27.024 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/wd719x.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb/ssb.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_file.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_iblock.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_mod.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_pscsi.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_user.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-15T17:43:27.025 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-15T17:43:27.026 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_gsm.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_hdlc.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/nozomi.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-15T17:43:27.027 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/arc_uart.ko 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio.ko 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_aec.ko 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_cif.ko 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_pci_generic.ko 2024-01-15T17:43:27.028 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_sercos3.ko 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-acm.ko 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usblp.ko 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usbtmc.ko 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/mdc800.ko 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/microtek.ko 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/adutux.ko 2024-01-15T17:43:27.029 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/appledisplay.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi26.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi62.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ezusb.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/idmouse.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/iowarrior.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ldusb.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/legousbtower.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usb3503.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usblcd.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-15T17:43:27.030 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/uss720.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/yurex.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-generic.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/aircable.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ark3116.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ch341.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cp210x.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cyberjack.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/empeg.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-15T17:43:27.031 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_ti.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipaq.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipw.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ir-usb.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mct_u232.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7720.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7840.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/navman.ko 2024-01-15T17:43:27.032 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/omninet.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/opticon.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/option.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/oti6858.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/pl2303.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcaux.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcserial.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/quatech2.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/safe_serial.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/sierra.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ssu100.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/symbolserial.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-15T17:43:27.033 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_debug.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/visor.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/whiteheat.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/uas.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-karma.ko 2024-01-15T17:43:27.034 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/usb-storage.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio.ko 2024-01-15T17:43:27.035 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_net.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_scsi.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/apple_bl.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lcd.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_balloon.ko 2024-01-15T17:43:27.036 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_mmio.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_ring.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2482.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2490.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-15T17:43:27.037 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/wire.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/hpwdt.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/i6300esb.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-15T17:43:27.038 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ib700wdt.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ibmasr.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it87_wdt.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/machzwd.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/nv_tco.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/softdog.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/via_wdt.ko 2024-01-15T17:43:27.039 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wdt_pci.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/xen_wdt.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-acpi-processor.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-evtchn.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntalloc.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntdev.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-privcmd.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-15T17:43:27.040 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/9p 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/9p/9p.ko 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/affs 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/affs/affs.ko 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/befs 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/befs/befs.ko 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/binfmt_misc.ko 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs/btrfs.ko 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles/cachefiles.ko 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph/ceph.ko 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/coda 2024-01-15T17:43:27.041 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/coda/coda.ko 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs/cramfs.ko 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm/dlm.ko 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ecryptfs 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ecryptfs/ecryptfs.ko 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs/f2fs.ko 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/fat.ko 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/msdos.ko 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/vfat.ko 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache 2024-01-15T17:43:27.042 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache/fscache.ko 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/cuse.ko 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/fuse.ko 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2/gfs2.ko 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs/hfs.ko 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus/hfsplus.ko 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs/isofs.ko 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs/jfs.ko 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd 2024-01-15T17:43:27.043 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd/lockd.ko 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/minix 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/minix/minix.ko 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs/netfs.ko 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfs.ko 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv3.ko 2024-01-15T17:43:27.044 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv4.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/grace.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/nfs_acl.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd/nfsd.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2/nilfs2.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-celtic.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-centeuro.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-croatian.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-cyrillic.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-gaelic.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-greek.ko 2024-01-15T17:43:27.045 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-iceland.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-inuit.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-roman.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-romanian.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-turkish.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1250.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1251.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1255.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp737.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp775.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp850.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp852.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp855.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp857.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp860.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp861.ko 2024-01-15T17:43:27.046 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp862.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp863.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp864.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp865.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp866.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp869.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp874.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp932.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp936.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp949.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp950.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_euc-jp.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-1.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-13.ko 2024-01-15T17:43:27.047 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-14.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-15.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-2.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-3.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-4.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-5.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-6.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-7.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-9.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-r.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-ru.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-u.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_utf8.ko 2024-01-15T17:43:27.048 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2.ko 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs/overlay.ko 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore 2024-01-15T17:43:27.049 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore/ramoops.ko 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs/reiserfs.ko 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs/romfs.ko 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client/cifs.ko 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_arc4.ko 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_md4.ko 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs/squashfs.ko 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv/sysv.ko 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/udf 2024-01-15T17:43:27.050 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/udf/udf.ko 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs/ufs.ko 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs/xfs.ko 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/rcu 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/rcu/rcutorture.ko 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/torture.ko 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/trace 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/asn1_encoder.ko 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crc-itu-t.ko 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crc8.ko 2024-01-15T17:43:27.051 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libarc4.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libdes.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/libcrc32c.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lru_cache.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4_compress.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4hc_compress.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/math 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/math/cordic.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6/raid6_pq.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_bm.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_fsm.ko 2024-01-15T17:43:27.052 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_kmp.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/zstd 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/zstd/zstd_compress.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/mm 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/mm/hwpoison-inject.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/6lowpan.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_dest.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_fragment.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_hop.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_mobility.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_routing.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_udp.ko 2024-01-15T17:43:27.053 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/garp.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/mrp.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/p8022.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/psnap.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/stp.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/8021q 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/8021q/8021q.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_fd.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_rdma.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_virtio.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv/batman-adv.ko 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge 2024-01-15T17:43:27.054 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/br_netfilter.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/bridge.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-15T17:43:27.055 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtables.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ceph 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ceph/libceph.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core/failover.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core/pktgen.ko 2024-01-15T17:43:27.056 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_diag.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv4.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv6.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dns_resolver 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dns_resolver/dns_resolver.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/dsa_core.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_brcm.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_dsa.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_none.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_trailer.ko 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan 2024-01-15T17:43:27.057 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ah4.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/esp4.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/fou.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/gre.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/inet_diag.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_gre.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_tunnel.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_vti.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipcomp.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipip.ko 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter 2024-01-15T17:43:27.058 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-15T17:43:27.059 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_bic.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_dctcp.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_diag.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_highspeed.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_htcp.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_hybla.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_illinois.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_lp.ko 2024-01-15T17:43:27.060 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_scalable.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_vegas.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_veno.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_westwood.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_yeah.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tunnel4.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_diag.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_tunnel.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ah6.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/esp6.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/fou6.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_tunnel.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-15T17:43:27.061 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_vti.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ipcomp6.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-15T17:43:27.062 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/sit.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/tunnel6.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/key 2024-01-15T17:43:27.063 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/key/af_key.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_core.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_eth.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip6.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_netlink.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ppp.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/llc 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/llc/llc.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154/mac802154.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mpls 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mpls/mpls_gso.ko 2024-01-15T17:43:27.064 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-15T17:43:27.065 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-15T17:43:27.066 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conncount.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-15T17:43:27.067 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_log_syslog.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_irc.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_sip.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_tables.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-15T17:43:27.068 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_log.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_chain_nat.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_compat.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_ct.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fib.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_hash.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_limit.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_log.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_masq.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_nat.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_numgen.ko 2024-01-15T17:43:27.069 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_queue.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_quota.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_redir.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject_inet.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_socket.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_tunnel.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_AUDIT.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CT.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_DSCP.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HL.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HMARK.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-15T17:43:27.070 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LED.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LOG.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NETMAP.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFLOG.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_RATEEST.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_SECMARK.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TEE.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TPROXY.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TRACE.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_addrtype.ko 2024-01-15T17:43:27.071 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_bpf.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cgroup.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cluster.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_comment.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connbytes.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlabel.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlimit.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connmark.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_conntrack.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cpu.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dccp.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_devgroup.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dscp.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ecn.ko 2024-01-15T17:43:27.072 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_esp.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hashlimit.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_helper.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hl.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipcomp.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_iprange.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipvs.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_l2tp.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_length.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_limit.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mac.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mark.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_multiport.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nat.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nfacct.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_osf.ko 2024-01-15T17:43:27.073 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_owner.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_physdev.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_pkttype.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_policy.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_quota.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_rateest.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_realm.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_recent.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_sctp.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_set.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_socket.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_state.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_statistic.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_string.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_tcpmss.ko 2024-01-15T17:43:27.074 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_time.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_u32.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netlink 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netlink/netlink_diag.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/nsh 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/nsh/nsh.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/openvswitch.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-geneve.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-gre.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-vxlan.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/packet 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/packet/af_packet_diag.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_rdma.ko 2024-01-15T17:43:27.075 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_tcp.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_csum.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_gact.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_ipt.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_mirred.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_nat.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_pedit.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_police.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_simple.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_skbedit.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_vlan.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_basic.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_bpf.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_flow.ko 2024-01-15T17:43:27.076 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_fw.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_route.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_u32.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_cmp.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_ipset.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_meta.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_nbyte.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_text.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_u32.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_choke.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_codel.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_drr.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_fq.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_gred.ko 2024-01-15T17:43:27.077 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hfsc.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hhf.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_htb.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_ingress.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio_lib.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_multiq.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_netem.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_pie.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_plug.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_prio.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_qfq.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_red.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfb.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfq.ko 2024-01-15T17:43:27.078 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_tbf.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_teql.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp_diag.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/sunrpc.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/diag.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/tipc.ko 2024-01-15T17:43:27.079 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/unix 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/unix/unix_diag.ko 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock.ko 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/encrypted-keys 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/trusted-keys 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/trusted-keys/trusted.ko 2024-01-15T17:43:27.080 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt/lib 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt/lib/irqbypass.ko 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.alias 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.alias.bin 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.alias.bin 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.bin 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.modinfo 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.dep 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.dep.bin 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.devname 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.order 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.softdep 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.symbols 2024-01-15T17:43:27.081 INFO:teuthology.orchestra.run.smithi124.stdout:/lib/modules/6.7.0-g2668949e0971/modules.symbols.bin 2024-01-15T17:43:27.082 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-g2668949e0971 2024-01-15T17:43:27.082 DEBUG:teuthology.orchestra.run.smithi124:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-01-15T17:43:27.104 INFO:teuthology.orchestra.run.smithi124.stdout:postinstall scriptlet (using /bin/sh): 2024-01-15T17:43:27.104 INFO:teuthology.orchestra.run.smithi124.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-g2668949e0971 -a -r /boot/System.map-6.7.0-g2668949e0971 ]; then 2024-01-15T17:43:27.104 INFO:teuthology.orchestra.run.smithi124.stdout:cp /boot/vmlinuz-6.7.0-g2668949e0971 /boot/.vmlinuz-6.7.0-g2668949e0971-rpm 2024-01-15T17:43:27.104 INFO:teuthology.orchestra.run.smithi124.stdout:cp /boot/System.map-6.7.0-g2668949e0971 /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-15T17:43:27.104 INFO:teuthology.orchestra.run.smithi124.stdout:rm -f /boot/vmlinuz-6.7.0-g2668949e0971 /boot/System.map-6.7.0-g2668949e0971 2024-01-15T17:43:27.104 INFO:teuthology.orchestra.run.smithi124.stdout:/sbin/installkernel 6.7.0-g2668949e0971 /boot/.vmlinuz-6.7.0-g2668949e0971-rpm /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-15T17:43:27.104 INFO:teuthology.orchestra.run.smithi124.stdout:rm -f /boot/.vmlinuz-6.7.0-g2668949e0971-rpm /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:fi 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:preuninstall scriptlet (using /bin/sh): 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:new-kernel-pkg --remove 6.7.0-g2668949e0971 --rminitrd --initrdfile=/boot/initramfs-6.7.0-g2668949e0971.img 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:kernel-install remove 6.7.0-g2668949e0971 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:fi 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:postuninstall scriptlet (using /bin/sh): 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:if [ -x /sbin/update-bootloader ]; then 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:/sbin/update-bootloader --remove 6.7.0-g2668949e0971 2024-01-15T17:43:27.105 INFO:teuthology.orchestra.run.smithi124.stdout:fi 2024-01-15T17:43:27.106 DEBUG:teuthology.orchestra.run.smithi124:> sudo rpm -qi grub2-tools 2024-01-15T17:43:27.181 INFO:teuthology.orchestra.run.smithi124.stdout:Name : grub2-tools 2024-01-15T17:43:27.181 INFO:teuthology.orchestra.run.smithi124.stdout:Epoch : 1 2024-01-15T17:43:27.181 INFO:teuthology.orchestra.run.smithi124.stdout:Version : 2.06 2024-01-15T17:43:27.181 INFO:teuthology.orchestra.run.smithi124.stdout:Release : 68.el9 2024-01-15T17:43:27.181 INFO:teuthology.orchestra.run.smithi124.stdout:Architecture: x86_64 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Group : Unspecified 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Size : 8264254 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:License : GPLv3+ 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Packager : builder@centos.org 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Vendor : CentOS 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Summary : Support tools for GRUB. 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:Description : 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:43:27.182 INFO:teuthology.orchestra.run.smithi124.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-15T17:43:27.183 INFO:teuthology.orchestra.run.smithi124.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-15T17:43:27.183 INFO:teuthology.orchestra.run.smithi124.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-15T17:43:27.183 INFO:teuthology.orchestra.run.smithi124.stdout:hardware devices. 2024-01-15T17:43:27.183 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:43:27.183 INFO:teuthology.orchestra.run.smithi124.stdout:This subpackage provides tools for support of all platforms. 2024-01-15T17:43:27.184 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-15T17:43:27.184 INFO:teuthology.task.kernel:Updating grub on smithi124 to boot 6.7.0-g2668949e0971 2024-01-15T17:43:27.184 DEBUG:teuthology.orchestra.run.smithi124:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-15T17:43:27.796 INFO:teuthology.orchestra.run.smithi124.stderr:Generating grub configuration file ... 2024-01-15T17:43:29.181 INFO:teuthology.orchestra.run.smithi124.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-15T17:43:29.221 INFO:teuthology.orchestra.run.smithi124.stderr:done 2024-01-15T17:43:29.223 DEBUG:teuthology.orchestra.run.smithi124:> mktemp 2024-01-15T17:43:29.238 INFO:teuthology.orchestra.run.smithi124.stdout:/tmp/tmp.ls3mjqbWkC 2024-01-15T17:43:29.239 DEBUG:teuthology.orchestra.run.smithi124:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ls3mjqbWkC 2024-01-15T17:43:29.307 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0666 /tmp/tmp.ls3mjqbWkC 2024-01-15T17:43:29.446 DEBUG:teuthology.orchestra.remote:smithi124:/tmp/tmp.ls3mjqbWkC is 6KB 2024-01-15T17:43:29.495 DEBUG:teuthology.orchestra.run.smithi124:> rm -fr /tmp/tmp.ls3mjqbWkC 2024-01-15T17:43:29.510 DEBUG:teuthology.orchestra.run.smithi124:> sudo /bin/ls /boot/loader/entries || true 2024-01-15T17:43:29.575 INFO:teuthology.orchestra.run.smithi124.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-15T17:43:29.575 INFO:teuthology.orchestra.run.smithi124.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-15T17:43:29.575 INFO:teuthology.orchestra.run.smithi124.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-15T17:43:29.575 INFO:teuthology.orchestra.run.smithi124.stdout:15949e3c88704300882482eddfd7baae-6.7.0-g2668949e0971.conf 2024-01-15T17:43:29.576 DEBUG:teuthology.orchestra.run.smithi124:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.7.0-g2668949e0971 2024-01-15T17:43:29.730 DEBUG:teuthology.orchestra.run.smithi124:> sudo shutdown -r now 2024-01-15T17:43:46.706 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi026.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-01-15T17:43:46.707 DEBUG:teuthology.orchestra.run.smithi026:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-01-15T17:43:46.790 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/System.map-6.7.0-g2668949e0971 2024-01-15T17:43:46.790 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/config-6.7.0-g2668949e0971 2024-01-15T17:43:46.790 INFO:teuthology.orchestra.run.smithi026.stdout:/boot/vmlinuz-6.7.0-g2668949e0971 2024-01-15T17:43:46.790 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971 2024-01-15T17:43:46.790 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel 2024-01-15T17:43:46.790 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch 2024-01-15T17:43:46.790 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86 2024-01-15T17:43:46.790 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crc32c-intel.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-15T17:43:46.791 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-15T17:43:46.792 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/arch/x86/kvm/kvm.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/ansi_cprng.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/anubis.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/arc4.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_memcpy.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_pq.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_tx.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/async_xor.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/async_tx/raid6test.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/authenc.ko 2024-01-15T17:43:46.793 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/authencesn.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blake2b_generic.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_common.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/blowfish_generic.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/camellia_generic.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast5_generic.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast6_generic.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cast_common.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/ccm.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cmac.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crc32_generic.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_engine.ko 2024-01-15T17:43:46.794 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/crypto_user.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/cts.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/des_generic.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/dh_generic.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/echainiv.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/essiv.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/fcrypt.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/khazad.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/lz4hc.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/md4.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/michael_mic.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcbc.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/pcrypt.ko 2024-01-15T17:43:46.795 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/rmd160.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/seed.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/serpent_generic.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/tcrypt.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/tea.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_common.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/twofish_generic.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/vmac.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/wp512.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xcbc.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xor.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/crypto/xxhash_generic.ko 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers 2024-01-15T17:43:46.796 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/acpi_pad.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/custom_method.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/ec_sys.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/platform_profile.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbs.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/sbshc.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/acpi/video.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/acard-ahci.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ahci_platform.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/ata_generic.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/libahci_platform.ko 2024-01-15T17:43:46.797 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_acpi.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ali.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_amd.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_artop.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atiixp.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_atp867x.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd640.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cmd64x.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_cypress.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_efar.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt366.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt37x.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-15T17:43:46.798 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it8213.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_it821x.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_jmicron.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_marvell.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_mpiix.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_netcell.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ninja32.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87410.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_ns87415.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_oldpiix.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_opti.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_optidma.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pcmcia.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-15T17:43:46.799 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_piccolo.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_rdc.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sch.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_serverworks.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sil680.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sis.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_sl82c105.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_triflex.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pata_via.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/pdc_adma.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_inic162x.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_mv.ko 2024-01-15T17:43:46.800 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_nv.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_promise.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_qstor.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sil24.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sis.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_svw.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_sx4.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_uli.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_via.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ata/sata_vsc.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/auxdisplay/ks0108.ko 2024-01-15T17:43:46.801 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap/regmap-i2c.ko 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/base/regmap/regmap-mmio.ko 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/bcma/bcma.ko 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/aoe/aoe.ko 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/brd.ko 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/drbd/drbd.ko 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/floppy.ko 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/loop.ko 2024-01-15T17:43:46.802 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/nbd.ko 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/null_blk/null_blk.ko 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/pktcdvd.ko 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/rbd.ko 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/virtio_blk.ko 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/xen-blkfront.ko 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/block/zram/zram.ko 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char 2024-01-15T17:43:46.803 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hangcheck-timer.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/via-rng.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-15T17:43:46.804 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/lp.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/mwave/mwave.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/ppdev.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tlclk.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_tis.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-01-15T17:43:46.805 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/uv_mmtimer.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/char/virtio_console.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/powernow-k8.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/ccp/ccp.ko 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel 2024-01-15T17:43:46.806 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-aes.ko 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/padlock-sha.ko 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dca/dca.ko 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/devfreq 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma 2024-01-15T17:43:46.807 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/amd64_edac.ko 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/e752x_edac.ko 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/edac_mce_amd.ko 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3000_edac.ko 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i3200_edac.ko 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5100_edac.ko 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i5400_edac.ko 2024-01-15T17:43:46.808 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7300_edac.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i7core_edac.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/i82975x_edac.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/ie31200_edac.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/sb_edac.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/edac/x38_edac.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/edd.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-ich.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-15T17:43:46.809 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-appleir.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-aureal.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-axff.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-dr.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elecom.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-elo.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-emsff.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gaff.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gt683r.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-gyration.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-holtekff.ko 2024-01-15T17:43:46.810 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-hyperv.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-icade.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-keytouch.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-kye.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lcpower.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-led.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-lenovo.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-multitouch.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-ortek.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-penmount.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-petalynx.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-picolcd.ko 2024-01-15T17:43:46.811 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-pl.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-plantronics.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-primax.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-rmi.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-common.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-15T17:43:46.812 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-roccat.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-saitek.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-samsung.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sjoy.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sony.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-speedlink.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-steelseries.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-sunplus.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tivo.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-tmff.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-topseed.ko 2024-01-15T17:43:46.813 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-twinhan.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-uclogic.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-waltop.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-wiimote.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-xinmo.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zpff.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/hid-zydacron.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/uhid.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hid/wacom.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_balloon.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_utils.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hv/hv_vmbus.ko 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon 2024-01-15T17:43:46.814 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/abituguru3.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7414.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ad7418.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adc128d818.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1025.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1026.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1029.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm1031.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adm9240.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ads7828.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7410.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7411.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7462.ko 2024-01-15T17:43:46.815 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7470.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7475.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/adt7x10.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/amc6821.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/applesmc.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asb100.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asc7621.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/atxp1.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/coretemp.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/dme1737.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds1621.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ds620.ko 2024-01-15T17:43:46.816 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc1403.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/emc6w201.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71805f.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f71882fg.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/f75375s.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fam15h_power.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/fschmd.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g760a.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/g762.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl518sm.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/gl520sm.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5500_temp.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/i5k_amb.ko 2024-01-15T17:43:46.817 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmaem.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ibmpex.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina209.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ina2xx.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/it87.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k10temp.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/k8temp.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lineage-pem.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm63.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm73.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm75.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm77.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm78.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm80.ko 2024-01-15T17:43:46.818 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm83.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm85.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm87.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm90.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm92.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm93.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95234.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95241.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/lm95245.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc2945.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4151.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4215.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4222.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4245.ko 2024-01-15T17:43:46.819 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4260.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/ltc4261.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max16065.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1619.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max1668.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max197.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6639.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6650.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/max6697.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/mcp3021.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6683.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775-core.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct6775.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/nct7802.ko 2024-01-15T17:43:46.820 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87360.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pc87427.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pcf8591.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-15T17:43:46.821 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/powr1220.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5627.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch5636.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht15.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sht21.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/shtc1.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/sis5595.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47b397.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m1.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/smsc47m192.ko 2024-01-15T17:43:46.822 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/thmc50.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp102.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp103.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp401.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/tmp421.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via-cputemp.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/via686a.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt1211.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/vt8231.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627ehf.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83627hf.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83781d.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83791d.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83792d.ko 2024-01-15T17:43:46.823 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83793.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83795.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l785ts.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/hwmon/w83l786ng.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-15T17:43:46.824 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-15T17:43:46.825 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-dev.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-mux.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-smbus.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/i2c/i2c-stub.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_core.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-15T17:43:46.826 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib 2024-01-15T17:43:46.827 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-15T17:43:46.828 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/gameport/gameport.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/joydev.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/appletouch.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/bcm5974.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/cyapatp.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/sermouse.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-15T17:43:46.829 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/altera_ps2.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/arc_ps2.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/serio/serio_raw.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/input/sparse-keymap.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-blinkm.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lm3530.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-lp3944.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/leds-ss4200.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-15T17:43:46.830 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/bcache/bcache.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-bio-prison.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache-smq.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-cache.ko 2024-01-15T17:43:46.831 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-crypt.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-delay.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-flakey.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-log-userspace.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-multipath.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-queue-length.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-raid.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-round-robin.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-service-time.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-switch.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-thin-pool.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/dm-verity.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/faulty.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/linear.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/multipath.ko 2024-01-15T17:43:46.832 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid0.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid1.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid10.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/md/raid456.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptbase.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptctl.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptfc.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptlan.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptsas.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptscsih.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/message/fusion/mptspi.ko 2024-01-15T17:43:46.833 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_ich.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/lpc_sch.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/mfd-core.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/sm501.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/viperboard.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/vx855.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/mfd/wl1273-core.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds9802als.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/apds990x.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/bh1770glc.ko 2024-01-15T17:43:46.834 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/cb710/cb710.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/at24.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/eeprom/max6875.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/enclosure.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/hpilo.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29003.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/isl29020.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei 2024-01-15T17:43:46.835 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-me.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei-txe.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/mei/mei.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-gru 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-gru/gru.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xp.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xpc.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_7xx1.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tifm_core.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/tsl2550.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_balloon.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-15T17:43:46.836 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/bonding/bonding.ko 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/dummy.ko 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/eql.ko 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com 2024-01-15T17:43:46.837 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon 2024-01-15T17:43:46.838 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx 2024-01-15T17:43:46.839 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna 2024-01-15T17:43:46.840 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec 2024-01-15T17:43:46.841 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/dnet.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ethoc.ko 2024-01-15T17:43:46.842 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/fealnx.ko 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-15T17:43:46.843 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/jme.ko 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-15T17:43:46.844 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia 2024-01-15T17:43:46.845 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-15T17:43:46.846 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rocker 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-15T17:43:46.847 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via 2024-01-15T17:43:46.848 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/geneve.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ifb.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-15T17:43:46.849 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvlan.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/macvtap.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/mii.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/net_failover.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/netconsole.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/nlmon.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-15T17:43:46.850 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/amd.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/bcm87xx.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/broadcom.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/cicada.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/davicom.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/dp83640.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/et1011c.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/icplus.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/lxt.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/marvell.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/micrel.ko 2024-01-15T17:43:46.851 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/national.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/phylink.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/qsemi.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/realtek.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/smsc.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/ste10Xp.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/phy/vitesse.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_async.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_generic.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-15T17:43:46.852 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppoe.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pppox.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/ppp/pptp.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slhc.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/slip/slip.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/sungem_phy.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tap.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_random.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-15T17:43:46.853 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/tun.ko 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/veth.ko 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/virtio_net.ko 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/vxlan/vxlan.ko 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/net/xen-netfront.ko 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ntb/ntb.ko 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme-core.ko 2024-01-15T17:43:46.854 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/nvme/host/nvme.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_cs.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_pc.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/parport/parport_serial.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pcie 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/pcie/aer_inject.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pci/xen-pcifront.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/i82092.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/pd6729.ko 2024-01-15T17:43:46.855 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/acerhdf.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-15T17:43:46.856 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/hdaps.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/intel_ips.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-15T17:43:46.857 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/platform/x86/wmi.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/clients/pps_parport.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/pps/pps_core.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ptp/ptp_kvm.ko 2024-01-15T17:43:46.858 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-em3027.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-15T17:43:46.859 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-max6900.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-15T17:43:46.860 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/rtc/rtc-x1205.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-9xxx.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-sas.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/3w-xxxx.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/BusLogic.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/a100u2w.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/advansys.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/am53c974.ko 2024-01-15T17:43:46.861 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/atp870u.ko 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bfa/bfa.ko 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ch.ko 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi 2024-01-15T17:43:46.862 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dc395x.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/dmx3191d.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-15T17:43:46.863 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/esp_scsi.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/fnic/fnic.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hpsa.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hptiop.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/hv_storvsc.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/initio.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ipr.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ips.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/isci/isci.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-15T17:43:46.864 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libfc/libfc.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/libsas/libsas.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-15T17:43:46.865 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/mvumi.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/pmcraid.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla1280.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/raid_class.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_debug.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-15T17:43:46.866 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/ses.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/st.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/stex.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/virtio_scsi.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/scsi/wd719x.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ssb/ssb.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-15T17:43:46.867 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_file.ko 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_iblock.ko 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_mod.ko 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_pscsi.ko 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/target_core_user.ko 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-15T17:43:46.868 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-15T17:43:46.869 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_gsm.ko 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/n_hdlc.ko 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/nozomi.ko 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/arc_uart.ko 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-15T17:43:46.870 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_aec.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_cif.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_pci_generic.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/uio/uio_sercos3.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-acm.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usblp.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/class/usbtmc.ko 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host 2024-01-15T17:43:46.871 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/mdc800.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/image/microtek.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/adutux.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/appledisplay.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi26.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/emi62.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ezusb.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/idmouse.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/iowarrior.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/ldusb.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/legousbtower.ko 2024-01-15T17:43:46.872 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usb3503.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usblcd.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/uss720.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/misc/yurex.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/phy/phy-generic.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/aircable.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ark3116.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ch341.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cp210x.ko 2024-01-15T17:43:46.873 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cyberjack.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/empeg.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/io_ti.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipaq.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ipw.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ir-usb.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan.ko 2024-01-15T17:43:46.874 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mct_u232.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7720.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/mos7840.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/navman.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/omninet.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/opticon.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/option.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/oti6858.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/pl2303.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcaux.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/qcserial.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/quatech2.ko 2024-01-15T17:43:46.875 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/safe_serial.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/sierra.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ssu100.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/symbolserial.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_debug.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/visor.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/whiteheat.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/uas.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-15T17:43:46.876 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-karma.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/storage/usb-storage.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-15T17:43:46.877 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_net.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/vhost/vhost_scsi.ko 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight 2024-01-15T17:43:46.878 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/apple_bl.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/lcd.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_balloon.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_mmio.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/virtio/virtio_ring.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2482.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/masters/ds2490.ko 2024-01-15T17:43:46.879 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/w1/wire.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-15T17:43:46.880 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/hpwdt.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/i6300esb.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ib700wdt.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ibmasr.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/it87_wdt.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/machzwd.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/nv_tco.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-15T17:43:46.881 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/softdog.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/via_wdt.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/wdt_pci.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/watchdog/xen_wdt.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-acpi-processor.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-evtchn.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntalloc.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-gntdev.ko 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback 2024-01-15T17:43:46.882 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xen-privcmd.ko 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/9p 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/9p/9p.ko 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/affs 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/affs/affs.ko 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/befs 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/befs/befs.ko 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/binfmt_misc.ko 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/btrfs/btrfs.ko 2024-01-15T17:43:46.883 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cachefiles/cachefiles.ko 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ceph/ceph.ko 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/coda 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/coda/coda.ko 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/cramfs/cramfs.ko 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/dlm/dlm.ko 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ecryptfs 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ecryptfs/ecryptfs.ko 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs 2024-01-15T17:43:46.884 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/f2fs/f2fs.ko 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/fat.ko 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/msdos.ko 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fat/vfat.ko 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fscache/fscache.ko 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/cuse.ko 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/fuse/fuse.ko 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/gfs2/gfs2.ko 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfs/hfs.ko 2024-01-15T17:43:46.885 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/hfsplus/hfsplus.ko 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/isofs/isofs.ko 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/jfs/jfs.ko 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/lockd/lockd.ko 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/minix 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/minix/minix.ko 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/netfs/netfs.ko 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout 2024-01-15T17:43:46.886 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfs.ko 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv3.ko 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs/nfsv4.ko 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/grace.ko 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfs_common/nfs_acl.ko 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nfsd/nfsd.ko 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2 2024-01-15T17:43:46.887 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nilfs2/nilfs2.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-celtic.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-centeuro.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-croatian.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-cyrillic.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-gaelic.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-greek.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-iceland.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-inuit.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-roman.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-romanian.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/mac-turkish.ko 2024-01-15T17:43:46.888 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1250.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1251.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp1255.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp737.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp775.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp850.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp852.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp855.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp857.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp860.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp861.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp862.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp863.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp864.ko 2024-01-15T17:43:46.889 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp865.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp866.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp869.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp874.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp932.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp936.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp949.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_cp950.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_euc-jp.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-1.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-13.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-14.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-15.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-2.ko 2024-01-15T17:43:46.890 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-3.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-4.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-5.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-6.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-7.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_iso8859-9.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-r.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-ru.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_koi8-u.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/nls/nls_utf8.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm 2024-01-15T17:43:46.891 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2.ko 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/overlayfs/overlay.ko 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/pstore/ramoops.ko 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/reiserfs/reiserfs.ko 2024-01-15T17:43:46.892 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/romfs/romfs.ko 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/client/cifs.ko 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_arc4.ko 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/smb/common/cifs_md4.ko 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/squashfs/squashfs.ko 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/sysv/sysv.ko 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/udf 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/udf/udf.ko 2024-01-15T17:43:46.893 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/ufs/ufs.ko 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/fs/xfs/xfs.ko 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/rcu 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/rcu/rcutorture.ko 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/torture.ko 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/trace 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/asn1_encoder.ko 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crc-itu-t.ko 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crc8.ko 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto 2024-01-15T17:43:46.894 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libarc4.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/crypto/libdes.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/libcrc32c.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lru_cache.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4_compress.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/lz4/lz4hc_compress.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/math 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/math/cordic.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/raid6/raid6_pq.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_bm.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_fsm.ko 2024-01-15T17:43:46.895 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/ts_kmp.ko 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/zstd 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/lib/zstd/zstd_compress.ko 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/mm 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/mm/hwpoison-inject.ko 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/6lowpan.ko 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_dest.ko 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_fragment.ko 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_hop.ko 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_mobility.ko 2024-01-15T17:43:46.896 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_routing.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/6lowpan/nhc_udp.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/garp.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/mrp.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/p8022.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/psnap.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/802/stp.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/8021q 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/8021q/8021q.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_fd.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_rdma.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/9p/9pnet_virtio.ko 2024-01-15T17:43:46.897 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/batman-adv/batman-adv.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/br_netfilter.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/bridge.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-15T17:43:46.898 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/ebtables.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-01-15T17:43:46.899 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ceph 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ceph/libceph.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core/failover.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/core/pktgen.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_diag.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv4.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dccp/dccp_ipv6.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dns_resolver 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dns_resolver/dns_resolver.ko 2024-01-15T17:43:46.900 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/dsa_core.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_brcm.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_dsa.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_none.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/dsa/tag_trailer.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ah4.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/esp4.ko 2024-01-15T17:43:46.901 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/fou.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/gre.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/inet_diag.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_gre.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_tunnel.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ip_vti.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipcomp.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/ipip.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-15T17:43:46.902 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-15T17:43:46.903 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_bic.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_dctcp.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_diag.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_highspeed.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_htcp.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_hybla.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_illinois.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_lp.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_scalable.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_vegas.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_veno.ko 2024-01-15T17:43:46.904 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_westwood.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tcp_yeah.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/tunnel4.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_diag.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/udp_tunnel.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ah6.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/esp6.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/fou6.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_tunnel.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ip6_vti.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/ipcomp6.ko 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter 2024-01-15T17:43:46.905 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-15T17:43:46.906 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/sit.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/tunnel6.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/key 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/key/af_key.ko 2024-01-15T17:43:46.907 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_core.ko 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_eth.ko 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip.ko 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ip6.ko 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_netlink.ko 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/l2tp/l2tp_ppp.ko 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/llc 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/llc/llc.ko 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mac802154/mac802154.ko 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mpls 2024-01-15T17:43:46.908 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/mpls/mpls_gso.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-15T17:43:46.909 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-15T17:43:46.910 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conncount.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-15T17:43:46.911 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_log_syslog.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_irc.ko 2024-01-15T17:43:46.912 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_sip.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nf_tables.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_log.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_chain_nat.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_compat.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_ct.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fib.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-15T17:43:46.913 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_hash.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_limit.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_log.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_masq.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_nat.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_numgen.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_queue.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_quota.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_redir.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_reject_inet.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_socket.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/nft_tunnel.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_AUDIT.ko 2024-01-15T17:43:46.914 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_CT.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_DSCP.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HL.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_HMARK.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LED.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_LOG.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NETMAP.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFLOG.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_RATEEST.ko 2024-01-15T17:43:46.915 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_SECMARK.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TEE.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TPROXY.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_TRACE.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_addrtype.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_bpf.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cgroup.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cluster.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_comment.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connbytes.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlabel.ko 2024-01-15T17:43:46.916 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connlimit.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_connmark.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_conntrack.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_cpu.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dccp.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_devgroup.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_dscp.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ecn.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_esp.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hashlimit.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_helper.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_hl.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipcomp.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_iprange.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_ipvs.ko 2024-01-15T17:43:46.917 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_l2tp.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_length.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_limit.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mac.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_mark.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_multiport.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nat.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_nfacct.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_osf.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_owner.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_physdev.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_pkttype.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_policy.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_quota.ko 2024-01-15T17:43:46.918 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_rateest.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_realm.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_recent.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_sctp.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_set.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_socket.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_state.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_statistic.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_string.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_tcpmss.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_time.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netfilter/xt_u32.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netlink 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/netlink/netlink_diag.ko 2024-01-15T17:43:46.919 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/nsh 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/nsh/nsh.ko 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/openvswitch.ko 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-geneve.ko 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-gre.ko 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/openvswitch/vport-vxlan.ko 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/packet 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/packet/af_packet_diag.ko 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds.ko 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_rdma.ko 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/rds/rds_tcp.ko 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched 2024-01-15T17:43:46.920 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_csum.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_gact.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_ipt.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_mirred.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_nat.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_pedit.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_police.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_simple.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_skbedit.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/act_vlan.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_basic.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_bpf.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_flow.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_fw.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_route.ko 2024-01-15T17:43:46.921 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/cls_u32.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_cmp.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_ipset.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_meta.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_nbyte.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_text.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/em_u32.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_choke.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_codel.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_drr.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_fq.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_gred.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hfsc.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_hhf.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_htb.ko 2024-01-15T17:43:46.922 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_ingress.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_mqprio_lib.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_multiq.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_netem.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_pie.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_plug.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_prio.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_qfq.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_red.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfb.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_sfq.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_tbf.ko 2024-01-15T17:43:46.923 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sched/sch_teql.ko 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp.ko 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sctp/sctp_diag.ko 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/sunrpc.ko 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/diag.ko 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/tipc/tipc.ko 2024-01-15T17:43:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/unix 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/unix/unix_diag.ko 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock.ko 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/encrypted-keys 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-01-15T17:43:46.925 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/trusted-keys 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/security/keys/trusted-keys/trusted.ko 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt/lib 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/kernel/virt/lib/irqbypass.ko 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.alias 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.alias.bin 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.alias.bin 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.bin 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.builtin.modinfo 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.dep 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.dep.bin 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.devname 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.order 2024-01-15T17:43:46.926 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.softdep 2024-01-15T17:43:46.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.symbols 2024-01-15T17:43:46.927 INFO:teuthology.orchestra.run.smithi026.stdout:/lib/modules/6.7.0-g2668949e0971/modules.symbols.bin 2024-01-15T17:43:46.927 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-g2668949e0971 2024-01-15T17:43:46.927 DEBUG:teuthology.orchestra.run.smithi026:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:postinstall scriptlet (using /bin/sh): 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-g2668949e0971 -a -r /boot/System.map-6.7.0-g2668949e0971 ]; then 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:cp /boot/vmlinuz-6.7.0-g2668949e0971 /boot/.vmlinuz-6.7.0-g2668949e0971-rpm 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:cp /boot/System.map-6.7.0-g2668949e0971 /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:rm -f /boot/vmlinuz-6.7.0-g2668949e0971 /boot/System.map-6.7.0-g2668949e0971 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/installkernel 6.7.0-g2668949e0971 /boot/.vmlinuz-6.7.0-g2668949e0971-rpm /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:rm -f /boot/.vmlinuz-6.7.0-g2668949e0971-rpm /boot/.System.map-6.7.0-g2668949e0971-rpm 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:preuninstall scriptlet (using /bin/sh): 2024-01-15T17:43:46.951 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-01-15T17:43:46.952 INFO:teuthology.orchestra.run.smithi026.stdout:new-kernel-pkg --remove 6.7.0-g2668949e0971 --rminitrd --initrdfile=/boot/initramfs-6.7.0-g2668949e0971.img 2024-01-15T17:43:46.952 INFO:teuthology.orchestra.run.smithi026.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-01-15T17:43:46.952 INFO:teuthology.orchestra.run.smithi026.stdout:kernel-install remove 6.7.0-g2668949e0971 2024-01-15T17:43:46.952 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-01-15T17:43:46.952 INFO:teuthology.orchestra.run.smithi026.stdout:postuninstall scriptlet (using /bin/sh): 2024-01-15T17:43:46.952 INFO:teuthology.orchestra.run.smithi026.stdout:if [ -x /sbin/update-bootloader ]; then 2024-01-15T17:43:46.952 INFO:teuthology.orchestra.run.smithi026.stdout:/sbin/update-bootloader --remove 6.7.0-g2668949e0971 2024-01-15T17:43:46.952 INFO:teuthology.orchestra.run.smithi026.stdout:fi 2024-01-15T17:43:46.953 DEBUG:teuthology.orchestra.run.smithi026:> sudo rpm -qi grub2-tools 2024-01-15T17:43:47.032 INFO:teuthology.orchestra.run.smithi026.stdout:Name : grub2-tools 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Epoch : 1 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Version : 2.06 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Release : 68.el9 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Architecture: x86_64 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Group : Unspecified 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Size : 8264254 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:License : GPLv3+ 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Packager : builder@centos.org 2024-01-15T17:43:47.033 INFO:teuthology.orchestra.run.smithi026.stdout:Vendor : CentOS 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout:Summary : Support tools for GRUB. 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout:Description : 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout:hardware devices. 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:43:47.034 INFO:teuthology.orchestra.run.smithi026.stdout:This subpackage provides tools for support of all platforms. 2024-01-15T17:43:47.035 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-15T17:43:47.035 INFO:teuthology.task.kernel:Updating grub on smithi026 to boot 6.7.0-g2668949e0971 2024-01-15T17:43:47.035 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-15T17:43:47.683 INFO:teuthology.orchestra.run.smithi026.stderr:Generating grub configuration file ... 2024-01-15T17:43:49.125 INFO:teuthology.orchestra.run.smithi026.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-15T17:43:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:done 2024-01-15T17:43:49.178 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-01-15T17:43:49.235 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.aC1jaYucPi 2024-01-15T17:43:49.235 DEBUG:teuthology.orchestra.run.smithi026:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aC1jaYucPi 2024-01-15T17:43:49.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.aC1jaYucPi 2024-01-15T17:43:49.467 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.aC1jaYucPi is 6KB 2024-01-15T17:43:49.519 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.aC1jaYucPi 2024-01-15T17:43:49.533 DEBUG:teuthology.orchestra.run.smithi026:> sudo /bin/ls /boot/loader/entries || true 2024-01-15T17:43:49.611 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-15T17:43:49.611 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-15T17:43:49.611 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-15T17:43:49.611 INFO:teuthology.orchestra.run.smithi026.stdout:15949e3c88704300882482eddfd7baae-6.7.0-g2668949e0971.conf 2024-01-15T17:43:49.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.7.0-g2668949e0971 2024-01-15T17:43:49.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo shutdown -r now 2024-01-15T17:43:59.740 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-01-15T17:43:59.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2024-01-15T17:43:59.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:44:18.236 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.124 2024-01-15T17:44:19.767 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-01-15T17:44:19.768 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-01-15T17:44:19.768 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:44:27.240 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2024-01-15T17:44:27.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:44:30.300 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.124 2024-01-15T17:44:38.204 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-01-15T17:44:42.306 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2024-01-15T17:44:42.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:44:47.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-01-15T17:44:47.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:45:05.596 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.26 2024-01-15T17:45:17.609 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-01-15T17:45:17.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:45:18.269 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-01-15T17:45:18.999 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-01-15T17:45:18.999 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.7.0-g2668949e0971"... 2024-01-15T17:45:18.999 DEBUG:teuthology.orchestra.run.smithi026:> uname -r 2024-01-15T17:45:19.039 INFO:teuthology.orchestra.run.smithi026.stdout:6.7.0-g2668949e0971 2024-01-15T17:45:19.039 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-g2668949e0971 vs 6.7.0-g2668949e0971 2024-01-15T17:45:19.040 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-15T17:45:19.040 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-15T17:45:20.040 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-01-15T17:45:20.040 DEBUG:teuthology.orchestra.run.smithi026:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-15T17:45:20.200 INFO:teuthology.orchestra.run.smithi026.stdout:ttyS1 2024-01-15T17:45:20.243 DEBUG:teuthology.parallel:result is None 2024-01-15T17:45:42.309 DEBUG:teuthology.orchestra.remote:timed out 2024-01-15T17:45:57.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2024-01-15T17:45:57.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:45:57.447 DEBUG:teuthology.orchestra.run.smithi124:> true 2024-01-15T17:45:57.603 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com' 2024-01-15T17:45:57.603 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.7.0-g2668949e0971"... 2024-01-15T17:45:57.603 DEBUG:teuthology.orchestra.run.smithi124:> uname -r 2024-01-15T17:45:57.650 INFO:teuthology.orchestra.run.smithi124.stdout:6.7.0-g2668949e0971 2024-01-15T17:45:57.650 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-g2668949e0971 vs 6.7.0-g2668949e0971 2024-01-15T17:45:57.650 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-15T17:45:57.651 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-15T17:45:58.652 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-01-15T17:45:58.652 DEBUG:teuthology.orchestra.run.smithi124:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-01-15T17:45:58.756 INFO:teuthology.orchestra.run.smithi124.stdout:ttyS1 2024-01-15T17:45:58.789 DEBUG:teuthology.parallel:result is None 2024-01-15T17:45:58.789 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-15T17:45:58.796 INFO:teuthology.task.internal:Creating test directory... 2024-01-15T17:45:58.796 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-15T17:45:58.806 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-15T17:45:58.865 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-15T17:45:58.872 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-15T17:45:58.880 INFO:teuthology.task.internal:Creating archive directory... 2024-01-15T17:45:58.880 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-15T17:45:58.885 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-15T17:45:58.952 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-15T17:45:58.960 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-15T17:45:58.960 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-15T17:45:58.968 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-15T17:45:59.070 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-15T17:45:59.088 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-15T17:45:59.112 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-15T17:45:59.134 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-15T17:45:59.139 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-15T17:45:59.146 INFO:teuthology.task.internal:Configuring sudo... 2024-01-15T17:45:59.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-15T17:45:59.153 DEBUG:teuthology.orchestra.run.smithi124:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-15T17:45:59.251 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-15T17:45:59.260 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-15T17:45:59.260 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-15T17:45:59.268 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-15T17:45:59.323 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-15T17:45:59.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-15T17:45:59.523 DEBUG:teuthology.orchestra.run.smithi026:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-15T17:45:59.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-15T17:45:59.715 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:45:59.715 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-15T17:45:59.815 DEBUG:teuthology.orchestra.run.smithi124:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-15T17:45:59.901 DEBUG:teuthology.orchestra.run.smithi124:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-15T17:46:00.009 DEBUG:teuthology.orchestra.run.smithi124:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-15T17:46:00.091 DEBUG:teuthology.orchestra.run.smithi124:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-15T17:46:00.175 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:46:00.175 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-15T17:46:00.255 DEBUG:teuthology.orchestra.run.smithi026:> sudo service rsyslog restart 2024-01-15T17:46:00.262 DEBUG:teuthology.orchestra.run.smithi124:> sudo service rsyslog restart 2024-01-15T17:46:00.393 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-15T17:46:00.397 INFO:teuthology.orchestra.run.smithi124.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-15T17:46:00.685 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-15T17:46:00.692 INFO:teuthology.task.internal:Starting timer... 2024-01-15T17:46:00.692 INFO:teuthology.run_tasks:Running task pcp... 2024-01-15T17:46:00.705 INFO:teuthology.run_tasks:Running task selinux... 2024-01-15T17:46:00.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo service auditd rotate 2024-01-15T17:46:00.850 INFO:teuthology.orchestra.run.smithi026.stdout:Rotating logs: 2024-01-15T17:46:00.854 DEBUG:teuthology.orchestra.run.smithi124:> sudo service auditd rotate 2024-01-15T17:46:00.983 INFO:teuthology.orchestra.run.smithi124.stdout:Rotating logs: 2024-01-15T17:46:00.987 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-15T17:46:00.987 DEBUG:teuthology.orchestra.run.smithi026:> /usr/sbin/getenforce 2024-01-15T17:46:01.050 INFO:teuthology.orchestra.run.smithi026.stdout:Permissive 2024-01-15T17:46:01.051 DEBUG:teuthology.orchestra.run.smithi124:> /usr/sbin/getenforce 2024-01-15T17:46:01.116 INFO:teuthology.orchestra.run.smithi124.stdout:Permissive 2024-01-15T17:46:01.116 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi026.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi124.front.sepia.ceph.com': 'permissive'} 2024-01-15T17:46:01.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-15T17:46:01.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:46:01.220 DEBUG:teuthology.orchestra.run.smithi124:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-15T17:46:01.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:46:01.319 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-15T17:46:01.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo /usr/sbin/setenforce permissive 2024-01-15T17:46:01.431 DEBUG:teuthology.orchestra.run.smithi124:> sudo /usr/sbin/setenforce permissive 2024-01-15T17:46:01.530 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-15T17:46:01.540 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-01-15T17:46:01.546 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-01-15T17:46:01.698 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-15T17:46:01.713 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-15T17:46:01.714 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi026.front.sepia.ceph.com,smithi124.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-15T17:51:47.645 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi026.front.sepia.ceph.com'), Remote(name='ubuntu@smithi124.front.sepia.ceph.com')] 2024-01-15T17:51:47.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-01-15T17:51:47.648 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi026.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:51:47.797 DEBUG:teuthology.orchestra.run.smithi026:> true 2024-01-15T17:51:47.947 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi026.front.sepia.ceph.com' 2024-01-15T17:51:47.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi124.front.sepia.ceph.com' 2024-01-15T17:51:47.948 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi124.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-15T17:51:48.087 DEBUG:teuthology.orchestra.run.smithi124:> true 2024-01-15T17:51:48.230 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi124.front.sepia.ceph.com' 2024-01-15T17:51:48.230 INFO:teuthology.run_tasks:Running task clock... 2024-01-15T17:51:48.244 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-15T17:51:48.244 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-15T17:51:48.245 DEBUG:teuthology.orchestra.run.smithi026:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-15T17:51:48.252 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-15T17:51:48.253 DEBUG:teuthology.orchestra.run.smithi124:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-15T17:51:48.351 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-15T17:51:48.354 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-15T17:51:48.400 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-15T17:51:48.409 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-15T17:51:48.492 INFO:teuthology.orchestra.run.smithi124.stderr:sudo: ntpd: command not found 2024-01-15T17:51:48.509 INFO:teuthology.orchestra.run.smithi026.stderr:sudo: ntpd: command not found 2024-01-15T17:51:48.535 INFO:teuthology.orchestra.run.smithi124.stdout:506 Cannot talk to daemon 2024-01-15T17:51:48.554 INFO:teuthology.orchestra.run.smithi026.stdout:506 Cannot talk to daemon 2024-01-15T17:51:48.587 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-15T17:51:48.608 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-15T17:51:48.637 INFO:teuthology.orchestra.run.smithi124.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-15T17:51:48.664 INFO:teuthology.orchestra.run.smithi026.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-15T17:51:48.752 INFO:teuthology.orchestra.run.smithi124.stderr:bash: line 1: ntpq: command not found 2024-01-15T17:51:48.759 INFO:teuthology.orchestra.run.smithi124.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-15T17:51:48.759 INFO:teuthology.orchestra.run.smithi124.stdout:=============================================================================== 2024-01-15T17:51:48.759 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T17:51:48.759 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T17:51:48.759 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T17:51:48.759 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T17:51:48.790 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-01-15T17:51:48.799 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-15T17:51:48.799 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-01-15T17:51:48.800 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T17:51:48.800 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T17:51:48.800 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T17:51:48.800 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T17:51:48.801 INFO:teuthology.run_tasks:Running task install... 2024-01-15T17:51:48.811 DEBUG:teuthology.task.install:project ceph 2024-01-15T17:51:48.811 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}} 2024-01-15T17:51:48.811 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'} 2024-01-15T17:51:48.811 INFO:teuthology.task.install:Using flavor: default 2024-01-15T17:51:48.817 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-15T17:51:48.817 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-01-15T17:51:48.818 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b', 'tag': None, 'wait_for_package': False} 2024-01-15T17:51:48.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T17:51:48.820 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b', 'tag': None, 'wait_for_package': False} 2024-01-15T17:51:48.820 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T17:51:48.960 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/centos/9/flavors/default/ 2024-01-15T17:51:48.960 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7629.g95c90f41 2024-01-15T17:51:48.963 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/centos/9/flavors/default/ 2024-01-15T17:51:48.963 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7629.g95c90f41 2024-01-15T17:51:49.216 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-15T17:51:49.216 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:51:49.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-15T17:51:49.222 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/rishabh-clone-stats/95c90f413c4bb65e3e56909855f2308fd9bda05b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-15T17:51:49.222 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:51:49.222 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-15T17:51:49.323 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-15T17:51:49.323 DEBUG:teuthology.orchestra.run.smithi026:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/95c90f413c4bb65e3e56909855f2308fd9bda05b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-15T17:51:49.324 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-15T17:51:49.324 DEBUG:teuthology.orchestra.run.smithi124:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/95c90f413c4bb65e3e56909855f2308fd9bda05b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-15T17:51:49.462 DEBUG:teuthology.orchestra.run.smithi124:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-15T17:51:49.467 DEBUG:teuthology.orchestra.run.smithi026:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-15T17:51:49.592 DEBUG:teuthology.orchestra.run.smithi124:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-15T17:51:49.605 DEBUG:teuthology.orchestra.run.smithi026:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-15T17:51:49.682 INFO:teuthology.orchestra.run.smithi124.stdout:check_obsoletes = 1 2024-01-15T17:51:49.686 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum clean all 2024-01-15T17:51:49.701 INFO:teuthology.orchestra.run.smithi026.stdout:check_obsoletes = 1 2024-01-15T17:51:49.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum clean all 2024-01-15T17:51:50.045 INFO:teuthology.orchestra.run.smithi124.stdout:73 files removed 2024-01-15T17:51:50.064 INFO:teuthology.orchestra.run.smithi026.stdout:73 files removed 2024-01-15T17:51:50.084 DEBUG:teuthology.orchestra.run.smithi124:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-15T17:51:50.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-15T17:51:51.185 INFO:teuthology.orchestra.run.smithi026.stdout:ceph packages for x86_64 141 kB/s | 78 kB 00:00 2024-01-15T17:51:51.258 INFO:teuthology.orchestra.run.smithi124.stdout:ceph packages for x86_64 117 kB/s | 78 kB 00:00 2024-01-15T17:51:51.826 INFO:teuthology.orchestra.run.smithi124.stdout:ceph noarch packages 20 kB/s | 9.5 kB 00:00 2024-01-15T17:51:51.878 INFO:teuthology.orchestra.run.smithi026.stdout:ceph noarch packages 16 kB/s | 9.5 kB 00:00 2024-01-15T17:51:52.526 INFO:teuthology.orchestra.run.smithi124.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-01-15T17:51:52.577 INFO:teuthology.orchestra.run.smithi026.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-01-15T17:51:53.697 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - AppStream 6.2 MB/s | 6.4 MB 00:01 2024-01-15T17:51:53.737 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS-9 - AppStream 5.6 MB/s | 6.4 MB 00:01 2024-01-15T17:51:55.856 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS-9 - BaseOS 2.2 MB/s | 2.0 MB 00:00 2024-01-15T17:51:56.299 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS-9 - BaseOS 1.5 MB/s | 2.0 MB 00:01 2024-01-15T17:51:56.943 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 7.9 MB 00:00 2024-01-15T17:51:58.129 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 7.9 MB 00:01 2024-01-15T17:51:59.144 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - AppStream 19 MB/s | 18 MB 00:00 2024-01-15T17:52:05.029 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 18 MB 00:05 2024-01-15T17:52:05.903 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 5.9 MB 00:02 2024-01-15T17:52:08.745 INFO:teuthology.orchestra.run.smithi026.stdout:CentOS Stream 9 - Extras packages 9.9 kB/s | 15 kB 00:01 2024-01-15T17:52:11.052 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 5.9 MB 00:02 2024-01-15T17:52:12.055 INFO:teuthology.orchestra.run.smithi026.stdout:Extra Packages for Enterprise Linux 6.4 MB/s | 20 MB 00:03 2024-01-15T17:52:13.273 INFO:teuthology.orchestra.run.smithi124.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 15 kB 00:00 2024-01-15T17:52:14.933 INFO:teuthology.orchestra.run.smithi124.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2024-01-15T17:52:18.171 INFO:teuthology.orchestra.run.smithi026.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-01-15T17:52:20.691 INFO:teuthology.orchestra.run.smithi124.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-01-15T17:52:20.976 INFO:teuthology.orchestra.run.smithi026.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-15T17:52:20.978 INFO:teuthology.orchestra.run.smithi026.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-15T17:52:20.994 INFO:teuthology.orchestra.run.smithi026.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-15T17:52:20.995 INFO:teuthology.orchestra.run.smithi026.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-15T17:52:20.997 INFO:teuthology.orchestra.run.smithi026.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-15T17:52:20.998 INFO:teuthology.orchestra.run.smithi026.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-15T17:52:21.001 INFO:teuthology.orchestra.run.smithi026.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-15T17:52:21.004 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-15T17:52:21.006 INFO:teuthology.orchestra.run.smithi026.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-15T17:52:21.091 INFO:teuthology.orchestra.run.smithi026.stdout:Dependencies resolved. 2024-01-15T17:52:21.105 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================ 2024-01-15T17:52:21.105 INFO:teuthology.orchestra.run.smithi026.stdout: Package Arch Version Repository Size 2024-01-15T17:52:21.105 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================ 2024-01-15T17:52:21.105 INFO:teuthology.orchestra.run.smithi026.stdout:Installing: 2024-01-15T17:52:21.105 INFO:teuthology.orchestra.run.smithi026.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-15T17:52:21.105 INFO:teuthology.orchestra.run.smithi026.stdout: ceph x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 6.6 k 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 5.2 M 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 860 k 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 141 k 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 1.5 M 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 133 k 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 1.8 M 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 7.4 M 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 50 k 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 8.6 M 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 41 M 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 262 k 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 774 k 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 217 k 2024-01-15T17:52:21.106 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 18 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 31 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 718 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 127 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 162 k 2024-01-15T17:52:21.107 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 320 k 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 298 k 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 99 k 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 85 k 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 3.0 M 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 170 k 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrading: 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager x86_64 1:1.45.9-1.el9 CentOS-BaseOS 2.2 M 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm x86_64 1:1.45.9-1.el9 CentOS-BaseOS 1.8 M 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team x86_64 1:1.45.9-1.el9 CentOS-BaseOS 42 k 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui x86_64 1:1.45.9-1.el9 CentOS-BaseOS 246 k 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-15T17:52:21.108 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: librados2 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 3.3 M 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 3.0 M 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout:Installing dependencies: 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-15T17:52:21.109 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 19 M 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 28 k 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 2.1 M 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 246 k 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 4.4 M 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 18 M 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 15 k 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 25 k 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-15T17:52:21.110 INFO:teuthology.orchestra.run.smithi026.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 163 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 476 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 4.8 M 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-15T17:52:21.111 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-15T17:52:21.112 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 45 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 129 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth noarch 1:2.26.0-1.el9 epel 202 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-15T17:52:21.113 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-15T17:52:21.114 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-15T17:52:21.115 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-15T17:52:21.116 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-15T17:52:21.117 INFO:teuthology.orchestra.run.smithi026.stdout:Installing weak dependencies: 2024-01-15T17:52:21.117 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-01-15T17:52:21.117 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:52:21.117 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction Summary 2024-01-15T17:52:21.117 INFO:teuthology.orchestra.run.smithi026.stdout:============================================================================================ 2024-01-15T17:52:21.117 INFO:teuthology.orchestra.run.smithi026.stdout:Install 135 Packages 2024-01-15T17:52:21.117 INFO:teuthology.orchestra.run.smithi026.stdout:Upgrade 16 Packages 2024-01-15T17:52:21.117 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:52:21.117 INFO:teuthology.orchestra.run.smithi026.stdout:Total download size: 204 M 2024-01-15T17:52:21.118 INFO:teuthology.orchestra.run.smithi026.stdout:Downloading Packages: 2024-01-15T17:52:22.506 INFO:teuthology.orchestra.run.smithi026.stdout:(1/151): ceph-18.0.0-7629.g95c90f41.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-01-15T17:52:22.898 INFO:teuthology.orchestra.run.smithi026.stdout:(2/151): ceph-base-18.0.0-7629.g95c90f41.el9.x8 6.5 MB/s | 5.2 MB 00:00 2024-01-15T17:52:23.156 INFO:teuthology.orchestra.run.smithi026.stdout:(3/151): ceph-immutable-object-cache-18.0.0-762 548 kB/s | 141 kB 00:00 2024-01-15T17:52:23.403 INFO:teuthology.orchestra.run.smithi124.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-15T17:52:23.405 INFO:teuthology.orchestra.run.smithi124.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-15T17:52:23.423 INFO:teuthology.orchestra.run.smithi124.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-15T17:52:23.424 INFO:teuthology.orchestra.run.smithi124.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-15T17:52:23.426 INFO:teuthology.orchestra.run.smithi124.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-15T17:52:23.427 INFO:teuthology.orchestra.run.smithi124.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-15T17:52:23.429 INFO:teuthology.orchestra.run.smithi124.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-15T17:52:23.433 INFO:teuthology.orchestra.run.smithi124.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-15T17:52:23.434 INFO:teuthology.orchestra.run.smithi124.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-15T17:52:23.456 INFO:teuthology.orchestra.run.smithi026.stdout:(4/151): ceph-mds-18.0.0-7629.g95c90f41.el9.x86 7.0 MB/s | 2.1 MB 00:00 2024-01-15T17:52:23.515 INFO:teuthology.orchestra.run.smithi124.stdout:Dependencies resolved. 2024-01-15T17:52:23.535 INFO:teuthology.orchestra.run.smithi124.stdout:============================================================================================ 2024-01-15T17:52:23.535 INFO:teuthology.orchestra.run.smithi124.stdout: Package Arch Version Repository Size 2024-01-15T17:52:23.536 INFO:teuthology.orchestra.run.smithi124.stdout:============================================================================================ 2024-01-15T17:52:23.536 INFO:teuthology.orchestra.run.smithi124.stdout:Installing: 2024-01-15T17:52:23.536 INFO:teuthology.orchestra.run.smithi124.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-15T17:52:23.536 INFO:teuthology.orchestra.run.smithi124.stdout: ceph x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 6.6 k 2024-01-15T17:52:23.536 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-base x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 5.2 M 2024-01-15T17:52:23.536 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-fuse x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 860 k 2024-01-15T17:52:23.536 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 141 k 2024-01-15T17:52:23.536 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 1.5 M 2024-01-15T17:52:23.537 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 133 k 2024-01-15T17:52:23.537 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 1.8 M 2024-01-15T17:52:23.537 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 7.4 M 2024-01-15T17:52:23.537 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-rook noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 50 k 2024-01-15T17:52:23.537 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-radosgw x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 8.6 M 2024-01-15T17:52:23.537 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-test x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 41 M 2024-01-15T17:52:23.537 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-volume noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 262 k 2024-01-15T17:52:23.537 INFO:teuthology.orchestra.run.smithi124.stdout: cephadm noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 774 k 2024-01-15T17:52:23.538 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-mirror x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 217 k 2024-01-15T17:52:23.538 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-top noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 18 k 2024-01-15T17:52:23.538 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-15T17:52:23.538 INFO:teuthology.orchestra.run.smithi124.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-15T17:52:23.538 INFO:teuthology.orchestra.run.smithi124.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-15T17:52:23.538 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs-devel x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 31 k 2024-01-15T17:52:23.538 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs2 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 718 k 2024-01-15T17:52:23.538 INFO:teuthology.orchestra.run.smithi124.stdout: librados-devel x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 127 k 2024-01-15T17:52:23.538 INFO:teuthology.orchestra.run.smithi124.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-15T17:52:23.539 INFO:teuthology.orchestra.run.smithi124.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-15T17:52:23.539 INFO:teuthology.orchestra.run.smithi124.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-15T17:52:23.539 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-15T17:52:23.539 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-15T17:52:23.539 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-15T17:52:23.539 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cephfs x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 162 k 2024-01-15T17:52:23.539 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rados x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 320 k 2024-01-15T17:52:23.539 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rbd x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 298 k 2024-01-15T17:52:23.540 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rgw x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 99 k 2024-01-15T17:52:23.540 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-fuse x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 85 k 2024-01-15T17:52:23.540 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-mirror x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 3.0 M 2024-01-15T17:52:23.540 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-nbd x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 170 k 2024-01-15T17:52:23.540 INFO:teuthology.orchestra.run.smithi124.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-15T17:52:23.540 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrading: 2024-01-15T17:52:23.540 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager x86_64 1:1.45.9-1.el9 CentOS-BaseOS 2.2 M 2024-01-15T17:52:23.540 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-libnm x86_64 1:1.45.9-1.el9 CentOS-BaseOS 1.8 M 2024-01-15T17:52:23.541 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-team x86_64 1:1.45.9-1.el9 CentOS-BaseOS 42 k 2024-01-15T17:52:23.541 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-tui x86_64 1:1.45.9-1.el9 CentOS-BaseOS 246 k 2024-01-15T17:52:23.541 INFO:teuthology.orchestra.run.smithi124.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-15T17:52:23.541 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-15T17:52:23.541 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-15T17:52:23.541 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-15T17:52:23.541 INFO:teuthology.orchestra.run.smithi124.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-15T17:52:23.541 INFO:teuthology.orchestra.run.smithi124.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-15T17:52:23.541 INFO:teuthology.orchestra.run.smithi124.stdout: librados2 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 3.3 M 2024-01-15T17:52:23.542 INFO:teuthology.orchestra.run.smithi124.stdout: librbd1 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 3.0 M 2024-01-15T17:52:23.542 INFO:teuthology.orchestra.run.smithi124.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-15T17:52:23.542 INFO:teuthology.orchestra.run.smithi124.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-15T17:52:23.542 INFO:teuthology.orchestra.run.smithi124.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-15T17:52:23.542 INFO:teuthology.orchestra.run.smithi124.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-15T17:52:23.542 INFO:teuthology.orchestra.run.smithi124.stdout:Installing dependencies: 2024-01-15T17:52:23.542 INFO:teuthology.orchestra.run.smithi124.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-15T17:52:23.542 INFO:teuthology.orchestra.run.smithi124.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-15T17:52:23.542 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-common x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 19 M 2024-01-15T17:52:23.543 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 28 k 2024-01-15T17:52:23.543 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mds x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 2.1 M 2024-01-15T17:52:23.543 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 246 k 2024-01-15T17:52:23.543 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mon x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 4.4 M 2024-01-15T17:52:23.543 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-osd x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 18 M 2024-01-15T17:52:23.543 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7629.g95c90f41.el9 ceph-noarch 15 k 2024-01-15T17:52:23.543 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-selinux x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 25 k 2024-01-15T17:52:23.543 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-15T17:52:23.544 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-15T17:52:23.544 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-15T17:52:23.544 INFO:teuthology.orchestra.run.smithi124.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-15T17:52:23.544 INFO:teuthology.orchestra.run.smithi124.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-15T17:52:23.544 INFO:teuthology.orchestra.run.smithi124.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-15T17:52:23.544 INFO:teuthology.orchestra.run.smithi124.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-15T17:52:23.544 INFO:teuthology.orchestra.run.smithi124.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-15T17:52:23.544 INFO:teuthology.orchestra.run.smithi124.stdout: libcephsqlite x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 163 k 2024-01-15T17:52:23.544 INFO:teuthology.orchestra.run.smithi124.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: libradosstriper1 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 476 k 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: librgw2 x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 4.8 M 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-15T17:52:23.545 INFO:teuthology.orchestra.run.smithi124.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-15T17:52:23.546 INFO:teuthology.orchestra.run.smithi124.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-15T17:52:23.546 INFO:teuthology.orchestra.run.smithi124.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-15T17:52:23.546 INFO:teuthology.orchestra.run.smithi124.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-15T17:52:23.546 INFO:teuthology.orchestra.run.smithi124.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-15T17:52:23.546 INFO:teuthology.orchestra.run.smithi124.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-15T17:52:23.546 INFO:teuthology.orchestra.run.smithi124.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-15T17:52:23.546 INFO:teuthology.orchestra.run.smithi124.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-15T17:52:23.546 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-15T17:52:23.546 INFO:teuthology.orchestra.run.smithi124.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-15T17:52:23.547 INFO:teuthology.orchestra.run.smithi124.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-15T17:52:23.547 INFO:teuthology.orchestra.run.smithi124.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-15T17:52:23.547 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-15T17:52:23.547 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-argparse x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 45 k 2024-01-15T17:52:23.547 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-common x86_64 2:18.0.0-7629.g95c90f41.el9 ceph 129 k 2024-01-15T17:52:23.547 INFO:teuthology.orchestra.run.smithi124.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-15T17:52:23.547 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-15T17:52:23.547 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-15T17:52:23.547 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-15T17:52:23.548 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-15T17:52:23.548 INFO:teuthology.orchestra.run.smithi124.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-15T17:52:23.548 INFO:teuthology.orchestra.run.smithi124.stdout: python3-google-auth noarch 1:2.26.0-1.el9 epel 202 k 2024-01-15T17:52:23.548 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-15T17:52:23.548 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-15T17:52:23.548 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-15T17:52:23.548 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-15T17:52:23.548 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-15T17:52:23.549 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-15T17:52:23.549 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-15T17:52:23.549 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-15T17:52:23.549 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-01-15T17:52:23.549 INFO:teuthology.orchestra.run.smithi124.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-15T17:52:23.549 INFO:teuthology.orchestra.run.smithi124.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-15T17:52:23.549 INFO:teuthology.orchestra.run.smithi124.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-15T17:52:23.549 INFO:teuthology.orchestra.run.smithi124.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-15T17:52:23.549 INFO:teuthology.orchestra.run.smithi124.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-15T17:52:23.550 INFO:teuthology.orchestra.run.smithi124.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-15T17:52:23.550 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-15T17:52:23.550 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-15T17:52:23.550 INFO:teuthology.orchestra.run.smithi124.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-15T17:52:23.550 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-15T17:52:23.550 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-15T17:52:23.550 INFO:teuthology.orchestra.run.smithi124.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-15T17:52:23.550 INFO:teuthology.orchestra.run.smithi124.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-15T17:52:23.550 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-15T17:52:23.551 INFO:teuthology.orchestra.run.smithi124.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-15T17:52:23.552 INFO:teuthology.orchestra.run.smithi124.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-15T17:52:23.552 INFO:teuthology.orchestra.run.smithi124.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-15T17:52:23.552 INFO:teuthology.orchestra.run.smithi124.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-15T17:52:23.552 INFO:teuthology.orchestra.run.smithi124.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-15T17:52:23.552 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-15T17:52:23.552 INFO:teuthology.orchestra.run.smithi124.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-15T17:52:23.552 INFO:teuthology.orchestra.run.smithi124.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-15T17:52:23.552 INFO:teuthology.orchestra.run.smithi124.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout:Installing weak dependencies: 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:52:23.553 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction Summary 2024-01-15T17:52:23.554 INFO:teuthology.orchestra.run.smithi124.stdout:============================================================================================ 2024-01-15T17:52:23.554 INFO:teuthology.orchestra.run.smithi124.stdout:Install 135 Packages 2024-01-15T17:52:23.554 INFO:teuthology.orchestra.run.smithi124.stdout:Upgrade 16 Packages 2024-01-15T17:52:23.554 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:52:23.554 INFO:teuthology.orchestra.run.smithi124.stdout:Total download size: 204 M 2024-01-15T17:52:23.555 INFO:teuthology.orchestra.run.smithi124.stdout:Downloading Packages: 2024-01-15T17:52:23.681 INFO:teuthology.orchestra.run.smithi026.stdout:(5/151): ceph-mgr-18.0.0-7629.g95c90f41.el9.x86 6.5 MB/s | 1.5 MB 00:00 2024-01-15T17:52:24.015 INFO:teuthology.orchestra.run.smithi026.stdout:(6/151): ceph-common-18.0.0-7629.g95c90f41.el9. 9.9 MB/s | 19 MB 00:01 2024-01-15T17:52:24.057 INFO:teuthology.orchestra.run.smithi026.stdout:(7/151): ceph-fuse-18.0.0-7629.g95c90f41.el9.x8 555 kB/s | 860 kB 00:01 2024-01-15T17:52:24.241 INFO:teuthology.orchestra.run.smithi026.stdout:(8/151): ceph-mon-18.0.0-7629.g95c90f41.el9.x86 7.9 MB/s | 4.4 MB 00:00 2024-01-15T17:52:24.610 INFO:teuthology.orchestra.run.smithi124.stdout:(1/151): ceph-18.0.0-7629.g95c90f41.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-01-15T17:52:24.649 INFO:teuthology.orchestra.run.smithi026.stdout:(9/151): ceph-selinux-18.0.0-7629.g95c90f41.el9 62 kB/s | 25 kB 00:00 2024-01-15T17:52:24.883 INFO:teuthology.orchestra.run.smithi026.stdout:(10/151): ceph-osd-18.0.0-7629.g95c90f41.el9.x8 20 MB/s | 18 MB 00:00 2024-01-15T17:52:24.958 INFO:teuthology.orchestra.run.smithi026.stdout:(11/151): cephfs-mirror-18.0.0-7629.g95c90f41.e 2.9 MB/s | 217 kB 00:00 2024-01-15T17:52:25.042 INFO:teuthology.orchestra.run.smithi026.stdout:(12/151): libcephfs-devel-18.0.0-7629.g95c90f41 377 kB/s | 31 kB 00:00 2024-01-15T17:52:25.242 INFO:teuthology.orchestra.run.smithi026.stdout:(13/151): libcephfs2-18.0.0-7629.g95c90f41.el9. 3.5 MB/s | 718 kB 00:00 2024-01-15T17:52:25.359 INFO:teuthology.orchestra.run.smithi026.stdout:(14/151): libcephsqlite-18.0.0-7629.g95c90f41.e 1.4 MB/s | 163 kB 00:00 2024-01-15T17:52:25.434 INFO:teuthology.orchestra.run.smithi026.stdout:(15/151): librados-devel-18.0.0-7629.g95c90f41. 1.7 MB/s | 127 kB 00:00 2024-01-15T17:52:25.543 INFO:teuthology.orchestra.run.smithi026.stdout:(16/151): libradosstriper1-18.0.0-7629.g95c90f4 4.3 MB/s | 476 kB 00:00 2024-01-15T17:52:25.602 INFO:teuthology.orchestra.run.smithi124.stdout:(2/151): ceph-fuse-18.0.0-7629.g95c90f41.el9.x8 868 kB/s | 860 kB 00:00 2024-01-15T17:52:26.111 INFO:teuthology.orchestra.run.smithi026.stdout:(17/151): librgw2-18.0.0-7629.g95c90f41.el9.x86 8.4 MB/s | 4.8 MB 00:00 2024-01-15T17:52:26.168 INFO:teuthology.orchestra.run.smithi026.stdout:(18/151): python3-ceph-argparse-18.0.0-7629.g95 788 kB/s | 45 kB 00:00 2024-01-15T17:52:26.227 INFO:teuthology.orchestra.run.smithi026.stdout:(19/151): python3-ceph-common-18.0.0-7629.g95c9 2.2 MB/s | 129 kB 00:00 2024-01-15T17:52:26.294 INFO:teuthology.orchestra.run.smithi026.stdout:(20/151): python3-cephfs-18.0.0-7629.g95c90f41. 2.4 MB/s | 162 kB 00:00 2024-01-15T17:52:26.369 INFO:teuthology.orchestra.run.smithi026.stdout:(21/151): python3-rados-18.0.0-7629.g95c90f41.e 4.2 MB/s | 320 kB 00:00 2024-01-15T17:52:26.385 INFO:teuthology.orchestra.run.smithi124.stdout:(3/151): ceph-immutable-object-cache-18.0.0-762 180 kB/s | 141 kB 00:00 2024-01-15T17:52:26.453 INFO:teuthology.orchestra.run.smithi026.stdout:(22/151): python3-rbd-18.0.0-7629.g95c90f41.el9 3.5 MB/s | 298 kB 00:00 2024-01-15T17:52:26.493 INFO:teuthology.orchestra.run.smithi124.stdout:(4/151): ceph-base-18.0.0-7629.g95c90f41.el9.x8 2.2 MB/s | 5.2 MB 00:02 2024-01-15T17:52:26.570 INFO:teuthology.orchestra.run.smithi026.stdout:(23/151): python3-rgw-18.0.0-7629.g95c90f41.el9 858 kB/s | 99 kB 00:00 2024-01-15T17:52:26.653 INFO:teuthology.orchestra.run.smithi026.stdout:(24/151): rbd-fuse-18.0.0-7629.g95c90f41.el9.x8 1.0 MB/s | 85 kB 00:00 2024-01-15T17:52:27.054 INFO:teuthology.orchestra.run.smithi026.stdout:(25/151): rbd-mirror-18.0.0-7629.g95c90f41.el9. 7.5 MB/s | 3.0 MB 00:00 2024-01-15T17:52:27.127 INFO:teuthology.orchestra.run.smithi124.stdout:(5/151): ceph-mgr-18.0.0-7629.g95c90f41.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-01-15T17:52:27.129 INFO:teuthology.orchestra.run.smithi026.stdout:(26/151): rbd-nbd-18.0.0-7629.g95c90f41.el9.x86 2.2 MB/s | 170 kB 00:00 2024-01-15T17:52:27.254 INFO:teuthology.orchestra.run.smithi026.stdout:(27/151): ceph-grafana-dashboards-18.0.0-7629.g 226 kB/s | 28 kB 00:00 2024-01-15T17:52:27.346 INFO:teuthology.orchestra.run.smithi026.stdout:(28/151): ceph-mgr-cephadm-18.0.0-7629.g95c90f4 1.4 MB/s | 133 kB 00:00 2024-01-15T17:52:27.622 INFO:teuthology.orchestra.run.smithi026.stdout:(29/151): ceph-mgr-dashboard-18.0.0-7629.g95c90 6.4 MB/s | 1.8 MB 00:00 2024-01-15T17:52:27.936 INFO:teuthology.orchestra.run.smithi124.stdout:(6/151): ceph-mds-18.0.0-7629.g95c90f41.el9.x86 1.3 MB/s | 2.1 MB 00:01 2024-01-15T17:52:28.080 INFO:teuthology.orchestra.run.smithi026.stdout:(30/151): ceph-test-18.0.0-7629.g95c90f41.el9.x 12 MB/s | 41 MB 00:03 2024-01-15T17:52:28.239 INFO:teuthology.orchestra.run.smithi026.stdout:(31/151): ceph-mgr-modules-core-18.0.0-7629.g95 1.5 MB/s | 246 kB 00:00 2024-01-15T17:52:28.422 INFO:teuthology.orchestra.run.smithi026.stdout:(32/151): ceph-mgr-diskprediction-local-18.0.0- 9.3 MB/s | 7.4 MB 00:00 2024-01-15T17:52:28.439 INFO:teuthology.orchestra.run.smithi026.stdout:(33/151): ceph-mgr-rook-18.0.0-7629.g95c90f41.e 252 kB/s | 50 kB 00:00 2024-01-15T17:52:28.464 INFO:teuthology.orchestra.run.smithi026.stdout:(34/151): ceph-prometheus-alerts-18.0.0-7629.g9 371 kB/s | 15 kB 00:00 2024-01-15T17:52:28.515 INFO:teuthology.orchestra.run.smithi026.stdout:(35/151): ceph-volume-18.0.0-7629.g95c90f41.el9 3.4 MB/s | 262 kB 00:00 2024-01-15T17:52:28.536 INFO:teuthology.orchestra.run.smithi124.stdout:(7/151): ceph-mon-18.0.0-7629.g95c90f41.el9.x86 3.1 MB/s | 4.4 MB 00:01 2024-01-15T17:52:28.573 INFO:teuthology.orchestra.run.smithi026.stdout:(36/151): cephadm-18.0.0-7629.g95c90f41.el9.noa 7.0 MB/s | 774 kB 00:00 2024-01-15T17:52:28.599 INFO:teuthology.orchestra.run.smithi026.stdout:(37/151): cephfs-top-18.0.0-7629.g95c90f41.el9. 220 kB/s | 18 kB 00:00 2024-01-15T17:52:28.891 INFO:teuthology.orchestra.run.smithi026.stdout:(38/151): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 662 kB 00:00 2024-01-15T17:52:28.941 INFO:teuthology.orchestra.run.smithi026.stdout:(39/151): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 920 kB 00:00 2024-01-15T17:52:28.966 INFO:teuthology.orchestra.run.smithi026.stdout:(40/151): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 106 kB 00:00 2024-01-15T17:52:28.991 INFO:teuthology.orchestra.run.smithi026.stdout:(41/151): elfutils-libelf-devel-0.190-2.el9.x86 421 kB/s | 20 kB 00:00 2024-01-15T17:52:29.025 INFO:teuthology.orchestra.run.smithi026.stdout:(42/151): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2024-01-15T17:52:29.050 INFO:teuthology.orchestra.run.smithi026.stdout:(43/151): flexiblas-3.0.4-8.el9.x86_64.rpm 575 kB/s | 32 kB 00:00 2024-01-15T17:52:29.192 INFO:teuthology.orchestra.run.smithi026.stdout:(44/151): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-01-15T17:52:29.217 INFO:teuthology.orchestra.run.smithi026.stdout:(45/151): flexiblas-openblas-openmp-3.0.4-8.el9 107 kB/s | 17 kB 00:00 2024-01-15T17:52:29.259 INFO:teuthology.orchestra.run.smithi026.stdout:(46/151): libaio-devel-0.3.111-13.el9.x86_64.rp 249 kB/s | 10 kB 00:00 2024-01-15T17:52:29.284 INFO:teuthology.orchestra.run.smithi026.stdout:(47/151): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 159 kB 00:00 2024-01-15T17:52:29.318 INFO:teuthology.orchestra.run.smithi026.stdout:(48/151): librabbitmq-0.11.0-7.el9.x86_64.rpm 779 kB/s | 44 kB 00:00 2024-01-15T17:52:29.368 INFO:teuthology.orchestra.run.smithi026.stdout:(49/151): librdkafka-1.6.1-102.el9.x86_64.rpm 7.9 MB/s | 661 kB 00:00 2024-01-15T17:52:29.410 INFO:teuthology.orchestra.run.smithi026.stdout:(50/151): libtool-2.4.6-45.el9.x86_64.rpm 6.6 MB/s | 578 kB 00:00 2024-01-15T17:52:29.436 INFO:teuthology.orchestra.run.smithi026.stdout:(51/151): libuuid-devel-2.37.4-15.el9.x86_64.rp 401 kB/s | 24 kB 00:00 2024-01-15T17:52:29.461 INFO:teuthology.orchestra.run.smithi026.stdout:(52/151): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 46 kB 00:00 2024-01-15T17:52:29.494 INFO:teuthology.orchestra.run.smithi026.stdout:(53/151): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-01-15T17:52:29.551 INFO:teuthology.orchestra.run.smithi026.stdout:(54/151): lua-5.4.4-4.el9.x86_64.rpm 2.1 MB/s | 187 kB 00:00 2024-01-15T17:52:29.579 INFO:teuthology.orchestra.run.smithi026.stdout:(55/151): openblas-0.3.21-2.el9.x86_64.rpm 421 kB/s | 34 kB 00:00 2024-01-15T17:52:29.811 INFO:teuthology.orchestra.run.smithi124.stdout:(8/151): ceph-common-18.0.0-7629.g95c90f41.el9. 3.4 MB/s | 19 MB 00:05 2024-01-15T17:52:29.821 INFO:teuthology.orchestra.run.smithi026.stdout:(56/151): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-01-15T17:52:29.886 INFO:teuthology.orchestra.run.smithi124.stdout:(9/151): ceph-selinux-18.0.0-7629.g95c90f41.el9 340 kB/s | 25 kB 00:00 2024-01-15T17:52:29.904 INFO:teuthology.orchestra.run.smithi026.stdout:(57/151): openssl-devel-3.0.7-25.el9.x86_64.rpm 9.2 MB/s | 3.0 MB 00:00 2024-01-15T17:52:29.963 INFO:teuthology.orchestra.run.smithi026.stdout:(58/151): postgresql-13.11-1.el9.x86_64.rpm 12 MB/s | 1.5 MB 00:00 2024-01-15T17:52:30.004 INFO:teuthology.orchestra.run.smithi026.stdout:(59/151): postgresql-contrib-13.11-1.el9.x86_64 9.2 MB/s | 811 kB 00:00 2024-01-15T17:52:30.030 INFO:teuthology.orchestra.run.smithi026.stdout:(60/151): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-01-15T17:52:30.180 INFO:teuthology.orchestra.run.smithi026.stdout:(61/151): ceph-radosgw-18.0.0-7629.g95c90f41.el 1.4 MB/s | 8.6 MB 00:06 2024-01-15T17:52:30.370 INFO:teuthology.orchestra.run.smithi124.stdout:(10/151): ceph-radosgw-18.0.0-7629.g95c90f41.el 4.7 MB/s | 8.6 MB 00:01 2024-01-15T17:52:30.445 INFO:teuthology.orchestra.run.smithi124.stdout:(11/151): cephfs-mirror-18.0.0-7629.g95c90f41.e 2.9 MB/s | 217 kB 00:00 2024-01-15T17:52:30.497 INFO:teuthology.orchestra.run.smithi026.stdout:(62/151): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2024-01-15T17:52:30.529 INFO:teuthology.orchestra.run.smithi124.stdout:(12/151): libcephfs-devel-18.0.0-7629.g95c90f41 375 kB/s | 31 kB 00:00 2024-01-15T17:52:30.756 INFO:teuthology.orchestra.run.smithi026.stdout:(63/151): python3-babel-2.9.1-2.el9.noarch.rpm 8.0 MB/s | 5.8 MB 00:00 2024-01-15T17:52:30.781 INFO:teuthology.orchestra.run.smithi026.stdout:(64/151): python3-jinja2-2.11.3-4.el9.noarch.rp 390 kB/s | 228 kB 00:00 2024-01-15T17:52:30.806 INFO:teuthology.orchestra.run.smithi026.stdout:(65/151): python3-jsonpatch-1.21-16.el9.noarch. 83 kB/s | 24 kB 00:00 2024-01-15T17:52:30.831 INFO:teuthology.orchestra.run.smithi026.stdout:(66/151): python3-jsonpointer-2.0-4.el9.noarch. 249 kB/s | 18 kB 00:00 2024-01-15T17:52:30.854 INFO:teuthology.orchestra.run.smithi124.stdout:(13/151): libcephfs2-18.0.0-7629.g95c90f41.el9. 2.2 MB/s | 718 kB 00:00 2024-01-15T17:52:30.857 INFO:teuthology.orchestra.run.smithi026.stdout:(67/151): python3-markupsafe-1.1.1-12.el9.x86_6 646 kB/s | 32 kB 00:00 2024-01-15T17:52:30.891 INFO:teuthology.orchestra.run.smithi026.stdout:(68/151): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-01-15T17:52:30.954 INFO:teuthology.orchestra.run.smithi124.stdout:(14/151): libcephsqlite-18.0.0-7629.g95c90f41.e 1.6 MB/s | 163 kB 00:00 2024-01-15T17:52:31.030 INFO:teuthology.orchestra.run.smithi124.stdout:(15/151): librados-devel-18.0.0-7629.g95c90f41. 1.7 MB/s | 127 kB 00:00 2024-01-15T17:52:31.107 INFO:teuthology.orchestra.run.smithi026.stdout:(69/151): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-01-15T17:52:31.141 INFO:teuthology.orchestra.run.smithi026.stdout:(70/151): python3-numpy-f2py-1.20.1-5.el9.x86_6 771 kB/s | 217 kB 00:00 2024-01-15T17:52:31.166 INFO:teuthology.orchestra.run.smithi026.stdout:(71/151): python3-oauthlib-3.1.1-5.el9.noarch.r 658 kB/s | 174 kB 00:00 2024-01-15T17:52:31.183 INFO:teuthology.orchestra.run.smithi026.stdout:(72/151): python3-prettytable-0.7.2-27.el9.noar 581 kB/s | 41 kB 00:00 2024-01-15T17:52:31.188 INFO:teuthology.orchestra.run.smithi124.stdout:(16/151): libradosstriper1-18.0.0-7629.g95c90f4 2.9 MB/s | 476 kB 00:00 2024-01-15T17:52:31.217 INFO:teuthology.orchestra.run.smithi026.stdout:(73/151): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2024-01-15T17:52:31.250 INFO:teuthology.orchestra.run.smithi026.stdout:(74/151): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-01-15T17:52:31.267 INFO:teuthology.orchestra.run.smithi026.stdout:(75/151): python3-pytz-2021.1-5.el9.noarch.rpm 573 kB/s | 47 kB 00:00 2024-01-15T17:52:31.292 INFO:teuthology.orchestra.run.smithi026.stdout:(76/151): python3-requests-oauthlib-1.3.0-12.el 589 kB/s | 43 kB 00:00 2024-01-15T17:52:31.326 INFO:teuthology.orchestra.run.smithi026.stdout:(77/151): python3-toml-0.10.2-6.el9.noarch.rpm 680 kB/s | 39 kB 00:00 2024-01-15T17:52:31.601 INFO:teuthology.orchestra.run.smithi026.stdout:(78/151): uuid-1.6.2-55.el9.x86_64.rpm 211 kB/s | 56 kB 00:00 2024-01-15T17:52:31.643 INFO:teuthology.orchestra.run.smithi026.stdout:(79/151): socat-1.7.4.1-5.el9.x86_64.rpm 859 kB/s | 300 kB 00:00 2024-01-15T17:52:32.052 INFO:teuthology.orchestra.run.smithi026.stdout:(80/151): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-01-15T17:52:32.069 INFO:teuthology.orchestra.run.smithi026.stdout:(81/151): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 188 kB/s | 87 kB 00:00 2024-01-15T17:52:32.094 INFO:teuthology.orchestra.run.smithi026.stdout:(82/151): xmlstarlet-1.6.1-20.el9.x86_64.rpm 144 kB/s | 63 kB 00:00 2024-01-15T17:52:32.136 INFO:teuthology.orchestra.run.smithi026.stdout:(83/151): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-01-15T17:52:32.161 INFO:teuthology.orchestra.run.smithi026.stdout:(84/151): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2024-01-15T17:52:32.186 INFO:teuthology.orchestra.run.smithi026.stdout:(85/151): jq-1.6-16.el9.x86_64.rpm 2.0 MB/s | 186 kB 00:00 2024-01-15T17:52:32.236 INFO:teuthology.orchestra.run.smithi026.stdout:(86/151): libgfortran-11.4.1-3.el9.x86_64.rpm 8.0 MB/s | 802 kB 00:00 2024-01-15T17:52:32.270 INFO:teuthology.orchestra.run.smithi026.stdout:(87/151): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.7 MB/s | 217 kB 00:00 2024-01-15T17:52:32.295 INFO:teuthology.orchestra.run.smithi026.stdout:(88/151): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 194 kB 00:00 2024-01-15T17:52:32.329 INFO:teuthology.orchestra.run.smithi026.stdout:(89/151): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-01-15T17:52:32.387 INFO:teuthology.orchestra.run.smithi026.stdout:(90/151): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.1 MB 00:00 2024-01-15T17:52:32.412 INFO:teuthology.orchestra.run.smithi026.stdout:(91/151): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2024-01-15T17:52:32.429 INFO:teuthology.orchestra.run.smithi026.stdout:(92/151): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-01-15T17:52:32.446 INFO:teuthology.orchestra.run.smithi026.stdout:(93/151): python3-requests-2.25.1-7.el9.noarch. 2.0 MB/s | 113 kB 00:00 2024-01-15T17:52:32.480 INFO:teuthology.orchestra.run.smithi026.stdout:(94/151): python3-urllib3-1.26.5-5.el9.noarch.r 3.8 MB/s | 187 kB 00:00 2024-01-15T17:52:32.589 INFO:teuthology.orchestra.run.smithi026.stdout:(95/151): lua-devel-5.4.4-4.el9.x86_64.rpm 159 kB/s | 22 kB 00:00 2024-01-15T17:52:32.614 INFO:teuthology.orchestra.run.smithi124.stdout:(17/151): librgw2-18.0.0-7629.g95c90f41.el9.x86 3.4 MB/s | 4.8 MB 00:01 2024-01-15T17:52:32.697 INFO:teuthology.orchestra.run.smithi124.stdout:(18/151): python3-ceph-argparse-18.0.0-7629.g95 548 kB/s | 45 kB 00:00 2024-01-15T17:52:32.772 INFO:teuthology.orchestra.run.smithi124.stdout:(19/151): python3-ceph-common-18.0.0-7629.g95c9 1.7 MB/s | 129 kB 00:00 2024-01-15T17:52:32.848 INFO:teuthology.orchestra.run.smithi124.stdout:(20/151): python3-cephfs-18.0.0-7629.g95c90f41. 2.1 MB/s | 162 kB 00:00 2024-01-15T17:52:32.973 INFO:teuthology.orchestra.run.smithi124.stdout:(21/151): python3-rados-18.0.0-7629.g95c90f41.e 2.5 MB/s | 320 kB 00:00 2024-01-15T17:52:33.080 INFO:teuthology.orchestra.run.smithi026.stdout:(96/151): python3-devel-3.9.17-2.el9.x86_64.rpm 378 kB/s | 245 kB 00:00 2024-01-15T17:52:33.090 INFO:teuthology.orchestra.run.smithi124.stdout:(22/151): python3-rbd-18.0.0-7629.g95c90f41.el9 2.5 MB/s | 298 kB 00:00 2024-01-15T17:52:33.165 INFO:teuthology.orchestra.run.smithi124.stdout:(23/151): python3-rgw-18.0.0-7629.g95c90f41.el9 1.3 MB/s | 99 kB 00:00 2024-01-15T17:52:33.241 INFO:teuthology.orchestra.run.smithi124.stdout:(24/151): rbd-fuse-18.0.0-7629.g95c90f41.el9.x8 1.1 MB/s | 85 kB 00:00 2024-01-15T17:52:33.514 INFO:teuthology.orchestra.run.smithi026.stdout:(97/151): gperftools-libs-2.9.1-2.el9.x86_64.rp 299 kB/s | 309 kB 00:01 2024-01-15T17:52:33.631 INFO:teuthology.orchestra.run.smithi026.stdout:(98/151): libarrow-doc-9.0.0-10.el9.noarch.rpm 47 kB/s | 26 kB 00:00 2024-01-15T17:52:33.656 INFO:teuthology.orchestra.run.smithi026.stdout:(99/151): liboath-2.6.7-2.el9.x86_64.rpm 344 kB/s | 49 kB 00:00 2024-01-15T17:52:33.848 INFO:teuthology.orchestra.run.smithi026.stdout:(100/151): libunwind-1.6.2-1.el9.x86_64.rpm 311 kB/s | 67 kB 00:00 2024-01-15T17:52:33.874 INFO:teuthology.orchestra.run.smithi026.stdout:(101/151): luarocks-3.9.2-1.el9.noarch.rpm 697 kB/s | 151 kB 00:00 2024-01-15T17:52:34.158 INFO:teuthology.orchestra.run.smithi124.stdout:(25/151): rbd-mirror-18.0.0-7629.g95c90f41.el9. 3.3 MB/s | 3.0 MB 00:00 2024-01-15T17:52:34.241 INFO:teuthology.orchestra.run.smithi124.stdout:(26/151): rbd-nbd-18.0.0-7629.g95c90f41.el9.x86 2.0 MB/s | 170 kB 00:00 2024-01-15T17:52:34.325 INFO:teuthology.orchestra.run.smithi124.stdout:(27/151): ceph-grafana-dashboards-18.0.0-7629.g 340 kB/s | 28 kB 00:00 2024-01-15T17:52:34.417 INFO:teuthology.orchestra.run.smithi124.stdout:(28/151): ceph-mgr-cephadm-18.0.0-7629.g95c90f4 1.4 MB/s | 133 kB 00:00 2024-01-15T17:52:34.849 INFO:teuthology.orchestra.run.smithi026.stdout:(102/151): python3-asyncssh-2.13.2-3.el9.noarch 562 kB/s | 548 kB 00:00 2024-01-15T17:52:35.007 INFO:teuthology.orchestra.run.smithi026.stdout:(103/151): python3-bcrypt-3.2.2-1.el9.x86_64.rp 275 kB/s | 43 kB 00:00 2024-01-15T17:52:35.166 INFO:teuthology.orchestra.run.smithi026.stdout:(104/151): python3-cachetools-4.2.4-1.el9.noarc 204 kB/s | 32 kB 00:00 2024-01-15T17:52:35.249 INFO:teuthology.orchestra.run.smithi026.stdout:(105/151): parquet-libs-9.0.0-10.el9.x86_64.rpm 599 kB/s | 838 kB 00:01 2024-01-15T17:52:35.275 INFO:teuthology.orchestra.run.smithi026.stdout:(106/151): python3-certifi-2021.10.8-2.el9.noar 139 kB/s | 15 kB 00:00 2024-01-15T17:52:35.359 INFO:teuthology.orchestra.run.smithi124.stdout:(29/151): ceph-osd-18.0.0-7629.g95c90f41.el9.x8 2.4 MB/s | 18 MB 00:07 2024-01-15T17:52:35.417 INFO:teuthology.orchestra.run.smithi124.stdout:(30/151): ceph-mgr-dashboard-18.0.0-7629.g95c90 1.8 MB/s | 1.8 MB 00:00 2024-01-15T17:52:35.550 INFO:teuthology.orchestra.run.smithi026.stdout:(107/151): python3-cheroot-8.6.0-4.el9.noarch.r 573 kB/s | 172 kB 00:00 2024-01-15T17:52:35.576 INFO:teuthology.orchestra.run.smithi124.stdout:(31/151): ceph-mgr-modules-core-18.0.0-7629.g95 1.5 MB/s | 246 kB 00:00 2024-01-15T17:52:35.643 INFO:teuthology.orchestra.run.smithi124.stdout:(32/151): ceph-mgr-rook-18.0.0-7629.g95c90f41.e 763 kB/s | 50 kB 00:00 2024-01-15T17:52:35.710 INFO:teuthology.orchestra.run.smithi124.stdout:(33/151): ceph-prometheus-alerts-18.0.0-7629.g9 229 kB/s | 15 kB 00:00 2024-01-15T17:52:35.827 INFO:teuthology.orchestra.run.smithi124.stdout:(34/151): ceph-volume-18.0.0-7629.g95c90f41.el9 2.2 MB/s | 262 kB 00:00 2024-01-15T17:52:35.834 INFO:teuthology.orchestra.run.smithi026.stdout:(108/151): python3-cherrypy-18.6.1-2.el9.noarch 642 kB/s | 358 kB 00:00 2024-01-15T17:52:35.867 INFO:teuthology.orchestra.run.smithi026.stdout:(109/151): python3-google-auth-2.26.0-1.el9.noa 639 kB/s | 202 kB 00:00 2024-01-15T17:52:35.968 INFO:teuthology.orchestra.run.smithi026.stdout:(110/151): python3-jaraco-8.2.1-3.el9.noarch.rp 80 kB/s | 11 kB 00:00 2024-01-15T17:52:35.993 INFO:teuthology.orchestra.run.smithi026.stdout:(111/151): python3-jaraco-classes-3.2.1-5.el9.n 142 kB/s | 18 kB 00:00 2024-01-15T17:52:36.093 INFO:teuthology.orchestra.run.smithi026.stdout:(112/151): python3-jaraco-collections-3.0.0-8.e 186 kB/s | 23 kB 00:00 2024-01-15T17:52:36.102 INFO:teuthology.orchestra.run.smithi124.stdout:(35/151): cephadm-18.0.0-7629.g95c90f41.el9.noa 2.8 MB/s | 774 kB 00:00 2024-01-15T17:52:36.118 INFO:teuthology.orchestra.run.smithi026.stdout:(113/151): python3-jaraco-functools-3.5.0-2.el9 156 kB/s | 19 kB 00:00 2024-01-15T17:52:36.169 INFO:teuthology.orchestra.run.smithi124.stdout:(36/151): cephfs-top-18.0.0-7629.g95c90f41.el9. 276 kB/s | 18 kB 00:00 2024-01-15T17:52:36.210 INFO:teuthology.orchestra.run.smithi026.stdout:(114/151): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-01-15T17:52:36.236 INFO:teuthology.orchestra.run.smithi026.stdout:(115/151): python3-jwt+crypto-2.4.0-1.el9.noarc 78 kB/s | 9.0 kB 00:00 2024-01-15T17:52:36.319 INFO:teuthology.orchestra.run.smithi026.stdout:(116/151): python3-jwt-2.4.0-1.el9.noarch.rpm 378 kB/s | 41 kB 00:00 2024-01-15T17:52:36.386 INFO:teuthology.orchestra.run.smithi124.stdout:(37/151): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 662 kB 00:00 2024-01-15T17:52:36.461 INFO:teuthology.orchestra.run.smithi124.stdout:(38/151): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 920 kB 00:00 2024-01-15T17:52:36.466 INFO:teuthology.orchestra.run.smithi026.stdout:(117/151): python3-logutils-0.3.5-21.el9.noarch 317 kB/s | 46 kB 00:00 2024-01-15T17:52:36.503 INFO:teuthology.orchestra.run.smithi124.stdout:(39/151): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-01-15T17:52:36.545 INFO:teuthology.orchestra.run.smithi124.stdout:(40/151): elfutils-libelf-devel-0.190-2.el9.x86 499 kB/s | 20 kB 00:00 2024-01-15T17:52:36.595 INFO:teuthology.orchestra.run.smithi026.stdout:(118/151): python3-more-itertools-8.12.0-2.el9. 617 kB/s | 79 kB 00:00 2024-01-15T17:52:36.604 INFO:teuthology.orchestra.run.smithi124.stdout:(41/151): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-01-15T17:52:36.646 INFO:teuthology.orchestra.run.smithi124.stdout:(42/151): flexiblas-3.0.4-8.el9.x86_64.rpm 774 kB/s | 32 kB 00:00 2024-01-15T17:52:36.737 INFO:teuthology.orchestra.run.smithi026.stdout:(119/151): python3-natsort-7.1.1-5.el9.noarch.r 409 kB/s | 58 kB 00:00 2024-01-15T17:52:36.805 INFO:teuthology.orchestra.run.smithi124.stdout:(43/151): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-01-15T17:52:36.847 INFO:teuthology.orchestra.run.smithi124.stdout:(44/151): flexiblas-openblas-openmp-3.0.4-8.el9 420 kB/s | 17 kB 00:00 2024-01-15T17:52:36.889 INFO:teuthology.orchestra.run.smithi124.stdout:(45/151): libaio-devel-0.3.111-13.el9.x86_64.rp 246 kB/s | 10 kB 00:00 2024-01-15T17:52:36.939 INFO:teuthology.orchestra.run.smithi124.stdout:(46/151): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2024-01-15T17:52:36.981 INFO:teuthology.orchestra.run.smithi124.stdout:(47/151): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-01-15T17:52:37.048 INFO:teuthology.orchestra.run.smithi124.stdout:(48/151): librdkafka-1.6.1-102.el9.x86_64.rpm 9.8 MB/s | 661 kB 00:00 2024-01-15T17:52:37.107 INFO:teuthology.orchestra.run.smithi124.stdout:(49/151): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-01-15T17:52:37.149 INFO:teuthology.orchestra.run.smithi124.stdout:(50/151): libuuid-devel-2.37.4-15.el9.x86_64.rp 595 kB/s | 24 kB 00:00 2024-01-15T17:52:37.191 INFO:teuthology.orchestra.run.smithi124.stdout:(51/151): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-01-15T17:52:37.203 INFO:teuthology.orchestra.run.smithi026.stdout:(120/151): python3-pecan-1.4.2-3.el9.noarch.rpm 585 kB/s | 272 kB 00:00 2024-01-15T17:52:37.241 INFO:teuthology.orchestra.run.smithi124.stdout:(52/151): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2024-01-15T17:52:37.291 INFO:teuthology.orchestra.run.smithi124.stdout:(53/151): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-01-15T17:52:37.329 INFO:teuthology.orchestra.run.smithi026.stdout:(121/151): python3-portend-3.1.0-2.el9.noarch.r 132 kB/s | 16 kB 00:00 2024-01-15T17:52:37.342 INFO:teuthology.orchestra.run.smithi124.stdout:(54/151): openblas-0.3.21-2.el9.x86_64.rpm 694 kB/s | 34 kB 00:00 2024-01-15T17:52:37.554 INFO:teuthology.orchestra.run.smithi026.stdout:(122/151): python3-pyOpenSSL-21.0.0-1.el9.noarc 401 kB/s | 90 kB 00:00 2024-01-15T17:52:37.621 INFO:teuthology.orchestra.run.smithi026.stdout:(123/151): python3-kubernetes-26.1.0-2.el9.noar 755 kB/s | 1.0 MB 00:01 2024-01-15T17:52:37.625 INFO:teuthology.orchestra.run.smithi124.stdout:(55/151): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-01-15T17:52:37.688 INFO:teuthology.orchestra.run.smithi026.stdout:(124/151): python3-repoze-lru-0.7-16.el9.noarch 232 kB/s | 31 kB 00:00 2024-01-15T17:52:37.792 INFO:teuthology.orchestra.run.smithi124.stdout:(56/151): openssl-devel-3.0.7-25.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-01-15T17:52:37.805 INFO:teuthology.orchestra.run.smithi026.stdout:(125/151): python3-rsa-4.9-2.el9.noarch.rpm 509 kB/s | 59 kB 00:00 2024-01-15T17:52:37.893 INFO:teuthology.orchestra.run.smithi124.stdout:(57/151): postgresql-13.11-1.el9.x86_64.rpm 15 MB/s | 1.5 MB 00:00 2024-01-15T17:52:37.930 INFO:teuthology.orchestra.run.smithi026.stdout:(126/151): python3-routes-2.5.1-5.el9.noarch.rp 611 kB/s | 188 kB 00:00 2024-01-15T17:52:37.955 INFO:teuthology.orchestra.run.smithi026.stdout:(127/151): python3-tempora-5.0.0-2.el9.noarch.r 239 kB/s | 36 kB 00:00 2024-01-15T17:52:37.968 INFO:teuthology.orchestra.run.smithi124.stdout:(58/151): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-01-15T17:52:38.018 INFO:teuthology.orchestra.run.smithi124.stdout:(59/151): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 134 kB 00:00 2024-01-15T17:52:38.056 INFO:teuthology.orchestra.run.smithi026.stdout:(128/151): python3-typing-extensions-4.4.0-2.el 409 kB/s | 51 kB 00:00 2024-01-15T17:52:38.260 INFO:teuthology.orchestra.run.smithi124.stdout:(60/151): postgresql-server-13.11-1.el9.x86_64. 24 MB/s | 5.7 MB 00:00 2024-01-15T17:52:38.281 INFO:teuthology.orchestra.run.smithi026.stdout:(129/151): python3-websocket-client-1.2.3-2.el9 400 kB/s | 90 kB 00:00 2024-01-15T17:52:38.431 INFO:teuthology.orchestra.run.smithi026.stdout:(130/151): python3-webob-1.8.7-6.el9.noarch.rpm 483 kB/s | 230 kB 00:00 2024-01-15T17:52:38.519 INFO:teuthology.orchestra.run.smithi124.stdout:(61/151): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-01-15T17:52:38.557 INFO:teuthology.orchestra.run.smithi026.stdout:(131/151): python3-zc-lockfile-2.0-10.el9.noarc 161 kB/s | 20 kB 00:00 2024-01-15T17:52:38.644 INFO:teuthology.orchestra.run.smithi124.stdout:(62/151): ceph-mgr-diskprediction-local-18.0.0- 2.3 MB/s | 7.4 MB 00:03 2024-01-15T17:52:38.670 INFO:teuthology.orchestra.run.smithi124.stdout:(63/151): python3-jinja2-2.11.3-4.el9.noarch.rp 1.5 MB/s | 228 kB 00:00 2024-01-15T17:52:38.695 INFO:teuthology.orchestra.run.smithi124.stdout:(64/151): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 24 kB 00:00 2024-01-15T17:52:38.720 INFO:teuthology.orchestra.run.smithi124.stdout:(65/151): python3-jsonpointer-2.0-4.el9.noarch. 367 kB/s | 18 kB 00:00 2024-01-15T17:52:38.745 INFO:teuthology.orchestra.run.smithi124.stdout:(66/151): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2024-01-15T17:52:38.771 INFO:teuthology.orchestra.run.smithi124.stdout:(67/151): python3-markupsafe-1.1.1-12.el9.x86_6 654 kB/s | 32 kB 00:00 2024-01-15T17:52:38.874 INFO:teuthology.orchestra.run.smithi026.stdout:(132/151): re2-20211101-3.el9.x86_64.rpm 606 kB/s | 192 kB 00:00 2024-01-15T17:52:38.996 INFO:teuthology.orchestra.run.smithi124.stdout:(68/151): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-01-15T17:52:39.007 INFO:teuthology.orchestra.run.smithi026.stdout:(133/151): python3-werkzeug-2.0.3-3.el9.1.noarc 589 kB/s | 427 kB 00:00 2024-01-15T17:52:39.038 INFO:teuthology.orchestra.run.smithi124.stdout:(69/151): python3-numpy-f2py-1.20.1-5.el9.x86_6 831 kB/s | 217 kB 00:00 2024-01-15T17:52:39.063 INFO:teuthology.orchestra.run.smithi124.stdout:(70/151): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-01-15T17:52:39.089 INFO:teuthology.orchestra.run.smithi124.stdout:(71/151): python3-prettytable-0.7.2-27.el9.noar 852 kB/s | 41 kB 00:00 2024-01-15T17:52:39.122 INFO:teuthology.orchestra.run.smithi124.stdout:(72/151): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 132 kB 00:00 2024-01-15T17:52:39.156 INFO:teuthology.orchestra.run.smithi124.stdout:(73/151): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2024-01-15T17:52:39.181 INFO:teuthology.orchestra.run.smithi124.stdout:(74/151): python3-pytz-2021.1-5.el9.noarch.rpm 840 kB/s | 47 kB 00:00 2024-01-15T17:52:39.215 INFO:teuthology.orchestra.run.smithi124.stdout:(75/151): python3-requests-oauthlib-1.3.0-12.el 746 kB/s | 43 kB 00:00 2024-01-15T17:52:39.482 INFO:teuthology.orchestra.run.smithi124.stdout:(76/151): python3-toml-0.10.2-6.el9.noarch.rpm 146 kB/s | 39 kB 00:00 2024-01-15T17:52:39.483 INFO:teuthology.orchestra.run.smithi026.stdout:(134/151): libarrow-9.0.0-10.el9.x86_64.rpm 663 kB/s | 4.4 MB 00:06 2024-01-15T17:52:39.783 INFO:teuthology.orchestra.run.smithi026.stdout:(135/151): librbd1-18.0.0-7629.g95c90f41.el9.x8 10 MB/s | 3.0 MB 00:00 2024-01-15T17:52:39.833 INFO:teuthology.orchestra.run.smithi026.stdout:(136/151): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2024-01-15T17:52:39.941 INFO:teuthology.orchestra.run.smithi124.stdout:(77/151): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-01-15T17:52:39.974 INFO:teuthology.orchestra.run.smithi124.stdout:(78/151): socat-1.7.4.1-5.el9.x86_64.rpm 622 kB/s | 300 kB 00:00 2024-01-15T17:52:39.999 INFO:teuthology.orchestra.run.smithi124.stdout:(79/151): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-01-15T17:52:40.000 INFO:teuthology.orchestra.run.smithi026.stdout:(137/151): NetworkManager-1.45.9-1.el9.x86_64.r 14 MB/s | 2.2 MB 00:00 2024-01-15T17:52:40.041 INFO:teuthology.orchestra.run.smithi124.stdout:(80/151): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2024-01-15T17:52:40.066 INFO:teuthology.orchestra.run.smithi124.stdout:(81/151): xmlstarlet-1.6.1-20.el9.x86_64.rpm 945 kB/s | 63 kB 00:00 2024-01-15T17:52:40.175 INFO:teuthology.orchestra.run.smithi124.stdout:(82/151): zlib-devel-1.2.11-40.el9.x86_64.rpm 333 kB/s | 44 kB 00:00 2024-01-15T17:52:40.192 INFO:teuthology.orchestra.run.smithi026.stdout:(138/151): NetworkManager-libnm-1.45.9-1.el9.x8 9.3 MB/s | 1.8 MB 00:00 2024-01-15T17:52:40.200 INFO:teuthology.orchestra.run.smithi124.stdout:(83/151): fuse-2.9.9-15.el9.x86_64.rpm 591 kB/s | 79 kB 00:00 2024-01-15T17:52:40.250 INFO:teuthology.orchestra.run.smithi026.stdout:(139/151): NetworkManager-team-1.45.9-1.el9.x86 738 kB/s | 42 kB 00:00 2024-01-15T17:52:40.284 INFO:teuthology.orchestra.run.smithi124.stdout:(84/151): jq-1.6-16.el9.x86_64.rpm 2.2 MB/s | 186 kB 00:00 2024-01-15T17:52:40.326 INFO:teuthology.orchestra.run.smithi026.stdout:(140/151): NetworkManager-tui-1.45.9-1.el9.x86_ 3.2 MB/s | 246 kB 00:00 2024-01-15T17:52:40.326 INFO:teuthology.orchestra.run.smithi124.stdout:(85/151): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 802 kB 00:00 2024-01-15T17:52:40.360 INFO:teuthology.orchestra.run.smithi124.stdout:(86/151): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 194 kB 00:00 2024-01-15T17:52:40.375 INFO:teuthology.orchestra.run.smithi026.stdout:(141/151): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 69 kB 00:00 2024-01-15T17:52:40.385 INFO:teuthology.orchestra.run.smithi124.stdout:(87/151): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.8 MB/s | 217 kB 00:00 2024-01-15T17:52:40.418 INFO:teuthology.orchestra.run.smithi124.stdout:(88/151): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 241 kB 00:00 2024-01-15T17:52:40.426 INFO:teuthology.orchestra.run.smithi026.stdout:(142/151): elfutils-debuginfod-client-0.190-2.e 714 kB/s | 35 kB 00:00 2024-01-15T17:52:40.486 INFO:teuthology.orchestra.run.smithi124.stdout:(89/151): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-01-15T17:52:40.493 INFO:teuthology.orchestra.run.smithi026.stdout:(143/151): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 191 kB 00:00 2024-01-15T17:52:40.511 INFO:teuthology.orchestra.run.smithi124.stdout:(90/151): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-01-15T17:52:40.536 INFO:teuthology.orchestra.run.smithi124.stdout:(91/151): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-01-15T17:52:40.570 INFO:teuthology.orchestra.run.smithi124.stdout:(92/151): python3-requests-2.25.1-7.el9.noarch. 1.9 MB/s | 113 kB 00:00 2024-01-15T17:52:40.604 INFO:teuthology.orchestra.run.smithi124.stdout:(93/151): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 187 kB 00:00 2024-01-15T17:52:40.618 INFO:teuthology.orchestra.run.smithi026.stdout:(144/151): elfutils-libs-0.190-2.el9.x86_64.rpm 2.0 MB/s | 254 kB 00:00 2024-01-15T17:52:40.669 INFO:teuthology.orchestra.run.smithi026.stdout:(145/151): libacl-2.3.1-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-01-15T17:52:40.721 INFO:teuthology.orchestra.run.smithi124.stdout:(94/151): lua-devel-5.4.4-4.el9.x86_64.rpm 193 kB/s | 22 kB 00:00 2024-01-15T17:52:40.802 INFO:teuthology.orchestra.run.smithi026.stdout:(146/151): openssl-3.0.7-25.el9.x86_64.rpm 8.7 MB/s | 1.2 MB 00:00 2024-01-15T17:52:41.011 INFO:teuthology.orchestra.run.smithi026.stdout:(147/151): openssl-libs-3.0.7-25.el9.x86_64.rpm 10 MB/s | 2.1 MB 00:00 2024-01-15T17:52:41.086 INFO:teuthology.orchestra.run.smithi026.stdout:(148/151): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-01-15T17:52:41.219 INFO:teuthology.orchestra.run.smithi026.stdout:(149/151): librados2-18.0.0-7629.g95c90f41.el9. 1.5 MB/s | 3.3 MB 00:02 2024-01-15T17:52:41.262 INFO:teuthology.orchestra.run.smithi026.stdout:(150/151): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.0 MB/s | 1.1 MB 00:00 2024-01-15T17:52:41.555 INFO:teuthology.orchestra.run.smithi124.stdout:(95/151): gperftools-libs-2.9.1-2.el9.x86_64.rp 371 kB/s | 309 kB 00:00 2024-01-15T17:52:41.713 INFO:teuthology.orchestra.run.smithi124.stdout:(96/151): python3-devel-3.9.17-2.el9.x86_64.rpm 215 kB/s | 245 kB 00:01 2024-01-15T17:52:41.945 INFO:teuthology.orchestra.run.smithi026.stdout:(151/151): thrift-0.15.0-2.el9.x86_64.rpm 527 kB/s | 1.6 MB 00:03 2024-01-15T17:52:41.959 INFO:teuthology.orchestra.run.smithi026.stdout:-------------------------------------------------------------------------------- 2024-01-15T17:52:41.960 INFO:teuthology.orchestra.run.smithi026.stdout:Total 9.8 MB/s | 204 MB 00:20 2024-01-15T17:52:42.048 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction check 2024-01-15T17:52:42.163 INFO:teuthology.orchestra.run.smithi124.stdout:(97/151): libarrow-doc-9.0.0-10.el9.noarch.rpm 58 kB/s | 26 kB 00:00 2024-01-15T17:52:42.356 INFO:teuthology.orchestra.run.smithi124.stdout:(98/151): liboath-2.6.7-2.el9.x86_64.rpm 254 kB/s | 49 kB 00:00 2024-01-15T17:52:42.413 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction check succeeded. 2024-01-15T17:52:42.413 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction test 2024-01-15T17:52:42.464 INFO:teuthology.orchestra.run.smithi124.stdout:(99/151): libunwind-1.6.2-1.el9.x86_64.rpm 627 kB/s | 67 kB 00:00 2024-01-15T17:52:42.648 INFO:teuthology.orchestra.run.smithi124.stdout:(100/151): luarocks-3.9.2-1.el9.noarch.rpm 828 kB/s | 151 kB 00:00 2024-01-15T17:52:42.881 INFO:teuthology.orchestra.run.smithi124.stdout:(101/151): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-01-15T17:52:43.082 INFO:teuthology.orchestra.run.smithi124.stdout:(102/151): libarrow-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-01-15T17:52:43.124 INFO:teuthology.orchestra.run.smithi124.stdout:(103/151): python3-asyncssh-2.13.2-3.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-01-15T17:52:43.232 INFO:teuthology.orchestra.run.smithi124.stdout:(104/151): python3-cachetools-4.2.4-1.el9.noarc 299 kB/s | 32 kB 00:00 2024-01-15T17:52:43.349 INFO:teuthology.orchestra.run.smithi124.stdout:(105/151): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-01-15T17:52:43.374 INFO:teuthology.orchestra.run.smithi124.stdout:(106/151): python3-bcrypt-3.2.2-1.el9.x86_64.rp 149 kB/s | 43 kB 00:00 2024-01-15T17:52:43.483 INFO:teuthology.orchestra.run.smithi124.stdout:(107/151): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-01-15T17:52:43.609 INFO:teuthology.orchestra.run.smithi124.stdout:(108/151): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-01-15T17:52:43.642 INFO:teuthology.orchestra.run.smithi124.stdout:(109/151): python3-google-auth-2.26.0-1.el9.noa 1.3 MB/s | 202 kB 00:00 2024-01-15T17:52:43.726 INFO:teuthology.orchestra.run.smithi124.stdout:(110/151): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-01-15T17:52:43.801 INFO:teuthology.orchestra.run.smithi124.stdout:(111/151): python3-jaraco-classes-3.2.1-5.el9.n 112 kB/s | 18 kB 00:00 2024-01-15T17:52:43.868 INFO:teuthology.orchestra.run.smithi124.stdout:(112/151): python3-jaraco-collections-3.0.0-8.e 164 kB/s | 23 kB 00:00 2024-01-15T17:52:43.960 INFO:teuthology.orchestra.run.smithi124.stdout:(113/151): python3-jaraco-functools-3.5.0-2.el9 123 kB/s | 19 kB 00:00 2024-01-15T17:52:43.985 INFO:teuthology.orchestra.run.smithi124.stdout:(114/151): python3-jaraco-text-3.2.0-6.el9.noar 169 kB/s | 20 kB 00:00 2024-01-15T17:52:44.086 INFO:teuthology.orchestra.run.smithi124.stdout:(115/151): python3-jwt+crypto-2.4.0-1.el9.noarc 73 kB/s | 9.0 kB 00:00 2024-01-15T17:52:44.111 INFO:teuthology.orchestra.run.smithi124.stdout:(116/151): python3-jwt-2.4.0-1.el9.noarch.rpm 328 kB/s | 41 kB 00:00 2024-01-15T17:52:44.245 INFO:teuthology.orchestra.run.smithi124.stdout:(117/151): python3-logutils-0.3.5-21.el9.noarch 349 kB/s | 46 kB 00:00 2024-01-15T17:52:44.295 INFO:teuthology.orchestra.run.smithi124.stdout:(118/151): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-01-15T17:52:44.370 INFO:teuthology.orchestra.run.smithi124.stdout:(119/151): python3-more-itertools-8.12.0-2.el9. 633 kB/s | 79 kB 00:00 2024-01-15T17:52:44.429 INFO:teuthology.orchestra.run.smithi124.stdout:(120/151): python3-natsort-7.1.1-5.el9.noarch.r 432 kB/s | 58 kB 00:00 2024-01-15T17:52:44.521 INFO:teuthology.orchestra.run.smithi124.stdout:(121/151): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-01-15T17:52:44.571 INFO:teuthology.orchestra.run.smithi124.stdout:(122/151): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-01-15T17:52:44.655 INFO:teuthology.orchestra.run.smithi124.stdout:(123/151): python3-pyOpenSSL-21.0.0-1.el9.noarc 676 kB/s | 90 kB 00:00 2024-01-15T17:52:44.704 INFO:teuthology.orchestra.run.smithi026.stdout:Transaction test succeeded. 2024-01-15T17:52:44.704 INFO:teuthology.orchestra.run.smithi026.stdout:Running transaction 2024-01-15T17:52:44.714 INFO:teuthology.orchestra.run.smithi124.stdout:(124/151): python3-repoze-lru-0.7-16.el9.noarch 217 kB/s | 31 kB 00:00 2024-01-15T17:52:44.814 INFO:teuthology.orchestra.run.smithi124.stdout:(125/151): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-01-15T17:52:44.856 INFO:teuthology.orchestra.run.smithi124.stdout:(126/151): python3-rsa-4.9-2.el9.noarch.rpm 418 kB/s | 59 kB 00:00 2024-01-15T17:52:44.981 INFO:teuthology.orchestra.run.smithi124.stdout:(127/151): python3-tempora-5.0.0-2.el9.noarch.r 215 kB/s | 36 kB 00:00 2024-01-15T17:52:45.015 INFO:teuthology.orchestra.run.smithi124.stdout:(128/151): python3-typing-extensions-4.4.0-2.el 323 kB/s | 51 kB 00:00 2024-01-15T17:52:45.149 INFO:teuthology.orchestra.run.smithi124.stdout:(129/151): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-01-15T17:52:45.232 INFO:teuthology.orchestra.run.smithi124.stdout:(130/151): python3-websocket-client-1.2.3-2.el9 414 kB/s | 90 kB 00:00 2024-01-15T17:52:45.399 INFO:teuthology.orchestra.run.smithi124.stdout:(131/151): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-01-15T17:52:45.633 INFO:teuthology.orchestra.run.smithi124.stdout:(132/151): re2-20211101-3.el9.x86_64.rpm 823 kB/s | 192 kB 00:00 2024-01-15T17:52:45.658 INFO:teuthology.orchestra.run.smithi124.stdout:(133/151): python3-zc-lockfile-2.0-10.el9.noarc 47 kB/s | 20 kB 00:00 2024-01-15T17:52:46.117 INFO:teuthology.orchestra.run.smithi124.stdout:(134/151): thrift-0.15.0-2.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2024-01-15T17:52:46.643 INFO:teuthology.orchestra.run.smithi124.stdout:(135/151): ceph-test-18.0.0-7629.g95c90f41.el9. 2.5 MB/s | 41 MB 00:16 2024-01-15T17:52:46.684 INFO:teuthology.orchestra.run.smithi124.stdout:(136/151): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-01-15T17:52:46.834 INFO:teuthology.orchestra.run.smithi124.stdout:(137/151): NetworkManager-1.45.9-1.el9.x86_64.r 15 MB/s | 2.2 MB 00:00 2024-01-15T17:52:46.952 INFO:teuthology.orchestra.run.smithi124.stdout:(138/151): NetworkManager-libnm-1.45.9-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-01-15T17:52:47.023 INFO:teuthology.orchestra.run.smithi124.stdout:(139/151): NetworkManager-team-1.45.9-1.el9.x86 604 kB/s | 42 kB 00:00 2024-01-15T17:52:47.077 INFO:teuthology.orchestra.run.smithi124.stdout:(140/151): NetworkManager-tui-1.45.9-1.el9.x86_ 4.5 MB/s | 246 kB 00:00 2024-01-15T17:52:47.119 INFO:teuthology.orchestra.run.smithi124.stdout:(141/151): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2024-01-15T17:52:47.169 INFO:teuthology.orchestra.run.smithi124.stdout:(142/151): elfutils-debuginfod-client-0.190-2.e 718 kB/s | 35 kB 00:00 2024-01-15T17:52:47.228 INFO:teuthology.orchestra.run.smithi124.stdout:(143/151): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 191 kB 00:00 2024-01-15T17:52:47.312 INFO:teuthology.orchestra.run.smithi124.stdout:(144/151): librados2-18.0.0-7629.g95c90f41.el9. 2.0 MB/s | 3.3 MB 00:01 2024-01-15T17:52:47.362 INFO:teuthology.orchestra.run.smithi124.stdout:(145/151): libacl-2.3.1-4.el9.x86_64.rpm 448 kB/s | 22 kB 00:00 2024-01-15T17:52:47.488 INFO:teuthology.orchestra.run.smithi124.stdout:(146/151): openssl-3.0.7-25.el9.x86_64.rpm 9.3 MB/s | 1.2 MB 00:00 2024-01-15T17:52:47.521 INFO:teuthology.orchestra.run.smithi124.stdout:(147/151): elfutils-libs-0.190-2.el9.x86_64.rpm 872 kB/s | 254 kB 00:00 2024-01-15T17:52:47.622 INFO:teuthology.orchestra.run.smithi124.stdout:(148/151): librbd1-18.0.0-7629.g95c90f41.el9.x8 2.0 MB/s | 3.0 MB 00:01 2024-01-15T17:52:47.706 INFO:teuthology.orchestra.run.smithi124.stdout:(149/151): openssl-libs-3.0.7-25.el9.x86_64.rpm 10 MB/s | 2.1 MB 00:00 2024-01-15T17:52:47.731 INFO:teuthology.orchestra.run.smithi124.stdout:(150/151): xfsdump-3.1.12-4.el9.x86_64.rpm 1.5 MB/s | 321 kB 00:00 2024-01-15T17:52:47.781 INFO:teuthology.orchestra.run.smithi124.stdout:(151/151): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.3 MB/s | 1.1 MB 00:00 2024-01-15T17:52:47.794 INFO:teuthology.orchestra.run.smithi124.stdout:-------------------------------------------------------------------------------- 2024-01-15T17:52:47.794 INFO:teuthology.orchestra.run.smithi124.stdout:Total 8.4 MB/s | 204 MB 00:24 2024-01-15T17:52:47.879 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction check 2024-01-15T17:52:48.224 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction check succeeded. 2024-01-15T17:52:48.224 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction test 2024-01-15T17:52:50.421 INFO:teuthology.orchestra.run.smithi124.stdout:Transaction test succeeded. 2024-01-15T17:52:50.421 INFO:teuthology.orchestra.run.smithi124.stdout:Running transaction 2024-01-15T17:52:51.333 INFO:teuthology.orchestra.run.smithi026.stdout: Preparing : 1/1 2024-01-15T17:52:51.538 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/167 2024-01-15T17:52:51.579 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/167 2024-01-15T17:52:51.677 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/167 2024-01-15T17:52:51.738 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/167 2024-01-15T17:52:51.771 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/167 2024-01-15T17:52:51.864 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/167 2024-01-15T17:52:52.283 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/167 2024-01-15T17:52:52.318 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librados2-2:18.0.0-7629.g95c90f41.el9.x86_64 8/167 2024-01-15T17:52:52.475 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:18.0.0-7629.g95c90f41.el9.x86_64 8/167 2024-01-15T17:52:52.502 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs2-2:18.0.0-7629.g95c90f41.el9.x86_64 9/167 2024-01-15T17:52:52.649 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephfs2-2:18.0.0-7629.g95c90f41.el9.x86_64 9/167 2024-01-15T17:52:52.679 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rados-2:18.0.0-7629.g95c90f41.el9.x86_64 10/167 2024-01-15T17:52:52.823 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/167 2024-01-15T17:52:52.936 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/167 2024-01-15T17:52:53.233 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/167 2024-01-15T17:52:53.293 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/167 2024-01-15T17:52:53.317 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephsqlite-2:18.0.0-7629.g95c90f41.el9.x86_64 15/167 2024-01-15T17:52:53.501 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7629.g95c90f41.el9.x86_64 15/167 2024-01-15T17:52:53.527 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libradosstriper1-2:18.0.0-7629.g95c90f41.el9.x86 16/167 2024-01-15T17:52:53.611 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7629.g95c90f41.el9.x86 16/167 2024-01-15T17:52:53.957 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/167 2024-01-15T17:52:54.101 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-libnm-1:1.45.9-1.el9.x86_64 18/167 2024-01-15T17:52:54.186 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.9-1.el9.x86_64 18/167 2024-01-15T17:52:54.382 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-15T17:52:54.428 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-15T17:52:54.880 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-15T17:52:54.926 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/167 2024-01-15T17:52:55.052 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/167 2024-01-15T17:52:55.134 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/167 2024-01-15T17:52:55.181 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 23/167 2024-01-15T17:52:55.244 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/167 2024-01-15T17:52:55.305 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/167 2024-01-15T17:52:55.410 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/167 2024-01-15T17:52:55.467 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/167 2024-01-15T17:52:55.512 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/167 2024-01-15T17:52:55.551 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/167 2024-01-15T17:52:55.674 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/167 2024-01-15T17:52:55.714 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-common-2:18.0.0-7629.g95c90f41.el9. 31/167 2024-01-15T17:52:55.770 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ceph-argparse-2:18.0.0-7629.g95c90f41.el 32/167 2024-01-15T17:52:55.803 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cephfs-2:18.0.0-7629.g95c90f41.el9.x86_6 33/167 2024-01-15T17:52:55.928 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/167 2024-01-15T17:52:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/167 2024-01-15T17:52:56.223 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/167 2024-01-15T17:52:56.291 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/167 2024-01-15T17:52:56.666 INFO:teuthology.orchestra.run.smithi124.stdout: Preparing : 1/1 2024-01-15T17:52:56.832 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/167 2024-01-15T17:52:56.865 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/167 2024-01-15T17:52:56.936 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/167 2024-01-15T17:52:56.991 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/167 2024-01-15T17:52:57.017 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/167 2024-01-15T17:52:57.022 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/167 2024-01-15T17:52:57.108 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/167 2024-01-15T17:52:57.120 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/167 2024-01-15T17:52:57.144 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/167 2024-01-15T17:52:57.213 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/167 2024-01-15T17:52:57.261 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librados-devel-2:18.0.0-7629.g95c90f41.el9.x86_6 42/167 2024-01-15T17:52:57.324 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/167 2024-01-15T17:52:57.530 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/167 2024-01-15T17:52:57.531 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/167 2024-01-15T17:52:57.573 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/167 2024-01-15T17:52:57.579 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : librados2-2:18.0.0-7629.g95c90f41.el9.x86_64 8/167 2024-01-15T17:52:57.730 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/167 2024-01-15T17:52:57.742 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librados2-2:18.0.0-7629.g95c90f41.el9.x86_64 8/167 2024-01-15T17:52:57.768 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/167 2024-01-15T17:52:57.780 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcephfs2-2:18.0.0-7629.g95c90f41.el9.x86_64 9/167 2024-01-15T17:52:57.834 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/167 2024-01-15T17:52:57.902 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libcephfs2-2:18.0.0-7629.g95c90f41.el9.x86_64 9/167 2024-01-15T17:52:57.922 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/167 2024-01-15T17:52:57.945 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rados-2:18.0.0-7629.g95c90f41.el9.x86_64 10/167 2024-01-15T17:52:57.960 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/167 2024-01-15T17:52:57.999 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/167 2024-01-15T17:52:58.067 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/167 2024-01-15T17:52:58.084 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/167 2024-01-15T17:52:58.125 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/167 2024-01-15T17:52:58.185 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/167 2024-01-15T17:52:58.229 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/167 2024-01-15T17:52:58.382 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/167 2024-01-15T17:52:58.450 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/167 2024-01-15T17:52:58.506 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 57/167 2024-01-15T17:52:58.597 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/167 2024-01-15T17:52:58.639 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/167 2024-01-15T17:52:58.655 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/167 2024-01-15T17:52:58.671 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcephsqlite-2:18.0.0-7629.g95c90f41.el9.x86_64 15/167 2024-01-15T17:52:58.674 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-google-auth-1:2.26.0-1.el9.noarch 59/167 2024-01-15T17:52:58.722 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/167 2024-01-15T17:52:58.757 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/167 2024-01-15T17:52:58.818 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7629.g95c90f41.el9.x86_64 15/167 2024-01-15T17:52:58.847 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libradosstriper1-2:18.0.0-7629.g95c90f41.el9.x86 16/167 2024-01-15T17:52:58.915 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 62/167 2024-01-15T17:52:58.930 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7629.g95c90f41.el9.x86 16/167 2024-01-15T17:52:58.970 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 63/167 2024-01-15T17:52:59.040 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-ply-3.11-14.el9.noarch 64/167 2024-01-15T17:52:59.124 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 65/167 2024-01-15T17:52:59.240 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/167 2024-01-15T17:52:59.409 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 66/167 2024-01-15T17:52:59.465 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 67/167 2024-01-15T17:52:59.474 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : NetworkManager-libnm-1:1.45.9-1.el9.x86_64 18/167 2024-01-15T17:52:59.566 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.9-1.el9.x86_64 18/167 2024-01-15T17:52:59.658 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 68/167 2024-01-15T17:52:59.719 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 69/167 2024-01-15T17:52:59.751 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 70/167 2024-01-15T17:52:59.772 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-15T17:52:59.793 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 71/167 2024-01-15T17:52:59.830 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-15T17:52:59.845 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 72/167 2024-01-15T17:52:59.893 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 73/167 2024-01-15T17:52:59.944 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : jq-1.6-16.el9.x86_64 74/167 2024-01-15T17:53:00.000 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : fuse-2.9.9-15.el9.x86_64 75/167 2024-01-15T17:53:00.044 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 76/167 2024-01-15T17:53:00.096 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 77/167 2024-01-15T17:53:00.167 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : uuid-1.6.2-55.el9.x86_64 78/167 2024-01-15T17:53:00.210 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 79/167 2024-01-15T17:53:00.266 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-1:1.45.9-1.el9.x86_64 19/167 2024-01-15T17:53:00.319 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/167 2024-01-15T17:53:00.346 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 80/167 2024-01-15T17:53:00.378 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 81/167 2024-01-15T17:53:00.413 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 82/167 2024-01-15T17:53:00.435 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/167 2024-01-15T17:53:00.468 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 83/167 2024-01-15T17:53:00.509 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 84/167 2024-01-15T17:53:00.512 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/167 2024-01-15T17:53:00.558 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 23/167 2024-01-15T17:53:00.621 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/167 2024-01-15T17:53:00.658 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 85/167 2024-01-15T17:53:00.677 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/167 2024-01-15T17:53:00.707 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 86/167 2024-01-15T17:53:00.791 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/167 2024-01-15T17:53:00.834 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/167 2024-01-15T17:53:00.855 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 87/167 2024-01-15T17:53:00.879 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/167 2024-01-15T17:53:00.911 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/167 2024-01-15T17:53:00.921 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 88/167 2024-01-15T17:53:01.024 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/167 2024-01-15T17:53:01.065 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-ceph-common-2:18.0.0-7629.g95c90f41.el9. 31/167 2024-01-15T17:53:01.107 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-ceph-argparse-2:18.0.0-7629.g95c90f41.el 32/167 2024-01-15T17:53:01.138 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cephfs-2:18.0.0-7629.g95c90f41.el9.x86_6 33/167 2024-01-15T17:53:01.251 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/167 2024-01-15T17:53:01.350 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/167 2024-01-15T17:53:01.526 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/167 2024-01-15T17:53:01.593 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/167 2024-01-15T17:53:01.949 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 89/167 2024-01-15T17:53:02.004 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 90/167 2024-01-15T17:53:02.037 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 91/167 2024-01-15T17:53:02.070 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 92/167 2024-01-15T17:53:02.282 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/167 2024-01-15T17:53:02.330 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/167 2024-01-15T17:53:02.357 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/167 2024-01-15T17:53:02.373 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/167 2024-01-15T17:53:02.430 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/167 2024-01-15T17:53:02.453 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/167 2024-01-15T17:53:02.476 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/167 2024-01-15T17:53:02.564 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librados-devel-2:18.0.0-7629.g95c90f41.el9.x86_6 42/167 2024-01-15T17:53:02.628 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/167 2024-01-15T17:53:02.828 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/167 2024-01-15T17:53:02.876 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/167 2024-01-15T17:53:03.006 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/167 2024-01-15T17:53:03.047 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/167 2024-01-15T17:53:03.109 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/167 2024-01-15T17:53:03.194 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/167 2024-01-15T17:53:03.230 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/167 2024-01-15T17:53:03.268 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/167 2024-01-15T17:53:03.303 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/167 2024-01-15T17:53:03.333 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/167 2024-01-15T17:53:03.393 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/167 2024-01-15T17:53:03.451 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/167 2024-01-15T17:53:03.454 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/167 2024-01-15T17:53:03.645 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/167 2024-01-15T17:53:03.696 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/167 2024-01-15T17:53:03.743 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 57/167 2024-01-15T17:53:03.871 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/167 2024-01-15T17:53:03.921 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-google-auth-1:2.26.0-1.el9.noarch 59/167 2024-01-15T17:53:03.970 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/167 2024-01-15T17:53:04.003 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/167 2024-01-15T17:53:04.157 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 62/167 2024-01-15T17:53:04.210 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 63/167 2024-01-15T17:53:04.277 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-ply-3.11-14.el9.noarch 64/167 2024-01-15T17:53:04.354 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 65/167 2024-01-15T17:53:04.624 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 66/167 2024-01-15T17:53:04.690 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 67/167 2024-01-15T17:53:04.861 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 68/167 2024-01-15T17:53:04.917 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 69/167 2024-01-15T17:53:04.942 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 70/167 2024-01-15T17:53:04.985 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 71/167 2024-01-15T17:53:05.033 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 72/167 2024-01-15T17:53:05.073 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/167 2024-01-15T17:53:05.074 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 73/167 2024-01-15T17:53:05.123 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : jq-1.6-16.el9.x86_64 74/167 2024-01-15T17:53:05.155 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/167 2024-01-15T17:53:05.164 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : fuse-2.9.9-15.el9.x86_64 75/167 2024-01-15T17:53:05.192 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/167 2024-01-15T17:53:05.216 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 76/167 2024-01-15T17:53:05.226 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 101/167 2024-01-15T17:53:05.277 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 77/167 2024-01-15T17:53:05.339 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : uuid-1.6.2-55.el9.x86_64 78/167 2024-01-15T17:53:05.398 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 79/167 2024-01-15T17:53:05.528 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 80/167 2024-01-15T17:53:05.583 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 81/167 2024-01-15T17:53:05.611 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/167 2024-01-15T17:53:05.627 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 82/167 2024-01-15T17:53:05.643 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : librbd1-2:18.0.0-7629.g95c90f41.el9.x86_64 103/167 2024-01-15T17:53:05.674 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 83/167 2024-01-15T17:53:05.714 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 84/167 2024-01-15T17:53:05.764 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:18.0.0-7629.g95c90f41.el9.x86_64 103/167 2024-01-15T17:53:05.799 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rbd-2:18.0.0-7629.g95c90f41.el9.x86_64 104/167 2024-01-15T17:53:05.855 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/167 2024-01-15T17:53:05.856 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 85/167 2024-01-15T17:53:05.905 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 86/167 2024-01-15T17:53:06.045 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 87/167 2024-01-15T17:53:06.111 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 88/167 2024-01-15T17:53:06.368 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 106/167 2024-01-15T17:53:06.421 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : librgw2-2:18.0.0-7629.g95c90f41.el9.x86_64 107/167 2024-01-15T17:53:06.515 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librgw2-2:18.0.0-7629.g95c90f41.el9.x86_64 107/167 2024-01-15T17:53:06.540 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : python3-rgw-2:18.0.0-7629.g95c90f41.el9.x86_64 108/167 2024-01-15T17:53:07.071 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 89/167 2024-01-15T17:53:07.126 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 90/167 2024-01-15T17:53:07.159 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 91/167 2024-01-15T17:53:07.192 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 92/167 2024-01-15T17:53:07.436 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/167 2024-01-15T17:53:07.464 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/167 2024-01-15T17:53:07.555 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/167 2024-01-15T17:53:08.352 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/167 2024-01-15T17:53:08.482 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/167 2024-01-15T17:53:09.411 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 109/167 2024-01-15T17:53:09.461 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 109/167 2024-01-15T17:53:09.989 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/167 2024-01-15T17:53:10.086 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/167 2024-01-15T17:53:10.123 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/167 2024-01-15T17:53:10.131 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 109/167 2024-01-15T17:53:10.157 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 101/167 2024-01-15T17:53:10.173 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-base-2:18.0.0-7629.g95c90f41.el9.x86_64 110/167 2024-01-15T17:53:10.372 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-base-2:18.0.0-7629.g95c90f41.el9.x86_64 110/167 2024-01-15T17:53:10.373 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-15T17:53:10.373 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-15T17:53:10.373 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:10.415 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-selinux-2:18.0.0-7629.g95c90f41.el9.x86_64 111/167 2024-01-15T17:53:10.514 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/167 2024-01-15T17:53:10.548 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : librbd1-2:18.0.0-7629.g95c90f41.el9.x86_64 103/167 2024-01-15T17:53:10.658 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librbd1-2:18.0.0-7629.g95c90f41.el9.x86_64 103/167 2024-01-15T17:53:10.720 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rbd-2:18.0.0-7629.g95c90f41.el9.x86_64 104/167 2024-01-15T17:53:10.768 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/167 2024-01-15T17:53:11.252 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 106/167 2024-01-15T17:53:11.284 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : librgw2-2:18.0.0-7629.g95c90f41.el9.x86_64 107/167 2024-01-15T17:53:11.358 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librgw2-2:18.0.0-7629.g95c90f41.el9.x86_64 107/167 2024-01-15T17:53:11.386 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : python3-rgw-2:18.0.0-7629.g95c90f41.el9.x86_64 108/167 2024-01-15T17:53:13.833 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 109/167 2024-01-15T17:53:13.874 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 109/167 2024-01-15T17:53:14.504 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 109/167 2024-01-15T17:53:14.552 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-base-2:18.0.0-7629.g95c90f41.el9.x86_64 110/167 2024-01-15T17:53:14.719 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-base-2:18.0.0-7629.g95c90f41.el9.x86_64 110/167 2024-01-15T17:53:14.719 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-15T17:53:14.719 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-15T17:53:14.719 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:14.761 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-selinux-2:18.0.0-7629.g95c90f41.el9.x86_64 111/167 2024-01-15T17:53:23.321 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7629.g95c90f41.el9.x86_64 111/167 2024-01-15T17:53:23.321 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /sys 2024-01-15T17:53:23.321 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /proc 2024-01-15T17:53:23.321 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /mnt 2024-01-15T17:53:23.321 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /var/tmp 2024-01-15T17:53:23.321 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /home 2024-01-15T17:53:23.321 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /root 2024-01-15T17:53:23.321 INFO:teuthology.orchestra.run.smithi026.stdout:skipping the directory /tmp 2024-01-15T17:53:23.322 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:23.545 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-volume-2:18.0.0-7629.g95c90f41.el9.noarch 112/167 2024-01-15T17:53:23.594 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-volume-2:18.0.0-7629.g95c90f41.el9.noarch 112/167 2024-01-15T17:53:23.594 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:23.594 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-15T17:53:23.594 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:25.374 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-osd-2:18.0.0-7629.g95c90f41.el9.x86_64 113/167 2024-01-15T17:53:25.455 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-osd-2:18.0.0-7629.g95c90f41.el9.x86_64 113/167 2024-01-15T17:53:25.455 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:25.455 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-15T17:53:25.455 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-15T17:53:25.455 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-15T17:53:25.455 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:25.690 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mds-2:18.0.0-7629.g95c90f41.el9.x86_64 114/167 2024-01-15T17:53:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mds-2:18.0.0-7629.g95c90f41.el9.x86_64 114/167 2024-01-15T17:53:25.772 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:25.772 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-15T17:53:25.772 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-15T17:53:25.772 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-15T17:53:25.772 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mon-2:18.0.0-7629.g95c90f41.el9.x86_64 115/167 2024-01-15T17:53:26.351 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mon-2:18.0.0-7629.g95c90f41.el9.x86_64 115/167 2024-01-15T17:53:26.351 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:26.351 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-15T17:53:26.351 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-15T17:53:26.351 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-15T17:53:26.352 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:26.468 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : automake-1.16.2-8.el9.noarch 116/167 2024-01-15T17:53:27.040 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 117/167 2024-01-15T17:53:27.077 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 117/167 2024-01-15T17:53:27.114 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7629.g95c90f41.e 118/167 2024-01-15T17:53:27.211 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7629.g95c90f41. 119/167 2024-01-15T17:53:27.247 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7629.g95c90f41.el9.noa 120/167 2024-01-15T17:53:27.494 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7629.g95c90f41.el9.x86_64 111/167 2024-01-15T17:53:27.495 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /sys 2024-01-15T17:53:27.495 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /proc 2024-01-15T17:53:27.495 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /mnt 2024-01-15T17:53:27.495 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /var/tmp 2024-01-15T17:53:27.495 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /home 2024-01-15T17:53:27.495 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /root 2024-01-15T17:53:27.495 INFO:teuthology.orchestra.run.smithi124.stdout:skipping the directory /tmp 2024-01-15T17:53:27.495 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:27.573 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7629.g95c90f41.el9.noa 120/167 2024-01-15T17:53:27.614 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7629.g95c90f41.el9.n 121/167 2024-01-15T17:53:27.729 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-volume-2:18.0.0-7629.g95c90f41.el9.noarch 112/167 2024-01-15T17:53:27.778 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-volume-2:18.0.0-7629.g95c90f41.el9.noarch 112/167 2024-01-15T17:53:27.778 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:27.778 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-15T17:53:27.778 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:28.797 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7629.g95c90f41.el9.n 121/167 2024-01-15T17:53:28.832 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7629.g95c 122/167 2024-01-15T17:53:28.979 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7629.g95c 122/167 2024-01-15T17:53:29.164 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7629.g95c90f41.el 123/167 2024-01-15T17:53:29.192 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-2:18.0.0-7629.g95c90f41.el9.x86_64 124/167 2024-01-15T17:53:29.267 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7629.g95c90f41.el9.x86_64 124/167 2024-01-15T17:53:29.267 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:29.267 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-15T17:53:29.267 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-15T17:53:29.267 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-15T17:53:29.267 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:29.326 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-mgr-rook-2:18.0.0-7629.g95c90f41.el9.noarch 125/167 2024-01-15T17:53:29.365 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7629.g95c90f41.el9.noarch 125/167 2024-01-15T17:53:29.453 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-2:18.0.0-7629.g95c90f41.el9.x86_64 126/167 2024-01-15T17:53:29.533 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-osd-2:18.0.0-7629.g95c90f41.el9.x86_64 113/167 2024-01-15T17:53:29.611 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-osd-2:18.0.0-7629.g95c90f41.el9.x86_64 113/167 2024-01-15T17:53:29.611 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:29.611 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-15T17:53:29.611 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-15T17:53:29.611 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-15T17:53:29.612 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:29.846 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mds-2:18.0.0-7629.g95c90f41.el9.x86_64 114/167 2024-01-15T17:53:29.912 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mds-2:18.0.0-7629.g95c90f41.el9.x86_64 114/167 2024-01-15T17:53:29.912 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:29.912 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-15T17:53:29.912 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-15T17:53:29.912 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-15T17:53:29.912 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:30.377 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libtool-2.4.6-45.el9.x86_64 127/167 2024-01-15T17:53:30.397 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mon-2:18.0.0-7629.g95c90f41.el9.x86_64 115/167 2024-01-15T17:53:30.411 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-radosgw-2:18.0.0-7629.g95c90f41.el9.x86_64 128/167 2024-01-15T17:53:30.462 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mon-2:18.0.0-7629.g95c90f41.el9.x86_64 115/167 2024-01-15T17:53:30.462 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:30.462 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-15T17:53:30.462 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-15T17:53:30.463 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-15T17:53:30.463 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:30.508 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7629.g95c90f41.el9.x86_64 128/167 2024-01-15T17:53:30.508 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:30.508 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-15T17:53:30.508 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-15T17:53:30.508 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-15T17:53:30.508 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:30.561 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7629.g95c90 129/167 2024-01-15T17:53:30.574 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : automake-1.16.2-8.el9.noarch 116/167 2024-01-15T17:53:30.651 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7629.g95c90 129/167 2024-01-15T17:53:30.651 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:30.651 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-15T17:53:30.651 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:30.720 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 130/167 2024-01-15T17:53:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 130/167 2024-01-15T17:53:30.823 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:30.823 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-15T17:53:30.823 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-15T17:53:30.823 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-15T17:53:30.823 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:31.083 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 117/167 2024-01-15T17:53:31.115 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 117/167 2024-01-15T17:53:31.151 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7629.g95c90f41.e 118/167 2024-01-15T17:53:31.196 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 131/167 2024-01-15T17:53:31.235 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7629.g95c90f41. 119/167 2024-01-15T17:53:31.261 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7629.g95c90f41.el9.noa 120/167 2024-01-15T17:53:31.272 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 131/167 2024-01-15T17:53:31.272 INFO:teuthology.orchestra.run.smithi026.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:31.273 INFO:teuthology.orchestra.run.smithi026.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-15T17:53:31.273 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-15T17:53:31.273 INFO:teuthology.orchestra.run.smithi026.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-15T17:53:31.273 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:31.574 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7629.g95c90f41.el9.noa 120/167 2024-01-15T17:53:31.621 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7629.g95c90f41.el9.n 121/167 2024-01-15T17:53:32.748 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7629.g95c90f41.el9.n 121/167 2024-01-15T17:53:32.779 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7629.g95c 122/167 2024-01-15T17:53:32.920 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7629.g95c 122/167 2024-01-15T17:53:33.122 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7629.g95c90f41.el 123/167 2024-01-15T17:53:33.148 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-2:18.0.0-7629.g95c90f41.el9.x86_64 124/167 2024-01-15T17:53:33.216 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7629.g95c90f41.el9.x86_64 124/167 2024-01-15T17:53:33.216 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:33.216 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-15T17:53:33.217 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-15T17:53:33.217 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-15T17:53:33.217 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:33.273 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-mgr-rook-2:18.0.0-7629.g95c90f41.el9.noarch 125/167 2024-01-15T17:53:33.311 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7629.g95c90f41.el9.noarch 125/167 2024-01-15T17:53:33.395 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-2:18.0.0-7629.g95c90f41.el9.x86_64 126/167 2024-01-15T17:53:34.299 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libtool-2.4.6-45.el9.x86_64 127/167 2024-01-15T17:53:34.334 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-radosgw-2:18.0.0-7629.g95c90f41.el9.x86_64 128/167 2024-01-15T17:53:34.398 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7629.g95c90f41.el9.x86_64 128/167 2024-01-15T17:53:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-15T17:53:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-15T17:53:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-15T17:53:34.399 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:34.451 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7629.g95c90 129/167 2024-01-15T17:53:34.518 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7629.g95c90 129/167 2024-01-15T17:53:34.518 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:34.519 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-15T17:53:34.519 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:34.585 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : cephfs-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 130/167 2024-01-15T17:53:34.659 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 130/167 2024-01-15T17:53:34.659 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:34.659 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-15T17:53:34.659 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-15T17:53:34.659 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-15T17:53:34.660 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:35.035 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : rbd-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 131/167 2024-01-15T17:53:35.103 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 131/167 2024-01-15T17:53:35.103 INFO:teuthology.orchestra.run.smithi124.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-15T17:53:35.103 INFO:teuthology.orchestra.run.smithi124.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-15T17:53:35.104 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-15T17:53:35.104 INFO:teuthology.orchestra.run.smithi124.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-15T17:53:35.104 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:35.679 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-test-2:18.0.0-7629.g95c90f41.el9.x86_64 132/167 2024-01-15T17:53:35.735 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 133/167 2024-01-15T17:53:35.799 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : rbd-nbd-2:18.0.0-7629.g95c90f41.el9.x86_64 134/167 2024-01-15T17:53:35.835 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 135/167 2024-01-15T17:53:36.143 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 136/167 2024-01-15T17:53:36.266 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 137/167 2024-01-15T17:53:36.305 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : ceph-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 138/167 2024-01-15T17:53:36.339 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libcephfs-devel-2:18.0.0-7629.g95c90f41.el9.x86_ 139/167 2024-01-15T17:53:36.369 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 140/167 2024-01-15T17:53:36.418 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-team-1:1.45.9-1.el9.x86_64 141/167 2024-01-15T17:53:36.524 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : NetworkManager-tui-1:1.45.9-1.el9.x86_64 142/167 2024-01-15T17:53:36.592 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 143/167 2024-01-15T17:53:36.631 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 144/167 2024-01-15T17:53:37.580 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-15T17:53:37.629 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-15T17:53:37.737 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-15T17:53:37.808 INFO:teuthology.orchestra.run.smithi026.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 146/167 2024-01-15T17:53:39.252 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-test-2:18.0.0-7629.g95c90f41.el9.x86_64 132/167 2024-01-15T17:53:39.266 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : cephfs-top-2:18.0.0-7629.g95c90f41.el9.noarch 147/167 2024-01-15T17:53:39.310 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : rbd-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 133/167 2024-01-15T17:53:39.343 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 148/167 2024-01-15T17:53:39.373 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : rbd-nbd-2:18.0.0-7629.g95c90f41.el9.x86_64 134/167 2024-01-15T17:53:39.392 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 149/167 2024-01-15T17:53:39.407 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 135/167 2024-01-15T17:53:39.681 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 136/167 2024-01-15T17:53:39.805 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 137/167 2024-01-15T17:53:39.835 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : ceph-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 138/167 2024-01-15T17:53:39.868 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libcephfs-devel-2:18.0.0-7629.g95c90f41.el9.x86_ 139/167 2024-01-15T17:53:39.908 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 140/167 2024-01-15T17:53:39.944 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : NetworkManager-team-1:1.45.9-1.el9.x86_64 141/167 2024-01-15T17:53:39.946 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : flex-2.6.4-9.el9.x86_64 150/167 2024-01-15T17:53:39.988 INFO:teuthology.orchestra.run.smithi026.stdout: Installing : bison-3.7.4-5.el9.x86_64 151/167 2024-01-15T17:53:40.021 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 152/167 2024-01-15T17:53:40.056 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/167 2024-01-15T17:53:40.067 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : NetworkManager-tui-1:1.45.9-1.el9.x86_64 142/167 2024-01-15T17:53:40.098 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 154/167 2024-01-15T17:53:40.099 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 155/167 2024-01-15T17:53:40.131 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 143/167 2024-01-15T17:53:40.203 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 144/167 2024-01-15T17:53:40.221 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 155/167 2024-01-15T17:53:40.299 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 156/167 2024-01-15T17:53:40.343 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 157/167 2024-01-15T17:53:40.385 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 158/167 2024-01-15T17:53:40.417 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 159/167 2024-01-15T17:53:40.417 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/167 2024-01-15T17:53:40.538 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/167 2024-01-15T17:53:40.569 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 161/167 2024-01-15T17:53:40.606 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-15T17:53:40.607 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-15T17:53:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-15T17:53:40.952 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 163/167 2024-01-15T17:53:41.078 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 163/167 2024-01-15T17:53:41.113 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 164/167 2024-01-15T17:53:41.141 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-15T17:53:41.153 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 165/167 2024-01-15T17:53:41.193 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-15T17:53:41.206 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 166/167 2024-01-15T17:53:41.289 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 145/167 2024-01-15T17:53:41.330 INFO:teuthology.orchestra.run.smithi026.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 167/167 2024-01-15T17:53:41.363 INFO:teuthology.orchestra.run.smithi124.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 146/167 2024-01-15T17:53:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 167/167 2024-01-15T17:53:41.631 INFO:teuthology.orchestra.run.smithi026.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-01-15T17:53:41.631 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:42.714 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : cephfs-top-2:18.0.0-7629.g95c90f41.el9.noarch 147/167 2024-01-15T17:53:43.247 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 148/167 2024-01-15T17:53:43.337 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 149/167 2024-01-15T17:53:43.503 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : flex-2.6.4-9.el9.x86_64 150/167 2024-01-15T17:53:43.542 INFO:teuthology.orchestra.run.smithi124.stdout: Installing : bison-3.7.4-5.el9.x86_64 151/167 2024-01-15T17:53:43.567 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 152/167 2024-01-15T17:53:43.593 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/167 2024-01-15T17:53:43.635 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 154/167 2024-01-15T17:53:43.635 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 155/167 2024-01-15T17:53:43.767 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 155/167 2024-01-15T17:53:43.817 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 156/167 2024-01-15T17:53:43.864 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 157/167 2024-01-15T17:53:43.906 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 158/167 2024-01-15T17:53:43.938 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 159/167 2024-01-15T17:53:43.939 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/167 2024-01-15T17:53:44.067 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/167 2024-01-15T17:53:44.089 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 161/167 2024-01-15T17:53:44.126 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-15T17:53:44.126 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-15T17:53:44.456 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 162/167 2024-01-15T17:53:44.456 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 163/167 2024-01-15T17:53:44.598 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 163/167 2024-01-15T17:53:44.634 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 164/167 2024-01-15T17:53:44.673 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 165/167 2024-01-15T17:53:44.719 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 166/167 2024-01-15T17:53:44.837 INFO:teuthology.orchestra.run.smithi124.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 167/167 2024-01-15T17:53:45.124 INFO:teuthology.orchestra.run.smithi124.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 167/167 2024-01-15T17:53:45.125 INFO:teuthology.orchestra.run.smithi124.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-01-15T17:53:45.125 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:46.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-2:18.0.0-7629.g95c90f41.el9.x86_64 1/167 2024-01-15T17:53:46.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-base-2:18.0.0-7629.g95c90f41.el9.x86_64 2/167 2024-01-15T17:53:46.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 3/167 2024-01-15T17:53:46.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 4/167 2024-01-15T17:53:46.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7629.g95c90 5/167 2024-01-15T17:53:46.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mds-2:18.0.0-7629.g95c90f41.el9.x86_64 6/167 2024-01-15T17:53:46.130 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-2:18.0.0-7629.g95c90f41.el9.x86_64 7/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mon-2:18.0.0-7629.g95c90f41.el9.x86_64 8/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-osd-2:18.0.0-7629.g95c90f41.el9.x86_64 9/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-radosgw-2:18.0.0-7629.g95c90f41.el9.x86_64 10/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-selinux-2:18.0.0-7629.g95c90f41.el9.x86_64 11/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-test-2:18.0.0-7629.g95c90f41.el9.x86_64 12/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 13/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs-devel-2:18.0.0-7629.g95c90f41.el9.x86_ 14/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephfs2-2:18.0.0-7629.g95c90f41.el9.x86_64 15/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libcephsqlite-2:18.0.0-7629.g95c90f41.el9.x86_64 16/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados-devel-2:18.0.0-7629.g95c90f41.el9.x86_6 17/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libradosstriper1-2:18.0.0-7629.g95c90f41.el9.x86 18/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librgw2-2:18.0.0-7629.g95c90f41.el9.x86_64 19/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7629.g95c90f41.el 20/167 2024-01-15T17:53:46.131 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ceph-common-2:18.0.0-7629.g95c90f41.el9. 21/167 2024-01-15T17:53:46.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cephfs-2:18.0.0-7629.g95c90f41.el9.x86_6 22/167 2024-01-15T17:53:46.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rados-2:18.0.0-7629.g95c90f41.el9.x86_64 23/167 2024-01-15T17:53:46.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rbd-2:18.0.0-7629.g95c90f41.el9.x86_64 24/167 2024-01-15T17:53:46.132 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rgw-2:18.0.0-7629.g95c90f41.el9.x86_64 25/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 26/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 27/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : rbd-nbd-2:18.0.0-7629.g95c90f41.el9.x86_64 28/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7629.g95c90f41. 29/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7629.g95c90f41.el9.noa 30/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7629.g95c90f41.el9.n 31/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7629.g95c 32/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7629.g95c90f41.el 33/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7629.g95c90f41.el9.noarch 34/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7629.g95c90f41.e 35/167 2024-01-15T17:53:46.145 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : ceph-volume-2:18.0.0-7629.g95c90f41.el9.noarch 36/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 37/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : cephfs-top-2:18.0.0-7629.g95c90f41.el9.noarch 38/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 42/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/167 2024-01-15T17:53:46.146 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 52/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 53/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 54/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-5.4.4-4.el9.x86_64 55/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 58/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/167 2024-01-15T17:53:46.147 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/167 2024-01-15T17:53:46.148 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 81/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : jq-1.6-16.el9.x86_64 85/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 86/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 87/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 88/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 89/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 90/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-ply-3.11-14.el9.noarch 91/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 92/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 93/167 2024-01-15T17:53:46.149 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 96/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 102/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 103/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 104/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 105/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 106/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 107/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 108/167 2024-01-15T17:53:46.150 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 109/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-google-auth-1:2.26.0-1.el9.noarch 110/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 111/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 112/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 113/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 114/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 115/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 116/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 117/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 118/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 119/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 120/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 121/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 122/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 123/167 2024-01-15T17:53:46.151 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 125/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 126/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 127/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 128/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 129/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 130/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 131/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 132/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 133/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : re2-1:20211101-3.el9.x86_64 134/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 135/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:18.0.0-7629.g95c90f41.el9.x86_64 136/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 137/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:18.0.0-7629.g95c90f41.el9.x86_64 138/167 2024-01-15T17:53:46.152 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 139/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 140/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 141/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.45.9-1.el9.x86_64 142/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 143/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.45.9-1.el9.x86_64 144/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 145/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.45.9-1.el9.x86_64 146/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.45.9-1.el9.x86_64 148/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 149/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : acl-2.3.1-4.el9.x86_64 150/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : acl-2.3.1-3.el9.x86_64 151/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 152/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/167 2024-01-15T17:53:46.153 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 154/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 155/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 156/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 157/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 158/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 159/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 160/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 161/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 162/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 163/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 164/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 165/167 2024-01-15T17:53:46.154 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 166/167 2024-01-15T17:53:48.295 INFO:teuthology.orchestra.run.smithi026.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 167/167 2024-01-15T17:53:48.295 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:48.295 INFO:teuthology.orchestra.run.smithi026.stdout:Upgraded: 2024-01-15T17:53:48.295 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-1:1.45.9-1.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-libnm-1:1.45.9-1.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-team-1:1.45.9-1.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: NetworkManager-tui-1:1.45.9-1.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: acl-2.3.1-4.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: librados2-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: librbd1-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.296 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout:Installed: 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: automake-1.16.2-8.el9.noarch 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: bison-3.7.4-5.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-base-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-grafana-dashboards-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-immutable-object-cache-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.297 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mds-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-dashboard-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-modules-core-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mgr-rook-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-mon-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-osd-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-prometheus-alerts-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-radosgw-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-selinux-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.298 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-test-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: ceph-volume-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: cephfs-top-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: flex-2.6.4-9.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: jq-1.6-16.el9.x86_64 2024-01-15T17:53:48.299 INFO:teuthology.orchestra.run.smithi026.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs-devel-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: libcephfs2-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: libcephsqlite-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: librados-devel-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: libradosstriper1-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.300 INFO:teuthology.orchestra.run.smithi026.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: librgw2-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: lua-5.4.4-4.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-15T17:53:48.301 INFO:teuthology.orchestra.run.smithi026.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-13.11-1.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-argparse-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cephfs-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.302 INFO:teuthology.orchestra.run.smithi026.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-google-auth-1:2.26.0-1.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-15T17:53:48.303 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-15T17:53:48.304 INFO:teuthology.orchestra.run.smithi026.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-ply-3.11-14.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rados-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rbd-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-15T17:53:48.305 INFO:teuthology.orchestra.run.smithi026.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rgw-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.306 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.307 INFO:teuthology.orchestra.run.smithi026.stdout: rbd-nbd-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:48.307 INFO:teuthology.orchestra.run.smithi026.stdout: re2-1:20211101-3.el9.x86_64 2024-01-15T17:53:48.307 INFO:teuthology.orchestra.run.smithi026.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-15T17:53:48.307 INFO:teuthology.orchestra.run.smithi026.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-15T17:53:48.307 INFO:teuthology.orchestra.run.smithi026.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-15T17:53:48.307 INFO:teuthology.orchestra.run.smithi026.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-15T17:53:48.307 INFO:teuthology.orchestra.run.smithi026.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-15T17:53:48.307 INFO:teuthology.orchestra.run.smithi026.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-15T17:53:48.307 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:48.308 INFO:teuthology.orchestra.run.smithi026.stdout:Complete! 2024-01-15T17:53:48.905 DEBUG:teuthology.parallel:result is None 2024-01-15T17:53:49.595 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-2:18.0.0-7629.g95c90f41.el9.x86_64 1/167 2024-01-15T17:53:49.595 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-base-2:18.0.0-7629.g95c90f41.el9.x86_64 2/167 2024-01-15T17:53:49.595 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 3/167 2024-01-15T17:53:49.595 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 4/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7629.g95c90 5/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mds-2:18.0.0-7629.g95c90f41.el9.x86_64 6/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-2:18.0.0-7629.g95c90f41.el9.x86_64 7/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mon-2:18.0.0-7629.g95c90f41.el9.x86_64 8/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-osd-2:18.0.0-7629.g95c90f41.el9.x86_64 9/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-radosgw-2:18.0.0-7629.g95c90f41.el9.x86_64 10/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-selinux-2:18.0.0-7629.g95c90f41.el9.x86_64 11/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-test-2:18.0.0-7629.g95c90f41.el9.x86_64 12/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : cephfs-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 13/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcephfs-devel-2:18.0.0-7629.g95c90f41.el9.x86_ 14/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcephfs2-2:18.0.0-7629.g95c90f41.el9.x86_64 15/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libcephsqlite-2:18.0.0-7629.g95c90f41.el9.x86_64 16/167 2024-01-15T17:53:49.596 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librados-devel-2:18.0.0-7629.g95c90f41.el9.x86_6 17/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libradosstriper1-2:18.0.0-7629.g95c90f41.el9.x86 18/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librgw2-2:18.0.0-7629.g95c90f41.el9.x86_64 19/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7629.g95c90f41.el 20/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-ceph-common-2:18.0.0-7629.g95c90f41.el9. 21/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cephfs-2:18.0.0-7629.g95c90f41.el9.x86_6 22/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rados-2:18.0.0-7629.g95c90f41.el9.x86_64 23/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rbd-2:18.0.0-7629.g95c90f41.el9.x86_64 24/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rgw-2:18.0.0-7629.g95c90f41.el9.x86_64 25/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : rbd-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 26/167 2024-01-15T17:53:49.597 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : rbd-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 27/167 2024-01-15T17:53:49.610 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : rbd-nbd-2:18.0.0-7629.g95c90f41.el9.x86_64 28/167 2024-01-15T17:53:49.610 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7629.g95c90f41. 29/167 2024-01-15T17:53:49.610 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7629.g95c90f41.el9.noa 30/167 2024-01-15T17:53:49.610 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7629.g95c90f41.el9.n 31/167 2024-01-15T17:53:49.610 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7629.g95c 32/167 2024-01-15T17:53:49.610 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7629.g95c90f41.el 33/167 2024-01-15T17:53:49.610 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7629.g95c90f41.el9.noarch 34/167 2024-01-15T17:53:49.610 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7629.g95c90f41.e 35/167 2024-01-15T17:53:49.610 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : ceph-volume-2:18.0.0-7629.g95c90f41.el9.noarch 36/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 37/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : cephfs-top-2:18.0.0-7629.g95c90f41.el9.noarch 38/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 42/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/167 2024-01-15T17:53:49.611 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 52/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 53/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 54/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : lua-5.4.4-4.el9.x86_64 55/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 58/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/167 2024-01-15T17:53:49.612 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/167 2024-01-15T17:53:49.613 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 81/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : jq-1.6-16.el9.x86_64 85/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 86/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 87/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 88/167 2024-01-15T17:53:49.614 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 89/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 90/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-ply-3.11-14.el9.noarch 91/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 92/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 93/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 95/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 96/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 97/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 98/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 99/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 100/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 101/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 102/167 2024-01-15T17:53:49.615 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 103/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 104/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 105/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 106/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 107/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 108/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 109/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-google-auth-1:2.26.0-1.el9.noarch 110/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 111/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 112/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 113/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 114/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 115/167 2024-01-15T17:53:49.616 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 116/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 117/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 118/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 119/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 120/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 121/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 122/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 123/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 124/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 125/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 126/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 127/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 128/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 129/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 130/167 2024-01-15T17:53:49.617 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 131/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 132/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 133/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : re2-1:20211101-3.el9.x86_64 134/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 135/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librados2-2:18.0.0-7629.g95c90f41.el9.x86_64 136/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 137/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librbd1-2:18.0.0-7629.g95c90f41.el9.x86_64 138/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 139/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 140/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 141/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-1:1.45.9-1.el9.x86_64 142/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 143/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-libnm-1:1.45.9-1.el9.x86_64 144/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 145/167 2024-01-15T17:53:49.618 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-team-1:1.45.9-1.el9.x86_64 146/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-tui-1:1.45.9-1.el9.x86_64 148/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 149/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : acl-2.3.1-4.el9.x86_64 150/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : acl-2.3.1-3.el9.x86_64 151/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 152/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 153/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 154/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 155/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 156/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 157/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 158/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 159/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 160/167 2024-01-15T17:53:49.619 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 161/167 2024-01-15T17:53:49.620 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 162/167 2024-01-15T17:53:49.620 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 163/167 2024-01-15T17:53:49.620 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 164/167 2024-01-15T17:53:49.620 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 165/167 2024-01-15T17:53:49.620 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 166/167 2024-01-15T17:53:51.762 INFO:teuthology.orchestra.run.smithi124.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 167/167 2024-01-15T17:53:51.762 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:51.762 INFO:teuthology.orchestra.run.smithi124.stdout:Upgraded: 2024-01-15T17:53:51.762 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-1:1.45.9-1.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-libnm-1:1.45.9-1.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-team-1:1.45.9-1.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: NetworkManager-tui-1:1.45.9-1.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: acl-2.3.1-4.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: librados2-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: librbd1-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.763 INFO:teuthology.orchestra.run.smithi124.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout:Installed: 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: automake-1.16.2-8.el9.noarch 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: bison-3.7.4-5.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-base-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.764 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-grafana-dashboards-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-immutable-object-cache-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mds-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-dashboard-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-modules-core-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mgr-rook-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-mon-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-osd-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-prometheus-alerts-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-radosgw-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.765 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-selinux-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-test-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: ceph-volume-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: cephadm-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: cephfs-top-2:18.0.0-7629.g95c90f41.el9.noarch 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: flex-2.6.4-9.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-15T17:53:51.766 INFO:teuthology.orchestra.run.smithi124.stdout: jq-1.6-16.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs-devel-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libcephfs2-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libcephsqlite-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: librados-devel-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: libradosstriper1-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.767 INFO:teuthology.orchestra.run.smithi124.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: librgw2-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: lua-5.4.4-4.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-15T17:53:51.768 INFO:teuthology.orchestra.run.smithi124.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-13.11-1.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-argparse-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ceph-common-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cephfs-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-15T17:53:51.769 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-google-auth-1:2.26.0-1.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-15T17:53:51.770 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-ply-3.11-14.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-15T17:53:51.771 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rados-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rbd-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rgw-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-15T17:53:51.772 INFO:teuthology.orchestra.run.smithi124.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-fuse-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-mirror-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: rbd-nbd-2:18.0.0-7629.g95c90f41.el9.x86_64 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: re2-1:20211101-3.el9.x86_64 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-15T17:53:51.773 INFO:teuthology.orchestra.run.smithi124.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-15T17:53:51.774 INFO:teuthology.orchestra.run.smithi124.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-15T17:53:51.774 INFO:teuthology.orchestra.run.smithi124.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-15T17:53:51.774 INFO:teuthology.orchestra.run.smithi124.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-15T17:53:51.774 INFO:teuthology.orchestra.run.smithi124.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-15T17:53:51.774 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:51.774 INFO:teuthology.orchestra.run.smithi124.stdout:Complete! 2024-01-15T17:53:52.392 DEBUG:teuthology.parallel:result is None 2024-01-15T17:53:52.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T17:53:52.569 DEBUG:teuthology.orchestra.run.smithi026:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-15T17:53:52.627 INFO:teuthology.orchestra.run.smithi026.stdout:18.0.0-7629.g95c90f41.el9 2024-01-15T17:53:52.628 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7629.g95c90f41.el9 2024-01-15T17:53:52.628 INFO:teuthology.task.install:The correct ceph version 18.0.0-7629.g95c90f41 is installed. 2024-01-15T17:53:52.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95c90f413c4bb65e3e56909855f2308fd9bda05b 2024-01-15T17:53:52.975 DEBUG:teuthology.orchestra.run.smithi124:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-15T17:53:53.028 INFO:teuthology.orchestra.run.smithi124.stdout:18.0.0-7629.g95c90f41.el9 2024-01-15T17:53:53.028 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7629.g95c90f41.el9 2024-01-15T17:53:53.029 INFO:teuthology.task.install:The correct ceph version 18.0.0-7629.g95c90f41 is installed. 2024-01-15T17:53:53.030 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-15T17:53:53.030 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:53.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-15T17:53:53.119 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:53:53.120 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-15T17:53:53.202 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-15T17:53:53.203 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:53.203 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/daemon-helper 2024-01-15T17:53:53.277 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-15T17:53:53.394 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:53:53.394 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/daemon-helper 2024-01-15T17:53:53.489 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-15T17:53:53.623 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-15T17:53:53.623 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:53.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-15T17:53:53.698 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-15T17:53:53.814 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:53:53.814 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-15T17:53:53.887 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-15T17:53:54.002 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-15T17:53:54.002 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:54.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/usr/bin/stdin-killer 2024-01-15T17:53:54.102 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-15T17:53:54.215 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:53:54.215 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/usr/bin/stdin-killer 2024-01-15T17:53:54.289 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-15T17:53:54.400 INFO:teuthology.run_tasks:Running task ceph... 2024-01-15T17:53:54.481 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-15T17:53:54.481 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 777 /var/log/ceph 2024-01-15T17:53:54.486 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 777 /var/log/ceph 2024-01-15T17:53:54.566 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-15T17:53:54.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-15T17:53:54.571 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-15T17:53:54.644 INFO:tasks.ceph:Creating extra log directories... 2024-01-15T17:53:54.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-15T17:53:54.649 DEBUG:teuthology.orchestra.run.smithi124:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-15T17:53:54.730 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-15T17:53:54.730 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-15T17:53:54.730 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-15T17:53:54.730 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:54.730 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-15T17:53:54.767 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-15T17:53:54.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-15T17:53:55.052 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:53:55.053 DEBUG:teuthology.orchestra.run.smithi124:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-15T17:53:55.100 DEBUG:teuthology.orchestra.run.smithi124:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-15T17:53:55.300 DEBUG:teuthology.orchestra.run.smithi124:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-15T17:53:55.379 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-15T17:53:55.379 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-15T17:53:55.379 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100', 'branch': 'rishabh-clone-stats', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}}', 'email': None, 'first_in_suite': False, 'job_id': '7517100', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-clone-stats'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-clone-stats', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4427, 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-clone-stats', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi026.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKC4M7/gUHS0qgi/R2ONHIyBfj0g7m7E7n2NR4YtUuMPa7fYJ+XHdPpjCv5mpg7wZvsKv3pKjLthZdgU6MmnXW4=', 'smithi124.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMnT11QoYwAOugPZL1E5SnA/BOcm9WyIhGutN3SBvKT0jssJcwBm3ezEJRkCO7YICWbWBHF3rH5E87M7LpTyhsQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '2668949e0971849893a3fb7f177ce9032fc65780'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '95c90f413c4bb65e3e56909855f2308fd9bda05b', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestCloneProgress']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'cd45576300487d997e5a85abed65500b9f5d143b', 'timestamp': '2024-01-15_17:35:13', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-15T17:53:55.380 DEBUG:teuthology.orchestra.run.smithi026:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-15T17:53:55.388 DEBUG:teuthology.orchestra.run.smithi124:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-15T17:53:55.440 DEBUG:teuthology.orchestra.run.smithi026:> sudo install -d -m0777 -- /var/run/ceph 2024-01-15T17:53:55.444 DEBUG:teuthology.orchestra.run.smithi124:> sudo install -d -m0777 -- /var/run/ceph 2024-01-15T17:53:55.539 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:55.539 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/scratch_devs of=/dev/stdout 2024-01-15T17:53:55.597 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-15T17:53:55.598 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_1 2024-01-15T17:53:55.657 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-15T17:53:55.657 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T17:53:55.657 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-01-15T17:53:55.657 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T17:53:55.657 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-01-15T17:53:55.657 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-01-15 17:53:42.076714134 +0000 2024-01-15T17:53:55.657 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-01-15 17:51:15.828643881 +0000 2024-01-15T17:53:55.657 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-01-15 17:51:15.828643881 +0000 2024-01-15T17:53:55.657 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-01-15 17:51:15.828643881 +0000 2024-01-15T17:53:55.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-15T17:53:55.742 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-01-15T17:53:55.742 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-01-15T17:53:55.742 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000295003 s, 1.7 MB/s 2024-01-15T17:53:55.745 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-15T17:53:55.799 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_2 2024-01-15T17:53:55.850 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-15T17:53:55.850 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T17:53:55.850 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-01-15T17:53:55.850 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T17:53:55.850 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-01-15T17:53:55.850 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-01-15 17:53:42.077714118 +0000 2024-01-15T17:53:55.851 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-01-15 17:51:16.380635502 +0000 2024-01-15T17:53:55.851 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-01-15 17:51:16.380635502 +0000 2024-01-15T17:53:55.851 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-01-15 17:51:16.380635502 +0000 2024-01-15T17:53:55.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-15T17:53:55.934 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-01-15T17:53:55.934 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-01-15T17:53:55.934 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000238374 s, 2.1 MB/s 2024-01-15T17:53:55.938 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-15T17:53:55.998 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_3 2024-01-15T17:53:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-15T17:53:56.044 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T17:53:56.044 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-01-15T17:53:56.044 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T17:53:56.044 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-01-15T17:53:56.044 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-01-15 17:53:42.077714118 +0000 2024-01-15T17:53:56.045 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-01-15 17:51:16.948626879 +0000 2024-01-15T17:53:56.045 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-01-15 17:51:16.949626864 +0000 2024-01-15T17:53:56.045 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-01-15 17:51:16.948626879 +0000 2024-01-15T17:53:56.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-15T17:53:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-01-15T17:53:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-01-15T17:53:56.182 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000222897 s, 2.3 MB/s 2024-01-15T17:53:56.187 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-15T17:53:56.245 DEBUG:teuthology.orchestra.run.smithi026:> stat /dev/vg_nvme/lv_4 2024-01-15T17:53:56.292 INFO:teuthology.orchestra.run.smithi026.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-15T17:53:56.292 INFO:teuthology.orchestra.run.smithi026.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T17:53:56.292 INFO:teuthology.orchestra.run.smithi026.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-01-15T17:53:56.292 INFO:teuthology.orchestra.run.smithi026.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T17:53:56.292 INFO:teuthology.orchestra.run.smithi026.stdout:Context: system_u:object_r:device_t:s0 2024-01-15T17:53:56.292 INFO:teuthology.orchestra.run.smithi026.stdout:Access: 2024-01-15 17:53:42.078714103 +0000 2024-01-15T17:53:56.292 INFO:teuthology.orchestra.run.smithi026.stdout:Modify: 2024-01-15 17:51:17.544617832 +0000 2024-01-15T17:53:56.292 INFO:teuthology.orchestra.run.smithi026.stdout:Change: 2024-01-15 17:51:17.544617832 +0000 2024-01-15T17:53:56.292 INFO:teuthology.orchestra.run.smithi026.stdout: Birth: 2024-01-15 17:51:17.544617832 +0000 2024-01-15T17:53:56.292 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-15T17:53:56.379 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records in 2024-01-15T17:53:56.379 INFO:teuthology.orchestra.run.smithi026.stderr:1+0 records out 2024-01-15T17:53:56.379 INFO:teuthology.orchestra.run.smithi026.stderr:512 bytes copied, 0.000213068 s, 2.4 MB/s 2024-01-15T17:53:56.384 DEBUG:teuthology.orchestra.run.smithi026:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-15T17:53:56.441 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T17:53:56.441 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:53:56.441 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/scratch_devs of=/dev/stdout 2024-01-15T17:53:56.486 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-15T17:53:56.486 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_1 2024-01-15T17:53:56.569 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-15T17:53:56.569 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T17:53:56.569 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-01-15T17:53:56.569 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T17:53:56.569 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2024-01-15T17:53:56.569 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2024-01-15 17:53:45.530151470 +0000 2024-01-15T17:53:56.569 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2024-01-15 17:51:06.574949034 +0000 2024-01-15T17:53:56.569 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2024-01-15 17:51:06.574949034 +0000 2024-01-15T17:53:56.569 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: 2024-01-15 17:51:06.574949034 +0000 2024-01-15T17:53:56.569 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-15T17:53:56.680 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2024-01-15T17:53:56.680 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2024-01-15T17:53:56.680 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000364558 s, 1.4 MB/s 2024-01-15T17:53:56.683 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-15T17:53:56.724 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_2 2024-01-15T17:53:56.803 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-15T17:53:56.804 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T17:53:56.804 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-01-15T17:53:56.804 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T17:53:56.804 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2024-01-15T17:53:56.804 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2024-01-15 17:53:45.530151470 +0000 2024-01-15T17:53:56.804 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2024-01-15 17:51:07.170938384 +0000 2024-01-15T17:53:56.804 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2024-01-15 17:51:07.170938384 +0000 2024-01-15T17:53:56.804 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: 2024-01-15 17:51:07.170938384 +0000 2024-01-15T17:53:56.804 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-15T17:53:56.931 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2024-01-15T17:53:56.931 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2024-01-15T17:53:56.931 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000361287 s, 1.4 MB/s 2024-01-15T17:53:56.935 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-15T17:53:56.991 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_3 2024-01-15T17:53:57.038 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-15T17:53:57.038 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T17:53:57.038 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-01-15T17:53:57.038 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T17:53:57.038 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2024-01-15T17:53:57.039 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2024-01-15 17:53:45.530151470 +0000 2024-01-15T17:53:57.039 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2024-01-15 17:51:07.748928056 +0000 2024-01-15T17:53:57.039 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2024-01-15 17:51:07.748928056 +0000 2024-01-15T17:53:57.039 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: 2024-01-15 17:51:07.748928056 +0000 2024-01-15T17:53:57.039 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-15T17:53:57.117 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2024-01-15T17:53:57.117 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2024-01-15T17:53:57.117 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000349352 s, 1.5 MB/s 2024-01-15T17:53:57.122 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-15T17:53:57.174 DEBUG:teuthology.orchestra.run.smithi124:> stat /dev/vg_nvme/lv_4 2024-01-15T17:53:57.216 INFO:teuthology.orchestra.run.smithi124.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-15T17:53:57.217 INFO:teuthology.orchestra.run.smithi124.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-15T17:53:57.217 INFO:teuthology.orchestra.run.smithi124.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-01-15T17:53:57.217 INFO:teuthology.orchestra.run.smithi124.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-15T17:53:57.217 INFO:teuthology.orchestra.run.smithi124.stdout:Context: system_u:object_r:device_t:s0 2024-01-15T17:53:57.217 INFO:teuthology.orchestra.run.smithi124.stdout:Access: 2024-01-15 17:53:45.531151452 +0000 2024-01-15T17:53:57.217 INFO:teuthology.orchestra.run.smithi124.stdout:Modify: 2024-01-15 17:51:08.326917728 +0000 2024-01-15T17:53:57.217 INFO:teuthology.orchestra.run.smithi124.stdout:Change: 2024-01-15 17:51:08.326917728 +0000 2024-01-15T17:53:57.217 INFO:teuthology.orchestra.run.smithi124.stdout: Birth: 2024-01-15 17:51:08.326917728 +0000 2024-01-15T17:53:57.217 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-15T17:53:57.348 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records in 2024-01-15T17:53:57.355 INFO:teuthology.orchestra.run.smithi124.stderr:1+0 records out 2024-01-15T17:53:57.355 INFO:teuthology.orchestra.run.smithi124.stderr:512 bytes copied, 0.000310967 s, 1.6 MB/s 2024-01-15T17:53:57.356 DEBUG:teuthology.orchestra.run.smithi124:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-15T17:53:57.409 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T17:53:57.409 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi124.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-15T17:53:57.409 INFO:tasks.ceph:Generating config... 2024-01-15T17:53:57.410 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-15T17:53:57.410 INFO:tasks.ceph:[client] debug client = 20 2024-01-15T17:53:57.410 INFO:tasks.ceph:[client] debug ms = 1 2024-01-15T17:53:57.410 INFO:tasks.ceph:[client] pid file = 2024-01-15T17:53:57.411 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-15T17:53:57.411 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mgr] debug client = 20 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-15T17:53:57.411 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-15T17:53:57.412 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-15T17:53:57.412 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-15T17:53:57.412 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-15T17:53:57.412 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] bdev async discard = True 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-15T17:53:57.412 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-01-15T17:53:57.413 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-15T17:53:57.413 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-15T17:53:57.413 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-15T17:53:57.413 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-15T17:53:57.413 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-15T17:53:57.413 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-15T17:53:57.415 INFO:tasks.ceph:Setting up mon.a... 2024-01-15T17:53:57.415 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-15T17:53:57.561 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.keyring 2024-01-15T17:53:57.570 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-15T17:53:57.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-15T17:53:57.823 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.26'), ('mon.b', '172.21.15.124'), ('mon.c', '[v2:172.21.15.124:3301,v1:172.21.15.124:6790]')] 2024-01-15T17:53:57.823 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.26,172.21.15.124,[v2:172.21.15.124:3301,v1:172.21.15.124:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-15T17:53:57.824 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:57.824 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-15T17:53:57.882 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.26 --add b 172.21.15.124 --addv c '[v2:172.21.15.124:3301,v1:172.21.15.124:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:53:58.039 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:53:58.039 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: generated fsid 64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1 2024-01-15T17:53:58.039 INFO:teuthology.orchestra.run.smithi026.stdout:setting min_mon_release = pacific 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:epoch 0 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:fsid 64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:last_changed 2024-01-15T17:53:58.037424+0000 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:created 2024-01-15T17:53:58.037424+0000 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:min_mon_release 16 (pacific) 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:election_strategy: 1 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:0: [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] mon.a 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:1: [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] mon.b 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:2: [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] mon.c 2024-01-15T17:53:58.040 INFO:teuthology.orchestra.run.smithi026.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-15T17:53:58.043 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-15T17:53:58.094 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1... 2024-01-15T17:53:58.095 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-15T17:53:58.100 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-15T17:53:58.256 INFO:teuthology.orchestra.run.smithi124.stdout:[global] 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: chdir = "" 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: auth supported = cephx 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: filestore xattr use omap = true 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: mon clock drift allowed = 1.000 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: osd crush chooseleaf type = 0 2024-01-15T17:53:58.257 INFO:teuthology.orchestra.run.smithi124.stdout: auth debug = true 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on old message = true 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: ms die on bug = true 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: mon pg warn max object skew = 0 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: # disable pg_autoscaler by default for new pools 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default size = 2 2024-01-15T17:53:58.258 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow primary affinity = true 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd allow pg remap = true 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on legacy crush tunables = false 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on crush straw calc version zero = false 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on no sortbitwise = false 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on osd down out interval zero = false 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon warn on too few osds = false 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: mon_allow_pool_size_one = true 2024-01-15T17:53:58.259 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: osd default data pool replay window = 5 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: mon allow pool delete = true 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: mon cluster log file level = debug 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: debug asserts on shutdown = true 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: mon health detail to clog = false 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: mon host = "172.21.15.26,172.21.15.124,[v2:172.21.15.124:3301,v1:172.21.15.124:6790]" 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: fsid = 64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1 2024-01-15T17:53:58.260 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout:[osd] 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: osd journal size = 100 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub load threshold = 5.0 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: osd scrub max interval = 600 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: osd mclock profile = high_recovery_ops 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: osd recover clone overlap = true 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: osd recovery max chunk = 1048576 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug shutdown = true 2024-01-15T17:53:58.261 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug op order = true 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug verify stray on activate = true 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug trim objects = true 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: osd open classes on start = true 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug pg log writeout = true 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: osd deep scrub update digest min age = 30 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: osd map max advance = 10 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: journal zero on create = true 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: filestore ondisk finisher threads = 3 2024-01-15T17:53:58.262 INFO:teuthology.orchestra.run.smithi124.stdout: filestore apply finisher threads = 3 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: bdev debug aio = true 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: osd debug misdirected ops = true 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: bdev async discard = True 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: bdev enable discard = True 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: bluestore allocator = bitmap 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: bluestore block size = 96636764160 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: bluestore fsck on mount = True 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: debug bluefs = 1/20 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: debug bluestore = 1/20 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: debug osd = 20 2024-01-15T17:53:58.263 INFO:teuthology.orchestra.run.smithi124.stdout: debug rocksdb = 4/10 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd backfillfull_ratio = 0.85 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd full ratio = 0.9 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd nearfull ratio = 0.8 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: osd failsafe full ratio = 0.95 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: osd objectstore = bluestore 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: osd op complaint time = 180 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr] 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: debug mgr = 20 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2024-01-15T17:53:58.264 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: mgr/telemetry/nag = false 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: debug client = 20 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: debug finisher = 20 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout:[mon] 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: debug mon = 20 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: debug paxos = 20 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: debug auth = 20 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: mon data avail warn = 5 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: mon mgr mkfs grace = 240 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min pgs per osd = 4 2024-01-15T17:53:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd reporter subtree level = osd 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: mon osd prime pg temp = true 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: mon reweight min bytes per osd = 10 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: auth mon ticket ttl = 660 # 11m 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: auth service ticket ttl = 240 # 4m 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: # don't complain about insecure global_id in the test suite 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: # 1m isn't quite enough 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: mon_down_mkfs_grace = 2m 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: mon_warn_on_filestore_osds = false 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: mon op complaint time = 120 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout:[client] 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: rgw cache enabled = true 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable ops log = true 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: rgw enable usage log = true 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: client mount timeout = 600 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: debug client = 20 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: pid file = "" 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: rados mon op timeout = 900 2024-01-15T17:53:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: rados osd op timeout = 900 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.a] 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.b] 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout:[mon.c] 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.a] 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.c] 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.b] 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.d] 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout:[mds] 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: debug mds = 20 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: debug mds balancer = 20 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: debug ms = 1 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: mds debug frag = True 2024-01-15T17:53:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: mds debug scatterstat = True 2024-01-15T17:53:58.269 INFO:teuthology.orchestra.run.smithi124.stdout: mds op complaint time = 180 2024-01-15T17:53:58.269 INFO:teuthology.orchestra.run.smithi124.stdout: mds verify scatter = True 2024-01-15T17:53:58.269 INFO:teuthology.orchestra.run.smithi124.stdout: osd op complaint time = 180 2024-01-15T17:53:58.269 INFO:teuthology.orchestra.run.smithi124.stdout: rados mon op timeout = 900 2024-01-15T17:53:58.269 INFO:teuthology.orchestra.run.smithi124.stdout: rados osd op timeout = 900 2024-01-15T17:53:58.275 INFO:teuthology.orchestra.run.smithi026.stdout:[global] 2024-01-15T17:53:58.275 INFO:teuthology.orchestra.run.smithi026.stdout: chdir = "" 2024-01-15T17:53:58.275 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-15T17:53:58.275 INFO:teuthology.orchestra.run.smithi026.stdout: auth supported = cephx 2024-01-15T17:53:58.275 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.275 INFO:teuthology.orchestra.run.smithi026.stdout: filestore xattr use omap = true 2024-01-15T17:53:58.275 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: mon clock drift allowed = 1.000 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: osd crush chooseleaf type = 0 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: auth debug = true 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on old message = true 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: ms die on bug = true 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: mon pg warn max object skew = 0 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: # disable pg_autoscaler by default for new pools 2024-01-15T17:53:58.276 INFO:teuthology.orchestra.run.smithi026.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default size = 2 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow primary affinity = true 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd allow pg remap = true 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on legacy crush tunables = false 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on crush straw calc version zero = false 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on no sortbitwise = false 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on osd down out interval zero = false 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon warn on too few osds = false 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-15T17:53:58.277 INFO:teuthology.orchestra.run.smithi026.stdout: mon_allow_pool_size_one = true 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: osd default data pool replay window = 5 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: mon allow pool delete = true 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: mon cluster log file level = debug 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: debug asserts on shutdown = true 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: mon health detail to clog = false 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: mon host = "172.21.15.26,172.21.15.124,[v2:172.21.15.124:3301,v1:172.21.15.124:6790]" 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: fsid = 64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.278 INFO:teuthology.orchestra.run.smithi026.stdout:[osd] 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd journal size = 100 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub load threshold = 5.0 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd scrub max interval = 600 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd mclock profile = high_recovery_ops 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd recover clone overlap = true 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd recovery max chunk = 1048576 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug shutdown = true 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug op order = true 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug verify stray on activate = true 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug trim objects = true 2024-01-15T17:53:58.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: osd open classes on start = true 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug pg log writeout = true 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: osd deep scrub update digest min age = 30 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: osd map max advance = 10 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: journal zero on create = true 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: filestore ondisk finisher threads = 3 2024-01-15T17:53:58.280 INFO:teuthology.orchestra.run.smithi026.stdout: filestore apply finisher threads = 3 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: bdev debug aio = true 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: osd debug misdirected ops = true 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: bdev async discard = True 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: bdev enable discard = True 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore allocator = bitmap 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore block size = 96636764160 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: bluestore fsck on mount = True 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluefs = 1/20 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: debug bluestore = 1/20 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: debug osd = 20 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: debug rocksdb = 4/10 2024-01-15T17:53:58.281 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd backfillfull_ratio = 0.85 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd full ratio = 0.9 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd nearfull ratio = 0.8 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: osd failsafe full ratio = 0.95 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: osd objectstore = bluestore 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr] 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: debug mgr = 20 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-01-15T17:53:58.282 INFO:teuthology.orchestra.run.smithi026.stdout: mgr/telemetry/nag = false 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: debug finisher = 20 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout:[mon] 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: debug mon = 20 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: debug paxos = 20 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: debug auth = 20 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: mon data avail warn = 5 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: mon mgr mkfs grace = 240 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min pgs per osd = 4 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd reporter subtree level = osd 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: mon osd prime pg temp = true 2024-01-15T17:53:58.283 INFO:teuthology.orchestra.run.smithi026.stdout: mon reweight min bytes per osd = 10 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: auth mon ticket ttl = 660 # 11m 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: auth service ticket ttl = 240 # 4m 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: # don't complain about insecure global_id in the test suite 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: # 1m isn't quite enough 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: mon_down_mkfs_grace = 2m 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: mon_warn_on_filestore_osds = false 2024-01-15T17:53:58.284 INFO:teuthology.orchestra.run.smithi026.stdout: mon op complaint time = 120 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout:[client] 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: rgw cache enabled = true 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable ops log = true 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: rgw enable usage log = true 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: client mount timeout = 600 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: debug client = 20 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: pid file = "" 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.a] 2024-01-15T17:53:58.285 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.b] 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout:[mon.c] 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout:[mds] 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds = 20 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout: debug mds balancer = 20 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout: debug ms = 1 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug frag = True 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout: mds debug scatterstat = True 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout: mds op complaint time = 180 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout: mds verify scatter = True 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout: osd op complaint time = 180 2024-01-15T17:53:58.286 INFO:teuthology.orchestra.run.smithi026.stdout: rados mon op timeout = 900 2024-01-15T17:53:58.287 INFO:teuthology.orchestra.run.smithi026.stdout: rados osd op timeout = 900 2024-01-15T17:53:58.325 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-15T17:53:58.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-15T17:53:58.482 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-15T17:53:58.482 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:58.482 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-15T17:53:58.536 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:58.536 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-15T17:53:58.589 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:53:58.589 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:58.589 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-15T17:53:58.589 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-15T17:53:58.744 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:53:58.744 DEBUG:teuthology.orchestra.run.smithi026:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:53:58.845 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:53:58.845 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:53:58.845 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-15T17:53:58.845 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-15T17:53:58.953 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:53:58.954 DEBUG:teuthology.orchestra.run.smithi124:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:53:59.044 INFO:tasks.ceph:Setting up mon nodes... 2024-01-15T17:53:59.044 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-15T17:53:59.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-15T17:53:59.245 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-15T17:53:59.255 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-15T17:53:59.444 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-15T17:53:59.455 INFO:tasks.ceph:Setting up mds nodes... 2024-01-15T17:53:59.455 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-15T17:53:59.653 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-15T17:53:59.662 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-15T17:53:59.762 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-15T17:53:59.943 INFO:teuthology.orchestra.run.smithi026.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-15T17:53:59.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-15T17:54:00.047 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-15T17:54:00.222 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-15T17:54:00.232 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-15T17:54:00.327 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-15T17:54:00.501 INFO:teuthology.orchestra.run.smithi124.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-15T17:54:00.509 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-15T17:54:00.600 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-15T17:54:00.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-15T17:54:00.738 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-15T17:54:00.808 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-01-15T17:54:00.939 INFO:teuthology.orchestra.run.smithi026.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-01-15T17:54:00.999 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-01-15T17:54:01.126 INFO:teuthology.orchestra.run.smithi124.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-01-15T17:54:01.182 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-01-15T17:54:01.304 INFO:teuthology.orchestra.run.smithi124.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-01-15T17:54:01.350 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-15T17:54:01.350 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi026.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi124.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-15T17:54:01.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-15T17:54:01.458 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T17:54:01.459 INFO:tasks.ceph:role: osd.0 2024-01-15T17:54:01.459 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:54:01.459 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T17:54:01.554 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T17:54:01.587 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-01-15T17:54:01.594 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-01-15T17:54:01.594 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-15T17:54:01.703 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-15T17:54:01.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-15T17:54:01.887 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T17:54:01.887 INFO:tasks.ceph:role: osd.1 2024-01-15T17:54:01.887 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:54:01.887 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T17:54:01.983 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T17:54:02.017 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-01-15T17:54:02.022 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-01-15T17:54:02.022 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-15T17:54:02.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-15T17:54:02.232 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-15T17:54:02.323 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T17:54:02.323 INFO:tasks.ceph:role: osd.2 2024-01-15T17:54:02.324 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:54:02.324 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T17:54:02.424 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T17:54:02.459 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-01-15T17:54:02.465 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-01-15T17:54:02.465 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-15T17:54:02.580 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-15T17:54:02.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-15T17:54:02.754 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-15T17:54:02.754 INFO:tasks.ceph:role: osd.3 2024-01-15T17:54:02.754 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:54:02.755 DEBUG:teuthology.orchestra.run.smithi026:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-15T17:54:02.849 INFO:teuthology.orchestra.run.smithi026.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T17:54:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T17:54:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T17:54:02.849 INFO:teuthology.orchestra.run.smithi026.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-15T17:54:02.850 INFO:teuthology.orchestra.run.smithi026.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T17:54:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: = sunit=0 swidth=0 blks 2024-01-15T17:54:02.850 INFO:teuthology.orchestra.run.smithi026.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T17:54:02.850 INFO:teuthology.orchestra.run.smithi026.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-15T17:54:02.850 INFO:teuthology.orchestra.run.smithi026.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T17:54:02.850 INFO:teuthology.orchestra.run.smithi026.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T17:54:02.883 INFO:teuthology.orchestra.run.smithi026.stdout:Discarding blocks...Done. 2024-01-15T17:54:02.890 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi026.front.sepia.ceph.com -o noatime 2024-01-15T17:54:02.890 DEBUG:teuthology.orchestra.run.smithi026:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-15T17:54:03.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-15T17:54:03.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:03.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:03.244+0000 7f9db280f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-15T17:54:03.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:03.245+0000 7f9db280f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-15T17:54:03.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:03.245+0000 7f9db280f500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-15T17:54:03.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:03.245+0000 7f9db280f500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-15T17:54:03.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:03.245+0000 7f9db280f500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-15T17:54:03.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:03.248+0000 7f9db280f500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-15T17:54:06.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-15T17:54:06.520 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:06.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:06.661+0000 7f09c2612500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-15T17:54:06.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:06.661+0000 7f09c2612500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-15T17:54:06.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:06.661+0000 7f09c2612500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-15T17:54:06.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:06.661+0000 7f09c2612500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-15T17:54:06.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:06.662+0000 7f09c2612500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-15T17:54:06.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:06.664+0000 7f09c2612500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-15T17:54:09.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-15T17:54:09.948 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:10.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:10.089+0000 7fd542f39500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-15T17:54:10.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:10.090+0000 7fd542f39500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-15T17:54:10.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:10.090+0000 7fd542f39500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-15T17:54:10.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:10.090+0000 7fd542f39500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-15T17:54:10.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:10.090+0000 7fd542f39500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-15T17:54:10.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:10.092+0000 7fd542f39500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-15T17:54:13.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-15T17:54:13.372 DEBUG:teuthology.orchestra.run.smithi026:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:13.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:13.515+0000 7f026ddde500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-15T17:54:13.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:13.515+0000 7f026ddde500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-15T17:54:13.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:13.515+0000 7f026ddde500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-15T17:54:13.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:13.515+0000 7f026ddde500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-15T17:54:13.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:13.516+0000 7f026ddde500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-15T17:54:13.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:13.518+0000 7f026ddde500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-15T17:54:16.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-15T17:54:16.799 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-15T17:54:16.898 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T17:54:16.898 INFO:tasks.ceph:role: osd.4 2024-01-15T17:54:16.898 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:54:16.899 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-15T17:54:16.994 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T17:54:16.994 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T17:54:16.994 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T17:54:16.994 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-15T17:54:16.994 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T17:54:16.994 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2024-01-15T17:54:16.995 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T17:54:16.995 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-15T17:54:16.995 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T17:54:16.995 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T17:54:17.517 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2024-01-15T17:54:17.524 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2024-01-15T17:54:17.524 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-15T17:54:17.705 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-15T17:54:17.809 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-15T17:54:17.906 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T17:54:17.907 INFO:tasks.ceph:role: osd.5 2024-01-15T17:54:17.907 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:54:17.907 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T17:54:18.006 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T17:54:18.498 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2024-01-15T17:54:18.505 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2024-01-15T17:54:18.505 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-15T17:54:18.651 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-15T17:54:18.747 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-15T17:54:18.840 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T17:54:18.841 INFO:tasks.ceph:role: osd.6 2024-01-15T17:54:18.841 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:54:18.841 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-15T17:54:18.935 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T17:54:18.936 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T17:54:18.936 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T17:54:18.936 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-15T17:54:18.936 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T17:54:18.936 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2024-01-15T17:54:18.936 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T17:54:18.936 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-15T17:54:18.936 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T17:54:18.936 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T17:54:19.444 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2024-01-15T17:54:19.452 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2024-01-15T17:54:19.452 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-15T17:54:19.637 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-15T17:54:19.737 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-15T17:54:19.829 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-15T17:54:19.829 INFO:tasks.ceph:role: osd.7 2024-01-15T17:54:19.829 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:54:19.829 DEBUG:teuthology.orchestra.run.smithi124:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-15T17:54:19.926 INFO:teuthology.orchestra.run.smithi124.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-15T17:54:19.927 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-15T17:54:19.927 INFO:teuthology.orchestra.run.smithi124.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-15T17:54:19.927 INFO:teuthology.orchestra.run.smithi124.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-15T17:54:19.927 INFO:teuthology.orchestra.run.smithi124.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-15T17:54:19.927 INFO:teuthology.orchestra.run.smithi124.stdout: = sunit=0 swidth=0 blks 2024-01-15T17:54:19.927 INFO:teuthology.orchestra.run.smithi124.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-15T17:54:19.927 INFO:teuthology.orchestra.run.smithi124.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-15T17:54:19.927 INFO:teuthology.orchestra.run.smithi124.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-15T17:54:19.927 INFO:teuthology.orchestra.run.smithi124.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-15T17:54:20.418 INFO:teuthology.orchestra.run.smithi124.stdout:Discarding blocks...Done. 2024-01-15T17:54:20.424 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi124.front.sepia.ceph.com -o noatime 2024-01-15T17:54:20.424 DEBUG:teuthology.orchestra.run.smithi124:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-15T17:54:20.565 DEBUG:teuthology.orchestra.run.smithi124:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-15T17:54:20.659 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:20.816 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:20.813+0000 7efe126c3500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-15T17:54:20.821 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:20.818+0000 7efe126c3500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-15T17:54:20.821 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:20.818+0000 7efe126c3500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-15T17:54:20.821 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:20.818+0000 7efe126c3500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-15T17:54:20.821 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:20.818+0000 7efe126c3500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-15T17:54:20.831 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:20.828+0000 7efe126c3500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-15T17:54:23.785 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-15T17:54:23.885 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:24.021 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:24.018+0000 7fcd04c2a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-15T17:54:24.022 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:24.019+0000 7fcd04c2a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-15T17:54:24.022 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:24.019+0000 7fcd04c2a500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-15T17:54:24.022 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:24.019+0000 7fcd04c2a500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-15T17:54:24.022 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:24.019+0000 7fcd04c2a500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-15T17:54:24.024 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:24.021+0000 7fcd04c2a500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-15T17:54:25.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:54:25.389 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:54:27.212 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-15T17:54:27.310 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:27.448 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:27.446+0000 7f525c21b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-15T17:54:27.449 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:27.446+0000 7f525c21b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-15T17:54:27.450 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:27.447+0000 7f525c21b500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-15T17:54:27.450 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:27.447+0000 7f525c21b500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-15T17:54:27.450 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:27.447+0000 7f525c21b500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-15T17:54:27.452 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:27.449+0000 7f525c21b500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-01-15T17:54:30.614 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-15T17:54:30.710 DEBUG:teuthology.orchestra.run.smithi124:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:30.851 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:30.847+0000 7f8260a49500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-15T17:54:30.851 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:30.848+0000 7f8260a49500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-15T17:54:30.851 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:30.848+0000 7f8260a49500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-15T17:54:30.851 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:30.848+0000 7f8260a49500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-15T17:54:30.851 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:30.848+0000 7f8260a49500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-15T17:54:30.853 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:30.850+0000 7f8260a49500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-01-15T17:54:34.017 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-15T17:54:34.105 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-15T17:54:34.105 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:34.105 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-15T17:54:34.213 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:34.213 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-15T17:54:34.317 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:34.317 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-15T17:54:34.417 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:34.417 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-15T17:54:34.518 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:34.518 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-15T17:54:34.617 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:34.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-15T17:54:34.718 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:34.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-15T17:54:34.819 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:34.819 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-15T17:54:34.916 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:34.917 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-15T17:54:35.006 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:35.006 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-15T17:54:35.094 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:35.094 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-15T17:54:35.185 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:35.185 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-15T17:54:35.321 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:35.321 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-15T17:54:35.408 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:35.410 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-15T17:54:35.504 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:35.504 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-15T17:54:35.564 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:35.565 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-01-15T17:54:35.621 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:35.621 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-01-15T17:54:35.676 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:35.676 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-01-15T17:54:35.726 INFO:tasks.ceph:Adding keys to all mons... 2024-01-15T17:54:35.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-15T17:54:35.734 DEBUG:teuthology.orchestra.run.smithi124:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr.y] 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA3caVle8Z+DhAA9cWxzc4puvrHAedqupPG3Q== 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.a] 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA3caVl3erNJhAANxPK+39yVM5neq9iKC1HkA== 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.c] 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA3caVlaIAVOBAAA8LEFM1HtzRtNooQCCjS/w== 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.0] 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA7caVlCTmgDhAAPWb0qEh134xxyF8lM9DoNg== 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.1] 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA+caVlaxF5JxAA23Z9j21ATy3GVlhgndDIxw== 2024-01-15T17:54:35.820 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.2] 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBCcaVlcRtjBRAAOFN9Xf5kN5MXL6yQfRI27g== 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.3] 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBFcaVlvAzEHhAACDoNh7BZY9gbKTdGwIrhlg== 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout:[mgr.x] 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA3caVlvuBcGhAAGqdCMSffJwnMKlFh3CtyCw== 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.b] 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA4caVlLaUlDRAAeKaen5hLlwDDWY6OAMRsPQ== 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout:[mds.d] 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA4caVlmpTBHRAAUwbZeVpysMsn56QRLRJCoA== 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.4] 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBMcaVl8yiOMBAAdI4NTeSB1d/nSHRaIDMcFw== 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.5] 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBQcaVltf8sARAAfPaJAfoTgQoEivw90jRz/Q== 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.6] 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBTcaVltuKiGhAAT0cIJiRkQ38QXT/h6Gyggg== 2024-01-15T17:54:35.821 INFO:teuthology.orchestra.run.smithi124.stdout:[osd.7] 2024-01-15T17:54:35.822 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQBWcaVlbJqVMhAAKjpl6pAHbytjZWMYeRbRjA== 2024-01-15T17:54:35.822 INFO:teuthology.orchestra.run.smithi124.stdout:[client.0] 2024-01-15T17:54:35.822 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA== 2024-01-15T17:54:35.822 INFO:teuthology.orchestra.run.smithi124.stdout:[client.2] 2024-01-15T17:54:35.822 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA4caVlO8vdNxAAlUE/TFCDQoXuxvwLPwUvtg== 2024-01-15T17:54:35.822 INFO:teuthology.orchestra.run.smithi124.stdout:[client.1] 2024-01-15T17:54:35.822 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA5caVlrVpqBxAAxOL0iuAlOvfR+KgSsin3vQ== 2024-01-15T17:54:35.822 INFO:teuthology.orchestra.run.smithi124.stdout:[client.3] 2024-01-15T17:54:35.822 INFO:teuthology.orchestra.run.smithi124.stdout: key = AQA5caVl5qgCEhAALgt7d3OjEmq5bc4nAD2rTw== 2024-01-15T17:54:35.824 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.y] 2024-01-15T17:54:35.824 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA3caVle8Z+DhAA9cWxzc4puvrHAedqupPG3Q== 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.a] 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA3caVl3erNJhAANxPK+39yVM5neq9iKC1HkA== 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.c] 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA3caVlaIAVOBAAA8LEFM1HtzRtNooQCCjS/w== 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.0] 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA7caVlCTmgDhAAPWb0qEh134xxyF8lM9DoNg== 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.1] 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA+caVlaxF5JxAA23Z9j21ATy3GVlhgndDIxw== 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.2] 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBCcaVlcRtjBRAAOFN9Xf5kN5MXL6yQfRI27g== 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.3] 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBFcaVlvAzEHhAACDoNh7BZY9gbKTdGwIrhlg== 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout:[mgr.x] 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA3caVlvuBcGhAAGqdCMSffJwnMKlFh3CtyCw== 2024-01-15T17:54:35.825 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.b] 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVlLaUlDRAAeKaen5hLlwDDWY6OAMRsPQ== 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout:[mds.d] 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVlmpTBHRAAUwbZeVpysMsn56QRLRJCoA== 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.4] 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBMcaVl8yiOMBAAdI4NTeSB1d/nSHRaIDMcFw== 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.5] 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBQcaVltf8sARAAfPaJAfoTgQoEivw90jRz/Q== 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.6] 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBTcaVltuKiGhAAT0cIJiRkQ38QXT/h6Gyggg== 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout:[osd.7] 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQBWcaVlbJqVMhAAKjpl6pAHbytjZWMYeRbRjA== 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA== 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout:[client.2] 2024-01-15T17:54:35.826 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVlO8vdNxAAlUE/TFCDQoXuxvwLPwUvtg== 2024-01-15T17:54:35.827 INFO:teuthology.orchestra.run.smithi026.stdout:[client.1] 2024-01-15T17:54:35.827 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA5caVlrVpqBxAAxOL0iuAlOvfR+KgSsin3vQ== 2024-01-15T17:54:35.827 INFO:teuthology.orchestra.run.smithi026.stdout:[client.3] 2024-01-15T17:54:35.827 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA5caVl5qgCEhAALgt7d3OjEmq5bc4nAD2rTw== 2024-01-15T17:54:35.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-15T17:54:35.834 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-15T17:54:36.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T17:54:36.015 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T17:54:36.162 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T17:54:36.169 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T17:54:36.316 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:36.321 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:36.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:36.467 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:36.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:36.616 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:36.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:36.761 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:36.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-15T17:54:36.907 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-15T17:54:37.052 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T17:54:37.058 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T17:54:37.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T17:54:37.209 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-15T17:54:37.361 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:37.369 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:37.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:37.511 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:37.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:37.657 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:37.802 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:37.807 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-15T17:54:37.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T17:54:37.960 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T17:54:38.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T17:54:38.113 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T17:54:38.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T17:54:38.258 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T17:54:38.407 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T17:54:38.411 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-15T17:54:38.563 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-15T17:54:38.563 DEBUG:teuthology.orchestra.run.smithi026:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-15T17:54:38.652 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-15T17:54:38.821 DEBUG:teuthology.orchestra.run.smithi026:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-15T17:54:38.899 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-15T17:54:38.999 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-15T17:54:39.155 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-15T17:54:39.231 DEBUG:teuthology.orchestra.run.smithi124:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-15T17:54:39.312 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-15T17:54:39.464 DEBUG:teuthology.orchestra.run.smithi124:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-15T17:54:39.552 DEBUG:teuthology.orchestra.run.smithi026:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:39.559 DEBUG:teuthology.orchestra.run.smithi124:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-15T17:54:39.613 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-15T17:54:39.613 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-15T17:54:39.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-01-15T17:54:39.620 INFO:tasks.ceph.mon.a:Started 2024-01-15T17:54:39.620 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-15T17:54:39.620 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-01-15T17:54:39.625 INFO:tasks.ceph.mon.b:Started 2024-01-15T17:54:39.626 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-15T17:54:39.626 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-01-15T17:54:39.630 INFO:tasks.ceph.mon.c:Started 2024-01-15T17:54:39.631 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-15T17:54:39.631 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-15T17:54:39.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-01-15T17:54:39.638 INFO:tasks.ceph.mgr.y:Started 2024-01-15T17:54:39.638 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-15T17:54:39.638 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-01-15T17:54:39.643 INFO:tasks.ceph.mgr.x:Started 2024-01-15T17:54:39.643 INFO:tasks.ceph:Setting crush tunables to default 2024-01-15T17:54:39.643 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph osd crush tunables default 2024-01-15T17:54:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.820+0000 7f4b09000640 1 Processor -- start 2024-01-15T17:54:39.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.820+0000 7f4b09000640 1 -- start start 2024-01-15T17:54:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.821+0000 7f4b09000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.821+0000 7f4b09000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.821+0000 7f4b09000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b040d0320 con 0x7f4b0415d4b0 2024-01-15T17:54:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.821+0000 7f4b09000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b040d1c60 con 0x7f4b040cf9a0 2024-01-15T17:54:39.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.821+0000 7f4b09000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040b2730 con 0x7f4b040b31d0 2024-01-15T17:54:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.821+0000 7f4b03400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4b0415d4b0 0x7f4b040d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42040/0 (socket says 172.21.15.26:42040) 2024-01-15T17:54:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.821+0000 7f4b03400640 1 -- 172.21.15.26:0/3054290157 learned_addr learned my addr 172.21.15.26:0/3054290157 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:54:39.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 msgr2=0x7f4b040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-15T17:54:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 msgr2=0x7f4b040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 msgr2=0x7f4b040b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-15T17:54:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 msgr2=0x7f4b040b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:39.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:39.822+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.023+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.023+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.024+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 msgr2=0x7f4b040b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-15T17:54:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.024+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 msgr2=0x7f4b040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-15T17:54:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.024+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 msgr2=0x7f4b040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.024+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 msgr2=0x7f4b040b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.024+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.024+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.024+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-15T17:54:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.024+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-15T17:54:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.425+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.425+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.426+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 msgr2=0x7f4b040b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-15T17:54:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.426+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 msgr2=0x7f4b040b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.426+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.426+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 msgr2=0x7f4b040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-15T17:54:40.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.426+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 msgr2=0x7f4b040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.426+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.426+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-15T17:54:40.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:40.426+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-15T17:54:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.227+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.227+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.228+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 msgr2=0x7f4b040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-15T17:54:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.228+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 msgr2=0x7f4b040cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.228+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.228+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-15T17:54:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.228+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 msgr2=0x7f4b040b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-15T17:54:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.228+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 msgr2=0x7f4b040b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.228+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:41.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:41.228+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-15T17:54:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.822+0000 7f4b01600640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 msgr2=0x7f4b040b3660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T17:54:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.822+0000 7f4b01600640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4b040b31d0 0x7f4b040b3660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.822+0000 7f4b01600640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 msgr2=0x7f4b040cfde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T17:54:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.822+0000 7f4b01600640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.124:3300/0 conn(0x7f4b040cf9a0 0x7f4b040cfde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.822+0000 7f4b01600640 1 -- 172.21.15.26:0/3054290157 >> v1:172.21.15.26:6789/0 conn(0x7f4b0415d4b0 legacy=0x7f4b040d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.822+0000 7f4b01600640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.822+0000 7f4b01600640 1 -- 172.21.15.26:0/3054290157 --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b040d1ec0 con 0x7f4adc0036c0 2024-01-15T17:54:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.822+0000 7f4b01600640 1 -- 172.21.15.26:0/3054290157 --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b040d1c60 con 0x7f4adc007350 2024-01-15T17:54:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.823+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.823+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 msgr2=0x7f4adc009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-15T17:54:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.823+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 msgr2=0x7f4adc009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.823+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:42.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:42.823+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:43.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.025+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.025+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 msgr2=0x7f4adc009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-15T17:54:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.025+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 msgr2=0x7f4adc009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.025+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:43.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.025+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-15T17:54:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.426+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:43.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.426+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 msgr2=0x7f4adc009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-15T17:54:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.426+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 msgr2=0x7f4adc009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.426+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:43.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:43.426+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-15T17:54:44.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.228+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.228+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 msgr2=0x7f4adc009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-15T17:54:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.228+0000 7f4afa800640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 msgr2=0x7f4adc009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.228+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:44.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.229+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-15T17:54:44.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.854+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215006105 0 0) 0x7f4b040d1ec0 con 0x7f4adc0036c0 2024-01-15T17:54:44.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.854+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ae4003520 con 0x7f4adc0036c0 2024-01-15T17:54:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4130431596 0 0) 0x7f4ae4003520 con 0x7f4adc0036c0 2024-01-15T17:54:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 msgr2=0x7f4adc009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T17:54:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 --2- 172.21.15.26:0/3054290157 >> v2:172.21.15.26:3300/0 conn(0x7f4adc007350 0x7f4adc009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc00af10 con 0x7f4adc0036c0 2024-01-15T17:54:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f4af40016f0 con 0x7f4adc0036c0 2024-01-15T17:54:44.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 >> v1:172.21.15.124:6789/0 conn(0x7f4adc0036c0 legacy=0x7f4adc005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:44.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 reap_dead start 2024-01-15T17:54:44.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae4003970 0x7f4ae4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:44.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae40047c0 0x7f4ae4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:44.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40052a0 0x7f4ae400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:44.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.860+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040d1c60 con 0x7f4ae4003970 2024-01-15T17:54:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.861+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae400c1f0 con 0x7f4ae40047c0 2024-01-15T17:54:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.861+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae400c3b0 con 0x7f4ae40052a0 2024-01-15T17:54:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.861+0000 7f4b03400640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae4003970 0x7f4ae4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.861+0000 7f4afa800640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae40047c0 0x7f4ae4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.861+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40052a0 0x7f4ae400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.862+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40052a0 msgr2=0x7f4ae400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-15T17:54:44.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.862+0000 7f4b03e00640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40052a0 msgr2=0x7f4ae400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.862+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40052a0 0x7f4ae400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.862+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40052a0 0x7f4ae400b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.862+0000 7f4b03400640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40052a0 msgr2=0x7f4ae400b980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T17:54:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.862+0000 7f4b03400640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40052a0 0x7f4ae400b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.862+0000 7f4b03400640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae40047c0 msgr2=0x7f4ae4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.862+0000 7f4b03400640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae40047c0 0x7f4ae4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.862+0000 7f4b03400640 1 -- 172.21.15.26:0/3054290157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae4003520 con 0x7f4ae4003970 2024-01-15T17:54:44.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.864+0000 7f4b03400640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae4003970 0x7f4ae4003d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4af4001eb0 tx=0x7f4ae40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e694621f151df89d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.865+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4005ad0 con 0x7f4ae4003970 2024-01-15T17:54:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.865+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af4005c30 con 0x7f4ae4003970 2024-01-15T17:54:44.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.865+0000 7f4b02a00640 1 -- 172.21.15.26:0/3054290157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af402c870 con 0x7f4ae4003970 2024-01-15T17:54:44.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.866+0000 7f4b09000640 1 -- 172.21.15.26:0/3054290157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae4003970 msgr2=0x7f4ae4003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.866+0000 7f4b09000640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae4003970 0x7f4ae4003d70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4af4001eb0 tx=0x7f4ae40057f0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.866+0000 7f4b09000640 1 -- 172.21.15.26:0/3054290157 shutdown_connections 2024-01-15T17:54:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.866+0000 7f4b09000640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40052a0 0x7f4ae400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.866+0000 7f4b09000640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae40047c0 0x7f4ae4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.866+0000 7f4b09000640 1 --2- 172.21.15.26:0/3054290157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae4003970 0x7f4ae4003d70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.866+0000 7f4b09000640 1 -- 172.21.15.26:0/3054290157 >> 172.21.15.26:0/3054290157 conn(0x7f4b040d3000 msgr2=0x7f4b040d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.867+0000 7f4b09000640 1 -- 172.21.15.26:0/3054290157 shutdown_connections 2024-01-15T17:54:44.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.867+0000 7f4b09000640 1 -- 172.21.15.26:0/3054290157 wait complete. 2024-01-15T17:54:44.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.870+0000 7f4b09000640 1 Processor -- start 2024-01-15T17:54:44.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.871+0000 7f4b09000640 1 -- start start 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.872+0000 7f4b09000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae4003970 0x7f4b04151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4b09000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40047c0 0x7f4b0415ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4b09000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae40052a0 0x7f4b041604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4b09000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b041f7000 con 0x7f4ae40052a0 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4b09000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040cbc00 con 0x7f4ae4003970 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4b09000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040cbee0 con 0x7f4ae40047c0 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4b03400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae4003970 0x7f4b04151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4b03e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae40052a0 0x7f4b041604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4b03400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae4003970 0x7f4b04151560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58550/0 (socket says 172.21.15.26:58550) 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4b03400640 1 -- 172.21.15.26:0/3390080208 learned_addr learned my addr 172.21.15.26:0/3390080208 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4afa800640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40047c0 0x7f4b0415ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4afa800640 1 -- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40047c0 msgr2=0x7f4b0415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-01-15T17:54:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4afa800640 1 -- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40047c0 msgr2=0x7f4b0415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-15T17:54:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.873+0000 7f4afa800640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40047c0 0x7f4b0415ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4afa800640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40047c0 0x7f4b0415ff90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4b03e00640 1 -- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40047c0 msgr2=0x7f4b0415ff90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:54:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40047c0 0x7f4b0415ff90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4b03e00640 1 -- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae4003970 msgr2=0x7f4b04151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae4003970 0x7f4b04151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4b03e00640 1 -- 172.21.15.26:0/3390080208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b040cc1c0 con 0x7f4ae40052a0 2024-01-15T17:54:44.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4b03e00640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae40052a0 0x7f4b041604d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b0406aaa0 tx=0x7f4afc009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4afbe00640 1 -- 172.21.15.26:0/3390080208 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc013a70 con 0x7f4ae40052a0 2024-01-15T17:54:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4afbe00640 1 -- 172.21.15.26:0/3390080208 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4afc00f040 con 0x7f4ae40052a0 2024-01-15T17:54:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.874+0000 7f4afbe00640 1 -- 172.21.15.26:0/3390080208 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4afc013e30 con 0x7f4ae40052a0 2024-01-15T17:54:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.875+0000 7f4b09000640 1 -- 172.21.15.26:0/3390080208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b040cc4a0 con 0x7f4ae40052a0 2024-01-15T17:54:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.875+0000 7f4b09000640 1 -- 172.21.15.26:0/3390080208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b04151aa0 con 0x7f4ae40052a0 2024-01-15T17:54:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.875+0000 7f4afbe00640 1 -- 172.21.15.26:0/3390080208 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4afc016730 con 0x7f4ae40052a0 2024-01-15T17:54:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.875+0000 7f4afbe00640 1 -- 172.21.15.26:0/3390080208 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4afc01fbe0 con 0x7f4ae40052a0 2024-01-15T17:54:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.876+0000 7f4b09000640 1 -- 172.21.15.26:0/3390080208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b040cc690 con 0x7f4ae40052a0 2024-01-15T17:54:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.880+0000 7f4afbe00640 1 -- 172.21.15.26:0/3390080208 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4afc01a030 con 0x7f4ae40052a0 2024-01-15T17:54:44.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.963+0000 7f4b09000640 1 -- 172.21.15.26:0/3390080208 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4b04067450 con 0x7f4ae40052a0 2024-01-15T17:54:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.967+0000 7f4afbe00640 1 -- 172.21.15.26:0/3390080208 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4afc003c30 con 0x7f4ae40052a0 2024-01-15T17:54:44.970 INFO:teuthology.orchestra.run.smithi026.stderr:adjusted tunables profile to default 2024-01-15T17:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.968+0000 7f4b09000640 1 -- 172.21.15.26:0/3390080208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae40052a0 msgr2=0x7f4b041604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:44.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.968+0000 7f4b09000640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae40052a0 0x7f4b041604d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4b0406aaa0 tx=0x7f4afc009d70 comp rx=0 tx=0).stop 2024-01-15T17:54:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.969+0000 7f4b09000640 1 -- 172.21.15.26:0/3390080208 shutdown_connections 2024-01-15T17:54:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.969+0000 7f4b09000640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ae40047c0 0x7f4b0415ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.969+0000 7f4b09000640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ae40052a0 0x7f4b041604d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.969+0000 7f4b09000640 1 --2- 172.21.15.26:0/3390080208 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ae4003970 0x7f4b04151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.969+0000 7f4b09000640 1 -- 172.21.15.26:0/3390080208 >> 172.21.15.26:0/3390080208 conn(0x7f4b040d3000 msgr2=0x7f4b040ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:44.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.969+0000 7f4b09000640 1 -- 172.21.15.26:0/3390080208 shutdown_connections 2024-01-15T17:54:44.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:44.969+0000 7f4b09000640 1 -- 172.21.15.26:0/3390080208 wait complete. 2024-01-15T17:54:44.992 INFO:tasks.ceph:check_enable_crimson: False 2024-01-15T17:54:44.992 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-15T17:54:44.992 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:44.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-15T17:54:45.091 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:45.091 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-15T17:54:45.196 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:45.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-15T17:54:45.298 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:54:45.298 DEBUG:teuthology.orchestra.run.smithi026:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-15T17:54:45.399 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:45.399 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-15T17:54:45.473 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:45.471+0000 7f8fdde3d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-15T17:54:45.488 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:45.488 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-15T17:54:45.540 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:45.537+0000 7f8fdde3d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-15T17:54:45.614 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:45.614 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-15T17:54:45.742 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:54:45.742 DEBUG:teuthology.orchestra.run.smithi124:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-15T17:54:45.821 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new c8f57d9b-0259-4478-80d0-1ed0a378a557 0 2024-01-15T17:54:46.052 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.049+0000 7f044be00640 1 Processor -- start 2024-01-15T17:54:46.052 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.049+0000 7f044be00640 1 -- start start 2024-01-15T17:54:46.053 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.050+0000 7f044be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f044c15b320 0x7f044c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.054 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.051+0000 7f044be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f044c0cc730 con 0x7f044c0ce5a0 2024-01-15T17:54:46.054 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.051+0000 7f044be00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f044c0cbe00 con 0x7f044c0ccbc0 2024-01-15T17:54:46.054 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.051+0000 7f044be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044c0cc0e0 con 0x7f044c15b320 2024-01-15T17:54:46.054 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.051+0000 7f044aa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f044c15b320 0x7f044c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.054 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.051+0000 7f044aa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f044c15b320 0x7f044c0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37294/0 (socket says 172.21.15.124:37294) 2024-01-15T17:54:46.054 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.051+0000 7f044aa00640 1 -- 172.21.15.124:0/3729989183 learned_addr learned my addr 172.21.15.124:0/3729989183 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:46.054 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.051+0000 7f044aa00640 1 -- 172.21.15.124:0/3729989183 >> v1:172.21.15.124:6789/0 conn(0x7f044c0ccbc0 legacy=0x7f044c15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.054 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.051+0000 7f044aa00640 1 -- 172.21.15.124:0/3729989183 >> v1:172.21.15.26:6789/0 conn(0x7f044c0ce5a0 legacy=0x7f044c0ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:46.055 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.051+0000 7f044aa00640 1 -- 172.21.15.124:0/3729989183 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f044c0cc3c0 con 0x7f044c15b320 2024-01-15T17:54:46.055 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.052+0000 7f044aa00640 1 --2- 172.21.15.124:0/3729989183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c15b320 0x7f044c0ce060 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f043c009870 tx=0x7f043c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=335b179675678cfb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:46.055 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.053+0000 7f044a000640 1 -- 172.21.15.124:0/3729989183 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c030860 con 0x7f044c15b320 2024-01-15T17:54:46.056 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.053+0000 7f044a000640 1 -- 172.21.15.124:0/3729989183 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c0309c0 con 0x7f044c15b320 2024-01-15T17:54:46.056 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.053+0000 7f044a000640 1 -- 172.21.15.124:0/3729989183 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c030d50 con 0x7f044c15b320 2024-01-15T17:54:46.056 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.053+0000 7f044be00640 1 -- 172.21.15.124:0/3729989183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c15b320 msgr2=0x7f044c0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.057 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.053+0000 7f044be00640 1 --2- 172.21.15.124:0/3729989183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c15b320 0x7f044c0ce060 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f043c009870 tx=0x7f043c02b9a0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.057 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.053+0000 7f044be00640 1 -- 172.21.15.124:0/3729989183 shutdown_connections 2024-01-15T17:54:46.057 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.053+0000 7f044be00640 1 --2- 172.21.15.124:0/3729989183 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c15b320 0x7f044c0ce060 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.057 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.053+0000 7f044be00640 1 -- 172.21.15.124:0/3729989183 >> 172.21.15.124:0/3729989183 conn(0x7f044c0b2a60 msgr2=0x7f044c0b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:46.057 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.054+0000 7f044be00640 1 -- 172.21.15.124:0/3729989183 shutdown_connections 2024-01-15T17:54:46.057 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.054+0000 7f044be00640 1 -- 172.21.15.124:0/3729989183 wait complete. 2024-01-15T17:54:46.058 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.055+0000 7f044be00640 1 Processor -- start 2024-01-15T17:54:46.058 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.055+0000 7f044be00640 1 -- start start 2024-01-15T17:54:46.061 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.056+0000 7f044be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0ccbc0 0x7f044c14d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.061 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.056+0000 7f044be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c0ce5a0 0x7f044c0c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.061 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.056+0000 7f044be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f044c15b320 0x7f044c0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.061 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.056+0000 7f044be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c0c6b40 con 0x7f044c15b320 2024-01-15T17:54:46.061 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.056+0000 7f044be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044c0c5560 con 0x7f044c0ccbc0 2024-01-15T17:54:46.061 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.056+0000 7f044be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044c0c5840 con 0x7f044c0ce5a0 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044aa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0ccbc0 0x7f044c14d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044aa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0ccbc0 0x7f044c14d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37296/0 (socket says 172.21.15.124:37296) 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044aa00640 1 -- 172.21.15.124:0/1822463153 learned_addr learned my addr 172.21.15.124:0/1822463153 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f0443e00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c0ce5a0 0x7f044c0c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f0443e00640 1 -- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c0ce5a0 msgr2=0x7f044c0c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f0443e00640 1 -- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c0ce5a0 msgr2=0x7f044c0c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f0443e00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c0ce5a0 0x7f044c0c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044b400640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f044c15b320 0x7f044c0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f0443e00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c0ce5a0 0x7f044c0c6600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044aa00640 1 -- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c0ce5a0 msgr2=0x7f044c0c6600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044aa00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c0ce5a0 0x7f044c0c6600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044aa00640 1 -- 172.21.15.124:0/1822463153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f044c15b320 msgr2=0x7f044c0b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044aa00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f044c15b320 0x7f044c0b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044aa00640 1 -- 172.21.15.124:0/1822463153 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c0094d0 con 0x7f044c0ccbc0 2024-01-15T17:54:46.062 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044b400640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f044c15b320 0x7f044c0b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:54:46.063 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.057+0000 7f044aa00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0ccbc0 0x7f044c14d1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f043c009870 tx=0x7f043c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:46.063 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.058+0000 7f0442a00640 1 -- 172.21.15.124:0/1822463153 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c030810 con 0x7f044c0ccbc0 2024-01-15T17:54:46.063 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.058+0000 7f0442a00640 1 -- 172.21.15.124:0/1822463153 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f043c039d90 con 0x7f044c0ccbc0 2024-01-15T17:54:46.063 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.058+0000 7f0442a00640 1 -- 172.21.15.124:0/1822463153 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c043610 con 0x7f044c0ccbc0 2024-01-15T17:54:46.063 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.058+0000 7f044be00640 1 -- 172.21.15.124:0/1822463153 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f044c0c5b20 con 0x7f044c0ccbc0 2024-01-15T17:54:46.063 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.058+0000 7f044be00640 1 -- 172.21.15.124:0/1822463153 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f044c0c60e0 con 0x7f044c0ccbc0 2024-01-15T17:54:46.063 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.058+0000 7f0442a00640 1 -- 172.21.15.124:0/1822463153 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f043c043a70 con 0x7f044c0ccbc0 2024-01-15T17:54:46.063 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.058+0000 7f0442a00640 1 -- 172.21.15.124:0/1822463153 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f043c04cb00 con 0x7f044c0ccbc0 2024-01-15T17:54:46.063 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.059+0000 7f044be00640 1 -- 172.21.15.124:0/1822463153 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f044c067b60 con 0x7f044c0ccbc0 2024-01-15T17:54:46.065 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.062+0000 7f0442a00640 1 -- 172.21.15.124:0/1822463153 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f043c048070 con 0x7f044c0ccbc0 2024-01-15T17:54:46.144 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.141+0000 7f044be00640 1 -- 172.21.15.124:0/1822463153 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c8f57d9b-0259-4478-80d0-1ed0a378a557", "id": 0} v 0) v1 -- 0x7f044c15b9b0 con 0x7f044c0ccbc0 2024-01-15T17:54:46.151 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.148+0000 7f0442a00640 1 -- 172.21.15.124:0/1822463153 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c8f57d9b-0259-4478-80d0-1ed0a378a557", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f043c034070 con 0x7f044c0ccbc0 2024-01-15T17:54:46.151 INFO:teuthology.orchestra.run.smithi124.stdout:0 2024-01-15T17:54:46.153 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.150+0000 7f044be00640 1 -- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0ccbc0 msgr2=0x7f044c14d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.153 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.150+0000 7f044be00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0ccbc0 0x7f044c14d1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f043c009870 tx=0x7f043c004730 comp rx=0 tx=0).stop 2024-01-15T17:54:46.153 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.150+0000 7f044be00640 1 -- 172.21.15.124:0/1822463153 shutdown_connections 2024-01-15T17:54:46.153 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.150+0000 7f044be00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f044c0ce5a0 0x7f044c0c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.153 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.150+0000 7f044be00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f044c15b320 0x7f044c0b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.153 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.150+0000 7f044be00640 1 --2- 172.21.15.124:0/1822463153 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f044c0ccbc0 0x7f044c14d1b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.153 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.150+0000 7f044be00640 1 -- 172.21.15.124:0/1822463153 >> 172.21.15.124:0/1822463153 conn(0x7f044c0b2a60 msgr2=0x7f044c155c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:46.153 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.150+0000 7f044be00640 1 -- 172.21.15.124:0/1822463153 shutdown_connections 2024-01-15T17:54:46.153 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.150+0000 7f044be00640 1 -- 172.21.15.124:0/1822463153 wait complete. 2024-01-15T17:54:46.175 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new b107ddc6-f573-40ca-8c09-62c5e8f4bfdc 1 2024-01-15T17:54:46.213 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.211+0000 7f8fdde3d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-15T17:54:46.278 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.275+0000 7f8fdde3d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-15T17:54:46.332 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.329+0000 7f8fdde3d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-15T17:54:46.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.350+0000 7f8d4b400640 1 Processor -- start 2024-01-15T17:54:46.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.350+0000 7f8d4b400640 1 -- start start 2024-01-15T17:54:46.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.351+0000 7f8d4b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d4415d4b0 0x7f8d440d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.355 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.352+0000 7f8d4b400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d440b2730 con 0x7f8d440cf9a0 2024-01-15T17:54:46.355 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.352+0000 7f8d4b400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d440d1ec0 con 0x7f8d440b41e0 2024-01-15T17:54:46.355 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.352+0000 7f8d4b400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d440b29d0 con 0x7f8d4415d4b0 2024-01-15T17:54:46.355 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.352+0000 7f8d4aa00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f8d440b41e0 0x7f8d440ce500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:49494/0 (socket says 172.21.15.124:49494) 2024-01-15T17:54:46.355 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.352+0000 7f8d4aa00640 1 -- 172.21.15.124:0/116062393 learned_addr learned my addr 172.21.15.124:0/116062393 (peer_addr_for_me v1:172.21.15.124:0/0) 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.352+0000 7f8d4a000640 1 --2- 172.21.15.124:0/116062393 >> v2:172.21.15.26:3300/0 conn(0x7f8d4415d4b0 0x7f8d440d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.352+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499814442 0 0) 0x7f8d440d1ec0 con 0x7f8d440b41e0 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.352+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d24003520 con 0x7f8d440b41e0 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867379096 0 0) 0x7f8d24003520 con 0x7f8d440b41e0 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 >> v2:172.21.15.26:3300/0 conn(0x7f8d4415d4b0 msgr2=0x7f8d440d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 --2- 172.21.15.124:0/116062393 >> v2:172.21.15.26:3300/0 conn(0x7f8d4415d4b0 0x7f8d440d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 >> v1:172.21.15.26:6789/0 conn(0x7f8d440cf9a0 legacy=0x7f8d440cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d440b2cb0 con 0x7f8d440b41e0 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8d3c0019f0 con 0x7f8d440b41e0 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 >> v1:172.21.15.124:6789/0 conn(0x7f8d440b41e0 legacy=0x7f8d440ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d24003970 0x7f8d24003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.357 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d240047c0 0x7f8d24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d2400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d440b29d0 con 0x7f8d240047c0 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2400c020 con 0x7f8d24005180 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2400c1e0 con 0x7f8d24003970 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d4a000640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d24003970 0x7f8d24003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d4aa00640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d2400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.353+0000 7f8d49600640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d240047c0 0x7f8d24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.354+0000 7f8d4aa00640 1 -- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d24003970 msgr2=0x7f8d24003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.354+0000 7f8d4aa00640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d24003970 0x7f8d24003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.354+0000 7f8d4aa00640 1 -- 172.21.15.124:0/116062393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d240047c0 msgr2=0x7f8d24004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.354+0000 7f8d4aa00640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d240047c0 0x7f8d24004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.358 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.354+0000 7f8d4aa00640 1 -- 172.21.15.124:0/116062393 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24003520 con 0x7f8d24005180 2024-01-15T17:54:46.359 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.354+0000 7f8d4aa00640 1 -- 172.21.15.124:0/116062393 reap_dead start 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.355+0000 7f8d4aa00640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d2400b980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00a800 tx=0x7f8d240057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eac0334435de053 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.355+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0072d0 con 0x7f8d24005180 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.355+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c02f070 con 0x7f8d24005180 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.355+0000 7f8d48c00640 1 -- 172.21.15.124:0/116062393 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c007690 con 0x7f8d24005180 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.356+0000 7f8d4b400640 1 -- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 msgr2=0x7f8d2400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.356+0000 7f8d4b400640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d2400b980 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00a800 tx=0x7f8d240057f0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.356+0000 7f8d4b400640 1 -- 172.21.15.124:0/116062393 shutdown_connections 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.356+0000 7f8d4b400640 1 --2- 172.21.15.124:0/116062393 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d2400b980 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.356+0000 7f8d4b400640 1 -- 172.21.15.124:0/116062393 >> 172.21.15.124:0/116062393 conn(0x7f8d440d3000 msgr2=0x7f8d440d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.356+0000 7f8d4b400640 1 -- 172.21.15.124:0/116062393 shutdown_connections 2024-01-15T17:54:46.360 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.356+0000 7f8d4b400640 1 -- 172.21.15.124:0/116062393 wait complete. 2024-01-15T17:54:46.361 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.358+0000 7f8d4b400640 1 Processor -- start 2024-01-15T17:54:46.361 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.358+0000 7f8d4b400640 1 -- start start 2024-01-15T17:54:46.363 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4b400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d4409cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.363 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 0x7f8d4415b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.363 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d441e3900 0x7f8d441e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.363 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d441e42c0 con 0x7f8d441e3900 2024-01-15T17:54:46.363 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4b400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4409e140 con 0x7f8d24005180 2024-01-15T17:54:46.363 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4409d220 con 0x7f8d4406b3c0 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4a000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d4409cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d49600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 0x7f8d4415b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d49600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 0x7f8d4415b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:40954/0 (socket says 172.21.15.124:40954) 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4a000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d4409cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37304/0 (socket says 172.21.15.124:37304) 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d49600640 1 -- 172.21.15.124:0/2842966926 learned_addr learned my addr 172.21.15.124:0/2842966926 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d49600640 1 -- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 msgr2=0x7f8d4415b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d49600640 1 -- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 msgr2=0x7f8d4415b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d49600640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 0x7f8d4415b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d49600640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 0x7f8d4415b7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4aa00640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d441e3900 0x7f8d441e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4a000640 1 -- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 msgr2=0x7f8d4415b7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4a000640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 0x7f8d4415b7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4a000640 1 -- 172.21.15.124:0/2842966926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d441e3900 msgr2=0x7f8d441e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.364 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4a000640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d441e3900 0x7f8d441e3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.359+0000 7f8d4a000640 1 -- 172.21.15.124:0/2842966926 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4415da10 con 0x7f8d24005180 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.360+0000 7f8d4a000640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d4409cce0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d34011960 tx=0x7f8d34011e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.360+0000 7f8d42000640 1 -- 172.21.15.124:0/2842966926 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3400fa30 con 0x7f8d24005180 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.360+0000 7f8d42000640 1 -- 172.21.15.124:0/2842966926 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3400fb90 con 0x7f8d24005180 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.360+0000 7f8d42000640 1 -- 172.21.15.124:0/2842966926 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34014440 con 0x7f8d24005180 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.360+0000 7f8d4b400640 1 -- 172.21.15.124:0/2842966926 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4409d4c0 con 0x7f8d24005180 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.360+0000 7f8d4b400640 1 -- 172.21.15.124:0/2842966926 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4409da80 con 0x7f8d24005180 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.360+0000 7f8d42000640 1 -- 172.21.15.124:0/2842966926 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8d340148a0 con 0x7f8d24005180 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.361+0000 7f8d42000640 1 -- 172.21.15.124:0/2842966926 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f8d3401db70 con 0x7f8d24005180 2024-01-15T17:54:46.365 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.361+0000 7f8d4b400640 1 -- 172.21.15.124:0/2842966926 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d44067450 con 0x7f8d24005180 2024-01-15T17:54:46.368 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.366+0000 7f8d42000640 1 -- 172.21.15.124:0/2842966926 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8d34022020 con 0x7f8d24005180 2024-01-15T17:54:46.409 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.406+0000 7f8fdde3d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-15T17:54:46.440 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:46.436+0000 7f627a030040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-15T17:54:46.446 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.444+0000 7f8d4b400640 1 -- 172.21.15.124:0/2842966926 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b107ddc6-f573-40ca-8c09-62c5e8f4bfdc", "id": 1} v 0) v1 -- 0x7f8d440d3b30 con 0x7f8d24005180 2024-01-15T17:54:46.452 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.449+0000 7f8d42000640 1 -- 172.21.15.124:0/2842966926 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b107ddc6-f573-40ca-8c09-62c5e8f4bfdc", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8d3401dd30 con 0x7f8d24005180 2024-01-15T17:54:46.452 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T17:54:46.454 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.451+0000 7f8d4b400640 1 -- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 msgr2=0x7f8d4409cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.454 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.451+0000 7f8d4b400640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d4409cce0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8d34011960 tx=0x7f8d34011e20 comp rx=0 tx=0).stop 2024-01-15T17:54:46.454 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.451+0000 7f8d4b400640 1 -- 172.21.15.124:0/2842966926 shutdown_connections 2024-01-15T17:54:46.454 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.451+0000 7f8d4b400640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d4406b3c0 0x7f8d4415b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.454 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.451+0000 7f8d4b400640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d441e3900 0x7f8d441e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.454 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.451+0000 7f8d4b400640 1 --2- 172.21.15.124:0/2842966926 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d24005180 0x7f8d4409cce0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.455 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.451+0000 7f8d4b400640 1 -- 172.21.15.124:0/2842966926 >> 172.21.15.124:0/2842966926 conn(0x7f8d440d3000 msgr2=0x7f8d4415c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:46.455 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.451+0000 7f8d4b400640 1 -- 172.21.15.124:0/2842966926 shutdown_connections 2024-01-15T17:54:46.455 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.451+0000 7f8d4b400640 1 -- 172.21.15.124:0/2842966926 wait complete. 2024-01-15T17:54:46.462 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.459+0000 7f8fdde3d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-15T17:54:46.471 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 2c4d3745-c37d-438b-8065-3bb12e54e54a 2 2024-01-15T17:54:46.507 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:46.505+0000 7f627a030040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-15T17:54:46.582 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.579+0000 7f8fdde3d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-15T17:54:46.667 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.662+0000 7f8fdde3d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-15T17:54:46.703 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.700+0000 7f588c600640 1 Processor -- start 2024-01-15T17:54:46.703 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.700+0000 7f588c600640 1 -- start start 2024-01-15T17:54:46.705 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.702+0000 7f588c600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5884159040 0x7f58840cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.705 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.702+0000 7f588c600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58840c6060 con 0x7f588415c300 2024-01-15T17:54:46.705 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.702+0000 7f588c600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58840c62d0 con 0x7f58840b34e0 2024-01-15T17:54:46.705 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.702+0000 7f588c600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58840c65b0 con 0x7f5884159040 2024-01-15T17:54:46.705 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.702+0000 7f588a600640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f58840b34e0 0x7f58840d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:49500/0 (socket says 172.21.15.124:49500) 2024-01-15T17:54:46.705 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.702+0000 7f588a600640 1 -- 172.21.15.124:0/2495761291 learned_addr learned my addr 172.21.15.124:0/2495761291 (peer_addr_for_me v1:172.21.15.124:0/0) 2024-01-15T17:54:46.705 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5889c00640 1 --2- 172.21.15.124:0/2495761291 >> v2:172.21.15.26:3300/0 conn(0x7f5884159040 0x7f58840cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.708 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037880871 0 0) 0x7f58840c62d0 con 0x7f58840b34e0 2024-01-15T17:54:46.708 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5864003520 con 0x7f58840b34e0 2024-01-15T17:54:46.708 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001574734 0 0) 0x7f5864003520 con 0x7f58840b34e0 2024-01-15T17:54:46.708 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 >> v2:172.21.15.26:3300/0 conn(0x7f5884159040 msgr2=0x7f58840cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 --2- 172.21.15.124:0/2495761291 >> v2:172.21.15.26:3300/0 conn(0x7f5884159040 0x7f58840cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 >> v1:172.21.15.26:6789/0 conn(0x7f588415c300 legacy=0x7f588415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588415eec0 con 0x7f58840b34e0 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f587c0019f0 con 0x7f58840b34e0 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 >> v1:172.21.15.124:6789/0 conn(0x7f58840b34e0 legacy=0x7f58840d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f5864003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58640047c0 0x7f5864004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5864005180 0x7f586400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840c65b0 con 0x7f58640047c0 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586400c020 con 0x7f5864003970 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.703+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586400c1e0 con 0x7f5864005180 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.704+0000 7f5889c00640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f5864003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.704+0000 7f588a600640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5864005180 0x7f586400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.709 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.704+0000 7f5889200640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58640047c0 0x7f5864004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.704+0000 7f5889c00640 1 -- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5864005180 msgr2=0x7f586400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.704+0000 7f5889c00640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5864005180 0x7f586400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.704+0000 7f5889c00640 1 -- 172.21.15.124:0/2495761291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58640047c0 msgr2=0x7f5864004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.704+0000 7f5889c00640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58640047c0 0x7f5864004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.704+0000 7f5889c00640 1 -- 172.21.15.124:0/2495761291 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5864003520 con 0x7f5864003970 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.704+0000 7f588a600640 1 -- 172.21.15.124:0/2495761291 reap_dead start 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.705+0000 7f5889c00640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f5864003d90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5874009ac0 tx=0x7f58640057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c73b6ebf80dd5eb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.705+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587402c6b0 con 0x7f5864003970 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.705+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587402c810 con 0x7f5864003970 2024-01-15T17:54:46.710 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.705+0000 7f5883e00640 1 -- 172.21.15.124:0/2495761291 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587402cba0 con 0x7f5864003970 2024-01-15T17:54:46.711 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.706+0000 7f588c600640 1 -- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 msgr2=0x7f5864003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.711 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.706+0000 7f588c600640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f5864003d90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5874009ac0 tx=0x7f58640057f0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.711 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.706+0000 7f588c600640 1 -- 172.21.15.124:0/2495761291 shutdown_connections 2024-01-15T17:54:46.711 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.706+0000 7f588c600640 1 --2- 172.21.15.124:0/2495761291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f5864003d90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.712 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.706+0000 7f588c600640 1 -- 172.21.15.124:0/2495761291 >> 172.21.15.124:0/2495761291 conn(0x7f5884066690 msgr2=0x7f588406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:46.712 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.706+0000 7f588c600640 1 -- 172.21.15.124:0/2495761291 shutdown_connections 2024-01-15T17:54:46.712 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.706+0000 7f588c600640 1 -- 172.21.15.124:0/2495761291 wait complete. 2024-01-15T17:54:46.712 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.708+0000 7f588c600640 1 Processor -- start 2024-01-15T17:54:46.712 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.708+0000 7f588c600640 1 -- start start 2024-01-15T17:54:46.714 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588c600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f58841f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.714 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588c600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5884154740 0x7f5884154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.714 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588c600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f588406bfe0 0x7f588406c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588c600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588406cc10 con 0x7f5884154740 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588c600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58840b04c0 con 0x7f5864003970 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588c600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58841550e0 con 0x7f588406bfe0 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f5889c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f58841f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588a600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f588406bfe0 0x7f588406c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f5889c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f58841f2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37326/0 (socket says 172.21.15.124:37326) 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f5889c00640 1 -- 172.21.15.124:0/1829428134 learned_addr learned my addr 172.21.15.124:0/1829428134 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588a600640 1 -- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f588406bfe0 msgr2=0x7f588406c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588a600640 1 -- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f588406bfe0 msgr2=0x7f588406c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588a600640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f588406bfe0 0x7f588406c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f5889200640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5884154740 0x7f5884154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.709+0000 7f588a600640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f588406bfe0 0x7f588406c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f5889c00640 1 -- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f588406bfe0 msgr2=0x7f588406c460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:54:46.715 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f5889c00640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f588406bfe0 0x7f588406c460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f5889c00640 1 -- 172.21.15.124:0/1829428134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5884154740 msgr2=0x7f5884154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f5889c00640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5884154740 0x7f5884154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f5889c00640 1 -- 172.21.15.124:0/1829428134 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588415a7d0 con 0x7f5864003970 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f5889c00640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f58841f2ea0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5874009ac0 tx=0x7f5874004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f5881600640 1 -- 172.21.15.124:0/1829428134 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874003810 con 0x7f5864003970 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f5881600640 1 -- 172.21.15.124:0/1829428134 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5874003970 con 0x7f5864003970 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f5881600640 1 -- 172.21.15.124:0/1829428134 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874003d00 con 0x7f5864003970 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f588c600640 1 -- 172.21.15.124:0/1829428134 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588415b350 con 0x7f5864003970 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.710+0000 7f588c600640 1 -- 172.21.15.124:0/1829428134 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588406c9a0 con 0x7f5864003970 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.711+0000 7f5881600640 1 -- 172.21.15.124:0/1829428134 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5874005c90 con 0x7f5864003970 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.711+0000 7f5881600640 1 -- 172.21.15.124:0/1829428134 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f5874006610 con 0x7f5864003970 2024-01-15T17:54:46.716 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.712+0000 7f588c600640 1 -- 172.21.15.124:0/1829428134 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5884074a50 con 0x7f5864003970 2024-01-15T17:54:46.717 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.714+0000 7f5881600640 1 -- 172.21.15.124:0/1829428134 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5874044020 con 0x7f5864003970 2024-01-15T17:54:46.722 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.719+0000 7f8fdde3d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-15T17:54:46.782 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.779+0000 7f8fdde3d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-15T17:54:46.795 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.792+0000 7f588c600640 1 -- 172.21.15.124:0/1829428134 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2c4d3745-c37d-438b-8065-3bb12e54e54a", "id": 2} v 0) v1 -- 0x7f58840675d0 con 0x7f5864003970 2024-01-15T17:54:46.801 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.798+0000 7f5881600640 1 -- 172.21.15.124:0/1829428134 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2c4d3745-c37d-438b-8065-3bb12e54e54a", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5874039070 con 0x7f5864003970 2024-01-15T17:54:46.801 INFO:teuthology.orchestra.run.smithi124.stdout:2 2024-01-15T17:54:46.803 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.800+0000 7f588c600640 1 -- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 msgr2=0x7f58841f2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:46.803 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.800+0000 7f588c600640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f58841f2ea0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5874009ac0 tx=0x7f5874004730 comp rx=0 tx=0).stop 2024-01-15T17:54:46.803 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.800+0000 7f588c600640 1 -- 172.21.15.124:0/1829428134 shutdown_connections 2024-01-15T17:54:46.803 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.800+0000 7f588c600640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f588406bfe0 0x7f588406c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.803 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.800+0000 7f588c600640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5884154740 0x7f5884154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.803 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.800+0000 7f588c600640 1 --2- 172.21.15.124:0/1829428134 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5864003970 0x7f58841f2ea0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:46.803 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.800+0000 7f588c600640 1 -- 172.21.15.124:0/1829428134 >> 172.21.15.124:0/1829428134 conn(0x7f5884066690 msgr2=0x7f5884156da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:46.803 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.800+0000 7f588c600640 1 -- 172.21.15.124:0/1829428134 shutdown_connections 2024-01-15T17:54:46.804 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.800+0000 7f588c600640 1 -- 172.21.15.124:0/1829428134 wait complete. 2024-01-15T17:54:46.818 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new e0b8df60-d19e-43a8-94bd-3607d215133f 3 2024-01-15T17:54:46.841 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:46.838+0000 7f8fdde3d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-15T17:54:46.997 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.994+0000 7f6081a00640 1 Processor -- start 2024-01-15T17:54:46.997 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.994+0000 7f6081a00640 1 -- start start 2024-01-15T17:54:46.999 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.996+0000 7f6081a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f607c0cfde0 0x7f607c0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:46.999 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.996+0000 7f6081a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c0d1c60 con 0x7f607c15d4b0 2024-01-15T17:54:46.999 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.996+0000 7f6081a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607c0b2730 con 0x7f607c0cf9a0 2024-01-15T17:54:46.999 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.996+0000 7f6081a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607c0d1ec0 con 0x7f607c0cfde0 2024-01-15T17:54:46.999 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.996+0000 7f6081000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f607c0cfde0 0x7f607c0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:46.999 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.996+0000 7f6081000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f607c0cfde0 0x7f607c0b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37332/0 (socket says 172.21.15.124:37332) 2024-01-15T17:54:46.999 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.996+0000 7f6081000640 1 -- 172.21.15.124:0/519241073 learned_addr learned my addr 172.21.15.124:0/519241073 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:47.000 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.996+0000 7f6081000640 1 -- 172.21.15.124:0/519241073 >> v1:172.21.15.124:6789/0 conn(0x7f607c0cf9a0 legacy=0x7f607c0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.000 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.996+0000 7f6081000640 1 -- 172.21.15.124:0/519241073 >> v1:172.21.15.26:6789/0 conn(0x7f607c15d4b0 legacy=0x7f607c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:47.000 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.997+0000 7f6081000640 1 -- 172.21.15.124:0/519241073 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c0b29d0 con 0x7f607c0cfde0 2024-01-15T17:54:47.001 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.998+0000 7f6081000640 1 --2- 172.21.15.124:0/519241073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cfde0 0x7f607c0b38e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6074009e50 tx=0x7f607402bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45aa6ab957e0d288 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:47.001 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.998+0000 7f607b400640 1 -- 172.21.15.124:0/519241073 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607402cb20 con 0x7f607c0cfde0 2024-01-15T17:54:47.001 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.998+0000 7f607b400640 1 -- 172.21.15.124:0/519241073 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607402cc80 con 0x7f607c0cfde0 2024-01-15T17:54:47.001 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.998+0000 7f607b400640 1 -- 172.21.15.124:0/519241073 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60740343f0 con 0x7f607c0cfde0 2024-01-15T17:54:47.002 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.998+0000 7f6081a00640 1 -- 172.21.15.124:0/519241073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cfde0 msgr2=0x7f607c0b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.002 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.998+0000 7f6081a00640 1 --2- 172.21.15.124:0/519241073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cfde0 0x7f607c0b38e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6074009e50 tx=0x7f607402bf80 comp rx=0 tx=0).stop 2024-01-15T17:54:47.002 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.999+0000 7f6081a00640 1 -- 172.21.15.124:0/519241073 shutdown_connections 2024-01-15T17:54:47.002 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.999+0000 7f6081a00640 1 --2- 172.21.15.124:0/519241073 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cfde0 0x7f607c0b38e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.002 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.999+0000 7f6081a00640 1 -- 172.21.15.124:0/519241073 >> 172.21.15.124:0/519241073 conn(0x7f607c0d3000 msgr2=0x7f607c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:47.002 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.999+0000 7f6081a00640 1 -- 172.21.15.124:0/519241073 shutdown_connections 2024-01-15T17:54:47.002 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:46.999+0000 7f6081a00640 1 -- 172.21.15.124:0/519241073 wait complete. 2024-01-15T17:54:47.003 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.000+0000 7f6081a00640 1 Processor -- start 2024-01-15T17:54:47.003 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.000+0000 7f6081a00640 1 -- start start 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.001+0000 7f6081a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cf9a0 0x7f607c1f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.001+0000 7f6081a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607c0cfde0 0x7f607c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.001+0000 7f6081a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c15d4b0 0x7f607c0cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.001+0000 7f6081a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0ca0d0 con 0x7f607c15d4b0 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.001+0000 7f6081a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c1f3ce0 con 0x7f607c0cf9a0 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.001+0000 7f6081a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c0ca2b0 con 0x7f607c0cfde0 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f607be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cf9a0 0x7f607c1f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f607be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cf9a0 0x7f607c1f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37338/0 (socket says 172.21.15.124:37338) 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f6073e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607c0cfde0 0x7f607c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.006 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f607be00640 1 -- 172.21.15.124:0/941007119 learned_addr learned my addr 172.21.15.124:0/941007119 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f6073e00640 1 -- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607c0cfde0 msgr2=0x7f607c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f6073e00640 1 -- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607c0cfde0 msgr2=0x7f607c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f6073e00640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607c0cfde0 0x7f607c15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f6081000640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c15d4b0 0x7f607c0cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f607be00640 1 -- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607c0cfde0 msgr2=0x7f607c15bfc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f6073e00640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607c0cfde0 0x7f607c15bfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f607be00640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607c0cfde0 0x7f607c15bfc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f607be00640 1 -- 172.21.15.124:0/941007119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c15d4b0 msgr2=0x7f607c0cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f607be00640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c15d4b0 0x7f607c0cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f607be00640 1 -- 172.21.15.124:0/941007119 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6074009b30 con 0x7f607c0cf9a0 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f6081000640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c15d4b0 0x7f607c0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f607be00640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cf9a0 0x7f607c1f37a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c430 tx=0x7f606c00c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.002+0000 7f6078c00640 1 -- 172.21.15.124:0/941007119 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c011790 con 0x7f607c0cf9a0 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.003+0000 7f6078c00640 1 -- 172.21.15.124:0/941007119 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606c0118f0 con 0x7f607c0cf9a0 2024-01-15T17:54:47.007 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.003+0000 7f6078c00640 1 -- 172.21.15.124:0/941007119 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c011ce0 con 0x7f607c0cf9a0 2024-01-15T17:54:47.008 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.003+0000 7f6081a00640 1 -- 172.21.15.124:0/941007119 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c0ca590 con 0x7f607c0cf9a0 2024-01-15T17:54:47.008 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.003+0000 7f6081a00640 1 -- 172.21.15.124:0/941007119 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c0cab50 con 0x7f607c0cf9a0 2024-01-15T17:54:47.008 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.003+0000 7f6078c00640 1 -- 172.21.15.124:0/941007119 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f606c01c760 con 0x7f607c0cf9a0 2024-01-15T17:54:47.008 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.003+0000 7f6078c00640 1 -- 172.21.15.124:0/941007119 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f606c011e40 con 0x7f607c0cf9a0 2024-01-15T17:54:47.008 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.004+0000 7f6081a00640 1 -- 172.21.15.124:0/941007119 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607c0d0250 con 0x7f607c0cf9a0 2024-01-15T17:54:47.009 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.006+0000 7f6078c00640 1 -- 172.21.15.124:0/941007119 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f607c0d0250 con 0x7f607c0cf9a0 2024-01-15T17:54:47.086 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.083+0000 7f6081a00640 1 -- 172.21.15.124:0/941007119 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e0b8df60-d19e-43a8-94bd-3607d215133f", "id": 3} v 0) v1 -- 0x7f607c067450 con 0x7f607c0cf9a0 2024-01-15T17:54:47.092 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.089+0000 7f6078c00640 1 -- 172.21.15.124:0/941007119 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e0b8df60-d19e-43a8-94bd-3607d215133f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f606c01ca10 con 0x7f607c0cf9a0 2024-01-15T17:54:47.092 INFO:teuthology.orchestra.run.smithi124.stdout:3 2024-01-15T17:54:47.093 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.090+0000 7f6081a00640 1 -- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cf9a0 msgr2=0x7f607c1f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.094 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.090+0000 7f6081a00640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cf9a0 0x7f607c1f37a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f606c00c430 tx=0x7f606c00c8f0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.094 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.090+0000 7f6081a00640 1 -- 172.21.15.124:0/941007119 shutdown_connections 2024-01-15T17:54:47.094 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.090+0000 7f6081a00640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607c0cfde0 0x7f607c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.094 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.090+0000 7f6081a00640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c15d4b0 0x7f607c0cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.094 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.090+0000 7f6081a00640 1 --2- 172.21.15.124:0/941007119 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0cf9a0 0x7f607c1f37a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.094 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.090+0000 7f6081a00640 1 -- 172.21.15.124:0/941007119 >> 172.21.15.124:0/941007119 conn(0x7f607c0d3000 msgr2=0x7f607c06b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:47.094 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.091+0000 7f6081a00640 1 -- 172.21.15.124:0/941007119 shutdown_connections 2024-01-15T17:54:47.094 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.091+0000 7f6081a00640 1 -- 172.21.15.124:0/941007119 wait complete. 2024-01-15T17:54:47.109 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 592f3f51-ada0-4adb-bfc0-86535d57b38e 4 2024-01-15T17:54:47.192 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.189+0000 7f627a030040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-15T17:54:47.256 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.253+0000 7f627a030040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-15T17:54:47.310 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.306+0000 7f627a030040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-15T17:54:47.314 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:47.311+0000 7f8fdde3d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-15T17:54:47.341 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.338+0000 7f60a2e00640 1 Processor -- start 2024-01-15T17:54:47.341 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.338+0000 7f60a2e00640 1 -- start start 2024-01-15T17:54:47.345 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.339+0000 7f60a2e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f609c15d4b0 0x7f609c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.345 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.339+0000 7f60a2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 0x7f609c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.345 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a2e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609c0d0320 con 0x7f609c0cddf0 2024-01-15T17:54:47.345 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a2e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609c0b2730 con 0x7f609c15d4b0 2024-01-15T17:54:47.345 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0d1ec0 con 0x7f609c0cf9a0 2024-01-15T17:54:47.345 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a1000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 0x7f609c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.345 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a2400640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f609c0cddf0 0x7f609c0ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:49522/0 (socket says 172.21.15.124:49522) 2024-01-15T17:54:47.345 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a1000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 0x7f609c0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:40984/0 (socket says 172.21.15.124:40984) 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a2400640 1 -- 172.21.15.124:0/95723156 learned_addr learned my addr 172.21.15.124:0/95723156 (peer_addr_for_me v1:172.21.15.124:0/0) 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a1a00640 1 --2- 172.21.15.124:0/95723156 >> v2:172.21.15.26:3300/0 conn(0x7f609c15d4b0 0x7f609c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a1000640 1 -- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 msgr2=0x7f609c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a1000640 1 -- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 msgr2=0x7f609c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a1000640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 0x7f609c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f60a1000640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 0x7f609c0cfde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002586346 0 0) 0x7f609c0d0320 con 0x7f609c0cddf0 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.340+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607c003520 con 0x7f609c0cddf0 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3313338286 0 0) 0x7f607c003520 con 0x7f609c0cddf0 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 msgr2=0x7f609c0cfde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 0x7f609c0cfde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 >> v2:172.21.15.26:3300/0 conn(0x7f609c15d4b0 msgr2=0x7f609c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:47.346 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 --2- 172.21.15.124:0/95723156 >> v2:172.21.15.26:3300/0 conn(0x7f609c15d4b0 0x7f609c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c0b29d0 con 0x7f609c0cddf0 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f60940019f0 con 0x7f609c0cddf0 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 >> v1:172.21.15.124:6789/0 conn(0x7f609c0cddf0 legacy=0x7f609c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c003970 0x7f607c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f607c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 0x7f607c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0b2730 con 0x7f607c003970 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0d1ec0 con 0x7f607c0047c0 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607c00b9d0 con 0x7f609c0cf9a0 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f60a1000640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f607c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f60a2400640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 0x7f607c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f60a1a00640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c003970 0x7f607c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f60a1000640 1 -- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 msgr2=0x7f607c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:47.347 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f60a1000640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0cf9a0 0x7f607c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.348 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f60a1000640 1 -- 172.21.15.124:0/95723156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c003970 msgr2=0x7f607c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:47.348 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f60a1000640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607c003970 0x7f607c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.348 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.341+0000 7f60a1000640 1 -- 172.21.15.124:0/95723156 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c003520 con 0x7f607c0047c0 2024-01-15T17:54:47.351 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.342+0000 7f60a1000640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f607c004c40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f608400b9e0 tx=0x7f607c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76d9461989af1206 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:47.351 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.343+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084032620 con 0x7f607c0047c0 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.343+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084032780 con 0x7f607c0047c0 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.343+0000 7f609be00640 1 -- 172.21.15.124:0/95723156 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084032b30 con 0x7f607c0047c0 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.343+0000 7f60a2e00640 1 -- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 msgr2=0x7f607c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.343+0000 7f60a2e00640 1 --2- 172.21.15.124:0/95723156 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f607c004c40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f608400b9e0 tx=0x7f607c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.343+0000 7f60a2400640 1 -- 172.21.15.124:0/95723156 reap_dead start 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.344+0000 7f60a2e00640 1 -- 172.21.15.124:0/95723156 shutdown_connections 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.344+0000 7f60a2e00640 1 -- 172.21.15.124:0/95723156 >> 172.21.15.124:0/95723156 conn(0x7f609c0d3000 msgr2=0x7f609c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.344+0000 7f60a2e00640 1 -- 172.21.15.124:0/95723156 shutdown_connections 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.344+0000 7f60a2e00640 1 -- 172.21.15.124:0/95723156 wait complete. 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.345+0000 7f60a2e00640 1 Processor -- start 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.345+0000 7f60a2e00640 1 -- start start 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.346+0000 7f60a2e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f609c0a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.346+0000 7f60a2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 0x7f609c0b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.352 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.346+0000 7f60a2e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f609c09daa0 0x7f609c09df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.346+0000 7f60a2e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c09d610 con 0x7f609c09daa0 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.346+0000 7f60a2e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0b4f10 con 0x7f607c0047c0 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.346+0000 7f60a2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609c0b8de0 con 0x7f609c0b8960 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f609c0a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 0x7f609c0b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 0x7f609c0b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:41002/0 (socket says 172.21.15.124:41002) 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1000640 1 -- 172.21.15.124:0/1186336366 learned_addr learned my addr 172.21.15.124:0/1186336366 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f609c0a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37354/0 (socket says 172.21.15.124:37354) 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1000640 1 -- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 msgr2=0x7f609c0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1000640 1 -- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 msgr2=0x7f609c0b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1000640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 0x7f609c0b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1000640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 0x7f609c0b49d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a2400640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f609c09daa0 0x7f609c09df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.353 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1a00640 1 -- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 msgr2=0x7f609c0b49d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1a00640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 0x7f609c0b49d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1a00640 1 -- 172.21.15.124:0/1186336366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f609c09daa0 msgr2=0x7f609c09df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1a00640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f609c09daa0 0x7f609c09df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1a00640 1 -- 172.21.15.124:0/1186336366 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609c15da10 con 0x7f607c0047c0 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a2400640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f609c09daa0 0x7f609c09df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a1a00640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f609c0a0200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f608c011590 tx=0x7f608c011a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f6099600640 1 -- 172.21.15.124:0/1186336366 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c004500 con 0x7f607c0047c0 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f6099600640 1 -- 172.21.15.124:0/1186336366 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c003610 con 0x7f607c0047c0 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f6099600640 1 -- 172.21.15.124:0/1186336366 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c003980 con 0x7f607c0047c0 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.347+0000 7f60a2e00640 1 -- 172.21.15.124:0/1186336366 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609c0a0900 con 0x7f607c0047c0 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.348+0000 7f60a2e00640 1 -- 172.21.15.124:0/1186336366 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609c09cfc0 con 0x7f607c0047c0 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.348+0000 7f6099600640 1 -- 172.21.15.124:0/1186336366 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f608c003ae0 con 0x7f607c0047c0 2024-01-15T17:54:47.354 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.348+0000 7f6099600640 1 -- 172.21.15.124:0/1186336366 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f608c003d90 con 0x7f607c0047c0 2024-01-15T17:54:47.355 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.349+0000 7f60a2e00640 1 -- 172.21.15.124:0/1186336366 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609c067450 con 0x7f607c0047c0 2024-01-15T17:54:47.355 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.351+0000 7f6099600640 1 -- 172.21.15.124:0/1186336366 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f608c023060 con 0x7f607c0047c0 2024-01-15T17:54:47.375 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.372+0000 7f627a030040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-15T17:54:47.427 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.424+0000 7f627a030040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-15T17:54:47.432 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.429+0000 7f60a2e00640 1 -- 172.21.15.124:0/1186336366 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "592f3f51-ada0-4adb-bfc0-86535d57b38e", "id": 4} v 0) v1 -- 0x7f609c067c00 con 0x7f607c0047c0 2024-01-15T17:54:47.438 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.435+0000 7f6099600640 1 -- 172.21.15.124:0/1186336366 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "592f3f51-ada0-4adb-bfc0-86535d57b38e", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f608c019070 con 0x7f607c0047c0 2024-01-15T17:54:47.438 INFO:teuthology.orchestra.run.smithi124.stdout:4 2024-01-15T17:54:47.440 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.436+0000 7f60a2e00640 1 -- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 msgr2=0x7f609c0a0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.440 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.436+0000 7f60a2e00640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f609c0a0200 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f608c011590 tx=0x7f608c011a50 comp rx=0 tx=0).stop 2024-01-15T17:54:47.440 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.436+0000 7f60a2e00640 1 -- 172.21.15.124:0/1186336366 shutdown_connections 2024-01-15T17:54:47.440 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.436+0000 7f60a2e00640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f609c0b8960 0x7f609c0b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.440 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.436+0000 7f60a2e00640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f609c09daa0 0x7f609c09df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.440 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.436+0000 7f60a2e00640 1 --2- 172.21.15.124:0/1186336366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f607c0047c0 0x7f609c0a0200 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.440 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.436+0000 7f60a2e00640 1 -- 172.21.15.124:0/1186336366 >> 172.21.15.124:0/1186336366 conn(0x7f609c0d3000 msgr2=0x7f609c0b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:47.440 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.436+0000 7f60a2e00640 1 -- 172.21.15.124:0/1186336366 shutdown_connections 2024-01-15T17:54:47.440 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.437+0000 7f60a2e00640 1 -- 172.21.15.124:0/1186336366 wait complete. 2024-01-15T17:54:47.456 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 9b649ed7-e5e5-4689-9746-d801e518f19b 5 2024-01-15T17:54:47.483 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:47.480+0000 7f8fdde3d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-15T17:54:47.547 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.544+0000 7f627a030040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-15T17:54:47.614 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:47.611+0000 7f8fdde3d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-15T17:54:47.630 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.627+0000 7f627a030040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-15T17:54:47.638 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.635+0000 7f13ecc00640 1 Processor -- start 2024-01-15T17:54:47.638 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.636+0000 7f13ecc00640 1 -- start start 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.637+0000 7f13ecc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f13e80ce8f0 0x7f13e80ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.637+0000 7f13ecc00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f13e80b2a00 0x7f13e80b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.637+0000 7f13ecc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e80d0de0 con 0x7f13e8160f80 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.637+0000 7f13ecc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e80d10c0 con 0x7f13e80ce8f0 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.637+0000 7f13ecc00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e80d13a0 con 0x7f13e80b2a00 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.637+0000 7f13e6800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f13e80b2a00 0x7f13e80b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.637+0000 7f13e6800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f13e80b2a00 0x7f13e80b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37366/0 (socket says 172.21.15.124:37366) 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.637+0000 7f13e6800640 1 -- 172.21.15.124:0/2466363137 learned_addr learned my addr 172.21.15.124:0/2466363137 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.638+0000 7f13dfe00640 1 --2- 172.21.15.124:0/2466363137 >> v2:172.21.15.26:3300/0 conn(0x7f13e80ce8f0 0x7f13e80ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.641 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.638+0000 7f13e6800640 1 -- 172.21.15.124:0/2466363137 >> v2:172.21.15.26:3300/0 conn(0x7f13e80ce8f0 msgr2=0x7f13e80ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:47.642 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.638+0000 7f13e6800640 1 --2- 172.21.15.124:0/2466363137 >> v2:172.21.15.26:3300/0 conn(0x7f13e80ce8f0 0x7f13e80ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.642 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.638+0000 7f13e6800640 1 -- 172.21.15.124:0/2466363137 >> v1:172.21.15.26:6789/0 conn(0x7f13e8160f80 legacy=0x7f13e8161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.642 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.638+0000 7f13e6800640 1 -- 172.21.15.124:0/2466363137 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80d1680 con 0x7f13e80b2a00 2024-01-15T17:54:47.642 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.638+0000 7f13dfe00640 1 --2- 172.21.15.124:0/2466363137 >> v2:172.21.15.26:3300/0 conn(0x7f13e80ce8f0 0x7f13e80ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:54:47.643 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.639+0000 7f13e6800640 1 --2- 172.21.15.124:0/2466363137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80b2a00 0x7f13e80b2e90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f13e80b4cb0 tx=0x7f13e002bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e01851ab3c8859 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:47.643 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.639+0000 7f13e5400640 1 -- 172.21.15.124:0/2466363137 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e002cb20 con 0x7f13e80b2a00 2024-01-15T17:54:47.643 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.639+0000 7f13e5400640 1 -- 172.21.15.124:0/2466363137 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e002cc80 con 0x7f13e80b2a00 2024-01-15T17:54:47.643 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.639+0000 7f13e5400640 1 -- 172.21.15.124:0/2466363137 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e00343f0 con 0x7f13e80b2a00 2024-01-15T17:54:47.643 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.640+0000 7f13ecc00640 1 -- 172.21.15.124:0/2466363137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80b2a00 msgr2=0x7f13e80b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.643 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.640+0000 7f13ecc00640 1 --2- 172.21.15.124:0/2466363137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80b2a00 0x7f13e80b2e90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f13e80b4cb0 tx=0x7f13e002bf80 comp rx=0 tx=0).stop 2024-01-15T17:54:47.643 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.640+0000 7f13ecc00640 1 -- 172.21.15.124:0/2466363137 shutdown_connections 2024-01-15T17:54:47.643 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.640+0000 7f13ecc00640 1 --2- 172.21.15.124:0/2466363137 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80b2a00 0x7f13e80b2e90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.643 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.640+0000 7f13ecc00640 1 --2- 172.21.15.124:0/2466363137 >> v2:172.21.15.26:3300/0 conn(0x7f13e80ce8f0 0x7f13e80ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.644 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.640+0000 7f13ecc00640 1 -- 172.21.15.124:0/2466363137 >> 172.21.15.124:0/2466363137 conn(0x7f13e809c550 msgr2=0x7f13e80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:47.644 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.640+0000 7f13ecc00640 1 -- 172.21.15.124:0/2466363137 shutdown_connections 2024-01-15T17:54:47.644 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.640+0000 7f13ecc00640 1 -- 172.21.15.124:0/2466363137 wait complete. 2024-01-15T17:54:47.644 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.642+0000 7f13ecc00640 1 Processor -- start 2024-01-15T17:54:47.644 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.642+0000 7f13ecc00640 1 -- start start 2024-01-15T17:54:47.647 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13ecc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13e80b2a00 0x7f13e80c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.647 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13ecc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80ce8f0 0x7f13e80cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.647 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13ecc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e8160f80 0x7f13e815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.647 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13ecc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80c7eb0 con 0x7f13e8160f80 2024-01-15T17:54:47.647 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13ecc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e815dc00 con 0x7f13e80ce8f0 2024-01-15T17:54:47.647 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13ecc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80c78f0 con 0x7f13e80b2a00 2024-01-15T17:54:47.647 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80ce8f0 0x7f13e80cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13e5e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13e80b2a00 0x7f13e80c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80ce8f0 0x7f13e80cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37376/0 (socket says 172.21.15.124:37376) 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13dfe00640 1 -- 172.21.15.124:0/963700572 learned_addr learned my addr 172.21.15.124:0/963700572 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13e5e00640 1 -- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13e80b2a00 msgr2=0x7f13e80c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13e5e00640 1 -- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13e80b2a00 msgr2=0x7f13e80c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13e5e00640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13e80b2a00 0x7f13e80c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13e6800640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e8160f80 0x7f13e815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13e5e00640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13e80b2a00 0x7f13e80c8090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13dfe00640 1 -- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13e80b2a00 msgr2=0x7f13e80c8090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13dfe00640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13e80b2a00 0x7f13e80c8090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13dfe00640 1 -- 172.21.15.124:0/963700572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e8160f80 msgr2=0x7f13e815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13dfe00640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e8160f80 0x7f13e815d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.643+0000 7f13dfe00640 1 -- 172.21.15.124:0/963700572 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e0009b30 con 0x7f13e80ce8f0 2024-01-15T17:54:47.648 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.644+0000 7f13dfe00640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80ce8f0 0x7f13e80cd520 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f13d000c7f0 tx=0x7f13d000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:47.649 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.644+0000 7f13de000640 1 -- 172.21.15.124:0/963700572 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0007a30 con 0x7f13e80ce8f0 2024-01-15T17:54:47.649 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.644+0000 7f13de000640 1 -- 172.21.15.124:0/963700572 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d0007b90 con 0x7f13e80ce8f0 2024-01-15T17:54:47.649 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.644+0000 7f13de000640 1 -- 172.21.15.124:0/963700572 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000ce40 con 0x7f13e80ce8f0 2024-01-15T17:54:47.649 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.644+0000 7f13ecc00640 1 -- 172.21.15.124:0/963700572 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e80c7bd0 con 0x7f13e80ce8f0 2024-01-15T17:54:47.649 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.644+0000 7f13ecc00640 1 -- 172.21.15.124:0/963700572 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e80b6240 con 0x7f13e80ce8f0 2024-01-15T17:54:47.649 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.644+0000 7f13de000640 1 -- 172.21.15.124:0/963700572 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f13d001a450 con 0x7f13e80ce8f0 2024-01-15T17:54:47.649 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.644+0000 7f13de000640 1 -- 172.21.15.124:0/963700572 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f13d001ac80 con 0x7f13e80ce8f0 2024-01-15T17:54:47.649 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.645+0000 7f13ecc00640 1 -- 172.21.15.124:0/963700572 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e80b6460 con 0x7f13e80ce8f0 2024-01-15T17:54:47.650 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.647+0000 7f13de000640 1 -- 172.21.15.124:0/963700572 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f13d0039ce0 con 0x7f13e80ce8f0 2024-01-15T17:54:47.671 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:47.668+0000 7f8fdde3d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-15T17:54:47.686 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.683+0000 7f627a030040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-15T17:54:47.728 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.725+0000 7f13ecc00640 1 -- 172.21.15.124:0/963700572 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9b649ed7-e5e5-4689-9746-d801e518f19b", "id": 5} v 0) v1 -- 0x7f13e80675d0 con 0x7f13e80ce8f0 2024-01-15T17:54:47.735 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.732+0000 7f13de000640 1 -- 172.21.15.124:0/963700572 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9b649ed7-e5e5-4689-9746-d801e518f19b", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f13d0028020 con 0x7f13e80ce8f0 2024-01-15T17:54:47.736 INFO:teuthology.orchestra.run.smithi124.stdout:5 2024-01-15T17:54:47.737 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.734+0000 7f13ecc00640 1 -- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80ce8f0 msgr2=0x7f13e80cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.737 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.734+0000 7f13ecc00640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80ce8f0 0x7f13e80cd520 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f13d000c7f0 tx=0x7f13d000cbf0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.737 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.734+0000 7f13ecc00640 1 -- 172.21.15.124:0/963700572 shutdown_connections 2024-01-15T17:54:47.737 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.734+0000 7f13ecc00640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13e8160f80 0x7f13e815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.737 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.734+0000 7f13ecc00640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13e80ce8f0 0x7f13e80cd520 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.737 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.734+0000 7f13ecc00640 1 --2- 172.21.15.124:0/963700572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13e80b2a00 0x7f13e80c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.737 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.734+0000 7f13ecc00640 1 -- 172.21.15.124:0/963700572 >> 172.21.15.124:0/963700572 conn(0x7f13e809c550 msgr2=0x7f13e8161360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:47.737 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.734+0000 7f13ecc00640 1 -- 172.21.15.124:0/963700572 shutdown_connections 2024-01-15T17:54:47.737 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.734+0000 7f13ecc00640 1 -- 172.21.15.124:0/963700572 wait complete. 2024-01-15T17:54:47.750 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.748+0000 7f627a030040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-15T17:54:47.753 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new 2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad 6 2024-01-15T17:54:47.785 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:47.782+0000 7f8fdde3d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-15T17:54:47.808 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:47.805+0000 7f627a030040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-15T17:54:47.902 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:47.899+0000 7f8fdde3d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-15T17:54:47.977 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.975+0000 7f04d7400640 1 Processor -- start 2024-01-15T17:54:47.978 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.975+0000 7f04d7400640 1 -- start start 2024-01-15T17:54:47.979 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.976+0000 7f04d7400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f04d00cfde0 0x7f04d00b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.980 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.976+0000 7f04d7400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d00d1c60 con 0x7f04d00cf9a0 2024-01-15T17:54:47.980 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.976+0000 7f04d7400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d00b2730 con 0x7f04d015d4b0 2024-01-15T17:54:47.980 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.976+0000 7f04d7400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d00d1ec0 con 0x7f04d00cfde0 2024-01-15T17:54:47.980 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.976+0000 7f04d6000640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f04d015d4b0 0x7f04d00d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:49524/0 (socket says 172.21.15.124:49524) 2024-01-15T17:54:47.980 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.977+0000 7f04d6000640 1 -- 172.21.15.124:0/2726606639 learned_addr learned my addr 172.21.15.124:0/2726606639 (peer_addr_for_me v1:172.21.15.124:0/0) 2024-01-15T17:54:47.980 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.977+0000 7f04d6a00640 1 --2- 172.21.15.124:0/2726606639 >> v2:172.21.15.124:3300/0 conn(0x7f04d00cfde0 0x7f04d00b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.980 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.977+0000 7f04d6a00640 1 -- 172.21.15.124:0/2726606639 >> v1:172.21.15.124:6789/0 conn(0x7f04d015d4b0 legacy=0x7f04d00d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.981 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.977+0000 7f04d6a00640 1 -- 172.21.15.124:0/2726606639 >> v1:172.21.15.26:6789/0 conn(0x7f04d00cf9a0 legacy=0x7f04d00b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.981 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.977+0000 7f04d6a00640 1 -- 172.21.15.124:0/2726606639 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d00b29d0 con 0x7f04d00cfde0 2024-01-15T17:54:47.981 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.978+0000 7f04d6a00640 1 --2- 172.21.15.124:0/2726606639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 0x7f04d00b38e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=218ebd49be1ff977 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:47.981 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.979+0000 7f04d4c00640 1 -- 172.21.15.124:0/2726606639 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c802cd00 con 0x7f04d00cfde0 2024-01-15T17:54:47.982 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.979+0000 7f04d4c00640 1 -- 172.21.15.124:0/2726606639 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c802ce60 con 0x7f04d00cfde0 2024-01-15T17:54:47.982 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.979+0000 7f04d4c00640 1 -- 172.21.15.124:0/2726606639 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8034660 con 0x7f04d00cfde0 2024-01-15T17:54:47.982 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.979+0000 7f04d7400640 1 -- 172.21.15.124:0/2726606639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 msgr2=0x7f04d00b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.983 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.979+0000 7f04d7400640 1 --2- 172.21.15.124:0/2726606639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 0x7f04d00b38e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c802bf50 comp rx=0 tx=0).stop 2024-01-15T17:54:47.983 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.980+0000 7f04d7400640 1 -- 172.21.15.124:0/2726606639 shutdown_connections 2024-01-15T17:54:47.983 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.980+0000 7f04d7400640 1 --2- 172.21.15.124:0/2726606639 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 0x7f04d00b38e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.983 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.980+0000 7f04d7400640 1 -- 172.21.15.124:0/2726606639 >> 172.21.15.124:0/2726606639 conn(0x7f04d00d3000 msgr2=0x7f04d00d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:47.983 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.980+0000 7f04d7400640 1 -- 172.21.15.124:0/2726606639 shutdown_connections 2024-01-15T17:54:47.983 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.980+0000 7f04d7400640 1 -- 172.21.15.124:0/2726606639 wait complete. 2024-01-15T17:54:47.984 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.981+0000 7f04d7400640 1 Processor -- start 2024-01-15T17:54:47.984 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.981+0000 7f04d7400640 1 -- start start 2024-01-15T17:54:47.987 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.982+0000 7f04d7400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04d00cf9a0 0x7f04d01f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.987 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.982+0000 7f04d7400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 0x7f04d01f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.987 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.982+0000 7f04d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d015d4b0 0x7f04d00b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:47.987 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.982+0000 7f04d7400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00c9740 con 0x7f04d015d4b0 2024-01-15T17:54:47.987 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.982+0000 7f04d7400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0149bc0 con 0x7f04d00cfde0 2024-01-15T17:54:47.987 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.982+0000 7f04d7400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0149290 con 0x7f04d00cf9a0 2024-01-15T17:54:47.987 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.982+0000 7f04d5600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 0x7f04d01f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.987 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04d00cf9a0 0x7f04d01f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.987 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d5600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 0x7f04d01f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.124:37392/0 (socket says 172.21.15.124:37392) 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d5600640 1 -- 172.21.15.124:0/3064630359 learned_addr learned my addr 172.21.15.124:0/3064630359 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d6000640 1 -- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04d00cf9a0 msgr2=0x7f04d01f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d6000640 1 -- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04d00cf9a0 msgr2=0x7f04d01f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d6000640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04d00cf9a0 0x7f04d01f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d6a00640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d015d4b0 0x7f04d00b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d6000640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04d00cf9a0 0x7f04d01f6690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d5600640 1 -- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04d00cf9a0 msgr2=0x7f04d01f6690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d5600640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04d00cf9a0 0x7f04d01f6690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d5600640 1 -- 172.21.15.124:0/3064630359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d015d4b0 msgr2=0x7f04d00b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d5600640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d015d4b0 0x7f04d00b79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d5600640 1 -- 172.21.15.124:0/3064630359 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c8009a70 con 0x7f04d00cfde0 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.983+0000 7f04d5600640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 0x7f04d01f6bd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c450 tx=0x7f04b800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:47.988 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.984+0000 7f04ce000640 1 -- 172.21.15.124:0/3064630359 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b80052d0 con 0x7f04d00cfde0 2024-01-15T17:54:47.989 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.984+0000 7f04ce000640 1 -- 172.21.15.124:0/3064630359 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b8005430 con 0x7f04d00cfde0 2024-01-15T17:54:47.989 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.984+0000 7f04d7400640 1 -- 172.21.15.124:0/3064630359 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d0149570 con 0x7f04d00cfde0 2024-01-15T17:54:47.989 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.984+0000 7f04ce000640 1 -- 172.21.15.124:0/3064630359 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8012870 con 0x7f04d00cfde0 2024-01-15T17:54:47.989 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.984+0000 7f04d7400640 1 -- 172.21.15.124:0/3064630359 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d014c5e0 con 0x7f04d00cfde0 2024-01-15T17:54:47.989 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.984+0000 7f04ce000640 1 -- 172.21.15.124:0/3064630359 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f04b8012cd0 con 0x7f04d00cfde0 2024-01-15T17:54:47.989 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.984+0000 7f04ce000640 1 -- 172.21.15.124:0/3064630359 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f04b801b490 con 0x7f04d00cfde0 2024-01-15T17:54:47.989 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.985+0000 7f04d7400640 1 -- 172.21.15.124:0/3064630359 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d014c800 con 0x7f04d00cfde0 2024-01-15T17:54:47.990 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:47.987+0000 7f04ce000640 1 -- 172.21.15.124:0/3064630359 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f04b80035f0 con 0x7f04d00cfde0 2024-01-15T17:54:48.067 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.064+0000 7f04d7400640 1 -- 172.21.15.124:0/3064630359 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad", "id": 6} v 0) v1 -- 0x7f04d0067450 con 0x7f04d00cfde0 2024-01-15T17:54:48.070 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:48.067+0000 7f8fdde3d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-15T17:54:48.075 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.072+0000 7f04ce000640 1 -- 172.21.15.124:0/3064630359 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f04b802a0b0 con 0x7f04d00cfde0 2024-01-15T17:54:48.075 INFO:teuthology.orchestra.run.smithi124.stdout:6 2024-01-15T17:54:48.077 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.073+0000 7f04d7400640 1 -- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 msgr2=0x7f04d01f6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.077 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.073+0000 7f04d7400640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 0x7f04d01f6bd0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f04b800c450 tx=0x7f04b800c910 comp rx=0 tx=0).stop 2024-01-15T17:54:48.077 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.073+0000 7f04d7400640 1 -- 172.21.15.124:0/3064630359 shutdown_connections 2024-01-15T17:54:48.077 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.073+0000 7f04d7400640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04d015d4b0 0x7f04d00b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.077 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.073+0000 7f04d7400640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04d00cfde0 0x7f04d01f6bd0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.077 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.073+0000 7f04d7400640 1 --2- 172.21.15.124:0/3064630359 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04d00cf9a0 0x7f04d01f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.077 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.073+0000 7f04d7400640 1 -- 172.21.15.124:0/3064630359 >> 172.21.15.124:0/3064630359 conn(0x7f04d00d3000 msgr2=0x7f04d006b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:48.077 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.074+0000 7f04d7400640 1 -- 172.21.15.124:0/3064630359 shutdown_connections 2024-01-15T17:54:48.077 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.074+0000 7f04d7400640 1 -- 172.21.15.124:0/3064630359 wait complete. 2024-01-15T17:54:48.094 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --cluster ceph osd new e21d396c-465f-43a6-b770-ac1f831974b2 7 2024-01-15T17:54:48.124 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:48.121+0000 7f8fdde3d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-15T17:54:48.279 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.276+0000 7f4c04400640 1 Processor -- start 2024-01-15T17:54:48.280 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.277+0000 7f4c04400640 1 -- start start 2024-01-15T17:54:48.282 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.278+0000 7f4c04400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4bfc0ae3f0 0x7f4bfc071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.282 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.278+0000 7f4c04400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc072640 con 0x7f4bfc072210 2024-01-15T17:54:48.282 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.278+0000 7f4c04400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc0ae8a0 con 0x7f4bfc15b3f0 2024-01-15T17:54:48.282 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.278+0000 7f4c04400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc15fd40 con 0x7f4bfc0ae3f0 2024-01-15T17:54:48.282 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.278+0000 7f4c01a00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f4bfc15b3f0 0x7f4bfc0adce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.124:49528/0 (socket says 172.21.15.124:49528) 2024-01-15T17:54:48.282 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.278+0000 7f4c01a00640 1 -- 172.21.15.124:0/2968023925 learned_addr learned my addr 172.21.15.124:0/2968023925 (peer_addr_for_me v1:172.21.15.124:0/0) 2024-01-15T17:54:48.282 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4c01000640 1 --2- 172.21.15.124:0/2968023925 >> v2:172.21.15.26:3300/0 conn(0x7f4bfc0ae3f0 0x7f4bfc071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.282 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3917935909 0 0) 0x7f4bfc0ae8a0 con 0x7f4bfc15b3f0 2024-01-15T17:54:48.282 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bdc003520 con 0x7f4bfc15b3f0 2024-01-15T17:54:48.283 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:48.279+0000 7f627a030040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-15T17:54:48.285 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3684379762 0 0) 0x7f4bdc003520 con 0x7f4bfc15b3f0 2024-01-15T17:54:48.285 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 >> v2:172.21.15.26:3300/0 conn(0x7f4bfc0ae3f0 msgr2=0x7f4bfc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:48.285 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 --2- 172.21.15.124:0/2968023925 >> v2:172.21.15.26:3300/0 conn(0x7f4bfc0ae3f0 0x7f4bfc071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.285 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 >> v1:172.21.15.26:6789/0 conn(0x7f4bfc072210 legacy=0x7f4bfc070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc15ef80 con 0x7f4bfc15b3f0 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f4bec001670 con 0x7f4bfc15b3f0 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 >> v1:172.21.15.124:6789/0 conn(0x7f4bfc15b3f0 legacy=0x7f4bfc0adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bdc0039c0 0x7f4bdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bdc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bdc005180 0x7f4bdc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc15fd40 con 0x7f4bdc005180 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc00c080 con 0x7f4bdc0039c0 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.279+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc00c240 con 0x7f4bdc0047c0 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.280+0000 7f4c01a00640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bdc0039c0 0x7f4bdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.280+0000 7f4c01000640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bdc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.280+0000 7f4c02400640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bdc005180 0x7f4bdc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.280+0000 7f4c01000640 1 -- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bdc0039c0 msgr2=0x7f4bdc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.280+0000 7f4c01000640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bdc0039c0 0x7f4bdc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.286 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.280+0000 7f4c01000640 1 -- 172.21.15.124:0/2968023925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bdc005180 msgr2=0x7f4bdc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.280+0000 7f4c01000640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bdc005180 0x7f4bdc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.280+0000 7f4c01000640 1 -- 172.21.15.124:0/2968023925 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc003520 con 0x7f4bdc0047c0 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.280+0000 7f4c02400640 1 -- 172.21.15.124:0/2968023925 reap_dead start 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.281+0000 7f4c01000640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bdc004c40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc073ca0 tx=0x7f4bdc0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3420469095711009 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.281+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be402c660 con 0x7f4bdc0047c0 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.281+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be402c7c0 con 0x7f4bdc0047c0 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.281+0000 7f4bfbe00640 1 -- 172.21.15.124:0/2968023925 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be402cb50 con 0x7f4bdc0047c0 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.282+0000 7f4c04400640 1 -- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 msgr2=0x7f4bdc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.282+0000 7f4c04400640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bdc004c40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc073ca0 tx=0x7f4bdc0057f0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.282+0000 7f4c04400640 1 -- 172.21.15.124:0/2968023925 shutdown_connections 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.282+0000 7f4c04400640 1 --2- 172.21.15.124:0/2968023925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bdc004c40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.282+0000 7f4c04400640 1 -- 172.21.15.124:0/2968023925 >> 172.21.15.124:0/2968023925 conn(0x7f4bfc0b0c00 msgr2=0x7f4bfc0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.282+0000 7f4c04400640 1 -- 172.21.15.124:0/2968023925 shutdown_connections 2024-01-15T17:54:48.287 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.282+0000 7f4c04400640 1 -- 172.21.15.124:0/2968023925 wait complete. 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.284+0000 7f4c04400640 1 Processor -- start 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.284+0000 7f4c04400640 1 -- start start 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c04400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bfc15e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c04400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bfc09a0e0 0x7f4bfc09a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c04400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bfc145470 0x7f4bfc145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c04400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc09aaa0 con 0x7f4bfc145470 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c04400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc145e60 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c04400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1f45a0 con 0x7f4bdc0047c0 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c01a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bfc15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c01000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bfc09a0e0 0x7f4bfc09a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c01a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bfc15e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.124:41042/0 (socket says 172.21.15.124:41042) 2024-01-15T17:54:48.290 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c01a00640 1 -- 172.21.15.124:0/3957715468 learned_addr learned my addr 172.21.15.124:0/3957715468 (peer_addr_for_me v2:172.21.15.124:0/0) 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c01a00640 1 -- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 msgr2=0x7f4bfc15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c01a00640 1 -- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 msgr2=0x7f4bfc15e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.285+0000 7f4c01a00640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bfc15e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c01a00640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bfc15e020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c02400640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bfc145470 0x7f4bfc145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c01000640 1 -- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 msgr2=0x7f4bfc15e020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c01000640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bfc15e020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c01000640 1 -- 172.21.15.124:0/3957715468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bfc145470 msgr2=0x7f4bfc145920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c01000640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bfc145470 0x7f4bfc145920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c01000640 1 -- 172.21.15.124:0/3957715468 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc1f4880 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c01000640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bfc09a0e0 0x7f4bfc09a560 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4be4033000 tx=0x7f4be4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4bf9600640 1 -- 172.21.15.124:0/3957715468 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4003810 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4bf9600640 1 -- 172.21.15.124:0/3957715468 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be4003970 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4bf9600640 1 -- 172.21.15.124:0/3957715468 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4003d00 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.291 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c04400640 1 -- 172.21.15.124:0/3957715468 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc1f4b60 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.292 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.286+0000 7f4c04400640 1 -- 172.21.15.124:0/3957715468 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc1f5120 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.292 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.287+0000 7f4bf9600640 1 -- 172.21.15.124:0/3957715468 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4be4047750 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.292 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.287+0000 7f4bf9600640 1 -- 172.21.15.124:0/3957715468 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f4be4034af0 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.292 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.287+0000 7f4c04400640 1 -- 172.21.15.124:0/3957715468 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc0cb730 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.294 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.292+0000 7f4bf9600640 1 -- 172.21.15.124:0/3957715468 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4be404c060 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.296 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:48.293+0000 7f8fdde3d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-15T17:54:48.372 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.369+0000 7f4c04400640 1 -- 172.21.15.124:0/3957715468 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e21d396c-465f-43a6-b770-ac1f831974b2", "id": 7} v 0) v1 -- 0x7f4bfc1f52b0 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.379 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.376+0000 7f4bf9600640 1 -- 172.21.15.124:0/3957715468 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e21d396c-465f-43a6-b770-ac1f831974b2", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4be4042070 con 0x7f4bfc09a0e0 2024-01-15T17:54:48.379 INFO:teuthology.orchestra.run.smithi124.stdout:7 2024-01-15T17:54:48.381 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.378+0000 7f4c04400640 1 -- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bfc09a0e0 msgr2=0x7f4bfc09a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.381 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.378+0000 7f4c04400640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bfc09a0e0 0x7f4bfc09a560 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4be4033000 tx=0x7f4be4004730 comp rx=0 tx=0).stop 2024-01-15T17:54:48.381 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.378+0000 7f4c04400640 1 -- 172.21.15.124:0/3957715468 shutdown_connections 2024-01-15T17:54:48.381 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.378+0000 7f4c04400640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4bfc145470 0x7f4bfc145920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.381 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.378+0000 7f4c04400640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4bfc09a0e0 0x7f4bfc09a560 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.381 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.378+0000 7f4c04400640 1 --2- 172.21.15.124:0/3957715468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4bdc0047c0 0x7f4bfc15e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.382 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.378+0000 7f4c04400640 1 -- 172.21.15.124:0/3957715468 >> 172.21.15.124:0/3957715468 conn(0x7f4bfc0b0c00 msgr2=0x7f4bfc0abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:48.382 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.378+0000 7f4c04400640 1 -- 172.21.15.124:0/3957715468 shutdown_connections 2024-01-15T17:54:48.382 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:54:48.378+0000 7f4c04400640 1 -- 172.21.15.124:0/3957715468 wait complete. 2024-01-15T17:54:48.397 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-15T17:54:48.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-01-15T17:54:48.405 INFO:tasks.ceph.osd.0:Started 2024-01-15T17:54:48.405 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-15T17:54:48.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-01-15T17:54:48.412 INFO:tasks.ceph.osd.1:Started 2024-01-15T17:54:48.412 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-15T17:54:48.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-01-15T17:54:48.418 INFO:tasks.ceph.osd.2:Started 2024-01-15T17:54:48.418 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-15T17:54:48.418 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-01-15T17:54:48.423 INFO:tasks.ceph.osd.3:Started 2024-01-15T17:54:48.424 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-15T17:54:48.424 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-01-15T17:54:48.444 INFO:tasks.ceph.osd.4:Started 2024-01-15T17:54:48.444 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-15T17:54:48.445 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-01-15T17:54:48.451 INFO:tasks.ceph.osd.5:Started 2024-01-15T17:54:48.451 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-15T17:54:48.451 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-01-15T17:54:48.457 INFO:tasks.ceph.osd.6:Started 2024-01-15T17:54:48.457 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-15T17:54:48.457 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-01-15T17:54:48.459 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:48.457+0000 7f627a030040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-15T17:54:48.462 INFO:tasks.ceph.osd.7:Started 2024-01-15T17:54:48.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:54:48.695 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:48.692+0000 7f627a030040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-15T17:54:48.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.703+0000 7fe933000640 1 Processor -- start 2024-01-15T17:54:48.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.703+0000 7fe933000640 1 -- start start 2024-01-15T17:54:48.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.704+0000 7fe933000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.704+0000 7fe933000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe92c0d0f40 0x7fe92c0d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.704+0000 7fe933000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c0d0a10 con 0x7fe92c160920 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.704+0000 7fe933000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0cf080 con 0x7fe92c0d0f40 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.704+0000 7fe933000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0c6be0 con 0x7fe92c0d0100 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe932600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe92c0d0f40 0x7fe92c0d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe932600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe92c0d0f40 0x7fe92c0d13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39904/0 (socket says 172.21.15.26:39904) 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe932600640 1 -- 172.21.15.26:0/1079242196 learned_addr learned my addr 172.21.15.26:0/1079242196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe931200640 1 --2- 172.21.15.26:0/1079242196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe931200640 1 -- 172.21.15.26:0/1079242196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 msgr2=0x7fe92c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe931200640 1 -- 172.21.15.26:0/1079242196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 msgr2=0x7fe92c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-15T17:54:48.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe931200640 1 --2- 172.21.15.26:0/1079242196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe931200640 1 --2- 172.21.15.26:0/1079242196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c0b3f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe932600640 1 -- 172.21.15.26:0/1079242196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 msgr2=0x7fe92c0b3f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe932600640 1 --2- 172.21.15.26:0/1079242196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c0b3f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe932600640 1 -- 172.21.15.26:0/1079242196 >> v1:172.21.15.26:6789/0 conn(0x7fe92c160920 legacy=0x7fe92c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe932600640 1 -- 172.21.15.26:0/1079242196 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0ce7e0 con 0x7fe92c0d0f40 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.705+0000 7fe92be00640 1 -- 172.21.15.26:0/1079242196 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575221487 0 0) 0x7fe92c0d0a10 con 0x7fe92c160920 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.706+0000 7fe932600640 1 --2- 172.21.15.26:0/1079242196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c0d0f40 0x7fe92c0d13f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe924009e50 tx=0x7fe92402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a69ac79f5d00f13 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe92be00640 1 -- 172.21.15.26:0/1079242196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92402cb00 con 0x7fe92c0d0f40 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe92be00640 1 -- 172.21.15.26:0/1079242196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92402cc60 con 0x7fe92c0d0f40 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe92be00640 1 -- 172.21.15.26:0/1079242196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9240343f0 con 0x7fe92c0d0f40 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe933000640 1 -- 172.21.15.26:0/1079242196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c0d0f40 msgr2=0x7fe92c0d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe933000640 1 --2- 172.21.15.26:0/1079242196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c0d0f40 0x7fe92c0d13f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe924009e50 tx=0x7fe92402bf60 comp rx=0 tx=0).stop 2024-01-15T17:54:48.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe933000640 1 -- 172.21.15.26:0/1079242196 shutdown_connections 2024-01-15T17:54:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe933000640 1 --2- 172.21.15.26:0/1079242196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c0d0f40 0x7fe92c0d13f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe933000640 1 --2- 172.21.15.26:0/1079242196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe933000640 1 -- 172.21.15.26:0/1079242196 >> 172.21.15.26:0/1079242196 conn(0x7fe92c0d3090 msgr2=0x7fe92c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe933000640 1 -- 172.21.15.26:0/1079242196 shutdown_connections 2024-01-15T17:54:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.707+0000 7fe933000640 1 -- 172.21.15.26:0/1079242196 wait complete. 2024-01-15T17:54:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.709+0000 7fe933000640 1 Processor -- start 2024-01-15T17:54:48.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.709+0000 7fe933000640 1 -- start start 2024-01-15T17:54:48.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe933000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c1f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe933000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe92c0d0f40 0x7fe92c15bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe933000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c160920 0x7fe92c1f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe933000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b5ff0 con 0x7fe92c160920 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe933000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b5c50 con 0x7fe92c0d0f40 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe933000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c1f3a30 con 0x7fe92c0d0100 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe932600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c160920 0x7fe92c1f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe932600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c160920 0x7fe92c1f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39920/0 (socket says 172.21.15.26:39920) 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe932600640 1 -- 172.21.15.26:0/981461425 learned_addr learned my addr 172.21.15.26:0/981461425 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe931200640 1 --2- 172.21.15.26:0/981461425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe92c0d0f40 0x7fe92c15bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.710+0000 7fe931c00640 1 --2- 172.21.15.26:0/981461425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c1f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe932600640 1 -- 172.21.15.26:0/981461425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 msgr2=0x7fe92c1f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe932600640 1 --2- 172.21.15.26:0/981461425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c1f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe932600640 1 -- 172.21.15.26:0/981461425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe92c0d0f40 msgr2=0x7fe92c15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe932600640 1 --2- 172.21.15.26:0/981461425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe92c0d0f40 0x7fe92c15bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe932600640 1 -- 172.21.15.26:0/981461425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe924009b30 con 0x7fe92c160920 2024-01-15T17:54:48.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe932600640 1 --2- 172.21.15.26:0/981461425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c160920 0x7fe92c1f3170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe924036040 tx=0x7fe924004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe929600640 1 -- 172.21.15.26:0/981461425 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924009510 con 0x7fe92c160920 2024-01-15T17:54:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe929600640 1 -- 172.21.15.26:0/981461425 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92403e450 con 0x7fe92c160920 2024-01-15T17:54:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe929600640 1 -- 172.21.15.26:0/981461425 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92403e7e0 con 0x7fe92c160920 2024-01-15T17:54:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe933000640 1 -- 172.21.15.26:0/981461425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c1f3d10 con 0x7fe92c160920 2024-01-15T17:54:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.711+0000 7fe933000640 1 -- 172.21.15.26:0/981461425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c15c210 con 0x7fe92c160920 2024-01-15T17:54:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.712+0000 7fe929600640 1 -- 172.21.15.26:0/981461425 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe92403ec40 con 0x7fe92c160920 2024-01-15T17:54:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.712+0000 7fe929600640 1 -- 172.21.15.26:0/981461425 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe924048b30 con 0x7fe92c160920 2024-01-15T17:54:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.712+0000 7fe933000640 1 -- 172.21.15.26:0/981461425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c0b6130 con 0x7fe92c160920 2024-01-15T17:54:48.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.715+0000 7fe929600640 1 -- 172.21.15.26:0/981461425 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fe92c0b6130 con 0x7fe92c160920 2024-01-15T17:54:48.753 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:48.750+0000 7f627a030040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-15T17:54:48.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.780+0000 7fe933000640 1 -- 172.21.15.26:0/981461425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe92c067f30 con 0x7fe92c160920 2024-01-15T17:54:48.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.780+0000 7fe929600640 1 -- 172.21.15.26:0/981461425 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe924043020 con 0x7fe92c160920 2024-01-15T17:54:48.784 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:54:48.784 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":10,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:54:48.372160+0000","last_up_change":"0.000000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:54:48.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.782+0000 7fe933000640 1 -- 172.21.15.26:0/981461425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c160920 msgr2=0x7fe92c1f3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:48.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.782+0000 7fe933000640 1 --2- 172.21.15.26:0/981461425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c160920 0x7fe92c1f3170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe924036040 tx=0x7fe924004a40 comp rx=0 tx=0).stop 2024-01-15T17:54:48.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.782+0000 7fe933000640 1 -- 172.21.15.26:0/981461425 shutdown_connections 2024-01-15T17:54:48.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.782+0000 7fe933000640 1 --2- 172.21.15.26:0/981461425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe92c160920 0x7fe92c1f3170 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.782+0000 7fe933000640 1 --2- 172.21.15.26:0/981461425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe92c0d0f40 0x7fe92c15bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.782+0000 7fe933000640 1 --2- 172.21.15.26:0/981461425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe92c0d0100 0x7fe92c1f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:48.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.782+0000 7fe933000640 1 -- 172.21.15.26:0/981461425 >> 172.21.15.26:0/981461425 conn(0x7fe92c0d3090 msgr2=0x7fe92c06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:48.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.782+0000 7fe933000640 1 -- 172.21.15.26:0/981461425 shutdown_connections 2024-01-15T17:54:48.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:48.782+0000 7fe933000640 1 -- 172.21.15.26:0/981461425 wait complete. 2024-01-15T17:54:48.801 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-15T17:54:48.801 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-15T17:54:48.870 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:48.867+0000 7f627a030040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-15T17:54:48.987 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:48.984+0000 7f627a030040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-15T17:54:49.102 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-01-15T17:54:49.099+0000 7fb55620c500 -1 Falling back to public interface 2024-01-15T17:54:49.118 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-01-15T17:54:49.115+0000 7fcd6c3e2500 -1 Falling back to public interface 2024-01-15T17:54:49.121 INFO:tasks.ceph.osd.4.smithi124.stderr:2024-01-15T17:54:49.118+0000 7f0cad4af500 -1 Falling back to public interface 2024-01-15T17:54:49.123 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-01-15T17:54:49.120+0000 7f26e84fa500 -1 Falling back to public interface 2024-01-15T17:54:49.144 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-01-15T17:54:49.141+0000 7f90973e0500 -1 Falling back to public interface 2024-01-15T17:54:49.145 INFO:tasks.ceph.osd.7.smithi124.stderr:2024-01-15T17:54:49.142+0000 7f570ce9a500 -1 Falling back to public interface 2024-01-15T17:54:49.157 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:49.154+0000 7f627a030040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-15T17:54:49.162 INFO:tasks.ceph.osd.6.smithi124.stderr:2024-01-15T17:54:49.159+0000 7fdc73838500 -1 Falling back to public interface 2024-01-15T17:54:49.178 INFO:tasks.ceph.osd.5.smithi124.stderr:2024-01-15T17:54:49.175+0000 7fddb3012500 -1 Falling back to public interface 2024-01-15T17:54:49.209 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:49.206+0000 7f627a030040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-15T17:54:49.378 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:49.374+0000 7f627a030040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-15T17:54:49.396 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:49.393+0000 7f8fdde3d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-15T17:54:49.607 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-15T17:54:49.701 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:49.698+0000 7f8fdde3d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-15T17:54:49.762 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:49.759+0000 7f8fdde3d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-15T17:54:49.766 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.763+0000 7f5ef1e00640 1 Processor -- start 2024-01-15T17:54:49.767 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.764+0000 7f5ef1e00640 1 -- start start 2024-01-15T17:54:49.768 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.765+0000 7f5ef1e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5eec144a00 0x7f5eec0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:49.768 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.765+0000 7f5ef1e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec070b30 con 0x7f5eec06c8f0 2024-01-15T17:54:49.768 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.765+0000 7f5ef1e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eec070520 con 0x7f5eec0cccc0 2024-01-15T17:54:49.768 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.765+0000 7f5ef1e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eec0cd470 con 0x7f5eec144a00 2024-01-15T17:54:49.768 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.765+0000 7f5eeb400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5eec06c8f0 0x7f5eec070fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55446/0 (socket says 172.21.15.26:55446) 2024-01-15T17:54:49.768 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.765+0000 7f5eeb400640 1 -- 172.21.15.26:0/1059537972 learned_addr learned my addr 172.21.15.26:0/1059537972 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:54:49.768 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.765+0000 7f5eeaa00640 1 --2- 172.21.15.26:0/1059537972 >> v2:172.21.15.124:3300/0 conn(0x7f5eec144a00 0x7f5eec0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:49.769 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.766+0000 7f5eea000640 1 -- 172.21.15.26:0/1059537972 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054484965 0 0) 0x7f5eec070b30 con 0x7f5eec06c8f0 2024-01-15T17:54:49.769 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.766+0000 7f5eea000640 1 -- 172.21.15.26:0/1059537972 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ecc003520 con 0x7f5eec06c8f0 2024-01-15T17:54:49.769 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.766+0000 7f5eeaa00640 1 -- 172.21.15.26:0/1059537972 >> v1:172.21.15.124:6789/0 conn(0x7f5eec0cccc0 legacy=0x7f5eec06c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:49.769 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.766+0000 7f5eeaa00640 1 -- 172.21.15.26:0/1059537972 >> v1:172.21.15.26:6789/0 conn(0x7f5eec06c8f0 legacy=0x7f5eec070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:49.769 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.766+0000 7f5eeaa00640 1 -- 172.21.15.26:0/1059537972 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eec0cd190 con 0x7f5eec144a00 2024-01-15T17:54:49.770 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.767+0000 7f5eeaa00640 1 --2- 172.21.15.26:0/1059537972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5eec144a00 0x7f5eec0cc780 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0097a0 tx=0x7f5edc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72495ef214cd056d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:49.770 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.767+0000 7f5eea000640 1 -- 172.21.15.26:0/1059537972 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc030610 con 0x7f5eec144a00 2024-01-15T17:54:49.770 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.767+0000 7f5eea000640 1 -- 172.21.15.26:0/1059537972 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc030770 con 0x7f5eec144a00 2024-01-15T17:54:49.770 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.767+0000 7f5eea000640 1 -- 172.21.15.26:0/1059537972 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc030b00 con 0x7f5eec144a00 2024-01-15T17:54:49.771 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.768+0000 7f5ef1e00640 1 -- 172.21.15.26:0/1059537972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5eec144a00 msgr2=0x7f5eec0cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:49.771 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.768+0000 7f5ef1e00640 1 --2- 172.21.15.26:0/1059537972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5eec144a00 0x7f5eec0cc780 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0097a0 tx=0x7f5edc02b960 comp rx=0 tx=0).stop 2024-01-15T17:54:49.771 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.768+0000 7f5ef1e00640 1 -- 172.21.15.26:0/1059537972 shutdown_connections 2024-01-15T17:54:49.771 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.768+0000 7f5ef1e00640 1 --2- 172.21.15.26:0/1059537972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5eec144a00 0x7f5eec0cc780 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:49.771 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.768+0000 7f5ef1e00640 1 -- 172.21.15.26:0/1059537972 >> 172.21.15.26:0/1059537972 conn(0x7f5eec06dd50 msgr2=0x7f5eec06a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:49.772 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.768+0000 7f5ef1e00640 1 -- 172.21.15.26:0/1059537972 shutdown_connections 2024-01-15T17:54:49.772 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.768+0000 7f5ef1e00640 1 -- 172.21.15.26:0/1059537972 wait complete. 2024-01-15T17:54:49.773 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.770+0000 7f5ef1e00640 1 Processor -- start 2024-01-15T17:54:49.773 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.770+0000 7f5ef1e00640 1 -- start start 2024-01-15T17:54:49.775 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5ef1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5eec06c8f0 0x7f5eec13b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:49.775 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5ef1e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5eec0cccc0 0x7f5eec15d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:49.775 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5ef1e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5eec144a00 0x7f5eec13cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5ef1e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec13b760 con 0x7f5eec06c8f0 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5ef1e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec13dbc0 con 0x7f5eec0cccc0 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5ef1e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec13dea0 con 0x7f5eec144a00 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5eeaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5eec06c8f0 0x7f5eec13b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5eeaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5eec06c8f0 0x7f5eec13b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39936/0 (socket says 172.21.15.26:39936) 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5eeaa00640 1 -- 172.21.15.26:0/953732127 learned_addr learned my addr 172.21.15.26:0/953732127 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5ee1e00640 1 --2- 172.21.15.26:0/953732127 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5eec0cccc0 0x7f5eec15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5eeaa00640 1 -- 172.21.15.26:0/953732127 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5eec144a00 msgr2=0x7f5eec13cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5eeaa00640 1 --2- 172.21.15.26:0/953732127 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5eec144a00 0x7f5eec13cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5eeaa00640 1 -- 172.21.15.26:0/953732127 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5eec0cccc0 msgr2=0x7f5eec15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5eeaa00640 1 --2- 172.21.15.26:0/953732127 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5eec0cccc0 0x7f5eec15d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.771+0000 7f5eeaa00640 1 -- 172.21.15.26:0/953732127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5edc009400 con 0x7f5eec06c8f0 2024-01-15T17:54:49.776 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.772+0000 7f5eeaa00640 1 --2- 172.21.15.26:0/953732127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5eec06c8f0 0x7f5eec13b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5edc02bf70 tx=0x7f5edc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:49.777 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.772+0000 7f5ee3e00640 1 -- 172.21.15.26:0/953732127 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc03e070 con 0x7f5eec06c8f0 2024-01-15T17:54:49.777 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.772+0000 7f5ee3e00640 1 -- 172.21.15.26:0/953732127 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc030e10 con 0x7f5eec06c8f0 2024-01-15T17:54:49.777 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.772+0000 7f5ee3e00640 1 -- 172.21.15.26:0/953732127 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc039c30 con 0x7f5eec06c8f0 2024-01-15T17:54:49.777 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.772+0000 7f5ef1e00640 1 -- 172.21.15.26:0/953732127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec15d640 con 0x7f5eec06c8f0 2024-01-15T17:54:49.777 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.772+0000 7f5ef1e00640 1 -- 172.21.15.26:0/953732127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec15dbc0 con 0x7f5eec06c8f0 2024-01-15T17:54:49.777 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.772+0000 7f5ee3e00640 1 -- 172.21.15.26:0/953732127 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5edc04b700 con 0x7f5eec06c8f0 2024-01-15T17:54:49.777 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.773+0000 7f5ee3e00640 1 -- 172.21.15.26:0/953732127 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5edc042b60 con 0x7f5eec06c8f0 2024-01-15T17:54:49.777 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.773+0000 7f5ef1e00640 1 -- 172.21.15.26:0/953732127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eec0675d0 con 0x7f5eec06c8f0 2024-01-15T17:54:49.778 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.776+0000 7f5ee3e00640 1 -- 172.21.15.26:0/953732127 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5edc034070 con 0x7f5eec06c8f0 2024-01-15T17:54:49.846 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.843+0000 7f5ef1e00640 1 -- 172.21.15.26:0/953732127 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5eec06d100 con 0x7f5eec06c8f0 2024-01-15T17:54:49.847 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.844+0000 7f5ee3e00640 1 -- 172.21.15.26:0/953732127 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5edc042dc0 con 0x7f5eec06c8f0 2024-01-15T17:54:49.847 INFO:teuthology.misc.health.smithi026.stdout: 2024-01-15T17:54:49.847 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":10,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:54:48.372160+0000","last_up_change":"0.000000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:54:49.848 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.845+0000 7f5ef1e00640 1 -- 172.21.15.26:0/953732127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5eec06c8f0 msgr2=0x7f5eec13b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:49.848 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.845+0000 7f5ef1e00640 1 --2- 172.21.15.26:0/953732127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5eec06c8f0 0x7f5eec13b220 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5edc02bf70 tx=0x7f5edc0046c0 comp rx=0 tx=0).stop 2024-01-15T17:54:49.849 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.845+0000 7f5ef1e00640 1 -- 172.21.15.26:0/953732127 shutdown_connections 2024-01-15T17:54:49.849 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.845+0000 7f5ef1e00640 1 --2- 172.21.15.26:0/953732127 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5eec144a00 0x7f5eec13cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:49.849 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.845+0000 7f5ef1e00640 1 --2- 172.21.15.26:0/953732127 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5eec0cccc0 0x7f5eec15d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:49.849 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.845+0000 7f5ef1e00640 1 --2- 172.21.15.26:0/953732127 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5eec06c8f0 0x7f5eec13b220 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:49.849 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.845+0000 7f5ef1e00640 1 -- 172.21.15.26:0/953732127 >> 172.21.15.26:0/953732127 conn(0x7f5eec06dd50 msgr2=0x7f5eec070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:49.849 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.846+0000 7f5ef1e00640 1 -- 172.21.15.26:0/953732127 shutdown_connections 2024-01-15T17:54:49.849 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:49.846+0000 7f5ef1e00640 1 -- 172.21.15.26:0/953732127 wait complete. 2024-01-15T17:54:49.860 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-15T17:54:49.886 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:49.883+0000 7f8fdde3d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-15T17:54:50.460 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:50.455+0000 7f627a030040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-15T17:54:50.778 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:50.775+0000 7f627a030040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-15T17:54:50.841 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:50.838+0000 7f627a030040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-15T17:54:50.923 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:50.919+0000 7f8f7ee00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-15T17:54:50.924 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:54:50.920+0000 7f8f7ee00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-15T17:54:50.961 INFO:tasks.ceph.mgr.y.smithi026.stderr:2024-01-15T17:54:50.958+0000 7f627a030040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-15T17:54:52.678 INFO:tasks.ceph.osd.7.smithi124.stderr:2024-01-15T17:54:52.674+0000 7f570ce9a500 -1 osd.7 0 log_to_monitors true 2024-01-15T17:54:52.696 INFO:tasks.ceph.osd.6.smithi124.stderr:2024-01-15T17:54:52.693+0000 7fdc73838500 -1 osd.6 0 log_to_monitors true 2024-01-15T17:54:52.710 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-01-15T17:54:52.707+0000 7f26e84fa500 -1 osd.2 0 log_to_monitors true 2024-01-15T17:54:52.714 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-01-15T17:54:52.711+0000 7f90973e0500 -1 osd.1 0 log_to_monitors true 2024-01-15T17:54:52.741 INFO:tasks.ceph.osd.5.smithi124.stderr:2024-01-15T17:54:52.737+0000 7fddb3012500 -1 osd.5 0 log_to_monitors true 2024-01-15T17:54:52.869 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-01-15T17:54:52.866+0000 7fb55620c500 -1 osd.0 0 log_to_monitors true 2024-01-15T17:54:52.875 INFO:tasks.ceph.osd.4.smithi124.stderr:2024-01-15T17:54:52.871+0000 7f0cad4af500 -1 osd.4 0 log_to_monitors true 2024-01-15T17:54:52.924 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-01-15T17:54:52.921+0000 7fcd6c3e2500 -1 osd.3 0 log_to_monitors true 2024-01-15T17:54:55.451 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-01-15T17:54:55.449+0000 7f26e2a00640 -1 osd.2 0 waiting for initial osdmap 2024-01-15T17:54:55.458 INFO:tasks.ceph.osd.2.smithi026.stderr:2024-01-15T17:54:55.456+0000 7f26d9400640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T17:54:55.480 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-01-15T17:54:55.477+0000 7fb550000640 -1 osd.0 0 waiting for initial osdmap 2024-01-15T17:54:55.487 INFO:tasks.ceph.osd.0.smithi026.stderr:2024-01-15T17:54:55.484+0000 7fb546a00640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T17:54:55.487 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-01-15T17:54:55.485+0000 7f9093600640 -1 osd.1 0 waiting for initial osdmap 2024-01-15T17:54:55.493 INFO:tasks.ceph.osd.1.smithi026.stderr:2024-01-15T17:54:55.491+0000 7f9088200640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T17:54:55.538 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:54:55.543 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:54:56.235 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-01-15T17:54:56.232+0000 7fcd66800640 -1 osd.3 0 waiting for initial osdmap 2024-01-15T17:54:56.250 INFO:tasks.ceph.osd.3.smithi026.stderr:2024-01-15T17:54:56.247+0000 7fcd5d200640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T17:54:56.252 INFO:tasks.ceph.osd.4.smithi124.stderr:2024-01-15T17:54:56.248+0000 7f0ca7a00640 -1 osd.4 0 waiting for initial osdmap 2024-01-15T17:54:56.255 INFO:tasks.ceph.osd.7.smithi124.stderr:2024-01-15T17:54:56.252+0000 7f5709200640 -1 osd.7 0 waiting for initial osdmap 2024-01-15T17:54:56.261 INFO:tasks.ceph.osd.4.smithi124.stderr:2024-01-15T17:54:56.257+0000 7f0c9e400640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T17:54:56.265 INFO:tasks.ceph.osd.5.smithi124.stderr:2024-01-15T17:54:56.261+0000 7fddace00640 -1 osd.5 0 waiting for initial osdmap 2024-01-15T17:54:56.265 INFO:tasks.ceph.osd.7.smithi124.stderr:2024-01-15T17:54:56.261+0000 7f56fde00640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T17:54:56.282 INFO:tasks.ceph.osd.5.smithi124.stderr:2024-01-15T17:54:56.278+0000 7fdda3800640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T17:54:56.344 INFO:tasks.ceph.osd.6.smithi124.stderr:2024-01-15T17:54:56.340+0000 7fdc6dc00640 -1 osd.6 0 waiting for initial osdmap 2024-01-15T17:54:56.357 INFO:tasks.ceph.osd.6.smithi124.stderr:2024-01-15T17:54:56.354+0000 7fdc64600640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-15T17:54:56.667 DEBUG:teuthology.orchestra.run.smithi026:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-15T17:54:56.864 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.861+0000 7fc827200640 1 Processor -- start 2024-01-15T17:54:56.864 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.861+0000 7fc827200640 1 -- start start 2024-01-15T17:54:56.866 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.863+0000 7fc827200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc8280d1c60 0x7fc8280d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:56.866 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.863+0000 7fc827200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280b21a0 0x7fc8280b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:56.866 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.863+0000 7fc827200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8280afd50 con 0x7fc82815d5d0 2024-01-15T17:54:56.866 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.863+0000 7fc827200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8280d11e0 con 0x7fc8280d1c60 2024-01-15T17:54:56.866 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.863+0000 7fc827200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280cef90 con 0x7fc8280b21a0 2024-01-15T17:54:56.866 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.863+0000 7fc825e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc82815d5d0 0x7fc8280d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48920/0 (socket says 172.21.15.26:48920) 2024-01-15T17:54:56.867 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.863+0000 7fc825e00640 1 -- 172.21.15.26:0/121947559 learned_addr learned my addr 172.21.15.26:0/121947559 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:54:56.868 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc825400640 1 --2- 172.21.15.26:0/121947559 >> v2:172.21.15.124:3300/0 conn(0x7fc8280d1c60 0x7fc8280d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc826800640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280b21a0 0x7fc8280b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978196370 0 0) 0x7fc8280afd50 con 0x7fc82815d5d0 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc808003520 con 0x7fc82815d5d0 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981446214 0 0) 0x7fc808003520 con 0x7fc82815d5d0 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280b21a0 msgr2=0x7fc8280b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280b21a0 0x7fc8280b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 >> v2:172.21.15.124:3300/0 conn(0x7fc8280d1c60 msgr2=0x7fc8280d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 --2- 172.21.15.26:0/121947559 >> v2:172.21.15.124:3300/0 conn(0x7fc8280d1c60 0x7fc8280d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8280cf270 con 0x7fc82815d5d0 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fc818001670 con 0x7fc82815d5d0 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 >> v1:172.21.15.26:6789/0 conn(0x7fc82815d5d0 legacy=0x7fc8280d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc808003970 0x7fc808003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:56.869 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280b21a0 0x7fc8080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc808004d00 0x7fc808005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280d11e0 con 0x7fc808004d00 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280cef90 con 0x7fc808003970 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.864+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80800b980 con 0x7fc8280b21a0 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.865+0000 7fc825400640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280b21a0 0x7fc8080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.865+0000 7fc825e00640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc808003970 0x7fc808003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.865+0000 7fc826800640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc808004d00 0x7fc808005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.865+0000 7fc825400640 1 -- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc808003970 msgr2=0x7fc808003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.865+0000 7fc825400640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc808003970 0x7fc808003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.865+0000 7fc825400640 1 -- 172.21.15.26:0/121947559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc808004d00 msgr2=0x7fc808005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.865+0000 7fc825400640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc808004d00 0x7fc808005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.865+0000 7fc825400640 1 -- 172.21.15.26:0/121947559 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808003520 con 0x7fc8280b21a0 2024-01-15T17:54:56.870 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.865+0000 7fc826800640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc808004d00 0x7fc808005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.867+0000 7fc825400640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280b21a0 0x7fc8080047c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc810009ac0 tx=0x7fc8080057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc7dc361778fe36 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.867+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81002c710 con 0x7fc8280b21a0 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.867+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81002c870 con 0x7fc8280b21a0 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.867+0000 7fc824a00640 1 -- 172.21.15.26:0/121947559 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81002cc00 con 0x7fc8280b21a0 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.868+0000 7fc827200640 1 -- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280b21a0 msgr2=0x7fc8080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.868+0000 7fc827200640 1 --2- 172.21.15.26:0/121947559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280b21a0 0x7fc8080047c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc810009ac0 tx=0x7fc8080057f0 comp rx=0 tx=0).stop 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.868+0000 7fc826800640 1 -- 172.21.15.26:0/121947559 reap_dead start 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.868+0000 7fc827200640 1 -- 172.21.15.26:0/121947559 shutdown_connections 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.868+0000 7fc827200640 1 -- 172.21.15.26:0/121947559 >> 172.21.15.26:0/121947559 conn(0x7fc8280d3090 msgr2=0x7fc8280d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.868+0000 7fc827200640 1 -- 172.21.15.26:0/121947559 shutdown_connections 2024-01-15T17:54:56.872 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.868+0000 7fc827200640 1 -- 172.21.15.26:0/121947559 wait complete. 2024-01-15T17:54:56.873 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.870+0000 7fc827200640 1 Processor -- start 2024-01-15T17:54:56.873 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.870+0000 7fc827200640 1 -- start start 2024-01-15T17:54:56.874 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.871+0000 7fc827200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8280b21a0 0x7fc82815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:56.874 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.871+0000 7fc827200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc82815f920 0x7fc8280cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:56.874 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.871+0000 7fc827200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280cf9d0 0x7fc82815fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:56.874 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.871+0000 7fc827200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280cfe80 con 0x7fc8280b21a0 2024-01-15T17:54:56.874 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.871+0000 7fc827200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b7e60 con 0x7fc82815f920 2024-01-15T17:54:56.874 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.871+0000 7fc827200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b7290 con 0x7fc8280cf9d0 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.871+0000 7fc825e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8280b21a0 0x7fc82815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc825e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8280b21a0 0x7fc82815f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55542/0 (socket says 172.21.15.26:55542) 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc825e00640 1 -- 172.21.15.26:0/113901688 learned_addr learned my addr 172.21.15.26:0/113901688 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc826800640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280cf9d0 0x7fc82815fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc825400640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc82815f920 0x7fc8280cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc825e00640 1 -- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280cf9d0 msgr2=0x7fc82815fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc825e00640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280cf9d0 0x7fc82815fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc825e00640 1 -- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc82815f920 msgr2=0x7fc8280cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc825e00640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc82815f920 0x7fc8280cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:56.876 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc825e00640 1 -- 172.21.15.26:0/113901688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8280b7530 con 0x7fc8280b21a0 2024-01-15T17:54:56.877 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.872+0000 7fc825e00640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8280b21a0 0x7fc82815f3e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc818007310 tx=0x7fc818007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:56.877 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.873+0000 7fc81e000640 1 -- 172.21.15.26:0/113901688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818001700 con 0x7fc8280b21a0 2024-01-15T17:54:56.877 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.873+0000 7fc81e000640 1 -- 172.21.15.26:0/113901688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81800f9c0 con 0x7fc8280b21a0 2024-01-15T17:54:56.877 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.873+0000 7fc81e000640 1 -- 172.21.15.26:0/113901688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800fd90 con 0x7fc8280b21a0 2024-01-15T17:54:56.877 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.873+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8280b7810 con 0x7fc8280b21a0 2024-01-15T17:54:56.877 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.873+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8280b82f0 con 0x7fc8280b21a0 2024-01-15T17:54:56.877 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.874+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc828067450 con 0x7fc8280b21a0 2024-01-15T17:54:56.878 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.875+0000 7fc81e000640 1 -- 172.21.15.26:0/113901688 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7fc81801d460 con 0x7fc8280b21a0 2024-01-15T17:54:56.879 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.876+0000 7fc81e000640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc80006b260 0x7fc80006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:56.879 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.876+0000 7fc81e000640 1 -- 172.21.15.26:0/113901688 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fc818089700 con 0x7fc8280b21a0 2024-01-15T17:54:56.879 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.877+0000 7fc825400640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc80006b260 0x7fc80006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:56.880 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.878+0000 7fc825400640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc80006b260 0x7fc80006d720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc810009ac0 tx=0x7fc810030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:56.884 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:56.881+0000 7fc81e000640 1 -- 172.21.15.26:0/113901688 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc818054a80 con 0x7fc8280b21a0 2024-01-15T17:54:57.087 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.084+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc82806c5a0 con 0x7fc8280b21a0 2024-01-15T17:54:57.089 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.086+0000 7fc81e000640 1 -- 172.21.15.26:0/113901688 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10884 (secure 0 0 0) 0x7fc8180580a0 con 0x7fc8280b21a0 2024-01-15T17:54:57.089 INFO:teuthology.misc.health.smithi026.stdout: 2024-01-15T17:54:57.089 INFO:teuthology.misc.health.smithi026.stdout:{"epoch":14,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:54:56.455609+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:54:57.094 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.088+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc80006b260 msgr2=0x7fc80006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc80006b260 0x7fc80006d720 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc810009ac0 tx=0x7fc810030000 comp rx=0 tx=0).stop 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8280b21a0 msgr2=0x7fc82815f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8280b21a0 0x7fc82815f3e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc818007310 tx=0x7fc818007710 comp rx=0 tx=0).stop 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 shutdown_connections 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc8280cf9d0 0x7fc82815fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc80006b260 0x7fc80006d720 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc82815f920 0x7fc8280cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 --2- 172.21.15.26:0/113901688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc8280b21a0 0x7fc82815f3e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 >> 172.21.15.26:0/113901688 conn(0x7fc8280d3090 msgr2=0x7fc8280d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 shutdown_connections 2024-01-15T17:54:57.095 INFO:teuthology.misc.health.smithi026.stderr:2024-01-15T17:54:57.089+0000 7fc827200640 1 -- 172.21.15.26:0/113901688 wait complete. 2024-01-15T17:54:57.107 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-15T17:54:57.107 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-15T17:54:57.108 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T17:54:57.108 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T17:54:57.112 INFO:tasks.ceph.mds.a:Started 2024-01-15T17:54:57.113 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T17:54:57.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T17:54:57.116 INFO:tasks.ceph.mds.c:Started 2024-01-15T17:54:57.117 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T17:54:57.117 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T17:54:57.124 INFO:tasks.ceph.mds.b:Started 2024-01-15T17:54:57.124 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T17:54:57.124 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T17:54:57.131 INFO:tasks.ceph.mds.d:Started 2024-01-15T17:54:57.132 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-15T17:54:57.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:54:57.265 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-15T17:54:57.266 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T17:54:57.291 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T17:54:57.298 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T17:54:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.323+0000 7ff8cbe00640 1 Processor -- start 2024-01-15T17:54:57.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.323+0000 7ff8cbe00640 1 -- start start 2024-01-15T17:54:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.324+0000 7ff8cbe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff8c4143ae0 0x7ff8c40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.324+0000 7ff8cbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c415e3b0 0x7ff8c40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.324+0000 7ff8cbe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c40cfc30 con 0x7ff8c40cf7d0 2024-01-15T17:54:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.324+0000 7ff8cbe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c40b1ee0 con 0x7ff8c4143ae0 2024-01-15T17:54:57.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.324+0000 7ff8cbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c40b2780 con 0x7ff8c415e3b0 2024-01-15T17:54:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.325+0000 7ff8c9400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff8c4143ae0 0x7ff8c40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.325+0000 7ff8c8a00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7ff8c40cf7d0 0x7ff8c415dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:42178/0 (socket says 172.21.15.26:42178) 2024-01-15T17:54:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.325+0000 7ff8c8a00640 1 -- 172.21.15.26:0/1201791658 learned_addr learned my addr 172.21.15.26:0/1201791658 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:54:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.325+0000 7ff8c9e00640 1 --2- 172.21.15.26:0/1201791658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c415e3b0 0x7ff8c40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.325+0000 7ff8c9400640 1 -- 172.21.15.26:0/1201791658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c415e3b0 msgr2=0x7ff8c40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.325+0000 7ff8c9400640 1 --2- 172.21.15.26:0/1201791658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c415e3b0 0x7ff8c40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.325+0000 7ff8c9400640 1 -- 172.21.15.26:0/1201791658 >> v1:172.21.15.124:6789/0 conn(0x7ff8c40cf7d0 legacy=0x7ff8c415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.325+0000 7ff8c9400640 1 -- 172.21.15.26:0/1201791658 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c415e990 con 0x7ff8c4143ae0 2024-01-15T17:54:57.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.325+0000 7ff8c3e00640 1 -- 172.21.15.26:0/1201791658 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992443738 0 0) 0x7ff8c40cfc30 con 0x7ff8c40cf7d0 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.326+0000 7ff8c9400640 1 --2- 172.21.15.26:0/1201791658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8c4143ae0 0x7ff8c40cf290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009dc0 tx=0x7ff8b402be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f29441fa7ca20cdf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8c3e00640 1 -- 172.21.15.26:0/1201791658 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b402cc40 con 0x7ff8c4143ae0 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8c3e00640 1 -- 172.21.15.26:0/1201791658 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b402cda0 con 0x7ff8c4143ae0 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8c3e00640 1 -- 172.21.15.26:0/1201791658 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4034660 con 0x7ff8c4143ae0 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8cbe00640 1 -- 172.21.15.26:0/1201791658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8c4143ae0 msgr2=0x7ff8c40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8cbe00640 1 --2- 172.21.15.26:0/1201791658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8c4143ae0 0x7ff8c40cf290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009dc0 tx=0x7ff8b402be90 comp rx=0 tx=0).stop 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8cbe00640 1 -- 172.21.15.26:0/1201791658 shutdown_connections 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8cbe00640 1 --2- 172.21.15.26:0/1201791658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c415e3b0 0x7ff8c40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8cbe00640 1 --2- 172.21.15.26:0/1201791658 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8c4143ae0 0x7ff8c40cf290 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8cbe00640 1 -- 172.21.15.26:0/1201791658 >> 172.21.15.26:0/1201791658 conn(0x7ff8c406a800 msgr2=0x7ff8c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.327+0000 7ff8cbe00640 1 -- 172.21.15.26:0/1201791658 shutdown_connections 2024-01-15T17:54:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.328+0000 7ff8cbe00640 1 -- 172.21.15.26:0/1201791658 wait complete. 2024-01-15T17:54:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.329+0000 7ff8cbe00640 1 Processor -- start 2024-01-15T17:54:57.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.329+0000 7ff8cbe00640 1 -- start start 2024-01-15T17:54:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.330+0000 7ff8cbe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8c40cf7d0 0x7ff8c41677f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.330+0000 7ff8cbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c4143ae0 0x7ff8c415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.330+0000 7ff8cbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c415e3b0 0x7ff8c415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.330+0000 7ff8cbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c4167430 con 0x7ff8c415e3b0 2024-01-15T17:54:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.330+0000 7ff8cbe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c4166fa0 con 0x7ff8c40cf7d0 2024-01-15T17:54:57.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.330+0000 7ff8cbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c4166390 con 0x7ff8c4143ae0 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c415e3b0 0x7ff8c415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c415e3b0 0x7ff8c415afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55574/0 (socket says 172.21.15.26:55574) 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9e00640 1 -- 172.21.15.26:0/330594984 learned_addr learned my addr 172.21.15.26:0/330594984 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9400640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8c40cf7d0 0x7ff8c41677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c8a00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c4143ae0 0x7ff8c415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9e00640 1 -- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c4143ae0 msgr2=0x7ff8c415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9e00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c4143ae0 0x7ff8c415aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9e00640 1 -- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8c40cf7d0 msgr2=0x7ff8c41677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9e00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8c40cf7d0 0x7ff8c41677f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9e00640 1 -- 172.21.15.26:0/330594984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4009a70 con 0x7ff8c415e3b0 2024-01-15T17:54:57.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.331+0000 7ff8c9e00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c415e3b0 0x7ff8c415afc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00e7f0 tx=0x7ff8bc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.332+0000 7ff8c1600640 1 -- 172.21.15.26:0/330594984 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc009510 con 0x7ff8c415e3b0 2024-01-15T17:54:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.332+0000 7ff8c1600640 1 -- 172.21.15.26:0/330594984 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc00ee40 con 0x7ff8c415e3b0 2024-01-15T17:54:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.332+0000 7ff8c1600640 1 -- 172.21.15.26:0/330594984 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00cb80 con 0x7ff8c415e3b0 2024-01-15T17:54:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.332+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c4166670 con 0x7ff8c415e3b0 2024-01-15T17:54:57.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.332+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c4166c30 con 0x7ff8c415e3b0 2024-01-15T17:54:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.333+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c40675d0 con 0x7ff8c415e3b0 2024-01-15T17:54:57.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.334+0000 7ff8c1600640 1 -- 172.21.15.26:0/330594984 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7ff8bc00cce0 con 0x7ff8c415e3b0 2024-01-15T17:54:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.335+0000 7ff8c1600640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff89c06b280 0x7ff89c06d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.335+0000 7ff8c1600640 1 -- 172.21.15.26:0/330594984 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7ff8bc089570 con 0x7ff8c415e3b0 2024-01-15T17:54:57.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.335+0000 7ff8c9400640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff89c06b280 0x7ff89c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.336+0000 7ff8c9400640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff89c06b280 0x7ff89c06d740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4036000 tx=0x7ff8b4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:57.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.340+0000 7ff8c1600640 1 -- 172.21.15.26:0/330594984 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff8bc054ab0 con 0x7ff8c415e3b0 2024-01-15T17:54:57.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.540+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8c41670e0 con 0x7ff8c415e3b0 2024-01-15T17:54:57.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.541+0000 7ff8c1600640 1 -- 172.21.15.26:0/330594984 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11068 (secure 0 0 0) 0x7ff8bc017090 con 0x7ff8c415e3b0 2024-01-15T17:54:57.545 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:54:57.545 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":15,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:54:57.459532+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:54:57.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff89c06b280 msgr2=0x7ff89c06d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff89c06b280 0x7ff89c06d740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4036000 tx=0x7ff8b4030000 comp rx=0 tx=0).stop 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c415e3b0 msgr2=0x7ff8c415afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c415e3b0 0x7ff8c415afc0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00e7f0 tx=0x7ff8bc00ebf0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 shutdown_connections 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff8c4143ae0 0x7ff8c415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff89c06b280 0x7ff89c06d740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff8c415e3b0 0x7ff8c415afc0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 --2- 172.21.15.26:0/330594984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff8c40cf7d0 0x7ff8c41677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 >> 172.21.15.26:0/330594984 conn(0x7ff8c406a800 msgr2=0x7ff8c406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 shutdown_connections 2024-01-15T17:54:57.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.544+0000 7ff8cbe00640 1 -- 172.21.15.26:0/330594984 wait complete. 2024-01-15T17:54:57.566 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-01-15T17:54:57.566 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:54:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.775+0000 7fd8a2400640 1 Processor -- start 2024-01-15T17:54:57.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.776+0000 7fd8a2400640 1 -- start start 2024-01-15T17:54:57.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.777+0000 7fd8a2400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd89c160920 0x7fd89c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.777+0000 7fd8a2400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd89c0cf9f0 0x7fd89c0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.777+0000 7fd8a2400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89c0d0a10 con 0x7fd89c161010 2024-01-15T17:54:57.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.777+0000 7fd8a2400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c0d03e0 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.777+0000 7fd8a2400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c0d1f50 con 0x7fd89c160920 2024-01-15T17:54:57.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.778+0000 7fd893e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd89c0cf9f0 0x7fd89c0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.778+0000 7fd8a1000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd89c160920 0x7fd89c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.778+0000 7fd893e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd89c0cf9f0 0x7fd89c0cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55582/0 (socket says 172.21.15.26:55582) 2024-01-15T17:54:57.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.778+0000 7fd893e00640 1 -- 172.21.15.26:0/2028001312 learned_addr learned my addr 172.21.15.26:0/2028001312 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:54:57.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.778+0000 7fd893e00640 1 -- 172.21.15.26:0/2028001312 >> v2:172.21.15.124:3300/0 conn(0x7fd89c160920 msgr2=0x7fd89c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:54:57.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.778+0000 7fd893e00640 1 --2- 172.21.15.26:0/2028001312 >> v2:172.21.15.124:3300/0 conn(0x7fd89c160920 0x7fd89c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.778+0000 7fd893e00640 1 -- 172.21.15.26:0/2028001312 >> v1:172.21.15.124:6789/0 conn(0x7fd89c161010 legacy=0x7fd89c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.778+0000 7fd893e00640 1 -- 172.21.15.26:0/2028001312 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c0cf080 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.779+0000 7fd893e00640 1 --2- 172.21.15.26:0/2028001312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 0x7fd89c0cfea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd89c08c060 tx=0x7fd88402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eabf0fe920629747 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd89be00640 1 -- 172.21.15.26:0/2028001312 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884030860 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd89be00640 1 -- 172.21.15.26:0/2028001312 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8840309c0 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd89be00640 1 -- 172.21.15.26:0/2028001312 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884030d50 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd8a2400640 1 -- 172.21.15.26:0/2028001312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 msgr2=0x7fd89c0cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd8a2400640 1 --2- 172.21.15.26:0/2028001312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 0x7fd89c0cfea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd89c08c060 tx=0x7fd88402b9a0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd8a2400640 1 -- 172.21.15.26:0/2028001312 shutdown_connections 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd8a2400640 1 --2- 172.21.15.26:0/2028001312 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 0x7fd89c0cfea0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd8a2400640 1 --2- 172.21.15.26:0/2028001312 >> v2:172.21.15.124:3300/0 conn(0x7fd89c160920 0x7fd89c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd8a2400640 1 -- 172.21.15.26:0/2028001312 >> 172.21.15.26:0/2028001312 conn(0x7fd89c0d3090 msgr2=0x7fd89c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd8a2400640 1 -- 172.21.15.26:0/2028001312 shutdown_connections 2024-01-15T17:54:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.780+0000 7fd8a2400640 1 -- 172.21.15.26:0/2028001312 wait complete. 2024-01-15T17:54:57.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.782+0000 7fd8a2400640 1 Processor -- start 2024-01-15T17:54:57.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.782+0000 7fd8a2400640 1 -- start start 2024-01-15T17:54:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.783+0000 7fd8a2400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 0x7fd89c1f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.783+0000 7fd8a2400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd89c160920 0x7fd89c0b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.783+0000 7fd8a2400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd89c161010 0x7fd89c0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.783+0000 7fd8a2400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c1f6810 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.783+0000 7fd8a2400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0cbfb0 con 0x7fd89c161010 2024-01-15T17:54:57.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.783+0000 7fd8a2400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0cc530 con 0x7fd89c160920 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 0x7fd89c1f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 0x7fd89c1f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55596/0 (socket says 172.21.15.26:55596) 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1000640 1 -- 172.21.15.26:0/2694017087 learned_addr learned my addr 172.21.15.26:0/2694017087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1a00640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd89c161010 0x7fd89c0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd893e00640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd89c160920 0x7fd89c0b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1000640 1 -- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd89c160920 msgr2=0x7fd89c0b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1000640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd89c160920 0x7fd89c0b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1000640 1 -- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd89c161010 msgr2=0x7fd89c0b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1000640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd89c161010 0x7fd89c0b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1000640 1 -- 172.21.15.26:0/2694017087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8840094d0 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd893e00640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd89c160920 0x7fd89c0b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:54:57.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.784+0000 7fd8a1000640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 0x7fd89c1f3470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00e9b0 tx=0x7fd88c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.785+0000 7fd899600640 1 -- 172.21.15.26:0/2694017087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c009670 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.785+0000 7fd899600640 1 -- 172.21.15.26:0/2694017087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c00c9a0 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.785+0000 7fd899600640 1 -- 172.21.15.26:0/2694017087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c00cd50 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.785+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c0cc250 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.785+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c15e290 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.786+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c067450 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.787+0000 7fd899600640 1 -- 172.21.15.26:0/2694017087 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7fd88c013470 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.788+0000 7fd899600640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd87406b230 0x7fd87406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:54:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.788+0000 7fd899600640 1 -- 172.21.15.26:0/2694017087 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fd88c0898a0 con 0x7fd89c0cf9f0 2024-01-15T17:54:57.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.789+0000 7fd893e00640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd87406b230 0x7fd87406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:54:57.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.790+0000 7fd893e00640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd87406b230 0x7fd87406d6f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd884036000 tx=0x7fd88403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:54:57.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:57.792+0000 7fd899600640 1 -- 172.21.15.26:0/2694017087 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd88c054cb0 con 0x7fd89c0cf9f0 2024-01-15T17:54:58.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.026+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd89c0cd550 con 0x7fd89c0cf9f0 2024-01-15T17:54:58.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.028+0000 7fd899600640 1 -- 172.21.15.26:0/2694017087 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7fd88c021540 con 0x7fd89c0cf9f0 2024-01-15T17:54:58.031 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:54:58.032 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-15T17:54:58.032 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 2 2024-01-15T17:54:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd87406b230 msgr2=0x7fd87406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd87406b230 0x7fd87406d6f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd884036000 tx=0x7fd88403b040 comp rx=0 tx=0).stop 2024-01-15T17:54:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 msgr2=0x7fd89c1f3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:54:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 0x7fd89c1f3470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00e9b0 tx=0x7fd88c00ee70 comp rx=0 tx=0).stop 2024-01-15T17:54:58.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 shutdown_connections 2024-01-15T17:54:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd89c160920 0x7fd89c0b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd87406b230 0x7fd87406d6f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd89c161010 0x7fd89c0b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 --2- 172.21.15.26:0/2694017087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd89c0cf9f0 0x7fd89c1f3470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:54:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 >> 172.21.15.26:0/2694017087 conn(0x7fd89c0d3090 msgr2=0x7fd89c06b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:54:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 shutdown_connections 2024-01-15T17:54:58.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:54:58.031+0000 7fd8a2400640 1 -- 172.21.15.26:0/2694017087 wait complete. 2024-01-15T17:55:00.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:55:00.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.270+0000 7f9ba0200640 1 Processor -- start 2024-01-15T17:55:00.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.270+0000 7f9ba0200640 1 -- start start 2024-01-15T17:55:00.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.272+0000 7f9ba0200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b9813df80 0x7f9b9815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.272+0000 7f9ba0200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813e360 0x7f9b980cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.272+0000 7f9ba0200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9806c470 con 0x7f9b9815f9c0 2024-01-15T17:55:00.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.272+0000 7f9ba0200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9806c750 con 0x7f9b9813df80 2024-01-15T17:55:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.272+0000 7f9ba0200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9806ca30 con 0x7f9b9813e360 2024-01-15T17:55:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.272+0000 7f9b9d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b9813df80 0x7f9b9815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.272+0000 7f9b9d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b9813df80 0x7f9b9815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55600/0 (socket says 172.21.15.26:55600) 2024-01-15T17:55:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.272+0000 7f9b9d800640 1 -- 172.21.15.26:0/3637650533 learned_addr learned my addr 172.21.15.26:0/3637650533 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:00.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.273+0000 7f9b9ce00640 1 --2- 172.21.15.26:0/3637650533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813e360 0x7f9b980cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.273+0000 7f9b9d800640 1 -- 172.21.15.26:0/3637650533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813e360 msgr2=0x7f9b980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:00.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.273+0000 7f9b9d800640 1 --2- 172.21.15.26:0/3637650533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813e360 0x7f9b980cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.273+0000 7f9b9d800640 1 -- 172.21.15.26:0/3637650533 >> v1:172.21.15.124:6789/0 conn(0x7f9b9815f9c0 legacy=0x7f9b981408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.273+0000 7f9b9d800640 1 -- 172.21.15.26:0/3637650533 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9806cd10 con 0x7f9b9813df80 2024-01-15T17:55:00.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.274+0000 7f9b9d800640 1 --2- 172.21.15.26:0/3637650533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813df80 0x7f9b9815bee0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009870 tx=0x7f9b8802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d5837ef2055b21 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:00.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.274+0000 7f9b97e00640 1 -- 172.21.15.26:0/3637650533 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88030840 con 0x7f9b9813df80 2024-01-15T17:55:00.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.274+0000 7f9b97e00640 1 -- 172.21.15.26:0/3637650533 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b880309a0 con 0x7f9b9813df80 2024-01-15T17:55:00.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.274+0000 7f9b97e00640 1 -- 172.21.15.26:0/3637650533 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88030d30 con 0x7f9b9813df80 2024-01-15T17:55:00.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.275+0000 7f9ba0200640 1 -- 172.21.15.26:0/3637650533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813df80 msgr2=0x7f9b9815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.275+0000 7f9ba0200640 1 --2- 172.21.15.26:0/3637650533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813df80 0x7f9b9815bee0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009870 tx=0x7f9b8802b980 comp rx=0 tx=0).stop 2024-01-15T17:55:00.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.275+0000 7f9ba0200640 1 -- 172.21.15.26:0/3637650533 shutdown_connections 2024-01-15T17:55:00.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.275+0000 7f9ba0200640 1 --2- 172.21.15.26:0/3637650533 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813e360 0x7f9b980cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.275+0000 7f9ba0200640 1 --2- 172.21.15.26:0/3637650533 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813df80 0x7f9b9815bee0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.275+0000 7f9ba0200640 1 -- 172.21.15.26:0/3637650533 >> 172.21.15.26:0/3637650533 conn(0x7f9b9806a800 msgr2=0x7f9b9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:00.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.275+0000 7f9ba0200640 1 -- 172.21.15.26:0/3637650533 shutdown_connections 2024-01-15T17:55:00.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.275+0000 7f9ba0200640 1 -- 172.21.15.26:0/3637650533 wait complete. 2024-01-15T17:55:00.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.277+0000 7f9ba0200640 1 Processor -- start 2024-01-15T17:55:00.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.277+0000 7f9ba0200640 1 -- start start 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.278+0000 7f9ba0200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813df80 0x7f9b98132060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.278+0000 7f9ba0200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813e360 0x7f9b9814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.278+0000 7f9ba0200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b9815f9c0 0x7f9b9814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.278+0000 7f9ba0200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98131c40 con 0x7f9b9813e360 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.278+0000 7f9ba0200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b981315c0 con 0x7f9b9815f9c0 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.278+0000 7f9ba0200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9814b5a0 con 0x7f9b9813df80 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.278+0000 7f9b9ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813e360 0x7f9b9814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.278+0000 7f9b9ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813e360 0x7f9b9814c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55606/0 (socket says 172.21.15.26:55606) 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.278+0000 7f9b9ce00640 1 -- 172.21.15.26:0/3240958075 learned_addr learned my addr 172.21.15.26:0/3240958075 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b9d800640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813df80 0x7f9b98132060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b9e200640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b9815f9c0 0x7f9b9814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b9ce00640 1 -- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813df80 msgr2=0x7f9b98132060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b9ce00640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813df80 0x7f9b98132060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b9ce00640 1 -- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b9815f9c0 msgr2=0x7f9b9814ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b9ce00640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b9815f9c0 0x7f9b9814ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b9ce00640 1 -- 172.21.15.26:0/3240958075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880094d0 con 0x7f9b9813e360 2024-01-15T17:55:00.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b9ce00640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813e360 0x7f9b9814c200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000d800 tx=0x7f9b8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b95600640 1 -- 172.21.15.26:0/3240958075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8000b6c0 con 0x7f9b9813e360 2024-01-15T17:55:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b95600640 1 -- 172.21.15.26:0/3240958075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8000b820 con 0x7f9b9813e360 2024-01-15T17:55:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.279+0000 7f9b95600640 1 -- 172.21.15.26:0/3240958075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8000bbb0 con 0x7f9b9813e360 2024-01-15T17:55:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.280+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9814b880 con 0x7f9b9813e360 2024-01-15T17:55:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.280+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9814c740 con 0x7f9b9813e360 2024-01-15T17:55:00.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.281+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b980675d0 con 0x7f9b9813e360 2024-01-15T17:55:00.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.282+0000 7f9b95600640 1 -- 172.21.15.26:0/3240958075 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7f9b8000bd10 con 0x7f9b9813e360 2024-01-15T17:55:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.283+0000 7f9b95600640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b7006b260 0x7f9b7006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.283+0000 7f9b95600640 1 -- 172.21.15.26:0/3240958075 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b80089c60 con 0x7f9b9813e360 2024-01-15T17:55:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.291+0000 7f9b9d800640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b7006b260 0x7f9b7006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.292+0000 7f9b9d800640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b7006b260 0x7f9b7006d720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b88036000 tx=0x7f9b8803b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:00.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.294+0000 7f9b95600640 1 -- 172.21.15.26:0/3240958075 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b80054f30 con 0x7f9b9813e360 2024-01-15T17:55:00.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.531+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b9815c850 con 0x7f9b9813e360 2024-01-15T17:55:00.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.532+0000 7f9b95600640 1 -- 172.21.15.26:0/3240958075 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f9b80058550 con 0x7f9b9813e360 2024-01-15T17:55:00.535 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:00.535 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/593578542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":593578542},{"type":"v1","addr":"172.21.15.124:6837","nonce":593578542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1776325954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1776325954},{"type":"v1","addr":"172.21.15.26:6835","nonce":1776325954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/3022017251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3022017251},{"type":"v1","addr":"172.21.15.124:6835","nonce":3022017251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-15T17:55:00.536 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 3 2024-01-15T17:55:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b7006b260 msgr2=0x7f9b7006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b7006b260 0x7f9b7006d720 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b88036000 tx=0x7f9b8803b040 comp rx=0 tx=0).stop 2024-01-15T17:55:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813e360 msgr2=0x7f9b9814c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813e360 0x7f9b9814c200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000d800 tx=0x7f9b8000dcc0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 shutdown_connections 2024-01-15T17:55:00.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b7006b260 0x7f9b7006d720 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b9815f9c0 0x7f9b9814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b9813e360 0x7f9b9814c200 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 --2- 172.21.15.26:0/3240958075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b9813df80 0x7f9b98132060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 >> 172.21.15.26:0/3240958075 conn(0x7f9b9806a800 msgr2=0x7f9b9815cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 shutdown_connections 2024-01-15T17:55:00.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.535+0000 7f9ba0200640 1 -- 172.21.15.26:0/3240958075 wait complete. 2024-01-15T17:55:00.555 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:00.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.740+0000 7f9db5600640 1 Processor -- start 2024-01-15T17:55:00.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.740+0000 7f9db5600640 1 -- start start 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.741+0000 7f9db5600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9db00ae5a0 0x7f9db015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.741+0000 7f9db5600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 0x7f9db006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.741+0000 7f9db5600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9db006d120 0x7f9db00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.741+0000 7f9db5600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db0071450 con 0x7f9db00ae5a0 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.741+0000 7f9db5600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db0070c40 con 0x7f9db006d120 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.741+0000 7f9db5600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db006c470 con 0x7f9db00b3ae0 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9daea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9db00ae5a0 0x7f9db015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9daea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9db00ae5a0 0x7f9db015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55622/0 (socket says 172.21.15.26:55622) 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9daea00640 1 -- 172.21.15.26:0/1223708379 learned_addr learned my addr 172.21.15.26:0/1223708379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:00.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9daf400640 1 --2- 172.21.15.26:0/1223708379 >> v2:172.21.15.124:3300/0 conn(0x7f9db006d120 0x7f9db00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9dae000640 1 --2- 172.21.15.26:0/1223708379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 0x7f9db006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9daea00640 1 -- 172.21.15.26:0/1223708379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 msgr2=0x7f9db006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:00.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9daea00640 1 --2- 172.21.15.26:0/1223708379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 0x7f9db006cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9daea00640 1 -- 172.21.15.26:0/1223708379 >> v2:172.21.15.124:3300/0 conn(0x7f9db006d120 msgr2=0x7f9db00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:00.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9daea00640 1 --2- 172.21.15.26:0/1223708379 >> v2:172.21.15.124:3300/0 conn(0x7f9db006d120 0x7f9db00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.742+0000 7f9daea00640 1 -- 172.21.15.26:0/1223708379 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db006c750 con 0x7f9db00ae5a0 2024-01-15T17:55:00.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.744+0000 7f9daea00640 1 --2- 172.21.15.26:0/1223708379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db015e430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9db006e230 tx=0x7f9da002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4350b345c1432c7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.744+0000 7f9dad600640 1 -- 172.21.15.26:0/1223708379 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0031870 con 0x7f9db00ae5a0 2024-01-15T17:55:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.744+0000 7f9dad600640 1 -- 172.21.15.26:0/1223708379 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da00319d0 con 0x7f9db00ae5a0 2024-01-15T17:55:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.744+0000 7f9dad600640 1 -- 172.21.15.26:0/1223708379 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0031d60 con 0x7f9db00ae5a0 2024-01-15T17:55:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.745+0000 7f9db5600640 1 -- 172.21.15.26:0/1223708379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 msgr2=0x7f9db015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.745+0000 7f9db5600640 1 --2- 172.21.15.26:0/1223708379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db015e430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9db006e230 tx=0x7f9da002b910 comp rx=0 tx=0).stop 2024-01-15T17:55:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.745+0000 7f9db5600640 1 -- 172.21.15.26:0/1223708379 shutdown_connections 2024-01-15T17:55:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.745+0000 7f9db5600640 1 --2- 172.21.15.26:0/1223708379 >> v2:172.21.15.124:3300/0 conn(0x7f9db006d120 0x7f9db00cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.745+0000 7f9db5600640 1 --2- 172.21.15.26:0/1223708379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 0x7f9db006cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.745+0000 7f9db5600640 1 --2- 172.21.15.26:0/1223708379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db015e430 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.745+0000 7f9db5600640 1 -- 172.21.15.26:0/1223708379 >> 172.21.15.26:0/1223708379 conn(0x7f9db006a800 msgr2=0x7f9db006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.745+0000 7f9db5600640 1 -- 172.21.15.26:0/1223708379 shutdown_connections 2024-01-15T17:55:00.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.745+0000 7f9db5600640 1 -- 172.21.15.26:0/1223708379 wait complete. 2024-01-15T17:55:00.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.747+0000 7f9db5600640 1 Processor -- start 2024-01-15T17:55:00.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.747+0000 7f9db5600640 1 -- start start 2024-01-15T17:55:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9db5600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db006d120 0x7f9db00cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9db5600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db00b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9db5600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9db5600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0156320 con 0x7f9db00ae5a0 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9db5600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db01565c0 con 0x7f9db00b3ae0 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9db5600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db01568a0 con 0x7f9db006d120 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9dae000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db00b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9dae000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db00b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55632/0 (socket says 172.21.15.26:55632) 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9dae000640 1 -- 172.21.15.26:0/3694189135 learned_addr learned my addr 172.21.15.26:0/3694189135 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9daea00640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db006d120 0x7f9db00cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.748+0000 7f9daf400640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9dae000640 1 -- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db006d120 msgr2=0x7f9db00cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9dae000640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db006d120 0x7f9db00cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9dae000640 1 -- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db00b3ae0 msgr2=0x7f9db00b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9dae000640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9dae000640 1 -- 172.21.15.26:0/3694189135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da00094d0 con 0x7f9db00ae5a0 2024-01-15T17:55:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9daf400640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9dae000640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db00b2a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800d5f0 tx=0x7f9d9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9da6a00640 1 -- 172.21.15.26:0/3694189135 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d980125f0 con 0x7f9db00ae5a0 2024-01-15T17:55:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9da6a00640 1 -- 172.21.15.26:0/3694189135 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d98012750 con 0x7f9db00ae5a0 2024-01-15T17:55:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db0156b80 con 0x7f9db00ae5a0 2024-01-15T17:55:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9da6a00640 1 -- 172.21.15.26:0/3694189135 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d98012b00 con 0x7f9db00ae5a0 2024-01-15T17:55:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.749+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db0157140 con 0x7f9db00ae5a0 2024-01-15T17:55:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.751+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db0157300 con 0x7f9db00ae5a0 2024-01-15T17:55:00.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.751+0000 7f9da6a00640 1 -- 172.21.15.26:0/3694189135 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 4) v1 ==== 89483+0+0 (secure 0 0 0) 0x7f9d98012c60 con 0x7f9db00ae5a0 2024-01-15T17:55:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.753+0000 7f9da6a00640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b260 0x7f9d8806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.753+0000 7f9da6a00640 1 -- 172.21.15.26:0/3694189135 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9d98089ca0 con 0x7f9db00ae5a0 2024-01-15T17:55:00.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.753+0000 7f9daea00640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b260 0x7f9d8806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:00.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.755+0000 7f9daea00640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b260 0x7f9d8806d720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9da0036000 tx=0x7f9da003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:00.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.757+0000 7f9da6a00640 1 -- 172.21.15.26:0/3694189135 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d98054ea0 con 0x7f9db00ae5a0 2024-01-15T17:55:00.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.818+0000 7f9da6a00640 1 -- 172.21.15.26:0/3694189135 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9d98055270 con 0x7f9db00ae5a0 2024-01-15T17:55:00.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.970+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9db00675d0 con 0x7f9db00ae5a0 2024-01-15T17:55:00.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.971+0000 7f9da6a00640 1 -- 172.21.15.26:0/3694189135 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12826 (secure 0 0 0) 0x7f9d98087e50 con 0x7f9db00ae5a0 2024-01-15T17:55:00.974 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:00.974 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":18,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:00.475281+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.973+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b260 msgr2=0x7f9d8806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.973+0000 7f9db5600640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b260 0x7f9d8806d720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9da0036000 tx=0x7f9da003c040 comp rx=0 tx=0).stop 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.973+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 msgr2=0x7f9db00b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.973+0000 7f9db5600640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db00b2a60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800d5f0 tx=0x7f9d9800dab0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.973+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 shutdown_connections 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.973+0000 7f9db5600640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b260 0x7f9d8806d720 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.973+0000 7f9db5600640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.973+0000 7f9db5600640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db00b2a60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.973+0000 7f9db5600640 1 --2- 172.21.15.26:0/3694189135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db006d120 0x7f9db00cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:00.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.974+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 >> 172.21.15.26:0/3694189135 conn(0x7f9db006a800 msgr2=0x7f9db006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:00.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.974+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 shutdown_connections 2024-01-15T17:55:00.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:00.974+0000 7f9db5600640 1 -- 172.21.15.26:0/3694189135 wait complete. 2024-01-15T17:55:00.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:00.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:01.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.182+0000 7f6918e00640 1 Processor -- start 2024-01-15T17:55:01.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.182+0000 7f6918e00640 1 -- start start 2024-01-15T17:55:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.183+0000 7f6918e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f691415d4b0 0x7f69140d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.183+0000 7f6918e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69140b41e0 0x7f69140cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.183+0000 7f6918e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69140d0320 con 0x7f69140cf9a0 2024-01-15T17:55:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.183+0000 7f6918e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69140b2730 con 0x7f69140b41e0 2024-01-15T17:55:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.183+0000 7f6918e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140d1ec0 con 0x7f691415d4b0 2024-01-15T17:55:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.184+0000 7f6913e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69140b41e0 0x7f69140cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.184+0000 7f6913e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f69140b41e0 0x7f69140cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55644/0 (socket says 172.21.15.26:55644) 2024-01-15T17:55:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.184+0000 7f6913e00640 1 -- 172.21.15.26:0/4041655412 learned_addr learned my addr 172.21.15.26:0/4041655412 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:01.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.184+0000 7f6913400640 1 --2- 172.21.15.26:0/4041655412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f691415d4b0 0x7f69140d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.184+0000 7f6913e00640 1 -- 172.21.15.26:0/4041655412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f691415d4b0 msgr2=0x7f69140d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:01.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.184+0000 7f6913e00640 1 --2- 172.21.15.26:0/4041655412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f691415d4b0 0x7f69140d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.184+0000 7f6913e00640 1 -- 172.21.15.26:0/4041655412 >> v1:172.21.15.124:6789/0 conn(0x7f69140cf9a0 legacy=0x7f69140cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.184+0000 7f6913e00640 1 -- 172.21.15.26:0/4041655412 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69140b29d0 con 0x7f69140b41e0 2024-01-15T17:55:01.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.185+0000 7f6913e00640 1 --2- 172.21.15.26:0/4041655412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69140b41e0 0x7f69140cfde0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f690c009e80 tx=0x7f690c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c9832e2861c68e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.186+0000 7f6912000640 1 -- 172.21.15.26:0/4041655412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c02caf0 con 0x7f69140b41e0 2024-01-15T17:55:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.186+0000 7f6912000640 1 -- 172.21.15.26:0/4041655412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c02cc50 con 0x7f69140b41e0 2024-01-15T17:55:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.186+0000 7f6912000640 1 -- 172.21.15.26:0/4041655412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c0343f0 con 0x7f69140b41e0 2024-01-15T17:55:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.186+0000 7f6918e00640 1 -- 172.21.15.26:0/4041655412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69140b41e0 msgr2=0x7f69140cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.186+0000 7f6918e00640 1 --2- 172.21.15.26:0/4041655412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69140b41e0 0x7f69140cfde0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f690c009e80 tx=0x7f690c02bf50 comp rx=0 tx=0).stop 2024-01-15T17:55:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.187+0000 7f6918e00640 1 -- 172.21.15.26:0/4041655412 shutdown_connections 2024-01-15T17:55:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.187+0000 7f6918e00640 1 --2- 172.21.15.26:0/4041655412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69140b41e0 0x7f69140cfde0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.187+0000 7f6918e00640 1 --2- 172.21.15.26:0/4041655412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f691415d4b0 0x7f69140d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.187+0000 7f6918e00640 1 -- 172.21.15.26:0/4041655412 >> 172.21.15.26:0/4041655412 conn(0x7f69140d3000 msgr2=0x7f69140d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:01.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.187+0000 7f6918e00640 1 -- 172.21.15.26:0/4041655412 shutdown_connections 2024-01-15T17:55:01.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.187+0000 7f6918e00640 1 -- 172.21.15.26:0/4041655412 wait complete. 2024-01-15T17:55:01.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.189+0000 7f6918e00640 1 Processor -- start 2024-01-15T17:55:01.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.189+0000 7f6918e00640 1 -- start start 2024-01-15T17:55:01.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6918e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69140b41e0 0x7f691414bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6918e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69140cf9a0 0x7f691414b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6918e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f691415d4b0 0x7f691415f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6918e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140b5cc0 con 0x7f691415d4b0 2024-01-15T17:55:01.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6918e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140b5fa0 con 0x7f69140b41e0 2024-01-15T17:55:01.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6918e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140b6280 con 0x7f69140cf9a0 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6913e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f691415d4b0 0x7f691415f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6913e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f691415d4b0 0x7f691415f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55652/0 (socket says 172.21.15.26:55652) 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6913e00640 1 -- 172.21.15.26:0/1494379346 learned_addr learned my addr 172.21.15.26:0/1494379346 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6913400640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69140b41e0 0x7f691414bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.190+0000 7f6912a00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69140cf9a0 0x7f691414b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f6913e00640 1 -- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69140cf9a0 msgr2=0x7f691414b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f6913e00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69140cf9a0 0x7f691414b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f6913e00640 1 -- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69140b41e0 msgr2=0x7f691414bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f6913e00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69140b41e0 0x7f691414bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f6913e00640 1 -- 172.21.15.26:0/1494379346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c009b30 con 0x7f691415d4b0 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f6913e00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f691415d4b0 0x7f691415f770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f690c036040 tx=0x7f690c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f690b400640 1 -- 172.21.15.26:0/1494379346 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c009510 con 0x7f691415d4b0 2024-01-15T17:55:01.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f690b400640 1 -- 172.21.15.26:0/1494379346 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c03e450 con 0x7f691415d4b0 2024-01-15T17:55:01.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691415da10 con 0x7f691415d4b0 2024-01-15T17:55:01.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f690b400640 1 -- 172.21.15.26:0/1494379346 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c03e800 con 0x7f691415d4b0 2024-01-15T17:55:01.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.191+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69140b6840 con 0x7f691415d4b0 2024-01-15T17:55:01.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.193+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691415cff0 con 0x7f691415d4b0 2024-01-15T17:55:01.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.193+0000 7f690b400640 1 -- 172.21.15.26:0/1494379346 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f690c03e960 con 0x7f691415d4b0 2024-01-15T17:55:01.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.194+0000 7f690b400640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ec06b2c0 0x7f68ec06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.194+0000 7f690b400640 1 -- 172.21.15.26:0/1494379346 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f690c0ac670 con 0x7f691415d4b0 2024-01-15T17:55:01.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.195+0000 7f6913400640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ec06b2c0 0x7f68ec06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.196+0000 7f6913400640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ec06b2c0 0x7f68ec06d780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69040079a0 tx=0x7f6904008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:01.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.199+0000 7f690b400640 1 -- 172.21.15.26:0/1494379346 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f690c077790 con 0x7f691415d4b0 2024-01-15T17:55:01.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.402+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f69140cf470 con 0x7f691415d4b0 2024-01-15T17:55:01.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.403+0000 7f690b400640 1 -- 172.21.15.26:0/1494379346 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f690c043020 con 0x7f691415d4b0 2024-01-15T17:55:01.406 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.405+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ec06b2c0 msgr2=0x7f68ec06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.405+0000 7f6918e00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ec06b2c0 0x7f68ec06d780 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69040079a0 tx=0x7f6904008040 comp rx=0 tx=0).stop 2024-01-15T17:55:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.405+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f691415d4b0 msgr2=0x7f691415f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.405+0000 7f6918e00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f691415d4b0 0x7f691415f770 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f690c036040 tx=0x7f690c004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.405+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 shutdown_connections 2024-01-15T17:55:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.406+0000 7f6918e00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69140cf9a0 0x7f691414b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.406+0000 7f6918e00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ec06b2c0 0x7f68ec06d780 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.406+0000 7f6918e00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f691415d4b0 0x7f691415f770 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.406+0000 7f6918e00640 1 --2- 172.21.15.26:0/1494379346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69140b41e0 0x7f691414bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.406+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 >> 172.21.15.26:0/1494379346 conn(0x7f69140d3000 msgr2=0x7f69140b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.406+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 shutdown_connections 2024-01-15T17:55:01.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.406+0000 7f6918e00640 1 -- 172.21.15.26:0/1494379346 wait complete. 2024-01-15T17:55:01.427 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T17:55:01.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.637+0000 7f70b9000640 1 Processor -- start 2024-01-15T17:55:01.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.638+0000 7f70b9000640 1 -- start start 2024-01-15T17:55:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.639+0000 7f70b9000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f70b40d3910 0x7f70b40d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.639+0000 7f70b9000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b40d4160 con 0x7f70b4160e10 2024-01-15T17:55:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.640+0000 7f70b9000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b40b2420 con 0x7f70b40cfcf0 2024-01-15T17:55:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.640+0000 7f70b9000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b40ceff0 con 0x7f70b40d3910 2024-01-15T17:55:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.640+0000 7f70b2800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f70b4160e10 0x7f70b40d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48930/0 (socket says 172.21.15.26:48930) 2024-01-15T17:55:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.640+0000 7f70b2800640 1 -- 172.21.15.26:0/1244902332 learned_addr learned my addr 172.21.15.26:0/1244902332 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:01.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.640+0000 7f70abe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f70b40d3910 0x7f70b40d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.640+0000 7f70b1400640 1 -- 172.21.15.26:0/1244902332 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040452405 0 0) 0x7f70b40d4160 con 0x7f70b4160e10 2024-01-15T17:55:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.640+0000 7f70b1400640 1 -- 172.21.15.26:0/1244902332 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7094003520 con 0x7f70b4160e10 2024-01-15T17:55:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.640+0000 7f70b1400640 1 -- 172.21.15.26:0/1244902332 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556731198 0 0) 0x7f70b40b2420 con 0x7f70b40cfcf0 2024-01-15T17:55:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.640+0000 7f70b1400640 1 -- 172.21.15.26:0/1244902332 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b40d4160 con 0x7f70b40cfcf0 2024-01-15T17:55:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.641+0000 7f70abe00640 1 -- 172.21.15.26:0/1244902332 >> v1:172.21.15.124:6789/0 conn(0x7f70b40cfcf0 legacy=0x7f70b4160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.641+0000 7f70abe00640 1 -- 172.21.15.26:0/1244902332 >> v1:172.21.15.26:6789/0 conn(0x7f70b4160e10 legacy=0x7f70b40d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.641+0000 7f70abe00640 1 -- 172.21.15.26:0/1244902332 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b40b2140 con 0x7f70b40d3910 2024-01-15T17:55:01.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.642+0000 7f70abe00640 1 --2- 172.21.15.26:0/1244902332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70b40d3910 0x7f70b40d1100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f709c0097a0 tx=0x7f709c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5354a0f064fcbfd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.642+0000 7f70b1400640 1 -- 172.21.15.26:0/1244902332 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c031830 con 0x7f70b40d3910 2024-01-15T17:55:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.642+0000 7f70b1400640 1 -- 172.21.15.26:0/1244902332 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c031990 con 0x7f70b40d3910 2024-01-15T17:55:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.642+0000 7f70b1400640 1 -- 172.21.15.26:0/1244902332 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c031d20 con 0x7f70b40d3910 2024-01-15T17:55:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.643+0000 7f70b9000640 1 -- 172.21.15.26:0/1244902332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70b40d3910 msgr2=0x7f70b40d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.643+0000 7f70b9000640 1 --2- 172.21.15.26:0/1244902332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70b40d3910 0x7f70b40d1100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f709c0097a0 tx=0x7f709c02b8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.643+0000 7f70b9000640 1 -- 172.21.15.26:0/1244902332 shutdown_connections 2024-01-15T17:55:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.643+0000 7f70b9000640 1 --2- 172.21.15.26:0/1244902332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70b40d3910 0x7f70b40d1100 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.643+0000 7f70b9000640 1 -- 172.21.15.26:0/1244902332 >> 172.21.15.26:0/1244902332 conn(0x7f70b409c530 msgr2=0x7f70b40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.643+0000 7f70b9000640 1 -- 172.21.15.26:0/1244902332 shutdown_connections 2024-01-15T17:55:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.643+0000 7f70b9000640 1 -- 172.21.15.26:0/1244902332 wait complete. 2024-01-15T17:55:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.645+0000 7f70b9000640 1 Processor -- start 2024-01-15T17:55:01.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.645+0000 7f70b9000640 1 -- start start 2024-01-15T17:55:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b9000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b40cfcf0 0x7f70b41f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b9000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70b40d3910 0x7f70b40c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b9000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70b4160e10 0x7f70b415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b9000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b41f3f20 con 0x7f70b40cfcf0 2024-01-15T17:55:01.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b9000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40c90b0 con 0x7f70b4160e10 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b9000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40c8780 con 0x7f70b40d3910 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b40cfcf0 0x7f70b41f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b40cfcf0 0x7f70b41f39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55660/0 (socket says 172.21.15.26:55660) 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b1e00640 1 -- 172.21.15.26:0/2873608851 learned_addr learned my addr 172.21.15.26:0/2873608851 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.646+0000 7f70b2800640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70b4160e10 0x7f70b415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70abe00640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70b40d3910 0x7f70b40c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70b1e00640 1 -- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70b40d3910 msgr2=0x7f70b40c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70b1e00640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70b40d3910 0x7f70b40c9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70b1e00640 1 -- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70b4160e10 msgr2=0x7f70b415cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70b1e00640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70b4160e10 0x7f70b415cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70b1e00640 1 -- 172.21.15.26:0/2873608851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c009400 con 0x7f70b40cfcf0 2024-01-15T17:55:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70abe00640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70b40d3910 0x7f70b40c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70b1e00640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b40cfcf0 0x7f70b41f39e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c4c0 tx=0x7f70a400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70aa000640 1 -- 172.21.15.26:0/2873608851 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40052d0 con 0x7f70b40cfcf0 2024-01-15T17:55:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70aa000640 1 -- 172.21.15.26:0/2873608851 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a4005430 con 0x7f70b40cfcf0 2024-01-15T17:55:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70aa000640 1 -- 172.21.15.26:0/2873608851 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4012870 con 0x7f70b40cfcf0 2024-01-15T17:55:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.647+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b40c8a60 con 0x7f70b40cfcf0 2024-01-15T17:55:01.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.648+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b40b7060 con 0x7f70b40cfcf0 2024-01-15T17:55:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.649+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b40675d0 con 0x7f70b40cfcf0 2024-01-15T17:55:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.649+0000 7f70aa000640 1 -- 172.21.15.26:0/2873608851 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f70a40129d0 con 0x7f70b40cfcf0 2024-01-15T17:55:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.651+0000 7f70aa000640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f708c06b2f0 0x7f708c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.651+0000 7f70aa000640 1 -- 172.21.15.26:0/2873608851 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f70a40893b0 con 0x7f70b40cfcf0 2024-01-15T17:55:01.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.651+0000 7f70abe00640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f708c06b2f0 0x7f708c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:01.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.652+0000 7f70abe00640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f708c06b2f0 0x7f708c06d7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f709c036000 tx=0x7f709c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:01.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.655+0000 7f70aa000640 1 -- 172.21.15.26:0/2873608851 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70a4054560 con 0x7f70b40cfcf0 2024-01-15T17:55:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.851+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f70b40b2d00 con 0x7f70b40cfcf0 2024-01-15T17:55:01.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.852+0000 7f70aa000640 1 -- 172.21.15.26:0/2873608851 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f70a4057b80 con 0x7f70b40cfcf0 2024-01-15T17:55:01.855 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:01.855 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T17:55:01.855 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:55:01.856 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-15T17:55:01.856 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-15T17:55:01.856 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:55:01.856 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T17:55:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.854+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f708c06b2f0 msgr2=0x7f708c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f708c06b2f0 0x7f708c06d7b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f709c036000 tx=0x7f709c008030 comp rx=0 tx=0).stop 2024-01-15T17:55:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b40cfcf0 msgr2=0x7f70b41f39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b40cfcf0 0x7f70b41f39e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c4c0 tx=0x7f70a400c980 comp rx=0 tx=0).stop 2024-01-15T17:55:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 shutdown_connections 2024-01-15T17:55:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70b40d3910 0x7f70b40c9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f708c06b2f0 0x7f708c06d7b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70b4160e10 0x7f70b415cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 --2- 172.21.15.26:0/2873608851 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70b40cfcf0 0x7f70b41f39e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 >> 172.21.15.26:0/2873608851 conn(0x7f70b409c530 msgr2=0x7f70b40d1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 shutdown_connections 2024-01-15T17:55:01.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:01.855+0000 7f70b9000640 1 -- 172.21.15.26:0/2873608851 wait complete. 2024-01-15T17:55:01.876 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T17:55:01.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T17:55:02.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.095+0000 7efd37e00640 1 Processor -- start 2024-01-15T17:55:02.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.096+0000 7efd37e00640 1 -- start start 2024-01-15T17:55:02.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.097+0000 7efd37e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd3815d440 0x7efd380d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.097+0000 7efd37e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380cfd00 con 0x7efd3806b700 2024-01-15T17:55:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.097+0000 7efd37e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380cffe0 con 0x7efd3806c220 2024-01-15T17:55:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.097+0000 7efd37e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380cfaf0 con 0x7efd3815d440 2024-01-15T17:55:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.098+0000 7efd36a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd3815d440 0x7efd380d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.098+0000 7efd36a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd3815d440 0x7efd380d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57402/0 (socket says 172.21.15.26:57402) 2024-01-15T17:55:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.098+0000 7efd36000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efd3806b700 0x7efd3806bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58640/0 (socket says 172.21.15.26:58640) 2024-01-15T17:55:02.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.098+0000 7efd36a00640 1 -- 172.21.15.26:0/947044351 learned_addr learned my addr 172.21.15.26:0/947044351 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.098+0000 7efd36a00640 1 -- 172.21.15.26:0/947044351 >> v1:172.21.15.124:6789/0 conn(0x7efd3806c220 legacy=0x7efd380cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.098+0000 7efd36a00640 1 -- 172.21.15.26:0/947044351 >> v1:172.21.15.26:6789/0 conn(0x7efd3806b700 legacy=0x7efd3806bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.098+0000 7efd36a00640 1 -- 172.21.15.26:0/947044351 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380d02c0 con 0x7efd3815d440 2024-01-15T17:55:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.098+0000 7efd35600640 1 -- 172.21.15.26:0/947044351 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275736299 0 0) 0x7efd380cfd00 con 0x7efd3806b700 2024-01-15T17:55:02.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.099+0000 7efd36a00640 1 --2- 172.21.15.26:0/947044351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 0x7efd380d1900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd28009e50 tx=0x7efd2802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d42bcedfc04404 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:02.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.100+0000 7efd35600640 1 -- 172.21.15.26:0/947044351 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2802cb20 con 0x7efd3815d440 2024-01-15T17:55:02.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.100+0000 7efd35600640 1 -- 172.21.15.26:0/947044351 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2802cc80 con 0x7efd3815d440 2024-01-15T17:55:02.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.100+0000 7efd35600640 1 -- 172.21.15.26:0/947044351 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd280343f0 con 0x7efd3815d440 2024-01-15T17:55:02.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.101+0000 7efd37e00640 1 -- 172.21.15.26:0/947044351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 msgr2=0x7efd380d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.101+0000 7efd37e00640 1 --2- 172.21.15.26:0/947044351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 0x7efd380d1900 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd28009e50 tx=0x7efd2802bf80 comp rx=0 tx=0).stop 2024-01-15T17:55:02.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.101+0000 7efd37e00640 1 -- 172.21.15.26:0/947044351 shutdown_connections 2024-01-15T17:55:02.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.101+0000 7efd37e00640 1 --2- 172.21.15.26:0/947044351 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 0x7efd380d1900 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.101+0000 7efd37e00640 1 -- 172.21.15.26:0/947044351 >> 172.21.15.26:0/947044351 conn(0x7efd380d0de0 msgr2=0x7efd380d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:02.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.101+0000 7efd37e00640 1 -- 172.21.15.26:0/947044351 shutdown_connections 2024-01-15T17:55:02.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.101+0000 7efd37e00640 1 -- 172.21.15.26:0/947044351 wait complete. 2024-01-15T17:55:02.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.103+0000 7efd37e00640 1 Processor -- start 2024-01-15T17:55:02.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.103+0000 7efd37e00640 1 -- start start 2024-01-15T17:55:02.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd3806b700 0x7efd381f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd3806c220 0x7efd380c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 0x7efd380c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380c9630 con 0x7efd3815d440 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd381f28e0 con 0x7efd3806c220 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380c8a20 con 0x7efd3806b700 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 0x7efd380c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 0x7efd380c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57412/0 (socket says 172.21.15.26:57412) 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd36a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd3806b700 0x7efd381f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37400640 1 -- 172.21.15.26:0/889676526 learned_addr learned my addr 172.21.15.26:0/889676526 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd36000640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd3806c220 0x7efd380c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.104+0000 7efd37400640 1 -- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd3806b700 msgr2=0x7efd381f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd37400640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd3806b700 0x7efd381f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd37400640 1 -- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd3806c220 msgr2=0x7efd380c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd37400640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd3806c220 0x7efd380c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd37400640 1 -- 172.21.15.26:0/889676526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd28009b30 con 0x7efd3815d440 2024-01-15T17:55:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd37400640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 0x7efd380c83f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c410 tx=0x7efd3000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd2ea00640 1 -- 172.21.15.26:0/889676526 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30011790 con 0x7efd3815d440 2024-01-15T17:55:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd2ea00640 1 -- 172.21.15.26:0/889676526 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd300118f0 con 0x7efd3815d440 2024-01-15T17:55:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd380c8d00 con 0x7efd3815d440 2024-01-15T17:55:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd2ea00640 1 -- 172.21.15.26:0/889676526 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30011ce0 con 0x7efd3815d440 2024-01-15T17:55:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.105+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd380c92c0 con 0x7efd3815d440 2024-01-15T17:55:02.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.106+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd380c97e0 con 0x7efd3815d440 2024-01-15T17:55:02.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.107+0000 7efd2ea00640 1 -- 172.21.15.26:0/889676526 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7efd3001c460 con 0x7efd3815d440 2024-01-15T17:55:02.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.108+0000 7efd2ea00640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd1006b2c0 0x7efd1006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.108+0000 7efd2ea00640 1 -- 172.21.15.26:0/889676526 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efd300894d0 con 0x7efd3815d440 2024-01-15T17:55:02.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.109+0000 7efd36a00640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd1006b2c0 0x7efd1006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.110+0000 7efd36a00640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd1006b2c0 0x7efd1006d780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd28035000 tx=0x7efd28030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:02.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.113+0000 7efd2ea00640 1 -- 172.21.15.26:0/889676526 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd380c97e0 con 0x7efd3815d440 2024-01-15T17:55:02.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.310+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efd38067450 con 0x7efd3815d440 2024-01-15T17:55:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.691+0000 7efd2ea00640 1 -- 172.21.15.26:0/889676526 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7efd30054620 con 0x7efd3815d440 2024-01-15T17:55:02.694 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-15T17:55:02.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd1006b2c0 msgr2=0x7efd1006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd1006b2c0 0x7efd1006d780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efd28035000 tx=0x7efd28030000 comp rx=0 tx=0).stop 2024-01-15T17:55:02.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 msgr2=0x7efd380c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 0x7efd380c83f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c410 tx=0x7efd3000c8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 shutdown_connections 2024-01-15T17:55:02.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd1006b2c0 0x7efd1006d780 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd3815d440 0x7efd380c83f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd3806c220 0x7efd380c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 --2- 172.21.15.26:0/889676526 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd3806b700 0x7efd381f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.695+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 >> 172.21.15.26:0/889676526 conn(0x7efd380d0de0 msgr2=0x7efd380cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:02.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.696+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 shutdown_connections 2024-01-15T17:55:02.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.696+0000 7efd37e00640 1 -- 172.21.15.26:0/889676526 wait complete. 2024-01-15T17:55:02.725 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T17:55:02.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.930+0000 7f0a8ba00640 1 Processor -- start 2024-01-15T17:55:02.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.930+0000 7f0a8ba00640 1 -- start start 2024-01-15T17:55:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.932+0000 7f0a8ba00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0a8415d320 0x7f0a840d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.932+0000 7f0a8ba00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a840b61b0 con 0x7f0a840d1d20 2024-01-15T17:55:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.932+0000 7f0a8ba00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a840b6450 con 0x7f0a840d02a0 2024-01-15T17:55:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.932+0000 7f0a8ba00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a840b6730 con 0x7f0a8415d320 2024-01-15T17:55:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.932+0000 7f0a7be00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0a840d1d20 0x7f0a840cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58648/0 (socket says 172.21.15.26:58648) 2024-01-15T17:55:02.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.932+0000 7f0a7be00640 1 -- 172.21.15.26:0/3864078608 learned_addr learned my addr 172.21.15.26:0/3864078608 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.932+0000 7f0a89000640 1 --2- 172.21.15.26:0/3864078608 >> v2:172.21.15.124:3300/0 conn(0x7f0a8415d320 0x7f0a840d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.933+0000 7f0a83e00640 1 -- 172.21.15.26:0/3864078608 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355244289 0 0) 0x7f0a840b61b0 con 0x7f0a840d1d20 2024-01-15T17:55:02.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.933+0000 7f0a83e00640 1 -- 172.21.15.26:0/3864078608 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a64003520 con 0x7f0a840d1d20 2024-01-15T17:55:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.933+0000 7f0a89000640 1 -- 172.21.15.26:0/3864078608 >> v1:172.21.15.124:6789/0 conn(0x7f0a840d02a0 legacy=0x7f0a841670e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.933+0000 7f0a89000640 1 -- 172.21.15.26:0/3864078608 >> v1:172.21.15.26:6789/0 conn(0x7f0a840d1d20 legacy=0x7f0a840cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.933+0000 7f0a89000640 1 -- 172.21.15.26:0/3864078608 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840b6a10 con 0x7f0a8415d320 2024-01-15T17:55:02.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.934+0000 7f0a89000640 1 --2- 172.21.15.26:0/3864078608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a8415d320 0x7f0a840d17e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a740097a0 tx=0x7f0a7402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9d92869d88d499 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.935+0000 7f0a83e00640 1 -- 172.21.15.26:0/3864078608 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74031830 con 0x7f0a8415d320 2024-01-15T17:55:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.935+0000 7f0a83e00640 1 -- 172.21.15.26:0/3864078608 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74031990 con 0x7f0a8415d320 2024-01-15T17:55:02.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.935+0000 7f0a83e00640 1 -- 172.21.15.26:0/3864078608 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74031d20 con 0x7f0a8415d320 2024-01-15T17:55:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.936+0000 7f0a8ba00640 1 -- 172.21.15.26:0/3864078608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a8415d320 msgr2=0x7f0a840d17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.936+0000 7f0a8ba00640 1 --2- 172.21.15.26:0/3864078608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a8415d320 0x7f0a840d17e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0a740097a0 tx=0x7f0a7402b8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.936+0000 7f0a8ba00640 1 -- 172.21.15.26:0/3864078608 shutdown_connections 2024-01-15T17:55:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.936+0000 7f0a8ba00640 1 --2- 172.21.15.26:0/3864078608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a8415d320 0x7f0a840d17e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.936+0000 7f0a8ba00640 1 -- 172.21.15.26:0/3864078608 >> 172.21.15.26:0/3864078608 conn(0x7f0a8409c570 msgr2=0x7f0a840d3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:02.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.936+0000 7f0a8ba00640 1 -- 172.21.15.26:0/3864078608 shutdown_connections 2024-01-15T17:55:02.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.936+0000 7f0a8ba00640 1 -- 172.21.15.26:0/3864078608 wait complete. 2024-01-15T17:55:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.938+0000 7f0a8ba00640 1 Processor -- start 2024-01-15T17:55:02.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.939+0000 7f0a8ba00640 1 -- start start 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.940+0000 7f0a8ba00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a840d02a0 0x7f0a8414b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.940+0000 7f0a8ba00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a840d1d20 0x7f0a8414a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.940+0000 7f0a8ba00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a8415d320 0x7f0a84148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.940+0000 7f0a8ba00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8414a8e0 con 0x7f0a840d02a0 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.940+0000 7f0a8ba00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a84149300 con 0x7f0a8415d320 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.940+0000 7f0a8ba00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841495e0 con 0x7f0a840d1d20 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.940+0000 7f0a89000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a840d02a0 0x7f0a8414b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a89000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a840d02a0 0x7f0a8414b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57422/0 (socket says 172.21.15.26:57422) 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a89000640 1 -- 172.21.15.26:0/61680625 learned_addr learned my addr 172.21.15.26:0/61680625 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a7be00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a840d1d20 0x7f0a8414a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a89a00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a8415d320 0x7f0a84148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a89000640 1 -- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a840d1d20 msgr2=0x7f0a8414a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a89000640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a840d1d20 0x7f0a8414a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a89000640 1 -- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a8415d320 msgr2=0x7f0a84148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a89000640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a8415d320 0x7f0a84148ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a89000640 1 -- 172.21.15.26:0/61680625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74009400 con 0x7f0a840d02a0 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a7be00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a840d1d20 0x7f0a8414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.941+0000 7f0a89000640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a840d02a0 0x7f0a8414b700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a74034040 tx=0x7f0a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.942+0000 7f0a81600640 1 -- 172.21.15.26:0/61680625 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7403a9e0 con 0x7f0a840d02a0 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.942+0000 7f0a81600640 1 -- 172.21.15.26:0/61680625 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7403ab60 con 0x7f0a840d02a0 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.942+0000 7f0a81600640 1 -- 172.21.15.26:0/61680625 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74044400 con 0x7f0a840d02a0 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.942+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a841498c0 con 0x7f0a840d02a0 2024-01-15T17:55:02.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.942+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a84149e80 con 0x7f0a840d02a0 2024-01-15T17:55:02.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.943+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a84067670 con 0x7f0a840d02a0 2024-01-15T17:55:02.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.945+0000 7f0a81600640 1 -- 172.21.15.26:0/61680625 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0a7404d440 con 0x7f0a840d02a0 2024-01-15T17:55:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.946+0000 7f0a81600640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a5c06b2c0 0x7f0a5c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.946+0000 7f0a81600640 1 -- 172.21.15.26:0/61680625 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f0a740ac150 con 0x7f0a840d02a0 2024-01-15T17:55:02.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.946+0000 7f0a7be00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a5c06b2c0 0x7f0a5c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:02.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.948+0000 7f0a7be00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a5c06b2c0 0x7f0a5c06d780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a840b3e70 tx=0x7f0a6c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:02.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:02.951+0000 7f0a81600640 1 -- 172.21.15.26:0/61680625 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a74077210 con 0x7f0a840d02a0 2024-01-15T17:55:03.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.146+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0a8414a040 con 0x7f0a840d02a0 2024-01-15T17:55:03.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.709+0000 7f0a81600640 1 -- 172.21.15.26:0/61680625 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0a740350d0 con 0x7f0a840d02a0 2024-01-15T17:55:03.713 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-15T17:55:03.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.714+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a5c06b2c0 msgr2=0x7f0a5c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:03.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.714+0000 7f0a8ba00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a5c06b2c0 0x7f0a5c06d780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a840b3e70 tx=0x7f0a6c009040 comp rx=0 tx=0).stop 2024-01-15T17:55:03.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.714+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a840d02a0 msgr2=0x7f0a8414b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:03.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.714+0000 7f0a8ba00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a840d02a0 0x7f0a8414b700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0a74034040 tx=0x7f0a740046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.715+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 shutdown_connections 2024-01-15T17:55:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.715+0000 7f0a8ba00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a840d1d20 0x7f0a8414a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.715+0000 7f0a8ba00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a5c06b2c0 0x7f0a5c06d780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.715+0000 7f0a8ba00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a8415d320 0x7f0a84148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.715+0000 7f0a8ba00640 1 --2- 172.21.15.26:0/61680625 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a840d02a0 0x7f0a8414b700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.715+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 >> 172.21.15.26:0/61680625 conn(0x7f0a8409c570 msgr2=0x7f0a840b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:03.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.718+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 shutdown_connections 2024-01-15T17:55:03.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.719+0000 7f0a8ba00640 1 -- 172.21.15.26:0/61680625 wait complete. 2024-01-15T17:55:03.760 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T17:55:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.944+0000 7f9133e00640 1 Processor -- start 2024-01-15T17:55:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.944+0000 7f9133e00640 1 -- start start 2024-01-15T17:55:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.945+0000 7f9133e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 0x7f913413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f9133e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91340cddf0 0x7f91340b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f9133e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91340ce440 con 0x7f913413fae0 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f9133e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913413ff60 con 0x7f91340cddf0 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f9133e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340ce720 con 0x7f913413cf10 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f9133400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91340cddf0 0x7f91340b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f912be00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f913413fae0 0x7f91340cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58664/0 (socket says 172.21.15.26:58664) 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f912be00640 1 -- 172.21.15.26:0/672581779 learned_addr learned my addr 172.21.15.26:0/672581779 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f9132a00640 1 --2- 172.21.15.26:0/672581779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 0x7f913413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f9133400640 1 -- 172.21.15.26:0/672581779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 msgr2=0x7f913413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f9133400640 1 --2- 172.21.15.26:0/672581779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 0x7f913413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.946+0000 7f9133400640 1 -- 172.21.15.26:0/672581779 >> v1:172.21.15.26:6789/0 conn(0x7f913413fae0 legacy=0x7f91340cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.947+0000 7f9133400640 1 -- 172.21.15.26:0/672581779 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340cea00 con 0x7f91340cddf0 2024-01-15T17:55:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.947+0000 7f9132000640 1 -- 172.21.15.26:0/672581779 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832160107 0 0) 0x7f91340ce440 con 0x7f913413fae0 2024-01-15T17:55:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.948+0000 7f9133400640 1 --2- 172.21.15.26:0/672581779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cddf0 0x7f91340b2590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f912c009d90 tx=0x7f912c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca39ec5018d2029 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.948+0000 7f9132000640 1 -- 172.21.15.26:0/672581779 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c02cc60 con 0x7f91340cddf0 2024-01-15T17:55:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.948+0000 7f9132000640 1 -- 172.21.15.26:0/672581779 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c02cdc0 con 0x7f91340cddf0 2024-01-15T17:55:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.948+0000 7f9132000640 1 -- 172.21.15.26:0/672581779 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c034660 con 0x7f91340cddf0 2024-01-15T17:55:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.948+0000 7f9133e00640 1 -- 172.21.15.26:0/672581779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cddf0 msgr2=0x7f91340b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.948+0000 7f9133e00640 1 --2- 172.21.15.26:0/672581779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cddf0 0x7f91340b2590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f912c009d90 tx=0x7f912c02beb0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.949+0000 7f9133e00640 1 -- 172.21.15.26:0/672581779 shutdown_connections 2024-01-15T17:55:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.949+0000 7f9133e00640 1 --2- 172.21.15.26:0/672581779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340cddf0 0x7f91340b2590 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.949+0000 7f9133e00640 1 --2- 172.21.15.26:0/672581779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 0x7f913413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.949+0000 7f9133e00640 1 -- 172.21.15.26:0/672581779 >> 172.21.15.26:0/672581779 conn(0x7f913406a7d0 msgr2=0x7f913406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.949+0000 7f9133e00640 1 -- 172.21.15.26:0/672581779 shutdown_connections 2024-01-15T17:55:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.949+0000 7f9133e00640 1 -- 172.21.15.26:0/672581779 wait complete. 2024-01-15T17:55:03.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.951+0000 7f9133e00640 1 Processor -- start 2024-01-15T17:55:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.951+0000 7f9133e00640 1 -- start start 2024-01-15T17:55:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91340cddf0 0x7f913415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 0x7f913415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f913413fae0 0x7f9134158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91341f3550 con 0x7f913413fae0 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913413d6b0 con 0x7f91340cddf0 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913415d530 con 0x7f913413cf10 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f913413fae0 0x7f9134158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f913413fae0 0x7f9134158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57434/0 (socket says 172.21.15.26:57434) 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133400640 1 -- 172.21.15.26:0/526840279 learned_addr learned my addr 172.21.15.26:0/526840279 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f912be00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 0x7f913415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9132a00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91340cddf0 0x7f913415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133400640 1 -- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 msgr2=0x7f913415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133400640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 0x7f913415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133400640 1 -- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91340cddf0 msgr2=0x7f913415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133400640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91340cddf0 0x7f913415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.952+0000 7f9133400640 1 -- 172.21.15.26:0/526840279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c009a70 con 0x7f913413fae0 2024-01-15T17:55:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.953+0000 7f9132a00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91340cddf0 0x7f913415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.953+0000 7f9133400640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f913413fae0 0x7f9134158860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f912c009eb0 tx=0x7f912c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.953+0000 7f912aa00640 1 -- 172.21.15.26:0/526840279 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c003b90 con 0x7f913413fae0 2024-01-15T17:55:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.953+0000 7f912aa00640 1 -- 172.21.15.26:0/526840279 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c003cf0 con 0x7f913413fae0 2024-01-15T17:55:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.953+0000 7f912aa00640 1 -- 172.21.15.26:0/526840279 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c004270 con 0x7f913413fae0 2024-01-15T17:55:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.953+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9134138790 con 0x7f913413fae0 2024-01-15T17:55:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.953+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9134138490 con 0x7f913413fae0 2024-01-15T17:55:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.955+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913415b3d0 con 0x7f913413fae0 2024-01-15T17:55:03.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.956+0000 7f912aa00640 1 -- 172.21.15.26:0/526840279 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f912c03b050 con 0x7f913413fae0 2024-01-15T17:55:03.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.957+0000 7f912aa00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06b2f0 0x7f910c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.957+0000 7f912aa00640 1 -- 172.21.15.26:0/526840279 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f912c0ac150 con 0x7f913413fae0 2024-01-15T17:55:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.957+0000 7f9132a00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06b2f0 0x7f910c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.958+0000 7f9132a00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06b2f0 0x7f910c06d7b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9124006fd0 tx=0x7f9124008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:03.962+0000 7f912aa00640 1 -- 172.21.15.26:0/526840279 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f912c076f80 con 0x7f913413fae0 2024-01-15T17:55:04.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.193+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f91341f3690 con 0x7f913413fae0 2024-01-15T17:55:04.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.714+0000 7f912aa00640 1 -- 172.21.15.26:0/526840279 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f912c07a5a0 con 0x7f913413fae0 2024-01-15T17:55:04.717 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 2 and data pool 3 2024-01-15T17:55:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.718+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06b2f0 msgr2=0x7f910c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.718+0000 7f9133e00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06b2f0 0x7f910c06d7b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9124006fd0 tx=0x7f9124008040 comp rx=0 tx=0).stop 2024-01-15T17:55:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.718+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f913413fae0 msgr2=0x7f9134158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.718+0000 7f9133e00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f913413fae0 0x7f9134158860 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f912c009eb0 tx=0x7f912c004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.719+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 shutdown_connections 2024-01-15T17:55:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.719+0000 7f9133e00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f913413cf10 0x7f913415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.719+0000 7f9133e00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06b2f0 0x7f910c06d7b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.719+0000 7f9133e00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f913413fae0 0x7f9134158860 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.719+0000 7f9133e00640 1 --2- 172.21.15.26:0/526840279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f91340cddf0 0x7f913415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:04.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.719+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 >> 172.21.15.26:0/526840279 conn(0x7f913406a7d0 msgr2=0x7f91340adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:04.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.720+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 shutdown_connections 2024-01-15T17:55:04.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.721+0000 7f9133e00640 1 -- 172.21.15.26:0/526840279 wait complete. 2024-01-15T17:55:04.758 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.942+0000 7f774f400640 1 Processor -- start 2024-01-15T17:55:04.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.942+0000 7f774f400640 1 -- start start 2024-01-15T17:55:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.943+0000 7f774f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77480ae5a0 0x7f774815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.944+0000 7f774f400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f77480cf290 0x7f77480cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.944+0000 7f774f400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7748070c40 con 0x7f77480b3ae0 2024-01-15T17:55:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.944+0000 7f774f400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7748071450 con 0x7f77480ae5a0 2024-01-15T17:55:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.944+0000 7f774f400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77480cfc60 con 0x7f77480cf290 2024-01-15T17:55:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.944+0000 7f774ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77480ae5a0 0x7f774815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.944+0000 7f774ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77480ae5a0 0x7f774815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57438/0 (socket says 172.21.15.26:57438) 2024-01-15T17:55:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.944+0000 7f774ca00640 1 -- 172.21.15.26:0/3615997582 learned_addr learned my addr 172.21.15.26:0/3615997582 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:04.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.944+0000 7f7747e00640 1 --1- 172.21.15.26:0/3615997582 >> v1:172.21.15.26:6789/0 conn(0x7f77480b3ae0 0x7f774806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58680/0 (socket says 172.21.15.26:58680) 2024-01-15T17:55:04.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.944+0000 7f774d400640 1 --2- 172.21.15.26:0/3615997582 >> v2:172.21.15.124:3300/0 conn(0x7f77480cf290 0x7f77480cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:04.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.945+0000 7f774ca00640 1 -- 172.21.15.26:0/3615997582 >> v2:172.21.15.124:3300/0 conn(0x7f77480cf290 msgr2=0x7f77480cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:04.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.945+0000 7f774ca00640 1 --2- 172.21.15.26:0/3615997582 >> v2:172.21.15.124:3300/0 conn(0x7f77480cf290 0x7f77480cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:04.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.945+0000 7f774ca00640 1 -- 172.21.15.26:0/3615997582 >> v1:172.21.15.26:6789/0 conn(0x7f77480b3ae0 legacy=0x7f774806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:04.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.945+0000 7f774ca00640 1 -- 172.21.15.26:0/3615997582 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774806c470 con 0x7f77480ae5a0 2024-01-15T17:55:04.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.945+0000 7f7747400640 1 -- 172.21.15.26:0/3615997582 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612504634 0 0) 0x7f7748070c40 con 0x7f77480b3ae0 2024-01-15T17:55:04.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.947+0000 7f774ca00640 1 --2- 172.21.15.26:0/3615997582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480ae5a0 0x7f774815e430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f77480b3f70 tx=0x7f773802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83ee50c5166133c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:04.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.947+0000 7f7747400640 1 -- 172.21.15.26:0/3615997582 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773802f850 con 0x7f77480ae5a0 2024-01-15T17:55:04.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.947+0000 7f7747400640 1 -- 172.21.15.26:0/3615997582 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773802f9b0 con 0x7f77480ae5a0 2024-01-15T17:55:04.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.947+0000 7f7747400640 1 -- 172.21.15.26:0/3615997582 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773802fd40 con 0x7f77480ae5a0 2024-01-15T17:55:04.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.948+0000 7f774f400640 1 -- 172.21.15.26:0/3615997582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480ae5a0 msgr2=0x7f774815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:04.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.948+0000 7f774f400640 1 --2- 172.21.15.26:0/3615997582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480ae5a0 0x7f774815e430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f77480b3f70 tx=0x7f773802ba30 comp rx=0 tx=0).stop 2024-01-15T17:55:04.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.948+0000 7f774f400640 1 -- 172.21.15.26:0/3615997582 shutdown_connections 2024-01-15T17:55:04.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.948+0000 7f774f400640 1 --2- 172.21.15.26:0/3615997582 >> v2:172.21.15.124:3300/0 conn(0x7f77480cf290 0x7f77480cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:04.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.948+0000 7f774f400640 1 --2- 172.21.15.26:0/3615997582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480ae5a0 0x7f774815e430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:04.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.948+0000 7f774f400640 1 -- 172.21.15.26:0/3615997582 >> 172.21.15.26:0/3615997582 conn(0x7f774806a800 msgr2=0x7f774806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:04.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.948+0000 7f774f400640 1 -- 172.21.15.26:0/3615997582 shutdown_connections 2024-01-15T17:55:04.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.948+0000 7f774f400640 1 -- 172.21.15.26:0/3615997582 wait complete. 2024-01-15T17:55:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.950+0000 7f774f400640 1 Processor -- start 2024-01-15T17:55:04.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.950+0000 7f774f400640 1 -- start start 2024-01-15T17:55:04.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f774f400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77480ae5a0 0x7f7748150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f774f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480b3ae0 0x7f7748156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f774f400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77480cf290 0x7f7748152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f774f400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7748157d40 con 0x7f77480b3ae0 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f774f400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7748157780 con 0x7f77480ae5a0 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f774f400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7748157a60 con 0x7f77480cf290 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f7747e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480b3ae0 0x7f7748156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f7747e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480b3ae0 0x7f7748156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57442/0 (socket says 172.21.15.26:57442) 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f7747e00640 1 -- 172.21.15.26:0/3549623244 learned_addr learned my addr 172.21.15.26:0/3549623244 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f774ca00640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77480ae5a0 0x7f7748150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f7747e00640 1 -- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77480cf290 msgr2=0x7f7748152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f774d400640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77480cf290 0x7f7748152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.951+0000 7f7747e00640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77480cf290 0x7f7748152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f7747e00640 1 -- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77480ae5a0 msgr2=0x7f7748150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f7747e00640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77480ae5a0 0x7f7748150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:04.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f7747e00640 1 -- 172.21.15.26:0/3549623244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77380094d0 con 0x7f77480b3ae0 2024-01-15T17:55:04.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f774ca00640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77480ae5a0 0x7f7748150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:04.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f7747e00640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480b3ae0 0x7f7748156e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f773000c4d0 tx=0x7f773000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f7744c00640 1 -- 172.21.15.26:0/3549623244 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77300074e0 con 0x7f77480b3ae0 2024-01-15T17:55:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f7744c00640 1 -- 172.21.15.26:0/3549623244 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7730007660 con 0x7f77480b3ae0 2024-01-15T17:55:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77481526f0 con 0x7f77480b3ae0 2024-01-15T17:55:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f7744c00640 1 -- 172.21.15.26:0/3549623244 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730004a20 con 0x7f77480b3ae0 2024-01-15T17:55:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.952+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7748150fa0 con 0x7f77480b3ae0 2024-01-15T17:55:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.953+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7748152550 con 0x7f77480b3ae0 2024-01-15T17:55:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.955+0000 7f7744c00640 1 -- 172.21.15.26:0/3549623244 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7730004b80 con 0x7f77480b3ae0 2024-01-15T17:55:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.956+0000 7f7744c00640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f772006b310 0x7f772006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.956+0000 7f7744c00640 1 -- 172.21.15.26:0/3549623244 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7730089210 con 0x7f77480b3ae0 2024-01-15T17:55:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.956+0000 7f774ca00640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f772006b310 0x7f772006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:04.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.957+0000 7f774ca00640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f772006b310 0x7f772006d7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7738036000 tx=0x7f7738008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:04.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:04.961+0000 7f7744c00640 1 -- 172.21.15.26:0/3549623244 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7748152550 con 0x7f77480b3ae0 2024-01-15T17:55:05.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.158+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7748152c10 con 0x7f77480b3ae0 2024-01-15T17:55:05.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.160+0000 7f7744c00640 1 -- 172.21.15.26:0/3549623244 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f7748152c10 con 0x7f77480b3ae0 2024-01-15T17:55:05.163 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:05.164 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:04.195228+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f772006b310 msgr2=0x7f772006d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f772006b310 0x7f772006d7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7738036000 tx=0x7f7738008030 comp rx=0 tx=0).stop 2024-01-15T17:55:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480b3ae0 msgr2=0x7f7748156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480b3ae0 0x7f7748156e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f773000c4d0 tx=0x7f773000c990 comp rx=0 tx=0).stop 2024-01-15T17:55:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 shutdown_connections 2024-01-15T17:55:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77480cf290 0x7f7748152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f772006b310 0x7f772006d7d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77480b3ae0 0x7f7748156e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 --2- 172.21.15.26:0/3549623244 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77480ae5a0 0x7f7748150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 >> 172.21.15.26:0/3549623244 conn(0x7f774806a800 msgr2=0x7f77480b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 shutdown_connections 2024-01-15T17:55:05.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.163+0000 7f774f400640 1 -- 172.21.15.26:0/3549623244 wait complete. 2024-01-15T17:55:05.185 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.396+0000 7f6779200640 1 Processor -- start 2024-01-15T17:55:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.396+0000 7f6779200640 1 -- start start 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.398+0000 7f6779200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f677415afb0 0x7f677413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.398+0000 7f6779200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677413ff30 con 0x7f677413cfa0 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.398+0000 7f6779200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740cdc40 con 0x7f677413fae0 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.398+0000 7f6779200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67740cdec0 con 0x7f677415afb0 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.398+0000 7f6773400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f677413cfa0 0x7f67740cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58692/0 (socket says 172.21.15.26:58692) 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.398+0000 7f6773400640 1 -- 172.21.15.26:0/3196554401 learned_addr learned my addr 172.21.15.26:0/3196554401 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.398+0000 7f676be00640 1 --2- 172.21.15.26:0/3196554401 >> v2:172.21.15.26:3300/0 conn(0x7f677415afb0 0x7f677413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.399+0000 7f6772a00640 1 -- 172.21.15.26:0/3196554401 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334740438 0 0) 0x7f6764001110 con 0x7f677413cfa0 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.399+0000 7f6772a00640 1 -- 172.21.15.26:0/3196554401 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6754003520 con 0x7f677413cfa0 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.399+0000 7f676be00640 1 -- 172.21.15.26:0/3196554401 >> v1:172.21.15.124:6789/0 conn(0x7f677413fae0 legacy=0x7f67740b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.399+0000 7f676be00640 1 -- 172.21.15.26:0/3196554401 >> v1:172.21.15.26:6789/0 conn(0x7f677413cfa0 legacy=0x7f67740cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.399+0000 7f676be00640 1 -- 172.21.15.26:0/3196554401 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740ce1a0 con 0x7f677415afb0 2024-01-15T17:55:05.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.400+0000 7f676be00640 1 --2- 172.21.15.26:0/3196554401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677415afb0 0x7f677413f5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f675c0097a0 tx=0x7f675c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2ea074ad990cfd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:05.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.400+0000 7f6772a00640 1 -- 172.21.15.26:0/3196554401 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c02e070 con 0x7f677415afb0 2024-01-15T17:55:05.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.400+0000 7f6772a00640 1 -- 172.21.15.26:0/3196554401 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c0315a0 con 0x7f677415afb0 2024-01-15T17:55:05.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.400+0000 7f6772a00640 1 -- 172.21.15.26:0/3196554401 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c031910 con 0x7f677415afb0 2024-01-15T17:55:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.401+0000 7f6779200640 1 -- 172.21.15.26:0/3196554401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677415afb0 msgr2=0x7f677413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.401+0000 7f6779200640 1 --2- 172.21.15.26:0/3196554401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677415afb0 0x7f677413f5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f675c0097a0 tx=0x7f675c02b840 comp rx=0 tx=0).stop 2024-01-15T17:55:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.401+0000 7f6779200640 1 -- 172.21.15.26:0/3196554401 shutdown_connections 2024-01-15T17:55:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.401+0000 7f6779200640 1 --2- 172.21.15.26:0/3196554401 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677415afb0 0x7f677413f5a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.401+0000 7f6779200640 1 -- 172.21.15.26:0/3196554401 >> 172.21.15.26:0/3196554401 conn(0x7f677406a800 msgr2=0x7f677406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.401+0000 7f6779200640 1 -- 172.21.15.26:0/3196554401 shutdown_connections 2024-01-15T17:55:05.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.401+0000 7f6779200640 1 -- 172.21.15.26:0/3196554401 wait complete. 2024-01-15T17:55:05.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.403+0000 7f6779200640 1 Processor -- start 2024-01-15T17:55:05.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.403+0000 7f6779200640 1 -- start start 2024-01-15T17:55:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6779200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677413cfa0 0x7f677412ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6779200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f677413fae0 0x7f67741532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6779200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f677415afb0 0x7f67740adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6779200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6774158430 con 0x7f677413cfa0 2024-01-15T17:55:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6779200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67741599e0 con 0x7f677413fae0 2024-01-15T17:55:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6779200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6774159cc0 con 0x7f677415afb0 2024-01-15T17:55:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677413cfa0 0x7f677412ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677413cfa0 0x7f677412ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57458/0 (socket says 172.21.15.26:57458) 2024-01-15T17:55:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773400640 1 -- 172.21.15.26:0/1701492509 learned_addr learned my addr 172.21.15.26:0/1701492509 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773e00640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f677415afb0 0x7f67740adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f676be00640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f677413fae0 0x7f67741532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773400640 1 -- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f677415afb0 msgr2=0x7f67740adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773400640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f677415afb0 0x7f67740adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773400640 1 -- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f677413fae0 msgr2=0x7f67741532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773400640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f677413fae0 0x7f67741532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773400640 1 -- 172.21.15.26:0/1701492509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c009400 con 0x7f677413cfa0 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f6773e00640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f677415afb0 0x7f67740adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.404+0000 7f676be00640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f677413fae0 0x7f67741532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.405+0000 7f6773400640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677413cfa0 0x7f677412ef70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f676400c700 tx=0x7f676400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.405+0000 7f676b400640 1 -- 172.21.15.26:0/1701492509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764005ab0 con 0x7f677413cfa0 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.405+0000 7f676b400640 1 -- 172.21.15.26:0/1701492509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6764005c10 con 0x7f677413cfa0 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.405+0000 7f676b400640 1 -- 172.21.15.26:0/1701492509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764001d50 con 0x7f677413cfa0 2024-01-15T17:55:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.405+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6774159fa0 con 0x7f677413cfa0 2024-01-15T17:55:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.405+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677415b6c0 con 0x7f677413cfa0 2024-01-15T17:55:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.406+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6774067450 con 0x7f677413cfa0 2024-01-15T17:55:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.407+0000 7f676b400640 1 -- 172.21.15.26:0/1701492509 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6764007070 con 0x7f677413cfa0 2024-01-15T17:55:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.408+0000 7f676b400640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f674c06b2c0 0x7f674c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.408+0000 7f676b400640 1 -- 172.21.15.26:0/1701492509 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6764089060 con 0x7f677413cfa0 2024-01-15T17:55:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.409+0000 7f676be00640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f674c06b2c0 0x7f674c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.410+0000 7f676be00640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f674c06b2c0 0x7f674c06d780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f675c036000 tx=0x7f675c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.412+0000 7f676b400640 1 -- 172.21.15.26:0/1701492509 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67640bbe90 con 0x7f677413cfa0 2024-01-15T17:55:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.608+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f677413ff30 con 0x7f677413cfa0 2024-01-15T17:55:05.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.611+0000 7f676b400640 1 -- 172.21.15.26:0/1701492509 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16494 (secure 0 0 0) 0x7f677413ff30 con 0x7f677413cfa0 2024-01-15T17:55:05.615 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:05.615 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":21,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:04.195228+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f674c06b2c0 msgr2=0x7f674c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f674c06b2c0 0x7f674c06d780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f675c036000 tx=0x7f675c03d040 comp rx=0 tx=0).stop 2024-01-15T17:55:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677413cfa0 msgr2=0x7f677412ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677413cfa0 0x7f677412ef70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f676400c700 tx=0x7f676400cbc0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 shutdown_connections 2024-01-15T17:55:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f677415afb0 0x7f67740adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f674c06b2c0 0x7f674c06d780 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f677413fae0 0x7f67741532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 --2- 172.21.15.26:0/1701492509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f677413cfa0 0x7f677412ef70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 >> 172.21.15.26:0/1701492509 conn(0x7f677406a800 msgr2=0x7f6774070a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 shutdown_connections 2024-01-15T17:55:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.614+0000 7f6779200640 1 -- 172.21.15.26:0/1701492509 wait complete. 2024-01-15T17:55:05.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T17:55:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.854+0000 7ff2b1600640 1 Processor -- start 2024-01-15T17:55:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.854+0000 7ff2b1600640 1 -- start start 2024-01-15T17:55:05.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.856+0000 7ff2b1600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2ac0adf20 0x7ff2ac070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.856+0000 7ff2b1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 0x7ff2ac143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.856+0000 7ff2b1600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac15f4e0 con 0x7ff2ac15bf10 2024-01-15T17:55:05.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.856+0000 7ff2b1600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2ac15f970 con 0x7ff2ac0adf20 2024-01-15T17:55:05.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.856+0000 7ff2b1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac071130 con 0x7ff2ac0ccb50 2024-01-15T17:55:05.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.856+0000 7ff2abe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2ac0adf20 0x7ff2ac070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.857+0000 7ff2abe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2ac0adf20 0x7ff2ac070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57470/0 (socket says 172.21.15.26:57470) 2024-01-15T17:55:05.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.857+0000 7ff2abe00640 1 -- 172.21.15.26:0/3653524308 learned_addr learned my addr 172.21.15.26:0/3653524308 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:05.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.857+0000 7ff2b0c00640 1 --2- 172.21.15.26:0/3653524308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 0x7ff2ac143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.857+0000 7ff2abe00640 1 -- 172.21.15.26:0/3653524308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 msgr2=0x7ff2ac143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.857+0000 7ff2abe00640 1 --2- 172.21.15.26:0/3653524308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 0x7ff2ac143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.857+0000 7ff2abe00640 1 -- 172.21.15.26:0/3653524308 >> v1:172.21.15.124:6789/0 conn(0x7ff2ac15bf10 legacy=0x7ff2ac0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.857+0000 7ff2abe00640 1 -- 172.21.15.26:0/3653524308 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac15c390 con 0x7ff2ac0adf20 2024-01-15T17:55:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.858+0000 7ff2abe00640 1 --2- 172.21.15.26:0/3653524308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 0x7ff2ac070a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009870 tx=0x7ff29c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3444400c08bf6f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.859+0000 7ff2aaa00640 1 -- 172.21.15.26:0/3653524308 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c031640 con 0x7ff2ac0adf20 2024-01-15T17:55:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.859+0000 7ff2aaa00640 1 -- 172.21.15.26:0/3653524308 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c0317a0 con 0x7ff2ac0adf20 2024-01-15T17:55:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.859+0000 7ff2aaa00640 1 -- 172.21.15.26:0/3653524308 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c031b30 con 0x7ff2ac0adf20 2024-01-15T17:55:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.859+0000 7ff2b1600640 1 -- 172.21.15.26:0/3653524308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 msgr2=0x7ff2ac070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.859+0000 7ff2b1600640 1 --2- 172.21.15.26:0/3653524308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 0x7ff2ac070a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009870 tx=0x7ff29c02b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.860+0000 7ff2b1600640 1 -- 172.21.15.26:0/3653524308 shutdown_connections 2024-01-15T17:55:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.860+0000 7ff2b1600640 1 --2- 172.21.15.26:0/3653524308 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 0x7ff2ac143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.860+0000 7ff2b1600640 1 --2- 172.21.15.26:0/3653524308 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 0x7ff2ac070a30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.860+0000 7ff2b1600640 1 -- 172.21.15.26:0/3653524308 >> 172.21.15.26:0/3653524308 conn(0x7ff2ac06a7d0 msgr2=0x7ff2ac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.860+0000 7ff2b1600640 1 -- 172.21.15.26:0/3653524308 shutdown_connections 2024-01-15T17:55:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.860+0000 7ff2b1600640 1 -- 172.21.15.26:0/3653524308 wait complete. 2024-01-15T17:55:05.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.861+0000 7ff2b1600640 1 Processor -- start 2024-01-15T17:55:05.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.861+0000 7ff2b1600640 1 -- start start 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.862+0000 7ff2b1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 0x7ff2ac0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.862+0000 7ff2b1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 0x7ff2ac1423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.862+0000 7ff2b1600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2ac15bf10 0x7ff2ac15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.862+0000 7ff2b1600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0ca0c0 con 0x7ff2ac0adf20 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.862+0000 7ff2b1600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0c9020 con 0x7ff2ac15bf10 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.862+0000 7ff2b1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0c9300 con 0x7ff2ac0ccb50 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 0x7ff2ac0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2abe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 0x7ff2ac0ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57480/0 (socket says 172.21.15.26:57480) 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2abe00640 1 -- 172.21.15.26:0/858690036 learned_addr learned my addr 172.21.15.26:0/858690036 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2ab400640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 0x7ff2ac1423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2b0c00640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2ac15bf10 0x7ff2ac15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2abe00640 1 -- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 msgr2=0x7ff2ac1423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2abe00640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 0x7ff2ac1423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2abe00640 1 -- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2ac15bf10 msgr2=0x7ff2ac15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2abe00640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2ac15bf10 0x7ff2ac15a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.863+0000 7ff2abe00640 1 -- 172.21.15.26:0/858690036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c0094d0 con 0x7ff2ac0adf20 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.864+0000 7ff2abe00640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 0x7ff2ac0ca710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff29c000c00 tx=0x7ff29c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.864+0000 7ff2a3e00640 1 -- 172.21.15.26:0/858690036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c03a930 con 0x7ff2ac0adf20 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.864+0000 7ff2a3e00640 1 -- 172.21.15.26:0/858690036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c03aa90 con 0x7ff2ac0adf20 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.864+0000 7ff2a3e00640 1 -- 172.21.15.26:0/858690036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c03ae40 con 0x7ff2ac0adf20 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.864+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac0c95e0 con 0x7ff2ac0adf20 2024-01-15T17:55:05.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.864+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac0c9ba0 con 0x7ff2ac0adf20 2024-01-15T17:55:05.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.865+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac0c9d90 con 0x7ff2ac0adf20 2024-01-15T17:55:05.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.866+0000 7ff2a3e00640 1 -- 172.21.15.26:0/858690036 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff29c02f050 con 0x7ff2ac0adf20 2024-01-15T17:55:05.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.867+0000 7ff2a3e00640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff28406b2c0 0x7ff28406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:05.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.867+0000 7ff2a3e00640 1 -- 172.21.15.26:0/858690036 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff29c0ab9c0 con 0x7ff2ac0adf20 2024-01-15T17:55:05.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.868+0000 7ff2ab400640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff28406b2c0 0x7ff28406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:05.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.869+0000 7ff2ab400640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff28406b2c0 0x7ff28406d780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac141990 tx=0x7ff294009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:05.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:05.872+0000 7ff2a3e00640 1 -- 172.21.15.26:0/858690036 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff29c076790 con 0x7ff2ac0adf20 2024-01-15T17:55:06.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.112+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff2ac067450 con 0x7ff2ac0adf20 2024-01-15T17:55:06.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.744+0000 7ff2a3e00640 1 -- 172.21.15.26:0/858690036 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff29c079db0 con 0x7ff2ac0adf20 2024-01-15T17:55:06.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff28406b2c0 msgr2=0x7ff28406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff28406b2c0 0x7ff28406d780 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac141990 tx=0x7ff294009040 comp rx=0 tx=0).stop 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 msgr2=0x7ff2ac0ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 0x7ff2ac0ca710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff29c000c00 tx=0x7ff29c0046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 shutdown_connections 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2ac0ccb50 0x7ff2ac1423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff28406b2c0 0x7ff28406d780 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2ac15bf10 0x7ff2ac15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 --2- 172.21.15.26:0/858690036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2ac0adf20 0x7ff2ac0ca710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.748+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 >> 172.21.15.26:0/858690036 conn(0x7ff2ac06a7d0 msgr2=0x7ff2ac0cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.749+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 shutdown_connections 2024-01-15T17:55:06.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.749+0000 7ff2b1600640 1 -- 172.21.15.26:0/858690036 wait complete. 2024-01-15T17:55:06.773 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T17:55:06.773 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T17:55:06.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.988+0000 7f3861800640 1 Processor -- start 2024-01-15T17:55:06.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.988+0000 7f3861800640 1 -- start start 2024-01-15T17:55:06.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.990+0000 7f3861800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f385c13e360 0x7f385c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:06.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.990+0000 7f3861800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c15f9c0 0x7f385c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:06.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.990+0000 7f3861800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c15c5f0 con 0x7f385c13df80 2024-01-15T17:55:06.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.990+0000 7f3861800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c06c470 con 0x7f385c13e360 2024-01-15T17:55:06.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.990+0000 7f3861800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c06c750 con 0x7f385c15f9c0 2024-01-15T17:55:06.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.990+0000 7f385aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f385c13df80 0x7f385c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58696/0 (socket says 172.21.15.26:58696) 2024-01-15T17:55:06.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.990+0000 7f385aa00640 1 -- 172.21.15.26:0/523993028 learned_addr learned my addr 172.21.15.26:0/523993028 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:06.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.991+0000 7f385a000640 1 --2- 172.21.15.26:0/523993028 >> v2:172.21.15.124:3300/0 conn(0x7f385c13e360 0x7f385c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:06.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.991+0000 7f385b400640 1 --2- 172.21.15.26:0/523993028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c15f9c0 0x7f385c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:06.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.991+0000 7f3859600640 1 -- 172.21.15.26:0/523993028 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143856935 0 0) 0x7f385c15c5f0 con 0x7f385c13df80 2024-01-15T17:55:06.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.991+0000 7f3859600640 1 -- 172.21.15.26:0/523993028 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f383c003520 con 0x7f385c13df80 2024-01-15T17:55:06.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.991+0000 7f385b400640 1 -- 172.21.15.26:0/523993028 >> v2:172.21.15.124:3300/0 conn(0x7f385c13e360 msgr2=0x7f385c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:06.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.991+0000 7f385b400640 1 --2- 172.21.15.26:0/523993028 >> v2:172.21.15.124:3300/0 conn(0x7f385c13e360 0x7f385c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:06.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.991+0000 7f385b400640 1 -- 172.21.15.26:0/523993028 >> v1:172.21.15.26:6789/0 conn(0x7f385c13df80 legacy=0x7f385c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:06.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.991+0000 7f385b400640 1 -- 172.21.15.26:0/523993028 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c06ca30 con 0x7f385c15f9c0 2024-01-15T17:55:06.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.993+0000 7f385b400640 1 --2- 172.21.15.26:0/523993028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c15f9c0 0x7f385c0cf7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3854009d90 tx=0x7f385402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5db9b5771fc7c1e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:06.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.993+0000 7f3859600640 1 -- 172.21.15.26:0/523993028 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385402cc60 con 0x7f385c15f9c0 2024-01-15T17:55:06.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.993+0000 7f3859600640 1 -- 172.21.15.26:0/523993028 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385402cdc0 con 0x7f385c15f9c0 2024-01-15T17:55:06.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.993+0000 7f3859600640 1 -- 172.21.15.26:0/523993028 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854034660 con 0x7f385c15f9c0 2024-01-15T17:55:06.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.994+0000 7f3861800640 1 -- 172.21.15.26:0/523993028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c15f9c0 msgr2=0x7f385c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:06.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.994+0000 7f3861800640 1 --2- 172.21.15.26:0/523993028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c15f9c0 0x7f385c0cf7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3854009d90 tx=0x7f385402beb0 comp rx=0 tx=0).stop 2024-01-15T17:55:06.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.994+0000 7f3861800640 1 -- 172.21.15.26:0/523993028 shutdown_connections 2024-01-15T17:55:06.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.994+0000 7f3861800640 1 --2- 172.21.15.26:0/523993028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c15f9c0 0x7f385c0cf7d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:06.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.994+0000 7f3861800640 1 --2- 172.21.15.26:0/523993028 >> v2:172.21.15.124:3300/0 conn(0x7f385c13e360 0x7f385c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:06.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.994+0000 7f3861800640 1 -- 172.21.15.26:0/523993028 >> 172.21.15.26:0/523993028 conn(0x7f385c06a800 msgr2=0x7f385c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:06.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.994+0000 7f3861800640 1 -- 172.21.15.26:0/523993028 shutdown_connections 2024-01-15T17:55:06.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.994+0000 7f3861800640 1 -- 172.21.15.26:0/523993028 wait complete. 2024-01-15T17:55:06.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.996+0000 7f3861800640 1 Processor -- start 2024-01-15T17:55:06.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.996+0000 7f3861800640 1 -- start start 2024-01-15T17:55:07.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.997+0000 7f3861800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f385c13df80 0x7f385c1e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:07.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.997+0000 7f3861800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c13e360 0x7f385c0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:07.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.997+0000 7f3861800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f385c15f9c0 0x7f385c13c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:07.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.997+0000 7f3861800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1f4f30 con 0x7f385c13df80 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.997+0000 7f3861800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1f4320 con 0x7f385c15f9c0 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.997+0000 7f3861800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1f4600 con 0x7f385c13e360 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.997+0000 7f385aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f385c13df80 0x7f385c1e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.997+0000 7f385aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f385c13df80 0x7f385c1e4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57482/0 (socket says 172.21.15.26:57482) 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.997+0000 7f385aa00640 1 -- 172.21.15.26:0/892947827 learned_addr learned my addr 172.21.15.26:0/892947827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f385a000640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c13e360 0x7f385c0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f385b400640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f385c15f9c0 0x7f385c13c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f385aa00640 1 -- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c13e360 msgr2=0x7f385c0b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f385aa00640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c13e360 0x7f385c0b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f385aa00640 1 -- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f385c15f9c0 msgr2=0x7f385c13c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f385aa00640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f385c15f9c0 0x7f385c13c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f385aa00640 1 -- 172.21.15.26:0/892947827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854009a70 con 0x7f385c13df80 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f385aa00640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f385c13df80 0x7f385c1e4ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c400 tx=0x7f384c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:07.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f3852a00640 1 -- 172.21.15.26:0/892947827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c005ce0 con 0x7f385c13df80 2024-01-15T17:55:07.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f3852a00640 1 -- 172.21.15.26:0/892947827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c005e40 con 0x7f385c13df80 2024-01-15T17:55:07.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c1f48e0 con 0x7f385c13df80 2024-01-15T17:55:07.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.998+0000 7f3852a00640 1 -- 172.21.15.26:0/892947827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c0072d0 con 0x7f385c13df80 2024-01-15T17:55:07.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:06.999+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c13d900 con 0x7f385c13df80 2024-01-15T17:55:07.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.000+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385c0675d0 con 0x7f385c13df80 2024-01-15T17:55:07.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.001+0000 7f3852a00640 1 -- 172.21.15.26:0/892947827 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f384c0074b0 con 0x7f385c13df80 2024-01-15T17:55:07.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.002+0000 7f3852a00640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f383406b2c0 0x7f383406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:07.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.002+0000 7f3852a00640 1 -- 172.21.15.26:0/892947827 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f384c0890d0 con 0x7f385c13df80 2024-01-15T17:55:07.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.002+0000 7f385a000640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f383406b2c0 0x7f383406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:07.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.003+0000 7f385a000640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f383406b2c0 0x7f383406d780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f385c06d830 tx=0x7f3844008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:07.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.006+0000 7f3852a00640 1 -- 172.21.15.26:0/892947827 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f384c053fe0 con 0x7f385c13df80 2024-01-15T17:55:07.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.243+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f385c06df00 con 0x7f385c13df80 2024-01-15T17:55:07.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.750+0000 7f3852a00640 1 -- 172.21.15.26:0/892947827 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f384c057600 con 0x7f385c13df80 2024-01-15T17:55:07.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f383406b2c0 msgr2=0x7f383406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:07.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f383406b2c0 0x7f383406d780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f385c06d830 tx=0x7f3844008040 comp rx=0 tx=0).stop 2024-01-15T17:55:07.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f385c13df80 msgr2=0x7f385c1e4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:07.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f385c13df80 0x7f385c1e4ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f384c00c400 tx=0x7f384c00c8c0 comp rx=0 tx=0).stop 2024-01-15T17:55:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 shutdown_connections 2024-01-15T17:55:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f385c13e360 0x7f385c0b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f383406b2c0 0x7f383406d780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f385c15f9c0 0x7f385c13c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 --2- 172.21.15.26:0/892947827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f385c13df80 0x7f385c1e4ae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 >> 172.21.15.26:0/892947827 conn(0x7f385c06a800 msgr2=0x7f385c0cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 shutdown_connections 2024-01-15T17:55:07.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.754+0000 7f3861800640 1 -- 172.21.15.26:0/892947827 wait complete. 2024-01-15T17:55:07.778 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T17:55:07.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:55:08.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.998+0000 7f8973e00640 1 Processor -- start 2024-01-15T17:55:08.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:07.999+0000 7f8973e00640 1 -- start start 2024-01-15T17:55:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.000+0000 7f8973e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8974143c40 0x7f8974071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.000+0000 7f8973e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89740717d0 0x7f89740cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.000+0000 7f8973e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8974070e50 con 0x7f89740cca40 2024-01-15T17:55:08.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.000+0000 7f8973e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89740723c0 con 0x7f8974143c40 2024-01-15T17:55:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.000+0000 7f8973e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89740cce40 con 0x7f89740717d0 2024-01-15T17:55:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.000+0000 7f8972a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8974143c40 0x7f8974071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.001+0000 7f8972a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8974143c40 0x7f8974071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57484/0 (socket says 172.21.15.26:57484) 2024-01-15T17:55:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.001+0000 7f8972a00640 1 -- 172.21.15.26:0/1555402217 learned_addr learned my addr 172.21.15.26:0/1555402217 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.001+0000 7f8969e00640 1 --2- 172.21.15.26:0/1555402217 >> v2:172.21.15.124:3300/0 conn(0x7f89740717d0 0x7f89740cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.001+0000 7f8972a00640 1 -- 172.21.15.26:0/1555402217 >> v2:172.21.15.124:3300/0 conn(0x7f89740717d0 msgr2=0x7f89740cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.001+0000 7f8972a00640 1 --2- 172.21.15.26:0/1555402217 >> v2:172.21.15.124:3300/0 conn(0x7f89740717d0 0x7f89740cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.001+0000 7f8972a00640 1 -- 172.21.15.26:0/1555402217 >> v1:172.21.15.124:6789/0 conn(0x7f89740cca40 legacy=0x7f897415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.001+0000 7f8972a00640 1 -- 172.21.15.26:0/1555402217 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897406e170 con 0x7f8974143c40 2024-01-15T17:55:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.001+0000 7f8969e00640 1 --2- 172.21.15.26:0/1555402217 >> v2:172.21.15.124:3300/0 conn(0x7f89740717d0 0x7f89740cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.002+0000 7f8972a00640 1 --2- 172.21.15.26:0/1555402217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8974143c40 0x7f8974071290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f89740b44a0 tx=0x7f896402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2b7d13af94c9f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:08.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.002+0000 7f8972000640 1 -- 172.21.15.26:0/1555402217 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640317e0 con 0x7f8974143c40 2024-01-15T17:55:08.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.003+0000 7f8972000640 1 -- 172.21.15.26:0/1555402217 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8964031940 con 0x7f8974143c40 2024-01-15T17:55:08.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.003+0000 7f8972000640 1 -- 172.21.15.26:0/1555402217 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964031cd0 con 0x7f8974143c40 2024-01-15T17:55:08.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.003+0000 7f8973e00640 1 -- 172.21.15.26:0/1555402217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8974143c40 msgr2=0x7f8974071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.003+0000 7f8973e00640 1 --2- 172.21.15.26:0/1555402217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8974143c40 0x7f8974071290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f89740b44a0 tx=0x7f896402b880 comp rx=0 tx=0).stop 2024-01-15T17:55:08.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.003+0000 7f8973e00640 1 -- 172.21.15.26:0/1555402217 shutdown_connections 2024-01-15T17:55:08.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.003+0000 7f8973e00640 1 --2- 172.21.15.26:0/1555402217 >> v2:172.21.15.124:3300/0 conn(0x7f89740717d0 0x7f89740cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.003+0000 7f8973e00640 1 --2- 172.21.15.26:0/1555402217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8974143c40 0x7f8974071290 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.003+0000 7f8973e00640 1 -- 172.21.15.26:0/1555402217 >> 172.21.15.26:0/1555402217 conn(0x7f897406a800 msgr2=0x7f897406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:08.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.004+0000 7f8973e00640 1 -- 172.21.15.26:0/1555402217 shutdown_connections 2024-01-15T17:55:08.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.004+0000 7f8973e00640 1 -- 172.21.15.26:0/1555402217 wait complete. 2024-01-15T17:55:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.005+0000 7f8973e00640 1 Processor -- start 2024-01-15T17:55:08.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.006+0000 7f8973e00640 1 -- start start 2024-01-15T17:55:08.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.006+0000 7f8973e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89740717d0 0x7f897412f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.006+0000 7f8973e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740cca40 0x7f8974130600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.006+0000 7f8973e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8974143c40 0x7f89740ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8973e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897412f980 con 0x7f89740717d0 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8973e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8974133180 con 0x7f8974143c40 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8973e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89741310c0 con 0x7f89740cca40 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8972a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89740717d0 0x7f897412f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8972a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89740717d0 0x7f897412f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57490/0 (socket says 172.21.15.26:57490) 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8972a00640 1 -- 172.21.15.26:0/3882528780 learned_addr learned my addr 172.21.15.26:0/3882528780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8969e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740cca40 0x7f8974130600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8973400640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8974143c40 0x7f89740ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8972a00640 1 -- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740cca40 msgr2=0x7f8974130600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8972a00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740cca40 0x7f8974130600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8972a00640 1 -- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8974143c40 msgr2=0x7f89740ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8972a00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8974143c40 0x7f89740ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8972a00640 1 -- 172.21.15.26:0/3882528780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89640094d0 con 0x7f89740717d0 2024-01-15T17:55:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8973400640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8974143c40 0x7f89740ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.007+0000 7f8969e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740cca40 0x7f8974130600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:08.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.008+0000 7f8972a00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89740717d0 0x7f897412f440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8964000c00 tx=0x7f89640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.008+0000 7f896b400640 1 -- 172.21.15.26:0/3882528780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640329c0 con 0x7f89740717d0 2024-01-15T17:55:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.008+0000 7f896b400640 1 -- 172.21.15.26:0/3882528780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8964032b40 con 0x7f89740717d0 2024-01-15T17:55:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.008+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89741313a0 con 0x7f89740717d0 2024-01-15T17:55:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.008+0000 7f896b400640 1 -- 172.21.15.26:0/3882528780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640453f0 con 0x7f89740717d0 2024-01-15T17:55:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.008+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8974135f00 con 0x7f89740717d0 2024-01-15T17:55:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.009+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8974133350 con 0x7f89740717d0 2024-01-15T17:55:08.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.010+0000 7f896b400640 1 -- 172.21.15.26:0/3882528780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8964045550 con 0x7f89740717d0 2024-01-15T17:55:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.012+0000 7f896b400640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f894c06b2c0 0x7f894c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.012+0000 7f896b400640 1 -- 172.21.15.26:0/3882528780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89640ac0f0 con 0x7f89740717d0 2024-01-15T17:55:08.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.012+0000 7f8969e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f894c06b2c0 0x7f894c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.013+0000 7f8969e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f894c06b2c0 0x7f894c06d780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89740af900 tx=0x7f895c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:08.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.016+0000 7f896b400640 1 -- 172.21.15.26:0/3882528780 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8964076fe0 con 0x7f89740717d0 2024-01-15T17:55:08.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.253+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8974067450 con 0x7f89740717d0 2024-01-15T17:55:08.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.255+0000 7f896b400640 1 -- 172.21.15.26:0/3882528780 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f896407a600 con 0x7f89740717d0 2024-01-15T17:55:08.258 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:08.258 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/593578542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":593578542},{"type":"v1","addr":"172.21.15.124:6837","nonce":593578542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1776325954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1776325954},{"type":"v1","addr":"172.21.15.26:6835","nonce":1776325954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:55:04.195734+0000","modified":"2024-01-15T17:55:07.244782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/3022017251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3022017251},{"type":"v1","addr":"172.21.15.124:6835","nonce":3022017251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T17:55:08.258 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.257+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f894c06b2c0 msgr2=0x7f894c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.257+0000 7f8973e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f894c06b2c0 0x7f894c06d780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f89740af900 tx=0x7f895c008040 comp rx=0 tx=0).stop 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.257+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89740717d0 msgr2=0x7f897412f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.257+0000 7f8973e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89740717d0 0x7f897412f440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8964000c00 tx=0x7f89640046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.257+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 shutdown_connections 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.258+0000 7f8973e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89740cca40 0x7f8974130600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.258+0000 7f8973e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f894c06b2c0 0x7f894c06d780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.258+0000 7f8973e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8974143c40 0x7f89740ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.258+0000 7f8973e00640 1 --2- 172.21.15.26:0/3882528780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89740717d0 0x7f897412f440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.258+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 >> 172.21.15.26:0/3882528780 conn(0x7f897406a800 msgr2=0x7f8974142560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.258+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 shutdown_connections 2024-01-15T17:55:08.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.258+0000 7f8973e00640 1 -- 172.21.15.26:0/3882528780 wait complete. 2024-01-15T17:55:08.279 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:08.306 INFO:tasks.ceph.mds.b.smithi124.stderr:2024-01-15T17:55:08.303+0000 7f3934800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T17:55:08.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.493+0000 7fdc92600640 1 Processor -- start 2024-01-15T17:55:08.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.494+0000 7fdc92600640 1 -- start start 2024-01-15T17:55:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.495+0000 7fdc92600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdc8c071410 0x7fdc8c146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.496+0000 7fdc92600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c0b2e10 con 0x7fdc8c071f00 2024-01-15T17:55:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.496+0000 7fdc92600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8c06cc50 con 0x7fdc8c06c7c0 2024-01-15T17:55:08.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.496+0000 7fdc92600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8c06cec0 con 0x7fdc8c071410 2024-01-15T17:55:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.496+0000 7fdc91c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdc8c071f00 0x7fdc8c166cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58704/0 (socket says 172.21.15.26:58704) 2024-01-15T17:55:08.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.496+0000 7fdc91c00640 1 -- 172.21.15.26:0/1601667960 learned_addr learned my addr 172.21.15.26:0/1601667960 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.496+0000 7fdc91200640 1 --2- 172.21.15.26:0/1601667960 >> v2:172.21.15.124:3300/0 conn(0x7fdc8c071410 0x7fdc8c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.496+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064285968 0 0) 0x7fdc8c0b2e10 con 0x7fdc8c071f00 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.496+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc6c003520 con 0x7fdc8c071f00 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3577391980 0 0) 0x7fdc6c003520 con 0x7fdc8c071f00 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 >> v2:172.21.15.124:3300/0 conn(0x7fdc8c071410 msgr2=0x7fdc8c146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 --2- 172.21.15.26:0/1601667960 >> v2:172.21.15.124:3300/0 conn(0x7fdc8c071410 0x7fdc8c146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 >> v1:172.21.15.124:6789/0 conn(0x7fdc8c06c7c0 legacy=0x7fdc8c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c06d1a0 con 0x7fdc8c071f00 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fdc840019f0 con 0x7fdc8c071f00 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 >> v1:172.21.15.26:6789/0 conn(0x7fdc8c071f00 legacy=0x7fdc8c166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc6c0039c0 0x7fdc6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c0047c0 0x7fdc6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc6c005180 0x7fdc6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c06cec0 con 0x7fdc6c005180 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c00c080 con 0x7fdc6c0047c0 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c00c240 con 0x7fdc6c0039c0 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc91c00640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc6c005180 0x7fdc6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc83e00640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c0047c0 0x7fdc6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.497+0000 7fdc91200640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc6c0039c0 0x7fdc6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.498+0000 7fdc91c00640 1 -- 172.21.15.26:0/1601667960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc6c0039c0 msgr2=0x7fdc6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.498+0000 7fdc91c00640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc6c0039c0 0x7fdc6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.498+0000 7fdc91c00640 1 -- 172.21.15.26:0/1601667960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c0047c0 msgr2=0x7fdc6c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.498+0000 7fdc91c00640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c0047c0 0x7fdc6c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.498+0000 7fdc91c00640 1 -- 172.21.15.26:0/1601667960 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c003520 con 0x7fdc6c005180 2024-01-15T17:55:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.499+0000 7fdc91c00640 1 -- 172.21.15.26:0/1601667960 reap_dead start 2024-01-15T17:55:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.499+0000 7fdc91c00640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc6c005180 0x7fdc6c00b980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400a760 tx=0x7fdc6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fd6cc3fded1a49 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.499+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84034630 con 0x7fdc6c005180 2024-01-15T17:55:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.499+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8402f070 con 0x7fdc6c005180 2024-01-15T17:55:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.499+0000 7fdc8be00640 1 -- 172.21.15.26:0/1601667960 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84034a10 con 0x7fdc6c005180 2024-01-15T17:55:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.500+0000 7fdc92600640 1 -- 172.21.15.26:0/1601667960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc6c005180 msgr2=0x7fdc6c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.500+0000 7fdc92600640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc6c005180 0x7fdc6c00b980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400a760 tx=0x7fdc6c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.500+0000 7fdc92600640 1 -- 172.21.15.26:0/1601667960 shutdown_connections 2024-01-15T17:55:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.500+0000 7fdc92600640 1 --2- 172.21.15.26:0/1601667960 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc6c005180 0x7fdc6c00b980 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.500+0000 7fdc92600640 1 -- 172.21.15.26:0/1601667960 >> 172.21.15.26:0/1601667960 conn(0x7fdc8c06a800 msgr2=0x7fdc8c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.500+0000 7fdc92600640 1 -- 172.21.15.26:0/1601667960 shutdown_connections 2024-01-15T17:55:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.500+0000 7fdc92600640 1 -- 172.21.15.26:0/1601667960 wait complete. 2024-01-15T17:55:08.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.502+0000 7fdc92600640 1 Processor -- start 2024-01-15T17:55:08.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.502+0000 7fdc92600640 1 -- start start 2024-01-15T17:55:08.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc92600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c005180 0x7fdc8c140d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc92600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc8c1412b0 0x7fdc8c0c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc92600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc8c0c4ed0 0x7fdc8c1f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc92600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c0c5380 con 0x7fdc8c1412b0 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc92600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c141a40 con 0x7fdc6c005180 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc92600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8c1f34b0 con 0x7fdc8c0c4ed0 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc83e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc8c1412b0 0x7fdc8c0c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc83e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc8c1412b0 0x7fdc8c0c4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57502/0 (socket says 172.21.15.26:57502) 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc83e00640 1 -- 172.21.15.26:0/3074401294 learned_addr learned my addr 172.21.15.26:0/3074401294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc91200640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c005180 0x7fdc8c140d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc91c00640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc8c0c4ed0 0x7fdc8c1f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc83e00640 1 -- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc8c0c4ed0 msgr2=0x7fdc8c1f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc83e00640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc8c0c4ed0 0x7fdc8c1f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc83e00640 1 -- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c005180 msgr2=0x7fdc8c140d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.503+0000 7fdc83e00640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c005180 0x7fdc8c140d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.504+0000 7fdc83e00640 1 -- 172.21.15.26:0/3074401294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c1f3790 con 0x7fdc8c1412b0 2024-01-15T17:55:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.504+0000 7fdc91200640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c005180 0x7fdc8c140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.504+0000 7fdc91c00640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc8c0c4ed0 0x7fdc8c1f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.504+0000 7fdc83e00640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc8c1412b0 0x7fdc8c0c4990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400e480 tx=0x7fdc7400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:08.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.504+0000 7fdc89600640 1 -- 172.21.15.26:0/3074401294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc740052e0 con 0x7fdc8c1412b0 2024-01-15T17:55:08.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.504+0000 7fdc89600640 1 -- 172.21.15.26:0/3074401294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74005440 con 0x7fdc8c1412b0 2024-01-15T17:55:08.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.504+0000 7fdc89600640 1 -- 172.21.15.26:0/3074401294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc74003830 con 0x7fdc8c1412b0 2024-01-15T17:55:08.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.504+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8c1f3a70 con 0x7fdc8c1412b0 2024-01-15T17:55:08.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.504+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8c1f4030 con 0x7fdc8c1412b0 2024-01-15T17:55:08.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.505+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc8c1f41f0 con 0x7fdc8c1412b0 2024-01-15T17:55:08.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.506+0000 7fdc89600640 1 -- 172.21.15.26:0/3074401294 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdc74003990 con 0x7fdc8c1412b0 2024-01-15T17:55:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.507+0000 7fdc89600640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc6406b2f0 0x7fdc6406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.508+0000 7fdc89600640 1 -- 172.21.15.26:0/3074401294 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc74089970 con 0x7fdc8c1412b0 2024-01-15T17:55:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.508+0000 7fdc91200640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc6406b2f0 0x7fdc6406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.509+0000 7fdc91200640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc6406b2f0 0x7fdc6406d7b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c1430d0 tx=0x7fdc7c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:08.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.512+0000 7fdc89600640 1 -- 172.21.15.26:0/3074401294 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc740547d0 con 0x7fdc8c1412b0 2024-01-15T17:55:08.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.711+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc8c067450 con 0x7fdc8c1412b0 2024-01-15T17:55:08.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.713+0000 7fdc89600640 1 -- 172.21.15.26:0/3074401294 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fdc74057df0 con 0x7fdc8c1412b0 2024-01-15T17:55:08.716 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:08.716 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc6406b2f0 msgr2=0x7fdc6406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc6406b2f0 0x7fdc6406d7b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c1430d0 tx=0x7fdc7c008040 comp rx=0 tx=0).stop 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc8c1412b0 msgr2=0x7fdc8c0c4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc8c1412b0 0x7fdc8c0c4990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400e480 tx=0x7fdc7400e940 comp rx=0 tx=0).stop 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 shutdown_connections 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc8c0c4ed0 0x7fdc8c1f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc6406b2f0 0x7fdc6406d7b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc8c1412b0 0x7fdc8c0c4990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 --2- 172.21.15.26:0/3074401294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc6c005180 0x7fdc8c140d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 >> 172.21.15.26:0/3074401294 conn(0x7fdc8c06a800 msgr2=0x7fdc8c0b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.715+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 shutdown_connections 2024-01-15T17:55:08.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.716+0000 7fdc92600640 1 -- 172.21.15.26:0/3074401294 wait complete. 2024-01-15T17:55:08.739 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T17:55:08.739 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:55:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.954+0000 7f9db4e00640 1 Processor -- start 2024-01-15T17:55:08.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.954+0000 7f9db4e00640 1 -- start start 2024-01-15T17:55:08.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.956+0000 7f9db4e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9db00ae5a0 0x7f9db015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.956+0000 7f9db4e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 0x7f9db006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.956+0000 7f9db4e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db0070c40 con 0x7f9db006d120 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.956+0000 7f9db4e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db0071450 con 0x7f9db00ae5a0 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.956+0000 7f9db4e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db006c470 con 0x7f9db00b3ae0 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.956+0000 7f9dade00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9db00ae5a0 0x7f9db015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.956+0000 7f9dade00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9db00ae5a0 0x7f9db015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57518/0 (socket says 172.21.15.26:57518) 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.956+0000 7f9dade00640 1 -- 172.21.15.26:0/1517387743 learned_addr learned my addr 172.21.15.26:0/1517387743 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.957+0000 7f9dad400640 1 --2- 172.21.15.26:0/1517387743 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 0x7f9db006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.957+0000 7f9dade00640 1 -- 172.21.15.26:0/1517387743 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 msgr2=0x7f9db006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.957+0000 7f9dade00640 1 --2- 172.21.15.26:0/1517387743 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 0x7f9db006cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.957+0000 7f9dade00640 1 -- 172.21.15.26:0/1517387743 >> v1:172.21.15.124:6789/0 conn(0x7f9db006d120 legacy=0x7f9db00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:08.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.957+0000 7f9dade00640 1 -- 172.21.15.26:0/1517387743 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db006c750 con 0x7f9db00ae5a0 2024-01-15T17:55:08.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.958+0000 7f9dade00640 1 --2- 172.21.15.26:0/1517387743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db015e430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009e50 tx=0x7f9da002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2afc9643b08326 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:08.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.958+0000 7f9daca00640 1 -- 172.21.15.26:0/1517387743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da002ca70 con 0x7f9db00ae5a0 2024-01-15T17:55:08.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.958+0000 7f9daca00640 1 -- 172.21.15.26:0/1517387743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da002cbd0 con 0x7f9db00ae5a0 2024-01-15T17:55:08.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.958+0000 7f9daca00640 1 -- 172.21.15.26:0/1517387743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da00343f0 con 0x7f9db00ae5a0 2024-01-15T17:55:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.959+0000 7f9db4e00640 1 -- 172.21.15.26:0/1517387743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 msgr2=0x7f9db015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.959+0000 7f9db4e00640 1 --2- 172.21.15.26:0/1517387743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db015e430 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9da0009e50 tx=0x7f9da002bed0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.959+0000 7f9db4e00640 1 -- 172.21.15.26:0/1517387743 shutdown_connections 2024-01-15T17:55:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.959+0000 7f9db4e00640 1 --2- 172.21.15.26:0/1517387743 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00b3ae0 0x7f9db006cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.959+0000 7f9db4e00640 1 --2- 172.21.15.26:0/1517387743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00ae5a0 0x7f9db015e430 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.959+0000 7f9db4e00640 1 -- 172.21.15.26:0/1517387743 >> 172.21.15.26:0/1517387743 conn(0x7f9db006a800 msgr2=0x7f9db006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.959+0000 7f9db4e00640 1 -- 172.21.15.26:0/1517387743 shutdown_connections 2024-01-15T17:55:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.959+0000 7f9db4e00640 1 -- 172.21.15.26:0/1517387743 wait complete. 2024-01-15T17:55:08.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.961+0000 7f9db4e00640 1 Processor -- start 2024-01-15T17:55:08.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.962+0000 7f9db4e00640 1 -- start start 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9db4e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db006d120 0x7f9db0153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9db4e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00ae5a0 0x7f9db00b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9db4e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9db4e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00cb950 con 0x7f9db00b3ae0 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9db4e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0157270 con 0x7f9db006d120 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9db4e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0158120 con 0x7f9db00ae5a0 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9dae800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9dae800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57532/0 (socket says 172.21.15.26:57532) 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9dae800640 1 -- 172.21.15.26:0/3140598836 learned_addr learned my addr 172.21.15.26:0/3140598836 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9dad400640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00ae5a0 0x7f9db00b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.963+0000 7f9dade00640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db006d120 0x7f9db0153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9dae800640 1 -- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00ae5a0 msgr2=0x7f9db00b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9dae800640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00ae5a0 0x7f9db00b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9dae800640 1 -- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db006d120 msgr2=0x7f9db0153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:08.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9dae800640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db006d120 0x7f9db0153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:08.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9dae800640 1 -- 172.21.15.26:0/3140598836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da0009b30 con 0x7f9db00b3ae0 2024-01-15T17:55:08.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9dae800640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9da800b410 tx=0x7f9da800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:08.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9da6000640 1 -- 172.21.15.26:0/3140598836 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8004020 con 0x7f9db00b3ae0 2024-01-15T17:55:08.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9da6000640 1 -- 172.21.15.26:0/3140598836 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da8004180 con 0x7f9db00b3ae0 2024-01-15T17:55:08.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9da6000640 1 -- 172.21.15.26:0/3140598836 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8011630 con 0x7f9db00b3ae0 2024-01-15T17:55:08.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db0158400 con 0x7f9db00b3ae0 2024-01-15T17:55:08.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.964+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db01589c0 con 0x7f9db00b3ae0 2024-01-15T17:55:08.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.966+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db00cbb00 con 0x7f9db00b3ae0 2024-01-15T17:55:08.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.967+0000 7f9da6000640 1 -- 172.21.15.26:0/3140598836 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9da8011790 con 0x7f9db00b3ae0 2024-01-15T17:55:08.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.968+0000 7f9da6000640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b2f0 0x7f9d8806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:08.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.968+0000 7f9da6000640 1 -- 172.21.15.26:0/3140598836 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9da8089840 con 0x7f9db00b3ae0 2024-01-15T17:55:08.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.968+0000 7f9dade00640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b2f0 0x7f9d8806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:08.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.970+0000 7f9dade00640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b2f0 0x7f9d8806d7b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9da0035000 tx=0x7f9da0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:08.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:08.972+0000 7f9da6000640 1 -- 172.21.15.26:0/3140598836 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9da80546a0 con 0x7f9db00b3ae0 2024-01-15T17:55:09.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.163+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9db0070550 con 0x7f9d8806b2f0 2024-01-15T17:55:09.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.189+0000 7f9da6000640 1 -- 172.21.15.26:0/3140598836 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264485 (secure 0 0 0) 0x7f9da0009510 con 0x7f9d8806b2f0 2024-01-15T17:55:09.196 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:09.198 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:55:09.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b2f0 msgr2=0x7f9d8806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b2f0 0x7f9d8806d7b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9da0035000 tx=0x7f9da0030000 comp rx=0 tx=0).stop 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00b3ae0 msgr2=0x7f9db00b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2cd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9da800b410 tx=0x7f9da800b8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 shutdown_connections 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9db00ae5a0 0x7f9db00b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d8806b2f0 0x7f9d8806d7b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9db00b3ae0 0x7f9db00b2cd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 --2- 172.21.15.26:0/3140598836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9db006d120 0x7f9db0153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 >> 172.21.15.26:0/3140598836 conn(0x7f9db006a800 msgr2=0x7f9db00ad5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 shutdown_connections 2024-01-15T17:55:09.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.196+0000 7f9db4e00640 1 -- 172.21.15.26:0/3140598836 wait complete. 2024-01-15T17:55:09.220 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-15T17:55:07.910271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236160,"kb_used_data":2948,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738560,"statfs":{"total":773094113280,"available":772852285440,"internally_reserved":0,"allocated":3018752,"data_stored":1797664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.979551"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590593+0000","last_change":"2024-01-15T17:55:04.755200+0000","last_active":"2024-01-15T17:55:06.590593+0000","last_peered":"2024-01-15T17:55:06.590593+0000","last_clean":"2024-01-15T17:55:06.590593+0000","last_became_active":"2024-01-15T17:55:04.753634+0000","last_became_peered":"2024-01-15T17:55:04.753634+0000","last_unstale":"2024-01-15T17:55:06.590593+0000","last_undegraded":"2024-01-15T17:55:06.590593+0000","last_fullsized":"2024-01-15T17:55:06.590593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:13.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721381+0000","last_change":"2024-01-15T17:55:04.760607+0000","last_active":"2024-01-15T17:55:05.721381+0000","last_peered":"2024-01-15T17:55:05.721381+0000","last_clean":"2024-01-15T17:55:05.721381+0000","last_became_active":"2024-01-15T17:55:04.760299+0000","last_became_peered":"2024-01-15T17:55:04.760299+0000","last_unstale":"2024-01-15T17:55:05.721381+0000","last_undegraded":"2024-01-15T17:55:05.721381+0000","last_fullsized":"2024-01-15T17:55:05.721381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787705+0000","last_change":"2024-01-15T17:55:04.755000+0000","last_active":"2024-01-15T17:55:06.787705+0000","last_peered":"2024-01-15T17:55:06.787705+0000","last_clean":"2024-01-15T17:55:06.787705+0000","last_became_active":"2024-01-15T17:55:04.754716+0000","last_became_peered":"2024-01-15T17:55:04.754716+0000","last_unstale":"2024-01-15T17:55:06.787705+0000","last_undegraded":"2024-01-15T17:55:06.787705+0000","last_fullsized":"2024-01-15T17:55:06.787705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:55.584774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789609+0000","last_change":"2024-01-15T17:55:04.767626+0000","last_active":"2024-01-15T17:55:06.789609+0000","last_peered":"2024-01-15T17:55:06.789609+0000","last_clean":"2024-01-15T17:55:06.789609+0000","last_became_active":"2024-01-15T17:55:04.767349+0000","last_became_peered":"2024-01-15T17:55:04.767349+0000","last_unstale":"2024-01-15T17:55:06.789609+0000","last_undegraded":"2024-01-15T17:55:06.789609+0000","last_fullsized":"2024-01-15T17:55:06.789609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:17.567853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590635+0000","last_change":"2024-01-15T17:55:04.756971+0000","last_active":"2024-01-15T17:55:06.590635+0000","last_peered":"2024-01-15T17:55:06.590635+0000","last_clean":"2024-01-15T17:55:06.590635+0000","last_became_active":"2024-01-15T17:55:04.756737+0000","last_became_peered":"2024-01-15T17:55:04.756737+0000","last_unstale":"2024-01-15T17:55:06.590635+0000","last_undegraded":"2024-01-15T17:55:06.590635+0000","last_fullsized":"2024-01-15T17:55:06.590635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:44.725915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789155+0000","last_change":"2024-01-15T17:55:04.763592+0000","last_active":"2024-01-15T17:55:06.789155+0000","last_peered":"2024-01-15T17:55:06.789155+0000","last_clean":"2024-01-15T17:55:06.789155+0000","last_became_active":"2024-01-15T17:55:04.763286+0000","last_became_peered":"2024-01-15T17:55:04.763286+0000","last_unstale":"2024-01-15T17:55:06.789155+0000","last_undegraded":"2024-01-15T17:55:06.789155+0000","last_fullsized":"2024-01-15T17:55:06.789155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:48:48.227594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589901+0000","last_change":"2024-01-15T17:55:04.754040+0000","last_active":"2024-01-15T17:55:06.589901+0000","last_peered":"2024-01-15T17:55:06.589901+0000","last_clean":"2024-01-15T17:55:06.589901+0000","last_became_active":"2024-01-15T17:55:04.753698+0000","last_became_peered":"2024-01-15T17:55:04.753698+0000","last_unstale":"2024-01-15T17:55:06.589901+0000","last_undegraded":"2024-01-15T17:55:06.589901+0000","last_fullsized":"2024-01-15T17:55:06.589901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:59.870088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788324+0000","last_change":"2024-01-15T17:55:04.754498+0000","last_active":"2024-01-15T17:55:06.788324+0000","last_peered":"2024-01-15T17:55:06.788324+0000","last_clean":"2024-01-15T17:55:06.788324+0000","last_became_active":"2024-01-15T17:55:04.754091+0000","last_became_peered":"2024-01-15T17:55:04.754091+0000","last_unstale":"2024-01-15T17:55:06.788324+0000","last_undegraded":"2024-01-15T17:55:06.788324+0000","last_fullsized":"2024-01-15T17:55:06.788324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:31.493057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590727+0000","last_change":"2024-01-15T17:55:04.755593+0000","last_active":"2024-01-15T17:55:06.590727+0000","last_peered":"2024-01-15T17:55:06.590727+0000","last_clean":"2024-01-15T17:55:06.590727+0000","last_became_active":"2024-01-15T17:55:04.755376+0000","last_became_peered":"2024-01-15T17:55:04.755376+0000","last_unstale":"2024-01-15T17:55:06.590727+0000","last_undegraded":"2024-01-15T17:55:06.590727+0000","last_fullsized":"2024-01-15T17:55:06.590727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:10:13.660425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726844+0000","last_change":"2024-01-15T17:55:04.745309+0000","last_active":"2024-01-15T17:55:05.726844+0000","last_peered":"2024-01-15T17:55:05.726844+0000","last_clean":"2024-01-15T17:55:05.726844+0000","last_became_active":"2024-01-15T17:55:04.744930+0000","last_became_peered":"2024-01-15T17:55:04.744930+0000","last_unstale":"2024-01-15T17:55:05.726844+0000","last_undegraded":"2024-01-15T17:55:05.726844+0000","last_fullsized":"2024-01-15T17:55:05.726844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726624+0000","last_change":"2024-01-15T17:55:04.743285+0000","last_active":"2024-01-15T17:55:05.726624+0000","last_peered":"2024-01-15T17:55:05.726624+0000","last_clean":"2024-01-15T17:55:05.726624+0000","last_became_active":"2024-01-15T17:55:04.743118+0000","last_became_peered":"2024-01-15T17:55:04.743118+0000","last_unstale":"2024-01-15T17:55:05.726624+0000","last_undegraded":"2024-01-15T17:55:05.726624+0000","last_fullsized":"2024-01-15T17:55:05.726624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726121+0000","last_change":"2024-01-15T17:55:04.739598+0000","last_active":"2024-01-15T17:55:05.726121+0000","last_peered":"2024-01-15T17:55:05.726121+0000","last_clean":"2024-01-15T17:55:05.726121+0000","last_became_active":"2024-01-15T17:55:04.739319+0000","last_became_peered":"2024-01-15T17:55:04.739319+0000","last_unstale":"2024-01-15T17:55:05.726121+0000","last_undegraded":"2024-01-15T17:55:05.726121+0000","last_fullsized":"2024-01-15T17:55:05.726121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:21.113714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788978+0000","last_change":"2024-01-15T17:55:04.751187+0000","last_active":"2024-01-15T17:55:06.788978+0000","last_peered":"2024-01-15T17:55:06.788978+0000","last_clean":"2024-01-15T17:55:06.788978+0000","last_became_active":"2024-01-15T17:55:04.750057+0000","last_became_peered":"2024-01-15T17:55:04.750057+0000","last_unstale":"2024-01-15T17:55:06.788978+0000","last_undegraded":"2024-01-15T17:55:06.788978+0000","last_fullsized":"2024-01-15T17:55:06.788978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:22:07.251406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787842+0000","last_change":"2024-01-15T17:55:04.759443+0000","last_active":"2024-01-15T17:55:06.787842+0000","last_peered":"2024-01-15T17:55:06.787842+0000","last_clean":"2024-01-15T17:55:06.787842+0000","last_became_active":"2024-01-15T17:55:04.759086+0000","last_became_peered":"2024-01-15T17:55:04.759086+0000","last_unstale":"2024-01-15T17:55:06.787842+0000","last_undegraded":"2024-01-15T17:55:06.787842+0000","last_fullsized":"2024-01-15T17:55:06.787842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:27.225767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721835+0000","last_change":"2024-01-15T17:55:04.741854+0000","last_active":"2024-01-15T17:55:05.721835+0000","last_peered":"2024-01-15T17:55:05.721835+0000","last_clean":"2024-01-15T17:55:05.721835+0000","last_became_active":"2024-01-15T17:55:04.741472+0000","last_became_peered":"2024-01-15T17:55:04.741472+0000","last_unstale":"2024-01-15T17:55:05.721835+0000","last_undegraded":"2024-01-15T17:55:05.721835+0000","last_fullsized":"2024-01-15T17:55:05.721835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789012+0000","last_change":"2024-01-15T17:55:04.757919+0000","last_active":"2024-01-15T17:55:06.789012+0000","last_peered":"2024-01-15T17:55:06.789012+0000","last_clean":"2024-01-15T17:55:06.789012+0000","last_became_active":"2024-01-15T17:55:04.757595+0000","last_became_peered":"2024-01-15T17:55:04.757595+0000","last_unstale":"2024-01-15T17:55:06.789012+0000","last_undegraded":"2024-01-15T17:55:06.789012+0000","last_fullsized":"2024-01-15T17:55:06.789012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790339+0000","last_change":"2024-01-15T17:55:04.745091+0000","last_active":"2024-01-15T17:55:06.790339+0000","last_peered":"2024-01-15T17:55:06.790339+0000","last_clean":"2024-01-15T17:55:06.790339+0000","last_became_active":"2024-01-15T17:55:04.744779+0000","last_became_peered":"2024-01-15T17:55:04.744779+0000","last_unstale":"2024-01-15T17:55:06.790339+0000","last_undegraded":"2024-01-15T17:55:06.790339+0000","last_fullsized":"2024-01-15T17:55:06.790339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:10:13.660425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787739+0000","last_change":"2024-01-15T17:55:04.754772+0000","last_active":"2024-01-15T17:55:06.787739+0000","last_peered":"2024-01-15T17:55:06.787739+0000","last_clean":"2024-01-15T17:55:06.787739+0000","last_became_active":"2024-01-15T17:55:04.743420+0000","last_became_peered":"2024-01-15T17:55:04.743420+0000","last_unstale":"2024-01-15T17:55:06.787739+0000","last_undegraded":"2024-01-15T17:55:06.787739+0000","last_fullsized":"2024-01-15T17:55:06.787739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:24:25.848060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591083+0000","last_change":"2024-01-15T17:55:04.741262+0000","last_active":"2024-01-15T17:55:06.591083+0000","last_peered":"2024-01-15T17:55:06.591083+0000","last_clean":"2024-01-15T17:55:06.591083+0000","last_became_active":"2024-01-15T17:55:04.740978+0000","last_became_peered":"2024-01-15T17:55:04.740978+0000","last_unstale":"2024-01-15T17:55:06.591083+0000","last_undegraded":"2024-01-15T17:55:06.591083+0000","last_fullsized":"2024-01-15T17:55:06.591083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:00.893650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726416+0000","last_change":"2024-01-15T17:55:04.758887+0000","last_active":"2024-01-15T17:55:05.726416+0000","last_peered":"2024-01-15T17:55:05.726416+0000","last_clean":"2024-01-15T17:55:05.726416+0000","last_became_active":"2024-01-15T17:55:04.758530+0000","last_became_peered":"2024-01-15T17:55:04.758530+0000","last_unstale":"2024-01-15T17:55:05.726416+0000","last_undegraded":"2024-01-15T17:55:05.726416+0000","last_fullsized":"2024-01-15T17:55:05.726416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788603+0000","last_change":"2024-01-15T17:55:04.756875+0000","last_active":"2024-01-15T17:55:06.788603+0000","last_peered":"2024-01-15T17:55:06.788603+0000","last_clean":"2024-01-15T17:55:06.788603+0000","last_became_active":"2024-01-15T17:55:04.756596+0000","last_became_peered":"2024-01-15T17:55:04.756596+0000","last_unstale":"2024-01-15T17:55:06.788603+0000","last_undegraded":"2024-01-15T17:55:06.788603+0000","last_fullsized":"2024-01-15T17:55:06.788603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789372+0000","last_change":"2024-01-15T17:55:04.746119+0000","last_active":"2024-01-15T17:55:06.789372+0000","last_peered":"2024-01-15T17:55:06.789372+0000","last_clean":"2024-01-15T17:55:06.789372+0000","last_became_active":"2024-01-15T17:55:04.745830+0000","last_became_peered":"2024-01-15T17:55:04.745830+0000","last_unstale":"2024-01-15T17:55:06.789372+0000","last_undegraded":"2024-01-15T17:55:06.789372+0000","last_fullsized":"2024-01-15T17:55:06.789372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:47:39.573232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789300+0000","last_change":"2024-01-15T17:55:04.764403+0000","last_active":"2024-01-15T17:55:06.789300+0000","last_peered":"2024-01-15T17:55:06.789300+0000","last_clean":"2024-01-15T17:55:06.789300+0000","last_became_active":"2024-01-15T17:55:04.763976+0000","last_became_peered":"2024-01-15T17:55:04.763976+0000","last_unstale":"2024-01-15T17:55:06.789300+0000","last_undegraded":"2024-01-15T17:55:06.789300+0000","last_fullsized":"2024-01-15T17:55:06.789300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:08:09.187744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789136+0000","last_change":"2024-01-15T17:55:04.750998+0000","last_active":"2024-01-15T17:55:06.789136+0000","last_peered":"2024-01-15T17:55:06.789136+0000","last_clean":"2024-01-15T17:55:06.789136+0000","last_became_active":"2024-01-15T17:55:04.750216+0000","last_became_peered":"2024-01-15T17:55:04.750216+0000","last_unstale":"2024-01-15T17:55:06.789136+0000","last_undegraded":"2024-01-15T17:55:06.789136+0000","last_fullsized":"2024-01-15T17:55:06.789136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790494+0000","last_change":"2024-01-15T17:55:04.758699+0000","last_active":"2024-01-15T17:55:06.790494+0000","last_peered":"2024-01-15T17:55:06.790494+0000","last_clean":"2024-01-15T17:55:06.790494+0000","last_became_active":"2024-01-15T17:55:04.758066+0000","last_became_peered":"2024-01-15T17:55:04.758066+0000","last_unstale":"2024-01-15T17:55:06.790494+0000","last_undegraded":"2024-01-15T17:55:06.790494+0000","last_fullsized":"2024-01-15T17:55:06.790494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:31:30.922640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727023+0000","last_change":"2024-01-15T17:55:04.749062+0000","last_active":"2024-01-15T17:55:05.727023+0000","last_peered":"2024-01-15T17:55:05.727023+0000","last_clean":"2024-01-15T17:55:05.727023+0000","last_became_active":"2024-01-15T17:55:04.746787+0000","last_became_peered":"2024-01-15T17:55:04.746787+0000","last_unstale":"2024-01-15T17:55:05.727023+0000","last_undegraded":"2024-01-15T17:55:05.727023+0000","last_fullsized":"2024-01-15T17:55:05.727023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721723+0000","last_change":"2024-01-15T17:55:04.753444+0000","last_active":"2024-01-15T17:55:05.721723+0000","last_peered":"2024-01-15T17:55:05.721723+0000","last_clean":"2024-01-15T17:55:05.721723+0000","last_became_active":"2024-01-15T17:55:04.753162+0000","last_became_peered":"2024-01-15T17:55:04.753162+0000","last_unstale":"2024-01-15T17:55:05.721723+0000","last_undegraded":"2024-01-15T17:55:05.721723+0000","last_fullsized":"2024-01-15T17:55:05.721723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790061+0000","last_change":"2024-01-15T17:55:04.764187+0000","last_active":"2024-01-15T17:55:06.790061+0000","last_peered":"2024-01-15T17:55:06.790061+0000","last_clean":"2024-01-15T17:55:06.790061+0000","last_became_active":"2024-01-15T17:55:04.763904+0000","last_became_peered":"2024-01-15T17:55:04.763904+0000","last_unstale":"2024-01-15T17:55:06.790061+0000","last_undegraded":"2024-01-15T17:55:06.790061+0000","last_fullsized":"2024-01-15T17:55:06.790061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:21:21.061668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788913+0000","last_change":"2024-01-15T17:55:04.769007+0000","last_active":"2024-01-15T17:55:06.788913+0000","last_peered":"2024-01-15T17:55:06.788913+0000","last_clean":"2024-01-15T17:55:06.788913+0000","last_became_active":"2024-01-15T17:55:04.767135+0000","last_became_peered":"2024-01-15T17:55:04.767135+0000","last_unstale":"2024-01-15T17:55:06.788913+0000","last_undegraded":"2024-01-15T17:55:06.788913+0000","last_fullsized":"2024-01-15T17:55:06.788913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789565+0000","last_change":"2024-01-15T17:55:04.751046+0000","last_active":"2024-01-15T17:55:06.789565+0000","last_peered":"2024-01-15T17:55:06.789565+0000","last_clean":"2024-01-15T17:55:06.789565+0000","last_became_active":"2024-01-15T17:55:04.750724+0000","last_became_peered":"2024-01-15T17:55:04.750724+0000","last_unstale":"2024-01-15T17:55:06.789565+0000","last_undegraded":"2024-01-15T17:55:06.789565+0000","last_fullsized":"2024-01-15T17:55:06.789565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:59.870088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787248+0000","last_change":"2024-01-15T17:55:04.732768+0000","last_active":"2024-01-15T17:55:06.787248+0000","last_peered":"2024-01-15T17:55:06.787248+0000","last_clean":"2024-01-15T17:55:06.787248+0000","last_became_active":"2024-01-15T17:55:04.732209+0000","last_became_peered":"2024-01-15T17:55:04.732209+0000","last_unstale":"2024-01-15T17:55:06.787248+0000","last_undegraded":"2024-01-15T17:55:06.787248+0000","last_fullsized":"2024-01-15T17:55:06.787248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:21.113714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.724905+0000","last_change":"2024-01-15T17:55:04.762744+0000","last_active":"2024-01-15T17:55:05.724905+0000","last_peered":"2024-01-15T17:55:05.724905+0000","last_clean":"2024-01-15T17:55:05.724905+0000","last_became_active":"2024-01-15T17:55:04.762234+0000","last_became_peered":"2024-01-15T17:55:04.762234+0000","last_unstale":"2024-01-15T17:55:05.724905+0000","last_undegraded":"2024-01-15T17:55:05.724905+0000","last_fullsized":"2024-01-15T17:55:05.724905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:10.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725553+0000","last_change":"2024-01-15T17:55:04.741937+0000","last_active":"2024-01-15T17:55:05.725553+0000","last_peered":"2024-01-15T17:55:05.725553+0000","last_clean":"2024-01-15T17:55:05.725553+0000","last_became_active":"2024-01-15T17:55:04.741629+0000","last_became_peered":"2024-01-15T17:55:04.741629+0000","last_unstale":"2024-01-15T17:55:05.725553+0000","last_undegraded":"2024-01-15T17:55:05.725553+0000","last_fullsized":"2024-01-15T17:55:05.725553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:30.356212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789134+0000","last_change":"2024-01-15T17:55:03.753426+0000","last_active":"2024-01-15T17:55:06.789134+0000","last_peered":"2024-01-15T17:55:06.789134+0000","last_clean":"2024-01-15T17:55:06.789134+0000","last_became_active":"2024-01-15T17:55:03.753141+0000","last_became_peered":"2024-01-15T17:55:03.753141+0000","last_unstale":"2024-01-15T17:55:06.789134+0000","last_undegraded":"2024-01-15T17:55:06.789134+0000","last_fullsized":"2024-01-15T17:55:06.789134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:38.142877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789190+0000","last_change":"2024-01-15T17:55:04.760185+0000","last_active":"2024-01-15T17:55:06.789190+0000","last_peered":"2024-01-15T17:55:06.789190+0000","last_clean":"2024-01-15T17:55:06.789190+0000","last_became_active":"2024-01-15T17:55:04.759883+0000","last_became_peered":"2024-01-15T17:55:04.759883+0000","last_unstale":"2024-01-15T17:55:06.789190+0000","last_undegraded":"2024-01-15T17:55:06.789190+0000","last_fullsized":"2024-01-15T17:55:06.789190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725694+0000","last_change":"2024-01-15T17:55:03.747953+0000","last_active":"2024-01-15T17:55:05.725694+0000","last_peered":"2024-01-15T17:55:05.725694+0000","last_clean":"2024-01-15T17:55:05.725694+0000","last_became_active":"2024-01-15T17:55:03.747648+0000","last_became_peered":"2024-01-15T17:55:03.747648+0000","last_unstale":"2024-01-15T17:55:05.725694+0000","last_undegraded":"2024-01-15T17:55:05.725694+0000","last_fullsized":"2024-01-15T17:55:05.725694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:54.384035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789495+0000","last_change":"2024-01-15T17:55:03.743175+0000","last_active":"2024-01-15T17:55:06.789495+0000","last_peered":"2024-01-15T17:55:06.789495+0000","last_clean":"2024-01-15T17:55:06.789495+0000","last_became_active":"2024-01-15T17:55:03.742887+0000","last_became_peered":"2024-01-15T17:55:03.742887+0000","last_unstale":"2024-01-15T17:55:06.789495+0000","last_undegraded":"2024-01-15T17:55:06.789495+0000","last_fullsized":"2024-01-15T17:55:06.789495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:03.274467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788556+0000","last_change":"2024-01-15T17:55:04.767350+0000","last_active":"2024-01-15T17:55:06.788556+0000","last_peered":"2024-01-15T17:55:06.788556+0000","last_clean":"2024-01-15T17:55:06.788556+0000","last_became_active":"2024-01-15T17:55:04.767083+0000","last_became_peered":"2024-01-15T17:55:04.767083+0000","last_unstale":"2024-01-15T17:55:06.788556+0000","last_undegraded":"2024-01-15T17:55:06.788556+0000","last_fullsized":"2024-01-15T17:55:06.788556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:01.549303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789294+0000","last_change":"2024-01-15T17:55:03.744198+0000","last_active":"2024-01-15T17:55:06.789294+0000","last_peered":"2024-01-15T17:55:06.789294+0000","last_clean":"2024-01-15T17:55:06.789294+0000","last_became_active":"2024-01-15T17:55:03.743838+0000","last_became_peered":"2024-01-15T17:55:03.743838+0000","last_unstale":"2024-01-15T17:55:06.789294+0000","last_undegraded":"2024-01-15T17:55:06.789294+0000","last_fullsized":"2024-01-15T17:55:06.789294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:19.912976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721537+0000","last_change":"2024-01-15T17:55:04.751636+0000","last_active":"2024-01-15T17:55:05.721537+0000","last_peered":"2024-01-15T17:55:05.721537+0000","last_clean":"2024-01-15T17:55:05.721537+0000","last_became_active":"2024-01-15T17:55:04.751353+0000","last_became_peered":"2024-01-15T17:55:04.751353+0000","last_unstale":"2024-01-15T17:55:05.721537+0000","last_undegraded":"2024-01-15T17:55:05.721537+0000","last_fullsized":"2024-01-15T17:55:05.721537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727326+0000","last_change":"2024-01-15T17:55:03.749204+0000","last_active":"2024-01-15T17:55:05.727326+0000","last_peered":"2024-01-15T17:55:05.727326+0000","last_clean":"2024-01-15T17:55:05.727326+0000","last_became_active":"2024-01-15T17:55:03.748941+0000","last_became_peered":"2024-01-15T17:55:03.748941+0000","last_unstale":"2024-01-15T17:55:05.727326+0000","last_undegraded":"2024-01-15T17:55:05.727326+0000","last_fullsized":"2024-01-15T17:55:05.727326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790411+0000","last_change":"2024-01-15T17:55:04.743467+0000","last_active":"2024-01-15T17:55:06.790411+0000","last_peered":"2024-01-15T17:55:06.790411+0000","last_clean":"2024-01-15T17:55:06.790411+0000","last_became_active":"2024-01-15T17:55:04.743161+0000","last_became_peered":"2024-01-15T17:55:04.743161+0000","last_unstale":"2024-01-15T17:55:06.790411+0000","last_undegraded":"2024-01-15T17:55:06.790411+0000","last_fullsized":"2024-01-15T17:55:06.790411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:00.908270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790246+0000","last_change":"2024-01-15T17:55:03.757214+0000","last_active":"2024-01-15T17:55:06.790246+0000","last_peered":"2024-01-15T17:55:06.790246+0000","last_clean":"2024-01-15T17:55:06.790246+0000","last_became_active":"2024-01-15T17:55:03.756900+0000","last_became_peered":"2024-01-15T17:55:03.756900+0000","last_unstale":"2024-01-15T17:55:06.790246+0000","last_undegraded":"2024-01-15T17:55:06.790246+0000","last_fullsized":"2024-01-15T17:55:06.790246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:19.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788839+0000","last_change":"2024-01-15T17:55:04.758670+0000","last_active":"2024-01-15T17:55:06.788839+0000","last_peered":"2024-01-15T17:55:06.788839+0000","last_clean":"2024-01-15T17:55:06.788839+0000","last_became_active":"2024-01-15T17:55:04.757659+0000","last_became_peered":"2024-01-15T17:55:04.757659+0000","last_unstale":"2024-01-15T17:55:06.788839+0000","last_undegraded":"2024-01-15T17:55:06.788839+0000","last_fullsized":"2024-01-15T17:55:06.788839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721816+0000","last_change":"2024-01-15T17:55:03.746513+0000","last_active":"2024-01-15T17:55:05.721816+0000","last_peered":"2024-01-15T17:55:05.721816+0000","last_clean":"2024-01-15T17:55:05.721816+0000","last_became_active":"2024-01-15T17:55:03.746147+0000","last_became_peered":"2024-01-15T17:55:03.746147+0000","last_unstale":"2024-01-15T17:55:05.721816+0000","last_undegraded":"2024-01-15T17:55:05.721816+0000","last_fullsized":"2024-01-15T17:55:05.721816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726333+0000","last_change":"2024-01-15T17:55:04.752615+0000","last_active":"2024-01-15T17:55:05.726333+0000","last_peered":"2024-01-15T17:55:05.726333+0000","last_clean":"2024-01-15T17:55:05.726333+0000","last_became_active":"2024-01-15T17:55:04.751582+0000","last_became_peered":"2024-01-15T17:55:04.751582+0000","last_unstale":"2024-01-15T17:55:05.726333+0000","last_undegraded":"2024-01-15T17:55:05.726333+0000","last_fullsized":"2024-01-15T17:55:05.726333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:24:25.848060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726379+0000","last_change":"2024-01-15T17:55:03.741760+0000","last_active":"2024-01-15T17:55:05.726379+0000","last_peered":"2024-01-15T17:55:05.726379+0000","last_clean":"2024-01-15T17:55:05.726379+0000","last_became_active":"2024-01-15T17:55:03.741327+0000","last_became_peered":"2024-01-15T17:55:03.741327+0000","last_unstale":"2024-01-15T17:55:05.726379+0000","last_undegraded":"2024-01-15T17:55:05.726379+0000","last_fullsized":"2024-01-15T17:55:05.726379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590242+0000","last_change":"2024-01-15T17:55:04.761237+0000","last_active":"2024-01-15T17:55:06.590242+0000","last_peered":"2024-01-15T17:55:06.590242+0000","last_clean":"2024-01-15T17:55:06.590242+0000","last_became_active":"2024-01-15T17:55:04.759551+0000","last_became_peered":"2024-01-15T17:55:04.759551+0000","last_unstale":"2024-01-15T17:55:06.590242+0000","last_undegraded":"2024-01-15T17:55:06.590242+0000","last_fullsized":"2024-01-15T17:55:06.590242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:17.567853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789391+0000","last_change":"2024-01-15T17:55:03.733161+0000","last_active":"2024-01-15T17:55:06.789391+0000","last_peered":"2024-01-15T17:55:06.789391+0000","last_clean":"2024-01-15T17:55:06.789391+0000","last_became_active":"2024-01-15T17:55:03.732839+0000","last_became_peered":"2024-01-15T17:55:03.732839+0000","last_unstale":"2024-01-15T17:55:06.789391+0000","last_undegraded":"2024-01-15T17:55:06.789391+0000","last_fullsized":"2024-01-15T17:55:06.789391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:17.540533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788038+0000","last_change":"2024-01-15T17:55:04.759368+0000","last_active":"2024-01-15T17:55:06.788038+0000","last_peered":"2024-01-15T17:55:06.788038+0000","last_clean":"2024-01-15T17:55:06.788038+0000","last_became_active":"2024-01-15T17:55:04.759069+0000","last_became_peered":"2024-01-15T17:55:04.759069+0000","last_unstale":"2024-01-15T17:55:06.788038+0000","last_undegraded":"2024-01-15T17:55:06.788038+0000","last_fullsized":"2024-01-15T17:55:06.788038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:10.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789653+0000","last_change":"2024-01-15T17:55:03.746380+0000","last_active":"2024-01-15T17:55:06.789653+0000","last_peered":"2024-01-15T17:55:06.789653+0000","last_clean":"2024-01-15T17:55:06.789653+0000","last_became_active":"2024-01-15T17:55:03.745986+0000","last_became_peered":"2024-01-15T17:55:03.745986+0000","last_unstale":"2024-01-15T17:55:06.789653+0000","last_undegraded":"2024-01-15T17:55:06.789653+0000","last_fullsized":"2024-01-15T17:55:06.789653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590808+0000","last_change":"2024-01-15T17:55:04.751522+0000","last_active":"2024-01-15T17:55:06.590808+0000","last_peered":"2024-01-15T17:55:06.590808+0000","last_clean":"2024-01-15T17:55:06.590808+0000","last_became_active":"2024-01-15T17:55:04.751235+0000","last_became_peered":"2024-01-15T17:55:04.751235+0000","last_unstale":"2024-01-15T17:55:06.590808+0000","last_undegraded":"2024-01-15T17:55:06.590808+0000","last_fullsized":"2024-01-15T17:55:06.590808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:00.908270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725362+0000","last_change":"2024-01-15T17:55:03.751331+0000","last_active":"2024-01-15T17:55:05.725362+0000","last_peered":"2024-01-15T17:55:05.725362+0000","last_clean":"2024-01-15T17:55:05.725362+0000","last_became_active":"2024-01-15T17:55:03.749400+0000","last_became_peered":"2024-01-15T17:55:03.749400+0000","last_unstale":"2024-01-15T17:55:05.725362+0000","last_undegraded":"2024-01-15T17:55:05.725362+0000","last_fullsized":"2024-01-15T17:55:05.725362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.724907+0000","last_change":"2024-01-15T17:55:04.761956+0000","last_active":"2024-01-15T17:55:05.724907+0000","last_peered":"2024-01-15T17:55:05.724907+0000","last_clean":"2024-01-15T17:55:05.724907+0000","last_became_active":"2024-01-15T17:55:04.761741+0000","last_became_peered":"2024-01-15T17:55:04.761741+0000","last_unstale":"2024-01-15T17:55:05.724907+0000","last_undegraded":"2024-01-15T17:55:05.724907+0000","last_fullsized":"2024-01-15T17:55:05.724907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787164+0000","last_change":"2024-01-15T17:55:03.735909+0000","last_active":"2024-01-15T17:55:06.787164+0000","last_peered":"2024-01-15T17:55:06.787164+0000","last_clean":"2024-01-15T17:55:06.787164+0000","last_became_active":"2024-01-15T17:55:03.734252+0000","last_became_peered":"2024-01-15T17:55:03.734252+0000","last_unstale":"2024-01-15T17:55:06.787164+0000","last_undegraded":"2024-01-15T17:55:06.787164+0000","last_fullsized":"2024-01-15T17:55:06.787164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590233+0000","last_change":"2024-01-15T17:55:04.753688+0000","last_active":"2024-01-15T17:55:06.590233+0000","last_peered":"2024-01-15T17:55:06.590233+0000","last_clean":"2024-01-15T17:55:06.590233+0000","last_became_active":"2024-01-15T17:55:04.753479+0000","last_became_peered":"2024-01-15T17:55:04.753479+0000","last_unstale":"2024-01-15T17:55:06.590233+0000","last_undegraded":"2024-01-15T17:55:06.590233+0000","last_fullsized":"2024-01-15T17:55:06.590233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:52.765350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726589+0000","last_change":"2024-01-15T17:55:03.740438+0000","last_active":"2024-01-15T17:55:05.726589+0000","last_peered":"2024-01-15T17:55:05.726589+0000","last_clean":"2024-01-15T17:55:05.726589+0000","last_became_active":"2024-01-15T17:55:03.737923+0000","last_became_peered":"2024-01-15T17:55:03.737923+0000","last_unstale":"2024-01-15T17:55:05.726589+0000","last_undegraded":"2024-01-15T17:55:05.726589+0000","last_fullsized":"2024-01-15T17:55:05.726589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591190+0000","last_change":"2024-01-15T17:55:04.756054+0000","last_active":"2024-01-15T17:55:06.591190+0000","last_peered":"2024-01-15T17:55:06.591190+0000","last_clean":"2024-01-15T17:55:06.591190+0000","last_became_active":"2024-01-15T17:55:04.754276+0000","last_became_peered":"2024-01-15T17:55:04.754276+0000","last_unstale":"2024-01-15T17:55:06.591190+0000","last_undegraded":"2024-01-15T17:55:06.591190+0000","last_fullsized":"2024-01-15T17:55:06.591190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:38.181261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787840+0000","last_change":"2024-01-15T17:55:03.744207+0000","last_active":"2024-01-15T17:55:06.787840+0000","last_peered":"2024-01-15T17:55:06.787840+0000","last_clean":"2024-01-15T17:55:06.787840+0000","last_became_active":"2024-01-15T17:55:03.743894+0000","last_became_peered":"2024-01-15T17:55:03.743894+0000","last_unstale":"2024-01-15T17:55:06.787840+0000","last_undegraded":"2024-01-15T17:55:06.787840+0000","last_fullsized":"2024-01-15T17:55:06.787840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591002+0000","last_change":"2024-01-15T17:55:04.758291+0000","last_active":"2024-01-15T17:55:06.591002+0000","last_peered":"2024-01-15T17:55:06.591002+0000","last_clean":"2024-01-15T17:55:06.591002+0000","last_became_active":"2024-01-15T17:55:04.757888+0000","last_became_peered":"2024-01-15T17:55:04.757888+0000","last_unstale":"2024-01-15T17:55:06.591002+0000","last_undegraded":"2024-01-15T17:55:06.591002+0000","last_fullsized":"2024-01-15T17:55:06.591002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:34:45.528954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721818+0000","last_change":"2024-01-15T17:55:03.744276+0000","last_active":"2024-01-15T17:55:05.721818+0000","last_peered":"2024-01-15T17:55:05.721818+0000","last_clean":"2024-01-15T17:55:05.721818+0000","last_became_active":"2024-01-15T17:55:03.744033+0000","last_became_peered":"2024-01-15T17:55:03.744033+0000","last_unstale":"2024-01-15T17:55:05.721818+0000","last_undegraded":"2024-01-15T17:55:05.721818+0000","last_fullsized":"2024-01-15T17:55:05.721818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:17.540533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790107+0000","last_change":"2024-01-15T17:55:04.751400+0000","last_active":"2024-01-15T17:55:06.790107+0000","last_peered":"2024-01-15T17:55:06.790107+0000","last_clean":"2024-01-15T17:55:06.790107+0000","last_became_active":"2024-01-15T17:55:04.751097+0000","last_became_peered":"2024-01-15T17:55:04.751097+0000","last_unstale":"2024-01-15T17:55:06.790107+0000","last_undegraded":"2024-01-15T17:55:06.790107+0000","last_fullsized":"2024-01-15T17:55:06.790107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:56:52.262406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787591+0000","last_change":"2024-01-15T17:55:03.751285+0000","last_active":"2024-01-15T17:55:06.787591+0000","last_peered":"2024-01-15T17:55:06.787591+0000","last_clean":"2024-01-15T17:55:06.787591+0000","last_became_active":"2024-01-15T17:55:03.750926+0000","last_became_peered":"2024-01-15T17:55:03.750926+0000","last_unstale":"2024-01-15T17:55:06.787591+0000","last_undegraded":"2024-01-15T17:55:06.787591+0000","last_fullsized":"2024-01-15T17:55:06.787591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590723+0000","last_change":"2024-01-15T17:55:04.741297+0000","last_active":"2024-01-15T17:55:06.590723+0000","last_peered":"2024-01-15T17:55:06.590723+0000","last_clean":"2024-01-15T17:55:06.590723+0000","last_became_active":"2024-01-15T17:55:04.741044+0000","last_became_peered":"2024-01-15T17:55:04.741044+0000","last_unstale":"2024-01-15T17:55:06.590723+0000","last_undegraded":"2024-01-15T17:55:06.590723+0000","last_fullsized":"2024-01-15T17:55:06.590723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:56:52.262406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-15T17:55:04.731969+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:55:04.731969+0000","last_peered":"2024-01-15T17:55:04.731969+0000","last_clean":"2024-01-15T17:55:04.731969+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:55:04.731969+0000","last_undegraded":"2024-01-15T17:55:04.731969+0000","last_fullsized":"2024-01-15T17:55:04.731969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789182+0000","last_change":"2024-01-15T17:55:03.744683+0000","last_active":"2024-01-15T17:55:06.789182+0000","last_peered":"2024-01-15T17:55:06.789182+0000","last_clean":"2024-01-15T17:55:06.789182+0000","last_became_active":"2024-01-15T17:55:03.743028+0000","last_became_peered":"2024-01-15T17:55:03.743028+0000","last_unstale":"2024-01-15T17:55:06.789182+0000","last_undegraded":"2024-01-15T17:55:06.789182+0000","last_fullsized":"2024-01-15T17:55:06.789182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790120+0000","last_change":"2024-01-15T17:55:04.755598+0000","last_active":"2024-01-15T17:55:06.790120+0000","last_peered":"2024-01-15T17:55:06.790120+0000","last_clean":"2024-01-15T17:55:06.790120+0000","last_became_active":"2024-01-15T17:55:04.755327+0000","last_became_peered":"2024-01-15T17:55:04.755327+0000","last_unstale":"2024-01-15T17:55:06.790120+0000","last_undegraded":"2024-01-15T17:55:06.790120+0000","last_fullsized":"2024-01-15T17:55:06.790120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:47.198851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589792+0000","last_change":"2024-01-15T17:55:03.755061+0000","last_active":"2024-01-15T17:55:06.589792+0000","last_peered":"2024-01-15T17:55:06.589792+0000","last_clean":"2024-01-15T17:55:06.589792+0000","last_became_active":"2024-01-15T17:55:03.754761+0000","last_became_peered":"2024-01-15T17:55:03.754761+0000","last_unstale":"2024-01-15T17:55:06.589792+0000","last_undegraded":"2024-01-15T17:55:06.589792+0000","last_fullsized":"2024-01-15T17:55:06.589792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787429+0000","last_change":"2024-01-15T17:55:04.760196+0000","last_active":"2024-01-15T17:55:06.787429+0000","last_peered":"2024-01-15T17:55:06.787429+0000","last_clean":"2024-01-15T17:55:06.787429+0000","last_became_active":"2024-01-15T17:55:04.759902+0000","last_became_peered":"2024-01-15T17:55:04.759902+0000","last_unstale":"2024-01-15T17:55:06.787429+0000","last_undegraded":"2024-01-15T17:55:06.787429+0000","last_fullsized":"2024-01-15T17:55:06.787429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:57.337185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787323+0000","last_change":"2024-01-15T17:55:03.741130+0000","last_active":"2024-01-15T17:55:06.787323+0000","last_peered":"2024-01-15T17:55:06.787323+0000","last_clean":"2024-01-15T17:55:06.787323+0000","last_became_active":"2024-01-15T17:55:03.740791+0000","last_became_peered":"2024-01-15T17:55:03.740791+0000","last_unstale":"2024-01-15T17:55:06.787323+0000","last_undegraded":"2024-01-15T17:55:06.787323+0000","last_fullsized":"2024-01-15T17:55:06.787323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:57.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721182+0000","last_change":"2024-01-15T17:55:04.739187+0000","last_active":"2024-01-15T17:55:05.721182+0000","last_peered":"2024-01-15T17:55:05.721182+0000","last_clean":"2024-01-15T17:55:05.721182+0000","last_became_active":"2024-01-15T17:55:04.738700+0000","last_became_peered":"2024-01-15T17:55:04.738700+0000","last_unstale":"2024-01-15T17:55:05.721182+0000","last_undegraded":"2024-01-15T17:55:05.721182+0000","last_fullsized":"2024-01-15T17:55:05.721182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790088+0000","last_change":"2024-01-15T17:55:03.741719+0000","last_active":"2024-01-15T17:55:06.790088+0000","last_peered":"2024-01-15T17:55:06.790088+0000","last_clean":"2024-01-15T17:55:06.790088+0000","last_became_active":"2024-01-15T17:55:03.739882+0000","last_became_peered":"2024-01-15T17:55:03.739882+0000","last_unstale":"2024-01-15T17:55:06.790088+0000","last_undegraded":"2024-01-15T17:55:06.790088+0000","last_fullsized":"2024-01-15T17:55:06.790088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725696+0000","last_change":"2024-01-15T17:55:04.766676+0000","last_active":"2024-01-15T17:55:05.725696+0000","last_peered":"2024-01-15T17:55:05.725696+0000","last_clean":"2024-01-15T17:55:05.725696+0000","last_became_active":"2024-01-15T17:55:04.766394+0000","last_became_peered":"2024-01-15T17:55:04.766394+0000","last_unstale":"2024-01-15T17:55:05.725696+0000","last_undegraded":"2024-01-15T17:55:05.725696+0000","last_fullsized":"2024-01-15T17:55:05.725696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:52.036315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721394+0000","last_change":"2024-01-15T17:55:03.735528+0000","last_active":"2024-01-15T17:55:05.721394+0000","last_peered":"2024-01-15T17:55:05.721394+0000","last_clean":"2024-01-15T17:55:05.721394+0000","last_became_active":"2024-01-15T17:55:03.733799+0000","last_became_peered":"2024-01-15T17:55:03.733799+0000","last_unstale":"2024-01-15T17:55:05.721394+0000","last_undegraded":"2024-01-15T17:55:05.721394+0000","last_fullsized":"2024-01-15T17:55:05.721394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789930+0000","last_change":"2024-01-15T17:55:04.750889+0000","last_active":"2024-01-15T17:55:06.789930+0000","last_peered":"2024-01-15T17:55:06.789930+0000","last_clean":"2024-01-15T17:55:06.789930+0000","last_became_active":"2024-01-15T17:55:04.750480+0000","last_became_peered":"2024-01-15T17:55:04.750480+0000","last_unstale":"2024-01-15T17:55:06.789930+0000","last_undegraded":"2024-01-15T17:55:06.789930+0000","last_fullsized":"2024-01-15T17:55:06.789930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:13.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590950+0000","last_change":"2024-01-15T17:55:03.758172+0000","last_active":"2024-01-15T17:55:06.590950+0000","last_peered":"2024-01-15T17:55:06.590950+0000","last_clean":"2024-01-15T17:55:06.590950+0000","last_became_active":"2024-01-15T17:55:03.757742+0000","last_became_peered":"2024-01-15T17:55:03.757742+0000","last_unstale":"2024-01-15T17:55:06.590950+0000","last_undegraded":"2024-01-15T17:55:06.590950+0000","last_fullsized":"2024-01-15T17:55:06.590950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:17.081091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721370+0000","last_change":"2024-01-15T17:55:04.729485+0000","last_active":"2024-01-15T17:55:05.721370+0000","last_peered":"2024-01-15T17:55:05.721370+0000","last_clean":"2024-01-15T17:55:05.721370+0000","last_became_active":"2024-01-15T17:55:04.729029+0000","last_became_peered":"2024-01-15T17:55:04.729029+0000","last_unstale":"2024-01-15T17:55:05.721370+0000","last_undegraded":"2024-01-15T17:55:05.721370+0000","last_fullsized":"2024-01-15T17:55:05.721370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721495+0000","last_change":"2024-01-15T17:55:03.752377+0000","last_active":"2024-01-15T17:55:05.721495+0000","last_peered":"2024-01-15T17:55:05.721495+0000","last_clean":"2024-01-15T17:55:05.721495+0000","last_became_active":"2024-01-15T17:55:03.752164+0000","last_became_peered":"2024-01-15T17:55:03.752164+0000","last_unstale":"2024-01-15T17:55:05.721495+0000","last_undegraded":"2024-01-15T17:55:05.721495+0000","last_fullsized":"2024-01-15T17:55:05.721495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790469+0000","last_change":"2024-01-15T17:55:04.758991+0000","last_active":"2024-01-15T17:55:06.790469+0000","last_peered":"2024-01-15T17:55:06.790469+0000","last_clean":"2024-01-15T17:55:06.790469+0000","last_became_active":"2024-01-15T17:55:04.758422+0000","last_became_peered":"2024-01-15T17:55:04.758422+0000","last_unstale":"2024-01-15T17:55:06.790469+0000","last_undegraded":"2024-01-15T17:55:06.790469+0000","last_fullsized":"2024-01-15T17:55:06.790469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:01:41.493666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727355+0000","last_change":"2024-01-15T17:55:03.754630+0000","last_active":"2024-01-15T17:55:05.727355+0000","last_peered":"2024-01-15T17:55:05.727355+0000","last_clean":"2024-01-15T17:55:05.727355+0000","last_became_active":"2024-01-15T17:55:03.754117+0000","last_became_peered":"2024-01-15T17:55:03.754117+0000","last_unstale":"2024-01-15T17:55:05.727355+0000","last_undegraded":"2024-01-15T17:55:05.727355+0000","last_fullsized":"2024-01-15T17:55:05.727355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:26.025029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788431+0000","last_change":"2024-01-15T17:55:04.754711+0000","last_active":"2024-01-15T17:55:06.788431+0000","last_peered":"2024-01-15T17:55:06.788431+0000","last_clean":"2024-01-15T17:55:06.788431+0000","last_became_active":"2024-01-15T17:55:04.753730+0000","last_became_peered":"2024-01-15T17:55:04.753730+0000","last_unstale":"2024-01-15T17:55:06.788431+0000","last_undegraded":"2024-01-15T17:55:06.788431+0000","last_fullsized":"2024-01-15T17:55:06.788431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589725+0000","last_change":"2024-01-15T17:55:03.755088+0000","last_active":"2024-01-15T17:55:06.589725+0000","last_peered":"2024-01-15T17:55:06.589725+0000","last_clean":"2024-01-15T17:55:06.589725+0000","last_became_active":"2024-01-15T17:55:03.754805+0000","last_became_peered":"2024-01-15T17:55:03.754805+0000","last_unstale":"2024-01-15T17:55:06.589725+0000","last_undegraded":"2024-01-15T17:55:06.589725+0000","last_fullsized":"2024-01-15T17:55:06.589725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725215+0000","last_change":"2024-01-15T17:55:04.747604+0000","last_active":"2024-01-15T17:55:05.725215+0000","last_peered":"2024-01-15T17:55:05.725215+0000","last_clean":"2024-01-15T17:55:05.725215+0000","last_became_active":"2024-01-15T17:55:04.746687+0000","last_became_peered":"2024-01-15T17:55:04.746687+0000","last_unstale":"2024-01-15T17:55:05.725215+0000","last_undegraded":"2024-01-15T17:55:05.725215+0000","last_fullsized":"2024-01-15T17:55:05.725215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:58.689690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721658+0000","last_change":"2024-01-15T17:55:03.749885+0000","last_active":"2024-01-15T17:55:05.721658+0000","last_peered":"2024-01-15T17:55:05.721658+0000","last_clean":"2024-01-15T17:55:05.721658+0000","last_became_active":"2024-01-15T17:55:03.749600+0000","last_became_peered":"2024-01-15T17:55:03.749600+0000","last_unstale":"2024-01-15T17:55:05.721658+0000","last_undegraded":"2024-01-15T17:55:05.721658+0000","last_fullsized":"2024-01-15T17:55:05.721658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:09.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788706+0000","last_change":"2024-01-15T17:55:04.736486+0000","last_active":"2024-01-15T17:55:06.788706+0000","last_peered":"2024-01-15T17:55:06.788706+0000","last_clean":"2024-01-15T17:55:06.788706+0000","last_became_active":"2024-01-15T17:55:04.736012+0000","last_became_peered":"2024-01-15T17:55:04.736012+0000","last_unstale":"2024-01-15T17:55:06.788706+0000","last_undegraded":"2024-01-15T17:55:06.788706+0000","last_fullsized":"2024-01-15T17:55:06.788706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:53:52.730234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721367+0000","last_change":"2024-01-15T17:55:03.738319+0000","last_active":"2024-01-15T17:55:05.721367+0000","last_peered":"2024-01-15T17:55:05.721367+0000","last_clean":"2024-01-15T17:55:05.721367+0000","last_became_active":"2024-01-15T17:55:03.737946+0000","last_became_peered":"2024-01-15T17:55:03.737946+0000","last_unstale":"2024-01-15T17:55:05.721367+0000","last_undegraded":"2024-01-15T17:55:05.721367+0000","last_fullsized":"2024-01-15T17:55:05.721367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:57.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789078+0000","last_change":"2024-01-15T17:55:04.766996+0000","last_active":"2024-01-15T17:55:06.789078+0000","last_peered":"2024-01-15T17:55:06.789078+0000","last_clean":"2024-01-15T17:55:06.789078+0000","last_became_active":"2024-01-15T17:55:04.766695+0000","last_became_peered":"2024-01-15T17:55:04.766695+0000","last_unstale":"2024-01-15T17:55:06.789078+0000","last_undegraded":"2024-01-15T17:55:06.789078+0000","last_fullsized":"2024-01-15T17:55:06.789078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:21.570608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590906+0000","last_change":"2024-01-15T17:55:03.757123+0000","last_active":"2024-01-15T17:55:06.590906+0000","last_peered":"2024-01-15T17:55:06.590906+0000","last_clean":"2024-01-15T17:55:06.590906+0000","last_became_active":"2024-01-15T17:55:03.756522+0000","last_became_peered":"2024-01-15T17:55:03.756522+0000","last_unstale":"2024-01-15T17:55:06.590906+0000","last_undegraded":"2024-01-15T17:55:06.590906+0000","last_fullsized":"2024-01-15T17:55:06.590906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725414+0000","last_change":"2024-01-15T17:55:04.756691+0000","last_active":"2024-01-15T17:55:05.725414+0000","last_peered":"2024-01-15T17:55:05.725414+0000","last_clean":"2024-01-15T17:55:05.725414+0000","last_became_active":"2024-01-15T17:55:04.756410+0000","last_became_peered":"2024-01-15T17:55:04.756410+0000","last_unstale":"2024-01-15T17:55:05.725414+0000","last_undegraded":"2024-01-15T17:55:05.725414+0000","last_fullsized":"2024-01-15T17:55:05.725414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789737+0000","last_change":"2024-01-15T17:55:04.747374+0000","last_active":"2024-01-15T17:55:06.789737+0000","last_peered":"2024-01-15T17:55:06.789737+0000","last_clean":"2024-01-15T17:55:06.789737+0000","last_became_active":"2024-01-15T17:55:04.747062+0000","last_became_peered":"2024-01-15T17:55:04.747062+0000","last_unstale":"2024-01-15T17:55:06.789737+0000","last_undegraded":"2024-01-15T17:55:06.789737+0000","last_fullsized":"2024-01-15T17:55:06.789737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:44.725915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590116+0000","last_change":"2024-01-15T17:55:03.759441+0000","last_active":"2024-01-15T17:55:06.590116+0000","last_peered":"2024-01-15T17:55:06.590116+0000","last_clean":"2024-01-15T17:55:06.590116+0000","last_became_active":"2024-01-15T17:55:03.759054+0000","last_became_peered":"2024-01-15T17:55:03.759054+0000","last_unstale":"2024-01-15T17:55:06.590116+0000","last_undegraded":"2024-01-15T17:55:06.590116+0000","last_fullsized":"2024-01-15T17:55:06.590116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:19.912976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789716+0000","last_change":"2024-01-15T17:55:04.743796+0000","last_active":"2024-01-15T17:55:06.789716+0000","last_peered":"2024-01-15T17:55:06.789716+0000","last_clean":"2024-01-15T17:55:06.789716+0000","last_became_active":"2024-01-15T17:55:04.743475+0000","last_became_peered":"2024-01-15T17:55:04.743475+0000","last_unstale":"2024-01-15T17:55:06.789716+0000","last_undegraded":"2024-01-15T17:55:06.789716+0000","last_fullsized":"2024-01-15T17:55:06.789716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:52.765350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590613+0000","last_change":"2024-01-15T17:55:03.759401+0000","last_active":"2024-01-15T17:55:06.590613+0000","last_peered":"2024-01-15T17:55:06.590613+0000","last_clean":"2024-01-15T17:55:06.590613+0000","last_became_active":"2024-01-15T17:55:03.759124+0000","last_became_peered":"2024-01-15T17:55:03.759124+0000","last_unstale":"2024-01-15T17:55:06.590613+0000","last_undegraded":"2024-01-15T17:55:06.590613+0000","last_fullsized":"2024-01-15T17:55:06.590613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:09.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788680+0000","last_change":"2024-01-15T17:55:04.757926+0000","last_active":"2024-01-15T17:55:06.788680+0000","last_peered":"2024-01-15T17:55:06.788680+0000","last_clean":"2024-01-15T17:55:06.788680+0000","last_became_active":"2024-01-15T17:55:04.757625+0000","last_became_peered":"2024-01-15T17:55:04.757625+0000","last_unstale":"2024-01-15T17:55:06.788680+0000","last_undegraded":"2024-01-15T17:55:06.788680+0000","last_fullsized":"2024-01-15T17:55:06.788680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790277+0000","last_change":"2024-01-15T17:55:03.736897+0000","last_active":"2024-01-15T17:55:06.790277+0000","last_peered":"2024-01-15T17:55:06.790277+0000","last_clean":"2024-01-15T17:55:06.790277+0000","last_became_active":"2024-01-15T17:55:03.736576+0000","last_became_peered":"2024-01-15T17:55:03.736576+0000","last_unstale":"2024-01-15T17:55:06.790277+0000","last_undegraded":"2024-01-15T17:55:06.790277+0000","last_fullsized":"2024-01-15T17:55:06.790277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:56:59.201106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790294+0000","last_change":"2024-01-15T17:55:04.760100+0000","last_active":"2024-01-15T17:55:06.790294+0000","last_peered":"2024-01-15T17:55:06.790294+0000","last_clean":"2024-01-15T17:55:06.790294+0000","last_became_active":"2024-01-15T17:55:04.756345+0000","last_became_peered":"2024-01-15T17:55:04.756345+0000","last_unstale":"2024-01-15T17:55:06.790294+0000","last_undegraded":"2024-01-15T17:55:06.790294+0000","last_fullsized":"2024-01-15T17:55:06.790294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:06.327821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788935+0000","last_change":"2024-01-15T17:55:03.750992+0000","last_active":"2024-01-15T17:55:06.788935+0000","last_peered":"2024-01-15T17:55:06.788935+0000","last_clean":"2024-01-15T17:55:06.788935+0000","last_became_active":"2024-01-15T17:55:03.750662+0000","last_became_peered":"2024-01-15T17:55:03.750662+0000","last_unstale":"2024-01-15T17:55:06.788935+0000","last_undegraded":"2024-01-15T17:55:06.788935+0000","last_fullsized":"2024-01-15T17:55:06.788935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":823296,"data_stored":669826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0779999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9059999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.726}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":79002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.583}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6919999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2970000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":74188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0470000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":72576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9339999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3370000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0510000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6070000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.954}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.573}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.106999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8510000000000009}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.228999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.272}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0209999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":831488,"data_stored":669882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.677}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0830000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1219999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2280000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:55:09.225 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:55:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.483+0000 7fc6cbe00640 1 Processor -- start 2024-01-15T17:55:09.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.483+0000 7fc6cbe00640 1 -- start start 2024-01-15T17:55:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.485+0000 7fc6cbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc13cfa0 0x7fc6cc15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.485+0000 7fc6cbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6cc13fcb0 0x7fc6cc0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.485+0000 7fc6cbe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6cc0cdec0 con 0x7fc6cc0cd530 2024-01-15T17:55:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.485+0000 7fc6cbe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0ce1a0 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.485+0000 7fc6cbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0ce480 con 0x7fc6cc13cfa0 2024-01-15T17:55:09.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.485+0000 7fc6ca000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc6cc0cd530 0x7fc6cc13f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58710/0 (socket says 172.21.15.26:58710) 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.485+0000 7fc6ca000640 1 -- 172.21.15.26:0/489986272 learned_addr learned my addr 172.21.15.26:0/489986272 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.485+0000 7fc6cb400640 1 --2- 172.21.15.26:0/489986272 >> v2:172.21.15.26:3300/0 conn(0x7fc6cc13fcb0 0x7fc6cc0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.486+0000 7fc6caa00640 1 --2- 172.21.15.26:0/489986272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc13cfa0 0x7fc6cc15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.486+0000 7fc6c9600640 1 -- 172.21.15.26:0/489986272 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103004765 0 0) 0x7fc6cc0cdec0 con 0x7fc6cc0cd530 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.486+0000 7fc6c9600640 1 -- 172.21.15.26:0/489986272 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6ac005730 con 0x7fc6cc0cd530 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.486+0000 7fc6cb400640 1 -- 172.21.15.26:0/489986272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc13cfa0 msgr2=0x7fc6cc15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.486+0000 7fc6cb400640 1 --2- 172.21.15.26:0/489986272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc13cfa0 0x7fc6cc15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.486+0000 7fc6cb400640 1 -- 172.21.15.26:0/489986272 >> v1:172.21.15.26:6789/0 conn(0x7fc6cc0cd530 legacy=0x7fc6cc13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.486+0000 7fc6cb400640 1 -- 172.21.15.26:0/489986272 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc0ce760 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.486+0000 7fc6caa00640 1 --2- 172.21.15.26:0/489986272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc13cfa0 0x7fc6cc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:09.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.487+0000 7fc6cb400640 1 --2- 172.21.15.26:0/489986272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 0x7fc6cc0b2040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009d90 tx=0x7fc6c402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee8ed1120a4dceb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.487+0000 7fc6c9600640 1 -- 172.21.15.26:0/489986272 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c402cbc0 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.487+0000 7fc6c9600640 1 -- 172.21.15.26:0/489986272 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c402cd20 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.487+0000 7fc6c9600640 1 -- 172.21.15.26:0/489986272 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4034660 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.488+0000 7fc6cbe00640 1 -- 172.21.15.26:0/489986272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 msgr2=0x7fc6cc0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.488+0000 7fc6cbe00640 1 --2- 172.21.15.26:0/489986272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 0x7fc6cc0b2040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009d90 tx=0x7fc6c402be10 comp rx=0 tx=0).stop 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.488+0000 7fc6cbe00640 1 -- 172.21.15.26:0/489986272 shutdown_connections 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.488+0000 7fc6cbe00640 1 --2- 172.21.15.26:0/489986272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 0x7fc6cc0b2040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.488+0000 7fc6cbe00640 1 --2- 172.21.15.26:0/489986272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc13cfa0 0x7fc6cc15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.488+0000 7fc6cbe00640 1 -- 172.21.15.26:0/489986272 >> 172.21.15.26:0/489986272 conn(0x7fc6cc06a800 msgr2=0x7fc6cc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.488+0000 7fc6cbe00640 1 -- 172.21.15.26:0/489986272 shutdown_connections 2024-01-15T17:55:09.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.488+0000 7fc6cbe00640 1 -- 172.21.15.26:0/489986272 wait complete. 2024-01-15T17:55:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.490+0000 7fc6cbe00640 1 Processor -- start 2024-01-15T17:55:09.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.490+0000 7fc6cbe00640 1 -- start start 2024-01-15T17:55:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.491+0000 7fc6cbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc0cd530 0x7fc6cc134140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.491+0000 7fc6cbe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6cc13cfa0 0x7fc6cc1f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.491+0000 7fc6cbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 0x7fc6cc0ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.491+0000 7fc6cbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0ad820 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.491+0000 7fc6cbe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc1f3050 con 0x7fc6cc13cfa0 2024-01-15T17:55:09.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.491+0000 7fc6cbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc1f7340 con 0x7fc6cc0cd530 2024-01-15T17:55:09.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.491+0000 7fc6cb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 0x7fc6cc0ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:09.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.491+0000 7fc6cb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 0x7fc6cc0ad2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57544/0 (socket says 172.21.15.26:57544) 2024-01-15T17:55:09.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.491+0000 7fc6cb400640 1 -- 172.21.15.26:0/2243232215 learned_addr learned my addr 172.21.15.26:0/2243232215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6caa00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc0cd530 0x7fc6cc134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6cb400640 1 -- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc0cd530 msgr2=0x7fc6cc134140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6cb400640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc0cd530 0x7fc6cc134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6cb400640 1 -- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6cc13cfa0 msgr2=0x7fc6cc1f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:09.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6ca000640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6cc13cfa0 0x7fc6cc1f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6cb400640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6cc13cfa0 0x7fc6cc1f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6cb400640 1 -- 172.21.15.26:0/2243232215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4009a70 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6ca000640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6cc13cfa0 0x7fc6cc1f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6cb400640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 0x7fc6cc0ad2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009d90 tx=0x7fc6c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6c2a00640 1 -- 172.21.15.26:0/2243232215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4040650 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6c2a00640 1 -- 172.21.15.26:0/2243232215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c402ce90 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.492+0000 7fc6c2a00640 1 -- 172.21.15.26:0/2243232215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c40409f0 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.493+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc1f6aa0 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.493+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc1f7060 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.494+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc1f31c0 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.495+0000 7fc6c2a00640 1 -- 172.21.15.26:0/2243232215 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc6c4040b50 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.496+0000 7fc6c2a00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6a406b2f0 0x7fc6a406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.496+0000 7fc6c2a00640 1 -- 172.21.15.26:0/2243232215 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6c40ac5c0 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.497+0000 7fc6caa00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6a406b2f0 0x7fc6a406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:09.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.498+0000 7fc6caa00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6a406b2f0 0x7fc6a406d7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc006fd0 tx=0x7fc6bc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:09.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.501+0000 7fc6c2a00640 1 -- 172.21.15.26:0/2243232215 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6c4077390 con 0x7fc6cc13fcb0 2024-01-15T17:55:09.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.692+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6cc0b2580 con 0x7fc6a406b2f0 2024-01-15T17:55:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.711+0000 7fc6c2a00640 1 -- 172.21.15.26:0/2243232215 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264485 (secure 0 0 0) 0x7fc6bc00de20 con 0x7fc6a406b2f0 2024-01-15T17:55:09.718 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:09.720 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.717+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6a406b2f0 msgr2=0x7fc6a406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6a406b2f0 0x7fc6a406d7b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc006fd0 tx=0x7fc6bc008040 comp rx=0 tx=0).stop 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 msgr2=0x7fc6cc0ad2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 0x7fc6cc0ad2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc6c4009d90 tx=0x7fc6c4004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 shutdown_connections 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6a406b2f0 0x7fc6a406d7b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6cc13fcb0 0x7fc6cc0ad2e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6cc13cfa0 0x7fc6cc1f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 --2- 172.21.15.26:0/2243232215 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6cc0cd530 0x7fc6cc134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 >> 172.21.15.26:0/2243232215 conn(0x7fc6cc06a800 msgr2=0x7fc6cc0afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 shutdown_connections 2024-01-15T17:55:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:09.718+0000 7fc6cbe00640 1 -- 172.21.15.26:0/2243232215 wait complete. 2024-01-15T17:55:09.741 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-01-15T17:55:07.910271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236160,"kb_used_data":2948,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738560,"statfs":{"total":773094113280,"available":772852285440,"internally_reserved":0,"allocated":3018752,"data_stored":1797664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.979551"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590593+0000","last_change":"2024-01-15T17:55:04.755200+0000","last_active":"2024-01-15T17:55:06.590593+0000","last_peered":"2024-01-15T17:55:06.590593+0000","last_clean":"2024-01-15T17:55:06.590593+0000","last_became_active":"2024-01-15T17:55:04.753634+0000","last_became_peered":"2024-01-15T17:55:04.753634+0000","last_unstale":"2024-01-15T17:55:06.590593+0000","last_undegraded":"2024-01-15T17:55:06.590593+0000","last_fullsized":"2024-01-15T17:55:06.590593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:13.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721381+0000","last_change":"2024-01-15T17:55:04.760607+0000","last_active":"2024-01-15T17:55:05.721381+0000","last_peered":"2024-01-15T17:55:05.721381+0000","last_clean":"2024-01-15T17:55:05.721381+0000","last_became_active":"2024-01-15T17:55:04.760299+0000","last_became_peered":"2024-01-15T17:55:04.760299+0000","last_unstale":"2024-01-15T17:55:05.721381+0000","last_undegraded":"2024-01-15T17:55:05.721381+0000","last_fullsized":"2024-01-15T17:55:05.721381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787705+0000","last_change":"2024-01-15T17:55:04.755000+0000","last_active":"2024-01-15T17:55:06.787705+0000","last_peered":"2024-01-15T17:55:06.787705+0000","last_clean":"2024-01-15T17:55:06.787705+0000","last_became_active":"2024-01-15T17:55:04.754716+0000","last_became_peered":"2024-01-15T17:55:04.754716+0000","last_unstale":"2024-01-15T17:55:06.787705+0000","last_undegraded":"2024-01-15T17:55:06.787705+0000","last_fullsized":"2024-01-15T17:55:06.787705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:55.584774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789609+0000","last_change":"2024-01-15T17:55:04.767626+0000","last_active":"2024-01-15T17:55:06.789609+0000","last_peered":"2024-01-15T17:55:06.789609+0000","last_clean":"2024-01-15T17:55:06.789609+0000","last_became_active":"2024-01-15T17:55:04.767349+0000","last_became_peered":"2024-01-15T17:55:04.767349+0000","last_unstale":"2024-01-15T17:55:06.789609+0000","last_undegraded":"2024-01-15T17:55:06.789609+0000","last_fullsized":"2024-01-15T17:55:06.789609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:17.567853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590635+0000","last_change":"2024-01-15T17:55:04.756971+0000","last_active":"2024-01-15T17:55:06.590635+0000","last_peered":"2024-01-15T17:55:06.590635+0000","last_clean":"2024-01-15T17:55:06.590635+0000","last_became_active":"2024-01-15T17:55:04.756737+0000","last_became_peered":"2024-01-15T17:55:04.756737+0000","last_unstale":"2024-01-15T17:55:06.590635+0000","last_undegraded":"2024-01-15T17:55:06.590635+0000","last_fullsized":"2024-01-15T17:55:06.590635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:44.725915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789155+0000","last_change":"2024-01-15T17:55:04.763592+0000","last_active":"2024-01-15T17:55:06.789155+0000","last_peered":"2024-01-15T17:55:06.789155+0000","last_clean":"2024-01-15T17:55:06.789155+0000","last_became_active":"2024-01-15T17:55:04.763286+0000","last_became_peered":"2024-01-15T17:55:04.763286+0000","last_unstale":"2024-01-15T17:55:06.789155+0000","last_undegraded":"2024-01-15T17:55:06.789155+0000","last_fullsized":"2024-01-15T17:55:06.789155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:48:48.227594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589901+0000","last_change":"2024-01-15T17:55:04.754040+0000","last_active":"2024-01-15T17:55:06.589901+0000","last_peered":"2024-01-15T17:55:06.589901+0000","last_clean":"2024-01-15T17:55:06.589901+0000","last_became_active":"2024-01-15T17:55:04.753698+0000","last_became_peered":"2024-01-15T17:55:04.753698+0000","last_unstale":"2024-01-15T17:55:06.589901+0000","last_undegraded":"2024-01-15T17:55:06.589901+0000","last_fullsized":"2024-01-15T17:55:06.589901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:59.870088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788324+0000","last_change":"2024-01-15T17:55:04.754498+0000","last_active":"2024-01-15T17:55:06.788324+0000","last_peered":"2024-01-15T17:55:06.788324+0000","last_clean":"2024-01-15T17:55:06.788324+0000","last_became_active":"2024-01-15T17:55:04.754091+0000","last_became_peered":"2024-01-15T17:55:04.754091+0000","last_unstale":"2024-01-15T17:55:06.788324+0000","last_undegraded":"2024-01-15T17:55:06.788324+0000","last_fullsized":"2024-01-15T17:55:06.788324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:31.493057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590727+0000","last_change":"2024-01-15T17:55:04.755593+0000","last_active":"2024-01-15T17:55:06.590727+0000","last_peered":"2024-01-15T17:55:06.590727+0000","last_clean":"2024-01-15T17:55:06.590727+0000","last_became_active":"2024-01-15T17:55:04.755376+0000","last_became_peered":"2024-01-15T17:55:04.755376+0000","last_unstale":"2024-01-15T17:55:06.590727+0000","last_undegraded":"2024-01-15T17:55:06.590727+0000","last_fullsized":"2024-01-15T17:55:06.590727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:10:13.660425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726844+0000","last_change":"2024-01-15T17:55:04.745309+0000","last_active":"2024-01-15T17:55:05.726844+0000","last_peered":"2024-01-15T17:55:05.726844+0000","last_clean":"2024-01-15T17:55:05.726844+0000","last_became_active":"2024-01-15T17:55:04.744930+0000","last_became_peered":"2024-01-15T17:55:04.744930+0000","last_unstale":"2024-01-15T17:55:05.726844+0000","last_undegraded":"2024-01-15T17:55:05.726844+0000","last_fullsized":"2024-01-15T17:55:05.726844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726624+0000","last_change":"2024-01-15T17:55:04.743285+0000","last_active":"2024-01-15T17:55:05.726624+0000","last_peered":"2024-01-15T17:55:05.726624+0000","last_clean":"2024-01-15T17:55:05.726624+0000","last_became_active":"2024-01-15T17:55:04.743118+0000","last_became_peered":"2024-01-15T17:55:04.743118+0000","last_unstale":"2024-01-15T17:55:05.726624+0000","last_undegraded":"2024-01-15T17:55:05.726624+0000","last_fullsized":"2024-01-15T17:55:05.726624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726121+0000","last_change":"2024-01-15T17:55:04.739598+0000","last_active":"2024-01-15T17:55:05.726121+0000","last_peered":"2024-01-15T17:55:05.726121+0000","last_clean":"2024-01-15T17:55:05.726121+0000","last_became_active":"2024-01-15T17:55:04.739319+0000","last_became_peered":"2024-01-15T17:55:04.739319+0000","last_unstale":"2024-01-15T17:55:05.726121+0000","last_undegraded":"2024-01-15T17:55:05.726121+0000","last_fullsized":"2024-01-15T17:55:05.726121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:21.113714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788978+0000","last_change":"2024-01-15T17:55:04.751187+0000","last_active":"2024-01-15T17:55:06.788978+0000","last_peered":"2024-01-15T17:55:06.788978+0000","last_clean":"2024-01-15T17:55:06.788978+0000","last_became_active":"2024-01-15T17:55:04.750057+0000","last_became_peered":"2024-01-15T17:55:04.750057+0000","last_unstale":"2024-01-15T17:55:06.788978+0000","last_undegraded":"2024-01-15T17:55:06.788978+0000","last_fullsized":"2024-01-15T17:55:06.788978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:22:07.251406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787842+0000","last_change":"2024-01-15T17:55:04.759443+0000","last_active":"2024-01-15T17:55:06.787842+0000","last_peered":"2024-01-15T17:55:06.787842+0000","last_clean":"2024-01-15T17:55:06.787842+0000","last_became_active":"2024-01-15T17:55:04.759086+0000","last_became_peered":"2024-01-15T17:55:04.759086+0000","last_unstale":"2024-01-15T17:55:06.787842+0000","last_undegraded":"2024-01-15T17:55:06.787842+0000","last_fullsized":"2024-01-15T17:55:06.787842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:27.225767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721835+0000","last_change":"2024-01-15T17:55:04.741854+0000","last_active":"2024-01-15T17:55:05.721835+0000","last_peered":"2024-01-15T17:55:05.721835+0000","last_clean":"2024-01-15T17:55:05.721835+0000","last_became_active":"2024-01-15T17:55:04.741472+0000","last_became_peered":"2024-01-15T17:55:04.741472+0000","last_unstale":"2024-01-15T17:55:05.721835+0000","last_undegraded":"2024-01-15T17:55:05.721835+0000","last_fullsized":"2024-01-15T17:55:05.721835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789012+0000","last_change":"2024-01-15T17:55:04.757919+0000","last_active":"2024-01-15T17:55:06.789012+0000","last_peered":"2024-01-15T17:55:06.789012+0000","last_clean":"2024-01-15T17:55:06.789012+0000","last_became_active":"2024-01-15T17:55:04.757595+0000","last_became_peered":"2024-01-15T17:55:04.757595+0000","last_unstale":"2024-01-15T17:55:06.789012+0000","last_undegraded":"2024-01-15T17:55:06.789012+0000","last_fullsized":"2024-01-15T17:55:06.789012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790339+0000","last_change":"2024-01-15T17:55:04.745091+0000","last_active":"2024-01-15T17:55:06.790339+0000","last_peered":"2024-01-15T17:55:06.790339+0000","last_clean":"2024-01-15T17:55:06.790339+0000","last_became_active":"2024-01-15T17:55:04.744779+0000","last_became_peered":"2024-01-15T17:55:04.744779+0000","last_unstale":"2024-01-15T17:55:06.790339+0000","last_undegraded":"2024-01-15T17:55:06.790339+0000","last_fullsized":"2024-01-15T17:55:06.790339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:10:13.660425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787739+0000","last_change":"2024-01-15T17:55:04.754772+0000","last_active":"2024-01-15T17:55:06.787739+0000","last_peered":"2024-01-15T17:55:06.787739+0000","last_clean":"2024-01-15T17:55:06.787739+0000","last_became_active":"2024-01-15T17:55:04.743420+0000","last_became_peered":"2024-01-15T17:55:04.743420+0000","last_unstale":"2024-01-15T17:55:06.787739+0000","last_undegraded":"2024-01-15T17:55:06.787739+0000","last_fullsized":"2024-01-15T17:55:06.787739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:24:25.848060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591083+0000","last_change":"2024-01-15T17:55:04.741262+0000","last_active":"2024-01-15T17:55:06.591083+0000","last_peered":"2024-01-15T17:55:06.591083+0000","last_clean":"2024-01-15T17:55:06.591083+0000","last_became_active":"2024-01-15T17:55:04.740978+0000","last_became_peered":"2024-01-15T17:55:04.740978+0000","last_unstale":"2024-01-15T17:55:06.591083+0000","last_undegraded":"2024-01-15T17:55:06.591083+0000","last_fullsized":"2024-01-15T17:55:06.591083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:00.893650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726416+0000","last_change":"2024-01-15T17:55:04.758887+0000","last_active":"2024-01-15T17:55:05.726416+0000","last_peered":"2024-01-15T17:55:05.726416+0000","last_clean":"2024-01-15T17:55:05.726416+0000","last_became_active":"2024-01-15T17:55:04.758530+0000","last_became_peered":"2024-01-15T17:55:04.758530+0000","last_unstale":"2024-01-15T17:55:05.726416+0000","last_undegraded":"2024-01-15T17:55:05.726416+0000","last_fullsized":"2024-01-15T17:55:05.726416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788603+0000","last_change":"2024-01-15T17:55:04.756875+0000","last_active":"2024-01-15T17:55:06.788603+0000","last_peered":"2024-01-15T17:55:06.788603+0000","last_clean":"2024-01-15T17:55:06.788603+0000","last_became_active":"2024-01-15T17:55:04.756596+0000","last_became_peered":"2024-01-15T17:55:04.756596+0000","last_unstale":"2024-01-15T17:55:06.788603+0000","last_undegraded":"2024-01-15T17:55:06.788603+0000","last_fullsized":"2024-01-15T17:55:06.788603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789372+0000","last_change":"2024-01-15T17:55:04.746119+0000","last_active":"2024-01-15T17:55:06.789372+0000","last_peered":"2024-01-15T17:55:06.789372+0000","last_clean":"2024-01-15T17:55:06.789372+0000","last_became_active":"2024-01-15T17:55:04.745830+0000","last_became_peered":"2024-01-15T17:55:04.745830+0000","last_unstale":"2024-01-15T17:55:06.789372+0000","last_undegraded":"2024-01-15T17:55:06.789372+0000","last_fullsized":"2024-01-15T17:55:06.789372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:47:39.573232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789300+0000","last_change":"2024-01-15T17:55:04.764403+0000","last_active":"2024-01-15T17:55:06.789300+0000","last_peered":"2024-01-15T17:55:06.789300+0000","last_clean":"2024-01-15T17:55:06.789300+0000","last_became_active":"2024-01-15T17:55:04.763976+0000","last_became_peered":"2024-01-15T17:55:04.763976+0000","last_unstale":"2024-01-15T17:55:06.789300+0000","last_undegraded":"2024-01-15T17:55:06.789300+0000","last_fullsized":"2024-01-15T17:55:06.789300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:08:09.187744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789136+0000","last_change":"2024-01-15T17:55:04.750998+0000","last_active":"2024-01-15T17:55:06.789136+0000","last_peered":"2024-01-15T17:55:06.789136+0000","last_clean":"2024-01-15T17:55:06.789136+0000","last_became_active":"2024-01-15T17:55:04.750216+0000","last_became_peered":"2024-01-15T17:55:04.750216+0000","last_unstale":"2024-01-15T17:55:06.789136+0000","last_undegraded":"2024-01-15T17:55:06.789136+0000","last_fullsized":"2024-01-15T17:55:06.789136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790494+0000","last_change":"2024-01-15T17:55:04.758699+0000","last_active":"2024-01-15T17:55:06.790494+0000","last_peered":"2024-01-15T17:55:06.790494+0000","last_clean":"2024-01-15T17:55:06.790494+0000","last_became_active":"2024-01-15T17:55:04.758066+0000","last_became_peered":"2024-01-15T17:55:04.758066+0000","last_unstale":"2024-01-15T17:55:06.790494+0000","last_undegraded":"2024-01-15T17:55:06.790494+0000","last_fullsized":"2024-01-15T17:55:06.790494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:31:30.922640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727023+0000","last_change":"2024-01-15T17:55:04.749062+0000","last_active":"2024-01-15T17:55:05.727023+0000","last_peered":"2024-01-15T17:55:05.727023+0000","last_clean":"2024-01-15T17:55:05.727023+0000","last_became_active":"2024-01-15T17:55:04.746787+0000","last_became_peered":"2024-01-15T17:55:04.746787+0000","last_unstale":"2024-01-15T17:55:05.727023+0000","last_undegraded":"2024-01-15T17:55:05.727023+0000","last_fullsized":"2024-01-15T17:55:05.727023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721723+0000","last_change":"2024-01-15T17:55:04.753444+0000","last_active":"2024-01-15T17:55:05.721723+0000","last_peered":"2024-01-15T17:55:05.721723+0000","last_clean":"2024-01-15T17:55:05.721723+0000","last_became_active":"2024-01-15T17:55:04.753162+0000","last_became_peered":"2024-01-15T17:55:04.753162+0000","last_unstale":"2024-01-15T17:55:05.721723+0000","last_undegraded":"2024-01-15T17:55:05.721723+0000","last_fullsized":"2024-01-15T17:55:05.721723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790061+0000","last_change":"2024-01-15T17:55:04.764187+0000","last_active":"2024-01-15T17:55:06.790061+0000","last_peered":"2024-01-15T17:55:06.790061+0000","last_clean":"2024-01-15T17:55:06.790061+0000","last_became_active":"2024-01-15T17:55:04.763904+0000","last_became_peered":"2024-01-15T17:55:04.763904+0000","last_unstale":"2024-01-15T17:55:06.790061+0000","last_undegraded":"2024-01-15T17:55:06.790061+0000","last_fullsized":"2024-01-15T17:55:06.790061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:21:21.061668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788913+0000","last_change":"2024-01-15T17:55:04.769007+0000","last_active":"2024-01-15T17:55:06.788913+0000","last_peered":"2024-01-15T17:55:06.788913+0000","last_clean":"2024-01-15T17:55:06.788913+0000","last_became_active":"2024-01-15T17:55:04.767135+0000","last_became_peered":"2024-01-15T17:55:04.767135+0000","last_unstale":"2024-01-15T17:55:06.788913+0000","last_undegraded":"2024-01-15T17:55:06.788913+0000","last_fullsized":"2024-01-15T17:55:06.788913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789565+0000","last_change":"2024-01-15T17:55:04.751046+0000","last_active":"2024-01-15T17:55:06.789565+0000","last_peered":"2024-01-15T17:55:06.789565+0000","last_clean":"2024-01-15T17:55:06.789565+0000","last_became_active":"2024-01-15T17:55:04.750724+0000","last_became_peered":"2024-01-15T17:55:04.750724+0000","last_unstale":"2024-01-15T17:55:06.789565+0000","last_undegraded":"2024-01-15T17:55:06.789565+0000","last_fullsized":"2024-01-15T17:55:06.789565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:59.870088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787248+0000","last_change":"2024-01-15T17:55:04.732768+0000","last_active":"2024-01-15T17:55:06.787248+0000","last_peered":"2024-01-15T17:55:06.787248+0000","last_clean":"2024-01-15T17:55:06.787248+0000","last_became_active":"2024-01-15T17:55:04.732209+0000","last_became_peered":"2024-01-15T17:55:04.732209+0000","last_unstale":"2024-01-15T17:55:06.787248+0000","last_undegraded":"2024-01-15T17:55:06.787248+0000","last_fullsized":"2024-01-15T17:55:06.787248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:21.113714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.724905+0000","last_change":"2024-01-15T17:55:04.762744+0000","last_active":"2024-01-15T17:55:05.724905+0000","last_peered":"2024-01-15T17:55:05.724905+0000","last_clean":"2024-01-15T17:55:05.724905+0000","last_became_active":"2024-01-15T17:55:04.762234+0000","last_became_peered":"2024-01-15T17:55:04.762234+0000","last_unstale":"2024-01-15T17:55:05.724905+0000","last_undegraded":"2024-01-15T17:55:05.724905+0000","last_fullsized":"2024-01-15T17:55:05.724905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:10.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725553+0000","last_change":"2024-01-15T17:55:04.741937+0000","last_active":"2024-01-15T17:55:05.725553+0000","last_peered":"2024-01-15T17:55:05.725553+0000","last_clean":"2024-01-15T17:55:05.725553+0000","last_became_active":"2024-01-15T17:55:04.741629+0000","last_became_peered":"2024-01-15T17:55:04.741629+0000","last_unstale":"2024-01-15T17:55:05.725553+0000","last_undegraded":"2024-01-15T17:55:05.725553+0000","last_fullsized":"2024-01-15T17:55:05.725553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:30.356212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789134+0000","last_change":"2024-01-15T17:55:03.753426+0000","last_active":"2024-01-15T17:55:06.789134+0000","last_peered":"2024-01-15T17:55:06.789134+0000","last_clean":"2024-01-15T17:55:06.789134+0000","last_became_active":"2024-01-15T17:55:03.753141+0000","last_became_peered":"2024-01-15T17:55:03.753141+0000","last_unstale":"2024-01-15T17:55:06.789134+0000","last_undegraded":"2024-01-15T17:55:06.789134+0000","last_fullsized":"2024-01-15T17:55:06.789134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:38.142877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789190+0000","last_change":"2024-01-15T17:55:04.760185+0000","last_active":"2024-01-15T17:55:06.789190+0000","last_peered":"2024-01-15T17:55:06.789190+0000","last_clean":"2024-01-15T17:55:06.789190+0000","last_became_active":"2024-01-15T17:55:04.759883+0000","last_became_peered":"2024-01-15T17:55:04.759883+0000","last_unstale":"2024-01-15T17:55:06.789190+0000","last_undegraded":"2024-01-15T17:55:06.789190+0000","last_fullsized":"2024-01-15T17:55:06.789190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725694+0000","last_change":"2024-01-15T17:55:03.747953+0000","last_active":"2024-01-15T17:55:05.725694+0000","last_peered":"2024-01-15T17:55:05.725694+0000","last_clean":"2024-01-15T17:55:05.725694+0000","last_became_active":"2024-01-15T17:55:03.747648+0000","last_became_peered":"2024-01-15T17:55:03.747648+0000","last_unstale":"2024-01-15T17:55:05.725694+0000","last_undegraded":"2024-01-15T17:55:05.725694+0000","last_fullsized":"2024-01-15T17:55:05.725694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:54.384035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789495+0000","last_change":"2024-01-15T17:55:03.743175+0000","last_active":"2024-01-15T17:55:06.789495+0000","last_peered":"2024-01-15T17:55:06.789495+0000","last_clean":"2024-01-15T17:55:06.789495+0000","last_became_active":"2024-01-15T17:55:03.742887+0000","last_became_peered":"2024-01-15T17:55:03.742887+0000","last_unstale":"2024-01-15T17:55:06.789495+0000","last_undegraded":"2024-01-15T17:55:06.789495+0000","last_fullsized":"2024-01-15T17:55:06.789495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:03.274467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788556+0000","last_change":"2024-01-15T17:55:04.767350+0000","last_active":"2024-01-15T17:55:06.788556+0000","last_peered":"2024-01-15T17:55:06.788556+0000","last_clean":"2024-01-15T17:55:06.788556+0000","last_became_active":"2024-01-15T17:55:04.767083+0000","last_became_peered":"2024-01-15T17:55:04.767083+0000","last_unstale":"2024-01-15T17:55:06.788556+0000","last_undegraded":"2024-01-15T17:55:06.788556+0000","last_fullsized":"2024-01-15T17:55:06.788556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:01.549303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789294+0000","last_change":"2024-01-15T17:55:03.744198+0000","last_active":"2024-01-15T17:55:06.789294+0000","last_peered":"2024-01-15T17:55:06.789294+0000","last_clean":"2024-01-15T17:55:06.789294+0000","last_became_active":"2024-01-15T17:55:03.743838+0000","last_became_peered":"2024-01-15T17:55:03.743838+0000","last_unstale":"2024-01-15T17:55:06.789294+0000","last_undegraded":"2024-01-15T17:55:06.789294+0000","last_fullsized":"2024-01-15T17:55:06.789294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:19.912976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721537+0000","last_change":"2024-01-15T17:55:04.751636+0000","last_active":"2024-01-15T17:55:05.721537+0000","last_peered":"2024-01-15T17:55:05.721537+0000","last_clean":"2024-01-15T17:55:05.721537+0000","last_became_active":"2024-01-15T17:55:04.751353+0000","last_became_peered":"2024-01-15T17:55:04.751353+0000","last_unstale":"2024-01-15T17:55:05.721537+0000","last_undegraded":"2024-01-15T17:55:05.721537+0000","last_fullsized":"2024-01-15T17:55:05.721537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727326+0000","last_change":"2024-01-15T17:55:03.749204+0000","last_active":"2024-01-15T17:55:05.727326+0000","last_peered":"2024-01-15T17:55:05.727326+0000","last_clean":"2024-01-15T17:55:05.727326+0000","last_became_active":"2024-01-15T17:55:03.748941+0000","last_became_peered":"2024-01-15T17:55:03.748941+0000","last_unstale":"2024-01-15T17:55:05.727326+0000","last_undegraded":"2024-01-15T17:55:05.727326+0000","last_fullsized":"2024-01-15T17:55:05.727326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790411+0000","last_change":"2024-01-15T17:55:04.743467+0000","last_active":"2024-01-15T17:55:06.790411+0000","last_peered":"2024-01-15T17:55:06.790411+0000","last_clean":"2024-01-15T17:55:06.790411+0000","last_became_active":"2024-01-15T17:55:04.743161+0000","last_became_peered":"2024-01-15T17:55:04.743161+0000","last_unstale":"2024-01-15T17:55:06.790411+0000","last_undegraded":"2024-01-15T17:55:06.790411+0000","last_fullsized":"2024-01-15T17:55:06.790411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:00.908270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790246+0000","last_change":"2024-01-15T17:55:03.757214+0000","last_active":"2024-01-15T17:55:06.790246+0000","last_peered":"2024-01-15T17:55:06.790246+0000","last_clean":"2024-01-15T17:55:06.790246+0000","last_became_active":"2024-01-15T17:55:03.756900+0000","last_became_peered":"2024-01-15T17:55:03.756900+0000","last_unstale":"2024-01-15T17:55:06.790246+0000","last_undegraded":"2024-01-15T17:55:06.790246+0000","last_fullsized":"2024-01-15T17:55:06.790246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:19.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788839+0000","last_change":"2024-01-15T17:55:04.758670+0000","last_active":"2024-01-15T17:55:06.788839+0000","last_peered":"2024-01-15T17:55:06.788839+0000","last_clean":"2024-01-15T17:55:06.788839+0000","last_became_active":"2024-01-15T17:55:04.757659+0000","last_became_peered":"2024-01-15T17:55:04.757659+0000","last_unstale":"2024-01-15T17:55:06.788839+0000","last_undegraded":"2024-01-15T17:55:06.788839+0000","last_fullsized":"2024-01-15T17:55:06.788839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721816+0000","last_change":"2024-01-15T17:55:03.746513+0000","last_active":"2024-01-15T17:55:05.721816+0000","last_peered":"2024-01-15T17:55:05.721816+0000","last_clean":"2024-01-15T17:55:05.721816+0000","last_became_active":"2024-01-15T17:55:03.746147+0000","last_became_peered":"2024-01-15T17:55:03.746147+0000","last_unstale":"2024-01-15T17:55:05.721816+0000","last_undegraded":"2024-01-15T17:55:05.721816+0000","last_fullsized":"2024-01-15T17:55:05.721816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726333+0000","last_change":"2024-01-15T17:55:04.752615+0000","last_active":"2024-01-15T17:55:05.726333+0000","last_peered":"2024-01-15T17:55:05.726333+0000","last_clean":"2024-01-15T17:55:05.726333+0000","last_became_active":"2024-01-15T17:55:04.751582+0000","last_became_peered":"2024-01-15T17:55:04.751582+0000","last_unstale":"2024-01-15T17:55:05.726333+0000","last_undegraded":"2024-01-15T17:55:05.726333+0000","last_fullsized":"2024-01-15T17:55:05.726333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:24:25.848060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726379+0000","last_change":"2024-01-15T17:55:03.741760+0000","last_active":"2024-01-15T17:55:05.726379+0000","last_peered":"2024-01-15T17:55:05.726379+0000","last_clean":"2024-01-15T17:55:05.726379+0000","last_became_active":"2024-01-15T17:55:03.741327+0000","last_became_peered":"2024-01-15T17:55:03.741327+0000","last_unstale":"2024-01-15T17:55:05.726379+0000","last_undegraded":"2024-01-15T17:55:05.726379+0000","last_fullsized":"2024-01-15T17:55:05.726379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590242+0000","last_change":"2024-01-15T17:55:04.761237+0000","last_active":"2024-01-15T17:55:06.590242+0000","last_peered":"2024-01-15T17:55:06.590242+0000","last_clean":"2024-01-15T17:55:06.590242+0000","last_became_active":"2024-01-15T17:55:04.759551+0000","last_became_peered":"2024-01-15T17:55:04.759551+0000","last_unstale":"2024-01-15T17:55:06.590242+0000","last_undegraded":"2024-01-15T17:55:06.590242+0000","last_fullsized":"2024-01-15T17:55:06.590242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:17.567853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789391+0000","last_change":"2024-01-15T17:55:03.733161+0000","last_active":"2024-01-15T17:55:06.789391+0000","last_peered":"2024-01-15T17:55:06.789391+0000","last_clean":"2024-01-15T17:55:06.789391+0000","last_became_active":"2024-01-15T17:55:03.732839+0000","last_became_peered":"2024-01-15T17:55:03.732839+0000","last_unstale":"2024-01-15T17:55:06.789391+0000","last_undegraded":"2024-01-15T17:55:06.789391+0000","last_fullsized":"2024-01-15T17:55:06.789391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:17.540533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788038+0000","last_change":"2024-01-15T17:55:04.759368+0000","last_active":"2024-01-15T17:55:06.788038+0000","last_peered":"2024-01-15T17:55:06.788038+0000","last_clean":"2024-01-15T17:55:06.788038+0000","last_became_active":"2024-01-15T17:55:04.759069+0000","last_became_peered":"2024-01-15T17:55:04.759069+0000","last_unstale":"2024-01-15T17:55:06.788038+0000","last_undegraded":"2024-01-15T17:55:06.788038+0000","last_fullsized":"2024-01-15T17:55:06.788038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:10.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789653+0000","last_change":"2024-01-15T17:55:03.746380+0000","last_active":"2024-01-15T17:55:06.789653+0000","last_peered":"2024-01-15T17:55:06.789653+0000","last_clean":"2024-01-15T17:55:06.789653+0000","last_became_active":"2024-01-15T17:55:03.745986+0000","last_became_peered":"2024-01-15T17:55:03.745986+0000","last_unstale":"2024-01-15T17:55:06.789653+0000","last_undegraded":"2024-01-15T17:55:06.789653+0000","last_fullsized":"2024-01-15T17:55:06.789653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590808+0000","last_change":"2024-01-15T17:55:04.751522+0000","last_active":"2024-01-15T17:55:06.590808+0000","last_peered":"2024-01-15T17:55:06.590808+0000","last_clean":"2024-01-15T17:55:06.590808+0000","last_became_active":"2024-01-15T17:55:04.751235+0000","last_became_peered":"2024-01-15T17:55:04.751235+0000","last_unstale":"2024-01-15T17:55:06.590808+0000","last_undegraded":"2024-01-15T17:55:06.590808+0000","last_fullsized":"2024-01-15T17:55:06.590808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:00.908270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725362+0000","last_change":"2024-01-15T17:55:03.751331+0000","last_active":"2024-01-15T17:55:05.725362+0000","last_peered":"2024-01-15T17:55:05.725362+0000","last_clean":"2024-01-15T17:55:05.725362+0000","last_became_active":"2024-01-15T17:55:03.749400+0000","last_became_peered":"2024-01-15T17:55:03.749400+0000","last_unstale":"2024-01-15T17:55:05.725362+0000","last_undegraded":"2024-01-15T17:55:05.725362+0000","last_fullsized":"2024-01-15T17:55:05.725362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.724907+0000","last_change":"2024-01-15T17:55:04.761956+0000","last_active":"2024-01-15T17:55:05.724907+0000","last_peered":"2024-01-15T17:55:05.724907+0000","last_clean":"2024-01-15T17:55:05.724907+0000","last_became_active":"2024-01-15T17:55:04.761741+0000","last_became_peered":"2024-01-15T17:55:04.761741+0000","last_unstale":"2024-01-15T17:55:05.724907+0000","last_undegraded":"2024-01-15T17:55:05.724907+0000","last_fullsized":"2024-01-15T17:55:05.724907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787164+0000","last_change":"2024-01-15T17:55:03.735909+0000","last_active":"2024-01-15T17:55:06.787164+0000","last_peered":"2024-01-15T17:55:06.787164+0000","last_clean":"2024-01-15T17:55:06.787164+0000","last_became_active":"2024-01-15T17:55:03.734252+0000","last_became_peered":"2024-01-15T17:55:03.734252+0000","last_unstale":"2024-01-15T17:55:06.787164+0000","last_undegraded":"2024-01-15T17:55:06.787164+0000","last_fullsized":"2024-01-15T17:55:06.787164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590233+0000","last_change":"2024-01-15T17:55:04.753688+0000","last_active":"2024-01-15T17:55:06.590233+0000","last_peered":"2024-01-15T17:55:06.590233+0000","last_clean":"2024-01-15T17:55:06.590233+0000","last_became_active":"2024-01-15T17:55:04.753479+0000","last_became_peered":"2024-01-15T17:55:04.753479+0000","last_unstale":"2024-01-15T17:55:06.590233+0000","last_undegraded":"2024-01-15T17:55:06.590233+0000","last_fullsized":"2024-01-15T17:55:06.590233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:52.765350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726589+0000","last_change":"2024-01-15T17:55:03.740438+0000","last_active":"2024-01-15T17:55:05.726589+0000","last_peered":"2024-01-15T17:55:05.726589+0000","last_clean":"2024-01-15T17:55:05.726589+0000","last_became_active":"2024-01-15T17:55:03.737923+0000","last_became_peered":"2024-01-15T17:55:03.737923+0000","last_unstale":"2024-01-15T17:55:05.726589+0000","last_undegraded":"2024-01-15T17:55:05.726589+0000","last_fullsized":"2024-01-15T17:55:05.726589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591190+0000","last_change":"2024-01-15T17:55:04.756054+0000","last_active":"2024-01-15T17:55:06.591190+0000","last_peered":"2024-01-15T17:55:06.591190+0000","last_clean":"2024-01-15T17:55:06.591190+0000","last_became_active":"2024-01-15T17:55:04.754276+0000","last_became_peered":"2024-01-15T17:55:04.754276+0000","last_unstale":"2024-01-15T17:55:06.591190+0000","last_undegraded":"2024-01-15T17:55:06.591190+0000","last_fullsized":"2024-01-15T17:55:06.591190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:38.181261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787840+0000","last_change":"2024-01-15T17:55:03.744207+0000","last_active":"2024-01-15T17:55:06.787840+0000","last_peered":"2024-01-15T17:55:06.787840+0000","last_clean":"2024-01-15T17:55:06.787840+0000","last_became_active":"2024-01-15T17:55:03.743894+0000","last_became_peered":"2024-01-15T17:55:03.743894+0000","last_unstale":"2024-01-15T17:55:06.787840+0000","last_undegraded":"2024-01-15T17:55:06.787840+0000","last_fullsized":"2024-01-15T17:55:06.787840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591002+0000","last_change":"2024-01-15T17:55:04.758291+0000","last_active":"2024-01-15T17:55:06.591002+0000","last_peered":"2024-01-15T17:55:06.591002+0000","last_clean":"2024-01-15T17:55:06.591002+0000","last_became_active":"2024-01-15T17:55:04.757888+0000","last_became_peered":"2024-01-15T17:55:04.757888+0000","last_unstale":"2024-01-15T17:55:06.591002+0000","last_undegraded":"2024-01-15T17:55:06.591002+0000","last_fullsized":"2024-01-15T17:55:06.591002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:34:45.528954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721818+0000","last_change":"2024-01-15T17:55:03.744276+0000","last_active":"2024-01-15T17:55:05.721818+0000","last_peered":"2024-01-15T17:55:05.721818+0000","last_clean":"2024-01-15T17:55:05.721818+0000","last_became_active":"2024-01-15T17:55:03.744033+0000","last_became_peered":"2024-01-15T17:55:03.744033+0000","last_unstale":"2024-01-15T17:55:05.721818+0000","last_undegraded":"2024-01-15T17:55:05.721818+0000","last_fullsized":"2024-01-15T17:55:05.721818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:17.540533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790107+0000","last_change":"2024-01-15T17:55:04.751400+0000","last_active":"2024-01-15T17:55:06.790107+0000","last_peered":"2024-01-15T17:55:06.790107+0000","last_clean":"2024-01-15T17:55:06.790107+0000","last_became_active":"2024-01-15T17:55:04.751097+0000","last_became_peered":"2024-01-15T17:55:04.751097+0000","last_unstale":"2024-01-15T17:55:06.790107+0000","last_undegraded":"2024-01-15T17:55:06.790107+0000","last_fullsized":"2024-01-15T17:55:06.790107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:56:52.262406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787591+0000","last_change":"2024-01-15T17:55:03.751285+0000","last_active":"2024-01-15T17:55:06.787591+0000","last_peered":"2024-01-15T17:55:06.787591+0000","last_clean":"2024-01-15T17:55:06.787591+0000","last_became_active":"2024-01-15T17:55:03.750926+0000","last_became_peered":"2024-01-15T17:55:03.750926+0000","last_unstale":"2024-01-15T17:55:06.787591+0000","last_undegraded":"2024-01-15T17:55:06.787591+0000","last_fullsized":"2024-01-15T17:55:06.787591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590723+0000","last_change":"2024-01-15T17:55:04.741297+0000","last_active":"2024-01-15T17:55:06.590723+0000","last_peered":"2024-01-15T17:55:06.590723+0000","last_clean":"2024-01-15T17:55:06.590723+0000","last_became_active":"2024-01-15T17:55:04.741044+0000","last_became_peered":"2024-01-15T17:55:04.741044+0000","last_unstale":"2024-01-15T17:55:06.590723+0000","last_undegraded":"2024-01-15T17:55:06.590723+0000","last_fullsized":"2024-01-15T17:55:06.590723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:56:52.262406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-15T17:55:04.731969+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:55:04.731969+0000","last_peered":"2024-01-15T17:55:04.731969+0000","last_clean":"2024-01-15T17:55:04.731969+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:55:04.731969+0000","last_undegraded":"2024-01-15T17:55:04.731969+0000","last_fullsized":"2024-01-15T17:55:04.731969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789182+0000","last_change":"2024-01-15T17:55:03.744683+0000","last_active":"2024-01-15T17:55:06.789182+0000","last_peered":"2024-01-15T17:55:06.789182+0000","last_clean":"2024-01-15T17:55:06.789182+0000","last_became_active":"2024-01-15T17:55:03.743028+0000","last_became_peered":"2024-01-15T17:55:03.743028+0000","last_unstale":"2024-01-15T17:55:06.789182+0000","last_undegraded":"2024-01-15T17:55:06.789182+0000","last_fullsized":"2024-01-15T17:55:06.789182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790120+0000","last_change":"2024-01-15T17:55:04.755598+0000","last_active":"2024-01-15T17:55:06.790120+0000","last_peered":"2024-01-15T17:55:06.790120+0000","last_clean":"2024-01-15T17:55:06.790120+0000","last_became_active":"2024-01-15T17:55:04.755327+0000","last_became_peered":"2024-01-15T17:55:04.755327+0000","last_unstale":"2024-01-15T17:55:06.790120+0000","last_undegraded":"2024-01-15T17:55:06.790120+0000","last_fullsized":"2024-01-15T17:55:06.790120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:47.198851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589792+0000","last_change":"2024-01-15T17:55:03.755061+0000","last_active":"2024-01-15T17:55:06.589792+0000","last_peered":"2024-01-15T17:55:06.589792+0000","last_clean":"2024-01-15T17:55:06.589792+0000","last_became_active":"2024-01-15T17:55:03.754761+0000","last_became_peered":"2024-01-15T17:55:03.754761+0000","last_unstale":"2024-01-15T17:55:06.589792+0000","last_undegraded":"2024-01-15T17:55:06.589792+0000","last_fullsized":"2024-01-15T17:55:06.589792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787429+0000","last_change":"2024-01-15T17:55:04.760196+0000","last_active":"2024-01-15T17:55:06.787429+0000","last_peered":"2024-01-15T17:55:06.787429+0000","last_clean":"2024-01-15T17:55:06.787429+0000","last_became_active":"2024-01-15T17:55:04.759902+0000","last_became_peered":"2024-01-15T17:55:04.759902+0000","last_unstale":"2024-01-15T17:55:06.787429+0000","last_undegraded":"2024-01-15T17:55:06.787429+0000","last_fullsized":"2024-01-15T17:55:06.787429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:57.337185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787323+0000","last_change":"2024-01-15T17:55:03.741130+0000","last_active":"2024-01-15T17:55:06.787323+0000","last_peered":"2024-01-15T17:55:06.787323+0000","last_clean":"2024-01-15T17:55:06.787323+0000","last_became_active":"2024-01-15T17:55:03.740791+0000","last_became_peered":"2024-01-15T17:55:03.740791+0000","last_unstale":"2024-01-15T17:55:06.787323+0000","last_undegraded":"2024-01-15T17:55:06.787323+0000","last_fullsized":"2024-01-15T17:55:06.787323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:57.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721182+0000","last_change":"2024-01-15T17:55:04.739187+0000","last_active":"2024-01-15T17:55:05.721182+0000","last_peered":"2024-01-15T17:55:05.721182+0000","last_clean":"2024-01-15T17:55:05.721182+0000","last_became_active":"2024-01-15T17:55:04.738700+0000","last_became_peered":"2024-01-15T17:55:04.738700+0000","last_unstale":"2024-01-15T17:55:05.721182+0000","last_undegraded":"2024-01-15T17:55:05.721182+0000","last_fullsized":"2024-01-15T17:55:05.721182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790088+0000","last_change":"2024-01-15T17:55:03.741719+0000","last_active":"2024-01-15T17:55:06.790088+0000","last_peered":"2024-01-15T17:55:06.790088+0000","last_clean":"2024-01-15T17:55:06.790088+0000","last_became_active":"2024-01-15T17:55:03.739882+0000","last_became_peered":"2024-01-15T17:55:03.739882+0000","last_unstale":"2024-01-15T17:55:06.790088+0000","last_undegraded":"2024-01-15T17:55:06.790088+0000","last_fullsized":"2024-01-15T17:55:06.790088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725696+0000","last_change":"2024-01-15T17:55:04.766676+0000","last_active":"2024-01-15T17:55:05.725696+0000","last_peered":"2024-01-15T17:55:05.725696+0000","last_clean":"2024-01-15T17:55:05.725696+0000","last_became_active":"2024-01-15T17:55:04.766394+0000","last_became_peered":"2024-01-15T17:55:04.766394+0000","last_unstale":"2024-01-15T17:55:05.725696+0000","last_undegraded":"2024-01-15T17:55:05.725696+0000","last_fullsized":"2024-01-15T17:55:05.725696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:52.036315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721394+0000","last_change":"2024-01-15T17:55:03.735528+0000","last_active":"2024-01-15T17:55:05.721394+0000","last_peered":"2024-01-15T17:55:05.721394+0000","last_clean":"2024-01-15T17:55:05.721394+0000","last_became_active":"2024-01-15T17:55:03.733799+0000","last_became_peered":"2024-01-15T17:55:03.733799+0000","last_unstale":"2024-01-15T17:55:05.721394+0000","last_undegraded":"2024-01-15T17:55:05.721394+0000","last_fullsized":"2024-01-15T17:55:05.721394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789930+0000","last_change":"2024-01-15T17:55:04.750889+0000","last_active":"2024-01-15T17:55:06.789930+0000","last_peered":"2024-01-15T17:55:06.789930+0000","last_clean":"2024-01-15T17:55:06.789930+0000","last_became_active":"2024-01-15T17:55:04.750480+0000","last_became_peered":"2024-01-15T17:55:04.750480+0000","last_unstale":"2024-01-15T17:55:06.789930+0000","last_undegraded":"2024-01-15T17:55:06.789930+0000","last_fullsized":"2024-01-15T17:55:06.789930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:13.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590950+0000","last_change":"2024-01-15T17:55:03.758172+0000","last_active":"2024-01-15T17:55:06.590950+0000","last_peered":"2024-01-15T17:55:06.590950+0000","last_clean":"2024-01-15T17:55:06.590950+0000","last_became_active":"2024-01-15T17:55:03.757742+0000","last_became_peered":"2024-01-15T17:55:03.757742+0000","last_unstale":"2024-01-15T17:55:06.590950+0000","last_undegraded":"2024-01-15T17:55:06.590950+0000","last_fullsized":"2024-01-15T17:55:06.590950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:17.081091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721370+0000","last_change":"2024-01-15T17:55:04.729485+0000","last_active":"2024-01-15T17:55:05.721370+0000","last_peered":"2024-01-15T17:55:05.721370+0000","last_clean":"2024-01-15T17:55:05.721370+0000","last_became_active":"2024-01-15T17:55:04.729029+0000","last_became_peered":"2024-01-15T17:55:04.729029+0000","last_unstale":"2024-01-15T17:55:05.721370+0000","last_undegraded":"2024-01-15T17:55:05.721370+0000","last_fullsized":"2024-01-15T17:55:05.721370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721495+0000","last_change":"2024-01-15T17:55:03.752377+0000","last_active":"2024-01-15T17:55:05.721495+0000","last_peered":"2024-01-15T17:55:05.721495+0000","last_clean":"2024-01-15T17:55:05.721495+0000","last_became_active":"2024-01-15T17:55:03.752164+0000","last_became_peered":"2024-01-15T17:55:03.752164+0000","last_unstale":"2024-01-15T17:55:05.721495+0000","last_undegraded":"2024-01-15T17:55:05.721495+0000","last_fullsized":"2024-01-15T17:55:05.721495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790469+0000","last_change":"2024-01-15T17:55:04.758991+0000","last_active":"2024-01-15T17:55:06.790469+0000","last_peered":"2024-01-15T17:55:06.790469+0000","last_clean":"2024-01-15T17:55:06.790469+0000","last_became_active":"2024-01-15T17:55:04.758422+0000","last_became_peered":"2024-01-15T17:55:04.758422+0000","last_unstale":"2024-01-15T17:55:06.790469+0000","last_undegraded":"2024-01-15T17:55:06.790469+0000","last_fullsized":"2024-01-15T17:55:06.790469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:01:41.493666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727355+0000","last_change":"2024-01-15T17:55:03.754630+0000","last_active":"2024-01-15T17:55:05.727355+0000","last_peered":"2024-01-15T17:55:05.727355+0000","last_clean":"2024-01-15T17:55:05.727355+0000","last_became_active":"2024-01-15T17:55:03.754117+0000","last_became_peered":"2024-01-15T17:55:03.754117+0000","last_unstale":"2024-01-15T17:55:05.727355+0000","last_undegraded":"2024-01-15T17:55:05.727355+0000","last_fullsized":"2024-01-15T17:55:05.727355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:26.025029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788431+0000","last_change":"2024-01-15T17:55:04.754711+0000","last_active":"2024-01-15T17:55:06.788431+0000","last_peered":"2024-01-15T17:55:06.788431+0000","last_clean":"2024-01-15T17:55:06.788431+0000","last_became_active":"2024-01-15T17:55:04.753730+0000","last_became_peered":"2024-01-15T17:55:04.753730+0000","last_unstale":"2024-01-15T17:55:06.788431+0000","last_undegraded":"2024-01-15T17:55:06.788431+0000","last_fullsized":"2024-01-15T17:55:06.788431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589725+0000","last_change":"2024-01-15T17:55:03.755088+0000","last_active":"2024-01-15T17:55:06.589725+0000","last_peered":"2024-01-15T17:55:06.589725+0000","last_clean":"2024-01-15T17:55:06.589725+0000","last_became_active":"2024-01-15T17:55:03.754805+0000","last_became_peered":"2024-01-15T17:55:03.754805+0000","last_unstale":"2024-01-15T17:55:06.589725+0000","last_undegraded":"2024-01-15T17:55:06.589725+0000","last_fullsized":"2024-01-15T17:55:06.589725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725215+0000","last_change":"2024-01-15T17:55:04.747604+0000","last_active":"2024-01-15T17:55:05.725215+0000","last_peered":"2024-01-15T17:55:05.725215+0000","last_clean":"2024-01-15T17:55:05.725215+0000","last_became_active":"2024-01-15T17:55:04.746687+0000","last_became_peered":"2024-01-15T17:55:04.746687+0000","last_unstale":"2024-01-15T17:55:05.725215+0000","last_undegraded":"2024-01-15T17:55:05.725215+0000","last_fullsized":"2024-01-15T17:55:05.725215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:58.689690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721658+0000","last_change":"2024-01-15T17:55:03.749885+0000","last_active":"2024-01-15T17:55:05.721658+0000","last_peered":"2024-01-15T17:55:05.721658+0000","last_clean":"2024-01-15T17:55:05.721658+0000","last_became_active":"2024-01-15T17:55:03.749600+0000","last_became_peered":"2024-01-15T17:55:03.749600+0000","last_unstale":"2024-01-15T17:55:05.721658+0000","last_undegraded":"2024-01-15T17:55:05.721658+0000","last_fullsized":"2024-01-15T17:55:05.721658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:09.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788706+0000","last_change":"2024-01-15T17:55:04.736486+0000","last_active":"2024-01-15T17:55:06.788706+0000","last_peered":"2024-01-15T17:55:06.788706+0000","last_clean":"2024-01-15T17:55:06.788706+0000","last_became_active":"2024-01-15T17:55:04.736012+0000","last_became_peered":"2024-01-15T17:55:04.736012+0000","last_unstale":"2024-01-15T17:55:06.788706+0000","last_undegraded":"2024-01-15T17:55:06.788706+0000","last_fullsized":"2024-01-15T17:55:06.788706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:53:52.730234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721367+0000","last_change":"2024-01-15T17:55:03.738319+0000","last_active":"2024-01-15T17:55:05.721367+0000","last_peered":"2024-01-15T17:55:05.721367+0000","last_clean":"2024-01-15T17:55:05.721367+0000","last_became_active":"2024-01-15T17:55:03.737946+0000","last_became_peered":"2024-01-15T17:55:03.737946+0000","last_unstale":"2024-01-15T17:55:05.721367+0000","last_undegraded":"2024-01-15T17:55:05.721367+0000","last_fullsized":"2024-01-15T17:55:05.721367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:57.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789078+0000","last_change":"2024-01-15T17:55:04.766996+0000","last_active":"2024-01-15T17:55:06.789078+0000","last_peered":"2024-01-15T17:55:06.789078+0000","last_clean":"2024-01-15T17:55:06.789078+0000","last_became_active":"2024-01-15T17:55:04.766695+0000","last_became_peered":"2024-01-15T17:55:04.766695+0000","last_unstale":"2024-01-15T17:55:06.789078+0000","last_undegraded":"2024-01-15T17:55:06.789078+0000","last_fullsized":"2024-01-15T17:55:06.789078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:21.570608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590906+0000","last_change":"2024-01-15T17:55:03.757123+0000","last_active":"2024-01-15T17:55:06.590906+0000","last_peered":"2024-01-15T17:55:06.590906+0000","last_clean":"2024-01-15T17:55:06.590906+0000","last_became_active":"2024-01-15T17:55:03.756522+0000","last_became_peered":"2024-01-15T17:55:03.756522+0000","last_unstale":"2024-01-15T17:55:06.590906+0000","last_undegraded":"2024-01-15T17:55:06.590906+0000","last_fullsized":"2024-01-15T17:55:06.590906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725414+0000","last_change":"2024-01-15T17:55:04.756691+0000","last_active":"2024-01-15T17:55:05.725414+0000","last_peered":"2024-01-15T17:55:05.725414+0000","last_clean":"2024-01-15T17:55:05.725414+0000","last_became_active":"2024-01-15T17:55:04.756410+0000","last_became_peered":"2024-01-15T17:55:04.756410+0000","last_unstale":"2024-01-15T17:55:05.725414+0000","last_undegraded":"2024-01-15T17:55:05.725414+0000","last_fullsized":"2024-01-15T17:55:05.725414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789737+0000","last_change":"2024-01-15T17:55:04.747374+0000","last_active":"2024-01-15T17:55:06.789737+0000","last_peered":"2024-01-15T17:55:06.789737+0000","last_clean":"2024-01-15T17:55:06.789737+0000","last_became_active":"2024-01-15T17:55:04.747062+0000","last_became_peered":"2024-01-15T17:55:04.747062+0000","last_unstale":"2024-01-15T17:55:06.789737+0000","last_undegraded":"2024-01-15T17:55:06.789737+0000","last_fullsized":"2024-01-15T17:55:06.789737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:44.725915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590116+0000","last_change":"2024-01-15T17:55:03.759441+0000","last_active":"2024-01-15T17:55:06.590116+0000","last_peered":"2024-01-15T17:55:06.590116+0000","last_clean":"2024-01-15T17:55:06.590116+0000","last_became_active":"2024-01-15T17:55:03.759054+0000","last_became_peered":"2024-01-15T17:55:03.759054+0000","last_unstale":"2024-01-15T17:55:06.590116+0000","last_undegraded":"2024-01-15T17:55:06.590116+0000","last_fullsized":"2024-01-15T17:55:06.590116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:19.912976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789716+0000","last_change":"2024-01-15T17:55:04.743796+0000","last_active":"2024-01-15T17:55:06.789716+0000","last_peered":"2024-01-15T17:55:06.789716+0000","last_clean":"2024-01-15T17:55:06.789716+0000","last_became_active":"2024-01-15T17:55:04.743475+0000","last_became_peered":"2024-01-15T17:55:04.743475+0000","last_unstale":"2024-01-15T17:55:06.789716+0000","last_undegraded":"2024-01-15T17:55:06.789716+0000","last_fullsized":"2024-01-15T17:55:06.789716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:52.765350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590613+0000","last_change":"2024-01-15T17:55:03.759401+0000","last_active":"2024-01-15T17:55:06.590613+0000","last_peered":"2024-01-15T17:55:06.590613+0000","last_clean":"2024-01-15T17:55:06.590613+0000","last_became_active":"2024-01-15T17:55:03.759124+0000","last_became_peered":"2024-01-15T17:55:03.759124+0000","last_unstale":"2024-01-15T17:55:06.590613+0000","last_undegraded":"2024-01-15T17:55:06.590613+0000","last_fullsized":"2024-01-15T17:55:06.590613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:09.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788680+0000","last_change":"2024-01-15T17:55:04.757926+0000","last_active":"2024-01-15T17:55:06.788680+0000","last_peered":"2024-01-15T17:55:06.788680+0000","last_clean":"2024-01-15T17:55:06.788680+0000","last_became_active":"2024-01-15T17:55:04.757625+0000","last_became_peered":"2024-01-15T17:55:04.757625+0000","last_unstale":"2024-01-15T17:55:06.788680+0000","last_undegraded":"2024-01-15T17:55:06.788680+0000","last_fullsized":"2024-01-15T17:55:06.788680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790277+0000","last_change":"2024-01-15T17:55:03.736897+0000","last_active":"2024-01-15T17:55:06.790277+0000","last_peered":"2024-01-15T17:55:06.790277+0000","last_clean":"2024-01-15T17:55:06.790277+0000","last_became_active":"2024-01-15T17:55:03.736576+0000","last_became_peered":"2024-01-15T17:55:03.736576+0000","last_unstale":"2024-01-15T17:55:06.790277+0000","last_undegraded":"2024-01-15T17:55:06.790277+0000","last_fullsized":"2024-01-15T17:55:06.790277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:56:59.201106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790294+0000","last_change":"2024-01-15T17:55:04.760100+0000","last_active":"2024-01-15T17:55:06.790294+0000","last_peered":"2024-01-15T17:55:06.790294+0000","last_clean":"2024-01-15T17:55:06.790294+0000","last_became_active":"2024-01-15T17:55:04.756345+0000","last_became_peered":"2024-01-15T17:55:04.756345+0000","last_unstale":"2024-01-15T17:55:06.790294+0000","last_undegraded":"2024-01-15T17:55:06.790294+0000","last_fullsized":"2024-01-15T17:55:06.790294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:06.327821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788935+0000","last_change":"2024-01-15T17:55:03.750992+0000","last_active":"2024-01-15T17:55:06.788935+0000","last_peered":"2024-01-15T17:55:06.788935+0000","last_clean":"2024-01-15T17:55:06.788935+0000","last_became_active":"2024-01-15T17:55:03.750662+0000","last_became_peered":"2024-01-15T17:55:03.750662+0000","last_unstale":"2024-01-15T17:55:06.788935+0000","last_undegraded":"2024-01-15T17:55:06.788935+0000","last_fullsized":"2024-01-15T17:55:06.788935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":823296,"data_stored":669826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0779999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9059999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8069999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542147,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5640000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.726}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.266}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]}]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":79002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.583}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6919999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9850000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3820000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2970000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":74188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0470000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6930000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":212992,"data_stored":72576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0860000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9339999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.008}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3370000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0510000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.6070000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.954}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.573}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":22.106999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8510000000000009}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":13.228999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.272}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0209999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":831488,"data_stored":669882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.756}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.677}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0830000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1219999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2280000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:55:09.744 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T17:55:09.745 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-15T17:55:09.745 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-15T17:55:09.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-15T17:55:09.745 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-15T17:55:10.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.001+0000 7fc535400640 1 Processor -- start 2024-01-15T17:55:10.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.001+0000 7fc535400640 1 -- start start 2024-01-15T17:55:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.003+0000 7fc535400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5300d0100 0x7fc5300b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.003+0000 7fc535400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5300d0a10 con 0x7fc5300d0f40 2024-01-15T17:55:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.003+0000 7fc535400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5300cf080 con 0x7fc530160920 2024-01-15T17:55:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.003+0000 7fc535400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5300c6be0 con 0x7fc5300d0100 2024-01-15T17:55:10.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.003+0000 7fc527e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5300d0100 0x7fc5300b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.003+0000 7fc534a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc5300d0f40 0x7fc5300d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58716/0 (socket says 172.21.15.26:58716) 2024-01-15T17:55:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.003+0000 7fc534a00640 1 -- 172.21.15.26:0/3297246974 learned_addr learned my addr 172.21.15.26:0/3297246974 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.004+0000 7fc527e00640 1 -- 172.21.15.26:0/3297246974 >> v1:172.21.15.124:6789/0 conn(0x7fc530160920 legacy=0x7fc5300cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.004+0000 7fc527e00640 1 -- 172.21.15.26:0/3297246974 >> v1:172.21.15.26:6789/0 conn(0x7fc5300d0f40 legacy=0x7fc5300d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.004+0000 7fc527e00640 1 -- 172.21.15.26:0/3297246974 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5300ce7e0 con 0x7fc5300d0100 2024-01-15T17:55:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.004+0000 7fc52f400640 1 -- 172.21.15.26:0/3297246974 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221548057 0 0) 0x7fc5300d0a10 con 0x7fc5300d0f40 2024-01-15T17:55:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.005+0000 7fc527e00640 1 --2- 172.21.15.26:0/3297246974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0100 0x7fc5300b3f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc5180097a0 tx=0x7fc51802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a21dfc7e34f6add server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.005+0000 7fc52f400640 1 -- 172.21.15.26:0/3297246974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518031830 con 0x7fc5300d0100 2024-01-15T17:55:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.005+0000 7fc52f400640 1 -- 172.21.15.26:0/3297246974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc518031990 con 0x7fc5300d0100 2024-01-15T17:55:10.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.005+0000 7fc52f400640 1 -- 172.21.15.26:0/3297246974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518031d20 con 0x7fc5300d0100 2024-01-15T17:55:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.006+0000 7fc535400640 1 -- 172.21.15.26:0/3297246974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0100 msgr2=0x7fc5300b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.006+0000 7fc535400640 1 --2- 172.21.15.26:0/3297246974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0100 0x7fc5300b3f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc5180097a0 tx=0x7fc51802b8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.006+0000 7fc535400640 1 -- 172.21.15.26:0/3297246974 shutdown_connections 2024-01-15T17:55:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.006+0000 7fc535400640 1 --2- 172.21.15.26:0/3297246974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0100 0x7fc5300b3f30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.006+0000 7fc535400640 1 -- 172.21.15.26:0/3297246974 >> 172.21.15.26:0/3297246974 conn(0x7fc5300d3090 msgr2=0x7fc5300d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.006+0000 7fc535400640 1 -- 172.21.15.26:0/3297246974 shutdown_connections 2024-01-15T17:55:10.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.006+0000 7fc535400640 1 -- 172.21.15.26:0/3297246974 wait complete. 2024-01-15T17:55:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.008+0000 7fc535400640 1 Processor -- start 2024-01-15T17:55:10.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.008+0000 7fc535400640 1 -- start start 2024-01-15T17:55:10.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.009+0000 7fc535400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5300d0100 0x7fc53015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.009+0000 7fc535400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0f40 0x7fc5300c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.009+0000 7fc535400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc530160920 0x7fc5300c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.009+0000 7fc535400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5300cb8d0 con 0x7fc5300d0f40 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.009+0000 7fc535400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5301f3f20 con 0x7fc5300d0100 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.009+0000 7fc535400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5301f4a70 con 0x7fc530160920 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc527e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0f40 0x7fc5300c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc527e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0f40 0x7fc5300c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57572/0 (socket says 172.21.15.26:57572) 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc527e00640 1 -- 172.21.15.26:0/2316285484 learned_addr learned my addr 172.21.15.26:0/2316285484 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc52fe00640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5300d0100 0x7fc53015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc527e00640 1 -- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc530160920 msgr2=0x7fc5300c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc534a00640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc530160920 0x7fc5300c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc527e00640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc530160920 0x7fc5300c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc527e00640 1 -- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5300d0100 msgr2=0x7fc53015c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc527e00640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5300d0100 0x7fc53015c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc527e00640 1 -- 172.21.15.26:0/2316285484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc518009400 con 0x7fc5300d0f40 2024-01-15T17:55:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.010+0000 7fc527e00640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0f40 0x7fc5300c9090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc51802bee0 tx=0x7fc5180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.011+0000 7fc52cc00640 1 -- 172.21.15.26:0/2316285484 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51803ba70 con 0x7fc5300d0f40 2024-01-15T17:55:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.011+0000 7fc52cc00640 1 -- 172.21.15.26:0/2316285484 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51803bbd0 con 0x7fc5300d0f40 2024-01-15T17:55:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.011+0000 7fc52cc00640 1 -- 172.21.15.26:0/2316285484 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5180453f0 con 0x7fc5300d0f40 2024-01-15T17:55:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.011+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5301f4140 con 0x7fc5300d0f40 2024-01-15T17:55:10.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.011+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5301f4700 con 0x7fc5300d0f40 2024-01-15T17:55:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.012+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc530067450 con 0x7fc5300d0f40 2024-01-15T17:55:10.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.013+0000 7fc52cc00640 1 -- 172.21.15.26:0/2316285484 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc518045550 con 0x7fc5300d0f40 2024-01-15T17:55:10.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.014+0000 7fc52cc00640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc50806b340 0x7fc50806d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.014+0000 7fc52cc00640 1 -- 172.21.15.26:0/2316285484 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5180abe70 con 0x7fc5300d0f40 2024-01-15T17:55:10.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.015+0000 7fc52fe00640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc50806b340 0x7fc50806d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.016+0000 7fc52fe00640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc50806b340 0x7fc50806d800 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc520006fd0 tx=0x7fc520008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:10.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.019+0000 7fc52cc00640 1 -- 172.21.15.26:0/2316285484 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc518076cc0 con 0x7fc5300d0f40 2024-01-15T17:55:10.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.266+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc53008c7e0 con 0x7fc5300d0f40 2024-01-15T17:55:10.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.276+0000 7fc52cc00640 1 -- 172.21.15.26:0/2316285484 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171565 (secure 0 0 0) 0x7fc51807a2e0 con 0x7fc5300d0f40 2024-01-15T17:55:10.281 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc50806b340 msgr2=0x7fc50806d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc50806b340 0x7fc50806d800 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc520006fd0 tx=0x7fc520008040 comp rx=0 tx=0).stop 2024-01-15T17:55:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0f40 msgr2=0x7fc5300c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0f40 0x7fc5300c9090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc51802bee0 tx=0x7fc5180046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 shutdown_connections 2024-01-15T17:55:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc530160920 0x7fc5300c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc50806b340 0x7fc50806d800 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5300d0f40 0x7fc5300c9090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 --2- 172.21.15.26:0/2316285484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5300d0100 0x7fc53015c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 >> 172.21.15.26:0/2316285484 conn(0x7fc5300d3090 msgr2=0x7fc530160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 shutdown_connections 2024-01-15T17:55:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.280+0000 7fc535400640 1 -- 172.21.15.26:0/2316285484 wait complete. 2024-01-15T17:55:10.304 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":5,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6832","nonce":2029932521},{"type":"v1","addr":"172.21.15.124:6833","nonce":2029932521}]},"active_addr":"172.21.15.124:6833/2029932521","active_change":"2024-01-15T17:54:49.888970+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"clone_delay_every_regfile":{"name":"clone_delay_every_regfile","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Sleep for specified seconds after copying a regfile","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"clone_delay_every_regfile":{"name":"clone_delay_every_regfile","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Sleep for specified seconds after copying a regfile","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":2084148042}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":3897041539}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":3885509634}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.124:0","nonce":1530511876}]}]} 2024-01-15T17:55:10.307 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-15T17:55:10.307 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-15T17:55:10.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.519+0000 7f40af200640 1 Processor -- start 2024-01-15T17:55:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.519+0000 7f40af200640 1 -- start start 2024-01-15T17:55:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.520+0000 7f40af200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f40b00d0100 0x7f40b00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.520+0000 7f40af200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f40b00d0f40 0x7f40b00d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.520+0000 7f40af200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b00d0a10 con 0x7f40b0160920 2024-01-15T17:55:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.520+0000 7f40af200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b00cf080 con 0x7f40b00d0100 2024-01-15T17:55:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.520+0000 7f40af200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b00c6be0 con 0x7f40b00d0f40 2024-01-15T17:55:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.521+0000 7f40ad400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f40b00d0100 0x7f40b00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.521+0000 7f40ad400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f40b00d0100 0x7f40b00b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57578/0 (socket says 172.21.15.26:57578) 2024-01-15T17:55:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.521+0000 7f40ad400640 1 -- 172.21.15.26:0/2490479868 learned_addr learned my addr 172.21.15.26:0/2490479868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:10.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.521+0000 7f40ae800640 1 --2- 172.21.15.26:0/2490479868 >> v2:172.21.15.124:3300/0 conn(0x7f40b00d0f40 0x7f40b00d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.521+0000 7f40ad400640 1 -- 172.21.15.26:0/2490479868 >> v2:172.21.15.124:3300/0 conn(0x7f40b00d0f40 msgr2=0x7f40b00d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.521+0000 7f40ad400640 1 --2- 172.21.15.26:0/2490479868 >> v2:172.21.15.124:3300/0 conn(0x7f40b00d0f40 0x7f40b00d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.521+0000 7f40ad400640 1 -- 172.21.15.26:0/2490479868 >> v1:172.21.15.124:6789/0 conn(0x7f40b0160920 legacy=0x7f40b00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.521+0000 7f40ad400640 1 -- 172.21.15.26:0/2490479868 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00ce7e0 con 0x7f40b00d0100 2024-01-15T17:55:10.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.522+0000 7f40ad400640 1 --2- 172.21.15.26:0/2490479868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b00d0100 0x7f40b00b3f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f40b01620b0 tx=0x7f409802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79df3717f3b8fa31 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:10.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40aca00640 1 -- 172.21.15.26:0/2490479868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098031870 con 0x7f40b00d0100 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40aca00640 1 -- 172.21.15.26:0/2490479868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40980319d0 con 0x7f40b00d0100 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40aca00640 1 -- 172.21.15.26:0/2490479868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098031d60 con 0x7f40b00d0100 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40af200640 1 -- 172.21.15.26:0/2490479868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b00d0100 msgr2=0x7f40b00b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40af200640 1 --2- 172.21.15.26:0/2490479868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b00d0100 0x7f40b00b3f30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f40b01620b0 tx=0x7f409802b910 comp rx=0 tx=0).stop 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40af200640 1 -- 172.21.15.26:0/2490479868 shutdown_connections 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40af200640 1 --2- 172.21.15.26:0/2490479868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b00d0100 0x7f40b00b3f30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40af200640 1 --2- 172.21.15.26:0/2490479868 >> v2:172.21.15.124:3300/0 conn(0x7f40b00d0f40 0x7f40b00d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40af200640 1 -- 172.21.15.26:0/2490479868 >> 172.21.15.26:0/2490479868 conn(0x7f40b00d3090 msgr2=0x7f40b00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40af200640 1 -- 172.21.15.26:0/2490479868 shutdown_connections 2024-01-15T17:55:10.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.523+0000 7f40af200640 1 -- 172.21.15.26:0/2490479868 wait complete. 2024-01-15T17:55:10.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.525+0000 7f40af200640 1 Processor -- start 2024-01-15T17:55:10.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.525+0000 7f40af200640 1 -- start start 2024-01-15T17:55:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.526+0000 7f40af200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40b00d0100 0x7f40b00afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.526+0000 7f40af200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40b00d0f40 0x7f40b015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.526+0000 7f40af200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b0160920 0x7f40b015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.526+0000 7f40af200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00b0190 con 0x7f40b0160920 2024-01-15T17:55:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.526+0000 7f40af200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00b7e30 con 0x7f40b00d0f40 2024-01-15T17:55:10.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.526+0000 7f40af200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00b8110 con 0x7f40b00d0100 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ae800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b0160920 0x7f40b015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ae800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b0160920 0x7f40b015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57582/0 (socket says 172.21.15.26:57582) 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ae800640 1 -- 172.21.15.26:0/1435764320 learned_addr learned my addr 172.21.15.26:0/1435764320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ad400640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40b00d0f40 0x7f40b015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ade00640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40b00d0100 0x7f40b00afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ae800640 1 -- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40b00d0100 msgr2=0x7f40b00afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ae800640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40b00d0100 0x7f40b00afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ae800640 1 -- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40b00d0f40 msgr2=0x7f40b015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ae800640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40b00d0f40 0x7f40b015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ae800640 1 -- 172.21.15.26:0/1435764320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40980094d0 con 0x7f40b0160920 2024-01-15T17:55:10.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.527+0000 7f40ae800640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b0160920 0x7f40b015af60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40a800efc0 tx=0x7f40a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.528+0000 7f40a6000640 1 -- 172.21.15.26:0/1435764320 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8014480 con 0x7f40b0160920 2024-01-15T17:55:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.528+0000 7f40a6000640 1 -- 172.21.15.26:0/1435764320 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a800f040 con 0x7f40b0160920 2024-01-15T17:55:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.528+0000 7f40a6000640 1 -- 172.21.15.26:0/1435764320 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8014820 con 0x7f40b0160920 2024-01-15T17:55:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.528+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b00b83f0 con 0x7f40b0160920 2024-01-15T17:55:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.528+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b015ab70 con 0x7f40b0160920 2024-01-15T17:55:10.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.529+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b00b8910 con 0x7f40b0160920 2024-01-15T17:55:10.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.530+0000 7f40a6000640 1 -- 172.21.15.26:0/1435764320 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f40a8014980 con 0x7f40b0160920 2024-01-15T17:55:10.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.531+0000 7f40a6000640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f408806b2c0 0x7f408806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.531+0000 7f40a6000640 1 -- 172.21.15.26:0/1435764320 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40a808a8a0 con 0x7f40b0160920 2024-01-15T17:55:10.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.532+0000 7f40ade00640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f408806b2c0 0x7f408806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.533+0000 7f40ade00640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f408806b2c0 0x7f408806d780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006fd0 tx=0x7f40a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:10.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.536+0000 7f40a6000640 1 -- 172.21.15.26:0/1435764320 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40b00b8910 con 0x7f40b0160920 2024-01-15T17:55:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.732+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40b00b8910 con 0x7f40b0160920 2024-01-15T17:55:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.735+0000 7f40a6000640 1 -- 172.21.15.26:0/1435764320 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f40b00b8910 con 0x7f40b0160920 2024-01-15T17:55:10.739 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:10.739 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:10.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.738+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f408806b2c0 msgr2=0x7f408806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f408806b2c0 0x7f408806d780 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006fd0 tx=0x7f40a0008040 comp rx=0 tx=0).stop 2024-01-15T17:55:10.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b0160920 msgr2=0x7f40b015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b0160920 0x7f40b015af60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f40a800efc0 tx=0x7f40a800c490 comp rx=0 tx=0).stop 2024-01-15T17:55:10.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 shutdown_connections 2024-01-15T17:55:10.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f408806b2c0 0x7f408806d780 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40b0160920 0x7f40b015af60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40b00d0f40 0x7f40b015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 --2- 172.21.15.26:0/1435764320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40b00d0100 0x7f40b00afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 >> 172.21.15.26:0/1435764320 conn(0x7f40b00d3090 msgr2=0x7f40b015d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 shutdown_connections 2024-01-15T17:55:10.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.739+0000 7f40af200640 1 -- 172.21.15.26:0/1435764320 wait complete. 2024-01-15T17:55:10.763 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-15T17:55:10.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.983+0000 7ff3f6000640 1 Processor -- start 2024-01-15T17:55:10.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.983+0000 7ff3f6000640 1 -- start start 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3f6000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff3f013df80 0x7ff3f015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3f6000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3f013e360 0x7ff3f00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3f6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 0x7ff3f00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3f6000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f006c470 con 0x7ff3f013e360 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3f6000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f006c750 con 0x7ff3f013df80 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3f6000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f006ca30 con 0x7ff3f015f9c0 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3eea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3f013e360 0x7ff3f00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3eea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3f013e360 0x7ff3f00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57590/0 (socket says 172.21.15.26:57590) 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3eea00640 1 -- 172.21.15.26:0/4110742896 learned_addr learned my addr 172.21.15.26:0/4110742896 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:10.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3efe00640 1 --2- 172.21.15.26:0/4110742896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 0x7ff3f00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3eea00640 1 -- 172.21.15.26:0/4110742896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 msgr2=0x7ff3f00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:10.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3ef400640 1 --2- 172.21.15.26:0/4110742896 >> v2:172.21.15.124:3300/0 conn(0x7ff3f013df80 0x7ff3f015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.985+0000 7ff3eea00640 1 --2- 172.21.15.26:0/4110742896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 0x7ff3f00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.986+0000 7ff3eea00640 1 -- 172.21.15.26:0/4110742896 >> v2:172.21.15.124:3300/0 conn(0x7ff3f013df80 msgr2=0x7ff3f015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:10.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.986+0000 7ff3eea00640 1 --2- 172.21.15.26:0/4110742896 >> v2:172.21.15.124:3300/0 conn(0x7ff3f013df80 0x7ff3f015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.986+0000 7ff3eea00640 1 -- 172.21.15.26:0/4110742896 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f006cd10 con 0x7ff3f013e360 2024-01-15T17:55:10.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.986+0000 7ff3efe00640 1 --2- 172.21.15.26:0/4110742896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 0x7ff3f00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:10.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.987+0000 7ff3eea00640 1 --2- 172.21.15.26:0/4110742896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013e360 0x7ff3f00cf290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0072770 tx=0x7ff3d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa223af5d22ea65 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:10.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.987+0000 7ff3ee000640 1 -- 172.21.15.26:0/4110742896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8031870 con 0x7ff3f013e360 2024-01-15T17:55:10.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.987+0000 7ff3ee000640 1 -- 172.21.15.26:0/4110742896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d80319d0 con 0x7ff3f013e360 2024-01-15T17:55:10.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.987+0000 7ff3ee000640 1 -- 172.21.15.26:0/4110742896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8031d60 con 0x7ff3f013e360 2024-01-15T17:55:10.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.987+0000 7ff3f6000640 1 -- 172.21.15.26:0/4110742896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013e360 msgr2=0x7ff3f00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.987+0000 7ff3f6000640 1 --2- 172.21.15.26:0/4110742896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013e360 0x7ff3f00cf290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0072770 tx=0x7ff3d802b910 comp rx=0 tx=0).stop 2024-01-15T17:55:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.988+0000 7ff3f6000640 1 -- 172.21.15.26:0/4110742896 shutdown_connections 2024-01-15T17:55:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.988+0000 7ff3f6000640 1 --2- 172.21.15.26:0/4110742896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 0x7ff3f00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.988+0000 7ff3f6000640 1 --2- 172.21.15.26:0/4110742896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013e360 0x7ff3f00cf290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.988+0000 7ff3f6000640 1 --2- 172.21.15.26:0/4110742896 >> v2:172.21.15.124:3300/0 conn(0x7ff3f013df80 0x7ff3f015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.988+0000 7ff3f6000640 1 -- 172.21.15.26:0/4110742896 >> 172.21.15.26:0/4110742896 conn(0x7ff3f006a800 msgr2=0x7ff3f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.988+0000 7ff3f6000640 1 -- 172.21.15.26:0/4110742896 shutdown_connections 2024-01-15T17:55:10.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.988+0000 7ff3f6000640 1 -- 172.21.15.26:0/4110742896 wait complete. 2024-01-15T17:55:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.989+0000 7ff3f6000640 1 Processor -- start 2024-01-15T17:55:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.990+0000 7ff3f6000640 1 -- start start 2024-01-15T17:55:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.990+0000 7ff3f6000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013df80 0x7ff3f013b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.990+0000 7ff3f6000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f013e360 0x7ff3f01f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.990+0000 7ff3f6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 0x7ff3f01f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3f6000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f01f5060 con 0x7ff3f013df80 2024-01-15T17:55:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3f6000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f013c3b0 con 0x7ff3f013e360 2024-01-15T17:55:10.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3f6000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f013b830 con 0x7ff3f015f9c0 2024-01-15T17:55:10.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3ef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013df80 0x7ff3f013b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3ef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013df80 0x7ff3f013b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57600/0 (socket says 172.21.15.26:57600) 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3ef400640 1 -- 172.21.15.26:0/795863190 learned_addr learned my addr 172.21.15.26:0/795863190 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3eea00640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f013e360 0x7ff3f01f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3efe00640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 0x7ff3f01f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3ef400640 1 -- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 msgr2=0x7ff3f01f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3ef400640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 0x7ff3f01f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3ef400640 1 -- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f013e360 msgr2=0x7ff3f01f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3ef400640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f013e360 0x7ff3f01f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.991+0000 7ff3ef400640 1 -- 172.21.15.26:0/795863190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d80094d0 con 0x7ff3f013df80 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.992+0000 7ff3ef400640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013df80 0x7ff3f013b250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000ccd0 tx=0x7ff3e000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:10.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.992+0000 7ff3e7400640 1 -- 172.21.15.26:0/795863190 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0003990 con 0x7ff3f013df80 2024-01-15T17:55:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.992+0000 7ff3e7400640 1 -- 172.21.15.26:0/795863190 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e0003af0 con 0x7ff3f013df80 2024-01-15T17:55:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.992+0000 7ff3e7400640 1 -- 172.21.15.26:0/795863190 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0004020 con 0x7ff3f013df80 2024-01-15T17:55:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.992+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f013bb10 con 0x7ff3f013df80 2024-01-15T17:55:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.992+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f013c0d0 con 0x7ff3f013df80 2024-01-15T17:55:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.993+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f0137cd0 con 0x7ff3f013df80 2024-01-15T17:55:10.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.994+0000 7ff3e7400640 1 -- 172.21.15.26:0/795863190 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff3e0004180 con 0x7ff3f013df80 2024-01-15T17:55:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.996+0000 7ff3e7400640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff3c806b2c0 0x7ff3c806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:10.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.996+0000 7ff3e7400640 1 -- 172.21.15.26:0/795863190 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3e008a0a0 con 0x7ff3f013df80 2024-01-15T17:55:10.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.996+0000 7ff3eea00640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff3c806b2c0 0x7ff3c806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:10.997+0000 7ff3eea00640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff3c806b2c0 0x7ff3c806d780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8036000 tx=0x7ff3d803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.000+0000 7ff3e7400640 1 -- 172.21.15.26:0/795863190 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3f0137cd0 con 0x7ff3f013df80 2024-01-15T17:55:11.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.197+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3f0137cd0 con 0x7ff3f013df80 2024-01-15T17:55:11.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.200+0000 7ff3e7400640 1 -- 172.21.15.26:0/795863190 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7ff3f0137cd0 con 0x7ff3f013df80 2024-01-15T17:55:11.204 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:11.204 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:11.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff3c806b2c0 msgr2=0x7ff3c806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff3c806b2c0 0x7ff3c806d780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8036000 tx=0x7ff3d803c040 comp rx=0 tx=0).stop 2024-01-15T17:55:11.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013df80 msgr2=0x7ff3f013b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013df80 0x7ff3f013b250 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000ccd0 tx=0x7ff3e000a5c0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 shutdown_connections 2024-01-15T17:55:11.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3f015f9c0 0x7ff3f01f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff3c806b2c0 0x7ff3c806d780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3f013e360 0x7ff3f01f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 --2- 172.21.15.26:0/795863190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3f013df80 0x7ff3f013b250 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 >> 172.21.15.26:0/795863190 conn(0x7ff3f006a800 msgr2=0x7ff3f006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 shutdown_connections 2024-01-15T17:55:11.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.203+0000 7ff3f6000640 1 -- 172.21.15.26:0/795863190 wait complete. 2024-01-15T17:55:11.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-15T17:55:11.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-15T17:55:11.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-15T17:55:11.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-15T17:55:11.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-15T17:55:11.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-15T17:55:11.229 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-15T17:55:11.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-15T17:55:11.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.530+0000 7f9d97400640 1 Processor -- start 2024-01-15T17:55:11.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.530+0000 7f9d97400640 1 -- start start 2024-01-15T17:55:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.532+0000 7f9d97400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9d900cf9a0 0x7f9d900cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.532+0000 7f9d97400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9d900b31d0 0x7f9d900b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.533+0000 7f9d97400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900d0320 con 0x7f9d9015d4b0 2024-01-15T17:55:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.533+0000 7f9d97400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900d1c60 con 0x7f9d900cf9a0 2024-01-15T17:55:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.533+0000 7f9d97400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900b2730 con 0x7f9d900b31d0 2024-01-15T17:55:11.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.533+0000 7f9d96000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9d9015d4b0 0x7f9d900d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58722/0 (socket says 172.21.15.26:58722) 2024-01-15T17:55:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.533+0000 7f9d96000640 1 -- 172.21.15.26:0/2637986351 learned_addr learned my addr 172.21.15.26:0/2637986351 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.533+0000 7f9d96a00640 1 --2- 172.21.15.26:0/2637986351 >> v2:172.21.15.124:3300/0 conn(0x7f9d900b31d0 0x7f9d900b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.534+0000 7f9d95600640 1 --2- 172.21.15.26:0/2637986351 >> v2:172.21.15.26:3300/0 conn(0x7f9d900cf9a0 0x7f9d900cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.534+0000 7f9d96a00640 1 -- 172.21.15.26:0/2637986351 >> v2:172.21.15.26:3300/0 conn(0x7f9d900cf9a0 msgr2=0x7f9d900cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.534+0000 7f9d96a00640 1 --2- 172.21.15.26:0/2637986351 >> v2:172.21.15.26:3300/0 conn(0x7f9d900cf9a0 0x7f9d900cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.534+0000 7f9d96a00640 1 -- 172.21.15.26:0/2637986351 >> v1:172.21.15.26:6789/0 conn(0x7f9d9015d4b0 legacy=0x7f9d900d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.534+0000 7f9d96a00640 1 -- 172.21.15.26:0/2637986351 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900d1ec0 con 0x7f9d900b31d0 2024-01-15T17:55:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.534+0000 7f9d94c00640 1 -- 172.21.15.26:0/2637986351 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477490100 0 0) 0x7f9d900d0320 con 0x7f9d9015d4b0 2024-01-15T17:55:11.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.535+0000 7f9d95600640 1 --2- 172.21.15.26:0/2637986351 >> v2:172.21.15.26:3300/0 conn(0x7f9d900cf9a0 0x7f9d900cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:11.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.535+0000 7f9d96a00640 1 --2- 172.21.15.26:0/2637986351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d900b31d0 0x7f9d900b3660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d90072a30 tx=0x7f9d8802bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b93f2200dabb15f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.536+0000 7f9d94c00640 1 -- 172.21.15.26:0/2637986351 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8802cc90 con 0x7f9d900b31d0 2024-01-15T17:55:11.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.536+0000 7f9d94c00640 1 -- 172.21.15.26:0/2637986351 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8802cdf0 con 0x7f9d900b31d0 2024-01-15T17:55:11.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.537+0000 7f9d94c00640 1 -- 172.21.15.26:0/2637986351 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d88034660 con 0x7f9d900b31d0 2024-01-15T17:55:11.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.538+0000 7f9d97400640 1 -- 172.21.15.26:0/2637986351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d900b31d0 msgr2=0x7f9d900b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.538+0000 7f9d97400640 1 --2- 172.21.15.26:0/2637986351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d900b31d0 0x7f9d900b3660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d90072a30 tx=0x7f9d8802bee0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.538+0000 7f9d97400640 1 -- 172.21.15.26:0/2637986351 shutdown_connections 2024-01-15T17:55:11.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.538+0000 7f9d97400640 1 --2- 172.21.15.26:0/2637986351 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d900b31d0 0x7f9d900b3660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.538+0000 7f9d97400640 1 --2- 172.21.15.26:0/2637986351 >> v2:172.21.15.26:3300/0 conn(0x7f9d900cf9a0 0x7f9d900cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.538+0000 7f9d97400640 1 -- 172.21.15.26:0/2637986351 >> 172.21.15.26:0/2637986351 conn(0x7f9d900d3000 msgr2=0x7f9d900d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.539+0000 7f9d97400640 1 -- 172.21.15.26:0/2637986351 shutdown_connections 2024-01-15T17:55:11.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.539+0000 7f9d97400640 1 -- 172.21.15.26:0/2637986351 wait complete. 2024-01-15T17:55:11.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.541+0000 7f9d97400640 1 Processor -- start 2024-01-15T17:55:11.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.542+0000 7fb154600640 1 Processor -- start 2024-01-15T17:55:11.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.543+0000 7f9d97400640 1 -- start start 2024-01-15T17:55:11.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.542+0000 7fb154600640 1 -- start start 2024-01-15T17:55:11.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.544+0000 7fb154600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb14c15d560 0x7fb14c0d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.544+0000 7f9d97400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d900b31d0 0x7f9d901ede40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.544+0000 7f9d97400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d900cf9a0 0x7f9d900b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.544+0000 7fb154600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c0d1970 con 0x7fb14c0b22b0 2024-01-15T17:55:11.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.544+0000 7fb154600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c0d1c50 con 0x7fb14c0d0140 2024-01-15T17:55:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.544+0000 7fb154600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14c0d1f30 con 0x7fb14c15d560 2024-01-15T17:55:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7fb151200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb14c0b22b0 0x7fb14c0cfa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58736/0 (socket says 172.21.15.26:58736) 2024-01-15T17:55:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7f9d97400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9015d4b0 0x7f9d900b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7f9d97400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9009e0b0 con 0x7f9d9015d4b0 2024-01-15T17:55:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7f9d97400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900b6160 con 0x7f9d900cf9a0 2024-01-15T17:55:11.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7f9d97400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900a05f0 con 0x7f9d900b31d0 2024-01-15T17:55:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7fb151200640 1 -- 172.21.15.26:0/3684202598 learned_addr learned my addr 172.21.15.26:0/3684202598 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7fb151c00640 1 --2- 172.21.15.26:0/3684202598 >> v2:172.21.15.26:3300/0 conn(0x7fb14c15d560 0x7fb14c0d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7f834cc00640 1 Processor -- start 2024-01-15T17:55:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7f9d96000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d900b31d0 0x7f9d901ede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.546+0000 7f9d96a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9015d4b0 0x7f9d900b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.546+0000 7f9d96a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9015d4b0 0x7f9d900b7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57626/0 (socket says 172.21.15.26:57626) 2024-01-15T17:55:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.546+0000 7f9d96a00640 1 -- 172.21.15.26:0/771752336 learned_addr learned my addr 172.21.15.26:0/771752336 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.546+0000 7f9d96a00640 1 -- 172.21.15.26:0/771752336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d900b31d0 msgr2=0x7f9d901ede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.546+0000 7f9d96a00640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d900b31d0 0x7f9d901ede40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.547+0000 7f9d96a00640 1 -- 172.21.15.26:0/771752336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d900cf9a0 msgr2=0x7f9d900b5c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T17:55:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.547+0000 7f9d96a00640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d900cf9a0 0x7f9d900b5c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.547+0000 7f9d96a00640 1 -- 172.21.15.26:0/771752336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009a70 con 0x7f9d9015d4b0 2024-01-15T17:55:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.548+0000 7f9d96a00640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9015d4b0 0x7f9d900b7ee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009e80 tx=0x7f9d88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.546+0000 7fb151c00640 1 -- 172.21.15.26:0/3684202598 >> v1:172.21.15.124:6789/0 conn(0x7fb14c0d0140 legacy=0x7fb14c0b2d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T17:55:11.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.546+0000 7fb14be00640 1 -- 172.21.15.26:0/3684202598 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142818885 0 0) 0x7fb14c0d1970 con 0x7fb14c0b22b0 2024-01-15T17:55:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f970ec00640 1 Processor -- start 2024-01-15T17:55:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f970ec00640 1 -- start start 2024-01-15T17:55:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f9d8e000640 1 -- 172.21.15.26:0/771752336 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8803e610 con 0x7f9d9015d4b0 2024-01-15T17:55:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f9d8e000640 1 -- 172.21.15.26:0/771752336 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8803e770 con 0x7f9d9015d4b0 2024-01-15T17:55:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f9d8e000640 1 -- 172.21.15.26:0/771752336 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8803eae0 con 0x7f9d9015d4b0 2024-01-15T17:55:11.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.550+0000 7f9d97400640 1 -- 172.21.15.26:0/771752336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d900b8420 con 0x7f9d9015d4b0 2024-01-15T17:55:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.550+0000 7f9d97400640 1 -- 172.21.15.26:0/771752336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9009da90 con 0x7f9d9015d4b0 2024-01-15T17:55:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.545+0000 7f834cc00640 1 -- start start 2024-01-15T17:55:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.548+0000 7f834cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 0x7f8348070c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.548+0000 7f834cc00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f83481625e0 0x7f834806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.548+0000 7f834cc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8348161b40 con 0x7f83480b2f10 2024-01-15T17:55:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.548+0000 7f834cc00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834815f5c0 con 0x7f83481625e0 2024-01-15T17:55:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.548+0000 7f834cc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834806dc50 con 0x7f834815f1e0 2024-01-15T17:55:11.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f8347400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 0x7f8348070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f8347400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 0x7f8348070c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:39582/0 (socket says 172.21.15.26:39582) 2024-01-15T17:55:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f8347400640 1 -- 172.21.15.26:0/1312802022 learned_addr learned my addr 172.21.15.26:0/1312802022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f8347e00640 1 --1- 172.21.15.26:0/1312802022 >> v1:172.21.15.26:6789/0 conn(0x7f83480b2f10 0x7f8348165110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58744/0 (socket says 172.21.15.26:58744) 2024-01-15T17:55:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7f8346a00640 1 --2- 172.21.15.26:0/1312802022 >> v2:172.21.15.124:3300/0 conn(0x7f83481625e0 0x7f834806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.550+0000 7f8346000640 1 -- 172.21.15.26:0/1312802022 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748048880 0 0) 0x7f8348161b40 con 0x7f83480b2f10 2024-01-15T17:55:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.550+0000 7f8346000640 1 -- 172.21.15.26:0/1312802022 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8328003520 con 0x7f83480b2f10 2024-01-15T17:55:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7fb151c00640 1 -- 172.21.15.26:0/3684202598 >> v1:172.21.15.26:6789/0 conn(0x7fb14c0b22b0 legacy=0x7fb14c0cfa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.549+0000 7fb151c00640 1 -- 172.21.15.26:0/3684202598 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c0d2210 con 0x7fb14c15d560 2024-01-15T17:55:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f970ec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 0x7f9708161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f970ec00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97080cf1b0 con 0x7f97080ced50 2024-01-15T17:55:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f970ec00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97081618a0 con 0x7f97080ce8f0 2024-01-15T17:55:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f970ec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97080d0a00 con 0x7f9708160f80 2024-01-15T17:55:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f8347400640 1 -- 172.21.15.26:0/1312802022 >> v2:172.21.15.124:3300/0 conn(0x7f83481625e0 msgr2=0x7f834806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f8347400640 1 --2- 172.21.15.26:0/1312802022 >> v2:172.21.15.124:3300/0 conn(0x7f83481625e0 0x7f834806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f8347400640 1 -- 172.21.15.26:0/1312802022 >> v1:172.21.15.26:6789/0 conn(0x7f83480b2f10 legacy=0x7f8348165110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f8347400640 1 -- 172.21.15.26:0/1312802022 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834806df00 con 0x7f834815f1e0 2024-01-15T17:55:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f970cc00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f97080ced50 0x7f97080d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58756/0 (socket says 172.21.15.26:58756) 2024-01-15T17:55:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.551+0000 7f970cc00640 1 -- 172.21.15.26:0/1421039053 learned_addr learned my addr 172.21.15.26:0/1421039053 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:11.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.552+0000 7fb151c00640 1 --2- 172.21.15.26:0/3684202598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c15d560 0x7fb14c0d0f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0097a0 tx=0x7fb13c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75158d31dfa6f38 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.552+0000 7f9707400640 1 -- 172.21.15.26:0/1421039053 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083261019 0 0) 0x7f97080cf1b0 con 0x7f97080ced50 2024-01-15T17:55:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.552+0000 7f9707400640 1 -- 172.21.15.26:0/1421039053 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96e8003520 con 0x7f97080ced50 2024-01-15T17:55:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7f9707e00640 1 --2- 172.21.15.26:0/1421039053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 0x7f9708161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7f8347400640 1 --2- 172.21.15.26:0/1312802022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 0x7f8348070c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8340009e50 tx=0x7f834002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aee790990df98fe server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7f9d87400640 1 -- 172.21.15.26:0/771752336 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d900d21a0 con 0x7f9d9015d4b0 2024-01-15T17:55:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7f8346000640 1 -- 172.21.15.26:0/1312802022 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834002ca80 con 0x7f834815f1e0 2024-01-15T17:55:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7f8346000640 1 -- 172.21.15.26:0/1312802022 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834002cbe0 con 0x7f834815f1e0 2024-01-15T17:55:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7f8346000640 1 -- 172.21.15.26:0/1312802022 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400343f0 con 0x7f834815f1e0 2024-01-15T17:55:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7f9707e00640 1 -- 172.21.15.26:0/1421039053 >> v1:172.21.15.124:6789/0 conn(0x7f97080ce8f0 legacy=0x7f97080cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.554+0000 7f9707400640 1 -- 172.21.15.26:0/1421039053 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 574258147 0 0) 0x7f96e8003520 con 0x7f97080ced50 2024-01-15T17:55:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.554+0000 7f834cc00640 1 -- 172.21.15.26:0/1312802022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 msgr2=0x7f8348070c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.554+0000 7f9d8e000640 1 -- 172.21.15.26:0/771752336 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9d8803b050 con 0x7f9d9015d4b0 2024-01-15T17:55:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.552+0000 7fb14be00640 1 -- 172.21.15.26:0/3684202598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c031640 con 0x7fb14c15d560 2024-01-15T17:55:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7fb14be00640 1 -- 172.21.15.26:0/3684202598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c0317a0 con 0x7fb14c15d560 2024-01-15T17:55:11.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7fb14be00640 1 -- 172.21.15.26:0/3684202598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c031b10 con 0x7fb14c15d560 2024-01-15T17:55:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7fb154600640 1 -- 172.21.15.26:0/3684202598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c15d560 msgr2=0x7fb14c0d0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.553+0000 7fb154600640 1 --2- 172.21.15.26:0/3684202598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c15d560 0x7fb14c0d0f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0097a0 tx=0x7fb13c02b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.554+0000 7fb154600640 1 -- 172.21.15.26:0/3684202598 shutdown_connections 2024-01-15T17:55:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.554+0000 7fb154600640 1 --2- 172.21.15.26:0/3684202598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c15d560 0x7fb14c0d0f20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.554+0000 7fb154600640 1 -- 172.21.15.26:0/3684202598 >> 172.21.15.26:0/3684202598 conn(0x7fb14c0d3020 msgr2=0x7fb14c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.554+0000 7fb154600640 1 -- 172.21.15.26:0/3684202598 shutdown_connections 2024-01-15T17:55:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.554+0000 7fb154600640 1 -- 172.21.15.26:0/3684202598 wait complete. 2024-01-15T17:55:11.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.555+0000 7f834cc00640 1 --2- 172.21.15.26:0/1312802022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 0x7f8348070c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8340009e50 tx=0x7f834002bee0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.555+0000 7f834cc00640 1 -- 172.21.15.26:0/1312802022 shutdown_connections 2024-01-15T17:55:11.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.555+0000 7f834cc00640 1 --2- 172.21.15.26:0/1312802022 >> v2:172.21.15.124:3300/0 conn(0x7f83481625e0 0x7f834806d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.555+0000 7f834cc00640 1 --2- 172.21.15.26:0/1312802022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 0x7f8348070c50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.555+0000 7f834cc00640 1 -- 172.21.15.26:0/1312802022 >> 172.21.15.26:0/1312802022 conn(0x7f83480b0b00 msgr2=0x7f83480b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.555+0000 7f9707e00640 1 -- 172.21.15.26:0/1421039053 >> v1:172.21.15.26:6789/0 conn(0x7f97080ced50 legacy=0x7f97080d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.555+0000 7f9707e00640 1 -- 172.21.15.26:0/1421039053 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97080d0de0 con 0x7f9708160f80 2024-01-15T17:55:11.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.555+0000 7f9d8e000640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d6806b310 0x7f9d6806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.555+0000 7f9d8e000640 1 -- 172.21.15.26:0/771752336 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d880ac520 con 0x7f9d9015d4b0 2024-01-15T17:55:11.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.556+0000 7f834cc00640 1 -- 172.21.15.26:0/1312802022 shutdown_connections 2024-01-15T17:55:11.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.556+0000 7f9d8e000640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7f9d68073530 0x7f9d68075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.556+0000 7f9d8e000640 1 -- 172.21.15.26:0/771752336 --> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d68076000 con 0x7f9d68073530 2024-01-15T17:55:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.556+0000 7f9d8e000640 1 -- 172.21.15.26:0/771752336 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d880aeb00 con 0x7f9d9015d4b0 2024-01-15T17:55:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.556+0000 7f9d96000640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d6806b310 0x7f9d6806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.556+0000 7f834cc00640 1 -- 172.21.15.26:0/1312802022 wait complete. 2024-01-15T17:55:11.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.557+0000 7f9d95600640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7f9d68073530 0x7f9d68075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.558+0000 7f9707e00640 1 --2- 172.21.15.26:0/1421039053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 0x7f9708161360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96f80097d0 tx=0x7f96f802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1dcb7929bd63bc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.558+0000 7fb154600640 1 Processor -- start 2024-01-15T17:55:11.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.558+0000 7f834cc00640 1 Processor -- start 2024-01-15T17:55:11.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.558+0000 7f9d95600640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7f9d68073530 0x7f9d68075950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.558+0000 7f9707400640 1 -- 172.21.15.26:0/1421039053 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97000019f0 con 0x7f9708160f80 2024-01-15T17:55:11.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.558+0000 7f9707400640 1 -- 172.21.15.26:0/1421039053 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f802e050 con 0x7f9708160f80 2024-01-15T17:55:11.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.558+0000 7f9707400640 1 -- 172.21.15.26:0/1421039053 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8030730 con 0x7f9708160f80 2024-01-15T17:55:11.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.559+0000 7f9d96000640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d6806b310 0x7f9d6806d7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000f60 tx=0x7f9d78009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.559+0000 7f834cc00640 1 -- start start 2024-01-15T17:55:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.559+0000 7fd186e00640 1 Processor -- start 2024-01-15T17:55:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.558+0000 7fb154600640 1 -- start start 2024-01-15T17:55:11.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7f970ec00640 1 -- 172.21.15.26:0/1421039053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 msgr2=0x7f9708161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7f970ec00640 1 --2- 172.21.15.26:0/1421039053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 0x7f9708161360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96f80097d0 tx=0x7f96f802b800 comp rx=0 tx=0).stop 2024-01-15T17:55:11.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7fb154600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb14c0b22b0 0x7fb14c0c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.559+0000 7fd186e00640 1 -- start start 2024-01-15T17:55:11.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7fb154600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c0d0140 0x7fb14c0c6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7f9d8e000640 1 -- 172.21.15.26:0/771752336 <== osd.0 v2:172.21.15.26:6800/3842439308 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9d68076000 con 0x7f9d68073530 2024-01-15T17:55:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7fb154600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb14c15d560 0x7fb14c0c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7f834cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83480b2f10 0x7f8348165320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7f834cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 0x7f83480c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7f834cc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83481625e0 0x7f83480c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7f834cc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8348165860 con 0x7f83480b2f10 2024-01-15T17:55:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7f834cc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83480c4c70 con 0x7f83481625e0 2024-01-15T17:55:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.560+0000 7f834cc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8348162d30 con 0x7f834815f1e0 2024-01-15T17:55:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f970ec00640 1 -- 172.21.15.26:0/1421039053 shutdown_connections 2024-01-15T17:55:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f970ec00640 1 --2- 172.21.15.26:0/1421039053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 0x7f9708161360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f970ec00640 1 -- 172.21.15.26:0/1421039053 >> 172.21.15.26:0/1421039053 conn(0x7f970809c550 msgr2=0x7f97080d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f8347e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83481625e0 0x7f83480c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7fb154600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0c75d0 con 0x7fb14c0d0140 2024-01-15T17:55:11.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f970ec00640 1 -- 172.21.15.26:0/1421039053 shutdown_connections 2024-01-15T17:55:11.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f8347400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83480b2f10 0x7f8348165320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f8347400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83480b2f10 0x7f8348165320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57638/0 (socket says 172.21.15.26:57638) 2024-01-15T17:55:11.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f8347400640 1 -- 172.21.15.26:0/716070652 learned_addr learned my addr 172.21.15.26:0/716070652 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f8346a00640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 0x7f83480c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7f970ec00640 1 -- 172.21.15.26:0/1421039053 wait complete. 2024-01-15T17:55:11.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7fb151200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c0d0140 0x7fb14c0c6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7fcf01800640 1 Processor -- start 2024-01-15T17:55:11.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7fb154600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0c77f0 con 0x7fb14c0b22b0 2024-01-15T17:55:11.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7fb154600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0c7ad0 con 0x7fb14c15d560 2024-01-15T17:55:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7f8347400640 1 -- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 msgr2=0x7f83480c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7fcf01800640 1 -- start start 2024-01-15T17:55:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7fb151200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c0d0140 0x7fb14c0c6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57654/0 (socket says 172.21.15.26:57654) 2024-01-15T17:55:11.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7fda3fa00640 1 Processor -- start 2024-01-15T17:55:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7fb151200640 1 -- 172.21.15.26:0/2155823034 learned_addr learned my addr 172.21.15.26:0/2155823034 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7f8347400640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834815f1e0 0x7f83480c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.561+0000 7fb151c00640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb14c0b22b0 0x7fb14c0c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7fb151200640 1 -- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb14c15d560 msgr2=0x7fb14c0c7090 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-15T17:55:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7fb151200640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb14c15d560 0x7fb14c0c7090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7fb151200640 1 -- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb14c0b22b0 msgr2=0x7fb14c0c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7fb151200640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb14c0b22b0 0x7fb14c0c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7fb151200640 1 -- 172.21.15.26:0/2155823034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13c009400 con 0x7fb14c0d0140 2024-01-15T17:55:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.563+0000 7fb151c00640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb14c0b22b0 0x7fb14c0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:55:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7f8347400640 1 -- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83481625e0 msgr2=0x7f83480c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7f8347400640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83481625e0 0x7f83480c5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7f8347400640 1 -- 172.21.15.26:0/716070652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8340009b30 con 0x7f83480b2f10 2024-01-15T17:55:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.562+0000 7fda3fa00640 1 -- start start 2024-01-15T17:55:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.563+0000 7fb151200640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c0d0140 0x7fb14c0c6b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c4a0 tx=0x7fb13400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.563+0000 7fb149600640 1 -- 172.21.15.26:0/2155823034 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134005ce0 con 0x7fb14c0d0140 2024-01-15T17:55:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.563+0000 7f8347400640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83480b2f10 0x7f8348165320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8340036040 tx=0x7f8340004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.563+0000 7fb149600640 1 -- 172.21.15.26:0/2155823034 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134005e40 con 0x7fb14c0d0140 2024-01-15T17:55:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.563+0000 7fb149600640 1 -- 172.21.15.26:0/2155823034 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1340072b0 con 0x7fb14c0d0140 2024-01-15T17:55:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.563+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c15dac0 con 0x7fb14c0d0140 2024-01-15T17:55:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c0c8090 con 0x7fb14c0d0140 2024-01-15T17:55:11.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7f970ec00640 1 Processor -- start 2024-01-15T17:55:11.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7f833f400640 1 -- 172.21.15.26:0/716070652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340007e10 con 0x7f83480b2f10 2024-01-15T17:55:11.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7f970ec00640 1 -- start start 2024-01-15T17:55:11.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7fcf01800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcefc14b2c0 0x7fcefc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7fcf01800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc0cfbf0 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7fcf01800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc071690 con 0x7fcefc071070 2024-01-15T17:55:11.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7fcf01800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcefc14b6a0 con 0x7fcefc14b2c0 2024-01-15T17:55:11.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8348163010 con 0x7f83480b2f10 2024-01-15T17:55:11.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7fd186e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd180160f80 0x7fd180161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.564+0000 7fd186e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1800ce8f0 0x7fd1800ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fd186e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd1800cfbe0 0x7fd1800d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fd186e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1801618a0 con 0x7fd1800ce8f0 2024-01-15T17:55:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fd186e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1800d0de0 con 0x7fd1800cfbe0 2024-01-15T17:55:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fd186e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800d10c0 con 0x7fd180160f80 2024-01-15T17:55:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fda3fa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda3814b2c0 0x7fda380cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fda3fa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fda380cf7d0 0x7fda38070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fda3fa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda380cfbf0 con 0x7fda38070ea0 2024-01-15T17:55:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fda3fa00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3814b6a0 con 0x7fda380cf7d0 2024-01-15T17:55:11.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fda3fa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda38071690 con 0x7fda3814b2c0 2024-01-15T17:55:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83481635d0 con 0x7f83480b2f10 2024-01-15T17:55:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7f833f400640 1 -- 172.21.15.26:0/716070652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834003e660 con 0x7f83480b2f10 2024-01-15T17:55:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7f833f400640 1 -- 172.21.15.26:0/716070652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834003ea10 con 0x7f83480b2f10 2024-01-15T17:55:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7f970ec00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080ce8f0 0x7f97080c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7f970ec00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97080ced50 0x7f97081f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7f970ec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 0x7f97080c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7f970ec00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97081f7820 con 0x7f97080ced50 2024-01-15T17:55:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7f970ec00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97080cad90 con 0x7f97080ce8f0 2024-01-15T17:55:11.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7f970ec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97080cb000 con 0x7f9708160f80 2024-01-15T17:55:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.565+0000 7fcefaa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcefc14b2c0 0x7fcefc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fcefaa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcefc14b2c0 0x7fcefc0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58834/0 (socket says 172.21.15.26:58834) 2024-01-15T17:55:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fcefaa00640 1 -- 172.21.15.26:0/2234472544 learned_addr learned my addr 172.21.15.26:0/2234472544 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fda3d000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda3814b2c0 0x7fda380cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fda3d000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda3814b2c0 0x7fda380cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:39622/0 (socket says 172.21.15.26:39622) 2024-01-15T17:55:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fda3d000640 1 -- 172.21.15.26:0/2663078516 learned_addr learned my addr 172.21.15.26:0/2663078516 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fda37e00640 1 --2- 172.21.15.26:0/2663078516 >> v2:172.21.15.26:3300/0 conn(0x7fda380cf7d0 0x7fda38070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7f9707e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080ce8f0 0x7f97080c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7f9707e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080ce8f0 0x7f97080c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58848/0 (socket says 172.21.15.26:58848) 2024-01-15T17:55:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7f9707e00640 1 -- 172.21.15.26:0/439273470 learned_addr learned my addr 172.21.15.26:0/439273470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fd184e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd1800cfbe0 0x7fd1800d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fd184e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd1800cfbe0 0x7fd1800d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58820/0 (socket says 172.21.15.26:58820) 2024-01-15T17:55:11.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fd184e00640 1 -- 172.21.15.26:0/977416136 learned_addr learned my addr 172.21.15.26:0/977416136 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.566+0000 7fd17f400640 1 --2- 172.21.15.26:0/977416136 >> v2:172.21.15.26:3300/0 conn(0x7fd1800ce8f0 0x7fd1800ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.567+0000 7f970cc00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 0x7f97080c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.567+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb14c15d940 con 0x7fb14c0d0140 2024-01-15T17:55:11.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83481620c0 con 0x7f83480b2f10 2024-01-15T17:55:11.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.567+0000 7fda3d000640 1 -- 172.21.15.26:0/2663078516 >> v2:172.21.15.26:3300/0 conn(0x7fda380cf7d0 msgr2=0x7fda38070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.567+0000 7fda3d000640 1 --2- 172.21.15.26:0/2663078516 >> v2:172.21.15.26:3300/0 conn(0x7fda380cf7d0 0x7fda38070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.567+0000 7fda3d000640 1 -- 172.21.15.26:0/2663078516 >> v1:172.21.15.26:6789/0 conn(0x7fda38070ea0 legacy=0x7fda380b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7f9707e00640 1 -- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 msgr2=0x7f97080c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7f9707e00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 0x7f97080c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7f9707e00640 1 -- 172.21.15.26:0/439273470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97080ced50 msgr2=0x7f97081f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7f9707e00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97080ced50 0x7f97081f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7f9707e00640 1 -- 172.21.15.26:0/439273470 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8009400 con 0x7f97080ce8f0 2024-01-15T17:55:11.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.567+0000 7fda3d000640 1 -- 172.21.15.26:0/2663078516 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda38071970 con 0x7fda3814b2c0 2024-01-15T17:55:11.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.567+0000 7fd17fe00640 1 --2- 172.21.15.26:0/977416136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd180160f80 0x7fd180161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7f970cc00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9708160f80 0x7f97080c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:11.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7fcef9600640 1 -- 172.21.15.26:0/2234472544 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417064935 0 0) 0x7fcefc071690 con 0x7fcefc071070 2024-01-15T17:55:11.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7fcef9600640 1 -- 172.21.15.26:0/2234472544 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcedc003520 con 0x7fcefc071070 2024-01-15T17:55:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.569+0000 7fcefaa00640 1 -- 172.21.15.26:0/2234472544 >> v1:172.21.15.124:6789/0 conn(0x7fcefc071070 legacy=0x7fcefc0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.569+0000 7fcefaa00640 1 -- 172.21.15.26:0/2234472544 >> v1:172.21.15.26:6789/0 conn(0x7fcefc0cf7d0 legacy=0x7fcefc070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.569+0000 7fcefaa00640 1 -- 172.21.15.26:0/2234472544 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc071970 con 0x7fcefc14b2c0 2024-01-15T17:55:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7fb149600640 1 -- 172.21.15.26:0/2155823034 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb134007410 con 0x7fb14c0d0140 2024-01-15T17:55:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.569+0000 7f9707e00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080ce8f0 0x7f97080c60b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96f8000c00 tx=0x7f96f80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.569+0000 7fda3d000640 1 --2- 172.21.15.26:0/2663078516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda3814b2c0 0x7fda380cf290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda28009870 tx=0x7fda2802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f0db36dfcc88fe5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.569+0000 7fa200a00640 1 Processor -- start 2024-01-15T17:55:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7fd17f400640 1 -- 172.21.15.26:0/977416136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd180160f80 msgr2=0x7fd180161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7fd17f400640 1 --2- 172.21.15.26:0/977416136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd180160f80 0x7fd180161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7fd17f400640 1 -- 172.21.15.26:0/977416136 >> v2:172.21.15.124:3300/0 conn(0x7fd1800cfbe0 msgr2=0x7fd1800d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7fd17f400640 1 --2- 172.21.15.26:0/977416136 >> v2:172.21.15.124:3300/0 conn(0x7fd1800cfbe0 0x7fd1800d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.568+0000 7fd17f400640 1 -- 172.21.15.26:0/977416136 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1800d13a0 con 0x7fd1800ce8f0 2024-01-15T17:55:11.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.569+0000 7fa200a00640 1 -- start start 2024-01-15T17:55:11.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.570+0000 7f9704c00640 1 -- 172.21.15.26:0/439273470 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80328d0 con 0x7f97080ce8f0 2024-01-15T17:55:11.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.570+0000 7f9704c00640 1 -- 172.21.15.26:0/439273470 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8032a70 con 0x7f97080ce8f0 2024-01-15T17:55:11.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.570+0000 7f9704c00640 1 -- 172.21.15.26:0/439273470 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8032e20 con 0x7f97080ce8f0 2024-01-15T17:55:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.570+0000 7fda37400640 1 -- 172.21.15.26:0/2663078516 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28031640 con 0x7fda3814b2c0 2024-01-15T17:55:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.570+0000 7fb149600640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb12406b390 0x7fb12406d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fcefaa00640 1 --2- 172.21.15.26:0/2234472544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcefc14b2c0 0x7fcefc0cf290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009870 tx=0x7fcee402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d93f68d43f21a95 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.570+0000 7fd17f400640 1 --2- 172.21.15.26:0/977416136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800ce8f0 0x7fd1800ced50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1800c6a40 tx=0x7fd17002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3b926b0f25c5a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fda37400640 1 -- 172.21.15.26:0/2663078516 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda280317a0 con 0x7fda3814b2c0 2024-01-15T17:55:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fda37400640 1 -- 172.21.15.26:0/2663078516 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28031b30 con 0x7fda3814b2c0 2024-01-15T17:55:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fd17ea00640 1 -- 172.21.15.26:0/977416136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17002e070 con 0x7fd1800ce8f0 2024-01-15T17:55:11.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fb149600640 1 -- 172.21.15.26:0/2155823034 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb134089170 con 0x7fb14c0d0140 2024-01-15T17:55:11.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fb149600640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7fb1240735b0 0x7fb1240759d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fb149600640 1 -- 172.21.15.26:0/2155823034 --> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb124076080 con 0x7fb1240735b0 2024-01-15T17:55:11.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fb149600640 1 -- 172.21.15.26:0/2155823034 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb13408b790 con 0x7fb14c0d0140 2024-01-15T17:55:11.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fb151c00640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb12406b390 0x7fb12406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fb152600640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7fb1240735b0 0x7fb1240759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fd17ea00640 1 -- 172.21.15.26:0/977416136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1700315a0 con 0x7fd1800ce8f0 2024-01-15T17:55:11.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fd17ea00640 1 -- 172.21.15.26:0/977416136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170031930 con 0x7fd1800ce8f0 2024-01-15T17:55:11.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fcef9600640 1 -- 172.21.15.26:0/2234472544 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4030860 con 0x7fcefc14b2c0 2024-01-15T17:55:11.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fcef9600640 1 -- 172.21.15.26:0/2234472544 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee40309c0 con 0x7fcefc14b2c0 2024-01-15T17:55:11.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7fcef9600640 1 -- 172.21.15.26:0/2234472544 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee4030d50 con 0x7fcefc14b2c0 2024-01-15T17:55:11.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fda3fa00640 1 -- 172.21.15.26:0/2663078516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda3814b2c0 msgr2=0x7fda380cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fda3fa00640 1 --2- 172.21.15.26:0/2663078516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda3814b2c0 0x7fda380cf290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda28009870 tx=0x7fda2802b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.571+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97080cb2e0 con 0x7f97080ce8f0 2024-01-15T17:55:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7f833f400640 1 -- 172.21.15.26:0/716070652 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f834002cd50 con 0x7f83480b2f10 2024-01-15T17:55:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97080cb8a0 con 0x7f97080ce8f0 2024-01-15T17:55:11.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fda3fa00640 1 -- 172.21.15.26:0/2663078516 shutdown_connections 2024-01-15T17:55:11.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fda3fa00640 1 --2- 172.21.15.26:0/2663078516 >> v2:172.21.15.26:3300/0 conn(0x7fda380cf7d0 0x7fda38070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fda3fa00640 1 --2- 172.21.15.26:0/2663078516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda3814b2c0 0x7fda380cf290 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fda3fa00640 1 -- 172.21.15.26:0/2663078516 >> 172.21.15.26:0/2663078516 conn(0x7fda3806a800 msgr2=0x7fda3806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fa200a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fa200a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa1fc06c7c0 0x7fa1fc0717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fa200a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc06cc20 con 0x7fa1fc071d30 2024-01-15T17:55:11.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fa200a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc06cec0 con 0x7fa1fc06c7c0 2024-01-15T17:55:11.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fa200a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc06d1a0 con 0x7fa1fc071410 2024-01-15T17:55:11.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fa1fb200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa1fc071d30 0x7fa1fc0b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58772/0 (socket says 172.21.15.26:58772) 2024-01-15T17:55:11.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.572+0000 7fa1fb200640 1 -- 172.21.15.26:0/1858511911 learned_addr learned my addr 172.21.15.26:0/1858511911 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:11.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fa1fa800640 1 --2- 172.21.15.26:0/1858511911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fd186e00640 1 -- 172.21.15.26:0/977416136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800ce8f0 msgr2=0x7fd1800ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fd186e00640 1 --2- 172.21.15.26:0/977416136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800ce8f0 0x7fd1800ced50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1800c6a40 tx=0x7fd17002b840 comp rx=0 tx=0).stop 2024-01-15T17:55:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fd186e00640 1 -- 172.21.15.26:0/977416136 shutdown_connections 2024-01-15T17:55:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fd186e00640 1 --2- 172.21.15.26:0/977416136 >> v2:172.21.15.124:3300/0 conn(0x7fd1800cfbe0 0x7fd1800d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f970808d2e0 con 0x7f97080ce8f0 2024-01-15T17:55:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fd186e00640 1 --2- 172.21.15.26:0/977416136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1800ce8f0 0x7fd1800ced50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fd186e00640 1 --2- 172.21.15.26:0/977416136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd180160f80 0x7fd180161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fd186e00640 1 -- 172.21.15.26:0/977416136 >> 172.21.15.26:0/977416136 conn(0x7fd18009c550 msgr2=0x7fd1800d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fda3fa00640 1 -- 172.21.15.26:0/2663078516 shutdown_connections 2024-01-15T17:55:11.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fda3fa00640 1 -- 172.21.15.26:0/2663078516 wait complete. 2024-01-15T17:55:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fa1f9400640 1 -- 172.21.15.26:0/1858511911 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611067932 0 0) 0x7fa1fc06cc20 con 0x7fa1fc071d30 2024-01-15T17:55:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fa1fa800640 1 -- 172.21.15.26:0/1858511911 >> v2:172.21.15.124:3300/0 conn(0x7fa1fc06c7c0 msgr2=0x7fa1fc0717f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T17:55:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fa1fa800640 1 --2- 172.21.15.26:0/1858511911 >> v2:172.21.15.124:3300/0 conn(0x7fa1fc06c7c0 0x7fa1fc0717f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fa1fa800640 1 -- 172.21.15.26:0/1858511911 >> v1:172.21.15.26:6789/0 conn(0x7fa1fc071d30 legacy=0x7fa1fc0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fa1fa800640 1 -- 172.21.15.26:0/1858511911 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc06d480 con 0x7fa1fc071410 2024-01-15T17:55:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fcf01800640 1 -- 172.21.15.26:0/2234472544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcefc14b2c0 msgr2=0x7fcefc0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fcf01800640 1 --2- 172.21.15.26:0/2234472544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcefc14b2c0 0x7fcefc0cf290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009870 tx=0x7fcee402b9a0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7f833f400640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f832006b2c0 0x7f832006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fb152600640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7fb1240735b0 0x7fb1240759d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.573+0000 7fb151c00640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb12406b390 0x7fb12406d850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb13c036000 tx=0x7fb13c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fb149600640 1 -- 172.21.15.26:0/2155823034 <== osd.1 v2:172.21.15.26:6824/639007827 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb124076080 con 0x7fb1240735b0 2024-01-15T17:55:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fd186e00640 1 -- 172.21.15.26:0/977416136 shutdown_connections 2024-01-15T17:55:11.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.574+0000 7fd186e00640 1 -- 172.21.15.26:0/977416136 wait complete. 2024-01-15T17:55:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.575+0000 7f833f400640 1 -- 172.21.15.26:0/716070652 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83400ace70 con 0x7f83480b2f10 2024-01-15T17:55:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.575+0000 7f8346a00640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f832006b2c0 0x7f832006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.575+0000 7f833f400640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f83200734e0 0x7f8320075900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.575+0000 7f833f400640 1 -- 172.21.15.26:0/716070652 --> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8320075fb0 con 0x7f83200734e0 2024-01-15T17:55:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.575+0000 7f833f400640 1 -- 172.21.15.26:0/716070652 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83400ad220 con 0x7f83480b2f10 2024-01-15T17:55:11.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.576+0000 7f8347e00640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f83200734e0 0x7f8320075900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.576+0000 7fa1fa800640 1 --2- 172.21.15.26:0/1858511911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc146770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0097a0 tx=0x7fa1ec02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=728a66c66e9b069d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.576+0000 7fd186e00640 1 Processor -- start 2024-01-15T17:55:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.576+0000 7fda3fa00640 1 Processor -- start 2024-01-15T17:55:11.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.577+0000 7fd186e00640 1 -- start start 2024-01-15T17:55:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.577+0000 7fa1f9400640 1 -- 172.21.15.26:0/1858511911 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec031830 con 0x7fa1fc071410 2024-01-15T17:55:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.577+0000 7fa1f9400640 1 -- 172.21.15.26:0/1858511911 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1ec031990 con 0x7fa1fc071410 2024-01-15T17:55:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.577+0000 7f8346a00640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f832006b2c0 0x7f832006d780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8338006fd0 tx=0x7f8338008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.577+0000 7f8347e00640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f83200734e0 0x7f8320075900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.577+0000 7fda3fa00640 1 -- start start 2024-01-15T17:55:11.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.578+0000 7f9704c00640 1 -- 172.21.15.26:0/439273470 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f96f8039070 con 0x7f97080ce8f0 2024-01-15T17:55:11.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.578+0000 7fa200a00640 1 -- 172.21.15.26:0/1858511911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 msgr2=0x7fa1fc146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.578+0000 7fa200a00640 1 --2- 172.21.15.26:0/1858511911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc146770 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0097a0 tx=0x7fa1ec02b8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.578+0000 7fd186e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1800cfbe0 0x7fd1800b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.578+0000 7fd186e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180160f80 0x7fd1800ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.578+0000 7fd186e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd180150b10 0x7fd1800ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.578+0000 7fd186e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18014d990 con 0x7fd180160f80 2024-01-15T17:55:11.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.578+0000 7fd186e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800cd220 con 0x7fd180150b10 2024-01-15T17:55:11.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.578+0000 7fd186e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1800cc980 con 0x7fd1800cfbe0 2024-01-15T17:55:11.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd17f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180160f80 0x7fd1800ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fcf01800640 1 -- 172.21.15.26:0/2234472544 shutdown_connections 2024-01-15T17:55:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fcf01800640 1 --2- 172.21.15.26:0/2234472544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcefc14b2c0 0x7fcefc0cf290 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fcf01800640 1 -- 172.21.15.26:0/2234472544 >> 172.21.15.26:0/2234472544 conn(0x7fcefc06a800 msgr2=0x7fcefc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd17fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1800cfbe0 0x7fd1800b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd17f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180160f80 0x7fd1800ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57690/0 (socket says 172.21.15.26:57690) 2024-01-15T17:55:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd17f400640 1 -- 172.21.15.26:0/3830878780 learned_addr learned my addr 172.21.15.26:0/3830878780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd184e00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd180150b10 0x7fd1800ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fa200a00640 1 -- 172.21.15.26:0/1858511911 shutdown_connections 2024-01-15T17:55:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fa200a00640 1 --2- 172.21.15.26:0/1858511911 >> v2:172.21.15.124:3300/0 conn(0x7fa1fc06c7c0 0x7fa1fc0717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fa200a00640 1 --2- 172.21.15.26:0/1858511911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc146770 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fa200a00640 1 -- 172.21.15.26:0/1858511911 >> 172.21.15.26:0/1858511911 conn(0x7fa1fc06a800 msgr2=0x7fa1fc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fa200a00640 1 -- 172.21.15.26:0/1858511911 shutdown_connections 2024-01-15T17:55:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7f833f400640 1 -- 172.21.15.26:0/716070652 <== osd.2 v2:172.21.15.26:6816/1591707592 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8320075fb0 con 0x7f83200734e0 2024-01-15T17:55:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fda3fa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda38070ea0 0x7fda38154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fda3fa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda380cf7d0 0x7fda38142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd17f400640 1 -- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1800cfbe0 msgr2=0x7fd1800b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd17f400640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1800cfbe0 0x7fd1800b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd17f400640 1 -- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd180150b10 msgr2=0x7fd1800ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd17f400640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd180150b10 0x7fd1800ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fd17f400640 1 -- 172.21.15.26:0/3830878780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170009400 con 0x7fd180160f80 2024-01-15T17:55:11.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fa200a00640 1 -- 172.21.15.26:0/1858511911 wait complete. 2024-01-15T17:55:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7fda3fa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda3814b2c0 0x7fda38143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7f9704c00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96e006b390 0x7f96e006d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7f9704c00640 1 -- 172.21.15.26:0/439273470 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96f80ac230 con 0x7f97080ce8f0 2024-01-15T17:55:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7f9704c00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f96e00735b0 0x7f96e00759d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7f9704c00640 1 -- 172.21.15.26:0/439273470 --> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f96e0076080 con 0x7f96e00735b0 2024-01-15T17:55:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7f9704c00640 1 -- 172.21.15.26:0/439273470 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96f80ac5e0 con 0x7f97080ce8f0 2024-01-15T17:55:11.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7f96ffe00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96e006b390 0x7f96e006d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7fda3d000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda38070ea0 0x7fda38154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7fda3d000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda38070ea0 0x7fda38154970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58856/0 (socket says 172.21.15.26:58856) 2024-01-15T17:55:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7fda3d000640 1 -- 172.21.15.26:0/869625517 learned_addr learned my addr 172.21.15.26:0/869625517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda38144b00 con 0x7fda3814b2c0 2024-01-15T17:55:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda38142670 con 0x7fda38070ea0 2024-01-15T17:55:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda38143ef0 con 0x7fda380cf7d0 2024-01-15T17:55:11.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7fd17f400640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180160f80 0x7fd1800ca2e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd17002bdd0 tx=0x7fd1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.580+0000 7fda37e00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda380cf7d0 0x7fda38142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fcf01800640 1 -- 172.21.15.26:0/2234472544 shutdown_connections 2024-01-15T17:55:11.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.579+0000 7fcf01800640 1 -- 172.21.15.26:0/2234472544 wait complete. 2024-01-15T17:55:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fd177e00640 1 -- 172.21.15.26:0/3830878780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170032840 con 0x7fd180160f80 2024-01-15T17:55:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fd177e00640 1 -- 172.21.15.26:0/3830878780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1700329c0 con 0x7fd180160f80 2024-01-15T17:55:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fd177e00640 1 -- 172.21.15.26:0/3830878780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170032d70 con 0x7fd180160f80 2024-01-15T17:55:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1800ccc60 con 0x7fd180160f80 2024-01-15T17:55:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1801607f0 con 0x7fd180160f80 2024-01-15T17:55:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fcf01800640 1 Processor -- start 2024-01-15T17:55:11.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.582+0000 7fcf01800640 1 -- start start 2024-01-15T17:55:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.582+0000 7f96ffe00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96e006b390 0x7f96e006d850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96f0000f60 tx=0x7f96f00079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.583+0000 7fa200a00640 1 Processor -- start 2024-01-15T17:55:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.583+0000 7fa200a00640 1 -- start start 2024-01-15T17:55:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.583+0000 7fcf01800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcefc071070 0x7fcefc147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.583+0000 7fcf01800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcefc0cf7d0 0x7fcefc144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fda3d000640 1 -- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda380cf7d0 msgr2=0x7fda38142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fda3d000640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda380cf7d0 0x7fda38142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fda3d000640 1 -- 172.21.15.26:0/869625517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda3814b2c0 msgr2=0x7fda38143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fda3d000640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda3814b2c0 0x7fda38143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.581+0000 7fda3d000640 1 -- 172.21.15.26:0/869625517 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280094d0 con 0x7fda38070ea0 2024-01-15T17:55:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.582+0000 7fda3d000640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda38070ea0 0x7fda38154970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda280090b0 tx=0x7fda280046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.583+0000 7fda34c00640 1 -- 172.21.15.26:0/869625517 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2803ac60 con 0x7fda38070ea0 2024-01-15T17:55:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.583+0000 7fda34c00640 1 -- 172.21.15.26:0/869625517 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2803adc0 con 0x7fda38070ea0 2024-01-15T17:55:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.583+0000 7fda34c00640 1 -- 172.21.15.26:0/869625517 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28044630 con 0x7fda38070ea0 2024-01-15T17:55:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.584+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda381441d0 con 0x7fda38070ea0 2024-01-15T17:55:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.583+0000 7fcf01800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcefc14b2c0 0x7fcefc141fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.584+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda38144790 con 0x7fda38070ea0 2024-01-15T17:55:11.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.583+0000 7f970cc00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f96e00735b0 0x7f96e00759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.584+0000 7fcefa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcefc0cf7d0 0x7fcefc144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.584+0000 7fcefa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcefc0cf7d0 0x7fcefc144fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57716/0 (socket says 172.21.15.26:57716) 2024-01-15T17:55:11.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.584+0000 7fcefa000640 1 -- 172.21.15.26:0/2172058657 learned_addr learned my addr 172.21.15.26:0/2172058657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.584+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc145510 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.584+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1424e0 con 0x7fcefc071070 2024-01-15T17:55:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.584+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc145770 con 0x7fcefc14b2c0 2024-01-15T17:55:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.585+0000 7fcefaa00640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcefc071070 0x7fcefc147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.585+0000 7fcefb400640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcefc14b2c0 0x7fcefc141fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.585+0000 7fd177e00640 1 -- 172.21.15.26:0/3830878780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd17003b3f0 con 0x7fd180160f80 2024-01-15T17:55:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.585+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd150000fc0 con 0x7fd180160f80 2024-01-15T17:55:11.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.585+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda380cbc40 con 0x7fda38070ea0 2024-01-15T17:55:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.585+0000 7fa200a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1fc06c7c0 0x7fa1fc15c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fcefa000640 1 -- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcefc14b2c0 msgr2=0x7fcefc141fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fcefa000640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcefc14b2c0 0x7fcefc141fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fcefa000640 1 -- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcefc071070 msgr2=0x7fcefc147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fcefa000640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcefc071070 0x7fcefc147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fcefa000640 1 -- 172.21.15.26:0/2172058657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40094d0 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fa200a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc130ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fcefb400640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcefc14b2c0 0x7fcefc141fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fa200a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1fc071d30 0x7fa1fc130ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fd177e00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd15806b2c0 0x7fd15806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fd177e00640 1 -- 172.21.15.26:0/3830878780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1700ab350 con 0x7fd180160f80 2024-01-15T17:55:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fa200a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1318a0 con 0x7fa1fc06c7c0 2024-01-15T17:55:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fa200a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc131b80 con 0x7fa1fc071d30 2024-01-15T17:55:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.586+0000 7fa200a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc131e60 con 0x7fa1fc071410 2024-01-15T17:55:11.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fcefaa00640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcefc071070 0x7fcefc147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fd177e00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fd158073460 0x7fd158075880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fd177e00640 1 -- 172.21.15.26:0/3830878780 --> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd158075f30 con 0x7fd158073460 2024-01-15T17:55:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fd177e00640 1 -- 172.21.15.26:0/3830878780 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1700ad930 con 0x7fd180160f80 2024-01-15T17:55:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fa1fa800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1fc06c7c0 0x7fa1fc15c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fa1fa800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1fc06c7c0 0x7fa1fc15c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57718/0 (socket says 172.21.15.26:57718) 2024-01-15T17:55:11.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fa1f9e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc130ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fa1f9e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc130ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:39678/0 (socket says 172.21.15.26:39678) 2024-01-15T17:55:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fa1f9e00640 1 -- 172.21.15.26:0/2249802551 learned_addr learned my addr 172.21.15.26:0/2249802551 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fa1fb200640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1fc071d30 0x7fa1fc130ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fcefa000640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcefc0cf7d0 0x7fcefc144fd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fceec00c430 tx=0x7fceec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.587+0000 7fd17fe00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd15806b2c0 0x7fd15806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fcef2a00640 1 -- 172.21.15.26:0/2172058657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec011790 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc145a50 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fa1f9e00640 1 -- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1fc071d30 msgr2=0x7fa1fc130ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fa1f9e00640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1fc071d30 0x7fa1fc130ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fa1f9e00640 1 -- 172.21.15.26:0/2249802551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1fc06c7c0 msgr2=0x7fa1fc15c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fa1f9e00640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1fc06c7c0 0x7fa1fc15c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fa1f9e00640 1 -- 172.21.15.26:0/2249802551 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec009400 con 0x7fa1fc071410 2024-01-15T17:55:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fda34c00640 1 -- 172.21.15.26:0/869625517 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fda2802f050 con 0x7fda38070ea0 2024-01-15T17:55:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7f970cc00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f96e00735b0 0x7f96e00759d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fd184e00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fd158073460 0x7fd158075880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fcef2a00640 1 -- 172.21.15.26:0/2172058657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0118f0 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.588+0000 7fcef2a00640 1 -- 172.21.15.26:0/2172058657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec011790 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.589+0000 7fd17fe00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd15806b2c0 0x7fd15806d780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd1800b22d0 tx=0x7fd17800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.589+0000 7fa1f9e00640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc130ab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400b450 tx=0x7fa1e400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.589+0000 7fa1f2a00640 1 -- 172.21.15.26:0/2249802551 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4011890 con 0x7fa1fc071410 2024-01-15T17:55:11.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.589+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc132140 con 0x7fa1fc071410 2024-01-15T17:55:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.589+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc15ca50 con 0x7fa1fc071410 2024-01-15T17:55:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.589+0000 7fa1f2a00640 1 -- 172.21.15.26:0/2249802551 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e40119f0 con 0x7fa1fc071410 2024-01-15T17:55:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.589+0000 7fa1f2a00640 1 -- 172.21.15.26:0/2249802551 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4011890 con 0x7fa1fc071410 2024-01-15T17:55:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.590+0000 7fda34c00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda1006b2f0 0x7fda1006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.590+0000 7fda34c00640 1 -- 172.21.15.26:0/869625517 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda280abd50 con 0x7fda38070ea0 2024-01-15T17:55:11.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.590+0000 7fda34c00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7fda10073490 0x7fda100758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.591+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1cc000fc0 con 0x7fa1fc071410 2024-01-15T17:55:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.591+0000 7f9704c00640 1 -- 172.21.15.26:0/439273470 <== osd.3 v2:172.21.15.26:6808/696680319 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f96e0076080 con 0x7f96e00735b0 2024-01-15T17:55:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.590+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc146010 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.590+0000 7fda34c00640 1 -- 172.21.15.26:0/869625517 --> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda10075f60 con 0x7fda10073490 2024-01-15T17:55:11.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.590+0000 7fda34c00640 1 -- 172.21.15.26:0/869625517 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda280ac1a0 con 0x7fda38070ea0 2024-01-15T17:55:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.591+0000 7fda37e00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda1006b2f0 0x7fda1006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.592+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcefc0cbc40 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.592+0000 7fda3da00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7fda10073490 0x7fda100758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.592+0000 7fcef2a00640 1 -- 172.21.15.26:0/2172058657 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fceec011aa0 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.592+0000 7fd184e00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fd158073460 0x7fd158075880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.594+0000 7fcef2a00640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fced406b320 0x7fced406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.595+0000 7fcef2a00640 1 -- 172.21.15.26:0/2172058657 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fceec055ba0 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.595+0000 7fcefaa00640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fced406b320 0x7fced406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.595+0000 7fcef2a00640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7fced40734c0 0x7fced40758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.595+0000 7fcef2a00640 1 -- 172.21.15.26:0/2172058657 --> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fced4075f90 con 0x7fced40734c0 2024-01-15T17:55:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.595+0000 7fcef2a00640 1 -- 172.21.15.26:0/2172058657 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceec053da0 con 0x7fcefc0cf7d0 2024-01-15T17:55:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.594+0000 7fd177e00640 1 -- 172.21.15.26:0/3830878780 <== osd.5 v2:172.21.15.124:6824/4019293128 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd158075f30 con 0x7fd158073460 2024-01-15T17:55:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.594+0000 7fda3da00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7fda10073490 0x7fda100758b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.594+0000 7fa1f2a00640 1 -- 172.21.15.26:0/2249802551 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa1e4011b60 con 0x7fa1fc071410 2024-01-15T17:55:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.595+0000 7fda37e00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda1006b2f0 0x7fda1006d7b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda380b28d0 tx=0x7fda20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.596+0000 7fa1f2a00640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa1d406b320 0x7fa1d406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.596+0000 7fcefb400640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7fced40734c0 0x7fced40758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.596+0000 7f9d87400640 1 -- 172.21.15.26:0/771752336 --> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d900b2cb0 con 0x7f9d68073530 2024-01-15T17:55:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.596+0000 7fa1f2a00640 1 -- 172.21.15.26:0/2249802551 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1e408a040 con 0x7fa1fc071410 2024-01-15T17:55:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.596+0000 7fa1f2a00640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fa1d40734c0 0x7fa1d40758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.596+0000 7fcefaa00640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fced406b320 0x7fced406d7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcee4036000 tx=0x7fcee403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.596+0000 7fa1f2a00640 1 -- 172.21.15.26:0/2249802551 --> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1d4075f90 con 0x7fa1d40734c0 2024-01-15T17:55:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.596+0000 7fa1f2a00640 1 -- 172.21.15.26:0/2249802551 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1e408a490 con 0x7fa1fc071410 2024-01-15T17:55:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7f9d8e000640 1 -- 172.21.15.26:0/771752336 <== osd.0 v2:172.21.15.26:6800/3842439308 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9d8000a9e0 con 0x7f9d68073530 2024-01-15T17:55:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7fa1fb200640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fa1d40734c0 0x7fa1d40758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7fda34c00640 1 -- 172.21.15.26:0/869625517 <== osd.6 v2:172.21.15.124:6816/4103149233 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fda10075f60 con 0x7fda10073490 2024-01-15T17:55:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7fa1fa800640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa1d406b320 0x7fa1d406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7f9d87400640 1 -- 172.21.15.26:0/771752336 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7f9d68073530 msgr2=0x7f9d68075950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7f9d87400640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7f9d68073530 0x7f9d68075950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7f9d87400640 1 -- 172.21.15.26:0/771752336 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d6806b310 msgr2=0x7f9d6806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7f9d87400640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d6806b310 0x7f9d6806d7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d78000f60 tx=0x7f9d78009040 comp rx=0 tx=0).stop 2024-01-15T17:55:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7f9d87400640 1 -- 172.21.15.26:0/771752336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9015d4b0 msgr2=0x7f9d900b7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.597+0000 7f9d87400640 1 --2- 172.21.15.26:0/771752336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d9015d4b0 0x7f9d900b7ee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009e80 tx=0x7f9d88004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.598+0000 7f9d96a00640 1 -- 172.21.15.26:0/771752336 reap_dead start 2024-01-15T17:55:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.598+0000 7f9d87400640 1 -- 172.21.15.26:0/771752336 shutdown_connections 2024-01-15T17:55:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.598+0000 7f9d87400640 1 -- 172.21.15.26:0/771752336 >> 172.21.15.26:0/771752336 conn(0x7f9d900d3000 msgr2=0x7f9d9006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.598+0000 7fa1fa800640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa1d406b320 0x7fa1d406d7e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec036000 tx=0x7fa1ec03e040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.598+0000 7fcefb400640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7fced40734c0 0x7fced40758e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.599+0000 7f9d87400640 1 -- 172.21.15.26:0/771752336 shutdown_connections 2024-01-15T17:55:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.599+0000 7f9d87400640 1 -- 172.21.15.26:0/771752336 wait complete. 2024-01-15T17:55:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.599+0000 7fa1fb200640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fa1d40734c0 0x7fa1d40758e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.602+0000 7fcef2a00640 1 -- 172.21.15.26:0/2172058657 <== osd.4 v2:172.21.15.124:6800/126047302 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fced4075f90 con 0x7fced40734c0 2024-01-15T17:55:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.602+0000 7fa1f2a00640 1 -- 172.21.15.26:0/2249802551 <== osd.7 v2:172.21.15.124:6808/1393332266 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fa1d4075f90 con 0x7fa1d40734c0 2024-01-15T17:55:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.610+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 --> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8348161de0 con 0x7f83200734e0 2024-01-15T17:55:11.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.611+0000 7f833f400640 1 -- 172.21.15.26:0/716070652 <== osd.2 v2:172.21.15.26:6816/1591707592 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8330001210 con 0x7f83200734e0 2024-01-15T17:55:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.610+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 --> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb14c0d1600 con 0x7fb1240735b0 2024-01-15T17:55:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.611+0000 7fb149600640 1 -- 172.21.15.26:0/2155823034 <== osd.1 v2:172.21.15.26:6824/639007827 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb14400a720 con 0x7fb1240735b0 2024-01-15T17:55:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.611+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7fb1240735b0 msgr2=0x7fb1240759d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.611+0000 7fb154600640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7fb1240735b0 0x7fb1240759d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f83200734e0 msgr2=0x7f8320075900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7f834cc00640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f83200734e0 0x7f8320075900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f832006b2c0 msgr2=0x7f832006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7f834cc00640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f832006b2c0 0x7f832006d780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8338006fd0 tx=0x7f8338008040 comp rx=0 tx=0).stop 2024-01-15T17:55:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83480b2f10 msgr2=0x7f8348165320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7f834cc00640 1 --2- 172.21.15.26:0/716070652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83480b2f10 0x7f8348165320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8340036040 tx=0x7f8340004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb12406b390 msgr2=0x7fb12406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7fb154600640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb12406b390 0x7fb12406d850 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb13c036000 tx=0x7fb13c008030 comp rx=0 tx=0).stop 2024-01-15T17:55:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c0d0140 msgr2=0x7fb14c0c6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7fb154600640 1 --2- 172.21.15.26:0/2155823034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb14c0d0140 0x7fb14c0c6b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb13400c4a0 tx=0x7fb13400c960 comp rx=0 tx=0).stop 2024-01-15T17:55:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7f8347e00640 1 -- 172.21.15.26:0/716070652 reap_dead start 2024-01-15T17:55:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.612+0000 7fb152600640 1 -- 172.21.15.26:0/2155823034 reap_dead start 2024-01-15T17:55:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.613+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 shutdown_connections 2024-01-15T17:55:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.613+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 >> 172.21.15.26:0/2155823034 conn(0x7fb14c0d3020 msgr2=0x7fb14c0d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.613+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 shutdown_connections 2024-01-15T17:55:11.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.613+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 >> 172.21.15.26:0/716070652 conn(0x7f83480b0b00 msgr2=0x7f83480b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.614+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 shutdown_connections 2024-01-15T17:55:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.614+0000 7f834cc00640 1 -- 172.21.15.26:0/716070652 wait complete. 2024-01-15T17:55:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.617+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 shutdown_connections 2024-01-15T17:55:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.617+0000 7fb154600640 1 -- 172.21.15.26:0/2155823034 wait complete. 2024-01-15T17:55:11.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.620+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 --> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97080d1680 con 0x7f96e00735b0 2024-01-15T17:55:11.646 INFO:teuthology.orchestra.run.smithi026.stdout:60129542150 2024-01-15T17:55:11.646 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-15T17:55:11.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.621+0000 7f9704c00640 1 -- 172.21.15.26:0/439273470 <== osd.3 v2:172.21.15.26:6808/696680319 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9700001f40 con 0x7f96e00735b0 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.622+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f96e00735b0 msgr2=0x7f96e00759d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.622+0000 7f970ec00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f96e00735b0 0x7f96e00759d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.622+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96e006b390 msgr2=0x7f96e006d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.622+0000 7f970ec00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96e006b390 0x7f96e006d850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96f0000f60 tx=0x7f96f00079b0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.622+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080ce8f0 msgr2=0x7f97080c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.622+0000 7f970ec00640 1 --2- 172.21.15.26:0/439273470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97080ce8f0 0x7f97080c60b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96f8000c00 tx=0x7f96f80046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.622+0000 7f970cc00640 1 -- 172.21.15.26:0/439273470 reap_dead start 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.623+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 shutdown_connections 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.623+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 >> 172.21.15.26:0/439273470 conn(0x7f970809c550 msgr2=0x7f97080d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.623+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 shutdown_connections 2024-01-15T17:55:11.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.623+0000 7f970ec00640 1 -- 172.21.15.26:0/439273470 wait complete. 2024-01-15T17:55:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.628+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 --> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcefc0cb960 con 0x7fced40734c0 2024-01-15T17:55:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.628+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 --> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa1cc002cb0 con 0x7fa1d40734c0 2024-01-15T17:55:11.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.629+0000 7fcef2a00640 1 -- 172.21.15.26:0/2172058657 <== osd.4 v2:172.21.15.124:6800/126047302 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcef40019d0 con 0x7fced40734c0 2024-01-15T17:55:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.629+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7fced40734c0 msgr2=0x7fced40758e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fcf01800640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7fced40734c0 0x7fced40758e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.629+0000 7fa1f2a00640 1 -- 172.21.15.26:0/2249802551 <== osd.7 v2:172.21.15.124:6808/1393332266 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa1cc002cb0 con 0x7fa1d40734c0 2024-01-15T17:55:11.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 --> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd150002cb0 con 0x7fd158073460 2024-01-15T17:55:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fa1d40734c0 msgr2=0x7fa1d40758e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fa200a00640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fa1d40734c0 0x7fa1d40758e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fced406b320 msgr2=0x7fced406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fcf01800640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fced406b320 0x7fced406d7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcee4036000 tx=0x7fcee403b040 comp rx=0 tx=0).stop 2024-01-15T17:55:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcefc0cf7d0 msgr2=0x7fcefc144fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fcf01800640 1 --2- 172.21.15.26:0/2172058657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcefc0cf7d0 0x7fcefc144fd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fceec00c430 tx=0x7fceec00c8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa1d406b320 msgr2=0x7fa1d406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fa200a00640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa1d406b320 0x7fa1d406d7e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec036000 tx=0x7fa1ec03e040 comp rx=0 tx=0).stop 2024-01-15T17:55:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 msgr2=0x7fa1fc130ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fa200a00640 1 --2- 172.21.15.26:0/2249802551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1fc071410 0x7fa1fc130ab0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400b450 tx=0x7fa1e400b910 comp rx=0 tx=0).stop 2024-01-15T17:55:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fa1fb200640 1 -- 172.21.15.26:0/2249802551 reap_dead start 2024-01-15T17:55:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.631+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 shutdown_connections 2024-01-15T17:55:11.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.631+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 >> 172.21.15.26:0/2249802551 conn(0x7fa1fc06a800 msgr2=0x7fa1fc0b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fcefb400640 1 -- 172.21.15.26:0/2172058657 reap_dead start 2024-01-15T17:55:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 shutdown_connections 2024-01-15T17:55:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.630+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 >> 172.21.15.26:0/2172058657 conn(0x7fcefc06a800 msgr2=0x7fcefc1492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.631+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 shutdown_connections 2024-01-15T17:55:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.631+0000 7fcf01800640 1 -- 172.21.15.26:0/2172058657 wait complete. 2024-01-15T17:55:11.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.631+0000 7fd177e00640 1 -- 172.21.15.26:0/3830878780 <== osd.5 v2:172.21.15.124:6824/4019293128 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd150002cb0 con 0x7fd158073460 2024-01-15T17:55:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.632+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fd158073460 msgr2=0x7fd158075880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.632+0000 7fd186e00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fd158073460 0x7fd158075880 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.632+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 shutdown_connections 2024-01-15T17:55:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.632+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd15806b2c0 msgr2=0x7fd15806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.632+0000 7fd186e00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd15806b2c0 0x7fd15806d780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd1800b22d0 tx=0x7fd17800b040 comp rx=0 tx=0).stop 2024-01-15T17:55:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.632+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180160f80 msgr2=0x7fd1800ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.632+0000 7fd186e00640 1 --2- 172.21.15.26:0/3830878780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd180160f80 0x7fd1800ca2e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd17002bdd0 tx=0x7fd1700046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.633+0000 7fd184e00640 1 -- 172.21.15.26:0/3830878780 reap_dead start 2024-01-15T17:55:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.633+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 shutdown_connections 2024-01-15T17:55:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.633+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 >> 172.21.15.26:0/3830878780 conn(0x7fd18009c550 msgr2=0x7fd1800cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.633+0000 7fa200a00640 1 -- 172.21.15.26:0/2249802551 wait complete. 2024-01-15T17:55:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.634+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 shutdown_connections 2024-01-15T17:55:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.634+0000 7fd186e00640 1 -- 172.21.15.26:0/3830878780 wait complete. 2024-01-15T17:55:11.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.636+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 --> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda380cb960 con 0x7fda10073490 2024-01-15T17:55:11.657 INFO:teuthology.orchestra.run.smithi026.stdout:60129542150 2024-01-15T17:55:11.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-15T17:55:11.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.639+0000 7fda34c00640 1 -- 172.21.15.26:0/869625517 <== osd.6 v2:172.21.15.124:6816/4103149233 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda30006b10 con 0x7fda10073490 2024-01-15T17:55:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7fda10073490 msgr2=0x7fda100758b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7fda10073490 0x7fda100758b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda1006b2f0 msgr2=0x7fda1006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda1006b2f0 0x7fda1006d7b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda380b28d0 tx=0x7fda20008040 comp rx=0 tx=0).stop 2024-01-15T17:55:11.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda38070ea0 msgr2=0x7fda38154970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 --2- 172.21.15.26:0/869625517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda38070ea0 0x7fda38154970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fda280090b0 tx=0x7fda280046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3da00640 1 -- 172.21.15.26:0/869625517 reap_dead start 2024-01-15T17:55:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 shutdown_connections 2024-01-15T17:55:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 >> 172.21.15.26:0/869625517 conn(0x7fda3806a800 msgr2=0x7fda380ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 shutdown_connections 2024-01-15T17:55:11.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.640+0000 7fda3fa00640 1 -- 172.21.15.26:0/869625517 wait complete. 2024-01-15T17:55:11.659 INFO:teuthology.orchestra.run.smithi026.stdout:60129542149 2024-01-15T17:55:11.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-15T17:55:11.661 INFO:teuthology.orchestra.run.smithi026.stdout:60129542150 2024-01-15T17:55:11.661 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-15T17:55:11.662 INFO:teuthology.orchestra.run.smithi026.stdout:60129542148 2024-01-15T17:55:11.662 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-15T17:55:11.663 INFO:teuthology.orchestra.run.smithi026.stdout:60129542148 2024-01-15T17:55:11.663 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-15T17:55:11.664 INFO:teuthology.orchestra.run.smithi026.stdout:60129542149 2024-01-15T17:55:11.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-15T17:55:11.666 INFO:teuthology.orchestra.run.smithi026.stdout:60129542148 2024-01-15T17:55:11.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-15T17:55:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.943+0000 7f7f9d800640 1 Processor -- start 2024-01-15T17:55:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.943+0000 7f7f9d800640 1 -- start start 2024-01-15T17:55:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.945+0000 7f7f9d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f98070c50 0x7f7f981625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.945+0000 7f7f9d800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f980b2f10 0x7f7f980b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.945+0000 7f7f9d800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f98162b20 con 0x7f7f9815f1e0 2024-01-15T17:55:11.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.945+0000 7f7f9d800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f98071100 con 0x7f7f98070c50 2024-01-15T17:55:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.945+0000 7f7f9d800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9815f5c0 con 0x7f7f980b2f10 2024-01-15T17:55:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.946+0000 7f7f97400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f98070c50 0x7f7f981625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.946+0000 7f7f97400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f98070c50 0x7f7f981625e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44340/0 (socket says 172.21.15.26:44340) 2024-01-15T17:55:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.946+0000 7f7f97400640 1 -- 172.21.15.26:0/1156532490 learned_addr learned my addr 172.21.15.26:0/1156532490 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.946+0000 7f7f9ce00640 1 --2- 172.21.15.26:0/1156532490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f980b2f10 0x7f7f980b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.947+0000 7f7f97400640 1 -- 172.21.15.26:0/1156532490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f980b2f10 msgr2=0x7f7f980b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.947+0000 7f7f97400640 1 --2- 172.21.15.26:0/1156532490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f980b2f10 0x7f7f980b3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.947+0000 7f7f97400640 1 -- 172.21.15.26:0/1156532490 >> v1:172.21.15.26:6789/0 conn(0x7f7f9815f1e0 legacy=0x7f7f9806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.947+0000 7f7f97400640 1 -- 172.21.15.26:0/1156532490 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f980b38a0 con 0x7f7f98070c50 2024-01-15T17:55:11.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.950+0000 7f7f97400640 1 --2- 172.21.15.26:0/1156532490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f98070c50 0x7f7f981625e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009870 tx=0x7f7f8002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7486729c7bbbd75 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.950+0000 7f7f96a00640 1 -- 172.21.15.26:0/1156532490 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80031640 con 0x7f7f98070c50 2024-01-15T17:55:11.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.951+0000 7f7f96a00640 1 -- 172.21.15.26:0/1156532490 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f800317a0 con 0x7f7f98070c50 2024-01-15T17:55:11.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.951+0000 7f7f96a00640 1 -- 172.21.15.26:0/1156532490 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80031b30 con 0x7f7f98070c50 2024-01-15T17:55:11.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.951+0000 7f7f9d800640 1 -- 172.21.15.26:0/1156532490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f98070c50 msgr2=0x7f7f981625e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.951+0000 7f7f9d800640 1 --2- 172.21.15.26:0/1156532490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f98070c50 0x7f7f981625e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009870 tx=0x7f7f8002b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.952+0000 7f7f9d800640 1 -- 172.21.15.26:0/1156532490 shutdown_connections 2024-01-15T17:55:11.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.952+0000 7f7f9d800640 1 --2- 172.21.15.26:0/1156532490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f980b2f10 0x7f7f980b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.952+0000 7f7f9d800640 1 --2- 172.21.15.26:0/1156532490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f98070c50 0x7f7f981625e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.952+0000 7f7f9d800640 1 -- 172.21.15.26:0/1156532490 >> 172.21.15.26:0/1156532490 conn(0x7f7f980b0b00 msgr2=0x7f7f980b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.953+0000 7f7f9d800640 1 -- 172.21.15.26:0/1156532490 shutdown_connections 2024-01-15T17:55:11.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.953+0000 7f7f9d800640 1 -- 172.21.15.26:0/1156532490 wait complete. 2024-01-15T17:55:11.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.955+0000 7f251ca00640 1 Processor -- start 2024-01-15T17:55:11.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.955+0000 7f7f9d800640 1 Processor -- start 2024-01-15T17:55:11.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.955+0000 7f7f9d800640 1 -- start start 2024-01-15T17:55:11.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.957+0000 7f7f9d800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f98070c50 0x7f7f98160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.957+0000 7f7f9d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f980b2f10 0x7f7f981418f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.957+0000 7f7f9d800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f9815f1e0 0x7f7f980ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.957+0000 7f7f9d800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9814d950 con 0x7f7f980b2f10 2024-01-15T17:55:11.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.957+0000 7f7f9d800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f98149370 con 0x7f7f98070c50 2024-01-15T17:55:11.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.957+0000 7f7f9d800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f98160e60 con 0x7f7f9815f1e0 2024-01-15T17:55:11.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.955+0000 7f251ca00640 1 -- start start 2024-01-15T17:55:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.957+0000 7f251ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f250c03c060 0x7f250c0e7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.957+0000 7f251ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f250c053c70 0x7f250c054100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f251ca00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250c054640 con 0x7f250c0e3120 2024-01-15T17:55:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f251ca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250c007b30 con 0x7f250c03c060 2024-01-15T17:55:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f251ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0587d0 con 0x7f250c053c70 2024-01-15T17:55:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f2519600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f250c03c060 0x7f250c0e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f251a000640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f250c0e3120 0x7f250c058010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:34938/0 (socket says 172.21.15.26:34938) 2024-01-15T17:55:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f251a000640 1 -- 172.21.15.26:0/991271417 learned_addr learned my addr 172.21.15.26:0/991271417 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f7f97e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f98070c50 0x7f7f98160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f7f97e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f98070c50 0x7f7f98160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58768/0 (socket says 172.21.15.26:58768) 2024-01-15T17:55:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f7f97e00640 1 -- 172.21.15.26:0/861987509 learned_addr learned my addr 172.21.15.26:0/861987509 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f7f9ce00640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f9815f1e0 0x7f7f980ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.958+0000 7f7f97400640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f980b2f10 0x7f7f981418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.959+0000 7f9f6dc00640 1 Processor -- start 2024-01-15T17:55:11.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.959+0000 7f0052600640 1 Processor -- start 2024-01-15T17:55:11.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.959+0000 7f9f6dc00640 1 -- start start 2024-01-15T17:55:11.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.960+0000 7f0052600640 1 -- start start 2024-01-15T17:55:11.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.960+0000 7f7f97e00640 1 -- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f9815f1e0 msgr2=0x7f7f980ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.960+0000 7f7f97e00640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f9815f1e0 0x7f7f980ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.960+0000 7f7f97e00640 1 -- 172.21.15.26:0/861987509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f980b2f10 msgr2=0x7f7f981418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.960+0000 7f7f97e00640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f980b2f10 0x7f7f981418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.960+0000 7f7f97e00640 1 -- 172.21.15.26:0/861987509 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f800094d0 con 0x7f7f98070c50 2024-01-15T17:55:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.959+0000 7f2519600640 1 -- 172.21.15.26:0/991271417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f250c053c70 msgr2=0x7f250c054100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.959+0000 7f2519600640 1 --2- 172.21.15.26:0/991271417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f250c053c70 0x7f250c054100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.959+0000 7f2519600640 1 -- 172.21.15.26:0/991271417 >> v1:172.21.15.124:6789/0 conn(0x7f250c0e3120 legacy=0x7f250c058010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.959+0000 7f2519600640 1 -- 172.21.15.26:0/991271417 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c058ab0 con 0x7f250c03c060 2024-01-15T17:55:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.960+0000 7f2519600640 1 --2- 172.21.15.26:0/991271417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 0x7f250c0e7740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0097a0 tx=0x7f24fc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6778cf413afbde2b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.961+0000 7f7f97e00640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f98070c50 0x7f7f98160880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000c730 tx=0x7f7f9000cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f7f8fe00640 1 -- 172.21.15.26:0/861987509 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90003b70 con 0x7f7f98070c50 2024-01-15T17:55:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f7f8fe00640 1 -- 172.21.15.26:0/861987509 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90003cf0 con 0x7f7f98070c50 2024-01-15T17:55:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f7f8fe00640 1 -- 172.21.15.26:0/861987509 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90003ea0 con 0x7f7f98070c50 2024-01-15T17:55:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.961+0000 7f9f6dc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f680b16d0 0x7f9f68153950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.961+0000 7f9f6dc00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9f6815d5a0 0x7f9f68164220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.961+0000 7f9f6dc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f680b2b70 con 0x7f9f680cb0d0 2024-01-15T17:55:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.961+0000 7f9f6dc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f680b2e50 con 0x7f9f680b16d0 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.961+0000 7f9f6dc00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f680b3130 con 0x7f9f6815d5a0 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.961+0000 7f9f66a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f680b16d0 0x7f9f68153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.961+0000 7f9f66a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f680b16d0 0x7f9f68153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44374/0 (socket says 172.21.15.26:44374) 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f9f66a00640 1 -- 172.21.15.26:0/3256079422 learned_addr learned my addr 172.21.15.26:0/3256079422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f9f67400640 1 --2- 172.21.15.26:0/3256079422 >> v2:172.21.15.124:3300/0 conn(0x7f9f6815d5a0 0x7f9f68164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f9f66a00640 1 -- 172.21.15.26:0/3256079422 >> v2:172.21.15.124:3300/0 conn(0x7f9f6815d5a0 msgr2=0x7f9f68164220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f9f66a00640 1 --2- 172.21.15.26:0/3256079422 >> v2:172.21.15.124:3300/0 conn(0x7f9f6815d5a0 0x7f9f68164220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f9f66a00640 1 -- 172.21.15.26:0/3256079422 >> v1:172.21.15.26:6789/0 conn(0x7f9f680cb0d0 legacy=0x7f9f6815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f9f66a00640 1 -- 172.21.15.26:0/3256079422 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f680b3410 con 0x7f9f680b16d0 2024-01-15T17:55:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f98148d20 con 0x7f7f98070c50 2024-01-15T17:55:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f2518c00640 1 -- 172.21.15.26:0/991271417 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc02e070 con 0x7f250c03c060 2024-01-15T17:55:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f2518c00640 1 -- 172.21.15.26:0/991271417 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc0315a0 con 0x7f250c03c060 2024-01-15T17:55:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f2518c00640 1 -- 172.21.15.26:0/991271417 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc031910 con 0x7f250c03c060 2024-01-15T17:55:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.962+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f98141e30 con 0x7f7f98070c50 2024-01-15T17:55:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.963+0000 7f9f67400640 1 --2- 172.21.15.26:0/3256079422 >> v2:172.21.15.124:3300/0 conn(0x7f9f6815d5a0 0x7f9f68164220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.963+0000 7f0052600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f004c0b16d0 0x7f004c0cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.963+0000 7f251ca00640 1 -- 172.21.15.26:0/991271417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 msgr2=0x7f250c0e7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.963+0000 7f251ca00640 1 --2- 172.21.15.26:0/991271417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 0x7f250c0e7740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0097a0 tx=0x7f24fc02b840 comp rx=0 tx=0).stop 2024-01-15T17:55:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.963+0000 7f0052600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f004c1536e0 0x7f004c15ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.963+0000 7f0052600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c153b70 0x7f004c15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.963+0000 7f0052600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c160200 con 0x7f004c1536e0 2024-01-15T17:55:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.963+0000 7f0052600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c0b2e50 con 0x7f004c0b16d0 2024-01-15T17:55:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.963+0000 7f0052600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0b3130 con 0x7f004c153b70 2024-01-15T17:55:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f004b400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f004c0b16d0 0x7f004c0cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f251ca00640 1 -- 172.21.15.26:0/991271417 shutdown_connections 2024-01-15T17:55:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f251ca00640 1 --2- 172.21.15.26:0/991271417 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f250c053c70 0x7f250c054100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f251ca00640 1 --2- 172.21.15.26:0/991271417 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 0x7f250c0e7740 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f251ca00640 1 -- 172.21.15.26:0/991271417 >> 172.21.15.26:0/991271417 conn(0x7f250c058ea0 msgr2=0x7f250c059280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f004b400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f004c0b16d0 0x7f004c0cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58790/0 (socket says 172.21.15.26:58790) 2024-01-15T17:55:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f004b400640 1 -- 172.21.15.26:0/1002047823 learned_addr learned my addr 172.21.15.26:0/1002047823 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.965+0000 7f251ca00640 1 -- 172.21.15.26:0/991271417 shutdown_connections 2024-01-15T17:55:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.965+0000 7f251ca00640 1 -- 172.21.15.26:0/991271417 wait complete. 2024-01-15T17:55:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.965+0000 7f9f66a00640 1 --2- 172.21.15.26:0/3256079422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680b16d0 0x7f9f68153950 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f6815df50 tx=0x7f9f6002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae9f79ba156d4c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.965+0000 7f004be00640 1 --2- 172.21.15.26:0/1002047823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c153b70 0x7f004c15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.965+0000 7f004aa00640 1 --2- 172.21.15.26:0/1002047823 >> v2:172.21.15.26:3300/0 conn(0x7f004c1536e0 0x7f004c15ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f98148b80 con 0x7f7f98070c50 2024-01-15T17:55:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f55e9200640 1 Processor -- start 2024-01-15T17:55:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.964+0000 7f55e9200640 1 -- start start 2024-01-15T17:55:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f88c1800640 1 Processor -- start 2024-01-15T17:55:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f88c1800640 1 -- start start 2024-01-15T17:55:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f9f65600640 1 -- 172.21.15.26:0/3256079422 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6002cb10 con 0x7f9f680b16d0 2024-01-15T17:55:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f9f65600640 1 -- 172.21.15.26:0/3256079422 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6002cc70 con 0x7f9f680b16d0 2024-01-15T17:55:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f9f65600640 1 -- 172.21.15.26:0/3256079422 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f600343f0 con 0x7f9f680b16d0 2024-01-15T17:55:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f004b400640 1 -- 172.21.15.26:0/1002047823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c153b70 msgr2=0x7f004c15d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f004b400640 1 --2- 172.21.15.26:0/1002047823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c153b70 0x7f004c15d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f004b400640 1 -- 172.21.15.26:0/1002047823 >> v2:172.21.15.26:3300/0 conn(0x7f004c1536e0 msgr2=0x7f004c15ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f004b400640 1 --2- 172.21.15.26:0/1002047823 >> v2:172.21.15.26:3300/0 conn(0x7f004c1536e0 0x7f004c15ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.966+0000 7f004b400640 1 -- 172.21.15.26:0/1002047823 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c0b3410 con 0x7f004c0b16d0 2024-01-15T17:55:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.967+0000 7f55e9200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f55e415b140 0x7f55e40cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.967+0000 7f55e9200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e40d1ac0 0x7f55e40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.967+0000 7f55e9200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f55e40cd890 0x7f55e40cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.967+0000 7f9f6dc00640 1 -- 172.21.15.26:0/3256079422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680b16d0 msgr2=0x7f9f68153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.967+0000 7f9f6dc00640 1 --2- 172.21.15.26:0/3256079422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680b16d0 0x7f9f68153950 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9f6815df50 tx=0x7f9f6002bf70 comp rx=0 tx=0).stop 2024-01-15T17:55:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.967+0000 7f251ca00640 1 Processor -- start 2024-01-15T17:55:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f004aa00640 1 --2- 172.21.15.26:0/1002047823 >> v2:172.21.15.26:3300/0 conn(0x7f004c1536e0 0x7f004c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f9f6dc00640 1 -- 172.21.15.26:0/3256079422 shutdown_connections 2024-01-15T17:55:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f9f6dc00640 1 --2- 172.21.15.26:0/3256079422 >> v2:172.21.15.124:3300/0 conn(0x7f9f6815d5a0 0x7f9f68164220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f9f6dc00640 1 --2- 172.21.15.26:0/3256079422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680b16d0 0x7f9f68153950 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f9f6dc00640 1 -- 172.21.15.26:0/3256079422 >> 172.21.15.26:0/3256079422 conn(0x7f9f680b0be0 msgr2=0x7f9f680b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f251ca00640 1 -- start start 2024-01-15T17:55:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f55e1e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f55e415b140 0x7f55e40cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f55e1e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f55e415b140 0x7f55e40cf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58800/0 (socket says 172.21.15.26:58800) 2024-01-15T17:55:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f55e1e00640 1 -- 172.21.15.26:0/2594714050 learned_addr learned my addr 172.21.15.26:0/2594714050 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f55e9200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e40ce200 con 0x7f55e40cd890 2024-01-15T17:55:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f55e1400640 1 --2- 172.21.15.26:0/2594714050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e40d1ac0 0x7f55e40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f88c1800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc06e380 0x7f88bc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f88c1800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc15d840 con 0x7f88bc0b30b0 2024-01-15T17:55:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f55e2800640 1 --2- 172.21.15.26:0/2594714050 >> v2:172.21.15.26:3300/0 conn(0x7f55e40cd890 0x7f55e40cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f88c1800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88bc072190 con 0x7f88bc15d460 2024-01-15T17:55:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f88c1800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc071490 con 0x7f88bc06e380 2024-01-15T17:55:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f9f6dc00640 1 -- 172.21.15.26:0/3256079422 shutdown_connections 2024-01-15T17:55:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f55e9200640 1 -- 172.21.15.26:0/2594714050 --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e40ccef0 con 0x7f55e415b140 2024-01-15T17:55:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f55e9200640 1 -- 172.21.15.26:0/2594714050 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e40ceb30 con 0x7f55e40d1ac0 2024-01-15T17:55:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f55e2800640 1 -- 172.21.15.26:0/2594714050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e40d1ac0 msgr2=0x7f55e40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f55e2800640 1 --2- 172.21.15.26:0/2594714050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e40d1ac0 0x7f55e40cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f55e2800640 1 -- 172.21.15.26:0/2594714050 >> v2:172.21.15.124:3300/0 conn(0x7f55e415b140 msgr2=0x7f55e40cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f55e2800640 1 --2- 172.21.15.26:0/2594714050 >> v2:172.21.15.124:3300/0 conn(0x7f55e415b140 0x7f55e40cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f55e2800640 1 -- 172.21.15.26:0/2594714050 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e40cee10 con 0x7f55e40cd890 2024-01-15T17:55:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f7f8fe00640 1 -- 172.21.15.26:0/861987509 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7f9000aa90 con 0x7f7f98070c50 2024-01-15T17:55:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f9f6dc00640 1 -- 172.21.15.26:0/3256079422 wait complete. 2024-01-15T17:55:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f004b400640 1 --2- 172.21.15.26:0/1002047823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c0b16d0 0x7f004c0cb0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f004c15dfb0 tx=0x7f003c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17b5b056d4238d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f88ba000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f88bc0b30b0 0x7f88bc06d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42946/0 (socket says 172.21.15.26:42946) 2024-01-15T17:55:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.968+0000 7f88ba000640 1 -- 172.21.15.26:0/3576066927 learned_addr learned my addr 172.21.15.26:0/3576066927 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f88bb400640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc06e380 0x7f88bc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860232635 0 0) 0x7f88bc072190 con 0x7f88bc15d460 2024-01-15T17:55:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f889c003520 con 0x7f88bc15d460 2024-01-15T17:55:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f251ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 0x7f250c0de480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.969+0000 7f251ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f250c053c70 0x7f250c051d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849944130 0 0) 0x7f88bc15d840 con 0x7f88bc0b30b0 2024-01-15T17:55:11.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88bc072190 con 0x7f88bc0b30b0 2024-01-15T17:55:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f250c0e3120 0x7f250c04d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c04c9d0 con 0x7f250c03c060 2024-01-15T17:55:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 0x7f250c0de480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 0x7f250c0de480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44388/0 (socket says 172.21.15.26:44388) 2024-01-15T17:55:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251a000640 1 -- 172.21.15.26:0/4061708071 learned_addr learned my addr 172.21.15.26:0/4061708071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c04d270 con 0x7f250c053c70 2024-01-15T17:55:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0de250 con 0x7f250c0e3120 2024-01-15T17:55:11.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f55e1e00640 1 --2- 172.21.15.26:0/2594714050 >> v2:172.21.15.124:3300/0 conn(0x7f55e415b140 0x7f55e40cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:11.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251aa00640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f250c0e3120 0x7f250c04d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251a000640 1 -- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f250c0e3120 msgr2=0x7f250c04d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251a000640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f250c0e3120 0x7f250c04d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251a000640 1 -- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f250c053c70 msgr2=0x7f250c051d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251a000640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f250c053c70 0x7f250c051d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.970+0000 7f251a000640 1 -- 172.21.15.26:0/4061708071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc009400 con 0x7f250c03c060 2024-01-15T17:55:11.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2366410168 0 0) 0x7f88bc072190 con 0x7f88bc0b30b0 2024-01-15T17:55:11.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc06e380 msgr2=0x7f88bc0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc06e380 0x7f88bc0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 >> v1:172.21.15.124:6789/0 conn(0x7f88bc15d460 legacy=0x7f88bc0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc072470 con 0x7f88bc0b30b0 2024-01-15T17:55:11.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f88ac001670 con 0x7f88bc0b30b0 2024-01-15T17:55:11.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 >> v1:172.21.15.26:6789/0 conn(0x7f88bc0b30b0 legacy=0x7f88bc06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f9f6dc00640 1 Processor -- start 2024-01-15T17:55:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c003b30 0x7f889c003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f889c0047c0 0x7f889c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc06e380 0x7f889c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc071490 con 0x7f889c003b30 2024-01-15T17:55:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c014240 con 0x7f889c0047c0 2024-01-15T17:55:11.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c014400 con 0x7f88bc06e380 2024-01-15T17:55:11.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f9f6dc00640 1 -- start start 2024-01-15T17:55:11.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f55e2800640 1 --2- 172.21.15.26:0/2594714050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 0x7f55e40cdcc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55e4068150 tx=0x7f55dc02c080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3e7d6e24402c11 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f251a000640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 0x7f250c0de480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f250400b460 tx=0x7f250400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.971+0000 7f7f8fe00640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f7006b310 0x7f7f7006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f7f8fe00640 1 -- 172.21.15.26:0/861987509 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f900896f0 con 0x7f7f98070c50 2024-01-15T17:55:11.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f88baa00640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c003b30 0x7f889c003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f55e0a00640 1 -- 172.21.15.26:0/2594714050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc02cc20 con 0x7f55e40cd890 2024-01-15T17:55:11.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f2512000640 1 -- 172.21.15.26:0/4061708071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011890 con 0x7f250c03c060 2024-01-15T17:55:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f88ba000640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f889c0047c0 0x7f889c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f2512000640 1 -- 172.21.15.26:0/4061708071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504011a10 con 0x7f250c03c060 2024-01-15T17:55:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f2512000640 1 -- 172.21.15.26:0/4061708071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504011dc0 con 0x7f250c03c060 2024-01-15T17:55:11.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250c0523b0 con 0x7f250c03c060 2024-01-15T17:55:11.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c04cf00 con 0x7f250c03c060 2024-01-15T17:55:11.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.972+0000 7f55e0a00640 1 -- 172.21.15.26:0/2594714050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc02cd80 con 0x7f55e40cd890 2024-01-15T17:55:11.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f88baa00640 1 -- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc06e380 msgr2=0x7f889c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f88baa00640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc06e380 0x7f889c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f88baa00640 1 -- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f889c0047c0 msgr2=0x7f889c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:11.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f88baa00640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f889c0047c0 0x7f889c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f88baa00640 1 -- 172.21.15.26:0/3576066927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc072190 con 0x7f889c003b30 2024-01-15T17:55:11.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f6dc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f680b16d0 0x7f9f68154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f55e0a00640 1 -- 172.21.15.26:0/2594714050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc034660 con 0x7f55e40cd890 2024-01-15T17:55:11.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f6dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680cb0d0 0x7f9f68157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f6dc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6815d5a0 0x7f9f68158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:11.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f6dc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6814c040 con 0x7f9f680cb0d0 2024-01-15T17:55:11.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f6dc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68157680 con 0x7f9f6815d5a0 2024-01-15T17:55:11.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f6dc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6814b7a0 con 0x7f9f680b16d0 2024-01-15T17:55:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f55e9200640 1 -- 172.21.15.26:0/2594714050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 msgr2=0x7f55e40cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f55e9200640 1 --2- 172.21.15.26:0/2594714050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 0x7f55e40cdcc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f55e4068150 tx=0x7f55dc02c080 comp rx=0 tx=0).stop 2024-01-15T17:55:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f67400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6815d5a0 0x7f9f68158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f66a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f680b16d0 0x7f9f68154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f66a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f680b16d0 0x7f9f68154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:37168/0 (socket says 172.21.15.26:37168) 2024-01-15T17:55:11.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f9f66a00640 1 -- 172.21.15.26:0/1043916714 learned_addr learned my addr 172.21.15.26:0/1043916714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:11.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.973+0000 7f7f97400640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f7006b310 0x7f7f7006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f9f66000640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680cb0d0 0x7f9f68157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:11.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f55e9200640 1 -- 172.21.15.26:0/2594714050 shutdown_connections 2024-01-15T17:55:11.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f55e9200640 1 --2- 172.21.15.26:0/2594714050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 0x7f55e40cdcc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f55e9200640 1 --2- 172.21.15.26:0/2594714050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e40d1ac0 0x7f55e40cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f55e9200640 1 --2- 172.21.15.26:0/2594714050 >> v2:172.21.15.124:3300/0 conn(0x7f55e415b140 0x7f55e40cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f55e9200640 1 -- 172.21.15.26:0/2594714050 >> 172.21.15.26:0/2594714050 conn(0x7f55e40d0ec0 msgr2=0x7f55e40d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f55e9200640 1 -- 172.21.15.26:0/2594714050 shutdown_connections 2024-01-15T17:55:11.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f250c005350 con 0x7f250c03c060 2024-01-15T17:55:11.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.975+0000 7f7f97400640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f7006b310 0x7f7f7006d7d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f80036000 tx=0x7f7f8003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.975+0000 7f88baa00640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c003b30 0x7f889c003f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88a4001900 tx=0x7f889c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b678ba0d861d78c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.975+0000 7f55e9200640 1 -- 172.21.15.26:0/2594714050 wait complete. 2024-01-15T17:55:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f9f66000640 1 -- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f680b16d0 msgr2=0x7f9f68154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f9f66000640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f680b16d0 0x7f9f68154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f9f66000640 1 -- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6815d5a0 msgr2=0x7f9f68158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f9f66000640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6815d5a0 0x7f9f68158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.974+0000 7f9f66000640 1 -- 172.21.15.26:0/1043916714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f60009b30 con 0x7f9f680cb0d0 2024-01-15T17:55:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.975+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a40052d0 con 0x7f889c003b30 2024-01-15T17:55:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.975+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a4005450 con 0x7f889c003b30 2024-01-15T17:55:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.975+0000 7f88b9600640 1 -- 172.21.15.26:0/3576066927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4003830 con 0x7f889c003b30 2024-01-15T17:55:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.975+0000 7f9f66000640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680cb0d0 0x7f9f68157b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000c370 tx=0x7f9f5000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:11.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f88c1800640 1 -- 172.21.15.26:0/3576066927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c003b30 msgr2=0x7f889c003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:11.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f88c1800640 1 --2- 172.21.15.26:0/3576066927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f889c003b30 0x7f889c003f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f88a4001900 tx=0x7f889c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:11.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f88bb400640 1 -- 172.21.15.26:0/3576066927 reap_dead start 2024-01-15T17:55:11.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f9f5ea00640 1 -- 172.21.15.26:0/1043916714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f500052d0 con 0x7f9f680cb0d0 2024-01-15T17:55:11.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f88c1800640 1 -- 172.21.15.26:0/3576066927 shutdown_connections 2024-01-15T17:55:11.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f88c1800640 1 -- 172.21.15.26:0/3576066927 >> 172.21.15.26:0/3576066927 conn(0x7f88bc0b0be0 msgr2=0x7f88bc0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:11.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f9f5ea00640 1 -- 172.21.15.26:0/1043916714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f50005450 con 0x7f9f680cb0d0 2024-01-15T17:55:11.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f9f5ea00640 1 -- 172.21.15.26:0/1043916714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50007260 con 0x7f9f680cb0d0 2024-01-15T17:55:11.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6814ba80 con 0x7f9f680cb0d0 2024-01-15T17:55:11.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.976+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6814c440 con 0x7f9f680cb0d0 2024-01-15T17:55:11.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.977+0000 7f2512000640 1 -- 172.21.15.26:0/4061708071 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f250401a410 con 0x7f250c03c060 2024-01-15T17:55:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.977+0000 7f88c1800640 1 -- 172.21.15.26:0/3576066927 shutdown_connections 2024-01-15T17:55:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.977+0000 7f55e9200640 1 Processor -- start 2024-01-15T17:55:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.977+0000 7f55e9200640 1 -- start start 2024-01-15T17:55:11.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.978+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f38005250 con 0x7f9f680cb0d0 2024-01-15T17:55:12.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.977+0000 7f88c1800640 1 -- 172.21.15.26:0/3576066927 wait complete. 2024-01-15T17:55:12.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.978+0000 7f2512000640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f24ec06b3c0 0x7f24ec06d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.978+0000 7f2512000640 1 -- 172.21.15.26:0/4061708071 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2504089940 con 0x7f250c03c060 2024-01-15T17:55:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 0x7f55e414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e9200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55e40d1ac0 0x7f55e41e5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e9200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e415b140 0x7f55e41e39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e9200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e41e4f90 con 0x7f55e40cd890 2024-01-15T17:55:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e9200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e41e4380 con 0x7f55e40d1ac0 2024-01-15T17:55:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e9200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e41e4660 con 0x7f55e415b140 2024-01-15T17:55:12.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.978+0000 7f3819e00640 1 Processor -- start 2024-01-15T17:55:12.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 0x7f55e414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 0x7f55e414c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44414/0 (socket says 172.21.15.26:44414) 2024-01-15T17:55:12.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e1e00640 1 -- 172.21.15.26:0/1491941518 learned_addr learned my addr 172.21.15.26:0/1491941518 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:12.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f2519600640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f24ec06b3c0 0x7f24ec06d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f3819e00640 1 -- start start 2024-01-15T17:55:12.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e1400640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55e40d1ac0 0x7f55e41e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.979+0000 7f55e2800640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e415b140 0x7f55e41e39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.980+0000 7f55e1e00640 1 -- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e415b140 msgr2=0x7f55e41e39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.980+0000 7f55e1e00640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e415b140 0x7f55e41e39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.980+0000 7f55e1e00640 1 -- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55e40d1ac0 msgr2=0x7f55e41e5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.980+0000 7f55e1e00640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55e40d1ac0 0x7f55e41e5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.980+0000 7f55e1e00640 1 -- 172.21.15.26:0/1491941518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc009a70 con 0x7f55e40cd890 2024-01-15T17:55:12.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.980+0000 7f55e1e00640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 0x7f55e414c0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c7f0 tx=0x7f55d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.980+0000 7f2519600640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f24ec06b3c0 0x7f24ec06d880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24fc036000 tx=0x7f24fc030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.981+0000 7f3819e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38140cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.981+0000 7f3819e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38140ce8c0 0x7f38140b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.981+0000 7f3819e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140b6750 con 0x7f38140d00b0 2024-01-15T17:55:12.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.981+0000 7f88c1800640 1 Processor -- start 2024-01-15T17:55:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.981+0000 7f3819e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38140ced20 con 0x7f38140ce8c0 2024-01-15T17:55:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.981+0000 7f3819e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140d0530 con 0x7f381415d590 2024-01-15T17:55:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.981+0000 7f88c1800640 1 -- start start 2024-01-15T17:55:12.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.982+0000 7f3813400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38140ce8c0 0x7f38140b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.982+0000 7f9f5ea00640 1 -- 172.21.15.26:0/1043916714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9f500073c0 con 0x7f9f680cb0d0 2024-01-15T17:55:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.982+0000 7f3813400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f38140ce8c0 0x7f38140b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44422/0 (socket says 172.21.15.26:44422) 2024-01-15T17:55:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.982+0000 7f3813400640 1 -- 172.21.15.26:0/1441554749 learned_addr learned my addr 172.21.15.26:0/1441554749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.982+0000 7f3812a00640 1 --2- 172.21.15.26:0/1441554749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38140cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f3813400640 1 -- 172.21.15.26:0/1441554749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 msgr2=0x7f38140cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f3813400640 1 --2- 172.21.15.26:0/1441554749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38140cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f3813400640 1 -- 172.21.15.26:0/1441554749 >> v1:172.21.15.26:6789/0 conn(0x7f38140d00b0 legacy=0x7f38140ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f88c1800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88bc14ce10 0x7f88bc14d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f88c1800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc0cd2f0 0x7f88bc0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f88c1800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88bc0cc730 0x7f88bc156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f3813400640 1 -- 172.21.15.26:0/1441554749 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140cf1f0 con 0x7f38140ce8c0 2024-01-15T17:55:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f88c1800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0cbdd0 con 0x7f88bc14ce10 2024-01-15T17:55:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f88c1800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0ce070 con 0x7f88bc0cc730 2024-01-15T17:55:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f88c1800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0cdab0 con 0x7f88bc0cd2f0 2024-01-15T17:55:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f55da000640 1 -- 172.21.15.26:0/1491941518 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4007820 con 0x7f55e40cd890 2024-01-15T17:55:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.984+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e41e4940 con 0x7f55e40cd890 2024-01-15T17:55:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.983+0000 7f9f5ea00640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f4006b310 0x7f9f4006d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.984+0000 7f9f5ea00640 1 -- 172.21.15.26:0/1043916714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f500891f0 con 0x7f9f680cb0d0 2024-01-15T17:55:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.984+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e414c600 con 0x7f55e40cd890 2024-01-15T17:55:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.984+0000 7f88bb400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88bc0cc730 0x7f88bc156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.984+0000 7f88bb400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88bc0cc730 0x7f88bc156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58836/0 (socket says 172.21.15.26:58836) 2024-01-15T17:55:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.984+0000 7f88bb400640 1 -- 172.21.15.26:0/547992365 learned_addr learned my addr 172.21.15.26:0/547992365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.984+0000 7f88ba000640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc0cd2f0 0x7f88bc0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.984+0000 7f9f66a00640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f4006b310 0x7f9f4006d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.984+0000 7f3813400640 1 --2- 172.21.15.26:0/1441554749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ce8c0 0x7f38140b6be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e50 tx=0x7f380c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a512ff26702b94f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.985+0000 7f88bb400640 1 -- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc0cd2f0 msgr2=0x7f88bc0cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.985+0000 7f88bb400640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc0cd2f0 0x7f88bc0cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.985+0000 7f88bb400640 1 -- 172.21.15.26:0/547992365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88bc14ce10 msgr2=0x7f88bc14d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.985+0000 7f88bb400640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88bc14ce10 0x7f88bc14d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.985+0000 7f88bb400640 1 -- 172.21.15.26:0/547992365 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0cdd90 con 0x7f88bc0cc730 2024-01-15T17:55:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.985+0000 7f2512000640 1 -- 172.21.15.26:0/4061708071 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2504054820 con 0x7f250c03c060 2024-01-15T17:55:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.985+0000 7f88bb400640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88bc0cc730 0x7f88bc156290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0aeb00 tx=0x7f88b400b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.985+0000 7f9f66a00640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f4006b310 0x7f9f4006d7d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f6002bfb0 tx=0x7f9f60030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.986+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e4067450 con 0x7f55e40cd890 2024-01-15T17:55:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.989+0000 7f55da000640 1 -- 172.21.15.26:0/1491941518 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d400a450 con 0x7f55e40cd890 2024-01-15T17:55:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.989+0000 7f55da000640 1 -- 172.21.15.26:0/1491941518 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4007d90 con 0x7f55e40cd890 2024-01-15T17:55:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.989+0000 7f3811600640 1 -- 172.21.15.26:0/1441554749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c02cb00 con 0x7f38140ce8c0 2024-01-15T17:55:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.989+0000 7f9f5ea00640 1 -- 172.21.15.26:0/1043916714 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f50054280 con 0x7f9f680cb0d0 2024-01-15T17:55:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.989+0000 7f3811600640 1 -- 172.21.15.26:0/1441554749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c02cc60 con 0x7f38140ce8c0 2024-01-15T17:55:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.989+0000 7f3811600640 1 -- 172.21.15.26:0/1441554749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c0343f0 con 0x7f38140ce8c0 2024-01-15T17:55:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.990+0000 7f004a000640 1 -- 172.21.15.26:0/1002047823 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c030860 con 0x7f004c0b16d0 2024-01-15T17:55:12.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.990+0000 7f004a000640 1 -- 172.21.15.26:0/1002047823 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c0309c0 con 0x7f004c0b16d0 2024-01-15T17:55:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.990+0000 7f004a000640 1 -- 172.21.15.26:0/1002047823 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c030d50 con 0x7f004c0b16d0 2024-01-15T17:55:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.990+0000 7f3819e00640 1 -- 172.21.15.26:0/1441554749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ce8c0 msgr2=0x7f38140b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.990+0000 7f3819e00640 1 --2- 172.21.15.26:0/1441554749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ce8c0 0x7f38140b6be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e50 tx=0x7f380c02bf60 comp rx=0 tx=0).stop 2024-01-15T17:55:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f3819e00640 1 -- 172.21.15.26:0/1441554749 shutdown_connections 2024-01-15T17:55:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f3819e00640 1 --2- 172.21.15.26:0/1441554749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ce8c0 0x7f38140b6be0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f3819e00640 1 --2- 172.21.15.26:0/1441554749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38140cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f3819e00640 1 -- 172.21.15.26:0/1441554749 >> 172.21.15.26:0/1441554749 conn(0x7f381409c540 msgr2=0x7f38140d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f88b2a00640 1 -- 172.21.15.26:0/547992365 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4010040 con 0x7f88bc0cc730 2024-01-15T17:55:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f88b2a00640 1 -- 172.21.15.26:0/547992365 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b400bd10 con 0x7f88bc0cc730 2024-01-15T17:55:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f88b2a00640 1 -- 172.21.15.26:0/547992365 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4014640 con 0x7f88bc0cc730 2024-01-15T17:55:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc0cb9f0 con 0x7f88bc0cc730 2024-01-15T17:55:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc14d1f0 con 0x7f88bc0cc730 2024-01-15T17:55:12.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.992+0000 7f7f8fe00640 1 -- 172.21.15.26:0/861987509 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f98148b80 con 0x7f7f98070c50 2024-01-15T17:55:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.992+0000 7f3819e00640 1 -- 172.21.15.26:0/1441554749 shutdown_connections 2024-01-15T17:55:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.992+0000 7f3819e00640 1 -- 172.21.15.26:0/1441554749 wait complete. 2024-01-15T17:55:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f0052600640 1 -- 172.21.15.26:0/1002047823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c0b16d0 msgr2=0x7f004c0cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f0052600640 1 --2- 172.21.15.26:0/1002047823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c0b16d0 0x7f004c0cb0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f004c15dfb0 tx=0x7f003c02b9a0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f0052600640 1 -- 172.21.15.26:0/1002047823 shutdown_connections 2024-01-15T17:55:12.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f0052600640 1 --2- 172.21.15.26:0/1002047823 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c153b70 0x7f004c15d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f0052600640 1 --2- 172.21.15.26:0/1002047823 >> v2:172.21.15.26:3300/0 conn(0x7f004c1536e0 0x7f004c15ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f0052600640 1 --2- 172.21.15.26:0/1002047823 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c0b16d0 0x7f004c0cb0d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f0052600640 1 -- 172.21.15.26:0/1002047823 >> 172.21.15.26:0/1002047823 conn(0x7f004c0b0be0 msgr2=0x7f004c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.991+0000 7f0052600640 1 -- 172.21.15.26:0/1002047823 shutdown_connections 2024-01-15T17:55:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.992+0000 7f0052600640 1 -- 172.21.15.26:0/1002047823 wait complete. 2024-01-15T17:55:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.993+0000 7f55da000640 1 -- 172.21.15.26:0/1491941518 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f55d4024480 con 0x7f55e40cd890 2024-01-15T17:55:12.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.995+0000 7f3819e00640 1 Processor -- start 2024-01-15T17:55:12.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.995+0000 7f55da000640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f55bc06b2f0 0x7f55bc06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.995+0000 7f55da000640 1 -- 172.21.15.26:0/1491941518 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55d408a2b0 con 0x7f55e40cd890 2024-01-15T17:55:12.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.994+0000 7f0052600640 1 Processor -- start 2024-01-15T17:55:12.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.993+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc0cc040 con 0x7f88bc0cc730 2024-01-15T17:55:12.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.995+0000 7f0052600640 1 -- start start 2024-01-15T17:55:12.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.995+0000 7f3819e00640 1 -- start start 2024-01-15T17:55:12.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.996+0000 7f88b2a00640 1 -- 172.21.15.26:0/547992365 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f88b40147a0 con 0x7f88bc0cc730 2024-01-15T17:55:12.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.996+0000 7f55e1400640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f55bc06b2f0 0x7f55bc06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.997+0000 7f88b2a00640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f889406b390 0x7f889406d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.997+0000 7f88b2a00640 1 -- 172.21.15.26:0/547992365 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88b408a310 con 0x7f88bc0cc730 2024-01-15T17:55:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.997+0000 7f0052600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0b16d0 0x7f004c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.997+0000 7f0052600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c1536e0 0x7f004c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.997+0000 7f0052600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c13b8d0 0x7f004c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f0052600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c144950 con 0x7f004c1536e0 2024-01-15T17:55:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f0052600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c1552e0 con 0x7f004c13b8d0 2024-01-15T17:55:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f004aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c1536e0 0x7f004c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f004aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c1536e0 0x7f004c1544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44432/0 (socket says 172.21.15.26:44432) 2024-01-15T17:55:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f004aa00640 1 -- 172.21.15.26:0/852854463 learned_addr learned my addr 172.21.15.26:0/852854463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f004b400640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0b16d0 0x7f004c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c155000 con 0x7f004c0b16d0 2024-01-15T17:55:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f3819e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140ce8c0 0x7f38140cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f3819e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140d00b0 0x7f381415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f3819e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38141f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f3819e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141fa780 con 0x7f38140d00b0 2024-01-15T17:55:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f3819e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141f7ac0 con 0x7f38140ce8c0 2024-01-15T17:55:12.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f3819e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140cc370 con 0x7f381415d590 2024-01-15T17:55:12.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f88baa00640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f889406b390 0x7f889406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.998+0000 7f55da000640 1 -- 172.21.15.26:0/1491941518 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55d4055190 con 0x7f55e40cd890 2024-01-15T17:55:12.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.999+0000 7f55e1400640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f55bc06b2f0 0x7f55bc06d7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55cc006fd0 tx=0x7f55cc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.999+0000 7f3812000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140d00b0 0x7f381415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.999+0000 7f3812a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140ce8c0 0x7f38140cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.999+0000 7f3813400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38141f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.999+0000 7f3813400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38141f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:37210/0 (socket says 172.21.15.26:37210) 2024-01-15T17:55:12.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:11.999+0000 7f3813400640 1 -- 172.21.15.26:0/2150599094 learned_addr learned my addr 172.21.15.26:0/2150599094 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:12.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f004be00640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c13b8d0 0x7f004c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f004aa00640 1 -- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0b16d0 msgr2=0x7f004c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f004aa00640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0b16d0 0x7f004c153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f004aa00640 1 -- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c13b8d0 msgr2=0x7f004c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f004aa00640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c13b8d0 0x7f004c1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f004aa00640 1 -- 172.21.15.26:0/852854463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0094d0 con 0x7f004c1536e0 2024-01-15T17:55:12.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.001+0000 7f004aa00640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c1536e0 0x7f004c1544a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f003400e810 tx=0x7f003400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f3813400640 1 -- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140ce8c0 msgr2=0x7f38140cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f3813400640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140ce8c0 0x7f38140cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f3813400640 1 -- 172.21.15.26:0/2150599094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140d00b0 msgr2=0x7f381415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f3813400640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140d00b0 0x7f381415f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.000+0000 7f3813400640 1 -- 172.21.15.26:0/2150599094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c009b30 con 0x7f381415d590 2024-01-15T17:55:12.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.002+0000 7f0043400640 1 -- 172.21.15.26:0/852854463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034004030 con 0x7f004c1536e0 2024-01-15T17:55:12.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.002+0000 7f0043400640 1 -- 172.21.15.26:0/852854463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0034004190 con 0x7f004c1536e0 2024-01-15T17:55:12.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.002+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c159e50 con 0x7f004c1536e0 2024-01-15T17:55:12.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.002+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c15a410 con 0x7f004c1536e0 2024-01-15T17:55:12.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.002+0000 7f0043400640 1 -- 172.21.15.26:0/852854463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034004520 con 0x7f004c1536e0 2024-01-15T17:55:12.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.004+0000 7f88baa00640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f889406b390 0x7f889406d850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88a4034000 tx=0x7f88a4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.004+0000 7f3813400640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38141f6ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e50 tx=0x7f380c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.005+0000 7f380b400640 1 -- 172.21.15.26:0/2150599094 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c009510 con 0x7f381415d590 2024-01-15T17:55:12.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.005+0000 7f380b400640 1 -- 172.21.15.26:0/2150599094 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c03e450 con 0x7f381415d590 2024-01-15T17:55:12.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.005+0000 7f380b400640 1 -- 172.21.15.26:0/2150599094 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c03e800 con 0x7f381415d590 2024-01-15T17:55:12.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.006+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38140cc650 con 0x7f381415d590 2024-01-15T17:55:12.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.006+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38141f7da0 con 0x7f381415d590 2024-01-15T17:55:12.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.004+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c005250 con 0x7f004c1536e0 2024-01-15T17:55:12.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.007+0000 7f0043400640 1 -- 172.21.15.26:0/852854463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f003400c6d0 con 0x7f004c1536e0 2024-01-15T17:55:12.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.008+0000 7f0043400640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b3c0 0x7f002406d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.008+0000 7f0043400640 1 -- 172.21.15.26:0/852854463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0034089560 con 0x7f004c1536e0 2024-01-15T17:55:12.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.009+0000 7f004b400640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b3c0 0x7f002406d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.010+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38140ccb70 con 0x7f381415d590 2024-01-15T17:55:12.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.010+0000 7f004b400640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b3c0 0x7f002406d880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f003c036000 tx=0x7f003c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.011+0000 7f380b400640 1 -- 172.21.15.26:0/2150599094 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f380c03e960 con 0x7f381415d590 2024-01-15T17:55:12.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.011+0000 7f380b400640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06b340 0x7f37ec06d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.012+0000 7f380b400640 1 -- 172.21.15.26:0/2150599094 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f380c0ac6b0 con 0x7f381415d590 2024-01-15T17:55:12.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.012+0000 7f3812a00640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06b340 0x7f37ec06d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.013+0000 7f88b2a00640 1 -- 172.21.15.26:0/547992365 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88b40552b0 con 0x7f88bc0cc730 2024-01-15T17:55:12.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.013+0000 7f3812a00640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06b340 0x7f37ec06d800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3804009aa0 tx=0x7f3804009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.015+0000 7f0043400640 1 -- 172.21.15.26:0/852854463 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00340544f0 con 0x7f004c1536e0 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.021+0000 7fe2b1800640 1 Processor -- start 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.022+0000 7fe2b1800640 1 -- start start 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.024+0000 7fe2b1800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.024+0000 7fe2b1800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac1613e0 con 0x7fe2ac15fa90 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.024+0000 7fe2b1800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ac161630 con 0x7fe2ac160e80 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.024+0000 7fe2b1800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0b4b70 con 0x7fe2ac0d1cc0 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.024+0000 7fe2aaa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe2ac15fa90 0x7fe2ac0b4460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42976/0 (socket says 172.21.15.26:42976) 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.024+0000 7fe2aaa00640 1 -- 172.21.15.26:0/3190496226 learned_addr learned my addr 172.21.15.26:0/3190496226 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.025+0000 7fe2ab400640 1 --2- 172.21.15.26:0/3190496226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.025+0000 7fe2a9600640 1 -- 172.21.15.26:0/3190496226 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380524115 0 0) 0x7fe2ac1613e0 con 0x7fe2ac15fa90 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.025+0000 7fe2a9600640 1 -- 172.21.15.26:0/3190496226 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe28c003520 con 0x7fe2ac15fa90 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.026+0000 7fe2a9600640 1 -- 172.21.15.26:0/3190496226 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966799202 0 0) 0x7fe2ac161630 con 0x7fe2ac160e80 2024-01-15T17:55:12.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.026+0000 7fe2a9600640 1 -- 172.21.15.26:0/3190496226 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ac1613e0 con 0x7fe2ac160e80 2024-01-15T17:55:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.026+0000 7fe2ab400640 1 -- 172.21.15.26:0/3190496226 >> v1:172.21.15.124:6789/0 conn(0x7fe2ac160e80 legacy=0x7fe2ac0d15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.026+0000 7fe2ab400640 1 -- 172.21.15.26:0/3190496226 >> v1:172.21.15.26:6789/0 conn(0x7fe2ac15fa90 legacy=0x7fe2ac0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.026+0000 7fe2ab400640 1 -- 172.21.15.26:0/3190496226 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac161910 con 0x7fe2ac0d1cc0 2024-01-15T17:55:12.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.029+0000 7f380b400640 1 -- 172.21.15.26:0/2150599094 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f380c0776c0 con 0x7f381415d590 2024-01-15T17:55:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.029+0000 7fe2ab400640 1 --2- 172.21.15.26:0/3190496226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0cde90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009dc0 tx=0x7fe2a402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ebf757f3cbbb7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.030+0000 7fe2a9600640 1 -- 172.21.15.26:0/3190496226 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a402cce0 con 0x7fe2ac0d1cc0 2024-01-15T17:55:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.030+0000 7fe2a9600640 1 -- 172.21.15.26:0/3190496226 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a402ce40 con 0x7fe2ac0d1cc0 2024-01-15T17:55:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.030+0000 7fe2a9600640 1 -- 172.21.15.26:0/3190496226 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4034640 con 0x7fe2ac0d1cc0 2024-01-15T17:55:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.031+0000 7fe2b1800640 1 -- 172.21.15.26:0/3190496226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 msgr2=0x7fe2ac0cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.031+0000 7fe2b1800640 1 --2- 172.21.15.26:0/3190496226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0cde90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009dc0 tx=0x7fe2a402bf30 comp rx=0 tx=0).stop 2024-01-15T17:55:12.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.031+0000 7fe2b1800640 1 -- 172.21.15.26:0/3190496226 shutdown_connections 2024-01-15T17:55:12.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.031+0000 7fe2b1800640 1 --2- 172.21.15.26:0/3190496226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0cde90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.032+0000 7fe2b1800640 1 -- 172.21.15.26:0/3190496226 >> 172.21.15.26:0/3190496226 conn(0x7fe2ac09c540 msgr2=0x7fe2ac0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.032+0000 7fe2b1800640 1 -- 172.21.15.26:0/3190496226 shutdown_connections 2024-01-15T17:55:12.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.032+0000 7fe2b1800640 1 -- 172.21.15.26:0/3190496226 wait complete. 2024-01-15T17:55:12.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.035+0000 7fe2b1800640 1 Processor -- start 2024-01-15T17:55:12.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.035+0000 7fe2b1800640 1 -- start start 2024-01-15T17:55:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.037+0000 7fe2b1800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.037+0000 7fe2b1800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2ac15fa90 0x7fe2ac1ff400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.037+0000 7fe2b1800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2ac160e80 0x7fe2ac0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.038+0000 7fe2ab400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2ac160e80 0x7fe2ac0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.038+0000 7fe2aaa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.038+0000 7fe2aaa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:37238/0 (socket says 172.21.15.26:37238) 2024-01-15T17:55:12.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.038+0000 7fe2ab400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2ac160e80 0x7fe2ac0b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44450/0 (socket says 172.21.15.26:44450) 2024-01-15T17:55:12.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.038+0000 7fe2b1800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0cd370 con 0x7fe2ac160e80 2024-01-15T17:55:12.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.039+0000 7fe2b1800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0c9430 con 0x7fe2ac15fa90 2024-01-15T17:55:12.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.039+0000 7fe2b1800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ac0caef0 con 0x7fe2ac0d1cc0 2024-01-15T17:55:12.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.039+0000 7fe2aaa00640 1 -- 172.21.15.26:0/2650934196 learned_addr learned my addr 172.21.15.26:0/2650934196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.040+0000 7fe2ab400640 1 -- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 msgr2=0x7fe2ac0c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.040+0000 7fe2ab400640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.040+0000 7fe2ab400640 1 -- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2ac15fa90 msgr2=0x7fe2ac1ff400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-15T17:55:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.040+0000 7fe2ab400640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2ac15fa90 0x7fe2ac1ff400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.040+0000 7fe2ab400640 1 -- 172.21.15.26:0/2650934196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4009a70 con 0x7fe2ac160e80 2024-01-15T17:55:12.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.041+0000 7fe2ab400640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2ac160e80 0x7fe2ac0b2cd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4033000 tx=0x7fe2a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.041+0000 7fe2aaa00640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-01-15T17:55:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.042+0000 7fe2a2a00640 1 -- 172.21.15.26:0/2650934196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4007000 con 0x7fe2ac160e80 2024-01-15T17:55:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.042+0000 7fe2a2a00640 1 -- 172.21.15.26:0/2650934196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a403e630 con 0x7fe2ac160e80 2024-01-15T17:55:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.042+0000 7fe2a2a00640 1 -- 172.21.15.26:0/2650934196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a403e9e0 con 0x7fe2ac160e80 2024-01-15T17:55:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.042+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ac1ff940 con 0x7fe2ac160e80 2024-01-15T17:55:12.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.042+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ac0ca5c0 con 0x7fe2ac160e80 2024-01-15T17:55:12.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.047+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ac0675d0 con 0x7fe2ac160e80 2024-01-15T17:55:12.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.054+0000 7fe2a2a00640 1 -- 172.21.15.26:0/2650934196 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe2a403b050 con 0x7fe2ac160e80 2024-01-15T17:55:12.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.055+0000 7fe2a2a00640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe28406b2c0 0x7fe28406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:12.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.055+0000 7fe2a2a00640 1 -- 172.21.15.26:0/2650934196 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2a40ac430 con 0x7fe2ac160e80 2024-01-15T17:55:12.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.056+0000 7fe2aaa00640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe28406b2c0 0x7fe28406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:12.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.057+0000 7fe2aaa00640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe28406b2c0 0x7fe28406d780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0011e0 tx=0x7fe29c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:12.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.059+0000 7fe2a2a00640 1 -- 172.21.15.26:0/2650934196 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2a40773e0 con 0x7fe2ac160e80 2024-01-15T17:55:12.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.401+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9f380057d0 con 0x7f9f680cb0d0 2024-01-15T17:55:12.407 INFO:teuthology.orchestra.run.smithi026.stdout:60129542148 2024-01-15T17:55:12.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.402+0000 7f9f5ea00640 1 -- 172.21.15.26:0/1043916714 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f50020020 con 0x7f9f680cb0d0 2024-01-15T17:55:12.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.406+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f4006b310 msgr2=0x7f9f4006d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.406+0000 7f9f6dc00640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f4006b310 0x7f9f4006d7d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f6002bfb0 tx=0x7f9f60030000 comp rx=0 tx=0).stop 2024-01-15T17:55:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.406+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680cb0d0 msgr2=0x7f9f68157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.406+0000 7f9f6dc00640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680cb0d0 0x7f9f68157b10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000c370 tx=0x7f9f5000c830 comp rx=0 tx=0).stop 2024-01-15T17:55:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.407+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 shutdown_connections 2024-01-15T17:55:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.407+0000 7f9f6dc00640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f4006b310 0x7f9f4006d7d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.407+0000 7f9f6dc00640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f6815d5a0 0x7f9f68158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.407+0000 7f9f6dc00640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f680cb0d0 0x7f9f68157b10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.407+0000 7f9f6dc00640 1 --2- 172.21.15.26:0/1043916714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f680b16d0 0x7f9f68154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.407+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 >> 172.21.15.26:0/1043916714 conn(0x7f9f680b0be0 msgr2=0x7f9f68071460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.408+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 shutdown_connections 2024-01-15T17:55:12.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.408+0000 7f9f6dc00640 1 -- 172.21.15.26:0/1043916714 wait complete. 2024-01-15T17:55:12.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.415+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f250c057d30 con 0x7f250c03c060 2024-01-15T17:55:12.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.416+0000 7f2512000640 1 -- 172.21.15.26:0/4061708071 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2504057e40 con 0x7f250c03c060 2024-01-15T17:55:12.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.416+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7f98162b20 con 0x7f7f98070c50 2024-01-15T17:55:12.419 INFO:teuthology.orchestra.run.smithi026.stdout:60129542149 2024-01-15T17:55:12.420 INFO:teuthology.orchestra.run.smithi026.stdout:60129542149 2024-01-15T17:55:12.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.417+0000 7f7f8fe00640 1 -- 172.21.15.26:0/861987509 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f98162b20 con 0x7f7f98070c50 2024-01-15T17:55:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.420+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f24ec06b3c0 msgr2=0x7f24ec06d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.420+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f7006b310 msgr2=0x7f7f7006d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.420+0000 7f7f9d800640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f7006b310 0x7f7f7006d7d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f80036000 tx=0x7f7f8003c040 comp rx=0 tx=0).stop 2024-01-15T17:55:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.420+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f98070c50 msgr2=0x7f7f98160880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.420+0000 7f7f9d800640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f98070c50 0x7f7f98160880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000c730 tx=0x7f7f9000cbf0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.420+0000 7f251ca00640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f24ec06b3c0 0x7f24ec06d880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24fc036000 tx=0x7f24fc030000 comp rx=0 tx=0).stop 2024-01-15T17:55:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.420+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 msgr2=0x7f250c0de480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.420+0000 7f251ca00640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 0x7f250c0de480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f250400b460 tx=0x7f250400b920 comp rx=0 tx=0).stop 2024-01-15T17:55:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 shutdown_connections 2024-01-15T17:55:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f7f9d800640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f9815f1e0 0x7f7f980ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f7f9d800640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f7006b310 0x7f7f7006d7d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f7f9d800640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f980b2f10 0x7f7f981418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f7f9d800640 1 --2- 172.21.15.26:0/861987509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f98070c50 0x7f7f98160880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 >> 172.21.15.26:0/861987509 conn(0x7f7f980b0b00 msgr2=0x7f7f980afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 shutdown_connections 2024-01-15T17:55:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f251ca00640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f250c0e3120 0x7f250c04d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f251ca00640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f24ec06b3c0 0x7f24ec06d880 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f24fc036000 tx=0x7f24fc030000 comp rx=0 tx=0).stop 2024-01-15T17:55:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f251ca00640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f250c053c70 0x7f250c051d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f251ca00640 1 --2- 172.21.15.26:0/4061708071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f250c03c060 0x7f250c0de480 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.421+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 >> 172.21.15.26:0/4061708071 conn(0x7f250c058ea0 msgr2=0x7f250c0e6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.422+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 shutdown_connections 2024-01-15T17:55:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.422+0000 7f7f9d800640 1 -- 172.21.15.26:0/861987509 wait complete. 2024-01-15T17:55:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.422+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 shutdown_connections 2024-01-15T17:55:12.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.422+0000 7f251ca00640 1 -- 172.21.15.26:0/4061708071 wait complete. 2024-01-15T17:55:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.427+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f55e40cd360 con 0x7f55e40cd890 2024-01-15T17:55:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.428+0000 7f55da000640 1 -- 172.21.15.26:0/1491941518 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f55d40587b0 con 0x7f55e40cd890 2024-01-15T17:55:12.431 INFO:teuthology.orchestra.run.smithi026.stdout:60129542147 2024-01-15T17:55:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.434+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f55bc06b2f0 msgr2=0x7f55bc06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.434+0000 7f55e9200640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f55bc06b2f0 0x7f55bc06d7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f55cc006fd0 tx=0x7f55cc008040 comp rx=0 tx=0).stop 2024-01-15T17:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.434+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 msgr2=0x7f55e414c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.434+0000 7f55e9200640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 0x7f55e414c0c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f55d400c7f0 tx=0x7f55d400ccb0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.435+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 shutdown_connections 2024-01-15T17:55:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.435+0000 7f55e9200640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f55e415b140 0x7f55e41e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.435+0000 7f55e9200640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f55bc06b2f0 0x7f55bc06d7b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.435+0000 7f55e9200640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f55e40d1ac0 0x7f55e41e5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.435+0000 7f55e9200640 1 --2- 172.21.15.26:0/1491941518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f55e40cd890 0x7f55e414c0c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.435+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 >> 172.21.15.26:0/1491941518 conn(0x7f55e40d0ec0 msgr2=0x7f55e40c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.435+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 shutdown_connections 2024-01-15T17:55:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.435+0000 7f55e9200640 1 -- 172.21.15.26:0/1491941518 wait complete. 2024-01-15T17:55:12.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2024-01-15T17:55:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.437+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f001c005500 con 0x7f004c1536e0 2024-01-15T17:55:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.438+0000 7f0043400640 1 -- 172.21.15.26:0/852854463 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0034057b10 con 0x7f004c1536e0 2024-01-15T17:55:12.441 INFO:teuthology.orchestra.run.smithi026.stdout:60129542149 2024-01-15T17:55:12.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.439+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f88bc0710c0 con 0x7f88bc0cc730 2024-01-15T17:55:12.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.439+0000 7f88b2a00640 1 -- 172.21.15.26:0/547992365 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88b40588d0 con 0x7f88bc0cc730 2024-01-15T17:55:12.443 INFO:teuthology.orchestra.run.smithi026.stdout:60129542147 2024-01-15T17:55:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.441+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b3c0 msgr2=0x7f002406d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.441+0000 7f0052600640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b3c0 0x7f002406d880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f003c036000 tx=0x7f003c008030 comp rx=0 tx=0).stop 2024-01-15T17:55:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.441+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c1536e0 msgr2=0x7f004c1544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.441+0000 7f0052600640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c1536e0 0x7f004c1544a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f003400e810 tx=0x7f003400ecd0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.442+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 shutdown_connections 2024-01-15T17:55:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.442+0000 7f0052600640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b3c0 0x7f002406d880 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.442+0000 7f0052600640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c13b8d0 0x7f004c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.442+0000 7f0052600640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c1536e0 0x7f004c1544a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.442+0000 7f0052600640 1 --2- 172.21.15.26:0/852854463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0b16d0 0x7f004c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.442+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 >> 172.21.15.26:0/852854463 conn(0x7f004c0b0be0 msgr2=0x7f004c0702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.442+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 shutdown_connections 2024-01-15T17:55:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.442+0000 7f0052600640 1 -- 172.21.15.26:0/852854463 wait complete. 2024-01-15T17:55:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f889406b390 msgr2=0x7f889406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f889406b390 0x7f889406d850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f88a4034000 tx=0x7f88a4030000 comp rx=0 tx=0).stop 2024-01-15T17:55:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88bc0cc730 msgr2=0x7f88bc156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88bc0cc730 0x7f88bc156290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0aeb00 tx=0x7f88b400b480 comp rx=0 tx=0).stop 2024-01-15T17:55:12.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.0 2024-01-15T17:55:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 shutdown_connections 2024-01-15T17:55:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88bc0cd2f0 0x7f88bc0cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f889406b390 0x7f889406d850 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88bc0cc730 0x7f88bc156290 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 --2- 172.21.15.26:0/547992365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88bc14ce10 0x7f88bc14d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 >> 172.21.15.26:0/547992365 conn(0x7f88bc0b0be0 msgr2=0x7f88bc0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 shutdown_connections 2024-01-15T17:55:12.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.444+0000 7f88c1800640 1 -- 172.21.15.26:0/547992365 wait complete. 2024-01-15T17:55:12.454 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.2 2024-01-15T17:55:12.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.451+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f38140675d0 con 0x7f381415d590 2024-01-15T17:55:12.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.452+0000 7f380b400640 1 -- 172.21.15.26:0/2150599094 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f380c07ace0 con 0x7f381415d590 2024-01-15T17:55:12.455 INFO:teuthology.orchestra.run.smithi026.stdout:60129542148 2024-01-15T17:55:12.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06b340 msgr2=0x7f37ec06d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06b340 0x7f37ec06d800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3804009aa0 tx=0x7f3804009190 comp rx=0 tx=0).stop 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 msgr2=0x7f38141f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38141f6ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e50 tx=0x7f380c004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 shutdown_connections 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381415d590 0x7f38141f6ea0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06b340 0x7f37ec06d800 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140d00b0 0x7f381415f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 --2- 172.21.15.26:0/2150599094 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38140ce8c0 0x7f38140cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.456+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 >> 172.21.15.26:0/2150599094 conn(0x7f381409c540 msgr2=0x7f38140b7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.457+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 shutdown_connections 2024-01-15T17:55:12.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.457+0000 7f3819e00640 1 -- 172.21.15.26:0/2150599094 wait complete. 2024-01-15T17:55:12.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.5 2024-01-15T17:55:12.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2024-01-15T17:55:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.467+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe2ac0cb030 con 0x7fe2ac160e80 2024-01-15T17:55:12.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.467+0000 7fe2a2a00640 1 -- 172.21.15.26:0/2650934196 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe2a4009450 con 0x7fe2ac160e80 2024-01-15T17:55:12.470 INFO:teuthology.orchestra.run.smithi026.stdout:60129542147 2024-01-15T17:55:12.471 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.469+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe28406b2c0 msgr2=0x7fe28406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.469+0000 7fe2b1800640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe28406b2c0 0x7fe28406d780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe29c0011e0 tx=0x7fe29c009040 comp rx=0 tx=0).stop 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2ac160e80 msgr2=0x7fe2ac0b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2ac160e80 0x7fe2ac0b2cd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4033000 tx=0x7fe2a4004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 shutdown_connections 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe28406b2c0 0x7fe28406d780 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2ac160e80 0x7fe2ac0b2cd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2ac15fa90 0x7fe2ac1ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 --2- 172.21.15.26:0/2650934196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2ac0d1cc0 0x7fe2ac0c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 >> 172.21.15.26:0/2650934196 conn(0x7fe2ac09c540 msgr2=0x7fe2ac0cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 shutdown_connections 2024-01-15T17:55:12.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:12.470+0000 7fe2b1800640 1 -- 172.21.15.26:0/2650934196 wait complete. 2024-01-15T17:55:12.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.7 2024-01-15T17:55:12.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.6 2024-01-15T17:55:13.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-15T17:55:13.448 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-15T17:55:13.455 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-15T17:55:13.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-15T17:55:13.468 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-15T17:55:13.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-15T17:55:13.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-15T17:55:13.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-15T17:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.737+0000 7f6065800640 1 Processor -- start 2024-01-15T17:55:13.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.737+0000 7f6065800640 1 -- start start 2024-01-15T17:55:13.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.739+0000 7f6065800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f606006c340 0x7f606006c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.740+0000 7f6065800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606006d0a0 con 0x7f60600b0530 2024-01-15T17:55:13.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.740+0000 7f6065800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606006d380 con 0x7f60600b3eb0 2024-01-15T17:55:13.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.740+0000 7f6065800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6060160040 con 0x7f606006c340 2024-01-15T17:55:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.741+0000 7f605e000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f606006c340 0x7f606006c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.741+0000 7f605f400640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f60600b3eb0 0x7f60600cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:34966/0 (socket says 172.21.15.26:34966) 2024-01-15T17:55:13.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.741+0000 7f605f400640 1 -- 172.21.15.26:0/729107730 learned_addr learned my addr 172.21.15.26:0/729107730 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:13.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.742+0000 7f605e000640 1 -- 172.21.15.26:0/729107730 >> v1:172.21.15.124:6789/0 conn(0x7f60600b3eb0 legacy=0x7f60600cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:13.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.742+0000 7f605e000640 1 -- 172.21.15.26:0/729107730 >> v1:172.21.15.26:6789/0 conn(0x7f60600b0530 legacy=0x7f606015a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:13.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.742+0000 7f605e000640 1 -- 172.21.15.26:0/729107730 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606006d660 con 0x7f606006c340 2024-01-15T17:55:13.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.744+0000 7f605e000640 1 --2- 172.21.15.26:0/729107730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f606006c340 0x7f606006c7c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60500097a0 tx=0x7f605002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1696f24dc835a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.745+0000 7f605d600640 1 -- 172.21.15.26:0/729107730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050031830 con 0x7f606006c340 2024-01-15T17:55:13.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.746+0000 7f605d600640 1 -- 172.21.15.26:0/729107730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6050031990 con 0x7f606006c340 2024-01-15T17:55:13.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.746+0000 7f605d600640 1 -- 172.21.15.26:0/729107730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050031d20 con 0x7f606006c340 2024-01-15T17:55:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.747+0000 7f6065800640 1 -- 172.21.15.26:0/729107730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f606006c340 msgr2=0x7f606006c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.747+0000 7f6065800640 1 --2- 172.21.15.26:0/729107730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f606006c340 0x7f606006c7c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f60500097a0 tx=0x7f605002b8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.748+0000 7f6065800640 1 -- 172.21.15.26:0/729107730 shutdown_connections 2024-01-15T17:55:13.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.748+0000 7f6065800640 1 --2- 172.21.15.26:0/729107730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f606006c340 0x7f606006c7c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.748+0000 7f6065800640 1 -- 172.21.15.26:0/729107730 >> 172.21.15.26:0/729107730 conn(0x7f60600aea00 msgr2=0x7f60600aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:13.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.748+0000 7f6065800640 1 -- 172.21.15.26:0/729107730 shutdown_connections 2024-01-15T17:55:13.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.748+0000 7f6065800640 1 -- 172.21.15.26:0/729107730 wait complete. 2024-01-15T17:55:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.752+0000 7f6065800640 1 Processor -- start 2024-01-15T17:55:13.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.752+0000 7f6065800640 1 -- start start 2024-01-15T17:55:13.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.755+0000 7f6065800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606006c340 0x7f6060135440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.755+0000 7f6065800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60600b0530 0x7f6060135980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.755+0000 7f6065800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60600b3eb0 0x7f60601f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.756+0000 7f6065800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60601f8eb0 con 0x7f60600b3eb0 2024-01-15T17:55:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.756+0000 7f6065800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60601f8ba0 con 0x7f60600b0530 2024-01-15T17:55:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.756+0000 7f6065800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60600cdb90 con 0x7f606006c340 2024-01-15T17:55:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.756+0000 7f605e000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60600b0530 0x7f6060135980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.756+0000 7f605f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60600b3eb0 0x7f60601f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.756+0000 7f605f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60600b3eb0 0x7f60601f92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44476/0 (socket says 172.21.15.26:44476) 2024-01-15T17:55:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.756+0000 7f605f400640 1 -- 172.21.15.26:0/1568368840 learned_addr learned my addr 172.21.15.26:0/1568368840 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.757+0000 7f605f400640 1 -- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606006c340 msgr2=0x7f6060135440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.757+0000 7f605ea00640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606006c340 0x7f6060135440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.757+0000 7f605f400640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606006c340 0x7f6060135440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.757+0000 7f605f400640 1 -- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60600b0530 msgr2=0x7f6060135980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.757+0000 7f605f400640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60600b0530 0x7f6060135980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.758+0000 7f605f400640 1 -- 172.21.15.26:0/1568368840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6050009400 con 0x7f60600b3eb0 2024-01-15T17:55:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.758+0000 7f605e000640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60600b0530 0x7f6060135980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:13.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.758+0000 7f605ea00640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606006c340 0x7f6060135440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:13.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.759+0000 7f605f400640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60600b3eb0 0x7f60601f92d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f604800b410 tx=0x7f6048009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.759+0000 7f6056a00640 1 -- 172.21.15.26:0/1568368840 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048011830 con 0x7f60600b3eb0 2024-01-15T17:55:13.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.760+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60600cde70 con 0x7f60600b3eb0 2024-01-15T17:55:13.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.760+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60601567f0 con 0x7f60600b3eb0 2024-01-15T17:55:13.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.760+0000 7f6056a00640 1 -- 172.21.15.26:0/1568368840 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6048011990 con 0x7f60600b3eb0 2024-01-15T17:55:13.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.760+0000 7f6056a00640 1 -- 172.21.15.26:0/1568368840 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048011830 con 0x7f60600b3eb0 2024-01-15T17:55:13.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.763+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6030005250 con 0x7f60600b3eb0 2024-01-15T17:55:13.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.765+0000 7f6056a00640 1 -- 172.21.15.26:0/1568368840 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6048011b40 con 0x7f60600b3eb0 2024-01-15T17:55:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.774+0000 7f6056a00640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603806b320 0x7f603806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.774+0000 7f6056a00640 1 -- 172.21.15.26:0/1568368840 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60480899a0 con 0x7f60600b3eb0 2024-01-15T17:55:13.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.775+0000 7f605ea00640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603806b320 0x7f603806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.777+0000 7f605ea00640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603806b320 0x7f603806d7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6058001dc0 tx=0x7f6058009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.778+0000 7f6056a00640 1 -- 172.21.15.26:0/1568368840 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6048054770 con 0x7f60600b3eb0 2024-01-15T17:55:13.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.784+0000 7fe347000640 1 Processor -- start 2024-01-15T17:55:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.784+0000 7fe347000640 1 -- start start 2024-01-15T17:55:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.787+0000 7fe347000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe338054390 0x7fe3380e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.787+0000 7fe347000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe338007b40 con 0x7fe33803c070 2024-01-15T17:55:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.787+0000 7fe347000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3380587e0 con 0x7fe3380e3130 2024-01-15T17:55:13.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.787+0000 7fe347000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe338058ac0 con 0x7fe338054390 2024-01-15T17:55:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.787+0000 7f4991000640 1 Processor -- start 2024-01-15T17:55:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.787+0000 7f4991000640 1 -- start start 2024-01-15T17:55:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.788+0000 7fe345c00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fe3380e3130 0x7fe338058020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:34974/0 (socket says 172.21.15.26:34974) 2024-01-15T17:55:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.788+0000 7fe345c00640 1 -- 172.21.15.26:0/227152050 learned_addr learned my addr 172.21.15.26:0/227152050 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:13.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.788+0000 7fe346600640 1 --2- 172.21.15.26:0/227152050 >> v2:172.21.15.26:3300/0 conn(0x7fe338054390 0x7fe3380e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.789+0000 7f4991000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f498c160920 0x7f498c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.789+0000 7f4991000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f498c0cf9f0 0x7f498c0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.789+0000 7f4991000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c161010 0x7f498c1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.789+0000 7f4991000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c0d03e0 con 0x7f498c0cf9f0 2024-01-15T17:55:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.789+0000 7f4991000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c0d0a10 con 0x7f498c160920 2024-01-15T17:55:13.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.789+0000 7f4991000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0cf080 con 0x7f498c161010 2024-01-15T17:55:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.789+0000 7fe33fe00640 1 -- 172.21.15.26:0/227152050 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917881306 0 0) 0x7fe3380587e0 con 0x7fe3380e3130 2024-01-15T17:55:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.790+0000 7fe33fe00640 1 -- 172.21.15.26:0/227152050 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe320003520 con 0x7fe3380e3130 2024-01-15T17:55:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.790+0000 7fe33fe00640 1 -- 172.21.15.26:0/227152050 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921135350 0 0) 0x7fe338007b40 con 0x7fe33803c070 2024-01-15T17:55:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.790+0000 7fe33fe00640 1 -- 172.21.15.26:0/227152050 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3380587e0 con 0x7fe33803c070 2024-01-15T17:55:13.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.790+0000 7f498b400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f498c160920 0x7f498c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.790+0000 7f4983e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f498c0cf9f0 0x7f498c0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.790+0000 7f498b400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f498c160920 0x7f498c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58894/0 (socket says 172.21.15.26:58894) 2024-01-15T17:55:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.790+0000 7f498b400640 1 -- 172.21.15.26:0/3780145375 learned_addr learned my addr 172.21.15.26:0/3780145375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.790+0000 7fe33fe00640 1 -- 172.21.15.26:0/227152050 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3477812564 0 0) 0x7fe3380587e0 con 0x7fe33803c070 2024-01-15T17:55:13.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.790+0000 7f498be00640 1 --2- 172.21.15.26:0/3780145375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c161010 0x7f498c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.791+0000 7fe346600640 1 -- 172.21.15.26:0/227152050 >> v1:172.21.15.124:6789/0 conn(0x7fe3380e3130 legacy=0x7fe338058020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.791+0000 7fe346600640 1 -- 172.21.15.26:0/227152050 >> v1:172.21.15.26:6789/0 conn(0x7fe33803c070 legacy=0x7fe338053c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.791+0000 7f498b400640 1 -- 172.21.15.26:0/3780145375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c161010 msgr2=0x7f498c1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.791+0000 7f498b400640 1 --2- 172.21.15.26:0/3780145375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c161010 0x7f498c1614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.791+0000 7f498b400640 1 -- 172.21.15.26:0/3780145375 >> v2:172.21.15.26:3300/0 conn(0x7f498c0cf9f0 msgr2=0x7f498c0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.791+0000 7f498b400640 1 --2- 172.21.15.26:0/3780145375 >> v2:172.21.15.26:3300/0 conn(0x7f498c0cf9f0 0x7f498c0cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.791+0000 7f498b400640 1 -- 172.21.15.26:0/3780145375 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0c6be0 con 0x7f498c160920 2024-01-15T17:55:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.791+0000 7f4983e00640 1 --2- 172.21.15.26:0/3780145375 >> v2:172.21.15.26:3300/0 conn(0x7f498c0cf9f0 0x7f498c0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:55:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.792+0000 7fe346600640 1 -- 172.21.15.26:0/227152050 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe338006330 con 0x7fe338054390 2024-01-15T17:55:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.793+0000 7f498b400640 1 --2- 172.21.15.26:0/3780145375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f498c160920 0x7f498c0b3f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f498c08c060 tx=0x7f497c02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6879e300942e86 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.794+0000 7f498aa00640 1 -- 172.21.15.26:0/3780145375 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c030800 con 0x7f498c160920 2024-01-15T17:55:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.794+0000 7fe346600640 1 --2- 172.21.15.26:0/227152050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe338054390 0x7fe3380e7750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097a0 tx=0x7fe33002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafb2df6152339be server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.794+0000 7f498aa00640 1 -- 172.21.15.26:0/3780145375 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c030960 con 0x7f498c160920 2024-01-15T17:55:13.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.794+0000 7fe33fe00640 1 -- 172.21.15.26:0/227152050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33002f050 con 0x7fe338054390 2024-01-15T17:55:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.794+0000 7fe33fe00640 1 -- 172.21.15.26:0/227152050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe330031500 con 0x7fe338054390 2024-01-15T17:55:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.794+0000 7fe33fe00640 1 -- 172.21.15.26:0/227152050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328001520 con 0x7fe338054390 2024-01-15T17:55:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.795+0000 7f4991000640 1 -- 172.21.15.26:0/3780145375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f498c160920 msgr2=0x7f498c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.795+0000 7f4991000640 1 --2- 172.21.15.26:0/3780145375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f498c160920 0x7f498c0b3f30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f498c08c060 tx=0x7f497c02b940 comp rx=0 tx=0).stop 2024-01-15T17:55:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.795+0000 7fe347000640 1 -- 172.21.15.26:0/227152050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe338054390 msgr2=0x7fe3380e7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.795+0000 7fe347000640 1 --2- 172.21.15.26:0/227152050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe338054390 0x7fe3380e7750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097a0 tx=0x7fe33002b820 comp rx=0 tx=0).stop 2024-01-15T17:55:13.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.795+0000 7f4991000640 1 -- 172.21.15.26:0/3780145375 shutdown_connections 2024-01-15T17:55:13.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.795+0000 7f4991000640 1 --2- 172.21.15.26:0/3780145375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c161010 0x7f498c1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.795+0000 7f4991000640 1 --2- 172.21.15.26:0/3780145375 >> v2:172.21.15.26:3300/0 conn(0x7f498c0cf9f0 0x7f498c0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.795+0000 7f4991000640 1 --2- 172.21.15.26:0/3780145375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f498c160920 0x7f498c0b3f30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.795+0000 7f4991000640 1 -- 172.21.15.26:0/3780145375 >> 172.21.15.26:0/3780145375 conn(0x7f498c0d3090 msgr2=0x7f498c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:13.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.796+0000 7fe347000640 1 -- 172.21.15.26:0/227152050 shutdown_connections 2024-01-15T17:55:13.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.796+0000 7fe347000640 1 --2- 172.21.15.26:0/227152050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe338054390 0x7fe3380e7750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.796+0000 7fe347000640 1 -- 172.21.15.26:0/227152050 >> 172.21.15.26:0/227152050 conn(0x7fe338058eb0 msgr2=0x7fe338059290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:13.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.796+0000 7f4991000640 1 -- 172.21.15.26:0/3780145375 shutdown_connections 2024-01-15T17:55:13.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.796+0000 7f7dd4e00640 1 Processor -- start 2024-01-15T17:55:13.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.796+0000 7fe347000640 1 -- 172.21.15.26:0/227152050 shutdown_connections 2024-01-15T17:55:13.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.796+0000 7f4991000640 1 -- 172.21.15.26:0/3780145375 wait complete. 2024-01-15T17:55:13.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.797+0000 7f7dd4e00640 1 -- start start 2024-01-15T17:55:13.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.796+0000 7fe347000640 1 -- 172.21.15.26:0/227152050 wait complete. 2024-01-15T17:55:13.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.799+0000 7f7dd4e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 0x7f7dd00d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.799+0000 7f7dd4e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7dd00b3890 0x7f7dd00b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.799+0000 7f7dd4e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd00d2180 con 0x7f7dd015d610 2024-01-15T17:55:13.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.799+0000 7f7dd4e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd00d1390 con 0x7f7dd00b3890 2024-01-15T17:55:13.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.799+0000 7f7dd4e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00b3400 con 0x7f7dd00d17e0 2024-01-15T17:55:13.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.799+0000 7fe347000640 1 Processor -- start 2024-01-15T17:55:13.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.800+0000 7fe347000640 1 -- start start 2024-01-15T17:55:13.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.800+0000 7f7dcde00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7dd015d610 0x7f7dd00ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43004/0 (socket says 172.21.15.26:43004) 2024-01-15T17:55:13.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.800+0000 7f7dcde00640 1 -- 172.21.15.26:0/4013867913 learned_addr learned my addr 172.21.15.26:0/4013867913 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:13.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.800+0000 7f7dcd400640 1 --2- 172.21.15.26:0/4013867913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 0x7f7dd00d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.800+0000 7f7dce800640 1 --2- 172.21.15.26:0/4013867913 >> v2:172.21.15.26:3300/0 conn(0x7f7dd00b3890 0x7f7dd00b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.799+0000 7f4991000640 1 Processor -- start 2024-01-15T17:55:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.801+0000 7f4991000640 1 -- start start 2024-01-15T17:55:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.801+0000 7f7dcd400640 1 -- 172.21.15.26:0/4013867913 >> v2:172.21.15.26:3300/0 conn(0x7f7dd00b3890 msgr2=0x7f7dd00b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.801+0000 7fe347000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe33803c070 0x7fe3380e16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.801+0000 7fe347000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe338054390 0x7fe338052150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.801+0000 7f8e54a00640 1 Processor -- start 2024-01-15T17:55:13.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.802+0000 7fe347000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3380e3130 0x7fe338052690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.802+0000 7f8e54a00640 1 -- start start 2024-01-15T17:55:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.802+0000 7fe347000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380e1bf0 con 0x7fe33803c070 2024-01-15T17:55:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.802+0000 7fe345200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe338054390 0x7fe338052150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.802+0000 7fe345200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe338054390 0x7fe338052150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58904/0 (socket says 172.21.15.26:58904) 2024-01-15T17:55:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.802+0000 7fe345200640 1 -- 172.21.15.26:0/2420352365 learned_addr learned my addr 172.21.15.26:0/2420352365 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.802+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3381859c0 con 0x7fe338054390 2024-01-15T17:55:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.802+0000 7fe345c00640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe33803c070 0x7fe3380e16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.802+0000 7fe346600640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3380e3130 0x7fe338052690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.803+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380e2800 con 0x7fe3380e3130 2024-01-15T17:55:13.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.801+0000 7f7dcd400640 1 --2- 172.21.15.26:0/4013867913 >> v2:172.21.15.26:3300/0 conn(0x7f7dd00b3890 0x7f7dd00b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.801+0000 7f7dcd400640 1 -- 172.21.15.26:0/4013867913 >> v1:172.21.15.26:6789/0 conn(0x7f7dd015d610 legacy=0x7f7dd00ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:13.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.801+0000 7f7dcd400640 1 -- 172.21.15.26:0/4013867913 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd00b27f0 con 0x7f7dd00d17e0 2024-01-15T17:55:13.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.803+0000 7f7dcd400640 1 --2- 172.21.15.26:0/4013867913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 0x7f7dd00d1c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009d90 tx=0x7f7dc002be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e1442e6303e740 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.803+0000 7fe345c00640 1 -- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3380e3130 msgr2=0x7fe338052690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.803+0000 7fe345c00640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3380e3130 0x7fe338052690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.803+0000 7fe345c00640 1 -- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe338054390 msgr2=0x7fe338052150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.803+0000 7fe345c00640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe338054390 0x7fe338052150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.803+0000 7fe345c00640 1 -- 172.21.15.26:0/2420352365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe330009400 con 0x7fe33803c070 2024-01-15T17:55:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f4991000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c0cf9f0 0x7f498c1fa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f4991000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c160920 0x7f498c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7fe345c00640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe33803c070 0x7fe3380e16b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe34002cee0 tx=0x7fe3400748f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7fe33d600640 1 -- 172.21.15.26:0/2420352365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34008e070 con 0x7fe33803c070 2024-01-15T17:55:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe338185c80 con 0x7fe33803c070 2024-01-15T17:55:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe338186240 con 0x7fe33803c070 2024-01-15T17:55:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f4991000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f498c161010 0x7f498c157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f4991000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0ca200 con 0x7f498c0cf9f0 2024-01-15T17:55:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f4991000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c1faf00 con 0x7f498c161010 2024-01-15T17:55:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f4991000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c1fb1e0 con 0x7f498c160920 2024-01-15T17:55:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f8e54a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8e4c0b3ae0 0x7f8e4c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f8e54a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c070c40 con 0x7f8e4c0cf290 2024-01-15T17:55:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f8e54a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4c06c470 con 0x7f8e4c0ae5a0 2024-01-15T17:55:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f8e54a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4c071450 con 0x7f8e4c0b3ae0 2024-01-15T17:55:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.804+0000 7f7dcca00640 1 -- 172.21.15.26:0/4013867913 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc002cbd0 con 0x7f7dd00d17e0 2024-01-15T17:55:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7f7dcca00640 1 -- 172.21.15.26:0/4013867913 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc002cd30 con 0x7f7dd00d17e0 2024-01-15T17:55:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7f7dcca00640 1 -- 172.21.15.26:0/4013867913 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc002cea0 con 0x7f7dd00d17e0 2024-01-15T17:55:13.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7f4983e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c160920 0x7f498c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7f4983e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c160920 0x7f498c1545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:37268/0 (socket says 172.21.15.26:37268) 2024-01-15T17:55:13.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7f4983e00640 1 -- 172.21.15.26:0/1999853912 learned_addr learned my addr 172.21.15.26:0/1999853912 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7f498be00640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f498c161010 0x7f498c157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7f8e52a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8e4c0cf290 0x7f8e4c0b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43008/0 (socket says 172.21.15.26:43008) 2024-01-15T17:55:13.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7f8e52a00640 1 -- 172.21.15.26:0/1905070389 learned_addr learned my addr 172.21.15.26:0/1905070389 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:13.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.806+0000 7f498b400640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c0cf9f0 0x7f498c1fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7fe33d600640 1 -- 172.21.15.26:0/2420352365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe340074a20 con 0x7fe33803c070 2024-01-15T17:55:13.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.805+0000 7fe33d600640 1 -- 172.21.15.26:0/2420352365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34006bc00 con 0x7fe33803c070 2024-01-15T17:55:13.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.806+0000 7f7dd4e00640 1 -- 172.21.15.26:0/4013867913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 msgr2=0x7f7dd00d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.806+0000 7f7dd4e00640 1 --2- 172.21.15.26:0/4013867913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 0x7f7dd00d1c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0009d90 tx=0x7f7dc002be20 comp rx=0 tx=0).stop 2024-01-15T17:55:13.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.806+0000 7f8e51600640 1 --2- 172.21.15.26:0/1905070389 >> v2:172.21.15.124:3300/0 conn(0x7f8e4c0b3ae0 0x7f8e4c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe310005250 con 0x7fe33803c070 2024-01-15T17:55:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f7dd4e00640 1 -- 172.21.15.26:0/4013867913 shutdown_connections 2024-01-15T17:55:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f7dd4e00640 1 --2- 172.21.15.26:0/4013867913 >> v2:172.21.15.26:3300/0 conn(0x7f7dd00b3890 0x7f7dd00b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f7dd4e00640 1 --2- 172.21.15.26:0/4013867913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 0x7f7dd00d1c40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f7dd4e00640 1 -- 172.21.15.26:0/4013867913 >> 172.21.15.26:0/4013867913 conn(0x7f7dd009c540 msgr2=0x7f7dd00d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f7dd4e00640 1 -- 172.21.15.26:0/4013867913 shutdown_connections 2024-01-15T17:55:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f8e50c00640 1 -- 172.21.15.26:0/1905070389 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812428021 0 0) 0x7f8e4c06c470 con 0x7f8e4c0ae5a0 2024-01-15T17:55:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f8e50c00640 1 -- 172.21.15.26:0/1905070389 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e2c003520 con 0x7f8e4c0ae5a0 2024-01-15T17:55:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f8e51600640 1 -- 172.21.15.26:0/1905070389 >> v1:172.21.15.124:6789/0 conn(0x7f8e4c0ae5a0 legacy=0x7f8e4c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f8e51600640 1 -- 172.21.15.26:0/1905070389 >> v1:172.21.15.26:6789/0 conn(0x7f8e4c0cf290 legacy=0x7f8e4c0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f8e51600640 1 -- 172.21.15.26:0/1905070389 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c06c750 con 0x7f8e4c0b3ae0 2024-01-15T17:55:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f4983e00640 1 -- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f498c161010 msgr2=0x7f498c157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f4983e00640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f498c161010 0x7f498c157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f4983e00640 1 -- 172.21.15.26:0/1999853912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c0cf9f0 msgr2=0x7f498c1fa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f4983e00640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c0cf9f0 0x7f498c1fa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f4983e00640 1 -- 172.21.15.26:0/1999853912 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4974009400 con 0x7f498c160920 2024-01-15T17:55:13.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.808+0000 7f4983e00640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c160920 0x7f498c1545b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f497400e8e0 tx=0x7f497400eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.809+0000 7fe33d600640 1 -- 172.21.15.26:0/2420352365 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe3400721c0 con 0x7fe33803c070 2024-01-15T17:55:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.807+0000 7f7dd4e00640 1 -- 172.21.15.26:0/4013867913 wait complete. 2024-01-15T17:55:13.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.809+0000 7fdc66200640 1 Processor -- start 2024-01-15T17:55:13.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.810+0000 7f7dd4e00640 1 Processor -- start 2024-01-15T17:55:13.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.810+0000 7f7dd4e00640 1 -- start start 2024-01-15T17:55:13.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.810+0000 7fdc66200640 1 -- start start 2024-01-15T17:55:13.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.809+0000 7f8e51600640 1 --2- 172.21.15.26:0/1905070389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e4c0b3ae0 0x7f8e4c15e430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e340097a0 tx=0x7f8e3402ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75df9d98004a1717 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.810+0000 7f4e9b200640 1 Processor -- start 2024-01-15T17:55:13.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.810+0000 7f8e50c00640 1 -- 172.21.15.26:0/1905070389 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3402f830 con 0x7f8e4c0b3ae0 2024-01-15T17:55:13.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.811+0000 7f8e50c00640 1 -- 172.21.15.26:0/1905070389 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3402f990 con 0x7f8e4c0b3ae0 2024-01-15T17:55:13.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.811+0000 7f8e50c00640 1 -- 172.21.15.26:0/1905070389 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3402fd20 con 0x7f8e4c0b3ae0 2024-01-15T17:55:13.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.811+0000 7f4e9b200640 1 -- start start 2024-01-15T17:55:13.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.811+0000 7fe33d600640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe31806b2c0 0x7fe31806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.811+0000 7fe33d600640 1 -- 172.21.15.26:0/2420352365 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe340103340 con 0x7fe33803c070 2024-01-15T17:55:13.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.812+0000 7f8e54a00640 1 -- 172.21.15.26:0/1905070389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e4c0b3ae0 msgr2=0x7f8e4c15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.812+0000 7f8e54a00640 1 --2- 172.21.15.26:0/1905070389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e4c0b3ae0 0x7f8e4c15e430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e340097a0 tx=0x7f8e3402ba10 comp rx=0 tx=0).stop 2024-01-15T17:55:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.812+0000 7fe345200640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe31806b2c0 0x7fe31806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.812+0000 7fdc66200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdc601536e0 0x7fdc6015ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.812+0000 7fdc66200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc60153b70 0x7fdc6015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.812+0000 7f7dd4e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dd00b3890 0x7f7dd015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.812+0000 7fdc66200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc600b2e50 con 0x7fdc600b16d0 2024-01-15T17:55:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.812+0000 7fdc66200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc600b3130 con 0x7fdc601536e0 2024-01-15T17:55:13.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4983400640 1 -- 172.21.15.26:0/1999853912 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400c940 con 0x7f498c160920 2024-01-15T17:55:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f7dd4e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 0x7f7dd015c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f7dd4e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dd015d610 0x7f7dd0203690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.812+0000 7fdc5f400640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fdc600b16d0 0x7fdc600cb0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:35004/0 (socket says 172.21.15.26:35004) 2024-01-15T17:55:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7fdc5f400640 1 -- 172.21.15.26:0/677339146 learned_addr learned my addr 172.21.15.26:0/677339146 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7fdc66200640 1 -- 172.21.15.26:0/677339146 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600b3410 con 0x7fdc60153b70 2024-01-15T17:55:13.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4e9b200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4e940d0280 0x7f4e940ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4e9b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e940ce6f0 0x7f4e940b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4e9b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e940b6750 con 0x7f4e9415d590 2024-01-15T17:55:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4983400640 1 -- 172.21.15.26:0/1999853912 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497400caa0 con 0x7f498c160920 2024-01-15T17:55:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4983400640 1 -- 172.21.15.26:0/1999853912 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497400ce30 con 0x7f498c160920 2024-01-15T17:55:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4e9b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e940cf1f0 con 0x7f4e940ce6f0 2024-01-15T17:55:13.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4e9b200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e940cef10 con 0x7f4e940d0280 2024-01-15T17:55:13.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7fdc5ea00640 1 --2- 172.21.15.26:0/677339146 >> v2:172.21.15.124:3300/0 conn(0x7fdc601536e0 0x7fdc6015ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7fdc5fe00640 1 --2- 172.21.15.26:0/677339146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc60153b70 0x7fdc6015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4e99200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e940ce6f0 0x7f4e940b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4e93400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4e940d0280 0x7f4e940ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4e99200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e940ce6f0 0x7f4e940b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44520/0 (socket says 172.21.15.26:44520) 2024-01-15T17:55:13.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f4e99200640 1 -- 172.21.15.26:0/770945984 learned_addr learned my addr 172.21.15.26:0/770945984 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.813+0000 7f7dcde00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dd00b3890 0x7f7dd015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f7dcde00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dd00b3890 0x7f7dd015bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58936/0 (socket says 172.21.15.26:58936) 2024-01-15T17:55:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7fe345200640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe31806b2c0 0x7fe31806d780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe33804fdb0 tx=0x7fe3280019e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f7dcde00640 1 -- 172.21.15.26:0/3448489425 learned_addr learned my addr 172.21.15.26:0/3448489425 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7fdc5ea00640 1 -- 172.21.15.26:0/677339146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc60153b70 msgr2=0x7fdc6015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7fdc5ea00640 1 --2- 172.21.15.26:0/677339146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc60153b70 0x7fdc6015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7fdc5e000640 1 -- 172.21.15.26:0/677339146 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342415395 0 0) 0x7fdc600b2e50 con 0x7fdc600b16d0 2024-01-15T17:55:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7fdc5ea00640 1 -- 172.21.15.26:0/677339146 >> v1:172.21.15.124:6789/0 conn(0x7fdc600b16d0 legacy=0x7fdc600cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7fdc5ea00640 1 -- 172.21.15.26:0/677339146 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc600b36f0 con 0x7fdc601536e0 2024-01-15T17:55:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f01e6800640 1 Processor -- start 2024-01-15T17:55:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00b6050 con 0x7f7dd015d610 2024-01-15T17:55:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00b6330 con 0x7f7dd00b3890 2024-01-15T17:55:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd00cb360 con 0x7f7dd00d17e0 2024-01-15T17:55:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f7dcd400640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 0x7f7dd015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f7dce800640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dd015d610 0x7f7dd0203690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c0094d0 con 0x7f498c160920 2024-01-15T17:55:13.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c154af0 con 0x7f498c160920 2024-01-15T17:55:13.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.814+0000 7f4e99200640 1 -- 172.21.15.26:0/770945984 >> v2:172.21.15.124:3300/0 conn(0x7f4e940d0280 msgr2=0x7f4e940ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:13.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f4e99200640 1 --2- 172.21.15.26:0/770945984 >> v2:172.21.15.124:3300/0 conn(0x7f4e940d0280 0x7f4e940ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f4e99200640 1 -- 172.21.15.26:0/770945984 >> v1:172.21.15.26:6789/0 conn(0x7f4e9415d590 legacy=0x7f4e940cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f8e54a00640 1 -- 172.21.15.26:0/1905070389 shutdown_connections 2024-01-15T17:55:13.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f8e54a00640 1 --2- 172.21.15.26:0/1905070389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e4c0b3ae0 0x7f8e4c15e430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f8e54a00640 1 -- 172.21.15.26:0/1905070389 >> 172.21.15.26:0/1905070389 conn(0x7f8e4c06a800 msgr2=0x7f8e4c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f4e99200640 1 -- 172.21.15.26:0/770945984 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940b6100 con 0x7f4e940ce6f0 2024-01-15T17:55:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f7dce800640 1 -- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 msgr2=0x7f7dd015c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f7dce800640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 0x7f7dd015c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f7dce800640 1 -- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dd00b3890 msgr2=0x7f7dd015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f7dce800640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dd00b3890 0x7f7dd015bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f7dce800640 1 -- 172.21.15.26:0/3448489425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc0009a70 con 0x7f7dd015d610 2024-01-15T17:55:13.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f8e54a00640 1 -- 172.21.15.26:0/1905070389 shutdown_connections 2024-01-15T17:55:13.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.815+0000 7f01e6800640 1 -- start start 2024-01-15T17:55:13.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.816+0000 7f4e93400640 1 --2- 172.21.15.26:0/770945984 >> v2:172.21.15.124:3300/0 conn(0x7f4e940d0280 0x7f4e940ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:13.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.816+0000 7fdc5ea00640 1 --2- 172.21.15.26:0/677339146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc601536e0 0x7fdc6015ce90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdc480097a0 tx=0x7fdc4802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d818ae9b987f7999 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.816+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c067450 con 0x7f498c160920 2024-01-15T17:55:13.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.816+0000 7f8e54a00640 1 -- 172.21.15.26:0/1905070389 wait complete. 2024-01-15T17:55:13.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7fdc5e000640 1 -- 172.21.15.26:0/677339146 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48030800 con 0x7fdc601536e0 2024-01-15T17:55:13.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7fdc5e000640 1 -- 172.21.15.26:0/677339146 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc48030960 con 0x7fdc601536e0 2024-01-15T17:55:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7fdc5e000640 1 -- 172.21.15.26:0/677339146 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48030cf0 con 0x7fdc601536e0 2024-01-15T17:55:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7f7dce800640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dd015d610 0x7f7dd0203690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7db800d560 tx=0x7f7db800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7f01e6800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01e00cf9a0 0x7f01e0070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7f01e6800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f01e0070ea0 0x7f01e00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7f01e6800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e0071690 con 0x7f01e014b2c0 2024-01-15T17:55:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7f01e6800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e014b6a0 con 0x7f01e00cf9a0 2024-01-15T17:55:13.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7f01e6800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e0071970 con 0x7f01e0070ea0 2024-01-15T17:55:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.817+0000 7f4e99200640 1 --2- 172.21.15.26:0/770945984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940ce6f0 0x7f4e940b6be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e9408d970 tx=0x7f4e8c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814c6631f8632efa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7fdc66200640 1 -- 172.21.15.26:0/677339146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc601536e0 msgr2=0x7fdc6015ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7fdc66200640 1 --2- 172.21.15.26:0/677339146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc601536e0 0x7fdc6015ce90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdc480097a0 tx=0x7fdc4802b940 comp rx=0 tx=0).stop 2024-01-15T17:55:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7fdc66200640 1 -- 172.21.15.26:0/677339146 shutdown_connections 2024-01-15T17:55:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7fdc66200640 1 --2- 172.21.15.26:0/677339146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc60153b70 0x7fdc6015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7fdc66200640 1 --2- 172.21.15.26:0/677339146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc601536e0 0x7fdc6015ce90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7fdc66200640 1 -- 172.21.15.26:0/677339146 >> 172.21.15.26:0/677339146 conn(0x7fdc600b0be0 msgr2=0x7fdc600b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7f01dea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01e00cf9a0 0x7f01e0070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7f01dea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f01e00cf9a0 0x7f01e0070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44534/0 (socket says 172.21.15.26:44534) 2024-01-15T17:55:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7f01dea00640 1 -- 172.21.15.26:0/2053684169 learned_addr learned my addr 172.21.15.26:0/2053684169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7f01dfe00640 1 --2- 172.21.15.26:0/2053684169 >> v2:172.21.15.124:3300/0 conn(0x7f01e0070ea0 0x7f01e00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7fdc66200640 1 -- 172.21.15.26:0/677339146 shutdown_connections 2024-01-15T17:55:13.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7fdc66200640 1 -- 172.21.15.26:0/677339146 wait complete. 2024-01-15T17:55:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.819+0000 7f01dea00640 1 -- 172.21.15.26:0/2053684169 >> v2:172.21.15.124:3300/0 conn(0x7f01e0070ea0 msgr2=0x7f01e00cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.819+0000 7f01dea00640 1 --2- 172.21.15.26:0/2053684169 >> v2:172.21.15.124:3300/0 conn(0x7f01e0070ea0 0x7f01e00cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.819+0000 7f01dea00640 1 -- 172.21.15.26:0/2053684169 >> v1:172.21.15.26:6789/0 conn(0x7f01e014b2c0 legacy=0x7f01e00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.819+0000 7f01dea00640 1 -- 172.21.15.26:0/2053684169 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0071c50 con 0x7f01e00cf9a0 2024-01-15T17:55:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.819+0000 7f01dfe00640 1 --2- 172.21.15.26:0/2053684169 >> v2:172.21.15.124:3300/0 conn(0x7f01e0070ea0 0x7f01e00cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:13.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.819+0000 7f4983400640 1 -- 172.21.15.26:0/1999853912 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4974013470 con 0x7f498c160920 2024-01-15T17:55:13.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.819+0000 7f4e92a00640 1 -- 172.21.15.26:0/770945984 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c02cb10 con 0x7f4e940ce6f0 2024-01-15T17:55:13.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.820+0000 7f4e92a00640 1 -- 172.21.15.26:0/770945984 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c02cc70 con 0x7f4e940ce6f0 2024-01-15T17:55:13.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.820+0000 7f4e92a00640 1 -- 172.21.15.26:0/770945984 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0343f0 con 0x7f4e940ce6f0 2024-01-15T17:55:13.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.820+0000 7f7dc6000640 1 -- 172.21.15.26:0/3448489425 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db80125f0 con 0x7f7dd015d610 2024-01-15T17:55:13.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.820+0000 7f7dc6000640 1 -- 172.21.15.26:0/3448489425 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db8012750 con 0x7f7dd015d610 2024-01-15T17:55:13.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.820+0000 7f7dc6000640 1 -- 172.21.15.26:0/3448489425 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8012ae0 con 0x7f7dd015d610 2024-01-15T17:55:13.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd01f5950 con 0x7f7dd015d610 2024-01-15T17:55:13.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd01f5f10 con 0x7f7dd015d610 2024-01-15T17:55:13.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.818+0000 7f8e54a00640 1 Processor -- start 2024-01-15T17:55:13.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.819+0000 7f8e54a00640 1 -- start start 2024-01-15T17:55:13.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f8e54a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e4c0ae5a0 0x7f8e4c150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f8e54a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4c0b3ae0 0x7f8e4c155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f8e54a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4c0cf290 0x7f8e4c153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f8e54a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c156340 con 0x7f8e4c0cf290 2024-01-15T17:55:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f8e54a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c153560 con 0x7f8e4c0ae5a0 2024-01-15T17:55:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f8e54a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c1512d0 con 0x7f8e4c0b3ae0 2024-01-15T17:55:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f4983400640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f496406b310 0x7f496406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f4983400640 1 -- 172.21.15.26:0/1999853912 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f497408a550 con 0x7f498c160920 2024-01-15T17:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f4e9b200640 1 -- 172.21.15.26:0/770945984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940ce6f0 msgr2=0x7f4e940b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f4e9b200640 1 --2- 172.21.15.26:0/770945984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940ce6f0 0x7f4e940b6be0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4e9408d970 tx=0x7f4e8c02bf70 comp rx=0 tx=0).stop 2024-01-15T17:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7fdc66200640 1 Processor -- start 2024-01-15T17:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f01dea00640 1 --2- 172.21.15.26:0/2053684169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 0x7f01e0070960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f01e0165330 tx=0x7f01c802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2824afee647fa9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7fe33d600640 1 -- 172.21.15.26:0/2420352365 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3400ce250 con 0x7fe33803c070 2024-01-15T17:55:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.822+0000 7f4e9b200640 1 -- 172.21.15.26:0/770945984 shutdown_connections 2024-01-15T17:55:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.822+0000 7f4e9b200640 1 --2- 172.21.15.26:0/770945984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940ce6f0 0x7f4e940b6be0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.822+0000 7f4e9b200640 1 --2- 172.21.15.26:0/770945984 >> v2:172.21.15.124:3300/0 conn(0x7f4e940d0280 0x7f4e940ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.822+0000 7f4e9b200640 1 -- 172.21.15.26:0/770945984 >> 172.21.15.26:0/770945984 conn(0x7f4e9409c540 msgr2=0x7f4e940d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.821+0000 7f01de000640 1 -- 172.21.15.26:0/2053684169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8030860 con 0x7f01e00cf9a0 2024-01-15T17:55:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.822+0000 7f01de000640 1 -- 172.21.15.26:0/2053684169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c80309c0 con 0x7f01e00cf9a0 2024-01-15T17:55:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.822+0000 7fdc66200640 1 -- start start 2024-01-15T17:55:13.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.822+0000 7f01de000640 1 -- 172.21.15.26:0/2053684169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8030d50 con 0x7f01e00cf9a0 2024-01-15T17:55:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.822+0000 7f4e9b200640 1 -- 172.21.15.26:0/770945984 shutdown_connections 2024-01-15T17:55:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.822+0000 7f8e51600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4c0b3ae0 0x7f8e4c155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f8e51600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4c0b3ae0 0x7f8e4c155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:37310/0 (socket says 172.21.15.26:37310) 2024-01-15T17:55:13.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f01e6800640 1 -- 172.21.15.26:0/2053684169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 msgr2=0x7f01e0070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f01e6800640 1 --2- 172.21.15.26:0/2053684169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 0x7f01e0070960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f01e0165330 tx=0x7f01c802b9a0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f8e51600640 1 -- 172.21.15.26:0/1756096855 learned_addr learned my addr 172.21.15.26:0/1756096855 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f01e6800640 1 -- 172.21.15.26:0/2053684169 shutdown_connections 2024-01-15T17:55:13.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f01e6800640 1 --2- 172.21.15.26:0/2053684169 >> v2:172.21.15.124:3300/0 conn(0x7f01e0070ea0 0x7f01e00cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f01e6800640 1 --2- 172.21.15.26:0/2053684169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 0x7f01e0070960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f01e6800640 1 -- 172.21.15.26:0/2053684169 >> 172.21.15.26:0/2053684169 conn(0x7f01e006a800 msgr2=0x7f01e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:13.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f4e9b200640 1 -- 172.21.15.26:0/770945984 wait complete. 2024-01-15T17:55:13.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f01e6800640 1 -- 172.21.15.26:0/2053684169 shutdown_connections 2024-01-15T17:55:13.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd00675d0 con 0x7f7dd015d610 2024-01-15T17:55:13.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.824+0000 7f8e52000640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e4c0ae5a0 0x7f8e4c150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.824+0000 7f498b400640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f496406b310 0x7f496406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7f8e52a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4c0cf290 0x7f8e4c153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.824+0000 7f01e6800640 1 -- 172.21.15.26:0/2053684169 wait complete. 2024-01-15T17:55:13.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7fdc66200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc600b16d0 0x7fdc60147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7fdc66200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc601536e0 0x7fdc60169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.823+0000 7fdc66200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc601485d0 0x7fdc60070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.824+0000 7fdc66200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601585e0 con 0x7fdc601485d0 2024-01-15T17:55:13.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.824+0000 7fdc66200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc60148310 con 0x7fdc601536e0 2024-01-15T17:55:13.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.824+0000 7fdc66200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601546f0 con 0x7fdc600b16d0 2024-01-15T17:55:13.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7f8e52a00640 1 -- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4c0b3ae0 msgr2=0x7f8e4c155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7f8e52a00640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4c0b3ae0 0x7f8e4c155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7f8e52a00640 1 -- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e4c0ae5a0 msgr2=0x7f8e4c150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7f8e52a00640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e4c0ae5a0 0x7f8e4c150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7f7dc6000640 1 -- 172.21.15.26:0/3448489425 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7db8012c40 con 0x7f7dd015d610 2024-01-15T17:55:13.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc601485d0 0x7fdc60070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7fdc5fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc601485d0 0x7fdc60070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44548/0 (socket says 172.21.15.26:44548) 2024-01-15T17:55:13.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7fdc5fe00640 1 -- 172.21.15.26:0/1917774473 learned_addr learned my addr 172.21.15.26:0/1917774473 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7f8e52a00640 1 -- 172.21.15.26:0/1756096855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e34009400 con 0x7f8e4c0cf290 2024-01-15T17:55:13.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.825+0000 7fdc5f400640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc600b16d0 0x7fdc60147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.826+0000 7fdc5ea00640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc601536e0 0x7fdc60169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.826+0000 7f498b400640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f496406b310 0x7f496406d7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f497c036000 tx=0x7f497c03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.826+0000 7f8e52a00640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4c0cf290 0x7f8e4c153020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400cbf0 tx=0x7f8e4400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.826+0000 7f01e6800640 1 Processor -- start 2024-01-15T17:55:13.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.826+0000 7f01e6800640 1 -- start start 2024-01-15T17:55:13.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7fdc5f400640 1 -- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc601536e0 msgr2=0x7fdc60169fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7fdc5f400640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc601536e0 0x7fdc60169fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7fdc5f400640 1 -- 172.21.15.26:0/1917774473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc601485d0 msgr2=0x7fdc60070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7fdc5f400640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc601485d0 0x7fdc60070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7fdc5f400640 1 -- 172.21.15.26:0/1917774473 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc48009400 con 0x7fdc600b16d0 2024-01-15T17:55:13.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7f4e9b200640 1 Processor -- start 2024-01-15T17:55:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7f7dc6000640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7da806b2c0 0x7f7da806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7f7dc6000640 1 -- 172.21.15.26:0/3448489425 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7db8089f60 con 0x7f7dd015d610 2024-01-15T17:55:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7f4e9b200640 1 -- start start 2024-01-15T17:55:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.827+0000 7fdc5f400640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc600b16d0 0x7fdc60147dd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c450 tx=0x7fdc5000c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01e6800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e0070ea0 0x7f01e015edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01e6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 0x7f01e01457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f7dcde00640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7da806b2c0 0x7f7da806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01e6800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01e014b2c0 0x7f01e0145d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01e6800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0143440 con 0x7f01e00cf9a0 2024-01-15T17:55:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01e6800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0143690 con 0x7f01e0070ea0 2024-01-15T17:55:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01e6800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0143970 con 0x7f01e014b2c0 2024-01-15T17:55:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01dea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 0x7f01e01457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01dea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 0x7f01e01457f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44560/0 (socket says 172.21.15.26:44560) 2024-01-15T17:55:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01dea00640 1 -- 172.21.15.26:0/1964618574 learned_addr learned my addr 172.21.15.26:0/1964618574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.828+0000 7f01df400640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e0070ea0 0x7f01e015edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f01dfe00640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01e014b2c0 0x7f01e0145d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f4e9b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e940ce6f0 0x7f4e9415dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f01dea00640 1 -- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01e014b2c0 msgr2=0x7f01e0145d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f01dea00640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01e014b2c0 0x7f01e0145d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f01dea00640 1 -- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e0070ea0 msgr2=0x7f01e015edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f01dea00640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e0070ea0 0x7f01e015edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f01dea00640 1 -- 172.21.15.26:0/1964618574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c80094d0 con 0x7f01e00cf9a0 2024-01-15T17:55:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f4e9b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940d0280 0x7f4e9414cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f4e9b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e9415d590 0x7f4e941f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f4e9b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9414d6b0 con 0x7f4e940d0280 2024-01-15T17:55:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f4e9b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9414d990 con 0x7f4e9415d590 2024-01-15T17:55:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f4e9b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9414dc70 con 0x7f4e940ce6f0 2024-01-15T17:55:13.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.829+0000 7f7dcde00640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7da806b2c0 0x7f7da806d780 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009370 tx=0x7f7dc8009300 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.830+0000 7f4e93400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940d0280 0x7f4e9414cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.830+0000 7f4e99200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e9415d590 0x7f4e941f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.830+0000 7f4e93400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940d0280 0x7f4e9414cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44570/0 (socket says 172.21.15.26:44570) 2024-01-15T17:55:13.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.830+0000 7f4e93400640 1 -- 172.21.15.26:0/492303764 learned_addr learned my addr 172.21.15.26:0/492303764 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:13.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.830+0000 7f4e93e00640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e940ce6f0 0x7f4e9415dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.830+0000 7f01dea00640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 0x7f01e01457f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f01c8000c00 tx=0x7f01c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.831+0000 7f4e93400640 1 -- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e940ce6f0 msgr2=0x7f4e9415dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.831+0000 7f4e93400640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e940ce6f0 0x7f4e9415dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.831+0000 7f4e93400640 1 -- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e9415d590 msgr2=0x7f4e941f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:13.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.831+0000 7f4e93400640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e9415d590 0x7f4e941f2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:13.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.831+0000 7f4e93400640 1 -- 172.21.15.26:0/492303764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c009b30 con 0x7f4e940d0280 2024-01-15T17:55:13.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.831+0000 7f4e93400640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940d0280 0x7f4e9414cd40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00e730 tx=0x7f4e7c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.832+0000 7f7dc6000640 1 -- 172.21.15.26:0/3448489425 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7db8054d30 con 0x7f7dd015d610 2024-01-15T17:55:13.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.832+0000 7f01d7400640 1 -- 172.21.15.26:0/1964618574 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8039d50 con 0x7f01e00cf9a0 2024-01-15T17:55:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.833+0000 7f01d7400640 1 -- 172.21.15.26:0/1964618574 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c8043400 con 0x7f01e00cf9a0 2024-01-15T17:55:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.833+0000 7f01d7400640 1 -- 172.21.15.26:0/1964618574 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c80437b0 con 0x7f01e00cf9a0 2024-01-15T17:55:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.833+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e0143c50 con 0x7f01e00cf9a0 2024-01-15T17:55:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.833+0000 7f8e4a000640 1 -- 172.21.15.26:0/1756096855 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4400aae0 con 0x7f8e4c0cf290 2024-01-15T17:55:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.833+0000 7f8e4a000640 1 -- 172.21.15.26:0/1756096855 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4400ac40 con 0x7f8e4c0cf290 2024-01-15T17:55:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.833+0000 7f8e4a000640 1 -- 172.21.15.26:0/1756096855 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44012400 con 0x7f8e4c0cf290 2024-01-15T17:55:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.833+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e4c1515b0 con 0x7f8e4c0cf290 2024-01-15T17:55:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.834+0000 7f4e8be00640 1 -- 172.21.15.26:0/492303764 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c004230 con 0x7f4e940d0280 2024-01-15T17:55:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.834+0000 7f4e8be00640 1 -- 172.21.15.26:0/492303764 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c004390 con 0x7f4e940d0280 2024-01-15T17:55:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.834+0000 7f4e8be00640 1 -- 172.21.15.26:0/492303764 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c00c8f0 con 0x7f4e940d0280 2024-01-15T17:55:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.834+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9414df50 con 0x7f4e940d0280 2024-01-15T17:55:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.834+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9415e480 con 0x7f4e940d0280 2024-01-15T17:55:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.833+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e00cc970 con 0x7f01e00cf9a0 2024-01-15T17:55:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.833+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c0aedd0 con 0x7f8e4c0cf290 2024-01-15T17:55:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.836+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e00675d0 con 0x7f01e00cf9a0 2024-01-15T17:55:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.837+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e940675d0 con 0x7f4e940d0280 2024-01-15T17:55:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.837+0000 7fdc57400640 1 -- 172.21.15.26:0/1917774473 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500074e0 con 0x7fdc600b16d0 2024-01-15T17:55:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.837+0000 7fdc57400640 1 -- 172.21.15.26:0/1917774473 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc50007640 con 0x7fdc600b16d0 2024-01-15T17:55:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.837+0000 7fdc57400640 1 -- 172.21.15.26:0/1917774473 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50004a20 con 0x7fdc600b16d0 2024-01-15T17:55:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.837+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc601549d0 con 0x7fdc600b16d0 2024-01-15T17:55:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.837+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc60154f90 con 0x7fdc600b16d0 2024-01-15T17:55:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.838+0000 7f4983400640 1 -- 172.21.15.26:0/1999853912 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49740554e0 con 0x7f498c160920 2024-01-15T17:55:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.837+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e4c0675d0 con 0x7f8e4c0cf290 2024-01-15T17:55:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.838+0000 7f8e4a000640 1 -- 172.21.15.26:0/1756096855 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8e44012560 con 0x7f8e4c0cf290 2024-01-15T17:55:13.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.839+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc30005250 con 0x7fdc600b16d0 2024-01-15T17:55:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.839+0000 7f8e4a000640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e2406b340 0x7f8e2406d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.839+0000 7f01d7400640 1 -- 172.21.15.26:0/1964618574 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f01c8043910 con 0x7f01e00cf9a0 2024-01-15T17:55:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.843+0000 7f01d7400640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f01b806b390 0x7f01b806d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.843+0000 7f01d7400640 1 -- 172.21.15.26:0/1964618574 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01c803e070 con 0x7f01e00cf9a0 2024-01-15T17:55:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.841+0000 7f4e8be00640 1 -- 172.21.15.26:0/492303764 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4e7c00ca50 con 0x7f4e940d0280 2024-01-15T17:55:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.842+0000 7f4e8be00640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e6c06b340 0x7f4e6c06d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.843+0000 7f4e8be00640 1 -- 172.21.15.26:0/492303764 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e7c089940 con 0x7f4e940d0280 2024-01-15T17:55:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.843+0000 7f4e93e00640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e6c06b340 0x7f4e6c06d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.843+0000 7fdc57400640 1 -- 172.21.15.26:0/1917774473 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdc50004b80 con 0x7fdc600b16d0 2024-01-15T17:55:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.844+0000 7f8e52000640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e2406b340 0x7f8e2406d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.844+0000 7f01df400640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f01b806b390 0x7f01b806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.845+0000 7fdc57400640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc3806b390 0x7fdc3806d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:13.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.845+0000 7f4e93e00640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e6c06b340 0x7f4e6c06d800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4e84000f60 tx=0x7f4e84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.845+0000 7fdc57400640 1 -- 172.21.15.26:0/1917774473 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc50089500 con 0x7fdc600b16d0 2024-01-15T17:55:13.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.845+0000 7f8e52000640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e2406b340 0x7f8e2406d800 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0011e0 tx=0x7f8e3c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.845+0000 7f01df400640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f01b806b390 0x7f01b806d850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f01d00016e0 tx=0x7f01d0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.846+0000 7fdc5ea00640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc3806b390 0x7fdc3806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:13.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.847+0000 7f01d7400640 1 -- 172.21.15.26:0/1964618574 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01c80785c0 con 0x7f01e00cf9a0 2024-01-15T17:55:13.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.847+0000 7fdc5ea00640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc3806b390 0x7fdc3806d850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdc48036000 tx=0x7fdc48008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:13.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.853+0000 7fdc57400640 1 -- 172.21.15.26:0/1917774473 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc50054400 con 0x7fdc600b16d0 2024-01-15T17:55:13.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.857+0000 7f4e8be00640 1 -- 172.21.15.26:0/492303764 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e7c0548d0 con 0x7f4e940d0280 2024-01-15T17:55:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.859+0000 7f8e4a000640 1 -- 172.21.15.26:0/1756096855 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e44058200 con 0x7f8e4c0cf290 2024-01-15T17:55:13.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:13.867+0000 7f8e4a000640 1 -- 172.21.15.26:0/1756096855 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e44054540 con 0x7f8e4c0cf290 2024-01-15T17:55:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.182+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6030005500 con 0x7f60600b3eb0 2024-01-15T17:55:14.188 INFO:teuthology.orchestra.run.smithi026.stdout:60129542149 2024-01-15T17:55:14.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.184+0000 7f6056a00640 1 -- 172.21.15.26:0/1568368840 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6048057d90 con 0x7f60600b3eb0 2024-01-15T17:55:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.187+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603806b320 msgr2=0x7f603806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.188+0000 7f6065800640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603806b320 0x7f603806d7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6058001dc0 tx=0x7f6058009040 comp rx=0 tx=0).stop 2024-01-15T17:55:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.188+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60600b3eb0 msgr2=0x7f60601f92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.188+0000 7f6065800640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60600b3eb0 0x7f60601f92d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f604800b410 tx=0x7f6048009f90 comp rx=0 tx=0).stop 2024-01-15T17:55:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.188+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 shutdown_connections 2024-01-15T17:55:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.188+0000 7f6065800640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603806b320 0x7f603806d7e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.188+0000 7f6065800640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60600b3eb0 0x7f60601f92d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.188+0000 7f6065800640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60600b0530 0x7f6060135980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.188+0000 7f6065800640 1 --2- 172.21.15.26:0/1568368840 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606006c340 0x7f6060135440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.188+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 >> 172.21.15.26:0/1568368840 conn(0x7f60600aea00 msgr2=0x7f6060158600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.189+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 shutdown_connections 2024-01-15T17:55:14.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.189+0000 7f6065800640 1 -- 172.21.15.26:0/1568368840 wait complete. 2024-01-15T17:55:14.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2024-01-15T17:55:14.218 DEBUG:teuthology.parallel:result is None 2024-01-15T17:55:14.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.241+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe3100050c0 con 0x7fe33803c070 2024-01-15T17:55:14.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.242+0000 7fe33d600640 1 -- 172.21.15.26:0/2420352365 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe3400d1870 con 0x7fe33803c070 2024-01-15T17:55:14.245 INFO:teuthology.orchestra.run.smithi026.stdout:60129542150 2024-01-15T17:55:14.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.243+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4e9415e480 con 0x7f4e940d0280 2024-01-15T17:55:14.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.243+0000 7f4e8be00640 1 -- 172.21.15.26:0/492303764 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e9415e480 con 0x7f4e940d0280 2024-01-15T17:55:14.247 INFO:teuthology.orchestra.run.smithi026.stdout:60129542150 2024-01-15T17:55:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.245+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe31806b2c0 msgr2=0x7fe31806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.245+0000 7fe347000640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe31806b2c0 0x7fe31806d780 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe33804fdb0 tx=0x7fe3280019e0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.245+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe33803c070 msgr2=0x7fe3380e16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.245+0000 7fe347000640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe33803c070 0x7fe3380e16b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe34002cee0 tx=0x7fe3400748f0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 shutdown_connections 2024-01-15T17:55:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7fe347000640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe3380e3130 0x7fe338052690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7fe347000640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe31806b2c0 0x7fe31806d780 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7fe347000640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe338054390 0x7fe338052150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7fe347000640 1 --2- 172.21.15.26:0/2420352365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe33803c070 0x7fe3380e16b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 >> 172.21.15.26:0/2420352365 conn(0x7fe338058eb0 msgr2=0x7fe3380386c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 shutdown_connections 2024-01-15T17:55:14.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e6c06b340 msgr2=0x7f4e6c06d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7fe347000640 1 -- 172.21.15.26:0/2420352365 wait complete. 2024-01-15T17:55:14.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7f4e9b200640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e6c06b340 0x7f4e6c06d800 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4e84000f60 tx=0x7f4e84009040 comp rx=0 tx=0).stop 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940d0280 msgr2=0x7f4e9414cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.246+0000 7f4e9b200640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940d0280 0x7f4e9414cd40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00e730 tx=0x7f4e7c00ebf0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.247+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 shutdown_connections 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.247+0000 7f4e9b200640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e6c06b340 0x7f4e6c06d800 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.247+0000 7f4e9b200640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e9415d590 0x7f4e941f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.247+0000 7f4e9b200640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e940d0280 0x7f4e9414cd40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.247+0000 7f4e9b200640 1 --2- 172.21.15.26:0/492303764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e940ce6f0 0x7f4e9415dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.247+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 >> 172.21.15.26:0/492303764 conn(0x7f4e9409c540 msgr2=0x7f4e940b74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.247+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 shutdown_connections 2024-01-15T17:55:14.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.248+0000 7f4e9b200640 1 -- 172.21.15.26:0/492303764 wait complete. 2024-01-15T17:55:14.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.254+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f498c1fb320 con 0x7f498c160920 2024-01-15T17:55:14.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.255+0000 7f4983400640 1 -- 172.21.15.26:0/1999853912 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f498c1fb320 con 0x7f498c160920 2024-01-15T17:55:14.260 INFO:teuthology.orchestra.run.smithi026.stdout:60129542150 2024-01-15T17:55:14.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.258+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7dd00b14e0 con 0x7f7dd015d610 2024-01-15T17:55:14.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.259+0000 7f7dc6000640 1 -- 172.21.15.26:0/3448489425 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7db8058350 con 0x7f7dd015d610 2024-01-15T17:55:14.262 INFO:teuthology.orchestra.run.smithi026.stdout:60129542148 2024-01-15T17:55:14.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.261+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f496406b310 msgr2=0x7f496406d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.261+0000 7f4991000640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f496406b310 0x7f496406d7d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f497c036000 tx=0x7f497c03b040 comp rx=0 tx=0).stop 2024-01-15T17:55:14.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.261+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c160920 msgr2=0x7f498c1545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.262+0000 7f4991000640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c160920 0x7f498c1545b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f497400e8e0 tx=0x7f497400eda0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.262+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 shutdown_connections 2024-01-15T17:55:14.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.262+0000 7f4991000640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f498c160920 0x7f498c1545b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.262+0000 7f4991000640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f496406b310 0x7f496406d7d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.262+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7da806b2c0 msgr2=0x7f7da806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.262+0000 7f7dd4e00640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7da806b2c0 0x7f7da806d780 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7dc8009370 tx=0x7f7dc8009300 comp rx=0 tx=0).stop 2024-01-15T17:55:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.262+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dd015d610 msgr2=0x7f7dd0203690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.262+0000 7f7dd4e00640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dd015d610 0x7f7dd0203690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7db800d560 tx=0x7f7db800d960 comp rx=0 tx=0).stop 2024-01-15T17:55:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f4991000640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f498c161010 0x7f498c157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f4991000640 1 --2- 172.21.15.26:0/1999853912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f498c0cf9f0 0x7f498c1fa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 shutdown_connections 2024-01-15T17:55:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f7dd4e00640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7dd00d17e0 0x7f7dd015c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f7dd4e00640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7da806b2c0 0x7f7da806d780 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f7dd4e00640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7dd015d610 0x7f7dd0203690 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f7dd4e00640 1 --2- 172.21.15.26:0/3448489425 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7dd00b3890 0x7f7dd015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 >> 172.21.15.26:0/3448489425 conn(0x7f7dd009c540 msgr2=0x7f7dd00733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 >> 172.21.15.26:0/1999853912 conn(0x7f498c0d3090 msgr2=0x7f498c0d3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 shutdown_connections 2024-01-15T17:55:14.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f7dd4e00640 1 -- 172.21.15.26:0/3448489425 wait complete. 2024-01-15T17:55:14.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.263+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 shutdown_connections 2024-01-15T17:55:14.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.264+0000 7f4991000640 1 -- 172.21.15.26:0/1999853912 wait complete. 2024-01-15T17:55:14.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-01-15T17:55:14.271 DEBUG:teuthology.parallel:result is None 2024-01-15T17:55:14.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.270+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f01e00b15f0 con 0x7f01e00cf9a0 2024-01-15T17:55:14.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.271+0000 7f01d7400640 1 -- 172.21.15.26:0/1964618574 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01c807bbe0 con 0x7f01e00cf9a0 2024-01-15T17:55:14.274 INFO:teuthology.orchestra.run.smithi026.stdout:60129542148 2024-01-15T17:55:14.276 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.0 2024-01-15T17:55:14.276 DEBUG:teuthology.parallel:result is None 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f01b806b390 msgr2=0x7f01b806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f01b806b390 0x7f01b806d850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f01d00016e0 tx=0x7f01d0009040 comp rx=0 tx=0).stop 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 msgr2=0x7f01e01457f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 0x7f01e01457f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f01c8000c00 tx=0x7f01c8004730 comp rx=0 tx=0).stop 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 shutdown_connections 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f01e014b2c0 0x7f01e0145d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f01b806b390 0x7f01b806d850 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f01e00cf9a0 0x7f01e01457f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 --2- 172.21.15.26:0/1964618574 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f01e0070ea0 0x7f01e015edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.273+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 >> 172.21.15.26:0/1964618574 conn(0x7f01e006a800 msgr2=0x7f01e00cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.274+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 shutdown_connections 2024-01-15T17:55:14.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.274+0000 7f01e6800640 1 -- 172.21.15.26:0/1964618574 wait complete. 2024-01-15T17:55:14.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.283+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdc300057d0 con 0x7fdc600b16d0 2024-01-15T17:55:14.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.284+0000 7fdc57400640 1 -- 172.21.15.26:0/1917774473 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdc50018020 con 0x7fdc600b16d0 2024-01-15T17:55:14.287 INFO:teuthology.orchestra.run.smithi026.stdout:60129542148 2024-01-15T17:55:14.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc3806b390 msgr2=0x7fdc3806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc3806b390 0x7fdc3806d850 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdc48036000 tx=0x7fdc48008030 comp rx=0 tx=0).stop 2024-01-15T17:55:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc600b16d0 msgr2=0x7fdc60147dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc600b16d0 0x7fdc60147dd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000c450 tx=0x7fdc5000c910 comp rx=0 tx=0).stop 2024-01-15T17:55:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 shutdown_connections 2024-01-15T17:55:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc3806b390 0x7fdc3806d850 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc601485d0 0x7fdc60070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc601536e0 0x7fdc60169fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 --2- 172.21.15.26:0/1917774473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc600b16d0 0x7fdc60147dd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 >> 172.21.15.26:0/1917774473 conn(0x7fdc600b0be0 msgr2=0x7fdc60157880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 shutdown_connections 2024-01-15T17:55:14.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.287+0000 7fdc66200640 1 -- 172.21.15.26:0/1917774473 wait complete. 2024-01-15T17:55:14.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.5 2024-01-15T17:55:14.291 DEBUG:teuthology.parallel:result is None 2024-01-15T17:55:14.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2024-01-15T17:55:14.295 DEBUG:teuthology.parallel:result is None 2024-01-15T17:55:14.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.6 2024-01-15T17:55:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.294+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8e4c161b90 con 0x7f8e4c0cf290 2024-01-15T17:55:14.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.294+0000 7f8e4a000640 1 -- 172.21.15.26:0/1756096855 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e44057b60 con 0x7f8e4c0cf290 2024-01-15T17:55:14.297 DEBUG:teuthology.parallel:result is None 2024-01-15T17:55:14.297 INFO:teuthology.orchestra.run.smithi026.stdout:60129542149 2024-01-15T17:55:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.296+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e2406b340 msgr2=0x7f8e2406d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.296+0000 7f8e54a00640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e2406b340 0x7f8e2406d800 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c0011e0 tx=0x7f8e3c009040 comp rx=0 tx=0).stop 2024-01-15T17:55:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.296+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4c0cf290 msgr2=0x7f8e4c153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.296+0000 7f8e54a00640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4c0cf290 0x7f8e4c153020 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400cbf0 tx=0x7f8e4400a430 comp rx=0 tx=0).stop 2024-01-15T17:55:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.297+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 shutdown_connections 2024-01-15T17:55:14.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.297+0000 7f8e54a00640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4c0b3ae0 0x7f8e4c155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.297+0000 7f8e54a00640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e2406b340 0x7f8e2406d800 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.297+0000 7f8e54a00640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4c0cf290 0x7f8e4c153020 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.297+0000 7f8e54a00640 1 --2- 172.21.15.26:0/1756096855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e4c0ae5a0 0x7f8e4c150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.297+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 >> 172.21.15.26:0/1756096855 conn(0x7f8e4c06a800 msgr2=0x7f8e4c15d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.297+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 shutdown_connections 2024-01-15T17:55:14.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.297+0000 7f8e54a00640 1 -- 172.21.15.26:0/1756096855 wait complete. 2024-01-15T17:55:14.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.4 2024-01-15T17:55:14.307 DEBUG:teuthology.parallel:result is None 2024-01-15T17:55:14.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2024-01-15T17:55:14.317 DEBUG:teuthology.parallel:result is None 2024-01-15T17:55:14.317 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-15T17:55:14.318 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:55:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.572+0000 7faf5a800640 1 Processor -- start 2024-01-15T17:55:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.573+0000 7faf5a800640 1 -- start start 2024-01-15T17:55:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.574+0000 7faf5a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf54160960 0x7faf540cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.574+0000 7faf5a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf54160d40 0x7faf540cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.574+0000 7faf5a800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf540d1d60 con 0x7faf540b40b0 2024-01-15T17:55:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.574+0000 7faf5a800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf540d4000 con 0x7faf54160960 2024-01-15T17:55:14.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.574+0000 7faf5a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf540b3f40 con 0x7faf54160d40 2024-01-15T17:55:14.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.574+0000 7faf59400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf54160960 0x7faf540cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:14.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.574+0000 7faf59400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faf54160960 0x7faf540cf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44572/0 (socket says 172.21.15.26:44572) 2024-01-15T17:55:14.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.574+0000 7faf59400640 1 -- 172.21.15.26:0/1749179711 learned_addr learned my addr 172.21.15.26:0/1749179711 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:14.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.575+0000 7faf58a00640 1 --2- 172.21.15.26:0/1749179711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf54160d40 0x7faf540cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:14.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.575+0000 7faf59400640 1 -- 172.21.15.26:0/1749179711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf54160d40 msgr2=0x7faf540cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.575+0000 7faf59400640 1 --2- 172.21.15.26:0/1749179711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf54160d40 0x7faf540cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.575+0000 7faf59400640 1 -- 172.21.15.26:0/1749179711 >> v1:172.21.15.124:6789/0 conn(0x7faf540b40b0 legacy=0x7faf540b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.575+0000 7faf59400640 1 -- 172.21.15.26:0/1749179711 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf540d3d20 con 0x7faf54160960 2024-01-15T17:55:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.576+0000 7faf59400640 1 --2- 172.21.15.26:0/1749179711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf54160960 0x7faf540cf790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf44009870 tx=0x7faf4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c319891479d9ff server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.576+0000 7faf53e00640 1 -- 172.21.15.26:0/1749179711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44031640 con 0x7faf54160960 2024-01-15T17:55:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.576+0000 7faf53e00640 1 -- 172.21.15.26:0/1749179711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf440317a0 con 0x7faf54160960 2024-01-15T17:55:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.576+0000 7faf53e00640 1 -- 172.21.15.26:0/1749179711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44031b30 con 0x7faf54160960 2024-01-15T17:55:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.577+0000 7faf5a800640 1 -- 172.21.15.26:0/1749179711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf54160960 msgr2=0x7faf540cf790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.577+0000 7faf5a800640 1 --2- 172.21.15.26:0/1749179711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf54160960 0x7faf540cf790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faf44009870 tx=0x7faf4402b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.577+0000 7faf5a800640 1 -- 172.21.15.26:0/1749179711 shutdown_connections 2024-01-15T17:55:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.577+0000 7faf5a800640 1 --2- 172.21.15.26:0/1749179711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf54160d40 0x7faf540cfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.577+0000 7faf5a800640 1 --2- 172.21.15.26:0/1749179711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf54160960 0x7faf540cf790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.577+0000 7faf5a800640 1 -- 172.21.15.26:0/1749179711 >> 172.21.15.26:0/1749179711 conn(0x7faf540d3060 msgr2=0x7faf540d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.577+0000 7faf5a800640 1 -- 172.21.15.26:0/1749179711 shutdown_connections 2024-01-15T17:55:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.577+0000 7faf5a800640 1 -- 172.21.15.26:0/1749179711 wait complete. 2024-01-15T17:55:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.579+0000 7faf5a800640 1 Processor -- start 2024-01-15T17:55:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.579+0000 7faf5a800640 1 -- start start 2024-01-15T17:55:14.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf5a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf540b40b0 0x7faf540b8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:14.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf5a800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf54160960 0x7faf541f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:14.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf5a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf54160d40 0x7faf540cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:14.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf5a800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541f2f90 con 0x7faf540b40b0 2024-01-15T17:55:14.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf5a800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5409d510 con 0x7faf54160960 2024-01-15T17:55:14.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf5a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf540b6c10 con 0x7faf54160d40 2024-01-15T17:55:14.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf59400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf540b40b0 0x7faf540b8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:14.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf59400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf540b40b0 0x7faf540b8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44576/0 (socket says 172.21.15.26:44576) 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf59400640 1 -- 172.21.15.26:0/1650523362 learned_addr learned my addr 172.21.15.26:0/1650523362 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf58a00640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf54160960 0x7faf541f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf59400640 1 -- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf54160d40 msgr2=0x7faf540cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf59400640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf54160d40 0x7faf540cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf59400640 1 -- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf54160960 msgr2=0x7faf541f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf59400640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf54160960 0x7faf541f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf59400640 1 -- 172.21.15.26:0/1650523362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf440094d0 con 0x7faf540b40b0 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.580+0000 7faf58a00640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf54160960 0x7faf541f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.581+0000 7faf59400640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf540b40b0 0x7faf540b8460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf44009870 tx=0x7faf440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.581+0000 7faf51600640 1 -- 172.21.15.26:0/1650523362 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4403ac60 con 0x7faf540b40b0 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.581+0000 7faf51600640 1 -- 172.21.15.26:0/1650523362 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4403ade0 con 0x7faf540b40b0 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.581+0000 7faf51600640 1 -- 172.21.15.26:0/1650523362 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44044630 con 0x7faf540b40b0 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.581+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf540b6ef0 con 0x7faf540b40b0 2024-01-15T17:55:14.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.581+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf540b74b0 con 0x7faf540b40b0 2024-01-15T17:55:14.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.582+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf54067450 con 0x7faf540b40b0 2024-01-15T17:55:14.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.583+0000 7faf51600640 1 -- 172.21.15.26:0/1650523362 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7faf4402f050 con 0x7faf540b40b0 2024-01-15T17:55:14.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.584+0000 7faf51600640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf2c06b2c0 0x7faf2c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.584+0000 7faf51600640 1 -- 172.21.15.26:0/1650523362 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf440abb60 con 0x7faf540b40b0 2024-01-15T17:55:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.584+0000 7faf58a00640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf2c06b2c0 0x7faf2c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:14.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.586+0000 7faf58a00640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf2c06b2c0 0x7faf2c06d780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf5406cb90 tx=0x7faf3c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:14.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.588+0000 7faf51600640 1 -- 172.21.15.26:0/1650523362 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf44076930 con 0x7faf540b40b0 2024-01-15T17:55:14.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.776+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf540d0820 con 0x7faf2c06b2c0 2024-01-15T17:55:14.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.798+0000 7faf51600640 1 -- 172.21.15.26:0/1650523362 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264330 (secure 0 0 0) 0x7faf3c009e10 con 0x7faf2c06b2c0 2024-01-15T17:55:14.805 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:14.806 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:55:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf2c06b2c0 msgr2=0x7faf2c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf2c06b2c0 0x7faf2c06d780 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faf5406cb90 tx=0x7faf3c008040 comp rx=0 tx=0).stop 2024-01-15T17:55:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf540b40b0 msgr2=0x7faf540b8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf540b40b0 0x7faf540b8460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf44009870 tx=0x7faf440046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 shutdown_connections 2024-01-15T17:55:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf54160d40 0x7faf540cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf2c06b2c0 0x7faf2c06d780 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf54160960 0x7faf541f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 --2- 172.21.15.26:0/1650523362 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf540b40b0 0x7faf540b8460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:14.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 >> 172.21.15.26:0/1650523362 conn(0x7faf540d3060 msgr2=0x7faf5406c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:14.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 shutdown_connections 2024-01-15T17:55:14.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:14.804+0000 7faf5a800640 1 -- 172.21.15.26:0/1650523362 wait complete. 2024-01-15T17:55:14.827 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-15T17:55:13.912863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236188,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738532,"statfs":{"total":773094113280,"available":772852256768,"internally_reserved":0,"allocated":3055616,"data_stored":1816564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.194431"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590593+0000","last_change":"2024-01-15T17:55:04.755200+0000","last_active":"2024-01-15T17:55:06.590593+0000","last_peered":"2024-01-15T17:55:06.590593+0000","last_clean":"2024-01-15T17:55:06.590593+0000","last_became_active":"2024-01-15T17:55:04.753634+0000","last_became_peered":"2024-01-15T17:55:04.753634+0000","last_unstale":"2024-01-15T17:55:06.590593+0000","last_undegraded":"2024-01-15T17:55:06.590593+0000","last_fullsized":"2024-01-15T17:55:06.590593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:13.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721381+0000","last_change":"2024-01-15T17:55:04.760607+0000","last_active":"2024-01-15T17:55:05.721381+0000","last_peered":"2024-01-15T17:55:05.721381+0000","last_clean":"2024-01-15T17:55:05.721381+0000","last_became_active":"2024-01-15T17:55:04.760299+0000","last_became_peered":"2024-01-15T17:55:04.760299+0000","last_unstale":"2024-01-15T17:55:05.721381+0000","last_undegraded":"2024-01-15T17:55:05.721381+0000","last_fullsized":"2024-01-15T17:55:05.721381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787705+0000","last_change":"2024-01-15T17:55:04.755000+0000","last_active":"2024-01-15T17:55:06.787705+0000","last_peered":"2024-01-15T17:55:06.787705+0000","last_clean":"2024-01-15T17:55:06.787705+0000","last_became_active":"2024-01-15T17:55:04.754716+0000","last_became_peered":"2024-01-15T17:55:04.754716+0000","last_unstale":"2024-01-15T17:55:06.787705+0000","last_undegraded":"2024-01-15T17:55:06.787705+0000","last_fullsized":"2024-01-15T17:55:06.787705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:55.584774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789609+0000","last_change":"2024-01-15T17:55:04.767626+0000","last_active":"2024-01-15T17:55:06.789609+0000","last_peered":"2024-01-15T17:55:06.789609+0000","last_clean":"2024-01-15T17:55:06.789609+0000","last_became_active":"2024-01-15T17:55:04.767349+0000","last_became_peered":"2024-01-15T17:55:04.767349+0000","last_unstale":"2024-01-15T17:55:06.789609+0000","last_undegraded":"2024-01-15T17:55:06.789609+0000","last_fullsized":"2024-01-15T17:55:06.789609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:17.567853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590635+0000","last_change":"2024-01-15T17:55:04.756971+0000","last_active":"2024-01-15T17:55:06.590635+0000","last_peered":"2024-01-15T17:55:06.590635+0000","last_clean":"2024-01-15T17:55:06.590635+0000","last_became_active":"2024-01-15T17:55:04.756737+0000","last_became_peered":"2024-01-15T17:55:04.756737+0000","last_unstale":"2024-01-15T17:55:06.590635+0000","last_undegraded":"2024-01-15T17:55:06.590635+0000","last_fullsized":"2024-01-15T17:55:06.590635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:44.725915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789155+0000","last_change":"2024-01-15T17:55:04.763592+0000","last_active":"2024-01-15T17:55:06.789155+0000","last_peered":"2024-01-15T17:55:06.789155+0000","last_clean":"2024-01-15T17:55:06.789155+0000","last_became_active":"2024-01-15T17:55:04.763286+0000","last_became_peered":"2024-01-15T17:55:04.763286+0000","last_unstale":"2024-01-15T17:55:06.789155+0000","last_undegraded":"2024-01-15T17:55:06.789155+0000","last_fullsized":"2024-01-15T17:55:06.789155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:48:48.227594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589901+0000","last_change":"2024-01-15T17:55:04.754040+0000","last_active":"2024-01-15T17:55:06.589901+0000","last_peered":"2024-01-15T17:55:06.589901+0000","last_clean":"2024-01-15T17:55:06.589901+0000","last_became_active":"2024-01-15T17:55:04.753698+0000","last_became_peered":"2024-01-15T17:55:04.753698+0000","last_unstale":"2024-01-15T17:55:06.589901+0000","last_undegraded":"2024-01-15T17:55:06.589901+0000","last_fullsized":"2024-01-15T17:55:06.589901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:59.870088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788324+0000","last_change":"2024-01-15T17:55:04.754498+0000","last_active":"2024-01-15T17:55:06.788324+0000","last_peered":"2024-01-15T17:55:06.788324+0000","last_clean":"2024-01-15T17:55:06.788324+0000","last_became_active":"2024-01-15T17:55:04.754091+0000","last_became_peered":"2024-01-15T17:55:04.754091+0000","last_unstale":"2024-01-15T17:55:06.788324+0000","last_undegraded":"2024-01-15T17:55:06.788324+0000","last_fullsized":"2024-01-15T17:55:06.788324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:31.493057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590727+0000","last_change":"2024-01-15T17:55:04.755593+0000","last_active":"2024-01-15T17:55:06.590727+0000","last_peered":"2024-01-15T17:55:06.590727+0000","last_clean":"2024-01-15T17:55:06.590727+0000","last_became_active":"2024-01-15T17:55:04.755376+0000","last_became_peered":"2024-01-15T17:55:04.755376+0000","last_unstale":"2024-01-15T17:55:06.590727+0000","last_undegraded":"2024-01-15T17:55:06.590727+0000","last_fullsized":"2024-01-15T17:55:06.590727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:10:13.660425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726844+0000","last_change":"2024-01-15T17:55:04.745309+0000","last_active":"2024-01-15T17:55:05.726844+0000","last_peered":"2024-01-15T17:55:05.726844+0000","last_clean":"2024-01-15T17:55:05.726844+0000","last_became_active":"2024-01-15T17:55:04.744930+0000","last_became_peered":"2024-01-15T17:55:04.744930+0000","last_unstale":"2024-01-15T17:55:05.726844+0000","last_undegraded":"2024-01-15T17:55:05.726844+0000","last_fullsized":"2024-01-15T17:55:05.726844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726624+0000","last_change":"2024-01-15T17:55:04.743285+0000","last_active":"2024-01-15T17:55:05.726624+0000","last_peered":"2024-01-15T17:55:05.726624+0000","last_clean":"2024-01-15T17:55:05.726624+0000","last_became_active":"2024-01-15T17:55:04.743118+0000","last_became_peered":"2024-01-15T17:55:04.743118+0000","last_unstale":"2024-01-15T17:55:05.726624+0000","last_undegraded":"2024-01-15T17:55:05.726624+0000","last_fullsized":"2024-01-15T17:55:05.726624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726121+0000","last_change":"2024-01-15T17:55:04.739598+0000","last_active":"2024-01-15T17:55:05.726121+0000","last_peered":"2024-01-15T17:55:05.726121+0000","last_clean":"2024-01-15T17:55:05.726121+0000","last_became_active":"2024-01-15T17:55:04.739319+0000","last_became_peered":"2024-01-15T17:55:04.739319+0000","last_unstale":"2024-01-15T17:55:05.726121+0000","last_undegraded":"2024-01-15T17:55:05.726121+0000","last_fullsized":"2024-01-15T17:55:05.726121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:21.113714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788978+0000","last_change":"2024-01-15T17:55:04.751187+0000","last_active":"2024-01-15T17:55:06.788978+0000","last_peered":"2024-01-15T17:55:06.788978+0000","last_clean":"2024-01-15T17:55:06.788978+0000","last_became_active":"2024-01-15T17:55:04.750057+0000","last_became_peered":"2024-01-15T17:55:04.750057+0000","last_unstale":"2024-01-15T17:55:06.788978+0000","last_undegraded":"2024-01-15T17:55:06.788978+0000","last_fullsized":"2024-01-15T17:55:06.788978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:22:07.251406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787842+0000","last_change":"2024-01-15T17:55:04.759443+0000","last_active":"2024-01-15T17:55:06.787842+0000","last_peered":"2024-01-15T17:55:06.787842+0000","last_clean":"2024-01-15T17:55:06.787842+0000","last_became_active":"2024-01-15T17:55:04.759086+0000","last_became_peered":"2024-01-15T17:55:04.759086+0000","last_unstale":"2024-01-15T17:55:06.787842+0000","last_undegraded":"2024-01-15T17:55:06.787842+0000","last_fullsized":"2024-01-15T17:55:06.787842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:27.225767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721835+0000","last_change":"2024-01-15T17:55:04.741854+0000","last_active":"2024-01-15T17:55:05.721835+0000","last_peered":"2024-01-15T17:55:05.721835+0000","last_clean":"2024-01-15T17:55:05.721835+0000","last_became_active":"2024-01-15T17:55:04.741472+0000","last_became_peered":"2024-01-15T17:55:04.741472+0000","last_unstale":"2024-01-15T17:55:05.721835+0000","last_undegraded":"2024-01-15T17:55:05.721835+0000","last_fullsized":"2024-01-15T17:55:05.721835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789012+0000","last_change":"2024-01-15T17:55:04.757919+0000","last_active":"2024-01-15T17:55:06.789012+0000","last_peered":"2024-01-15T17:55:06.789012+0000","last_clean":"2024-01-15T17:55:06.789012+0000","last_became_active":"2024-01-15T17:55:04.757595+0000","last_became_peered":"2024-01-15T17:55:04.757595+0000","last_unstale":"2024-01-15T17:55:06.789012+0000","last_undegraded":"2024-01-15T17:55:06.789012+0000","last_fullsized":"2024-01-15T17:55:06.789012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790339+0000","last_change":"2024-01-15T17:55:04.745091+0000","last_active":"2024-01-15T17:55:06.790339+0000","last_peered":"2024-01-15T17:55:06.790339+0000","last_clean":"2024-01-15T17:55:06.790339+0000","last_became_active":"2024-01-15T17:55:04.744779+0000","last_became_peered":"2024-01-15T17:55:04.744779+0000","last_unstale":"2024-01-15T17:55:06.790339+0000","last_undegraded":"2024-01-15T17:55:06.790339+0000","last_fullsized":"2024-01-15T17:55:06.790339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:10:13.660425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787739+0000","last_change":"2024-01-15T17:55:04.754772+0000","last_active":"2024-01-15T17:55:06.787739+0000","last_peered":"2024-01-15T17:55:06.787739+0000","last_clean":"2024-01-15T17:55:06.787739+0000","last_became_active":"2024-01-15T17:55:04.743420+0000","last_became_peered":"2024-01-15T17:55:04.743420+0000","last_unstale":"2024-01-15T17:55:06.787739+0000","last_undegraded":"2024-01-15T17:55:06.787739+0000","last_fullsized":"2024-01-15T17:55:06.787739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:24:25.848060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591083+0000","last_change":"2024-01-15T17:55:04.741262+0000","last_active":"2024-01-15T17:55:06.591083+0000","last_peered":"2024-01-15T17:55:06.591083+0000","last_clean":"2024-01-15T17:55:06.591083+0000","last_became_active":"2024-01-15T17:55:04.740978+0000","last_became_peered":"2024-01-15T17:55:04.740978+0000","last_unstale":"2024-01-15T17:55:06.591083+0000","last_undegraded":"2024-01-15T17:55:06.591083+0000","last_fullsized":"2024-01-15T17:55:06.591083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:00.893650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726416+0000","last_change":"2024-01-15T17:55:04.758887+0000","last_active":"2024-01-15T17:55:05.726416+0000","last_peered":"2024-01-15T17:55:05.726416+0000","last_clean":"2024-01-15T17:55:05.726416+0000","last_became_active":"2024-01-15T17:55:04.758530+0000","last_became_peered":"2024-01-15T17:55:04.758530+0000","last_unstale":"2024-01-15T17:55:05.726416+0000","last_undegraded":"2024-01-15T17:55:05.726416+0000","last_fullsized":"2024-01-15T17:55:05.726416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788603+0000","last_change":"2024-01-15T17:55:04.756875+0000","last_active":"2024-01-15T17:55:06.788603+0000","last_peered":"2024-01-15T17:55:06.788603+0000","last_clean":"2024-01-15T17:55:06.788603+0000","last_became_active":"2024-01-15T17:55:04.756596+0000","last_became_peered":"2024-01-15T17:55:04.756596+0000","last_unstale":"2024-01-15T17:55:06.788603+0000","last_undegraded":"2024-01-15T17:55:06.788603+0000","last_fullsized":"2024-01-15T17:55:06.788603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789372+0000","last_change":"2024-01-15T17:55:04.746119+0000","last_active":"2024-01-15T17:55:06.789372+0000","last_peered":"2024-01-15T17:55:06.789372+0000","last_clean":"2024-01-15T17:55:06.789372+0000","last_became_active":"2024-01-15T17:55:04.745830+0000","last_became_peered":"2024-01-15T17:55:04.745830+0000","last_unstale":"2024-01-15T17:55:06.789372+0000","last_undegraded":"2024-01-15T17:55:06.789372+0000","last_fullsized":"2024-01-15T17:55:06.789372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:47:39.573232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789300+0000","last_change":"2024-01-15T17:55:04.764403+0000","last_active":"2024-01-15T17:55:06.789300+0000","last_peered":"2024-01-15T17:55:06.789300+0000","last_clean":"2024-01-15T17:55:06.789300+0000","last_became_active":"2024-01-15T17:55:04.763976+0000","last_became_peered":"2024-01-15T17:55:04.763976+0000","last_unstale":"2024-01-15T17:55:06.789300+0000","last_undegraded":"2024-01-15T17:55:06.789300+0000","last_fullsized":"2024-01-15T17:55:06.789300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:08:09.187744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789136+0000","last_change":"2024-01-15T17:55:04.750998+0000","last_active":"2024-01-15T17:55:06.789136+0000","last_peered":"2024-01-15T17:55:06.789136+0000","last_clean":"2024-01-15T17:55:06.789136+0000","last_became_active":"2024-01-15T17:55:04.750216+0000","last_became_peered":"2024-01-15T17:55:04.750216+0000","last_unstale":"2024-01-15T17:55:06.789136+0000","last_undegraded":"2024-01-15T17:55:06.789136+0000","last_fullsized":"2024-01-15T17:55:06.789136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790494+0000","last_change":"2024-01-15T17:55:04.758699+0000","last_active":"2024-01-15T17:55:06.790494+0000","last_peered":"2024-01-15T17:55:06.790494+0000","last_clean":"2024-01-15T17:55:06.790494+0000","last_became_active":"2024-01-15T17:55:04.758066+0000","last_became_peered":"2024-01-15T17:55:04.758066+0000","last_unstale":"2024-01-15T17:55:06.790494+0000","last_undegraded":"2024-01-15T17:55:06.790494+0000","last_fullsized":"2024-01-15T17:55:06.790494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:31:30.922640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727023+0000","last_change":"2024-01-15T17:55:04.749062+0000","last_active":"2024-01-15T17:55:05.727023+0000","last_peered":"2024-01-15T17:55:05.727023+0000","last_clean":"2024-01-15T17:55:05.727023+0000","last_became_active":"2024-01-15T17:55:04.746787+0000","last_became_peered":"2024-01-15T17:55:04.746787+0000","last_unstale":"2024-01-15T17:55:05.727023+0000","last_undegraded":"2024-01-15T17:55:05.727023+0000","last_fullsized":"2024-01-15T17:55:05.727023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721723+0000","last_change":"2024-01-15T17:55:04.753444+0000","last_active":"2024-01-15T17:55:05.721723+0000","last_peered":"2024-01-15T17:55:05.721723+0000","last_clean":"2024-01-15T17:55:05.721723+0000","last_became_active":"2024-01-15T17:55:04.753162+0000","last_became_peered":"2024-01-15T17:55:04.753162+0000","last_unstale":"2024-01-15T17:55:05.721723+0000","last_undegraded":"2024-01-15T17:55:05.721723+0000","last_fullsized":"2024-01-15T17:55:05.721723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790061+0000","last_change":"2024-01-15T17:55:04.764187+0000","last_active":"2024-01-15T17:55:06.790061+0000","last_peered":"2024-01-15T17:55:06.790061+0000","last_clean":"2024-01-15T17:55:06.790061+0000","last_became_active":"2024-01-15T17:55:04.763904+0000","last_became_peered":"2024-01-15T17:55:04.763904+0000","last_unstale":"2024-01-15T17:55:06.790061+0000","last_undegraded":"2024-01-15T17:55:06.790061+0000","last_fullsized":"2024-01-15T17:55:06.790061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:21:21.061668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788913+0000","last_change":"2024-01-15T17:55:04.769007+0000","last_active":"2024-01-15T17:55:06.788913+0000","last_peered":"2024-01-15T17:55:06.788913+0000","last_clean":"2024-01-15T17:55:06.788913+0000","last_became_active":"2024-01-15T17:55:04.767135+0000","last_became_peered":"2024-01-15T17:55:04.767135+0000","last_unstale":"2024-01-15T17:55:06.788913+0000","last_undegraded":"2024-01-15T17:55:06.788913+0000","last_fullsized":"2024-01-15T17:55:06.788913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789565+0000","last_change":"2024-01-15T17:55:04.751046+0000","last_active":"2024-01-15T17:55:06.789565+0000","last_peered":"2024-01-15T17:55:06.789565+0000","last_clean":"2024-01-15T17:55:06.789565+0000","last_became_active":"2024-01-15T17:55:04.750724+0000","last_became_peered":"2024-01-15T17:55:04.750724+0000","last_unstale":"2024-01-15T17:55:06.789565+0000","last_undegraded":"2024-01-15T17:55:06.789565+0000","last_fullsized":"2024-01-15T17:55:06.789565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:59.870088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787248+0000","last_change":"2024-01-15T17:55:04.732768+0000","last_active":"2024-01-15T17:55:06.787248+0000","last_peered":"2024-01-15T17:55:06.787248+0000","last_clean":"2024-01-15T17:55:06.787248+0000","last_became_active":"2024-01-15T17:55:04.732209+0000","last_became_peered":"2024-01-15T17:55:04.732209+0000","last_unstale":"2024-01-15T17:55:06.787248+0000","last_undegraded":"2024-01-15T17:55:06.787248+0000","last_fullsized":"2024-01-15T17:55:06.787248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:21.113714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.724905+0000","last_change":"2024-01-15T17:55:04.762744+0000","last_active":"2024-01-15T17:55:05.724905+0000","last_peered":"2024-01-15T17:55:05.724905+0000","last_clean":"2024-01-15T17:55:05.724905+0000","last_became_active":"2024-01-15T17:55:04.762234+0000","last_became_peered":"2024-01-15T17:55:04.762234+0000","last_unstale":"2024-01-15T17:55:05.724905+0000","last_undegraded":"2024-01-15T17:55:05.724905+0000","last_fullsized":"2024-01-15T17:55:05.724905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:10.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725553+0000","last_change":"2024-01-15T17:55:04.741937+0000","last_active":"2024-01-15T17:55:05.725553+0000","last_peered":"2024-01-15T17:55:05.725553+0000","last_clean":"2024-01-15T17:55:05.725553+0000","last_became_active":"2024-01-15T17:55:04.741629+0000","last_became_peered":"2024-01-15T17:55:04.741629+0000","last_unstale":"2024-01-15T17:55:05.725553+0000","last_undegraded":"2024-01-15T17:55:05.725553+0000","last_fullsized":"2024-01-15T17:55:05.725553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:30.356212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789134+0000","last_change":"2024-01-15T17:55:03.753426+0000","last_active":"2024-01-15T17:55:06.789134+0000","last_peered":"2024-01-15T17:55:06.789134+0000","last_clean":"2024-01-15T17:55:06.789134+0000","last_became_active":"2024-01-15T17:55:03.753141+0000","last_became_peered":"2024-01-15T17:55:03.753141+0000","last_unstale":"2024-01-15T17:55:06.789134+0000","last_undegraded":"2024-01-15T17:55:06.789134+0000","last_fullsized":"2024-01-15T17:55:06.789134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:38.142877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789190+0000","last_change":"2024-01-15T17:55:04.760185+0000","last_active":"2024-01-15T17:55:06.789190+0000","last_peered":"2024-01-15T17:55:06.789190+0000","last_clean":"2024-01-15T17:55:06.789190+0000","last_became_active":"2024-01-15T17:55:04.759883+0000","last_became_peered":"2024-01-15T17:55:04.759883+0000","last_unstale":"2024-01-15T17:55:06.789190+0000","last_undegraded":"2024-01-15T17:55:06.789190+0000","last_fullsized":"2024-01-15T17:55:06.789190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725694+0000","last_change":"2024-01-15T17:55:03.747953+0000","last_active":"2024-01-15T17:55:05.725694+0000","last_peered":"2024-01-15T17:55:05.725694+0000","last_clean":"2024-01-15T17:55:05.725694+0000","last_became_active":"2024-01-15T17:55:03.747648+0000","last_became_peered":"2024-01-15T17:55:03.747648+0000","last_unstale":"2024-01-15T17:55:05.725694+0000","last_undegraded":"2024-01-15T17:55:05.725694+0000","last_fullsized":"2024-01-15T17:55:05.725694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:54.384035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789495+0000","last_change":"2024-01-15T17:55:03.743175+0000","last_active":"2024-01-15T17:55:06.789495+0000","last_peered":"2024-01-15T17:55:06.789495+0000","last_clean":"2024-01-15T17:55:06.789495+0000","last_became_active":"2024-01-15T17:55:03.742887+0000","last_became_peered":"2024-01-15T17:55:03.742887+0000","last_unstale":"2024-01-15T17:55:06.789495+0000","last_undegraded":"2024-01-15T17:55:06.789495+0000","last_fullsized":"2024-01-15T17:55:06.789495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:03.274467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788556+0000","last_change":"2024-01-15T17:55:04.767350+0000","last_active":"2024-01-15T17:55:06.788556+0000","last_peered":"2024-01-15T17:55:06.788556+0000","last_clean":"2024-01-15T17:55:06.788556+0000","last_became_active":"2024-01-15T17:55:04.767083+0000","last_became_peered":"2024-01-15T17:55:04.767083+0000","last_unstale":"2024-01-15T17:55:06.788556+0000","last_undegraded":"2024-01-15T17:55:06.788556+0000","last_fullsized":"2024-01-15T17:55:06.788556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:01.549303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789294+0000","last_change":"2024-01-15T17:55:03.744198+0000","last_active":"2024-01-15T17:55:06.789294+0000","last_peered":"2024-01-15T17:55:06.789294+0000","last_clean":"2024-01-15T17:55:06.789294+0000","last_became_active":"2024-01-15T17:55:03.743838+0000","last_became_peered":"2024-01-15T17:55:03.743838+0000","last_unstale":"2024-01-15T17:55:06.789294+0000","last_undegraded":"2024-01-15T17:55:06.789294+0000","last_fullsized":"2024-01-15T17:55:06.789294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:19.912976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721537+0000","last_change":"2024-01-15T17:55:04.751636+0000","last_active":"2024-01-15T17:55:05.721537+0000","last_peered":"2024-01-15T17:55:05.721537+0000","last_clean":"2024-01-15T17:55:05.721537+0000","last_became_active":"2024-01-15T17:55:04.751353+0000","last_became_peered":"2024-01-15T17:55:04.751353+0000","last_unstale":"2024-01-15T17:55:05.721537+0000","last_undegraded":"2024-01-15T17:55:05.721537+0000","last_fullsized":"2024-01-15T17:55:05.721537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727326+0000","last_change":"2024-01-15T17:55:03.749204+0000","last_active":"2024-01-15T17:55:05.727326+0000","last_peered":"2024-01-15T17:55:05.727326+0000","last_clean":"2024-01-15T17:55:05.727326+0000","last_became_active":"2024-01-15T17:55:03.748941+0000","last_became_peered":"2024-01-15T17:55:03.748941+0000","last_unstale":"2024-01-15T17:55:05.727326+0000","last_undegraded":"2024-01-15T17:55:05.727326+0000","last_fullsized":"2024-01-15T17:55:05.727326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790411+0000","last_change":"2024-01-15T17:55:04.743467+0000","last_active":"2024-01-15T17:55:06.790411+0000","last_peered":"2024-01-15T17:55:06.790411+0000","last_clean":"2024-01-15T17:55:06.790411+0000","last_became_active":"2024-01-15T17:55:04.743161+0000","last_became_peered":"2024-01-15T17:55:04.743161+0000","last_unstale":"2024-01-15T17:55:06.790411+0000","last_undegraded":"2024-01-15T17:55:06.790411+0000","last_fullsized":"2024-01-15T17:55:06.790411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:00.908270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790246+0000","last_change":"2024-01-15T17:55:03.757214+0000","last_active":"2024-01-15T17:55:06.790246+0000","last_peered":"2024-01-15T17:55:06.790246+0000","last_clean":"2024-01-15T17:55:06.790246+0000","last_became_active":"2024-01-15T17:55:03.756900+0000","last_became_peered":"2024-01-15T17:55:03.756900+0000","last_unstale":"2024-01-15T17:55:06.790246+0000","last_undegraded":"2024-01-15T17:55:06.790246+0000","last_fullsized":"2024-01-15T17:55:06.790246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:19.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788839+0000","last_change":"2024-01-15T17:55:04.758670+0000","last_active":"2024-01-15T17:55:06.788839+0000","last_peered":"2024-01-15T17:55:06.788839+0000","last_clean":"2024-01-15T17:55:06.788839+0000","last_became_active":"2024-01-15T17:55:04.757659+0000","last_became_peered":"2024-01-15T17:55:04.757659+0000","last_unstale":"2024-01-15T17:55:06.788839+0000","last_undegraded":"2024-01-15T17:55:06.788839+0000","last_fullsized":"2024-01-15T17:55:06.788839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721816+0000","last_change":"2024-01-15T17:55:03.746513+0000","last_active":"2024-01-15T17:55:05.721816+0000","last_peered":"2024-01-15T17:55:05.721816+0000","last_clean":"2024-01-15T17:55:05.721816+0000","last_became_active":"2024-01-15T17:55:03.746147+0000","last_became_peered":"2024-01-15T17:55:03.746147+0000","last_unstale":"2024-01-15T17:55:05.721816+0000","last_undegraded":"2024-01-15T17:55:05.721816+0000","last_fullsized":"2024-01-15T17:55:05.721816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726333+0000","last_change":"2024-01-15T17:55:04.752615+0000","last_active":"2024-01-15T17:55:05.726333+0000","last_peered":"2024-01-15T17:55:05.726333+0000","last_clean":"2024-01-15T17:55:05.726333+0000","last_became_active":"2024-01-15T17:55:04.751582+0000","last_became_peered":"2024-01-15T17:55:04.751582+0000","last_unstale":"2024-01-15T17:55:05.726333+0000","last_undegraded":"2024-01-15T17:55:05.726333+0000","last_fullsized":"2024-01-15T17:55:05.726333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:24:25.848060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726379+0000","last_change":"2024-01-15T17:55:03.741760+0000","last_active":"2024-01-15T17:55:05.726379+0000","last_peered":"2024-01-15T17:55:05.726379+0000","last_clean":"2024-01-15T17:55:05.726379+0000","last_became_active":"2024-01-15T17:55:03.741327+0000","last_became_peered":"2024-01-15T17:55:03.741327+0000","last_unstale":"2024-01-15T17:55:05.726379+0000","last_undegraded":"2024-01-15T17:55:05.726379+0000","last_fullsized":"2024-01-15T17:55:05.726379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590242+0000","last_change":"2024-01-15T17:55:04.761237+0000","last_active":"2024-01-15T17:55:06.590242+0000","last_peered":"2024-01-15T17:55:06.590242+0000","last_clean":"2024-01-15T17:55:06.590242+0000","last_became_active":"2024-01-15T17:55:04.759551+0000","last_became_peered":"2024-01-15T17:55:04.759551+0000","last_unstale":"2024-01-15T17:55:06.590242+0000","last_undegraded":"2024-01-15T17:55:06.590242+0000","last_fullsized":"2024-01-15T17:55:06.590242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:17.567853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789391+0000","last_change":"2024-01-15T17:55:03.733161+0000","last_active":"2024-01-15T17:55:06.789391+0000","last_peered":"2024-01-15T17:55:06.789391+0000","last_clean":"2024-01-15T17:55:06.789391+0000","last_became_active":"2024-01-15T17:55:03.732839+0000","last_became_peered":"2024-01-15T17:55:03.732839+0000","last_unstale":"2024-01-15T17:55:06.789391+0000","last_undegraded":"2024-01-15T17:55:06.789391+0000","last_fullsized":"2024-01-15T17:55:06.789391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:17.540533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788038+0000","last_change":"2024-01-15T17:55:04.759368+0000","last_active":"2024-01-15T17:55:06.788038+0000","last_peered":"2024-01-15T17:55:06.788038+0000","last_clean":"2024-01-15T17:55:06.788038+0000","last_became_active":"2024-01-15T17:55:04.759069+0000","last_became_peered":"2024-01-15T17:55:04.759069+0000","last_unstale":"2024-01-15T17:55:06.788038+0000","last_undegraded":"2024-01-15T17:55:06.788038+0000","last_fullsized":"2024-01-15T17:55:06.788038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:10.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789653+0000","last_change":"2024-01-15T17:55:03.746380+0000","last_active":"2024-01-15T17:55:06.789653+0000","last_peered":"2024-01-15T17:55:06.789653+0000","last_clean":"2024-01-15T17:55:06.789653+0000","last_became_active":"2024-01-15T17:55:03.745986+0000","last_became_peered":"2024-01-15T17:55:03.745986+0000","last_unstale":"2024-01-15T17:55:06.789653+0000","last_undegraded":"2024-01-15T17:55:06.789653+0000","last_fullsized":"2024-01-15T17:55:06.789653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590808+0000","last_change":"2024-01-15T17:55:04.751522+0000","last_active":"2024-01-15T17:55:06.590808+0000","last_peered":"2024-01-15T17:55:06.590808+0000","last_clean":"2024-01-15T17:55:06.590808+0000","last_became_active":"2024-01-15T17:55:04.751235+0000","last_became_peered":"2024-01-15T17:55:04.751235+0000","last_unstale":"2024-01-15T17:55:06.590808+0000","last_undegraded":"2024-01-15T17:55:06.590808+0000","last_fullsized":"2024-01-15T17:55:06.590808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:00.908270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725362+0000","last_change":"2024-01-15T17:55:03.751331+0000","last_active":"2024-01-15T17:55:05.725362+0000","last_peered":"2024-01-15T17:55:05.725362+0000","last_clean":"2024-01-15T17:55:05.725362+0000","last_became_active":"2024-01-15T17:55:03.749400+0000","last_became_peered":"2024-01-15T17:55:03.749400+0000","last_unstale":"2024-01-15T17:55:05.725362+0000","last_undegraded":"2024-01-15T17:55:05.725362+0000","last_fullsized":"2024-01-15T17:55:05.725362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.724907+0000","last_change":"2024-01-15T17:55:04.761956+0000","last_active":"2024-01-15T17:55:05.724907+0000","last_peered":"2024-01-15T17:55:05.724907+0000","last_clean":"2024-01-15T17:55:05.724907+0000","last_became_active":"2024-01-15T17:55:04.761741+0000","last_became_peered":"2024-01-15T17:55:04.761741+0000","last_unstale":"2024-01-15T17:55:05.724907+0000","last_undegraded":"2024-01-15T17:55:05.724907+0000","last_fullsized":"2024-01-15T17:55:05.724907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787164+0000","last_change":"2024-01-15T17:55:03.735909+0000","last_active":"2024-01-15T17:55:06.787164+0000","last_peered":"2024-01-15T17:55:06.787164+0000","last_clean":"2024-01-15T17:55:06.787164+0000","last_became_active":"2024-01-15T17:55:03.734252+0000","last_became_peered":"2024-01-15T17:55:03.734252+0000","last_unstale":"2024-01-15T17:55:06.787164+0000","last_undegraded":"2024-01-15T17:55:06.787164+0000","last_fullsized":"2024-01-15T17:55:06.787164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590233+0000","last_change":"2024-01-15T17:55:04.753688+0000","last_active":"2024-01-15T17:55:06.590233+0000","last_peered":"2024-01-15T17:55:06.590233+0000","last_clean":"2024-01-15T17:55:06.590233+0000","last_became_active":"2024-01-15T17:55:04.753479+0000","last_became_peered":"2024-01-15T17:55:04.753479+0000","last_unstale":"2024-01-15T17:55:06.590233+0000","last_undegraded":"2024-01-15T17:55:06.590233+0000","last_fullsized":"2024-01-15T17:55:06.590233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:52.765350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726589+0000","last_change":"2024-01-15T17:55:03.740438+0000","last_active":"2024-01-15T17:55:05.726589+0000","last_peered":"2024-01-15T17:55:05.726589+0000","last_clean":"2024-01-15T17:55:05.726589+0000","last_became_active":"2024-01-15T17:55:03.737923+0000","last_became_peered":"2024-01-15T17:55:03.737923+0000","last_unstale":"2024-01-15T17:55:05.726589+0000","last_undegraded":"2024-01-15T17:55:05.726589+0000","last_fullsized":"2024-01-15T17:55:05.726589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591190+0000","last_change":"2024-01-15T17:55:04.756054+0000","last_active":"2024-01-15T17:55:06.591190+0000","last_peered":"2024-01-15T17:55:06.591190+0000","last_clean":"2024-01-15T17:55:06.591190+0000","last_became_active":"2024-01-15T17:55:04.754276+0000","last_became_peered":"2024-01-15T17:55:04.754276+0000","last_unstale":"2024-01-15T17:55:06.591190+0000","last_undegraded":"2024-01-15T17:55:06.591190+0000","last_fullsized":"2024-01-15T17:55:06.591190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:38.181261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787840+0000","last_change":"2024-01-15T17:55:03.744207+0000","last_active":"2024-01-15T17:55:06.787840+0000","last_peered":"2024-01-15T17:55:06.787840+0000","last_clean":"2024-01-15T17:55:06.787840+0000","last_became_active":"2024-01-15T17:55:03.743894+0000","last_became_peered":"2024-01-15T17:55:03.743894+0000","last_unstale":"2024-01-15T17:55:06.787840+0000","last_undegraded":"2024-01-15T17:55:06.787840+0000","last_fullsized":"2024-01-15T17:55:06.787840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591002+0000","last_change":"2024-01-15T17:55:04.758291+0000","last_active":"2024-01-15T17:55:06.591002+0000","last_peered":"2024-01-15T17:55:06.591002+0000","last_clean":"2024-01-15T17:55:06.591002+0000","last_became_active":"2024-01-15T17:55:04.757888+0000","last_became_peered":"2024-01-15T17:55:04.757888+0000","last_unstale":"2024-01-15T17:55:06.591002+0000","last_undegraded":"2024-01-15T17:55:06.591002+0000","last_fullsized":"2024-01-15T17:55:06.591002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:34:45.528954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721818+0000","last_change":"2024-01-15T17:55:03.744276+0000","last_active":"2024-01-15T17:55:05.721818+0000","last_peered":"2024-01-15T17:55:05.721818+0000","last_clean":"2024-01-15T17:55:05.721818+0000","last_became_active":"2024-01-15T17:55:03.744033+0000","last_became_peered":"2024-01-15T17:55:03.744033+0000","last_unstale":"2024-01-15T17:55:05.721818+0000","last_undegraded":"2024-01-15T17:55:05.721818+0000","last_fullsized":"2024-01-15T17:55:05.721818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:17.540533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790107+0000","last_change":"2024-01-15T17:55:04.751400+0000","last_active":"2024-01-15T17:55:06.790107+0000","last_peered":"2024-01-15T17:55:06.790107+0000","last_clean":"2024-01-15T17:55:06.790107+0000","last_became_active":"2024-01-15T17:55:04.751097+0000","last_became_peered":"2024-01-15T17:55:04.751097+0000","last_unstale":"2024-01-15T17:55:06.790107+0000","last_undegraded":"2024-01-15T17:55:06.790107+0000","last_fullsized":"2024-01-15T17:55:06.790107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:56:52.262406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787591+0000","last_change":"2024-01-15T17:55:03.751285+0000","last_active":"2024-01-15T17:55:06.787591+0000","last_peered":"2024-01-15T17:55:06.787591+0000","last_clean":"2024-01-15T17:55:06.787591+0000","last_became_active":"2024-01-15T17:55:03.750926+0000","last_became_peered":"2024-01-15T17:55:03.750926+0000","last_unstale":"2024-01-15T17:55:06.787591+0000","last_undegraded":"2024-01-15T17:55:06.787591+0000","last_fullsized":"2024-01-15T17:55:06.787591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590723+0000","last_change":"2024-01-15T17:55:04.741297+0000","last_active":"2024-01-15T17:55:06.590723+0000","last_peered":"2024-01-15T17:55:06.590723+0000","last_clean":"2024-01-15T17:55:06.590723+0000","last_became_active":"2024-01-15T17:55:04.741044+0000","last_became_peered":"2024-01-15T17:55:04.741044+0000","last_unstale":"2024-01-15T17:55:06.590723+0000","last_undegraded":"2024-01-15T17:55:06.590723+0000","last_fullsized":"2024-01-15T17:55:06.590723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:56:52.262406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-15T17:55:04.731969+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:55:04.731969+0000","last_peered":"2024-01-15T17:55:04.731969+0000","last_clean":"2024-01-15T17:55:04.731969+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:55:04.731969+0000","last_undegraded":"2024-01-15T17:55:04.731969+0000","last_fullsized":"2024-01-15T17:55:04.731969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789182+0000","last_change":"2024-01-15T17:55:03.744683+0000","last_active":"2024-01-15T17:55:06.789182+0000","last_peered":"2024-01-15T17:55:06.789182+0000","last_clean":"2024-01-15T17:55:06.789182+0000","last_became_active":"2024-01-15T17:55:03.743028+0000","last_became_peered":"2024-01-15T17:55:03.743028+0000","last_unstale":"2024-01-15T17:55:06.789182+0000","last_undegraded":"2024-01-15T17:55:06.789182+0000","last_fullsized":"2024-01-15T17:55:06.789182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790120+0000","last_change":"2024-01-15T17:55:04.755598+0000","last_active":"2024-01-15T17:55:06.790120+0000","last_peered":"2024-01-15T17:55:06.790120+0000","last_clean":"2024-01-15T17:55:06.790120+0000","last_became_active":"2024-01-15T17:55:04.755327+0000","last_became_peered":"2024-01-15T17:55:04.755327+0000","last_unstale":"2024-01-15T17:55:06.790120+0000","last_undegraded":"2024-01-15T17:55:06.790120+0000","last_fullsized":"2024-01-15T17:55:06.790120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:47.198851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589792+0000","last_change":"2024-01-15T17:55:03.755061+0000","last_active":"2024-01-15T17:55:06.589792+0000","last_peered":"2024-01-15T17:55:06.589792+0000","last_clean":"2024-01-15T17:55:06.589792+0000","last_became_active":"2024-01-15T17:55:03.754761+0000","last_became_peered":"2024-01-15T17:55:03.754761+0000","last_unstale":"2024-01-15T17:55:06.589792+0000","last_undegraded":"2024-01-15T17:55:06.589792+0000","last_fullsized":"2024-01-15T17:55:06.589792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787429+0000","last_change":"2024-01-15T17:55:04.760196+0000","last_active":"2024-01-15T17:55:06.787429+0000","last_peered":"2024-01-15T17:55:06.787429+0000","last_clean":"2024-01-15T17:55:06.787429+0000","last_became_active":"2024-01-15T17:55:04.759902+0000","last_became_peered":"2024-01-15T17:55:04.759902+0000","last_unstale":"2024-01-15T17:55:06.787429+0000","last_undegraded":"2024-01-15T17:55:06.787429+0000","last_fullsized":"2024-01-15T17:55:06.787429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:57.337185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787323+0000","last_change":"2024-01-15T17:55:03.741130+0000","last_active":"2024-01-15T17:55:06.787323+0000","last_peered":"2024-01-15T17:55:06.787323+0000","last_clean":"2024-01-15T17:55:06.787323+0000","last_became_active":"2024-01-15T17:55:03.740791+0000","last_became_peered":"2024-01-15T17:55:03.740791+0000","last_unstale":"2024-01-15T17:55:06.787323+0000","last_undegraded":"2024-01-15T17:55:06.787323+0000","last_fullsized":"2024-01-15T17:55:06.787323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:57.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721182+0000","last_change":"2024-01-15T17:55:04.739187+0000","last_active":"2024-01-15T17:55:05.721182+0000","last_peered":"2024-01-15T17:55:05.721182+0000","last_clean":"2024-01-15T17:55:05.721182+0000","last_became_active":"2024-01-15T17:55:04.738700+0000","last_became_peered":"2024-01-15T17:55:04.738700+0000","last_unstale":"2024-01-15T17:55:05.721182+0000","last_undegraded":"2024-01-15T17:55:05.721182+0000","last_fullsized":"2024-01-15T17:55:05.721182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790088+0000","last_change":"2024-01-15T17:55:03.741719+0000","last_active":"2024-01-15T17:55:06.790088+0000","last_peered":"2024-01-15T17:55:06.790088+0000","last_clean":"2024-01-15T17:55:06.790088+0000","last_became_active":"2024-01-15T17:55:03.739882+0000","last_became_peered":"2024-01-15T17:55:03.739882+0000","last_unstale":"2024-01-15T17:55:06.790088+0000","last_undegraded":"2024-01-15T17:55:06.790088+0000","last_fullsized":"2024-01-15T17:55:06.790088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725696+0000","last_change":"2024-01-15T17:55:04.766676+0000","last_active":"2024-01-15T17:55:05.725696+0000","last_peered":"2024-01-15T17:55:05.725696+0000","last_clean":"2024-01-15T17:55:05.725696+0000","last_became_active":"2024-01-15T17:55:04.766394+0000","last_became_peered":"2024-01-15T17:55:04.766394+0000","last_unstale":"2024-01-15T17:55:05.725696+0000","last_undegraded":"2024-01-15T17:55:05.725696+0000","last_fullsized":"2024-01-15T17:55:05.725696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:52.036315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721394+0000","last_change":"2024-01-15T17:55:03.735528+0000","last_active":"2024-01-15T17:55:05.721394+0000","last_peered":"2024-01-15T17:55:05.721394+0000","last_clean":"2024-01-15T17:55:05.721394+0000","last_became_active":"2024-01-15T17:55:03.733799+0000","last_became_peered":"2024-01-15T17:55:03.733799+0000","last_unstale":"2024-01-15T17:55:05.721394+0000","last_undegraded":"2024-01-15T17:55:05.721394+0000","last_fullsized":"2024-01-15T17:55:05.721394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789930+0000","last_change":"2024-01-15T17:55:04.750889+0000","last_active":"2024-01-15T17:55:06.789930+0000","last_peered":"2024-01-15T17:55:06.789930+0000","last_clean":"2024-01-15T17:55:06.789930+0000","last_became_active":"2024-01-15T17:55:04.750480+0000","last_became_peered":"2024-01-15T17:55:04.750480+0000","last_unstale":"2024-01-15T17:55:06.789930+0000","last_undegraded":"2024-01-15T17:55:06.789930+0000","last_fullsized":"2024-01-15T17:55:06.789930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:13.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590950+0000","last_change":"2024-01-15T17:55:03.758172+0000","last_active":"2024-01-15T17:55:06.590950+0000","last_peered":"2024-01-15T17:55:06.590950+0000","last_clean":"2024-01-15T17:55:06.590950+0000","last_became_active":"2024-01-15T17:55:03.757742+0000","last_became_peered":"2024-01-15T17:55:03.757742+0000","last_unstale":"2024-01-15T17:55:06.590950+0000","last_undegraded":"2024-01-15T17:55:06.590950+0000","last_fullsized":"2024-01-15T17:55:06.590950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:17.081091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721370+0000","last_change":"2024-01-15T17:55:04.729485+0000","last_active":"2024-01-15T17:55:05.721370+0000","last_peered":"2024-01-15T17:55:05.721370+0000","last_clean":"2024-01-15T17:55:05.721370+0000","last_became_active":"2024-01-15T17:55:04.729029+0000","last_became_peered":"2024-01-15T17:55:04.729029+0000","last_unstale":"2024-01-15T17:55:05.721370+0000","last_undegraded":"2024-01-15T17:55:05.721370+0000","last_fullsized":"2024-01-15T17:55:05.721370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.956145+0000","last_change":"2024-01-15T17:55:03.752377+0000","last_active":"2024-01-15T17:55:06.956145+0000","last_peered":"2024-01-15T17:55:06.956145+0000","last_clean":"2024-01-15T17:55:06.956145+0000","last_became_active":"2024-01-15T17:55:03.752164+0000","last_became_peered":"2024-01-15T17:55:03.752164+0000","last_unstale":"2024-01-15T17:55:06.956145+0000","last_undegraded":"2024-01-15T17:55:06.956145+0000","last_fullsized":"2024-01-15T17:55:06.956145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790469+0000","last_change":"2024-01-15T17:55:04.758991+0000","last_active":"2024-01-15T17:55:06.790469+0000","last_peered":"2024-01-15T17:55:06.790469+0000","last_clean":"2024-01-15T17:55:06.790469+0000","last_became_active":"2024-01-15T17:55:04.758422+0000","last_became_peered":"2024-01-15T17:55:04.758422+0000","last_unstale":"2024-01-15T17:55:06.790469+0000","last_undegraded":"2024-01-15T17:55:06.790469+0000","last_fullsized":"2024-01-15T17:55:06.790469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:01:41.493666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727355+0000","last_change":"2024-01-15T17:55:03.754630+0000","last_active":"2024-01-15T17:55:05.727355+0000","last_peered":"2024-01-15T17:55:05.727355+0000","last_clean":"2024-01-15T17:55:05.727355+0000","last_became_active":"2024-01-15T17:55:03.754117+0000","last_became_peered":"2024-01-15T17:55:03.754117+0000","last_unstale":"2024-01-15T17:55:05.727355+0000","last_undegraded":"2024-01-15T17:55:05.727355+0000","last_fullsized":"2024-01-15T17:55:05.727355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:26.025029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788431+0000","last_change":"2024-01-15T17:55:04.754711+0000","last_active":"2024-01-15T17:55:06.788431+0000","last_peered":"2024-01-15T17:55:06.788431+0000","last_clean":"2024-01-15T17:55:06.788431+0000","last_became_active":"2024-01-15T17:55:04.753730+0000","last_became_peered":"2024-01-15T17:55:04.753730+0000","last_unstale":"2024-01-15T17:55:06.788431+0000","last_undegraded":"2024-01-15T17:55:06.788431+0000","last_fullsized":"2024-01-15T17:55:06.788431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589725+0000","last_change":"2024-01-15T17:55:03.755088+0000","last_active":"2024-01-15T17:55:06.589725+0000","last_peered":"2024-01-15T17:55:06.589725+0000","last_clean":"2024-01-15T17:55:06.589725+0000","last_became_active":"2024-01-15T17:55:03.754805+0000","last_became_peered":"2024-01-15T17:55:03.754805+0000","last_unstale":"2024-01-15T17:55:06.589725+0000","last_undegraded":"2024-01-15T17:55:06.589725+0000","last_fullsized":"2024-01-15T17:55:06.589725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725215+0000","last_change":"2024-01-15T17:55:04.747604+0000","last_active":"2024-01-15T17:55:05.725215+0000","last_peered":"2024-01-15T17:55:05.725215+0000","last_clean":"2024-01-15T17:55:05.725215+0000","last_became_active":"2024-01-15T17:55:04.746687+0000","last_became_peered":"2024-01-15T17:55:04.746687+0000","last_unstale":"2024-01-15T17:55:05.725215+0000","last_undegraded":"2024-01-15T17:55:05.725215+0000","last_fullsized":"2024-01-15T17:55:05.725215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:58.689690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721658+0000","last_change":"2024-01-15T17:55:03.749885+0000","last_active":"2024-01-15T17:55:05.721658+0000","last_peered":"2024-01-15T17:55:05.721658+0000","last_clean":"2024-01-15T17:55:05.721658+0000","last_became_active":"2024-01-15T17:55:03.749600+0000","last_became_peered":"2024-01-15T17:55:03.749600+0000","last_unstale":"2024-01-15T17:55:05.721658+0000","last_undegraded":"2024-01-15T17:55:05.721658+0000","last_fullsized":"2024-01-15T17:55:05.721658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:09.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788706+0000","last_change":"2024-01-15T17:55:04.736486+0000","last_active":"2024-01-15T17:55:06.788706+0000","last_peered":"2024-01-15T17:55:06.788706+0000","last_clean":"2024-01-15T17:55:06.788706+0000","last_became_active":"2024-01-15T17:55:04.736012+0000","last_became_peered":"2024-01-15T17:55:04.736012+0000","last_unstale":"2024-01-15T17:55:06.788706+0000","last_undegraded":"2024-01-15T17:55:06.788706+0000","last_fullsized":"2024-01-15T17:55:06.788706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:53:52.730234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721367+0000","last_change":"2024-01-15T17:55:03.738319+0000","last_active":"2024-01-15T17:55:05.721367+0000","last_peered":"2024-01-15T17:55:05.721367+0000","last_clean":"2024-01-15T17:55:05.721367+0000","last_became_active":"2024-01-15T17:55:03.737946+0000","last_became_peered":"2024-01-15T17:55:03.737946+0000","last_unstale":"2024-01-15T17:55:05.721367+0000","last_undegraded":"2024-01-15T17:55:05.721367+0000","last_fullsized":"2024-01-15T17:55:05.721367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:57.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789078+0000","last_change":"2024-01-15T17:55:04.766996+0000","last_active":"2024-01-15T17:55:06.789078+0000","last_peered":"2024-01-15T17:55:06.789078+0000","last_clean":"2024-01-15T17:55:06.789078+0000","last_became_active":"2024-01-15T17:55:04.766695+0000","last_became_peered":"2024-01-15T17:55:04.766695+0000","last_unstale":"2024-01-15T17:55:06.789078+0000","last_undegraded":"2024-01-15T17:55:06.789078+0000","last_fullsized":"2024-01-15T17:55:06.789078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:21.570608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590906+0000","last_change":"2024-01-15T17:55:03.757123+0000","last_active":"2024-01-15T17:55:06.590906+0000","last_peered":"2024-01-15T17:55:06.590906+0000","last_clean":"2024-01-15T17:55:06.590906+0000","last_became_active":"2024-01-15T17:55:03.756522+0000","last_became_peered":"2024-01-15T17:55:03.756522+0000","last_unstale":"2024-01-15T17:55:06.590906+0000","last_undegraded":"2024-01-15T17:55:06.590906+0000","last_fullsized":"2024-01-15T17:55:06.590906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725414+0000","last_change":"2024-01-15T17:55:04.756691+0000","last_active":"2024-01-15T17:55:05.725414+0000","last_peered":"2024-01-15T17:55:05.725414+0000","last_clean":"2024-01-15T17:55:05.725414+0000","last_became_active":"2024-01-15T17:55:04.756410+0000","last_became_peered":"2024-01-15T17:55:04.756410+0000","last_unstale":"2024-01-15T17:55:05.725414+0000","last_undegraded":"2024-01-15T17:55:05.725414+0000","last_fullsized":"2024-01-15T17:55:05.725414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789737+0000","last_change":"2024-01-15T17:55:04.747374+0000","last_active":"2024-01-15T17:55:06.789737+0000","last_peered":"2024-01-15T17:55:06.789737+0000","last_clean":"2024-01-15T17:55:06.789737+0000","last_became_active":"2024-01-15T17:55:04.747062+0000","last_became_peered":"2024-01-15T17:55:04.747062+0000","last_unstale":"2024-01-15T17:55:06.789737+0000","last_undegraded":"2024-01-15T17:55:06.789737+0000","last_fullsized":"2024-01-15T17:55:06.789737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:44.725915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590116+0000","last_change":"2024-01-15T17:55:03.759441+0000","last_active":"2024-01-15T17:55:06.590116+0000","last_peered":"2024-01-15T17:55:06.590116+0000","last_clean":"2024-01-15T17:55:06.590116+0000","last_became_active":"2024-01-15T17:55:03.759054+0000","last_became_peered":"2024-01-15T17:55:03.759054+0000","last_unstale":"2024-01-15T17:55:06.590116+0000","last_undegraded":"2024-01-15T17:55:06.590116+0000","last_fullsized":"2024-01-15T17:55:06.590116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:19.912976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789716+0000","last_change":"2024-01-15T17:55:04.743796+0000","last_active":"2024-01-15T17:55:06.789716+0000","last_peered":"2024-01-15T17:55:06.789716+0000","last_clean":"2024-01-15T17:55:06.789716+0000","last_became_active":"2024-01-15T17:55:04.743475+0000","last_became_peered":"2024-01-15T17:55:04.743475+0000","last_unstale":"2024-01-15T17:55:06.789716+0000","last_undegraded":"2024-01-15T17:55:06.789716+0000","last_fullsized":"2024-01-15T17:55:06.789716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:52.765350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590613+0000","last_change":"2024-01-15T17:55:03.759401+0000","last_active":"2024-01-15T17:55:06.590613+0000","last_peered":"2024-01-15T17:55:06.590613+0000","last_clean":"2024-01-15T17:55:06.590613+0000","last_became_active":"2024-01-15T17:55:03.759124+0000","last_became_peered":"2024-01-15T17:55:03.759124+0000","last_unstale":"2024-01-15T17:55:06.590613+0000","last_undegraded":"2024-01-15T17:55:06.590613+0000","last_fullsized":"2024-01-15T17:55:06.590613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:09.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788680+0000","last_change":"2024-01-15T17:55:04.757926+0000","last_active":"2024-01-15T17:55:06.788680+0000","last_peered":"2024-01-15T17:55:06.788680+0000","last_clean":"2024-01-15T17:55:06.788680+0000","last_became_active":"2024-01-15T17:55:04.757625+0000","last_became_peered":"2024-01-15T17:55:04.757625+0000","last_unstale":"2024-01-15T17:55:06.788680+0000","last_undegraded":"2024-01-15T17:55:06.788680+0000","last_fullsized":"2024-01-15T17:55:06.788680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790277+0000","last_change":"2024-01-15T17:55:03.736897+0000","last_active":"2024-01-15T17:55:06.790277+0000","last_peered":"2024-01-15T17:55:06.790277+0000","last_clean":"2024-01-15T17:55:06.790277+0000","last_became_active":"2024-01-15T17:55:03.736576+0000","last_became_peered":"2024-01-15T17:55:03.736576+0000","last_unstale":"2024-01-15T17:55:06.790277+0000","last_undegraded":"2024-01-15T17:55:06.790277+0000","last_fullsized":"2024-01-15T17:55:06.790277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:56:59.201106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790294+0000","last_change":"2024-01-15T17:55:04.760100+0000","last_active":"2024-01-15T17:55:06.790294+0000","last_peered":"2024-01-15T17:55:06.790294+0000","last_clean":"2024-01-15T17:55:06.790294+0000","last_became_active":"2024-01-15T17:55:04.756345+0000","last_became_peered":"2024-01-15T17:55:04.756345+0000","last_unstale":"2024-01-15T17:55:06.790294+0000","last_undegraded":"2024-01-15T17:55:06.790294+0000","last_fullsized":"2024-01-15T17:55:06.790294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:06.327821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788935+0000","last_change":"2024-01-15T17:55:03.750992+0000","last_active":"2024-01-15T17:55:06.788935+0000","last_peered":"2024-01-15T17:55:06.788935+0000","last_clean":"2024-01-15T17:55:06.788935+0000","last_became_active":"2024-01-15T17:55:03.750662+0000","last_became_peered":"2024-01-15T17:55:03.750662+0000","last_unstale":"2024-01-15T17:55:06.788935+0000","last_undegraded":"2024-01-15T17:55:06.788935+0000","last_fullsized":"2024-01-15T17:55:06.788935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":823296,"data_stored":669826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.629}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0979999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.036}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1589999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":78966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.835}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":79002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7410000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":78876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":831488,"data_stored":669882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9139999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:55:14.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:55:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.068+0000 7fa736000640 1 Processor -- start 2024-01-15T17:55:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.068+0000 7fa736000640 1 -- start start 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.070+0000 7fa736000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa730160920 0x7fa7300b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.070+0000 7fa736000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7300cf9f0 0x7fa7300cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.070+0000 7fa736000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7300d0a10 con 0x7fa730161010 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.070+0000 7fa736000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7300d03b0 con 0x7fa7300cf9f0 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.070+0000 7fa736000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300d1f50 con 0x7fa730160920 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.070+0000 7fa72fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7300cf9f0 0x7fa7300cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.070+0000 7fa72fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7300cf9f0 0x7fa7300cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44590/0 (socket says 172.21.15.26:44590) 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.070+0000 7fa72fe00640 1 -- 172.21.15.26:0/1840855845 learned_addr learned my addr 172.21.15.26:0/1840855845 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.070+0000 7fa735600640 1 --1- 172.21.15.26:0/1840855845 >> v1:172.21.15.26:6789/0 conn(0x7fa730161010 0x7fa7301614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43024/0 (socket says 172.21.15.26:43024) 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.071+0000 7fa734c00640 1 --2- 172.21.15.26:0/1840855845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa730160920 0x7fa7300b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.071+0000 7fa72fe00640 1 -- 172.21.15.26:0/1840855845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa730160920 msgr2=0x7fa7300b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.071+0000 7fa72fe00640 1 --2- 172.21.15.26:0/1840855845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa730160920 0x7fa7300b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.071+0000 7fa72fe00640 1 -- 172.21.15.26:0/1840855845 >> v1:172.21.15.26:6789/0 conn(0x7fa730161010 legacy=0x7fa7301614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.071+0000 7fa72fe00640 1 -- 172.21.15.26:0/1840855845 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7300cf080 con 0x7fa7300cf9f0 2024-01-15T17:55:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.072+0000 7fa72fe00640 1 --2- 172.21.15.26:0/1840855845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7300cf9f0 0x7fa7300cfe70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa718009870 tx=0x7fa71802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a394e2ec7bf919 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.072+0000 7fa72f400640 1 -- 172.21.15.26:0/1840855845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718030840 con 0x7fa7300cf9f0 2024-01-15T17:55:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.072+0000 7fa72f400640 1 -- 172.21.15.26:0/1840855845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7180309a0 con 0x7fa7300cf9f0 2024-01-15T17:55:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.072+0000 7fa72f400640 1 -- 172.21.15.26:0/1840855845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718030d30 con 0x7fa7300cf9f0 2024-01-15T17:55:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.073+0000 7fa736000640 1 -- 172.21.15.26:0/1840855845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7300cf9f0 msgr2=0x7fa7300cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.073+0000 7fa736000640 1 --2- 172.21.15.26:0/1840855845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7300cf9f0 0x7fa7300cfe70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa718009870 tx=0x7fa71802b980 comp rx=0 tx=0).stop 2024-01-15T17:55:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.073+0000 7fa736000640 1 -- 172.21.15.26:0/1840855845 shutdown_connections 2024-01-15T17:55:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.073+0000 7fa736000640 1 --2- 172.21.15.26:0/1840855845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7300cf9f0 0x7fa7300cfe70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.073+0000 7fa736000640 1 --2- 172.21.15.26:0/1840855845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa730160920 0x7fa7300b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.073+0000 7fa736000640 1 -- 172.21.15.26:0/1840855845 >> 172.21.15.26:0/1840855845 conn(0x7fa7300d3090 msgr2=0x7fa7300d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.073+0000 7fa736000640 1 -- 172.21.15.26:0/1840855845 shutdown_connections 2024-01-15T17:55:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.073+0000 7fa736000640 1 -- 172.21.15.26:0/1840855845 wait complete. 2024-01-15T17:55:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.075+0000 7fa736000640 1 Processor -- start 2024-01-15T17:55:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.075+0000 7fa736000640 1 -- start start 2024-01-15T17:55:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa736000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7300cf9f0 0x7fa7300b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa736000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa730160920 0x7fa73014b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa736000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa730161010 0x7fa730157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa736000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa730155930 con 0x7fa730160920 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa736000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300b7e40 con 0x7fa7300cf9f0 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa736000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7300b6860 con 0x7fa730161010 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa72fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa730160920 0x7fa73014b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa72fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa730160920 0x7fa73014b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44596/0 (socket says 172.21.15.26:44596) 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa72fe00640 1 -- 172.21.15.26:0/638577116 learned_addr learned my addr 172.21.15.26:0/638577116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.076+0000 7fa734c00640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7300cf9f0 0x7fa7300b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa72fe00640 1 -- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa730161010 msgr2=0x7fa730157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa72fe00640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa730161010 0x7fa730157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa72fe00640 1 -- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7300cf9f0 msgr2=0x7fa7300b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa72fe00640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7300cf9f0 0x7fa7300b7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa72fe00640 1 -- 172.21.15.26:0/638577116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180094d0 con 0x7fa730160920 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa72fe00640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa730160920 0x7fa73014b280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa718009870 tx=0x7fa718004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa72cc00640 1 -- 172.21.15.26:0/638577116 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718039a90 con 0x7fa730160920 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa72cc00640 1 -- 172.21.15.26:0/638577116 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa718039bf0 con 0x7fa730160920 2024-01-15T17:55:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7300b6b40 con 0x7fa730160920 2024-01-15T17:55:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa72cc00640 1 -- 172.21.15.26:0/638577116 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180433f0 con 0x7fa730160920 2024-01-15T17:55:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.077+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7300b7100 con 0x7fa730160920 2024-01-15T17:55:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.079+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa730067450 con 0x7fa730160920 2024-01-15T17:55:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.079+0000 7fa72cc00640 1 -- 172.21.15.26:0/638577116 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa71804c3b0 con 0x7fa730160920 2024-01-15T17:55:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.080+0000 7fa72cc00640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa70806b2c0 0x7fa70806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.080+0000 7fa72cc00640 1 -- 172.21.15.26:0/638577116 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7180abff0 con 0x7fa730160920 2024-01-15T17:55:15.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.081+0000 7fa734c00640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa70806b2c0 0x7fa70806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.082+0000 7fa734c00640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa70806b2c0 0x7fa70806d780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa720006fd0 tx=0x7fa720008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:15.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.085+0000 7fa72cc00640 1 -- 172.21.15.26:0/638577116 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa718076d90 con 0x7fa730160920 2024-01-15T17:55:15.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.269+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7300d0c60 con 0x7fa70806b2c0 2024-01-15T17:55:15.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.289+0000 7fa72cc00640 1 -- 172.21.15.26:0/638577116 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264330 (secure 0 0 0) 0x7fa7300d0c60 con 0x7fa70806b2c0 2024-01-15T17:55:15.296 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:15.298 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:55:15.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa70806b2c0 msgr2=0x7fa70806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa70806b2c0 0x7fa70806d780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa720006fd0 tx=0x7fa720008040 comp rx=0 tx=0).stop 2024-01-15T17:55:15.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa730160920 msgr2=0x7fa73014b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa730160920 0x7fa73014b280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa718009870 tx=0x7fa718004730 comp rx=0 tx=0).stop 2024-01-15T17:55:15.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 shutdown_connections 2024-01-15T17:55:15.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa730161010 0x7fa730157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa70806b2c0 0x7fa70806d780 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa730160920 0x7fa73014b280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 --2- 172.21.15.26:0/638577116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7300cf9f0 0x7fa7300b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.295+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 >> 172.21.15.26:0/638577116 conn(0x7fa7300d3090 msgr2=0x7fa73006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:15.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.296+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 shutdown_connections 2024-01-15T17:55:15.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.296+0000 7fa736000640 1 -- 172.21.15.26:0/638577116 wait complete. 2024-01-15T17:55:15.316 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-01-15T17:55:13.912863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236188,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738532,"statfs":{"total":773094113280,"available":772852256768,"internally_reserved":0,"allocated":3055616,"data_stored":1816564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.194431"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590593+0000","last_change":"2024-01-15T17:55:04.755200+0000","last_active":"2024-01-15T17:55:06.590593+0000","last_peered":"2024-01-15T17:55:06.590593+0000","last_clean":"2024-01-15T17:55:06.590593+0000","last_became_active":"2024-01-15T17:55:04.753634+0000","last_became_peered":"2024-01-15T17:55:04.753634+0000","last_unstale":"2024-01-15T17:55:06.590593+0000","last_undegraded":"2024-01-15T17:55:06.590593+0000","last_fullsized":"2024-01-15T17:55:06.590593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:13.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721381+0000","last_change":"2024-01-15T17:55:04.760607+0000","last_active":"2024-01-15T17:55:05.721381+0000","last_peered":"2024-01-15T17:55:05.721381+0000","last_clean":"2024-01-15T17:55:05.721381+0000","last_became_active":"2024-01-15T17:55:04.760299+0000","last_became_peered":"2024-01-15T17:55:04.760299+0000","last_unstale":"2024-01-15T17:55:05.721381+0000","last_undegraded":"2024-01-15T17:55:05.721381+0000","last_fullsized":"2024-01-15T17:55:05.721381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787705+0000","last_change":"2024-01-15T17:55:04.755000+0000","last_active":"2024-01-15T17:55:06.787705+0000","last_peered":"2024-01-15T17:55:06.787705+0000","last_clean":"2024-01-15T17:55:06.787705+0000","last_became_active":"2024-01-15T17:55:04.754716+0000","last_became_peered":"2024-01-15T17:55:04.754716+0000","last_unstale":"2024-01-15T17:55:06.787705+0000","last_undegraded":"2024-01-15T17:55:06.787705+0000","last_fullsized":"2024-01-15T17:55:06.787705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:55.584774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789609+0000","last_change":"2024-01-15T17:55:04.767626+0000","last_active":"2024-01-15T17:55:06.789609+0000","last_peered":"2024-01-15T17:55:06.789609+0000","last_clean":"2024-01-15T17:55:06.789609+0000","last_became_active":"2024-01-15T17:55:04.767349+0000","last_became_peered":"2024-01-15T17:55:04.767349+0000","last_unstale":"2024-01-15T17:55:06.789609+0000","last_undegraded":"2024-01-15T17:55:06.789609+0000","last_fullsized":"2024-01-15T17:55:06.789609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:17.567853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590635+0000","last_change":"2024-01-15T17:55:04.756971+0000","last_active":"2024-01-15T17:55:06.590635+0000","last_peered":"2024-01-15T17:55:06.590635+0000","last_clean":"2024-01-15T17:55:06.590635+0000","last_became_active":"2024-01-15T17:55:04.756737+0000","last_became_peered":"2024-01-15T17:55:04.756737+0000","last_unstale":"2024-01-15T17:55:06.590635+0000","last_undegraded":"2024-01-15T17:55:06.590635+0000","last_fullsized":"2024-01-15T17:55:06.590635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:44.725915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789155+0000","last_change":"2024-01-15T17:55:04.763592+0000","last_active":"2024-01-15T17:55:06.789155+0000","last_peered":"2024-01-15T17:55:06.789155+0000","last_clean":"2024-01-15T17:55:06.789155+0000","last_became_active":"2024-01-15T17:55:04.763286+0000","last_became_peered":"2024-01-15T17:55:04.763286+0000","last_unstale":"2024-01-15T17:55:06.789155+0000","last_undegraded":"2024-01-15T17:55:06.789155+0000","last_fullsized":"2024-01-15T17:55:06.789155+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:48:48.227594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589901+0000","last_change":"2024-01-15T17:55:04.754040+0000","last_active":"2024-01-15T17:55:06.589901+0000","last_peered":"2024-01-15T17:55:06.589901+0000","last_clean":"2024-01-15T17:55:06.589901+0000","last_became_active":"2024-01-15T17:55:04.753698+0000","last_became_peered":"2024-01-15T17:55:04.753698+0000","last_unstale":"2024-01-15T17:55:06.589901+0000","last_undegraded":"2024-01-15T17:55:06.589901+0000","last_fullsized":"2024-01-15T17:55:06.589901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:59.870088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788324+0000","last_change":"2024-01-15T17:55:04.754498+0000","last_active":"2024-01-15T17:55:06.788324+0000","last_peered":"2024-01-15T17:55:06.788324+0000","last_clean":"2024-01-15T17:55:06.788324+0000","last_became_active":"2024-01-15T17:55:04.754091+0000","last_became_peered":"2024-01-15T17:55:04.754091+0000","last_unstale":"2024-01-15T17:55:06.788324+0000","last_undegraded":"2024-01-15T17:55:06.788324+0000","last_fullsized":"2024-01-15T17:55:06.788324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:50:31.493057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590727+0000","last_change":"2024-01-15T17:55:04.755593+0000","last_active":"2024-01-15T17:55:06.590727+0000","last_peered":"2024-01-15T17:55:06.590727+0000","last_clean":"2024-01-15T17:55:06.590727+0000","last_became_active":"2024-01-15T17:55:04.755376+0000","last_became_peered":"2024-01-15T17:55:04.755376+0000","last_unstale":"2024-01-15T17:55:06.590727+0000","last_undegraded":"2024-01-15T17:55:06.590727+0000","last_fullsized":"2024-01-15T17:55:06.590727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:10:13.660425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726844+0000","last_change":"2024-01-15T17:55:04.745309+0000","last_active":"2024-01-15T17:55:05.726844+0000","last_peered":"2024-01-15T17:55:05.726844+0000","last_clean":"2024-01-15T17:55:05.726844+0000","last_became_active":"2024-01-15T17:55:04.744930+0000","last_became_peered":"2024-01-15T17:55:04.744930+0000","last_unstale":"2024-01-15T17:55:05.726844+0000","last_undegraded":"2024-01-15T17:55:05.726844+0000","last_fullsized":"2024-01-15T17:55:05.726844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726624+0000","last_change":"2024-01-15T17:55:04.743285+0000","last_active":"2024-01-15T17:55:05.726624+0000","last_peered":"2024-01-15T17:55:05.726624+0000","last_clean":"2024-01-15T17:55:05.726624+0000","last_became_active":"2024-01-15T17:55:04.743118+0000","last_became_peered":"2024-01-15T17:55:04.743118+0000","last_unstale":"2024-01-15T17:55:05.726624+0000","last_undegraded":"2024-01-15T17:55:05.726624+0000","last_fullsized":"2024-01-15T17:55:05.726624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726121+0000","last_change":"2024-01-15T17:55:04.739598+0000","last_active":"2024-01-15T17:55:05.726121+0000","last_peered":"2024-01-15T17:55:05.726121+0000","last_clean":"2024-01-15T17:55:05.726121+0000","last_became_active":"2024-01-15T17:55:04.739319+0000","last_became_peered":"2024-01-15T17:55:04.739319+0000","last_unstale":"2024-01-15T17:55:05.726121+0000","last_undegraded":"2024-01-15T17:55:05.726121+0000","last_fullsized":"2024-01-15T17:55:05.726121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:21.113714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788978+0000","last_change":"2024-01-15T17:55:04.751187+0000","last_active":"2024-01-15T17:55:06.788978+0000","last_peered":"2024-01-15T17:55:06.788978+0000","last_clean":"2024-01-15T17:55:06.788978+0000","last_became_active":"2024-01-15T17:55:04.750057+0000","last_became_peered":"2024-01-15T17:55:04.750057+0000","last_unstale":"2024-01-15T17:55:06.788978+0000","last_undegraded":"2024-01-15T17:55:06.788978+0000","last_fullsized":"2024-01-15T17:55:06.788978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:22:07.251406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787842+0000","last_change":"2024-01-15T17:55:04.759443+0000","last_active":"2024-01-15T17:55:06.787842+0000","last_peered":"2024-01-15T17:55:06.787842+0000","last_clean":"2024-01-15T17:55:06.787842+0000","last_became_active":"2024-01-15T17:55:04.759086+0000","last_became_peered":"2024-01-15T17:55:04.759086+0000","last_unstale":"2024-01-15T17:55:06.787842+0000","last_undegraded":"2024-01-15T17:55:06.787842+0000","last_fullsized":"2024-01-15T17:55:06.787842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:27.225767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721835+0000","last_change":"2024-01-15T17:55:04.741854+0000","last_active":"2024-01-15T17:55:05.721835+0000","last_peered":"2024-01-15T17:55:05.721835+0000","last_clean":"2024-01-15T17:55:05.721835+0000","last_became_active":"2024-01-15T17:55:04.741472+0000","last_became_peered":"2024-01-15T17:55:04.741472+0000","last_unstale":"2024-01-15T17:55:05.721835+0000","last_undegraded":"2024-01-15T17:55:05.721835+0000","last_fullsized":"2024-01-15T17:55:05.721835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789012+0000","last_change":"2024-01-15T17:55:04.757919+0000","last_active":"2024-01-15T17:55:06.789012+0000","last_peered":"2024-01-15T17:55:06.789012+0000","last_clean":"2024-01-15T17:55:06.789012+0000","last_became_active":"2024-01-15T17:55:04.757595+0000","last_became_peered":"2024-01-15T17:55:04.757595+0000","last_unstale":"2024-01-15T17:55:06.789012+0000","last_undegraded":"2024-01-15T17:55:06.789012+0000","last_fullsized":"2024-01-15T17:55:06.789012+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790339+0000","last_change":"2024-01-15T17:55:04.745091+0000","last_active":"2024-01-15T17:55:06.790339+0000","last_peered":"2024-01-15T17:55:06.790339+0000","last_clean":"2024-01-15T17:55:06.790339+0000","last_became_active":"2024-01-15T17:55:04.744779+0000","last_became_peered":"2024-01-15T17:55:04.744779+0000","last_unstale":"2024-01-15T17:55:06.790339+0000","last_undegraded":"2024-01-15T17:55:06.790339+0000","last_fullsized":"2024-01-15T17:55:06.790339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:10:13.660425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787739+0000","last_change":"2024-01-15T17:55:04.754772+0000","last_active":"2024-01-15T17:55:06.787739+0000","last_peered":"2024-01-15T17:55:06.787739+0000","last_clean":"2024-01-15T17:55:06.787739+0000","last_became_active":"2024-01-15T17:55:04.743420+0000","last_became_peered":"2024-01-15T17:55:04.743420+0000","last_unstale":"2024-01-15T17:55:06.787739+0000","last_undegraded":"2024-01-15T17:55:06.787739+0000","last_fullsized":"2024-01-15T17:55:06.787739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:24:25.848060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591083+0000","last_change":"2024-01-15T17:55:04.741262+0000","last_active":"2024-01-15T17:55:06.591083+0000","last_peered":"2024-01-15T17:55:06.591083+0000","last_clean":"2024-01-15T17:55:06.591083+0000","last_became_active":"2024-01-15T17:55:04.740978+0000","last_became_peered":"2024-01-15T17:55:04.740978+0000","last_unstale":"2024-01-15T17:55:06.591083+0000","last_undegraded":"2024-01-15T17:55:06.591083+0000","last_fullsized":"2024-01-15T17:55:06.591083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:00.893650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726416+0000","last_change":"2024-01-15T17:55:04.758887+0000","last_active":"2024-01-15T17:55:05.726416+0000","last_peered":"2024-01-15T17:55:05.726416+0000","last_clean":"2024-01-15T17:55:05.726416+0000","last_became_active":"2024-01-15T17:55:04.758530+0000","last_became_peered":"2024-01-15T17:55:04.758530+0000","last_unstale":"2024-01-15T17:55:05.726416+0000","last_undegraded":"2024-01-15T17:55:05.726416+0000","last_fullsized":"2024-01-15T17:55:05.726416+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788603+0000","last_change":"2024-01-15T17:55:04.756875+0000","last_active":"2024-01-15T17:55:06.788603+0000","last_peered":"2024-01-15T17:55:06.788603+0000","last_clean":"2024-01-15T17:55:06.788603+0000","last_became_active":"2024-01-15T17:55:04.756596+0000","last_became_peered":"2024-01-15T17:55:04.756596+0000","last_unstale":"2024-01-15T17:55:06.788603+0000","last_undegraded":"2024-01-15T17:55:06.788603+0000","last_fullsized":"2024-01-15T17:55:06.788603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789372+0000","last_change":"2024-01-15T17:55:04.746119+0000","last_active":"2024-01-15T17:55:06.789372+0000","last_peered":"2024-01-15T17:55:06.789372+0000","last_clean":"2024-01-15T17:55:06.789372+0000","last_became_active":"2024-01-15T17:55:04.745830+0000","last_became_peered":"2024-01-15T17:55:04.745830+0000","last_unstale":"2024-01-15T17:55:06.789372+0000","last_undegraded":"2024-01-15T17:55:06.789372+0000","last_fullsized":"2024-01-15T17:55:06.789372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:47:39.573232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789300+0000","last_change":"2024-01-15T17:55:04.764403+0000","last_active":"2024-01-15T17:55:06.789300+0000","last_peered":"2024-01-15T17:55:06.789300+0000","last_clean":"2024-01-15T17:55:06.789300+0000","last_became_active":"2024-01-15T17:55:04.763976+0000","last_became_peered":"2024-01-15T17:55:04.763976+0000","last_unstale":"2024-01-15T17:55:06.789300+0000","last_undegraded":"2024-01-15T17:55:06.789300+0000","last_fullsized":"2024-01-15T17:55:06.789300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:08:09.187744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789136+0000","last_change":"2024-01-15T17:55:04.750998+0000","last_active":"2024-01-15T17:55:06.789136+0000","last_peered":"2024-01-15T17:55:06.789136+0000","last_clean":"2024-01-15T17:55:06.789136+0000","last_became_active":"2024-01-15T17:55:04.750216+0000","last_became_peered":"2024-01-15T17:55:04.750216+0000","last_unstale":"2024-01-15T17:55:06.789136+0000","last_undegraded":"2024-01-15T17:55:06.789136+0000","last_fullsized":"2024-01-15T17:55:06.789136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790494+0000","last_change":"2024-01-15T17:55:04.758699+0000","last_active":"2024-01-15T17:55:06.790494+0000","last_peered":"2024-01-15T17:55:06.790494+0000","last_clean":"2024-01-15T17:55:06.790494+0000","last_became_active":"2024-01-15T17:55:04.758066+0000","last_became_peered":"2024-01-15T17:55:04.758066+0000","last_unstale":"2024-01-15T17:55:06.790494+0000","last_undegraded":"2024-01-15T17:55:06.790494+0000","last_fullsized":"2024-01-15T17:55:06.790494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:31:30.922640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727023+0000","last_change":"2024-01-15T17:55:04.749062+0000","last_active":"2024-01-15T17:55:05.727023+0000","last_peered":"2024-01-15T17:55:05.727023+0000","last_clean":"2024-01-15T17:55:05.727023+0000","last_became_active":"2024-01-15T17:55:04.746787+0000","last_became_peered":"2024-01-15T17:55:04.746787+0000","last_unstale":"2024-01-15T17:55:05.727023+0000","last_undegraded":"2024-01-15T17:55:05.727023+0000","last_fullsized":"2024-01-15T17:55:05.727023+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721723+0000","last_change":"2024-01-15T17:55:04.753444+0000","last_active":"2024-01-15T17:55:05.721723+0000","last_peered":"2024-01-15T17:55:05.721723+0000","last_clean":"2024-01-15T17:55:05.721723+0000","last_became_active":"2024-01-15T17:55:04.753162+0000","last_became_peered":"2024-01-15T17:55:04.753162+0000","last_unstale":"2024-01-15T17:55:05.721723+0000","last_undegraded":"2024-01-15T17:55:05.721723+0000","last_fullsized":"2024-01-15T17:55:05.721723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790061+0000","last_change":"2024-01-15T17:55:04.764187+0000","last_active":"2024-01-15T17:55:06.790061+0000","last_peered":"2024-01-15T17:55:06.790061+0000","last_clean":"2024-01-15T17:55:06.790061+0000","last_became_active":"2024-01-15T17:55:04.763904+0000","last_became_peered":"2024-01-15T17:55:04.763904+0000","last_unstale":"2024-01-15T17:55:06.790061+0000","last_undegraded":"2024-01-15T17:55:06.790061+0000","last_fullsized":"2024-01-15T17:55:06.790061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:21:21.061668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788913+0000","last_change":"2024-01-15T17:55:04.769007+0000","last_active":"2024-01-15T17:55:06.788913+0000","last_peered":"2024-01-15T17:55:06.788913+0000","last_clean":"2024-01-15T17:55:06.788913+0000","last_became_active":"2024-01-15T17:55:04.767135+0000","last_became_peered":"2024-01-15T17:55:04.767135+0000","last_unstale":"2024-01-15T17:55:06.788913+0000","last_undegraded":"2024-01-15T17:55:06.788913+0000","last_fullsized":"2024-01-15T17:55:06.788913+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:38:51.068753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789565+0000","last_change":"2024-01-15T17:55:04.751046+0000","last_active":"2024-01-15T17:55:06.789565+0000","last_peered":"2024-01-15T17:55:06.789565+0000","last_clean":"2024-01-15T17:55:06.789565+0000","last_became_active":"2024-01-15T17:55:04.750724+0000","last_became_peered":"2024-01-15T17:55:04.750724+0000","last_unstale":"2024-01-15T17:55:06.789565+0000","last_undegraded":"2024-01-15T17:55:06.789565+0000","last_fullsized":"2024-01-15T17:55:06.789565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:10:59.870088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787248+0000","last_change":"2024-01-15T17:55:04.732768+0000","last_active":"2024-01-15T17:55:06.787248+0000","last_peered":"2024-01-15T17:55:06.787248+0000","last_clean":"2024-01-15T17:55:06.787248+0000","last_became_active":"2024-01-15T17:55:04.732209+0000","last_became_peered":"2024-01-15T17:55:04.732209+0000","last_unstale":"2024-01-15T17:55:06.787248+0000","last_undegraded":"2024-01-15T17:55:06.787248+0000","last_fullsized":"2024-01-15T17:55:06.787248+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:21.113714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.724905+0000","last_change":"2024-01-15T17:55:04.762744+0000","last_active":"2024-01-15T17:55:05.724905+0000","last_peered":"2024-01-15T17:55:05.724905+0000","last_clean":"2024-01-15T17:55:05.724905+0000","last_became_active":"2024-01-15T17:55:04.762234+0000","last_became_peered":"2024-01-15T17:55:04.762234+0000","last_unstale":"2024-01-15T17:55:05.724905+0000","last_undegraded":"2024-01-15T17:55:05.724905+0000","last_fullsized":"2024-01-15T17:55:05.724905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:10.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725553+0000","last_change":"2024-01-15T17:55:04.741937+0000","last_active":"2024-01-15T17:55:05.725553+0000","last_peered":"2024-01-15T17:55:05.725553+0000","last_clean":"2024-01-15T17:55:05.725553+0000","last_became_active":"2024-01-15T17:55:04.741629+0000","last_became_peered":"2024-01-15T17:55:04.741629+0000","last_unstale":"2024-01-15T17:55:05.725553+0000","last_undegraded":"2024-01-15T17:55:05.725553+0000","last_fullsized":"2024-01-15T17:55:05.725553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:30.356212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789134+0000","last_change":"2024-01-15T17:55:03.753426+0000","last_active":"2024-01-15T17:55:06.789134+0000","last_peered":"2024-01-15T17:55:06.789134+0000","last_clean":"2024-01-15T17:55:06.789134+0000","last_became_active":"2024-01-15T17:55:03.753141+0000","last_became_peered":"2024-01-15T17:55:03.753141+0000","last_unstale":"2024-01-15T17:55:06.789134+0000","last_undegraded":"2024-01-15T17:55:06.789134+0000","last_fullsized":"2024-01-15T17:55:06.789134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:38.142877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789190+0000","last_change":"2024-01-15T17:55:04.760185+0000","last_active":"2024-01-15T17:55:06.789190+0000","last_peered":"2024-01-15T17:55:06.789190+0000","last_clean":"2024-01-15T17:55:06.789190+0000","last_became_active":"2024-01-15T17:55:04.759883+0000","last_became_peered":"2024-01-15T17:55:04.759883+0000","last_unstale":"2024-01-15T17:55:06.789190+0000","last_undegraded":"2024-01-15T17:55:06.789190+0000","last_fullsized":"2024-01-15T17:55:06.789190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:22.194505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725694+0000","last_change":"2024-01-15T17:55:03.747953+0000","last_active":"2024-01-15T17:55:05.725694+0000","last_peered":"2024-01-15T17:55:05.725694+0000","last_clean":"2024-01-15T17:55:05.725694+0000","last_became_active":"2024-01-15T17:55:03.747648+0000","last_became_peered":"2024-01-15T17:55:03.747648+0000","last_unstale":"2024-01-15T17:55:05.725694+0000","last_undegraded":"2024-01-15T17:55:05.725694+0000","last_fullsized":"2024-01-15T17:55:05.725694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:54.384035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789495+0000","last_change":"2024-01-15T17:55:03.743175+0000","last_active":"2024-01-15T17:55:06.789495+0000","last_peered":"2024-01-15T17:55:06.789495+0000","last_clean":"2024-01-15T17:55:06.789495+0000","last_became_active":"2024-01-15T17:55:03.742887+0000","last_became_peered":"2024-01-15T17:55:03.742887+0000","last_unstale":"2024-01-15T17:55:06.789495+0000","last_undegraded":"2024-01-15T17:55:06.789495+0000","last_fullsized":"2024-01-15T17:55:06.789495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:03.274467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788556+0000","last_change":"2024-01-15T17:55:04.767350+0000","last_active":"2024-01-15T17:55:06.788556+0000","last_peered":"2024-01-15T17:55:06.788556+0000","last_clean":"2024-01-15T17:55:06.788556+0000","last_became_active":"2024-01-15T17:55:04.767083+0000","last_became_peered":"2024-01-15T17:55:04.767083+0000","last_unstale":"2024-01-15T17:55:06.788556+0000","last_undegraded":"2024-01-15T17:55:06.788556+0000","last_fullsized":"2024-01-15T17:55:06.788556+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:01.549303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789294+0000","last_change":"2024-01-15T17:55:03.744198+0000","last_active":"2024-01-15T17:55:06.789294+0000","last_peered":"2024-01-15T17:55:06.789294+0000","last_clean":"2024-01-15T17:55:06.789294+0000","last_became_active":"2024-01-15T17:55:03.743838+0000","last_became_peered":"2024-01-15T17:55:03.743838+0000","last_unstale":"2024-01-15T17:55:06.789294+0000","last_undegraded":"2024-01-15T17:55:06.789294+0000","last_fullsized":"2024-01-15T17:55:06.789294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:19.912976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721537+0000","last_change":"2024-01-15T17:55:04.751636+0000","last_active":"2024-01-15T17:55:05.721537+0000","last_peered":"2024-01-15T17:55:05.721537+0000","last_clean":"2024-01-15T17:55:05.721537+0000","last_became_active":"2024-01-15T17:55:04.751353+0000","last_became_peered":"2024-01-15T17:55:04.751353+0000","last_unstale":"2024-01-15T17:55:05.721537+0000","last_undegraded":"2024-01-15T17:55:05.721537+0000","last_fullsized":"2024-01-15T17:55:05.721537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727326+0000","last_change":"2024-01-15T17:55:03.749204+0000","last_active":"2024-01-15T17:55:05.727326+0000","last_peered":"2024-01-15T17:55:05.727326+0000","last_clean":"2024-01-15T17:55:05.727326+0000","last_became_active":"2024-01-15T17:55:03.748941+0000","last_became_peered":"2024-01-15T17:55:03.748941+0000","last_unstale":"2024-01-15T17:55:05.727326+0000","last_undegraded":"2024-01-15T17:55:05.727326+0000","last_fullsized":"2024-01-15T17:55:05.727326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790411+0000","last_change":"2024-01-15T17:55:04.743467+0000","last_active":"2024-01-15T17:55:06.790411+0000","last_peered":"2024-01-15T17:55:06.790411+0000","last_clean":"2024-01-15T17:55:06.790411+0000","last_became_active":"2024-01-15T17:55:04.743161+0000","last_became_peered":"2024-01-15T17:55:04.743161+0000","last_unstale":"2024-01-15T17:55:06.790411+0000","last_undegraded":"2024-01-15T17:55:06.790411+0000","last_fullsized":"2024-01-15T17:55:06.790411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:00.908270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790246+0000","last_change":"2024-01-15T17:55:03.757214+0000","last_active":"2024-01-15T17:55:06.790246+0000","last_peered":"2024-01-15T17:55:06.790246+0000","last_clean":"2024-01-15T17:55:06.790246+0000","last_became_active":"2024-01-15T17:55:03.756900+0000","last_became_peered":"2024-01-15T17:55:03.756900+0000","last_unstale":"2024-01-15T17:55:06.790246+0000","last_undegraded":"2024-01-15T17:55:06.790246+0000","last_fullsized":"2024-01-15T17:55:06.790246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:19.514066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788839+0000","last_change":"2024-01-15T17:55:04.758670+0000","last_active":"2024-01-15T17:55:06.788839+0000","last_peered":"2024-01-15T17:55:06.788839+0000","last_clean":"2024-01-15T17:55:06.788839+0000","last_became_active":"2024-01-15T17:55:04.757659+0000","last_became_peered":"2024-01-15T17:55:04.757659+0000","last_unstale":"2024-01-15T17:55:06.788839+0000","last_undegraded":"2024-01-15T17:55:06.788839+0000","last_fullsized":"2024-01-15T17:55:06.788839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:49:20.714805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721816+0000","last_change":"2024-01-15T17:55:03.746513+0000","last_active":"2024-01-15T17:55:05.721816+0000","last_peered":"2024-01-15T17:55:05.721816+0000","last_clean":"2024-01-15T17:55:05.721816+0000","last_became_active":"2024-01-15T17:55:03.746147+0000","last_became_peered":"2024-01-15T17:55:03.746147+0000","last_unstale":"2024-01-15T17:55:05.721816+0000","last_undegraded":"2024-01-15T17:55:05.721816+0000","last_fullsized":"2024-01-15T17:55:05.721816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726333+0000","last_change":"2024-01-15T17:55:04.752615+0000","last_active":"2024-01-15T17:55:05.726333+0000","last_peered":"2024-01-15T17:55:05.726333+0000","last_clean":"2024-01-15T17:55:05.726333+0000","last_became_active":"2024-01-15T17:55:04.751582+0000","last_became_peered":"2024-01-15T17:55:04.751582+0000","last_unstale":"2024-01-15T17:55:05.726333+0000","last_undegraded":"2024-01-15T17:55:05.726333+0000","last_fullsized":"2024-01-15T17:55:05.726333+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:24:25.848060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726379+0000","last_change":"2024-01-15T17:55:03.741760+0000","last_active":"2024-01-15T17:55:05.726379+0000","last_peered":"2024-01-15T17:55:05.726379+0000","last_clean":"2024-01-15T17:55:05.726379+0000","last_became_active":"2024-01-15T17:55:03.741327+0000","last_became_peered":"2024-01-15T17:55:03.741327+0000","last_unstale":"2024-01-15T17:55:05.726379+0000","last_undegraded":"2024-01-15T17:55:05.726379+0000","last_fullsized":"2024-01-15T17:55:05.726379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590242+0000","last_change":"2024-01-15T17:55:04.761237+0000","last_active":"2024-01-15T17:55:06.590242+0000","last_peered":"2024-01-15T17:55:06.590242+0000","last_clean":"2024-01-15T17:55:06.590242+0000","last_became_active":"2024-01-15T17:55:04.759551+0000","last_became_peered":"2024-01-15T17:55:04.759551+0000","last_unstale":"2024-01-15T17:55:06.590242+0000","last_undegraded":"2024-01-15T17:55:06.590242+0000","last_fullsized":"2024-01-15T17:55:06.590242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:17.567853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789391+0000","last_change":"2024-01-15T17:55:03.733161+0000","last_active":"2024-01-15T17:55:06.789391+0000","last_peered":"2024-01-15T17:55:06.789391+0000","last_clean":"2024-01-15T17:55:06.789391+0000","last_became_active":"2024-01-15T17:55:03.732839+0000","last_became_peered":"2024-01-15T17:55:03.732839+0000","last_unstale":"2024-01-15T17:55:06.789391+0000","last_undegraded":"2024-01-15T17:55:06.789391+0000","last_fullsized":"2024-01-15T17:55:06.789391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:17.540533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788038+0000","last_change":"2024-01-15T17:55:04.759368+0000","last_active":"2024-01-15T17:55:06.788038+0000","last_peered":"2024-01-15T17:55:06.788038+0000","last_clean":"2024-01-15T17:55:06.788038+0000","last_became_active":"2024-01-15T17:55:04.759069+0000","last_became_peered":"2024-01-15T17:55:04.759069+0000","last_unstale":"2024-01-15T17:55:06.788038+0000","last_undegraded":"2024-01-15T17:55:06.788038+0000","last_fullsized":"2024-01-15T17:55:06.788038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:10.846049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789653+0000","last_change":"2024-01-15T17:55:03.746380+0000","last_active":"2024-01-15T17:55:06.789653+0000","last_peered":"2024-01-15T17:55:06.789653+0000","last_clean":"2024-01-15T17:55:06.789653+0000","last_became_active":"2024-01-15T17:55:03.745986+0000","last_became_peered":"2024-01-15T17:55:03.745986+0000","last_unstale":"2024-01-15T17:55:06.789653+0000","last_undegraded":"2024-01-15T17:55:06.789653+0000","last_fullsized":"2024-01-15T17:55:06.789653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590808+0000","last_change":"2024-01-15T17:55:04.751522+0000","last_active":"2024-01-15T17:55:06.590808+0000","last_peered":"2024-01-15T17:55:06.590808+0000","last_clean":"2024-01-15T17:55:06.590808+0000","last_became_active":"2024-01-15T17:55:04.751235+0000","last_became_peered":"2024-01-15T17:55:04.751235+0000","last_unstale":"2024-01-15T17:55:06.590808+0000","last_undegraded":"2024-01-15T17:55:06.590808+0000","last_fullsized":"2024-01-15T17:55:06.590808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:49:00.908270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725362+0000","last_change":"2024-01-15T17:55:03.751331+0000","last_active":"2024-01-15T17:55:05.725362+0000","last_peered":"2024-01-15T17:55:05.725362+0000","last_clean":"2024-01-15T17:55:05.725362+0000","last_became_active":"2024-01-15T17:55:03.749400+0000","last_became_peered":"2024-01-15T17:55:03.749400+0000","last_unstale":"2024-01-15T17:55:05.725362+0000","last_undegraded":"2024-01-15T17:55:05.725362+0000","last_fullsized":"2024-01-15T17:55:05.725362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.724907+0000","last_change":"2024-01-15T17:55:04.761956+0000","last_active":"2024-01-15T17:55:05.724907+0000","last_peered":"2024-01-15T17:55:05.724907+0000","last_clean":"2024-01-15T17:55:05.724907+0000","last_became_active":"2024-01-15T17:55:04.761741+0000","last_became_peered":"2024-01-15T17:55:04.761741+0000","last_unstale":"2024-01-15T17:55:05.724907+0000","last_undegraded":"2024-01-15T17:55:05.724907+0000","last_fullsized":"2024-01-15T17:55:05.724907+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787164+0000","last_change":"2024-01-15T17:55:03.735909+0000","last_active":"2024-01-15T17:55:06.787164+0000","last_peered":"2024-01-15T17:55:06.787164+0000","last_clean":"2024-01-15T17:55:06.787164+0000","last_became_active":"2024-01-15T17:55:03.734252+0000","last_became_peered":"2024-01-15T17:55:03.734252+0000","last_unstale":"2024-01-15T17:55:06.787164+0000","last_undegraded":"2024-01-15T17:55:06.787164+0000","last_fullsized":"2024-01-15T17:55:06.787164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590233+0000","last_change":"2024-01-15T17:55:04.753688+0000","last_active":"2024-01-15T17:55:06.590233+0000","last_peered":"2024-01-15T17:55:06.590233+0000","last_clean":"2024-01-15T17:55:06.590233+0000","last_became_active":"2024-01-15T17:55:04.753479+0000","last_became_peered":"2024-01-15T17:55:04.753479+0000","last_unstale":"2024-01-15T17:55:06.590233+0000","last_undegraded":"2024-01-15T17:55:06.590233+0000","last_fullsized":"2024-01-15T17:55:06.590233+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:52.765350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.726589+0000","last_change":"2024-01-15T17:55:03.740438+0000","last_active":"2024-01-15T17:55:05.726589+0000","last_peered":"2024-01-15T17:55:05.726589+0000","last_clean":"2024-01-15T17:55:05.726589+0000","last_became_active":"2024-01-15T17:55:03.737923+0000","last_became_peered":"2024-01-15T17:55:03.737923+0000","last_unstale":"2024-01-15T17:55:05.726589+0000","last_undegraded":"2024-01-15T17:55:05.726589+0000","last_fullsized":"2024-01-15T17:55:05.726589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591190+0000","last_change":"2024-01-15T17:55:04.756054+0000","last_active":"2024-01-15T17:55:06.591190+0000","last_peered":"2024-01-15T17:55:06.591190+0000","last_clean":"2024-01-15T17:55:06.591190+0000","last_became_active":"2024-01-15T17:55:04.754276+0000","last_became_peered":"2024-01-15T17:55:04.754276+0000","last_unstale":"2024-01-15T17:55:06.591190+0000","last_undegraded":"2024-01-15T17:55:06.591190+0000","last_fullsized":"2024-01-15T17:55:06.591190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:33:38.181261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787840+0000","last_change":"2024-01-15T17:55:03.744207+0000","last_active":"2024-01-15T17:55:06.787840+0000","last_peered":"2024-01-15T17:55:06.787840+0000","last_clean":"2024-01-15T17:55:06.787840+0000","last_became_active":"2024-01-15T17:55:03.743894+0000","last_became_peered":"2024-01-15T17:55:03.743894+0000","last_unstale":"2024-01-15T17:55:06.787840+0000","last_undegraded":"2024-01-15T17:55:06.787840+0000","last_fullsized":"2024-01-15T17:55:06.787840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.591002+0000","last_change":"2024-01-15T17:55:04.758291+0000","last_active":"2024-01-15T17:55:06.591002+0000","last_peered":"2024-01-15T17:55:06.591002+0000","last_clean":"2024-01-15T17:55:06.591002+0000","last_became_active":"2024-01-15T17:55:04.757888+0000","last_became_peered":"2024-01-15T17:55:04.757888+0000","last_unstale":"2024-01-15T17:55:06.591002+0000","last_undegraded":"2024-01-15T17:55:06.591002+0000","last_fullsized":"2024-01-15T17:55:06.591002+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:34:45.528954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721818+0000","last_change":"2024-01-15T17:55:03.744276+0000","last_active":"2024-01-15T17:55:05.721818+0000","last_peered":"2024-01-15T17:55:05.721818+0000","last_clean":"2024-01-15T17:55:05.721818+0000","last_became_active":"2024-01-15T17:55:03.744033+0000","last_became_peered":"2024-01-15T17:55:03.744033+0000","last_unstale":"2024-01-15T17:55:05.721818+0000","last_undegraded":"2024-01-15T17:55:05.721818+0000","last_fullsized":"2024-01-15T17:55:05.721818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:17.540533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790107+0000","last_change":"2024-01-15T17:55:04.751400+0000","last_active":"2024-01-15T17:55:06.790107+0000","last_peered":"2024-01-15T17:55:06.790107+0000","last_clean":"2024-01-15T17:55:06.790107+0000","last_became_active":"2024-01-15T17:55:04.751097+0000","last_became_peered":"2024-01-15T17:55:04.751097+0000","last_unstale":"2024-01-15T17:55:06.790107+0000","last_undegraded":"2024-01-15T17:55:06.790107+0000","last_fullsized":"2024-01-15T17:55:06.790107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:56:52.262406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787591+0000","last_change":"2024-01-15T17:55:03.751285+0000","last_active":"2024-01-15T17:55:06.787591+0000","last_peered":"2024-01-15T17:55:06.787591+0000","last_clean":"2024-01-15T17:55:06.787591+0000","last_became_active":"2024-01-15T17:55:03.750926+0000","last_became_peered":"2024-01-15T17:55:03.750926+0000","last_unstale":"2024-01-15T17:55:06.787591+0000","last_undegraded":"2024-01-15T17:55:06.787591+0000","last_fullsized":"2024-01-15T17:55:06.787591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590723+0000","last_change":"2024-01-15T17:55:04.741297+0000","last_active":"2024-01-15T17:55:06.590723+0000","last_peered":"2024-01-15T17:55:06.590723+0000","last_clean":"2024-01-15T17:55:06.590723+0000","last_became_active":"2024-01-15T17:55:04.741044+0000","last_became_peered":"2024-01-15T17:55:04.741044+0000","last_unstale":"2024-01-15T17:55:06.590723+0000","last_undegraded":"2024-01-15T17:55:06.590723+0000","last_fullsized":"2024-01-15T17:55:06.590723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:56:52.262406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-01-15T17:55:04.731969+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:55:04.731969+0000","last_peered":"2024-01-15T17:55:04.731969+0000","last_clean":"2024-01-15T17:55:04.731969+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:55:04.731969+0000","last_undegraded":"2024-01-15T17:55:04.731969+0000","last_fullsized":"2024-01-15T17:55:04.731969+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789182+0000","last_change":"2024-01-15T17:55:03.744683+0000","last_active":"2024-01-15T17:55:06.789182+0000","last_peered":"2024-01-15T17:55:06.789182+0000","last_clean":"2024-01-15T17:55:06.789182+0000","last_became_active":"2024-01-15T17:55:03.743028+0000","last_became_peered":"2024-01-15T17:55:03.743028+0000","last_unstale":"2024-01-15T17:55:06.789182+0000","last_undegraded":"2024-01-15T17:55:06.789182+0000","last_fullsized":"2024-01-15T17:55:06.789182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:41.233221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790120+0000","last_change":"2024-01-15T17:55:04.755598+0000","last_active":"2024-01-15T17:55:06.790120+0000","last_peered":"2024-01-15T17:55:06.790120+0000","last_clean":"2024-01-15T17:55:06.790120+0000","last_became_active":"2024-01-15T17:55:04.755327+0000","last_became_peered":"2024-01-15T17:55:04.755327+0000","last_unstale":"2024-01-15T17:55:06.790120+0000","last_undegraded":"2024-01-15T17:55:06.790120+0000","last_fullsized":"2024-01-15T17:55:06.790120+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:47.198851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589792+0000","last_change":"2024-01-15T17:55:03.755061+0000","last_active":"2024-01-15T17:55:06.589792+0000","last_peered":"2024-01-15T17:55:06.589792+0000","last_clean":"2024-01-15T17:55:06.589792+0000","last_became_active":"2024-01-15T17:55:03.754761+0000","last_became_peered":"2024-01-15T17:55:03.754761+0000","last_unstale":"2024-01-15T17:55:06.589792+0000","last_undegraded":"2024-01-15T17:55:06.589792+0000","last_fullsized":"2024-01-15T17:55:06.589792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787429+0000","last_change":"2024-01-15T17:55:04.760196+0000","last_active":"2024-01-15T17:55:06.787429+0000","last_peered":"2024-01-15T17:55:06.787429+0000","last_clean":"2024-01-15T17:55:06.787429+0000","last_became_active":"2024-01-15T17:55:04.759902+0000","last_became_peered":"2024-01-15T17:55:04.759902+0000","last_unstale":"2024-01-15T17:55:06.787429+0000","last_undegraded":"2024-01-15T17:55:06.787429+0000","last_fullsized":"2024-01-15T17:55:06.787429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:57.337185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.787323+0000","last_change":"2024-01-15T17:55:03.741130+0000","last_active":"2024-01-15T17:55:06.787323+0000","last_peered":"2024-01-15T17:55:06.787323+0000","last_clean":"2024-01-15T17:55:06.787323+0000","last_became_active":"2024-01-15T17:55:03.740791+0000","last_became_peered":"2024-01-15T17:55:03.740791+0000","last_unstale":"2024-01-15T17:55:06.787323+0000","last_undegraded":"2024-01-15T17:55:06.787323+0000","last_fullsized":"2024-01-15T17:55:06.787323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:57.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721182+0000","last_change":"2024-01-15T17:55:04.739187+0000","last_active":"2024-01-15T17:55:05.721182+0000","last_peered":"2024-01-15T17:55:05.721182+0000","last_clean":"2024-01-15T17:55:05.721182+0000","last_became_active":"2024-01-15T17:55:04.738700+0000","last_became_peered":"2024-01-15T17:55:04.738700+0000","last_unstale":"2024-01-15T17:55:05.721182+0000","last_undegraded":"2024-01-15T17:55:05.721182+0000","last_fullsized":"2024-01-15T17:55:05.721182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790088+0000","last_change":"2024-01-15T17:55:03.741719+0000","last_active":"2024-01-15T17:55:06.790088+0000","last_peered":"2024-01-15T17:55:06.790088+0000","last_clean":"2024-01-15T17:55:06.790088+0000","last_became_active":"2024-01-15T17:55:03.739882+0000","last_became_peered":"2024-01-15T17:55:03.739882+0000","last_unstale":"2024-01-15T17:55:06.790088+0000","last_undegraded":"2024-01-15T17:55:06.790088+0000","last_fullsized":"2024-01-15T17:55:06.790088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725696+0000","last_change":"2024-01-15T17:55:04.766676+0000","last_active":"2024-01-15T17:55:05.725696+0000","last_peered":"2024-01-15T17:55:05.725696+0000","last_clean":"2024-01-15T17:55:05.725696+0000","last_became_active":"2024-01-15T17:55:04.766394+0000","last_became_peered":"2024-01-15T17:55:04.766394+0000","last_unstale":"2024-01-15T17:55:05.725696+0000","last_undegraded":"2024-01-15T17:55:05.725696+0000","last_fullsized":"2024-01-15T17:55:05.725696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:52.036315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721394+0000","last_change":"2024-01-15T17:55:03.735528+0000","last_active":"2024-01-15T17:55:05.721394+0000","last_peered":"2024-01-15T17:55:05.721394+0000","last_clean":"2024-01-15T17:55:05.721394+0000","last_became_active":"2024-01-15T17:55:03.733799+0000","last_became_peered":"2024-01-15T17:55:03.733799+0000","last_unstale":"2024-01-15T17:55:05.721394+0000","last_undegraded":"2024-01-15T17:55:05.721394+0000","last_fullsized":"2024-01-15T17:55:05.721394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:50.835576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789930+0000","last_change":"2024-01-15T17:55:04.750889+0000","last_active":"2024-01-15T17:55:06.789930+0000","last_peered":"2024-01-15T17:55:06.789930+0000","last_clean":"2024-01-15T17:55:06.789930+0000","last_became_active":"2024-01-15T17:55:04.750480+0000","last_became_peered":"2024-01-15T17:55:04.750480+0000","last_unstale":"2024-01-15T17:55:06.789930+0000","last_undegraded":"2024-01-15T17:55:06.789930+0000","last_fullsized":"2024-01-15T17:55:06.789930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:13.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590950+0000","last_change":"2024-01-15T17:55:03.758172+0000","last_active":"2024-01-15T17:55:06.590950+0000","last_peered":"2024-01-15T17:55:06.590950+0000","last_clean":"2024-01-15T17:55:06.590950+0000","last_became_active":"2024-01-15T17:55:03.757742+0000","last_became_peered":"2024-01-15T17:55:03.757742+0000","last_unstale":"2024-01-15T17:55:06.590950+0000","last_undegraded":"2024-01-15T17:55:06.590950+0000","last_fullsized":"2024-01-15T17:55:06.590950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:17.081091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721370+0000","last_change":"2024-01-15T17:55:04.729485+0000","last_active":"2024-01-15T17:55:05.721370+0000","last_peered":"2024-01-15T17:55:05.721370+0000","last_clean":"2024-01-15T17:55:05.721370+0000","last_became_active":"2024-01-15T17:55:04.729029+0000","last_became_peered":"2024-01-15T17:55:04.729029+0000","last_unstale":"2024-01-15T17:55:05.721370+0000","last_undegraded":"2024-01-15T17:55:05.721370+0000","last_fullsized":"2024-01-15T17:55:05.721370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.956145+0000","last_change":"2024-01-15T17:55:03.752377+0000","last_active":"2024-01-15T17:55:06.956145+0000","last_peered":"2024-01-15T17:55:06.956145+0000","last_clean":"2024-01-15T17:55:06.956145+0000","last_became_active":"2024-01-15T17:55:03.752164+0000","last_became_peered":"2024-01-15T17:55:03.752164+0000","last_unstale":"2024-01-15T17:55:06.956145+0000","last_undegraded":"2024-01-15T17:55:06.956145+0000","last_fullsized":"2024-01-15T17:55:06.956145+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:24:56.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790469+0000","last_change":"2024-01-15T17:55:04.758991+0000","last_active":"2024-01-15T17:55:06.790469+0000","last_peered":"2024-01-15T17:55:06.790469+0000","last_clean":"2024-01-15T17:55:06.790469+0000","last_became_active":"2024-01-15T17:55:04.758422+0000","last_became_peered":"2024-01-15T17:55:04.758422+0000","last_unstale":"2024-01-15T17:55:06.790469+0000","last_undegraded":"2024-01-15T17:55:06.790469+0000","last_fullsized":"2024-01-15T17:55:06.790469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:01:41.493666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.727355+0000","last_change":"2024-01-15T17:55:03.754630+0000","last_active":"2024-01-15T17:55:05.727355+0000","last_peered":"2024-01-15T17:55:05.727355+0000","last_clean":"2024-01-15T17:55:05.727355+0000","last_became_active":"2024-01-15T17:55:03.754117+0000","last_became_peered":"2024-01-15T17:55:03.754117+0000","last_unstale":"2024-01-15T17:55:05.727355+0000","last_undegraded":"2024-01-15T17:55:05.727355+0000","last_fullsized":"2024-01-15T17:55:05.727355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:45:26.025029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788431+0000","last_change":"2024-01-15T17:55:04.754711+0000","last_active":"2024-01-15T17:55:06.788431+0000","last_peered":"2024-01-15T17:55:06.788431+0000","last_clean":"2024-01-15T17:55:06.788431+0000","last_became_active":"2024-01-15T17:55:04.753730+0000","last_became_peered":"2024-01-15T17:55:04.753730+0000","last_unstale":"2024-01-15T17:55:06.788431+0000","last_undegraded":"2024-01-15T17:55:06.788431+0000","last_fullsized":"2024-01-15T17:55:06.788431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:41.585450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.589725+0000","last_change":"2024-01-15T17:55:03.755088+0000","last_active":"2024-01-15T17:55:06.589725+0000","last_peered":"2024-01-15T17:55:06.589725+0000","last_clean":"2024-01-15T17:55:06.589725+0000","last_became_active":"2024-01-15T17:55:03.754805+0000","last_became_peered":"2024-01-15T17:55:03.754805+0000","last_unstale":"2024-01-15T17:55:06.589725+0000","last_undegraded":"2024-01-15T17:55:06.589725+0000","last_fullsized":"2024-01-15T17:55:06.589725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:29.155474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725215+0000","last_change":"2024-01-15T17:55:04.747604+0000","last_active":"2024-01-15T17:55:05.725215+0000","last_peered":"2024-01-15T17:55:05.725215+0000","last_clean":"2024-01-15T17:55:05.725215+0000","last_became_active":"2024-01-15T17:55:04.746687+0000","last_became_peered":"2024-01-15T17:55:04.746687+0000","last_unstale":"2024-01-15T17:55:05.725215+0000","last_undegraded":"2024-01-15T17:55:05.725215+0000","last_fullsized":"2024-01-15T17:55:05.725215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:58.689690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721658+0000","last_change":"2024-01-15T17:55:03.749885+0000","last_active":"2024-01-15T17:55:05.721658+0000","last_peered":"2024-01-15T17:55:05.721658+0000","last_clean":"2024-01-15T17:55:05.721658+0000","last_became_active":"2024-01-15T17:55:03.749600+0000","last_became_peered":"2024-01-15T17:55:03.749600+0000","last_unstale":"2024-01-15T17:55:05.721658+0000","last_undegraded":"2024-01-15T17:55:05.721658+0000","last_fullsized":"2024-01-15T17:55:05.721658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:09.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788706+0000","last_change":"2024-01-15T17:55:04.736486+0000","last_active":"2024-01-15T17:55:06.788706+0000","last_peered":"2024-01-15T17:55:06.788706+0000","last_clean":"2024-01-15T17:55:06.788706+0000","last_became_active":"2024-01-15T17:55:04.736012+0000","last_became_peered":"2024-01-15T17:55:04.736012+0000","last_unstale":"2024-01-15T17:55:06.788706+0000","last_undegraded":"2024-01-15T17:55:06.788706+0000","last_fullsized":"2024-01-15T17:55:06.788706+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:53:52.730234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.721367+0000","last_change":"2024-01-15T17:55:03.738319+0000","last_active":"2024-01-15T17:55:05.721367+0000","last_peered":"2024-01-15T17:55:05.721367+0000","last_clean":"2024-01-15T17:55:05.721367+0000","last_became_active":"2024-01-15T17:55:03.737946+0000","last_became_peered":"2024-01-15T17:55:03.737946+0000","last_unstale":"2024-01-15T17:55:05.721367+0000","last_undegraded":"2024-01-15T17:55:05.721367+0000","last_fullsized":"2024-01-15T17:55:05.721367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:04:57.488951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789078+0000","last_change":"2024-01-15T17:55:04.766996+0000","last_active":"2024-01-15T17:55:06.789078+0000","last_peered":"2024-01-15T17:55:06.789078+0000","last_clean":"2024-01-15T17:55:06.789078+0000","last_became_active":"2024-01-15T17:55:04.766695+0000","last_became_peered":"2024-01-15T17:55:04.766695+0000","last_unstale":"2024-01-15T17:55:06.789078+0000","last_undegraded":"2024-01-15T17:55:06.789078+0000","last_fullsized":"2024-01-15T17:55:06.789078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:21.570608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590906+0000","last_change":"2024-01-15T17:55:03.757123+0000","last_active":"2024-01-15T17:55:06.590906+0000","last_peered":"2024-01-15T17:55:06.590906+0000","last_clean":"2024-01-15T17:55:06.590906+0000","last_became_active":"2024-01-15T17:55:03.756522+0000","last_became_peered":"2024-01-15T17:55:03.756522+0000","last_unstale":"2024-01-15T17:55:06.590906+0000","last_undegraded":"2024-01-15T17:55:06.590906+0000","last_fullsized":"2024-01-15T17:55:06.590906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:05.725414+0000","last_change":"2024-01-15T17:55:04.756691+0000","last_active":"2024-01-15T17:55:05.725414+0000","last_peered":"2024-01-15T17:55:05.725414+0000","last_clean":"2024-01-15T17:55:05.725414+0000","last_became_active":"2024-01-15T17:55:04.756410+0000","last_became_peered":"2024-01-15T17:55:04.756410+0000","last_unstale":"2024-01-15T17:55:05.725414+0000","last_undegraded":"2024-01-15T17:55:05.725414+0000","last_fullsized":"2024-01-15T17:55:05.725414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:51:18.281829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789737+0000","last_change":"2024-01-15T17:55:04.747374+0000","last_active":"2024-01-15T17:55:06.789737+0000","last_peered":"2024-01-15T17:55:06.789737+0000","last_clean":"2024-01-15T17:55:06.789737+0000","last_became_active":"2024-01-15T17:55:04.747062+0000","last_became_peered":"2024-01-15T17:55:04.747062+0000","last_unstale":"2024-01-15T17:55:06.789737+0000","last_undegraded":"2024-01-15T17:55:06.789737+0000","last_fullsized":"2024-01-15T17:55:06.789737+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:44.725915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590116+0000","last_change":"2024-01-15T17:55:03.759441+0000","last_active":"2024-01-15T17:55:06.590116+0000","last_peered":"2024-01-15T17:55:06.590116+0000","last_clean":"2024-01-15T17:55:06.590116+0000","last_became_active":"2024-01-15T17:55:03.759054+0000","last_became_peered":"2024-01-15T17:55:03.759054+0000","last_unstale":"2024-01-15T17:55:06.590116+0000","last_undegraded":"2024-01-15T17:55:06.590116+0000","last_fullsized":"2024-01-15T17:55:06.590116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:19.912976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.789716+0000","last_change":"2024-01-15T17:55:04.743796+0000","last_active":"2024-01-15T17:55:06.789716+0000","last_peered":"2024-01-15T17:55:06.789716+0000","last_clean":"2024-01-15T17:55:06.789716+0000","last_became_active":"2024-01-15T17:55:04.743475+0000","last_became_peered":"2024-01-15T17:55:04.743475+0000","last_unstale":"2024-01-15T17:55:06.789716+0000","last_undegraded":"2024-01-15T17:55:06.789716+0000","last_fullsized":"2024-01-15T17:55:06.789716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:52.765350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.590613+0000","last_change":"2024-01-15T17:55:03.759401+0000","last_active":"2024-01-15T17:55:06.590613+0000","last_peered":"2024-01-15T17:55:06.590613+0000","last_clean":"2024-01-15T17:55:06.590613+0000","last_became_active":"2024-01-15T17:55:03.759124+0000","last_became_peered":"2024-01-15T17:55:03.759124+0000","last_unstale":"2024-01-15T17:55:06.590613+0000","last_undegraded":"2024-01-15T17:55:06.590613+0000","last_fullsized":"2024-01-15T17:55:06.590613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:09.645311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788680+0000","last_change":"2024-01-15T17:55:04.757926+0000","last_active":"2024-01-15T17:55:06.788680+0000","last_peered":"2024-01-15T17:55:06.788680+0000","last_clean":"2024-01-15T17:55:06.788680+0000","last_became_active":"2024-01-15T17:55:04.757625+0000","last_became_peered":"2024-01-15T17:55:04.757625+0000","last_unstale":"2024-01-15T17:55:06.788680+0000","last_undegraded":"2024-01-15T17:55:06.788680+0000","last_fullsized":"2024-01-15T17:55:06.788680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:55.004149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790277+0000","last_change":"2024-01-15T17:55:03.736897+0000","last_active":"2024-01-15T17:55:06.790277+0000","last_peered":"2024-01-15T17:55:06.790277+0000","last_clean":"2024-01-15T17:55:06.790277+0000","last_became_active":"2024-01-15T17:55:03.736576+0000","last_became_peered":"2024-01-15T17:55:03.736576+0000","last_unstale":"2024-01-15T17:55:06.790277+0000","last_undegraded":"2024-01-15T17:55:06.790277+0000","last_fullsized":"2024-01-15T17:55:06.790277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:56:59.201106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.790294+0000","last_change":"2024-01-15T17:55:04.760100+0000","last_active":"2024-01-15T17:55:06.790294+0000","last_peered":"2024-01-15T17:55:06.790294+0000","last_clean":"2024-01-15T17:55:06.790294+0000","last_became_active":"2024-01-15T17:55:04.756345+0000","last_became_peered":"2024-01-15T17:55:04.756345+0000","last_unstale":"2024-01-15T17:55:06.790294+0000","last_undegraded":"2024-01-15T17:55:06.790294+0000","last_fullsized":"2024-01-15T17:55:06.790294+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:03.693357+0000","last_clean_scrub_stamp":"2024-01-15T17:55:03.693357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:06.327821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-15T17:55:06.788935+0000","last_change":"2024-01-15T17:55:03.750992+0000","last_active":"2024-01-15T17:55:06.788935+0000","last_peered":"2024-01-15T17:55:06.788935+0000","last_clean":"2024-01-15T17:55:06.788935+0000","last_became_active":"2024-01-15T17:55:03.750662+0000","last_became_peered":"2024-01-15T17:55:03.750662+0000","last_unstale":"2024-01-15T17:55:06.788935+0000","last_undegraded":"2024-01-15T17:55:06.788935+0000","last_fullsized":"2024-01-15T17:55:06.788935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:55:02.492619+0000","last_clean_scrub_stamp":"2024-01-15T17:55:02.492619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:53:33.271369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":823296,"data_stored":669826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4039999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.347}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.629}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0979999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.036}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1589999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":78966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.835}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.399}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.623}]}]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":79002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3720000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7410000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.806}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":78876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.171}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.054}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":233472,"data_stored":79036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5409999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":831488,"data_stored":669882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7230000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9139999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:55:15.320 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-15T17:55:15.320 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-15T17:55:15.320 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-15T17:55:15.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-15T17:55:15.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.574+0000 7f8a97e00640 1 Processor -- start 2024-01-15T17:55:15.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.574+0000 7f8a97e00640 1 -- start start 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.575+0000 7f8a97e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8a98162810 0x7f8a98071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.575+0000 7f8a97e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8a98071780 0x7f8a9806c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.575+0000 7f8a97e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a98071c10 con 0x7f8a98162430 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.575+0000 7f8a97e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a980702e0 con 0x7f8a98071780 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.575+0000 7f8a97e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a980705c0 con 0x7f8a98162810 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.575+0000 7f8a96a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8a98162430 0x7f8a980b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43032/0 (socket says 172.21.15.26:43032) 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.575+0000 7f8a96a00640 1 -- 172.21.15.26:0/1134159057 learned_addr learned my addr 172.21.15.26:0/1134159057 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.576+0000 7f8a97400640 1 --2- 172.21.15.26:0/1134159057 >> v2:172.21.15.26:3300/0 conn(0x7f8a98071780 0x7f8a9806c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.576+0000 7f8a8de00640 1 --2- 172.21.15.26:0/1134159057 >> v2:172.21.15.124:3300/0 conn(0x7f8a98162810 0x7f8a98071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.576+0000 7f8a96000640 1 -- 172.21.15.26:0/1134159057 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312530075 0 0) 0x7f8a98071c10 con 0x7f8a98162430 2024-01-15T17:55:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.576+0000 7f8a96000640 1 -- 172.21.15.26:0/1134159057 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a78003520 con 0x7f8a98162430 2024-01-15T17:55:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.576+0000 7f8a97400640 1 -- 172.21.15.26:0/1134159057 >> v2:172.21.15.124:3300/0 conn(0x7f8a98162810 msgr2=0x7f8a98071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.576+0000 7f8a97400640 1 --2- 172.21.15.26:0/1134159057 >> v2:172.21.15.124:3300/0 conn(0x7f8a98162810 0x7f8a98071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.576+0000 7f8a97400640 1 -- 172.21.15.26:0/1134159057 >> v1:172.21.15.26:6789/0 conn(0x7f8a98162430 legacy=0x7f8a980b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.576+0000 7f8a97400640 1 -- 172.21.15.26:0/1134159057 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a980708a0 con 0x7f8a98071780 2024-01-15T17:55:15.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.577+0000 7f8a97400640 1 --2- 172.21.15.26:0/1134159057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a98071780 0x7f8a9806c8c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8a9815c090 tx=0x7f8a9002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cceee16ffa6cdd13 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:15.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.577+0000 7f8a96000640 1 -- 172.21.15.26:0/1134159057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9002cc80 con 0x7f8a98071780 2024-01-15T17:55:15.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.578+0000 7f8a96000640 1 -- 172.21.15.26:0/1134159057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9002cde0 con 0x7f8a98071780 2024-01-15T17:55:15.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.578+0000 7f8a96000640 1 -- 172.21.15.26:0/1134159057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90034640 con 0x7f8a98071780 2024-01-15T17:55:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.578+0000 7f8a97e00640 1 -- 172.21.15.26:0/1134159057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a98071780 msgr2=0x7f8a9806c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.578+0000 7f8a97e00640 1 --2- 172.21.15.26:0/1134159057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a98071780 0x7f8a9806c8c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8a9815c090 tx=0x7f8a9002bed0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.578+0000 7f8a97e00640 1 -- 172.21.15.26:0/1134159057 shutdown_connections 2024-01-15T17:55:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.578+0000 7f8a97e00640 1 --2- 172.21.15.26:0/1134159057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a98071780 0x7f8a9806c8c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.578+0000 7f8a97e00640 1 --2- 172.21.15.26:0/1134159057 >> v2:172.21.15.124:3300/0 conn(0x7f8a98162810 0x7f8a98071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.578+0000 7f8a97e00640 1 -- 172.21.15.26:0/1134159057 >> 172.21.15.26:0/1134159057 conn(0x7f8a9806a800 msgr2=0x7f8a9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.579+0000 7f8a97e00640 1 -- 172.21.15.26:0/1134159057 shutdown_connections 2024-01-15T17:55:15.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.579+0000 7f8a97e00640 1 -- 172.21.15.26:0/1134159057 wait complete. 2024-01-15T17:55:15.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.580+0000 7f8a97e00640 1 Processor -- start 2024-01-15T17:55:15.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.581+0000 7f8a97e00640 1 -- start start 2024-01-15T17:55:15.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.581+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a98071780 0x7f8a98163c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.581+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a98162430 0x7f8a981652d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.581+0000 7f8a97e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a981433c0 0x7f8a980addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.581+0000 7f8a97e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98143870 con 0x7f8a981433c0 2024-01-15T17:55:15.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.581+0000 7f8a97e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a980ae300 con 0x7f8a98162430 2024-01-15T17:55:15.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.581+0000 7f8a97e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98164230 con 0x7f8a98071780 2024-01-15T17:55:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a97400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a981433c0 0x7f8a980addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a97400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a981433c0 0x7f8a980addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44620/0 (socket says 172.21.15.26:44620) 2024-01-15T17:55:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a97400640 1 -- 172.21.15.26:0/2997163046 learned_addr learned my addr 172.21.15.26:0/2997163046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a96a00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a98071780 0x7f8a98163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a8de00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a98162430 0x7f8a981652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a97400640 1 -- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a98071780 msgr2=0x7f8a98163c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a97400640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a98071780 0x7f8a98163c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a97400640 1 -- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a98162430 msgr2=0x7f8a981652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a97400640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a98162430 0x7f8a981652d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.582+0000 7f8a97400640 1 -- 172.21.15.26:0/2997163046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90009a70 con 0x7f8a981433c0 2024-01-15T17:55:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.583+0000 7f8a97400640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a981433c0 0x7f8a980addc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a90037040 tx=0x7f8a90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.583+0000 7f8a8f400640 1 -- 172.21.15.26:0/2997163046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90004450 con 0x7f8a981433c0 2024-01-15T17:55:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.583+0000 7f8a8f400640 1 -- 172.21.15.26:0/2997163046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a900045d0 con 0x7f8a981433c0 2024-01-15T17:55:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.583+0000 7f8a8f400640 1 -- 172.21.15.26:0/2997163046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90034440 con 0x7f8a981433c0 2024-01-15T17:55:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.583+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a98164510 con 0x7f8a981433c0 2024-01-15T17:55:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.583+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a98164ad0 con 0x7f8a981433c0 2024-01-15T17:55:15.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.584+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a98067450 con 0x7f8a981433c0 2024-01-15T17:55:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.585+0000 7f8a8f400640 1 -- 172.21.15.26:0/2997163046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8a900345a0 con 0x7f8a981433c0 2024-01-15T17:55:15.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.586+0000 7f8a8f400640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a7006b2f0 0x7f8a7006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:15.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.586+0000 7f8a8f400640 1 -- 172.21.15.26:0/2997163046 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a900ac6b0 con 0x7f8a981433c0 2024-01-15T17:55:15.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.586+0000 7f8a96a00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a7006b2f0 0x7f8a7006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.587+0000 7f8a96a00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a7006b2f0 0x7f8a7006d7b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a880011e0 tx=0x7f8a88009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.590+0000 7f8a8f400640 1 -- 172.21.15.26:0/2997163046 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a900775c0 con 0x7f8a981433c0 2024-01-15T17:55:15.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.823+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8a980b2ef0 con 0x7f8a981433c0 2024-01-15T17:55:15.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.824+0000 7f8a8f400640 1 -- 172.21.15.26:0/2997163046 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8a9007abe0 con 0x7f8a981433c0 2024-01-15T17:55:15.827 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:15.827 INFO:teuthology.orchestra.run.smithi026.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-15T17:55:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a7006b2f0 msgr2=0x7f8a7006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a7006b2f0 0x7f8a7006d7b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a880011e0 tx=0x7f8a88009040 comp rx=0 tx=0).stop 2024-01-15T17:55:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a981433c0 msgr2=0x7f8a980addc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a981433c0 0x7f8a980addc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a90037040 tx=0x7f8a90004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:15.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 shutdown_connections 2024-01-15T17:55:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a7006b2f0 0x7f8a7006d7b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8a981433c0 0x7f8a980addc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8a98162430 0x7f8a981652d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 --2- 172.21.15.26:0/2997163046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8a98071780 0x7f8a98163c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 >> 172.21.15.26:0/2997163046 conn(0x7f8a9806a800 msgr2=0x7f8a980aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 shutdown_connections 2024-01-15T17:55:15.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:15.827+0000 7f8a97e00640 1 -- 172.21.15.26:0/2997163046 wait complete. 2024-01-15T17:55:15.850 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-15T17:55:15.850 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:16.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.063+0000 7f217ee00640 1 Processor -- start 2024-01-15T17:55:16.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.063+0000 7f217ee00640 1 -- start start 2024-01-15T17:55:16.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.064+0000 7f217ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cf9a0 0x7f217815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.064+0000 7f217ee00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21780b1ee0 con 0x7f2178143ae0 2024-01-15T17:55:16.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.064+0000 7f217ee00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21780b2780 con 0x7f217815e1e0 2024-01-15T17:55:16.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.064+0000 7f217ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217815e990 con 0x7f21780cf9a0 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.065+0000 7f2177e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2178143ae0 0x7f21780cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43048/0 (socket says 172.21.15.26:43048) 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.065+0000 7f2177e00640 1 -- 172.21.15.26:0/487320871 learned_addr learned my addr 172.21.15.26:0/487320871 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.065+0000 7f2177400640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cf9a0 0x7f217815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.065+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310319764 0 0) 0x7f21780b1ee0 con 0x7f2178143ae0 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.065+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2158005730 con 0x7f2178143ae0 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.065+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3354442292 0 0) 0x7f2158005730 con 0x7f2178143ae0 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.065+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cf9a0 msgr2=0x7f217815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.065+0000 7f2176a00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cf9a0 0x7f217815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 >> v1:172.21.15.124:6789/0 conn(0x7f217815e1e0 legacy=0x7f21780702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21780b2130 con 0x7f2178143ae0 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f2168001670 con 0x7f2178143ae0 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 >> v1:172.21.15.26:6789/0 conn(0x7f2178143ae0 legacy=0x7f21780cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2158005bb0 0x7f2158005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cf9a0 0x7f21580069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158006f10 0x7f2158007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217815e990 con 0x7f2158005bb0 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215800dc70 con 0x7f2158006f10 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215800de30 con 0x7f21780cf9a0 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2177e00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2158005bb0 0x7f2158005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2177e00640 1 -- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cf9a0 msgr2=0x7f21580069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2177e00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cf9a0 0x7f21580069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f217ce00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158006f10 0x7f2158007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2177e00640 1 -- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158006f10 msgr2=0x7f2158007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2177e00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158006f10 0x7f2158007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.066+0000 7f2177e00640 1 -- 172.21.15.26:0/487320871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158005730 con 0x7f2158005bb0 2024-01-15T17:55:16.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.067+0000 7f2177e00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2158005bb0 0x7f2158005fd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2168007fb0 tx=0x7f2158007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b4e0aeb345ee5d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:16.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.068+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168005ad0 con 0x7f2158005bb0 2024-01-15T17:55:16.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.068+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2168005c50 con 0x7f2158005bb0 2024-01-15T17:55:16.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.068+0000 7f2176a00640 1 -- 172.21.15.26:0/487320871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216802c7c0 con 0x7f2158005bb0 2024-01-15T17:55:16.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.068+0000 7f217ee00640 1 -- 172.21.15.26:0/487320871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2158005bb0 msgr2=0x7f2158005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.068+0000 7f217ee00640 1 --2- 172.21.15.26:0/487320871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2158005bb0 0x7f2158005fd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2168007fb0 tx=0x7f2158007a00 comp rx=0 tx=0).stop 2024-01-15T17:55:16.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.068+0000 7f217ce00640 1 -- 172.21.15.26:0/487320871 reap_dead start 2024-01-15T17:55:16.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.068+0000 7f217ee00640 1 -- 172.21.15.26:0/487320871 shutdown_connections 2024-01-15T17:55:16.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.068+0000 7f217ee00640 1 -- 172.21.15.26:0/487320871 >> 172.21.15.26:0/487320871 conn(0x7f217806a800 msgr2=0x7f217806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:16.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.069+0000 7f217ee00640 1 -- 172.21.15.26:0/487320871 shutdown_connections 2024-01-15T17:55:16.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.069+0000 7f217ee00640 1 -- 172.21.15.26:0/487320871 wait complete. 2024-01-15T17:55:16.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.070+0000 7f217ee00640 1 Processor -- start 2024-01-15T17:55:16.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.070+0000 7f217ee00640 1 -- start start 2024-01-15T17:55:16.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.071+0000 7f217ee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158005bb0 0x7f21780ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.071+0000 7f217ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21780ce1f0 0x7f2178159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.071+0000 7f217ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cd1f0 0x7f21780cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.071+0000 7f217ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21780ce070 con 0x7f21780ce1f0 2024-01-15T17:55:16.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.071+0000 7f217ee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21780cdbb0 con 0x7f2158005bb0 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.071+0000 7f217ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21780ae260 con 0x7f21780cd1f0 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21780ce1f0 0x7f2178159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21780ce1f0 0x7f2178159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44646/0 (socket says 172.21.15.26:44646) 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177400640 1 -- 172.21.15.26:0/1540219878 learned_addr learned my addr 172.21.15.26:0/1540219878 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177e00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158005bb0 0x7f21780ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f217ce00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cd1f0 0x7f21780cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177400640 1 -- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cd1f0 msgr2=0x7f21780cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177400640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cd1f0 0x7f21780cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177400640 1 -- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158005bb0 msgr2=0x7f21780ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177400640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158005bb0 0x7f21780ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177400640 1 -- 172.21.15.26:0/1540219878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21780ad9c0 con 0x7f21780ce1f0 2024-01-15T17:55:16.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177e00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158005bb0 0x7f21780ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:16.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.072+0000 7f2177400640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21780ce1f0 0x7f2178159870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f216000e7c0 tx=0x7f216000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:16.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.073+0000 7f216fe00640 1 -- 172.21.15.26:0/1540219878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216000eed0 con 0x7f21780ce1f0 2024-01-15T17:55:16.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.073+0000 7f216fe00640 1 -- 172.21.15.26:0/1540219878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216000c940 con 0x7f21780ce1f0 2024-01-15T17:55:16.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.073+0000 7f216fe00640 1 -- 172.21.15.26:0/1540219878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216000ccd0 con 0x7f21780ce1f0 2024-01-15T17:55:16.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.073+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21780adca0 con 0x7f21780ce1f0 2024-01-15T17:55:16.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.073+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21780aeb50 con 0x7f21780ce1f0 2024-01-15T17:55:16.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.074+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21780675d0 con 0x7f21780ce1f0 2024-01-15T17:55:16.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.075+0000 7f216fe00640 1 -- 172.21.15.26:0/1540219878 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2160013470 con 0x7f21780ce1f0 2024-01-15T17:55:16.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.076+0000 7f216fe00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215006b2c0 0x7f215006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.076+0000 7f216fe00640 1 -- 172.21.15.26:0/1540219878 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2160089be0 con 0x7f21780ce1f0 2024-01-15T17:55:16.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.077+0000 7f2177e00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215006b2c0 0x7f215006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.078+0000 7f2177e00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215006b2c0 0x7f215006d780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2168034000 tx=0x7f2168009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:16.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.081+0000 7f216fe00640 1 -- 172.21.15.26:0/1540219878 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2160054ae0 con 0x7f21780ce1f0 2024-01-15T17:55:16.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.275+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21781410e0 con 0x7f21780ce1f0 2024-01-15T17:55:16.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.278+0000 7f216fe00640 1 -- 172.21.15.26:0/1540219878 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f2160021090 con 0x7f21780ce1f0 2024-01-15T17:55:16.282 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:16.282 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:16.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215006b2c0 msgr2=0x7f215006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215006b2c0 0x7f215006d780 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2168034000 tx=0x7f2168009030 comp rx=0 tx=0).stop 2024-01-15T17:55:16.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21780ce1f0 msgr2=0x7f2178159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21780ce1f0 0x7f2178159870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f216000e7c0 tx=0x7f216000ec80 comp rx=0 tx=0).stop 2024-01-15T17:55:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 shutdown_connections 2024-01-15T17:55:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21780cd1f0 0x7f21780cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215006b2c0 0x7f215006d780 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21780ce1f0 0x7f2178159870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 --2- 172.21.15.26:0/1540219878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2158005bb0 0x7f21780ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 >> 172.21.15.26:0/1540219878 conn(0x7f217806a800 msgr2=0x7f217815a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 shutdown_connections 2024-01-15T17:55:16.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.281+0000 7f217ee00640 1 -- 172.21.15.26:0/1540219878 wait complete. 2024-01-15T17:55:16.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:16.304 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.529+0000 7f8e6d400640 1 Processor -- start 2024-01-15T17:55:16.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.530+0000 7f8e6d400640 1 -- start start 2024-01-15T17:55:16.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.531+0000 7f8e6d400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8e680cf9a0 0x7f8e680cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.531+0000 7f8e6d400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e680b31d0 0x7f8e680b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.531+0000 7f8e6d400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e680d0320 con 0x7f8e6815d4b0 2024-01-15T17:55:16.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.531+0000 7f8e6d400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e680d1c60 con 0x7f8e680b31d0 2024-01-15T17:55:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.531+0000 7f8e6d400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e680b2730 con 0x7f8e680cf9a0 2024-01-15T17:55:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.532+0000 7f8e6ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e680b31d0 0x7f8e680b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.532+0000 7f8e6ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e680b31d0 0x7f8e680b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44648/0 (socket says 172.21.15.26:44648) 2024-01-15T17:55:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.532+0000 7f8e6ca00640 1 -- 172.21.15.26:0/1382303679 learned_addr learned my addr 172.21.15.26:0/1382303679 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.532+0000 7f8e67400640 1 --2- 172.21.15.26:0/1382303679 >> v2:172.21.15.124:3300/0 conn(0x7f8e680cf9a0 0x7f8e680cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.532+0000 7f8e6ca00640 1 -- 172.21.15.26:0/1382303679 >> v2:172.21.15.124:3300/0 conn(0x7f8e680cf9a0 msgr2=0x7f8e680cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.532+0000 7f8e6ca00640 1 --2- 172.21.15.26:0/1382303679 >> v2:172.21.15.124:3300/0 conn(0x7f8e680cf9a0 0x7f8e680cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.532+0000 7f8e6ca00640 1 -- 172.21.15.26:0/1382303679 >> v1:172.21.15.124:6789/0 conn(0x7f8e6815d4b0 legacy=0x7f8e680d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.532+0000 7f8e6ca00640 1 -- 172.21.15.26:0/1382303679 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e680d1ec0 con 0x7f8e680b31d0 2024-01-15T17:55:16.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.533+0000 7f8e6ca00640 1 --2- 172.21.15.26:0/1382303679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680b31d0 0x7f8e680b3660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8e6808b580 tx=0x7f8e6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c667fd5e9234b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:16.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.534+0000 7f8e66a00640 1 -- 172.21.15.26:0/1382303679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e60031870 con 0x7f8e680b31d0 2024-01-15T17:55:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.534+0000 7f8e66a00640 1 -- 172.21.15.26:0/1382303679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e600319d0 con 0x7f8e680b31d0 2024-01-15T17:55:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.534+0000 7f8e66a00640 1 -- 172.21.15.26:0/1382303679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e60031d60 con 0x7f8e680b31d0 2024-01-15T17:55:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.535+0000 7f8e6d400640 1 -- 172.21.15.26:0/1382303679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680b31d0 msgr2=0x7f8e680b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.535+0000 7f8e6d400640 1 --2- 172.21.15.26:0/1382303679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680b31d0 0x7f8e680b3660 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8e6808b580 tx=0x7f8e6002b910 comp rx=0 tx=0).stop 2024-01-15T17:55:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.535+0000 7f8e6d400640 1 -- 172.21.15.26:0/1382303679 shutdown_connections 2024-01-15T17:55:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.535+0000 7f8e6d400640 1 --2- 172.21.15.26:0/1382303679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680b31d0 0x7f8e680b3660 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.535+0000 7f8e6d400640 1 --2- 172.21.15.26:0/1382303679 >> v2:172.21.15.124:3300/0 conn(0x7f8e680cf9a0 0x7f8e680cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.535+0000 7f8e6d400640 1 -- 172.21.15.26:0/1382303679 >> 172.21.15.26:0/1382303679 conn(0x7f8e680d3000 msgr2=0x7f8e680d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.535+0000 7f8e6d400640 1 -- 172.21.15.26:0/1382303679 shutdown_connections 2024-01-15T17:55:16.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.535+0000 7f8e6d400640 1 -- 172.21.15.26:0/1382303679 wait complete. 2024-01-15T17:55:16.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.537+0000 7f8e6d400640 1 Processor -- start 2024-01-15T17:55:16.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.537+0000 7f8e6d400640 1 -- start start 2024-01-15T17:55:16.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e6d400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e680b31d0 0x7f8e681542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e6d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680cf9a0 0x7f8e6809f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e6d400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e6815d4b0 0x7f8e680afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e6d400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680a0d20 con 0x7f8e680cf9a0 2024-01-15T17:55:16.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e6d400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680a0110 con 0x7f8e6815d4b0 2024-01-15T17:55:16.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e6d400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e680a03f0 con 0x7f8e680b31d0 2024-01-15T17:55:16.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e67400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680cf9a0 0x7f8e6809f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e67400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680cf9a0 0x7f8e6809f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44664/0 (socket says 172.21.15.26:44664) 2024-01-15T17:55:16.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e67400640 1 -- 172.21.15.26:0/2643475373 learned_addr learned my addr 172.21.15.26:0/2643475373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:16.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.538+0000 7f8e67e00640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e680b31d0 0x7f8e681542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e6ca00640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e6815d4b0 0x7f8e680afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e67400640 1 -- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e680b31d0 msgr2=0x7f8e681542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e67400640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e680b31d0 0x7f8e681542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e67400640 1 -- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e6815d4b0 msgr2=0x7f8e680afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e67400640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e6815d4b0 0x7f8e680afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e67400640 1 -- 172.21.15.26:0/2643475373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e600094d0 con 0x7f8e680cf9a0 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e67400640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680cf9a0 0x7f8e6809f6a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8e50007e40 tx=0x7f8e5000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e5fe00640 1 -- 172.21.15.26:0/2643475373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e50014620 con 0x7f8e680cf9a0 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e5fe00640 1 -- 172.21.15.26:0/2643475373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5000ee90 con 0x7f8e680cf9a0 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e5fe00640 1 -- 172.21.15.26:0/2643475373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e500149c0 con 0x7f8e680cf9a0 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.539+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e680a06d0 con 0x7f8e680cf9a0 2024-01-15T17:55:16.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.540+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e68154830 con 0x7f8e680cf9a0 2024-01-15T17:55:16.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.541+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e680cb540 con 0x7f8e680cf9a0 2024-01-15T17:55:16.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.541+0000 7f8e5fe00640 1 -- 172.21.15.26:0/2643475373 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8e50014b20 con 0x7f8e680cf9a0 2024-01-15T17:55:16.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.542+0000 7f8e5fe00640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e4006b2c0 0x7f8e4006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.542+0000 7f8e5fe00640 1 -- 172.21.15.26:0/2643475373 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e5008a5a0 con 0x7f8e680cf9a0 2024-01-15T17:55:16.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.543+0000 7f8e67e00640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e4006b2c0 0x7f8e4006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.544+0000 7f8e67e00640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e4006b2c0 0x7f8e4006d780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006fd0 tx=0x7f8e58008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:16.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.547+0000 7f8e5fe00640 1 -- 172.21.15.26:0/2643475373 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e680cb540 con 0x7f8e680cf9a0 2024-01-15T17:55:16.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.741+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e680cb540 con 0x7f8e680cf9a0 2024-01-15T17:55:16.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.742+0000 7f8e5fe00640 1 -- 172.21.15.26:0/2643475373 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e680cb540 con 0x7f8e680cf9a0 2024-01-15T17:55:16.745 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:16.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e4006b2c0 msgr2=0x7f8e4006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e4006b2c0 0x7f8e4006d780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8e58006fd0 tx=0x7f8e58008040 comp rx=0 tx=0).stop 2024-01-15T17:55:16.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680cf9a0 msgr2=0x7f8e6809f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680cf9a0 0x7f8e6809f6a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8e50007e40 tx=0x7f8e5000e4b0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 shutdown_connections 2024-01-15T17:55:16.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e4006b2c0 0x7f8e4006d780 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e6815d4b0 0x7f8e680afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e680cf9a0 0x7f8e6809f6a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 --2- 172.21.15.26:0/2643475373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e680b31d0 0x7f8e681542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 >> 172.21.15.26:0/2643475373 conn(0x7f8e680d3000 msgr2=0x7f8e6806ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 shutdown_connections 2024-01-15T17:55:16.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.745+0000 7f8e6d400640 1 -- 172.21.15.26:0/2643475373 wait complete. 2024-01-15T17:55:16.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:16.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.990+0000 7faf97200640 1 Processor -- start 2024-01-15T17:55:16.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.990+0000 7faf97200640 1 -- start start 2024-01-15T17:55:16.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.992+0000 7faf97200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faf980ccb50 0x7faf98070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.992+0000 7faf97200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9815bf10 0x7faf98143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.992+0000 7faf97200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9815f4e0 con 0x7faf980adf20 2024-01-15T17:55:16.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.992+0000 7faf97200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9815f970 con 0x7faf980ccb50 2024-01-15T17:55:16.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.992+0000 7faf97200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9815c3c0 con 0x7faf9815bf10 2024-01-15T17:55:16.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.992+0000 7faf95e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faf980adf20 0x7faf980cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43056/0 (socket says 172.21.15.26:43056) 2024-01-15T17:55:16.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.992+0000 7faf95e00640 1 -- 172.21.15.26:0/1907312291 learned_addr learned my addr 172.21.15.26:0/1907312291 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:16.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf8fe00640 1 --2- 172.21.15.26:0/1907312291 >> v2:172.21.15.124:3300/0 conn(0x7faf980ccb50 0x7faf98070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307199381 0 0) 0x7faf9815f4e0 con 0x7faf980adf20 2024-01-15T17:55:16.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf96800640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9815bf10 0x7faf98143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf78003520 con 0x7faf980adf20 2024-01-15T17:55:16.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421918613 0 0) 0x7faf78003520 con 0x7faf980adf20 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9815bf10 msgr2=0x7faf98143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9815bf10 0x7faf98143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 >> v2:172.21.15.124:3300/0 conn(0x7faf980ccb50 msgr2=0x7faf98070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 --2- 172.21.15.26:0/1907312291 >> v2:172.21.15.124:3300/0 conn(0x7faf980ccb50 0x7faf98070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf98071130 con 0x7faf980adf20 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7faf88001670 con 0x7faf980adf20 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 >> v1:172.21.15.26:6789/0 conn(0x7faf980adf20 legacy=0x7faf980cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf78003970 0x7faf78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf780047c0 0x7faf78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9815bf10 0x7faf78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.993+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9815f970 con 0x7faf78003970 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.994+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9815c3c0 con 0x7faf780047c0 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.994+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7800b9d0 con 0x7faf9815bf10 2024-01-15T17:55:16.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.994+0000 7faf95e00640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf78003970 0x7faf78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.994+0000 7faf8fe00640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf780047c0 0x7faf78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:16.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.994+0000 7faf95e00640 1 -- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9815bf10 msgr2=0x7faf78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:16.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.994+0000 7faf95e00640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf9815bf10 0x7faf78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.994+0000 7faf95e00640 1 -- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf780047c0 msgr2=0x7faf78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:16.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.994+0000 7faf95e00640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf780047c0 0x7faf78004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:16.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.994+0000 7faf95e00640 1 -- 172.21.15.26:0/1907312291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78003520 con 0x7faf78003970 2024-01-15T17:55:17.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.995+0000 7faf95e00640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf78003970 0x7faf78003d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c110 tx=0x7faf780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cf8b0d8027ce28 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:17.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.995+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf88005ad0 con 0x7faf78003970 2024-01-15T17:55:17.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.995+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf88005c50 con 0x7faf78003970 2024-01-15T17:55:17.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.995+0000 7faf95400640 1 -- 172.21.15.26:0/1907312291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8802c7e0 con 0x7faf78003970 2024-01-15T17:55:17.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.996+0000 7faf97200640 1 -- 172.21.15.26:0/1907312291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf78003970 msgr2=0x7faf78003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.996+0000 7faf97200640 1 --2- 172.21.15.26:0/1907312291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf78003970 0x7faf78003d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7faf8800c110 tx=0x7faf780057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.996+0000 7faf96800640 1 -- 172.21.15.26:0/1907312291 reap_dead start 2024-01-15T17:55:17.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.996+0000 7faf97200640 1 -- 172.21.15.26:0/1907312291 shutdown_connections 2024-01-15T17:55:17.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.996+0000 7faf97200640 1 -- 172.21.15.26:0/1907312291 >> 172.21.15.26:0/1907312291 conn(0x7faf9806a7d0 msgr2=0x7faf9806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:17.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.996+0000 7faf97200640 1 -- 172.21.15.26:0/1907312291 shutdown_connections 2024-01-15T17:55:17.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.996+0000 7faf97200640 1 -- 172.21.15.26:0/1907312291 wait complete. 2024-01-15T17:55:17.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.998+0000 7faf97200640 1 Processor -- start 2024-01-15T17:55:17.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:16.999+0000 7faf97200640 1 -- start start 2024-01-15T17:55:17.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf97200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf78003970 0x7faf98138810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf97200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf980ac8d0 0x7faf980acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf97200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9813b0c0 0x7faf9813b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf97200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf980ad290 con 0x7faf9813b0c0 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf97200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9813ba60 con 0x7faf78003970 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf97200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf980b0510 con 0x7faf980ac8d0 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf96800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9813b0c0 0x7faf9813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf96800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9813b0c0 0x7faf9813b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44684/0 (socket says 172.21.15.26:44684) 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf96800640 1 -- 172.21.15.26:0/3508923952 learned_addr learned my addr 172.21.15.26:0/3508923952 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf8fe00640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf980ac8d0 0x7faf980acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.000+0000 7faf95e00640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf78003970 0x7faf98138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf96800640 1 -- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf980ac8d0 msgr2=0x7faf980acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf96800640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf980ac8d0 0x7faf980acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf96800640 1 -- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf78003970 msgr2=0x7faf98138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf96800640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf78003970 0x7faf98138810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf96800640 1 -- 172.21.15.26:0/3508923952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9813bd30 con 0x7faf9813b0c0 2024-01-15T17:55:17.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf96800640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9813b0c0 0x7faf9813b520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf980b4270 tx=0x7faf9000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:17.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf8e000640 1 -- 172.21.15.26:0/3508923952 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90010040 con 0x7faf9813b0c0 2024-01-15T17:55:17.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf8e000640 1 -- 172.21.15.26:0/3508923952 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9000bce0 con 0x7faf9813b0c0 2024-01-15T17:55:17.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf8e000640 1 -- 172.21.15.26:0/3508923952 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90014640 con 0x7faf9813b0c0 2024-01-15T17:55:17.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.001+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9813c010 con 0x7faf9813b0c0 2024-01-15T17:55:17.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.002+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf980b2210 con 0x7faf9813b0c0 2024-01-15T17:55:17.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.003+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf98067450 con 0x7faf9813b0c0 2024-01-15T17:55:17.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.003+0000 7faf8e000640 1 -- 172.21.15.26:0/3508923952 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7faf900147a0 con 0x7faf9813b0c0 2024-01-15T17:55:17.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.004+0000 7faf8e000640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf7006b2c0 0x7faf7006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.005+0000 7faf8e000640 1 -- 172.21.15.26:0/3508923952 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf9008a460 con 0x7faf9813b0c0 2024-01-15T17:55:17.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.005+0000 7faf95e00640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf7006b2c0 0x7faf7006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.006+0000 7faf95e00640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf7006b2c0 0x7faf7006d780 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf88034000 tx=0x7faf88007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:17.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.010+0000 7faf8e000640 1 -- 172.21.15.26:0/3508923952 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf90055370 con 0x7faf9813b0c0 2024-01-15T17:55:17.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.209+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf9806ae30 con 0x7faf9813b0c0 2024-01-15T17:55:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.210+0000 7faf8e000640 1 -- 172.21.15.26:0/3508923952 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf90058990 con 0x7faf9813b0c0 2024-01-15T17:55:17.213 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf7006b2c0 msgr2=0x7faf7006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf7006b2c0 0x7faf7006d780 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf88034000 tx=0x7faf88007000 comp rx=0 tx=0).stop 2024-01-15T17:55:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9813b0c0 msgr2=0x7faf9813b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9813b0c0 0x7faf9813b520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf980b4270 tx=0x7faf9000b480 comp rx=0 tx=0).stop 2024-01-15T17:55:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 shutdown_connections 2024-01-15T17:55:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faf980ac8d0 0x7faf980acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf7006b2c0 0x7faf7006d780 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faf9813b0c0 0x7faf9813b520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 --2- 172.21.15.26:0/3508923952 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faf78003970 0x7faf98138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 >> 172.21.15.26:0/3508923952 conn(0x7faf9806a7d0 msgr2=0x7faf98142620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 shutdown_connections 2024-01-15T17:55:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.213+0000 7faf97200640 1 -- 172.21.15.26:0/3508923952 wait complete. 2024-01-15T17:55:17.236 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.462+0000 7fb112200640 1 Processor -- start 2024-01-15T17:55:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.462+0000 7fb112200640 1 -- start start 2024-01-15T17:55:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.463+0000 7fb112200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb10c15d610 0x7fb10c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb112200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c0d1ef0 0x7fb10c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb112200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0b3400 con 0x7fb10c0ced60 2024-01-15T17:55:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb112200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0d1390 con 0x7fb10c15d610 2024-01-15T17:55:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb112200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0b27f0 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb10b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb10c15d610 0x7fb10c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb10b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb10c15d610 0x7fb10c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44688/0 (socket says 172.21.15.26:44688) 2024-01-15T17:55:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb10b400640 1 -- 172.21.15.26:0/3166274284 learned_addr learned my addr 172.21.15.26:0/3166274284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb10be00640 1 --2- 172.21.15.26:0/3166274284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c0d1ef0 0x7fb10c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb10b400640 1 -- 172.21.15.26:0/3166274284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c0d1ef0 msgr2=0x7fb10c0b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb10b400640 1 --2- 172.21.15.26:0/3166274284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c0d1ef0 0x7fb10c0b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb10b400640 1 -- 172.21.15.26:0/3166274284 >> v1:172.21.15.124:6789/0 conn(0x7fb10c0ced60 legacy=0x7fb10c0d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.464+0000 7fb10b400640 1 -- 172.21.15.26:0/3166274284 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c0b2ad0 con 0x7fb10c15d610 2024-01-15T17:55:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.466+0000 7fb10b400640 1 --2- 172.21.15.26:0/3166274284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c15d610 0x7fb10c0ce820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009870 tx=0x7fb0fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0d6d9e6efb21fa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.466+0000 7fb10aa00640 1 -- 172.21.15.26:0/3166274284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc031640 con 0x7fb10c15d610 2024-01-15T17:55:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.466+0000 7fb10aa00640 1 -- 172.21.15.26:0/3166274284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc0317a0 con 0x7fb10c15d610 2024-01-15T17:55:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.466+0000 7fb10aa00640 1 -- 172.21.15.26:0/3166274284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc031b30 con 0x7fb10c15d610 2024-01-15T17:55:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.467+0000 7fb112200640 1 -- 172.21.15.26:0/3166274284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c15d610 msgr2=0x7fb10c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.467+0000 7fb112200640 1 --2- 172.21.15.26:0/3166274284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c15d610 0x7fb10c0ce820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc009870 tx=0x7fb0fc02b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.467+0000 7fb112200640 1 -- 172.21.15.26:0/3166274284 shutdown_connections 2024-01-15T17:55:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.467+0000 7fb112200640 1 --2- 172.21.15.26:0/3166274284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c0d1ef0 0x7fb10c0b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.467+0000 7fb112200640 1 --2- 172.21.15.26:0/3166274284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c15d610 0x7fb10c0ce820 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.467+0000 7fb112200640 1 -- 172.21.15.26:0/3166274284 >> 172.21.15.26:0/3166274284 conn(0x7fb10c09c540 msgr2=0x7fb10c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.467+0000 7fb112200640 1 -- 172.21.15.26:0/3166274284 shutdown_connections 2024-01-15T17:55:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.467+0000 7fb112200640 1 -- 172.21.15.26:0/3166274284 wait complete. 2024-01-15T17:55:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.469+0000 7fb112200640 1 Processor -- start 2024-01-15T17:55:17.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.469+0000 7fb112200640 1 -- start start 2024-01-15T17:55:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb112200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb10c0ced60 0x7fb10c14c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb112200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0d1ef0 0x7fb10c0b8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb112200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c15d610 0x7fb10c14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb112200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c14cb50 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb112200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c14cda0 con 0x7fb10c0ced60 2024-01-15T17:55:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb112200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c14d080 con 0x7fb10c15d610 2024-01-15T17:55:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb102800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0d1ef0 0x7fb10c0b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb102800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0d1ef0 0x7fb10c0b8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44698/0 (socket says 172.21.15.26:44698) 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb102800640 1 -- 172.21.15.26:0/2546031512 learned_addr learned my addr 172.21.15.26:0/2546031512 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb10b400640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb10c0ced60 0x7fb10c14c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb10be00640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c15d610 0x7fb10c14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb102800640 1 -- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c15d610 msgr2=0x7fb10c14ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb102800640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c15d610 0x7fb10c14ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb102800640 1 -- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb10c0ced60 msgr2=0x7fb10c14c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb102800640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb10c0ced60 0x7fb10c14c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.470+0000 7fb102800640 1 -- 172.21.15.26:0/2546031512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc0094d0 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.471+0000 7fb102800640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0d1ef0 0x7fb10c0b8840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400c430 tx=0x7fb0f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.471+0000 7fb103e00640 1 -- 172.21.15.26:0/2546031512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4011790 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.471+0000 7fb103e00640 1 -- 172.21.15.26:0/2546031512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f40118f0 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.471+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c14d360 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.471+0000 7fb103e00640 1 -- 172.21.15.26:0/2546031512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4011ce0 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.471+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c14c7b0 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.472+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c14d880 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.473+0000 7fb103e00640 1 -- 172.21.15.26:0/2546031512 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb0f401c460 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.474+0000 7fb103e00640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0e406b2c0 0x7fb0e406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.474+0000 7fb103e00640 1 -- 172.21.15.26:0/2546031512 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0f4089af0 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.475+0000 7fb10b400640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0e406b2c0 0x7fb0e406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.476+0000 7fb10b400640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0e406b2c0 0x7fb0e406d780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc036000 tx=0x7fb0fc03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:17.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.478+0000 7fb103e00640 1 -- 172.21.15.26:0/2546031512 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb10c14d880 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.669+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb10c14d880 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.670+0000 7fb103e00640 1 -- 172.21.15.26:0/2546031512 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb10c14d880 con 0x7fb10c0d1ef0 2024-01-15T17:55:17.673 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0e406b2c0 msgr2=0x7fb0e406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0e406b2c0 0x7fb0e406d780 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc036000 tx=0x7fb0fc03c040 comp rx=0 tx=0).stop 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0d1ef0 msgr2=0x7fb10c0b8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0d1ef0 0x7fb10c0b8840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400c430 tx=0x7fb0f400c8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 shutdown_connections 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10c15d610 0x7fb10c14ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0e406b2c0 0x7fb0e406d780 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10c0d1ef0 0x7fb10c0b8840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 --2- 172.21.15.26:0/2546031512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb10c0ced60 0x7fb10c14c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.673+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 >> 172.21.15.26:0/2546031512 conn(0x7fb10c09c540 msgr2=0x7fb10c0b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.674+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 shutdown_connections 2024-01-15T17:55:17.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.674+0000 7fb112200640 1 -- 172.21.15.26:0/2546031512 wait complete. 2024-01-15T17:55:17.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:55:17.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.920+0000 7fabc0c00640 1 Processor -- start 2024-01-15T17:55:17.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.920+0000 7fabc0c00640 1 -- start start 2024-01-15T17:55:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.922+0000 7fabc0c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabbc160920 0x7fabbc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.922+0000 7fabc0c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 0x7fabbc0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.922+0000 7fabc0c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fabbc161010 0x7fabbc1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.922+0000 7fabc0c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabbc0d03b0 con 0x7fabbc160920 2024-01-15T17:55:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.922+0000 7fabc0c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabbc0d0a10 con 0x7fabbc161010 2024-01-15T17:55:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.922+0000 7fabc0c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0cf080 con 0x7fabbc0cf9f0 2024-01-15T17:55:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.922+0000 7fabbb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabbc160920 0x7fabbc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.922+0000 7fabbb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabbc160920 0x7fabbc0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44704/0 (socket says 172.21.15.26:44704) 2024-01-15T17:55:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.922+0000 7fabbb400640 1 -- 172.21.15.26:0/3102173984 learned_addr learned my addr 172.21.15.26:0/3102173984 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.923+0000 7fabbbe00640 1 --2- 172.21.15.26:0/3102173984 >> v2:172.21.15.124:3300/0 conn(0x7fabbc161010 0x7fabbc1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.923+0000 7fabbb400640 1 -- 172.21.15.26:0/3102173984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 msgr2=0x7fabbc0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.923+0000 7fabb3e00640 1 --2- 172.21.15.26:0/3102173984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 0x7fabbc0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.923+0000 7fabbb400640 1 --2- 172.21.15.26:0/3102173984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 0x7fabbc0cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.923+0000 7fabbb400640 1 -- 172.21.15.26:0/3102173984 >> v2:172.21.15.124:3300/0 conn(0x7fabbc161010 msgr2=0x7fabbc1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.923+0000 7fabbb400640 1 --2- 172.21.15.26:0/3102173984 >> v2:172.21.15.124:3300/0 conn(0x7fabbc161010 0x7fabbc1614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.923+0000 7fabbb400640 1 -- 172.21.15.26:0/3102173984 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc0c6be0 con 0x7fabbc160920 2024-01-15T17:55:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.923+0000 7fabbbe00640 1 --2- 172.21.15.26:0/3102173984 >> v2:172.21.15.124:3300/0 conn(0x7fabbc161010 0x7fabbc1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.924+0000 7fabbb400640 1 --2- 172.21.15.26:0/3102173984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 0x7fabbc0b3f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0c6930 tx=0x7fabac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7443ae27bdb08f11 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.924+0000 7fabbaa00640 1 -- 172.21.15.26:0/3102173984 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac031870 con 0x7fabbc160920 2024-01-15T17:55:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.924+0000 7fabbaa00640 1 -- 172.21.15.26:0/3102173984 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac0319d0 con 0x7fabbc160920 2024-01-15T17:55:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.924+0000 7fabbaa00640 1 -- 172.21.15.26:0/3102173984 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac031d60 con 0x7fabbc160920 2024-01-15T17:55:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.925+0000 7fabc0c00640 1 -- 172.21.15.26:0/3102173984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 msgr2=0x7fabbc0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.925+0000 7fabc0c00640 1 --2- 172.21.15.26:0/3102173984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 0x7fabbc0b3f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0c6930 tx=0x7fabac02b910 comp rx=0 tx=0).stop 2024-01-15T17:55:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.925+0000 7fabc0c00640 1 -- 172.21.15.26:0/3102173984 shutdown_connections 2024-01-15T17:55:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.925+0000 7fabc0c00640 1 --2- 172.21.15.26:0/3102173984 >> v2:172.21.15.124:3300/0 conn(0x7fabbc161010 0x7fabbc1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.925+0000 7fabc0c00640 1 --2- 172.21.15.26:0/3102173984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 0x7fabbc0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.925+0000 7fabc0c00640 1 --2- 172.21.15.26:0/3102173984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 0x7fabbc0b3f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.925+0000 7fabc0c00640 1 -- 172.21.15.26:0/3102173984 >> 172.21.15.26:0/3102173984 conn(0x7fabbc0d3090 msgr2=0x7fabbc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.925+0000 7fabc0c00640 1 -- 172.21.15.26:0/3102173984 shutdown_connections 2024-01-15T17:55:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.925+0000 7fabc0c00640 1 -- 172.21.15.26:0/3102173984 wait complete. 2024-01-15T17:55:17.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.927+0000 7fabc0c00640 1 Processor -- start 2024-01-15T17:55:17.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.927+0000 7fabc0c00640 1 -- start start 2024-01-15T17:55:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.928+0000 7fabc0c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 0x7fabbc1f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.928+0000 7fabc0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 0x7fabbc0b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.928+0000 7fabc0c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabbc161010 0x7fabbc0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.928+0000 7fabc0c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0b7210 con 0x7fabbc160920 2024-01-15T17:55:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.928+0000 7fabc0c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0cb5d0 con 0x7fabbc161010 2024-01-15T17:55:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.928+0000 7fabc0c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc0cb8b0 con 0x7fabbc0cf9f0 2024-01-15T17:55:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.928+0000 7fabb3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 0x7fabbc0b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabb3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 0x7fabbc0b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44706/0 (socket says 172.21.15.26:44706) 2024-01-15T17:55:17.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabb3e00640 1 -- 172.21.15.26:0/2833958930 learned_addr learned my addr 172.21.15.26:0/2833958930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabbb400640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 0x7fabbc1f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabbbe00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabbc161010 0x7fabbc0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabb3e00640 1 -- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 msgr2=0x7fabbc1f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabb3e00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 0x7fabbc1f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabb3e00640 1 -- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabbc161010 msgr2=0x7fabbc0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabb3e00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabbc161010 0x7fabbc0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabb3e00640 1 -- 172.21.15.26:0/2833958930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0094d0 con 0x7fabbc160920 2024-01-15T17:55:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.929+0000 7fabb3e00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 0x7fabbc0b68e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faba400c7f0 tx=0x7faba400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.930+0000 7fabb3400640 1 -- 172.21.15.26:0/2833958930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4004220 con 0x7fabbc160920 2024-01-15T17:55:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.930+0000 7fabb3400640 1 -- 172.21.15.26:0/2833958930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4004380 con 0x7fabbc160920 2024-01-15T17:55:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.930+0000 7fabb3400640 1 -- 172.21.15.26:0/2833958930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4007860 con 0x7fabbc160920 2024-01-15T17:55:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.930+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabbc0cbb90 con 0x7fabbc160920 2024-01-15T17:55:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.930+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabbc1f6070 con 0x7fabbc160920 2024-01-15T17:55:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.931+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabbc067450 con 0x7fabbc160920 2024-01-15T17:55:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.932+0000 7fabb3400640 1 -- 172.21.15.26:0/2833958930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7faba4007bd0 con 0x7fabbc160920 2024-01-15T17:55:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.933+0000 7fabb3400640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9406b2f0 0x7fab9406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.934+0000 7fabb3400640 1 -- 172.21.15.26:0/2833958930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faba4089ba0 con 0x7fabbc160920 2024-01-15T17:55:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.934+0000 7fabbb400640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9406b2f0 0x7fab9406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:17.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.935+0000 7fabbb400640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9406b2f0 0x7fab9406d7b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fabac036000 tx=0x7fabac03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:17.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:17.938+0000 7fabb3400640 1 -- 172.21.15.26:0/2833958930 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faba4054970 con 0x7fabbc160920 2024-01-15T17:55:18.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.173+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabbc08c7e0 con 0x7fabbc160920 2024-01-15T17:55:18.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.174+0000 7fabb3400640 1 -- 172.21.15.26:0/2833958930 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7faba4057f90 con 0x7fabbc160920 2024-01-15T17:55:18.177 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:18.178 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/593578542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":593578542},{"type":"v1","addr":"172.21.15.124:6837","nonce":593578542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1776325954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1776325954},{"type":"v1","addr":"172.21.15.26:6835","nonce":1776325954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:55:04.195734+0000","modified":"2024-01-15T17:55:07.244782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/3022017251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3022017251},{"type":"v1","addr":"172.21.15.124:6835","nonce":3022017251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T17:55:18.178 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9406b2f0 msgr2=0x7fab9406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9406b2f0 0x7fab9406d7b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fabac036000 tx=0x7fabac03c040 comp rx=0 tx=0).stop 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 msgr2=0x7fabbc0b68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 0x7fabbc0b68e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7faba400c7f0 tx=0x7faba400ccb0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 shutdown_connections 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9406b2f0 0x7fab9406d7b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabbc161010 0x7fabbc0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabbc160920 0x7fabbc0b68e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 --2- 172.21.15.26:0/2833958930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabbc0cf9f0 0x7fabbc1f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.177+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 >> 172.21.15.26:0/2833958930 conn(0x7fabbc0d3090 msgr2=0x7fabbc06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.178+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 shutdown_connections 2024-01-15T17:55:18.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.178+0000 7fabc0c00640 1 -- 172.21.15.26:0/2833958930 wait complete. 2024-01-15T17:55:18.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:18.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.406+0000 7f9492c00640 1 Processor -- start 2024-01-15T17:55:18.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.406+0000 7f9492c00640 1 -- start start 2024-01-15T17:55:18.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.408+0000 7f9492c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f948c143c40 0x7f948c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.408+0000 7f9492c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f948c0717d0 0x7f948c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.408+0000 7f9492c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c070e50 con 0x7f948c0cca40 2024-01-15T17:55:18.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.408+0000 7f9492c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0723c0 con 0x7f948c143c40 2024-01-15T17:55:18.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.408+0000 7f9492c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0cce40 con 0x7f948c0717d0 2024-01-15T17:55:18.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.408+0000 7f9491800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f948c143c40 0x7f948c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.408+0000 7f9491800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f948c143c40 0x7f948c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44722/0 (socket says 172.21.15.26:44722) 2024-01-15T17:55:18.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.408+0000 7f9491800640 1 -- 172.21.15.26:0/3466600743 learned_addr learned my addr 172.21.15.26:0/3466600743 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:18.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.409+0000 7f9490e00640 1 --2- 172.21.15.26:0/3466600743 >> v2:172.21.15.124:3300/0 conn(0x7f948c0717d0 0x7f948c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.409+0000 7f9491800640 1 -- 172.21.15.26:0/3466600743 >> v2:172.21.15.124:3300/0 conn(0x7f948c0717d0 msgr2=0x7f948c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:18.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.409+0000 7f9491800640 1 --2- 172.21.15.26:0/3466600743 >> v2:172.21.15.124:3300/0 conn(0x7f948c0717d0 0x7f948c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.409+0000 7f9491800640 1 -- 172.21.15.26:0/3466600743 >> v1:172.21.15.124:6789/0 conn(0x7f948c0cca40 legacy=0x7f948c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:18.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.409+0000 7f9491800640 1 -- 172.21.15.26:0/3466600743 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c06e170 con 0x7f948c143c40 2024-01-15T17:55:18.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.409+0000 7f9490e00640 1 --2- 172.21.15.26:0/3466600743 >> v2:172.21.15.124:3300/0 conn(0x7f948c0717d0 0x7f948c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.410+0000 7f9491800640 1 --2- 172.21.15.26:0/3466600743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c143c40 0x7f948c071290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f948c0b44a0 tx=0x7f947c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1091e57f729ce801 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.410+0000 7f948be00640 1 -- 172.21.15.26:0/3466600743 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0317e0 con 0x7f948c143c40 2024-01-15T17:55:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.410+0000 7f948be00640 1 -- 172.21.15.26:0/3466600743 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c031940 con 0x7f948c143c40 2024-01-15T17:55:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.410+0000 7f948be00640 1 -- 172.21.15.26:0/3466600743 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c031cd0 con 0x7f948c143c40 2024-01-15T17:55:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.411+0000 7f9492c00640 1 -- 172.21.15.26:0/3466600743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c143c40 msgr2=0x7f948c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.411+0000 7f9492c00640 1 --2- 172.21.15.26:0/3466600743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c143c40 0x7f948c071290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f948c0b44a0 tx=0x7f947c02b880 comp rx=0 tx=0).stop 2024-01-15T17:55:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.411+0000 7f9492c00640 1 -- 172.21.15.26:0/3466600743 shutdown_connections 2024-01-15T17:55:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.411+0000 7f9492c00640 1 --2- 172.21.15.26:0/3466600743 >> v2:172.21.15.124:3300/0 conn(0x7f948c0717d0 0x7f948c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.411+0000 7f9492c00640 1 --2- 172.21.15.26:0/3466600743 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c143c40 0x7f948c071290 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.411+0000 7f9492c00640 1 -- 172.21.15.26:0/3466600743 >> 172.21.15.26:0/3466600743 conn(0x7f948c06a800 msgr2=0x7f948c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.411+0000 7f9492c00640 1 -- 172.21.15.26:0/3466600743 shutdown_connections 2024-01-15T17:55:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.411+0000 7f9492c00640 1 -- 172.21.15.26:0/3466600743 wait complete. 2024-01-15T17:55:18.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.413+0000 7f9492c00640 1 Processor -- start 2024-01-15T17:55:18.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.413+0000 7f9492c00640 1 -- start start 2024-01-15T17:55:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.414+0000 7f9492c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c0717d0 0x7f948c1f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.414+0000 7f9492c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f948c0cca40 0x7f948c1f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.414+0000 7f9492c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f948c143c40 0x7f948c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.414+0000 7f9492c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c1f7560 con 0x7f948c0717d0 2024-01-15T17:55:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.414+0000 7f9492c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0b0890 con 0x7f948c0cca40 2024-01-15T17:55:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.414+0000 7f9492c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c1f6cc0 con 0x7f948c143c40 2024-01-15T17:55:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.414+0000 7f9491800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c0717d0 0x7f948c1f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.414+0000 7f9491800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c0717d0 0x7f948c1f65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44726/0 (socket says 172.21.15.26:44726) 2024-01-15T17:55:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.414+0000 7f9491800640 1 -- 172.21.15.26:0/148693090 learned_addr learned my addr 172.21.15.26:0/148693090 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.415+0000 7f9490e00640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f948c0cca40 0x7f948c1f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.415+0000 7f9491800640 1 -- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f948c143c40 msgr2=0x7f948c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.415+0000 7f9492200640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f948c143c40 0x7f948c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.415+0000 7f9491800640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f948c143c40 0x7f948c1537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.415+0000 7f9491800640 1 -- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f948c0cca40 msgr2=0x7f948c1f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.415+0000 7f9491800640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f948c0cca40 0x7f948c1f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.415+0000 7f9491800640 1 -- 172.21.15.26:0/148693090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0094d0 con 0x7f948c0717d0 2024-01-15T17:55:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.415+0000 7f9491800640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c0717d0 0x7f948c1f65b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f947c02b8c0 tx=0x7f947c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.416+0000 7f9489600640 1 -- 172.21.15.26:0/148693090 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0329c0 con 0x7f948c0717d0 2024-01-15T17:55:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.416+0000 7f9489600640 1 -- 172.21.15.26:0/148693090 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947c032b40 con 0x7f948c0717d0 2024-01-15T17:55:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.416+0000 7f9489600640 1 -- 172.21.15.26:0/148693090 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947c0453f0 con 0x7f948c0717d0 2024-01-15T17:55:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.416+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948c1f6fa0 con 0x7f948c0717d0 2024-01-15T17:55:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.416+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c1532d0 con 0x7f948c0717d0 2024-01-15T17:55:18.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.417+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c153130 con 0x7f948c0717d0 2024-01-15T17:55:18.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.418+0000 7f9489600640 1 -- 172.21.15.26:0/148693090 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f947c04e450 con 0x7f948c0717d0 2024-01-15T17:55:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.419+0000 7f9489600640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f946406b2c0 0x7f946406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.419+0000 7f9489600640 1 -- 172.21.15.26:0/148693090 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f947c0abf00 con 0x7f948c0717d0 2024-01-15T17:55:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.419+0000 7f9490e00640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f946406b2c0 0x7f946406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.420+0000 7f9490e00640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f946406b2c0 0x7f946406d780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f948c0af4f0 tx=0x7f9474008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.423+0000 7f9489600640 1 -- 172.21.15.26:0/148693090 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f948c153130 con 0x7f948c0717d0 2024-01-15T17:55:18.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.617+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f948c153130 con 0x7f948c0717d0 2024-01-15T17:55:18.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.619+0000 7f9489600640 1 -- 172.21.15.26:0/148693090 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f948c153130 con 0x7f948c0717d0 2024-01-15T17:55:18.622 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:18.622 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.621+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f946406b2c0 msgr2=0x7f946406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.621+0000 7f9492c00640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f946406b2c0 0x7f946406d780 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f948c0af4f0 tx=0x7f9474008040 comp rx=0 tx=0).stop 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.621+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c0717d0 msgr2=0x7f948c1f65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.621+0000 7f9492c00640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c0717d0 0x7f948c1f65b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f947c02b8c0 tx=0x7f947c0046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.622+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 shutdown_connections 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.622+0000 7f9492c00640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f948c143c40 0x7f948c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.622+0000 7f9492c00640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f946406b2c0 0x7f946406d780 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.622+0000 7f9492c00640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f948c0cca40 0x7f948c1f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.622+0000 7f9492c00640 1 --2- 172.21.15.26:0/148693090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f948c0717d0 0x7f948c1f65b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.622+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 >> 172.21.15.26:0/148693090 conn(0x7f948c06a800 msgr2=0x7f948c13f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:18.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.622+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 shutdown_connections 2024-01-15T17:55:18.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.622+0000 7f9492c00640 1 -- 172.21.15.26:0/148693090 wait complete. 2024-01-15T17:55:18.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:18.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:18.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.870+0000 7fa140a00640 1 Processor -- start 2024-01-15T17:55:18.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.870+0000 7fa140a00640 1 -- start start 2024-01-15T17:55:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.872+0000 7fa140a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13fcb0 0x7fa13c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.872+0000 7fa140a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa13c0cdc20 0x7fa13c0b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.872+0000 7fa140a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13c0ce440 con 0x7fa13c13cf10 2024-01-15T17:55:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.872+0000 7fa140a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13c0ce720 con 0x7fa13c0cdc20 2024-01-15T17:55:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.872+0000 7fa140a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c0cea00 con 0x7fa13c13fcb0 2024-01-15T17:55:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.872+0000 7fa13be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa13c0cdc20 0x7fa13c0b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.872+0000 7fa13be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa13c0cdc20 0x7fa13c0b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:59114/0 (socket says 172.21.15.26:59114) 2024-01-15T17:55:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.872+0000 7fa13be00640 1 -- 172.21.15.26:0/2990483123 learned_addr learned my addr 172.21.15.26:0/2990483123 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.873+0000 7fa133e00640 1 --2- 172.21.15.26:0/2990483123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13fcb0 0x7fa13c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.873+0000 7fa13be00640 1 -- 172.21.15.26:0/2990483123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13fcb0 msgr2=0x7fa13c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.873+0000 7fa13be00640 1 --2- 172.21.15.26:0/2990483123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13fcb0 0x7fa13c0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.873+0000 7fa13be00640 1 -- 172.21.15.26:0/2990483123 >> v1:172.21.15.124:6789/0 conn(0x7fa13c13cf10 legacy=0x7fa13c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.873+0000 7fa13be00640 1 -- 172.21.15.26:0/2990483123 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c0b1d90 con 0x7fa13c0cdc20 2024-01-15T17:55:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.875+0000 7fa13be00640 1 --2- 172.21.15.26:0/2990483123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c0cdc20 0x7fa13c0b2590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa134009e50 tx=0x7fa13402e170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ec11b3734c0ac56 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.876+0000 7fa13aa00640 1 -- 172.21.15.26:0/2990483123 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13402ed10 con 0x7fa13c0cdc20 2024-01-15T17:55:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.876+0000 7fa13aa00640 1 -- 172.21.15.26:0/2990483123 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13402ee70 con 0x7fa13c0cdc20 2024-01-15T17:55:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.876+0000 7fa13aa00640 1 -- 172.21.15.26:0/2990483123 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1340365f0 con 0x7fa13c0cdc20 2024-01-15T17:55:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.876+0000 7fa140a00640 1 -- 172.21.15.26:0/2990483123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c0cdc20 msgr2=0x7fa13c0b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.876+0000 7fa140a00640 1 --2- 172.21.15.26:0/2990483123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c0cdc20 0x7fa13c0b2590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa134009e50 tx=0x7fa13402e170 comp rx=0 tx=0).stop 2024-01-15T17:55:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.877+0000 7fa140a00640 1 -- 172.21.15.26:0/2990483123 shutdown_connections 2024-01-15T17:55:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.877+0000 7fa140a00640 1 --2- 172.21.15.26:0/2990483123 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13fcb0 0x7fa13c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.877+0000 7fa140a00640 1 --2- 172.21.15.26:0/2990483123 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c0cdc20 0x7fa13c0b2590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.877+0000 7fa140a00640 1 -- 172.21.15.26:0/2990483123 >> 172.21.15.26:0/2990483123 conn(0x7fa13c06a7d0 msgr2=0x7fa13c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.877+0000 7fa140a00640 1 -- 172.21.15.26:0/2990483123 shutdown_connections 2024-01-15T17:55:18.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.877+0000 7fa140a00640 1 -- 172.21.15.26:0/2990483123 wait complete. 2024-01-15T17:55:18.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.879+0000 7fa140a00640 1 Processor -- start 2024-01-15T17:55:18.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.879+0000 7fa140a00640 1 -- start start 2024-01-15T17:55:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa140a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c0cdc20 0x7fa13c13e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa140a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13cf10 0x7fa13c152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa140a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c13fcb0 0x7fa13c070160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa140a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c13d780 con 0x7fa13c0cdc20 2024-01-15T17:55:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa140a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c13da60 con 0x7fa13c13fcb0 2024-01-15T17:55:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa140a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c13dd40 con 0x7fa13c13cf10 2024-01-15T17:55:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa13b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c0cdc20 0x7fa13c13e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa13b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c0cdc20 0x7fa13c13e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44738/0 (socket says 172.21.15.26:44738) 2024-01-15T17:55:18.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa13b400640 1 -- 172.21.15.26:0/2942320040 learned_addr learned my addr 172.21.15.26:0/2942320040 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:18.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa133e00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13cf10 0x7fa13c152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.880+0000 7fa13be00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c13fcb0 0x7fa13c070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa13b400640 1 -- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13cf10 msgr2=0x7fa13c152650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa13b400640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13cf10 0x7fa13c152650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa13b400640 1 -- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c13fcb0 msgr2=0x7fa13c070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa13b400640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c13fcb0 0x7fa13c070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa13b400640 1 -- 172.21.15.26:0/2942320040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa134009b30 con 0x7fa13c0cdc20 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa133e00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13cf10 0x7fa13c152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa13be00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c13fcb0 0x7fa13c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa13b400640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c0cdc20 0x7fa13c13e820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00c450 tx=0x7fa12c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa133400640 1 -- 172.21.15.26:0/2942320040 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c0052d0 con 0x7fa13c0cdc20 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa133400640 1 -- 172.21.15.26:0/2942320040 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12c005430 con 0x7fa13c0cdc20 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa133400640 1 -- 172.21.15.26:0/2942320040 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12c012870 con 0x7fa13c0cdc20 2024-01-15T17:55:18.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c13e020 con 0x7fa13c0cdc20 2024-01-15T17:55:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.881+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c152b90 con 0x7fa13c0cdc20 2024-01-15T17:55:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.883+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c067450 con 0x7fa13c0cdc20 2024-01-15T17:55:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.884+0000 7fa133400640 1 -- 172.21.15.26:0/2942320040 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa12c0129d0 con 0x7fa13c0cdc20 2024-01-15T17:55:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.885+0000 7fa133400640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa11406b2c0 0x7fa11406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:18.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.885+0000 7fa133400640 1 -- 172.21.15.26:0/2942320040 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa12c089700 con 0x7fa13c0cdc20 2024-01-15T17:55:18.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.885+0000 7fa133e00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa11406b2c0 0x7fa11406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:18.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.886+0000 7fa133e00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa11406b2c0 0x7fa11406d780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa13c13c3d0 tx=0x7fa124008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:18.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:18.889+0000 7fa133400640 1 -- 172.21.15.26:0/2942320040 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa12c054570 con 0x7fa13c0cdc20 2024-01-15T17:55:19.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.083+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa13c0cbf10 con 0x7fa13c0cdc20 2024-01-15T17:55:19.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.084+0000 7fa133400640 1 -- 172.21.15.26:0/2942320040 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa12c057b90 con 0x7fa13c0cdc20 2024-01-15T17:55:19.087 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.086+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa11406b2c0 msgr2=0x7fa11406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.086+0000 7fa140a00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa11406b2c0 0x7fa11406d780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa13c13c3d0 tx=0x7fa124008040 comp rx=0 tx=0).stop 2024-01-15T17:55:19.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.086+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c0cdc20 msgr2=0x7fa13c13e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.086+0000 7fa140a00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c0cdc20 0x7fa13c13e820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa12c00c450 tx=0x7fa12c00c910 comp rx=0 tx=0).stop 2024-01-15T17:55:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.087+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 shutdown_connections 2024-01-15T17:55:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.087+0000 7fa140a00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa13c13cf10 0x7fa13c152650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.087+0000 7fa140a00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa11406b2c0 0x7fa11406d780 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.087+0000 7fa140a00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa13c13fcb0 0x7fa13c070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.087+0000 7fa140a00640 1 --2- 172.21.15.26:0/2942320040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa13c0cdc20 0x7fa13c13e820 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.087+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 >> 172.21.15.26:0/2942320040 conn(0x7fa13c06a7d0 msgr2=0x7fa13c06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:19.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.087+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 shutdown_connections 2024-01-15T17:55:19.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.087+0000 7fa140a00640 1 -- 172.21.15.26:0/2942320040 wait complete. 2024-01-15T17:55:19.108 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:19.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.318+0000 7fcedb200640 1 Processor -- start 2024-01-15T17:55:19.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.318+0000 7fcedb200640 1 -- start start 2024-01-15T17:55:19.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fcedb200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcedc0adf20 0x7fcedc070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fcedb200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 0x7fcedc0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fcedb200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcedc0cc980 0x7fcedc143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fcedb200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc15f970 con 0x7fcedc0cc980 2024-01-15T17:55:19.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fcedb200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc15f4e0 con 0x7fcedc0adf20 2024-01-15T17:55:19.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fcedb200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc0cce10 con 0x7fcedc15bf10 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fceda800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcedc0cc980 0x7fcedc143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fceda800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcedc0cc980 0x7fcedc143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44740/0 (socket says 172.21.15.26:44740) 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fceda800640 1 -- 172.21.15.26:0/3105468745 learned_addr learned my addr 172.21.15.26:0/3105468745 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fced9e00640 1 --2- 172.21.15.26:0/3105468745 >> v2:172.21.15.124:3300/0 conn(0x7fcedc0adf20 0x7fcedc070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.320+0000 7fced1200640 1 --2- 172.21.15.26:0/3105468745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 0x7fcedc0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.321+0000 7fceda800640 1 -- 172.21.15.26:0/3105468745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 msgr2=0x7fcedc0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.321+0000 7fceda800640 1 --2- 172.21.15.26:0/3105468745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 0x7fcedc0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.321+0000 7fceda800640 1 -- 172.21.15.26:0/3105468745 >> v2:172.21.15.124:3300/0 conn(0x7fcedc0adf20 msgr2=0x7fcedc070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.321+0000 7fceda800640 1 --2- 172.21.15.26:0/3105468745 >> v2:172.21.15.124:3300/0 conn(0x7fcedc0adf20 0x7fcedc070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.321+0000 7fceda800640 1 -- 172.21.15.26:0/3105468745 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc071130 con 0x7fcedc0cc980 2024-01-15T17:55:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.321+0000 7fced9e00640 1 --2- 172.21.15.26:0/3105468745 >> v2:172.21.15.124:3300/0 conn(0x7fcedc0adf20 0x7fcedc070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.322+0000 7fceda800640 1 --2- 172.21.15.26:0/3105468745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0cc980 0x7fcedc143d30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0b4150 tx=0x7fced402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a44e524992e331 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:19.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fced9400640 1 -- 172.21.15.26:0/3105468745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced40317e0 con 0x7fcedc0cc980 2024-01-15T17:55:19.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fced9400640 1 -- 172.21.15.26:0/3105468745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced4031940 con 0x7fcedc0cc980 2024-01-15T17:55:19.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fced9400640 1 -- 172.21.15.26:0/3105468745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4031cd0 con 0x7fcedc0cc980 2024-01-15T17:55:19.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fcedb200640 1 -- 172.21.15.26:0/3105468745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0cc980 msgr2=0x7fcedc143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fcedb200640 1 --2- 172.21.15.26:0/3105468745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0cc980 0x7fcedc143d30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0b4150 tx=0x7fced402b880 comp rx=0 tx=0).stop 2024-01-15T17:55:19.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fcedb200640 1 -- 172.21.15.26:0/3105468745 shutdown_connections 2024-01-15T17:55:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fcedb200640 1 --2- 172.21.15.26:0/3105468745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0cc980 0x7fcedc143d30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fcedb200640 1 --2- 172.21.15.26:0/3105468745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 0x7fcedc0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fcedb200640 1 --2- 172.21.15.26:0/3105468745 >> v2:172.21.15.124:3300/0 conn(0x7fcedc0adf20 0x7fcedc070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fcedb200640 1 -- 172.21.15.26:0/3105468745 >> 172.21.15.26:0/3105468745 conn(0x7fcedc06a7d0 msgr2=0x7fcedc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.323+0000 7fcedb200640 1 -- 172.21.15.26:0/3105468745 shutdown_connections 2024-01-15T17:55:19.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.324+0000 7fcedb200640 1 -- 172.21.15.26:0/3105468745 wait complete. 2024-01-15T17:55:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.325+0000 7fcedb200640 1 Processor -- start 2024-01-15T17:55:19.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.326+0000 7fcedb200640 1 -- start start 2024-01-15T17:55:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fcedb200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0adf20 0x7fcedc1375e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fcedb200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcedc0cc980 0x7fcedc15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fcedb200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 0x7fcedc1f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fcedb200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc13ade0 con 0x7fcedc0adf20 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fcedb200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc137150 con 0x7fcedc0cc980 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fcedb200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc1f6c90 con 0x7fcedc15bf10 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fced9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0adf20 0x7fcedc1375e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fced9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0adf20 0x7fcedc1375e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44756/0 (socket says 172.21.15.26:44756) 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fced9e00640 1 -- 172.21.15.26:0/211953527 learned_addr learned my addr 172.21.15.26:0/211953527 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fced1200640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcedc0cc980 0x7fcedc15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fceda800640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 0x7fcedc1f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fced9e00640 1 -- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 msgr2=0x7fcedc1f5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fced9e00640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 0x7fcedc1f5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fced9e00640 1 -- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcedc0cc980 msgr2=0x7fcedc15b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fced9e00640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcedc0cc980 0x7fcedc15b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fced9e00640 1 -- 172.21.15.26:0/211953527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced40094d0 con 0x7fcedc0adf20 2024-01-15T17:55:19.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.327+0000 7fceda800640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 0x7fcedc1f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.328+0000 7fced9e00640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0adf20 0x7fcedc1375e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007e10 tx=0x7fcecc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.328+0000 7fced2a00640 1 -- 172.21.15.26:0/211953527 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc003b90 con 0x7fcedc0adf20 2024-01-15T17:55:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.328+0000 7fced2a00640 1 -- 172.21.15.26:0/211953527 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc00f040 con 0x7fcedc0adf20 2024-01-15T17:55:19.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.328+0000 7fced2a00640 1 -- 172.21.15.26:0/211953527 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc00de80 con 0x7fcedc0adf20 2024-01-15T17:55:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.328+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc1f69b0 con 0x7fcedc0adf20 2024-01-15T17:55:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.328+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc136de0 con 0x7fcedc0adf20 2024-01-15T17:55:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.329+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc15eb90 con 0x7fcedc0adf20 2024-01-15T17:55:19.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.331+0000 7fced2a00640 1 -- 172.21.15.26:0/211953527 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fcecc004030 con 0x7fcedc0adf20 2024-01-15T17:55:19.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.332+0000 7fced2a00640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fceb406b2c0 0x7fceb406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.332+0000 7fced2a00640 1 -- 172.21.15.26:0/211953527 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcecc089e10 con 0x7fcedc0adf20 2024-01-15T17:55:19.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.332+0000 7fced1200640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fceb406b2c0 0x7fceb406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.333+0000 7fced1200640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fceb406b2c0 0x7fceb406d780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcedc071c70 tx=0x7fcec4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:19.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.336+0000 7fced2a00640 1 -- 172.21.15.26:0/211953527 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcedc15eb90 con 0x7fcedc0adf20 2024-01-15T17:55:19.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.531+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcedc15eb90 con 0x7fcedc0adf20 2024-01-15T17:55:19.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.532+0000 7fced2a00640 1 -- 172.21.15.26:0/211953527 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcecc01a020 con 0x7fcedc0adf20 2024-01-15T17:55:19.535 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:19.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fceb406b2c0 msgr2=0x7fceb406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fceb406b2c0 0x7fceb406d780 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcedc071c70 tx=0x7fcec4008040 comp rx=0 tx=0).stop 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0adf20 msgr2=0x7fcedc1375e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0adf20 0x7fcedc1375e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcecc007e10 tx=0x7fcecc00d4a0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 shutdown_connections 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcedc15bf10 0x7fcedc1f5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fceb406b2c0 0x7fceb406d780 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcedc0cc980 0x7fcedc15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 --2- 172.21.15.26:0/211953527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcedc0adf20 0x7fcedc1375e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 >> 172.21.15.26:0/211953527 conn(0x7fcedc06a7d0 msgr2=0x7fcedc0cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.535+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 shutdown_connections 2024-01-15T17:55:19.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.536+0000 7fcedb200640 1 -- 172.21.15.26:0/211953527 wait complete. 2024-01-15T17:55:19.558 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.764+0000 7f460ce00640 1 Processor -- start 2024-01-15T17:55:19.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.764+0000 7f460ce00640 1 -- start start 2024-01-15T17:55:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.765+0000 7f460ce00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f46080adf20 0x7f4608070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.765+0000 7f460ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f460815bf10 0x7f46080cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.765+0000 7f460ce00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460815f4e0 con 0x7f46080cc980 2024-01-15T17:55:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.765+0000 7f460ce00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460815f970 con 0x7f460815bf10 2024-01-15T17:55:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.765+0000 7f460ce00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46080ccdd0 con 0x7f46080adf20 2024-01-15T17:55:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.765+0000 7f45ffe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f460815bf10 0x7f46080cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.765+0000 7f4607e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f46080cc980 0x7f4608143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43060/0 (socket says 172.21.15.26:43060) 2024-01-15T17:55:19.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.765+0000 7f4607e00640 1 -- 172.21.15.26:0/2038903617 learned_addr learned my addr 172.21.15.26:0/2038903617 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.766+0000 7f4607400640 1 --2- 172.21.15.26:0/2038903617 >> v2:172.21.15.124:3300/0 conn(0x7f46080adf20 0x7f4608070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.766+0000 7f45ffe00640 1 -- 172.21.15.26:0/2038903617 >> v2:172.21.15.124:3300/0 conn(0x7f46080adf20 msgr2=0x7f4608070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.766+0000 7f45ffe00640 1 --2- 172.21.15.26:0/2038903617 >> v2:172.21.15.124:3300/0 conn(0x7f46080adf20 0x7f4608070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.766+0000 7f45ffe00640 1 -- 172.21.15.26:0/2038903617 >> v1:172.21.15.26:6789/0 conn(0x7f46080cc980 legacy=0x7f4608143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.766+0000 7f45ffe00640 1 -- 172.21.15.26:0/2038903617 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608071130 con 0x7f460815bf10 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.767+0000 7f45ffe00640 1 --2- 172.21.15.26:0/2038903617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f460815bf10 0x7f46080cc440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46080b4150 tx=0x7f45f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7788e2ed19b2a25d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.767+0000 7f4606a00640 1 -- 172.21.15.26:0/2038903617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f002e070 con 0x7f460815bf10 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.767+0000 7f4606a00640 1 -- 172.21.15.26:0/2038903617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f00315a0 con 0x7f460815bf10 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.767+0000 7f4606a00640 1 -- 172.21.15.26:0/2038903617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0031930 con 0x7f460815bf10 2024-01-15T17:55:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.768+0000 7f460ce00640 1 -- 172.21.15.26:0/2038903617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f460815bf10 msgr2=0x7f46080cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.768+0000 7f460ce00640 1 --2- 172.21.15.26:0/2038903617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f460815bf10 0x7f46080cc440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46080b4150 tx=0x7f45f002b840 comp rx=0 tx=0).stop 2024-01-15T17:55:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.768+0000 7f460ce00640 1 -- 172.21.15.26:0/2038903617 shutdown_connections 2024-01-15T17:55:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.768+0000 7f460ce00640 1 --2- 172.21.15.26:0/2038903617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f460815bf10 0x7f46080cc440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.768+0000 7f460ce00640 1 --2- 172.21.15.26:0/2038903617 >> v2:172.21.15.124:3300/0 conn(0x7f46080adf20 0x7f4608070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.768+0000 7f460ce00640 1 -- 172.21.15.26:0/2038903617 >> 172.21.15.26:0/2038903617 conn(0x7f460806a7d0 msgr2=0x7f460806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.768+0000 7f460ce00640 1 -- 172.21.15.26:0/2038903617 shutdown_connections 2024-01-15T17:55:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.768+0000 7f460ce00640 1 -- 172.21.15.26:0/2038903617 wait complete. 2024-01-15T17:55:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.770+0000 7f460ce00640 1 Processor -- start 2024-01-15T17:55:19.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.770+0000 7f460ce00640 1 -- start start 2024-01-15T17:55:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f460ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46080adf20 0x7f46081f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f460ce00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46080cc980 0x7f4608157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f460ce00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f460815bf10 0x7f46080ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f460ce00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46081f6080 con 0x7f46080adf20 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f460ce00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46081f2c50 con 0x7f460815bf10 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f460ce00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4608158460 con 0x7f46080cc980 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f4607400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46080adf20 0x7f46081f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f4607400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46080adf20 0x7f46081f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44770/0 (socket says 172.21.15.26:44770) 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f4607400640 1 -- 172.21.15.26:0/4014568465 learned_addr learned my addr 172.21.15.26:0/4014568465 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.771+0000 7f4607e00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f460815bf10 0x7f46080ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f45ffe00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46080cc980 0x7f4608157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f4607400640 1 -- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46080cc980 msgr2=0x7f4608157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f4607400640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46080cc980 0x7f4608157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f4607400640 1 -- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f460815bf10 msgr2=0x7f46080ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f4607400640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f460815bf10 0x7f46080ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f4607400640 1 -- 172.21.15.26:0/4014568465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0009400 con 0x7f46080adf20 2024-01-15T17:55:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f4607e00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f460815bf10 0x7f46080ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:19.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f4607400640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46080adf20 0x7f46081f5b40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d5f0 tx=0x7f45f800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:19.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f45ff400640 1 -- 172.21.15.26:0/4014568465 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80125f0 con 0x7f46080adf20 2024-01-15T17:55:19.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f45ff400640 1 -- 172.21.15.26:0/4014568465 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f8012750 con 0x7f46080adf20 2024-01-15T17:55:19.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f45ff400640 1 -- 172.21.15.26:0/4014568465 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8012b00 con 0x7f46080adf20 2024-01-15T17:55:19.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46081f2e60 con 0x7f46080adf20 2024-01-15T17:55:19.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.772+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46081f3420 con 0x7f46080adf20 2024-01-15T17:55:19.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.773+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4608067450 con 0x7f46080adf20 2024-01-15T17:55:19.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.774+0000 7f45ff400640 1 -- 172.21.15.26:0/4014568465 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f45f8012c60 con 0x7f46080adf20 2024-01-15T17:55:19.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.775+0000 7f45ff400640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45e006b2c0 0x7f45e006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:19.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.775+0000 7f45ff400640 1 -- 172.21.15.26:0/4014568465 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f808a140 con 0x7f46080adf20 2024-01-15T17:55:19.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.776+0000 7f45ffe00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45e006b2c0 0x7f45e006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:19.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.777+0000 7f45ffe00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45e006b2c0 0x7f45e006d780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f45f0036000 tx=0x7f45f003d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.779+0000 7f45ff400640 1 -- 172.21.15.26:0/4014568465 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45f8054f10 con 0x7f46080adf20 2024-01-15T17:55:19.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.975+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4608158690 con 0x7f46080adf20 2024-01-15T17:55:19.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.975+0000 7f45ff400640 1 -- 172.21.15.26:0/4014568465 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45f8058530 con 0x7f46080adf20 2024-01-15T17:55:19.979 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:19.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45e006b2c0 msgr2=0x7f45e006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45e006b2c0 0x7f45e006d780 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f45f0036000 tx=0x7f45f003d040 comp rx=0 tx=0).stop 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46080adf20 msgr2=0x7f46081f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46080adf20 0x7f46081f5b40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45f800d5f0 tx=0x7f45f800dab0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 shutdown_connections 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46080cc980 0x7f4608157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45e006b2c0 0x7f45e006d780 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f460815bf10 0x7f46080ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 --2- 172.21.15.26:0/4014568465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46080adf20 0x7f46081f5b40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 >> 172.21.15.26:0/4014568465 conn(0x7f460806a7d0 msgr2=0x7f460815a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 shutdown_connections 2024-01-15T17:55:19.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:19.978+0000 7f460ce00640 1 -- 172.21.15.26:0/4014568465 wait complete. 2024-01-15T17:55:19.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:55:20.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.216+0000 7f9f49400640 1 Processor -- start 2024-01-15T17:55:20.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.217+0000 7f9f49400640 1 -- start start 2024-01-15T17:55:20.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.218+0000 7f9f49400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9f440cc990 0x7f9f4406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.218+0000 7f9f49400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f44070e50 con 0x7f9f44143c40 2024-01-15T17:55:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.218+0000 7f9f49400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f44144020 con 0x7f9f440cc500 2024-01-15T17:55:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.218+0000 7f9f49400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f440723c0 con 0x7f9f440cc990 2024-01-15T17:55:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9f44143c40 0x7f9f44071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43076/0 (socket says 172.21.15.26:43076) 2024-01-15T17:55:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43e00640 1 -- 172.21.15.26:0/718210985 learned_addr learned my addr 172.21.15.26:0/718210985 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714062432 0 0) 0x7f9f44070e50 con 0x7f9f44143c40 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f24003520 con 0x7f9f44143c40 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f48a00640 1 --2- 172.21.15.26:0/718210985 >> v2:172.21.15.124:3300/0 conn(0x7f9f440cc990 0x7f9f4406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574599484 0 0) 0x7f9f24003520 con 0x7f9f44143c40 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 >> v2:172.21.15.124:3300/0 conn(0x7f9f440cc990 msgr2=0x7f9f4406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43400640 1 --2- 172.21.15.26:0/718210985 >> v2:172.21.15.124:3300/0 conn(0x7f9f440cc990 0x7f9f4406d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 >> v1:172.21.15.124:6789/0 conn(0x7f9f440cc500 legacy=0x7f9f4406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44071e00 con 0x7f9f44143c40 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.219+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9f34001670 con 0x7f9f44143c40 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 >> v1:172.21.15.26:6789/0 conn(0x7f9f44143c40 legacy=0x7f9f44071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43400640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f24003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43400640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f240047c0 0x7f9f24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43400640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f240051d0 0x7f9f2400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440723c0 con 0x7f9f240039c0 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2400c120 con 0x7f9f240047c0 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2400c2e0 con 0x7f9f240051d0 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43e00640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f24003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43e00640 1 -- 172.21.15.26:0/718210985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f240051d0 msgr2=0x7f9f2400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43e00640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f240051d0 0x7f9f2400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f3be00640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f240047c0 0x7f9f24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43e00640 1 -- 172.21.15.26:0/718210985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f240047c0 msgr2=0x7f9f24004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43e00640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f240047c0 0x7f9f24004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f43e00640 1 -- 172.21.15.26:0/718210985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f24003520 con 0x7f9f240039c0 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f48a00640 1 -- 172.21.15.26:0/718210985 reap_dead start 2024-01-15T17:55:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.220+0000 7f9f3be00640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f240047c0 0x7f9f24004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.221+0000 7f9f43e00640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f24003de0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f34001eb0 tx=0x7f9f240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555ca91d9754592f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.222+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34005ad0 con 0x7f9f240039c0 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.222+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f34005c30 con 0x7f9f240039c0 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.222+0000 7f9f43400640 1 -- 172.21.15.26:0/718210985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3402c9d0 con 0x7f9f240039c0 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.222+0000 7f9f49400640 1 -- 172.21.15.26:0/718210985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 msgr2=0x7f9f24003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.222+0000 7f9f49400640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f24003de0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f34001eb0 tx=0x7f9f240057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.222+0000 7f9f49400640 1 -- 172.21.15.26:0/718210985 shutdown_connections 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.222+0000 7f9f49400640 1 --2- 172.21.15.26:0/718210985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f24003de0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.222+0000 7f9f49400640 1 -- 172.21.15.26:0/718210985 >> 172.21.15.26:0/718210985 conn(0x7f9f4406a800 msgr2=0x7f9f4406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.223+0000 7f9f49400640 1 -- 172.21.15.26:0/718210985 shutdown_connections 2024-01-15T17:55:20.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.223+0000 7f9f49400640 1 -- 172.21.15.26:0/718210985 wait complete. 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.224+0000 7f9f49400640 1 Processor -- start 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.224+0000 7f9f49400640 1 -- start start 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.225+0000 7f9f49400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f4415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.225+0000 7f9f49400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4415a3f0 0x7f9f44159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.225+0000 7f9f49400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4413b070 0x7f9f4413b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.225+0000 7f9f49400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4413ba30 con 0x7f9f240039c0 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.225+0000 7f9f49400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4415f430 con 0x7f9f4415a3f0 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.225+0000 7f9f49400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4415e820 con 0x7f9f4413b070 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f43e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f4415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f43e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f4415e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44798/0 (socket says 172.21.15.26:44798) 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f43e00640 1 -- 172.21.15.26:0/887509115 learned_addr learned my addr 172.21.15.26:0/887509115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:20.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f3be00640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4415a3f0 0x7f9f44159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f48a00640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4413b070 0x7f9f4413b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f43e00640 1 -- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4413b070 msgr2=0x7f9f4413b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f43e00640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4413b070 0x7f9f4413b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f43e00640 1 -- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4415a3f0 msgr2=0x7f9f44159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f43e00640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4415a3f0 0x7f9f44159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f43e00640 1 -- 172.21.15.26:0/887509115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4415eb00 con 0x7f9f240039c0 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f48a00640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4413b070 0x7f9f4413b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.226+0000 7f9f43e00640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f4415e1d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f3402c400 tx=0x7f9f34001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.227+0000 7f9f40c00640 1 -- 172.21.15.26:0/887509115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34043070 con 0x7f9f240039c0 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.227+0000 7f9f40c00640 1 -- 172.21.15.26:0/887509115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3403a070 con 0x7f9f240039c0 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.227+0000 7f9f40c00640 1 -- 172.21.15.26:0/887509115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3402c9b0 con 0x7f9f240039c0 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.227+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4415ede0 con 0x7f9f240039c0 2024-01-15T17:55:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.227+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4415f8c0 con 0x7f9f240039c0 2024-01-15T17:55:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.228+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f44067450 con 0x7f9f240039c0 2024-01-15T17:55:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.229+0000 7f9f40c00640 1 -- 172.21.15.26:0/887509115 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9f34032070 con 0x7f9f240039c0 2024-01-15T17:55:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.230+0000 7f9f40c00640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f1c06b2c0 0x7f9f1c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.230+0000 7f9f40c00640 1 -- 172.21.15.26:0/887509115 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f340ab230 con 0x7f9f240039c0 2024-01-15T17:55:20.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.230+0000 7f9f3be00640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f1c06b2c0 0x7f9f1c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.231+0000 7f9f3be00640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f1c06b2c0 0x7f9f1c06d780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f440b0370 tx=0x7f9f2c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.235+0000 7f9f40c00640 1 -- 172.21.15.26:0/887509115 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f34076090 con 0x7f9f240039c0 2024-01-15T17:55:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.457+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f440ceb30 con 0x7f9f240039c0 2024-01-15T17:55:20.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.459+0000 7f9f40c00640 1 -- 172.21.15.26:0/887509115 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f9f340796b0 con 0x7f9f240039c0 2024-01-15T17:55:20.462 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:20.462 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/593578542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":593578542},{"type":"v1","addr":"172.21.15.124:6837","nonce":593578542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1776325954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1776325954},{"type":"v1","addr":"172.21.15.26:6835","nonce":1776325954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:55:04.195734+0000","modified":"2024-01-15T17:55:07.244782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/3022017251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3022017251},{"type":"v1","addr":"172.21.15.124:6835","nonce":3022017251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T17:55:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-01-15T17:55:20.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f1c06b2c0 msgr2=0x7f9f1c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f1c06b2c0 0x7f9f1c06d780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f440b0370 tx=0x7f9f2c009040 comp rx=0 tx=0).stop 2024-01-15T17:55:20.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 msgr2=0x7f9f4415e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f4415e1d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f3402c400 tx=0x7f9f34001930 comp rx=0 tx=0).stop 2024-01-15T17:55:20.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 shutdown_connections 2024-01-15T17:55:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f4413b070 0x7f9f4413b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f1c06b2c0 0x7f9f1c06d780 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f4415a3f0 0x7f9f44159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 --2- 172.21.15.26:0/887509115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f240039c0 0x7f9f4415e1d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 >> 172.21.15.26:0/887509115 conn(0x7f9f4406a800 msgr2=0x7f9f4413ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 shutdown_connections 2024-01-15T17:55:20.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.461+0000 7f9f49400640 1 -- 172.21.15.26:0/887509115 wait complete. 2024-01-15T17:55:20.483 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.690+0000 7f8052000640 1 Processor -- start 2024-01-15T17:55:20.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.690+0000 7f8052000640 1 -- start start 2024-01-15T17:55:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.692+0000 7f8052000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f804c071410 0x7f804c146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.692+0000 7f8052000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f804c071f00 0x7f804c166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.692+0000 7f8052000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804c0b2e10 con 0x7f804c06c7c0 2024-01-15T17:55:20.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.692+0000 7f8052000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c06cc50 con 0x7f804c071f00 2024-01-15T17:55:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.692+0000 7f8052000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c06cec0 con 0x7f804c071410 2024-01-15T17:55:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.692+0000 7f8051600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f804c071f00 0x7f804c166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.692+0000 7f8051600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f804c071f00 0x7f804c166cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44808/0 (socket says 172.21.15.26:44808) 2024-01-15T17:55:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.692+0000 7f8051600640 1 -- 172.21.15.26:0/3679018938 learned_addr learned my addr 172.21.15.26:0/3679018938 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:20.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.692+0000 7f8050c00640 1 --2- 172.21.15.26:0/3679018938 >> v2:172.21.15.124:3300/0 conn(0x7f804c071410 0x7f804c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.693+0000 7f8051600640 1 -- 172.21.15.26:0/3679018938 >> v2:172.21.15.124:3300/0 conn(0x7f804c071410 msgr2=0x7f804c146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.693+0000 7f8051600640 1 --2- 172.21.15.26:0/3679018938 >> v2:172.21.15.124:3300/0 conn(0x7f804c071410 0x7f804c146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.693+0000 7f8051600640 1 -- 172.21.15.26:0/3679018938 >> v1:172.21.15.124:6789/0 conn(0x7f804c06c7c0 legacy=0x7f804c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.693+0000 7f8051600640 1 -- 172.21.15.26:0/3679018938 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c06d1a0 con 0x7f804c071f00 2024-01-15T17:55:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.693+0000 7f8050c00640 1 --2- 172.21.15.26:0/3679018938 >> v2:172.21.15.124:3300/0 conn(0x7f804c071410 0x7f804c146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:20.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.694+0000 7f8051600640 1 --2- 172.21.15.26:0/3679018938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c071f00 0x7f804c166cf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f804c0ad5f0 tx=0x7f804402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22c13f5cef80f2f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.694+0000 7f804b400640 1 -- 172.21.15.26:0/3679018938 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804402cb20 con 0x7f804c071f00 2024-01-15T17:55:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.694+0000 7f804b400640 1 -- 172.21.15.26:0/3679018938 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804402cc80 con 0x7f804c071f00 2024-01-15T17:55:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.694+0000 7f804b400640 1 -- 172.21.15.26:0/3679018938 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80440343f0 con 0x7f804c071f00 2024-01-15T17:55:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.695+0000 7f8052000640 1 -- 172.21.15.26:0/3679018938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c071f00 msgr2=0x7f804c166cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.695+0000 7f8052000640 1 --2- 172.21.15.26:0/3679018938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c071f00 0x7f804c166cf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f804c0ad5f0 tx=0x7f804402bf80 comp rx=0 tx=0).stop 2024-01-15T17:55:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.695+0000 7f8052000640 1 -- 172.21.15.26:0/3679018938 shutdown_connections 2024-01-15T17:55:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.695+0000 7f8052000640 1 --2- 172.21.15.26:0/3679018938 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c071f00 0x7f804c166cf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.695+0000 7f8052000640 1 --2- 172.21.15.26:0/3679018938 >> v2:172.21.15.124:3300/0 conn(0x7f804c071410 0x7f804c146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.695+0000 7f8052000640 1 -- 172.21.15.26:0/3679018938 >> 172.21.15.26:0/3679018938 conn(0x7f804c06a800 msgr2=0x7f804c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.695+0000 7f8052000640 1 -- 172.21.15.26:0/3679018938 shutdown_connections 2024-01-15T17:55:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.695+0000 7f8052000640 1 -- 172.21.15.26:0/3679018938 wait complete. 2024-01-15T17:55:20.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.697+0000 7f8052000640 1 Processor -- start 2024-01-15T17:55:20.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.697+0000 7f8052000640 1 -- start start 2024-01-15T17:55:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8052000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c06c7c0 0x7f804c1396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8052000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f804c071410 0x7f804c13bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8052000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f804c071f00 0x7f804c15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8052000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c139c20 con 0x7f804c06c7c0 2024-01-15T17:55:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8052000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c13d0a0 con 0x7f804c071f00 2024-01-15T17:55:20.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8052000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c13c800 con 0x7f804c071410 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8050c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c06c7c0 0x7f804c1396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8050c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c06c7c0 0x7f804c1396e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44816/0 (socket says 172.21.15.26:44816) 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8050c00640 1 -- 172.21.15.26:0/3693417877 learned_addr learned my addr 172.21.15.26:0/3693417877 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8050c00640 1 -- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f804c071410 msgr2=0x7f804c13bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f804be00640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f804c071410 0x7f804c13bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8051600640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f804c071f00 0x7f804c15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8050c00640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f804c071410 0x7f804c13bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8050c00640 1 -- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f804c071f00 msgr2=0x7f804c15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.698+0000 7f8050c00640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f804c071f00 0x7f804c15dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.699+0000 7f8050c00640 1 -- 172.21.15.26:0/3693417877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8044009b30 con 0x7f804c06c7c0 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.699+0000 7f8050c00640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c06c7c0 0x7f804c1396e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f803c00e760 tx=0x7f803c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.699+0000 7f8048c00640 1 -- 172.21.15.26:0/3693417877 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c004230 con 0x7f804c06c7c0 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.699+0000 7f8048c00640 1 -- 172.21.15.26:0/3693417877 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c004390 con 0x7f804c06c7c0 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.699+0000 7f8048c00640 1 -- 172.21.15.26:0/3693417877 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c00c8f0 con 0x7f804c06c7c0 2024-01-15T17:55:20.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.699+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c13cae0 con 0x7f804c06c7c0 2024-01-15T17:55:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.699+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c15d870 con 0x7f804c06c7c0 2024-01-15T17:55:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.701+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c15da30 con 0x7f804c06c7c0 2024-01-15T17:55:20.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.701+0000 7f8048c00640 1 -- 172.21.15.26:0/3693417877 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f803c00ca50 con 0x7f804c06c7c0 2024-01-15T17:55:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.702+0000 7f8048c00640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f802406b2c0 0x7f802406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:20.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.702+0000 7f8048c00640 1 -- 172.21.15.26:0/3693417877 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f803c089800 con 0x7f804c06c7c0 2024-01-15T17:55:20.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.703+0000 7f804be00640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f802406b2c0 0x7f802406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:20.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.704+0000 7f804be00640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f802406b2c0 0x7f802406d780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f804c0b0910 tx=0x7f8034009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:20.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.707+0000 7f8048c00640 1 -- 172.21.15.26:0/3693417877 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f803c054660 con 0x7f804c06c7c0 2024-01-15T17:55:20.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.911+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f804c067450 con 0x7f804c06c7c0 2024-01-15T17:55:20.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.913+0000 7f8048c00640 1 -- 172.21.15.26:0/3693417877 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f803c057c80 con 0x7f804c06c7c0 2024-01-15T17:55:20.916 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:20.917 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f802406b2c0 msgr2=0x7f802406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f802406b2c0 0x7f802406d780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f804c0b0910 tx=0x7f8034009040 comp rx=0 tx=0).stop 2024-01-15T17:55:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c06c7c0 msgr2=0x7f804c1396e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:20.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c06c7c0 0x7f804c1396e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f803c00e760 tx=0x7f803c00ec20 comp rx=0 tx=0).stop 2024-01-15T17:55:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 shutdown_connections 2024-01-15T17:55:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f804c071410 0x7f804c13bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f802406b2c0 0x7f802406d780 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f804c071f00 0x7f804c15dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 --2- 172.21.15.26:0/3693417877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f804c06c7c0 0x7f804c1396e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 >> 172.21.15.26:0/3693417877 conn(0x7f804c06a800 msgr2=0x7f804c144680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 shutdown_connections 2024-01-15T17:55:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:20.916+0000 7f8052000640 1 -- 172.21.15.26:0/3693417877 wait complete. 2024-01-15T17:55:20.939 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:55:21.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.163+0000 7f196fe00640 1 Processor -- start 2024-01-15T17:55:21.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.164+0000 7f196fe00640 1 -- start start 2024-01-15T17:55:21.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.165+0000 7f196fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1970143c40 0x7f1970071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:21.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.165+0000 7f196fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19700ccc10 0x7f197006d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:21.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.165+0000 7f196fe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1970070e50 con 0x7f19700717d0 2024-01-15T17:55:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.165+0000 7f196fe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19700723c0 con 0x7f1970143c40 2024-01-15T17:55:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.165+0000 7f196fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1970144020 con 0x7f19700ccc10 2024-01-15T17:55:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.166+0000 7f196ea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1970143c40 0x7f1970071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.166+0000 7f196ea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1970143c40 0x7f1970071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44826/0 (socket says 172.21.15.26:44826) 2024-01-15T17:55:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.166+0000 7f196ea00640 1 -- 172.21.15.26:0/2860323787 learned_addr learned my addr 172.21.15.26:0/2860323787 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.166+0000 7f196f400640 1 --2- 172.21.15.26:0/2860323787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19700ccc10 0x7f197006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.167+0000 7f196ea00640 1 -- 172.21.15.26:0/2860323787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19700ccc10 msgr2=0x7f197006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.167+0000 7f196ea00640 1 --2- 172.21.15.26:0/2860323787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19700ccc10 0x7f197006d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.167+0000 7f196ea00640 1 -- 172.21.15.26:0/2860323787 >> v1:172.21.15.124:6789/0 conn(0x7f19700717d0 legacy=0x7f19700cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.167+0000 7f196ea00640 1 -- 172.21.15.26:0/2860323787 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970071e00 con 0x7f1970143c40 2024-01-15T17:55:21.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.167+0000 7f196f400640 1 --2- 172.21.15.26:0/2860323787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19700ccc10 0x7f197006d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:21.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.169+0000 7f196ea00640 1 --2- 172.21.15.26:0/2860323787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1970143c40 0x7f1970071290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1960009870 tx=0x7f196002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b728b426c5f6d157 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:21.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.169+0000 7f196e000640 1 -- 172.21.15.26:0/2860323787 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960031640 con 0x7f1970143c40 2024-01-15T17:55:21.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.169+0000 7f196e000640 1 -- 172.21.15.26:0/2860323787 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19600317a0 con 0x7f1970143c40 2024-01-15T17:55:21.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.169+0000 7f196e000640 1 -- 172.21.15.26:0/2860323787 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960031b30 con 0x7f1970143c40 2024-01-15T17:55:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.170+0000 7f196fe00640 1 -- 172.21.15.26:0/2860323787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1970143c40 msgr2=0x7f1970071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.170+0000 7f196fe00640 1 --2- 172.21.15.26:0/2860323787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1970143c40 0x7f1970071290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1960009870 tx=0x7f196002b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.170+0000 7f196fe00640 1 -- 172.21.15.26:0/2860323787 shutdown_connections 2024-01-15T17:55:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.170+0000 7f196fe00640 1 --2- 172.21.15.26:0/2860323787 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19700ccc10 0x7f197006d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.170+0000 7f196fe00640 1 --2- 172.21.15.26:0/2860323787 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1970143c40 0x7f1970071290 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.170+0000 7f196fe00640 1 -- 172.21.15.26:0/2860323787 >> 172.21.15.26:0/2860323787 conn(0x7f197006a800 msgr2=0x7f197006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.170+0000 7f196fe00640 1 -- 172.21.15.26:0/2860323787 shutdown_connections 2024-01-15T17:55:21.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.170+0000 7f196fe00640 1 -- 172.21.15.26:0/2860323787 wait complete. 2024-01-15T17:55:21.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.173+0000 7f196fe00640 1 Processor -- start 2024-01-15T17:55:21.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.173+0000 7f196fe00640 1 -- start start 2024-01-15T17:55:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.174+0000 7f196fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19700717d0 0x7f197013e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.174+0000 7f196fe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19700ccc10 0x7f19700c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.174+0000 7f196fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1970143c40 0x7f19700c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.174+0000 7f196fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700ca500 con 0x7f19700717d0 2024-01-15T17:55:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.174+0000 7f196fe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700ca7e0 con 0x7f19700ccc10 2024-01-15T17:55:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.174+0000 7f196fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19700caac0 con 0x7f1970143c40 2024-01-15T17:55:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.174+0000 7f196ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19700717d0 0x7f197013e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:21.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.174+0000 7f196ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19700717d0 0x7f197013e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44828/0 (socket says 172.21.15.26:44828) 2024-01-15T17:55:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.174+0000 7f196ea00640 1 -- 172.21.15.26:0/1317287986 learned_addr learned my addr 172.21.15.26:0/1317287986 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.175+0000 7f196ea00640 1 -- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1970143c40 msgr2=0x7f19700c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.175+0000 7f196ea00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1970143c40 0x7f19700c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.175+0000 7f196ea00640 1 -- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19700ccc10 msgr2=0x7f19700c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.175+0000 7f1967e00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19700ccc10 0x7f19700c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.175+0000 7f196ea00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19700ccc10 0x7f19700c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.175+0000 7f196ea00640 1 -- 172.21.15.26:0/1317287986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19600094d0 con 0x7f19700717d0 2024-01-15T17:55:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.175+0000 7f1967e00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19700ccc10 0x7f19700c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:21.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.175+0000 7f196ea00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19700717d0 0x7f197013e6a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1960000c00 tx=0x7f19600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:21.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.176+0000 7f1967400640 1 -- 172.21.15.26:0/1317287986 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196003ac60 con 0x7f19700717d0 2024-01-15T17:55:21.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.176+0000 7f1967400640 1 -- 172.21.15.26:0/1317287986 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196003adc0 con 0x7f19700717d0 2024-01-15T17:55:21.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.176+0000 7f1967400640 1 -- 172.21.15.26:0/1317287986 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1960044630 con 0x7f19700717d0 2024-01-15T17:55:21.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.176+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19700cada0 con 0x7f19700717d0 2024-01-15T17:55:21.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.176+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197013a9b0 con 0x7f19700717d0 2024-01-15T17:55:21.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.177+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1970067450 con 0x7f19700717d0 2024-01-15T17:55:21.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.178+0000 7f1967400640 1 -- 172.21.15.26:0/1317287986 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f196002f050 con 0x7f19700717d0 2024-01-15T17:55:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.179+0000 7f1967400640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f194806b2c0 0x7f194806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.179+0000 7f1967400640 1 -- 172.21.15.26:0/1317287986 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19600abb10 con 0x7f19700717d0 2024-01-15T17:55:21.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.179+0000 7f1967e00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f194806b2c0 0x7f194806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:21.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.181+0000 7f1967e00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f194806b2c0 0x7f194806d780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19700cc360 tx=0x7f1958009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.183+0000 7f1967400640 1 -- 172.21.15.26:0/1317287986 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19600768e0 con 0x7f19700717d0 2024-01-15T17:55:21.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.414+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19700ceb30 con 0x7f19700717d0 2024-01-15T17:55:21.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.416+0000 7f1967400640 1 -- 172.21.15.26:0/1317287986 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f1960035090 con 0x7f19700717d0 2024-01-15T17:55:21.419 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:21.419 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/593578542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":593578542},{"type":"v1","addr":"172.21.15.124:6837","nonce":593578542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1776325954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1776325954},{"type":"v1","addr":"172.21.15.26:6835","nonce":1776325954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:55:04.195734+0000","modified":"2024-01-15T17:55:07.244782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/3022017251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3022017251},{"type":"v1","addr":"172.21.15.124:6835","nonce":3022017251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T17:55:21.419 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-01-15T17:55:21.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f194806b2c0 msgr2=0x7f194806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f194806b2c0 0x7f194806d780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19700cc360 tx=0x7f1958009040 comp rx=0 tx=0).stop 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19700717d0 msgr2=0x7f197013e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19700717d0 0x7f197013e6a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1960000c00 tx=0x7f19600046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 shutdown_connections 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1970143c40 0x7f19700c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f194806b2c0 0x7f194806d780 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19700ccc10 0x7f19700c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 --2- 172.21.15.26:0/1317287986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19700717d0 0x7f197013e6a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 >> 172.21.15.26:0/1317287986 conn(0x7f197006a800 msgr2=0x7f19700aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 shutdown_connections 2024-01-15T17:55:21.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:21.418+0000 7f196fe00640 1 -- 172.21.15.26:0/1317287986 wait complete. 2024-01-15T17:55:21.842 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T17:55:04.195734+0000', 'modified': '2024-01-15T17:55:07.244782+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34160}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34160': {'gid': 34160, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6835/3022017251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6834', 'nonce': 3022017251}, {'type': 'v1', 'addr': '172.21.15.124:6835', 'nonce': 3022017251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T17:55:21.843 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T17:55:21.843 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-15T17:55:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.072+0000 7f588e600640 1 Processor -- start 2024-01-15T17:55:22.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.072+0000 7f588e600640 1 -- start start 2024-01-15T17:55:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.073+0000 7f588e600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58880cc500 0x7f58880cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.073+0000 7f588e600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f588806d160 0x7f588806d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.073+0000 7f588e600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5888070e50 con 0x7f5888143c40 2024-01-15T17:55:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.073+0000 7f588e600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58880723c0 con 0x7f588806d160 2024-01-15T17:55:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.073+0000 7f588e600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5888144020 con 0x7f58880cc500 2024-01-15T17:55:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.074+0000 7f588dc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f588806d160 0x7f588806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.074+0000 7f588dc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f588806d160 0x7f588806d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54084/0 (socket says 172.21.15.26:54084) 2024-01-15T17:55:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.074+0000 7f588dc00640 1 -- 172.21.15.26:0/959526574 learned_addr learned my addr 172.21.15.26:0/959526574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.074+0000 7f5887e00640 1 --2- 172.21.15.26:0/959526574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58880cc500 0x7f58880cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.074+0000 7f588dc00640 1 -- 172.21.15.26:0/959526574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58880cc500 msgr2=0x7f58880cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.074+0000 7f588dc00640 1 --2- 172.21.15.26:0/959526574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58880cc500 0x7f58880cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.074+0000 7f588dc00640 1 -- 172.21.15.26:0/959526574 >> v1:172.21.15.124:6789/0 conn(0x7f5888143c40 legacy=0x7f5888071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.074+0000 7f588dc00640 1 -- 172.21.15.26:0/959526574 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58880ccea0 con 0x7f588806d160 2024-01-15T17:55:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.075+0000 7f588dc00640 1 --2- 172.21.15.26:0/959526574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f588806d160 0x7f588806d590 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5880009e50 tx=0x7f588002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aea2998baf9bee6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.076+0000 7f5887400640 1 -- 172.21.15.26:0/959526574 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588002ca70 con 0x7f588806d160 2024-01-15T17:55:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.076+0000 7f5887400640 1 -- 172.21.15.26:0/959526574 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588002cbd0 con 0x7f588806d160 2024-01-15T17:55:22.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.076+0000 7f5887400640 1 -- 172.21.15.26:0/959526574 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58800343f0 con 0x7f588806d160 2024-01-15T17:55:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.076+0000 7f588e600640 1 -- 172.21.15.26:0/959526574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f588806d160 msgr2=0x7f588806d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.076+0000 7f588e600640 1 --2- 172.21.15.26:0/959526574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f588806d160 0x7f588806d590 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5880009e50 tx=0x7f588002bed0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.076+0000 7f588e600640 1 -- 172.21.15.26:0/959526574 shutdown_connections 2024-01-15T17:55:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.076+0000 7f588e600640 1 --2- 172.21.15.26:0/959526574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f588806d160 0x7f588806d590 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.076+0000 7f588e600640 1 --2- 172.21.15.26:0/959526574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58880cc500 0x7f58880cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.076+0000 7f588e600640 1 -- 172.21.15.26:0/959526574 >> 172.21.15.26:0/959526574 conn(0x7f588806a800 msgr2=0x7f588806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.077+0000 7f588e600640 1 -- 172.21.15.26:0/959526574 shutdown_connections 2024-01-15T17:55:22.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.077+0000 7f588e600640 1 -- 172.21.15.26:0/959526574 wait complete. 2024-01-15T17:55:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.078+0000 7f588e600640 1 Processor -- start 2024-01-15T17:55:22.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.078+0000 7f588e600640 1 -- start start 2024-01-15T17:55:22.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.079+0000 7f588e600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f588806d160 0x7f588815a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.079+0000 7f588e600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58880cc500 0x7f588813db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.079+0000 7f588e600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5888143c40 0x7f58881424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.079+0000 7f588e600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58881e6640 con 0x7f58880cc500 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.079+0000 7f588e600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58881e1160 con 0x7f588806d160 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.079+0000 7f588e600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58881e5a30 con 0x7f5888143c40 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f5887e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58880cc500 0x7f588813db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f5887e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58880cc500 0x7f588813db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54086/0 (socket says 172.21.15.26:54086) 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f5887e00640 1 -- 172.21.15.26:0/1745322541 learned_addr learned my addr 172.21.15.26:0/1745322541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f588d200640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f588806d160 0x7f588815a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f5887e00640 1 -- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5888143c40 msgr2=0x7f58881424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f588dc00640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5888143c40 0x7f58881424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f5887e00640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5888143c40 0x7f58881424e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f5887e00640 1 -- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f588806d160 msgr2=0x7f588815a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f5887e00640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f588806d160 0x7f588815a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f5887e00640 1 -- 172.21.15.26:0/1745322541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5880009b30 con 0x7f58880cc500 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f588d200640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f588806d160 0x7f588815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f588dc00640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5888143c40 0x7f58881424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.080+0000 7f5887e00640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58880cc500 0x7f588813db00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f587000d800 tx=0x7f587000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.081+0000 7f5884c00640 1 -- 172.21.15.26:0/1745322541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587000b6c0 con 0x7f58880cc500 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.081+0000 7f5884c00640 1 -- 172.21.15.26:0/1745322541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587000b820 con 0x7f58880cc500 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.081+0000 7f5884c00640 1 -- 172.21.15.26:0/1745322541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587000bbb0 con 0x7f58880cc500 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.081+0000 7f588e600640 1 -- 172.21.15.26:0/1745322541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58881e5d10 con 0x7f58880cc500 2024-01-15T17:55:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.081+0000 7f588e600640 1 -- 172.21.15.26:0/1745322541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58881e62d0 con 0x7f58880cc500 2024-01-15T17:55:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.084+0000 7f5884c00640 1 -- 172.21.15.26:0/1745322541 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f587000bd10 con 0x7f58880cc500 2024-01-15T17:55:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.085+0000 7f5884c00640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f586006b340 0x7f586006d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.085+0000 7f5884c00640 1 -- 172.21.15.26:0/1745322541 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f587008a110 con 0x7f58880cc500 2024-01-15T17:55:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.085+0000 7f588d200640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f586006b340 0x7f586006d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.085+0000 7f58929d2740 1 Processor -- start 2024-01-15T17:55:22.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.086+0000 7f58929d2740 1 -- start start 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.086+0000 7f58929d2740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b29450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.086+0000 7f588d200640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f586006b340 0x7f586006d800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f58880b08a0 tx=0x7f5878009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.086+0000 7f58929d2740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 0x5592d8b28270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.086+0000 7f58929d2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b6c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f58929d2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592d8a87880 con 0x5592d8b287b0 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f58929d2740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592d8a96ae0 con 0x5592d8b29070 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f58929d2740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592d8a941d0 con 0x5592d8b23d70 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b6c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b6c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54090/0 (socket says 172.21.15.26:54090) 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588dc00640 1 -- 172.21.15.26:0/3442804647 learned_addr learned my addr 172.21.15.26:0/3442804647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588d200640 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b29450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f5887e00640 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 0x5592d8b28270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588dc00640 1 -- 172.21.15.26:0/3442804647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 msgr2=0x5592d8b28270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588dc00640 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 0x5592d8b28270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588dc00640 1 -- 172.21.15.26:0/3442804647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 msgr2=0x5592d8b29450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588dc00640 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b29450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588dc00640 1 -- 172.21.15.26:0/3442804647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592d8a57860 con 0x5592d8b287b0 2024-01-15T17:55:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.087+0000 7f588d200640 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b29450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.088+0000 7f588dc00640 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b6c730 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f588002c4e0 tx=0x7f588002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5bbc5799e639a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.088+0000 7f587d600640 1 -- 172.21.15.26:0/3442804647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880040470 con 0x5592d8b287b0 2024-01-15T17:55:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.088+0000 7f587d600640 1 -- 172.21.15.26:0/3442804647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58800405d0 con 0x5592d8b287b0 2024-01-15T17:55:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.088+0000 7f587d600640 1 -- 172.21.15.26:0/3442804647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880040940 con 0x5592d8b287b0 2024-01-15T17:55:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.088+0000 7f58929d2740 1 -- 172.21.15.26:0/3442804647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 msgr2=0x5592d8b6c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.088+0000 7f58929d2740 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b6c730 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f588002c4e0 tx=0x7f588002e000 comp rx=0 tx=0).stop 2024-01-15T17:55:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.089+0000 7f58929d2740 1 -- 172.21.15.26:0/3442804647 shutdown_connections 2024-01-15T17:55:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.089+0000 7f58929d2740 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 0x5592d8b28270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.089+0000 7f58929d2740 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b6c730 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.089+0000 7f58929d2740 1 --2- 172.21.15.26:0/3442804647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b29450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.089+0000 7f58929d2740 1 -- 172.21.15.26:0/3442804647 >> 172.21.15.26:0/3442804647 conn(0x5592d8aeac70 msgr2=0x5592d8b6eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.089+0000 7f58929d2740 1 -- 172.21.15.26:0/3442804647 shutdown_connections 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.089+0000 7f58929d2740 1 -- 172.21.15.26:0/3442804647 wait complete. 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.089+0000 7f58929d2740 1 -- start start 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.090+0000 7f58929d2740 1 Processor -- start 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f58929d2740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 0x5592d8b52730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f58929d2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b52c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f58929d2740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b51470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f58929d2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592d8a8fac0 con 0x5592d8b287b0 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f58929d2740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592d8a2c2f0 con 0x5592d8b29070 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f58929d2740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592d8a49c70 con 0x5592d8b23d70 2024-01-15T17:55:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f58929d2740 10 client.0 _ll_register_callbacks cb 0x5592d8a4e3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f5887e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b52c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f5887e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b52c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54104/0 (socket says 172.21.15.26:54104) 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f5887e00640 1 -- 172.21.15.26:0/2088466771 learned_addr learned my addr 172.21.15.26:0/2088466771 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f588d200640 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 0x5592d8b52730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f588dc00640 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b51470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f5887e00640 1 -- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 msgr2=0x5592d8b52730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f5887e00640 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 0x5592d8b52730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f5887e00640 1 -- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 msgr2=0x5592d8b51470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f5887e00640 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b51470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.091+0000 7f5887e00640 1 -- 172.21.15.26:0/2088466771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58880ccea0 con 0x5592d8b287b0 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f5887e00640 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b52c70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58700609f0 tx=0x7f5870058c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1671b1c6af07d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f5876000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T17:55:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f5876000640 1 -- 172.21.15.26:0/2088466771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870058110 con 0x5592d8b287b0 2024-01-15T17:55:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f5876000640 1 -- 172.21.15.26:0/2088466771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5870058270 con 0x5592d8b287b0 2024-01-15T17:55:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f58929d2740 1 -- 172.21.15.26:0/2088466771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592d8a88aa0 con 0x5592d8b287b0 2024-01-15T17:55:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f5876000640 1 -- 172.21.15.26:0/2088466771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5870088100 con 0x5592d8b287b0 2024-01-15T17:55:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f58929d2740 10 client.4647 fetch_fsmap learned FSMap version 7 2024-01-15T17:55:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f58929d2740 1 -- 172.21.15.26:0/2088466771 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5592d8a8e140 con 0x5592d8b287b0 2024-01-15T17:55:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f5876000640 1 -- 172.21.15.26:0/2088466771 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592d8a88aa0 con 0x5592d8b287b0 2024-01-15T17:55:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.092+0000 7f5876000640 1 -- 172.21.15.26:0/2088466771 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5870060a90 con 0x5592d8b287b0 2024-01-15T17:55:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.093+0000 7f58929d2740 10 client.4647 fetch_fsmap finished waiting for FSMap version 7 2024-01-15T17:55:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.093+0000 7f58929d2740 10 client.4647 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-15T17:55:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.093+0000 7f58929d2740 20 client.4647 populate_metadata read hostname 'smithi026' 2024-01-15T17:55:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.093+0000 7f58929d2740 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] conn(0x5592d8b51fd0 0x5592d8b73510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.093+0000 7f58929d2740 4 client.4647 mds_command: new command op to 34160 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T17:55:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.093+0000 7f58929d2740 1 -- 172.21.15.26:0/2088466771 --> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592d8a887c0 con 0x5592d8b51fd0 2024-01-15T17:55:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.093+0000 7f588d200640 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] conn(0x5592d8b51fd0 0x5592d8b73510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.094+0000 7f588d200640 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] conn(0x5592d8b51fd0 0x5592d8b73510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2514c152d6edc2ae server_cookie=53ee05af0fb78e46 in_seq=0 out_seq=0 2024-01-15T17:55:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.094+0000 7f5876000640 10 client.4647 ms_handle_connect on v2:172.21.15.124:6834/3022017251 2024-01-15T17:55:22.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.096+0000 7f5876000640 1 -- 172.21.15.26:0/2088466771 <== mds.0 v2:172.21.15.124:6834/3022017251 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592d8a887c0 con 0x5592d8b51fd0 2024-01-15T17:55:22.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.096+0000 7f5876000640 10 client.4647 handle_command_reply: tid=0 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.096+0000 7f58929d2740 1 client.4647 shutdown 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.096+0000 7f58929d2740 1 -- 172.21.15.26:0/2088466771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 msgr2=0x5592d8b52c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.096+0000 7f58929d2740 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b52c70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58700609f0 tx=0x7f5870058c20 comp rx=0 tx=0).stop 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f58929d2740 1 -- 172.21.15.26:0/2088466771 shutdown_connections 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f58929d2740 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] conn(0x5592d8b51fd0 0x5592d8b73510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f58929d2740 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b51470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f58929d2740 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b287b0 0x5592d8b52c70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f58929d2740 1 --2- 172.21.15.26:0/2088466771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b23d70 0x5592d8b52730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f58929d2740 1 -- 172.21.15.26:0/2088466771 >> 172.21.15.26:0/2088466771 conn(0x5592d8aeac70 msgr2=0x5592d8b6bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f5876000640 0 client.4647 ms_handle_reset on v2:172.21.15.124:6834/3022017251 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f58929d2740 1 -- 172.21.15.26:0/2088466771 shutdown_connections 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f58929d2740 1 -- 172.21.15.26:0/2088466771 wait complete. 2024-01-15T17:55:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.097+0000 7f58929d2740 20 client.4647 trim_cache size 0 max 16384 2024-01-15T17:55:22.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.114+0000 7f58929d2740 1 Processor -- start 2024-01-15T17:55:22.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.114+0000 7f58929d2740 1 -- start start 2024-01-15T17:55:22.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.115+0000 7f58929d2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b23d70 0x5592d8b6aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.115+0000 7f58929d2740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b287b0 0x5592d8b29450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.115+0000 7f58929d2740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.115+0000 7f58929d2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592d87f2f20 con 0x5592d8b23d70 2024-01-15T17:55:22.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.115+0000 7f58929d2740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592d8b4e8e0 con 0x5592d8b29070 2024-01-15T17:55:22.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.115+0000 7f58929d2740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592d8b4e680 con 0x5592d8b287b0 2024-01-15T17:55:22.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b23d70 0x5592d8b6aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b23d70 0x5592d8b6aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54106/0 (socket says 172.21.15.26:54106) 2024-01-15T17:55:22.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588d200640 1 -- 172.21.15.26:0/2422919283 learned_addr learned my addr 172.21.15.26:0/2422919283 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:22.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588dc00640 1 --2- 172.21.15.26:0/2422919283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588d200640 1 -- 172.21.15.26:0/2422919283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b287b0 msgr2=0x5592d8b29450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588d200640 1 --2- 172.21.15.26:0/2422919283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b287b0 0x5592d8b29450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588d200640 1 -- 172.21.15.26:0/2422919283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 msgr2=0x5592d8b736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588d200640 1 --2- 172.21.15.26:0/2422919283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588d200640 1 -- 172.21.15.26:0/2422919283 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592d8a2bb80 con 0x5592d8b23d70 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588dc00640 1 --2- 172.21.15.26:0/2422919283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.116+0000 7f588d200640 1 --2- 172.21.15.26:0/2422919283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b23d70 0x5592d8b6aa50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f58780053a0 tx=0x7f5878013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ea43a9e05aa805 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f587e000640 1 -- 172.21.15.26:0/2422919283 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878003b90 con 0x5592d8b23d70 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f587e000640 1 -- 172.21.15.26:0/2422919283 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587801d070 con 0x5592d8b23d70 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f587e000640 1 -- 172.21.15.26:0/2422919283 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878007030 con 0x5592d8b23d70 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f58929d2740 1 -- 172.21.15.26:0/2422919283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b23d70 msgr2=0x5592d8b6aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f58929d2740 1 --2- 172.21.15.26:0/2422919283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b23d70 0x5592d8b6aa50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f58780053a0 tx=0x7f5878013000 comp rx=0 tx=0).stop 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f58929d2740 1 -- 172.21.15.26:0/2422919283 shutdown_connections 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f58929d2740 1 --2- 172.21.15.26:0/2422919283 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b287b0 0x5592d8b29450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f58929d2740 1 --2- 172.21.15.26:0/2422919283 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b29070 0x5592d8b736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f58929d2740 1 --2- 172.21.15.26:0/2422919283 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b23d70 0x5592d8b6aa50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f58929d2740 1 -- 172.21.15.26:0/2422919283 >> 172.21.15.26:0/2422919283 conn(0x5592d8aeac70 msgr2=0x5592d8b29f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:22.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f58929d2740 1 -- 172.21.15.26:0/2422919283 shutdown_connections 2024-01-15T17:55:22.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.117+0000 7f58929d2740 1 -- 172.21.15.26:0/2422919283 wait complete. 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.118+0000 7f58929d2740 1 -- start start 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.118+0000 7f58929d2740 1 Processor -- start 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.119+0000 7f58929d2740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b23d70 0x5592d8bd8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.119+0000 7f58929d2740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b287b0 0x5592d8b736f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.119+0000 7f58929d2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b29070 0x5592d8b73c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.119+0000 7f58929d2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592d8b57b70 con 0x5592d8b29070 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.119+0000 7f58929d2740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592d8bd93a0 con 0x5592d8b23d70 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.119+0000 7f58929d2740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592d8b57920 con 0x5592d8b287b0 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.119+0000 7f58929d2740 10 client.0 _ll_register_callbacks cb 0x7f5870004660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b29070 0x5592d8b73c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b29070 0x5592d8b73c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54116/0 (socket says 172.21.15.26:54116) 2024-01-15T17:55:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588dc00640 1 -- 172.21.15.26:0/417764253 learned_addr learned my addr 172.21.15.26:0/417764253 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588d200640 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b23d70 0x5592d8bd8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f5887e00640 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b287b0 0x5592d8b736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588dc00640 1 -- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b287b0 msgr2=0x5592d8b736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588dc00640 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b287b0 0x5592d8b736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588dc00640 1 -- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b23d70 msgr2=0x5592d8bd8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588dc00640 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b23d70 0x5592d8bd8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588dc00640 1 -- 172.21.15.26:0/417764253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5888070e50 con 0x5592d8b29070 2024-01-15T17:55:22.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f588dc00640 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b29070 0x5592d8b73c30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5880043040 tx=0x7f5880004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b4f83bbf589bf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.120+0000 7f5876000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f5876000640 1 -- 172.21.15.26:0/417764253 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880040470 con 0x5592d8b29070 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f5876000640 1 -- 172.21.15.26:0/417764253 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f588004caf0 con 0x5592d8b29070 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f58929d2740 1 -- 172.21.15.26:0/417764253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58800405d0 con 0x5592d8b29070 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f5876000640 1 -- 172.21.15.26:0/417764253 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f588002c550 con 0x5592d8b29070 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f58929d2740 10 client.4653 fetch_fsmap learned FSMap version 7 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f58929d2740 1 -- 172.21.15.26:0/417764253 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5592d8a51170 con 0x5592d8b29070 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f5876000640 1 -- 172.21.15.26:0/417764253 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58800405d0 con 0x5592d8b29070 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f5876000640 1 -- 172.21.15.26:0/417764253 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f588003c040 con 0x5592d8b29070 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f58929d2740 10 client.4653 fetch_fsmap finished waiting for FSMap version 7 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f58929d2740 10 client.4653 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f58929d2740 20 client.4653 populate_metadata read hostname 'smithi026' 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f58929d2740 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] conn(0x5592d8bdc650 0x5592d8bdca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f58929d2740 4 client.4653 mds_command: new command op to 34160 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T17:55:22.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.121+0000 7f58929d2740 1 -- 172.21.15.26:0/417764253 --> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5592d87906c0 con 0x5592d8bdc650 2024-01-15T17:55:22.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.122+0000 7f588d200640 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] conn(0x5592d8bdc650 0x5592d8bdca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.123+0000 7f588d200640 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] conn(0x5592d8bdc650 0x5592d8bdca30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea173f3010119621 server_cookie=4187b2ee61cacc9c in_seq=0 out_seq=0 2024-01-15T17:55:22.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.123+0000 7f5876000640 10 client.4653 ms_handle_connect on v2:172.21.15.124:6834/3022017251 2024-01-15T17:55:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.123+0000 7f5876000640 1 -- 172.21.15.26:0/417764253 <== mds.0 v2:172.21.15.124:6834/3022017251 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5592d87906c0 con 0x5592d8bdc650 2024-01-15T17:55:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.123+0000 7f5876000640 10 client.4653 handle_command_reply: tid=0 2024-01-15T17:55:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.123+0000 7f58929d2740 1 client.4653 shutdown 2024-01-15T17:55:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.123+0000 7f58929d2740 1 -- 172.21.15.26:0/417764253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b29070 msgr2=0x5592d8b73c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.123+0000 7f58929d2740 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b29070 0x5592d8b73c30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5880043040 tx=0x7f5880004ab0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f58929d2740 1 -- 172.21.15.26:0/417764253 shutdown_connections 2024-01-15T17:55:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f58929d2740 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:6834/3022017251,v1:172.21.15.124:6835/3022017251] conn(0x5592d8bdc650 0x5592d8bdca30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f58929d2740 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5592d8b287b0 0x5592d8b736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f58929d2740 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5592d8b29070 0x5592d8b73c30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f58929d2740 1 --2- 172.21.15.26:0/417764253 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5592d8b23d70 0x5592d8bd8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f58929d2740 1 -- 172.21.15.26:0/417764253 >> 172.21.15.26:0/417764253 conn(0x5592d8aeac70 msgr2=0x5592d8b29450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f5876000640 0 client.4653 ms_handle_reset on v2:172.21.15.124:6834/3022017251 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f58929d2740 1 -- 172.21.15.26:0/417764253 shutdown_connections 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f58929d2740 1 -- 172.21.15.26:0/417764253 wait complete. 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f58929d2740 20 client.4653 trim_cache size 0 max 16384 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 34160, 2024-01-15T17:55:22.129 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-15T17:55:22.130 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-15T17:55:22.130 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-15T17:55:22.130 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 17.405591926, 2024-01-15T17:55:22.130 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 7, 2024-01-15T17:55:22.130 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 22, 2024-01-15T17:55:22.130 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 22, 2024-01-15T17:55:22.130 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 24.831194325999999 2024-01-15T17:55:22.130 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f588e600640 1 -- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f586006b340 msgr2=0x7f586006d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f588e600640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f586006b340 0x7f586006d800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f58880b08a0 tx=0x7f5878009040 comp rx=0 tx=0).stop 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f588e600640 1 -- 172.21.15.26:0/1745322541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58880cc500 msgr2=0x7f588813db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.124+0000 7f588e600640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58880cc500 0x7f588813db00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f587000d800 tx=0x7f587000dcc0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.125+0000 7f588e600640 1 -- 172.21.15.26:0/1745322541 shutdown_connections 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.125+0000 7f588e600640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5888143c40 0x7f58881424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.125+0000 7f588e600640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f586006b340 0x7f586006d800 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.125+0000 7f588e600640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58880cc500 0x7f588813db00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.125+0000 7f588e600640 1 --2- 172.21.15.26:0/1745322541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f588806d160 0x7f588815a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.125+0000 7f588e600640 1 -- 172.21.15.26:0/1745322541 >> 172.21.15.26:0/1745322541 conn(0x7f588806a800 msgr2=0x7f588806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.125+0000 7f588e600640 1 -- 172.21.15.26:0/1745322541 shutdown_connections 2024-01-15T17:55:22.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.125+0000 7f588e600640 1 -- 172.21.15.26:0/1745322541 wait complete. 2024-01-15T17:55:22.144 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-01-15T17:55:22.167 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-01-15T17:55:22.167 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-01-15T17:55:22.167 INFO:tasks.ceph_fuse:client.0 config is {} 2024-01-15T17:55:22.167 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-15T17:55:22.167 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:22.167 INFO:tasks.ceph_fuse:client.2 config is {} 2024-01-15T17:55:22.167 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-15T17:55:22.167 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-15T17:55:22.167 INFO:tasks.ceph_fuse:client.1 config is {} 2024-01-15T17:55:22.167 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-15T17:55:22.168 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:22.168 INFO:tasks.ceph_fuse:client.3 config is {} 2024-01-15T17:55:22.168 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-15T17:55:22.168 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-15T17:55:22.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:22.168 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T17:55:22.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:22.228 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2024-01-15T17:55:22.327 INFO:teuthology.orchestra.run.smithi124.stderr:Cannot find device "ceph-brx" 2024-01-15T17:55:22.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:55:22.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:22.333 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T17:55:22.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:22.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-01-15T17:55:22.499 INFO:teuthology.orchestra.run.smithi026.stderr:Cannot find device "ceph-brx" 2024-01-15T17:55:22.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:55:22.504 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-01-15T17:55:22.504 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T17:55:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.726+0000 7fd798000640 1 Processor -- start 2024-01-15T17:55:22.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.726+0000 7fd798000640 1 -- start start 2024-01-15T17:55:22.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.728+0000 7fd798000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd79013df80 0x7fd79015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.728+0000 7fd798000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7900cf9a0 0x7fd7900cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.728+0000 7fd798000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79006c470 con 0x7fd79013e360 2024-01-15T17:55:22.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.728+0000 7fd798000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79006c750 con 0x7fd7900cf9a0 2024-01-15T17:55:22.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.728+0000 7fd798000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79006ca30 con 0x7fd79013df80 2024-01-15T17:55:22.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.728+0000 7fd796000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7900cf9a0 0x7fd7900cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.728+0000 7fd796000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7900cf9a0 0x7fd7900cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54130/0 (socket says 172.21.15.26:54130) 2024-01-15T17:55:22.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.728+0000 7fd796000640 1 -- 172.21.15.26:0/888850202 learned_addr learned my addr 172.21.15.26:0/888850202 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:22.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.729+0000 7fd795600640 1 --2- 172.21.15.26:0/888850202 >> v2:172.21.15.124:3300/0 conn(0x7fd79013df80 0x7fd79015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.729+0000 7fd796000640 1 -- 172.21.15.26:0/888850202 >> v2:172.21.15.124:3300/0 conn(0x7fd79013df80 msgr2=0x7fd79015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.729+0000 7fd796000640 1 --2- 172.21.15.26:0/888850202 >> v2:172.21.15.124:3300/0 conn(0x7fd79013df80 0x7fd79015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.729+0000 7fd796000640 1 -- 172.21.15.26:0/888850202 >> v1:172.21.15.124:6789/0 conn(0x7fd79013e360 legacy=0x7fd7900cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:22.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.729+0000 7fd796000640 1 -- 172.21.15.26:0/888850202 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79006cd10 con 0x7fd7900cf9a0 2024-01-15T17:55:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.730+0000 7fd796000640 1 --2- 172.21.15.26:0/888850202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7900cf9a0 0x7fd7900cd840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd790072770 tx=0x7fd78802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6095ca4091cd2376 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.730+0000 7fd78fe00640 1 -- 172.21.15.26:0/888850202 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78802ca80 con 0x7fd7900cf9a0 2024-01-15T17:55:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd78fe00640 1 -- 172.21.15.26:0/888850202 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78802cbe0 con 0x7fd7900cf9a0 2024-01-15T17:55:22.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd78fe00640 1 -- 172.21.15.26:0/888850202 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7880343f0 con 0x7fd7900cf9a0 2024-01-15T17:55:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd798000640 1 -- 172.21.15.26:0/888850202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7900cf9a0 msgr2=0x7fd7900cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd798000640 1 --2- 172.21.15.26:0/888850202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7900cf9a0 0x7fd7900cd840 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd790072770 tx=0x7fd78802bee0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd798000640 1 -- 172.21.15.26:0/888850202 shutdown_connections 2024-01-15T17:55:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd798000640 1 --2- 172.21.15.26:0/888850202 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7900cf9a0 0x7fd7900cd840 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd798000640 1 --2- 172.21.15.26:0/888850202 >> v2:172.21.15.124:3300/0 conn(0x7fd79013df80 0x7fd79015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd798000640 1 -- 172.21.15.26:0/888850202 >> 172.21.15.26:0/888850202 conn(0x7fd79006a800 msgr2=0x7fd79006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd798000640 1 -- 172.21.15.26:0/888850202 shutdown_connections 2024-01-15T17:55:22.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.731+0000 7fd798000640 1 -- 172.21.15.26:0/888850202 wait complete. 2024-01-15T17:55:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.733+0000 7fd798000640 1 Processor -- start 2024-01-15T17:55:22.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.733+0000 7fd798000640 1 -- start start 2024-01-15T17:55:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.734+0000 7fd798000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7900cf9a0 0x7fd790071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.734+0000 7fd798000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd79013df80 0x7fd790070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.734+0000 7fd798000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd79013e360 0x7fd79015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.734+0000 7fd798000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd790071960 con 0x7fd79013e360 2024-01-15T17:55:22.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.734+0000 7fd798000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd790072330 con 0x7fd79013df80 2024-01-15T17:55:22.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.734+0000 7fd798000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd790070ad0 con 0x7fd7900cf9a0 2024-01-15T17:55:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd794c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd79013df80 0x7fd790070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd796000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd79013e360 0x7fd79015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd794c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd79013df80 0x7fd790070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:55606/0 (socket says 172.21.15.26:55606) 2024-01-15T17:55:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd794c00640 1 -- 172.21.15.26:0/4291776998 learned_addr learned my addr 172.21.15.26:0/4291776998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd796000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd79013e360 0x7fd79015a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54144/0 (socket says 172.21.15.26:54144) 2024-01-15T17:55:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd795600640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7900cf9a0 0x7fd790071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd796000640 1 -- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7900cf9a0 msgr2=0x7fd790071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd796000640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7900cf9a0 0x7fd790071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd796000640 1 -- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd79013df80 msgr2=0x7fd790070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd796000640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd79013df80 0x7fd790070590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.735+0000 7fd796000640 1 -- 172.21.15.26:0/4291776998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd788009b30 con 0x7fd79013e360 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.736+0000 7fd796000640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd79013e360 0x7fd79015a3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd788032000 tx=0x7fd788004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.736+0000 7fd78d600640 1 -- 172.21.15.26:0/4291776998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788009510 con 0x7fd79013e360 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.736+0000 7fd78d600640 1 -- 172.21.15.26:0/4291776998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78803e450 con 0x7fd79013e360 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.736+0000 7fd78d600640 1 -- 172.21.15.26:0/4291776998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78803e800 con 0x7fd79013e360 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.736+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd790070d50 con 0x7fd79013e360 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.736+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd790071310 con 0x7fd79013e360 2024-01-15T17:55:22.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.737+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd790071b10 con 0x7fd79013e360 2024-01-15T17:55:22.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.738+0000 7fd78d600640 1 -- 172.21.15.26:0/4291776998 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd78802cd50 con 0x7fd79013e360 2024-01-15T17:55:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.739+0000 7fd78d600640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd76806b2c0 0x7fd76806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:22.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.739+0000 7fd78d600640 1 -- 172.21.15.26:0/4291776998 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7880ac9d0 con 0x7fd79013e360 2024-01-15T17:55:22.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.740+0000 7fd795600640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd76806b2c0 0x7fd76806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:22.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.741+0000 7fd795600640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd76806b2c0 0x7fd76806d780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd780006fd0 tx=0x7fd780008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:22.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.744+0000 7fd78d600640 1 -- 172.21.15.26:0/4291776998 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd790071b10 con 0x7fd79013e360 2024-01-15T17:55:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.982+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd790071b10 con 0x7fd79013e360 2024-01-15T17:55:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.983+0000 7fd78d600640 1 -- 172.21.15.26:0/4291776998 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd790071b10 con 0x7fd79013e360 2024-01-15T17:55:22.986 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.985+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd76806b2c0 msgr2=0x7fd76806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.985+0000 7fd798000640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd76806b2c0 0x7fd76806d780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd780006fd0 tx=0x7fd780008040 comp rx=0 tx=0).stop 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.985+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd79013e360 msgr2=0x7fd79015a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.985+0000 7fd798000640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd79013e360 0x7fd79015a3f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd788032000 tx=0x7fd788004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.985+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 shutdown_connections 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.985+0000 7fd798000640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd76806b2c0 0x7fd76806d780 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.985+0000 7fd798000640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd79013e360 0x7fd79015a3f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.985+0000 7fd798000640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd79013df80 0x7fd790070590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.985+0000 7fd798000640 1 --2- 172.21.15.26:0/4291776998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7900cf9a0 0x7fd790071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.986+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 >> 172.21.15.26:0/4291776998 conn(0x7fd79006a800 msgr2=0x7fd7900b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.986+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 shutdown_connections 2024-01-15T17:55:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:22.986+0000 7fd798000640 1 -- 172.21.15.26:0/4291776998 wait complete. 2024-01-15T17:55:23.009 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T17:55:23.010 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T17:55:23.010 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-15T17:55:23.010 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T17:55:23.010 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:23.010 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T17:55:23.010 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T17:55:23.010 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T17:55:23.010 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T17:55:23.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:23.010 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-15T17:55:23.062 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:55:23.062 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-15T17:55:23.063 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-15T17:55:23.064 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42595sec preferred_lft 42595sec 2024-01-15T17:55:23.064 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-01-15T17:55:23.064 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:23.064 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:23.064 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:23.064 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-15T17:55:23.065 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-15T17:55:23.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:23.065 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:55:23.065 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:55:23.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add name ceph-brx type bridge 2024-01-15T17:55:23.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr flush dev ceph-brx 2024-01-15T17:55:23.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set ceph-brx up 2024-01-15T17:55:23.065 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-15T17:55:23.065 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:55:23.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:23.527 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T17:55:23.614 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-15T17:55:23.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:23.618 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-15T17:55:23.681 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-15T17:55:23.681 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T17:55:23.681 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-15T17:55:23.681 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-15T17:55:23.682 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T17:55:23.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:23.682 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:55:23.682 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:55:23.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-15T17:55:23.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-15T17:55:23.682 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-15T17:55:23.682 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:55:24.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:24.445 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T17:55:24.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:24.491 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-01-15T17:55:24.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:24.585 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:55:24.585 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:55:24.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:24.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-15T17:55:24.586 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:55:24.726 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-15T17:55:24.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:24.726 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:55:24.726 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:55:24.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-15T17:55:24.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-15T17:55:24.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-15T17:55:24.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-15T17:55:24.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-15T17:55:24.726 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:55:25.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:25.000 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:55:25.001 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:55:25.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-15T17:55:25.001 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.0 master ceph-brx 2024-01-15T17:55:25.001 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:55:25.162 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T17:55:25.162 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:55:25.162 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:25.201 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T17:55:25.202 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:55:25.202 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:25.289 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:55:25.372 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:25.416 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:55:25.416 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:55:25.416 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:55:25.416 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:55:25.416 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:55:25.416 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:55:25.416 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:55:25.416 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:55:25.417 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:55:25.418 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:55:25.418 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:55:25.418 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:55:25.418 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:55:25.418 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:55:25.418 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:25.418 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:25.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:25.419 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:55:25.509 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T17:55:25.510 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T17:55:25.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:55:25.584 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:25.627 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:55:25.627 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:55:25.627 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:55:25.627 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:55:25.627 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:55:25.627 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:55:25.627 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:55:25.627 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:25.628 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:25.629 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:55:25.629 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:55:25.629 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:55:25.629 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:55:25.629 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:55:25.629 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:55:25.629 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:55:25.629 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:25.629 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:25.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:25.630 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:55:25.688 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:55:25.707 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:2024-01-15T17:55:25.703+0000 7fe7ac26d480 -1 init, newargv = 0x55e2fc5d5a10 newargc=15 2024-01-15T17:55:25.707 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[43236]: starting ceph client 2024-01-15T17:55:25.711 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:55:25.719 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[43236]: starting fuse 2024-01-15T17:55:26.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:55:26.917 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:26.976 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:55:26.976 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:55:26.976 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:55:26.977 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:26.978 INFO:teuthology.orchestra.run.smithi026.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:26.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:26.979 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:55:27.036 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T17:55:27.037 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-01-15T17:55:27.037 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.0")) 2024-01-15T17:55:27.038 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-01-15T17:55:27.225 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.0.43236.asok 2024-01-15T17:55:27.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T17:55:27.244 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43236.asok 2024-01-15T17:55:27.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:27.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43236.asok status 2024-01-15T17:55:27.493 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:55:27.493 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "0", 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "43236", 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4665, 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-01-15T17:55:27.494 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4665 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 295221709 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 295221709 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:55:27.495 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.4665 192.168.0.1:0/295221709", 2024-01-15T17:55:27.496 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/295221709", 2024-01-15T17:55:27.496 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-01-15T17:55:27.496 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 7, 2024-01-15T17:55:27.496 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 22, 2024-01-15T17:55:27.496 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-01-15T17:55:27.496 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-01-15T17:55:27.496 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-01-15T17:55:27.496 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:55:27.509 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T17:55:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.727+0000 7fe2fb200640 1 Processor -- start 2024-01-15T17:55:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.727+0000 7fe2fb200640 1 -- start start 2024-01-15T17:55:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.729+0000 7fe2fb200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe2fc071f00 0x7fe2fc146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.729+0000 7fe2fb200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc0b2e10 con 0x7fe2fc06c7c0 2024-01-15T17:55:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.729+0000 7fe2fb200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc06cc10 con 0x7fe2fc071410 2024-01-15T17:55:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.729+0000 7fe2fb200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2fc06cec0 con 0x7fe2fc071f00 2024-01-15T17:55:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.729+0000 7fe2f1200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe2fc071f00 0x7fe2fc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.729+0000 7fe2fa800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe2fc06c7c0 0x7fe2fc166cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38522/0 (socket says 172.21.15.26:38522) 2024-01-15T17:55:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.729+0000 7fe2fa800640 1 -- 172.21.15.26:0/814844242 learned_addr learned my addr 172.21.15.26:0/814844242 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.730+0000 7fe2f9400640 1 -- 172.21.15.26:0/814844242 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443578205 0 0) 0x7fe2fc0b2e10 con 0x7fe2fc06c7c0 2024-01-15T17:55:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.730+0000 7fe2f9400640 1 -- 172.21.15.26:0/814844242 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2dc003520 con 0x7fe2fc06c7c0 2024-01-15T17:55:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.730+0000 7fe2f1200640 1 -- 172.21.15.26:0/814844242 >> v1:172.21.15.124:6789/0 conn(0x7fe2fc071410 legacy=0x7fe2fc0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.730+0000 7fe2f1200640 1 -- 172.21.15.26:0/814844242 >> v1:172.21.15.26:6789/0 conn(0x7fe2fc06c7c0 legacy=0x7fe2fc166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.730+0000 7fe2f1200640 1 -- 172.21.15.26:0/814844242 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc06d1a0 con 0x7fe2fc071f00 2024-01-15T17:55:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.731+0000 7fe2f1200640 1 --2- 172.21.15.26:0/814844242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 0x7fe2fc146770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40097a0 tx=0x7fe2e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19af5d992996f6df server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.731+0000 7fe2f9400640 1 -- 172.21.15.26:0/814844242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4031830 con 0x7fe2fc071f00 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.731+0000 7fe2f9400640 1 -- 172.21.15.26:0/814844242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e4031990 con 0x7fe2fc071f00 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.731+0000 7fe2f9400640 1 -- 172.21.15.26:0/814844242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4031d20 con 0x7fe2fc071f00 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.732+0000 7fe2fb200640 1 -- 172.21.15.26:0/814844242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 msgr2=0x7fe2fc146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.732+0000 7fe2fb200640 1 --2- 172.21.15.26:0/814844242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 0x7fe2fc146770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40097a0 tx=0x7fe2e402b8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.732+0000 7fe2fb200640 1 -- 172.21.15.26:0/814844242 shutdown_connections 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.732+0000 7fe2fb200640 1 --2- 172.21.15.26:0/814844242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 0x7fe2fc146770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.732+0000 7fe2fb200640 1 -- 172.21.15.26:0/814844242 >> 172.21.15.26:0/814844242 conn(0x7fe2fc06a800 msgr2=0x7fe2fc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.732+0000 7fe2fb200640 1 -- 172.21.15.26:0/814844242 shutdown_connections 2024-01-15T17:55:27.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.732+0000 7fe2fb200640 1 -- 172.21.15.26:0/814844242 wait complete. 2024-01-15T17:55:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.734+0000 7fe2fb200640 1 Processor -- start 2024-01-15T17:55:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.734+0000 7fe2fb200640 1 -- start start 2024-01-15T17:55:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.735+0000 7fe2fb200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2fc06c7c0 0x7fe2fc1f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.735+0000 7fe2fb200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2fc071410 0x7fe2fc0c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.735+0000 7fe2fb200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 0x7fe2fc15c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.735+0000 7fe2fb200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc13c6e0 con 0x7fe2fc071f00 2024-01-15T17:55:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.735+0000 7fe2fb200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc13ba60 con 0x7fe2fc06c7c0 2024-01-15T17:55:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.735+0000 7fe2fb200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc13bd40 con 0x7fe2fc071410 2024-01-15T17:55:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2fa800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 0x7fe2fc15c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2fa800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 0x7fe2fc15c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54164/0 (socket says 172.21.15.26:54164) 2024-01-15T17:55:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2fa800640 1 -- 172.21.15.26:0/985962152 learned_addr learned my addr 172.21.15.26:0/985962152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2f1200640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2fc071410 0x7fe2fc0c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2fa800640 1 -- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2fc071410 msgr2=0x7fe2fc0c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2fa800640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2fc071410 0x7fe2fc0c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2f9e00640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2fc06c7c0 0x7fe2fc1f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2fa800640 1 -- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2fc06c7c0 msgr2=0x7fe2fc1f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2fa800640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2fc06c7c0 0x7fe2fc1f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2fa800640 1 -- 172.21.15.26:0/985962152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4009400 con 0x7fe2fc071f00 2024-01-15T17:55:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2f9e00640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2fc06c7c0 0x7fe2fc1f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.736+0000 7fe2fa800640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 0x7fe2fc15c500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400cb40 tx=0x7fe2f400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.737+0000 7fe2f2a00640 1 -- 172.21.15.26:0/985962152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400aca0 con 0x7fe2fc071f00 2024-01-15T17:55:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.737+0000 7fe2f2a00640 1 -- 172.21.15.26:0/985962152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f400ae00 con 0x7fe2fc071f00 2024-01-15T17:55:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.737+0000 7fe2f2a00640 1 -- 172.21.15.26:0/985962152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4012620 con 0x7fe2fc071f00 2024-01-15T17:55:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.737+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc13c020 con 0x7fe2fc071f00 2024-01-15T17:55:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.737+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc1f6a40 con 0x7fe2fc071f00 2024-01-15T17:55:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.738+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc1f6c30 con 0x7fe2fc071f00 2024-01-15T17:55:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.739+0000 7fe2f2a00640 1 -- 172.21.15.26:0/985962152 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe2f4007070 con 0x7fe2fc071f00 2024-01-15T17:55:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.740+0000 7fe2f2a00640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe2d406b2c0 0x7fe2d406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.740+0000 7fe2f2a00640 1 -- 172.21.15.26:0/985962152 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2f4089cc0 con 0x7fe2fc071f00 2024-01-15T17:55:27.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.741+0000 7fe2f9e00640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe2d406b2c0 0x7fe2d406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.742+0000 7fe2f9e00640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe2d406b2c0 0x7fe2d406d780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec000f60 tx=0x7fe2ec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.745+0000 7fe2f2a00640 1 -- 172.21.15.26:0/985962152 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2f4090080 con 0x7fe2fc071f00 2024-01-15T17:55:27.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.979+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe2fc13c820 con 0x7fe2fc071f00 2024-01-15T17:55:27.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.980+0000 7fe2f2a00640 1 -- 172.21.15.26:0/985962152 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe2fc13c820 con 0x7fe2fc071f00 2024-01-15T17:55:27.983 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T17:55:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe2d406b2c0 msgr2=0x7fe2d406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe2d406b2c0 0x7fe2d406d780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec000f60 tx=0x7fe2ec009040 comp rx=0 tx=0).stop 2024-01-15T17:55:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 msgr2=0x7fe2fc15c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 0x7fe2fc15c500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400cb40 tx=0x7fe2f400a060 comp rx=0 tx=0).stop 2024-01-15T17:55:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 shutdown_connections 2024-01-15T17:55:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2fc071410 0x7fe2fc0c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe2d406b2c0 0x7fe2d406d780 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2fc071f00 0x7fe2fc15c500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 --2- 172.21.15.26:0/985962152 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2fc06c7c0 0x7fe2fc1f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 >> 172.21.15.26:0/985962152 conn(0x7fe2fc06a800 msgr2=0x7fe2fc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 shutdown_connections 2024-01-15T17:55:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:27.983+0000 7fe2fb200640 1 -- 172.21.15.26:0/985962152 wait complete. 2024-01-15T17:55:28.004 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T17:55:28.004 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T17:55:28.004 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-15T17:55:28.004 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-01-15T17:55:28.004 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-01-15T17:55:28.004 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T17:55:28.004 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T17:55:28.004 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T17:55:28.004 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-01-15T17:55:28.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.005 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-15T17:55:28.061 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42590sec preferred_lft 42590sec 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:55:28.062 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T17:55:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::a4e5:3ff:fe91:fcad/64 scope link 2024-01-15T17:55:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.063 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:55:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::f464:8fff:fecd:58d2/64 scope link 2024-01-15T17:55:28.063 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.064 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T17:55:28.154 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-15T17:55:28.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.159 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-15T17:55:28.216 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-15T17:55:28.216 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T17:55:28.216 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-15T17:55:28.216 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-15T17:55:28.216 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T17:55:28.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.217 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:55:28.217 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:55:28.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-15T17:55:28.218 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-15T17:55:28.218 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-15T17:55:28.218 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:55:28.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.417 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T17:55:28.470 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T17:55:28.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.472 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list-id 2024-01-15T17:55:28.524 INFO:teuthology.orchestra.run.smithi026.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-01-15T17:55:28.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.526 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:55:28.526 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:55:28.526 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:55:28.526 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-01-15T17:55:28.526 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:55:28.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.673 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-15T17:55:28.773 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 6a:dd:9b:42:80:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::68dd:9bff:fe42:80db/64 scope link 2024-01-15T17:55:28.774 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.779 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-01-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi026.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 6a:dd:9b:42:80:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-15T17:55:28.872 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-15T17:55:28.873 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.873 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::68dd:9bff:fe42:80db/64 scope link 2024-01-15T17:55:28.873 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:28.877 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-01-15T17:55:28.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:28.877 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:55:28.877 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:55:28.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-01-15T17:55:28.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-15T17:55:28.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-01-15T17:55:28.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-01-15T17:55:28.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-01-15T17:55:28.877 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:55:29.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:29.144 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:55:29.145 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:55:29.145 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.1 up 2024-01-15T17:55:29.145 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set dev brx.1 master ceph-brx 2024-01-15T17:55:29.145 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:55:29.308 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-01-15T17:55:29.308 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:55:29.308 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-01-15T17:55:29.358 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-01-15T17:55:29.359 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:55:29.359 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-01-15T17:55:29.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:55:29.508 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:29.566 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:55:29.567 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:55:29.568 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:55:29.569 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:55:29.569 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:55:29.569 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:55:29.569 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:29.569 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:29.569 INFO:teuthology.orchestra.run.smithi026.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:29.569 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:55:29.569 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:55:29.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:29.570 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:55:29.627 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T17:55:29.627 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-01-15T17:55:29.628 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-01-15T17:55:29.633 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:55:29.708 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:29.755 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:55:29.755 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:55:29.755 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:55:29.755 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:55:29.755 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:55:29.755 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:55:29.755 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:55:29.755 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:55:29.755 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:29.756 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:55:29.757 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:55:29.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:29.758 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:55:29.806 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T17:55:29.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi026.stderr:2024-01-15T17:55:29.815+0000 7f12d0f57480 -1 init, newargv = 0x55ba2d06bd10 newargc=15 2024-01-15T17:55:29.819 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi026.stderr:ceph-fuse[43823]: starting ceph client 2024-01-15T17:55:29.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi026.stderr:ceph-fuse[43823]: starting fuse 2024-01-15T17:55:30.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:55:31.018 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:31.078 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:55:31.078 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:55:31.078 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:55:31.078 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:55:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:31.081 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:55:31.081 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:55:31.081 INFO:teuthology.orchestra.run.smithi026.stdout:/home/ubuntu/cephtest/mnt.2 2024-01-15T17:55:31.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:31.082 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:55:31.134 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T17:55:31.134 INFO:teuthology.orchestra.run.smithi026.stdout:42 2024-01-15T17:55:31.134 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 42] 2024-01-15T17:55:31.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:55:31.135 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.2")) 2024-01-15T17:55:31.136 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-01-15T17:55:31.323 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.2.43823.asok 2024-01-15T17:55:31.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T17:55:31.341 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.43823.asok 2024-01-15T17:55:31.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:31.341 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.43823.asok status 2024-01-15T17:55:31.588 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:55:31.588 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "2", 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "43823", 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4683, 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-01-15T17:55:31.589 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 4683 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.2:0", 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1641309417 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T17:55:31.590 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.2:0", 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1641309417 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.4683 192.168.0.2:0/1641309417", 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.2:0/1641309417", 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 7, 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 22, 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-01-15T17:55:31.591 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:55:31.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T17:55:31.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.825+0000 7fc3be800640 1 Processor -- start 2024-01-15T17:55:31.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.825+0000 7fc3be800640 1 -- start start 2024-01-15T17:55:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.827+0000 7fc3be800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc3b814b2c0 0x7fc3b80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.827+0000 7fc3be800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3b80cf7d0 0x7fc3b8070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:31.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.827+0000 7fc3be800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b8070ea0 0x7fc3b80b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:31.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.827+0000 7fc3be800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b814b6a0 con 0x7fc3b80cf7d0 2024-01-15T17:55:31.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.827+0000 7fc3be800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b80cfbf0 con 0x7fc3b814b2c0 2024-01-15T17:55:31.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.827+0000 7fc3be800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8071690 con 0x7fc3b8070ea0 2024-01-15T17:55:31.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.827+0000 7fc3afe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3b80cf7d0 0x7fc3b8070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:31.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.828+0000 7fc3afe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc3b80cf7d0 0x7fc3b8070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54176/0 (socket says 172.21.15.26:54176) 2024-01-15T17:55:31.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.828+0000 7fc3afe00640 1 -- 172.21.15.26:0/2444739321 learned_addr learned my addr 172.21.15.26:0/2444739321 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:31.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.828+0000 7fc3b7400640 1 --2- 172.21.15.26:0/2444739321 >> v2:172.21.15.124:3300/0 conn(0x7fc3b814b2c0 0x7fc3b80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:31.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.828+0000 7fc3b7e00640 1 --2- 172.21.15.26:0/2444739321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b8070ea0 0x7fc3b80b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:31.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.828+0000 7fc3afe00640 1 -- 172.21.15.26:0/2444739321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b8070ea0 msgr2=0x7fc3b80b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:31.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.828+0000 7fc3afe00640 1 --2- 172.21.15.26:0/2444739321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b8070ea0 0x7fc3b80b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:31.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.828+0000 7fc3afe00640 1 -- 172.21.15.26:0/2444739321 >> v2:172.21.15.124:3300/0 conn(0x7fc3b814b2c0 msgr2=0x7fc3b80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:31.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.828+0000 7fc3afe00640 1 --2- 172.21.15.26:0/2444739321 >> v2:172.21.15.124:3300/0 conn(0x7fc3b814b2c0 0x7fc3b80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:31.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.828+0000 7fc3afe00640 1 -- 172.21.15.26:0/2444739321 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b8071970 con 0x7fc3b80cf7d0 2024-01-15T17:55:31.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.829+0000 7fc3afe00640 1 --2- 172.21.15.26:0/2444739321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b80cf7d0 0x7fc3b8070960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3b806e650 tx=0x7fc3a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6bdd2fa95f47bf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:31.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.830+0000 7fc3b6a00640 1 -- 172.21.15.26:0/2444739321 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0031870 con 0x7fc3b80cf7d0 2024-01-15T17:55:31.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.830+0000 7fc3b6a00640 1 -- 172.21.15.26:0/2444739321 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a00319d0 con 0x7fc3b80cf7d0 2024-01-15T17:55:31.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.830+0000 7fc3b6a00640 1 -- 172.21.15.26:0/2444739321 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0031d60 con 0x7fc3b80cf7d0 2024-01-15T17:55:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.831+0000 7fc3be800640 1 -- 172.21.15.26:0/2444739321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b80cf7d0 msgr2=0x7fc3b8070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.831+0000 7fc3be800640 1 --2- 172.21.15.26:0/2444739321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b80cf7d0 0x7fc3b8070960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc3b806e650 tx=0x7fc3a002b910 comp rx=0 tx=0).stop 2024-01-15T17:55:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.831+0000 7fc3be800640 1 -- 172.21.15.26:0/2444739321 shutdown_connections 2024-01-15T17:55:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.831+0000 7fc3be800640 1 --2- 172.21.15.26:0/2444739321 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b8070ea0 0x7fc3b80b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.831+0000 7fc3be800640 1 --2- 172.21.15.26:0/2444739321 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b80cf7d0 0x7fc3b8070960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.831+0000 7fc3be800640 1 --2- 172.21.15.26:0/2444739321 >> v2:172.21.15.124:3300/0 conn(0x7fc3b814b2c0 0x7fc3b80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.831+0000 7fc3be800640 1 -- 172.21.15.26:0/2444739321 >> 172.21.15.26:0/2444739321 conn(0x7fc3b806a800 msgr2=0x7fc3b806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.831+0000 7fc3be800640 1 -- 172.21.15.26:0/2444739321 shutdown_connections 2024-01-15T17:55:31.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.831+0000 7fc3be800640 1 -- 172.21.15.26:0/2444739321 wait complete. 2024-01-15T17:55:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.833+0000 7fc3be800640 1 Processor -- start 2024-01-15T17:55:31.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.833+0000 7fc3be800640 1 -- start start 2024-01-15T17:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.834+0000 7fc3be800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b8070ea0 0x7fc3b815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.834+0000 7fc3be800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b80cf7d0 0x7fc3b80cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.834+0000 7fc3be800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b814b2c0 0x7fc3b80cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.834+0000 7fc3be800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8163620 con 0x7fc3b8070ea0 2024-01-15T17:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.834+0000 7fc3be800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8163900 con 0x7fc3b80cf7d0 2024-01-15T17:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.834+0000 7fc3be800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b8163be0 con 0x7fc3b814b2c0 2024-01-15T17:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.834+0000 7fc3b7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b8070ea0 0x7fc3b815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.834+0000 7fc3b7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b8070ea0 0x7fc3b815e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54186/0 (socket says 172.21.15.26:54186) 2024-01-15T17:55:31.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.834+0000 7fc3b7400640 1 -- 172.21.15.26:0/2228732636 learned_addr learned my addr 172.21.15.26:0/2228732636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.835+0000 7fc3afe00640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b80cf7d0 0x7fc3b80cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.835+0000 7fc3b7e00640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b814b2c0 0x7fc3b80cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.835+0000 7fc3b7400640 1 -- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b814b2c0 msgr2=0x7fc3b80cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.835+0000 7fc3b7400640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b814b2c0 0x7fc3b80cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.835+0000 7fc3b7400640 1 -- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b80cf7d0 msgr2=0x7fc3b80cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.835+0000 7fc3b7400640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b80cf7d0 0x7fc3b80cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.835+0000 7fc3b7400640 1 -- 172.21.15.26:0/2228732636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a00094d0 con 0x7fc3b8070ea0 2024-01-15T17:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.835+0000 7fc3afe00640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b80cf7d0 0x7fc3b80cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:31.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.835+0000 7fc3b7400640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b8070ea0 0x7fc3b815e0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800d800 tx=0x7fc3a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.836+0000 7fc3af400640 1 -- 172.21.15.26:0/2228732636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800b6c0 con 0x7fc3b8070ea0 2024-01-15T17:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.836+0000 7fc3af400640 1 -- 172.21.15.26:0/2228732636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a800b820 con 0x7fc3b8070ea0 2024-01-15T17:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.836+0000 7fc3af400640 1 -- 172.21.15.26:0/2228732636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800bbb0 con 0x7fc3b8070ea0 2024-01-15T17:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.836+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b8163ec0 con 0x7fc3b8070ea0 2024-01-15T17:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.836+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b8148570 con 0x7fc3b8070ea0 2024-01-15T17:55:31.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.837+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b80675d0 con 0x7fc3b8070ea0 2024-01-15T17:55:31.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.838+0000 7fc3af400640 1 -- 172.21.15.26:0/2228732636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc3a800bd10 con 0x7fc3b8070ea0 2024-01-15T17:55:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.839+0000 7fc3af400640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc39006b2c0 0x7fc39006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.839+0000 7fc3af400640 1 -- 172.21.15.26:0/2228732636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3a808a180 con 0x7fc3b8070ea0 2024-01-15T17:55:31.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.839+0000 7fc3afe00640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc39006b2c0 0x7fc39006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:31.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.840+0000 7fc3afe00640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc39006b2c0 0x7fc39006d780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0036000 tx=0x7fc3a003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:31.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:31.843+0000 7fc3af400640 1 -- 172.21.15.26:0/2228732636 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3a8054fe0 con 0x7fc3b8070ea0 2024-01-15T17:55:32.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.080+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc3b80ce320 con 0x7fc3b8070ea0 2024-01-15T17:55:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.081+0000 7fc3af400640 1 -- 172.21.15.26:0/2228732636 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc3a8058600 con 0x7fc3b8070ea0 2024-01-15T17:55:32.084 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T17:55:32.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc39006b2c0 msgr2=0x7fc39006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:32.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc39006b2c0 0x7fc39006d780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0036000 tx=0x7fc3a003c040 comp rx=0 tx=0).stop 2024-01-15T17:55:32.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b8070ea0 msgr2=0x7fc3b815e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:32.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b8070ea0 0x7fc3b815e0c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800d800 tx=0x7fc3a800dcc0 comp rx=0 tx=0).stop 2024-01-15T17:55:32.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 shutdown_connections 2024-01-15T17:55:32.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc3b814b2c0 0x7fc3b80cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:32.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc39006b2c0 0x7fc39006d780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc3b80cf7d0 0x7fc3b80cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 --2- 172.21.15.26:0/2228732636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc3b8070ea0 0x7fc3b815e0c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 >> 172.21.15.26:0/2228732636 conn(0x7fc3b806a800 msgr2=0x7fc3b806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 shutdown_connections 2024-01-15T17:55:32.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:32.084+0000 7fc3be800640 1 -- 172.21.15.26:0/2228732636 wait complete. 2024-01-15T17:55:32.105 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T17:55:32.105 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T17:55:32.105 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2024-01-15T17:55:32.105 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T17:55:32.105 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:32.105 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T17:55:32.105 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T17:55:32.105 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T17:55:32.106 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T17:55:32.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:32.106 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2024-01-15T17:55:32.162 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:55:32.162 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:55:32.162 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:55:32.162 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-15T17:55:32.163 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42583sec preferred_lft 42583sec 2024-01-15T17:55:32.164 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2024-01-15T17:55:32.164 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:32.164 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-15T17:55:32.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:32.165 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:55:32.165 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:55:32.165 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link add name ceph-brx type bridge 2024-01-15T17:55:32.165 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip addr flush dev ceph-brx 2024-01-15T17:55:32.165 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set ceph-brx up 2024-01-15T17:55:32.166 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-15T17:55:32.166 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:55:32.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:32.584 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T17:55:32.661 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T17:55:32.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:32.665 DEBUG:teuthology.orchestra.run.smithi124:> route 2024-01-15T17:55:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2024-01-15T17:55:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T17:55:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T17:55:32.724 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T17:55:32.725 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T17:55:32.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:32.725 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:55:32.725 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:55:32.725 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T17:55:32.725 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T17:55:32.725 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T17:55:32.726 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:55:33.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:33.401 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T17:55:33.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:33.459 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list-id 2024-01-15T17:55:33.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:33.510 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:55:33.510 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:55:33.510 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:33.510 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-01-15T17:55:33.511 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:55:33.650 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-01-15T17:55:33.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:33.650 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:55:33.650 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:55:33.650 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-01-15T17:55:33.651 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-15T17:55:33.651 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-01-15T17:55:33.651 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-01-15T17:55:33.651 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-01-15T17:55:33.651 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:55:33.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:33.961 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:55:33.961 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:55:33.961 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2024-01-15T17:55:33.961 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set dev brx.0 master ceph-brx 2024-01-15T17:55:33.961 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:55:34.118 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T17:55:34.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:55:34.119 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:34.161 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T17:55:34.161 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:55:34.161 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:34.257 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T17:55:34.349 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T17:55:34.398 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T17:55:34.399 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T17:55:34.400 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T17:55:34.400 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T17:55:34.400 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T17:55:34.400 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T17:55:34.400 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:34.400 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:34.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:34.401 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T17:55:34.446 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T17:55:34.446 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T17:55:34.493 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T17:55:34.574 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:34.616 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T17:55:34.628 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T17:55:34.628 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T17:55:34.628 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:34.629 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:34.630 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:34.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:34.632 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T17:55:34.683 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:2024-01-15T17:55:34.680+0000 7fad0aa47480 -1 init, newargv = 0x55b0899b86c0 newargc=15 2024-01-15T17:55:34.683 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[41950]: starting ceph client 2024-01-15T17:55:34.693 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[41950]: starting fuse 2024-01-15T17:55:34.713 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T17:55:34.713 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T17:55:34.713 DEBUG:teuthology.orchestra.run.smithi124:> sudo stdin-killer -- python3 -c ' 2024-01-15T17:55:34.713 DEBUG:teuthology.orchestra.run.smithi124:> import glob 2024-01-15T17:55:34.713 DEBUG:teuthology.orchestra.run.smithi124:> import re 2024-01-15T17:55:34.713 DEBUG:teuthology.orchestra.run.smithi124:> import os 2024-01-15T17:55:34.713 DEBUG:teuthology.orchestra.run.smithi124:> import subprocess 2024-01-15T17:55:34.713 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> def _find_admin_socket(client_name): 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> files = glob.glob(asok_path) 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> # Given a non-glob path, it better be there 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> if "*" not in asok_path: 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> assert(len(files) == 1) 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> return files[0] 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> for f in files: 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T17:55:34.714 DEBUG:teuthology.orchestra.run.smithi124:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T17:55:34.715 DEBUG:teuthology.orchestra.run.smithi124:> contents = proc_f.read() 2024-01-15T17:55:34.715 DEBUG:teuthology.orchestra.run.smithi124:> if mountpoint in contents: 2024-01-15T17:55:34.715 DEBUG:teuthology.orchestra.run.smithi124:> return f 2024-01-15T17:55:34.715 DEBUG:teuthology.orchestra.run.smithi124:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T17:55:34.715 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:55:34.715 DEBUG:teuthology.orchestra.run.smithi124:> print(_find_admin_socket("client.1")) 2024-01-15T17:55:34.715 DEBUG:teuthology.orchestra.run.smithi124:> ' 2024-01-15T17:55:34.924 INFO:teuthology.orchestra.run.smithi124.stdout:/var/run/ceph/ceph-client.1.41950.asok 2024-01-15T17:55:34.928 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:55:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T17:55:34.941 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.41950.asok 2024-01-15T17:55:34.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:34.941 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.41950.asok status 2024-01-15T17:55:35.177 INFO:teuthology.orchestra.run.smithi124.stdout:{ 2024-01-15T17:55:35.177 INFO:teuthology.orchestra.run.smithi124.stdout: "metadata": { 2024-01-15T17:55:35.177 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: "entity_id": "1", 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: "hostname": "smithi124", 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: "pid": "41950", 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: "root": "/" 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_count": 0, 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_pinned_count": 0, 2024-01-15T17:55:35.178 INFO:teuthology.orchestra.run.smithi124.stdout: "id": 34400, 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: "inst": { 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: "name": { 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "client", 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: "num": 34400 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 4226682904 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: } 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:55:35.179 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 4226682904 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "inst_str": "client.34400 192.168.0.1:0/4226682904", 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "addr_str": "192.168.0.1:0/4226682904", 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "inode_count": 1, 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "mds_epoch": 7, 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch": 22, 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch_barrier": 0, 2024-01-15T17:55:35.180 INFO:teuthology.orchestra.run.smithi124.stdout: "blocklisted": false, 2024-01-15T17:55:35.181 INFO:teuthology.orchestra.run.smithi124.stdout: "fs_name": "cephfs" 2024-01-15T17:55:35.181 INFO:teuthology.orchestra.run.smithi124.stdout:} 2024-01-15T17:55:35.193 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T17:55:35.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.418+0000 7f5bcd200640 1 Processor -- start 2024-01-15T17:55:35.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.418+0000 7f5bcd200640 1 -- start start 2024-01-15T17:55:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.420+0000 7f5bcd200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5bc813df80 0x7f5bc815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.420+0000 7f5bcd200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc813e360 0x7f5bc80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.420+0000 7f5bcd200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc806c470 con 0x7f5bc815f9c0 2024-01-15T17:55:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.420+0000 7f5bcd200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc806c750 con 0x7f5bc813df80 2024-01-15T17:55:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.420+0000 7f5bcd200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc806ca30 con 0x7f5bc813e360 2024-01-15T17:55:35.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.420+0000 7f5bc5e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5bc813df80 0x7f5bc815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.421+0000 7f5bc5e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5bc813df80 0x7f5bc815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44366/0 (socket says 172.21.15.26:44366) 2024-01-15T17:55:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.421+0000 7f5bc5e00640 1 -- 172.21.15.26:0/2972816326 learned_addr learned my addr 172.21.15.26:0/2972816326 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.421+0000 7f5bc5400640 1 --2- 172.21.15.26:0/2972816326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc813e360 0x7f5bc80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.421+0000 7f5bc5e00640 1 -- 172.21.15.26:0/2972816326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc813e360 msgr2=0x7f5bc80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.421+0000 7f5bc5e00640 1 --2- 172.21.15.26:0/2972816326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc813e360 0x7f5bc80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.421+0000 7f5bc5e00640 1 -- 172.21.15.26:0/2972816326 >> v1:172.21.15.124:6789/0 conn(0x7f5bc815f9c0 legacy=0x7f5bc81408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:35.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.421+0000 7f5bc5e00640 1 -- 172.21.15.26:0/2972816326 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc806cd10 con 0x7f5bc813df80 2024-01-15T17:55:35.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.423+0000 7f5bc5e00640 1 --2- 172.21.15.26:0/2972816326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813df80 0x7f5bc815bee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009e50 tx=0x7f5bc002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd866cef12d1aef server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:35.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.423+0000 7f5bc4a00640 1 -- 172.21.15.26:0/2972816326 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc002ca80 con 0x7f5bc813df80 2024-01-15T17:55:35.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.424+0000 7f5bc4a00640 1 -- 172.21.15.26:0/2972816326 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc002cbe0 con 0x7f5bc813df80 2024-01-15T17:55:35.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.424+0000 7f5bc4a00640 1 -- 172.21.15.26:0/2972816326 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00343f0 con 0x7f5bc813df80 2024-01-15T17:55:35.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.424+0000 7f5bcd200640 1 -- 172.21.15.26:0/2972816326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813df80 msgr2=0x7f5bc815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:35.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.424+0000 7f5bcd200640 1 --2- 172.21.15.26:0/2972816326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813df80 0x7f5bc815bee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009e50 tx=0x7f5bc002bee0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.424+0000 7f5bcd200640 1 -- 172.21.15.26:0/2972816326 shutdown_connections 2024-01-15T17:55:35.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.424+0000 7f5bcd200640 1 --2- 172.21.15.26:0/2972816326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc813e360 0x7f5bc80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.424+0000 7f5bcd200640 1 --2- 172.21.15.26:0/2972816326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813df80 0x7f5bc815bee0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.424+0000 7f5bcd200640 1 -- 172.21.15.26:0/2972816326 >> 172.21.15.26:0/2972816326 conn(0x7f5bc806a800 msgr2=0x7f5bc806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:35.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.425+0000 7f5bcd200640 1 -- 172.21.15.26:0/2972816326 shutdown_connections 2024-01-15T17:55:35.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.425+0000 7f5bcd200640 1 -- 172.21.15.26:0/2972816326 wait complete. 2024-01-15T17:55:35.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.426+0000 7f5bcd200640 1 Processor -- start 2024-01-15T17:55:35.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.426+0000 7f5bcd200640 1 -- start start 2024-01-15T17:55:35.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.427+0000 7f5bcd200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bc813df80 0x7f5bc8157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:35.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.427+0000 7f5bcd200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813e360 0x7f5bc815a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:35.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.427+0000 7f5bcd200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc815f9c0 0x7f5bc8152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:35.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.427+0000 7f5bcd200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc815a6b0 con 0x7f5bc813e360 2024-01-15T17:55:35.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.427+0000 7f5bcd200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc815a930 con 0x7f5bc813df80 2024-01-15T17:55:35.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.427+0000 7f5bcd200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc815ac10 con 0x7f5bc815f9c0 2024-01-15T17:55:35.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813e360 0x7f5bc815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813e360 0x7f5bc815a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44376/0 (socket says 172.21.15.26:44376) 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5400640 1 -- 172.21.15.26:0/548489716 learned_addr learned my addr 172.21.15.26:0/548489716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc6800640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc815f9c0 0x7f5bc8152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5e00640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bc813df80 0x7f5bc8157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5400640 1 -- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc815f9c0 msgr2=0x7f5bc8152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5400640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc815f9c0 0x7f5bc8152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5400640 1 -- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bc813df80 msgr2=0x7f5bc8157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5400640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bc813df80 0x7f5bc8157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5400640 1 -- 172.21.15.26:0/548489716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc0009b30 con 0x7f5bc813e360 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc6800640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc815f9c0 0x7f5bc8152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.428+0000 7f5bc5e00640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bc813df80 0x7f5bc8157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:35.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.429+0000 7f5bc5400640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813e360 0x7f5bc815a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000d800 tx=0x7f5bb000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:35.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.429+0000 7f5bbe000640 1 -- 172.21.15.26:0/548489716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000b6c0 con 0x7f5bc813e360 2024-01-15T17:55:35.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.429+0000 7f5bbe000640 1 -- 172.21.15.26:0/548489716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb000b820 con 0x7f5bc813e360 2024-01-15T17:55:35.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.429+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc8070360 con 0x7f5bc813e360 2024-01-15T17:55:35.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.429+0000 7f5bbe000640 1 -- 172.21.15.26:0/548489716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000bbb0 con 0x7f5bc813e360 2024-01-15T17:55:35.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.429+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc8070920 con 0x7f5bc813e360 2024-01-15T17:55:35.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.431+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc80675d0 con 0x7f5bc813e360 2024-01-15T17:55:35.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.431+0000 7f5bbe000640 1 -- 172.21.15.26:0/548489716 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5bb000bd10 con 0x7f5bc813e360 2024-01-15T17:55:35.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.433+0000 7f5bbe000640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ba006b2c0 0x7f5ba006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:35.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.433+0000 7f5bbe000640 1 -- 172.21.15.26:0/548489716 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bb008a180 con 0x7f5bc813e360 2024-01-15T17:55:35.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.433+0000 7f5bc5e00640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ba006b2c0 0x7f5ba006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:35.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.434+0000 7f5bc5e00640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ba006b2c0 0x7f5ba006d780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0035000 tx=0x7f5bc0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.437+0000 7f5bbe000640 1 -- 172.21.15.26:0/548489716 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5bb0054fe0 con 0x7f5bc813e360 2024-01-15T17:55:35.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.688+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5bc80cc190 con 0x7f5bc813e360 2024-01-15T17:55:35.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.689+0000 7f5bbe000640 1 -- 172.21.15.26:0/548489716 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5bb0058600 con 0x7f5bc813e360 2024-01-15T17:55:35.693 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T17:55:35.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ba006b2c0 msgr2=0x7f5ba006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ba006b2c0 0x7f5ba006d780 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0035000 tx=0x7f5bc0030000 comp rx=0 tx=0).stop 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813e360 msgr2=0x7f5bc815a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813e360 0x7f5bc815a170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000d800 tx=0x7f5bb000dcc0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 shutdown_connections 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bc815f9c0 0x7f5bc8152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ba006b2c0 0x7f5ba006d780 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bc813e360 0x7f5bc815a170 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 --2- 172.21.15.26:0/548489716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bc813df80 0x7f5bc8157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 >> 172.21.15.26:0/548489716 conn(0x7f5bc806a800 msgr2=0x7f5bc80b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 shutdown_connections 2024-01-15T17:55:35.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:35.692+0000 7f5bcd200640 1 -- 172.21.15.26:0/548489716 wait complete. 2024-01-15T17:55:35.715 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T17:55:35.715 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T17:55:35.715 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2024-01-15T17:55:35.716 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-01-15T17:55:35.716 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-01-15T17:55:35.716 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T17:55:35.716 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T17:55:35.716 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T17:55:35.716 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-01-15T17:55:35.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:35.716 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2024-01-15T17:55:35.771 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:55:35.771 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:55:35.771 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2024-01-15T17:55:35.772 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42580sec preferred_lft 42580sec 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 26:4f:4f:5b:0b:31 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4c2d:94ff:fe47:f54c/64 scope link 2024-01-15T17:55:35.773 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:35.774 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:55:35.774 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 26:4f:4f:5b:0b:31 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:35.774 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::244f:4fff:fe5b:b31/64 scope link 2024-01-15T17:55:35.774 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:35.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:35.774 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T17:55:35.864 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T17:55:35.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:35.869 DEBUG:teuthology.orchestra.run.smithi124:> route 2024-01-15T17:55:35.920 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2024-01-15T17:55:35.920 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T17:55:35.920 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T17:55:35.920 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T17:55:35.921 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T17:55:35.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:35.922 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:55:35.922 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:55:35.922 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T17:55:35.922 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T17:55:35.922 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T17:55:35.922 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:55:36.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:36.099 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T17:55:36.152 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T17:55:36.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:36.154 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list-id 2024-01-15T17:55:36.203 INFO:teuthology.orchestra.run.smithi124.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-01-15T17:55:36.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:36.204 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:55:36.205 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:55:36.205 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:55:36.205 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-01-15T17:55:36.205 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:55:36.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:36.343 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-15T17:55:36.429 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:55:36.430 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:55:36.430 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:55:36.430 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:36.430 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T17:55:36.430 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:36.430 INFO:teuthology.orchestra.run.smithi124.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:55:36.430 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 7e:f7:e0:57:6a:be brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-15T17:55:36.430 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-15T17:55:36.430 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:36.431 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::7cf7:e0ff:fe57:6abe/64 scope link 2024-01-15T17:55:36.431 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:36.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:36.434 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-01-15T17:55:36.526 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:55:36.526 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 7e:f7:e0:57:6a:be brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::7cf7:e0ff:fe57:6abe/64 scope link 2024-01-15T17:55:36.527 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:55:36.531 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-01-15T17:55:36.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:36.532 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:55:36.532 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:55:36.532 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-01-15T17:55:36.532 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-01-15T17:55:36.532 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-01-15T17:55:36.532 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-01-15T17:55:36.532 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-01-15T17:55:36.532 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:55:36.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:36.800 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:55:36.800 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:55:36.800 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.1 up 2024-01-15T17:55:36.800 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set dev brx.1 master ceph-brx 2024-01-15T17:55:36.801 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:55:36.964 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-01-15T17:55:36.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:55:36.964 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-01-15T17:55:37.010 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-01-15T17:55:37.011 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:55:37.011 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-01-15T17:55:37.102 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T17:55:37.187 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:37.243 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T17:55:37.243 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T17:55:37.243 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T17:55:37.243 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T17:55:37.243 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T17:55:37.243 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T17:55:37.243 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T17:55:37.243 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T17:55:37.244 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T17:55:37.244 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T17:55:37.244 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T17:55:37.244 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:37.244 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T17:55:37.244 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T17:55:37.244 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T17:55:37.244 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T17:55:37.245 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:37.245 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T17:55:37.245 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:37.245 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:37.245 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:37.245 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:37.245 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T17:55:37.245 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T17:55:37.245 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T17:55:37.246 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T17:55:37.246 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T17:55:37.246 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T17:55:37.246 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T17:55:37.246 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:37.246 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:37.246 INFO:teuthology.orchestra.run.smithi124.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:37.246 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:55:37.246 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:55:37.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:37.248 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T17:55:37.299 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T17:55:37.300 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-01-15T17:55:37.300 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-01-15T17:55:37.308 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T17:55:37.385 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:55:37.424 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T17:55:37.425 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T17:55:37.426 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T17:55:37.427 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:37.427 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:55:37.427 INFO:teuthology.orchestra.run.smithi124.stdout:/home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:37.427 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:55:37.427 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:55:37.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:37.428 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T17:55:37.490 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi124.stderr:2024-01-15T17:55:37.487+0000 7f8a5720f480 -1 init, newargv = 0x558770199970 newargc=15 2024-01-15T17:55:37.490 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi124.stderr:ceph-fuse[42402]: starting ceph client 2024-01-15T17:55:37.500 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi124.stderr:ceph-fuse[42402]: starting fuse 2024-01-15T17:55:37.513 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T17:55:37.514 INFO:teuthology.orchestra.run.smithi124.stdout:42 2024-01-15T17:55:37.514 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 42] 2024-01-15T17:55:37.514 DEBUG:teuthology.orchestra.run.smithi124:> sudo stdin-killer -- python3 -c ' 2024-01-15T17:55:37.514 DEBUG:teuthology.orchestra.run.smithi124:> import glob 2024-01-15T17:55:37.514 DEBUG:teuthology.orchestra.run.smithi124:> import re 2024-01-15T17:55:37.514 DEBUG:teuthology.orchestra.run.smithi124:> import os 2024-01-15T17:55:37.514 DEBUG:teuthology.orchestra.run.smithi124:> import subprocess 2024-01-15T17:55:37.514 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:55:37.514 DEBUG:teuthology.orchestra.run.smithi124:> def _find_admin_socket(client_name): 2024-01-15T17:55:37.514 DEBUG:teuthology.orchestra.run.smithi124:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-01-15T17:55:37.514 DEBUG:teuthology.orchestra.run.smithi124:> files = glob.glob(asok_path) 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> # Given a non-glob path, it better be there 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> if "*" not in asok_path: 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> assert(len(files) == 1) 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> return files[0] 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> for f in files: 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> contents = proc_f.read() 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> if mountpoint in contents: 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> return f 2024-01-15T17:55:37.515 DEBUG:teuthology.orchestra.run.smithi124:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T17:55:37.516 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:55:37.516 DEBUG:teuthology.orchestra.run.smithi124:> print(_find_admin_socket("client.3")) 2024-01-15T17:55:37.516 DEBUG:teuthology.orchestra.run.smithi124:> ' 2024-01-15T17:55:37.713 INFO:teuthology.orchestra.run.smithi124.stdout:/var/run/ceph/ceph-client.3.42402.asok 2024-01-15T17:55:37.717 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:55:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T17:55:37.730 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.42402.asok 2024-01-15T17:55:37.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:37.730 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.42402.asok status 2024-01-15T17:55:37.970 INFO:teuthology.orchestra.run.smithi124.stdout:{ 2024-01-15T17:55:37.970 INFO:teuthology.orchestra.run.smithi124.stdout: "metadata": { 2024-01-15T17:55:37.970 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T17:55:37.970 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T17:55:37.970 INFO:teuthology.orchestra.run.smithi124.stdout: "entity_id": "3", 2024-01-15T17:55:37.970 INFO:teuthology.orchestra.run.smithi124.stdout: "hostname": "smithi124", 2024-01-15T17:55:37.970 INFO:teuthology.orchestra.run.smithi124.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-01-15T17:55:37.970 INFO:teuthology.orchestra.run.smithi124.stdout: "pid": "42402", 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: "root": "/" 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_count": 0, 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_pinned_count": 0, 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: "id": 34406, 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: "inst": { 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: "name": { 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "client", 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: "num": 34406 2024-01-15T17:55:37.971 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.2:0", 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 3844022589 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: } 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.2:0", 2024-01-15T17:55:37.972 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 3844022589 2024-01-15T17:55:37.973 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:55:37.973 INFO:teuthology.orchestra.run.smithi124.stdout: "inst_str": "client.34406 192.168.0.2:0/3844022589", 2024-01-15T17:55:37.973 INFO:teuthology.orchestra.run.smithi124.stdout: "addr_str": "192.168.0.2:0/3844022589", 2024-01-15T17:55:37.973 INFO:teuthology.orchestra.run.smithi124.stdout: "inode_count": 1, 2024-01-15T17:55:37.973 INFO:teuthology.orchestra.run.smithi124.stdout: "mds_epoch": 7, 2024-01-15T17:55:37.973 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch": 22, 2024-01-15T17:55:37.973 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch_barrier": 0, 2024-01-15T17:55:37.974 INFO:teuthology.orchestra.run.smithi124.stdout: "blocklisted": false, 2024-01-15T17:55:37.975 INFO:teuthology.orchestra.run.smithi124.stdout: "fs_name": "cephfs" 2024-01-15T17:55:37.975 INFO:teuthology.orchestra.run.smithi124.stdout:} 2024-01-15T17:55:37.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:37.985 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-01-15T17:55:37.985 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:38.046 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-01-15T17:55:38.047 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:38.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:38.047 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:38.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:38.152 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-01-15T17:55:38.152 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.2 2024-01-15T17:55:38.208 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-01-15T17:55:38.209 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-01-15T17:55:38.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:38.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-01-15T17:55:38.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:38.318 DEBUG:teuthology.orchestra.run.smithi124:> stat --file-system '--printf=%T 2024-01-15T17:55:38.318 DEBUG:teuthology.orchestra.run.smithi124:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:38.373 INFO:teuthology.orchestra.run.smithi124.stdout:fuseblk 2024-01-15T17:55:38.373 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:38.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:38.374 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:38.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:38.470 DEBUG:teuthology.orchestra.run.smithi124:> stat --file-system '--printf=%T 2024-01-15T17:55:38.470 DEBUG:teuthology.orchestra.run.smithi124:> ' -- /home/ubuntu/cephtest/mnt.3 2024-01-15T17:55:38.525 INFO:teuthology.orchestra.run.smithi124.stdout:fuseblk 2024-01-15T17:55:38.525 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-01-15T17:55:38.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:38.525 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-01-15T17:55:38.621 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-01-15T17:55:38.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.859+0000 7f9b85e00640 1 Processor -- start 2024-01-15T17:55:38.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.859+0000 7f9b85e00640 1 -- start start 2024-01-15T17:55:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.861+0000 7f9b85e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9b8006aaf0 0x7f9b800b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.861+0000 7f9b85e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b800ad170 0x7f9b800ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:38.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.861+0000 7f9b85e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8006fc40 con 0x7f9b80154220 2024-01-15T17:55:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.861+0000 7f9b85e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b80154600 con 0x7f9b8006aaf0 2024-01-15T17:55:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.861+0000 7f9b85e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80160d70 con 0x7f9b800ad170 2024-01-15T17:55:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.861+0000 7f9b84a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9b80154220 0x7f9b8006a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55638/0 (socket says 172.21.15.26:55638) 2024-01-15T17:55:38.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.861+0000 7f9b84a00640 1 -- 172.21.15.26:0/2062568546 learned_addr learned my addr 172.21.15.26:0/2062568546 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242708816 0 0) 0x7f9b8006fc40 con 0x7f9b80154220 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b60003520 con 0x7f9b80154220 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7fe00640 1 --2- 172.21.15.26:0/2062568546 >> v2:172.21.15.124:3300/0 conn(0x7f9b8006aaf0 0x7f9b800b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b85400640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b800ad170 0x7f9b800ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1864046976 0 0) 0x7f9b60003520 con 0x7f9b80154220 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b800ad170 msgr2=0x7f9b800ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b800ad170 0x7f9b800ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 >> v2:172.21.15.124:3300/0 conn(0x7f9b8006aaf0 msgr2=0x7f9b800b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 --2- 172.21.15.26:0/2062568546 >> v2:172.21.15.124:3300/0 conn(0x7f9b8006aaf0 0x7f9b800b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b80161050 con 0x7f9b80154220 2024-01-15T17:55:38.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9b70001690 con 0x7f9b80154220 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 >> v1:172.21.15.26:6789/0 conn(0x7f9b80154220 legacy=0x7f9b8006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b800ad170 0x7f9b60003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60003f40 0x7f9b600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60004d00 0x7f9b60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80154600 con 0x7f9b60004d00 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b80160d70 con 0x7f9b60003f40 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.862+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6000ba30 con 0x7f9b800ad170 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.863+0000 7f9b85400640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60004d00 0x7f9b60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.863+0000 7f9b7fe00640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60003f40 0x7f9b600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.863+0000 7f9b84a00640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b800ad170 0x7f9b60003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.863+0000 7f9b85400640 1 -- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b800ad170 msgr2=0x7f9b60003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.863+0000 7f9b85400640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b800ad170 0x7f9b60003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.863+0000 7f9b85400640 1 -- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60003f40 msgr2=0x7f9b600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:38.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.863+0000 7f9b85400640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60003f40 0x7f9b600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.863+0000 7f9b85400640 1 -- 172.21.15.26:0/2062568546 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60003520 con 0x7f9b60004d00 2024-01-15T17:55:38.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.863+0000 7f9b84a00640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b800ad170 0x7f9b60003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.864+0000 7f9b85400640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60004d00 0x7f9b60005180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b78007360 tx=0x7f9b600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e668bbfab27b4463 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7802ccb0 con 0x7f9b60004d00 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7802ce10 con 0x7f9b60004d00 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b7f400640 1 -- 172.21.15.26:0/2062568546 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78034660 con 0x7f9b60004d00 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b85e00640 1 -- 172.21.15.26:0/2062568546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60004d00 msgr2=0x7f9b60005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b85e00640 1 --2- 172.21.15.26:0/2062568546 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b60004d00 0x7f9b60005180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9b78007360 tx=0x7f9b600057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b85400640 1 -- 172.21.15.26:0/2062568546 reap_dead start 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b85e00640 1 -- 172.21.15.26:0/2062568546 shutdown_connections 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b85e00640 1 -- 172.21.15.26:0/2062568546 >> 172.21.15.26:0/2062568546 conn(0x7f9b800b1a50 msgr2=0x7f9b800b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b85e00640 1 -- 172.21.15.26:0/2062568546 shutdown_connections 2024-01-15T17:55:38.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.865+0000 7f9b85e00640 1 -- 172.21.15.26:0/2062568546 wait complete. 2024-01-15T17:55:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.867+0000 7f9b85e00640 1 Processor -- start 2024-01-15T17:55:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.867+0000 7f9b85e00640 1 -- start start 2024-01-15T17:55:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.868+0000 7f9b85e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60004d00 0x7f9b80150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.868+0000 7f9b85e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b801412d0 0x7f9b80141730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:38.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.868+0000 7f9b85e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b8015f3b0 0x7f9b8015f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.868+0000 7f9b85e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8015e4c0 con 0x7f9b801412d0 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.868+0000 7f9b85e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8015e8e0 con 0x7f9b60004d00 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.868+0000 7f9b85e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8015d880 con 0x7f9b8015f3b0 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.868+0000 7f9b7fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b801412d0 0x7f9b80141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b7fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b801412d0 0x7f9b80141730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44396/0 (socket says 172.21.15.26:44396) 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b7fe00640 1 -- 172.21.15.26:0/2659573824 learned_addr learned my addr 172.21.15.26:0/2659573824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b84a00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60004d00 0x7f9b80150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b7fe00640 1 -- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b8015f3b0 msgr2=0x7f9b8015f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b7fe00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b8015f3b0 0x7f9b8015f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b7fe00640 1 -- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60004d00 msgr2=0x7f9b80150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b7fe00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60004d00 0x7f9b80150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b7fe00640 1 -- 172.21.15.26:0/2659573824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b80141c70 con 0x7f9b801412d0 2024-01-15T17:55:38.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b84a00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60004d00 0x7f9b80150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:38.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.869+0000 7f9b7fe00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b801412d0 0x7f9b80141730 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9b680115b0 tx=0x7f9b68011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.870+0000 7f9b7cc00640 1 -- 172.21.15.26:0/2659573824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68015870 con 0x7f9b801412d0 2024-01-15T17:55:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.870+0000 7f9b7cc00640 1 -- 172.21.15.26:0/2659573824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b680159d0 con 0x7f9b801412d0 2024-01-15T17:55:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.870+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8015db20 con 0x7f9b801412d0 2024-01-15T17:55:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.870+0000 7f9b7cc00640 1 -- 172.21.15.26:0/2659573824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b68015d40 con 0x7f9b801412d0 2024-01-15T17:55:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.870+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8015e0e0 con 0x7f9b801412d0 2024-01-15T17:55:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.871+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b80074a00 con 0x7f9b801412d0 2024-01-15T17:55:38.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.872+0000 7f9b7cc00640 1 -- 172.21.15.26:0/2659573824 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9b6800f480 con 0x7f9b801412d0 2024-01-15T17:55:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.873+0000 7f9b7cc00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5806b2c0 0x7f9b5806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:38.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.874+0000 7f9b7cc00640 1 -- 172.21.15.26:0/2659573824 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b6808a3e0 con 0x7f9b801412d0 2024-01-15T17:55:38.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.874+0000 7f9b84a00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5806b2c0 0x7f9b5806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:38.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.875+0000 7f9b84a00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5806b2c0 0x7f9b5806d780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b700017c0 tx=0x7f9b70009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:38.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:38.878+0000 7f9b7cc00640 1 -- 172.21.15.26:0/2659573824 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b68055240 con 0x7f9b801412d0 2024-01-15T17:55:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.079+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b8015e600 con 0x7f9b801412d0 2024-01-15T17:55:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.080+0000 7f9b7cc00640 1 -- 172.21.15.26:0/2659573824 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f9b8015e600 con 0x7f9b801412d0 2024-01-15T17:55:39.084 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:39.085 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5806b2c0 msgr2=0x7f9b5806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5806b2c0 0x7f9b5806d780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b700017c0 tx=0x7f9b70009040 comp rx=0 tx=0).stop 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b801412d0 msgr2=0x7f9b80141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b801412d0 0x7f9b80141730 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9b680115b0 tx=0x7f9b68011a70 comp rx=0 tx=0).stop 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 shutdown_connections 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b8015f3b0 0x7f9b8015f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5806b2c0 0x7f9b5806d780 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b801412d0 0x7f9b80141730 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 --2- 172.21.15.26:0/2659573824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b60004d00 0x7f9b80150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 >> 172.21.15.26:0/2659573824 conn(0x7f9b800b1a50 msgr2=0x7f9b80071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 shutdown_connections 2024-01-15T17:55:39.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.083+0000 7f9b85e00640 1 -- 172.21.15.26:0/2659573824 wait complete. 2024-01-15T17:55:39.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:39.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:39.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.334+0000 7fd290c00640 1 Processor -- start 2024-01-15T17:55:39.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.335+0000 7fd290c00640 1 -- start start 2024-01-15T17:55:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.336+0000 7fd290c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c15b7a0 0x7fd28c0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.336+0000 7fd290c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28c0b2130 con 0x7fd28c0cfec0 2024-01-15T17:55:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.336+0000 7fd290c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28c144f50 con 0x7fd28c144b70 2024-01-15T17:55:39.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.336+0000 7fd290c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c15bc00 con 0x7fd28c15b7a0 2024-01-15T17:55:39.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd28a800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd28c0cfec0 0x7fd28c0ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55652/0 (socket says 172.21.15.26:55652) 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd28a800640 1 -- 172.21.15.26:0/522432086 learned_addr learned my addr 172.21.15.26:0/522432086 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd289400640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c15b7a0 0x7fd28c0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312276883 0 0) 0x7fd28c0b2130 con 0x7fd28c0cfec0 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd26c003520 con 0x7fd28c0cfec0 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 45228050 0 0) 0x7fd26c003520 con 0x7fd28c0cfec0 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c15b7a0 msgr2=0x7fd28c0b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd288a00640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c15b7a0 0x7fd28c0b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 >> v1:172.21.15.124:6789/0 conn(0x7fd28c144b70 legacy=0x7fd28c15f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c15edc0 con 0x7fd28c0cfec0 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.337+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd284001670 con 0x7fd28c0cfec0 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 >> v1:172.21.15.26:6789/0 conn(0x7fd28c0cfec0 legacy=0x7fd28c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd288a00640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26c0039a0 0x7fd26c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd288a00640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c15b7a0 0x7fd26c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd288a00640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c004d00 0x7fd26c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c15bc00 con 0x7fd26c0039a0 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c00b980 con 0x7fd26c004d00 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26c00bb40 con 0x7fd28c15b7a0 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd289e00640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26c0039a0 0x7fd26c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd289400640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c15b7a0 0x7fd26c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd289e00640 1 -- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c15b7a0 msgr2=0x7fd26c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd28a800640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c004d00 0x7fd26c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd289e00640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c15b7a0 0x7fd26c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd289e00640 1 -- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c004d00 msgr2=0x7fd26c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:39.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd289e00640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c004d00 0x7fd26c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd289e00640 1 -- 172.21.15.26:0/522432086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c003520 con 0x7fd26c0039a0 2024-01-15T17:55:39.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.338+0000 7fd28a800640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c004d00 0x7fd26c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.340+0000 7fd289e00640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26c0039a0 0x7fd26c003d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd27c001860 tx=0x7fd26c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b56a3397c9361e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.340+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c033640 con 0x7fd26c0039a0 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.340+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27c0337c0 con 0x7fd26c0039a0 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.340+0000 7fd288a00640 1 -- 172.21.15.26:0/522432086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27c033bb0 con 0x7fd26c0039a0 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.341+0000 7fd290c00640 1 -- 172.21.15.26:0/522432086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26c0039a0 msgr2=0x7fd26c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.341+0000 7fd290c00640 1 --2- 172.21.15.26:0/522432086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd26c0039a0 0x7fd26c003d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd27c001860 tx=0x7fd26c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.341+0000 7fd28a800640 1 -- 172.21.15.26:0/522432086 reap_dead start 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.341+0000 7fd290c00640 1 -- 172.21.15.26:0/522432086 shutdown_connections 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.341+0000 7fd290c00640 1 -- 172.21.15.26:0/522432086 >> 172.21.15.26:0/522432086 conn(0x7fd28c06b4f0 msgr2=0x7fd28c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.341+0000 7fd290c00640 1 -- 172.21.15.26:0/522432086 shutdown_connections 2024-01-15T17:55:39.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.341+0000 7fd290c00640 1 -- 172.21.15.26:0/522432086 wait complete. 2024-01-15T17:55:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.343+0000 7fd290c00640 1 Processor -- start 2024-01-15T17:55:39.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.343+0000 7fd290c00640 1 -- start start 2024-01-15T17:55:39.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd290c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c0039a0 0x7fd28c13ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd290c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c157490 0x7fd28c13ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd290c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd28c13bce0 0x7fd28c13c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd290c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c13a430 con 0x7fd28c13bce0 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd290c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c13a710 con 0x7fd26c0039a0 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd290c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c13a9f0 con 0x7fd28c157490 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd28a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd28c13bce0 0x7fd28c13c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd28a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd28c13bce0 0x7fd28c13c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44410/0 (socket says 172.21.15.26:44410) 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd28a800640 1 -- 172.21.15.26:0/1909170300 learned_addr learned my addr 172.21.15.26:0/1909170300 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd289400640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c157490 0x7fd28c13ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd289e00640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c0039a0 0x7fd28c13ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd28a800640 1 -- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c157490 msgr2=0x7fd28c13ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd28a800640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c157490 0x7fd28c13ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd28a800640 1 -- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c0039a0 msgr2=0x7fd28c13ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd28a800640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c0039a0 0x7fd28c13ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd28a800640 1 -- 172.21.15.26:0/1909170300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c13acd0 con 0x7fd28c13bce0 2024-01-15T17:55:39.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.344+0000 7fd289400640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c157490 0x7fd28c13ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:39.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.345+0000 7fd28a800640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd28c13bce0 0x7fd28c13c160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0cda50 tx=0x7fd28400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:39.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.345+0000 7fd282000640 1 -- 172.21.15.26:0/1909170300 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284007440 con 0x7fd28c13bce0 2024-01-15T17:55:39.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.345+0000 7fd282000640 1 -- 172.21.15.26:0/1909170300 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2840075c0 con 0x7fd28c13bce0 2024-01-15T17:55:39.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.345+0000 7fd282000640 1 -- 172.21.15.26:0/1909170300 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284004a20 con 0x7fd28c13bce0 2024-01-15T17:55:39.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.345+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28c13afb0 con 0x7fd28c13bce0 2024-01-15T17:55:39.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.345+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28c0afbd0 con 0x7fd28c13bce0 2024-01-15T17:55:39.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.346+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28c13b1a0 con 0x7fd28c13bce0 2024-01-15T17:55:39.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.347+0000 7fd282000640 1 -- 172.21.15.26:0/1909170300 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd284004b80 con 0x7fd28c13bce0 2024-01-15T17:55:39.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.348+0000 7fd282000640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd26406b2c0 0x7fd26406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.348+0000 7fd282000640 1 -- 172.21.15.26:0/1909170300 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd284089920 con 0x7fd28c13bce0 2024-01-15T17:55:39.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.349+0000 7fd289e00640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd26406b2c0 0x7fd26406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.350+0000 7fd289e00640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd26406b2c0 0x7fd26406d780 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd27c035000 tx=0x7fd27c005040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:39.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.353+0000 7fd282000640 1 -- 172.21.15.26:0/1909170300 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2840546f0 con 0x7fd28c13bce0 2024-01-15T17:55:39.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.546+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd28c0675d0 con 0x7fd28c13bce0 2024-01-15T17:55:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.547+0000 7fd282000640 1 -- 172.21.15.26:0/1909170300 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd284057d10 con 0x7fd28c13bce0 2024-01-15T17:55:39.551 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd26406b2c0 msgr2=0x7fd26406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd26406b2c0 0x7fd26406d780 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd27c035000 tx=0x7fd27c005040 comp rx=0 tx=0).stop 2024-01-15T17:55:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd28c13bce0 msgr2=0x7fd28c13c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd28c13bce0 0x7fd28c13c160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0cda50 tx=0x7fd28400d9d0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 shutdown_connections 2024-01-15T17:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd28c157490 0x7fd28c13ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd26406b2c0 0x7fd26406d780 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd28c13bce0 0x7fd28c13c160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 --2- 172.21.15.26:0/1909170300 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd26c0039a0 0x7fd28c13ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 >> 172.21.15.26:0/1909170300 conn(0x7fd28c06b4f0 msgr2=0x7fd28c15dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 shutdown_connections 2024-01-15T17:55:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.550+0000 7fd290c00640 1 -- 172.21.15.26:0/1909170300 wait complete. 2024-01-15T17:55:39.574 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.798+0000 7fbd8b800640 1 Processor -- start 2024-01-15T17:55:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.798+0000 7fbd8b800640 1 -- start start 2024-01-15T17:55:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.799+0000 7fbd8b800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840d3910 0x7fbd840d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.800+0000 7fbd8b800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbd84160e10 0x7fbd840d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.800+0000 7fbd8b800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd840d4160 con 0x7fbd840cfcf0 2024-01-15T17:55:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.800+0000 7fbd8b800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd840ceff0 con 0x7fbd84160e10 2024-01-15T17:55:39.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.800+0000 7fbd8b800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840cf410 con 0x7fbd840d3910 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.800+0000 7fbd89800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbd84160e10 0x7fbd840d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.800+0000 7fbd83e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840d3910 0x7fbd840d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.800+0000 7fbd83e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840d3910 0x7fbd840d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:50846/0 (socket says 172.21.15.26:50846) 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.800+0000 7fbd83e00640 1 -- 172.21.15.26:0/405829880 learned_addr learned my addr 172.21.15.26:0/405829880 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.800+0000 7fbd88e00640 1 --1- 172.21.15.26:0/405829880 >> v1:172.21.15.124:6789/0 conn(0x7fbd840cfcf0 0x7fbd84160700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:55924/0 (socket says 172.21.15.26:55924) 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.801+0000 7fbd83e00640 1 -- 172.21.15.26:0/405829880 >> v2:172.21.15.124:3300/0 conn(0x7fbd84160e10 msgr2=0x7fbd840d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.801+0000 7fbd83e00640 1 --2- 172.21.15.26:0/405829880 >> v2:172.21.15.124:3300/0 conn(0x7fbd84160e10 0x7fbd840d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.801+0000 7fbd83e00640 1 -- 172.21.15.26:0/405829880 >> v1:172.21.15.124:6789/0 conn(0x7fbd840cfcf0 legacy=0x7fbd84160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.801+0000 7fbd83e00640 1 -- 172.21.15.26:0/405829880 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd840b2420 con 0x7fbd840d3910 2024-01-15T17:55:39.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.801+0000 7fbd89800640 1 --2- 172.21.15.26:0/405829880 >> v2:172.21.15.124:3300/0 conn(0x7fbd84160e10 0x7fbd840d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:55:39.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.802+0000 7fbd83e00640 1 --2- 172.21.15.26:0/405829880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840d3910 0x7fbd840d1100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009870 tx=0x7fbd6c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80d8f128421f7d45 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.803+0000 7fbd83400640 1 -- 172.21.15.26:0/405829880 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c031640 con 0x7fbd840d3910 2024-01-15T17:55:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.803+0000 7fbd83400640 1 -- 172.21.15.26:0/405829880 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c0317a0 con 0x7fbd840d3910 2024-01-15T17:55:39.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.803+0000 7fbd83400640 1 -- 172.21.15.26:0/405829880 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c031b30 con 0x7fbd840d3910 2024-01-15T17:55:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.804+0000 7fbd8b800640 1 -- 172.21.15.26:0/405829880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840d3910 msgr2=0x7fbd840d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.804+0000 7fbd8b800640 1 --2- 172.21.15.26:0/405829880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840d3910 0x7fbd840d1100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c009870 tx=0x7fbd6c02b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.804+0000 7fbd8b800640 1 -- 172.21.15.26:0/405829880 shutdown_connections 2024-01-15T17:55:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.804+0000 7fbd8b800640 1 --2- 172.21.15.26:0/405829880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840d3910 0x7fbd840d1100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.804+0000 7fbd8b800640 1 --2- 172.21.15.26:0/405829880 >> v2:172.21.15.124:3300/0 conn(0x7fbd84160e10 0x7fbd840d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.804+0000 7fbd8b800640 1 -- 172.21.15.26:0/405829880 >> 172.21.15.26:0/405829880 conn(0x7fbd8409c530 msgr2=0x7fbd840d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.804+0000 7fbd8b800640 1 -- 172.21.15.26:0/405829880 shutdown_connections 2024-01-15T17:55:39.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.804+0000 7fbd8b800640 1 -- 172.21.15.26:0/405829880 wait complete. 2024-01-15T17:55:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.806+0000 7fbd8b800640 1 Processor -- start 2024-01-15T17:55:39.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.806+0000 7fbd8b800640 1 -- start start 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.807+0000 7fbd8b800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840cfcf0 0x7fbd8414a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.807+0000 7fbd8b800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840d3910 0x7fbd840ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.807+0000 7fbd8b800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd84160e10 0x7fbd840ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.807+0000 7fbd8b800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8414d460 con 0x7fbd840d3910 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.807+0000 7fbd8b800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840cd250 con 0x7fbd84160e10 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.807+0000 7fbd8b800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8414cb30 con 0x7fbd840cfcf0 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.807+0000 7fbd83e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840d3910 0x7fbd840ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.807+0000 7fbd83e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840d3910 0x7fbd840ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44424/0 (socket says 172.21.15.26:44424) 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.807+0000 7fbd83e00640 1 -- 172.21.15.26:0/930555728 learned_addr learned my addr 172.21.15.26:0/930555728 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.808+0000 7fbd88e00640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840cfcf0 0x7fbd8414a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.808+0000 7fbd89800640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd84160e10 0x7fbd840ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.808+0000 7fbd83e00640 1 -- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840cfcf0 msgr2=0x7fbd8414a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.808+0000 7fbd83e00640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840cfcf0 0x7fbd8414a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.808+0000 7fbd83e00640 1 -- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd84160e10 msgr2=0x7fbd840ca810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:39.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.808+0000 7fbd83e00640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd84160e10 0x7fbd840ca810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:39.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.808+0000 7fbd83e00640 1 -- 172.21.15.26:0/930555728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c009a70 con 0x7fbd840d3910 2024-01-15T17:55:39.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.808+0000 7fbd83e00640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840d3910 0x7fbd840ca2d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c000c00 tx=0x7fbd6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.809+0000 7fbd80c00640 1 -- 172.21.15.26:0/930555728 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c03a9f0 con 0x7fbd840d3910 2024-01-15T17:55:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.809+0000 7fbd80c00640 1 -- 172.21.15.26:0/930555728 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c03ab50 con 0x7fbd840d3910 2024-01-15T17:55:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.809+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c0094d0 con 0x7fbd840d3910 2024-01-15T17:55:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.809+0000 7fbd80c00640 1 -- 172.21.15.26:0/930555728 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0443f0 con 0x7fbd840d3910 2024-01-15T17:55:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.809+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8414d0f0 con 0x7fbd840d3910 2024-01-15T17:55:39.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.810+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd840675d0 con 0x7fbd840d3910 2024-01-15T17:55:39.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.812+0000 7fbd80c00640 1 -- 172.21.15.26:0/930555728 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbd6c02f050 con 0x7fbd840d3910 2024-01-15T17:55:39.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.813+0000 7fbd80c00640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd5c06b2f0 0x7fbd5c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:39.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.813+0000 7fbd80c00640 1 -- 172.21.15.26:0/930555728 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd6c0abab0 con 0x7fbd840d3910 2024-01-15T17:55:39.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.813+0000 7fbd88e00640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd5c06b2f0 0x7fbd5c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:39.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.815+0000 7fbd88e00640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd5c06b2f0 0x7fbd5c06d7b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd74000f60 tx=0x7fbd74007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:39.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:39.818+0000 7fbd80c00640 1 -- 172.21.15.26:0/930555728 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd6c0767f0 con 0x7fbd840d3910 2024-01-15T17:55:40.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.017+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd8415fad0 con 0x7fbd840d3910 2024-01-15T17:55:40.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.018+0000 7fbd80c00640 1 -- 172.21.15.26:0/930555728 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd6c035020 con 0x7fbd840d3910 2024-01-15T17:55:40.022 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd5c06b2f0 msgr2=0x7fbd5c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd5c06b2f0 0x7fbd5c06d7b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbd74000f60 tx=0x7fbd74007cc0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840d3910 msgr2=0x7fbd840ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840d3910 0x7fbd840ca2d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c000c00 tx=0x7fbd6c0046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 shutdown_connections 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd5c06b2f0 0x7fbd5c06d7b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd84160e10 0x7fbd840ca810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd840d3910 0x7fbd840ca2d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 --2- 172.21.15.26:0/930555728 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd840cfcf0 0x7fbd8414a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 >> 172.21.15.26:0/930555728 conn(0x7fbd8409c530 msgr2=0x7fbd8406b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.020+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 shutdown_connections 2024-01-15T17:55:40.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.021+0000 7fbd8b800640 1 -- 172.21.15.26:0/930555728 wait complete. 2024-01-15T17:55:40.044 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.268+0000 7f27cbe00640 1 Processor -- start 2024-01-15T17:55:40.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.268+0000 7f27cbe00640 1 -- start start 2024-01-15T17:55:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.270+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 0x7f27cc0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.270+0000 7f27cbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f27cc0cf9a0 0x7f27cc0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.270+0000 7f27cbe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f27cc0cddf0 0x7f27cc0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.270+0000 7f27cbe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc0d0320 con 0x7f27cc0cf9a0 2024-01-15T17:55:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.270+0000 7f27cbe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc0b2730 con 0x7f27cc0cddf0 2024-01-15T17:55:40.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.270+0000 7f27cbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0d1ec0 con 0x7f27cc15d4b0 2024-01-15T17:55:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.271+0000 7f27caa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 0x7f27cc0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.271+0000 7f27caa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 0x7f27cc0d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:50856/0 (socket says 172.21.15.26:50856) 2024-01-15T17:55:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.271+0000 7f27caa00640 1 -- 172.21.15.26:0/3260845190 learned_addr learned my addr 172.21.15.26:0/3260845190 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:40.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.271+0000 7f27ca000640 1 --2- 172.21.15.26:0/3260845190 >> v2:172.21.15.26:3300/0 conn(0x7f27cc0cf9a0 0x7f27cc0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.272+0000 7f27cb400640 1 --2- 172.21.15.26:0/3260845190 >> v2:172.21.15.124:3300/0 conn(0x7f27cc0cddf0 0x7f27cc0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.272+0000 7f27ca000640 1 -- 172.21.15.26:0/3260845190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 msgr2=0x7f27cc0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.272+0000 7f27ca000640 1 --2- 172.21.15.26:0/3260845190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 0x7f27cc0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.272+0000 7f27ca000640 1 -- 172.21.15.26:0/3260845190 >> v2:172.21.15.124:3300/0 conn(0x7f27cc0cddf0 msgr2=0x7f27cc0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.272+0000 7f27ca000640 1 --2- 172.21.15.26:0/3260845190 >> v2:172.21.15.124:3300/0 conn(0x7f27cc0cddf0 0x7f27cc0ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.272+0000 7f27ca000640 1 -- 172.21.15.26:0/3260845190 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0b29d0 con 0x7f27cc0cf9a0 2024-01-15T17:55:40.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.273+0000 7f27ca000640 1 --2- 172.21.15.26:0/3260845190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cf9a0 0x7f27cc0cfde0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f27b40097a0 tx=0x7f27b402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faa5be294447d39 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:40.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.273+0000 7f27c9600640 1 -- 172.21.15.26:0/3260845190 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4031780 con 0x7f27cc0cf9a0 2024-01-15T17:55:40.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.273+0000 7f27c9600640 1 -- 172.21.15.26:0/3260845190 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b40318e0 con 0x7f27cc0cf9a0 2024-01-15T17:55:40.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.273+0000 7f27c9600640 1 -- 172.21.15.26:0/3260845190 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4031c70 con 0x7f27cc0cf9a0 2024-01-15T17:55:40.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.274+0000 7f27cbe00640 1 -- 172.21.15.26:0/3260845190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cf9a0 msgr2=0x7f27cc0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.274+0000 7f27cbe00640 1 --2- 172.21.15.26:0/3260845190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cf9a0 0x7f27cc0cfde0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f27b40097a0 tx=0x7f27b402b820 comp rx=0 tx=0).stop 2024-01-15T17:55:40.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.274+0000 7f27cbe00640 1 -- 172.21.15.26:0/3260845190 shutdown_connections 2024-01-15T17:55:40.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.274+0000 7f27cbe00640 1 --2- 172.21.15.26:0/3260845190 >> v2:172.21.15.124:3300/0 conn(0x7f27cc0cddf0 0x7f27cc0ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.274+0000 7f27cbe00640 1 --2- 172.21.15.26:0/3260845190 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cf9a0 0x7f27cc0cfde0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.274+0000 7f27cbe00640 1 --2- 172.21.15.26:0/3260845190 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 0x7f27cc0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.274+0000 7f27cbe00640 1 -- 172.21.15.26:0/3260845190 >> 172.21.15.26:0/3260845190 conn(0x7f27cc0d3000 msgr2=0x7f27cc0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.274+0000 7f27cbe00640 1 -- 172.21.15.26:0/3260845190 shutdown_connections 2024-01-15T17:55:40.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.275+0000 7f27cbe00640 1 -- 172.21.15.26:0/3260845190 wait complete. 2024-01-15T17:55:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.276+0000 7f27cbe00640 1 Processor -- start 2024-01-15T17:55:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.277+0000 7f27cbe00640 1 -- start start 2024-01-15T17:55:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.277+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cddf0 0x7f27cc14d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.277+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27cc0cf9a0 0x7f27cc0cab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27cbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 0x7f27cc15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27cbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc14daf0 con 0x7f27cc0cddf0 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27cbe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0ca0d0 con 0x7f27cc0cf9a0 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27cbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc1609f0 con 0x7f27cc15d4b0 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27caa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cddf0 0x7f27cc14d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27caa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cddf0 0x7f27cc14d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44448/0 (socket says 172.21.15.26:44448) 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27caa00640 1 -- 172.21.15.26:0/2424159184 learned_addr learned my addr 172.21.15.26:0/2424159184 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27ca000640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27cc0cf9a0 0x7f27cc0cab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27cb400640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 0x7f27cc15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27caa00640 1 -- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 msgr2=0x7f27cc15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27caa00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 0x7f27cc15dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27caa00640 1 -- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27cc0cf9a0 msgr2=0x7f27cc0cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27caa00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27cc0cf9a0 0x7f27cc0cab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27caa00640 1 -- 172.21.15.26:0/2424159184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b4009400 con 0x7f27cc0cddf0 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.278+0000 7f27ca000640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27cc0cf9a0 0x7f27cc0cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:40.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.279+0000 7f27caa00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cddf0 0x7f27cc14d5b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00e840 tx=0x7f27bc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.279+0000 7f27c2a00640 1 -- 172.21.15.26:0/2424159184 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00c8e0 con 0x7f27cc0cddf0 2024-01-15T17:55:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.279+0000 7f27c2a00640 1 -- 172.21.15.26:0/2424159184 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc00ca40 con 0x7f27cc0cddf0 2024-01-15T17:55:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.279+0000 7f27c2a00640 1 -- 172.21.15.26:0/2424159184 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00cdf0 con 0x7f27cc0cddf0 2024-01-15T17:55:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.279+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc15da10 con 0x7f27cc0cddf0 2024-01-15T17:55:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.279+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc0ca390 con 0x7f27cc0cddf0 2024-01-15T17:55:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.281+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc067450 con 0x7f27cc0cddf0 2024-01-15T17:55:40.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.281+0000 7f27c2a00640 1 -- 172.21.15.26:0/2424159184 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f27bc013470 con 0x7f27cc0cddf0 2024-01-15T17:55:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.282+0000 7f27c2a00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f27a406b2c0 0x7f27a406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.282+0000 7f27c2a00640 1 -- 172.21.15.26:0/2424159184 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27bc089cc0 con 0x7f27cc0cddf0 2024-01-15T17:55:40.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.283+0000 7f27ca000640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f27a406b2c0 0x7f27a406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.284+0000 7f27ca000640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f27a406b2c0 0x7f27a406d780 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27b4036000 tx=0x7f27b403d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.287+0000 7f27c2a00640 1 -- 172.21.15.26:0/2424159184 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27bc054b30 con 0x7f27cc0cddf0 2024-01-15T17:55:40.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.483+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27cc0b2040 con 0x7f27cc0cddf0 2024-01-15T17:55:40.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.484+0000 7f27c2a00640 1 -- 172.21.15.26:0/2424159184 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27bc0170a0 con 0x7f27cc0cddf0 2024-01-15T17:55:40.488 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:40.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f27a406b2c0 msgr2=0x7f27a406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f27a406b2c0 0x7f27a406d780 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f27b4036000 tx=0x7f27b403d040 comp rx=0 tx=0).stop 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cddf0 msgr2=0x7f27cc14d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cddf0 0x7f27cc14d5b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00e840 tx=0x7f27bc00ed00 comp rx=0 tx=0).stop 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 shutdown_connections 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27cc15d4b0 0x7f27cc15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f27a406b2c0 0x7f27a406d780 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27cc0cf9a0 0x7f27cc0cab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 --2- 172.21.15.26:0/2424159184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27cc0cddf0 0x7f27cc14d5b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 >> 172.21.15.26:0/2424159184 conn(0x7f27cc0d3000 msgr2=0x7f27cc0b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 shutdown_connections 2024-01-15T17:55:40.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.487+0000 7f27cbe00640 1 -- 172.21.15.26:0/2424159184 wait complete. 2024-01-15T17:55:40.511 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.725+0000 7f8b0d000640 1 Processor -- start 2024-01-15T17:55:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.726+0000 7f8b0d000640 1 -- start start 2024-01-15T17:55:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.727+0000 7f8b0d000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813cfa0 0x7f8b0815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.727+0000 7f8b0d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b0813fcb0 0x7f8b080b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.727+0000 7f8b0d000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b080cdec0 con 0x7f8b080cd530 2024-01-15T17:55:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.727+0000 7f8b0d000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b080ce1a0 con 0x7f8b0813fcb0 2024-01-15T17:55:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.727+0000 7f8b0d000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b080ce480 con 0x7f8b0813cfa0 2024-01-15T17:55:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.727+0000 7f8b07e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b0813fcb0 0x7f8b080b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.727+0000 7f8b07e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b0813fcb0 0x7f8b080b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44456/0 (socket says 172.21.15.26:44456) 2024-01-15T17:55:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.727+0000 7f8b07e00640 1 -- 172.21.15.26:0/3705325151 learned_addr learned my addr 172.21.15.26:0/3705325151 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.728+0000 7f8b07400640 1 --2- 172.21.15.26:0/3705325151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813cfa0 0x7f8b0815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.728+0000 7f8b07e00640 1 -- 172.21.15.26:0/3705325151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813cfa0 msgr2=0x7f8b0815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.728+0000 7f8b07e00640 1 --2- 172.21.15.26:0/3705325151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813cfa0 0x7f8b0815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.728+0000 7f8b07e00640 1 -- 172.21.15.26:0/3705325151 >> v1:172.21.15.124:6789/0 conn(0x7f8b080cd530 legacy=0x7f8b0813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.728+0000 7f8b07e00640 1 -- 172.21.15.26:0/3705325151 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b080ce760 con 0x7f8b0813fcb0 2024-01-15T17:55:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.728+0000 7f8b07400640 1 --2- 172.21.15.26:0/3705325151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813cfa0 0x7f8b0815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.729+0000 7f8b07e00640 1 --2- 172.21.15.26:0/3705325151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813fcb0 0x7f8b080b2040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009e50 tx=0x7f8b0002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12080fbbd662147e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.729+0000 7f8b06a00640 1 -- 172.21.15.26:0/3705325151 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0002c9e0 con 0x7f8b0813fcb0 2024-01-15T17:55:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.729+0000 7f8b06a00640 1 -- 172.21.15.26:0/3705325151 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0002cb40 con 0x7f8b0813fcb0 2024-01-15T17:55:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.729+0000 7f8b06a00640 1 -- 172.21.15.26:0/3705325151 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b000343f0 con 0x7f8b0813fcb0 2024-01-15T17:55:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.730+0000 7f8b0d000640 1 -- 172.21.15.26:0/3705325151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813fcb0 msgr2=0x7f8b080b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.730+0000 7f8b0d000640 1 --2- 172.21.15.26:0/3705325151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813fcb0 0x7f8b080b2040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009e50 tx=0x7f8b0002be40 comp rx=0 tx=0).stop 2024-01-15T17:55:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.730+0000 7f8b0d000640 1 -- 172.21.15.26:0/3705325151 shutdown_connections 2024-01-15T17:55:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.730+0000 7f8b0d000640 1 --2- 172.21.15.26:0/3705325151 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813fcb0 0x7f8b080b2040 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.730+0000 7f8b0d000640 1 --2- 172.21.15.26:0/3705325151 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813cfa0 0x7f8b0815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.730+0000 7f8b0d000640 1 -- 172.21.15.26:0/3705325151 >> 172.21.15.26:0/3705325151 conn(0x7f8b0806a800 msgr2=0x7f8b0806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.730+0000 7f8b0d000640 1 -- 172.21.15.26:0/3705325151 shutdown_connections 2024-01-15T17:55:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.730+0000 7f8b0d000640 1 -- 172.21.15.26:0/3705325151 wait complete. 2024-01-15T17:55:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.732+0000 7f8b0d000640 1 Processor -- start 2024-01-15T17:55:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.732+0000 7f8b0d000640 1 -- start start 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.733+0000 7f8b0d000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b080cd530 0x7f8b081575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.733+0000 7f8b0d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813cfa0 0x7f8b0812f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.733+0000 7f8b0d000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813fcb0 0x7f8b08155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.733+0000 7f8b0d000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0812e070 con 0x7f8b0813cfa0 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.733+0000 7f8b0d000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0812d3f0 con 0x7f8b080cd530 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.733+0000 7f8b0d000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0812d6d0 con 0x7f8b0813fcb0 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.733+0000 7f8afe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813cfa0 0x7f8b0812f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.733+0000 7f8afe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813cfa0 0x7f8b0812f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44462/0 (socket says 172.21.15.26:44462) 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.733+0000 7f8afe800640 1 -- 172.21.15.26:0/1055383584 learned_addr learned my addr 172.21.15.26:0/1055383584 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8b07e00640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813fcb0 0x7f8b08155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8b07400640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b080cd530 0x7f8b081575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8afe800640 1 -- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813fcb0 msgr2=0x7f8b08155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8afe800640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813fcb0 0x7f8b08155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8afe800640 1 -- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b080cd530 msgr2=0x7f8b081575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8afe800640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b080cd530 0x7f8b081575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8afe800640 1 -- 172.21.15.26:0/1055383584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b00009b30 con 0x7f8b0813cfa0 2024-01-15T17:55:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8afe800640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813cfa0 0x7f8b0812f9b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8af000c430 tx=0x7f8af000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8affe00640 1 -- 172.21.15.26:0/1055383584 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011790 con 0x7f8b0813cfa0 2024-01-15T17:55:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8affe00640 1 -- 172.21.15.26:0/1055383584 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af00118f0 con 0x7f8b0813cfa0 2024-01-15T17:55:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8affe00640 1 -- 172.21.15.26:0/1055383584 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0011ce0 con 0x7f8b0813cfa0 2024-01-15T17:55:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.734+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0812d9b0 con 0x7f8b0813cfa0 2024-01-15T17:55:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.735+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0812fef0 con 0x7f8b0813cfa0 2024-01-15T17:55:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.736+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0812e1b0 con 0x7f8b0813cfa0 2024-01-15T17:55:40.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.737+0000 7f8affe00640 1 -- 172.21.15.26:0/1055383584 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8af001c460 con 0x7f8b0813cfa0 2024-01-15T17:55:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.738+0000 7f8affe00640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8ae006b2f0 0x7f8ae006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.738+0000 7f8affe00640 1 -- 172.21.15.26:0/1055383584 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8af0089b70 con 0x7f8b0813cfa0 2024-01-15T17:55:40.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.739+0000 7f8b07400640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8ae006b2f0 0x7f8ae006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:40.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.740+0000 7f8b07400640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8ae006b2f0 0x7f8ae006d7b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8af8008850 tx=0x7f8af8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:40.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.742+0000 7f8affe00640 1 -- 172.21.15.26:0/1055383584 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b0812e1b0 con 0x7f8b0813cfa0 2024-01-15T17:55:40.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.941+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b0812e1b0 con 0x7f8b0813cfa0 2024-01-15T17:55:40.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.944+0000 7f8affe00640 1 -- 172.21.15.26:0/1055383584 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f8b0812e1b0 con 0x7f8b0813cfa0 2024-01-15T17:55:40.949 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:40.950 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:40.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8ae006b2f0 msgr2=0x7f8ae006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8ae006b2f0 0x7f8ae006d7b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8af8008850 tx=0x7f8af8008040 comp rx=0 tx=0).stop 2024-01-15T17:55:40.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813cfa0 msgr2=0x7f8b0812f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:40.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813cfa0 0x7f8b0812f9b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8af000c430 tx=0x7f8af000c8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 shutdown_connections 2024-01-15T17:55:40.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b0813fcb0 0x7f8b08155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8ae006b2f0 0x7f8ae006d7b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b0813cfa0 0x7f8b0812f9b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 --2- 172.21.15.26:0/1055383584 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b080cd530 0x7f8b081575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.947+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 >> 172.21.15.26:0/1055383584 conn(0x7f8b0806a800 msgr2=0x7f8b08071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.948+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 shutdown_connections 2024-01-15T17:55:40.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:40.948+0000 7f8b0d000640 1 -- 172.21.15.26:0/1055383584 wait complete. 2024-01-15T17:55:40.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:40.972 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:41.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.194+0000 7f0055200640 1 Processor -- start 2024-01-15T17:55:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.194+0000 7f0055200640 1 -- start start 2024-01-15T17:55:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.195+0000 7f0055200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f005013cf10 0x7f005013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.195+0000 7f0055200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f00500cddf0 0x7f00500b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.195+0000 7f0055200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00500ce440 con 0x7f005013fae0 2024-01-15T17:55:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.195+0000 7f0055200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f005013ff90 con 0x7f005013cf10 2024-01-15T17:55:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.195+0000 7f0055200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00500ce220 con 0x7f00500cddf0 2024-01-15T17:55:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.196+0000 7f004ea00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f005013fae0 0x7f00500cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55666/0 (socket says 172.21.15.26:55666) 2024-01-15T17:55:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.196+0000 7f004ea00640 1 -- 172.21.15.26:0/349071196 learned_addr learned my addr 172.21.15.26:0/349071196 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.196+0000 7f004f400640 1 --2- 172.21.15.26:0/349071196 >> v2:172.21.15.26:3300/0 conn(0x7f005013cf10 0x7f005013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.196+0000 7f004f400640 1 -- 172.21.15.26:0/349071196 >> v2:172.21.15.124:3300/0 conn(0x7f00500cddf0 msgr2=0x7f00500b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.196+0000 7f004f400640 1 --2- 172.21.15.26:0/349071196 >> v2:172.21.15.124:3300/0 conn(0x7f00500cddf0 0x7f00500b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.196+0000 7f004f400640 1 -- 172.21.15.26:0/349071196 >> v1:172.21.15.26:6789/0 conn(0x7f005013fae0 legacy=0x7f00500cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.196+0000 7f004f400640 1 -- 172.21.15.26:0/349071196 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00500ce720 con 0x7f005013cf10 2024-01-15T17:55:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.198+0000 7f004f400640 1 --2- 172.21.15.26:0/349071196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 0x7f005013f5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0050157f90 tx=0x7f004002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b36810b01cbb4cf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.198+0000 7f004e000640 1 -- 172.21.15.26:0/349071196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040031830 con 0x7f005013cf10 2024-01-15T17:55:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.198+0000 7f004e000640 1 -- 172.21.15.26:0/349071196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0040031990 con 0x7f005013cf10 2024-01-15T17:55:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.198+0000 7f004e000640 1 -- 172.21.15.26:0/349071196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040031d20 con 0x7f005013cf10 2024-01-15T17:55:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.198+0000 7f0055200640 1 -- 172.21.15.26:0/349071196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 msgr2=0x7f005013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.198+0000 7f0055200640 1 --2- 172.21.15.26:0/349071196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 0x7f005013f5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0050157f90 tx=0x7f004002b8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.199+0000 7f0055200640 1 -- 172.21.15.26:0/349071196 shutdown_connections 2024-01-15T17:55:41.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.199+0000 7f0055200640 1 --2- 172.21.15.26:0/349071196 >> v2:172.21.15.124:3300/0 conn(0x7f00500cddf0 0x7f00500b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.199+0000 7f0055200640 1 --2- 172.21.15.26:0/349071196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 0x7f005013f5a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.199+0000 7f0055200640 1 -- 172.21.15.26:0/349071196 >> 172.21.15.26:0/349071196 conn(0x7f005006a7d0 msgr2=0x7f005006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:41.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.199+0000 7f0055200640 1 -- 172.21.15.26:0/349071196 shutdown_connections 2024-01-15T17:55:41.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.199+0000 7f0055200640 1 -- 172.21.15.26:0/349071196 wait complete. 2024-01-15T17:55:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.200+0000 7f0055200640 1 Processor -- start 2024-01-15T17:55:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.201+0000 7f0055200640 1 -- start start 2024-01-15T17:55:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.201+0000 7f0055200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00500cddf0 0x7f0050138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.201+0000 7f0055200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 0x7f00501385d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f0055200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f005013fae0 0x7f00501e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f0055200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005015b8e0 con 0x7f005013cf10 2024-01-15T17:55:41.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f0055200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005015bbc0 con 0x7f00500cddf0 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f0055200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005015bea0 con 0x7f005013fae0 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 0x7f00501385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 0x7f00501385d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44486/0 (socket says 172.21.15.26:44486) 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004ea00640 1 -- 172.21.15.26:0/191653618 learned_addr learned my addr 172.21.15.26:0/191653618 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004f400640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00500cddf0 0x7f0050138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004fe00640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f005013fae0 0x7f00501e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004ea00640 1 -- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f005013fae0 msgr2=0x7f00501e69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004ea00640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f005013fae0 0x7f00501e69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004ea00640 1 -- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00500cddf0 msgr2=0x7f0050138ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004ea00640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00500cddf0 0x7f0050138ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004ea00640 1 -- 172.21.15.26:0/191653618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0040009400 con 0x7f005013cf10 2024-01-15T17:55:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.202+0000 7f004fe00640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f005013fae0 0x7f00501e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.203+0000 7f004ea00640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 0x7f00501385d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f003800c480 tx=0x7f003800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.203+0000 7f0047400640 1 -- 172.21.15.26:0/191653618 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038005ce0 con 0x7f005013cf10 2024-01-15T17:55:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.203+0000 7f0047400640 1 -- 172.21.15.26:0/191653618 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0038005e40 con 0x7f005013cf10 2024-01-15T17:55:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.203+0000 7f0047400640 1 -- 172.21.15.26:0/191653618 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380072f0 con 0x7f005013cf10 2024-01-15T17:55:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.203+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f005015c180 con 0x7f005013cf10 2024-01-15T17:55:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.203+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f005015c980 con 0x7f005013cf10 2024-01-15T17:55:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.204+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f005015c6a0 con 0x7f005013cf10 2024-01-15T17:55:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.205+0000 7f0047400640 1 -- 172.21.15.26:0/191653618 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0038007510 con 0x7f005013cf10 2024-01-15T17:55:41.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.206+0000 7f0047400640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002806b2c0 0x7f002806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.206+0000 7f0047400640 1 -- 172.21.15.26:0/191653618 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0038089100 con 0x7f005013cf10 2024-01-15T17:55:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.207+0000 7f004f400640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002806b2c0 0x7f002806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.208+0000 7f004f400640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002806b2c0 0x7f002806d780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0040036000 tx=0x7f004003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:41.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.211+0000 7f0047400640 1 -- 172.21.15.26:0/191653618 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f005015c6a0 con 0x7f005013cf10 2024-01-15T17:55:41.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.402+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f005015c6a0 con 0x7f005013cf10 2024-01-15T17:55:41.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.403+0000 7f0047400640 1 -- 172.21.15.26:0/191653618 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f005015c6a0 con 0x7f005013cf10 2024-01-15T17:55:41.408 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002806b2c0 msgr2=0x7f002806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002806b2c0 0x7f002806d780 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0040036000 tx=0x7f004003c040 comp rx=0 tx=0).stop 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 msgr2=0x7f00501385d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 0x7f00501385d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f003800c480 tx=0x7f003800c940 comp rx=0 tx=0).stop 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 shutdown_connections 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f005013fae0 0x7f00501e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002806b2c0 0x7f002806d780 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f005013cf10 0x7f00501385d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 --2- 172.21.15.26:0/191653618 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00500cddf0 0x7f0050138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.406+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 >> 172.21.15.26:0/191653618 conn(0x7f005006a7d0 msgr2=0x7f0050159590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.407+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 shutdown_connections 2024-01-15T17:55:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.407+0000 7f0055200640 1 -- 172.21.15.26:0/191653618 wait complete. 2024-01-15T17:55:41.429 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:41.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.647+0000 7fed62a00640 1 Processor -- start 2024-01-15T17:55:41.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.647+0000 7fed62a00640 1 -- start start 2024-01-15T17:55:41.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.648+0000 7fed62a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed5c0d17e0 0x7fed5c0d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.648+0000 7fed62a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5c0d2180 con 0x7fed5c0b3890 2024-01-15T17:55:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.648+0000 7fed62a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5c0b3400 con 0x7fed5c15d610 2024-01-15T17:55:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.648+0000 7fed62a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed5c0d1390 con 0x7fed5c0d17e0 2024-01-15T17:55:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.649+0000 7fed5b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed5c0d17e0 0x7fed5c0d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.649+0000 7fed5b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed5c0d17e0 0x7fed5c0d1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44502/0 (socket says 172.21.15.26:44502) 2024-01-15T17:55:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.649+0000 7fed5b400640 1 -- 172.21.15.26:0/464853572 learned_addr learned my addr 172.21.15.26:0/464853572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.649+0000 7fed5b400640 1 -- 172.21.15.26:0/464853572 >> v1:172.21.15.124:6789/0 conn(0x7fed5c15d610 legacy=0x7fed5c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.649+0000 7fed5b400640 1 -- 172.21.15.26:0/464853572 >> v1:172.21.15.26:6789/0 conn(0x7fed5c0b3890 legacy=0x7fed5c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.649+0000 7fed5b400640 1 -- 172.21.15.26:0/464853572 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c0b27f0 con 0x7fed5c0d17e0 2024-01-15T17:55:41.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.650+0000 7fed5b400640 1 --2- 172.21.15.26:0/464853572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 0x7fed5c0d1c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed44009870 tx=0x7fed4402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184ec99f57f34131 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:41.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed5aa00640 1 -- 172.21.15.26:0/464853572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44030860 con 0x7fed5c0d17e0 2024-01-15T17:55:41.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed5aa00640 1 -- 172.21.15.26:0/464853572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed440309c0 con 0x7fed5c0d17e0 2024-01-15T17:55:41.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed5aa00640 1 -- 172.21.15.26:0/464853572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44030d50 con 0x7fed5c0d17e0 2024-01-15T17:55:41.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed62a00640 1 -- 172.21.15.26:0/464853572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 msgr2=0x7fed5c0d1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed62a00640 1 --2- 172.21.15.26:0/464853572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 0x7fed5c0d1c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fed44009870 tx=0x7fed4402b9a0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed62a00640 1 -- 172.21.15.26:0/464853572 shutdown_connections 2024-01-15T17:55:41.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed62a00640 1 --2- 172.21.15.26:0/464853572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 0x7fed5c0d1c40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed62a00640 1 -- 172.21.15.26:0/464853572 >> 172.21.15.26:0/464853572 conn(0x7fed5c09c540 msgr2=0x7fed5c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:41.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed62a00640 1 -- 172.21.15.26:0/464853572 shutdown_connections 2024-01-15T17:55:41.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.651+0000 7fed62a00640 1 -- 172.21.15.26:0/464853572 wait complete. 2024-01-15T17:55:41.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.653+0000 7fed62a00640 1 Processor -- start 2024-01-15T17:55:41.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.653+0000 7fed62a00640 1 -- start start 2024-01-15T17:55:41.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.654+0000 7fed62a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed5c0b3890 0x7fed5c14f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.654+0000 7fed62a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 0x7fed5c0b8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.654+0000 7fed62a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed5c15d610 0x7fed5c0b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.654+0000 7fed62a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c14ff10 con 0x7fed5c0d17e0 2024-01-15T17:55:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.654+0000 7fed62a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c0b60f0 con 0x7fed5c15d610 2024-01-15T17:55:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.654+0000 7fed62a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed5c0b63d0 con 0x7fed5c0b3890 2024-01-15T17:55:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.654+0000 7fed5b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 0x7fed5c0b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.655+0000 7fed5b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 0x7fed5c0b8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44508/0 (socket says 172.21.15.26:44508) 2024-01-15T17:55:41.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.655+0000 7fed5b400640 1 -- 172.21.15.26:0/1000936911 learned_addr learned my addr 172.21.15.26:0/1000936911 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.655+0000 7fed60a00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed5c15d610 0x7fed5c0b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.655+0000 7fed5be00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed5c0b3890 0x7fed5c14f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.655+0000 7fed5b400640 1 -- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed5c0b3890 msgr2=0x7fed5c14f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.655+0000 7fed5b400640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed5c0b3890 0x7fed5c14f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.655+0000 7fed5b400640 1 -- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed5c15d610 msgr2=0x7fed5c0b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.655+0000 7fed5b400640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed5c15d610 0x7fed5c0b7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.655+0000 7fed5b400640 1 -- 172.21.15.26:0/1000936911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed440094d0 con 0x7fed5c0d17e0 2024-01-15T17:55:41.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.656+0000 7fed5b400640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 0x7fed5c0b8930 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fed44009870 tx=0x7fed44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:41.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.656+0000 7fed53e00640 1 -- 172.21.15.26:0/1000936911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed44030810 con 0x7fed5c0d17e0 2024-01-15T17:55:41.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.656+0000 7fed53e00640 1 -- 172.21.15.26:0/1000936911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed44043400 con 0x7fed5c0d17e0 2024-01-15T17:55:41.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.656+0000 7fed53e00640 1 -- 172.21.15.26:0/1000936911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed440437b0 con 0x7fed5c0d17e0 2024-01-15T17:55:41.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.656+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed5c0b66b0 con 0x7fed5c0d17e0 2024-01-15T17:55:41.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.656+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5c0b6c70 con 0x7fed5c0d17e0 2024-01-15T17:55:41.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.657+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5c0675d0 con 0x7fed5c0d17e0 2024-01-15T17:55:41.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.658+0000 7fed53e00640 1 -- 172.21.15.26:0/1000936911 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fed44043910 con 0x7fed5c0d17e0 2024-01-15T17:55:41.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.659+0000 7fed53e00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed3406b2c0 0x7fed3406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:41.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.660+0000 7fed53e00640 1 -- 172.21.15.26:0/1000936911 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed4403e070 con 0x7fed5c0d17e0 2024-01-15T17:55:41.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.660+0000 7fed5be00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed3406b2c0 0x7fed3406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:41.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.661+0000 7fed5be00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed3406b2c0 0x7fed3406d780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006fd0 tx=0x7fed4c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:41.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.664+0000 7fed53e00640 1 -- 172.21.15.26:0/1000936911 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed44077ea0 con 0x7fed5c0d17e0 2024-01-15T17:55:41.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.858+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed5c0b1470 con 0x7fed5c0d17e0 2024-01-15T17:55:41.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.859+0000 7fed53e00640 1 -- 172.21.15.26:0/1000936911 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed44034020 con 0x7fed5c0d17e0 2024-01-15T17:55:41.864 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.862+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed3406b2c0 msgr2=0x7fed3406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.862+0000 7fed62a00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed3406b2c0 0x7fed3406d780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006fd0 tx=0x7fed4c008040 comp rx=0 tx=0).stop 2024-01-15T17:55:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.862+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 msgr2=0x7fed5c0b8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.862+0000 7fed62a00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 0x7fed5c0b8930 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fed44009870 tx=0x7fed44004730 comp rx=0 tx=0).stop 2024-01-15T17:55:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.863+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 shutdown_connections 2024-01-15T17:55:41.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.863+0000 7fed62a00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed3406b2c0 0x7fed3406d780 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.863+0000 7fed62a00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed5c15d610 0x7fed5c0b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.863+0000 7fed62a00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed5c0d17e0 0x7fed5c0b8930 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.863+0000 7fed62a00640 1 --2- 172.21.15.26:0/1000936911 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed5c0b3890 0x7fed5c14f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.863+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 >> 172.21.15.26:0/1000936911 conn(0x7fed5c09c540 msgr2=0x7fed5c0b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.863+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 shutdown_connections 2024-01-15T17:55:41.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:41.863+0000 7fed62a00640 1 -- 172.21.15.26:0/1000936911 wait complete. 2024-01-15T17:55:41.887 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.109+0000 7f0ed3200640 1 Processor -- start 2024-01-15T17:55:42.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.110+0000 7f0ed3200640 1 -- start start 2024-01-15T17:55:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.111+0000 7f0ed3200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0ecc15d4b0 0x7f0ecc0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.111+0000 7f0ed3200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0b2730 con 0x7f0ecc0cf9a0 2024-01-15T17:55:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.111+0000 7f0ed3200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0d1ec0 con 0x7f0ecc0b41e0 2024-01-15T17:55:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.111+0000 7f0ed3200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc0b29d0 con 0x7f0ecc15d4b0 2024-01-15T17:55:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.111+0000 7f0ed1400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0ecc0cf9a0 0x7f0ecc0cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57310/0 (socket says 172.21.15.26:57310) 2024-01-15T17:55:42.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.112+0000 7f0ed1400640 1 -- 172.21.15.26:0/2276507210 learned_addr learned my addr 172.21.15.26:0/2276507210 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.112+0000 7f0ed1e00640 1 --2- 172.21.15.26:0/2276507210 >> v2:172.21.15.124:3300/0 conn(0x7f0ecc15d4b0 0x7f0ecc0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.112+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702726570 0 0) 0x7f0ecc0b2730 con 0x7f0ecc0cf9a0 2024-01-15T17:55:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.112+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eac003520 con 0x7f0ecc0cf9a0 2024-01-15T17:55:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1154999559 0 0) 0x7f0eac003520 con 0x7f0ecc0cf9a0 2024-01-15T17:55:42.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 >> v2:172.21.15.124:3300/0 conn(0x7f0ecc15d4b0 msgr2=0x7f0ecc0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 --2- 172.21.15.26:0/2276507210 >> v2:172.21.15.124:3300/0 conn(0x7f0ecc15d4b0 0x7f0ecc0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 >> v1:172.21.15.124:6789/0 conn(0x7f0ecc0b41e0 legacy=0x7f0ecc0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0b2cb0 con 0x7f0ecc0cf9a0 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f0eb40019f0 con 0x7f0ecc0cf9a0 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 >> v1:172.21.15.26:6789/0 conn(0x7f0ecc0cf9a0 legacy=0x7f0ecc0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0eac0039c0 0x7f0eac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0eac0047c0 0x7f0eac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eac005180 0x7f0eac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0b29d0 con 0x7f0eac005180 2024-01-15T17:55:42.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac00c080 con 0x7f0eac0039c0 2024-01-15T17:55:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eac00c240 con 0x7f0eac0047c0 2024-01-15T17:55:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed2800640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eac005180 0x7f0eac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.113+0000 7f0ed1e00640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0eac0039c0 0x7f0eac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.114+0000 7f0ed1400640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0eac0047c0 0x7f0eac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.114+0000 7f0ed2800640 1 -- 172.21.15.26:0/2276507210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0eac0047c0 msgr2=0x7f0eac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.114+0000 7f0ed2800640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0eac0047c0 0x7f0eac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.114+0000 7f0ed2800640 1 -- 172.21.15.26:0/2276507210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0eac0039c0 msgr2=0x7f0eac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.114+0000 7f0ed2800640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0eac0039c0 0x7f0eac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.114+0000 7f0ed2800640 1 -- 172.21.15.26:0/2276507210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eac003520 con 0x7f0eac005180 2024-01-15T17:55:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.114+0000 7f0ed1400640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0eac0047c0 0x7f0eac004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.115+0000 7f0ed2800640 1 -- 172.21.15.26:0/2276507210 reap_dead start 2024-01-15T17:55:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.115+0000 7f0ed2800640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eac005180 0x7f0eac00b980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001720 tx=0x7f0eac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294ee14b4e172e59 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.115+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40052d0 con 0x7f0eac005180 2024-01-15T17:55:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.115+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec402e050 con 0x7f0eac005180 2024-01-15T17:55:42.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.115+0000 7f0ed0a00640 1 -- 172.21.15.26:0/2276507210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40035f0 con 0x7f0eac005180 2024-01-15T17:55:42.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.116+0000 7f0ed3200640 1 -- 172.21.15.26:0/2276507210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eac005180 msgr2=0x7f0eac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.116+0000 7f0ed3200640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eac005180 0x7f0eac00b980 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001720 tx=0x7f0eac0057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.116+0000 7f0ed3200640 1 -- 172.21.15.26:0/2276507210 shutdown_connections 2024-01-15T17:55:42.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.116+0000 7f0ed3200640 1 --2- 172.21.15.26:0/2276507210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0eac005180 0x7f0eac00b980 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.116+0000 7f0ed3200640 1 -- 172.21.15.26:0/2276507210 >> 172.21.15.26:0/2276507210 conn(0x7f0ecc0d3000 msgr2=0x7f0ecc0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:42.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.116+0000 7f0ed3200640 1 -- 172.21.15.26:0/2276507210 shutdown_connections 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.116+0000 7f0ed3200640 1 -- 172.21.15.26:0/2276507210 wait complete. 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.118+0000 7f0ed3200640 1 Processor -- start 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.118+0000 7f0ed3200640 1 -- start start 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed3200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0eac005180 0x7f0ecc0c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ecc0a0ac0 0x7f0ecc0cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ecc06ae20 0x7f0ecc06b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc06b7e0 con 0x7f0ecc06ae20 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed3200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0cd2f0 con 0x7f0eac005180 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0c9150 con 0x7f0ecc0a0ac0 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ecc06ae20 0x7f0ecc06b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ecc06ae20 0x7f0ecc06b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60816/0 (socket says 172.21.15.26:60816) 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed2800640 1 -- 172.21.15.26:0/1917515209 learned_addr learned my addr 172.21.15.26:0/1917515209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed2800640 1 -- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ecc0a0ac0 msgr2=0x7f0ecc0cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed2800640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ecc0a0ac0 0x7f0ecc0cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed2800640 1 -- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0eac005180 msgr2=0x7f0ecc0c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed2800640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0eac005180 0x7f0ecc0c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.119+0000 7f0ed2800640 1 -- 172.21.15.26:0/1917515209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc15da10 con 0x7f0ecc06ae20 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.120+0000 7f0ed2800640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ecc06ae20 0x7f0ecc06b2a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001840 tx=0x7f0ec4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.120+0000 7f0eca000640 1 -- 172.21.15.26:0/1917515209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40054c0 con 0x7f0ecc06ae20 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.120+0000 7f0eca000640 1 -- 172.21.15.26:0/1917515209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec4038040 con 0x7f0ecc06ae20 2024-01-15T17:55:42.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.120+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc0cc6e0 con 0x7f0ecc06ae20 2024-01-15T17:55:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.120+0000 7f0eca000640 1 -- 172.21.15.26:0/1917515209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4007250 con 0x7f0ecc06ae20 2024-01-15T17:55:42.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.120+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc0ccca0 con 0x7f0ecc06ae20 2024-01-15T17:55:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.122+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc067450 con 0x7f0ecc06ae20 2024-01-15T17:55:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.122+0000 7f0eca000640 1 -- 172.21.15.26:0/1917515209 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0ec40073b0 con 0x7f0ecc06ae20 2024-01-15T17:55:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.123+0000 7f0eca000640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ea406b2f0 0x7f0ea406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.123+0000 7f0eca000640 1 -- 172.21.15.26:0/1917515209 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ec40aa8b0 con 0x7f0ecc06ae20 2024-01-15T17:55:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.124+0000 7f0ed1e00640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ea406b2f0 0x7f0ea406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.125+0000 7f0ed1e00640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ea406b2f0 0x7f0ea406d7b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009890 tx=0x7f0ebc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:42.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.128+0000 7f0eca000640 1 -- 172.21.15.26:0/1917515209 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ec4075680 con 0x7f0ecc06ae20 2024-01-15T17:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.325+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ecc06cb30 con 0x7f0ecc06ae20 2024-01-15T17:55:42.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.326+0000 7f0eca000640 1 -- 172.21.15.26:0/1917515209 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0ec4042020 con 0x7f0ecc06ae20 2024-01-15T17:55:42.331 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:42.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ea406b2f0 msgr2=0x7f0ea406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ea406b2f0 0x7f0ea406d7b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc009890 tx=0x7f0ebc008040 comp rx=0 tx=0).stop 2024-01-15T17:55:42.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ecc06ae20 msgr2=0x7f0ecc06b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ecc06ae20 0x7f0ecc06b2a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001840 tx=0x7f0ec4001120 comp rx=0 tx=0).stop 2024-01-15T17:55:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 shutdown_connections 2024-01-15T17:55:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ecc0a0ac0 0x7f0ecc0cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ea406b2f0 0x7f0ea406d7b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ecc06ae20 0x7f0ecc06b2a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 --2- 172.21.15.26:0/1917515209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0eac005180 0x7f0ecc0c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.329+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 >> 172.21.15.26:0/1917515209 conn(0x7f0ecc0d3000 msgr2=0x7f0ecc0cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:42.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.330+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 shutdown_connections 2024-01-15T17:55:42.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.330+0000 7f0ed3200640 1 -- 172.21.15.26:0/1917515209 wait complete. 2024-01-15T17:55:42.354 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:42.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.573+0000 7fb6c4e00640 1 Processor -- start 2024-01-15T17:55:42.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.573+0000 7fb6c4e00640 1 -- start start 2024-01-15T17:55:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.575+0000 7fb6c4e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c0160920 0x7fb6c00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.575+0000 7fb6c4e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb6c00cf9f0 0x7fb6c00cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.575+0000 7fb6c4e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c00d0a10 con 0x7fb6c0161010 2024-01-15T17:55:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.575+0000 7fb6c4e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c00d03b0 con 0x7fb6c00cf9f0 2024-01-15T17:55:42.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.575+0000 7fb6c4e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00d1f50 con 0x7fb6c0160920 2024-01-15T17:55:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.576+0000 7fb6bea00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb6c00cf9f0 0x7fb6c00cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.576+0000 7fb6bf400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c0160920 0x7fb6c00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.576+0000 7fb6bf400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c0160920 0x7fb6c00b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:42752/0 (socket says 172.21.15.26:42752) 2024-01-15T17:55:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.576+0000 7fb6bf400640 1 -- 172.21.15.26:0/1626300724 learned_addr learned my addr 172.21.15.26:0/1626300724 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.577+0000 7fb6bea00640 1 -- 172.21.15.26:0/1626300724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c0160920 msgr2=0x7fb6c00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.577+0000 7fb6bea00640 1 --2- 172.21.15.26:0/1626300724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c0160920 0x7fb6c00b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.577+0000 7fb6bea00640 1 -- 172.21.15.26:0/1626300724 >> v1:172.21.15.124:6789/0 conn(0x7fb6c0161010 legacy=0x7fb6c01614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.577+0000 7fb6bea00640 1 -- 172.21.15.26:0/1626300724 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c00cf080 con 0x7fb6c00cf9f0 2024-01-15T17:55:42.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.578+0000 7fb6bea00640 1 --2- 172.21.15.26:0/1626300724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c00cf9f0 0x7fb6c00cfe70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80097a0 tx=0x7fb6a802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6179597745dd6da0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:42.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.578+0000 7fb6be000640 1 -- 172.21.15.26:0/1626300724 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a802d040 con 0x7fb6c00cf9f0 2024-01-15T17:55:42.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6be000640 1 -- 172.21.15.26:0/1626300724 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a8031810 con 0x7fb6c00cf9f0 2024-01-15T17:55:42.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6be000640 1 -- 172.21.15.26:0/1626300724 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8031ba0 con 0x7fb6c00cf9f0 2024-01-15T17:55:42.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6c4e00640 1 -- 172.21.15.26:0/1626300724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c00cf9f0 msgr2=0x7fb6c00cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6c4e00640 1 --2- 172.21.15.26:0/1626300724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c00cf9f0 0x7fb6c00cfe70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80097a0 tx=0x7fb6a802b8b0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6c4e00640 1 -- 172.21.15.26:0/1626300724 shutdown_connections 2024-01-15T17:55:42.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6c4e00640 1 --2- 172.21.15.26:0/1626300724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c00cf9f0 0x7fb6c00cfe70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6c4e00640 1 --2- 172.21.15.26:0/1626300724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c0160920 0x7fb6c00b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6c4e00640 1 -- 172.21.15.26:0/1626300724 >> 172.21.15.26:0/1626300724 conn(0x7fb6c00d3090 msgr2=0x7fb6c00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6c4e00640 1 -- 172.21.15.26:0/1626300724 shutdown_connections 2024-01-15T17:55:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.579+0000 7fb6c4e00640 1 -- 172.21.15.26:0/1626300724 wait complete. 2024-01-15T17:55:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.581+0000 7fb6c4e00640 1 Processor -- start 2024-01-15T17:55:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.581+0000 7fb6c4e00640 1 -- start start 2024-01-15T17:55:42.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.582+0000 7fb6c4e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c00cf9f0 0x7fb6c01503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.582+0000 7fb6c4e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6c0160920 0x7fb6c00cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.582+0000 7fb6c4e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c0161010 0x7fb6c015c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.582+0000 7fb6c4e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00cc0a0 con 0x7fb6c0160920 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.582+0000 7fb6c4e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00cc380 con 0x7fb6c0161010 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.582+0000 7fb6c4e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00cc660 con 0x7fb6c00cf9f0 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6c0160920 0x7fb6c00cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6c0160920 0x7fb6c00cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60828/0 (socket says 172.21.15.26:60828) 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bea00640 1 -- 172.21.15.26:0/70491512 learned_addr learned my addr 172.21.15.26:0/70491512 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bf400640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c00cf9f0 0x7fb6c01503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bfe00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c0161010 0x7fb6c015c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bea00640 1 -- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c00cf9f0 msgr2=0x7fb6c01503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bea00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c00cf9f0 0x7fb6c01503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bea00640 1 -- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c0161010 msgr2=0x7fb6c015c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bea00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c0161010 0x7fb6c015c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bea00640 1 -- 172.21.15.26:0/70491512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a8009400 con 0x7fb6c0160920 2024-01-15T17:55:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bf400640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c00cf9f0 0x7fb6c01503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.583+0000 7fb6bfe00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c0161010 0x7fb6c015c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.584+0000 7fb6bea00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6c0160920 0x7fb6c00cb6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098c0 tx=0x7fb6a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.584+0000 7fb6b7400640 1 -- 172.21.15.26:0/70491512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a803ac60 con 0x7fb6c0160920 2024-01-15T17:55:42.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.584+0000 7fb6b7400640 1 -- 172.21.15.26:0/70491512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a803ade0 con 0x7fb6c0160920 2024-01-15T17:55:42.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.584+0000 7fb6b7400640 1 -- 172.21.15.26:0/70491512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8044630 con 0x7fb6c0160920 2024-01-15T17:55:42.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.584+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c00cc940 con 0x7fb6c0160920 2024-01-15T17:55:42.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.584+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c0150910 con 0x7fb6c0160920 2024-01-15T17:55:42.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.586+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c00cce60 con 0x7fb6c0160920 2024-01-15T17:55:42.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.587+0000 7fb6b7400640 1 -- 172.21.15.26:0/70491512 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb6a804d450 con 0x7fb6c0160920 2024-01-15T17:55:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.587+0000 7fb6b7400640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb69806b310 0x7fb69806d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.588+0000 7fb6b7400640 1 -- 172.21.15.26:0/70491512 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6a80abe50 con 0x7fb6c0160920 2024-01-15T17:55:42.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.588+0000 7fb6bf400640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb69806b310 0x7fb69806d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:42.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.589+0000 7fb6bf400640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb69806b310 0x7fb69806d7d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009d70 tx=0x7fb6b000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:42.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.592+0000 7fb6b7400640 1 -- 172.21.15.26:0/70491512 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6c00cce60 con 0x7fb6c0160920 2024-01-15T17:55:42.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.788+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6c00cce60 con 0x7fb6c0160920 2024-01-15T17:55:42.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.791+0000 7fb6b7400640 1 -- 172.21.15.26:0/70491512 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7fb6c00cce60 con 0x7fb6c0160920 2024-01-15T17:55:42.795 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:42.796 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:42.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb69806b310 msgr2=0x7fb69806d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb69806b310 0x7fb69806d7d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009d70 tx=0x7fb6b000b040 comp rx=0 tx=0).stop 2024-01-15T17:55:42.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6c0160920 msgr2=0x7fb6c00cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:42.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6c0160920 0x7fb6c00cb6c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098c0 tx=0x7fb6a80046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 shutdown_connections 2024-01-15T17:55:42.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb69806b310 0x7fb69806d7d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6c0161010 0x7fb6c015c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6c0160920 0x7fb6c00cb6c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 --2- 172.21.15.26:0/70491512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6c00cf9f0 0x7fb6c01503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:42.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 >> 172.21.15.26:0/70491512 conn(0x7fb6c00d3090 msgr2=0x7fb6c006c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:42.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 shutdown_connections 2024-01-15T17:55:42.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:42.794+0000 7fb6c4e00640 1 -- 172.21.15.26:0/70491512 wait complete. 2024-01-15T17:55:42.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:42.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.039+0000 7ff149400640 1 Processor -- start 2024-01-15T17:55:43.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.039+0000 7ff149400640 1 -- start start 2024-01-15T17:55:43.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.041+0000 7ff149400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff14413cf10 0x7ff14413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.041+0000 7ff149400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1440ce440 con 0x7ff1440cddf0 2024-01-15T17:55:43.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.041+0000 7ff149400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1440ce720 con 0x7ff14413fae0 2024-01-15T17:55:43.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.041+0000 7ff149400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14413ff90 con 0x7ff14413cf10 2024-01-15T17:55:43.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.041+0000 7ff148a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff1440cddf0 0x7ff1440b2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57322/0 (socket says 172.21.15.26:57322) 2024-01-15T17:55:43.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.041+0000 7ff148a00640 1 -- 172.21.15.26:0/1766436966 learned_addr learned my addr 172.21.15.26:0/1766436966 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff143e00640 1 --2- 172.21.15.26:0/1766436966 >> v2:172.21.15.124:3300/0 conn(0x7ff14413cf10 0x7ff14413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080221283 0 0) 0x7ff1440ce440 con 0x7ff1440cddf0 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff124005730 con 0x7ff1440cddf0 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3631664453 0 0) 0x7ff124005730 con 0x7ff1440cddf0 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 >> v2:172.21.15.124:3300/0 conn(0x7ff14413cf10 msgr2=0x7ff14413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 --2- 172.21.15.26:0/1766436966 >> v2:172.21.15.124:3300/0 conn(0x7ff14413cf10 0x7ff14413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 >> v1:172.21.15.124:6789/0 conn(0x7ff14413fae0 legacy=0x7ff1440cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1440cea00 con 0x7ff1440cddf0 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7ff13c0019f0 con 0x7ff1440cddf0 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 >> v1:172.21.15.26:6789/0 conn(0x7ff1440cddf0 legacy=0x7ff1440b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff124005b80 0x7ff124005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1240069d0 0x7ff124006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.042+0000 7ff142a00640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff12400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14413ff90 con 0x7ff124007390 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12400e210 con 0x7ff1240069d0 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12400e3d0 con 0x7ff124005b80 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff148a00640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff12400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff143e00640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff124005b80 0x7ff124005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff143400640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1240069d0 0x7ff124006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff148a00640 1 -- 172.21.15.26:0/1766436966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff124005b80 msgr2=0x7ff124005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff148a00640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff124005b80 0x7ff124005f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff148a00640 1 -- 172.21.15.26:0/1766436966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1240069d0 msgr2=0x7ff124006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff148a00640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1240069d0 0x7ff124006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff148a00640 1 -- 172.21.15.26:0/1766436966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff124005730 con 0x7ff124007390 2024-01-15T17:55:43.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.043+0000 7ff143400640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1240069d0 0x7ff124006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.044+0000 7ff148a00640 1 -- 172.21.15.26:0/1766436966 reap_dead start 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.044+0000 7ff148a00640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff12400db90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00a830 tx=0x7ff124007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b57fc564d94a2d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.045+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c02cea0 con 0x7ff124007390 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.045+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c02f070 con 0x7ff124007390 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.045+0000 7ff142a00640 1 -- 172.21.15.26:0/1766436966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c034890 con 0x7ff124007390 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.045+0000 7ff149400640 1 -- 172.21.15.26:0/1766436966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 msgr2=0x7ff12400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.045+0000 7ff149400640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff12400db90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff13c00a830 tx=0x7ff124007a00 comp rx=0 tx=0).stop 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.045+0000 7ff149400640 1 -- 172.21.15.26:0/1766436966 shutdown_connections 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.045+0000 7ff149400640 1 --2- 172.21.15.26:0/1766436966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff12400db90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.045+0000 7ff149400640 1 -- 172.21.15.26:0/1766436966 >> 172.21.15.26:0/1766436966 conn(0x7ff14406a7d0 msgr2=0x7ff14406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:43.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.046+0000 7ff149400640 1 -- 172.21.15.26:0/1766436966 shutdown_connections 2024-01-15T17:55:43.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.046+0000 7ff149400640 1 -- 172.21.15.26:0/1766436966 wait complete. 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.047+0000 7ff149400640 1 Processor -- start 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.048+0000 7ff149400640 1 -- start start 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.048+0000 7ff149400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff144154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff149400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff144137780 0x7ff1441f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff149400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1440711b0 0x7ff144071630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff149400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff144071b70 con 0x7ff124007390 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff149400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff144137be0 con 0x7ff1440711b0 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff149400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff144136320 con 0x7ff144137780 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff144154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff144154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60842/0 (socket says 172.21.15.26:60842) 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143e00640 1 -- 172.21.15.26:0/2691167335 learned_addr learned my addr 172.21.15.26:0/2691167335 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff144137780 0x7ff1441f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143e00640 1 -- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff144137780 msgr2=0x7ff1441f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff148a00640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1440711b0 0x7ff144071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143e00640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff144137780 0x7ff1441f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143e00640 1 -- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1440711b0 msgr2=0x7ff144071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143e00640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1440711b0 0x7ff144071630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143e00640 1 -- 172.21.15.26:0/2691167335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff144137e20 con 0x7ff124007390 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.049+0000 7ff143400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff144137780 0x7ff1441f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.050+0000 7ff143e00640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff144154050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff134011540 tx=0x7ff134011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.050+0000 7ff13be00640 1 -- 172.21.15.26:0/2691167335 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134015930 con 0x7ff124007390 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.050+0000 7ff13be00640 1 -- 172.21.15.26:0/2691167335 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff134015a90 con 0x7ff124007390 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.050+0000 7ff13be00640 1 -- 172.21.15.26:0/2691167335 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff134015e60 con 0x7ff124007390 2024-01-15T17:55:43.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.050+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff144138100 con 0x7ff124007390 2024-01-15T17:55:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.050+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1441386c0 con 0x7ff124007390 2024-01-15T17:55:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.051+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff144067450 con 0x7ff124007390 2024-01-15T17:55:43.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.052+0000 7ff13be00640 1 -- 172.21.15.26:0/2691167335 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff13401d400 con 0x7ff124007390 2024-01-15T17:55:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.053+0000 7ff13be00640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff11c06b310 0x7ff11c06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.053+0000 7ff13be00640 1 -- 172.21.15.26:0/2691167335 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff13408a400 con 0x7ff124007390 2024-01-15T17:55:43.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.054+0000 7ff143400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff11c06b310 0x7ff11c06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.055+0000 7ff143400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff11c06b310 0x7ff11c06d7d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff12c007fd0 tx=0x7ff12c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:43.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.057+0000 7ff13be00640 1 -- 172.21.15.26:0/2691167335 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff134055250 con 0x7ff124007390 2024-01-15T17:55:43.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.254+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1440cbf10 con 0x7ff124007390 2024-01-15T17:55:43.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.255+0000 7ff13be00640 1 -- 172.21.15.26:0/2691167335 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff134058870 con 0x7ff124007390 2024-01-15T17:55:43.259 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff11c06b310 msgr2=0x7ff11c06d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff11c06b310 0x7ff11c06d7d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff12c007fd0 tx=0x7ff12c009040 comp rx=0 tx=0).stop 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 msgr2=0x7ff144154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff144154050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff134011540 tx=0x7ff134011a00 comp rx=0 tx=0).stop 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 shutdown_connections 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff144137780 0x7ff1441f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff11c06b310 0x7ff11c06d7d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1440711b0 0x7ff144071630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 --2- 172.21.15.26:0/2691167335 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff124007390 0x7ff144154050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 >> 172.21.15.26:0/2691167335 conn(0x7ff14406a7d0 msgr2=0x7ff1440b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:43.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.258+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 shutdown_connections 2024-01-15T17:55:43.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.259+0000 7ff149400640 1 -- 172.21.15.26:0/2691167335 wait complete. 2024-01-15T17:55:43.283 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:43.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.502+0000 7f4342800640 1 Processor -- start 2024-01-15T17:55:43.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.502+0000 7f4342800640 1 -- start start 2024-01-15T17:55:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.504+0000 7f4342800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f433c160f80 0x7f433c161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.504+0000 7f4342800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ced50 0x7f433c0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.504+0000 7f4342800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433c1618a0 con 0x7f433c0ce8f0 2024-01-15T17:55:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.504+0000 7f4342800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c0cf1e0 con 0x7f433c160f80 2024-01-15T17:55:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.504+0000 7f4342800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c0d0de0 con 0x7f433c0ced50 2024-01-15T17:55:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.504+0000 7f433aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f433c0ce8f0 0x7f433c0cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57324/0 (socket says 172.21.15.26:57324) 2024-01-15T17:55:43.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.504+0000 7f433aa00640 1 -- 172.21.15.26:0/451401652 learned_addr learned my addr 172.21.15.26:0/451401652 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:43.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.504+0000 7f433b400640 1 --2- 172.21.15.26:0/451401652 >> v2:172.21.15.124:3300/0 conn(0x7f433c160f80 0x7f433c161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433be00640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ced50 0x7f433c0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882390897 0 0) 0x7f433c1618a0 con 0x7f433c0ce8f0 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f431c003520 con 0x7f433c0ce8f0 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 844637894 0 0) 0x7f431c003520 con 0x7f433c0ce8f0 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ced50 msgr2=0x7f433c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ced50 0x7f433c0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 >> v2:172.21.15.124:3300/0 conn(0x7f433c160f80 msgr2=0x7f433c161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 --2- 172.21.15.26:0/451401652 >> v2:172.21.15.124:3300/0 conn(0x7f433c160f80 0x7f433c161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0d10c0 con 0x7f433c0ce8f0 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f4324001670 con 0x7f433c0ce8f0 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 >> v1:172.21.15.26:6789/0 conn(0x7f433c0ce8f0 legacy=0x7f433c0cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f431c003970 0x7f431c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ced50 0x7f431c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c004d00 0x7f431c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c0cf1e0 con 0x7f431c003970 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c0d0de0 con 0x7f431c004d00 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.505+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c00b980 con 0x7f433c0ced50 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.506+0000 7f433b400640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f431c003970 0x7f431c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.506+0000 7f433aa00640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ced50 0x7f431c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.506+0000 7f433be00640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c004d00 0x7f431c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.506+0000 7f433b400640 1 -- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ced50 msgr2=0x7f431c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.506+0000 7f433b400640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ced50 0x7f431c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.506+0000 7f433b400640 1 -- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c004d00 msgr2=0x7f431c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.506+0000 7f433b400640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c004d00 0x7f431c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.506+0000 7f433b400640 1 -- 172.21.15.26:0/451401652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c003520 con 0x7f431c003970 2024-01-15T17:55:43.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.506+0000 7f433aa00640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ced50 0x7f431c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.507+0000 7f433b400640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f431c003970 0x7f431c003d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f432c009af0 tx=0x7f431c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5966b8d3687eafb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.508+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c02c710 con 0x7f431c003970 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.508+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c02c870 con 0x7f431c003970 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.508+0000 7f433a000640 1 -- 172.21.15.26:0/451401652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c02cc00 con 0x7f431c003970 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.508+0000 7f4342800640 1 -- 172.21.15.26:0/451401652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f431c003970 msgr2=0x7f431c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.508+0000 7f4342800640 1 --2- 172.21.15.26:0/451401652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f431c003970 0x7f431c003d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f432c009af0 tx=0x7f431c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.508+0000 7f433be00640 1 -- 172.21.15.26:0/451401652 reap_dead start 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.509+0000 7f4342800640 1 -- 172.21.15.26:0/451401652 shutdown_connections 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.509+0000 7f4342800640 1 -- 172.21.15.26:0/451401652 >> 172.21.15.26:0/451401652 conn(0x7f433c09c550 msgr2=0x7f433c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.509+0000 7f4342800640 1 -- 172.21.15.26:0/451401652 shutdown_connections 2024-01-15T17:55:43.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.509+0000 7f4342800640 1 -- 172.21.15.26:0/451401652 wait complete. 2024-01-15T17:55:43.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.510+0000 7f4342800640 1 Processor -- start 2024-01-15T17:55:43.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.511+0000 7f4342800640 1 -- start start 2024-01-15T17:55:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f4342800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c003970 0x7f433c06cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f4342800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f433c155b10 0x7f433c151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f4342800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ca2e0 0x7f433c0c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f4342800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c151980 con 0x7f433c155b10 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f4342800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c0ca7e0 con 0x7f431c003970 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f4342800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c0caac0 con 0x7f433c0ca2e0 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f433c155b10 0x7f433c151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f433c155b10 0x7f433c151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60860/0 (socket says 172.21.15.26:60860) 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433aa00640 1 -- 172.21.15.26:0/2565507082 learned_addr learned my addr 172.21.15.26:0/2565507082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433b400640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c003970 0x7f433c06cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433be00640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ca2e0 0x7f433c0c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433aa00640 1 -- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ca2e0 msgr2=0x7f433c0c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433aa00640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ca2e0 0x7f433c0c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433aa00640 1 -- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c003970 msgr2=0x7f433c06cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433aa00640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c003970 0x7f433c06cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.512+0000 7f433aa00640 1 -- 172.21.15.26:0/2565507082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c0cada0 con 0x7f433c155b10 2024-01-15T17:55:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.513+0000 7f433aa00640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f433c155b10 0x7f433c151440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f432400d760 tx=0x7f432400dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.513+0000 7f4333400640 1 -- 172.21.15.26:0/2565507082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43240074e0 con 0x7f433c155b10 2024-01-15T17:55:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.513+0000 7f4333400640 1 -- 172.21.15.26:0/2565507082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4324007640 con 0x7f433c155b10 2024-01-15T17:55:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.513+0000 7f4333400640 1 -- 172.21.15.26:0/2565507082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324004a00 con 0x7f433c155b10 2024-01-15T17:55:43.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.513+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c0cb080 con 0x7f433c155b10 2024-01-15T17:55:43.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.513+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c0cd020 con 0x7f433c155b10 2024-01-15T17:55:43.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.515+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433c0675d0 con 0x7f433c155b10 2024-01-15T17:55:43.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.515+0000 7f4333400640 1 -- 172.21.15.26:0/2565507082 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4324004b60 con 0x7f433c155b10 2024-01-15T17:55:43.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.517+0000 7f4333400640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f431406b2c0 0x7f431406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.517+0000 7f4333400640 1 -- 172.21.15.26:0/2565507082 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43240894e0 con 0x7f433c155b10 2024-01-15T17:55:43.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.518+0000 7f433b400640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f431406b2c0 0x7f431406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.519+0000 7f433b400640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f431406b2c0 0x7f431406d780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f432c036000 tx=0x7f432c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:43.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.522+0000 7f4333400640 1 -- 172.21.15.26:0/2565507082 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4324054350 con 0x7f433c155b10 2024-01-15T17:55:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.715+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f433c0d3920 con 0x7f433c155b10 2024-01-15T17:55:43.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.717+0000 7f4333400640 1 -- 172.21.15.26:0/2565507082 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4324057970 con 0x7f433c155b10 2024-01-15T17:55:43.721 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f431406b2c0 msgr2=0x7f431406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f431406b2c0 0x7f431406d780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f432c036000 tx=0x7f432c030000 comp rx=0 tx=0).stop 2024-01-15T17:55:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f433c155b10 msgr2=0x7f433c151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f433c155b10 0x7f433c151440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f432400d760 tx=0x7f432400dc20 comp rx=0 tx=0).stop 2024-01-15T17:55:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 shutdown_connections 2024-01-15T17:55:43.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f433c0ca2e0 0x7f433c0c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f431406b2c0 0x7f431406d780 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f433c155b10 0x7f433c151440 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 --2- 172.21.15.26:0/2565507082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f431c003970 0x7f433c06cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 >> 172.21.15.26:0/2565507082 conn(0x7f433c09c550 msgr2=0x7f433c0b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 shutdown_connections 2024-01-15T17:55:43.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.719+0000 7f4342800640 1 -- 172.21.15.26:0/2565507082 wait complete. 2024-01-15T17:55:43.743 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:43.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.964+0000 7efd5b400640 1 Processor -- start 2024-01-15T17:55:43.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.965+0000 7efd5b400640 1 -- start start 2024-01-15T17:55:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd54070a30 0x7efd5415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5b400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd5415f4e0 con 0x7efd540adf20 2024-01-15T17:55:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5b400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd54070ea0 con 0x7efd540ccb50 2024-01-15T17:55:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5b400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5415c450 con 0x7efd54070a30 2024-01-15T17:55:43.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5a000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efd540adf20 0x7efd540cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57334/0 (socket says 172.21.15.26:57334) 2024-01-15T17:55:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5a000640 1 -- 172.21.15.26:0/612875793 learned_addr learned my addr 172.21.15.26:0/612875793 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5aa00640 1 --2- 172.21.15.26:0/612875793 >> v2:172.21.15.26:3300/0 conn(0x7efd54070a30 0x7efd5415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd58c00640 1 -- 172.21.15.26:0/612875793 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297158247 0 0) 0x7efd5415f4e0 con 0x7efd540adf20 2024-01-15T17:55:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd58c00640 1 -- 172.21.15.26:0/612875793 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd34003520 con 0x7efd540adf20 2024-01-15T17:55:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5aa00640 1 -- 172.21.15.26:0/612875793 >> v1:172.21.15.124:6789/0 conn(0x7efd540ccb50 legacy=0x7efd54143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5aa00640 1 -- 172.21.15.26:0/612875793 >> v1:172.21.15.26:6789/0 conn(0x7efd540adf20 legacy=0x7efd540cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.967+0000 7efd5aa00640 1 -- 172.21.15.26:0/612875793 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd54071130 con 0x7efd54070a30 2024-01-15T17:55:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.969+0000 7efd5aa00640 1 --2- 172.21.15.26:0/612875793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54070a30 0x7efd5415bf10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009d90 tx=0x7efd4c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd4e9a0e719b050 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.969+0000 7efd58c00640 1 -- 172.21.15.26:0/612875793 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c02cbd0 con 0x7efd54070a30 2024-01-15T17:55:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.969+0000 7efd58c00640 1 -- 172.21.15.26:0/612875793 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c02cd30 con 0x7efd54070a30 2024-01-15T17:55:43.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.969+0000 7efd58c00640 1 -- 172.21.15.26:0/612875793 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c02cea0 con 0x7efd54070a30 2024-01-15T17:55:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.969+0000 7efd5b400640 1 -- 172.21.15.26:0/612875793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54070a30 msgr2=0x7efd5415bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.969+0000 7efd5b400640 1 --2- 172.21.15.26:0/612875793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54070a30 0x7efd5415bf10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009d90 tx=0x7efd4c02be20 comp rx=0 tx=0).stop 2024-01-15T17:55:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.970+0000 7efd5b400640 1 -- 172.21.15.26:0/612875793 shutdown_connections 2024-01-15T17:55:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.970+0000 7efd5b400640 1 --2- 172.21.15.26:0/612875793 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd54070a30 0x7efd5415bf10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.970+0000 7efd5b400640 1 -- 172.21.15.26:0/612875793 >> 172.21.15.26:0/612875793 conn(0x7efd5406a7d0 msgr2=0x7efd5406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.970+0000 7efd5b400640 1 -- 172.21.15.26:0/612875793 shutdown_connections 2024-01-15T17:55:43.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.970+0000 7efd5b400640 1 -- 172.21.15.26:0/612875793 wait complete. 2024-01-15T17:55:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.972+0000 7efd5b400640 1 Processor -- start 2024-01-15T17:55:43.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.972+0000 7efd5b400640 1 -- start start 2024-01-15T17:55:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd5b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd54070a30 0x7efd541f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd5b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd540adf20 0x7efd5415a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd5b400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd540ccb50 0x7efd5413ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd5b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5415ae90 con 0x7efd540adf20 2024-01-15T17:55:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd5b400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5413f830 con 0x7efd540ccb50 2024-01-15T17:55:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd5b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd541f3270 con 0x7efd54070a30 2024-01-15T17:55:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd59600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd540adf20 0x7efd5415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd59600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd540adf20 0x7efd5415a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60876/0 (socket says 172.21.15.26:60876) 2024-01-15T17:55:43.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd59600640 1 -- 172.21.15.26:0/2851214033 learned_addr learned my addr 172.21.15.26:0/2851214033 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:43.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.973+0000 7efd5a000640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd54070a30 0x7efd541f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd59600640 1 -- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd54070a30 msgr2=0x7efd541f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd59600640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd54070a30 0x7efd541f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd59600640 1 -- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd540ccb50 msgr2=0x7efd5413ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd59600640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd540ccb50 0x7efd5413ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd59600640 1 -- 172.21.15.26:0/2851214033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c009a70 con 0x7efd540adf20 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd59600640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd540adf20 0x7efd5415a950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c400 tx=0x7efd3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd52000640 1 -- 172.21.15.26:0/2851214033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c011790 con 0x7efd540adf20 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd52000640 1 -- 172.21.15.26:0/2851214033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c011930 con 0x7efd540adf20 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd52000640 1 -- 172.21.15.26:0/2851214033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c011ce0 con 0x7efd540adf20 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd541f3550 con 0x7efd540adf20 2024-01-15T17:55:43.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.974+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5413fb10 con 0x7efd540adf20 2024-01-15T17:55:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.975+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd541f3a70 con 0x7efd540adf20 2024-01-15T17:55:43.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.976+0000 7efd52000640 1 -- 172.21.15.26:0/2851214033 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7efd3c01c460 con 0x7efd540adf20 2024-01-15T17:55:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.977+0000 7efd52000640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd2c06b2c0 0x7efd2c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:43.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.977+0000 7efd52000640 1 -- 172.21.15.26:0/2851214033 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd3c089a60 con 0x7efd540adf20 2024-01-15T17:55:43.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.978+0000 7efd5a000640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd2c06b2c0 0x7efd2c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:43.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.979+0000 7efd5a000640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd2c06b2c0 0x7efd2c06d780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd440011e0 tx=0x7efd44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:43.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:43.982+0000 7efd52000640 1 -- 172.21.15.26:0/2851214033 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd541f3a70 con 0x7efd540adf20 2024-01-15T17:55:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.174+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd541f3a70 con 0x7efd540adf20 2024-01-15T17:55:44.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.175+0000 7efd52000640 1 -- 172.21.15.26:0/2851214033 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd541f3a70 con 0x7efd540adf20 2024-01-15T17:55:44.179 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.178+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd2c06b2c0 msgr2=0x7efd2c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.178+0000 7efd5b400640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd2c06b2c0 0x7efd2c06d780 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd440011e0 tx=0x7efd44009040 comp rx=0 tx=0).stop 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.178+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd540adf20 msgr2=0x7efd5415a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.178+0000 7efd5b400640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd540adf20 0x7efd5415a950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd3c00c400 tx=0x7efd3c00c8c0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.179+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 shutdown_connections 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.179+0000 7efd5b400640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd2c06b2c0 0x7efd2c06d780 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.179+0000 7efd5b400640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd540ccb50 0x7efd5413ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.179+0000 7efd5b400640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd540adf20 0x7efd5415a950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.179+0000 7efd5b400640 1 --2- 172.21.15.26:0/2851214033 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd54070a30 0x7efd541f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.179+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 >> 172.21.15.26:0/2851214033 conn(0x7efd5406a7d0 msgr2=0x7efd5406c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.179+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 shutdown_connections 2024-01-15T17:55:44.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.179+0000 7efd5b400640 1 -- 172.21.15.26:0/2851214033 wait complete. 2024-01-15T17:55:44.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:44.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.422+0000 7f162b200640 1 Processor -- start 2024-01-15T17:55:44.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.423+0000 7f162b200640 1 -- start start 2024-01-15T17:55:44.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.424+0000 7f162b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f162c0cc500 0x7f162c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.424+0000 7f162b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162c070e50 con 0x7f162c143c40 2024-01-15T17:55:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.424+0000 7f162b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162c144020 con 0x7f162c06d160 2024-01-15T17:55:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.424+0000 7f162b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c0723c0 con 0x7f162c0cc500 2024-01-15T17:55:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.425+0000 7f1629e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f162c143c40 0x7f162c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57346/0 (socket says 172.21.15.26:57346) 2024-01-15T17:55:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.425+0000 7f1629e00640 1 -- 172.21.15.26:0/2556555089 learned_addr learned my addr 172.21.15.26:0/2556555089 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.425+0000 7f1621200640 1 --2- 172.21.15.26:0/2556555089 >> v2:172.21.15.26:3300/0 conn(0x7f162c0cc500 0x7f162c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.425+0000 7f1621200640 1 -- 172.21.15.26:0/2556555089 >> v1:172.21.15.124:6789/0 conn(0x7f162c06d160 legacy=0x7f162c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.425+0000 7f1621200640 1 -- 172.21.15.26:0/2556555089 >> v1:172.21.15.26:6789/0 conn(0x7f162c143c40 legacy=0x7f162c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.425+0000 7f1621200640 1 -- 172.21.15.26:0/2556555089 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c071e00 con 0x7f162c0cc500 2024-01-15T17:55:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.425+0000 7f1629400640 1 -- 172.21.15.26:0/2556555089 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821561980 0 0) 0x7f162c070e50 con 0x7f162c143c40 2024-01-15T17:55:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.427+0000 7f1621200640 1 --2- 172.21.15.26:0/2556555089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c0cc500 0x7f162c0cc990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1614009d90 tx=0x7f161402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5345d4c74e12f42f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:44.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.427+0000 7f1629400640 1 -- 172.21.15.26:0/2556555089 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161402cc60 con 0x7f162c0cc500 2024-01-15T17:55:44.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.427+0000 7f1629400640 1 -- 172.21.15.26:0/2556555089 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161402cdc0 con 0x7f162c0cc500 2024-01-15T17:55:44.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.427+0000 7f1629400640 1 -- 172.21.15.26:0/2556555089 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614034660 con 0x7f162c0cc500 2024-01-15T17:55:44.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.428+0000 7f162b200640 1 -- 172.21.15.26:0/2556555089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c0cc500 msgr2=0x7f162c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.428+0000 7f162b200640 1 --2- 172.21.15.26:0/2556555089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c0cc500 0x7f162c0cc990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1614009d90 tx=0x7f161402beb0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.428+0000 7f162b200640 1 -- 172.21.15.26:0/2556555089 shutdown_connections 2024-01-15T17:55:44.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.428+0000 7f162b200640 1 --2- 172.21.15.26:0/2556555089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c0cc500 0x7f162c0cc990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.428+0000 7f162b200640 1 -- 172.21.15.26:0/2556555089 >> 172.21.15.26:0/2556555089 conn(0x7f162c06a800 msgr2=0x7f162c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:44.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.428+0000 7f162b200640 1 -- 172.21.15.26:0/2556555089 shutdown_connections 2024-01-15T17:55:44.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.428+0000 7f162b200640 1 -- 172.21.15.26:0/2556555089 wait complete. 2024-01-15T17:55:44.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.430+0000 7f162b200640 1 Processor -- start 2024-01-15T17:55:44.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.430+0000 7f162b200640 1 -- start start 2024-01-15T17:55:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.431+0000 7f162b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f162c06d160 0x7f162c1f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.431+0000 7f162b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f162c0cc500 0x7f162c15a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c143c40 0x7f162c0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c1f38d0 con 0x7f162c143c40 2024-01-15T17:55:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0cab00 con 0x7f162c0cc500 2024-01-15T17:55:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c0c5540 con 0x7f162c06d160 2024-01-15T17:55:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c143c40 0x7f162c0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c143c40 0x7f162c0c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60902/0 (socket says 172.21.15.26:60902) 2024-01-15T17:55:44.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162a800640 1 -- 172.21.15.26:0/2572927651 learned_addr learned my addr 172.21.15.26:0/2572927651 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:44.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f1621200640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f162c0cc500 0x7f162c15a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f1629e00640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f162c06d160 0x7f162c1f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162a800640 1 -- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f162c06d160 msgr2=0x7f162c1f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162a800640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f162c06d160 0x7f162c1f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162a800640 1 -- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f162c0cc500 msgr2=0x7f162c15a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162a800640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f162c0cc500 0x7f162c15a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.432+0000 7f162a800640 1 -- 172.21.15.26:0/2572927651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614009a70 con 0x7f162c143c40 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.433+0000 7f162a800640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c143c40 0x7f162c0c49d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f162400c390 tx=0x7f162400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.433+0000 7f1622a00640 1 -- 172.21.15.26:0/2572927651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16240052d0 con 0x7f162c143c40 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.433+0000 7f1622a00640 1 -- 172.21.15.26:0/2572927651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624005430 con 0x7f162c143c40 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.433+0000 7f1622a00640 1 -- 172.21.15.26:0/2572927651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624003810 con 0x7f162c143c40 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.433+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c13c930 con 0x7f162c143c40 2024-01-15T17:55:44.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.433+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c13cef0 con 0x7f162c143c40 2024-01-15T17:55:44.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.435+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c067450 con 0x7f162c143c40 2024-01-15T17:55:44.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.435+0000 7f1622a00640 1 -- 172.21.15.26:0/2572927651 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1624003970 con 0x7f162c143c40 2024-01-15T17:55:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.436+0000 7f1622a00640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f160406b2c0 0x7f160406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.436+0000 7f1622a00640 1 -- 172.21.15.26:0/2572927651 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1624089190 con 0x7f162c143c40 2024-01-15T17:55:44.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.437+0000 7f1629e00640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f160406b2c0 0x7f160406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.438+0000 7f1629e00640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f160406b2c0 0x7f160406d780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f161c000f60 tx=0x7f161c007bb0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:44.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.441+0000 7f1622a00640 1 -- 172.21.15.26:0/2572927651 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1624053f60 con 0x7f162c143c40 2024-01-15T17:55:44.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.637+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f162c0ceb30 con 0x7f162c143c40 2024-01-15T17:55:44.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.639+0000 7f1622a00640 1 -- 172.21.15.26:0/2572927651 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f1624057580 con 0x7f162c143c40 2024-01-15T17:55:44.643 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:44.644 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.641+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f160406b2c0 msgr2=0x7f160406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.641+0000 7f162b200640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f160406b2c0 0x7f160406d780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f161c000f60 tx=0x7f161c007bb0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.641+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c143c40 msgr2=0x7f162c0c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.641+0000 7f162b200640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c143c40 0x7f162c0c49d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f162400c390 tx=0x7f162400c850 comp rx=0 tx=0).stop 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.641+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 shutdown_connections 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.641+0000 7f162b200640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f160406b2c0 0x7f160406d780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.641+0000 7f162b200640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f162c143c40 0x7f162c0c49d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.641+0000 7f162b200640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f162c0cc500 0x7f162c15a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.641+0000 7f162b200640 1 --2- 172.21.15.26:0/2572927651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f162c06d160 0x7f162c1f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.642+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 >> 172.21.15.26:0/2572927651 conn(0x7f162c06a800 msgr2=0x7f162c0addd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:44.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.642+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 shutdown_connections 2024-01-15T17:55:44.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.642+0000 7f162b200640 1 -- 172.21.15.26:0/2572927651 wait complete. 2024-01-15T17:55:44.664 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:44.665 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.888+0000 7f5f8de00640 1 Processor -- start 2024-01-15T17:55:44.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.889+0000 7f5f8de00640 1 -- start start 2024-01-15T17:55:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.891+0000 7f5f8de00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5f8813e360 0x7f5f880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.891+0000 7f5f8de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8815f9c0 0x7f5f880cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.891+0000 7f5f8de00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8815c5f0 con 0x7f5f8813df80 2024-01-15T17:55:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.891+0000 7f5f8de00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8806c470 con 0x7f5f8813e360 2024-01-15T17:55:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.891+0000 7f5f8de00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8806c750 con 0x7f5f8815f9c0 2024-01-15T17:55:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.891+0000 7f5f86a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5f8813df80 0x7f5f8815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57362/0 (socket says 172.21.15.26:57362) 2024-01-15T17:55:44.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.891+0000 7f5f86a00640 1 -- 172.21.15.26:0/1792955944 learned_addr learned my addr 172.21.15.26:0/1792955944 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.891+0000 7f5f7de00640 1 --2- 172.21.15.26:0/1792955944 >> v2:172.21.15.124:3300/0 conn(0x7f5f8813e360 0x7f5f880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f87400640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8815f9c0 0x7f5f880cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217236779 0 0) 0x7f5f8815c5f0 con 0x7f5f8813df80 2024-01-15T17:55:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f68003520 con 0x7f5f8813df80 2024-01-15T17:55:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3357421279 0 0) 0x7f5f68003520 con 0x7f5f8813df80 2024-01-15T17:55:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8815f9c0 msgr2=0x7f5f880cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8815f9c0 0x7f5f880cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 >> v2:172.21.15.124:3300/0 conn(0x7f5f8813e360 msgr2=0x7f5f880cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 --2- 172.21.15.26:0/1792955944 >> v2:172.21.15.124:3300/0 conn(0x7f5f8813e360 0x7f5f880cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8806ca30 con 0x7f5f8813df80 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f5f78001690 con 0x7f5f8813df80 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f7de00640 1 --2- 172.21.15.26:0/1792955944 >> v2:172.21.15.124:3300/0 conn(0x7f5f8813e360 0x7f5f880cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 >> v1:172.21.15.26:6789/0 conn(0x7f5f8813df80 legacy=0x7f5f8815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f68003970 0x7f5f68003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8815f9c0 0x7f5f680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f68004d00 0x7f5f68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8806c470 con 0x7f5f68003970 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8806c750 con 0x7f5f68004d00 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6800ba30 con 0x7f5f8815f9c0 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.892+0000 7f5f86a00640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f68003970 0x7f5f68003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.893+0000 7f5f86a00640 1 -- 172.21.15.26:0/1792955944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8815f9c0 msgr2=0x7f5f680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:44.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.893+0000 7f5f86a00640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8815f9c0 0x7f5f680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.893+0000 7f5f86a00640 1 -- 172.21.15.26:0/1792955944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f68004d00 msgr2=0x7f5f68005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.893+0000 7f5f86a00640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f68004d00 0x7f5f68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.893+0000 7f5f86a00640 1 -- 172.21.15.26:0/1792955944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f68003520 con 0x7f5f68003970 2024-01-15T17:55:44.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.894+0000 7f5f86a00640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f68003970 0x7f5f68003d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c240 tx=0x7f5f680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2079f9434dcffafa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:44.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.897+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f78005ad0 con 0x7f5f68003970 2024-01-15T17:55:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.897+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f78005c30 con 0x7f5f68003970 2024-01-15T17:55:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.897+0000 7f5f86000640 1 -- 172.21.15.26:0/1792955944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7802c8a0 con 0x7f5f68003970 2024-01-15T17:55:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.897+0000 7f5f8de00640 1 -- 172.21.15.26:0/1792955944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f68003970 msgr2=0x7f5f68003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.897+0000 7f5f8de00640 1 --2- 172.21.15.26:0/1792955944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f68003970 0x7f5f68003d90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800c240 tx=0x7f5f680057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.897+0000 7f5f87400640 1 -- 172.21.15.26:0/1792955944 reap_dead start 2024-01-15T17:55:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.897+0000 7f5f8de00640 1 -- 172.21.15.26:0/1792955944 shutdown_connections 2024-01-15T17:55:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.898+0000 7f5f8de00640 1 -- 172.21.15.26:0/1792955944 >> 172.21.15.26:0/1792955944 conn(0x7f5f8806a800 msgr2=0x7f5f8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.898+0000 7f5f8de00640 1 -- 172.21.15.26:0/1792955944 shutdown_connections 2024-01-15T17:55:44.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.898+0000 7f5f8de00640 1 -- 172.21.15.26:0/1792955944 wait complete. 2024-01-15T17:55:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.899+0000 7f5f8de00640 1 Processor -- start 2024-01-15T17:55:44.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.900+0000 7f5f8de00640 1 -- start start 2024-01-15T17:55:44.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.900+0000 7f5f8de00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f68003970 0x7f5f8815ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.900+0000 7f5f8de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8813d910 0x7f5f8813d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f8de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f880b3290 0x7f5f880b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f8de00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f881f4910 con 0x7f5f8813d910 2024-01-15T17:55:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f8de00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8814bcf0 con 0x7f5f68003970 2024-01-15T17:55:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f8de00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880b3c50 con 0x7f5f880b3290 2024-01-15T17:55:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f7de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8813d910 0x7f5f8813d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f7de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8813d910 0x7f5f8813d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60928/0 (socket says 172.21.15.26:60928) 2024-01-15T17:55:44.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f7de00640 1 -- 172.21.15.26:0/3083703652 learned_addr learned my addr 172.21.15.26:0/3083703652 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f87400640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f880b3290 0x7f5f880b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f86a00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f68003970 0x7f5f8815ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f7de00640 1 -- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f880b3290 msgr2=0x7f5f880b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f7de00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f880b3290 0x7f5f880b3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f7de00640 1 -- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f68003970 msgr2=0x7f5f8815ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:44.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f7de00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f68003970 0x7f5f8815ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.901+0000 7f5f7de00640 1 -- 172.21.15.26:0/3083703652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8815b040 con 0x7f5f8813d910 2024-01-15T17:55:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.902+0000 7f5f7de00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8813d910 0x7f5f8813d1e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000e7e0 tx=0x7f5f7000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.902+0000 7f5f7fe00640 1 -- 172.21.15.26:0/3083703652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000c870 con 0x7f5f8813d910 2024-01-15T17:55:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.902+0000 7f5f7fe00640 1 -- 172.21.15.26:0/3083703652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7000ee90 con 0x7f5f8813d910 2024-01-15T17:55:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.902+0000 7f5f7fe00640 1 -- 172.21.15.26:0/3083703652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000cc10 con 0x7f5f8813d910 2024-01-15T17:55:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.902+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f881f3d20 con 0x7f5f8813d910 2024-01-15T17:55:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.902+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8813d720 con 0x7f5f8813d910 2024-01-15T17:55:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.903+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f880675d0 con 0x7f5f8813d910 2024-01-15T17:55:44.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.904+0000 7f5f7fe00640 1 -- 172.21.15.26:0/3083703652 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5f70013410 con 0x7f5f8813d910 2024-01-15T17:55:44.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.905+0000 7f5f7fe00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5f6006b2c0 0x7f5f6006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:44.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.905+0000 7f5f7fe00640 1 -- 172.21.15.26:0/3083703652 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f70089950 con 0x7f5f8813d910 2024-01-15T17:55:44.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.906+0000 7f5f86a00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5f6006b2c0 0x7f5f6006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:44.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.907+0000 7f5f86a00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5f6006b2c0 0x7f5f6006d780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f78034000 tx=0x7f5f78037040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:44.910+0000 7f5f7fe00640 1 -- 172.21.15.26:0/3083703652 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f70058240 con 0x7f5f8813d910 2024-01-15T17:55:45.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.104+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f8806df00 con 0x7f5f8813d910 2024-01-15T17:55:45.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.104+0000 7f5f7fe00640 1 -- 172.21.15.26:0/3083703652 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f700210a0 con 0x7f5f8813d910 2024-01-15T17:55:45.109 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:45.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5f6006b2c0 msgr2=0x7f5f6006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5f6006b2c0 0x7f5f6006d780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5f78034000 tx=0x7f5f78037040 comp rx=0 tx=0).stop 2024-01-15T17:55:45.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8813d910 msgr2=0x7f5f8813d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8813d910 0x7f5f8813d1e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000e7e0 tx=0x7f5f7000eca0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 shutdown_connections 2024-01-15T17:55:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f880b3290 0x7f5f880b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5f6006b2c0 0x7f5f6006d780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8813d910 0x7f5f8813d1e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 --2- 172.21.15.26:0/3083703652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f68003970 0x7f5f8815ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 >> 172.21.15.26:0/3083703652 conn(0x7f5f8806a800 msgr2=0x7f5f8815e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.107+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 shutdown_connections 2024-01-15T17:55:45.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.108+0000 7f5f8de00640 1 -- 172.21.15.26:0/3083703652 wait complete. 2024-01-15T17:55:45.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:45.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.356+0000 7f1210e00640 1 Processor -- start 2024-01-15T17:55:45.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.357+0000 7f1210e00640 1 -- start start 2024-01-15T17:55:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.358+0000 7f1210e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f120c0ccb50 0x7f120c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.358+0000 7f1210e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c15f4e0 con 0x7f120c15bf10 2024-01-15T17:55:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.358+0000 7f1210e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c15c3c0 con 0x7f120c0adf20 2024-01-15T17:55:45.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.358+0000 7f1210e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c15f970 con 0x7f120c0ccb50 2024-01-15T17:55:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.359+0000 7f120be00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f120c15bf10 0x7f120c143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57376/0 (socket says 172.21.15.26:57376) 2024-01-15T17:55:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.359+0000 7f120be00640 1 -- 172.21.15.26:0/4026113783 learned_addr learned my addr 172.21.15.26:0/4026113783 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.359+0000 7f120aa00640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f120c0ccb50 0x7f120c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.359+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688587902 0 0) 0x7f120c15f4e0 con 0x7f120c15bf10 2024-01-15T17:55:45.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.359+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11ec003520 con 0x7f120c15bf10 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3100026215 0 0) 0x7f11ec003520 con 0x7f120c15bf10 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f120c0ccb50 msgr2=0x7f120c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f120c0ccb50 0x7f120c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 >> v1:172.21.15.124:6789/0 conn(0x7f120c0adf20 legacy=0x7f120c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c071130 con 0x7f120c15bf10 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f1204001670 con 0x7f120c15bf10 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 >> v1:172.21.15.26:6789/0 conn(0x7f120c15bf10 legacy=0x7f120c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec0039a0 0x7f11ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f120c0ccb50 0x7f11ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11ec004d00 0x7f11ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c15f970 con 0x7f11ec0039a0 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec00bab0 con 0x7f11ec004d00 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec00bc70 con 0x7f120c0ccb50 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.360+0000 7f120b400640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec0039a0 0x7f11ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.361+0000 7f120be00640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11ec004d00 0x7f11ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.361+0000 7f120aa00640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f120c0ccb50 0x7f11ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.361+0000 7f120b400640 1 -- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f120c0ccb50 msgr2=0x7f11ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.361+0000 7f120b400640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f120c0ccb50 0x7f11ec0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.361+0000 7f120b400640 1 -- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11ec004d00 msgr2=0x7f11ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.361+0000 7f120b400640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11ec004d00 0x7f11ec005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.361+0000 7f120b400640 1 -- 172.21.15.26:0/4026113783 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec003520 con 0x7f11ec0039a0 2024-01-15T17:55:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.362+0000 7f120b400640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec0039a0 0x7f11ec003dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11fc001e50 tx=0x7f11ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec348ee82c55cb7c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.362+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc003b70 con 0x7f11ec0039a0 2024-01-15T17:55:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.362+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc003cd0 con 0x7f11ec0039a0 2024-01-15T17:55:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.362+0000 7f120a000640 1 -- 172.21.15.26:0/4026113783 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc005310 con 0x7f11ec0039a0 2024-01-15T17:55:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.363+0000 7f1210e00640 1 -- 172.21.15.26:0/4026113783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec0039a0 msgr2=0x7f11ec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.363+0000 7f1210e00640 1 --2- 172.21.15.26:0/4026113783 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11ec0039a0 0x7f11ec003dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f11fc001e50 tx=0x7f11ec0057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.363+0000 7f120be00640 1 -- 172.21.15.26:0/4026113783 reap_dead start 2024-01-15T17:55:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.363+0000 7f1210e00640 1 -- 172.21.15.26:0/4026113783 shutdown_connections 2024-01-15T17:55:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.363+0000 7f1210e00640 1 -- 172.21.15.26:0/4026113783 >> 172.21.15.26:0/4026113783 conn(0x7f120c06a7d0 msgr2=0x7f120c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.363+0000 7f1210e00640 1 -- 172.21.15.26:0/4026113783 shutdown_connections 2024-01-15T17:55:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.363+0000 7f1210e00640 1 -- 172.21.15.26:0/4026113783 wait complete. 2024-01-15T17:55:45.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.365+0000 7f1210e00640 1 Processor -- start 2024-01-15T17:55:45.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.365+0000 7f1210e00640 1 -- start start 2024-01-15T17:55:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.366+0000 7f1210e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11ec0039a0 0x7f120c141c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.366+0000 7f1210e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f120c0ac8d0 0x7f120c0acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.366+0000 7f1210e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c135de0 0x7f120c136260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.366+0000 7f1210e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0ad250 con 0x7f120c135de0 2024-01-15T17:55:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.366+0000 7f1210e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c1367a0 con 0x7f120c0ac8d0 2024-01-15T17:55:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.366+0000 7f1210e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c1308d0 con 0x7f11ec0039a0 2024-01-15T17:55:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.366+0000 7f120be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c135de0 0x7f120c136260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c135de0 0x7f120c136260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60950/0 (socket says 172.21.15.26:60950) 2024-01-15T17:55:45.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120be00640 1 -- 172.21.15.26:0/4049983543 learned_addr learned my addr 172.21.15.26:0/4049983543 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120aa00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f120c0ac8d0 0x7f120c0acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120b400640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11ec0039a0 0x7f120c141c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120be00640 1 -- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11ec0039a0 msgr2=0x7f120c141c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120be00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11ec0039a0 0x7f120c141c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120be00640 1 -- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f120c0ac8d0 msgr2=0x7f120c0acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120be00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f120c0ac8d0 0x7f120c0acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120be00640 1 -- 172.21.15.26:0/4049983543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c0b0e00 con 0x7f120c135de0 2024-01-15T17:55:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.367+0000 7f120be00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c135de0 0x7f120c136260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f120400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.368+0000 7f1203400640 1 -- 172.21.15.26:0/4049983543 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204007230 con 0x7f120c135de0 2024-01-15T17:55:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.368+0000 7f1203400640 1 -- 172.21.15.26:0/4049983543 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1204007390 con 0x7f120c135de0 2024-01-15T17:55:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.368+0000 7f1203400640 1 -- 172.21.15.26:0/4049983543 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12040047e0 con 0x7f120c135de0 2024-01-15T17:55:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.368+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c0b10e0 con 0x7f120c135de0 2024-01-15T17:55:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.368+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c130e70 con 0x7f120c135de0 2024-01-15T17:55:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.369+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c0b5080 con 0x7f120c135de0 2024-01-15T17:55:45.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.370+0000 7f1203400640 1 -- 172.21.15.26:0/4049983543 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1204004940 con 0x7f120c135de0 2024-01-15T17:55:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.371+0000 7f1203400640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11e406b2c0 0x7f11e406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.371+0000 7f1203400640 1 -- 172.21.15.26:0/4049983543 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1204089840 con 0x7f120c135de0 2024-01-15T17:55:45.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.372+0000 7f120b400640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11e406b2c0 0x7f11e406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.373+0000 7f120b400640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11e406b2c0 0x7f11e406d780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f11fc034000 tx=0x7f11fc009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:45.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.375+0000 7f1203400640 1 -- 172.21.15.26:0/4049983543 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1204054620 con 0x7f120c135de0 2024-01-15T17:55:45.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.569+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f120c067450 con 0x7f120c135de0 2024-01-15T17:55:45.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.570+0000 7f1203400640 1 -- 172.21.15.26:0/4049983543 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1204057c40 con 0x7f120c135de0 2024-01-15T17:55:45.574 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11e406b2c0 msgr2=0x7f11e406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11e406b2c0 0x7f11e406d780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f11fc034000 tx=0x7f11fc009030 comp rx=0 tx=0).stop 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c135de0 msgr2=0x7f120c136260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c135de0 0x7f120c136260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1204000c00 tx=0x7f120400dc00 comp rx=0 tx=0).stop 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 shutdown_connections 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11e406b2c0 0x7f11e406d780 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f120c135de0 0x7f120c136260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f120c0ac8d0 0x7f120c0acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 --2- 172.21.15.26:0/4049983543 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11ec0039a0 0x7f120c141c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 >> 172.21.15.26:0/4049983543 conn(0x7f120c06a7d0 msgr2=0x7f120c0cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 shutdown_connections 2024-01-15T17:55:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.573+0000 7f1210e00640 1 -- 172.21.15.26:0/4049983543 wait complete. 2024-01-15T17:55:45.596 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.818+0000 7f8d75e00640 1 Processor -- start 2024-01-15T17:55:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.819+0000 7f8d75e00640 1 -- start start 2024-01-15T17:55:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.820+0000 7f8d75e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8d7015d4b0 0x7f8d700d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.820+0000 7f8d75e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700cf9a0 0x7f8d700cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.820+0000 7f8d75e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700d0320 con 0x7f8d700cddf0 2024-01-15T17:55:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.820+0000 7f8d75e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d700b2730 con 0x7f8d7015d4b0 2024-01-15T17:55:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.820+0000 7f8d75e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700d1ec0 con 0x7f8d700cf9a0 2024-01-15T17:55:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d75400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8d700cddf0 0x7f8d700ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57386/0 (socket says 172.21.15.26:57386) 2024-01-15T17:55:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d75400640 1 -- 172.21.15.26:0/2968429166 learned_addr learned my addr 172.21.15.26:0/2968429166 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d74a00640 1 --2- 172.21.15.26:0/2968429166 >> v2:172.21.15.124:3300/0 conn(0x7f8d7015d4b0 0x7f8d700d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6fe00640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700cf9a0 0x7f8d700cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447555858 0 0) 0x7f8d700d0320 con 0x7f8d700cddf0 2024-01-15T17:55:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d50003520 con 0x7f8d700cddf0 2024-01-15T17:55:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 154725603 0 0) 0x7f8d50003520 con 0x7f8d700cddf0 2024-01-15T17:55:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700cf9a0 msgr2=0x7f8d700cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6f400640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700cf9a0 0x7f8d700cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 >> v2:172.21.15.124:3300/0 conn(0x7f8d7015d4b0 msgr2=0x7f8d700d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6f400640 1 --2- 172.21.15.26:0/2968429166 >> v2:172.21.15.124:3300/0 conn(0x7f8d7015d4b0 0x7f8d700d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700b29d0 con 0x7f8d700cddf0 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.821+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8d68001670 con 0x7f8d700cddf0 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 >> v1:172.21.15.26:6789/0 conn(0x7f8d700cddf0 legacy=0x7f8d700ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6f400640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d50003970 0x7f8d50003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6f400640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d50004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6f400640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700cf9a0 0x7f8d50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700b2730 con 0x7f8d500047c0 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700d1ec0 con 0x7f8d50003970 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5000b980 con 0x7f8d700cf9a0 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6fe00640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d50004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d74a00640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d50003970 0x7f8d50003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d75400640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700cf9a0 0x7f8d50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6fe00640 1 -- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700cf9a0 msgr2=0x7f8d50005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6fe00640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700cf9a0 0x7f8d50005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6fe00640 1 -- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d50003970 msgr2=0x7f8d50003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6fe00640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d50003970 0x7f8d50003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.822+0000 7f8d6fe00640 1 -- 172.21.15.26:0/2968429166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50003520 con 0x7f8d500047c0 2024-01-15T17:55:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.823+0000 7f8d75400640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700cf9a0 0x7f8d50005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:45.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.824+0000 7f8d6fe00640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d50004c40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009ac0 tx=0x7f8d500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689eb22b730d76a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:45.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.824+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5802c6b0 con 0x7f8d500047c0 2024-01-15T17:55:45.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.824+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5802c810 con 0x7f8d500047c0 2024-01-15T17:55:45.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.824+0000 7f8d6f400640 1 -- 172.21.15.26:0/2968429166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5802cba0 con 0x7f8d500047c0 2024-01-15T17:55:45.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.824+0000 7f8d75e00640 1 -- 172.21.15.26:0/2968429166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 msgr2=0x7f8d50004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.824+0000 7f8d75e00640 1 --2- 172.21.15.26:0/2968429166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d50004c40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d58009ac0 tx=0x7f8d500057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.825+0000 7f8d75400640 1 -- 172.21.15.26:0/2968429166 reap_dead start 2024-01-15T17:55:45.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.825+0000 7f8d75e00640 1 -- 172.21.15.26:0/2968429166 shutdown_connections 2024-01-15T17:55:45.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.825+0000 7f8d75e00640 1 -- 172.21.15.26:0/2968429166 >> 172.21.15.26:0/2968429166 conn(0x7f8d700d3000 msgr2=0x7f8d700d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:45.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.825+0000 7f8d75e00640 1 -- 172.21.15.26:0/2968429166 shutdown_connections 2024-01-15T17:55:45.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.825+0000 7f8d75e00640 1 -- 172.21.15.26:0/2968429166 wait complete. 2024-01-15T17:55:45.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.827+0000 7f8d75e00640 1 Processor -- start 2024-01-15T17:55:45.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.827+0000 7f8d75e00640 1 -- start start 2024-01-15T17:55:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d75e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d7009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d75e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d7009f7c0 0x7f8d701f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d75e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700b0470 0x7f8d700b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d75e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700b0e30 con 0x7f8d500047c0 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d75e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700a0030 con 0x7f8d7009f7c0 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d75e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700a0310 con 0x7f8d700b0470 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d74a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d7009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d74a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d7009cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60972/0 (socket says 172.21.15.26:60972) 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d74a00640 1 -- 172.21.15.26:0/2933768650 learned_addr learned my addr 172.21.15.26:0/2933768650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d6fe00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d7009f7c0 0x7f8d701f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d74a00640 1 -- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700b0470 msgr2=0x7f8d700b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d74a00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700b0470 0x7f8d700b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d74a00640 1 -- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d7009f7c0 msgr2=0x7f8d701f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d74a00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d7009f7c0 0x7f8d701f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.828+0000 7f8d74a00640 1 -- 172.21.15.26:0/2933768650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7015da10 con 0x7f8d500047c0 2024-01-15T17:55:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.829+0000 7f8d6fe00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d7009f7c0 0x7f8d701f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:45.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.829+0000 7f8d74a00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d7009cce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8d60007f30 tx=0x7f8d60011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:45.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.829+0000 7f8d6cc00640 1 -- 172.21.15.26:0/2933768650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000e030 con 0x7f8d500047c0 2024-01-15T17:55:45.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.829+0000 7f8d6cc00640 1 -- 172.21.15.26:0/2933768650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d60016470 con 0x7f8d500047c0 2024-01-15T17:55:45.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.829+0000 7f8d6cc00640 1 -- 172.21.15.26:0/2933768650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60016820 con 0x7f8d500047c0 2024-01-15T17:55:45.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.829+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d700a05f0 con 0x7f8d500047c0 2024-01-15T17:55:45.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.829+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d700a0bb0 con 0x7f8d500047c0 2024-01-15T17:55:45.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.830+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d70067450 con 0x7f8d500047c0 2024-01-15T17:55:45.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.831+0000 7f8d6cc00640 1 -- 172.21.15.26:0/2933768650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8d60016980 con 0x7f8d500047c0 2024-01-15T17:55:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.832+0000 7f8d6cc00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d4806b2f0 0x7f8d4806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:45.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.832+0000 7f8d6cc00640 1 -- 172.21.15.26:0/2933768650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d6000f070 con 0x7f8d500047c0 2024-01-15T17:55:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.833+0000 7f8d6fe00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d4806b2f0 0x7f8d4806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:45.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.834+0000 7f8d6fe00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d4806b2f0 0x7f8d4806d7b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d58036000 tx=0x7f8d58030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:45.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:45.836+0000 7f8d6cc00640 1 -- 172.21.15.26:0/2933768650 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d60055a50 con 0x7f8d500047c0 2024-01-15T17:55:46.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.031+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d7006b610 con 0x7f8d500047c0 2024-01-15T17:55:46.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.032+0000 7f8d6cc00640 1 -- 172.21.15.26:0/2933768650 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d60059070 con 0x7f8d500047c0 2024-01-15T17:55:46.036 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d4806b2f0 msgr2=0x7f8d4806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d4806b2f0 0x7f8d4806d7b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d58036000 tx=0x7f8d58030000 comp rx=0 tx=0).stop 2024-01-15T17:55:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 msgr2=0x7f8d7009cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d7009cce0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8d60007f30 tx=0x7f8d60011660 comp rx=0 tx=0).stop 2024-01-15T17:55:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 shutdown_connections 2024-01-15T17:55:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d700b0470 0x7f8d700b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d4806b2f0 0x7f8d4806d7b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d7009f7c0 0x7f8d701f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 --2- 172.21.15.26:0/2933768650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d500047c0 0x7f8d7009cce0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 >> 172.21.15.26:0/2933768650 conn(0x7f8d700d3000 msgr2=0x7f8d700d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 shutdown_connections 2024-01-15T17:55:46.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.035+0000 7f8d75e00640 1 -- 172.21.15.26:0/2933768650 wait complete. 2024-01-15T17:55:46.122 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgress) Test that when one clone is in progress, one progress bar is printed 2024-01-15T17:55:46.122 INFO:tasks.cephfs_test_runner:test_progress_bar_for_1_clone (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T17:55:46.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:46.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.344+0000 7f7ae1200640 1 Processor -- start 2024-01-15T17:55:46.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.345+0000 7f7ae1200640 1 -- start start 2024-01-15T17:55:46.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.346+0000 7f7ae1200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7adc160920 0x7f7adc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.346+0000 7f7ae1200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7adc0cf9f0 0x7f7adc0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.346+0000 7f7ae1200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc161010 0x7f7adc1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.346+0000 7f7ae1200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7adc0d03e0 con 0x7f7adc160920 2024-01-15T17:55:46.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.346+0000 7f7ae1200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7adc0d0a10 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.346+0000 7f7ae1200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0cf080 con 0x7f7adc161010 2024-01-15T17:55:46.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.346+0000 7f7adb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7adc160920 0x7f7adc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.346+0000 7f7adb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7adc160920 0x7f7adc0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60984/0 (socket says 172.21.15.26:60984) 2024-01-15T17:55:46.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.346+0000 7f7adb400640 1 -- 172.21.15.26:0/2617221695 learned_addr learned my addr 172.21.15.26:0/2617221695 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:46.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.347+0000 7f7ad3e00640 1 --2- 172.21.15.26:0/2617221695 >> v2:172.21.15.124:3300/0 conn(0x7f7adc0cf9f0 0x7f7adc0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.347+0000 7f7adbe00640 1 --2- 172.21.15.26:0/2617221695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc161010 0x7f7adc1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.347+0000 7f7adb400640 1 -- 172.21.15.26:0/2617221695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc161010 msgr2=0x7f7adc1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:46.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.347+0000 7f7adb400640 1 --2- 172.21.15.26:0/2617221695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc161010 0x7f7adc1614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.347+0000 7f7adb400640 1 -- 172.21.15.26:0/2617221695 >> v2:172.21.15.124:3300/0 conn(0x7f7adc0cf9f0 msgr2=0x7f7adc0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:46.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.347+0000 7f7adb400640 1 --2- 172.21.15.26:0/2617221695 >> v2:172.21.15.124:3300/0 conn(0x7f7adc0cf9f0 0x7f7adc0cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.347+0000 7f7adb400640 1 -- 172.21.15.26:0/2617221695 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0c6be0 con 0x7f7adc160920 2024-01-15T17:55:46.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.347+0000 7f7ad3e00640 1 --2- 172.21.15.26:0/2617221695 >> v2:172.21.15.124:3300/0 conn(0x7f7adc0cf9f0 0x7f7adc0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:46.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.347+0000 7f7adbe00640 1 --2- 172.21.15.26:0/2617221695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc161010 0x7f7adc1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:46.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.348+0000 7f7adb400640 1 --2- 172.21.15.26:0/2617221695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc160920 0x7f7adc0b3f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7adc08c1f0 tx=0x7f7acc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f15abcaf32de166 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:46.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.348+0000 7f7adaa00640 1 -- 172.21.15.26:0/2617221695 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc031870 con 0x7f7adc160920 2024-01-15T17:55:46.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.348+0000 7f7adaa00640 1 -- 172.21.15.26:0/2617221695 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc0319d0 con 0x7f7adc160920 2024-01-15T17:55:46.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.348+0000 7f7adaa00640 1 -- 172.21.15.26:0/2617221695 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc031d60 con 0x7f7adc160920 2024-01-15T17:55:46.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.349+0000 7f7ae1200640 1 -- 172.21.15.26:0/2617221695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc160920 msgr2=0x7f7adc0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.349+0000 7f7ae1200640 1 --2- 172.21.15.26:0/2617221695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc160920 0x7f7adc0b3f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7adc08c1f0 tx=0x7f7acc02b910 comp rx=0 tx=0).stop 2024-01-15T17:55:46.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.349+0000 7f7ae1200640 1 -- 172.21.15.26:0/2617221695 shutdown_connections 2024-01-15T17:55:46.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.349+0000 7f7ae1200640 1 --2- 172.21.15.26:0/2617221695 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc161010 0x7f7adc1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.349+0000 7f7ae1200640 1 --2- 172.21.15.26:0/2617221695 >> v2:172.21.15.124:3300/0 conn(0x7f7adc0cf9f0 0x7f7adc0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.349+0000 7f7ae1200640 1 --2- 172.21.15.26:0/2617221695 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc160920 0x7f7adc0b3f30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.349+0000 7f7ae1200640 1 -- 172.21.15.26:0/2617221695 >> 172.21.15.26:0/2617221695 conn(0x7f7adc0d3090 msgr2=0x7f7adc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:46.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.349+0000 7f7ae1200640 1 -- 172.21.15.26:0/2617221695 shutdown_connections 2024-01-15T17:55:46.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.349+0000 7f7ae1200640 1 -- 172.21.15.26:0/2617221695 wait complete. 2024-01-15T17:55:46.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.351+0000 7f7ae1200640 1 Processor -- start 2024-01-15T17:55:46.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.351+0000 7f7ae1200640 1 -- start start 2024-01-15T17:55:46.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.352+0000 7f7ae1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc0cf9f0 0x7f7adc0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.352+0000 7f7ae1200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc160920 0x7f7adc15a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.352+0000 7f7ae1200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7adc161010 0x7f7adc15d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.352+0000 7f7ae1200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0cbae0 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.352+0000 7f7ae1200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0cbdc0 con 0x7f7adc161010 2024-01-15T17:55:46.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.352+0000 7f7ae1200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc0cc0a0 con 0x7f7adc160920 2024-01-15T17:55:46.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.352+0000 7f7adb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc0cf9f0 0x7f7adc0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.352+0000 7f7adb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc0cf9f0 0x7f7adc0cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60988/0 (socket says 172.21.15.26:60988) 2024-01-15T17:55:46.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.352+0000 7f7adb400640 1 -- 172.21.15.26:0/1754641738 learned_addr learned my addr 172.21.15.26:0/1754641738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:46.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7adb400640 1 -- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc160920 msgr2=0x7f7adc15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7adb400640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc160920 0x7f7adc15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7adb400640 1 -- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7adc161010 msgr2=0x7f7adc15d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7adbe00640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7adc161010 0x7f7adc15d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7adb400640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7adc161010 0x7f7adc15d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7adb400640 1 -- 172.21.15.26:0/1754641738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc0094d0 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7adb400640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc0cf9f0 0x7f7adc0cb450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7acc02b950 tx=0x7f7acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:46.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7ad3400640 1 -- 172.21.15.26:0/1754641738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc031860 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7ad3400640 1 -- 172.21.15.26:0/1754641738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc0443f0 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7ad3400640 1 -- 172.21.15.26:0/1754641738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0447a0 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.353+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc0cc380 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.354+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc0ccb80 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.355+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc067450 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.355+0000 7f7ad3400640 1 -- 172.21.15.26:0/1754641738 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7acc04d450 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.357+0000 7f7ad3400640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ab406b2f0 0x7f7ab406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.357+0000 7f7ad3400640 1 -- 172.21.15.26:0/1754641738 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7acc02d040 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.357+0000 7f7ad3e00640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ab406b2f0 0x7f7ab406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.358+0000 7f7ad3e00640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ab406b2f0 0x7f7ab406d7b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4006b10 tx=0x7f7ac4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:46.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.361+0000 7f7ad3400640 1 -- 172.21.15.26:0/1754641738 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7acc077150 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.558+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7adc08c7e0 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.562+0000 7f7ad3400640 1 -- 172.21.15.26:0/1754641738 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f7acc07a770 con 0x7f7adc0cf9f0 2024-01-15T17:55:46.566 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:46.566 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:46.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ab406b2f0 msgr2=0x7f7ab406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ab406b2f0 0x7f7ab406d7b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4006b10 tx=0x7f7ac4008040 comp rx=0 tx=0).stop 2024-01-15T17:55:46.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc0cf9f0 msgr2=0x7f7adc0cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc0cf9f0 0x7f7adc0cb450 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7acc02b950 tx=0x7f7acc0046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 shutdown_connections 2024-01-15T17:55:46.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7adc160920 0x7f7adc15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ab406b2f0 0x7f7ab406d7b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7adc161010 0x7f7adc15d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 --2- 172.21.15.26:0/1754641738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7adc0cf9f0 0x7f7adc0cb450 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 >> 172.21.15.26:0/1754641738 conn(0x7f7adc0d3090 msgr2=0x7f7adc06c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:46.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 shutdown_connections 2024-01-15T17:55:46.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.565+0000 7f7ae1200640 1 -- 172.21.15.26:0/1754641738 wait complete. 2024-01-15T17:55:46.588 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:46.588 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:46.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.806+0000 7f7703e00640 1 Processor -- start 2024-01-15T17:55:46.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.806+0000 7f7703e00640 1 -- start start 2024-01-15T17:55:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.808+0000 7f7703e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7704070a30 0x7f770415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.808+0000 7f7703e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770415f4e0 con 0x7f77040adf20 2024-01-15T17:55:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.808+0000 7f7703e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7704070ee0 con 0x7f77040ccb50 2024-01-15T17:55:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.808+0000 7f7703e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f770415c450 con 0x7f7704070a30 2024-01-15T17:55:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.808+0000 7f7702a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f77040adf20 0x7f77040cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57396/0 (socket says 172.21.15.26:57396) 2024-01-15T17:55:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.808+0000 7f7702a00640 1 -- 172.21.15.26:0/3454525623 learned_addr learned my addr 172.21.15.26:0/3454525623 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.808+0000 7f7703400640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7704070a30 0x7f770415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367203210 0 0) 0x7f770415f4e0 con 0x7f77040adf20 2024-01-15T17:55:46.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76e4003520 con 0x7f77040adf20 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 75372885 0 0) 0x7f76e4003520 con 0x7f77040adf20 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7704070a30 msgr2=0x7f770415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7704070a30 0x7f770415bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 >> v1:172.21.15.124:6789/0 conn(0x7f77040ccb50 legacy=0x7f7704143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7704071130 con 0x7f77040adf20 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f76f4001670 con 0x7f77040adf20 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 >> v1:172.21.15.26:6789/0 conn(0x7f77040adf20 legacy=0x7f77040cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76e40039a0 0x7f76e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7704070a30 0x7f76e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e4004d00 0x7f76e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770415c450 con 0x7f76e4004d00 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e400bb10 con 0x7f76e40039a0 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.809+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e400bcd0 con 0x7f7704070a30 2024-01-15T17:55:46.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.810+0000 7f7703400640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e4004d00 0x7f76e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.810+0000 7f7702a00640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76e40039a0 0x7f76e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.810+0000 7f76fbe00640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7704070a30 0x7f76e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.810+0000 7f7703400640 1 -- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7704070a30 msgr2=0x7f76e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.810+0000 7f7703400640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7704070a30 0x7f76e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.810+0000 7f7703400640 1 -- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76e40039a0 msgr2=0x7f76e4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.810+0000 7f7703400640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76e40039a0 0x7f76e4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.810+0000 7f7703400640 1 -- 172.21.15.26:0/3454525623 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e4003520 con 0x7f76e4004d00 2024-01-15T17:55:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.811+0000 7f7703400640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e4004d00 0x7f76e4005180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009fc0 tx=0x7f76e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6184cf5b2753b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.811+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc02cc20 con 0x7f76e4004d00 2024-01-15T17:55:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.811+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76fc00f040 con 0x7f76e4004d00 2024-01-15T17:55:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.811+0000 7f7702000640 1 -- 172.21.15.26:0/3454525623 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc034440 con 0x7f76e4004d00 2024-01-15T17:55:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.812+0000 7f7703e00640 1 -- 172.21.15.26:0/3454525623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e4004d00 msgr2=0x7f76e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.812+0000 7f7703e00640 1 --2- 172.21.15.26:0/3454525623 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76e4004d00 0x7f76e4005180 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009fc0 tx=0x7f76e40057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.812+0000 7f7703400640 1 -- 172.21.15.26:0/3454525623 reap_dead start 2024-01-15T17:55:46.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.812+0000 7f7703e00640 1 -- 172.21.15.26:0/3454525623 shutdown_connections 2024-01-15T17:55:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.812+0000 7f7703e00640 1 -- 172.21.15.26:0/3454525623 >> 172.21.15.26:0/3454525623 conn(0x7f770406a7d0 msgr2=0x7f770406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.812+0000 7f7703e00640 1 -- 172.21.15.26:0/3454525623 shutdown_connections 2024-01-15T17:55:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.812+0000 7f7703e00640 1 -- 172.21.15.26:0/3454525623 wait complete. 2024-01-15T17:55:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.814+0000 7f7703e00640 1 Processor -- start 2024-01-15T17:55:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.814+0000 7f7703e00640 1 -- start start 2024-01-15T17:55:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.815+0000 7f7703e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76e4004d00 0x7f770415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.815+0000 7f7703e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7704140530 0x7f7704140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.815+0000 7f7703e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77041f2cc0 0x7f77041f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.815+0000 7f7703e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77040b0ad0 con 0x7f77041f2cc0 2024-01-15T17:55:46.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.815+0000 7f7703e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770415a870 con 0x7f7704140530 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.815+0000 7f7703e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77040afe50 con 0x7f76e4004d00 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.815+0000 7f7703400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77041f2cc0 0x7f77041f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f7703400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77041f2cc0 0x7f77041f3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32780/0 (socket says 172.21.15.26:32780) 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f7703400640 1 -- 172.21.15.26:0/758212725 learned_addr learned my addr 172.21.15.26:0/758212725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f76fbe00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7704140530 0x7f7704140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f7702a00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76e4004d00 0x7f770415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f7703400640 1 -- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76e4004d00 msgr2=0x7f770415a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f7703400640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76e4004d00 0x7f770415a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f7703400640 1 -- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7704140530 msgr2=0x7f7704140990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f7703400640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7704140530 0x7f7704140990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f7703400640 1 -- 172.21.15.26:0/758212725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040b0130 con 0x7f77041f2cc0 2024-01-15T17:55:46.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.816+0000 7f7703400640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77041f2cc0 0x7f77041f3140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009f60 tx=0x7f76fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.817+0000 7f76fb400640 1 -- 172.21.15.26:0/758212725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc046070 con 0x7f77041f2cc0 2024-01-15T17:55:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.817+0000 7f76fb400640 1 -- 172.21.15.26:0/758212725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76fc035ae0 con 0x7f77041f2cc0 2024-01-15T17:55:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.817+0000 7f76fb400640 1 -- 172.21.15.26:0/758212725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76fc035e70 con 0x7f77041f2cc0 2024-01-15T17:55:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.817+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77040b0410 con 0x7f77041f2cc0 2024-01-15T17:55:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.817+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77040afa70 con 0x7f77041f2cc0 2024-01-15T17:55:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.818+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77040b0c10 con 0x7f77041f2cc0 2024-01-15T17:55:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.819+0000 7f76fb400640 1 -- 172.21.15.26:0/758212725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f76fc03b050 con 0x7f77041f2cc0 2024-01-15T17:55:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.820+0000 7f76fb400640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76dc06b2c0 0x7f76dc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.820+0000 7f76fb400640 1 -- 172.21.15.26:0/758212725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76fc0ab0c0 con 0x7f77041f2cc0 2024-01-15T17:55:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.821+0000 7f7702a00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76dc06b2c0 0x7f76dc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.822+0000 7f7702a00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76dc06b2c0 0x7f76dc06d780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f76f4001050 tx=0x7f76f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:46.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:46.824+0000 7f76fb400640 1 -- 172.21.15.26:0/758212725 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77040b0c10 con 0x7f77041f2cc0 2024-01-15T17:55:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.021+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7704067450 con 0x7f77041f2cc0 2024-01-15T17:55:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.022+0000 7f76fb400640 1 -- 172.21.15.26:0/758212725 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76fc075fc0 con 0x7f77041f2cc0 2024-01-15T17:55:47.026 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76dc06b2c0 msgr2=0x7f76dc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76dc06b2c0 0x7f76dc06d780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f76f4001050 tx=0x7f76f4009040 comp rx=0 tx=0).stop 2024-01-15T17:55:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77041f2cc0 msgr2=0x7f77041f3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77041f2cc0 0x7f77041f3140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f76fc009f60 tx=0x7f76fc009750 comp rx=0 tx=0).stop 2024-01-15T17:55:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 shutdown_connections 2024-01-15T17:55:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76dc06b2c0 0x7f76dc06d780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77041f2cc0 0x7f77041f3140 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7704140530 0x7f7704140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 --2- 172.21.15.26:0/758212725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76e4004d00 0x7f770415a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 >> 172.21.15.26:0/758212725 conn(0x7f770406a7d0 msgr2=0x7f770415dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:47.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 shutdown_connections 2024-01-15T17:55:47.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.025+0000 7f7703e00640 1 -- 172.21.15.26:0/758212725 wait complete. 2024-01-15T17:55:47.049 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:47.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.266+0000 7fd83f000640 1 Processor -- start 2024-01-15T17:55:47.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.266+0000 7fd83f000640 1 -- start start 2024-01-15T17:55:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.268+0000 7fd83f000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8380cf9a0 0x7fd8380cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.268+0000 7fd83f000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd8380b31d0 0x7fd8380b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.268+0000 7fd83f000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8380d0320 con 0x7fd83815d4b0 2024-01-15T17:55:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.268+0000 7fd83f000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8380d1c60 con 0x7fd8380b31d0 2024-01-15T17:55:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.268+0000 7fd83f000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380b2730 con 0x7fd8380cf9a0 2024-01-15T17:55:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.269+0000 7fd83e600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd8380b31d0 0x7fd8380b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.269+0000 7fd83e600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd8380b31d0 0x7fd8380b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32790/0 (socket says 172.21.15.26:32790) 2024-01-15T17:55:47.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.269+0000 7fd83e600640 1 -- 172.21.15.26:0/3975861799 learned_addr learned my addr 172.21.15.26:0/3975861799 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.269+0000 7fd83d200640 1 --2- 172.21.15.26:0/3975861799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8380cf9a0 0x7fd8380cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.269+0000 7fd83e600640 1 -- 172.21.15.26:0/3975861799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8380cf9a0 msgr2=0x7fd8380cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.269+0000 7fd83e600640 1 --2- 172.21.15.26:0/3975861799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8380cf9a0 0x7fd8380cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.269+0000 7fd83e600640 1 -- 172.21.15.26:0/3975861799 >> v1:172.21.15.124:6789/0 conn(0x7fd83815d4b0 legacy=0x7fd8380d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.269+0000 7fd83e600640 1 -- 172.21.15.26:0/3975861799 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380d1ec0 con 0x7fd8380b31d0 2024-01-15T17:55:47.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.270+0000 7fd83e600640 1 --2- 172.21.15.26:0/3975861799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 0x7fd8380b3660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd830009e80 tx=0x7fd83002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4ca2044f12e859 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:47.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.271+0000 7fd837e00640 1 -- 172.21.15.26:0/3975861799 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83002cae0 con 0x7fd8380b31d0 2024-01-15T17:55:47.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.271+0000 7fd837e00640 1 -- 172.21.15.26:0/3975861799 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83002cc40 con 0x7fd8380b31d0 2024-01-15T17:55:47.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.271+0000 7fd837e00640 1 -- 172.21.15.26:0/3975861799 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8300343f0 con 0x7fd8380b31d0 2024-01-15T17:55:47.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.271+0000 7fd83f000640 1 -- 172.21.15.26:0/3975861799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 msgr2=0x7fd8380b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.271+0000 7fd83f000640 1 --2- 172.21.15.26:0/3975861799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 0x7fd8380b3660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd830009e80 tx=0x7fd83002bf40 comp rx=0 tx=0).stop 2024-01-15T17:55:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.272+0000 7fd83f000640 1 -- 172.21.15.26:0/3975861799 shutdown_connections 2024-01-15T17:55:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.272+0000 7fd83f000640 1 --2- 172.21.15.26:0/3975861799 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 0x7fd8380b3660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.272+0000 7fd83f000640 1 --2- 172.21.15.26:0/3975861799 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd8380cf9a0 0x7fd8380cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.272+0000 7fd83f000640 1 -- 172.21.15.26:0/3975861799 >> 172.21.15.26:0/3975861799 conn(0x7fd8380d3000 msgr2=0x7fd8380d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.272+0000 7fd83f000640 1 -- 172.21.15.26:0/3975861799 shutdown_connections 2024-01-15T17:55:47.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.272+0000 7fd83f000640 1 -- 172.21.15.26:0/3975861799 wait complete. 2024-01-15T17:55:47.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.273+0000 7fd83f000640 1 Processor -- start 2024-01-15T17:55:47.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.274+0000 7fd83f000640 1 -- start start 2024-01-15T17:55:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83f000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 0x7fd838159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83f000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8380cf9a0 0x7fd8380cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83f000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd83815d4b0 0x7fd8381f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83f000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83815ad00 con 0x7fd8380b31d0 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83f000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380cc820 con 0x7fd8380cf9a0 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83f000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8380cd030 con 0x7fd83815d4b0 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 0x7fd838159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 0x7fd838159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32796/0 (socket says 172.21.15.26:32796) 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83dc00640 1 -- 172.21.15.26:0/697203965 learned_addr learned my addr 172.21.15.26:0/697203965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83d200640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8380cf9a0 0x7fd8380cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83e600640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd83815d4b0 0x7fd8381f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83dc00640 1 -- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd83815d4b0 msgr2=0x7fd8381f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83dc00640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd83815d4b0 0x7fd8381f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83dc00640 1 -- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8380cf9a0 msgr2=0x7fd8380cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83dc00640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8380cf9a0 0x7fd8380cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83dc00640 1 -- 172.21.15.26:0/697203965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd830009b30 con 0x7fd8380b31d0 2024-01-15T17:55:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.275+0000 7fd83e600640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd83815d4b0 0x7fd8381f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.276+0000 7fd83dc00640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 0x7fd838159c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c430 tx=0x7fd82800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.276+0000 7fd835600640 1 -- 172.21.15.26:0/697203965 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8280052d0 con 0x7fd8380b31d0 2024-01-15T17:55:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.276+0000 7fd835600640 1 -- 172.21.15.26:0/697203965 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828005430 con 0x7fd8380b31d0 2024-01-15T17:55:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.276+0000 7fd835600640 1 -- 172.21.15.26:0/697203965 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828003830 con 0x7fd8380b31d0 2024-01-15T17:55:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.276+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8380cca70 con 0x7fd8380b31d0 2024-01-15T17:55:47.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.276+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83815e6c0 con 0x7fd8380b31d0 2024-01-15T17:55:47.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.277+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd838067450 con 0x7fd8380b31d0 2024-01-15T17:55:47.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.278+0000 7fd835600640 1 -- 172.21.15.26:0/697203965 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd828003990 con 0x7fd8380b31d0 2024-01-15T17:55:47.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.279+0000 7fd835600640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd81006b2f0 0x7fd81006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.279+0000 7fd835600640 1 -- 172.21.15.26:0/697203965 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8280891d0 con 0x7fd8380b31d0 2024-01-15T17:55:47.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.280+0000 7fd83d200640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd81006b2f0 0x7fd81006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.281+0000 7fd83d200640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd81006b2f0 0x7fd81006d7b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd820006fd0 tx=0x7fd820008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:47.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.283+0000 7fd835600640 1 -- 172.21.15.26:0/697203965 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd828053fa0 con 0x7fd8380b31d0 2024-01-15T17:55:47.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.482+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd83815ca30 con 0x7fd8380b31d0 2024-01-15T17:55:47.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.484+0000 7fd835600640 1 -- 172.21.15.26:0/697203965 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd8280575c0 con 0x7fd8380b31d0 2024-01-15T17:55:47.488 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd81006b2f0 msgr2=0x7fd81006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd81006b2f0 0x7fd81006d7b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd820006fd0 tx=0x7fd820008040 comp rx=0 tx=0).stop 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 msgr2=0x7fd838159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 0x7fd838159c20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c430 tx=0x7fd82800c8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 shutdown_connections 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd83815d4b0 0x7fd8381f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd81006b2f0 0x7fd81006d7b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd8380cf9a0 0x7fd8380cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 --2- 172.21.15.26:0/697203965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd8380b31d0 0x7fd838159c20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.486+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 >> 172.21.15.26:0/697203965 conn(0x7fd8380d3000 msgr2=0x7fd83806bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.487+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 shutdown_connections 2024-01-15T17:55:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.487+0000 7fd83f000640 1 -- 172.21.15.26:0/697203965 wait complete. 2024-01-15T17:55:47.510 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.730+0000 7fad9c400640 1 Processor -- start 2024-01-15T17:55:47.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.730+0000 7fad9c400640 1 -- start start 2024-01-15T17:55:47.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad9c400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fad9413e360 0x7fad940cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad9c400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 0x7fad940cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad9c400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9415c5f0 con 0x7fad9413df80 2024-01-15T17:55:47.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad9c400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9406c470 con 0x7fad9413e360 2024-01-15T17:55:47.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad9c400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9406c750 con 0x7fad9415f9c0 2024-01-15T17:55:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad9a400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 0x7fad940cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad99000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fad9413e360 0x7fad940cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad99000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fad9413e360 0x7fad940cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:48456/0 (socket says 172.21.15.26:48456) 2024-01-15T17:55:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad99a00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fad9413df80 0x7fad9415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:51198/0 (socket says 172.21.15.26:51198) 2024-01-15T17:55:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.732+0000 7fad9a400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 0x7fad940cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:42964/0 (socket says 172.21.15.26:42964) 2024-01-15T17:55:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.733+0000 7fad99000640 1 -- 172.21.15.26:0/1260456863 learned_addr learned my addr 172.21.15.26:0/1260456863 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.733+0000 7fad99000640 1 -- 172.21.15.26:0/1260456863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 msgr2=0x7fad940cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:47.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.733+0000 7fad99000640 1 --2- 172.21.15.26:0/1260456863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 0x7fad940cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.733+0000 7fad99000640 1 -- 172.21.15.26:0/1260456863 >> v1:172.21.15.124:6789/0 conn(0x7fad9413df80 legacy=0x7fad9415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.733+0000 7fad99000640 1 -- 172.21.15.26:0/1260456863 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9406ca30 con 0x7fad9413e360 2024-01-15T17:55:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.735+0000 7fad99000640 1 --2- 172.21.15.26:0/1260456863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413e360 0x7fad940cf290 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e86b461013e0312a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.735+0000 7fad93e00640 1 -- 172.21.15.26:0/1260456863 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c030840 con 0x7fad9413e360 2024-01-15T17:55:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.736+0000 7fad93e00640 1 -- 172.21.15.26:0/1260456863 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c0309a0 con 0x7fad9413e360 2024-01-15T17:55:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.736+0000 7fad93e00640 1 -- 172.21.15.26:0/1260456863 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c030d30 con 0x7fad9413e360 2024-01-15T17:55:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.736+0000 7fad9c400640 1 -- 172.21.15.26:0/1260456863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413e360 msgr2=0x7fad940cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.736+0000 7fad9c400640 1 --2- 172.21.15.26:0/1260456863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413e360 0x7fad940cf290 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fad7c009870 tx=0x7fad7c02b980 comp rx=0 tx=0).stop 2024-01-15T17:55:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.736+0000 7fad9c400640 1 -- 172.21.15.26:0/1260456863 shutdown_connections 2024-01-15T17:55:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.736+0000 7fad9c400640 1 --2- 172.21.15.26:0/1260456863 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 0x7fad940cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.736+0000 7fad9c400640 1 --2- 172.21.15.26:0/1260456863 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413e360 0x7fad940cf290 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.736+0000 7fad9c400640 1 -- 172.21.15.26:0/1260456863 >> 172.21.15.26:0/1260456863 conn(0x7fad9406a800 msgr2=0x7fad9406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.736+0000 7fad9c400640 1 -- 172.21.15.26:0/1260456863 shutdown_connections 2024-01-15T17:55:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.737+0000 7fad9c400640 1 -- 172.21.15.26:0/1260456863 wait complete. 2024-01-15T17:55:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.739+0000 7fad9c400640 1 Processor -- start 2024-01-15T17:55:47.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.739+0000 7fad9c400640 1 -- start start 2024-01-15T17:55:47.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.740+0000 7fad9c400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413df80 0x7fad94153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.740+0000 7fad9c400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9413e360 0x7fad940a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.740+0000 7fad9c400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 0x7fad9415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.740+0000 7fad9c400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad94151f60 con 0x7fad9413e360 2024-01-15T17:55:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.740+0000 7fad9c400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad94151630 con 0x7fad9413df80 2024-01-15T17:55:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.740+0000 7fad9c400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad94151910 con 0x7fad9415f9c0 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.741+0000 7fad99000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9413e360 0x7fad940a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.741+0000 7fad99000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9413e360 0x7fad940a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32812/0 (socket says 172.21.15.26:32812) 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.741+0000 7fad99000640 1 -- 172.21.15.26:0/3272770740 learned_addr learned my addr 172.21.15.26:0/3272770740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.741+0000 7fad99a00640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413df80 0x7fad94153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.741+0000 7fad9a400640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 0x7fad9415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.741+0000 7fad99000640 1 -- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 msgr2=0x7fad9415db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.741+0000 7fad99000640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 0x7fad9415db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.741+0000 7fad99000640 1 -- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413df80 msgr2=0x7fad94153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.741+0000 7fad99000640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413df80 0x7fad94153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.742+0000 7fad99000640 1 -- 172.21.15.26:0/3272770740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c0094d0 con 0x7fad9413e360 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.742+0000 7fad99a00640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413df80 0x7fad94153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.742+0000 7fad99000640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9413e360 0x7fad940a8c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000c00 tx=0x7fad7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.742+0000 7fad91600640 1 -- 172.21.15.26:0/3272770740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c039c90 con 0x7fad9413e360 2024-01-15T17:55:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.742+0000 7fad91600640 1 -- 172.21.15.26:0/3272770740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c039e10 con 0x7fad9413e360 2024-01-15T17:55:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.742+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad94151bf0 con 0x7fad9413e360 2024-01-15T17:55:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.742+0000 7fad91600640 1 -- 172.21.15.26:0/3272770740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c043640 con 0x7fad9413e360 2024-01-15T17:55:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.743+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9415e050 con 0x7fad9413e360 2024-01-15T17:55:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.744+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad94152110 con 0x7fad9413e360 2024-01-15T17:55:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.745+0000 7fad91600640 1 -- 172.21.15.26:0/3272770740 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fad7c0437a0 con 0x7fad9413e360 2024-01-15T17:55:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.746+0000 7fad91600640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad6c06b310 0x7fad6c06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:47.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.746+0000 7fad91600640 1 -- 172.21.15.26:0/3272770740 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad7c03e070 con 0x7fad9413e360 2024-01-15T17:55:47.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.746+0000 7fad99a00640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad6c06b310 0x7fad6c06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.747+0000 7fad99a00640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad6c06b310 0x7fad6c06d7d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad84000f60 tx=0x7fad84007ce0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.750+0000 7fad91600640 1 -- 172.21.15.26:0/3272770740 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad94152110 con 0x7fad9413e360 2024-01-15T17:55:47.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.950+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad940675d0 con 0x7fad9413e360 2024-01-15T17:55:47.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.950+0000 7fad91600640 1 -- 172.21.15.26:0/3272770740 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fad7c0340a0 con 0x7fad9413e360 2024-01-15T17:55:47.955 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad6c06b310 msgr2=0x7fad6c06d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad6c06b310 0x7fad6c06d7d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad84000f60 tx=0x7fad84007ce0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9413e360 msgr2=0x7fad940a8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9413e360 0x7fad940a8c40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000c00 tx=0x7fad7c004730 comp rx=0 tx=0).stop 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 shutdown_connections 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad9415f9c0 0x7fad9415db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad6c06b310 0x7fad6c06d7d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad9413e360 0x7fad940a8c40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 --2- 172.21.15.26:0/3272770740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad9413df80 0x7fad94153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 >> 172.21.15.26:0/3272770740 conn(0x7fad9406a800 msgr2=0x7fad9415eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:47.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.953+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 shutdown_connections 2024-01-15T17:55:47.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:47.954+0000 7fad9c400640 1 -- 172.21.15.26:0/3272770740 wait complete. 2024-01-15T17:55:47.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone' 2024-01-15T17:55:48.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.195+0000 7fe123e00640 1 Processor -- start 2024-01-15T17:55:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.196+0000 7fe123e00640 1 -- start start 2024-01-15T17:55:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.197+0000 7fe123e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe124160920 0x7fe1240b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.197+0000 7fe123e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 0x7fe1240cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.197+0000 7fe123e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1240d0a10 con 0x7fe124161010 2024-01-15T17:55:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.197+0000 7fe123e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1240d03b0 con 0x7fe124160920 2024-01-15T17:55:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.197+0000 7fe123e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1240d1f50 con 0x7fe1240cf9f0 2024-01-15T17:55:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.198+0000 7fe123400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe124161010 0x7fe1241614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57398/0 (socket says 172.21.15.26:57398) 2024-01-15T17:55:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.198+0000 7fe123400640 1 -- 172.21.15.26:0/5778347 learned_addr learned my addr 172.21.15.26:0/5778347 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.198+0000 7fe122a00640 1 --2- 172.21.15.26:0/5778347 >> v2:172.21.15.124:3300/0 conn(0x7fe124160920 0x7fe1240b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.198+0000 7fe11be00640 1 --2- 172.21.15.26:0/5778347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 0x7fe1240cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.198+0000 7fe122000640 1 -- 172.21.15.26:0/5778347 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244229506 0 0) 0x7fe1240d0a10 con 0x7fe124161010 2024-01-15T17:55:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.198+0000 7fe122000640 1 -- 172.21.15.26:0/5778347 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe104003520 con 0x7fe124161010 2024-01-15T17:55:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.199+0000 7fe122000640 1 -- 172.21.15.26:0/5778347 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4052594410 0 0) 0x7fe104003520 con 0x7fe124161010 2024-01-15T17:55:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.199+0000 7fe122a00640 1 -- 172.21.15.26:0/5778347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 msgr2=0x7fe1240cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.199+0000 7fe122a00640 1 --2- 172.21.15.26:0/5778347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 0x7fe1240cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.199+0000 7fe122a00640 1 -- 172.21.15.26:0/5778347 >> v1:172.21.15.26:6789/0 conn(0x7fe124161010 legacy=0x7fe1241614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.199+0000 7fe122a00640 1 -- 172.21.15.26:0/5778347 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1240cf080 con 0x7fe124160920 2024-01-15T17:55:48.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.200+0000 7fe122a00640 1 --2- 172.21.15.26:0/5778347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 0x7fe1240b3f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe1140097a0 tx=0x7fe11402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e419b25915c0e25 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.201+0000 7fe122000640 1 -- 172.21.15.26:0/5778347 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0019f0 con 0x7fe124160920 2024-01-15T17:55:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.201+0000 7fe122000640 1 -- 172.21.15.26:0/5778347 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11402e050 con 0x7fe124160920 2024-01-15T17:55:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.201+0000 7fe122000640 1 -- 172.21.15.26:0/5778347 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114030730 con 0x7fe124160920 2024-01-15T17:55:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.202+0000 7fe123e00640 1 -- 172.21.15.26:0/5778347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 msgr2=0x7fe1240b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.202+0000 7fe123e00640 1 --2- 172.21.15.26:0/5778347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 0x7fe1240b3f30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe1140097a0 tx=0x7fe11402b790 comp rx=0 tx=0).stop 2024-01-15T17:55:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.202+0000 7fe123e00640 1 -- 172.21.15.26:0/5778347 shutdown_connections 2024-01-15T17:55:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.202+0000 7fe123e00640 1 --2- 172.21.15.26:0/5778347 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 0x7fe1240cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.202+0000 7fe123e00640 1 --2- 172.21.15.26:0/5778347 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 0x7fe1240b3f30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.202+0000 7fe123e00640 1 -- 172.21.15.26:0/5778347 >> 172.21.15.26:0/5778347 conn(0x7fe1240d3090 msgr2=0x7fe1240d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.202+0000 7fe123e00640 1 -- 172.21.15.26:0/5778347 shutdown_connections 2024-01-15T17:55:48.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.202+0000 7fe123e00640 1 -- 172.21.15.26:0/5778347 wait complete. 2024-01-15T17:55:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.204+0000 7fe123e00640 1 Processor -- start 2024-01-15T17:55:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.204+0000 7fe123e00640 1 -- start start 2024-01-15T17:55:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe123e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 0x7fe1240b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe123e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 0x7fe124157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe123e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe124161010 0x7fe124159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:48.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe123e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12415d010 con 0x7fe124161010 2024-01-15T17:55:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe123e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12415d2f0 con 0x7fe124160920 2024-01-15T17:55:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe123e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12415d5d0 con 0x7fe1240cf9f0 2024-01-15T17:55:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe123400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe124161010 0x7fe124159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe123400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe124161010 0x7fe124159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32828/0 (socket says 172.21.15.26:32828) 2024-01-15T17:55:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe123400640 1 -- 172.21.15.26:0/1983889463 learned_addr learned my addr 172.21.15.26:0/1983889463 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe11be00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 0x7fe124157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:48.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.206+0000 7fe122a00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 0x7fe1240b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe123400640 1 -- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 msgr2=0x7fe1240b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe123400640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 0x7fe1240b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe123400640 1 -- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 msgr2=0x7fe124157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe123400640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 0x7fe124157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe123400640 1 -- 172.21.15.26:0/1983889463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe114009400 con 0x7fe124161010 2024-01-15T17:55:48.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe11be00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 0x7fe124157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe123400640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe124161010 0x7fe124159000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00cc70 tx=0x7fe11c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe11aa00640 1 -- 172.21.15.26:0/1983889463 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c00ce80 con 0x7fe124161010 2024-01-15T17:55:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe11aa00640 1 -- 172.21.15.26:0/1983889463 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c012400 con 0x7fe124161010 2024-01-15T17:55:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.207+0000 7fe11aa00640 1 -- 172.21.15.26:0/1983889463 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0127b0 con 0x7fe124161010 2024-01-15T17:55:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.208+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12415d8b0 con 0x7fe124161010 2024-01-15T17:55:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.208+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1240b6190 con 0x7fe124161010 2024-01-15T17:55:48.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.209+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12415ddd0 con 0x7fe124161010 2024-01-15T17:55:48.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.210+0000 7fe11aa00640 1 -- 172.21.15.26:0/1983889463 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe11c012910 con 0x7fe124161010 2024-01-15T17:55:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.211+0000 7fe11aa00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:48.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.211+0000 7fe11aa00640 1 -- 172.21.15.26:0/1983889463 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe11c08a310 con 0x7fe124161010 2024-01-15T17:55:48.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.212+0000 7fe122a00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:48.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.213+0000 7fe122a00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe114034000 tx=0x7fe11402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:48.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.216+0000 7fe11aa00640 1 -- 172.21.15.26:0/1983889463 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe12415ddd0 con 0x7fe124161010 2024-01-15T17:55:48.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.485+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]} v 0) v1 -- 0x7fe12415ddd0 con 0x7fe124161010 2024-01-15T17:55:48.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.916+0000 7fe11aa00640 1 -- 172.21.15.26:0/1983889463 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]}]=0 v58) v1 ==== 155+0+0 (secure 0 0 0) 0x7fe12415ddd0 con 0x7fe124161010 2024-01-15T17:55:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe0fc06b2c0 msgr2=0x7fe0fc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe114034000 tx=0x7fe11402f000 comp rx=0 tx=0).stop 2024-01-15T17:55:48.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe124161010 msgr2=0x7fe124159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe124161010 0x7fe124159000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe11c00cc70 tx=0x7fe11c00a430 comp rx=0 tx=0).stop 2024-01-15T17:55:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 shutdown_connections 2024-01-15T17:55:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe0fc06b2c0 0x7fe0fc06d780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe124161010 0x7fe124159000 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe124160920 0x7fe124157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 --2- 172.21.15.26:0/1983889463 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1240cf9f0 0x7fe1240b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:48.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 >> 172.21.15.26:0/1983889463 conn(0x7fe1240d3090 msgr2=0x7fe12406bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:48.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.919+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 shutdown_connections 2024-01-15T17:55:48.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:48.920+0000 7fe123e00640 1 -- 172.21.15.26:0/1983889463 wait complete. 2024-01-15T17:55:48.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T17:55:49.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.166+0000 7fd1ebe00640 1 Processor -- start 2024-01-15T17:55:49.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.166+0000 7fd1ebe00640 1 -- start start 2024-01-15T17:55:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1ebe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec1447f0 0x7fd1ec15c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1ebe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1ec06c470 0x7fd1ec0cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1ebe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec15c5c0 con 0x7fd1ec06dab0 2024-01-15T17:55:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1ebe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec06d690 con 0x7fd1ec06c470 2024-01-15T17:55:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1ebe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0cebb0 con 0x7fd1ec1447f0 2024-01-15T17:55:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1e3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1ec06c470 0x7fd1ec0cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1e3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1ec06c470 0x7fd1ec0cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32844/0 (socket says 172.21.15.26:32844) 2024-01-15T17:55:49.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1e3e00640 1 -- 172.21.15.26:0/2992083006 learned_addr learned my addr 172.21.15.26:0/2992083006 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1eaa00640 1 --2- 172.21.15.26:0/2992083006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec1447f0 0x7fd1ec15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1e3e00640 1 -- 172.21.15.26:0/2992083006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec1447f0 msgr2=0x7fd1ec15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1e3e00640 1 --2- 172.21.15.26:0/2992083006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec1447f0 0x7fd1ec15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1e3e00640 1 -- 172.21.15.26:0/2992083006 >> v1:172.21.15.26:6789/0 conn(0x7fd1ec06dab0 legacy=0x7fd1ec15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.168+0000 7fd1e3e00640 1 -- 172.21.15.26:0/2992083006 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec06ca10 con 0x7fd1ec06c470 2024-01-15T17:55:49.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.169+0000 7fd1e3e00640 1 --2- 172.21.15.26:0/2992083006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06c470 0x7fd1ec0cc5e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009870 tx=0x7fd1d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d34f3d12a131f15 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.170+0000 7fd1ea000640 1 -- 172.21.15.26:0/2992083006 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4031640 con 0x7fd1ec06c470 2024-01-15T17:55:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.170+0000 7fd1ea000640 1 -- 172.21.15.26:0/2992083006 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d40317a0 con 0x7fd1ec06c470 2024-01-15T17:55:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.170+0000 7fd1ea000640 1 -- 172.21.15.26:0/2992083006 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4031b30 con 0x7fd1ec06c470 2024-01-15T17:55:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.170+0000 7fd1ebe00640 1 -- 172.21.15.26:0/2992083006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06c470 msgr2=0x7fd1ec0cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.170+0000 7fd1ebe00640 1 --2- 172.21.15.26:0/2992083006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06c470 0x7fd1ec0cc5e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009870 tx=0x7fd1d402b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.170+0000 7fd1ebe00640 1 -- 172.21.15.26:0/2992083006 shutdown_connections 2024-01-15T17:55:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.170+0000 7fd1ebe00640 1 --2- 172.21.15.26:0/2992083006 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06c470 0x7fd1ec0cc5e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.170+0000 7fd1ebe00640 1 --2- 172.21.15.26:0/2992083006 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec1447f0 0x7fd1ec15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.170+0000 7fd1ebe00640 1 -- 172.21.15.26:0/2992083006 >> 172.21.15.26:0/2992083006 conn(0x7fd1ec06a3e0 msgr2=0x7fd1ec06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.171+0000 7fd1ebe00640 1 -- 172.21.15.26:0/2992083006 shutdown_connections 2024-01-15T17:55:49.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.171+0000 7fd1ebe00640 1 -- 172.21.15.26:0/2992083006 wait complete. 2024-01-15T17:55:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.173+0000 7fd1ebe00640 1 Processor -- start 2024-01-15T17:55:49.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.173+0000 7fd1ebe00640 1 -- start start 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1ebe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec06c470 0x7fd1ec1f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06dab0 0x7fd1ec13af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1ebe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1ec1447f0 0x7fd1ec157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1ebe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec13d1f0 con 0x7fd1ec06dab0 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1ebe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec13b4c0 con 0x7fd1ec1447f0 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1ebe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec13c890 con 0x7fd1ec06c470 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06dab0 0x7fd1ec13af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06dab0 0x7fd1ec13af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32854/0 (socket says 172.21.15.26:32854) 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e3e00640 1 -- 172.21.15.26:0/487178830 learned_addr learned my addr 172.21.15.26:0/487178830 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1eaa00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec06c470 0x7fd1ec1f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e3e00640 1 -- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec06c470 msgr2=0x7fd1ec1f0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1eb400640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1ec1447f0 0x7fd1ec157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e3e00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec06c470 0x7fd1ec1f0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e3e00640 1 -- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1ec1447f0 msgr2=0x7fd1ec157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e3e00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1ec1447f0 0x7fd1ec157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e3e00640 1 -- 172.21.15.26:0/487178830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d40094d0 con 0x7fd1ec06dab0 2024-01-15T17:55:49.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1eb400640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1ec1447f0 0x7fd1ec157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:49.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e3e00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06dab0 0x7fd1ec13af80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4000c00 tx=0x7fd1d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:49.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e2a00640 1 -- 172.21.15.26:0/487178830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d403ac60 con 0x7fd1ec06dab0 2024-01-15T17:55:49.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e2a00640 1 -- 172.21.15.26:0/487178830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d403adc0 con 0x7fd1ec06dab0 2024-01-15T17:55:49.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1e2a00640 1 -- 172.21.15.26:0/487178830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4044630 con 0x7fd1ec06dab0 2024-01-15T17:55:49.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.175+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec13cb30 con 0x7fd1ec06dab0 2024-01-15T17:55:49.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.176+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec13b6f0 con 0x7fd1ec06dab0 2024-01-15T17:55:49.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.177+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec13d330 con 0x7fd1ec06dab0 2024-01-15T17:55:49.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.178+0000 7fd1e2a00640 1 -- 172.21.15.26:0/487178830 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd1d402f050 con 0x7fd1ec06dab0 2024-01-15T17:55:49.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.180+0000 7fd1e2a00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd1c406b2c0 0x7fd1c406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.180+0000 7fd1e2a00640 1 -- 172.21.15.26:0/487178830 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1d40aba40 con 0x7fd1ec06dab0 2024-01-15T17:55:49.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.180+0000 7fd1eaa00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd1c406b2c0 0x7fd1c406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.180+0000 7fd1eaa00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd1c406b2c0 0x7fd1c406d780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc006fd0 tx=0x7fd1dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:49.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.183+0000 7fd1e2a00640 1 -- 172.21.15.26:0/487178830 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1d40ade60 con 0x7fd1ec06dab0 2024-01-15T17:55:49.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.376+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd1ec0ce0b0 con 0x7fd1ec06dab0 2024-01-15T17:55:49.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.381+0000 7fd1e2a00640 1 -- 172.21.15.26:0/487178830 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1ec0ce0b0 con 0x7fd1ec06dab0 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd1c406b2c0 msgr2=0x7fd1c406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd1c406b2c0 0x7fd1c406d780 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc006fd0 tx=0x7fd1dc008040 comp rx=0 tx=0).stop 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06dab0 msgr2=0x7fd1ec13af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06dab0 0x7fd1ec13af80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4000c00 tx=0x7fd1d40046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 shutdown_connections 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd1c406b2c0 0x7fd1c406d780 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1ec1447f0 0x7fd1ec157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1ec06dab0 0x7fd1ec13af80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 --2- 172.21.15.26:0/487178830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1ec06c470 0x7fd1ec1f0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.384+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 >> 172.21.15.26:0/487178830 conn(0x7fd1ec06a3e0 msgr2=0x7fd1ec06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.385+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 shutdown_connections 2024-01-15T17:55:49.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.385+0000 7fd1ebe00640 1 -- 172.21.15.26:0/487178830 wait complete. 2024-01-15T17:55:49.409 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T17:55:49.409 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T17:55:49.531 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T17:55:49.650 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T17:55:49.650 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:55:49.650 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:55:49.710 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:55:49.710 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:55:49.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.982+0000 7f56e1a00640 1 Processor -- start 2024-01-15T17:55:49.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.982+0000 7f56e1a00640 1 -- start start 2024-01-15T17:55:49.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.984+0000 7f56e1a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56dc160920 0x7f56dc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.985+0000 7f56e1a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc0cf9f0 0x7f56dc0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.985+0000 7f56e1a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56dc0d0a10 con 0x7f56dc161010 2024-01-15T17:55:49.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.985+0000 7f56e1a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56dc0d03b0 con 0x7f56dc160920 2024-01-15T17:55:49.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.985+0000 7f56e1a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0d1f50 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.985+0000 7f56dbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56dc160920 0x7f56dc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.985+0000 7f56dbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f56dc160920 0x7f56dc0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32870/0 (socket says 172.21.15.26:32870) 2024-01-15T17:55:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.985+0000 7f56dbe00640 1 -- 172.21.15.26:0/1710416958 learned_addr learned my addr 172.21.15.26:0/1710416958 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.985+0000 7f56d3200640 1 --2- 172.21.15.26:0/1710416958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc0cf9f0 0x7f56dc0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.986+0000 7f56dbe00640 1 -- 172.21.15.26:0/1710416958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc0cf9f0 msgr2=0x7f56dc0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.986+0000 7f56dbe00640 1 --2- 172.21.15.26:0/1710416958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc0cf9f0 0x7f56dc0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.986+0000 7f56dbe00640 1 -- 172.21.15.26:0/1710416958 >> v1:172.21.15.124:6789/0 conn(0x7f56dc161010 legacy=0x7f56dc1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.986+0000 7f56dbe00640 1 -- 172.21.15.26:0/1710416958 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc0cf080 con 0x7f56dc160920 2024-01-15T17:55:49.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.986+0000 7f56dbe00640 1 --2- 172.21.15.26:0/1710416958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc160920 0x7f56dc0b3f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009870 tx=0x7f56cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c022b5494e52ce server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:49.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.986+0000 7f56db400640 1 -- 172.21.15.26:0/1710416958 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc030840 con 0x7f56dc160920 2024-01-15T17:55:49.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.986+0000 7f56db400640 1 -- 172.21.15.26:0/1710416958 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc0309a0 con 0x7f56dc160920 2024-01-15T17:55:49.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.986+0000 7f56db400640 1 -- 172.21.15.26:0/1710416958 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc030d30 con 0x7f56dc160920 2024-01-15T17:55:49.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.987+0000 7f56e1a00640 1 -- 172.21.15.26:0/1710416958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc160920 msgr2=0x7f56dc0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.987+0000 7f56e1a00640 1 --2- 172.21.15.26:0/1710416958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc160920 0x7f56dc0b3f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009870 tx=0x7f56cc02b980 comp rx=0 tx=0).stop 2024-01-15T17:55:49.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.987+0000 7f56e1a00640 1 -- 172.21.15.26:0/1710416958 shutdown_connections 2024-01-15T17:55:49.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.987+0000 7f56e1a00640 1 --2- 172.21.15.26:0/1710416958 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc0cf9f0 0x7f56dc0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.987+0000 7f56e1a00640 1 --2- 172.21.15.26:0/1710416958 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc160920 0x7f56dc0b3f30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.987+0000 7f56e1a00640 1 -- 172.21.15.26:0/1710416958 >> 172.21.15.26:0/1710416958 conn(0x7f56dc0d3090 msgr2=0x7f56dc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:49.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.987+0000 7f56e1a00640 1 -- 172.21.15.26:0/1710416958 shutdown_connections 2024-01-15T17:55:49.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.987+0000 7f56e1a00640 1 -- 172.21.15.26:0/1710416958 wait complete. 2024-01-15T17:55:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.989+0000 7f56e1a00640 1 Processor -- start 2024-01-15T17:55:49.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.989+0000 7f56e1a00640 1 -- start start 2024-01-15T17:55:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56e1a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc0cf9f0 0x7f56dc0b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56e1a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc160920 0x7f56dc1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56e1a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56dc161010 0x7f56dc0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56e1a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0cbe20 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56e1a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0b8a70 con 0x7f56dc161010 2024-01-15T17:55:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56e1a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0b6150 con 0x7f56dc160920 2024-01-15T17:55:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56dbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc0cf9f0 0x7f56dc0b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56dbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc0cf9f0 0x7f56dc0b8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32880/0 (socket says 172.21.15.26:32880) 2024-01-15T17:55:49.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56dbe00640 1 -- 172.21.15.26:0/1170545482 learned_addr learned my addr 172.21.15.26:0/1170545482 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56d3200640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc160920 0x7f56dc1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56e1000640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56dc161010 0x7f56dc0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56dbe00640 1 -- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc160920 msgr2=0x7f56dc1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56dbe00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc160920 0x7f56dc1571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56dbe00640 1 -- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56dc161010 msgr2=0x7f56dc0b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56dbe00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56dc161010 0x7f56dc0b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.991+0000 7f56dbe00640 1 -- 172.21.15.26:0/1170545482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56cc0094d0 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.993+0000 7f56dbe00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc0cf9f0 0x7f56dc0b8530 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009870 tx=0x7f56cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.993+0000 7f56d8c00640 1 -- 172.21.15.26:0/1170545482 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc039c90 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.993+0000 7f56d8c00640 1 -- 172.21.15.26:0/1170545482 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc039df0 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.993+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56dc0cc070 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.993+0000 7f56d8c00640 1 -- 172.21.15.26:0/1170545482 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc043640 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.993+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc0cc630 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.995+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56dc160180 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.996+0000 7f56d8c00640 1 -- 172.21.15.26:0/1170545482 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f56cc0437a0 con 0x7f56dc0cf9f0 2024-01-15T17:55:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.996+0000 7f56d8c00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56b406b2c0 0x7f56b406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:49.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.997+0000 7f56d8c00640 1 -- 172.21.15.26:0/1170545482 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56cc03e070 con 0x7f56dc0cf9f0 2024-01-15T17:55:50.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.997+0000 7f56d3200640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56b406b2c0 0x7f56b406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:49.997+0000 7f56d3200640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56b406b2c0 0x7f56b406d780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f56dc159f60 tx=0x7f56c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:50.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.000+0000 7f56d8c00640 1 -- 172.21.15.26:0/1170545482 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56cc077b70 con 0x7f56dc0cf9f0 2024-01-15T17:55:50.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.195+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56dc067450 con 0x7f56dc0cf9f0 2024-01-15T17:55:50.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.198+0000 7f56d8c00640 1 -- 172.21.15.26:0/1170545482 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f56cc07b190 con 0x7f56dc0cf9f0 2024-01-15T17:55:50.202 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:50.202 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":22,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:05.714386+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:50.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56b406b2c0 msgr2=0x7f56b406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56b406b2c0 0x7f56b406d780 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f56dc159f60 tx=0x7f56c4008040 comp rx=0 tx=0).stop 2024-01-15T17:55:50.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc0cf9f0 msgr2=0x7f56dc0b8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc0cf9f0 0x7f56dc0b8530 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009870 tx=0x7f56cc004730 comp rx=0 tx=0).stop 2024-01-15T17:55:50.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 shutdown_connections 2024-01-15T17:55:50.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56dc160920 0x7f56dc1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56b406b2c0 0x7f56b406d780 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56dc161010 0x7f56dc0b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 --2- 172.21.15.26:0/1170545482 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56dc0cf9f0 0x7f56dc0b8530 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 >> 172.21.15.26:0/1170545482 conn(0x7f56dc0d3090 msgr2=0x7f56dc0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:50.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 shutdown_connections 2024-01-15T17:55:50.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.201+0000 7f56e1a00640 1 -- 172.21.15.26:0/1170545482 wait complete. 2024-01-15T17:55:50.223 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:50.224 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:50.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.441+0000 7f577c400640 1 Processor -- start 2024-01-15T17:55:50.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.442+0000 7f577c400640 1 -- start start 2024-01-15T17:55:50.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.443+0000 7f577c400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57740b3ae0 0x7f577415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.443+0000 7f577c400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5774070c40 con 0x7f57740cf290 2024-01-15T17:55:50.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.443+0000 7f577c400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f577406c470 con 0x7f57740ae5a0 2024-01-15T17:55:50.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.443+0000 7f577c400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5774071450 con 0x7f57740b3ae0 2024-01-15T17:55:50.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.444+0000 7f5779000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57740b3ae0 0x7f577415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.444+0000 7f577a400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f57740cf290 0x7f57740b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57418/0 (socket says 172.21.15.26:57418) 2024-01-15T17:55:50.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.444+0000 7f577a400640 1 -- 172.21.15.26:0/529696653 learned_addr learned my addr 172.21.15.26:0/529696653 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:50.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.444+0000 7f5779000640 1 -- 172.21.15.26:0/529696653 >> v1:172.21.15.124:6789/0 conn(0x7f57740ae5a0 legacy=0x7f577406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.444+0000 7f5779000640 1 -- 172.21.15.26:0/529696653 >> v1:172.21.15.26:6789/0 conn(0x7f57740cf290 legacy=0x7f57740b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.444+0000 7f5773e00640 1 -- 172.21.15.26:0/529696653 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950623143 0 0) 0x7f5774070c40 con 0x7f57740cf290 2024-01-15T17:55:50.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.444+0000 7f5779000640 1 -- 172.21.15.26:0/529696653 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577406c750 con 0x7f57740b3ae0 2024-01-15T17:55:50.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.445+0000 7f5779000640 1 --2- 172.21.15.26:0/529696653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740b3ae0 0x7f577415e430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f575c0097a0 tx=0x7f575c02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323213ffa2b63d24 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:50.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.446+0000 7f5773e00640 1 -- 172.21.15.26:0/529696653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c02f810 con 0x7f57740b3ae0 2024-01-15T17:55:50.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.446+0000 7f5773e00640 1 -- 172.21.15.26:0/529696653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c02f970 con 0x7f57740b3ae0 2024-01-15T17:55:50.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.446+0000 7f5773e00640 1 -- 172.21.15.26:0/529696653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c02fd00 con 0x7f57740b3ae0 2024-01-15T17:55:50.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.446+0000 7f577c400640 1 -- 172.21.15.26:0/529696653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740b3ae0 msgr2=0x7f577415e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.446+0000 7f577c400640 1 --2- 172.21.15.26:0/529696653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740b3ae0 0x7f577415e430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f575c0097a0 tx=0x7f575c02b9f0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.447+0000 7f577c400640 1 -- 172.21.15.26:0/529696653 shutdown_connections 2024-01-15T17:55:50.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.447+0000 7f577c400640 1 --2- 172.21.15.26:0/529696653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740b3ae0 0x7f577415e430 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.447+0000 7f577c400640 1 -- 172.21.15.26:0/529696653 >> 172.21.15.26:0/529696653 conn(0x7f577406a800 msgr2=0x7f577406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:50.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.447+0000 7f577c400640 1 -- 172.21.15.26:0/529696653 shutdown_connections 2024-01-15T17:55:50.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.447+0000 7f577c400640 1 -- 172.21.15.26:0/529696653 wait complete. 2024-01-15T17:55:50.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.448+0000 7f577c400640 1 Processor -- start 2024-01-15T17:55:50.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.449+0000 7f577c400640 1 -- start start 2024-01-15T17:55:50.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.449+0000 7f577c400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57740ae5a0 0x7f5774153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.449+0000 7f577c400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57740b3ae0 0x7f57741f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.449+0000 7f577c400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740cf290 0x7f577414f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.449+0000 7f577c400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57741566c0 con 0x7f57740cf290 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.449+0000 7f577c400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57741f3820 con 0x7f57740ae5a0 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.449+0000 7f577c400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57741f3b00 con 0x7f57740b3ae0 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f577a400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740cf290 0x7f577414f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f577a400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740cf290 0x7f577414f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32896/0 (socket says 172.21.15.26:32896) 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f577a400640 1 -- 172.21.15.26:0/3387056744 learned_addr learned my addr 172.21.15.26:0/3387056744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f5779a00640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57740ae5a0 0x7f5774153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f5779000640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57740b3ae0 0x7f57741f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f577a400640 1 -- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57740b3ae0 msgr2=0x7f57741f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f577a400640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57740b3ae0 0x7f57741f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f577a400640 1 -- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57740ae5a0 msgr2=0x7f5774153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f577a400640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57740ae5a0 0x7f5774153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.450+0000 7f577a400640 1 -- 172.21.15.26:0/3387056744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c009400 con 0x7f57740cf290 2024-01-15T17:55:50.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.451+0000 7f577a400640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740cf290 0x7f577414f090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f576c00c780 tx=0x7f576c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:50.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.451+0000 7f5771600640 1 -- 172.21.15.26:0/3387056744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c00ce90 con 0x7f57740cf290 2024-01-15T17:55:50.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.451+0000 7f5771600640 1 -- 172.21.15.26:0/3387056744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c00a640 con 0x7f57740cf290 2024-01-15T17:55:50.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.451+0000 7f5771600640 1 -- 172.21.15.26:0/3387056744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c00a9d0 con 0x7f57740cf290 2024-01-15T17:55:50.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.451+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577415ecf0 con 0x7f57740cf290 2024-01-15T17:55:50.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.451+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5774152f60 con 0x7f57740cf290 2024-01-15T17:55:50.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.453+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577415eb50 con 0x7f57740cf290 2024-01-15T17:55:50.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.453+0000 7f5771600640 1 -- 172.21.15.26:0/3387056744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f576c00ab30 con 0x7f57740cf290 2024-01-15T17:55:50.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.455+0000 7f5771600640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f574c06b2c0 0x7f574c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.455+0000 7f5771600640 1 -- 172.21.15.26:0/3387056744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f576c0890f0 con 0x7f57740cf290 2024-01-15T17:55:50.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.455+0000 7f5779a00640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f574c06b2c0 0x7f574c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.456+0000 7f5779a00640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f574c06b2c0 0x7f574c06d780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f57740b2680 tx=0x7f5764009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:50.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.459+0000 7f5771600640 1 -- 172.21.15.26:0/3387056744 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f577415eb50 con 0x7f57740cf290 2024-01-15T17:55:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.658+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f577415eb50 con 0x7f57740cf290 2024-01-15T17:55:50.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.659+0000 7f5771600640 1 -- 172.21.15.26:0/3387056744 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f577415eb50 con 0x7f57740cf290 2024-01-15T17:55:50.662 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:50.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f574c06b2c0 msgr2=0x7f574c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f574c06b2c0 0x7f574c06d780 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f57740b2680 tx=0x7f5764009040 comp rx=0 tx=0).stop 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740cf290 msgr2=0x7f577414f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740cf290 0x7f577414f090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f576c00c780 tx=0x7f576c00cc40 comp rx=0 tx=0).stop 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 shutdown_connections 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57740b3ae0 0x7f57741f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f574c06b2c0 0x7f574c06d780 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57740cf290 0x7f577414f090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 --2- 172.21.15.26:0/3387056744 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57740ae5a0 0x7f5774153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 >> 172.21.15.26:0/3387056744 conn(0x7f577406a800 msgr2=0x7f577415aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 shutdown_connections 2024-01-15T17:55:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.662+0000 7f577c400640 1 -- 172.21.15.26:0/3387056744 wait complete. 2024-01-15T17:55:50.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.908+0000 7fe6e2e00640 1 Processor -- start 2024-01-15T17:55:50.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.908+0000 7fe6e2e00640 1 -- start start 2024-01-15T17:55:50.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.910+0000 7fe6e2e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe6dc13e360 0x7fe6dc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.910+0000 7fe6e2e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe6dc15f9c0 0x7fe6dc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.910+0000 7fe6e2e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6dc15c5f0 con 0x7fe6dc13df80 2024-01-15T17:55:50.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.910+0000 7fe6e2e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6dc06c470 con 0x7fe6dc15f9c0 2024-01-15T17:55:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.910+0000 7fe6e2e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6dc06c750 con 0x7fe6dc13e360 2024-01-15T17:55:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.910+0000 7fe6dbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe6dc13df80 0x7fe6dc15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57426/0 (socket says 172.21.15.26:57426) 2024-01-15T17:55:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.910+0000 7fe6dbe00640 1 -- 172.21.15.26:0/4192933532 learned_addr learned my addr 172.21.15.26:0/4192933532 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.910+0000 7fe6e0e00640 1 --2- 172.21.15.26:0/4192933532 >> v2:172.21.15.26:3300/0 conn(0x7fe6dc15f9c0 0x7fe6dc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.911+0000 7fe6db400640 1 --2- 172.21.15.26:0/4192933532 >> v2:172.21.15.124:3300/0 conn(0x7fe6dc13e360 0x7fe6dc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.911+0000 7fe6daa00640 1 -- 172.21.15.26:0/4192933532 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029729341 0 0) 0x7fe6dc15c5f0 con 0x7fe6dc13df80 2024-01-15T17:55:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.911+0000 7fe6daa00640 1 -- 172.21.15.26:0/4192933532 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6bc003520 con 0x7fe6dc13df80 2024-01-15T17:55:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.911+0000 7fe6e0e00640 1 -- 172.21.15.26:0/4192933532 >> v2:172.21.15.124:3300/0 conn(0x7fe6dc13e360 msgr2=0x7fe6dc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.911+0000 7fe6e0e00640 1 --2- 172.21.15.26:0/4192933532 >> v2:172.21.15.124:3300/0 conn(0x7fe6dc13e360 0x7fe6dc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.911+0000 7fe6e0e00640 1 -- 172.21.15.26:0/4192933532 >> v1:172.21.15.26:6789/0 conn(0x7fe6dc13df80 legacy=0x7fe6dc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.911+0000 7fe6e0e00640 1 -- 172.21.15.26:0/4192933532 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc06ca30 con 0x7fe6dc15f9c0 2024-01-15T17:55:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.912+0000 7fe6e0e00640 1 --2- 172.21.15.26:0/4192933532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc15f9c0 0x7fe6dc0cf7d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc06ded0 tx=0x7fe6d402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def950f60882fff6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.913+0000 7fe6daa00640 1 -- 172.21.15.26:0/4192933532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d402cc70 con 0x7fe6dc15f9c0 2024-01-15T17:55:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.913+0000 7fe6daa00640 1 -- 172.21.15.26:0/4192933532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d402cdd0 con 0x7fe6dc15f9c0 2024-01-15T17:55:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.913+0000 7fe6daa00640 1 -- 172.21.15.26:0/4192933532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4034660 con 0x7fe6dc15f9c0 2024-01-15T17:55:50.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.913+0000 7fe6e2e00640 1 -- 172.21.15.26:0/4192933532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc15f9c0 msgr2=0x7fe6dc0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.913+0000 7fe6e2e00640 1 --2- 172.21.15.26:0/4192933532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc15f9c0 0x7fe6dc0cf7d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc06ded0 tx=0x7fe6d402bec0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.914+0000 7fe6e2e00640 1 -- 172.21.15.26:0/4192933532 shutdown_connections 2024-01-15T17:55:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.914+0000 7fe6e2e00640 1 --2- 172.21.15.26:0/4192933532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc15f9c0 0x7fe6dc0cf7d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.914+0000 7fe6e2e00640 1 --2- 172.21.15.26:0/4192933532 >> v2:172.21.15.124:3300/0 conn(0x7fe6dc13e360 0x7fe6dc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.914+0000 7fe6e2e00640 1 -- 172.21.15.26:0/4192933532 >> 172.21.15.26:0/4192933532 conn(0x7fe6dc06a800 msgr2=0x7fe6dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.914+0000 7fe6e2e00640 1 -- 172.21.15.26:0/4192933532 shutdown_connections 2024-01-15T17:55:50.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.914+0000 7fe6e2e00640 1 -- 172.21.15.26:0/4192933532 wait complete. 2024-01-15T17:55:50.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.915+0000 7fe6e2e00640 1 Processor -- start 2024-01-15T17:55:50.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.916+0000 7fe6e2e00640 1 -- start start 2024-01-15T17:55:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.916+0000 7fe6e2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6dc13df80 0x7fe6dc1f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.916+0000 7fe6e2e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc13e360 0x7fe6dc1f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.916+0000 7fe6e2e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc15f9c0 0x7fe6dc1f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6e2e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc0a8d50 con 0x7fe6dc13e360 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6e2e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc1f6970 con 0x7fe6dc15f9c0 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6e2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc1f6c50 con 0x7fe6dc13df80 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6db400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc13e360 0x7fe6dc1f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6db400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc13e360 0x7fe6dc1f5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32928/0 (socket says 172.21.15.26:32928) 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6db400640 1 -- 172.21.15.26:0/2935175023 learned_addr learned my addr 172.21.15.26:0/2935175023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6dbe00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6dc13df80 0x7fe6dc1f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6e0e00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc15f9c0 0x7fe6dc1f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6db400640 1 -- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6dc13df80 msgr2=0x7fe6dc1f3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6db400640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6dc13df80 0x7fe6dc1f3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6db400640 1 -- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc15f9c0 msgr2=0x7fe6dc1f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6db400640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc15f9c0 0x7fe6dc1f6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.917+0000 7fe6db400640 1 -- 172.21.15.26:0/2935175023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d4009a70 con 0x7fe6dc13e360 2024-01-15T17:55:50.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.918+0000 7fe6db400640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc13e360 0x7fe6dc1f5e00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d7d0 tx=0x7fe6c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.918+0000 7fe6d3e00640 1 -- 172.21.15.26:0/2935175023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4018070 con 0x7fe6dc13e360 2024-01-15T17:55:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.918+0000 7fe6d3e00640 1 -- 172.21.15.26:0/2935175023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c4004020 con 0x7fe6dc13e360 2024-01-15T17:55:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.918+0000 7fe6d3e00640 1 -- 172.21.15.26:0/2935175023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c40041b0 con 0x7fe6dc13e360 2024-01-15T17:55:50.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.918+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6dc1f2b60 con 0x7fe6dc13e360 2024-01-15T17:55:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.918+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6dc1f2810 con 0x7fe6dc13e360 2024-01-15T17:55:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.919+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6dc1f29a0 con 0x7fe6dc13e360 2024-01-15T17:55:50.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.920+0000 7fe6d3e00640 1 -- 172.21.15.26:0/2935175023 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe6c4004310 con 0x7fe6dc13e360 2024-01-15T17:55:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.921+0000 7fe6d3e00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6b406b310 0x7fe6b406d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.921+0000 7fe6d3e00640 1 -- 172.21.15.26:0/2935175023 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6c4089960 con 0x7fe6dc13e360 2024-01-15T17:55:50.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.922+0000 7fe6dbe00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6b406b310 0x7fe6b406d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:50.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.923+0000 7fe6dbe00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6b406b310 0x7fe6b406d7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc005880 tx=0x7fe6cc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:50.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:50.925+0000 7fe6d3e00640 1 -- 172.21.15.26:0/2935175023 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6c4054940 con 0x7fe6dc13e360 2024-01-15T17:55:51.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.124+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6dc0675d0 con 0x7fe6dc13e360 2024-01-15T17:55:51.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.125+0000 7fe6d3e00640 1 -- 172.21.15.26:0/2935175023 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6c4057f60 con 0x7fe6dc13e360 2024-01-15T17:55:51.128 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:51.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6b406b310 msgr2=0x7fe6b406d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6b406b310 0x7fe6b406d7d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc005880 tx=0x7fe6cc009040 comp rx=0 tx=0).stop 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc13e360 msgr2=0x7fe6dc1f5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc13e360 0x7fe6dc1f5e00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400d7d0 tx=0x7fe6c400dc90 comp rx=0 tx=0).stop 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 shutdown_connections 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6b406b310 0x7fe6b406d7d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc15f9c0 0x7fe6dc1f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc13e360 0x7fe6dc1f5e00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 --2- 172.21.15.26:0/2935175023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6dc13df80 0x7fe6dc1f3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 >> 172.21.15.26:0/2935175023 conn(0x7fe6dc06a800 msgr2=0x7fe6dc0b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 shutdown_connections 2024-01-15T17:55:51.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.127+0000 7fe6e2e00640 1 -- 172.21.15.26:0/2935175023 wait complete. 2024-01-15T17:55:51.151 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.363+0000 7f2d79000640 1 Processor -- start 2024-01-15T17:55:51.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.363+0000 7f2d79000640 1 -- start start 2024-01-15T17:55:51.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.364+0000 7f2d79000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d74152250 0x7f2d7415c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.364+0000 7f2d79000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2d7406b3f0 0x7f2d740b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.364+0000 7f2d79000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d740af150 con 0x7f2d7406ebc0 2024-01-15T17:55:51.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.364+0000 7f2d79000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d740ce050 con 0x7f2d7406b3f0 2024-01-15T17:55:51.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.364+0000 7f2d79000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d74152630 con 0x7f2d74152250 2024-01-15T17:55:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.365+0000 7f2d71e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d74152250 0x7f2d7415c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.365+0000 7f2d71e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d74152250 0x7f2d7415c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:43064/0 (socket says 172.21.15.26:43064) 2024-01-15T17:55:51.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.365+0000 7f2d71e00640 1 -- 172.21.15.26:0/1353944467 learned_addr learned my addr 172.21.15.26:0/1353944467 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.366+0000 7f2d71400640 1 --2- 172.21.15.26:0/1353944467 >> v2:172.21.15.124:3300/0 conn(0x7f2d7406b3f0 0x7f2d740b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.366+0000 7f2d71e00640 1 -- 172.21.15.26:0/1353944467 >> v2:172.21.15.124:3300/0 conn(0x7f2d7406b3f0 msgr2=0x7f2d740b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.366+0000 7f2d71e00640 1 --2- 172.21.15.26:0/1353944467 >> v2:172.21.15.124:3300/0 conn(0x7f2d7406b3f0 0x7f2d740b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.366+0000 7f2d71e00640 1 -- 172.21.15.26:0/1353944467 >> v1:172.21.15.124:6789/0 conn(0x7f2d7406ebc0 legacy=0x7f2d740ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.366+0000 7f2d71e00640 1 -- 172.21.15.26:0/1353944467 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740aeb90 con 0x7f2d74152250 2024-01-15T17:55:51.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.367+0000 7f2d71e00640 1 --2- 172.21.15.26:0/1353944467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d74152250 0x7f2d7415c360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009870 tx=0x7f2d5c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2661590668b2ba0c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.368+0000 7f2d70a00640 1 -- 172.21.15.26:0/1353944467 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c031640 con 0x7f2d74152250 2024-01-15T17:55:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.368+0000 7f2d70a00640 1 -- 172.21.15.26:0/1353944467 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c0317a0 con 0x7f2d74152250 2024-01-15T17:55:51.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.368+0000 7f2d70a00640 1 -- 172.21.15.26:0/1353944467 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c031b30 con 0x7f2d74152250 2024-01-15T17:55:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.369+0000 7f2d79000640 1 -- 172.21.15.26:0/1353944467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d74152250 msgr2=0x7f2d7415c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.369+0000 7f2d79000640 1 --2- 172.21.15.26:0/1353944467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d74152250 0x7f2d7415c360 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009870 tx=0x7f2d5c02b8f0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.369+0000 7f2d79000640 1 -- 172.21.15.26:0/1353944467 shutdown_connections 2024-01-15T17:55:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.369+0000 7f2d79000640 1 --2- 172.21.15.26:0/1353944467 >> v2:172.21.15.124:3300/0 conn(0x7f2d7406b3f0 0x7f2d740b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.369+0000 7f2d79000640 1 --2- 172.21.15.26:0/1353944467 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d74152250 0x7f2d7415c360 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.369+0000 7f2d79000640 1 -- 172.21.15.26:0/1353944467 >> 172.21.15.26:0/1353944467 conn(0x7f2d74070b10 msgr2=0x7f2d74070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.369+0000 7f2d79000640 1 -- 172.21.15.26:0/1353944467 shutdown_connections 2024-01-15T17:55:51.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.369+0000 7f2d79000640 1 -- 172.21.15.26:0/1353944467 wait complete. 2024-01-15T17:55:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.371+0000 7f2d79000640 1 Processor -- start 2024-01-15T17:55:51.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.371+0000 7f2d79000640 1 -- start start 2024-01-15T17:55:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.372+0000 7f2d79000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7406b3f0 0x7f2d741432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.372+0000 7f2d79000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d7406ebc0 0x7f2d7415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.372+0000 7f2d79000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d74152250 0x7f2d74154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.372+0000 7f2d79000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d74159cc0 con 0x7f2d7406b3f0 2024-01-15T17:55:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.372+0000 7f2d79000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741590f0 con 0x7f2d74152250 2024-01-15T17:55:51.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.372+0000 7f2d79000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741557d0 con 0x7f2d7406ebc0 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.372+0000 7f2d71e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7406b3f0 0x7f2d741432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.372+0000 7f2d71e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7406b3f0 0x7f2d741432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32940/0 (socket says 172.21.15.26:32940) 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.372+0000 7f2d71e00640 1 -- 172.21.15.26:0/1906187456 learned_addr learned my addr 172.21.15.26:0/1906187456 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.373+0000 7f2d72800640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d74152250 0x7f2d74154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.373+0000 7f2d71400640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d7406ebc0 0x7f2d7415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.373+0000 7f2d71e00640 1 -- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d7406ebc0 msgr2=0x7f2d7415a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.373+0000 7f2d71e00640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d7406ebc0 0x7f2d7415a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.373+0000 7f2d71e00640 1 -- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d74152250 msgr2=0x7f2d74154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.373+0000 7f2d71e00640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d74152250 0x7f2d74154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.373+0000 7f2d71e00640 1 -- 172.21.15.26:0/1906187456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0094d0 con 0x7f2d7406b3f0 2024-01-15T17:55:51.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.373+0000 7f2d71e00640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7406b3f0 0x7f2d741432f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c000c00 tx=0x7f2d5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.374+0000 7f2d6a000640 1 -- 172.21.15.26:0/1906187456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c03a9f0 con 0x7f2d7406b3f0 2024-01-15T17:55:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.374+0000 7f2d6a000640 1 -- 172.21.15.26:0/1906187456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c03ab90 con 0x7f2d7406b3f0 2024-01-15T17:55:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.374+0000 7f2d6a000640 1 -- 172.21.15.26:0/1906187456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c0443f0 con 0x7f2d7406b3f0 2024-01-15T17:55:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.374+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d74159320 con 0x7f2d7406b3f0 2024-01-15T17:55:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.374+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d741598e0 con 0x7f2d7406b3f0 2024-01-15T17:55:51.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.375+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d74159e00 con 0x7f2d7406b3f0 2024-01-15T17:55:51.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.376+0000 7f2d6a000640 1 -- 172.21.15.26:0/1906187456 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2d5c02f050 con 0x7f2d7406b3f0 2024-01-15T17:55:51.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.377+0000 7f2d6a000640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4c06b2c0 0x7f2d4c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.377+0000 7f2d6a000640 1 -- 172.21.15.26:0/1906187456 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2d5c0abb60 con 0x7f2d7406b3f0 2024-01-15T17:55:51.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.378+0000 7f2d71400640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4c06b2c0 0x7f2d4c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.379+0000 7f2d71400640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4c06b2c0 0x7f2d4c06d780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2d7414fd10 tx=0x7f2d6400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:51.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.381+0000 7f2d6a000640 1 -- 172.21.15.26:0/1906187456 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d5c076af0 con 0x7f2d7406b3f0 2024-01-15T17:55:51.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.574+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d740675d0 con 0x7f2d7406b3f0 2024-01-15T17:55:51.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.575+0000 7f2d6a000640 1 -- 172.21.15.26:0/1906187456 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d5c0350a0 con 0x7f2d7406b3f0 2024-01-15T17:55:51.579 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:51.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4c06b2c0 msgr2=0x7f2d4c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4c06b2c0 0x7f2d4c06d780 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2d7414fd10 tx=0x7f2d6400b040 comp rx=0 tx=0).stop 2024-01-15T17:55:51.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7406b3f0 msgr2=0x7f2d741432f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7406b3f0 0x7f2d741432f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c000c00 tx=0x7f2d5c0046c0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 shutdown_connections 2024-01-15T17:55:51.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d7406ebc0 0x7f2d7415a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4c06b2c0 0x7f2d4c06d780 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d74152250 0x7f2d74154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 --2- 172.21.15.26:0/1906187456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d7406b3f0 0x7f2d741432f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 >> 172.21.15.26:0/1906187456 conn(0x7f2d74070b10 msgr2=0x7f2d74151480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:51.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 shutdown_connections 2024-01-15T17:55:51.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.578+0000 7f2d79000640 1 -- 172.21.15.26:0/1906187456 wait complete. 2024-01-15T17:55:51.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:55:51.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.818+0000 7fdcb7200640 1 Processor -- start 2024-01-15T17:55:51.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.818+0000 7fdcb7200640 1 -- start start 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071410 0x7fdcb8146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcb7200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcb806c7c0 0x7fdcb80717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcb7200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb806cc20 con 0x7fdcb8071d30 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcb7200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb806cec0 con 0x7fdcb806c7c0 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcb7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb806d1a0 con 0x7fdcb8071410 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcafe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcb806c7c0 0x7fdcb80717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcafe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdcb806c7c0 0x7fdcb80717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32950/0 (socket says 172.21.15.26:32950) 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcafe00640 1 -- 172.21.15.26:0/3827377325 learned_addr learned my addr 172.21.15.26:0/3827377325 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcb6800640 1 --1- 172.21.15.26:0/3827377325 >> v1:172.21.15.26:6789/0 conn(0x7fdcb8071d30 0x7fdcb80b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57430/0 (socket says 172.21.15.26:57430) 2024-01-15T17:55:51.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.820+0000 7fdcb5e00640 1 --2- 172.21.15.26:0/3827377325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071410 0x7fdcb8146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.821+0000 7fdcb5400640 1 -- 172.21.15.26:0/3827377325 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354660440 0 0) 0x7fdcb806cc20 con 0x7fdcb8071d30 2024-01-15T17:55:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.821+0000 7fdcb5400640 1 -- 172.21.15.26:0/3827377325 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc98003520 con 0x7fdcb8071d30 2024-01-15T17:55:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.821+0000 7fdcafe00640 1 -- 172.21.15.26:0/3827377325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071410 msgr2=0x7fdcb8146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.821+0000 7fdcafe00640 1 --2- 172.21.15.26:0/3827377325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071410 0x7fdcb8146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.821+0000 7fdcafe00640 1 -- 172.21.15.26:0/3827377325 >> v1:172.21.15.26:6789/0 conn(0x7fdcb8071d30 legacy=0x7fdcb80b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.821+0000 7fdcafe00640 1 -- 172.21.15.26:0/3827377325 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb806d480 con 0x7fdcb806c7c0 2024-01-15T17:55:51.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.821+0000 7fdcb5e00640 1 --2- 172.21.15.26:0/3827377325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071410 0x7fdcb8146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.822+0000 7fdcafe00640 1 --2- 172.21.15.26:0/3827377325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb806c7c0 0x7fdcb80717f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009870 tx=0x7fdca002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba9ff7b7ce2776a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.822+0000 7fdcb5400640 1 -- 172.21.15.26:0/3827377325 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0030840 con 0x7fdcb806c7c0 2024-01-15T17:55:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.822+0000 7fdcb5400640 1 -- 172.21.15.26:0/3827377325 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca00309a0 con 0x7fdcb806c7c0 2024-01-15T17:55:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.822+0000 7fdcb5400640 1 -- 172.21.15.26:0/3827377325 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0030d10 con 0x7fdcb806c7c0 2024-01-15T17:55:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.823+0000 7fdcb7200640 1 -- 172.21.15.26:0/3827377325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb806c7c0 msgr2=0x7fdcb80717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.823+0000 7fdcb7200640 1 --2- 172.21.15.26:0/3827377325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb806c7c0 0x7fdcb80717f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009870 tx=0x7fdca002b980 comp rx=0 tx=0).stop 2024-01-15T17:55:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.823+0000 7fdcb7200640 1 -- 172.21.15.26:0/3827377325 shutdown_connections 2024-01-15T17:55:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.823+0000 7fdcb7200640 1 --2- 172.21.15.26:0/3827377325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb806c7c0 0x7fdcb80717f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.823+0000 7fdcb7200640 1 --2- 172.21.15.26:0/3827377325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071410 0x7fdcb8146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.823+0000 7fdcb7200640 1 -- 172.21.15.26:0/3827377325 >> 172.21.15.26:0/3827377325 conn(0x7fdcb806a800 msgr2=0x7fdcb806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.823+0000 7fdcb7200640 1 -- 172.21.15.26:0/3827377325 shutdown_connections 2024-01-15T17:55:51.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.823+0000 7fdcb7200640 1 -- 172.21.15.26:0/3827377325 wait complete. 2024-01-15T17:55:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.825+0000 7fdcb7200640 1 Processor -- start 2024-01-15T17:55:51.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.825+0000 7fdcb7200640 1 -- start start 2024-01-15T17:55:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.826+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcb806c7c0 0x7fdcb80c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.826+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb8071410 0x7fdcb813e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.826+0000 7fdcb7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071d30 0x7fdcb80c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.826+0000 7fdcb7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c5e10 con 0x7fdcb8071410 2024-01-15T17:55:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.826+0000 7fdcb7200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c4830 con 0x7fdcb806c7c0 2024-01-15T17:55:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.826+0000 7fdcb7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c4b10 con 0x7fdcb8071d30 2024-01-15T17:55:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.826+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb8071410 0x7fdcb813e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.826+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb8071410 0x7fdcb813e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32962/0 (socket says 172.21.15.26:32962) 2024-01-15T17:55:51.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.826+0000 7fdcafe00640 1 -- 172.21.15.26:0/1810670610 learned_addr learned my addr 172.21.15.26:0/1810670610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcb6800640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071d30 0x7fdcb80c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcb5e00640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcb806c7c0 0x7fdcb80c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcafe00640 1 -- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071d30 msgr2=0x7fdcb80c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcafe00640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071d30 0x7fdcb80c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcafe00640 1 -- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcb806c7c0 msgr2=0x7fdcb80c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcafe00640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcb806c7c0 0x7fdcb80c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcafe00640 1 -- 172.21.15.26:0/1810670610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca00094d0 con 0x7fdcb8071410 2024-01-15T17:55:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcb6800640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071d30 0x7fdcb80c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:51.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcafe00640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb8071410 0x7fdcb813e950 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdca0000c00 tx=0x7fdca0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcae000640 1 -- 172.21.15.26:0/1810670610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0039c90 con 0x7fdcb8071410 2024-01-15T17:55:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcae000640 1 -- 172.21.15.26:0/1810670610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca0039df0 con 0x7fdcb8071410 2024-01-15T17:55:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcae000640 1 -- 172.21.15.26:0/1810670610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0043600 con 0x7fdcb8071410 2024-01-15T17:55:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.827+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb80c4df0 con 0x7fdcb8071410 2024-01-15T17:55:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.828+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb80c53b0 con 0x7fdcb8071410 2024-01-15T17:55:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.829+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb80c55a0 con 0x7fdcb8071410 2024-01-15T17:55:51.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.830+0000 7fdcae000640 1 -- 172.21.15.26:0/1810670610 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdca0043760 con 0x7fdcb8071410 2024-01-15T17:55:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.831+0000 7fdcae000640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc9006b2c0 0x7fdc9006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.831+0000 7fdcae000640 1 -- 172.21.15.26:0/1810670610 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdca003e070 con 0x7fdcb8071410 2024-01-15T17:55:51.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.832+0000 7fdcb5e00640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc9006b2c0 0x7fdc9006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:51.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.833+0000 7fdcb5e00640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc9006b2c0 0x7fdc9006d780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdca8006fd0 tx=0x7fdca8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:51.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:51.836+0000 7fdcae000640 1 -- 172.21.15.26:0/1810670610 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdca0077bd0 con 0x7fdcb8071410 2024-01-15T17:55:52.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.058+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdcb8067450 con 0x7fdcb8071410 2024-01-15T17:55:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.058+0000 7fdcae000640 1 -- 172.21.15.26:0/1810670610 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdca0034080 con 0x7fdcb8071410 2024-01-15T17:55:52.062 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.061+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc9006b2c0 msgr2=0x7fdc9006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.061+0000 7fdcb7200640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc9006b2c0 0x7fdc9006d780 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdca8006fd0 tx=0x7fdca8008040 comp rx=0 tx=0).stop 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.061+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb8071410 msgr2=0x7fdcb813e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.061+0000 7fdcb7200640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb8071410 0x7fdcb813e950 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdca0000c00 tx=0x7fdca0004730 comp rx=0 tx=0).stop 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.062+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 shutdown_connections 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.062+0000 7fdcb7200640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdcb8071d30 0x7fdcb80c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.062+0000 7fdcb7200640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc9006b2c0 0x7fdc9006d780 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.062+0000 7fdcb7200640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdcb8071410 0x7fdcb813e950 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.062+0000 7fdcb7200640 1 --2- 172.21.15.26:0/1810670610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdcb806c7c0 0x7fdcb80c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.062+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 >> 172.21.15.26:0/1810670610 conn(0x7fdcb806a800 msgr2=0x7fdcb81410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:52.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.062+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 shutdown_connections 2024-01-15T17:55:52.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.062+0000 7fdcb7200640 1 -- 172.21.15.26:0/1810670610 wait complete. 2024-01-15T17:55:52.085 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:55:52.085 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:55:52.142 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:52.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.355+0000 7fed35a00640 1 Processor -- start 2024-01-15T17:55:52.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.355+0000 7fed35a00640 1 -- start start 2024-01-15T17:55:52.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.357+0000 7fed35a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fed30143c40 0x7fed30071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.357+0000 7fed35a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed300ccc10 0x7fed3006d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.357+0000 7fed35a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed30070e50 con 0x7fed300717d0 2024-01-15T17:55:52.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.357+0000 7fed35a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed300723c0 con 0x7fed300ccc10 2024-01-15T17:55:52.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.357+0000 7fed35a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed30144020 con 0x7fed30143c40 2024-01-15T17:55:52.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.357+0000 7fed35000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed300ccc10 0x7fed3006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.357+0000 7fed35000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fed300ccc10 0x7fed3006d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48176/0 (socket says 172.21.15.26:48176) 2024-01-15T17:55:52.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.357+0000 7fed35000640 1 -- 172.21.15.26:0/707364613 learned_addr learned my addr 172.21.15.26:0/707364613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:52.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.358+0000 7fed2fe00640 1 --2- 172.21.15.26:0/707364613 >> v2:172.21.15.124:3300/0 conn(0x7fed30143c40 0x7fed30071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.358+0000 7fed35000640 1 -- 172.21.15.26:0/707364613 >> v2:172.21.15.124:3300/0 conn(0x7fed30143c40 msgr2=0x7fed30071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:52.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.358+0000 7fed35000640 1 --2- 172.21.15.26:0/707364613 >> v2:172.21.15.124:3300/0 conn(0x7fed30143c40 0x7fed30071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.358+0000 7fed35000640 1 -- 172.21.15.26:0/707364613 >> v1:172.21.15.124:6789/0 conn(0x7fed300717d0 legacy=0x7fed300cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:52.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.358+0000 7fed35000640 1 -- 172.21.15.26:0/707364613 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed30071e00 con 0x7fed300ccc10 2024-01-15T17:55:52.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.359+0000 7fed35000640 1 --2- 172.21.15.26:0/707364613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed300ccc10 0x7fed3006d160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fed3006e410 tx=0x7fed2802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df65aa3e1984a735 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:52.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.360+0000 7fed2ea00640 1 -- 172.21.15.26:0/707364613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2802c9f0 con 0x7fed300ccc10 2024-01-15T17:55:52.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.360+0000 7fed2ea00640 1 -- 172.21.15.26:0/707364613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2802cb50 con 0x7fed300ccc10 2024-01-15T17:55:52.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.360+0000 7fed2ea00640 1 -- 172.21.15.26:0/707364613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280343f0 con 0x7fed300ccc10 2024-01-15T17:55:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.360+0000 7fed35a00640 1 -- 172.21.15.26:0/707364613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed300ccc10 msgr2=0x7fed3006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.360+0000 7fed35a00640 1 --2- 172.21.15.26:0/707364613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed300ccc10 0x7fed3006d160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fed3006e410 tx=0x7fed2802be50 comp rx=0 tx=0).stop 2024-01-15T17:55:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.361+0000 7fed35a00640 1 -- 172.21.15.26:0/707364613 shutdown_connections 2024-01-15T17:55:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.361+0000 7fed35a00640 1 --2- 172.21.15.26:0/707364613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed300ccc10 0x7fed3006d160 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.361+0000 7fed35a00640 1 --2- 172.21.15.26:0/707364613 >> v2:172.21.15.124:3300/0 conn(0x7fed30143c40 0x7fed30071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.361+0000 7fed35a00640 1 -- 172.21.15.26:0/707364613 >> 172.21.15.26:0/707364613 conn(0x7fed3006a800 msgr2=0x7fed3006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.361+0000 7fed35a00640 1 -- 172.21.15.26:0/707364613 shutdown_connections 2024-01-15T17:55:52.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.361+0000 7fed35a00640 1 -- 172.21.15.26:0/707364613 wait complete. 2024-01-15T17:55:52.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.362+0000 7fed35a00640 1 Processor -- start 2024-01-15T17:55:52.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.363+0000 7fed35a00640 1 -- start start 2024-01-15T17:55:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed300717d0 0x7fed301f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed300ccc10 0x7fed3013c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30143c40 0x7fed3013c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3013cc60 con 0x7fed30143c40 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3013df30 con 0x7fed300ccc10 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed300c5480 con 0x7fed300717d0 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30143c40 0x7fed3013c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30143c40 0x7fed3013c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48178/0 (socket says 172.21.15.26:48178) 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35000640 1 -- 172.21.15.26:0/1104955026 learned_addr learned my addr 172.21.15.26:0/1104955026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed2fe00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed300717d0 0x7fed301f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed2f400640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed300ccc10 0x7fed3013c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35000640 1 -- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed300717d0 msgr2=0x7fed301f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35000640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed300717d0 0x7fed301f29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35000640 1 -- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed300ccc10 msgr2=0x7fed3013c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35000640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed300ccc10 0x7fed3013c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.364+0000 7fed35000640 1 -- 172.21.15.26:0/1104955026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28009b30 con 0x7fed30143c40 2024-01-15T17:55:52.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.365+0000 7fed35000640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30143c40 0x7fed3013c720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fed28036040 tx=0x7fed28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.365+0000 7fed27e00640 1 -- 172.21.15.26:0/1104955026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28009510 con 0x7fed30143c40 2024-01-15T17:55:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.365+0000 7fed27e00640 1 -- 172.21.15.26:0/1104955026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2803e450 con 0x7fed30143c40 2024-01-15T17:55:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.365+0000 7fed27e00640 1 -- 172.21.15.26:0/1104955026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2803e7e0 con 0x7fed30143c40 2024-01-15T17:55:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.365+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3013d600 con 0x7fed30143c40 2024-01-15T17:55:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.365+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3013dbc0 con 0x7fed30143c40 2024-01-15T17:55:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.366+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3013e0a0 con 0x7fed30143c40 2024-01-15T17:55:52.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.367+0000 7fed27e00640 1 -- 172.21.15.26:0/1104955026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fed2803e940 con 0x7fed30143c40 2024-01-15T17:55:52.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.368+0000 7fed27e00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed0806b2c0 0x7fed0806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.368+0000 7fed27e00640 1 -- 172.21.15.26:0/1104955026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fed280acad0 con 0x7fed30143c40 2024-01-15T17:55:52.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.369+0000 7fed2fe00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed0806b2c0 0x7fed0806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.371+0000 7fed2fe00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed0806b2c0 0x7fed0806d780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fed20006fd0 tx=0x7fed20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:52.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.373+0000 7fed27e00640 1 -- 172.21.15.26:0/1104955026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed28077810 con 0x7fed30143c40 2024-01-15T17:55:52.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.567+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed30067450 con 0x7fed30143c40 2024-01-15T17:55:52.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.570+0000 7fed27e00640 1 -- 172.21.15.26:0/1104955026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7fed28043090 con 0x7fed30143c40 2024-01-15T17:55:52.573 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:52.573 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed0806b2c0 msgr2=0x7fed0806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed0806b2c0 0x7fed0806d780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fed20006fd0 tx=0x7fed20008040 comp rx=0 tx=0).stop 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30143c40 msgr2=0x7fed3013c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30143c40 0x7fed3013c720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fed28036040 tx=0x7fed28004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 shutdown_connections 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed0806b2c0 0x7fed0806d780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed30143c40 0x7fed3013c720 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed300ccc10 0x7fed3013c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 --2- 172.21.15.26:0/1104955026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed300717d0 0x7fed301f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 >> 172.21.15.26:0/1104955026 conn(0x7fed3006a800 msgr2=0x7fed300cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:52.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 shutdown_connections 2024-01-15T17:55:52.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.573+0000 7fed35a00640 1 -- 172.21.15.26:0/1104955026 wait complete. 2024-01-15T17:55:52.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:52.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:52.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.819+0000 7f1744e00640 1 Processor -- start 2024-01-15T17:55:52.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.819+0000 7f1744e00640 1 -- start start 2024-01-15T17:55:52.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.820+0000 7f1744e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f174015d4b0 0x7f17400d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.820+0000 7f1744e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 0x7f17400cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.820+0000 7f1744e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f17400cddf0 0x7f17400ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.820+0000 7f1744e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17400d0320 con 0x7f174015d4b0 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.820+0000 7f1744e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17400b2730 con 0x7f17400cddf0 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.820+0000 7f1744e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400d1ec0 con 0x7f17400cf9a0 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f174015d4b0 0x7f17400d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f174015d4b0 0x7f17400d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48182/0 (socket says 172.21.15.26:48182) 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173f400640 1 -- 172.21.15.26:0/3149922918 learned_addr learned my addr 172.21.15.26:0/3149922918 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173fe00640 1 --2- 172.21.15.26:0/3149922918 >> v2:172.21.15.124:3300/0 conn(0x7f17400cddf0 0x7f17400ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173ea00640 1 --2- 172.21.15.26:0/3149922918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 0x7f17400cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173f400640 1 -- 172.21.15.26:0/3149922918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 msgr2=0x7f17400cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173f400640 1 --2- 172.21.15.26:0/3149922918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 0x7f17400cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173f400640 1 -- 172.21.15.26:0/3149922918 >> v2:172.21.15.124:3300/0 conn(0x7f17400cddf0 msgr2=0x7f17400ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173f400640 1 --2- 172.21.15.26:0/3149922918 >> v2:172.21.15.124:3300/0 conn(0x7f17400cddf0 0x7f17400ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173f400640 1 -- 172.21.15.26:0/3149922918 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400b29d0 con 0x7f174015d4b0 2024-01-15T17:55:52.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.821+0000 7f173ea00640 1 --2- 172.21.15.26:0/3149922918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 0x7f17400cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:52.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.823+0000 7f173f400640 1 --2- 172.21.15.26:0/3149922918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 0x7f17400d1550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1730009e50 tx=0x7f173002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58d30cf3ace187e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:52.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.823+0000 7f173e000640 1 -- 172.21.15.26:0/3149922918 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173002ca70 con 0x7f174015d4b0 2024-01-15T17:55:52.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.823+0000 7f173e000640 1 -- 172.21.15.26:0/3149922918 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173002cbd0 con 0x7f174015d4b0 2024-01-15T17:55:52.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.823+0000 7f173e000640 1 -- 172.21.15.26:0/3149922918 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17300343f0 con 0x7f174015d4b0 2024-01-15T17:55:52.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.824+0000 7f1744e00640 1 -- 172.21.15.26:0/3149922918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 msgr2=0x7f17400d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.824+0000 7f1744e00640 1 --2- 172.21.15.26:0/3149922918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 0x7f17400d1550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1730009e50 tx=0x7f173002bed0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.824+0000 7f1744e00640 1 -- 172.21.15.26:0/3149922918 shutdown_connections 2024-01-15T17:55:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.824+0000 7f1744e00640 1 --2- 172.21.15.26:0/3149922918 >> v2:172.21.15.124:3300/0 conn(0x7f17400cddf0 0x7f17400ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.824+0000 7f1744e00640 1 --2- 172.21.15.26:0/3149922918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 0x7f17400cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.824+0000 7f1744e00640 1 --2- 172.21.15.26:0/3149922918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 0x7f17400d1550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.824+0000 7f1744e00640 1 -- 172.21.15.26:0/3149922918 >> 172.21.15.26:0/3149922918 conn(0x7f17400d3000 msgr2=0x7f17400d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.824+0000 7f1744e00640 1 -- 172.21.15.26:0/3149922918 shutdown_connections 2024-01-15T17:55:52.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.824+0000 7f1744e00640 1 -- 172.21.15.26:0/3149922918 wait complete. 2024-01-15T17:55:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.826+0000 7f1744e00640 1 Processor -- start 2024-01-15T17:55:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.826+0000 7f1744e00640 1 -- start start 2024-01-15T17:55:52.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.827+0000 7f1744e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17400cddf0 0x7f174009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.827+0000 7f1744e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 0x7f17400b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.827+0000 7f1744e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 0x7f1740153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.827+0000 7f1744e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400b6030 con 0x7f174015d4b0 2024-01-15T17:55:52.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.827+0000 7f1744e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17400b8300 con 0x7f17400cddf0 2024-01-15T17:55:52.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.827+0000 7f1744e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174009d350 con 0x7f17400cf9a0 2024-01-15T17:55:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.827+0000 7f173fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 0x7f1740153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 0x7f1740153bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48188/0 (socket says 172.21.15.26:48188) 2024-01-15T17:55:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173fe00640 1 -- 172.21.15.26:0/1645906025 learned_addr learned my addr 172.21.15.26:0/1645906025 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173f400640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17400cddf0 0x7f174009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173ea00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 0x7f17400b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173fe00640 1 -- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 msgr2=0x7f17400b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173fe00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 0x7f17400b7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173fe00640 1 -- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17400cddf0 msgr2=0x7f174009cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:52.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173fe00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17400cddf0 0x7f174009cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173fe00640 1 -- 172.21.15.26:0/1645906025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1730009b30 con 0x7f174015d4b0 2024-01-15T17:55:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.828+0000 7f173fe00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 0x7f1740153bc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f173800c4c0 tx=0x7f173800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.829+0000 7f1737400640 1 -- 172.21.15.26:0/1645906025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738004220 con 0x7f174015d4b0 2024-01-15T17:55:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.829+0000 7f1737400640 1 -- 172.21.15.26:0/1645906025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1738004380 con 0x7f174015d4b0 2024-01-15T17:55:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.829+0000 7f1737400640 1 -- 172.21.15.26:0/1645906025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738003810 con 0x7f174015d4b0 2024-01-15T17:55:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.829+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174015da10 con 0x7f174015d4b0 2024-01-15T17:55:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.829+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17400b5cc0 con 0x7f174015d4b0 2024-01-15T17:55:52.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.830+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17400b61e0 con 0x7f174015d4b0 2024-01-15T17:55:52.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.831+0000 7f1737400640 1 -- 172.21.15.26:0/1645906025 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1738003970 con 0x7f174015d4b0 2024-01-15T17:55:52.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.832+0000 7f1737400640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f171806b2f0 0x7f171806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:52.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.832+0000 7f1737400640 1 -- 172.21.15.26:0/1645906025 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f17380898c0 con 0x7f174015d4b0 2024-01-15T17:55:52.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.833+0000 7f173f400640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f171806b2f0 0x7f171806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:52.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.834+0000 7f173f400640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f171806b2f0 0x7f171806d7b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1730035000 tx=0x7f1730030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:52.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:52.836+0000 7f1737400640 1 -- 172.21.15.26:0/1645906025 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17400b61e0 con 0x7f174015d4b0 2024-01-15T17:55:53.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.031+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1740067450 con 0x7f174015d4b0 2024-01-15T17:55:53.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.032+0000 7f1737400640 1 -- 172.21.15.26:0/1645906025 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f173801e020 con 0x7f174015d4b0 2024-01-15T17:55:53.035 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f171806b2f0 msgr2=0x7f171806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f171806b2f0 0x7f171806d7b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1730035000 tx=0x7f1730030000 comp rx=0 tx=0).stop 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 msgr2=0x7f1740153bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 0x7f1740153bc0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f173800c4c0 tx=0x7f173800c980 comp rx=0 tx=0).stop 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 shutdown_connections 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f17400cf9a0 0x7f17400b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f171806b2f0 0x7f171806d7b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f174015d4b0 0x7f1740153bc0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 --2- 172.21.15.26:0/1645906025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f17400cddf0 0x7f174009cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 >> 172.21.15.26:0/1645906025 conn(0x7f17400d3000 msgr2=0x7f174006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 shutdown_connections 2024-01-15T17:55:53.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.035+0000 7f1744e00640 1 -- 172.21.15.26:0/1645906025 wait complete. 2024-01-15T17:55:53.058 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:53.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.285+0000 7f550be00640 1 Processor -- start 2024-01-15T17:55:53.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.285+0000 7f550be00640 1 -- start start 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.287+0000 7f550be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f550c0adf20 0x7f550c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.287+0000 7f550be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f550c15bf10 0x7f550c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.287+0000 7f550be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 0x7f550c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.287+0000 7f550be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550c15f970 con 0x7f550c0adf20 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.287+0000 7f550be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550c15f4e0 con 0x7f550c15bf10 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.287+0000 7f550be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0cce10 con 0x7f550c0cc980 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f550c0adf20 0x7f550c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f550c0adf20 0x7f550c070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48200/0 (socket says 172.21.15.26:48200) 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550aa00640 1 -- 172.21.15.26:0/2730059307 learned_addr learned my addr 172.21.15.26:0/2730059307 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:53.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550b400640 1 --2- 172.21.15.26:0/2730059307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 0x7f550c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f5503e00640 1 --2- 172.21.15.26:0/2730059307 >> v2:172.21.15.124:3300/0 conn(0x7f550c15bf10 0x7f550c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550aa00640 1 -- 172.21.15.26:0/2730059307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 msgr2=0x7f550c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550aa00640 1 --2- 172.21.15.26:0/2730059307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 0x7f550c143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550aa00640 1 -- 172.21.15.26:0/2730059307 >> v2:172.21.15.124:3300/0 conn(0x7f550c15bf10 msgr2=0x7f550c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550aa00640 1 --2- 172.21.15.26:0/2730059307 >> v2:172.21.15.124:3300/0 conn(0x7f550c15bf10 0x7f550c0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550aa00640 1 -- 172.21.15.26:0/2730059307 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c071130 con 0x7f550c0adf20 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f550b400640 1 --2- 172.21.15.26:0/2730059307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 0x7f550c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.288+0000 7f5503e00640 1 --2- 172.21.15.26:0/2730059307 >> v2:172.21.15.124:3300/0 conn(0x7f550c15bf10 0x7f550c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.289+0000 7f550aa00640 1 --2- 172.21.15.26:0/2730059307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c0adf20 0x7f550c070a30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f550c15e9e0 tx=0x7f54fc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2d28a35edc3b0d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.290+0000 7f550a000640 1 -- 172.21.15.26:0/2730059307 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc0317e0 con 0x7f550c0adf20 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.290+0000 7f550a000640 1 -- 172.21.15.26:0/2730059307 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc031940 con 0x7f550c0adf20 2024-01-15T17:55:53.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.290+0000 7f550a000640 1 -- 172.21.15.26:0/2730059307 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc031cd0 con 0x7f550c0adf20 2024-01-15T17:55:53.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.290+0000 7f550be00640 1 -- 172.21.15.26:0/2730059307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c0adf20 msgr2=0x7f550c070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.290+0000 7f550be00640 1 --2- 172.21.15.26:0/2730059307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c0adf20 0x7f550c070a30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f550c15e9e0 tx=0x7f54fc02b880 comp rx=0 tx=0).stop 2024-01-15T17:55:53.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.291+0000 7f550be00640 1 -- 172.21.15.26:0/2730059307 shutdown_connections 2024-01-15T17:55:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.291+0000 7f550be00640 1 --2- 172.21.15.26:0/2730059307 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 0x7f550c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.291+0000 7f550be00640 1 --2- 172.21.15.26:0/2730059307 >> v2:172.21.15.124:3300/0 conn(0x7f550c15bf10 0x7f550c0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.291+0000 7f550be00640 1 --2- 172.21.15.26:0/2730059307 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c0adf20 0x7f550c070a30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.291+0000 7f550be00640 1 -- 172.21.15.26:0/2730059307 >> 172.21.15.26:0/2730059307 conn(0x7f550c06a7d0 msgr2=0x7f550c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.291+0000 7f550be00640 1 -- 172.21.15.26:0/2730059307 shutdown_connections 2024-01-15T17:55:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.291+0000 7f550be00640 1 -- 172.21.15.26:0/2730059307 wait complete. 2024-01-15T17:55:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.292+0000 7f550be00640 1 Processor -- start 2024-01-15T17:55:53.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.293+0000 7f550be00640 1 -- start start 2024-01-15T17:55:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.293+0000 7f550be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0adf20 0x7f550c1f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.293+0000 7f550be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 0x7f550c1f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c15bf10 0x7f550c0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c1f6cd0 con 0x7f550c15bf10 2024-01-15T17:55:53.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0b21c0 con 0x7f550c0adf20 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0b24a0 con 0x7f550c0cc980 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c15bf10 0x7f550c0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c15bf10 0x7f550c0afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48216/0 (socket says 172.21.15.26:48216) 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550b400640 1 -- 172.21.15.26:0/2025515820 learned_addr learned my addr 172.21.15.26:0/2025515820 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f5503e00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 0x7f550c1f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550aa00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0adf20 0x7f550c1f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550b400640 1 -- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 msgr2=0x7f550c1f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550b400640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 0x7f550c1f2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550b400640 1 -- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0adf20 msgr2=0x7f550c1f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550b400640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0adf20 0x7f550c1f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.294+0000 7f550b400640 1 -- 172.21.15.26:0/2025515820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0094d0 con 0x7f550c15bf10 2024-01-15T17:55:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.295+0000 7f550b400640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c15bf10 0x7f550c0afe20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5504007e10 tx=0x7f550400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.295+0000 7f5503400640 1 -- 172.21.15.26:0/2025515820 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504003b90 con 0x7f550c15bf10 2024-01-15T17:55:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.295+0000 7f5503400640 1 -- 172.21.15.26:0/2025515820 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550400f040 con 0x7f550c15bf10 2024-01-15T17:55:53.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.295+0000 7f5503400640 1 -- 172.21.15.26:0/2025515820 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550400de80 con 0x7f550c15bf10 2024-01-15T17:55:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.295+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550c0b2780 con 0x7f550c15bf10 2024-01-15T17:55:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.295+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550c0b2d40 con 0x7f550c15bf10 2024-01-15T17:55:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.296+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550c067450 con 0x7f550c15bf10 2024-01-15T17:55:53.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.297+0000 7f5503400640 1 -- 172.21.15.26:0/2025515820 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5504004030 con 0x7f550c15bf10 2024-01-15T17:55:53.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.298+0000 7f5503400640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f54e406b2c0 0x7f54e406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.298+0000 7f5503400640 1 -- 172.21.15.26:0/2025515820 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5504089fe0 con 0x7f550c15bf10 2024-01-15T17:55:53.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.299+0000 7f550aa00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f54e406b2c0 0x7f54e406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.300+0000 7f550aa00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f54e406b2c0 0x7f54e406d780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54fc036000 tx=0x7f54fc03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:53.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.303+0000 7f5503400640 1 -- 172.21.15.26:0/2025515820 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5504054d60 con 0x7f550c15bf10 2024-01-15T17:55:53.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.497+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f550c0c8110 con 0x7f550c15bf10 2024-01-15T17:55:53.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.498+0000 7f5503400640 1 -- 172.21.15.26:0/2025515820 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f550401a020 con 0x7f550c15bf10 2024-01-15T17:55:53.502 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:53.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f54e406b2c0 msgr2=0x7f54e406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f54e406b2c0 0x7f54e406d780 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54fc036000 tx=0x7f54fc03d040 comp rx=0 tx=0).stop 2024-01-15T17:55:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c15bf10 msgr2=0x7f550c0afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c15bf10 0x7f550c0afe20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5504007e10 tx=0x7f550400d4a0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 shutdown_connections 2024-01-15T17:55:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f550c0cc980 0x7f550c1f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f54e406b2c0 0x7f54e406d780 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f550c15bf10 0x7f550c0afe20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 --2- 172.21.15.26:0/2025515820 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f550c0adf20 0x7f550c1f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 >> 172.21.15.26:0/2025515820 conn(0x7f550c06a7d0 msgr2=0x7f550c0aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 shutdown_connections 2024-01-15T17:55:53.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.501+0000 7f550be00640 1 -- 172.21.15.26:0/2025515820 wait complete. 2024-01-15T17:55:53.524 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:53.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.748+0000 7f9f0be00640 1 Processor -- start 2024-01-15T17:55:53.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.748+0000 7f9f0be00640 1 -- start start 2024-01-15T17:55:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.750+0000 7f9f0be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f0c0adf20 0x7f9f0c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.750+0000 7f9f0be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9f0c0ccb50 0x7f9f0c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.750+0000 7f9f0be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0c15f4e0 con 0x7f9f0c15bf10 2024-01-15T17:55:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.750+0000 7f9f0be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0c15f970 con 0x7f9f0c0adf20 2024-01-15T17:55:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.750+0000 7f9f0be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0c071130 con 0x7f9f0c0ccb50 2024-01-15T17:55:53.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.750+0000 7f9f0aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f0c0adf20 0x7f9f0c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.750+0000 7f9f0aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f0c0adf20 0x7f9f0c070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48218/0 (socket says 172.21.15.26:48218) 2024-01-15T17:55:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.750+0000 7f9f0aa00640 1 -- 172.21.15.26:0/1925495857 learned_addr learned my addr 172.21.15.26:0/1925495857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.750+0000 7f9f0b400640 1 --2- 172.21.15.26:0/1925495857 >> v2:172.21.15.124:3300/0 conn(0x7f9f0c0ccb50 0x7f9f0c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.751+0000 7f9f0aa00640 1 -- 172.21.15.26:0/1925495857 >> v2:172.21.15.124:3300/0 conn(0x7f9f0c0ccb50 msgr2=0x7f9f0c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.751+0000 7f9f0aa00640 1 --2- 172.21.15.26:0/1925495857 >> v2:172.21.15.124:3300/0 conn(0x7f9f0c0ccb50 0x7f9f0c143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.751+0000 7f9f0aa00640 1 -- 172.21.15.26:0/1925495857 >> v1:172.21.15.26:6789/0 conn(0x7f9f0c15bf10 legacy=0x7f9f0c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.751+0000 7f9f0aa00640 1 -- 172.21.15.26:0/1925495857 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c15c390 con 0x7f9f0c0adf20 2024-01-15T17:55:53.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.751+0000 7f9f0b400640 1 --2- 172.21.15.26:0/1925495857 >> v2:172.21.15.124:3300/0 conn(0x7f9f0c0ccb50 0x7f9f0c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.752+0000 7f9f0aa00640 1 --2- 172.21.15.26:0/1925495857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c0adf20 0x7f9f0c070a30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c06ff20 tx=0x7f9efc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3ddd0811ff05a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.752+0000 7f9f0a000640 1 -- 172.21.15.26:0/1925495857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc02ca80 con 0x7f9f0c0adf20 2024-01-15T17:55:53.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.752+0000 7f9f0a000640 1 -- 172.21.15.26:0/1925495857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc02cbe0 con 0x7f9f0c0adf20 2024-01-15T17:55:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.752+0000 7f9f0a000640 1 -- 172.21.15.26:0/1925495857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0343f0 con 0x7f9f0c0adf20 2024-01-15T17:55:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.753+0000 7f9f0be00640 1 -- 172.21.15.26:0/1925495857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c0adf20 msgr2=0x7f9f0c070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.753+0000 7f9f0be00640 1 --2- 172.21.15.26:0/1925495857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c0adf20 0x7f9f0c070a30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c06ff20 tx=0x7f9efc02bee0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.753+0000 7f9f0be00640 1 -- 172.21.15.26:0/1925495857 shutdown_connections 2024-01-15T17:55:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.753+0000 7f9f0be00640 1 --2- 172.21.15.26:0/1925495857 >> v2:172.21.15.124:3300/0 conn(0x7f9f0c0ccb50 0x7f9f0c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.753+0000 7f9f0be00640 1 --2- 172.21.15.26:0/1925495857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c0adf20 0x7f9f0c070a30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.753+0000 7f9f0be00640 1 -- 172.21.15.26:0/1925495857 >> 172.21.15.26:0/1925495857 conn(0x7f9f0c06a7d0 msgr2=0x7f9f0c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:53.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.753+0000 7f9f0be00640 1 -- 172.21.15.26:0/1925495857 shutdown_connections 2024-01-15T17:55:53.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.753+0000 7f9f0be00640 1 -- 172.21.15.26:0/1925495857 wait complete. 2024-01-15T17:55:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.755+0000 7f9f0be00640 1 Processor -- start 2024-01-15T17:55:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.755+0000 7f9f0be00640 1 -- start start 2024-01-15T17:55:53.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.756+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f0c0adf20 0x7f9f0c13be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.756+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f0c0ccb50 0x7f9f0c0ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.756+0000 7f9f0be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c15bf10 0x7f9f0c152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.756+0000 7f9f0be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c152c40 con 0x7f9f0c15bf10 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.756+0000 7f9f0be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c0afdf0 con 0x7f9f0c0adf20 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.756+0000 7f9f0be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c0b00d0 con 0x7f9f0c0ccb50 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.756+0000 7f9f0b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c15bf10 0x7f9f0c152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.756+0000 7f9f0b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c15bf10 0x7f9f0c152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48222/0 (socket says 172.21.15.26:48222) 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.756+0000 7f9f0b400640 1 -- 172.21.15.26:0/1938052421 learned_addr learned my addr 172.21.15.26:0/1938052421 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f0aa00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f0c0adf20 0x7f9f0c13be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f01e00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f0c0ccb50 0x7f9f0c0ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f0b400640 1 -- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f0c0ccb50 msgr2=0x7f9f0c0ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f0b400640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f0c0ccb50 0x7f9f0c0ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f0b400640 1 -- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f0c0adf20 msgr2=0x7f9f0c13be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f0b400640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f0c0adf20 0x7f9f0c13be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f0b400640 1 -- 172.21.15.26:0/1938052421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc009b30 con 0x7f9f0c15bf10 2024-01-15T17:55:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f0b400640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c15bf10 0x7f9f0c152700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400c7f0 tx=0x7f9f0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f03e00640 1 -- 172.21.15.26:0/1938052421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f04007820 con 0x7f9f0c15bf10 2024-01-15T17:55:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f03e00640 1 -- 172.21.15.26:0/1938052421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f04007980 con 0x7f9f0c15bf10 2024-01-15T17:55:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f03e00640 1 -- 172.21.15.26:0/1938052421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f04007d70 con 0x7f9f0c15bf10 2024-01-15T17:55:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.757+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0c13b360 con 0x7f9f0c15bf10 2024-01-15T17:55:53.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.758+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c13b920 con 0x7f9f0c15bf10 2024-01-15T17:55:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.759+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0c13b1c0 con 0x7f9f0c15bf10 2024-01-15T17:55:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.760+0000 7f9f03e00640 1 -- 172.21.15.26:0/1938052421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9f040113f0 con 0x7f9f0c15bf10 2024-01-15T17:55:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.761+0000 7f9f03e00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ee406b2c0 0x7f9ee406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.761+0000 7f9f03e00640 1 -- 172.21.15.26:0/1938052421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9f0408a040 con 0x7f9f0c15bf10 2024-01-15T17:55:53.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.761+0000 7f9f0aa00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ee406b2c0 0x7f9ee406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:53.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.762+0000 7f9f0aa00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ee406b2c0 0x7f9ee406d780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9efc035000 tx=0x7f9efc030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:53.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.765+0000 7f9f03e00640 1 -- 172.21.15.26:0/1938052421 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f0c13b1c0 con 0x7f9f0c15bf10 2024-01-15T17:55:53.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.959+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f0c13b1c0 con 0x7f9f0c15bf10 2024-01-15T17:55:53.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.960+0000 7f9f03e00640 1 -- 172.21.15.26:0/1938052421 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f0c13b1c0 con 0x7f9f0c15bf10 2024-01-15T17:55:53.963 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ee406b2c0 msgr2=0x7f9ee406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ee406b2c0 0x7f9ee406d780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9efc035000 tx=0x7f9efc030000 comp rx=0 tx=0).stop 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c15bf10 msgr2=0x7f9f0c152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c15bf10 0x7f9f0c152700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400c7f0 tx=0x7f9f0400ccb0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 shutdown_connections 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f0c0ccb50 0x7f9f0c0ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ee406b2c0 0x7f9ee406d780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f0c15bf10 0x7f9f0c152700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 --2- 172.21.15.26:0/1938052421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f0c0adf20 0x7f9f0c13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:53.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.963+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 >> 172.21.15.26:0/1938052421 conn(0x7f9f0c06a7d0 msgr2=0x7f9f0c143500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.964+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 shutdown_connections 2024-01-15T17:55:53.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:53.964+0000 7f9f0be00640 1 -- 172.21.15.26:0/1938052421 wait complete. 2024-01-15T17:55:53.987 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:55:54.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.193+0000 7fbe37200640 1 Processor -- start 2024-01-15T17:55:54.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.194+0000 7fbe37200640 1 -- start start 2024-01-15T17:55:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.195+0000 7fbe37200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe38160920 0x7fbe380b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.195+0000 7fbe37200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbe380cfea0 0x7fbe38161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.195+0000 7fbe37200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380d0a10 con 0x7fbe380cf9f0 2024-01-15T17:55:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.195+0000 7fbe37200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe380d1f50 con 0x7fbe38160920 2024-01-15T17:55:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.195+0000 7fbe37200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe380cf080 con 0x7fbe380cfea0 2024-01-15T17:55:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.195+0000 7fbe35e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe38160920 0x7fbe380b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.195+0000 7fbe35e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbe38160920 0x7fbe380b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48228/0 (socket says 172.21.15.26:48228) 2024-01-15T17:55:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.195+0000 7fbe35e00640 1 -- 172.21.15.26:0/409006405 learned_addr learned my addr 172.21.15.26:0/409006405 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:54.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.196+0000 7fbe36800640 1 --2- 172.21.15.26:0/409006405 >> v2:172.21.15.124:3300/0 conn(0x7fbe380cfea0 0x7fbe38161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.196+0000 7fbe35e00640 1 -- 172.21.15.26:0/409006405 >> v2:172.21.15.124:3300/0 conn(0x7fbe380cfea0 msgr2=0x7fbe38161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.196+0000 7fbe35e00640 1 --2- 172.21.15.26:0/409006405 >> v2:172.21.15.124:3300/0 conn(0x7fbe380cfea0 0x7fbe38161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.196+0000 7fbe35e00640 1 -- 172.21.15.26:0/409006405 >> v1:172.21.15.124:6789/0 conn(0x7fbe380cf9f0 legacy=0x7fbe38161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.196+0000 7fbe35e00640 1 -- 172.21.15.26:0/409006405 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380d0310 con 0x7fbe38160920 2024-01-15T17:55:54.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.196+0000 7fbe36800640 1 --2- 172.21.15.26:0/409006405 >> v2:172.21.15.124:3300/0 conn(0x7fbe380cfea0 0x7fbe38161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.197+0000 7fbe35e00640 1 --2- 172.21.15.26:0/409006405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38160920 0x7fbe380b3f30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe3808c2c0 tx=0x7fbe2802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964c452df974ef02 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.197+0000 7fbe34a00640 1 -- 172.21.15.26:0/409006405 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2802ca80 con 0x7fbe38160920 2024-01-15T17:55:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.197+0000 7fbe34a00640 1 -- 172.21.15.26:0/409006405 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2802cbe0 con 0x7fbe38160920 2024-01-15T17:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.197+0000 7fbe34a00640 1 -- 172.21.15.26:0/409006405 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe280343f0 con 0x7fbe38160920 2024-01-15T17:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.198+0000 7fbe37200640 1 -- 172.21.15.26:0/409006405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38160920 msgr2=0x7fbe380b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.198+0000 7fbe37200640 1 --2- 172.21.15.26:0/409006405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38160920 0x7fbe380b3f30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbe3808c2c0 tx=0x7fbe2802bee0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.198+0000 7fbe37200640 1 -- 172.21.15.26:0/409006405 shutdown_connections 2024-01-15T17:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.198+0000 7fbe37200640 1 --2- 172.21.15.26:0/409006405 >> v2:172.21.15.124:3300/0 conn(0x7fbe380cfea0 0x7fbe38161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.198+0000 7fbe37200640 1 --2- 172.21.15.26:0/409006405 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe38160920 0x7fbe380b3f30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.198+0000 7fbe37200640 1 -- 172.21.15.26:0/409006405 >> 172.21.15.26:0/409006405 conn(0x7fbe380d3090 msgr2=0x7fbe380d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.198+0000 7fbe37200640 1 -- 172.21.15.26:0/409006405 shutdown_connections 2024-01-15T17:55:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.198+0000 7fbe37200640 1 -- 172.21.15.26:0/409006405 wait complete. 2024-01-15T17:55:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.200+0000 7fbe37200640 1 Processor -- start 2024-01-15T17:55:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.200+0000 7fbe37200640 1 -- start start 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.201+0000 7fbe37200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380cf9f0 0x7fbe38151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.201+0000 7fbe37200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe380cfea0 0x7fbe380b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.201+0000 7fbe37200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe38160920 0x7fbe380b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.201+0000 7fbe37200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380b6a00 con 0x7fbe380cf9f0 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.201+0000 7fbe37200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3815b880 con 0x7fbe38160920 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.201+0000 7fbe37200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380b5df0 con 0x7fbe380cfea0 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.201+0000 7fbe35e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380cf9f0 0x7fbe38151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.201+0000 7fbe35e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380cf9f0 0x7fbe38151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48240/0 (socket says 172.21.15.26:48240) 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.201+0000 7fbe35e00640 1 -- 172.21.15.26:0/36489009 learned_addr learned my addr 172.21.15.26:0/36489009 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:54.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe36800640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe38160920 0x7fbe380b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe35e00640 1 -- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe380cfea0 msgr2=0x7fbe380b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe35e00640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe380cfea0 0x7fbe380b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe35e00640 1 -- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe38160920 msgr2=0x7fbe380b85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe35e00640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe38160920 0x7fbe380b85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe35e00640 1 -- 172.21.15.26:0/36489009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe28009b30 con 0x7fbe380cf9f0 2024-01-15T17:55:54.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe35e00640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380cf9f0 0x7fbe38151270 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbe28036040 tx=0x7fbe28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:54.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe2e000640 1 -- 172.21.15.26:0/36489009 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2803e450 con 0x7fbe380cf9f0 2024-01-15T17:55:54.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe2e000640 1 -- 172.21.15.26:0/36489009 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2803e5f0 con 0x7fbe380cf9f0 2024-01-15T17:55:54.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe2e000640 1 -- 172.21.15.26:0/36489009 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2803e9a0 con 0x7fbe380cf9f0 2024-01-15T17:55:54.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.202+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe380b60d0 con 0x7fbe380cf9f0 2024-01-15T17:55:54.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.203+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe380b6690 con 0x7fbe380cf9f0 2024-01-15T17:55:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.204+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe38067450 con 0x7fbe380cf9f0 2024-01-15T17:55:54.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.204+0000 7fbe2e000640 1 -- 172.21.15.26:0/36489009 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbe2802cd50 con 0x7fbe380cf9f0 2024-01-15T17:55:54.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.206+0000 7fbe2e000640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbe1006b2c0 0x7fbe1006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.206+0000 7fbe2e000640 1 -- 172.21.15.26:0/36489009 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbe280ac900 con 0x7fbe380cf9f0 2024-01-15T17:55:54.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.207+0000 7fbe35400640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbe1006b2c0 0x7fbe1006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.208+0000 7fbe35400640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbe1006b2c0 0x7fbe1006d780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbe20006fd0 tx=0x7fbe20006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:54.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.211+0000 7fbe2e000640 1 -- 172.21.15.26:0/36489009 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbe28077640 con 0x7fbe380cf9f0 2024-01-15T17:55:54.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.420+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbe3808c7e0 con 0x7fbe380cf9f0 2024-01-15T17:55:54.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.421+0000 7fbe2e000640 1 -- 172.21.15.26:0/36489009 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbe2807ac60 con 0x7fbe380cf9f0 2024-01-15T17:55:54.424 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T17:55:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbe1006b2c0 msgr2=0x7fbe1006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbe1006b2c0 0x7fbe1006d780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbe20006fd0 tx=0x7fbe20006bc0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380cf9f0 msgr2=0x7fbe38151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380cf9f0 0x7fbe38151270 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbe28036040 tx=0x7fbe28004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 shutdown_connections 2024-01-15T17:55:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbe380cfea0 0x7fbe380b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbe1006b2c0 0x7fbe1006d780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbe38160920 0x7fbe380b85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 --2- 172.21.15.26:0/36489009 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbe380cf9f0 0x7fbe38151270 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 >> 172.21.15.26:0/36489009 conn(0x7fbe380d3090 msgr2=0x7fbe3806c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 shutdown_connections 2024-01-15T17:55:54.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.424+0000 7fbe37200640 1 -- 172.21.15.26:0/36489009 wait complete. 2024-01-15T17:55:54.445 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-01-15T17:55:54.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:54.445 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T17:55:54.548 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:55:54.548 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:55:54.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:54.548 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T17:55:54.616 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:55:54.617 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:55:54.675 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:55:54.675 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:55:54.732 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.958+0000 7ff6e6800640 1 Processor -- start 2024-01-15T17:55:54.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.958+0000 7ff6e6800640 1 -- start start 2024-01-15T17:55:54.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.960+0000 7ff6e6800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff6e0071410 0x7ff6e0146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.960+0000 7ff6e6800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e00b2e10 con 0x7ff6e006c7c0 2024-01-15T17:55:54.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.960+0000 7ff6e6800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e006cc50 con 0x7ff6e0071f00 2024-01-15T17:55:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.960+0000 7ff6e6800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e006cec0 con 0x7ff6e0071410 2024-01-15T17:55:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.960+0000 7ff6e4a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff6e006c7c0 0x7ff6e00717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55262/0 (socket says 172.21.15.26:55262) 2024-01-15T17:55:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.960+0000 7ff6e4a00640 1 -- 172.21.15.26:0/738331385 learned_addr learned my addr 172.21.15.26:0/738331385 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:54.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6e5400640 1 --2- 172.21.15.26:0/738331385 >> v2:172.21.15.124:3300/0 conn(0x7ff6e0071410 0x7ff6e0146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793165478 0 0) 0x7ff6e00b2e10 con 0x7ff6e006c7c0 2024-01-15T17:55:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6c0003520 con 0x7ff6e006c7c0 2024-01-15T17:55:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1858716466 0 0) 0x7ff6c0003520 con 0x7ff6e006c7c0 2024-01-15T17:55:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 >> v2:172.21.15.124:3300/0 conn(0x7ff6e0071410 msgr2=0x7ff6e0146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 --2- 172.21.15.26:0/738331385 >> v2:172.21.15.124:3300/0 conn(0x7ff6e0071410 0x7ff6e0146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 >> v1:172.21.15.124:6789/0 conn(0x7ff6e0071f00 legacy=0x7ff6e0166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e006d1a0 con 0x7ff6e006c7c0 2024-01-15T17:55:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7ff6c8001670 con 0x7ff6e006c7c0 2024-01-15T17:55:54.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 >> v1:172.21.15.26:6789/0 conn(0x7ff6e006c7c0 legacy=0x7ff6e00717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6c00047c0 0x7ff6c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6c0005180 0x7ff6c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e006cec0 con 0x7ff6c00039c0 2024-01-15T17:55:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c000c080 con 0x7ff6c0005180 2024-01-15T17:55:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.961+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c000c240 con 0x7ff6c00047c0 2024-01-15T17:55:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.962+0000 7ff6e5400640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.962+0000 7ff6e5e00640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6c0005180 0x7ff6c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.962+0000 7ff6e5400640 1 -- 172.21.15.26:0/738331385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6c00047c0 msgr2=0x7ff6c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.962+0000 7ff6e5400640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6c00047c0 0x7ff6c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.962+0000 7ff6e5400640 1 -- 172.21.15.26:0/738331385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6c0005180 msgr2=0x7ff6c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.962+0000 7ff6e5400640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6c0005180 0x7ff6c000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.962+0000 7ff6e5400640 1 -- 172.21.15.26:0/738331385 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0003520 con 0x7ff6c00039c0 2024-01-15T17:55:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.962+0000 7ff6e5e00640 1 -- 172.21.15.26:0/738331385 reap_dead start 2024-01-15T17:55:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.963+0000 7ff6e5400640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6c0003de0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009fc0 tx=0x7ff6c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c37e9c7c4d0ad1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.964+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d002cc20 con 0x7ff6c00039c0 2024-01-15T17:55:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.964+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d000f040 con 0x7ff6c00039c0 2024-01-15T17:55:54.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.964+0000 7ff6dfe00640 1 -- 172.21.15.26:0/738331385 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0034440 con 0x7ff6c00039c0 2024-01-15T17:55:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.964+0000 7ff6e6800640 1 -- 172.21.15.26:0/738331385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 msgr2=0x7ff6c0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.964+0000 7ff6e6800640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6c0003de0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff6d0009fc0 tx=0x7ff6c00057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.964+0000 7ff6e6800640 1 -- 172.21.15.26:0/738331385 shutdown_connections 2024-01-15T17:55:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.964+0000 7ff6e6800640 1 --2- 172.21.15.26:0/738331385 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6c0003de0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.964+0000 7ff6e6800640 1 -- 172.21.15.26:0/738331385 >> 172.21.15.26:0/738331385 conn(0x7ff6e006a800 msgr2=0x7ff6e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.965+0000 7ff6e6800640 1 -- 172.21.15.26:0/738331385 shutdown_connections 2024-01-15T17:55:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.965+0000 7ff6e6800640 1 -- 172.21.15.26:0/738331385 wait complete. 2024-01-15T17:55:54.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.966+0000 7ff6e6800640 1 Processor -- start 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.967+0000 7ff6e6800640 1 -- start start 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6e015cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e6800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e01f6d80 0x7ff6e00c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e6800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e00c5790 0x7ff6e00cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e6800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e01f7200 con 0x7ff6c00039c0 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e6800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e00c8800 con 0x7ff6e00c5790 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e6800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e00c8ae0 con 0x7ff6e01f6d80 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6e015cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6e015cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48258/0 (socket says 172.21.15.26:48258) 2024-01-15T17:55:54.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e5400640 1 -- 172.21.15.26:0/1059948334 learned_addr learned my addr 172.21.15.26:0/1059948334 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e5400640 1 -- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e01f6d80 msgr2=0x7ff6e00c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e5400640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e01f6d80 0x7ff6e00c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e5e00640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e00c5790 0x7ff6e00cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e5400640 1 -- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e00c5790 msgr2=0x7ff6e00cd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e5400640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e00c5790 0x7ff6e00cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.968+0000 7ff6e5400640 1 -- 172.21.15.26:0/1059948334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e01f62a0 con 0x7ff6c00039c0 2024-01-15T17:55:54.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.969+0000 7ff6e5400640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6e015cce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000e230 tx=0x7ff6d0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:54.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.969+0000 7ff6dd600640 1 -- 172.21.15.26:0/1059948334 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0034c50 con 0x7ff6c00039c0 2024-01-15T17:55:54.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.969+0000 7ff6dd600640 1 -- 172.21.15.26:0/1059948334 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d0034db0 con 0x7ff6c00039c0 2024-01-15T17:55:54.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.969+0000 7ff6dd600640 1 -- 172.21.15.26:0/1059948334 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d0003bb0 con 0x7ff6c00039c0 2024-01-15T17:55:54.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.969+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e01f6580 con 0x7ff6c00039c0 2024-01-15T17:55:54.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.969+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e00c5c10 con 0x7ff6c00039c0 2024-01-15T17:55:54.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.971+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e0067450 con 0x7ff6c00039c0 2024-01-15T17:55:54.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.971+0000 7ff6dd600640 1 -- 172.21.15.26:0/1059948334 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ff6d0040450 con 0x7ff6c00039c0 2024-01-15T17:55:54.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.972+0000 7ff6dd600640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff6b806b2c0 0x7ff6b806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:54.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.973+0000 7ff6dd600640 1 -- 172.21.15.26:0/1059948334 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff6d00ac490 con 0x7ff6c00039c0 2024-01-15T17:55:54.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.973+0000 7ff6e4a00640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff6b806b2c0 0x7ff6b806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:54.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.974+0000 7ff6e4a00640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff6b806b2c0 0x7ff6b806d780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00cadb0 tx=0x7ff6c8001540 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:54.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:54.977+0000 7ff6dd600640 1 -- 172.21.15.26:0/1059948334 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6d0077260 con 0x7ff6c00039c0 2024-01-15T17:55:55.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.173+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6e0070b70 con 0x7ff6c00039c0 2024-01-15T17:55:55.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.176+0000 7ff6dd600640 1 -- 172.21.15.26:0/1059948334 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7ff6d007a880 con 0x7ff6c00039c0 2024-01-15T17:55:55.180 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:55.180 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff6b806b2c0 msgr2=0x7ff6b806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff6b806b2c0 0x7ff6b806d780 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00cadb0 tx=0x7ff6c8001540 comp rx=0 tx=0).stop 2024-01-15T17:55:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 msgr2=0x7ff6e015cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6e015cce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000e230 tx=0x7ff6d0009750 comp rx=0 tx=0).stop 2024-01-15T17:55:55.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 shutdown_connections 2024-01-15T17:55:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e01f6d80 0x7ff6e00c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff6b806b2c0 0x7ff6b806d780 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e00c5790 0x7ff6e00cd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 --2- 172.21.15.26:0/1059948334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6c00039c0 0x7ff6e015cce0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 >> 172.21.15.26:0/1059948334 conn(0x7ff6e006a800 msgr2=0x7ff6e00717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.179+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 shutdown_connections 2024-01-15T17:55:55.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.180+0000 7ff6e6800640 1 -- 172.21.15.26:0/1059948334 wait complete. 2024-01-15T17:55:55.202 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:55.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:55.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.417+0000 7fce70200640 1 Processor -- start 2024-01-15T17:55:55.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.417+0000 7fce70200640 1 -- start start 2024-01-15T17:55:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.419+0000 7fce70200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce6813df80 0x7fce6815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.419+0000 7fce70200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fce6813e360 0x7fce680cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.419+0000 7fce70200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce6815f9c0 0x7fce680cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.419+0000 7fce70200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6806c470 con 0x7fce6813df80 2024-01-15T17:55:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.419+0000 7fce70200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6806c750 con 0x7fce6813e360 2024-01-15T17:55:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.419+0000 7fce70200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6806ca30 con 0x7fce6815f9c0 2024-01-15T17:55:55.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.419+0000 7fce6d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce6813df80 0x7fce6815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.419+0000 7fce6d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fce6813df80 0x7fce6815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48268/0 (socket says 172.21.15.26:48268) 2024-01-15T17:55:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.419+0000 7fce6d800640 1 -- 172.21.15.26:0/2550133451 learned_addr learned my addr 172.21.15.26:0/2550133451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.420+0000 7fce6ce00640 1 --2- 172.21.15.26:0/2550133451 >> v2:172.21.15.124:3300/0 conn(0x7fce6813e360 0x7fce680cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.420+0000 7fce6d800640 1 -- 172.21.15.26:0/2550133451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce6815f9c0 msgr2=0x7fce680cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.420+0000 7fce6e200640 1 --2- 172.21.15.26:0/2550133451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce6815f9c0 0x7fce680cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.420+0000 7fce6d800640 1 --2- 172.21.15.26:0/2550133451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce6815f9c0 0x7fce680cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.420+0000 7fce6d800640 1 -- 172.21.15.26:0/2550133451 >> v2:172.21.15.124:3300/0 conn(0x7fce6813e360 msgr2=0x7fce680cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.420+0000 7fce6d800640 1 --2- 172.21.15.26:0/2550133451 >> v2:172.21.15.124:3300/0 conn(0x7fce6813e360 0x7fce680cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.420+0000 7fce6d800640 1 -- 172.21.15.26:0/2550133451 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6806cd10 con 0x7fce6813df80 2024-01-15T17:55:55.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.420+0000 7fce6ce00640 1 --2- 172.21.15.26:0/2550133451 >> v2:172.21.15.124:3300/0 conn(0x7fce6813e360 0x7fce680cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:55.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.421+0000 7fce6d800640 1 --2- 172.21.15.26:0/2550133451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6813df80 0x7fce6815bee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fce6806ded0 tx=0x7fce5802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e654e29205bf45 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:55.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.421+0000 7fce67e00640 1 -- 172.21.15.26:0/2550133451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58031860 con 0x7fce6813df80 2024-01-15T17:55:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.421+0000 7fce67e00640 1 -- 172.21.15.26:0/2550133451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce580319c0 con 0x7fce6813df80 2024-01-15T17:55:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.421+0000 7fce67e00640 1 -- 172.21.15.26:0/2550133451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58031d50 con 0x7fce6813df80 2024-01-15T17:55:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.422+0000 7fce70200640 1 -- 172.21.15.26:0/2550133451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6813df80 msgr2=0x7fce6815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.422+0000 7fce70200640 1 --2- 172.21.15.26:0/2550133451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6813df80 0x7fce6815bee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fce6806ded0 tx=0x7fce5802b900 comp rx=0 tx=0).stop 2024-01-15T17:55:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.422+0000 7fce70200640 1 -- 172.21.15.26:0/2550133451 shutdown_connections 2024-01-15T17:55:55.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.422+0000 7fce70200640 1 --2- 172.21.15.26:0/2550133451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce6815f9c0 0x7fce680cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.422+0000 7fce70200640 1 --2- 172.21.15.26:0/2550133451 >> v2:172.21.15.124:3300/0 conn(0x7fce6813e360 0x7fce680cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.422+0000 7fce70200640 1 --2- 172.21.15.26:0/2550133451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6813df80 0x7fce6815bee0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.422+0000 7fce70200640 1 -- 172.21.15.26:0/2550133451 >> 172.21.15.26:0/2550133451 conn(0x7fce6806a800 msgr2=0x7fce6806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.422+0000 7fce70200640 1 -- 172.21.15.26:0/2550133451 shutdown_connections 2024-01-15T17:55:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.422+0000 7fce70200640 1 -- 172.21.15.26:0/2550133451 wait complete. 2024-01-15T17:55:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.424+0000 7fce70200640 1 Processor -- start 2024-01-15T17:55:55.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.424+0000 7fce70200640 1 -- start start 2024-01-15T17:55:55.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.425+0000 7fce70200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce6813df80 0x7fce680cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.425+0000 7fce70200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce6813e360 0x7fce680ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.425+0000 7fce70200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6815f9c0 0x7fce680ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.425+0000 7fce70200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6813d970 con 0x7fce6815f9c0 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.425+0000 7fce70200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce681293a0 con 0x7fce6813df80 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.425+0000 7fce70200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce68129680 con 0x7fce6813e360 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.426+0000 7fce6e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6815f9c0 0x7fce680ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.426+0000 7fce6e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6815f9c0 0x7fce680ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48272/0 (socket says 172.21.15.26:48272) 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.426+0000 7fce6e200640 1 -- 172.21.15.26:0/564552158 learned_addr learned my addr 172.21.15.26:0/564552158 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.426+0000 7fce6d800640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce6813df80 0x7fce680cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.426+0000 7fce6e200640 1 -- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce6813e360 msgr2=0x7fce680ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.426+0000 7fce6e200640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce6813e360 0x7fce680ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.426+0000 7fce6e200640 1 -- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce6813df80 msgr2=0x7fce680cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.426+0000 7fce6e200640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce6813df80 0x7fce680cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.426+0000 7fce6e200640 1 -- 172.21.15.26:0/564552158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580094d0 con 0x7fce6815f9c0 2024-01-15T17:55:55.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.427+0000 7fce6e200640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6815f9c0 0x7fce680ad510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fce6000cd60 tx=0x7fce6000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:55.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.427+0000 7fce65600640 1 -- 172.21.15.26:0/564552158 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60003b90 con 0x7fce6815f9c0 2024-01-15T17:55:55.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.427+0000 7fce65600640 1 -- 172.21.15.26:0/564552158 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce60003d10 con 0x7fce6815f9c0 2024-01-15T17:55:55.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.427+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce68129960 con 0x7fce6815f9c0 2024-01-15T17:55:55.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.427+0000 7fce65600640 1 -- 172.21.15.26:0/564552158 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60004260 con 0x7fce6815f9c0 2024-01-15T17:55:55.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.427+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce680ada50 con 0x7fce6815f9c0 2024-01-15T17:55:55.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.428+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce680675d0 con 0x7fce6815f9c0 2024-01-15T17:55:55.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.429+0000 7fce65600640 1 -- 172.21.15.26:0/564552158 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fce600043c0 con 0x7fce6815f9c0 2024-01-15T17:55:55.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.430+0000 7fce65600640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fce4006b2c0 0x7fce4006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.430+0000 7fce65600640 1 -- 172.21.15.26:0/564552158 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fce60012070 con 0x7fce6815f9c0 2024-01-15T17:55:55.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.431+0000 7fce6d800640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fce4006b2c0 0x7fce4006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.432+0000 7fce6d800640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fce4006b2c0 0x7fce4006d780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce58036000 tx=0x7fce5803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:55.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.434+0000 7fce65600640 1 -- 172.21.15.26:0/564552158 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce60055330 con 0x7fce6815f9c0 2024-01-15T17:55:55.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.629+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce6806df00 con 0x7fce6815f9c0 2024-01-15T17:55:55.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.629+0000 7fce65600640 1 -- 172.21.15.26:0/564552158 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce60058950 con 0x7fce6815f9c0 2024-01-15T17:55:55.633 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:55.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fce4006b2c0 msgr2=0x7fce4006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fce4006b2c0 0x7fce4006d780 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce58036000 tx=0x7fce5803c040 comp rx=0 tx=0).stop 2024-01-15T17:55:55.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6815f9c0 msgr2=0x7fce680ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6815f9c0 0x7fce680ad510 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fce6000cd60 tx=0x7fce6000a660 comp rx=0 tx=0).stop 2024-01-15T17:55:55.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 shutdown_connections 2024-01-15T17:55:55.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce6813e360 0x7fce680ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fce4006b2c0 0x7fce4006d780 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce6815f9c0 0x7fce680ad510 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 --2- 172.21.15.26:0/564552158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce6813df80 0x7fce680cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 >> 172.21.15.26:0/564552158 conn(0x7fce6806a800 msgr2=0x7fce6806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:55.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 shutdown_connections 2024-01-15T17:55:55.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.632+0000 7fce70200640 1 -- 172.21.15.26:0/564552158 wait complete. 2024-01-15T17:55:55.654 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:55.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:55:55.832 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:55:55.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.835+0000 7fdd66c00640 1 Processor -- start 2024-01-15T17:55:55.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.835+0000 7fdd66c00640 1 -- start start 2024-01-15T17:55:55.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.836+0000 7fdd66c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdd60143810 0x7fdd60071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.836+0000 7fdd66c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd600ae170 0x7fdd6015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.836+0000 7fdd66c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6006c730 con 0x7fdd6015d770 2024-01-15T17:55:55.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.836+0000 7fdd66c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd60143bf0 con 0x7fdd60143810 2024-01-15T17:55:55.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.836+0000 7fdd66c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600ae5f0 con 0x7fdd600ae170 2024-01-15T17:55:55.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.836+0000 7fdd66200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdd6015d770 0x7fdd6015dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55264/0 (socket says 172.21.15.26:55264) 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.836+0000 7fdd66200640 1 -- 172.21.15.26:0/183752241 learned_addr learned my addr 172.21.15.26:0/183752241 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd65800640 1 --2- 172.21.15.26:0/183752241 >> v2:172.21.15.124:3300/0 conn(0x7fdd60143810 0x7fdd60071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd64e00640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd600ae170 0x7fdd6015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962345750 0 0) 0x7fdd6006c730 con 0x7fdd6015d770 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd40003520 con 0x7fdd6015d770 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 740337336 0 0) 0x7fdd40003520 con 0x7fdd6015d770 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd600ae170 msgr2=0x7fdd6015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd600ae170 0x7fdd6015e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 >> v2:172.21.15.124:3300/0 conn(0x7fdd60143810 msgr2=0x7fdd60071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 --2- 172.21.15.26:0/183752241 >> v2:172.21.15.124:3300/0 conn(0x7fdd60143810 0x7fdd60071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600724d0 con 0x7fdd6015d770 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fdd580019f0 con 0x7fdd6015d770 2024-01-15T17:55:55.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 >> v1:172.21.15.26:6789/0 conn(0x7fdd6015d770 legacy=0x7fdd6015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.837+0000 7fdd5fe00640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd40003970 0x7fdd40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd5fe00640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd400047c0 0x7fdd40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd5fe00640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd600ae170 0x7fdd40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60143bf0 con 0x7fdd40003970 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600ae5f0 con 0x7fdd400047c0 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4000b980 con 0x7fdd600ae170 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd65800640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd40003970 0x7fdd40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd65800640 1 -- 172.21.15.26:0/183752241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd600ae170 msgr2=0x7fdd40005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd65800640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd600ae170 0x7fdd40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd65800640 1 -- 172.21.15.26:0/183752241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd400047c0 msgr2=0x7fdd40004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd65800640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd400047c0 0x7fdd40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.838+0000 7fdd65800640 1 -- 172.21.15.26:0/183752241 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd40003520 con 0x7fdd40003970 2024-01-15T17:55:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.839+0000 7fdd65800640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd40003970 0x7fdd40003d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009af0 tx=0x7fdd400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122820bcc85a81e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5002c710 con 0x7fdd40003970 2024-01-15T17:55:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5002c870 con 0x7fdd40003970 2024-01-15T17:55:55.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd5fe00640 1 -- 172.21.15.26:0/183752241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5002cc00 con 0x7fdd40003970 2024-01-15T17:55:55.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd66c00640 1 -- 172.21.15.26:0/183752241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd40003970 msgr2=0x7fdd40003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd66c00640 1 --2- 172.21.15.26:0/183752241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd40003970 0x7fdd40003d90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009af0 tx=0x7fdd400057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd66200640 1 -- 172.21.15.26:0/183752241 reap_dead start 2024-01-15T17:55:55.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd66c00640 1 -- 172.21.15.26:0/183752241 shutdown_connections 2024-01-15T17:55:55.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd66c00640 1 -- 172.21.15.26:0/183752241 >> 172.21.15.26:0/183752241 conn(0x7fdd6006a7d0 msgr2=0x7fdd6006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:55.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd66c00640 1 -- 172.21.15.26:0/183752241 shutdown_connections 2024-01-15T17:55:55.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.840+0000 7fdd66c00640 1 -- 172.21.15.26:0/183752241 wait complete. 2024-01-15T17:55:55.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.842+0000 7fdd66c00640 1 Processor -- start 2024-01-15T17:55:55.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.842+0000 7fdd66c00640 1 -- start start 2024-01-15T17:55:55.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.843+0000 7fdd66c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd40003970 0x7fdd601f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.843+0000 7fdd66c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd60143d30 0x7fdd60144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.843+0000 7fdd66c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd6013a090 0x7fdd6013a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.843+0000 7fdd66c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6013ae30 con 0x7fdd6013a090 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.843+0000 7fdd66c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601380c0 con 0x7fdd40003970 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.843+0000 7fdd66c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601446d0 con 0x7fdd60143d30 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd66200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd6013a090 0x7fdd6013a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd66200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd6013a090 0x7fdd6013a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48298/0 (socket says 172.21.15.26:48298) 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd66200640 1 -- 172.21.15.26:0/1570309763 learned_addr learned my addr 172.21.15.26:0/1570309763 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd65800640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd40003970 0x7fdd601f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd64e00640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd60143d30 0x7fdd60144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd66200640 1 -- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd60143d30 msgr2=0x7fdd60144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd66200640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd60143d30 0x7fdd60144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd66200640 1 -- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd40003970 msgr2=0x7fdd601f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:55.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd66200640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd40003970 0x7fdd601f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd66200640 1 -- 172.21.15.26:0/1570309763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600b0260 con 0x7fdd6013a090 2024-01-15T17:55:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd65800640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd40003970 0x7fdd601f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:55:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.844+0000 7fdd66200640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd6013a090 0x7fdd6013a510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd6015db50 tx=0x7fdd5800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.845+0000 7fdd5d600640 1 -- 172.21.15.26:0/1570309763 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800a670 con 0x7fdd6013a090 2024-01-15T17:55:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.845+0000 7fdd5d600640 1 -- 172.21.15.26:0/1570309763 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5800a7d0 con 0x7fdd6013a090 2024-01-15T17:55:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.845+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd601f2950 con 0x7fdd6013a090 2024-01-15T17:55:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.845+0000 7fdd5d600640 1 -- 172.21.15.26:0/1570309763 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800ab80 con 0x7fdd6013a090 2024-01-15T17:55:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.845+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd600afc10 con 0x7fdd6013a090 2024-01-15T17:55:55.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.846+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd600b03f0 con 0x7fdd6013a090 2024-01-15T17:55:55.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.848+0000 7fdd5d600640 1 -- 172.21.15.26:0/1570309763 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdd5800ace0 con 0x7fdd6013a090 2024-01-15T17:55:55.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.849+0000 7fdd5d600640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdd3806b2f0 0x7fdd3806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:55.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.849+0000 7fdd5d600640 1 -- 172.21.15.26:0/1570309763 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdd58089430 con 0x7fdd6013a090 2024-01-15T17:55:55.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.850+0000 7fdd65800640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdd3806b2f0 0x7fdd3806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:55.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.851+0000 7fdd65800640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdd3806b2f0 0x7fdd3806d7b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd50036000 tx=0x7fdd50030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:55.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:55.857+0000 7fdd5d600640 1 -- 172.21.15.26:0/1570309763 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd58054160 con 0x7fdd6013a090 2024-01-15T17:55:56.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.052+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd60067450 con 0x7fdd6013a090 2024-01-15T17:55:56.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.053+0000 7fdd5d600640 1 -- 172.21.15.26:0/1570309763 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd58057780 con 0x7fdd6013a090 2024-01-15T17:55:56.056 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:56.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdd3806b2f0 msgr2=0x7fdd3806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdd3806b2f0 0x7fdd3806d7b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd50036000 tx=0x7fdd50030000 comp rx=0 tx=0).stop 2024-01-15T17:55:56.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd6013a090 msgr2=0x7fdd6013a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd6013a090 0x7fdd6013a510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd6015db50 tx=0x7fdd5800cf00 comp rx=0 tx=0).stop 2024-01-15T17:55:56.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 shutdown_connections 2024-01-15T17:55:56.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdd60143d30 0x7fdd60144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdd3806b2f0 0x7fdd3806d7b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdd6013a090 0x7fdd6013a510 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 --2- 172.21.15.26:0/1570309763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdd40003970 0x7fdd601f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 >> 172.21.15.26:0/1570309763 conn(0x7fdd6006a7d0 msgr2=0x7fdd600b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:56.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 shutdown_connections 2024-01-15T17:55:56.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.056+0000 7fdd66c00640 1 -- 172.21.15.26:0/1570309763 wait complete. 2024-01-15T17:55:56.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:56.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.282+0000 7f9725600640 1 Processor -- start 2024-01-15T17:55:56.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.282+0000 7f9725600640 1 -- start start 2024-01-15T17:55:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f9725600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9720143ae0 0x7f97200cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f9725600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97200cf7d0 0x7f972015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f9725600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f972015e1e0 0x7f972015a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f9725600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97200cfc30 con 0x7f972015e1e0 2024-01-15T17:55:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f9725600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97200b1ee0 con 0x7f9720143ae0 2024-01-15T17:55:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f9725600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97200b2780 con 0x7f97200cf7d0 2024-01-15T17:55:56.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f971f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f972015e1e0 0x7f972015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f971f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f972015e1e0 0x7f972015a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48314/0 (socket says 172.21.15.26:48314) 2024-01-15T17:55:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f971f400640 1 -- 172.21.15.26:0/1195273492 learned_addr learned my addr 172.21.15.26:0/1195273492 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:56.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f971e000640 1 --2- 172.21.15.26:0/1195273492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97200cf7d0 0x7f972015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.284+0000 7f971ea00640 1 --2- 172.21.15.26:0/1195273492 >> v2:172.21.15.124:3300/0 conn(0x7f9720143ae0 0x7f97200cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.285+0000 7f971f400640 1 -- 172.21.15.26:0/1195273492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97200cf7d0 msgr2=0x7f972015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:56.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.285+0000 7f971f400640 1 --2- 172.21.15.26:0/1195273492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97200cf7d0 0x7f972015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.285+0000 7f971f400640 1 -- 172.21.15.26:0/1195273492 >> v2:172.21.15.124:3300/0 conn(0x7f9720143ae0 msgr2=0x7f97200cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:56.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.285+0000 7f971f400640 1 --2- 172.21.15.26:0/1195273492 >> v2:172.21.15.124:3300/0 conn(0x7f9720143ae0 0x7f97200cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.285+0000 7f971f400640 1 -- 172.21.15.26:0/1195273492 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972015e990 con 0x7f972015e1e0 2024-01-15T17:55:56.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.286+0000 7f971f400640 1 --2- 172.21.15.26:0/1195273492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 0x7f972015a670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97200cfea0 tx=0x7f971802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe21f0f4d434185 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:56.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.286+0000 7f971d600640 1 -- 172.21.15.26:0/1195273492 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971802ca70 con 0x7f972015e1e0 2024-01-15T17:55:56.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.286+0000 7f971d600640 1 -- 172.21.15.26:0/1195273492 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971802cbd0 con 0x7f972015e1e0 2024-01-15T17:55:56.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.286+0000 7f971d600640 1 -- 172.21.15.26:0/1195273492 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180343f0 con 0x7f972015e1e0 2024-01-15T17:55:56.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.287+0000 7f9725600640 1 -- 172.21.15.26:0/1195273492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 msgr2=0x7f972015a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.287+0000 7f9725600640 1 --2- 172.21.15.26:0/1195273492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 0x7f972015a670 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f97200cfea0 tx=0x7f971802bed0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.287+0000 7f9725600640 1 -- 172.21.15.26:0/1195273492 shutdown_connections 2024-01-15T17:55:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.287+0000 7f9725600640 1 --2- 172.21.15.26:0/1195273492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 0x7f972015a670 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.287+0000 7f9725600640 1 --2- 172.21.15.26:0/1195273492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97200cf7d0 0x7f972015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.287+0000 7f9725600640 1 --2- 172.21.15.26:0/1195273492 >> v2:172.21.15.124:3300/0 conn(0x7f9720143ae0 0x7f97200cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.287+0000 7f9725600640 1 -- 172.21.15.26:0/1195273492 >> 172.21.15.26:0/1195273492 conn(0x7f972006a800 msgr2=0x7f972006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.287+0000 7f9725600640 1 -- 172.21.15.26:0/1195273492 shutdown_connections 2024-01-15T17:55:56.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.287+0000 7f9725600640 1 -- 172.21.15.26:0/1195273492 wait complete. 2024-01-15T17:55:56.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.289+0000 7f9725600640 1 Processor -- start 2024-01-15T17:55:56.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.289+0000 7f9725600640 1 -- start start 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.290+0000 7f9725600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97200cf7d0 0x7f97201f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.290+0000 7f9725600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9720143ae0 0x7f97200adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.290+0000 7f9725600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 0x7f97201f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.290+0000 7f9725600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97200cc740 con 0x7f972015e1e0 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.290+0000 7f9725600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97201f3f90 con 0x7f97200cf7d0 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.290+0000 7f9725600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97201f2ec0 con 0x7f9720143ae0 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.290+0000 7f971f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 0x7f97201f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 0x7f97201f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48324/0 (socket says 172.21.15.26:48324) 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971f400640 1 -- 172.21.15.26:0/3746511173 learned_addr learned my addr 172.21.15.26:0/3746511173 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971ea00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97200cf7d0 0x7f97201f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971e000640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9720143ae0 0x7f97200adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971f400640 1 -- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9720143ae0 msgr2=0x7f97200adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971f400640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9720143ae0 0x7f97200adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971f400640 1 -- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97200cf7d0 msgr2=0x7f97201f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971f400640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97200cf7d0 0x7f97201f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971f400640 1 -- 172.21.15.26:0/3746511173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9718009b30 con 0x7f972015e1e0 2024-01-15T17:55:56.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.291+0000 7f971f400640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 0x7f97201f3a50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9718036040 tx=0x7f9718004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:56.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.292+0000 7f9716a00640 1 -- 172.21.15.26:0/3746511173 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718040670 con 0x7f972015e1e0 2024-01-15T17:55:56.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.292+0000 7f9716a00640 1 -- 172.21.15.26:0/3746511173 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97180407f0 con 0x7f972015e1e0 2024-01-15T17:55:56.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.292+0000 7f9716a00640 1 -- 172.21.15.26:0/3746511173 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718040ba0 con 0x7f972015e1e0 2024-01-15T17:55:56.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.292+0000 7f9725600640 1 -- 172.21.15.26:0/3746511173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97201f31a0 con 0x7f972015e1e0 2024-01-15T17:55:56.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.292+0000 7f9725600640 1 -- 172.21.15.26:0/3746511173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97201f3760 con 0x7f972015e1e0 2024-01-15T17:55:56.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.293+0000 7f970fe00640 1 -- 172.21.15.26:0/3746511173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97200cc8f0 con 0x7f972015e1e0 2024-01-15T17:55:56.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.294+0000 7f9716a00640 1 -- 172.21.15.26:0/3746511173 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9718040d00 con 0x7f972015e1e0 2024-01-15T17:55:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.295+0000 7f9716a00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96f806b2c0 0x7f96f806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.295+0000 7f9716a00640 1 -- 172.21.15.26:0/3746511173 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f97180acde0 con 0x7f972015e1e0 2024-01-15T17:55:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.296+0000 7f971ea00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96f806b2c0 0x7f96f806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.297+0000 7f971ea00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96f806b2c0 0x7f96f806d780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9710006fd0 tx=0x7f9710008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.299+0000 7f9716a00640 1 -- 172.21.15.26:0/3746511173 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9718077c40 con 0x7f972015e1e0 2024-01-15T17:55:56.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.483+0000 7f970fe00640 1 -- 172.21.15.26:0/3746511173 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97200675d0 con 0x7f972015e1e0 2024-01-15T17:55:56.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.483+0000 7f9716a00640 1 -- 172.21.15.26:0/3746511173 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f971807b260 con 0x7f972015e1e0 2024-01-15T17:55:56.487 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 -- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96f806b2c0 msgr2=0x7f96f806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96f806b2c0 0x7f96f806d780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9710006fd0 tx=0x7f9710008040 comp rx=0 tx=0).stop 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 -- 172.21.15.26:0/3746511173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 msgr2=0x7f97201f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 0x7f97201f3a50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9718036040 tx=0x7f9718004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 -- 172.21.15.26:0/3746511173 shutdown_connections 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9720143ae0 0x7f97200adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96f806b2c0 0x7f96f806d780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f972015e1e0 0x7f97201f3a50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 --2- 172.21.15.26:0/3746511173 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97200cf7d0 0x7f97201f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 -- 172.21.15.26:0/3746511173 >> 172.21.15.26:0/3746511173 conn(0x7f972006a800 msgr2=0x7f97200b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 -- 172.21.15.26:0/3746511173 shutdown_connections 2024-01-15T17:55:56.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.486+0000 7f970fe00640 1 -- 172.21.15.26:0/3746511173 wait complete. 2024-01-15T17:55:56.507 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:55:56.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.709+0000 7fa41b200640 1 Processor -- start 2024-01-15T17:55:56.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.710+0000 7fa41b200640 1 -- start start 2024-01-15T17:55:56.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.711+0000 7fa41b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa4140cf100 0x7fa4140d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.711+0000 7fa41b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140b23c0 con 0x7fa4140d0220 2024-01-15T17:55:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.711+0000 7fa41b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4140d0800 con 0x7fa41415d3c0 2024-01-15T17:55:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.711+0000 7fa41b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4140ce5d0 con 0x7fa4140cf100 2024-01-15T17:55:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.712+0000 7fa41a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa4140cf100 0x7fa4140d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.712+0000 7fa41a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa4140cf100 0x7fa4140d0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48328/0 (socket says 172.21.15.26:48328) 2024-01-15T17:55:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.712+0000 7fa41a800640 1 -- 172.21.15.26:0/318355671 learned_addr learned my addr 172.21.15.26:0/318355671 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.712+0000 7fa419400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa4140d0220 0x7fa4140ce9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55270/0 (socket says 172.21.15.26:55270) 2024-01-15T17:55:56.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.712+0000 7fa41a800640 1 -- 172.21.15.26:0/318355671 >> v1:172.21.15.124:6789/0 conn(0x7fa41415d3c0 legacy=0x7fa4140afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:56.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.712+0000 7fa41a800640 1 -- 172.21.15.26:0/318355671 >> v1:172.21.15.26:6789/0 conn(0x7fa4140d0220 legacy=0x7fa4140ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.712+0000 7fa41a800640 1 -- 172.21.15.26:0/318355671 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140cdf10 con 0x7fa4140cf100 2024-01-15T17:55:56.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.712+0000 7fa418a00640 1 -- 172.21.15.26:0/318355671 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478368518 0 0) 0x7fa4140b23c0 con 0x7fa4140d0220 2024-01-15T17:55:56.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.713+0000 7fa41a800640 1 --2- 172.21.15.26:0/318355671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4140cf100 0x7fa4140d0e90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009e50 tx=0x7fa40c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd33d823d89a4962 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:56.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.714+0000 7fa418a00640 1 -- 172.21.15.26:0/318355671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c02cb20 con 0x7fa4140cf100 2024-01-15T17:55:56.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.714+0000 7fa418a00640 1 -- 172.21.15.26:0/318355671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c02cc80 con 0x7fa4140cf100 2024-01-15T17:55:56.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.714+0000 7fa418a00640 1 -- 172.21.15.26:0/318355671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0343f0 con 0x7fa4140cf100 2024-01-15T17:55:56.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.714+0000 7fa41b200640 1 -- 172.21.15.26:0/318355671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4140cf100 msgr2=0x7fa4140d0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.714+0000 7fa41b200640 1 --2- 172.21.15.26:0/318355671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4140cf100 0x7fa4140d0e90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009e50 tx=0x7fa40c02bf80 comp rx=0 tx=0).stop 2024-01-15T17:55:56.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.714+0000 7fa41b200640 1 -- 172.21.15.26:0/318355671 shutdown_connections 2024-01-15T17:55:56.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.714+0000 7fa41b200640 1 --2- 172.21.15.26:0/318355671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4140cf100 0x7fa4140d0e90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.714+0000 7fa41b200640 1 -- 172.21.15.26:0/318355671 >> 172.21.15.26:0/318355671 conn(0x7fa4140d30b0 msgr2=0x7fa4140d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:56.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.714+0000 7fa41b200640 1 -- 172.21.15.26:0/318355671 shutdown_connections 2024-01-15T17:55:56.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.715+0000 7fa41b200640 1 -- 172.21.15.26:0/318355671 wait complete. 2024-01-15T17:55:56.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.716+0000 7fa41b200640 1 Processor -- start 2024-01-15T17:55:56.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.716+0000 7fa41b200640 1 -- start start 2024-01-15T17:55:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.717+0000 7fa41b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4140cf100 0x7fa4140ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.717+0000 7fa41b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4140d0220 0x7fa4140c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.717+0000 7fa41b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41415d3c0 0x7fa4140b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.717+0000 7fa41b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa414158090 con 0x7fa41415d3c0 2024-01-15T17:55:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.717+0000 7fa41b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140ca760 con 0x7fa4140d0220 2024-01-15T17:55:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.717+0000 7fa41b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4140b5af0 con 0x7fa4140cf100 2024-01-15T17:55:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.717+0000 7fa41a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41415d3c0 0x7fa4140b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.717+0000 7fa41a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41415d3c0 0x7fa4140b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48342/0 (socket says 172.21.15.26:48342) 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.717+0000 7fa41a800640 1 -- 172.21.15.26:0/3346043885 learned_addr learned my addr 172.21.15.26:0/3346043885 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa419400640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4140d0220 0x7fa4140c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa419e00640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4140cf100 0x7fa4140ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa41a800640 1 -- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4140cf100 msgr2=0x7fa4140ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa41a800640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4140cf100 0x7fa4140ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa41a800640 1 -- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4140d0220 msgr2=0x7fa4140c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa41a800640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4140d0220 0x7fa4140c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa41a800640 1 -- 172.21.15.26:0/3346043885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c009b30 con 0x7fa41415d3c0 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa419400640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4140d0220 0x7fa4140c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa419e00640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4140cf100 0x7fa4140ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:56.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.718+0000 7fa41a800640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41415d3c0 0x7fa4140b55b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009e50 tx=0x7fa40c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.719+0000 7fa412000640 1 -- 172.21.15.26:0/3346043885 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c009510 con 0x7fa41415d3c0 2024-01-15T17:55:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.719+0000 7fa412000640 1 -- 172.21.15.26:0/3346043885 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c007e50 con 0x7fa41415d3c0 2024-01-15T17:55:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.719+0000 7fa412000640 1 -- 172.21.15.26:0/3346043885 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c03e690 con 0x7fa41415d3c0 2024-01-15T17:55:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.719+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4140caba0 con 0x7fa41415d3c0 2024-01-15T17:55:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.719+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4140c7b90 con 0x7fa41415d3c0 2024-01-15T17:55:56.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.720+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4140cad90 con 0x7fa41415d3c0 2024-01-15T17:55:56.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.721+0000 7fa412000640 1 -- 172.21.15.26:0/3346043885 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa40c03e7f0 con 0x7fa41415d3c0 2024-01-15T17:55:56.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.722+0000 7fa412000640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3ec06b2c0 0x7fa3ec06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:56.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.722+0000 7fa412000640 1 -- 172.21.15.26:0/3346043885 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa40c0ac9c0 con 0x7fa41415d3c0 2024-01-15T17:55:56.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.722+0000 7fa419e00640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3ec06b2c0 0x7fa3ec06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:56.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.724+0000 7fa419e00640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3ec06b2c0 0x7fa3ec06d780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa404000f60 tx=0x7fa404009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:56.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.730+0000 7fa412000640 1 -- 172.21.15.26:0/3346043885 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa40c077700 con 0x7fa41415d3c0 2024-01-15T17:55:56.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.927+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa41415dc10 con 0x7fa41415d3c0 2024-01-15T17:55:56.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.928+0000 7fa412000640 1 -- 172.21.15.26:0/3346043885 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa40c043080 con 0x7fa41415d3c0 2024-01-15T17:55:56.931 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.930+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3ec06b2c0 msgr2=0x7fa3ec06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.930+0000 7fa41b200640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3ec06b2c0 0x7fa3ec06d780 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa404000f60 tx=0x7fa404009040 comp rx=0 tx=0).stop 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.930+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41415d3c0 msgr2=0x7fa4140b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.930+0000 7fa41b200640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41415d3c0 0x7fa4140b55b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009e50 tx=0x7fa40c004a40 comp rx=0 tx=0).stop 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.931+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 shutdown_connections 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.931+0000 7fa41b200640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3ec06b2c0 0x7fa3ec06d780 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.931+0000 7fa41b200640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41415d3c0 0x7fa4140b55b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.931+0000 7fa41b200640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4140d0220 0x7fa4140c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.931+0000 7fa41b200640 1 --2- 172.21.15.26:0/3346043885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4140cf100 0x7fa4140ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:56.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.931+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 >> 172.21.15.26:0/3346043885 conn(0x7fa4140d30b0 msgr2=0x7fa4140b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:56.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.931+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 shutdown_connections 2024-01-15T17:55:56.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:56.931+0000 7fa41b200640 1 -- 172.21.15.26:0/3346043885 wait complete. 2024-01-15T17:55:56.953 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:55:56.953 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:55:57.009 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.231+0000 7f6bd5400640 1 Processor -- start 2024-01-15T17:55:57.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.231+0000 7f6bd5400640 1 -- start start 2024-01-15T17:55:57.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.232+0000 7f6bd5400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6bd00ae5a0 0x7f6bd015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.232+0000 7f6bd5400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6bd00cf290 0x7f6bd00cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.232+0000 7f6bd5400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd0070c40 con 0x7f6bd00b3ae0 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.232+0000 7f6bd5400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd0071450 con 0x7f6bd00ae5a0 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.232+0000 7f6bd5400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd00cfc60 con 0x7f6bd00cf290 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.233+0000 7f6bce800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6bd00ae5a0 0x7f6bd015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.233+0000 7f6bce800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6bd00ae5a0 0x7f6bd015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48354/0 (socket says 172.21.15.26:48354) 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.233+0000 7f6bce800640 1 -- 172.21.15.26:0/388882242 learned_addr learned my addr 172.21.15.26:0/388882242 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.233+0000 7f6bcf200640 1 --2- 172.21.15.26:0/388882242 >> v2:172.21.15.124:3300/0 conn(0x7f6bd00cf290 0x7f6bd00cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.233+0000 7f6bce800640 1 -- 172.21.15.26:0/388882242 >> v2:172.21.15.124:3300/0 conn(0x7f6bd00cf290 msgr2=0x7f6bd00cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:57.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.233+0000 7f6bce800640 1 --2- 172.21.15.26:0/388882242 >> v2:172.21.15.124:3300/0 conn(0x7f6bd00cf290 0x7f6bd00cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.233+0000 7f6bce800640 1 -- 172.21.15.26:0/388882242 >> v1:172.21.15.124:6789/0 conn(0x7f6bd00b3ae0 legacy=0x7f6bd006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:57.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.233+0000 7f6bce800640 1 -- 172.21.15.26:0/388882242 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd006c470 con 0x7f6bd00ae5a0 2024-01-15T17:55:57.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.235+0000 7f6bce800640 1 --2- 172.21.15.26:0/388882242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00ae5a0 0x7f6bd015e430 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00b3f70 tx=0x7f6bc002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a1dc78169d743a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:57.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.235+0000 7f6bcd400640 1 -- 172.21.15.26:0/388882242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0030860 con 0x7f6bd00ae5a0 2024-01-15T17:55:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.235+0000 7f6bcd400640 1 -- 172.21.15.26:0/388882242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc00309c0 con 0x7f6bd00ae5a0 2024-01-15T17:55:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.235+0000 7f6bcd400640 1 -- 172.21.15.26:0/388882242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0030d50 con 0x7f6bd00ae5a0 2024-01-15T17:55:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.236+0000 7f6bd5400640 1 -- 172.21.15.26:0/388882242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00ae5a0 msgr2=0x7f6bd015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.236+0000 7f6bd5400640 1 --2- 172.21.15.26:0/388882242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00ae5a0 0x7f6bd015e430 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00b3f70 tx=0x7f6bc002b9a0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.236+0000 7f6bd5400640 1 -- 172.21.15.26:0/388882242 shutdown_connections 2024-01-15T17:55:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.236+0000 7f6bd5400640 1 --2- 172.21.15.26:0/388882242 >> v2:172.21.15.124:3300/0 conn(0x7f6bd00cf290 0x7f6bd00cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.236+0000 7f6bd5400640 1 --2- 172.21.15.26:0/388882242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00ae5a0 0x7f6bd015e430 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.236+0000 7f6bd5400640 1 -- 172.21.15.26:0/388882242 >> 172.21.15.26:0/388882242 conn(0x7f6bd006a800 msgr2=0x7f6bd006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:57.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.236+0000 7f6bd5400640 1 -- 172.21.15.26:0/388882242 shutdown_connections 2024-01-15T17:55:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.236+0000 7f6bd5400640 1 -- 172.21.15.26:0/388882242 wait complete. 2024-01-15T17:55:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.238+0000 7f6bd5400640 1 Processor -- start 2024-01-15T17:55:57.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.238+0000 7f6bd5400640 1 -- start start 2024-01-15T17:55:57.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bd5400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd00ae5a0 0x7f6bd014fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bd5400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd00b3ae0 0x7f6bd00aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bd5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00cf290 0x7f6bd00b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bd5400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00af3a0 con 0x7f6bd00cf290 2024-01-15T17:55:57.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bd5400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00af680 con 0x7f6bd00ae5a0 2024-01-15T17:55:57.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bd5400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00af960 con 0x7f6bd00b3ae0 2024-01-15T17:55:57.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bcf200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00cf290 0x7f6bd00b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bcf200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00cf290 0x7f6bd00b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48368/0 (socket says 172.21.15.26:48368) 2024-01-15T17:55:57.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bcf200640 1 -- 172.21.15.26:0/539289500 learned_addr learned my addr 172.21.15.26:0/539289500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:57.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.239+0000 7f6bce800640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd00ae5a0 0x7f6bd014fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bcf200640 1 -- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd00b3ae0 msgr2=0x7f6bd00aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bcde00640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd00b3ae0 0x7f6bd00aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bcf200640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd00b3ae0 0x7f6bd00aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bcf200640 1 -- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd00ae5a0 msgr2=0x7f6bd014fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bcf200640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd00ae5a0 0x7f6bd014fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bcf200640 1 -- 172.21.15.26:0/539289500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00094d0 con 0x7f6bd00cf290 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bcde00640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd00b3ae0 0x7f6bd00aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bcf200640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00cf290 0x7f6bd00b27a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8007e60 tx=0x7f6bc800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bc6a00640 1 -- 172.21.15.26:0/539289500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8014420 con 0x7f6bd00cf290 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bc6a00640 1 -- 172.21.15.26:0/539289500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc80145a0 con 0x7f6bd00cf290 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd00afc40 con 0x7f6bd00cf290 2024-01-15T17:55:57.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bc6a00640 1 -- 172.21.15.26:0/539289500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8014950 con 0x7f6bd00cf290 2024-01-15T17:55:57.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.240+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd0150130 con 0x7f6bd00cf290 2024-01-15T17:55:57.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.241+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd00675d0 con 0x7f6bd00cf290 2024-01-15T17:55:57.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.242+0000 7f6bc6a00640 1 -- 172.21.15.26:0/539289500 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6bc8014ab0 con 0x7f6bd00cf290 2024-01-15T17:55:57.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.243+0000 7f6bc6a00640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ba806b2c0 0x7f6ba806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.243+0000 7f6bc6a00640 1 -- 172.21.15.26:0/539289500 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6bc808a5a0 con 0x7f6bd00cf290 2024-01-15T17:55:57.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.244+0000 7f6bce800640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ba806b2c0 0x7f6ba806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.245+0000 7f6bce800640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ba806b2c0 0x7f6ba806d780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0036000 tx=0x7f6bc0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:57.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.247+0000 7f6bc6a00640 1 -- 172.21.15.26:0/539289500 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bc8055370 con 0x7f6bd00cf290 2024-01-15T17:55:57.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.436+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bd00cb620 con 0x7f6bd00cf290 2024-01-15T17:55:57.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.438+0000 7f6bc6a00640 1 -- 172.21.15.26:0/539289500 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f6bc8058990 con 0x7f6bd00cf290 2024-01-15T17:55:57.442 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:57.442 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:57.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ba806b2c0 msgr2=0x7f6ba806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ba806b2c0 0x7f6ba806d780 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0036000 tx=0x7f6bc0008030 comp rx=0 tx=0).stop 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00cf290 msgr2=0x7f6bd00b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00cf290 0x7f6bd00b27a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8007e60 tx=0x7f6bc800e4b0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 shutdown_connections 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd00b3ae0 0x7f6bd00aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ba806b2c0 0x7f6ba806d780 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bd00cf290 0x7f6bd00b27a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 --2- 172.21.15.26:0/539289500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd00ae5a0 0x7f6bd014fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 >> 172.21.15.26:0/539289500 conn(0x7f6bd006a800 msgr2=0x7f6bd0160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 shutdown_connections 2024-01-15T17:55:57.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.441+0000 7f6bd5400640 1 -- 172.21.15.26:0/539289500 wait complete. 2024-01-15T17:55:57.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:55:57.463 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:55:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.679+0000 7fbb00a00640 1 Processor -- start 2024-01-15T17:55:57.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.679+0000 7fbb00a00640 1 -- start start 2024-01-15T17:55:57.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.680+0000 7fbb00a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbafc1608b0 0x7fbafc161970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbb00a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbafc0ce000 0x7fbafc15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbb00a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbafc0d04e0 con 0x7fbafc0cf150 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbb00a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc161eb0 con 0x7fbafc0ce000 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbb00a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbafc160c90 con 0x7fbafc1608b0 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbafb200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbafc0ce000 0x7fbafc15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbaf9e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbafc0cf150 0x7fbafc0cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55274/0 (socket says 172.21.15.26:55274) 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbafb200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbafc0ce000 0x7fbafc15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48370/0 (socket says 172.21.15.26:48370) 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbaf9e00640 1 -- 172.21.15.26:0/3848331052 learned_addr learned my addr 172.21.15.26:0/3848331052 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbafa800640 1 --2- 172.21.15.26:0/3848331052 >> v2:172.21.15.124:3300/0 conn(0x7fbafc1608b0 0x7fbafc161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbafb200640 1 -- 172.21.15.26:0/3848331052 >> v2:172.21.15.124:3300/0 conn(0x7fbafc1608b0 msgr2=0x7fbafc161970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbafb200640 1 --2- 172.21.15.26:0/3848331052 >> v2:172.21.15.124:3300/0 conn(0x7fbafc1608b0 0x7fbafc161970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbafb200640 1 -- 172.21.15.26:0/3848331052 >> v1:172.21.15.26:6789/0 conn(0x7fbafc0cf150 legacy=0x7fbafc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.681+0000 7fbafb200640 1 -- 172.21.15.26:0/3848331052 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc0cf8d0 con 0x7fbafc0ce000 2024-01-15T17:55:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.683+0000 7fbafb200640 1 --2- 172.21.15.26:0/3848331052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 0x7fbafc15cb80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0719e0 tx=0x7fbaf402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e5952665b2e970 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.683+0000 7fbaf9400640 1 -- 172.21.15.26:0/3848331052 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4031830 con 0x7fbafc0ce000 2024-01-15T17:55:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.683+0000 7fbaf9400640 1 -- 172.21.15.26:0/3848331052 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf4031990 con 0x7fbafc0ce000 2024-01-15T17:55:57.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.683+0000 7fbaf9400640 1 -- 172.21.15.26:0/3848331052 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4031d20 con 0x7fbafc0ce000 2024-01-15T17:55:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.683+0000 7fbb00a00640 1 -- 172.21.15.26:0/3848331052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 msgr2=0x7fbafc15cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.683+0000 7fbb00a00640 1 --2- 172.21.15.26:0/3848331052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 0x7fbafc15cb80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0719e0 tx=0x7fbaf402b8d0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.684+0000 7fbb00a00640 1 -- 172.21.15.26:0/3848331052 shutdown_connections 2024-01-15T17:55:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.684+0000 7fbb00a00640 1 --2- 172.21.15.26:0/3848331052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 0x7fbafc15cb80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.684+0000 7fbb00a00640 1 --2- 172.21.15.26:0/3848331052 >> v2:172.21.15.124:3300/0 conn(0x7fbafc1608b0 0x7fbafc161970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.684+0000 7fbb00a00640 1 -- 172.21.15.26:0/3848331052 >> 172.21.15.26:0/3848331052 conn(0x7fbafc0b2a60 msgr2=0x7fbafc0b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.684+0000 7fbb00a00640 1 -- 172.21.15.26:0/3848331052 shutdown_connections 2024-01-15T17:55:57.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.684+0000 7fbb00a00640 1 -- 172.21.15.26:0/3848331052 wait complete. 2024-01-15T17:55:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.685+0000 7fbb00a00640 1 Processor -- start 2024-01-15T17:55:57.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.686+0000 7fbb00a00640 1 -- start start 2024-01-15T17:55:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbb00a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 0x7fbafc1f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbb00a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbafc0cf150 0x7fbafc0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbb00a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbafc1608b0 0x7fbafc0d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbb00a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc1593b0 con 0x7fbafc0ce000 2024-01-15T17:55:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbb00a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc1587a0 con 0x7fbafc0cf150 2024-01-15T17:55:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbb00a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbafc158a80 con 0x7fbafc1608b0 2024-01-15T17:55:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbafa800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 0x7fbafc1f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbafa800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 0x7fbafc1f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48386/0 (socket says 172.21.15.26:48386) 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbafa800640 1 -- 172.21.15.26:0/2417060140 learned_addr learned my addr 172.21.15.26:0/2417060140 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbafa800640 1 -- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbafc1608b0 msgr2=0x7fbafc0d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbafa800640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbafc1608b0 0x7fbafc0d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbafa800640 1 -- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbafc0cf150 msgr2=0x7fbafc0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbafa800640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbafc0cf150 0x7fbafc0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.687+0000 7fbafa800640 1 -- 172.21.15.26:0/2417060140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf4009400 con 0x7fbafc0ce000 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.688+0000 7fbafa800640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 0x7fbafc1f2670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d5f0 tx=0x7fbaec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.688+0000 7fbaf2a00640 1 -- 172.21.15.26:0/2417060140 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0125f0 con 0x7fbafc0ce000 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.688+0000 7fbaf2a00640 1 -- 172.21.15.26:0/2417060140 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec012750 con 0x7fbafc0ce000 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.688+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc158d60 con 0x7fbafc0ce000 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.688+0000 7fbaf2a00640 1 -- 172.21.15.26:0/2417060140 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec012ae0 con 0x7fbafc0ce000 2024-01-15T17:55:57.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.688+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc1f2bb0 con 0x7fbafc0ce000 2024-01-15T17:55:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.689+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbafc0b56b0 con 0x7fbafc0ce000 2024-01-15T17:55:57.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.690+0000 7fbaf2a00640 1 -- 172.21.15.26:0/2417060140 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbaec012c40 con 0x7fbafc0ce000 2024-01-15T17:55:57.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.691+0000 7fbaf2a00640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbad406b2f0 0x7fbad406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.692+0000 7fbaf2a00640 1 -- 172.21.15.26:0/2417060140 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbaec08a240 con 0x7fbafc0ce000 2024-01-15T17:55:57.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.692+0000 7fbaf9e00640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbad406b2f0 0x7fbad406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.693+0000 7fbaf9e00640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbad406b2f0 0x7fbad406d7b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0d2030 tx=0x7fbae4001660 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:57.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.696+0000 7fbaf2a00640 1 -- 172.21.15.26:0/2417060140 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbaec054ef0 con 0x7fbafc0ce000 2024-01-15T17:55:57.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.890+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbafc067b60 con 0x7fbafc0ce000 2024-01-15T17:55:57.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.891+0000 7fbaf2a00640 1 -- 172.21.15.26:0/2417060140 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbaec058510 con 0x7fbafc0ce000 2024-01-15T17:55:57.894 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:55:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.893+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbad406b2f0 msgr2=0x7fbad406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.893+0000 7fbb00a00640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbad406b2f0 0x7fbad406d7b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0d2030 tx=0x7fbae4001660 comp rx=0 tx=0).stop 2024-01-15T17:55:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.893+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 msgr2=0x7fbafc1f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.893+0000 7fbb00a00640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 0x7fbafc1f2670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d5f0 tx=0x7fbaec00dab0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.894+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 shutdown_connections 2024-01-15T17:55:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.894+0000 7fbb00a00640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbafc1608b0 0x7fbafc0d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.894+0000 7fbb00a00640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbad406b2f0 0x7fbad406d7b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.894+0000 7fbb00a00640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbafc0cf150 0x7fbafc0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.894+0000 7fbb00a00640 1 --2- 172.21.15.26:0/2417060140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbafc0ce000 0x7fbafc1f2670 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:57.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.894+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 >> 172.21.15.26:0/2417060140 conn(0x7fbafc0b2a60 msgr2=0x7fbafc15d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:57.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.894+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 shutdown_connections 2024-01-15T17:55:57.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:57.894+0000 7fbb00a00640 1 -- 172.21.15.26:0/2417060140 wait complete. 2024-01-15T17:55:57.916 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:55:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.131+0000 7f7a4b200640 1 Processor -- start 2024-01-15T17:55:58.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.131+0000 7f7a4b200640 1 -- start start 2024-01-15T17:55:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.133+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13cf10 0x7f7a4c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.133+0000 7f7a4b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c0ce440 con 0x7f7a4c13fae0 2024-01-15T17:55:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.133+0000 7f7a4b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c0ce720 con 0x7f7a4c0cddf0 2024-01-15T17:55:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.133+0000 7f7a4b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c13ff60 con 0x7f7a4c13cf10 2024-01-15T17:55:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.133+0000 7f7a41200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7a4c13fae0 0x7f7a4c0cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55284/0 (socket says 172.21.15.26:55284) 2024-01-15T17:55:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.133+0000 7f7a41200640 1 -- 172.21.15.26:0/4138310184 learned_addr learned my addr 172.21.15.26:0/4138310184 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202650341 0 0) 0x7f7a4c0ce440 con 0x7f7a4c13fae0 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49e00640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13cf10 0x7f7a4c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a2c003520 con 0x7f7a4c13fae0 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3487853119 0 0) 0x7f7a2c003520 con 0x7f7a4c13fae0 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13cf10 msgr2=0x7f7a4c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13cf10 0x7f7a4c13f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 >> v1:172.21.15.124:6789/0 conn(0x7f7a4c0cddf0 legacy=0x7f7a4c0b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0cea00 con 0x7f7a4c13fae0 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f7a34001670 con 0x7f7a4c13fae0 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 >> v1:172.21.15.26:6789/0 conn(0x7f7a4c13fae0 legacy=0x7f7a4c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a2c0039a0 0x7f7a2c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13cf10 0x7f7a2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.134+0000 7f7a49400640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c13ff60 con 0x7f7a2c004d00 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c00ba60 con 0x7f7a2c0039a0 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2c00bc20 con 0x7f7a4c13cf10 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a4a800640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a49e00640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a2c0039a0 0x7f7a2c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a41200640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13cf10 0x7f7a2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a4a800640 1 -- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13cf10 msgr2=0x7f7a2c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a4a800640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13cf10 0x7f7a2c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a4a800640 1 -- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a2c0039a0 msgr2=0x7f7a2c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a4a800640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a2c0039a0 0x7f7a2c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.135+0000 7f7a4a800640 1 -- 172.21.15.26:0/4138310184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c003520 con 0x7f7a2c004d00 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.136+0000 7f7a4a800640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a2c005180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7a44001e20 tx=0x7f7a2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bf2713584362e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.137+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44003b70 con 0x7f7a2c004d00 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.137+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a44003cd0 con 0x7f7a2c004d00 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.137+0000 7f7a49400640 1 -- 172.21.15.26:0/4138310184 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44005310 con 0x7f7a2c004d00 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.137+0000 7f7a4b200640 1 -- 172.21.15.26:0/4138310184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 msgr2=0x7f7a2c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.137+0000 7f7a4b200640 1 --2- 172.21.15.26:0/4138310184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a2c005180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7a44001e20 tx=0x7f7a2c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.137+0000 7f7a4a800640 1 -- 172.21.15.26:0/4138310184 reap_dead start 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.137+0000 7f7a4b200640 1 -- 172.21.15.26:0/4138310184 shutdown_connections 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.137+0000 7f7a4b200640 1 -- 172.21.15.26:0/4138310184 >> 172.21.15.26:0/4138310184 conn(0x7f7a4c06a7d0 msgr2=0x7f7a4c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.138+0000 7f7a4b200640 1 -- 172.21.15.26:0/4138310184 shutdown_connections 2024-01-15T17:55:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.138+0000 7f7a4b200640 1 -- 172.21.15.26:0/4138310184 wait complete. 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.139+0000 7f7a4b200640 1 Processor -- start 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.139+0000 7f7a4b200640 1 -- start start 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.140+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a4c13bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.140+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13e340 0x7f7a4c13e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.140+0000 7f7a4b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a4c15bb70 0x7f7a4c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.140+0000 7f7a4b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c155e70 con 0x7f7a2c004d00 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.140+0000 7f7a4b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c13ed00 con 0x7f7a4c15bb70 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.140+0000 7f7a4b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c15d180 con 0x7f7a4c13e340 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a49e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a4c13bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a49e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a4c13bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48404/0 (socket says 172.21.15.26:48404) 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a49e00640 1 -- 172.21.15.26:0/1888635984 learned_addr learned my addr 172.21.15.26:0/1888635984 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a49e00640 1 -- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13e340 msgr2=0x7f7a4c13e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a49e00640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13e340 0x7f7a4c13e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a49e00640 1 -- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a4c15bb70 msgr2=0x7f7a4c15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a49e00640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a4c15bb70 0x7f7a4c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a49e00640 1 -- 172.21.15.26:0/1888635984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c13c4e0 con 0x7f7a2c004d00 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a49e00640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a4c13bfa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c010ad0 tx=0x7f7a3c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.141+0000 7f7a42a00640 1 -- 172.21.15.26:0/1888635984 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c01a470 con 0x7f7a2c004d00 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.142+0000 7f7a42a00640 1 -- 172.21.15.26:0/1888635984 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3c01a5d0 con 0x7f7a2c004d00 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.142+0000 7f7a42a00640 1 -- 172.21.15.26:0/1888635984 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3c01a940 con 0x7f7a2c004d00 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.142+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c15c8e0 con 0x7f7a2c004d00 2024-01-15T17:55:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.142+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c15cea0 con 0x7f7a2c004d00 2024-01-15T17:55:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.143+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c0ce440 con 0x7f7a2c004d00 2024-01-15T17:55:58.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.144+0000 7f7a42a00640 1 -- 172.21.15.26:0/1888635984 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7a3c017070 con 0x7f7a2c004d00 2024-01-15T17:55:58.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.145+0000 7f7a42a00640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a2406b2f0 0x7f7a2406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.145+0000 7f7a42a00640 1 -- 172.21.15.26:0/1888635984 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7a3c08a120 con 0x7f7a2c004d00 2024-01-15T17:55:58.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.146+0000 7f7a41200640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a2406b2f0 0x7f7a2406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.147+0000 7f7a41200640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a2406b2f0 0x7f7a2406d7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c1f2340 tx=0x7f7a34009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:58.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.149+0000 7f7a42a00640 1 -- 172.21.15.26:0/1888635984 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a3c054e60 con 0x7f7a2c004d00 2024-01-15T17:55:58.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.337+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a4c067450 con 0x7f7a2c004d00 2024-01-15T17:55:58.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.338+0000 7f7a42a00640 1 -- 172.21.15.26:0/1888635984 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a3c058480 con 0x7f7a2c004d00 2024-01-15T17:55:58.342 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:55:58.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a2406b2f0 msgr2=0x7f7a2406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a2406b2f0 0x7f7a2406d7b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c1f2340 tx=0x7f7a34009040 comp rx=0 tx=0).stop 2024-01-15T17:55:58.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 msgr2=0x7f7a4c13bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a4c13bfa0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c010ad0 tx=0x7f7a3c010f90 comp rx=0 tx=0).stop 2024-01-15T17:55:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 shutdown_connections 2024-01-15T17:55:58.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4c13e340 0x7f7a4c13e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a2406b2f0 0x7f7a2406d7b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a4c15bb70 0x7f7a4c15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 --2- 172.21.15.26:0/1888635984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a2c004d00 0x7f7a4c13bfa0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 >> 172.21.15.26:0/1888635984 conn(0x7f7a4c06a7d0 msgr2=0x7f7a4c0aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.341+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 shutdown_connections 2024-01-15T17:55:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.342+0000 7f7a4b200640 1 -- 172.21.15.26:0/1888635984 wait complete. 2024-01-15T17:55:58.363 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:55:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.578+0000 7f5157e00640 1 Processor -- start 2024-01-15T17:55:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.579+0000 7f5157e00640 1 -- start start 2024-01-15T17:55:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.580+0000 7f5157e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51580cf9a0 0x7f51580cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.580+0000 7f5157e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51580d0320 con 0x7f51580b31d0 2024-01-15T17:55:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.580+0000 7f5157e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51580d1c60 con 0x7f515815d4b0 2024-01-15T17:55:58.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.580+0000 7f5157e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51580b2730 con 0x7f51580cf9a0 2024-01-15T17:55:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.581+0000 7f5157400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f51580b31d0 0x7f51580b3660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55300/0 (socket says 172.21.15.26:55300) 2024-01-15T17:55:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.581+0000 7f5157400640 1 -- 172.21.15.26:0/278573104 learned_addr learned my addr 172.21.15.26:0/278573104 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.581+0000 7f5156000640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51580cf9a0 0x7f51580cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.581+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075779183 0 0) 0x7f51580d0320 con 0x7f51580b31d0 2024-01-15T17:55:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.581+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5138003520 con 0x7f51580b31d0 2024-01-15T17:55:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3402599851 0 0) 0x7f5138003520 con 0x7f51580b31d0 2024-01-15T17:55:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51580cf9a0 msgr2=0x7f51580cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51580cf9a0 0x7f51580cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 >> v1:172.21.15.124:6789/0 conn(0x7f515815d4b0 legacy=0x7f51580d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580d1ec0 con 0x7f51580b31d0 2024-01-15T17:55:58.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f5150001670 con 0x7f51580b31d0 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 >> v1:172.21.15.26:6789/0 conn(0x7f51580b31d0 legacy=0x7f51580b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51580cf9a0 0x7f51380039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138003ee0 0x7f51380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5138004d00 0x7f5138005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51580b2730 con 0x7f5138004d00 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513800ba30 con 0x7f5138003ee0 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513800bbf0 con 0x7f51580cf9a0 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.582+0000 7f5157400640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5138004d00 0x7f5138005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.583+0000 7f5156000640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138003ee0 0x7f51380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.583+0000 7f5156a00640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51580cf9a0 0x7f51380039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.583+0000 7f5157400640 1 -- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51580cf9a0 msgr2=0x7f51380039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.583+0000 7f5157400640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51580cf9a0 0x7f51380039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.583+0000 7f5157400640 1 -- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138003ee0 msgr2=0x7f51380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.583+0000 7f5157400640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138003ee0 0x7f51380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.583+0000 7f5157400640 1 -- 172.21.15.26:0/278573104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5138003520 con 0x7f5138004d00 2024-01-15T17:55:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.584+0000 7f5157400640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5138004d00 0x7f5138005180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f515000c190 tx=0x7f51380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad725e51a972b17 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.584+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150005ad0 con 0x7f5138004d00 2024-01-15T17:55:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.584+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5150005c30 con 0x7f5138004d00 2024-01-15T17:55:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.584+0000 7f5155600640 1 -- 172.21.15.26:0/278573104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515002c810 con 0x7f5138004d00 2024-01-15T17:55:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.585+0000 7f5157e00640 1 -- 172.21.15.26:0/278573104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5138004d00 msgr2=0x7f5138005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.585+0000 7f5157e00640 1 --2- 172.21.15.26:0/278573104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5138004d00 0x7f5138005180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f515000c190 tx=0x7f51380057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.585+0000 7f5157400640 1 -- 172.21.15.26:0/278573104 reap_dead start 2024-01-15T17:55:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.585+0000 7f5157e00640 1 -- 172.21.15.26:0/278573104 shutdown_connections 2024-01-15T17:55:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.585+0000 7f5157e00640 1 -- 172.21.15.26:0/278573104 >> 172.21.15.26:0/278573104 conn(0x7f51580d3000 msgr2=0x7f51580d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.585+0000 7f5157e00640 1 -- 172.21.15.26:0/278573104 shutdown_connections 2024-01-15T17:55:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.585+0000 7f5157e00640 1 -- 172.21.15.26:0/278573104 wait complete. 2024-01-15T17:55:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.587+0000 7f5157e00640 1 Processor -- start 2024-01-15T17:55:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.587+0000 7f5157e00640 1 -- start start 2024-01-15T17:55:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.588+0000 7f5157e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138004d00 0x7f51581f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.588+0000 7f5157e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f515815b750 0x7f51580a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.588+0000 7f5157e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51581554c0 0x7f51580ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.588+0000 7f5157e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515815bfc0 con 0x7f515815b750 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.588+0000 7f5157e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515809e600 con 0x7f5138004d00 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.588+0000 7f5157e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515809e850 con 0x7f51581554c0 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.588+0000 7f5156000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f515815b750 0x7f51580a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.588+0000 7f5156000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f515815b750 0x7f51580a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48414/0 (socket says 172.21.15.26:48414) 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.588+0000 7f5156000640 1 -- 172.21.15.26:0/3583528875 learned_addr learned my addr 172.21.15.26:0/3583528875 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f5156000640 1 -- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51581554c0 msgr2=0x7f51580ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f5156000640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51581554c0 0x7f51580ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f5156a00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138004d00 0x7f51581f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f5156000640 1 -- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138004d00 msgr2=0x7f51581f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f5156000640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138004d00 0x7f51581f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f5156000640 1 -- 172.21.15.26:0/3583528875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f515809eb30 con 0x7f515815b750 2024-01-15T17:55:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f5156a00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138004d00 0x7f51581f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f5156000640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f515815b750 0x7f51580a05f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f51400115b0 tx=0x7f5140011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f514ea00640 1 -- 172.21.15.26:0/3583528875 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140015870 con 0x7f515815b750 2024-01-15T17:55:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.589+0000 7f514ea00640 1 -- 172.21.15.26:0/3583528875 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51400159d0 con 0x7f515815b750 2024-01-15T17:55:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.590+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515809ee10 con 0x7f515815b750 2024-01-15T17:55:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.590+0000 7f514ea00640 1 -- 172.21.15.26:0/3583528875 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140015d40 con 0x7f515815b750 2024-01-15T17:55:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.590+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5158155940 con 0x7f515815b750 2024-01-15T17:55:58.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.591+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515809cd70 con 0x7f515815b750 2024-01-15T17:55:58.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.592+0000 7f514ea00640 1 -- 172.21.15.26:0/3583528875 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f514000f480 con 0x7f515815b750 2024-01-15T17:55:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.593+0000 7f514ea00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f513006b2f0 0x7f513006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:58.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.593+0000 7f514ea00640 1 -- 172.21.15.26:0/3583528875 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f514008a0b0 con 0x7f515815b750 2024-01-15T17:55:58.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.594+0000 7f5156a00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f513006b2f0 0x7f513006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:58.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.595+0000 7f5156a00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f513006b2f0 0x7f513006d7b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5148001ec0 tx=0x7f5148007ef0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:58.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.598+0000 7f514ea00640 1 -- 172.21.15.26:0/3583528875 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f515809cd70 con 0x7f515815b750 2024-01-15T17:55:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.789+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f515809cd70 con 0x7f515815b750 2024-01-15T17:55:58.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.790+0000 7f514ea00640 1 -- 172.21.15.26:0/3583528875 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f515809cd70 con 0x7f515815b750 2024-01-15T17:55:58.794 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:55:58.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f513006b2f0 msgr2=0x7f513006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f513006b2f0 0x7f513006d7b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5148001ec0 tx=0x7f5148007ef0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f515815b750 msgr2=0x7f51580a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f515815b750 0x7f51580a05f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f51400115b0 tx=0x7f5140011a70 comp rx=0 tx=0).stop 2024-01-15T17:55:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 shutdown_connections 2024-01-15T17:55:58.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f51581554c0 0x7f51580ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f513006b2f0 0x7f513006d7b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f515815b750 0x7f51580a05f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 --2- 172.21.15.26:0/3583528875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5138004d00 0x7f51581f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 >> 172.21.15.26:0/3583528875 conn(0x7f51580d3000 msgr2=0x7f51580b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 shutdown_connections 2024-01-15T17:55:58.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:58.793+0000 7f5157e00640 1 -- 172.21.15.26:0/3583528875 wait complete. 2024-01-15T17:55:58.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:55:59.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.024+0000 7f03bb000640 1 Processor -- start 2024-01-15T17:55:59.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.024+0000 7f03bb000640 1 -- start start 2024-01-15T17:55:59.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.026+0000 7f03bb000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b413e360 0x7f03b40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.026+0000 7f03bb000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b415c5f0 con 0x7f03b415f9c0 2024-01-15T17:55:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.026+0000 7f03bb000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b406c470 con 0x7f03b413df80 2024-01-15T17:55:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.026+0000 7f03bb000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b406c750 con 0x7f03b413e360 2024-01-15T17:55:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.026+0000 7f03b9000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f03b415f9c0 0x7f03b40cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55304/0 (socket says 172.21.15.26:55304) 2024-01-15T17:55:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.026+0000 7f03b9000640 1 -- 172.21.15.26:0/2887696568 learned_addr learned my addr 172.21.15.26:0/2887696568 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:59.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.027+0000 7f03b3400640 1 --2- 172.21.15.26:0/2887696568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b413e360 0x7f03b40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.027+0000 7f03b2a00640 1 -- 172.21.15.26:0/2887696568 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062027730 0 0) 0x7f03b415c5f0 con 0x7f03b415f9c0 2024-01-15T17:55:59.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.027+0000 7f03b2a00640 1 -- 172.21.15.26:0/2887696568 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0394003520 con 0x7f03b415f9c0 2024-01-15T17:55:59.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.027+0000 7f03b3400640 1 -- 172.21.15.26:0/2887696568 >> v1:172.21.15.124:6789/0 conn(0x7f03b413df80 legacy=0x7f03b415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.027+0000 7f03b3400640 1 -- 172.21.15.26:0/2887696568 >> v1:172.21.15.26:6789/0 conn(0x7f03b415f9c0 legacy=0x7f03b40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.027+0000 7f03b3400640 1 -- 172.21.15.26:0/2887696568 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b406ca30 con 0x7f03b413e360 2024-01-15T17:55:59.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.029+0000 7f03b3400640 1 --2- 172.21.15.26:0/2887696568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b413e360 0x7f03b40cf290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f039c0097d0 tx=0x7f039c02b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc0dcc620bbe1a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:59.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.029+0000 7f03b2a00640 1 -- 172.21.15.26:0/2887696568 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c030800 con 0x7f03b413e360 2024-01-15T17:55:59.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.029+0000 7f03b2a00640 1 -- 172.21.15.26:0/2887696568 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c030960 con 0x7f03b413e360 2024-01-15T17:55:59.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.029+0000 7f03b2a00640 1 -- 172.21.15.26:0/2887696568 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c030cf0 con 0x7f03b413e360 2024-01-15T17:55:59.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.030+0000 7f03bb000640 1 -- 172.21.15.26:0/2887696568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b413e360 msgr2=0x7f03b40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.030+0000 7f03bb000640 1 --2- 172.21.15.26:0/2887696568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b413e360 0x7f03b40cf290 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f039c0097d0 tx=0x7f039c02b940 comp rx=0 tx=0).stop 2024-01-15T17:55:59.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.030+0000 7f03bb000640 1 -- 172.21.15.26:0/2887696568 shutdown_connections 2024-01-15T17:55:59.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.030+0000 7f03bb000640 1 --2- 172.21.15.26:0/2887696568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b413e360 0x7f03b40cf290 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.030+0000 7f03bb000640 1 -- 172.21.15.26:0/2887696568 >> 172.21.15.26:0/2887696568 conn(0x7f03b406a800 msgr2=0x7f03b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:59.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.030+0000 7f03bb000640 1 -- 172.21.15.26:0/2887696568 shutdown_connections 2024-01-15T17:55:59.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.030+0000 7f03bb000640 1 -- 172.21.15.26:0/2887696568 wait complete. 2024-01-15T17:55:59.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.032+0000 7f03bb000640 1 Processor -- start 2024-01-15T17:55:59.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.032+0000 7f03bb000640 1 -- start start 2024-01-15T17:55:59.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.033+0000 7f03bb000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b413df80 0x7f03b413cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.033+0000 7f03bb000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b413e360 0x7f03b41e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.033+0000 7f03bb000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b415f9c0 0x7f03b40b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.033+0000 7f03bb000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41f5340 con 0x7f03b413df80 2024-01-15T17:55:59.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.033+0000 7f03bb000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41f46c0 con 0x7f03b413e360 2024-01-15T17:55:59.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.033+0000 7f03bb000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b41f49a0 con 0x7f03b415f9c0 2024-01-15T17:55:59.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.033+0000 7f03b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b413df80 0x7f03b413cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.033+0000 7f03b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b413df80 0x7f03b413cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48422/0 (socket says 172.21.15.26:48422) 2024-01-15T17:55:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.033+0000 7f03b3e00640 1 -- 172.21.15.26:0/1200917818 learned_addr learned my addr 172.21.15.26:0/1200917818 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.034+0000 7f03b9000640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b415f9c0 0x7f03b40b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.034+0000 7f03b3400640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b413e360 0x7f03b41e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.034+0000 7f03b3e00640 1 -- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b415f9c0 msgr2=0x7f03b40b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.034+0000 7f03b3e00640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b415f9c0 0x7f03b40b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.034+0000 7f03b3e00640 1 -- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b413e360 msgr2=0x7f03b41e5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.034+0000 7f03b3e00640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b413e360 0x7f03b41e5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.034+0000 7f03b3e00640 1 -- 172.21.15.26:0/1200917818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c009400 con 0x7f03b413df80 2024-01-15T17:55:59.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.034+0000 7f03b3e00640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b413df80 0x7f03b413cf70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f03a400c3c0 tx=0x7f03a400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:59.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.035+0000 7f03abe00640 1 -- 172.21.15.26:0/1200917818 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40052d0 con 0x7f03b413df80 2024-01-15T17:55:59.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.035+0000 7f03abe00640 1 -- 172.21.15.26:0/1200917818 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a4005450 con 0x7f03b413df80 2024-01-15T17:55:59.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.035+0000 7f03abe00640 1 -- 172.21.15.26:0/1200917818 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4007260 con 0x7f03b413df80 2024-01-15T17:55:59.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.035+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b41f4c80 con 0x7f03b413df80 2024-01-15T17:55:59.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.035+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b413d4b0 con 0x7f03b413df80 2024-01-15T17:55:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.036+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b413d640 con 0x7f03b413df80 2024-01-15T17:55:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.037+0000 7f03abe00640 1 -- 172.21.15.26:0/1200917818 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f03a40073c0 con 0x7f03b413df80 2024-01-15T17:55:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.038+0000 7f03abe00640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038c06b2c0 0x7f038c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.038+0000 7f03abe00640 1 -- 172.21.15.26:0/1200917818 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f03a4088ff0 con 0x7f03b413df80 2024-01-15T17:55:59.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.039+0000 7f03b3400640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038c06b2c0 0x7f038c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.040+0000 7f03b3400640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038c06b2c0 0x7f038c06d780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f039c036000 tx=0x7f039c03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.043+0000 7f03abe00640 1 -- 172.21.15.26:0/1200917818 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03b413d640 con 0x7f03b413df80 2024-01-15T17:55:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.251+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f03b413d640 con 0x7f03b413df80 2024-01-15T17:55:59.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.252+0000 7f03abe00640 1 -- 172.21.15.26:0/1200917818 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f03b413d640 con 0x7f03b413df80 2024-01-15T17:55:59.255 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T17:55:59.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038c06b2c0 msgr2=0x7f038c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038c06b2c0 0x7f038c06d780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f039c036000 tx=0x7f039c03b040 comp rx=0 tx=0).stop 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b413df80 msgr2=0x7f03b413cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b413df80 0x7f03b413cf70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f03a400c3c0 tx=0x7f03a400c880 comp rx=0 tx=0).stop 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 shutdown_connections 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03b415f9c0 0x7f03b40b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038c06b2c0 0x7f038c06d780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03b413e360 0x7f03b41e5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 --2- 172.21.15.26:0/1200917818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03b413df80 0x7f03b413cf70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 >> 172.21.15.26:0/1200917818 conn(0x7f03b406a800 msgr2=0x7f03b40b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 shutdown_connections 2024-01-15T17:55:59.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.254+0000 7f03bb000640 1 -- 172.21.15.26:0/1200917818 wait complete. 2024-01-15T17:55:59.277 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi124.front.sepia.ceph.com... 2024-01-15T17:55:59.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:59.277 DEBUG:teuthology.orchestra.run.smithi124:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T17:55:59.380 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:55:59.380 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:55:59.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:55:59.380 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T17:55:59.434 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:55:59.434 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:55:59.494 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:55:59.494 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:55:59.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:55:59.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.759+0000 7f9dfb800640 1 Processor -- start 2024-01-15T17:55:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.760+0000 7f9dfb800640 1 -- start start 2024-01-15T17:55:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.761+0000 7f9dfb800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9df40cf9a0 0x7f9df4166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.761+0000 7f9dfb800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df415c5f0 con 0x7f9df413df80 2024-01-15T17:55:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.761+0000 7f9dfb800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df406c470 con 0x7f9df413e360 2024-01-15T17:55:59.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.761+0000 7f9dfb800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df406c750 con 0x7f9df40cf9a0 2024-01-15T17:55:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.761+0000 7f9df8e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9df413df80 0x7f9df415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55316/0 (socket says 172.21.15.26:55316) 2024-01-15T17:55:59.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.761+0000 7f9df8e00640 1 -- 172.21.15.26:0/1334925880 learned_addr learned my addr 172.21.15.26:0/1334925880 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df9800640 1 --2- 172.21.15.26:0/1334925880 >> v2:172.21.15.124:3300/0 conn(0x7f9df40cf9a0 0x7f9df4166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888539558 0 0) 0x7f9df415c5f0 con 0x7f9df413df80 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dd4003520 con 0x7f9df413df80 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3279055781 0 0) 0x7f9dd4003520 con 0x7f9df413df80 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 >> v2:172.21.15.124:3300/0 conn(0x7f9df40cf9a0 msgr2=0x7f9df4166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 --2- 172.21.15.26:0/1334925880 >> v2:172.21.15.124:3300/0 conn(0x7f9df40cf9a0 0x7f9df4166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 >> v1:172.21.15.124:6789/0 conn(0x7f9df413e360 legacy=0x7f9df40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df406ca30 con 0x7f9df413df80 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9de4001670 con 0x7f9df413df80 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 >> v1:172.21.15.26:6789/0 conn(0x7f9df413df80 legacy=0x7f9df415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd40047c0 0x7f9dd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.762+0000 7f9df3400640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd4005180 0x7f9dd400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df406c750 con 0x7f9dd40039c0 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd400c080 con 0x7f9dd4005180 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd400c240 con 0x7f9dd40047c0 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df8e00640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df9800640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd4005180 0x7f9dd400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df8e00640 1 -- 172.21.15.26:0/1334925880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd40047c0 msgr2=0x7f9dd4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df3e00640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd40047c0 0x7f9dd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df8e00640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd40047c0 0x7f9dd4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df8e00640 1 -- 172.21.15.26:0/1334925880 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd4005180 msgr2=0x7f9dd400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df8e00640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd4005180 0x7f9dd400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df8e00640 1 -- 172.21.15.26:0/1334925880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd4003520 con 0x7f9dd40039c0 2024-01-15T17:55:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.763+0000 7f9df9800640 1 -- 172.21.15.26:0/1334925880 reap_dead start 2024-01-15T17:55:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.764+0000 7f9df8e00640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9de4001050 tx=0x7f9dd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafc925babeae41f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.765+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4005ad0 con 0x7f9dd40039c0 2024-01-15T17:55:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.765+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de4005c30 con 0x7f9dd40039c0 2024-01-15T17:55:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.765+0000 7f9df3400640 1 -- 172.21.15.26:0/1334925880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de402c7c0 con 0x7f9dd40039c0 2024-01-15T17:55:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.765+0000 7f9dfb800640 1 -- 172.21.15.26:0/1334925880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd40039c0 msgr2=0x7f9dd4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.765+0000 7f9dfb800640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003de0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9de4001050 tx=0x7f9dd40057f0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.765+0000 7f9dfb800640 1 -- 172.21.15.26:0/1334925880 shutdown_connections 2024-01-15T17:55:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.765+0000 7f9dfb800640 1 --2- 172.21.15.26:0/1334925880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd40039c0 0x7f9dd4003de0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.765+0000 7f9dfb800640 1 -- 172.21.15.26:0/1334925880 >> 172.21.15.26:0/1334925880 conn(0x7f9df406a800 msgr2=0x7f9df406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.765+0000 7f9dfb800640 1 -- 172.21.15.26:0/1334925880 shutdown_connections 2024-01-15T17:55:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.766+0000 7f9dfb800640 1 -- 172.21.15.26:0/1334925880 wait complete. 2024-01-15T17:55:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.767+0000 7f9dfb800640 1 Processor -- start 2024-01-15T17:55:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.767+0000 7f9dfb800640 1 -- start start 2024-01-15T17:55:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.768+0000 7f9dfb800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd40039c0 0x7f9df4138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.768+0000 7f9dfb800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df40b3950 0x7f9df4150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.768+0000 7f9dfb800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9df414e590 0x7f9df415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.768+0000 7f9dfb800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df41390e0 con 0x7f9df40b3950 2024-01-15T17:55:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.768+0000 7f9dfb800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df41512a0 con 0x7f9df414e590 2024-01-15T17:55:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.768+0000 7f9dfb800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df40b2e70 con 0x7f9dd40039c0 2024-01-15T17:55:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df40b3950 0x7f9df4150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df40b3950 0x7f9df4150950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48444/0 (socket says 172.21.15.26:48444) 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df3e00640 1 -- 172.21.15.26:0/2952040560 learned_addr learned my addr 172.21.15.26:0/2952040560 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df8e00640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd40039c0 0x7f9df4138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df9800640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9df414e590 0x7f9df415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df3e00640 1 -- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd40039c0 msgr2=0x7f9df4138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df3e00640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd40039c0 0x7f9df4138ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df3e00640 1 -- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9df414e590 msgr2=0x7f9df415a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df3e00640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9df414e590 0x7f9df415a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df3e00640 1 -- 172.21.15.26:0/2952040560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df40b3150 con 0x7f9df40b3950 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.769+0000 7f9df9800640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9df414e590 0x7f9df415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.770+0000 7f9df3e00640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df40b3950 0x7f9df4150950 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00d4a0 tx=0x7f9ddc00d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.770+0000 7f9df0c00640 1 -- 172.21.15.26:0/2952040560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc005480 con 0x7f9df40b3950 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.770+0000 7f9df0c00640 1 -- 172.21.15.26:0/2952040560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ddc003610 con 0x7f9df40b3950 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.770+0000 7f9df0c00640 1 -- 172.21.15.26:0/2952040560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc003980 con 0x7f9df40b3950 2024-01-15T17:55:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.770+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df40b3430 con 0x7f9df40b3950 2024-01-15T17:55:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.770+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df4151070 con 0x7f9df40b3950 2024-01-15T17:55:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.771+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df40675d0 con 0x7f9df40b3950 2024-01-15T17:55:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.772+0000 7f9df0c00640 1 -- 172.21.15.26:0/2952040560 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9ddc003ae0 con 0x7f9df40b3950 2024-01-15T17:55:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.773+0000 7f9df0c00640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9dcc06b2c0 0x7f9dcc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:55:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.773+0000 7f9df0c00640 1 -- 172.21.15.26:0/2952040560 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9ddc089a60 con 0x7f9df40b3950 2024-01-15T17:55:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.774+0000 7f9df8e00640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9dcc06b2c0 0x7f9dcc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:55:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.775+0000 7f9df8e00640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9dcc06b2c0 0x7f9dcc06d780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9de4034000 tx=0x7f9de4007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:55:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.777+0000 7f9df0c00640 1 -- 172.21.15.26:0/2952040560 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ddc0548f0 con 0x7f9df40b3950 2024-01-15T17:55:59.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.974+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9df406df00 con 0x7f9df40b3950 2024-01-15T17:55:59.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.976+0000 7f9df0c00640 1 -- 172.21.15.26:0/2952040560 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f9ddc057f10 con 0x7f9df40b3950 2024-01-15T17:55:59.980 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:55:59.980 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:55:59.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.979+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9dcc06b2c0 msgr2=0x7f9dcc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.979+0000 7f9dfb800640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9dcc06b2c0 0x7f9dcc06d780 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9de4034000 tx=0x7f9de4007000 comp rx=0 tx=0).stop 2024-01-15T17:55:59.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.979+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df40b3950 msgr2=0x7f9df4150950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:55:59.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.980+0000 7f9dfb800640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df40b3950 0x7f9df4150950 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc00d4a0 tx=0x7f9ddc00d960 comp rx=0 tx=0).stop 2024-01-15T17:55:59.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.980+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 shutdown_connections 2024-01-15T17:55:59.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.980+0000 7f9dfb800640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9dcc06b2c0 0x7f9dcc06d780 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.980+0000 7f9dfb800640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9df414e590 0x7f9df415a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.980+0000 7f9dfb800640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9df40b3950 0x7f9df4150950 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.980+0000 7f9dfb800640 1 --2- 172.21.15.26:0/2952040560 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd40039c0 0x7f9df4138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:55:59.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.980+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 >> 172.21.15.26:0/2952040560 conn(0x7f9df406a800 msgr2=0x7f9df415c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:55:59.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.980+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 shutdown_connections 2024-01-15T17:55:59.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:55:59.980+0000 7f9dfb800640 1 -- 172.21.15.26:0/2952040560 wait complete. 2024-01-15T17:56:00.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:56:00.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:56:00.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.228+0000 7fd7b3e00640 1 Processor -- start 2024-01-15T17:56:00.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.228+0000 7fd7b3e00640 1 -- start start 2024-01-15T17:56:00.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.230+0000 7fd7b3e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac13df80 0x7fd7ac15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.230+0000 7fd7b3e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac06c470 con 0x7fd7ac0cf9a0 2024-01-15T17:56:00.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.230+0000 7fd7b3e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7ac06c750 con 0x7fd7ac13e360 2024-01-15T17:56:00.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.230+0000 7fd7b3e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac06ca30 con 0x7fd7ac13df80 2024-01-15T17:56:00.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.230+0000 7fd7b1e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd7ac0cf9a0 0x7fd7ac0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55330/0 (socket says 172.21.15.26:55330) 2024-01-15T17:56:00.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.230+0000 7fd7b1e00640 1 -- 172.21.15.26:0/2565276102 learned_addr learned my addr 172.21.15.26:0/2565276102 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:00.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.231+0000 7fd7b1400640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac13df80 0x7fd7ac15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.231+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250815107 0 0) 0x7fd7ac06c470 con 0x7fd7ac0cf9a0 2024-01-15T17:56:00.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.231+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd78c003520 con 0x7fd7ac0cf9a0 2024-01-15T17:56:00.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.231+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 95475561 0 0) 0x7fd78c003520 con 0x7fd7ac0cf9a0 2024-01-15T17:56:00.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.231+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac13df80 msgr2=0x7fd7ac15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:00.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.231+0000 7fd7abe00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac13df80 0x7fd7ac15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.231+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 >> v1:172.21.15.124:6789/0 conn(0x7fd7ac13e360 legacy=0x7fd7ac0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.231+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac06cd10 con 0x7fd7ac0cf9a0 2024-01-15T17:56:00.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.231+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd7a40019f0 con 0x7fd7ac0cf9a0 2024-01-15T17:56:00.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 >> v1:172.21.15.26:6789/0 conn(0x7fd7ac0cf9a0 legacy=0x7fd7ac0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7abe00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd78c0039a0 0x7fd78c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7abe00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac13df80 0x7fd78c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7abe00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78c004d00 0x7fd78c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac06ca30 con 0x7fd78c004d00 2024-01-15T17:56:00.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c00b980 con 0x7fd78c0039a0 2024-01-15T17:56:00.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c00bb40 con 0x7fd7ac13df80 2024-01-15T17:56:00.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7b1e00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78c004d00 0x7fd78c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7b1400640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd78c0039a0 0x7fd78c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.232+0000 7fd7b0a00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac13df80 0x7fd78c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.233+0000 7fd7b1e00640 1 -- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac13df80 msgr2=0x7fd78c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:00.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.233+0000 7fd7b1e00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac13df80 0x7fd78c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.233+0000 7fd7b1e00640 1 -- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd78c0039a0 msgr2=0x7fd78c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:00.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.233+0000 7fd7b1e00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd78c0039a0 0x7fd78c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.233+0000 7fd7b1e00640 1 -- 172.21.15.26:0/2565276102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c003520 con 0x7fd78c004d00 2024-01-15T17:56:00.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.234+0000 7fd7b1e00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78c004d00 0x7fd78c005180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400a7d0 tx=0x7fd78c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f792d7e9b4f85a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:00.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.234+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4034630 con 0x7fd78c004d00 2024-01-15T17:56:00.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.234+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a402f070 con 0x7fd78c004d00 2024-01-15T17:56:00.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.234+0000 7fd7abe00640 1 -- 172.21.15.26:0/2565276102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a40349f0 con 0x7fd78c004d00 2024-01-15T17:56:00.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.235+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2565276102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78c004d00 msgr2=0x7fd78c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.235+0000 7fd7b3e00640 1 --2- 172.21.15.26:0/2565276102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd78c004d00 0x7fd78c005180 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400a7d0 tx=0x7fd78c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.235+0000 7fd7b1e00640 1 -- 172.21.15.26:0/2565276102 reap_dead start 2024-01-15T17:56:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.235+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2565276102 shutdown_connections 2024-01-15T17:56:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.235+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2565276102 >> 172.21.15.26:0/2565276102 conn(0x7fd7ac06a800 msgr2=0x7fd7ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.235+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2565276102 shutdown_connections 2024-01-15T17:56:00.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.235+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2565276102 wait complete. 2024-01-15T17:56:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.237+0000 7fd7b3e00640 1 Processor -- start 2024-01-15T17:56:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.237+0000 7fd7b3e00640 1 -- start start 2024-01-15T17:56:00.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.238+0000 7fd7b3e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd78c004d00 0x7fd7ac1f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.238+0000 7fd7b3e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac1e53e0 0x7fd7ac1f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.238+0000 7fd7b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7ac1f4220 0x7fd7ac0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.238+0000 7fd7b3e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac1591a0 con 0x7fd7ac1f4220 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.238+0000 7fd7b3e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac1f4020 con 0x7fd78c004d00 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.238+0000 7fd7b3e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac158870 con 0x7fd7ac1e53e0 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.238+0000 7fd7b1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7ac1f4220 0x7fd7ac0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.238+0000 7fd7b1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7ac1f4220 0x7fd7ac0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48462/0 (socket says 172.21.15.26:48462) 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.238+0000 7fd7b1e00640 1 -- 172.21.15.26:0/2131279631 learned_addr learned my addr 172.21.15.26:0/2131279631 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.239+0000 7fd7b0a00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac1e53e0 0x7fd7ac1f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.239+0000 7fd7b1400640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd78c004d00 0x7fd7ac1f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.239+0000 7fd7b1e00640 1 -- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac1e53e0 msgr2=0x7fd7ac1f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.239+0000 7fd7b1e00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac1e53e0 0x7fd7ac1f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.239+0000 7fd7b1e00640 1 -- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd78c004d00 msgr2=0x7fd7ac1f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.239+0000 7fd7b1e00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd78c004d00 0x7fd7ac1f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.239+0000 7fd7b1e00640 1 -- 172.21.15.26:0/2131279631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac158b50 con 0x7fd7ac1f4220 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.239+0000 7fd7b1e00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7ac1f4220 0x7fd7ac0b2cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd7a402caf0 tx=0x7fd7a4001d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.239+0000 7fd7a9600640 1 -- 172.21.15.26:0/2131279631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4034640 con 0x7fd7ac1f4220 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.240+0000 7fd7a9600640 1 -- 172.21.15.26:0/2131279631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a40347a0 con 0x7fd7ac1f4220 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.240+0000 7fd7a9600640 1 -- 172.21.15.26:0/2131279631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4034b30 con 0x7fd7ac1f4220 2024-01-15T17:56:00.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.240+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ac158e30 con 0x7fd7ac1f4220 2024-01-15T17:56:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.240+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ac1f4680 con 0x7fd7ac1f4220 2024-01-15T17:56:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.241+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ac1f4870 con 0x7fd7ac1f4220 2024-01-15T17:56:00.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.242+0000 7fd7a9600640 1 -- 172.21.15.26:0/2131279631 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd7a4031030 con 0x7fd7ac1f4220 2024-01-15T17:56:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.243+0000 7fd7a9600640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd78406b2c0 0x7fd78406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.243+0000 7fd7a9600640 1 -- 172.21.15.26:0/2131279631 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd7a40ac2c0 con 0x7fd7ac1f4220 2024-01-15T17:56:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.244+0000 7fd7b1400640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd78406b2c0 0x7fd78406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.245+0000 7fd7b1400640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd78406b2c0 0x7fd78406d780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009b00 tx=0x7fd79c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:00.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.247+0000 7fd7a9600640 1 -- 172.21.15.26:0/2131279631 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7a40771b0 con 0x7fd7ac1f4220 2024-01-15T17:56:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.443+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7ac1f4870 con 0x7fd7ac1f4220 2024-01-15T17:56:00.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.444+0000 7fd7a9600640 1 -- 172.21.15.26:0/2131279631 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7a407a7d0 con 0x7fd7ac1f4220 2024-01-15T17:56:00.448 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:56:00.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.447+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd78406b2c0 msgr2=0x7fd78406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.447+0000 7fd7b3e00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd78406b2c0 0x7fd78406d780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd79c009b00 tx=0x7fd79c008040 comp rx=0 tx=0).stop 2024-01-15T17:56:00.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.447+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7ac1f4220 msgr2=0x7fd7ac0b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.447+0000 7fd7b3e00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7ac1f4220 0x7fd7ac0b2cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd7a402caf0 tx=0x7fd7a4001d00 comp rx=0 tx=0).stop 2024-01-15T17:56:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.447+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 shutdown_connections 2024-01-15T17:56:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.447+0000 7fd7b3e00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7ac1e53e0 0x7fd7ac1f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.447+0000 7fd7b3e00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd78406b2c0 0x7fd78406d780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.447+0000 7fd7b3e00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7ac1f4220 0x7fd7ac0b2cf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.447+0000 7fd7b3e00640 1 --2- 172.21.15.26:0/2131279631 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd78c004d00 0x7fd7ac1f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.448+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 >> 172.21.15.26:0/2131279631 conn(0x7fd7ac06a800 msgr2=0x7fd7ac0ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.448+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 shutdown_connections 2024-01-15T17:56:00.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.448+0000 7fd7b3e00640 1 -- 172.21.15.26:0/2131279631 wait complete. 2024-01-15T17:56:00.471 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:56:00.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.693+0000 7f30c7200640 1 Processor -- start 2024-01-15T17:56:00.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.693+0000 7f30c7200640 1 -- start start 2024-01-15T17:56:00.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.695+0000 7f30c7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80adf20 0x7f30c8070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.695+0000 7f30c7200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c815f4e0 con 0x7f30c80ccb50 2024-01-15T17:56:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.695+0000 7f30c7200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c8071130 con 0x7f30c815bf10 2024-01-15T17:56:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.695+0000 7f30c7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c815f970 con 0x7f30c80adf20 2024-01-15T17:56:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.696+0000 7f30c6800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f30c80ccb50 0x7f30c8143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55332/0 (socket says 172.21.15.26:55332) 2024-01-15T17:56:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.696+0000 7f30c6800640 1 -- 172.21.15.26:0/2338106643 learned_addr learned my addr 172.21.15.26:0/2338106643 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.696+0000 7f30c5e00640 1 --2- 172.21.15.26:0/2338106643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80adf20 0x7f30c8070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.696+0000 7f30c5400640 1 -- 172.21.15.26:0/2338106643 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42964541 0 0) 0x7f30c815f4e0 con 0x7f30c80ccb50 2024-01-15T17:56:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.696+0000 7f30c5400640 1 -- 172.21.15.26:0/2338106643 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30a8003520 con 0x7f30c80ccb50 2024-01-15T17:56:00.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.697+0000 7f30c5e00640 1 -- 172.21.15.26:0/2338106643 >> v1:172.21.15.124:6789/0 conn(0x7f30c815bf10 legacy=0x7f30c80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.697+0000 7f30c5e00640 1 -- 172.21.15.26:0/2338106643 >> v1:172.21.15.26:6789/0 conn(0x7f30c80ccb50 legacy=0x7f30c8143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.697+0000 7f30c5e00640 1 -- 172.21.15.26:0/2338106643 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c815c390 con 0x7f30c80adf20 2024-01-15T17:56:00.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.698+0000 7f30c5e00640 1 --2- 172.21.15.26:0/2338106643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80adf20 0x7f30c8070a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f30b80097d0 tx=0x7f30b802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=690759bf1f0e6230 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.698+0000 7f30c5400640 1 -- 172.21.15.26:0/2338106643 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8031800 con 0x7f30c80adf20 2024-01-15T17:56:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.699+0000 7f30c5400640 1 -- 172.21.15.26:0/2338106643 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b8031960 con 0x7f30c80adf20 2024-01-15T17:56:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.699+0000 7f30c5400640 1 -- 172.21.15.26:0/2338106643 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8031cf0 con 0x7f30c80adf20 2024-01-15T17:56:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.699+0000 7f30c7200640 1 -- 172.21.15.26:0/2338106643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80adf20 msgr2=0x7f30c8070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.699+0000 7f30c7200640 1 --2- 172.21.15.26:0/2338106643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80adf20 0x7f30c8070a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f30b80097d0 tx=0x7f30b802b8a0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.699+0000 7f30c7200640 1 -- 172.21.15.26:0/2338106643 shutdown_connections 2024-01-15T17:56:00.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.699+0000 7f30c7200640 1 --2- 172.21.15.26:0/2338106643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c80adf20 0x7f30c8070a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.699+0000 7f30c7200640 1 -- 172.21.15.26:0/2338106643 >> 172.21.15.26:0/2338106643 conn(0x7f30c806a7d0 msgr2=0x7f30c806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.699+0000 7f30c7200640 1 -- 172.21.15.26:0/2338106643 shutdown_connections 2024-01-15T17:56:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.699+0000 7f30c7200640 1 -- 172.21.15.26:0/2338106643 wait complete. 2024-01-15T17:56:00.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.701+0000 7f30c7200640 1 Processor -- start 2024-01-15T17:56:00.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.701+0000 7f30c7200640 1 -- start start 2024-01-15T17:56:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.702+0000 7f30c7200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80adf20 0x7f30c81f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.702+0000 7f30c7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c80ccb50 0x7f30c80c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.702+0000 7f30c7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c815bf10 0x7f30c80b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.702+0000 7f30c7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8133cc0 con 0x7f30c80ccb50 2024-01-15T17:56:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.702+0000 7f30c7200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8133fa0 con 0x7f30c80adf20 2024-01-15T17:56:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.702+0000 7f30c7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c8134280 con 0x7f30c815bf10 2024-01-15T17:56:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30bfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c80ccb50 0x7f30c80c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30bfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c80ccb50 0x7f30c80c3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48476/0 (socket says 172.21.15.26:48476) 2024-01-15T17:56:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30bfe00640 1 -- 172.21.15.26:0/4240512327 learned_addr learned my addr 172.21.15.26:0/4240512327 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:00.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30c5e00640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80adf20 0x7f30c81f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30c6800640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c815bf10 0x7f30c80b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30bfe00640 1 -- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c815bf10 msgr2=0x7f30c80b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30bfe00640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c815bf10 0x7f30c80b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30bfe00640 1 -- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80adf20 msgr2=0x7f30c81f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30bfe00640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80adf20 0x7f30c81f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30bfe00640 1 -- 172.21.15.26:0/4240512327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8009400 con 0x7f30c80ccb50 2024-01-15T17:56:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.703+0000 7f30c6800640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c815bf10 0x7f30c80b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:00.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.704+0000 7f30bfe00640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c80ccb50 0x7f30c80c3e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f30b000c450 tx=0x7f30b000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.704+0000 7f30be000640 1 -- 172.21.15.26:0/4240512327 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b00052d0 con 0x7f30c80ccb50 2024-01-15T17:56:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.704+0000 7f30be000640 1 -- 172.21.15.26:0/4240512327 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b0005430 con 0x7f30c80ccb50 2024-01-15T17:56:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.704+0000 7f30be000640 1 -- 172.21.15.26:0/4240512327 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b0012870 con 0x7f30c80ccb50 2024-01-15T17:56:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.704+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c8134560 con 0x7f30c80ccb50 2024-01-15T17:56:00.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.704+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c81f2710 con 0x7f30c80ccb50 2024-01-15T17:56:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.705+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c8134a80 con 0x7f30c80ccb50 2024-01-15T17:56:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.706+0000 7f30be000640 1 -- 172.21.15.26:0/4240512327 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f30b00129d0 con 0x7f30c80ccb50 2024-01-15T17:56:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.707+0000 7f30be000640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30a006b2c0 0x7f30a006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.707+0000 7f30be000640 1 -- 172.21.15.26:0/4240512327 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f30b00898a0 con 0x7f30c80ccb50 2024-01-15T17:56:00.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.708+0000 7f30c5e00640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30a006b2c0 0x7f30a006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:00.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.709+0000 7f30c5e00640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30a006b2c0 0x7f30a006d780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036000 tx=0x7f30b803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:00.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.711+0000 7f30be000640 1 -- 172.21.15.26:0/4240512327 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f30c8134a80 con 0x7f30c80ccb50 2024-01-15T17:56:00.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.918+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30c8134a80 con 0x7f30c80ccb50 2024-01-15T17:56:00.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.919+0000 7f30be000640 1 -- 172.21.15.26:0/4240512327 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30c8134a80 con 0x7f30c80ccb50 2024-01-15T17:56:00.922 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:56:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30a006b2c0 msgr2=0x7f30a006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30a006b2c0 0x7f30a006d780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f30b8036000 tx=0x7f30b803c040 comp rx=0 tx=0).stop 2024-01-15T17:56:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c80ccb50 msgr2=0x7f30c80c3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c80ccb50 0x7f30c80c3e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f30b000c450 tx=0x7f30b000c910 comp rx=0 tx=0).stop 2024-01-15T17:56:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 shutdown_connections 2024-01-15T17:56:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30c815bf10 0x7f30c80b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30a006b2c0 0x7f30a006d780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30c80ccb50 0x7f30c80c3e50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 --2- 172.21.15.26:0/4240512327 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30c80adf20 0x7f30c81f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 >> 172.21.15.26:0/4240512327 conn(0x7f30c806a7d0 msgr2=0x7f30c815b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 shutdown_connections 2024-01-15T17:56:00.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:00.922+0000 7f30c7200640 1 -- 172.21.15.26:0/4240512327 wait complete. 2024-01-15T17:56:00.946 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:56:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.169+0000 7f1891e00640 1 Processor -- start 2024-01-15T17:56:01.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.169+0000 7f1891e00640 1 -- start start 2024-01-15T17:56:01.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.170+0000 7f1891e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f188c0cf9a0 0x7f188c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.171+0000 7f1891e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c070c40 con 0x7f188c0b3ae0 2024-01-15T17:56:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.171+0000 7f1891e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c06c470 con 0x7f188c0ae5a0 2024-01-15T17:56:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.171+0000 7f1891e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c071450 con 0x7f188c0cf9a0 2024-01-15T17:56:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.171+0000 7f188b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f188c0cf9a0 0x7f188c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.171+0000 7f188a000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f188c0b3ae0 0x7f188c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55334/0 (socket says 172.21.15.26:55334) 2024-01-15T17:56:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.171+0000 7f188a000640 1 -- 172.21.15.26:0/4101567560 learned_addr learned my addr 172.21.15.26:0/4101567560 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.171+0000 7f188b400640 1 -- 172.21.15.26:0/4101567560 >> v1:172.21.15.124:6789/0 conn(0x7f188c0ae5a0 legacy=0x7f188c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:01.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.172+0000 7f188b400640 1 -- 172.21.15.26:0/4101567560 >> v1:172.21.15.26:6789/0 conn(0x7f188c0b3ae0 legacy=0x7f188c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.172+0000 7f188b400640 1 -- 172.21.15.26:0/4101567560 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c06c750 con 0x7f188c0cf9a0 2024-01-15T17:56:01.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.173+0000 7f188b400640 1 --2- 172.21.15.26:0/4101567560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 0x7f188c15e430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f18840097a0 tx=0x7f188402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7296367bfd4ef1a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:01.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.173+0000 7f1889600640 1 -- 172.21.15.26:0/4101567560 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884030610 con 0x7f188c0cf9a0 2024-01-15T17:56:01.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.173+0000 7f1889600640 1 -- 172.21.15.26:0/4101567560 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1884030770 con 0x7f188c0cf9a0 2024-01-15T17:56:01.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.173+0000 7f1889600640 1 -- 172.21.15.26:0/4101567560 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884030b00 con 0x7f188c0cf9a0 2024-01-15T17:56:01.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.174+0000 7f1891e00640 1 -- 172.21.15.26:0/4101567560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 msgr2=0x7f188c15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.174+0000 7f1891e00640 1 --2- 172.21.15.26:0/4101567560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 0x7f188c15e430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f18840097a0 tx=0x7f188402b960 comp rx=0 tx=0).stop 2024-01-15T17:56:01.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.174+0000 7f1891e00640 1 -- 172.21.15.26:0/4101567560 shutdown_connections 2024-01-15T17:56:01.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.174+0000 7f1891e00640 1 --2- 172.21.15.26:0/4101567560 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 0x7f188c15e430 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.174+0000 7f1891e00640 1 -- 172.21.15.26:0/4101567560 >> 172.21.15.26:0/4101567560 conn(0x7f188c06a800 msgr2=0x7f188c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:01.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.174+0000 7f1891e00640 1 -- 172.21.15.26:0/4101567560 shutdown_connections 2024-01-15T17:56:01.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.174+0000 7f1891e00640 1 -- 172.21.15.26:0/4101567560 wait complete. 2024-01-15T17:56:01.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.176+0000 7f1891e00640 1 Processor -- start 2024-01-15T17:56:01.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.176+0000 7f1891e00640 1 -- start start 2024-01-15T17:56:01.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.177+0000 7f1891e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188c0ae5a0 0x7f188c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.177+0000 7f1891e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f188c0b3ae0 0x7f188c149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.177+0000 7f1891e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 0x7f188c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.177+0000 7f1891e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0cb9d0 con 0x7f188c0cf9a0 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.177+0000 7f1891e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0cbcb0 con 0x7f188c0b3ae0 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.177+0000 7f1891e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0cbf90 con 0x7f188c0ae5a0 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.177+0000 7f188b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 0x7f188c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.177+0000 7f188b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 0x7f188c159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48498/0 (socket says 172.21.15.26:48498) 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.177+0000 7f188b400640 1 -- 172.21.15.26:0/3411945918 learned_addr learned my addr 172.21.15.26:0/3411945918 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.178+0000 7f188a000640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f188c0b3ae0 0x7f188c149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.178+0000 7f188aa00640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188c0ae5a0 0x7f188c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.178+0000 7f188b400640 1 -- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188c0ae5a0 msgr2=0x7f188c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.178+0000 7f188b400640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188c0ae5a0 0x7f188c0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.178+0000 7f188b400640 1 -- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f188c0b3ae0 msgr2=0x7f188c149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.178+0000 7f188b400640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f188c0b3ae0 0x7f188c149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.178+0000 7f188b400640 1 -- 172.21.15.26:0/3411945918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1884009400 con 0x7f188c0cf9a0 2024-01-15T17:56:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.178+0000 7f188b400640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 0x7f188c159460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f18840098c0 tx=0x7f18840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.179+0000 7f1883400640 1 -- 172.21.15.26:0/3411945918 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188403e070 con 0x7f188c0cf9a0 2024-01-15T17:56:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.179+0000 7f1883400640 1 -- 172.21.15.26:0/3411945918 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1884039990 con 0x7f188c0cf9a0 2024-01-15T17:56:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.179+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c0cc270 con 0x7f188c0cf9a0 2024-01-15T17:56:01.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.179+0000 7f1883400640 1 -- 172.21.15.26:0/3411945918 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884039d40 con 0x7f188c0cf9a0 2024-01-15T17:56:01.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.179+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c149dd0 con 0x7f188c0cf9a0 2024-01-15T17:56:01.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.180+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c0cc790 con 0x7f188c0cf9a0 2024-01-15T17:56:01.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.181+0000 7f1883400640 1 -- 172.21.15.26:0/3411945918 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f18840423f0 con 0x7f188c0cf9a0 2024-01-15T17:56:01.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.182+0000 7f1883400640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f186406b2f0 0x7f186406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.182+0000 7f1883400640 1 -- 172.21.15.26:0/3411945918 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f18840abbb0 con 0x7f188c0cf9a0 2024-01-15T17:56:01.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.182+0000 7f188aa00640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f186406b2f0 0x7f186406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.183+0000 7f188aa00640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f186406b2f0 0x7f186406d7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f187c005520 tx=0x7f187c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:01.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.186+0000 7f1883400640 1 -- 172.21.15.26:0/3411945918 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f188c0cc790 con 0x7f188c0cf9a0 2024-01-15T17:56:01.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.383+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f188c0cc790 con 0x7f188c0cf9a0 2024-01-15T17:56:01.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.384+0000 7f1883400640 1 -- 172.21.15.26:0/3411945918 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f188c0cc790 con 0x7f188c0cf9a0 2024-01-15T17:56:01.387 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:56:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.386+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f186406b2f0 msgr2=0x7f186406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.386+0000 7f1891e00640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f186406b2f0 0x7f186406d7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f187c005520 tx=0x7f187c009040 comp rx=0 tx=0).stop 2024-01-15T17:56:01.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.386+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 msgr2=0x7f188c159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.386+0000 7f1891e00640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 0x7f188c159460 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f18840098c0 tx=0x7f18840046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.386+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 shutdown_connections 2024-01-15T17:56:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.386+0000 7f1891e00640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f186406b2f0 0x7f186406d7b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.387+0000 7f1891e00640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f188c0cf9a0 0x7f188c159460 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.387+0000 7f1891e00640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f188c0b3ae0 0x7f188c149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.387+0000 7f1891e00640 1 --2- 172.21.15.26:0/3411945918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f188c0ae5a0 0x7f188c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.387+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 >> 172.21.15.26:0/3411945918 conn(0x7f188c06a800 msgr2=0x7f188c0cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.387+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 shutdown_connections 2024-01-15T17:56:01.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.387+0000 7f1891e00640 1 -- 172.21.15.26:0/3411945918 wait complete. 2024-01-15T17:56:01.410 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:56:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.620+0000 7f8ee1e00640 1 Processor -- start 2024-01-15T17:56:01.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.621+0000 7f8ee1e00640 1 -- start start 2024-01-15T17:56:01.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.623+0000 7f8ee1e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc146770 0x7f8edc06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.623+0000 7f8ee1e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc0b2e10 con 0x7f8edc071410 2024-01-15T17:56:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.623+0000 7f8ee1e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8edc146bc0 con 0x7f8edc071f00 2024-01-15T17:56:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.623+0000 7f8ee1e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc06cec0 con 0x7f8edc146770 2024-01-15T17:56:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.623+0000 7f8ee0a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8edc071410 0x7f8edc0717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55350/0 (socket says 172.21.15.26:55350) 2024-01-15T17:56:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.623+0000 7f8ee0a00640 1 -- 172.21.15.26:0/1951194768 learned_addr learned my addr 172.21.15.26:0/1951194768 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.623+0000 7f8ee1400640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc146770 0x7f8edc06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.623+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885891260 0 0) 0x7f8edc0b2e10 con 0x7f8edc071410 2024-01-15T17:56:01.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.623+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ebc003520 con 0x7f8edc071410 2024-01-15T17:56:01.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2086719846 0 0) 0x7f8ebc003520 con 0x7f8edc071410 2024-01-15T17:56:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc146770 msgr2=0x7f8edc06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc146770 0x7f8edc06c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 >> v1:172.21.15.124:6789/0 conn(0x7f8edc071f00 legacy=0x7f8edc164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc06d1a0 con 0x7f8edc071410 2024-01-15T17:56:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8ecc001670 con 0x7f8edc071410 2024-01-15T17:56:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 >> v1:172.21.15.26:6789/0 conn(0x7f8edc071410 legacy=0x7f8edc0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8ebc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc146770 0x7f8ebc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ebc004d00 0x7f8ebc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc06cec0 con 0x7f8ebc0039a0 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc00bb40 con 0x7f8ebc004d00 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc00bd00 con 0x7f8edc146770 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.624+0000 7f8ee0a00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8ebc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.625+0000 7f8ee1400640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ebc004d00 0x7f8ebc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.625+0000 7f8ed3e00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc146770 0x7f8ebc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.625+0000 7f8ee0a00640 1 -- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc146770 msgr2=0x7f8ebc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.625+0000 7f8ee0a00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc146770 0x7f8ebc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.625+0000 7f8ee0a00640 1 -- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ebc004d00 msgr2=0x7f8ebc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:01.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.625+0000 7f8ee0a00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ebc004d00 0x7f8ebc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.625+0000 7f8ee0a00640 1 -- 172.21.15.26:0/1951194768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc003520 con 0x7f8ebc0039a0 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.626+0000 7f8ee0a00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8ebc003dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00c0e0 tx=0x7f8ebc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96a5cbee637472e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.626+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc005ad0 con 0x7f8ebc0039a0 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.626+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc005c30 con 0x7f8ebc0039a0 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.626+0000 7f8edbe00640 1 -- 172.21.15.26:0/1951194768 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc02c810 con 0x7f8ebc0039a0 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.627+0000 7f8ee1e00640 1 -- 172.21.15.26:0/1951194768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 msgr2=0x7f8ebc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.627+0000 7f8ee1e00640 1 --2- 172.21.15.26:0/1951194768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8ebc003dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00c0e0 tx=0x7f8ebc0057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.627+0000 7f8ee1400640 1 -- 172.21.15.26:0/1951194768 reap_dead start 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.627+0000 7f8ee1e00640 1 -- 172.21.15.26:0/1951194768 shutdown_connections 2024-01-15T17:56:01.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.627+0000 7f8ee1e00640 1 -- 172.21.15.26:0/1951194768 >> 172.21.15.26:0/1951194768 conn(0x7f8edc06a800 msgr2=0x7f8edc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.627+0000 7f8ee1e00640 1 -- 172.21.15.26:0/1951194768 shutdown_connections 2024-01-15T17:56:01.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.627+0000 7f8ee1e00640 1 -- 172.21.15.26:0/1951194768 wait complete. 2024-01-15T17:56:01.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.629+0000 7f8ee1e00640 1 Processor -- start 2024-01-15T17:56:01.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.629+0000 7f8ee1e00640 1 -- start start 2024-01-15T17:56:01.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.630+0000 7f8ee1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8edc15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.630+0000 7f8ee1e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc0cdec0 0x7f8edc1f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.630+0000 7f8ee1e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8edc1f6210 0x7f8edc1f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.630+0000 7f8ee1e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc1f6be0 con 0x7f8ebc0039a0 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.630+0000 7f8ee1e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc130610 con 0x7f8edc1f6210 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.630+0000 7f8ee1e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc160140 con 0x7f8edc0cdec0 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.630+0000 7f8ee0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8edc15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ee0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8edc15f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48504/0 (socket says 172.21.15.26:48504) 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ee0a00640 1 -- 172.21.15.26:0/4035421239 learned_addr learned my addr 172.21.15.26:0/4035421239 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ed3e00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc0cdec0 0x7f8edc1f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ee1400640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8edc1f6210 0x7f8edc1f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ee0a00640 1 -- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc0cdec0 msgr2=0x7f8edc1f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ee0a00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc0cdec0 0x7f8edc1f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ee0a00640 1 -- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8edc1f6210 msgr2=0x7f8edc1f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ee0a00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8edc1f6210 0x7f8edc1f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ee0a00640 1 -- 172.21.15.26:0/4035421239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc15f8a0 con 0x7f8ebc0039a0 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.631+0000 7f8ee0a00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8edc15f1a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00c0e0 tx=0x7f8ecc001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.632+0000 7f8ed9600640 1 -- 172.21.15.26:0/4035421239 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc03c040 con 0x7f8ebc0039a0 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.632+0000 7f8ed9600640 1 -- 172.21.15.26:0/4035421239 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc02dcb0 con 0x7f8ebc0039a0 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.632+0000 7f8ed9600640 1 -- 172.21.15.26:0/4035421239 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc007270 con 0x7f8ebc0039a0 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.632+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc15fb80 con 0x7f8ebc0039a0 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.632+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc1ef9e0 con 0x7f8ebc0039a0 2024-01-15T17:56:01.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.633+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc0b1970 con 0x7f8ebc0039a0 2024-01-15T17:56:01.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.634+0000 7f8ed9600640 1 -- 172.21.15.26:0/4035421239 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f8ecc032070 con 0x7f8ebc0039a0 2024-01-15T17:56:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.635+0000 7f8ed9600640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8eb406b2c0 0x7f8eb406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.635+0000 7f8ed9600640 1 -- 172.21.15.26:0/4035421239 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8ecc0ab690 con 0x7f8ebc0039a0 2024-01-15T17:56:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.636+0000 7f8ed3e00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8eb406b2c0 0x7f8eb406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:01.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.637+0000 7f8ed3e00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8eb406b2c0 0x7f8eb406d780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8edc1400f0 tx=0x7f8ec4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:01.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.639+0000 7f8ed9600640 1 -- 172.21.15.26:0/4035421239 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ecc076460 con 0x7f8ebc0039a0 2024-01-15T17:56:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.847+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8edc067450 con 0x7f8ebc0039a0 2024-01-15T17:56:01.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.847+0000 7f8ed9600640 1 -- 172.21.15.26:0/4035421239 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8ecc079a80 con 0x7f8ebc0039a0 2024-01-15T17:56:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T17:56:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8eb406b2c0 msgr2=0x7f8eb406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8eb406b2c0 0x7f8eb406d780 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8edc1400f0 tx=0x7f8ec4009040 comp rx=0 tx=0).stop 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 msgr2=0x7f8edc15f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8edc15f1a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00c0e0 tx=0x7f8ecc001930 comp rx=0 tx=0).stop 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 shutdown_connections 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8edc0cdec0 0x7f8edc1f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8eb406b2c0 0x7f8eb406d780 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8edc1f6210 0x7f8edc1f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 --2- 172.21.15.26:0/4035421239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ebc0039a0 0x7f8edc15f1a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 >> 172.21.15.26:0/4035421239 conn(0x7f8edc06a800 msgr2=0x7f8edc144530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 shutdown_connections 2024-01-15T17:56:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:01.850+0000 7f8ee1e00640 1 -- 172.21.15.26:0/4035421239 wait complete. 2024-01-15T17:56:01.871 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:56:01.872 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:56:01.924 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:02.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.146+0000 7f79a5c00640 1 Processor -- start 2024-01-15T17:56:02.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.146+0000 7f79a5c00640 1 -- start start 2024-01-15T17:56:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.147+0000 7f79a5c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f79a00d0100 0x7f79a00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.147+0000 7f79a5c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a00d0a10 con 0x7f79a00d0f40 2024-01-15T17:56:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.147+0000 7f79a5c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a00cf080 con 0x7f79a0160920 2024-01-15T17:56:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.147+0000 7f79a5c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a00c6be0 con 0x7f79a00d0100 2024-01-15T17:56:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.148+0000 7f79a5200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f79a00d0f40 0x7f79a00d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51236/0 (socket says 172.21.15.26:51236) 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.148+0000 7f79a5200640 1 -- 172.21.15.26:0/3665045845 learned_addr learned my addr 172.21.15.26:0/3665045845 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.148+0000 7f799f400640 1 --2- 172.21.15.26:0/3665045845 >> v2:172.21.15.124:3300/0 conn(0x7f79a00d0100 0x7f79a00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.148+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843224108 0 0) 0x7f79a00d0a10 con 0x7f79a00d0f40 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.148+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7980003520 con 0x7f79a00d0f40 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2511872885 0 0) 0x7f7980003520 con 0x7f79a00d0f40 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 >> v2:172.21.15.124:3300/0 conn(0x7f79a00d0100 msgr2=0x7f79a00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 --2- 172.21.15.26:0/3665045845 >> v2:172.21.15.124:3300/0 conn(0x7f79a00d0100 0x7f79a00b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 >> v1:172.21.15.124:6789/0 conn(0x7f79a0160920 legacy=0x7f79a00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a00ce7e0 con 0x7f79a00d0f40 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f7998001670 con 0x7f79a00d0f40 2024-01-15T17:56:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 >> v1:172.21.15.26:6789/0 conn(0x7f79a00d0f40 legacy=0x7f79a00d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7980003970 0x7f7980003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79800047c0 0x7f7980004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980005180 0x7f798000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a00c6be0 con 0x7f7980003970 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798000c020 con 0x7f7980005180 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798000c1e0 con 0x7f79800047c0 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799fe00640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7980003970 0x7f7980003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f79a5200640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980005180 0x7f798000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.149+0000 7f799f400640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79800047c0 0x7f7980004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.150+0000 7f799fe00640 1 -- 172.21.15.26:0/3665045845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79800047c0 msgr2=0x7f7980004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.150+0000 7f799fe00640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79800047c0 0x7f7980004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.150+0000 7f799fe00640 1 -- 172.21.15.26:0/3665045845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980005180 msgr2=0x7f798000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.150+0000 7f799fe00640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980005180 0x7f798000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.150+0000 7f799fe00640 1 -- 172.21.15.26:0/3665045845 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7980003520 con 0x7f7980003970 2024-01-15T17:56:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.150+0000 7f79a5200640 1 -- 172.21.15.26:0/3665045845 reap_dead start 2024-01-15T17:56:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.151+0000 7f799fe00640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7980003970 0x7f7980003d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7990001e50 tx=0x7f79800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfd8d7916b2dfa5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.151+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7990003b70 con 0x7f7980003970 2024-01-15T17:56:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.151+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7990003cd0 con 0x7f7980003970 2024-01-15T17:56:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.151+0000 7f799ea00640 1 -- 172.21.15.26:0/3665045845 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7990005310 con 0x7f7980003970 2024-01-15T17:56:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.152+0000 7f79a5c00640 1 -- 172.21.15.26:0/3665045845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7980003970 msgr2=0x7f7980003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.152+0000 7f79a5c00640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7980003970 0x7f7980003d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7990001e50 tx=0x7f79800057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.152+0000 7f79a5c00640 1 -- 172.21.15.26:0/3665045845 shutdown_connections 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.152+0000 7f79a5c00640 1 --2- 172.21.15.26:0/3665045845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7980003970 0x7f7980003d90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.152+0000 7f79a5c00640 1 -- 172.21.15.26:0/3665045845 >> 172.21.15.26:0/3665045845 conn(0x7f79a00d3090 msgr2=0x7f79a00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.152+0000 7f79a5c00640 1 -- 172.21.15.26:0/3665045845 shutdown_connections 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.152+0000 7f79a5c00640 1 -- 172.21.15.26:0/3665045845 wait complete. 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.154+0000 7f79a5c00640 1 Processor -- start 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.154+0000 7f79a5c00640 1 -- start start 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f79a5c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980003970 0x7f79a015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f79a5c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a01f6530 0x7f79a015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f79a5c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79a01f3720 0x7f79a00b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f79a5c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a015f730 con 0x7f79a01f6530 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f79a5c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a01f3290 con 0x7f7980003970 2024-01-15T17:56:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f79a5c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a01f2f40 con 0x7f79a01f3720 2024-01-15T17:56:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f799f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a01f6530 0x7f79a015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f799f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a01f6530 0x7f79a015a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59496/0 (socket says 172.21.15.26:59496) 2024-01-15T17:56:02.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f799f400640 1 -- 172.21.15.26:0/2644191561 learned_addr learned my addr 172.21.15.26:0/2644191561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:02.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f799fe00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980003970 0x7f79a015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.155+0000 7f79a5200640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79a01f3720 0x7f79a00b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f799f400640 1 -- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79a01f3720 msgr2=0x7f79a00b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f799f400640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79a01f3720 0x7f79a00b7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f799f400640 1 -- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980003970 msgr2=0x7f79a015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f799f400640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980003970 0x7f79a015dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f799f400640 1 -- 172.21.15.26:0/2644191561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a01f6ca0 con 0x7f79a01f6530 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f799fe00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980003970 0x7f79a015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f799f400640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a01f6530 0x7f79a015a610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f79880108e0 tx=0x7f7988010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f7997e00640 1 -- 172.21.15.26:0/2644191561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798800eb60 con 0x7f79a01f6530 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f7997e00640 1 -- 172.21.15.26:0/2644191561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798800ecc0 con 0x7f79a01f6530 2024-01-15T17:56:02.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f7997e00640 1 -- 172.21.15.26:0/2644191561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798801c400 con 0x7f79a01f6530 2024-01-15T17:56:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a01f6f80 con 0x7f79a01f6530 2024-01-15T17:56:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.156+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a01f69b0 con 0x7f79a01f6530 2024-01-15T17:56:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.157+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a01f3420 con 0x7f79a01f6530 2024-01-15T17:56:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.158+0000 7f7997e00640 1 -- 172.21.15.26:0/2644191561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f798801c560 con 0x7f79a01f6530 2024-01-15T17:56:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.159+0000 7f7997e00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f797806b2c0 0x7f797806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.159+0000 7f7997e00640 1 -- 172.21.15.26:0/2644191561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f798808a270 con 0x7f79a01f6530 2024-01-15T17:56:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.160+0000 7f799fe00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f797806b2c0 0x7f797806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.161+0000 7f799fe00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f797806b2c0 0x7f797806d780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7990034000 tx=0x7f7990009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:02.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.164+0000 7f7997e00640 1 -- 172.21.15.26:0/2644191561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79880550f0 con 0x7f79a01f6530 2024-01-15T17:56:02.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.359+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79a0067450 con 0x7f79a01f6530 2024-01-15T17:56:02.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.362+0000 7f7997e00640 1 -- 172.21.15.26:0/2644191561 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f7988058710 con 0x7f79a01f6530 2024-01-15T17:56:02.366 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:02.366 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.364+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f797806b2c0 msgr2=0x7f797806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.364+0000 7f79a5c00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f797806b2c0 0x7f797806d780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7990034000 tx=0x7f7990009030 comp rx=0 tx=0).stop 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.364+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a01f6530 msgr2=0x7f79a015a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.364+0000 7f79a5c00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a01f6530 0x7f79a015a610 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f79880108e0 tx=0x7f7988010da0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.365+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 shutdown_connections 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.365+0000 7f79a5c00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f79a01f3720 0x7f79a00b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.365+0000 7f79a5c00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f797806b2c0 0x7f797806d780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.365+0000 7f79a5c00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f79a01f6530 0x7f79a015a610 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.365+0000 7f79a5c00640 1 --2- 172.21.15.26:0/2644191561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7980003970 0x7f79a015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.365+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 >> 172.21.15.26:0/2644191561 conn(0x7f79a00d3090 msgr2=0x7f79a006c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.365+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 shutdown_connections 2024-01-15T17:56:02.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.365+0000 7f79a5c00640 1 -- 172.21.15.26:0/2644191561 wait complete. 2024-01-15T17:56:02.389 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:56:02.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:56:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.618+0000 7f1b0a400640 1 Processor -- start 2024-01-15T17:56:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.619+0000 7f1b0a400640 1 -- start start 2024-01-15T17:56:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.620+0000 7f1b0a400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b041610e0 0x7f1b040d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.620+0000 7f1b0a400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 0x7f1b0415d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.620+0000 7f1b0a400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b040b2920 con 0x7f1b040cf4c0 2024-01-15T17:56:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.620+0000 7f1b0a400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b040cb670 con 0x7f1b041610e0 2024-01-15T17:56:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.620+0000 7f1b0a400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b040cb950 con 0x7f1b0415cc90 2024-01-15T17:56:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.621+0000 7f1b03400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b041610e0 0x7f1b040d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.621+0000 7f1b03400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b041610e0 0x7f1b040d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59502/0 (socket says 172.21.15.26:59502) 2024-01-15T17:56:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.621+0000 7f1b03400640 1 -- 172.21.15.26:0/3879687492 learned_addr learned my addr 172.21.15.26:0/3879687492 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.621+0000 7f1b03e00640 1 --2- 172.21.15.26:0/3879687492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 0x7f1b0415d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.621+0000 7f1b03400640 1 -- 172.21.15.26:0/3879687492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 msgr2=0x7f1b0415d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.621+0000 7f1b03400640 1 --2- 172.21.15.26:0/3879687492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 0x7f1b0415d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.621+0000 7f1b03400640 1 -- 172.21.15.26:0/3879687492 >> v1:172.21.15.124:6789/0 conn(0x7f1b040cf4c0 legacy=0x7f1b041614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.621+0000 7f1b03400640 1 -- 172.21.15.26:0/3879687492 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b040cbc30 con 0x7f1b041610e0 2024-01-15T17:56:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.622+0000 7f1b03400640 1 --2- 172.21.15.26:0/3879687492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b041610e0 0x7f1b040d0340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1af40098a0 tx=0x7f1af402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5129707c2a8b51b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.623+0000 7f1b02000640 1 -- 172.21.15.26:0/3879687492 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4031640 con 0x7f1b041610e0 2024-01-15T17:56:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.623+0000 7f1b02000640 1 -- 172.21.15.26:0/3879687492 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af40317a0 con 0x7f1b041610e0 2024-01-15T17:56:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.623+0000 7f1b02000640 1 -- 172.21.15.26:0/3879687492 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4031b30 con 0x7f1b041610e0 2024-01-15T17:56:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.623+0000 7f1b0a400640 1 -- 172.21.15.26:0/3879687492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b041610e0 msgr2=0x7f1b040d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.623+0000 7f1b0a400640 1 --2- 172.21.15.26:0/3879687492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b041610e0 0x7f1b040d0340 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1af40098a0 tx=0x7f1af402b8f0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.624+0000 7f1b0a400640 1 -- 172.21.15.26:0/3879687492 shutdown_connections 2024-01-15T17:56:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.624+0000 7f1b0a400640 1 --2- 172.21.15.26:0/3879687492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 0x7f1b0415d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.624+0000 7f1b0a400640 1 --2- 172.21.15.26:0/3879687492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b041610e0 0x7f1b040d0340 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.624+0000 7f1b0a400640 1 -- 172.21.15.26:0/3879687492 >> 172.21.15.26:0/3879687492 conn(0x7f1b040d1020 msgr2=0x7f1b040d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.624+0000 7f1b0a400640 1 -- 172.21.15.26:0/3879687492 shutdown_connections 2024-01-15T17:56:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.624+0000 7f1b0a400640 1 -- 172.21.15.26:0/3879687492 wait complete. 2024-01-15T17:56:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.626+0000 7f1b0a400640 1 Processor -- start 2024-01-15T17:56:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.626+0000 7f1b0a400640 1 -- start start 2024-01-15T17:56:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.627+0000 7f1b0a400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b040cf4c0 0x7f1b040c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.627+0000 7f1b0a400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 0x7f1b0415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.627+0000 7f1b0a400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b041610e0 0x7f1b04160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.627+0000 7f1b0a400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b04153e20 con 0x7f1b040cf4c0 2024-01-15T17:56:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.627+0000 7f1b0a400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0415e740 con 0x7f1b041610e0 2024-01-15T17:56:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.627+0000 7f1b0a400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0415ea20 con 0x7f1b0415cc90 2024-01-15T17:56:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.627+0000 7f1b03400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b040cf4c0 0x7f1b040c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.627+0000 7f1b03400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b040cf4c0 0x7f1b040c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59514/0 (socket says 172.21.15.26:59514) 2024-01-15T17:56:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.627+0000 7f1b03400640 1 -- 172.21.15.26:0/160246448 learned_addr learned my addr 172.21.15.26:0/160246448 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b02a00640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 0x7f1b0415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b03e00640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b041610e0 0x7f1b04160480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b03400640 1 -- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 msgr2=0x7f1b0415ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b03400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 0x7f1b0415ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b03400640 1 -- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b041610e0 msgr2=0x7f1b04160480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b03400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b041610e0 0x7f1b04160480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b03400640 1 -- 172.21.15.26:0/160246448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af40094d0 con 0x7f1b040cf4c0 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b02a00640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 0x7f1b0415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b03e00640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b041610e0 0x7f1b04160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1b03400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b040cf4c0 0x7f1b040c9d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1af40098a0 tx=0x7f1af40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.628+0000 7f1afb400640 1 -- 172.21.15.26:0/160246448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af403ac60 con 0x7f1b040cf4c0 2024-01-15T17:56:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.629+0000 7f1afb400640 1 -- 172.21.15.26:0/160246448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af403adc0 con 0x7f1b040cf4c0 2024-01-15T17:56:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.629+0000 7f1afb400640 1 -- 172.21.15.26:0/160246448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af4044630 con 0x7f1b040cf4c0 2024-01-15T17:56:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.629+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0415ed00 con 0x7f1b040cf4c0 2024-01-15T17:56:02.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.629+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0415f2c0 con 0x7f1b040cf4c0 2024-01-15T17:56:02.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.630+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0415f4b0 con 0x7f1b040cf4c0 2024-01-15T17:56:02.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.631+0000 7f1afb400640 1 -- 172.21.15.26:0/160246448 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1af402f050 con 0x7f1b040cf4c0 2024-01-15T17:56:02.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.632+0000 7f1afb400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1adc06b2f0 0x7f1adc06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:02.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.632+0000 7f1afb400640 1 -- 172.21.15.26:0/160246448 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1af40abb00 con 0x7f1b040cf4c0 2024-01-15T17:56:02.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.633+0000 7f1b02a00640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1adc06b2f0 0x7f1adc06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:02.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.634+0000 7f1b02a00640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1adc06b2f0 0x7f1adc06d7b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b040cac20 tx=0x7f1aec009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:02.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.637+0000 7f1afb400640 1 -- 172.21.15.26:0/160246448 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1af40ade60 con 0x7f1b040cf4c0 2024-01-15T17:56:02.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.839+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b040675d0 con 0x7f1b040cf4c0 2024-01-15T17:56:02.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.840+0000 7f1afb400640 1 -- 172.21.15.26:0/160246448 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1af4076840 con 0x7f1b040cf4c0 2024-01-15T17:56:02.843 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:56:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.842+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1adc06b2f0 msgr2=0x7f1adc06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.842+0000 7f1b0a400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1adc06b2f0 0x7f1adc06d7b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1b040cac20 tx=0x7f1aec009040 comp rx=0 tx=0).stop 2024-01-15T17:56:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.842+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b040cf4c0 msgr2=0x7f1b040c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.842+0000 7f1b0a400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b040cf4c0 0x7f1b040c9d30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1af40098a0 tx=0x7f1af40046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.843+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 shutdown_connections 2024-01-15T17:56:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.843+0000 7f1b0a400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b0415cc90 0x7f1b0415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.843+0000 7f1b0a400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1adc06b2f0 0x7f1adc06d7b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.843+0000 7f1b0a400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b041610e0 0x7f1b04160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.843+0000 7f1b0a400640 1 --2- 172.21.15.26:0/160246448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b040cf4c0 0x7f1b040c9d30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:02.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.843+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 >> 172.21.15.26:0/160246448 conn(0x7f1b040d1020 msgr2=0x7f1b04155620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:02.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.843+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 shutdown_connections 2024-01-15T17:56:02.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:02.843+0000 7f1b0a400640 1 -- 172.21.15.26:0/160246448 wait complete. 2024-01-15T17:56:02.866 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:56:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.083+0000 7f4e1fe00640 1 Processor -- start 2024-01-15T17:56:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.084+0000 7f4e1fe00640 1 -- start start 2024-01-15T17:56:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.085+0000 7f4e1fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e2015d4b0 0x7f4e200d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.085+0000 7f4e1fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e200cf9a0 0x7f4e200cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.085+0000 7f4e1fe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e200d0320 con 0x7f4e200cddf0 2024-01-15T17:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.085+0000 7f4e1fe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e200b2730 con 0x7f4e200cf9a0 2024-01-15T17:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.085+0000 7f4e1fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200d1ec0 con 0x7f4e2015d4b0 2024-01-15T17:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.086+0000 7f4e1e000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e200cf9a0 0x7f4e200cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.086+0000 7f4e1e000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e200cf9a0 0x7f4e200cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59520/0 (socket says 172.21.15.26:59520) 2024-01-15T17:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.086+0000 7f4e1e000640 1 -- 172.21.15.26:0/2047263339 learned_addr learned my addr 172.21.15.26:0/2047263339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.086+0000 7f4e1e000640 1 -- 172.21.15.26:0/2047263339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e2015d4b0 msgr2=0x7f4e200d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.086+0000 7f4e1e000640 1 --2- 172.21.15.26:0/2047263339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e2015d4b0 0x7f4e200d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.086+0000 7f4e1e000640 1 -- 172.21.15.26:0/2047263339 >> v1:172.21.15.124:6789/0 conn(0x7f4e200cddf0 legacy=0x7f4e200ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.086+0000 7f4e1e000640 1 -- 172.21.15.26:0/2047263339 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e200b29d0 con 0x7f4e200cf9a0 2024-01-15T17:56:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.087+0000 7f4e1e000640 1 --2- 172.21.15.26:0/2047263339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e200cf9a0 0x7f4e200cfde0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e080098a0 tx=0x7f4e0802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c06494b116609f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1d600640 1 -- 172.21.15.26:0/2047263339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08031640 con 0x7f4e200cf9a0 2024-01-15T17:56:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1d600640 1 -- 172.21.15.26:0/2047263339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e080317a0 con 0x7f4e200cf9a0 2024-01-15T17:56:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1d600640 1 -- 172.21.15.26:0/2047263339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08031b30 con 0x7f4e200cf9a0 2024-01-15T17:56:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2047263339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e200cf9a0 msgr2=0x7f4e200cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1fe00640 1 --2- 172.21.15.26:0/2047263339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e200cf9a0 0x7f4e200cfde0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e080098a0 tx=0x7f4e0802b8f0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2047263339 shutdown_connections 2024-01-15T17:56:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1fe00640 1 --2- 172.21.15.26:0/2047263339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e200cf9a0 0x7f4e200cfde0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1fe00640 1 --2- 172.21.15.26:0/2047263339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e2015d4b0 0x7f4e200d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2047263339 >> 172.21.15.26:0/2047263339 conn(0x7f4e200d3000 msgr2=0x7f4e200d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.088+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2047263339 shutdown_connections 2024-01-15T17:56:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.089+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2047263339 wait complete. 2024-01-15T17:56:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.090+0000 7f4e1fe00640 1 Processor -- start 2024-01-15T17:56:03.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.090+0000 7f4e1fe00640 1 -- start start 2024-01-15T17:56:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.091+0000 7f4e1fe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200cddf0 0x7f4e201574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.091+0000 7f4e1fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e200cf9a0 0x7f4e200b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.091+0000 7f4e1fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2015d4b0 0x7f4e200b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.091+0000 7f4e1fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200ca250 con 0x7f4e2015d4b0 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.091+0000 7f4e1fe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200ca530 con 0x7f4e200cddf0 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.091+0000 7f4e1fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e200ca810 con 0x7f4e200cf9a0 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2015d4b0 0x7f4e200b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2015d4b0 0x7f4e200b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59526/0 (socket says 172.21.15.26:59526) 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1f400640 1 -- 172.21.15.26:0/2001687887 learned_addr learned my addr 172.21.15.26:0/2001687887 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1ea00640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200cddf0 0x7f4e201574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1e000640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e200cf9a0 0x7f4e200b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1f400640 1 -- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e200cf9a0 msgr2=0x7f4e200b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1f400640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e200cf9a0 0x7f4e200b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1f400640 1 -- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200cddf0 msgr2=0x7f4e201574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1f400640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200cddf0 0x7f4e201574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.092+0000 7f4e1f400640 1 -- 172.21.15.26:0/2001687887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e080094d0 con 0x7f4e2015d4b0 2024-01-15T17:56:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.093+0000 7f4e1f400640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2015d4b0 0x7f4e200b1380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800c780 tx=0x7f4e1800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.093+0000 7f4e17400640 1 -- 172.21.15.26:0/2001687887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e18004b60 con 0x7f4e2015d4b0 2024-01-15T17:56:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.093+0000 7f4e17400640 1 -- 172.21.15.26:0/2001687887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1800ce90 con 0x7f4e2015d4b0 2024-01-15T17:56:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.093+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2015da10 con 0x7f4e2015d4b0 2024-01-15T17:56:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.093+0000 7f4e17400640 1 -- 172.21.15.26:0/2001687887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1800a850 con 0x7f4e2015d4b0 2024-01-15T17:56:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.093+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e20157a00 con 0x7f4e2015d4b0 2024-01-15T17:56:03.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.094+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e200cad30 con 0x7f4e2015d4b0 2024-01-15T17:56:03.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.096+0000 7f4e17400640 1 -- 172.21.15.26:0/2001687887 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4e1800a9b0 con 0x7f4e2015d4b0 2024-01-15T17:56:03.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.096+0000 7f4e17400640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4df806b2c0 0x7f4df806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.096+0000 7f4e17400640 1 -- 172.21.15.26:0/2001687887 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4e1808a130 con 0x7f4e2015d4b0 2024-01-15T17:56:03.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.097+0000 7f4e1ea00640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4df806b2c0 0x7f4df806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.098+0000 7f4e1ea00640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4df806b2c0 0x7f4df806d780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000b810 tx=0x7f4e10008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:03.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.101+0000 7f4e17400640 1 -- 172.21.15.26:0/2001687887 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e200cad30 con 0x7f4e2015d4b0 2024-01-15T17:56:03.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.294+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e200cad30 con 0x7f4e2015d4b0 2024-01-15T17:56:03.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.295+0000 7f4e17400640 1 -- 172.21.15.26:0/2001687887 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e200cad30 con 0x7f4e2015d4b0 2024-01-15T17:56:03.298 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:56:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4df806b2c0 msgr2=0x7f4df806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4df806b2c0 0x7f4df806d780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000b810 tx=0x7f4e10008040 comp rx=0 tx=0).stop 2024-01-15T17:56:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2015d4b0 msgr2=0x7f4e200b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2015d4b0 0x7f4e200b1380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800c780 tx=0x7f4e1800cc40 comp rx=0 tx=0).stop 2024-01-15T17:56:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 shutdown_connections 2024-01-15T17:56:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e200cf9a0 0x7f4e200b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4df806b2c0 0x7f4df806d780 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e2015d4b0 0x7f4e200b1380 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 --2- 172.21.15.26:0/2001687887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e200cddf0 0x7f4e201574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 >> 172.21.15.26:0/2001687887 conn(0x7f4e200d3000 msgr2=0x7f4e2006aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 shutdown_connections 2024-01-15T17:56:03.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.298+0000 7f4e1fe00640 1 -- 172.21.15.26:0/2001687887 wait complete. 2024-01-15T17:56:03.321 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:56:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.542+0000 7f4cacc00640 1 Processor -- start 2024-01-15T17:56:03.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.542+0000 7f4cacc00640 1 -- start start 2024-01-15T17:56:03.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.543+0000 7f4cacc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ca80cf9a0 0x7f4ca80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.543+0000 7f4cacc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca80d0320 con 0x7f4ca80b31d0 2024-01-15T17:56:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.544+0000 7f4cacc00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca80d1c60 con 0x7f4ca815d4b0 2024-01-15T17:56:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.544+0000 7f4cacc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca80b2730 con 0x7f4ca80cf9a0 2024-01-15T17:56:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.544+0000 7f4ca6a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ca80cf9a0 0x7f4ca80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.544+0000 7f4ca7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4ca80b31d0 0x7f4ca80b3660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51252/0 (socket says 172.21.15.26:51252) 2024-01-15T17:56:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.544+0000 7f4ca7e00640 1 -- 172.21.15.26:0/1869970469 learned_addr learned my addr 172.21.15.26:0/1869970469 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.544+0000 7f4ca6a00640 1 -- 172.21.15.26:0/1869970469 >> v1:172.21.15.124:6789/0 conn(0x7f4ca815d4b0 legacy=0x7f4ca80d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.544+0000 7f4ca6a00640 1 -- 172.21.15.26:0/1869970469 >> v1:172.21.15.26:6789/0 conn(0x7f4ca80b31d0 legacy=0x7f4ca80b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.544+0000 7f4ca6a00640 1 -- 172.21.15.26:0/1869970469 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80d1ec0 con 0x7f4ca80cf9a0 2024-01-15T17:56:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.545+0000 7f4ca6a00640 1 --2- 172.21.15.26:0/1869970469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80cf9a0 0x7f4ca80cfde0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4c900097a0 tx=0x7f4c9002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803b618f7418af69 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:03.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.546+0000 7f4ca6000640 1 -- 172.21.15.26:0/1869970469 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90030610 con 0x7f4ca80cf9a0 2024-01-15T17:56:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.546+0000 7f4ca6000640 1 -- 172.21.15.26:0/1869970469 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90030770 con 0x7f4ca80cf9a0 2024-01-15T17:56:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.546+0000 7f4ca6000640 1 -- 172.21.15.26:0/1869970469 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90030b00 con 0x7f4ca80cf9a0 2024-01-15T17:56:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.546+0000 7f4cacc00640 1 -- 172.21.15.26:0/1869970469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80cf9a0 msgr2=0x7f4ca80cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.546+0000 7f4cacc00640 1 --2- 172.21.15.26:0/1869970469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80cf9a0 0x7f4ca80cfde0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4c900097a0 tx=0x7f4c9002b960 comp rx=0 tx=0).stop 2024-01-15T17:56:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.546+0000 7f4cacc00640 1 -- 172.21.15.26:0/1869970469 shutdown_connections 2024-01-15T17:56:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.546+0000 7f4cacc00640 1 --2- 172.21.15.26:0/1869970469 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80cf9a0 0x7f4ca80cfde0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.546+0000 7f4cacc00640 1 -- 172.21.15.26:0/1869970469 >> 172.21.15.26:0/1869970469 conn(0x7f4ca80d3000 msgr2=0x7f4ca80d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.547+0000 7f4cacc00640 1 -- 172.21.15.26:0/1869970469 shutdown_connections 2024-01-15T17:56:03.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.547+0000 7f4cacc00640 1 -- 172.21.15.26:0/1869970469 wait complete. 2024-01-15T17:56:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.548+0000 7f4cacc00640 1 Processor -- start 2024-01-15T17:56:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.548+0000 7f4cacc00640 1 -- start start 2024-01-15T17:56:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.549+0000 7f4cacc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80b31d0 0x7f4ca81f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.549+0000 7f4cacc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca80cf9a0 0x7f4ca809e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.549+0000 7f4cacc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ca815d4b0 0x7f4ca814d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.549+0000 7f4cacc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80ccbc0 con 0x7f4ca80b31d0 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.549+0000 7f4cacc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca814e770 con 0x7f4ca815d4b0 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.549+0000 7f4cacc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca80cc8e0 con 0x7f4ca80cf9a0 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.549+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80b31d0 0x7f4ca81f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.549+0000 7f4ca7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80b31d0 0x7f4ca81f39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59546/0 (socket says 172.21.15.26:59546) 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.549+0000 7f4ca7400640 1 -- 172.21.15.26:0/3283581637 learned_addr learned my addr 172.21.15.26:0/3283581637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4ca7400640 1 -- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca80cf9a0 msgr2=0x7f4ca809e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4ca7400640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca80cf9a0 0x7f4ca809e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4ca7400640 1 -- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ca815d4b0 msgr2=0x7f4ca814d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4ca7400640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ca815d4b0 0x7f4ca814d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4ca7400640 1 -- 172.21.15.26:0/3283581637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c90009400 con 0x7f4ca80b31d0 2024-01-15T17:56:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4ca7400640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80b31d0 0x7f4ca81f39e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c400 tx=0x7f4c9800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4c9f400640 1 -- 172.21.15.26:0/3283581637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980052d0 con 0x7f4ca80b31d0 2024-01-15T17:56:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4c9f400640 1 -- 172.21.15.26:0/3283581637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c98005430 con 0x7f4ca80b31d0 2024-01-15T17:56:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4c9f400640 1 -- 172.21.15.26:0/3283581637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98003830 con 0x7f4ca80b31d0 2024-01-15T17:56:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.550+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca80ca170 con 0x7f4ca80b31d0 2024-01-15T17:56:03.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.551+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca80ca730 con 0x7f4ca80b31d0 2024-01-15T17:56:03.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.552+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca8067450 con 0x7f4ca80b31d0 2024-01-15T17:56:03.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.553+0000 7f4c9f400640 1 -- 172.21.15.26:0/3283581637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4c98003990 con 0x7f4ca80b31d0 2024-01-15T17:56:03.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.554+0000 7f4c9f400640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4c8006b2c0 0x7f4c8006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.554+0000 7f4c9f400640 1 -- 172.21.15.26:0/3283581637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4c980891f0 con 0x7f4ca80b31d0 2024-01-15T17:56:03.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.554+0000 7f4ca6a00640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4c8006b2c0 0x7f4c8006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.555+0000 7f4ca6a00640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4c8006b2c0 0x7f4c8006d780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c90036000 tx=0x7f4c9003b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:03.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.558+0000 7f4c9f400640 1 -- 172.21.15.26:0/3283581637 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c98053fc0 con 0x7f4ca80b31d0 2024-01-15T17:56:03.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.744+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ca80b21b0 con 0x7f4ca80b31d0 2024-01-15T17:56:03.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.745+0000 7f4c9f400640 1 -- 172.21.15.26:0/3283581637 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c980575e0 con 0x7f4ca80b31d0 2024-01-15T17:56:03.748 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:56:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.747+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4c8006b2c0 msgr2=0x7f4c8006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.747+0000 7f4cacc00640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4c8006b2c0 0x7f4c8006d780 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c90036000 tx=0x7f4c9003b040 comp rx=0 tx=0).stop 2024-01-15T17:56:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80b31d0 msgr2=0x7f4ca81f39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80b31d0 0x7f4ca81f39e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800c400 tx=0x7f4c9800c8c0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 shutdown_connections 2024-01-15T17:56:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ca80cf9a0 0x7f4ca809e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4c8006b2c0 0x7f4c8006d780 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ca815d4b0 0x7f4ca814d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 --2- 172.21.15.26:0/3283581637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ca80b31d0 0x7f4ca81f39e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 >> 172.21.15.26:0/3283581637 conn(0x7f4ca80d3000 msgr2=0x7f4ca80b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:03.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 shutdown_connections 2024-01-15T17:56:03.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.748+0000 7f4cacc00640 1 -- 172.21.15.26:0/3283581637 wait complete. 2024-01-15T17:56:03.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:56:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.986+0000 7fd5fe000640 1 Processor -- start 2024-01-15T17:56:03.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.986+0000 7fd5fe000640 1 -- start start 2024-01-15T17:56:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.987+0000 7fd5fe000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5f80cdfd0 0x7fd5f815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fe000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd5f80cf120 0x7fd5f80cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fe000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f806bbc0 con 0x7fd5f806b700 2024-01-15T17:56:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fe000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f80b04d0 con 0x7fd5f80cdfd0 2024-01-15T17:56:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fe000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f80b07b0 con 0x7fd5f80cf120 2024-01-15T17:56:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fcc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5f80cdfd0 0x7fd5f815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fcc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5f80cdfd0 0x7fd5f815cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59560/0 (socket says 172.21.15.26:59560) 2024-01-15T17:56:03.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fcc00640 1 -- 172.21.15.26:0/3611276858 learned_addr learned my addr 172.21.15.26:0/3611276858 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fd600640 1 --2- 172.21.15.26:0/3611276858 >> v2:172.21.15.124:3300/0 conn(0x7fd5f80cf120 0x7fd5f80cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fcc00640 1 -- 172.21.15.26:0/3611276858 >> v2:172.21.15.124:3300/0 conn(0x7fd5f80cf120 msgr2=0x7fd5f80cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fcc00640 1 --2- 172.21.15.26:0/3611276858 >> v2:172.21.15.124:3300/0 conn(0x7fd5f80cf120 0x7fd5f80cf5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fcc00640 1 -- 172.21.15.26:0/3611276858 >> v1:172.21.15.26:6789/0 conn(0x7fd5f806b700 legacy=0x7fd5f815d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.988+0000 7fd5fcc00640 1 -- 172.21.15.26:0/3611276858 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f80b0a90 con 0x7fd5f80cdfd0 2024-01-15T17:56:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.990+0000 7fd5fcc00640 1 --2- 172.21.15.26:0/3611276858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cdfd0 0x7fd5f815cac0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80718e0 tx=0x7fd5e802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454bb0e443671b95 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.990+0000 7fd5f7400640 1 -- 172.21.15.26:0/3611276858 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8030860 con 0x7fd5f80cdfd0 2024-01-15T17:56:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.990+0000 7fd5f7400640 1 -- 172.21.15.26:0/3611276858 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e80309c0 con 0x7fd5f80cdfd0 2024-01-15T17:56:03.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.990+0000 7fd5f7400640 1 -- 172.21.15.26:0/3611276858 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8030d50 con 0x7fd5f80cdfd0 2024-01-15T17:56:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.990+0000 7fd5fe000640 1 -- 172.21.15.26:0/3611276858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cdfd0 msgr2=0x7fd5f815cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.990+0000 7fd5fe000640 1 --2- 172.21.15.26:0/3611276858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cdfd0 0x7fd5f815cac0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80718e0 tx=0x7fd5e802b9a0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.991+0000 7fd5fe000640 1 -- 172.21.15.26:0/3611276858 shutdown_connections 2024-01-15T17:56:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.991+0000 7fd5fe000640 1 --2- 172.21.15.26:0/3611276858 >> v2:172.21.15.124:3300/0 conn(0x7fd5f80cf120 0x7fd5f80cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.991+0000 7fd5fe000640 1 --2- 172.21.15.26:0/3611276858 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cdfd0 0x7fd5f815cac0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.991+0000 7fd5fe000640 1 -- 172.21.15.26:0/3611276858 >> 172.21.15.26:0/3611276858 conn(0x7fd5f80d0e50 msgr2=0x7fd5f80d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.991+0000 7fd5fe000640 1 -- 172.21.15.26:0/3611276858 shutdown_connections 2024-01-15T17:56:03.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.991+0000 7fd5fe000640 1 -- 172.21.15.26:0/3611276858 wait complete. 2024-01-15T17:56:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.992+0000 7fd5fe000640 1 Processor -- start 2024-01-15T17:56:03.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.993+0000 7fd5fe000640 1 -- start start 2024-01-15T17:56:03.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.993+0000 7fd5fe000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f806b700 0x7fd5f815f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.993+0000 7fd5fe000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f80cdfd0 0x7fd5f814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.993+0000 7fd5fe000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cf120 0x7fd5f80ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.993+0000 7fd5fe000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f815f550 con 0x7fd5f80cf120 2024-01-15T17:56:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.993+0000 7fd5fe000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f815e940 con 0x7fd5f806b700 2024-01-15T17:56:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.993+0000 7fd5fe000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f815ec20 con 0x7fd5f80cdfd0 2024-01-15T17:56:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.994+0000 7fd5fd600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cf120 0x7fd5f80ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.994+0000 7fd5fd600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cf120 0x7fd5f80ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59566/0 (socket says 172.21.15.26:59566) 2024-01-15T17:56:03.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.994+0000 7fd5fd600640 1 -- 172.21.15.26:0/77501762 learned_addr learned my addr 172.21.15.26:0/77501762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.994+0000 7fd5fcc00640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f806b700 0x7fd5f815f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.994+0000 7fd5fd600640 1 -- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f80cdfd0 msgr2=0x7fd5f814b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.994+0000 7fd5fd600640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f80cdfd0 0x7fd5f814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.994+0000 7fd5fd600640 1 -- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f806b700 msgr2=0x7fd5f815f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:03.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.994+0000 7fd5fd600640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f806b700 0x7fd5f815f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.994+0000 7fd5fd600640 1 -- 172.21.15.26:0/77501762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e80094d0 con 0x7fd5f80cf120 2024-01-15T17:56:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.995+0000 7fd5fd600640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cf120 0x7fd5f80ae6d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0007e60 tx=0x7fd5f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.995+0000 7fd5f4c00640 1 -- 172.21.15.26:0/77501762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0014420 con 0x7fd5f80cf120 2024-01-15T17:56:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.995+0000 7fd5f4c00640 1 -- 172.21.15.26:0/77501762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f00145a0 con 0x7fd5f80cf120 2024-01-15T17:56:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.995+0000 7fd5f4c00640 1 -- 172.21.15.26:0/77501762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0014950 con 0x7fd5f80cf120 2024-01-15T17:56:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.995+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f815ef00 con 0x7fd5f80cf120 2024-01-15T17:56:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.995+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f815ff20 con 0x7fd5f80cf120 2024-01-15T17:56:03.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.996+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f815f700 con 0x7fd5f80cf120 2024-01-15T17:56:04.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.997+0000 7fd5f4c00640 1 -- 172.21.15.26:0/77501762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd5f0014ab0 con 0x7fd5f80cf120 2024-01-15T17:56:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.998+0000 7fd5f4c00640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d006b2c0 0x7fd5d006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.998+0000 7fd5f4c00640 1 -- 172.21.15.26:0/77501762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd5f008a680 con 0x7fd5f80cf120 2024-01-15T17:56:04.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:03.998+0000 7fd5fcc00640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d006b2c0 0x7fd5d006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:04.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.000+0000 7fd5fcc00640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d006b2c0 0x7fd5d006d780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8036000 tx=0x7fd5e8008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:04.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.002+0000 7fd5f4c00640 1 -- 172.21.15.26:0/77501762 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5f815f700 con 0x7fd5f80cf120 2024-01-15T17:56:04.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.216+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd5f8067450 con 0x7fd5f80cf120 2024-01-15T17:56:04.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.217+0000 7fd5f4c00640 1 -- 172.21.15.26:0/77501762 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd5f0058e50 con 0x7fd5f80cf120 2024-01-15T17:56:04.220 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T17:56:04.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d006b2c0 msgr2=0x7fd5d006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d006b2c0 0x7fd5d006d780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8036000 tx=0x7fd5e8008030 comp rx=0 tx=0).stop 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cf120 msgr2=0x7fd5f80ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cf120 0x7fd5f80ae6d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0007e60 tx=0x7fd5f000e4b0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 shutdown_connections 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5f80cdfd0 0x7fd5f814b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d006b2c0 0x7fd5d006d780 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5f80cf120 0x7fd5f80ae6d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 --2- 172.21.15.26:0/77501762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5f806b700 0x7fd5f815f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.220+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 >> 172.21.15.26:0/77501762 conn(0x7fd5f80d0e50 msgr2=0x7fd5f806f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.221+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 shutdown_connections 2024-01-15T17:56:04.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.221+0000 7fd5fe000640 1 -- 172.21.15.26:0/77501762 wait complete. 2024-01-15T17:56:04.244 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-01-15T17:56:04.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:04.244 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-01-15T17:56:04.342 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:56:04.342 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:56:04.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:04.343 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T17:56:04.397 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:56:04.397 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:56:04.456 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:56:04.457 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:56:04.507 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:04.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.732+0000 7fa724800640 1 Processor -- start 2024-01-15T17:56:04.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.733+0000 7fa724800640 1 -- start start 2024-01-15T17:56:04.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.734+0000 7fa724800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c0ae5a0 0x7fa71c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa724800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa71c0cf290 0x7fa71c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa724800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71c070c40 con 0x7fa71c0b3ae0 2024-01-15T17:56:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa724800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71c071450 con 0x7fa71c0cf290 2024-01-15T17:56:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa724800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0cfc60 con 0x7fa71c0ae5a0 2024-01-15T17:56:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa721400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa71c0b3ae0 0x7fa71c06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51266/0 (socket says 172.21.15.26:51266) 2024-01-15T17:56:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa721400640 1 -- 172.21.15.26:0/3516253617 learned_addr learned my addr 172.21.15.26:0/3516253617 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:04.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa722800640 1 --2- 172.21.15.26:0/3516253617 >> v2:172.21.15.124:3300/0 conn(0x7fa71c0cf290 0x7fa71c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa721e00640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c0ae5a0 0x7fa71c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532033681 0 0) 0x7fa71c070c40 con 0x7fa71c0b3ae0 2024-01-15T17:56:04.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.735+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6fc003520 con 0x7fa71c0b3ae0 2024-01-15T17:56:04.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3969358279 0 0) 0x7fa6fc003520 con 0x7fa71c0b3ae0 2024-01-15T17:56:04.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c0ae5a0 msgr2=0x7fa71c15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:04.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c0ae5a0 0x7fa71c15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 >> v2:172.21.15.124:3300/0 conn(0x7fa71c0cf290 msgr2=0x7fa71c0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 --2- 172.21.15.26:0/3516253617 >> v2:172.21.15.124:3300/0 conn(0x7fa71c0cf290 0x7fa71c0cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c06c470 con 0x7fa71c0b3ae0 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa704001670 con 0x7fa71c0b3ae0 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 >> v1:172.21.15.26:6789/0 conn(0x7fa71c0b3ae0 legacy=0x7fa71c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6fc003970 0x7fa6fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc0047c0 0x7fa6fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c0ae5a0 0x7fa6fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c071450 con 0x7fa6fc003970 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0cfc60 con 0x7fa6fc0047c0 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc00ba30 con 0x7fa71c0ae5a0 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.736+0000 7fa721e00640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6fc003970 0x7fa6fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.737+0000 7fa721400640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc0047c0 0x7fa6fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.737+0000 7fa721e00640 1 -- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c0ae5a0 msgr2=0x7fa6fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:04.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.737+0000 7fa721e00640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c0ae5a0 0x7fa6fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.737+0000 7fa721e00640 1 -- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc0047c0 msgr2=0x7fa6fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:04.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.737+0000 7fa721e00640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc0047c0 0x7fa6fc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.737+0000 7fa721e00640 1 -- 172.21.15.26:0/3516253617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc003520 con 0x7fa6fc003970 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.738+0000 7fa721e00640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6fc003970 0x7fa6fc003d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00ba30 tx=0x7fa6fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6355a7aa0b902347 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.738+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c032680 con 0x7fa6fc003970 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.738+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c0327e0 con 0x7fa6fc003970 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.738+0000 7fa720a00640 1 -- 172.21.15.26:0/3516253617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c032b70 con 0x7fa6fc003970 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.739+0000 7fa724800640 1 -- 172.21.15.26:0/3516253617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6fc003970 msgr2=0x7fa6fc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.739+0000 7fa724800640 1 --2- 172.21.15.26:0/3516253617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6fc003970 0x7fa6fc003d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00ba30 tx=0x7fa6fc0057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.739+0000 7fa722800640 1 -- 172.21.15.26:0/3516253617 reap_dead start 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.739+0000 7fa724800640 1 -- 172.21.15.26:0/3516253617 shutdown_connections 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.739+0000 7fa724800640 1 -- 172.21.15.26:0/3516253617 >> 172.21.15.26:0/3516253617 conn(0x7fa71c06a800 msgr2=0x7fa71c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.739+0000 7fa724800640 1 -- 172.21.15.26:0/3516253617 shutdown_connections 2024-01-15T17:56:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.739+0000 7fa724800640 1 -- 172.21.15.26:0/3516253617 wait complete. 2024-01-15T17:56:04.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.741+0000 7fa724800640 1 Processor -- start 2024-01-15T17:56:04.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.741+0000 7fa724800640 1 -- start start 2024-01-15T17:56:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa724800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc003970 0x7fa71c156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa724800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa71c15c5f0 0x7fa71c15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa724800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c157770 0x7fa71c157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa724800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c15cfb0 con 0x7fa71c15c5f0 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa724800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c1f3e10 con 0x7fa6fc003970 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa724800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c1f4060 con 0x7fa71c157770 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa721400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa71c15c5f0 0x7fa71c15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa721400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa71c15c5f0 0x7fa71c15ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59582/0 (socket says 172.21.15.26:59582) 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa721400640 1 -- 172.21.15.26:0/4039583007 learned_addr learned my addr 172.21.15.26:0/4039583007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa721e00640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc003970 0x7fa71c156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.742+0000 7fa722800640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c157770 0x7fa71c157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa721400640 1 -- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c157770 msgr2=0x7fa71c157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa721400640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c157770 0x7fa71c157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa721400640 1 -- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc003970 msgr2=0x7fa71c156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa721400640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc003970 0x7fa71c156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa721400640 1 -- 172.21.15.26:0/4039583007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c1f4340 con 0x7fa71c15c5f0 2024-01-15T17:56:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa721e00640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc003970 0x7fa71c156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa721400640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa71c15c5f0 0x7fa71c15ca70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa70400d830 tx=0x7fa70400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa71a000640 1 -- 172.21.15.26:0/4039583007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70400dee0 con 0x7fa71c15c5f0 2024-01-15T17:56:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa71a000640 1 -- 172.21.15.26:0/4039583007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7040074e0 con 0x7fa71c15c5f0 2024-01-15T17:56:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa71a000640 1 -- 172.21.15.26:0/4039583007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040049e0 con 0x7fa71c15c5f0 2024-01-15T17:56:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71c1f4620 con 0x7fa71c15c5f0 2024-01-15T17:56:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.743+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c158160 con 0x7fa71c15c5f0 2024-01-15T17:56:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.744+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71c0675d0 con 0x7fa71c15c5f0 2024-01-15T17:56:04.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.746+0000 7fa71a000640 1 -- 172.21.15.26:0/4039583007 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa704004b40 con 0x7fa71c15c5f0 2024-01-15T17:56:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.746+0000 7fa71a000640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa6f406b2f0 0x7fa6f406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:04.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.746+0000 7fa71a000640 1 -- 172.21.15.26:0/4039583007 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa7040895c0 con 0x7fa71c15c5f0 2024-01-15T17:56:04.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.747+0000 7fa721e00640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa6f406b2f0 0x7fa6f406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:04.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.748+0000 7fa721e00640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa6f406b2f0 0x7fa6f406d7b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00ba00 tx=0x7fa70c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:04.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.751+0000 7fa71a000640 1 -- 172.21.15.26:0/4039583007 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa704054300 con 0x7fa71c15c5f0 2024-01-15T17:56:04.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.952+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa71c0b33d0 con 0x7fa71c15c5f0 2024-01-15T17:56:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.955+0000 7fa71a000640 1 -- 172.21.15.26:0/4039583007 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7fa704057920 con 0x7fa71c15c5f0 2024-01-15T17:56:04.959 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:04.959 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.957+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa6f406b2f0 msgr2=0x7fa6f406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.957+0000 7fa724800640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa6f406b2f0 0x7fa6f406d7b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00ba00 tx=0x7fa70c03d040 comp rx=0 tx=0).stop 2024-01-15T17:56:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.957+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa71c15c5f0 msgr2=0x7fa71c15ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.957+0000 7fa724800640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa71c15c5f0 0x7fa71c15ca70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa70400d830 tx=0x7fa70400dcf0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.958+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 shutdown_connections 2024-01-15T17:56:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.958+0000 7fa724800640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa71c157770 0x7fa71c157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.958+0000 7fa724800640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa6f406b2f0 0x7fa6f406d7b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.958+0000 7fa724800640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa71c15c5f0 0x7fa71c15ca70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.958+0000 7fa724800640 1 --2- 172.21.15.26:0/4039583007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6fc003970 0x7fa71c156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:04.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.958+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 >> 172.21.15.26:0/4039583007 conn(0x7fa71c06a800 msgr2=0x7fa71c160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:04.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.958+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 shutdown_connections 2024-01-15T17:56:04.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:04.958+0000 7fa724800640 1 -- 172.21.15.26:0/4039583007 wait complete. 2024-01-15T17:56:04.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:56:04.980 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:56:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.207+0000 7f96c1a00640 1 Processor -- start 2024-01-15T17:56:05.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.207+0000 7f96c1a00640 1 -- start start 2024-01-15T17:56:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.209+0000 7f96c1a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 0x7f96bc070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.209+0000 7f96c1a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f96bc0ccb50 0x7f96bc143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.209+0000 7f96c1a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc15f4e0 con 0x7f96bc15bf10 2024-01-15T17:56:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.209+0000 7f96c1a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc15f970 con 0x7f96bc0ccb50 2024-01-15T17:56:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.209+0000 7f96c1a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc071130 con 0x7f96bc0adf20 2024-01-15T17:56:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.210+0000 7f96c1000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f96bc0ccb50 0x7f96bc143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.210+0000 7f96c1000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f96bc0ccb50 0x7f96bc143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59586/0 (socket says 172.21.15.26:59586) 2024-01-15T17:56:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.210+0000 7f96c1000640 1 -- 172.21.15.26:0/1873777415 learned_addr learned my addr 172.21.15.26:0/1873777415 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.210+0000 7f96bbe00640 1 --2- 172.21.15.26:0/1873777415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 0x7f96bc070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.210+0000 7f96c1000640 1 -- 172.21.15.26:0/1873777415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 msgr2=0x7f96bc070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.210+0000 7f96c1000640 1 --2- 172.21.15.26:0/1873777415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 0x7f96bc070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.210+0000 7f96c1000640 1 -- 172.21.15.26:0/1873777415 >> v1:172.21.15.26:6789/0 conn(0x7f96bc15bf10 legacy=0x7f96bc0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.210+0000 7f96c1000640 1 -- 172.21.15.26:0/1873777415 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc15c390 con 0x7f96bc0ccb50 2024-01-15T17:56:05.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.211+0000 7f96c1000640 1 --2- 172.21.15.26:0/1873777415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc0ccb50 0x7f96bc143d30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e50 tx=0x7f96b402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3f9b3736d2b05c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:05.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.212+0000 7f96bb400640 1 -- 172.21.15.26:0/1873777415 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b402c9e0 con 0x7f96bc0ccb50 2024-01-15T17:56:05.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.212+0000 7f96bb400640 1 -- 172.21.15.26:0/1873777415 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b402cb40 con 0x7f96bc0ccb50 2024-01-15T17:56:05.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.212+0000 7f96bb400640 1 -- 172.21.15.26:0/1873777415 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b40343f0 con 0x7f96bc0ccb50 2024-01-15T17:56:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.212+0000 7f96c1a00640 1 -- 172.21.15.26:0/1873777415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc0ccb50 msgr2=0x7f96bc143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.213+0000 7f96c1a00640 1 --2- 172.21.15.26:0/1873777415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc0ccb50 0x7f96bc143d30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e50 tx=0x7f96b402be40 comp rx=0 tx=0).stop 2024-01-15T17:56:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.213+0000 7f96c1a00640 1 -- 172.21.15.26:0/1873777415 shutdown_connections 2024-01-15T17:56:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.213+0000 7f96c1a00640 1 --2- 172.21.15.26:0/1873777415 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc0ccb50 0x7f96bc143d30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.213+0000 7f96c1a00640 1 --2- 172.21.15.26:0/1873777415 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 0x7f96bc070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.213+0000 7f96c1a00640 1 -- 172.21.15.26:0/1873777415 >> 172.21.15.26:0/1873777415 conn(0x7f96bc06a7d0 msgr2=0x7f96bc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.213+0000 7f96c1a00640 1 -- 172.21.15.26:0/1873777415 shutdown_connections 2024-01-15T17:56:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.213+0000 7f96c1a00640 1 -- 172.21.15.26:0/1873777415 wait complete. 2024-01-15T17:56:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.215+0000 7f96c1a00640 1 Processor -- start 2024-01-15T17:56:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.215+0000 7f96c1a00640 1 -- start start 2024-01-15T17:56:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 0x7f96bc13fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc0ccb50 0x7f96bc15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc15bf10 0x7f96bc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc13d5a0 con 0x7f96bc15bf10 2024-01-15T17:56:05.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0b2660 con 0x7f96bc0ccb50 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc13d2c0 con 0x7f96bc0adf20 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc15bf10 0x7f96bc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc15bf10 0x7f96bc0ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59592/0 (socket says 172.21.15.26:59592) 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1000640 1 -- 172.21.15.26:0/1871834464 learned_addr learned my addr 172.21.15.26:0/1871834464 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96bbe00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 0x7f96bc13fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96b3e00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc0ccb50 0x7f96bc15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1000640 1 -- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 msgr2=0x7f96bc13fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1000640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 0x7f96bc13fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1000640 1 -- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc0ccb50 msgr2=0x7f96bc15a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1000640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc0ccb50 0x7f96bc15a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.216+0000 7f96c1000640 1 -- 172.21.15.26:0/1871834464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4009b30 con 0x7f96bc15bf10 2024-01-15T17:56:05.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.217+0000 7f96b3e00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc0ccb50 0x7f96bc15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.217+0000 7f96c1000640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc15bf10 0x7f96bc0ac8d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f96b4036040 tx=0x7f96b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.217+0000 7f96b8c00640 1 -- 172.21.15.26:0/1871834464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4009510 con 0x7f96bc15bf10 2024-01-15T17:56:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.217+0000 7f96b8c00640 1 -- 172.21.15.26:0/1871834464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b403e450 con 0x7f96bc15bf10 2024-01-15T17:56:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.217+0000 7f96b8c00640 1 -- 172.21.15.26:0/1871834464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b403e7e0 con 0x7f96bc15bf10 2024-01-15T17:56:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.217+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc0b2010 con 0x7f96bc15bf10 2024-01-15T17:56:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.217+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc15ada0 con 0x7f96bc15bf10 2024-01-15T17:56:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.219+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc0ce1a0 con 0x7f96bc15bf10 2024-01-15T17:56:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.220+0000 7f96b8c00640 1 -- 172.21.15.26:0/1871834464 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f96b403e940 con 0x7f96bc15bf10 2024-01-15T17:56:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.220+0000 7f96b8c00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f969406b2c0 0x7f969406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.221+0000 7f96b8c00640 1 -- 172.21.15.26:0/1871834464 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f96b40ac9f0 con 0x7f96bc15bf10 2024-01-15T17:56:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.221+0000 7f96bbe00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f969406b2c0 0x7f969406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.222+0000 7f96bbe00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f969406b2c0 0x7f969406d780 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f96ac006fd0 tx=0x7f96ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:05.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.228+0000 7f96b8c00640 1 -- 172.21.15.26:0/1871834464 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96b40777d0 con 0x7f96bc15bf10 2024-01-15T17:56:05.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.424+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96bc0b27d0 con 0x7f96bc15bf10 2024-01-15T17:56:05.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.425+0000 7f96b8c00640 1 -- 172.21.15.26:0/1871834464 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96b40430a0 con 0x7f96bc15bf10 2024-01-15T17:56:05.428 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:56:05.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f969406b2c0 msgr2=0x7f969406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f969406b2c0 0x7f969406d780 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f96ac006fd0 tx=0x7f96ac008040 comp rx=0 tx=0).stop 2024-01-15T17:56:05.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc15bf10 msgr2=0x7f96bc0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc15bf10 0x7f96bc0ac8d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f96b4036040 tx=0x7f96b4004a40 comp rx=0 tx=0).stop 2024-01-15T17:56:05.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 shutdown_connections 2024-01-15T17:56:05.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f969406b2c0 0x7f969406d780 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96bc15bf10 0x7f96bc0ac8d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96bc0ccb50 0x7f96bc15a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 --2- 172.21.15.26:0/1871834464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96bc0adf20 0x7f96bc13fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 >> 172.21.15.26:0/1871834464 conn(0x7f96bc06a7d0 msgr2=0x7f96bc06c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:05.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 shutdown_connections 2024-01-15T17:56:05.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.428+0000 7f96c1a00640 1 -- 172.21.15.26:0/1871834464 wait complete. 2024-01-15T17:56:05.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:56:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.672+0000 7faebf200640 1 Processor -- start 2024-01-15T17:56:05.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.673+0000 7faebf200640 1 -- start start 2024-01-15T17:56:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.674+0000 7faebf200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faec0143810 0x7faec0071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.674+0000 7faebf200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec00ae620 con 0x7faec00ae170 2024-01-15T17:56:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.674+0000 7faebf200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec006c730 con 0x7faec015d770 2024-01-15T17:56:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.674+0000 7faebf200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec0143bf0 con 0x7faec0143810 2024-01-15T17:56:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.675+0000 7faebd400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faec00ae170 0x7faec015e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51290/0 (socket says 172.21.15.26:51290) 2024-01-15T17:56:05.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.675+0000 7faebd400640 1 -- 172.21.15.26:0/3838051002 learned_addr learned my addr 172.21.15.26:0/3838051002 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.675+0000 7faebde00640 1 --2- 172.21.15.26:0/3838051002 >> v2:172.21.15.124:3300/0 conn(0x7faec0143810 0x7faec0071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.675+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723978476 0 0) 0x7faec00ae620 con 0x7faec00ae170 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.675+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea0003520 con 0x7faec00ae170 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2779634494 0 0) 0x7faea0003520 con 0x7faec00ae170 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 >> v2:172.21.15.124:3300/0 conn(0x7faec0143810 msgr2=0x7faec0071e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 --2- 172.21.15.26:0/3838051002 >> v2:172.21.15.124:3300/0 conn(0x7faec0143810 0x7faec0071e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 >> v1:172.21.15.124:6789/0 conn(0x7faec015d770 legacy=0x7faec015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec00724d0 con 0x7faec00ae170 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7faea8001670 con 0x7faec00ae170 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 >> v1:172.21.15.26:6789/0 conn(0x7faec00ae170 legacy=0x7faec015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faea0003970 0x7faea0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea00047c0 0x7faea0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea00052a0 0x7faea000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec0143bf0 con 0x7faea0003970 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea000c170 con 0x7faea00052a0 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea000c330 con 0x7faea00047c0 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebde00640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faea0003970 0x7faea0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebd400640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea00047c0 0x7faea0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.676+0000 7faebe800640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea00052a0 0x7faea000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.677+0000 7faebde00640 1 -- 172.21.15.26:0/3838051002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea00047c0 msgr2=0x7faea0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.677+0000 7faebde00640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faea00047c0 0x7faea0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.677+0000 7faebde00640 1 -- 172.21.15.26:0/3838051002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea00052a0 msgr2=0x7faea000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.677+0000 7faebde00640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea00052a0 0x7faea000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.677+0000 7faebde00640 1 -- 172.21.15.26:0/3838051002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea0003520 con 0x7faea0003970 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.677+0000 7faebe800640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea00052a0 0x7faea000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:05.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.677+0000 7faebe800640 1 -- 172.21.15.26:0/3838051002 reap_dead start 2024-01-15T17:56:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.678+0000 7faebde00640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faea0003970 0x7faea0003d90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009a90 tx=0x7faea00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ac506e66f98b5d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:05.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.678+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb002c680 con 0x7faea0003970 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.678+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb002c7e0 con 0x7faea0003970 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.678+0000 7faebca00640 1 -- 172.21.15.26:0/3838051002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb002cb70 con 0x7faea0003970 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.679+0000 7faebf200640 1 -- 172.21.15.26:0/3838051002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faea0003970 msgr2=0x7faea0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.679+0000 7faebf200640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faea0003970 0x7faea0003d90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009a90 tx=0x7faea00057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.679+0000 7faebf200640 1 -- 172.21.15.26:0/3838051002 shutdown_connections 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.679+0000 7faebf200640 1 --2- 172.21.15.26:0/3838051002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faea0003970 0x7faea0003d90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.679+0000 7faebf200640 1 -- 172.21.15.26:0/3838051002 >> 172.21.15.26:0/3838051002 conn(0x7faec006a7d0 msgr2=0x7faec006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.679+0000 7faebf200640 1 -- 172.21.15.26:0/3838051002 shutdown_connections 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.679+0000 7faebf200640 1 -- 172.21.15.26:0/3838051002 wait complete. 2024-01-15T17:56:05.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.681+0000 7faebf200640 1 Processor -- start 2024-01-15T17:56:05.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.681+0000 7faebf200640 1 -- start start 2024-01-15T17:56:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebf200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea0003970 0x7faec01f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebf200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec00ae9e0 0x7faec00aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebf200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faec01f2cc0 0x7faec01f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebf200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec00af3a0 con 0x7faec01f2cc0 2024-01-15T17:56:05.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebf200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec01f3c90 con 0x7faea0003970 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebf200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec01f39b0 con 0x7faec00ae9e0 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faec01f2cc0 0x7faec01f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faec01f2cc0 0x7faec01f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59614/0 (socket says 172.21.15.26:59614) 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebe800640 1 -- 172.21.15.26:0/4080039090 learned_addr learned my addr 172.21.15.26:0/4080039090 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebd400640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec00ae9e0 0x7faec00aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.682+0000 7faebde00640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea0003970 0x7faec01f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebe800640 1 -- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec00ae9e0 msgr2=0x7faec00aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebe800640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec00ae9e0 0x7faec00aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebe800640 1 -- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea0003970 msgr2=0x7faec01f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebe800640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea0003970 0x7faec01f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebe800640 1 -- 172.21.15.26:0/4080039090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec00cc740 con 0x7faec01f2cc0 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebd400640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec00ae9e0 0x7faec00aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebde00640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea0003970 0x7faec01f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebe800640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faec01f2cc0 0x7faec01f3120 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faec00ad100 tx=0x7faeb800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faeb6000640 1 -- 172.21.15.26:0/4080039090 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8003d80 con 0x7faec01f2cc0 2024-01-15T17:56:05.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faeb6000640 1 -- 172.21.15.26:0/4080039090 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb80050f0 con 0x7faec01f2cc0 2024-01-15T17:56:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faeb6000640 1 -- 172.21.15.26:0/4080039090 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80054a0 con 0x7faec01f2cc0 2024-01-15T17:56:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec00cca20 con 0x7faec01f2cc0 2024-01-15T17:56:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.683+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec01f3660 con 0x7faec01f2cc0 2024-01-15T17:56:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.685+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec00ccf40 con 0x7faec01f2cc0 2024-01-15T17:56:05.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.685+0000 7faeb6000640 1 -- 172.21.15.26:0/4080039090 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7faeb8005600 con 0x7faec01f2cc0 2024-01-15T17:56:05.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.686+0000 7faeb6000640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fae9806b2f0 0x7fae9806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:05.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.686+0000 7faeb6000640 1 -- 172.21.15.26:0/4080039090 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7faeb8089610 con 0x7faec01f2cc0 2024-01-15T17:56:05.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.687+0000 7faebde00640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fae9806b2f0 0x7fae9806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:05.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.688+0000 7faebde00640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fae9806b2f0 0x7fae9806d7b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faeb0036000 tx=0x7faeb0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:05.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.691+0000 7faeb6000640 1 -- 172.21.15.26:0/4080039090 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faec00ccf40 con 0x7faec01f2cc0 2024-01-15T17:56:05.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.885+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faec00ccf40 con 0x7faec01f2cc0 2024-01-15T17:56:05.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.886+0000 7faeb6000640 1 -- 172.21.15.26:0/4080039090 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7faec00ccf40 con 0x7faec01f2cc0 2024-01-15T17:56:05.889 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:56:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fae9806b2f0 msgr2=0x7fae9806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fae9806b2f0 0x7fae9806d7b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faeb0036000 tx=0x7faeb0030000 comp rx=0 tx=0).stop 2024-01-15T17:56:05.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faec01f2cc0 msgr2=0x7faec01f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faec01f2cc0 0x7faec01f3120 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faec00ad100 tx=0x7faeb800d9f0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 shutdown_connections 2024-01-15T17:56:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faec00ae9e0 0x7faec00aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fae9806b2f0 0x7fae9806d7b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faec01f2cc0 0x7faec01f3120 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 --2- 172.21.15.26:0/4080039090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faea0003970 0x7faec01f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 >> 172.21.15.26:0/4080039090 conn(0x7faec006a7d0 msgr2=0x7faec00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 shutdown_connections 2024-01-15T17:56:05.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:05.889+0000 7faebf200640 1 -- 172.21.15.26:0/4080039090 wait complete. 2024-01-15T17:56:05.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:56:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.133+0000 7f344b000640 1 Processor -- start 2024-01-15T17:56:06.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.133+0000 7f344b000640 1 -- start start 2024-01-15T17:56:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.134+0000 7f344b000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34440c9870 0x7f34440ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.134+0000 7f344b000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34440d0560 0x7f3444161030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f344b000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34440cf4c0 con 0x7f34440d2260 2024-01-15T17:56:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f344b000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34440d2ef0 con 0x7f34440c9870 2024-01-15T17:56:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f344b000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34440c9c50 con 0x7f34440d0560 2024-01-15T17:56:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f3443e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34440c9870 0x7f34440ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f3449000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f34440d2260 0x7f34440d0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51304/0 (socket says 172.21.15.26:51304) 2024-01-15T17:56:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f3449000640 1 -- 172.21.15.26:0/3142236293 learned_addr learned my addr 172.21.15.26:0/3142236293 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:06.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f3443400640 1 --2- 172.21.15.26:0/3142236293 >> v2:172.21.15.124:3300/0 conn(0x7f34440d0560 0x7f3444161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f3443e00640 1 -- 172.21.15.26:0/3142236293 >> v2:172.21.15.124:3300/0 conn(0x7f34440d0560 msgr2=0x7f3444161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:06.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f3443e00640 1 --2- 172.21.15.26:0/3142236293 >> v2:172.21.15.124:3300/0 conn(0x7f34440d0560 0x7f3444161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.135+0000 7f3443e00640 1 -- 172.21.15.26:0/3142236293 >> v1:172.21.15.26:6789/0 conn(0x7f34440d2260 legacy=0x7f34440d0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.136+0000 7f3443e00640 1 -- 172.21.15.26:0/3142236293 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34440cf7a0 con 0x7f34440c9870 2024-01-15T17:56:06.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.136+0000 7f3442a00640 1 -- 172.21.15.26:0/3142236293 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049220499 0 0) 0x7f34440cf4c0 con 0x7f34440d2260 2024-01-15T17:56:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.137+0000 7f3443e00640 1 --2- 172.21.15.26:0/3142236293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 0x7f34440ceb50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f34440b5de0 tx=0x7f343402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d66d617df48ed38 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f3442a00640 1 -- 172.21.15.26:0/3142236293 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343402cc60 con 0x7f34440c9870 2024-01-15T17:56:06.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f3442a00640 1 -- 172.21.15.26:0/3142236293 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343402cdc0 con 0x7f34440c9870 2024-01-15T17:56:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f3442a00640 1 -- 172.21.15.26:0/3142236293 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434034660 con 0x7f34440c9870 2024-01-15T17:56:06.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f344b000640 1 -- 172.21.15.26:0/3142236293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 msgr2=0x7f34440ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f344b000640 1 --2- 172.21.15.26:0/3142236293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 0x7f34440ceb50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f34440b5de0 tx=0x7f343402beb0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f344b000640 1 -- 172.21.15.26:0/3142236293 shutdown_connections 2024-01-15T17:56:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f344b000640 1 --2- 172.21.15.26:0/3142236293 >> v2:172.21.15.124:3300/0 conn(0x7f34440d0560 0x7f3444161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f344b000640 1 --2- 172.21.15.26:0/3142236293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 0x7f34440ceb50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f344b000640 1 -- 172.21.15.26:0/3142236293 >> 172.21.15.26:0/3142236293 conn(0x7f344409c560 msgr2=0x7f34440d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f344b000640 1 -- 172.21.15.26:0/3142236293 shutdown_connections 2024-01-15T17:56:06.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.138+0000 7f344b000640 1 -- 172.21.15.26:0/3142236293 wait complete. 2024-01-15T17:56:06.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.140+0000 7f344b000640 1 Processor -- start 2024-01-15T17:56:06.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.140+0000 7f344b000640 1 -- start start 2024-01-15T17:56:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.141+0000 7f344b000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 0x7f34440cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.141+0000 7f344b000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34440d0560 0x7f344415a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.141+0000 7f344b000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34440d2260 0x7f344414f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.141+0000 7f344b000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34440b66e0 con 0x7f34440c9870 2024-01-15T17:56:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.141+0000 7f344b000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34440cc360 con 0x7f34440d0560 2024-01-15T17:56:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.141+0000 7f344b000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34440cc640 con 0x7f34440d2260 2024-01-15T17:56:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3443e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 0x7f34440cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3443e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 0x7f34440cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59630/0 (socket says 172.21.15.26:59630) 2024-01-15T17:56:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3443e00640 1 -- 172.21.15.26:0/539258912 learned_addr learned my addr 172.21.15.26:0/539258912 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3443400640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34440d0560 0x7f344415a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3449000640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34440d2260 0x7f344414f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3443e00640 1 -- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34440d2260 msgr2=0x7f344414f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3443e00640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34440d2260 0x7f344414f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3443e00640 1 -- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34440d0560 msgr2=0x7f344415a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3443e00640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34440d0560 0x7f344415a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.142+0000 7f3443e00640 1 -- 172.21.15.26:0/539258912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434009a70 con 0x7f34440c9870 2024-01-15T17:56:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.143+0000 7f3443e00640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 0x7f34440cbd80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3434009d90 tx=0x7f3434004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.143+0000 7f343be00640 1 -- 172.21.15.26:0/539258912 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343403e610 con 0x7f34440c9870 2024-01-15T17:56:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.143+0000 7f343be00640 1 -- 172.21.15.26:0/539258912 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343403e790 con 0x7f34440c9870 2024-01-15T17:56:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.143+0000 7f343be00640 1 -- 172.21.15.26:0/539258912 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343403eb40 con 0x7f34440c9870 2024-01-15T17:56:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.143+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34440cc920 con 0x7f34440c9870 2024-01-15T17:56:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.143+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34440ccc00 con 0x7f34440c9870 2024-01-15T17:56:06.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.144+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34440b68b0 con 0x7f34440c9870 2024-01-15T17:56:06.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.145+0000 7f343be00640 1 -- 172.21.15.26:0/539258912 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f343403b050 con 0x7f34440c9870 2024-01-15T17:56:06.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.146+0000 7f343be00640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f341c06b2c0 0x7f341c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.146+0000 7f343be00640 1 -- 172.21.15.26:0/539258912 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f34340abff0 con 0x7f34440c9870 2024-01-15T17:56:06.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.147+0000 7f3443400640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f341c06b2c0 0x7f341c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.148+0000 7f3443400640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f341c06b2c0 0x7f341c06d780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f34441f67d0 tx=0x7f342c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:06.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.150+0000 7f343be00640 1 -- 172.21.15.26:0/539258912 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34340dee90 con 0x7f34440c9870 2024-01-15T17:56:06.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.349+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34440675d0 con 0x7f34440c9870 2024-01-15T17:56:06.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.350+0000 7f343be00640 1 -- 172.21.15.26:0/539258912 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f343404d0a0 con 0x7f34440c9870 2024-01-15T17:56:06.354 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:56:06.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.353+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f341c06b2c0 msgr2=0x7f341c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.353+0000 7f344b000640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f341c06b2c0 0x7f341c06d780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f34441f67d0 tx=0x7f342c008040 comp rx=0 tx=0).stop 2024-01-15T17:56:06.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.353+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 msgr2=0x7f34440cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.353+0000 7f344b000640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 0x7f34440cbd80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3434009d90 tx=0x7f3434004a40 comp rx=0 tx=0).stop 2024-01-15T17:56:06.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.354+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 shutdown_connections 2024-01-15T17:56:06.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.354+0000 7f344b000640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34440d2260 0x7f344414f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.354+0000 7f344b000640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f341c06b2c0 0x7f341c06d780 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.354+0000 7f344b000640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34440d0560 0x7f344415a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.354+0000 7f344b000640 1 --2- 172.21.15.26:0/539258912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34440c9870 0x7f34440cbd80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.354+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 >> 172.21.15.26:0/539258912 conn(0x7f344409c560 msgr2=0x7f344406c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:06.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.354+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 shutdown_connections 2024-01-15T17:56:06.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.354+0000 7f344b000640 1 -- 172.21.15.26:0/539258912 wait complete. 2024-01-15T17:56:06.376 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:56:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.597+0000 7f07c2a00640 1 Processor -- start 2024-01-15T17:56:06.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.597+0000 7f07c2a00640 1 -- start start 2024-01-15T17:56:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.599+0000 7f07c2a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cc500 0x7f07bc0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.599+0000 7f07c2a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07bc06d160 0x7f07bc06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.599+0000 7f07c2a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07bc070e50 con 0x7f07bc143c40 2024-01-15T17:56:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.599+0000 7f07c2a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc0723c0 con 0x7f07bc06d160 2024-01-15T17:56:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.599+0000 7f07c2a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07bc0cced0 con 0x7f07bc0cc500 2024-01-15T17:56:06.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.599+0000 7f07c2000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07bc06d160 0x7f07bc06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.600+0000 7f07c2000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07bc06d160 0x7f07bc06d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59646/0 (socket says 172.21.15.26:59646) 2024-01-15T17:56:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.600+0000 7f07c2000640 1 -- 172.21.15.26:0/991353818 learned_addr learned my addr 172.21.15.26:0/991353818 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.600+0000 7f07c0c00640 1 --2- 172.21.15.26:0/991353818 >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cc500 0x7f07bc0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.600+0000 7f07c2000640 1 -- 172.21.15.26:0/991353818 >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cc500 msgr2=0x7f07bc0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.600+0000 7f07c2000640 1 --2- 172.21.15.26:0/991353818 >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cc500 0x7f07bc0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.600+0000 7f07c2000640 1 -- 172.21.15.26:0/991353818 >> v1:172.21.15.124:6789/0 conn(0x7f07bc143c40 legacy=0x7f07bc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:06.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.600+0000 7f07c2000640 1 -- 172.21.15.26:0/991353818 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc144020 con 0x7f07bc06d160 2024-01-15T17:56:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.601+0000 7f07c2000640 1 --2- 172.21.15.26:0/991353818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc06d160 0x7f07bc06d560 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0b44a0 tx=0x7f07b402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448730ccfd0e4f26 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.602+0000 7f07bbe00640 1 -- 172.21.15.26:0/991353818 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b402c9f0 con 0x7f07bc06d160 2024-01-15T17:56:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.602+0000 7f07bbe00640 1 -- 172.21.15.26:0/991353818 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b402cb50 con 0x7f07bc06d160 2024-01-15T17:56:06.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.602+0000 7f07bbe00640 1 -- 172.21.15.26:0/991353818 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b40343f0 con 0x7f07bc06d160 2024-01-15T17:56:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.602+0000 7f07c2a00640 1 -- 172.21.15.26:0/991353818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc06d160 msgr2=0x7f07bc06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.602+0000 7f07c2a00640 1 --2- 172.21.15.26:0/991353818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc06d160 0x7f07bc06d560 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0b44a0 tx=0x7f07b402be50 comp rx=0 tx=0).stop 2024-01-15T17:56:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.602+0000 7f07c2a00640 1 -- 172.21.15.26:0/991353818 shutdown_connections 2024-01-15T17:56:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.602+0000 7f07c2a00640 1 --2- 172.21.15.26:0/991353818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc06d160 0x7f07bc06d560 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.602+0000 7f07c2a00640 1 --2- 172.21.15.26:0/991353818 >> v2:172.21.15.124:3300/0 conn(0x7f07bc0cc500 0x7f07bc0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.602+0000 7f07c2a00640 1 -- 172.21.15.26:0/991353818 >> 172.21.15.26:0/991353818 conn(0x7f07bc06a800 msgr2=0x7f07bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.603+0000 7f07c2a00640 1 -- 172.21.15.26:0/991353818 shutdown_connections 2024-01-15T17:56:06.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.603+0000 7f07c2a00640 1 -- 172.21.15.26:0/991353818 wait complete. 2024-01-15T17:56:06.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.604+0000 7f07c2a00640 1 Processor -- start 2024-01-15T17:56:06.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.605+0000 7f07c2a00640 1 -- start start 2024-01-15T17:56:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c2a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc06d160 0x7f07bc15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc0cc500 0x7f07bc1494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c2a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc143c40 0x7f07bc14b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c2a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc15a980 con 0x7f07bc0cc500 2024-01-15T17:56:06.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c2a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc15ade0 con 0x7f07bc06d160 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c2a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07bc1f2950 con 0x7f07bc143c40 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc0cc500 0x7f07bc1494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc0cc500 0x7f07bc1494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59662/0 (socket says 172.21.15.26:59662) 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c0c00640 1 -- 172.21.15.26:0/2751226604 learned_addr learned my addr 172.21.15.26:0/2751226604 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c1600640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc06d160 0x7f07bc15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c0c00640 1 -- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc143c40 msgr2=0x7f07bc14b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c2000640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc143c40 0x7f07bc14b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c0c00640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc143c40 0x7f07bc14b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c0c00640 1 -- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc06d160 msgr2=0x7f07bc15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c0c00640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc06d160 0x7f07bc15a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.606+0000 7f07c0c00640 1 -- 172.21.15.26:0/2751226604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b4009b30 con 0x7f07bc0cc500 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.607+0000 7f07c0c00640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc0cc500 0x7f07bc1494b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f07a400d800 tx=0x7f07a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.607+0000 7f07b9600640 1 -- 172.21.15.26:0/2751226604 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400b6c0 con 0x7f07bc0cc500 2024-01-15T17:56:06.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.607+0000 7f07b9600640 1 -- 172.21.15.26:0/2751226604 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a400b820 con 0x7f07bc0cc500 2024-01-15T17:56:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.607+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07bc14c0b0 con 0x7f07bc0cc500 2024-01-15T17:56:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.607+0000 7f07b9600640 1 -- 172.21.15.26:0/2751226604 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400bbb0 con 0x7f07bc0cc500 2024-01-15T17:56:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.607+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07bc13a1e0 con 0x7f07bc0cc500 2024-01-15T17:56:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.608+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07bc067450 con 0x7f07bc0cc500 2024-01-15T17:56:06.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.609+0000 7f07b9600640 1 -- 172.21.15.26:0/2751226604 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f07a400bd10 con 0x7f07bc0cc500 2024-01-15T17:56:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.611+0000 7f07b9600640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079406b2c0 0x7f079406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.611+0000 7f07b9600640 1 -- 172.21.15.26:0/2751226604 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f07a408a260 con 0x7f07bc0cc500 2024-01-15T17:56:06.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.611+0000 7f07c1600640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079406b2c0 0x7f079406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:06.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.612+0000 7f07c1600640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079406b2c0 0x7f079406d780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f07ac006fd0 tx=0x7f07ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:06.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.616+0000 7f07b9600640 1 -- 172.21.15.26:0/2751226604 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07a4054fa0 con 0x7f07bc0cc500 2024-01-15T17:56:06.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.841+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f07bc0ceb30 con 0x7f07bc0cc500 2024-01-15T17:56:06.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.842+0000 7f07b9600640 1 -- 172.21.15.26:0/2751226604 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f07a40585c0 con 0x7f07bc0cc500 2024-01-15T17:56:06.845 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079406b2c0 msgr2=0x7f079406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079406b2c0 0x7f079406d780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f07ac006fd0 tx=0x7f07ac008040 comp rx=0 tx=0).stop 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc0cc500 msgr2=0x7f07bc1494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc0cc500 0x7f07bc1494b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f07a400d800 tx=0x7f07a400dcc0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 shutdown_connections 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07bc143c40 0x7f07bc14b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079406b2c0 0x7f079406d780 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07bc0cc500 0x7f07bc1494b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 --2- 172.21.15.26:0/2751226604 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07bc06d160 0x7f07bc15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.845+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 >> 172.21.15.26:0/2751226604 conn(0x7f07bc06a800 msgr2=0x7f07bc071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.846+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 shutdown_connections 2024-01-15T17:56:06.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:06.846+0000 7f07c2a00640 1 -- 172.21.15.26:0/2751226604 wait complete. 2024-01-15T17:56:06.869 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:56:06.869 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:56:06.917 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.145+0000 7fa422000640 1 Processor -- start 2024-01-15T17:56:07.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.145+0000 7fa422000640 1 -- start start 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.147+0000 7fa422000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa41c143c40 0x7fa41c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.147+0000 7fa422000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa41c0717d0 0x7fa41c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.147+0000 7fa422000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa41c070e50 con 0x7fa41c0cca40 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.147+0000 7fa422000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41c0723c0 con 0x7fa41c143c40 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.147+0000 7fa422000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa41c0cce40 con 0x7fa41c0717d0 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.147+0000 7fa420c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa41c143c40 0x7fa41c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.147+0000 7fa420c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa41c143c40 0x7fa41c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59676/0 (socket says 172.21.15.26:59676) 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.148+0000 7fa420c00640 1 -- 172.21.15.26:0/3652551268 learned_addr learned my addr 172.21.15.26:0/3652551268 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.148+0000 7fa413c00640 1 --2- 172.21.15.26:0/3652551268 >> v2:172.21.15.124:3300/0 conn(0x7fa41c0717d0 0x7fa41c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.148+0000 7fa420c00640 1 -- 172.21.15.26:0/3652551268 >> v2:172.21.15.124:3300/0 conn(0x7fa41c0717d0 msgr2=0x7fa41c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:07.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.148+0000 7fa420c00640 1 --2- 172.21.15.26:0/3652551268 >> v2:172.21.15.124:3300/0 conn(0x7fa41c0717d0 0x7fa41c0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.148+0000 7fa420c00640 1 -- 172.21.15.26:0/3652551268 >> v1:172.21.15.124:6789/0 conn(0x7fa41c0cca40 legacy=0x7fa41c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:07.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.148+0000 7fa420c00640 1 -- 172.21.15.26:0/3652551268 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c06e170 con 0x7fa41c143c40 2024-01-15T17:56:07.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.149+0000 7fa420c00640 1 --2- 172.21.15.26:0/3652551268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c143c40 0x7fa41c071290 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0b44a0 tx=0x7fa40c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c73b486ceea653 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:07.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.149+0000 7fa41be00640 1 -- 172.21.15.26:0/3652551268 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0317e0 con 0x7fa41c143c40 2024-01-15T17:56:07.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.149+0000 7fa41be00640 1 -- 172.21.15.26:0/3652551268 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c031940 con 0x7fa41c143c40 2024-01-15T17:56:07.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.149+0000 7fa41be00640 1 -- 172.21.15.26:0/3652551268 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c031cd0 con 0x7fa41c143c40 2024-01-15T17:56:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.150+0000 7fa422000640 1 -- 172.21.15.26:0/3652551268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c143c40 msgr2=0x7fa41c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.150+0000 7fa422000640 1 --2- 172.21.15.26:0/3652551268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c143c40 0x7fa41c071290 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0b44a0 tx=0x7fa40c02b880 comp rx=0 tx=0).stop 2024-01-15T17:56:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.150+0000 7fa422000640 1 -- 172.21.15.26:0/3652551268 shutdown_connections 2024-01-15T17:56:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.150+0000 7fa422000640 1 --2- 172.21.15.26:0/3652551268 >> v2:172.21.15.124:3300/0 conn(0x7fa41c0717d0 0x7fa41c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.150+0000 7fa422000640 1 --2- 172.21.15.26:0/3652551268 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c143c40 0x7fa41c071290 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.150+0000 7fa422000640 1 -- 172.21.15.26:0/3652551268 >> 172.21.15.26:0/3652551268 conn(0x7fa41c06a800 msgr2=0x7fa41c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.150+0000 7fa422000640 1 -- 172.21.15.26:0/3652551268 shutdown_connections 2024-01-15T17:56:07.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.150+0000 7fa422000640 1 -- 172.21.15.26:0/3652551268 wait complete. 2024-01-15T17:56:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.152+0000 7fa422000640 1 Processor -- start 2024-01-15T17:56:07.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.152+0000 7fa422000640 1 -- start start 2024-01-15T17:56:07.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.153+0000 7fa422000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c0717d0 0x7fa41c15a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.153+0000 7fa422000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa41c0cca40 0x7fa41c1388d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.153+0000 7fa422000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa41c143c40 0x7fa41c138e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.153+0000 7fa422000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0ada30 con 0x7fa41c0717d0 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.153+0000 7fa422000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0ace20 con 0x7fa41c143c40 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.153+0000 7fa422000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c0ad100 con 0x7fa41c0cca40 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa420c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c0717d0 0x7fa41c15a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa420c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c0717d0 0x7fa41c15a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59690/0 (socket says 172.21.15.26:59690) 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa420c00640 1 -- 172.21.15.26:0/1387649349 learned_addr learned my addr 172.21.15.26:0/1387649349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa413c00640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa41c0cca40 0x7fa41c1388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa421600640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa41c143c40 0x7fa41c138e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa420c00640 1 -- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa41c0cca40 msgr2=0x7fa41c1388d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa420c00640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa41c0cca40 0x7fa41c1388d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa420c00640 1 -- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa41c143c40 msgr2=0x7fa41c138e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa420c00640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa41c143c40 0x7fa41c138e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.154+0000 7fa420c00640 1 -- 172.21.15.26:0/1387649349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0094d0 con 0x7fa41c0717d0 2024-01-15T17:56:07.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.155+0000 7fa420c00640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c0717d0 0x7fa41c15a8a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa40c02b8c0 tx=0x7fa40c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.155+0000 7fa419600640 1 -- 172.21.15.26:0/1387649349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c032880 con 0x7fa41c0717d0 2024-01-15T17:56:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.155+0000 7fa419600640 1 -- 172.21.15.26:0/1387649349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c0329e0 con 0x7fa41c0717d0 2024-01-15T17:56:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.155+0000 7fa419600640 1 -- 172.21.15.26:0/1387649349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c032d90 con 0x7fa41c0717d0 2024-01-15T17:56:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.155+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41c0ad3e0 con 0x7fa41c0717d0 2024-01-15T17:56:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.155+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41c15ade0 con 0x7fa41c0717d0 2024-01-15T17:56:07.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.156+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41c067450 con 0x7fa41c0717d0 2024-01-15T17:56:07.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.157+0000 7fa419600640 1 -- 172.21.15.26:0/1387649349 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa40c04e450 con 0x7fa41c0717d0 2024-01-15T17:56:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.158+0000 7fa419600640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3f406b2c0 0x7fa3f406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.158+0000 7fa419600640 1 -- 172.21.15.26:0/1387649349 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa40c0abdd0 con 0x7fa41c0717d0 2024-01-15T17:56:07.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.159+0000 7fa413c00640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3f406b2c0 0x7fa3f406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.160+0000 7fa413c00640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3f406b2c0 0x7fa3f406d780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa41c138070 tx=0x7fa404003990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:07.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.163+0000 7fa419600640 1 -- 172.21.15.26:0/1387649349 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa40c076bc0 con 0x7fa41c0717d0 2024-01-15T17:56:07.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.359+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa41c0ceb30 con 0x7fa41c0717d0 2024-01-15T17:56:07.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.363+0000 7fa419600640 1 -- 172.21.15.26:0/1387649349 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7fa40c07a1e0 con 0x7fa41c0717d0 2024-01-15T17:56:07.366 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:07.366 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:07.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3f406b2c0 msgr2=0x7fa3f406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3f406b2c0 0x7fa3f406d780 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa41c138070 tx=0x7fa404003990 comp rx=0 tx=0).stop 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c0717d0 msgr2=0x7fa41c15a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c0717d0 0x7fa41c15a8a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa40c02b8c0 tx=0x7fa40c0046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 shutdown_connections 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa41c0cca40 0x7fa41c1388d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa3f406b2c0 0x7fa3f406d780 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa41c143c40 0x7fa41c138e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 --2- 172.21.15.26:0/1387649349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa41c0717d0 0x7fa41c15a8a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.365+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 >> 172.21.15.26:0/1387649349 conn(0x7fa41c06a800 msgr2=0x7fa41c140380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.366+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 shutdown_connections 2024-01-15T17:56:07.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.366+0000 7fa422000640 1 -- 172.21.15.26:0/1387649349 wait complete. 2024-01-15T17:56:07.390 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:56:07.390 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:56:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.621+0000 7f4d0d200640 1 Processor -- start 2024-01-15T17:56:07.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.621+0000 7f4d0d200640 1 -- start start 2024-01-15T17:56:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.623+0000 7f4d0d200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4d0813df80 0x7f4d0815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.623+0000 7f4d0d200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4d080cf9a0 0x7f4d080cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.623+0000 7f4d0d200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0806c470 con 0x7f4d0813e360 2024-01-15T17:56:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.623+0000 7f4d0d200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0806c750 con 0x7f4d080cf9a0 2024-01-15T17:56:07.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.623+0000 7f4d0d200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0806ca30 con 0x7f4d0813df80 2024-01-15T17:56:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.623+0000 7f4d06800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4d080cf9a0 0x7f4d080cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.623+0000 7f4d06800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4d080cf9a0 0x7f4d080cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59696/0 (socket says 172.21.15.26:59696) 2024-01-15T17:56:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.623+0000 7f4d06800640 1 -- 172.21.15.26:0/1082714922 learned_addr learned my addr 172.21.15.26:0/1082714922 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.624+0000 7f4d05e00640 1 --2- 172.21.15.26:0/1082714922 >> v2:172.21.15.124:3300/0 conn(0x7f4d0813df80 0x7f4d0815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.624+0000 7f4d06800640 1 -- 172.21.15.26:0/1082714922 >> v2:172.21.15.124:3300/0 conn(0x7f4d0813df80 msgr2=0x7f4d0815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.624+0000 7f4d06800640 1 --2- 172.21.15.26:0/1082714922 >> v2:172.21.15.124:3300/0 conn(0x7f4d0813df80 0x7f4d0815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.624+0000 7f4d06800640 1 -- 172.21.15.26:0/1082714922 >> v1:172.21.15.26:6789/0 conn(0x7f4d0813e360 legacy=0x7f4d080cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.624+0000 7f4d06800640 1 -- 172.21.15.26:0/1082714922 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0806cd10 con 0x7f4d080cf9a0 2024-01-15T17:56:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.624+0000 7f4d05e00640 1 --2- 172.21.15.26:0/1082714922 >> v2:172.21.15.124:3300/0 conn(0x7f4d0813df80 0x7f4d0815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.625+0000 7f4d06800640 1 --2- 172.21.15.26:0/1082714922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d080cf9a0 0x7f4d080cd840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4d08072770 tx=0x7f4d0002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a179e2e20ec9a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.625+0000 7f4d05400640 1 -- 172.21.15.26:0/1082714922 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0002cb10 con 0x7f4d080cf9a0 2024-01-15T17:56:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.625+0000 7f4d05400640 1 -- 172.21.15.26:0/1082714922 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0002cc70 con 0x7f4d080cf9a0 2024-01-15T17:56:07.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.625+0000 7f4d05400640 1 -- 172.21.15.26:0/1082714922 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d000343f0 con 0x7f4d080cf9a0 2024-01-15T17:56:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.626+0000 7f4d0d200640 1 -- 172.21.15.26:0/1082714922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d080cf9a0 msgr2=0x7f4d080cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.626+0000 7f4d0d200640 1 --2- 172.21.15.26:0/1082714922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d080cf9a0 0x7f4d080cd840 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4d08072770 tx=0x7f4d0002bf70 comp rx=0 tx=0).stop 2024-01-15T17:56:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.626+0000 7f4d0d200640 1 -- 172.21.15.26:0/1082714922 shutdown_connections 2024-01-15T17:56:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.626+0000 7f4d0d200640 1 --2- 172.21.15.26:0/1082714922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d080cf9a0 0x7f4d080cd840 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.626+0000 7f4d0d200640 1 --2- 172.21.15.26:0/1082714922 >> v2:172.21.15.124:3300/0 conn(0x7f4d0813df80 0x7f4d0815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.626+0000 7f4d0d200640 1 -- 172.21.15.26:0/1082714922 >> 172.21.15.26:0/1082714922 conn(0x7f4d0806a800 msgr2=0x7f4d0806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.626+0000 7f4d0d200640 1 -- 172.21.15.26:0/1082714922 shutdown_connections 2024-01-15T17:56:07.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.626+0000 7f4d0d200640 1 -- 172.21.15.26:0/1082714922 wait complete. 2024-01-15T17:56:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.628+0000 7f4d0d200640 1 Processor -- start 2024-01-15T17:56:07.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.628+0000 7f4d0d200640 1 -- start start 2024-01-15T17:56:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.629+0000 7f4d0d200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d080cf9a0 0x7f4d081532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.629+0000 7f4d0d200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d0813df80 0x7f4d08153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.629+0000 7f4d0d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d0813e360 0x7f4d08154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.629+0000 7f4d0d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08071240 con 0x7f4d0813e360 2024-01-15T17:56:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.629+0000 7f4d0d200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08070630 con 0x7f4d0813df80 2024-01-15T17:56:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.629+0000 7f4d0d200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d08070910 con 0x7f4d080cf9a0 2024-01-15T17:56:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.629+0000 7f4d06800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d0813e360 0x7f4d08154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.629+0000 7f4d06800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d0813e360 0x7f4d08154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59710/0 (socket says 172.21.15.26:59710) 2024-01-15T17:56:07.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.629+0000 7f4d06800640 1 -- 172.21.15.26:0/4096032398 learned_addr learned my addr 172.21.15.26:0/4096032398 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.630+0000 7f4d05e00640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d080cf9a0 0x7f4d081532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.630+0000 7f4d06800640 1 -- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d080cf9a0 msgr2=0x7f4d081532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.630+0000 7f4d06800640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d080cf9a0 0x7f4d081532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.630+0000 7f4d06800640 1 -- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d0813df80 msgr2=0x7f4d08153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.630+0000 7f4cffe00640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d0813df80 0x7f4d08153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.630+0000 7f4d06800640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d0813df80 0x7f4d08153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.630+0000 7f4d06800640 1 -- 172.21.15.26:0/4096032398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00009b30 con 0x7f4d0813e360 2024-01-15T17:56:07.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.630+0000 7f4d05e00640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d080cf9a0 0x7f4d081532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.630+0000 7f4d06800640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d0813e360 0x7f4d08154470 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4d00036040 tx=0x7f4d00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.631+0000 7f4cfe000640 1 -- 172.21.15.26:0/4096032398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d00009510 con 0x7f4d0813e360 2024-01-15T17:56:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.631+0000 7f4cfe000640 1 -- 172.21.15.26:0/4096032398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0003e450 con 0x7f4d0813e360 2024-01-15T17:56:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.631+0000 7f4cfe000640 1 -- 172.21.15.26:0/4096032398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0003e800 con 0x7f4d0813e360 2024-01-15T17:56:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.631+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d08070bf0 con 0x7f4d0813e360 2024-01-15T17:56:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.631+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d08151810 con 0x7f4d0813e360 2024-01-15T17:56:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.632+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d080713f0 con 0x7f4d0813e360 2024-01-15T17:56:07.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.633+0000 7f4cfe000640 1 -- 172.21.15.26:0/4096032398 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f4d0003e960 con 0x7f4d0813e360 2024-01-15T17:56:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.634+0000 7f4cfe000640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ce006b2f0 0x7f4ce006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.634+0000 7f4cfe000640 1 -- 172.21.15.26:0/4096032398 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4d000acbb0 con 0x7f4d0813e360 2024-01-15T17:56:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.635+0000 7f4d05e00640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ce006b2f0 0x7f4ce006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.636+0000 7f4d05e00640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ce006b2f0 0x7f4ce006d7b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8006fd0 tx=0x7f4cf8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.639+0000 7f4cfe000640 1 -- 172.21.15.26:0/4096032398 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d080713f0 con 0x7f4d0813e360 2024-01-15T17:56:07.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.836+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d080675d0 con 0x7f4d0813e360 2024-01-15T17:56:07.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.837+0000 7f4cfe000640 1 -- 172.21.15.26:0/4096032398 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d000430a0 con 0x7f4d0813e360 2024-01-15T17:56:07.840 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:56:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ce006b2f0 msgr2=0x7f4ce006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ce006b2f0 0x7f4ce006d7b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8006fd0 tx=0x7f4cf8008040 comp rx=0 tx=0).stop 2024-01-15T17:56:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d0813e360 msgr2=0x7f4d08154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d0813e360 0x7f4d08154470 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4d00036040 tx=0x7f4d00004a40 comp rx=0 tx=0).stop 2024-01-15T17:56:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 shutdown_connections 2024-01-15T17:56:07.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ce006b2f0 0x7f4ce006d7b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d0813e360 0x7f4d08154470 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d0813df80 0x7f4d08153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 --2- 172.21.15.26:0/4096032398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d080cf9a0 0x7f4d081532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 >> 172.21.15.26:0/4096032398 conn(0x7f4d0806a800 msgr2=0x7f4d0815c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 shutdown_connections 2024-01-15T17:56:07.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:07.840+0000 7f4d0d200640 1 -- 172.21.15.26:0/4096032398 wait complete. 2024-01-15T17:56:07.862 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:56:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.079+0000 7f57aae00640 1 Processor -- start 2024-01-15T17:56:08.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.079+0000 7f57aae00640 1 -- start start 2024-01-15T17:56:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.081+0000 7f57aae00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57a413cf10 0x7f57a413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.081+0000 7f57aae00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a40ce440 con 0x7f57a413fae0 2024-01-15T17:56:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.081+0000 7f57aae00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a40ce720 con 0x7f57a40cddf0 2024-01-15T17:56:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.081+0000 7f57aae00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a413ff90 con 0x7f57a413cf10 2024-01-15T17:56:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.081+0000 7f57a9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57a413cf10 0x7f57a413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.081+0000 7f57a9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57a413cf10 0x7f57a413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59722/0 (socket says 172.21.15.26:59722) 2024-01-15T17:56:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.081+0000 7f57a9a00640 1 -- 172.21.15.26:0/2665098200 learned_addr learned my addr 172.21.15.26:0/2665098200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:08.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.081+0000 7f57a9000640 1 --1- 172.21.15.26:0/2665098200 >> v1:172.21.15.26:6789/0 conn(0x7f57a413fae0 0x7f57a40cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51324/0 (socket says 172.21.15.26:51324) 2024-01-15T17:56:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.082+0000 7f57a9a00640 1 -- 172.21.15.26:0/2665098200 >> v1:172.21.15.124:6789/0 conn(0x7f57a40cddf0 legacy=0x7f57a40b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.082+0000 7f57a9a00640 1 -- 172.21.15.26:0/2665098200 >> v1:172.21.15.26:6789/0 conn(0x7f57a413fae0 legacy=0x7f57a40cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.082+0000 7f57a9a00640 1 -- 172.21.15.26:0/2665098200 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a40cea00 con 0x7f57a413cf10 2024-01-15T17:56:08.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.083+0000 7f57a9a00640 1 --2- 172.21.15.26:0/2665098200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413cf10 0x7f57a413f5a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5794009870 tx=0x7f579402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473b9e1e19265e56 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.083+0000 7f57a3e00640 1 -- 172.21.15.26:0/2665098200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794030850 con 0x7f57a413cf10 2024-01-15T17:56:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.083+0000 7f57a3e00640 1 -- 172.21.15.26:0/2665098200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57940309b0 con 0x7f57a413cf10 2024-01-15T17:56:08.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.083+0000 7f57a3e00640 1 -- 172.21.15.26:0/2665098200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5794030d40 con 0x7f57a413cf10 2024-01-15T17:56:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.084+0000 7f57aae00640 1 -- 172.21.15.26:0/2665098200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413cf10 msgr2=0x7f57a413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.084+0000 7f57aae00640 1 --2- 172.21.15.26:0/2665098200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413cf10 0x7f57a413f5a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5794009870 tx=0x7f579402b990 comp rx=0 tx=0).stop 2024-01-15T17:56:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.084+0000 7f57aae00640 1 -- 172.21.15.26:0/2665098200 shutdown_connections 2024-01-15T17:56:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.084+0000 7f57aae00640 1 --2- 172.21.15.26:0/2665098200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413cf10 0x7f57a413f5a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.084+0000 7f57aae00640 1 -- 172.21.15.26:0/2665098200 >> 172.21.15.26:0/2665098200 conn(0x7f57a406a7d0 msgr2=0x7f57a406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.084+0000 7f57aae00640 1 -- 172.21.15.26:0/2665098200 shutdown_connections 2024-01-15T17:56:08.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.084+0000 7f57aae00640 1 -- 172.21.15.26:0/2665098200 wait complete. 2024-01-15T17:56:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.086+0000 7f57aae00640 1 Processor -- start 2024-01-15T17:56:08.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.086+0000 7f57aae00640 1 -- start start 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.087+0000 7f57aae00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57a40cddf0 0x7f57a413b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.087+0000 7f57aae00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57a413cf10 0x7f57a413af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.087+0000 7f57aae00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413fae0 0x7f57a41556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.087+0000 7f57aae00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a41f7420 con 0x7f57a413fae0 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.087+0000 7f57aae00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4155f20 con 0x7f57a40cddf0 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.087+0000 7f57aae00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a4156200 con 0x7f57a413cf10 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57aa400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413fae0 0x7f57a41556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57aa400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413fae0 0x7f57a41556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59732/0 (socket says 172.21.15.26:59732) 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57aa400640 1 -- 172.21.15.26:0/3872269184 learned_addr learned my addr 172.21.15.26:0/3872269184 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57a9a00640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57a40cddf0 0x7f57a413b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57a9000640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57a413cf10 0x7f57a413af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57aa400640 1 -- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57a413cf10 msgr2=0x7f57a413af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57aa400640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57a413cf10 0x7f57a413af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57aa400640 1 -- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57a40cddf0 msgr2=0x7f57a413b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57aa400640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57a40cddf0 0x7f57a413b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.088+0000 7f57aa400640 1 -- 172.21.15.26:0/3872269184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57940094d0 con 0x7f57a413fae0 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.089+0000 7f57aa400640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413fae0 0x7f57a41556d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f579c00c780 tx=0x7f579c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.089+0000 7f57a1600640 1 -- 172.21.15.26:0/3872269184 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c004b60 con 0x7f57a413fae0 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.089+0000 7f57a1600640 1 -- 172.21.15.26:0/3872269184 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c00ce90 con 0x7f57a413fae0 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.089+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a41f6d60 con 0x7f57a413fae0 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.089+0000 7f57a1600640 1 -- 172.21.15.26:0/3872269184 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c00a850 con 0x7f57a413fae0 2024-01-15T17:56:08.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.089+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a4155c10 con 0x7f57a413fae0 2024-01-15T17:56:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.090+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a41f6bc0 con 0x7f57a413fae0 2024-01-15T17:56:08.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.091+0000 7f57a1600640 1 -- 172.21.15.26:0/3872269184 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f579c00a9b0 con 0x7f57a413fae0 2024-01-15T17:56:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.092+0000 7f57a1600640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f577c06b2c0 0x7f577c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.092+0000 7f57a1600640 1 -- 172.21.15.26:0/3872269184 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f579c089dd0 con 0x7f57a413fae0 2024-01-15T17:56:08.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.093+0000 7f57a9a00640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f577c06b2c0 0x7f577c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.094+0000 7f57a9a00640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f577c06b2c0 0x7f577c06d780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5794036000 tx=0x7f5794008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:08.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.096+0000 7f57a1600640 1 -- 172.21.15.26:0/3872269184 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57a41f6bc0 con 0x7f57a413fae0 2024-01-15T17:56:08.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.288+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57a41f6bc0 con 0x7f57a413fae0 2024-01-15T17:56:08.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.289+0000 7f57a1600640 1 -- 172.21.15.26:0/3872269184 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57a41f6bc0 con 0x7f57a413fae0 2024-01-15T17:56:08.292 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:56:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.291+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f577c06b2c0 msgr2=0x7f577c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.291+0000 7f57aae00640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f577c06b2c0 0x7f577c06d780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5794036000 tx=0x7f5794008030 comp rx=0 tx=0).stop 2024-01-15T17:56:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.291+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413fae0 msgr2=0x7f57a41556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.291+0000 7f57aae00640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413fae0 0x7f57a41556d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f579c00c780 tx=0x7f579c00cc40 comp rx=0 tx=0).stop 2024-01-15T17:56:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.292+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 shutdown_connections 2024-01-15T17:56:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.292+0000 7f57aae00640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57a413cf10 0x7f57a413af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.292+0000 7f57aae00640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f577c06b2c0 0x7f577c06d780 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.292+0000 7f57aae00640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57a413fae0 0x7f57a41556d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.292+0000 7f57aae00640 1 --2- 172.21.15.26:0/3872269184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57a40cddf0 0x7f57a413b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.292+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 >> 172.21.15.26:0/3872269184 conn(0x7f57a406a7d0 msgr2=0x7f57a406abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.292+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 shutdown_connections 2024-01-15T17:56:08.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.292+0000 7f57aae00640 1 -- 172.21.15.26:0/3872269184 wait complete. 2024-01-15T17:56:08.314 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:56:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.537+0000 7fe181000640 1 Processor -- start 2024-01-15T17:56:08.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.537+0000 7fe181000640 1 -- start start 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.538+0000 7fe181000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe17c15d960 0x7fe17c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.538+0000 7fe181000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe17c0b20d0 0x7fe17c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.538+0000 7fe181000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe17c0ce760 con 0x7fe17c15d580 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.538+0000 7fe181000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c0d0a00 con 0x7fe17c15d960 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.538+0000 7fe181000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17c0cf250 con 0x7fe17c0b20d0 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.538+0000 7fe17aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe17c15d960 0x7fe17c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.538+0000 7fe17aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe17c15d960 0x7fe17c0cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59734/0 (socket says 172.21.15.26:59734) 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.538+0000 7fe17aa00640 1 -- 172.21.15.26:0/125795738 learned_addr learned my addr 172.21.15.26:0/125795738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.539+0000 7fe17be00640 1 --2- 172.21.15.26:0/125795738 >> v2:172.21.15.124:3300/0 conn(0x7fe17c0b20d0 0x7fe17c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.539+0000 7fe17aa00640 1 -- 172.21.15.26:0/125795738 >> v2:172.21.15.124:3300/0 conn(0x7fe17c0b20d0 msgr2=0x7fe17c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.539+0000 7fe17aa00640 1 --2- 172.21.15.26:0/125795738 >> v2:172.21.15.124:3300/0 conn(0x7fe17c0b20d0 0x7fe17c0b0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.539+0000 7fe17aa00640 1 -- 172.21.15.26:0/125795738 >> v1:172.21.15.124:6789/0 conn(0x7fe17c15d580 legacy=0x7fe17c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.539+0000 7fe17aa00640 1 -- 172.21.15.26:0/125795738 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe17c0d1f50 con 0x7fe17c15d960 2024-01-15T17:56:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.540+0000 7fe17aa00640 1 --2- 172.21.15.26:0/125795738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d960 0x7fe17c0cfd60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe17c06adb0 tx=0x7fe16402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9f0ae932906d19 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.540+0000 7fe17a000640 1 -- 172.21.15.26:0/125795738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1640317e0 con 0x7fe17c15d960 2024-01-15T17:56:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.540+0000 7fe17a000640 1 -- 172.21.15.26:0/125795738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe164031940 con 0x7fe17c15d960 2024-01-15T17:56:08.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.540+0000 7fe17a000640 1 -- 172.21.15.26:0/125795738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164031cd0 con 0x7fe17c15d960 2024-01-15T17:56:08.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.541+0000 7fe181000640 1 -- 172.21.15.26:0/125795738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d960 msgr2=0x7fe17c0cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.541+0000 7fe181000640 1 --2- 172.21.15.26:0/125795738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d960 0x7fe17c0cfd60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe17c06adb0 tx=0x7fe16402b880 comp rx=0 tx=0).stop 2024-01-15T17:56:08.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.541+0000 7fe181000640 1 -- 172.21.15.26:0/125795738 shutdown_connections 2024-01-15T17:56:08.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.541+0000 7fe181000640 1 --2- 172.21.15.26:0/125795738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d960 0x7fe17c0cfd60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.541+0000 7fe181000640 1 --2- 172.21.15.26:0/125795738 >> v2:172.21.15.124:3300/0 conn(0x7fe17c0b20d0 0x7fe17c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.541+0000 7fe181000640 1 -- 172.21.15.26:0/125795738 >> 172.21.15.26:0/125795738 conn(0x7fe17c0d3040 msgr2=0x7fe17c0d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.541+0000 7fe181000640 1 -- 172.21.15.26:0/125795738 shutdown_connections 2024-01-15T17:56:08.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.541+0000 7fe181000640 1 -- 172.21.15.26:0/125795738 wait complete. 2024-01-15T17:56:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.543+0000 7fe181000640 1 Processor -- start 2024-01-15T17:56:08.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.543+0000 7fe181000640 1 -- start start 2024-01-15T17:56:08.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.544+0000 7fe181000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe17c0b20d0 0x7fe17c0cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.544+0000 7fe181000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d580 0x7fe17c0cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.544+0000 7fe181000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17c15d960 0x7fe17c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.544+0000 7fe181000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1f3e10 con 0x7fe17c15d580 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.544+0000 7fe181000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c1f3b30 con 0x7fe17c0b20d0 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.544+0000 7fe181000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe17c0ca2b0 con 0x7fe17c15d960 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.544+0000 7fe17aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d580 0x7fe17c0cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.544+0000 7fe17aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d580 0x7fe17c0cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59744/0 (socket says 172.21.15.26:59744) 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.544+0000 7fe17aa00640 1 -- 172.21.15.26:0/1258852260 learned_addr learned my addr 172.21.15.26:0/1258852260 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe17b400640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe17c0b20d0 0x7fe17c0cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe17aa00640 1 -- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17c15d960 msgr2=0x7fe17c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe17aa00640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17c15d960 0x7fe17c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe17aa00640 1 -- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe17c0b20d0 msgr2=0x7fe17c0cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe17aa00640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe17c0b20d0 0x7fe17c0cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe17aa00640 1 -- 172.21.15.26:0/1258852260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1640094d0 con 0x7fe17c15d580 2024-01-15T17:56:08.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe17b400640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe17c0b20d0 0x7fe17c0cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:08.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe17aa00640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d580 0x7fe17c0cb940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe16402b8c0 tx=0x7fe1640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:08.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe173400640 1 -- 172.21.15.26:0/1258852260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164032940 con 0x7fe17c15d580 2024-01-15T17:56:08.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe173400640 1 -- 172.21.15.26:0/1258852260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe164032aa0 con 0x7fe17c15d580 2024-01-15T17:56:08.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe173400640 1 -- 172.21.15.26:0/1258852260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe164032e50 con 0x7fe17c15d580 2024-01-15T17:56:08.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.545+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe17c0ca590 con 0x7fe17c15d580 2024-01-15T17:56:08.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.546+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17c0cb2d0 con 0x7fe17c15d580 2024-01-15T17:56:08.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.547+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17c0caab0 con 0x7fe17c15d580 2024-01-15T17:56:08.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.548+0000 7fe173400640 1 -- 172.21.15.26:0/1258852260 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe16404e450 con 0x7fe17c15d580 2024-01-15T17:56:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.549+0000 7fe173400640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe15406b2c0 0x7fe15406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.549+0000 7fe173400640 1 -- 172.21.15.26:0/1258852260 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe1640abec0 con 0x7fe17c15d580 2024-01-15T17:56:08.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.549+0000 7fe17b400640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe15406b2c0 0x7fe15406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.551+0000 7fe17b400640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe15406b2c0 0x7fe15406d780 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0cd110 tx=0x7fe16c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:08.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.553+0000 7fe173400640 1 -- 172.21.15.26:0/1258852260 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe17c0caab0 con 0x7fe17c15d580 2024-01-15T17:56:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.747+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe17c0caab0 con 0x7fe17c15d580 2024-01-15T17:56:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.748+0000 7fe173400640 1 -- 172.21.15.26:0/1258852260 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe17c0caab0 con 0x7fe17c15d580 2024-01-15T17:56:08.751 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.750+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe15406b2c0 msgr2=0x7fe15406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.750+0000 7fe181000640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe15406b2c0 0x7fe15406d780 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe17c0cd110 tx=0x7fe16c009040 comp rx=0 tx=0).stop 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.750+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d580 msgr2=0x7fe17c0cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.750+0000 7fe181000640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d580 0x7fe17c0cb940 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe16402b8c0 tx=0x7fe1640046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.750+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 shutdown_connections 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.750+0000 7fe181000640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17c15d960 0x7fe17c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.750+0000 7fe181000640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe15406b2c0 0x7fe15406d780 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.751+0000 7fe181000640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe17c15d580 0x7fe17c0cb940 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.751+0000 7fe181000640 1 --2- 172.21.15.26:0/1258852260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe17c0b20d0 0x7fe17c0cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.751+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 >> 172.21.15.26:0/1258852260 conn(0x7fe17c0d3040 msgr2=0x7fe17c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.751+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 shutdown_connections 2024-01-15T17:56:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.751+0000 7fe181000640 1 -- 172.21.15.26:0/1258852260 wait complete. 2024-01-15T17:56:08.776 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:56:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.988+0000 7f352f200640 1 Processor -- start 2024-01-15T17:56:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.988+0000 7f352f200640 1 -- start start 2024-01-15T17:56:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.990+0000 7f352f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3530143c40 0x7f3530071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.990+0000 7f352f200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f35300717d0 0x7f35300cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.990+0000 7f352f200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3530070e50 con 0x7f35300cca40 2024-01-15T17:56:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.990+0000 7f352f200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35300723c0 con 0x7f35300717d0 2024-01-15T17:56:08.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.990+0000 7f352f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35300cce70 con 0x7f3530143c40 2024-01-15T17:56:08.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.991+0000 7f3527e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f35300717d0 0x7f35300cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.991+0000 7f352de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3530143c40 0x7f3530071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:08.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.990+0000 7f352e800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f35300cca40 0x7f353015eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51338/0 (socket says 172.21.15.26:51338) 2024-01-15T17:56:08.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.991+0000 7f352e800640 1 -- 172.21.15.26:0/2385388857 learned_addr learned my addr 172.21.15.26:0/2385388857 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.991+0000 7f3527e00640 1 -- 172.21.15.26:0/2385388857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3530143c40 msgr2=0x7f3530071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.991+0000 7f3527e00640 1 --2- 172.21.15.26:0/2385388857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3530143c40 0x7f3530071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.991+0000 7f3527e00640 1 -- 172.21.15.26:0/2385388857 >> v1:172.21.15.26:6789/0 conn(0x7f35300cca40 legacy=0x7f353015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.991+0000 7f3527e00640 1 -- 172.21.15.26:0/2385388857 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353006e170 con 0x7f35300717d0 2024-01-15T17:56:08.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.993+0000 7f3527e00640 1 --2- 172.21.15.26:0/2385388857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300717d0 0x7f35300cc500 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f35180097a0 tx=0x7f351802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=667f6444d123baae server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.993+0000 7f352d400640 1 -- 172.21.15.26:0/2385388857 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518031780 con 0x7f35300717d0 2024-01-15T17:56:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.993+0000 7f352d400640 1 -- 172.21.15.26:0/2385388857 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35180318e0 con 0x7f35300717d0 2024-01-15T17:56:08.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.993+0000 7f352d400640 1 -- 172.21.15.26:0/2385388857 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518031c70 con 0x7f35300717d0 2024-01-15T17:56:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.994+0000 7f352f200640 1 -- 172.21.15.26:0/2385388857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300717d0 msgr2=0x7f35300cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.994+0000 7f352f200640 1 --2- 172.21.15.26:0/2385388857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300717d0 0x7f35300cc500 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f35180097a0 tx=0x7f351802b820 comp rx=0 tx=0).stop 2024-01-15T17:56:08.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.994+0000 7f352f200640 1 -- 172.21.15.26:0/2385388857 shutdown_connections 2024-01-15T17:56:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.994+0000 7f352f200640 1 --2- 172.21.15.26:0/2385388857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35300717d0 0x7f35300cc500 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.994+0000 7f352f200640 1 --2- 172.21.15.26:0/2385388857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3530143c40 0x7f3530071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.994+0000 7f352f200640 1 -- 172.21.15.26:0/2385388857 >> 172.21.15.26:0/2385388857 conn(0x7f353006a800 msgr2=0x7f353006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.994+0000 7f352f200640 1 -- 172.21.15.26:0/2385388857 shutdown_connections 2024-01-15T17:56:08.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.994+0000 7f352f200640 1 -- 172.21.15.26:0/2385388857 wait complete. 2024-01-15T17:56:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.996+0000 7f352f200640 1 Processor -- start 2024-01-15T17:56:08.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.996+0000 7f352f200640 1 -- start start 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.997+0000 7f352f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35300717d0 0x7f35301587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.997+0000 7f352f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300cca40 0x7f35300a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.997+0000 7f352f200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3530143c40 0x7f353013a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.997+0000 7f352f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353013ae70 con 0x7f35300717d0 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.997+0000 7f352f200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35300a93e0 con 0x7f3530143c40 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.997+0000 7f352f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353013b8e0 con 0x7f35300cca40 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.997+0000 7f352de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35300717d0 0x7f35301587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f352de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35300717d0 0x7f35301587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59756/0 (socket says 172.21.15.26:59756) 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f352de00640 1 -- 172.21.15.26:0/639679165 learned_addr learned my addr 172.21.15.26:0/639679165 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f352e800640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3530143c40 0x7f353013a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f3527e00640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300cca40 0x7f35300a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f352de00640 1 -- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300cca40 msgr2=0x7f35300a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f352de00640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300cca40 0x7f35300a8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f352de00640 1 -- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3530143c40 msgr2=0x7f353013a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f352de00640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3530143c40 0x7f353013a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f352de00640 1 -- 172.21.15.26:0/639679165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009400 con 0x7f35300717d0 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.998+0000 7f352de00640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35300717d0 0x7f35301587a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f352000e870 tx=0x7f352000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.999+0000 7f3526000640 1 -- 172.21.15.26:0/639679165 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35200094f0 con 0x7f35300717d0 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.999+0000 7f3526000640 1 -- 172.21.15.26:0/639679165 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352000c940 con 0x7f35300717d0 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.999+0000 7f3526000640 1 -- 172.21.15.26:0/639679165 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352000ccf0 con 0x7f35300717d0 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.999+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353013bbc0 con 0x7f35300717d0 2024-01-15T17:56:09.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:08.999+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353013c180 con 0x7f35300717d0 2024-01-15T17:56:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.000+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3530067450 con 0x7f35300717d0 2024-01-15T17:56:09.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.001+0000 7f3526000640 1 -- 172.21.15.26:0/639679165 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f3520013470 con 0x7f35300717d0 2024-01-15T17:56:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.002+0000 7f3526000640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f350806b2c0 0x7f350806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.002+0000 7f3526000640 1 -- 172.21.15.26:0/639679165 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3520089d80 con 0x7f35300717d0 2024-01-15T17:56:09.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.003+0000 7f3527e00640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f350806b2c0 0x7f350806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.004+0000 7f3527e00640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f350806b2c0 0x7f350806d780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3518036000 tx=0x7f351803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:09.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.007+0000 7f3526000640 1 -- 172.21.15.26:0/639679165 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3520054b60 con 0x7f35300717d0 2024-01-15T17:56:09.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.214+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f35300ceb30 con 0x7f35300717d0 2024-01-15T17:56:09.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.215+0000 7f3526000640 1 -- 172.21.15.26:0/639679165 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3520021080 con 0x7f35300717d0 2024-01-15T17:56:09.218 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.217+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f350806b2c0 msgr2=0x7f350806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.217+0000 7f352f200640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f350806b2c0 0x7f350806d780 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3518036000 tx=0x7f351803d040 comp rx=0 tx=0).stop 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.217+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35300717d0 msgr2=0x7f35301587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.218+0000 7f352f200640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35300717d0 0x7f35301587a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f352000e870 tx=0x7f352000ed30 comp rx=0 tx=0).stop 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.218+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 shutdown_connections 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.218+0000 7f352f200640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35300cca40 0x7f35300a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.218+0000 7f352f200640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f350806b2c0 0x7f350806d780 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.218+0000 7f352f200640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3530143c40 0x7f353013a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.218+0000 7f352f200640 1 --2- 172.21.15.26:0/639679165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f35300717d0 0x7f35301587a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.218+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 >> 172.21.15.26:0/639679165 conn(0x7f353006a800 msgr2=0x7f353006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.218+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 shutdown_connections 2024-01-15T17:56:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.218+0000 7f352f200640 1 -- 172.21.15.26:0/639679165 wait complete. 2024-01-15T17:56:09.241 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi124.front.sepia.ceph.com... 2024-01-15T17:56:09.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:09.241 DEBUG:teuthology.orchestra.run.smithi124:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-01-15T17:56:09.343 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:56:09.344 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:56:09.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:09.344 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T17:56:09.393 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T17:56:09.393 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:56:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.617+0000 7f2575000640 1 Processor -- start 2024-01-15T17:56:09.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.617+0000 7f2575000640 1 -- start start 2024-01-15T17:56:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.619+0000 7f2575000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25700cc500 0x7f25700cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.619+0000 7f2575000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2570070e50 con 0x7f257006d160 2024-01-15T17:56:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.619+0000 7f2575000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25700ccea0 con 0x7f2570143c40 2024-01-15T17:56:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.619+0000 7f2575000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25700723c0 con 0x7f25700cc500 2024-01-15T17:56:09.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.619+0000 7f256fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f257006d160 0x7f257006d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51346/0 (socket says 172.21.15.26:51346) 2024-01-15T17:56:09.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.619+0000 7f256fe00640 1 -- 172.21.15.26:0/2916169154 learned_addr learned my addr 172.21.15.26:0/2916169154 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f2567e00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25700cc500 0x7f25700cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824949957 0 0) 0x7f2570070e50 con 0x7f257006d160 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2550003520 con 0x7f257006d160 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904631520 0 0) 0x7f2550003520 con 0x7f257006d160 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25700cc500 msgr2=0x7f25700cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25700cc500 0x7f25700cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 >> v1:172.21.15.124:6789/0 conn(0x7f2570143c40 legacy=0x7f2570071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2570144020 con 0x7f257006d160 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f25680019f0 con 0x7f257006d160 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 >> v1:172.21.15.26:6789/0 conn(0x7f257006d160 legacy=0x7f257006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.620+0000 7f256ea00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25700cc500 0x7f25500039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256ea00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2550003f70 0x7f25500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256ea00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f2550005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25700723c0 con 0x7f2550004d00 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255000bb70 con 0x7f2550003f70 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255000bd30 con 0x7f25700cc500 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256fe00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f2550005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f2567e00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2550003f70 0x7f25500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256f400640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25700cc500 0x7f25500039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256fe00640 1 -- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25700cc500 msgr2=0x7f25500039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256fe00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25700cc500 0x7f25500039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256fe00640 1 -- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2550003f70 msgr2=0x7f25500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256fe00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2550003f70 0x7f25500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256fe00640 1 -- 172.21.15.26:0/2916169154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2550003520 con 0x7f2550004d00 2024-01-15T17:56:09.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.621+0000 7f256f400640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25700cc500 0x7f25500039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:09.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.622+0000 7f256fe00640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f2550005180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f256800a760 tx=0x7f25500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0d36b27ed0de2d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.623+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568034630 con 0x7f2550004d00 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.623+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256802f070 con 0x7f2550004d00 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.623+0000 7f256ea00640 1 -- 172.21.15.26:0/2916169154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568034a50 con 0x7f2550004d00 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.623+0000 7f2575000640 1 -- 172.21.15.26:0/2916169154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 msgr2=0x7f2550005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.623+0000 7f2575000640 1 --2- 172.21.15.26:0/2916169154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f2550005180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f256800a760 tx=0x7f25500057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.623+0000 7f256fe00640 1 -- 172.21.15.26:0/2916169154 reap_dead start 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.623+0000 7f2575000640 1 -- 172.21.15.26:0/2916169154 shutdown_connections 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.623+0000 7f2575000640 1 -- 172.21.15.26:0/2916169154 >> 172.21.15.26:0/2916169154 conn(0x7f257006a800 msgr2=0x7f257006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.623+0000 7f2575000640 1 -- 172.21.15.26:0/2916169154 shutdown_connections 2024-01-15T17:56:09.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.624+0000 7f2575000640 1 -- 172.21.15.26:0/2916169154 wait complete. 2024-01-15T17:56:09.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.625+0000 7f2575000640 1 Processor -- start 2024-01-15T17:56:09.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.625+0000 7f2575000640 1 -- start start 2024-01-15T17:56:09.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.626+0000 7f2575000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f25700cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.626+0000 7f2575000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25700aa4f0 0x7f25701312a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.626+0000 7f2575000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257015ebc0 0x7f257015f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.626+0000 7f2575000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257012f250 con 0x7f2550004d00 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.626+0000 7f2575000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257015f560 con 0x7f25700aa4f0 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.626+0000 7f2575000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2570131ac0 con 0x7f257015ebc0 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f256f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f25700cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f256f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f25700cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59780/0 (socket says 172.21.15.26:59780) 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f256f400640 1 -- 172.21.15.26:0/1823874270 learned_addr learned my addr 172.21.15.26:0/1823874270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f2567e00640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25700aa4f0 0x7f25701312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f256fe00640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257015ebc0 0x7f257015f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f256f400640 1 -- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257015ebc0 msgr2=0x7f257015f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f256f400640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257015ebc0 0x7f257015f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f256f400640 1 -- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25700aa4f0 msgr2=0x7f25701312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f256f400640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25700aa4f0 0x7f25701312a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f256f400640 1 -- 172.21.15.26:0/1823874270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2570131da0 con 0x7f2550004d00 2024-01-15T17:56:09.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.627+0000 7f2567e00640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25700aa4f0 0x7f25701312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:09.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.628+0000 7f256f400640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f25700cb0d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f256000d7d0 tx=0x7f256000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:09.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.628+0000 7f2567400640 1 -- 172.21.15.26:0/1823874270 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560017070 con 0x7f2550004d00 2024-01-15T17:56:09.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.628+0000 7f2567400640 1 -- 172.21.15.26:0/1823874270 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2560005070 con 0x7f2550004d00 2024-01-15T17:56:09.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.628+0000 7f2567400640 1 -- 172.21.15.26:0/1823874270 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560005440 con 0x7f2550004d00 2024-01-15T17:56:09.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.628+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2570132080 con 0x7f2550004d00 2024-01-15T17:56:09.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.628+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25701317e0 con 0x7f2550004d00 2024-01-15T17:56:09.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.630+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2570132270 con 0x7f2550004d00 2024-01-15T17:56:09.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.630+0000 7f2567400640 1 -- 172.21.15.26:0/1823874270 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2560007030 con 0x7f2550004d00 2024-01-15T17:56:09.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.632+0000 7f2567400640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f254806b2f0 0x7f254806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:09.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.632+0000 7f2567400640 1 -- 172.21.15.26:0/1823874270 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f25600895a0 con 0x7f2550004d00 2024-01-15T17:56:09.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.632+0000 7f2567e00640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f254806b2f0 0x7f254806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:09.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.634+0000 7f2567e00640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f254806b2f0 0x7f254806d7b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f257013fa70 tx=0x7f2558008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:09.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.636+0000 7f2567400640 1 -- 172.21.15.26:0/1823874270 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2570132270 con 0x7f2550004d00 2024-01-15T17:56:09.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.867+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2570132270 con 0x7f2550004d00 2024-01-15T17:56:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.868+0000 7f2567400640 1 -- 172.21.15.26:0/1823874270 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f2570132270 con 0x7f2550004d00 2024-01-15T17:56:09.871 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:09.871 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/593578542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":593578542},{"type":"v1","addr":"172.21.15.124:6837","nonce":593578542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1776325954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1776325954},{"type":"v1","addr":"172.21.15.26:6835","nonce":1776325954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:55:04.195734+0000","modified":"2024-01-15T17:55:07.244782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/3022017251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3022017251},{"type":"v1","addr":"172.21.15.124:6835","nonce":3022017251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T17:56:09.871 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-01-15T17:56:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f254806b2f0 msgr2=0x7f254806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f254806b2f0 0x7f254806d7b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f257013fa70 tx=0x7f2558008040 comp rx=0 tx=0).stop 2024-01-15T17:56:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 msgr2=0x7f25700cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f25700cb0d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f256000d7d0 tx=0x7f256000dc90 comp rx=0 tx=0).stop 2024-01-15T17:56:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 shutdown_connections 2024-01-15T17:56:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257015ebc0 0x7f257015f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f254806b2f0 0x7f254806d7b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25700aa4f0 0x7f25701312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 --2- 172.21.15.26:0/1823874270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2550004d00 0x7f25700cb0d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 >> 172.21.15.26:0/1823874270 conn(0x7f257006a800 msgr2=0x7f25700b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 shutdown_connections 2024-01-15T17:56:09.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:09.871+0000 7f2575000640 1 -- 172.21.15.26:0/1823874270 wait complete. 2024-01-15T17:56:09.893 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:10.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.116+0000 7fbeffe00640 1 Processor -- start 2024-01-15T17:56:10.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.117+0000 7fbeffe00640 1 -- start start 2024-01-15T17:56:10.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.118+0000 7fbeffe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbf00160920 0x7fbf000b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.118+0000 7fbeffe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000cfea0 0x7fbf00161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.118+0000 7fbeffe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf000d0a10 con 0x7fbf000cf9f0 2024-01-15T17:56:10.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.118+0000 7fbeffe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf000d0330 con 0x7fbf00160920 2024-01-15T17:56:10.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.118+0000 7fbeffe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000d1f50 con 0x7fbf000cfea0 2024-01-15T17:56:10.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.118+0000 7fbefe000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbf000cf9f0 0x7fbf00161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51348/0 (socket says 172.21.15.26:51348) 2024-01-15T17:56:10.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.118+0000 7fbefe000640 1 -- 172.21.15.26:0/375389891 learned_addr learned my addr 172.21.15.26:0/375389891 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbeff400640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000cfea0 0x7fbf00161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefea00640 1 --2- 172.21.15.26:0/375389891 >> v2:172.21.15.124:3300/0 conn(0x7fbf00160920 0x7fbf000b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736438909 0 0) 0x7fbf000d0a10 con 0x7fbf000cf9f0 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee0003520 con 0x7fbf000cf9f0 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353010215 0 0) 0x7fbee0003520 con 0x7fbf000cf9f0 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000cfea0 msgr2=0x7fbf00161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000cfea0 0x7fbf00161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 >> v2:172.21.15.124:3300/0 conn(0x7fbf00160920 msgr2=0x7fbf000b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 --2- 172.21.15.26:0/375389891 >> v2:172.21.15.124:3300/0 conn(0x7fbf00160920 0x7fbf000b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf000cf080 con 0x7fbf000cf9f0 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fbee8001670 con 0x7fbf000cf9f0 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 >> v1:172.21.15.26:6789/0 conn(0x7fbf000cf9f0 legacy=0x7fbf00161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.119+0000 7fbefd600640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbee0003970 0x7fbee0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbefd600640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000cfea0 0x7fbee00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbefd600640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbee0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000d0330 con 0x7fbee0004d00 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000d1f50 con 0x7fbee0003970 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee000b980 con 0x7fbf000cfea0 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbeff400640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbee0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbefe000640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000cfea0 0x7fbee00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbefea00640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbee0003970 0x7fbee0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbeff400640 1 -- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000cfea0 msgr2=0x7fbee00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbeff400640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000cfea0 0x7fbee00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbeff400640 1 -- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbee0003970 msgr2=0x7fbee0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:10.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbeff400640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbee0003970 0x7fbee0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbeff400640 1 -- 172.21.15.26:0/375389891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0003520 con 0x7fbee0004d00 2024-01-15T17:56:10.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.120+0000 7fbefea00640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbee0003970 0x7fbee0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.121+0000 7fbeff400640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbee0005180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009fc0 tx=0x7fbee00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6208ee9ee8b1c21e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.122+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef802cc20 con 0x7fbee0004d00 2024-01-15T17:56:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.122+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef800f040 con 0x7fbee0004d00 2024-01-15T17:56:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.122+0000 7fbefd600640 1 -- 172.21.15.26:0/375389891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef8034440 con 0x7fbee0004d00 2024-01-15T17:56:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.122+0000 7fbeffe00640 1 -- 172.21.15.26:0/375389891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 msgr2=0x7fbee0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.122+0000 7fbeffe00640 1 --2- 172.21.15.26:0/375389891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbee0005180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009fc0 tx=0x7fbee00057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.122+0000 7fbeff400640 1 -- 172.21.15.26:0/375389891 reap_dead start 2024-01-15T17:56:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.122+0000 7fbeffe00640 1 -- 172.21.15.26:0/375389891 shutdown_connections 2024-01-15T17:56:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.122+0000 7fbeffe00640 1 -- 172.21.15.26:0/375389891 >> 172.21.15.26:0/375389891 conn(0x7fbf000d3090 msgr2=0x7fbf000d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.122+0000 7fbeffe00640 1 -- 172.21.15.26:0/375389891 shutdown_connections 2024-01-15T17:56:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.123+0000 7fbeffe00640 1 -- 172.21.15.26:0/375389891 wait complete. 2024-01-15T17:56:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.124+0000 7fbeffe00640 1 Processor -- start 2024-01-15T17:56:10.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.124+0000 7fbeffe00640 1 -- start start 2024-01-15T17:56:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.125+0000 7fbeffe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbf000cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.125+0000 7fbeffe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf001540f0 0x7fbf000cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.125+0000 7fbeffe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000b8cc0 0x7fbf0015ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbeffe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000b7ff0 con 0x7fbee0004d00 2024-01-15T17:56:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbeffe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000cb8d0 con 0x7fbf001540f0 2024-01-15T17:56:10.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbeffe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000cbbb0 con 0x7fbf000b8cc0 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbefea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbf000cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbefea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbf000cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59798/0 (socket says 172.21.15.26:59798) 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbefea00640 1 -- 172.21.15.26:0/1583480373 learned_addr learned my addr 172.21.15.26:0/1583480373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbeff400640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000b8cc0 0x7fbf0015ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbefe000640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf001540f0 0x7fbf000cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbefea00640 1 -- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000b8cc0 msgr2=0x7fbf0015ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbefea00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000b8cc0 0x7fbf0015ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbefea00640 1 -- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf001540f0 msgr2=0x7fbf000cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbefea00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf001540f0 0x7fbf000cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.126+0000 7fbefea00640 1 -- 172.21.15.26:0/1583480373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf000b81e0 con 0x7fbee0004d00 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.127+0000 7fbefea00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbf000cd070 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbef00109d0 tx=0x7fbef0010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.127+0000 7fbef6a00640 1 -- 172.21.15.26:0/1583480373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef000ec20 con 0x7fbee0004d00 2024-01-15T17:56:10.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.127+0000 7fbef6a00640 1 -- 172.21.15.26:0/1583480373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef000ed80 con 0x7fbee0004d00 2024-01-15T17:56:10.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.127+0000 7fbef6a00640 1 -- 172.21.15.26:0/1583480373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef001c620 con 0x7fbee0004d00 2024-01-15T17:56:10.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.127+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf000b84c0 con 0x7fbee0004d00 2024-01-15T17:56:10.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.127+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf000cb5c0 con 0x7fbee0004d00 2024-01-15T17:56:10.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.129+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf00067450 con 0x7fbee0004d00 2024-01-15T17:56:10.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.130+0000 7fbef6a00640 1 -- 172.21.15.26:0/1583480373 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fbef001c780 con 0x7fbee0004d00 2024-01-15T17:56:10.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.131+0000 7fbef6a00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbed806b2f0 0x7fbed806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.131+0000 7fbef6a00640 1 -- 172.21.15.26:0/1583480373 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbef008a340 con 0x7fbee0004d00 2024-01-15T17:56:10.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.131+0000 7fbefe000640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbed806b2f0 0x7fbed806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.133+0000 7fbefe000640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbed806b2f0 0x7fbed806d7b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbee8001050 tx=0x7fbee8001d60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:10.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.136+0000 7fbef6a00640 1 -- 172.21.15.26:0/1583480373 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbef0055130 con 0x7fbee0004d00 2024-01-15T17:56:10.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.334+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf0006ccb0 con 0x7fbee0004d00 2024-01-15T17:56:10.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.337+0000 7fbef6a00640 1 -- 172.21.15.26:0/1583480373 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7fbef0058750 con 0x7fbee0004d00 2024-01-15T17:56:10.340 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:10.340 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:10.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbed806b2f0 msgr2=0x7fbed806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbed806b2f0 0x7fbed806d7b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbee8001050 tx=0x7fbee8001d60 comp rx=0 tx=0).stop 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 msgr2=0x7fbf000cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbf000cd070 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbef00109d0 tx=0x7fbef0010e90 comp rx=0 tx=0).stop 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 shutdown_connections 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf000b8cc0 0x7fbf0015ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbed806b2f0 0x7fbed806d7b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf001540f0 0x7fbf000cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 --2- 172.21.15.26:0/1583480373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbee0004d00 0x7fbf000cd070 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 >> 172.21.15.26:0/1583480373 conn(0x7fbf000d3090 msgr2=0x7fbf000b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 shutdown_connections 2024-01-15T17:56:10.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.340+0000 7fbeffe00640 1 -- 172.21.15.26:0/1583480373 wait complete. 2024-01-15T17:56:10.363 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T17:55:02.467034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T17:55:03.199274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:56:10.364 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:56:10.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.588+0000 7fc016400640 1 Processor -- start 2024-01-15T17:56:10.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.589+0000 7fc016400640 1 -- start start 2024-01-15T17:56:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.590+0000 7fc016400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0100adf20 0x7fc010070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.590+0000 7fc016400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0100ccb50 0x7fc010143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.590+0000 7fc016400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01015f4e0 con 0x7fc01015bf10 2024-01-15T17:56:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.590+0000 7fc016400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01015f970 con 0x7fc0100ccb50 2024-01-15T17:56:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.590+0000 7fc016400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc010071130 con 0x7fc0100adf20 2024-01-15T17:56:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.591+0000 7fc015a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0100ccb50 0x7fc010143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.591+0000 7fc015a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0100ccb50 0x7fc010143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59800/0 (socket says 172.21.15.26:59800) 2024-01-15T17:56:10.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.591+0000 7fc015a00640 1 -- 172.21.15.26:0/1208414759 learned_addr learned my addr 172.21.15.26:0/1208414759 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:10.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.591+0000 7fc015000640 1 --2- 172.21.15.26:0/1208414759 >> v2:172.21.15.124:3300/0 conn(0x7fc0100adf20 0x7fc010070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.591+0000 7fc015a00640 1 -- 172.21.15.26:0/1208414759 >> v2:172.21.15.124:3300/0 conn(0x7fc0100adf20 msgr2=0x7fc010070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:10.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.591+0000 7fc015a00640 1 --2- 172.21.15.26:0/1208414759 >> v2:172.21.15.124:3300/0 conn(0x7fc0100adf20 0x7fc010070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.591+0000 7fc015a00640 1 -- 172.21.15.26:0/1208414759 >> v1:172.21.15.124:6789/0 conn(0x7fc01015bf10 legacy=0x7fc0100cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:10.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.591+0000 7fc015a00640 1 -- 172.21.15.26:0/1208414759 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01015c390 con 0x7fc0100ccb50 2024-01-15T17:56:10.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.592+0000 7fc015a00640 1 --2- 172.21.15.26:0/1208414759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100ccb50 0x7fc010143d30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc0100b4150 tx=0x7fc00802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148df9a917cbfc96 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:10.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.593+0000 7fc00f400640 1 -- 172.21.15.26:0/1208414759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0080317e0 con 0x7fc0100ccb50 2024-01-15T17:56:10.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.593+0000 7fc00f400640 1 -- 172.21.15.26:0/1208414759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc008031940 con 0x7fc0100ccb50 2024-01-15T17:56:10.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.593+0000 7fc00f400640 1 -- 172.21.15.26:0/1208414759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc008031cd0 con 0x7fc0100ccb50 2024-01-15T17:56:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.594+0000 7fc016400640 1 -- 172.21.15.26:0/1208414759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100ccb50 msgr2=0x7fc010143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.594+0000 7fc016400640 1 --2- 172.21.15.26:0/1208414759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100ccb50 0x7fc010143d30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc0100b4150 tx=0x7fc00802b880 comp rx=0 tx=0).stop 2024-01-15T17:56:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.594+0000 7fc016400640 1 -- 172.21.15.26:0/1208414759 shutdown_connections 2024-01-15T17:56:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.594+0000 7fc016400640 1 --2- 172.21.15.26:0/1208414759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100ccb50 0x7fc010143d30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.594+0000 7fc016400640 1 --2- 172.21.15.26:0/1208414759 >> v2:172.21.15.124:3300/0 conn(0x7fc0100adf20 0x7fc010070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.594+0000 7fc016400640 1 -- 172.21.15.26:0/1208414759 >> 172.21.15.26:0/1208414759 conn(0x7fc01006a7d0 msgr2=0x7fc01006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.594+0000 7fc016400640 1 -- 172.21.15.26:0/1208414759 shutdown_connections 2024-01-15T17:56:10.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.594+0000 7fc016400640 1 -- 172.21.15.26:0/1208414759 wait complete. 2024-01-15T17:56:10.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.596+0000 7fc016400640 1 Processor -- start 2024-01-15T17:56:10.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.596+0000 7fc016400640 1 -- start start 2024-01-15T17:56:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc016400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100adf20 0x7fc010159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc016400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0100ccb50 0x7fc010139dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc016400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc01015bf10 0x7fc0100caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc016400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0101398f0 con 0x7fc0100adf20 2024-01-15T17:56:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc016400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01015a360 con 0x7fc01015bf10 2024-01-15T17:56:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc016400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01015a640 con 0x7fc0100ccb50 2024-01-15T17:56:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc015000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100adf20 0x7fc010159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc015000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100adf20 0x7fc010159b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59814/0 (socket says 172.21.15.26:59814) 2024-01-15T17:56:10.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc015000640 1 -- 172.21.15.26:0/3158919491 learned_addr learned my addr 172.21.15.26:0/3158919491 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc00fe00640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0100ccb50 0x7fc010139dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.597+0000 7fc015a00640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc01015bf10 0x7fc0100caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc015000640 1 -- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0100ccb50 msgr2=0x7fc010139dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc015000640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0100ccb50 0x7fc010139dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc015000640 1 -- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc01015bf10 msgr2=0x7fc0100caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc015000640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc01015bf10 0x7fc0100caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc015000640 1 -- 172.21.15.26:0/3158919491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0080094d0 con 0x7fc0100adf20 2024-01-15T17:56:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc015000640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100adf20 0x7fc010159b00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc00000d800 tx=0x7fc00000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc00cc00640 1 -- 172.21.15.26:0/3158919491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00000b6c0 con 0x7fc0100adf20 2024-01-15T17:56:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc00cc00640 1 -- 172.21.15.26:0/3158919491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00000b820 con 0x7fc0100adf20 2024-01-15T17:56:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01015a920 con 0x7fc0100adf20 2024-01-15T17:56:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.598+0000 7fc00cc00640 1 -- 172.21.15.26:0/3158919491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00000bbb0 con 0x7fc0100adf20 2024-01-15T17:56:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.599+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01015a040 con 0x7fc0100adf20 2024-01-15T17:56:10.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.600+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc010067450 con 0x7fc0100adf20 2024-01-15T17:56:10.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.601+0000 7fc00cc00640 1 -- 172.21.15.26:0/3158919491 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc00000bd10 con 0x7fc0100adf20 2024-01-15T17:56:10.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.602+0000 7fc00cc00640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbfe806b2f0 0x7fbfe806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:10.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.602+0000 7fc00cc00640 1 -- 172.21.15.26:0/3158919491 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fc00008a370 con 0x7fc0100adf20 2024-01-15T17:56:10.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.602+0000 7fc00fe00640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbfe806b2f0 0x7fbfe806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:10.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.603+0000 7fc00fe00640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbfe806b2f0 0x7fbfe806d7b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0101f6490 tx=0x7fbff8007310 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:10.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.606+0000 7fc00cc00640 1 -- 172.21.15.26:0/3158919491 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc000055020 con 0x7fc0100adf20 2024-01-15T17:56:10.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.795+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0100ad280 con 0x7fc0100adf20 2024-01-15T17:56:10.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.796+0000 7fc00cc00640 1 -- 172.21.15.26:0/3158919491 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc000058640 con 0x7fc0100adf20 2024-01-15T17:56:10.799 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:56:10.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbfe806b2f0 msgr2=0x7fbfe806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbfe806b2f0 0x7fbfe806d7b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc0101f6490 tx=0x7fbff8007310 comp rx=0 tx=0).stop 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100adf20 msgr2=0x7fc010159b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100adf20 0x7fc010159b00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc00000d800 tx=0x7fc00000dcc0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 shutdown_connections 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0100ccb50 0x7fc010139dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbfe806b2f0 0x7fbfe806d7b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc01015bf10 0x7fc0100caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 --2- 172.21.15.26:0/3158919491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0100adf20 0x7fc010159b00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 >> 172.21.15.26:0/3158919491 conn(0x7fc01006a7d0 msgr2=0x7fc01006abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 shutdown_connections 2024-01-15T17:56:10.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:10.798+0000 7fc016400640 1 -- 172.21.15.26:0/3158919491 wait complete. 2024-01-15T17:56:10.820 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T17:56:11.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.043+0000 7f07cc200640 1 Processor -- start 2024-01-15T17:56:11.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.043+0000 7f07cc200640 1 -- start start 2024-01-15T17:56:11.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.045+0000 7f07cc200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f07c40b34e0 0x7f07c40cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.045+0000 7f07cc200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f07c40d1090 0x7f07c40d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.045+0000 7f07cc200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c4165230 con 0x7f07c4164e50 2024-01-15T17:56:11.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.045+0000 7f07cc200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c40c6860 con 0x7f07c40d1090 2024-01-15T17:56:11.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.045+0000 7f07cc200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c40d0df0 con 0x7f07c40b34e0 2024-01-15T17:56:11.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07c9800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f07c4164e50 0x7f07c40ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51350/0 (socket says 172.21.15.26:51350) 2024-01-15T17:56:11.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07c9800640 1 -- 172.21.15.26:0/4031669697 learned_addr learned my addr 172.21.15.26:0/4031669697 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:11.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07ca200640 1 --2- 172.21.15.26:0/4031669697 >> v2:172.21.15.26:3300/0 conn(0x7f07c40d1090 0x7f07c40d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07c8e00640 1 --2- 172.21.15.26:0/4031669697 >> v2:172.21.15.124:3300/0 conn(0x7f07c40b34e0 0x7f07c40cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07c3e00640 1 -- 172.21.15.26:0/4031669697 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234938806 0 0) 0x7f07c4165230 con 0x7f07c4164e50 2024-01-15T17:56:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07c3e00640 1 -- 172.21.15.26:0/4031669697 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07a4003520 con 0x7f07c4164e50 2024-01-15T17:56:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07ca200640 1 -- 172.21.15.26:0/4031669697 >> v2:172.21.15.124:3300/0 conn(0x7f07c40b34e0 msgr2=0x7f07c40cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07ca200640 1 --2- 172.21.15.26:0/4031669697 >> v2:172.21.15.124:3300/0 conn(0x7f07c40b34e0 0x7f07c40cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07ca200640 1 -- 172.21.15.26:0/4031669697 >> v1:172.21.15.26:6789/0 conn(0x7f07c4164e50 legacy=0x7f07c40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.046+0000 7f07ca200640 1 -- 172.21.15.26:0/4031669697 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c40d19f0 con 0x7f07c40d1090 2024-01-15T17:56:11.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.048+0000 7f07ca200640 1 --2- 172.21.15.26:0/4031669697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40d1090 0x7f07c40d14b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f07c40d1c30 tx=0x7f07bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b79e0c44f732de server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:11.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.048+0000 7f07c3e00640 1 -- 172.21.15.26:0/4031669697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc031830 con 0x7f07c40d1090 2024-01-15T17:56:11.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.048+0000 7f07c3e00640 1 -- 172.21.15.26:0/4031669697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc031990 con 0x7f07c40d1090 2024-01-15T17:56:11.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.048+0000 7f07c3e00640 1 -- 172.21.15.26:0/4031669697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc031d20 con 0x7f07c40d1090 2024-01-15T17:56:11.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.048+0000 7f07cc200640 1 -- 172.21.15.26:0/4031669697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40d1090 msgr2=0x7f07c40d14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.048+0000 7f07cc200640 1 --2- 172.21.15.26:0/4031669697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40d1090 0x7f07c40d14b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f07c40d1c30 tx=0x7f07bc02b8d0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.049+0000 7f07cc200640 1 -- 172.21.15.26:0/4031669697 shutdown_connections 2024-01-15T17:56:11.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.049+0000 7f07cc200640 1 --2- 172.21.15.26:0/4031669697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40d1090 0x7f07c40d14b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.049+0000 7f07cc200640 1 --2- 172.21.15.26:0/4031669697 >> v2:172.21.15.124:3300/0 conn(0x7f07c40b34e0 0x7f07c40cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.049+0000 7f07cc200640 1 -- 172.21.15.26:0/4031669697 >> 172.21.15.26:0/4031669697 conn(0x7f07c4066690 msgr2=0x7f07c406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:11.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.049+0000 7f07cc200640 1 -- 172.21.15.26:0/4031669697 shutdown_connections 2024-01-15T17:56:11.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.049+0000 7f07cc200640 1 -- 172.21.15.26:0/4031669697 wait complete. 2024-01-15T17:56:11.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.051+0000 7f07cc200640 1 Processor -- start 2024-01-15T17:56:11.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.051+0000 7f07cc200640 1 -- start start 2024-01-15T17:56:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07cc200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40b34e0 0x7f07c4162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07cc200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07c40d1090 0x7f07c4154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07cc200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07c4164e50 0x7f07c40b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07cc200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c4159680 con 0x7f07c40b34e0 2024-01-15T17:56:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07cc200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c40b6a40 con 0x7f07c4164e50 2024-01-15T17:56:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07cc200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c4163180 con 0x7f07c40d1090 2024-01-15T17:56:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07c9800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40b34e0 0x7f07c4162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07c9800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40b34e0 0x7f07c4162aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59826/0 (socket says 172.21.15.26:59826) 2024-01-15T17:56:11.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07c9800640 1 -- 172.21.15.26:0/1932040976 learned_addr learned my addr 172.21.15.26:0/1932040976 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07c8e00640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07c40d1090 0x7f07c4154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07c9800640 1 -- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07c40d1090 msgr2=0x7f07c4154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07c9800640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07c40d1090 0x7f07c4154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07c9800640 1 -- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07c4164e50 msgr2=0x7f07c40b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07c9800640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07c4164e50 0x7f07c40b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.052+0000 7f07c9800640 1 -- 172.21.15.26:0/1932040976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07bc009400 con 0x7f07c40b34e0 2024-01-15T17:56:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.053+0000 7f07c9800640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40b34e0 0x7f07c4162aa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f07b400cbc0 tx=0x7f07b400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.054+0000 7f07c1600640 1 -- 172.21.15.26:0/1932040976 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b400ad40 con 0x7f07c40b34e0 2024-01-15T17:56:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.054+0000 7f07c1600640 1 -- 172.21.15.26:0/1932040976 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b4012400 con 0x7f07c40b34e0 2024-01-15T17:56:11.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.054+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c4158d50 con 0x7f07c40b34e0 2024-01-15T17:56:11.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.054+0000 7f07c1600640 1 -- 172.21.15.26:0/1932040976 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b400aea0 con 0x7f07c40b34e0 2024-01-15T17:56:11.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.054+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c4159310 con 0x7f07c40b34e0 2024-01-15T17:56:11.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.055+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c4074a50 con 0x7f07c40b34e0 2024-01-15T17:56:11.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.057+0000 7f07c1600640 1 -- 172.21.15.26:0/1932040976 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f07b4007070 con 0x7f07c40b34e0 2024-01-15T17:56:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.057+0000 7f07c1600640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079c06b2f0 0x7f079c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.058+0000 7f07c1600640 1 -- 172.21.15.26:0/1932040976 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f07b4089db0 con 0x7f07c40b34e0 2024-01-15T17:56:11.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.058+0000 7f07c8e00640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079c06b2f0 0x7f079c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.059+0000 7f07c8e00640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079c06b2f0 0x7f079c06d7b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f07c41633e0 tx=0x7f07ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:11.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.062+0000 7f07c1600640 1 -- 172.21.15.26:0/1932040976 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07b408e070 con 0x7f07c40b34e0 2024-01-15T17:56:11.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.254+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07c40ce5d0 con 0x7f07c40b34e0 2024-01-15T17:56:11.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.255+0000 7f07c1600640 1 -- 172.21.15.26:0/1932040976 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07b4017020 con 0x7f07c40b34e0 2024-01-15T17:56:11.258 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T17:56:11.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079c06b2f0 msgr2=0x7f079c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079c06b2f0 0x7f079c06d7b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f07c41633e0 tx=0x7f07ac008040 comp rx=0 tx=0).stop 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40b34e0 msgr2=0x7f07c4162aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40b34e0 0x7f07c4162aa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f07b400cbc0 tx=0x7f07b400a430 comp rx=0 tx=0).stop 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 shutdown_connections 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07c40d1090 0x7f07c4154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f079c06b2f0 0x7f079c06d7b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07c4164e50 0x7f07c40b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 --2- 172.21.15.26:0/1932040976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07c40b34e0 0x7f07c4162aa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 >> 172.21.15.26:0/1932040976 conn(0x7f07c4066690 msgr2=0x7f07c40cdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 shutdown_connections 2024-01-15T17:56:11.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.257+0000 7f07cc200640 1 -- 172.21.15.26:0/1932040976 wait complete. 2024-01-15T17:56:11.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T17:56:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.498+0000 7f9148e00640 1 Processor -- start 2024-01-15T17:56:11.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.498+0000 7f9148e00640 1 -- start start 2024-01-15T17:56:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.500+0000 7f9148e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f91440ccb50 0x7f9144070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.500+0000 7f9148e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f914415f4e0 con 0x7f91440adf20 2024-01-15T17:56:11.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.500+0000 7f9148e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9144071130 con 0x7f914415bf10 2024-01-15T17:56:11.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.500+0000 7f9148e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914415f970 con 0x7f91440ccb50 2024-01-15T17:56:11.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.501+0000 7f9143400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f91440adf20 0x7f91440cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51362/0 (socket says 172.21.15.26:51362) 2024-01-15T17:56:11.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.501+0000 7f9143400640 1 -- 172.21.15.26:0/2077056913 learned_addr learned my addr 172.21.15.26:0/2077056913 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.501+0000 7f9142a00640 1 --2- 172.21.15.26:0/2077056913 >> v2:172.21.15.124:3300/0 conn(0x7f91440ccb50 0x7f9144070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.501+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875145552 0 0) 0x7f914415f4e0 con 0x7f91440adf20 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.501+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9124003520 con 0x7f91440adf20 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4178358426 0 0) 0x7f9124003520 con 0x7f91440adf20 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 >> v2:172.21.15.124:3300/0 conn(0x7f91440ccb50 msgr2=0x7f9144070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 --2- 172.21.15.26:0/2077056913 >> v2:172.21.15.124:3300/0 conn(0x7f91440ccb50 0x7f9144070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 >> v1:172.21.15.124:6789/0 conn(0x7f914415bf10 legacy=0x7f9144143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914415ebb0 con 0x7f91440adf20 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f91340019f0 con 0x7f91440adf20 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 >> v1:172.21.15.26:6789/0 conn(0x7f91440adf20 legacy=0x7f91440cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9124003970 0x7f9124003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91240047c0 0x7f9124004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9124005180 0x7f912400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914415f970 con 0x7f9124003970 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912400c120 con 0x7f9124005180 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912400c2e0 con 0x7f91240047c0 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9143400640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9124003970 0x7f9124003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.502+0000 7f9143e00640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9124005180 0x7f912400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.503+0000 7f9142a00640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91240047c0 0x7f9124004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.503+0000 7f9143400640 1 -- 172.21.15.26:0/2077056913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91240047c0 msgr2=0x7f9124004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.503+0000 7f9143400640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91240047c0 0x7f9124004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.503+0000 7f9143400640 1 -- 172.21.15.26:0/2077056913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9124005180 msgr2=0x7f912400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.503+0000 7f9143400640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9124005180 0x7f912400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.503+0000 7f9143400640 1 -- 172.21.15.26:0/2077056913 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124003520 con 0x7f9124003970 2024-01-15T17:56:11.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.503+0000 7f9143e00640 1 -- 172.21.15.26:0/2077056913 reap_dead start 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.504+0000 7f9143400640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9124003970 0x7f9124003d90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f913400a7f0 tx=0x7f91240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38136f2e2783abc3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.504+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134034630 con 0x7f9124003970 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.504+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913402f070 con 0x7f9124003970 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.504+0000 7f9142000640 1 -- 172.21.15.26:0/2077056913 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91340349f0 con 0x7f9124003970 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.505+0000 7f9148e00640 1 -- 172.21.15.26:0/2077056913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9124003970 msgr2=0x7f9124003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.505+0000 7f9148e00640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9124003970 0x7f9124003d90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f913400a7f0 tx=0x7f91240057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.505+0000 7f9148e00640 1 -- 172.21.15.26:0/2077056913 shutdown_connections 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.505+0000 7f9148e00640 1 --2- 172.21.15.26:0/2077056913 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9124003970 0x7f9124003d90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.505+0000 7f9148e00640 1 -- 172.21.15.26:0/2077056913 >> 172.21.15.26:0/2077056913 conn(0x7f914406a7d0 msgr2=0x7f914406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.505+0000 7f9148e00640 1 -- 172.21.15.26:0/2077056913 shutdown_connections 2024-01-15T17:56:11.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.505+0000 7f9148e00640 1 -- 172.21.15.26:0/2077056913 wait complete. 2024-01-15T17:56:11.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.507+0000 7f9148e00640 1 Processor -- start 2024-01-15T17:56:11.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.507+0000 7f9148e00640 1 -- start start 2024-01-15T17:56:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9148e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9124003970 0x7f91440acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9148e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91440b28a0 0x7f91440b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9148e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f914415a150 0x7f914415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9148e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440cb9c0 con 0x7f91440b28a0 2024-01-15T17:56:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9148e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440b3260 con 0x7f9124003970 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9148e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914413d800 con 0x7f914415a150 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9142a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91440b28a0 0x7f91440b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9142a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91440b28a0 0x7f91440b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59830/0 (socket says 172.21.15.26:59830) 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9142a00640 1 -- 172.21.15.26:0/341849196 learned_addr learned my addr 172.21.15.26:0/341849196 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9143400640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9124003970 0x7f91440acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9143e00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f914415a150 0x7f914415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9142a00640 1 -- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f914415a150 msgr2=0x7f914415a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.508+0000 7f9142a00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f914415a150 0x7f914415a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f9142a00640 1 -- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9124003970 msgr2=0x7f91440acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f9142a00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9124003970 0x7f91440acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f9142a00640 1 -- 172.21.15.26:0/341849196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440cb120 con 0x7f91440b28a0 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f9143e00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f914415a150 0x7f914415a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f9142a00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91440b28a0 0x7f91440b2d20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f912c0115e0 tx=0x7f912c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f913b400640 1 -- 172.21.15.26:0/341849196 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c004220 con 0x7f91440b28a0 2024-01-15T17:56:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f913b400640 1 -- 172.21.15.26:0/341849196 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c004380 con 0x7f91440b28a0 2024-01-15T17:56:11.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91440cb400 con 0x7f91440b28a0 2024-01-15T17:56:11.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f913b400640 1 -- 172.21.15.26:0/341849196 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00f930 con 0x7f91440b28a0 2024-01-15T17:56:11.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.509+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914415aad0 con 0x7f91440b28a0 2024-01-15T17:56:11.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.510+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914413d660 con 0x7f91440b28a0 2024-01-15T17:56:11.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.511+0000 7f913b400640 1 -- 172.21.15.26:0/341849196 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f912c00fa90 con 0x7f91440b28a0 2024-01-15T17:56:11.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.512+0000 7f913b400640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f911c06b2c0 0x7f911c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.512+0000 7f913b400640 1 -- 172.21.15.26:0/341849196 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f912c089e80 con 0x7f91440b28a0 2024-01-15T17:56:11.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.513+0000 7f9143400640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f911c06b2c0 0x7f911c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.514+0000 7f9143400640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f911c06b2c0 0x7f911c06d780 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9134034000 tx=0x7f9134009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:11.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.517+0000 7f913b400640 1 -- 172.21.15.26:0/341849196 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f914413d660 con 0x7f91440b28a0 2024-01-15T17:56:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.713+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f914413d660 con 0x7f91440b28a0 2024-01-15T17:56:11.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.714+0000 7f913b400640 1 -- 172.21.15.26:0/341849196 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f914413d660 con 0x7f91440b28a0 2024-01-15T17:56:11.718 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f911c06b2c0 msgr2=0x7f911c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f911c06b2c0 0x7f911c06d780 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9134034000 tx=0x7f9134009030 comp rx=0 tx=0).stop 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91440b28a0 msgr2=0x7f91440b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91440b28a0 0x7f91440b2d20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f912c0115e0 tx=0x7f912c011aa0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 shutdown_connections 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f914415a150 0x7f914415a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f911c06b2c0 0x7f911c06d780 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91440b28a0 0x7f91440b2d20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 --2- 172.21.15.26:0/341849196 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9124003970 0x7f91440acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 >> 172.21.15.26:0/341849196 conn(0x7f914406a7d0 msgr2=0x7f914415c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:11.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 shutdown_connections 2024-01-15T17:56:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.717+0000 7f9148e00640 1 -- 172.21.15.26:0/341849196 wait complete. 2024-01-15T17:56:11.741 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:56:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.963+0000 7fa0e9e00640 1 Processor -- start 2024-01-15T17:56:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.963+0000 7fa0e9e00640 1 -- start start 2024-01-15T17:56:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.964+0000 7fa0e9e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0e415d3c0 0x7fa0e40d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.964+0000 7fa0e9e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e40b4110 0x7fa0e40cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.964+0000 7fa0e9e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e40b2c70 con 0x7fa0e40cfa40 2024-01-15T17:56:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.964+0000 7fa0e9e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e40d03f0 con 0x7fa0e415d3c0 2024-01-15T17:56:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.964+0000 7fa0e9e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e40cf6d0 con 0x7fa0e40b4110 2024-01-15T17:56:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.965+0000 7fa0e8a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0e415d3c0 0x7fa0e40d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.965+0000 7fa0e8a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa0e415d3c0 0x7fa0e40d1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58574/0 (socket says 172.21.15.26:58574) 2024-01-15T17:56:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.965+0000 7fa0e8a00640 1 -- 172.21.15.26:0/1907062058 learned_addr learned my addr 172.21.15.26:0/1907062058 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.965+0000 7fa0e9400640 1 --2- 172.21.15.26:0/1907062058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e40b4110 0x7fa0e40cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.965+0000 7fa0e8a00640 1 -- 172.21.15.26:0/1907062058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e40b4110 msgr2=0x7fa0e40cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.965+0000 7fa0e8a00640 1 --2- 172.21.15.26:0/1907062058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e40b4110 0x7fa0e40cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.965+0000 7fa0e8a00640 1 -- 172.21.15.26:0/1907062058 >> v1:172.21.15.124:6789/0 conn(0x7fa0e40cfa40 legacy=0x7fa0e40cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.965+0000 7fa0e8a00640 1 -- 172.21.15.26:0/1907062058 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e40cf3f0 con 0x7fa0e415d3c0 2024-01-15T17:56:11.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.967+0000 7fa0e8a00640 1 --2- 172.21.15.26:0/1907062058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e415d3c0 0x7fa0e40d1f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009e50 tx=0x7fa0d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e68cd920b7dd51b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.968+0000 7fa0e3400640 1 -- 172.21.15.26:0/1907062058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d402ca80 con 0x7fa0e415d3c0 2024-01-15T17:56:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.968+0000 7fa0e3400640 1 -- 172.21.15.26:0/1907062058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d402cbe0 con 0x7fa0e415d3c0 2024-01-15T17:56:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.968+0000 7fa0e3400640 1 -- 172.21.15.26:0/1907062058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40343f0 con 0x7fa0e415d3c0 2024-01-15T17:56:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.968+0000 7fa0e9e00640 1 -- 172.21.15.26:0/1907062058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e415d3c0 msgr2=0x7fa0e40d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.968+0000 7fa0e9e00640 1 --2- 172.21.15.26:0/1907062058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e415d3c0 0x7fa0e40d1f20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009e50 tx=0x7fa0d402bee0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.968+0000 7fa0e9e00640 1 -- 172.21.15.26:0/1907062058 shutdown_connections 2024-01-15T17:56:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.968+0000 7fa0e9e00640 1 --2- 172.21.15.26:0/1907062058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e40b4110 0x7fa0e40cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.968+0000 7fa0e9e00640 1 --2- 172.21.15.26:0/1907062058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e415d3c0 0x7fa0e40d1f20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.968+0000 7fa0e9e00640 1 -- 172.21.15.26:0/1907062058 >> 172.21.15.26:0/1907062058 conn(0x7fa0e40d3050 msgr2=0x7fa0e40d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.969+0000 7fa0e9e00640 1 -- 172.21.15.26:0/1907062058 shutdown_connections 2024-01-15T17:56:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.969+0000 7fa0e9e00640 1 -- 172.21.15.26:0/1907062058 wait complete. 2024-01-15T17:56:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.970+0000 7fa0e9e00640 1 Processor -- start 2024-01-15T17:56:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.970+0000 7fa0e9e00640 1 -- start start 2024-01-15T17:56:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.971+0000 7fa0e9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e40b4110 0x7fa0e414a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.971+0000 7fa0e9e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0e40cfa40 0x7fa0e415daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.971+0000 7fa0e9e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e415d3c0 0x7fa0e415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.971+0000 7fa0e9e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e40c6b80 con 0x7fa0e40b4110 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.971+0000 7fa0e9e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e40cad70 con 0x7fa0e40cfa40 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.971+0000 7fa0e9e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e40cb030 con 0x7fa0e415d3c0 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e40b4110 0x7fa0e414a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e40b4110 0x7fa0e414a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58588/0 (socket says 172.21.15.26:58588) 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e8a00640 1 -- 172.21.15.26:0/312183824 learned_addr learned my addr 172.21.15.26:0/312183824 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e3e00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0e40cfa40 0x7fa0e415daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e9400640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e415d3c0 0x7fa0e415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e8a00640 1 -- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e415d3c0 msgr2=0x7fa0e415dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e8a00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e415d3c0 0x7fa0e415dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e8a00640 1 -- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0e40cfa40 msgr2=0x7fa0e415daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e8a00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0e40cfa40 0x7fa0e415daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e8a00640 1 -- 172.21.15.26:0/312183824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d4009b30 con 0x7fa0e40b4110 2024-01-15T17:56:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.972+0000 7fa0e9400640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e415d3c0 0x7fa0e415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.973+0000 7fa0e8a00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e40b4110 0x7fa0e414a310 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4000f80 tx=0x7fa0d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.973+0000 7fa0e0c00640 1 -- 172.21.15.26:0/312183824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d403e450 con 0x7fa0e40b4110 2024-01-15T17:56:11.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.973+0000 7fa0e0c00640 1 -- 172.21.15.26:0/312183824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d403e5f0 con 0x7fa0e40b4110 2024-01-15T17:56:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.973+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e40cb310 con 0x7fa0e40b4110 2024-01-15T17:56:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.973+0000 7fa0e0c00640 1 -- 172.21.15.26:0/312183824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d403e9a0 con 0x7fa0e40b4110 2024-01-15T17:56:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.973+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e40cb8d0 con 0x7fa0e40b4110 2024-01-15T17:56:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.974+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e408ba80 con 0x7fa0e40b4110 2024-01-15T17:56:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.975+0000 7fa0e0c00640 1 -- 172.21.15.26:0/312183824 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fa0d402cd50 con 0x7fa0e40b4110 2024-01-15T17:56:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.976+0000 7fa0e0c00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa0bc06b310 0x7fa0bc06d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:11.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.976+0000 7fa0e0c00640 1 -- 172.21.15.26:0/312183824 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa0d40aca00 con 0x7fa0e40b4110 2024-01-15T17:56:11.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.977+0000 7fa0e3e00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa0bc06b310 0x7fa0bc06d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:11.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.978+0000 7fa0e3e00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa0bc06b310 0x7fa0bc06d7d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc000f60 tx=0x7fa0cc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:11.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:11.980+0000 7fa0e0c00640 1 -- 172.21.15.26:0/312183824 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa0d4077900 con 0x7fa0e40b4110 2024-01-15T17:56:12.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.215+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0e4067450 con 0x7fa0e40b4110 2024-01-15T17:56:12.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.216+0000 7fa0e0c00640 1 -- 172.21.15.26:0/312183824 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7fa0d407af20 con 0x7fa0e40b4110 2024-01-15T17:56:12.220 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:12.220 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/593578542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":593578542},{"type":"v1","addr":"172.21.15.124:6837","nonce":593578542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1776325954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1776325954},{"type":"v1","addr":"172.21.15.26:6835","nonce":1776325954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:55:04.195734+0000","modified":"2024-01-15T17:55:07.244782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/3022017251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3022017251},{"type":"v1","addr":"172.21.15.124:6835","nonce":3022017251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T17:56:12.220 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.219+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa0bc06b310 msgr2=0x7fa0bc06d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.219+0000 7fa0e9e00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa0bc06b310 0x7fa0bc06d7d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc000f60 tx=0x7fa0cc009040 comp rx=0 tx=0).stop 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.219+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e40b4110 msgr2=0x7fa0e414a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.219+0000 7fa0e9e00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e40b4110 0x7fa0e414a310 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4000f80 tx=0x7fa0d4004a40 comp rx=0 tx=0).stop 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.220+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 shutdown_connections 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.220+0000 7fa0e9e00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0e415d3c0 0x7fa0e415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.220+0000 7fa0e9e00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa0bc06b310 0x7fa0bc06d7d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.220+0000 7fa0e9e00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0e40cfa40 0x7fa0e415daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.220+0000 7fa0e9e00640 1 --2- 172.21.15.26:0/312183824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0e40b4110 0x7fa0e414a310 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.220+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 >> 172.21.15.26:0/312183824 conn(0x7fa0e40d3050 msgr2=0x7fa0e40b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.220+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 shutdown_connections 2024-01-15T17:56:12.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.220+0000 7fa0e9e00640 1 -- 172.21.15.26:0/312183824 wait complete. 2024-01-15T17:56:12.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.459+0000 7f698ca00640 1 Processor -- start 2024-01-15T17:56:12.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.460+0000 7f698ca00640 1 -- start start 2024-01-15T17:56:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.461+0000 7f698ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6988160920 0x7f69880b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.461+0000 7f698ca00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f69880cfea0 0x7f6988161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.461+0000 7f698ca00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69880d0a10 con 0x7f69880cf9f0 2024-01-15T17:56:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.461+0000 7f698ca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69880d1f50 con 0x7f6988160920 2024-01-15T17:56:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.461+0000 7f698ca00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69880cf080 con 0x7f69880cfea0 2024-01-15T17:56:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.462+0000 7f6987400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6988160920 0x7f69880b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.462+0000 7f6987400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6988160920 0x7f69880b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58602/0 (socket says 172.21.15.26:58602) 2024-01-15T17:56:12.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.462+0000 7f6987400640 1 -- 172.21.15.26:0/4246055677 learned_addr learned my addr 172.21.15.26:0/4246055677 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.462+0000 7f6986a00640 1 --1- 172.21.15.26:0/4246055677 >> v1:172.21.15.26:6789/0 conn(0x7f69880cf9f0 0x7f6988161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51196/0 (socket says 172.21.15.26:51196) 2024-01-15T17:56:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.462+0000 7f6987e00640 1 --2- 172.21.15.26:0/4246055677 >> v2:172.21.15.124:3300/0 conn(0x7f69880cfea0 0x7f6988161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.462+0000 7f6987400640 1 -- 172.21.15.26:0/4246055677 >> v2:172.21.15.124:3300/0 conn(0x7f69880cfea0 msgr2=0x7f6988161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.462+0000 7f6987400640 1 --2- 172.21.15.26:0/4246055677 >> v2:172.21.15.124:3300/0 conn(0x7f69880cfea0 0x7f6988161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.462+0000 7f6987400640 1 -- 172.21.15.26:0/4246055677 >> v1:172.21.15.26:6789/0 conn(0x7f69880cf9f0 legacy=0x7f6988161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.462+0000 7f6987400640 1 -- 172.21.15.26:0/4246055677 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880d0310 con 0x7f6988160920 2024-01-15T17:56:12.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.464+0000 7f6987400640 1 --2- 172.21.15.26:0/4246055677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988160920 0x7f69880b3f30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f698808c2c0 tx=0x7f697802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b28caea39a20b64 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.464+0000 7f6986000640 1 -- 172.21.15.26:0/4246055677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978030860 con 0x7f6988160920 2024-01-15T17:56:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.464+0000 7f6986000640 1 -- 172.21.15.26:0/4246055677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69780309c0 con 0x7f6988160920 2024-01-15T17:56:12.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.464+0000 7f6986000640 1 -- 172.21.15.26:0/4246055677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978030d50 con 0x7f6988160920 2024-01-15T17:56:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.464+0000 7f698ca00640 1 -- 172.21.15.26:0/4246055677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988160920 msgr2=0x7f69880b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.464+0000 7f698ca00640 1 --2- 172.21.15.26:0/4246055677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988160920 0x7f69880b3f30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f698808c2c0 tx=0x7f697802b9a0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.465+0000 7f698ca00640 1 -- 172.21.15.26:0/4246055677 shutdown_connections 2024-01-15T17:56:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.465+0000 7f698ca00640 1 --2- 172.21.15.26:0/4246055677 >> v2:172.21.15.124:3300/0 conn(0x7f69880cfea0 0x7f6988161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.465+0000 7f698ca00640 1 --2- 172.21.15.26:0/4246055677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6988160920 0x7f69880b3f30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.465+0000 7f698ca00640 1 -- 172.21.15.26:0/4246055677 >> 172.21.15.26:0/4246055677 conn(0x7f69880d3090 msgr2=0x7f69880d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.465+0000 7f698ca00640 1 -- 172.21.15.26:0/4246055677 shutdown_connections 2024-01-15T17:56:12.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.465+0000 7f698ca00640 1 -- 172.21.15.26:0/4246055677 wait complete. 2024-01-15T17:56:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.466+0000 7f698ca00640 1 Processor -- start 2024-01-15T17:56:12.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.467+0000 7f698ca00640 1 -- start start 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f698ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69880cf9f0 0x7f6988151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f698ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69880cfea0 0x7f69880b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f698ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6988160920 0x7f69881510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f698ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880ccc00 con 0x7f69880cfea0 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f698ca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b5c50 con 0x7f69880cf9f0 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f698ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b5ea0 con 0x7f6988160920 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f6986a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69880cfea0 0x7f69880b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f6986a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69880cfea0 0x7f69880b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58618/0 (socket says 172.21.15.26:58618) 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f6986a00640 1 -- 172.21.15.26:0/1167275703 learned_addr learned my addr 172.21.15.26:0/1167275703 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:12.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f6987400640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69880cf9f0 0x7f6988151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f6986a00640 1 -- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6988160920 msgr2=0x7f69881510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f6986a00640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6988160920 0x7f69881510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f6986a00640 1 -- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69880cf9f0 msgr2=0x7f6988151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f6986a00640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69880cf9f0 0x7f6988151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.468+0000 7f6986a00640 1 -- 172.21.15.26:0/1167275703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69780094d0 con 0x7f69880cfea0 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.469+0000 7f6986a00640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69880cfea0 0x7f69880b7960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f697000c3f0 tx=0x7f697000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.469+0000 7f697f400640 1 -- 172.21.15.26:0/1167275703 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700074e0 con 0x7f69880cfea0 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.469+0000 7f697f400640 1 -- 172.21.15.26:0/1167275703 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6970007640 con 0x7f69880cfea0 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.469+0000 7f697f400640 1 -- 172.21.15.26:0/1167275703 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970004a00 con 0x7f69880cfea0 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.469+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69880b6180 con 0x7f69880cfea0 2024-01-15T17:56:12.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.469+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69880b6740 con 0x7f69880cfea0 2024-01-15T17:56:12.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.470+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6988067450 con 0x7f69880cfea0 2024-01-15T17:56:12.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.471+0000 7f697f400640 1 -- 172.21.15.26:0/1167275703 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f6970004b60 con 0x7f69880cfea0 2024-01-15T17:56:12.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.472+0000 7f697f400640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696006b2c0 0x7f696006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.472+0000 7f697f400640 1 -- 172.21.15.26:0/1167275703 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6970089610 con 0x7f69880cfea0 2024-01-15T17:56:12.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.473+0000 7f6987400640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696006b2c0 0x7f696006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:12.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.474+0000 7f6987400640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696006b2c0 0x7f696006d780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6978036000 tx=0x7f6978008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:12.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.477+0000 7f697f400640 1 -- 172.21.15.26:0/1167275703 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6970054490 con 0x7f69880cfea0 2024-01-15T17:56:12.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.672+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f698808c7e0 con 0x7f69880cfea0 2024-01-15T17:56:12.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.674+0000 7f697f400640 1 -- 172.21.15.26:0/1167275703 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f6970057ab0 con 0x7f69880cfea0 2024-01-15T17:56:12.677 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:12.678 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696006b2c0 msgr2=0x7f696006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696006b2c0 0x7f696006d780 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6978036000 tx=0x7f6978008030 comp rx=0 tx=0).stop 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69880cfea0 msgr2=0x7f69880b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69880cfea0 0x7f69880b7960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f697000c3f0 tx=0x7f697000c8b0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 shutdown_connections 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6988160920 0x7f69881510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696006b2c0 0x7f696006d780 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69880cfea0 0x7f69880b7960 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 --2- 172.21.15.26:0/1167275703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69880cf9f0 0x7f6988151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 >> 172.21.15.26:0/1167275703 conn(0x7f69880d3090 msgr2=0x7f69881616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:12.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 shutdown_connections 2024-01-15T17:56:12.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.677+0000 7f698ca00640 1 -- 172.21.15.26:0/1167275703 wait complete. 2024-01-15T17:56:12.700 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-15T17:56:12.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:56:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.917+0000 7f0df6a00640 1 Processor -- start 2024-01-15T17:56:12.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.917+0000 7f0df6a00640 1 -- start start 2024-01-15T17:56:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.919+0000 7f0df6a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0df013df80 0x7f0df015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.919+0000 7f0df6a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0df00cf9a0 0x7f0df00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.919+0000 7f0df6a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df006c470 con 0x7f0df013e360 2024-01-15T17:56:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.919+0000 7f0df6a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df006c750 con 0x7f0df00cf9a0 2024-01-15T17:56:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.919+0000 7f0df6a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df006ca30 con 0x7f0df013df80 2024-01-15T17:56:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.919+0000 7f0df4a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0df00cf9a0 0x7f0df00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.919+0000 7f0df4a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0df00cf9a0 0x7f0df00cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58632/0 (socket says 172.21.15.26:58632) 2024-01-15T17:56:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.919+0000 7f0df4a00640 1 -- 172.21.15.26:0/586250591 learned_addr learned my addr 172.21.15.26:0/586250591 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:12.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.919+0000 7f0defe00640 1 --2- 172.21.15.26:0/586250591 >> v2:172.21.15.124:3300/0 conn(0x7f0df013df80 0x7f0df015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:12.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.920+0000 7f0df4a00640 1 -- 172.21.15.26:0/586250591 >> v2:172.21.15.124:3300/0 conn(0x7f0df013df80 msgr2=0x7f0df015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:12.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.920+0000 7f0df4a00640 1 --2- 172.21.15.26:0/586250591 >> v2:172.21.15.124:3300/0 conn(0x7f0df013df80 0x7f0df015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.920+0000 7f0df4a00640 1 -- 172.21.15.26:0/586250591 >> v1:172.21.15.26:6789/0 conn(0x7f0df013e360 legacy=0x7f0df00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.920+0000 7f0df4a00640 1 -- 172.21.15.26:0/586250591 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df006cd10 con 0x7f0df00cf9a0 2024-01-15T17:56:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.921+0000 7f0df4a00640 1 --2- 172.21.15.26:0/586250591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 0x7f0df00cd840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0df0072770 tx=0x7f0de802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9ab320c46218eb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.921+0000 7f0deea00640 1 -- 172.21.15.26:0/586250591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de802cb10 con 0x7f0df00cf9a0 2024-01-15T17:56:12.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.921+0000 7f0deea00640 1 -- 172.21.15.26:0/586250591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de802cc70 con 0x7f0df00cf9a0 2024-01-15T17:56:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.921+0000 7f0deea00640 1 -- 172.21.15.26:0/586250591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de80343f0 con 0x7f0df00cf9a0 2024-01-15T17:56:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.922+0000 7f0df6a00640 1 -- 172.21.15.26:0/586250591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 msgr2=0x7f0df00cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.922+0000 7f0df6a00640 1 --2- 172.21.15.26:0/586250591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 0x7f0df00cd840 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0df0072770 tx=0x7f0de802bf70 comp rx=0 tx=0).stop 2024-01-15T17:56:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.922+0000 7f0df6a00640 1 -- 172.21.15.26:0/586250591 shutdown_connections 2024-01-15T17:56:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.922+0000 7f0df6a00640 1 --2- 172.21.15.26:0/586250591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 0x7f0df00cd840 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.922+0000 7f0df6a00640 1 --2- 172.21.15.26:0/586250591 >> v2:172.21.15.124:3300/0 conn(0x7f0df013df80 0x7f0df015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.922+0000 7f0df6a00640 1 -- 172.21.15.26:0/586250591 >> 172.21.15.26:0/586250591 conn(0x7f0df006a800 msgr2=0x7f0df006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.922+0000 7f0df6a00640 1 -- 172.21.15.26:0/586250591 shutdown_connections 2024-01-15T17:56:12.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.922+0000 7f0df6a00640 1 -- 172.21.15.26:0/586250591 wait complete. 2024-01-15T17:56:12.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.924+0000 7f0df6a00640 1 Processor -- start 2024-01-15T17:56:12.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.924+0000 7f0df6a00640 1 -- start start 2024-01-15T17:56:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.925+0000 7f0df6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 0x7f0df00a9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.925+0000 7f0df6a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0df013df80 0x7f0df00a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.925+0000 7f0df6a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0df013e360 0x7f0df01f32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.925+0000 7f0df6a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df00a9ea0 con 0x7f0df00cf9a0 2024-01-15T17:56:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.925+0000 7f0df6a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df01f4580 con 0x7f0df013e360 2024-01-15T17:56:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.925+0000 7f0df6a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df01f4840 con 0x7f0df013df80 2024-01-15T17:56:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.925+0000 7f0defe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 0x7f0df00a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:12.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.925+0000 7f0defe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 0x7f0df00a9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58636/0 (socket says 172.21.15.26:58636) 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.925+0000 7f0defe00640 1 -- 172.21.15.26:0/288805047 learned_addr learned my addr 172.21.15.26:0/288805047 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0defe00640 1 -- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0df013df80 msgr2=0x7f0df00a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0defe00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0df013df80 0x7f0df00a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0defe00640 1 -- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0df013e360 msgr2=0x7f0df01f32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0defe00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0df013e360 0x7f0df01f32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0defe00640 1 -- 172.21.15.26:0/288805047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8009b30 con 0x7f0df00cf9a0 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0defe00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 0x7f0df00a9420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0de000d800 tx=0x7f0de000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0de7e00640 1 -- 172.21.15.26:0/288805047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de000b640 con 0x7f0df00cf9a0 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0de7e00640 1 -- 172.21.15.26:0/288805047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de000b7a0 con 0x7f0df00cf9a0 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0de7e00640 1 -- 172.21.15.26:0/288805047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de000bb30 con 0x7f0df00cf9a0 2024-01-15T17:56:12.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df01f4b20 con 0x7f0df00cf9a0 2024-01-15T17:56:12.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.926+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df01f50e0 con 0x7f0df00cf9a0 2024-01-15T17:56:12.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.927+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df01f52a0 con 0x7f0df00cf9a0 2024-01-15T17:56:12.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.928+0000 7f0de7e00640 1 -- 172.21.15.26:0/288805047 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0de000bc90 con 0x7f0df00cf9a0 2024-01-15T17:56:12.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.929+0000 7f0de7e00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0dc806b2f0 0x7f0dc806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:12.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.929+0000 7f0de7e00640 1 -- 172.21.15.26:0/288805047 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0de008a1e0 con 0x7f0df00cf9a0 2024-01-15T17:56:12.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.930+0000 7f0def400640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0dc806b2f0 0x7f0dc806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:12.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.931+0000 7f0def400640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0dc806b2f0 0x7f0dc806d7b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8000f60 tx=0x7f0dd80079d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:12.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:12.933+0000 7f0de7e00640 1 -- 172.21.15.26:0/288805047 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0de0054f20 con 0x7f0df00cf9a0 2024-01-15T17:56:13.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.171+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0df00675d0 con 0x7f0df00cf9a0 2024-01-15T17:56:13.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.172+0000 7f0de7e00640 1 -- 172.21.15.26:0/288805047 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7f0de0058540 con 0x7f0df00cf9a0 2024-01-15T17:56:13.176 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:13.176 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/593578542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":593578542},{"type":"v1","addr":"172.21.15.124:6837","nonce":593578542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1776325954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1776325954},{"type":"v1","addr":"172.21.15.26:6835","nonce":1776325954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:55:04.195734+0000","modified":"2024-01-15T17:55:07.244782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/3022017251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3022017251},{"type":"v1","addr":"172.21.15.124:6835","nonce":3022017251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T17:56:13.176 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0dc806b2f0 msgr2=0x7f0dc806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0dc806b2f0 0x7f0dc806d7b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8000f60 tx=0x7f0dd80079d0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 msgr2=0x7f0df00a9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 0x7f0df00a9420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0de000d800 tx=0x7f0de000dcc0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 shutdown_connections 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0df013df80 0x7f0df00a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0dc806b2f0 0x7f0dc806d7b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0df013e360 0x7f0df01f32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 --2- 172.21.15.26:0/288805047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0df00cf9a0 0x7f0df00a9420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.175+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 >> 172.21.15.26:0/288805047 conn(0x7f0df006a800 msgr2=0x7f0df00aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:13.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.176+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 shutdown_connections 2024-01-15T17:56:13.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.176+0000 7f0df6a00640 1 -- 172.21.15.26:0/288805047 wait complete. 2024-01-15T17:56:13.197 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:56:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.419+0000 7ffb93200640 1 Processor -- start 2024-01-15T17:56:13.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.419+0000 7ffb93200640 1 -- start start 2024-01-15T17:56:13.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.421+0000 7ffb93200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ffb94143c40 0x7ffb94071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.421+0000 7ffb93200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb940ccc10 0x7ffb9406d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.421+0000 7ffb93200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb94070e50 con 0x7ffb940717d0 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.421+0000 7ffb93200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb940723c0 con 0x7ffb940ccc10 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.421+0000 7ffb93200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb94144020 con 0x7ffb94143c40 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.421+0000 7ffb92800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb940ccc10 0x7ffb9406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.421+0000 7ffb92800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb940ccc10 0x7ffb9406d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58638/0 (socket says 172.21.15.26:58638) 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.421+0000 7ffb92800640 1 -- 172.21.15.26:0/1896741095 learned_addr learned my addr 172.21.15.26:0/1896741095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.421+0000 7ffb91e00640 1 --2- 172.21.15.26:0/1896741095 >> v2:172.21.15.124:3300/0 conn(0x7ffb94143c40 0x7ffb94071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.422+0000 7ffb92800640 1 -- 172.21.15.26:0/1896741095 >> v2:172.21.15.124:3300/0 conn(0x7ffb94143c40 msgr2=0x7ffb94071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.422+0000 7ffb92800640 1 --2- 172.21.15.26:0/1896741095 >> v2:172.21.15.124:3300/0 conn(0x7ffb94143c40 0x7ffb94071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.422+0000 7ffb92800640 1 -- 172.21.15.26:0/1896741095 >> v1:172.21.15.124:6789/0 conn(0x7ffb940717d0 legacy=0x7ffb940cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:13.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.422+0000 7ffb92800640 1 -- 172.21.15.26:0/1896741095 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb94071e00 con 0x7ffb940ccc10 2024-01-15T17:56:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.423+0000 7ffb92800640 1 --2- 172.21.15.26:0/1896741095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 0x7ffb9406d160 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb9406e410 tx=0x7ffb8c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01504d7a7906e99 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.423+0000 7ffb91400640 1 -- 172.21.15.26:0/1896741095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c02c9f0 con 0x7ffb940ccc10 2024-01-15T17:56:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.423+0000 7ffb91400640 1 -- 172.21.15.26:0/1896741095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8c02cb50 con 0x7ffb940ccc10 2024-01-15T17:56:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.423+0000 7ffb91400640 1 -- 172.21.15.26:0/1896741095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c0343f0 con 0x7ffb940ccc10 2024-01-15T17:56:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.424+0000 7ffb93200640 1 -- 172.21.15.26:0/1896741095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 msgr2=0x7ffb9406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.424+0000 7ffb93200640 1 --2- 172.21.15.26:0/1896741095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 0x7ffb9406d160 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ffb9406e410 tx=0x7ffb8c02be50 comp rx=0 tx=0).stop 2024-01-15T17:56:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.424+0000 7ffb93200640 1 -- 172.21.15.26:0/1896741095 shutdown_connections 2024-01-15T17:56:13.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.424+0000 7ffb93200640 1 --2- 172.21.15.26:0/1896741095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 0x7ffb9406d160 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.424+0000 7ffb93200640 1 --2- 172.21.15.26:0/1896741095 >> v2:172.21.15.124:3300/0 conn(0x7ffb94143c40 0x7ffb94071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.424+0000 7ffb93200640 1 -- 172.21.15.26:0/1896741095 >> 172.21.15.26:0/1896741095 conn(0x7ffb9406a800 msgr2=0x7ffb9406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.424+0000 7ffb93200640 1 -- 172.21.15.26:0/1896741095 shutdown_connections 2024-01-15T17:56:13.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.424+0000 7ffb93200640 1 -- 172.21.15.26:0/1896741095 wait complete. 2024-01-15T17:56:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.426+0000 7ffb93200640 1 Processor -- start 2024-01-15T17:56:13.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.426+0000 7ffb93200640 1 -- start start 2024-01-15T17:56:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb93200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb940717d0 0x7ffb940b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb93200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 0x7ffb9413bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb93200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb94143c40 0x7ffb9413c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb93200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940b2b80 con 0x7ffb940ccc10 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb93200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9413cb20 con 0x7ffb940717d0 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb93200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940b1f00 con 0x7ffb94143c40 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb89200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 0x7ffb9413bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb89200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 0x7ffb9413bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58654/0 (socket says 172.21.15.26:58654) 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb89200640 1 -- 172.21.15.26:0/2021252757 learned_addr learned my addr 172.21.15.26:0/2021252757 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb92800640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb94143c40 0x7ffb9413c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.427+0000 7ffb91e00640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb940717d0 0x7ffb940b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb89200640 1 -- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb94143c40 msgr2=0x7ffb9413c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb89200640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb94143c40 0x7ffb9413c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb89200640 1 -- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb940717d0 msgr2=0x7ffb940b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb89200640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb940717d0 0x7ffb940b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb89200640 1 -- 172.21.15.26:0/2021252757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c009b30 con 0x7ffb940ccc10 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb91e00640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb940717d0 0x7ffb940b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb89200640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 0x7ffb9413bbe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00b3f0 tx=0x7ffb7c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb8aa00640 1 -- 172.21.15.26:0/2021252757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c011830 con 0x7ffb940ccc10 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb8aa00640 1 -- 172.21.15.26:0/2021252757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c011990 con 0x7ffb940ccc10 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb8aa00640 1 -- 172.21.15.26:0/2021252757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c011d40 con 0x7ffb940ccc10 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb940b21e0 con 0x7ffb940ccc10 2024-01-15T17:56:13.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.428+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb940b27a0 con 0x7ffb940ccc10 2024-01-15T17:56:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.430+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb940b2cc0 con 0x7ffb940ccc10 2024-01-15T17:56:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.430+0000 7ffb8aa00640 1 -- 172.21.15.26:0/2021252757 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7ffb7c01a400 con 0x7ffb940ccc10 2024-01-15T17:56:13.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.431+0000 7ffb8aa00640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb6c06b2c0 0x7ffb6c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.431+0000 7ffb8aa00640 1 -- 172.21.15.26:0/2021252757 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ffb7c089af0 con 0x7ffb940ccc10 2024-01-15T17:56:13.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.432+0000 7ffb91e00640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb6c06b2c0 0x7ffb6c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.433+0000 7ffb91e00640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb6c06b2c0 0x7ffb6c06d780 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffb84006fd0 tx=0x7ffb84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:13.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.436+0000 7ffb8aa00640 1 -- 172.21.15.26:0/2021252757 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb940b2cc0 con 0x7ffb940ccc10 2024-01-15T17:56:13.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.668+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb940b2cc0 con 0x7ffb940ccc10 2024-01-15T17:56:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.669+0000 7ffb8aa00640 1 -- 172.21.15.26:0/2021252757 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4915 (secure 0 0 0) 0x7ffb940b2cc0 con 0x7ffb940ccc10 2024-01-15T17:56:13.673 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:13.673 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/593578542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":593578542},{"type":"v1","addr":"172.21.15.124:6837","nonce":593578542}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":34148,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/3271422799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3271422799},{"type":"v1","addr":"172.21.15.26:6834","nonce":3271422799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":34151,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1776325954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1776325954},{"type":"v1","addr":"172.21.15.26:6835","nonce":1776325954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:55:04.195734+0000","modified":"2024-01-15T17:55:07.244782+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34160},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34160":{"gid":34160,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6835/3022017251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3022017251},{"type":"v1","addr":"172.21.15.124:6835","nonce":3022017251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-15T17:56:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 7 2024-01-15T17:56:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb6c06b2c0 msgr2=0x7ffb6c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb6c06b2c0 0x7ffb6c06d780 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffb84006fd0 tx=0x7ffb84008040 comp rx=0 tx=0).stop 2024-01-15T17:56:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 msgr2=0x7ffb9413bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 0x7ffb9413bbe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00b3f0 tx=0x7ffb7c00b8b0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 shutdown_connections 2024-01-15T17:56:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb94143c40 0x7ffb9413c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb6c06b2c0 0x7ffb6c06d780 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb940ccc10 0x7ffb9413bbe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 --2- 172.21.15.26:0/2021252757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb940717d0 0x7ffb940b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 >> 172.21.15.26:0/2021252757 conn(0x7ffb9406a800 msgr2=0x7ffb940cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 shutdown_connections 2024-01-15T17:56:13.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.672+0000 7ffb93200640 1 -- 172.21.15.26:0/2021252757 wait complete. 2024-01-15T17:56:13.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.914+0000 7f2bbb400640 1 Processor -- start 2024-01-15T17:56:13.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.914+0000 7f2bbb400640 1 -- start start 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.916+0000 7f2bbb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 0x7f2bb413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.916+0000 7f2bbb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2bb413fae0 0x7f2bb40b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.916+0000 7f2bbb400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb413ff00 con 0x7f2bb413cfa0 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.916+0000 7f2bbb400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb40cdc40 con 0x7f2bb413fae0 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.916+0000 7f2bbb400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb40cdec0 con 0x7f2bb415afb0 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.916+0000 7f2bba000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2bb413cfa0 0x7f2bb40cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51212/0 (socket says 172.21.15.26:51212) 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.916+0000 7f2bba000640 1 -- 172.21.15.26:0/351654257 learned_addr learned my addr 172.21.15.26:0/351654257 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.916+0000 7f2bbaa00640 1 --2- 172.21.15.26:0/351654257 >> v2:172.21.15.26:3300/0 conn(0x7f2bb413fae0 0x7f2bb40b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.917+0000 7f2bb9600640 1 --2- 172.21.15.26:0/351654257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 0x7f2bb413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.917+0000 7f2bb8c00640 1 -- 172.21.15.26:0/351654257 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592276552 0 0) 0x7f2bb413ff00 con 0x7f2bb413cfa0 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.917+0000 7f2bb8c00640 1 -- 172.21.15.26:0/351654257 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b94003520 con 0x7f2bb413cfa0 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.917+0000 7f2bbaa00640 1 -- 172.21.15.26:0/351654257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 msgr2=0x7f2bb413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.917+0000 7f2bbaa00640 1 --2- 172.21.15.26:0/351654257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 0x7f2bb413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.917+0000 7f2bbaa00640 1 -- 172.21.15.26:0/351654257 >> v1:172.21.15.26:6789/0 conn(0x7f2bb413cfa0 legacy=0x7f2bb40cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.917+0000 7f2bbaa00640 1 -- 172.21.15.26:0/351654257 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40ce1a0 con 0x7f2bb413fae0 2024-01-15T17:56:13.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.917+0000 7f2bb9600640 1 --2- 172.21.15.26:0/351654257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 0x7f2bb413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.918+0000 7f2bbaa00640 1 --2- 172.21.15.26:0/351654257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413fae0 0x7f2bb40b2040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009d90 tx=0x7f2bac02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43319045c95a0e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.918+0000 7f2bb8c00640 1 -- 172.21.15.26:0/351654257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac02cbc0 con 0x7f2bb413fae0 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.918+0000 7f2bb8c00640 1 -- 172.21.15.26:0/351654257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac02cd20 con 0x7f2bb413fae0 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.918+0000 7f2bb8c00640 1 -- 172.21.15.26:0/351654257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac034660 con 0x7f2bb413fae0 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.919+0000 7f2bbb400640 1 -- 172.21.15.26:0/351654257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413fae0 msgr2=0x7f2bb40b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.919+0000 7f2bbb400640 1 --2- 172.21.15.26:0/351654257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413fae0 0x7f2bb40b2040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009d90 tx=0x7f2bac02be10 comp rx=0 tx=0).stop 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.919+0000 7f2bbb400640 1 -- 172.21.15.26:0/351654257 shutdown_connections 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.919+0000 7f2bbb400640 1 --2- 172.21.15.26:0/351654257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413fae0 0x7f2bb40b2040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.919+0000 7f2bbb400640 1 --2- 172.21.15.26:0/351654257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 0x7f2bb413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.919+0000 7f2bbb400640 1 -- 172.21.15.26:0/351654257 >> 172.21.15.26:0/351654257 conn(0x7f2bb406a800 msgr2=0x7f2bb406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.919+0000 7f2bbb400640 1 -- 172.21.15.26:0/351654257 shutdown_connections 2024-01-15T17:56:13.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.919+0000 7f2bbb400640 1 -- 172.21.15.26:0/351654257 wait complete. 2024-01-15T17:56:13.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.921+0000 7f2bbb400640 1 Processor -- start 2024-01-15T17:56:13.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.921+0000 7f2bbb400640 1 -- start start 2024-01-15T17:56:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.922+0000 7f2bbb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413cfa0 0x7f2bb41536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.922+0000 7f2bbb400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bb413fae0 0x7f2bb415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.922+0000 7f2bbb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 0x7f2bb4159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.922+0000 7f2bbb400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb41596c0 con 0x7f2bb413cfa0 2024-01-15T17:56:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.922+0000 7f2bbb400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb4157990 con 0x7f2bb413fae0 2024-01-15T17:56:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.922+0000 7f2bbb400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb415c460 con 0x7f2bb415afb0 2024-01-15T17:56:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.922+0000 7f2bba000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413cfa0 0x7f2bb41536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bba000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413cfa0 0x7f2bb41536a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58662/0 (socket says 172.21.15.26:58662) 2024-01-15T17:56:13.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bba000640 1 -- 172.21.15.26:0/2341420994 learned_addr learned my addr 172.21.15.26:0/2341420994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bb9600640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bb413fae0 0x7f2bb415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bba000640 1 -- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 msgr2=0x7f2bb4159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bba000640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 0x7f2bb4159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bba000640 1 -- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bb413fae0 msgr2=0x7f2bb415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bba000640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bb413fae0 0x7f2bb415b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bba000640 1 -- 172.21.15.26:0/2341420994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac009a70 con 0x7f2bb413cfa0 2024-01-15T17:56:13.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bba000640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413cfa0 0x7f2bb41536a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400c530 tx=0x7f2ba400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:13.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.923+0000 7f2bb2000640 1 -- 172.21.15.26:0/2341420994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4005ce0 con 0x7f2bb413cfa0 2024-01-15T17:56:13.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.924+0000 7f2bb2000640 1 -- 172.21.15.26:0/2341420994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba4005e40 con 0x7f2bb413cfa0 2024-01-15T17:56:13.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.924+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb415bea0 con 0x7f2bb413cfa0 2024-01-15T17:56:13.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.924+0000 7f2bb2000640 1 -- 172.21.15.26:0/2341420994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40072b0 con 0x7f2bb413cfa0 2024-01-15T17:56:13.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.924+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb41592e0 con 0x7f2bb413cfa0 2024-01-15T17:56:13.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.925+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb4067450 con 0x7f2bb413cfa0 2024-01-15T17:56:13.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.926+0000 7f2bb2000640 1 -- 172.21.15.26:0/2341420994 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2ba4007410 con 0x7f2bb413cfa0 2024-01-15T17:56:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.927+0000 7f2bb2000640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b8c06b2c0 0x7f2b8c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:13.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.927+0000 7f2bb2000640 1 -- 172.21.15.26:0/2341420994 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2ba4089230 con 0x7f2bb413cfa0 2024-01-15T17:56:13.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.928+0000 7f2bb9600640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b8c06b2c0 0x7f2b8c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:13.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.929+0000 7f2bb9600640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b8c06b2c0 0x7f2b8c06d780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006fd0 tx=0x7f2b9c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:13.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:13.931+0000 7f2bb2000640 1 -- 172.21.15.26:0/2341420994 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ba40540c0 con 0x7f2bb413cfa0 2024-01-15T17:56:14.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.126+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bb40cbff0 con 0x7f2bb413cfa0 2024-01-15T17:56:14.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.129+0000 7f2bb2000640 1 -- 172.21.15.26:0/2341420994 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f2ba40576e0 con 0x7f2bb413cfa0 2024-01-15T17:56:14.132 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:14.132 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":24,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:55:51.939894+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:55:02.467034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-15T17:55:03.199274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.131+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b8c06b2c0 msgr2=0x7f2b8c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.131+0000 7f2bbb400640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b8c06b2c0 0x7f2b8c06d780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006fd0 tx=0x7f2b9c008040 comp rx=0 tx=0).stop 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.131+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413cfa0 msgr2=0x7f2bb41536a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.131+0000 7f2bbb400640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413cfa0 0x7f2bb41536a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400c530 tx=0x7f2ba400c9f0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.132+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 shutdown_connections 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.132+0000 7f2bbb400640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bb415afb0 0x7f2bb4159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.132+0000 7f2bbb400640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b8c06b2c0 0x7f2b8c06d780 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.132+0000 7f2bbb400640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bb413fae0 0x7f2bb415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.132+0000 7f2bbb400640 1 --2- 172.21.15.26:0/2341420994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bb413cfa0 0x7f2bb41536a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.132+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 >> 172.21.15.26:0/2341420994 conn(0x7f2bb406a800 msgr2=0x7f2bb406d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.132+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 shutdown_connections 2024-01-15T17:56:14.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.132+0000 7f2bbb400640 1 -- 172.21.15.26:0/2341420994 wait complete. 2024-01-15T17:56:14.156 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-15T17:56:14.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.380+0000 7f3347200640 1 Processor -- start 2024-01-15T17:56:14.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.381+0000 7f3347200640 1 -- start start 2024-01-15T17:56:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.382+0000 7f3347200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3340162430 0x7f33400b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.382+0000 7f3347200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33400702e0 con 0x7f33400b3650 2024-01-15T17:56:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.383+0000 7f3347200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33400705c0 con 0x7f3340162810 2024-01-15T17:56:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.383+0000 7f3347200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400708a0 con 0x7f3340162430 2024-01-15T17:56:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.383+0000 7f3346800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f33400b3650 0x7f334006c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51216/0 (socket says 172.21.15.26:51216) 2024-01-15T17:56:14.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.383+0000 7f3346800640 1 -- 172.21.15.26:0/2231190305 learned_addr learned my addr 172.21.15.26:0/2231190305 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:14.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.383+0000 7f3345e00640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3340162430 0x7f33400b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.383+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1300275855 0 0) 0x7f33400702e0 con 0x7f33400b3650 2024-01-15T17:56:14.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.383+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3320003520 con 0x7f33400b3650 2024-01-15T17:56:14.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2787940831 0 0) 0x7f3320003520 con 0x7f33400b3650 2024-01-15T17:56:14.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3340162430 msgr2=0x7f33400b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:14.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3340162430 0x7f33400b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 >> v1:172.21.15.124:6789/0 conn(0x7f3340162810 legacy=0x7f3340071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340070b80 con 0x7f33400b3650 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f33380019f0 con 0x7f33400b3650 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 >> v1:172.21.15.26:6789/0 conn(0x7f33400b3650 legacy=0x7f334006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33200039a0 0x7f3320003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3340162430 0x7f33200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3320004d00 0x7f3320005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400708a0 con 0x7f33200039a0 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332000ba60 con 0x7f3320004d00 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332000bc20 con 0x7f3340162430 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.384+0000 7f3345e00640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33200039a0 0x7f3320003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.385+0000 7f3346800640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3320004d00 0x7f3320005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.385+0000 7f3345e00640 1 -- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3340162430 msgr2=0x7f33200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:14.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.385+0000 7f3345e00640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3340162430 0x7f33200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.385+0000 7f3345e00640 1 -- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3320004d00 msgr2=0x7f3320005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:14.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.385+0000 7f3345e00640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3320004d00 0x7f3320005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.385+0000 7f3345e00640 1 -- 172.21.15.26:0/2231190305 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3320003520 con 0x7f33200039a0 2024-01-15T17:56:14.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.385+0000 7f3346800640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3320004d00 0x7f3320005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.386+0000 7f3345e00640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33200039a0 0x7f3320003dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3330009a90 tx=0x7f33200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71db503823739538 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.386+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333002c6b0 con 0x7f33200039a0 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.386+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333002c810 con 0x7f33200039a0 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.386+0000 7f3344a00640 1 -- 172.21.15.26:0/2231190305 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333002cba0 con 0x7f33200039a0 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.387+0000 7f3347200640 1 -- 172.21.15.26:0/2231190305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33200039a0 msgr2=0x7f3320003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.387+0000 7f3347200640 1 --2- 172.21.15.26:0/2231190305 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33200039a0 0x7f3320003dc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3330009a90 tx=0x7f33200057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.387+0000 7f3346800640 1 -- 172.21.15.26:0/2231190305 reap_dead start 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.387+0000 7f3347200640 1 -- 172.21.15.26:0/2231190305 shutdown_connections 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.387+0000 7f3347200640 1 -- 172.21.15.26:0/2231190305 >> 172.21.15.26:0/2231190305 conn(0x7f334006a800 msgr2=0x7f334006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.387+0000 7f3347200640 1 -- 172.21.15.26:0/2231190305 shutdown_connections 2024-01-15T17:56:14.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.387+0000 7f3347200640 1 -- 172.21.15.26:0/2231190305 wait complete. 2024-01-15T17:56:14.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.389+0000 7f3347200640 1 Processor -- start 2024-01-15T17:56:14.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.389+0000 7f3347200640 1 -- start start 2024-01-15T17:56:14.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.390+0000 7f3347200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33200039a0 0x7f3340150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.390+0000 7f3347200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33401f7d30 0x7f33401507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.390+0000 7f3347200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33401577f0 0x7f3340157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.390+0000 7f3347200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33401581b0 con 0x7f33401577f0 2024-01-15T17:56:14.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.390+0000 7f3347200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33401505a0 con 0x7f33200039a0 2024-01-15T17:56:14.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.390+0000 7f3347200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3340151080 con 0x7f33401f7d30 2024-01-15T17:56:14.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.390+0000 7f3346800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33401577f0 0x7f3340157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.390+0000 7f3346800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33401577f0 0x7f3340157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58674/0 (socket says 172.21.15.26:58674) 2024-01-15T17:56:14.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.390+0000 7f3346800640 1 -- 172.21.15.26:0/3767389668 learned_addr learned my addr 172.21.15.26:0/3767389668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3345e00640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33200039a0 0x7f3340150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3345400640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33401f7d30 0x7f33401507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3346800640 1 -- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33401f7d30 msgr2=0x7f33401507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3346800640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33401f7d30 0x7f33401507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3346800640 1 -- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33200039a0 msgr2=0x7f3340150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3346800640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33200039a0 0x7f3340150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3346800640 1 -- 172.21.15.26:0/3767389668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3340151360 con 0x7f33401577f0 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3345e00640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33200039a0 0x7f3340150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3345400640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33401f7d30 0x7f33401507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:14.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f3346800640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33401577f0 0x7f3340157c70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f33400b3d60 tx=0x7f333800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f333e000640 1 -- 172.21.15.26:0/3767389668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338004d60 con 0x7f33401577f0 2024-01-15T17:56:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.391+0000 7f333e000640 1 -- 172.21.15.26:0/3767389668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333800bb80 con 0x7f33401577f0 2024-01-15T17:56:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.392+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3340151640 con 0x7f33401577f0 2024-01-15T17:56:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.392+0000 7f333e000640 1 -- 172.21.15.26:0/3767389668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3338007070 con 0x7f33401577f0 2024-01-15T17:56:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.392+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3340150ce0 con 0x7f33401577f0 2024-01-15T17:56:14.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.393+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3340151b60 con 0x7f33401577f0 2024-01-15T17:56:14.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.394+0000 7f333e000640 1 -- 172.21.15.26:0/3767389668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f33380071d0 con 0x7f33401577f0 2024-01-15T17:56:14.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.395+0000 7f333e000640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f331806b2f0 0x7f331806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.395+0000 7f333e000640 1 -- 172.21.15.26:0/3767389668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f333808a070 con 0x7f33401577f0 2024-01-15T17:56:14.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.395+0000 7f3345e00640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f331806b2f0 0x7f331806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.396+0000 7f3345e00640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f331806b2f0 0x7f331806d7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3330036000 tx=0x7f3330030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:14.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.399+0000 7f333e000640 1 -- 172.21.15.26:0/3767389668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3340151b60 con 0x7f33401577f0 2024-01-15T17:56:14.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.632+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3340151b60 con 0x7f33401577f0 2024-01-15T17:56:14.637 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-01-15T17:56:14.634+0000 7f9819c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T17:56:14.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.637+0000 7f333e000640 1 -- 172.21.15.26:0/3767389668 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3340151b60 con 0x7f33401577f0 2024-01-15T17:56:14.640 INFO:teuthology.orchestra.run.smithi026.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-15T17:56:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.641+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f331806b2f0 msgr2=0x7f331806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.641+0000 7f3347200640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f331806b2f0 0x7f331806d7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3330036000 tx=0x7f3330030000 comp rx=0 tx=0).stop 2024-01-15T17:56:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.641+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33401577f0 msgr2=0x7f3340157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.641+0000 7f3347200640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33401577f0 0x7f3340157c70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f33400b3d60 tx=0x7f333800b4a0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.642+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 shutdown_connections 2024-01-15T17:56:14.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.642+0000 7f3347200640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33401f7d30 0x7f33401507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.642+0000 7f3347200640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f331806b2f0 0x7f331806d7b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.642+0000 7f3347200640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33401577f0 0x7f3340157c70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.642+0000 7f3347200640 1 --2- 172.21.15.26:0/3767389668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33200039a0 0x7f3340150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.642+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 >> 172.21.15.26:0/3767389668 conn(0x7f334006a800 msgr2=0x7f33400ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.642+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 shutdown_connections 2024-01-15T17:56:14.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.642+0000 7f3347200640 1 -- 172.21.15.26:0/3767389668 wait complete. 2024-01-15T17:56:14.666 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-15T17:56:14.670 INFO:tasks.ceph.mds.b.smithi124.stderr: -3911> 2024-01-15T17:55:08.303+0000 7f3934800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T17:56:14.769 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T17:56:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.886+0000 7fc0dd600640 1 Processor -- start 2024-01-15T17:56:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.887+0000 7fc0dd600640 1 -- start start 2024-01-15T17:56:14.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.888+0000 7fc0dd600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0d80cf9a0 0x7fc0d815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.888+0000 7fc0dd600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0d815e1e0 0x7fc0d80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.888+0000 7fc0dd600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0d80b1ee0 con 0x7fc0d8143ae0 2024-01-15T17:56:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.888+0000 7fc0dd600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d80b2780 con 0x7fc0d815e1e0 2024-01-15T17:56:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.888+0000 7fc0dd600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d815e990 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d6a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc0d8143ae0 0x7fc0d80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51218/0 (socket says 172.21.15.26:51218) 2024-01-15T17:56:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d6a00640 1 -- 172.21.15.26:0/1562866686 learned_addr learned my addr 172.21.15.26:0/1562866686 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d7400640 1 --2- 172.21.15.26:0/1562866686 >> v2:172.21.15.26:3300/0 conn(0x7fc0d815e1e0 0x7fc0d80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d6000640 1 --2- 172.21.15.26:0/1562866686 >> v2:172.21.15.124:3300/0 conn(0x7fc0d80cf9a0 0x7fc0d815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d5600640 1 -- 172.21.15.26:0/1562866686 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226401622 0 0) 0x7fc0d80b1ee0 con 0x7fc0d8143ae0 2024-01-15T17:56:14.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d5600640 1 -- 172.21.15.26:0/1562866686 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b8005730 con 0x7fc0d8143ae0 2024-01-15T17:56:14.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d7400640 1 -- 172.21.15.26:0/1562866686 >> v2:172.21.15.124:3300/0 conn(0x7fc0d80cf9a0 msgr2=0x7fc0d815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:14.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d7400640 1 --2- 172.21.15.26:0/1562866686 >> v2:172.21.15.124:3300/0 conn(0x7fc0d80cf9a0 0x7fc0d815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d7400640 1 -- 172.21.15.26:0/1562866686 >> v1:172.21.15.26:6789/0 conn(0x7fc0d8143ae0 legacy=0x7fc0d80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.889+0000 7fc0d7400640 1 -- 172.21.15.26:0/1562866686 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d80b2130 con 0x7fc0d815e1e0 2024-01-15T17:56:14.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.890+0000 7fc0d7400640 1 --2- 172.21.15.26:0/1562866686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d815e1e0 0x7fc0d80702c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80cb830 tx=0x7fc0d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2337bee034ca742c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0d5600640 1 -- 172.21.15.26:0/1562866686 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0031830 con 0x7fc0d815e1e0 2024-01-15T17:56:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0d5600640 1 -- 172.21.15.26:0/1562866686 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d0031990 con 0x7fc0d815e1e0 2024-01-15T17:56:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0d5600640 1 -- 172.21.15.26:0/1562866686 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0031d20 con 0x7fc0d815e1e0 2024-01-15T17:56:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0dd600640 1 -- 172.21.15.26:0/1562866686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d815e1e0 msgr2=0x7fc0d80702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0dd600640 1 --2- 172.21.15.26:0/1562866686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d815e1e0 0x7fc0d80702c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc0d80cb830 tx=0x7fc0d002b8d0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0dd600640 1 -- 172.21.15.26:0/1562866686 shutdown_connections 2024-01-15T17:56:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0dd600640 1 --2- 172.21.15.26:0/1562866686 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d815e1e0 0x7fc0d80702c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0dd600640 1 --2- 172.21.15.26:0/1562866686 >> v2:172.21.15.124:3300/0 conn(0x7fc0d80cf9a0 0x7fc0d815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0dd600640 1 -- 172.21.15.26:0/1562866686 >> 172.21.15.26:0/1562866686 conn(0x7fc0d806a800 msgr2=0x7fc0d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0dd600640 1 -- 172.21.15.26:0/1562866686 shutdown_connections 2024-01-15T17:56:14.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.891+0000 7fc0dd600640 1 -- 172.21.15.26:0/1562866686 wait complete. 2024-01-15T17:56:14.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.893+0000 7fc0dd600640 1 Processor -- start 2024-01-15T17:56:14.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.893+0000 7fc0dd600640 1 -- start start 2024-01-15T17:56:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.894+0000 7fc0dd600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80cf9a0 0x7fc0d81341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.894+0000 7fc0dd600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d8143ae0 0x7fc0d81302d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.894+0000 7fc0dd600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d815e1e0 0x7fc0d8140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.894+0000 7fc0dd600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d80ab6d0 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.894+0000 7fc0dd600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d81f1ed0 con 0x7fc0d8143ae0 2024-01-15T17:56:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.894+0000 7fc0dd600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d81f2170 con 0x7fc0d815e1e0 2024-01-15T17:56:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80cf9a0 0x7fc0d81341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80cf9a0 0x7fc0d81341e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58692/0 (socket says 172.21.15.26:58692) 2024-01-15T17:56:14.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6a00640 1 -- 172.21.15.26:0/3293545364 learned_addr learned my addr 172.21.15.26:0/3293545364 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6000640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d8143ae0 0x7fc0d81302d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6a00640 1 -- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d815e1e0 msgr2=0x7fc0d8140990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6a00640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d815e1e0 0x7fc0d8140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6a00640 1 -- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d8143ae0 msgr2=0x7fc0d81302d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6a00640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d8143ae0 0x7fc0d81302d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6a00640 1 -- 172.21.15.26:0/3293545364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d0009400 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.895+0000 7fc0d6a00640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80cf9a0 0x7fc0d81341e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800c500 tx=0x7fc0c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.896+0000 7fc0cea00640 1 -- 172.21.15.26:0/3293545364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8005ce0 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.896+0000 7fc0cea00640 1 -- 172.21.15.26:0/3293545364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c8005e40 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.896+0000 7fc0cea00640 1 -- 172.21.15.26:0/3293545364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8007330 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.896+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d8130810 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.896+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d8130d90 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.897+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d80675d0 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.898+0000 7fc0cea00640 1 -- 172.21.15.26:0/3293545364 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fc0c8007490 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.899+0000 7fc0cea00640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006b2c0 0x7fc0b006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:14.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.899+0000 7fc0cea00640 1 -- 172.21.15.26:0/3293545364 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fc0c8089340 con 0x7fc0d80cf9a0 2024-01-15T17:56:14.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.899+0000 7fc0d6000640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006b2c0 0x7fc0b006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:14.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.901+0000 7fc0d6000640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006b2c0 0x7fc0b006d780 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc0d81f1e30 tx=0x7fc0c000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:14.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:14.904+0000 7fc0cea00640 1 -- 172.21.15.26:0/3293545364 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0c8054170 con 0x7fc0d80cf9a0 2024-01-15T17:56:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.141+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc0d806af60 con 0x7fc0d80cf9a0 2024-01-15T17:56:15.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.640+0000 7fc0cea00640 1 -- 172.21.15.26:0/3293545364 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc0c8057790 con 0x7fc0d80cf9a0 2024-01-15T17:56:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006b2c0 msgr2=0x7fc0b006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006b2c0 0x7fc0b006d780 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc0d81f1e30 tx=0x7fc0c000b040 comp rx=0 tx=0).stop 2024-01-15T17:56:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80cf9a0 msgr2=0x7fc0d81341e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80cf9a0 0x7fc0d81341e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800c500 tx=0x7fc0c800c9c0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 shutdown_connections 2024-01-15T17:56:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d815e1e0 0x7fc0d8140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006b2c0 0x7fc0b006d780 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d8143ae0 0x7fc0d81302d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 --2- 172.21.15.26:0/3293545364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80cf9a0 0x7fc0d81341e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 >> 172.21.15.26:0/3293545364 conn(0x7fc0d806a800 msgr2=0x7fc0d806ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 shutdown_connections 2024-01-15T17:56:15.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.644+0000 7fc0dd600640 1 -- 172.21.15.26:0/3293545364 wait complete. 2024-01-15T17:56:15.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-15T17:56:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.900+0000 7f1408800640 1 Processor -- start 2024-01-15T17:56:15.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.900+0000 7f1408800640 1 -- start start 2024-01-15T17:56:15.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.901+0000 7f1408800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f14000ae5a0 0x7f140015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.901+0000 7f1408800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f14000cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.901+0000 7f1408800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1400070c40 con 0x7f14000b3ae0 2024-01-15T17:56:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.901+0000 7f1408800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1400071450 con 0x7f14000ae5a0 2024-01-15T17:56:15.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.901+0000 7f1408800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000cfc60 con 0x7f14000cf290 2024-01-15T17:56:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.902+0000 7f1406800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f14000cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.902+0000 7f1406800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f14000cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:40822/0 (socket says 172.21.15.26:40822) 2024-01-15T17:56:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.902+0000 7f1406800640 1 -- 172.21.15.26:0/1285956104 learned_addr learned my addr 172.21.15.26:0/1285956104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.902+0000 7f1405e00640 1 --2- 172.21.15.26:0/1285956104 >> v2:172.21.15.124:3300/0 conn(0x7f14000ae5a0 0x7f140015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.903+0000 7f1406800640 1 -- 172.21.15.26:0/1285956104 >> v2:172.21.15.124:3300/0 conn(0x7f14000ae5a0 msgr2=0x7f140015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:15.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.903+0000 7f1406800640 1 --2- 172.21.15.26:0/1285956104 >> v2:172.21.15.124:3300/0 conn(0x7f14000ae5a0 0x7f140015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.903+0000 7f1406800640 1 -- 172.21.15.26:0/1285956104 >> v1:172.21.15.124:6789/0 conn(0x7f14000b3ae0 legacy=0x7f140006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.903+0000 7f1406800640 1 -- 172.21.15.26:0/1285956104 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140006c470 con 0x7f14000cf290 2024-01-15T17:56:15.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.904+0000 7f1406800640 1 --2- 172.21.15.26:0/1285956104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f14000cf720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009870 tx=0x7f13f802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39772188929c9eb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.905+0000 7f1404a00640 1 -- 172.21.15.26:0/1285956104 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f802f830 con 0x7f14000cf290 2024-01-15T17:56:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.905+0000 7f1404a00640 1 -- 172.21.15.26:0/1285956104 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f802f990 con 0x7f14000cf290 2024-01-15T17:56:15.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.905+0000 7f1404a00640 1 -- 172.21.15.26:0/1285956104 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f802fd20 con 0x7f14000cf290 2024-01-15T17:56:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.906+0000 7f1408800640 1 -- 172.21.15.26:0/1285956104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 msgr2=0x7f14000cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.906+0000 7f1408800640 1 --2- 172.21.15.26:0/1285956104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f14000cf720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13f8009870 tx=0x7f13f802ba10 comp rx=0 tx=0).stop 2024-01-15T17:56:15.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.906+0000 7f1408800640 1 -- 172.21.15.26:0/1285956104 shutdown_connections 2024-01-15T17:56:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.906+0000 7f1408800640 1 --2- 172.21.15.26:0/1285956104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f14000cf720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.906+0000 7f1408800640 1 --2- 172.21.15.26:0/1285956104 >> v2:172.21.15.124:3300/0 conn(0x7f14000ae5a0 0x7f140015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.906+0000 7f1408800640 1 -- 172.21.15.26:0/1285956104 >> 172.21.15.26:0/1285956104 conn(0x7f140006a800 msgr2=0x7f140006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.906+0000 7f1408800640 1 -- 172.21.15.26:0/1285956104 shutdown_connections 2024-01-15T17:56:15.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.906+0000 7f1408800640 1 -- 172.21.15.26:0/1285956104 wait complete. 2024-01-15T17:56:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.908+0000 7f1408800640 1 Processor -- start 2024-01-15T17:56:15.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.908+0000 7f1408800640 1 -- start start 2024-01-15T17:56:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.909+0000 7f1408800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14000ae5a0 0x7f1400153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.909+0000 7f1408800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14000b3ae0 0x7f140015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.909+0000 7f1408800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f1400158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.909+0000 7f1408800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1400153680 con 0x7f14000ae5a0 2024-01-15T17:56:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.909+0000 7f1408800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14001f7640 con 0x7f14000b3ae0 2024-01-15T17:56:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.909+0000 7f1408800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14001f78e0 con 0x7f14000cf290 2024-01-15T17:56:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.909+0000 7f1405e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14000ae5a0 0x7f1400153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:15.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.909+0000 7f1405e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14000ae5a0 0x7f1400153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58700/0 (socket says 172.21.15.26:58700) 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.909+0000 7f1405e00640 1 -- 172.21.15.26:0/76068354 learned_addr learned my addr 172.21.15.26:0/76068354 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f1406800640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f1400158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f1405400640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14000b3ae0 0x7f140015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f1405e00640 1 -- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 msgr2=0x7f1400158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f1405e00640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f1400158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f1405e00640 1 -- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14000b3ae0 msgr2=0x7f140015fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f1405e00640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14000b3ae0 0x7f140015fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f1405e00640 1 -- 172.21.15.26:0/76068354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f80094d0 con 0x7f14000ae5a0 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f1405400640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14000b3ae0 0x7f140015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:15.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f1405e00640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14000ae5a0 0x7f1400153140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f13f000e9b0 tx=0x7f13f000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.910+0000 7f13fe000640 1 -- 172.21.15.26:0/76068354 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0009670 con 0x7f14000ae5a0 2024-01-15T17:56:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.911+0000 7f13fe000640 1 -- 172.21.15.26:0/76068354 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f000c9a0 con 0x7f14000ae5a0 2024-01-15T17:56:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.911+0000 7f13fe000640 1 -- 172.21.15.26:0/76068354 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000cd50 con 0x7f14000ae5a0 2024-01-15T17:56:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.911+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14001f7bc0 con 0x7f14000ae5a0 2024-01-15T17:56:15.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.911+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14001f8180 con 0x7f14000ae5a0 2024-01-15T17:56:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.912+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14000675b0 con 0x7f14000ae5a0 2024-01-15T17:56:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.913+0000 7f13fe000640 1 -- 172.21.15.26:0/76068354 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f13f0013470 con 0x7f14000ae5a0 2024-01-15T17:56:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.914+0000 7f13fe000640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13d806b2c0 0x7f13d806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.914+0000 7f13fe000640 1 -- 172.21.15.26:0/76068354 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f13f008a010 con 0x7f14000ae5a0 2024-01-15T17:56:15.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.914+0000 7f1405400640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13d806b2c0 0x7f13d806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:15.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.915+0000 7f1405400640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13d806b2c0 0x7f13d806d780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f140015a6e0 tx=0x7f13e8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:15.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:15.919+0000 7f13fe000640 1 -- 172.21.15.26:0/76068354 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13f0054cf0 con 0x7f14000ae5a0 2024-01-15T17:56:16.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:16.117+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1400161be0 con 0x7f14000ae5a0 2024-01-15T17:56:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.094+0000 7f13fe000640 1 -- 172.21.15.26:0/76068354 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f13f00170d0 con 0x7f14000ae5a0 2024-01-15T17:56:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' removed 2024-01-15T17:56:17.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13d806b2c0 msgr2=0x7f13d806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13d806b2c0 0x7f13d806d780 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f140015a6e0 tx=0x7f13e8009040 comp rx=0 tx=0).stop 2024-01-15T17:56:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14000ae5a0 msgr2=0x7f1400153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14000ae5a0 0x7f1400153140 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f13f000e9b0 tx=0x7f13f000ee70 comp rx=0 tx=0).stop 2024-01-15T17:56:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 shutdown_connections 2024-01-15T17:56:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14000cf290 0x7f1400158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13d806b2c0 0x7f13d806d780 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14000b3ae0 0x7f140015fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 --2- 172.21.15.26:0/76068354 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14000ae5a0 0x7f1400153140 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.098+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 >> 172.21.15.26:0/76068354 conn(0x7f140006a800 msgr2=0x7f14000b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:17.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.102+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 shutdown_connections 2024-01-15T17:56:17.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.103+0000 7f1408800640 1 -- 172.21.15.26:0/76068354 wait complete. 2024-01-15T17:56:17.128 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-15T17:56:17.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.336+0000 7f1555400640 1 Processor -- start 2024-01-15T17:56:17.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.337+0000 7f1555400640 1 -- start start 2024-01-15T17:56:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.338+0000 7f1555400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1550071410 0x7f1550146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.338+0000 7f1555400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f155006c7c0 0x7f15500717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.338+0000 7f1555400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1550071d30 0x7f15500b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.338+0000 7f1555400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155006cc50 con 0x7f155006c7c0 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.338+0000 7f1555400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155006cec0 con 0x7f1550071410 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.338+0000 7f1555400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155006d1a0 con 0x7f1550071d30 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f154f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f155006c7c0 0x7f15500717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f154f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f155006c7c0 0x7f15500717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58708/0 (socket says 172.21.15.26:58708) 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f154f400640 1 -- 172.21.15.26:0/3922325504 learned_addr learned my addr 172.21.15.26:0/3922325504 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f154fe00640 1 --2- 172.21.15.26:0/3922325504 >> v2:172.21.15.124:3300/0 conn(0x7f1550071410 0x7f1550146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f1554a00640 1 --2- 172.21.15.26:0/3922325504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1550071d30 0x7f15500b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f154f400640 1 -- 172.21.15.26:0/3922325504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1550071d30 msgr2=0x7f15500b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:17.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f154f400640 1 --2- 172.21.15.26:0/3922325504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1550071d30 0x7f15500b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f154f400640 1 -- 172.21.15.26:0/3922325504 >> v2:172.21.15.124:3300/0 conn(0x7f1550071410 msgr2=0x7f1550146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:17.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f154f400640 1 --2- 172.21.15.26:0/3922325504 >> v2:172.21.15.124:3300/0 conn(0x7f1550071410 0x7f1550146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.339+0000 7f154f400640 1 -- 172.21.15.26:0/3922325504 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155006d480 con 0x7f155006c7c0 2024-01-15T17:56:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.341+0000 7f154f400640 1 --2- 172.21.15.26:0/3922325504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f155006c7c0 0x7f15500717f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f15500ad5f0 tx=0x7f153802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194683ebe3e441cc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.341+0000 7f154ea00640 1 -- 172.21.15.26:0/3922325504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538031860 con 0x7f155006c7c0 2024-01-15T17:56:17.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.341+0000 7f154ea00640 1 -- 172.21.15.26:0/3922325504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15380319c0 con 0x7f155006c7c0 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.341+0000 7f154ea00640 1 -- 172.21.15.26:0/3922325504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538031d50 con 0x7f155006c7c0 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.342+0000 7f1555400640 1 -- 172.21.15.26:0/3922325504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f155006c7c0 msgr2=0x7f15500717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.342+0000 7f1555400640 1 --2- 172.21.15.26:0/3922325504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f155006c7c0 0x7f15500717f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f15500ad5f0 tx=0x7f153802b900 comp rx=0 tx=0).stop 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.342+0000 7f1555400640 1 -- 172.21.15.26:0/3922325504 shutdown_connections 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.342+0000 7f1555400640 1 --2- 172.21.15.26:0/3922325504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1550071d30 0x7f15500b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.342+0000 7f1555400640 1 --2- 172.21.15.26:0/3922325504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f155006c7c0 0x7f15500717f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.342+0000 7f1555400640 1 --2- 172.21.15.26:0/3922325504 >> v2:172.21.15.124:3300/0 conn(0x7f1550071410 0x7f1550146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.342+0000 7f1555400640 1 -- 172.21.15.26:0/3922325504 >> 172.21.15.26:0/3922325504 conn(0x7f155006a800 msgr2=0x7f155006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.342+0000 7f1555400640 1 -- 172.21.15.26:0/3922325504 shutdown_connections 2024-01-15T17:56:17.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.342+0000 7f1555400640 1 -- 172.21.15.26:0/3922325504 wait complete. 2024-01-15T17:56:17.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.344+0000 7f1555400640 1 Processor -- start 2024-01-15T17:56:17.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.344+0000 7f1555400640 1 -- start start 2024-01-15T17:56:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.345+0000 7f1555400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f155006c7c0 0x7f15501f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.345+0000 7f1555400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1550071410 0x7f155015d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.345+0000 7f1555400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550071d30 0x7f155015c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.345+0000 7f1555400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155015db70 con 0x7f1550071d30 2024-01-15T17:56:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.345+0000 7f1555400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155015cf10 con 0x7f1550071410 2024-01-15T17:56:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.345+0000 7f1555400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15501412e0 con 0x7f155006c7c0 2024-01-15T17:56:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.345+0000 7f1554a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550071d30 0x7f155015c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:17.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.345+0000 7f1554a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550071d30 0x7f155015c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58714/0 (socket says 172.21.15.26:58714) 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.345+0000 7f1554a00640 1 -- 172.21.15.26:0/293028930 learned_addr learned my addr 172.21.15.26:0/293028930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f154f400640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1550071410 0x7f155015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f154fe00640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f155006c7c0 0x7f15501f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f1554a00640 1 -- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f155006c7c0 msgr2=0x7f15501f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f1554a00640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f155006c7c0 0x7f15501f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f1554a00640 1 -- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1550071410 msgr2=0x7f155015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f1554a00640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1550071410 0x7f155015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f1554a00640 1 -- 172.21.15.26:0/293028930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15380094d0 con 0x7f1550071d30 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f154f400640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1550071410 0x7f155015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f154fe00640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f155006c7c0 0x7f15501f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f1554a00640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550071d30 0x7f155015c9d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f154800d5d0 tx=0x7f154800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:17.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f1547e00640 1 -- 172.21.15.26:0/293028930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15480125f0 con 0x7f1550071d30 2024-01-15T17:56:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f1547e00640 1 -- 172.21.15.26:0/293028930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1548012770 con 0x7f1550071d30 2024-01-15T17:56:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.346+0000 7f1547e00640 1 -- 172.21.15.26:0/293028930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548012b20 con 0x7f1550071d30 2024-01-15T17:56:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.347+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1550160020 con 0x7f1550071d30 2024-01-15T17:56:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.347+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15501605d0 con 0x7f1550071d30 2024-01-15T17:56:17.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.348+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1550067450 con 0x7f1550071d30 2024-01-15T17:56:17.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.349+0000 7f1547e00640 1 -- 172.21.15.26:0/293028930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1548012c80 con 0x7f1550071d30 2024-01-15T17:56:17.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.349+0000 7f1547e00640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f152806b2c0 0x7f152806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:17.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.350+0000 7f1547e00640 1 -- 172.21.15.26:0/293028930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f1548089f20 con 0x7f1550071d30 2024-01-15T17:56:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.350+0000 7f154fe00640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f152806b2c0 0x7f152806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.351+0000 7f154fe00640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f152806b2c0 0x7f152806d780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1540007330 tx=0x7f154000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.354+0000 7f1547e00640 1 -- 172.21.15.26:0/293028930 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1548054e70 con 0x7f1550071d30 2024-01-15T17:56:17.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:17.546+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f155013f140 con 0x7f1550071d30 2024-01-15T17:56:18.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.106+0000 7f1547e00640 1 -- 172.21.15.26:0/293028930 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1548058490 con 0x7f1550071d30 2024-01-15T17:56:18.110 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' removed 2024-01-15T17:56:18.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f152806b2c0 msgr2=0x7f152806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:18.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f152806b2c0 0x7f152806d780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1540007330 tx=0x7f154000b040 comp rx=0 tx=0).stop 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550071d30 msgr2=0x7f155015c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550071d30 0x7f155015c9d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f154800d5d0 tx=0x7f154800da90 comp rx=0 tx=0).stop 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 shutdown_connections 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f152806b2c0 0x7f152806d780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1550071d30 0x7f155015c9d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1550071410 0x7f155015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 --2- 172.21.15.26:0/293028930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f155006c7c0 0x7f15501f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 >> 172.21.15.26:0/293028930 conn(0x7f155006a800 msgr2=0x7f15501440a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 shutdown_connections 2024-01-15T17:56:18.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.110+0000 7f1555400640 1 -- 172.21.15.26:0/293028930 wait complete. 2024-01-15T17:56:18.143 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T17:56:18.143 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T17:56:18.143 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T17:56:18.143 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:56:18.144 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T17:56:18.144 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T17:56:18.144 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T17:56:18.144 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:56:18.144 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T17:56:18.145 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T17:56:18.145 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T17:56:18.145 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:56:18.145 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T17:56:18.145 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T17:56:18.146 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T17:56:18.146 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:56:18.161 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T17:56:18.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T17:56:18.164 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T17:56:18.164 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T17:56:18.166 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T17:56:18.166 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T17:56:18.167 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T17:56:18.167 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T17:56:18.212 INFO:tasks.ceph.mds.c:Started 2024-01-15T17:56:18.213 DEBUG:teuthology.parallel:result is None 2024-01-15T17:56:18.214 INFO:tasks.ceph.mds.d:Started 2024-01-15T17:56:18.214 DEBUG:teuthology.parallel:result is None 2024-01-15T17:56:18.219 INFO:tasks.ceph.mds.b:Started 2024-01-15T17:56:18.220 INFO:tasks.ceph.mds.a:Started 2024-01-15T17:56:18.220 DEBUG:teuthology.parallel:result is None 2024-01-15T17:56:18.220 DEBUG:teuthology.parallel:result is None 2024-01-15T17:56:18.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T17:56:18.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-01-15T17:56:18.369 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-01-15T17:56:18.371 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T17:56:18.374 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T17:56:18.375 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T17:56:18.377 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-15T17:56:18.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.409+0000 7fb95f800640 1 Processor -- start 2024-01-15T17:56:18.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.409+0000 7fb95f800640 1 -- start start 2024-01-15T17:56:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.411+0000 7fb95f800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb95813df80 0x7fb95815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.411+0000 7fb95f800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 0x7fb9580cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.411+0000 7fb95f800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95806c470 con 0x7fb95815f9c0 2024-01-15T17:56:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.411+0000 7fb95f800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95806c750 con 0x7fb95813df80 2024-01-15T17:56:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.411+0000 7fb95f800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95806ca30 con 0x7fb95813e360 2024-01-15T17:56:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.411+0000 7fb95d800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb95815f9c0 0x7fb9581408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51242/0 (socket says 172.21.15.26:51242) 2024-01-15T17:56:18.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.411+0000 7fb95d800640 1 -- 172.21.15.26:0/3010906895 learned_addr learned my addr 172.21.15.26:0/3010906895 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.411+0000 7fb95ce00640 1 --2- 172.21.15.26:0/3010906895 >> v2:172.21.15.124:3300/0 conn(0x7fb95813df80 0x7fb95815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.411+0000 7fb94fc00640 1 --2- 172.21.15.26:0/3010906895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 0x7fb9580cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.412+0000 7fb957e00640 1 -- 172.21.15.26:0/3010906895 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339678400 0 0) 0x7fb95806c470 con 0x7fb95815f9c0 2024-01-15T17:56:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.412+0000 7fb957e00640 1 -- 172.21.15.26:0/3010906895 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb938003520 con 0x7fb95815f9c0 2024-01-15T17:56:18.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.412+0000 7fb95ce00640 1 -- 172.21.15.26:0/3010906895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 msgr2=0x7fb9580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:18.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.412+0000 7fb95ce00640 1 --2- 172.21.15.26:0/3010906895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 0x7fb9580cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.412+0000 7fb95ce00640 1 -- 172.21.15.26:0/3010906895 >> v1:172.21.15.26:6789/0 conn(0x7fb95815f9c0 legacy=0x7fb9581408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:18.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.412+0000 7fb95ce00640 1 -- 172.21.15.26:0/3010906895 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95806cd10 con 0x7fb95813df80 2024-01-15T17:56:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.413+0000 7fb95ce00640 1 --2- 172.21.15.26:0/3010906895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 0x7fb95815bee0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb9480097a0 tx=0x7fb94802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af326f5ff39512a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.413+0000 7fb957e00640 1 -- 172.21.15.26:0/3010906895 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94802d040 con 0x7fb95813df80 2024-01-15T17:56:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.413+0000 7fb957e00640 1 -- 172.21.15.26:0/3010906895 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb948031810 con 0x7fb95813df80 2024-01-15T17:56:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.413+0000 7fb957e00640 1 -- 172.21.15.26:0/3010906895 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948031ba0 con 0x7fb95813df80 2024-01-15T17:56:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.414+0000 7fb95f800640 1 -- 172.21.15.26:0/3010906895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 msgr2=0x7fb95815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.414+0000 7fb95f800640 1 --2- 172.21.15.26:0/3010906895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 0x7fb95815bee0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb9480097a0 tx=0x7fb94802b8b0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.414+0000 7fb95f800640 1 -- 172.21.15.26:0/3010906895 shutdown_connections 2024-01-15T17:56:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.414+0000 7fb95f800640 1 --2- 172.21.15.26:0/3010906895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 0x7fb9580cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.414+0000 7fb95f800640 1 --2- 172.21.15.26:0/3010906895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 0x7fb95815bee0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.414+0000 7fb95f800640 1 -- 172.21.15.26:0/3010906895 >> 172.21.15.26:0/3010906895 conn(0x7fb95806a800 msgr2=0x7fb95806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.414+0000 7fb95f800640 1 -- 172.21.15.26:0/3010906895 shutdown_connections 2024-01-15T17:56:18.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.414+0000 7fb95f800640 1 -- 172.21.15.26:0/3010906895 wait complete. 2024-01-15T17:56:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.416+0000 7fb95f800640 1 Processor -- start 2024-01-15T17:56:18.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.416+0000 7fb95f800640 1 -- start start 2024-01-15T17:56:18.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95f800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 0x7fb95813d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:18.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95f800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 0x7fb95813cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:18.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95f800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb95815f9c0 0x7fb95814d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:18.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95f800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9581524b0 con 0x7fb95815f9c0 2024-01-15T17:56:18.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95f800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95814e380 con 0x7fb95813df80 2024-01-15T17:56:18.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95f800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95814e660 con 0x7fb95813e360 2024-01-15T17:56:18.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb95815f9c0 0x7fb95814d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb95815f9c0 0x7fb95814d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58750/0 (socket says 172.21.15.26:58750) 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95d800640 1 -- 172.21.15.26:0/2494607729 learned_addr learned my addr 172.21.15.26:0/2494607729 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.417+0000 7fb95ce00640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 0x7fb95813d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.418+0000 7fb94fc00640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 0x7fb95813cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.418+0000 7fb95d800640 1 -- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 msgr2=0x7fb95813cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.418+0000 7fb95d800640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 0x7fb95813cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.418+0000 7fb95d800640 1 -- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 msgr2=0x7fb95813d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.418+0000 7fb95d800640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 0x7fb95813d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.418+0000 7fb95d800640 1 -- 172.21.15.26:0/2494607729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb948009400 con 0x7fb95815f9c0 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.418+0000 7fb95ce00640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 0x7fb95813d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:56:18.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.418+0000 7fb95d800640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb95815f9c0 0x7fb95814d950 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb95000cb40 tx=0x7fb95000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.418+0000 7fb955600640 1 -- 172.21.15.26:0/2494607729 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000aca0 con 0x7fb95815f9c0 2024-01-15T17:56:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.419+0000 7fb955600640 1 -- 172.21.15.26:0/2494607729 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95000ae00 con 0x7fb95815f9c0 2024-01-15T17:56:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.419+0000 7fb955600640 1 -- 172.21.15.26:0/2494607729 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950012620 con 0x7fb95815f9c0 2024-01-15T17:56:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.419+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95814e940 con 0x7fb95815f9c0 2024-01-15T17:56:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.419+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95814de90 con 0x7fb95815f9c0 2024-01-15T17:56:18.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.420+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb958155490 con 0x7fb95815f9c0 2024-01-15T17:56:18.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.421+0000 7fb955600640 1 -- 172.21.15.26:0/2494607729 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb950007070 con 0x7fb95815f9c0 2024-01-15T17:56:18.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.422+0000 7fb955600640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb93006b2c0 0x7fb93006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:18.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.422+0000 7fb955600640 1 -- 172.21.15.26:0/2494607729 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb950089900 con 0x7fb95815f9c0 2024-01-15T17:56:18.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.422+0000 7fb95ce00640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb93006b2c0 0x7fb93006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:18.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.423+0000 7fb95ce00640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb93006b2c0 0x7fb93006d780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb948036000 tx=0x7fb94803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:18.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.426+0000 7fb955600640 1 -- 172.21.15.26:0/2494607729 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb950054a80 con 0x7fb95815f9c0 2024-01-15T17:56:18.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:18.644+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb958152260 con 0x7fb95815f9c0 2024-01-15T17:56:19.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.130+0000 7fb955600640 1 -- 172.21.15.26:0/2494607729 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb950017020 con 0x7fb95815f9c0 2024-01-15T17:56:19.133 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb93006b2c0 msgr2=0x7fb93006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb93006b2c0 0x7fb93006d780 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb948036000 tx=0x7fb94803c040 comp rx=0 tx=0).stop 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb95815f9c0 msgr2=0x7fb95814d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb95815f9c0 0x7fb95814d950 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb95000cb40 tx=0x7fb95000a060 comp rx=0 tx=0).stop 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 shutdown_connections 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb95813e360 0x7fb95813cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb93006b2c0 0x7fb93006d780 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb95815f9c0 0x7fb95814d950 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 --2- 172.21.15.26:0/2494607729 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb95813df80 0x7fb95813d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.133+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 >> 172.21.15.26:0/2494607729 conn(0x7fb95806a800 msgr2=0x7fb9580af330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.134+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 shutdown_connections 2024-01-15T17:56:19.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.134+0000 7fb95f800640 1 -- 172.21.15.26:0/2494607729 wait complete. 2024-01-15T17:56:19.159 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T17:56:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.372+0000 7f976ce00640 1 Processor -- start 2024-01-15T17:56:19.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.372+0000 7f976ce00640 1 -- start start 2024-01-15T17:56:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.373+0000 7f976ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97681447f0 0x7f976815c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.373+0000 7f976ce00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f97680cccf0 0x7f976815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.373+0000 7f976ce00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976815c5c0 con 0x7f976806c470 2024-01-15T17:56:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.373+0000 7f976ce00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976806d690 con 0x7f97681447f0 2024-01-15T17:56:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.373+0000 7f976ce00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97680cebb0 con 0x7f97680cccf0 2024-01-15T17:56:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.374+0000 7f9767400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f97681447f0 0x7f976815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.374+0000 7f975fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f976806c470 0x7f97680cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51254/0 (socket says 172.21.15.26:51254) 2024-01-15T17:56:19.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.374+0000 7f975fe00640 1 -- 172.21.15.26:0/4017310109 learned_addr learned my addr 172.21.15.26:0/4017310109 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:19.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.374+0000 7f9767400640 1 -- 172.21.15.26:0/4017310109 >> v2:172.21.15.124:3300/0 conn(0x7f97680cccf0 msgr2=0x7f976815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:19.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.374+0000 7f9767400640 1 --2- 172.21.15.26:0/4017310109 >> v2:172.21.15.124:3300/0 conn(0x7f97680cccf0 0x7f976815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.374+0000 7f9767400640 1 -- 172.21.15.26:0/4017310109 >> v1:172.21.15.26:6789/0 conn(0x7f976806c470 legacy=0x7f97680cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.374+0000 7f9767400640 1 -- 172.21.15.26:0/4017310109 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976806ca10 con 0x7f97681447f0 2024-01-15T17:56:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.374+0000 7f9766a00640 1 -- 172.21.15.26:0/4017310109 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439684432 0 0) 0x7f976815c5c0 con 0x7f976806c470 2024-01-15T17:56:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.375+0000 7f9767400640 1 --2- 172.21.15.26:0/4017310109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97681447f0 0x7f976815c080 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f97680aea60 tx=0x7f975802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b421b60451e5b0c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f9766a00640 1 -- 172.21.15.26:0/4017310109 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975802e070 con 0x7f97681447f0 2024-01-15T17:56:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f9766a00640 1 -- 172.21.15.26:0/4017310109 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97580315a0 con 0x7f97681447f0 2024-01-15T17:56:19.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f9766a00640 1 -- 172.21.15.26:0/4017310109 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758031930 con 0x7f97681447f0 2024-01-15T17:56:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f976ce00640 1 -- 172.21.15.26:0/4017310109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97681447f0 msgr2=0x7f976815c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f976ce00640 1 --2- 172.21.15.26:0/4017310109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97681447f0 0x7f976815c080 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f97680aea60 tx=0x7f975802b840 comp rx=0 tx=0).stop 2024-01-15T17:56:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f976ce00640 1 -- 172.21.15.26:0/4017310109 shutdown_connections 2024-01-15T17:56:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f976ce00640 1 --2- 172.21.15.26:0/4017310109 >> v2:172.21.15.124:3300/0 conn(0x7f97680cccf0 0x7f976815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f976ce00640 1 --2- 172.21.15.26:0/4017310109 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97681447f0 0x7f976815c080 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f976ce00640 1 -- 172.21.15.26:0/4017310109 >> 172.21.15.26:0/4017310109 conn(0x7f976806a3e0 msgr2=0x7f976806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f976ce00640 1 -- 172.21.15.26:0/4017310109 shutdown_connections 2024-01-15T17:56:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.376+0000 7f976ce00640 1 -- 172.21.15.26:0/4017310109 wait complete. 2024-01-15T17:56:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.378+0000 7f976ce00640 1 Processor -- start 2024-01-15T17:56:19.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.378+0000 7f976ce00640 1 -- start start 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.379+0000 7f976ce00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f976806c470 0x7f9768137a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.379+0000 7f976ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97680cccf0 0x7f976815a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.379+0000 7f976ce00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97681447f0 0x7f97680caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.379+0000 7f976ce00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97680cb310 con 0x7f97680cccf0 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.379+0000 7f976ce00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97680cb5f0 con 0x7f97681447f0 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.379+0000 7f976ce00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97680cb8d0 con 0x7f976806c470 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.379+0000 7f975fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97680cccf0 0x7f976815a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.380+0000 7f975fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97680cccf0 0x7f976815a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58770/0 (socket says 172.21.15.26:58770) 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.380+0000 7f975fe00640 1 -- 172.21.15.26:0/479890611 learned_addr learned my addr 172.21.15.26:0/479890611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:19.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.380+0000 7f975fe00640 1 -- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f976806c470 msgr2=0x7f9768137a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.380+0000 7f975fe00640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f976806c470 0x7f9768137a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.380+0000 7f975fe00640 1 -- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97681447f0 msgr2=0x7f97680caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.380+0000 7f975fe00640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97681447f0 0x7f97680caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.380+0000 7f975fe00640 1 -- 172.21.15.26:0/479890611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758009400 con 0x7f97680cccf0 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.380+0000 7f975fe00640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97680cccf0 0x7f976815a840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f975000c4d0 tx=0x7f975000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.380+0000 7f975f400640 1 -- 172.21.15.26:0/479890611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97500074e0 con 0x7f97680cccf0 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.381+0000 7f975f400640 1 -- 172.21.15.26:0/479890611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9750007660 con 0x7f97680cccf0 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.381+0000 7f975f400640 1 -- 172.21.15.26:0/479890611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750004a20 con 0x7f97680cccf0 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.381+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97680cbbb0 con 0x7f97680cccf0 2024-01-15T17:56:19.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.381+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976815ad80 con 0x7f97680cccf0 2024-01-15T17:56:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.382+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9768067b60 con 0x7f97680cccf0 2024-01-15T17:56:19.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.383+0000 7f975f400640 1 -- 172.21.15.26:0/479890611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9750004b80 con 0x7f97680cccf0 2024-01-15T17:56:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.384+0000 7f975f400640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f974006b2c0 0x7f974006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.384+0000 7f975f400640 1 -- 172.21.15.26:0/479890611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9750088a10 con 0x7f97680cccf0 2024-01-15T17:56:19.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.384+0000 7f9767400640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f974006b2c0 0x7f974006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.385+0000 7f9767400640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f974006b2c0 0x7f974006d780 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9758036000 tx=0x7f9758030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:19.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.389+0000 7f975f400640 1 -- 172.21.15.26:0/479890611 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9750053bd0 con 0x7f97680cccf0 2024-01-15T17:56:19.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.657+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9768067450 con 0x7f97680cccf0 2024-01-15T17:56:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.658+0000 7f975f400640 1 -- 172.21.15.26:0/479890611 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v5) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f9750018020 con 0x7f97680cccf0 2024-01-15T17:56:19.661 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVl3erNJhAANxPK+39yVM5neq9iKC1HkA==", 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:56:19.662 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.663 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.663 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.663 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.663 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-01-15T17:56:19.663 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlLaUlDRAAeKaen5hLlwDDWY6OAMRsPQ==", 2024-01-15T17:56:19.663 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.663 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:56:19.663 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:56:19.663 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:56:19.664 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.664 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.664 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.664 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.664 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-01-15T17:56:19.664 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlaIAVOBAAA8LEFM1HtzRtNooQCCjS/w==", 2024-01-15T17:56:19.664 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.664 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:56:19.664 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlmpTBHRAAUwbZeVpysMsn56QRLRJCoA==", 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:56:19.665 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA7caVlCTmgDhAAPWb0qEh134xxyF8lM9DoNg==", 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:56:19.666 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA+caVlaxF5JxAA23Z9j21ATy3GVlhgndDIxw==", 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:56:19.667 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.668 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.668 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.668 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.668 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-01-15T17:56:19.668 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBCcaVlcRtjBRAAOFN9Xf5kN5MXL6yQfRI27g==", 2024-01-15T17:56:19.668 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.668 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:56:19.668 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:56:19.668 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.669 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.669 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.669 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.669 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-01-15T17:56:19.669 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBFcaVlvAzEHhAACDoNh7BZY9gbKTdGwIrhlg==", 2024-01-15T17:56:19.669 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.669 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:56:19.669 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:56:19.669 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBMcaVl8yiOMBAAdI4NTeSB1d/nSHRaIDMcFw==", 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.670 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBQcaVltf8sARAAfPaJAfoTgQoEivw90jRz/Q==", 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.671 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBTcaVltuKiGhAAT0cIJiRkQ38QXT/h6Gyggg==", 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.672 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBWcaVlbJqVMhAAKjpl6pAHbytjZWMYeRbRjA==", 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.673 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA==", 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-01-15T17:56:19.674 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVlrVpqBxAAxOL0iuAlOvfR+KgSsin3vQ==", 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlO8vdNxAAlUE/TFCDQoXuxvwLPwUvtg==", 2024-01-15T17:56:19.675 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVl5qgCEhAALgt7d3OjEmq5bc4nAD2rTw==", 2024-01-15T17:56:19.676 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA2caVlyg4eHBAAwTCXVk8hO9+zcZ1tUA3gfA==", 2024-01-15T17:56:19.677 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mds", 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBkcaVl1XkMMRAA+DlUZoKI8x/izrN4rpuynQ==", 2024-01-15T17:56:19.678 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mds" 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-mgr", 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBkcaVlkbMMMRAAQ/gXfmUptZI3VoHxCwQRUQ==", 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-mgr" 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-osd", 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBkcaVlyeAMMRAA/gC2BptzqWaBNdnGQSwlMA==", 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-osd" 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd", 2024-01-15T17:56:19.680 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBkcaVluAwNMRAAtbCOCTjBE7n3gOD88aC8EQ==", 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd" 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBkcaVlAzgNMRAAJx8u0u/asDK1CbnCzzgOfQ==", 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.bootstrap-rgw", 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBkcaVlX1QNMRAAhnDTq5UBsi8kigep/MhuIg==", 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile bootstrap-rgw" 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlvuBcGhAAGqdCMSffJwnMKlFh3CtyCw==", 2024-01-15T17:56:19.682 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVle8Z+DhAA9cWxzc4puvrHAedqupPG3Q==", 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:56:19.683 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T17:56:19.684 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T17:56:19.684 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:56:19.684 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.684 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:19.684 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-01-15T17:56:19.684 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:56:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f974006b2c0 msgr2=0x7f974006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f974006b2c0 0x7f974006d780 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9758036000 tx=0x7f9758030000 comp rx=0 tx=0).stop 2024-01-15T17:56:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97680cccf0 msgr2=0x7f976815a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97680cccf0 0x7f976815a840 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f975000c4d0 tx=0x7f975000c990 comp rx=0 tx=0).stop 2024-01-15T17:56:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 shutdown_connections 2024-01-15T17:56:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f974006b2c0 0x7f974006d780 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97681447f0 0x7f97680caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97680cccf0 0x7f976815a840 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 --2- 172.21.15.26:0/479890611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f976806c470 0x7f9768137a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 >> 172.21.15.26:0/479890611 conn(0x7f976806a3e0 msgr2=0x7f976815c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 shutdown_connections 2024-01-15T17:56:19.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.661+0000 7f976ce00640 1 -- 172.21.15.26:0/479890611 wait complete. 2024-01-15T17:56:19.687 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-01-15T17:56:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.913+0000 7f2577e00640 1 Processor -- start 2024-01-15T17:56:19.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.913+0000 7f2577e00640 1 -- start start 2024-01-15T17:56:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.914+0000 7f2577e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25780d0100 0x7f25780b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.915+0000 7f2577e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25780d0a10 con 0x7f2578160920 2024-01-15T17:56:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.915+0000 7f2577e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25780cf080 con 0x7f25780d0f40 2024-01-15T17:56:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.915+0000 7f2577e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25780c6be0 con 0x7f25780d0100 2024-01-15T17:56:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.915+0000 7f2576a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2578160920 0x7f25780cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51256/0 (socket says 172.21.15.26:51256) 2024-01-15T17:56:19.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.915+0000 7f2576a00640 1 -- 172.21.15.26:0/3129222826 learned_addr learned my addr 172.21.15.26:0/3129222826 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.915+0000 7f2576000640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25780d0100 0x7f25780b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.915+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224590853 0 0) 0x7f25780d0a10 con 0x7f2578160920 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2558003520 con 0x7f2578160920 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2185038594 0 0) 0x7f2558003520 con 0x7f2578160920 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25780d0100 msgr2=0x7f25780b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25780d0100 0x7f25780b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 >> v1:172.21.15.124:6789/0 conn(0x7f25780d0f40 legacy=0x7f25780d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25780ce7e0 con 0x7f2578160920 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f2568001670 con 0x7f2578160920 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 >> v1:172.21.15.26:6789/0 conn(0x7f2578160920 legacy=0x7f25780cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25580039a0 0x7f2558003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580047c0 0x7f2558004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25780d0100 0x7f2558005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25780c6be0 con 0x7f25580039a0 2024-01-15T17:56:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255800ba60 con 0x7f25580047c0 2024-01-15T17:56:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255800bc20 con 0x7f25780d0100 2024-01-15T17:56:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.916+0000 7f2576a00640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25580039a0 0x7f2558003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.917+0000 7f2576000640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580047c0 0x7f2558004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.917+0000 7f2577400640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25780d0100 0x7f2558005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.917+0000 7f2576a00640 1 -- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25780d0100 msgr2=0x7f2558005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.917+0000 7f2576a00640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25780d0100 0x7f2558005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.917+0000 7f2576a00640 1 -- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580047c0 msgr2=0x7f2558004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.917+0000 7f2576a00640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580047c0 0x7f2558004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.917+0000 7f2576a00640 1 -- 172.21.15.26:0/3129222826 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558003520 con 0x7f25580039a0 2024-01-15T17:56:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.917+0000 7f2577400640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25780d0100 0x7f2558005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:19.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.919+0000 7f2576a00640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25580039a0 0x7f2558003dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2568001eb0 tx=0x7f25580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816be924a1816259 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.919+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568005ad0 con 0x7f25580039a0 2024-01-15T17:56:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.919+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2568005c50 con 0x7f25580039a0 2024-01-15T17:56:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.919+0000 7f2575600640 1 -- 172.21.15.26:0/3129222826 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256802c7c0 con 0x7f25580039a0 2024-01-15T17:56:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.920+0000 7f2577e00640 1 -- 172.21.15.26:0/3129222826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25580039a0 msgr2=0x7f2558003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.920+0000 7f2577e00640 1 --2- 172.21.15.26:0/3129222826 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25580039a0 0x7f2558003dc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2568001eb0 tx=0x7f25580057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.920+0000 7f2577400640 1 -- 172.21.15.26:0/3129222826 reap_dead start 2024-01-15T17:56:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.920+0000 7f2577e00640 1 -- 172.21.15.26:0/3129222826 shutdown_connections 2024-01-15T17:56:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.920+0000 7f2577e00640 1 -- 172.21.15.26:0/3129222826 >> 172.21.15.26:0/3129222826 conn(0x7f25780d3090 msgr2=0x7f25780d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:19.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.920+0000 7f2577e00640 1 -- 172.21.15.26:0/3129222826 shutdown_connections 2024-01-15T17:56:19.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.920+0000 7f2577e00640 1 -- 172.21.15.26:0/3129222826 wait complete. 2024-01-15T17:56:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.922+0000 7f2577e00640 1 Processor -- start 2024-01-15T17:56:19.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.922+0000 7f2577e00640 1 -- start start 2024-01-15T17:56:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.923+0000 7f2577e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580039a0 0x7f25781f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.923+0000 7f2577e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f257815e510 0x7f2578148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.923+0000 7f2577e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257815c520 0x7f257815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.923+0000 7f2577e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257815f6e0 con 0x7f257815e510 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.923+0000 7f2577e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257815cee0 con 0x7f25580039a0 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.923+0000 7f2577e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25781f74f0 con 0x7f257815c520 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.923+0000 7f2576000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f257815e510 0x7f2578148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.923+0000 7f2576000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f257815e510 0x7f2578148e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58792/0 (socket says 172.21.15.26:58792) 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.923+0000 7f2576000640 1 -- 172.21.15.26:0/747590289 learned_addr learned my addr 172.21.15.26:0/747590289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.924+0000 7f2576a00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580039a0 0x7f25781f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.924+0000 7f2577400640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257815c520 0x7f257815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.924+0000 7f2576000640 1 -- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257815c520 msgr2=0x7f257815c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.924+0000 7f2576000640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257815c520 0x7f257815c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.924+0000 7f2576000640 1 -- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580039a0 msgr2=0x7f25781f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:19.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.924+0000 7f2576000640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580039a0 0x7f25781f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.924+0000 7f2576000640 1 -- 172.21.15.26:0/747590289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257815ea60 con 0x7f257815e510 2024-01-15T17:56:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.924+0000 7f2576a00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580039a0 0x7f25781f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.924+0000 7f2576000640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f257815e510 0x7f2578148e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2560011950 tx=0x7f2560011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.925+0000 7f256f400640 1 -- 172.21.15.26:0/747590289 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256000faf0 con 0x7f257815e510 2024-01-15T17:56:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.925+0000 7f256f400640 1 -- 172.21.15.26:0/747590289 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256000fc50 con 0x7f257815e510 2024-01-15T17:56:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.925+0000 7f256f400640 1 -- 172.21.15.26:0/747590289 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2560014440 con 0x7f257815e510 2024-01-15T17:56:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.925+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257815ed40 con 0x7f257815e510 2024-01-15T17:56:19.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.925+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257815f300 con 0x7f257815e510 2024-01-15T17:56:19.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.926+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2578067450 con 0x7f257815e510 2024-01-15T17:56:19.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.927+0000 7f256f400640 1 -- 172.21.15.26:0/747590289 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f25600145a0 con 0x7f257815e510 2024-01-15T17:56:19.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.928+0000 7f256f400640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f255006b2c0 0x7f255006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:19.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.928+0000 7f256f400640 1 -- 172.21.15.26:0/747590289 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f256008a0e0 con 0x7f257815e510 2024-01-15T17:56:19.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.929+0000 7f2576a00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f255006b2c0 0x7f255006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:19.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.930+0000 7f2576a00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f255006b2c0 0x7f255006d780 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2568034000 tx=0x7f2568009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:19.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:19.933+0000 7f256f400640 1 -- 172.21.15.26:0/747590289 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2560055200 con 0x7f257815e510 2024-01-15T17:56:20.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.200+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f257808c7e0 con 0x7f257815e510 2024-01-15T17:56:20.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.207+0000 7f256f400640 1 -- 172.21.15.26:0/747590289 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2560058820 con 0x7f257815e510 2024-01-15T17:56:20.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f255006b2c0 msgr2=0x7f255006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f255006b2c0 0x7f255006d780 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2568034000 tx=0x7f2568009030 comp rx=0 tx=0).stop 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f257815e510 msgr2=0x7f2578148e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f257815e510 0x7f2578148e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2560011950 tx=0x7f2560011e10 comp rx=0 tx=0).stop 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 shutdown_connections 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f257815c520 0x7f257815c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f255006b2c0 0x7f255006d780 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f257815e510 0x7f2578148e10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 --2- 172.21.15.26:0/747590289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25580039a0 0x7f25781f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 >> 172.21.15.26:0/747590289 conn(0x7f25780d3090 msgr2=0x7f25780cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 shutdown_connections 2024-01-15T17:56:20.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.210+0000 7f2577e00640 1 -- 172.21.15.26:0/747590289 wait complete. 2024-01-15T17:56:20.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-01-15T17:56:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.442+0000 7fd602e00640 1 Processor -- start 2024-01-15T17:56:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.442+0000 7fd602e00640 1 -- start start 2024-01-15T17:56:20.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.444+0000 7fd602e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5fc0ced50 0x7fd5fc0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.444+0000 7fd602e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc0b1430 con 0x7fd5fc0ce310 2024-01-15T17:56:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.444+0000 7fd602e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc0d0c40 con 0x7fd5fc0cd820 2024-01-15T17:56:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.444+0000 7fd602e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5fc0cf160 con 0x7fd5fc0ced50 2024-01-15T17:56:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.445+0000 7fd602400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5fc0ced50 0x7fd5fc0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.445+0000 7fd601000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5fc0ce310 0x7fd5fc164f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51260/0 (socket says 172.21.15.26:51260) 2024-01-15T17:56:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.445+0000 7fd601000640 1 -- 172.21.15.26:0/2596930713 learned_addr learned my addr 172.21.15.26:0/2596930713 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.445+0000 7fd602400640 1 -- 172.21.15.26:0/2596930713 >> v1:172.21.15.124:6789/0 conn(0x7fd5fc0cd820 legacy=0x7fd5fc0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.445+0000 7fd602400640 1 -- 172.21.15.26:0/2596930713 >> v1:172.21.15.26:6789/0 conn(0x7fd5fc0ce310 legacy=0x7fd5fc164f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.445+0000 7fd5fbe00640 1 -- 172.21.15.26:0/2596930713 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013581267 0 0) 0x7fd5fc0b1430 con 0x7fd5fc0ce310 2024-01-15T17:56:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.445+0000 7fd602400640 1 -- 172.21.15.26:0/2596930713 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc0d0f20 con 0x7fd5fc0ced50 2024-01-15T17:56:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.446+0000 7fd602400640 1 --2- 172.21.15.26:0/2596930713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0ced50 0x7fd5fc0d0640 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009d90 tx=0x7fd5f402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2af81ded635ec1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.447+0000 7fd5fbe00640 1 -- 172.21.15.26:0/2596930713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f402cc60 con 0x7fd5fc0ced50 2024-01-15T17:56:20.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.447+0000 7fd5fbe00640 1 -- 172.21.15.26:0/2596930713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f402cdc0 con 0x7fd5fc0ced50 2024-01-15T17:56:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.447+0000 7fd5fbe00640 1 -- 172.21.15.26:0/2596930713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4034660 con 0x7fd5fc0ced50 2024-01-15T17:56:20.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.447+0000 7fd602e00640 1 -- 172.21.15.26:0/2596930713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0ced50 msgr2=0x7fd5fc0d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.447+0000 7fd602e00640 1 --2- 172.21.15.26:0/2596930713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0ced50 0x7fd5fc0d0640 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4009d90 tx=0x7fd5f402beb0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.447+0000 7fd602e00640 1 -- 172.21.15.26:0/2596930713 shutdown_connections 2024-01-15T17:56:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.447+0000 7fd602e00640 1 --2- 172.21.15.26:0/2596930713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0ced50 0x7fd5fc0d0640 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.447+0000 7fd602e00640 1 -- 172.21.15.26:0/2596930713 >> 172.21.15.26:0/2596930713 conn(0x7fd5fc099a60 msgr2=0x7fd5fc099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.448+0000 7fd602e00640 1 -- 172.21.15.26:0/2596930713 shutdown_connections 2024-01-15T17:56:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.448+0000 7fd602e00640 1 -- 172.21.15.26:0/2596930713 wait complete. 2024-01-15T17:56:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.449+0000 7fd602e00640 1 Processor -- start 2024-01-15T17:56:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.449+0000 7fd602e00640 1 -- start start 2024-01-15T17:56:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.450+0000 7fd602e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0cd820 0x7fd5fc15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.450+0000 7fd602e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5fc0ce310 0x7fd5fc15f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.450+0000 7fd602e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5fc0ced50 0x7fd5fc15f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.450+0000 7fd602e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b4520 con 0x7fd5fc0cd820 2024-01-15T17:56:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.450+0000 7fd602e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc15ee40 con 0x7fd5fc0ce310 2024-01-15T17:56:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.450+0000 7fd602e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0b3b80 con 0x7fd5fc0ced50 2024-01-15T17:56:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0cd820 0x7fd5fc15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0cd820 0x7fd5fc15e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58798/0 (socket says 172.21.15.26:58798) 2024-01-15T17:56:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601a00640 1 -- 172.21.15.26:0/252379105 learned_addr learned my addr 172.21.15.26:0/252379105 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601000640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5fc0ce310 0x7fd5fc15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601a00640 1 -- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5fc0ced50 msgr2=0x7fd5fc15f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601a00640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5fc0ced50 0x7fd5fc15f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601a00640 1 -- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5fc0ce310 msgr2=0x7fd5fc15f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601a00640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5fc0ce310 0x7fd5fc15f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601a00640 1 -- 172.21.15.26:0/252379105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4009a70 con 0x7fd5fc0cd820 2024-01-15T17:56:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.451+0000 7fd601000640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5fc0ce310 0x7fd5fc15f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.452+0000 7fd601a00640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0cd820 0x7fd5fc15e3c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00c400 tx=0x7fd5ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.452+0000 7fd5f9600640 1 -- 172.21.15.26:0/252379105 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec0052d0 con 0x7fd5fc0cd820 2024-01-15T17:56:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.452+0000 7fd5f9600640 1 -- 172.21.15.26:0/252379105 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec005430 con 0x7fd5fc0cd820 2024-01-15T17:56:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.452+0000 7fd5f9600640 1 -- 172.21.15.26:0/252379105 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec003830 con 0x7fd5fc0cd820 2024-01-15T17:56:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.452+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc0b3e60 con 0x7fd5fc0cd820 2024-01-15T17:56:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.452+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc0af9a0 con 0x7fd5fc0cd820 2024-01-15T17:56:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.453+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc0b4660 con 0x7fd5fc0cd820 2024-01-15T17:56:20.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.454+0000 7fd5f9600640 1 -- 172.21.15.26:0/252379105 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fd5ec003990 con 0x7fd5fc0cd820 2024-01-15T17:56:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.455+0000 7fd5f9600640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d406b2c0 0x7fd5d406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.455+0000 7fd5f9600640 1 -- 172.21.15.26:0/252379105 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd5ec088df0 con 0x7fd5fc0cd820 2024-01-15T17:56:20.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.456+0000 7fd601000640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d406b2c0 0x7fd5d406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.457+0000 7fd601000640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d406b2c0 0x7fd5d406d780 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc06e860 tx=0x7fd5e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:20.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.459+0000 7fd5f9600640 1 -- 172.21.15.26:0/252379105 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5ec053fa0 con 0x7fd5fc0cd820 2024-01-15T17:56:20.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.724+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd5fc067450 con 0x7fd5fc0cd820 2024-01-15T17:56:20.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.727+0000 7fd5f9600640 1 -- 172.21.15.26:0/252379105 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd5ec0170a0 con 0x7fd5fc0cd820 2024-01-15T17:56:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d406b2c0 msgr2=0x7fd5d406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d406b2c0 0x7fd5d406d780 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc06e860 tx=0x7fd5e4009040 comp rx=0 tx=0).stop 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0cd820 msgr2=0x7fd5fc15e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0cd820 0x7fd5fc15e3c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00c400 tx=0x7fd5ec00c8c0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 shutdown_connections 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5fc0ced50 0x7fd5fc15f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd5d406b2c0 0x7fd5d406d780 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5fc0ce310 0x7fd5fc15f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 --2- 172.21.15.26:0/252379105 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5fc0cd820 0x7fd5fc15e3c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 >> 172.21.15.26:0/252379105 conn(0x7fd5fc099a60 msgr2=0x7fd5fc06ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 shutdown_connections 2024-01-15T17:56:20.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.731+0000 7fd602e00640 1 -- 172.21.15.26:0/252379105 wait complete. 2024-01-15T17:56:20.753 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-01-15T17:56:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.970+0000 7f1ddce00640 1 Processor -- start 2024-01-15T17:56:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.970+0000 7f1ddce00640 1 -- start start 2024-01-15T17:56:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.972+0000 7f1ddce00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1dd813cda0 0x7f1dd815da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.972+0000 7f1ddce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1dd80ce0c0 0x7f1dd8070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.972+0000 7f1ddce00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dd80ce4f0 con 0x7f1dd815dfa0 2024-01-15T17:56:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.972+0000 7f1ddce00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd80700f0 con 0x7f1dd80ce0c0 2024-01-15T17:56:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.972+0000 7f1ddce00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dd815e3e0 con 0x7f1dd813cda0 2024-01-15T17:56:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.972+0000 7f1dd7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1dd80ce0c0 0x7f1dd8070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.972+0000 7f1dd7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1dd80ce0c0 0x7f1dd8070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58810/0 (socket says 172.21.15.26:58810) 2024-01-15T17:56:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.972+0000 7f1dd7e00640 1 -- 172.21.15.26:0/1051478942 learned_addr learned my addr 172.21.15.26:0/1051478942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.972+0000 7f1dd7400640 1 --2- 172.21.15.26:0/1051478942 >> v2:172.21.15.124:3300/0 conn(0x7f1dd813cda0 0x7f1dd815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.973+0000 7f1dd7e00640 1 -- 172.21.15.26:0/1051478942 >> v2:172.21.15.124:3300/0 conn(0x7f1dd813cda0 msgr2=0x7f1dd815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.973+0000 7f1dd7e00640 1 --2- 172.21.15.26:0/1051478942 >> v2:172.21.15.124:3300/0 conn(0x7f1dd813cda0 0x7f1dd815da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.973+0000 7f1dd7e00640 1 -- 172.21.15.26:0/1051478942 >> v1:172.21.15.124:6789/0 conn(0x7f1dd815dfa0 legacy=0x7f1dd80cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.973+0000 7f1dd7e00640 1 -- 172.21.15.26:0/1051478942 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd80ce720 con 0x7f1dd80ce0c0 2024-01-15T17:56:20.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.974+0000 7f1dd7e00640 1 --2- 172.21.15.26:0/1051478942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd80ce0c0 0x7f1dd8070b40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1dd806df10 tx=0x7f1dd002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe9c20d57df62ca server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:20.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.974+0000 7f1dd6000640 1 -- 172.21.15.26:0/1051478942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd00317e0 con 0x7f1dd80ce0c0 2024-01-15T17:56:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.974+0000 7f1dd6000640 1 -- 172.21.15.26:0/1051478942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd0031940 con 0x7f1dd80ce0c0 2024-01-15T17:56:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.974+0000 7f1dd6000640 1 -- 172.21.15.26:0/1051478942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0031cd0 con 0x7f1dd80ce0c0 2024-01-15T17:56:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.975+0000 7f1ddce00640 1 -- 172.21.15.26:0/1051478942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd80ce0c0 msgr2=0x7f1dd8070b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.975+0000 7f1ddce00640 1 --2- 172.21.15.26:0/1051478942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd80ce0c0 0x7f1dd8070b40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1dd806df10 tx=0x7f1dd002b880 comp rx=0 tx=0).stop 2024-01-15T17:56:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.975+0000 7f1ddce00640 1 -- 172.21.15.26:0/1051478942 shutdown_connections 2024-01-15T17:56:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.975+0000 7f1ddce00640 1 --2- 172.21.15.26:0/1051478942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd80ce0c0 0x7f1dd8070b40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.975+0000 7f1ddce00640 1 --2- 172.21.15.26:0/1051478942 >> v2:172.21.15.124:3300/0 conn(0x7f1dd813cda0 0x7f1dd815da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.975+0000 7f1ddce00640 1 -- 172.21.15.26:0/1051478942 >> 172.21.15.26:0/1051478942 conn(0x7f1dd806a3e0 msgr2=0x7f1dd806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.975+0000 7f1ddce00640 1 -- 172.21.15.26:0/1051478942 shutdown_connections 2024-01-15T17:56:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.975+0000 7f1ddce00640 1 -- 172.21.15.26:0/1051478942 wait complete. 2024-01-15T17:56:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.977+0000 7f1ddce00640 1 Processor -- start 2024-01-15T17:56:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.977+0000 7f1ddce00640 1 -- start start 2024-01-15T17:56:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1ddce00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd80ce0c0 0x7f1dd8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1ddce00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1dd813cda0 0x7f1dd8154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1ddce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd815dfa0 0x7f1dd80adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1ddce00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd81314f0 con 0x7f1dd815dfa0 2024-01-15T17:56:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1ddce00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd8131a70 con 0x7f1dd813cda0 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1ddce00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd8132440 con 0x7f1dd80ce0c0 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1dd7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd815dfa0 0x7f1dd80adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1dd7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd815dfa0 0x7f1dd80adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58818/0 (socket says 172.21.15.26:58818) 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1dd7e00640 1 -- 172.21.15.26:0/104756052 learned_addr learned my addr 172.21.15.26:0/104756052 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1dd7400640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd80ce0c0 0x7f1dd8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.978+0000 7f1dd6a00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1dd813cda0 0x7f1dd8154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1dd7e00640 1 -- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd80ce0c0 msgr2=0x7f1dd8131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1dd7e00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd80ce0c0 0x7f1dd8131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1dd7e00640 1 -- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1dd813cda0 msgr2=0x7f1dd8154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1dd7e00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1dd813cda0 0x7f1dd8154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1dd7e00640 1 -- 172.21.15.26:0/104756052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd00094d0 con 0x7f1dd815dfa0 2024-01-15T17:56:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1dd7e00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd815dfa0 0x7f1dd80adcc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00090b0 tx=0x7f1dd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1dcf400640 1 -- 172.21.15.26:0/104756052 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0032880 con 0x7f1dd815dfa0 2024-01-15T17:56:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1dcf400640 1 -- 172.21.15.26:0/104756052 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd0032a00 con 0x7f1dd815dfa0 2024-01-15T17:56:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1dcf400640 1 -- 172.21.15.26:0/104756052 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0032db0 con 0x7f1dd815dfa0 2024-01-15T17:56:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.979+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd81516e0 con 0x7f1dd815dfa0 2024-01-15T17:56:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.980+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd8151ca0 con 0x7f1dd815dfa0 2024-01-15T17:56:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.981+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd8131c00 con 0x7f1dd815dfa0 2024-01-15T17:56:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.982+0000 7f1dcf400640 1 -- 172.21.15.26:0/104756052 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1dd004d3f0 con 0x7f1dd815dfa0 2024-01-15T17:56:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.983+0000 7f1dcf400640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1db006b2f0 0x7f1db006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.983+0000 7f1dcf400640 1 -- 172.21.15.26:0/104756052 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1dd00ab520 con 0x7f1dd815dfa0 2024-01-15T17:56:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.984+0000 7f1dd7400640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1db006b2f0 0x7f1db006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.985+0000 7f1dd7400640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1db006b2f0 0x7f1db006d7b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009a60 tx=0x7f1dc8003990 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:20.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:20.988+0000 7f1dcf400640 1 -- 172.21.15.26:0/104756052 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1dd00766e0 con 0x7f1dd815dfa0 2024-01-15T17:56:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.250+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f1dd8067450 con 0x7f1dd815dfa0 2024-01-15T17:56:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.253+0000 7f1dcf400640 1 -- 172.21.15.26:0/104756052 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1dd0079d00 con 0x7f1dd815dfa0 2024-01-15T17:56:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1db006b2f0 msgr2=0x7f1db006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1db006b2f0 0x7f1db006d7b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009a60 tx=0x7f1dc8003990 comp rx=0 tx=0).stop 2024-01-15T17:56:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd815dfa0 msgr2=0x7f1dd80adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd815dfa0 0x7f1dd80adcc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1dd00090b0 tx=0x7f1dd00046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 shutdown_connections 2024-01-15T17:56:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1db006b2f0 0x7f1db006d7b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1dd815dfa0 0x7f1dd80adcc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1dd813cda0 0x7f1dd8154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 --2- 172.21.15.26:0/104756052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1dd80ce0c0 0x7f1dd8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.256+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 >> 172.21.15.26:0/104756052 conn(0x7f1dd806a3e0 msgr2=0x7f1dd806c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.257+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 shutdown_connections 2024-01-15T17:56:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.257+0000 7f1ddce00640 1 -- 172.21.15.26:0/104756052 wait complete. 2024-01-15T17:56:21.278 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-01-15T17:56:21.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.488+0000 7fcc2b200640 1 Processor -- start 2024-01-15T17:56:21.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.488+0000 7fcc2b200640 1 -- start start 2024-01-15T17:56:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.490+0000 7fcc2b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c15b7f0 0x7fcc2c0adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.490+0000 7fcc2b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc2c0d0900 0x7fcc2c0d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.490+0000 7fcc2b200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcc2c15bbd0 0x7fcc2c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.490+0000 7fcc2b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c0b2170 con 0x7fcc2c0d0900 2024-01-15T17:56:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.490+0000 7fcc2b200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2c0b2450 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.490+0000 7fcc2b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c0b2730 con 0x7fcc2c15b7f0 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc23e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc2c0d0900 0x7fcc2c0d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc23e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc2c0d0900 0x7fcc2c0d17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58822/0 (socket says 172.21.15.26:58822) 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc23e00640 1 -- 172.21.15.26:0/2461788889 learned_addr learned my addr 172.21.15.26:0/2461788889 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc2a800640 1 --2- 172.21.15.26:0/2461788889 >> v2:172.21.15.124:3300/0 conn(0x7fcc2c15bbd0 0x7fcc2c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc29e00640 1 --2- 172.21.15.26:0/2461788889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c15b7f0 0x7fcc2c0adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc23e00640 1 -- 172.21.15.26:0/2461788889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c15b7f0 msgr2=0x7fcc2c0adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc23e00640 1 --2- 172.21.15.26:0/2461788889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c15b7f0 0x7fcc2c0adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc23e00640 1 -- 172.21.15.26:0/2461788889 >> v2:172.21.15.124:3300/0 conn(0x7fcc2c15bbd0 msgr2=0x7fcc2c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc23e00640 1 --2- 172.21.15.26:0/2461788889 >> v2:172.21.15.124:3300/0 conn(0x7fcc2c15bbd0 0x7fcc2c15c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc23e00640 1 -- 172.21.15.26:0/2461788889 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2c0b08d0 con 0x7fcc2c0d0900 2024-01-15T17:56:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.491+0000 7fcc2a800640 1 --2- 172.21.15.26:0/2461788889 >> v2:172.21.15.124:3300/0 conn(0x7fcc2c15bbd0 0x7fcc2c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.493+0000 7fcc23e00640 1 --2- 172.21.15.26:0/2461788889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c0d0900 0x7fcc2c0d17d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c074620 tx=0x7fcc1402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4004dec47f67f8eb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.493+0000 7fcc29400640 1 -- 172.21.15.26:0/2461788889 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140317e0 con 0x7fcc2c0d0900 2024-01-15T17:56:21.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.493+0000 7fcc29400640 1 -- 172.21.15.26:0/2461788889 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc14031940 con 0x7fcc2c0d0900 2024-01-15T17:56:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.493+0000 7fcc29400640 1 -- 172.21.15.26:0/2461788889 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14031cd0 con 0x7fcc2c0d0900 2024-01-15T17:56:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.494+0000 7fcc2b200640 1 -- 172.21.15.26:0/2461788889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c0d0900 msgr2=0x7fcc2c0d17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.494+0000 7fcc2b200640 1 --2- 172.21.15.26:0/2461788889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c0d0900 0x7fcc2c0d17d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c074620 tx=0x7fcc1402b880 comp rx=0 tx=0).stop 2024-01-15T17:56:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.494+0000 7fcc2b200640 1 -- 172.21.15.26:0/2461788889 shutdown_connections 2024-01-15T17:56:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.494+0000 7fcc2b200640 1 --2- 172.21.15.26:0/2461788889 >> v2:172.21.15.124:3300/0 conn(0x7fcc2c15bbd0 0x7fcc2c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.494+0000 7fcc2b200640 1 --2- 172.21.15.26:0/2461788889 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c0d0900 0x7fcc2c0d17d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.494+0000 7fcc2b200640 1 --2- 172.21.15.26:0/2461788889 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c15b7f0 0x7fcc2c0adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.494+0000 7fcc2b200640 1 -- 172.21.15.26:0/2461788889 >> 172.21.15.26:0/2461788889 conn(0x7fcc2c099a60 msgr2=0x7fcc2c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.494+0000 7fcc2b200640 1 -- 172.21.15.26:0/2461788889 shutdown_connections 2024-01-15T17:56:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.494+0000 7fcc2b200640 1 -- 172.21.15.26:0/2461788889 wait complete. 2024-01-15T17:56:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.496+0000 7fcc2b200640 1 Processor -- start 2024-01-15T17:56:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.496+0000 7fcc2b200640 1 -- start start 2024-01-15T17:56:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc2b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c0d0900 0x7fcc2c1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc2b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc2c15b7f0 0x7fcc2c156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc2b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c15bbd0 0x7fcc2c154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc2b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1575a0 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc2b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c156ab0 con 0x7fcc2c15b7f0 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc2b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2c1550d0 con 0x7fcc2c0d0900 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc2a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c15bbd0 0x7fcc2c154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc2a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c15bbd0 0x7fcc2c154b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58832/0 (socket says 172.21.15.26:58832) 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc2a800640 1 -- 172.21.15.26:0/295895862 learned_addr learned my addr 172.21.15.26:0/295895862 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.497+0000 7fcc29e00640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c0d0900 0x7fcc2c1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc23e00640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc2c15b7f0 0x7fcc2c156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc2a800640 1 -- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c0d0900 msgr2=0x7fcc2c1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc2a800640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c0d0900 0x7fcc2c1534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc2a800640 1 -- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc2c15b7f0 msgr2=0x7fcc2c156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc2a800640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc2c15b7f0 0x7fcc2c156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc2a800640 1 -- 172.21.15.26:0/295895862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc140094d0 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc23e00640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc2c15b7f0 0x7fcc2c156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc2a800640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c15bbd0 0x7fcc2c154b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007df0 tx=0x7fcc2400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc22000640 1 -- 172.21.15.26:0/295895862 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24003990 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc22000640 1 -- 172.21.15.26:0/295895862 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2400f040 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc2c156d00 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc22000640 1 -- 172.21.15.26:0/295895862 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24003d30 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.498+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2c1572c0 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.500+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2c067450 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.501+0000 7fcc22000640 1 -- 172.21.15.26:0/295895862 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fcc24004030 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.502+0000 7fcc22000640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc0406b2c0 0x7fcc0406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.502+0000 7fcc22000640 1 -- 172.21.15.26:0/295895862 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcc24089c30 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.502+0000 7fcc29e00640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc0406b2c0 0x7fcc0406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.503+0000 7fcc29e00640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc0406b2c0 0x7fcc0406d780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c006fd0 tx=0x7fcc1c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:21.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.506+0000 7fcc22000640 1 -- 172.21.15.26:0/295895862 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc24054d60 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.771+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fcc2c15dae0 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.775+0000 7fcc22000640 1 -- 172.21.15.26:0/295895862 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcc24003e90 con 0x7fcc2c15bbd0 2024-01-15T17:56:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc0406b2c0 msgr2=0x7fcc0406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc0406b2c0 0x7fcc0406d780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c006fd0 tx=0x7fcc1c008040 comp rx=0 tx=0).stop 2024-01-15T17:56:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c15bbd0 msgr2=0x7fcc2c154b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c15bbd0 0x7fcc2c154b90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007df0 tx=0x7fcc2400d4a0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 shutdown_connections 2024-01-15T17:56:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc0406b2c0 0x7fcc0406d780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc2c15bbd0 0x7fcc2c154b90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc2c15b7f0 0x7fcc2c156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 --2- 172.21.15.26:0/295895862 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc2c0d0900 0x7fcc2c1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 >> 172.21.15.26:0/295895862 conn(0x7fcc2c099a60 msgr2=0x7fcc2c0b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 shutdown_connections 2024-01-15T17:56:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:21.778+0000 7fcc2b200640 1 -- 172.21.15.26:0/295895862 wait complete. 2024-01-15T17:56:21.801 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-01-15T17:56:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.021+0000 7f3013200640 1 Processor -- start 2024-01-15T17:56:22.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.022+0000 7f3013200640 1 -- start start 2024-01-15T17:56:22.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.023+0000 7f3013200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140d0100 0x7f30140b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.023+0000 7f3013200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30140d0a10 con 0x7f3014160920 2024-01-15T17:56:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.023+0000 7f3013200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30140cf080 con 0x7f30140d0f40 2024-01-15T17:56:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.023+0000 7f3013200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30140c6be0 con 0x7f30140d0100 2024-01-15T17:56:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.024+0000 7f3011e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3014160920 0x7f30140cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54852/0 (socket says 172.21.15.26:54852) 2024-01-15T17:56:22.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.024+0000 7f3011e00640 1 -- 172.21.15.26:0/2610257223 learned_addr learned my addr 172.21.15.26:0/2610257223 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.024+0000 7f3011400640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140d0100 0x7f30140b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.024+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974087088 0 0) 0x7f30140d0a10 con 0x7f3014160920 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.024+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff4003520 con 0x7f3014160920 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270119807 0 0) 0x7f2ff4003520 con 0x7f3014160920 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140d0100 msgr2=0x7f30140b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140d0100 0x7f30140b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 >> v1:172.21.15.124:6789/0 conn(0x7f30140d0f40 legacy=0x7f30140d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30140ce7e0 con 0x7f3014160920 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f3004001670 con 0x7f3014160920 2024-01-15T17:56:22.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 >> v1:172.21.15.26:6789/0 conn(0x7f3014160920 legacy=0x7f30140cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ff40039a0 0x7f2ff4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140d0100 0x7f2ff40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4004d00 0x7f2ff4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30140c6be0 con 0x7f2ff4004d00 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff400bb10 con 0x7f2ff40039a0 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff400bca0 con 0x7f30140d0100 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3012800640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4004d00 0x7f2ff4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3011e00640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ff40039a0 0x7f2ff4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.025+0000 7f3011400640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140d0100 0x7f2ff40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.026+0000 7f3012800640 1 -- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140d0100 msgr2=0x7f2ff40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.026+0000 7f3012800640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140d0100 0x7f2ff40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.026+0000 7f3012800640 1 -- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ff40039a0 msgr2=0x7f2ff4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.026+0000 7f3012800640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ff40039a0 0x7f2ff4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.026+0000 7f3012800640 1 -- 172.21.15.26:0/2610257223 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4003520 con 0x7f2ff4004d00 2024-01-15T17:56:22.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.027+0000 7f3012800640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4004d00 0x7f2ff4005180 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f300c001e80 tx=0x7f2ff40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1393fc3862d82791 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.027+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c003b70 con 0x7f2ff4004d00 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.027+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c003cd0 con 0x7f2ff4004d00 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.027+0000 7f3010a00640 1 -- 172.21.15.26:0/2610257223 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c005310 con 0x7f2ff4004d00 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.028+0000 7f3013200640 1 -- 172.21.15.26:0/2610257223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4004d00 msgr2=0x7f2ff4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.028+0000 7f3013200640 1 --2- 172.21.15.26:0/2610257223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ff4004d00 0x7f2ff4005180 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f300c001e80 tx=0x7f2ff40057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.028+0000 7f3012800640 1 -- 172.21.15.26:0/2610257223 reap_dead start 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.028+0000 7f3013200640 1 -- 172.21.15.26:0/2610257223 shutdown_connections 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.028+0000 7f3013200640 1 -- 172.21.15.26:0/2610257223 >> 172.21.15.26:0/2610257223 conn(0x7f30140d3090 msgr2=0x7f30140d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.028+0000 7f3013200640 1 -- 172.21.15.26:0/2610257223 shutdown_connections 2024-01-15T17:56:22.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.028+0000 7f3013200640 1 -- 172.21.15.26:0/2610257223 wait complete. 2024-01-15T17:56:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.030+0000 7f3013200640 1 Processor -- start 2024-01-15T17:56:22.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.030+0000 7f3013200640 1 -- start start 2024-01-15T17:56:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.031+0000 7f3013200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ff4004d00 0x7f30140ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.031+0000 7f3013200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140afc50 0x7f301415e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.031+0000 7f3013200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301415b140 0x7f301415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.031+0000 7f3013200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301415ba00 con 0x7f301415b140 2024-01-15T17:56:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.031+0000 7f3013200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30140b3070 con 0x7f2ff4004d00 2024-01-15T17:56:22.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.031+0000 7f3013200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30140cd490 con 0x7f30140afc50 2024-01-15T17:56:22.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3012800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301415b140 0x7f301415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3012800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301415b140 0x7f301415edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41162/0 (socket says 172.21.15.26:41162) 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3012800640 1 -- 172.21.15.26:0/3142188988 learned_addr learned my addr 172.21.15.26:0/3142188988 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3011400640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140afc50 0x7f301415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3011e00640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ff4004d00 0x7f30140ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3012800640 1 -- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140afc50 msgr2=0x7f301415e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3012800640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140afc50 0x7f301415e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3012800640 1 -- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ff4004d00 msgr2=0x7f30140ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3012800640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ff4004d00 0x7f30140ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.032+0000 7f3012800640 1 -- 172.21.15.26:0/3142188988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30140b3310 con 0x7f301415b140 2024-01-15T17:56:22.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.033+0000 7f3012800640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301415b140 0x7f301415edd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f300c005040 tx=0x7f300c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:22.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.033+0000 7f300a000640 1 -- 172.21.15.26:0/3142188988 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c03e070 con 0x7f301415b140 2024-01-15T17:56:22.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.033+0000 7f300a000640 1 -- 172.21.15.26:0/3142188988 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300c003e40 con 0x7f301415b140 2024-01-15T17:56:22.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.033+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30140b35f0 con 0x7f301415b140 2024-01-15T17:56:22.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.033+0000 7f300a000640 1 -- 172.21.15.26:0/3142188988 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300c006d90 con 0x7f301415b140 2024-01-15T17:56:22.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.033+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30140b01c0 con 0x7f301415b140 2024-01-15T17:56:22.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.034+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3014067450 con 0x7f301415b140 2024-01-15T17:56:22.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.035+0000 7f300a000640 1 -- 172.21.15.26:0/3142188988 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f300c007030 con 0x7f301415b140 2024-01-15T17:56:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.036+0000 7f300a000640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2fec06b2f0 0x7f2fec06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.036+0000 7f300a000640 1 -- 172.21.15.26:0/3142188988 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f300c0aaae0 con 0x7f301415b140 2024-01-15T17:56:22.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.037+0000 7f3011e00640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2fec06b2f0 0x7f2fec06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.038+0000 7f3011e00640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2fec06b2f0 0x7f2fec06d7b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3004001c90 tx=0x7f3004009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:22.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.040+0000 7f300a000640 1 -- 172.21.15.26:0/3142188988 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f300c075cb0 con 0x7f301415b140 2024-01-15T17:56:22.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.314+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f301408c7e0 con 0x7f301415b140 2024-01-15T17:56:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.318+0000 7f300a000640 1 -- 172.21.15.26:0/3142188988 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f300c048070 con 0x7f301415b140 2024-01-15T17:56:22.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2fec06b2f0 msgr2=0x7f2fec06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2fec06b2f0 0x7f2fec06d7b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3004001c90 tx=0x7f3004009040 comp rx=0 tx=0).stop 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301415b140 msgr2=0x7f301415edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301415b140 0x7f301415edd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f300c005040 tx=0x7f300c0014a0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 shutdown_connections 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30140afc50 0x7f301415e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2fec06b2f0 0x7f2fec06d7b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f301415b140 0x7f301415edd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 --2- 172.21.15.26:0/3142188988 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ff4004d00 0x7f30140ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 >> 172.21.15.26:0/3142188988 conn(0x7f30140d3090 msgr2=0x7f30140d1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 shutdown_connections 2024-01-15T17:56:22.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.322+0000 7f3013200640 1 -- 172.21.15.26:0/3142188988 wait complete. 2024-01-15T17:56:22.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-01-15T17:56:22.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.565+0000 7f249f800640 1 Processor -- start 2024-01-15T17:56:22.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.565+0000 7f249f800640 1 -- start start 2024-01-15T17:56:22.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.567+0000 7f249f800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2498143ae0 0x7f24980cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.567+0000 7f249f800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24980cfc30 con 0x7f24980cf7d0 2024-01-15T17:56:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.567+0000 7f249f800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24980b1ee0 con 0x7f249815e3b0 2024-01-15T17:56:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.567+0000 7f249f800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24980b2780 con 0x7f2498143ae0 2024-01-15T17:56:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.568+0000 7f2497e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f24980cf7d0 0x7f249815dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54862/0 (socket says 172.21.15.26:54862) 2024-01-15T17:56:22.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.568+0000 7f2497e00640 1 -- 172.21.15.26:0/4141972447 learned_addr learned my addr 172.21.15.26:0/4141972447 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.568+0000 7f249ce00640 1 --2- 172.21.15.26:0/4141972447 >> v2:172.21.15.124:3300/0 conn(0x7f2498143ae0 0x7f24980cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.568+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554932507 0 0) 0x7f24980cfc30 con 0x7f24980cf7d0 2024-01-15T17:56:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.568+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2478003520 con 0x7f24980cf7d0 2024-01-15T17:56:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2815863724 0 0) 0x7f2478003520 con 0x7f24980cf7d0 2024-01-15T17:56:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 >> v2:172.21.15.124:3300/0 conn(0x7f2498143ae0 msgr2=0x7f24980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 --2- 172.21.15.26:0/4141972447 >> v2:172.21.15.124:3300/0 conn(0x7f2498143ae0 0x7f24980cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 >> v1:172.21.15.124:6789/0 conn(0x7f249815e3b0 legacy=0x7f24980702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249815e990 con 0x7f24980cf7d0 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f24800019f0 con 0x7f24980cf7d0 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 >> v1:172.21.15.26:6789/0 conn(0x7f24980cf7d0 legacy=0x7f249815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24780039c0 0x7f2478003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24780047c0 0x7f2478004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478005180 0x7f247800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24980b2780 con 0x7f2478005180 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247800e290 con 0x7f24780047c0 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247800e450 con 0x7f24780039c0 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.569+0000 7f249d800640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478005180 0x7f247800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.570+0000 7f2497e00640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24780047c0 0x7f2478004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.570+0000 7f249ce00640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24780039c0 0x7f2478003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.570+0000 7f249d800640 1 -- 172.21.15.26:0/4141972447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24780039c0 msgr2=0x7f2478003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.570+0000 7f249d800640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24780039c0 0x7f2478003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.570+0000 7f249d800640 1 -- 172.21.15.26:0/4141972447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24780047c0 msgr2=0x7f2478004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:22.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.570+0000 7f249d800640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24780047c0 0x7f2478004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.570+0000 7f249d800640 1 -- 172.21.15.26:0/4141972447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478003520 con 0x7f2478005180 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.571+0000 7f249d800640 1 -- 172.21.15.26:0/4141972447 reap_dead start 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.571+0000 7f249d800640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478005180 0x7f247800db90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2490001750 tx=0x7f24780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d110746e5a599a61 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.571+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24900052d0 con 0x7f2478005180 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.571+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249002e050 con 0x7f2478005180 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.571+0000 7f2497400640 1 -- 172.21.15.26:0/4141972447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24900035f0 con 0x7f2478005180 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.572+0000 7f249f800640 1 -- 172.21.15.26:0/4141972447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478005180 msgr2=0x7f247800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.572+0000 7f249f800640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478005180 0x7f247800db90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2490001750 tx=0x7f24780057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.572+0000 7f249f800640 1 -- 172.21.15.26:0/4141972447 shutdown_connections 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.572+0000 7f249f800640 1 --2- 172.21.15.26:0/4141972447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2478005180 0x7f247800db90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.572+0000 7f249f800640 1 -- 172.21.15.26:0/4141972447 >> 172.21.15.26:0/4141972447 conn(0x7f249806a800 msgr2=0x7f249806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.572+0000 7f249f800640 1 -- 172.21.15.26:0/4141972447 shutdown_connections 2024-01-15T17:56:22.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.572+0000 7f249f800640 1 -- 172.21.15.26:0/4141972447 wait complete. 2024-01-15T17:56:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.574+0000 7f249f800640 1 Processor -- start 2024-01-15T17:56:22.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.574+0000 7f249f800640 1 -- start start 2024-01-15T17:56:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.575+0000 7f249f800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2478005180 0x7f2498141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.575+0000 7f249f800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980c8210 0x7f24980c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.575+0000 7f249f800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24980702c0 0x7f2498070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.575+0000 7f249f800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498070cb0 con 0x7f24980702c0 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.575+0000 7f249f800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249806cd60 con 0x7f2478005180 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.575+0000 7f249f800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24981f6eb0 con 0x7f24980c8210 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.575+0000 7f249d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24980702c0 0x7f2498070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.575+0000 7f249d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24980702c0 0x7f2498070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41186/0 (socket says 172.21.15.26:41186) 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.575+0000 7f249d800640 1 -- 172.21.15.26:0/4191700210 learned_addr learned my addr 172.21.15.26:0/4191700210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f249ce00640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2478005180 0x7f2498141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f2497e00640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980c8210 0x7f24980c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f249d800640 1 -- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980c8210 msgr2=0x7f24980c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f249d800640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980c8210 0x7f24980c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f249d800640 1 -- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2478005180 msgr2=0x7f2498141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f249d800640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2478005180 0x7f2498141930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f249d800640 1 -- 172.21.15.26:0/4191700210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24981f6bd0 con 0x7f24980702c0 2024-01-15T17:56:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f249ce00640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2478005180 0x7f2498141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f2497e00640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980c8210 0x7f24980c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f249d800640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24980702c0 0x7f2498070770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2490001840 tx=0x7f2490001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f2494c00640 1 -- 172.21.15.26:0/4191700210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490039070 con 0x7f24980702c0 2024-01-15T17:56:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f2494c00640 1 -- 172.21.15.26:0/4191700210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2490007a30 con 0x7f24980702c0 2024-01-15T17:56:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f2494c00640 1 -- 172.21.15.26:0/4191700210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490007270 con 0x7f24980702c0 2024-01-15T17:56:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.576+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24980c3f80 con 0x7f24980702c0 2024-01-15T17:56:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.577+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24980c4540 con 0x7f24980702c0 2024-01-15T17:56:22.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.578+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24980675d0 con 0x7f24980702c0 2024-01-15T17:56:22.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.579+0000 7f2494c00640 1 -- 172.21.15.26:0/4191700210 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f24900073d0 con 0x7f24980702c0 2024-01-15T17:56:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.580+0000 7f2494c00640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f247006b2c0 0x7f247006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.580+0000 7f2494c00640 1 -- 172.21.15.26:0/4191700210 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f24900a9f50 con 0x7f24980702c0 2024-01-15T17:56:22.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.580+0000 7f249ce00640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f247006b2c0 0x7f247006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:22.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.582+0000 7f249ce00640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f247006b2c0 0x7f247006d780 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2488009980 tx=0x7f2488008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:22.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.584+0000 7f2494c00640 1 -- 172.21.15.26:0/4191700210 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2490075100 con 0x7f24980702c0 2024-01-15T17:56:22.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.850+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f24980c4700 con 0x7f24980702c0 2024-01-15T17:56:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.853+0000 7f2494c00640 1 -- 172.21.15.26:0/4191700210 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2490078720 con 0x7f24980702c0 2024-01-15T17:56:22.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f247006b2c0 msgr2=0x7f247006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f247006b2c0 0x7f247006d780 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2488009980 tx=0x7f2488008040 comp rx=0 tx=0).stop 2024-01-15T17:56:22.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24980702c0 msgr2=0x7f2498070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24980702c0 0x7f2498070770 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2490001840 tx=0x7f2490001120 comp rx=0 tx=0).stop 2024-01-15T17:56:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 shutdown_connections 2024-01-15T17:56:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24980c8210 0x7f24980c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f247006b2c0 0x7f247006d780 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24980702c0 0x7f2498070770 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 --2- 172.21.15.26:0/4191700210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2478005180 0x7f2498141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 >> 172.21.15.26:0/4191700210 conn(0x7f249806a800 msgr2=0x7f24980adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 shutdown_connections 2024-01-15T17:56:22.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:22.856+0000 7f249f800640 1 -- 172.21.15.26:0/4191700210 wait complete. 2024-01-15T17:56:22.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.098+0000 7f192a600640 1 Processor -- start 2024-01-15T17:56:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.098+0000 7f192a600640 1 -- start start 2024-01-15T17:56:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.099+0000 7f192a600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1924143ae0 0x7f19240cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f192a600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f19240702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f192a600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240cfc30 con 0x7f19240cf7d0 2024-01-15T17:56:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f192a600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19240b1ee0 con 0x7f1924143ae0 2024-01-15T17:56:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f192a600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b2780 con 0x7f192415e3b0 2024-01-15T17:56:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f1922a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f19240cf7d0 0x7f192415dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54870/0 (socket says 172.21.15.26:54870) 2024-01-15T17:56:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f1923400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1924143ae0 0x7f19240cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f1922a00640 1 -- 172.21.15.26:0/2463928970 learned_addr learned my addr 172.21.15.26:0/2463928970 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f1923e00640 1 --2- 172.21.15.26:0/2463928970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f19240702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f1923400640 1 -- 172.21.15.26:0/2463928970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 msgr2=0x7f19240702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f1923400640 1 --2- 172.21.15.26:0/2463928970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f19240702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.101+0000 7f1923400640 1 -- 172.21.15.26:0/2463928970 >> v1:172.21.15.26:6789/0 conn(0x7f19240cf7d0 legacy=0x7f192415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.101+0000 7f1923400640 1 -- 172.21.15.26:0/2463928970 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192415e990 con 0x7f1924143ae0 2024-01-15T17:56:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.100+0000 7f1922000640 1 -- 172.21.15.26:0/2463928970 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931037430 0 0) 0x7f19240cfc30 con 0x7f19240cf7d0 2024-01-15T17:56:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.101+0000 7f1923e00640 1 --2- 172.21.15.26:0/2463928970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f19240702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.102+0000 7f1923400640 1 --2- 172.21.15.26:0/2463928970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1924143ae0 0x7f19240cf290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f19140097a0 tx=0x7f191402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bdb86832368ac7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.102+0000 7f1922000640 1 -- 172.21.15.26:0/2463928970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191402d040 con 0x7f1924143ae0 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.102+0000 7f1922000640 1 -- 172.21.15.26:0/2463928970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1914031810 con 0x7f1924143ae0 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.102+0000 7f1922000640 1 -- 172.21.15.26:0/2463928970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914031ba0 con 0x7f1924143ae0 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.103+0000 7f192a600640 1 -- 172.21.15.26:0/2463928970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1924143ae0 msgr2=0x7f19240cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.103+0000 7f192a600640 1 --2- 172.21.15.26:0/2463928970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1924143ae0 0x7f19240cf290 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f19140097a0 tx=0x7f191402b8b0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.103+0000 7f192a600640 1 -- 172.21.15.26:0/2463928970 shutdown_connections 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.103+0000 7f192a600640 1 --2- 172.21.15.26:0/2463928970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f19240702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.103+0000 7f192a600640 1 --2- 172.21.15.26:0/2463928970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1924143ae0 0x7f19240cf290 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.103+0000 7f192a600640 1 -- 172.21.15.26:0/2463928970 >> 172.21.15.26:0/2463928970 conn(0x7f192406a800 msgr2=0x7f192406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.103+0000 7f192a600640 1 -- 172.21.15.26:0/2463928970 shutdown_connections 2024-01-15T17:56:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.103+0000 7f192a600640 1 -- 172.21.15.26:0/2463928970 wait complete. 2024-01-15T17:56:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.105+0000 7f192a600640 1 Processor -- start 2024-01-15T17:56:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.105+0000 7f192a600640 1 -- start start 2024-01-15T17:56:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f192a600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19240cf7d0 0x7f19240cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f192a600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1924143ae0 0x7f192412d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f192a600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f192412d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f192a600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192412b1f0 con 0x7f19240cf7d0 2024-01-15T17:56:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f192a600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192412e120 con 0x7f1924143ae0 2024-01-15T17:56:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f192a600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19241416f0 con 0x7f192415e3b0 2024-01-15T17:56:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19240cf7d0 0x7f19240cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19240cf7d0 0x7f19240cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41200/0 (socket says 172.21.15.26:41200) 2024-01-15T17:56:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923400640 1 -- 172.21.15.26:0/3621673498 learned_addr learned my addr 172.21.15.26:0/3621673498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1922a00640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1924143ae0 0x7f192412d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923e00640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f192412d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923400640 1 -- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 msgr2=0x7f192412d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923400640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f192412d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923400640 1 -- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1924143ae0 msgr2=0x7f192412d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923400640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1924143ae0 0x7f192412d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923400640 1 -- 172.21.15.26:0/3621673498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1914009400 con 0x7f19240cf7d0 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1923e00640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f192412d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.106+0000 7f1922a00640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1924143ae0 0x7f192412d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.107+0000 7f1923400640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19240cf7d0 0x7f19240cdbc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f19140098c0 tx=0x7f19140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.107+0000 7f191b400640 1 -- 172.21.15.26:0/3621673498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191403ad60 con 0x7f19240cf7d0 2024-01-15T17:56:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.107+0000 7f191b400640 1 -- 172.21.15.26:0/3621673498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1914044400 con 0x7f19240cf7d0 2024-01-15T17:56:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.107+0000 7f191b400640 1 -- 172.21.15.26:0/3621673498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19140447b0 con 0x7f19240cf7d0 2024-01-15T17:56:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.107+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19241419d0 con 0x7f19240cf7d0 2024-01-15T17:56:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.107+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19240ce100 con 0x7f19240cf7d0 2024-01-15T17:56:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.109+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19240675d0 con 0x7f19240cf7d0 2024-01-15T17:56:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.110+0000 7f191b400640 1 -- 172.21.15.26:0/3621673498 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f191404d440 con 0x7f19240cf7d0 2024-01-15T17:56:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.110+0000 7f191b400640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18fc06b2c0 0x7f18fc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.110+0000 7f191b400640 1 -- 172.21.15.26:0/3621673498 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f19140abde0 con 0x7f19240cf7d0 2024-01-15T17:56:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.111+0000 7f1922a00640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18fc06b2c0 0x7f18fc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.112+0000 7f1922a00640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18fc06b2c0 0x7f18fc06d780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f192415df60 tx=0x7f190c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.115+0000 7f191b400640 1 -- 172.21.15.26:0/3621673498 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1914076f00 con 0x7f19240cf7d0 2024-01-15T17:56:23.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.316+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19240ae600 con 0x7f19240cf7d0 2024-01-15T17:56:23.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.318+0000 7f191b400640 1 -- 172.21.15.26:0/3621673498 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12833 (secure 0 0 0) 0x7f191407a520 con 0x7f19240cf7d0 2024-01-15T17:56:23.321 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:23.321 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":31,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:56:19.145834+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:23.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18fc06b2c0 msgr2=0x7f18fc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18fc06b2c0 0x7f18fc06d780 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f192415df60 tx=0x7f190c009040 comp rx=0 tx=0).stop 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19240cf7d0 msgr2=0x7f19240cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19240cf7d0 0x7f19240cdbc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f19140098c0 tx=0x7f19140046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 shutdown_connections 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f192415e3b0 0x7f192412d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18fc06b2c0 0x7f18fc06d780 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1924143ae0 0x7f192412d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 --2- 172.21.15.26:0/3621673498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19240cf7d0 0x7f19240cdbc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 >> 172.21.15.26:0/3621673498 conn(0x7f192406a800 msgr2=0x7f192406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 shutdown_connections 2024-01-15T17:56:23.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.321+0000 7f192a600640 1 -- 172.21.15.26:0/3621673498 wait complete. 2024-01-15T17:56:23.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:56:23.345 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:56:23.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.564+0000 7efc67200640 1 Processor -- start 2024-01-15T17:56:23.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.564+0000 7efc67200640 1 -- start start 2024-01-15T17:56:23.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.566+0000 7efc67200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc6013df80 0x7efc6015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.566+0000 7efc67200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6013e360 0x7efc600cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.566+0000 7efc67200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efc6015f9c0 0x7efc600cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.566+0000 7efc67200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6006c470 con 0x7efc6013df80 2024-01-15T17:56:23.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.566+0000 7efc67200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6006c750 con 0x7efc6015f9c0 2024-01-15T17:56:23.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.566+0000 7efc67200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6006ca30 con 0x7efc6013e360 2024-01-15T17:56:23.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.566+0000 7efc5fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc6013df80 0x7efc6015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.567+0000 7efc5fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efc6013df80 0x7efc6015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41202/0 (socket says 172.21.15.26:41202) 2024-01-15T17:56:23.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.567+0000 7efc5fe00640 1 -- 172.21.15.26:0/1759215771 learned_addr learned my addr 172.21.15.26:0/1759215771 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:23.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.567+0000 7efc65200640 1 --2- 172.21.15.26:0/1759215771 >> v2:172.21.15.124:3300/0 conn(0x7efc6015f9c0 0x7efc600cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.567+0000 7efc57200640 1 --2- 172.21.15.26:0/1759215771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6013e360 0x7efc600cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.567+0000 7efc5fe00640 1 -- 172.21.15.26:0/1759215771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6013e360 msgr2=0x7efc600cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:23.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.567+0000 7efc5fe00640 1 --2- 172.21.15.26:0/1759215771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6013e360 0x7efc600cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.567+0000 7efc5fe00640 1 -- 172.21.15.26:0/1759215771 >> v2:172.21.15.124:3300/0 conn(0x7efc6015f9c0 msgr2=0x7efc600cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:23.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.567+0000 7efc5fe00640 1 --2- 172.21.15.26:0/1759215771 >> v2:172.21.15.124:3300/0 conn(0x7efc6015f9c0 0x7efc600cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.567+0000 7efc5fe00640 1 -- 172.21.15.26:0/1759215771 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6006cd10 con 0x7efc6013df80 2024-01-15T17:56:23.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.568+0000 7efc5fe00640 1 --2- 172.21.15.26:0/1759215771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013df80 0x7efc6015bee0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efc6006e6d0 tx=0x7efc5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416cc0378fcd4040 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:23.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.569+0000 7efc5f400640 1 -- 172.21.15.26:0/1759215771 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5002ca70 con 0x7efc6013df80 2024-01-15T17:56:23.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.569+0000 7efc5f400640 1 -- 172.21.15.26:0/1759215771 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5002cbd0 con 0x7efc6013df80 2024-01-15T17:56:23.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.569+0000 7efc5f400640 1 -- 172.21.15.26:0/1759215771 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc500343f0 con 0x7efc6013df80 2024-01-15T17:56:23.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.569+0000 7efc67200640 1 -- 172.21.15.26:0/1759215771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013df80 msgr2=0x7efc6015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.570+0000 7efc67200640 1 --2- 172.21.15.26:0/1759215771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013df80 0x7efc6015bee0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efc6006e6d0 tx=0x7efc5002bed0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.570+0000 7efc67200640 1 -- 172.21.15.26:0/1759215771 shutdown_connections 2024-01-15T17:56:23.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.570+0000 7efc67200640 1 --2- 172.21.15.26:0/1759215771 >> v2:172.21.15.124:3300/0 conn(0x7efc6015f9c0 0x7efc600cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.570+0000 7efc67200640 1 --2- 172.21.15.26:0/1759215771 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6013e360 0x7efc600cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.570+0000 7efc67200640 1 --2- 172.21.15.26:0/1759215771 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013df80 0x7efc6015bee0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.570+0000 7efc67200640 1 -- 172.21.15.26:0/1759215771 >> 172.21.15.26:0/1759215771 conn(0x7efc6006a800 msgr2=0x7efc6006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:23.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.570+0000 7efc67200640 1 -- 172.21.15.26:0/1759215771 shutdown_connections 2024-01-15T17:56:23.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.570+0000 7efc67200640 1 -- 172.21.15.26:0/1759215771 wait complete. 2024-01-15T17:56:23.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.572+0000 7efc67200640 1 Processor -- start 2024-01-15T17:56:23.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.572+0000 7efc67200640 1 -- start start 2024-01-15T17:56:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.573+0000 7efc67200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc6013df80 0x7efc60155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.573+0000 7efc67200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013e360 0x7efc600b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.573+0000 7efc67200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6015f9c0 0x7efc600af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.573+0000 7efc67200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc600aa5d0 con 0x7efc6013e360 2024-01-15T17:56:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.573+0000 7efc67200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc600ad6d0 con 0x7efc6013df80 2024-01-15T17:56:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.573+0000 7efc67200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc600ad9b0 con 0x7efc6015f9c0 2024-01-15T17:56:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.573+0000 7efc57200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013e360 0x7efc600b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.573+0000 7efc57200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013e360 0x7efc600b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41204/0 (socket says 172.21.15.26:41204) 2024-01-15T17:56:23.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.573+0000 7efc57200640 1 -- 172.21.15.26:0/1983001130 learned_addr learned my addr 172.21.15.26:0/1983001130 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc65200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6015f9c0 0x7efc600af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc57200640 1 -- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6015f9c0 msgr2=0x7efc600af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc57200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6015f9c0 0x7efc600af6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc57200640 1 -- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc6013df80 msgr2=0x7efc60155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc57200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc6013df80 0x7efc60155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc57200640 1 -- 172.21.15.26:0/1983001130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50009b30 con 0x7efc6013e360 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc57200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013e360 0x7efc600b3380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efc4800d800 tx=0x7efc4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc5cc00640 1 -- 172.21.15.26:0/1983001130 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800b6c0 con 0x7efc6013e360 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc5cc00640 1 -- 172.21.15.26:0/1983001130 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4800b840 con 0x7efc6013e360 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc600adc90 con 0x7efc6013e360 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc5cc00640 1 -- 172.21.15.26:0/1983001130 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800bbf0 con 0x7efc6013e360 2024-01-15T17:56:23.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.574+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc600a84c0 con 0x7efc6013e360 2024-01-15T17:56:23.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.575+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc600675d0 con 0x7efc6013e360 2024-01-15T17:56:23.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.576+0000 7efc5cc00640 1 -- 172.21.15.26:0/1983001130 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7efc4800bd50 con 0x7efc6013e360 2024-01-15T17:56:23.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.577+0000 7efc5cc00640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efc3806b2c0 0x7efc3806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:23.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.577+0000 7efc5cc00640 1 -- 172.21.15.26:0/1983001130 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efc48089f20 con 0x7efc6013e360 2024-01-15T17:56:23.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.578+0000 7efc5fe00640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efc3806b2c0 0x7efc3806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:23.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.579+0000 7efc5fe00640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efc3806b2c0 0x7efc3806d780 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc50035000 tx=0x7efc5002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:23.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.582+0000 7efc5cc00640 1 -- 172.21.15.26:0/1983001130 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc48055040 con 0x7efc6013e360 2024-01-15T17:56:23.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.776+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc6006df00 con 0x7efc6013e360 2024-01-15T17:56:23.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.777+0000 7efc5cc00640 1 -- 172.21.15.26:0/1983001130 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc48058660 con 0x7efc6013e360 2024-01-15T17:56:23.780 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:56:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efc3806b2c0 msgr2=0x7efc3806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efc3806b2c0 0x7efc3806d780 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc50035000 tx=0x7efc5002e000 comp rx=0 tx=0).stop 2024-01-15T17:56:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013e360 msgr2=0x7efc600b3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013e360 0x7efc600b3380 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efc4800d800 tx=0x7efc4800dcc0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 shutdown_connections 2024-01-15T17:56:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efc6015f9c0 0x7efc600af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efc3806b2c0 0x7efc3806d780 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efc6013e360 0x7efc600b3380 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 --2- 172.21.15.26:0/1983001130 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efc6013df80 0x7efc60155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 >> 172.21.15.26:0/1983001130 conn(0x7efc6006a800 msgr2=0x7efc600ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 shutdown_connections 2024-01-15T17:56:23.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:23.780+0000 7efc67200640 1 -- 172.21.15.26:0/1983001130 wait complete. 2024-01-15T17:56:23.803 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T17:56:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.026+0000 7f9945a00640 1 Processor -- start 2024-01-15T17:56:24.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.026+0000 7f9945a00640 1 -- start start 2024-01-15T17:56:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.027+0000 7f9945a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9940160920 0x7f99400b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.027+0000 7f9945a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99400d0a10 con 0x7f99400cf9f0 2024-01-15T17:56:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.027+0000 7f9945a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9940161e30 con 0x7f99400cfea0 2024-01-15T17:56:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.027+0000 7f9945a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99400d1f50 con 0x7f9940160920 2024-01-15T17:56:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.028+0000 7f993f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f99400cf9f0 0x7f9940161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54872/0 (socket says 172.21.15.26:54872) 2024-01-15T17:56:24.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.028+0000 7f993f400640 1 -- 172.21.15.26:0/1695776905 learned_addr learned my addr 172.21.15.26:0/1695776905 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.028+0000 7f993fe00640 1 --2- 172.21.15.26:0/1695776905 >> v2:172.21.15.124:3300/0 conn(0x7f9940160920 0x7f99400b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.028+0000 7f993ea00640 1 -- 172.21.15.26:0/1695776905 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609982384 0 0) 0x7f99400d0a10 con 0x7f99400cf9f0 2024-01-15T17:56:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.028+0000 7f993ea00640 1 -- 172.21.15.26:0/1695776905 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9920003520 con 0x7f99400cf9f0 2024-01-15T17:56:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.029+0000 7f993fe00640 1 -- 172.21.15.26:0/1695776905 >> v1:172.21.15.124:6789/0 conn(0x7f99400cfea0 legacy=0x7f9940161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.029+0000 7f993fe00640 1 -- 172.21.15.26:0/1695776905 >> v1:172.21.15.26:6789/0 conn(0x7f99400cf9f0 legacy=0x7f9940161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.029+0000 7f993fe00640 1 -- 172.21.15.26:0/1695776905 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400cf080 con 0x7f9940160920 2024-01-15T17:56:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.030+0000 7f993fe00640 1 --2- 172.21.15.26:0/1695776905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9940160920 0x7f99400b3f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f99300097a0 tx=0x7f993002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c339468ebdcfcb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f993ea00640 1 -- 172.21.15.26:0/1695776905 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930030610 con 0x7f9940160920 2024-01-15T17:56:24.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f993ea00640 1 -- 172.21.15.26:0/1695776905 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9930030770 con 0x7f9940160920 2024-01-15T17:56:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f993ea00640 1 -- 172.21.15.26:0/1695776905 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930030ae0 con 0x7f9940160920 2024-01-15T17:56:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f9945a00640 1 -- 172.21.15.26:0/1695776905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9940160920 msgr2=0x7f99400b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f9945a00640 1 --2- 172.21.15.26:0/1695776905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9940160920 0x7f99400b3f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f99300097a0 tx=0x7f993002b960 comp rx=0 tx=0).stop 2024-01-15T17:56:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f9945a00640 1 -- 172.21.15.26:0/1695776905 shutdown_connections 2024-01-15T17:56:24.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f9945a00640 1 --2- 172.21.15.26:0/1695776905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9940160920 0x7f99400b3f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f9945a00640 1 -- 172.21.15.26:0/1695776905 >> 172.21.15.26:0/1695776905 conn(0x7f99400d3090 msgr2=0x7f99400d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:24.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f9945a00640 1 -- 172.21.15.26:0/1695776905 shutdown_connections 2024-01-15T17:56:24.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.031+0000 7f9945a00640 1 -- 172.21.15.26:0/1695776905 wait complete. 2024-01-15T17:56:24.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.033+0000 7f9945a00640 1 Processor -- start 2024-01-15T17:56:24.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.033+0000 7f9945a00640 1 -- start start 2024-01-15T17:56:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.034+0000 7f9945a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cf9f0 0x7f99401516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.034+0000 7f9945a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99400cfea0 0x7f99400caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.034+0000 7f9945a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9940160920 0x7f99401f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.034+0000 7f9945a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9940151be0 con 0x7f99400cf9f0 2024-01-15T17:56:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.034+0000 7f9945a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99401f7540 con 0x7f99400cfea0 2024-01-15T17:56:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.034+0000 7f9945a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99401f6f80 con 0x7f9940160920 2024-01-15T17:56:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.034+0000 7f993fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cf9f0 0x7f99401516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f993fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cf9f0 0x7f99401516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41210/0 (socket says 172.21.15.26:41210) 2024-01-15T17:56:24.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f993fe00640 1 -- 172.21.15.26:0/2495828290 learned_addr learned my addr 172.21.15.26:0/2495828290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f993f400640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99400cfea0 0x7f99400caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f9945000640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9940160920 0x7f99401f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f993fe00640 1 -- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9940160920 msgr2=0x7f99401f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f993fe00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9940160920 0x7f99401f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f993fe00640 1 -- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99400cfea0 msgr2=0x7f99400caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f993fe00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99400cfea0 0x7f99400caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f993fe00640 1 -- 172.21.15.26:0/2495828290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930009400 con 0x7f99400cf9f0 2024-01-15T17:56:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.035+0000 7f993fe00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cf9f0 0x7f99401516a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f99300097a0 tx=0x7f99300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.036+0000 7f9937e00640 1 -- 172.21.15.26:0/2495828290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930040070 con 0x7f99400cf9f0 2024-01-15T17:56:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.036+0000 7f9937e00640 1 -- 172.21.15.26:0/2495828290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9930006e10 con 0x7f99400cf9f0 2024-01-15T17:56:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.036+0000 7f9937e00640 1 -- 172.21.15.26:0/2495828290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930030ea0 con 0x7f99400cf9f0 2024-01-15T17:56:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.036+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99401f7260 con 0x7f99400cf9f0 2024-01-15T17:56:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.036+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99400ca570 con 0x7f99400cf9f0 2024-01-15T17:56:24.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.037+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9940067450 con 0x7f99400cf9f0 2024-01-15T17:56:24.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.038+0000 7f9937e00640 1 -- 172.21.15.26:0/2495828290 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f9930003810 con 0x7f99400cf9f0 2024-01-15T17:56:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.039+0000 7f9937e00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f991806b2c0 0x7f991806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.039+0000 7f9937e00640 1 -- 172.21.15.26:0/2495828290 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f99300ab0d0 con 0x7f99400cf9f0 2024-01-15T17:56:24.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.040+0000 7f993f400640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f991806b2c0 0x7f991806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.041+0000 7f993f400640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f991806b2c0 0x7f991806d780 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f994008b060 tx=0x7f9928009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:24.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.043+0000 7f9937e00640 1 -- 172.21.15.26:0/2495828290 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f99300764f0 con 0x7f99400cf9f0 2024-01-15T17:56:24.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.243+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f994008c7e0 con 0x7f99400cf9f0 2024-01-15T17:56:24.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.244+0000 7f9937e00640 1 -- 172.21.15.26:0/2495828290 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9930079b10 con 0x7f99400cf9f0 2024-01-15T17:56:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:24.248 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T17:56:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-15T17:56:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-15T17:56:24.248 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:24.248 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T17:56:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f991806b2c0 msgr2=0x7f991806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f991806b2c0 0x7f991806d780 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f994008b060 tx=0x7f9928009040 comp rx=0 tx=0).stop 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cf9f0 msgr2=0x7f99401516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cf9f0 0x7f99401516a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f99300097a0 tx=0x7f99300046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 shutdown_connections 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9940160920 0x7f99401f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f991806b2c0 0x7f991806d780 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99400cfea0 0x7f99400caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 --2- 172.21.15.26:0/2495828290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99400cf9f0 0x7f99401516a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 >> 172.21.15.26:0/2495828290 conn(0x7f99400d3090 msgr2=0x7f994006c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 shutdown_connections 2024-01-15T17:56:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.247+0000 7f9945a00640 1 -- 172.21.15.26:0/2495828290 wait complete. 2024-01-15T17:56:24.270 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T17:56:24.270 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T17:56:24.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.498+0000 7fcf43000640 1 Processor -- start 2024-01-15T17:56:24.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.499+0000 7fcf43000640 1 -- start start 2024-01-15T17:56:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.500+0000 7fcf43000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 0x7fcf3c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.500+0000 7fcf43000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcf3c0cf7d0 0x7fcf3c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.500+0000 7fcf43000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf3c15e1e0 0x7fcf3c15a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.500+0000 7fcf43000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3c0cfc30 con 0x7fcf3c15e1e0 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.500+0000 7fcf43000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3c0b1ee0 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.500+0000 7fcf43000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c0b2780 con 0x7fcf3c143ae0 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf41000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf3c15e1e0 0x7fcf3c15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf41000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcf3c15e1e0 0x7fcf3c15a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41216/0 (socket says 172.21.15.26:41216) 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf41000640 1 -- 172.21.15.26:0/867131034 learned_addr learned my addr 172.21.15.26:0/867131034 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf3be00640 1 --2- 172.21.15.26:0/867131034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 0x7fcf3c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf33e00640 1 --2- 172.21.15.26:0/867131034 >> v2:172.21.15.124:3300/0 conn(0x7fcf3c0cf7d0 0x7fcf3c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf41000640 1 -- 172.21.15.26:0/867131034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 msgr2=0x7fcf3c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf41000640 1 --2- 172.21.15.26:0/867131034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 0x7fcf3c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf41000640 1 -- 172.21.15.26:0/867131034 >> v2:172.21.15.124:3300/0 conn(0x7fcf3c0cf7d0 msgr2=0x7fcf3c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf41000640 1 --2- 172.21.15.26:0/867131034 >> v2:172.21.15.124:3300/0 conn(0x7fcf3c0cf7d0 0x7fcf3c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf41000640 1 -- 172.21.15.26:0/867131034 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c15e990 con 0x7fcf3c15e1e0 2024-01-15T17:56:24.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf3be00640 1 --2- 172.21.15.26:0/867131034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 0x7fcf3c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:24.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.501+0000 7fcf33e00640 1 --2- 172.21.15.26:0/867131034 >> v2:172.21.15.124:3300/0 conn(0x7fcf3c0cf7d0 0x7fcf3c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:24.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.503+0000 7fcf41000640 1 --2- 172.21.15.26:0/867131034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c15e1e0 0x7fcf3c15a670 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c06e350 tx=0x7fcf3402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f323f8b230acc4fd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:24.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.503+0000 7fcf3b400640 1 -- 172.21.15.26:0/867131034 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3402ca80 con 0x7fcf3c15e1e0 2024-01-15T17:56:24.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.503+0000 7fcf3b400640 1 -- 172.21.15.26:0/867131034 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3402cbe0 con 0x7fcf3c15e1e0 2024-01-15T17:56:24.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.503+0000 7fcf3b400640 1 -- 172.21.15.26:0/867131034 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf340343f0 con 0x7fcf3c15e1e0 2024-01-15T17:56:24.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.504+0000 7fcf43000640 1 -- 172.21.15.26:0/867131034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c15e1e0 msgr2=0x7fcf3c15a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.504+0000 7fcf43000640 1 --2- 172.21.15.26:0/867131034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c15e1e0 0x7fcf3c15a670 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c06e350 tx=0x7fcf3402bee0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.504+0000 7fcf43000640 1 -- 172.21.15.26:0/867131034 shutdown_connections 2024-01-15T17:56:24.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.504+0000 7fcf43000640 1 --2- 172.21.15.26:0/867131034 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c15e1e0 0x7fcf3c15a670 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.504+0000 7fcf43000640 1 --2- 172.21.15.26:0/867131034 >> v2:172.21.15.124:3300/0 conn(0x7fcf3c0cf7d0 0x7fcf3c15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.504+0000 7fcf43000640 1 --2- 172.21.15.26:0/867131034 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 0x7fcf3c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.504+0000 7fcf43000640 1 -- 172.21.15.26:0/867131034 >> 172.21.15.26:0/867131034 conn(0x7fcf3c06a800 msgr2=0x7fcf3c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:24.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.504+0000 7fcf43000640 1 -- 172.21.15.26:0/867131034 shutdown_connections 2024-01-15T17:56:24.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.504+0000 7fcf43000640 1 -- 172.21.15.26:0/867131034 wait complete. 2024-01-15T17:56:24.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.506+0000 7fcf43000640 1 Processor -- start 2024-01-15T17:56:24.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.507+0000 7fcf43000640 1 -- start start 2024-01-15T17:56:24.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.508+0000 7fcf43000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c0cf7d0 0x7fcf3c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.508+0000 7fcf43000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 0x7fcf3c13a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.508+0000 7fcf43000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf3c15e1e0 0x7fcf3c13ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.508+0000 7fcf43000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c15aa20 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.508+0000 7fcf43000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c141680 con 0x7fcf3c15e1e0 2024-01-15T17:56:24.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.508+0000 7fcf43000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c141960 con 0x7fcf3c143ae0 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.508+0000 7fcf3be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c0cf7d0 0x7fcf3c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.508+0000 7fcf3be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c0cf7d0 0x7fcf3c15a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41230/0 (socket says 172.21.15.26:41230) 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.508+0000 7fcf3be00640 1 -- 172.21.15.26:0/3286582881 learned_addr learned my addr 172.21.15.26:0/3286582881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.509+0000 7fcf33e00640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 0x7fcf3c13a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.509+0000 7fcf41000640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf3c15e1e0 0x7fcf3c13ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.509+0000 7fcf3be00640 1 -- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 msgr2=0x7fcf3c13a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.509+0000 7fcf3be00640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 0x7fcf3c13a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.509+0000 7fcf3be00640 1 -- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf3c15e1e0 msgr2=0x7fcf3c13ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.509+0000 7fcf3be00640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf3c15e1e0 0x7fcf3c13ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.509+0000 7fcf3be00640 1 -- 172.21.15.26:0/3286582881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34009b30 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.509+0000 7fcf3be00640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c0cf7d0 0x7fcf3c15a4e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00e7e0 tx=0x7fcf2c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.509+0000 7fcf38c00640 1 -- 172.21.15.26:0/3286582881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c004030 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.510+0000 7fcf38c00640 1 -- 172.21.15.26:0/3286582881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c004190 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.510+0000 7fcf38c00640 1 -- 172.21.15.26:0/3286582881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c004520 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.510+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c141c40 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.510+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c142200 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.511+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c0675d0 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.512+0000 7fcf38c00640 1 -- 172.21.15.26:0/3286582881 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fcf2c00c6d0 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.513+0000 7fcf38c00640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf1406b2c0 0x7fcf1406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:24.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.513+0000 7fcf38c00640 1 -- 172.21.15.26:0/3286582881 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcf2c089520 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.514+0000 7fcf33e00640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf1406b2c0 0x7fcf1406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:24.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.515+0000 7fcf33e00640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf1406b2c0 0x7fcf1406d780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf24006fd0 tx=0x7fcf24008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:24.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.517+0000 7fcf38c00640 1 -- 172.21.15.26:0/3286582881 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf2c0546e0 con 0x7fcf3c0cf7d0 2024-01-15T17:56:24.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:24.711+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcf3c13e480 con 0x7fcf3c0cf7d0 2024-01-15T17:56:25.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-01-15T17:56:25.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-01-15T17:56:25.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.164+0000 7fcf38c00640 1 -- 172.21.15.26:0/3286582881 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcf2c057d00 con 0x7fcf3c0cf7d0 2024-01-15T17:56:25.167 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-15T17:56:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf1406b2c0 msgr2=0x7fcf1406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf1406b2c0 0x7fcf1406d780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcf24006fd0 tx=0x7fcf24008040 comp rx=0 tx=0).stop 2024-01-15T17:56:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c0cf7d0 msgr2=0x7fcf3c15a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c0cf7d0 0x7fcf3c15a4e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00e7e0 tx=0x7fcf2c00eca0 comp rx=0 tx=0).stop 2024-01-15T17:56:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 shutdown_connections 2024-01-15T17:56:25.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcf3c143ae0 0x7fcf3c13a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf1406b2c0 0x7fcf1406d780 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcf3c15e1e0 0x7fcf3c13ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 --2- 172.21.15.26:0/3286582881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcf3c0cf7d0 0x7fcf3c15a4e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.168+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 >> 172.21.15.26:0/3286582881 conn(0x7fcf3c06a800 msgr2=0x7fcf3c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:25.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.169+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 shutdown_connections 2024-01-15T17:56:25.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.169+0000 7fcf43000640 1 -- 172.21.15.26:0/3286582881 wait complete. 2024-01-15T17:56:25.198 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T17:56:25.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.405+0000 7fdfa7400640 1 Processor -- start 2024-01-15T17:56:25.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.406+0000 7fdfa7400640 1 -- start start 2024-01-15T17:56:25.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.407+0000 7fdfa7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdfa0141360 0x7fdfa00cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:25.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.407+0000 7fdfa7400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa013e440 con 0x7fdfa00cdcc0 2024-01-15T17:56:25.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.407+0000 7fdfa7400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa00ce1d0 con 0x7fdfa015ed20 2024-01-15T17:56:25.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.407+0000 7fdfa7400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa00ce4b0 con 0x7fdfa0141360 2024-01-15T17:56:25.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.408+0000 7fdfa5400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdfa00cdcc0 0x7fdfa006c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54884/0 (socket says 172.21.15.26:54884) 2024-01-15T17:56:25.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.408+0000 7fdfa5400640 1 -- 172.21.15.26:0/2306886528 learned_addr learned my addr 172.21.15.26:0/2306886528 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:25.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.408+0000 7fdf9fe00640 1 --2- 172.21.15.26:0/2306886528 >> v2:172.21.15.26:3300/0 conn(0x7fdfa0141360 0x7fdfa00cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:25.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.408+0000 7fdf9f400640 1 -- 172.21.15.26:0/2306886528 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625462231 0 0) 0x7fdfa013e440 con 0x7fdfa00cdcc0 2024-01-15T17:56:25.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.408+0000 7fdf9f400640 1 -- 172.21.15.26:0/2306886528 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf80003520 con 0x7fdfa00cdcc0 2024-01-15T17:56:25.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.408+0000 7fdf9fe00640 1 -- 172.21.15.26:0/2306886528 >> v1:172.21.15.124:6789/0 conn(0x7fdfa015ed20 legacy=0x7fdfa015f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:25.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.408+0000 7fdf9fe00640 1 -- 172.21.15.26:0/2306886528 >> v1:172.21.15.26:6789/0 conn(0x7fdfa00cdcc0 legacy=0x7fdfa006c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:25.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.408+0000 7fdf9fe00640 1 -- 172.21.15.26:0/2306886528 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa00ce790 con 0x7fdfa0141360 2024-01-15T17:56:25.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.410+0000 7fdf9fe00640 1 --2- 172.21.15.26:0/2306886528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 0x7fdfa00cd780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdf880097a0 tx=0x7fdf8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319d804b702e6e84 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:25.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.410+0000 7fdf9f400640 1 -- 172.21.15.26:0/2306886528 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88031830 con 0x7fdfa0141360 2024-01-15T17:56:25.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.410+0000 7fdf9f400640 1 -- 172.21.15.26:0/2306886528 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf88031990 con 0x7fdfa0141360 2024-01-15T17:56:25.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.410+0000 7fdf9f400640 1 -- 172.21.15.26:0/2306886528 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88031d20 con 0x7fdfa0141360 2024-01-15T17:56:25.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.411+0000 7fdfa7400640 1 -- 172.21.15.26:0/2306886528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 msgr2=0x7fdfa00cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:25.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.411+0000 7fdfa7400640 1 --2- 172.21.15.26:0/2306886528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 0x7fdfa00cd780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdf880097a0 tx=0x7fdf8802b8d0 comp rx=0 tx=0).stop 2024-01-15T17:56:25.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.411+0000 7fdfa7400640 1 -- 172.21.15.26:0/2306886528 shutdown_connections 2024-01-15T17:56:25.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.411+0000 7fdfa7400640 1 --2- 172.21.15.26:0/2306886528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 0x7fdfa00cd780 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:25.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.411+0000 7fdfa7400640 1 -- 172.21.15.26:0/2306886528 >> 172.21.15.26:0/2306886528 conn(0x7fdfa00b0be0 msgr2=0x7fdfa00b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:25.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.411+0000 7fdfa7400640 1 -- 172.21.15.26:0/2306886528 shutdown_connections 2024-01-15T17:56:25.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.411+0000 7fdfa7400640 1 -- 172.21.15.26:0/2306886528 wait complete. 2024-01-15T17:56:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.413+0000 7fdfa7400640 1 Processor -- start 2024-01-15T17:56:25.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.413+0000 7fdfa7400640 1 -- start start 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfa00cdcc0 0x7fdfa00adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 0x7fdfa01ee070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdfa7400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfa015ed20 0x7fdfa0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdfa7400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa0072030 con 0x7fdfa0141360 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdfa7400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00b2b90 con 0x7fdfa015ed20 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdfa7400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa00b2e70 con 0x7fdfa00cdcc0 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdf9fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 0x7fdfa01ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdf9fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 0x7fdfa01ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41248/0 (socket says 172.21.15.26:41248) 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdf9fe00640 1 -- 172.21.15.26:0/2077871432 learned_addr learned my addr 172.21.15.26:0/2077871432 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.414+0000 7fdfa4a00640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfa00cdcc0 0x7fdfa00adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.415+0000 7fdfa5400640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfa015ed20 0x7fdfa0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.415+0000 7fdf9fe00640 1 -- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfa00cdcc0 msgr2=0x7fdfa00adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:25.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.415+0000 7fdf9fe00640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfa00cdcc0 0x7fdfa00adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:25.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.415+0000 7fdf9fe00640 1 -- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfa015ed20 msgr2=0x7fdfa0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:25.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.415+0000 7fdf9fe00640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfa015ed20 0x7fdfa0070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:25.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.415+0000 7fdf9fe00640 1 -- 172.21.15.26:0/2077871432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf88009400 con 0x7fdfa0141360 2024-01-15T17:56:25.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.415+0000 7fdfa4a00640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfa00cdcc0 0x7fdfa00adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:25.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.415+0000 7fdf9fe00640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 0x7fdfa01ee070 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdf8802bee0 tx=0x7fdf880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:25.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.416+0000 7fdf9cc00640 1 -- 172.21.15.26:0/2077871432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8803a6f0 con 0x7fdfa0141360 2024-01-15T17:56:25.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.416+0000 7fdf9cc00640 1 -- 172.21.15.26:0/2077871432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8803a870 con 0x7fdfa0141360 2024-01-15T17:56:25.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.416+0000 7fdf9cc00640 1 -- 172.21.15.26:0/2077871432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8803ac20 con 0x7fdfa0141360 2024-01-15T17:56:25.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.416+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa00b3150 con 0x7fdfa0141360 2024-01-15T17:56:25.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.416+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa01ee5b0 con 0x7fdfa0141360 2024-01-15T17:56:25.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.417+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa00b3950 con 0x7fdfa0141360 2024-01-15T17:56:25.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.418+0000 7fdf9cc00640 1 -- 172.21.15.26:0/2077871432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdf8804c3b0 con 0x7fdfa0141360 2024-01-15T17:56:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.419+0000 7fdf9cc00640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf7806b2c0 0x7fdf7806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.419+0000 7fdf9cc00640 1 -- 172.21.15.26:0/2077871432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fdf880ab7a0 con 0x7fdfa0141360 2024-01-15T17:56:25.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.420+0000 7fdfa4a00640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf7806b2c0 0x7fdf7806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:25.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.421+0000 7fdfa4a00640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf7806b2c0 0x7fdf7806d780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdfa015b190 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:25.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.425+0000 7fdf9cc00640 1 -- 172.21.15.26:0/2077871432 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdfa00b3950 con 0x7fdfa0141360 2024-01-15T17:56:25.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:25.618+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdfa00b3950 con 0x7fdfa0141360 2024-01-15T17:56:25.970 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:56:25.978 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:56:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.170+0000 7fdf9cc00640 1 -- 172.21.15.26:0/2077871432 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdfa00b3950 con 0x7fdfa0141360 2024-01-15T17:56:26.173 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-15T17:56:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf7806b2c0 msgr2=0x7fdf7806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf7806b2c0 0x7fdf7806d780 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdfa015b190 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2024-01-15T17:56:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 msgr2=0x7fdfa01ee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 0x7fdfa01ee070 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdf8802bee0 tx=0x7fdf880046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 shutdown_connections 2024-01-15T17:56:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf7806b2c0 0x7fdf7806d780 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfa015ed20 0x7fdfa0070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfa0141360 0x7fdfa01ee070 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 --2- 172.21.15.26:0/2077871432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdfa00cdcc0 0x7fdfa00adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.174+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 >> 172.21.15.26:0/2077871432 conn(0x7fdfa00b0be0 msgr2=0x7fdfa015f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.175+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 shutdown_connections 2024-01-15T17:56:26.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.175+0000 7fdfa7400640 1 -- 172.21.15.26:0/2077871432 wait complete. 2024-01-15T17:56:26.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T17:56:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.397+0000 7f8988a00640 1 Processor -- start 2024-01-15T17:56:26.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.397+0000 7f8988a00640 1 -- start start 2024-01-15T17:56:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.398+0000 7f8988a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89800cf9a0 0x7f898015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.398+0000 7f8988a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8980070c40 con 0x7f89800ae5a0 2024-01-15T17:56:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.398+0000 7f8988a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898006c470 con 0x7f89800b3ae0 2024-01-15T17:56:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.398+0000 7f8988a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8980071450 con 0x7f89800cf9a0 2024-01-15T17:56:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.399+0000 7f8986000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f89800ae5a0 0x7f898006cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54896/0 (socket says 172.21.15.26:54896) 2024-01-15T17:56:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.399+0000 7f8986000640 1 -- 172.21.15.26:0/1917589275 learned_addr learned my addr 172.21.15.26:0/1917589275 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.399+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685453801 0 0) 0x7f8980070c40 con 0x7f89800ae5a0 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.399+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8960003520 con 0x7f89800ae5a0 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.399+0000 7f8986a00640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89800cf9a0 0x7f898015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248665037 0 0) 0x7f8960003520 con 0x7f89800ae5a0 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89800cf9a0 msgr2=0x7f898015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89800cf9a0 0x7f898015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 >> v1:172.21.15.124:6789/0 conn(0x7f89800b3ae0 legacy=0x7f89800cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898006c750 con 0x7f89800ae5a0 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8970001670 con 0x7f89800ae5a0 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 >> v1:172.21.15.26:6789/0 conn(0x7f89800ae5a0 legacy=0x7f898006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89800cf9a0 0x7f89600039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8960003fd0 0x7f89600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960004d00 0x7f8960005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8980071450 con 0x7f8960003fd0 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896000bbd0 con 0x7f8960004d00 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896000bd90 con 0x7f89800cf9a0 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8985600640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8960003fd0 0x7f89600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8986a00640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960004d00 0x7f8960005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.400+0000 7f8986000640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89800cf9a0 0x7f89600039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.401+0000 7f8985600640 1 -- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89800cf9a0 msgr2=0x7f89600039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.401+0000 7f8985600640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89800cf9a0 0x7f89600039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.401+0000 7f8985600640 1 -- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960004d00 msgr2=0x7f8960005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.401+0000 7f8985600640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960004d00 0x7f8960005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.401+0000 7f8985600640 1 -- 172.21.15.26:0/1917589275 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8960003520 con 0x7f8960003fd0 2024-01-15T17:56:26.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.401+0000 7f8986000640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89800cf9a0 0x7f89600039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.402+0000 7f8985600640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8960003fd0 0x7f89600047c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8968001750 tx=0x7f89600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267cae016d15bbb6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.402+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680052d0 con 0x7f8960003fd0 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.402+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896802e050 con 0x7f8960003fd0 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.402+0000 7f8984c00640 1 -- 172.21.15.26:0/1917589275 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968003830 con 0x7f8960003fd0 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.402+0000 7f8988a00640 1 -- 172.21.15.26:0/1917589275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8960003fd0 msgr2=0x7f89600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.402+0000 7f8988a00640 1 --2- 172.21.15.26:0/1917589275 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8960003fd0 0x7f89600047c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8968001750 tx=0x7f89600057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.403+0000 7f8986a00640 1 -- 172.21.15.26:0/1917589275 reap_dead start 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.403+0000 7f8988a00640 1 -- 172.21.15.26:0/1917589275 shutdown_connections 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.403+0000 7f8988a00640 1 -- 172.21.15.26:0/1917589275 >> 172.21.15.26:0/1917589275 conn(0x7f898006a800 msgr2=0x7f898006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.403+0000 7f8988a00640 1 -- 172.21.15.26:0/1917589275 shutdown_connections 2024-01-15T17:56:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.403+0000 7f8988a00640 1 -- 172.21.15.26:0/1917589275 wait complete. 2024-01-15T17:56:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.405+0000 7f8988a00640 1 Processor -- start 2024-01-15T17:56:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.405+0000 7f8988a00640 1 -- start start 2024-01-15T17:56:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8988a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960003fd0 0x7f89800cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8988a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89801471a0 0x7f8980159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8988a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89801599b0 0x7f89800cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8988a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8980148240 con 0x7f89801599b0 2024-01-15T17:56:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8988a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8980147630 con 0x7f8960003fd0 2024-01-15T17:56:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8988a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8980147910 con 0x7f89801471a0 2024-01-15T17:56:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89801599b0 0x7f89800cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89801599b0 0x7f89800cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41274/0 (socket says 172.21.15.26:41274) 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986a00640 1 -- 172.21.15.26:0/945930564 learned_addr learned my addr 172.21.15.26:0/945930564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986000640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960003fd0 0x7f89800cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986a00640 1 -- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89801471a0 msgr2=0x7f8980159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986a00640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89801471a0 0x7f8980159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986a00640 1 -- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960003fd0 msgr2=0x7f89800cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986a00640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960003fd0 0x7f89800cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986a00640 1 -- 172.21.15.26:0/945930564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8980147bf0 con 0x7f89801599b0 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.406+0000 7f8986000640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960003fd0 0x7f89800cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.407+0000 7f8986a00640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89801599b0 0x7f89800cdc10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f89800cff50 tx=0x7f8978009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.407+0000 7f897e000640 1 -- 172.21.15.26:0/945930564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978007cf0 con 0x7f89801599b0 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.407+0000 7f897e000640 1 -- 172.21.15.26:0/945930564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897800f050 con 0x7f89801599b0 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.407+0000 7f897e000640 1 -- 172.21.15.26:0/945930564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8978014650 con 0x7f89801599b0 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.407+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8980147ed0 con 0x7f89801599b0 2024-01-15T17:56:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.407+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89800ce330 con 0x7f89801599b0 2024-01-15T17:56:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.408+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89800675d0 con 0x7f89801599b0 2024-01-15T17:56:26.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.410+0000 7f897e000640 1 -- 172.21.15.26:0/945930564 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f89780147b0 con 0x7f89801599b0 2024-01-15T17:56:26.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.410+0000 7f897e000640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f895806b2f0 0x7f895806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:26.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.411+0000 7f897e000640 1 -- 172.21.15.26:0/945930564 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f8978089ff0 con 0x7f89801599b0 2024-01-15T17:56:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.411+0000 7f8986000640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f895806b2f0 0x7f895806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:26.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.412+0000 7f8986000640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f895806b2f0 0x7f895806d7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f89800cbbd0 tx=0x7f8970009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:26.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.416+0000 7f897e000640 1 -- 172.21.15.26:0/945930564 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89780bce90 con 0x7f89801599b0 2024-01-15T17:56:26.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:26.645+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f89800b06a0 con 0x7f89801599b0 2024-01-15T17:56:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.211+0000 7f897e000640 1 -- 172.21.15.26:0/945930564 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8978054ef0 con 0x7f89801599b0 2024-01-15T17:56:27.217 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 4 and data pool 5 2024-01-15T17:56:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f895806b2f0 msgr2=0x7f895806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f895806b2f0 0x7f895806d7b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f89800cbbd0 tx=0x7f8970009040 comp rx=0 tx=0).stop 2024-01-15T17:56:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89801599b0 msgr2=0x7f89800cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89801599b0 0x7f89800cdc10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f89800cff50 tx=0x7f8978009d50 comp rx=0 tx=0).stop 2024-01-15T17:56:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 shutdown_connections 2024-01-15T17:56:27.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89801471a0 0x7f8980159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f895806b2f0 0x7f895806d7b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89801599b0 0x7f89800cdc10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 --2- 172.21.15.26:0/945930564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8960003fd0 0x7f89800cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.220+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 >> 172.21.15.26:0/945930564 conn(0x7f898006a800 msgr2=0x7f89800ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.222+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 shutdown_connections 2024-01-15T17:56:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.222+0000 7f8988a00640 1 -- 172.21.15.26:0/945930564 wait complete. 2024-01-15T17:56:27.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:27.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.436+0000 7f2960200640 1 Processor -- start 2024-01-15T17:56:27.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.436+0000 7f2960200640 1 -- start start 2024-01-15T17:56:27.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.438+0000 7f2960200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29580b3ae0 0x7f295815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.438+0000 7f2960200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 0x7f29580cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.438+0000 7f2960200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2958070c40 con 0x7f29580ae5a0 2024-01-15T17:56:27.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.438+0000 7f2960200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2958071450 con 0x7f29580b3ae0 2024-01-15T17:56:27.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.438+0000 7f2960200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29580cfc60 con 0x7f29580cf290 2024-01-15T17:56:27.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.439+0000 7f295ce00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29580b3ae0 0x7f295815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.439+0000 7f295ce00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29580b3ae0 0x7f295815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:56874/0 (socket says 172.21.15.26:56874) 2024-01-15T17:56:27.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.439+0000 7f295ce00640 1 -- 172.21.15.26:0/659233500 learned_addr learned my addr 172.21.15.26:0/659233500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:27.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.439+0000 7f295e200640 1 --2- 172.21.15.26:0/659233500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 0x7f29580cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.439+0000 7f295ce00640 1 -- 172.21.15.26:0/659233500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 msgr2=0x7f29580cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:27.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.439+0000 7f295ce00640 1 --2- 172.21.15.26:0/659233500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 0x7f29580cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.439+0000 7f295ce00640 1 -- 172.21.15.26:0/659233500 >> v1:172.21.15.124:6789/0 conn(0x7f29580ae5a0 legacy=0x7f295806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.439+0000 7f295ce00640 1 -- 172.21.15.26:0/659233500 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295806c470 con 0x7f29580b3ae0 2024-01-15T17:56:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.441+0000 7f295ce00640 1 --2- 172.21.15.26:0/659233500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580b3ae0 0x7f295815e430 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2940009870 tx=0x7f294002ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=168315fe9625efcf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.441+0000 7f2957e00640 1 -- 172.21.15.26:0/659233500 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294002f840 con 0x7f29580b3ae0 2024-01-15T17:56:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.442+0000 7f2957e00640 1 -- 172.21.15.26:0/659233500 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294002f9a0 con 0x7f29580b3ae0 2024-01-15T17:56:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.442+0000 7f2957e00640 1 -- 172.21.15.26:0/659233500 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294002fd30 con 0x7f29580b3ae0 2024-01-15T17:56:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.442+0000 7f2960200640 1 -- 172.21.15.26:0/659233500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580b3ae0 msgr2=0x7f295815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.442+0000 7f2960200640 1 --2- 172.21.15.26:0/659233500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580b3ae0 0x7f295815e430 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2940009870 tx=0x7f294002ba20 comp rx=0 tx=0).stop 2024-01-15T17:56:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.442+0000 7f2960200640 1 -- 172.21.15.26:0/659233500 shutdown_connections 2024-01-15T17:56:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.442+0000 7f2960200640 1 --2- 172.21.15.26:0/659233500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 0x7f29580cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.442+0000 7f2960200640 1 --2- 172.21.15.26:0/659233500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580b3ae0 0x7f295815e430 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.442+0000 7f2960200640 1 -- 172.21.15.26:0/659233500 >> 172.21.15.26:0/659233500 conn(0x7f295806a800 msgr2=0x7f295806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.443+0000 7f2960200640 1 -- 172.21.15.26:0/659233500 shutdown_connections 2024-01-15T17:56:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.443+0000 7f2960200640 1 -- 172.21.15.26:0/659233500 wait complete. 2024-01-15T17:56:27.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.444+0000 7f2960200640 1 Processor -- start 2024-01-15T17:56:27.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.444+0000 7f2960200640 1 -- start start 2024-01-15T17:56:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.445+0000 7f2960200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580ae5a0 0x7f29581514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.445+0000 7f2960200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29580b3ae0 0x7f295815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.445+0000 7f2960200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 0x7f295815f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.445+0000 7f2960200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295815cce0 con 0x7f29580b3ae0 2024-01-15T17:56:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.445+0000 7f2960200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295815c060 con 0x7f29580ae5a0 2024-01-15T17:56:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.445+0000 7f2960200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295815c340 con 0x7f29580cf290 2024-01-15T17:56:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29580b3ae0 0x7f295815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29580b3ae0 0x7f295815b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41278/0 (socket says 172.21.15.26:41278) 2024-01-15T17:56:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295ce00640 1 -- 172.21.15.26:0/3941018201 learned_addr learned my addr 172.21.15.26:0/3941018201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295e200640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 0x7f295815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295d800640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580ae5a0 0x7f29581514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295ce00640 1 -- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 msgr2=0x7f295815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295ce00640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 0x7f295815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295ce00640 1 -- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580ae5a0 msgr2=0x7f29581514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295ce00640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580ae5a0 0x7f29581514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295ce00640 1 -- 172.21.15.26:0/3941018201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29400094d0 con 0x7f29580b3ae0 2024-01-15T17:56:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.446+0000 7f295d800640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580ae5a0 0x7f29581514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:27.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.447+0000 7f295ce00640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29580b3ae0 0x7f295815b5a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2940000c00 tx=0x7f29400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:27.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.447+0000 7f2955600640 1 -- 172.21.15.26:0/3941018201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294003b040 con 0x7f29580b3ae0 2024-01-15T17:56:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.447+0000 7f2955600640 1 -- 172.21.15.26:0/3941018201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940037c60 con 0x7f29580b3ae0 2024-01-15T17:56:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.447+0000 7f2955600640 1 -- 172.21.15.26:0/3941018201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940042480 con 0x7f29580b3ae0 2024-01-15T17:56:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.447+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295815c620 con 0x7f29580b3ae0 2024-01-15T17:56:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.447+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29581506f0 con 0x7f29580b3ae0 2024-01-15T17:56:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.448+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295815ce20 con 0x7f29580b3ae0 2024-01-15T17:56:27.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.449+0000 7f2955600640 1 -- 172.21.15.26:0/3941018201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f29400425e0 con 0x7f29580b3ae0 2024-01-15T17:56:27.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.450+0000 7f2955600640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f293006b2c0 0x7f293006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.450+0000 7f2955600640 1 -- 172.21.15.26:0/3941018201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29400acc70 con 0x7f29580b3ae0 2024-01-15T17:56:27.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.451+0000 7f295d800640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f293006b2c0 0x7f293006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.452+0000 7f295d800640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f293006b2c0 0x7f293006d780 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2948001680 tx=0x7f2948009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.454+0000 7f2955600640 1 -- 172.21.15.26:0/3941018201 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f295815ce20 con 0x7f29580b3ae0 2024-01-15T17:56:27.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.648+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f295815ce20 con 0x7f29580b3ae0 2024-01-15T17:56:27.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.651+0000 7f2955600640 1 -- 172.21.15.26:0/3941018201 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16558 (secure 0 0 0) 0x7f295815ce20 con 0x7f29580b3ae0 2024-01-15T17:56:27.654 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:27.654 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":34,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:56:26.647342+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:56:24.764583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-15T17:56:25.673498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f293006b2c0 msgr2=0x7f293006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f293006b2c0 0x7f293006d780 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2948001680 tx=0x7f2948009040 comp rx=0 tx=0).stop 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29580b3ae0 msgr2=0x7f295815b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29580b3ae0 0x7f295815b5a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2940000c00 tx=0x7f29400046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 shutdown_connections 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29580cf290 0x7f295815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f293006b2c0 0x7f293006d780 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29580b3ae0 0x7f295815b5a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 --2- 172.21.15.26:0/3941018201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29580ae5a0 0x7f29581514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.653+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 >> 172.21.15.26:0/3941018201 conn(0x7f295806a800 msgr2=0x7f295815d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.654+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 shutdown_connections 2024-01-15T17:56:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.654+0000 7f2960200640 1 -- 172.21.15.26:0/3941018201 wait complete. 2024-01-15T17:56:27.676 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:27.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.875+0000 7f2247200640 1 Processor -- start 2024-01-15T17:56:27.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.875+0000 7f2247200640 1 -- start start 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f2247200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f224813cfa0 0x7f224815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f2247200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22480cd530 0x7f224813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f2247200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 0x7f224813fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f2247200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22480cdec0 con 0x7f22480cd530 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f2247200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22480ce1a0 con 0x7f224813cfa0 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f2247200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22480ce480 con 0x7f22480cd9e0 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f223fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22480cd530 0x7f224813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f223fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22480cd530 0x7f224813f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41282/0 (socket says 172.21.15.26:41282) 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f223fe00640 1 -- 172.21.15.26:0/3677489076 learned_addr learned my addr 172.21.15.26:0/3677489076 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f2246800640 1 --2- 172.21.15.26:0/3677489076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 0x7f224813fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f223fe00640 1 -- 172.21.15.26:0/3677489076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 msgr2=0x7f224813fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f2245e00640 1 --2- 172.21.15.26:0/3677489076 >> v2:172.21.15.124:3300/0 conn(0x7f224813cfa0 0x7f224815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f223fe00640 1 --2- 172.21.15.26:0/3677489076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 0x7f224813fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.877+0000 7f223fe00640 1 -- 172.21.15.26:0/3677489076 >> v2:172.21.15.124:3300/0 conn(0x7f224813cfa0 msgr2=0x7f224815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:27.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.878+0000 7f223fe00640 1 --2- 172.21.15.26:0/3677489076 >> v2:172.21.15.124:3300/0 conn(0x7f224813cfa0 0x7f224815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.878+0000 7f223fe00640 1 -- 172.21.15.26:0/3677489076 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22480ce760 con 0x7f22480cd530 2024-01-15T17:56:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.879+0000 7f223fe00640 1 --2- 172.21.15.26:0/3677489076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 0x7f224813f5a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f224806cf00 tx=0x7f223002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87abac3990c38bd4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.879+0000 7f2245400640 1 -- 172.21.15.26:0/3677489076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22300317d0 con 0x7f22480cd530 2024-01-15T17:56:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.879+0000 7f2245400640 1 -- 172.21.15.26:0/3677489076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2230031930 con 0x7f22480cd530 2024-01-15T17:56:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.879+0000 7f2245400640 1 -- 172.21.15.26:0/3677489076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230031cc0 con 0x7f22480cd530 2024-01-15T17:56:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.880+0000 7f2247200640 1 -- 172.21.15.26:0/3677489076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 msgr2=0x7f224813f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.880+0000 7f2247200640 1 --2- 172.21.15.26:0/3677489076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 0x7f224813f5a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f224806cf00 tx=0x7f223002b870 comp rx=0 tx=0).stop 2024-01-15T17:56:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.880+0000 7f2247200640 1 -- 172.21.15.26:0/3677489076 shutdown_connections 2024-01-15T17:56:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.880+0000 7f2247200640 1 --2- 172.21.15.26:0/3677489076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 0x7f224813fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.880+0000 7f2247200640 1 --2- 172.21.15.26:0/3677489076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 0x7f224813f5a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.880+0000 7f2247200640 1 --2- 172.21.15.26:0/3677489076 >> v2:172.21.15.124:3300/0 conn(0x7f224813cfa0 0x7f224815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.880+0000 7f2247200640 1 -- 172.21.15.26:0/3677489076 >> 172.21.15.26:0/3677489076 conn(0x7f224806a800 msgr2=0x7f224806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.880+0000 7f2247200640 1 -- 172.21.15.26:0/3677489076 shutdown_connections 2024-01-15T17:56:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.880+0000 7f2247200640 1 -- 172.21.15.26:0/3677489076 wait complete. 2024-01-15T17:56:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.881+0000 7f2247200640 1 Processor -- start 2024-01-15T17:56:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.882+0000 7f2247200640 1 -- start start 2024-01-15T17:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.882+0000 7f2247200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 0x7f22481548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.882+0000 7f2247200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 0x7f22481f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.882+0000 7f2247200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f224813cfa0 0x7f2248156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2247200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224813cb50 con 0x7f22480cd530 2024-01-15T17:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2247200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22481f3670 con 0x7f224813cfa0 2024-01-15T17:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2247200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2248154fb0 con 0x7f22480cd9e0 2024-01-15T17:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2245e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 0x7f22481548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2245e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 0x7f22481548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41286/0 (socket says 172.21.15.26:41286) 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2245e00640 1 -- 172.21.15.26:0/3925678483 learned_addr learned my addr 172.21.15.26:0/3925678483 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f223fe00640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 0x7f22481f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2245e00640 1 -- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 msgr2=0x7f22481f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2245e00640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 0x7f22481f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2245e00640 1 -- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f224813cfa0 msgr2=0x7f2248156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2246800640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f224813cfa0 0x7f2248156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2245e00640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f224813cfa0 0x7f2248156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2245e00640 1 -- 172.21.15.26:0/3925678483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300094d0 con 0x7f22480cd530 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.883+0000 7f2245e00640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 0x7f22481548d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f223800c7f0 tx=0x7f223800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.884+0000 7f223e000640 1 -- 172.21.15.26:0/3925678483 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238004220 con 0x7f22480cd530 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.884+0000 7f223e000640 1 -- 172.21.15.26:0/3925678483 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2238004380 con 0x7f22480cd530 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.884+0000 7f223e000640 1 -- 172.21.15.26:0/3925678483 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238007840 con 0x7f22480cd530 2024-01-15T17:56:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.884+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2248155290 con 0x7f22480cd530 2024-01-15T17:56:27.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.884+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2248155850 con 0x7f22480cd530 2024-01-15T17:56:27.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.885+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2248067450 con 0x7f22480cd530 2024-01-15T17:56:27.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.886+0000 7f223e000640 1 -- 172.21.15.26:0/3925678483 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2238007bb0 con 0x7f22480cd530 2024-01-15T17:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.887+0000 7f223e000640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f222006b2c0 0x7f222006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.887+0000 7f223e000640 1 -- 172.21.15.26:0/3925678483 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2238089aa0 con 0x7f22480cd530 2024-01-15T17:56:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.887+0000 7f223fe00640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f222006b2c0 0x7f222006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.888+0000 7f223fe00640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f222006b2c0 0x7f222006d780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2230036000 tx=0x7f223003d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:27.891+0000 7f223e000640 1 -- 172.21.15.26:0/3925678483 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2238054900 con 0x7f22480cd530 2024-01-15T17:56:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.082+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22480cbff0 con 0x7f22480cd530 2024-01-15T17:56:28.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.086+0000 7f223e000640 1 -- 172.21.15.26:0/3925678483 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16558 (secure 0 0 0) 0x7f223801e020 con 0x7f22480cd530 2024-01-15T17:56:28.089 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:28.089 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":34,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:56:26.647342+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:56:24.764583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-15T17:56:25.673498+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.088+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f222006b2c0 msgr2=0x7f222006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.088+0000 7f2247200640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f222006b2c0 0x7f222006d780 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2230036000 tx=0x7f223003d040 comp rx=0 tx=0).stop 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.088+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 msgr2=0x7f22481548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.088+0000 7f2247200640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 0x7f22481548d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f223800c7f0 tx=0x7f223800ccb0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.088+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 shutdown_connections 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.088+0000 7f2247200640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22480cd9e0 0x7f22481f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.088+0000 7f2247200640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f222006b2c0 0x7f222006d780 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.088+0000 7f2247200640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f224813cfa0 0x7f2248156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.088+0000 7f2247200640 1 --2- 172.21.15.26:0/3925678483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22480cd530 0x7f22481548d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.089+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 >> 172.21.15.26:0/3925678483 conn(0x7f224806a800 msgr2=0x7f22480afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.089+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 shutdown_connections 2024-01-15T17:56:28.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.089+0000 7f2247200640 1 -- 172.21.15.26:0/3925678483 wait complete. 2024-01-15T17:56:28.112 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T17:56:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.330+0000 7fde75800640 1 Processor -- start 2024-01-15T17:56:28.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.330+0000 7fde75800640 1 -- start start 2024-01-15T17:56:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.331+0000 7fde75800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fde7013cf10 0x7fde7013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.331+0000 7fde75800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde7013fae0 0x7fde700cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.332+0000 7fde75800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde700ce440 con 0x7fde700cdc20 2024-01-15T17:56:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.332+0000 7fde75800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde7013ff90 con 0x7fde7013fae0 2024-01-15T17:56:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.332+0000 7fde75800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde700ce720 con 0x7fde7013cf10 2024-01-15T17:56:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.332+0000 7fde6f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde7013fae0 0x7fde700cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.332+0000 7fde74e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fde700cdc20 0x7fde7015d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54900/0 (socket says 172.21.15.26:54900) 2024-01-15T17:56:28.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.332+0000 7fde74e00640 1 -- 172.21.15.26:0/1366314561 learned_addr learned my addr 172.21.15.26:0/1366314561 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.332+0000 7fde6fe00640 1 --2- 172.21.15.26:0/1366314561 >> v2:172.21.15.124:3300/0 conn(0x7fde7013cf10 0x7fde7013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.332+0000 7fde6f400640 1 -- 172.21.15.26:0/1366314561 >> v2:172.21.15.124:3300/0 conn(0x7fde7013cf10 msgr2=0x7fde7013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.333+0000 7fde6f400640 1 --2- 172.21.15.26:0/1366314561 >> v2:172.21.15.124:3300/0 conn(0x7fde7013cf10 0x7fde7013f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.333+0000 7fde6f400640 1 -- 172.21.15.26:0/1366314561 >> v1:172.21.15.26:6789/0 conn(0x7fde700cdc20 legacy=0x7fde7015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.333+0000 7fde6f400640 1 -- 172.21.15.26:0/1366314561 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde700cea00 con 0x7fde7013fae0 2024-01-15T17:56:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.333+0000 7fde6ea00640 1 -- 172.21.15.26:0/1366314561 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904885839 0 0) 0x7fde700ce440 con 0x7fde700cdc20 2024-01-15T17:56:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.334+0000 7fde6f400640 1 --2- 172.21.15.26:0/1366314561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013fae0 0x7fde700cd6e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fde7006e390 tx=0x7fde5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee1a97026f04fd2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.334+0000 7fde6ea00640 1 -- 172.21.15.26:0/1366314561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58031830 con 0x7fde7013fae0 2024-01-15T17:56:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.334+0000 7fde6ea00640 1 -- 172.21.15.26:0/1366314561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58031990 con 0x7fde7013fae0 2024-01-15T17:56:28.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.334+0000 7fde6ea00640 1 -- 172.21.15.26:0/1366314561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58031d20 con 0x7fde7013fae0 2024-01-15T17:56:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.335+0000 7fde75800640 1 -- 172.21.15.26:0/1366314561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013fae0 msgr2=0x7fde700cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.335+0000 7fde75800640 1 --2- 172.21.15.26:0/1366314561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013fae0 0x7fde700cd6e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fde7006e390 tx=0x7fde5802b8d0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.335+0000 7fde75800640 1 -- 172.21.15.26:0/1366314561 shutdown_connections 2024-01-15T17:56:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.335+0000 7fde75800640 1 --2- 172.21.15.26:0/1366314561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013fae0 0x7fde700cd6e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.335+0000 7fde75800640 1 --2- 172.21.15.26:0/1366314561 >> v2:172.21.15.124:3300/0 conn(0x7fde7013cf10 0x7fde7013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.335+0000 7fde75800640 1 -- 172.21.15.26:0/1366314561 >> 172.21.15.26:0/1366314561 conn(0x7fde7006a7d0 msgr2=0x7fde7006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.335+0000 7fde75800640 1 -- 172.21.15.26:0/1366314561 shutdown_connections 2024-01-15T17:56:28.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.335+0000 7fde75800640 1 -- 172.21.15.26:0/1366314561 wait complete. 2024-01-15T17:56:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.337+0000 7fde75800640 1 Processor -- start 2024-01-15T17:56:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.337+0000 7fde75800640 1 -- start start 2024-01-15T17:56:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde75800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde700cdc20 0x7fde701573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde75800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013cf10 0x7fde70135c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde75800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde70154a20 0x7fde70154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde75800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde70155f80 con 0x7fde7013cf10 2024-01-15T17:56:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde75800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde70157920 con 0x7fde700cdc20 2024-01-15T17:56:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde75800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde701559c0 con 0x7fde70154a20 2024-01-15T17:56:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013cf10 0x7fde70135c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013cf10 0x7fde70135c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41302/0 (socket says 172.21.15.26:41302) 2024-01-15T17:56:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6f400640 1 -- 172.21.15.26:0/1182272315 learned_addr learned my addr 172.21.15.26:0/1182272315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6fe00640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde700cdc20 0x7fde701573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6f400640 1 -- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde70154a20 msgr2=0x7fde70154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6f400640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde70154a20 0x7fde70154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6f400640 1 -- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde700cdc20 msgr2=0x7fde701573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6f400640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde700cdc20 0x7fde701573e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6f400640 1 -- 172.21.15.26:0/1182272315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde58009400 con 0x7fde7013cf10 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.339+0000 7fde6fe00640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde700cdc20 0x7fde701573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.340+0000 7fde6f400640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013cf10 0x7fde70135c50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fde5802be60 tx=0x7fde580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.340+0000 7fde67e00640 1 -- 172.21.15.26:0/1182272315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58004230 con 0x7fde7013cf10 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.340+0000 7fde67e00640 1 -- 172.21.15.26:0/1182272315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58004390 con 0x7fde7013cf10 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.340+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde70155ca0 con 0x7fde7013cf10 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.340+0000 7fde67e00640 1 -- 172.21.15.26:0/1182272315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5803a850 con 0x7fde7013cf10 2024-01-15T17:56:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.340+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7013d9a0 con 0x7fde7013cf10 2024-01-15T17:56:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.342+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7013d520 con 0x7fde7013cf10 2024-01-15T17:56:28.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.342+0000 7fde67e00640 1 -- 172.21.15.26:0/1182272315 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fde5803a9b0 con 0x7fde7013cf10 2024-01-15T17:56:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.343+0000 7fde67e00640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde4806b2c0 0x7fde4806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.343+0000 7fde67e00640 1 -- 172.21.15.26:0/1182272315 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde580abc20 con 0x7fde7013cf10 2024-01-15T17:56:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.344+0000 7fde6fe00640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde4806b2c0 0x7fde4806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.345+0000 7fde6fe00640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde4806b2c0 0x7fde4806d780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fde60006fd0 tx=0x7fde60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:28.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.348+0000 7fde67e00640 1 -- 172.21.15.26:0/1182272315 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde7013d520 con 0x7fde7013cf10 2024-01-15T17:56:28.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:28.585+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fde7013d520 con 0x7fde7013cf10 2024-01-15T17:56:29.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.237+0000 7fde67e00640 1 -- 172.21.15.26:0/1182272315 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fde7013d520 con 0x7fde7013cf10 2024-01-15T17:56:29.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde4806b2c0 msgr2=0x7fde4806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde4806b2c0 0x7fde4806d780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fde60006fd0 tx=0x7fde60008040 comp rx=0 tx=0).stop 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013cf10 msgr2=0x7fde70135c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013cf10 0x7fde70135c50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fde5802be60 tx=0x7fde580046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 shutdown_connections 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde70154a20 0x7fde70154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde4806b2c0 0x7fde4806d780 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde7013cf10 0x7fde70135c50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 --2- 172.21.15.26:0/1182272315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde700cdc20 0x7fde701573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 >> 172.21.15.26:0/1182272315 conn(0x7fde7006a7d0 msgr2=0x7fde70139130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.240+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 shutdown_connections 2024-01-15T17:56:29.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.241+0000 7fde75800640 1 -- 172.21.15.26:0/1182272315 wait complete. 2024-01-15T17:56:29.263 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T17:56:29.263 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T17:56:29.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.489+0000 7f7bb2e00640 1 Processor -- start 2024-01-15T17:56:29.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.489+0000 7f7bb2e00640 1 -- start start 2024-01-15T17:56:29.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.491+0000 7f7bb2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac13e360 0x7f7bac0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:29.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.491+0000 7f7bb2e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7bac15f9c0 0x7f7bac0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:29.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.491+0000 7f7bb2e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac15c5f0 con 0x7f7bac13df80 2024-01-15T17:56:29.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.491+0000 7f7bb2e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bac06c470 con 0x7f7bac15f9c0 2024-01-15T17:56:29.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.491+0000 7f7bb2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac06c750 con 0x7f7bac13e360 2024-01-15T17:56:29.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.491+0000 7f7babe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7bac13df80 0x7f7bac15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54912/0 (socket says 172.21.15.26:54912) 2024-01-15T17:56:29.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.491+0000 7f7babe00640 1 -- 172.21.15.26:0/1847532323 learned_addr learned my addr 172.21.15.26:0/1847532323 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:29.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.492+0000 7f7bb0e00640 1 --2- 172.21.15.26:0/1847532323 >> v2:172.21.15.124:3300/0 conn(0x7f7bac15f9c0 0x7f7bac0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:29.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.492+0000 7f7ba3200640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac13e360 0x7f7bac0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:29.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.492+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581724720 0 0) 0x7f7bac15c5f0 con 0x7f7bac13df80 2024-01-15T17:56:29.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.492+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b8c003520 con 0x7f7bac13df80 2024-01-15T17:56:29.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.492+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2965838801 0 0) 0x7f7b8c003520 con 0x7f7bac13df80 2024-01-15T17:56:29.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac13e360 msgr2=0x7f7bac0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:29.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac13e360 0x7f7bac0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 >> v2:172.21.15.124:3300/0 conn(0x7f7bac15f9c0 msgr2=0x7f7bac0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 --2- 172.21.15.26:0/1847532323 >> v2:172.21.15.124:3300/0 conn(0x7f7bac15f9c0 0x7f7bac0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac06ca30 con 0x7f7bac13df80 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f7b9c0019f0 con 0x7f7bac13df80 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 >> v1:172.21.15.26:6789/0 conn(0x7f7bac13df80 legacy=0x7f7bac15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b8c003970 0x7f7b8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7b8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac13e360 0x7f7b8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac06c470 con 0x7f7b8c0047c0 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac06c750 con 0x7f7b8c003970 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c00b980 con 0x7f7bac13e360 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7ba3200640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7b8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:29.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7babe00640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b8c003970 0x7f7b8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7ba3200640 1 -- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac13e360 msgr2=0x7f7b8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7ba3200640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac13e360 0x7f7b8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7ba3200640 1 -- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b8c003970 msgr2=0x7f7b8c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7ba3200640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b8c003970 0x7f7b8c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.493+0000 7f7ba3200640 1 -- 172.21.15.26:0/1847532323 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c003520 con 0x7f7b8c0047c0 2024-01-15T17:56:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.494+0000 7f7ba3200640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7b8c004c40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ac0 tx=0x7f7b8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4d40a5f46c0085 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:29.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.495+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9402c710 con 0x7f7b8c0047c0 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.495+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9402c870 con 0x7f7b8c0047c0 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.495+0000 7f7bab400640 1 -- 172.21.15.26:0/1847532323 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9402cc00 con 0x7f7b8c0047c0 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.495+0000 7f7bb2e00640 1 -- 172.21.15.26:0/1847532323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 msgr2=0x7f7b8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.495+0000 7f7bb2e00640 1 --2- 172.21.15.26:0/1847532323 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7b8c004c40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009ac0 tx=0x7f7b8c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.495+0000 7f7bb0e00640 1 -- 172.21.15.26:0/1847532323 reap_dead start 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.495+0000 7f7bb2e00640 1 -- 172.21.15.26:0/1847532323 shutdown_connections 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.495+0000 7f7bb2e00640 1 -- 172.21.15.26:0/1847532323 >> 172.21.15.26:0/1847532323 conn(0x7f7bac06a800 msgr2=0x7f7bac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.496+0000 7f7bb2e00640 1 -- 172.21.15.26:0/1847532323 shutdown_connections 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.496+0000 7f7bb2e00640 1 -- 172.21.15.26:0/1847532323 wait complete. 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.497+0000 7f7bb2e00640 1 Processor -- start 2024-01-15T17:56:29.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.497+0000 7f7bb2e00640 1 -- start start 2024-01-15T17:56:29.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.498+0000 7f7bb2e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7bac137700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:29.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.498+0000 7f7bb2e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7bac13a3e0 0x7f7bac0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:29.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.498+0000 7f7bb2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac1f6520 0x7f7bac1f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.498+0000 7f7bb2e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac13a860 con 0x7f7b8c0047c0 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.498+0000 7f7bb2e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac13ab30 con 0x7f7bac13a3e0 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.498+0000 7f7bb2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac13ae10 con 0x7f7bac1f6520 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7babe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7bac137700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7babe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7bac137700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41316/0 (socket says 172.21.15.26:41316) 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7babe00640 1 -- 172.21.15.26:0/3500613043 learned_addr learned my addr 172.21.15.26:0/3500613043 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7ba3200640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7bac13a3e0 0x7f7bac0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7babe00640 1 -- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac1f6520 msgr2=0x7f7bac1f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7babe00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac1f6520 0x7f7bac1f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7babe00640 1 -- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7bac13a3e0 msgr2=0x7f7bac0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:29.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7babe00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7bac13a3e0 0x7f7bac0afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:29.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7babe00640 1 -- 172.21.15.26:0/3500613043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac13b0f0 con 0x7f7b8c0047c0 2024-01-15T17:56:29.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.499+0000 7f7babe00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7bac137700 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c002110 tx=0x7f7b9c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:29.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.500+0000 7f7ba8c00640 1 -- 172.21.15.26:0/3500613043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c004b60 con 0x7f7b8c0047c0 2024-01-15T17:56:29.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.500+0000 7f7ba8c00640 1 -- 172.21.15.26:0/3500613043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c004cc0 con 0x7f7b8c0047c0 2024-01-15T17:56:29.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.500+0000 7f7ba8c00640 1 -- 172.21.15.26:0/3500613043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c007310 con 0x7f7b8c0047c0 2024-01-15T17:56:29.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.500+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac13b3d0 con 0x7f7b8c0047c0 2024-01-15T17:56:29.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.500+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac0b0440 con 0x7f7b8c0047c0 2024-01-15T17:56:29.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.501+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac0675d0 con 0x7f7b8c0047c0 2024-01-15T17:56:29.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.502+0000 7f7ba8c00640 1 -- 172.21.15.26:0/3500613043 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f7b9c007470 con 0x7f7b8c0047c0 2024-01-15T17:56:29.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.503+0000 7f7ba8c00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b8406b2f0 0x7f7b8406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:29.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.503+0000 7f7ba8c00640 1 -- 172.21.15.26:0/3500613043 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b9c0899f0 con 0x7f7b8c0047c0 2024-01-15T17:56:29.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.504+0000 7f7ba3200640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b8406b2f0 0x7f7b8406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:29.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.505+0000 7f7ba3200640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b8406b2f0 0x7f7b8406d7b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7b94036000 tx=0x7f7b94030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:29.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.507+0000 7f7ba8c00640 1 -- 172.21.15.26:0/3500613043 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b9c054850 con 0x7f7b8c0047c0 2024-01-15T17:56:29.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:29.737+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7bac0b0440 con 0x7f7b8c0047c0 2024-01-15T17:56:30.148 INFO:tasks.ceph.mds.b.smithi124.stderr:2024-01-15T17:56:30.144+0000 7f0d29600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T17:56:30.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.240+0000 7f7ba8c00640 1 -- 172.21.15.26:0/3500613043 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7b9c019020 con 0x7f7b8c0047c0 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b8406b2f0 msgr2=0x7f7b8406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b8406b2f0 0x7f7b8406d7b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7b94036000 tx=0x7f7b94030000 comp rx=0 tx=0).stop 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 msgr2=0x7f7bac137700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7bac137700 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c002110 tx=0x7f7b9c00b4a0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 shutdown_connections 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7bac1f6520 0x7f7bac1f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b8406b2f0 0x7f7b8406d7b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7bac13a3e0 0x7f7bac0afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 --2- 172.21.15.26:0/3500613043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b8c0047c0 0x7f7bac137700 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.243+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 >> 172.21.15.26:0/3500613043 conn(0x7f7bac06a800 msgr2=0x7f7bac15cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:30.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.244+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 shutdown_connections 2024-01-15T17:56:30.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.244+0000 7f7bb2e00640 1 -- 172.21.15.26:0/3500613043 wait complete. 2024-01-15T17:56:30.269 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T17:56:30.269 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:56:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.473+0000 7f0be5600640 1 Processor -- start 2024-01-15T17:56:30.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.473+0000 7f0be5600640 1 -- start start 2024-01-15T17:56:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.475+0000 7f0be5600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0be0143c40 0x7f0be0071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.475+0000 7f0be5600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0be00717d0 0x7f0be00cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.475+0000 7f0be5600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be00cca40 0x7f0be015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.475+0000 7f0be5600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be00723c0 con 0x7f0be0143c40 2024-01-15T17:56:30.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.475+0000 7f0be5600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be0070e50 con 0x7f0be00717d0 2024-01-15T17:56:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.475+0000 7f0be5600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be00cce40 con 0x7f0be00cca40 2024-01-15T17:56:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.475+0000 7f0bdfe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0be0143c40 0x7f0be0071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.475+0000 7f0bdfe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0be0143c40 0x7f0be0071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41330/0 (socket says 172.21.15.26:41330) 2024-01-15T17:56:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.475+0000 7f0bdfe00640 1 -- 172.21.15.26:0/3325700585 learned_addr learned my addr 172.21.15.26:0/3325700585 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.476+0000 7f0bdf400640 1 --2- 172.21.15.26:0/3325700585 >> v2:172.21.15.124:3300/0 conn(0x7f0be00717d0 0x7f0be00cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.476+0000 7f0bdfe00640 1 -- 172.21.15.26:0/3325700585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be00cca40 msgr2=0x7f0be015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.476+0000 7f0bdfe00640 1 --2- 172.21.15.26:0/3325700585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be00cca40 0x7f0be015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.476+0000 7f0bdfe00640 1 -- 172.21.15.26:0/3325700585 >> v2:172.21.15.124:3300/0 conn(0x7f0be00717d0 msgr2=0x7f0be00cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:30.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.476+0000 7f0bdfe00640 1 --2- 172.21.15.26:0/3325700585 >> v2:172.21.15.124:3300/0 conn(0x7f0be00717d0 0x7f0be00cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.476+0000 7f0bdfe00640 1 -- 172.21.15.26:0/3325700585 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be0144020 con 0x7f0be0143c40 2024-01-15T17:56:30.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.477+0000 7f0bdfe00640 1 --2- 172.21.15.26:0/3325700585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be0143c40 0x7f0be0071290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0be00b44a0 tx=0x7f0bd002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3d6aeeb77bb24e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:30.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.477+0000 7f0bdea00640 1 -- 172.21.15.26:0/3325700585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00317e0 con 0x7f0be0143c40 2024-01-15T17:56:30.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.477+0000 7f0bdea00640 1 -- 172.21.15.26:0/3325700585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0031940 con 0x7f0be0143c40 2024-01-15T17:56:30.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.477+0000 7f0bdea00640 1 -- 172.21.15.26:0/3325700585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0031cd0 con 0x7f0be0143c40 2024-01-15T17:56:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.478+0000 7f0be5600640 1 -- 172.21.15.26:0/3325700585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be0143c40 msgr2=0x7f0be0071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.478+0000 7f0be5600640 1 --2- 172.21.15.26:0/3325700585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be0143c40 0x7f0be0071290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0be00b44a0 tx=0x7f0bd002b880 comp rx=0 tx=0).stop 2024-01-15T17:56:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.478+0000 7f0be5600640 1 -- 172.21.15.26:0/3325700585 shutdown_connections 2024-01-15T17:56:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.478+0000 7f0be5600640 1 --2- 172.21.15.26:0/3325700585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be00cca40 0x7f0be015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.478+0000 7f0be5600640 1 --2- 172.21.15.26:0/3325700585 >> v2:172.21.15.124:3300/0 conn(0x7f0be00717d0 0x7f0be00cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.478+0000 7f0be5600640 1 --2- 172.21.15.26:0/3325700585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be0143c40 0x7f0be0071290 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.478+0000 7f0be5600640 1 -- 172.21.15.26:0/3325700585 >> 172.21.15.26:0/3325700585 conn(0x7f0be006a800 msgr2=0x7f0be006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.478+0000 7f0be5600640 1 -- 172.21.15.26:0/3325700585 shutdown_connections 2024-01-15T17:56:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.478+0000 7f0be5600640 1 -- 172.21.15.26:0/3325700585 wait complete. 2024-01-15T17:56:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.480+0000 7f0be5600640 1 Processor -- start 2024-01-15T17:56:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.480+0000 7f0be5600640 1 -- start start 2024-01-15T17:56:30.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0be5600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00717d0 0x7f0be0151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0be5600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0be00cca40 0x7f0be014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0be5600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be0143c40 0x7f0be00af730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0be5600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01f51e0 con 0x7f0be00717d0 2024-01-15T17:56:30.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0be5600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01f4330 con 0x7f0be00cca40 2024-01-15T17:56:30.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0be5600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be01f45d0 con 0x7f0be0143c40 2024-01-15T17:56:30.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00717d0 0x7f0be0151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00717d0 0x7f0be0151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41334/0 (socket says 172.21.15.26:41334) 2024-01-15T17:56:30.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0bdfe00640 1 -- 172.21.15.26:0/1688608324 learned_addr learned my addr 172.21.15.26:0/1688608324 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:30.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0be4c00640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be0143c40 0x7f0be00af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0bdf400640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0be00cca40 0x7f0be014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0bdfe00640 1 -- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be0143c40 msgr2=0x7f0be00af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0bdfe00640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be0143c40 0x7f0be00af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0bdfe00640 1 -- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0be00cca40 msgr2=0x7f0be014e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0bdfe00640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0be00cca40 0x7f0be014e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0bdfe00640 1 -- 172.21.15.26:0/1688608324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd00094d0 con 0x7f0be00717d0 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.481+0000 7f0be4c00640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be0143c40 0x7f0be00af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.482+0000 7f0bdfe00640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00717d0 0x7f0be0151620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0bd002b8c0 tx=0x7f0bd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.482+0000 7f0bd7e00640 1 -- 172.21.15.26:0/1688608324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0032880 con 0x7f0be00717d0 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.482+0000 7f0bd7e00640 1 -- 172.21.15.26:0/1688608324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd00329e0 con 0x7f0be00717d0 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.482+0000 7f0bd7e00640 1 -- 172.21.15.26:0/1688608324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0032d90 con 0x7f0be00717d0 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.482+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be01f48b0 con 0x7f0be00717d0 2024-01-15T17:56:30.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.482+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be01f4e70 con 0x7f0be00717d0 2024-01-15T17:56:30.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.483+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be0067450 con 0x7f0be00717d0 2024-01-15T17:56:30.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.484+0000 7f0bd7e00640 1 -- 172.21.15.26:0/1688608324 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0bd004e450 con 0x7f0be00717d0 2024-01-15T17:56:30.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.485+0000 7f0bd7e00640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bb806b2f0 0x7f0bb806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.486+0000 7f0bd7e00640 1 -- 172.21.15.26:0/1688608324 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bd00abd40 con 0x7f0be00717d0 2024-01-15T17:56:30.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.486+0000 7f0bdf400640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bb806b2f0 0x7f0bb806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.487+0000 7f0bdf400640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bb806b2f0 0x7f0bb806d7b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8006fd0 tx=0x7f0bc8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:30.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.491+0000 7f0bd7e00640 1 -- 172.21.15.26:0/1688608324 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bd0076bc0 con 0x7f0be00717d0 2024-01-15T17:56:30.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.721+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0be00ceb30 con 0x7f0be00717d0 2024-01-15T17:56:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.723+0000 7f0bd7e00640 1 -- 172.21.15.26:0/1688608324 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f0bd007a1e0 con 0x7f0be00717d0 2024-01-15T17:56:30.726 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:30.726 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1651915477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":1651915477},{"type":"v1","addr":"172.21.15.124:6837","nonce":1651915477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/506946946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":506946946},{"type":"v1","addr":"172.21.15.26:6833","nonce":506946946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3855677701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3855677701},{"type":"v1","addr":"172.21.15.26:6835","nonce":3855677701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:56:26.648174+0000","modified":"2024-01-15T17:56:29.738878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34619":{"gid":34619,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6836/2548050157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2548050157},{"type":"v1","addr":"172.21.15.124:6836","nonce":2548050157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-15T17:56:30.727 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 16 2024-01-15T17:56:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.725+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bb806b2f0 msgr2=0x7f0bb806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.725+0000 7f0be5600640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bb806b2f0 0x7f0bb806d7b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8006fd0 tx=0x7f0bc8008040 comp rx=0 tx=0).stop 2024-01-15T17:56:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.725+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00717d0 msgr2=0x7f0be0151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.725+0000 7f0be5600640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00717d0 0x7f0be0151620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0bd002b8c0 tx=0x7f0bd00046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.726+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 shutdown_connections 2024-01-15T17:56:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.726+0000 7f0be5600640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0be0143c40 0x7f0be00af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.726+0000 7f0be5600640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bb806b2f0 0x7f0bb806d7b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.726+0000 7f0be5600640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0be00cca40 0x7f0be014e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.726+0000 7f0be5600640 1 --2- 172.21.15.26:0/1688608324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0be00717d0 0x7f0be0151620 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.726+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 >> 172.21.15.26:0/1688608324 conn(0x7f0be006a800 msgr2=0x7f0be013ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.726+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 shutdown_connections 2024-01-15T17:56:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.726+0000 7f0be5600640 1 -- 172.21.15.26:0/1688608324 wait complete. 2024-01-15T17:56:30.748 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:56:30.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.958+0000 7f2a5b200640 1 Processor -- start 2024-01-15T17:56:30.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.959+0000 7f2a5b200640 1 -- start start 2024-01-15T17:56:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.960+0000 7f2a5b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a5415d610 0x7f2a540ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.960+0000 7f2a5b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 0x7f2a540b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.960+0000 7f2a5b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a540b3400 con 0x7f2a540ced60 2024-01-15T17:56:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.960+0000 7f2a5b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a540d1390 con 0x7f2a5415d610 2024-01-15T17:56:30.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.960+0000 7f2a5b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540b27f0 con 0x7f2a540d1ef0 2024-01-15T17:56:30.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.961+0000 7f2a53e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a5415d610 0x7f2a540ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.961+0000 7f2a53e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a5415d610 0x7f2a540ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41348/0 (socket says 172.21.15.26:41348) 2024-01-15T17:56:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.961+0000 7f2a53e00640 1 -- 172.21.15.26:0/2763758924 learned_addr learned my addr 172.21.15.26:0/2763758924 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.961+0000 7f2a59200640 1 --2- 172.21.15.26:0/2763758924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 0x7f2a540b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.961+0000 7f2a53e00640 1 -- 172.21.15.26:0/2763758924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 msgr2=0x7f2a540b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.961+0000 7f2a53e00640 1 --2- 172.21.15.26:0/2763758924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 0x7f2a540b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.961+0000 7f2a53e00640 1 -- 172.21.15.26:0/2763758924 >> v1:172.21.15.124:6789/0 conn(0x7f2a540ced60 legacy=0x7f2a540d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.961+0000 7f2a53e00640 1 -- 172.21.15.26:0/2763758924 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540b2ad0 con 0x7f2a5415d610 2024-01-15T17:56:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.963+0000 7f2a53e00640 1 --2- 172.21.15.26:0/2763758924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 0x7f2a540ce820 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009870 tx=0x7f2a4402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39f0a81ee5ca714 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.963+0000 7f2a52a00640 1 -- 172.21.15.26:0/2763758924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44030840 con 0x7f2a5415d610 2024-01-15T17:56:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.963+0000 7f2a52a00640 1 -- 172.21.15.26:0/2763758924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a440309a0 con 0x7f2a5415d610 2024-01-15T17:56:30.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.963+0000 7f2a52a00640 1 -- 172.21.15.26:0/2763758924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44030d30 con 0x7f2a5415d610 2024-01-15T17:56:30.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.964+0000 7f2a5b200640 1 -- 172.21.15.26:0/2763758924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 msgr2=0x7f2a540ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.964+0000 7f2a5b200640 1 --2- 172.21.15.26:0/2763758924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 0x7f2a540ce820 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009870 tx=0x7f2a4402b980 comp rx=0 tx=0).stop 2024-01-15T17:56:30.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.964+0000 7f2a5b200640 1 -- 172.21.15.26:0/2763758924 shutdown_connections 2024-01-15T17:56:30.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.964+0000 7f2a5b200640 1 --2- 172.21.15.26:0/2763758924 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 0x7f2a540b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.964+0000 7f2a5b200640 1 --2- 172.21.15.26:0/2763758924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 0x7f2a540ce820 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.964+0000 7f2a5b200640 1 -- 172.21.15.26:0/2763758924 >> 172.21.15.26:0/2763758924 conn(0x7f2a5409c540 msgr2=0x7f2a540d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:30.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.964+0000 7f2a5b200640 1 -- 172.21.15.26:0/2763758924 shutdown_connections 2024-01-15T17:56:30.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.964+0000 7f2a5b200640 1 -- 172.21.15.26:0/2763758924 wait complete. 2024-01-15T17:56:30.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.966+0000 7f2a5b200640 1 Processor -- start 2024-01-15T17:56:30.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.966+0000 7f2a5b200640 1 -- start start 2024-01-15T17:56:30.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.967+0000 7f2a5b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a540ced60 0x7f2a540ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.967+0000 7f2a5b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 0x7f2a540cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.967+0000 7f2a5b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 0x7f2a541512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.967+0000 7f2a5b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540b7450 con 0x7f2a5415d610 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.967+0000 7f2a5b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540cd470 con 0x7f2a540ced60 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.967+0000 7f2a5b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540b7630 con 0x7f2a540d1ef0 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a59200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 0x7f2a541512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a59200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 0x7f2a541512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41350/0 (socket says 172.21.15.26:41350) 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a59200640 1 -- 172.21.15.26:0/2038350881 learned_addr learned my addr 172.21.15.26:0/2038350881 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a53e00640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a540ced60 0x7f2a540ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a53400640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 0x7f2a540cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a59200640 1 -- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 msgr2=0x7f2a540cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a59200640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 0x7f2a540cb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a59200640 1 -- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a540ced60 msgr2=0x7f2a540ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:30.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a59200640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a540ced60 0x7f2a540ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:30.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a59200640 1 -- 172.21.15.26:0/2038350881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440094d0 con 0x7f2a5415d610 2024-01-15T17:56:30.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a53e00640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a540ced60 0x7f2a540ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:30.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.968+0000 7f2a53400640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 0x7f2a540cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:30.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.969+0000 7f2a59200640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 0x7f2a541512a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c007de0 tx=0x7f2a4c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:30.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.969+0000 7f2a4be00640 1 -- 172.21.15.26:0/2038350881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c014620 con 0x7f2a5415d610 2024-01-15T17:56:30.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.969+0000 7f2a4be00640 1 -- 172.21.15.26:0/2038350881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c00ee90 con 0x7f2a5415d610 2024-01-15T17:56:30.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.969+0000 7f2a4be00640 1 -- 172.21.15.26:0/2038350881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0149e0 con 0x7f2a5415d610 2024-01-15T17:56:30.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.969+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a540b7910 con 0x7f2a5415d610 2024-01-15T17:56:30.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.969+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5415b020 con 0x7f2a5415d610 2024-01-15T17:56:30.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.970+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a540b7e30 con 0x7f2a5415d610 2024-01-15T17:56:30.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.971+0000 7f2a4be00640 1 -- 172.21.15.26:0/2038350881 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f2a4c014b40 con 0x7f2a5415d610 2024-01-15T17:56:30.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.972+0000 7f2a4be00640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2c06b2c0 0x7f2a2c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:30.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.972+0000 7f2a4be00640 1 -- 172.21.15.26:0/2038350881 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a4c08a660 con 0x7f2a5415d610 2024-01-15T17:56:30.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.973+0000 7f2a53e00640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2c06b2c0 0x7f2a2c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:30.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.974+0000 7f2a53e00640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2c06b2c0 0x7f2a2c06d780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a44036000 tx=0x7f2a4403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:30.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:30.976+0000 7f2a4be00640 1 -- 172.21.15.26:0/2038350881 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a540b7e30 con 0x7f2a5415d610 2024-01-15T17:56:31.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.174+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a5415b1b0 con 0x7f2a5415d610 2024-01-15T17:56:31.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.176+0000 7f2a4be00640 1 -- 172.21.15.26:0/2038350881 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f2a4c058ec0 con 0x7f2a5415d610 2024-01-15T17:56:31.180 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:31.180 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":35,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:56:28.211106+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:56:24.764583+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-01-15T17:56:25.673498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:56:31.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2c06b2c0 msgr2=0x7f2a2c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2c06b2c0 0x7f2a2c06d780 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2a44036000 tx=0x7f2a4403b040 comp rx=0 tx=0).stop 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 msgr2=0x7f2a541512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 0x7f2a541512a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c007de0 tx=0x7f2a4c00e4b0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 shutdown_connections 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a540d1ef0 0x7f2a540cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2c06b2c0 0x7f2a2c06d780 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a5415d610 0x7f2a541512a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 --2- 172.21.15.26:0/2038350881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a540ced60 0x7f2a540ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 >> 172.21.15.26:0/2038350881 conn(0x7f2a5409c540 msgr2=0x7f2a540b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.179+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 shutdown_connections 2024-01-15T17:56:31.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.180+0000 7f2a5b200640 1 -- 172.21.15.26:0/2038350881 wait complete. 2024-01-15T17:56:31.202 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T17:56:31.202 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:56:31.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.420+0000 7fdbf7200640 1 Processor -- start 2024-01-15T17:56:31.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.420+0000 7fdbf7200640 1 -- start start 2024-01-15T17:56:31.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.422+0000 7fdbf7200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbf80cc500 0x7fdbf80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.422+0000 7fdbf7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf806d160 0x7fdbf806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.422+0000 7fdbf7200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf8070e50 con 0x7fdbf8143c40 2024-01-15T17:56:31.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.422+0000 7fdbf7200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf80723c0 con 0x7fdbf80cc500 2024-01-15T17:56:31.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.422+0000 7fdbf7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80cced0 con 0x7fdbf806d160 2024-01-15T17:56:31.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.422+0000 7fdbefe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbf80cc500 0x7fdbf80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.422+0000 7fdbefe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdbf80cc500 0x7fdbf80cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41358/0 (socket says 172.21.15.26:41358) 2024-01-15T17:56:31.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.422+0000 7fdbefe00640 1 -- 172.21.15.26:0/4125105426 learned_addr learned my addr 172.21.15.26:0/4125105426 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:31.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.423+0000 7fdbefe00640 1 -- 172.21.15.26:0/4125105426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf806d160 msgr2=0x7fdbf806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:31.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.423+0000 7fdbefe00640 1 --2- 172.21.15.26:0/4125105426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf806d160 0x7fdbf806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.423+0000 7fdbefe00640 1 -- 172.21.15.26:0/4125105426 >> v1:172.21.15.124:6789/0 conn(0x7fdbf8143c40 legacy=0x7fdbf8071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:31.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.423+0000 7fdbefe00640 1 -- 172.21.15.26:0/4125105426 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf8144020 con 0x7fdbf80cc500 2024-01-15T17:56:31.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.424+0000 7fdbefe00640 1 --2- 172.21.15.26:0/4125105426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 0x7fdbf80cc990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80b44a0 tx=0x7fdbe002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0504372a9d0388 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:31.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.424+0000 7fdbf5400640 1 -- 172.21.15.26:0/4125105426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe002e070 con 0x7fdbf80cc500 2024-01-15T17:56:31.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.424+0000 7fdbf5400640 1 -- 172.21.15.26:0/4125105426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe00315a0 con 0x7fdbf80cc500 2024-01-15T17:56:31.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.424+0000 7fdbf5400640 1 -- 172.21.15.26:0/4125105426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0031930 con 0x7fdbf80cc500 2024-01-15T17:56:31.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.425+0000 7fdbf7200640 1 -- 172.21.15.26:0/4125105426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 msgr2=0x7fdbf80cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.425+0000 7fdbf7200640 1 --2- 172.21.15.26:0/4125105426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 0x7fdbf80cc990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80b44a0 tx=0x7fdbe002b840 comp rx=0 tx=0).stop 2024-01-15T17:56:31.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.425+0000 7fdbf7200640 1 -- 172.21.15.26:0/4125105426 shutdown_connections 2024-01-15T17:56:31.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.425+0000 7fdbf7200640 1 --2- 172.21.15.26:0/4125105426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf806d160 0x7fdbf806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.425+0000 7fdbf7200640 1 --2- 172.21.15.26:0/4125105426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 0x7fdbf80cc990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.425+0000 7fdbf7200640 1 -- 172.21.15.26:0/4125105426 >> 172.21.15.26:0/4125105426 conn(0x7fdbf806a800 msgr2=0x7fdbf806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:31.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.425+0000 7fdbf7200640 1 -- 172.21.15.26:0/4125105426 shutdown_connections 2024-01-15T17:56:31.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.425+0000 7fdbf7200640 1 -- 172.21.15.26:0/4125105426 wait complete. 2024-01-15T17:56:31.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.427+0000 7fdbf7200640 1 Processor -- start 2024-01-15T17:56:31.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.428+0000 7fdbf7200640 1 -- start start 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbf7200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf806d160 0x7fdbf81380a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbf7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 0x7fdbf8142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbf7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf8143c40 0x7fdbf81427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbf7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf8134bb0 con 0x7fdbf80cc500 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbf7200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf8135a10 con 0x7fdbf806d160 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbf7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf8134e00 con 0x7fdbf8143c40 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbefe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 0x7fdbf8142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbefe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 0x7fdbf8142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41364/0 (socket says 172.21.15.26:41364) 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbefe00640 1 -- 172.21.15.26:0/603769295 learned_addr learned my addr 172.21.15.26:0/603769295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbf6800640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf8143c40 0x7fdbf81427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbf5e00640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf806d160 0x7fdbf81380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbefe00640 1 -- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf8143c40 msgr2=0x7fdbf81427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbefe00640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf8143c40 0x7fdbf81427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.429+0000 7fdbefe00640 1 -- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf806d160 msgr2=0x7fdbf81380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.430+0000 7fdbefe00640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf806d160 0x7fdbf81380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.430+0000 7fdbefe00640 1 -- 172.21.15.26:0/603769295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00094d0 con 0x7fdbf80cc500 2024-01-15T17:56:31.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.430+0000 7fdbf5e00640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf806d160 0x7fdbf81380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:31.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.430+0000 7fdbefe00640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 0x7fdbf8142280 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdbe002b880 tx=0x7fdbe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.430+0000 7fdbee000640 1 -- 172.21.15.26:0/603769295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0032900 con 0x7fdbf80cc500 2024-01-15T17:56:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.430+0000 7fdbee000640 1 -- 172.21.15.26:0/603769295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe0032a60 con 0x7fdbf80cc500 2024-01-15T17:56:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.430+0000 7fdbee000640 1 -- 172.21.15.26:0/603769295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0032e10 con 0x7fdbf80cc500 2024-01-15T17:56:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.430+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf81350e0 con 0x7fdbf80cc500 2024-01-15T17:56:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.430+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf81356a0 con 0x7fdbf80cc500 2024-01-15T17:56:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.432+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf8135bc0 con 0x7fdbf80cc500 2024-01-15T17:56:31.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.432+0000 7fdbee000640 1 -- 172.21.15.26:0/603769295 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fdbe004d3f0 con 0x7fdbf80cc500 2024-01-15T17:56:31.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.433+0000 7fdbee000640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006b2c0 0x7fdbd006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.433+0000 7fdbee000640 1 -- 172.21.15.26:0/603769295 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbe00ab850 con 0x7fdbf80cc500 2024-01-15T17:56:31.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.434+0000 7fdbf5e00640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006b2c0 0x7fdbd006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.435+0000 7fdbf5e00640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006b2c0 0x7fdbd006d780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdbf813ac90 tx=0x7fdbe8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:31.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.438+0000 7fdbee000640 1 -- 172.21.15.26:0/603769295 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdbf8135bc0 con 0x7fdbf80cc500 2024-01-15T17:56:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.627+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbf80c94a0 con 0x7fdbd006b2c0 2024-01-15T17:56:31.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.655+0000 7fdbee000640 1 -- 172.21.15.26:0/603769295 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280833 (secure 0 0 0) 0x7fdbf80c94a0 con 0x7fdbd006b2c0 2024-01-15T17:56:31.662 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:31.664 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:56:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006b2c0 msgr2=0x7fdbd006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006b2c0 0x7fdbd006d780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdbf813ac90 tx=0x7fdbe8008040 comp rx=0 tx=0).stop 2024-01-15T17:56:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 msgr2=0x7fdbf8142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 0x7fdbf8142280 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdbe002b880 tx=0x7fdbe00046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 shutdown_connections 2024-01-15T17:56:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf8143c40 0x7fdbf81427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006b2c0 0x7fdbd006d780 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbf80cc500 0x7fdbf8142280 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 --2- 172.21.15.26:0/603769295 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf806d160 0x7fdbf81380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.661+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 >> 172.21.15.26:0/603769295 conn(0x7fdbf806a800 msgr2=0x7fdbf806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.662+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 shutdown_connections 2024-01-15T17:56:31.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.662+0000 7fdbf7200640 1 -- 172.21.15.26:0/603769295 wait complete. 2024-01-15T17:56:31.684 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-01-15T17:56:29.938855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236848,"kb_used_data":4028,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737872,"statfs":{"total":773094113280,"available":772851580928,"internally_reserved":0,"allocated":4124672,"data_stored":2307629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.771783"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184384+0000","last_change":"2024-01-15T17:56:26.184384+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184384+0000","last_undegraded":"2024-01-15T17:56:26.184384+0000","last_fullsized":"2024-01-15T17:56:26.184384+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.195931+0000","last_change":"2024-01-15T17:56:26.195931+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.195931+0000","last_undegraded":"2024-01-15T17:56:26.195931+0000","last_fullsized":"2024-01-15T17:56:26.195931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185022+0000","last_change":"2024-01-15T17:56:26.185022+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185022+0000","last_undegraded":"2024-01-15T17:56:26.185022+0000","last_fullsized":"2024-01-15T17:56:26.185022+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189153+0000","last_change":"2024-01-15T17:56:26.189153+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189153+0000","last_undegraded":"2024-01-15T17:56:26.189153+0000","last_fullsized":"2024-01-15T17:56:26.189153+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188820+0000","last_change":"2024-01-15T17:56:26.188820+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188820+0000","last_undegraded":"2024-01-15T17:56:26.188820+0000","last_fullsized":"2024-01-15T17:56:26.188820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.192671+0000","last_change":"2024-01-15T17:56:26.192671+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.192671+0000","last_undegraded":"2024-01-15T17:56:26.192671+0000","last_fullsized":"2024-01-15T17:56:26.192671+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.197878+0000","last_change":"2024-01-15T17:56:26.197878+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.197878+0000","last_undegraded":"2024-01-15T17:56:26.197878+0000","last_fullsized":"2024-01-15T17:56:26.197878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.205932+0000","last_change":"2024-01-15T17:56:26.205932+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.205932+0000","last_undegraded":"2024-01-15T17:56:26.205932+0000","last_fullsized":"2024-01-15T17:56:26.205932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190196+0000","last_change":"2024-01-15T17:56:26.190196+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190196+0000","last_undegraded":"2024-01-15T17:56:26.190196+0000","last_fullsized":"2024-01-15T17:56:26.190196+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190603+0000","last_change":"2024-01-15T17:56:26.190603+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190603+0000","last_undegraded":"2024-01-15T17:56:26.190603+0000","last_fullsized":"2024-01-15T17:56:26.190603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202815+0000","last_change":"2024-01-15T17:56:26.202815+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202815+0000","last_undegraded":"2024-01-15T17:56:26.202815+0000","last_fullsized":"2024-01-15T17:56:26.202815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190684+0000","last_change":"2024-01-15T17:56:26.190684+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190684+0000","last_undegraded":"2024-01-15T17:56:26.190684+0000","last_fullsized":"2024-01-15T17:56:26.190684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188271+0000","last_change":"2024-01-15T17:56:26.188271+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188271+0000","last_undegraded":"2024-01-15T17:56:26.188271+0000","last_fullsized":"2024-01-15T17:56:26.188271+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190649+0000","last_change":"2024-01-15T17:56:26.190649+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190649+0000","last_undegraded":"2024-01-15T17:56:26.190649+0000","last_fullsized":"2024-01-15T17:56:26.190649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181879+0000","last_change":"2024-01-15T17:56:26.181879+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181879+0000","last_undegraded":"2024-01-15T17:56:26.181879+0000","last_fullsized":"2024-01-15T17:56:26.181879+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.197426+0000","last_change":"2024-01-15T17:56:26.197426+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.197426+0000","last_undegraded":"2024-01-15T17:56:26.197426+0000","last_fullsized":"2024-01-15T17:56:26.197426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202504+0000","last_change":"2024-01-15T17:56:26.202504+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202504+0000","last_undegraded":"2024-01-15T17:56:26.202504+0000","last_fullsized":"2024-01-15T17:56:26.202504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.192203+0000","last_change":"2024-01-15T17:56:26.192203+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.192203+0000","last_undegraded":"2024-01-15T17:56:26.192203+0000","last_fullsized":"2024-01-15T17:56:26.192203+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193807+0000","last_change":"2024-01-15T17:56:26.193807+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193807+0000","last_undegraded":"2024-01-15T17:56:26.193807+0000","last_fullsized":"2024-01-15T17:56:26.193807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184195+0000","last_change":"2024-01-15T17:56:26.184195+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184195+0000","last_undegraded":"2024-01-15T17:56:26.184195+0000","last_fullsized":"2024-01-15T17:56:26.184195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.204185+0000","last_change":"2024-01-15T17:56:26.204185+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.204185+0000","last_undegraded":"2024-01-15T17:56:26.204185+0000","last_fullsized":"2024-01-15T17:56:26.204185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189019+0000","last_change":"2024-01-15T17:56:26.189019+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189019+0000","last_undegraded":"2024-01-15T17:56:26.189019+0000","last_fullsized":"2024-01-15T17:56:26.189019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184645+0000","last_change":"2024-01-15T17:56:26.184645+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184645+0000","last_undegraded":"2024-01-15T17:56:26.184645+0000","last_fullsized":"2024-01-15T17:56:26.184645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.182486+0000","last_change":"2024-01-15T17:56:26.182486+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.182486+0000","last_undegraded":"2024-01-15T17:56:26.182486+0000","last_fullsized":"2024-01-15T17:56:26.182486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193649+0000","last_change":"2024-01-15T17:56:26.193649+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193649+0000","last_undegraded":"2024-01-15T17:56:26.193649+0000","last_fullsized":"2024-01-15T17:56:26.193649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191687+0000","last_change":"2024-01-15T17:56:26.191687+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191687+0000","last_undegraded":"2024-01-15T17:56:26.191687+0000","last_fullsized":"2024-01-15T17:56:26.191687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193341+0000","last_change":"2024-01-15T17:56:26.193341+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193341+0000","last_undegraded":"2024-01-15T17:56:26.193341+0000","last_fullsized":"2024-01-15T17:56:26.193341+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185835+0000","last_change":"2024-01-15T17:56:26.185835+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185835+0000","last_undegraded":"2024-01-15T17:56:26.185835+0000","last_fullsized":"2024-01-15T17:56:26.185835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.196524+0000","last_change":"2024-01-15T17:56:26.196524+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.196524+0000","last_undegraded":"2024-01-15T17:56:26.196524+0000","last_fullsized":"2024-01-15T17:56:26.196524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202508+0000","last_change":"2024-01-15T17:56:26.202508+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202508+0000","last_undegraded":"2024-01-15T17:56:26.202508+0000","last_fullsized":"2024-01-15T17:56:26.202508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183698+0000","last_change":"2024-01-15T17:56:26.183698+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183698+0000","last_undegraded":"2024-01-15T17:56:26.183698+0000","last_fullsized":"2024-01-15T17:56:26.183698+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.205444+0000","last_change":"2024-01-15T17:56:26.205444+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.205444+0000","last_undegraded":"2024-01-15T17:56:26.205444+0000","last_fullsized":"2024-01-15T17:56:26.205444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185930+0000","last_change":"2024-01-15T17:56:26.185930+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185930+0000","last_undegraded":"2024-01-15T17:56:26.185930+0000","last_fullsized":"2024-01-15T17:56:26.185930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207378+0000","last_change":"2024-01-15T17:56:26.202527+0000","last_active":"2024-01-15T17:56:26.207378+0000","last_peered":"2024-01-15T17:56:26.207378+0000","last_clean":"2024-01-15T17:56:26.207378+0000","last_became_active":"2024-01-15T17:56:26.202254+0000","last_became_peered":"2024-01-15T17:56:26.202254+0000","last_unstale":"2024-01-15T17:56:26.207378+0000","last_undegraded":"2024-01-15T17:56:26.207378+0000","last_fullsized":"2024-01-15T17:56:26.207378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:58:49.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183024+0000","last_change":"2024-01-15T17:56:26.183024+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183024+0000","last_undegraded":"2024-01-15T17:56:26.183024+0000","last_fullsized":"2024-01-15T17:56:26.183024+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.214222+0000","last_change":"2024-01-15T17:56:26.213479+0000","last_active":"2024-01-15T17:56:26.214222+0000","last_peered":"2024-01-15T17:56:26.214222+0000","last_clean":"2024-01-15T17:56:26.214222+0000","last_became_active":"2024-01-15T17:56:26.213165+0000","last_became_peered":"2024-01-15T17:56:26.213165+0000","last_unstale":"2024-01-15T17:56:26.214222+0000","last_undegraded":"2024-01-15T17:56:26.214222+0000","last_fullsized":"2024-01-15T17:56:26.214222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:43.670278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191397+0000","last_change":"2024-01-15T17:56:26.191397+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191397+0000","last_undegraded":"2024-01-15T17:56:26.191397+0000","last_fullsized":"2024-01-15T17:56:26.191397+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207053+0000","last_change":"2024-01-15T17:56:26.199957+0000","last_active":"2024-01-15T17:56:26.207053+0000","last_peered":"2024-01-15T17:56:26.207053+0000","last_clean":"2024-01-15T17:56:26.207053+0000","last_became_active":"2024-01-15T17:56:26.199632+0000","last_became_peered":"2024-01-15T17:56:26.199632+0000","last_unstale":"2024-01-15T17:56:26.207053+0000","last_undegraded":"2024-01-15T17:56:26.207053+0000","last_fullsized":"2024-01-15T17:56:26.207053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:01:25.558146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190219+0000","last_change":"2024-01-15T17:56:26.190219+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190219+0000","last_undegraded":"2024-01-15T17:56:26.190219+0000","last_fullsized":"2024-01-15T17:56:26.190219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.211037+0000","last_change":"2024-01-15T17:56:26.209481+0000","last_active":"2024-01-15T17:56:26.211037+0000","last_peered":"2024-01-15T17:56:26.211037+0000","last_clean":"2024-01-15T17:56:26.211037+0000","last_became_active":"2024-01-15T17:56:26.209251+0000","last_became_peered":"2024-01-15T17:56:26.209251+0000","last_unstale":"2024-01-15T17:56:26.211037+0000","last_undegraded":"2024-01-15T17:56:26.211037+0000","last_fullsized":"2024-01-15T17:56:26.211037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:13:34.770667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.196248+0000","last_change":"2024-01-15T17:56:26.196248+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.196248+0000","last_undegraded":"2024-01-15T17:56:26.196248+0000","last_fullsized":"2024-01-15T17:56:26.196248+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.211475+0000","last_change":"2024-01-15T17:56:26.209952+0000","last_active":"2024-01-15T17:56:26.211475+0000","last_peered":"2024-01-15T17:56:26.211475+0000","last_clean":"2024-01-15T17:56:26.211475+0000","last_became_active":"2024-01-15T17:56:26.209696+0000","last_became_peered":"2024-01-15T17:56:26.209696+0000","last_unstale":"2024-01-15T17:56:26.211475+0000","last_undegraded":"2024-01-15T17:56:26.211475+0000","last_fullsized":"2024-01-15T17:56:26.211475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:41.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181863+0000","last_change":"2024-01-15T17:56:26.181863+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181863+0000","last_undegraded":"2024-01-15T17:56:26.181863+0000","last_fullsized":"2024-01-15T17:56:26.181863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213022+0000","last_change":"2024-01-15T17:56:26.211397+0000","last_active":"2024-01-15T17:56:26.213022+0000","last_peered":"2024-01-15T17:56:26.213022+0000","last_clean":"2024-01-15T17:56:26.213022+0000","last_became_active":"2024-01-15T17:56:26.210076+0000","last_became_peered":"2024-01-15T17:56:26.210076+0000","last_unstale":"2024-01-15T17:56:26.213022+0000","last_undegraded":"2024-01-15T17:56:26.213022+0000","last_fullsized":"2024-01-15T17:56:26.213022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:31:26.304443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188872+0000","last_change":"2024-01-15T17:56:26.188872+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188872+0000","last_undegraded":"2024-01-15T17:56:26.188872+0000","last_fullsized":"2024-01-15T17:56:26.188872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207337+0000","last_change":"2024-01-15T17:56:26.203486+0000","last_active":"2024-01-15T17:56:26.207337+0000","last_peered":"2024-01-15T17:56:26.207337+0000","last_clean":"2024-01-15T17:56:26.207337+0000","last_became_active":"2024-01-15T17:56:26.203141+0000","last_became_peered":"2024-01-15T17:56:26.203141+0000","last_unstale":"2024-01-15T17:56:26.207337+0000","last_undegraded":"2024-01-15T17:56:26.207337+0000","last_fullsized":"2024-01-15T17:56:26.207337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:28:33.523943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183545+0000","last_change":"2024-01-15T17:56:26.183545+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183545+0000","last_undegraded":"2024-01-15T17:56:26.183545+0000","last_fullsized":"2024-01-15T17:56:26.183545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.203978+0000","last_change":"2024-01-15T17:56:26.200676+0000","last_active":"2024-01-15T17:56:26.203978+0000","last_peered":"2024-01-15T17:56:26.203978+0000","last_clean":"2024-01-15T17:56:26.203978+0000","last_became_active":"2024-01-15T17:56:26.199502+0000","last_became_peered":"2024-01-15T17:56:26.199502+0000","last_unstale":"2024-01-15T17:56:26.203978+0000","last_undegraded":"2024-01-15T17:56:26.203978+0000","last_fullsized":"2024-01-15T17:56:26.203978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:17.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185090+0000","last_change":"2024-01-15T17:56:26.185090+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185090+0000","last_undegraded":"2024-01-15T17:56:26.185090+0000","last_fullsized":"2024-01-15T17:56:26.185090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213865+0000","last_change":"2024-01-15T17:56:26.212810+0000","last_active":"2024-01-15T17:56:26.213865+0000","last_peered":"2024-01-15T17:56:26.213865+0000","last_clean":"2024-01-15T17:56:26.213865+0000","last_became_active":"2024-01-15T17:56:26.212482+0000","last_became_peered":"2024-01-15T17:56:26.212482+0000","last_unstale":"2024-01-15T17:56:26.213865+0000","last_undegraded":"2024-01-15T17:56:26.213865+0000","last_fullsized":"2024-01-15T17:56:26.213865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:26.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188955+0000","last_change":"2024-01-15T17:56:26.188955+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188955+0000","last_undegraded":"2024-01-15T17:56:26.188955+0000","last_fullsized":"2024-01-15T17:56:26.188955+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210403+0000","last_change":"2024-01-15T17:56:26.208053+0000","last_active":"2024-01-15T17:56:26.210403+0000","last_peered":"2024-01-15T17:56:26.210403+0000","last_clean":"2024-01-15T17:56:26.210403+0000","last_became_active":"2024-01-15T17:56:26.207220+0000","last_became_peered":"2024-01-15T17:56:26.207220+0000","last_unstale":"2024-01-15T17:56:26.210403+0000","last_undegraded":"2024-01-15T17:56:26.210403+0000","last_fullsized":"2024-01-15T17:56:26.210403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:31.045653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188235+0000","last_change":"2024-01-15T17:56:26.188235+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188235+0000","last_undegraded":"2024-01-15T17:56:26.188235+0000","last_fullsized":"2024-01-15T17:56:26.188235+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.215003+0000","last_change":"2024-01-15T17:56:26.213932+0000","last_active":"2024-01-15T17:56:26.215003+0000","last_peered":"2024-01-15T17:56:26.215003+0000","last_clean":"2024-01-15T17:56:26.215003+0000","last_became_active":"2024-01-15T17:56:26.213653+0000","last_became_peered":"2024-01-15T17:56:26.213653+0000","last_unstale":"2024-01-15T17:56:26.215003+0000","last_undegraded":"2024-01-15T17:56:26.215003+0000","last_fullsized":"2024-01-15T17:56:26.215003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:46.243898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188570+0000","last_change":"2024-01-15T17:56:26.188570+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188570+0000","last_undegraded":"2024-01-15T17:56:26.188570+0000","last_fullsized":"2024-01-15T17:56:26.188570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.209612+0000","last_change":"2024-01-15T17:56:26.203678+0000","last_active":"2024-01-15T17:56:26.209612+0000","last_peered":"2024-01-15T17:56:26.209612+0000","last_clean":"2024-01-15T17:56:26.209612+0000","last_became_active":"2024-01-15T17:56:26.203341+0000","last_became_peered":"2024-01-15T17:56:26.203341+0000","last_unstale":"2024-01-15T17:56:26.209612+0000","last_undegraded":"2024-01-15T17:56:26.209612+0000","last_fullsized":"2024-01-15T17:56:26.209612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:14:31.052513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184920+0000","last_change":"2024-01-15T17:56:26.184920+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184920+0000","last_undegraded":"2024-01-15T17:56:26.184920+0000","last_fullsized":"2024-01-15T17:56:26.184920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212414+0000","last_change":"2024-01-15T17:56:26.210551+0000","last_active":"2024-01-15T17:56:26.212414+0000","last_peered":"2024-01-15T17:56:26.212414+0000","last_clean":"2024-01-15T17:56:26.212414+0000","last_became_active":"2024-01-15T17:56:26.209399+0000","last_became_peered":"2024-01-15T17:56:26.209399+0000","last_unstale":"2024-01-15T17:56:26.212414+0000","last_undegraded":"2024-01-15T17:56:26.212414+0000","last_fullsized":"2024-01-15T17:56:26.212414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:12:05.549396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185817+0000","last_change":"2024-01-15T17:56:26.185817+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185817+0000","last_undegraded":"2024-01-15T17:56:26.185817+0000","last_fullsized":"2024-01-15T17:56:26.185817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208828+0000","last_change":"2024-01-15T17:56:26.201105+0000","last_active":"2024-01-15T17:56:26.208828+0000","last_peered":"2024-01-15T17:56:26.208828+0000","last_clean":"2024-01-15T17:56:26.208828+0000","last_became_active":"2024-01-15T17:56:26.200818+0000","last_became_peered":"2024-01-15T17:56:26.200818+0000","last_unstale":"2024-01-15T17:56:26.208828+0000","last_undegraded":"2024-01-15T17:56:26.208828+0000","last_fullsized":"2024-01-15T17:56:26.208828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:58:49.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.199505+0000","last_change":"2024-01-15T17:56:26.199505+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.199505+0000","last_undegraded":"2024-01-15T17:56:26.199505+0000","last_fullsized":"2024-01-15T17:56:26.199505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212669+0000","last_change":"2024-01-15T17:56:26.211092+0000","last_active":"2024-01-15T17:56:26.212669+0000","last_peered":"2024-01-15T17:56:26.212669+0000","last_clean":"2024-01-15T17:56:26.212669+0000","last_became_active":"2024-01-15T17:56:26.210804+0000","last_became_peered":"2024-01-15T17:56:26.210804+0000","last_unstale":"2024-01-15T17:56:26.212669+0000","last_undegraded":"2024-01-15T17:56:26.212669+0000","last_fullsized":"2024-01-15T17:56:26.212669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:02:16.155632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181524+0000","last_change":"2024-01-15T17:56:26.181524+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181524+0000","last_undegraded":"2024-01-15T17:56:26.181524+0000","last_fullsized":"2024-01-15T17:56:26.181524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207189+0000","last_change":"2024-01-15T17:56:26.201849+0000","last_active":"2024-01-15T17:56:26.207189+0000","last_peered":"2024-01-15T17:56:26.207189+0000","last_clean":"2024-01-15T17:56:26.207189+0000","last_became_active":"2024-01-15T17:56:26.201558+0000","last_became_peered":"2024-01-15T17:56:26.201558+0000","last_unstale":"2024-01-15T17:56:26.207189+0000","last_undegraded":"2024-01-15T17:56:26.207189+0000","last_fullsized":"2024-01-15T17:56:26.207189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:43.277391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190247+0000","last_change":"2024-01-15T17:56:26.190247+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190247+0000","last_undegraded":"2024-01-15T17:56:26.190247+0000","last_fullsized":"2024-01-15T17:56:26.190247+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213314+0000","last_change":"2024-01-15T17:56:26.211508+0000","last_active":"2024-01-15T17:56:26.213314+0000","last_peered":"2024-01-15T17:56:26.213314+0000","last_clean":"2024-01-15T17:56:26.213314+0000","last_became_active":"2024-01-15T17:56:26.207011+0000","last_became_peered":"2024-01-15T17:56:26.207011+0000","last_unstale":"2024-01-15T17:56:26.213314+0000","last_undegraded":"2024-01-15T17:56:26.213314+0000","last_fullsized":"2024-01-15T17:56:26.213314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:40.566415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191272+0000","last_change":"2024-01-15T17:56:26.191272+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191272+0000","last_undegraded":"2024-01-15T17:56:26.191272+0000","last_fullsized":"2024-01-15T17:56:26.191272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.203619+0000","last_change":"2024-01-15T17:56:26.200302+0000","last_active":"2024-01-15T17:56:26.203619+0000","last_peered":"2024-01-15T17:56:26.203619+0000","last_clean":"2024-01-15T17:56:26.203619+0000","last_became_active":"2024-01-15T17:56:26.200001+0000","last_became_peered":"2024-01-15T17:56:26.200001+0000","last_unstale":"2024-01-15T17:56:26.203619+0000","last_undegraded":"2024-01-15T17:56:26.203619+0000","last_fullsized":"2024-01-15T17:56:26.203619+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:06:34.072861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187437+0000","last_change":"2024-01-15T17:56:26.187437+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187437+0000","last_undegraded":"2024-01-15T17:56:26.187437+0000","last_fullsized":"2024-01-15T17:56:26.187437+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210901+0000","last_change":"2024-01-15T17:56:26.207943+0000","last_active":"2024-01-15T17:56:26.210901+0000","last_peered":"2024-01-15T17:56:26.210901+0000","last_clean":"2024-01-15T17:56:26.210901+0000","last_became_active":"2024-01-15T17:56:26.207604+0000","last_became_peered":"2024-01-15T17:56:26.207604+0000","last_unstale":"2024-01-15T17:56:26.210901+0000","last_undegraded":"2024-01-15T17:56:26.210901+0000","last_fullsized":"2024-01-15T17:56:26.210901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:42.102953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187383+0000","last_change":"2024-01-15T17:56:26.187383+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187383+0000","last_undegraded":"2024-01-15T17:56:26.187383+0000","last_fullsized":"2024-01-15T17:56:26.187383+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210215+0000","last_change":"2024-01-15T17:56:26.207505+0000","last_active":"2024-01-15T17:56:26.210215+0000","last_peered":"2024-01-15T17:56:26.210215+0000","last_clean":"2024-01-15T17:56:26.210215+0000","last_became_active":"2024-01-15T17:56:26.207210+0000","last_became_peered":"2024-01-15T17:56:26.207210+0000","last_unstale":"2024-01-15T17:56:26.210215+0000","last_undegraded":"2024-01-15T17:56:26.210215+0000","last_fullsized":"2024-01-15T17:56:26.210215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:13:34.770667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183722+0000","last_change":"2024-01-15T17:56:26.183722+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183722+0000","last_undegraded":"2024-01-15T17:56:26.183722+0000","last_fullsized":"2024-01-15T17:56:26.183722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.205077+0000","last_change":"2024-01-15T17:56:26.202274+0000","last_active":"2024-01-15T17:56:26.205077+0000","last_peered":"2024-01-15T17:56:26.205077+0000","last_clean":"2024-01-15T17:56:26.205077+0000","last_became_active":"2024-01-15T17:56:26.201924+0000","last_became_peered":"2024-01-15T17:56:26.201924+0000","last_unstale":"2024-01-15T17:56:26.205077+0000","last_undegraded":"2024-01-15T17:56:26.205077+0000","last_fullsized":"2024-01-15T17:56:26.205077+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:35.925710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183777+0000","last_change":"2024-01-15T17:56:26.183777+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183777+0000","last_undegraded":"2024-01-15T17:56:26.183777+0000","last_fullsized":"2024-01-15T17:56:26.183777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.205313+0000","last_change":"2024-01-15T17:56:26.200301+0000","last_active":"2024-01-15T17:56:26.205313+0000","last_peered":"2024-01-15T17:56:26.205313+0000","last_clean":"2024-01-15T17:56:26.205313+0000","last_became_active":"2024-01-15T17:56:26.200003+0000","last_became_peered":"2024-01-15T17:56:26.200003+0000","last_unstale":"2024-01-15T17:56:26.205313+0000","last_undegraded":"2024-01-15T17:56:26.205313+0000","last_fullsized":"2024-01-15T17:56:26.205313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:02:29.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.186611+0000","last_change":"2024-01-15T17:56:26.186611+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.186611+0000","last_undegraded":"2024-01-15T17:56:26.186611+0000","last_fullsized":"2024-01-15T17:56:26.186611+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.215099+0000","last_change":"2024-01-15T17:56:26.214358+0000","last_active":"2024-01-15T17:56:26.215099+0000","last_peered":"2024-01-15T17:56:26.215099+0000","last_clean":"2024-01-15T17:56:26.215099+0000","last_became_active":"2024-01-15T17:56:26.213452+0000","last_became_peered":"2024-01-15T17:56:26.213452+0000","last_unstale":"2024-01-15T17:56:26.215099+0000","last_undegraded":"2024-01-15T17:56:26.215099+0000","last_fullsized":"2024-01-15T17:56:26.215099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:05:49.120775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.198507+0000","last_change":"2024-01-15T17:56:26.198507+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.198507+0000","last_undegraded":"2024-01-15T17:56:26.198507+0000","last_fullsized":"2024-01-15T17:56:26.198507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212927+0000","last_change":"2024-01-15T17:56:26.212040+0000","last_active":"2024-01-15T17:56:26.212927+0000","last_peered":"2024-01-15T17:56:26.212927+0000","last_clean":"2024-01-15T17:56:26.212927+0000","last_became_active":"2024-01-15T17:56:26.211151+0000","last_became_peered":"2024-01-15T17:56:26.211151+0000","last_unstale":"2024-01-15T17:56:26.212927+0000","last_undegraded":"2024-01-15T17:56:26.212927+0000","last_fullsized":"2024-01-15T17:56:26.212927+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T17:58:44.566548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185596+0000","last_change":"2024-01-15T17:56:26.185596+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185596+0000","last_undegraded":"2024-01-15T17:56:26.185596+0000","last_fullsized":"2024-01-15T17:56:26.185596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210500+0000","last_change":"2024-01-15T17:56:26.201506+0000","last_active":"2024-01-15T17:56:26.210500+0000","last_peered":"2024-01-15T17:56:26.210500+0000","last_clean":"2024-01-15T17:56:26.210500+0000","last_became_active":"2024-01-15T17:56:26.201268+0000","last_became_peered":"2024-01-15T17:56:26.201268+0000","last_unstale":"2024-01-15T17:56:26.210500+0000","last_undegraded":"2024-01-15T17:56:26.210500+0000","last_fullsized":"2024-01-15T17:56:26.210500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:43.670278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190043+0000","last_change":"2024-01-15T17:56:26.190043+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190043+0000","last_undegraded":"2024-01-15T17:56:26.190043+0000","last_fullsized":"2024-01-15T17:56:26.190043+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.202533+0000","last_change":"2024-01-15T17:56:26.200072+0000","last_active":"2024-01-15T17:56:26.202533+0000","last_peered":"2024-01-15T17:56:26.202533+0000","last_clean":"2024-01-15T17:56:26.202533+0000","last_became_active":"2024-01-15T17:56:26.199764+0000","last_became_peered":"2024-01-15T17:56:26.199764+0000","last_unstale":"2024-01-15T17:56:26.202533+0000","last_undegraded":"2024-01-15T17:56:26.202533+0000","last_fullsized":"2024-01-15T17:56:26.202533+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189595+0000","last_change":"2024-01-15T17:56:26.189595+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189595+0000","last_undegraded":"2024-01-15T17:56:26.189595+0000","last_fullsized":"2024-01-15T17:56:26.189595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208497+0000","last_change":"2024-01-15T17:56:26.200854+0000","last_active":"2024-01-15T17:56:26.208497+0000","last_peered":"2024-01-15T17:56:26.208497+0000","last_clean":"2024-01-15T17:56:26.208497+0000","last_became_active":"2024-01-15T17:56:26.200317+0000","last_became_peered":"2024-01-15T17:56:26.200317+0000","last_unstale":"2024-01-15T17:56:26.208497+0000","last_undegraded":"2024-01-15T17:56:26.208497+0000","last_fullsized":"2024-01-15T17:56:26.208497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181422+0000","last_change":"2024-01-15T17:56:26.181422+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181422+0000","last_undegraded":"2024-01-15T17:56:26.181422+0000","last_fullsized":"2024-01-15T17:56:26.181422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208623+0000","last_change":"2024-01-15T17:56:26.205649+0000","last_active":"2024-01-15T17:56:26.208623+0000","last_peered":"2024-01-15T17:56:26.208623+0000","last_clean":"2024-01-15T17:56:26.208623+0000","last_became_active":"2024-01-15T17:56:26.205337+0000","last_became_peered":"2024-01-15T17:56:26.205337+0000","last_unstale":"2024-01-15T17:56:26.208623+0000","last_undegraded":"2024-01-15T17:56:26.208623+0000","last_fullsized":"2024-01-15T17:56:26.208623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:40.566415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187391+0000","last_change":"2024-01-15T17:56:26.187391+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187391+0000","last_undegraded":"2024-01-15T17:56:26.187391+0000","last_fullsized":"2024-01-15T17:56:26.187391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208848+0000","last_change":"2024-01-15T17:56:26.201883+0000","last_active":"2024-01-15T17:56:26.208848+0000","last_peered":"2024-01-15T17:56:26.208848+0000","last_clean":"2024-01-15T17:56:26.208848+0000","last_became_active":"2024-01-15T17:56:26.201275+0000","last_became_peered":"2024-01-15T17:56:26.201275+0000","last_unstale":"2024-01-15T17:56:26.208848+0000","last_undegraded":"2024-01-15T17:56:26.208848+0000","last_fullsized":"2024-01-15T17:56:26.208848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:02:29.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.182677+0000","last_change":"2024-01-15T17:56:26.182677+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.182677+0000","last_undegraded":"2024-01-15T17:56:26.182677+0000","last_fullsized":"2024-01-15T17:56:26.182677+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207896+0000","last_change":"2024-01-15T17:56:26.205102+0000","last_active":"2024-01-15T17:56:26.207896+0000","last_peered":"2024-01-15T17:56:26.207896+0000","last_clean":"2024-01-15T17:56:26.207896+0000","last_became_active":"2024-01-15T17:56:26.204747+0000","last_became_peered":"2024-01-15T17:56:26.204747+0000","last_unstale":"2024-01-15T17:56:26.207896+0000","last_undegraded":"2024-01-15T17:56:26.207896+0000","last_fullsized":"2024-01-15T17:56:26.207896+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:48.459125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.195525+0000","last_change":"2024-01-15T17:56:26.195525+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.195525+0000","last_undegraded":"2024-01-15T17:56:26.195525+0000","last_fullsized":"2024-01-15T17:56:26.195525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212667+0000","last_change":"2024-01-15T17:56:26.211169+0000","last_active":"2024-01-15T17:56:26.212667+0000","last_peered":"2024-01-15T17:56:26.212667+0000","last_clean":"2024-01-15T17:56:26.212667+0000","last_became_active":"2024-01-15T17:56:26.210842+0000","last_became_peered":"2024-01-15T17:56:26.210842+0000","last_unstale":"2024-01-15T17:56:26.212667+0000","last_undegraded":"2024-01-15T17:56:26.212667+0000","last_fullsized":"2024-01-15T17:56:26.212667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:25.400140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.201814+0000","last_change":"2024-01-15T17:56:26.201814+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.201814+0000","last_undegraded":"2024-01-15T17:56:26.201814+0000","last_fullsized":"2024-01-15T17:56:26.201814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210656+0000","last_change":"2024-01-15T17:56:26.208654+0000","last_active":"2024-01-15T17:56:26.210656+0000","last_peered":"2024-01-15T17:56:26.210656+0000","last_clean":"2024-01-15T17:56:26.210656+0000","last_became_active":"2024-01-15T17:56:26.208305+0000","last_became_peered":"2024-01-15T17:56:26.208305+0000","last_unstale":"2024-01-15T17:56:26.210656+0000","last_undegraded":"2024-01-15T17:56:26.210656+0000","last_fullsized":"2024-01-15T17:56:26.210656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":172,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-01-15T17:56:25.185666+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:56:25.185666+0000","last_peered":"2024-01-15T17:56:25.185666+0000","last_clean":"2024-01-15T17:56:25.185666+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:56:25.185666+0000","last_undegraded":"2024-01-15T17:56:25.185666+0000","last_fullsized":"2024-01-15T17:56:25.185666+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":962560,"data_stored":732013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.643}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.373},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.534}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.28},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.081}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.181},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.052,"5min":1.052,"15min":1.052},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.2330000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.444}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.49},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.1339999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.714,"15min":1.714},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.325},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.611}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":0.876}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.155},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.2569999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.3280000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.417},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1140000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.605},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.4810000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.758,"5min":1.758,"15min":1.758},"min":{"1min":1.202,"5min":1.202,"15min":1.202},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.5209999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.212},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"min":{"1min":1.286,"5min":1.286,"15min":1.286},"max":{"1min":3.141,"5min":3.141,"15min":3.141},"last":1.456}]}]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.3840000000000003,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.22},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":5.2290000000000001,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.3440000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.625}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.835,"5min":2.835,"15min":2.835},"last":1.546},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.413}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.264}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.419,"15min":1.419},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.5720000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.083,"5min":1.083,"15min":1.083},"max":{"1min":5.218,"5min":5.218,"15min":5.218},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.512}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":1.2,"5min":1.2,"15min":1.2},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.4430000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.996},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0740000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.597},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.127}]},{"osd":2,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.339},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.383}]},{"osd":3,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.4410000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.536},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"min":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.794}]},{"osd":6,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":1.706,"15min":1.706},"min":{"1min":1.117,"5min":1.117,"15min":1.117},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.742},{"interface":"front","average":{"1min":1.845,"5min":1.845,"15min":1.845},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.8520000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"min":{"1min":1.385,"5min":1.385,"15min":1.385},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.899},{"interface":"front","average":{"1min":1.929,"5min":1.929,"15min":1.929},"min":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":1.929}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.2989999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.4410000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.238}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.353}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.643},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":1.167}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.673,"15min":1.673},"min":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.5820000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.698,"15min":1.698},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"min":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.538}]}]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.208}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.135}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":4.008,"5min":4.008,"15min":4.008},"last":1.278},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.0289999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.171}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.6070000000000002,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.389},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.0529999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.028},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.61199999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1990000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.23,"15min":2.23},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":22.106999999999999,"5min":22.106999999999999,"15min":22.106999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.0740000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":9.8510000000000009,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.272},{"interface":"front","average":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":14.359999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.113}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":13.228999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":4.9900000000000002,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":1.4039999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":14.272,"5min":14.272,"15min":14.272},"last":1.357},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.1659999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.474}]}]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":962560,"data_stored":732013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.032}]},{"osd":2,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.496},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.93300000000000005}]},{"osd":3,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.597},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.542}]},{"osd":4,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.401},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.6619999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.6419999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.9990000000000001,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":1.792},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.274}]},{"osd":7,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.8100000000000001,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.857},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.74}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:56:31.689 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:56:31.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.953+0000 7f9566800640 1 Processor -- start 2024-01-15T17:56:31.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.953+0000 7f9566800640 1 -- start start 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f9566800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f956015d940 0x7f956006b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f9566800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95600c7a90 0x7f956015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f9566800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9560160eb0 0x7f9560161340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f9566800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956006d1d0 con 0x7f9560160eb0 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f9566800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95600c7290 con 0x7f956015d940 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f9566800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95600c7ef0 con 0x7f95600c7a90 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f955fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9560160eb0 0x7f9560161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f955fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9560160eb0 0x7f9560161340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47740/0 (socket says 172.21.15.26:47740) 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f955fe00640 1 -- 172.21.15.26:0/1406442306 learned_addr learned my addr 172.21.15.26:0/1406442306 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f955f400640 1 --2- 172.21.15.26:0/1406442306 >> v2:172.21.15.124:3300/0 conn(0x7f956015d940 0x7f956006b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.955+0000 7f9557e00640 1 --2- 172.21.15.26:0/1406442306 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95600c7a90 0x7f956015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.956+0000 7f955fe00640 1 -- 172.21.15.26:0/1406442306 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95600c7a90 msgr2=0x7f956015dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.956+0000 7f955fe00640 1 --2- 172.21.15.26:0/1406442306 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95600c7a90 0x7f956015dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.956+0000 7f955fe00640 1 -- 172.21.15.26:0/1406442306 >> v2:172.21.15.124:3300/0 conn(0x7f956015d940 msgr2=0x7f956006b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:31.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.956+0000 7f955fe00640 1 --2- 172.21.15.26:0/1406442306 >> v2:172.21.15.124:3300/0 conn(0x7f956015d940 0x7f956006b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.956+0000 7f955fe00640 1 -- 172.21.15.26:0/1406442306 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956006bb80 con 0x7f9560160eb0 2024-01-15T17:56:31.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.957+0000 7f955fe00640 1 --2- 172.21.15.26:0/1406442306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9560160eb0 0x7f9560161340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f95600ceb80 tx=0x7f955802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e90d4b7b2b96f42 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:31.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.957+0000 7f955ea00640 1 -- 172.21.15.26:0/1406442306 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955802ca70 con 0x7f9560160eb0 2024-01-15T17:56:31.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.957+0000 7f955ea00640 1 -- 172.21.15.26:0/1406442306 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955802cbd0 con 0x7f9560160eb0 2024-01-15T17:56:31.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.957+0000 7f955ea00640 1 -- 172.21.15.26:0/1406442306 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95580343f0 con 0x7f9560160eb0 2024-01-15T17:56:31.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.958+0000 7f9566800640 1 -- 172.21.15.26:0/1406442306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9560160eb0 msgr2=0x7f9560161340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.958+0000 7f9566800640 1 --2- 172.21.15.26:0/1406442306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9560160eb0 0x7f9560161340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f95600ceb80 tx=0x7f955802bed0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.958+0000 7f9566800640 1 -- 172.21.15.26:0/1406442306 shutdown_connections 2024-01-15T17:56:31.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.958+0000 7f9566800640 1 --2- 172.21.15.26:0/1406442306 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9560160eb0 0x7f9560161340 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.958+0000 7f9566800640 1 --2- 172.21.15.26:0/1406442306 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95600c7a90 0x7f956015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.958+0000 7f9566800640 1 --2- 172.21.15.26:0/1406442306 >> v2:172.21.15.124:3300/0 conn(0x7f956015d940 0x7f956006b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.958+0000 7f9566800640 1 -- 172.21.15.26:0/1406442306 >> 172.21.15.26:0/1406442306 conn(0x7f95600b2ce0 msgr2=0x7f95600b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:31.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.958+0000 7f9566800640 1 -- 172.21.15.26:0/1406442306 shutdown_connections 2024-01-15T17:56:31.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.958+0000 7f9566800640 1 -- 172.21.15.26:0/1406442306 wait complete. 2024-01-15T17:56:31.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.960+0000 7f9566800640 1 Processor -- start 2024-01-15T17:56:31.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.960+0000 7f9566800640 1 -- start start 2024-01-15T17:56:31.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f9566800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95600c7a90 0x7f95600ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f9566800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f956015d940 0x7f95601f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f9566800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9560160eb0 0x7f95600cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f9566800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600b4790 con 0x7f95600c7a90 2024-01-15T17:56:31.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f9566800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600ad7b0 con 0x7f956015d940 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f9566800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95601f7090 con 0x7f9560160eb0 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f955f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95600c7a90 0x7f95600ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f955f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95600c7a90 0x7f95600ad270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47744/0 (socket says 172.21.15.26:47744) 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f955f400640 1 -- 172.21.15.26:0/3140044087 learned_addr learned my addr 172.21.15.26:0/3140044087 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.961+0000 7f955fe00640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9560160eb0 0x7f95600cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f9557e00640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f956015d940 0x7f95601f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f955f400640 1 -- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9560160eb0 msgr2=0x7f95600cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f955f400640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9560160eb0 0x7f95600cfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f955f400640 1 -- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f956015d940 msgr2=0x7f95601f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f955f400640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f956015d940 0x7f95601f6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f955f400640 1 -- 172.21.15.26:0/3140044087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9558009b30 con 0x7f95600c7a90 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f955f400640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95600c7a90 0x7f95600ad270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f955000d800 tx=0x7f955000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f9557400640 1 -- 172.21.15.26:0/3140044087 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955000b6c0 con 0x7f95600c7a90 2024-01-15T17:56:31.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f9557400640 1 -- 172.21.15.26:0/3140044087 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955000b820 con 0x7f95600c7a90 2024-01-15T17:56:31.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95600ad9a0 con 0x7f95600c7a90 2024-01-15T17:56:31.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f9557400640 1 -- 172.21.15.26:0/3140044087 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955000bbb0 con 0x7f95600c7a90 2024-01-15T17:56:31.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.962+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95600b40d0 con 0x7f95600c7a90 2024-01-15T17:56:31.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.964+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95600b4a00 con 0x7f95600c7a90 2024-01-15T17:56:31.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.965+0000 7f9557400640 1 -- 172.21.15.26:0/3140044087 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f955000bd10 con 0x7f95600c7a90 2024-01-15T17:56:31.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.966+0000 7f9557400640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f953806b2c0 0x7f953806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:31.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.966+0000 7f9557400640 1 -- 172.21.15.26:0/3140044087 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f955008a140 con 0x7f95600c7a90 2024-01-15T17:56:31.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.966+0000 7f9557e00640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f953806b2c0 0x7f953806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:31.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.967+0000 7f9557e00640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f953806b2c0 0x7f953806d780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9548006fd0 tx=0x7f9548008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:31.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:31.970+0000 7f9557400640 1 -- 172.21.15.26:0/3140044087 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9550054fa0 con 0x7f95600c7a90 2024-01-15T17:56:32.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.164+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95600c6f20 con 0x7f953806b2c0 2024-01-15T17:56:32.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.185+0000 7f9557400640 1 -- 172.21.15.26:0/3140044087 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280833 (secure 0 0 0) 0x7f95480133f0 con 0x7f953806b2c0 2024-01-15T17:56:32.190 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:32.194 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:56:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.192+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f953806b2c0 msgr2=0x7f953806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.192+0000 7f9566800640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f953806b2c0 0x7f953806d780 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9548006fd0 tx=0x7f9548008040 comp rx=0 tx=0).stop 2024-01-15T17:56:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.192+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95600c7a90 msgr2=0x7f95600ad270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.192+0000 7f9566800640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95600c7a90 0x7f95600ad270 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f955000d800 tx=0x7f955000dcc0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.193+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 shutdown_connections 2024-01-15T17:56:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.193+0000 7f9566800640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9560160eb0 0x7f95600cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.193+0000 7f9566800640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f953806b2c0 0x7f953806d780 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.193+0000 7f9566800640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f956015d940 0x7f95601f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.193+0000 7f9566800640 1 --2- 172.21.15.26:0/3140044087 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95600c7a90 0x7f95600ad270 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.193+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 >> 172.21.15.26:0/3140044087 conn(0x7f95600b2ce0 msgr2=0x7f95600b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.193+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 shutdown_connections 2024-01-15T17:56:32.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.193+0000 7f9566800640 1 -- 172.21.15.26:0/3140044087 wait complete. 2024-01-15T17:56:32.218 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-01-15T17:56:31.939478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236848,"kb_used_data":4028,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737872,"statfs":{"total":773094113280,"available":772851580928,"internally_reserved":0,"allocated":4124672,"data_stored":2307629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001737"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184384+0000","last_change":"2024-01-15T17:56:26.184384+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184384+0000","last_undegraded":"2024-01-15T17:56:26.184384+0000","last_fullsized":"2024-01-15T17:56:26.184384+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.195931+0000","last_change":"2024-01-15T17:56:26.195931+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.195931+0000","last_undegraded":"2024-01-15T17:56:26.195931+0000","last_fullsized":"2024-01-15T17:56:26.195931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185022+0000","last_change":"2024-01-15T17:56:26.185022+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185022+0000","last_undegraded":"2024-01-15T17:56:26.185022+0000","last_fullsized":"2024-01-15T17:56:26.185022+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189153+0000","last_change":"2024-01-15T17:56:26.189153+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189153+0000","last_undegraded":"2024-01-15T17:56:26.189153+0000","last_fullsized":"2024-01-15T17:56:26.189153+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188820+0000","last_change":"2024-01-15T17:56:26.188820+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188820+0000","last_undegraded":"2024-01-15T17:56:26.188820+0000","last_fullsized":"2024-01-15T17:56:26.188820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.192671+0000","last_change":"2024-01-15T17:56:26.192671+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.192671+0000","last_undegraded":"2024-01-15T17:56:26.192671+0000","last_fullsized":"2024-01-15T17:56:26.192671+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.197878+0000","last_change":"2024-01-15T17:56:26.197878+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.197878+0000","last_undegraded":"2024-01-15T17:56:26.197878+0000","last_fullsized":"2024-01-15T17:56:26.197878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.205932+0000","last_change":"2024-01-15T17:56:26.205932+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.205932+0000","last_undegraded":"2024-01-15T17:56:26.205932+0000","last_fullsized":"2024-01-15T17:56:26.205932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190196+0000","last_change":"2024-01-15T17:56:26.190196+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190196+0000","last_undegraded":"2024-01-15T17:56:26.190196+0000","last_fullsized":"2024-01-15T17:56:26.190196+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190603+0000","last_change":"2024-01-15T17:56:26.190603+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190603+0000","last_undegraded":"2024-01-15T17:56:26.190603+0000","last_fullsized":"2024-01-15T17:56:26.190603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202815+0000","last_change":"2024-01-15T17:56:26.202815+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202815+0000","last_undegraded":"2024-01-15T17:56:26.202815+0000","last_fullsized":"2024-01-15T17:56:26.202815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190684+0000","last_change":"2024-01-15T17:56:26.190684+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190684+0000","last_undegraded":"2024-01-15T17:56:26.190684+0000","last_fullsized":"2024-01-15T17:56:26.190684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188271+0000","last_change":"2024-01-15T17:56:26.188271+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188271+0000","last_undegraded":"2024-01-15T17:56:26.188271+0000","last_fullsized":"2024-01-15T17:56:26.188271+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190649+0000","last_change":"2024-01-15T17:56:26.190649+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190649+0000","last_undegraded":"2024-01-15T17:56:26.190649+0000","last_fullsized":"2024-01-15T17:56:26.190649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181879+0000","last_change":"2024-01-15T17:56:26.181879+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181879+0000","last_undegraded":"2024-01-15T17:56:26.181879+0000","last_fullsized":"2024-01-15T17:56:26.181879+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.197426+0000","last_change":"2024-01-15T17:56:26.197426+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.197426+0000","last_undegraded":"2024-01-15T17:56:26.197426+0000","last_fullsized":"2024-01-15T17:56:26.197426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202504+0000","last_change":"2024-01-15T17:56:26.202504+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202504+0000","last_undegraded":"2024-01-15T17:56:26.202504+0000","last_fullsized":"2024-01-15T17:56:26.202504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.192203+0000","last_change":"2024-01-15T17:56:26.192203+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.192203+0000","last_undegraded":"2024-01-15T17:56:26.192203+0000","last_fullsized":"2024-01-15T17:56:26.192203+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193807+0000","last_change":"2024-01-15T17:56:26.193807+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193807+0000","last_undegraded":"2024-01-15T17:56:26.193807+0000","last_fullsized":"2024-01-15T17:56:26.193807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184195+0000","last_change":"2024-01-15T17:56:26.184195+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184195+0000","last_undegraded":"2024-01-15T17:56:26.184195+0000","last_fullsized":"2024-01-15T17:56:26.184195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.204185+0000","last_change":"2024-01-15T17:56:26.204185+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.204185+0000","last_undegraded":"2024-01-15T17:56:26.204185+0000","last_fullsized":"2024-01-15T17:56:26.204185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189019+0000","last_change":"2024-01-15T17:56:26.189019+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189019+0000","last_undegraded":"2024-01-15T17:56:26.189019+0000","last_fullsized":"2024-01-15T17:56:26.189019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184645+0000","last_change":"2024-01-15T17:56:26.184645+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184645+0000","last_undegraded":"2024-01-15T17:56:26.184645+0000","last_fullsized":"2024-01-15T17:56:26.184645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.182486+0000","last_change":"2024-01-15T17:56:26.182486+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.182486+0000","last_undegraded":"2024-01-15T17:56:26.182486+0000","last_fullsized":"2024-01-15T17:56:26.182486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193649+0000","last_change":"2024-01-15T17:56:26.193649+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193649+0000","last_undegraded":"2024-01-15T17:56:26.193649+0000","last_fullsized":"2024-01-15T17:56:26.193649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191687+0000","last_change":"2024-01-15T17:56:26.191687+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191687+0000","last_undegraded":"2024-01-15T17:56:26.191687+0000","last_fullsized":"2024-01-15T17:56:26.191687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193341+0000","last_change":"2024-01-15T17:56:26.193341+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193341+0000","last_undegraded":"2024-01-15T17:56:26.193341+0000","last_fullsized":"2024-01-15T17:56:26.193341+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185835+0000","last_change":"2024-01-15T17:56:26.185835+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185835+0000","last_undegraded":"2024-01-15T17:56:26.185835+0000","last_fullsized":"2024-01-15T17:56:26.185835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.196524+0000","last_change":"2024-01-15T17:56:26.196524+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.196524+0000","last_undegraded":"2024-01-15T17:56:26.196524+0000","last_fullsized":"2024-01-15T17:56:26.196524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202508+0000","last_change":"2024-01-15T17:56:26.202508+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202508+0000","last_undegraded":"2024-01-15T17:56:26.202508+0000","last_fullsized":"2024-01-15T17:56:26.202508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183698+0000","last_change":"2024-01-15T17:56:26.183698+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183698+0000","last_undegraded":"2024-01-15T17:56:26.183698+0000","last_fullsized":"2024-01-15T17:56:26.183698+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.205444+0000","last_change":"2024-01-15T17:56:26.205444+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.205444+0000","last_undegraded":"2024-01-15T17:56:26.205444+0000","last_fullsized":"2024-01-15T17:56:26.205444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185930+0000","last_change":"2024-01-15T17:56:26.185930+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185930+0000","last_undegraded":"2024-01-15T17:56:26.185930+0000","last_fullsized":"2024-01-15T17:56:26.185930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207378+0000","last_change":"2024-01-15T17:56:26.202527+0000","last_active":"2024-01-15T17:56:26.207378+0000","last_peered":"2024-01-15T17:56:26.207378+0000","last_clean":"2024-01-15T17:56:26.207378+0000","last_became_active":"2024-01-15T17:56:26.202254+0000","last_became_peered":"2024-01-15T17:56:26.202254+0000","last_unstale":"2024-01-15T17:56:26.207378+0000","last_undegraded":"2024-01-15T17:56:26.207378+0000","last_fullsized":"2024-01-15T17:56:26.207378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:58:49.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183024+0000","last_change":"2024-01-15T17:56:26.183024+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183024+0000","last_undegraded":"2024-01-15T17:56:26.183024+0000","last_fullsized":"2024-01-15T17:56:26.183024+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.214222+0000","last_change":"2024-01-15T17:56:26.213479+0000","last_active":"2024-01-15T17:56:26.214222+0000","last_peered":"2024-01-15T17:56:26.214222+0000","last_clean":"2024-01-15T17:56:26.214222+0000","last_became_active":"2024-01-15T17:56:26.213165+0000","last_became_peered":"2024-01-15T17:56:26.213165+0000","last_unstale":"2024-01-15T17:56:26.214222+0000","last_undegraded":"2024-01-15T17:56:26.214222+0000","last_fullsized":"2024-01-15T17:56:26.214222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:43.670278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191397+0000","last_change":"2024-01-15T17:56:26.191397+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191397+0000","last_undegraded":"2024-01-15T17:56:26.191397+0000","last_fullsized":"2024-01-15T17:56:26.191397+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207053+0000","last_change":"2024-01-15T17:56:26.199957+0000","last_active":"2024-01-15T17:56:26.207053+0000","last_peered":"2024-01-15T17:56:26.207053+0000","last_clean":"2024-01-15T17:56:26.207053+0000","last_became_active":"2024-01-15T17:56:26.199632+0000","last_became_peered":"2024-01-15T17:56:26.199632+0000","last_unstale":"2024-01-15T17:56:26.207053+0000","last_undegraded":"2024-01-15T17:56:26.207053+0000","last_fullsized":"2024-01-15T17:56:26.207053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:01:25.558146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190219+0000","last_change":"2024-01-15T17:56:26.190219+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190219+0000","last_undegraded":"2024-01-15T17:56:26.190219+0000","last_fullsized":"2024-01-15T17:56:26.190219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.211037+0000","last_change":"2024-01-15T17:56:26.209481+0000","last_active":"2024-01-15T17:56:26.211037+0000","last_peered":"2024-01-15T17:56:26.211037+0000","last_clean":"2024-01-15T17:56:26.211037+0000","last_became_active":"2024-01-15T17:56:26.209251+0000","last_became_peered":"2024-01-15T17:56:26.209251+0000","last_unstale":"2024-01-15T17:56:26.211037+0000","last_undegraded":"2024-01-15T17:56:26.211037+0000","last_fullsized":"2024-01-15T17:56:26.211037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:13:34.770667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.196248+0000","last_change":"2024-01-15T17:56:26.196248+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.196248+0000","last_undegraded":"2024-01-15T17:56:26.196248+0000","last_fullsized":"2024-01-15T17:56:26.196248+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.211475+0000","last_change":"2024-01-15T17:56:26.209952+0000","last_active":"2024-01-15T17:56:26.211475+0000","last_peered":"2024-01-15T17:56:26.211475+0000","last_clean":"2024-01-15T17:56:26.211475+0000","last_became_active":"2024-01-15T17:56:26.209696+0000","last_became_peered":"2024-01-15T17:56:26.209696+0000","last_unstale":"2024-01-15T17:56:26.211475+0000","last_undegraded":"2024-01-15T17:56:26.211475+0000","last_fullsized":"2024-01-15T17:56:26.211475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:41.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181863+0000","last_change":"2024-01-15T17:56:26.181863+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181863+0000","last_undegraded":"2024-01-15T17:56:26.181863+0000","last_fullsized":"2024-01-15T17:56:26.181863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213022+0000","last_change":"2024-01-15T17:56:26.211397+0000","last_active":"2024-01-15T17:56:26.213022+0000","last_peered":"2024-01-15T17:56:26.213022+0000","last_clean":"2024-01-15T17:56:26.213022+0000","last_became_active":"2024-01-15T17:56:26.210076+0000","last_became_peered":"2024-01-15T17:56:26.210076+0000","last_unstale":"2024-01-15T17:56:26.213022+0000","last_undegraded":"2024-01-15T17:56:26.213022+0000","last_fullsized":"2024-01-15T17:56:26.213022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:31:26.304443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188872+0000","last_change":"2024-01-15T17:56:26.188872+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188872+0000","last_undegraded":"2024-01-15T17:56:26.188872+0000","last_fullsized":"2024-01-15T17:56:26.188872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207337+0000","last_change":"2024-01-15T17:56:26.203486+0000","last_active":"2024-01-15T17:56:26.207337+0000","last_peered":"2024-01-15T17:56:26.207337+0000","last_clean":"2024-01-15T17:56:26.207337+0000","last_became_active":"2024-01-15T17:56:26.203141+0000","last_became_peered":"2024-01-15T17:56:26.203141+0000","last_unstale":"2024-01-15T17:56:26.207337+0000","last_undegraded":"2024-01-15T17:56:26.207337+0000","last_fullsized":"2024-01-15T17:56:26.207337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:28:33.523943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183545+0000","last_change":"2024-01-15T17:56:26.183545+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183545+0000","last_undegraded":"2024-01-15T17:56:26.183545+0000","last_fullsized":"2024-01-15T17:56:26.183545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.203978+0000","last_change":"2024-01-15T17:56:26.200676+0000","last_active":"2024-01-15T17:56:26.203978+0000","last_peered":"2024-01-15T17:56:26.203978+0000","last_clean":"2024-01-15T17:56:26.203978+0000","last_became_active":"2024-01-15T17:56:26.199502+0000","last_became_peered":"2024-01-15T17:56:26.199502+0000","last_unstale":"2024-01-15T17:56:26.203978+0000","last_undegraded":"2024-01-15T17:56:26.203978+0000","last_fullsized":"2024-01-15T17:56:26.203978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:17.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185090+0000","last_change":"2024-01-15T17:56:26.185090+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185090+0000","last_undegraded":"2024-01-15T17:56:26.185090+0000","last_fullsized":"2024-01-15T17:56:26.185090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213865+0000","last_change":"2024-01-15T17:56:26.212810+0000","last_active":"2024-01-15T17:56:26.213865+0000","last_peered":"2024-01-15T17:56:26.213865+0000","last_clean":"2024-01-15T17:56:26.213865+0000","last_became_active":"2024-01-15T17:56:26.212482+0000","last_became_peered":"2024-01-15T17:56:26.212482+0000","last_unstale":"2024-01-15T17:56:26.213865+0000","last_undegraded":"2024-01-15T17:56:26.213865+0000","last_fullsized":"2024-01-15T17:56:26.213865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:26.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188955+0000","last_change":"2024-01-15T17:56:26.188955+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188955+0000","last_undegraded":"2024-01-15T17:56:26.188955+0000","last_fullsized":"2024-01-15T17:56:26.188955+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210403+0000","last_change":"2024-01-15T17:56:26.208053+0000","last_active":"2024-01-15T17:56:26.210403+0000","last_peered":"2024-01-15T17:56:26.210403+0000","last_clean":"2024-01-15T17:56:26.210403+0000","last_became_active":"2024-01-15T17:56:26.207220+0000","last_became_peered":"2024-01-15T17:56:26.207220+0000","last_unstale":"2024-01-15T17:56:26.210403+0000","last_undegraded":"2024-01-15T17:56:26.210403+0000","last_fullsized":"2024-01-15T17:56:26.210403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:31.045653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188235+0000","last_change":"2024-01-15T17:56:26.188235+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188235+0000","last_undegraded":"2024-01-15T17:56:26.188235+0000","last_fullsized":"2024-01-15T17:56:26.188235+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.215003+0000","last_change":"2024-01-15T17:56:26.213932+0000","last_active":"2024-01-15T17:56:26.215003+0000","last_peered":"2024-01-15T17:56:26.215003+0000","last_clean":"2024-01-15T17:56:26.215003+0000","last_became_active":"2024-01-15T17:56:26.213653+0000","last_became_peered":"2024-01-15T17:56:26.213653+0000","last_unstale":"2024-01-15T17:56:26.215003+0000","last_undegraded":"2024-01-15T17:56:26.215003+0000","last_fullsized":"2024-01-15T17:56:26.215003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:46.243898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188570+0000","last_change":"2024-01-15T17:56:26.188570+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188570+0000","last_undegraded":"2024-01-15T17:56:26.188570+0000","last_fullsized":"2024-01-15T17:56:26.188570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.209612+0000","last_change":"2024-01-15T17:56:26.203678+0000","last_active":"2024-01-15T17:56:26.209612+0000","last_peered":"2024-01-15T17:56:26.209612+0000","last_clean":"2024-01-15T17:56:26.209612+0000","last_became_active":"2024-01-15T17:56:26.203341+0000","last_became_peered":"2024-01-15T17:56:26.203341+0000","last_unstale":"2024-01-15T17:56:26.209612+0000","last_undegraded":"2024-01-15T17:56:26.209612+0000","last_fullsized":"2024-01-15T17:56:26.209612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:14:31.052513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184920+0000","last_change":"2024-01-15T17:56:26.184920+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184920+0000","last_undegraded":"2024-01-15T17:56:26.184920+0000","last_fullsized":"2024-01-15T17:56:26.184920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212414+0000","last_change":"2024-01-15T17:56:26.210551+0000","last_active":"2024-01-15T17:56:26.212414+0000","last_peered":"2024-01-15T17:56:26.212414+0000","last_clean":"2024-01-15T17:56:26.212414+0000","last_became_active":"2024-01-15T17:56:26.209399+0000","last_became_peered":"2024-01-15T17:56:26.209399+0000","last_unstale":"2024-01-15T17:56:26.212414+0000","last_undegraded":"2024-01-15T17:56:26.212414+0000","last_fullsized":"2024-01-15T17:56:26.212414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:12:05.549396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185817+0000","last_change":"2024-01-15T17:56:26.185817+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185817+0000","last_undegraded":"2024-01-15T17:56:26.185817+0000","last_fullsized":"2024-01-15T17:56:26.185817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208828+0000","last_change":"2024-01-15T17:56:26.201105+0000","last_active":"2024-01-15T17:56:26.208828+0000","last_peered":"2024-01-15T17:56:26.208828+0000","last_clean":"2024-01-15T17:56:26.208828+0000","last_became_active":"2024-01-15T17:56:26.200818+0000","last_became_peered":"2024-01-15T17:56:26.200818+0000","last_unstale":"2024-01-15T17:56:26.208828+0000","last_undegraded":"2024-01-15T17:56:26.208828+0000","last_fullsized":"2024-01-15T17:56:26.208828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:58:49.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.199505+0000","last_change":"2024-01-15T17:56:26.199505+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.199505+0000","last_undegraded":"2024-01-15T17:56:26.199505+0000","last_fullsized":"2024-01-15T17:56:26.199505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212669+0000","last_change":"2024-01-15T17:56:26.211092+0000","last_active":"2024-01-15T17:56:26.212669+0000","last_peered":"2024-01-15T17:56:26.212669+0000","last_clean":"2024-01-15T17:56:26.212669+0000","last_became_active":"2024-01-15T17:56:26.210804+0000","last_became_peered":"2024-01-15T17:56:26.210804+0000","last_unstale":"2024-01-15T17:56:26.212669+0000","last_undegraded":"2024-01-15T17:56:26.212669+0000","last_fullsized":"2024-01-15T17:56:26.212669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:02:16.155632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181524+0000","last_change":"2024-01-15T17:56:26.181524+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181524+0000","last_undegraded":"2024-01-15T17:56:26.181524+0000","last_fullsized":"2024-01-15T17:56:26.181524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207189+0000","last_change":"2024-01-15T17:56:26.201849+0000","last_active":"2024-01-15T17:56:26.207189+0000","last_peered":"2024-01-15T17:56:26.207189+0000","last_clean":"2024-01-15T17:56:26.207189+0000","last_became_active":"2024-01-15T17:56:26.201558+0000","last_became_peered":"2024-01-15T17:56:26.201558+0000","last_unstale":"2024-01-15T17:56:26.207189+0000","last_undegraded":"2024-01-15T17:56:26.207189+0000","last_fullsized":"2024-01-15T17:56:26.207189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:43.277391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190247+0000","last_change":"2024-01-15T17:56:26.190247+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190247+0000","last_undegraded":"2024-01-15T17:56:26.190247+0000","last_fullsized":"2024-01-15T17:56:26.190247+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213314+0000","last_change":"2024-01-15T17:56:26.211508+0000","last_active":"2024-01-15T17:56:26.213314+0000","last_peered":"2024-01-15T17:56:26.213314+0000","last_clean":"2024-01-15T17:56:26.213314+0000","last_became_active":"2024-01-15T17:56:26.207011+0000","last_became_peered":"2024-01-15T17:56:26.207011+0000","last_unstale":"2024-01-15T17:56:26.213314+0000","last_undegraded":"2024-01-15T17:56:26.213314+0000","last_fullsized":"2024-01-15T17:56:26.213314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:40.566415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191272+0000","last_change":"2024-01-15T17:56:26.191272+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191272+0000","last_undegraded":"2024-01-15T17:56:26.191272+0000","last_fullsized":"2024-01-15T17:56:26.191272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.203619+0000","last_change":"2024-01-15T17:56:26.200302+0000","last_active":"2024-01-15T17:56:26.203619+0000","last_peered":"2024-01-15T17:56:26.203619+0000","last_clean":"2024-01-15T17:56:26.203619+0000","last_became_active":"2024-01-15T17:56:26.200001+0000","last_became_peered":"2024-01-15T17:56:26.200001+0000","last_unstale":"2024-01-15T17:56:26.203619+0000","last_undegraded":"2024-01-15T17:56:26.203619+0000","last_fullsized":"2024-01-15T17:56:26.203619+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:06:34.072861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187437+0000","last_change":"2024-01-15T17:56:26.187437+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187437+0000","last_undegraded":"2024-01-15T17:56:26.187437+0000","last_fullsized":"2024-01-15T17:56:26.187437+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210901+0000","last_change":"2024-01-15T17:56:26.207943+0000","last_active":"2024-01-15T17:56:26.210901+0000","last_peered":"2024-01-15T17:56:26.210901+0000","last_clean":"2024-01-15T17:56:26.210901+0000","last_became_active":"2024-01-15T17:56:26.207604+0000","last_became_peered":"2024-01-15T17:56:26.207604+0000","last_unstale":"2024-01-15T17:56:26.210901+0000","last_undegraded":"2024-01-15T17:56:26.210901+0000","last_fullsized":"2024-01-15T17:56:26.210901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:42.102953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187383+0000","last_change":"2024-01-15T17:56:26.187383+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187383+0000","last_undegraded":"2024-01-15T17:56:26.187383+0000","last_fullsized":"2024-01-15T17:56:26.187383+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210215+0000","last_change":"2024-01-15T17:56:26.207505+0000","last_active":"2024-01-15T17:56:26.210215+0000","last_peered":"2024-01-15T17:56:26.210215+0000","last_clean":"2024-01-15T17:56:26.210215+0000","last_became_active":"2024-01-15T17:56:26.207210+0000","last_became_peered":"2024-01-15T17:56:26.207210+0000","last_unstale":"2024-01-15T17:56:26.210215+0000","last_undegraded":"2024-01-15T17:56:26.210215+0000","last_fullsized":"2024-01-15T17:56:26.210215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:13:34.770667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183722+0000","last_change":"2024-01-15T17:56:26.183722+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183722+0000","last_undegraded":"2024-01-15T17:56:26.183722+0000","last_fullsized":"2024-01-15T17:56:26.183722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.205077+0000","last_change":"2024-01-15T17:56:26.202274+0000","last_active":"2024-01-15T17:56:26.205077+0000","last_peered":"2024-01-15T17:56:26.205077+0000","last_clean":"2024-01-15T17:56:26.205077+0000","last_became_active":"2024-01-15T17:56:26.201924+0000","last_became_peered":"2024-01-15T17:56:26.201924+0000","last_unstale":"2024-01-15T17:56:26.205077+0000","last_undegraded":"2024-01-15T17:56:26.205077+0000","last_fullsized":"2024-01-15T17:56:26.205077+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:35.925710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183777+0000","last_change":"2024-01-15T17:56:26.183777+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183777+0000","last_undegraded":"2024-01-15T17:56:26.183777+0000","last_fullsized":"2024-01-15T17:56:26.183777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.205313+0000","last_change":"2024-01-15T17:56:26.200301+0000","last_active":"2024-01-15T17:56:26.205313+0000","last_peered":"2024-01-15T17:56:26.205313+0000","last_clean":"2024-01-15T17:56:26.205313+0000","last_became_active":"2024-01-15T17:56:26.200003+0000","last_became_peered":"2024-01-15T17:56:26.200003+0000","last_unstale":"2024-01-15T17:56:26.205313+0000","last_undegraded":"2024-01-15T17:56:26.205313+0000","last_fullsized":"2024-01-15T17:56:26.205313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:02:29.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.186611+0000","last_change":"2024-01-15T17:56:26.186611+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.186611+0000","last_undegraded":"2024-01-15T17:56:26.186611+0000","last_fullsized":"2024-01-15T17:56:26.186611+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.215099+0000","last_change":"2024-01-15T17:56:26.214358+0000","last_active":"2024-01-15T17:56:26.215099+0000","last_peered":"2024-01-15T17:56:26.215099+0000","last_clean":"2024-01-15T17:56:26.215099+0000","last_became_active":"2024-01-15T17:56:26.213452+0000","last_became_peered":"2024-01-15T17:56:26.213452+0000","last_unstale":"2024-01-15T17:56:26.215099+0000","last_undegraded":"2024-01-15T17:56:26.215099+0000","last_fullsized":"2024-01-15T17:56:26.215099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:05:49.120775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.198507+0000","last_change":"2024-01-15T17:56:26.198507+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.198507+0000","last_undegraded":"2024-01-15T17:56:26.198507+0000","last_fullsized":"2024-01-15T17:56:26.198507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212927+0000","last_change":"2024-01-15T17:56:26.212040+0000","last_active":"2024-01-15T17:56:26.212927+0000","last_peered":"2024-01-15T17:56:26.212927+0000","last_clean":"2024-01-15T17:56:26.212927+0000","last_became_active":"2024-01-15T17:56:26.211151+0000","last_became_peered":"2024-01-15T17:56:26.211151+0000","last_unstale":"2024-01-15T17:56:26.212927+0000","last_undegraded":"2024-01-15T17:56:26.212927+0000","last_fullsized":"2024-01-15T17:56:26.212927+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T17:58:44.566548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185596+0000","last_change":"2024-01-15T17:56:26.185596+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185596+0000","last_undegraded":"2024-01-15T17:56:26.185596+0000","last_fullsized":"2024-01-15T17:56:26.185596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210500+0000","last_change":"2024-01-15T17:56:26.201506+0000","last_active":"2024-01-15T17:56:26.210500+0000","last_peered":"2024-01-15T17:56:26.210500+0000","last_clean":"2024-01-15T17:56:26.210500+0000","last_became_active":"2024-01-15T17:56:26.201268+0000","last_became_peered":"2024-01-15T17:56:26.201268+0000","last_unstale":"2024-01-15T17:56:26.210500+0000","last_undegraded":"2024-01-15T17:56:26.210500+0000","last_fullsized":"2024-01-15T17:56:26.210500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:43.670278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190043+0000","last_change":"2024-01-15T17:56:26.190043+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190043+0000","last_undegraded":"2024-01-15T17:56:26.190043+0000","last_fullsized":"2024-01-15T17:56:26.190043+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.202533+0000","last_change":"2024-01-15T17:56:26.200072+0000","last_active":"2024-01-15T17:56:26.202533+0000","last_peered":"2024-01-15T17:56:26.202533+0000","last_clean":"2024-01-15T17:56:26.202533+0000","last_became_active":"2024-01-15T17:56:26.199764+0000","last_became_peered":"2024-01-15T17:56:26.199764+0000","last_unstale":"2024-01-15T17:56:26.202533+0000","last_undegraded":"2024-01-15T17:56:26.202533+0000","last_fullsized":"2024-01-15T17:56:26.202533+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189595+0000","last_change":"2024-01-15T17:56:26.189595+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189595+0000","last_undegraded":"2024-01-15T17:56:26.189595+0000","last_fullsized":"2024-01-15T17:56:26.189595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208497+0000","last_change":"2024-01-15T17:56:26.200854+0000","last_active":"2024-01-15T17:56:26.208497+0000","last_peered":"2024-01-15T17:56:26.208497+0000","last_clean":"2024-01-15T17:56:26.208497+0000","last_became_active":"2024-01-15T17:56:26.200317+0000","last_became_peered":"2024-01-15T17:56:26.200317+0000","last_unstale":"2024-01-15T17:56:26.208497+0000","last_undegraded":"2024-01-15T17:56:26.208497+0000","last_fullsized":"2024-01-15T17:56:26.208497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181422+0000","last_change":"2024-01-15T17:56:26.181422+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181422+0000","last_undegraded":"2024-01-15T17:56:26.181422+0000","last_fullsized":"2024-01-15T17:56:26.181422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208623+0000","last_change":"2024-01-15T17:56:26.205649+0000","last_active":"2024-01-15T17:56:26.208623+0000","last_peered":"2024-01-15T17:56:26.208623+0000","last_clean":"2024-01-15T17:56:26.208623+0000","last_became_active":"2024-01-15T17:56:26.205337+0000","last_became_peered":"2024-01-15T17:56:26.205337+0000","last_unstale":"2024-01-15T17:56:26.208623+0000","last_undegraded":"2024-01-15T17:56:26.208623+0000","last_fullsized":"2024-01-15T17:56:26.208623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:40.566415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187391+0000","last_change":"2024-01-15T17:56:26.187391+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187391+0000","last_undegraded":"2024-01-15T17:56:26.187391+0000","last_fullsized":"2024-01-15T17:56:26.187391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208848+0000","last_change":"2024-01-15T17:56:26.201883+0000","last_active":"2024-01-15T17:56:26.208848+0000","last_peered":"2024-01-15T17:56:26.208848+0000","last_clean":"2024-01-15T17:56:26.208848+0000","last_became_active":"2024-01-15T17:56:26.201275+0000","last_became_peered":"2024-01-15T17:56:26.201275+0000","last_unstale":"2024-01-15T17:56:26.208848+0000","last_undegraded":"2024-01-15T17:56:26.208848+0000","last_fullsized":"2024-01-15T17:56:26.208848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:02:29.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.182677+0000","last_change":"2024-01-15T17:56:26.182677+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.182677+0000","last_undegraded":"2024-01-15T17:56:26.182677+0000","last_fullsized":"2024-01-15T17:56:26.182677+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207896+0000","last_change":"2024-01-15T17:56:26.205102+0000","last_active":"2024-01-15T17:56:26.207896+0000","last_peered":"2024-01-15T17:56:26.207896+0000","last_clean":"2024-01-15T17:56:26.207896+0000","last_became_active":"2024-01-15T17:56:26.204747+0000","last_became_peered":"2024-01-15T17:56:26.204747+0000","last_unstale":"2024-01-15T17:56:26.207896+0000","last_undegraded":"2024-01-15T17:56:26.207896+0000","last_fullsized":"2024-01-15T17:56:26.207896+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:48.459125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.195525+0000","last_change":"2024-01-15T17:56:26.195525+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.195525+0000","last_undegraded":"2024-01-15T17:56:26.195525+0000","last_fullsized":"2024-01-15T17:56:26.195525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212667+0000","last_change":"2024-01-15T17:56:26.211169+0000","last_active":"2024-01-15T17:56:26.212667+0000","last_peered":"2024-01-15T17:56:26.212667+0000","last_clean":"2024-01-15T17:56:26.212667+0000","last_became_active":"2024-01-15T17:56:26.210842+0000","last_became_peered":"2024-01-15T17:56:26.210842+0000","last_unstale":"2024-01-15T17:56:26.212667+0000","last_undegraded":"2024-01-15T17:56:26.212667+0000","last_fullsized":"2024-01-15T17:56:26.212667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:25.400140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.201814+0000","last_change":"2024-01-15T17:56:26.201814+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.201814+0000","last_undegraded":"2024-01-15T17:56:26.201814+0000","last_fullsized":"2024-01-15T17:56:26.201814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210656+0000","last_change":"2024-01-15T17:56:26.208654+0000","last_active":"2024-01-15T17:56:26.210656+0000","last_peered":"2024-01-15T17:56:26.210656+0000","last_clean":"2024-01-15T17:56:26.210656+0000","last_became_active":"2024-01-15T17:56:26.208305+0000","last_became_peered":"2024-01-15T17:56:26.208305+0000","last_unstale":"2024-01-15T17:56:26.210656+0000","last_undegraded":"2024-01-15T17:56:26.210656+0000","last_fullsized":"2024-01-15T17:56:26.210656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":172,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-01-15T17:56:25.185666+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:56:25.185666+0000","last_peered":"2024-01-15T17:56:25.185666+0000","last_clean":"2024-01-15T17:56:25.185666+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:56:25.185666+0000","last_undegraded":"2024-01-15T17:56:25.185666+0000","last_fullsized":"2024-01-15T17:56:25.185666+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":962560,"data_stored":732013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.643}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.373},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.534}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.28},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.081}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.181},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.052,"5min":1.052,"15min":1.052},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.2330000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.444}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.49},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.1339999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.714,"15min":1.714},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.325},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.611}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":0.876}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.155},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.2569999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.3280000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.417},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1140000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.605},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.4810000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.758,"5min":1.758,"15min":1.758},"min":{"1min":1.202,"5min":1.202,"15min":1.202},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.5209999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.212},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"min":{"1min":1.286,"5min":1.286,"15min":1.286},"max":{"1min":3.141,"5min":3.141,"15min":3.141},"last":1.456}]}]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.3840000000000003,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.22},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":5.2290000000000001,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.3440000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.625}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.835,"5min":2.835,"15min":2.835},"last":1.546},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.413}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.264}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.419,"15min":1.419},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.5720000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.083,"5min":1.083,"15min":1.083},"max":{"1min":5.218,"5min":5.218,"15min":5.218},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.512}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":1.2,"5min":1.2,"15min":1.2},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.4430000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.996},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0740000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.597},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.127}]},{"osd":2,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.339},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.383}]},{"osd":3,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.4410000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.536},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"min":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.794}]},{"osd":6,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":1.706,"15min":1.706},"min":{"1min":1.117,"5min":1.117,"15min":1.117},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.742},{"interface":"front","average":{"1min":1.845,"5min":1.845,"15min":1.845},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.8520000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"min":{"1min":1.385,"5min":1.385,"15min":1.385},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.899},{"interface":"front","average":{"1min":1.929,"5min":1.929,"15min":1.929},"min":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":1.929}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.2989999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.4410000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.238}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.353}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.643},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":1.167}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.673,"15min":1.673},"min":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.5820000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.698,"15min":1.698},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"min":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.538}]}]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.208}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.135}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":4.008,"5min":4.008,"15min":4.008},"last":1.278},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.0289999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.171}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.6070000000000002,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.389},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.0529999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.028},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.61199999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1990000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.23,"15min":2.23},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":22.106999999999999,"5min":22.106999999999999,"15min":22.106999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.0740000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":9.8510000000000009,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.272},{"interface":"front","average":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":14.359999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.113}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":13.228999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":4.9900000000000002,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":1.4039999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":14.272,"5min":14.272,"15min":14.272},"last":1.357},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.1659999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.474}]}]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":962560,"data_stored":732013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.032}]},{"osd":2,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.496},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.93300000000000005}]},{"osd":3,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.597},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.542}]},{"osd":4,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.401},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.6619999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.6419999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.9990000000000001,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":1.792},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.274}]},{"osd":7,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.8100000000000001,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.857},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.74}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:56:32.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.476+0000 7f8665200640 1 Processor -- start 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.476+0000 7f8665200640 1 -- start start 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.478+0000 7f8665200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f866006d750 0x7f86600b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.478+0000 7f8665200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f86600b3450 0x7f8660070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.478+0000 7f8665200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866006dbe0 con 0x7f8660164230 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.478+0000 7f8665200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866008aea0 con 0x7f866006d750 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.478+0000 7f8665200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86600b3860 con 0x7f86600b3450 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.478+0000 7f8657e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f866006d750 0x7f86600b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.478+0000 7f8657e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f866006d750 0x7f86600b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47754/0 (socket says 172.21.15.26:47754) 2024-01-15T17:56:32.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.478+0000 7f8657e00640 1 -- 172.21.15.26:0/3046039965 learned_addr learned my addr 172.21.15.26:0/3046039965 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.478+0000 7f865fe00640 1 --2- 172.21.15.26:0/3046039965 >> v2:172.21.15.124:3300/0 conn(0x7f86600b3450 0x7f8660070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.479+0000 7f865ea00640 1 -- 172.21.15.26:0/3046039965 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217793792 0 0) 0x7f866006dbe0 con 0x7f8660164230 2024-01-15T17:56:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.495+0000 7f865ea00640 1 -- 172.21.15.26:0/3046039965 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8640003500 con 0x7f8660164230 2024-01-15T17:56:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.495+0000 7f8657e00640 1 -- 172.21.15.26:0/3046039965 >> v2:172.21.15.124:3300/0 conn(0x7f86600b3450 msgr2=0x7f8660070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.495+0000 7f8657e00640 1 --2- 172.21.15.26:0/3046039965 >> v2:172.21.15.124:3300/0 conn(0x7f86600b3450 0x7f8660070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.495+0000 7f8657e00640 1 -- 172.21.15.26:0/3046039965 >> v1:172.21.15.124:6789/0 conn(0x7f8660164230 legacy=0x7f866016e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:32.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.495+0000 7f8657e00640 1 -- 172.21.15.26:0/3046039965 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866016da90 con 0x7f866006d750 2024-01-15T17:56:32.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.498+0000 7f8657e00640 1 --2- 172.21.15.26:0/3046039965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866006d750 0x7f86600b2f10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f866016d830 tx=0x7f864802c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d36cc84dc8c0d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:32.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.499+0000 7f865ea00640 1 -- 172.21.15.26:0/3046039965 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648036400 con 0x7f866006d750 2024-01-15T17:56:32.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.499+0000 7f865ea00640 1 -- 172.21.15.26:0/3046039965 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8648030070 con 0x7f866006d750 2024-01-15T17:56:32.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.499+0000 7f865ea00640 1 -- 172.21.15.26:0/3046039965 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648036780 con 0x7f866006d750 2024-01-15T17:56:32.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.500+0000 7f8665200640 1 -- 172.21.15.26:0/3046039965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866006d750 msgr2=0x7f86600b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:32.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.500+0000 7f8665200640 1 --2- 172.21.15.26:0/3046039965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866006d750 0x7f86600b2f10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f866016d830 tx=0x7f864802c1a0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.500+0000 7f8665200640 1 -- 172.21.15.26:0/3046039965 shutdown_connections 2024-01-15T17:56:32.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.500+0000 7f8665200640 1 --2- 172.21.15.26:0/3046039965 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f866006d750 0x7f86600b2f10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.500+0000 7f8665200640 1 --2- 172.21.15.26:0/3046039965 >> v2:172.21.15.124:3300/0 conn(0x7f86600b3450 0x7f8660070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.500+0000 7f8665200640 1 -- 172.21.15.26:0/3046039965 >> 172.21.15.26:0/3046039965 conn(0x7f86600b0b00 msgr2=0x7f86600b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:32.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.500+0000 7f8665200640 1 -- 172.21.15.26:0/3046039965 shutdown_connections 2024-01-15T17:56:32.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.500+0000 7f8665200640 1 -- 172.21.15.26:0/3046039965 wait complete. 2024-01-15T17:56:32.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.502+0000 7f8665200640 1 Processor -- start 2024-01-15T17:56:32.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.503+0000 7f8665200640 1 -- start start 2024-01-15T17:56:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f8665200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f866006d750 0x7f8660156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f8665200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86600b3450 0x7f86601504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f8665200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8660164230 0x7f8660151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f8665200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8660150a30 con 0x7f8660164230 2024-01-15T17:56:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f8665200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8660150c60 con 0x7f86600b3450 2024-01-15T17:56:32.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f8665200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8660150f40 con 0x7f866006d750 2024-01-15T17:56:32.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f865fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8660164230 0x7f8660151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:32.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f865fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8660164230 0x7f8660151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47764/0 (socket says 172.21.15.26:47764) 2024-01-15T17:56:32.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f865fe00640 1 -- 172.21.15.26:0/1897378007 learned_addr learned my addr 172.21.15.26:0/1897378007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:32.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.504+0000 7f865f400640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f866006d750 0x7f8660156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:32.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f8657e00640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86600b3450 0x7f86601504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:32.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f865fe00640 1 -- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f866006d750 msgr2=0x7f8660156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:32.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f865fe00640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f866006d750 0x7f8660156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f865fe00640 1 -- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86600b3450 msgr2=0x7f86601504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:32.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f865fe00640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86600b3450 0x7f86601504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f865fe00640 1 -- 172.21.15.26:0/1897378007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864800b5d0 con 0x7f8660164230 2024-01-15T17:56:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f865f400640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f866006d750 0x7f8660156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f865fe00640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8660164230 0x7f8660151d00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f865800efc0 tx=0x7f86580075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f8657400640 1 -- 172.21.15.26:0/1897378007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865800e2d0 con 0x7f8660164230 2024-01-15T17:56:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f8657400640 1 -- 172.21.15.26:0/1897378007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8658014040 con 0x7f8660164230 2024-01-15T17:56:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.505+0000 7f8657400640 1 -- 172.21.15.26:0/1897378007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865800c610 con 0x7f8660164230 2024-01-15T17:56:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.506+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8660151220 con 0x7f8660164230 2024-01-15T17:56:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.506+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86601517e0 con 0x7f8660164230 2024-01-15T17:56:32.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.507+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8660067450 con 0x7f8660164230 2024-01-15T17:56:32.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.508+0000 7f8657400640 1 -- 172.21.15.26:0/1897378007 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f865800c770 con 0x7f8660164230 2024-01-15T17:56:32.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.509+0000 7f8657400640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f863806b2c0 0x7f863806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:32.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.509+0000 7f8657400640 1 -- 172.21.15.26:0/1897378007 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8658008040 con 0x7f8660164230 2024-01-15T17:56:32.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.510+0000 7f865f400640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f863806b2c0 0x7f863806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:32.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.511+0000 7f865f400640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f863806b2c0 0x7f863806d780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86500011e0 tx=0x7f8650009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:32.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.515+0000 7f8657400640 1 -- 172.21.15.26:0/1897378007 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8658059660 con 0x7f8660164230 2024-01-15T17:56:32.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.704+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f866006e140 con 0x7f863806b2c0 2024-01-15T17:56:32.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.725+0000 7f8657400640 1 -- 172.21.15.26:0/1897378007 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280833 (secure 0 0 0) 0x7f8650001d70 con 0x7f863806b2c0 2024-01-15T17:56:32.730 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:32.734 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f863806b2c0 msgr2=0x7f863806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f863806b2c0 0x7f863806d780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f86500011e0 tx=0x7f8650009040 comp rx=0 tx=0).stop 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8660164230 msgr2=0x7f8660151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8660164230 0x7f8660151d00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f865800efc0 tx=0x7f86580075c0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 shutdown_connections 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f863806b2c0 0x7f863806d780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8660164230 0x7f8660151d00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86600b3450 0x7f86601504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 --2- 172.21.15.26:0/1897378007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f866006d750 0x7f8660156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:32.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.732+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 >> 172.21.15.26:0/1897378007 conn(0x7f86600b0b00 msgr2=0x7f8660163720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.733+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 shutdown_connections 2024-01-15T17:56:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:32.733+0000 7f8665200640 1 -- 172.21.15.26:0/1897378007 wait complete. 2024-01-15T17:56:32.757 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-01-15T17:56:31.939478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236848,"kb_used_data":4028,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737872,"statfs":{"total":773094113280,"available":772851580928,"internally_reserved":0,"allocated":4124672,"data_stored":2307629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001737"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184384+0000","last_change":"2024-01-15T17:56:26.184384+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184384+0000","last_undegraded":"2024-01-15T17:56:26.184384+0000","last_fullsized":"2024-01-15T17:56:26.184384+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.195931+0000","last_change":"2024-01-15T17:56:26.195931+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.195931+0000","last_undegraded":"2024-01-15T17:56:26.195931+0000","last_fullsized":"2024-01-15T17:56:26.195931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185022+0000","last_change":"2024-01-15T17:56:26.185022+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185022+0000","last_undegraded":"2024-01-15T17:56:26.185022+0000","last_fullsized":"2024-01-15T17:56:26.185022+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189153+0000","last_change":"2024-01-15T17:56:26.189153+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189153+0000","last_undegraded":"2024-01-15T17:56:26.189153+0000","last_fullsized":"2024-01-15T17:56:26.189153+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188820+0000","last_change":"2024-01-15T17:56:26.188820+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188820+0000","last_undegraded":"2024-01-15T17:56:26.188820+0000","last_fullsized":"2024-01-15T17:56:26.188820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.192671+0000","last_change":"2024-01-15T17:56:26.192671+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.192671+0000","last_undegraded":"2024-01-15T17:56:26.192671+0000","last_fullsized":"2024-01-15T17:56:26.192671+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.197878+0000","last_change":"2024-01-15T17:56:26.197878+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.197878+0000","last_undegraded":"2024-01-15T17:56:26.197878+0000","last_fullsized":"2024-01-15T17:56:26.197878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.205932+0000","last_change":"2024-01-15T17:56:26.205932+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.205932+0000","last_undegraded":"2024-01-15T17:56:26.205932+0000","last_fullsized":"2024-01-15T17:56:26.205932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190196+0000","last_change":"2024-01-15T17:56:26.190196+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190196+0000","last_undegraded":"2024-01-15T17:56:26.190196+0000","last_fullsized":"2024-01-15T17:56:26.190196+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190603+0000","last_change":"2024-01-15T17:56:26.190603+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190603+0000","last_undegraded":"2024-01-15T17:56:26.190603+0000","last_fullsized":"2024-01-15T17:56:26.190603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202815+0000","last_change":"2024-01-15T17:56:26.202815+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202815+0000","last_undegraded":"2024-01-15T17:56:26.202815+0000","last_fullsized":"2024-01-15T17:56:26.202815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190684+0000","last_change":"2024-01-15T17:56:26.190684+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190684+0000","last_undegraded":"2024-01-15T17:56:26.190684+0000","last_fullsized":"2024-01-15T17:56:26.190684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188271+0000","last_change":"2024-01-15T17:56:26.188271+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188271+0000","last_undegraded":"2024-01-15T17:56:26.188271+0000","last_fullsized":"2024-01-15T17:56:26.188271+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190649+0000","last_change":"2024-01-15T17:56:26.190649+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190649+0000","last_undegraded":"2024-01-15T17:56:26.190649+0000","last_fullsized":"2024-01-15T17:56:26.190649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181879+0000","last_change":"2024-01-15T17:56:26.181879+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181879+0000","last_undegraded":"2024-01-15T17:56:26.181879+0000","last_fullsized":"2024-01-15T17:56:26.181879+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.197426+0000","last_change":"2024-01-15T17:56:26.197426+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.197426+0000","last_undegraded":"2024-01-15T17:56:26.197426+0000","last_fullsized":"2024-01-15T17:56:26.197426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202504+0000","last_change":"2024-01-15T17:56:26.202504+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202504+0000","last_undegraded":"2024-01-15T17:56:26.202504+0000","last_fullsized":"2024-01-15T17:56:26.202504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.192203+0000","last_change":"2024-01-15T17:56:26.192203+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.192203+0000","last_undegraded":"2024-01-15T17:56:26.192203+0000","last_fullsized":"2024-01-15T17:56:26.192203+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193807+0000","last_change":"2024-01-15T17:56:26.193807+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193807+0000","last_undegraded":"2024-01-15T17:56:26.193807+0000","last_fullsized":"2024-01-15T17:56:26.193807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184195+0000","last_change":"2024-01-15T17:56:26.184195+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184195+0000","last_undegraded":"2024-01-15T17:56:26.184195+0000","last_fullsized":"2024-01-15T17:56:26.184195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.204185+0000","last_change":"2024-01-15T17:56:26.204185+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.204185+0000","last_undegraded":"2024-01-15T17:56:26.204185+0000","last_fullsized":"2024-01-15T17:56:26.204185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189019+0000","last_change":"2024-01-15T17:56:26.189019+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189019+0000","last_undegraded":"2024-01-15T17:56:26.189019+0000","last_fullsized":"2024-01-15T17:56:26.189019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184645+0000","last_change":"2024-01-15T17:56:26.184645+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184645+0000","last_undegraded":"2024-01-15T17:56:26.184645+0000","last_fullsized":"2024-01-15T17:56:26.184645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.182486+0000","last_change":"2024-01-15T17:56:26.182486+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.182486+0000","last_undegraded":"2024-01-15T17:56:26.182486+0000","last_fullsized":"2024-01-15T17:56:26.182486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193649+0000","last_change":"2024-01-15T17:56:26.193649+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193649+0000","last_undegraded":"2024-01-15T17:56:26.193649+0000","last_fullsized":"2024-01-15T17:56:26.193649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191687+0000","last_change":"2024-01-15T17:56:26.191687+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191687+0000","last_undegraded":"2024-01-15T17:56:26.191687+0000","last_fullsized":"2024-01-15T17:56:26.191687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193341+0000","last_change":"2024-01-15T17:56:26.193341+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193341+0000","last_undegraded":"2024-01-15T17:56:26.193341+0000","last_fullsized":"2024-01-15T17:56:26.193341+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185835+0000","last_change":"2024-01-15T17:56:26.185835+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185835+0000","last_undegraded":"2024-01-15T17:56:26.185835+0000","last_fullsized":"2024-01-15T17:56:26.185835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.196524+0000","last_change":"2024-01-15T17:56:26.196524+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.196524+0000","last_undegraded":"2024-01-15T17:56:26.196524+0000","last_fullsized":"2024-01-15T17:56:26.196524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202508+0000","last_change":"2024-01-15T17:56:26.202508+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202508+0000","last_undegraded":"2024-01-15T17:56:26.202508+0000","last_fullsized":"2024-01-15T17:56:26.202508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183698+0000","last_change":"2024-01-15T17:56:26.183698+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183698+0000","last_undegraded":"2024-01-15T17:56:26.183698+0000","last_fullsized":"2024-01-15T17:56:26.183698+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.205444+0000","last_change":"2024-01-15T17:56:26.205444+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.205444+0000","last_undegraded":"2024-01-15T17:56:26.205444+0000","last_fullsized":"2024-01-15T17:56:26.205444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185930+0000","last_change":"2024-01-15T17:56:26.185930+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185930+0000","last_undegraded":"2024-01-15T17:56:26.185930+0000","last_fullsized":"2024-01-15T17:56:26.185930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207378+0000","last_change":"2024-01-15T17:56:26.202527+0000","last_active":"2024-01-15T17:56:26.207378+0000","last_peered":"2024-01-15T17:56:26.207378+0000","last_clean":"2024-01-15T17:56:26.207378+0000","last_became_active":"2024-01-15T17:56:26.202254+0000","last_became_peered":"2024-01-15T17:56:26.202254+0000","last_unstale":"2024-01-15T17:56:26.207378+0000","last_undegraded":"2024-01-15T17:56:26.207378+0000","last_fullsized":"2024-01-15T17:56:26.207378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:58:49.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183024+0000","last_change":"2024-01-15T17:56:26.183024+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183024+0000","last_undegraded":"2024-01-15T17:56:26.183024+0000","last_fullsized":"2024-01-15T17:56:26.183024+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.214222+0000","last_change":"2024-01-15T17:56:26.213479+0000","last_active":"2024-01-15T17:56:26.214222+0000","last_peered":"2024-01-15T17:56:26.214222+0000","last_clean":"2024-01-15T17:56:26.214222+0000","last_became_active":"2024-01-15T17:56:26.213165+0000","last_became_peered":"2024-01-15T17:56:26.213165+0000","last_unstale":"2024-01-15T17:56:26.214222+0000","last_undegraded":"2024-01-15T17:56:26.214222+0000","last_fullsized":"2024-01-15T17:56:26.214222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:43.670278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191397+0000","last_change":"2024-01-15T17:56:26.191397+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191397+0000","last_undegraded":"2024-01-15T17:56:26.191397+0000","last_fullsized":"2024-01-15T17:56:26.191397+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207053+0000","last_change":"2024-01-15T17:56:26.199957+0000","last_active":"2024-01-15T17:56:26.207053+0000","last_peered":"2024-01-15T17:56:26.207053+0000","last_clean":"2024-01-15T17:56:26.207053+0000","last_became_active":"2024-01-15T17:56:26.199632+0000","last_became_peered":"2024-01-15T17:56:26.199632+0000","last_unstale":"2024-01-15T17:56:26.207053+0000","last_undegraded":"2024-01-15T17:56:26.207053+0000","last_fullsized":"2024-01-15T17:56:26.207053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:01:25.558146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190219+0000","last_change":"2024-01-15T17:56:26.190219+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190219+0000","last_undegraded":"2024-01-15T17:56:26.190219+0000","last_fullsized":"2024-01-15T17:56:26.190219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.211037+0000","last_change":"2024-01-15T17:56:26.209481+0000","last_active":"2024-01-15T17:56:26.211037+0000","last_peered":"2024-01-15T17:56:26.211037+0000","last_clean":"2024-01-15T17:56:26.211037+0000","last_became_active":"2024-01-15T17:56:26.209251+0000","last_became_peered":"2024-01-15T17:56:26.209251+0000","last_unstale":"2024-01-15T17:56:26.211037+0000","last_undegraded":"2024-01-15T17:56:26.211037+0000","last_fullsized":"2024-01-15T17:56:26.211037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:13:34.770667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.196248+0000","last_change":"2024-01-15T17:56:26.196248+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.196248+0000","last_undegraded":"2024-01-15T17:56:26.196248+0000","last_fullsized":"2024-01-15T17:56:26.196248+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.211475+0000","last_change":"2024-01-15T17:56:26.209952+0000","last_active":"2024-01-15T17:56:26.211475+0000","last_peered":"2024-01-15T17:56:26.211475+0000","last_clean":"2024-01-15T17:56:26.211475+0000","last_became_active":"2024-01-15T17:56:26.209696+0000","last_became_peered":"2024-01-15T17:56:26.209696+0000","last_unstale":"2024-01-15T17:56:26.211475+0000","last_undegraded":"2024-01-15T17:56:26.211475+0000","last_fullsized":"2024-01-15T17:56:26.211475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:41.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181863+0000","last_change":"2024-01-15T17:56:26.181863+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181863+0000","last_undegraded":"2024-01-15T17:56:26.181863+0000","last_fullsized":"2024-01-15T17:56:26.181863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213022+0000","last_change":"2024-01-15T17:56:26.211397+0000","last_active":"2024-01-15T17:56:26.213022+0000","last_peered":"2024-01-15T17:56:26.213022+0000","last_clean":"2024-01-15T17:56:26.213022+0000","last_became_active":"2024-01-15T17:56:26.210076+0000","last_became_peered":"2024-01-15T17:56:26.210076+0000","last_unstale":"2024-01-15T17:56:26.213022+0000","last_undegraded":"2024-01-15T17:56:26.213022+0000","last_fullsized":"2024-01-15T17:56:26.213022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:31:26.304443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188872+0000","last_change":"2024-01-15T17:56:26.188872+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188872+0000","last_undegraded":"2024-01-15T17:56:26.188872+0000","last_fullsized":"2024-01-15T17:56:26.188872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207337+0000","last_change":"2024-01-15T17:56:26.203486+0000","last_active":"2024-01-15T17:56:26.207337+0000","last_peered":"2024-01-15T17:56:26.207337+0000","last_clean":"2024-01-15T17:56:26.207337+0000","last_became_active":"2024-01-15T17:56:26.203141+0000","last_became_peered":"2024-01-15T17:56:26.203141+0000","last_unstale":"2024-01-15T17:56:26.207337+0000","last_undegraded":"2024-01-15T17:56:26.207337+0000","last_fullsized":"2024-01-15T17:56:26.207337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:28:33.523943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183545+0000","last_change":"2024-01-15T17:56:26.183545+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183545+0000","last_undegraded":"2024-01-15T17:56:26.183545+0000","last_fullsized":"2024-01-15T17:56:26.183545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.203978+0000","last_change":"2024-01-15T17:56:26.200676+0000","last_active":"2024-01-15T17:56:26.203978+0000","last_peered":"2024-01-15T17:56:26.203978+0000","last_clean":"2024-01-15T17:56:26.203978+0000","last_became_active":"2024-01-15T17:56:26.199502+0000","last_became_peered":"2024-01-15T17:56:26.199502+0000","last_unstale":"2024-01-15T17:56:26.203978+0000","last_undegraded":"2024-01-15T17:56:26.203978+0000","last_fullsized":"2024-01-15T17:56:26.203978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:17.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185090+0000","last_change":"2024-01-15T17:56:26.185090+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185090+0000","last_undegraded":"2024-01-15T17:56:26.185090+0000","last_fullsized":"2024-01-15T17:56:26.185090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213865+0000","last_change":"2024-01-15T17:56:26.212810+0000","last_active":"2024-01-15T17:56:26.213865+0000","last_peered":"2024-01-15T17:56:26.213865+0000","last_clean":"2024-01-15T17:56:26.213865+0000","last_became_active":"2024-01-15T17:56:26.212482+0000","last_became_peered":"2024-01-15T17:56:26.212482+0000","last_unstale":"2024-01-15T17:56:26.213865+0000","last_undegraded":"2024-01-15T17:56:26.213865+0000","last_fullsized":"2024-01-15T17:56:26.213865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:26.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188955+0000","last_change":"2024-01-15T17:56:26.188955+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188955+0000","last_undegraded":"2024-01-15T17:56:26.188955+0000","last_fullsized":"2024-01-15T17:56:26.188955+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210403+0000","last_change":"2024-01-15T17:56:26.208053+0000","last_active":"2024-01-15T17:56:26.210403+0000","last_peered":"2024-01-15T17:56:26.210403+0000","last_clean":"2024-01-15T17:56:26.210403+0000","last_became_active":"2024-01-15T17:56:26.207220+0000","last_became_peered":"2024-01-15T17:56:26.207220+0000","last_unstale":"2024-01-15T17:56:26.210403+0000","last_undegraded":"2024-01-15T17:56:26.210403+0000","last_fullsized":"2024-01-15T17:56:26.210403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:31.045653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188235+0000","last_change":"2024-01-15T17:56:26.188235+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188235+0000","last_undegraded":"2024-01-15T17:56:26.188235+0000","last_fullsized":"2024-01-15T17:56:26.188235+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.215003+0000","last_change":"2024-01-15T17:56:26.213932+0000","last_active":"2024-01-15T17:56:26.215003+0000","last_peered":"2024-01-15T17:56:26.215003+0000","last_clean":"2024-01-15T17:56:26.215003+0000","last_became_active":"2024-01-15T17:56:26.213653+0000","last_became_peered":"2024-01-15T17:56:26.213653+0000","last_unstale":"2024-01-15T17:56:26.215003+0000","last_undegraded":"2024-01-15T17:56:26.215003+0000","last_fullsized":"2024-01-15T17:56:26.215003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:46.243898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188570+0000","last_change":"2024-01-15T17:56:26.188570+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188570+0000","last_undegraded":"2024-01-15T17:56:26.188570+0000","last_fullsized":"2024-01-15T17:56:26.188570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.209612+0000","last_change":"2024-01-15T17:56:26.203678+0000","last_active":"2024-01-15T17:56:26.209612+0000","last_peered":"2024-01-15T17:56:26.209612+0000","last_clean":"2024-01-15T17:56:26.209612+0000","last_became_active":"2024-01-15T17:56:26.203341+0000","last_became_peered":"2024-01-15T17:56:26.203341+0000","last_unstale":"2024-01-15T17:56:26.209612+0000","last_undegraded":"2024-01-15T17:56:26.209612+0000","last_fullsized":"2024-01-15T17:56:26.209612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:14:31.052513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184920+0000","last_change":"2024-01-15T17:56:26.184920+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184920+0000","last_undegraded":"2024-01-15T17:56:26.184920+0000","last_fullsized":"2024-01-15T17:56:26.184920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212414+0000","last_change":"2024-01-15T17:56:26.210551+0000","last_active":"2024-01-15T17:56:26.212414+0000","last_peered":"2024-01-15T17:56:26.212414+0000","last_clean":"2024-01-15T17:56:26.212414+0000","last_became_active":"2024-01-15T17:56:26.209399+0000","last_became_peered":"2024-01-15T17:56:26.209399+0000","last_unstale":"2024-01-15T17:56:26.212414+0000","last_undegraded":"2024-01-15T17:56:26.212414+0000","last_fullsized":"2024-01-15T17:56:26.212414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:12:05.549396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185817+0000","last_change":"2024-01-15T17:56:26.185817+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185817+0000","last_undegraded":"2024-01-15T17:56:26.185817+0000","last_fullsized":"2024-01-15T17:56:26.185817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208828+0000","last_change":"2024-01-15T17:56:26.201105+0000","last_active":"2024-01-15T17:56:26.208828+0000","last_peered":"2024-01-15T17:56:26.208828+0000","last_clean":"2024-01-15T17:56:26.208828+0000","last_became_active":"2024-01-15T17:56:26.200818+0000","last_became_peered":"2024-01-15T17:56:26.200818+0000","last_unstale":"2024-01-15T17:56:26.208828+0000","last_undegraded":"2024-01-15T17:56:26.208828+0000","last_fullsized":"2024-01-15T17:56:26.208828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:58:49.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.199505+0000","last_change":"2024-01-15T17:56:26.199505+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.199505+0000","last_undegraded":"2024-01-15T17:56:26.199505+0000","last_fullsized":"2024-01-15T17:56:26.199505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212669+0000","last_change":"2024-01-15T17:56:26.211092+0000","last_active":"2024-01-15T17:56:26.212669+0000","last_peered":"2024-01-15T17:56:26.212669+0000","last_clean":"2024-01-15T17:56:26.212669+0000","last_became_active":"2024-01-15T17:56:26.210804+0000","last_became_peered":"2024-01-15T17:56:26.210804+0000","last_unstale":"2024-01-15T17:56:26.212669+0000","last_undegraded":"2024-01-15T17:56:26.212669+0000","last_fullsized":"2024-01-15T17:56:26.212669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:02:16.155632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181524+0000","last_change":"2024-01-15T17:56:26.181524+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181524+0000","last_undegraded":"2024-01-15T17:56:26.181524+0000","last_fullsized":"2024-01-15T17:56:26.181524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207189+0000","last_change":"2024-01-15T17:56:26.201849+0000","last_active":"2024-01-15T17:56:26.207189+0000","last_peered":"2024-01-15T17:56:26.207189+0000","last_clean":"2024-01-15T17:56:26.207189+0000","last_became_active":"2024-01-15T17:56:26.201558+0000","last_became_peered":"2024-01-15T17:56:26.201558+0000","last_unstale":"2024-01-15T17:56:26.207189+0000","last_undegraded":"2024-01-15T17:56:26.207189+0000","last_fullsized":"2024-01-15T17:56:26.207189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:43.277391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190247+0000","last_change":"2024-01-15T17:56:26.190247+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190247+0000","last_undegraded":"2024-01-15T17:56:26.190247+0000","last_fullsized":"2024-01-15T17:56:26.190247+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213314+0000","last_change":"2024-01-15T17:56:26.211508+0000","last_active":"2024-01-15T17:56:26.213314+0000","last_peered":"2024-01-15T17:56:26.213314+0000","last_clean":"2024-01-15T17:56:26.213314+0000","last_became_active":"2024-01-15T17:56:26.207011+0000","last_became_peered":"2024-01-15T17:56:26.207011+0000","last_unstale":"2024-01-15T17:56:26.213314+0000","last_undegraded":"2024-01-15T17:56:26.213314+0000","last_fullsized":"2024-01-15T17:56:26.213314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:40.566415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191272+0000","last_change":"2024-01-15T17:56:26.191272+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191272+0000","last_undegraded":"2024-01-15T17:56:26.191272+0000","last_fullsized":"2024-01-15T17:56:26.191272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.203619+0000","last_change":"2024-01-15T17:56:26.200302+0000","last_active":"2024-01-15T17:56:26.203619+0000","last_peered":"2024-01-15T17:56:26.203619+0000","last_clean":"2024-01-15T17:56:26.203619+0000","last_became_active":"2024-01-15T17:56:26.200001+0000","last_became_peered":"2024-01-15T17:56:26.200001+0000","last_unstale":"2024-01-15T17:56:26.203619+0000","last_undegraded":"2024-01-15T17:56:26.203619+0000","last_fullsized":"2024-01-15T17:56:26.203619+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:06:34.072861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187437+0000","last_change":"2024-01-15T17:56:26.187437+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187437+0000","last_undegraded":"2024-01-15T17:56:26.187437+0000","last_fullsized":"2024-01-15T17:56:26.187437+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210901+0000","last_change":"2024-01-15T17:56:26.207943+0000","last_active":"2024-01-15T17:56:26.210901+0000","last_peered":"2024-01-15T17:56:26.210901+0000","last_clean":"2024-01-15T17:56:26.210901+0000","last_became_active":"2024-01-15T17:56:26.207604+0000","last_became_peered":"2024-01-15T17:56:26.207604+0000","last_unstale":"2024-01-15T17:56:26.210901+0000","last_undegraded":"2024-01-15T17:56:26.210901+0000","last_fullsized":"2024-01-15T17:56:26.210901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:42.102953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187383+0000","last_change":"2024-01-15T17:56:26.187383+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187383+0000","last_undegraded":"2024-01-15T17:56:26.187383+0000","last_fullsized":"2024-01-15T17:56:26.187383+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210215+0000","last_change":"2024-01-15T17:56:26.207505+0000","last_active":"2024-01-15T17:56:26.210215+0000","last_peered":"2024-01-15T17:56:26.210215+0000","last_clean":"2024-01-15T17:56:26.210215+0000","last_became_active":"2024-01-15T17:56:26.207210+0000","last_became_peered":"2024-01-15T17:56:26.207210+0000","last_unstale":"2024-01-15T17:56:26.210215+0000","last_undegraded":"2024-01-15T17:56:26.210215+0000","last_fullsized":"2024-01-15T17:56:26.210215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:13:34.770667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183722+0000","last_change":"2024-01-15T17:56:26.183722+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183722+0000","last_undegraded":"2024-01-15T17:56:26.183722+0000","last_fullsized":"2024-01-15T17:56:26.183722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.205077+0000","last_change":"2024-01-15T17:56:26.202274+0000","last_active":"2024-01-15T17:56:26.205077+0000","last_peered":"2024-01-15T17:56:26.205077+0000","last_clean":"2024-01-15T17:56:26.205077+0000","last_became_active":"2024-01-15T17:56:26.201924+0000","last_became_peered":"2024-01-15T17:56:26.201924+0000","last_unstale":"2024-01-15T17:56:26.205077+0000","last_undegraded":"2024-01-15T17:56:26.205077+0000","last_fullsized":"2024-01-15T17:56:26.205077+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:35.925710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183777+0000","last_change":"2024-01-15T17:56:26.183777+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183777+0000","last_undegraded":"2024-01-15T17:56:26.183777+0000","last_fullsized":"2024-01-15T17:56:26.183777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.205313+0000","last_change":"2024-01-15T17:56:26.200301+0000","last_active":"2024-01-15T17:56:26.205313+0000","last_peered":"2024-01-15T17:56:26.205313+0000","last_clean":"2024-01-15T17:56:26.205313+0000","last_became_active":"2024-01-15T17:56:26.200003+0000","last_became_peered":"2024-01-15T17:56:26.200003+0000","last_unstale":"2024-01-15T17:56:26.205313+0000","last_undegraded":"2024-01-15T17:56:26.205313+0000","last_fullsized":"2024-01-15T17:56:26.205313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:02:29.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.186611+0000","last_change":"2024-01-15T17:56:26.186611+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.186611+0000","last_undegraded":"2024-01-15T17:56:26.186611+0000","last_fullsized":"2024-01-15T17:56:26.186611+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.215099+0000","last_change":"2024-01-15T17:56:26.214358+0000","last_active":"2024-01-15T17:56:26.215099+0000","last_peered":"2024-01-15T17:56:26.215099+0000","last_clean":"2024-01-15T17:56:26.215099+0000","last_became_active":"2024-01-15T17:56:26.213452+0000","last_became_peered":"2024-01-15T17:56:26.213452+0000","last_unstale":"2024-01-15T17:56:26.215099+0000","last_undegraded":"2024-01-15T17:56:26.215099+0000","last_fullsized":"2024-01-15T17:56:26.215099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:05:49.120775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.198507+0000","last_change":"2024-01-15T17:56:26.198507+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.198507+0000","last_undegraded":"2024-01-15T17:56:26.198507+0000","last_fullsized":"2024-01-15T17:56:26.198507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212927+0000","last_change":"2024-01-15T17:56:26.212040+0000","last_active":"2024-01-15T17:56:26.212927+0000","last_peered":"2024-01-15T17:56:26.212927+0000","last_clean":"2024-01-15T17:56:26.212927+0000","last_became_active":"2024-01-15T17:56:26.211151+0000","last_became_peered":"2024-01-15T17:56:26.211151+0000","last_unstale":"2024-01-15T17:56:26.212927+0000","last_undegraded":"2024-01-15T17:56:26.212927+0000","last_fullsized":"2024-01-15T17:56:26.212927+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T17:58:44.566548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185596+0000","last_change":"2024-01-15T17:56:26.185596+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185596+0000","last_undegraded":"2024-01-15T17:56:26.185596+0000","last_fullsized":"2024-01-15T17:56:26.185596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210500+0000","last_change":"2024-01-15T17:56:26.201506+0000","last_active":"2024-01-15T17:56:26.210500+0000","last_peered":"2024-01-15T17:56:26.210500+0000","last_clean":"2024-01-15T17:56:26.210500+0000","last_became_active":"2024-01-15T17:56:26.201268+0000","last_became_peered":"2024-01-15T17:56:26.201268+0000","last_unstale":"2024-01-15T17:56:26.210500+0000","last_undegraded":"2024-01-15T17:56:26.210500+0000","last_fullsized":"2024-01-15T17:56:26.210500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:43.670278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190043+0000","last_change":"2024-01-15T17:56:26.190043+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190043+0000","last_undegraded":"2024-01-15T17:56:26.190043+0000","last_fullsized":"2024-01-15T17:56:26.190043+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.202533+0000","last_change":"2024-01-15T17:56:26.200072+0000","last_active":"2024-01-15T17:56:26.202533+0000","last_peered":"2024-01-15T17:56:26.202533+0000","last_clean":"2024-01-15T17:56:26.202533+0000","last_became_active":"2024-01-15T17:56:26.199764+0000","last_became_peered":"2024-01-15T17:56:26.199764+0000","last_unstale":"2024-01-15T17:56:26.202533+0000","last_undegraded":"2024-01-15T17:56:26.202533+0000","last_fullsized":"2024-01-15T17:56:26.202533+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189595+0000","last_change":"2024-01-15T17:56:26.189595+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189595+0000","last_undegraded":"2024-01-15T17:56:26.189595+0000","last_fullsized":"2024-01-15T17:56:26.189595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208497+0000","last_change":"2024-01-15T17:56:26.200854+0000","last_active":"2024-01-15T17:56:26.208497+0000","last_peered":"2024-01-15T17:56:26.208497+0000","last_clean":"2024-01-15T17:56:26.208497+0000","last_became_active":"2024-01-15T17:56:26.200317+0000","last_became_peered":"2024-01-15T17:56:26.200317+0000","last_unstale":"2024-01-15T17:56:26.208497+0000","last_undegraded":"2024-01-15T17:56:26.208497+0000","last_fullsized":"2024-01-15T17:56:26.208497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181422+0000","last_change":"2024-01-15T17:56:26.181422+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181422+0000","last_undegraded":"2024-01-15T17:56:26.181422+0000","last_fullsized":"2024-01-15T17:56:26.181422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208623+0000","last_change":"2024-01-15T17:56:26.205649+0000","last_active":"2024-01-15T17:56:26.208623+0000","last_peered":"2024-01-15T17:56:26.208623+0000","last_clean":"2024-01-15T17:56:26.208623+0000","last_became_active":"2024-01-15T17:56:26.205337+0000","last_became_peered":"2024-01-15T17:56:26.205337+0000","last_unstale":"2024-01-15T17:56:26.208623+0000","last_undegraded":"2024-01-15T17:56:26.208623+0000","last_fullsized":"2024-01-15T17:56:26.208623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:40.566415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187391+0000","last_change":"2024-01-15T17:56:26.187391+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187391+0000","last_undegraded":"2024-01-15T17:56:26.187391+0000","last_fullsized":"2024-01-15T17:56:26.187391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208848+0000","last_change":"2024-01-15T17:56:26.201883+0000","last_active":"2024-01-15T17:56:26.208848+0000","last_peered":"2024-01-15T17:56:26.208848+0000","last_clean":"2024-01-15T17:56:26.208848+0000","last_became_active":"2024-01-15T17:56:26.201275+0000","last_became_peered":"2024-01-15T17:56:26.201275+0000","last_unstale":"2024-01-15T17:56:26.208848+0000","last_undegraded":"2024-01-15T17:56:26.208848+0000","last_fullsized":"2024-01-15T17:56:26.208848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:02:29.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.182677+0000","last_change":"2024-01-15T17:56:26.182677+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.182677+0000","last_undegraded":"2024-01-15T17:56:26.182677+0000","last_fullsized":"2024-01-15T17:56:26.182677+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207896+0000","last_change":"2024-01-15T17:56:26.205102+0000","last_active":"2024-01-15T17:56:26.207896+0000","last_peered":"2024-01-15T17:56:26.207896+0000","last_clean":"2024-01-15T17:56:26.207896+0000","last_became_active":"2024-01-15T17:56:26.204747+0000","last_became_peered":"2024-01-15T17:56:26.204747+0000","last_unstale":"2024-01-15T17:56:26.207896+0000","last_undegraded":"2024-01-15T17:56:26.207896+0000","last_fullsized":"2024-01-15T17:56:26.207896+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:48.459125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.195525+0000","last_change":"2024-01-15T17:56:26.195525+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.195525+0000","last_undegraded":"2024-01-15T17:56:26.195525+0000","last_fullsized":"2024-01-15T17:56:26.195525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212667+0000","last_change":"2024-01-15T17:56:26.211169+0000","last_active":"2024-01-15T17:56:26.212667+0000","last_peered":"2024-01-15T17:56:26.212667+0000","last_clean":"2024-01-15T17:56:26.212667+0000","last_became_active":"2024-01-15T17:56:26.210842+0000","last_became_peered":"2024-01-15T17:56:26.210842+0000","last_unstale":"2024-01-15T17:56:26.212667+0000","last_undegraded":"2024-01-15T17:56:26.212667+0000","last_fullsized":"2024-01-15T17:56:26.212667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:25.400140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.201814+0000","last_change":"2024-01-15T17:56:26.201814+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.201814+0000","last_undegraded":"2024-01-15T17:56:26.201814+0000","last_fullsized":"2024-01-15T17:56:26.201814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210656+0000","last_change":"2024-01-15T17:56:26.208654+0000","last_active":"2024-01-15T17:56:26.210656+0000","last_peered":"2024-01-15T17:56:26.210656+0000","last_clean":"2024-01-15T17:56:26.210656+0000","last_became_active":"2024-01-15T17:56:26.208305+0000","last_became_peered":"2024-01-15T17:56:26.208305+0000","last_unstale":"2024-01-15T17:56:26.210656+0000","last_undegraded":"2024-01-15T17:56:26.210656+0000","last_fullsized":"2024-01-15T17:56:26.210656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":172,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-01-15T17:56:25.185666+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:56:25.185666+0000","last_peered":"2024-01-15T17:56:25.185666+0000","last_clean":"2024-01-15T17:56:25.185666+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:56:25.185666+0000","last_undegraded":"2024-01-15T17:56:25.185666+0000","last_fullsized":"2024-01-15T17:56:25.185666+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":962560,"data_stored":732013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.643}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.373},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.534}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.28},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.081}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.181},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.052,"5min":1.052,"15min":1.052},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.2330000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.444}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.49},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.1339999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.714,"15min":1.714},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.325},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.611}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":0.876}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.155},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.2569999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.3280000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.417},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1140000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.605},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.4810000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.758,"5min":1.758,"15min":1.758},"min":{"1min":1.202,"5min":1.202,"15min":1.202},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.5209999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.212},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"min":{"1min":1.286,"5min":1.286,"15min":1.286},"max":{"1min":3.141,"5min":3.141,"15min":3.141},"last":1.456}]}]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.3840000000000003,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.22},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":5.2290000000000001,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.3440000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.625}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.835,"5min":2.835,"15min":2.835},"last":1.546},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.413}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.264}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.419,"15min":1.419},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.5720000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.083,"5min":1.083,"15min":1.083},"max":{"1min":5.218,"5min":5.218,"15min":5.218},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.512}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":1.2,"5min":1.2,"15min":1.2},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.4430000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.996},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0740000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.597},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.127}]},{"osd":2,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.339},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.383}]},{"osd":3,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.4410000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.536},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"min":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.794}]},{"osd":6,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":1.706,"15min":1.706},"min":{"1min":1.117,"5min":1.117,"15min":1.117},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.742},{"interface":"front","average":{"1min":1.845,"5min":1.845,"15min":1.845},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.8520000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"min":{"1min":1.385,"5min":1.385,"15min":1.385},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.899},{"interface":"front","average":{"1min":1.929,"5min":1.929,"15min":1.929},"min":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":1.929}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.2989999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.4410000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.238}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.353}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.643},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":1.167}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.673,"15min":1.673},"min":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.5820000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.698,"15min":1.698},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"min":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.538}]}]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.208}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.135}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":4.008,"5min":4.008,"15min":4.008},"last":1.278},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.0289999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.171}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.6070000000000002,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.389},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.0529999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.028},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.61199999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1990000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.23,"15min":2.23},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":22.106999999999999,"5min":22.106999999999999,"15min":22.106999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.0740000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":9.8510000000000009,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.272},{"interface":"front","average":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":14.359999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.113}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":13.228999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":4.9900000000000002,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":1.4039999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":14.272,"5min":14.272,"15min":14.272},"last":1.357},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.1659999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.474}]}]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":962560,"data_stored":732013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.032}]},{"osd":2,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.496},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.93300000000000005}]},{"osd":3,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.597},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.542}]},{"osd":4,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.401},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.6619999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.6419999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.9990000000000001,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":1.792},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.274}]},{"osd":7,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.8100000000000001,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.857},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.74}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:56:32.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-01-15T17:56:32.762 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.184384+0000', 'last_change': '2024-01-15T17:56:26.184384+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.184384+0000', 'last_undegraded': '2024-01-15T17:56:26.184384+0000', 'last_fullsized': '2024-01-15T17:56:26.184384+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T17:56:32.762 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.195931+0000', 'last_change': '2024-01-15T17:56:26.195931+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.195931+0000', 'last_undegraded': '2024-01-15T17:56:26.195931+0000', 'last_fullsized': '2024-01-15T17:56:26.195931+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.185022+0000', 'last_change': '2024-01-15T17:56:26.185022+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.185022+0000', 'last_undegraded': '2024-01-15T17:56:26.185022+0000', 'last_fullsized': '2024-01-15T17:56:26.185022+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.189153+0000', 'last_change': '2024-01-15T17:56:26.189153+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.189153+0000', 'last_undegraded': '2024-01-15T17:56:26.189153+0000', 'last_fullsized': '2024-01-15T17:56:26.189153+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.188820+0000', 'last_change': '2024-01-15T17:56:26.188820+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.188820+0000', 'last_undegraded': '2024-01-15T17:56:26.188820+0000', 'last_fullsized': '2024-01-15T17:56:26.188820+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.192671+0000', 'last_change': '2024-01-15T17:56:26.192671+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.192671+0000', 'last_undegraded': '2024-01-15T17:56:26.192671+0000', 'last_fullsized': '2024-01-15T17:56:26.192671+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.763 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.197878+0000', 'last_change': '2024-01-15T17:56:26.197878+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.197878+0000', 'last_undegraded': '2024-01-15T17:56:26.197878+0000', 'last_fullsized': '2024-01-15T17:56:26.197878+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.205932+0000', 'last_change': '2024-01-15T17:56:26.205932+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.205932+0000', 'last_undegraded': '2024-01-15T17:56:26.205932+0000', 'last_fullsized': '2024-01-15T17:56:26.205932+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.190196+0000', 'last_change': '2024-01-15T17:56:26.190196+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.190196+0000', 'last_undegraded': '2024-01-15T17:56:26.190196+0000', 'last_fullsized': '2024-01-15T17:56:26.190196+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.190603+0000', 'last_change': '2024-01-15T17:56:26.190603+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.190603+0000', 'last_undegraded': '2024-01-15T17:56:26.190603+0000', 'last_fullsized': '2024-01-15T17:56:26.190603+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.202815+0000', 'last_change': '2024-01-15T17:56:26.202815+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.202815+0000', 'last_undegraded': '2024-01-15T17:56:26.202815+0000', 'last_fullsized': '2024-01-15T17:56:26.202815+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.764 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.190684+0000', 'last_change': '2024-01-15T17:56:26.190684+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.190684+0000', 'last_undegraded': '2024-01-15T17:56:26.190684+0000', 'last_fullsized': '2024-01-15T17:56:26.190684+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.188271+0000', 'last_change': '2024-01-15T17:56:26.188271+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.188271+0000', 'last_undegraded': '2024-01-15T17:56:26.188271+0000', 'last_fullsized': '2024-01-15T17:56:26.188271+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.190649+0000', 'last_change': '2024-01-15T17:56:26.190649+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.190649+0000', 'last_undegraded': '2024-01-15T17:56:26.190649+0000', 'last_fullsized': '2024-01-15T17:56:26.190649+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.181879+0000', 'last_change': '2024-01-15T17:56:26.181879+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.181879+0000', 'last_undegraded': '2024-01-15T17:56:26.181879+0000', 'last_fullsized': '2024-01-15T17:56:26.181879+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.197426+0000', 'last_change': '2024-01-15T17:56:26.197426+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.197426+0000', 'last_undegraded': '2024-01-15T17:56:26.197426+0000', 'last_fullsized': '2024-01-15T17:56:26.197426+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.765 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.202504+0000', 'last_change': '2024-01-15T17:56:26.202504+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.202504+0000', 'last_undegraded': '2024-01-15T17:56:26.202504+0000', 'last_fullsized': '2024-01-15T17:56:26.202504+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.192203+0000', 'last_change': '2024-01-15T17:56:26.192203+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.192203+0000', 'last_undegraded': '2024-01-15T17:56:26.192203+0000', 'last_fullsized': '2024-01-15T17:56:26.192203+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.193807+0000', 'last_change': '2024-01-15T17:56:26.193807+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.193807+0000', 'last_undegraded': '2024-01-15T17:56:26.193807+0000', 'last_fullsized': '2024-01-15T17:56:26.193807+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.184195+0000', 'last_change': '2024-01-15T17:56:26.184195+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.184195+0000', 'last_undegraded': '2024-01-15T17:56:26.184195+0000', 'last_fullsized': '2024-01-15T17:56:26.184195+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.204185+0000', 'last_change': '2024-01-15T17:56:26.204185+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.204185+0000', 'last_undegraded': '2024-01-15T17:56:26.204185+0000', 'last_fullsized': '2024-01-15T17:56:26.204185+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-01-15T17:56:32.766 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.189019+0000', 'last_change': '2024-01-15T17:56:26.189019+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.189019+0000', 'last_undegraded': '2024-01-15T17:56:26.189019+0000', 'last_fullsized': '2024-01-15T17:56:26.189019+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.184645+0000', 'last_change': '2024-01-15T17:56:26.184645+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.184645+0000', 'last_undegraded': '2024-01-15T17:56:26.184645+0000', 'last_fullsized': '2024-01-15T17:56:26.184645+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.182486+0000', 'last_change': '2024-01-15T17:56:26.182486+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.182486+0000', 'last_undegraded': '2024-01-15T17:56:26.182486+0000', 'last_fullsized': '2024-01-15T17:56:26.182486+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.193649+0000', 'last_change': '2024-01-15T17:56:26.193649+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.193649+0000', 'last_undegraded': '2024-01-15T17:56:26.193649+0000', 'last_fullsized': '2024-01-15T17:56:26.193649+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.191687+0000', 'last_change': '2024-01-15T17:56:26.191687+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.191687+0000', 'last_undegraded': '2024-01-15T17:56:26.191687+0000', 'last_fullsized': '2024-01-15T17:56:26.191687+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-01-15T17:56:32.767 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.193341+0000', 'last_change': '2024-01-15T17:56:26.193341+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.193341+0000', 'last_undegraded': '2024-01-15T17:56:26.193341+0000', 'last_fullsized': '2024-01-15T17:56:26.193341+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T17:56:32.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-01-15T17:56:32.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.185835+0000', 'last_change': '2024-01-15T17:56:26.185835+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.185835+0000', 'last_undegraded': '2024-01-15T17:56:26.185835+0000', 'last_fullsized': '2024-01-15T17:56:26.185835+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-01-15T17:56:32.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.196524+0000', 'last_change': '2024-01-15T17:56:26.196524+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.196524+0000', 'last_undegraded': '2024-01-15T17:56:26.196524+0000', 'last_fullsized': '2024-01-15T17:56:26.196524+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T17:56:32.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-01-15T17:56:32.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.202508+0000', 'last_change': '2024-01-15T17:56:26.202508+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.202508+0000', 'last_undegraded': '2024-01-15T17:56:26.202508+0000', 'last_fullsized': '2024-01-15T17:56:26.202508+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.768 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-01-15T17:56:32.768 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.183698+0000', 'last_change': '2024-01-15T17:56:26.183698+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.183698+0000', 'last_undegraded': '2024-01-15T17:56:26.183698+0000', 'last_fullsized': '2024-01-15T17:56:26.183698+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.205444+0000', 'last_change': '2024-01-15T17:56:26.205444+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.205444+0000', 'last_undegraded': '2024-01-15T17:56:26.205444+0000', 'last_fullsized': '2024-01-15T17:56:26.205444+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.185930+0000', 'last_change': '2024-01-15T17:56:26.185930+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.185930+0000', 'last_undegraded': '2024-01-15T17:56:26.185930+0000', 'last_fullsized': '2024-01-15T17:56:26.185930+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.183024+0000', 'last_change': '2024-01-15T17:56:26.183024+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.183024+0000', 'last_undegraded': '2024-01-15T17:56:26.183024+0000', 'last_fullsized': '2024-01-15T17:56:26.183024+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.191397+0000', 'last_change': '2024-01-15T17:56:26.191397+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.191397+0000', 'last_undegraded': '2024-01-15T17:56:26.191397+0000', 'last_fullsized': '2024-01-15T17:56:26.191397+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.190219+0000', 'last_change': '2024-01-15T17:56:26.190219+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.190219+0000', 'last_undegraded': '2024-01-15T17:56:26.190219+0000', 'last_fullsized': '2024-01-15T17:56:26.190219+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T17:56:32.769 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.196248+0000', 'last_change': '2024-01-15T17:56:26.196248+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.196248+0000', 'last_undegraded': '2024-01-15T17:56:26.196248+0000', 'last_fullsized': '2024-01-15T17:56:26.196248+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.181863+0000', 'last_change': '2024-01-15T17:56:26.181863+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.181863+0000', 'last_undegraded': '2024-01-15T17:56:26.181863+0000', 'last_fullsized': '2024-01-15T17:56:26.181863+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.188872+0000', 'last_change': '2024-01-15T17:56:26.188872+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.188872+0000', 'last_undegraded': '2024-01-15T17:56:26.188872+0000', 'last_fullsized': '2024-01-15T17:56:26.188872+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.183545+0000', 'last_change': '2024-01-15T17:56:26.183545+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.183545+0000', 'last_undegraded': '2024-01-15T17:56:26.183545+0000', 'last_fullsized': '2024-01-15T17:56:26.183545+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.185090+0000', 'last_change': '2024-01-15T17:56:26.185090+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.185090+0000', 'last_undegraded': '2024-01-15T17:56:26.185090+0000', 'last_fullsized': '2024-01-15T17:56:26.185090+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.770 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.188955+0000', 'last_change': '2024-01-15T17:56:26.188955+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.188955+0000', 'last_undegraded': '2024-01-15T17:56:26.188955+0000', 'last_fullsized': '2024-01-15T17:56:26.188955+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.188235+0000', 'last_change': '2024-01-15T17:56:26.188235+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.188235+0000', 'last_undegraded': '2024-01-15T17:56:26.188235+0000', 'last_fullsized': '2024-01-15T17:56:26.188235+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.188570+0000', 'last_change': '2024-01-15T17:56:26.188570+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.188570+0000', 'last_undegraded': '2024-01-15T17:56:26.188570+0000', 'last_fullsized': '2024-01-15T17:56:26.188570+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.184920+0000', 'last_change': '2024-01-15T17:56:26.184920+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.184920+0000', 'last_undegraded': '2024-01-15T17:56:26.184920+0000', 'last_fullsized': '2024-01-15T17:56:26.184920+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.185817+0000', 'last_change': '2024-01-15T17:56:26.185817+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.185817+0000', 'last_undegraded': '2024-01-15T17:56:26.185817+0000', 'last_fullsized': '2024-01-15T17:56:26.185817+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-01-15T17:56:32.771 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.199505+0000', 'last_change': '2024-01-15T17:56:26.199505+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.199505+0000', 'last_undegraded': '2024-01-15T17:56:26.199505+0000', 'last_fullsized': '2024-01-15T17:56:26.199505+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-01-15T17:56:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.181524+0000', 'last_change': '2024-01-15T17:56:26.181524+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.181524+0000', 'last_undegraded': '2024-01-15T17:56:26.181524+0000', 'last_fullsized': '2024-01-15T17:56:26.181524+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T17:56:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-01-15T17:56:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.190247+0000', 'last_change': '2024-01-15T17:56:26.190247+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.190247+0000', 'last_undegraded': '2024-01-15T17:56:26.190247+0000', 'last_fullsized': '2024-01-15T17:56:26.190247+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T17:56:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-01-15T17:56:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.191272+0000', 'last_change': '2024-01-15T17:56:26.191272+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.191272+0000', 'last_undegraded': '2024-01-15T17:56:26.191272+0000', 'last_fullsized': '2024-01-15T17:56:26.191272+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-01-15T17:56:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.187437+0000', 'last_change': '2024-01-15T17:56:26.187437+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.187437+0000', 'last_undegraded': '2024-01-15T17:56:26.187437+0000', 'last_fullsized': '2024-01-15T17:56:26.187437+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T17:56:32.772 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.187383+0000', 'last_change': '2024-01-15T17:56:26.187383+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.187383+0000', 'last_undegraded': '2024-01-15T17:56:26.187383+0000', 'last_fullsized': '2024-01-15T17:56:26.187383+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.183722+0000', 'last_change': '2024-01-15T17:56:26.183722+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.183722+0000', 'last_undegraded': '2024-01-15T17:56:26.183722+0000', 'last_fullsized': '2024-01-15T17:56:26.183722+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.183777+0000', 'last_change': '2024-01-15T17:56:26.183777+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.183777+0000', 'last_undegraded': '2024-01-15T17:56:26.183777+0000', 'last_fullsized': '2024-01-15T17:56:26.183777+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.186611+0000', 'last_change': '2024-01-15T17:56:26.186611+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.186611+0000', 'last_undegraded': '2024-01-15T17:56:26.186611+0000', 'last_fullsized': '2024-01-15T17:56:26.186611+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.198507+0000', 'last_change': '2024-01-15T17:56:26.198507+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.198507+0000', 'last_undegraded': '2024-01-15T17:56:26.198507+0000', 'last_fullsized': '2024-01-15T17:56:26.198507+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-01-15T17:56:32.773 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.185596+0000', 'last_change': '2024-01-15T17:56:26.185596+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.185596+0000', 'last_undegraded': '2024-01-15T17:56:26.185596+0000', 'last_fullsized': '2024-01-15T17:56:26.185596+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.190043+0000', 'last_change': '2024-01-15T17:56:26.190043+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.190043+0000', 'last_undegraded': '2024-01-15T17:56:26.190043+0000', 'last_fullsized': '2024-01-15T17:56:26.190043+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.189595+0000', 'last_change': '2024-01-15T17:56:26.189595+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.189595+0000', 'last_undegraded': '2024-01-15T17:56:26.189595+0000', 'last_fullsized': '2024-01-15T17:56:26.189595+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.181422+0000', 'last_change': '2024-01-15T17:56:26.181422+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.181422+0000', 'last_undegraded': '2024-01-15T17:56:26.181422+0000', 'last_fullsized': '2024-01-15T17:56:26.181422+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.187391+0000', 'last_change': '2024-01-15T17:56:26.187391+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.187391+0000', 'last_undegraded': '2024-01-15T17:56:26.187391+0000', 'last_fullsized': '2024-01-15T17:56:26.187391+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.182677+0000', 'last_change': '2024-01-15T17:56:26.182677+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.182677+0000', 'last_undegraded': '2024-01-15T17:56:26.182677+0000', 'last_fullsized': '2024-01-15T17:56:26.182677+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-01-15T17:56:32.774 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.195525+0000', 'last_change': '2024-01-15T17:56:26.195525+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.195525+0000', 'last_undegraded': '2024-01-15T17:56:26.195525+0000', 'last_fullsized': '2024-01-15T17:56:26.195525+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T17:56:32.775 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-01-15T17:56:32.775 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-01-15T17:56:26.201814+0000', 'last_change': '2024-01-15T17:56:26.201814+0000', 'last_active': '2024-01-15T17:56:26.164505+0000', 'last_peered': '2024-01-15T17:56:26.164505+0000', 'last_clean': '2024-01-15T17:56:26.164505+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T17:56:26.201814+0000', 'last_undegraded': '2024-01-15T17:56:26.201814+0000', 'last_fullsized': '2024-01-15T17:56:26.201814+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'last_clean_scrub_stamp': '2024-01-15T17:56:26.164505+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T17:56:32.775 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-15T17:56:33.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.023+0000 7f0acaa00640 1 Processor -- start 2024-01-15T17:56:33.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.023+0000 7f0acaa00640 1 -- start start 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.024+0000 7f0acaa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0ac40cc500 0x7f0ac40cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.024+0000 7f0acaa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0ac406d160 0x7f0ac406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.025+0000 7f0acaa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac4070e50 con 0x7f0ac4143c40 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.025+0000 7f0acaa00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac40723c0 con 0x7f0ac40cc500 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.025+0000 7f0acaa00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac40cced0 con 0x7f0ac406d160 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.025+0000 7f0ac9600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0ac4143c40 0x7f0ac4071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39794/0 (socket says 172.21.15.26:39794) 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.025+0000 7f0ac9600640 1 -- 172.21.15.26:0/2831283143 learned_addr learned my addr 172.21.15.26:0/2831283143 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.025+0000 7f0ac8c00640 1 --2- 172.21.15.26:0/2831283143 >> v2:172.21.15.26:3300/0 conn(0x7f0ac40cc500 0x7f0ac40cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.025+0000 7f0aca000640 1 --2- 172.21.15.26:0/2831283143 >> v2:172.21.15.124:3300/0 conn(0x7f0ac406d160 0x7f0ac406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.025+0000 7f0ac3e00640 1 -- 172.21.15.26:0/2831283143 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421417343 0 0) 0x7f0ac4070e50 con 0x7f0ac4143c40 2024-01-15T17:56:33.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.025+0000 7f0ac3e00640 1 -- 172.21.15.26:0/2831283143 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa4003520 con 0x7f0ac4143c40 2024-01-15T17:56:33.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.026+0000 7f0ac8c00640 1 -- 172.21.15.26:0/2831283143 >> v2:172.21.15.124:3300/0 conn(0x7f0ac406d160 msgr2=0x7f0ac406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:33.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.026+0000 7f0ac8c00640 1 --2- 172.21.15.26:0/2831283143 >> v2:172.21.15.124:3300/0 conn(0x7f0ac406d160 0x7f0ac406d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.026+0000 7f0ac8c00640 1 -- 172.21.15.26:0/2831283143 >> v1:172.21.15.26:6789/0 conn(0x7f0ac4143c40 legacy=0x7f0ac4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.026+0000 7f0ac8c00640 1 -- 172.21.15.26:0/2831283143 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac4144020 con 0x7f0ac40cc500 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.027+0000 7f0ac8c00640 1 --2- 172.21.15.26:0/2831283143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac40cc500 0x7f0ac40cc990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4070690 tx=0x7f0aac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aab8347d9d7e39 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.027+0000 7f0ac3e00640 1 -- 172.21.15.26:0/2831283143 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac02e070 con 0x7f0ac40cc500 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.027+0000 7f0ac3e00640 1 -- 172.21.15.26:0/2831283143 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac0315a0 con 0x7f0ac40cc500 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.027+0000 7f0ac3e00640 1 -- 172.21.15.26:0/2831283143 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac031930 con 0x7f0ac40cc500 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.028+0000 7f0acaa00640 1 -- 172.21.15.26:0/2831283143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac40cc500 msgr2=0x7f0ac40cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.028+0000 7f0acaa00640 1 --2- 172.21.15.26:0/2831283143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac40cc500 0x7f0ac40cc990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4070690 tx=0x7f0aac02b840 comp rx=0 tx=0).stop 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.028+0000 7f0acaa00640 1 -- 172.21.15.26:0/2831283143 shutdown_connections 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.028+0000 7f0acaa00640 1 --2- 172.21.15.26:0/2831283143 >> v2:172.21.15.124:3300/0 conn(0x7f0ac406d160 0x7f0ac406d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.028+0000 7f0acaa00640 1 --2- 172.21.15.26:0/2831283143 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac40cc500 0x7f0ac40cc990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.028+0000 7f0acaa00640 1 -- 172.21.15.26:0/2831283143 >> 172.21.15.26:0/2831283143 conn(0x7f0ac406a800 msgr2=0x7f0ac406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:33.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.028+0000 7f0acaa00640 1 -- 172.21.15.26:0/2831283143 shutdown_connections 2024-01-15T17:56:33.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.028+0000 7f0acaa00640 1 -- 172.21.15.26:0/2831283143 wait complete. 2024-01-15T17:56:33.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.030+0000 7f0acaa00640 1 Processor -- start 2024-01-15T17:56:33.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.030+0000 7f0acaa00640 1 -- start start 2024-01-15T17:56:33.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.031+0000 7f0acaa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ac406d160 0x7f0ac41f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.031+0000 7f0acaa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ac40cc500 0x7f0ac40b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.031+0000 7f0acaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac4143c40 0x7f0ac40b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.031+0000 7f0acaa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac40c5530 con 0x7f0ac4143c40 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.031+0000 7f0acaa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac40cb030 con 0x7f0ac406d160 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.031+0000 7f0acaa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac40cb310 con 0x7f0ac40cc500 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.031+0000 7f0aca000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac4143c40 0x7f0ac40b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.031+0000 7f0aca000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac4143c40 0x7f0ac40b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47772/0 (socket says 172.21.15.26:47772) 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.031+0000 7f0aca000640 1 -- 172.21.15.26:0/4205308577 learned_addr learned my addr 172.21.15.26:0/4205308577 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0ac9600640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ac406d160 0x7f0ac41f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0ac8c00640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ac40cc500 0x7f0ac40b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0aca000640 1 -- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ac40cc500 msgr2=0x7f0ac40b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0aca000640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ac40cc500 0x7f0ac40b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0aca000640 1 -- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ac406d160 msgr2=0x7f0ac41f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0aca000640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ac406d160 0x7f0ac41f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0aca000640 1 -- 172.21.15.26:0/4205308577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac009400 con 0x7f0ac4143c40 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0ac9600640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ac406d160 0x7f0ac41f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0aca000640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac4143c40 0x7f0ac40b3490 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0abc007e40 tx=0x7f0abc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0ac1600640 1 -- 172.21.15.26:0/4205308577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc014620 con 0x7f0ac4143c40 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0ac1600640 1 -- 172.21.15.26:0/4205308577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0abc00ee90 con 0x7f0ac4143c40 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0ac1600640 1 -- 172.21.15.26:0/4205308577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0abc0149e0 con 0x7f0ac4143c40 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.032+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac40c5740 con 0x7f0ac4143c40 2024-01-15T17:56:33.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.033+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac41f60c0 con 0x7f0ac4143c40 2024-01-15T17:56:33.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.034+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac4067450 con 0x7f0ac4143c40 2024-01-15T17:56:33.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.034+0000 7f0ac1600640 1 -- 172.21.15.26:0/4205308577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f0abc014b40 con 0x7f0ac4143c40 2024-01-15T17:56:33.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.035+0000 7f0ac1600640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a9c06b2f0 0x7f0a9c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.035+0000 7f0ac1600640 1 -- 172.21.15.26:0/4205308577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0abc08a580 con 0x7f0ac4143c40 2024-01-15T17:56:33.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.036+0000 7f0ac9600640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a9c06b2f0 0x7f0a9c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.037+0000 7f0ac9600640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a9c06b2f0 0x7f0a9c06d7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40011e0 tx=0x7f0ab4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:33.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.040+0000 7f0ac1600640 1 -- 172.21.15.26:0/4205308577 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0abc0553f0 con 0x7f0ac4143c40 2024-01-15T17:56:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.310+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0ac40cb4d0 con 0x7f0ac4143c40 2024-01-15T17:56:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.311+0000 7f0ac1600640 1 -- 172.21.15.26:0/4205308577 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1034 (secure 0 0 0) 0x7f0abc058a10 con 0x7f0ac4143c40 2024-01-15T17:56:33.314 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:33.314 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":107,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1705341296,"num_in_osds":8,"osd_in_since":1705341288,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":242532352,"bytes_avail":772851580928,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":34619}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-01-15T17:56:09.933010+0000","services":{}},"progress_events":{}} 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a9c06b2f0 msgr2=0x7f0a9c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a9c06b2f0 0x7f0a9c06d7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40011e0 tx=0x7f0ab4009040 comp rx=0 tx=0).stop 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac4143c40 msgr2=0x7f0ac40b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac4143c40 0x7f0ac40b3490 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0abc007e40 tx=0x7f0abc00e4b0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 shutdown_connections 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ac40cc500 0x7f0ac40b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a9c06b2f0 0x7f0a9c06d7b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ac4143c40 0x7f0ac40b3490 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 --2- 172.21.15.26:0/4205308577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ac406d160 0x7f0ac41f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.313+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 >> 172.21.15.26:0/4205308577 conn(0x7f0ac406a800 msgr2=0x7f0ac406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.314+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 shutdown_connections 2024-01-15T17:56:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.314+0000 7f0acaa00640 1 -- 172.21.15.26:0/4205308577 wait complete. 2024-01-15T17:56:33.336 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-15T17:56:33.336 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:56:33.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.558+0000 7f1c45a00640 1 Processor -- start 2024-01-15T17:56:33.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.558+0000 7f1c45a00640 1 -- start start 2024-01-15T17:56:33.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.560+0000 7f1c45a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1c40071410 0x7f1c40146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.560+0000 7f1c45a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1c4006c7c0 0x7f1c400717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.560+0000 7f1c45a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4006cc50 con 0x7f1c40071d30 2024-01-15T17:56:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.560+0000 7f1c45a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4006cec0 con 0x7f1c40071410 2024-01-15T17:56:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.560+0000 7f1c45a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4006d1a0 con 0x7f1c4006c7c0 2024-01-15T17:56:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.560+0000 7f1c45000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1c40071d30 0x7f1c400b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39796/0 (socket says 172.21.15.26:39796) 2024-01-15T17:56:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.561+0000 7f1c45000640 1 -- 172.21.15.26:0/600231474 learned_addr learned my addr 172.21.15.26:0/600231474 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.560+0000 7f1c3fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1c40071410 0x7f1c40146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.561+0000 7f1c37e00640 1 --2- 172.21.15.26:0/600231474 >> v2:172.21.15.124:3300/0 conn(0x7f1c4006c7c0 0x7f1c400717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.561+0000 7f1c3f400640 1 -- 172.21.15.26:0/600231474 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580195794 0 0) 0x7f1c4006cc50 con 0x7f1c40071d30 2024-01-15T17:56:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.561+0000 7f1c3fe00640 1 -- 172.21.15.26:0/600231474 >> v2:172.21.15.124:3300/0 conn(0x7f1c4006c7c0 msgr2=0x7f1c400717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.561+0000 7f1c3fe00640 1 --2- 172.21.15.26:0/600231474 >> v2:172.21.15.124:3300/0 conn(0x7f1c4006c7c0 0x7f1c400717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.561+0000 7f1c3fe00640 1 -- 172.21.15.26:0/600231474 >> v1:172.21.15.26:6789/0 conn(0x7f1c40071d30 legacy=0x7f1c400b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.561+0000 7f1c3fe00640 1 -- 172.21.15.26:0/600231474 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4006d480 con 0x7f1c40071410 2024-01-15T17:56:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.562+0000 7f1c3fe00640 1 --2- 172.21.15.26:0/600231474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 0x7f1c40146770 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1c40146ef0 tx=0x7f1c3002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bcd4a5936984af server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.563+0000 7f1c3f400640 1 -- 172.21.15.26:0/600231474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30031830 con 0x7f1c40071410 2024-01-15T17:56:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.563+0000 7f1c3f400640 1 -- 172.21.15.26:0/600231474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c30031990 con 0x7f1c40071410 2024-01-15T17:56:33.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.563+0000 7f1c3f400640 1 -- 172.21.15.26:0/600231474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30031d20 con 0x7f1c40071410 2024-01-15T17:56:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.563+0000 7f1c45a00640 1 -- 172.21.15.26:0/600231474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 msgr2=0x7f1c40146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.563+0000 7f1c45a00640 1 --2- 172.21.15.26:0/600231474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 0x7f1c40146770 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1c40146ef0 tx=0x7f1c3002b8d0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.564+0000 7f1c45a00640 1 -- 172.21.15.26:0/600231474 shutdown_connections 2024-01-15T17:56:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.564+0000 7f1c45a00640 1 --2- 172.21.15.26:0/600231474 >> v2:172.21.15.124:3300/0 conn(0x7f1c4006c7c0 0x7f1c400717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.564+0000 7f1c45a00640 1 --2- 172.21.15.26:0/600231474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 0x7f1c40146770 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.564+0000 7f1c45a00640 1 -- 172.21.15.26:0/600231474 >> 172.21.15.26:0/600231474 conn(0x7f1c4006a800 msgr2=0x7f1c4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.564+0000 7f1c45a00640 1 -- 172.21.15.26:0/600231474 shutdown_connections 2024-01-15T17:56:33.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.564+0000 7f1c45a00640 1 -- 172.21.15.26:0/600231474 wait complete. 2024-01-15T17:56:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.565+0000 7f1c45a00640 1 Processor -- start 2024-01-15T17:56:33.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.566+0000 7f1c45a00640 1 -- start start 2024-01-15T17:56:33.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c45a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c4006c7c0 0x7f1c400c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c45a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 0x7f1c401f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c45a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c40071d30 0x7f1c401f5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c45a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015d050 con 0x7f1c40071410 2024-01-15T17:56:33.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c45a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c401f3630 con 0x7f1c4006c7c0 2024-01-15T17:56:33.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c45a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c401f59c0 con 0x7f1c40071d30 2024-01-15T17:56:33.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 0x7f1c401f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 0x7f1c401f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47786/0 (socket says 172.21.15.26:47786) 2024-01-15T17:56:33.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c37e00640 1 -- 172.21.15.26:0/3323798721 learned_addr learned my addr 172.21.15.26:0/3323798721 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c3fe00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c4006c7c0 0x7f1c400c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c45000640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c40071d30 0x7f1c401f5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c37e00640 1 -- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c40071d30 msgr2=0x7f1c401f5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c37e00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c40071d30 0x7f1c401f5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c37e00640 1 -- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c4006c7c0 msgr2=0x7f1c400c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c37e00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c4006c7c0 0x7f1c400c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.567+0000 7f1c37e00640 1 -- 172.21.15.26:0/3323798721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c30009400 con 0x7f1c40071410 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.568+0000 7f1c37e00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 0x7f1c401f30f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800d620 tx=0x7f1c2800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.568+0000 7f1c3cc00640 1 -- 172.21.15.26:0/3323798721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800b620 con 0x7f1c40071410 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.568+0000 7f1c3cc00640 1 -- 172.21.15.26:0/3323798721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c2800b780 con 0x7f1c40071410 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.568+0000 7f1c3cc00640 1 -- 172.21.15.26:0/3323798721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2800bb30 con 0x7f1c40071410 2024-01-15T17:56:33.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.568+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c400c4350 con 0x7f1c40071410 2024-01-15T17:56:33.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.568+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c400c4910 con 0x7f1c40071410 2024-01-15T17:56:33.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.569+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c401f3830 con 0x7f1c40071410 2024-01-15T17:56:33.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.570+0000 7f1c3cc00640 1 -- 172.21.15.26:0/3323798721 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f1c2800bc90 con 0x7f1c40071410 2024-01-15T17:56:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.571+0000 7f1c3cc00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1806b2c0 0x7f1c1806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:33.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.571+0000 7f1c3cc00640 1 -- 172.21.15.26:0/3323798721 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c2808a140 con 0x7f1c40071410 2024-01-15T17:56:33.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.572+0000 7f1c3fe00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1806b2c0 0x7f1c1806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:33.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.573+0000 7f1c3fe00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1806b2c0 0x7f1c1806d780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c30036000 tx=0x7f1c30008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:33.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.576+0000 7f1c3cc00640 1 -- 172.21.15.26:0/3323798721 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c401f3830 con 0x7f1c40071410 2024-01-15T17:56:33.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.767+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c40072180 con 0x7f1c1806b2c0 2024-01-15T17:56:33.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.790+0000 7f1c3cc00640 1 -- 172.21.15.26:0/3323798721 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280833 (secure 0 0 0) 0x7f1c30031630 con 0x7f1c1806b2c0 2024-01-15T17:56:33.797 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:33.798 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:56:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1806b2c0 msgr2=0x7f1c1806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1806b2c0 0x7f1c1806d780 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1c30036000 tx=0x7f1c30008030 comp rx=0 tx=0).stop 2024-01-15T17:56:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 msgr2=0x7f1c401f30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 0x7f1c401f30f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800d620 tx=0x7f1c2800dae0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 shutdown_connections 2024-01-15T17:56:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c40071d30 0x7f1c401f5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1806b2c0 0x7f1c1806d780 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c40071410 0x7f1c401f30f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 --2- 172.21.15.26:0/3323798721 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c4006c7c0 0x7f1c400c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:33.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 >> 172.21.15.26:0/3323798721 conn(0x7f1c4006a800 msgr2=0x7f1c400b01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:33.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 shutdown_connections 2024-01-15T17:56:33.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:33.795+0000 7f1c45a00640 1 -- 172.21.15.26:0/3323798721 wait complete. 2024-01-15T17:56:33.817 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-01-15T17:56:31.939478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236848,"kb_used_data":4028,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737872,"statfs":{"total":773094113280,"available":772851580928,"internally_reserved":0,"allocated":4124672,"data_stored":2307629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001737"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184384+0000","last_change":"2024-01-15T17:56:26.184384+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184384+0000","last_undegraded":"2024-01-15T17:56:26.184384+0000","last_fullsized":"2024-01-15T17:56:26.184384+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.195931+0000","last_change":"2024-01-15T17:56:26.195931+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.195931+0000","last_undegraded":"2024-01-15T17:56:26.195931+0000","last_fullsized":"2024-01-15T17:56:26.195931+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185022+0000","last_change":"2024-01-15T17:56:26.185022+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185022+0000","last_undegraded":"2024-01-15T17:56:26.185022+0000","last_fullsized":"2024-01-15T17:56:26.185022+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189153+0000","last_change":"2024-01-15T17:56:26.189153+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189153+0000","last_undegraded":"2024-01-15T17:56:26.189153+0000","last_fullsized":"2024-01-15T17:56:26.189153+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188820+0000","last_change":"2024-01-15T17:56:26.188820+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188820+0000","last_undegraded":"2024-01-15T17:56:26.188820+0000","last_fullsized":"2024-01-15T17:56:26.188820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.192671+0000","last_change":"2024-01-15T17:56:26.192671+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.192671+0000","last_undegraded":"2024-01-15T17:56:26.192671+0000","last_fullsized":"2024-01-15T17:56:26.192671+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.197878+0000","last_change":"2024-01-15T17:56:26.197878+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.197878+0000","last_undegraded":"2024-01-15T17:56:26.197878+0000","last_fullsized":"2024-01-15T17:56:26.197878+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.205932+0000","last_change":"2024-01-15T17:56:26.205932+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.205932+0000","last_undegraded":"2024-01-15T17:56:26.205932+0000","last_fullsized":"2024-01-15T17:56:26.205932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190196+0000","last_change":"2024-01-15T17:56:26.190196+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190196+0000","last_undegraded":"2024-01-15T17:56:26.190196+0000","last_fullsized":"2024-01-15T17:56:26.190196+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190603+0000","last_change":"2024-01-15T17:56:26.190603+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190603+0000","last_undegraded":"2024-01-15T17:56:26.190603+0000","last_fullsized":"2024-01-15T17:56:26.190603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202815+0000","last_change":"2024-01-15T17:56:26.202815+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202815+0000","last_undegraded":"2024-01-15T17:56:26.202815+0000","last_fullsized":"2024-01-15T17:56:26.202815+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190684+0000","last_change":"2024-01-15T17:56:26.190684+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190684+0000","last_undegraded":"2024-01-15T17:56:26.190684+0000","last_fullsized":"2024-01-15T17:56:26.190684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188271+0000","last_change":"2024-01-15T17:56:26.188271+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188271+0000","last_undegraded":"2024-01-15T17:56:26.188271+0000","last_fullsized":"2024-01-15T17:56:26.188271+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190649+0000","last_change":"2024-01-15T17:56:26.190649+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190649+0000","last_undegraded":"2024-01-15T17:56:26.190649+0000","last_fullsized":"2024-01-15T17:56:26.190649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181879+0000","last_change":"2024-01-15T17:56:26.181879+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181879+0000","last_undegraded":"2024-01-15T17:56:26.181879+0000","last_fullsized":"2024-01-15T17:56:26.181879+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.197426+0000","last_change":"2024-01-15T17:56:26.197426+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.197426+0000","last_undegraded":"2024-01-15T17:56:26.197426+0000","last_fullsized":"2024-01-15T17:56:26.197426+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202504+0000","last_change":"2024-01-15T17:56:26.202504+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202504+0000","last_undegraded":"2024-01-15T17:56:26.202504+0000","last_fullsized":"2024-01-15T17:56:26.202504+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.192203+0000","last_change":"2024-01-15T17:56:26.192203+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.192203+0000","last_undegraded":"2024-01-15T17:56:26.192203+0000","last_fullsized":"2024-01-15T17:56:26.192203+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193807+0000","last_change":"2024-01-15T17:56:26.193807+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193807+0000","last_undegraded":"2024-01-15T17:56:26.193807+0000","last_fullsized":"2024-01-15T17:56:26.193807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184195+0000","last_change":"2024-01-15T17:56:26.184195+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184195+0000","last_undegraded":"2024-01-15T17:56:26.184195+0000","last_fullsized":"2024-01-15T17:56:26.184195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.204185+0000","last_change":"2024-01-15T17:56:26.204185+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.204185+0000","last_undegraded":"2024-01-15T17:56:26.204185+0000","last_fullsized":"2024-01-15T17:56:26.204185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189019+0000","last_change":"2024-01-15T17:56:26.189019+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189019+0000","last_undegraded":"2024-01-15T17:56:26.189019+0000","last_fullsized":"2024-01-15T17:56:26.189019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184645+0000","last_change":"2024-01-15T17:56:26.184645+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184645+0000","last_undegraded":"2024-01-15T17:56:26.184645+0000","last_fullsized":"2024-01-15T17:56:26.184645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.182486+0000","last_change":"2024-01-15T17:56:26.182486+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.182486+0000","last_undegraded":"2024-01-15T17:56:26.182486+0000","last_fullsized":"2024-01-15T17:56:26.182486+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193649+0000","last_change":"2024-01-15T17:56:26.193649+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193649+0000","last_undegraded":"2024-01-15T17:56:26.193649+0000","last_fullsized":"2024-01-15T17:56:26.193649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191687+0000","last_change":"2024-01-15T17:56:26.191687+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191687+0000","last_undegraded":"2024-01-15T17:56:26.191687+0000","last_fullsized":"2024-01-15T17:56:26.191687+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.193341+0000","last_change":"2024-01-15T17:56:26.193341+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.193341+0000","last_undegraded":"2024-01-15T17:56:26.193341+0000","last_fullsized":"2024-01-15T17:56:26.193341+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185835+0000","last_change":"2024-01-15T17:56:26.185835+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185835+0000","last_undegraded":"2024-01-15T17:56:26.185835+0000","last_fullsized":"2024-01-15T17:56:26.185835+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.196524+0000","last_change":"2024-01-15T17:56:26.196524+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.196524+0000","last_undegraded":"2024-01-15T17:56:26.196524+0000","last_fullsized":"2024-01-15T17:56:26.196524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.202508+0000","last_change":"2024-01-15T17:56:26.202508+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.202508+0000","last_undegraded":"2024-01-15T17:56:26.202508+0000","last_fullsized":"2024-01-15T17:56:26.202508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183698+0000","last_change":"2024-01-15T17:56:26.183698+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183698+0000","last_undegraded":"2024-01-15T17:56:26.183698+0000","last_fullsized":"2024-01-15T17:56:26.183698+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.205444+0000","last_change":"2024-01-15T17:56:26.205444+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.205444+0000","last_undegraded":"2024-01-15T17:56:26.205444+0000","last_fullsized":"2024-01-15T17:56:26.205444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185930+0000","last_change":"2024-01-15T17:56:26.185930+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185930+0000","last_undegraded":"2024-01-15T17:56:26.185930+0000","last_fullsized":"2024-01-15T17:56:26.185930+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207378+0000","last_change":"2024-01-15T17:56:26.202527+0000","last_active":"2024-01-15T17:56:26.207378+0000","last_peered":"2024-01-15T17:56:26.207378+0000","last_clean":"2024-01-15T17:56:26.207378+0000","last_became_active":"2024-01-15T17:56:26.202254+0000","last_became_peered":"2024-01-15T17:56:26.202254+0000","last_unstale":"2024-01-15T17:56:26.207378+0000","last_undegraded":"2024-01-15T17:56:26.207378+0000","last_fullsized":"2024-01-15T17:56:26.207378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:58:49.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183024+0000","last_change":"2024-01-15T17:56:26.183024+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183024+0000","last_undegraded":"2024-01-15T17:56:26.183024+0000","last_fullsized":"2024-01-15T17:56:26.183024+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.214222+0000","last_change":"2024-01-15T17:56:26.213479+0000","last_active":"2024-01-15T17:56:26.214222+0000","last_peered":"2024-01-15T17:56:26.214222+0000","last_clean":"2024-01-15T17:56:26.214222+0000","last_became_active":"2024-01-15T17:56:26.213165+0000","last_became_peered":"2024-01-15T17:56:26.213165+0000","last_unstale":"2024-01-15T17:56:26.214222+0000","last_undegraded":"2024-01-15T17:56:26.214222+0000","last_fullsized":"2024-01-15T17:56:26.214222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:43.670278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191397+0000","last_change":"2024-01-15T17:56:26.191397+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191397+0000","last_undegraded":"2024-01-15T17:56:26.191397+0000","last_fullsized":"2024-01-15T17:56:26.191397+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207053+0000","last_change":"2024-01-15T17:56:26.199957+0000","last_active":"2024-01-15T17:56:26.207053+0000","last_peered":"2024-01-15T17:56:26.207053+0000","last_clean":"2024-01-15T17:56:26.207053+0000","last_became_active":"2024-01-15T17:56:26.199632+0000","last_became_peered":"2024-01-15T17:56:26.199632+0000","last_unstale":"2024-01-15T17:56:26.207053+0000","last_undegraded":"2024-01-15T17:56:26.207053+0000","last_fullsized":"2024-01-15T17:56:26.207053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:01:25.558146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190219+0000","last_change":"2024-01-15T17:56:26.190219+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190219+0000","last_undegraded":"2024-01-15T17:56:26.190219+0000","last_fullsized":"2024-01-15T17:56:26.190219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.211037+0000","last_change":"2024-01-15T17:56:26.209481+0000","last_active":"2024-01-15T17:56:26.211037+0000","last_peered":"2024-01-15T17:56:26.211037+0000","last_clean":"2024-01-15T17:56:26.211037+0000","last_became_active":"2024-01-15T17:56:26.209251+0000","last_became_peered":"2024-01-15T17:56:26.209251+0000","last_unstale":"2024-01-15T17:56:26.211037+0000","last_undegraded":"2024-01-15T17:56:26.211037+0000","last_fullsized":"2024-01-15T17:56:26.211037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:13:34.770667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.196248+0000","last_change":"2024-01-15T17:56:26.196248+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.196248+0000","last_undegraded":"2024-01-15T17:56:26.196248+0000","last_fullsized":"2024-01-15T17:56:26.196248+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.211475+0000","last_change":"2024-01-15T17:56:26.209952+0000","last_active":"2024-01-15T17:56:26.211475+0000","last_peered":"2024-01-15T17:56:26.211475+0000","last_clean":"2024-01-15T17:56:26.211475+0000","last_became_active":"2024-01-15T17:56:26.209696+0000","last_became_peered":"2024-01-15T17:56:26.209696+0000","last_unstale":"2024-01-15T17:56:26.211475+0000","last_undegraded":"2024-01-15T17:56:26.211475+0000","last_fullsized":"2024-01-15T17:56:26.211475+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:41.012171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181863+0000","last_change":"2024-01-15T17:56:26.181863+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181863+0000","last_undegraded":"2024-01-15T17:56:26.181863+0000","last_fullsized":"2024-01-15T17:56:26.181863+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213022+0000","last_change":"2024-01-15T17:56:26.211397+0000","last_active":"2024-01-15T17:56:26.213022+0000","last_peered":"2024-01-15T17:56:26.213022+0000","last_clean":"2024-01-15T17:56:26.213022+0000","last_became_active":"2024-01-15T17:56:26.210076+0000","last_became_peered":"2024-01-15T17:56:26.210076+0000","last_unstale":"2024-01-15T17:56:26.213022+0000","last_undegraded":"2024-01-15T17:56:26.213022+0000","last_fullsized":"2024-01-15T17:56:26.213022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:31:26.304443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188872+0000","last_change":"2024-01-15T17:56:26.188872+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188872+0000","last_undegraded":"2024-01-15T17:56:26.188872+0000","last_fullsized":"2024-01-15T17:56:26.188872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207337+0000","last_change":"2024-01-15T17:56:26.203486+0000","last_active":"2024-01-15T17:56:26.207337+0000","last_peered":"2024-01-15T17:56:26.207337+0000","last_clean":"2024-01-15T17:56:26.207337+0000","last_became_active":"2024-01-15T17:56:26.203141+0000","last_became_peered":"2024-01-15T17:56:26.203141+0000","last_unstale":"2024-01-15T17:56:26.207337+0000","last_undegraded":"2024-01-15T17:56:26.207337+0000","last_fullsized":"2024-01-15T17:56:26.207337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:28:33.523943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183545+0000","last_change":"2024-01-15T17:56:26.183545+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183545+0000","last_undegraded":"2024-01-15T17:56:26.183545+0000","last_fullsized":"2024-01-15T17:56:26.183545+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.203978+0000","last_change":"2024-01-15T17:56:26.200676+0000","last_active":"2024-01-15T17:56:26.203978+0000","last_peered":"2024-01-15T17:56:26.203978+0000","last_clean":"2024-01-15T17:56:26.203978+0000","last_became_active":"2024-01-15T17:56:26.199502+0000","last_became_peered":"2024-01-15T17:56:26.199502+0000","last_unstale":"2024-01-15T17:56:26.203978+0000","last_undegraded":"2024-01-15T17:56:26.203978+0000","last_fullsized":"2024-01-15T17:56:26.203978+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:44:17.350330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185090+0000","last_change":"2024-01-15T17:56:26.185090+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185090+0000","last_undegraded":"2024-01-15T17:56:26.185090+0000","last_fullsized":"2024-01-15T17:56:26.185090+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213865+0000","last_change":"2024-01-15T17:56:26.212810+0000","last_active":"2024-01-15T17:56:26.213865+0000","last_peered":"2024-01-15T17:56:26.213865+0000","last_clean":"2024-01-15T17:56:26.213865+0000","last_became_active":"2024-01-15T17:56:26.212482+0000","last_became_peered":"2024-01-15T17:56:26.212482+0000","last_unstale":"2024-01-15T17:56:26.213865+0000","last_undegraded":"2024-01-15T17:56:26.213865+0000","last_fullsized":"2024-01-15T17:56:26.213865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:26.782884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188955+0000","last_change":"2024-01-15T17:56:26.188955+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188955+0000","last_undegraded":"2024-01-15T17:56:26.188955+0000","last_fullsized":"2024-01-15T17:56:26.188955+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210403+0000","last_change":"2024-01-15T17:56:26.208053+0000","last_active":"2024-01-15T17:56:26.210403+0000","last_peered":"2024-01-15T17:56:26.210403+0000","last_clean":"2024-01-15T17:56:26.210403+0000","last_became_active":"2024-01-15T17:56:26.207220+0000","last_became_peered":"2024-01-15T17:56:26.207220+0000","last_unstale":"2024-01-15T17:56:26.210403+0000","last_undegraded":"2024-01-15T17:56:26.210403+0000","last_fullsized":"2024-01-15T17:56:26.210403+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:31.045653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188235+0000","last_change":"2024-01-15T17:56:26.188235+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188235+0000","last_undegraded":"2024-01-15T17:56:26.188235+0000","last_fullsized":"2024-01-15T17:56:26.188235+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.215003+0000","last_change":"2024-01-15T17:56:26.213932+0000","last_active":"2024-01-15T17:56:26.215003+0000","last_peered":"2024-01-15T17:56:26.215003+0000","last_clean":"2024-01-15T17:56:26.215003+0000","last_became_active":"2024-01-15T17:56:26.213653+0000","last_became_peered":"2024-01-15T17:56:26.213653+0000","last_unstale":"2024-01-15T17:56:26.215003+0000","last_undegraded":"2024-01-15T17:56:26.215003+0000","last_fullsized":"2024-01-15T17:56:26.215003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:46.243898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.188570+0000","last_change":"2024-01-15T17:56:26.188570+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.188570+0000","last_undegraded":"2024-01-15T17:56:26.188570+0000","last_fullsized":"2024-01-15T17:56:26.188570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.209612+0000","last_change":"2024-01-15T17:56:26.203678+0000","last_active":"2024-01-15T17:56:26.209612+0000","last_peered":"2024-01-15T17:56:26.209612+0000","last_clean":"2024-01-15T17:56:26.209612+0000","last_became_active":"2024-01-15T17:56:26.203341+0000","last_became_peered":"2024-01-15T17:56:26.203341+0000","last_unstale":"2024-01-15T17:56:26.209612+0000","last_undegraded":"2024-01-15T17:56:26.209612+0000","last_fullsized":"2024-01-15T17:56:26.209612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:14:31.052513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.184920+0000","last_change":"2024-01-15T17:56:26.184920+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.184920+0000","last_undegraded":"2024-01-15T17:56:26.184920+0000","last_fullsized":"2024-01-15T17:56:26.184920+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212414+0000","last_change":"2024-01-15T17:56:26.210551+0000","last_active":"2024-01-15T17:56:26.212414+0000","last_peered":"2024-01-15T17:56:26.212414+0000","last_clean":"2024-01-15T17:56:26.212414+0000","last_became_active":"2024-01-15T17:56:26.209399+0000","last_became_peered":"2024-01-15T17:56:26.209399+0000","last_unstale":"2024-01-15T17:56:26.212414+0000","last_undegraded":"2024-01-15T17:56:26.212414+0000","last_fullsized":"2024-01-15T17:56:26.212414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:12:05.549396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185817+0000","last_change":"2024-01-15T17:56:26.185817+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185817+0000","last_undegraded":"2024-01-15T17:56:26.185817+0000","last_fullsized":"2024-01-15T17:56:26.185817+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208828+0000","last_change":"2024-01-15T17:56:26.201105+0000","last_active":"2024-01-15T17:56:26.208828+0000","last_peered":"2024-01-15T17:56:26.208828+0000","last_clean":"2024-01-15T17:56:26.208828+0000","last_became_active":"2024-01-15T17:56:26.200818+0000","last_became_peered":"2024-01-15T17:56:26.200818+0000","last_unstale":"2024-01-15T17:56:26.208828+0000","last_undegraded":"2024-01-15T17:56:26.208828+0000","last_fullsized":"2024-01-15T17:56:26.208828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:58:49.255624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.199505+0000","last_change":"2024-01-15T17:56:26.199505+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.199505+0000","last_undegraded":"2024-01-15T17:56:26.199505+0000","last_fullsized":"2024-01-15T17:56:26.199505+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212669+0000","last_change":"2024-01-15T17:56:26.211092+0000","last_active":"2024-01-15T17:56:26.212669+0000","last_peered":"2024-01-15T17:56:26.212669+0000","last_clean":"2024-01-15T17:56:26.212669+0000","last_became_active":"2024-01-15T17:56:26.210804+0000","last_became_peered":"2024-01-15T17:56:26.210804+0000","last_unstale":"2024-01-15T17:56:26.212669+0000","last_undegraded":"2024-01-15T17:56:26.212669+0000","last_fullsized":"2024-01-15T17:56:26.212669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:02:16.155632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181524+0000","last_change":"2024-01-15T17:56:26.181524+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181524+0000","last_undegraded":"2024-01-15T17:56:26.181524+0000","last_fullsized":"2024-01-15T17:56:26.181524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207189+0000","last_change":"2024-01-15T17:56:26.201849+0000","last_active":"2024-01-15T17:56:26.207189+0000","last_peered":"2024-01-15T17:56:26.207189+0000","last_clean":"2024-01-15T17:56:26.207189+0000","last_became_active":"2024-01-15T17:56:26.201558+0000","last_became_peered":"2024-01-15T17:56:26.201558+0000","last_unstale":"2024-01-15T17:56:26.207189+0000","last_undegraded":"2024-01-15T17:56:26.207189+0000","last_fullsized":"2024-01-15T17:56:26.207189+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:32:43.277391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190247+0000","last_change":"2024-01-15T17:56:26.190247+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190247+0000","last_undegraded":"2024-01-15T17:56:26.190247+0000","last_fullsized":"2024-01-15T17:56:26.190247+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.213314+0000","last_change":"2024-01-15T17:56:26.211508+0000","last_active":"2024-01-15T17:56:26.213314+0000","last_peered":"2024-01-15T17:56:26.213314+0000","last_clean":"2024-01-15T17:56:26.213314+0000","last_became_active":"2024-01-15T17:56:26.207011+0000","last_became_peered":"2024-01-15T17:56:26.207011+0000","last_unstale":"2024-01-15T17:56:26.213314+0000","last_undegraded":"2024-01-15T17:56:26.213314+0000","last_fullsized":"2024-01-15T17:56:26.213314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:40.566415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.191272+0000","last_change":"2024-01-15T17:56:26.191272+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.191272+0000","last_undegraded":"2024-01-15T17:56:26.191272+0000","last_fullsized":"2024-01-15T17:56:26.191272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.203619+0000","last_change":"2024-01-15T17:56:26.200302+0000","last_active":"2024-01-15T17:56:26.203619+0000","last_peered":"2024-01-15T17:56:26.203619+0000","last_clean":"2024-01-15T17:56:26.203619+0000","last_became_active":"2024-01-15T17:56:26.200001+0000","last_became_peered":"2024-01-15T17:56:26.200001+0000","last_unstale":"2024-01-15T17:56:26.203619+0000","last_undegraded":"2024-01-15T17:56:26.203619+0000","last_fullsized":"2024-01-15T17:56:26.203619+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:06:34.072861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187437+0000","last_change":"2024-01-15T17:56:26.187437+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187437+0000","last_undegraded":"2024-01-15T17:56:26.187437+0000","last_fullsized":"2024-01-15T17:56:26.187437+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210901+0000","last_change":"2024-01-15T17:56:26.207943+0000","last_active":"2024-01-15T17:56:26.210901+0000","last_peered":"2024-01-15T17:56:26.210901+0000","last_clean":"2024-01-15T17:56:26.210901+0000","last_became_active":"2024-01-15T17:56:26.207604+0000","last_became_peered":"2024-01-15T17:56:26.207604+0000","last_unstale":"2024-01-15T17:56:26.210901+0000","last_undegraded":"2024-01-15T17:56:26.210901+0000","last_fullsized":"2024-01-15T17:56:26.210901+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:42.102953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187383+0000","last_change":"2024-01-15T17:56:26.187383+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187383+0000","last_undegraded":"2024-01-15T17:56:26.187383+0000","last_fullsized":"2024-01-15T17:56:26.187383+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210215+0000","last_change":"2024-01-15T17:56:26.207505+0000","last_active":"2024-01-15T17:56:26.210215+0000","last_peered":"2024-01-15T17:56:26.210215+0000","last_clean":"2024-01-15T17:56:26.210215+0000","last_became_active":"2024-01-15T17:56:26.207210+0000","last_became_peered":"2024-01-15T17:56:26.207210+0000","last_unstale":"2024-01-15T17:56:26.210215+0000","last_undegraded":"2024-01-15T17:56:26.210215+0000","last_fullsized":"2024-01-15T17:56:26.210215+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:13:34.770667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183722+0000","last_change":"2024-01-15T17:56:26.183722+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183722+0000","last_undegraded":"2024-01-15T17:56:26.183722+0000","last_fullsized":"2024-01-15T17:56:26.183722+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.205077+0000","last_change":"2024-01-15T17:56:26.202274+0000","last_active":"2024-01-15T17:56:26.205077+0000","last_peered":"2024-01-15T17:56:26.205077+0000","last_clean":"2024-01-15T17:56:26.205077+0000","last_became_active":"2024-01-15T17:56:26.201924+0000","last_became_peered":"2024-01-15T17:56:26.201924+0000","last_unstale":"2024-01-15T17:56:26.205077+0000","last_undegraded":"2024-01-15T17:56:26.205077+0000","last_fullsized":"2024-01-15T17:56:26.205077+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:35.925710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.183777+0000","last_change":"2024-01-15T17:56:26.183777+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.183777+0000","last_undegraded":"2024-01-15T17:56:26.183777+0000","last_fullsized":"2024-01-15T17:56:26.183777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.205313+0000","last_change":"2024-01-15T17:56:26.200301+0000","last_active":"2024-01-15T17:56:26.205313+0000","last_peered":"2024-01-15T17:56:26.205313+0000","last_clean":"2024-01-15T17:56:26.205313+0000","last_became_active":"2024-01-15T17:56:26.200003+0000","last_became_peered":"2024-01-15T17:56:26.200003+0000","last_unstale":"2024-01-15T17:56:26.205313+0000","last_undegraded":"2024-01-15T17:56:26.205313+0000","last_fullsized":"2024-01-15T17:56:26.205313+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:02:29.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.186611+0000","last_change":"2024-01-15T17:56:26.186611+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.186611+0000","last_undegraded":"2024-01-15T17:56:26.186611+0000","last_fullsized":"2024-01-15T17:56:26.186611+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.215099+0000","last_change":"2024-01-15T17:56:26.214358+0000","last_active":"2024-01-15T17:56:26.215099+0000","last_peered":"2024-01-15T17:56:26.215099+0000","last_clean":"2024-01-15T17:56:26.215099+0000","last_became_active":"2024-01-15T17:56:26.213452+0000","last_became_peered":"2024-01-15T17:56:26.213452+0000","last_unstale":"2024-01-15T17:56:26.215099+0000","last_undegraded":"2024-01-15T17:56:26.215099+0000","last_fullsized":"2024-01-15T17:56:26.215099+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:05:49.120775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.198507+0000","last_change":"2024-01-15T17:56:26.198507+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.198507+0000","last_undegraded":"2024-01-15T17:56:26.198507+0000","last_fullsized":"2024-01-15T17:56:26.198507+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212927+0000","last_change":"2024-01-15T17:56:26.212040+0000","last_active":"2024-01-15T17:56:26.212927+0000","last_peered":"2024-01-15T17:56:26.212927+0000","last_clean":"2024-01-15T17:56:26.212927+0000","last_became_active":"2024-01-15T17:56:26.211151+0000","last_became_peered":"2024-01-15T17:56:26.211151+0000","last_unstale":"2024-01-15T17:56:26.212927+0000","last_undegraded":"2024-01-15T17:56:26.212927+0000","last_fullsized":"2024-01-15T17:56:26.212927+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T17:58:44.566548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.185596+0000","last_change":"2024-01-15T17:56:26.185596+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.185596+0000","last_undegraded":"2024-01-15T17:56:26.185596+0000","last_fullsized":"2024-01-15T17:56:26.185596+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210500+0000","last_change":"2024-01-15T17:56:26.201506+0000","last_active":"2024-01-15T17:56:26.210500+0000","last_peered":"2024-01-15T17:56:26.210500+0000","last_clean":"2024-01-15T17:56:26.210500+0000","last_became_active":"2024-01-15T17:56:26.201268+0000","last_became_peered":"2024-01-15T17:56:26.201268+0000","last_unstale":"2024-01-15T17:56:26.210500+0000","last_undegraded":"2024-01-15T17:56:26.210500+0000","last_fullsized":"2024-01-15T17:56:26.210500+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:43.670278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.190043+0000","last_change":"2024-01-15T17:56:26.190043+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.190043+0000","last_undegraded":"2024-01-15T17:56:26.190043+0000","last_fullsized":"2024-01-15T17:56:26.190043+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.202533+0000","last_change":"2024-01-15T17:56:26.200072+0000","last_active":"2024-01-15T17:56:26.202533+0000","last_peered":"2024-01-15T17:56:26.202533+0000","last_clean":"2024-01-15T17:56:26.202533+0000","last_became_active":"2024-01-15T17:56:26.199764+0000","last_became_peered":"2024-01-15T17:56:26.199764+0000","last_unstale":"2024-01-15T17:56:26.202533+0000","last_undegraded":"2024-01-15T17:56:26.202533+0000","last_fullsized":"2024-01-15T17:56:26.202533+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.189595+0000","last_change":"2024-01-15T17:56:26.189595+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.189595+0000","last_undegraded":"2024-01-15T17:56:26.189595+0000","last_fullsized":"2024-01-15T17:56:26.189595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208497+0000","last_change":"2024-01-15T17:56:26.200854+0000","last_active":"2024-01-15T17:56:26.208497+0000","last_peered":"2024-01-15T17:56:26.208497+0000","last_clean":"2024-01-15T17:56:26.208497+0000","last_became_active":"2024-01-15T17:56:26.200317+0000","last_became_peered":"2024-01-15T17:56:26.200317+0000","last_unstale":"2024-01-15T17:56:26.208497+0000","last_undegraded":"2024-01-15T17:56:26.208497+0000","last_fullsized":"2024-01-15T17:56:26.208497+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.181422+0000","last_change":"2024-01-15T17:56:26.181422+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.181422+0000","last_undegraded":"2024-01-15T17:56:26.181422+0000","last_fullsized":"2024-01-15T17:56:26.181422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208623+0000","last_change":"2024-01-15T17:56:26.205649+0000","last_active":"2024-01-15T17:56:26.208623+0000","last_peered":"2024-01-15T17:56:26.208623+0000","last_clean":"2024-01-15T17:56:26.208623+0000","last_became_active":"2024-01-15T17:56:26.205337+0000","last_became_peered":"2024-01-15T17:56:26.205337+0000","last_unstale":"2024-01-15T17:56:26.208623+0000","last_undegraded":"2024-01-15T17:56:26.208623+0000","last_fullsized":"2024-01-15T17:56:26.208623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:23:40.566415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.187391+0000","last_change":"2024-01-15T17:56:26.187391+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.187391+0000","last_undegraded":"2024-01-15T17:56:26.187391+0000","last_fullsized":"2024-01-15T17:56:26.187391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.208848+0000","last_change":"2024-01-15T17:56:26.201883+0000","last_active":"2024-01-15T17:56:26.208848+0000","last_peered":"2024-01-15T17:56:26.208848+0000","last_clean":"2024-01-15T17:56:26.208848+0000","last_became_active":"2024-01-15T17:56:26.201275+0000","last_became_peered":"2024-01-15T17:56:26.201275+0000","last_unstale":"2024-01-15T17:56:26.208848+0000","last_undegraded":"2024-01-15T17:56:26.208848+0000","last_fullsized":"2024-01-15T17:56:26.208848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:02:29.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.182677+0000","last_change":"2024-01-15T17:56:26.182677+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.182677+0000","last_undegraded":"2024-01-15T17:56:26.182677+0000","last_fullsized":"2024-01-15T17:56:26.182677+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.207896+0000","last_change":"2024-01-15T17:56:26.205102+0000","last_active":"2024-01-15T17:56:26.207896+0000","last_peered":"2024-01-15T17:56:26.207896+0000","last_clean":"2024-01-15T17:56:26.207896+0000","last_became_active":"2024-01-15T17:56:26.204747+0000","last_became_peered":"2024-01-15T17:56:26.204747+0000","last_unstale":"2024-01-15T17:56:26.207896+0000","last_undegraded":"2024-01-15T17:56:26.207896+0000","last_fullsized":"2024-01-15T17:56:26.207896+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:48.459125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.195525+0000","last_change":"2024-01-15T17:56:26.195525+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.195525+0000","last_undegraded":"2024-01-15T17:56:26.195525+0000","last_fullsized":"2024-01-15T17:56:26.195525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.212667+0000","last_change":"2024-01-15T17:56:26.211169+0000","last_active":"2024-01-15T17:56:26.212667+0000","last_peered":"2024-01-15T17:56:26.212667+0000","last_clean":"2024-01-15T17:56:26.212667+0000","last_became_active":"2024-01-15T17:56:26.210842+0000","last_became_peered":"2024-01-15T17:56:26.210842+0000","last_unstale":"2024-01-15T17:56:26.212667+0000","last_undegraded":"2024-01-15T17:56:26.212667+0000","last_fullsized":"2024-01-15T17:56:26.212667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:28:25.400140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-01-15T17:56:26.201814+0000","last_change":"2024-01-15T17:56:26.201814+0000","last_active":"2024-01-15T17:56:26.164505+0000","last_peered":"2024-01-15T17:56:26.164505+0000","last_clean":"2024-01-15T17:56:26.164505+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:56:26.201814+0000","last_undegraded":"2024-01-15T17:56:26.201814+0000","last_fullsized":"2024-01-15T17:56:26.201814+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-01-15T17:56:26.210656+0000","last_change":"2024-01-15T17:56:26.208654+0000","last_active":"2024-01-15T17:56:26.210656+0000","last_peered":"2024-01-15T17:56:26.210656+0000","last_clean":"2024-01-15T17:56:26.210656+0000","last_became_active":"2024-01-15T17:56:26.208305+0000","last_became_peered":"2024-01-15T17:56:26.208305+0000","last_unstale":"2024-01-15T17:56:26.210656+0000","last_undegraded":"2024-01-15T17:56:26.210656+0000","last_fullsized":"2024-01-15T17:56:26.210656+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:42:18.904263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":172,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-01-15T17:56:25.185666+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:56:25.185666+0000","last_peered":"2024-01-15T17:56:25.185666+0000","last_clean":"2024-01-15T17:56:25.185666+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:56:25.185666+0000","last_undegraded":"2024-01-15T17:56:25.185666+0000","last_fullsized":"2024-01-15T17:56:25.185666+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":962560,"data_stored":732013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.643}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.373},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.534}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.28},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":1.081}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.181},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.052,"5min":1.052,"15min":1.052},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.2330000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.444}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.49},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.1339999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.714,"15min":1.714},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.325},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.611}]}]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":0.876}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.155},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.2569999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.3280000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.417},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1140000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.605},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":1.4810000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.758,"5min":1.758,"15min":1.758},"min":{"1min":1.202,"5min":1.202,"15min":1.202},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.5209999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.212},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"min":{"1min":1.286,"5min":1.286,"15min":1.286},"max":{"1min":3.141,"5min":3.141,"15min":3.141},"last":1.456}]}]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.3840000000000003,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.22},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":5.2290000000000001,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.3440000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.625}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.835,"5min":2.835,"15min":2.835},"last":1.546},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.413}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.264}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.419,"15min":1.419},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.5720000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.083,"5min":1.083,"15min":1.083},"max":{"1min":5.218,"5min":5.218,"15min":5.218},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.512}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":1.2,"5min":1.2,"15min":1.2},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.4430000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.996},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.0740000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.597},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.127}]},{"osd":2,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.339},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.383}]},{"osd":3,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.4410000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":1.536},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"min":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.794}]},{"osd":6,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":1.706,"15min":1.706},"min":{"1min":1.117,"5min":1.117,"15min":1.117},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.742},{"interface":"front","average":{"1min":1.845,"5min":1.845,"15min":1.845},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.8520000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"min":{"1min":1.385,"5min":1.385,"15min":1.385},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.899},{"interface":"front","average":{"1min":1.929,"5min":1.929,"15min":1.929},"min":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":1.929}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.2989999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.4410000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.238}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.353}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.643},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":1.167}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.673,"5min":1.673,"15min":1.673},"min":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.5820000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.698,"15min":1.698},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"min":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.538}]}]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.208}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.135}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":4.008,"5min":4.008,"15min":4.008},"last":1.278},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.0289999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":1.171}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.6070000000000002,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.389},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.0529999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542165,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":368640,"data_stored":141645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.028},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.61199999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1990000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.23,"15min":2.23},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":22.106999999999999,"5min":22.106999999999999,"15min":22.106999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.0740000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":9.8510000000000009,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.272},{"interface":"front","average":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":14.359999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.113}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":13.228999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":4.9900000000000002,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":1.4039999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":14.272,"5min":14.272,"15min":14.272},"last":1.357},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.1659999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.474}]}]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339716,"statfs":{"total":96636764160,"available":96603869184,"internally_reserved":0,"allocated":962560,"data_stored":732013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.032}]},{"osd":2,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.496},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.93300000000000005}]},{"osd":3,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.597},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.542}]},{"osd":4,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.401},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.6619999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.6419999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.9990000000000001,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":1.792},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.274}]},{"osd":7,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.8100000000000001,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.857},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.74}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:56:36.822 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:56:37.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.051+0000 7fef04a00640 1 Processor -- start 2024-01-15T17:56:37.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.051+0000 7fef04a00640 1 -- start start 2024-01-15T17:56:37.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.053+0000 7fef04a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feefc160f80 0x7feefc161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.053+0000 7fef04a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0ce8f0 0x7feefc0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.053+0000 7fef04a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc1618a0 con 0x7feefc0cfbe0 2024-01-15T17:56:37.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.053+0000 7fef04a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc0d0de0 con 0x7feefc160f80 2024-01-15T17:56:37.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.053+0000 7fef04a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0d10c0 con 0x7feefc0ce8f0 2024-01-15T17:56:37.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.053+0000 7fef01600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0ce8f0 0x7feefc0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.053+0000 7fef01600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0ce8f0 0x7feefc0ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:36082/0 (socket says 172.21.15.26:36082) 2024-01-15T17:56:37.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.053+0000 7fef01600640 1 -- 172.21.15.26:0/1002925345 learned_addr learned my addr 172.21.15.26:0/1002925345 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:37.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.053+0000 7fef02000640 1 --2- 172.21.15.26:0/1002925345 >> v2:172.21.15.124:3300/0 conn(0x7feefc160f80 0x7feefc161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.054+0000 7fef01600640 1 -- 172.21.15.26:0/1002925345 >> v2:172.21.15.124:3300/0 conn(0x7feefc160f80 msgr2=0x7feefc161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:37.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.054+0000 7fef01600640 1 --2- 172.21.15.26:0/1002925345 >> v2:172.21.15.124:3300/0 conn(0x7feefc160f80 0x7feefc161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.054+0000 7fef01600640 1 -- 172.21.15.26:0/1002925345 >> v1:172.21.15.124:6789/0 conn(0x7feefc0cfbe0 legacy=0x7feefc0d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.054+0000 7fef01600640 1 -- 172.21.15.26:0/1002925345 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0d13a0 con 0x7feefc0ce8f0 2024-01-15T17:56:37.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.056+0000 7fef01600640 1 --2- 172.21.15.26:0/1002925345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0ce8f0 0x7feefc0ced50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feee4009870 tx=0x7feee402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12eca8d292ccd969 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:37.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.056+0000 7fef00c00640 1 -- 172.21.15.26:0/1002925345 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4030840 con 0x7feefc0ce8f0 2024-01-15T17:56:37.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.056+0000 7fef00c00640 1 -- 172.21.15.26:0/1002925345 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee40309a0 con 0x7feefc0ce8f0 2024-01-15T17:56:37.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.056+0000 7fef00c00640 1 -- 172.21.15.26:0/1002925345 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee4030d30 con 0x7feefc0ce8f0 2024-01-15T17:56:37.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.057+0000 7fef04a00640 1 -- 172.21.15.26:0/1002925345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0ce8f0 msgr2=0x7feefc0ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.057+0000 7fef04a00640 1 --2- 172.21.15.26:0/1002925345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0ce8f0 0x7feefc0ced50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feee4009870 tx=0x7feee402b980 comp rx=0 tx=0).stop 2024-01-15T17:56:37.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.057+0000 7fef04a00640 1 -- 172.21.15.26:0/1002925345 shutdown_connections 2024-01-15T17:56:37.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.057+0000 7fef04a00640 1 --2- 172.21.15.26:0/1002925345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0ce8f0 0x7feefc0ced50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.057+0000 7fef04a00640 1 --2- 172.21.15.26:0/1002925345 >> v2:172.21.15.124:3300/0 conn(0x7feefc160f80 0x7feefc161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.057+0000 7fef04a00640 1 -- 172.21.15.26:0/1002925345 >> 172.21.15.26:0/1002925345 conn(0x7feefc09c550 msgr2=0x7feefc0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:37.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.057+0000 7fef04a00640 1 -- 172.21.15.26:0/1002925345 shutdown_connections 2024-01-15T17:56:37.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.057+0000 7fef04a00640 1 -- 172.21.15.26:0/1002925345 wait complete. 2024-01-15T17:56:37.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.059+0000 7fef04a00640 1 Processor -- start 2024-01-15T17:56:37.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.059+0000 7fef04a00640 1 -- start start 2024-01-15T17:56:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef04a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ce8f0 0x7feefc14c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef04a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0cfbe0 0x7feefc0c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef04a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc160f80 0x7feefc15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef04a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0c8d40 con 0x7feefc0ce8f0 2024-01-15T17:56:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef04a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0c9710 con 0x7feefc160f80 2024-01-15T17:56:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef04a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0c8130 con 0x7feefc0cfbe0 2024-01-15T17:56:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef02000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ce8f0 0x7feefc14c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef02000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ce8f0 0x7feefc14c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47794/0 (socket says 172.21.15.26:47794) 2024-01-15T17:56:37.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef02000640 1 -- 172.21.15.26:0/3582568042 learned_addr learned my addr 172.21.15.26:0/3582568042 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:37.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef01600640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0cfbe0 0x7feefc0c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.060+0000 7fef02a00640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc160f80 0x7feefc15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7fef02000640 1 -- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0cfbe0 msgr2=0x7feefc0c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7fef02000640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0cfbe0 0x7feefc0c91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7fef02000640 1 -- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc160f80 msgr2=0x7feefc15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7fef02000640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc160f80 0x7feefc15fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7fef02000640 1 -- 172.21.15.26:0/3582568042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee40094d0 con 0x7feefc0ce8f0 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7fef02000640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ce8f0 0x7feefc14c740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feeec00e9b0 tx=0x7feeec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7feefa000640 1 -- 172.21.15.26:0/3582568042 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec009670 con 0x7feefc0ce8f0 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7feefa000640 1 -- 172.21.15.26:0/3582568042 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec00c9a0 con 0x7feefc0ce8f0 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7feefa000640 1 -- 172.21.15.26:0/3582568042 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec00cd50 con 0x7feefc0ce8f0 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.061+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc0c8410 con 0x7feefc0ce8f0 2024-01-15T17:56:37.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.062+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc0c89d0 con 0x7feefc0ce8f0 2024-01-15T17:56:37.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.063+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc0d1920 con 0x7feefc0ce8f0 2024-01-15T17:56:37.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.063+0000 7feefa000640 1 -- 172.21.15.26:0/3582568042 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7feeec013470 con 0x7feefc0ce8f0 2024-01-15T17:56:37.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.064+0000 7feefa000640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406b2c0 0x7feed406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.064+0000 7feefa000640 1 -- 172.21.15.26:0/3582568042 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feeec089ec0 con 0x7feefc0ce8f0 2024-01-15T17:56:37.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.065+0000 7fef01600640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406b2c0 0x7feed406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.066+0000 7fef01600640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406b2c0 0x7feed406d780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feee4036000 tx=0x7feee403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:37.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.069+0000 7feefa000640 1 -- 172.21.15.26:0/3582568042 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feeec054c90 con 0x7feefc0ce8f0 2024-01-15T17:56:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.260+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feefc0d21d0 con 0x7feed406b2c0 2024-01-15T17:56:37.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.284+0000 7feefa000640 1 -- 172.21.15.26:0/3582568042 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286295 (secure 0 0 0) 0x7feefc0d21d0 con 0x7feed406b2c0 2024-01-15T17:56:37.292 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:37.293 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:56:37.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406b2c0 msgr2=0x7feed406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406b2c0 0x7feed406d780 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feee4036000 tx=0x7feee403b040 comp rx=0 tx=0).stop 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ce8f0 msgr2=0x7feefc14c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ce8f0 0x7feefc14c740 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feeec00e9b0 tx=0x7feeec00ee70 comp rx=0 tx=0).stop 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 shutdown_connections 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0cfbe0 0x7feefc0c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406b2c0 0x7feed406d780 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc160f80 0x7feefc15fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 --2- 172.21.15.26:0/3582568042 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ce8f0 0x7feefc14c740 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 >> 172.21.15.26:0/3582568042 conn(0x7feefc09c550 msgr2=0x7feefc0b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 shutdown_connections 2024-01-15T17:56:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.291+0000 7fef04a00640 1 -- 172.21.15.26:0/3582568042 wait complete. 2024-01-15T17:56:37.314 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-01-15T17:56:35.941170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237100,"kb_used_data":4296,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737620,"statfs":{"total":773094113280,"available":772851322880,"internally_reserved":0,"allocated":4399104,"data_stored":2423316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.743112"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.830927+0000","last_change":"2024-01-15T17:56:27.245471+0000","last_active":"2024-01-15T17:56:28.830927+0000","last_peered":"2024-01-15T17:56:28.830927+0000","last_clean":"2024-01-15T17:56:28.830927+0000","last_became_active":"2024-01-15T17:56:27.245115+0000","last_became_peered":"2024-01-15T17:56:27.245115+0000","last_unstale":"2024-01-15T17:56:28.830927+0000","last_undegraded":"2024-01-15T17:56:28.830927+0000","last_fullsized":"2024-01-15T17:56:28.830927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:47.254698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.221479+0000","last_change":"2024-01-15T17:56:27.236496+0000","last_active":"2024-01-15T17:56:28.221479+0000","last_peered":"2024-01-15T17:56:28.221479+0000","last_clean":"2024-01-15T17:56:28.221479+0000","last_became_active":"2024-01-15T17:56:27.236075+0000","last_became_peered":"2024-01-15T17:56:27.236075+0000","last_unstale":"2024-01-15T17:56:28.221479+0000","last_undegraded":"2024-01-15T17:56:28.221479+0000","last_fullsized":"2024-01-15T17:56:28.221479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:28:45.446976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.944492+0000","last_change":"2024-01-15T17:56:27.242340+0000","last_active":"2024-01-15T17:56:28.944492+0000","last_peered":"2024-01-15T17:56:28.944492+0000","last_clean":"2024-01-15T17:56:28.944492+0000","last_became_active":"2024-01-15T17:56:27.240271+0000","last_became_peered":"2024-01-15T17:56:27.240271+0000","last_unstale":"2024-01-15T17:56:28.944492+0000","last_undegraded":"2024-01-15T17:56:28.944492+0000","last_fullsized":"2024-01-15T17:56:28.944492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:21:55.000829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.387001+0000","last_change":"2024-01-15T17:56:27.240086+0000","last_active":"2024-01-15T17:56:28.387001+0000","last_peered":"2024-01-15T17:56:28.387001+0000","last_clean":"2024-01-15T17:56:28.387001+0000","last_became_active":"2024-01-15T17:56:27.238271+0000","last_became_peered":"2024-01-15T17:56:27.238271+0000","last_unstale":"2024-01-15T17:56:28.387001+0000","last_undegraded":"2024-01-15T17:56:28.387001+0000","last_fullsized":"2024-01-15T17:56:28.387001+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:03:17.223427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.638677+0000","last_change":"2024-01-15T17:56:27.246645+0000","last_active":"2024-01-15T17:56:28.638677+0000","last_peered":"2024-01-15T17:56:28.638677+0000","last_clean":"2024-01-15T17:56:28.638677+0000","last_became_active":"2024-01-15T17:56:27.246338+0000","last_became_peered":"2024-01-15T17:56:27.246338+0000","last_unstale":"2024-01-15T17:56:28.638677+0000","last_undegraded":"2024-01-15T17:56:28.638677+0000","last_fullsized":"2024-01-15T17:56:28.638677+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:32.056453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.220391+0000","last_change":"2024-01-15T17:56:27.247508+0000","last_active":"2024-01-15T17:56:28.220391+0000","last_peered":"2024-01-15T17:56:28.220391+0000","last_clean":"2024-01-15T17:56:28.220391+0000","last_became_active":"2024-01-15T17:56:27.247124+0000","last_became_peered":"2024-01-15T17:56:27.247124+0000","last_unstale":"2024-01-15T17:56:28.220391+0000","last_undegraded":"2024-01-15T17:56:28.220391+0000","last_fullsized":"2024-01-15T17:56:28.220391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:31:36.096274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.222275+0000","last_change":"2024-01-15T17:56:27.234511+0000","last_active":"2024-01-15T17:56:28.222275+0000","last_peered":"2024-01-15T17:56:28.222275+0000","last_clean":"2024-01-15T17:56:28.222275+0000","last_became_active":"2024-01-15T17:56:27.234238+0000","last_became_peered":"2024-01-15T17:56:27.234238+0000","last_unstale":"2024-01-15T17:56:28.222275+0000","last_undegraded":"2024-01-15T17:56:28.222275+0000","last_fullsized":"2024-01-15T17:56:28.222275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:16:58.628832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.222083+0000","last_change":"2024-01-15T17:56:27.225028+0000","last_active":"2024-01-15T17:56:28.222083+0000","last_peered":"2024-01-15T17:56:28.222083+0000","last_clean":"2024-01-15T17:56:28.222083+0000","last_became_active":"2024-01-15T17:56:27.224738+0000","last_became_peered":"2024-01-15T17:56:27.224738+0000","last_unstale":"2024-01-15T17:56:28.222083+0000","last_undegraded":"2024-01-15T17:56:28.222083+0000","last_fullsized":"2024-01-15T17:56:28.222083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:42:41.679605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.224984+0000","last_change":"2024-01-15T17:56:27.229527+0000","last_active":"2024-01-15T17:56:28.224984+0000","last_peered":"2024-01-15T17:56:28.224984+0000","last_clean":"2024-01-15T17:56:28.224984+0000","last_became_active":"2024-01-15T17:56:27.229212+0000","last_became_peered":"2024-01-15T17:56:27.229212+0000","last_unstale":"2024-01-15T17:56:28.224984+0000","last_undegraded":"2024-01-15T17:56:28.224984+0000","last_fullsized":"2024-01-15T17:56:28.224984+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:56.908299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.387050+0000","last_change":"2024-01-15T17:56:27.241481+0000","last_active":"2024-01-15T17:56:28.387050+0000","last_peered":"2024-01-15T17:56:28.387050+0000","last_clean":"2024-01-15T17:56:28.387050+0000","last_became_active":"2024-01-15T17:56:27.241199+0000","last_became_peered":"2024-01-15T17:56:27.241199+0000","last_unstale":"2024-01-15T17:56:28.387050+0000","last_undegraded":"2024-01-15T17:56:28.387050+0000","last_fullsized":"2024-01-15T17:56:28.387050+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:40:47.989143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.223836+0000","last_change":"2024-01-15T17:56:27.233146+0000","last_active":"2024-01-15T17:56:28.223836+0000","last_peered":"2024-01-15T17:56:28.223836+0000","last_clean":"2024-01-15T17:56:28.223836+0000","last_became_active":"2024-01-15T17:56:27.232850+0000","last_became_peered":"2024-01-15T17:56:27.232850+0000","last_unstale":"2024-01-15T17:56:28.223836+0000","last_undegraded":"2024-01-15T17:56:28.223836+0000","last_fullsized":"2024-01-15T17:56:28.223836+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:46:38.143462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.943916+0000","last_change":"2024-01-15T17:56:27.238541+0000","last_active":"2024-01-15T17:56:28.943916+0000","last_peered":"2024-01-15T17:56:28.943916+0000","last_clean":"2024-01-15T17:56:28.943916+0000","last_became_active":"2024-01-15T17:56:27.237719+0000","last_became_peered":"2024-01-15T17:56:27.237719+0000","last_unstale":"2024-01-15T17:56:28.943916+0000","last_undegraded":"2024-01-15T17:56:28.943916+0000","last_fullsized":"2024-01-15T17:56:28.943916+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:22.730772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.638994+0000","last_change":"2024-01-15T17:56:27.225254+0000","last_active":"2024-01-15T17:56:28.638994+0000","last_peered":"2024-01-15T17:56:28.638994+0000","last_clean":"2024-01-15T17:56:28.638994+0000","last_became_active":"2024-01-15T17:56:27.223544+0000","last_became_peered":"2024-01-15T17:56:27.223544+0000","last_unstale":"2024-01-15T17:56:28.638994+0000","last_undegraded":"2024-01-15T17:56:28.638994+0000","last_fullsized":"2024-01-15T17:56:28.638994+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:38:56.255093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.220627+0000","last_change":"2024-01-15T17:56:27.238403+0000","last_active":"2024-01-15T17:56:28.220627+0000","last_peered":"2024-01-15T17:56:28.220627+0000","last_clean":"2024-01-15T17:56:28.220627+0000","last_became_active":"2024-01-15T17:56:27.238144+0000","last_became_peered":"2024-01-15T17:56:27.238144+0000","last_unstale":"2024-01-15T17:56:28.220627+0000","last_undegraded":"2024-01-15T17:56:28.220627+0000","last_fullsized":"2024-01-15T17:56:28.220627+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:45.200541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.943600+0000","last_change":"2024-01-15T17:56:27.236516+0000","last_active":"2024-01-15T17:56:28.943600+0000","last_peered":"2024-01-15T17:56:28.943600+0000","last_clean":"2024-01-15T17:56:28.943600+0000","last_became_active":"2024-01-15T17:56:27.234635+0000","last_became_peered":"2024-01-15T17:56:27.234635+0000","last_unstale":"2024-01-15T17:56:28.943600+0000","last_undegraded":"2024-01-15T17:56:28.943600+0000","last_fullsized":"2024-01-15T17:56:28.943600+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:06:36.733815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.223394+0000","last_change":"2024-01-15T17:56:27.240868+0000","last_active":"2024-01-15T17:56:28.223394+0000","last_peered":"2024-01-15T17:56:28.223394+0000","last_clean":"2024-01-15T17:56:28.223394+0000","last_became_active":"2024-01-15T17:56:27.239644+0000","last_became_peered":"2024-01-15T17:56:27.239644+0000","last_unstale":"2024-01-15T17:56:28.223394+0000","last_undegraded":"2024-01-15T17:56:28.223394+0000","last_fullsized":"2024-01-15T17:56:28.223394+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:49.108881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.225495+0000","last_change":"2024-01-15T17:56:27.237984+0000","last_active":"2024-01-15T17:56:28.225495+0000","last_peered":"2024-01-15T17:56:28.225495+0000","last_clean":"2024-01-15T17:56:28.225495+0000","last_became_active":"2024-01-15T17:56:27.235950+0000","last_became_peered":"2024-01-15T17:56:27.235950+0000","last_unstale":"2024-01-15T17:56:28.225495+0000","last_undegraded":"2024-01-15T17:56:28.225495+0000","last_fullsized":"2024-01-15T17:56:28.225495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:41.256378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.222036+0000","last_change":"2024-01-15T17:56:27.238299+0000","last_active":"2024-01-15T17:56:28.222036+0000","last_peered":"2024-01-15T17:56:28.222036+0000","last_clean":"2024-01-15T17:56:28.222036+0000","last_became_active":"2024-01-15T17:56:27.237994+0000","last_became_peered":"2024-01-15T17:56:27.237994+0000","last_unstale":"2024-01-15T17:56:28.222036+0000","last_undegraded":"2024-01-15T17:56:28.222036+0000","last_fullsized":"2024-01-15T17:56:28.222036+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:47.868299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.220612+0000","last_change":"2024-01-15T17:56:27.237985+0000","last_active":"2024-01-15T17:56:28.220612+0000","last_peered":"2024-01-15T17:56:28.220612+0000","last_clean":"2024-01-15T17:56:28.220612+0000","last_became_active":"2024-01-15T17:56:27.237682+0000","last_became_peered":"2024-01-15T17:56:27.237682+0000","last_unstale":"2024-01-15T17:56:28.220612+0000","last_undegraded":"2024-01-15T17:56:28.220612+0000","last_fullsized":"2024-01-15T17:56:28.220612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:47:34.560173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.639041+0000","last_change":"2024-01-15T17:56:27.243145+0000","last_active":"2024-01-15T17:56:28.639041+0000","last_peered":"2024-01-15T17:56:28.639041+0000","last_clean":"2024-01-15T17:56:28.639041+0000","last_became_active":"2024-01-15T17:56:27.242896+0000","last_became_peered":"2024-01-15T17:56:27.242896+0000","last_unstale":"2024-01-15T17:56:28.639041+0000","last_undegraded":"2024-01-15T17:56:28.639041+0000","last_fullsized":"2024-01-15T17:56:28.639041+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:35.345222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.223935+0000","last_change":"2024-01-15T17:56:27.223116+0000","last_active":"2024-01-15T17:56:28.223935+0000","last_peered":"2024-01-15T17:56:28.223935+0000","last_clean":"2024-01-15T17:56:28.223935+0000","last_became_active":"2024-01-15T17:56:27.222681+0000","last_became_peered":"2024-01-15T17:56:27.222681+0000","last_unstale":"2024-01-15T17:56:28.223935+0000","last_undegraded":"2024-01-15T17:56:28.223935+0000","last_fullsized":"2024-01-15T17:56:28.223935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:03:17.223427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.224181+0000","last_change":"2024-01-15T17:56:27.240510+0000","last_active":"2024-01-15T17:56:28.224181+0000","last_peered":"2024-01-15T17:56:28.224181+0000","last_clean":"2024-01-15T17:56:28.224181+0000","last_became_active":"2024-01-15T17:56:27.240265+0000","last_became_peered":"2024-01-15T17:56:27.240265+0000","last_unstale":"2024-01-15T17:56:28.224181+0000","last_undegraded":"2024-01-15T17:56:28.224181+0000","last_fullsized":"2024-01-15T17:56:28.224181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:40:47.989143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.382394+0000","last_change":"2024-01-15T17:56:27.228443+0000","last_active":"2024-01-15T17:56:28.382394+0000","last_peered":"2024-01-15T17:56:28.382394+0000","last_clean":"2024-01-15T17:56:28.382394+0000","last_became_active":"2024-01-15T17:56:27.228170+0000","last_became_peered":"2024-01-15T17:56:27.228170+0000","last_unstale":"2024-01-15T17:56:28.382394+0000","last_undegraded":"2024-01-15T17:56:28.382394+0000","last_fullsized":"2024-01-15T17:56:28.382394+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:47.254698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.943301+0000","last_change":"2024-01-15T17:56:27.238155+0000","last_active":"2024-01-15T17:56:28.943301+0000","last_peered":"2024-01-15T17:56:28.943301+0000","last_clean":"2024-01-15T17:56:28.943301+0000","last_became_active":"2024-01-15T17:56:27.237835+0000","last_became_peered":"2024-01-15T17:56:27.237835+0000","last_unstale":"2024-01-15T17:56:28.943301+0000","last_undegraded":"2024-01-15T17:56:28.943301+0000","last_fullsized":"2024-01-15T17:56:28.943301+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:16:58.628832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.387432+0000","last_change":"2024-01-15T17:56:27.233054+0000","last_active":"2024-01-15T17:56:28.387432+0000","last_peered":"2024-01-15T17:56:28.387432+0000","last_clean":"2024-01-15T17:56:28.387432+0000","last_became_active":"2024-01-15T17:56:27.232747+0000","last_became_peered":"2024-01-15T17:56:27.232747+0000","last_unstale":"2024-01-15T17:56:28.387432+0000","last_undegraded":"2024-01-15T17:56:28.387432+0000","last_fullsized":"2024-01-15T17:56:28.387432+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:18.374165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.222079+0000","last_change":"2024-01-15T17:56:27.240378+0000","last_active":"2024-01-15T17:56:28.222079+0000","last_peered":"2024-01-15T17:56:28.222079+0000","last_clean":"2024-01-15T17:56:28.222079+0000","last_became_active":"2024-01-15T17:56:27.239804+0000","last_became_peered":"2024-01-15T17:56:27.239804+0000","last_unstale":"2024-01-15T17:56:28.222079+0000","last_undegraded":"2024-01-15T17:56:28.222079+0000","last_fullsized":"2024-01-15T17:56:28.222079+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:13.005998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.387538+0000","last_change":"2024-01-15T17:56:27.240041+0000","last_active":"2024-01-15T17:56:28.387538+0000","last_peered":"2024-01-15T17:56:28.387538+0000","last_clean":"2024-01-15T17:56:28.387538+0000","last_became_active":"2024-01-15T17:56:27.236275+0000","last_became_peered":"2024-01-15T17:56:27.236275+0000","last_unstale":"2024-01-15T17:56:28.387538+0000","last_undegraded":"2024-01-15T17:56:28.387538+0000","last_fullsized":"2024-01-15T17:56:28.387538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:56.908299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.944046+0000","last_change":"2024-01-15T17:56:27.230499+0000","last_active":"2024-01-15T17:56:28.944046+0000","last_peered":"2024-01-15T17:56:28.944046+0000","last_clean":"2024-01-15T17:56:28.944046+0000","last_became_active":"2024-01-15T17:56:27.230188+0000","last_became_peered":"2024-01-15T17:56:27.230188+0000","last_unstale":"2024-01-15T17:56:28.944046+0000","last_undegraded":"2024-01-15T17:56:28.944046+0000","last_fullsized":"2024-01-15T17:56:28.944046+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:52:30.484393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.224327+0000","last_change":"2024-01-15T17:56:27.242589+0000","last_active":"2024-01-15T17:56:28.224327+0000","last_peered":"2024-01-15T17:56:28.224327+0000","last_clean":"2024-01-15T17:56:28.224327+0000","last_became_active":"2024-01-15T17:56:27.242322+0000","last_became_peered":"2024-01-15T17:56:27.242322+0000","last_unstale":"2024-01-15T17:56:28.224327+0000","last_undegraded":"2024-01-15T17:56:28.224327+0000","last_fullsized":"2024-01-15T17:56:28.224327+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:36.651616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.222956+0000","last_change":"2024-01-15T17:56:27.241228+0000","last_active":"2024-01-15T17:56:28.222956+0000","last_peered":"2024-01-15T17:56:28.222956+0000","last_clean":"2024-01-15T17:56:28.222956+0000","last_became_active":"2024-01-15T17:56:27.240954+0000","last_became_peered":"2024-01-15T17:56:27.240954+0000","last_unstale":"2024-01-15T17:56:28.222956+0000","last_undegraded":"2024-01-15T17:56:28.222956+0000","last_fullsized":"2024-01-15T17:56:28.222956+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:22.730772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.830506+0000","last_change":"2024-01-15T17:56:27.231132+0000","last_active":"2024-01-15T17:56:28.830506+0000","last_peered":"2024-01-15T17:56:28.830506+0000","last_clean":"2024-01-15T17:56:28.830506+0000","last_became_active":"2024-01-15T17:56:27.230843+0000","last_became_peered":"2024-01-15T17:56:27.230843+0000","last_unstale":"2024-01-15T17:56:28.830506+0000","last_undegraded":"2024-01-15T17:56:28.830506+0000","last_fullsized":"2024-01-15T17:56:28.830506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:13:37.287651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.222433+0000","last_change":"2024-01-15T17:56:27.241941+0000","last_active":"2024-01-15T17:56:28.222433+0000","last_peered":"2024-01-15T17:56:28.222433+0000","last_clean":"2024-01-15T17:56:28.222433+0000","last_became_active":"2024-01-15T17:56:27.238372+0000","last_became_peered":"2024-01-15T17:56:27.238372+0000","last_unstale":"2024-01-15T17:56:28.222433+0000","last_undegraded":"2024-01-15T17:56:28.222433+0000","last_fullsized":"2024-01-15T17:56:28.222433+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:06:36.733815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.383148+0000","last_change":"2024-01-15T17:56:27.230511+0000","last_active":"2024-01-15T17:56:28.383148+0000","last_peered":"2024-01-15T17:56:28.383148+0000","last_clean":"2024-01-15T17:56:28.383148+0000","last_became_active":"2024-01-15T17:56:27.229925+0000","last_became_peered":"2024-01-15T17:56:27.229925+0000","last_unstale":"2024-01-15T17:56:28.383148+0000","last_undegraded":"2024-01-15T17:56:28.383148+0000","last_fullsized":"2024-01-15T17:56:28.383148+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:38:27.412944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.256490+0000","last_change":"2024-01-15T17:56:26.202527+0000","last_active":"2024-01-15T17:56:27.256490+0000","last_peered":"2024-01-15T17:56:27.256490+0000","last_clean":"2024-01-15T17:56:27.256490+0000","last_became_active":"2024-01-15T17:56:26.202254+0000","last_became_peered":"2024-01-15T17:56:26.202254+0000","last_unstale":"2024-01-15T17:56:27.256490+0000","last_undegraded":"2024-01-15T17:56:27.256490+0000","last_fullsized":"2024-01-15T17:56:27.256490+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:34.334422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.831282+0000","last_change":"2024-01-15T17:56:27.237529+0000","last_active":"2024-01-15T17:56:28.831282+0000","last_peered":"2024-01-15T17:56:28.831282+0000","last_clean":"2024-01-15T17:56:28.831282+0000","last_became_active":"2024-01-15T17:56:27.237224+0000","last_became_peered":"2024-01-15T17:56:27.237224+0000","last_unstale":"2024-01-15T17:56:28.831282+0000","last_undegraded":"2024-01-15T17:56:28.831282+0000","last_fullsized":"2024-01-15T17:56:28.831282+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:42.735524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.251492+0000","last_change":"2024-01-15T17:56:26.213479+0000","last_active":"2024-01-15T17:56:27.251492+0000","last_peered":"2024-01-15T17:56:27.251492+0000","last_clean":"2024-01-15T17:56:27.251492+0000","last_became_active":"2024-01-15T17:56:26.213165+0000","last_became_peered":"2024-01-15T17:56:26.213165+0000","last_unstale":"2024-01-15T17:56:27.251492+0000","last_undegraded":"2024-01-15T17:56:27.251492+0000","last_fullsized":"2024-01-15T17:56:27.251492+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:22:59.067583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.224768+0000","last_change":"2024-01-15T17:56:27.240386+0000","last_active":"2024-01-15T17:56:28.224768+0000","last_peered":"2024-01-15T17:56:28.224768+0000","last_clean":"2024-01-15T17:56:28.224768+0000","last_became_active":"2024-01-15T17:56:27.240101+0000","last_became_peered":"2024-01-15T17:56:27.240101+0000","last_unstale":"2024-01-15T17:56:28.224768+0000","last_undegraded":"2024-01-15T17:56:28.224768+0000","last_fullsized":"2024-01-15T17:56:28.224768+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:18.374165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.258583+0000","last_change":"2024-01-15T17:56:26.199957+0000","last_active":"2024-01-15T17:56:27.258583+0000","last_peered":"2024-01-15T17:56:27.258583+0000","last_clean":"2024-01-15T17:56:27.258583+0000","last_became_active":"2024-01-15T17:56:26.199632+0000","last_became_peered":"2024-01-15T17:56:26.199632+0000","last_unstale":"2024-01-15T17:56:27.258583+0000","last_undegraded":"2024-01-15T17:56:27.258583+0000","last_fullsized":"2024-01-15T17:56:27.258583+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:04:15.978631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.387788+0000","last_change":"2024-01-15T17:56:27.242475+0000","last_active":"2024-01-15T17:56:28.387788+0000","last_peered":"2024-01-15T17:56:28.387788+0000","last_clean":"2024-01-15T17:56:28.387788+0000","last_became_active":"2024-01-15T17:56:27.240281+0000","last_became_peered":"2024-01-15T17:56:27.240281+0000","last_unstale":"2024-01-15T17:56:28.387788+0000","last_undegraded":"2024-01-15T17:56:28.387788+0000","last_fullsized":"2024-01-15T17:56:28.387788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:46.268557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.254864+0000","last_change":"2024-01-15T17:56:26.209481+0000","last_active":"2024-01-15T17:56:27.254864+0000","last_peered":"2024-01-15T17:56:27.254864+0000","last_clean":"2024-01-15T17:56:27.254864+0000","last_became_active":"2024-01-15T17:56:26.209251+0000","last_became_peered":"2024-01-15T17:56:26.209251+0000","last_unstale":"2024-01-15T17:56:27.254864+0000","last_undegraded":"2024-01-15T17:56:27.254864+0000","last_fullsized":"2024-01-15T17:56:27.254864+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:43.670278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.223126+0000","last_change":"2024-01-15T17:56:27.240026+0000","last_active":"2024-01-15T17:56:28.223126+0000","last_peered":"2024-01-15T17:56:28.223126+0000","last_clean":"2024-01-15T17:56:28.223126+0000","last_became_active":"2024-01-15T17:56:27.239742+0000","last_became_peered":"2024-01-15T17:56:27.239742+0000","last_unstale":"2024-01-15T17:56:28.223126+0000","last_undegraded":"2024-01-15T17:56:28.223126+0000","last_fullsized":"2024-01-15T17:56:28.223126+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:52:30.484393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.255429+0000","last_change":"2024-01-15T17:56:26.209952+0000","last_active":"2024-01-15T17:56:27.255429+0000","last_peered":"2024-01-15T17:56:27.255429+0000","last_clean":"2024-01-15T17:56:27.255429+0000","last_became_active":"2024-01-15T17:56:26.209696+0000","last_became_peered":"2024-01-15T17:56:26.209696+0000","last_unstale":"2024-01-15T17:56:27.255429+0000","last_undegraded":"2024-01-15T17:56:27.255429+0000","last_fullsized":"2024-01-15T17:56:27.255429+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:38:55.244293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.943745+0000","last_change":"2024-01-15T17:56:27.236450+0000","last_active":"2024-01-15T17:56:28.943745+0000","last_peered":"2024-01-15T17:56:28.943745+0000","last_clean":"2024-01-15T17:56:28.943745+0000","last_became_active":"2024-01-15T17:56:27.234863+0000","last_became_peered":"2024-01-15T17:56:27.234863+0000","last_unstale":"2024-01-15T17:56:28.943745+0000","last_undegraded":"2024-01-15T17:56:28.943745+0000","last_fullsized":"2024-01-15T17:56:28.943745+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:33:30.592329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.251519+0000","last_change":"2024-01-15T17:56:26.211397+0000","last_active":"2024-01-15T17:56:27.251519+0000","last_peered":"2024-01-15T17:56:27.251519+0000","last_clean":"2024-01-15T17:56:27.251519+0000","last_became_active":"2024-01-15T17:56:26.210076+0000","last_became_peered":"2024-01-15T17:56:26.210076+0000","last_unstale":"2024-01-15T17:56:27.251519+0000","last_undegraded":"2024-01-15T17:56:27.251519+0000","last_fullsized":"2024-01-15T17:56:27.251519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:13:36.276851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.223054+0000","last_change":"2024-01-15T17:56:27.228829+0000","last_active":"2024-01-15T17:56:28.223054+0000","last_peered":"2024-01-15T17:56:28.223054+0000","last_clean":"2024-01-15T17:56:28.223054+0000","last_became_active":"2024-01-15T17:56:27.228329+0000","last_became_peered":"2024-01-15T17:56:27.228329+0000","last_unstale":"2024-01-15T17:56:28.223054+0000","last_undegraded":"2024-01-15T17:56:28.223054+0000","last_fullsized":"2024-01-15T17:56:28.223054+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:38:27.412944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.229001+0000","last_change":"2024-01-15T17:56:26.203486+0000","last_active":"2024-01-15T17:56:27.229001+0000","last_peered":"2024-01-15T17:56:27.229001+0000","last_clean":"2024-01-15T17:56:27.229001+0000","last_became_active":"2024-01-15T17:56:26.203141+0000","last_became_peered":"2024-01-15T17:56:26.203141+0000","last_unstale":"2024-01-15T17:56:27.229001+0000","last_undegraded":"2024-01-15T17:56:27.229001+0000","last_fullsized":"2024-01-15T17:56:27.229001+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:22:10.300002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.386704+0000","last_change":"2024-01-15T17:56:27.248539+0000","last_active":"2024-01-15T17:56:28.386704+0000","last_peered":"2024-01-15T17:56:28.386704+0000","last_clean":"2024-01-15T17:56:28.386704+0000","last_became_active":"2024-01-15T17:56:27.248191+0000","last_became_peered":"2024-01-15T17:56:27.248191+0000","last_unstale":"2024-01-15T17:56:28.386704+0000","last_undegraded":"2024-01-15T17:56:28.386704+0000","last_fullsized":"2024-01-15T17:56:28.386704+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:06:44.454991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.204900+0000","last_change":"2024-01-15T17:56:26.200676+0000","last_active":"2024-01-15T17:56:27.204900+0000","last_peered":"2024-01-15T17:56:27.204900+0000","last_clean":"2024-01-15T17:56:27.204900+0000","last_became_active":"2024-01-15T17:56:26.199502+0000","last_became_peered":"2024-01-15T17:56:26.199502+0000","last_unstale":"2024-01-15T17:56:27.204900+0000","last_undegraded":"2024-01-15T17:56:27.204900+0000","last_fullsized":"2024-01-15T17:56:27.204900+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:35.925710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.944302+0000","last_change":"2024-01-15T17:56:27.247967+0000","last_active":"2024-01-15T17:56:28.944302+0000","last_peered":"2024-01-15T17:56:28.944302+0000","last_clean":"2024-01-15T17:56:28.944302+0000","last_became_active":"2024-01-15T17:56:27.247574+0000","last_became_peered":"2024-01-15T17:56:27.247574+0000","last_unstale":"2024-01-15T17:56:28.944302+0000","last_undegraded":"2024-01-15T17:56:28.944302+0000","last_fullsized":"2024-01-15T17:56:28.944302+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:49.108881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.237071+0000","last_change":"2024-01-15T17:56:26.212810+0000","last_active":"2024-01-15T17:56:27.237071+0000","last_peered":"2024-01-15T17:56:27.237071+0000","last_clean":"2024-01-15T17:56:27.237071+0000","last_became_active":"2024-01-15T17:56:26.212482+0000","last_became_peered":"2024-01-15T17:56:26.212482+0000","last_unstale":"2024-01-15T17:56:27.237071+0000","last_undegraded":"2024-01-15T17:56:27.237071+0000","last_fullsized":"2024-01-15T17:56:27.237071+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:27:20.426376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.220864+0000","last_change":"2024-01-15T17:56:27.237200+0000","last_active":"2024-01-15T17:56:28.220864+0000","last_peered":"2024-01-15T17:56:28.220864+0000","last_clean":"2024-01-15T17:56:28.220864+0000","last_became_active":"2024-01-15T17:56:27.236671+0000","last_became_peered":"2024-01-15T17:56:27.236671+0000","last_unstale":"2024-01-15T17:56:28.220864+0000","last_undegraded":"2024-01-15T17:56:28.220864+0000","last_fullsized":"2024-01-15T17:56:28.220864+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:37:47.463858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.201835+0000","last_change":"2024-01-15T17:56:26.208053+0000","last_active":"2024-01-15T17:56:27.201835+0000","last_peered":"2024-01-15T17:56:27.201835+0000","last_clean":"2024-01-15T17:56:27.201835+0000","last_became_active":"2024-01-15T17:56:26.207220+0000","last_became_peered":"2024-01-15T17:56:26.207220+0000","last_unstale":"2024-01-15T17:56:27.201835+0000","last_undegraded":"2024-01-15T17:56:27.201835+0000","last_fullsized":"2024-01-15T17:56:27.201835+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:44.509811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.221139+0000","last_change":"2024-01-15T17:56:27.239615+0000","last_active":"2024-01-15T17:56:28.221139+0000","last_peered":"2024-01-15T17:56:28.221139+0000","last_clean":"2024-01-15T17:56:28.221139+0000","last_became_active":"2024-01-15T17:56:27.238916+0000","last_became_peered":"2024-01-15T17:56:27.238916+0000","last_unstale":"2024-01-15T17:56:28.221139+0000","last_undegraded":"2024-01-15T17:56:28.221139+0000","last_fullsized":"2024-01-15T17:56:28.221139+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:26:22.004051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.253963+0000","last_change":"2024-01-15T17:56:26.213932+0000","last_active":"2024-01-15T17:56:27.253963+0000","last_peered":"2024-01-15T17:56:27.253963+0000","last_clean":"2024-01-15T17:56:27.253963+0000","last_became_active":"2024-01-15T17:56:26.213653+0000","last_became_peered":"2024-01-15T17:56:26.213653+0000","last_unstale":"2024-01-15T17:56:27.253963+0000","last_undegraded":"2024-01-15T17:56:27.253963+0000","last_fullsized":"2024-01-15T17:56:27.253963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:27:25.653448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.944363+0000","last_change":"2024-01-15T17:56:27.234915+0000","last_active":"2024-01-15T17:56:28.944363+0000","last_peered":"2024-01-15T17:56:28.944363+0000","last_clean":"2024-01-15T17:56:28.944363+0000","last_became_active":"2024-01-15T17:56:27.234611+0000","last_became_peered":"2024-01-15T17:56:27.234611+0000","last_unstale":"2024-01-15T17:56:28.944363+0000","last_undegraded":"2024-01-15T17:56:28.944363+0000","last_fullsized":"2024-01-15T17:56:28.944363+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:46:38.143462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.260879+0000","last_change":"2024-01-15T17:56:26.203678+0000","last_active":"2024-01-15T17:56:27.260879+0000","last_peered":"2024-01-15T17:56:27.260879+0000","last_clean":"2024-01-15T17:56:27.260879+0000","last_became_active":"2024-01-15T17:56:26.203341+0000","last_became_peered":"2024-01-15T17:56:26.203341+0000","last_unstale":"2024-01-15T17:56:27.260879+0000","last_undegraded":"2024-01-15T17:56:27.260879+0000","last_fullsized":"2024-01-15T17:56:27.260879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:15:41.063833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.386620+0000","last_change":"2024-01-15T17:56:27.241711+0000","last_active":"2024-01-15T17:56:28.386620+0000","last_peered":"2024-01-15T17:56:28.386620+0000","last_clean":"2024-01-15T17:56:28.386620+0000","last_became_active":"2024-01-15T17:56:27.237749+0000","last_became_peered":"2024-01-15T17:56:27.237749+0000","last_unstale":"2024-01-15T17:56:28.386620+0000","last_undegraded":"2024-01-15T17:56:28.386620+0000","last_fullsized":"2024-01-15T17:56:28.386620+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:38:27.412944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.254935+0000","last_change":"2024-01-15T17:56:26.210551+0000","last_active":"2024-01-15T17:56:27.254935+0000","last_peered":"2024-01-15T17:56:27.254935+0000","last_clean":"2024-01-15T17:56:27.254935+0000","last_became_active":"2024-01-15T17:56:26.209399+0000","last_became_peered":"2024-01-15T17:56:26.209399+0000","last_unstale":"2024-01-15T17:56:27.254935+0000","last_undegraded":"2024-01-15T17:56:27.254935+0000","last_fullsized":"2024-01-15T17:56:27.254935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:30:34.334422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.220273+0000","last_change":"2024-01-15T17:56:27.239363+0000","last_active":"2024-01-15T17:56:28.220273+0000","last_peered":"2024-01-15T17:56:28.220273+0000","last_clean":"2024-01-15T17:56:28.220273+0000","last_became_active":"2024-01-15T17:56:27.233073+0000","last_became_peered":"2024-01-15T17:56:27.233073+0000","last_unstale":"2024-01-15T17:56:28.220273+0000","last_undegraded":"2024-01-15T17:56:28.220273+0000","last_fullsized":"2024-01-15T17:56:28.220273+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:04:14.915974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.221700+0000","last_change":"2024-01-15T17:56:26.201105+0000","last_active":"2024-01-15T17:56:27.221700+0000","last_peered":"2024-01-15T17:56:27.221700+0000","last_clean":"2024-01-15T17:56:27.221700+0000","last_became_active":"2024-01-15T17:56:26.200818+0000","last_became_peered":"2024-01-15T17:56:26.200818+0000","last_unstale":"2024-01-15T17:56:27.221700+0000","last_undegraded":"2024-01-15T17:56:27.221700+0000","last_fullsized":"2024-01-15T17:56:27.221700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:13:36.276851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.222743+0000","last_change":"2024-01-15T17:56:27.240671+0000","last_active":"2024-01-15T17:56:28.222743+0000","last_peered":"2024-01-15T17:56:28.222743+0000","last_clean":"2024-01-15T17:56:28.222743+0000","last_became_active":"2024-01-15T17:56:27.239280+0000","last_became_peered":"2024-01-15T17:56:27.239280+0000","last_unstale":"2024-01-15T17:56:28.222743+0000","last_undegraded":"2024-01-15T17:56:28.222743+0000","last_fullsized":"2024-01-15T17:56:28.222743+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:33:30.592329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.228870+0000","last_change":"2024-01-15T17:56:26.211092+0000","last_active":"2024-01-15T17:56:27.228870+0000","last_peered":"2024-01-15T17:56:27.228870+0000","last_clean":"2024-01-15T17:56:27.228870+0000","last_became_active":"2024-01-15T17:56:26.210804+0000","last_became_peered":"2024-01-15T17:56:26.210804+0000","last_unstale":"2024-01-15T17:56:27.228870+0000","last_undegraded":"2024-01-15T17:56:27.228870+0000","last_fullsized":"2024-01-15T17:56:27.228870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:58:46.164665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.830615+0000","last_change":"2024-01-15T17:56:27.236533+0000","last_active":"2024-01-15T17:56:28.830615+0000","last_peered":"2024-01-15T17:56:28.830615+0000","last_clean":"2024-01-15T17:56:28.830615+0000","last_became_active":"2024-01-15T17:56:27.236220+0000","last_became_peered":"2024-01-15T17:56:27.236220+0000","last_unstale":"2024-01-15T17:56:28.830615+0000","last_undegraded":"2024-01-15T17:56:28.830615+0000","last_fullsized":"2024-01-15T17:56:28.830615+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:45.520611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.259222+0000","last_change":"2024-01-15T17:56:26.201849+0000","last_active":"2024-01-15T17:56:27.259222+0000","last_peered":"2024-01-15T17:56:27.259222+0000","last_clean":"2024-01-15T17:56:27.259222+0000","last_became_active":"2024-01-15T17:56:26.201558+0000","last_became_peered":"2024-01-15T17:56:26.201558+0000","last_unstale":"2024-01-15T17:56:27.259222+0000","last_undegraded":"2024-01-15T17:56:27.259222+0000","last_fullsized":"2024-01-15T17:56:27.259222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:38:00.675361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.225554+0000","last_change":"2024-01-15T17:56:27.236277+0000","last_active":"2024-01-15T17:56:28.225554+0000","last_peered":"2024-01-15T17:56:28.225554+0000","last_clean":"2024-01-15T17:56:28.225554+0000","last_became_active":"2024-01-15T17:56:27.235644+0000","last_became_peered":"2024-01-15T17:56:27.235644+0000","last_unstale":"2024-01-15T17:56:28.225554+0000","last_undegraded":"2024-01-15T17:56:28.225554+0000","last_fullsized":"2024-01-15T17:56:28.225554+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:46.268557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.258892+0000","last_change":"2024-01-15T17:56:26.211508+0000","last_active":"2024-01-15T17:56:27.258892+0000","last_peered":"2024-01-15T17:56:27.258892+0000","last_clean":"2024-01-15T17:56:27.258892+0000","last_became_active":"2024-01-15T17:56:26.207011+0000","last_became_peered":"2024-01-15T17:56:26.207011+0000","last_unstale":"2024-01-15T17:56:27.258892+0000","last_undegraded":"2024-01-15T17:56:27.258892+0000","last_fullsized":"2024-01-15T17:56:27.258892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:46.243898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.944359+0000","last_change":"2024-01-15T17:56:27.241082+0000","last_active":"2024-01-15T17:56:28.944359+0000","last_peered":"2024-01-15T17:56:28.944359+0000","last_clean":"2024-01-15T17:56:28.944359+0000","last_became_active":"2024-01-15T17:56:27.240745+0000","last_became_peered":"2024-01-15T17:56:27.240745+0000","last_unstale":"2024-01-15T17:56:28.944359+0000","last_undegraded":"2024-01-15T17:56:28.944359+0000","last_fullsized":"2024-01-15T17:56:28.944359+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:41.256378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.254058+0000","last_change":"2024-01-15T17:56:26.200302+0000","last_active":"2024-01-15T17:56:27.254058+0000","last_peered":"2024-01-15T17:56:27.254058+0000","last_clean":"2024-01-15T17:56:27.254058+0000","last_became_active":"2024-01-15T17:56:26.200001+0000","last_became_peered":"2024-01-15T17:56:26.200001+0000","last_unstale":"2024-01-15T17:56:27.254058+0000","last_undegraded":"2024-01-15T17:56:27.254058+0000","last_fullsized":"2024-01-15T17:56:27.254058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:44.509811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.387196+0000","last_change":"2024-01-15T17:56:27.241207+0000","last_active":"2024-01-15T17:56:28.387196+0000","last_peered":"2024-01-15T17:56:28.387196+0000","last_clean":"2024-01-15T17:56:28.387196+0000","last_became_active":"2024-01-15T17:56:27.240812+0000","last_became_peered":"2024-01-15T17:56:27.240812+0000","last_unstale":"2024-01-15T17:56:28.387196+0000","last_undegraded":"2024-01-15T17:56:28.387196+0000","last_fullsized":"2024-01-15T17:56:28.387196+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:36.651616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.254280+0000","last_change":"2024-01-15T17:56:26.207943+0000","last_active":"2024-01-15T17:56:27.254280+0000","last_peered":"2024-01-15T17:56:27.254280+0000","last_clean":"2024-01-15T17:56:27.254280+0000","last_became_active":"2024-01-15T17:56:26.207604+0000","last_became_peered":"2024-01-15T17:56:26.207604+0000","last_unstale":"2024-01-15T17:56:27.254280+0000","last_undegraded":"2024-01-15T17:56:27.254280+0000","last_fullsized":"2024-01-15T17:56:27.254280+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:46.243898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.639282+0000","last_change":"2024-01-15T17:56:27.232067+0000","last_active":"2024-01-15T17:56:28.639282+0000","last_peered":"2024-01-15T17:56:28.639282+0000","last_clean":"2024-01-15T17:56:28.639282+0000","last_became_active":"2024-01-15T17:56:27.230704+0000","last_became_peered":"2024-01-15T17:56:27.230704+0000","last_unstale":"2024-01-15T17:56:28.639282+0000","last_undegraded":"2024-01-15T17:56:28.639282+0000","last_fullsized":"2024-01-15T17:56:28.639282+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:56:57.008319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.205743+0000","last_change":"2024-01-15T17:56:26.207505+0000","last_active":"2024-01-15T17:56:27.205743+0000","last_peered":"2024-01-15T17:56:27.205743+0000","last_clean":"2024-01-15T17:56:27.205743+0000","last_became_active":"2024-01-15T17:56:26.207210+0000","last_became_peered":"2024-01-15T17:56:26.207210+0000","last_unstale":"2024-01-15T17:56:27.205743+0000","last_undegraded":"2024-01-15T17:56:27.205743+0000","last_fullsized":"2024-01-15T17:56:27.205743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:56:55.997520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.638727+0000","last_change":"2024-01-15T17:56:27.234110+0000","last_active":"2024-01-15T17:56:28.638727+0000","last_peered":"2024-01-15T17:56:28.638727+0000","last_clean":"2024-01-15T17:56:28.638727+0000","last_became_active":"2024-01-15T17:56:27.231600+0000","last_became_peered":"2024-01-15T17:56:27.231600+0000","last_unstale":"2024-01-15T17:56:28.638727+0000","last_undegraded":"2024-01-15T17:56:28.638727+0000","last_fullsized":"2024-01-15T17:56:28.638727+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:23:00.078383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.253228+0000","last_change":"2024-01-15T17:56:26.202274+0000","last_active":"2024-01-15T17:56:27.253228+0000","last_peered":"2024-01-15T17:56:27.253228+0000","last_clean":"2024-01-15T17:56:27.253228+0000","last_became_active":"2024-01-15T17:56:26.201924+0000","last_became_peered":"2024-01-15T17:56:26.201924+0000","last_unstale":"2024-01-15T17:56:27.253228+0000","last_undegraded":"2024-01-15T17:56:27.253228+0000","last_fullsized":"2024-01-15T17:56:27.253228+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:13:36.276851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.638909+0000","last_change":"2024-01-15T17:56:27.225774+0000","last_active":"2024-01-15T17:56:28.638909+0000","last_peered":"2024-01-15T17:56:28.638909+0000","last_clean":"2024-01-15T17:56:28.638909+0000","last_became_active":"2024-01-15T17:56:27.223931+0000","last_became_peered":"2024-01-15T17:56:27.223931+0000","last_unstale":"2024-01-15T17:56:28.638909+0000","last_undegraded":"2024-01-15T17:56:28.638909+0000","last_fullsized":"2024-01-15T17:56:28.638909+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:21.470726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.217716+0000","last_change":"2024-01-15T17:56:26.200301+0000","last_active":"2024-01-15T17:56:27.217716+0000","last_peered":"2024-01-15T17:56:27.217716+0000","last_clean":"2024-01-15T17:56:27.217716+0000","last_became_active":"2024-01-15T17:56:26.200003+0000","last_became_peered":"2024-01-15T17:56:26.200003+0000","last_unstale":"2024-01-15T17:56:27.217716+0000","last_undegraded":"2024-01-15T17:56:27.217716+0000","last_fullsized":"2024-01-15T17:56:27.217716+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:35.925710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.638619+0000","last_change":"2024-01-15T17:56:27.223377+0000","last_active":"2024-01-15T17:56:28.638619+0000","last_peered":"2024-01-15T17:56:28.638619+0000","last_clean":"2024-01-15T17:56:28.638619+0000","last_became_active":"2024-01-15T17:56:27.222401+0000","last_became_peered":"2024-01-15T17:56:27.222401+0000","last_unstale":"2024-01-15T17:56:28.638619+0000","last_undegraded":"2024-01-15T17:56:28.638619+0000","last_fullsized":"2024-01-15T17:56:28.638619+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:36.936510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.204930+0000","last_change":"2024-01-15T17:56:26.214358+0000","last_active":"2024-01-15T17:56:27.204930+0000","last_peered":"2024-01-15T17:56:27.204930+0000","last_clean":"2024-01-15T17:56:27.204930+0000","last_became_active":"2024-01-15T17:56:26.213452+0000","last_became_peered":"2024-01-15T17:56:26.213452+0000","last_unstale":"2024-01-15T17:56:27.204930+0000","last_undegraded":"2024-01-15T17:56:27.204930+0000","last_fullsized":"2024-01-15T17:56:27.204930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:31.045653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.222272+0000","last_change":"2024-01-15T17:56:27.234846+0000","last_active":"2024-01-15T17:56:28.222272+0000","last_peered":"2024-01-15T17:56:28.222272+0000","last_clean":"2024-01-15T17:56:28.222272+0000","last_became_active":"2024-01-15T17:56:27.234585+0000","last_became_peered":"2024-01-15T17:56:27.234585+0000","last_unstale":"2024-01-15T17:56:28.222272+0000","last_undegraded":"2024-01-15T17:56:28.222272+0000","last_fullsized":"2024-01-15T17:56:28.222272+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:27:26.664248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.226350+0000","last_change":"2024-01-15T17:56:26.212040+0000","last_active":"2024-01-15T17:56:27.226350+0000","last_peered":"2024-01-15T17:56:27.226350+0000","last_clean":"2024-01-15T17:56:27.226350+0000","last_became_active":"2024-01-15T17:56:26.211151+0000","last_became_peered":"2024-01-15T17:56:26.211151+0000","last_unstale":"2024-01-15T17:56:27.226350+0000","last_undegraded":"2024-01-15T17:56:27.226350+0000","last_fullsized":"2024-01-15T17:56:27.226350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:00:19.864569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.638909+0000","last_change":"2024-01-15T17:56:27.239933+0000","last_active":"2024-01-15T17:56:28.638909+0000","last_peered":"2024-01-15T17:56:28.638909+0000","last_clean":"2024-01-15T17:56:28.638909+0000","last_became_active":"2024-01-15T17:56:27.239573+0000","last_became_peered":"2024-01-15T17:56:27.239573+0000","last_unstale":"2024-01-15T17:56:28.638909+0000","last_undegraded":"2024-01-15T17:56:28.638909+0000","last_fullsized":"2024-01-15T17:56:28.638909+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:07.936327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.257839+0000","last_change":"2024-01-15T17:56:26.201506+0000","last_active":"2024-01-15T17:56:27.257839+0000","last_peered":"2024-01-15T17:56:27.257839+0000","last_clean":"2024-01-15T17:56:27.257839+0000","last_became_active":"2024-01-15T17:56:26.201268+0000","last_became_peered":"2024-01-15T17:56:26.201268+0000","last_unstale":"2024-01-15T17:56:27.257839+0000","last_undegraded":"2024-01-15T17:56:27.257839+0000","last_fullsized":"2024-01-15T17:56:27.257839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:41.724724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.944456+0000","last_change":"2024-01-15T17:56:27.234186+0000","last_active":"2024-01-15T17:56:28.944456+0000","last_peered":"2024-01-15T17:56:28.944456+0000","last_clean":"2024-01-15T17:56:28.944456+0000","last_became_active":"2024-01-15T17:56:27.233708+0000","last_became_peered":"2024-01-15T17:56:27.233708+0000","last_unstale":"2024-01-15T17:56:28.944456+0000","last_undegraded":"2024-01-15T17:56:28.944456+0000","last_fullsized":"2024-01-15T17:56:28.944456+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:13:12.057319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.383225+0000","last_change":"2024-01-15T17:56:26.200072+0000","last_active":"2024-01-15T17:56:28.383225+0000","last_peered":"2024-01-15T17:56:28.383225+0000","last_clean":"2024-01-15T17:56:28.383225+0000","last_became_active":"2024-01-15T17:56:26.199764+0000","last_became_peered":"2024-01-15T17:56:26.199764+0000","last_unstale":"2024-01-15T17:56:28.383225+0000","last_undegraded":"2024-01-15T17:56:28.383225+0000","last_fullsized":"2024-01-15T17:56:28.383225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:56:55.997520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.383150+0000","last_change":"2024-01-15T17:56:27.225578+0000","last_active":"2024-01-15T17:56:28.383150+0000","last_peered":"2024-01-15T17:56:28.383150+0000","last_clean":"2024-01-15T17:56:28.383150+0000","last_became_active":"2024-01-15T17:56:27.225289+0000","last_became_peered":"2024-01-15T17:56:27.225289+0000","last_unstale":"2024-01-15T17:56:28.383150+0000","last_undegraded":"2024-01-15T17:56:28.383150+0000","last_fullsized":"2024-01-15T17:56:28.383150+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:06:44.454991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.229250+0000","last_change":"2024-01-15T17:56:26.200854+0000","last_active":"2024-01-15T17:56:27.229250+0000","last_peered":"2024-01-15T17:56:27.229250+0000","last_clean":"2024-01-15T17:56:27.229250+0000","last_became_active":"2024-01-15T17:56:26.200317+0000","last_became_peered":"2024-01-15T17:56:26.200317+0000","last_unstale":"2024-01-15T17:56:27.229250+0000","last_undegraded":"2024-01-15T17:56:27.229250+0000","last_fullsized":"2024-01-15T17:56:27.229250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T17:58:44.566548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.830765+0000","last_change":"2024-01-15T17:56:27.233480+0000","last_active":"2024-01-15T17:56:28.830765+0000","last_peered":"2024-01-15T17:56:28.830765+0000","last_clean":"2024-01-15T17:56:28.830765+0000","last_became_active":"2024-01-15T17:56:27.230369+0000","last_became_peered":"2024-01-15T17:56:27.230369+0000","last_unstale":"2024-01-15T17:56:28.830765+0000","last_undegraded":"2024-01-15T17:56:28.830765+0000","last_fullsized":"2024-01-15T17:56:28.830765+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T17:58:45.577348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.201964+0000","last_change":"2024-01-15T17:56:26.205649+0000","last_active":"2024-01-15T17:56:27.201964+0000","last_peered":"2024-01-15T17:56:27.201964+0000","last_clean":"2024-01-15T17:56:27.201964+0000","last_became_active":"2024-01-15T17:56:26.205337+0000","last_became_peered":"2024-01-15T17:56:26.205337+0000","last_unstale":"2024-01-15T17:56:27.201964+0000","last_undegraded":"2024-01-15T17:56:27.201964+0000","last_fullsized":"2024-01-15T17:56:27.201964+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T17:58:44.566548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.382657+0000","last_change":"2024-01-15T17:56:27.231005+0000","last_active":"2024-01-15T17:56:28.382657+0000","last_peered":"2024-01-15T17:56:28.382657+0000","last_clean":"2024-01-15T17:56:28.382657+0000","last_became_active":"2024-01-15T17:56:27.230303+0000","last_became_peered":"2024-01-15T17:56:27.230303+0000","last_unstale":"2024-01-15T17:56:28.382657+0000","last_undegraded":"2024-01-15T17:56:28.382657+0000","last_fullsized":"2024-01-15T17:56:28.382657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:24:42.735524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.221853+0000","last_change":"2024-01-15T17:56:26.201883+0000","last_active":"2024-01-15T17:56:27.221853+0000","last_peered":"2024-01-15T17:56:27.221853+0000","last_clean":"2024-01-15T17:56:27.221853+0000","last_became_active":"2024-01-15T17:56:26.201275+0000","last_became_peered":"2024-01-15T17:56:26.201275+0000","last_unstale":"2024-01-15T17:56:27.221853+0000","last_undegraded":"2024-01-15T17:56:27.221853+0000","last_fullsized":"2024-01-15T17:56:27.221853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:30:44.509811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.219828+0000","last_change":"2024-01-15T17:56:27.238165+0000","last_active":"2024-01-15T17:56:28.219828+0000","last_peered":"2024-01-15T17:56:28.219828+0000","last_clean":"2024-01-15T17:56:28.219828+0000","last_became_active":"2024-01-15T17:56:27.237857+0000","last_became_peered":"2024-01-15T17:56:27.237857+0000","last_unstale":"2024-01-15T17:56:28.219828+0000","last_undegraded":"2024-01-15T17:56:28.219828+0000","last_fullsized":"2024-01-15T17:56:28.219828+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:43:42.810052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.205311+0000","last_change":"2024-01-15T17:56:26.205102+0000","last_active":"2024-01-15T17:56:27.205311+0000","last_peered":"2024-01-15T17:56:27.205311+0000","last_clean":"2024-01-15T17:56:27.205311+0000","last_became_active":"2024-01-15T17:56:26.204747+0000","last_became_peered":"2024-01-15T17:56:26.204747+0000","last_unstale":"2024-01-15T17:56:27.205311+0000","last_undegraded":"2024-01-15T17:56:27.205311+0000","last_fullsized":"2024-01-15T17:56:27.205311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:35:06.925527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.223257+0000","last_change":"2024-01-15T17:56:27.233057+0000","last_active":"2024-01-15T17:56:28.223257+0000","last_peered":"2024-01-15T17:56:28.223257+0000","last_clean":"2024-01-15T17:56:28.223257+0000","last_became_active":"2024-01-15T17:56:27.232788+0000","last_became_peered":"2024-01-15T17:56:27.232788+0000","last_unstale":"2024-01-15T17:56:28.223257+0000","last_undegraded":"2024-01-15T17:56:28.223257+0000","last_fullsized":"2024-01-15T17:56:28.223257+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:06:44.454991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.228184+0000","last_change":"2024-01-15T17:56:26.211169+0000","last_active":"2024-01-15T17:56:27.228184+0000","last_peered":"2024-01-15T17:56:27.228184+0000","last_clean":"2024-01-15T17:56:27.228184+0000","last_became_active":"2024-01-15T17:56:26.210842+0000","last_became_peered":"2024-01-15T17:56:26.210842+0000","last_unstale":"2024-01-15T17:56:27.228184+0000","last_undegraded":"2024-01-15T17:56:27.228184+0000","last_fullsized":"2024-01-15T17:56:27.228184+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:45:12.684802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-01-15T17:56:28.222077+0000","last_change":"2024-01-15T17:56:27.224266+0000","last_active":"2024-01-15T17:56:28.222077+0000","last_peered":"2024-01-15T17:56:28.222077+0000","last_clean":"2024-01-15T17:56:28.222077+0000","last_became_active":"2024-01-15T17:56:27.224003+0000","last_became_peered":"2024-01-15T17:56:27.224003+0000","last_unstale":"2024-01-15T17:56:28.222077+0000","last_undegraded":"2024-01-15T17:56:28.222077+0000","last_fullsized":"2024-01-15T17:56:28.222077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:26.164505+0000","last_clean_scrub_stamp":"2024-01-15T17:56:26.164505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:00:20.875369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.222048+0000","last_change":"2024-01-15T17:56:26.208654+0000","last_active":"2024-01-15T17:56:27.222048+0000","last_peered":"2024-01-15T17:56:27.222048+0000","last_clean":"2024-01-15T17:56:27.222048+0000","last_became_active":"2024-01-15T17:56:26.208305+0000","last_became_peered":"2024-01-15T17:56:26.208305+0000","last_unstale":"2024-01-15T17:56:27.222048+0000","last_undegraded":"2024-01-15T17:56:27.222048+0000","last_fullsized":"2024-01-15T17:56:27.222048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:56:25.153705+0000","last_clean_scrub_stamp":"2024-01-15T17:56:25.153705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:38:55.244293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-01-15T17:56:27.215281+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:56:27.215281+0000","last_peered":"2024-01-15T17:56:27.215281+0000","last_clean":"2024-01-15T17:56:27.215281+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:56:27.215281+0000","last_undegraded":"2024-01-15T17:56:27.215281+0000","last_fullsized":"2024-01-15T17:56:27.215281+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":745178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.522},{"interface":"front","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.4490000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.341}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.391,"5min":1.391,"15min":1.391},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":1.9950000000000001},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.052,"5min":1.052,"15min":1.052},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.601}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"min":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.446,"15min":1.446},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.486}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.6060000000000001,"5min":1.6060000000000001,"15min":1.6060000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.6499999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.714,"15min":1.714},"min":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"max":{"1min":2.556,"5min":2.556,"15min":2.556},"last":1.794},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":1.175}]}]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":155336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.734,"5min":1.734,"15min":1.734},"last":1.47}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.9139999999999999,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.3999999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.738},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"max":{"1min":2.0259999999999998,"5min":2.0259999999999998,"15min":2.0259999999999998},"last":1.157}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":1.589},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.323}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.524},{"interface":"front","average":{"1min":1.593,"5min":1.593,"15min":1.593},"min":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"max":{"1min":2.6099999999999999,"5min":2.6099999999999999,"15min":2.6099999999999999},"last":0.94699999999999995}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.758,"5min":1.758,"15min":1.758},"min":{"1min":1.202,"5min":1.202,"15min":1.202},"max":{"1min":3.4830000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.6339999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.044,"5min":1.044,"15min":1.044},"max":{"1min":2.4550000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.7649999999999999},"min":{"1min":1.286,"5min":1.286,"15min":1.286},"max":{"1min":3.141,"5min":3.141,"15min":3.141},"last":1.9570000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542165,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":155392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":5.3840000000000003,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":5.2290000000000001,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":0.999}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.2050000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.226,"15min":1.226},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.835,"5min":2.835,"15min":2.835},"last":1.534},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.46}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.3640000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.419,"15min":1.419},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.649},{"interface":"front","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.4119999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"min":{"1min":1.083,"5min":1.083,"15min":1.083},"max":{"1min":5.218,"5min":5.218,"15min":5.218},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"max":{"1min":2.1259999999999999,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.76}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"min":{"1min":1.2,"5min":1.2,"15min":1.2},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"min":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"max":{"1min":5.3259999999999996,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.71}]}]},{"osd":4,"up_from":14,"seq":60129542165,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":154824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.371}]},{"osd":1,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3400000000000001,"15min":1.3400000000000001},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":1.139},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.524}]},{"osd":2,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"max":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"last":1.042},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.93300000000000005}]},{"osd":3,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.587,"15min":1.587},"min":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":2.056},{"interface":"front","average":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.2370000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":2.1869999999999998},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"min":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.6559999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.706,"5min":1.706,"15min":1.706},"min":{"1min":1.117,"5min":1.117,"15min":1.117},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.9830000000000001},{"interface":"front","average":{"1min":1.845,"5min":1.845,"15min":1.845},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":2.2909999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"min":{"1min":1.385,"5min":1.385,"15min":1.385},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.861},{"interface":"front","average":{"1min":1.929,"5min":1.929,"15min":1.929},"min":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"max":{"1min":2.669,"5min":2.669,"15min":2.669},"last":2.4169999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":409600,"data_stored":156354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.94699999999999995}]},{"osd":1,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.75},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.536}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.3519999999999999,"5min":2.3519999999999999,"15min":2.3519999999999999},"last":1.2889999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.5309999999999999,"15min":1.5309999999999999},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.4159999999999999,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.619,"5min":1.619,"15min":1.619},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":1.6459999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.107},{"interface":"front","average":{"1min":1.673,"5min":1.673,"15min":1.673},"min":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.421}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":1.698,"15min":1.698},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.351},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.7170000000000001,"15min":1.7170000000000001},"min":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.1970000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":397312,"data_stored":154790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":1.1120000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":0.66200000000000003}]},{"osd":3,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":2.2799999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4550000000000001,"15min":1.4550000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.5350000000000001,"5min":2.5350000000000001,"15min":2.5350000000000001},"last":1.0189999999999999}]},{"osd":4,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":1.617,"15min":1.617},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":4.008,"5min":4.008,"15min":4.008},"last":1.073},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.2929999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"min":{"1min":1.165,"5min":1.165,"15min":1.165},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.254}]},{"osd":6,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.494,"15min":1.494},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.98799999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:55:57 2024","interfaces":[{"interface":"back","average":{"1min":1.629,"5min":1.629,"15min":1.629},"min":{"1min":1.042,"5min":1.042,"15min":1.042},"max":{"1min":3.6070000000000002,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.04},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"min":{"1min":1.218,"5min":1.218,"15min":1.218},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":1.087}]}]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":409600,"data_stored":156354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.84199999999999997}]},{"osd":2,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.4159999999999999}]},{"osd":3,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.23,"5min":2.23,"15min":2.23},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":22.106999999999999,"5min":22.106999999999999,"15min":22.106999999999999},"last":1.286},{"interface":"front","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":1.4399999999999999}]},{"osd":4,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.7889999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":9.8510000000000009,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.6499999999999999},{"interface":"front","average":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":14.359999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.625}]},{"osd":5,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":13.228999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.708},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":4.9900000000000002,"5min":4.9900000000000002,"15min":4.9900000000000002},"last":1.4850000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":14.272,"5min":14.272,"15min":14.272},"last":1.552},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":2.7759999999999998,"5min":2.7759999999999998,"15min":2.7759999999999998},"last":1.3320000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:55:59 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":1.077,"5min":1.077,"15min":1.077},"max":{"1min":2.1549999999999998,"5min":2.1549999999999998,"15min":2.1549999999999998},"last":1.597},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.419,"5min":2.419,"15min":2.419},"last":1.371}]}]},{"osd":0,"up_from":14,"seq":60129542167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":987136,"data_stored":745088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.756,"5min":1.756,"15min":1.756},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.96299999999999997}]},{"osd":2,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.70699999999999996}]},{"osd":3,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.569}]},{"osd":4,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":3.7010000000000001,"5min":3.7010000000000001,"15min":3.7010000000000001},"last":1.3300000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.1819999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.679}]},{"osd":6,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.9990000000000001,"5min":2.9990000000000001,"15min":2.9990000000000001},"last":1.478},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.28}]},{"osd":7,"last update":"Mon Jan 15 17:56:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.8100000000000001,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.448},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.4700000000000002,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.7929999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:56:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T17:56:37.319 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T17:56:37.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.587+0000 7f6068c00640 1 Processor -- start 2024-01-15T17:56:37.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.587+0000 7f6068c00640 1 -- start start 2024-01-15T17:56:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.589+0000 7f6068c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6064157cb0 0x7f60640ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.589+0000 7f6068c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f606406d8c0 0x7f606406ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.589+0000 7f6068c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60640713f0 con 0x7f60640cec20 2024-01-15T17:56:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.589+0000 7f6068c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6064158090 con 0x7f606406d8c0 2024-01-15T17:56:37.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.589+0000 7f6068c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60640cfd70 con 0x7f6064157cb0 2024-01-15T17:56:37.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.590+0000 7f6062800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f606406d8c0 0x7f606406ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.590+0000 7f6061e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6064157cb0 0x7f60640ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.590+0000 7f6062800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f606406d8c0 0x7f606406ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:35876/0 (socket says 172.21.15.26:35876) 2024-01-15T17:56:37.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.590+0000 7f6062800640 1 -- 172.21.15.26:0/4177150407 learned_addr learned my addr 172.21.15.26:0/4177150407 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:37.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.591+0000 7f6062800640 1 -- 172.21.15.26:0/4177150407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6064157cb0 msgr2=0x7f60640ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:37.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.591+0000 7f6062800640 1 --2- 172.21.15.26:0/4177150407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6064157cb0 0x7f60640ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.591+0000 7f6062800640 1 -- 172.21.15.26:0/4177150407 >> v1:172.21.15.124:6789/0 conn(0x7f60640cec20 legacy=0x7f606406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.591+0000 7f6062800640 1 -- 172.21.15.26:0/4177150407 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6064071c90 con 0x7f606406d8c0 2024-01-15T17:56:37.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.591+0000 7f6061e00640 1 --2- 172.21.15.26:0/4177150407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6064157cb0 0x7f60640ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:56:37.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.592+0000 7f6062800640 1 --2- 172.21.15.26:0/4177150407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f606406d8c0 0x7f606406ecd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f605c009e50 tx=0x7f605c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb8c50088387501e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.593+0000 7f6061400640 1 -- 172.21.15.26:0/4177150407 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c02cb00 con 0x7f606406d8c0 2024-01-15T17:56:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.593+0000 7f6061400640 1 -- 172.21.15.26:0/4177150407 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c02cc60 con 0x7f606406d8c0 2024-01-15T17:56:37.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.593+0000 7f6061400640 1 -- 172.21.15.26:0/4177150407 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c0343f0 con 0x7f606406d8c0 2024-01-15T17:56:37.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.594+0000 7f6068c00640 1 -- 172.21.15.26:0/4177150407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f606406d8c0 msgr2=0x7f606406ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.594+0000 7f6068c00640 1 --2- 172.21.15.26:0/4177150407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f606406d8c0 0x7f606406ecd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f605c009e50 tx=0x7f605c02bf60 comp rx=0 tx=0).stop 2024-01-15T17:56:37.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.594+0000 7f6068c00640 1 -- 172.21.15.26:0/4177150407 shutdown_connections 2024-01-15T17:56:37.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.594+0000 7f6068c00640 1 --2- 172.21.15.26:0/4177150407 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f606406d8c0 0x7f606406ecd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.594+0000 7f6068c00640 1 --2- 172.21.15.26:0/4177150407 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6064157cb0 0x7f60640ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.594+0000 7f6068c00640 1 -- 172.21.15.26:0/4177150407 >> 172.21.15.26:0/4177150407 conn(0x7f60640b1c60 msgr2=0x7f60640b2040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:37.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.594+0000 7f6068c00640 1 -- 172.21.15.26:0/4177150407 shutdown_connections 2024-01-15T17:56:37.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.594+0000 7f6068c00640 1 -- 172.21.15.26:0/4177150407 wait complete. 2024-01-15T17:56:37.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.596+0000 7f6068c00640 1 Processor -- start 2024-01-15T17:56:37.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.596+0000 7f6068c00640 1 -- start start 2024-01-15T17:56:37.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.597+0000 7f6068c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606406d8c0 0x7f6064148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.597+0000 7f6068c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60640cec20 0x7f606415df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.597+0000 7f6068c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6064157cb0 0x7f606415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.597+0000 7f6068c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640c9fd0 con 0x7f6064157cb0 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.597+0000 7f6068c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640ca2b0 con 0x7f60640cec20 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.597+0000 7f6068c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60640ca590 con 0x7f606406d8c0 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6062800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6064157cb0 0x7f606415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6062800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6064157cb0 0x7f606415e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47808/0 (socket says 172.21.15.26:47808) 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6062800640 1 -- 172.21.15.26:0/2251988856 learned_addr learned my addr 172.21.15.26:0/2251988856 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f605be00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60640cec20 0x7f606415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6061e00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606406d8c0 0x7f6064148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6062800640 1 -- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606406d8c0 msgr2=0x7f6064148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6062800640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606406d8c0 0x7f6064148930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6062800640 1 -- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60640cec20 msgr2=0x7f606415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6062800640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60640cec20 0x7f606415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6062800640 1 -- 172.21.15.26:0/2251988856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6054009400 con 0x7f6064157cb0 2024-01-15T17:56:37.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f605be00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60640cec20 0x7f606415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.598+0000 7f6061e00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606406d8c0 0x7f6064148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.599+0000 7f6062800640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6064157cb0 0x7f606415e4c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f605c009e50 tx=0x7f605c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.599+0000 7f605a000640 1 -- 172.21.15.26:0/2251988856 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c03e450 con 0x7f6064157cb0 2024-01-15T17:56:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.599+0000 7f605a000640 1 -- 172.21.15.26:0/2251988856 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c03e5b0 con 0x7f6064157cb0 2024-01-15T17:56:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.599+0000 7f605a000640 1 -- 172.21.15.26:0/2251988856 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c03e960 con 0x7f6064157cb0 2024-01-15T17:56:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.599+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c009b30 con 0x7f6064157cb0 2024-01-15T17:56:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.599+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60640cab50 con 0x7f6064157cb0 2024-01-15T17:56:37.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.601+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6064067670 con 0x7f6064157cb0 2024-01-15T17:56:37.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.601+0000 7f605a000640 1 -- 172.21.15.26:0/2251988856 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f605c03eac0 con 0x7f6064157cb0 2024-01-15T17:56:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.602+0000 7f605a000640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603c06b2c0 0x7f603c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:37.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.602+0000 7f605a000640 1 -- 172.21.15.26:0/2251988856 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f605c0ac7f0 con 0x7f6064157cb0 2024-01-15T17:56:37.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.603+0000 7f6061e00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603c06b2c0 0x7f603c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:37.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.604+0000 7f6061e00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603c06b2c0 0x7f603c06d780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6054009c80 tx=0x7f6054008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:37.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.607+0000 7f605a000640 1 -- 172.21.15.26:0/2251988856 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f605c077660 con 0x7f6064157cb0 2024-01-15T17:56:37.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.871+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f60640cad10 con 0x7f6064157cb0 2024-01-15T17:56:37.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.876+0000 7f605a000640 1 -- 172.21.15.26:0/2251988856 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f605c0430b0 con 0x7f6064157cb0 2024-01-15T17:56:37.880 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-15T17:56:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA== 2024-01-15T17:56:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-01-15T17:56:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-01-15T17:56:37.880 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T17:56:37.880 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-01-15T17:56:37.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603c06b2c0 msgr2=0x7f603c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603c06b2c0 0x7f603c06d780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6054009c80 tx=0x7f6054008040 comp rx=0 tx=0).stop 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6064157cb0 msgr2=0x7f606415e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6064157cb0 0x7f606415e4c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f605c009e50 tx=0x7f605c004a40 comp rx=0 tx=0).stop 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 shutdown_connections 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f603c06b2c0 0x7f603c06d780 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6064157cb0 0x7f606415e4c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60640cec20 0x7f606415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 --2- 172.21.15.26:0/2251988856 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f606406d8c0 0x7f6064148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 >> 172.21.15.26:0/2251988856 conn(0x7f60640b1c60 msgr2=0x7f606406f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 shutdown_connections 2024-01-15T17:56:37.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:37.879+0000 7f6068c00640 1 -- 172.21.15.26:0/2251988856 wait complete. 2024-01-15T17:56:37.901 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:56:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.111+0000 7fde2d400640 1 Processor -- start 2024-01-15T17:56:38.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.111+0000 7fde2d400640 1 -- start start 2024-01-15T17:56:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.113+0000 7fde2d400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde28162430 0x7fde280b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.113+0000 7fde2d400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde28162810 0x7fde28071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.113+0000 7fde2d400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde280702e0 con 0x7fde280b3650 2024-01-15T17:56:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.113+0000 7fde2d400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde280705c0 con 0x7fde28162430 2024-01-15T17:56:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.113+0000 7fde2d400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde280708a0 con 0x7fde28162810 2024-01-15T17:56:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.114+0000 7fde27e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde28162430 0x7fde280b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.114+0000 7fde27e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde28162430 0x7fde280b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47816/0 (socket says 172.21.15.26:47816) 2024-01-15T17:56:38.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.114+0000 7fde27e00640 1 -- 172.21.15.26:0/1325682757 learned_addr learned my addr 172.21.15.26:0/1325682757 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:38.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.114+0000 7fde27e00640 1 -- 172.21.15.26:0/1325682757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde28162810 msgr2=0x7fde28071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:38.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.114+0000 7fde27e00640 1 --2- 172.21.15.26:0/1325682757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde28162810 0x7fde28071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.114+0000 7fde27e00640 1 -- 172.21.15.26:0/1325682757 >> v1:172.21.15.124:6789/0 conn(0x7fde280b3650 legacy=0x7fde281650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:38.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.114+0000 7fde27e00640 1 -- 172.21.15.26:0/1325682757 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde28070b80 con 0x7fde28162430 2024-01-15T17:56:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.115+0000 7fde27e00640 1 --2- 172.21.15.26:0/1325682757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde28162430 0x7fde280b3110 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fde18009e50 tx=0x7fde1802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c3b2aea303ab37 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.116+0000 7fde26a00640 1 -- 172.21.15.26:0/1325682757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1802ca70 con 0x7fde28162430 2024-01-15T17:56:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.116+0000 7fde26a00640 1 -- 172.21.15.26:0/1325682757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1802cbd0 con 0x7fde28162430 2024-01-15T17:56:38.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.116+0000 7fde26a00640 1 -- 172.21.15.26:0/1325682757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde180343f0 con 0x7fde28162430 2024-01-15T17:56:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.116+0000 7fde2d400640 1 -- 172.21.15.26:0/1325682757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde28162430 msgr2=0x7fde280b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.116+0000 7fde2d400640 1 --2- 172.21.15.26:0/1325682757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde28162430 0x7fde280b3110 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fde18009e50 tx=0x7fde1802bed0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.117+0000 7fde2d400640 1 -- 172.21.15.26:0/1325682757 shutdown_connections 2024-01-15T17:56:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.117+0000 7fde2d400640 1 --2- 172.21.15.26:0/1325682757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde28162810 0x7fde28071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.117+0000 7fde2d400640 1 --2- 172.21.15.26:0/1325682757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde28162430 0x7fde280b3110 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.117+0000 7fde2d400640 1 -- 172.21.15.26:0/1325682757 >> 172.21.15.26:0/1325682757 conn(0x7fde2806a800 msgr2=0x7fde2806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.117+0000 7fde2d400640 1 -- 172.21.15.26:0/1325682757 shutdown_connections 2024-01-15T17:56:38.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.117+0000 7fde2d400640 1 -- 172.21.15.26:0/1325682757 wait complete. 2024-01-15T17:56:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.119+0000 7fde2d400640 1 Processor -- start 2024-01-15T17:56:38.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.119+0000 7fde2d400640 1 -- start start 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.121+0000 7fde2d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde280b3650 0x7fde28143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.121+0000 7fde2d400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde28162430 0x7fde2815b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.121+0000 7fde2d400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde2815b840 0x7fde28163810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.121+0000 7fde2d400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2815bcf0 con 0x7fde280b3650 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.121+0000 7fde2d400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde281645e0 con 0x7fde2815b840 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.121+0000 7fde2d400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde28143e30 con 0x7fde28162430 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.121+0000 7fde27e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde280b3650 0x7fde28143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.121+0000 7fde27e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde280b3650 0x7fde28143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47826/0 (socket says 172.21.15.26:47826) 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.121+0000 7fde27e00640 1 -- 172.21.15.26:0/328372648 learned_addr learned my addr 172.21.15.26:0/328372648 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde27e00640 1 -- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde28162430 msgr2=0x7fde2815b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:38.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde2ca00640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde2815b840 0x7fde28163810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde27e00640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde28162430 0x7fde2815b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde27e00640 1 -- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde2815b840 msgr2=0x7fde28163810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde27e00640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde2815b840 0x7fde28163810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde27e00640 1 -- 172.21.15.26:0/328372648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde18009b30 con 0x7fde280b3650 2024-01-15T17:56:38.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde27e00640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde280b3650 0x7fde28143520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fde18009e50 tx=0x7fde18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde1fe00640 1 -- 172.21.15.26:0/328372648 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1803e450 con 0x7fde280b3650 2024-01-15T17:56:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde1fe00640 1 -- 172.21.15.26:0/328372648 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1803e5b0 con 0x7fde280b3650 2024-01-15T17:56:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde1fe00640 1 -- 172.21.15.26:0/328372648 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1803e960 con 0x7fde280b3650 2024-01-15T17:56:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.122+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde28144110 con 0x7fde280b3650 2024-01-15T17:56:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.123+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde281446d0 con 0x7fde280b3650 2024-01-15T17:56:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.124+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde28164750 con 0x7fde280b3650 2024-01-15T17:56:38.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.124+0000 7fde1fe00640 1 -- 172.21.15.26:0/328372648 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fde1803eac0 con 0x7fde280b3650 2024-01-15T17:56:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.125+0000 7fde1fe00640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde0006b2c0 0x7fde0006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:38.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.125+0000 7fde1fe00640 1 -- 172.21.15.26:0/328372648 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde180acb80 con 0x7fde280b3650 2024-01-15T17:56:38.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.126+0000 7fde27400640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde0006b2c0 0x7fde0006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:38.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.128+0000 7fde27400640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde0006b2c0 0x7fde0006d780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fde28067c30 tx=0x7fde10006bf0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:38.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.130+0000 7fde1fe00640 1 -- 172.21.15.26:0/328372648 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde18077950 con 0x7fde280b3650 2024-01-15T17:56:38.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.363+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde28067450 con 0x7fde280b3650 2024-01-15T17:56:38.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.364+0000 7fde1fe00640 1 -- 172.21.15.26:0/328372648 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fde1807af70 con 0x7fde280b3650 2024-01-15T17:56:38.368 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:56:38.368 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1651915477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":1651915477},{"type":"v1","addr":"172.21.15.124:6837","nonce":1651915477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/506946946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":506946946},{"type":"v1","addr":"172.21.15.26:6833","nonce":506946946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3855677701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3855677701},{"type":"v1","addr":"172.21.15.26:6835","nonce":3855677701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:56:26.648174+0000","modified":"2024-01-15T17:56:29.738878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34619},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34619":{"gid":34619,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.124:6836/2548050157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2548050157},{"type":"v1","addr":"172.21.15.124:6836","nonce":2548050157}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-01-15T17:56:38.368 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 16 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde0006b2c0 msgr2=0x7fde0006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde0006b2c0 0x7fde0006d780 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fde28067c30 tx=0x7fde10006bf0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde280b3650 msgr2=0x7fde28143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde280b3650 0x7fde28143520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fde18009e50 tx=0x7fde18004a40 comp rx=0 tx=0).stop 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 shutdown_connections 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde28162430 0x7fde2815b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde0006b2c0 0x7fde0006d780 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde2815b840 0x7fde28163810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 --2- 172.21.15.26:0/328372648 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde280b3650 0x7fde28143520 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.367+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 >> 172.21.15.26:0/328372648 conn(0x7fde2806a800 msgr2=0x7fde2806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:38.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.368+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 shutdown_connections 2024-01-15T17:56:38.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:38.368+0000 7fde2d400640 1 -- 172.21.15.26:0/328372648 wait complete. 2024-01-15T17:56:38.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T17:56:26.648174+0000', 'modified': '2024-01-15T17:56:29.738878+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34619}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34619': {'gid': 34619, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.124:6836/2548050157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6834', 'nonce': 2548050157}, {'type': 'v1', 'addr': '172.21.15.124:6836', 'nonce': 2548050157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T17:56:38.794 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T17:56:38.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-15T17:56:39.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.024+0000 7f77ba600640 1 Processor -- start 2024-01-15T17:56:39.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.025+0000 7f77ba600640 1 -- start start 2024-01-15T17:56:39.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.026+0000 7f77ba600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f77b40cc990 0x7f77b406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.026+0000 7f77ba600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b4070e50 con 0x7f77b40cc500 2024-01-15T17:56:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.026+0000 7f77ba600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b4144020 con 0x7f77b4143c40 2024-01-15T17:56:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.026+0000 7f77ba600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b40723c0 con 0x7f77b40cc990 2024-01-15T17:56:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f77b40cc500 0x7f77b406d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39808/0 (socket says 172.21.15.26:39808) 2024-01-15T17:56:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3e00640 1 -- 172.21.15.26:0/44402846 learned_addr learned my addr 172.21.15.26:0/44402846 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b9c00640 1 --2- 172.21.15.26:0/44402846 >> v2:172.21.15.124:3300/0 conn(0x7f77b40cc990 0x7f77b406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842280880 0 0) 0x7f77b4070e50 con 0x7f77b40cc500 2024-01-15T17:56:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7794003520 con 0x7f77b40cc500 2024-01-15T17:56:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1150175533 0 0) 0x7f7794003520 con 0x7f77b40cc500 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 >> v2:172.21.15.124:3300/0 conn(0x7f77b40cc990 msgr2=0x7f77b406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3400640 1 --2- 172.21.15.26:0/44402846 >> v2:172.21.15.124:3300/0 conn(0x7f77b40cc990 0x7f77b406d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 >> v1:172.21.15.124:6789/0 conn(0x7f77b4143c40 legacy=0x7f77b4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b4071e00 con 0x7f77b40cc500 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.027+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f779c001670 con 0x7f77b40cc500 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 >> v1:172.21.15.26:6789/0 conn(0x7f77b40cc500 legacy=0x7f77b406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b3400640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7794003970 0x7f7794003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b3400640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77940047c0 0x7f7794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b3400640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f779400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40723c0 con 0x7f7794005180 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779400c120 con 0x7f77940047c0 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779400c2e0 con 0x7f7794003970 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b9c00640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f779400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b3e00640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77940047c0 0x7f7794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b9c00640 1 -- 172.21.15.26:0/44402846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7794003970 msgr2=0x7f7794003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b9c00640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7794003970 0x7f7794003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b9c00640 1 -- 172.21.15.26:0/44402846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77940047c0 msgr2=0x7f7794004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b9c00640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77940047c0 0x7f7794004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b9c00640 1 -- 172.21.15.26:0/44402846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7794003520 con 0x7f7794005180 2024-01-15T17:56:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.028+0000 7f77b3e00640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77940047c0 0x7f7794004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.029+0000 7f77b9c00640 1 -- 172.21.15.26:0/44402846 reap_dead start 2024-01-15T17:56:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.029+0000 7f77b9c00640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f779400b980 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009fc0 tx=0x7f77940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732de3154679631b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.030+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac02cc60 con 0x7f7794005180 2024-01-15T17:56:39.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.030+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac00f040 con 0x7f7794005180 2024-01-15T17:56:39.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.030+0000 7f77b3400640 1 -- 172.21.15.26:0/44402846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac003990 con 0x7f7794005180 2024-01-15T17:56:39.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.030+0000 7f77ba600640 1 -- 172.21.15.26:0/44402846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 msgr2=0x7f779400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.030+0000 7f77ba600640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f779400b980 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f77ac009fc0 tx=0x7f77940057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.030+0000 7f77ba600640 1 -- 172.21.15.26:0/44402846 shutdown_connections 2024-01-15T17:56:39.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.030+0000 7f77ba600640 1 --2- 172.21.15.26:0/44402846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f779400b980 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.030+0000 7f77ba600640 1 -- 172.21.15.26:0/44402846 >> 172.21.15.26:0/44402846 conn(0x7f77b406a800 msgr2=0x7f77b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:39.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.031+0000 7f77ba600640 1 -- 172.21.15.26:0/44402846 shutdown_connections 2024-01-15T17:56:39.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.031+0000 7f77ba600640 1 -- 172.21.15.26:0/44402846 wait complete. 2024-01-15T17:56:39.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.032+0000 7f77ba600640 1 Processor -- start 2024-01-15T17:56:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.032+0000 7f77ba600640 1 -- start start 2024-01-15T17:56:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.033+0000 7f77ba600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f77b40aa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.033+0000 7f77ba600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77b40ca590 0x7f77b40a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.033+0000 7f77ba600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77b40a9940 0x7f77b41306c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.033+0000 7f77ba600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b41302a0 con 0x7f7794005180 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.033+0000 7f77ba600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40c3de0 con 0x7f77b40a9940 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.033+0000 7f77ba600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b412f620 con 0x7f77b40ca590 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f77b40aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f77b40aa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47830/0 (socket says 172.21.15.26:47830) 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9200640 1 -- 172.21.15.26:0/4204254466 learned_addr learned my addr 172.21.15.26:0/4204254466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9c00640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77b40a9940 0x7f77b41306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b3e00640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77b40ca590 0x7f77b40a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9200640 1 -- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77b40ca590 msgr2=0x7f77b40a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9200640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77b40ca590 0x7f77b40a9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9200640 1 -- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77b40a9940 msgr2=0x7f77b41306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9200640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77b40a9940 0x7f77b41306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9200640 1 -- 172.21.15.26:0/4204254466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b412f900 con 0x7f7794005180 2024-01-15T17:56:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.034+0000 7f77b9c00640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77b40a9940 0x7f77b41306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:39.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.035+0000 7f77b9200640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f77b40aa100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f77a400c400 tx=0x7f77a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.035+0000 7f77b0c00640 1 -- 172.21.15.26:0/4204254466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a40052d0 con 0x7f7794005180 2024-01-15T17:56:39.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.035+0000 7f77b0c00640 1 -- 172.21.15.26:0/4204254466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4005470 con 0x7f7794005180 2024-01-15T17:56:39.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.035+0000 7f77b0c00640 1 -- 172.21.15.26:0/4204254466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4003830 con 0x7f7794005180 2024-01-15T17:56:39.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.035+0000 7f77ba600640 1 -- 172.21.15.26:0/4204254466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b412fbe0 con 0x7f7794005180 2024-01-15T17:56:39.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.035+0000 7f77ba600640 1 -- 172.21.15.26:0/4204254466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b40a9dc0 con 0x7f7794005180 2024-01-15T17:56:39.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.038+0000 7f77b0c00640 1 -- 172.21.15.26:0/4204254466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f77a4003990 con 0x7f7794005180 2024-01-15T17:56:39.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.038+0000 7f77b0c00640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f778c06b2f0 0x7f778c06d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.039+0000 7f77b0c00640 1 -- 172.21.15.26:0/4204254466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77a4088d00 con 0x7f7794005180 2024-01-15T17:56:39.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.039+0000 7f77b3e00640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f778c06b2f0 0x7f778c06d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.039+0000 7f77be97e740 1 Processor -- start 2024-01-15T17:56:39.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.039+0000 7f77be97e740 1 -- start start 2024-01-15T17:56:39.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.040+0000 7f77b3e00640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f778c06b2f0 0x7f778c06d7b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77b40ae4a0 tx=0x7f779c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.040+0000 7f77be97e740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a547b070 0x55f9a547b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.040+0000 7f77be97e740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a5475d70 0x55f9a547a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.040+0000 7f77be97e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54be730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.040+0000 7f77be97e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a53d9880 con 0x55f9a547a7b0 2024-01-15T17:56:39.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.040+0000 7f77be97e740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a53e8ae0 con 0x55f9a547b070 2024-01-15T17:56:39.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.041+0000 7f77be97e740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9a53e61d0 con 0x55f9a5475d70 2024-01-15T17:56:39.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.041+0000 7f77b9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54be730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.041+0000 7f77b9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54be730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47842/0 (socket says 172.21.15.26:47842) 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.041+0000 7f77b9c00640 1 -- 172.21.15.26:0/3582068279 learned_addr learned my addr 172.21.15.26:0/3582068279 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.041+0000 7f77b9200640 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a547b070 0x55f9a547b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.041+0000 7f77b3e00640 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a5475d70 0x55f9a547a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77b9c00640 1 -- 172.21.15.26:0/3582068279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a5475d70 msgr2=0x55f9a547a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77b9c00640 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a5475d70 0x55f9a547a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77b9c00640 1 -- 172.21.15.26:0/3582068279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a547b070 msgr2=0x55f9a547b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77b9c00640 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a547b070 0x55f9a547b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77b9c00640 1 -- 172.21.15.26:0/3582068279 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9a53a9860 con 0x55f9a547a7b0 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77b3e00640 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a5475d70 0x55f9a547a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77b9c00640 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54be730 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00e200 tx=0x7f77ac02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72c1773e2c2a125 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77a9600640 1 -- 172.21.15.26:0/3582068279 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac02cb20 con 0x55f9a547a7b0 2024-01-15T17:56:39.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77a9600640 1 -- 172.21.15.26:0/3582068279 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac003990 con 0x55f9a547a7b0 2024-01-15T17:56:39.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.042+0000 7f77a9600640 1 -- 172.21.15.26:0/3582068279 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac02cc80 con 0x55f9a547a7b0 2024-01-15T17:56:39.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.043+0000 7f77be97e740 1 -- 172.21.15.26:0/3582068279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 msgr2=0x55f9a54be730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.043+0000 7f77be97e740 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54be730 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00e200 tx=0x7f77ac02e070 comp rx=0 tx=0).stop 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.043+0000 7f77be97e740 1 -- 172.21.15.26:0/3582068279 shutdown_connections 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.043+0000 7f77be97e740 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a5475d70 0x55f9a547a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.043+0000 7f77be97e740 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54be730 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.043+0000 7f77be97e740 1 --2- 172.21.15.26:0/3582068279 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a547b070 0x55f9a547b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.043+0000 7f77be97e740 1 -- 172.21.15.26:0/3582068279 >> 172.21.15.26:0/3582068279 conn(0x55f9a543cc70 msgr2=0x55f9a54c0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.043+0000 7f77be97e740 1 -- 172.21.15.26:0/3582068279 shutdown_connections 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.043+0000 7f77be97e740 1 -- 172.21.15.26:0/3582068279 wait complete. 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.044+0000 7f77be97e740 1 -- start start 2024-01-15T17:56:39.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.044+0000 7f77be97e740 1 Processor -- start 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.045+0000 7f77be97e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a5475d70 0x55f9a54d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.045+0000 7f77be97e740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547a7b0 0x55f9a54bf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.045+0000 7f77be97e740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a547b070 0x55f9a549c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.045+0000 7f77be97e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a53e1ac0 con 0x55f9a5475d70 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.045+0000 7f77be97e740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a537e2f0 con 0x55f9a547b070 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.045+0000 7f77be97e740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9a539bc70 con 0x55f9a547a7b0 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.045+0000 7f77be97e740 10 client.0 _ll_register_callbacks cb 0x55f9a53a03b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.045+0000 7f77b9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a5475d70 0x55f9a54d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77b9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a5475d70 0x55f9a54d4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47846/0 (socket says 172.21.15.26:47846) 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77b9200640 1 -- 172.21.15.26:0/1646571566 learned_addr learned my addr 172.21.15.26:0/1646571566 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77b9200640 1 -- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547a7b0 msgr2=0x55f9a54bf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77b9200640 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547a7b0 0x55f9a54bf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77b9200640 1 -- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a547b070 msgr2=0x55f9a549c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77b9200640 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a547b070 0x55f9a549c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77b9200640 1 -- 172.21.15.26:0/1646571566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7794003520 con 0x55f9a5475d70 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77b9200640 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a5475d70 0x55f9a54d4790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f77a405f5f0 tx=0x7f77a405fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3268cefc370818 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77a2000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77a2000640 1 -- 172.21.15.26:0/1646571566 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a405f890 con 0x55f9a5475d70 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77a2000640 1 -- 172.21.15.26:0/1646571566 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77a4058b50 con 0x55f9a5475d70 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.046+0000 7f77be97e740 1 -- 172.21.15.26:0/1646571566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9a53daaa0 con 0x55f9a5475d70 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77a2000640 1 -- 172.21.15.26:0/1646571566 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77a4058d10 con 0x55f9a5475d70 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77be97e740 10 client.5439 fetch_fsmap learned FSMap version 16 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77be97e740 1 -- 172.21.15.26:0/1646571566 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f9a53e0140 con 0x55f9a5475d70 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77a2000640 1 -- 172.21.15.26:0/1646571566 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4075070 con 0x55f9a5475d70 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77a2000640 1 -- 172.21.15.26:0/1646571566 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f77a40536c0 con 0x55f9a5475d70 2024-01-15T17:56:39.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77be97e740 10 client.5439 fetch_fsmap finished waiting for FSMap version 16 2024-01-15T17:56:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77be97e740 10 client.5439 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-15T17:56:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77be97e740 20 client.5439 populate_metadata read hostname 'smithi026' 2024-01-15T17:56:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77be97e740 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] conn(0x55f9a54c3880 0x55f9a54c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77be97e740 4 client.5439 mds_command: new command op to 34619 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T17:56:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.047+0000 7f77be97e740 1 -- 172.21.15.26:0/1646571566 --> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9a53da7c0 con 0x55f9a54c3880 2024-01-15T17:56:39.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.048+0000 7f77b3e00640 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] conn(0x55f9a54c3880 0x55f9a54c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.049+0000 7f77b3e00640 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] conn(0x55f9a54c3880 0x55f9a54c1c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d10cf1b40d7060f4 server_cookie=1f92d33042833f8f in_seq=0 out_seq=0 2024-01-15T17:56:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.049+0000 7f77a2000640 10 client.5439 ms_handle_connect on v2:172.21.15.124:6834/2548050157 2024-01-15T17:56:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77a2000640 1 -- 172.21.15.26:0/1646571566 <== mds.0 v2:172.21.15.124:6834/2548050157 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9a53da7c0 con 0x55f9a54c3880 2024-01-15T17:56:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77a2000640 10 client.5439 handle_command_reply: tid=0 2024-01-15T17:56:39.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 client.5439 shutdown 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 -- 172.21.15.26:0/1646571566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a5475d70 msgr2=0x55f9a54d4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a5475d70 0x55f9a54d4790 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f77a405f5f0 tx=0x7f77a405fbe0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 -- 172.21.15.26:0/1646571566 shutdown_connections 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] conn(0x55f9a54c3880 0x55f9a54c1c90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547a7b0 0x55f9a54bf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a547b070 0x55f9a549c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 --2- 172.21.15.26:0/1646571566 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a5475d70 0x55f9a54d4790 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77a2000640 0 client.5439 ms_handle_reset on v2:172.21.15.124:6834/2548050157 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 -- 172.21.15.26:0/1646571566 >> 172.21.15.26:0/1646571566 conn(0x55f9a543cc70 msgr2=0x55f9a5474dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 -- 172.21.15.26:0/1646571566 shutdown_connections 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.051+0000 7f77be97e740 1 -- 172.21.15.26:0/1646571566 wait complete. 2024-01-15T17:56:39.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.052+0000 7f77be97e740 20 client.5439 trim_cache size 0 max 16384 2024-01-15T17:56:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.069+0000 7f77be97e740 1 Processor -- start 2024-01-15T17:56:39.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.069+0000 7f77be97e740 1 -- start start 2024-01-15T17:56:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77be97e740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 0x55f9a5497c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77be97e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a5474dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77be97e740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547b070 0x55f9a5475310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77be97e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a5144f20 con 0x55f9a547a7b0 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77be97e740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a547c9b0 con 0x55f9a5475d70 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77be97e740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9a54c9030 con 0x55f9a547b070 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a5474dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a5474dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47848/0 (socket says 172.21.15.26:47848) 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77b3e00640 1 -- 172.21.15.26:0/715008715 learned_addr learned my addr 172.21.15.26:0/715008715 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77b3e00640 1 -- 172.21.15.26:0/715008715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547b070 msgr2=0x55f9a5475310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77b3e00640 1 --2- 172.21.15.26:0/715008715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547b070 0x55f9a5475310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77b3e00640 1 -- 172.21.15.26:0/715008715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 msgr2=0x55f9a5497c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77b3e00640 1 --2- 172.21.15.26:0/715008715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 0x55f9a5497c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.070+0000 7f77b3e00640 1 -- 172.21.15.26:0/715008715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9a537db80 con 0x55f9a547a7b0 2024-01-15T17:56:39.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.071+0000 7f77b3e00640 1 --2- 172.21.15.26:0/715008715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a5474dd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f779c000f60 tx=0x7f779c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbf1a6b43c5c776 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.071+0000 7f77aa000640 1 -- 172.21.15.26:0/715008715 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c00c070 con 0x55f9a547a7b0 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.071+0000 7f77aa000640 1 -- 172.21.15.26:0/715008715 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c00d040 con 0x55f9a547a7b0 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.071+0000 7f77aa000640 1 -- 172.21.15.26:0/715008715 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c023440 con 0x55f9a547a7b0 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 -- 172.21.15.26:0/715008715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 msgr2=0x55f9a5474dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 --2- 172.21.15.26:0/715008715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a5474dd0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f779c000f60 tx=0x7f779c011000 comp rx=0 tx=0).stop 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 -- 172.21.15.26:0/715008715 shutdown_connections 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 --2- 172.21.15.26:0/715008715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547b070 0x55f9a5475310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 --2- 172.21.15.26:0/715008715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a5474dd0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 --2- 172.21.15.26:0/715008715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 0x55f9a5497c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 -- 172.21.15.26:0/715008715 >> 172.21.15.26:0/715008715 conn(0x55f9a543cc70 msgr2=0x55f9a54c0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 -- 172.21.15.26:0/715008715 shutdown_connections 2024-01-15T17:56:39.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 -- 172.21.15.26:0/715008715 wait complete. 2024-01-15T17:56:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.072+0000 7f77be97e740 1 -- start start 2024-01-15T17:56:39.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.073+0000 7f77be97e740 1 Processor -- start 2024-01-15T17:56:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77be97e740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 0x55f9a5532ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77be97e740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77be97e740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547b070 0x55f9a549f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77be97e740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a54c27a0 con 0x55f9a547a7b0 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77be97e740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9a54c7d80 con 0x55f9a5475d70 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77be97e740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9a547bf70 con 0x55f9a547b070 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77be97e740 10 client.0 _ll_register_callbacks cb 0x7f77a405b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54c75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47862/0 (socket says 172.21.15.26:47862) 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b3e00640 1 -- 172.21.15.26:0/250700959 learned_addr learned my addr 172.21.15.26:0/250700959 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b3e00640 1 -- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547b070 msgr2=0x55f9a549f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b3e00640 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547b070 0x55f9a549f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b3e00640 1 -- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 msgr2=0x55f9a5532ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b9200640 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 0x55f9a5532ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b3e00640 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 0x55f9a5532ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b3e00640 1 -- 172.21.15.26:0/250700959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c01d040 con 0x55f9a547a7b0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.074+0000 7f77b9200640 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 0x55f9a5532ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.075+0000 7f77b3e00640 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54c75c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f779c00e590 tx=0x7f779c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457cd9e679993e01 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.075+0000 7f77a2000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.075+0000 7f77a2000640 1 -- 172.21.15.26:0/250700959 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c023790 con 0x55f9a547a7b0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.075+0000 7f77a2000640 1 -- 172.21.15.26:0/250700959 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f779c0070b0 con 0x55f9a547a7b0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.075+0000 7f77be97e740 1 -- 172.21.15.26:0/250700959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77ac003990 con 0x55f9a547a7b0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.075+0000 7f77a2000640 1 -- 172.21.15.26:0/250700959 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f779c0238f0 con 0x55f9a547a7b0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.075+0000 7f77be97e740 10 client.5445 fetch_fsmap learned FSMap version 16 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.075+0000 7f77be97e740 1 -- 172.21.15.26:0/250700959 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55f9a53a3170 con 0x55f9a547a7b0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.075+0000 7f77a2000640 1 -- 172.21.15.26:0/250700959 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac003990 con 0x55f9a547a7b0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.076+0000 7f77a2000640 1 -- 172.21.15.26:0/250700959 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f779c020070 con 0x55f9a547a7b0 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.076+0000 7f77be97e740 10 client.5445 fetch_fsmap finished waiting for FSMap version 16 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.076+0000 7f77be97e740 10 client.5445 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.076+0000 7f77be97e740 20 client.5445 populate_metadata read hostname 'smithi026' 2024-01-15T17:56:39.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.076+0000 7f77be97e740 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] conn(0x55f9a54a26b0 0x55f9a5498ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.076+0000 7f77be97e740 4 client.5445 mds_command: new command op to 34619 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T17:56:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.076+0000 7f77be97e740 1 -- 172.21.15.26:0/250700959 --> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f9a50e26c0 con 0x55f9a54a26b0 2024-01-15T17:56:39.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.076+0000 7f77b9200640 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] conn(0x55f9a54a26b0 0x55f9a5498ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.077+0000 7f77b9200640 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] conn(0x55f9a54a26b0 0x55f9a5498ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf2eb01e27d66ae server_cookie=993c5d75af48ced6 in_seq=0 out_seq=0 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.077+0000 7f77a2000640 10 client.5445 ms_handle_connect on v2:172.21.15.124:6834/2548050157 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77a2000640 1 -- 172.21.15.26:0/250700959 <== mds.0 v2:172.21.15.124:6834/2548050157 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x55f9a50e26c0 con 0x55f9a54a26b0 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77a2000640 10 client.5445 handle_command_reply: tid=0 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 client.5445 shutdown 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 -- 172.21.15.26:0/250700959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 msgr2=0x55f9a54c75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54c75c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f779c00e590 tx=0x7f779c001a00 comp rx=0 tx=0).stop 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 -- 172.21.15.26:0/250700959 shutdown_connections 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:6834/2548050157,v1:172.21.15.124:6836/2548050157] conn(0x55f9a54a26b0 0x55f9a5498ac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x55f9a547b070 0x55f9a549f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x55f9a547a7b0 0x55f9a54c75c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 --2- 172.21.15.26:0/250700959 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x55f9a5475d70 0x55f9a5532ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77a2000640 0 client.5445 ms_handle_reset on v2:172.21.15.124:6834/2548050157 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 -- 172.21.15.26:0/250700959 >> 172.21.15.26:0/250700959 conn(0x55f9a543cc70 msgr2=0x55f9a54e3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 -- 172.21.15.26:0/250700959 shutdown_connections 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 1 -- 172.21.15.26:0/250700959 wait complete. 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.078+0000 7f77be97e740 20 client.5445 trim_cache size 0 max 16384 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-15T17:56:39.084 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 34619, 2024-01-15T17:56:39.085 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-15T17:56:39.085 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-15T17:56:39.085 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-15T17:56:39.085 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 11.85931744, 2024-01-15T17:56:39.085 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 16, 2024-01-15T17:56:39.085 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 35, 2024-01-15T17:56:39.085 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 35, 2024-01-15T17:56:39.085 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 20.702800563 2024-01-15T17:56:39.085 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 -- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f778c06b2f0 msgr2=0x7f778c06d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f778c06b2f0 0x7f778c06d7b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f77b40ae4a0 tx=0x7f779c009040 comp rx=0 tx=0).stop 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 -- 172.21.15.26:0/4204254466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 msgr2=0x7f77b40aa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f77b40aa100 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f77a400c400 tx=0x7f77a400c8c0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 -- 172.21.15.26:0/4204254466 shutdown_connections 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77b40ca590 0x7f77b40a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f778c06b2f0 0x7f778c06d7b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77b40a9940 0x7f77b41306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 --2- 172.21.15.26:0/4204254466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7794005180 0x7f77b40aa100 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 -- 172.21.15.26:0/4204254466 >> 172.21.15.26:0/4204254466 conn(0x7f77b406a800 msgr2=0x7f77b40b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:39.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 -- 172.21.15.26:0/4204254466 shutdown_connections 2024-01-15T17:56:39.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.079+0000 7f77ba600640 1 -- 172.21.15.26:0/4204254466 wait complete. 2024-01-15T17:56:39.099 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T17:56:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.308+0000 7f5b07c00640 1 Processor -- start 2024-01-15T17:56:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.308+0000 7f5b07c00640 1 -- start start 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.310+0000 7f5b07c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5b0013df80 0x7f5b0015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.310+0000 7f5b07c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b000cf9a0 0x7f5b000cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.310+0000 7f5b07c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0006c470 con 0x7f5b0013e360 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.310+0000 7f5b07c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0006c750 con 0x7f5b0013df80 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.310+0000 7f5b07c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0006ca30 con 0x7f5b000cf9a0 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.310+0000 7f5affe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5b0013e360 0x7f5b000cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39810/0 (socket says 172.21.15.26:39810) 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.310+0000 7f5affe00640 1 -- 172.21.15.26:0/2646425213 learned_addr learned my addr 172.21.15.26:0/2646425213 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.310+0000 7f5b05200640 1 --2- 172.21.15.26:0/2646425213 >> v2:172.21.15.124:3300/0 conn(0x7f5b0013df80 0x7f5b0015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5b05c00640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b000cf9a0 0x7f5b000cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671501917 0 0) 0x7f5b0006c470 con 0x7f5b0013e360 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae0003520 con 0x7f5b0013e360 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1609922018 0 0) 0x7f5ae0003520 con 0x7f5b0013e360 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b000cf9a0 msgr2=0x7f5b000cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b000cf9a0 0x7f5b000cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 >> v2:172.21.15.124:3300/0 conn(0x7f5b0013df80 msgr2=0x7f5b0015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 --2- 172.21.15.26:0/2646425213 >> v2:172.21.15.124:3300/0 conn(0x7f5b0013df80 0x7f5b0015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0006cd10 con 0x7f5b0013e360 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f5ae8001670 con 0x7f5b0013e360 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 >> v1:172.21.15.26:6789/0 conn(0x7f5b0013e360 legacy=0x7f5b000cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5ae0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ae00047c0 0x7f5ae0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b000cf9a0 0x7f5ae0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0006c750 con 0x7f5ae0003970 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0006ca30 con 0x7f5ae00047c0 2024-01-15T17:56:39.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.311+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae000b980 con 0x7f5b000cf9a0 2024-01-15T17:56:39.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.312+0000 7f5b05200640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5ae0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.312+0000 7f5b05200640 1 -- 172.21.15.26:0/2646425213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b000cf9a0 msgr2=0x7f5ae0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.312+0000 7f5b05200640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b000cf9a0 0x7f5ae0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.312+0000 7f5b05200640 1 -- 172.21.15.26:0/2646425213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ae00047c0 msgr2=0x7f5ae0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:39.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.312+0000 7f5b05200640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ae00047c0 0x7f5ae0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.312+0000 7f5b05200640 1 -- 172.21.15.26:0/2646425213 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0003520 con 0x7f5ae0003970 2024-01-15T17:56:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.313+0000 7f5b05200640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5ae0003d90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009ac0 tx=0x7f5ae00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c4272168bac56c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.313+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af002c710 con 0x7f5ae0003970 2024-01-15T17:56:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.314+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af002c870 con 0x7f5ae0003970 2024-01-15T17:56:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.314+0000 7f5aff400640 1 -- 172.21.15.26:0/2646425213 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af002cc00 con 0x7f5ae0003970 2024-01-15T17:56:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.314+0000 7f5b07c00640 1 -- 172.21.15.26:0/2646425213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 msgr2=0x7f5ae0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.314+0000 7f5b07c00640 1 --2- 172.21.15.26:0/2646425213 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5ae0003d90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009ac0 tx=0x7f5ae00057f0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.314+0000 7f5b05c00640 1 -- 172.21.15.26:0/2646425213 reap_dead start 2024-01-15T17:56:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.314+0000 7f5b07c00640 1 -- 172.21.15.26:0/2646425213 shutdown_connections 2024-01-15T17:56:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.314+0000 7f5b07c00640 1 -- 172.21.15.26:0/2646425213 >> 172.21.15.26:0/2646425213 conn(0x7f5b0006a800 msgr2=0x7f5b0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.314+0000 7f5b07c00640 1 -- 172.21.15.26:0/2646425213 shutdown_connections 2024-01-15T17:56:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.314+0000 7f5b07c00640 1 -- 172.21.15.26:0/2646425213 wait complete. 2024-01-15T17:56:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.316+0000 7f5b07c00640 1 Processor -- start 2024-01-15T17:56:39.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.316+0000 7f5b07c00640 1 -- start start 2024-01-15T17:56:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.317+0000 7f5b07c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5b00152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.317+0000 7f5b07c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0013a660 0x7f5b000ce070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.317+0000 7f5b07c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0013aae0 0x7f5b000ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.317+0000 7f5b07c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b00152ff0 con 0x7f5ae0003970 2024-01-15T17:56:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.317+0000 7f5b07c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0013b460 con 0x7f5b0013aae0 2024-01-15T17:56:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.317+0000 7f5b07c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0013b740 con 0x7f5b0013a660 2024-01-15T17:56:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.317+0000 7f5b05200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5b00152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5b00152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47882/0 (socket says 172.21.15.26:47882) 2024-01-15T17:56:39.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05200640 1 -- 172.21.15.26:0/2481309821 learned_addr learned my addr 172.21.15.26:0/2481309821 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5affe00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0013a660 0x7f5b000ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05200640 1 -- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0013a660 msgr2=0x7f5b000ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05200640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0013a660 0x7f5b000ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05c00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0013aae0 0x7f5b000ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05200640 1 -- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0013aae0 msgr2=0x7f5b000ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05200640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0013aae0 0x7f5b000ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05200640 1 -- 172.21.15.26:0/2481309821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0013ba20 con 0x7f5ae0003970 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5affe00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0013a660 0x7f5b000ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05c00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0013aae0 0x7f5b000ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.318+0000 7f5b05200640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5b00152ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5af0000c00 tx=0x7f5af0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.319+0000 7f5afcc00640 1 -- 172.21.15.26:0/2481309821 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0003810 con 0x7f5ae0003970 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.319+0000 7f5afcc00640 1 -- 172.21.15.26:0/2481309821 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af0003990 con 0x7f5ae0003970 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.319+0000 7f5afcc00640 1 -- 172.21.15.26:0/2481309821 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0003d40 con 0x7f5ae0003970 2024-01-15T17:56:39.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.319+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0013bd00 con 0x7f5ae0003970 2024-01-15T17:56:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.319+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0013aff0 con 0x7f5ae0003970 2024-01-15T17:56:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.320+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b000675d0 con 0x7f5ae0003970 2024-01-15T17:56:39.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.321+0000 7f5afcc00640 1 -- 172.21.15.26:0/2481309821 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f5af003b070 con 0x7f5ae0003970 2024-01-15T17:56:39.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.322+0000 7f5afcc00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ad806b2c0 0x7f5ad806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:39.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.322+0000 7f5afcc00640 1 -- 172.21.15.26:0/2481309821 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5af00ab9c0 con 0x7f5ae0003970 2024-01-15T17:56:39.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.323+0000 7f5affe00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ad806b2c0 0x7f5ad806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:39.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.324+0000 7f5affe00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ad806b2c0 0x7f5ad806d780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8000c00 tx=0x7f5ae8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:39.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.327+0000 7f5afcc00640 1 -- 172.21.15.26:0/2481309821 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5af0076a00 con 0x7f5ae0003970 2024-01-15T17:56:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.559+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5b000b42a0 con 0x7f5ae0003970 2024-01-15T17:56:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.560+0000 7f5afcc00640 1 -- 172.21.15.26:0/2481309821 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5af0039070 con 0x7f5ae0003970 2024-01-15T17:56:39.563 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T17:56:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ad806b2c0 msgr2=0x7f5ad806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ad806b2c0 0x7f5ad806d780 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8000c00 tx=0x7f5ae8009040 comp rx=0 tx=0).stop 2024-01-15T17:56:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 msgr2=0x7f5b00152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5b00152ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5af0000c00 tx=0x7f5af0004730 comp rx=0 tx=0).stop 2024-01-15T17:56:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 shutdown_connections 2024-01-15T17:56:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b0013a660 0x7f5b000ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5ad806b2c0 0x7f5ad806d780 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b0013aae0 0x7f5b000ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 --2- 172.21.15.26:0/2481309821 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ae0003970 0x7f5b00152ab0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 >> 172.21.15.26:0/2481309821 conn(0x7f5b0006a800 msgr2=0x7f5b000cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 shutdown_connections 2024-01-15T17:56:39.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:39.563+0000 7f5b07c00640 1 -- 172.21.15.26:0/2481309821 wait complete. 2024-01-15T17:56:39.585 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T17:56:39.585 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T17:56:39.585 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-15T17:56:39.585 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T17:56:39.585 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T17:56:39.585 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T17:56:39.585 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T17:56:39.585 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T17:56:39.585 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T17:56:39.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:39.586 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-15T17:56:39.640 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:39.641 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42518sec preferred_lft 42518sec 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::a4e5:3ff:fe91:fcad/64 scope link 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:56:39.642 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:56:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::f464:8fff:fecd:58d2/64 scope link 2024-01-15T17:56:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:39.643 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:56:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether fe:82:c8:bc:3f:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:56:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::fc82:c8ff:febc:3fd2/64 scope link 2024-01-15T17:56:39.643 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:39.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:39.644 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T17:56:39.735 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-15T17:56:39.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:39.740 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-15T17:56:39.795 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-15T17:56:39.796 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T17:56:39.796 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-15T17:56:39.796 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-15T17:56:39.796 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T17:56:39.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:39.797 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:56:39.797 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:56:39.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-15T17:56:39.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-15T17:56:39.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-15T17:56:39.797 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:56:39.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:39.993 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T17:56:40.047 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T17:56:40.047 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T17:56:40.048 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T17:56:40.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:40.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-15T17:56:40.151 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T17:56:40.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:56:40.151 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T17:56:40.206 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T17:56:40.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:56:40.206 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T17:56:40.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:56:40.363 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:56:40.421 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:56:40.421 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:56:40.421 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:56:40.421 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:56:40.421 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:56:40.421 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:56:40.422 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:56:40.423 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:56:40.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:40.424 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:56:40.480 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T17:56:40.480 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T17:56:40.487 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:56:40.572 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:56:40.614 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:56:40.614 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:56:40.614 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:56:40.615 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:56:40.616 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:56:40.617 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:56:40.617 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:56:40.617 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:56:40.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:40.618 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:56:40.682 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:2024-01-15T17:56:40.679+0000 7f9eb92d3480 -1 init, newargv = 0x556b5a5b4a60 newargc=15 2024-01-15T17:56:40.683 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[49154]: starting ceph client 2024-01-15T17:56:40.693 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[49154]: starting fuse 2024-01-15T17:56:40.705 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T17:56:40.705 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T17:56:40.705 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-01-15T17:56:40.706 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T17:56:40.707 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T17:56:40.707 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T17:56:40.707 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-01-15T17:56:40.707 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-01-15T17:56:40.707 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-01-15T17:56:40.707 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T17:56:40.707 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:56:40.707 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.0")) 2024-01-15T17:56:40.707 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-01-15T17:56:40.904 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.0.49154.asok 2024-01-15T17:56:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T17:56:40.923 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49154.asok 2024-01-15T17:56:40.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:40.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49154.asok status 2024-01-15T17:56:41.169 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:56:41.169 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-01-15T17:56:41.169 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T17:56:41.169 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "0", 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "49154", 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5454, 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-01-15T17:56:41.170 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 5454 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2777528500 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2777528500 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:56:41.171 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.5454 192.168.0.1:0/2777528500", 2024-01-15T17:56:41.172 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/2777528500", 2024-01-15T17:56:41.172 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-01-15T17:56:41.172 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 16, 2024-01-15T17:56:41.172 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 35, 2024-01-15T17:56:41.172 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-01-15T17:56:41.172 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-01-15T17:56:41.172 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-01-15T17:56:41.172 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:56:41.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:41.185 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-01-15T17:56:41.185 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T17:56:41.242 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-01-15T17:56:41.242 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T17:56:41.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:41.242 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T17:56:41.342 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T17:56:41.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.565+0000 7f3f0b200640 1 Processor -- start 2024-01-15T17:56:41.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.565+0000 7f3f0b200640 1 -- start start 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f0b200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3f0c143c40 0x7f3f0c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f0b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f0c0717d0 0x7f3f0c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f0b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0c070e50 con 0x7f3f0c0cca40 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f0b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c0723c0 con 0x7f3f0c0717d0 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f0b200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0c0cce40 con 0x7f3f0c143c40 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f03e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f0c0717d0 0x7f3f0c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f03e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3f0c0717d0 0x7f3f0c0cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47898/0 (socket says 172.21.15.26:47898) 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f03e00640 1 -- 172.21.15.26:0/1462394025 learned_addr learned my addr 172.21.15.26:0/1462394025 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f0a800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3f0c0cca40 0x7f3f0c15eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39820/0 (socket says 172.21.15.26:39820) 2024-01-15T17:56:41.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.567+0000 7f3f09e00640 1 --2- 172.21.15.26:0/1462394025 >> v2:172.21.15.124:3300/0 conn(0x7f3f0c143c40 0x7f3f0c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.568+0000 7f3f03e00640 1 -- 172.21.15.26:0/1462394025 >> v2:172.21.15.124:3300/0 conn(0x7f3f0c143c40 msgr2=0x7f3f0c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.568+0000 7f3f03e00640 1 --2- 172.21.15.26:0/1462394025 >> v2:172.21.15.124:3300/0 conn(0x7f3f0c143c40 0x7f3f0c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.568+0000 7f3f03e00640 1 -- 172.21.15.26:0/1462394025 >> v1:172.21.15.26:6789/0 conn(0x7f3f0c0cca40 legacy=0x7f3f0c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.568+0000 7f3f03e00640 1 -- 172.21.15.26:0/1462394025 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c06e170 con 0x7f3f0c0717d0 2024-01-15T17:56:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.568+0000 7f3f09400640 1 -- 172.21.15.26:0/1462394025 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961273873 0 0) 0x7f3f0c070e50 con 0x7f3f0c0cca40 2024-01-15T17:56:41.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.568+0000 7f3f09e00640 1 --2- 172.21.15.26:0/1462394025 >> v2:172.21.15.124:3300/0 conn(0x7f3f0c143c40 0x7f3f0c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.569+0000 7f3f03e00640 1 --2- 172.21.15.26:0/1462394025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0717d0 0x7f3f0c0cc500 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06e410 tx=0x7f3ef402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2302847fe8883290 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.569+0000 7f3f09400640 1 -- 172.21.15.26:0/1462394025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4031870 con 0x7f3f0c0717d0 2024-01-15T17:56:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.569+0000 7f3f09400640 1 -- 172.21.15.26:0/1462394025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40319d0 con 0x7f3f0c0717d0 2024-01-15T17:56:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.569+0000 7f3f09400640 1 -- 172.21.15.26:0/1462394025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4031d60 con 0x7f3f0c0717d0 2024-01-15T17:56:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.570+0000 7f3f0b200640 1 -- 172.21.15.26:0/1462394025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0717d0 msgr2=0x7f3f0c0cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.570+0000 7f3f0b200640 1 --2- 172.21.15.26:0/1462394025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0717d0 0x7f3f0c0cc500 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c06e410 tx=0x7f3ef402b910 comp rx=0 tx=0).stop 2024-01-15T17:56:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.570+0000 7f3f0b200640 1 -- 172.21.15.26:0/1462394025 shutdown_connections 2024-01-15T17:56:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.570+0000 7f3f0b200640 1 --2- 172.21.15.26:0/1462394025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0717d0 0x7f3f0c0cc500 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.570+0000 7f3f0b200640 1 --2- 172.21.15.26:0/1462394025 >> v2:172.21.15.124:3300/0 conn(0x7f3f0c143c40 0x7f3f0c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.570+0000 7f3f0b200640 1 -- 172.21.15.26:0/1462394025 >> 172.21.15.26:0/1462394025 conn(0x7f3f0c06a800 msgr2=0x7f3f0c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.570+0000 7f3f0b200640 1 -- 172.21.15.26:0/1462394025 shutdown_connections 2024-01-15T17:56:41.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.570+0000 7f3f0b200640 1 -- 172.21.15.26:0/1462394025 wait complete. 2024-01-15T17:56:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.572+0000 7f3f0b200640 1 Processor -- start 2024-01-15T17:56:41.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.572+0000 7f3f0b200640 1 -- start start 2024-01-15T17:56:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.573+0000 7f3f0b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f0c0717d0 0x7f3f0c13c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.573+0000 7f3f0b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0cca40 0x7f3f0c13e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.573+0000 7f3f0b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c143c40 0x7f3f0c13e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.573+0000 7f3f0b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c152140 con 0x7f3f0c0cca40 2024-01-15T17:56:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.573+0000 7f3f0b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c152420 con 0x7f3f0c0717d0 2024-01-15T17:56:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.573+0000 7f3f0b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0c152700 con 0x7f3f0c143c40 2024-01-15T17:56:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.573+0000 7f3f03e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0cca40 0x7f3f0c13e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:41.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f03e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0cca40 0x7f3f0c13e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47912/0 (socket says 172.21.15.26:47912) 2024-01-15T17:56:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f03e00640 1 -- 172.21.15.26:0/1841986855 learned_addr learned my addr 172.21.15.26:0/1841986855 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f09e00640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f0c0717d0 0x7f3f0c13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f0a800640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c143c40 0x7f3f0c13e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f03e00640 1 -- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c143c40 msgr2=0x7f3f0c13e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f03e00640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c143c40 0x7f3f0c13e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f03e00640 1 -- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f0c0717d0 msgr2=0x7f3f0c13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f03e00640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f0c0717d0 0x7f3f0c13c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f03e00640 1 -- 172.21.15.26:0/1841986855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef40094d0 con 0x7f3f0c0cca40 2024-01-15T17:56:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.574+0000 7f3f03e00640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0cca40 0x7f3f0c13e360 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3ef402b950 tx=0x7f3ef40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.575+0000 7f3f02000640 1 -- 172.21.15.26:0/1841986855 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4031860 con 0x7f3f0c0cca40 2024-01-15T17:56:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.575+0000 7f3f02000640 1 -- 172.21.15.26:0/1841986855 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef40443f0 con 0x7f3f0c0cca40 2024-01-15T17:56:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.575+0000 7f3f02000640 1 -- 172.21.15.26:0/1841986855 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40447a0 con 0x7f3f0c0cca40 2024-01-15T17:56:41.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.575+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0c1529e0 con 0x7f3f0c0cca40 2024-01-15T17:56:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.575+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f0c13cb40 con 0x7f3f0c0cca40 2024-01-15T17:56:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.576+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0c067450 con 0x7f3f0c0cca40 2024-01-15T17:56:41.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.577+0000 7f3f02000640 1 -- 172.21.15.26:0/1841986855 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f3ef404d450 con 0x7f3f0c0cca40 2024-01-15T17:56:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.579+0000 7f3f02000640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ee406b2c0 0x7f3ee406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:41.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.579+0000 7f3f02000640 1 -- 172.21.15.26:0/1841986855 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ef402d040 con 0x7f3f0c0cca40 2024-01-15T17:56:41.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.579+0000 7f3f09e00640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ee406b2c0 0x7f3ee406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:41.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.580+0000 7f3f09e00640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ee406b2c0 0x7f3ee406d780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00a950 tx=0x7f3efc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:41.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.584+0000 7f3f02000640 1 -- 172.21.15.26:0/1841986855 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ef4077210 con 0x7f3f0c0cca40 2024-01-15T17:56:41.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.816+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3f0c0cdf80 con 0x7f3f0c0cca40 2024-01-15T17:56:41.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.817+0000 7f3f02000640 1 -- 172.21.15.26:0/1841986855 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3ef4035060 con 0x7f3f0c0cca40 2024-01-15T17:56:41.821 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T17:56:41.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ee406b2c0 msgr2=0x7f3ee406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ee406b2c0 0x7f3ee406d780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00a950 tx=0x7f3efc008040 comp rx=0 tx=0).stop 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0cca40 msgr2=0x7f3f0c13e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0cca40 0x7f3f0c13e360 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3ef402b950 tx=0x7f3ef40046c0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 shutdown_connections 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3f0c143c40 0x7f3f0c13e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ee406b2c0 0x7f3ee406d780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3f0c0cca40 0x7f3f0c13e360 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 --2- 172.21.15.26:0/1841986855 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3f0c0717d0 0x7f3f0c13c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.819+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 >> 172.21.15.26:0/1841986855 conn(0x7f3f0c06a800 msgr2=0x7f3f0c0b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.820+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 shutdown_connections 2024-01-15T17:56:41.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:41.820+0000 7f3f0b200640 1 -- 172.21.15.26:0/1841986855 wait complete. 2024-01-15T17:56:41.843 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T17:56:41.843 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T17:56:41.843 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2024-01-15T17:56:41.843 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T17:56:41.843 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T17:56:41.843 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T17:56:41.843 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T17:56:41.843 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T17:56:41.843 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T17:56:41.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:41.844 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2024-01-15T17:56:41.903 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42514sec preferred_lft 42514sec 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:41.904 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4c2d:94ff:fe47:f54c/64 scope link 2024-01-15T17:56:41.905 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:41.905 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:56:41.905 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 26:4f:4f:5b:0b:31 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:56:41.905 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::244f:4fff:fe5b:b31/64 scope link 2024-01-15T17:56:41.905 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:41.905 INFO:teuthology.orchestra.run.smithi124.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:56:41.905 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:56:41.905 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::2070:16ff:fe92:4a4a/64 scope link 2024-01-15T17:56:41.905 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:56:41.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:41.906 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T17:56:41.994 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T17:56:41.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:41.999 DEBUG:teuthology.orchestra.run.smithi124:> route 2024-01-15T17:56:42.047 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2024-01-15T17:56:42.047 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T17:56:42.047 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T17:56:42.047 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T17:56:42.047 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T17:56:42.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:42.047 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:56:42.048 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:56:42.048 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T17:56:42.048 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T17:56:42.048 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T17:56:42.048 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:56:42.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:42.222 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T17:56:42.270 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T17:56:42.271 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T17:56:42.271 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T17:56:42.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:42.272 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2024-01-15T17:56:42.363 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T17:56:42.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:56:42.363 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T17:56:42.415 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T17:56:42.415 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:56:42.415 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T17:56:42.463 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T17:56:42.554 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:56:42.610 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T17:56:42.611 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T17:56:42.612 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T17:56:42.612 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:56:42.612 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:56:42.612 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:56:42.612 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:56:42.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:42.613 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T17:56:42.663 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T17:56:42.663 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T17:56:42.670 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T17:56:42.743 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:56:42.784 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T17:56:42.785 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:56:42.786 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:56:42.787 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:56:42.787 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:56:42.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:42.788 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T17:56:42.850 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:2024-01-15T17:56:42.847+0000 7fa84b960480 -1 init, newargv = 0x55e0fa942b70 newargc=15 2024-01-15T17:56:42.850 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[43154]: starting ceph client 2024-01-15T17:56:42.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[43154]: starting fuse 2024-01-15T17:56:42.872 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T17:56:42.872 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> sudo stdin-killer -- python3 -c ' 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> import glob 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> import re 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> import os 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> import subprocess 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> def _find_admin_socket(client_name): 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> files = glob.glob(asok_path) 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> # Given a non-glob path, it better be there 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> if "*" not in asok_path: 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> assert(len(files) == 1) 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> return files[0] 2024-01-15T17:56:42.873 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> for f in files: 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> contents = proc_f.read() 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> if mountpoint in contents: 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> return f 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> print(_find_admin_socket("client.1")) 2024-01-15T17:56:42.874 DEBUG:teuthology.orchestra.run.smithi124:> ' 2024-01-15T17:56:43.066 INFO:teuthology.orchestra.run.smithi124.stdout:/var/run/ceph/ceph-client.1.43154.asok 2024-01-15T17:56:43.071 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:56:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T17:56:43.083 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43154.asok 2024-01-15T17:56:43.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:43.084 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43154.asok status 2024-01-15T17:56:43.323 INFO:teuthology.orchestra.run.smithi124.stdout:{ 2024-01-15T17:56:43.323 INFO:teuthology.orchestra.run.smithi124.stdout: "metadata": { 2024-01-15T17:56:43.323 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: "entity_id": "1", 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: "hostname": "smithi124", 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: "pid": "43154", 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: "root": "/" 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_count": 0, 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_pinned_count": 0, 2024-01-15T17:56:43.324 INFO:teuthology.orchestra.run.smithi124.stdout: "id": 34724, 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: "inst": { 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: "name": { 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "client", 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: "num": 34724 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 3835980547 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: } 2024-01-15T17:56:43.325 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 3835980547 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "inst_str": "client.34724 192.168.0.1:0/3835980547", 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "addr_str": "192.168.0.1:0/3835980547", 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "inode_count": 1, 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "mds_epoch": 16, 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch": 35, 2024-01-15T17:56:43.326 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch_barrier": 0, 2024-01-15T17:56:43.327 INFO:teuthology.orchestra.run.smithi124.stdout: "blocklisted": false, 2024-01-15T17:56:43.327 INFO:teuthology.orchestra.run.smithi124.stdout: "fs_name": "cephfs" 2024-01-15T17:56:43.327 INFO:teuthology.orchestra.run.smithi124.stdout:} 2024-01-15T17:56:43.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:43.338 DEBUG:teuthology.orchestra.run.smithi124:> stat --file-system '--printf=%T 2024-01-15T17:56:43.338 DEBUG:teuthology.orchestra.run.smithi124:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T17:56:43.388 INFO:teuthology.orchestra.run.smithi124.stdout:fuseblk 2024-01-15T17:56:43.388 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T17:56:43.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:43.388 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T17:56:43.527 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T17:56:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.762+0000 7fe5e9a00640 1 Processor -- start 2024-01-15T17:56:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.762+0000 7fe5e9a00640 1 -- start start 2024-01-15T17:56:43.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.764+0000 7fe5e9a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe5e40ae5a0 0x7fe5e415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:43.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.764+0000 7fe5e9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5e40cf290 0x7fe5e40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:43.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.764+0000 7fe5e9a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e4070c40 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.764+0000 7fe5e9a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e4071450 con 0x7fe5e40cf290 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.764+0000 7fe5e9a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e40cfc60 con 0x7fe5e40ae5a0 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.764+0000 7fe5e3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5e40cf290 0x7fe5e40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.765+0000 7fe5e3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5e40cf290 0x7fe5e40cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56072/0 (socket says 172.21.15.26:56072) 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.765+0000 7fe5e3400640 1 -- 172.21.15.26:0/3958910241 learned_addr learned my addr 172.21.15.26:0/3958910241 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.765+0000 7fe5e2a00640 1 --2- 172.21.15.26:0/3958910241 >> v2:172.21.15.124:3300/0 conn(0x7fe5e40ae5a0 0x7fe5e415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.765+0000 7fe5e3400640 1 -- 172.21.15.26:0/3958910241 >> v2:172.21.15.124:3300/0 conn(0x7fe5e40ae5a0 msgr2=0x7fe5e415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.765+0000 7fe5e3400640 1 --2- 172.21.15.26:0/3958910241 >> v2:172.21.15.124:3300/0 conn(0x7fe5e40ae5a0 0x7fe5e415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.765+0000 7fe5e3400640 1 -- 172.21.15.26:0/3958910241 >> v1:172.21.15.124:6789/0 conn(0x7fe5e40b3ae0 legacy=0x7fe5e406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:43.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.765+0000 7fe5e3400640 1 -- 172.21.15.26:0/3958910241 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e406c470 con 0x7fe5e40cf290 2024-01-15T17:56:43.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.766+0000 7fe5e3400640 1 --2- 172.21.15.26:0/3958910241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40cf290 0x7fe5e40cf720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40cfe20 tx=0x7fe5dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4b4d664c61ebe8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:43.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.767+0000 7fe5e1600640 1 -- 172.21.15.26:0/3958910241 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc030860 con 0x7fe5e40cf290 2024-01-15T17:56:43.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.767+0000 7fe5e1600640 1 -- 172.21.15.26:0/3958910241 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5dc0309c0 con 0x7fe5e40cf290 2024-01-15T17:56:43.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.767+0000 7fe5e1600640 1 -- 172.21.15.26:0/3958910241 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc030d50 con 0x7fe5e40cf290 2024-01-15T17:56:43.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.767+0000 7fe5e9a00640 1 -- 172.21.15.26:0/3958910241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40cf290 msgr2=0x7fe5e40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.767+0000 7fe5e9a00640 1 --2- 172.21.15.26:0/3958910241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40cf290 0x7fe5e40cf720 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40cfe20 tx=0x7fe5dc02b9a0 comp rx=0 tx=0).stop 2024-01-15T17:56:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.768+0000 7fe5e9a00640 1 -- 172.21.15.26:0/3958910241 shutdown_connections 2024-01-15T17:56:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.768+0000 7fe5e9a00640 1 --2- 172.21.15.26:0/3958910241 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40cf290 0x7fe5e40cf720 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.768+0000 7fe5e9a00640 1 --2- 172.21.15.26:0/3958910241 >> v2:172.21.15.124:3300/0 conn(0x7fe5e40ae5a0 0x7fe5e415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.768+0000 7fe5e9a00640 1 -- 172.21.15.26:0/3958910241 >> 172.21.15.26:0/3958910241 conn(0x7fe5e406a800 msgr2=0x7fe5e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.768+0000 7fe5e9a00640 1 -- 172.21.15.26:0/3958910241 shutdown_connections 2024-01-15T17:56:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.768+0000 7fe5e9a00640 1 -- 172.21.15.26:0/3958910241 wait complete. 2024-01-15T17:56:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.770+0000 7fe5e9a00640 1 Processor -- start 2024-01-15T17:56:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.770+0000 7fe5e9a00640 1 -- start start 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ae5a0 0x7fe5e414f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40b3ae0 0x7fe5e4158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e9a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e40cf290 0x7fe5e414d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e9a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e413ec30 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e9a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e413f050 con 0x7fe5e40ae5a0 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e9a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e413e070 con 0x7fe5e40cf290 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40b3ae0 0x7fe5e4158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40b3ae0 0x7fe5e4158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56088/0 (socket says 172.21.15.26:56088) 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e2000640 1 -- 172.21.15.26:0/267915293 learned_addr learned my addr 172.21.15.26:0/267915293 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:43.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e2a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ae5a0 0x7fe5e414f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e3400640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e40cf290 0x7fe5e414d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e2000640 1 -- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e40cf290 msgr2=0x7fe5e414d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e2000640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e40cf290 0x7fe5e414d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e2000640 1 -- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ae5a0 msgr2=0x7fe5e414f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.771+0000 7fe5e2000640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ae5a0 0x7fe5e414f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.772+0000 7fe5e2000640 1 -- 172.21.15.26:0/267915293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc0094d0 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.772+0000 7fe5e2a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ae5a0 0x7fe5e414f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.772+0000 7fe5e2000640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40b3ae0 0x7fe5e4158a00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c490 tx=0x7fe5cc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.772+0000 7fe5daa00640 1 -- 172.21.15.26:0/267915293 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc012830 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.772+0000 7fe5daa00640 1 -- 172.21.15.26:0/267915293 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5cc012990 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.772+0000 7fe5daa00640 1 -- 172.21.15.26:0/267915293 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc012d40 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.772+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e4158f40 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.772+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e413e290 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.774+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e40675d0 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.775+0000 7fe5daa00640 1 -- 172.21.15.26:0/267915293 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fe5cc01b400 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.776+0000 7fe5daa00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe5bc06b2c0 0x7fe5bc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.776+0000 7fe5daa00640 1 -- 172.21.15.26:0/267915293 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5cc089d60 con 0x7fe5e40b3ae0 2024-01-15T17:56:43.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.776+0000 7fe5e2a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe5bc06b2c0 0x7fe5bc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:43.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.777+0000 7fe5e2a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe5bc06b2c0 0x7fe5bc06d780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4007330 tx=0x7fe5d400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:43.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:43.780+0000 7fe5daa00640 1 -- 172.21.15.26:0/267915293 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5cc054aa0 con 0x7fe5e40b3ae0 2024-01-15T17:56:44.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.015+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe5e4158f40 con 0x7fe5e40b3ae0 2024-01-15T17:56:44.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.903+0000 7fe5daa00640 1 -- 172.21.15.26:0/267915293 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 136+0+0 (secure 0 0 0) 0x7fe5e4158f40 con 0x7fe5e40b3ae0 2024-01-15T17:56:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe5bc06b2c0 msgr2=0x7fe5bc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe5bc06b2c0 0x7fe5bc06d780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4007330 tx=0x7fe5d400b040 comp rx=0 tx=0).stop 2024-01-15T17:56:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40b3ae0 msgr2=0x7fe5e4158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40b3ae0 0x7fe5e4158a00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc00c490 tx=0x7fe5cc00c950 comp rx=0 tx=0).stop 2024-01-15T17:56:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 shutdown_connections 2024-01-15T17:56:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5e40cf290 0x7fe5e414d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe5bc06b2c0 0x7fe5bc06d780 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5e40b3ae0 0x7fe5e4158a00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 --2- 172.21.15.26:0/267915293 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe5e40ae5a0 0x7fe5e414f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 >> 172.21.15.26:0/267915293 conn(0x7fe5e406a800 msgr2=0x7fe5e40b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 shutdown_connections 2024-01-15T17:56:44.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:44.907+0000 7fe5e9a00640 1 -- 172.21.15.26:0/267915293 wait complete. 2024-01-15T17:56:44.934 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T17:56:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.158+0000 7fb2fb400640 1 Processor -- start 2024-01-15T17:56:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.158+0000 7fb2fb400640 1 -- start start 2024-01-15T17:56:45.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.160+0000 7fb2fb400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb2f413cfa0 0x7fb2f415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.160+0000 7fb2fb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2f40cd530 0x7fb2f413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.160+0000 7fb2fb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 0x7fb2f413fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.160+0000 7fb2fb400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f40cdec0 con 0x7fb2f40cd530 2024-01-15T17:56:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.160+0000 7fb2fb400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f40ce1a0 con 0x7fb2f413cfa0 2024-01-15T17:56:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.160+0000 7fb2fb400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f40ce480 con 0x7fb2f40cd9e0 2024-01-15T17:56:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.160+0000 7fb2f9600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2f40cd530 0x7fb2f413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.160+0000 7fb2f9600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb2f40cd530 0x7fb2f413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56090/0 (socket says 172.21.15.26:56090) 2024-01-15T17:56:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.160+0000 7fb2f9600640 1 -- 172.21.15.26:0/4143827979 learned_addr learned my addr 172.21.15.26:0/4143827979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.161+0000 7fb2faa00640 1 --2- 172.21.15.26:0/4143827979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 0x7fb2f413fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.161+0000 7fb2fa000640 1 --2- 172.21.15.26:0/4143827979 >> v2:172.21.15.124:3300/0 conn(0x7fb2f413cfa0 0x7fb2f415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.161+0000 7fb2f9600640 1 -- 172.21.15.26:0/4143827979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 msgr2=0x7fb2f413fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.161+0000 7fb2f9600640 1 --2- 172.21.15.26:0/4143827979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 0x7fb2f413fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.161+0000 7fb2f9600640 1 -- 172.21.15.26:0/4143827979 >> v2:172.21.15.124:3300/0 conn(0x7fb2f413cfa0 msgr2=0x7fb2f415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.161+0000 7fb2f9600640 1 --2- 172.21.15.26:0/4143827979 >> v2:172.21.15.124:3300/0 conn(0x7fb2f413cfa0 0x7fb2f415afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.161+0000 7fb2f9600640 1 -- 172.21.15.26:0/4143827979 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f40ce760 con 0x7fb2f40cd530 2024-01-15T17:56:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.162+0000 7fb2f9600640 1 --2- 172.21.15.26:0/4143827979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f40cd530 0x7fb2f413f5a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb2f406cf00 tx=0x7fb2d802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693efb93a6f2e756 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2f8c00640 1 -- 172.21.15.26:0/4143827979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80317e0 con 0x7fb2f40cd530 2024-01-15T17:56:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2f8c00640 1 -- 172.21.15.26:0/4143827979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d8031940 con 0x7fb2f40cd530 2024-01-15T17:56:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2f8c00640 1 -- 172.21.15.26:0/4143827979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8031cd0 con 0x7fb2f40cd530 2024-01-15T17:56:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2fb400640 1 -- 172.21.15.26:0/4143827979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f40cd530 msgr2=0x7fb2f413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2fb400640 1 --2- 172.21.15.26:0/4143827979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f40cd530 0x7fb2f413f5a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb2f406cf00 tx=0x7fb2d802b880 comp rx=0 tx=0).stop 2024-01-15T17:56:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2fb400640 1 -- 172.21.15.26:0/4143827979 shutdown_connections 2024-01-15T17:56:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2fb400640 1 --2- 172.21.15.26:0/4143827979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 0x7fb2f413fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2fb400640 1 --2- 172.21.15.26:0/4143827979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f40cd530 0x7fb2f413f5a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2fb400640 1 --2- 172.21.15.26:0/4143827979 >> v2:172.21.15.124:3300/0 conn(0x7fb2f413cfa0 0x7fb2f415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.163+0000 7fb2fb400640 1 -- 172.21.15.26:0/4143827979 >> 172.21.15.26:0/4143827979 conn(0x7fb2f406a800 msgr2=0x7fb2f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.164+0000 7fb2fb400640 1 -- 172.21.15.26:0/4143827979 shutdown_connections 2024-01-15T17:56:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.164+0000 7fb2fb400640 1 -- 172.21.15.26:0/4143827979 wait complete. 2024-01-15T17:56:45.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.165+0000 7fb2fb400640 1 Processor -- start 2024-01-15T17:56:45.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.166+0000 7fb2fb400640 1 -- start start 2024-01-15T17:56:45.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.166+0000 7fb2fb400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2f40cd530 0x7fb2f4155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2fb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 0x7fb2f4157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2fb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f413cfa0 0x7fb2f4138760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2fb400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4155d50 con 0x7fb2f413cfa0 2024-01-15T17:56:45.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2fb400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4157650 con 0x7fb2f40cd530 2024-01-15T17:56:45.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2fb400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f414d430 con 0x7fb2f40cd9e0 2024-01-15T17:56:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2faa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f413cfa0 0x7fb2f4138760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2faa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f413cfa0 0x7fb2f4138760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56102/0 (socket says 172.21.15.26:56102) 2024-01-15T17:56:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2faa00640 1 -- 172.21.15.26:0/1892856169 learned_addr learned my addr 172.21.15.26:0/1892856169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2f9600640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 0x7fb2f4157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2fa000640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2f40cd530 0x7fb2f4155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2faa00640 1 -- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 msgr2=0x7fb2f4157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2faa00640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 0x7fb2f4157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2faa00640 1 -- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2f40cd530 msgr2=0x7fb2f4155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2faa00640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2f40cd530 0x7fb2f4155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2faa00640 1 -- 172.21.15.26:0/1892856169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80094d0 con 0x7fb2f413cfa0 2024-01-15T17:56:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.167+0000 7fb2f9600640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 0x7fb2f4157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:56:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.168+0000 7fb2faa00640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f413cfa0 0x7fb2f4138760 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000d600 tx=0x7fb2e000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.168+0000 7fb2f2000640 1 -- 172.21.15.26:0/1892856169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00125f0 con 0x7fb2f413cfa0 2024-01-15T17:56:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.168+0000 7fb2f2000640 1 -- 172.21.15.26:0/1892856169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e0012750 con 0x7fb2f413cfa0 2024-01-15T17:56:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.168+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f413c7a0 con 0x7fb2f413cfa0 2024-01-15T17:56:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.168+0000 7fb2f2000640 1 -- 172.21.15.26:0/1892856169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0012b00 con 0x7fb2f413cfa0 2024-01-15T17:56:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.168+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f414cb00 con 0x7fb2f413cfa0 2024-01-15T17:56:45.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.169+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f414d5a0 con 0x7fb2f413cfa0 2024-01-15T17:56:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.170+0000 7fb2f2000640 1 -- 172.21.15.26:0/1892856169 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7fb2e0012c60 con 0x7fb2f413cfa0 2024-01-15T17:56:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.171+0000 7fb2f2000640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb2c806b2c0 0x7fb2c806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.171+0000 7fb2f2000640 1 -- 172.21.15.26:0/1892856169 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2e008a1a0 con 0x7fb2f413cfa0 2024-01-15T17:56:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.172+0000 7fb2fa000640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb2c806b2c0 0x7fb2c806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.173+0000 7fb2fa000640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb2c806b2c0 0x7fb2c806d780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009940 tx=0x7fb2e8003cd0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:45.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.176+0000 7fb2f2000640 1 -- 172.21.15.26:0/1892856169 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2e0054f70 con 0x7fb2f413cfa0 2024-01-15T17:56:45.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.399+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2f4067450 con 0x7fb2c806b2c0 2024-01-15T17:56:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.403+0000 7fb2f2000640 1 -- 172.21.15.26:0/1892856169 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb2f4067450 con 0x7fb2c806b2c0 2024-01-15T17:56:45.407 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T17:56:45.407 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:56:45.407 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T17:56:45.407 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:56:45.407 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T17:56:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.405+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb2c806b2c0 msgr2=0x7fb2c806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.405+0000 7fb2fb400640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb2c806b2c0 0x7fb2c806d780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009940 tx=0x7fb2e8003cd0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.405+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f413cfa0 msgr2=0x7fb2f4138760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.405+0000 7fb2fb400640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f413cfa0 0x7fb2f4138760 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000d600 tx=0x7fb2e000dac0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.406+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 shutdown_connections 2024-01-15T17:56:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.406+0000 7fb2fb400640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb2f40cd9e0 0x7fb2f4157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.406+0000 7fb2fb400640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb2c806b2c0 0x7fb2c806d780 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.406+0000 7fb2fb400640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb2f413cfa0 0x7fb2f4138760 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.406+0000 7fb2fb400640 1 --2- 172.21.15.26:0/1892856169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb2f40cd530 0x7fb2f4155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.406+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 >> 172.21.15.26:0/1892856169 conn(0x7fb2f406a800 msgr2=0x7fb2f415b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.406+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 shutdown_connections 2024-01-15T17:56:45.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.406+0000 7fb2fb400640 1 -- 172.21.15.26:0/1892856169 wait complete. 2024-01-15T17:56:45.430 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T17:56:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.640+0000 7f5719600640 1 Processor -- start 2024-01-15T17:56:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.640+0000 7f5719600640 1 -- start start 2024-01-15T17:56:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5719600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f571414b2c0 0x7f57140cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5719600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57140cf7d0 0x7f5714070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5719600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5714070ea0 0x7f57140b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5719600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571414b6a0 con 0x7f57140cf7d0 2024-01-15T17:56:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5719600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57140cfbf0 con 0x7f571414b2c0 2024-01-15T17:56:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5719600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5714071690 con 0x7f5714070ea0 2024-01-15T17:56:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5712000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57140cf7d0 0x7f5714070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5712000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f57140cf7d0 0x7f5714070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56108/0 (socket says 172.21.15.26:56108) 2024-01-15T17:56:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5712000640 1 -- 172.21.15.26:0/116186450 learned_addr learned my addr 172.21.15.26:0/116186450 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5712a00640 1 --2- 172.21.15.26:0/116186450 >> v2:172.21.15.124:3300/0 conn(0x7f571414b2c0 0x7f57140cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.642+0000 7f5713400640 1 --2- 172.21.15.26:0/116186450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5714070ea0 0x7f57140b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.643+0000 7f5712000640 1 -- 172.21.15.26:0/116186450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5714070ea0 msgr2=0x7f57140b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.643+0000 7f5712000640 1 --2- 172.21.15.26:0/116186450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5714070ea0 0x7f57140b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.643+0000 7f5712000640 1 -- 172.21.15.26:0/116186450 >> v2:172.21.15.124:3300/0 conn(0x7f571414b2c0 msgr2=0x7f57140cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.643+0000 7f5712000640 1 --2- 172.21.15.26:0/116186450 >> v2:172.21.15.124:3300/0 conn(0x7f571414b2c0 0x7f57140cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.643+0000 7f5712000640 1 -- 172.21.15.26:0/116186450 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5714071970 con 0x7f57140cf7d0 2024-01-15T17:56:45.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.643+0000 7f5712a00640 1 --2- 172.21.15.26:0/116186450 >> v2:172.21.15.124:3300/0 conn(0x7f571414b2c0 0x7f57140cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:56:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.644+0000 7f5712000640 1 --2- 172.21.15.26:0/116186450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57140cf7d0 0x7f5714070960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f571406e650 tx=0x7f56fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca01ef2d61f6cb5a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.644+0000 7f5711600640 1 -- 172.21.15.26:0/116186450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc031870 con 0x7f57140cf7d0 2024-01-15T17:56:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.644+0000 7f5711600640 1 -- 172.21.15.26:0/116186450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc0319d0 con 0x7f57140cf7d0 2024-01-15T17:56:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.644+0000 7f5711600640 1 -- 172.21.15.26:0/116186450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc031d60 con 0x7f57140cf7d0 2024-01-15T17:56:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.645+0000 7f5719600640 1 -- 172.21.15.26:0/116186450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57140cf7d0 msgr2=0x7f5714070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.645+0000 7f5719600640 1 --2- 172.21.15.26:0/116186450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57140cf7d0 0x7f5714070960 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f571406e650 tx=0x7f56fc02b910 comp rx=0 tx=0).stop 2024-01-15T17:56:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.645+0000 7f5719600640 1 -- 172.21.15.26:0/116186450 shutdown_connections 2024-01-15T17:56:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.645+0000 7f5719600640 1 --2- 172.21.15.26:0/116186450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5714070ea0 0x7f57140b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.645+0000 7f5719600640 1 --2- 172.21.15.26:0/116186450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57140cf7d0 0x7f5714070960 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.645+0000 7f5719600640 1 --2- 172.21.15.26:0/116186450 >> v2:172.21.15.124:3300/0 conn(0x7f571414b2c0 0x7f57140cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.645+0000 7f5719600640 1 -- 172.21.15.26:0/116186450 >> 172.21.15.26:0/116186450 conn(0x7f571406a800 msgr2=0x7f571406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.645+0000 7f5719600640 1 -- 172.21.15.26:0/116186450 shutdown_connections 2024-01-15T17:56:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.645+0000 7f5719600640 1 -- 172.21.15.26:0/116186450 wait complete. 2024-01-15T17:56:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.647+0000 7f5719600640 1 Processor -- start 2024-01-15T17:56:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.647+0000 7f5719600640 1 -- start start 2024-01-15T17:56:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5719600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714070ea0 0x7f57141432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5719600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57140cf7d0 0x7f57140b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5719600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f571414b2c0 0x7f57140b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5719600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57141624a0 con 0x7f5714070ea0 2024-01-15T17:56:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5719600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5714145c50 con 0x7f57140cf7d0 2024-01-15T17:56:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5719600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5714145f30 con 0x7f571414b2c0 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5712a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714070ea0 0x7f57141432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5712a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714070ea0 0x7f57141432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56112/0 (socket says 172.21.15.26:56112) 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5712a00640 1 -- 172.21.15.26:0/496802734 learned_addr learned my addr 172.21.15.26:0/496802734 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5712a00640 1 -- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f571414b2c0 msgr2=0x7f57140b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5712a00640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f571414b2c0 0x7f57140b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5712a00640 1 -- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57140cf7d0 msgr2=0x7f57140b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5712a00640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57140cf7d0 0x7f57140b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.649+0000 7f5712a00640 1 -- 172.21.15.26:0/496802734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc0094d0 con 0x7f5714070ea0 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.650+0000 7f5712a00640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714070ea0 0x7f57141432f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f570400e7e0 tx=0x7f570400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.650+0000 7f570aa00640 1 -- 172.21.15.26:0/496802734 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704004030 con 0x7f5714070ea0 2024-01-15T17:56:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.650+0000 7f570aa00640 1 -- 172.21.15.26:0/496802734 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704004190 con 0x7f5714070ea0 2024-01-15T17:56:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.650+0000 7f570aa00640 1 -- 172.21.15.26:0/496802734 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570400c6d0 con 0x7f5714070ea0 2024-01-15T17:56:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.650+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5714162680 con 0x7f5714070ea0 2024-01-15T17:56:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.650+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5714143830 con 0x7f5714070ea0 2024-01-15T17:56:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.651+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5714146150 con 0x7f5714070ea0 2024-01-15T17:56:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.652+0000 7f570aa00640 1 -- 172.21.15.26:0/496802734 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f570400c830 con 0x7f5714070ea0 2024-01-15T17:56:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.653+0000 7f570aa00640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56ec06b2c0 0x7f56ec06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.653+0000 7f570aa00640 1 -- 172.21.15.26:0/496802734 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5704089920 con 0x7f5714070ea0 2024-01-15T17:56:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.654+0000 7f5712000640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56ec06b2c0 0x7f56ec06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:45.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.655+0000 7f5712000640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56ec06b2c0 0x7f56ec06d780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f56fc036000 tx=0x7f56fc030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.658+0000 7f570aa00640 1 -- 172.21.15.26:0/496802734 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5704054780 con 0x7f5714070ea0 2024-01-15T17:56:45.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.858+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f57140675d0 con 0x7f5714070ea0 2024-01-15T17:56:45.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.860+0000 7f570aa00640 1 -- 172.21.15.26:0/496802734 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f57040200c0 con 0x7f5714070ea0 2024-01-15T17:56:45.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56ec06b2c0 msgr2=0x7f56ec06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56ec06b2c0 0x7f56ec06d780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f56fc036000 tx=0x7f56fc030000 comp rx=0 tx=0).stop 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714070ea0 msgr2=0x7f57141432f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714070ea0 0x7f57141432f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f570400e7e0 tx=0x7f570400eca0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 shutdown_connections 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f571414b2c0 0x7f57140b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f56ec06b2c0 0x7f56ec06d780 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57140cf7d0 0x7f57140b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 --2- 172.21.15.26:0/496802734 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5714070ea0 0x7f57141432f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 >> 172.21.15.26:0/496802734 conn(0x7f571406a800 msgr2=0x7f571406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.862+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 shutdown_connections 2024-01-15T17:56:45.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:45.863+0000 7f5719600640 1 -- 172.21.15.26:0/496802734 wait complete. 2024-01-15T17:56:45.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T17:56:46.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.105+0000 7f5912000640 1 Processor -- start 2024-01-15T17:56:46.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.106+0000 7f5912000640 1 -- start start 2024-01-15T17:56:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.107+0000 7f5912000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f590c071410 0x7f590c146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.107+0000 7f5912000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c06c7c0 0x7f590c0717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.108+0000 7f5912000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590c06cc20 con 0x7f590c071d30 2024-01-15T17:56:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.108+0000 7f5912000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590c06cec0 con 0x7f590c071410 2024-01-15T17:56:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.108+0000 7f5912000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c06d1a0 con 0x7f590c06c7c0 2024-01-15T17:56:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.108+0000 7f5910c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f590c071410 0x7f590c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.108+0000 7f5903e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c06c7c0 0x7f590c0717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.108+0000 7f5910c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f590c071410 0x7f590c146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58294/0 (socket says 172.21.15.26:58294) 2024-01-15T17:56:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.108+0000 7f5910c00640 1 -- 172.21.15.26:0/4164940375 learned_addr learned my addr 172.21.15.26:0/4164940375 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:46.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.108+0000 7f5911600640 1 --1- 172.21.15.26:0/4164940375 >> v1:172.21.15.124:6789/0 conn(0x7f590c071d30 0x7f590c0b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:44918/0 (socket says 172.21.15.26:44918) 2024-01-15T17:56:46.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.109+0000 7f5910c00640 1 -- 172.21.15.26:0/4164940375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c06c7c0 msgr2=0x7f590c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:46.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.109+0000 7f5910c00640 1 --2- 172.21.15.26:0/4164940375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c06c7c0 0x7f590c0717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.109+0000 7f5910c00640 1 -- 172.21.15.26:0/4164940375 >> v1:172.21.15.124:6789/0 conn(0x7f590c071d30 legacy=0x7f590c0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.109+0000 7f5910c00640 1 -- 172.21.15.26:0/4164940375 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590c06d480 con 0x7f590c071410 2024-01-15T17:56:46.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.111+0000 7f5910c00640 1 --2- 172.21.15.26:0/4164940375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f590c071410 0x7f590c146770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009870 tx=0x7f58fc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8da8d9c95a436d99 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.112+0000 7f590be00640 1 -- 172.21.15.26:0/4164940375 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc030840 con 0x7f590c071410 2024-01-15T17:56:46.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.112+0000 7f590be00640 1 -- 172.21.15.26:0/4164940375 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc0309a0 con 0x7f590c071410 2024-01-15T17:56:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.112+0000 7f590be00640 1 -- 172.21.15.26:0/4164940375 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc030d30 con 0x7f590c071410 2024-01-15T17:56:46.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.113+0000 7f5912000640 1 -- 172.21.15.26:0/4164940375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f590c071410 msgr2=0x7f590c146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.113+0000 7f5912000640 1 --2- 172.21.15.26:0/4164940375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f590c071410 0x7f590c146770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009870 tx=0x7f58fc02b980 comp rx=0 tx=0).stop 2024-01-15T17:56:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.113+0000 7f5912000640 1 -- 172.21.15.26:0/4164940375 shutdown_connections 2024-01-15T17:56:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.113+0000 7f5912000640 1 --2- 172.21.15.26:0/4164940375 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c06c7c0 0x7f590c0717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.113+0000 7f5912000640 1 --2- 172.21.15.26:0/4164940375 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f590c071410 0x7f590c146770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.113+0000 7f5912000640 1 -- 172.21.15.26:0/4164940375 >> 172.21.15.26:0/4164940375 conn(0x7f590c06a800 msgr2=0x7f590c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.113+0000 7f5912000640 1 -- 172.21.15.26:0/4164940375 shutdown_connections 2024-01-15T17:56:46.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.113+0000 7f5912000640 1 -- 172.21.15.26:0/4164940375 wait complete. 2024-01-15T17:56:46.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.115+0000 7f5912000640 1 Processor -- start 2024-01-15T17:56:46.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.115+0000 7f5912000640 1 -- start start 2024-01-15T17:56:46.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.116+0000 7f5912000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c06c7c0 0x7f590c1f5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.116+0000 7f5912000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c071410 0x7f590c0c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.116+0000 7f5912000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f590c071d30 0x7f590c0c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.116+0000 7f5912000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c0c6d20 con 0x7f590c06c7c0 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.116+0000 7f5912000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590c0c7000 con 0x7f590c071d30 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.116+0000 7f5912000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590c0c72e0 con 0x7f590c071410 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5910c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c06c7c0 0x7f590c1f5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5910c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c06c7c0 0x7f590c1f5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56124/0 (socket says 172.21.15.26:56124) 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5910c00640 1 -- 172.21.15.26:0/1253369267 learned_addr learned my addr 172.21.15.26:0/1253369267 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5911600640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f590c071d30 0x7f590c0c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5903e00640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c071410 0x7f590c0c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5910c00640 1 -- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c071410 msgr2=0x7f590c0c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5910c00640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c071410 0x7f590c0c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5910c00640 1 -- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f590c071d30 msgr2=0x7f590c0c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5910c00640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f590c071d30 0x7f590c0c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.117+0000 7f5910c00640 1 -- 172.21.15.26:0/1253369267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc0094d0 con 0x7f590c06c7c0 2024-01-15T17:56:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.118+0000 7f5910c00640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c06c7c0 0x7f590c1f5d70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f58fc000c00 tx=0x7f58fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.118+0000 7f5909600640 1 -- 172.21.15.26:0/1253369267 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc039c90 con 0x7f590c06c7c0 2024-01-15T17:56:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.118+0000 7f5909600640 1 -- 172.21.15.26:0/1253369267 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc039e10 con 0x7f590c06c7c0 2024-01-15T17:56:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.118+0000 7f5909600640 1 -- 172.21.15.26:0/1253369267 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc043640 con 0x7f590c06c7c0 2024-01-15T17:56:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.118+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590c0c75c0 con 0x7f590c06c7c0 2024-01-15T17:56:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.118+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590c0c6910 con 0x7f590c06c7c0 2024-01-15T17:56:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.119+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590c067450 con 0x7f590c06c7c0 2024-01-15T17:56:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.120+0000 7f5909600640 1 -- 172.21.15.26:0/1253369267 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7f58fc0437a0 con 0x7f590c06c7c0 2024-01-15T17:56:46.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.121+0000 7f5909600640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f58e406b2c0 0x7f58e406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.122+0000 7f5909600640 1 -- 172.21.15.26:0/1253369267 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58fc03e070 con 0x7f590c06c7c0 2024-01-15T17:56:46.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.122+0000 7f5903e00640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f58e406b2c0 0x7f58e406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.123+0000 7f5903e00640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f58e406b2c0 0x7f58e406d780 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f590c15f700 tx=0x7f58f4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.126+0000 7f5909600640 1 -- 172.21.15.26:0/1253369267 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58fc077ab0 con 0x7f590c06c7c0 2024-01-15T17:56:46.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.347+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f590c072150 con 0x7f58e406b2c0 2024-01-15T17:56:46.365 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:56:46.361+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:56:46.365 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:56:46.361+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:56:46.365 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:56:46.361+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:56:46.365 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:56:46.361+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:56:46.365 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:56:46.361+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:56:46.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.429+0000 7f5909600640 1 -- 172.21.15.26:0/1253369267 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f58f4009800 con 0x7f58e406b2c0 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f58e406b2c0 msgr2=0x7f58e406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f58e406b2c0 0x7f58e406d780 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f590c15f700 tx=0x7f58f4008040 comp rx=0 tx=0).stop 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c06c7c0 msgr2=0x7f590c1f5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c06c7c0 0x7f590c1f5d70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f58fc000c00 tx=0x7f58fc004730 comp rx=0 tx=0).stop 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 shutdown_connections 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f590c071410 0x7f590c0c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f58e406b2c0 0x7f58e406d780 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f590c071d30 0x7f590c0c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 --2- 172.21.15.26:0/1253369267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f590c06c7c0 0x7f590c1f5d70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 >> 172.21.15.26:0/1253369267 conn(0x7f590c06a800 msgr2=0x7f590c0ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 shutdown_connections 2024-01-15T17:56:46.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.432+0000 7f5912000640 1 -- 172.21.15.26:0/1253369267 wait complete. 2024-01-15T17:56:46.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T17:56:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.668+0000 7faa1be00640 1 Processor -- start 2024-01-15T17:56:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.668+0000 7faa1be00640 1 -- start start 2024-01-15T17:56:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.670+0000 7faa1be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c15d4b0 0x7faa1c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.670+0000 7faa1be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa1c0b41e0 0x7faa1c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.670+0000 7faa1be00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c0d0320 con 0x7faa1c0cf9a0 2024-01-15T17:56:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.670+0000 7faa1be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c0b2730 con 0x7faa1c0b41e0 2024-01-15T17:56:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.670+0000 7faa1be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0d1ec0 con 0x7faa1c15d4b0 2024-01-15T17:56:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.670+0000 7faa1b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa1c0b41e0 0x7faa1c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.670+0000 7faa1b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa1c0b41e0 0x7faa1c0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56130/0 (socket says 172.21.15.26:56130) 2024-01-15T17:56:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.670+0000 7faa1b400640 1 -- 172.21.15.26:0/628159616 learned_addr learned my addr 172.21.15.26:0/628159616 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.671+0000 7faa1aa00640 1 --2- 172.21.15.26:0/628159616 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c15d4b0 0x7faa1c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.671+0000 7faa1b400640 1 -- 172.21.15.26:0/628159616 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c15d4b0 msgr2=0x7faa1c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:56:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.671+0000 7faa1b400640 1 --2- 172.21.15.26:0/628159616 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c15d4b0 0x7faa1c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.671+0000 7faa1b400640 1 -- 172.21.15.26:0/628159616 >> v1:172.21.15.124:6789/0 conn(0x7faa1c0cf9a0 legacy=0x7faa1c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.671+0000 7faa1b400640 1 -- 172.21.15.26:0/628159616 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c0b29d0 con 0x7faa1c0b41e0 2024-01-15T17:56:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.672+0000 7faa1b400640 1 --2- 172.21.15.26:0/628159616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c0b41e0 0x7faa1c0cfde0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7faa140098a0 tx=0x7faa1402b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3050be294b3f59 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.672+0000 7faa1a000640 1 -- 172.21.15.26:0/628159616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14030830 con 0x7faa1c0b41e0 2024-01-15T17:56:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1a000640 1 -- 172.21.15.26:0/628159616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14030990 con 0x7faa1c0b41e0 2024-01-15T17:56:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1a000640 1 -- 172.21.15.26:0/628159616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14030d20 con 0x7faa1c0b41e0 2024-01-15T17:56:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1be00640 1 -- 172.21.15.26:0/628159616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c0b41e0 msgr2=0x7faa1c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1be00640 1 --2- 172.21.15.26:0/628159616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c0b41e0 0x7faa1c0cfde0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7faa140098a0 tx=0x7faa1402b970 comp rx=0 tx=0).stop 2024-01-15T17:56:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1be00640 1 -- 172.21.15.26:0/628159616 shutdown_connections 2024-01-15T17:56:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1be00640 1 --2- 172.21.15.26:0/628159616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c0b41e0 0x7faa1c0cfde0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1be00640 1 --2- 172.21.15.26:0/628159616 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c15d4b0 0x7faa1c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1be00640 1 -- 172.21.15.26:0/628159616 >> 172.21.15.26:0/628159616 conn(0x7faa1c0d3000 msgr2=0x7faa1c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1be00640 1 -- 172.21.15.26:0/628159616 shutdown_connections 2024-01-15T17:56:46.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.673+0000 7faa1be00640 1 -- 172.21.15.26:0/628159616 wait complete. 2024-01-15T17:56:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.675+0000 7faa1be00640 1 Processor -- start 2024-01-15T17:56:46.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.675+0000 7faa1be00640 1 -- start start 2024-01-15T17:56:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.676+0000 7faa1be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa1c0b41e0 0x7faa1c160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.676+0000 7faa1be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c0cf9a0 0x7faa1c1f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.676+0000 7faa1be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c15d4b0 0x7faa1c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.676+0000 7faa1be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c159db0 con 0x7faa1c15d4b0 2024-01-15T17:56:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.676+0000 7faa1be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c1f7690 con 0x7faa1c0b41e0 2024-01-15T17:56:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.676+0000 7faa1be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c15dba0 con 0x7faa1c0cf9a0 2024-01-15T17:56:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c15d4b0 0x7faa1c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c15d4b0 0x7faa1c15bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56136/0 (socket says 172.21.15.26:56136) 2024-01-15T17:56:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1b400640 1 -- 172.21.15.26:0/3449659468 learned_addr learned my addr 172.21.15.26:0/3449659468 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:56:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa11e00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c0cf9a0 0x7faa1c1f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1aa00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa1c0b41e0 0x7faa1c160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1b400640 1 -- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c0cf9a0 msgr2=0x7faa1c1f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1b400640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c0cf9a0 0x7faa1c1f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1b400640 1 -- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa1c0b41e0 msgr2=0x7faa1c160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1b400640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa1c0b41e0 0x7faa1c160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1b400640 1 -- 172.21.15.26:0/3449659468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa140094d0 con 0x7faa1c15d4b0 2024-01-15T17:56:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.677+0000 7faa1b400640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c15d4b0 0x7faa1c15bc90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faa140099c0 tx=0x7faa14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.678+0000 7faa13400640 1 -- 172.21.15.26:0/3449659468 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14039920 con 0x7faa1c15d4b0 2024-01-15T17:56:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.678+0000 7faa13400640 1 -- 172.21.15.26:0/3449659468 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14039aa0 con 0x7faa1c15d4b0 2024-01-15T17:56:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.678+0000 7faa13400640 1 -- 172.21.15.26:0/3449659468 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14039e50 con 0x7faa1c15d4b0 2024-01-15T17:56:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.678+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c15da10 con 0x7faa1c15d4b0 2024-01-15T17:56:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.678+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c160950 con 0x7faa1c15d4b0 2024-01-15T17:56:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.679+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c067450 con 0x7faa1c15d4b0 2024-01-15T17:56:46.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.680+0000 7faa13400640 1 -- 172.21.15.26:0/3449659468 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 5) v1 ==== 89539+0+0 (secure 0 0 0) 0x7faa1404d460 con 0x7faa1c15d4b0 2024-01-15T17:56:46.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.681+0000 7faa13400640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f406b2c0 0x7fa9f406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:56:46.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.681+0000 7faa13400640 1 -- 172.21.15.26:0/3449659468 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa1403e070 con 0x7faa1c15d4b0 2024-01-15T17:56:46.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.681+0000 7faa1aa00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f406b2c0 0x7fa9f406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:56:46.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.682+0000 7faa1aa00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f406b2c0 0x7fa9f406d780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009e90 tx=0x7faa0c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:56:46.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.685+0000 7faa13400640 1 -- 172.21.15.26:0/3449659468 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa14077960 con 0x7faa1c15d4b0 2024-01-15T17:56:46.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.905+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7faa1c15ccc0 con 0x7fa9f406b2c0 2024-01-15T17:56:46.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.920+0000 7faa13400640 1 -- 172.21.15.26:0/3449659468 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7faa1c15ccc0 con 0x7fa9f406b2c0 2024-01-15T17:56:46.924 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8 2024-01-15T17:56:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f406b2c0 msgr2=0x7fa9f406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f406b2c0 0x7fa9f406d780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009e90 tx=0x7faa0c00b040 comp rx=0 tx=0).stop 2024-01-15T17:56:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c15d4b0 msgr2=0x7faa1c15bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:56:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c15d4b0 0x7faa1c15bc90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7faa140099c0 tx=0x7faa14004730 comp rx=0 tx=0).stop 2024-01-15T17:56:46.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 shutdown_connections 2024-01-15T17:56:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa1c0cf9a0 0x7faa1c1f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f406b2c0 0x7fa9f406d780 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1c15d4b0 0x7faa1c15bc90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 --2- 172.21.15.26:0/3449659468 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa1c0b41e0 0x7faa1c160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:56:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 >> 172.21.15.26:0/3449659468 conn(0x7faa1c0d3000 msgr2=0x7faa1c0b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:56:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 shutdown_connections 2024-01-15T17:56:46.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:56:46.923+0000 7faa1be00640 1 -- 172.21.15.26:0/3449659468 wait complete. 2024-01-15T17:56:46.945 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8 2024-01-15T17:56:46.945 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:56:46.945 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:56:46.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:56:46.996 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T17:56:56.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:56:56.119 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:57:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T17:57:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T17:57:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.5568 s, 64.9 MB/s 2024-01-15T17:57:03.680 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:57:03.680 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:57:03.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:57:03.734 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T17:57:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T17:57:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T17:57:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.1649 s, 75.8 MB/s 2024-01-15T17:57:17.963 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:57:17.963 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:57:18.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:57:18.022 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T17:57:26.251 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:57:26.262 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:57:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T17:57:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T17:57:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.5437 s, 79.3 MB/s 2024-01-15T17:57:31.632 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-15T17:57:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.854+0000 7f11e1a00640 1 Processor -- start 2024-01-15T17:57:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.854+0000 7f11e1a00640 1 -- start start 2024-01-15T17:57:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.856+0000 7f11e1a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11dc13df80 0x7f11dc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.856+0000 7f11e1a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11dc06c470 con 0x7f11dc0cf9a0 2024-01-15T17:57:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.856+0000 7f11e1a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11dc06c750 con 0x7f11dc13e360 2024-01-15T17:57:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.856+0000 7f11e1a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11dc06ca30 con 0x7f11dc13df80 2024-01-15T17:57:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.856+0000 7f11daa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11dc13df80 0x7f11dc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.857+0000 7f11daa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f11dc13df80 0x7f11dc15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49204/0 (socket says 172.21.15.26:49204) 2024-01-15T17:57:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.857+0000 7f11daa00640 1 -- 172.21.15.26:0/1424311139 learned_addr learned my addr 172.21.15.26:0/1424311139 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.857+0000 7f11db400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f11dc0cf9a0 0x7f11dc0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51150/0 (socket says 172.21.15.26:51150) 2024-01-15T17:57:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.857+0000 7f11daa00640 1 -- 172.21.15.26:0/1424311139 >> v1:172.21.15.124:6789/0 conn(0x7f11dc13e360 legacy=0x7f11dc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.857+0000 7f11daa00640 1 -- 172.21.15.26:0/1424311139 >> v1:172.21.15.26:6789/0 conn(0x7f11dc0cf9a0 legacy=0x7f11dc0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.857+0000 7f11daa00640 1 -- 172.21.15.26:0/1424311139 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc06cd10 con 0x7f11dc13df80 2024-01-15T17:57:31.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.858+0000 7f11daa00640 1 --2- 172.21.15.26:0/1424311139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 0x7f11dc15bee0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009870 tx=0x7f11cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c14202f2a4a5d28 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:31.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.859+0000 7f11da000640 1 -- 172.21.15.26:0/1424311139 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc030860 con 0x7f11dc13df80 2024-01-15T17:57:31.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.859+0000 7f11da000640 1 -- 172.21.15.26:0/1424311139 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc0309c0 con 0x7f11dc13df80 2024-01-15T17:57:31.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.859+0000 7f11da000640 1 -- 172.21.15.26:0/1424311139 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc030d50 con 0x7f11dc13df80 2024-01-15T17:57:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.859+0000 7f11e1a00640 1 -- 172.21.15.26:0/1424311139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 msgr2=0x7f11dc15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.859+0000 7f11e1a00640 1 --2- 172.21.15.26:0/1424311139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 0x7f11dc15bee0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009870 tx=0x7f11cc02b9a0 comp rx=0 tx=0).stop 2024-01-15T17:57:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.860+0000 7f11e1a00640 1 -- 172.21.15.26:0/1424311139 shutdown_connections 2024-01-15T17:57:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.860+0000 7f11e1a00640 1 --2- 172.21.15.26:0/1424311139 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 0x7f11dc15bee0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.860+0000 7f11e1a00640 1 -- 172.21.15.26:0/1424311139 >> 172.21.15.26:0/1424311139 conn(0x7f11dc06a800 msgr2=0x7f11dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.860+0000 7f11e1a00640 1 -- 172.21.15.26:0/1424311139 shutdown_connections 2024-01-15T17:57:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.860+0000 7f11e1a00640 1 -- 172.21.15.26:0/1424311139 wait complete. 2024-01-15T17:57:31.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.861+0000 7f11e1a00640 1 Processor -- start 2024-01-15T17:57:31.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.862+0000 7f11e1a00640 1 -- start start 2024-01-15T17:57:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11e1a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11dc0cf9a0 0x7f11dc13a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11e1a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 0x7f11dc0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11e1a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11dc13e360 0x7f11dc0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:31.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11e1a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc13b770 con 0x7f11dc13df80 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11e1a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc13c5d0 con 0x7f11dc13e360 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11e1a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc13b9c0 con 0x7f11dc0cf9a0 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 0x7f11dc0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 0x7f11dc0afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49214/0 (socket says 172.21.15.26:49214) 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11d3e00640 1 -- 172.21.15.26:0/3770959647 learned_addr learned my addr 172.21.15.26:0/3770959647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11daa00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11dc0cf9a0 0x7f11dc13a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11db400640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11dc13e360 0x7f11dc0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11d3e00640 1 -- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11dc0cf9a0 msgr2=0x7f11dc13a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.863+0000 7f11d3e00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11dc0cf9a0 0x7f11dc13a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11d3e00640 1 -- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11dc13e360 msgr2=0x7f11dc0b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11d3e00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11dc13e360 0x7f11dc0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11d3e00640 1 -- 172.21.15.26:0/3770959647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11cc0094d0 con 0x7f11dc13df80 2024-01-15T17:57:31.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11daa00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11dc0cf9a0 0x7f11dc13a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:31.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11d3e00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 0x7f11dc0afdc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c430 tx=0x7f11c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:31.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11d2a00640 1 -- 172.21.15.26:0/3770959647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4011790 con 0x7f11dc13df80 2024-01-15T17:57:31.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11d2a00640 1 -- 172.21.15.26:0/3770959647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c40118f0 con 0x7f11dc13df80 2024-01-15T17:57:31.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11d2a00640 1 -- 172.21.15.26:0/3770959647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4011ca0 con 0x7f11dc13df80 2024-01-15T17:57:31.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11dc13bca0 con 0x7f11dc13df80 2024-01-15T17:57:31.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.864+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11dc13c260 con 0x7f11dc13df80 2024-01-15T17:57:31.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.866+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11dc0675d0 con 0x7f11dc13df80 2024-01-15T17:57:31.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.866+0000 7f11d2a00640 1 -- 172.21.15.26:0/3770959647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f11c401c460 con 0x7f11dc13df80 2024-01-15T17:57:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.867+0000 7f11d2a00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11b406b3a0 0x7f11b406d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:31.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.868+0000 7f11d2a00640 1 -- 172.21.15.26:0/3770959647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11c4089b50 con 0x7f11dc13df80 2024-01-15T17:57:31.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.868+0000 7f11daa00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11b406b3a0 0x7f11b406d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.869+0000 7f11daa00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11b406b3a0 0x7f11b406d860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f11cc036000 tx=0x7f11cc03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:31.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:31.874+0000 7f11d2a00640 1 -- 172.21.15.26:0/3770959647 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11c40549e0 con 0x7f11dc13df80 2024-01-15T17:57:32.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.071+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f11dc13c710 con 0x7f11dc13df80 2024-01-15T17:57:32.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.078+0000 7f11d2a00640 1 -- 172.21.15.26:0/3770959647 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v11)=0 v11) v1 ==== 135+0+0 (secure 0 0 0) 0x7f11c4058000 con 0x7f11dc13df80 2024-01-15T17:57:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.080+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11b406b3a0 msgr2=0x7f11b406d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.080+0000 7f11e1a00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11b406b3a0 0x7f11b406d860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f11cc036000 tx=0x7f11cc03b040 comp rx=0 tx=0).stop 2024-01-15T17:57:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.080+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 msgr2=0x7f11dc0afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.080+0000 7f11e1a00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 0x7f11dc0afdc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f11c400c430 tx=0x7f11c400c8f0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.081+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 shutdown_connections 2024-01-15T17:57:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.081+0000 7f11e1a00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11b406b3a0 0x7f11b406d860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.081+0000 7f11e1a00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f11dc13e360 0x7f11dc0b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.081+0000 7f11e1a00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f11dc13df80 0x7f11dc0afdc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.081+0000 7f11e1a00640 1 --2- 172.21.15.26:0/3770959647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f11dc0cf9a0 0x7f11dc13a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.081+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 >> 172.21.15.26:0/3770959647 conn(0x7f11dc06a800 msgr2=0x7f11dc153b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.081+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 shutdown_connections 2024-01-15T17:57:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.081+0000 7f11e1a00640 1 -- 172.21.15.26:0/3770959647 wait complete. 2024-01-15T17:57:32.104 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T17:57:32.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.331+0000 7f0014a00640 1 Processor -- start 2024-01-15T17:57:32.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.331+0000 7f0014a00640 1 -- start start 2024-01-15T17:57:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.333+0000 7f0014a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f000c0ac8d0 0x7f000c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.333+0000 7f0014a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c071de0 con 0x7f000c06a3e0 2024-01-15T17:57:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.333+0000 7f0014a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c072030 con 0x7f000c0d0b60 2024-01-15T17:57:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.333+0000 7f0014a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0acd60 con 0x7f000c0ac8d0 2024-01-15T17:57:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.333+0000 7f0012000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f000c06a3e0 0x7f000c06a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36430/0 (socket says 172.21.15.26:36430) 2024-01-15T17:57:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.333+0000 7f0012a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f000c0ac8d0 0x7f000c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.333+0000 7f0012000640 1 -- 172.21.15.26:0/3282926357 learned_addr learned my addr 172.21.15.26:0/3282926357 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:57:32.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.333+0000 7f0010c00640 1 -- 172.21.15.26:0/3282926357 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346109072 0 0) 0x7f000c071de0 con 0x7f000c06a3e0 2024-01-15T17:57:32.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.334+0000 7f0010c00640 1 -- 172.21.15.26:0/3282926357 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effec003520 con 0x7f000c06a3e0 2024-01-15T17:57:32.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.334+0000 7f0012a00640 1 -- 172.21.15.26:0/3282926357 >> v1:172.21.15.124:6789/0 conn(0x7f000c0d0b60 legacy=0x7f000c06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.334+0000 7f0012a00640 1 -- 172.21.15.26:0/3282926357 >> v1:172.21.15.26:6789/0 conn(0x7f000c06a3e0 legacy=0x7f000c06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.334+0000 7f0012a00640 1 -- 172.21.15.26:0/3282926357 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c072310 con 0x7f000c0ac8d0 2024-01-15T17:57:32.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.335+0000 7f0012a00640 1 --2- 172.21.15.26:0/3282926357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0ac8d0 0x7f000c0aede0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0004009d90 tx=0x7f000402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524f2396ca17e9ad server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:32.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.335+0000 7f0010c00640 1 -- 172.21.15.26:0/3282926357 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000402cd00 con 0x7f000c0ac8d0 2024-01-15T17:57:32.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.336+0000 7f0010c00640 1 -- 172.21.15.26:0/3282926357 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000402ce60 con 0x7f000c0ac8d0 2024-01-15T17:57:32.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.336+0000 7f0010c00640 1 -- 172.21.15.26:0/3282926357 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0004034660 con 0x7f000c0ac8d0 2024-01-15T17:57:32.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.336+0000 7f0014a00640 1 -- 172.21.15.26:0/3282926357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0ac8d0 msgr2=0x7f000c0aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.336+0000 7f0014a00640 1 --2- 172.21.15.26:0/3282926357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0ac8d0 0x7f000c0aede0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0004009d90 tx=0x7f000402bf50 comp rx=0 tx=0).stop 2024-01-15T17:57:32.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.336+0000 7f0014a00640 1 -- 172.21.15.26:0/3282926357 shutdown_connections 2024-01-15T17:57:32.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.336+0000 7f0014a00640 1 --2- 172.21.15.26:0/3282926357 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0ac8d0 0x7f000c0aede0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.336+0000 7f0014a00640 1 -- 172.21.15.26:0/3282926357 >> 172.21.15.26:0/3282926357 conn(0x7f000c0cfef0 msgr2=0x7f000c0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:32.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.336+0000 7f0014a00640 1 -- 172.21.15.26:0/3282926357 shutdown_connections 2024-01-15T17:57:32.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.336+0000 7f0014a00640 1 -- 172.21.15.26:0/3282926357 wait complete. 2024-01-15T17:57:32.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.339+0000 7f0014a00640 1 Processor -- start 2024-01-15T17:57:32.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.339+0000 7f0014a00640 1 -- start start 2024-01-15T17:57:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0014a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f000c06a3e0 0x7f000c13e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0014a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000c0ac8d0 0x7f000c154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0014a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0d0b60 0x7f000c0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0014a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c156450 con 0x7f000c0d0b60 2024-01-15T17:57:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0014a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c156730 con 0x7f000c0ac8d0 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0014a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c156a10 con 0x7f000c06a3e0 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0012a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0d0b60 0x7f000c0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0012a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0d0b60 0x7f000c0b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53506/0 (socket says 172.21.15.26:53506) 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0012a00640 1 -- 172.21.15.26:0/3653149699 learned_addr learned my addr 172.21.15.26:0/3653149699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0011600640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000c0ac8d0 0x7f000c154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.341+0000 7f0012000640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f000c06a3e0 0x7f000c13e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f0012a00640 1 -- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f000c06a3e0 msgr2=0x7f000c13e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f0012a00640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f000c06a3e0 0x7f000c13e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f0012a00640 1 -- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000c0ac8d0 msgr2=0x7f000c154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f0012a00640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000c0ac8d0 0x7f000c154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f0012a00640 1 -- 172.21.15.26:0/3653149699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0004009a70 con 0x7f000c0d0b60 2024-01-15T17:57:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f0011600640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000c0ac8d0 0x7f000c154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:57:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f0012a00640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0d0b60 0x7f000c0b0f00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0004009d90 tx=0x7f0004004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f000a000640 1 -- 172.21.15.26:0/3653149699 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000403f680 con 0x7f000c0d0b60 2024-01-15T17:57:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f000a000640 1 -- 172.21.15.26:0/3653149699 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000403f7e0 con 0x7f000c0d0b60 2024-01-15T17:57:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f000a000640 1 -- 172.21.15.26:0/3653149699 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000403fb70 con 0x7f000c0d0b60 2024-01-15T17:57:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.342+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c156cf0 con 0x7f000c0d0b60 2024-01-15T17:57:32.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.343+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c13eaf0 con 0x7f000c0d0b60 2024-01-15T17:57:32.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.344+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c157210 con 0x7f000c0d0b60 2024-01-15T17:57:32.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.345+0000 7f000a000640 1 -- 172.21.15.26:0/3653149699 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f000403b040 con 0x7f000c0d0b60 2024-01-15T17:57:32.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.346+0000 7f000a000640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effe406b350 0x7effe406d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.346+0000 7f000a000640 1 -- 172.21.15.26:0/3653149699 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00040ac990 con 0x7f000c0d0b60 2024-01-15T17:57:32.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.346+0000 7f0012000640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effe406b350 0x7effe406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.347+0000 7f0012000640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effe406b350 0x7effe406d810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efffc0011e0 tx=0x7efffc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:32.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.352+0000 7f000a000640 1 -- 172.21.15.26:0/3653149699 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00040777a0 con 0x7f000c0d0b60 2024-01-15T17:57:32.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.583+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7f000c13ec80 con 0x7effe406b350 2024-01-15T17:57:32.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.620+0000 7f000a000640 1 -- 172.21.15.26:0/3653149699 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efffc001ce0 con 0x7effe406b350 2024-01-15T17:57:32.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effe406b350 msgr2=0x7effe406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effe406b350 0x7effe406d810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efffc0011e0 tx=0x7efffc009040 comp rx=0 tx=0).stop 2024-01-15T17:57:32.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0d0b60 msgr2=0x7f000c0b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0d0b60 0x7f000c0b0f00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f0004009d90 tx=0x7f0004004a40 comp rx=0 tx=0).stop 2024-01-15T17:57:32.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 shutdown_connections 2024-01-15T17:57:32.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effe406b350 0x7effe406d810 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f000c0d0b60 0x7f000c0b0f00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000c0ac8d0 0x7f000c154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 --2- 172.21.15.26:0/3653149699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f000c06a3e0 0x7f000c13e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 >> 172.21.15.26:0/3653149699 conn(0x7f000c0cfef0 msgr2=0x7f000c06d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:32.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 shutdown_connections 2024-01-15T17:57:32.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.623+0000 7f0014a00640 1 -- 172.21.15.26:0/3653149699 wait complete. 2024-01-15T17:57:32.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T17:57:32.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.872+0000 7f2c2b200640 1 Processor -- start 2024-01-15T17:57:32.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.873+0000 7f2c2b200640 1 -- start start 2024-01-15T17:57:32.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.874+0000 7f2c2b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.874+0000 7f2c2b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c240cfc30 con 0x7f2c2415e3b0 2024-01-15T17:57:32.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.874+0000 7f2c2b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c240b1ee0 con 0x7f2c240cf7d0 2024-01-15T17:57:32.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.874+0000 7f2c2b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240b2780 con 0x7f2c24143ae0 2024-01-15T17:57:32.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.875+0000 7f2c29200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2c2415e3b0 0x7f2c240702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36434/0 (socket says 172.21.15.26:36434) 2024-01-15T17:57:32.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.875+0000 7f2c29200640 1 -- 172.21.15.26:0/2361034057 learned_addr learned my addr 172.21.15.26:0/2361034057 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:57:32.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.875+0000 7f2c23e00640 1 --2- 172.21.15.26:0/2361034057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.876+0000 7f2c22a00640 1 -- 172.21.15.26:0/2361034057 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398464397 0 0) 0x7f2c240cfc30 con 0x7f2c2415e3b0 2024-01-15T17:57:32.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.876+0000 7f2c22a00640 1 -- 172.21.15.26:0/2361034057 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c04003520 con 0x7f2c2415e3b0 2024-01-15T17:57:32.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.876+0000 7f2c22a00640 1 -- 172.21.15.26:0/2361034057 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26979252 0 0) 0x7f2c240b1ee0 con 0x7f2c240cf7d0 2024-01-15T17:57:32.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.876+0000 7f2c22a00640 1 -- 172.21.15.26:0/2361034057 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c240cfc30 con 0x7f2c240cf7d0 2024-01-15T17:57:32.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.876+0000 7f2c22a00640 1 -- 172.21.15.26:0/2361034057 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2211628332 0 0) 0x7f2c04003520 con 0x7f2c2415e3b0 2024-01-15T17:57:32.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.876+0000 7f2c23e00640 1 -- 172.21.15.26:0/2361034057 >> v1:172.21.15.124:6789/0 conn(0x7f2c240cf7d0 legacy=0x7f2c2415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.876+0000 7f2c23e00640 1 -- 172.21.15.26:0/2361034057 >> v1:172.21.15.26:6789/0 conn(0x7f2c2415e3b0 legacy=0x7f2c240702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.876+0000 7f2c23e00640 1 -- 172.21.15.26:0/2361034057 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2415e990 con 0x7f2c24143ae0 2024-01-15T17:57:32.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.877+0000 7f2c23e00640 1 --2- 172.21.15.26:0/2361034057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240cf290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c140097d0 tx=0x7f2c1402b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9e83658110b6fd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:32.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.878+0000 7f2c22a00640 1 -- 172.21.15.26:0/2361034057 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c001970 con 0x7f2c24143ae0 2024-01-15T17:57:32.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.878+0000 7f2c22a00640 1 -- 172.21.15.26:0/2361034057 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1402e050 con 0x7f2c24143ae0 2024-01-15T17:57:32.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.878+0000 7f2c22a00640 1 -- 172.21.15.26:0/2361034057 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c14030710 con 0x7f2c24143ae0 2024-01-15T17:57:32.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.879+0000 7f2c2b200640 1 -- 172.21.15.26:0/2361034057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 msgr2=0x7f2c240cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.879+0000 7f2c2b200640 1 --2- 172.21.15.26:0/2361034057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240cf290 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c140097d0 tx=0x7f2c1402b800 comp rx=0 tx=0).stop 2024-01-15T17:57:32.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.879+0000 7f2c2b200640 1 -- 172.21.15.26:0/2361034057 shutdown_connections 2024-01-15T17:57:32.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.879+0000 7f2c2b200640 1 --2- 172.21.15.26:0/2361034057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240cf290 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.879+0000 7f2c2b200640 1 -- 172.21.15.26:0/2361034057 >> 172.21.15.26:0/2361034057 conn(0x7f2c2406a800 msgr2=0x7f2c2406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:32.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.879+0000 7f2c2b200640 1 -- 172.21.15.26:0/2361034057 shutdown_connections 2024-01-15T17:57:32.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.879+0000 7f2c2b200640 1 -- 172.21.15.26:0/2361034057 wait complete. 2024-01-15T17:57:32.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.881+0000 7f2c2b200640 1 Processor -- start 2024-01-15T17:57:32.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.882+0000 7f2c2b200640 1 -- start start 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.883+0000 7f2c2b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240cf7d0 0x7f2c240b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.883+0000 7f2c2b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.883+0000 7f2c2b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2415e3b0 0x7f2c24159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.883+0000 7f2c2b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240b0b60 con 0x7f2c240cf7d0 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.883+0000 7f2c2b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240b0db0 con 0x7f2c2415e3b0 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.883+0000 7f2c2b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c240b1090 con 0x7f2c24143ae0 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.883+0000 7f2c23e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240cf7d0 0x7f2c240b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.883+0000 7f2c23e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240cf7d0 0x7f2c240b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53520/0 (socket says 172.21.15.26:53520) 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.883+0000 7f2c23e00640 1 -- 172.21.15.26:0/88632894 learned_addr learned my addr 172.21.15.26:0/88632894 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:32.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c23400640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c29200640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2415e3b0 0x7f2c24159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c23e00640 1 -- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 msgr2=0x7f2c240aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c23e00640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c23e00640 1 -- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2415e3b0 msgr2=0x7f2c24159b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c23e00640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2415e3b0 0x7f2c24159b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c23e00640 1 -- 172.21.15.26:0/88632894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14009400 con 0x7f2c240cf7d0 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c29200640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2415e3b0 0x7f2c24159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c23400640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c23e00640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240cf7d0 0x7f2c240b0620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.884+0000 7f2c1be00640 1 -- 172.21.15.26:0/88632894 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1403c620 con 0x7f2c240cf7d0 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.885+0000 7f2c1be00640 1 -- 172.21.15.26:0/88632894 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1403c7a0 con 0x7f2c240cf7d0 2024-01-15T17:57:32.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.885+0000 7f2c1be00640 1 -- 172.21.15.26:0/88632894 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1403cb50 con 0x7f2c240cf7d0 2024-01-15T17:57:32.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.885+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c240b1370 con 0x7f2c240cf7d0 2024-01-15T17:57:32.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.885+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c240af150 con 0x7f2c240cf7d0 2024-01-15T17:57:32.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.886+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c240675d0 con 0x7f2c240cf7d0 2024-01-15T17:57:32.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.887+0000 7f2c1be00640 1 -- 172.21.15.26:0/88632894 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f2c14039070 con 0x7f2c240cf7d0 2024-01-15T17:57:32.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.888+0000 7f2c1be00640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2bfc06b350 0x7f2bfc06d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:32.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.888+0000 7f2c1be00640 1 -- 172.21.15.26:0/88632894 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c140ab470 con 0x7f2c240cf7d0 2024-01-15T17:57:32.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.889+0000 7f2c23400640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2bfc06b350 0x7f2bfc06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:32.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.890+0000 7f2c23400640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2bfc06b350 0x7f2bfc06d810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c007fd0 tx=0x7f2c0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:32.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:32.893+0000 7f2c1be00640 1 -- 172.21.15.26:0/88632894 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c14076170 con 0x7f2c240cf7d0 2024-01-15T17:57:33.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.117+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c240b2410 con 0x7f2bfc06b350 2024-01-15T17:57:33.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.129+0000 7f2c1be00640 1 -- 172.21.15.26:0/88632894 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f2c0c0013f0 con 0x7f2bfc06b350 2024-01-15T17:57:33.132 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8 2024-01-15T17:57:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.131+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2bfc06b350 msgr2=0x7f2bfc06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.131+0000 7f2c2b200640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2bfc06b350 0x7f2bfc06d810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c007fd0 tx=0x7f2c0c009040 comp rx=0 tx=0).stop 2024-01-15T17:57:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.131+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240cf7d0 msgr2=0x7f2c240b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:33.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.131+0000 7f2c2b200640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240cf7d0 0x7f2c240b0620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2c14000c00 tx=0x7f2c140046c0 comp rx=0 tx=0).stop 2024-01-15T17:57:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.132+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 shutdown_connections 2024-01-15T17:57:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.132+0000 7f2c2b200640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c24143ae0 0x7f2c240aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.132+0000 7f2c2b200640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2bfc06b350 0x7f2bfc06d810 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.132+0000 7f2c2b200640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c2415e3b0 0x7f2c24159b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.132+0000 7f2c2b200640 1 --2- 172.21.15.26:0/88632894 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c240cf7d0 0x7f2c240b0620 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.132+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 >> 172.21.15.26:0/88632894 conn(0x7f2c2406a800 msgr2=0x7f2c2415c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.132+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 shutdown_connections 2024-01-15T17:57:33.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:33.132+0000 7f2c2b200640 1 -- 172.21.15.26:0/88632894 wait complete. 2024-01-15T17:57:33.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:57:33.155 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8) 2024-01-15T17:57:33.217 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8 2024-01-15T17:57:33.217 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T17:57:33.217 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:57:33.219 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T17:57:35.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:57:35.220 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8) 2024-01-15T17:57:35.289 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/f659b49d-1d79-4a5c-87ed-9218971f4da8 2024-01-15T17:57:35.289 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T17:57:35.289 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:57:35.291 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T17:57:35.291 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-01-15T17:57:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.515+0000 7f8b46200640 1 Processor -- start 2024-01-15T17:57:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.516+0000 7f8b46200640 1 -- start start 2024-01-15T17:57:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.517+0000 7f8b46200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8b40143c40 0x7f8b40071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.517+0000 7f8b46200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b400ccc10 0x7f8b4006d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.517+0000 7f8b46200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b40070e50 con 0x7f8b400717d0 2024-01-15T17:57:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.517+0000 7f8b46200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b400723c0 con 0x7f8b400ccc10 2024-01-15T17:57:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.517+0000 7f8b46200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b40144020 con 0x7f8b40143c40 2024-01-15T17:57:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.517+0000 7f8b45800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b400ccc10 0x7f8b4006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.517+0000 7f8b37e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8b400717d0 0x7f8b400cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36448/0 (socket says 172.21.15.26:36448) 2024-01-15T17:57:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.517+0000 7f8b37e00640 1 -- 172.21.15.26:0/76192094 learned_addr learned my addr 172.21.15.26:0/76192094 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:57:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.518+0000 7f8b44e00640 1 --2- 172.21.15.26:0/76192094 >> v2:172.21.15.124:3300/0 conn(0x7f8b40143c40 0x7f8b40071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.518+0000 7f8b45800640 1 -- 172.21.15.26:0/76192094 >> v2:172.21.15.124:3300/0 conn(0x7f8b40143c40 msgr2=0x7f8b40071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.518+0000 7f8b45800640 1 --2- 172.21.15.26:0/76192094 >> v2:172.21.15.124:3300/0 conn(0x7f8b40143c40 0x7f8b40071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.518+0000 7f8b45800640 1 -- 172.21.15.26:0/76192094 >> v1:172.21.15.26:6789/0 conn(0x7f8b400717d0 legacy=0x7f8b400cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.518+0000 7f8b45800640 1 -- 172.21.15.26:0/76192094 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40071e00 con 0x7f8b400ccc10 2024-01-15T17:57:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.518+0000 7f8b3fe00640 1 -- 172.21.15.26:0/76192094 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877767900 0 0) 0x7f8b40070e50 con 0x7f8b400717d0 2024-01-15T17:57:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.519+0000 7f8b45800640 1 --2- 172.21.15.26:0/76192094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400ccc10 0x7f8b4006d160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8b4006e410 tx=0x7f8b3802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a35b47faef962d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.519+0000 7f8b3fe00640 1 -- 172.21.15.26:0/76192094 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3802cbf0 con 0x7f8b400ccc10 2024-01-15T17:57:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.519+0000 7f8b3fe00640 1 -- 172.21.15.26:0/76192094 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3802cd50 con 0x7f8b400ccc10 2024-01-15T17:57:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.519+0000 7f8b3fe00640 1 -- 172.21.15.26:0/76192094 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38034660 con 0x7f8b400ccc10 2024-01-15T17:57:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.520+0000 7f8b46200640 1 -- 172.21.15.26:0/76192094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400ccc10 msgr2=0x7f8b4006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.520+0000 7f8b46200640 1 --2- 172.21.15.26:0/76192094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400ccc10 0x7f8b4006d160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8b4006e410 tx=0x7f8b3802be40 comp rx=0 tx=0).stop 2024-01-15T17:57:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.520+0000 7f8b46200640 1 -- 172.21.15.26:0/76192094 shutdown_connections 2024-01-15T17:57:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.520+0000 7f8b46200640 1 --2- 172.21.15.26:0/76192094 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400ccc10 0x7f8b4006d160 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.520+0000 7f8b46200640 1 --2- 172.21.15.26:0/76192094 >> v2:172.21.15.124:3300/0 conn(0x7f8b40143c40 0x7f8b40071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.520+0000 7f8b46200640 1 -- 172.21.15.26:0/76192094 >> 172.21.15.26:0/76192094 conn(0x7f8b4006a800 msgr2=0x7f8b4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.520+0000 7f8b46200640 1 -- 172.21.15.26:0/76192094 shutdown_connections 2024-01-15T17:57:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.520+0000 7f8b46200640 1 -- 172.21.15.26:0/76192094 wait complete. 2024-01-15T17:57:35.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.522+0000 7f8b46200640 1 Processor -- start 2024-01-15T17:57:35.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.522+0000 7f8b46200640 1 -- start start 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.523+0000 7f8b46200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400717d0 0x7f8b40143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.523+0000 7f8b46200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b400ccc10 0x7f8b4015a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.523+0000 7f8b46200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b40143c40 0x7f8b401f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.523+0000 7f8b46200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400c5040 con 0x7f8b400717d0 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.523+0000 7f8b46200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4013b0f0 con 0x7f8b40143c40 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.523+0000 7f8b46200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4015ad50 con 0x7f8b400ccc10 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.523+0000 7f8b44e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400717d0 0x7f8b40143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.523+0000 7f8b44e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400717d0 0x7f8b40143080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53534/0 (socket says 172.21.15.26:53534) 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.523+0000 7f8b44e00640 1 -- 172.21.15.26:0/2104488741 learned_addr learned my addr 172.21.15.26:0/2104488741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b44e00640 1 -- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b400ccc10 msgr2=0x7f8b4015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b45800640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b40143c40 0x7f8b401f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b37e00640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b400ccc10 0x7f8b4015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b44e00640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b400ccc10 0x7f8b4015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b44e00640 1 -- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b40143c40 msgr2=0x7f8b401f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b44e00640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b40143c40 0x7f8b401f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b44e00640 1 -- 172.21.15.26:0/2104488741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b38009a70 con 0x7f8b400717d0 2024-01-15T17:57:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b45800640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b40143c40 0x7f8b401f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b37e00640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b400ccc10 0x7f8b4015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.524+0000 7f8b44e00640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400717d0 0x7f8b40143080 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c820 tx=0x7f8b3000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.525+0000 7f8b3d600640 1 -- 172.21.15.26:0/2104488741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30004220 con 0x7f8b400717d0 2024-01-15T17:57:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.525+0000 7f8b3d600640 1 -- 172.21.15.26:0/2104488741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b300043a0 con 0x7f8b400717d0 2024-01-15T17:57:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.525+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b401f3630 con 0x7f8b400717d0 2024-01-15T17:57:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.525+0000 7f8b3d600640 1 -- 172.21.15.26:0/2104488741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30007a70 con 0x7f8b400717d0 2024-01-15T17:57:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.525+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b401f3bc0 con 0x7f8b400717d0 2024-01-15T17:57:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.526+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b40067450 con 0x7f8b400717d0 2024-01-15T17:57:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.527+0000 7f8b3d600640 1 -- 172.21.15.26:0/2104488741 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f8b30007bd0 con 0x7f8b400717d0 2024-01-15T17:57:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.528+0000 7f8b3d600640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b1806b350 0x7f8b1806d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:35.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.528+0000 7f8b3d600640 1 -- 172.21.15.26:0/2104488741 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b30089af0 con 0x7f8b400717d0 2024-01-15T17:57:35.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.528+0000 7f8b37e00640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b1806b350 0x7f8b1806d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:35.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.530+0000 7f8b37e00640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b1806b350 0x7f8b1806d810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b400c58f0 tx=0x7f8b28009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:35.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.533+0000 7f8b3d600640 1 -- 172.21.15.26:0/2104488741 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b300547f0 con 0x7f8b400717d0 2024-01-15T17:57:35.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.765+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b40072710 con 0x7f8b1806b350 2024-01-15T17:57:35.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.922+0000 7f8b3d600640 1 -- 172.21.15.26:0/2104488741 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8b40072710 con 0x7f8b1806b350 2024-01-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b1806b350 msgr2=0x7f8b1806d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b1806b350 0x7f8b1806d810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b400c58f0 tx=0x7f8b28009040 comp rx=0 tx=0).stop 2024-01-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400717d0 msgr2=0x7f8b40143080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400717d0 0x7f8b40143080 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c820 tx=0x7f8b3000cce0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 shutdown_connections 2024-01-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b400ccc10 0x7f8b4015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b1806b350 0x7f8b1806d810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b40143c40 0x7f8b401f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 --2- 172.21.15.26:0/2104488741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b400717d0 0x7f8b40143080 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.925+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 >> 172.21.15.26:0/2104488741 conn(0x7f8b4006a800 msgr2=0x7f8b40144190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.926+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 shutdown_connections 2024-01-15T17:57:35.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:35.926+0000 7f8b46200640 1 -- 172.21.15.26:0/2104488741 wait complete. 2024-01-15T17:57:35.935 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.931+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:35.935 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.931+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:35.935 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.931+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:35.935 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.931+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:35.935 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.931+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:35.949 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T17:57:35.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T17:57:35.960 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.957+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:35.960 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.957+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:35.960 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.957+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:35.960 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.957+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:35.960 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:35.957+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.165+0000 7f47b5200640 1 Processor -- start 2024-01-15T17:57:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.166+0000 7f47b5200640 1 -- start start 2024-01-15T17:57:36.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.167+0000 7f47b5200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f47b013df80 0x7f47b015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.167+0000 7f47b5200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47b013e360 0x7f47b00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.167+0000 7f47b5200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b006c470 con 0x7f47b015f9c0 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.167+0000 7f47b5200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b006c750 con 0x7f47b013e360 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.167+0000 7f47b5200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47b006ca30 con 0x7f47b013df80 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.168+0000 7f47ad400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47b013e360 0x7f47b00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.168+0000 7f47ad400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47b013e360 0x7f47b00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53536/0 (socket says 172.21.15.26:53536) 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.168+0000 7f47ad400640 1 -- 172.21.15.26:0/351399774 learned_addr learned my addr 172.21.15.26:0/351399774 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.168+0000 7f47ade00640 1 --2- 172.21.15.26:0/351399774 >> v2:172.21.15.124:3300/0 conn(0x7f47b013df80 0x7f47b015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.168+0000 7f47ad400640 1 -- 172.21.15.26:0/351399774 >> v2:172.21.15.124:3300/0 conn(0x7f47b013df80 msgr2=0x7f47b015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:36.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.168+0000 7f47ad400640 1 --2- 172.21.15.26:0/351399774 >> v2:172.21.15.124:3300/0 conn(0x7f47b013df80 0x7f47b015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.168+0000 7f47ad400640 1 -- 172.21.15.26:0/351399774 >> v1:172.21.15.124:6789/0 conn(0x7f47b015f9c0 legacy=0x7f47b01408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.168+0000 7f47ad400640 1 -- 172.21.15.26:0/351399774 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b006cd10 con 0x7f47b013e360 2024-01-15T17:57:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.170+0000 7f47ad400640 1 --2- 172.21.15.26:0/351399774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013e360 0x7f47b00cf290 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f47b0072770 tx=0x7f479802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4251f5e99c88dc9f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.170+0000 7f47aca00640 1 -- 172.21.15.26:0/351399774 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479802cb20 con 0x7f47b013e360 2024-01-15T17:57:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.170+0000 7f47aca00640 1 -- 172.21.15.26:0/351399774 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479802cc80 con 0x7f47b013e360 2024-01-15T17:57:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.170+0000 7f47aca00640 1 -- 172.21.15.26:0/351399774 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47980343f0 con 0x7f47b013e360 2024-01-15T17:57:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.170+0000 7f47b5200640 1 -- 172.21.15.26:0/351399774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013e360 msgr2=0x7f47b00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.170+0000 7f47b5200640 1 --2- 172.21.15.26:0/351399774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013e360 0x7f47b00cf290 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f47b0072770 tx=0x7f479802bf80 comp rx=0 tx=0).stop 2024-01-15T17:57:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.171+0000 7f47b5200640 1 -- 172.21.15.26:0/351399774 shutdown_connections 2024-01-15T17:57:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.171+0000 7f47b5200640 1 --2- 172.21.15.26:0/351399774 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013e360 0x7f47b00cf290 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.171+0000 7f47b5200640 1 --2- 172.21.15.26:0/351399774 >> v2:172.21.15.124:3300/0 conn(0x7f47b013df80 0x7f47b015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.171+0000 7f47b5200640 1 -- 172.21.15.26:0/351399774 >> 172.21.15.26:0/351399774 conn(0x7f47b006a800 msgr2=0x7f47b006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.171+0000 7f47b5200640 1 -- 172.21.15.26:0/351399774 shutdown_connections 2024-01-15T17:57:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.171+0000 7f47b5200640 1 -- 172.21.15.26:0/351399774 wait complete. 2024-01-15T17:57:36.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.173+0000 7f47b5200640 1 Processor -- start 2024-01-15T17:57:36.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.173+0000 7f47b5200640 1 -- start start 2024-01-15T17:57:36.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47b5200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013df80 0x7f47b01f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47b5200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47b013e360 0x7f47b0153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47b5200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47b015f9c0 0x7f47b015d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47b5200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b00a9aa0 con 0x7f47b013df80 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47b5200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0153b90 con 0x7f47b015f9c0 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47b5200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b01f6d00 con 0x7f47b013e360 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47ade00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013df80 0x7f47b01f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47ade00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013df80 0x7f47b01f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53542/0 (socket says 172.21.15.26:53542) 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47ade00640 1 -- 172.21.15.26:0/56554998 learned_addr learned my addr 172.21.15.26:0/56554998 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47ade00640 1 -- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47b013e360 msgr2=0x7f47b0153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47ade00640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47b013e360 0x7f47b0153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47ae800640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47b015f9c0 0x7f47b015d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47ade00640 1 -- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47b015f9c0 msgr2=0x7f47b015d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.174+0000 7f47ade00640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47b015f9c0 0x7f47b015d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.175+0000 7f47ade00640 1 -- 172.21.15.26:0/56554998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798009b30 con 0x7f47b013df80 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.175+0000 7f47ae800640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47b015f9c0 0x7f47b015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.175+0000 7f47ade00640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013df80 0x7f47b01f6720 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f47a000d600 tx=0x7f47a000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.175+0000 7f47a6000640 1 -- 172.21.15.26:0/56554998 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a00125f0 con 0x7f47b013df80 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.175+0000 7f47a6000640 1 -- 172.21.15.26:0/56554998 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a0012770 con 0x7f47b013df80 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.175+0000 7f47a6000640 1 -- 172.21.15.26:0/56554998 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0012b20 con 0x7f47b013df80 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.175+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b01f6fe0 con 0x7f47b013df80 2024-01-15T17:57:36.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.175+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b01f75a0 con 0x7f47b013df80 2024-01-15T17:57:36.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.176+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b00675d0 con 0x7f47b013df80 2024-01-15T17:57:36.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.177+0000 7f47a6000640 1 -- 172.21.15.26:0/56554998 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 6) v1 ==== 89590+0+0 (secure 0 0 0) 0x7f47a0012c80 con 0x7f47b013df80 2024-01-15T17:57:36.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.178+0000 7f47a6000640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f478806b3d0 0x7f478806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:36.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.178+0000 7f47a6000640 1 -- 172.21.15.26:0/56554998 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47a008a0f0 con 0x7f47b013df80 2024-01-15T17:57:36.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.179+0000 7f47ad400640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f478806b3d0 0x7f478806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:36.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.180+0000 7f47ad400640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f478806b3d0 0x7f478806d890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4798035000 tx=0x7f479802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:36.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.183+0000 7f47a6000640 1 -- 172.21.15.26:0/56554998 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47a0054f80 con 0x7f47b013df80 2024-01-15T17:57:36.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.413+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f47b00ce850 con 0x7f478806b3d0 2024-01-15T17:57:36.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:36.889+0000 7f47a6000640 1 -- 172.21.15.26:0/56554998 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f47a0055350 con 0x7f47b013df80 2024-01-15T17:57:38.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.408+0000 7f47a6000640 1 -- 172.21.15.26:0/56554998 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f479802caa0 con 0x7f478806b3d0 2024-01-15T17:57:38.411 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:38.411 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:57:38.411 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T17:57:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:57:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:57:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:57:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:57:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:38.412 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:38.412 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f478806b3d0 msgr2=0x7f478806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f478806b3d0 0x7f478806d890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4798035000 tx=0x7f479802e000 comp rx=0 tx=0).stop 2024-01-15T17:57:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013df80 msgr2=0x7f47b01f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013df80 0x7f47b01f6720 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f47a000d600 tx=0x7f47a000dac0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 shutdown_connections 2024-01-15T17:57:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47b013e360 0x7f47b0153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f478806b3d0 0x7f478806d890 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47b015f9c0 0x7f47b015d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 --2- 172.21.15.26:0/56554998 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47b013df80 0x7f47b01f6720 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.411+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 >> 172.21.15.26:0/56554998 conn(0x7f47b006a800 msgr2=0x7f47b00cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.412+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 shutdown_connections 2024-01-15T17:57:38.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.412+0000 7f47b5200640 1 -- 172.21.15.26:0/56554998 wait complete. 2024-01-15T17:57:38.434 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T17:57:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.644+0000 7f527f600640 1 Processor -- start 2024-01-15T17:57:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.644+0000 7f527f600640 1 -- start start 2024-01-15T17:57:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.646+0000 7f527f600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f527814b2c0 0x7f52780cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.646+0000 7f527f600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52780cf7d0 0x7f5278070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.646+0000 7f527f600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52780cfbf0 con 0x7f5278070ea0 2024-01-15T17:57:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.646+0000 7f527f600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527814b6a0 con 0x7f52780cf7d0 2024-01-15T17:57:38.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.646+0000 7f527f600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5278071690 con 0x7f527814b2c0 2024-01-15T17:57:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.646+0000 7f526fc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52780cf7d0 0x7f5278070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.646+0000 7f526fc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52780cf7d0 0x7f5278070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53544/0 (socket says 172.21.15.26:53544) 2024-01-15T17:57:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.646+0000 7f526fc00640 1 -- 172.21.15.26:0/3478713332 learned_addr learned my addr 172.21.15.26:0/3478713332 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.647+0000 7f527cc00640 1 --2- 172.21.15.26:0/3478713332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f527814b2c0 0x7f52780cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.647+0000 7f526fc00640 1 -- 172.21.15.26:0/3478713332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f527814b2c0 msgr2=0x7f52780cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.647+0000 7f526fc00640 1 --2- 172.21.15.26:0/3478713332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f527814b2c0 0x7f52780cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.647+0000 7f526fc00640 1 -- 172.21.15.26:0/3478713332 >> v1:172.21.15.124:6789/0 conn(0x7f5278070ea0 legacy=0x7f52780b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.647+0000 7f526fc00640 1 -- 172.21.15.26:0/3478713332 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5278071970 con 0x7f52780cf7d0 2024-01-15T17:57:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.648+0000 7f526fc00640 1 --2- 172.21.15.26:0/3478713332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52780cf7d0 0x7f5278070960 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5260009870 tx=0x7f526002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e0ae06b400a1c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:38.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.648+0000 7f5277e00640 1 -- 172.21.15.26:0/3478713332 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260030840 con 0x7f52780cf7d0 2024-01-15T17:57:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f5277e00640 1 -- 172.21.15.26:0/3478713332 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52600309a0 con 0x7f52780cf7d0 2024-01-15T17:57:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f5277e00640 1 -- 172.21.15.26:0/3478713332 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260030d30 con 0x7f52780cf7d0 2024-01-15T17:57:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f527f600640 1 -- 172.21.15.26:0/3478713332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52780cf7d0 msgr2=0x7f5278070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:38.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f527f600640 1 --2- 172.21.15.26:0/3478713332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52780cf7d0 0x7f5278070960 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5260009870 tx=0x7f526002b980 comp rx=0 tx=0).stop 2024-01-15T17:57:38.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f527f600640 1 -- 172.21.15.26:0/3478713332 shutdown_connections 2024-01-15T17:57:38.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f527f600640 1 --2- 172.21.15.26:0/3478713332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52780cf7d0 0x7f5278070960 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f527f600640 1 --2- 172.21.15.26:0/3478713332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f527814b2c0 0x7f52780cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f527f600640 1 -- 172.21.15.26:0/3478713332 >> 172.21.15.26:0/3478713332 conn(0x7f527806a800 msgr2=0x7f527806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:38.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f527f600640 1 -- 172.21.15.26:0/3478713332 shutdown_connections 2024-01-15T17:57:38.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.649+0000 7f527f600640 1 -- 172.21.15.26:0/3478713332 wait complete. 2024-01-15T17:57:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.651+0000 7f527f600640 1 Processor -- start 2024-01-15T17:57:38.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.651+0000 7f527f600640 1 -- start start 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.652+0000 7f527f600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5278070ea0 0x7f527815f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.652+0000 7f527f600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52780cf7d0 0x7f5278160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.652+0000 7f527f600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527814b2c0 0x7f52781fa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.652+0000 7f527f600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52781fb430 con 0x7f527814b2c0 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.652+0000 7f527f600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5278142e60 con 0x7f52780cf7d0 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.652+0000 7f527f600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5278143140 con 0x7f5278070ea0 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.652+0000 7f527d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527814b2c0 0x7f52781fa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.652+0000 7f527d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527814b2c0 0x7f52781fa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53556/0 (socket says 172.21.15.26:53556) 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.652+0000 7f527d600640 1 -- 172.21.15.26:0/4257931216 learned_addr learned my addr 172.21.15.26:0/4257931216 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:38.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.653+0000 7f526fc00640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52780cf7d0 0x7f5278160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:38.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.653+0000 7f527cc00640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5278070ea0 0x7f527815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:38.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.653+0000 7f527d600640 1 -- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5278070ea0 msgr2=0x7f527815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:38.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.653+0000 7f527d600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5278070ea0 0x7f527815f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.653+0000 7f527d600640 1 -- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52780cf7d0 msgr2=0x7f5278160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:38.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.653+0000 7f527d600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52780cf7d0 0x7f5278160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.653+0000 7f527d600640 1 -- 172.21.15.26:0/4257931216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600094d0 con 0x7f527814b2c0 2024-01-15T17:57:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.653+0000 7f527d600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527814b2c0 0x7f52781fa5d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f527000c7a0 tx=0x7f527000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.654+0000 7f5275600640 1 -- 172.21.15.26:0/4257931216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270003b70 con 0x7f527814b2c0 2024-01-15T17:57:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.654+0000 7f5275600640 1 -- 172.21.15.26:0/4257931216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5270003cf0 con 0x7f527814b2c0 2024-01-15T17:57:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.654+0000 7f5275600640 1 -- 172.21.15.26:0/4257931216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270003ea0 con 0x7f527814b2c0 2024-01-15T17:57:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.654+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5278143420 con 0x7f527814b2c0 2024-01-15T17:57:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.654+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52781439e0 con 0x7f527814b2c0 2024-01-15T17:57:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.655+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52780675d0 con 0x7f527814b2c0 2024-01-15T17:57:38.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.656+0000 7f5275600640 1 -- 172.21.15.26:0/4257931216 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f527000aa90 con 0x7f527814b2c0 2024-01-15T17:57:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.657+0000 7f5275600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f525006b4a0 0x7f525006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.657+0000 7f5275600640 1 -- 172.21.15.26:0/4257931216 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5270089d10 con 0x7f527814b2c0 2024-01-15T17:57:38.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.657+0000 7f527cc00640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f525006b4a0 0x7f525006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:38.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.658+0000 7f527cc00640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f525006b4a0 0x7f525006d960 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5268006fd0 tx=0x7f5268008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.661+0000 7f5275600640 1 -- 172.21.15.26:0/4257931216 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f527008f020 con 0x7f527814b2c0 2024-01-15T17:57:38.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.929+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5278147da0 con 0x7f527814b2c0 2024-01-15T17:57:38.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.930+0000 7f5275600640 1 -- 172.21.15.26:0/4257931216 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1856 (secure 0 0 0) 0x7f52700548f0 con 0x7f527814b2c0 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T17:57:38.934 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 173, 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T17:57:38.935 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 35, 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T17:57:38.936 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 796, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3222104795, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 6702850048, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 766391263232, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 85, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 65347728, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 0, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 18 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T17:57:38.937 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 17, 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 2, 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 2, 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "b", 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 34619 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:57:38.938 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 9, 2024-01-15T17:57:38.939 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:57:31.962932+0000", 2024-01-15T17:57:38.940 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:57:38.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:38.940 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": {} 2024-01-15T17:57:38.940 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f525006b4a0 msgr2=0x7f525006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f525006b4a0 0x7f525006d960 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5268006fd0 tx=0x7f5268008040 comp rx=0 tx=0).stop 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527814b2c0 msgr2=0x7f52781fa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527814b2c0 0x7f52781fa5d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f527000c7a0 tx=0x7f527000cc60 comp rx=0 tx=0).stop 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 shutdown_connections 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f525006b4a0 0x7f525006d960 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f527814b2c0 0x7f52781fa5d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52780cf7d0 0x7f5278160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 --2- 172.21.15.26:0/4257931216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5278070ea0 0x7f527815f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:38.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 >> 172.21.15.26:0/4257931216 conn(0x7f527806a800 msgr2=0x7f52780ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 shutdown_connections 2024-01-15T17:57:38.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:38.933+0000 7f527f600640 1 -- 172.21.15.26:0/4257931216 wait complete. 2024-01-15T17:57:38.956 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T17:57:38.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T17:57:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.177+0000 7f2371600640 1 Processor -- start 2024-01-15T17:57:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.177+0000 7f2371600640 1 -- start start 2024-01-15T17:57:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.178+0000 7f2371600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f236c15d4b0 0x7f236c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.178+0000 7f2371600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f236c0cf9a0 0x7f236c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f2371600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236c0d0320 con 0x7f236c0cddf0 2024-01-15T17:57:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f2371600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0b2730 con 0x7f236c15d4b0 2024-01-15T17:57:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f2371600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236c0d1ec0 con 0x7f236c0cf9a0 2024-01-15T17:57:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f236be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f236c15d4b0 0x7f236c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f236be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f236c15d4b0 0x7f236c0d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53564/0 (socket says 172.21.15.26:53564) 2024-01-15T17:57:39.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f236be00640 1 -- 172.21.15.26:0/1038425470 learned_addr learned my addr 172.21.15.26:0/1038425470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f236b400640 1 --2- 172.21.15.26:0/1038425470 >> v2:172.21.15.124:3300/0 conn(0x7f236c0cf9a0 0x7f236c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f236be00640 1 -- 172.21.15.26:0/1038425470 >> v2:172.21.15.124:3300/0 conn(0x7f236c0cf9a0 msgr2=0x7f236c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f236be00640 1 --2- 172.21.15.26:0/1038425470 >> v2:172.21.15.124:3300/0 conn(0x7f236c0cf9a0 0x7f236c0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f236be00640 1 -- 172.21.15.26:0/1038425470 >> v1:172.21.15.26:6789/0 conn(0x7f236c0cddf0 legacy=0x7f236c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f236be00640 1 -- 172.21.15.26:0/1038425470 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0b29d0 con 0x7f236c15d4b0 2024-01-15T17:57:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.179+0000 7f236b400640 1 --2- 172.21.15.26:0/1038425470 >> v2:172.21.15.124:3300/0 conn(0x7f236c0cf9a0 0x7f236c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:39.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.181+0000 7f236be00640 1 --2- 172.21.15.26:0/1038425470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 0x7f236c0d1550 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f236c08b580 tx=0x7f235c02b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f641aebecb846b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:39.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.181+0000 7f236aa00640 1 -- 172.21.15.26:0/1038425470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c030880 con 0x7f236c15d4b0 2024-01-15T17:57:39.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.181+0000 7f236aa00640 1 -- 172.21.15.26:0/1038425470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235c0309e0 con 0x7f236c15d4b0 2024-01-15T17:57:39.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.181+0000 7f236aa00640 1 -- 172.21.15.26:0/1038425470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235c030d70 con 0x7f236c15d4b0 2024-01-15T17:57:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.182+0000 7f2371600640 1 -- 172.21.15.26:0/1038425470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 msgr2=0x7f236c0d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.182+0000 7f2371600640 1 --2- 172.21.15.26:0/1038425470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 0x7f236c0d1550 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f236c08b580 tx=0x7f235c02b9c0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.182+0000 7f2371600640 1 -- 172.21.15.26:0/1038425470 shutdown_connections 2024-01-15T17:57:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.182+0000 7f2371600640 1 --2- 172.21.15.26:0/1038425470 >> v2:172.21.15.124:3300/0 conn(0x7f236c0cf9a0 0x7f236c0cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.182+0000 7f2371600640 1 --2- 172.21.15.26:0/1038425470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 0x7f236c0d1550 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.182+0000 7f2371600640 1 -- 172.21.15.26:0/1038425470 >> 172.21.15.26:0/1038425470 conn(0x7f236c0d3000 msgr2=0x7f236c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.182+0000 7f2371600640 1 -- 172.21.15.26:0/1038425470 shutdown_connections 2024-01-15T17:57:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.182+0000 7f2371600640 1 -- 172.21.15.26:0/1038425470 wait complete. 2024-01-15T17:57:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.184+0000 7f2371600640 1 Processor -- start 2024-01-15T17:57:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.184+0000 7f2371600640 1 -- start start 2024-01-15T17:57:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f2371600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236c0cddf0 0x7f236c0a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f2371600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236c0cf9a0 0x7f236c1f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f2371600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 0x7f236c09e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f2371600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c0a0d00 con 0x7f236c15d4b0 2024-01-15T17:57:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f2371600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c09d150 con 0x7f236c0cf9a0 2024-01-15T17:57:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f2371600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c09d430 con 0x7f236c0cddf0 2024-01-15T17:57:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f2370c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 0x7f236c09e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f2370c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 0x7f236c09e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53576/0 (socket says 172.21.15.26:53576) 2024-01-15T17:57:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f2370c00640 1 -- 172.21.15.26:0/2221423472 learned_addr learned my addr 172.21.15.26:0/2221423472 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.185+0000 7f236b400640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236c0cf9a0 0x7f236c1f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:39.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f236be00640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236c0cddf0 0x7f236c0a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2370c00640 1 -- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236c0cddf0 msgr2=0x7f236c0a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2370c00640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236c0cddf0 0x7f236c0a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2370c00640 1 -- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236c0cf9a0 msgr2=0x7f236c1f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2370c00640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236c0cf9a0 0x7f236c1f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2370c00640 1 -- 172.21.15.26:0/2221423472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0094d0 con 0x7f236c15d4b0 2024-01-15T17:57:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2370c00640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 0x7f236c09e7b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2364001eb0 tx=0x7f236400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:39.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2363e00640 1 -- 172.21.15.26:0/2221423472 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640052d0 con 0x7f236c15d4b0 2024-01-15T17:57:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2363e00640 1 -- 172.21.15.26:0/2221423472 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2364005430 con 0x7f236c15d4b0 2024-01-15T17:57:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2363e00640 1 -- 172.21.15.26:0/2221423472 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364003810 con 0x7f236c15d4b0 2024-01-15T17:57:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.186+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c15da10 con 0x7f236c15d4b0 2024-01-15T17:57:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.187+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c09d9f0 con 0x7f236c15d4b0 2024-01-15T17:57:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.188+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c067450 con 0x7f236c15d4b0 2024-01-15T17:57:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.189+0000 7f2363e00640 1 -- 172.21.15.26:0/2221423472 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f2364003970 con 0x7f236c15d4b0 2024-01-15T17:57:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.190+0000 7f2363e00640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234406b470 0x7f234406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.190+0000 7f2363e00640 1 -- 172.21.15.26:0/2221423472 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2364089550 con 0x7f236c15d4b0 2024-01-15T17:57:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.190+0000 7f236be00640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234406b470 0x7f234406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.191+0000 7f236be00640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234406b470 0x7f234406d930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f235c036000 tx=0x7f235c03b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:39.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.194+0000 7f2363e00640 1 -- 172.21.15.26:0/2221423472 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23640540a0 con 0x7f236c15d4b0 2024-01-15T17:57:39.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.434+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f236c0cd990 con 0x7f234406b470 2024-01-15T17:57:39.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.448+0000 7f2363e00640 1 -- 172.21.15.26:0/2221423472 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f236c0cd990 con 0x7f234406b470 2024-01-15T17:57:39.451 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 0.0, 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "0 B/3 GB", 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "0/3", 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T17:57:39.452 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T17:57:39.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:39.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:39.453 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:39.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234406b470 msgr2=0x7f234406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234406b470 0x7f234406d930 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f235c036000 tx=0x7f235c03b040 comp rx=0 tx=0).stop 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 msgr2=0x7f236c09e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 0x7f236c09e7b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2364001eb0 tx=0x7f236400c890 comp rx=0 tx=0).stop 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 shutdown_connections 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234406b470 0x7f234406d930 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236c15d4b0 0x7f236c09e7b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236c0cf9a0 0x7f236c1f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 --2- 172.21.15.26:0/2221423472 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236c0cddf0 0x7f236c0a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 >> 172.21.15.26:0/2221423472 conn(0x7f236c0d3000 msgr2=0x7f236c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 shutdown_connections 2024-01-15T17:57:39.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:39.451+0000 7f2371600640 1 -- 172.21.15.26:0/2221423472 wait complete. 2024-01-15T17:57:39.472 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T17:57:40.473 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T17:57:40.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.721+0000 7f5040200640 1 Processor -- start 2024-01-15T17:57:40.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.722+0000 7f5040200640 1 -- start start 2024-01-15T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.723+0000 7f5040200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f50380d0280 0x7f50380ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.723+0000 7f5040200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380b6750 con 0x7f503815d590 2024-01-15T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.723+0000 7f5040200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50380cf1f0 con 0x7f50380ce6f0 2024-01-15T17:57:40.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.723+0000 7f5040200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50380cef10 con 0x7f50380d0280 2024-01-15T17:57:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.723+0000 7f503d800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f503815d590 0x7f50380cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36474/0 (socket says 172.21.15.26:36474) 2024-01-15T17:57:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.723+0000 7f503d800640 1 -- 172.21.15.26:0/560374182 learned_addr learned my addr 172.21.15.26:0/560374182 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:57:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998140467 0 0) 0x7f50380b6750 con 0x7f503815d590 2024-01-15T17:57:40.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5018003520 con 0x7f503815d590 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f503ce00640 1 --2- 172.21.15.26:0/560374182 >> v2:172.21.15.124:3300/0 conn(0x7f50380d0280 0x7f50380ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2500929169 0 0) 0x7f5018003520 con 0x7f503815d590 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 >> v2:172.21.15.124:3300/0 conn(0x7f50380d0280 msgr2=0x7f50380ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 --2- 172.21.15.26:0/560374182 >> v2:172.21.15.124:3300/0 conn(0x7f50380d0280 0x7f50380ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 >> v1:172.21.15.124:6789/0 conn(0x7f50380ce6f0 legacy=0x7f50380b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50380b6100 con 0x7f503815d590 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f50300019f0 con 0x7f503815d590 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 >> v1:172.21.15.26:6789/0 conn(0x7f503815d590 legacy=0x7f50380cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 --2- 172.21.15.26:0/560374182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50180039c0 0x7f5018003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.724+0000 7f5037e00640 1 --2- 172.21.15.26:0/560374182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50180047c0 0x7f5018004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f5037e00640 1 --2- 172.21.15.26:0/560374182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180051d0 0x7f501800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50380cef10 con 0x7f50180047c0 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501800c120 con 0x7f50180051d0 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501800c2e0 con 0x7f50180039c0 2024-01-15T17:57:40.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f503ce00640 1 --2- 172.21.15.26:0/560374182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50180047c0 0x7f5018004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f503ce00640 1 -- 172.21.15.26:0/560374182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50180039c0 msgr2=0x7f5018003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f503ce00640 1 --2- 172.21.15.26:0/560374182 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50180039c0 0x7f5018003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f503ce00640 1 -- 172.21.15.26:0/560374182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180051d0 msgr2=0x7f501800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f503ce00640 1 --2- 172.21.15.26:0/560374182 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180051d0 0x7f501800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f503ce00640 1 -- 172.21.15.26:0/560374182 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5018003520 con 0x7f50180047c0 2024-01-15T17:57:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.725+0000 7f503e200640 1 -- 172.21.15.26:0/560374182 reap_dead start 2024-01-15T17:57:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.727+0000 7f503ce00640 1 --2- 172.21.15.26:0/560374182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50180047c0 0x7f5018004c40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f502800ba30 tx=0x7f50180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c00d695c61e771 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:40.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.727+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028032680 con 0x7f50180047c0 2024-01-15T17:57:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.727+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50280327e0 con 0x7f50180047c0 2024-01-15T17:57:40.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.727+0000 7f5037e00640 1 -- 172.21.15.26:0/560374182 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5028032b70 con 0x7f50180047c0 2024-01-15T17:57:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.728+0000 7f5040200640 1 -- 172.21.15.26:0/560374182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50180047c0 msgr2=0x7f5018004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.728+0000 7f5040200640 1 --2- 172.21.15.26:0/560374182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50180047c0 0x7f5018004c40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f502800ba30 tx=0x7f50180057f0 comp rx=0 tx=0).stop 2024-01-15T17:57:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.728+0000 7f5040200640 1 -- 172.21.15.26:0/560374182 shutdown_connections 2024-01-15T17:57:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.728+0000 7f5040200640 1 --2- 172.21.15.26:0/560374182 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50180047c0 0x7f5018004c40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.728+0000 7f5040200640 1 -- 172.21.15.26:0/560374182 >> 172.21.15.26:0/560374182 conn(0x7f503809c540 msgr2=0x7f50380d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.728+0000 7f5040200640 1 -- 172.21.15.26:0/560374182 shutdown_connections 2024-01-15T17:57:40.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.729+0000 7f5040200640 1 -- 172.21.15.26:0/560374182 wait complete. 2024-01-15T17:57:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.730+0000 7f5040200640 1 Processor -- start 2024-01-15T17:57:40.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.730+0000 7f5040200640 1 -- start start 2024-01-15T17:57:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.731+0000 7f5040200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180047c0 0x7f503806c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.731+0000 7f5040200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f503806c820 0x7f50380ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.731+0000 7f5040200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cd130 0x7f50380ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.731+0000 7f5040200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50380c9a00 con 0x7f50380cd130 2024-01-15T17:57:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.731+0000 7f5040200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503806cca0 con 0x7f50180047c0 2024-01-15T17:57:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f5040200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50380cd5b0 con 0x7f503806c820 2024-01-15T17:57:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f503e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cd130 0x7f50380ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f503e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cd130 0x7f50380ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53582/0 (socket says 172.21.15.26:53582) 2024-01-15T17:57:40.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f503e200640 1 -- 172.21.15.26:0/423930325 learned_addr learned my addr 172.21.15.26:0/423930325 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:40.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f503d800640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180047c0 0x7f503806c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f503e200640 1 -- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f503806c820 msgr2=0x7f50380ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f503e200640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f503806c820 0x7f50380ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f503e200640 1 -- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180047c0 msgr2=0x7f503806c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f503e200640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180047c0 0x7f503806c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.732+0000 7f503e200640 1 -- 172.21.15.26:0/423930325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50381f6410 con 0x7f50380cd130 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.733+0000 7f503e200640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cd130 0x7f50380ca2e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f50380d4310 tx=0x7f502000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.733+0000 7f5035600640 1 -- 172.21.15.26:0/423930325 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200072d0 con 0x7f50380cd130 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.733+0000 7f5035600640 1 -- 172.21.15.26:0/423930325 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5020007450 con 0x7f50380cd130 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.733+0000 7f5035600640 1 -- 172.21.15.26:0/423930325 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200057c0 con 0x7f50380cd130 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.733+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50381f66f0 con 0x7f50380cd130 2024-01-15T17:57:40.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.733+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50381f6cb0 con 0x7f50380cd130 2024-01-15T17:57:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.734+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50381f71d0 con 0x7f50380cd130 2024-01-15T17:57:40.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.735+0000 7f5035600640 1 -- 172.21.15.26:0/423930325 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f5020005b30 con 0x7f50380cd130 2024-01-15T17:57:40.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.737+0000 7f5035600640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f501006b4a0 0x7f501006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:40.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.737+0000 7f5035600640 1 -- 172.21.15.26:0/423930325 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50200893d0 con 0x7f50380cd130 2024-01-15T17:57:40.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.741+0000 7f5035600640 1 -- 172.21.15.26:0/423930325 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50381f71d0 con 0x7f50380cd130 2024-01-15T17:57:40.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.745+0000 7f503d800640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f501006b4a0 0x7f501006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:40.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:40.746+0000 7f503d800640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f501006b4a0 0x7f501006d960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f50380695b0 tx=0x7f5030009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:41.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.072+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f50380ceba0 con 0x7f501006b4a0 2024-01-15T17:57:41.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.087+0000 7f5035600640 1 -- 172.21.15.26:0/423930325 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f5030001940 con 0x7f501006b4a0 2024-01-15T17:57:41.091 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:41.091 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 0.0, 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "0 B/3 GB", 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "0/3", 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T17:57:41.092 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:41.093 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:41.093 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.091+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f501006b4a0 msgr2=0x7f501006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.091+0000 7f5040200640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f501006b4a0 0x7f501006d960 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f50380695b0 tx=0x7f5030009040 comp rx=0 tx=0).stop 2024-01-15T17:57:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.091+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cd130 msgr2=0x7f50380ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:41.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.091+0000 7f5040200640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cd130 0x7f50380ca2e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f50380d4310 tx=0x7f502000c820 comp rx=0 tx=0).stop 2024-01-15T17:57:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.092+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 shutdown_connections 2024-01-15T17:57:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.092+0000 7f5040200640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f503806c820 0x7f50380ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.092+0000 7f5040200640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f501006b4a0 0x7f501006d960 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.092+0000 7f5040200640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50380cd130 0x7f50380ca2e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.092+0000 7f5040200640 1 --2- 172.21.15.26:0/423930325 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50180047c0 0x7f503806c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:41.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.092+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 >> 172.21.15.26:0/423930325 conn(0x7f503809c540 msgr2=0x7f503815fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.092+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 shutdown_connections 2024-01-15T17:57:41.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:41.092+0000 7f5040200640 1 -- 172.21.15.26:0/423930325 wait complete. 2024-01-15T17:57:41.120 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T17:57:42.122 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T17:57:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.421+0000 7f9fb3800640 1 Processor -- start 2024-01-15T17:57:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.421+0000 7f9fb3800640 1 -- start start 2024-01-15T17:57:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.422+0000 7f9fb3800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9fac14b2c0 0x7f9fac0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.422+0000 7f9fb3800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9fac0cf7d0 0x7f9fac070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.422+0000 7f9fb3800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 0x7f9fac0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.422+0000 7f9fb3800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fac14b6a0 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.422+0000 7f9fb3800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fac0cfbf0 con 0x7f9fac14b2c0 2024-01-15T17:57:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.422+0000 7f9fb3800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac071690 con 0x7f9fac070ea0 2024-01-15T17:57:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.423+0000 7f9fabe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9fac0cf7d0 0x7f9fac070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.423+0000 7f9fabe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9fac0cf7d0 0x7f9fac070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39170/0 (socket says 172.21.15.26:39170) 2024-01-15T17:57:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.423+0000 7f9fabe00640 1 -- 172.21.15.26:0/2074232611 learned_addr learned my addr 172.21.15.26:0/2074232611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.423+0000 7f9fb0e00640 1 --2- 172.21.15.26:0/2074232611 >> v2:172.21.15.124:3300/0 conn(0x7f9fac14b2c0 0x7f9fac0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.424+0000 7f9fabe00640 1 -- 172.21.15.26:0/2074232611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 msgr2=0x7f9fac0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.424+0000 7f9fb1800640 1 --2- 172.21.15.26:0/2074232611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 0x7f9fac0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.424+0000 7f9fabe00640 1 --2- 172.21.15.26:0/2074232611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 0x7f9fac0b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.424+0000 7f9fabe00640 1 -- 172.21.15.26:0/2074232611 >> v2:172.21.15.124:3300/0 conn(0x7f9fac14b2c0 msgr2=0x7f9fac0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.424+0000 7f9fabe00640 1 --2- 172.21.15.26:0/2074232611 >> v2:172.21.15.124:3300/0 conn(0x7f9fac14b2c0 0x7f9fac0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.424+0000 7f9fabe00640 1 -- 172.21.15.26:0/2074232611 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac071970 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.424+0000 7f9fb1800640 1 --2- 172.21.15.26:0/2074232611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 0x7f9fac0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.425+0000 7f9fabe00640 1 --2- 172.21.15.26:0/2074232611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 0x7f9fac070960 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9fac06e650 tx=0x7f9f9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45c15482185bb30 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.425+0000 7f9fab400640 1 -- 172.21.15.26:0/2074232611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94031870 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.425+0000 7f9fab400640 1 -- 172.21.15.26:0/2074232611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f940319d0 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.425+0000 7f9fab400640 1 -- 172.21.15.26:0/2074232611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f94031d60 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.426+0000 7f9fb3800640 1 -- 172.21.15.26:0/2074232611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 msgr2=0x7f9fac070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.426+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2074232611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 0x7f9fac070960 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9fac06e650 tx=0x7f9f9402b910 comp rx=0 tx=0).stop 2024-01-15T17:57:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.426+0000 7f9fb3800640 1 -- 172.21.15.26:0/2074232611 shutdown_connections 2024-01-15T17:57:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.426+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2074232611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 0x7f9fac0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.426+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2074232611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 0x7f9fac070960 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.426+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2074232611 >> v2:172.21.15.124:3300/0 conn(0x7f9fac14b2c0 0x7f9fac0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.426+0000 7f9fb3800640 1 -- 172.21.15.26:0/2074232611 >> 172.21.15.26:0/2074232611 conn(0x7f9fac06a800 msgr2=0x7f9fac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.426+0000 7f9fb3800640 1 -- 172.21.15.26:0/2074232611 shutdown_connections 2024-01-15T17:57:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.427+0000 7f9fb3800640 1 -- 172.21.15.26:0/2074232611 wait complete. 2024-01-15T17:57:42.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.429+0000 7f9fb3800640 1 Processor -- start 2024-01-15T17:57:42.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.429+0000 7f9fb3800640 1 -- start start 2024-01-15T17:57:42.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.431+0000 7f9fb3800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 0x7f9fac15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:42.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.431+0000 7f9fb3800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 0x7f9fac146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:42.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.431+0000 7f9fb3800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fac14b2c0 0x7f9fac164610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:42.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.431+0000 7f9fb3800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac147290 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.431+0000 7f9fb3800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac163e30 con 0x7f9fac14b2c0 2024-01-15T17:57:42.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.431+0000 7f9fb3800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fac15dab0 con 0x7f9fac070ea0 2024-01-15T17:57:42.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.431+0000 7f9fabe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 0x7f9fac146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:42.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.431+0000 7f9fabe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 0x7f9fac146d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39172/0 (socket says 172.21.15.26:39172) 2024-01-15T17:57:42.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.431+0000 7f9fabe00640 1 -- 172.21.15.26:0/2212856238 learned_addr learned my addr 172.21.15.26:0/2212856238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:42.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.432+0000 7f9fb0e00640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 0x7f9fac15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:42.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.432+0000 7f9fb1800640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fac14b2c0 0x7f9fac164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:42.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.432+0000 7f9fabe00640 1 -- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 msgr2=0x7f9fac15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:42.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.432+0000 7f9fabe00640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 0x7f9fac15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.432+0000 7f9fabe00640 1 -- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fac14b2c0 msgr2=0x7f9fac164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.432+0000 7f9fabe00640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fac14b2c0 0x7f9fac164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.432+0000 7f9fabe00640 1 -- 172.21.15.26:0/2212856238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f940094d0 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.432+0000 7f9fb1800640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fac14b2c0 0x7f9fac164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.432+0000 7f9fabe00640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 0x7f9fac146d50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9f9402b950 tx=0x7f9f940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.433+0000 7f9fa8c00640 1 -- 172.21.15.26:0/2212856238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9403ad30 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.433+0000 7f9fa8c00640 1 -- 172.21.15.26:0/2212856238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f940443f0 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.433+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fac15dd90 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.433+0000 7f9fa8c00640 1 -- 172.21.15.26:0/2212856238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f940447a0 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.433+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fac164b50 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.435+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fac164330 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.437+0000 7f9fa8c00640 1 -- 172.21.15.26:0/2212856238 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f9f9404d450 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.438+0000 7f9fa8c00640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f8406b540 0x7f9f8406da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:42.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.438+0000 7f9fa8c00640 1 -- 172.21.15.26:0/2212856238 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f9402d040 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.440+0000 7f9fb0e00640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f8406b540 0x7f9f8406da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:42.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.443+0000 7f9fa8c00640 1 -- 172.21.15.26:0/2212856238 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fac164330 con 0x7f9fac0cf7d0 2024-01-15T17:57:42.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.444+0000 7f9fb0e00640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f8406b540 0x7f9f8406da00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c006fd0 tx=0x7f9f9c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:42.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.754+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fac0cbc40 con 0x7f9f8406b540 2024-01-15T17:57:42.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.767+0000 7f9fa8c00640 1 -- 172.21.15.26:0/2212856238 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f9f9c00b200 con 0x7f9f8406b540 2024-01-15T17:57:42.770 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:42.770 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:57:42.770 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T17:57:42.770 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:42.771 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f8406b540 msgr2=0x7f9f8406da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f8406b540 0x7f9f8406da00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c006fd0 tx=0x7f9f9c008040 comp rx=0 tx=0).stop 2024-01-15T17:57:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 msgr2=0x7f9fac146d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 0x7f9fac146d50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9f9402b950 tx=0x7f9f940046c0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 shutdown_connections 2024-01-15T17:57:42.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f8406b540 0x7f9f8406da00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fac14b2c0 0x7f9fac164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fac0cf7d0 0x7f9fac146d50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 --2- 172.21.15.26:0/2212856238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fac070ea0 0x7f9fac15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.770+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 >> 172.21.15.26:0/2212856238 conn(0x7f9fac06a800 msgr2=0x7f9fac0b0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.771+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 shutdown_connections 2024-01-15T17:57:42.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:42.771+0000 7f9fb3800640 1 -- 172.21.15.26:0/2212856238 wait complete. 2024-01-15T17:57:42.792 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T17:57:43.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T17:57:44.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.028+0000 7fbdd5200640 1 Processor -- start 2024-01-15T17:57:44.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.028+0000 7fbdd5200640 1 -- start start 2024-01-15T17:57:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.029+0000 7fbdd5200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdd00ce8f0 0x7fbdd00ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.030+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd00b2a00 0x7fbdd00b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.030+0000 7fbdd5200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd00d0de0 con 0x7fbdd0160f80 2024-01-15T17:57:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.030+0000 7fbdd5200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd00d10c0 con 0x7fbdd00ce8f0 2024-01-15T17:57:44.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.030+0000 7fbdd5200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd00d13a0 con 0x7fbdd00b2a00 2024-01-15T17:57:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.030+0000 7fbdcd400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdd00ce8f0 0x7fbdd00ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.030+0000 7fbdcd400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbdd00ce8f0 0x7fbdd00ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39184/0 (socket says 172.21.15.26:39184) 2024-01-15T17:57:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.030+0000 7fbdcd400640 1 -- 172.21.15.26:0/928172248 learned_addr learned my addr 172.21.15.26:0/928172248 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.030+0000 7fbdce800640 1 --2- 172.21.15.26:0/928172248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd00b2a00 0x7fbdd00b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.031+0000 7fbdcd400640 1 -- 172.21.15.26:0/928172248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd00b2a00 msgr2=0x7fbdd00b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.031+0000 7fbdcd400640 1 --2- 172.21.15.26:0/928172248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd00b2a00 0x7fbdd00b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.031+0000 7fbdcd400640 1 -- 172.21.15.26:0/928172248 >> v1:172.21.15.124:6789/0 conn(0x7fbdd0160f80 legacy=0x7fbdd0161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.031+0000 7fbdcd400640 1 -- 172.21.15.26:0/928172248 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd00d1680 con 0x7fbdd00ce8f0 2024-01-15T17:57:44.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.031+0000 7fbdce800640 1 --2- 172.21.15.26:0/928172248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd00b2a00 0x7fbdd00b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:44.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.032+0000 7fbdcd400640 1 --2- 172.21.15.26:0/928172248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00ce8f0 0x7fbdd00ced30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009870 tx=0x7fbdb802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd283602014615eb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:44.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.032+0000 7fbdcca00640 1 -- 172.21.15.26:0/928172248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8031640 con 0x7fbdd00ce8f0 2024-01-15T17:57:44.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.032+0000 7fbdcca00640 1 -- 172.21.15.26:0/928172248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb80317a0 con 0x7fbdd00ce8f0 2024-01-15T17:57:44.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.032+0000 7fbdcca00640 1 -- 172.21.15.26:0/928172248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8031b30 con 0x7fbdd00ce8f0 2024-01-15T17:57:44.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.033+0000 7fbdd5200640 1 -- 172.21.15.26:0/928172248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00ce8f0 msgr2=0x7fbdd00ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:44.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.033+0000 7fbdd5200640 1 --2- 172.21.15.26:0/928172248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00ce8f0 0x7fbdd00ced30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8009870 tx=0x7fbdb802b8f0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.033+0000 7fbdd5200640 1 -- 172.21.15.26:0/928172248 shutdown_connections 2024-01-15T17:57:44.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.033+0000 7fbdd5200640 1 --2- 172.21.15.26:0/928172248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd00b2a00 0x7fbdd00b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.033+0000 7fbdd5200640 1 --2- 172.21.15.26:0/928172248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00ce8f0 0x7fbdd00ced30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.033+0000 7fbdd5200640 1 -- 172.21.15.26:0/928172248 >> 172.21.15.26:0/928172248 conn(0x7fbdd009c550 msgr2=0x7fbdd00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:44.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.033+0000 7fbdd5200640 1 -- 172.21.15.26:0/928172248 shutdown_connections 2024-01-15T17:57:44.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.033+0000 7fbdd5200640 1 -- 172.21.15.26:0/928172248 wait complete. 2024-01-15T17:57:44.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.035+0000 7fbdd5200640 1 Processor -- start 2024-01-15T17:57:44.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.035+0000 7fbdd5200640 1 -- start start 2024-01-15T17:57:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.036+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00b2a00 0x7fbdd0160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.036+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd00ce8f0 0x7fbdd01f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.036+0000 7fbdd5200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd0160f80 0x7fbdd00b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.036+0000 7fbdd5200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd0160650 con 0x7fbdd00b2a00 2024-01-15T17:57:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.036+0000 7fbdd5200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd01f3db0 con 0x7fbdd00ce8f0 2024-01-15T17:57:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.036+0000 7fbdd5200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd00b90c0 con 0x7fbdd0160f80 2024-01-15T17:57:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.036+0000 7fbdcde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00b2a00 0x7fbdd0160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:44.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.036+0000 7fbdcde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00b2a00 0x7fbdd0160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39200/0 (socket says 172.21.15.26:39200) 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.036+0000 7fbdcde00640 1 -- 172.21.15.26:0/3703339147 learned_addr learned my addr 172.21.15.26:0/3703339147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdcd400640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd00ce8f0 0x7fbdd01f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdcde00640 1 -- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd0160f80 msgr2=0x7fbdd00b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdcde00640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd0160f80 0x7fbdd00b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdcde00640 1 -- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd00ce8f0 msgr2=0x7fbdd01f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdcde00640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd00ce8f0 0x7fbdd01f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdcde00640 1 -- 172.21.15.26:0/3703339147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb80094d0 con 0x7fbdd00b2a00 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdcd400640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd00ce8f0 0x7fbdd01f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdcde00640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00b2a00 0x7fbdd0160110 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000b3f0 tx=0x7fbdc000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdc6000640 1 -- 172.21.15.26:0/3703339147 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0011830 con 0x7fbdd00b2a00 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdc6000640 1 -- 172.21.15.26:0/3703339147 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc0011990 con 0x7fbdd00b2a00 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.037+0000 7fbdc6000640 1 -- 172.21.15.26:0/3703339147 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0011d40 con 0x7fbdd00b2a00 2024-01-15T17:57:44.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.038+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd00b8020 con 0x7fbdd00b2a00 2024-01-15T17:57:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.038+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd00b85e0 con 0x7fbdd00b2a00 2024-01-15T17:57:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.039+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd00675d0 con 0x7fbdd00b2a00 2024-01-15T17:57:44.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.040+0000 7fbdc6000640 1 -- 172.21.15.26:0/3703339147 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fbdc001a400 con 0x7fbdd00b2a00 2024-01-15T17:57:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.041+0000 7fbdc6000640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbda806b4a0 0x7fbda806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.041+0000 7fbdc6000640 1 -- 172.21.15.26:0/3703339147 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdc0089ea0 con 0x7fbdd00b2a00 2024-01-15T17:57:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.041+0000 7fbdcd400640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbda806b4a0 0x7fbda806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.042+0000 7fbdcd400640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbda806b4a0 0x7fbda806d960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8036000 tx=0x7fbdb803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.045+0000 7fbdc6000640 1 -- 172.21.15.26:0/3703339147 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbdc00549f0 con 0x7fbdd00b2a00 2024-01-15T17:57:44.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.278+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdd00d1c40 con 0x7fbda806b4a0 2024-01-15T17:57:44.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.296+0000 7fbdc6000640 1 -- 172.21.15.26:0/3703339147 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fbdb8031a90 con 0x7fbda806b4a0 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-15T17:57:44.299 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T17:57:44.300 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T17:57:44.300 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:44.300 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:44.300 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:44.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbda806b4a0 msgr2=0x7fbda806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbda806b4a0 0x7fbda806d960 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8036000 tx=0x7fbdb803c040 comp rx=0 tx=0).stop 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00b2a00 msgr2=0x7fbdd0160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00b2a00 0x7fbdd0160110 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000b3f0 tx=0x7fbdc000b8b0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 shutdown_connections 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd0160f80 0x7fbdd00b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbda806b4a0 0x7fbda806d960 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd00ce8f0 0x7fbdd01f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 --2- 172.21.15.26:0/3703339147 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdd00b2a00 0x7fbdd0160110 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 >> 172.21.15.26:0/3703339147 conn(0x7fbdd009c550 msgr2=0x7fbdd006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 shutdown_connections 2024-01-15T17:57:44.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:44.299+0000 7fbdd5200640 1 -- 172.21.15.26:0/3703339147 wait complete. 2024-01-15T17:57:44.323 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T17:57:45.324 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T17:57:45.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.593+0000 7fb559e00640 1 Processor -- start 2024-01-15T17:57:45.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.594+0000 7fb559e00640 1 -- start start 2024-01-15T17:57:45.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.595+0000 7fb559e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5540cf9a0 0x7fb554166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:45.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.595+0000 7fb559e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55415c5f0 con 0x7fb55413df80 2024-01-15T17:57:45.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.595+0000 7fb559e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55406c470 con 0x7fb55413e360 2024-01-15T17:57:45.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.595+0000 7fb559e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55406c750 con 0x7fb5540cf9a0 2024-01-15T17:57:45.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.596+0000 7fb552a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb55413df80 0x7fb55415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50704/0 (socket says 172.21.15.26:50704) 2024-01-15T17:57:45.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.596+0000 7fb552a00640 1 -- 172.21.15.26:0/3848541868 learned_addr learned my addr 172.21.15.26:0/3848541868 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:57:45.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.596+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411171489 0 0) 0x7fb55415c5f0 con 0x7fb55413df80 2024-01-15T17:57:45.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.596+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb534003520 con 0x7fb55413df80 2024-01-15T17:57:45.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.596+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1889492346 0 0) 0x7fb534003520 con 0x7fb55413df80 2024-01-15T17:57:45.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.596+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5540cf9a0 msgr2=0x7fb554166030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T17:57:45.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.596+0000 7fb552000640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5540cf9a0 0x7fb554166030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 >> v1:172.21.15.124:6789/0 conn(0x7fb55413e360 legacy=0x7fb5540cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55406ca30 con 0x7fb55413df80 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb544001670 con 0x7fb55413df80 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 >> v1:172.21.15.26:6789/0 conn(0x7fb55413df80 legacy=0x7fb55415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5540cf9a0 0x7fb5340039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5340040b0 0x7fb5340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb534004e20 0x7fb53400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55406c750 con 0x7fb5340040b0 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb534005500 con 0x7fb534004e20 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53400bec0 con 0x7fb5540cf9a0 2024-01-15T17:57:45.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb549e00640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5340040b0 0x7fb5340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.597+0000 7fb552a00640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5540cf9a0 0x7fb5340039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.598+0000 7fb549e00640 1 -- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5540cf9a0 msgr2=0x7fb5340039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.598+0000 7fb553400640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb534004e20 0x7fb53400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.598+0000 7fb549e00640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5540cf9a0 0x7fb5340039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.598+0000 7fb549e00640 1 -- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb534004e20 msgr2=0x7fb53400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.598+0000 7fb549e00640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb534004e20 0x7fb53400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:45.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.598+0000 7fb549e00640 1 -- 172.21.15.26:0/3848541868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb534003520 con 0x7fb5340040b0 2024-01-15T17:57:45.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.598+0000 7fb552a00640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5540cf9a0 0x7fb5340039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:45.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.598+0000 7fb553400640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb534004e20 0x7fb53400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:45.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.599+0000 7fb549e00640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5340040b0 0x7fb5340047c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb53c001d40 tx=0x7fb5340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a697a719c2824e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:45.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.600+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c003990 con 0x7fb5340040b0 2024-01-15T17:57:45.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.600+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c003af0 con 0x7fb5340040b0 2024-01-15T17:57:45.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.600+0000 7fb552000640 1 -- 172.21.15.26:0/3848541868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c003ea0 con 0x7fb5340040b0 2024-01-15T17:57:45.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.601+0000 7fb559e00640 1 -- 172.21.15.26:0/3848541868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5340040b0 msgr2=0x7fb5340047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:45.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.601+0000 7fb559e00640 1 --2- 172.21.15.26:0/3848541868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5340040b0 0x7fb5340047c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb53c001d40 tx=0x7fb5340057f0 comp rx=0 tx=0).stop 2024-01-15T17:57:45.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.601+0000 7fb553400640 1 -- 172.21.15.26:0/3848541868 reap_dead start 2024-01-15T17:57:45.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.601+0000 7fb559e00640 1 -- 172.21.15.26:0/3848541868 shutdown_connections 2024-01-15T17:57:45.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.601+0000 7fb559e00640 1 -- 172.21.15.26:0/3848541868 >> 172.21.15.26:0/3848541868 conn(0x7fb55406a800 msgr2=0x7fb55406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:45.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.601+0000 7fb559e00640 1 -- 172.21.15.26:0/3848541868 shutdown_connections 2024-01-15T17:57:45.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.601+0000 7fb559e00640 1 -- 172.21.15.26:0/3848541868 wait complete. 2024-01-15T17:57:45.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.603+0000 7fb559e00640 1 Processor -- start 2024-01-15T17:57:45.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.603+0000 7fb559e00640 1 -- start start 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.603+0000 7fb559e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5340040b0 0x7fb55414b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.603+0000 7fb559e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb55414a750 0x7fb554149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.603+0000 7fb559e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb55415a180 0x7fb55415a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.603+0000 7fb559e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55415ab40 con 0x7fb55415a180 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.603+0000 7fb559e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5541f3730 con 0x7fb5340040b0 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.603+0000 7fb559e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb554151010 con 0x7fb55414a750 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb553400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb55415a180 0x7fb55415a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb553400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb55415a180 0x7fb55415a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39212/0 (socket says 172.21.15.26:39212) 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb553400640 1 -- 172.21.15.26:0/1229950927 learned_addr learned my addr 172.21.15.26:0/1229950927 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:45.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb552a00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5340040b0 0x7fb55414b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb553400640 1 -- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb55414a750 msgr2=0x7fb554149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb553400640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb55414a750 0x7fb554149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb553400640 1 -- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5340040b0 msgr2=0x7fb55414b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb553400640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5340040b0 0x7fb55414b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb553400640 1 -- 172.21.15.26:0/1229950927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5541f2e00 con 0x7fb55415a180 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.604+0000 7fb553400640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb55415a180 0x7fb55415a600 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb554155d40 tx=0x7fb54c00a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.605+0000 7fb54be00640 1 -- 172.21.15.26:0/1229950927 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c011820 con 0x7fb55415a180 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.605+0000 7fb54be00640 1 -- 172.21.15.26:0/1229950927 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c00c050 con 0x7fb55415a180 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.605+0000 7fb54be00640 1 -- 172.21.15.26:0/1229950927 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c011c00 con 0x7fb55415a180 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.605+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5541f30e0 con 0x7fb55415a180 2024-01-15T17:57:45.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.605+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541f3bc0 con 0x7fb55415a180 2024-01-15T17:57:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.606+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5540675d0 con 0x7fb55415a180 2024-01-15T17:57:45.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.607+0000 7fb54be00640 1 -- 172.21.15.26:0/1229950927 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fb54c01b410 con 0x7fb55415a180 2024-01-15T17:57:45.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.608+0000 7fb54be00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb52c06b4a0 0x7fb52c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:45.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.608+0000 7fb54be00640 1 -- 172.21.15.26:0/1229950927 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb54c0892b0 con 0x7fb55415a180 2024-01-15T17:57:45.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.613+0000 7fb552a00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb52c06b4a0 0x7fb52c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:45.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.616+0000 7fb54be00640 1 -- 172.21.15.26:0/1229950927 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb54c053e50 con 0x7fb55415a180 2024-01-15T17:57:45.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.616+0000 7fb552a00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb52c06b4a0 0x7fb52c06d960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb544001200 tx=0x7fb544009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:45.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:45.990+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb55406cff0 con 0x7fb52c06b4a0 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.008+0000 7fb54be00640 1 -- 172.21.15.26:0/1229950927 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fb54400d050 con 0x7fb52c06b4a0 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T17:57:46.013 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-15T17:57:46.014 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T17:57:46.014 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-15T17:57:46.014 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T17:57:46.014 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T17:57:46.014 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:46.014 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:46.014 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:46.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb52c06b4a0 msgr2=0x7fb52c06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb52c06b4a0 0x7fb52c06d960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb544001200 tx=0x7fb544009040 comp rx=0 tx=0).stop 2024-01-15T17:57:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb55415a180 msgr2=0x7fb55415a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb55415a180 0x7fb55415a600 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb554155d40 tx=0x7fb54c00a870 comp rx=0 tx=0).stop 2024-01-15T17:57:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 shutdown_connections 2024-01-15T17:57:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb55414a750 0x7fb554149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb52c06b4a0 0x7fb52c06d960 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb55415a180 0x7fb55415a600 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 --2- 172.21.15.26:0/1229950927 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5340040b0 0x7fb55414b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:46.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.012+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 >> 172.21.15.26:0/1229950927 conn(0x7fb55406a800 msgr2=0x7fb5540cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:46.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.013+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 shutdown_connections 2024-01-15T17:57:46.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:46.013+0000 7fb559e00640 1 -- 172.21.15.26:0/1229950927 wait complete. 2024-01-15T17:57:46.044 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T17:57:47.046 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T17:57:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.280+0000 7f3cdd200640 1 Processor -- start 2024-01-15T17:57:47.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.280+0000 7f3cdd200640 1 -- start start 2024-01-15T17:57:47.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.282+0000 7f3cdd200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3cd815da10 0x7f3cd8072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:47.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.282+0000 7f3cdd200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd80b3730 con 0x7f3cd8150760 2024-01-15T17:57:47.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.282+0000 7f3cdd200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd8071190 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.282+0000 7f3cdd200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd8071470 con 0x7f3cd815da10 2024-01-15T17:57:47.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.282+0000 7f3cd5e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3cd8150760 0x7f3cd815d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50708/0 (socket says 172.21.15.26:50708) 2024-01-15T17:57:47.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.282+0000 7f3cd5e00640 1 -- 172.21.15.26:0/316737949 learned_addr learned my addr 172.21.15.26:0/316737949 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:57:47.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.282+0000 7f3ccfe00640 1 --2- 172.21.15.26:0/316737949 >> v2:172.21.15.26:3300/0 conn(0x7f3cd815da10 0x7f3cd8072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:47.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.283+0000 7f3ccfe00640 1 -- 172.21.15.26:0/316737949 >> v1:172.21.15.124:6789/0 conn(0x7f3cd80b2bd0 legacy=0x7f3cd80b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:47.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.283+0000 7f3ccfe00640 1 -- 172.21.15.26:0/316737949 >> v1:172.21.15.26:6789/0 conn(0x7f3cd8150760 legacy=0x7f3cd815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:47.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.283+0000 7f3ccfe00640 1 -- 172.21.15.26:0/316737949 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd8071750 con 0x7f3cd815da10 2024-01-15T17:57:47.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.283+0000 7f3cd5400640 1 -- 172.21.15.26:0/316737949 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054703035 0 0) 0x7f3cd80b3730 con 0x7f3cd8150760 2024-01-15T17:57:47.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.284+0000 7f3ccfe00640 1 --2- 172.21.15.26:0/316737949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815da10 0x7f3cd8072230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00097a0 tx=0x7f3cc002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437d45e71cfcb599 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:47.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.284+0000 7f3cd5400640 1 -- 172.21.15.26:0/316737949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0031830 con 0x7f3cd815da10 2024-01-15T17:57:47.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.284+0000 7f3cd5400640 1 -- 172.21.15.26:0/316737949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc0031990 con 0x7f3cd815da10 2024-01-15T17:57:47.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.284+0000 7f3cd5400640 1 -- 172.21.15.26:0/316737949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0031d20 con 0x7f3cd815da10 2024-01-15T17:57:47.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.285+0000 7f3cdd200640 1 -- 172.21.15.26:0/316737949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815da10 msgr2=0x7f3cd8072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:47.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.285+0000 7f3cdd200640 1 --2- 172.21.15.26:0/316737949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815da10 0x7f3cd8072230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00097a0 tx=0x7f3cc002b8d0 comp rx=0 tx=0).stop 2024-01-15T17:57:47.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.285+0000 7f3cdd200640 1 -- 172.21.15.26:0/316737949 shutdown_connections 2024-01-15T17:57:47.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.285+0000 7f3cdd200640 1 --2- 172.21.15.26:0/316737949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd815da10 0x7f3cd8072230 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:47.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.285+0000 7f3cdd200640 1 -- 172.21.15.26:0/316737949 >> 172.21.15.26:0/316737949 conn(0x7f3cd806a800 msgr2=0x7f3cd806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:47.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.285+0000 7f3cdd200640 1 -- 172.21.15.26:0/316737949 shutdown_connections 2024-01-15T17:57:47.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.285+0000 7f3cdd200640 1 -- 172.21.15.26:0/316737949 wait complete. 2024-01-15T17:57:47.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.287+0000 7f3cdd200640 1 Processor -- start 2024-01-15T17:57:47.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.287+0000 7f3cdd200640 1 -- start start 2024-01-15T17:57:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.288+0000 7f3cdd200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd80b2bd0 0x7f3cd8156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.288+0000 7f3cdd200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cd8150760 0x7f3cd8158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.288+0000 7f3cdd200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cd815da10 0x7f3cd81550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.288+0000 7f3cdd200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd8154cc0 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.288+0000 7f3cdd200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd80cddf0 con 0x7f3cd815da10 2024-01-15T17:57:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.288+0000 7f3cdd200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd80ce0d0 con 0x7f3cd8150760 2024-01-15T17:57:47.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd80b2bd0 0x7f3cd8156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd80b2bd0 0x7f3cd8156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39240/0 (socket says 172.21.15.26:39240) 2024-01-15T17:57:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd5e00640 1 -- 172.21.15.26:0/2587913451 learned_addr learned my addr 172.21.15.26:0/2587913451 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3ccfe00640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cd8150760 0x7f3cd8158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd6800640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cd815da10 0x7f3cd81550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd5e00640 1 -- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cd8150760 msgr2=0x7f3cd8158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd5e00640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cd8150760 0x7f3cd8158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd5e00640 1 -- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cd815da10 msgr2=0x7f3cd81550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd5e00640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cd815da10 0x7f3cd81550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd5e00640 1 -- 172.21.15.26:0/2587913451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0009400 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.289+0000 7f3cd5e00640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd80b2bd0 0x7f3cd8156fd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c4d0 tx=0x7f3cc800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.290+0000 7f3cce000640 1 -- 172.21.15.26:0/2587913451 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80074e0 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.290+0000 7f3cce000640 1 -- 172.21.15.26:0/2587913451 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc8007660 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.290+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd8154600 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.290+0000 7f3cce000640 1 -- 172.21.15.26:0/2587913451 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8004a20 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.290+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd8157510 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.291+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd80675d0 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.292+0000 7f3cce000640 1 -- 172.21.15.26:0/2587913451 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f3cc8004b80 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.293+0000 7f3cce000640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3cb006b4a0 0x7f3cb006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.293+0000 7f3cce000640 1 -- 172.21.15.26:0/2587913451 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cc80894f0 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.294+0000 7f3ccfe00640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3cb006b4a0 0x7f3cb006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.295+0000 7f3ccfe00640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3cb006b4a0 0x7f3cb006d960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0036000 tx=0x7f3cc003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:47.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.298+0000 7f3cce000640 1 -- 172.21.15.26:0/2587913451 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3cc8053fb0 con 0x7f3cd80b2bd0 2024-01-15T17:57:47.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.529+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cd8071d10 con 0x7f3cb006b4a0 2024-01-15T17:57:47.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.544+0000 7f3cce000640 1 -- 172.21.15.26:0/2587913451 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f3cc003a420 con 0x7f3cb006b4a0 2024-01-15T17:57:47.547 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 66.66666666666666, 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "2/3", 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T17:57:47.548 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T17:57:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:47.549 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:47.549 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3cb006b4a0 msgr2=0x7f3cb006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3cb006b4a0 0x7f3cb006d960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0036000 tx=0x7f3cc003c040 comp rx=0 tx=0).stop 2024-01-15T17:57:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd80b2bd0 msgr2=0x7f3cd8156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:47.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd80b2bd0 0x7f3cd8156fd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800c4d0 tx=0x7f3cc800c990 comp rx=0 tx=0).stop 2024-01-15T17:57:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 shutdown_connections 2024-01-15T17:57:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3cd8150760 0x7f3cd8158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3cb006b4a0 0x7f3cb006d960 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3cd815da10 0x7f3cd81550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 --2- 172.21.15.26:0/2587913451 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3cd80b2bd0 0x7f3cd8156fd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 >> 172.21.15.26:0/2587913451 conn(0x7f3cd806a800 msgr2=0x7f3cd81599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 shutdown_connections 2024-01-15T17:57:47.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:47.546+0000 7f3cdd200640 1 -- 172.21.15.26:0/2587913451 wait complete. 2024-01-15T17:57:47.570 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T17:57:48.572 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T17:57:48.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.815+0000 7f4dfe200640 1 Processor -- start 2024-01-15T17:57:48.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.816+0000 7f4dfe200640 1 -- start start 2024-01-15T17:57:48.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.817+0000 7f4dfe200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 0x7f4df80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:48.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.817+0000 7f4dfe200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4df80d0b60 0x7f4df80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:48.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.817+0000 7f4dfe200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df8071de0 con 0x7f4df806a7c0 2024-01-15T17:57:48.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.817+0000 7f4dfe200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df80af320 con 0x7f4df80d0b60 2024-01-15T17:57:48.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.817+0000 7f4dfe200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df8072030 con 0x7f4df806a3e0 2024-01-15T17:57:48.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.818+0000 7f4df7400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 0x7f4df80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:48.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.818+0000 7f4df7400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 0x7f4df80ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:54658/0 (socket says 172.21.15.26:54658) 2024-01-15T17:57:48.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.818+0000 7f4df7400640 1 -- 172.21.15.26:0/225367176 learned_addr learned my addr 172.21.15.26:0/225367176 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:48.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.818+0000 7f4df6a00640 1 --2- 172.21.15.26:0/225367176 >> v2:172.21.15.124:3300/0 conn(0x7f4df80d0b60 0x7f4df80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:48.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.819+0000 7f4df6000640 1 -- 172.21.15.26:0/225367176 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950941910 0 0) 0x7f4df8071de0 con 0x7f4df806a7c0 2024-01-15T17:57:48.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.819+0000 7f4df6a00640 1 -- 172.21.15.26:0/225367176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 msgr2=0x7f4df80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:48.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.819+0000 7f4df6a00640 1 --2- 172.21.15.26:0/225367176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 0x7f4df80ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:48.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.819+0000 7f4df6a00640 1 -- 172.21.15.26:0/225367176 >> v1:172.21.15.124:6789/0 conn(0x7f4df806a7c0 legacy=0x7f4df806f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:48.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.819+0000 7f4df6a00640 1 -- 172.21.15.26:0/225367176 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df8072310 con 0x7f4df80d0b60 2024-01-15T17:57:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.821+0000 7f4df6a00640 1 --2- 172.21.15.26:0/225367176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4df80d0b60 0x7f4df80aede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4de00097a0 tx=0x7f4de002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c7bf91f6c55fd6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.821+0000 7f4df6000640 1 -- 172.21.15.26:0/225367176 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de002d040 con 0x7f4df80d0b60 2024-01-15T17:57:48.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.821+0000 7f4df6000640 1 -- 172.21.15.26:0/225367176 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de0031810 con 0x7f4df80d0b60 2024-01-15T17:57:48.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.821+0000 7f4df6000640 1 -- 172.21.15.26:0/225367176 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0031ba0 con 0x7f4df80d0b60 2024-01-15T17:57:48.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.823+0000 7f4dfe200640 1 -- 172.21.15.26:0/225367176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4df80d0b60 msgr2=0x7f4df80aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:48.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.823+0000 7f4dfe200640 1 --2- 172.21.15.26:0/225367176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4df80d0b60 0x7f4df80aede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4de00097a0 tx=0x7f4de002b8b0 comp rx=0 tx=0).stop 2024-01-15T17:57:48.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.823+0000 7f4dfe200640 1 -- 172.21.15.26:0/225367176 shutdown_connections 2024-01-15T17:57:48.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.823+0000 7f4dfe200640 1 --2- 172.21.15.26:0/225367176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4df80d0b60 0x7f4df80aede0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:48.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.823+0000 7f4dfe200640 1 --2- 172.21.15.26:0/225367176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 0x7f4df80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:48.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.823+0000 7f4dfe200640 1 -- 172.21.15.26:0/225367176 >> 172.21.15.26:0/225367176 conn(0x7f4df80cfef0 msgr2=0x7f4df80d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:48.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.823+0000 7f4dfe200640 1 -- 172.21.15.26:0/225367176 shutdown_connections 2024-01-15T17:57:48.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.823+0000 7f4dfe200640 1 -- 172.21.15.26:0/225367176 wait complete. 2024-01-15T17:57:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.825+0000 7f4dfe200640 1 Processor -- start 2024-01-15T17:57:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.826+0000 7f4dfe200640 1 -- start start 2024-01-15T17:57:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.827+0000 7f4dfe200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 0x7f4df81ec4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.827+0000 7f4dfe200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4df806a7c0 0x7f4df81eca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.827+0000 7f4dfe200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df80d0b60 0x7f4df8203050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.827+0000 7f4dfe200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80cf6e0 con 0x7f4df80d0b60 2024-01-15T17:57:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.827+0000 7f4dfe200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80cd630 con 0x7f4df806a7c0 2024-01-15T17:57:48.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.827+0000 7f4dfe200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df80cd910 con 0x7f4df806a3e0 2024-01-15T17:57:48.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.828+0000 7f4df7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df80d0b60 0x7f4df8203050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:48.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.828+0000 7f4df7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df80d0b60 0x7f4df8203050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39242/0 (socket says 172.21.15.26:39242) 2024-01-15T17:57:48.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.828+0000 7f4df7e00640 1 -- 172.21.15.26:0/2362302516 learned_addr learned my addr 172.21.15.26:0/2362302516 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.829+0000 7f4df7400640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 0x7f4df81ec4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.829+0000 7f4df7e00640 1 -- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 msgr2=0x7f4df81ec4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.829+0000 7f4df7e00640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 0x7f4df81ec4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.829+0000 7f4df7e00640 1 -- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4df806a7c0 msgr2=0x7f4df81eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.829+0000 7f4df6a00640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4df806a7c0 0x7f4df81eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.829+0000 7f4df7e00640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4df806a7c0 0x7f4df81eca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.829+0000 7f4df7e00640 1 -- 172.21.15.26:0/2362302516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de0009400 con 0x7f4df80d0b60 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.830+0000 7f4df7e00640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df80d0b60 0x7f4df8203050 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4df0001560 tx=0x7f4df000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.831+0000 7f4def400640 1 -- 172.21.15.26:0/2362302516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0003da0 con 0x7f4df80d0b60 2024-01-15T17:57:48.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.831+0000 7f4def400640 1 -- 172.21.15.26:0/2362302516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df0005070 con 0x7f4df80d0b60 2024-01-15T17:57:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.831+0000 7f4def400640 1 -- 172.21.15.26:0/2362302516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0005420 con 0x7f4df80d0b60 2024-01-15T17:57:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.831+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df80cdbf0 con 0x7f4df80d0b60 2024-01-15T17:57:48.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.831+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df80ce110 con 0x7f4df80d0b60 2024-01-15T17:57:48.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.834+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df80cdde0 con 0x7f4df80d0b60 2024-01-15T17:57:48.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.837+0000 7f4def400640 1 -- 172.21.15.26:0/2362302516 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f4df0005580 con 0x7f4df80d0b60 2024-01-15T17:57:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.840+0000 7f4def400640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4dd006b540 0x7f4dd006da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:48.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.841+0000 7f4def400640 1 -- 172.21.15.26:0/2362302516 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4df0089cd0 con 0x7f4df80d0b60 2024-01-15T17:57:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.844+0000 7f4def400640 1 -- 172.21.15.26:0/2362302516 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4df00548a0 con 0x7f4df80d0b60 2024-01-15T17:57:48.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.844+0000 7f4df7400640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4dd006b540 0x7f4dd006da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:48.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:48.846+0000 7f4df7400640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4dd006b540 0x7f4dd006da00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009aa0 tx=0x7f4de8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:49.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.183+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4df80cd7a0 con 0x7f4dd006b540 2024-01-15T17:57:49.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.215+0000 7f4def400640 1 -- 172.21.15.26:0/2362302516 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f4df80cd7a0 con 0x7f4dd006b540 2024-01-15T17:57:49.223 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 66.66666666666666, 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "2/3", 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T17:57:49.224 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T17:57:49.225 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:49.225 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:49.225 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4dd006b540 msgr2=0x7f4dd006da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4dd006b540 0x7f4dd006da00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009aa0 tx=0x7f4de8008040 comp rx=0 tx=0).stop 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df80d0b60 msgr2=0x7f4df8203050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df80d0b60 0x7f4df8203050 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4df0001560 tx=0x7f4df000cc70 comp rx=0 tx=0).stop 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 shutdown_connections 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4dd006b540 0x7f4dd006da00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4df80d0b60 0x7f4df8203050 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4df806a7c0 0x7f4df81eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 --2- 172.21.15.26:0/2362302516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4df806a3e0 0x7f4df81ec4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 >> 172.21.15.26:0/2362302516 conn(0x7f4df80cfef0 msgr2=0x7f4df80b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 shutdown_connections 2024-01-15T17:57:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:49.223+0000 7f4dfe200640 1 -- 172.21.15.26:0/2362302516 wait complete. 2024-01-15T17:57:49.259 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T17:57:50.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T17:57:50.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.566+0000 7f47cd800640 1 Processor -- start 2024-01-15T17:57:50.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.566+0000 7f47cd800640 1 -- start start 2024-01-15T17:57:50.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.568+0000 7f47cd800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813df80 0x7f47c815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:50.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.568+0000 7f47cd800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47c813e360 0x7f47c80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:50.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.568+0000 7f47cd800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f47c815f9c0 0x7f47c80cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:50.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.568+0000 7f47cd800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c806c470 con 0x7f47c813e360 2024-01-15T17:57:50.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.568+0000 7f47cd800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c806c750 con 0x7f47c815f9c0 2024-01-15T17:57:50.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.568+0000 7f47cd800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c806ca30 con 0x7f47c813df80 2024-01-15T17:57:50.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.569+0000 7f47c7400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f47c815f9c0 0x7f47c80cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:50.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.569+0000 7f47c7400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f47c815f9c0 0x7f47c80cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:55060/0 (socket says 172.21.15.26:55060) 2024-01-15T17:57:50.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.569+0000 7f47c7400640 1 -- 172.21.15.26:0/2463920805 learned_addr learned my addr 172.21.15.26:0/2463920805 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:50.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.569+0000 7f47c6a00640 1 --2- 172.21.15.26:0/2463920805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813df80 0x7f47c815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:50.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.569+0000 7f47bfe00640 1 --2- 172.21.15.26:0/2463920805 >> v2:172.21.15.26:3300/0 conn(0x7f47c813e360 0x7f47c80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:50.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.570+0000 7f47c7400640 1 -- 172.21.15.26:0/2463920805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813df80 msgr2=0x7f47c815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:50.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.570+0000 7f47c7400640 1 --2- 172.21.15.26:0/2463920805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813df80 0x7f47c815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:50.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.570+0000 7f47c7400640 1 -- 172.21.15.26:0/2463920805 >> v2:172.21.15.26:3300/0 conn(0x7f47c813e360 msgr2=0x7f47c80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:50.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.570+0000 7f47c7400640 1 --2- 172.21.15.26:0/2463920805 >> v2:172.21.15.26:3300/0 conn(0x7f47c813e360 0x7f47c80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:50.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.570+0000 7f47c7400640 1 -- 172.21.15.26:0/2463920805 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c806cd10 con 0x7f47c815f9c0 2024-01-15T17:57:50.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.573+0000 7f47c7400640 1 --2- 172.21.15.26:0/2463920805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 0x7f47c80cf7d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47c806e6d0 tx=0x7f47c0030310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee80282d059e6ac9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:50.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.573+0000 7f47c6000640 1 -- 172.21.15.26:0/2463920805 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0030e90 con 0x7f47c815f9c0 2024-01-15T17:57:50.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.573+0000 7f47c6000640 1 -- 172.21.15.26:0/2463920805 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c00383f0 con 0x7f47c815f9c0 2024-01-15T17:57:50.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.574+0000 7f47c6000640 1 -- 172.21.15.26:0/2463920805 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c00387e0 con 0x7f47c815f9c0 2024-01-15T17:57:50.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.575+0000 7f47cd800640 1 -- 172.21.15.26:0/2463920805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 msgr2=0x7f47c80cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:50.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.575+0000 7f47cd800640 1 --2- 172.21.15.26:0/2463920805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 0x7f47c80cf7d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47c806e6d0 tx=0x7f47c0030310 comp rx=0 tx=0).stop 2024-01-15T17:57:50.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.575+0000 7f47cd800640 1 -- 172.21.15.26:0/2463920805 shutdown_connections 2024-01-15T17:57:50.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.575+0000 7f47cd800640 1 --2- 172.21.15.26:0/2463920805 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 0x7f47c80cf7d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.575+0000 7f47cd800640 1 --2- 172.21.15.26:0/2463920805 >> v2:172.21.15.26:3300/0 conn(0x7f47c813e360 0x7f47c80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.575+0000 7f47cd800640 1 --2- 172.21.15.26:0/2463920805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813df80 0x7f47c815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.575+0000 7f47cd800640 1 -- 172.21.15.26:0/2463920805 >> 172.21.15.26:0/2463920805 conn(0x7f47c806a800 msgr2=0x7f47c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.576+0000 7f47cd800640 1 -- 172.21.15.26:0/2463920805 shutdown_connections 2024-01-15T17:57:50.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.576+0000 7f47cd800640 1 -- 172.21.15.26:0/2463920805 wait complete. 2024-01-15T17:57:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.580+0000 7f47cd800640 1 Processor -- start 2024-01-15T17:57:50.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.580+0000 7f47cd800640 1 -- start start 2024-01-15T17:57:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.583+0000 7f47cd800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c813df80 0x7f47c8153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:50.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.584+0000 7f47cd800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813e360 0x7f47c8070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.584+0000 7f47cd800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 0x7f47c80b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.584+0000 7f47cd800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8071c00 con 0x7f47c813df80 2024-01-15T17:57:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.584+0000 7f47cd800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8139810 con 0x7f47c815f9c0 2024-01-15T17:57:50.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.584+0000 7f47cd800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8071640 con 0x7f47c813e360 2024-01-15T17:57:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.584+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813e360 0x7f47c8070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:50.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.584+0000 7f47bfe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813e360 0x7f47c8070cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:54688/0 (socket says 172.21.15.26:54688) 2024-01-15T17:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.584+0000 7f47bfe00640 1 -- 172.21.15.26:0/459260845 learned_addr learned my addr 172.21.15.26:0/459260845 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.585+0000 7f47c6a00640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c813df80 0x7f47c8153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.585+0000 7f47c7400640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 0x7f47c80b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.585+0000 7f47c7400640 1 -- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813e360 msgr2=0x7f47c8070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:50.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.586+0000 7f47c7400640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813e360 0x7f47c8070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.586+0000 7f47c7400640 1 -- 172.21.15.26:0/459260845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c813df80 msgr2=0x7f47c8153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.586+0000 7f47c7400640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c813df80 0x7f47c8153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.586+0000 7f47c7400640 1 -- 172.21.15.26:0/459260845 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c0009b30 con 0x7f47c815f9c0 2024-01-15T17:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.586+0000 7f47c6a00640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c813df80 0x7f47c8153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.586+0000 7f47bfe00640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813e360 0x7f47c8070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:57:50.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.586+0000 7f47c7400640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 0x7f47c80b2490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f47c0036000 tx=0x7f47c0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.587+0000 7f47bea00640 1 -- 172.21.15.26:0/459260845 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0004600 con 0x7f47c815f9c0 2024-01-15T17:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.587+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c8071920 con 0x7f47c815f9c0 2024-01-15T17:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.587+0000 7f47bea00640 1 -- 172.21.15.26:0/459260845 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c0004760 con 0x7f47c815f9c0 2024-01-15T17:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.587+0000 7f47bea00640 1 -- 172.21.15.26:0/459260845 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c0004600 con 0x7f47c815f9c0 2024-01-15T17:57:50.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.587+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c8071210 con 0x7f47c815f9c0 2024-01-15T17:57:50.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.589+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c8139980 con 0x7f47c815f9c0 2024-01-15T17:57:50.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.589+0000 7f47bea00640 1 -- 172.21.15.26:0/459260845 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f47c0038560 con 0x7f47c815f9c0 2024-01-15T17:57:50.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.590+0000 7f47bea00640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47a006b4d0 0x7f47a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:50.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.591+0000 7f47bea00640 1 -- 172.21.15.26:0/459260845 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47c00b0ad0 con 0x7f47c815f9c0 2024-01-15T17:57:50.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.591+0000 7f47c6a00640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47a006b4d0 0x7f47a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:50.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.592+0000 7f47c6a00640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47a006b4d0 0x7f47a006d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47b8007d90 tx=0x7f47b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:50.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.596+0000 7f47bea00640 1 -- 172.21.15.26:0/459260845 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47c007b7d0 con 0x7f47c815f9c0 2024-01-15T17:57:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:50.832+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f47c81408d0 con 0x7f47a006b4d0 2024-01-15T17:57:51.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.694+0000 7f47bea00640 1 -- 172.21.15.26:0/459260845 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f47b80079e0 con 0x7f47a006b4d0 2024-01-15T17:57:51.698 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:57:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:57:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T17:57:51.699 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:51.699 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:57:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47a006b4d0 msgr2=0x7f47a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47a006b4d0 0x7f47a006d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47b8007d90 tx=0x7f47b8008040 comp rx=0 tx=0).stop 2024-01-15T17:57:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 msgr2=0x7f47c80b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 0x7f47c80b2490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f47c0036000 tx=0x7f47c0004a40 comp rx=0 tx=0).stop 2024-01-15T17:57:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 shutdown_connections 2024-01-15T17:57:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47c813e360 0x7f47c8070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47a006b4d0 0x7f47a006d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47c815f9c0 0x7f47c80b2490 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 --2- 172.21.15.26:0/459260845 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47c813df80 0x7f47c8153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 >> 172.21.15.26:0/459260845 conn(0x7f47c806a800 msgr2=0x7f47c80b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:51.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.697+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 shutdown_connections 2024-01-15T17:57:51.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.698+0000 7f47cd800640 1 -- 172.21.15.26:0/459260845 wait complete. 2024-01-15T17:57:51.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T17:57:51.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.943+0000 7fa395e00640 1 Processor -- start 2024-01-15T17:57:51.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.943+0000 7fa395e00640 1 -- start start 2024-01-15T17:57:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.945+0000 7fa395e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa3900cf9a0 0x7fa39015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.945+0000 7fa395e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3900b1ee0 con 0x7fa39015e1e0 2024-01-15T17:57:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.945+0000 7fa395e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3900b2780 con 0x7fa390143ae0 2024-01-15T17:57:51.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.945+0000 7fa395e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39015e990 con 0x7fa3900cf9a0 2024-01-15T17:57:51.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.945+0000 7fa38f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa39015e1e0 0x7fa3900702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36776/0 (socket says 172.21.15.26:36776) 2024-01-15T17:57:51.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.945+0000 7fa38f400640 1 -- 172.21.15.26:0/3969684725 learned_addr learned my addr 172.21.15.26:0/3969684725 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:57:51.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.945+0000 7fa38e000640 1 --2- 172.21.15.26:0/3969684725 >> v2:172.21.15.124:3300/0 conn(0x7fa3900cf9a0 0x7fa39015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:51.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367747946 0 0) 0x7fa3900b1ee0 con 0x7fa39015e1e0 2024-01-15T17:57:51.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa370003520 con 0x7fa39015e1e0 2024-01-15T17:57:51.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3777138087 0 0) 0x7fa370003520 con 0x7fa39015e1e0 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 >> v2:172.21.15.124:3300/0 conn(0x7fa3900cf9a0 msgr2=0x7fa39015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 --2- 172.21.15.26:0/3969684725 >> v2:172.21.15.124:3300/0 conn(0x7fa3900cf9a0 0x7fa39015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 >> v1:172.21.15.124:6789/0 conn(0x7fa390143ae0 legacy=0x7fa3900cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900b2130 con 0x7fa39015e1e0 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa388001670 con 0x7fa39015e1e0 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 >> v1:172.21.15.26:6789/0 conn(0x7fa39015e1e0 legacy=0x7fa3900702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 --2- 172.21.15.26:0/3969684725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3700039c0 0x7fa370003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 --2- 172.21.15.26:0/3969684725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3700047c0 0x7fa370004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 --2- 172.21.15.26:0/3969684725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa370005180 0x7fa37000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39015e990 con 0x7fa370005180 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37000c080 con 0x7fa3700039c0 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.946+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37000c240 con 0x7fa3700047c0 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.947+0000 7fa38f400640 1 --2- 172.21.15.26:0/3969684725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa370005180 0x7fa37000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:51.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.947+0000 7fa38f400640 1 -- 172.21.15.26:0/3969684725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3700047c0 msgr2=0x7fa370004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.947+0000 7fa38f400640 1 --2- 172.21.15.26:0/3969684725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa3700047c0 0x7fa370004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.947+0000 7fa38f400640 1 -- 172.21.15.26:0/3969684725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3700039c0 msgr2=0x7fa370003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.947+0000 7fa38f400640 1 --2- 172.21.15.26:0/3969684725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa3700039c0 0x7fa370003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.947+0000 7fa38f400640 1 -- 172.21.15.26:0/3969684725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa370003520 con 0x7fa370005180 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.948+0000 7fa38f400640 1 -- 172.21.15.26:0/3969684725 reap_dead start 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.948+0000 7fa38f400640 1 --2- 172.21.15.26:0/3969684725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa370005180 0x7fa37000b980 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa38800c0e0 tx=0x7fa3700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dcaa01d4c8e106 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.948+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388005ad0 con 0x7fa370005180 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.948+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa388005c70 con 0x7fa370005180 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.948+0000 7fa38d600640 1 -- 172.21.15.26:0/3969684725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38802c810 con 0x7fa370005180 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.949+0000 7fa395e00640 1 -- 172.21.15.26:0/3969684725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa370005180 msgr2=0x7fa37000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.949+0000 7fa395e00640 1 --2- 172.21.15.26:0/3969684725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa370005180 0x7fa37000b980 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa38800c0e0 tx=0x7fa3700057f0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.949+0000 7fa395e00640 1 -- 172.21.15.26:0/3969684725 shutdown_connections 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.949+0000 7fa395e00640 1 --2- 172.21.15.26:0/3969684725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa370005180 0x7fa37000b980 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.949+0000 7fa395e00640 1 -- 172.21.15.26:0/3969684725 >> 172.21.15.26:0/3969684725 conn(0x7fa39006a800 msgr2=0x7fa39006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.949+0000 7fa395e00640 1 -- 172.21.15.26:0/3969684725 shutdown_connections 2024-01-15T17:57:51.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.949+0000 7fa395e00640 1 -- 172.21.15.26:0/3969684725 wait complete. 2024-01-15T17:57:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.951+0000 7fa395e00640 1 Processor -- start 2024-01-15T17:57:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.951+0000 7fa395e00640 1 -- start start 2024-01-15T17:57:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.952+0000 7fa395e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa370005180 0x7fa390159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.952+0000 7fa395e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa39013ffd0 0x7fa390140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:51.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.952+0000 7fa395e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39015b550 0x7fa39015b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.952+0000 7fa395e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900ae290 con 0x7fa39015b550 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.952+0000 7fa395e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900ad410 con 0x7fa370005180 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.952+0000 7fa395e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa390140990 con 0x7fa39013ffd0 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39015b550 0x7fa39015b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39015b550 0x7fa39015b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48442/0 (socket says 172.21.15.26:48442) 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38f400640 1 -- 172.21.15.26:0/1291677693 learned_addr learned my addr 172.21.15.26:0/1291677693 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38e000640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa39013ffd0 0x7fa390140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38ea00640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa370005180 0x7fa390159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38f400640 1 -- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa39013ffd0 msgr2=0x7fa390140450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38f400640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa39013ffd0 0x7fa390140450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38f400640 1 -- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa370005180 msgr2=0x7fa390159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38f400640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa370005180 0x7fa390159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38f400640 1 -- 172.21.15.26:0/1291677693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900ad610 con 0x7fa39015b550 2024-01-15T17:57:51.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.953+0000 7fa38f400640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39015b550 0x7fa39015b9d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa38800c0e0 tx=0x7fa388001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:51.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.954+0000 7fa387400640 1 -- 172.21.15.26:0/1291677693 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38802cc20 con 0x7fa39015b550 2024-01-15T17:57:51.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.954+0000 7fa387400640 1 -- 172.21.15.26:0/1291677693 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38802cda0 con 0x7fa39015b550 2024-01-15T17:57:51.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.954+0000 7fa387400640 1 -- 172.21.15.26:0/1291677693 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3880072b0 con 0x7fa39015b550 2024-01-15T17:57:51.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.954+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3900ad8f0 con 0x7fa39015b550 2024-01-15T17:57:51.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.954+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3900adeb0 con 0x7fa39015b550 2024-01-15T17:57:51.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.955+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3900ae040 con 0x7fa39015b550 2024-01-15T17:57:51.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.956+0000 7fa387400640 1 -- 172.21.15.26:0/1291677693 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fa38803d070 con 0x7fa39015b550 2024-01-15T17:57:51.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.957+0000 7fa387400640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa36806b490 0x7fa36806d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:51.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.957+0000 7fa387400640 1 -- 172.21.15.26:0/1291677693 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3880aaaa0 con 0x7fa39015b550 2024-01-15T17:57:51.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.958+0000 7fa38ea00640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa36806b490 0x7fa36806d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:51.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.959+0000 7fa38ea00640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa36806b490 0x7fa36806d950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa380001670 tx=0x7fa380009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:51.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:51.961+0000 7fa387400640 1 -- 172.21.15.26:0/1291677693 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3880755f0 con 0x7fa39015b550 2024-01-15T17:57:52.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.182+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa39015e670 con 0x7fa36806b490 2024-01-15T17:57:52.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.186+0000 7fa387400640 1 -- 172.21.15.26:0/1291677693 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa380001bf0 con 0x7fa36806b490 2024-01-15T17:57:52.190 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T17:57:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:57:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T17:57:52.190 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:52.190 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T17:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa36806b490 msgr2=0x7fa36806d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa36806b490 0x7fa36806d950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa380001670 tx=0x7fa380009040 comp rx=0 tx=0).stop 2024-01-15T17:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39015b550 msgr2=0x7fa39015b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39015b550 0x7fa39015b9d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa38800c0e0 tx=0x7fa388001930 comp rx=0 tx=0).stop 2024-01-15T17:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 shutdown_connections 2024-01-15T17:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa39013ffd0 0x7fa390140450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa36806b490 0x7fa36806d950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa39015b550 0x7fa39015b9d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 --2- 172.21.15.26:0/1291677693 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa370005180 0x7fa390159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 >> 172.21.15.26:0/1291677693 conn(0x7fa39006a800 msgr2=0x7fa3900702c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 shutdown_connections 2024-01-15T17:57:52.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.188+0000 7fa395e00640 1 -- 172.21.15.26:0/1291677693 wait complete. 2024-01-15T17:57:52.209 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T17:57:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.426+0000 7f2227a00640 1 Processor -- start 2024-01-15T17:57:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.426+0000 7f2227a00640 1 -- start start 2024-01-15T17:57:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.427+0000 7f2227a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 0x7f2220150760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.427+0000 7f2227a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22200cf290 0x7f222015d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.428+0000 7f2227a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2220071190 con 0x7f22200cf6d0 2024-01-15T17:57:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.428+0000 7f2227a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2220072610 con 0x7f22200cf290 2024-01-15T17:57:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.428+0000 7f2227a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2220071470 con 0x7f2220072230 2024-01-15T17:57:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.428+0000 7f2217e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22200cf290 0x7f222015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.428+0000 7f2225a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f22200cf6d0 0x7f22200b2bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36780/0 (socket says 172.21.15.26:36780) 2024-01-15T17:57:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.428+0000 7f2225a00640 1 -- 172.21.15.26:0/4268095504 learned_addr learned my addr 172.21.15.26:0/4268095504 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:57:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.428+0000 7f2225000640 1 --2- 172.21.15.26:0/4268095504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 0x7f2220150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.429+0000 7f2217e00640 1 -- 172.21.15.26:0/4268095504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 msgr2=0x7f2220150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.429+0000 7f2217e00640 1 --2- 172.21.15.26:0/4268095504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 0x7f2220150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.429+0000 7f2217e00640 1 -- 172.21.15.26:0/4268095504 >> v1:172.21.15.26:6789/0 conn(0x7f22200cf6d0 legacy=0x7f22200b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.429+0000 7f2217e00640 1 -- 172.21.15.26:0/4268095504 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2220071750 con 0x7f22200cf290 2024-01-15T17:57:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.429+0000 7f221fe00640 1 -- 172.21.15.26:0/4268095504 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019712970 0 0) 0x7f2220071190 con 0x7f22200cf6d0 2024-01-15T17:57:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.430+0000 7f2217e00640 1 --2- 172.21.15.26:0/4268095504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf290 0x7f222015d300 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f220800ba40 tx=0x7f220802e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21bb76e2ae313ae server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.430+0000 7f221fe00640 1 -- 172.21.15.26:0/4268095504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220802ee30 con 0x7f22200cf290 2024-01-15T17:57:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.430+0000 7f221fe00640 1 -- 172.21.15.26:0/4268095504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2208036440 con 0x7f22200cf290 2024-01-15T17:57:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.430+0000 7f221fe00640 1 -- 172.21.15.26:0/4268095504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22080367d0 con 0x7f22200cf290 2024-01-15T17:57:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.431+0000 7f2227a00640 1 -- 172.21.15.26:0/4268095504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf290 msgr2=0x7f222015d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.431+0000 7f2227a00640 1 --2- 172.21.15.26:0/4268095504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf290 0x7f222015d300 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f220800ba40 tx=0x7f220802e080 comp rx=0 tx=0).stop 2024-01-15T17:57:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.431+0000 7f2227a00640 1 -- 172.21.15.26:0/4268095504 shutdown_connections 2024-01-15T17:57:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.431+0000 7f2227a00640 1 --2- 172.21.15.26:0/4268095504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf290 0x7f222015d300 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.431+0000 7f2227a00640 1 --2- 172.21.15.26:0/4268095504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 0x7f2220150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.431+0000 7f2227a00640 1 -- 172.21.15.26:0/4268095504 >> 172.21.15.26:0/4268095504 conn(0x7f222006a800 msgr2=0x7f222006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.431+0000 7f2227a00640 1 -- 172.21.15.26:0/4268095504 shutdown_connections 2024-01-15T17:57:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.431+0000 7f2227a00640 1 -- 172.21.15.26:0/4268095504 wait complete. 2024-01-15T17:57:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.432+0000 7f2227a00640 1 Processor -- start 2024-01-15T17:57:52.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.433+0000 7f2227a00640 1 -- start start 2024-01-15T17:57:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.433+0000 7f2227a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 0x7f2220146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2227a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22200cf290 0x7f22200ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2227a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf6d0 0x7f22201593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2227a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22200ace10 con 0x7f22200cf6d0 2024-01-15T17:57:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2227a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22200ad040 con 0x7f22200cf290 2024-01-15T17:57:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2227a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22200ad320 con 0x7f2220072230 2024-01-15T17:57:52.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2225a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf6d0 0x7f22201593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2225a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf6d0 0x7f22201593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48458/0 (socket says 172.21.15.26:48458) 2024-01-15T17:57:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2225a00640 1 -- 172.21.15.26:0/3901849872 learned_addr learned my addr 172.21.15.26:0/3901849872 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2225000640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 0x7f2220146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2217e00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22200cf290 0x7f22200ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2225a00640 1 -- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 msgr2=0x7f2220146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2225a00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 0x7f2220146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2225a00640 1 -- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22200cf290 msgr2=0x7f22200ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.434+0000 7f2225a00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22200cf290 0x7f22200ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.435+0000 7f2225a00640 1 -- 172.21.15.26:0/3901849872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220800b6a0 con 0x7f22200cf6d0 2024-01-15T17:57:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.435+0000 7f2225000640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 0x7f2220146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:57:52.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.435+0000 7f2225a00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf6d0 0x7f22201593a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f221800c700 tx=0x7f221800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.435+0000 7f221d600640 1 -- 172.21.15.26:0/3901849872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221800ce10 con 0x7f22200cf6d0 2024-01-15T17:57:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.435+0000 7f221d600640 1 -- 172.21.15.26:0/3901849872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221800a640 con 0x7f22200cf6d0 2024-01-15T17:57:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.435+0000 7f221d600640 1 -- 172.21.15.26:0/3901849872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221800a9f0 con 0x7f22200cf6d0 2024-01-15T17:57:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.435+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22200ad600 con 0x7f22200cf6d0 2024-01-15T17:57:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.435+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22200adbc0 con 0x7f22200cf6d0 2024-01-15T17:57:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.437+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22200addb0 con 0x7f22200cf6d0 2024-01-15T17:57:52.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.437+0000 7f221d600640 1 -- 172.21.15.26:0/3901849872 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f221800ab50 con 0x7f22200cf6d0 2024-01-15T17:57:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.438+0000 7f221d600640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21f806b470 0x7f21f806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.438+0000 7f221d600640 1 -- 172.21.15.26:0/3901849872 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2218089300 con 0x7f22200cf6d0 2024-01-15T17:57:52.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.439+0000 7f2225000640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21f806b470 0x7f21f806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.440+0000 7f2225000640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21f806b470 0x7f21f806d930 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2210006fd0 tx=0x7f2210008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:52.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.443+0000 7f221d600640 1 -- 172.21.15.26:0/3901849872 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2218053ee0 con 0x7f22200cf6d0 2024-01-15T17:57:52.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.665+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2220071d10 con 0x7f21f806b470 2024-01-15T17:57:52.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.677+0000 7f221d600640 1 -- 172.21.15.26:0/3901849872 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+79 (secure 0 0 0) 0x7f221000f920 con 0x7f21f806b470 2024-01-15T17:57:52.681 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:57:52.681 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T17:57:52.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:57:52.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sv1" 2024-01-15T17:57:52.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:57:52.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:57:52.681 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1clone1" 2024-01-15T17:57:52.682 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:52.682 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T17:57:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.679+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21f806b470 msgr2=0x7f21f806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.679+0000 7f2227a00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21f806b470 0x7f21f806d930 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2210006fd0 tx=0x7f2210008040 comp rx=0 tx=0).stop 2024-01-15T17:57:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.679+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf6d0 msgr2=0x7f22201593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.679+0000 7f2227a00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf6d0 0x7f22201593a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f221800c700 tx=0x7f221800cbc0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.680+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 shutdown_connections 2024-01-15T17:57:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.680+0000 7f2227a00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21f806b470 0x7f21f806d930 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.680+0000 7f2227a00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22200cf6d0 0x7f22201593a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.680+0000 7f2227a00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22200cf290 0x7f22200ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.680+0000 7f2227a00640 1 --2- 172.21.15.26:0/3901849872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2220072230 0x7f2220146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.680+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 >> 172.21.15.26:0/3901849872 conn(0x7f222006a800 msgr2=0x7f222006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.680+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 shutdown_connections 2024-01-15T17:57:52.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.680+0000 7f2227a00640 1 -- 172.21.15.26:0/3901849872 wait complete. 2024-01-15T17:57:52.704 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T17:57:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.920+0000 7fc22a200640 1 Processor -- start 2024-01-15T17:57:52.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.921+0000 7fc22a200640 1 -- start start 2024-01-15T17:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.922+0000 7fc22a200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc22413df80 0x7fc22415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.922+0000 7fc22a200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc22413e360 0x7fc2240cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.922+0000 7fc22a200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22415f9c0 0x7fc2240cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.922+0000 7fc22a200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22406c470 con 0x7fc22413df80 2024-01-15T17:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.922+0000 7fc22a200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22406c750 con 0x7fc22413e360 2024-01-15T17:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.922+0000 7fc22a200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22406ca30 con 0x7fc22415f9c0 2024-01-15T17:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.923+0000 7fc223400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc22413df80 0x7fc22415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.923+0000 7fc223400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc22413df80 0x7fc22415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48460/0 (socket says 172.21.15.26:48460) 2024-01-15T17:57:52.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.923+0000 7fc223400640 1 -- 172.21.15.26:0/1584784906 learned_addr learned my addr 172.21.15.26:0/1584784906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.923+0000 7fc223400640 1 -- 172.21.15.26:0/1584784906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22415f9c0 msgr2=0x7fc2240cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.923+0000 7fc223400640 1 --2- 172.21.15.26:0/1584784906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22415f9c0 0x7fc2240cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.923+0000 7fc223400640 1 -- 172.21.15.26:0/1584784906 >> v2:172.21.15.124:3300/0 conn(0x7fc22413e360 msgr2=0x7fc2240cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.923+0000 7fc223400640 1 --2- 172.21.15.26:0/1584784906 >> v2:172.21.15.124:3300/0 conn(0x7fc22413e360 0x7fc2240cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.923+0000 7fc223400640 1 -- 172.21.15.26:0/1584784906 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22406cd10 con 0x7fc22413df80 2024-01-15T17:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.924+0000 7fc223400640 1 --2- 172.21.15.26:0/1584784906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22413df80 0x7fc22415bee0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc22406ded0 tx=0x7fc21402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54001225514fcc1a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.925+0000 7fc222a00640 1 -- 172.21.15.26:0/1584784906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214031870 con 0x7fc22413df80 2024-01-15T17:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.925+0000 7fc222a00640 1 -- 172.21.15.26:0/1584784906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2140319d0 con 0x7fc22413df80 2024-01-15T17:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.925+0000 7fc222a00640 1 -- 172.21.15.26:0/1584784906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214031d60 con 0x7fc22413df80 2024-01-15T17:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.925+0000 7fc22a200640 1 -- 172.21.15.26:0/1584784906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22413df80 msgr2=0x7fc22415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.925+0000 7fc22a200640 1 --2- 172.21.15.26:0/1584784906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22413df80 0x7fc22415bee0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc22406ded0 tx=0x7fc21402b910 comp rx=0 tx=0).stop 2024-01-15T17:57:52.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.926+0000 7fc22a200640 1 -- 172.21.15.26:0/1584784906 shutdown_connections 2024-01-15T17:57:52.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.926+0000 7fc22a200640 1 --2- 172.21.15.26:0/1584784906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22415f9c0 0x7fc2240cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.926+0000 7fc22a200640 1 --2- 172.21.15.26:0/1584784906 >> v2:172.21.15.124:3300/0 conn(0x7fc22413e360 0x7fc2240cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.926+0000 7fc22a200640 1 --2- 172.21.15.26:0/1584784906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22413df80 0x7fc22415bee0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.926+0000 7fc22a200640 1 -- 172.21.15.26:0/1584784906 >> 172.21.15.26:0/1584784906 conn(0x7fc22406a800 msgr2=0x7fc22406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.926+0000 7fc22a200640 1 -- 172.21.15.26:0/1584784906 shutdown_connections 2024-01-15T17:57:52.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.926+0000 7fc22a200640 1 -- 172.21.15.26:0/1584784906 wait complete. 2024-01-15T17:57:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.928+0000 7fc22a200640 1 Processor -- start 2024-01-15T17:57:52.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.928+0000 7fc22a200640 1 -- start start 2024-01-15T17:57:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc22a200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22413df80 0x7fc22413c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc22a200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22413e360 0x7fc2240a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc22a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22415f9c0 0x7fc224071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc22a200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240aa0d0 con 0x7fc22415f9c0 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc22a200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240aa320 con 0x7fc22413e360 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc22a200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2240aa600 con 0x7fc22413df80 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc223e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22415f9c0 0x7fc224071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc223e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22415f9c0 0x7fc224071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48462/0 (socket says 172.21.15.26:48462) 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc223e00640 1 -- 172.21.15.26:0/373395046 learned_addr learned my addr 172.21.15.26:0/373395046 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc223400640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22413df80 0x7fc22413c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.929+0000 7fc21be00640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22413e360 0x7fc2240a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc223e00640 1 -- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22413df80 msgr2=0x7fc22413c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc223e00640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22413df80 0x7fc22413c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc223e00640 1 -- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22413e360 msgr2=0x7fc2240a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:52.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc223e00640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22413e360 0x7fc2240a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:52.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc223e00640 1 -- 172.21.15.26:0/373395046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2140094d0 con 0x7fc22415f9c0 2024-01-15T17:57:52.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc223e00640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22415f9c0 0x7fc224071570 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b9e0 tx=0x7fc21c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:52.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc21b400640 1 -- 172.21.15.26:0/373395046 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00cb20 con 0x7fc22415f9c0 2024-01-15T17:57:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc21b400640 1 -- 172.21.15.26:0/373395046 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21c00cc80 con 0x7fc22415f9c0 2024-01-15T17:57:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc21b400640 1 -- 172.21.15.26:0/373395046 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c014680 con 0x7fc22415f9c0 2024-01-15T17:57:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2240aa8e0 con 0x7fc22415f9c0 2024-01-15T17:57:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.930+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2240aaea0 con 0x7fc22415f9c0 2024-01-15T17:57:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.932+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2240ab0c0 con 0x7fc22415f9c0 2024-01-15T17:57:52.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.932+0000 7fc21b400640 1 -- 172.21.15.26:0/373395046 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fc21c01d3b0 con 0x7fc22415f9c0 2024-01-15T17:57:52.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.933+0000 7fc21b400640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1fc06b470 0x7fc1fc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.933+0000 7fc21b400640 1 -- 172.21.15.26:0/373395046 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc21c089ea0 con 0x7fc22415f9c0 2024-01-15T17:57:52.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.934+0000 7fc223400640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1fc06b470 0x7fc1fc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:52.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.935+0000 7fc223400640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1fc06b470 0x7fc1fc06d930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc214036000 tx=0x7fc21403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:52.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:52.938+0000 7fc21b400640 1 -- 172.21.15.26:0/373395046 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc21c0549f0 con 0x7fc22415f9c0 2024-01-15T17:57:53.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.166+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc2241408d0 con 0x7fc1fc06b470 2024-01-15T17:57:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.199+0000 7fc21b400640 1 -- 172.21.15.26:0/373395046 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc214031cc0 con 0x7fc1fc06b470 2024-01-15T17:57:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:57:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T17:57:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:57:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1" 2024-01-15T17:57:53.203 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:57:53.203 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1fc06b470 msgr2=0x7fc1fc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1fc06b470 0x7fc1fc06d930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc214036000 tx=0x7fc21403c040 comp rx=0 tx=0).stop 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22415f9c0 msgr2=0x7fc224071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22415f9c0 0x7fc224071570 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b9e0 tx=0x7fc21c00bea0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 shutdown_connections 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1fc06b470 0x7fc1fc06d930 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc22415f9c0 0x7fc224071570 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc22413e360 0x7fc2240a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 --2- 172.21.15.26:0/373395046 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc22413df80 0x7fc22413c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.202+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 >> 172.21.15.26:0/373395046 conn(0x7fc22406a800 msgr2=0x7fc22406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.203+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 shutdown_connections 2024-01-15T17:57:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.203+0000 7fc22a200640 1 -- 172.21.15.26:0/373395046 wait complete. 2024-01-15T17:57:53.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T17:57:53.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.455+0000 7f8ac9c00640 1 Processor -- start 2024-01-15T17:57:53.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.455+0000 7f8ac9c00640 1 -- start start 2024-01-15T17:57:53.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.457+0000 7f8ac9c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8ac413df80 0x7f8ac415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:53.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.457+0000 7f8ac9c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ac40cf9a0 0x7f8ac40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.457+0000 7f8ac9c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac406c470 con 0x7f8ac413e360 2024-01-15T17:57:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.457+0000 7f8ac9c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac406c750 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.457+0000 7f8ac9c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac406ca30 con 0x7f8ac413df80 2024-01-15T17:57:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ac40cf9a0 0x7f8ac40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ac40cf9a0 0x7f8ac40cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48476/0 (socket says 172.21.15.26:48476) 2024-01-15T17:57:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac3400640 1 -- 172.21.15.26:0/3211565444 learned_addr learned my addr 172.21.15.26:0/3211565444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac2000640 1 --1- 172.21.15.26:0/3211565444 >> v1:172.21.15.26:6789/0 conn(0x7f8ac413e360 0x7f8ac40cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36792/0 (socket says 172.21.15.26:36792) 2024-01-15T17:57:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac2a00640 1 --2- 172.21.15.26:0/3211565444 >> v2:172.21.15.124:3300/0 conn(0x7f8ac413df80 0x7f8ac415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac3400640 1 -- 172.21.15.26:0/3211565444 >> v2:172.21.15.124:3300/0 conn(0x7f8ac413df80 msgr2=0x7f8ac415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac3400640 1 --2- 172.21.15.26:0/3211565444 >> v2:172.21.15.124:3300/0 conn(0x7f8ac413df80 0x7f8ac415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac3400640 1 -- 172.21.15.26:0/3211565444 >> v1:172.21.15.26:6789/0 conn(0x7f8ac413e360 legacy=0x7f8ac40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac3400640 1 -- 172.21.15.26:0/3211565444 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac406cd10 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.458+0000 7f8ac1600640 1 -- 172.21.15.26:0/3211565444 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523841882 0 0) 0x7f8ac406c470 con 0x7f8ac413e360 2024-01-15T17:57:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.459+0000 7f8ac3400640 1 --2- 172.21.15.26:0/3211565444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 0x7f8ac40cd840 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4072770 tx=0x7f8abc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d47a9dc62a4b7fd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac1600640 1 -- 172.21.15.26:0/3211565444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc030860 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac1600640 1 -- 172.21.15.26:0/3211565444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8abc0309c0 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac1600640 1 -- 172.21.15.26:0/3211565444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8abc030d50 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac9c00640 1 -- 172.21.15.26:0/3211565444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 msgr2=0x7f8ac40cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac9c00640 1 --2- 172.21.15.26:0/3211565444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 0x7f8ac40cd840 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4072770 tx=0x7f8abc02b9a0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac9c00640 1 -- 172.21.15.26:0/3211565444 shutdown_connections 2024-01-15T17:57:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac9c00640 1 --2- 172.21.15.26:0/3211565444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 0x7f8ac40cd840 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac9c00640 1 --2- 172.21.15.26:0/3211565444 >> v2:172.21.15.124:3300/0 conn(0x7f8ac413df80 0x7f8ac415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac9c00640 1 -- 172.21.15.26:0/3211565444 >> 172.21.15.26:0/3211565444 conn(0x7f8ac406a800 msgr2=0x7f8ac406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac9c00640 1 -- 172.21.15.26:0/3211565444 shutdown_connections 2024-01-15T17:57:53.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.460+0000 7f8ac9c00640 1 -- 172.21.15.26:0/3211565444 wait complete. 2024-01-15T17:57:53.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.462+0000 7f8ac9c00640 1 Processor -- start 2024-01-15T17:57:53.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.462+0000 7f8ac9c00640 1 -- start start 2024-01-15T17:57:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.463+0000 7f8ac9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 0x7f8ac4147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.463+0000 7f8ac9c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac413df80 0x7f8ac40b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.463+0000 7f8ac9c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ac413e360 0x7f8ac40b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:53.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.463+0000 7f8ac9c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac4148c10 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.463+0000 7f8ac9c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac41448a0 con 0x7f8ac413df80 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.463+0000 7f8ac9c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac4147d20 con 0x7f8ac413e360 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.463+0000 7f8ac2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 0x7f8ac4147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 0x7f8ac4147240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48480/0 (socket says 172.21.15.26:48480) 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac2a00640 1 -- 172.21.15.26:0/2092443195 learned_addr learned my addr 172.21.15.26:0/2092443195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac3400640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ac413e360 0x7f8ac40b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac2a00640 1 -- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ac413e360 msgr2=0x7f8ac40b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac2a00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ac413e360 0x7f8ac40b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac2a00640 1 -- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac413df80 msgr2=0x7f8ac40b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac2000640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac413df80 0x7f8ac40b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac2a00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac413df80 0x7f8ac40b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac2a00640 1 -- 172.21.15.26:0/2092443195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0094d0 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac3400640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ac413e360 0x7f8ac40b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:53.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.464+0000 7f8ac2a00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 0x7f8ac4147240 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4007ef0 tx=0x7f8ab400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.465+0000 7f8abaa00640 1 -- 172.21.15.26:0/2092443195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4014620 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.465+0000 7f8abaa00640 1 -- 172.21.15.26:0/2092443195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab4014780 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.465+0000 7f8abaa00640 1 -- 172.21.15.26:0/2092443195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4014b30 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.465+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac4148000 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.465+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac41485c0 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.466+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac40675d0 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.467+0000 7f8abaa00640 1 -- 172.21.15.26:0/2092443195 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7f8ab4014c90 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.468+0000 7f8abaa00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a9c06b470 0x7f8a9c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:53.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.468+0000 7f8abaa00640 1 -- 172.21.15.26:0/2092443195 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ab408a940 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.469+0000 7f8ac2000640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a9c06b470 0x7f8a9c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:53.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.470+0000 7f8ac2000640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a9c06b470 0x7f8a9c06d930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007fd0 tx=0x7f8aac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:53.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.473+0000 7f8abaa00640 1 -- 172.21.15.26:0/2092443195 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ab4055490 con 0x7f8ac40cf9a0 2024-01-15T17:57:53.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.702+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ac415f9c0 con 0x7f8a9c06b470 2024-01-15T17:57:53.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.771+0000 7f8abaa00640 1 -- 172.21.15.26:0/2092443195 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8aac001c70 con 0x7f8a9c06b470 2024-01-15T17:57:53.774 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:57:53.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.773+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a9c06b470 msgr2=0x7f8a9c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:53.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.773+0000 7f8ac9c00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a9c06b470 0x7f8a9c06d930 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8aac007fd0 tx=0x7f8aac009040 comp rx=0 tx=0).stop 2024-01-15T17:57:53.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.773+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 msgr2=0x7f8ac4147240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:53.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.773+0000 7f8ac9c00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 0x7f8ac4147240 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4007ef0 tx=0x7f8ab400e4b0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.774+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 shutdown_connections 2024-01-15T17:57:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.774+0000 7f8ac9c00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ac413e360 0x7f8ac40b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.774+0000 7f8ac9c00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a9c06b470 0x7f8a9c06d930 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.774+0000 7f8ac9c00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac413df80 0x7f8ac40b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.774+0000 7f8ac9c00640 1 --2- 172.21.15.26:0/2092443195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40cf9a0 0x7f8ac4147240 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.774+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 >> 172.21.15.26:0/2092443195 conn(0x7f8ac406a800 msgr2=0x7f8ac40cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.774+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 shutdown_connections 2024-01-15T17:57:53.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:53.774+0000 7f8ac9c00640 1 -- 172.21.15.26:0/2092443195 wait complete. 2024-01-15T17:57:53.798 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T17:57:54.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.019+0000 7fbca7e00640 1 Processor -- start 2024-01-15T17:57:54.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.019+0000 7fbca7e00640 1 -- start start 2024-01-15T17:57:54.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.021+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806c730 0x7fbca80b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.021+0000 7fbca7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbca806cb10 0x7fbca806cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.021+0000 7fbca7e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca80708d0 con 0x7fbca8070160 2024-01-15T17:57:54.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.021+0000 7fbca7e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca8070bb0 con 0x7fbca806cb10 2024-01-15T17:57:54.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.021+0000 7fbca7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8070e90 con 0x7fbca806c730 2024-01-15T17:57:54.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.021+0000 7fbca7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbca806cb10 0x7fbca806cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.021+0000 7fbca7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbca806cb10 0x7fbca806cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48490/0 (socket says 172.21.15.26:48490) 2024-01-15T17:57:54.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.021+0000 7fbca7400640 1 -- 172.21.15.26:0/564553071 learned_addr learned my addr 172.21.15.26:0/564553071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:54.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.021+0000 7fbca6a00640 1 --2- 172.21.15.26:0/564553071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806c730 0x7fbca80b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.022+0000 7fbca7400640 1 -- 172.21.15.26:0/564553071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806c730 msgr2=0x7fbca80b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:54.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.022+0000 7fbca7400640 1 --2- 172.21.15.26:0/564553071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806c730 0x7fbca80b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.022+0000 7fbca7400640 1 -- 172.21.15.26:0/564553071 >> v1:172.21.15.124:6789/0 conn(0x7fbca8070160 legacy=0x7fbca8071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.022+0000 7fbca7400640 1 -- 172.21.15.26:0/564553071 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca8071170 con 0x7fbca806cb10 2024-01-15T17:57:54.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.022+0000 7fbca6a00640 1 --2- 172.21.15.26:0/564553071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806c730 0x7fbca80b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:57:54.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.023+0000 7fbca7400640 1 --2- 172.21.15.26:0/564553071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca806cb10 0x7fbca806cf80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009e50 tx=0x7fbca002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24decfaa2cacae42 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:54.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.023+0000 7fbca6000640 1 -- 172.21.15.26:0/564553071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca002cb00 con 0x7fbca806cb10 2024-01-15T17:57:54.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.023+0000 7fbca6000640 1 -- 172.21.15.26:0/564553071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca002cc60 con 0x7fbca806cb10 2024-01-15T17:57:54.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.023+0000 7fbca6000640 1 -- 172.21.15.26:0/564553071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca00343f0 con 0x7fbca806cb10 2024-01-15T17:57:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.024+0000 7fbca7e00640 1 -- 172.21.15.26:0/564553071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca806cb10 msgr2=0x7fbca806cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.024+0000 7fbca7e00640 1 --2- 172.21.15.26:0/564553071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca806cb10 0x7fbca806cf80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fbca0009e50 tx=0x7fbca002bf60 comp rx=0 tx=0).stop 2024-01-15T17:57:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.024+0000 7fbca7e00640 1 -- 172.21.15.26:0/564553071 shutdown_connections 2024-01-15T17:57:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.024+0000 7fbca7e00640 1 --2- 172.21.15.26:0/564553071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca806cb10 0x7fbca806cf80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.024+0000 7fbca7e00640 1 --2- 172.21.15.26:0/564553071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806c730 0x7fbca80b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.024+0000 7fbca7e00640 1 -- 172.21.15.26:0/564553071 >> 172.21.15.26:0/564553071 conn(0x7fbca806a7d0 msgr2=0x7fbca806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.024+0000 7fbca7e00640 1 -- 172.21.15.26:0/564553071 shutdown_connections 2024-01-15T17:57:54.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.024+0000 7fbca7e00640 1 -- 172.21.15.26:0/564553071 wait complete. 2024-01-15T17:57:54.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.026+0000 7fbca7e00640 1 Processor -- start 2024-01-15T17:57:54.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.026+0000 7fbca7e00640 1 -- start start 2024-01-15T17:57:54.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.027+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbca806c730 0x7fbca815d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.027+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806cb10 0x7fbca80ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.027+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca8070160 0x7fbca80ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.027+0000 7fbca7e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8147fd0 con 0x7fbca8070160 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.027+0000 7fbca7e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80a92f0 con 0x7fbca806c730 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.027+0000 7fbca7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca815ddc0 con 0x7fbca806cb10 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbca7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca8070160 0x7fbca80ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbca7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca8070160 0x7fbca80ab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48500/0 (socket says 172.21.15.26:48500) 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbca7400640 1 -- 172.21.15.26:0/3335250953 learned_addr learned my addr 172.21.15.26:0/3335250953 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbc9fe00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806cb10 0x7fbca80ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbca6a00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbca806c730 0x7fbca815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbca7400640 1 -- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806cb10 msgr2=0x7fbca80ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbca7400640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806cb10 0x7fbca80ab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbca7400640 1 -- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbca806c730 msgr2=0x7fbca815d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbca7400640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbca806c730 0x7fbca815d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbca7400640 1 -- 172.21.15.26:0/3335250953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca0009b30 con 0x7fbca8070160 2024-01-15T17:57:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.028+0000 7fbc9fe00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806cb10 0x7fbca80ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:57:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.029+0000 7fbca7400640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca8070160 0x7fbca80ab770 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbca0036040 tx=0x7fbca0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:54.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.029+0000 7fbc9ea00640 1 -- 172.21.15.26:0/3335250953 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca0009510 con 0x7fbca8070160 2024-01-15T17:57:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.029+0000 7fbc9ea00640 1 -- 172.21.15.26:0/3335250953 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca003e450 con 0x7fbca8070160 2024-01-15T17:57:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.029+0000 7fbc9ea00640 1 -- 172.21.15.26:0/3335250953 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca003e800 con 0x7fbca8070160 2024-01-15T17:57:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.029+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca80a89c0 con 0x7fbca8070160 2024-01-15T17:57:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.029+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca80a8f80 con 0x7fbca8070160 2024-01-15T17:57:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.030+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca8067450 con 0x7fbca8070160 2024-01-15T17:57:54.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.031+0000 7fbc9ea00640 1 -- 172.21.15.26:0/3335250953 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fbca003e960 con 0x7fbca8070160 2024-01-15T17:57:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.032+0000 7fbc9ea00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbc8006b4a0 0x7fbc8006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.032+0000 7fbc9ea00640 1 -- 172.21.15.26:0/3335250953 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbca00acce0 con 0x7fbca8070160 2024-01-15T17:57:54.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.033+0000 7fbca6a00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbc8006b4a0 0x7fbc8006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.034+0000 7fbca6a00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbc8006b4a0 0x7fbc8006d960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbca81482d0 tx=0x7fbc98008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:54.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.037+0000 7fbc9ea00640 1 -- 172.21.15.26:0/3335250953 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbca0077830 con 0x7fbca8070160 2024-01-15T17:57:54.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.261+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbca806d4c0 con 0x7fbc8006b4a0 2024-01-15T17:57:54.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.314+0000 7fbc9ea00640 1 -- 172.21.15.26:0/3335250953 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbc9800c590 con 0x7fbc8006b4a0 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbc8006b4a0 msgr2=0x7fbc8006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbc8006b4a0 0x7fbc8006d960 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbca81482d0 tx=0x7fbc98008040 comp rx=0 tx=0).stop 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca8070160 msgr2=0x7fbca80ab770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca8070160 0x7fbca80ab770 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbca0036040 tx=0x7fbca0004a40 comp rx=0 tx=0).stop 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 shutdown_connections 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbca806cb10 0x7fbca80ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbc8006b4a0 0x7fbc8006d960 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbca8070160 0x7fbca80ab770 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 --2- 172.21.15.26:0/3335250953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbca806c730 0x7fbca815d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 >> 172.21.15.26:0/3335250953 conn(0x7fbca806a7d0 msgr2=0x7fbca80b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 shutdown_connections 2024-01-15T17:57:54.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.316+0000 7fbca7e00640 1 -- 172.21.15.26:0/3335250953 wait complete. 2024-01-15T17:57:54.327 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.324+0000 7f8f56e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.327 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.324+0000 7f8f56e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.327 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.324+0000 7f8f56e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.327 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.324+0000 7f8f56e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.327 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.324+0000 7f8f56e00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.338 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-01-15T17:57:54.350 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.346+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.350 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.346+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.350 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.346+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.350 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.346+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.350 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:57:54.346+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:57:54.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.544+0000 7fe53ee00640 1 Processor -- start 2024-01-15T17:57:54.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.544+0000 7fe53ee00640 1 -- start start 2024-01-15T17:57:54.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.546+0000 7fe53ee00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe53813cd70 0x7fe53813d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.546+0000 7fe53ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5380af7b0 0x7fe53815a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.546+0000 7fe53ee00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe53815ade0 con 0x7fe53806e7e0 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.546+0000 7fe53ee00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53815b0c0 con 0x7fe53813cd70 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.546+0000 7fe53ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53815b3a0 con 0x7fe5380af7b0 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.546+0000 7fe537e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe53813cd70 0x7fe53813d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.546+0000 7fe537e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe53813cd70 0x7fe53813d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48502/0 (socket says 172.21.15.26:48502) 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.546+0000 7fe537e00640 1 -- 172.21.15.26:0/1982256805 learned_addr learned my addr 172.21.15.26:0/1982256805 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.547+0000 7fe537e00640 1 -- 172.21.15.26:0/1982256805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5380af7b0 msgr2=0x7fe53815a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.547+0000 7fe537e00640 1 --2- 172.21.15.26:0/1982256805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5380af7b0 0x7fe53815a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.547+0000 7fe537e00640 1 -- 172.21.15.26:0/1982256805 >> v1:172.21.15.124:6789/0 conn(0x7fe53806e7e0 legacy=0x7fe5380cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:57:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.547+0000 7fe537e00640 1 -- 172.21.15.26:0/1982256805 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5380ced80 con 0x7fe53813cd70 2024-01-15T17:57:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.548+0000 7fe537e00640 1 --2- 172.21.15.26:0/1982256805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53813cd70 0x7fe53813d5d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe528009870 tx=0x7fe52802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5735f1de505e25df server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.548+0000 7fe536a00640 1 -- 172.21.15.26:0/1982256805 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528031640 con 0x7fe53813cd70 2024-01-15T17:57:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.548+0000 7fe536a00640 1 -- 172.21.15.26:0/1982256805 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5280317a0 con 0x7fe53813cd70 2024-01-15T17:57:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.548+0000 7fe536a00640 1 -- 172.21.15.26:0/1982256805 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528031b30 con 0x7fe53813cd70 2024-01-15T17:57:54.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.549+0000 7fe53ee00640 1 -- 172.21.15.26:0/1982256805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53813cd70 msgr2=0x7fe53813d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.549+0000 7fe53ee00640 1 --2- 172.21.15.26:0/1982256805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53813cd70 0x7fe53813d5d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe528009870 tx=0x7fe52802b8f0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.549+0000 7fe53ee00640 1 -- 172.21.15.26:0/1982256805 shutdown_connections 2024-01-15T17:57:54.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.549+0000 7fe53ee00640 1 --2- 172.21.15.26:0/1982256805 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5380af7b0 0x7fe53815a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.549+0000 7fe53ee00640 1 --2- 172.21.15.26:0/1982256805 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe53813cd70 0x7fe53813d5d0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.549+0000 7fe53ee00640 1 -- 172.21.15.26:0/1982256805 >> 172.21.15.26:0/1982256805 conn(0x7fe5380b1d40 msgr2=0x7fe5380b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:54.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.549+0000 7fe53ee00640 1 -- 172.21.15.26:0/1982256805 shutdown_connections 2024-01-15T17:57:54.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.549+0000 7fe53ee00640 1 -- 172.21.15.26:0/1982256805 wait complete. 2024-01-15T17:57:54.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.551+0000 7fe53ee00640 1 Processor -- start 2024-01-15T17:57:54.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.551+0000 7fe53ee00640 1 -- start start 2024-01-15T17:57:54.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe53ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe53806e7e0 0x7fe5381505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe53ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5380af7b0 0x7fe5380b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe53ee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe53813cd70 0x7fe5381e5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe53ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5381524e0 con 0x7fe5380af7b0 2024-01-15T17:57:54.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe53ee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5381e9c00 con 0x7fe53813cd70 2024-01-15T17:57:54.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe53ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5381e4d80 con 0x7fe53806e7e0 2024-01-15T17:57:54.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe537400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5380af7b0 0x7fe5380b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe537400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5380af7b0 0x7fe5380b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48512/0 (socket says 172.21.15.26:48512) 2024-01-15T17:57:54.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe537400640 1 -- 172.21.15.26:0/2656110090 learned_addr learned my addr 172.21.15.26:0/2656110090 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:57:54.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe537e00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe53806e7e0 0x7fe5381505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe53ce00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe53813cd70 0x7fe5381e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe537400640 1 -- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe53806e7e0 msgr2=0x7fe5381505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe537400640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe53806e7e0 0x7fe5381505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe537400640 1 -- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe53813cd70 msgr2=0x7fe5381e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe537400640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe53813cd70 0x7fe5381e5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.552+0000 7fe537400640 1 -- 172.21.15.26:0/2656110090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280094d0 con 0x7fe5380af7b0 2024-01-15T17:57:54.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.554+0000 7fe53ce00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe53813cd70 0x7fe5381e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:57:54.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.554+0000 7fe537400640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5380af7b0 0x7fe5380b2e30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b4d0 tx=0x7fe52000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:54.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.554+0000 7fe52fe00640 1 -- 172.21.15.26:0/2656110090 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe520010870 con 0x7fe5380af7b0 2024-01-15T17:57:54.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.554+0000 7fe52fe00640 1 -- 172.21.15.26:0/2656110090 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5200109d0 con 0x7fe5380af7b0 2024-01-15T17:57:54.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.554+0000 7fe52fe00640 1 -- 172.21.15.26:0/2656110090 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe520010d80 con 0x7fe5380af7b0 2024-01-15T17:57:54.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.554+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53815abb0 con 0x7fe5380af7b0 2024-01-15T17:57:54.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.554+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5381e4730 con 0x7fe5380af7b0 2024-01-15T17:57:54.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.555+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53815a9b0 con 0x7fe5380af7b0 2024-01-15T17:57:54.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.556+0000 7fe52fe00640 1 -- 172.21.15.26:0/2656110090 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 7) v1 ==== 89692+0+0 (secure 0 0 0) 0x7fe52000f3c0 con 0x7fe5380af7b0 2024-01-15T17:57:54.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.558+0000 7fe52fe00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe51006b470 0x7fe51006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:57:54.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.558+0000 7fe52fe00640 1 -- 172.21.15.26:0/2656110090 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe520089fd0 con 0x7fe5380af7b0 2024-01-15T17:57:54.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.558+0000 7fe537e00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe51006b470 0x7fe51006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:57:54.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.559+0000 7fe537e00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe51006b470 0x7fe51006d930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe528036000 tx=0x7fe52803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:57:54.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.566+0000 7fe52fe00640 1 -- 172.21.15.26:0/2656110090 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe520054a90 con 0x7fe5380af7b0 2024-01-15T17:57:54.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.781+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5380ce9a0 con 0x7fe51006b470 2024-01-15T17:57:54.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.823+0000 7fe52fe00640 1 -- 172.21.15.26:0/2656110090 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe528031a90 con 0x7fe51006b470 2024-01-15T17:57:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.825+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe51006b470 msgr2=0x7fe51006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.825+0000 7fe53ee00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe51006b470 0x7fe51006d930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe528036000 tx=0x7fe52803c040 comp rx=0 tx=0).stop 2024-01-15T17:57:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.825+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5380af7b0 msgr2=0x7fe5380b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:57:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.825+0000 7fe53ee00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5380af7b0 0x7fe5380b2e30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b4d0 tx=0x7fe52000b990 comp rx=0 tx=0).stop 2024-01-15T17:57:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.826+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 shutdown_connections 2024-01-15T17:57:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.826+0000 7fe53ee00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe51006b470 0x7fe51006d930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.826+0000 7fe53ee00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe53813cd70 0x7fe5381e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.826+0000 7fe53ee00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5380af7b0 0x7fe5380b2e30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.826+0000 7fe53ee00640 1 --2- 172.21.15.26:0/2656110090 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe53806e7e0 0x7fe5381505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:57:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.826+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 >> 172.21.15.26:0/2656110090 conn(0x7fe5380b1d40 msgr2=0x7fe5380b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:57:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.826+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 shutdown_connections 2024-01-15T17:57:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:57:54.826+0000 7fe53ee00640 1 -- 172.21.15.26:0/2656110090 wait complete. 2024-01-15T17:57:54.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:57:54.848 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T17:57:54.906 INFO:teuthology.orchestra.run.smithi026.stdout:3 2024-01-15T17:57:56.397 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:57:56.414 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:57:59.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:57:59.908 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T17:57:59.973 INFO:teuthology.orchestra.run.smithi026.stdout:2 2024-01-15T17:57:59.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T17:58:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.178+0000 7f30e4c00640 1 Processor -- start 2024-01-15T17:58:00.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.178+0000 7f30e4c00640 1 -- start start 2024-01-15T17:58:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.179+0000 7f30e4c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 0x7f30e015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.179+0000 7f30e4c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30e00cf9a0 0x7f30e00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:00.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.179+0000 7f30e4c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e006c470 con 0x7f30e013e360 2024-01-15T17:58:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.179+0000 7f30e4c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e006c750 con 0x7f30e00cf9a0 2024-01-15T17:58:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.179+0000 7f30e4c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e006ca30 con 0x7f30e013df80 2024-01-15T17:58:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.179+0000 7f30de800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30e00cf9a0 0x7f30e00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.179+0000 7f30de800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f30e00cf9a0 0x7f30e00cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48522/0 (socket says 172.21.15.26:48522) 2024-01-15T17:58:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.179+0000 7f30de800640 1 -- 172.21.15.26:0/3177266205 learned_addr learned my addr 172.21.15.26:0/3177266205 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:00.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.179+0000 7f30dde00640 1 --2- 172.21.15.26:0/3177266205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 0x7f30e015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.180+0000 7f30de800640 1 -- 172.21.15.26:0/3177266205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 msgr2=0x7f30e015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.180+0000 7f30de800640 1 --2- 172.21.15.26:0/3177266205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 0x7f30e015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.180+0000 7f30de800640 1 -- 172.21.15.26:0/3177266205 >> v1:172.21.15.124:6789/0 conn(0x7f30e013e360 legacy=0x7f30e00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.180+0000 7f30de800640 1 -- 172.21.15.26:0/3177266205 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e006cd10 con 0x7f30e00cf9a0 2024-01-15T17:58:00.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.180+0000 7f30dde00640 1 --2- 172.21.15.26:0/3177266205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 0x7f30e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:58:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.181+0000 7f30de800640 1 --2- 172.21.15.26:0/3177266205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 0x7f30e00cd840 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009e50 tx=0x7f30d802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5977b3d01dda4af server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.181+0000 7f30dd400640 1 -- 172.21.15.26:0/3177266205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d802cb00 con 0x7f30e00cf9a0 2024-01-15T17:58:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30dd400640 1 -- 172.21.15.26:0/3177266205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d802cc60 con 0x7f30e00cf9a0 2024-01-15T17:58:00.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30dd400640 1 -- 172.21.15.26:0/3177266205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d80343f0 con 0x7f30e00cf9a0 2024-01-15T17:58:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30e4c00640 1 -- 172.21.15.26:0/3177266205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 msgr2=0x7f30e00cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30e4c00640 1 --2- 172.21.15.26:0/3177266205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 0x7f30e00cd840 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009e50 tx=0x7f30d802bf60 comp rx=0 tx=0).stop 2024-01-15T17:58:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30e4c00640 1 -- 172.21.15.26:0/3177266205 shutdown_connections 2024-01-15T17:58:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30e4c00640 1 --2- 172.21.15.26:0/3177266205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 0x7f30e00cd840 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30e4c00640 1 --2- 172.21.15.26:0/3177266205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 0x7f30e015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30e4c00640 1 -- 172.21.15.26:0/3177266205 >> 172.21.15.26:0/3177266205 conn(0x7f30e006a800 msgr2=0x7f30e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30e4c00640 1 -- 172.21.15.26:0/3177266205 shutdown_connections 2024-01-15T17:58:00.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.182+0000 7f30e4c00640 1 -- 172.21.15.26:0/3177266205 wait complete. 2024-01-15T17:58:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.184+0000 7f30e4c00640 1 Processor -- start 2024-01-15T17:58:00.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.184+0000 7f30e4c00640 1 -- start start 2024-01-15T17:58:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.185+0000 7f30e4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 0x7f30e00b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.185+0000 7f30e4c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 0x7f30e013d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.185+0000 7f30e4c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30e013e360 0x7f30e00afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.185+0000 7f30e4c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00ae370 con 0x7f30e00cf9a0 2024-01-15T17:58:00.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.185+0000 7f30e4c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e0071de0 con 0x7f30e013e360 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.185+0000 7f30e4c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b31f0 con 0x7f30e013df80 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.185+0000 7f30dde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 0x7f30e00b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30dde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 0x7f30e00b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48532/0 (socket says 172.21.15.26:48532) 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30dde00640 1 -- 172.21.15.26:0/2088132084 learned_addr learned my addr 172.21.15.26:0/2088132084 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30de800640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30e013e360 0x7f30e00afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30d7e00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 0x7f30e013d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30dde00640 1 -- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 msgr2=0x7f30e013d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30dde00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 0x7f30e013d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30dde00640 1 -- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30e013e360 msgr2=0x7f30e00afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30dde00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30e013e360 0x7f30e00afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30dde00640 1 -- 172.21.15.26:0/2088132084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d8009b30 con 0x7f30e00cf9a0 2024-01-15T17:58:00.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.186+0000 7f30dde00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 0x7f30e00b2cb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f30d000e760 tx=0x7f30d000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.187+0000 7f30d6000640 1 -- 172.21.15.26:0/2088132084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0004230 con 0x7f30e00cf9a0 2024-01-15T17:58:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.187+0000 7f30d6000640 1 -- 172.21.15.26:0/2088132084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0004390 con 0x7f30e00cf9a0 2024-01-15T17:58:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.187+0000 7f30d6000640 1 -- 172.21.15.26:0/2088132084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d000c8f0 con 0x7f30e00cf9a0 2024-01-15T17:58:00.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.187+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e0072270 con 0x7f30e00cf9a0 2024-01-15T17:58:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.187+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e00714b0 con 0x7f30e00cf9a0 2024-01-15T17:58:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.188+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e00675d0 con 0x7f30e00cf9a0 2024-01-15T17:58:00.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.189+0000 7f30d6000640 1 -- 172.21.15.26:0/2088132084 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 8) v1 ==== 89794+0+0 (secure 0 0 0) 0x7f30d000ca50 con 0x7f30e00cf9a0 2024-01-15T17:58:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.190+0000 7f30d6000640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30b806b590 0x7f30b806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:00.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.191+0000 7f30d6000640 1 -- 172.21.15.26:0/2088132084 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f30d0089e20 con 0x7f30e00cf9a0 2024-01-15T17:58:00.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.191+0000 7f30d7e00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30b806b590 0x7f30b806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:00.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.193+0000 7f30d7e00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30b806b590 0x7f30b806da50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f30e01f5e30 tx=0x7f30c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:00.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.195+0000 7f30d6000640 1 -- 172.21.15.26:0/2088132084 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f30d00547b0 con 0x7f30e00cf9a0 2024-01-15T17:58:00.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:00.418+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f30e015f9c0 con 0x7f30b806b590 2024-01-15T17:58:00.479 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-01-15T17:58:00.474+0000 7f9819c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T17:58:00.579 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T17:58:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:02.984+0000 7f30d6000640 1 -- 172.21.15.26:0/2088132084 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f30d008f020 con 0x7f30e00cf9a0 2024-01-15T17:58:03.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.519+0000 7f30d6000640 1 -- 172.21.15.26:0/2088132084 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f30c8001c70 con 0x7f30b806b590 2024-01-15T17:58:03.522 INFO:teuthology.orchestra.run.smithi026.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T17:58:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30b806b590 msgr2=0x7f30b806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30b806b590 0x7f30b806da50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f30e01f5e30 tx=0x7f30c8009040 comp rx=0 tx=0).stop 2024-01-15T17:58:03.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 msgr2=0x7f30e00b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 0x7f30e00b2cb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f30d000e760 tx=0x7f30d000ec20 comp rx=0 tx=0).stop 2024-01-15T17:58:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 shutdown_connections 2024-01-15T17:58:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f30e013df80 0x7f30e013d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f30b806b590 0x7f30b806da50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:03.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f30e013e360 0x7f30e00afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 --2- 172.21.15.26:0/2088132084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f30e00cf9a0 0x7f30e00b2cb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.523+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 >> 172.21.15.26:0/2088132084 conn(0x7f30e006a800 msgr2=0x7f30e015c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.524+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 shutdown_connections 2024-01-15T17:58:03.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:03.524+0000 7f30e4c00640 1 -- 172.21.15.26:0/2088132084 wait complete. 2024-01-15T17:58:03.550 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T17:58:03.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T17:58:03.654 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T17:58:03.775 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T17:58:03.775 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:58:03.775 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:03.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:58:04.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.061+0000 7fcb86e00640 1 Processor -- start 2024-01-15T17:58:04.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.061+0000 7fcb86e00640 1 -- start start 2024-01-15T17:58:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.063+0000 7fcb86e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb80143ae0 0x7fcb800cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.063+0000 7fcb86e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb8015e3b0 0x7fcb800702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.063+0000 7fcb86e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb800cfc30 con 0x7fcb800cf7d0 2024-01-15T17:58:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.063+0000 7fcb86e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb800b1ee0 con 0x7fcb80143ae0 2024-01-15T17:58:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.063+0000 7fcb86e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb800b2780 con 0x7fcb8015e3b0 2024-01-15T17:58:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.063+0000 7fcb7fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb80143ae0 0x7fcb800cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.063+0000 7fcb7fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb80143ae0 0x7fcb800cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40644/0 (socket says 172.21.15.26:40644) 2024-01-15T17:58:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.063+0000 7fcb7fe00640 1 -- 172.21.15.26:0/3529517295 learned_addr learned my addr 172.21.15.26:0/3529517295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:04.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.064+0000 7fcb84e00640 1 --2- 172.21.15.26:0/3529517295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb8015e3b0 0x7fcb800702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.064+0000 7fcb7fe00640 1 -- 172.21.15.26:0/3529517295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb8015e3b0 msgr2=0x7fcb800702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.064+0000 7fcb7fe00640 1 --2- 172.21.15.26:0/3529517295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb8015e3b0 0x7fcb800702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.064+0000 7fcb7fe00640 1 -- 172.21.15.26:0/3529517295 >> v1:172.21.15.124:6789/0 conn(0x7fcb800cf7d0 legacy=0x7fcb8015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.064+0000 7fcb7fe00640 1 -- 172.21.15.26:0/3529517295 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8015e990 con 0x7fcb80143ae0 2024-01-15T17:58:04.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.065+0000 7fcb7fe00640 1 --2- 172.21.15.26:0/3529517295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 0x7fcb800cf290 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009870 tx=0x7fcb7002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecd1f523e5461b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.065+0000 7fcb7ea00640 1 -- 172.21.15.26:0/3529517295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70030840 con 0x7fcb80143ae0 2024-01-15T17:58:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.065+0000 7fcb7ea00640 1 -- 172.21.15.26:0/3529517295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb700309a0 con 0x7fcb80143ae0 2024-01-15T17:58:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.065+0000 7fcb7ea00640 1 -- 172.21.15.26:0/3529517295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb70030d30 con 0x7fcb80143ae0 2024-01-15T17:58:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.066+0000 7fcb86e00640 1 -- 172.21.15.26:0/3529517295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 msgr2=0x7fcb800cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.066+0000 7fcb86e00640 1 --2- 172.21.15.26:0/3529517295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 0x7fcb800cf290 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcb70009870 tx=0x7fcb7002b980 comp rx=0 tx=0).stop 2024-01-15T17:58:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.066+0000 7fcb86e00640 1 -- 172.21.15.26:0/3529517295 shutdown_connections 2024-01-15T17:58:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.066+0000 7fcb86e00640 1 --2- 172.21.15.26:0/3529517295 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb8015e3b0 0x7fcb800702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.066+0000 7fcb86e00640 1 --2- 172.21.15.26:0/3529517295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 0x7fcb800cf290 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.066+0000 7fcb86e00640 1 -- 172.21.15.26:0/3529517295 >> 172.21.15.26:0/3529517295 conn(0x7fcb8006a800 msgr2=0x7fcb8006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.066+0000 7fcb86e00640 1 -- 172.21.15.26:0/3529517295 shutdown_connections 2024-01-15T17:58:04.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.066+0000 7fcb86e00640 1 -- 172.21.15.26:0/3529517295 wait complete. 2024-01-15T17:58:04.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.068+0000 7fcb86e00640 1 Processor -- start 2024-01-15T17:58:04.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.068+0000 7fcb86e00640 1 -- start start 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.069+0000 7fcb86e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb800cf7d0 0x7fcb8013be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.069+0000 7fcb86e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 0x7fcb80159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.069+0000 7fcb86e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb8015e3b0 0x7fcb800acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.069+0000 7fcb86e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8013c4f0 con 0x7fcb80143ae0 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.069+0000 7fcb86e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8013c7d0 con 0x7fcb8015e3b0 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.069+0000 7fcb86e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8013cab0 con 0x7fcb800cf7d0 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.069+0000 7fcb7f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 0x7fcb80159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.069+0000 7fcb7f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 0x7fcb80159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40654/0 (socket says 172.21.15.26:40654) 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.069+0000 7fcb7f400640 1 -- 172.21.15.26:0/1700231494 learned_addr learned my addr 172.21.15.26:0/1700231494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb7fe00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb800cf7d0 0x7fcb8013be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb84e00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb8015e3b0 0x7fcb800acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb7f400640 1 -- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb800cf7d0 msgr2=0x7fcb8013be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb7f400640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb800cf7d0 0x7fcb8013be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb7f400640 1 -- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb8015e3b0 msgr2=0x7fcb800acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb7f400640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb8015e3b0 0x7fcb800acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb7f400640 1 -- 172.21.15.26:0/1700231494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb700094d0 con 0x7fcb80143ae0 2024-01-15T17:58:04.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb84e00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb8015e3b0 0x7fcb800acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb7f400640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 0x7fcb80159a00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c450 tx=0x7fcb6800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb77e00640 1 -- 172.21.15.26:0/1700231494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb680052d0 con 0x7fcb80143ae0 2024-01-15T17:58:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb77e00640 1 -- 172.21.15.26:0/1700231494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb68005430 con 0x7fcb80143ae0 2024-01-15T17:58:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb77e00640 1 -- 172.21.15.26:0/1700231494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb68012870 con 0x7fcb80143ae0 2024-01-15T17:58:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.070+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb800aca70 con 0x7fcb80143ae0 2024-01-15T17:58:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.071+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb800ad4d0 con 0x7fcb80143ae0 2024-01-15T17:58:04.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.072+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb800ac8d0 con 0x7fcb80143ae0 2024-01-15T17:58:04.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.073+0000 7fcb77e00640 1 -- 172.21.15.26:0/1700231494 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fcb680129d0 con 0x7fcb80143ae0 2024-01-15T17:58:04.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.074+0000 7fcb77e00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb5806b500 0x7fcb5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.074+0000 7fcb77e00640 1 -- 172.21.15.26:0/1700231494 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fcb680897b0 con 0x7fcb80143ae0 2024-01-15T17:58:04.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.074+0000 7fcb7fe00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb5806b500 0x7fcb5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.075+0000 7fcb7fe00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb5806b500 0x7fcb5806d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcb70036000 tx=0x7fcb7003b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:04.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.079+0000 7fcb77e00640 1 -- 172.21.15.26:0/1700231494 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb800ac8d0 con 0x7fcb80143ae0 2024-01-15T17:58:04.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.275+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb800accb0 con 0x7fcb80143ae0 2024-01-15T17:58:04.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.277+0000 7fcb77e00640 1 -- 172.21.15.26:0/1700231494 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13007 (secure 0 0 0) 0x7fcb800accb0 con 0x7fcb80143ae0 2024-01-15T17:58:04.281 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:04.281 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":39,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:58:03.490741+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6836/2548050157":"2024-01-16T17:58:00.474566+0000","172.21.15.124:6834/2548050157":"2024-01-16T17:58:00.474566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb5806b500 msgr2=0x7fcb5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb5806b500 0x7fcb5806d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcb70036000 tx=0x7fcb7003b040 comp rx=0 tx=0).stop 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 msgr2=0x7fcb80159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 0x7fcb80159a00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcb6800c450 tx=0x7fcb6800c910 comp rx=0 tx=0).stop 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 shutdown_connections 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb5806b500 0x7fcb5806d9c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb8015e3b0 0x7fcb800acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb80143ae0 0x7fcb80159a00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 --2- 172.21.15.26:0/1700231494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb800cf7d0 0x7fcb8013be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.280+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 >> 172.21.15.26:0/1700231494 conn(0x7fcb8006a800 msgr2=0x7fcb8015da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:04.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.281+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 shutdown_connections 2024-01-15T17:58:04.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.281+0000 7fcb86e00640 1 -- 172.21.15.26:0/1700231494 wait complete. 2024-01-15T17:58:04.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:58:04.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:58:04.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.526+0000 7f1ef7800640 1 Processor -- start 2024-01-15T17:58:04.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.526+0000 7f1ef7800640 1 -- start start 2024-01-15T17:58:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.527+0000 7f1ef7800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ef00cd240 0x7f1ef00b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.528+0000 7f1ef7800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef006cea0 con 0x7f1ef009a170 2024-01-15T17:58:04.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.528+0000 7f1ef7800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef006cbc0 con 0x7f1ef006c6b0 2024-01-15T17:58:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.528+0000 7f1ef7800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef006f350 con 0x7f1ef00cd240 2024-01-15T17:58:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.528+0000 7f1ef4e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1ef00cd240 0x7f1ef00b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.528+0000 7f1ef5800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1ef009a170 0x7f1ef00cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39012/0 (socket says 172.21.15.26:39012) 2024-01-15T17:58:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.528+0000 7f1ef5800640 1 -- 172.21.15.26:0/448664639 learned_addr learned my addr 172.21.15.26:0/448664639 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.528+0000 7f1ef4e00640 1 -- 172.21.15.26:0/448664639 >> v1:172.21.15.124:6789/0 conn(0x7f1ef006c6b0 legacy=0x7f1ef0099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.528+0000 7f1ef4e00640 1 -- 172.21.15.26:0/448664639 >> v1:172.21.15.26:6789/0 conn(0x7f1ef009a170 legacy=0x7f1ef00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.528+0000 7f1ef4e00640 1 -- 172.21.15.26:0/448664639 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef006f630 con 0x7f1ef00cd240 2024-01-15T17:58:04.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.530+0000 7f1ef4e00640 1 --2- 172.21.15.26:0/448664639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef00cd240 0x7f1ef00b3820 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00097a0 tx=0x7f1ee002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9946d8578f5b43a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:04.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.530+0000 7f1eefe00640 1 -- 172.21.15.26:0/448664639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0031830 con 0x7f1ef00cd240 2024-01-15T17:58:04.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.530+0000 7f1eefe00640 1 -- 172.21.15.26:0/448664639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee0031990 con 0x7f1ef00cd240 2024-01-15T17:58:04.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.530+0000 7f1eefe00640 1 -- 172.21.15.26:0/448664639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0031d20 con 0x7f1ef00cd240 2024-01-15T17:58:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.531+0000 7f1ef7800640 1 -- 172.21.15.26:0/448664639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef00cd240 msgr2=0x7f1ef00b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.531+0000 7f1ef7800640 1 --2- 172.21.15.26:0/448664639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef00cd240 0x7f1ef00b3820 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1ee00097a0 tx=0x7f1ee002b8d0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.531+0000 7f1ef7800640 1 -- 172.21.15.26:0/448664639 shutdown_connections 2024-01-15T17:58:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.531+0000 7f1ef7800640 1 --2- 172.21.15.26:0/448664639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef00cd240 0x7f1ef00b3820 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.531+0000 7f1ef7800640 1 -- 172.21.15.26:0/448664639 >> 172.21.15.26:0/448664639 conn(0x7f1ef00b2d00 msgr2=0x7f1ef00b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.531+0000 7f1ef7800640 1 -- 172.21.15.26:0/448664639 shutdown_connections 2024-01-15T17:58:04.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.531+0000 7f1ef7800640 1 -- 172.21.15.26:0/448664639 wait complete. 2024-01-15T17:58:04.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.533+0000 7f1ef7800640 1 Processor -- start 2024-01-15T17:58:04.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.533+0000 7f1ef7800640 1 -- start start 2024-01-15T17:58:04.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef7800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef006c6b0 0x7f1ef00ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef7800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef009a170 0x7f1ef00b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef7800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef00cd240 0x7f1ef00b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef7800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00b4420 con 0x7f1ef006c6b0 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef7800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00b4700 con 0x7f1ef009a170 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef7800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef00b49e0 con 0x7f1ef00cd240 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef4e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef006c6b0 0x7f1ef00ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef4e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef006c6b0 0x7f1ef00ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40674/0 (socket says 172.21.15.26:40674) 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef4e00640 1 -- 172.21.15.26:0/3335171069 learned_addr learned my addr 172.21.15.26:0/3335171069 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef4e00640 1 -- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef00cd240 msgr2=0x7f1ef00b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef4e00640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef00cd240 0x7f1ef00b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef4e00640 1 -- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef009a170 msgr2=0x7f1ef00b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef4e00640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef009a170 0x7f1ef00b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.534+0000 7f1ef4e00640 1 -- 172.21.15.26:0/3335171069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee0009400 con 0x7f1ef006c6b0 2024-01-15T17:58:04.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.535+0000 7f1ef4e00640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef006c6b0 0x7f1ef00ca7a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0000c00 tx=0x7f1ee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:04.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.535+0000 7f1eed600640 1 -- 172.21.15.26:0/3335171069 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee003ab60 con 0x7f1ef006c6b0 2024-01-15T17:58:04.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.535+0000 7f1eed600640 1 -- 172.21.15.26:0/3335171069 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee003ace0 con 0x7f1ef006c6b0 2024-01-15T17:58:04.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.535+0000 7f1eed600640 1 -- 172.21.15.26:0/3335171069 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0044630 con 0x7f1ef006c6b0 2024-01-15T17:58:04.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.535+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef00b4cc0 con 0x7f1ef006c6b0 2024-01-15T17:58:04.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.535+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef00b5a00 con 0x7f1ef006c6b0 2024-01-15T17:58:04.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.537+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef00b51e0 con 0x7f1ef006c6b0 2024-01-15T17:58:04.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.537+0000 7f1eed600640 1 -- 172.21.15.26:0/3335171069 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f1ee004d450 con 0x7f1ef006c6b0 2024-01-15T17:58:04.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.538+0000 7f1eed600640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ec806b550 0x7f1ec806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.538+0000 7f1eed600640 1 -- 172.21.15.26:0/3335171069 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f1ee00abf30 con 0x7f1ef006c6b0 2024-01-15T17:58:04.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.539+0000 7f1ee7e00640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ec806b550 0x7f1ec806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.540+0000 7f1ee7e00640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ec806b550 0x7f1ec806da10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00c55e0 tx=0x7f1ed80079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:04.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.543+0000 7f1eed600640 1 -- 172.21.15.26:0/3335171069 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ee00dee90 con 0x7f1ef006c6b0 2024-01-15T17:58:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.737+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ef0073d90 con 0x7f1ef006c6b0 2024-01-15T17:58:04.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.739+0000 7f1eed600640 1 -- 172.21.15.26:0/3335171069 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ef00b51e0 con 0x7f1ef006c6b0 2024-01-15T17:58:04.742 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:58:04.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ec806b550 msgr2=0x7f1ec806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ec806b550 0x7f1ec806da10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00c55e0 tx=0x7f1ed80079b0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef006c6b0 msgr2=0x7f1ef00ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef006c6b0 0x7f1ef00ca7a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0000c00 tx=0x7f1ee00046c0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 shutdown_connections 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1ef00cd240 0x7f1ef00b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ec806b550 0x7f1ec806da10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1ef009a170 0x7f1ef00b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 --2- 172.21.15.26:0/3335171069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1ef006c6b0 0x7f1ef00ca7a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.741+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 >> 172.21.15.26:0/3335171069 conn(0x7f1ef00b2d00 msgr2=0x7f1ef00b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.742+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 shutdown_connections 2024-01-15T17:58:04.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.742+0000 7f1ef7800640 1 -- 172.21.15.26:0/3335171069 wait complete. 2024-01-15T17:58:04.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:58:04.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.978+0000 7fc59b800640 1 Processor -- start 2024-01-15T17:58:04.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.979+0000 7fc59b800640 1 -- start start 2024-01-15T17:58:04.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.980+0000 7fc59b800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc59413df80 0x7fc59415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.980+0000 7fc59b800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc59413e360 0x7fc5940cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.980+0000 7fc59b800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59415f9c0 0x7fc5940cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.980+0000 7fc59b800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59406c470 con 0x7fc59413e360 2024-01-15T17:58:04.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.980+0000 7fc59b800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59406c750 con 0x7fc59413df80 2024-01-15T17:58:04.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.980+0000 7fc59b800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59406ca30 con 0x7fc59415f9c0 2024-01-15T17:58:04.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc58be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc59413e360 0x7fc5940cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc58be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc59413e360 0x7fc5940cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40682/0 (socket says 172.21.15.26:40682) 2024-01-15T17:58:04.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc58be00640 1 -- 172.21.15.26:0/267987370 learned_addr learned my addr 172.21.15.26:0/267987370 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:04.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc598e00640 1 --2- 172.21.15.26:0/267987370 >> v2:172.21.15.124:3300/0 conn(0x7fc59413df80 0x7fc59415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc599800640 1 --2- 172.21.15.26:0/267987370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59415f9c0 0x7fc5940cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc58be00640 1 -- 172.21.15.26:0/267987370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59415f9c0 msgr2=0x7fc5940cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:04.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc58be00640 1 --2- 172.21.15.26:0/267987370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59415f9c0 0x7fc5940cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc58be00640 1 -- 172.21.15.26:0/267987370 >> v2:172.21.15.124:3300/0 conn(0x7fc59413df80 msgr2=0x7fc59415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:04.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc58be00640 1 --2- 172.21.15.26:0/267987370 >> v2:172.21.15.124:3300/0 conn(0x7fc59413df80 0x7fc59415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.981+0000 7fc58be00640 1 -- 172.21.15.26:0/267987370 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59406cd10 con 0x7fc59413e360 2024-01-15T17:58:04.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.983+0000 7fc58be00640 1 --2- 172.21.15.26:0/267987370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 0x7fc5940cf290 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc594072770 tx=0x7fc57c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cfd445dbb398b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:04.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.983+0000 7fc593e00640 1 -- 172.21.15.26:0/267987370 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c031860 con 0x7fc59413e360 2024-01-15T17:58:04.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.983+0000 7fc593e00640 1 -- 172.21.15.26:0/267987370 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c0319c0 con 0x7fc59413e360 2024-01-15T17:58:04.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.983+0000 7fc593e00640 1 -- 172.21.15.26:0/267987370 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c031d50 con 0x7fc59413e360 2024-01-15T17:58:04.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.983+0000 7fc59b800640 1 -- 172.21.15.26:0/267987370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 msgr2=0x7fc5940cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.983+0000 7fc59b800640 1 --2- 172.21.15.26:0/267987370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 0x7fc5940cf290 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc594072770 tx=0x7fc57c02b900 comp rx=0 tx=0).stop 2024-01-15T17:58:04.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.984+0000 7fc59b800640 1 -- 172.21.15.26:0/267987370 shutdown_connections 2024-01-15T17:58:04.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.984+0000 7fc59b800640 1 --2- 172.21.15.26:0/267987370 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59415f9c0 0x7fc5940cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.984+0000 7fc59b800640 1 --2- 172.21.15.26:0/267987370 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 0x7fc5940cf290 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.984+0000 7fc59b800640 1 --2- 172.21.15.26:0/267987370 >> v2:172.21.15.124:3300/0 conn(0x7fc59413df80 0x7fc59415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.984+0000 7fc59b800640 1 -- 172.21.15.26:0/267987370 >> 172.21.15.26:0/267987370 conn(0x7fc59406a800 msgr2=0x7fc59406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:04.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.984+0000 7fc59b800640 1 -- 172.21.15.26:0/267987370 shutdown_connections 2024-01-15T17:58:04.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.984+0000 7fc59b800640 1 -- 172.21.15.26:0/267987370 wait complete. 2024-01-15T17:58:04.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.985+0000 7fc59b800640 1 Processor -- start 2024-01-15T17:58:04.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.986+0000 7fc59b800640 1 -- start start 2024-01-15T17:58:04.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc59b800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59413df80 0x7fc5941f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc59b800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 0x7fc59415d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc59b800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc59415f9c0 0x7fc5940aab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc59b800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941f3220 con 0x7fc59413e360 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc59b800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59413d330 con 0x7fc59415f9c0 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc59b800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940ab050 con 0x7fc59413df80 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc58be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 0x7fc59415d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc58be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 0x7fc59415d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40692/0 (socket says 172.21.15.26:40692) 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc58be00640 1 -- 172.21.15.26:0/2641792284 learned_addr learned my addr 172.21.15.26:0/2641792284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc598e00640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59413df80 0x7fc5941f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc599800640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc59415f9c0 0x7fc5940aab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc58be00640 1 -- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59413df80 msgr2=0x7fc5941f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc58be00640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59413df80 0x7fc5941f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc58be00640 1 -- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc59415f9c0 msgr2=0x7fc5940aab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc58be00640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc59415f9c0 0x7fc5940aab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc58be00640 1 -- 172.21.15.26:0/2641792284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c0094d0 con 0x7fc59413e360 2024-01-15T17:58:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.987+0000 7fc599800640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc59415f9c0 0x7fc5940aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.988+0000 7fc58be00640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 0x7fc59415d260 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02b940 tx=0x7fc57c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.988+0000 7fc591600640 1 -- 172.21.15.26:0/2641792284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c03ad30 con 0x7fc59413e360 2024-01-15T17:58:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.988+0000 7fc591600640 1 -- 172.21.15.26:0/2641792284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c0443f0 con 0x7fc59413e360 2024-01-15T17:58:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.988+0000 7fc591600640 1 -- 172.21.15.26:0/2641792284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0447a0 con 0x7fc59413e360 2024-01-15T17:58:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.988+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5941f74f0 con 0x7fc59413e360 2024-01-15T17:58:04.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.988+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5941f6f30 con 0x7fc59413e360 2024-01-15T17:58:04.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.990+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5941f3360 con 0x7fc59413e360 2024-01-15T17:58:04.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.990+0000 7fc591600640 1 -- 172.21.15.26:0/2641792284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fc57c04d450 con 0x7fc59413e360 2024-01-15T17:58:04.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.991+0000 7fc591600640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc56c06b550 0x7fc56c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:04.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.991+0000 7fc591600640 1 -- 172.21.15.26:0/2641792284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fc57c0ac350 con 0x7fc59413e360 2024-01-15T17:58:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.992+0000 7fc598e00640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc56c06b550 0x7fc56c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:04.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.993+0000 7fc598e00640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc56c06b550 0x7fc56c06da10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc584006fd0 tx=0x7fc584008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:04.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:04.996+0000 7fc591600640 1 -- 172.21.15.26:0/2641792284 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5941f3360 con 0x7fc59413e360 2024-01-15T17:58:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.209+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc5941f3360 con 0x7fc59413e360 2024-01-15T17:58:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.210+0000 7fc591600640 1 -- 172.21.15.26:0/2641792284 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v39) v1 ==== 80+0+124 (secure 0 0 0) 0x7fc57c035090 con 0x7fc59413e360 2024-01-15T17:58:05.213 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6836/2548050157 2024-01-16T17:58:00.474566+0000 2024-01-15T17:58:05.213 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6834/2548050157 2024-01-16T17:58:00.474566+0000 2024-01-15T17:58:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T17:58:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.212+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc56c06b550 msgr2=0x7fc56c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.212+0000 7fc59b800640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc56c06b550 0x7fc56c06da10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc584006fd0 tx=0x7fc584008040 comp rx=0 tx=0).stop 2024-01-15T17:58:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.212+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 msgr2=0x7fc59415d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.212+0000 7fc59b800640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 0x7fc59415d260 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc57c02b940 tx=0x7fc57c0046c0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.213+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 shutdown_connections 2024-01-15T17:58:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.213+0000 7fc59b800640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc56c06b550 0x7fc56c06da10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.213+0000 7fc59b800640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc59415f9c0 0x7fc5940aab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.213+0000 7fc59b800640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc59413e360 0x7fc59415d260 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.213+0000 7fc59b800640 1 --2- 172.21.15.26:0/2641792284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc59413df80 0x7fc5941f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.213+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 >> 172.21.15.26:0/2641792284 conn(0x7fc59406a800 msgr2=0x7fc5940af230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.213+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 shutdown_connections 2024-01-15T17:58:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.213+0000 7fc59b800640 1 -- 172.21.15.26:0/2641792284 wait complete. 2024-01-15T17:58:05.234 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-01-15T17:58:05.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:05.235 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T17:58:05.325 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[49154]: fuse finished with error 0 and tester_r 0 2024-01-15T17:58:05.332 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:58:05.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:05.332 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T17:58:05.543 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:58:05.543 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:05.598 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:58:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.818+0000 7efd8d200640 1 Processor -- start 2024-01-15T17:58:05.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.818+0000 7efd8d200640 1 -- start start 2024-01-15T17:58:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.820+0000 7efd8d200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd88071410 0x7efd88146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.820+0000 7efd8d200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8806c7c0 0x7efd880717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.820+0000 7efd8d200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8806cc20 con 0x7efd88071d30 2024-01-15T17:58:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.820+0000 7efd8d200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8806cec0 con 0x7efd88071410 2024-01-15T17:58:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.820+0000 7efd8d200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8806d1a0 con 0x7efd8806c7c0 2024-01-15T17:58:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.821+0000 7efd87400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efd88071410 0x7efd88146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.821+0000 7efd87e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efd88071d30 0x7efd880b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39022/0 (socket says 172.21.15.26:39022) 2024-01-15T17:58:05.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.821+0000 7efd87e00640 1 -- 172.21.15.26:0/800265563 learned_addr learned my addr 172.21.15.26:0/800265563 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.821+0000 7efd7fe00640 1 --2- 172.21.15.26:0/800265563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8806c7c0 0x7efd880717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.822+0000 7efd86a00640 1 -- 172.21.15.26:0/800265563 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952279011 0 0) 0x7efd8806cc20 con 0x7efd88071d30 2024-01-15T17:58:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.822+0000 7efd86a00640 1 -- 172.21.15.26:0/800265563 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd68003520 con 0x7efd88071d30 2024-01-15T17:58:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.822+0000 7efd87400640 1 -- 172.21.15.26:0/800265563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8806c7c0 msgr2=0x7efd880717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:05.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.822+0000 7efd87400640 1 --2- 172.21.15.26:0/800265563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8806c7c0 0x7efd880717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.822+0000 7efd87400640 1 -- 172.21.15.26:0/800265563 >> v1:172.21.15.26:6789/0 conn(0x7efd88071d30 legacy=0x7efd880b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:05.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.822+0000 7efd87400640 1 -- 172.21.15.26:0/800265563 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8806d480 con 0x7efd88071410 2024-01-15T17:58:05.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.823+0000 7efd87400640 1 --2- 172.21.15.26:0/800265563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd88071410 0x7efd88146770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd780097a0 tx=0x7efd7802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4664d01bf14c75 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.824+0000 7efd86a00640 1 -- 172.21.15.26:0/800265563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78031830 con 0x7efd88071410 2024-01-15T17:58:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.824+0000 7efd86a00640 1 -- 172.21.15.26:0/800265563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd78031990 con 0x7efd88071410 2024-01-15T17:58:05.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.824+0000 7efd86a00640 1 -- 172.21.15.26:0/800265563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78031d20 con 0x7efd88071410 2024-01-15T17:58:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.824+0000 7efd8d200640 1 -- 172.21.15.26:0/800265563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd88071410 msgr2=0x7efd88146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.824+0000 7efd8d200640 1 --2- 172.21.15.26:0/800265563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd88071410 0x7efd88146770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7efd780097a0 tx=0x7efd7802b8d0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.825+0000 7efd8d200640 1 -- 172.21.15.26:0/800265563 shutdown_connections 2024-01-15T17:58:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.825+0000 7efd8d200640 1 --2- 172.21.15.26:0/800265563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd8806c7c0 0x7efd880717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.825+0000 7efd8d200640 1 --2- 172.21.15.26:0/800265563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd88071410 0x7efd88146770 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.825+0000 7efd8d200640 1 -- 172.21.15.26:0/800265563 >> 172.21.15.26:0/800265563 conn(0x7efd8806a800 msgr2=0x7efd8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.825+0000 7efd8d200640 1 -- 172.21.15.26:0/800265563 shutdown_connections 2024-01-15T17:58:05.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.825+0000 7efd8d200640 1 -- 172.21.15.26:0/800265563 wait complete. 2024-01-15T17:58:05.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.827+0000 7efd8d200640 1 Processor -- start 2024-01-15T17:58:05.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.827+0000 7efd8d200640 1 -- start start 2024-01-15T17:58:05.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd8d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8806c7c0 0x7efd8815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd8d200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88071410 0x7efd881306e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd8d200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd88071d30 0x7efd880c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd8d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8815d150 con 0x7efd8806c7c0 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd8d200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd88130c20 con 0x7efd88071410 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd8d200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8813d000 con 0x7efd88071d30 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd87400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8806c7c0 0x7efd8815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd87400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8806c7c0 0x7efd8815cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40704/0 (socket says 172.21.15.26:40704) 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd87400640 1 -- 172.21.15.26:0/1609185676 learned_addr learned my addr 172.21.15.26:0/1609185676 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd87400640 1 -- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd88071d30 msgr2=0x7efd880c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd87400640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd88071d30 0x7efd880c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd87400640 1 -- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88071410 msgr2=0x7efd881306e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd87400640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88071410 0x7efd881306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.828+0000 7efd87400640 1 -- 172.21.15.26:0/1609185676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd78009400 con 0x7efd8806c7c0 2024-01-15T17:58:05.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.829+0000 7efd87400640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8806c7c0 0x7efd8815cc10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efd78009890 tx=0x7efd780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.829+0000 7efd7f400640 1 -- 172.21.15.26:0/1609185676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7803a8f0 con 0x7efd8806c7c0 2024-01-15T17:58:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.829+0000 7efd7f400640 1 -- 172.21.15.26:0/1609185676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7803aa70 con 0x7efd8806c7c0 2024-01-15T17:58:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.829+0000 7efd7f400640 1 -- 172.21.15.26:0/1609185676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7803ae20 con 0x7efd8806c7c0 2024-01-15T17:58:05.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.829+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd881f3510 con 0x7efd8806c7c0 2024-01-15T17:58:05.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.829+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd881f3ad0 con 0x7efd8806c7c0 2024-01-15T17:58:05.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.830+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd88067450 con 0x7efd8806c7c0 2024-01-15T17:58:05.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.831+0000 7efd7f400640 1 -- 172.21.15.26:0/1609185676 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7efd7804d440 con 0x7efd8806c7c0 2024-01-15T17:58:05.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.832+0000 7efd7f400640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd6006b550 0x7efd6006da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:05.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.832+0000 7efd7f400640 1 -- 172.21.15.26:0/1609185676 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7efd780abf60 con 0x7efd8806c7c0 2024-01-15T17:58:05.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.833+0000 7efd7fe00640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd6006b550 0x7efd6006da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:05.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.834+0000 7efd7fe00640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd6006b550 0x7efd6006da10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd70006a90 tx=0x7efd70006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:05.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:05.836+0000 7efd7f400640 1 -- 172.21.15.26:0/1609185676 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd78076cb0 con 0x7efd8806c7c0 2024-01-15T17:58:06.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.038+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd880acd10 con 0x7efd8806c7c0 2024-01-15T17:58:06.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.039+0000 7efd7f400640 1 -- 172.21.15.26:0/1609185676 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13007 (secure 0 0 0) 0x7efd7807a2d0 con 0x7efd8806c7c0 2024-01-15T17:58:06.043 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:06.043 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":39,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:58:03.490741+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6836/2548050157":"2024-01-16T17:58:00.474566+0000","172.21.15.124:6834/2548050157":"2024-01-16T17:58:00.474566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:58:06.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd6006b550 msgr2=0x7efd6006da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd6006b550 0x7efd6006da10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd70006a90 tx=0x7efd70006a20 comp rx=0 tx=0).stop 2024-01-15T17:58:06.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8806c7c0 msgr2=0x7efd8815cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8806c7c0 0x7efd8815cc10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7efd78009890 tx=0x7efd780046c0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 shutdown_connections 2024-01-15T17:58:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efd88071d30 0x7efd880c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efd6006b550 0x7efd6006da10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efd88071410 0x7efd881306e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 --2- 172.21.15.26:0/1609185676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efd8806c7c0 0x7efd8815cc10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.042+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 >> 172.21.15.26:0/1609185676 conn(0x7efd8806a800 msgr2=0x7efd880b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.043+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 shutdown_connections 2024-01-15T17:58:06.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.043+0000 7efd8d200640 1 -- 172.21.15.26:0/1609185676 wait complete. 2024-01-15T17:58:06.066 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:58:06.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:58:06.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.285+0000 7f76fd000640 1 Processor -- start 2024-01-15T17:58:06.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.285+0000 7f76fd000640 1 -- start start 2024-01-15T17:58:06.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.287+0000 7f76fd000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f813df80 0x7f76f815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.287+0000 7f76fd000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f806c470 con 0x7f76f813e360 2024-01-15T17:58:06.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.287+0000 7f76fd000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f806c750 con 0x7f76f80cf9a0 2024-01-15T17:58:06.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.287+0000 7f76fd000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f806ca30 con 0x7f76f813df80 2024-01-15T17:58:06.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.288+0000 7f76ed200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f76f813e360 0x7f76f80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39024/0 (socket says 172.21.15.26:39024) 2024-01-15T17:58:06.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.288+0000 7f76ed200640 1 -- 172.21.15.26:0/3711833775 learned_addr learned my addr 172.21.15.26:0/3711833775 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.288+0000 7f76f5e00640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f813df80 0x7f76f815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.288+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154256919 0 0) 0x7f76f806c470 con 0x7f76f813e360 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.288+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76d8003520 con 0x7f76f813e360 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691312092 0 0) 0x7f76f806c750 con 0x7f76f80cf9a0 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76f806c470 con 0x7f76f80cf9a0 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2505833981 0 0) 0x7f76d8003520 con 0x7f76f813e360 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f813df80 msgr2=0x7f76f815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f813df80 0x7f76f815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 >> v1:172.21.15.124:6789/0 conn(0x7f76f80cf9a0 legacy=0x7f76f80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f806cd10 con 0x7f76f813e360 2024-01-15T17:58:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f76e0001670 con 0x7f76f813e360 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 >> v1:172.21.15.26:6789/0 conn(0x7f76f813e360 legacy=0x7f76f80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f813df80 0x7f76d8003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76d8004d00 0x7f76d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f806ca30 con 0x7f76d80040e0 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d800bac0 con 0x7f76d8004d00 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d800bc80 con 0x7f76f813df80 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76ed200640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.289+0000 7f76f5e00640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f813df80 0x7f76d8003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.290+0000 7f76ed200640 1 -- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f813df80 msgr2=0x7f76d8003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.290+0000 7f76ed200640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f813df80 0x7f76d8003b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.290+0000 7f76ed200640 1 -- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76d8004d00 msgr2=0x7f76d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.290+0000 7f76f6800640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76d8004d00 0x7f76d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.290+0000 7f76ed200640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76d8004d00 0x7f76d8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.290+0000 7f76ed200640 1 -- 172.21.15.26:0/3711833775 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d8003520 con 0x7f76d80040e0 2024-01-15T17:58:06.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.290+0000 7f76f6800640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76d8004d00 0x7f76d8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.291+0000 7f76ed200640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76d80047c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f76e0007f90 tx=0x7f76d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2300967c6b2d9b0b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.291+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0005ad0 con 0x7f76d80040e0 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.291+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e0005c30 con 0x7f76d80040e0 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.291+0000 7f76f5400640 1 -- 172.21.15.26:0/3711833775 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0007290 con 0x7f76d80040e0 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.292+0000 7f76fd000640 1 -- 172.21.15.26:0/3711833775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 msgr2=0x7f76d80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.292+0000 7f76fd000640 1 --2- 172.21.15.26:0/3711833775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76d80047c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f76e0007f90 tx=0x7f76d80057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.292+0000 7f76f6800640 1 -- 172.21.15.26:0/3711833775 reap_dead start 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.292+0000 7f76fd000640 1 -- 172.21.15.26:0/3711833775 shutdown_connections 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.292+0000 7f76fd000640 1 -- 172.21.15.26:0/3711833775 >> 172.21.15.26:0/3711833775 conn(0x7f76f806a800 msgr2=0x7f76f806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:06.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.292+0000 7f76fd000640 1 -- 172.21.15.26:0/3711833775 shutdown_connections 2024-01-15T17:58:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.292+0000 7f76fd000640 1 -- 172.21.15.26:0/3711833775 wait complete. 2024-01-15T17:58:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.294+0000 7f76fd000640 1 Processor -- start 2024-01-15T17:58:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.294+0000 7f76fd000640 1 -- start start 2024-01-15T17:58:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76fd000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76f8159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76fd000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f80b2cf0 0x7f76f80b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76fd000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76f81e5f40 0x7f76f81e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76fd000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f80b36b0 con 0x7f76d80040e0 2024-01-15T17:58:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76fd000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f81e6c90 con 0x7f76f81e5f40 2024-01-15T17:58:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76fd000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f81e6f70 con 0x7f76f80b2cf0 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76f5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76f8159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76f5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76f8159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40722/0 (socket says 172.21.15.26:40722) 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76f5e00640 1 -- 172.21.15.26:0/390032270 learned_addr learned my addr 172.21.15.26:0/390032270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76f5e00640 1 -- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f80b2cf0 msgr2=0x7f76f80b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76f5e00640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f80b2cf0 0x7f76f80b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76f5e00640 1 -- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76f81e5f40 msgr2=0x7f76f81e63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76f6800640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76f81e5f40 0x7f76f81e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76f5e00640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76f81e5f40 0x7f76f81e63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.295+0000 7f76f5e00640 1 -- 172.21.15.26:0/390032270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f81e7250 con 0x7f76d80040e0 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.296+0000 7f76f5e00640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76f8159990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f76e800f350 tx=0x7f76e800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.296+0000 7f76eea00640 1 -- 172.21.15.26:0/390032270 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8017600 con 0x7f76d80040e0 2024-01-15T17:58:06.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.296+0000 7f76eea00640 1 -- 172.21.15.26:0/390032270 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e8011040 con 0x7f76d80040e0 2024-01-15T17:58:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.296+0000 7f76eea00640 1 -- 172.21.15.26:0/390032270 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e80179c0 con 0x7f76d80040e0 2024-01-15T17:58:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.296+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f81e7530 con 0x7f76d80040e0 2024-01-15T17:58:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.296+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f8159ed0 con 0x7f76d80040e0 2024-01-15T17:58:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.297+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f80675d0 con 0x7f76d80040e0 2024-01-15T17:58:06.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.299+0000 7f76eea00640 1 -- 172.21.15.26:0/390032270 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f76e8017b20 con 0x7f76d80040e0 2024-01-15T17:58:06.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.300+0000 7f76eea00640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76d006b500 0x7f76d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.300+0000 7f76eea00640 1 -- 172.21.15.26:0/390032270 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f76e808a550 con 0x7f76d80040e0 2024-01-15T17:58:06.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.300+0000 7f76ed200640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76d006b500 0x7f76d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.301+0000 7f76ed200640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76d006b500 0x7f76d006d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76e0034000 tx=0x7f76e0007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:06.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.304+0000 7f76eea00640 1 -- 172.21.15.26:0/390032270 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76e8055250 con 0x7f76d80040e0 2024-01-15T17:58:06.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.502+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76f80cc220 con 0x7f76d80040e0 2024-01-15T17:58:06.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.503+0000 7f76eea00640 1 -- 172.21.15.26:0/390032270 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76e801c020 con 0x7f76d80040e0 2024-01-15T17:58:06.506 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:58:06.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76d006b500 msgr2=0x7f76d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76d006b500 0x7f76d006d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76e0034000 tx=0x7f76e0007000 comp rx=0 tx=0).stop 2024-01-15T17:58:06.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 msgr2=0x7f76f8159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76f8159990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f76e800f350 tx=0x7f76e800f810 comp rx=0 tx=0).stop 2024-01-15T17:58:06.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 shutdown_connections 2024-01-15T17:58:06.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76f80b2cf0 0x7f76f80b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f76d006b500 0x7f76d006d9c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f76f81e5f40 0x7f76f81e63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 --2- 172.21.15.26:0/390032270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76d80040e0 0x7f76f8159990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 >> 172.21.15.26:0/390032270 conn(0x7f76f806a800 msgr2=0x7f76f80ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:06.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 shutdown_connections 2024-01-15T17:58:06.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.505+0000 7f76fd000640 1 -- 172.21.15.26:0/390032270 wait complete. 2024-01-15T17:58:06.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T17:58:06.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.754+0000 7f278d600640 1 Processor -- start 2024-01-15T17:58:06.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.755+0000 7f278d600640 1 -- start start 2024-01-15T17:58:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.756+0000 7f278d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f27880cc500 0x7f27880cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.756+0000 7f278d600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2788070e50 con 0x7f278806d160 2024-01-15T17:58:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.756+0000 7f278d600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2788144020 con 0x7f2788143c40 2024-01-15T17:58:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.756+0000 7f278d600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27880723c0 con 0x7f27880cc500 2024-01-15T17:58:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.757+0000 7f2787400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f27880cc500 0x7f27880cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.757+0000 7f2787400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f27880cc500 0x7f27880cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40734/0 (socket says 172.21.15.26:40734) 2024-01-15T17:58:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.757+0000 7f2787400640 1 -- 172.21.15.26:0/1076311120 learned_addr learned my addr 172.21.15.26:0/1076311120 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:06.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.757+0000 7f2787400640 1 -- 172.21.15.26:0/1076311120 >> v1:172.21.15.124:6789/0 conn(0x7f2788143c40 legacy=0x7f2788071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:06.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.757+0000 7f2787400640 1 -- 172.21.15.26:0/1076311120 >> v1:172.21.15.26:6789/0 conn(0x7f278806d160 legacy=0x7f278806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.757+0000 7f2787400640 1 -- 172.21.15.26:0/1076311120 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2788071e00 con 0x7f27880cc500 2024-01-15T17:58:06.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.758+0000 7f2787400640 1 --2- 172.21.15.26:0/1076311120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27880cc500 0x7f27880cc990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2770009870 tx=0x7f277002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c8265a377cc440 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:06.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.759+0000 7f2786a00640 1 -- 172.21.15.26:0/1076311120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770031870 con 0x7f27880cc500 2024-01-15T17:58:06.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.759+0000 7f2786a00640 1 -- 172.21.15.26:0/1076311120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27700319d0 con 0x7f27880cc500 2024-01-15T17:58:06.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.759+0000 7f2786a00640 1 -- 172.21.15.26:0/1076311120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2770031d60 con 0x7f27880cc500 2024-01-15T17:58:06.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.759+0000 7f278d600640 1 -- 172.21.15.26:0/1076311120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27880cc500 msgr2=0x7f27880cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.759+0000 7f278d600640 1 --2- 172.21.15.26:0/1076311120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27880cc500 0x7f27880cc990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2770009870 tx=0x7f277002b910 comp rx=0 tx=0).stop 2024-01-15T17:58:06.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.760+0000 7f278d600640 1 -- 172.21.15.26:0/1076311120 shutdown_connections 2024-01-15T17:58:06.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.760+0000 7f278d600640 1 --2- 172.21.15.26:0/1076311120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27880cc500 0x7f27880cc990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.760+0000 7f278d600640 1 -- 172.21.15.26:0/1076311120 >> 172.21.15.26:0/1076311120 conn(0x7f278806a800 msgr2=0x7f278806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:06.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.760+0000 7f278d600640 1 -- 172.21.15.26:0/1076311120 shutdown_connections 2024-01-15T17:58:06.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.760+0000 7f278d600640 1 -- 172.21.15.26:0/1076311120 wait complete. 2024-01-15T17:58:06.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.762+0000 7f278d600640 1 Processor -- start 2024-01-15T17:58:06.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.762+0000 7f278d600640 1 -- start start 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f278d600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f278806d160 0x7f27880a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f278d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27880cc500 0x7f278813e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f278d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2788143c40 0x7f278813e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f278d600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278814a3a0 con 0x7f2788143c40 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f278d600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27881f1e10 con 0x7f278806d160 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f278d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278814a0c0 con 0x7f27880cc500 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f278cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2788143c40 0x7f278813e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f278cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2788143c40 0x7f278813e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40740/0 (socket says 172.21.15.26:40740) 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f278cc00640 1 -- 172.21.15.26:0/2310832297 learned_addr learned my addr 172.21.15.26:0/2310832297 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.763+0000 7f2787e00640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f278806d160 0x7f27880a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f2787400640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27880cc500 0x7f278813e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f278cc00640 1 -- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27880cc500 msgr2=0x7f278813e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f278cc00640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27880cc500 0x7f278813e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f278cc00640 1 -- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f278806d160 msgr2=0x7f27880a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f278cc00640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f278806d160 0x7f27880a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f278cc00640 1 -- 172.21.15.26:0/2310832297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700094d0 con 0x7f2788143c40 2024-01-15T17:58:06.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f2787400640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27880cc500 0x7f278813e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:06.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f278cc00640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2788143c40 0x7f278813e610 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f278000c390 tx=0x7f278000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f277fe00640 1 -- 172.21.15.26:0/2310832297 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27800052d0 con 0x7f2788143c40 2024-01-15T17:58:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f277fe00640 1 -- 172.21.15.26:0/2310832297 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2780005430 con 0x7f2788143c40 2024-01-15T17:58:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f277fe00640 1 -- 172.21.15.26:0/2310832297 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780003810 con 0x7f2788143c40 2024-01-15T17:58:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.764+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27881474e0 con 0x7f2788143c40 2024-01-15T17:58:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.765+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2788147aa0 con 0x7f2788143c40 2024-01-15T17:58:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.766+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2788147340 con 0x7f2788143c40 2024-01-15T17:58:06.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.767+0000 7f277fe00640 1 -- 172.21.15.26:0/2310832297 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f2780003970 con 0x7f2788143c40 2024-01-15T17:58:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.768+0000 7f277fe00640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f276006b500 0x7f276006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:06.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.768+0000 7f277fe00640 1 -- 172.21.15.26:0/2310832297 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f2780089170 con 0x7f2788143c40 2024-01-15T17:58:06.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.769+0000 7f2787e00640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f276006b500 0x7f276006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:06.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.770+0000 7f2787e00640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f276006b500 0x7f276006d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2778005520 tx=0x7f2778009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:06.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.772+0000 7f277fe00640 1 -- 172.21.15.26:0/2310832297 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2788147340 con 0x7f2788143c40 2024-01-15T17:58:06.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.980+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2788147340 con 0x7f2788143c40 2024-01-15T17:58:06.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.981+0000 7f277fe00640 1 -- 172.21.15.26:0/2310832297 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v39) v1 ==== 80+0+124 (secure 0 0 0) 0x7f2788147340 con 0x7f2788143c40 2024-01-15T17:58:06.984 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6836/2548050157 2024-01-16T17:58:00.474566+0000 2024-01-15T17:58:06.984 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6834/2548050157 2024-01-16T17:58:00.474566+0000 2024-01-15T17:58:06.984 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.983+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f276006b500 msgr2=0x7f276006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.983+0000 7f278d600640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f276006b500 0x7f276006d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2778005520 tx=0x7f2778009040 comp rx=0 tx=0).stop 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.983+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2788143c40 msgr2=0x7f278813e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.983+0000 7f278d600640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2788143c40 0x7f278813e610 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f278000c390 tx=0x7f278000c850 comp rx=0 tx=0).stop 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.984+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 shutdown_connections 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.984+0000 7f278d600640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f27880cc500 0x7f278813e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.984+0000 7f278d600640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f276006b500 0x7f276006d9c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.984+0000 7f278d600640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2788143c40 0x7f278813e610 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.984+0000 7f278d600640 1 --2- 172.21.15.26:0/2310832297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f278806d160 0x7f27880a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:06.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.984+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 >> 172.21.15.26:0/2310832297 conn(0x7f278806a800 msgr2=0x7f27880aff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:06.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.984+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 shutdown_connections 2024-01-15T17:58:06.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:06.984+0000 7f278d600640 1 -- 172.21.15.26:0/2310832297 wait complete. 2024-01-15T17:58:07.008 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi124.front.sepia.ceph.com... 2024-01-15T17:58:07.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:07.008 DEBUG:teuthology.orchestra.run.smithi124:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T17:58:07.095 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[43154]: fuse finished with error 0 and tester_r 0 2024-01-15T17:58:07.101 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:58:07.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:07.102 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T17:58:07.275 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:58:07.275 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:07.336 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T17:58:07.336 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T17:58:07.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:07.336 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T17:58:07.390 INFO:teuthology.orchestra.run.smithi026.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T17:58:07.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:58:07.392 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:58:07.392 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:07.448 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T17:58:07.449 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T17:58:07.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:07.449 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T17:58:07.499 INFO:teuthology.orchestra.run.smithi124.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T17:58:07.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T17:58:07.501 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:58:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.732+0000 7f5f0aa00640 1 Processor -- start 2024-01-15T17:58:07.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.732+0000 7f5f0aa00640 1 -- start start 2024-01-15T17:58:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.733+0000 7f5f0aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5f040cf9a0 0x7f5f04166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:07.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.733+0000 7f5f0aa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0415c5f0 con 0x7f5f0413e360 2024-01-15T17:58:07.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.733+0000 7f5f0aa00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0406c470 con 0x7f5f0413df80 2024-01-15T17:58:07.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.733+0000 7f5f0aa00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0406c750 con 0x7f5f040cf9a0 2024-01-15T17:58:07.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.733+0000 7f5efb200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5f0413e360 0x7f5f040cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39054/0 (socket says 172.21.15.26:39054) 2024-01-15T17:58:07.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.734+0000 7f5efb200640 1 -- 172.21.15.26:0/3347078550 learned_addr learned my addr 172.21.15.26:0/3347078550 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:07.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.733+0000 7f5f08a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5f040cf9a0 0x7f5f04166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:07.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.734+0000 7f5f08a00640 1 -- 172.21.15.26:0/3347078550 >> v1:172.21.15.124:6789/0 conn(0x7f5f0413df80 legacy=0x7f5f0415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:07.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.734+0000 7f5f08a00640 1 -- 172.21.15.26:0/3347078550 >> v1:172.21.15.26:6789/0 conn(0x7f5f0413e360 legacy=0x7f5f040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.734+0000 7f5f08a00640 1 -- 172.21.15.26:0/3347078550 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0406ca30 con 0x7f5f040cf9a0 2024-01-15T17:58:07.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.736+0000 7f5f08a00640 1 --2- 172.21.15.26:0/3347078550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f040cf9a0 0x7f5f04166030 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009d90 tx=0x7f5efc02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7b0adb4621e40e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.736+0000 7f5f03400640 1 -- 172.21.15.26:0/3347078550 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc02ccf0 con 0x7f5f040cf9a0 2024-01-15T17:58:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.736+0000 7f5f03400640 1 -- 172.21.15.26:0/3347078550 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc02ce50 con 0x7f5f040cf9a0 2024-01-15T17:58:07.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.736+0000 7f5f03400640 1 -- 172.21.15.26:0/3347078550 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc034660 con 0x7f5f040cf9a0 2024-01-15T17:58:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.737+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3347078550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f040cf9a0 msgr2=0x7f5f04166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.737+0000 7f5f0aa00640 1 --2- 172.21.15.26:0/3347078550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f040cf9a0 0x7f5f04166030 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009d90 tx=0x7f5efc02bf40 comp rx=0 tx=0).stop 2024-01-15T17:58:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.737+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3347078550 shutdown_connections 2024-01-15T17:58:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.737+0000 7f5f0aa00640 1 --2- 172.21.15.26:0/3347078550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f040cf9a0 0x7f5f04166030 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.737+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3347078550 >> 172.21.15.26:0/3347078550 conn(0x7f5f0406a800 msgr2=0x7f5f0406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.737+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3347078550 shutdown_connections 2024-01-15T17:58:07.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.737+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3347078550 wait complete. 2024-01-15T17:58:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.738+0000 7f5f0aa00640 1 Processor -- start 2024-01-15T17:58:07.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.739+0000 7f5f0aa00640 1 -- start start 2024-01-15T17:58:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f0aa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f040cf9a0 0x7f5f041588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f0aa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f0413df80 0x7f5f04135750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f0aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0413e360 0x7f5f040702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f0aa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f04158de0 con 0x7f5f0413e360 2024-01-15T17:58:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f0aa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f04135c90 con 0x7f5f040cf9a0 2024-01-15T17:58:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f0aa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f04070800 con 0x7f5f0413df80 2024-01-15T17:58:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f08a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0413e360 0x7f5f040702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f08a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0413e360 0x7f5f040702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40764/0 (socket says 172.21.15.26:40764) 2024-01-15T17:58:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f08a00640 1 -- 172.21.15.26:0/3426436761 learned_addr learned my addr 172.21.15.26:0/3426436761 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5efb200640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f0413df80 0x7f5f04135750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f03e00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f040cf9a0 0x7f5f041588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f08a00640 1 -- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f0413df80 msgr2=0x7f5f04135750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f08a00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f0413df80 0x7f5f04135750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f08a00640 1 -- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f040cf9a0 msgr2=0x7f5f041588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f08a00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f040cf9a0 0x7f5f041588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.740+0000 7f5f08a00640 1 -- 172.21.15.26:0/3426436761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5efc009a70 con 0x7f5f0413e360 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.741+0000 7f5f08a00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0413e360 0x7f5f040702c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5efc037040 tx=0x7f5efc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.741+0000 7f5f00c00640 1 -- 172.21.15.26:0/3426436761 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc03d600 con 0x7f5f0413e360 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.741+0000 7f5f00c00640 1 -- 172.21.15.26:0/3426436761 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc03d760 con 0x7f5f0413e360 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.741+0000 7f5f00c00640 1 -- 172.21.15.26:0/3426436761 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc03daf0 con 0x7f5f0413e360 2024-01-15T17:58:07.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.741+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f04070aa0 con 0x7f5f0413e360 2024-01-15T17:58:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.741+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f04071060 con 0x7f5f0413e360 2024-01-15T17:58:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.743+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f040675d0 con 0x7f5f0413e360 2024-01-15T17:58:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.743+0000 7f5f00c00640 1 -- 172.21.15.26:0/3426436761 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f5efc03b040 con 0x7f5f0413e360 2024-01-15T17:58:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.744+0000 7f5f00c00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5edc06b500 0x7f5edc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.744+0000 7f5f00c00640 1 -- 172.21.15.26:0/3426436761 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f5efc0ac360 con 0x7f5f0413e360 2024-01-15T17:58:07.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.745+0000 7f5f03e00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5edc06b500 0x7f5edc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:07.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.746+0000 7f5f03e00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5edc06b500 0x7f5edc06d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4000f60 tx=0x7f5ef4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:07.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.749+0000 7f5f00c00640 1 -- 172.21.15.26:0/3426436761 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5efc0771e0 con 0x7f5f0413e360 2024-01-15T17:58:07.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.984+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f0406df00 con 0x7f5f0413e360 2024-01-15T17:58:07.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.985+0000 7f5f00c00640 1 -- 172.21.15.26:0/3426436761 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f5efc04ee90 con 0x7f5f0413e360 2024-01-15T17:58:07.988 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:07.989 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1651915477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":1651915477},{"type":"v1","addr":"172.21.15.124:6837","nonce":1651915477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2848117329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2848117329},{"type":"v1","addr":"172.21.15.124:6836","nonce":2848117329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/506946946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":506946946},{"type":"v1","addr":"172.21.15.26:6833","nonce":506946946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3855677701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3855677701},{"type":"v1","addr":"172.21.15.26:6835","nonce":3855677701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2024-01-15T17:58:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 19 2024-01-15T17:58:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5edc06b500 msgr2=0x7f5edc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5edc06b500 0x7f5edc06d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4000f60 tx=0x7f5ef4009040 comp rx=0 tx=0).stop 2024-01-15T17:58:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0413e360 msgr2=0x7f5f040702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0413e360 0x7f5f040702c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5efc037040 tx=0x7f5efc004a40 comp rx=0 tx=0).stop 2024-01-15T17:58:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 shutdown_connections 2024-01-15T17:58:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f0413df80 0x7f5f04135750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5edc06b500 0x7f5edc06d9c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f0413e360 0x7f5f040702c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 --2- 172.21.15.26:0/3426436761 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f040cf9a0 0x7f5f041588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 >> 172.21.15.26:0/3426436761 conn(0x7f5f0406a800 msgr2=0x7f5f0406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 shutdown_connections 2024-01-15T17:58:07.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:07.988+0000 7f5f0aa00640 1 -- 172.21.15.26:0/3426436761 wait complete. 2024-01-15T17:58:08.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T17:58:08.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.233+0000 7f59f7800640 1 Processor -- start 2024-01-15T17:58:08.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.233+0000 7f59f7800640 1 -- start start 2024-01-15T17:58:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.235+0000 7f59f7800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f0143ae0 0x7f59f00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.235+0000 7f59f7800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f59f00cf7d0 0x7f59f015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.236+0000 7f59f7800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f00cfc30 con 0x7f59f015e1e0 2024-01-15T17:58:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.236+0000 7f59f7800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f00b1ee0 con 0x7f59f00cf7d0 2024-01-15T17:58:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.236+0000 7f59f7800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00b2780 con 0x7f59f0143ae0 2024-01-15T17:58:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.236+0000 7f59f5800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f59f015e1e0 0x7f59f015a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39066/0 (socket says 172.21.15.26:39066) 2024-01-15T17:58:08.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.236+0000 7f59f5800640 1 -- 172.21.15.26:0/451613124 learned_addr learned my addr 172.21.15.26:0/451613124 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.236+0000 7f59e7e00640 1 --2- 172.21.15.26:0/451613124 >> v2:172.21.15.124:3300/0 conn(0x7f59f00cf7d0 0x7f59f015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.236+0000 7f59f4e00640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f0143ae0 0x7f59f00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.236+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139962500 0 0) 0x7f59f00cfc30 con 0x7f59f015e1e0 2024-01-15T17:58:08.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.236+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59d0005730 con 0x7f59f015e1e0 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1872003715 0 0) 0x7f59d0005730 con 0x7f59f015e1e0 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f0143ae0 msgr2=0x7f59f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f0143ae0 0x7f59f00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 >> v2:172.21.15.124:3300/0 conn(0x7f59f00cf7d0 msgr2=0x7f59f015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 --2- 172.21.15.26:0/451613124 >> v2:172.21.15.124:3300/0 conn(0x7f59f00cf7d0 0x7f59f015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f015e990 con 0x7f59f015e1e0 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f59e80019f0 con 0x7f59f015e1e0 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 >> v1:172.21.15.26:6789/0 conn(0x7f59f015e1e0 legacy=0x7f59f015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f0143ae0 0x7f59d0005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006100 0x7f59d00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d0006f10 0x7f59d0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00b1ee0 con 0x7f59d0006f10 2024-01-15T17:58:08.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00b2780 con 0x7f59d0006100 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d000db90 con 0x7f59f0143ae0 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59f5800640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d0006f10 0x7f59d0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.237+0000 7f59f4e00640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f0143ae0 0x7f59d0005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.238+0000 7f59e7e00640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006100 0x7f59d00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.238+0000 7f59f5800640 1 -- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f0143ae0 msgr2=0x7f59d0005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.238+0000 7f59f5800640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f0143ae0 0x7f59d0005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.238+0000 7f59f5800640 1 -- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006100 msgr2=0x7f59d00069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.238+0000 7f59f5800640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006100 0x7f59d00069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.238+0000 7f59f5800640 1 -- 172.21.15.26:0/451613124 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0005730 con 0x7f59d0006f10 2024-01-15T17:58:08.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.238+0000 7f59e7e00640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006100 0x7f59d00069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.239+0000 7f59f5800640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d0006f10 0x7f59d0007390 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f59e800a830 tx=0x7f59d0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca740c87e831857 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.239+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8034630 con 0x7f59d0006f10 2024-01-15T17:58:08.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.239+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e802f070 con 0x7f59d0006f10 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.239+0000 7f59efe00640 1 -- 172.21.15.26:0/451613124 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8034a30 con 0x7f59d0006f10 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.240+0000 7f59f7800640 1 -- 172.21.15.26:0/451613124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d0006f10 msgr2=0x7f59d0007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.240+0000 7f59f7800640 1 --2- 172.21.15.26:0/451613124 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59d0006f10 0x7f59d0007390 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f59e800a830 tx=0x7f59d0007a00 comp rx=0 tx=0).stop 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.240+0000 7f59f5800640 1 -- 172.21.15.26:0/451613124 reap_dead start 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.240+0000 7f59f7800640 1 -- 172.21.15.26:0/451613124 shutdown_connections 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.240+0000 7f59f7800640 1 -- 172.21.15.26:0/451613124 >> 172.21.15.26:0/451613124 conn(0x7f59f006a800 msgr2=0x7f59f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.240+0000 7f59f7800640 1 -- 172.21.15.26:0/451613124 shutdown_connections 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.240+0000 7f59f7800640 1 -- 172.21.15.26:0/451613124 wait complete. 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.242+0000 7f59f7800640 1 Processor -- start 2024-01-15T17:58:08.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.242+0000 7f59f7800640 1 -- start start 2024-01-15T17:58:08.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.243+0000 7f59f7800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006f10 0x7f59f015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.243+0000 7f59f7800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f015acd0 0x7f59f006c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.243+0000 7f59f7800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59f015b150 0x7f59f006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.243+0000 7f59f7800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f015d5e0 con 0x7f59f015b150 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.243+0000 7f59f7800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f015b660 con 0x7f59d0006f10 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.243+0000 7f59f7800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00cd6a0 con 0x7f59f015acd0 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.243+0000 7f59f5800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59f015b150 0x7f59f006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.243+0000 7f59f5800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59f015b150 0x7f59f006cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40776/0 (socket says 172.21.15.26:40776) 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.243+0000 7f59f5800640 1 -- 172.21.15.26:0/3342160694 learned_addr learned my addr 172.21.15.26:0/3342160694 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59e7e00640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f015acd0 0x7f59f006c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59f4e00640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006f10 0x7f59f015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59f5800640 1 -- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f015acd0 msgr2=0x7f59f006c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59f5800640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f015acd0 0x7f59f006c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59f5800640 1 -- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006f10 msgr2=0x7f59f015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59f5800640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006f10 0x7f59f015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59f5800640 1 -- 172.21.15.26:0/3342160694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00cd980 con 0x7f59f015b150 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59e7e00640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f015acd0 0x7f59f006c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59f4e00640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006f10 0x7f59f015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59f5800640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59f015b150 0x7f59f006cc90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f59e802cbe0 tx=0x7f59f00b26f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59ed600640 1 -- 172.21.15.26:0/3342160694 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8001fc0 con 0x7f59f015b150 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59ed600640 1 -- 172.21.15.26:0/3342160694 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e8034430 con 0x7f59f015b150 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f015d5e0 con 0x7f59f015b150 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.244+0000 7f59ed600640 1 -- 172.21.15.26:0/3342160694 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e80347a0 con 0x7f59f015b150 2024-01-15T17:58:08.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.245+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f00cdf40 con 0x7f59f015b150 2024-01-15T17:58:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.246+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f00675d0 con 0x7f59f015b150 2024-01-15T17:58:08.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.247+0000 7f59ed600640 1 -- 172.21.15.26:0/3342160694 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f59e8031030 con 0x7f59f015b150 2024-01-15T17:58:08.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.248+0000 7f59ed600640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f59c806b500 0x7f59c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.248+0000 7f59ed600640 1 -- 172.21.15.26:0/3342160694 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f59e80ab220 con 0x7f59f015b150 2024-01-15T17:58:08.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.248+0000 7f59f4e00640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f59c806b500 0x7f59c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.250+0000 7f59f4e00640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f59c806b500 0x7f59c806d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f59f015ac60 tx=0x7f59e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:08.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.252+0000 7f59ed600640 1 -- 172.21.15.26:0/3342160694 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59e8076080 con 0x7f59f015b150 2024-01-15T17:58:08.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.447+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f59f00b1500 con 0x7f59f015b150 2024-01-15T17:58:08.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.452+0000 7f59ed600640 1 -- 172.21.15.26:0/3342160694 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f59e80796a0 con 0x7f59f015b150 2024-01-15T17:58:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f59c806b500 msgr2=0x7f59c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f59c806b500 0x7f59c806d9c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f59f015ac60 tx=0x7f59e0008040 comp rx=0 tx=0).stop 2024-01-15T17:58:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59f015b150 msgr2=0x7f59f006cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59f015b150 0x7f59f006cc90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f59e802cbe0 tx=0x7f59f00b26f0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 shutdown_connections 2024-01-15T17:58:08.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59f015acd0 0x7f59f006c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f59c806b500 0x7f59c806d9c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59f015b150 0x7f59f006cc90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 --2- 172.21.15.26:0/3342160694 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59d0006f10 0x7f59f015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 >> 172.21.15.26:0/3342160694 conn(0x7f59f006a800 msgr2=0x7f59f00cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 shutdown_connections 2024-01-15T17:58:08.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.455+0000 7f59f7800640 1 -- 172.21.15.26:0/3342160694 wait complete. 2024-01-15T17:58:08.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T17:58:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.690+0000 7f8349600640 1 Processor -- start 2024-01-15T17:58:08.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.690+0000 7f8349600640 1 -- start start 2024-01-15T17:58:08.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.692+0000 7f8349600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83440adf20 0x7f8344070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.692+0000 7f8349600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f834415bf10 0x7f83440cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.692+0000 7f8349600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834415f4e0 con 0x7f83440cc980 2024-01-15T17:58:08.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.692+0000 7f8349600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834415f970 con 0x7f834415bf10 2024-01-15T17:58:08.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.692+0000 7f8349600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83440cce10 con 0x7f83440adf20 2024-01-15T17:58:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.692+0000 7f8348c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f83440cc980 0x7f8344143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39082/0 (socket says 172.21.15.26:39082) 2024-01-15T17:58:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.692+0000 7f8348c00640 1 -- 172.21.15.26:0/2024681266 learned_addr learned my addr 172.21.15.26:0/2024681266 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.692+0000 7f833b200640 1 --2- 172.21.15.26:0/2024681266 >> v2:172.21.15.26:3300/0 conn(0x7f834415bf10 0x7f83440cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.693+0000 7f8343e00640 1 --2- 172.21.15.26:0/2024681266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83440adf20 0x7f8344070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.693+0000 7f8343400640 1 -- 172.21.15.26:0/2024681266 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439996799 0 0) 0x7f834415f4e0 con 0x7f83440cc980 2024-01-15T17:58:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.693+0000 7f8343400640 1 -- 172.21.15.26:0/2024681266 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8324003520 con 0x7f83440cc980 2024-01-15T17:58:08.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.693+0000 7f833b200640 1 -- 172.21.15.26:0/2024681266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83440adf20 msgr2=0x7f8344070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:08.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.693+0000 7f833b200640 1 --2- 172.21.15.26:0/2024681266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83440adf20 0x7f8344070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.693+0000 7f833b200640 1 -- 172.21.15.26:0/2024681266 >> v1:172.21.15.26:6789/0 conn(0x7f83440cc980 legacy=0x7f8344143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.693+0000 7f833b200640 1 -- 172.21.15.26:0/2024681266 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8344071130 con 0x7f834415bf10 2024-01-15T17:58:08.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.694+0000 7f833b200640 1 --2- 172.21.15.26:0/2024681266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834415bf10 0x7f83440cc440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f832c0097a0 tx=0x7f832c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e898e986fbb756bf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8343400640 1 -- 172.21.15.26:0/2024681266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c031780 con 0x7f834415bf10 2024-01-15T17:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8343400640 1 -- 172.21.15.26:0/2024681266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c0318e0 con 0x7f834415bf10 2024-01-15T17:58:08.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8343400640 1 -- 172.21.15.26:0/2024681266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c031c70 con 0x7f834415bf10 2024-01-15T17:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8349600640 1 -- 172.21.15.26:0/2024681266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834415bf10 msgr2=0x7f83440cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8349600640 1 --2- 172.21.15.26:0/2024681266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834415bf10 0x7f83440cc440 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f832c0097a0 tx=0x7f832c02b820 comp rx=0 tx=0).stop 2024-01-15T17:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8349600640 1 -- 172.21.15.26:0/2024681266 shutdown_connections 2024-01-15T17:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8349600640 1 --2- 172.21.15.26:0/2024681266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f834415bf10 0x7f83440cc440 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8349600640 1 --2- 172.21.15.26:0/2024681266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83440adf20 0x7f8344070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8349600640 1 -- 172.21.15.26:0/2024681266 >> 172.21.15.26:0/2024681266 conn(0x7f834406a7d0 msgr2=0x7f834406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.695+0000 7f8349600640 1 -- 172.21.15.26:0/2024681266 shutdown_connections 2024-01-15T17:58:08.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.696+0000 7f8349600640 1 -- 172.21.15.26:0/2024681266 wait complete. 2024-01-15T17:58:08.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.697+0000 7f8349600640 1 Processor -- start 2024-01-15T17:58:08.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.697+0000 7f8349600640 1 -- start start 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.698+0000 7f8349600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83440adf20 0x7f83441f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.698+0000 7f8349600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83440cc980 0x7f834414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.698+0000 7f8349600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834415bf10 0x7f8344143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.698+0000 7f8349600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834413c3d0 con 0x7f83440adf20 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.698+0000 7f8349600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83441f6dc0 con 0x7f83440cc980 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.698+0000 7f8349600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834413bed0 con 0x7f834415bf10 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8343e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83440adf20 0x7f83441f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8343e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83440adf20 0x7f83441f6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40784/0 (socket says 172.21.15.26:40784) 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8343e00640 1 -- 172.21.15.26:0/1775898396 learned_addr learned my addr 172.21.15.26:0/1775898396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f833b200640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83440cc980 0x7f834414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8348c00640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834415bf10 0x7f8344143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8343e00640 1 -- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834415bf10 msgr2=0x7f8344143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8343e00640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834415bf10 0x7f8344143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8343e00640 1 -- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83440cc980 msgr2=0x7f834414dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8343e00640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83440cc980 0x7f834414dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8343e00640 1 -- 172.21.15.26:0/1775898396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c009400 con 0x7f83440adf20 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f833b200640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83440cc980 0x7f834414dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.699+0000 7f8343e00640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83440adf20 0x7f83441f6880 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f833400efc0 tx=0x7f833400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.700+0000 7f8340c00640 1 -- 172.21.15.26:0/1775898396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334009b80 con 0x7f83440adf20 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.700+0000 7f8340c00640 1 -- 172.21.15.26:0/1775898396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833400f040 con 0x7f83440adf20 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.700+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834414e780 con 0x7f83440adf20 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.700+0000 7f8340c00640 1 -- 172.21.15.26:0/1775898396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83340146a0 con 0x7f83440adf20 2024-01-15T17:58:08.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.700+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834414ed40 con 0x7f83440adf20 2024-01-15T17:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.701+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8344067450 con 0x7f83440adf20 2024-01-15T17:58:08.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.702+0000 7f8340c00640 1 -- 172.21.15.26:0/1775898396 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f8334014800 con 0x7f83440adf20 2024-01-15T17:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.703+0000 7f8340c00640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f831c06b500 0x7f831c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:08.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.703+0000 7f8340c00640 1 -- 172.21.15.26:0/1775898396 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f833408aa00 con 0x7f83440adf20 2024-01-15T17:58:08.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.704+0000 7f833b200640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f831c06b500 0x7f831c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:08.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.705+0000 7f833b200640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f831c06b500 0x7f831c06d9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f832c036000 tx=0x7f832c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:08.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.708+0000 7f8340c00640 1 -- 172.21.15.26:0/1775898396 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8334055640 con 0x7f83440adf20 2024-01-15T17:58:08.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.903+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f8344144920 con 0x7f83440adf20 2024-01-15T17:58:08.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.908+0000 7f8340c00640 1 -- 172.21.15.26:0/1775898396 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v13) v1 ==== 120+0+0 (secure 0 0 0) 0x7f8334058c60 con 0x7f83440adf20 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.911+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f831c06b500 msgr2=0x7f831c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.911+0000 7f8349600640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f831c06b500 0x7f831c06d9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f832c036000 tx=0x7f832c03d040 comp rx=0 tx=0).stop 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.911+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83440adf20 msgr2=0x7f83441f6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.911+0000 7f8349600640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83440adf20 0x7f83441f6880 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f833400efc0 tx=0x7f833400c490 comp rx=0 tx=0).stop 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.912+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 shutdown_connections 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.912+0000 7f8349600640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f834415bf10 0x7f8344143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.912+0000 7f8349600640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f831c06b500 0x7f831c06d9c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.912+0000 7f8349600640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83440cc980 0x7f834414dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.912+0000 7f8349600640 1 --2- 172.21.15.26:0/1775898396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83440adf20 0x7f83441f6880 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.912+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 >> 172.21.15.26:0/1775898396 conn(0x7f834406a7d0 msgr2=0x7f834406c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:08.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.912+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 shutdown_connections 2024-01-15T17:58:08.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:08.912+0000 7f8349600640 1 -- 172.21.15.26:0/1775898396 wait complete. 2024-01-15T17:58:08.933 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone' 2024-01-15T17:58:09.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.140+0000 7f1092400640 1 Processor -- start 2024-01-15T17:58:09.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.140+0000 7f1092400640 1 -- start start 2024-01-15T17:58:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.141+0000 7f1092400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c160920 0x7f108c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.141+0000 7f1092400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f108c0cfe70 0x7f108c161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.141+0000 7f1092400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f108c0d0a10 con 0x7f108c0cf9f0 2024-01-15T17:58:09.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.141+0000 7f1092400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108c0d1f50 con 0x7f108c0cfe70 2024-01-15T17:58:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.141+0000 7f1092400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0cf080 con 0x7f108c160920 2024-01-15T17:58:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.141+0000 7f1083c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f108c0cf9f0 0x7f108c161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39092/0 (socket says 172.21.15.26:39092) 2024-01-15T17:58:09.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.141+0000 7f1083c00640 1 -- 172.21.15.26:0/3937638751 learned_addr learned my addr 172.21.15.26:0/3937638751 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f1091a00640 1 --2- 172.21.15.26:0/3937638751 >> v2:172.21.15.124:3300/0 conn(0x7f108c0cfe70 0x7f108c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f1091000640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c160920 0x7f108c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494677132 0 0) 0x7f108c0d0a10 con 0x7f108c0cf9f0 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f106c003520 con 0x7f108c0cf9f0 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 39235467 0 0) 0x7f106c003520 con 0x7f108c0cf9f0 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c160920 msgr2=0x7f108c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c160920 0x7f108c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 >> v2:172.21.15.124:3300/0 conn(0x7f108c0cfe70 msgr2=0x7f108c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 --2- 172.21.15.26:0/3937638751 >> v2:172.21.15.124:3300/0 conn(0x7f108c0cfe70 0x7f108c161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c0c6be0 con 0x7f108c0cf9f0 2024-01-15T17:58:09.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f1074001670 con 0x7f108c0cf9f0 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 >> v1:172.21.15.26:6789/0 conn(0x7f108c0cf9f0 legacy=0x7f108c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c003970 0x7f106c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.142+0000 7f108be00640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c160920 0x7f106c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f108be00640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f106c004d00 0x7f106c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0d1f50 con 0x7f106c003970 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0cf080 con 0x7f106c004d00 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106c00b980 con 0x7f108c160920 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f1091000640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c003970 0x7f106c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f1083c00640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c160920 0x7f106c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f1091000640 1 -- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c160920 msgr2=0x7f106c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f1091000640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c160920 0x7f106c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f1091000640 1 -- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f106c004d00 msgr2=0x7f106c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f1091000640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f106c004d00 0x7f106c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f1091000640 1 -- 172.21.15.26:0/3937638751 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c003520 con 0x7f106c003970 2024-01-15T17:58:09.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.143+0000 7f1083c00640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c160920 0x7f106c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.144+0000 7f1091000640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c003970 0x7f106c003d90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f107c009fc0 tx=0x7f106c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e608c12cfaa66b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.145+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c02cc20 con 0x7f106c003970 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.145+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c00f040 con 0x7f106c003970 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.145+0000 7f108be00640 1 -- 172.21.15.26:0/3937638751 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c034440 con 0x7f106c003970 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.145+0000 7f1092400640 1 -- 172.21.15.26:0/3937638751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c003970 msgr2=0x7f106c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.145+0000 7f1092400640 1 --2- 172.21.15.26:0/3937638751 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f106c003970 0x7f106c003d90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f107c009fc0 tx=0x7f106c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.145+0000 7f1091a00640 1 -- 172.21.15.26:0/3937638751 reap_dead start 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.145+0000 7f1092400640 1 -- 172.21.15.26:0/3937638751 shutdown_connections 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.145+0000 7f1092400640 1 -- 172.21.15.26:0/3937638751 >> 172.21.15.26:0/3937638751 conn(0x7f108c0d3090 msgr2=0x7f108c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.146+0000 7f1092400640 1 -- 172.21.15.26:0/3937638751 shutdown_connections 2024-01-15T17:58:09.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.146+0000 7f1092400640 1 -- 172.21.15.26:0/3937638751 wait complete. 2024-01-15T17:58:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.147+0000 7f1092400640 1 Processor -- start 2024-01-15T17:58:09.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.148+0000 7f1092400640 1 -- start start 2024-01-15T17:58:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.148+0000 7f1092400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c15a510 0x7f108c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1092400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f108c15a8f0 0x7f108c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1092400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c15f690 0x7f108c0b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1092400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0b87b0 con 0x7f108c15f690 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1092400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0ca300 con 0x7f108c15a8f0 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1092400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108c0ca5e0 con 0x7f108c15a510 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1091a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c15f690 0x7f108c0b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1091a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c15f690 0x7f108c0b4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40806/0 (socket says 172.21.15.26:40806) 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1091a00640 1 -- 172.21.15.26:0/1247580613 learned_addr learned my addr 172.21.15.26:0/1247580613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1091000640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c15a510 0x7f108c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1083c00640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f108c15a8f0 0x7f108c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1091a00640 1 -- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c15a510 msgr2=0x7f108c15f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1091a00640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c15a510 0x7f108c15f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1091a00640 1 -- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f108c15a8f0 msgr2=0x7f108c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1091a00640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f108c15a8f0 0x7f108c0b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.149+0000 7f1091a00640 1 -- 172.21.15.26:0/1247580613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c0ca8c0 con 0x7f108c15f690 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.150+0000 7f1091a00640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c15f690 0x7f108c0b4da0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f108c08bfe0 tx=0x7f108400e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.150+0000 7f1089600640 1 -- 172.21.15.26:0/1247580613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084013680 con 0x7f108c15f690 2024-01-15T17:58:09.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.150+0000 7f1089600640 1 -- 172.21.15.26:0/1247580613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10840137e0 con 0x7f108c15f690 2024-01-15T17:58:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.150+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108c0caba0 con 0x7f108c15f690 2024-01-15T17:58:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.150+0000 7f1089600640 1 -- 172.21.15.26:0/1247580613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084013b90 con 0x7f108c15f690 2024-01-15T17:58:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.150+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108c15c670 con 0x7f108c15f690 2024-01-15T17:58:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.151+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f108c0cb0c0 con 0x7f108c15f690 2024-01-15T17:58:09.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.153+0000 7f1089600640 1 -- 172.21.15.26:0/1247580613 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f1084013cf0 con 0x7f108c15f690 2024-01-15T17:58:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.154+0000 7f1089600640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f106406b530 0x7f106406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.154+0000 7f1089600640 1 -- 172.21.15.26:0/1247580613 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f1084089950 con 0x7f108c15f690 2024-01-15T17:58:09.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.154+0000 7f1091000640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f106406b530 0x7f106406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.155+0000 7f1091000640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f106406b530 0x7f106406d9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f107c036000 tx=0x7f107c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:09.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.158+0000 7f1089600640 1 -- 172.21.15.26:0/1247580613 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f108c0cb0c0 con 0x7f108c15f690 2024-01-15T17:58:09.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.420+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]} v 0) v1 -- 0x7f108c0cb0c0 con 0x7f108c15f690 2024-01-15T17:58:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.550+0000 7f1089600640 1 -- 172.21.15.26:0/1247580613 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_1_clone"]}]=0 v168) v1 ==== 152+0+0 (secure 0 0 0) 0x7f108c0cb0c0 con 0x7f108c15f690 2024-01-15T17:58:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f106406b530 msgr2=0x7f106406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f106406b530 0x7f106406d9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f107c036000 tx=0x7f107c02e000 comp rx=0 tx=0).stop 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c15f690 msgr2=0x7f108c0b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c15f690 0x7f108c0b4da0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f108c08bfe0 tx=0x7f108400e9f0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 shutdown_connections 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f106406b530 0x7f106406d9f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f108c15f690 0x7f108c0b4da0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f108c15a8f0 0x7f108c0b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 --2- 172.21.15.26:0/1247580613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f108c15a510 0x7f108c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 >> 172.21.15.26:0/1247580613 conn(0x7f108c0d3090 msgr2=0x7f108c0c6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 shutdown_connections 2024-01-15T17:58:09.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.554+0000 7f1092400640 1 -- 172.21.15.26:0/1247580613 wait complete. 2024-01-15T17:58:09.578 INFO:tasks.cephfs_test_runner:Test that when one clone is in progress, one progress bar is printed ... ok 2024-01-15T17:58:09.579 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bar_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgress) X = number of clones issued. 2024-01-15T17:58:09.579 INFO:tasks.cephfs_test_runner:test_progress_bar_for_4_clones (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T17:58:09.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:58:09.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.797+0000 7f9556200640 1 Processor -- start 2024-01-15T17:58:09.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.798+0000 7f9556200640 1 -- start start 2024-01-15T17:58:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.799+0000 7f9556200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9550160920 0x7f95500b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.799+0000 7f9556200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95500cfea0 0x7f9550161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.799+0000 7f9556200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95500d0a10 con 0x7f95500cf9f0 2024-01-15T17:58:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.799+0000 7f9556200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95500d0330 con 0x7f9550160920 2024-01-15T17:58:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.799+0000 7f9556200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500d1f50 con 0x7f95500cfea0 2024-01-15T17:58:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.800+0000 7f954fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95500cf9f0 0x7f9550161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39102/0 (socket says 172.21.15.26:39102) 2024-01-15T17:58:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.800+0000 7f9554e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9550160920 0x7f95500b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.800+0000 7f954fe00640 1 -- 172.21.15.26:0/1840840827 learned_addr learned my addr 172.21.15.26:0/1840840827 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:09.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.800+0000 7f9555800640 1 --2- 172.21.15.26:0/1840840827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95500cfea0 0x7f9550161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.800+0000 7f9554e00640 1 -- 172.21.15.26:0/1840840827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95500cfea0 msgr2=0x7f9550161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:09.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.800+0000 7f9554e00640 1 --2- 172.21.15.26:0/1840840827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95500cfea0 0x7f9550161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.800+0000 7f9554e00640 1 -- 172.21.15.26:0/1840840827 >> v1:172.21.15.26:6789/0 conn(0x7f95500cf9f0 legacy=0x7f9550161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.800+0000 7f9554e00640 1 -- 172.21.15.26:0/1840840827 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500cf080 con 0x7f9550160920 2024-01-15T17:58:09.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.801+0000 7f9554e00640 1 --2- 172.21.15.26:0/1840840827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9550160920 0x7f95500b3f30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f95400097a0 tx=0x7f954002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2befea56f844f284 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:09.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.802+0000 7f954f400640 1 -- 172.21.15.26:0/1840840827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954002d040 con 0x7f9550160920 2024-01-15T17:58:09.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.802+0000 7f954f400640 1 -- 172.21.15.26:0/1840840827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9540031810 con 0x7f9550160920 2024-01-15T17:58:09.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.802+0000 7f954f400640 1 -- 172.21.15.26:0/1840840827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9540031ba0 con 0x7f9550160920 2024-01-15T17:58:09.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.802+0000 7f9556200640 1 -- 172.21.15.26:0/1840840827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9550160920 msgr2=0x7f95500b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.802+0000 7f9556200640 1 --2- 172.21.15.26:0/1840840827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9550160920 0x7f95500b3f30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f95400097a0 tx=0x7f954002b8b0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.802+0000 7f9556200640 1 -- 172.21.15.26:0/1840840827 shutdown_connections 2024-01-15T17:58:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.802+0000 7f9556200640 1 --2- 172.21.15.26:0/1840840827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95500cfea0 0x7f9550161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.802+0000 7f9556200640 1 --2- 172.21.15.26:0/1840840827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9550160920 0x7f95500b3f30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.802+0000 7f9556200640 1 -- 172.21.15.26:0/1840840827 >> 172.21.15.26:0/1840840827 conn(0x7f95500d3090 msgr2=0x7f95500d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.803+0000 7f9556200640 1 -- 172.21.15.26:0/1840840827 shutdown_connections 2024-01-15T17:58:09.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.803+0000 7f9556200640 1 -- 172.21.15.26:0/1840840827 wait complete. 2024-01-15T17:58:09.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.805+0000 7f9556200640 1 Processor -- start 2024-01-15T17:58:09.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.805+0000 7f9556200640 1 -- start start 2024-01-15T17:58:09.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.806+0000 7f9556200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95500cf9f0 0x7f95500b6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.806+0000 7f9556200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95500cfea0 0x7f95500b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.806+0000 7f9556200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9550160920 0x7f95500b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.806+0000 7f9556200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500b6b80 con 0x7f95500cfea0 2024-01-15T17:58:09.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.806+0000 7f9556200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95500b73e0 con 0x7f95500cf9f0 2024-01-15T17:58:09.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.806+0000 7f9556200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95500b76c0 con 0x7f9550160920 2024-01-15T17:58:09.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f954fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95500cfea0 0x7f95500b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f954fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95500cfea0 0x7f95500b8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40830/0 (socket says 172.21.15.26:40830) 2024-01-15T17:58:09.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f954fe00640 1 -- 172.21.15.26:0/1468655004 learned_addr learned my addr 172.21.15.26:0/1468655004 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:09.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f9555800640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9550160920 0x7f95500b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f9554e00640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95500cf9f0 0x7f95500b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f954fe00640 1 -- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9550160920 msgr2=0x7f95500b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f954fe00640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9550160920 0x7f95500b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f954fe00640 1 -- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95500cf9f0 msgr2=0x7f95500b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f954fe00640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95500cf9f0 0x7f95500b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.807+0000 7f954fe00640 1 -- 172.21.15.26:0/1468655004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9540009400 con 0x7f95500cfea0 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.808+0000 7f954fe00640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95500cfea0 0x7f95500b8480 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f953800c480 tx=0x7f953800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.808+0000 7f954cc00640 1 -- 172.21.15.26:0/1468655004 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538005ce0 con 0x7f95500cfea0 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.808+0000 7f954cc00640 1 -- 172.21.15.26:0/1468655004 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9538005e40 con 0x7f95500cfea0 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.808+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95500b79a0 con 0x7f95500cfea0 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.808+0000 7f954cc00640 1 -- 172.21.15.26:0/1468655004 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380072f0 con 0x7f95500cfea0 2024-01-15T17:58:09.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.809+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95500b7f60 con 0x7f95500cfea0 2024-01-15T17:58:09.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.810+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95500d3fa0 con 0x7f95500cfea0 2024-01-15T17:58:09.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.811+0000 7f954cc00640 1 -- 172.21.15.26:0/1468655004 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f9538007450 con 0x7f95500cfea0 2024-01-15T17:58:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.812+0000 7f954cc00640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f952806b500 0x7f952806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:09.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.812+0000 7f954cc00640 1 -- 172.21.15.26:0/1468655004 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f95380891b0 con 0x7f95500cfea0 2024-01-15T17:58:09.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.813+0000 7f9554e00640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f952806b500 0x7f952806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:09.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.814+0000 7f9554e00640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f952806b500 0x7f952806d9c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9540036000 tx=0x7f954003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:09.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:09.823+0000 7f954cc00640 1 -- 172.21.15.26:0/1468655004 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9538054040 con 0x7f95500cfea0 2024-01-15T17:58:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.021+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9550067450 con 0x7f95500cfea0 2024-01-15T17:58:10.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.023+0000 7f954cc00640 1 -- 172.21.15.26:0/1468655004 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13007 (secure 0 0 0) 0x7f9538057660 con 0x7f95500cfea0 2024-01-15T17:58:10.027 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:10.027 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":39,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:58:03.490741+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6836/2548050157":"2024-01-16T17:58:00.474566+0000","172.21.15.124:6834/2548050157":"2024-01-16T17:58:00.474566+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f952806b500 msgr2=0x7f952806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f952806b500 0x7f952806d9c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9540036000 tx=0x7f954003c040 comp rx=0 tx=0).stop 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95500cfea0 msgr2=0x7f95500b8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95500cfea0 0x7f95500b8480 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f953800c480 tx=0x7f953800c940 comp rx=0 tx=0).stop 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 shutdown_connections 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9550160920 0x7f95500b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f952806b500 0x7f952806d9c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95500cfea0 0x7f95500b8480 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 --2- 172.21.15.26:0/1468655004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95500cf9f0 0x7f95500b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.026+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 >> 172.21.15.26:0/1468655004 conn(0x7f95500d3090 msgr2=0x7f955006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.027+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 shutdown_connections 2024-01-15T17:58:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.027+0000 7f9556200640 1 -- 172.21.15.26:0/1468655004 wait complete. 2024-01-15T17:58:10.051 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:58:10.051 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:58:10.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.268+0000 7fda35800640 1 Processor -- start 2024-01-15T17:58:10.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.268+0000 7fda35800640 1 -- start start 2024-01-15T17:58:10.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.270+0000 7fda35800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fda30071290 0x7fda30071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.270+0000 7fda35800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fda3015eb60 0x7fda3015f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.270+0000 7fda35800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda300cd520 0x7fda300cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.270+0000 7fda35800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda300ccb10 con 0x7fda3015eb60 2024-01-15T17:58:10.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.270+0000 7fda35800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda30071bb0 con 0x7fda30071290 2024-01-15T17:58:10.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.270+0000 7fda35800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3015f620 con 0x7fda300cd520 2024-01-15T17:58:10.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.270+0000 7fda2f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fda3015eb60 0x7fda3015f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.270+0000 7fda2f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fda3015eb60 0x7fda3015f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40840/0 (socket says 172.21.15.26:40840) 2024-01-15T17:58:10.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.270+0000 7fda2f400640 1 -- 172.21.15.26:0/680675735 learned_addr learned my addr 172.21.15.26:0/680675735 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:10.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.271+0000 7fda34e00640 1 --2- 172.21.15.26:0/680675735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda300cd520 0x7fda300cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.271+0000 7fda2fe00640 1 --2- 172.21.15.26:0/680675735 >> v2:172.21.15.124:3300/0 conn(0x7fda30071290 0x7fda30071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.271+0000 7fda2f400640 1 -- 172.21.15.26:0/680675735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda300cd520 msgr2=0x7fda300cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:10.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.271+0000 7fda2f400640 1 --2- 172.21.15.26:0/680675735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda300cd520 0x7fda300cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.271+0000 7fda2f400640 1 -- 172.21.15.26:0/680675735 >> v2:172.21.15.124:3300/0 conn(0x7fda30071290 msgr2=0x7fda30071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:10.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.271+0000 7fda2f400640 1 --2- 172.21.15.26:0/680675735 >> v2:172.21.15.124:3300/0 conn(0x7fda30071290 0x7fda30071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.271+0000 7fda2f400640 1 -- 172.21.15.26:0/680675735 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3015f900 con 0x7fda3015eb60 2024-01-15T17:58:10.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.273+0000 7fda2f400640 1 --2- 172.21.15.26:0/680675735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda3015eb60 0x7fda3015f010 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fda3006e3a0 tx=0x7fda1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b5b4bf2a07098a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:10.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.273+0000 7fda2ea00640 1 -- 172.21.15.26:0/680675735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18031870 con 0x7fda3015eb60 2024-01-15T17:58:10.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.273+0000 7fda2ea00640 1 -- 172.21.15.26:0/680675735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda180319d0 con 0x7fda3015eb60 2024-01-15T17:58:10.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.273+0000 7fda2ea00640 1 -- 172.21.15.26:0/680675735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18031d60 con 0x7fda3015eb60 2024-01-15T17:58:10.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.274+0000 7fda35800640 1 -- 172.21.15.26:0/680675735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda3015eb60 msgr2=0x7fda3015f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.274+0000 7fda35800640 1 --2- 172.21.15.26:0/680675735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda3015eb60 0x7fda3015f010 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fda3006e3a0 tx=0x7fda1802b910 comp rx=0 tx=0).stop 2024-01-15T17:58:10.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.274+0000 7fda35800640 1 -- 172.21.15.26:0/680675735 shutdown_connections 2024-01-15T17:58:10.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.274+0000 7fda35800640 1 --2- 172.21.15.26:0/680675735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda300cd520 0x7fda300cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.274+0000 7fda35800640 1 --2- 172.21.15.26:0/680675735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda3015eb60 0x7fda3015f010 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.274+0000 7fda35800640 1 --2- 172.21.15.26:0/680675735 >> v2:172.21.15.124:3300/0 conn(0x7fda30071290 0x7fda30071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.274+0000 7fda35800640 1 -- 172.21.15.26:0/680675735 >> 172.21.15.26:0/680675735 conn(0x7fda300b0ed0 msgr2=0x7fda300cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:10.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.274+0000 7fda35800640 1 -- 172.21.15.26:0/680675735 shutdown_connections 2024-01-15T17:58:10.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.274+0000 7fda35800640 1 -- 172.21.15.26:0/680675735 wait complete. 2024-01-15T17:58:10.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.276+0000 7fda35800640 1 Processor -- start 2024-01-15T17:58:10.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.276+0000 7fda35800640 1 -- start start 2024-01-15T17:58:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.277+0000 7fda35800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda30071290 0x7fda300ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.277+0000 7fda35800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda300cd520 0x7fda300ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.277+0000 7fda35800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda3015eb60 0x7fda301f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.277+0000 7fda35800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300c70b0 con 0x7fda300cd520 2024-01-15T17:58:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.277+0000 7fda35800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300c44e0 con 0x7fda3015eb60 2024-01-15T17:58:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.277+0000 7fda35800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda301f54a0 con 0x7fda30071290 2024-01-15T17:58:10.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda2f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda300cd520 0x7fda300ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda2f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda300cd520 0x7fda300ceba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40854/0 (socket says 172.21.15.26:40854) 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda2f400640 1 -- 172.21.15.26:0/3170848827 learned_addr learned my addr 172.21.15.26:0/3170848827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda2fe00640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda30071290 0x7fda300ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda34e00640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda3015eb60 0x7fda301f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda2f400640 1 -- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda30071290 msgr2=0x7fda300ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda2f400640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda30071290 0x7fda300ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda2f400640 1 -- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda3015eb60 msgr2=0x7fda301f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda2f400640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda3015eb60 0x7fda301f4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda2f400640 1 -- 172.21.15.26:0/3170848827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180094d0 con 0x7fda300cd520 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.278+0000 7fda34e00640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda3015eb60 0x7fda301f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.279+0000 7fda2f400640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda300cd520 0x7fda300ceba0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fda18000c00 tx=0x7fda180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.279+0000 7fda27e00640 1 -- 172.21.15.26:0/3170848827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1803acb0 con 0x7fda300cd520 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.279+0000 7fda27e00640 1 -- 172.21.15.26:0/3170848827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1803ae10 con 0x7fda300cd520 2024-01-15T17:58:10.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.279+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda300c3bb0 con 0x7fda300cd520 2024-01-15T17:58:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.279+0000 7fda27e00640 1 -- 172.21.15.26:0/3170848827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18044630 con 0x7fda300cd520 2024-01-15T17:58:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.279+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda300c4170 con 0x7fda300cd520 2024-01-15T17:58:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.280+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda300b1b40 con 0x7fda300cd520 2024-01-15T17:58:10.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.281+0000 7fda27e00640 1 -- 172.21.15.26:0/3170848827 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fda18044790 con 0x7fda300cd520 2024-01-15T17:58:10.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.283+0000 7fda27e00640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0806b530 0x7fda0806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.283+0000 7fda27e00640 1 -- 172.21.15.26:0/3170848827 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fda1802d040 con 0x7fda300cd520 2024-01-15T17:58:10.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.283+0000 7fda2fe00640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0806b530 0x7fda0806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.284+0000 7fda2fe00640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0806b530 0x7fda0806d9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fda20006fd0 tx=0x7fda20008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:10.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.287+0000 7fda27e00640 1 -- 172.21.15.26:0/3170848827 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda180770e0 con 0x7fda300cd520 2024-01-15T17:58:10.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.481+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda300c4300 con 0x7fda300cd520 2024-01-15T17:58:10.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.482+0000 7fda27e00640 1 -- 172.21.15.26:0/3170848827 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda1807a700 con 0x7fda300cd520 2024-01-15T17:58:10.486 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:58:10.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0806b530 msgr2=0x7fda0806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0806b530 0x7fda0806d9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fda20006fd0 tx=0x7fda20008040 comp rx=0 tx=0).stop 2024-01-15T17:58:10.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda300cd520 msgr2=0x7fda300ceba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda300cd520 0x7fda300ceba0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fda18000c00 tx=0x7fda180046c0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 shutdown_connections 2024-01-15T17:58:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0806b530 0x7fda0806d9f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda3015eb60 0x7fda301f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda300cd520 0x7fda300ceba0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 --2- 172.21.15.26:0/3170848827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda30071290 0x7fda300ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 >> 172.21.15.26:0/3170848827 conn(0x7fda300b0ed0 msgr2=0x7fda3015e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 shutdown_connections 2024-01-15T17:58:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.485+0000 7fda35800640 1 -- 172.21.15.26:0/3170848827 wait complete. 2024-01-15T17:58:10.508 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones' 2024-01-15T17:58:10.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.721+0000 7f1f9b000640 1 Processor -- start 2024-01-15T17:58:10.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.722+0000 7f1f9b000640 1 -- start start 2024-01-15T17:58:10.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.723+0000 7f1f9b000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f94162810 0x7f1f94071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.723+0000 7f1f9b000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f940702e0 con 0x7f1f94071780 2024-01-15T17:58:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.723+0000 7f1f9b000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f940705c0 con 0x7f1f94162430 2024-01-15T17:58:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.723+0000 7f1f9b000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f940708a0 con 0x7f1f94162810 2024-01-15T17:58:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.724+0000 7f1f9a600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f94071780 0x7f1f9406c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39118/0 (socket says 172.21.15.26:39118) 2024-01-15T17:58:10.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.724+0000 7f1f9a600640 1 -- 172.21.15.26:0/4197526266 learned_addr learned my addr 172.21.15.26:0/4197526266 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.724+0000 7f1f99200640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f94162810 0x7f1f94071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.724+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821673960 0 0) 0x7f1f940702e0 con 0x7f1f94071780 2024-01-15T17:58:10.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.724+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f74003520 con 0x7f1f94071780 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2869721729 0 0) 0x7f1f74003520 con 0x7f1f94071780 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f94162810 msgr2=0x7f1f94071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f94162810 0x7f1f94071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 >> v1:172.21.15.124:6789/0 conn(0x7f1f94162430 legacy=0x7f1f940b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f94070b80 con 0x7f1f94071780 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f1f8c0019f0 con 0x7f1f94071780 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 >> v1:172.21.15.26:6789/0 conn(0x7f1f94071780 legacy=0x7f1f9406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f740039a0 0x7f1f74003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740047c0 0x7f1f74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740051d0 0x7f1f7400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f940708a0 con 0x7f1f740039a0 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7400bf20 con 0x7f1f740047c0 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7400c0e0 con 0x7f1f740051d0 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.725+0000 7f1f99c00640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f740039a0 0x7f1f74003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.726+0000 7f1f99200640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740047c0 0x7f1f74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.726+0000 7f1f9a600640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740051d0 0x7f1f7400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.726+0000 7f1f99c00640 1 -- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740051d0 msgr2=0x7f1f7400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.726+0000 7f1f99c00640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740051d0 0x7f1f7400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.726+0000 7f1f99c00640 1 -- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740047c0 msgr2=0x7f1f74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.726+0000 7f1f99c00640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740047c0 0x7f1f74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.726+0000 7f1f99c00640 1 -- 172.21.15.26:0/4197526266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74003520 con 0x7f1f740039a0 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.727+0000 7f1f99c00640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f740039a0 0x7f1f74003dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001720 tx=0x7f1f740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97cf790d0d2327e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.727+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840052d0 con 0x7f1f740039a0 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.727+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8402e050 con 0x7f1f740039a0 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.727+0000 7f1f93e00640 1 -- 172.21.15.26:0/4197526266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840035f0 con 0x7f1f740039a0 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.728+0000 7f1f9b000640 1 -- 172.21.15.26:0/4197526266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f740039a0 msgr2=0x7f1f74003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.728+0000 7f1f9b000640 1 --2- 172.21.15.26:0/4197526266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f740039a0 0x7f1f74003dc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001720 tx=0x7f1f740057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.728+0000 7f1f9a600640 1 -- 172.21.15.26:0/4197526266 reap_dead start 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.728+0000 7f1f9b000640 1 -- 172.21.15.26:0/4197526266 shutdown_connections 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.728+0000 7f1f9b000640 1 -- 172.21.15.26:0/4197526266 >> 172.21.15.26:0/4197526266 conn(0x7f1f9406a800 msgr2=0x7f1f9406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:10.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.728+0000 7f1f9b000640 1 -- 172.21.15.26:0/4197526266 shutdown_connections 2024-01-15T17:58:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.728+0000 7f1f9b000640 1 -- 172.21.15.26:0/4197526266 wait complete. 2024-01-15T17:58:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.730+0000 7f1f9b000640 1 Processor -- start 2024-01-15T17:58:10.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.730+0000 7f1f9b000640 1 -- start start 2024-01-15T17:58:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.731+0000 7f1f9b000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740039a0 0x7f1f94158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.731+0000 7f1f9b000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f94153a50 0x7f1f94142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.731+0000 7f1f9b000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f94153ed0 0x7f1f94142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.731+0000 7f1f9b000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f941434a0 con 0x7f1f94153a50 2024-01-15T17:58:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.731+0000 7f1f9b000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f941437f0 con 0x7f1f94153ed0 2024-01-15T17:58:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.731+0000 7f1f9b000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f94143ad0 con 0x7f1f740039a0 2024-01-15T17:58:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f94153a50 0x7f1f94142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f94153a50 0x7f1f94142a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40870/0 (socket says 172.21.15.26:40870) 2024-01-15T17:58:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99200640 1 -- 172.21.15.26:0/2907279461 learned_addr learned my addr 172.21.15.26:0/2907279461 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99c00640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740039a0 0x7f1f94158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99200640 1 -- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740039a0 msgr2=0x7f1f94158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99200640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740039a0 0x7f1f94158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99200640 1 -- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f94153ed0 msgr2=0x7f1f94142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99200640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f94153ed0 0x7f1f94142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99200640 1 -- 172.21.15.26:0/2907279461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f94143db0 con 0x7f1f94153a50 2024-01-15T17:58:10.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99c00640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740039a0 0x7f1f94158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.732+0000 7f1f99200640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f94153a50 0x7f1f94142a20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00f580 tx=0x7f1f7c00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.733+0000 7f1f91600640 1 -- 172.21.15.26:0/2907279461 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c014870 con 0x7f1f94153a50 2024-01-15T17:58:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.733+0000 7f1f91600640 1 -- 172.21.15.26:0/2907279461 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c0149d0 con 0x7f1f94153a50 2024-01-15T17:58:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.733+0000 7f1f91600640 1 -- 172.21.15.26:0/2907279461 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c014d80 con 0x7f1f94153a50 2024-01-15T17:58:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.733+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f94144090 con 0x7f1f94153a50 2024-01-15T17:58:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.733+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9415b110 con 0x7f1f94153a50 2024-01-15T17:58:10.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.734+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f94067450 con 0x7f1f94153a50 2024-01-15T17:58:10.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.735+0000 7f1f91600640 1 -- 172.21.15.26:0/2907279461 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f1f7c015410 con 0x7f1f94153a50 2024-01-15T17:58:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.736+0000 7f1f91600640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f6c06b580 0x7f1f6c06da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:10.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.736+0000 7f1f91600640 1 -- 172.21.15.26:0/2907279461 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f1f7c08a570 con 0x7f1f94153a50 2024-01-15T17:58:10.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.737+0000 7f1f99c00640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f6c06b580 0x7f1f6c06da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:10.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.738+0000 7f1f99c00640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f6c06b580 0x7f1f6c06da40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1f84033000 tx=0x7f1f84009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:10.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:10.741+0000 7f1f91600640 1 -- 172.21.15.26:0/2907279461 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f7c0553f0 con 0x7f1f94153a50 2024-01-15T17:58:11.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.005+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones"]} v 0) v1 -- 0x7f1f940b2ef0 con 0x7f1f94153a50 2024-01-15T17:58:11.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.882+0000 7f1f91600640 1 -- 172.21.15.26:0/2907279461 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones"]}]=0 v170) v1 ==== 156+0+0 (secure 0 0 0) 0x7f1f7c058a10 con 0x7f1f94153a50 2024-01-15T17:58:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f6c06b580 msgr2=0x7f1f6c06da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f6c06b580 0x7f1f6c06da40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1f84033000 tx=0x7f1f84009030 comp rx=0 tx=0).stop 2024-01-15T17:58:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f94153a50 msgr2=0x7f1f94142a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f94153a50 0x7f1f94142a20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00f580 tx=0x7f1f7c00fa40 comp rx=0 tx=0).stop 2024-01-15T17:58:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 shutdown_connections 2024-01-15T17:58:11.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f6c06b580 0x7f1f6c06da40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:11.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f94153ed0 0x7f1f94142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:11.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f94153a50 0x7f1f94142a20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:11.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 --2- 172.21.15.26:0/2907279461 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740039a0 0x7f1f94158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:11.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 >> 172.21.15.26:0/2907279461 conn(0x7f1f9406a800 msgr2=0x7f1f940b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:11.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 shutdown_connections 2024-01-15T17:58:11.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:11.886+0000 7f1f9b000640 1 -- 172.21.15.26:0/2907279461 wait complete. 2024-01-15T17:58:11.911 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T17:58:12.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.133+0000 7f9ecf200640 1 Processor -- start 2024-01-15T17:58:12.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.134+0000 7f9ecf200640 1 -- start start 2024-01-15T17:58:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.135+0000 7f9ecf200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9ed0160920 0x7f9ed00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.135+0000 7f9ecf200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ed00cfea0 0x7f9ed0161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.135+0000 7f9ecf200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00d0a10 con 0x7f9ed00cf9f0 2024-01-15T17:58:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.135+0000 7f9ecf200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00d1f50 con 0x7f9ed00cfea0 2024-01-15T17:58:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.135+0000 7f9ecf200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00cf080 con 0x7f9ed0160920 2024-01-15T17:58:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.136+0000 7f9ece800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ed00cfea0 0x7f9ed0161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.136+0000 7f9ece800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ed00cfea0 0x7f9ed0161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60624/0 (socket says 172.21.15.26:60624) 2024-01-15T17:58:12.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.136+0000 7f9ece800640 1 -- 172.21.15.26:0/1089713199 learned_addr learned my addr 172.21.15.26:0/1089713199 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:12.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.136+0000 7f9ece800640 1 -- 172.21.15.26:0/1089713199 >> v2:172.21.15.124:3300/0 conn(0x7f9ed0160920 msgr2=0x7f9ed00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:12.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.136+0000 7f9ece800640 1 --2- 172.21.15.26:0/1089713199 >> v2:172.21.15.124:3300/0 conn(0x7f9ed0160920 0x7f9ed00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.136+0000 7f9ece800640 1 -- 172.21.15.26:0/1089713199 >> v1:172.21.15.124:6789/0 conn(0x7f9ed00cf9f0 legacy=0x7f9ed0161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:12.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.136+0000 7f9ece800640 1 -- 172.21.15.26:0/1089713199 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00d0310 con 0x7f9ed00cfea0 2024-01-15T17:58:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.137+0000 7f9ece800640 1 --2- 172.21.15.26:0/1089713199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed00cfea0 0x7f9ed0161720 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9ed008c060 tx=0x7f9ec802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b5334effa3a059 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.138+0000 7f9ecca00640 1 -- 172.21.15.26:0/1089713199 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8031870 con 0x7f9ed00cfea0 2024-01-15T17:58:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.138+0000 7f9ecca00640 1 -- 172.21.15.26:0/1089713199 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec80319d0 con 0x7f9ed00cfea0 2024-01-15T17:58:12.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.138+0000 7f9ecca00640 1 -- 172.21.15.26:0/1089713199 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8031d60 con 0x7f9ed00cfea0 2024-01-15T17:58:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.138+0000 7f9ecf200640 1 -- 172.21.15.26:0/1089713199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed00cfea0 msgr2=0x7f9ed0161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.138+0000 7f9ecf200640 1 --2- 172.21.15.26:0/1089713199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed00cfea0 0x7f9ed0161720 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9ed008c060 tx=0x7f9ec802b910 comp rx=0 tx=0).stop 2024-01-15T17:58:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.138+0000 7f9ecf200640 1 -- 172.21.15.26:0/1089713199 shutdown_connections 2024-01-15T17:58:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.138+0000 7f9ecf200640 1 --2- 172.21.15.26:0/1089713199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed00cfea0 0x7f9ed0161720 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.138+0000 7f9ecf200640 1 --2- 172.21.15.26:0/1089713199 >> v2:172.21.15.124:3300/0 conn(0x7f9ed0160920 0x7f9ed00b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.138+0000 7f9ecf200640 1 -- 172.21.15.26:0/1089713199 >> 172.21.15.26:0/1089713199 conn(0x7f9ed00d3090 msgr2=0x7f9ed00d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.139+0000 7f9ecf200640 1 -- 172.21.15.26:0/1089713199 shutdown_connections 2024-01-15T17:58:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.139+0000 7f9ecf200640 1 -- 172.21.15.26:0/1089713199 wait complete. 2024-01-15T17:58:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.140+0000 7f9ecf200640 1 Processor -- start 2024-01-15T17:58:12.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.140+0000 7f9ecf200640 1 -- start start 2024-01-15T17:58:12.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.141+0000 7f9ecf200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ed00cf9f0 0x7f9ed00cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:12.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.141+0000 7f9ecf200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed00cfea0 0x7f9ed00b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:12.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.141+0000 7f9ecf200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0160920 0x7f9ed00ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:12.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.141+0000 7f9ecf200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00cbc30 con 0x7f9ed0160920 2024-01-15T17:58:12.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.141+0000 7f9ecf200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00cb020 con 0x7f9ed00cf9f0 2024-01-15T17:58:12.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.141+0000 7f9ecf200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00cb300 con 0x7f9ed00cfea0 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ece800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0160920 0x7f9ed00ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ece800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0160920 0x7f9ed00ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60628/0 (socket says 172.21.15.26:60628) 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ece800640 1 -- 172.21.15.26:0/2833314165 learned_addr learned my addr 172.21.15.26:0/2833314165 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ece800640 1 -- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed00cfea0 msgr2=0x7f9ed00b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ecd400640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed00cfea0 0x7f9ed00b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ecde00640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ed00cf9f0 0x7f9ed00cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ece800640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed00cfea0 0x7f9ed00b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ece800640 1 -- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ed00cf9f0 msgr2=0x7f9ed00cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ece800640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ed00cf9f0 0x7f9ed00cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ece800640 1 -- 172.21.15.26:0/2833314165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80094d0 con 0x7f9ed0160920 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ecd400640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed00cfea0 0x7f9ed00b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.142+0000 7f9ecde00640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ed00cf9f0 0x7f9ed00cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.143+0000 7f9ece800640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0160920 0x7f9ed00ca2b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8034040 tx=0x7f9ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:12.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.143+0000 7f9ec6000640 1 -- 172.21.15.26:0/2833314165 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec803ad70 con 0x7f9ed0160920 2024-01-15T17:58:12.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.143+0000 7f9ec6000640 1 -- 172.21.15.26:0/2833314165 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec80443f0 con 0x7f9ed0160920 2024-01-15T17:58:12.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.143+0000 7f9ec6000640 1 -- 172.21.15.26:0/2833314165 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8044760 con 0x7f9ed0160920 2024-01-15T17:58:12.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.143+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed00cb5e0 con 0x7f9ed0160920 2024-01-15T17:58:12.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.143+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed00b8410 con 0x7f9ed0160920 2024-01-15T17:58:12.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.144+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed00cbde0 con 0x7f9ed0160920 2024-01-15T17:58:12.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.145+0000 7f9ec6000640 1 -- 172.21.15.26:0/2833314165 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f9ec804d450 con 0x7f9ed0160920 2024-01-15T17:58:12.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.147+0000 7f9ec6000640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806b500 0x7f9ea806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:12.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.147+0000 7f9ec6000640 1 -- 172.21.15.26:0/2833314165 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f9ec802d040 con 0x7f9ed0160920 2024-01-15T17:58:12.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.147+0000 7f9ecde00640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806b500 0x7f9ea806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:12.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.148+0000 7f9ecde00640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806b500 0x7f9ea806d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0006fd0 tx=0x7f9ec0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:12.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.151+0000 7f9ec6000640 1 -- 172.21.15.26:0/2833314165 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ed00cbde0 con 0x7f9ed0160920 2024-01-15T17:58:12.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.343+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9ed00b85d0 con 0x7f9ed0160920 2024-01-15T17:58:12.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.349+0000 7f9ec6000640 1 -- 172.21.15.26:0/2833314165 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9ec8077150 con 0x7f9ed0160920 2024-01-15T17:58:12.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.351+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806b500 msgr2=0x7f9ea806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:12.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.351+0000 7f9ecf200640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806b500 0x7f9ea806d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0006fd0 tx=0x7f9ec0008040 comp rx=0 tx=0).stop 2024-01-15T17:58:12.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.351+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0160920 msgr2=0x7f9ed00ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:12.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.351+0000 7f9ecf200640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0160920 0x7f9ed00ca2b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8034040 tx=0x7f9ec80046c0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.352+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 shutdown_connections 2024-01-15T17:58:12.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.352+0000 7f9ecf200640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed00cfea0 0x7f9ed00b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.352+0000 7f9ecf200640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806b500 0x7f9ea806d9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.352+0000 7f9ecf200640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed0160920 0x7f9ed00ca2b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.352+0000 7f9ecf200640 1 --2- 172.21.15.26:0/2833314165 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ed00cf9f0 0x7f9ed00cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:12.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.352+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 >> 172.21.15.26:0/2833314165 conn(0x7f9ed00d3090 msgr2=0x7f9ed006c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:12.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.352+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 shutdown_connections 2024-01-15T17:58:12.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:12.352+0000 7f9ecf200640 1 -- 172.21.15.26:0/2833314165 wait complete. 2024-01-15T17:58:12.374 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T17:58:12.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T17:58:12.468 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T17:58:12.585 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T17:58:12.585 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:58:12.585 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:12.646 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T17:58:12.646 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:58:12.646 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:12.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T17:58:12.700 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:58:12.700 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:12.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T17:58:12.759 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T17:58:12.759 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:12.817 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T17:58:12.817 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T17:58:12.817 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:58:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.047+0000 7f052e000640 1 Processor -- start 2024-01-15T17:58:13.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.048+0000 7f052e000640 1 -- start start 2024-01-15T17:58:13.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.049+0000 7f052e000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f052813cfa0 0x7f052815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.049+0000 7f052e000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f052813fcb0 0x7f05280b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.049+0000 7f052e000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05280cdec0 con 0x7f05280cd530 2024-01-15T17:58:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.049+0000 7f052e000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05280ce1a0 con 0x7f052813cfa0 2024-01-15T17:58:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.049+0000 7f052e000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05280ce480 con 0x7f052813fcb0 2024-01-15T17:58:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.050+0000 7f0527e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f05280cd530 0x7f052813f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39012/0 (socket says 172.21.15.26:39012) 2024-01-15T17:58:13.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.050+0000 7f0527e00640 1 -- 172.21.15.26:0/1295135570 learned_addr learned my addr 172.21.15.26:0/1295135570 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:13.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.050+0000 7f052cc00640 1 --2- 172.21.15.26:0/1295135570 >> v2:172.21.15.124:3300/0 conn(0x7f052813cfa0 0x7f052815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.050+0000 7f052d600640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f052813fcb0 0x7f05280b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.050+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730132970 0 0) 0x7f05280cdec0 con 0x7f05280cd530 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.050+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0508005730 con 0x7f05280cd530 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1837957289 0 0) 0x7f0508005730 con 0x7f05280cd530 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f052813fcb0 msgr2=0x7f05280b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f052813fcb0 0x7f05280b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 >> v2:172.21.15.124:3300/0 conn(0x7f052813cfa0 msgr2=0x7f052815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 --2- 172.21.15.26:0/1295135570 >> v2:172.21.15.124:3300/0 conn(0x7f052813cfa0 0x7f052815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05280ce760 con 0x7f05280cd530 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f0510001670 con 0x7f05280cd530 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 >> v1:172.21.15.26:6789/0 conn(0x7f05280cd530 legacy=0x7f052813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0508005b80 0x7f0508005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080069d0 0x7f0508006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f052813fcb0 0x7f0508007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05280ce1a0 con 0x7f05080069d0 2024-01-15T17:58:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05280ce480 con 0x7f0508005b80 2024-01-15T17:58:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f050800dbc0 con 0x7f052813fcb0 2024-01-15T17:58:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f0527e00640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080069d0 0x7f0508006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.051+0000 7f052cc00640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0508005b80 0x7f0508005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.052+0000 7f0527e00640 1 -- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f052813fcb0 msgr2=0x7f0508007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.052+0000 7f0527e00640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f052813fcb0 0x7f0508007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.052+0000 7f0527e00640 1 -- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0508005b80 msgr2=0x7f0508005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.052+0000 7f0527e00640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0508005b80 0x7f0508005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.052+0000 7f0527e00640 1 -- 172.21.15.26:0/1295135570 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0508005730 con 0x7f05080069d0 2024-01-15T17:58:13.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.053+0000 7f0527e00640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080069d0 0x7f0508006e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f051000c210 tx=0x7f0508007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbd9358db35f28b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:13.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.053+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510005ad0 con 0x7f05080069d0 2024-01-15T17:58:13.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.053+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0510005c30 con 0x7f05080069d0 2024-01-15T17:58:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.053+0000 7f0527400640 1 -- 172.21.15.26:0/1295135570 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051002c8d0 con 0x7f05080069d0 2024-01-15T17:58:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.053+0000 7f052e000640 1 -- 172.21.15.26:0/1295135570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080069d0 msgr2=0x7f0508006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.053+0000 7f052e000640 1 --2- 172.21.15.26:0/1295135570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f05080069d0 0x7f0508006e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f051000c210 tx=0x7f0508007a00 comp rx=0 tx=0).stop 2024-01-15T17:58:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.053+0000 7f052d600640 1 -- 172.21.15.26:0/1295135570 reap_dead start 2024-01-15T17:58:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.054+0000 7f052e000640 1 -- 172.21.15.26:0/1295135570 shutdown_connections 2024-01-15T17:58:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.054+0000 7f052e000640 1 -- 172.21.15.26:0/1295135570 >> 172.21.15.26:0/1295135570 conn(0x7f052806a800 msgr2=0x7f052806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.054+0000 7f052e000640 1 -- 172.21.15.26:0/1295135570 shutdown_connections 2024-01-15T17:58:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.054+0000 7f052e000640 1 -- 172.21.15.26:0/1295135570 wait complete. 2024-01-15T17:58:13.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.056+0000 7f052e000640 1 Processor -- start 2024-01-15T17:58:13.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.056+0000 7f052e000640 1 -- start start 2024-01-15T17:58:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f052e000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05080069d0 0x7f052813a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f052e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0528154a90 0x7f05281560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f052e000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05281565e0 0x7f0528156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f052e000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0528155c10 con 0x7f0528154a90 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f052e000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052813aba0 con 0x7f05080069d0 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f052e000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0528155000 con 0x7f05281565e0 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f0527e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0528154a90 0x7f05281560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f0527e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0528154a90 0x7f05281560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60650/0 (socket says 172.21.15.26:60650) 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f0527e00640 1 -- 172.21.15.26:0/3936828931 learned_addr learned my addr 172.21.15.26:0/3936828931 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f052cc00640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05080069d0 0x7f052813a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.057+0000 7f052d600640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05281565e0 0x7f0528156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f0527e00640 1 -- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05281565e0 msgr2=0x7f0528156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f0527e00640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05281565e0 0x7f0528156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f0527e00640 1 -- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05080069d0 msgr2=0x7f052813a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f0527e00640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05080069d0 0x7f052813a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f0527e00640 1 -- 172.21.15.26:0/3936828931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05281552e0 con 0x7f0528154a90 2024-01-15T17:58:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f0527e00640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0528154a90 0x7f05281560a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f051000c210 tx=0x7f052815a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:13.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f0524c00640 1 -- 172.21.15.26:0/3936828931 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510039030 con 0x7f0528154a90 2024-01-15T17:58:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f0524c00640 1 -- 172.21.15.26:0/3936828931 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051002cce0 con 0x7f0528154a90 2024-01-15T17:58:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f0524c00640 1 -- 172.21.15.26:0/3936828931 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051003c040 con 0x7f0528154a90 2024-01-15T17:58:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05281555c0 con 0x7f0528154a90 2024-01-15T17:58:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.058+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0528154460 con 0x7f0528154a90 2024-01-15T17:58:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.060+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0528155d80 con 0x7f0528154a90 2024-01-15T17:58:13.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.060+0000 7f0524c00640 1 -- 172.21.15.26:0/3936828931 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f0510032070 con 0x7f0528154a90 2024-01-15T17:58:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.061+0000 7f0524c00640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f050006b500 0x7f050006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.062+0000 7f0524c00640 1 -- 172.21.15.26:0/3936828931 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f05100ab0b0 con 0x7f0528154a90 2024-01-15T17:58:13.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.062+0000 7f052cc00640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f050006b500 0x7f050006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.063+0000 7f052cc00640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f050006b500 0x7f050006d9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0518009990 tx=0x7f0518008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:13.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.066+0000 7f0524c00640 1 -- 172.21.15.26:0/3936828931 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0510079920 con 0x7f0528154a90 2024-01-15T17:58:13.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.301+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0528067450 con 0x7f0528154a90 2024-01-15T17:58:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.302+0000 7f0524c00640 1 -- 172.21.15.26:0/3936828931 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f05100794b0 con 0x7f0528154a90 2024-01-15T17:58:13.306 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:13.306 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1651915477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":1651915477},{"type":"v1","addr":"172.21.15.124:6837","nonce":1651915477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5155,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/2848117329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":2848117329},{"type":"v1","addr":"172.21.15.124:6836","nonce":2848117329}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/506946946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":506946946},{"type":"v1","addr":"172.21.15.26:6833","nonce":506946946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5247,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/3855677701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":3855677701},{"type":"v1","addr":"172.21.15.26:6835","nonce":3855677701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[]} 2024-01-15T17:58:13.306 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 19 2024-01-15T17:58:13.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.305+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f050006b500 msgr2=0x7f050006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.305+0000 7f052e000640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f050006b500 0x7f050006d9c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0518009990 tx=0x7f0518008040 comp rx=0 tx=0).stop 2024-01-15T17:58:13.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.305+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0528154a90 msgr2=0x7f05281560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.305+0000 7f052e000640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0528154a90 0x7f05281560a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f051000c210 tx=0x7f052815a400 comp rx=0 tx=0).stop 2024-01-15T17:58:13.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.306+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 shutdown_connections 2024-01-15T17:58:13.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.306+0000 7f052e000640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f05281565e0 0x7f0528156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.306+0000 7f052e000640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f050006b500 0x7f050006d9c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.306+0000 7f052e000640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0528154a90 0x7f05281560a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.306+0000 7f052e000640 1 --2- 172.21.15.26:0/3936828931 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f05080069d0 0x7f052813a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.306+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 >> 172.21.15.26:0/3936828931 conn(0x7f052806a800 msgr2=0x7f05280aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:13.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.306+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 shutdown_connections 2024-01-15T17:58:13.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.306+0000 7f052e000640 1 -- 172.21.15.26:0/3936828931 wait complete. 2024-01-15T17:58:13.329 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T17:58:13.329 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T17:58:13.329 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T17:58:13.330 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:58:13.330 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T17:58:13.330 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T17:58:13.330 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T17:58:13.330 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:58:13.330 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T17:58:13.330 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T17:58:13.331 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T17:58:13.331 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:58:13.331 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T17:58:13.331 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T17:58:13.331 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T17:58:13.331 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T17:58:13.348 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T17:58:13.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T17:58:13.351 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T17:58:13.351 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T17:58:13.353 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T17:58:13.353 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T17:58:13.355 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T17:58:13.355 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T17:58:13.398 INFO:tasks.ceph.mds.a:Started 2024-01-15T17:58:13.398 DEBUG:teuthology.parallel:result is None 2024-01-15T17:58:13.401 INFO:tasks.ceph.mds.d:Started 2024-01-15T17:58:13.402 DEBUG:teuthology.parallel:result is None 2024-01-15T17:58:13.404 INFO:tasks.ceph.mds.c:Started 2024-01-15T17:58:13.404 DEBUG:teuthology.parallel:result is None 2024-01-15T17:58:13.408 INFO:tasks.ceph.mds.b:Started 2024-01-15T17:58:13.408 DEBUG:teuthology.parallel:result is None 2024-01-15T17:58:13.408 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T17:58:13.559 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T17:58:13.559 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-15T17:58:13.560 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T17:58:13.560 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T17:58:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.593+0000 7f2d61600640 1 Processor -- start 2024-01-15T17:58:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.594+0000 7f2d61600640 1 -- start start 2024-01-15T17:58:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.595+0000 7f2d61600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d5c0b3f30 0x7f2d5c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.595+0000 7f2d61600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5c0d0a10 con 0x7f2d5c0d0100 2024-01-15T17:58:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.595+0000 7f2d61600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5c0cf080 con 0x7f2d5c160920 2024-01-15T17:58:13.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.595+0000 7f2d61600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5c0c6be0 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.595+0000 7f2d60c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d5c0b3f30 0x7f2d5c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.595+0000 7f2d53200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2d5c0d0100 0x7f2d5c0d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39038/0 (socket says 172.21.15.26:39038) 2024-01-15T17:58:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.595+0000 7f2d53200640 1 -- 172.21.15.26:0/4036452285 learned_addr learned my addr 172.21.15.26:0/4036452285 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.596+0000 7f2d60c00640 1 -- 172.21.15.26:0/4036452285 >> v1:172.21.15.124:6789/0 conn(0x7f2d5c160920 legacy=0x7f2d5c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.596+0000 7f2d60c00640 1 -- 172.21.15.26:0/4036452285 >> v1:172.21.15.26:6789/0 conn(0x7f2d5c0d0100 legacy=0x7f2d5c0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.596+0000 7f2d5b400640 1 -- 172.21.15.26:0/4036452285 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372798157 0 0) 0x7f2d5c0cf080 con 0x7f2d5c160920 2024-01-15T17:58:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.596+0000 7f2d60c00640 1 -- 172.21.15.26:0/4036452285 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c0ce7e0 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.597+0000 7f2d60c00640 1 --2- 172.21.15.26:0/4036452285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 0x7f2d5c0d1650 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009d90 tx=0x7f2d5402bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6ee84978ef2dc0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.597+0000 7f2d5b400640 1 -- 172.21.15.26:0/4036452285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5402ccf0 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.597+0000 7f2d5b400640 1 -- 172.21.15.26:0/4036452285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5402ce50 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.597+0000 7f2d5b400640 1 -- 172.21.15.26:0/4036452285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54034660 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.598+0000 7f2d61600640 1 -- 172.21.15.26:0/4036452285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 msgr2=0x7f2d5c0d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.598+0000 7f2d61600640 1 --2- 172.21.15.26:0/4036452285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 0x7f2d5c0d1650 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009d90 tx=0x7f2d5402bf40 comp rx=0 tx=0).stop 2024-01-15T17:58:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.598+0000 7f2d61600640 1 -- 172.21.15.26:0/4036452285 shutdown_connections 2024-01-15T17:58:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.598+0000 7f2d61600640 1 --2- 172.21.15.26:0/4036452285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 0x7f2d5c0d1650 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.598+0000 7f2d61600640 1 -- 172.21.15.26:0/4036452285 >> 172.21.15.26:0/4036452285 conn(0x7f2d5c0d3090 msgr2=0x7f2d5c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.598+0000 7f2d61600640 1 -- 172.21.15.26:0/4036452285 shutdown_connections 2024-01-15T17:58:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.598+0000 7f2d61600640 1 -- 172.21.15.26:0/4036452285 wait complete. 2024-01-15T17:58:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.600+0000 7f2d61600640 1 Processor -- start 2024-01-15T17:58:13.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.600+0000 7f2d61600640 1 -- start start 2024-01-15T17:58:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d61600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 0x7f2d5c1562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d61600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d5c0d0100 0x7f2d5c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d61600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d5c160920 0x7f2d5c06b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d61600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c0cb1e0 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d61600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c0ca660 con 0x7f2d5c0d0100 2024-01-15T17:58:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d61600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c0ca940 con 0x7f2d5c160920 2024-01-15T17:58:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d5be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 0x7f2d5c1562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d5be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 0x7f2d5c1562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60692/0 (socket says 172.21.15.26:60692) 2024-01-15T17:58:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d5be00640 1 -- 172.21.15.26:0/2635362356 learned_addr learned my addr 172.21.15.26:0/2635362356 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d60c00640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d5c160920 0x7f2d5c06b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.601+0000 7f2d53200640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d5c0d0100 0x7f2d5c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d5be00640 1 -- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d5c160920 msgr2=0x7f2d5c06b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d5be00640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d5c160920 0x7f2d5c06b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d5be00640 1 -- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d5c0d0100 msgr2=0x7f2d5c155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d5be00640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d5c0d0100 0x7f2d5c155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d5be00640 1 -- 172.21.15.26:0/2635362356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d54009a70 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d5be00640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 0x7f2d5c1562d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c3c0 tx=0x7f2d4c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d58c00640 1 -- 172.21.15.26:0/2635362356 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0035f0 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d58c00640 1 -- 172.21.15.26:0/2635362356 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c003770 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d58c00640 1 -- 172.21.15.26:0/2635362356 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c003b20 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5c0cac20 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.602+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5c156970 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.604+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5c156b90 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.604+0000 7f2d58c00640 1 -- 172.21.15.26:0/2635362356 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f2d4c003c80 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.605+0000 7f2d58c00640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d3406b500 0x7f2d3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.605+0000 7f2d58c00640 1 -- 172.21.15.26:0/2635362356 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f2d4c0892a0 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.606+0000 7f2d53200640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d3406b500 0x7f2d3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:13.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.606+0000 7f2d53200640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d3406b500 0x7f2d3406d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2d44005880 tx=0x7f2d44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:13.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.610+0000 7f2d58c00640 1 -- 172.21.15.26:0/2635362356 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d4c0540a0 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.816+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2d5c067450 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.914+0000 7f2d58c00640 1 -- 172.21.15.26:0/2635362356 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2d4c0576c0 con 0x7f2d5c0b3f30 2024-01-15T17:58:13.918 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d3406b500 msgr2=0x7f2d3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d3406b500 0x7f2d3406d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2d44005880 tx=0x7f2d44009040 comp rx=0 tx=0).stop 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 msgr2=0x7f2d5c1562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 0x7f2d5c1562d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c3c0 tx=0x7f2d4c00c880 comp rx=0 tx=0).stop 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 shutdown_connections 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d5c160920 0x7f2d5c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d3406b500 0x7f2d3406d9c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d5c0d0100 0x7f2d5c155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 --2- 172.21.15.26:0/2635362356 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d5c0b3f30 0x7f2d5c1562d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.917+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 >> 172.21.15.26:0/2635362356 conn(0x7f2d5c0d3090 msgr2=0x7f2d5c0b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.918+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 shutdown_connections 2024-01-15T17:58:13.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:13.918+0000 7f2d61600640 1 -- 172.21.15.26:0/2635362356 wait complete. 2024-01-15T17:58:13.941 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T17:58:14.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.141+0000 7f6573200640 1 Processor -- start 2024-01-15T17:58:14.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.141+0000 7f6573200640 1 -- start start 2024-01-15T17:58:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.143+0000 7f6573200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f65741447f0 0x7f657415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.143+0000 7f6573200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f657406c470 0x7f65740cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.143+0000 7f6573200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f657415c5c0 con 0x7f657406dab0 2024-01-15T17:58:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.143+0000 7f6573200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657406d690 con 0x7f657406c470 2024-01-15T17:58:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.143+0000 7f6573200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65740cebb0 con 0x7f65741447f0 2024-01-15T17:58:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.143+0000 7f656be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f657406c470 0x7f65740cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.143+0000 7f6572800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f657406dab0 0x7f657415c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39042/0 (socket says 172.21.15.26:39042) 2024-01-15T17:58:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.143+0000 7f6572800640 1 -- 172.21.15.26:0/3490820359 learned_addr learned my addr 172.21.15.26:0/3490820359 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:14.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.144+0000 7f6571e00640 1 --2- 172.21.15.26:0/3490820359 >> v2:172.21.15.124:3300/0 conn(0x7f65741447f0 0x7f657415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.144+0000 7f6571400640 1 -- 172.21.15.26:0/3490820359 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768100064 0 0) 0x7f657415c5c0 con 0x7f657406dab0 2024-01-15T17:58:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.144+0000 7f656be00640 1 -- 172.21.15.26:0/3490820359 >> v2:172.21.15.124:3300/0 conn(0x7f65741447f0 msgr2=0x7f657415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.144+0000 7f656be00640 1 --2- 172.21.15.26:0/3490820359 >> v2:172.21.15.124:3300/0 conn(0x7f65741447f0 0x7f657415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.144+0000 7f656be00640 1 -- 172.21.15.26:0/3490820359 >> v1:172.21.15.26:6789/0 conn(0x7f657406dab0 legacy=0x7f657415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.144+0000 7f656be00640 1 -- 172.21.15.26:0/3490820359 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657406ca10 con 0x7f657406c470 2024-01-15T17:58:14.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.145+0000 7f656be00640 1 --2- 172.21.15.26:0/3490820359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657406c470 0x7f65740cc5e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f65740cecf0 tx=0x7f655c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c34d13bce5869f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.146+0000 7f6571400640 1 -- 172.21.15.26:0/3490820359 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c02e070 con 0x7f657406c470 2024-01-15T17:58:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.146+0000 7f6571400640 1 -- 172.21.15.26:0/3490820359 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c0315a0 con 0x7f657406c470 2024-01-15T17:58:14.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.146+0000 7f6571400640 1 -- 172.21.15.26:0/3490820359 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c031930 con 0x7f657406c470 2024-01-15T17:58:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.146+0000 7f6573200640 1 -- 172.21.15.26:0/3490820359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657406c470 msgr2=0x7f65740cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.146+0000 7f6573200640 1 --2- 172.21.15.26:0/3490820359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657406c470 0x7f65740cc5e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f65740cecf0 tx=0x7f655c02b840 comp rx=0 tx=0).stop 2024-01-15T17:58:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.147+0000 7f6573200640 1 -- 172.21.15.26:0/3490820359 shutdown_connections 2024-01-15T17:58:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.147+0000 7f6573200640 1 --2- 172.21.15.26:0/3490820359 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f657406c470 0x7f65740cc5e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.147+0000 7f6573200640 1 --2- 172.21.15.26:0/3490820359 >> v2:172.21.15.124:3300/0 conn(0x7f65741447f0 0x7f657415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.147+0000 7f6573200640 1 -- 172.21.15.26:0/3490820359 >> 172.21.15.26:0/3490820359 conn(0x7f657406a3e0 msgr2=0x7f657406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.147+0000 7f6573200640 1 -- 172.21.15.26:0/3490820359 shutdown_connections 2024-01-15T17:58:14.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.147+0000 7f6573200640 1 -- 172.21.15.26:0/3490820359 wait complete. 2024-01-15T17:58:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.149+0000 7f6573200640 1 Processor -- start 2024-01-15T17:58:14.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.149+0000 7f6573200640 1 -- start start 2024-01-15T17:58:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.150+0000 7f6573200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f657406c470 0x7f657413dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.150+0000 7f6573200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f657406dab0 0x7f657415d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.150+0000 7f6573200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65741447f0 0x7f6574158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.150+0000 7f6573200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65740affe0 con 0x7f65741447f0 2024-01-15T17:58:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.150+0000 7f6573200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65740b02c0 con 0x7f657406c470 2024-01-15T17:58:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.150+0000 7f6573200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65740b05a0 con 0x7f657406dab0 2024-01-15T17:58:14.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.151+0000 7f6572800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65741447f0 0x7f6574158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.151+0000 7f6572800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65741447f0 0x7f6574158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60710/0 (socket says 172.21.15.26:60710) 2024-01-15T17:58:14.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.151+0000 7f6572800640 1 -- 172.21.15.26:0/3318675925 learned_addr learned my addr 172.21.15.26:0/3318675925 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.151+0000 7f6572800640 1 -- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f657406dab0 msgr2=0x7f657415d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.151+0000 7f656be00640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f657406dab0 0x7f657415d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f6572800640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f657406dab0 0x7f657415d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f6572800640 1 -- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f657406c470 msgr2=0x7f657413dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f6572800640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f657406c470 0x7f657413dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f6572800640 1 -- 172.21.15.26:0/3318675925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c009400 con 0x7f65741447f0 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f6572800640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65741447f0 0x7f6574158690 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f656c00c7a0 tx=0x7f656c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f656a000640 1 -- 172.21.15.26:0/3318675925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c00a640 con 0x7f65741447f0 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f656a000640 1 -- 172.21.15.26:0/3318675925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c00a7a0 con 0x7f65741447f0 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65740b0880 con 0x7f65741447f0 2024-01-15T17:58:14.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f656a000640 1 -- 172.21.15.26:0/3318675925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c00abd0 con 0x7f65741447f0 2024-01-15T17:58:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.152+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657413e2c0 con 0x7f65741447f0 2024-01-15T17:58:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.154+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657415c9e0 con 0x7f65741447f0 2024-01-15T17:58:14.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.155+0000 7f656a000640 1 -- 172.21.15.26:0/3318675925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f656c00ad30 con 0x7f65741447f0 2024-01-15T17:58:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.155+0000 7f656a000640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f654c06b500 0x7f654c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.156+0000 7f656a000640 1 -- 172.21.15.26:0/3318675925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f656c0894e0 con 0x7f65741447f0 2024-01-15T17:58:14.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.156+0000 7f6571e00640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f654c06b500 0x7f654c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.157+0000 7f6571e00640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f654c06b500 0x7f654c06d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6564006fd0 tx=0x7f6564008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:14.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.160+0000 7f656a000640 1 -- 172.21.15.26:0/3318675925 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f656c054310 con 0x7f65741447f0 2024-01-15T17:58:14.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.425+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6574067450 con 0x7f65741447f0 2024-01-15T17:58:14.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.426+0000 7f656a000640 1 -- 172.21.15.26:0/3318675925 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f656c05ce60 con 0x7f65741447f0 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVl3erNJhAANxPK+39yVM5neq9iKC1HkA==", 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.430 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlLaUlDRAAeKaen5hLlwDDWY6OAMRsPQ==", 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlaIAVOBAAA8LEFM1HtzRtNooQCCjS/w==", 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.431 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlmpTBHRAAUwbZeVpysMsn56QRLRJCoA==", 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.432 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA7caVlCTmgDhAAPWb0qEh134xxyF8lM9DoNg==", 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.433 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA+caVlaxF5JxAA23Z9j21ATy3GVlhgndDIxw==", 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBCcaVlcRtjBRAAOFN9Xf5kN5MXL6yQfRI27g==", 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:58:14.434 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBFcaVlvAzEHhAACDoNh7BZY9gbKTdGwIrhlg==", 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBMcaVl8yiOMBAAdI4NTeSB1d/nSHRaIDMcFw==", 2024-01-15T17:58:14.435 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBQcaVltf8sARAAfPaJAfoTgQoEivw90jRz/Q==", 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:58:14.436 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBTcaVltuKiGhAAT0cIJiRkQ38QXT/h6Gyggg==", 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBWcaVlbJqVMhAAKjpl6pAHbytjZWMYeRbRjA==", 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA==", 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.438 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVlrVpqBxAAxOL0iuAlOvfR+KgSsin3vQ==", 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.439 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlO8vdNxAAlUE/TFCDQoXuxvwLPwUvtg==", 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVl5qgCEhAALgt7d3OjEmq5bc4nAD2rTw==", 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T17:58:14.440 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA2caVlyg4eHBAAwTCXVk8hO9+zcZ1tUA3gfA==", 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.441 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlvuBcGhAAGqdCMSffJwnMKlFh3CtyCw==", 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVle8Z+DhAA9cWxzc4puvrHAedqupPG3Q==", 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T17:58:14.442 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.443 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:14.443 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-01-15T17:58:14.443 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:58:14.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f654c06b500 msgr2=0x7f654c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f654c06b500 0x7f654c06d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6564006fd0 tx=0x7f6564008040 comp rx=0 tx=0).stop 2024-01-15T17:58:14.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65741447f0 msgr2=0x7f6574158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65741447f0 0x7f6574158690 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f656c00c7a0 tx=0x7f656c00cc60 comp rx=0 tx=0).stop 2024-01-15T17:58:14.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 shutdown_connections 2024-01-15T17:58:14.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f657406dab0 0x7f657415d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f654c06b500 0x7f654c06d9c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65741447f0 0x7f6574158690 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 --2- 172.21.15.26:0/3318675925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f657406c470 0x7f657413dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 >> 172.21.15.26:0/3318675925 conn(0x7f657406a3e0 msgr2=0x7f65740adc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:14.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 shutdown_connections 2024-01-15T17:58:14.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.429+0000 7f6573200640 1 -- 172.21.15.26:0/3318675925 wait complete. 2024-01-15T17:58:14.450 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:58:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.664+0000 7f32dfe00640 1 Processor -- start 2024-01-15T17:58:14.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.664+0000 7f32dfe00640 1 -- start start 2024-01-15T17:58:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.666+0000 7f32dfe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f32e00cc500 0x7f32e00cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.666+0000 7f32dfe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e0070e50 con 0x7f32e006d160 2024-01-15T17:58:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.666+0000 7f32dfe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e0144020 con 0x7f32e0143c40 2024-01-15T17:58:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.666+0000 7f32dfe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e00723c0 con 0x7f32e00cc500 2024-01-15T17:58:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.666+0000 7f32df400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f32e006d160 0x7f32e006d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39054/0 (socket says 172.21.15.26:39054) 2024-01-15T17:58:14.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.666+0000 7f32df400640 1 -- 172.21.15.26:0/3810875076 learned_addr learned my addr 172.21.15.26:0/3810875076 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32de000640 1 --2- 172.21.15.26:0/3810875076 >> v2:172.21.15.124:3300/0 conn(0x7f32e00cc500 0x7f32e00cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901911319 0 0) 0x7f32e0070e50 con 0x7f32e006d160 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32c0003520 con 0x7f32e006d160 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1390784789 0 0) 0x7f32c0003520 con 0x7f32e006d160 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 >> v2:172.21.15.124:3300/0 conn(0x7f32e00cc500 msgr2=0x7f32e00cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 --2- 172.21.15.26:0/3810875076 >> v2:172.21.15.124:3300/0 conn(0x7f32e00cc500 0x7f32e00cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 >> v1:172.21.15.124:6789/0 conn(0x7f32e0143c40 legacy=0x7f32e0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0071e00 con 0x7f32e006d160 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f32d80019f0 con 0x7f32e006d160 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 >> v1:172.21.15.26:6789/0 conn(0x7f32e006d160 legacy=0x7f32e006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c0003970 0x7f32c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32c00047c0 0x7f32c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32c0005180 0x7f32c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e00723c0 con 0x7f32c0003970 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c000c120 con 0x7f32c00047c0 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.667+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c000c2e0 con 0x7f32c0005180 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.668+0000 7f32dea00640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c0003970 0x7f32c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.668+0000 7f32de000640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32c00047c0 0x7f32c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.668+0000 7f32dea00640 1 -- 172.21.15.26:0/3810875076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32c0005180 msgr2=0x7f32c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.668+0000 7f32dea00640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32c0005180 0x7f32c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.668+0000 7f32dea00640 1 -- 172.21.15.26:0/3810875076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32c00047c0 msgr2=0x7f32c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.668+0000 7f32dea00640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32c00047c0 0x7f32c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.668+0000 7f32dea00640 1 -- 172.21.15.26:0/3810875076 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c0003520 con 0x7f32c0003970 2024-01-15T17:58:14.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.668+0000 7f32df400640 1 -- 172.21.15.26:0/3810875076 reap_dead start 2024-01-15T17:58:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.669+0000 7f32dea00640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c0003970 0x7f32c0003d90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f32d0001750 tx=0x7f32c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f5f124417ebbb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d00052d0 con 0x7f32c0003970 2024-01-15T17:58:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d002e050 con 0x7f32c0003970 2024-01-15T17:58:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dd600640 1 -- 172.21.15.26:0/3810875076 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d0003830 con 0x7f32c0003970 2024-01-15T17:58:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dfe00640 1 -- 172.21.15.26:0/3810875076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c0003970 msgr2=0x7f32c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dfe00640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c0003970 0x7f32c0003d90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f32d0001750 tx=0x7f32c00057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dfe00640 1 -- 172.21.15.26:0/3810875076 shutdown_connections 2024-01-15T17:58:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dfe00640 1 --2- 172.21.15.26:0/3810875076 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32c0003970 0x7f32c0003d90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dfe00640 1 -- 172.21.15.26:0/3810875076 >> 172.21.15.26:0/3810875076 conn(0x7f32e006a800 msgr2=0x7f32e006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dfe00640 1 -- 172.21.15.26:0/3810875076 shutdown_connections 2024-01-15T17:58:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.670+0000 7f32dfe00640 1 -- 172.21.15.26:0/3810875076 wait complete. 2024-01-15T17:58:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.672+0000 7f32dfe00640 1 Processor -- start 2024-01-15T17:58:14.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.672+0000 7f32dfe00640 1 -- start start 2024-01-15T17:58:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.673+0000 7f32dfe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32c0003970 0x7f32e015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.673+0000 7f32dfe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e00b2c40 0x7f32e00b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.673+0000 7f32dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e0138f10 0x7f32e01393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.673+0000 7f32dfe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e00b3600 con 0x7f32e0138f10 2024-01-15T17:58:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.673+0000 7f32dfe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e0139c60 con 0x7f32c0003970 2024-01-15T17:58:14.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.673+0000 7f32dfe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e0139f40 con 0x7f32e00b2c40 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32df400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e0138f10 0x7f32e01393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32df400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e0138f10 0x7f32e01393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60726/0 (socket says 172.21.15.26:60726) 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32df400640 1 -- 172.21.15.26:0/3390313766 learned_addr learned my addr 172.21.15.26:0/3390313766 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32dea00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32c0003970 0x7f32e015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32de000640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e00b2c40 0x7f32e00b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32df400640 1 -- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e00b2c40 msgr2=0x7f32e00b30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32df400640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e00b2c40 0x7f32e00b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32df400640 1 -- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32c0003970 msgr2=0x7f32e015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32df400640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32c0003970 0x7f32e015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.674+0000 7f32df400640 1 -- 172.21.15.26:0/3390313766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e013a220 con 0x7f32e0138f10 2024-01-15T17:58:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.675+0000 7f32df400640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e0138f10 0x7f32e01393c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32e0144ca0 tx=0x7f32d800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.675+0000 7f32d6a00640 1 -- 172.21.15.26:0/3390313766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d800a610 con 0x7f32e0138f10 2024-01-15T17:58:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.675+0000 7f32d6a00640 1 -- 172.21.15.26:0/3390313766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d800a790 con 0x7f32e0138f10 2024-01-15T17:58:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.675+0000 7f32d6a00640 1 -- 172.21.15.26:0/3390313766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d800ab80 con 0x7f32e0138f10 2024-01-15T17:58:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.675+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e013a500 con 0x7f32e0138f10 2024-01-15T17:58:14.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.675+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e015ae10 con 0x7f32e0138f10 2024-01-15T17:58:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.676+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e013aa20 con 0x7f32e0138f10 2024-01-15T17:58:14.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.677+0000 7f32d6a00640 1 -- 172.21.15.26:0/3390313766 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f32d800ace0 con 0x7f32e0138f10 2024-01-15T17:58:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.678+0000 7f32d6a00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32b806b500 0x7f32b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:14.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.678+0000 7f32d6a00640 1 -- 172.21.15.26:0/3390313766 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f32d8089590 con 0x7f32e0138f10 2024-01-15T17:58:14.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.679+0000 7f32dea00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32b806b500 0x7f32b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:14.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.680+0000 7f32dea00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32b806b500 0x7f32b806d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f32d0033000 tx=0x7f32d0009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:14.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.683+0000 7f32d6a00640 1 -- 172.21.15.26:0/3390313766 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32e013aa20 con 0x7f32e0138f10 2024-01-15T17:58:14.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.881+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32e013aa20 con 0x7f32e0138f10 2024-01-15T17:58:14.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.883+0000 7f32d6a00640 1 -- 172.21.15.26:0/3390313766 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+12876 (secure 0 0 0) 0x7f32e013aa20 con 0x7f32e0138f10 2024-01-15T17:58:14.887 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:14.887 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":43,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:58:13.931164+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:58:14.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32b806b500 msgr2=0x7f32b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32b806b500 0x7f32b806d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f32d0033000 tx=0x7f32d0009030 comp rx=0 tx=0).stop 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e0138f10 msgr2=0x7f32e01393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e0138f10 0x7f32e01393c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f32e0144ca0 tx=0x7f32d800cf00 comp rx=0 tx=0).stop 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 shutdown_connections 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f32e00b2c40 0x7f32e00b30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32b806b500 0x7f32b806d9c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f32e0138f10 0x7f32e01393c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 --2- 172.21.15.26:0/3390313766 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f32c0003970 0x7f32e015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 >> 172.21.15.26:0/3390313766 conn(0x7f32e006a800 msgr2=0x7f32e00aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.886+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 shutdown_connections 2024-01-15T17:58:14.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:14.887+0000 7f32dfe00640 1 -- 172.21.15.26:0/3390313766 wait complete. 2024-01-15T17:58:14.910 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T17:58:14.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T17:58:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.131+0000 7f65dd400640 1 Processor -- start 2024-01-15T17:58:15.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.132+0000 7f65dd400640 1 -- start start 2024-01-15T17:58:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.133+0000 7f65dd400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d80ccb50 0x7f65d8070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.134+0000 7f65dd400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d815f4e0 con 0x7f65d80adf20 2024-01-15T17:58:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.134+0000 7f65dd400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d815c3c0 con 0x7f65d815bf10 2024-01-15T17:58:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.134+0000 7f65dd400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d815f970 con 0x7f65d80ccb50 2024-01-15T17:58:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.134+0000 7f65d7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f65d80adf20 0x7f65d80cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39062/0 (socket says 172.21.15.26:39062) 2024-01-15T17:58:15.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.134+0000 7f65d7e00640 1 -- 172.21.15.26:0/3372115604 learned_addr learned my addr 172.21.15.26:0/3372115604 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:15.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.134+0000 7f65cfe00640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d80ccb50 0x7f65d8070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.134+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350663945 0 0) 0x7f65d815f4e0 con 0x7f65d80adf20 2024-01-15T17:58:15.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.134+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b8003520 con 0x7f65d80adf20 2024-01-15T17:58:15.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 728077801 0 0) 0x7f65b8003520 con 0x7f65d80adf20 2024-01-15T17:58:15.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d80ccb50 msgr2=0x7f65d8070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d80ccb50 0x7f65d8070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 >> v1:172.21.15.124:6789/0 conn(0x7f65d815bf10 legacy=0x7f65d8143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d8071130 con 0x7f65d80adf20 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f65c8001670 con 0x7f65d80adf20 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 >> v1:172.21.15.26:6789/0 conn(0x7f65d80adf20 legacy=0x7f65d80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d80ccb50 0x7f65b80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65b8003f70 0x7f65b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d815f970 con 0x7f65b8004d00 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b800bb70 con 0x7f65b8003f70 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b800bd30 con 0x7f65d80ccb50 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.135+0000 7f65dca00640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.136+0000 7f65d7e00640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d80ccb50 0x7f65b80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.136+0000 7f65cfe00640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65b8003f70 0x7f65b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.136+0000 7f65dca00640 1 -- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d80ccb50 msgr2=0x7f65b80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:15.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.136+0000 7f65dca00640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d80ccb50 0x7f65b80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.136+0000 7f65dca00640 1 -- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65b8003f70 msgr2=0x7f65b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:15.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.136+0000 7f65dca00640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65b8003f70 0x7f65b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.136+0000 7f65dca00640 1 -- 172.21.15.26:0/3372115604 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b8003520 con 0x7f65b8004d00 2024-01-15T17:58:15.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.137+0000 7f65dca00640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65b8005180 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f65d0001e50 tx=0x7f65b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b60364fc59b1145 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:15.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.137+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0003b70 con 0x7f65b8004d00 2024-01-15T17:58:15.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.137+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d0003cd0 con 0x7f65b8004d00 2024-01-15T17:58:15.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.138+0000 7f65d7400640 1 -- 172.21.15.26:0/3372115604 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0005310 con 0x7f65b8004d00 2024-01-15T17:58:15.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.138+0000 7f65dd400640 1 -- 172.21.15.26:0/3372115604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 msgr2=0x7f65b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.138+0000 7f65dd400640 1 --2- 172.21.15.26:0/3372115604 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65b8005180 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f65d0001e50 tx=0x7f65b80057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.138+0000 7f65dca00640 1 -- 172.21.15.26:0/3372115604 reap_dead start 2024-01-15T17:58:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.138+0000 7f65dd400640 1 -- 172.21.15.26:0/3372115604 shutdown_connections 2024-01-15T17:58:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.138+0000 7f65dd400640 1 -- 172.21.15.26:0/3372115604 >> 172.21.15.26:0/3372115604 conn(0x7f65d806a7d0 msgr2=0x7f65d806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.138+0000 7f65dd400640 1 -- 172.21.15.26:0/3372115604 shutdown_connections 2024-01-15T17:58:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.138+0000 7f65dd400640 1 -- 172.21.15.26:0/3372115604 wait complete. 2024-01-15T17:58:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.140+0000 7f65dd400640 1 Processor -- start 2024-01-15T17:58:15.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.140+0000 7f65dd400640 1 -- start start 2024-01-15T17:58:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.141+0000 7f65dd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65d8149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.141+0000 7f65dd400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65d8149a40 0x7f65d8149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.141+0000 7f65dd400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d815dca0 0x7f65d815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.141+0000 7f65dd400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80c7d60 con 0x7f65b8004d00 2024-01-15T17:58:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.141+0000 7f65dd400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d815e620 con 0x7f65d8149a40 2024-01-15T17:58:15.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.141+0000 7f65dd400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8149760 con 0x7f65d815dca0 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65d8149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65d8149220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60748/0 (socket says 172.21.15.26:60748) 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65d7e00640 1 -- 172.21.15.26:0/274148916 learned_addr learned my addr 172.21.15.26:0/274148916 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65cfe00640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65d8149a40 0x7f65d8149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65dca00640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d815dca0 0x7f65d815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65d7e00640 1 -- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d815dca0 msgr2=0x7f65d815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65d7e00640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d815dca0 0x7f65d815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65d7e00640 1 -- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65d8149a40 msgr2=0x7f65d8149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65d7e00640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65d8149a40 0x7f65d8149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.142+0000 7f65d7e00640 1 -- 172.21.15.26:0/274148916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d814a5b0 con 0x7f65b8004d00 2024-01-15T17:58:15.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.143+0000 7f65d7e00640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65d8149220 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f65c8007370 tx=0x7f65c8007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:15.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.143+0000 7f65d4c00640 1 -- 172.21.15.26:0/274148916 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800f9c0 con 0x7f65b8004d00 2024-01-15T17:58:15.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.143+0000 7f65d4c00640 1 -- 172.21.15.26:0/274148916 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c800fb20 con 0x7f65b8004d00 2024-01-15T17:58:15.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.143+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d814a890 con 0x7f65b8004d00 2024-01-15T17:58:15.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.143+0000 7f65d4c00640 1 -- 172.21.15.26:0/274148916 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c800fe90 con 0x7f65b8004d00 2024-01-15T17:58:15.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.143+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d814ae50 con 0x7f65b8004d00 2024-01-15T17:58:15.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.144+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d8067450 con 0x7f65b8004d00 2024-01-15T17:58:15.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.146+0000 7f65d4c00640 1 -- 172.21.15.26:0/274148916 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f65c801e3b0 con 0x7f65b8004d00 2024-01-15T17:58:15.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.147+0000 7f65d4c00640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f65b006b500 0x7f65b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.147+0000 7f65d4c00640 1 -- 172.21.15.26:0/274148916 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f65c80898f0 con 0x7f65b8004d00 2024-01-15T17:58:15.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.148+0000 7f65cfe00640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f65b006b500 0x7f65b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.149+0000 7f65cfe00640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f65b006b500 0x7f65b006d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f65d80b3490 tx=0x7f65c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:15.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.152+0000 7f65d4c00640 1 -- 172.21.15.26:0/274148916 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65c8058250 con 0x7f65b8004d00 2024-01-15T17:58:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.348+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65d80cd030 con 0x7f65b8004d00 2024-01-15T17:58:15.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.348+0000 7f65d4c00640 1 -- 172.21.15.26:0/274148916 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65c801a070 con 0x7f65b8004d00 2024-01-15T17:58:15.352 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f65b006b500 msgr2=0x7f65b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f65b006b500 0x7f65b006d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f65d80b3490 tx=0x7f65c0008040 comp rx=0 tx=0).stop 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 msgr2=0x7f65d8149220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65d8149220 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f65c8007370 tx=0x7f65c8007830 comp rx=0 tx=0).stop 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 shutdown_connections 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f65d815dca0 0x7f65d815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f65b006b500 0x7f65b006d9c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f65d8149a40 0x7f65d8149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 --2- 172.21.15.26:0/274148916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f65b8004d00 0x7f65d8149220 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.351+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 >> 172.21.15.26:0/274148916 conn(0x7f65d806a7d0 msgr2=0x7f65d80cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.352+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 shutdown_connections 2024-01-15T17:58:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.352+0000 7f65dd400640 1 -- 172.21.15.26:0/274148916 wait complete. 2024-01-15T17:58:15.373 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T17:58:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.583+0000 7f94dd600640 1 Processor -- start 2024-01-15T17:58:15.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.583+0000 7f94dd600640 1 -- start start 2024-01-15T17:58:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.585+0000 7f94dd600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d80d0900 0x7f94d80adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.585+0000 7f94dd600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f94d815bbd0 0x7f94d815c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.585+0000 7f94dd600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d80b2170 con 0x7f94d815b7f0 2024-01-15T17:58:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.585+0000 7f94dd600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d80b2450 con 0x7f94d815bbd0 2024-01-15T17:58:15.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.585+0000 7f94dd600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80b2730 con 0x7f94d80d0900 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.585+0000 7f94d7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f94d815b7f0 0x7f94d80d17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39078/0 (socket says 172.21.15.26:39078) 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.585+0000 7f94d7e00640 1 -- 172.21.15.26:0/2146086052 learned_addr learned my addr 172.21.15.26:0/2146086052 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.585+0000 7f94dcc00640 1 --2- 172.21.15.26:0/2146086052 >> v2:172.21.15.124:3300/0 conn(0x7f94d815bbd0 0x7f94d815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.585+0000 7f94d7400640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d80d0900 0x7f94d80adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269378469 0 0) 0x7f94d80b2170 con 0x7f94d815b7f0 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b8003520 con 0x7f94d815b7f0 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1246556086 0 0) 0x7f94b8003520 con 0x7f94d815b7f0 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d80d0900 msgr2=0x7f94d80adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d80d0900 0x7f94d80adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 >> v2:172.21.15.124:3300/0 conn(0x7f94d815bbd0 msgr2=0x7f94d815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 --2- 172.21.15.26:0/2146086052 >> v2:172.21.15.124:3300/0 conn(0x7f94d815bbd0 0x7f94d815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d80b08d0 con 0x7f94d815b7f0 2024-01-15T17:58:15.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f94c8001670 con 0x7f94d815b7f0 2024-01-15T17:58:15.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94dcc00640 1 --2- 172.21.15.26:0/2146086052 >> v2:172.21.15.124:3300/0 conn(0x7f94d815bbd0 0x7f94d815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:58:15.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 >> v1:172.21.15.26:6789/0 conn(0x7f94d815b7f0 legacy=0x7f94d80d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d80d0900 0x7f94b8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b8003ef0 0x7f94b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94b8004d00 0x7f94b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80b2450 con 0x7f94b8003ef0 2024-01-15T17:58:15.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d80b2730 con 0x7f94b8004d00 2024-01-15T17:58:15.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.586+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b800b9d0 con 0x7f94d80d0900 2024-01-15T17:58:15.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.587+0000 7f94d7400640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b8003ef0 0x7f94b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.587+0000 7f94d7e00640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d80d0900 0x7f94b8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.587+0000 7f94d7400640 1 -- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d80d0900 msgr2=0x7f94b8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.587+0000 7f94d7400640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d80d0900 0x7f94b8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.587+0000 7f94d7400640 1 -- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94b8004d00 msgr2=0x7f94b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.587+0000 7f94d7400640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94b8004d00 0x7f94b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.587+0000 7f94d7400640 1 -- 172.21.15.26:0/2146086052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8003520 con 0x7f94b8003ef0 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.587+0000 7f94d7e00640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d80d0900 0x7f94b8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.588+0000 7f94d7400640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b8003ef0 0x7f94b80047c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009ac0 tx=0x7f94b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615f83ac05febfd8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.589+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c002c660 con 0x7f94b8003ef0 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.589+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c002c7c0 con 0x7f94b8003ef0 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.589+0000 7f94d6a00640 1 -- 172.21.15.26:0/2146086052 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c002cb70 con 0x7f94b8003ef0 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.589+0000 7f94dd600640 1 -- 172.21.15.26:0/2146086052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b8003ef0 msgr2=0x7f94b80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.589+0000 7f94dd600640 1 --2- 172.21.15.26:0/2146086052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94b8003ef0 0x7f94b80047c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009ac0 tx=0x7f94b80057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.589+0000 7f94dcc00640 1 -- 172.21.15.26:0/2146086052 reap_dead start 2024-01-15T17:58:15.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.589+0000 7f94dd600640 1 -- 172.21.15.26:0/2146086052 shutdown_connections 2024-01-15T17:58:15.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.589+0000 7f94dd600640 1 -- 172.21.15.26:0/2146086052 >> 172.21.15.26:0/2146086052 conn(0x7f94d8099a60 msgr2=0x7f94d8099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:15.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.589+0000 7f94dd600640 1 -- 172.21.15.26:0/2146086052 shutdown_connections 2024-01-15T17:58:15.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.590+0000 7f94dd600640 1 -- 172.21.15.26:0/2146086052 wait complete. 2024-01-15T17:58:15.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.591+0000 7f94dd600640 1 Processor -- start 2024-01-15T17:58:15.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.592+0000 7f94dd600640 1 -- start start 2024-01-15T17:58:15.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.592+0000 7f94dd600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94b8003ef0 0x7f94d81519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94dd600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d81f4f10 0x7f94d81477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94dd600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d815d710 0x7f94d815dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94dd600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d8154ff0 con 0x7f94d81f4f10 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94dd600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d815e0e0 con 0x7f94b8003ef0 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94dd600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d81544e0 con 0x7f94d815d710 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d81f4f10 0x7f94d81477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d81f4f10 0x7f94d81477a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60764/0 (socket says 172.21.15.26:60764) 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7400640 1 -- 172.21.15.26:0/3200802004 learned_addr learned my addr 172.21.15.26:0/3200802004 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7e00640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94b8003ef0 0x7f94d81519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94dcc00640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d815d710 0x7f94d815dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7400640 1 -- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d815d710 msgr2=0x7f94d815dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7400640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d815d710 0x7f94d815dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7400640 1 -- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94b8003ef0 msgr2=0x7f94d81519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7400640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94b8003ef0 0x7f94d81519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7400640 1 -- 172.21.15.26:0/3200802004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d8156550 con 0x7f94d81f4f10 2024-01-15T17:58:15.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.593+0000 7f94d7e00640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94b8003ef0 0x7f94d81519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:58:15.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.594+0000 7f94d7400640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d81f4f10 0x7f94d81477a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f94c0037040 tx=0x7f94c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:15.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.594+0000 7f94cfe00640 1 -- 172.21.15.26:0/3200802004 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0034a80 con 0x7f94d81f4f10 2024-01-15T17:58:15.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.594+0000 7f94cfe00640 1 -- 172.21.15.26:0/3200802004 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c0034c00 con 0x7f94d81f4f10 2024-01-15T17:58:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.594+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d8156830 con 0x7f94d81f4f10 2024-01-15T17:58:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.594+0000 7f94cfe00640 1 -- 172.21.15.26:0/3200802004 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c003e400 con 0x7f94d81f4f10 2024-01-15T17:58:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.594+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d81547c0 con 0x7f94d81f4f10 2024-01-15T17:58:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.596+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d81563b0 con 0x7f94d81f4f10 2024-01-15T17:58:15.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.596+0000 7f94cfe00640 1 -- 172.21.15.26:0/3200802004 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f94c003e560 con 0x7f94d81f4f10 2024-01-15T17:58:15.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.597+0000 7f94cfe00640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f94b006b500 0x7f94b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:15.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.597+0000 7f94cfe00640 1 -- 172.21.15.26:0/3200802004 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f94c00abe40 con 0x7f94d81f4f10 2024-01-15T17:58:15.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.598+0000 7f94d7e00640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f94b006b500 0x7f94b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:15.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.599+0000 7f94d7e00640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f94b006b500 0x7f94b006d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f94d8070250 tx=0x7f94c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:15.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.602+0000 7f94cfe00640 1 -- 172.21.15.26:0/3200802004 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94d81563b0 con 0x7f94d81f4f10 2024-01-15T17:58:15.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.803+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f94d81563b0 con 0x7f94d81f4f10 2024-01-15T17:58:15.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.804+0000 7f94cfe00640 1 -- 172.21.15.26:0/3200802004 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) v1 ==== 84+0+65 (secure 0 0 0) 0x7f94d81563b0 con 0x7f94d81f4f10 2024-01-15T17:58:15.807 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:15.807 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T17:58:15.807 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:15.807 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-15T17:58:15.807 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-15T17:58:15.807 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:15.808 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T17:58:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f94b006b500 msgr2=0x7f94b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f94b006b500 0x7f94b006d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f94d8070250 tx=0x7f94c8009040 comp rx=0 tx=0).stop 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d81f4f10 msgr2=0x7f94d81477a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d81f4f10 0x7f94d81477a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f94c0037040 tx=0x7f94c0004730 comp rx=0 tx=0).stop 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 shutdown_connections 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94d815d710 0x7f94d815dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f94b006b500 0x7f94b006d9c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94d81f4f10 0x7f94d81477a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 --2- 172.21.15.26:0/3200802004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94b8003ef0 0x7f94d81519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 >> 172.21.15.26:0/3200802004 conn(0x7f94d8099a60 msgr2=0x7f94d806e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 shutdown_connections 2024-01-15T17:58:15.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:15.807+0000 7f94dd600640 1 -- 172.21.15.26:0/3200802004 wait complete. 2024-01-15T17:58:15.827 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T17:58:15.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T17:58:16.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.048+0000 7ff236c00640 1 Processor -- start 2024-01-15T17:58:16.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.048+0000 7ff236c00640 1 -- start start 2024-01-15T17:58:16.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.050+0000 7ff236c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff230071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:16.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.050+0000 7ff236c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff2300717d0 0x7ff2300cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:16.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.050+0000 7ff236c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff230070e50 con 0x7ff2300cca40 2024-01-15T17:58:16.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.050+0000 7ff236c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2300723c0 con 0x7ff2300717d0 2024-01-15T17:58:16.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.050+0000 7ff236c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2300cce70 con 0x7ff230143c40 2024-01-15T17:58:16.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.050+0000 7ff235800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff230071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:16.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.051+0000 7ff235800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff230071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:39052/0 (socket says 172.21.15.26:39052) 2024-01-15T17:58:16.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.051+0000 7ff235800640 1 -- 172.21.15.26:0/2022444969 learned_addr learned my addr 172.21.15.26:0/2022444969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:16.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.051+0000 7ff236200640 1 --1- 172.21.15.26:0/2022444969 >> v1:172.21.15.124:6789/0 conn(0x7ff2300cca40 0x7ff23015eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:40686/0 (socket says 172.21.15.26:40686) 2024-01-15T17:58:16.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.051+0000 7ff234e00640 1 --2- 172.21.15.26:0/2022444969 >> v2:172.21.15.124:3300/0 conn(0x7ff2300717d0 0x7ff2300cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:16.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.051+0000 7ff235800640 1 -- 172.21.15.26:0/2022444969 >> v2:172.21.15.124:3300/0 conn(0x7ff2300717d0 msgr2=0x7ff2300cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:16.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.051+0000 7ff235800640 1 --2- 172.21.15.26:0/2022444969 >> v2:172.21.15.124:3300/0 conn(0x7ff2300717d0 0x7ff2300cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.051+0000 7ff235800640 1 -- 172.21.15.26:0/2022444969 >> v1:172.21.15.124:6789/0 conn(0x7ff2300cca40 legacy=0x7ff23015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:16.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.051+0000 7ff235800640 1 -- 172.21.15.26:0/2022444969 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23006e170 con 0x7ff230143c40 2024-01-15T17:58:16.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.053+0000 7ff235800640 1 --2- 172.21.15.26:0/2022444969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff230071290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff220009870 tx=0x7ff22002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953735d5628434d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.053+0000 7ff22fe00640 1 -- 172.21.15.26:0/2022444969 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220031640 con 0x7ff230143c40 2024-01-15T17:58:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.054+0000 7ff22fe00640 1 -- 172.21.15.26:0/2022444969 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2200317a0 con 0x7ff230143c40 2024-01-15T17:58:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.054+0000 7ff22fe00640 1 -- 172.21.15.26:0/2022444969 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff220031b30 con 0x7ff230143c40 2024-01-15T17:58:16.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.054+0000 7ff236c00640 1 -- 172.21.15.26:0/2022444969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 msgr2=0x7ff230071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.054+0000 7ff236c00640 1 --2- 172.21.15.26:0/2022444969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff230071290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff220009870 tx=0x7ff22002b8f0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.054+0000 7ff236c00640 1 -- 172.21.15.26:0/2022444969 shutdown_connections 2024-01-15T17:58:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.054+0000 7ff236c00640 1 --2- 172.21.15.26:0/2022444969 >> v2:172.21.15.124:3300/0 conn(0x7ff2300717d0 0x7ff2300cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.054+0000 7ff236c00640 1 --2- 172.21.15.26:0/2022444969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff230071290 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.054+0000 7ff236c00640 1 -- 172.21.15.26:0/2022444969 >> 172.21.15.26:0/2022444969 conn(0x7ff23006a800 msgr2=0x7ff23006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.054+0000 7ff236c00640 1 -- 172.21.15.26:0/2022444969 shutdown_connections 2024-01-15T17:58:16.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.055+0000 7ff236c00640 1 -- 172.21.15.26:0/2022444969 wait complete. 2024-01-15T17:58:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.056+0000 7ff236c00640 1 Processor -- start 2024-01-15T17:58:16.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.056+0000 7ff236c00640 1 -- start start 2024-01-15T17:58:16.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.057+0000 7ff236c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2300717d0 0x7ff23013d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.057+0000 7ff236c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2300cca40 0x7ff2300aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.057+0000 7ff236c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff2300af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.057+0000 7ff236c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2301e2340 con 0x7ff2300717d0 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.057+0000 7ff236c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2301e2550 con 0x7ff2300cca40 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.057+0000 7ff236c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2301e2830 con 0x7ff230143c40 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff235800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2300717d0 0x7ff23013d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff235800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2300717d0 0x7ff23013d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60772/0 (socket says 172.21.15.26:60772) 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff235800640 1 -- 172.21.15.26:0/1957986142 learned_addr learned my addr 172.21.15.26:0/1957986142 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff236200640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff2300af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff234e00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2300cca40 0x7ff2300aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff235800640 1 -- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 msgr2=0x7ff2300af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff235800640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff2300af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff235800640 1 -- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2300cca40 msgr2=0x7ff2300aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:16.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff235800640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2300cca40 0x7ff2300aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff235800640 1 -- 172.21.15.26:0/1957986142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2200094d0 con 0x7ff2300717d0 2024-01-15T17:58:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.058+0000 7ff234e00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2300cca40 0x7ff2300aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.059+0000 7ff235800640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2300717d0 0x7ff23013d5b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff2200090b0 tx=0x7ff2200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.059+0000 7ff22d600640 1 -- 172.21.15.26:0/1957986142 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22003a9f0 con 0x7ff2300717d0 2024-01-15T17:58:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.059+0000 7ff22d600640 1 -- 172.21.15.26:0/1957986142 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22003ab90 con 0x7ff2300717d0 2024-01-15T17:58:16.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.059+0000 7ff22d600640 1 -- 172.21.15.26:0/1957986142 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2200443f0 con 0x7ff2300717d0 2024-01-15T17:58:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.059+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2301e2b10 con 0x7ff2300717d0 2024-01-15T17:58:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.059+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2301e30d0 con 0x7ff2300717d0 2024-01-15T17:58:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.060+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2301e32c0 con 0x7ff2300717d0 2024-01-15T17:58:16.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.061+0000 7ff22d600640 1 -- 172.21.15.26:0/1957986142 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7ff22002f050 con 0x7ff2300717d0 2024-01-15T17:58:16.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.062+0000 7ff22d600640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff20806b530 0x7ff20806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:16.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.062+0000 7ff22d600640 1 -- 172.21.15.26:0/1957986142 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4885+0+0 (secure 0 0 0) 0x7ff2200abc90 con 0x7ff2300717d0 2024-01-15T17:58:16.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.063+0000 7ff234e00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff20806b530 0x7ff20806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:16.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.064+0000 7ff234e00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff20806b530 0x7ff20806d9f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff2300b2310 tx=0x7ff218008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:16.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.068+0000 7ff22d600640 1 -- 172.21.15.26:0/1957986142 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2200769b0 con 0x7ff2300717d0 2024-01-15T17:58:16.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.263+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff230067450 con 0x7ff2300717d0 2024-01-15T17:58:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.933+0000 7ff22d600640 1 -- 172.21.15.26:0/1957986142 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff2200350b0 con 0x7ff2300717d0 2024-01-15T17:58:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-15T17:58:16.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff20806b530 msgr2=0x7ff20806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:16.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff20806b530 0x7ff20806d9f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff2300b2310 tx=0x7ff218008040 comp rx=0 tx=0).stop 2024-01-15T17:58:16.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2300717d0 msgr2=0x7ff23013d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:16.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2300717d0 0x7ff23013d5b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff2200090b0 tx=0x7ff2200046c0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 shutdown_connections 2024-01-15T17:58:16.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff230143c40 0x7ff2300af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff20806b530 0x7ff20806d9f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2300cca40 0x7ff2300aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 --2- 172.21.15.26:0/1957986142 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2300717d0 0x7ff23013d5b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:16.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.938+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 >> 172.21.15.26:0/1957986142 conn(0x7ff23006a800 msgr2=0x7ff23013f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:16.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.939+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 shutdown_connections 2024-01-15T17:58:16.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:16.939+0000 7ff236c00640 1 -- 172.21.15.26:0/1957986142 wait complete. 2024-01-15T17:58:16.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T17:58:17.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.194+0000 7fb71d600640 1 Processor -- start 2024-01-15T17:58:17.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.195+0000 7fb71d600640 1 -- start start 2024-01-15T17:58:17.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.197+0000 7fb71d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb718160920 0x7fb7180b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:17.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.197+0000 7fb71d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 0x7fb7180cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:17.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.197+0000 7fb71d600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7180d0a10 con 0x7fb718161010 2024-01-15T17:58:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.197+0000 7fb71d600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7180d03b0 con 0x7fb718160920 2024-01-15T17:58:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.197+0000 7fb71d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7180d1f50 con 0x7fb7180cf9f0 2024-01-15T17:58:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.197+0000 7fb717e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb718160920 0x7fb7180b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.197+0000 7fb71cc00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb718161010 0x7fb7181614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39090/0 (socket says 172.21.15.26:39090) 2024-01-15T17:58:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.197+0000 7fb71cc00640 1 -- 172.21.15.26:0/394703917 learned_addr learned my addr 172.21.15.26:0/394703917 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.197+0000 7fb717400640 1 --2- 172.21.15.26:0/394703917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 0x7fb7180cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.198+0000 7fb716a00640 1 -- 172.21.15.26:0/394703917 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10808324 0 0) 0x7fb7180d0a10 con 0x7fb718161010 2024-01-15T17:58:17.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.198+0000 7fb716a00640 1 -- 172.21.15.26:0/394703917 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6f8003520 con 0x7fb718161010 2024-01-15T17:58:17.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.198+0000 7fb717e00640 1 -- 172.21.15.26:0/394703917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 msgr2=0x7fb7180cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:17.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.198+0000 7fb717e00640 1 --2- 172.21.15.26:0/394703917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 0x7fb7180cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.198+0000 7fb717e00640 1 -- 172.21.15.26:0/394703917 >> v1:172.21.15.26:6789/0 conn(0x7fb718161010 legacy=0x7fb7181614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:17.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.198+0000 7fb717e00640 1 -- 172.21.15.26:0/394703917 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7180cf080 con 0x7fb718160920 2024-01-15T17:58:17.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.199+0000 7fb717e00640 1 --2- 172.21.15.26:0/394703917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718160920 0x7fb7180b3f30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb7080097a0 tx=0x7fb70802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ddbe5e3686cca1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb716a00640 1 -- 172.21.15.26:0/394703917 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708031830 con 0x7fb718160920 2024-01-15T17:58:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb716a00640 1 -- 172.21.15.26:0/394703917 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb708031990 con 0x7fb718160920 2024-01-15T17:58:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb716a00640 1 -- 172.21.15.26:0/394703917 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708031d00 con 0x7fb718160920 2024-01-15T17:58:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb71d600640 1 -- 172.21.15.26:0/394703917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718160920 msgr2=0x7fb7180b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb71d600640 1 --2- 172.21.15.26:0/394703917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718160920 0x7fb7180b3f30 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb7080097a0 tx=0x7fb70802b8d0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb71d600640 1 -- 172.21.15.26:0/394703917 shutdown_connections 2024-01-15T17:58:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb71d600640 1 --2- 172.21.15.26:0/394703917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 0x7fb7180cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb71d600640 1 --2- 172.21.15.26:0/394703917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718160920 0x7fb7180b3f30 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb71d600640 1 -- 172.21.15.26:0/394703917 >> 172.21.15.26:0/394703917 conn(0x7fb7180d3090 msgr2=0x7fb7180d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb71d600640 1 -- 172.21.15.26:0/394703917 shutdown_connections 2024-01-15T17:58:17.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.200+0000 7fb71d600640 1 -- 172.21.15.26:0/394703917 wait complete. 2024-01-15T17:58:17.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.202+0000 7fb71d600640 1 Processor -- start 2024-01-15T17:58:17.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.202+0000 7fb71d600640 1 -- start start 2024-01-15T17:58:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.203+0000 7fb71d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 0x7fb7180cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.203+0000 7fb71d600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb718160920 0x7fb718157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.203+0000 7fb71d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718161010 0x7fb7180ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.203+0000 7fb71d600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7180cb2d0 con 0x7fb718161010 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.203+0000 7fb71d600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb718157860 con 0x7fb718160920 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.203+0000 7fb71d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7180cd1b0 con 0x7fb7180cf9f0 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb71cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718161010 0x7fb7180ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb71cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718161010 0x7fb7180ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60792/0 (socket says 172.21.15.26:60792) 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb71cc00640 1 -- 172.21.15.26:0/3480687317 learned_addr learned my addr 172.21.15.26:0/3480687317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb717400640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb718160920 0x7fb718157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb717e00640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 0x7fb7180cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb71cc00640 1 -- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 msgr2=0x7fb7180cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb71cc00640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 0x7fb7180cad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb71cc00640 1 -- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb718160920 msgr2=0x7fb718157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb71cc00640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb718160920 0x7fb718157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.204+0000 7fb71cc00640 1 -- 172.21.15.26:0/3480687317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708009400 con 0x7fb718161010 2024-01-15T17:58:17.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.205+0000 7fb71cc00640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718161010 0x7fb7180ca160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb71000caa0 tx=0x7fb71000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:17.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.205+0000 7fb70fe00640 1 -- 172.21.15.26:0/3480687317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71000aba0 con 0x7fb718161010 2024-01-15T17:58:17.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.205+0000 7fb70fe00640 1 -- 172.21.15.26:0/3480687317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71000ad00 con 0x7fb718161010 2024-01-15T17:58:17.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.205+0000 7fb70fe00640 1 -- 172.21.15.26:0/3480687317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb710012620 con 0x7fb718161010 2024-01-15T17:58:17.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.205+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7180b6f70 con 0x7fb718161010 2024-01-15T17:58:17.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.205+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7180cb7c0 con 0x7fb718161010 2024-01-15T17:58:17.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.206+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7180cb9e0 con 0x7fb718161010 2024-01-15T17:58:17.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.208+0000 7fb70fe00640 1 -- 172.21.15.26:0/3480687317 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fb710007070 con 0x7fb718161010 2024-01-15T17:58:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.208+0000 7fb70fe00640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6f006b530 0x7fb6f006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.209+0000 7fb70fe00640 1 -- 172.21.15.26:0/3480687317 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5255+0+0 (secure 0 0 0) 0x7fb710089d10 con 0x7fb718161010 2024-01-15T17:58:17.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.209+0000 7fb717e00640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6f006b530 0x7fb6f006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.210+0000 7fb717e00640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6f006b530 0x7fb6f006d9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb708036000 tx=0x7fb708008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.214+0000 7fb70fe00640 1 -- 172.21.15.26:0/3480687317 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7180cb9e0 con 0x7fb718161010 2024-01-15T17:58:17.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.409+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb7180cb9e0 con 0x7fb718161010 2024-01-15T17:58:17.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.951+0000 7fb70fe00640 1 -- 172.21.15.26:0/3480687317 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb7180cb9e0 con 0x7fb718161010 2024-01-15T17:58:17.957 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-15T17:58:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.956+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6f006b530 msgr2=0x7fb6f006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:17.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.956+0000 7fb71d600640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6f006b530 0x7fb6f006d9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb708036000 tx=0x7fb708008030 comp rx=0 tx=0).stop 2024-01-15T17:58:17.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.956+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718161010 msgr2=0x7fb7180ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:17.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.956+0000 7fb71d600640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718161010 0x7fb7180ca160 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb71000caa0 tx=0x7fb71000cf60 comp rx=0 tx=0).stop 2024-01-15T17:58:17.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.957+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 shutdown_connections 2024-01-15T17:58:17.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.957+0000 7fb71d600640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6f006b530 0x7fb6f006d9f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.957+0000 7fb71d600640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb718161010 0x7fb7180ca160 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.957+0000 7fb71d600640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb718160920 0x7fb718157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.957+0000 7fb71d600640 1 --2- 172.21.15.26:0/3480687317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb7180cf9f0 0x7fb7180cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.957+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 >> 172.21.15.26:0/3480687317 conn(0x7fb7180d3090 msgr2=0x7fb718161420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:17.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.958+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 shutdown_connections 2024-01-15T17:58:17.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:17.960+0000 7fb71d600640 1 -- 172.21.15.26:0/3480687317 wait complete. 2024-01-15T17:58:17.998 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T17:58:18.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.184+0000 7fa02ac00640 1 Processor -- start 2024-01-15T17:58:18.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.184+0000 7fa02ac00640 1 -- start start 2024-01-15T17:58:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.186+0000 7fa02ac00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa02413e360 0x7fa0240cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.186+0000 7fa02ac00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02415c5f0 con 0x7fa02415f9c0 2024-01-15T17:58:18.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.186+0000 7fa02ac00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02406c470 con 0x7fa02413df80 2024-01-15T17:58:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.186+0000 7fa02ac00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02406c750 con 0x7fa02413e360 2024-01-15T17:58:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.186+0000 7fa028c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa02415f9c0 0x7fa0240cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39094/0 (socket says 172.21.15.26:39094) 2024-01-15T17:58:18.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.186+0000 7fa028c00640 1 -- 172.21.15.26:0/1779279262 learned_addr learned my addr 172.21.15.26:0/1779279262 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa023400640 1 --2- 172.21.15.26:0/1779279262 >> v2:172.21.15.124:3300/0 conn(0x7fa02413e360 0x7fa0240cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:18.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335834791 0 0) 0x7fa02415c5f0 con 0x7fa02415f9c0 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa004003520 con 0x7fa02415f9c0 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2704480769 0 0) 0x7fa004003520 con 0x7fa02415f9c0 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 >> v2:172.21.15.124:3300/0 conn(0x7fa02413e360 msgr2=0x7fa0240cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 --2- 172.21.15.26:0/1779279262 >> v2:172.21.15.124:3300/0 conn(0x7fa02413e360 0x7fa0240cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 >> v1:172.21.15.124:6789/0 conn(0x7fa02413df80 legacy=0x7fa02415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02406ca30 con 0x7fa02415f9c0 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa01c001a10 con 0x7fa02415f9c0 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 >> v1:172.21.15.26:6789/0 conn(0x7fa02415f9c0 legacy=0x7fa0240cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa004003970 0x7fa004003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0040047c0 0x7fa004004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004005180 0x7fa00400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02406c750 con 0x7fa004005180 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00400c020 con 0x7fa004003970 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.187+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00400c1e0 con 0x7fa0040047c0 2024-01-15T17:58:18.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.188+0000 7fa028c00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004005180 0x7fa00400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.188+0000 7fa023e00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa004003970 0x7fa004003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.188+0000 7fa023400640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0040047c0 0x7fa004004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.188+0000 7fa028c00640 1 -- 172.21.15.26:0/1779279262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0040047c0 msgr2=0x7fa004004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.188+0000 7fa028c00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa0040047c0 0x7fa004004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.188+0000 7fa028c00640 1 -- 172.21.15.26:0/1779279262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa004003970 msgr2=0x7fa004003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.188+0000 7fa028c00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa004003970 0x7fa004003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.188+0000 7fa028c00640 1 -- 172.21.15.26:0/1779279262 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa004003520 con 0x7fa004005180 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.189+0000 7fa028c00640 1 -- 172.21.15.26:0/1779279262 reap_dead start 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.189+0000 7fa028c00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004005180 0x7fa00400b980 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa02415b6f0 tx=0x7fa0040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e4bc29566960a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.189+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c034630 con 0x7fa004005180 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.190+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01c02f070 con 0x7fa004005180 2024-01-15T17:58:18.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.190+0000 7fa022a00640 1 -- 172.21.15.26:0/1779279262 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01c034990 con 0x7fa004005180 2024-01-15T17:58:18.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.190+0000 7fa02ac00640 1 -- 172.21.15.26:0/1779279262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004005180 msgr2=0x7fa00400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:18.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.190+0000 7fa02ac00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004005180 0x7fa00400b980 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa02415b6f0 tx=0x7fa0040057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.190+0000 7fa02ac00640 1 -- 172.21.15.26:0/1779279262 shutdown_connections 2024-01-15T17:58:18.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.190+0000 7fa02ac00640 1 --2- 172.21.15.26:0/1779279262 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa004005180 0x7fa00400b980 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.190+0000 7fa02ac00640 1 -- 172.21.15.26:0/1779279262 >> 172.21.15.26:0/1779279262 conn(0x7fa02406a800 msgr2=0x7fa02406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.190+0000 7fa02ac00640 1 -- 172.21.15.26:0/1779279262 shutdown_connections 2024-01-15T17:58:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.190+0000 7fa02ac00640 1 -- 172.21.15.26:0/1779279262 wait complete. 2024-01-15T17:58:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.192+0000 7fa02ac00640 1 Processor -- start 2024-01-15T17:58:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.192+0000 7fa02ac00640 1 -- start start 2024-01-15T17:58:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.193+0000 7fa02ac00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa004005180 0x7fa02413d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.193+0000 7fa02ac00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0240a8ed0 0x7fa024159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.193+0000 7fa02ac00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0240cd840 0x7fa0240cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.193+0000 7fa02ac00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240aa380 con 0x7fa0240a8ed0 2024-01-15T17:58:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.193+0000 7fa02ac00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240ce200 con 0x7fa0240cd840 2024-01-15T17:58:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.193+0000 7fa02ac00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240a9420 con 0x7fa004005180 2024-01-15T17:58:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0240a8ed0 0x7fa024159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0240a8ed0 0x7fa024159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60800/0 (socket says 172.21.15.26:60800) 2024-01-15T17:58:18.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023400640 1 -- 172.21.15.26:0/3231574225 learned_addr learned my addr 172.21.15.26:0/3231574225 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa028c00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0240cd840 0x7fa0240cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:18.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023e00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa004005180 0x7fa02413d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023400640 1 -- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa004005180 msgr2=0x7fa02413d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023400640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa004005180 0x7fa02413d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023400640 1 -- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0240cd840 msgr2=0x7fa0240cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023400640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0240cd840 0x7fa0240cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023400640 1 -- 172.21.15.26:0/3231574225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240a9700 con 0x7fa0240a8ed0 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.194+0000 7fa023400640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0240a8ed0 0x7fa024159fb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0107f0 tx=0x7fa00c010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.195+0000 7fa01be00640 1 -- 172.21.15.26:0/3231574225 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0095d0 con 0x7fa0240a8ed0 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.195+0000 7fa01be00640 1 -- 172.21.15.26:0/3231574225 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c00eb60 con 0x7fa0240a8ed0 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.195+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0240a99e0 con 0x7fa0240a8ed0 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.195+0000 7fa01be00640 1 -- 172.21.15.26:0/3231574225 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c01c400 con 0x7fa0240a8ed0 2024-01-15T17:58:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.195+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02413dac0 con 0x7fa0240a8ed0 2024-01-15T17:58:18.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.196+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0240675d0 con 0x7fa0240a8ed0 2024-01-15T17:58:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.198+0000 7fa01be00640 1 -- 172.21.15.26:0/3231574225 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fa00c01c560 con 0x7fa0240a8ed0 2024-01-15T17:58:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.199+0000 7fa01be00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.199+0000 7fa01be00640 1 -- 172.21.15.26:0/3231574225 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5637+0+0 (secure 0 0 0) 0x7fa00c08a4b0 con 0x7fa0240a8ed0 2024-01-15T17:58:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.199+0000 7fa023e00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:18.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.201+0000 7fa023e00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa014005490 tx=0x7fa014009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:18.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.204+0000 7fa01be00640 1 -- 172.21.15.26:0/3231574225 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa00c0550a0 con 0x7fa0240a8ed0 2024-01-15T17:58:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.440+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa02406df00 con 0x7fa0240a8ed0 2024-01-15T17:58:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.974+0000 7fa01be00640 1 -- 172.21.15.26:0/3231574225 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa00c0586c0 con 0x7fa0240a8ed0 2024-01-15T17:58:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 6 and data pool 7 2024-01-15T17:58:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.987+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ffc06b500 msgr2=0x7f9ffc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.987+0000 7fa02ac00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa014005490 tx=0x7fa014009040 comp rx=0 tx=0).stop 2024-01-15T17:58:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.987+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0240a8ed0 msgr2=0x7fa024159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.987+0000 7fa02ac00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0240a8ed0 0x7fa024159fb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0107f0 tx=0x7fa00c010cb0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.988+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 shutdown_connections 2024-01-15T17:58:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.988+0000 7fa02ac00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ffc06b500 0x7f9ffc06d9c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.988+0000 7fa02ac00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa0240cd840 0x7fa0240cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.988+0000 7fa02ac00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa0240a8ed0 0x7fa024159fb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.988+0000 7fa02ac00640 1 --2- 172.21.15.26:0/3231574225 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa004005180 0x7fa02413d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.988+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 >> 172.21.15.26:0/3231574225 conn(0x7fa02406a800 msgr2=0x7fa02406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.989+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 shutdown_connections 2024-01-15T17:58:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:18.989+0000 7fa02ac00640 1 -- 172.21.15.26:0/3231574225 wait complete. 2024-01-15T17:58:19.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:58:19.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.208+0000 7f720da00640 1 Processor -- start 2024-01-15T17:58:19.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.208+0000 7f720da00640 1 -- start start 2024-01-15T17:58:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.210+0000 7f720da00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f720813df80 0x7f720815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.210+0000 7f720da00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720806c470 con 0x7f72080cf9a0 2024-01-15T17:58:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.210+0000 7f720da00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f720806c750 con 0x7f720813e360 2024-01-15T17:58:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.210+0000 7f720da00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720806ca30 con 0x7f720813df80 2024-01-15T17:58:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.210+0000 7f7206a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f720813df80 0x7f720815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.210+0000 7f7206a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f720813df80 0x7f720815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60814/0 (socket says 172.21.15.26:60814) 2024-01-15T17:58:19.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.210+0000 7f7206a00640 1 -- 172.21.15.26:0/996781236 learned_addr learned my addr 172.21.15.26:0/996781236 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:19.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.210+0000 7f7206a00640 1 -- 172.21.15.26:0/996781236 >> v1:172.21.15.124:6789/0 conn(0x7f720813e360 legacy=0x7f72080cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:19.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.211+0000 7f7206a00640 1 -- 172.21.15.26:0/996781236 >> v1:172.21.15.26:6789/0 conn(0x7f72080cf9a0 legacy=0x7f72080cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.211+0000 7f7205600640 1 -- 172.21.15.26:0/996781236 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706969744 0 0) 0x7f720806c470 con 0x7f72080cf9a0 2024-01-15T17:58:19.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.211+0000 7f7206a00640 1 -- 172.21.15.26:0/996781236 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720806cd10 con 0x7f720813df80 2024-01-15T17:58:19.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.212+0000 7f7206a00640 1 --2- 172.21.15.26:0/996781236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813df80 0x7f720815bee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7200009870 tx=0x7f720002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c135cf366299de48 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:19.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.212+0000 7f7205600640 1 -- 172.21.15.26:0/996781236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200030860 con 0x7f720813df80 2024-01-15T17:58:19.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.212+0000 7f7205600640 1 -- 172.21.15.26:0/996781236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72000309c0 con 0x7f720813df80 2024-01-15T17:58:19.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.212+0000 7f7205600640 1 -- 172.21.15.26:0/996781236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200030d50 con 0x7f720813df80 2024-01-15T17:58:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.213+0000 7f720da00640 1 -- 172.21.15.26:0/996781236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813df80 msgr2=0x7f720815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.213+0000 7f720da00640 1 --2- 172.21.15.26:0/996781236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813df80 0x7f720815bee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7200009870 tx=0x7f720002b9a0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.213+0000 7f720da00640 1 -- 172.21.15.26:0/996781236 shutdown_connections 2024-01-15T17:58:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.213+0000 7f720da00640 1 --2- 172.21.15.26:0/996781236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813df80 0x7f720815bee0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.213+0000 7f720da00640 1 -- 172.21.15.26:0/996781236 >> 172.21.15.26:0/996781236 conn(0x7f720806a800 msgr2=0x7f720806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.213+0000 7f720da00640 1 -- 172.21.15.26:0/996781236 shutdown_connections 2024-01-15T17:58:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.213+0000 7f720da00640 1 -- 172.21.15.26:0/996781236 wait complete. 2024-01-15T17:58:19.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.215+0000 7f720da00640 1 Processor -- start 2024-01-15T17:58:19.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.215+0000 7f720da00640 1 -- start start 2024-01-15T17:58:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f720da00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72080cf9a0 0x7f7208071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f720da00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f720813df80 0x7f720813d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f720da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813e360 0x7f72081527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f720da00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720814dba0 con 0x7f720813e360 2024-01-15T17:58:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f720da00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7208072260 con 0x7f720813df80 2024-01-15T17:58:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f720da00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720814b820 con 0x7f72080cf9a0 2024-01-15T17:58:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f7207400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813e360 0x7f72081527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f7207400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813e360 0x7f72081527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60824/0 (socket says 172.21.15.26:60824) 2024-01-15T17:58:19.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f7207400640 1 -- 172.21.15.26:0/891535191 learned_addr learned my addr 172.21.15.26:0/891535191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:19.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f7207400640 1 -- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72080cf9a0 msgr2=0x7f7208071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f7207400640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72080cf9a0 0x7f7208071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f7206000640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f720813df80 0x7f720813d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f7207400640 1 -- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f720813df80 msgr2=0x7f720813d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.216+0000 7f7207400640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f720813df80 0x7f720813d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.217+0000 7f7207400640 1 -- 172.21.15.26:0/891535191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72000094d0 con 0x7f720813e360 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.217+0000 7f7207400640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813e360 0x7f72081527d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c410 tx=0x7f71f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.217+0000 7f71fea00640 1 -- 172.21.15.26:0/891535191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f80035f0 con 0x7f720813e360 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.217+0000 7f71fea00640 1 -- 172.21.15.26:0/891535191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8003770 con 0x7f720813e360 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.217+0000 7f71fea00640 1 -- 172.21.15.26:0/891535191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8003b20 con 0x7f720813e360 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.217+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720814bb00 con 0x7f720813e360 2024-01-15T17:58:19.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.217+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720814c0c0 con 0x7f720813e360 2024-01-15T17:58:19.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.218+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72080675d0 con 0x7f720813e360 2024-01-15T17:58:19.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.219+0000 7f71fea00640 1 -- 172.21.15.26:0/891535191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f71f8003c80 con 0x7f720813e360 2024-01-15T17:58:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.220+0000 7f71fea00640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71e006b500 0x7f71e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.220+0000 7f71fea00640 1 -- 172.21.15.26:0/891535191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f71f80896a0 con 0x7f720813e360 2024-01-15T17:58:19.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.221+0000 7f7206a00640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71e006b500 0x7f71e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:19.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.222+0000 7f7206a00640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71e006b500 0x7f71e006d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7200036000 tx=0x7f720003b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:19.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.225+0000 7f71fea00640 1 -- 172.21.15.26:0/891535191 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71f80540f0 con 0x7f720813e360 2024-01-15T17:58:19.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.419+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f720806df00 con 0x7f720813e360 2024-01-15T17:58:19.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.422+0000 7f71fea00640 1 -- 172.21.15.26:0/891535191 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16545 (secure 0 0 0) 0x7f71f8057710 con 0x7f720813e360 2024-01-15T17:58:19.426 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:19.426 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":46,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:58:18.442403+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:58:16.318608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-15T17:58:17.464623+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:58:19.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71e006b500 msgr2=0x7f71e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71e006b500 0x7f71e006d9c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7200036000 tx=0x7f720003b040 comp rx=0 tx=0).stop 2024-01-15T17:58:19.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813e360 msgr2=0x7f72081527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813e360 0x7f72081527d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c410 tx=0x7f71f800c8d0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 shutdown_connections 2024-01-15T17:58:19.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71e006b500 0x7f71e006d9c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f720813e360 0x7f72081527d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f720813df80 0x7f720813d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 --2- 172.21.15.26:0/891535191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72080cf9a0 0x7f7208071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.425+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 >> 172.21.15.26:0/891535191 conn(0x7f720806a800 msgr2=0x7f72080cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:19.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.426+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 shutdown_connections 2024-01-15T17:58:19.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.426+0000 7f720da00640 1 -- 172.21.15.26:0/891535191 wait complete. 2024-01-15T17:58:19.447 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:58:19.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.648+0000 7fdbae600640 1 Processor -- start 2024-01-15T17:58:19.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.648+0000 7fdbae600640 1 -- start start 2024-01-15T17:58:19.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.650+0000 7fdbae600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdba8160890 0x7fdba80d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.650+0000 7fdbae600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdba8160c70 0x7fdba80d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.650+0000 7fdbae600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba80cf6a0 con 0x7fdba80c6be0 2024-01-15T17:58:19.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.650+0000 7fdbae600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba80d1430 con 0x7fdba8160890 2024-01-15T17:58:19.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.650+0000 7fdbae600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba80b4370 con 0x7fdba8160c70 2024-01-15T17:58:19.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.650+0000 7fdbad200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdba8160890 0x7fdba80d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:19.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.650+0000 7fdbad200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdba8160890 0x7fdba80d3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60836/0 (socket says 172.21.15.26:60836) 2024-01-15T17:58:19.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.650+0000 7fdbad200640 1 -- 172.21.15.26:0/538923843 learned_addr learned my addr 172.21.15.26:0/538923843 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:19.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.650+0000 7fdbadc00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdba80c6be0 0x7fdba8161110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39120/0 (socket says 172.21.15.26:39120) 2024-01-15T17:58:19.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.651+0000 7fdb9fe00640 1 --2- 172.21.15.26:0/538923843 >> v2:172.21.15.124:3300/0 conn(0x7fdba8160c70 0x7fdba80d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:19.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.651+0000 7fdbad200640 1 -- 172.21.15.26:0/538923843 >> v2:172.21.15.124:3300/0 conn(0x7fdba8160c70 msgr2=0x7fdba80d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:19.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.651+0000 7fdbad200640 1 --2- 172.21.15.26:0/538923843 >> v2:172.21.15.124:3300/0 conn(0x7fdba8160c70 0x7fdba80d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.651+0000 7fdbad200640 1 -- 172.21.15.26:0/538923843 >> v1:172.21.15.26:6789/0 conn(0x7fdba80c6be0 legacy=0x7fdba8161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.651+0000 7fdbad200640 1 -- 172.21.15.26:0/538923843 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba80ce7c0 con 0x7fdba8160890 2024-01-15T17:58:19.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.651+0000 7fdba7e00640 1 -- 172.21.15.26:0/538923843 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632293018 0 0) 0x7fdba80cf6a0 con 0x7fdba80c6be0 2024-01-15T17:58:19.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.652+0000 7fdbad200640 1 --2- 172.21.15.26:0/538923843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba8160890 0x7fdba80d3bb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdba80b5c80 tx=0x7fdb9802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6811972b159aa94 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:19.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdba7e00640 1 -- 172.21.15.26:0/538923843 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb980317e0 con 0x7fdba8160890 2024-01-15T17:58:19.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdba7e00640 1 -- 172.21.15.26:0/538923843 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98031940 con 0x7fdba8160890 2024-01-15T17:58:19.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdba7e00640 1 -- 172.21.15.26:0/538923843 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98031cd0 con 0x7fdba8160890 2024-01-15T17:58:19.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdbae600640 1 -- 172.21.15.26:0/538923843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba8160890 msgr2=0x7fdba80d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdbae600640 1 --2- 172.21.15.26:0/538923843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba8160890 0x7fdba80d3bb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fdba80b5c80 tx=0x7fdb9802b880 comp rx=0 tx=0).stop 2024-01-15T17:58:19.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdbae600640 1 -- 172.21.15.26:0/538923843 shutdown_connections 2024-01-15T17:58:19.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdbae600640 1 --2- 172.21.15.26:0/538923843 >> v2:172.21.15.124:3300/0 conn(0x7fdba8160c70 0x7fdba80d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdbae600640 1 --2- 172.21.15.26:0/538923843 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba8160890 0x7fdba80d3bb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdbae600640 1 -- 172.21.15.26:0/538923843 >> 172.21.15.26:0/538923843 conn(0x7fdba80d3090 msgr2=0x7fdba80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:19.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.653+0000 7fdbae600640 1 -- 172.21.15.26:0/538923843 shutdown_connections 2024-01-15T17:58:19.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.654+0000 7fdbae600640 1 -- 172.21.15.26:0/538923843 wait complete. 2024-01-15T17:58:19.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.655+0000 7fdbae600640 1 Processor -- start 2024-01-15T17:58:19.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.655+0000 7fdbae600640 1 -- start start 2024-01-15T17:58:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.656+0000 7fdbae600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba80c6be0 0x7fdba815dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.656+0000 7fdbae600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdba8160890 0x7fdba815e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.656+0000 7fdbae600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdba8160c70 0x7fdba815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.656+0000 7fdbae600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b5e20 con 0x7fdba80c6be0 2024-01-15T17:58:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.656+0000 7fdbae600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b6100 con 0x7fdba8160890 2024-01-15T17:58:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.656+0000 7fdbae600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba80b63e0 con 0x7fdba8160c70 2024-01-15T17:58:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdbad200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba80c6be0 0x7fdba815dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdbad200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba80c6be0 0x7fdba815dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60844/0 (socket says 172.21.15.26:60844) 2024-01-15T17:58:19.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdbad200640 1 -- 172.21.15.26:0/1466703930 learned_addr learned my addr 172.21.15.26:0/1466703930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdb9fe00640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdba8160890 0x7fdba815e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdbad200640 1 -- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdba8160c70 msgr2=0x7fdba815ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdbad200640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdba8160c70 0x7fdba815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdbad200640 1 -- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdba8160890 msgr2=0x7fdba815e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdbad200640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdba8160890 0x7fdba815e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdbad200640 1 -- 172.21.15.26:0/1466703930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980094d0 con 0x7fdba80c6be0 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.657+0000 7fdbad200640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba80c6be0 0x7fdba815dfb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdb98000c00 tx=0x7fdb980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.658+0000 7fdba5600640 1 -- 172.21.15.26:0/1466703930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98032880 con 0x7fdba80c6be0 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.658+0000 7fdba5600640 1 -- 172.21.15.26:0/1466703930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98032a20 con 0x7fdba80c6be0 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.658+0000 7fdba5600640 1 -- 172.21.15.26:0/1466703930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98032dd0 con 0x7fdba80c6be0 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.658+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba80b66c0 con 0x7fdba80c6be0 2024-01-15T17:58:19.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.658+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba815d390 con 0x7fdba80c6be0 2024-01-15T17:58:19.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.659+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba80b6be0 con 0x7fdba80c6be0 2024-01-15T17:58:19.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.660+0000 7fdba5600640 1 -- 172.21.15.26:0/1466703930 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fdb9804e450 con 0x7fdba80c6be0 2024-01-15T17:58:19.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.661+0000 7fdba5600640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:19.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.662+0000 7fdba5600640 1 -- 172.21.15.26:0/1466703930 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdb980ac300 con 0x7fdba80c6be0 2024-01-15T17:58:19.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.662+0000 7fdb9fe00640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:19.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.663+0000 7fdb9fe00640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8006b500 0x7fdb8006d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdba815fea0 tx=0x7fdb90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:19.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.666+0000 7fdba5600640 1 -- 172.21.15.26:0/1466703930 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdba80b6be0 con 0x7fdba80c6be0 2024-01-15T17:58:19.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.859+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdba80b6be0 con 0x7fdba80c6be0 2024-01-15T17:58:19.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.862+0000 7fdba5600640 1 -- 172.21.15.26:0/1466703930 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16545 (secure 0 0 0) 0x7fdba80b6be0 con 0x7fdba80c6be0 2024-01-15T17:58:19.866 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:19.866 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":46,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:58:18.442403+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:58:16.318608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-15T17:58:17.464623+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:58:19.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8006b500 msgr2=0x7fdb8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8006b500 0x7fdb8006d9c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdba815fea0 tx=0x7fdb90008040 comp rx=0 tx=0).stop 2024-01-15T17:58:19.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba80c6be0 msgr2=0x7fdba815dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:19.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba80c6be0 0x7fdba815dfb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdb98000c00 tx=0x7fdb980046c0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 shutdown_connections 2024-01-15T17:58:19.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdba8160c70 0x7fdba815ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8006b500 0x7fdb8006d9c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdba8160890 0x7fdba815e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 --2- 172.21.15.26:0/1466703930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdba80c6be0 0x7fdba815dfb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:19.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 >> 172.21.15.26:0/1466703930 conn(0x7fdba80d3090 msgr2=0x7fdba80b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:19.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 shutdown_connections 2024-01-15T17:58:19.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:19.865+0000 7fdbae600640 1 -- 172.21.15.26:0/1466703930 wait complete. 2024-01-15T17:58:19.887 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T17:58:19.931 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-01-15T17:58:19.928+0000 7fdd59200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T17:58:20.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.094+0000 7f8f93200640 1 Processor -- start 2024-01-15T17:58:20.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.094+0000 7f8f93200640 1 -- start start 2024-01-15T17:58:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.096+0000 7f8f93200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8f9413fcb0 0x7f8f940cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.096+0000 7f8f93200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f940ce440 con 0x7f8f940cdc20 2024-01-15T17:58:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.096+0000 7f8f93200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f940ce720 con 0x7f8f9413cf10 2024-01-15T17:58:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.096+0000 7f8f93200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f940cea00 con 0x7f8f9413fcb0 2024-01-15T17:58:20.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.096+0000 7f8f92800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8f940cdc20 0x7f8f940b2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39132/0 (socket says 172.21.15.26:39132) 2024-01-15T17:58:20.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.096+0000 7f8f92800640 1 -- 172.21.15.26:0/600085601 learned_addr learned my addr 172.21.15.26:0/600085601 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:20.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072729305 0 0) 0x7f8f940ce440 con 0x7f8f940cdc20 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f74003520 con 0x7f8f940cdc20 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f89200640 1 --2- 172.21.15.26:0/600085601 >> v2:172.21.15.124:3300/0 conn(0x7f8f9413fcb0 0x7f8f940cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 671061408 0 0) 0x7f8f74003520 con 0x7f8f940cdc20 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 >> v2:172.21.15.124:3300/0 conn(0x7f8f9413fcb0 msgr2=0x7f8f940cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 --2- 172.21.15.26:0/600085601 >> v2:172.21.15.124:3300/0 conn(0x7f8f9413fcb0 0x7f8f940cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 >> v1:172.21.15.124:6789/0 conn(0x7f8f9413cf10 legacy=0x7f8f9413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940b1d90 con 0x7f8f940cdc20 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8f8c001670 con 0x7f8f940cdc20 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 >> v1:172.21.15.26:6789/0 conn(0x7f8f940cdc20 legacy=0x7f8f940b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f74003970 0x7f8f74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740047c0 0x7f8f74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f74005180 0x7f8f7400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940cea00 con 0x7f8f740047c0 2024-01-15T17:58:20.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7400c0d0 con 0x7f8f74005180 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.097+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7400c290 con 0x7f8f74003970 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.098+0000 7f8f89200640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740047c0 0x7f8f74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.098+0000 7f8f89200640 1 -- 172.21.15.26:0/600085601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f74003970 msgr2=0x7f8f74003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.098+0000 7f8f89200640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f74003970 0x7f8f74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.098+0000 7f8f92800640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f74005180 0x7f8f7400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.098+0000 7f8f89200640 1 -- 172.21.15.26:0/600085601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f74005180 msgr2=0x7f8f7400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.098+0000 7f8f89200640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f74005180 0x7f8f7400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.098+0000 7f8f89200640 1 -- 172.21.15.26:0/600085601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f74003520 con 0x7f8f740047c0 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.098+0000 7f8f92800640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f74005180 0x7f8f7400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:20.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.098+0000 7f8f92800640 1 -- 172.21.15.26:0/600085601 reap_dead start 2024-01-15T17:58:20.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.099+0000 7f8f89200640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740047c0 0x7f8f74004c40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00ba30 tx=0x7f8f740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2282c756e757a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c032640 con 0x7f8f740047c0 2024-01-15T17:58:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c0327a0 con 0x7f8f740047c0 2024-01-15T17:58:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f91400640 1 -- 172.21.15.26:0/600085601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c032b30 con 0x7f8f740047c0 2024-01-15T17:58:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f93200640 1 -- 172.21.15.26:0/600085601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740047c0 msgr2=0x7f8f74004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f93200640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740047c0 0x7f8f74004c40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00ba30 tx=0x7f8f740057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f93200640 1 -- 172.21.15.26:0/600085601 shutdown_connections 2024-01-15T17:58:20.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f93200640 1 --2- 172.21.15.26:0/600085601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f740047c0 0x7f8f74004c40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:20.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f93200640 1 -- 172.21.15.26:0/600085601 >> 172.21.15.26:0/600085601 conn(0x7f8f9406a7d0 msgr2=0x7f8f9406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:20.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f93200640 1 -- 172.21.15.26:0/600085601 shutdown_connections 2024-01-15T17:58:20.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.100+0000 7f8f93200640 1 -- 172.21.15.26:0/600085601 wait complete. 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.102+0000 7f8f93200640 1 Processor -- start 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.102+0000 7f8f93200640 1 -- start start 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.103+0000 7f8f93200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f740047c0 0x7f8f94152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.103+0000 7f8f93200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9413ba10 0x7f8f940ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.103+0000 7f8f93200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f94156860 0x7f8f9415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.103+0000 7f8f93200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94157de0 con 0x7f8f94156860 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.103+0000 7f8f93200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94156d40 con 0x7f8f9413ba10 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.103+0000 7f8f93200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94157020 con 0x7f8f740047c0 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f92800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f94156860 0x7f8f9415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f92800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f94156860 0x7f8f9415c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60862/0 (socket says 172.21.15.26:60862) 2024-01-15T17:58:20.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f92800640 1 -- 172.21.15.26:0/2335940920 learned_addr learned my addr 172.21.15.26:0/2335940920 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f89200640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9413ba10 0x7f8f940ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f91e00640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f740047c0 0x7f8f94152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f92800640 1 -- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f740047c0 msgr2=0x7f8f94152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f92800640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f740047c0 0x7f8f94152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f92800640 1 -- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9413ba10 msgr2=0x7f8f940ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f92800640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9413ba10 0x7f8f940ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f92800640 1 -- 172.21.15.26:0/2335940920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94157300 con 0x7f8f94156860 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f89200640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9413ba10 0x7f8f940ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f91e00640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f740047c0 0x7f8f94152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.104+0000 7f8f92800640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f94156860 0x7f8f9415c5b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8f940ae9b0 tx=0x7f8f8c00d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.105+0000 7f8f8aa00640 1 -- 172.21.15.26:0/2335940920 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c007440 con 0x7f8f94156860 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.105+0000 7f8f8aa00640 1 -- 172.21.15.26:0/2335940920 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c0075c0 con 0x7f8f94156860 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.105+0000 7f8f8aa00640 1 -- 172.21.15.26:0/2335940920 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c004a20 con 0x7f8f94156860 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.105+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f941575e0 con 0x7f8f94156860 2024-01-15T17:58:20.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.105+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9415caf0 con 0x7f8f94156860 2024-01-15T17:58:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.106+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f94157b00 con 0x7f8f94156860 2024-01-15T17:58:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.107+0000 7f8f8aa00640 1 -- 172.21.15.26:0/2335940920 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f8f8c004b80 con 0x7f8f94156860 2024-01-15T17:58:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.108+0000 7f8f8aa00640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f6c06b500 0x7f8f6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.108+0000 7f8f8aa00640 1 -- 172.21.15.26:0/2335940920 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8f8c089e20 con 0x7f8f94156860 2024-01-15T17:58:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.108+0000 7f8f91e00640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f6c06b500 0x7f8f6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:20.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.110+0000 7f8f91e00640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f6c06b500 0x7f8f6c06d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8f94156650 tx=0x7f8f84009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:20.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.113+0000 7f8f8aa00640 1 -- 172.21.15.26:0/2335940920 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f94157b00 con 0x7f8f94156860 2024-01-15T17:58:20.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:20.345+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8f94157b00 con 0x7f8f94156860 2024-01-15T17:58:21.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.001+0000 7f8f8aa00640 1 -- 172.21.15.26:0/2335940920 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8f94157b00 con 0x7f8f94156860 2024-01-15T17:58:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f6c06b500 msgr2=0x7f8f6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f6c06b500 0x7f8f6c06d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8f94156650 tx=0x7f8f84009040 comp rx=0 tx=0).stop 2024-01-15T17:58:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f94156860 msgr2=0x7f8f9415c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f94156860 0x7f8f9415c5b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f8f940ae9b0 tx=0x7f8f8c00d950 comp rx=0 tx=0).stop 2024-01-15T17:58:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 shutdown_connections 2024-01-15T17:58:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f6c06b500 0x7f8f6c06d9c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f94156860 0x7f8f9415c5b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f9413ba10 0x7f8f940ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 --2- 172.21.15.26:0/2335940920 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f740047c0 0x7f8f94152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 >> 172.21.15.26:0/2335940920 conn(0x7f8f9406a7d0 msgr2=0x7f8f9406abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 shutdown_connections 2024-01-15T17:58:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.005+0000 7f8f93200640 1 -- 172.21.15.26:0/2335940920 wait complete. 2024-01-15T17:58:21.031 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T17:58:21.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T17:58:21.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.289+0000 7fadcbe00640 1 Processor -- start 2024-01-15T17:58:21.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.290+0000 7fadcbe00640 1 -- start start 2024-01-15T17:58:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.291+0000 7fadcbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fadcc0cf9a0 0x7fadcc0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.291+0000 7fadcbe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0d0320 con 0x7fadcc0b31d0 2024-01-15T17:58:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.291+0000 7fadcbe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc0d1c60 con 0x7fadcc15d4b0 2024-01-15T17:58:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.291+0000 7fadcbe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc0b2730 con 0x7fadcc0cf9a0 2024-01-15T17:58:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.292+0000 7fadca000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fadcc0cf9a0 0x7fadcc0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.292+0000 7fadca000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fadcc0cf9a0 0x7fadcc0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60872/0 (socket says 172.21.15.26:60872) 2024-01-15T17:58:21.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.292+0000 7fadca000640 1 -- 172.21.15.26:0/3444131099 learned_addr learned my addr 172.21.15.26:0/3444131099 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.292+0000 7fadca000640 1 -- 172.21.15.26:0/3444131099 >> v1:172.21.15.124:6789/0 conn(0x7fadcc15d4b0 legacy=0x7fadcc0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.292+0000 7fadca000640 1 -- 172.21.15.26:0/3444131099 >> v1:172.21.15.26:6789/0 conn(0x7fadcc0b31d0 legacy=0x7fadcc0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.292+0000 7fadca000640 1 -- 172.21.15.26:0/3444131099 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc0d1ec0 con 0x7fadcc0cf9a0 2024-01-15T17:58:21.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.292+0000 7fadc9600640 1 -- 172.21.15.26:0/3444131099 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185903462 0 0) 0x7fadcc0d0320 con 0x7fadcc0b31d0 2024-01-15T17:58:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.293+0000 7fadca000640 1 --2- 172.21.15.26:0/3444131099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0cf9a0 0x7fadcc0cfde0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009e50 tx=0x7fadb402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6619f3e44b4fdce1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.294+0000 7fadc9600640 1 -- 172.21.15.26:0/3444131099 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb402cbb0 con 0x7fadcc0cf9a0 2024-01-15T17:58:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.294+0000 7fadc9600640 1 -- 172.21.15.26:0/3444131099 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb402cd10 con 0x7fadcc0cf9a0 2024-01-15T17:58:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.294+0000 7fadc9600640 1 -- 172.21.15.26:0/3444131099 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4034610 con 0x7fadcc0cf9a0 2024-01-15T17:58:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.294+0000 7fadcbe00640 1 -- 172.21.15.26:0/3444131099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0cf9a0 msgr2=0x7fadcc0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:21.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.294+0000 7fadcbe00640 1 --2- 172.21.15.26:0/3444131099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0cf9a0 0x7fadcc0cfde0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009e50 tx=0x7fadb402c010 comp rx=0 tx=0).stop 2024-01-15T17:58:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.294+0000 7fadcbe00640 1 -- 172.21.15.26:0/3444131099 shutdown_connections 2024-01-15T17:58:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.294+0000 7fadcbe00640 1 --2- 172.21.15.26:0/3444131099 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0cf9a0 0x7fadcc0cfde0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.294+0000 7fadcbe00640 1 -- 172.21.15.26:0/3444131099 >> 172.21.15.26:0/3444131099 conn(0x7fadcc0d3000 msgr2=0x7fadcc0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.295+0000 7fadcbe00640 1 -- 172.21.15.26:0/3444131099 shutdown_connections 2024-01-15T17:58:21.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.295+0000 7fadcbe00640 1 -- 172.21.15.26:0/3444131099 wait complete. 2024-01-15T17:58:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.296+0000 7fadcbe00640 1 Processor -- start 2024-01-15T17:58:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.297+0000 7fadcbe00640 1 -- start start 2024-01-15T17:58:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.297+0000 7fadcbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0b31d0 0x7fadcc1f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.297+0000 7fadcbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadcc0cf9a0 0x7fadcc09f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcbe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fadcc15d4b0 0x7fadcc09fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0a0d50 con 0x7fadcc0b31d0 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcbe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0a0140 con 0x7fadcc15d4b0 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc0a0420 con 0x7fadcc0cf9a0 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0b31d0 0x7fadcc1f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0b31d0 0x7fadcc1f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60884/0 (socket says 172.21.15.26:60884) 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcaa00640 1 -- 172.21.15.26:0/3596100993 learned_addr learned my addr 172.21.15.26:0/3596100993 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcaa00640 1 -- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadcc0cf9a0 msgr2=0x7fadcc09f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcaa00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadcc0cf9a0 0x7fadcc09f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcaa00640 1 -- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fadcc15d4b0 msgr2=0x7fadcc09fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcaa00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fadcc15d4b0 0x7fadcc09fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.298+0000 7fadcaa00640 1 -- 172.21.15.26:0/3596100993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb4009b30 con 0x7fadcc0b31d0 2024-01-15T17:58:21.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.299+0000 7fadcaa00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0b31d0 0x7fadcc1f2a90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00c450 tx=0x7fadbc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.299+0000 7fadc2a00640 1 -- 172.21.15.26:0/3596100993 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0052d0 con 0x7fadcc0b31d0 2024-01-15T17:58:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.299+0000 7fadc2a00640 1 -- 172.21.15.26:0/3596100993 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc005430 con 0x7fadcc0b31d0 2024-01-15T17:58:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.299+0000 7fadc2a00640 1 -- 172.21.15.26:0/3596100993 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc012870 con 0x7fadcc0b31d0 2024-01-15T17:58:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.299+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc0a0700 con 0x7fadcc0b31d0 2024-01-15T17:58:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.299+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc15aa60 con 0x7fadcc0b31d0 2024-01-15T17:58:21.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.300+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc067450 con 0x7fadcc0b31d0 2024-01-15T17:58:21.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.301+0000 7fadc2a00640 1 -- 172.21.15.26:0/3596100993 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fadbc0129d0 con 0x7fadcc0b31d0 2024-01-15T17:58:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.302+0000 7fadc2a00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fada406b500 0x7fada406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.302+0000 7fadc2a00640 1 -- 172.21.15.26:0/3596100993 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fadbc089c10 con 0x7fadcc0b31d0 2024-01-15T17:58:21.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.303+0000 7fadca000640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fada406b500 0x7fada406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:21.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.304+0000 7fadca000640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fada406b500 0x7fada406d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fadb4035000 tx=0x7fadb402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:21.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.306+0000 7fadc2a00640 1 -- 172.21.15.26:0/3596100993 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadbc054660 con 0x7fadcc0b31d0 2024-01-15T17:58:21.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:21.536+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fadcc15cbc0 con 0x7fadcc0b31d0 2024-01-15T17:58:22.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.014+0000 7fadc2a00640 1 -- 172.21.15.26:0/3596100993 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7fadbc057c80 con 0x7fadcc0b31d0 2024-01-15T17:58:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fada406b500 msgr2=0x7fada406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fada406b500 0x7fada406d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fadb4035000 tx=0x7fadb402e000 comp rx=0 tx=0).stop 2024-01-15T17:58:22.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0b31d0 msgr2=0x7fadcc1f2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0b31d0 0x7fadcc1f2a90 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00c450 tx=0x7fadbc00c910 comp rx=0 tx=0).stop 2024-01-15T17:58:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 shutdown_connections 2024-01-15T17:58:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fadcc0cf9a0 0x7fadcc09f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fada406b500 0x7fada406d9c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fadcc15d4b0 0x7fadcc09fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 --2- 172.21.15.26:0/3596100993 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fadcc0b31d0 0x7fadcc1f2a90 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 >> 172.21.15.26:0/3596100993 conn(0x7fadcc0d3000 msgr2=0x7fadcc0b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 shutdown_connections 2024-01-15T17:58:22.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.018+0000 7fadcbe00640 1 -- 172.21.15.26:0/3596100993 wait complete. 2024-01-15T17:58:22.043 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T17:58:22.044 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:58:22.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.267+0000 7f5650c00640 1 Processor -- start 2024-01-15T17:58:22.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.268+0000 7f5650c00640 1 -- start start 2024-01-15T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.269+0000 7f5650c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071f00 0x7f564c146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.269+0000 7f5650c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f564c06c7c0 0x7f564c166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.269+0000 7f5650c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564c0b2e10 con 0x7f564c071410 2024-01-15T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.270+0000 7f5650c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564c06cec0 con 0x7f564c06c7c0 2024-01-15T17:58:22.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.270+0000 7f5650c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c06d1a0 con 0x7f564c071f00 2024-01-15T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.270+0000 7f5643e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071f00 0x7f564c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.270+0000 7f5643e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071f00 0x7f564c146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:35494/0 (socket says 172.21.15.26:35494) 2024-01-15T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.270+0000 7f5643e00640 1 -- 172.21.15.26:0/4142744547 learned_addr learned my addr 172.21.15.26:0/4142744547 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.271+0000 7f564be00640 1 --2- 172.21.15.26:0/4142744547 >> v2:172.21.15.124:3300/0 conn(0x7f564c06c7c0 0x7f564c166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.271+0000 7f5643e00640 1 -- 172.21.15.26:0/4142744547 >> v2:172.21.15.124:3300/0 conn(0x7f564c06c7c0 msgr2=0x7f564c166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.271+0000 7f5643e00640 1 --2- 172.21.15.26:0/4142744547 >> v2:172.21.15.124:3300/0 conn(0x7f564c06c7c0 0x7f564c166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.271+0000 7f5643e00640 1 -- 172.21.15.26:0/4142744547 >> v1:172.21.15.124:6789/0 conn(0x7f564c071410 legacy=0x7f564c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.271+0000 7f5643e00640 1 -- 172.21.15.26:0/4142744547 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c06d480 con 0x7f564c071f00 2024-01-15T17:58:22.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.272+0000 7f5643e00640 1 --2- 172.21.15.26:0/4142744547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071f00 0x7f564c146770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5634009870 tx=0x7f563402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33607803374d3383 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.273+0000 7f564aa00640 1 -- 172.21.15.26:0/4142744547 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634030840 con 0x7f564c071f00 2024-01-15T17:58:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.273+0000 7f564aa00640 1 -- 172.21.15.26:0/4142744547 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56340309a0 con 0x7f564c071f00 2024-01-15T17:58:22.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.273+0000 7f564aa00640 1 -- 172.21.15.26:0/4142744547 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634030d30 con 0x7f564c071f00 2024-01-15T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.274+0000 7f5650c00640 1 -- 172.21.15.26:0/4142744547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071f00 msgr2=0x7f564c146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.274+0000 7f5650c00640 1 --2- 172.21.15.26:0/4142744547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071f00 0x7f564c146770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5634009870 tx=0x7f563402b980 comp rx=0 tx=0).stop 2024-01-15T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.274+0000 7f5650c00640 1 -- 172.21.15.26:0/4142744547 shutdown_connections 2024-01-15T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.274+0000 7f5650c00640 1 --2- 172.21.15.26:0/4142744547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071f00 0x7f564c146770 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.274+0000 7f5650c00640 1 --2- 172.21.15.26:0/4142744547 >> v2:172.21.15.124:3300/0 conn(0x7f564c06c7c0 0x7f564c166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.274+0000 7f5650c00640 1 -- 172.21.15.26:0/4142744547 >> 172.21.15.26:0/4142744547 conn(0x7f564c06a800 msgr2=0x7f564c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:22.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.274+0000 7f5650c00640 1 -- 172.21.15.26:0/4142744547 shutdown_connections 2024-01-15T17:58:22.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.274+0000 7f5650c00640 1 -- 172.21.15.26:0/4142744547 wait complete. 2024-01-15T17:58:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.276+0000 7f5650c00640 1 Processor -- start 2024-01-15T17:58:22.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.276+0000 7f5650c00640 1 -- start start 2024-01-15T17:58:22.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f5650c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f564c06c7c0 0x7f564c15c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f5650c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071410 0x7f564c1451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f5650c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c071f00 0x7f564c140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f5650c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c1f4770 con 0x7f564c06c7c0 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f5650c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c1f3af0 con 0x7f564c071f00 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f5650c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564c1f3dd0 con 0x7f564c071410 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f564b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f564c06c7c0 0x7f564c15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f564b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f564c06c7c0 0x7f564c15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57506/0 (socket says 172.21.15.26:57506) 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f564b400640 1 -- 172.21.15.26:0/397779865 learned_addr learned my addr 172.21.15.26:0/397779865 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f5643e00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071410 0x7f564c1451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f564be00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c071f00 0x7f564c140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f564b400640 1 -- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071410 msgr2=0x7f564c1451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f564b400640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071410 0x7f564c1451d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.278+0000 7f564b400640 1 -- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c071f00 msgr2=0x7f564c140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.279+0000 7f564b400640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c071f00 0x7f564c140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.279+0000 7f564b400640 1 -- 172.21.15.26:0/397779865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340094d0 con 0x7f564c06c7c0 2024-01-15T17:58:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.279+0000 7f564b400640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f564c06c7c0 0x7f564c15c910 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f563c00c450 tx=0x7f563c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:22.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.279+0000 7f5643400640 1 -- 172.21.15.26:0/397779865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c0052d0 con 0x7f564c06c7c0 2024-01-15T17:58:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.279+0000 7f5643400640 1 -- 172.21.15.26:0/397779865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c005430 con 0x7f564c06c7c0 2024-01-15T17:58:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.279+0000 7f5643400640 1 -- 172.21.15.26:0/397779865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c012870 con 0x7f564c06c7c0 2024-01-15T17:58:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.279+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c1f40b0 con 0x7f564c06c7c0 2024-01-15T17:58:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.279+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c145710 con 0x7f564c06c7c0 2024-01-15T17:58:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.281+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c067450 con 0x7f564c06c7c0 2024-01-15T17:58:22.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.282+0000 7f5643400640 1 -- 172.21.15.26:0/397779865 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f563c0129d0 con 0x7f564c06c7c0 2024-01-15T17:58:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.282+0000 7f5643400640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f562406b530 0x7f562406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.282+0000 7f5643400640 1 -- 172.21.15.26:0/397779865 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f563c089bb0 con 0x7f564c06c7c0 2024-01-15T17:58:22.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.283+0000 7f5643e00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f562406b530 0x7f562406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.284+0000 7f5643e00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f562406b530 0x7f562406d9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5634036000 tx=0x7f563403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:22.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.287+0000 7f5643400640 1 -- 172.21.15.26:0/397779865 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f563c054600 con 0x7f564c06c7c0 2024-01-15T17:58:22.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.523+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f564c06b3f0 con 0x7f564c06c7c0 2024-01-15T17:58:22.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.524+0000 7f5643400640 1 -- 172.21.15.26:0/397779865 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f563c057c20 con 0x7f564c06c7c0 2024-01-15T17:58:22.527 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:22.527 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5218,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3293492366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3293492366},{"type":"v1","addr":"172.21.15.124:6836","nonce":3293492366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1049920857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":1049920857},{"type":"v1","addr":"172.21.15.124:6837","nonce":1049920857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2644667232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2644667232},{"type":"v1","addr":"172.21.15.26:6834","nonce":2644667232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:58:18.443247+0000","modified":"2024-01-15T17:58:21.537724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5802":{"gid":5802,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1948779240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1948779240},{"type":"v1","addr":"172.21.15.26:6835","nonce":1948779240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-15T17:58:22.527 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2024-01-15T17:58:22.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f562406b530 msgr2=0x7f562406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f562406b530 0x7f562406d9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5634036000 tx=0x7f563403b040 comp rx=0 tx=0).stop 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f564c06c7c0 msgr2=0x7f564c15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f564c06c7c0 0x7f564c15c910 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f563c00c450 tx=0x7f563c00c910 comp rx=0 tx=0).stop 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 shutdown_connections 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f564c071410 0x7f564c1451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f562406b530 0x7f562406d9f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f564c071f00 0x7f564c140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 --2- 172.21.15.26:0/397779865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f564c06c7c0 0x7f564c15c910 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 >> 172.21.15.26:0/397779865 conn(0x7f564c06a800 msgr2=0x7f564c0717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 shutdown_connections 2024-01-15T17:58:22.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.527+0000 7f5650c00640 1 -- 172.21.15.26:0/397779865 wait complete. 2024-01-15T17:58:22.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T17:58:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.756+0000 7fa697e00640 1 Processor -- start 2024-01-15T17:58:22.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.756+0000 7fa697e00640 1 -- start start 2024-01-15T17:58:22.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.758+0000 7fa697e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 0x7fa698071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.758+0000 7fa697e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa6980717d0 0x7fa6980cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.758+0000 7fa697e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa698070e50 con 0x7fa6980cca40 2024-01-15T17:58:22.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.758+0000 7fa697e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6980723c0 con 0x7fa6980717d0 2024-01-15T17:58:22.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.758+0000 7fa697e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980cce70 con 0x7fa698143c40 2024-01-15T17:58:22.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.759+0000 7fa696000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa6980717d0 0x7fa6980cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.759+0000 7fa697400640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fa6980cca40 0x7fa69815eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:49496/0 (socket says 172.21.15.26:49496) 2024-01-15T17:58:22.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.759+0000 7fa696a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 0x7fa698071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.759+0000 7fa697400640 1 -- 172.21.15.26:0/1569791529 learned_addr learned my addr 172.21.15.26:0/1569791529 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:22.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.760+0000 7fa696a00640 1 -- 172.21.15.26:0/1569791529 >> v2:172.21.15.124:3300/0 conn(0x7fa6980717d0 msgr2=0x7fa6980cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:22.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.760+0000 7fa696a00640 1 --2- 172.21.15.26:0/1569791529 >> v2:172.21.15.124:3300/0 conn(0x7fa6980717d0 0x7fa6980cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.760+0000 7fa696a00640 1 -- 172.21.15.26:0/1569791529 >> v1:172.21.15.124:6789/0 conn(0x7fa6980cca40 legacy=0x7fa69815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.760+0000 7fa696a00640 1 -- 172.21.15.26:0/1569791529 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69806e170 con 0x7fa698143c40 2024-01-15T17:58:22.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.761+0000 7fa696a00640 1 --2- 172.21.15.26:0/1569791529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 0x7fa698071290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6880097a0 tx=0x7fa68802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86df841f5d749a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.762+0000 7fa695600640 1 -- 172.21.15.26:0/1569791529 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68802e070 con 0x7fa698143c40 2024-01-15T17:58:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.762+0000 7fa695600640 1 -- 172.21.15.26:0/1569791529 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6880315a0 con 0x7fa698143c40 2024-01-15T17:58:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.762+0000 7fa695600640 1 -- 172.21.15.26:0/1569791529 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688031930 con 0x7fa698143c40 2024-01-15T17:58:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.762+0000 7fa697e00640 1 -- 172.21.15.26:0/1569791529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 msgr2=0x7fa698071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.763+0000 7fa697e00640 1 --2- 172.21.15.26:0/1569791529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 0x7fa698071290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa6880097a0 tx=0x7fa68802b840 comp rx=0 tx=0).stop 2024-01-15T17:58:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.763+0000 7fa697e00640 1 -- 172.21.15.26:0/1569791529 shutdown_connections 2024-01-15T17:58:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.763+0000 7fa697e00640 1 --2- 172.21.15.26:0/1569791529 >> v2:172.21.15.124:3300/0 conn(0x7fa6980717d0 0x7fa6980cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.763+0000 7fa697e00640 1 --2- 172.21.15.26:0/1569791529 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 0x7fa698071290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.763+0000 7fa697e00640 1 -- 172.21.15.26:0/1569791529 >> 172.21.15.26:0/1569791529 conn(0x7fa69806a800 msgr2=0x7fa69806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.763+0000 7fa697e00640 1 -- 172.21.15.26:0/1569791529 shutdown_connections 2024-01-15T17:58:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.763+0000 7fa697e00640 1 -- 172.21.15.26:0/1569791529 wait complete. 2024-01-15T17:58:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.765+0000 7fa697e00640 1 Processor -- start 2024-01-15T17:58:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.765+0000 7fa697e00640 1 -- start start 2024-01-15T17:58:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.766+0000 7fa697e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6980717d0 0x7fa6980adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.766+0000 7fa697e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6980cca40 0x7fa69813d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.766+0000 7fa697e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 0x7fa69813dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.766+0000 7fa697e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69813cb60 con 0x7fa6980717d0 2024-01-15T17:58:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.766+0000 7fa697e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69813bc70 con 0x7fa6980cca40 2024-01-15T17:58:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.766+0000 7fa697e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69813bf50 con 0x7fa698143c40 2024-01-15T17:58:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa696a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6980717d0 0x7fa6980adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa696a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6980717d0 0x7fa6980adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57516/0 (socket says 172.21.15.26:57516) 2024-01-15T17:58:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa696a00640 1 -- 172.21.15.26:0/3827380391 learned_addr learned my addr 172.21.15.26:0/3827380391 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa697400640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 0x7fa69813dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa696000640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6980cca40 0x7fa69813d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa696a00640 1 -- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 msgr2=0x7fa69813dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa696a00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 0x7fa69813dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa696a00640 1 -- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6980cca40 msgr2=0x7fa69813d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa696a00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6980cca40 0x7fa69813d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.767+0000 7fa696a00640 1 -- 172.21.15.26:0/3827380391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688009400 con 0x7fa6980717d0 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.768+0000 7fa696a00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6980717d0 0x7fa6980adc60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa688034040 tx=0x7fa6880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.768+0000 7fa68ea00640 1 -- 172.21.15.26:0/3827380391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688032840 con 0x7fa6980717d0 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.768+0000 7fa68ea00640 1 -- 172.21.15.26:0/3827380391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa688031f00 con 0x7fa6980717d0 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.768+0000 7fa68ea00640 1 -- 172.21.15.26:0/3827380391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688032be0 con 0x7fa6980717d0 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.768+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69813c230 con 0x7fa6980717d0 2024-01-15T17:58:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.768+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69813c7f0 con 0x7fa6980717d0 2024-01-15T17:58:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.769+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa698067450 con 0x7fa6980717d0 2024-01-15T17:58:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.770+0000 7fa68ea00640 1 -- 172.21.15.26:0/3827380391 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fa688032d40 con 0x7fa6980717d0 2024-01-15T17:58:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.772+0000 7fa68ea00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa67006b500 0x7fa67006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.772+0000 7fa68ea00640 1 -- 172.21.15.26:0/3827380391 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa6880ac1a0 con 0x7fa6980717d0 2024-01-15T17:58:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.772+0000 7fa696000640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa67006b500 0x7fa67006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.773+0000 7fa696000640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa67006b500 0x7fa67006d9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa6981f1c30 tx=0x7fa680008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:22.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.776+0000 7fa68ea00640 1 -- 172.21.15.26:0/3827380391 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa688076b00 con 0x7fa6980717d0 2024-01-15T17:58:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.971+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6980ceb30 con 0x7fa6980717d0 2024-01-15T17:58:22.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.974+0000 7fa68ea00640 1 -- 172.21.15.26:0/3827380391 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16532 (secure 0 0 0) 0x7fa68807a120 con 0x7fa6980717d0 2024-01-15T17:58:22.978 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:22.978 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":47,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T17:58:19.972833+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-01-15T17:58:16.318608+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-01-15T17:58:17.464623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T17:58:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa67006b500 msgr2=0x7fa67006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa67006b500 0x7fa67006d9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa6981f1c30 tx=0x7fa680008040 comp rx=0 tx=0).stop 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6980717d0 msgr2=0x7fa6980adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6980717d0 0x7fa6980adc60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa688034040 tx=0x7fa6880046c0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 shutdown_connections 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa698143c40 0x7fa69813dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa67006b500 0x7fa67006d9c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6980cca40 0x7fa69813d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 --2- 172.21.15.26:0/3827380391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6980717d0 0x7fa6980adc60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.977+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 >> 172.21.15.26:0/3827380391 conn(0x7fa69806a800 msgr2=0x7fa69815a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.978+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 shutdown_connections 2024-01-15T17:58:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:22.978+0000 7fa697e00640 1 -- 172.21.15.26:0/3827380391 wait complete. 2024-01-15T17:58:22.999 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T17:58:22.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:58:23.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.198+0000 7f0d0e600640 1 Processor -- start 2024-01-15T17:58:23.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.198+0000 7f0d0e600640 1 -- start start 2024-01-15T17:58:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.200+0000 7f0d0e600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d08143c40 0x7f0d08071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.200+0000 7f0d0e600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0d080ccc10 0x7f0d0806d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.200+0000 7f0d0e600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d08070e50 con 0x7f0d080717d0 2024-01-15T17:58:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.200+0000 7f0d0e600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d080723c0 con 0x7f0d080ccc10 2024-01-15T17:58:23.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.200+0000 7f0d0e600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d08144020 con 0x7f0d08143c40 2024-01-15T17:58:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.200+0000 7f0d0d200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d08143c40 0x7f0d08071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.201+0000 7f0d0d200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d08143c40 0x7f0d08071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:35518/0 (socket says 172.21.15.26:35518) 2024-01-15T17:58:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.201+0000 7f0d0dc00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0d080ccc10 0x7f0d0806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.201+0000 7f0d0d200640 1 -- 172.21.15.26:0/878769078 learned_addr learned my addr 172.21.15.26:0/878769078 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:23.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.201+0000 7f0d0dc00640 1 -- 172.21.15.26:0/878769078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d08143c40 msgr2=0x7f0d08071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:23.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.201+0000 7f0d0dc00640 1 --2- 172.21.15.26:0/878769078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d08143c40 0x7f0d08071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.201+0000 7f0d0dc00640 1 -- 172.21.15.26:0/878769078 >> v1:172.21.15.124:6789/0 conn(0x7f0d080717d0 legacy=0x7f0d080cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.201+0000 7f0d0dc00640 1 -- 172.21.15.26:0/878769078 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08071e00 con 0x7f0d080ccc10 2024-01-15T17:58:23.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.201+0000 7f0d0d200640 1 --2- 172.21.15.26:0/878769078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d08143c40 0x7f0d08071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:58:23.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.203+0000 7f0d0dc00640 1 --2- 172.21.15.26:0/878769078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d080ccc10 0x7f0d0806d160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009d90 tx=0x7f0d0002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c86f6390f1cff6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:23.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.203+0000 7f0d07e00640 1 -- 172.21.15.26:0/878769078 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0002cbc0 con 0x7f0d080ccc10 2024-01-15T17:58:23.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.203+0000 7f0d07e00640 1 -- 172.21.15.26:0/878769078 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0002cd20 con 0x7f0d080ccc10 2024-01-15T17:58:23.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.203+0000 7f0d07e00640 1 -- 172.21.15.26:0/878769078 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d00034660 con 0x7f0d080ccc10 2024-01-15T17:58:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.204+0000 7f0d0e600640 1 -- 172.21.15.26:0/878769078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d080ccc10 msgr2=0x7f0d0806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.204+0000 7f0d0e600640 1 --2- 172.21.15.26:0/878769078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d080ccc10 0x7f0d0806d160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009d90 tx=0x7f0d0002be10 comp rx=0 tx=0).stop 2024-01-15T17:58:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.204+0000 7f0d0e600640 1 -- 172.21.15.26:0/878769078 shutdown_connections 2024-01-15T17:58:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.204+0000 7f0d0e600640 1 --2- 172.21.15.26:0/878769078 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d080ccc10 0x7f0d0806d160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.204+0000 7f0d0e600640 1 --2- 172.21.15.26:0/878769078 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d08143c40 0x7f0d08071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.204+0000 7f0d0e600640 1 -- 172.21.15.26:0/878769078 >> 172.21.15.26:0/878769078 conn(0x7f0d0806a800 msgr2=0x7f0d0806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.204+0000 7f0d0e600640 1 -- 172.21.15.26:0/878769078 shutdown_connections 2024-01-15T17:58:23.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.204+0000 7f0d0e600640 1 -- 172.21.15.26:0/878769078 wait complete. 2024-01-15T17:58:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.206+0000 7f0d0e600640 1 Processor -- start 2024-01-15T17:58:23.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.206+0000 7f0d0e600640 1 -- start start 2024-01-15T17:58:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0e600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d080717d0 0x7f0d080ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0e600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d080ccc10 0x7f0d0812d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0e600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d08143c40 0x7f0d081f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0e600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080acbe0 con 0x7f0d08143c40 2024-01-15T17:58:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0e600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d080ac8d0 con 0x7f0d080717d0 2024-01-15T17:58:23.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0e600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0812d980 con 0x7f0d080ccc10 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d08143c40 0x7f0d081f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d08143c40 0x7f0d081f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57518/0 (socket says 172.21.15.26:57518) 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0dc00640 1 -- 172.21.15.26:0/3305397547 learned_addr learned my addr 172.21.15.26:0/3305397547 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0cffc00640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d080ccc10 0x7f0d0812d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.208+0000 7f0d0d200640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d080717d0 0x7f0d080ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.209+0000 7f0d0dc00640 1 -- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d080ccc10 msgr2=0x7f0d0812d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.209+0000 7f0d0dc00640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d080ccc10 0x7f0d0812d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.209+0000 7f0d0dc00640 1 -- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d080717d0 msgr2=0x7f0d080ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.209+0000 7f0d0dc00640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d080717d0 0x7f0d080ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.209+0000 7f0d0dc00640 1 -- 172.21.15.26:0/3305397547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf80094d0 con 0x7f0d08143c40 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.209+0000 7f0d0dc00640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d08143c40 0x7f0d081f2690 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009eb0 tx=0x7f0d00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.209+0000 7f0d05600640 1 -- 172.21.15.26:0/3305397547 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0003e610 con 0x7f0d08143c40 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.209+0000 7f0d05600640 1 -- 172.21.15.26:0/3305397547 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0002ce90 con 0x7f0d08143c40 2024-01-15T17:58:23.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.209+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d00009a70 con 0x7f0d08143c40 2024-01-15T17:58:23.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.210+0000 7f0d05600640 1 -- 172.21.15.26:0/3305397547 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0003e9b0 con 0x7f0d08143c40 2024-01-15T17:58:23.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.210+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0812dee0 con 0x7f0d08143c40 2024-01-15T17:58:23.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.211+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d08067450 con 0x7f0d08143c40 2024-01-15T17:58:23.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.212+0000 7f0d05600640 1 -- 172.21.15.26:0/3305397547 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f0d0003b050 con 0x7f0d08143c40 2024-01-15T17:58:23.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.213+0000 7f0d05600640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ce006b500 0x7f0ce006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.213+0000 7f0d05600640 1 -- 172.21.15.26:0/3305397547 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0d000ac5f0 con 0x7f0d08143c40 2024-01-15T17:58:23.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.213+0000 7f0d0d200640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ce006b500 0x7f0ce006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.214+0000 7f0d0d200640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ce006b500 0x7f0ce006d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006ca0 tx=0x7f0cf8009240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:23.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.217+0000 7f0d05600640 1 -- 172.21.15.26:0/3305397547 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d00077340 con 0x7f0d08143c40 2024-01-15T17:58:23.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.397+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d0806cc40 con 0x7f0ce006b500 2024-01-15T17:58:23.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.424+0000 7f0d05600640 1 -- 172.21.15.26:0/3305397547 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274146 (secure 0 0 0) 0x7f0cf8009940 con 0x7f0ce006b500 2024-01-15T17:58:23.431 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:23.432 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:58:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ce006b500 msgr2=0x7f0ce006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ce006b500 0x7f0ce006d9c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8006ca0 tx=0x7f0cf8009240 comp rx=0 tx=0).stop 2024-01-15T17:58:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d08143c40 msgr2=0x7f0d081f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d08143c40 0x7f0d081f2690 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009eb0 tx=0x7f0d00004a40 comp rx=0 tx=0).stop 2024-01-15T17:58:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 shutdown_connections 2024-01-15T17:58:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0d080ccc10 0x7f0d0812d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0ce006b500 0x7f0ce006d9c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0d08143c40 0x7f0d081f2690 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 --2- 172.21.15.26:0/3305397547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0d080717d0 0x7f0d080ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 >> 172.21.15.26:0/3305397547 conn(0x7f0d0806a800 msgr2=0x7f0d0806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:23.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 shutdown_connections 2024-01-15T17:58:23.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.430+0000 7f0d0e600640 1 -- 172.21.15.26:0/3305397547 wait complete. 2024-01-15T17:58:23.450 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-01-15T17:58:21.979139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":267048,"kb_used_data":5076,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754707672,"statfs":{"total":773094113280,"available":772820656128,"internally_reserved":0,"allocated":5197824,"data_stored":2781245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.042459"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.941701+0000","last_change":"2024-01-15T17:58:16.941701+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.941701+0000","last_undegraded":"2024-01-15T17:58:16.941701+0000","last_fullsized":"2024-01-15T17:58:16.941701+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942072+0000","last_change":"2024-01-15T17:58:16.942072+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942072+0000","last_undegraded":"2024-01-15T17:58:16.942072+0000","last_fullsized":"2024-01-15T17:58:16.942072+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.948787+0000","last_change":"2024-01-15T17:58:16.948787+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.948787+0000","last_undegraded":"2024-01-15T17:58:16.948787+0000","last_fullsized":"2024-01-15T17:58:16.948787+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.953681+0000","last_change":"2024-01-15T17:58:16.953681+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.953681+0000","last_undegraded":"2024-01-15T17:58:16.953681+0000","last_fullsized":"2024-01-15T17:58:16.953681+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942609+0000","last_change":"2024-01-15T17:58:16.942609+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942609+0000","last_undegraded":"2024-01-15T17:58:16.942609+0000","last_fullsized":"2024-01-15T17:58:16.942609+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942183+0000","last_change":"2024-01-15T17:58:16.942183+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942183+0000","last_undegraded":"2024-01-15T17:58:16.942183+0000","last_fullsized":"2024-01-15T17:58:16.942183+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949897+0000","last_change":"2024-01-15T17:58:16.949897+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949897+0000","last_undegraded":"2024-01-15T17:58:16.949897+0000","last_fullsized":"2024-01-15T17:58:16.949897+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.948193+0000","last_change":"2024-01-15T17:58:16.948193+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.948193+0000","last_undegraded":"2024-01-15T17:58:16.948193+0000","last_fullsized":"2024-01-15T17:58:16.948193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.940884+0000","last_change":"2024-01-15T17:58:16.940884+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.940884+0000","last_undegraded":"2024-01-15T17:58:16.940884+0000","last_fullsized":"2024-01-15T17:58:16.940884+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.946664+0000","last_change":"2024-01-15T17:58:16.946664+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.946664+0000","last_undegraded":"2024-01-15T17:58:16.946664+0000","last_fullsized":"2024-01-15T17:58:16.946664+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.955768+0000","last_change":"2024-01-15T17:58:16.955768+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.955768+0000","last_undegraded":"2024-01-15T17:58:16.955768+0000","last_fullsized":"2024-01-15T17:58:16.955768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.940096+0000","last_change":"2024-01-15T17:58:16.940096+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.940096+0000","last_undegraded":"2024-01-15T17:58:16.940096+0000","last_fullsized":"2024-01-15T17:58:16.940096+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942131+0000","last_change":"2024-01-15T17:58:16.942131+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942131+0000","last_undegraded":"2024-01-15T17:58:16.942131+0000","last_fullsized":"2024-01-15T17:58:16.942131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.955523+0000","last_change":"2024-01-15T17:58:16.955523+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.955523+0000","last_undegraded":"2024-01-15T17:58:16.955523+0000","last_fullsized":"2024-01-15T17:58:16.955523+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.941797+0000","last_change":"2024-01-15T17:58:16.941797+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.941797+0000","last_undegraded":"2024-01-15T17:58:16.941797+0000","last_fullsized":"2024-01-15T17:58:16.941797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.947364+0000","last_change":"2024-01-15T17:58:16.947364+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.947364+0000","last_undegraded":"2024-01-15T17:58:16.947364+0000","last_fullsized":"2024-01-15T17:58:16.947364+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.939468+0000","last_change":"2024-01-15T17:58:16.939468+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.939468+0000","last_undegraded":"2024-01-15T17:58:16.939468+0000","last_fullsized":"2024-01-15T17:58:16.939468+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.940707+0000","last_change":"2024-01-15T17:58:16.940707+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.940707+0000","last_undegraded":"2024-01-15T17:58:16.940707+0000","last_fullsized":"2024-01-15T17:58:16.940707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949966+0000","last_change":"2024-01-15T17:58:16.949966+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949966+0000","last_undegraded":"2024-01-15T17:58:16.949966+0000","last_fullsized":"2024-01-15T17:58:16.949966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.941821+0000","last_change":"2024-01-15T17:58:16.941821+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.941821+0000","last_undegraded":"2024-01-15T17:58:16.941821+0000","last_fullsized":"2024-01-15T17:58:16.941821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.941556+0000","last_change":"2024-01-15T17:58:16.941556+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.941556+0000","last_undegraded":"2024-01-15T17:58:16.941556+0000","last_fullsized":"2024-01-15T17:58:16.941556+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949183+0000","last_change":"2024-01-15T17:58:16.949183+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949183+0000","last_undegraded":"2024-01-15T17:58:16.949183+0000","last_fullsized":"2024-01-15T17:58:16.949183+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.946623+0000","last_change":"2024-01-15T17:58:16.946623+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.946623+0000","last_undegraded":"2024-01-15T17:58:16.946623+0000","last_fullsized":"2024-01-15T17:58:16.946623+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949918+0000","last_change":"2024-01-15T17:58:16.949918+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949918+0000","last_undegraded":"2024-01-15T17:58:16.949918+0000","last_fullsized":"2024-01-15T17:58:16.949918+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942644+0000","last_change":"2024-01-15T17:58:16.942644+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942644+0000","last_undegraded":"2024-01-15T17:58:16.942644+0000","last_fullsized":"2024-01-15T17:58:16.942644+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.953143+0000","last_change":"2024-01-15T17:58:16.953143+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.953143+0000","last_undegraded":"2024-01-15T17:58:16.953143+0000","last_fullsized":"2024-01-15T17:58:16.953143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.948646+0000","last_change":"2024-01-15T17:58:16.948646+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.948646+0000","last_undegraded":"2024-01-15T17:58:16.948646+0000","last_fullsized":"2024-01-15T17:58:16.948646+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.953154+0000","last_change":"2024-01-15T17:58:16.953154+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.953154+0000","last_undegraded":"2024-01-15T17:58:16.953154+0000","last_fullsized":"2024-01-15T17:58:16.953154+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.940882+0000","last_change":"2024-01-15T17:58:16.940882+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.940882+0000","last_undegraded":"2024-01-15T17:58:16.940882+0000","last_fullsized":"2024-01-15T17:58:16.940882+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949783+0000","last_change":"2024-01-15T17:58:16.949783+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949783+0000","last_undegraded":"2024-01-15T17:58:16.949783+0000","last_fullsized":"2024-01-15T17:58:16.949783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.946633+0000","last_change":"2024-01-15T17:58:16.946633+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.946633+0000","last_undegraded":"2024-01-15T17:58:16.946633+0000","last_fullsized":"2024-01-15T17:58:16.946633+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949542+0000","last_change":"2024-01-15T17:58:16.949542+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949542+0000","last_undegraded":"2024-01-15T17:58:16.949542+0000","last_fullsized":"2024-01-15T17:58:16.949542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-15T17:58:14.763036+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:58:14.763036+0000","last_peered":"2024-01-15T17:58:14.763036+0000","last_clean":"2024-01-15T17:58:14.763036+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:58:14.763036+0000","last_undegraded":"2024-01-15T17:58:14.763036+0000","last_fullsized":"2024-01-15T17:58:14.763036+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1093632,"data_stored":789701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1559999999999999,"15min":1.1539999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5700000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.175,"15min":1.1679999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.6619999999999999,"5min":1.804,"15min":1.804},"last":1.3560000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.177},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.274,"15min":1.262},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.619}]},{"osd":2,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.3460000000000001,"15min":1.3759999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.504}]},{"osd":3,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.294,"15min":1.298},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.385},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.4890000000000001,"15min":1.4930000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":3.831,"5min":3.831,"15min":3.831},"last":1.5820000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4350000000000001,"15min":1.48},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.1890000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.593,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":4.4729999999999999,"5min":4.4729999999999999,"15min":4.4729999999999999},"last":1.2170000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.6000000000000001,"15min":1.605},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1589999999999998,"5min":7.3819999999999997,"15min":7.3819999999999997},"last":1.675},{"interface":"front","average":{"1min":1.456,"5min":1.548,"15min":1.5860000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.1230000000000002,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4450000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.649,"15min":1.6919999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9219999999999999,"5min":2.556,"15min":2.556},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5880000000000001,"15min":1.625},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1400000000000001,"5min":2.649,"15min":2.649},"last":1.3140000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.409,"15min":1.3149999999999999},"min":{"1min":0.753,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":12.544,"5min":12.544,"15min":12.544},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.206,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.165}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.29,"15min":1.202},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":12.811999999999999,"5min":12.811999999999999,"15min":12.811999999999999},"last":1.655},{"interface":"front","average":{"1min":1.206,"5min":1.1870000000000001,"15min":1.179},"min":{"1min":0.83899999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4990000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.875}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.214,"15min":1.2},"min":{"1min":0.94299999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5549999999999999,"5min":1.788,"15min":1.788},"last":1.101},{"interface":"front","average":{"1min":2.0169999999999999,"5min":1.5680000000000001,"15min":1.5269999999999999},"min":{"1min":0.96899999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":12.897,"5min":12.897,"15min":12.897},"last":1.0209999999999999}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.169,"5min":1.597,"15min":1.5069999999999999},"min":{"1min":1.23,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":12.717000000000001,"5min":12.717000000000001,"15min":12.717000000000001},"last":2.048},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.95399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.97,"5min":2.29,"15min":2.29},"last":1.468}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.90700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6839999999999999,"5min":2.343,"15min":2.343},"last":1.986},{"interface":"front","average":{"1min":2.3740000000000001,"5min":1.7,"15min":1.6279999999999999},"min":{"1min":0.92000000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":16.98,"5min":16.98,"15min":16.98},"last":1.9330000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.413,"15min":1.4390000000000001},"min":{"1min":0.93500000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.101,"5min":2.548,"15min":2.548},"last":1.885},{"interface":"front","average":{"1min":1.583,"5min":1.6799999999999999,"15min":1.732},"min":{"1min":1.4339999999999999,"5min":1.163,"15min":1.163},"max":{"1min":1.8220000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":2.0960000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5249999999999999,"15min":1.5629999999999999},"min":{"1min":1.1459999999999999,"5min":1.016,"15min":1.016},"max":{"1min":2.1930000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.6819999999999999,"15min":1.7370000000000001},"min":{"1min":1.3540000000000001,"5min":1.286,"15min":1.286},"max":{"1min":2.2480000000000002,"5min":3.141,"15min":3.141},"last":1.55}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.206,"15min":1.2150000000000001},"min":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.681,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.109},{"interface":"front","average":{"1min":1.26,"5min":1.3029999999999999,"15min":1.3600000000000001},"min":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.988,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":0.874}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2130000000000001,"15min":1.1990000000000001},"min":{"1min":0.86499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.181},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2190000000000001,"15min":1.1970000000000001},"min":{"1min":0.91500000000000004,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.782,"5min":1.917,"15min":1.917},"last":1.538}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2150000000000001,"15min":1.222},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7130000000000001,"5min":2.835,"15min":2.835},"last":1.03},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.256},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.3280000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.0670000000000002,"5min":1.48,"15min":1.3440000000000001},"min":{"1min":0.76000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":14.914,"5min":14.914,"15min":14.914},"last":1.468},{"interface":"front","average":{"1min":1.46,"5min":1.407,"15min":1.427},"min":{"1min":1.0409999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.944,"5min":3.302,"15min":3.302},"last":1.371}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3839999999999999,"15min":1.407},"min":{"1min":1.03,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.8009999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.476,"15min":1.48},"min":{"1min":0.96899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":1.224}]},{"osd":6,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.577,"15min":1.625},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.0529999999999999,"5min":5.218,"15min":5.218},"last":1.266},{"interface":"front","average":{"1min":1.496,"5min":1.524,"15min":1.546},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.8260000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.4330000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.607,"15min":1.6220000000000001},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":4.4059999999999997,"5min":4.4059999999999997,"15min":4.4059999999999997},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6879999999999999,"15min":1.756},"min":{"1min":1.1379999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.2149999999999999,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.504}]}]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2090000000000001,"15min":1.2110000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.526},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.756}]},{"osd":1,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":1.399,"15min":1.359},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":11.987,"5min":11.987,"15min":11.987},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2929999999999999,"15min":1.331},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9810000000000001,"5min":2.117,"15min":2.117},"last":1.147}]},{"osd":2,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.4450000000000001,"15min":1.4830000000000001},"min":{"1min":0.92800000000000005,"5min":0.745,"15min":0.745},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.333,"15min":1.3360000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0009999999999999,"5min":2.141,"15min":2.141},"last":1.3300000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.528,"15min":1.5669999999999999},"min":{"1min":1.077,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.198},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.506,"15min":1.6040000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.931,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.468}]},{"osd":5,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.599,"15min":1.673},"min":{"1min":0.98299999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.393},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.5640000000000001,"15min":1.6699999999999999},"min":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1179999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.6870000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.5660000000000001,"15min":1.659},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.0419999999999998,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.821,"15min":1.837},"min":{"1min":0.92100000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":1.637}]},{"osd":7,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":2.0489999999999999,"5min":1.8759999999999999,"15min":1.8979999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":11.894,"5min":11.894,"15min":11.894},"last":1.804},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.851,"15min":1.903},"min":{"1min":1.419,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.8440000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3660000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.129,"15min":1.089},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5589999999999999,"5min":3.008,"15min":3.008},"last":1.159}]},{"osd":1,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.772,"5min":1.1699999999999999,"15min":1.0609999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":12.391,"5min":12.391,"15min":12.391},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.0880000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.2230000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.169,"15min":1.1559999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.9399999999999999,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.373,"15min":1.3720000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.925,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.7090000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.377},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.6850000000000001,"5min":4.226,"15min":4.226},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.4670000000000001,"15min":1.4299999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":7.3600000000000003,"5min":7.3600000000000003,"15min":7.3600000000000003},"last":1.306}]},{"osd":5,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.516,"15min":1.526},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.3090000000000002,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.8680000000000001},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.631,"15min":1.623},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.7360000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":1.75}]},{"osd":6,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":2.052,"5min":1.7010000000000001,"15min":1.675},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":7.9630000000000001,"5min":7.9630000000000001,"15min":7.9630000000000001},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.788,"5min":1.671,"15min":1.6719999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.556}]},{"osd":7,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.671,"15min":1.6890000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.621,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":1.653},{"interface":"front","average":{"1min":1.9259999999999999,"5min":1.73,"15min":1.7210000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":7.407,"5min":7.407,"15min":7.407},"last":1.504}]}]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.1040000000000001,"15min":1.0409999999999999},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.427,"5min":14.427,"15min":14.427},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0049999999999999,"15min":0.99299999999999999},"min":{"1min":0.437,"5min":0.432,"15min":0.432},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.79700000000000004}]},{"osd":1,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.8859999999999999,"5min":1.371,"15min":1.3149999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":14.609,"5min":14.609,"15min":14.609},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.137,"5min":1.1819999999999999,"15min":1.2010000000000001},"min":{"1min":0.56100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.722,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.337}]},{"osd":3,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.286,"15min":1.208},"min":{"1min":0.54500000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":16.567,"5min":16.567,"15min":16.567},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.9470000000000001,"5min":1.5089999999999999,"15min":1.4730000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":14.528,"5min":14.528,"15min":14.528},"last":1.226}]},{"osd":4,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.2789999999999999,"5min":1.716,"15min":1.6499999999999999},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":14.779,"5min":14.779,"15min":14.779},"last":1.175},{"interface":"front","average":{"1min":1.296,"5min":1.4239999999999999,"15min":1.462},"min":{"1min":0.878,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6519999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.528}]},{"osd":5,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.1459999999999999,"5min":1.5669999999999999,"15min":1.508},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":17.167000000000002,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.649},{"interface":"front","average":{"1min":2.2320000000000002,"5min":1.728,"15min":1.6599999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":14.702999999999999,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.387}]},{"osd":6,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.47,"15min":1.486},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7689999999999999,"5min":2.246,"15min":2.246},"last":1.4339999999999999},{"interface":"front","average":{"1min":2.302,"5min":1.5880000000000001,"15min":1.4850000000000001},"min":{"1min":0.85399999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":17.07,"5min":17.07,"15min":17.07},"last":1.7729999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5760000000000001,"15min":1.611},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8129999999999999,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.484},{"interface":"front","average":{"1min":3.0960000000000001,"5min":1.806,"15min":1.6060000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":16.922000000000001,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.871}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":512000,"data_stored":205178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":0.94799999999999995,"15min":0.879},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":0.747},{"interface":"front","average":{"1min":1.103,"5min":0.998,"15min":0.98599999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":1.0760000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.93200000000000005,"15min":0.86799999999999999},"min":{"1min":0.48999999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.637,"5min":1.234,"15min":1.155},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":1.196}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.9079999999999999,"15min":2.1219999999999999},"min":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":6.5949999999999998,"5min":22.106999999999999,"15min":22.106999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":2.4590000000000001,"5min":1.401,"15min":1.2370000000000001},"min":{"1min":0.47499999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":15.510999999999999,"5min":15.510999999999999,"15min":15.510999999999999},"last":1.234}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.6819999999999999,"15min":1.7529999999999999},"min":{"1min":0.98399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3230000000000004,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.744,"15min":1.9179999999999999},"min":{"1min":0.88500000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.9729999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.722}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.734,"15min":1.893},"min":{"1min":0.96999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.9359999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":3.2770000000000001,"5min":1.8979999999999999,"15min":1.7},"min":{"1min":0.88100000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":20.823,"5min":20.823,"15min":20.823},"last":1.5660000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.885,"15min":2.073},"min":{"1min":1.0469999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.8980000000000001,"5min":14.272,"15min":14.272},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.5049999999999999,"15min":1.4870000000000001},"min":{"1min":0.82599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":4.3639999999999999,"5min":4.3639999999999999,"15min":4.3639999999999999},"last":1.292}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.589,"15min":1.556},"min":{"1min":0.94999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.546,"15min":1.554},"min":{"1min":1.111,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.649}]}]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1093632,"data_stored":789701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89300000000000002,"15min":0.875},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5580000000000001,"5min":1.756,"15min":1.756},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.085,"15min":1.0760000000000001},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5509999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.268}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0509999999999999,"15min":0.995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.391},{"interface":"front","average":{"1min":1.03,"5min":0.89800000000000002,"15min":0.84599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.1020000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2669999999999999,"15min":1.2609999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.179,"5min":1.1040000000000001,"15min":1.056},"min":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.873}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3049999999999999,"15min":1.27},"min":{"1min":0.85199999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.202},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.5189999999999999,"15min":1.5},"min":{"1min":1.119,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.514}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7829999999999999,"5min":1.421,"15min":1.3460000000000001},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":8.0120000000000005,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.603},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4710000000000001,"15min":1.48},"min":{"1min":0.93200000000000005,"5min":0.754,"15min":0.754},"max":{"1min":2.161,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.321}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5249999999999999,"15min":1.502},"min":{"1min":1.03,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.444},{"interface":"front","average":{"1min":1.474,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.026,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.2730000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.482}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.778,"5min":1.653,"15min":1.645},"min":{"1min":0.97999999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.0259999999999998,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.724,"5min":1.581,"15min":1.55},"min":{"1min":0.94899999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.5609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:58:23.456 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:58:23.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.709+0000 7fc60ca00640 1 Processor -- start 2024-01-15T17:58:23.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.709+0000 7fc60ca00640 1 -- start start 2024-01-15T17:58:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.710+0000 7fc60ca00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc60813cfa0 0x7fc60815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.710+0000 7fc60ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6080cd530 0x7fc60813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.711+0000 7fc60ca00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6080cdec0 con 0x7fc6080cd9e0 2024-01-15T17:58:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.711+0000 7fc60ca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6080ce1a0 con 0x7fc6080cd530 2024-01-15T17:58:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.711+0000 7fc60ca00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6080ce480 con 0x7fc60813cfa0 2024-01-15T17:58:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.711+0000 7fc607e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc6080cd9e0 0x7fc60815d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46688/0 (socket says 172.21.15.26:46688) 2024-01-15T17:58:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.711+0000 7fc607e00640 1 -- 172.21.15.26:0/235315358 learned_addr learned my addr 172.21.15.26:0/235315358 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.711+0000 7fc5ffe00640 1 --2- 172.21.15.26:0/235315358 >> v2:172.21.15.26:3300/0 conn(0x7fc6080cd530 0x7fc60813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.711+0000 7fc607400640 1 --2- 172.21.15.26:0/235315358 >> v2:172.21.15.124:3300/0 conn(0x7fc60813cfa0 0x7fc60815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.711+0000 7fc606a00640 1 -- 172.21.15.26:0/235315358 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005171780 0 0) 0x7fc6080cdec0 con 0x7fc6080cd9e0 2024-01-15T17:58:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.711+0000 7fc606a00640 1 -- 172.21.15.26:0/235315358 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5e8005730 con 0x7fc6080cd9e0 2024-01-15T17:58:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.712+0000 7fc5ffe00640 1 -- 172.21.15.26:0/235315358 >> v2:172.21.15.124:3300/0 conn(0x7fc60813cfa0 msgr2=0x7fc60815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.712+0000 7fc5ffe00640 1 --2- 172.21.15.26:0/235315358 >> v2:172.21.15.124:3300/0 conn(0x7fc60813cfa0 0x7fc60815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.712+0000 7fc5ffe00640 1 -- 172.21.15.26:0/235315358 >> v1:172.21.15.26:6789/0 conn(0x7fc6080cd9e0 legacy=0x7fc60815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.712+0000 7fc5ffe00640 1 -- 172.21.15.26:0/235315358 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080ce760 con 0x7fc6080cd530 2024-01-15T17:58:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.713+0000 7fc5ffe00640 1 --2- 172.21.15.26:0/235315358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 0x7fc60813f5a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc60806cf00 tx=0x7fc5f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6dcb4764c30d1e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.713+0000 7fc606a00640 1 -- 172.21.15.26:0/235315358 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f002e070 con 0x7fc6080cd530 2024-01-15T17:58:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.713+0000 7fc606a00640 1 -- 172.21.15.26:0/235315358 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f00315a0 con 0x7fc6080cd530 2024-01-15T17:58:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.713+0000 7fc606a00640 1 -- 172.21.15.26:0/235315358 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0031910 con 0x7fc6080cd530 2024-01-15T17:58:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.714+0000 7fc60ca00640 1 -- 172.21.15.26:0/235315358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 msgr2=0x7fc60813f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.714+0000 7fc60ca00640 1 --2- 172.21.15.26:0/235315358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 0x7fc60813f5a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc60806cf00 tx=0x7fc5f002b840 comp rx=0 tx=0).stop 2024-01-15T17:58:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.714+0000 7fc60ca00640 1 -- 172.21.15.26:0/235315358 shutdown_connections 2024-01-15T17:58:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.714+0000 7fc60ca00640 1 --2- 172.21.15.26:0/235315358 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 0x7fc60813f5a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.714+0000 7fc60ca00640 1 --2- 172.21.15.26:0/235315358 >> v2:172.21.15.124:3300/0 conn(0x7fc60813cfa0 0x7fc60815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.714+0000 7fc60ca00640 1 -- 172.21.15.26:0/235315358 >> 172.21.15.26:0/235315358 conn(0x7fc60806a800 msgr2=0x7fc60806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.714+0000 7fc60ca00640 1 -- 172.21.15.26:0/235315358 shutdown_connections 2024-01-15T17:58:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.714+0000 7fc60ca00640 1 -- 172.21.15.26:0/235315358 wait complete. 2024-01-15T17:58:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.716+0000 7fc60ca00640 1 Processor -- start 2024-01-15T17:58:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.716+0000 7fc60ca00640 1 -- start start 2024-01-15T17:58:23.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc60ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 0x7fc608152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc60ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6080cd9e0 0x7fc608156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc60ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc60813cfa0 0x7fc6081591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc60ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6081528b0 con 0x7fc6080cd530 2024-01-15T17:58:23.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc60ca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6081f32f0 con 0x7fc60813cfa0 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc60ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6081579c0 con 0x7fc6080cd9e0 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc607400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 0x7fc608152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc607400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 0x7fc608152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57546/0 (socket says 172.21.15.26:57546) 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc607400640 1 -- 172.21.15.26:0/211702577 learned_addr learned my addr 172.21.15.26:0/211702577 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc607e00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc60813cfa0 0x7fc6081591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc5ffe00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6080cd9e0 0x7fc608156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc607400640 1 -- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6080cd9e0 msgr2=0x7fc608156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc607400640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6080cd9e0 0x7fc608156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc607400640 1 -- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc60813cfa0 msgr2=0x7fc6081591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc607400640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc60813cfa0 0x7fc6081591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.717+0000 7fc607400640 1 -- 172.21.15.26:0/211702577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f0009400 con 0x7fc6080cd530 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.718+0000 7fc607e00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc60813cfa0 0x7fc6081591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.718+0000 7fc5ffe00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6080cd9e0 0x7fc608156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:23.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.718+0000 7fc607400640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 0x7fc608152370 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800d620 tx=0x7fc5f800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:23.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.718+0000 7fc5ff400640 1 -- 172.21.15.26:0/211702577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800b620 con 0x7fc6080cd530 2024-01-15T17:58:23.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.718+0000 7fc5ff400640 1 -- 172.21.15.26:0/211702577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f800b780 con 0x7fc6080cd530 2024-01-15T17:58:23.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.718+0000 7fc5ff400640 1 -- 172.21.15.26:0/211702577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800bb30 con 0x7fc6080cd530 2024-01-15T17:58:23.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.718+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6081576e0 con 0x7fc6080cd530 2024-01-15T17:58:23.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.718+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6081e2a10 con 0x7fc6080cd530 2024-01-15T17:58:23.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.720+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc608067450 con 0x7fc6080cd530 2024-01-15T17:58:23.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.721+0000 7fc5ff400640 1 -- 172.21.15.26:0/211702577 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fc5f800bc90 con 0x7fc6080cd530 2024-01-15T17:58:23.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.722+0000 7fc5ff400640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5e006b500 0x7fc5e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:23.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.722+0000 7fc5ff400640 1 -- 172.21.15.26:0/211702577 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc5f808a4f0 con 0x7fc6080cd530 2024-01-15T17:58:23.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.723+0000 7fc5ffe00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5e006b500 0x7fc5e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:23.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.724+0000 7fc5ffe00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5e006b500 0x7fc5e006d9c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0036000 tx=0x7fc5f003d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:23.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.727+0000 7fc5ff400640 1 -- 172.21.15.26:0/211702577 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5f8054fd0 con 0x7fc6080cd530 2024-01-15T17:58:23.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.917+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6080cbff0 con 0x7fc5e006b500 2024-01-15T17:58:23.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.938+0000 7fc5ff400640 1 -- 172.21.15.26:0/211702577 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274146 (secure 0 0 0) 0x7fc6080cbff0 con 0x7fc5e006b500 2024-01-15T17:58:23.945 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:23.946 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:58:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.943+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5e006b500 msgr2=0x7fc5e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.943+0000 7fc60ca00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5e006b500 0x7fc5e006d9c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0036000 tx=0x7fc5f003d040 comp rx=0 tx=0).stop 2024-01-15T17:58:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 msgr2=0x7fc608152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 0x7fc608152370 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800d620 tx=0x7fc5f800dae0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 shutdown_connections 2024-01-15T17:58:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6080cd9e0 0x7fc608156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5e006b500 0x7fc5e006d9c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc60813cfa0 0x7fc6081591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 --2- 172.21.15.26:0/211702577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6080cd530 0x7fc608152370 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 >> 172.21.15.26:0/211702577 conn(0x7fc60806a800 msgr2=0x7fc60815ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 shutdown_connections 2024-01-15T17:58:23.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:23.944+0000 7fc60ca00640 1 -- 172.21.15.26:0/211702577 wait complete. 2024-01-15T17:58:23.967 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-01-15T17:58:21.979139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":267048,"kb_used_data":5076,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754707672,"statfs":{"total":773094113280,"available":772820656128,"internally_reserved":0,"allocated":5197824,"data_stored":2781245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":268095024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.042459"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.941701+0000","last_change":"2024-01-15T17:58:16.941701+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.941701+0000","last_undegraded":"2024-01-15T17:58:16.941701+0000","last_fullsized":"2024-01-15T17:58:16.941701+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942072+0000","last_change":"2024-01-15T17:58:16.942072+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942072+0000","last_undegraded":"2024-01-15T17:58:16.942072+0000","last_fullsized":"2024-01-15T17:58:16.942072+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.948787+0000","last_change":"2024-01-15T17:58:16.948787+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.948787+0000","last_undegraded":"2024-01-15T17:58:16.948787+0000","last_fullsized":"2024-01-15T17:58:16.948787+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.953681+0000","last_change":"2024-01-15T17:58:16.953681+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.953681+0000","last_undegraded":"2024-01-15T17:58:16.953681+0000","last_fullsized":"2024-01-15T17:58:16.953681+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942609+0000","last_change":"2024-01-15T17:58:16.942609+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942609+0000","last_undegraded":"2024-01-15T17:58:16.942609+0000","last_fullsized":"2024-01-15T17:58:16.942609+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942183+0000","last_change":"2024-01-15T17:58:16.942183+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942183+0000","last_undegraded":"2024-01-15T17:58:16.942183+0000","last_fullsized":"2024-01-15T17:58:16.942183+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949897+0000","last_change":"2024-01-15T17:58:16.949897+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949897+0000","last_undegraded":"2024-01-15T17:58:16.949897+0000","last_fullsized":"2024-01-15T17:58:16.949897+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.948193+0000","last_change":"2024-01-15T17:58:16.948193+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.948193+0000","last_undegraded":"2024-01-15T17:58:16.948193+0000","last_fullsized":"2024-01-15T17:58:16.948193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.940884+0000","last_change":"2024-01-15T17:58:16.940884+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.940884+0000","last_undegraded":"2024-01-15T17:58:16.940884+0000","last_fullsized":"2024-01-15T17:58:16.940884+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.946664+0000","last_change":"2024-01-15T17:58:16.946664+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.946664+0000","last_undegraded":"2024-01-15T17:58:16.946664+0000","last_fullsized":"2024-01-15T17:58:16.946664+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.955768+0000","last_change":"2024-01-15T17:58:16.955768+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.955768+0000","last_undegraded":"2024-01-15T17:58:16.955768+0000","last_fullsized":"2024-01-15T17:58:16.955768+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.940096+0000","last_change":"2024-01-15T17:58:16.940096+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.940096+0000","last_undegraded":"2024-01-15T17:58:16.940096+0000","last_fullsized":"2024-01-15T17:58:16.940096+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942131+0000","last_change":"2024-01-15T17:58:16.942131+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942131+0000","last_undegraded":"2024-01-15T17:58:16.942131+0000","last_fullsized":"2024-01-15T17:58:16.942131+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.955523+0000","last_change":"2024-01-15T17:58:16.955523+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.955523+0000","last_undegraded":"2024-01-15T17:58:16.955523+0000","last_fullsized":"2024-01-15T17:58:16.955523+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.941797+0000","last_change":"2024-01-15T17:58:16.941797+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.941797+0000","last_undegraded":"2024-01-15T17:58:16.941797+0000","last_fullsized":"2024-01-15T17:58:16.941797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.947364+0000","last_change":"2024-01-15T17:58:16.947364+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.947364+0000","last_undegraded":"2024-01-15T17:58:16.947364+0000","last_fullsized":"2024-01-15T17:58:16.947364+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.939468+0000","last_change":"2024-01-15T17:58:16.939468+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.939468+0000","last_undegraded":"2024-01-15T17:58:16.939468+0000","last_fullsized":"2024-01-15T17:58:16.939468+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.940707+0000","last_change":"2024-01-15T17:58:16.940707+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.940707+0000","last_undegraded":"2024-01-15T17:58:16.940707+0000","last_fullsized":"2024-01-15T17:58:16.940707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949966+0000","last_change":"2024-01-15T17:58:16.949966+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949966+0000","last_undegraded":"2024-01-15T17:58:16.949966+0000","last_fullsized":"2024-01-15T17:58:16.949966+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.941821+0000","last_change":"2024-01-15T17:58:16.941821+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.941821+0000","last_undegraded":"2024-01-15T17:58:16.941821+0000","last_fullsized":"2024-01-15T17:58:16.941821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.941556+0000","last_change":"2024-01-15T17:58:16.941556+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.941556+0000","last_undegraded":"2024-01-15T17:58:16.941556+0000","last_fullsized":"2024-01-15T17:58:16.941556+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949183+0000","last_change":"2024-01-15T17:58:16.949183+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949183+0000","last_undegraded":"2024-01-15T17:58:16.949183+0000","last_fullsized":"2024-01-15T17:58:16.949183+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.946623+0000","last_change":"2024-01-15T17:58:16.946623+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.946623+0000","last_undegraded":"2024-01-15T17:58:16.946623+0000","last_fullsized":"2024-01-15T17:58:16.946623+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949918+0000","last_change":"2024-01-15T17:58:16.949918+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949918+0000","last_undegraded":"2024-01-15T17:58:16.949918+0000","last_fullsized":"2024-01-15T17:58:16.949918+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.942644+0000","last_change":"2024-01-15T17:58:16.942644+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.942644+0000","last_undegraded":"2024-01-15T17:58:16.942644+0000","last_fullsized":"2024-01-15T17:58:16.942644+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.953143+0000","last_change":"2024-01-15T17:58:16.953143+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.953143+0000","last_undegraded":"2024-01-15T17:58:16.953143+0000","last_fullsized":"2024-01-15T17:58:16.953143+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.948646+0000","last_change":"2024-01-15T17:58:16.948646+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.948646+0000","last_undegraded":"2024-01-15T17:58:16.948646+0000","last_fullsized":"2024-01-15T17:58:16.948646+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.953154+0000","last_change":"2024-01-15T17:58:16.953154+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.953154+0000","last_undegraded":"2024-01-15T17:58:16.953154+0000","last_fullsized":"2024-01-15T17:58:16.953154+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.940882+0000","last_change":"2024-01-15T17:58:16.940882+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.940882+0000","last_undegraded":"2024-01-15T17:58:16.940882+0000","last_fullsized":"2024-01-15T17:58:16.940882+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949783+0000","last_change":"2024-01-15T17:58:16.949783+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949783+0000","last_undegraded":"2024-01-15T17:58:16.949783+0000","last_fullsized":"2024-01-15T17:58:16.949783+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.946633+0000","last_change":"2024-01-15T17:58:16.946633+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.946633+0000","last_undegraded":"2024-01-15T17:58:16.946633+0000","last_fullsized":"2024-01-15T17:58:16.946633+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-01-15T17:58:17.935043+0000","last_change":"2024-01-15T17:58:17.935043+0000","last_active":"2024-01-15T17:58:17.935043+0000","last_peered":"2024-01-15T17:58:17.935043+0000","last_clean":"2024-01-15T17:58:17.935043+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:17.935043+0000","last_undegraded":"2024-01-15T17:58:17.935043+0000","last_fullsized":"2024-01-15T17:58:17.935043+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":44,"state":"creating+peering","last_fresh":"2024-01-15T17:58:16.949542+0000","last_change":"2024-01-15T17:58:16.949542+0000","last_active":"2024-01-15T17:58:16.921286+0000","last_peered":"2024-01-15T17:58:16.921286+0000","last_clean":"2024-01-15T17:58:16.921286+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T17:58:16.949542+0000","last_undegraded":"2024-01-15T17:58:16.949542+0000","last_fullsized":"2024-01-15T17:58:16.949542+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":182,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-01-15T17:58:14.763036+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:58:14.763036+0000","last_peered":"2024-01-15T17:58:14.763036+0000","last_clean":"2024-01-15T17:58:14.763036+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:58:14.763036+0000","last_undegraded":"2024-01-15T17:58:14.763036+0000","last_fullsized":"2024-01-15T17:58:14.763036+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542187,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1093632,"data_stored":789701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1559999999999999,"15min":1.1539999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5700000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.175,"15min":1.1679999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.6619999999999999,"5min":1.804,"15min":1.804},"last":1.3560000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.177},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.274,"15min":1.262},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.619}]},{"osd":2,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.3460000000000001,"15min":1.3759999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.504}]},{"osd":3,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.294,"15min":1.298},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.385},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.4890000000000001,"15min":1.4930000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":3.831,"5min":3.831,"15min":3.831},"last":1.5820000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4350000000000001,"15min":1.48},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.1890000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.593,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":4.4729999999999999,"5min":4.4729999999999999,"15min":4.4729999999999999},"last":1.2170000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.6000000000000001,"15min":1.605},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1589999999999998,"5min":7.3819999999999997,"15min":7.3819999999999997},"last":1.675},{"interface":"front","average":{"1min":1.456,"5min":1.548,"15min":1.5860000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.1230000000000002,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4450000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.649,"15min":1.6919999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9219999999999999,"5min":2.556,"15min":2.556},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5880000000000001,"15min":1.625},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1400000000000001,"5min":2.649,"15min":2.649},"last":1.3140000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.409,"15min":1.3149999999999999},"min":{"1min":0.753,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":12.544,"5min":12.544,"15min":12.544},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.206,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.165}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.29,"15min":1.202},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":12.811999999999999,"5min":12.811999999999999,"15min":12.811999999999999},"last":1.655},{"interface":"front","average":{"1min":1.206,"5min":1.1870000000000001,"15min":1.179},"min":{"1min":0.83899999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4990000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.875}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.214,"15min":1.2},"min":{"1min":0.94299999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5549999999999999,"5min":1.788,"15min":1.788},"last":1.101},{"interface":"front","average":{"1min":2.0169999999999999,"5min":1.5680000000000001,"15min":1.5269999999999999},"min":{"1min":0.96899999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":12.897,"5min":12.897,"15min":12.897},"last":1.0209999999999999}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.169,"5min":1.597,"15min":1.5069999999999999},"min":{"1min":1.23,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":12.717000000000001,"5min":12.717000000000001,"15min":12.717000000000001},"last":2.048},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.95399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.97,"5min":2.29,"15min":2.29},"last":1.468}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.90700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6839999999999999,"5min":2.343,"15min":2.343},"last":1.986},{"interface":"front","average":{"1min":2.3740000000000001,"5min":1.7,"15min":1.6279999999999999},"min":{"1min":0.92000000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":16.98,"5min":16.98,"15min":16.98},"last":1.9330000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.413,"15min":1.4390000000000001},"min":{"1min":0.93500000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.101,"5min":2.548,"15min":2.548},"last":1.885},{"interface":"front","average":{"1min":1.583,"5min":1.6799999999999999,"15min":1.732},"min":{"1min":1.4339999999999999,"5min":1.163,"15min":1.163},"max":{"1min":1.8220000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":2.0960000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5249999999999999,"15min":1.5629999999999999},"min":{"1min":1.1459999999999999,"5min":1.016,"15min":1.016},"max":{"1min":2.1930000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.7749999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.6819999999999999,"15min":1.7370000000000001},"min":{"1min":1.3540000000000001,"5min":1.286,"15min":1.286},"max":{"1min":2.2480000000000002,"5min":3.141,"15min":3.141},"last":1.55}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.206,"15min":1.2150000000000001},"min":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.681,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.109},{"interface":"front","average":{"1min":1.26,"5min":1.3029999999999999,"15min":1.3600000000000001},"min":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.988,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":0.874}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2130000000000001,"15min":1.1990000000000001},"min":{"1min":0.86499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.181},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2190000000000001,"15min":1.1970000000000001},"min":{"1min":0.91500000000000004,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.782,"5min":1.917,"15min":1.917},"last":1.538}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2150000000000001,"15min":1.222},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7130000000000001,"5min":2.835,"15min":2.835},"last":1.03},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.256},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.3280000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.0670000000000002,"5min":1.48,"15min":1.3440000000000001},"min":{"1min":0.76000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":14.914,"5min":14.914,"15min":14.914},"last":1.468},{"interface":"front","average":{"1min":1.46,"5min":1.407,"15min":1.427},"min":{"1min":1.0409999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.944,"5min":3.302,"15min":3.302},"last":1.371}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3839999999999999,"15min":1.407},"min":{"1min":1.03,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.8009999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.476,"15min":1.48},"min":{"1min":0.96899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":1.224}]},{"osd":6,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.577,"15min":1.625},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.0529999999999999,"5min":5.218,"15min":5.218},"last":1.266},{"interface":"front","average":{"1min":1.496,"5min":1.524,"15min":1.546},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.8260000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.4330000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.607,"15min":1.6220000000000001},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":4.4059999999999997,"5min":4.4059999999999997,"15min":4.4059999999999997},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6879999999999999,"15min":1.756},"min":{"1min":1.1379999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.2149999999999999,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.504}]}]},{"osd":4,"up_from":14,"seq":60129542186,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2090000000000001,"15min":1.2110000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.526},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.756}]},{"osd":1,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":1.399,"15min":1.359},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":11.987,"5min":11.987,"15min":11.987},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2929999999999999,"15min":1.331},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9810000000000001,"5min":2.117,"15min":2.117},"last":1.147}]},{"osd":2,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.4450000000000001,"15min":1.4830000000000001},"min":{"1min":0.92800000000000005,"5min":0.745,"15min":0.745},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.333,"15min":1.3360000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0009999999999999,"5min":2.141,"15min":2.141},"last":1.3300000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.528,"15min":1.5669999999999999},"min":{"1min":1.077,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.198},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.506,"15min":1.6040000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.931,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.468}]},{"osd":5,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.599,"15min":1.673},"min":{"1min":0.98299999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.393},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.5640000000000001,"15min":1.6699999999999999},"min":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1179999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.6870000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.5660000000000001,"15min":1.659},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.0419999999999998,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.821,"15min":1.837},"min":{"1min":0.92100000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":1.637}]},{"osd":7,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":2.0489999999999999,"5min":1.8759999999999999,"15min":1.8979999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":11.894,"5min":11.894,"15min":11.894},"last":1.804},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.851,"15min":1.903},"min":{"1min":1.419,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.8440000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3660000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.121,"5min":1.129,"15min":1.089},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5589999999999999,"5min":3.008,"15min":3.008},"last":1.159}]},{"osd":1,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.772,"5min":1.1699999999999999,"15min":1.0609999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":12.391,"5min":12.391,"15min":12.391},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.0880000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.2230000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.169,"15min":1.1559999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.9399999999999999,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.373,"15min":1.3720000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.925,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":1.7090000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.377},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.6850000000000001,"5min":4.226,"15min":4.226},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.4670000000000001,"15min":1.4299999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":7.3600000000000003,"5min":7.3600000000000003,"15min":7.3600000000000003},"last":1.306}]},{"osd":5,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.516,"15min":1.526},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.3090000000000002,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.8680000000000001},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.631,"15min":1.623},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.7360000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":1.75}]},{"osd":6,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":2.052,"5min":1.7010000000000001,"15min":1.675},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":7.9630000000000001,"5min":7.9630000000000001,"15min":7.9630000000000001},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.788,"5min":1.671,"15min":1.6719999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.556}]},{"osd":7,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.671,"15min":1.6890000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.621,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":1.653},{"interface":"front","average":{"1min":1.9259999999999999,"5min":1.73,"15min":1.7210000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":7.407,"5min":7.407,"15min":7.407},"last":1.504}]}]},{"osd":2,"up_from":14,"seq":60129542188,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":499712,"data_stored":199333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.1040000000000001,"15min":1.0409999999999999},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.427,"5min":14.427,"15min":14.427},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0049999999999999,"15min":0.99299999999999999},"min":{"1min":0.437,"5min":0.432,"15min":0.432},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.79700000000000004}]},{"osd":1,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.8859999999999999,"5min":1.371,"15min":1.3149999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":14.609,"5min":14.609,"15min":14.609},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.137,"5min":1.1819999999999999,"15min":1.2010000000000001},"min":{"1min":0.56100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.722,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.337}]},{"osd":3,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.286,"15min":1.208},"min":{"1min":0.54500000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":16.567,"5min":16.567,"15min":16.567},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.9470000000000001,"5min":1.5089999999999999,"15min":1.4730000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":14.528,"5min":14.528,"15min":14.528},"last":1.226}]},{"osd":4,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.2789999999999999,"5min":1.716,"15min":1.6499999999999999},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":14.779,"5min":14.779,"15min":14.779},"last":1.175},{"interface":"front","average":{"1min":1.296,"5min":1.4239999999999999,"15min":1.462},"min":{"1min":0.878,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6519999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.528}]},{"osd":5,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.1459999999999999,"5min":1.5669999999999999,"15min":1.508},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":17.167000000000002,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.649},{"interface":"front","average":{"1min":2.2320000000000002,"5min":1.728,"15min":1.6599999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":14.702999999999999,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.387}]},{"osd":6,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.47,"15min":1.486},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7689999999999999,"5min":2.246,"15min":2.246},"last":1.4339999999999999},{"interface":"front","average":{"1min":2.302,"5min":1.5880000000000001,"15min":1.4850000000000001},"min":{"1min":0.85399999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":17.07,"5min":17.07,"15min":17.07},"last":1.7729999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5760000000000001,"15min":1.611},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8129999999999999,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.484},{"interface":"front","average":{"1min":3.0960000000000001,"5min":1.806,"15min":1.6060000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":16.922000000000001,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.871}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35840,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336000,"statfs":{"total":96636764160,"available":96600064000,"internally_reserved":0,"allocated":512000,"data_stored":205178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":0.94799999999999995,"15min":0.879},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":0.747},{"interface":"front","average":{"1min":1.103,"5min":0.998,"15min":0.98599999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":1.0760000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.93200000000000005,"15min":0.86799999999999999},"min":{"1min":0.48999999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.637,"5min":1.234,"15min":1.155},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":1.196}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.9079999999999999,"15min":2.1219999999999999},"min":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":6.5949999999999998,"5min":22.106999999999999,"15min":22.106999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":2.4590000000000001,"5min":1.401,"15min":1.2370000000000001},"min":{"1min":0.47499999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":15.510999999999999,"5min":15.510999999999999,"15min":15.510999999999999},"last":1.234}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.6819999999999999,"15min":1.7529999999999999},"min":{"1min":0.98399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3230000000000004,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.744,"15min":1.9179999999999999},"min":{"1min":0.88500000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.9729999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.722}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.734,"15min":1.893},"min":{"1min":0.96999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.9359999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":3.2770000000000001,"5min":1.8979999999999999,"15min":1.7},"min":{"1min":0.88100000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":20.823,"5min":20.823,"15min":20.823},"last":1.5660000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.885,"15min":2.073},"min":{"1min":1.0469999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.8980000000000001,"5min":14.272,"15min":14.272},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.5049999999999999,"15min":1.4870000000000001},"min":{"1min":0.82599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":4.3639999999999999,"5min":4.3639999999999999,"15min":4.3639999999999999},"last":1.292}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.589,"15min":1.556},"min":{"1min":0.94999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.546,"15min":1.554},"min":{"1min":1.111,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.649}]}]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335428,"statfs":{"total":96636764160,"available":96599478272,"internally_reserved":0,"allocated":1093632,"data_stored":789701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89300000000000002,"15min":0.875},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5580000000000001,"5min":1.756,"15min":1.756},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.085,"15min":1.0760000000000001},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5509999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.268}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0509999999999999,"15min":0.995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":1.391},{"interface":"front","average":{"1min":1.03,"5min":0.89800000000000002,"15min":0.84599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":1.1020000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2669999999999999,"15min":1.2609999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.179,"5min":1.1040000000000001,"15min":1.056},"min":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.873}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3049999999999999,"15min":1.27},"min":{"1min":0.85199999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.202},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.5189999999999999,"15min":1.5},"min":{"1min":1.119,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.514}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7829999999999999,"5min":1.421,"15min":1.3460000000000001},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":8.0120000000000005,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.603},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4710000000000001,"15min":1.48},"min":{"1min":0.93200000000000005,"5min":0.754,"15min":0.754},"max":{"1min":2.161,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.321}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5249999999999999,"15min":1.502},"min":{"1min":1.03,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.444},{"interface":"front","average":{"1min":1.474,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.026,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.2730000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.482}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.778,"5min":1.653,"15min":1.645},"min":{"1min":0.97999999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.0259999999999998,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.724,"5min":1.581,"15min":1.55},"min":{"1min":0.94899999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.5609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:58:23.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:58:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.234+0000 7fb951400640 1 Processor -- start 2024-01-15T17:58:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.235+0000 7fb951400640 1 -- start start 2024-01-15T17:58:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.236+0000 7fb951400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c13cfa0 0x7fb94c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.236+0000 7fb951400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb94c13fcb0 0x7fb94c0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.237+0000 7fb951400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94c0cdec0 con 0x7fb94c0cd530 2024-01-15T17:58:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.237+0000 7fb951400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c0ce1a0 con 0x7fb94c13fcb0 2024-01-15T17:58:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.237+0000 7fb951400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0ce480 con 0x7fb94c13cfa0 2024-01-15T17:58:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.237+0000 7fb94b400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb94c0cd530 0x7fb94c13f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46690/0 (socket says 172.21.15.26:46690) 2024-01-15T17:58:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.237+0000 7fb94b400640 1 -- 172.21.15.26:0/787129879 learned_addr learned my addr 172.21.15.26:0/787129879 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.237+0000 7fb950a00640 1 --2- 172.21.15.26:0/787129879 >> v2:172.21.15.124:3300/0 conn(0x7fb94c13fcb0 0x7fb94c0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.237+0000 7fb94be00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c13cfa0 0x7fb94c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.237+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731507118 0 0) 0x7fb94c0cdec0 con 0x7fb94c0cd530 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb92c003520 con 0x7fb94c0cd530 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 185013894 0 0) 0x7fb92c003520 con 0x7fb94c0cd530 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c13cfa0 msgr2=0x7fb94c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c13cfa0 0x7fb94c15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 >> v2:172.21.15.124:3300/0 conn(0x7fb94c13fcb0 msgr2=0x7fb94c0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 --2- 172.21.15.26:0/787129879 >> v2:172.21.15.124:3300/0 conn(0x7fb94c13fcb0 0x7fb94c0b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0ce760 con 0x7fb94c0cd530 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb934001670 con 0x7fb94c0cd530 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 >> v1:172.21.15.26:6789/0 conn(0x7fb94c0cd530 legacy=0x7fb94c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c13cfa0 0x7fb92c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb92c003f40 0x7fb92c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb92c004d00 0x7fb92c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0ce1a0 con 0x7fb92c004d00 2024-01-15T17:58:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0ce480 con 0x7fb92c003f40 2024-01-15T17:58:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c00ba30 con 0x7fb94c13cfa0 2024-01-15T17:58:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.238+0000 7fb950a00640 1 --2- 172.21.15.26:0/787129879 >> v2:172.21.15.124:3300/0 conn(0x7fb94c13fcb0 0x7fb94c0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:58:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.239+0000 7fb950a00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb92c004d00 0x7fb92c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.239+0000 7fb94b400640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb92c003f40 0x7fb92c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.239+0000 7fb94be00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c13cfa0 0x7fb92c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.239+0000 7fb950a00640 1 -- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c13cfa0 msgr2=0x7fb92c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.239+0000 7fb950a00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c13cfa0 0x7fb92c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.239+0000 7fb950a00640 1 -- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb92c003f40 msgr2=0x7fb92c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.239+0000 7fb950a00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb92c003f40 0x7fb92c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.239+0000 7fb950a00640 1 -- 172.21.15.26:0/787129879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c003520 con 0x7fb92c004d00 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.240+0000 7fb950a00640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb92c004d00 0x7fb92c005180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb94400e350 tx=0x7fb92c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77e96f0b1d9794 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.240+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94402cde0 con 0x7fb92c004d00 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.240+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94400f040 con 0x7fb92c004d00 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.240+0000 7fb94aa00640 1 -- 172.21.15.26:0/787129879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944003b90 con 0x7fb92c004d00 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.241+0000 7fb951400640 1 -- 172.21.15.26:0/787129879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb92c004d00 msgr2=0x7fb92c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.241+0000 7fb951400640 1 --2- 172.21.15.26:0/787129879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb92c004d00 0x7fb92c005180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb94400e350 tx=0x7fb92c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.241+0000 7fb950a00640 1 -- 172.21.15.26:0/787129879 reap_dead start 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.241+0000 7fb951400640 1 -- 172.21.15.26:0/787129879 shutdown_connections 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.241+0000 7fb951400640 1 -- 172.21.15.26:0/787129879 >> 172.21.15.26:0/787129879 conn(0x7fb94c06a800 msgr2=0x7fb94c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.241+0000 7fb951400640 1 -- 172.21.15.26:0/787129879 shutdown_connections 2024-01-15T17:58:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.241+0000 7fb951400640 1 -- 172.21.15.26:0/787129879 wait complete. 2024-01-15T17:58:24.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.243+0000 7fb951400640 1 Processor -- start 2024-01-15T17:58:24.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.243+0000 7fb951400640 1 -- start start 2024-01-15T17:58:24.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.244+0000 7fb951400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb92c004d00 0x7fb94c13c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.244+0000 7fb951400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c1521d0 0x7fb94c158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.244+0000 7fb951400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb94c1598a0 0x7fb94c14a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.244+0000 7fb951400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c14dd50 con 0x7fb94c1598a0 2024-01-15T17:58:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.244+0000 7fb951400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c13cb50 con 0x7fb92c004d00 2024-01-15T17:58:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.244+0000 7fb951400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c158560 con 0x7fb94c1521d0 2024-01-15T17:58:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.244+0000 7fb950a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb94c1598a0 0x7fb94c14a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.244+0000 7fb950a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb94c1598a0 0x7fb94c14a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57558/0 (socket says 172.21.15.26:57558) 2024-01-15T17:58:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.244+0000 7fb950a00640 1 -- 172.21.15.26:0/981249484 learned_addr learned my addr 172.21.15.26:0/981249484 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.245+0000 7fb94be00640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb92c004d00 0x7fb94c13c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.245+0000 7fb94b400640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c1521d0 0x7fb94c158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.245+0000 7fb950a00640 1 -- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c1521d0 msgr2=0x7fb94c158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.245+0000 7fb950a00640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c1521d0 0x7fb94c158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.245+0000 7fb950a00640 1 -- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb92c004d00 msgr2=0x7fb94c13c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.245+0000 7fb950a00640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb92c004d00 0x7fb94c13c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.245+0000 7fb950a00640 1 -- 172.21.15.26:0/981249484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c158800 con 0x7fb94c1598a0 2024-01-15T17:58:24.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.245+0000 7fb950a00640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb94c1598a0 0x7fb94c14a530 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb94402c780 tx=0x7fb944009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.246+0000 7fb943e00640 1 -- 172.21.15.26:0/981249484 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944045070 con 0x7fb94c1598a0 2024-01-15T17:58:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.246+0000 7fb943e00640 1 -- 172.21.15.26:0/981249484 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb944034bd0 con 0x7fb94c1598a0 2024-01-15T17:58:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.246+0000 7fb943e00640 1 -- 172.21.15.26:0/981249484 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944003990 con 0x7fb94c1598a0 2024-01-15T17:58:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.246+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c158ae0 con 0x7fb94c1598a0 2024-01-15T17:58:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.246+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c1590a0 con 0x7fb94c1598a0 2024-01-15T17:58:24.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.247+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c067450 con 0x7fb94c1598a0 2024-01-15T17:58:24.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.248+0000 7fb943e00640 1 -- 172.21.15.26:0/981249484 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fb944040470 con 0x7fb94c1598a0 2024-01-15T17:58:24.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.249+0000 7fb943e00640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb92406b530 0x7fb92406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.249+0000 7fb943e00640 1 -- 172.21.15.26:0/981249484 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb9440ac140 con 0x7fb94c1598a0 2024-01-15T17:58:24.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.250+0000 7fb94be00640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb92406b530 0x7fb92406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.251+0000 7fb94be00640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb92406b530 0x7fb92406d9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009ac0 tx=0x7fb93c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:24.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.253+0000 7fb943e00640 1 -- 172.21.15.26:0/981249484 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb944076de0 con 0x7fb94c1598a0 2024-01-15T17:58:24.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.447+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb94c0b1d00 con 0x7fb92406b530 2024-01-15T17:58:24.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.470+0000 7fb943e00640 1 -- 172.21.15.26:0/981249484 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286474 (secure 0 0 0) 0x7fb94c0b1d00 con 0x7fb92406b530 2024-01-15T17:58:24.477 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:24.478 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb92406b530 msgr2=0x7fb92406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb92406b530 0x7fb92406d9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009ac0 tx=0x7fb93c008040 comp rx=0 tx=0).stop 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb94c1598a0 msgr2=0x7fb94c14a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb94c1598a0 0x7fb94c14a530 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb94402c780 tx=0x7fb944009750 comp rx=0 tx=0).stop 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 shutdown_connections 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb94c1521d0 0x7fb94c158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb92406b530 0x7fb92406d9f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb94c1598a0 0x7fb94c14a530 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 --2- 172.21.15.26:0/981249484 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb92c004d00 0x7fb94c13c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 >> 172.21.15.26:0/981249484 conn(0x7fb94c06a800 msgr2=0x7fb94c0aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:24.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 shutdown_connections 2024-01-15T17:58:24.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.476+0000 7fb951400640 1 -- 172.21.15.26:0/981249484 wait complete. 2024-01-15T17:58:24.500 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-01-15T17:58:23.979687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271588,"kb_used_data":5500,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754703132,"statfs":{"total":773094113280,"available":772816007168,"internally_reserved":0,"allocated":5632000,"data_stored":2976060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.022365"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159357+0000","last_change":"2024-01-15T17:58:18.991217+0000","last_active":"2024-01-15T17:58:21.159357+0000","last_peered":"2024-01-15T17:58:21.159357+0000","last_clean":"2024-01-15T17:58:21.159357+0000","last_became_active":"2024-01-15T17:58:18.990850+0000","last_became_peered":"2024-01-15T17:58:18.990850+0000","last_unstale":"2024-01-15T17:58:21.159357+0000","last_undegraded":"2024-01-15T17:58:21.159357+0000","last_fullsized":"2024-01-15T17:58:21.159357+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:48.160968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987226+0000","last_change":"2024-01-15T17:58:19.015695+0000","last_active":"2024-01-15T17:58:19.987226+0000","last_peered":"2024-01-15T17:58:19.987226+0000","last_clean":"2024-01-15T17:58:19.987226+0000","last_became_active":"2024-01-15T17:58:19.015324+0000","last_became_peered":"2024-01-15T17:58:19.015324+0000","last_unstale":"2024-01-15T17:58:19.987226+0000","last_undegraded":"2024-01-15T17:58:19.987226+0000","last_fullsized":"2024-01-15T17:58:19.987226+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:09.274759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345611+0000","last_change":"2024-01-15T17:58:18.984806+0000","last_active":"2024-01-15T17:58:20.345611+0000","last_peered":"2024-01-15T17:58:20.345611+0000","last_clean":"2024-01-15T17:58:20.345611+0000","last_became_active":"2024-01-15T17:58:18.984445+0000","last_became_peered":"2024-01-15T17:58:18.984445+0000","last_unstale":"2024-01-15T17:58:20.345611+0000","last_undegraded":"2024-01-15T17:58:20.345611+0000","last_fullsized":"2024-01-15T17:58:20.345611+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:07.065888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163376+0000","last_change":"2024-01-15T17:58:19.015296+0000","last_active":"2024-01-15T17:58:21.163376+0000","last_peered":"2024-01-15T17:58:21.163376+0000","last_clean":"2024-01-15T17:58:21.163376+0000","last_became_active":"2024-01-15T17:58:19.011703+0000","last_became_peered":"2024-01-15T17:58:19.011703+0000","last_unstale":"2024-01-15T17:58:21.163376+0000","last_undegraded":"2024-01-15T17:58:21.163376+0000","last_fullsized":"2024-01-15T17:58:21.163376+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:43:54.733897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986294+0000","last_change":"2024-01-15T17:58:19.009586+0000","last_active":"2024-01-15T17:58:19.986294+0000","last_peered":"2024-01-15T17:58:19.986294+0000","last_clean":"2024-01-15T17:58:19.986294+0000","last_became_active":"2024-01-15T17:58:19.009120+0000","last_became_peered":"2024-01-15T17:58:19.009120+0000","last_unstale":"2024-01-15T17:58:19.986294+0000","last_undegraded":"2024-01-15T17:58:19.986294+0000","last_fullsized":"2024-01-15T17:58:19.986294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:51.429644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160792+0000","last_change":"2024-01-15T17:58:18.994318+0000","last_active":"2024-01-15T17:58:21.160792+0000","last_peered":"2024-01-15T17:58:21.160792+0000","last_clean":"2024-01-15T17:58:21.160792+0000","last_became_active":"2024-01-15T17:58:18.994008+0000","last_became_peered":"2024-01-15T17:58:18.994008+0000","last_unstale":"2024-01-15T17:58:21.160792+0000","last_undegraded":"2024-01-15T17:58:21.160792+0000","last_fullsized":"2024-01-15T17:58:21.160792+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:51.745964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158955+0000","last_change":"2024-01-15T17:58:18.992678+0000","last_active":"2024-01-15T17:58:21.158955+0000","last_peered":"2024-01-15T17:58:21.158955+0000","last_clean":"2024-01-15T17:58:21.158955+0000","last_became_active":"2024-01-15T17:58:18.992276+0000","last_became_peered":"2024-01-15T17:58:18.992276+0000","last_unstale":"2024-01-15T17:58:21.158955+0000","last_undegraded":"2024-01-15T17:58:21.158955+0000","last_fullsized":"2024-01-15T17:58:21.158955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:18.389244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160076+0000","last_change":"2024-01-15T17:58:19.005373+0000","last_active":"2024-01-15T17:58:21.160076+0000","last_peered":"2024-01-15T17:58:21.160076+0000","last_clean":"2024-01-15T17:58:21.160076+0000","last_became_active":"2024-01-15T17:58:19.003358+0000","last_became_peered":"2024-01-15T17:58:19.003358+0000","last_unstale":"2024-01-15T17:58:21.160076+0000","last_undegraded":"2024-01-15T17:58:21.160076+0000","last_fullsized":"2024-01-15T17:58:21.160076+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:07.065888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164063+0000","last_change":"2024-01-15T17:58:19.003597+0000","last_active":"2024-01-15T17:58:21.164063+0000","last_peered":"2024-01-15T17:58:21.164063+0000","last_clean":"2024-01-15T17:58:21.164063+0000","last_became_active":"2024-01-15T17:58:19.003270+0000","last_became_peered":"2024-01-15T17:58:19.003270+0000","last_unstale":"2024-01-15T17:58:21.164063+0000","last_undegraded":"2024-01-15T17:58:21.164063+0000","last_fullsized":"2024-01-15T17:58:21.164063+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:59.015634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164080+0000","last_change":"2024-01-15T17:58:19.002263+0000","last_active":"2024-01-15T17:58:21.164080+0000","last_peered":"2024-01-15T17:58:21.164080+0000","last_clean":"2024-01-15T17:58:21.164080+0000","last_became_active":"2024-01-15T17:58:19.000576+0000","last_became_peered":"2024-01-15T17:58:19.000576+0000","last_unstale":"2024-01-15T17:58:21.164080+0000","last_undegraded":"2024-01-15T17:58:21.164080+0000","last_fullsized":"2024-01-15T17:58:21.164080+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:26.728833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163770+0000","last_change":"2024-01-15T17:58:19.014552+0000","last_active":"2024-01-15T17:58:21.163770+0000","last_peered":"2024-01-15T17:58:21.163770+0000","last_clean":"2024-01-15T17:58:21.163770+0000","last_became_active":"2024-01-15T17:58:19.008921+0000","last_became_peered":"2024-01-15T17:58:19.008921+0000","last_unstale":"2024-01-15T17:58:21.163770+0000","last_undegraded":"2024-01-15T17:58:21.163770+0000","last_fullsized":"2024-01-15T17:58:21.163770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:51.429644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986412+0000","last_change":"2024-01-15T17:58:18.988600+0000","last_active":"2024-01-15T17:58:19.986412+0000","last_peered":"2024-01-15T17:58:19.986412+0000","last_clean":"2024-01-15T17:58:19.986412+0000","last_became_active":"2024-01-15T17:58:18.987658+0000","last_became_peered":"2024-01-15T17:58:18.987658+0000","last_unstale":"2024-01-15T17:58:19.986412+0000","last_undegraded":"2024-01-15T17:58:19.986412+0000","last_fullsized":"2024-01-15T17:58:19.986412+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:35.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158316+0000","last_change":"2024-01-15T17:58:18.998942+0000","last_active":"2024-01-15T17:58:21.158316+0000","last_peered":"2024-01-15T17:58:21.158316+0000","last_clean":"2024-01-15T17:58:21.158316+0000","last_became_active":"2024-01-15T17:58:18.998650+0000","last_became_peered":"2024-01-15T17:58:18.998650+0000","last_unstale":"2024-01-15T17:58:21.158316+0000","last_undegraded":"2024-01-15T17:58:21.158316+0000","last_fullsized":"2024-01-15T17:58:21.158316+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:39.127046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162172+0000","last_change":"2024-01-15T17:58:19.001928+0000","last_active":"2024-01-15T17:58:21.162172+0000","last_peered":"2024-01-15T17:58:21.162172+0000","last_clean":"2024-01-15T17:58:21.162172+0000","last_became_active":"2024-01-15T17:58:19.000617+0000","last_became_peered":"2024-01-15T17:58:19.000617+0000","last_unstale":"2024-01-15T17:58:21.162172+0000","last_undegraded":"2024-01-15T17:58:21.162172+0000","last_fullsized":"2024-01-15T17:58:21.162172+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:01.355486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159929+0000","last_change":"2024-01-15T17:58:18.987718+0000","last_active":"2024-01-15T17:58:21.159929+0000","last_peered":"2024-01-15T17:58:21.159929+0000","last_clean":"2024-01-15T17:58:21.159929+0000","last_became_active":"2024-01-15T17:58:18.987390+0000","last_became_peered":"2024-01-15T17:58:18.987390+0000","last_unstale":"2024-01-15T17:58:21.159929+0000","last_undegraded":"2024-01-15T17:58:21.159929+0000","last_fullsized":"2024-01-15T17:58:21.159929+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:05.757201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985536+0000","last_change":"2024-01-15T17:58:19.004638+0000","last_active":"2024-01-15T17:58:19.985536+0000","last_peered":"2024-01-15T17:58:19.985536+0000","last_clean":"2024-01-15T17:58:19.985536+0000","last_became_active":"2024-01-15T17:58:19.004325+0000","last_became_peered":"2024-01-15T17:58:19.004325+0000","last_unstale":"2024-01-15T17:58:19.985536+0000","last_undegraded":"2024-01-15T17:58:19.985536+0000","last_fullsized":"2024-01-15T17:58:19.985536+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:47.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346067+0000","last_change":"2024-01-15T17:58:18.987202+0000","last_active":"2024-01-15T17:58:20.346067+0000","last_peered":"2024-01-15T17:58:20.346067+0000","last_clean":"2024-01-15T17:58:20.346067+0000","last_became_active":"2024-01-15T17:58:18.986911+0000","last_became_peered":"2024-01-15T17:58:18.986911+0000","last_unstale":"2024-01-15T17:58:20.346067+0000","last_undegraded":"2024-01-15T17:58:20.346067+0000","last_fullsized":"2024-01-15T17:58:20.346067+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:01.355486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346145+0000","last_change":"2024-01-15T17:58:19.016790+0000","last_active":"2024-01-15T17:58:20.346145+0000","last_peered":"2024-01-15T17:58:20.346145+0000","last_clean":"2024-01-15T17:58:20.346145+0000","last_became_active":"2024-01-15T17:58:19.016452+0000","last_became_peered":"2024-01-15T17:58:19.016452+0000","last_unstale":"2024-01-15T17:58:20.346145+0000","last_undegraded":"2024-01-15T17:58:20.346145+0000","last_fullsized":"2024-01-15T17:58:20.346145+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:01:54.772028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987535+0000","last_change":"2024-01-15T17:58:19.012051+0000","last_active":"2024-01-15T17:58:19.987535+0000","last_peered":"2024-01-15T17:58:19.987535+0000","last_clean":"2024-01-15T17:58:19.987535+0000","last_became_active":"2024-01-15T17:58:19.011643+0000","last_became_peered":"2024-01-15T17:58:19.011643+0000","last_unstale":"2024-01-15T17:58:19.987535+0000","last_undegraded":"2024-01-15T17:58:19.987535+0000","last_fullsized":"2024-01-15T17:58:19.987535+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:28:46.759912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158121+0000","last_change":"2024-01-15T17:58:19.011350+0000","last_active":"2024-01-15T17:58:21.158121+0000","last_peered":"2024-01-15T17:58:21.158121+0000","last_clean":"2024-01-15T17:58:21.158121+0000","last_became_active":"2024-01-15T17:58:19.011046+0000","last_became_peered":"2024-01-15T17:58:19.011046+0000","last_unstale":"2024-01-15T17:58:21.158121+0000","last_undegraded":"2024-01-15T17:58:21.158121+0000","last_fullsized":"2024-01-15T17:58:21.158121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:12:37.308652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161075+0000","last_change":"2024-01-15T17:58:18.996921+0000","last_active":"2024-01-15T17:58:21.161075+0000","last_peered":"2024-01-15T17:58:21.161075+0000","last_clean":"2024-01-15T17:58:21.161075+0000","last_became_active":"2024-01-15T17:58:18.994377+0000","last_became_peered":"2024-01-15T17:58:18.994377+0000","last_unstale":"2024-01-15T17:58:21.161075+0000","last_undegraded":"2024-01-15T17:58:21.161075+0000","last_fullsized":"2024-01-15T17:58:21.161075+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:54:37.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162353+0000","last_change":"2024-01-15T17:58:19.006417+0000","last_active":"2024-01-15T17:58:21.162353+0000","last_peered":"2024-01-15T17:58:21.162353+0000","last_clean":"2024-01-15T17:58:21.162353+0000","last_became_active":"2024-01-15T17:58:19.002993+0000","last_became_peered":"2024-01-15T17:58:19.002993+0000","last_unstale":"2024-01-15T17:58:21.162353+0000","last_undegraded":"2024-01-15T17:58:21.162353+0000","last_fullsized":"2024-01-15T17:58:21.162353+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:01:54.772028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986705+0000","last_change":"2024-01-15T17:58:18.995694+0000","last_active":"2024-01-15T17:58:19.986705+0000","last_peered":"2024-01-15T17:58:19.986705+0000","last_clean":"2024-01-15T17:58:19.986705+0000","last_became_active":"2024-01-15T17:58:18.993968+0000","last_became_peered":"2024-01-15T17:58:18.993968+0000","last_unstale":"2024-01-15T17:58:19.986705+0000","last_undegraded":"2024-01-15T17:58:19.986705+0000","last_fullsized":"2024-01-15T17:58:19.986705+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:25:46.220536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985649+0000","last_change":"2024-01-15T17:58:18.995548+0000","last_active":"2024-01-15T17:58:19.985649+0000","last_peered":"2024-01-15T17:58:19.985649+0000","last_clean":"2024-01-15T17:58:19.985649+0000","last_became_active":"2024-01-15T17:58:18.995281+0000","last_became_peered":"2024-01-15T17:58:18.995281+0000","last_unstale":"2024-01-15T17:58:19.985649+0000","last_undegraded":"2024-01-15T17:58:19.985649+0000","last_fullsized":"2024-01-15T17:58:19.985649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:28:02.329866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986241+0000","last_change":"2024-01-15T17:58:18.992172+0000","last_active":"2024-01-15T17:58:19.986241+0000","last_peered":"2024-01-15T17:58:19.986241+0000","last_clean":"2024-01-15T17:58:19.986241+0000","last_became_active":"2024-01-15T17:58:18.991535+0000","last_became_peered":"2024-01-15T17:58:18.991535+0000","last_unstale":"2024-01-15T17:58:19.986241+0000","last_undegraded":"2024-01-15T17:58:19.986241+0000","last_fullsized":"2024-01-15T17:58:19.986241+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:07.285598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158273+0000","last_change":"2024-01-15T17:58:18.996294+0000","last_active":"2024-01-15T17:58:21.158273+0000","last_peered":"2024-01-15T17:58:21.158273+0000","last_clean":"2024-01-15T17:58:21.158273+0000","last_became_active":"2024-01-15T17:58:18.996025+0000","last_became_peered":"2024-01-15T17:58:18.996025+0000","last_unstale":"2024-01-15T17:58:21.158273+0000","last_undegraded":"2024-01-15T17:58:21.158273+0000","last_fullsized":"2024-01-15T17:58:21.158273+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:23:51.721700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346062+0000","last_change":"2024-01-15T17:58:18.984770+0000","last_active":"2024-01-15T17:58:20.346062+0000","last_peered":"2024-01-15T17:58:20.346062+0000","last_clean":"2024-01-15T17:58:20.346062+0000","last_became_active":"2024-01-15T17:58:18.984406+0000","last_became_peered":"2024-01-15T17:58:18.984406+0000","last_unstale":"2024-01-15T17:58:20.346062+0000","last_undegraded":"2024-01-15T17:58:20.346062+0000","last_fullsized":"2024-01-15T17:58:20.346062+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:10.953697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986646+0000","last_change":"2024-01-15T17:58:18.988385+0000","last_active":"2024-01-15T17:58:19.986646+0000","last_peered":"2024-01-15T17:58:19.986646+0000","last_clean":"2024-01-15T17:58:19.986646+0000","last_became_active":"2024-01-15T17:58:18.987296+0000","last_became_peered":"2024-01-15T17:58:18.987296+0000","last_unstale":"2024-01-15T17:58:19.986646+0000","last_undegraded":"2024-01-15T17:58:19.986646+0000","last_fullsized":"2024-01-15T17:58:19.986646+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:26.728833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986444+0000","last_change":"2024-01-15T17:58:18.987849+0000","last_active":"2024-01-15T17:58:19.986444+0000","last_peered":"2024-01-15T17:58:19.986444+0000","last_clean":"2024-01-15T17:58:19.986444+0000","last_became_active":"2024-01-15T17:58:18.987504+0000","last_became_peered":"2024-01-15T17:58:18.987504+0000","last_unstale":"2024-01-15T17:58:19.986444+0000","last_undegraded":"2024-01-15T17:58:19.986444+0000","last_fullsized":"2024-01-15T17:58:19.986444+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:43.781847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345606+0000","last_change":"2024-01-15T17:58:18.994812+0000","last_active":"2024-01-15T17:58:20.345606+0000","last_peered":"2024-01-15T17:58:20.345606+0000","last_clean":"2024-01-15T17:58:20.345606+0000","last_became_active":"2024-01-15T17:58:18.994420+0000","last_became_peered":"2024-01-15T17:58:18.994420+0000","last_unstale":"2024-01-15T17:58:20.345606+0000","last_undegraded":"2024-01-15T17:58:20.345606+0000","last_fullsized":"2024-01-15T17:58:20.345606+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:33.875326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160190+0000","last_change":"2024-01-15T17:58:18.987732+0000","last_active":"2024-01-15T17:58:21.160190+0000","last_peered":"2024-01-15T17:58:21.160190+0000","last_clean":"2024-01-15T17:58:21.160190+0000","last_became_active":"2024-01-15T17:58:18.986144+0000","last_became_peered":"2024-01-15T17:58:18.986144+0000","last_unstale":"2024-01-15T17:58:21.160190+0000","last_undegraded":"2024-01-15T17:58:21.160190+0000","last_fullsized":"2024-01-15T17:58:21.160190+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:52.164647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.157748+0000","last_change":"2024-01-15T17:58:18.983829+0000","last_active":"2024-01-15T17:58:21.157748+0000","last_peered":"2024-01-15T17:58:21.157748+0000","last_clean":"2024-01-15T17:58:21.157748+0000","last_became_active":"2024-01-15T17:58:18.983528+0000","last_became_peered":"2024-01-15T17:58:18.983528+0000","last_unstale":"2024-01-15T17:58:21.157748+0000","last_undegraded":"2024-01-15T17:58:21.157748+0000","last_fullsized":"2024-01-15T17:58:21.157748+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:53.234232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.979342+0000","last_change":"2024-01-15T17:58:18.985545+0000","last_active":"2024-01-15T17:58:19.979342+0000","last_peered":"2024-01-15T17:58:19.979342+0000","last_clean":"2024-01-15T17:58:19.979342+0000","last_became_active":"2024-01-15T17:58:18.981858+0000","last_became_peered":"2024-01-15T17:58:18.981858+0000","last_unstale":"2024-01-15T17:58:19.979342+0000","last_undegraded":"2024-01-15T17:58:19.979342+0000","last_fullsized":"2024-01-15T17:58:19.979342+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:43.781847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.018482+0000","last_change":"2024-01-15T17:58:17.980564+0000","last_active":"2024-01-15T17:58:19.018482+0000","last_peered":"2024-01-15T17:58:19.018482+0000","last_clean":"2024-01-15T17:58:19.018482+0000","last_became_active":"2024-01-15T17:58:17.979889+0000","last_became_peered":"2024-01-15T17:58:17.979889+0000","last_unstale":"2024-01-15T17:58:19.018482+0000","last_undegraded":"2024-01-15T17:58:19.018482+0000","last_fullsized":"2024-01-15T17:58:19.018482+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:14:34.561030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986677+0000","last_change":"2024-01-15T17:58:18.987759+0000","last_active":"2024-01-15T17:58:19.986677+0000","last_peered":"2024-01-15T17:58:19.986677+0000","last_clean":"2024-01-15T17:58:19.986677+0000","last_became_active":"2024-01-15T17:58:18.987413+0000","last_became_peered":"2024-01-15T17:58:18.987413+0000","last_unstale":"2024-01-15T17:58:19.986677+0000","last_undegraded":"2024-01-15T17:58:19.986677+0000","last_fullsized":"2024-01-15T17:58:19.986677+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:22.217018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019379+0000","last_change":"2024-01-15T17:58:17.981282+0000","last_active":"2024-01-15T17:58:19.019379+0000","last_peered":"2024-01-15T17:58:19.019379+0000","last_clean":"2024-01-15T17:58:19.019379+0000","last_became_active":"2024-01-15T17:58:17.980794+0000","last_became_peered":"2024-01-15T17:58:17.980794+0000","last_unstale":"2024-01-15T17:58:19.019379+0000","last_undegraded":"2024-01-15T17:58:19.019379+0000","last_fullsized":"2024-01-15T17:58:19.019379+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:15.887231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986915+0000","last_change":"2024-01-15T17:58:18.988826+0000","last_active":"2024-01-15T17:58:19.986915+0000","last_peered":"2024-01-15T17:58:19.986915+0000","last_clean":"2024-01-15T17:58:19.986915+0000","last_became_active":"2024-01-15T17:58:18.988087+0000","last_became_peered":"2024-01-15T17:58:18.988087+0000","last_unstale":"2024-01-15T17:58:19.986915+0000","last_undegraded":"2024-01-15T17:58:19.986915+0000","last_fullsized":"2024-01-15T17:58:19.986915+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:18.389244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.023715+0000","last_change":"2024-01-15T17:58:17.990072+0000","last_active":"2024-01-15T17:58:19.023715+0000","last_peered":"2024-01-15T17:58:19.023715+0000","last_clean":"2024-01-15T17:58:19.023715+0000","last_became_active":"2024-01-15T17:58:17.989788+0000","last_became_peered":"2024-01-15T17:58:17.989788+0000","last_unstale":"2024-01-15T17:58:19.023715+0000","last_undegraded":"2024-01-15T17:58:19.023715+0000","last_fullsized":"2024-01-15T17:58:19.023715+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:09:54.096692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.979523+0000","last_change":"2024-01-15T17:58:18.997500+0000","last_active":"2024-01-15T17:58:19.979523+0000","last_peered":"2024-01-15T17:58:19.979523+0000","last_clean":"2024-01-15T17:58:19.979523+0000","last_became_active":"2024-01-15T17:58:18.997234+0000","last_became_peered":"2024-01-15T17:58:18.997234+0000","last_unstale":"2024-01-15T17:58:19.979523+0000","last_undegraded":"2024-01-15T17:58:19.979523+0000","last_fullsized":"2024-01-15T17:58:19.979523+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:22.217018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021174+0000","last_change":"2024-01-15T17:58:17.989934+0000","last_active":"2024-01-15T17:58:19.021174+0000","last_peered":"2024-01-15T17:58:19.021174+0000","last_clean":"2024-01-15T17:58:19.021174+0000","last_became_active":"2024-01-15T17:58:17.989582+0000","last_became_peered":"2024-01-15T17:58:17.989582+0000","last_unstale":"2024-01-15T17:58:19.021174+0000","last_undegraded":"2024-01-15T17:58:19.021174+0000","last_fullsized":"2024-01-15T17:58:19.021174+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:04.365732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345619+0000","last_change":"2024-01-15T17:58:18.986303+0000","last_active":"2024-01-15T17:58:20.345619+0000","last_peered":"2024-01-15T17:58:20.345619+0000","last_clean":"2024-01-15T17:58:20.345619+0000","last_became_active":"2024-01-15T17:58:18.985470+0000","last_became_peered":"2024-01-15T17:58:18.985470+0000","last_unstale":"2024-01-15T17:58:20.345619+0000","last_undegraded":"2024-01-15T17:58:20.345619+0000","last_fullsized":"2024-01-15T17:58:20.345619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:05.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.018703+0000","last_change":"2024-01-15T17:58:17.984103+0000","last_active":"2024-01-15T17:58:19.018703+0000","last_peered":"2024-01-15T17:58:19.018703+0000","last_clean":"2024-01-15T17:58:19.018703+0000","last_became_active":"2024-01-15T17:58:17.983807+0000","last_became_peered":"2024-01-15T17:58:17.983807+0000","last_unstale":"2024-01-15T17:58:19.018703+0000","last_undegraded":"2024-01-15T17:58:19.018703+0000","last_fullsized":"2024-01-15T17:58:19.018703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:25.210788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163150+0000","last_change":"2024-01-15T17:58:19.002031+0000","last_active":"2024-01-15T17:58:21.163150+0000","last_peered":"2024-01-15T17:58:21.163150+0000","last_clean":"2024-01-15T17:58:21.163150+0000","last_became_active":"2024-01-15T17:58:19.001699+0000","last_became_peered":"2024-01-15T17:58:19.001699+0000","last_unstale":"2024-01-15T17:58:21.163150+0000","last_undegraded":"2024-01-15T17:58:21.163150+0000","last_fullsized":"2024-01-15T17:58:21.163150+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:48.531831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019861+0000","last_change":"2024-01-15T17:58:17.987855+0000","last_active":"2024-01-15T17:58:19.019861+0000","last_peered":"2024-01-15T17:58:19.019861+0000","last_clean":"2024-01-15T17:58:19.019861+0000","last_became_active":"2024-01-15T17:58:17.987507+0000","last_became_peered":"2024-01-15T17:58:17.987507+0000","last_unstale":"2024-01-15T17:58:19.019861+0000","last_undegraded":"2024-01-15T17:58:19.019861+0000","last_fullsized":"2024-01-15T17:58:19.019861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:42.768090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161051+0000","last_change":"2024-01-15T17:58:19.003102+0000","last_active":"2024-01-15T17:58:21.161051+0000","last_peered":"2024-01-15T17:58:21.161051+0000","last_clean":"2024-01-15T17:58:21.161051+0000","last_became_active":"2024-01-15T17:58:19.002788+0000","last_became_peered":"2024-01-15T17:58:19.002788+0000","last_unstale":"2024-01-15T17:58:21.161051+0000","last_undegraded":"2024-01-15T17:58:21.161051+0000","last_fullsized":"2024-01-15T17:58:21.161051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:05.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.975625+0000","last_change":"2024-01-15T17:58:17.991304+0000","last_active":"2024-01-15T17:58:18.975625+0000","last_peered":"2024-01-15T17:58:18.975625+0000","last_clean":"2024-01-15T17:58:18.975625+0000","last_became_active":"2024-01-15T17:58:17.990871+0000","last_became_peered":"2024-01-15T17:58:17.990871+0000","last_unstale":"2024-01-15T17:58:18.975625+0000","last_undegraded":"2024-01-15T17:58:18.975625+0000","last_fullsized":"2024-01-15T17:58:18.975625+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:09.939940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985537+0000","last_change":"2024-01-15T17:58:18.995504+0000","last_active":"2024-01-15T17:58:19.985537+0000","last_peered":"2024-01-15T17:58:19.985537+0000","last_clean":"2024-01-15T17:58:19.985537+0000","last_became_active":"2024-01-15T17:58:18.995235+0000","last_became_peered":"2024-01-15T17:58:18.995235+0000","last_unstale":"2024-01-15T17:58:19.985537+0000","last_undegraded":"2024-01-15T17:58:19.985537+0000","last_fullsized":"2024-01-15T17:58:19.985537+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:39.127046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.987590+0000","last_change":"2024-01-15T17:58:17.977262+0000","last_active":"2024-01-15T17:58:18.987590+0000","last_peered":"2024-01-15T17:58:18.987590+0000","last_clean":"2024-01-15T17:58:18.987590+0000","last_became_active":"2024-01-15T17:58:17.976960+0000","last_became_peered":"2024-01-15T17:58:17.976960+0000","last_unstale":"2024-01-15T17:58:18.987590+0000","last_undegraded":"2024-01-15T17:58:18.987590+0000","last_fullsized":"2024-01-15T17:58:18.987590+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:21.203261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346224+0000","last_change":"2024-01-15T17:58:19.017344+0000","last_active":"2024-01-15T17:58:20.346224+0000","last_peered":"2024-01-15T17:58:20.346224+0000","last_clean":"2024-01-15T17:58:20.346224+0000","last_became_active":"2024-01-15T17:58:19.016961+0000","last_became_peered":"2024-01-15T17:58:19.016961+0000","last_unstale":"2024-01-15T17:58:20.346224+0000","last_undegraded":"2024-01-15T17:58:20.346224+0000","last_fullsized":"2024-01-15T17:58:20.346224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:48.531831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.967198+0000","last_change":"2024-01-15T17:58:17.984002+0000","last_active":"2024-01-15T17:58:18.967198+0000","last_peered":"2024-01-15T17:58:18.967198+0000","last_clean":"2024-01-15T17:58:18.967198+0000","last_became_active":"2024-01-15T17:58:17.983646+0000","last_became_peered":"2024-01-15T17:58:17.983646+0000","last_unstale":"2024-01-15T17:58:18.967198+0000","last_undegraded":"2024-01-15T17:58:18.967198+0000","last_fullsized":"2024-01-15T17:58:18.967198+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:39:43.632893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159543+0000","last_change":"2024-01-15T17:58:18.994672+0000","last_active":"2024-01-15T17:58:21.159543+0000","last_peered":"2024-01-15T17:58:21.159543+0000","last_clean":"2024-01-15T17:58:21.159543+0000","last_became_active":"2024-01-15T17:58:18.994367+0000","last_became_peered":"2024-01-15T17:58:18.994367+0000","last_unstale":"2024-01-15T17:58:21.159543+0000","last_undegraded":"2024-01-15T17:58:21.159543+0000","last_fullsized":"2024-01-15T17:58:21.159543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:53.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.997468+0000","last_change":"2024-01-15T17:58:17.988879+0000","last_active":"2024-01-15T17:58:18.997468+0000","last_peered":"2024-01-15T17:58:18.997468+0000","last_clean":"2024-01-15T17:58:18.997468+0000","last_became_active":"2024-01-15T17:58:17.988538+0000","last_became_peered":"2024-01-15T17:58:17.988538+0000","last_unstale":"2024-01-15T17:58:18.997468+0000","last_undegraded":"2024-01-15T17:58:18.997468+0000","last_fullsized":"2024-01-15T17:58:18.997468+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:06:27.277594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162839+0000","last_change":"2024-01-15T17:58:18.990318+0000","last_active":"2024-01-15T17:58:21.162839+0000","last_peered":"2024-01-15T17:58:21.162839+0000","last_clean":"2024-01-15T17:58:21.162839+0000","last_became_active":"2024-01-15T17:58:18.987210+0000","last_became_peered":"2024-01-15T17:58:18.987210+0000","last_unstale":"2024-01-15T17:58:21.162839+0000","last_undegraded":"2024-01-15T17:58:21.162839+0000","last_fullsized":"2024-01-15T17:58:21.162839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:01:42.464858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021391+0000","last_change":"2024-01-15T17:58:17.990723+0000","last_active":"2024-01-15T17:58:19.021391+0000","last_peered":"2024-01-15T17:58:19.021391+0000","last_clean":"2024-01-15T17:58:19.021391+0000","last_became_active":"2024-01-15T17:58:17.989841+0000","last_became_peered":"2024-01-15T17:58:17.989841+0000","last_unstale":"2024-01-15T17:58:19.021391+0000","last_undegraded":"2024-01-15T17:58:19.021391+0000","last_fullsized":"2024-01-15T17:58:19.021391+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:47.518074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159377+0000","last_change":"2024-01-15T17:58:18.992357+0000","last_active":"2024-01-15T17:58:21.159377+0000","last_peered":"2024-01-15T17:58:21.159377+0000","last_clean":"2024-01-15T17:58:21.159377+0000","last_became_active":"2024-01-15T17:58:18.992039+0000","last_became_peered":"2024-01-15T17:58:18.992039+0000","last_unstale":"2024-01-15T17:58:21.159377+0000","last_undegraded":"2024-01-15T17:58:21.159377+0000","last_fullsized":"2024-01-15T17:58:21.159377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:59:47.920817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.028902+0000","last_change":"2024-01-15T17:58:17.984206+0000","last_active":"2024-01-15T17:58:19.028902+0000","last_peered":"2024-01-15T17:58:19.028902+0000","last_clean":"2024-01-15T17:58:19.028902+0000","last_became_active":"2024-01-15T17:58:17.983944+0000","last_became_peered":"2024-01-15T17:58:17.983944+0000","last_unstale":"2024-01-15T17:58:19.028902+0000","last_undegraded":"2024-01-15T17:58:19.028902+0000","last_fullsized":"2024-01-15T17:58:19.028902+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:16.883155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987487+0000","last_change":"2024-01-15T17:58:19.004493+0000","last_active":"2024-01-15T17:58:19.987487+0000","last_peered":"2024-01-15T17:58:19.987487+0000","last_clean":"2024-01-15T17:58:19.987487+0000","last_became_active":"2024-01-15T17:58:19.004276+0000","last_became_peered":"2024-01-15T17:58:19.004276+0000","last_unstale":"2024-01-15T17:58:19.987487+0000","last_undegraded":"2024-01-15T17:58:19.987487+0000","last_fullsized":"2024-01-15T17:58:19.987487+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:12:41.726472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021349+0000","last_change":"2024-01-15T17:58:17.980792+0000","last_active":"2024-01-15T17:58:19.021349+0000","last_peered":"2024-01-15T17:58:19.021349+0000","last_clean":"2024-01-15T17:58:19.021349+0000","last_became_active":"2024-01-15T17:58:17.980499+0000","last_became_peered":"2024-01-15T17:58:17.980499+0000","last_unstale":"2024-01-15T17:58:19.021349+0000","last_undegraded":"2024-01-15T17:58:19.021349+0000","last_fullsized":"2024-01-15T17:58:19.021349+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:51:40.514469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163360+0000","last_change":"2024-01-15T17:58:19.001940+0000","last_active":"2024-01-15T17:58:21.163360+0000","last_peered":"2024-01-15T17:58:21.163360+0000","last_clean":"2024-01-15T17:58:21.163360+0000","last_became_active":"2024-01-15T17:58:19.000612+0000","last_became_peered":"2024-01-15T17:58:19.000612+0000","last_unstale":"2024-01-15T17:58:21.163360+0000","last_undegraded":"2024-01-15T17:58:21.163360+0000","last_fullsized":"2024-01-15T17:58:21.163360+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:28.109432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.986632+0000","last_change":"2024-01-15T17:58:17.988834+0000","last_active":"2024-01-15T17:58:18.986632+0000","last_peered":"2024-01-15T17:58:18.986632+0000","last_clean":"2024-01-15T17:58:18.986632+0000","last_became_active":"2024-01-15T17:58:17.988417+0000","last_became_peered":"2024-01-15T17:58:17.988417+0000","last_unstale":"2024-01-15T17:58:18.986632+0000","last_undegraded":"2024-01-15T17:58:18.986632+0000","last_fullsized":"2024-01-15T17:58:18.986632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:25.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159690+0000","last_change":"2024-01-15T17:58:18.987877+0000","last_active":"2024-01-15T17:58:21.159690+0000","last_peered":"2024-01-15T17:58:21.159690+0000","last_clean":"2024-01-15T17:58:21.159690+0000","last_became_active":"2024-01-15T17:58:18.987494+0000","last_became_peered":"2024-01-15T17:58:18.987494+0000","last_unstale":"2024-01-15T17:58:21.159690+0000","last_undegraded":"2024-01-15T17:58:21.159690+0000","last_fullsized":"2024-01-15T17:58:21.159690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:56:15.494821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.963210+0000","last_change":"2024-01-15T17:58:17.978177+0000","last_active":"2024-01-15T17:58:18.963210+0000","last_peered":"2024-01-15T17:58:18.963210+0000","last_clean":"2024-01-15T17:58:18.963210+0000","last_became_active":"2024-01-15T17:58:17.977406+0000","last_became_peered":"2024-01-15T17:58:17.977406+0000","last_unstale":"2024-01-15T17:58:18.963210+0000","last_undegraded":"2024-01-15T17:58:18.963210+0000","last_fullsized":"2024-01-15T17:58:18.963210+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:33:14.070510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158955+0000","last_change":"2024-01-15T17:58:18.994744+0000","last_active":"2024-01-15T17:58:21.158955+0000","last_peered":"2024-01-15T17:58:21.158955+0000","last_clean":"2024-01-15T17:58:21.158955+0000","last_became_active":"2024-01-15T17:58:18.994405+0000","last_became_peered":"2024-01-15T17:58:18.994405+0000","last_unstale":"2024-01-15T17:58:21.158955+0000","last_undegraded":"2024-01-15T17:58:21.158955+0000","last_fullsized":"2024-01-15T17:58:21.158955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:09.274759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.026888+0000","last_change":"2024-01-15T17:58:17.987803+0000","last_active":"2024-01-15T17:58:19.026888+0000","last_peered":"2024-01-15T17:58:19.026888+0000","last_clean":"2024-01-15T17:58:19.026888+0000","last_became_active":"2024-01-15T17:58:17.987169+0000","last_became_peered":"2024-01-15T17:58:17.987169+0000","last_unstale":"2024-01-15T17:58:19.026888+0000","last_undegraded":"2024-01-15T17:58:19.026888+0000","last_fullsized":"2024-01-15T17:58:19.026888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:25:45.206779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158413+0000","last_change":"2024-01-15T17:58:18.987935+0000","last_active":"2024-01-15T17:58:21.158413+0000","last_peered":"2024-01-15T17:58:21.158413+0000","last_clean":"2024-01-15T17:58:21.158413+0000","last_became_active":"2024-01-15T17:58:18.987605+0000","last_became_peered":"2024-01-15T17:58:18.987605+0000","last_unstale":"2024-01-15T17:58:21.158413+0000","last_undegraded":"2024-01-15T17:58:21.158413+0000","last_fullsized":"2024-01-15T17:58:21.158413+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:47.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.020407+0000","last_change":"2024-01-15T17:58:17.983214+0000","last_active":"2024-01-15T17:58:19.020407+0000","last_peered":"2024-01-15T17:58:19.020407+0000","last_clean":"2024-01-15T17:58:19.020407+0000","last_became_active":"2024-01-15T17:58:17.981263+0000","last_became_peered":"2024-01-15T17:58:17.981263+0000","last_unstale":"2024-01-15T17:58:19.020407+0000","last_undegraded":"2024-01-15T17:58:19.020407+0000","last_fullsized":"2024-01-15T17:58:19.020407+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:40.241498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164081+0000","last_change":"2024-01-15T17:58:19.003039+0000","last_active":"2024-01-15T17:58:21.164081+0000","last_peered":"2024-01-15T17:58:21.164081+0000","last_clean":"2024-01-15T17:58:21.164081+0000","last_became_active":"2024-01-15T17:58:19.002748+0000","last_became_peered":"2024-01-15T17:58:19.002748+0000","last_unstale":"2024-01-15T17:58:21.164081+0000","last_undegraded":"2024-01-15T17:58:21.164081+0000","last_fullsized":"2024-01-15T17:58:21.164081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:35.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019305+0000","last_change":"2024-01-15T17:58:17.981090+0000","last_active":"2024-01-15T17:58:19.019305+0000","last_peered":"2024-01-15T17:58:19.019305+0000","last_clean":"2024-01-15T17:58:19.019305+0000","last_became_active":"2024-01-15T17:58:17.978330+0000","last_became_peered":"2024-01-15T17:58:17.978330+0000","last_unstale":"2024-01-15T17:58:19.019305+0000","last_undegraded":"2024-01-15T17:58:19.019305+0000","last_fullsized":"2024-01-15T17:58:19.019305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:28:45.746156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985724+0000","last_change":"2024-01-15T17:58:18.997258+0000","last_active":"2024-01-15T17:58:19.985724+0000","last_peered":"2024-01-15T17:58:19.985724+0000","last_clean":"2024-01-15T17:58:19.985724+0000","last_became_active":"2024-01-15T17:58:18.996811+0000","last_became_peered":"2024-01-15T17:58:18.996811+0000","last_unstale":"2024-01-15T17:58:19.985724+0000","last_undegraded":"2024-01-15T17:58:19.985724+0000","last_fullsized":"2024-01-15T17:58:19.985724+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:09:55.110449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.018745+0000","last_change":"2024-01-15T17:58:17.991610+0000","last_active":"2024-01-15T17:58:19.018745+0000","last_peered":"2024-01-15T17:58:19.018745+0000","last_clean":"2024-01-15T17:58:19.018745+0000","last_became_active":"2024-01-15T17:58:17.991339+0000","last_became_peered":"2024-01-15T17:58:17.991339+0000","last_unstale":"2024-01-15T17:58:19.018745+0000","last_undegraded":"2024-01-15T17:58:19.018745+0000","last_fullsized":"2024-01-15T17:58:19.018745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:04.365732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987156+0000","last_change":"2024-01-15T17:58:18.989452+0000","last_active":"2024-01-15T17:58:19.987156+0000","last_peered":"2024-01-15T17:58:19.987156+0000","last_clean":"2024-01-15T17:58:19.987156+0000","last_became_active":"2024-01-15T17:58:18.989257+0000","last_became_peered":"2024-01-15T17:58:18.989257+0000","last_unstale":"2024-01-15T17:58:19.987156+0000","last_undegraded":"2024-01-15T17:58:19.987156+0000","last_fullsized":"2024-01-15T17:58:19.987156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:15:10.413760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.962717+0000","last_change":"2024-01-15T17:58:17.979360+0000","last_active":"2024-01-15T17:58:18.962717+0000","last_peered":"2024-01-15T17:58:18.962717+0000","last_clean":"2024-01-15T17:58:18.962717+0000","last_became_active":"2024-01-15T17:58:17.979089+0000","last_became_peered":"2024-01-15T17:58:17.979089+0000","last_unstale":"2024-01-15T17:58:18.962717+0000","last_undegraded":"2024-01-15T17:58:18.962717+0000","last_fullsized":"2024-01-15T17:58:18.962717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:01:09.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161038+0000","last_change":"2024-01-15T17:58:18.989730+0000","last_active":"2024-01-15T17:58:21.161038+0000","last_peered":"2024-01-15T17:58:21.161038+0000","last_clean":"2024-01-15T17:58:21.161038+0000","last_became_active":"2024-01-15T17:58:18.989337+0000","last_became_peered":"2024-01-15T17:58:18.989337+0000","last_unstale":"2024-01-15T17:58:21.161038+0000","last_undegraded":"2024-01-15T17:58:21.161038+0000","last_fullsized":"2024-01-15T17:58:21.161038+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:32:02.467555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021507+0000","last_change":"2024-01-15T17:58:17.984692+0000","last_active":"2024-01-15T17:58:19.021507+0000","last_peered":"2024-01-15T17:58:19.021507+0000","last_clean":"2024-01-15T17:58:19.021507+0000","last_became_active":"2024-01-15T17:58:17.983559+0000","last_became_peered":"2024-01-15T17:58:17.983559+0000","last_unstale":"2024-01-15T17:58:19.021507+0000","last_undegraded":"2024-01-15T17:58:19.021507+0000","last_fullsized":"2024-01-15T17:58:19.021507+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:10:46.123640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.979093+0000","last_change":"2024-01-15T17:58:18.996551+0000","last_active":"2024-01-15T17:58:19.979093+0000","last_peered":"2024-01-15T17:58:19.979093+0000","last_clean":"2024-01-15T17:58:19.979093+0000","last_became_active":"2024-01-15T17:58:18.996211+0000","last_became_peered":"2024-01-15T17:58:18.996211+0000","last_unstale":"2024-01-15T17:58:19.979093+0000","last_undegraded":"2024-01-15T17:58:19.979093+0000","last_fullsized":"2024-01-15T17:58:19.979093+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:07.285598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.979050+0000","last_change":"2024-01-15T17:58:17.984051+0000","last_active":"2024-01-15T17:58:18.979050+0000","last_peered":"2024-01-15T17:58:18.979050+0000","last_clean":"2024-01-15T17:58:18.979050+0000","last_became_active":"2024-01-15T17:58:17.983756+0000","last_became_peered":"2024-01-15T17:58:17.983756+0000","last_unstale":"2024-01-15T17:58:18.979050+0000","last_undegraded":"2024-01-15T17:58:18.979050+0000","last_fullsized":"2024-01-15T17:58:18.979050+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:27.095675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.157598+0000","last_change":"2024-01-15T17:58:18.995108+0000","last_active":"2024-01-15T17:58:21.157598+0000","last_peered":"2024-01-15T17:58:21.157598+0000","last_clean":"2024-01-15T17:58:21.157598+0000","last_became_active":"2024-01-15T17:58:18.994708+0000","last_became_peered":"2024-01-15T17:58:18.994708+0000","last_unstale":"2024-01-15T17:58:21.157598+0000","last_undegraded":"2024-01-15T17:58:21.157598+0000","last_fullsized":"2024-01-15T17:58:21.157598+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.885376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.990894+0000","last_change":"2024-01-15T17:58:17.983625+0000","last_active":"2024-01-15T17:58:18.990894+0000","last_peered":"2024-01-15T17:58:18.990894+0000","last_clean":"2024-01-15T17:58:18.990894+0000","last_became_active":"2024-01-15T17:58:17.983282+0000","last_became_peered":"2024-01-15T17:58:17.983282+0000","last_unstale":"2024-01-15T17:58:18.990894+0000","last_undegraded":"2024-01-15T17:58:18.990894+0000","last_fullsized":"2024-01-15T17:58:18.990894+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:15.887231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161926+0000","last_change":"2024-01-15T17:58:19.013008+0000","last_active":"2024-01-15T17:58:21.161926+0000","last_peered":"2024-01-15T17:58:21.161926+0000","last_clean":"2024-01-15T17:58:21.161926+0000","last_became_active":"2024-01-15T17:58:19.007927+0000","last_became_peered":"2024-01-15T17:58:19.007927+0000","last_unstale":"2024-01-15T17:58:21.161926+0000","last_undegraded":"2024-01-15T17:58:21.161926+0000","last_fullsized":"2024-01-15T17:58:21.161926+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:10.953697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.983895+0000","last_change":"2024-01-15T17:58:17.990404+0000","last_active":"2024-01-15T17:58:18.983895+0000","last_peered":"2024-01-15T17:58:18.983895+0000","last_clean":"2024-01-15T17:58:18.983895+0000","last_became_active":"2024-01-15T17:58:17.989721+0000","last_became_peered":"2024-01-15T17:58:17.989721+0000","last_unstale":"2024-01-15T17:58:18.983895+0000","last_undegraded":"2024-01-15T17:58:18.983895+0000","last_fullsized":"2024-01-15T17:58:18.983895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:34.474345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987041+0000","last_change":"2024-01-15T17:58:19.004632+0000","last_active":"2024-01-15T17:58:19.987041+0000","last_peered":"2024-01-15T17:58:19.987041+0000","last_clean":"2024-01-15T17:58:19.987041+0000","last_became_active":"2024-01-15T17:58:19.004331+0000","last_became_peered":"2024-01-15T17:58:19.004331+0000","last_unstale":"2024-01-15T17:58:19.987041+0000","last_undegraded":"2024-01-15T17:58:19.987041+0000","last_fullsized":"2024-01-15T17:58:19.987041+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:39:44.646650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.963869+0000","last_change":"2024-01-15T17:58:17.984411+0000","last_active":"2024-01-15T17:58:18.963869+0000","last_peered":"2024-01-15T17:58:18.963869+0000","last_clean":"2024-01-15T17:58:18.963869+0000","last_became_active":"2024-01-15T17:58:17.984086+0000","last_became_peered":"2024-01-15T17:58:17.984086+0000","last_unstale":"2024-01-15T17:58:18.963869+0000","last_undegraded":"2024-01-15T17:58:18.963869+0000","last_fullsized":"2024-01-15T17:58:18.963869+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:03.255576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163518+0000","last_change":"2024-01-15T17:58:19.015017+0000","last_active":"2024-01-15T17:58:21.163518+0000","last_peered":"2024-01-15T17:58:21.163518+0000","last_clean":"2024-01-15T17:58:21.163518+0000","last_became_active":"2024-01-15T17:58:19.008911+0000","last_became_peered":"2024-01-15T17:58:19.008911+0000","last_unstale":"2024-01-15T17:58:21.163518+0000","last_undegraded":"2024-01-15T17:58:21.163518+0000","last_fullsized":"2024-01-15T17:58:21.163518+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:30.537768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.979570+0000","last_change":"2024-01-15T17:58:17.990981+0000","last_active":"2024-01-15T17:58:18.979570+0000","last_peered":"2024-01-15T17:58:18.979570+0000","last_clean":"2024-01-15T17:58:18.979570+0000","last_became_active":"2024-01-15T17:58:17.990431+0000","last_became_peered":"2024-01-15T17:58:17.990431+0000","last_unstale":"2024-01-15T17:58:18.979570+0000","last_undegraded":"2024-01-15T17:58:18.979570+0000","last_fullsized":"2024-01-15T17:58:18.979570+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:27.095675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159663+0000","last_change":"2024-01-15T17:58:18.991128+0000","last_active":"2024-01-15T17:58:21.159663+0000","last_peered":"2024-01-15T17:58:21.159663+0000","last_clean":"2024-01-15T17:58:21.159663+0000","last_became_active":"2024-01-15T17:58:18.990636+0000","last_became_peered":"2024-01-15T17:58:18.990636+0000","last_unstale":"2024-01-15T17:58:21.159663+0000","last_undegraded":"2024-01-15T17:58:21.159663+0000","last_fullsized":"2024-01-15T17:58:21.159663+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:25.096243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.982822+0000","last_change":"2024-01-15T17:58:17.988011+0000","last_active":"2024-01-15T17:58:18.982822+0000","last_peered":"2024-01-15T17:58:18.982822+0000","last_clean":"2024-01-15T17:58:18.982822+0000","last_became_active":"2024-01-15T17:58:17.987726+0000","last_became_peered":"2024-01-15T17:58:17.987726+0000","last_unstale":"2024-01-15T17:58:18.982822+0000","last_undegraded":"2024-01-15T17:58:18.982822+0000","last_fullsized":"2024-01-15T17:58:18.982822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:28:01.316109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160955+0000","last_change":"2024-01-15T17:58:18.989299+0000","last_active":"2024-01-15T17:58:21.160955+0000","last_peered":"2024-01-15T17:58:21.160955+0000","last_clean":"2024-01-15T17:58:21.160955+0000","last_became_active":"2024-01-15T17:58:18.988977+0000","last_became_peered":"2024-01-15T17:58:18.988977+0000","last_unstale":"2024-01-15T17:58:21.160955+0000","last_undegraded":"2024-01-15T17:58:21.160955+0000","last_fullsized":"2024-01-15T17:58:21.160955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:14:36.844811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.977953+0000","last_change":"2024-01-15T17:58:17.991529+0000","last_active":"2024-01-15T17:58:18.977953+0000","last_peered":"2024-01-15T17:58:18.977953+0000","last_clean":"2024-01-15T17:58:18.977953+0000","last_became_active":"2024-01-15T17:58:17.990561+0000","last_became_peered":"2024-01-15T17:58:17.990561+0000","last_unstale":"2024-01-15T17:58:18.977953+0000","last_undegraded":"2024-01-15T17:58:18.977953+0000","last_fullsized":"2024-01-15T17:58:18.977953+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:00.341729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161249+0000","last_change":"2024-01-15T17:58:18.992026+0000","last_active":"2024-01-15T17:58:21.161249+0000","last_peered":"2024-01-15T17:58:21.161249+0000","last_clean":"2024-01-15T17:58:21.161249+0000","last_became_active":"2024-01-15T17:58:18.991771+0000","last_became_peered":"2024-01-15T17:58:18.991771+0000","last_unstale":"2024-01-15T17:58:21.161249+0000","last_undegraded":"2024-01-15T17:58:21.161249+0000","last_fullsized":"2024-01-15T17:58:21.161249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:10.367838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.964680+0000","last_change":"2024-01-15T17:58:17.984128+0000","last_active":"2024-01-15T17:58:18.964680+0000","last_peered":"2024-01-15T17:58:18.964680+0000","last_clean":"2024-01-15T17:58:18.964680+0000","last_became_active":"2024-01-15T17:58:17.983852+0000","last_became_peered":"2024-01-15T17:58:17.983852+0000","last_unstale":"2024-01-15T17:58:18.964680+0000","last_undegraded":"2024-01-15T17:58:18.964680+0000","last_fullsized":"2024-01-15T17:58:18.964680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:06.271842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986298+0000","last_change":"2024-01-15T17:58:18.987205+0000","last_active":"2024-01-15T17:58:19.986298+0000","last_peered":"2024-01-15T17:58:19.986298+0000","last_clean":"2024-01-15T17:58:19.986298+0000","last_became_active":"2024-01-15T17:58:18.986891+0000","last_became_peered":"2024-01-15T17:58:18.986891+0000","last_unstale":"2024-01-15T17:58:19.986298+0000","last_undegraded":"2024-01-15T17:58:19.986298+0000","last_fullsized":"2024-01-15T17:58:19.986298+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:59.015634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.982669+0000","last_change":"2024-01-15T17:58:17.983233+0000","last_active":"2024-01-15T17:58:18.982669+0000","last_peered":"2024-01-15T17:58:18.982669+0000","last_clean":"2024-01-15T17:58:18.982669+0000","last_became_active":"2024-01-15T17:58:17.981967+0000","last_became_peered":"2024-01-15T17:58:17.981967+0000","last_unstale":"2024-01-15T17:58:18.982669+0000","last_undegraded":"2024-01-15T17:58:18.982669+0000","last_fullsized":"2024-01-15T17:58:18.982669+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:46.693010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345605+0000","last_change":"2024-01-15T17:58:18.979487+0000","last_active":"2024-01-15T17:58:20.345605+0000","last_peered":"2024-01-15T17:58:20.345605+0000","last_clean":"2024-01-15T17:58:20.345605+0000","last_became_active":"2024-01-15T17:58:18.977153+0000","last_became_peered":"2024-01-15T17:58:18.977153+0000","last_unstale":"2024-01-15T17:58:20.345605+0000","last_undegraded":"2024-01-15T17:58:20.345605+0000","last_fullsized":"2024-01-15T17:58:20.345605+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:10.697813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.987930+0000","last_change":"2024-01-15T17:58:17.976886+0000","last_active":"2024-01-15T17:58:18.987930+0000","last_peered":"2024-01-15T17:58:18.987930+0000","last_clean":"2024-01-15T17:58:18.987930+0000","last_became_active":"2024-01-15T17:58:17.976596+0000","last_became_peered":"2024-01-15T17:58:17.976596+0000","last_unstale":"2024-01-15T17:58:18.987930+0000","last_undegraded":"2024-01-15T17:58:18.987930+0000","last_fullsized":"2024-01-15T17:58:18.987930+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:39:43.632893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164256+0000","last_change":"2024-01-15T17:58:19.013503+0000","last_active":"2024-01-15T17:58:21.164256+0000","last_peered":"2024-01-15T17:58:21.164256+0000","last_clean":"2024-01-15T17:58:21.164256+0000","last_became_active":"2024-01-15T17:58:19.008625+0000","last_became_peered":"2024-01-15T17:58:19.008625+0000","last_unstale":"2024-01-15T17:58:21.164256+0000","last_undegraded":"2024-01-15T17:58:21.164256+0000","last_fullsized":"2024-01-15T17:58:21.164256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:18.389244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019643+0000","last_change":"2024-01-15T17:58:17.989391+0000","last_active":"2024-01-15T17:58:19.019643+0000","last_peered":"2024-01-15T17:58:19.019643+0000","last_clean":"2024-01-15T17:58:19.019643+0000","last_became_active":"2024-01-15T17:58:17.989106+0000","last_became_peered":"2024-01-15T17:58:17.989106+0000","last_unstale":"2024-01-15T17:58:19.019643+0000","last_undegraded":"2024-01-15T17:58:19.019643+0000","last_fullsized":"2024-01-15T17:58:19.019643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:47.518074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":185,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.983999+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:58:18.983999+0000","last_peered":"2024-01-15T17:58:18.983999+0000","last_clean":"2024-01-15T17:58:18.983999+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:58:18.983999+0000","last_undegraded":"2024-01-15T17:58:18.983999+0000","last_fullsized":"2024-01-15T17:58:18.983999+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":815062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1559999999999999,"15min":1.1539999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5700000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.175,"15min":1.1679999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.6619999999999999,"5min":1.804,"15min":1.804},"last":0.95599999999999996}]},{"osd":1,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.177},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.274,"15min":1.262},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.3460000000000001,"15min":1.3759999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.5389999999999999}]},{"osd":3,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.294,"15min":1.298},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.403},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.4890000000000001,"15min":1.4930000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":3.831,"5min":3.831,"15min":3.831},"last":1.49}]},{"osd":4,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4350000000000001,"15min":1.48},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.1890000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.222},{"interface":"front","average":{"1min":1.593,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":4.4729999999999999,"5min":4.4729999999999999,"15min":4.4729999999999999},"last":1.3360000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.6000000000000001,"15min":1.605},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1589999999999998,"5min":7.3819999999999997,"15min":7.3819999999999997},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.548,"15min":1.5860000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.1230000000000002,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.28}]},{"osd":6,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.649,"15min":1.6919999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9219999999999999,"5min":2.556,"15min":2.556},"last":1.377},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5880000000000001,"15min":1.625},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1400000000000001,"5min":2.649,"15min":2.649},"last":1.4650000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":225276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.409,"15min":1.3149999999999999},"min":{"1min":0.753,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":12.544,"5min":12.544,"15min":12.544},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.206,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.083}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.29,"15min":1.202},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":12.811999999999999,"5min":12.811999999999999,"15min":12.811999999999999},"last":1.514},{"interface":"front","average":{"1min":1.206,"5min":1.1870000000000001,"15min":1.179},"min":{"1min":0.83899999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4990000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.88}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.214,"15min":1.2},"min":{"1min":0.94299999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5549999999999999,"5min":1.788,"15min":1.788},"last":1.145},{"interface":"front","average":{"1min":2.0169999999999999,"5min":1.5680000000000001,"15min":1.5269999999999999},"min":{"1min":0.96899999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":12.897,"5min":12.897,"15min":12.897},"last":1.363}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.169,"5min":1.597,"15min":1.5069999999999999},"min":{"1min":1.23,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":12.717000000000001,"5min":12.717000000000001,"15min":12.717000000000001},"last":1.841},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.95399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.97,"5min":2.29,"15min":2.29},"last":1.7490000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.90700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6839999999999999,"5min":2.343,"15min":2.343},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.3740000000000001,"5min":1.7,"15min":1.6279999999999999},"min":{"1min":0.92000000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":16.98,"5min":16.98,"15min":16.98},"last":1.6319999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.413,"15min":1.4390000000000001},"min":{"1min":0.93500000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.101,"5min":2.548,"15min":2.548},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.583,"5min":1.6799999999999999,"15min":1.732},"min":{"1min":1.4339999999999999,"5min":1.163,"15min":1.163},"max":{"1min":1.8220000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.6839999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5249999999999999,"15min":1.5629999999999999},"min":{"1min":1.1459999999999999,"5min":1.016,"15min":1.016},"max":{"1min":2.1930000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.234},{"interface":"front","average":{"1min":1.575,"5min":1.6819999999999999,"15min":1.7370000000000001},"min":{"1min":1.3540000000000001,"5min":1.286,"15min":1.286},"max":{"1min":2.2480000000000002,"5min":3.141,"15min":3.141},"last":1.897}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":552960,"data_stored":224674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.206,"15min":1.2150000000000001},"min":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.681,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.26,"5min":1.3029999999999999,"15min":1.3600000000000001},"min":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.988,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.5449999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2130000000000001,"15min":1.1990000000000001},"min":{"1min":0.86499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2190000000000001,"15min":1.1970000000000001},"min":{"1min":0.91500000000000004,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.782,"5min":1.917,"15min":1.917},"last":1.1339999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2150000000000001,"15min":1.222},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7130000000000001,"5min":2.835,"15min":2.835},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.256},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.008}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.0670000000000002,"5min":1.48,"15min":1.3440000000000001},"min":{"1min":0.76000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":14.914,"5min":14.914,"15min":14.914},"last":1.23},{"interface":"front","average":{"1min":1.46,"5min":1.407,"15min":1.427},"min":{"1min":1.0409999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.944,"5min":3.302,"15min":3.302},"last":1.3580000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3839999999999999,"15min":1.407},"min":{"1min":1.03,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.8009999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.107},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.476,"15min":1.48},"min":{"1min":0.96899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":1.4910000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.577,"15min":1.625},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.0529999999999999,"5min":5.218,"15min":5.218},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.496,"5min":1.524,"15min":1.546},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.8260000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.4359999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.607,"15min":1.6220000000000001},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":4.4059999999999997,"5min":4.4059999999999997,"15min":4.4059999999999997},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6879999999999999,"15min":1.756},"min":{"1min":1.1379999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.2149999999999999,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.6619999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":561152,"data_stored":226216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2090000000000001,"15min":1.2110000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.2130000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":1.399,"15min":1.359},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":11.987,"5min":11.987,"15min":11.987},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2929999999999999,"15min":1.331},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9810000000000001,"5min":2.117,"15min":2.117},"last":1.242}]},{"osd":2,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.4450000000000001,"15min":1.4830000000000001},"min":{"1min":0.92800000000000005,"5min":0.745,"15min":0.745},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.181},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.333,"15min":1.3360000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0009999999999999,"5min":2.141,"15min":2.141},"last":1.5469999999999999}]},{"osd":3,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.528,"15min":1.5669999999999999},"min":{"1min":1.077,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.611},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.506,"15min":1.6040000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.931,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.1120000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.599,"15min":1.673},"min":{"1min":0.98299999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.5640000000000001,"15min":1.6699999999999999},"min":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1179999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.502}]},{"osd":6,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.5660000000000001,"15min":1.659},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.0419999999999998,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.821,"15min":1.837},"min":{"1min":0.92100000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":1.796}]},{"osd":7,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":2.0489999999999999,"5min":1.8759999999999999,"15min":1.8979999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":11.894,"5min":11.894,"15min":11.894},"last":1.395},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.851,"15min":1.903},"min":{"1min":1.419,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.7350000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35880,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335960,"statfs":{"total":96636764160,"available":96600023040,"internally_reserved":0,"allocated":548864,"data_stored":218914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3660000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.105},{"interface":"front","average":{"1min":1.121,"5min":1.129,"15min":1.089},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5589999999999999,"5min":3.008,"15min":3.008},"last":1.038}]},{"osd":1,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.772,"5min":1.1699999999999999,"15min":1.0609999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":12.391,"5min":12.391,"15min":12.391},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.0880000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.88800000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.169,"15min":1.1559999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.9399999999999999,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.373,"15min":1.3720000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.925,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.79200000000000004}]},{"osd":4,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.377},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.6850000000000001,"5min":4.226,"15min":4.226},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.4670000000000001,"15min":1.4299999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":7.3600000000000003,"5min":7.3600000000000003,"15min":7.3600000000000003},"last":1.6120000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.516,"15min":1.526},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.3090000000000002,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.631,"15min":1.623},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.7360000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":1.569}]},{"osd":6,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":2.052,"5min":1.7010000000000001,"15min":1.675},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":7.9630000000000001,"5min":7.9630000000000001,"15min":7.9630000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.788,"5min":1.671,"15min":1.6719999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.1659999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.671,"15min":1.6890000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.621,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":1.232},{"interface":"front","average":{"1min":1.9259999999999999,"5min":1.73,"15min":1.7210000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":7.407,"5min":7.407,"15min":7.407},"last":1.488}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":565248,"data_stored":226238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.1040000000000001,"15min":1.0409999999999999},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.427,"5min":14.427,"15min":14.427},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0049999999999999,"15min":0.99299999999999999},"min":{"1min":0.437,"5min":0.432,"15min":0.432},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.74199999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.8859999999999999,"5min":1.371,"15min":1.3149999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":14.609,"5min":14.609,"15min":14.609},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.137,"5min":1.1819999999999999,"15min":1.2010000000000001},"min":{"1min":0.56100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.722,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":0.87}]},{"osd":3,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.286,"15min":1.208},"min":{"1min":0.54500000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":16.567,"5min":16.567,"15min":16.567},"last":0.62},{"interface":"front","average":{"1min":1.9470000000000001,"5min":1.5089999999999999,"15min":1.4730000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":14.528,"5min":14.528,"15min":14.528},"last":1.25}]},{"osd":4,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.2789999999999999,"5min":1.716,"15min":1.6499999999999999},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":14.779,"5min":14.779,"15min":14.779},"last":1.149},{"interface":"front","average":{"1min":1.296,"5min":1.4239999999999999,"15min":1.462},"min":{"1min":0.878,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6519999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.458}]},{"osd":5,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.1459999999999999,"5min":1.5669999999999999,"15min":1.508},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":17.167000000000002,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.3160000000000001},{"interface":"front","average":{"1min":2.2320000000000002,"5min":1.728,"15min":1.6599999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":14.702999999999999,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.3819999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.47,"15min":1.486},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7689999999999999,"5min":2.246,"15min":2.246},"last":1.3420000000000001},{"interface":"front","average":{"1min":2.302,"5min":1.5880000000000001,"15min":1.4850000000000001},"min":{"1min":0.85399999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":17.07,"5min":17.07,"15min":17.07},"last":1.0189999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5760000000000001,"15min":1.611},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8129999999999999,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.5429999999999999},{"interface":"front","average":{"1min":3.0960000000000001,"5min":1.806,"15min":1.6060000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":16.922000000000001,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.208}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":552960,"data_stored":224674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":0.94799999999999995,"15min":0.879},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.103,"5min":0.998,"15min":0.98599999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":2.1190000000000002}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.93200000000000005,"15min":0.86799999999999999},"min":{"1min":0.48999999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.637,"5min":1.234,"15min":1.155},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.9079999999999999,"15min":2.1219999999999999},"min":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":6.5949999999999998,"5min":22.106999999999999,"15min":22.106999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":2.4590000000000001,"5min":1.401,"15min":1.2370000000000001},"min":{"1min":0.47499999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":15.510999999999999,"5min":15.510999999999999,"15min":15.510999999999999},"last":2.181}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.6819999999999999,"15min":1.7529999999999999},"min":{"1min":0.98399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3230000000000004,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":2.234},{"interface":"front","average":{"1min":1.423,"5min":1.744,"15min":1.9179999999999999},"min":{"1min":0.88500000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.9729999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":0.95199999999999996}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.734,"15min":1.893},"min":{"1min":0.96999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.9359999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":3.2770000000000001,"5min":1.8979999999999999,"15min":1.7},"min":{"1min":0.88100000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":20.823,"5min":20.823,"15min":20.823},"last":0.96899999999999997}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.885,"15min":2.073},"min":{"1min":1.0469999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.8980000000000001,"5min":14.272,"15min":14.272},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.5049999999999999,"15min":1.4870000000000001},"min":{"1min":0.82599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":4.3639999999999999,"5min":4.3639999999999999,"15min":4.3639999999999999},"last":2.2829999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.589,"15min":1.556},"min":{"1min":0.94999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.546,"15min":1.554},"min":{"1min":1.111,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.997}]}]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36460,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335380,"statfs":{"total":96636764160,"available":96599429120,"internally_reserved":0,"allocated":1146880,"data_stored":815006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89300000000000002,"15min":0.875},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5580000000000001,"5min":1.756,"15min":1.756},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.109,"5min":1.085,"15min":1.0760000000000001},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5509999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.86599999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0509999999999999,"15min":0.995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.03,"5min":0.89800000000000002,"15min":0.84599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":0.78200000000000003}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2669999999999999,"15min":1.2609999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.238},{"interface":"front","average":{"1min":1.179,"5min":1.1040000000000001,"15min":1.056},"min":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.325}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3049999999999999,"15min":1.27},"min":{"1min":0.85199999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.5189999999999999,"15min":1.5},"min":{"1min":1.119,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.405}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7829999999999999,"5min":1.421,"15min":1.3460000000000001},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":8.0120000000000005,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4710000000000001,"15min":1.48},"min":{"1min":0.93200000000000005,"5min":0.754,"15min":0.754},"max":{"1min":2.161,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.381}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5249999999999999,"15min":1.502},"min":{"1min":1.03,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.421},{"interface":"front","average":{"1min":1.474,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.026,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.2730000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.266}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.778,"5min":1.653,"15min":1.645},"min":{"1min":0.97999999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.0259999999999998,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.724,"5min":1.581,"15min":1.55},"min":{"1min":0.94899999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.526}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:58:24.505 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-15T17:58:24.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.763+0000 7f13f1600640 1 Processor -- start 2024-01-15T17:58:24.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.763+0000 7f13f1600640 1 -- start start 2024-01-15T17:58:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.765+0000 7f13f1600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f13ec162810 0x7f13ec071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.765+0000 7f13f1600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec071c10 con 0x7f13ec162430 2024-01-15T17:58:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.765+0000 7f13f1600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ec0702e0 con 0x7f13ec071780 2024-01-15T17:58:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.765+0000 7f13f1600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13ec0705c0 con 0x7f13ec162810 2024-01-15T17:58:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.765+0000 7f13ebe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f13ec162430 0x7f13ec0b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46698/0 (socket says 172.21.15.26:46698) 2024-01-15T17:58:24.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.765+0000 7f13ebe00640 1 -- 172.21.15.26:0/1450939824 learned_addr learned my addr 172.21.15.26:0/1450939824 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536073319 0 0) 0x7f13ec071c10 con 0x7f13ec162430 2024-01-15T17:58:24.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13cc003520 con 0x7f13ec162430 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13e3e00640 1 --2- 172.21.15.26:0/1450939824 >> v2:172.21.15.124:3300/0 conn(0x7f13ec162810 0x7f13ec071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2962572180 0 0) 0x7f13cc003520 con 0x7f13ec162430 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 >> v2:172.21.15.124:3300/0 conn(0x7f13ec162810 msgr2=0x7f13ec071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 --2- 172.21.15.26:0/1450939824 >> v2:172.21.15.124:3300/0 conn(0x7f13ec162810 0x7f13ec071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 >> v1:172.21.15.124:6789/0 conn(0x7f13ec071780 legacy=0x7f13ec06c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0708a0 con 0x7f13ec162430 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f13dc001670 con 0x7f13ec162430 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 >> v1:172.21.15.26:6789/0 conn(0x7f13ec162430 legacy=0x7f13ec0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13cc0039c0 0x7f13cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.766+0000 7f13eb400640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13cc0047c0 0x7f13cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13eb400640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13cc005180 0x7f13cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec0705c0 con 0x7f13cc005180 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc00c080 con 0x7f13cc0039c0 2024-01-15T17:58:24.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13cc00c240 con 0x7f13cc0047c0 2024-01-15T17:58:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13f0c00640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13cc005180 0x7f13cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13ebe00640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13cc0039c0 0x7f13cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13e3e00640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13cc0047c0 0x7f13cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13f0c00640 1 -- 172.21.15.26:0/1450939824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13cc0047c0 msgr2=0x7f13cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13f0c00640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13cc0047c0 0x7f13cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13f0c00640 1 -- 172.21.15.26:0/1450939824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13cc0039c0 msgr2=0x7f13cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13f0c00640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13cc0039c0 0x7f13cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.767+0000 7f13f0c00640 1 -- 172.21.15.26:0/1450939824 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc003520 con 0x7f13cc005180 2024-01-15T17:58:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.768+0000 7f13f0c00640 1 -- 172.21.15.26:0/1450939824 reap_dead start 2024-01-15T17:58:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.768+0000 7f13f0c00640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13cc005180 0x7f13cc00b980 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13e4001e50 tx=0x7f13cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e95099f3b02dea2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4003b70 con 0x7f13cc005180 2024-01-15T17:58:24.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e4003d10 con 0x7f13cc005180 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13eb400640 1 -- 172.21.15.26:0/1450939824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4005310 con 0x7f13cc005180 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13f1600640 1 -- 172.21.15.26:0/1450939824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13cc005180 msgr2=0x7f13cc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13f1600640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13cc005180 0x7f13cc00b980 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13e4001e50 tx=0x7f13cc0057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13f1600640 1 -- 172.21.15.26:0/1450939824 shutdown_connections 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13f1600640 1 --2- 172.21.15.26:0/1450939824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13cc005180 0x7f13cc00b980 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13f1600640 1 -- 172.21.15.26:0/1450939824 >> 172.21.15.26:0/1450939824 conn(0x7f13ec06a800 msgr2=0x7f13ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13f1600640 1 -- 172.21.15.26:0/1450939824 shutdown_connections 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.769+0000 7f13f1600640 1 -- 172.21.15.26:0/1450939824 wait complete. 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.771+0000 7f13f1600640 1 Processor -- start 2024-01-15T17:58:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.771+0000 7f13f1600640 1 -- start start 2024-01-15T17:58:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.772+0000 7f13f1600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13cc005180 0x7f13ec14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.772+0000 7f13f1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13ec152640 0x7f13ec1f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.772+0000 7f13f1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13ec153100 0x7f13ec1535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.772+0000 7f13f1600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec152220 con 0x7f13ec152640 2024-01-15T17:58:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.772+0000 7f13f1600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec153af0 con 0x7f13cc005180 2024-01-15T17:58:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.772+0000 7f13f1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ec14f5f0 con 0x7f13ec153100 2024-01-15T17:58:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.772+0000 7f13e3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13ec152640 0x7f13ec1f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.772+0000 7f13e3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13ec152640 0x7f13ec1f7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57576/0 (socket says 172.21.15.26:57576) 2024-01-15T17:58:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.772+0000 7f13e3e00640 1 -- 172.21.15.26:0/952450844 learned_addr learned my addr 172.21.15.26:0/952450844 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13e3e00640 1 -- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13ec153100 msgr2=0x7f13ec1535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13e3e00640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13ec153100 0x7f13ec1535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13e3e00640 1 -- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13cc005180 msgr2=0x7f13ec14f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13e3e00640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13cc005180 0x7f13ec14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13e3e00640 1 -- 172.21.15.26:0/952450844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec1515a0 con 0x7f13ec152640 2024-01-15T17:58:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13e3e00640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13ec152640 0x7f13ec1f7ae0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f13d4011560 tx=0x7f13d4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13e8c00640 1 -- 172.21.15.26:0/952450844 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d4015930 con 0x7f13ec152640 2024-01-15T17:58:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13e8c00640 1 -- 172.21.15.26:0/952450844 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d4015a90 con 0x7f13ec152640 2024-01-15T17:58:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13e8c00640 1 -- 172.21.15.26:0/952450844 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d4015e40 con 0x7f13ec152640 2024-01-15T17:58:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.773+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ec151880 con 0x7f13ec152640 2024-01-15T17:58:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.774+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ec151e40 con 0x7f13ec152640 2024-01-15T17:58:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.775+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ec152490 con 0x7f13ec152640 2024-01-15T17:58:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.776+0000 7f13e8c00640 1 -- 172.21.15.26:0/952450844 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f13d401d400 con 0x7f13ec152640 2024-01-15T17:58:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.777+0000 7f13e8c00640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13c406b580 0x7f13c406da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.777+0000 7f13e8c00640 1 -- 172.21.15.26:0/952450844 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f13d408a750 con 0x7f13ec152640 2024-01-15T17:58:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.777+0000 7f13ebe00640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13c406b580 0x7f13c406da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:24.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.778+0000 7f13ebe00640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13c406b580 0x7f13c406da40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f13dc001050 tx=0x7f13dc000f60 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:24.781+0000 7f13e8c00640 1 -- 172.21.15.26:0/952450844 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13d40552b0 con 0x7f13ec152640 2024-01-15T17:58:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.050+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f13ec067450 con 0x7f13ec152640 2024-01-15T17:58:25.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.051+0000 7f13e8c00640 1 -- 172.21.15.26:0/952450844 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1012 (secure 0 0 0) 0x7f13d40588d0 con 0x7f13ec152640 2024-01-15T17:58:25.054 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:25.054 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":219,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":47,"num_osds":8,"num_up_osds":8,"osd_up_since":1705341296,"num_in_osds":8,"osd_in_since":1705341288,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":278106112,"bytes_avail":772816007168,"bytes_total":773094113280,"write_bytes_sec":2380,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5802}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-01-15T17:57:31.962932+0000","services":{}},"progress_events":{}} 2024-01-15T17:58:25.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13c406b580 msgr2=0x7f13c406da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:25.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13c406b580 0x7f13c406da40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f13dc001050 tx=0x7f13dc000f60 comp rx=0 tx=0).stop 2024-01-15T17:58:25.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13ec152640 msgr2=0x7f13ec1f7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:25.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13ec152640 0x7f13ec1f7ae0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f13d4011560 tx=0x7f13d4009f90 comp rx=0 tx=0).stop 2024-01-15T17:58:25.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 shutdown_connections 2024-01-15T17:58:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13ec153100 0x7f13ec1535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13c406b580 0x7f13c406da40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13ec152640 0x7f13ec1f7ae0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 --2- 172.21.15.26:0/952450844 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f13cc005180 0x7f13ec14f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 >> 172.21.15.26:0/952450844 conn(0x7f13ec06a800 msgr2=0x7f13ec0af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 shutdown_connections 2024-01-15T17:58:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.054+0000 7f13f1600640 1 -- 172.21.15.26:0/952450844 wait complete. 2024-01-15T17:58:25.075 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-15T17:58:25.075 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:58:25.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.290+0000 7f29f0e00640 1 Processor -- start 2024-01-15T17:58:25.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.291+0000 7f29f0e00640 1 -- start start 2024-01-15T17:58:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.292+0000 7f29f0e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f29ec071410 0x7f29ec146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.292+0000 7f29f0e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29ec06c7c0 0x7f29ec0717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.292+0000 7f29f0e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071d30 0x7f29ec0b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.292+0000 7f29f0e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec06cc50 con 0x7f29ec071410 2024-01-15T17:58:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.292+0000 7f29f0e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec06cec0 con 0x7f29ec06c7c0 2024-01-15T17:58:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.292+0000 7f29f0e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec06d1a0 con 0x7f29ec071d30 2024-01-15T17:58:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29eb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f29ec071410 0x7f29ec146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29eb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f29ec071410 0x7f29ec146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57582/0 (socket says 172.21.15.26:57582) 2024-01-15T17:58:25.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29eb400640 1 -- 172.21.15.26:0/3263758622 learned_addr learned my addr 172.21.15.26:0/3263758622 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29e2800640 1 --2- 172.21.15.26:0/3263758622 >> v2:172.21.15.124:3300/0 conn(0x7f29ec06c7c0 0x7f29ec0717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29ebe00640 1 --2- 172.21.15.26:0/3263758622 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071d30 0x7f29ec0b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29eb400640 1 -- 172.21.15.26:0/3263758622 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071d30 msgr2=0x7f29ec0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29eb400640 1 --2- 172.21.15.26:0/3263758622 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071d30 0x7f29ec0b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29eb400640 1 -- 172.21.15.26:0/3263758622 >> v2:172.21.15.124:3300/0 conn(0x7f29ec06c7c0 msgr2=0x7f29ec0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29eb400640 1 --2- 172.21.15.26:0/3263758622 >> v2:172.21.15.124:3300/0 conn(0x7f29ec06c7c0 0x7f29ec0717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29eb400640 1 -- 172.21.15.26:0/3263758622 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec06d480 con 0x7f29ec071410 2024-01-15T17:58:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29ebe00640 1 --2- 172.21.15.26:0/3263758622 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071d30 0x7f29ec0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:25.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.293+0000 7f29e2800640 1 --2- 172.21.15.26:0/3263758622 >> v2:172.21.15.124:3300/0 conn(0x7f29ec06c7c0 0x7f29ec0717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.294+0000 7f29eb400640 1 --2- 172.21.15.26:0/3263758622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071410 0x7f29ec146770 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f29ec146ef0 tx=0x7f29dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b350e123389903e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:25.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29eaa00640 1 -- 172.21.15.26:0/3263758622 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc02ca80 con 0x7f29ec071410 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29eaa00640 1 -- 172.21.15.26:0/3263758622 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29dc02cbe0 con 0x7f29ec071410 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29eaa00640 1 -- 172.21.15.26:0/3263758622 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0343f0 con 0x7f29ec071410 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29f0e00640 1 -- 172.21.15.26:0/3263758622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071410 msgr2=0x7f29ec146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29f0e00640 1 --2- 172.21.15.26:0/3263758622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071410 0x7f29ec146770 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f29ec146ef0 tx=0x7f29dc02bee0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29f0e00640 1 -- 172.21.15.26:0/3263758622 shutdown_connections 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29f0e00640 1 --2- 172.21.15.26:0/3263758622 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071d30 0x7f29ec0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29f0e00640 1 --2- 172.21.15.26:0/3263758622 >> v2:172.21.15.124:3300/0 conn(0x7f29ec06c7c0 0x7f29ec0717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29f0e00640 1 --2- 172.21.15.26:0/3263758622 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071410 0x7f29ec146770 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29f0e00640 1 -- 172.21.15.26:0/3263758622 >> 172.21.15.26:0/3263758622 conn(0x7f29ec06a800 msgr2=0x7f29ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29f0e00640 1 -- 172.21.15.26:0/3263758622 shutdown_connections 2024-01-15T17:58:25.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.295+0000 7f29f0e00640 1 -- 172.21.15.26:0/3263758622 wait complete. 2024-01-15T17:58:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.297+0000 7f29f0e00640 1 Processor -- start 2024-01-15T17:58:25.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.297+0000 7f29f0e00640 1 -- start start 2024-01-15T17:58:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.298+0000 7f29f0e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29ec06c7c0 0x7f29ec15aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.298+0000 7f29f0e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071410 0x7f29ec15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.298+0000 7f29f0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071d30 0x7f29ec13d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.298+0000 7f29f0e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec15d4a0 con 0x7f29ec071d30 2024-01-15T17:58:25.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.298+0000 7f29f0e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec13de90 con 0x7f29ec06c7c0 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.298+0000 7f29f0e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec15d680 con 0x7f29ec071410 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071d30 0x7f29ec13d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071d30 0x7f29ec13d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57588/0 (socket says 172.21.15.26:57588) 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29ebe00640 1 -- 172.21.15.26:0/114766916 learned_addr learned my addr 172.21.15.26:0/114766916 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29e2800640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071410 0x7f29ec15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29eb400640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29ec06c7c0 0x7f29ec15aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29ebe00640 1 -- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071410 msgr2=0x7f29ec15a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29ebe00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071410 0x7f29ec15a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29ebe00640 1 -- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29ec06c7c0 msgr2=0x7f29ec15aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29ebe00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29ec06c7c0 0x7f29ec15aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.299+0000 7f29ebe00640 1 -- 172.21.15.26:0/114766916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc009b30 con 0x7f29ec071d30 2024-01-15T17:58:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.300+0000 7f29ebe00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071d30 0x7f29ec13d440 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f29e400c7c0 tx=0x7f29e400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.300+0000 7f29e3e00640 1 -- 172.21.15.26:0/114766916 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4004230 con 0x7f29ec071d30 2024-01-15T17:58:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.300+0000 7f29e3e00640 1 -- 172.21.15.26:0/114766916 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e4004390 con 0x7f29ec071d30 2024-01-15T17:58:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.300+0000 7f29e3e00640 1 -- 172.21.15.26:0/114766916 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4007860 con 0x7f29ec071d30 2024-01-15T17:58:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.300+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec15d960 con 0x7f29ec071d30 2024-01-15T17:58:25.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.300+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec15a7d0 con 0x7f29ec071d30 2024-01-15T17:58:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.302+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ec067db0 con 0x7f29ec071d30 2024-01-15T17:58:25.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.303+0000 7f29e3e00640 1 -- 172.21.15.26:0/114766916 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f29e40079c0 con 0x7f29ec071d30 2024-01-15T17:58:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.303+0000 7f29e3e00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f29c406b500 0x7f29c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:25.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.304+0000 7f29e3e00640 1 -- 172.21.15.26:0/114766916 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f29e408a300 con 0x7f29ec071d30 2024-01-15T17:58:25.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.304+0000 7f29eb400640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f29c406b500 0x7f29c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:25.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.305+0000 7f29eb400640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f29c406b500 0x7f29c406d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f29dc035000 tx=0x7f29dc030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:25.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.308+0000 7f29e3e00640 1 -- 172.21.15.26:0/114766916 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29e4054d90 con 0x7f29ec071d30 2024-01-15T17:58:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.490+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29ec0b3a90 con 0x7f29c406b500 2024-01-15T17:58:25.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.514+0000 7f29e3e00640 1 -- 172.21.15.26:0/114766916 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286474 (secure 0 0 0) 0x7f29ec0b3a90 con 0x7f29c406b500 2024-01-15T17:58:25.520 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f29c406b500 msgr2=0x7f29c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f29c406b500 0x7f29c406d9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f29dc035000 tx=0x7f29dc030000 comp rx=0 tx=0).stop 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071d30 msgr2=0x7f29ec13d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071d30 0x7f29ec13d440 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f29e400c7c0 tx=0x7f29e400cc80 comp rx=0 tx=0).stop 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 shutdown_connections 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29ec071410 0x7f29ec15a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f29c406b500 0x7f29c406d9c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29ec071d30 0x7f29ec13d440 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 --2- 172.21.15.26:0/114766916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29ec06c7c0 0x7f29ec15aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 >> 172.21.15.26:0/114766916 conn(0x7f29ec06a800 msgr2=0x7f29ec0b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 shutdown_connections 2024-01-15T17:58:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:25.520+0000 7f29f0e00640 1 -- 172.21.15.26:0/114766916 wait complete. 2024-01-15T17:58:25.543 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-01-15T17:58:23.979687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271588,"kb_used_data":5500,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754703132,"statfs":{"total":773094113280,"available":772816007168,"internally_reserved":0,"allocated":5632000,"data_stored":2976060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.022365"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159357+0000","last_change":"2024-01-15T17:58:18.991217+0000","last_active":"2024-01-15T17:58:21.159357+0000","last_peered":"2024-01-15T17:58:21.159357+0000","last_clean":"2024-01-15T17:58:21.159357+0000","last_became_active":"2024-01-15T17:58:18.990850+0000","last_became_peered":"2024-01-15T17:58:18.990850+0000","last_unstale":"2024-01-15T17:58:21.159357+0000","last_undegraded":"2024-01-15T17:58:21.159357+0000","last_fullsized":"2024-01-15T17:58:21.159357+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:48.160968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987226+0000","last_change":"2024-01-15T17:58:19.015695+0000","last_active":"2024-01-15T17:58:19.987226+0000","last_peered":"2024-01-15T17:58:19.987226+0000","last_clean":"2024-01-15T17:58:19.987226+0000","last_became_active":"2024-01-15T17:58:19.015324+0000","last_became_peered":"2024-01-15T17:58:19.015324+0000","last_unstale":"2024-01-15T17:58:19.987226+0000","last_undegraded":"2024-01-15T17:58:19.987226+0000","last_fullsized":"2024-01-15T17:58:19.987226+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:09.274759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345611+0000","last_change":"2024-01-15T17:58:18.984806+0000","last_active":"2024-01-15T17:58:20.345611+0000","last_peered":"2024-01-15T17:58:20.345611+0000","last_clean":"2024-01-15T17:58:20.345611+0000","last_became_active":"2024-01-15T17:58:18.984445+0000","last_became_peered":"2024-01-15T17:58:18.984445+0000","last_unstale":"2024-01-15T17:58:20.345611+0000","last_undegraded":"2024-01-15T17:58:20.345611+0000","last_fullsized":"2024-01-15T17:58:20.345611+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:07.065888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163376+0000","last_change":"2024-01-15T17:58:19.015296+0000","last_active":"2024-01-15T17:58:21.163376+0000","last_peered":"2024-01-15T17:58:21.163376+0000","last_clean":"2024-01-15T17:58:21.163376+0000","last_became_active":"2024-01-15T17:58:19.011703+0000","last_became_peered":"2024-01-15T17:58:19.011703+0000","last_unstale":"2024-01-15T17:58:21.163376+0000","last_undegraded":"2024-01-15T17:58:21.163376+0000","last_fullsized":"2024-01-15T17:58:21.163376+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:43:54.733897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986294+0000","last_change":"2024-01-15T17:58:19.009586+0000","last_active":"2024-01-15T17:58:19.986294+0000","last_peered":"2024-01-15T17:58:19.986294+0000","last_clean":"2024-01-15T17:58:19.986294+0000","last_became_active":"2024-01-15T17:58:19.009120+0000","last_became_peered":"2024-01-15T17:58:19.009120+0000","last_unstale":"2024-01-15T17:58:19.986294+0000","last_undegraded":"2024-01-15T17:58:19.986294+0000","last_fullsized":"2024-01-15T17:58:19.986294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:51.429644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160792+0000","last_change":"2024-01-15T17:58:18.994318+0000","last_active":"2024-01-15T17:58:21.160792+0000","last_peered":"2024-01-15T17:58:21.160792+0000","last_clean":"2024-01-15T17:58:21.160792+0000","last_became_active":"2024-01-15T17:58:18.994008+0000","last_became_peered":"2024-01-15T17:58:18.994008+0000","last_unstale":"2024-01-15T17:58:21.160792+0000","last_undegraded":"2024-01-15T17:58:21.160792+0000","last_fullsized":"2024-01-15T17:58:21.160792+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:51.745964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158955+0000","last_change":"2024-01-15T17:58:18.992678+0000","last_active":"2024-01-15T17:58:21.158955+0000","last_peered":"2024-01-15T17:58:21.158955+0000","last_clean":"2024-01-15T17:58:21.158955+0000","last_became_active":"2024-01-15T17:58:18.992276+0000","last_became_peered":"2024-01-15T17:58:18.992276+0000","last_unstale":"2024-01-15T17:58:21.158955+0000","last_undegraded":"2024-01-15T17:58:21.158955+0000","last_fullsized":"2024-01-15T17:58:21.158955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:18.389244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160076+0000","last_change":"2024-01-15T17:58:19.005373+0000","last_active":"2024-01-15T17:58:21.160076+0000","last_peered":"2024-01-15T17:58:21.160076+0000","last_clean":"2024-01-15T17:58:21.160076+0000","last_became_active":"2024-01-15T17:58:19.003358+0000","last_became_peered":"2024-01-15T17:58:19.003358+0000","last_unstale":"2024-01-15T17:58:21.160076+0000","last_undegraded":"2024-01-15T17:58:21.160076+0000","last_fullsized":"2024-01-15T17:58:21.160076+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:07.065888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164063+0000","last_change":"2024-01-15T17:58:19.003597+0000","last_active":"2024-01-15T17:58:21.164063+0000","last_peered":"2024-01-15T17:58:21.164063+0000","last_clean":"2024-01-15T17:58:21.164063+0000","last_became_active":"2024-01-15T17:58:19.003270+0000","last_became_peered":"2024-01-15T17:58:19.003270+0000","last_unstale":"2024-01-15T17:58:21.164063+0000","last_undegraded":"2024-01-15T17:58:21.164063+0000","last_fullsized":"2024-01-15T17:58:21.164063+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:59.015634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164080+0000","last_change":"2024-01-15T17:58:19.002263+0000","last_active":"2024-01-15T17:58:21.164080+0000","last_peered":"2024-01-15T17:58:21.164080+0000","last_clean":"2024-01-15T17:58:21.164080+0000","last_became_active":"2024-01-15T17:58:19.000576+0000","last_became_peered":"2024-01-15T17:58:19.000576+0000","last_unstale":"2024-01-15T17:58:21.164080+0000","last_undegraded":"2024-01-15T17:58:21.164080+0000","last_fullsized":"2024-01-15T17:58:21.164080+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:26.728833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163770+0000","last_change":"2024-01-15T17:58:19.014552+0000","last_active":"2024-01-15T17:58:21.163770+0000","last_peered":"2024-01-15T17:58:21.163770+0000","last_clean":"2024-01-15T17:58:21.163770+0000","last_became_active":"2024-01-15T17:58:19.008921+0000","last_became_peered":"2024-01-15T17:58:19.008921+0000","last_unstale":"2024-01-15T17:58:21.163770+0000","last_undegraded":"2024-01-15T17:58:21.163770+0000","last_fullsized":"2024-01-15T17:58:21.163770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:51.429644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986412+0000","last_change":"2024-01-15T17:58:18.988600+0000","last_active":"2024-01-15T17:58:19.986412+0000","last_peered":"2024-01-15T17:58:19.986412+0000","last_clean":"2024-01-15T17:58:19.986412+0000","last_became_active":"2024-01-15T17:58:18.987658+0000","last_became_peered":"2024-01-15T17:58:18.987658+0000","last_unstale":"2024-01-15T17:58:19.986412+0000","last_undegraded":"2024-01-15T17:58:19.986412+0000","last_fullsized":"2024-01-15T17:58:19.986412+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:35.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158316+0000","last_change":"2024-01-15T17:58:18.998942+0000","last_active":"2024-01-15T17:58:21.158316+0000","last_peered":"2024-01-15T17:58:21.158316+0000","last_clean":"2024-01-15T17:58:21.158316+0000","last_became_active":"2024-01-15T17:58:18.998650+0000","last_became_peered":"2024-01-15T17:58:18.998650+0000","last_unstale":"2024-01-15T17:58:21.158316+0000","last_undegraded":"2024-01-15T17:58:21.158316+0000","last_fullsized":"2024-01-15T17:58:21.158316+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:39.127046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162172+0000","last_change":"2024-01-15T17:58:19.001928+0000","last_active":"2024-01-15T17:58:21.162172+0000","last_peered":"2024-01-15T17:58:21.162172+0000","last_clean":"2024-01-15T17:58:21.162172+0000","last_became_active":"2024-01-15T17:58:19.000617+0000","last_became_peered":"2024-01-15T17:58:19.000617+0000","last_unstale":"2024-01-15T17:58:21.162172+0000","last_undegraded":"2024-01-15T17:58:21.162172+0000","last_fullsized":"2024-01-15T17:58:21.162172+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:01.355486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159929+0000","last_change":"2024-01-15T17:58:18.987718+0000","last_active":"2024-01-15T17:58:21.159929+0000","last_peered":"2024-01-15T17:58:21.159929+0000","last_clean":"2024-01-15T17:58:21.159929+0000","last_became_active":"2024-01-15T17:58:18.987390+0000","last_became_peered":"2024-01-15T17:58:18.987390+0000","last_unstale":"2024-01-15T17:58:21.159929+0000","last_undegraded":"2024-01-15T17:58:21.159929+0000","last_fullsized":"2024-01-15T17:58:21.159929+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:05.757201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985536+0000","last_change":"2024-01-15T17:58:19.004638+0000","last_active":"2024-01-15T17:58:19.985536+0000","last_peered":"2024-01-15T17:58:19.985536+0000","last_clean":"2024-01-15T17:58:19.985536+0000","last_became_active":"2024-01-15T17:58:19.004325+0000","last_became_peered":"2024-01-15T17:58:19.004325+0000","last_unstale":"2024-01-15T17:58:19.985536+0000","last_undegraded":"2024-01-15T17:58:19.985536+0000","last_fullsized":"2024-01-15T17:58:19.985536+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:47.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346067+0000","last_change":"2024-01-15T17:58:18.987202+0000","last_active":"2024-01-15T17:58:20.346067+0000","last_peered":"2024-01-15T17:58:20.346067+0000","last_clean":"2024-01-15T17:58:20.346067+0000","last_became_active":"2024-01-15T17:58:18.986911+0000","last_became_peered":"2024-01-15T17:58:18.986911+0000","last_unstale":"2024-01-15T17:58:20.346067+0000","last_undegraded":"2024-01-15T17:58:20.346067+0000","last_fullsized":"2024-01-15T17:58:20.346067+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:01.355486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346145+0000","last_change":"2024-01-15T17:58:19.016790+0000","last_active":"2024-01-15T17:58:20.346145+0000","last_peered":"2024-01-15T17:58:20.346145+0000","last_clean":"2024-01-15T17:58:20.346145+0000","last_became_active":"2024-01-15T17:58:19.016452+0000","last_became_peered":"2024-01-15T17:58:19.016452+0000","last_unstale":"2024-01-15T17:58:20.346145+0000","last_undegraded":"2024-01-15T17:58:20.346145+0000","last_fullsized":"2024-01-15T17:58:20.346145+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:01:54.772028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987535+0000","last_change":"2024-01-15T17:58:19.012051+0000","last_active":"2024-01-15T17:58:19.987535+0000","last_peered":"2024-01-15T17:58:19.987535+0000","last_clean":"2024-01-15T17:58:19.987535+0000","last_became_active":"2024-01-15T17:58:19.011643+0000","last_became_peered":"2024-01-15T17:58:19.011643+0000","last_unstale":"2024-01-15T17:58:19.987535+0000","last_undegraded":"2024-01-15T17:58:19.987535+0000","last_fullsized":"2024-01-15T17:58:19.987535+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:28:46.759912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158121+0000","last_change":"2024-01-15T17:58:19.011350+0000","last_active":"2024-01-15T17:58:21.158121+0000","last_peered":"2024-01-15T17:58:21.158121+0000","last_clean":"2024-01-15T17:58:21.158121+0000","last_became_active":"2024-01-15T17:58:19.011046+0000","last_became_peered":"2024-01-15T17:58:19.011046+0000","last_unstale":"2024-01-15T17:58:21.158121+0000","last_undegraded":"2024-01-15T17:58:21.158121+0000","last_fullsized":"2024-01-15T17:58:21.158121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:12:37.308652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161075+0000","last_change":"2024-01-15T17:58:18.996921+0000","last_active":"2024-01-15T17:58:21.161075+0000","last_peered":"2024-01-15T17:58:21.161075+0000","last_clean":"2024-01-15T17:58:21.161075+0000","last_became_active":"2024-01-15T17:58:18.994377+0000","last_became_peered":"2024-01-15T17:58:18.994377+0000","last_unstale":"2024-01-15T17:58:21.161075+0000","last_undegraded":"2024-01-15T17:58:21.161075+0000","last_fullsized":"2024-01-15T17:58:21.161075+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:54:37.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162353+0000","last_change":"2024-01-15T17:58:19.006417+0000","last_active":"2024-01-15T17:58:21.162353+0000","last_peered":"2024-01-15T17:58:21.162353+0000","last_clean":"2024-01-15T17:58:21.162353+0000","last_became_active":"2024-01-15T17:58:19.002993+0000","last_became_peered":"2024-01-15T17:58:19.002993+0000","last_unstale":"2024-01-15T17:58:21.162353+0000","last_undegraded":"2024-01-15T17:58:21.162353+0000","last_fullsized":"2024-01-15T17:58:21.162353+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:01:54.772028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986705+0000","last_change":"2024-01-15T17:58:18.995694+0000","last_active":"2024-01-15T17:58:19.986705+0000","last_peered":"2024-01-15T17:58:19.986705+0000","last_clean":"2024-01-15T17:58:19.986705+0000","last_became_active":"2024-01-15T17:58:18.993968+0000","last_became_peered":"2024-01-15T17:58:18.993968+0000","last_unstale":"2024-01-15T17:58:19.986705+0000","last_undegraded":"2024-01-15T17:58:19.986705+0000","last_fullsized":"2024-01-15T17:58:19.986705+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:25:46.220536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985649+0000","last_change":"2024-01-15T17:58:18.995548+0000","last_active":"2024-01-15T17:58:19.985649+0000","last_peered":"2024-01-15T17:58:19.985649+0000","last_clean":"2024-01-15T17:58:19.985649+0000","last_became_active":"2024-01-15T17:58:18.995281+0000","last_became_peered":"2024-01-15T17:58:18.995281+0000","last_unstale":"2024-01-15T17:58:19.985649+0000","last_undegraded":"2024-01-15T17:58:19.985649+0000","last_fullsized":"2024-01-15T17:58:19.985649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:28:02.329866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986241+0000","last_change":"2024-01-15T17:58:18.992172+0000","last_active":"2024-01-15T17:58:19.986241+0000","last_peered":"2024-01-15T17:58:19.986241+0000","last_clean":"2024-01-15T17:58:19.986241+0000","last_became_active":"2024-01-15T17:58:18.991535+0000","last_became_peered":"2024-01-15T17:58:18.991535+0000","last_unstale":"2024-01-15T17:58:19.986241+0000","last_undegraded":"2024-01-15T17:58:19.986241+0000","last_fullsized":"2024-01-15T17:58:19.986241+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:07.285598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158273+0000","last_change":"2024-01-15T17:58:18.996294+0000","last_active":"2024-01-15T17:58:21.158273+0000","last_peered":"2024-01-15T17:58:21.158273+0000","last_clean":"2024-01-15T17:58:21.158273+0000","last_became_active":"2024-01-15T17:58:18.996025+0000","last_became_peered":"2024-01-15T17:58:18.996025+0000","last_unstale":"2024-01-15T17:58:21.158273+0000","last_undegraded":"2024-01-15T17:58:21.158273+0000","last_fullsized":"2024-01-15T17:58:21.158273+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:23:51.721700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346062+0000","last_change":"2024-01-15T17:58:18.984770+0000","last_active":"2024-01-15T17:58:20.346062+0000","last_peered":"2024-01-15T17:58:20.346062+0000","last_clean":"2024-01-15T17:58:20.346062+0000","last_became_active":"2024-01-15T17:58:18.984406+0000","last_became_peered":"2024-01-15T17:58:18.984406+0000","last_unstale":"2024-01-15T17:58:20.346062+0000","last_undegraded":"2024-01-15T17:58:20.346062+0000","last_fullsized":"2024-01-15T17:58:20.346062+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:10.953697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986646+0000","last_change":"2024-01-15T17:58:18.988385+0000","last_active":"2024-01-15T17:58:19.986646+0000","last_peered":"2024-01-15T17:58:19.986646+0000","last_clean":"2024-01-15T17:58:19.986646+0000","last_became_active":"2024-01-15T17:58:18.987296+0000","last_became_peered":"2024-01-15T17:58:18.987296+0000","last_unstale":"2024-01-15T17:58:19.986646+0000","last_undegraded":"2024-01-15T17:58:19.986646+0000","last_fullsized":"2024-01-15T17:58:19.986646+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:26.728833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986444+0000","last_change":"2024-01-15T17:58:18.987849+0000","last_active":"2024-01-15T17:58:19.986444+0000","last_peered":"2024-01-15T17:58:19.986444+0000","last_clean":"2024-01-15T17:58:19.986444+0000","last_became_active":"2024-01-15T17:58:18.987504+0000","last_became_peered":"2024-01-15T17:58:18.987504+0000","last_unstale":"2024-01-15T17:58:19.986444+0000","last_undegraded":"2024-01-15T17:58:19.986444+0000","last_fullsized":"2024-01-15T17:58:19.986444+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:43.781847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345606+0000","last_change":"2024-01-15T17:58:18.994812+0000","last_active":"2024-01-15T17:58:20.345606+0000","last_peered":"2024-01-15T17:58:20.345606+0000","last_clean":"2024-01-15T17:58:20.345606+0000","last_became_active":"2024-01-15T17:58:18.994420+0000","last_became_peered":"2024-01-15T17:58:18.994420+0000","last_unstale":"2024-01-15T17:58:20.345606+0000","last_undegraded":"2024-01-15T17:58:20.345606+0000","last_fullsized":"2024-01-15T17:58:20.345606+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:33.875326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160190+0000","last_change":"2024-01-15T17:58:18.987732+0000","last_active":"2024-01-15T17:58:21.160190+0000","last_peered":"2024-01-15T17:58:21.160190+0000","last_clean":"2024-01-15T17:58:21.160190+0000","last_became_active":"2024-01-15T17:58:18.986144+0000","last_became_peered":"2024-01-15T17:58:18.986144+0000","last_unstale":"2024-01-15T17:58:21.160190+0000","last_undegraded":"2024-01-15T17:58:21.160190+0000","last_fullsized":"2024-01-15T17:58:21.160190+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:52.164647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.157748+0000","last_change":"2024-01-15T17:58:18.983829+0000","last_active":"2024-01-15T17:58:21.157748+0000","last_peered":"2024-01-15T17:58:21.157748+0000","last_clean":"2024-01-15T17:58:21.157748+0000","last_became_active":"2024-01-15T17:58:18.983528+0000","last_became_peered":"2024-01-15T17:58:18.983528+0000","last_unstale":"2024-01-15T17:58:21.157748+0000","last_undegraded":"2024-01-15T17:58:21.157748+0000","last_fullsized":"2024-01-15T17:58:21.157748+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:53.234232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.979342+0000","last_change":"2024-01-15T17:58:18.985545+0000","last_active":"2024-01-15T17:58:19.979342+0000","last_peered":"2024-01-15T17:58:19.979342+0000","last_clean":"2024-01-15T17:58:19.979342+0000","last_became_active":"2024-01-15T17:58:18.981858+0000","last_became_peered":"2024-01-15T17:58:18.981858+0000","last_unstale":"2024-01-15T17:58:19.979342+0000","last_undegraded":"2024-01-15T17:58:19.979342+0000","last_fullsized":"2024-01-15T17:58:19.979342+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:43.781847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.018482+0000","last_change":"2024-01-15T17:58:17.980564+0000","last_active":"2024-01-15T17:58:19.018482+0000","last_peered":"2024-01-15T17:58:19.018482+0000","last_clean":"2024-01-15T17:58:19.018482+0000","last_became_active":"2024-01-15T17:58:17.979889+0000","last_became_peered":"2024-01-15T17:58:17.979889+0000","last_unstale":"2024-01-15T17:58:19.018482+0000","last_undegraded":"2024-01-15T17:58:19.018482+0000","last_fullsized":"2024-01-15T17:58:19.018482+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:14:34.561030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986677+0000","last_change":"2024-01-15T17:58:18.987759+0000","last_active":"2024-01-15T17:58:19.986677+0000","last_peered":"2024-01-15T17:58:19.986677+0000","last_clean":"2024-01-15T17:58:19.986677+0000","last_became_active":"2024-01-15T17:58:18.987413+0000","last_became_peered":"2024-01-15T17:58:18.987413+0000","last_unstale":"2024-01-15T17:58:19.986677+0000","last_undegraded":"2024-01-15T17:58:19.986677+0000","last_fullsized":"2024-01-15T17:58:19.986677+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:22.217018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019379+0000","last_change":"2024-01-15T17:58:17.981282+0000","last_active":"2024-01-15T17:58:19.019379+0000","last_peered":"2024-01-15T17:58:19.019379+0000","last_clean":"2024-01-15T17:58:19.019379+0000","last_became_active":"2024-01-15T17:58:17.980794+0000","last_became_peered":"2024-01-15T17:58:17.980794+0000","last_unstale":"2024-01-15T17:58:19.019379+0000","last_undegraded":"2024-01-15T17:58:19.019379+0000","last_fullsized":"2024-01-15T17:58:19.019379+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:15.887231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986915+0000","last_change":"2024-01-15T17:58:18.988826+0000","last_active":"2024-01-15T17:58:19.986915+0000","last_peered":"2024-01-15T17:58:19.986915+0000","last_clean":"2024-01-15T17:58:19.986915+0000","last_became_active":"2024-01-15T17:58:18.988087+0000","last_became_peered":"2024-01-15T17:58:18.988087+0000","last_unstale":"2024-01-15T17:58:19.986915+0000","last_undegraded":"2024-01-15T17:58:19.986915+0000","last_fullsized":"2024-01-15T17:58:19.986915+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:18.389244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.023715+0000","last_change":"2024-01-15T17:58:17.990072+0000","last_active":"2024-01-15T17:58:19.023715+0000","last_peered":"2024-01-15T17:58:19.023715+0000","last_clean":"2024-01-15T17:58:19.023715+0000","last_became_active":"2024-01-15T17:58:17.989788+0000","last_became_peered":"2024-01-15T17:58:17.989788+0000","last_unstale":"2024-01-15T17:58:19.023715+0000","last_undegraded":"2024-01-15T17:58:19.023715+0000","last_fullsized":"2024-01-15T17:58:19.023715+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:09:54.096692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.979523+0000","last_change":"2024-01-15T17:58:18.997500+0000","last_active":"2024-01-15T17:58:19.979523+0000","last_peered":"2024-01-15T17:58:19.979523+0000","last_clean":"2024-01-15T17:58:19.979523+0000","last_became_active":"2024-01-15T17:58:18.997234+0000","last_became_peered":"2024-01-15T17:58:18.997234+0000","last_unstale":"2024-01-15T17:58:19.979523+0000","last_undegraded":"2024-01-15T17:58:19.979523+0000","last_fullsized":"2024-01-15T17:58:19.979523+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:22.217018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021174+0000","last_change":"2024-01-15T17:58:17.989934+0000","last_active":"2024-01-15T17:58:19.021174+0000","last_peered":"2024-01-15T17:58:19.021174+0000","last_clean":"2024-01-15T17:58:19.021174+0000","last_became_active":"2024-01-15T17:58:17.989582+0000","last_became_peered":"2024-01-15T17:58:17.989582+0000","last_unstale":"2024-01-15T17:58:19.021174+0000","last_undegraded":"2024-01-15T17:58:19.021174+0000","last_fullsized":"2024-01-15T17:58:19.021174+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:04.365732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345619+0000","last_change":"2024-01-15T17:58:18.986303+0000","last_active":"2024-01-15T17:58:20.345619+0000","last_peered":"2024-01-15T17:58:20.345619+0000","last_clean":"2024-01-15T17:58:20.345619+0000","last_became_active":"2024-01-15T17:58:18.985470+0000","last_became_peered":"2024-01-15T17:58:18.985470+0000","last_unstale":"2024-01-15T17:58:20.345619+0000","last_undegraded":"2024-01-15T17:58:20.345619+0000","last_fullsized":"2024-01-15T17:58:20.345619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:05.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.018703+0000","last_change":"2024-01-15T17:58:17.984103+0000","last_active":"2024-01-15T17:58:19.018703+0000","last_peered":"2024-01-15T17:58:19.018703+0000","last_clean":"2024-01-15T17:58:19.018703+0000","last_became_active":"2024-01-15T17:58:17.983807+0000","last_became_peered":"2024-01-15T17:58:17.983807+0000","last_unstale":"2024-01-15T17:58:19.018703+0000","last_undegraded":"2024-01-15T17:58:19.018703+0000","last_fullsized":"2024-01-15T17:58:19.018703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:25.210788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163150+0000","last_change":"2024-01-15T17:58:19.002031+0000","last_active":"2024-01-15T17:58:21.163150+0000","last_peered":"2024-01-15T17:58:21.163150+0000","last_clean":"2024-01-15T17:58:21.163150+0000","last_became_active":"2024-01-15T17:58:19.001699+0000","last_became_peered":"2024-01-15T17:58:19.001699+0000","last_unstale":"2024-01-15T17:58:21.163150+0000","last_undegraded":"2024-01-15T17:58:21.163150+0000","last_fullsized":"2024-01-15T17:58:21.163150+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:48.531831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019861+0000","last_change":"2024-01-15T17:58:17.987855+0000","last_active":"2024-01-15T17:58:19.019861+0000","last_peered":"2024-01-15T17:58:19.019861+0000","last_clean":"2024-01-15T17:58:19.019861+0000","last_became_active":"2024-01-15T17:58:17.987507+0000","last_became_peered":"2024-01-15T17:58:17.987507+0000","last_unstale":"2024-01-15T17:58:19.019861+0000","last_undegraded":"2024-01-15T17:58:19.019861+0000","last_fullsized":"2024-01-15T17:58:19.019861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:42.768090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161051+0000","last_change":"2024-01-15T17:58:19.003102+0000","last_active":"2024-01-15T17:58:21.161051+0000","last_peered":"2024-01-15T17:58:21.161051+0000","last_clean":"2024-01-15T17:58:21.161051+0000","last_became_active":"2024-01-15T17:58:19.002788+0000","last_became_peered":"2024-01-15T17:58:19.002788+0000","last_unstale":"2024-01-15T17:58:21.161051+0000","last_undegraded":"2024-01-15T17:58:21.161051+0000","last_fullsized":"2024-01-15T17:58:21.161051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:05.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.975625+0000","last_change":"2024-01-15T17:58:17.991304+0000","last_active":"2024-01-15T17:58:18.975625+0000","last_peered":"2024-01-15T17:58:18.975625+0000","last_clean":"2024-01-15T17:58:18.975625+0000","last_became_active":"2024-01-15T17:58:17.990871+0000","last_became_peered":"2024-01-15T17:58:17.990871+0000","last_unstale":"2024-01-15T17:58:18.975625+0000","last_undegraded":"2024-01-15T17:58:18.975625+0000","last_fullsized":"2024-01-15T17:58:18.975625+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:09.939940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985537+0000","last_change":"2024-01-15T17:58:18.995504+0000","last_active":"2024-01-15T17:58:19.985537+0000","last_peered":"2024-01-15T17:58:19.985537+0000","last_clean":"2024-01-15T17:58:19.985537+0000","last_became_active":"2024-01-15T17:58:18.995235+0000","last_became_peered":"2024-01-15T17:58:18.995235+0000","last_unstale":"2024-01-15T17:58:19.985537+0000","last_undegraded":"2024-01-15T17:58:19.985537+0000","last_fullsized":"2024-01-15T17:58:19.985537+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:39.127046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.987590+0000","last_change":"2024-01-15T17:58:17.977262+0000","last_active":"2024-01-15T17:58:18.987590+0000","last_peered":"2024-01-15T17:58:18.987590+0000","last_clean":"2024-01-15T17:58:18.987590+0000","last_became_active":"2024-01-15T17:58:17.976960+0000","last_became_peered":"2024-01-15T17:58:17.976960+0000","last_unstale":"2024-01-15T17:58:18.987590+0000","last_undegraded":"2024-01-15T17:58:18.987590+0000","last_fullsized":"2024-01-15T17:58:18.987590+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:21.203261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346224+0000","last_change":"2024-01-15T17:58:19.017344+0000","last_active":"2024-01-15T17:58:20.346224+0000","last_peered":"2024-01-15T17:58:20.346224+0000","last_clean":"2024-01-15T17:58:20.346224+0000","last_became_active":"2024-01-15T17:58:19.016961+0000","last_became_peered":"2024-01-15T17:58:19.016961+0000","last_unstale":"2024-01-15T17:58:20.346224+0000","last_undegraded":"2024-01-15T17:58:20.346224+0000","last_fullsized":"2024-01-15T17:58:20.346224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:48.531831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.967198+0000","last_change":"2024-01-15T17:58:17.984002+0000","last_active":"2024-01-15T17:58:18.967198+0000","last_peered":"2024-01-15T17:58:18.967198+0000","last_clean":"2024-01-15T17:58:18.967198+0000","last_became_active":"2024-01-15T17:58:17.983646+0000","last_became_peered":"2024-01-15T17:58:17.983646+0000","last_unstale":"2024-01-15T17:58:18.967198+0000","last_undegraded":"2024-01-15T17:58:18.967198+0000","last_fullsized":"2024-01-15T17:58:18.967198+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:39:43.632893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159543+0000","last_change":"2024-01-15T17:58:18.994672+0000","last_active":"2024-01-15T17:58:21.159543+0000","last_peered":"2024-01-15T17:58:21.159543+0000","last_clean":"2024-01-15T17:58:21.159543+0000","last_became_active":"2024-01-15T17:58:18.994367+0000","last_became_peered":"2024-01-15T17:58:18.994367+0000","last_unstale":"2024-01-15T17:58:21.159543+0000","last_undegraded":"2024-01-15T17:58:21.159543+0000","last_fullsized":"2024-01-15T17:58:21.159543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:53.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.997468+0000","last_change":"2024-01-15T17:58:17.988879+0000","last_active":"2024-01-15T17:58:18.997468+0000","last_peered":"2024-01-15T17:58:18.997468+0000","last_clean":"2024-01-15T17:58:18.997468+0000","last_became_active":"2024-01-15T17:58:17.988538+0000","last_became_peered":"2024-01-15T17:58:17.988538+0000","last_unstale":"2024-01-15T17:58:18.997468+0000","last_undegraded":"2024-01-15T17:58:18.997468+0000","last_fullsized":"2024-01-15T17:58:18.997468+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:06:27.277594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162839+0000","last_change":"2024-01-15T17:58:18.990318+0000","last_active":"2024-01-15T17:58:21.162839+0000","last_peered":"2024-01-15T17:58:21.162839+0000","last_clean":"2024-01-15T17:58:21.162839+0000","last_became_active":"2024-01-15T17:58:18.987210+0000","last_became_peered":"2024-01-15T17:58:18.987210+0000","last_unstale":"2024-01-15T17:58:21.162839+0000","last_undegraded":"2024-01-15T17:58:21.162839+0000","last_fullsized":"2024-01-15T17:58:21.162839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:01:42.464858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021391+0000","last_change":"2024-01-15T17:58:17.990723+0000","last_active":"2024-01-15T17:58:19.021391+0000","last_peered":"2024-01-15T17:58:19.021391+0000","last_clean":"2024-01-15T17:58:19.021391+0000","last_became_active":"2024-01-15T17:58:17.989841+0000","last_became_peered":"2024-01-15T17:58:17.989841+0000","last_unstale":"2024-01-15T17:58:19.021391+0000","last_undegraded":"2024-01-15T17:58:19.021391+0000","last_fullsized":"2024-01-15T17:58:19.021391+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:47.518074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159377+0000","last_change":"2024-01-15T17:58:18.992357+0000","last_active":"2024-01-15T17:58:21.159377+0000","last_peered":"2024-01-15T17:58:21.159377+0000","last_clean":"2024-01-15T17:58:21.159377+0000","last_became_active":"2024-01-15T17:58:18.992039+0000","last_became_peered":"2024-01-15T17:58:18.992039+0000","last_unstale":"2024-01-15T17:58:21.159377+0000","last_undegraded":"2024-01-15T17:58:21.159377+0000","last_fullsized":"2024-01-15T17:58:21.159377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:59:47.920817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.028902+0000","last_change":"2024-01-15T17:58:17.984206+0000","last_active":"2024-01-15T17:58:19.028902+0000","last_peered":"2024-01-15T17:58:19.028902+0000","last_clean":"2024-01-15T17:58:19.028902+0000","last_became_active":"2024-01-15T17:58:17.983944+0000","last_became_peered":"2024-01-15T17:58:17.983944+0000","last_unstale":"2024-01-15T17:58:19.028902+0000","last_undegraded":"2024-01-15T17:58:19.028902+0000","last_fullsized":"2024-01-15T17:58:19.028902+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:16.883155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987487+0000","last_change":"2024-01-15T17:58:19.004493+0000","last_active":"2024-01-15T17:58:19.987487+0000","last_peered":"2024-01-15T17:58:19.987487+0000","last_clean":"2024-01-15T17:58:19.987487+0000","last_became_active":"2024-01-15T17:58:19.004276+0000","last_became_peered":"2024-01-15T17:58:19.004276+0000","last_unstale":"2024-01-15T17:58:19.987487+0000","last_undegraded":"2024-01-15T17:58:19.987487+0000","last_fullsized":"2024-01-15T17:58:19.987487+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:12:41.726472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021349+0000","last_change":"2024-01-15T17:58:17.980792+0000","last_active":"2024-01-15T17:58:19.021349+0000","last_peered":"2024-01-15T17:58:19.021349+0000","last_clean":"2024-01-15T17:58:19.021349+0000","last_became_active":"2024-01-15T17:58:17.980499+0000","last_became_peered":"2024-01-15T17:58:17.980499+0000","last_unstale":"2024-01-15T17:58:19.021349+0000","last_undegraded":"2024-01-15T17:58:19.021349+0000","last_fullsized":"2024-01-15T17:58:19.021349+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:51:40.514469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163360+0000","last_change":"2024-01-15T17:58:19.001940+0000","last_active":"2024-01-15T17:58:21.163360+0000","last_peered":"2024-01-15T17:58:21.163360+0000","last_clean":"2024-01-15T17:58:21.163360+0000","last_became_active":"2024-01-15T17:58:19.000612+0000","last_became_peered":"2024-01-15T17:58:19.000612+0000","last_unstale":"2024-01-15T17:58:21.163360+0000","last_undegraded":"2024-01-15T17:58:21.163360+0000","last_fullsized":"2024-01-15T17:58:21.163360+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:28.109432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.986632+0000","last_change":"2024-01-15T17:58:17.988834+0000","last_active":"2024-01-15T17:58:18.986632+0000","last_peered":"2024-01-15T17:58:18.986632+0000","last_clean":"2024-01-15T17:58:18.986632+0000","last_became_active":"2024-01-15T17:58:17.988417+0000","last_became_peered":"2024-01-15T17:58:17.988417+0000","last_unstale":"2024-01-15T17:58:18.986632+0000","last_undegraded":"2024-01-15T17:58:18.986632+0000","last_fullsized":"2024-01-15T17:58:18.986632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:25.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159690+0000","last_change":"2024-01-15T17:58:18.987877+0000","last_active":"2024-01-15T17:58:21.159690+0000","last_peered":"2024-01-15T17:58:21.159690+0000","last_clean":"2024-01-15T17:58:21.159690+0000","last_became_active":"2024-01-15T17:58:18.987494+0000","last_became_peered":"2024-01-15T17:58:18.987494+0000","last_unstale":"2024-01-15T17:58:21.159690+0000","last_undegraded":"2024-01-15T17:58:21.159690+0000","last_fullsized":"2024-01-15T17:58:21.159690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:56:15.494821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.963210+0000","last_change":"2024-01-15T17:58:17.978177+0000","last_active":"2024-01-15T17:58:18.963210+0000","last_peered":"2024-01-15T17:58:18.963210+0000","last_clean":"2024-01-15T17:58:18.963210+0000","last_became_active":"2024-01-15T17:58:17.977406+0000","last_became_peered":"2024-01-15T17:58:17.977406+0000","last_unstale":"2024-01-15T17:58:18.963210+0000","last_undegraded":"2024-01-15T17:58:18.963210+0000","last_fullsized":"2024-01-15T17:58:18.963210+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:33:14.070510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158955+0000","last_change":"2024-01-15T17:58:18.994744+0000","last_active":"2024-01-15T17:58:21.158955+0000","last_peered":"2024-01-15T17:58:21.158955+0000","last_clean":"2024-01-15T17:58:21.158955+0000","last_became_active":"2024-01-15T17:58:18.994405+0000","last_became_peered":"2024-01-15T17:58:18.994405+0000","last_unstale":"2024-01-15T17:58:21.158955+0000","last_undegraded":"2024-01-15T17:58:21.158955+0000","last_fullsized":"2024-01-15T17:58:21.158955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:09.274759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.026888+0000","last_change":"2024-01-15T17:58:17.987803+0000","last_active":"2024-01-15T17:58:19.026888+0000","last_peered":"2024-01-15T17:58:19.026888+0000","last_clean":"2024-01-15T17:58:19.026888+0000","last_became_active":"2024-01-15T17:58:17.987169+0000","last_became_peered":"2024-01-15T17:58:17.987169+0000","last_unstale":"2024-01-15T17:58:19.026888+0000","last_undegraded":"2024-01-15T17:58:19.026888+0000","last_fullsized":"2024-01-15T17:58:19.026888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:25:45.206779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158413+0000","last_change":"2024-01-15T17:58:18.987935+0000","last_active":"2024-01-15T17:58:21.158413+0000","last_peered":"2024-01-15T17:58:21.158413+0000","last_clean":"2024-01-15T17:58:21.158413+0000","last_became_active":"2024-01-15T17:58:18.987605+0000","last_became_peered":"2024-01-15T17:58:18.987605+0000","last_unstale":"2024-01-15T17:58:21.158413+0000","last_undegraded":"2024-01-15T17:58:21.158413+0000","last_fullsized":"2024-01-15T17:58:21.158413+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:47.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.020407+0000","last_change":"2024-01-15T17:58:17.983214+0000","last_active":"2024-01-15T17:58:19.020407+0000","last_peered":"2024-01-15T17:58:19.020407+0000","last_clean":"2024-01-15T17:58:19.020407+0000","last_became_active":"2024-01-15T17:58:17.981263+0000","last_became_peered":"2024-01-15T17:58:17.981263+0000","last_unstale":"2024-01-15T17:58:19.020407+0000","last_undegraded":"2024-01-15T17:58:19.020407+0000","last_fullsized":"2024-01-15T17:58:19.020407+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:40.241498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164081+0000","last_change":"2024-01-15T17:58:19.003039+0000","last_active":"2024-01-15T17:58:21.164081+0000","last_peered":"2024-01-15T17:58:21.164081+0000","last_clean":"2024-01-15T17:58:21.164081+0000","last_became_active":"2024-01-15T17:58:19.002748+0000","last_became_peered":"2024-01-15T17:58:19.002748+0000","last_unstale":"2024-01-15T17:58:21.164081+0000","last_undegraded":"2024-01-15T17:58:21.164081+0000","last_fullsized":"2024-01-15T17:58:21.164081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:35.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019305+0000","last_change":"2024-01-15T17:58:17.981090+0000","last_active":"2024-01-15T17:58:19.019305+0000","last_peered":"2024-01-15T17:58:19.019305+0000","last_clean":"2024-01-15T17:58:19.019305+0000","last_became_active":"2024-01-15T17:58:17.978330+0000","last_became_peered":"2024-01-15T17:58:17.978330+0000","last_unstale":"2024-01-15T17:58:19.019305+0000","last_undegraded":"2024-01-15T17:58:19.019305+0000","last_fullsized":"2024-01-15T17:58:19.019305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:28:45.746156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985724+0000","last_change":"2024-01-15T17:58:18.997258+0000","last_active":"2024-01-15T17:58:19.985724+0000","last_peered":"2024-01-15T17:58:19.985724+0000","last_clean":"2024-01-15T17:58:19.985724+0000","last_became_active":"2024-01-15T17:58:18.996811+0000","last_became_peered":"2024-01-15T17:58:18.996811+0000","last_unstale":"2024-01-15T17:58:19.985724+0000","last_undegraded":"2024-01-15T17:58:19.985724+0000","last_fullsized":"2024-01-15T17:58:19.985724+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:09:55.110449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.018745+0000","last_change":"2024-01-15T17:58:17.991610+0000","last_active":"2024-01-15T17:58:19.018745+0000","last_peered":"2024-01-15T17:58:19.018745+0000","last_clean":"2024-01-15T17:58:19.018745+0000","last_became_active":"2024-01-15T17:58:17.991339+0000","last_became_peered":"2024-01-15T17:58:17.991339+0000","last_unstale":"2024-01-15T17:58:19.018745+0000","last_undegraded":"2024-01-15T17:58:19.018745+0000","last_fullsized":"2024-01-15T17:58:19.018745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:04.365732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987156+0000","last_change":"2024-01-15T17:58:18.989452+0000","last_active":"2024-01-15T17:58:19.987156+0000","last_peered":"2024-01-15T17:58:19.987156+0000","last_clean":"2024-01-15T17:58:19.987156+0000","last_became_active":"2024-01-15T17:58:18.989257+0000","last_became_peered":"2024-01-15T17:58:18.989257+0000","last_unstale":"2024-01-15T17:58:19.987156+0000","last_undegraded":"2024-01-15T17:58:19.987156+0000","last_fullsized":"2024-01-15T17:58:19.987156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:15:10.413760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.962717+0000","last_change":"2024-01-15T17:58:17.979360+0000","last_active":"2024-01-15T17:58:18.962717+0000","last_peered":"2024-01-15T17:58:18.962717+0000","last_clean":"2024-01-15T17:58:18.962717+0000","last_became_active":"2024-01-15T17:58:17.979089+0000","last_became_peered":"2024-01-15T17:58:17.979089+0000","last_unstale":"2024-01-15T17:58:18.962717+0000","last_undegraded":"2024-01-15T17:58:18.962717+0000","last_fullsized":"2024-01-15T17:58:18.962717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:01:09.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161038+0000","last_change":"2024-01-15T17:58:18.989730+0000","last_active":"2024-01-15T17:58:21.161038+0000","last_peered":"2024-01-15T17:58:21.161038+0000","last_clean":"2024-01-15T17:58:21.161038+0000","last_became_active":"2024-01-15T17:58:18.989337+0000","last_became_peered":"2024-01-15T17:58:18.989337+0000","last_unstale":"2024-01-15T17:58:21.161038+0000","last_undegraded":"2024-01-15T17:58:21.161038+0000","last_fullsized":"2024-01-15T17:58:21.161038+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:32:02.467555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021507+0000","last_change":"2024-01-15T17:58:17.984692+0000","last_active":"2024-01-15T17:58:19.021507+0000","last_peered":"2024-01-15T17:58:19.021507+0000","last_clean":"2024-01-15T17:58:19.021507+0000","last_became_active":"2024-01-15T17:58:17.983559+0000","last_became_peered":"2024-01-15T17:58:17.983559+0000","last_unstale":"2024-01-15T17:58:19.021507+0000","last_undegraded":"2024-01-15T17:58:19.021507+0000","last_fullsized":"2024-01-15T17:58:19.021507+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:10:46.123640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.979093+0000","last_change":"2024-01-15T17:58:18.996551+0000","last_active":"2024-01-15T17:58:19.979093+0000","last_peered":"2024-01-15T17:58:19.979093+0000","last_clean":"2024-01-15T17:58:19.979093+0000","last_became_active":"2024-01-15T17:58:18.996211+0000","last_became_peered":"2024-01-15T17:58:18.996211+0000","last_unstale":"2024-01-15T17:58:19.979093+0000","last_undegraded":"2024-01-15T17:58:19.979093+0000","last_fullsized":"2024-01-15T17:58:19.979093+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:07.285598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.979050+0000","last_change":"2024-01-15T17:58:17.984051+0000","last_active":"2024-01-15T17:58:18.979050+0000","last_peered":"2024-01-15T17:58:18.979050+0000","last_clean":"2024-01-15T17:58:18.979050+0000","last_became_active":"2024-01-15T17:58:17.983756+0000","last_became_peered":"2024-01-15T17:58:17.983756+0000","last_unstale":"2024-01-15T17:58:18.979050+0000","last_undegraded":"2024-01-15T17:58:18.979050+0000","last_fullsized":"2024-01-15T17:58:18.979050+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:27.095675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.157598+0000","last_change":"2024-01-15T17:58:18.995108+0000","last_active":"2024-01-15T17:58:21.157598+0000","last_peered":"2024-01-15T17:58:21.157598+0000","last_clean":"2024-01-15T17:58:21.157598+0000","last_became_active":"2024-01-15T17:58:18.994708+0000","last_became_peered":"2024-01-15T17:58:18.994708+0000","last_unstale":"2024-01-15T17:58:21.157598+0000","last_undegraded":"2024-01-15T17:58:21.157598+0000","last_fullsized":"2024-01-15T17:58:21.157598+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.885376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.990894+0000","last_change":"2024-01-15T17:58:17.983625+0000","last_active":"2024-01-15T17:58:18.990894+0000","last_peered":"2024-01-15T17:58:18.990894+0000","last_clean":"2024-01-15T17:58:18.990894+0000","last_became_active":"2024-01-15T17:58:17.983282+0000","last_became_peered":"2024-01-15T17:58:17.983282+0000","last_unstale":"2024-01-15T17:58:18.990894+0000","last_undegraded":"2024-01-15T17:58:18.990894+0000","last_fullsized":"2024-01-15T17:58:18.990894+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:15.887231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161926+0000","last_change":"2024-01-15T17:58:19.013008+0000","last_active":"2024-01-15T17:58:21.161926+0000","last_peered":"2024-01-15T17:58:21.161926+0000","last_clean":"2024-01-15T17:58:21.161926+0000","last_became_active":"2024-01-15T17:58:19.007927+0000","last_became_peered":"2024-01-15T17:58:19.007927+0000","last_unstale":"2024-01-15T17:58:21.161926+0000","last_undegraded":"2024-01-15T17:58:21.161926+0000","last_fullsized":"2024-01-15T17:58:21.161926+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:10.953697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.983895+0000","last_change":"2024-01-15T17:58:17.990404+0000","last_active":"2024-01-15T17:58:18.983895+0000","last_peered":"2024-01-15T17:58:18.983895+0000","last_clean":"2024-01-15T17:58:18.983895+0000","last_became_active":"2024-01-15T17:58:17.989721+0000","last_became_peered":"2024-01-15T17:58:17.989721+0000","last_unstale":"2024-01-15T17:58:18.983895+0000","last_undegraded":"2024-01-15T17:58:18.983895+0000","last_fullsized":"2024-01-15T17:58:18.983895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:34.474345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987041+0000","last_change":"2024-01-15T17:58:19.004632+0000","last_active":"2024-01-15T17:58:19.987041+0000","last_peered":"2024-01-15T17:58:19.987041+0000","last_clean":"2024-01-15T17:58:19.987041+0000","last_became_active":"2024-01-15T17:58:19.004331+0000","last_became_peered":"2024-01-15T17:58:19.004331+0000","last_unstale":"2024-01-15T17:58:19.987041+0000","last_undegraded":"2024-01-15T17:58:19.987041+0000","last_fullsized":"2024-01-15T17:58:19.987041+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:39:44.646650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.963869+0000","last_change":"2024-01-15T17:58:17.984411+0000","last_active":"2024-01-15T17:58:18.963869+0000","last_peered":"2024-01-15T17:58:18.963869+0000","last_clean":"2024-01-15T17:58:18.963869+0000","last_became_active":"2024-01-15T17:58:17.984086+0000","last_became_peered":"2024-01-15T17:58:17.984086+0000","last_unstale":"2024-01-15T17:58:18.963869+0000","last_undegraded":"2024-01-15T17:58:18.963869+0000","last_fullsized":"2024-01-15T17:58:18.963869+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:03.255576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163518+0000","last_change":"2024-01-15T17:58:19.015017+0000","last_active":"2024-01-15T17:58:21.163518+0000","last_peered":"2024-01-15T17:58:21.163518+0000","last_clean":"2024-01-15T17:58:21.163518+0000","last_became_active":"2024-01-15T17:58:19.008911+0000","last_became_peered":"2024-01-15T17:58:19.008911+0000","last_unstale":"2024-01-15T17:58:21.163518+0000","last_undegraded":"2024-01-15T17:58:21.163518+0000","last_fullsized":"2024-01-15T17:58:21.163518+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:30.537768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.979570+0000","last_change":"2024-01-15T17:58:17.990981+0000","last_active":"2024-01-15T17:58:18.979570+0000","last_peered":"2024-01-15T17:58:18.979570+0000","last_clean":"2024-01-15T17:58:18.979570+0000","last_became_active":"2024-01-15T17:58:17.990431+0000","last_became_peered":"2024-01-15T17:58:17.990431+0000","last_unstale":"2024-01-15T17:58:18.979570+0000","last_undegraded":"2024-01-15T17:58:18.979570+0000","last_fullsized":"2024-01-15T17:58:18.979570+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:27.095675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159663+0000","last_change":"2024-01-15T17:58:18.991128+0000","last_active":"2024-01-15T17:58:21.159663+0000","last_peered":"2024-01-15T17:58:21.159663+0000","last_clean":"2024-01-15T17:58:21.159663+0000","last_became_active":"2024-01-15T17:58:18.990636+0000","last_became_peered":"2024-01-15T17:58:18.990636+0000","last_unstale":"2024-01-15T17:58:21.159663+0000","last_undegraded":"2024-01-15T17:58:21.159663+0000","last_fullsized":"2024-01-15T17:58:21.159663+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:25.096243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.982822+0000","last_change":"2024-01-15T17:58:17.988011+0000","last_active":"2024-01-15T17:58:18.982822+0000","last_peered":"2024-01-15T17:58:18.982822+0000","last_clean":"2024-01-15T17:58:18.982822+0000","last_became_active":"2024-01-15T17:58:17.987726+0000","last_became_peered":"2024-01-15T17:58:17.987726+0000","last_unstale":"2024-01-15T17:58:18.982822+0000","last_undegraded":"2024-01-15T17:58:18.982822+0000","last_fullsized":"2024-01-15T17:58:18.982822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:28:01.316109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160955+0000","last_change":"2024-01-15T17:58:18.989299+0000","last_active":"2024-01-15T17:58:21.160955+0000","last_peered":"2024-01-15T17:58:21.160955+0000","last_clean":"2024-01-15T17:58:21.160955+0000","last_became_active":"2024-01-15T17:58:18.988977+0000","last_became_peered":"2024-01-15T17:58:18.988977+0000","last_unstale":"2024-01-15T17:58:21.160955+0000","last_undegraded":"2024-01-15T17:58:21.160955+0000","last_fullsized":"2024-01-15T17:58:21.160955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:14:36.844811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.977953+0000","last_change":"2024-01-15T17:58:17.991529+0000","last_active":"2024-01-15T17:58:18.977953+0000","last_peered":"2024-01-15T17:58:18.977953+0000","last_clean":"2024-01-15T17:58:18.977953+0000","last_became_active":"2024-01-15T17:58:17.990561+0000","last_became_peered":"2024-01-15T17:58:17.990561+0000","last_unstale":"2024-01-15T17:58:18.977953+0000","last_undegraded":"2024-01-15T17:58:18.977953+0000","last_fullsized":"2024-01-15T17:58:18.977953+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:00.341729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161249+0000","last_change":"2024-01-15T17:58:18.992026+0000","last_active":"2024-01-15T17:58:21.161249+0000","last_peered":"2024-01-15T17:58:21.161249+0000","last_clean":"2024-01-15T17:58:21.161249+0000","last_became_active":"2024-01-15T17:58:18.991771+0000","last_became_peered":"2024-01-15T17:58:18.991771+0000","last_unstale":"2024-01-15T17:58:21.161249+0000","last_undegraded":"2024-01-15T17:58:21.161249+0000","last_fullsized":"2024-01-15T17:58:21.161249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:10.367838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.964680+0000","last_change":"2024-01-15T17:58:17.984128+0000","last_active":"2024-01-15T17:58:18.964680+0000","last_peered":"2024-01-15T17:58:18.964680+0000","last_clean":"2024-01-15T17:58:18.964680+0000","last_became_active":"2024-01-15T17:58:17.983852+0000","last_became_peered":"2024-01-15T17:58:17.983852+0000","last_unstale":"2024-01-15T17:58:18.964680+0000","last_undegraded":"2024-01-15T17:58:18.964680+0000","last_fullsized":"2024-01-15T17:58:18.964680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:06.271842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986298+0000","last_change":"2024-01-15T17:58:18.987205+0000","last_active":"2024-01-15T17:58:19.986298+0000","last_peered":"2024-01-15T17:58:19.986298+0000","last_clean":"2024-01-15T17:58:19.986298+0000","last_became_active":"2024-01-15T17:58:18.986891+0000","last_became_peered":"2024-01-15T17:58:18.986891+0000","last_unstale":"2024-01-15T17:58:19.986298+0000","last_undegraded":"2024-01-15T17:58:19.986298+0000","last_fullsized":"2024-01-15T17:58:19.986298+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:59.015634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.982669+0000","last_change":"2024-01-15T17:58:17.983233+0000","last_active":"2024-01-15T17:58:18.982669+0000","last_peered":"2024-01-15T17:58:18.982669+0000","last_clean":"2024-01-15T17:58:18.982669+0000","last_became_active":"2024-01-15T17:58:17.981967+0000","last_became_peered":"2024-01-15T17:58:17.981967+0000","last_unstale":"2024-01-15T17:58:18.982669+0000","last_undegraded":"2024-01-15T17:58:18.982669+0000","last_fullsized":"2024-01-15T17:58:18.982669+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:46.693010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345605+0000","last_change":"2024-01-15T17:58:18.979487+0000","last_active":"2024-01-15T17:58:20.345605+0000","last_peered":"2024-01-15T17:58:20.345605+0000","last_clean":"2024-01-15T17:58:20.345605+0000","last_became_active":"2024-01-15T17:58:18.977153+0000","last_became_peered":"2024-01-15T17:58:18.977153+0000","last_unstale":"2024-01-15T17:58:20.345605+0000","last_undegraded":"2024-01-15T17:58:20.345605+0000","last_fullsized":"2024-01-15T17:58:20.345605+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:10.697813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.987930+0000","last_change":"2024-01-15T17:58:17.976886+0000","last_active":"2024-01-15T17:58:18.987930+0000","last_peered":"2024-01-15T17:58:18.987930+0000","last_clean":"2024-01-15T17:58:18.987930+0000","last_became_active":"2024-01-15T17:58:17.976596+0000","last_became_peered":"2024-01-15T17:58:17.976596+0000","last_unstale":"2024-01-15T17:58:18.987930+0000","last_undegraded":"2024-01-15T17:58:18.987930+0000","last_fullsized":"2024-01-15T17:58:18.987930+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:39:43.632893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164256+0000","last_change":"2024-01-15T17:58:19.013503+0000","last_active":"2024-01-15T17:58:21.164256+0000","last_peered":"2024-01-15T17:58:21.164256+0000","last_clean":"2024-01-15T17:58:21.164256+0000","last_became_active":"2024-01-15T17:58:19.008625+0000","last_became_peered":"2024-01-15T17:58:19.008625+0000","last_unstale":"2024-01-15T17:58:21.164256+0000","last_undegraded":"2024-01-15T17:58:21.164256+0000","last_fullsized":"2024-01-15T17:58:21.164256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:18.389244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019643+0000","last_change":"2024-01-15T17:58:17.989391+0000","last_active":"2024-01-15T17:58:19.019643+0000","last_peered":"2024-01-15T17:58:19.019643+0000","last_clean":"2024-01-15T17:58:19.019643+0000","last_became_active":"2024-01-15T17:58:17.989106+0000","last_became_peered":"2024-01-15T17:58:17.989106+0000","last_unstale":"2024-01-15T17:58:19.019643+0000","last_undegraded":"2024-01-15T17:58:19.019643+0000","last_fullsized":"2024-01-15T17:58:19.019643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:47.518074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":185,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.983999+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:58:18.983999+0000","last_peered":"2024-01-15T17:58:18.983999+0000","last_clean":"2024-01-15T17:58:18.983999+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:58:18.983999+0000","last_undegraded":"2024-01-15T17:58:18.983999+0000","last_fullsized":"2024-01-15T17:58:18.983999+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542188,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":815062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1559999999999999,"15min":1.1539999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5700000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.175,"15min":1.1679999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.6619999999999999,"5min":1.804,"15min":1.804},"last":0.95599999999999996}]},{"osd":1,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.177},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.274,"15min":1.262},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.3460000000000001,"15min":1.3759999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.5389999999999999}]},{"osd":3,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.294,"15min":1.298},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.403},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.4890000000000001,"15min":1.4930000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":3.831,"5min":3.831,"15min":3.831},"last":1.49}]},{"osd":4,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4350000000000001,"15min":1.48},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.1890000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.222},{"interface":"front","average":{"1min":1.593,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":4.4729999999999999,"5min":4.4729999999999999,"15min":4.4729999999999999},"last":1.3360000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.6000000000000001,"15min":1.605},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1589999999999998,"5min":7.3819999999999997,"15min":7.3819999999999997},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.548,"15min":1.5860000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.1230000000000002,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.28}]},{"osd":6,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.649,"15min":1.6919999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9219999999999999,"5min":2.556,"15min":2.556},"last":1.377},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5880000000000001,"15min":1.625},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1400000000000001,"5min":2.649,"15min":2.649},"last":1.4650000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542187,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":225276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.409,"15min":1.3149999999999999},"min":{"1min":0.753,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":12.544,"5min":12.544,"15min":12.544},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.206,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.083}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.29,"15min":1.202},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":12.811999999999999,"5min":12.811999999999999,"15min":12.811999999999999},"last":1.514},{"interface":"front","average":{"1min":1.206,"5min":1.1870000000000001,"15min":1.179},"min":{"1min":0.83899999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4990000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":0.88}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.214,"15min":1.2},"min":{"1min":0.94299999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5549999999999999,"5min":1.788,"15min":1.788},"last":1.145},{"interface":"front","average":{"1min":2.0169999999999999,"5min":1.5680000000000001,"15min":1.5269999999999999},"min":{"1min":0.96899999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":12.897,"5min":12.897,"15min":12.897},"last":1.363}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.169,"5min":1.597,"15min":1.5069999999999999},"min":{"1min":1.23,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":12.717000000000001,"5min":12.717000000000001,"15min":12.717000000000001},"last":1.841},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.95399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.97,"5min":2.29,"15min":2.29},"last":1.7490000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.90700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6839999999999999,"5min":2.343,"15min":2.343},"last":1.4350000000000001},{"interface":"front","average":{"1min":2.3740000000000001,"5min":1.7,"15min":1.6279999999999999},"min":{"1min":0.92000000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":16.98,"5min":16.98,"15min":16.98},"last":1.6319999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.413,"15min":1.4390000000000001},"min":{"1min":0.93500000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.101,"5min":2.548,"15min":2.548},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.583,"5min":1.6799999999999999,"15min":1.732},"min":{"1min":1.4339999999999999,"5min":1.163,"15min":1.163},"max":{"1min":1.8220000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.6839999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5249999999999999,"15min":1.5629999999999999},"min":{"1min":1.1459999999999999,"5min":1.016,"15min":1.016},"max":{"1min":2.1930000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.234},{"interface":"front","average":{"1min":1.575,"5min":1.6819999999999999,"15min":1.7370000000000001},"min":{"1min":1.3540000000000001,"5min":1.286,"15min":1.286},"max":{"1min":2.2480000000000002,"5min":3.141,"15min":3.141},"last":1.897}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":552960,"data_stored":224674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.206,"15min":1.2150000000000001},"min":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.681,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.26,"5min":1.3029999999999999,"15min":1.3600000000000001},"min":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.988,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":1.5449999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2130000000000001,"15min":1.1990000000000001},"min":{"1min":0.86499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2190000000000001,"15min":1.1970000000000001},"min":{"1min":0.91500000000000004,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.782,"5min":1.917,"15min":1.917},"last":1.1339999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2150000000000001,"15min":1.222},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7130000000000001,"5min":2.835,"15min":2.835},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.256},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.008}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.0670000000000002,"5min":1.48,"15min":1.3440000000000001},"min":{"1min":0.76000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":14.914,"5min":14.914,"15min":14.914},"last":1.23},{"interface":"front","average":{"1min":1.46,"5min":1.407,"15min":1.427},"min":{"1min":1.0409999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.944,"5min":3.302,"15min":3.302},"last":1.3580000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3839999999999999,"15min":1.407},"min":{"1min":1.03,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.8009999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.107},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.476,"15min":1.48},"min":{"1min":0.96899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":1.4910000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.577,"15min":1.625},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.0529999999999999,"5min":5.218,"15min":5.218},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.496,"5min":1.524,"15min":1.546},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.8260000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.4359999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.607,"15min":1.6220000000000001},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":4.4059999999999997,"5min":4.4059999999999997,"15min":4.4059999999999997},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6879999999999999,"15min":1.756},"min":{"1min":1.1379999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.2149999999999999,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.6619999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542187,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":561152,"data_stored":226216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2090000000000001,"15min":1.2110000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.2130000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":1.399,"15min":1.359},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":11.987,"5min":11.987,"15min":11.987},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2929999999999999,"15min":1.331},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9810000000000001,"5min":2.117,"15min":2.117},"last":1.242}]},{"osd":2,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.4450000000000001,"15min":1.4830000000000001},"min":{"1min":0.92800000000000005,"5min":0.745,"15min":0.745},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.181},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.333,"15min":1.3360000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0009999999999999,"5min":2.141,"15min":2.141},"last":1.5469999999999999}]},{"osd":3,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.528,"15min":1.5669999999999999},"min":{"1min":1.077,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.611},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.506,"15min":1.6040000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.931,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.1120000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.599,"15min":1.673},"min":{"1min":0.98299999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.5640000000000001,"15min":1.6699999999999999},"min":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1179999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.502}]},{"osd":6,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.5660000000000001,"15min":1.659},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.0419999999999998,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.821,"15min":1.837},"min":{"1min":0.92100000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":1.796}]},{"osd":7,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":2.0489999999999999,"5min":1.8759999999999999,"15min":1.8979999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":11.894,"5min":11.894,"15min":11.894},"last":1.395},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.851,"15min":1.903},"min":{"1min":1.419,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.7350000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35880,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335960,"statfs":{"total":96636764160,"available":96600023040,"internally_reserved":0,"allocated":548864,"data_stored":218914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3660000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.105},{"interface":"front","average":{"1min":1.121,"5min":1.129,"15min":1.089},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5589999999999999,"5min":3.008,"15min":3.008},"last":1.038}]},{"osd":1,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.772,"5min":1.1699999999999999,"15min":1.0609999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":12.391,"5min":12.391,"15min":12.391},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.0880000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.88800000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.169,"15min":1.1559999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.9399999999999999,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.373,"15min":1.3720000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.925,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.79200000000000004}]},{"osd":4,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.377},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.6850000000000001,"5min":4.226,"15min":4.226},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.4670000000000001,"15min":1.4299999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":7.3600000000000003,"5min":7.3600000000000003,"15min":7.3600000000000003},"last":1.6120000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.516,"15min":1.526},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.3090000000000002,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.631,"15min":1.623},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.7360000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":1.569}]},{"osd":6,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":2.052,"5min":1.7010000000000001,"15min":1.675},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":7.9630000000000001,"5min":7.9630000000000001,"15min":7.9630000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.788,"5min":1.671,"15min":1.6719999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.1659999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.671,"15min":1.6890000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.621,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":1.232},{"interface":"front","average":{"1min":1.9259999999999999,"5min":1.73,"15min":1.7210000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":7.407,"5min":7.407,"15min":7.407},"last":1.488}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":565248,"data_stored":226238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.1040000000000001,"15min":1.0409999999999999},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.427,"5min":14.427,"15min":14.427},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0049999999999999,"15min":0.99299999999999999},"min":{"1min":0.437,"5min":0.432,"15min":0.432},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.74199999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.8859999999999999,"5min":1.371,"15min":1.3149999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":14.609,"5min":14.609,"15min":14.609},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.137,"5min":1.1819999999999999,"15min":1.2010000000000001},"min":{"1min":0.56100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.722,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":0.87}]},{"osd":3,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.286,"15min":1.208},"min":{"1min":0.54500000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":16.567,"5min":16.567,"15min":16.567},"last":0.62},{"interface":"front","average":{"1min":1.9470000000000001,"5min":1.5089999999999999,"15min":1.4730000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":14.528,"5min":14.528,"15min":14.528},"last":1.25}]},{"osd":4,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.2789999999999999,"5min":1.716,"15min":1.6499999999999999},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":14.779,"5min":14.779,"15min":14.779},"last":1.149},{"interface":"front","average":{"1min":1.296,"5min":1.4239999999999999,"15min":1.462},"min":{"1min":0.878,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6519999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.458}]},{"osd":5,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.1459999999999999,"5min":1.5669999999999999,"15min":1.508},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":17.167000000000002,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.3160000000000001},{"interface":"front","average":{"1min":2.2320000000000002,"5min":1.728,"15min":1.6599999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":14.702999999999999,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.3819999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.47,"15min":1.486},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7689999999999999,"5min":2.246,"15min":2.246},"last":1.3420000000000001},{"interface":"front","average":{"1min":2.302,"5min":1.5880000000000001,"15min":1.4850000000000001},"min":{"1min":0.85399999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":17.07,"5min":17.07,"15min":17.07},"last":1.0189999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5760000000000001,"15min":1.611},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8129999999999999,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.5429999999999999},{"interface":"front","average":{"1min":3.0960000000000001,"5min":1.806,"15min":1.6060000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":16.922000000000001,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.208}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":552960,"data_stored":224674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":0.94799999999999995,"15min":0.879},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.103,"5min":0.998,"15min":0.98599999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":2.1190000000000002}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.93200000000000005,"15min":0.86799999999999999},"min":{"1min":0.48999999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.637,"5min":1.234,"15min":1.155},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":0.86199999999999999}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.9079999999999999,"15min":2.1219999999999999},"min":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":6.5949999999999998,"5min":22.106999999999999,"15min":22.106999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":2.4590000000000001,"5min":1.401,"15min":1.2370000000000001},"min":{"1min":0.47499999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":15.510999999999999,"5min":15.510999999999999,"15min":15.510999999999999},"last":2.181}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.6819999999999999,"15min":1.7529999999999999},"min":{"1min":0.98399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3230000000000004,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":2.234},{"interface":"front","average":{"1min":1.423,"5min":1.744,"15min":1.9179999999999999},"min":{"1min":0.88500000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.9729999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":0.95199999999999996}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.734,"15min":1.893},"min":{"1min":0.96999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.9359999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":3.2770000000000001,"5min":1.8979999999999999,"15min":1.7},"min":{"1min":0.88100000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":20.823,"5min":20.823,"15min":20.823},"last":0.96899999999999997}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.885,"15min":2.073},"min":{"1min":1.0469999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.8980000000000001,"5min":14.272,"15min":14.272},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.5049999999999999,"15min":1.4870000000000001},"min":{"1min":0.82599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":4.3639999999999999,"5min":4.3639999999999999,"15min":4.3639999999999999},"last":2.2829999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.589,"15min":1.556},"min":{"1min":0.94999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.615,"5min":1.546,"15min":1.554},"min":{"1min":1.111,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":0.997}]}]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36460,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335380,"statfs":{"total":96636764160,"available":96599429120,"internally_reserved":0,"allocated":1146880,"data_stored":815006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89300000000000002,"15min":0.875},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5580000000000001,"5min":1.756,"15min":1.756},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.109,"5min":1.085,"15min":1.0760000000000001},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5509999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.86599999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0509999999999999,"15min":0.995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.03,"5min":0.89800000000000002,"15min":0.84599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":0.78200000000000003}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2669999999999999,"15min":1.2609999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.238},{"interface":"front","average":{"1min":1.179,"5min":1.1040000000000001,"15min":1.056},"min":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.325}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3049999999999999,"15min":1.27},"min":{"1min":0.85199999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.5189999999999999,"15min":1.5},"min":{"1min":1.119,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.405}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7829999999999999,"5min":1.421,"15min":1.3460000000000001},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":8.0120000000000005,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4710000000000001,"15min":1.48},"min":{"1min":0.93200000000000005,"5min":0.754,"15min":0.754},"max":{"1min":2.161,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.381}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5249999999999999,"15min":1.502},"min":{"1min":1.03,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.421},{"interface":"front","average":{"1min":1.474,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.026,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.2730000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.266}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.778,"5min":1.653,"15min":1.645},"min":{"1min":0.97999999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.0259999999999998,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.724,"5min":1.581,"15min":1.55},"min":{"1min":0.94899999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.526}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:58:26.573 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:58:26.581 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:58:28.548 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T17:58:28.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.774+0000 7f8d53e00640 1 Processor -- start 2024-01-15T17:58:28.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.775+0000 7f8d53e00640 1 -- start start 2024-01-15T17:58:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.776+0000 7f8d53e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d5415b7f0 0x7f8d540adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.776+0000 7f8d53e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8d540d0900 0x7f8d540d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.776+0000 7f8d53e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540b2170 con 0x7f8d5415bbd0 2024-01-15T17:58:28.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.776+0000 7f8d53e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540b2450 con 0x7f8d5415b7f0 2024-01-15T17:58:28.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.776+0000 7f8d53e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540b2730 con 0x7f8d540d0900 2024-01-15T17:58:28.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.777+0000 7f8d52a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d5415b7f0 0x7f8d540adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:28.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.777+0000 7f8d53400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8d5415bbd0 0x7f8d5415c040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46702/0 (socket says 172.21.15.26:46702) 2024-01-15T17:58:28.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.777+0000 7f8d53400640 1 -- 172.21.15.26:0/3164323931 learned_addr learned my addr 172.21.15.26:0/3164323931 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:28.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.777+0000 7f8d49e00640 1 --2- 172.21.15.26:0/3164323931 >> v2:172.21.15.124:3300/0 conn(0x7f8d540d0900 0x7f8d540d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:28.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.777+0000 7f8d52a00640 1 -- 172.21.15.26:0/3164323931 >> v2:172.21.15.124:3300/0 conn(0x7f8d540d0900 msgr2=0x7f8d540d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:28.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.777+0000 7f8d52a00640 1 --2- 172.21.15.26:0/3164323931 >> v2:172.21.15.124:3300/0 conn(0x7f8d540d0900 0x7f8d540d17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:28.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.777+0000 7f8d52a00640 1 -- 172.21.15.26:0/3164323931 >> v1:172.21.15.26:6789/0 conn(0x7f8d5415bbd0 legacy=0x7f8d5415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:28.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.777+0000 7f8d52a00640 1 -- 172.21.15.26:0/3164323931 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540b08d0 con 0x7f8d5415b7f0 2024-01-15T17:58:28.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.778+0000 7f8d52a00640 1 --2- 172.21.15.26:0/3164323931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d5415b7f0 0x7f8d540adca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8d540b3a80 tx=0x7f8d4402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75b13869b5441a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:28.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.779+0000 7f8d52000640 1 -- 172.21.15.26:0/3164323931 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4402cbe0 con 0x7f8d5415b7f0 2024-01-15T17:58:28.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.779+0000 7f8d52000640 1 -- 172.21.15.26:0/3164323931 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4402cd40 con 0x7f8d5415b7f0 2024-01-15T17:58:28.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.779+0000 7f8d52000640 1 -- 172.21.15.26:0/3164323931 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44034660 con 0x7f8d5415b7f0 2024-01-15T17:58:28.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.779+0000 7f8d53e00640 1 -- 172.21.15.26:0/3164323931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d5415b7f0 msgr2=0x7f8d540adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:28.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.779+0000 7f8d53e00640 1 --2- 172.21.15.26:0/3164323931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d5415b7f0 0x7f8d540adca0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8d540b3a80 tx=0x7f8d4402be30 comp rx=0 tx=0).stop 2024-01-15T17:58:28.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.779+0000 7f8d53e00640 1 -- 172.21.15.26:0/3164323931 shutdown_connections 2024-01-15T17:58:28.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.779+0000 7f8d53e00640 1 --2- 172.21.15.26:0/3164323931 >> v2:172.21.15.124:3300/0 conn(0x7f8d540d0900 0x7f8d540d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:28.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.779+0000 7f8d53e00640 1 --2- 172.21.15.26:0/3164323931 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d5415b7f0 0x7f8d540adca0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:28.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.779+0000 7f8d53e00640 1 -- 172.21.15.26:0/3164323931 >> 172.21.15.26:0/3164323931 conn(0x7f8d54099a60 msgr2=0x7f8d54099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:28.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.780+0000 7f8d53e00640 1 -- 172.21.15.26:0/3164323931 shutdown_connections 2024-01-15T17:58:28.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.780+0000 7f8d53e00640 1 -- 172.21.15.26:0/3164323931 wait complete. 2024-01-15T17:58:28.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.781+0000 7f8d53e00640 1 Processor -- start 2024-01-15T17:58:28.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.781+0000 7f8d53e00640 1 -- start start 2024-01-15T17:58:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.782+0000 7f8d53e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d540d0900 0x7f8d54152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.782+0000 7f8d53e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5415b7f0 0x7f8d5406e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.782+0000 7f8d53e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415bbd0 0x7f8d5406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.782+0000 7f8d53e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5415cf50 con 0x7f8d540d0900 2024-01-15T17:58:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.782+0000 7f8d53e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5415c920 con 0x7f8d5415bbd0 2024-01-15T17:58:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.782+0000 7f8d53e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d54155740 con 0x7f8d5415b7f0 2024-01-15T17:58:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d540d0900 0x7f8d54152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d52a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d540d0900 0x7f8d54152af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57608/0 (socket says 172.21.15.26:57608) 2024-01-15T17:58:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d52a00640 1 -- 172.21.15.26:0/2438781146 learned_addr learned my addr 172.21.15.26:0/2438781146 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d53400640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415bbd0 0x7f8d5406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d52a00640 1 -- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5415b7f0 msgr2=0x7f8d5406e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d49e00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5415b7f0 0x7f8d5406e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d52a00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5415b7f0 0x7f8d5406e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d52a00640 1 -- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415bbd0 msgr2=0x7f8d5406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d52a00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415bbd0 0x7f8d5406edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d52a00640 1 -- 172.21.15.26:0/2438781146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d44009a70 con 0x7f8d540d0900 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.783+0000 7f8d53400640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415bbd0 0x7f8d5406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.784+0000 7f8d52a00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d540d0900 0x7f8d54152af0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8d44037040 tx=0x7f8d44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.784+0000 7f8d4b400640 1 -- 172.21.15.26:0/2438781146 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44040650 con 0x7f8d540d0900 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.784+0000 7f8d4b400640 1 -- 172.21.15.26:0/2438781146 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d440407b0 con 0x7f8d540d0900 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.784+0000 7f8d4b400640 1 -- 172.21.15.26:0/2438781146 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44040b60 con 0x7f8d540d0900 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.784+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d54155a20 con 0x7f8d540d0900 2024-01-15T17:58:28.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.784+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d540b56c0 con 0x7f8d540d0900 2024-01-15T17:58:28.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.785+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d54067450 con 0x7f8d540d0900 2024-01-15T17:58:28.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.786+0000 7f8d4b400640 1 -- 172.21.15.26:0/2438781146 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f8d44040cc0 con 0x7f8d540d0900 2024-01-15T17:58:28.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.787+0000 7f8d4b400640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d2c06b500 0x7f8d2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:28.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.787+0000 7f8d4b400640 1 -- 172.21.15.26:0/2438781146 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8d440acee0 con 0x7f8d540d0900 2024-01-15T17:58:28.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.788+0000 7f8d49e00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d2c06b500 0x7f8d2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:28.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.789+0000 7f8d49e00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d2c06b500 0x7f8d2c06d9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8d541f7270 tx=0x7f8d3c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:28.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.792+0000 7f8d4b400640 1 -- 172.21.15.26:0/2438781146 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d440779d0 con 0x7f8d540d0900 2024-01-15T17:58:28.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:28.982+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d540d1d10 con 0x7f8d2c06b500 2024-01-15T17:58:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.006+0000 7f8d4b400640 1 -- 172.21.15.26:0/2438781146 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286479 (secure 0 0 0) 0x7f8d3c009d80 con 0x7f8d2c06b500 2024-01-15T17:58:29.011 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:29.015 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T17:58:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.012+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d2c06b500 msgr2=0x7f8d2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.012+0000 7f8d53e00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d2c06b500 0x7f8d2c06d9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8d541f7270 tx=0x7f8d3c008040 comp rx=0 tx=0).stop 2024-01-15T17:58:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.012+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d540d0900 msgr2=0x7f8d54152af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.013+0000 7f8d53e00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d540d0900 0x7f8d54152af0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8d44037040 tx=0x7f8d44004a40 comp rx=0 tx=0).stop 2024-01-15T17:58:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.013+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 shutdown_connections 2024-01-15T17:58:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.013+0000 7f8d53e00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d5415b7f0 0x7f8d5406e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.013+0000 7f8d53e00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8d2c06b500 0x7f8d2c06d9c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.013+0000 7f8d53e00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d5415bbd0 0x7f8d5406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.013+0000 7f8d53e00640 1 --2- 172.21.15.26:0/2438781146 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d540d0900 0x7f8d54152af0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.013+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 >> 172.21.15.26:0/2438781146 conn(0x7f8d54099a60 msgr2=0x7f8d540d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.013+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 shutdown_connections 2024-01-15T17:58:29.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.013+0000 7f8d53e00640 1 -- 172.21.15.26:0/2438781146 wait complete. 2024-01-15T17:58:29.037 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-01-15T17:58:27.981293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":271596,"kb_used_data":5512,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754703124,"statfs":{"total":773094113280,"available":772815998976,"internally_reserved":0,"allocated":5644288,"data_stored":2982388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":272354864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.022816"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159357+0000","last_change":"2024-01-15T17:58:18.991217+0000","last_active":"2024-01-15T17:58:21.159357+0000","last_peered":"2024-01-15T17:58:21.159357+0000","last_clean":"2024-01-15T17:58:21.159357+0000","last_became_active":"2024-01-15T17:58:18.990850+0000","last_became_peered":"2024-01-15T17:58:18.990850+0000","last_unstale":"2024-01-15T17:58:21.159357+0000","last_undegraded":"2024-01-15T17:58:21.159357+0000","last_fullsized":"2024-01-15T17:58:21.159357+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:48.160968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987226+0000","last_change":"2024-01-15T17:58:19.015695+0000","last_active":"2024-01-15T17:58:19.987226+0000","last_peered":"2024-01-15T17:58:19.987226+0000","last_clean":"2024-01-15T17:58:19.987226+0000","last_became_active":"2024-01-15T17:58:19.015324+0000","last_became_peered":"2024-01-15T17:58:19.015324+0000","last_unstale":"2024-01-15T17:58:19.987226+0000","last_undegraded":"2024-01-15T17:58:19.987226+0000","last_fullsized":"2024-01-15T17:58:19.987226+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:09.274759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345611+0000","last_change":"2024-01-15T17:58:18.984806+0000","last_active":"2024-01-15T17:58:20.345611+0000","last_peered":"2024-01-15T17:58:20.345611+0000","last_clean":"2024-01-15T17:58:20.345611+0000","last_became_active":"2024-01-15T17:58:18.984445+0000","last_became_peered":"2024-01-15T17:58:18.984445+0000","last_unstale":"2024-01-15T17:58:20.345611+0000","last_undegraded":"2024-01-15T17:58:20.345611+0000","last_fullsized":"2024-01-15T17:58:20.345611+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:07.065888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163376+0000","last_change":"2024-01-15T17:58:19.015296+0000","last_active":"2024-01-15T17:58:21.163376+0000","last_peered":"2024-01-15T17:58:21.163376+0000","last_clean":"2024-01-15T17:58:21.163376+0000","last_became_active":"2024-01-15T17:58:19.011703+0000","last_became_peered":"2024-01-15T17:58:19.011703+0000","last_unstale":"2024-01-15T17:58:21.163376+0000","last_undegraded":"2024-01-15T17:58:21.163376+0000","last_fullsized":"2024-01-15T17:58:21.163376+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:43:54.733897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986294+0000","last_change":"2024-01-15T17:58:19.009586+0000","last_active":"2024-01-15T17:58:19.986294+0000","last_peered":"2024-01-15T17:58:19.986294+0000","last_clean":"2024-01-15T17:58:19.986294+0000","last_became_active":"2024-01-15T17:58:19.009120+0000","last_became_peered":"2024-01-15T17:58:19.009120+0000","last_unstale":"2024-01-15T17:58:19.986294+0000","last_undegraded":"2024-01-15T17:58:19.986294+0000","last_fullsized":"2024-01-15T17:58:19.986294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:51.429644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160792+0000","last_change":"2024-01-15T17:58:18.994318+0000","last_active":"2024-01-15T17:58:21.160792+0000","last_peered":"2024-01-15T17:58:21.160792+0000","last_clean":"2024-01-15T17:58:21.160792+0000","last_became_active":"2024-01-15T17:58:18.994008+0000","last_became_peered":"2024-01-15T17:58:18.994008+0000","last_unstale":"2024-01-15T17:58:21.160792+0000","last_undegraded":"2024-01-15T17:58:21.160792+0000","last_fullsized":"2024-01-15T17:58:21.160792+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:51.745964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158955+0000","last_change":"2024-01-15T17:58:18.992678+0000","last_active":"2024-01-15T17:58:21.158955+0000","last_peered":"2024-01-15T17:58:21.158955+0000","last_clean":"2024-01-15T17:58:21.158955+0000","last_became_active":"2024-01-15T17:58:18.992276+0000","last_became_peered":"2024-01-15T17:58:18.992276+0000","last_unstale":"2024-01-15T17:58:21.158955+0000","last_undegraded":"2024-01-15T17:58:21.158955+0000","last_fullsized":"2024-01-15T17:58:21.158955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:18.389244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160076+0000","last_change":"2024-01-15T17:58:19.005373+0000","last_active":"2024-01-15T17:58:21.160076+0000","last_peered":"2024-01-15T17:58:21.160076+0000","last_clean":"2024-01-15T17:58:21.160076+0000","last_became_active":"2024-01-15T17:58:19.003358+0000","last_became_peered":"2024-01-15T17:58:19.003358+0000","last_unstale":"2024-01-15T17:58:21.160076+0000","last_undegraded":"2024-01-15T17:58:21.160076+0000","last_fullsized":"2024-01-15T17:58:21.160076+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:07.065888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164063+0000","last_change":"2024-01-15T17:58:19.003597+0000","last_active":"2024-01-15T17:58:21.164063+0000","last_peered":"2024-01-15T17:58:21.164063+0000","last_clean":"2024-01-15T17:58:21.164063+0000","last_became_active":"2024-01-15T17:58:19.003270+0000","last_became_peered":"2024-01-15T17:58:19.003270+0000","last_unstale":"2024-01-15T17:58:21.164063+0000","last_undegraded":"2024-01-15T17:58:21.164063+0000","last_fullsized":"2024-01-15T17:58:21.164063+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:59.015634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164080+0000","last_change":"2024-01-15T17:58:19.002263+0000","last_active":"2024-01-15T17:58:21.164080+0000","last_peered":"2024-01-15T17:58:21.164080+0000","last_clean":"2024-01-15T17:58:21.164080+0000","last_became_active":"2024-01-15T17:58:19.000576+0000","last_became_peered":"2024-01-15T17:58:19.000576+0000","last_unstale":"2024-01-15T17:58:21.164080+0000","last_undegraded":"2024-01-15T17:58:21.164080+0000","last_fullsized":"2024-01-15T17:58:21.164080+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:26.728833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163770+0000","last_change":"2024-01-15T17:58:19.014552+0000","last_active":"2024-01-15T17:58:21.163770+0000","last_peered":"2024-01-15T17:58:21.163770+0000","last_clean":"2024-01-15T17:58:21.163770+0000","last_became_active":"2024-01-15T17:58:19.008921+0000","last_became_peered":"2024-01-15T17:58:19.008921+0000","last_unstale":"2024-01-15T17:58:21.163770+0000","last_undegraded":"2024-01-15T17:58:21.163770+0000","last_fullsized":"2024-01-15T17:58:21.163770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:51.429644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986412+0000","last_change":"2024-01-15T17:58:18.988600+0000","last_active":"2024-01-15T17:58:19.986412+0000","last_peered":"2024-01-15T17:58:19.986412+0000","last_clean":"2024-01-15T17:58:19.986412+0000","last_became_active":"2024-01-15T17:58:18.987658+0000","last_became_peered":"2024-01-15T17:58:18.987658+0000","last_unstale":"2024-01-15T17:58:19.986412+0000","last_undegraded":"2024-01-15T17:58:19.986412+0000","last_fullsized":"2024-01-15T17:58:19.986412+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:35.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158316+0000","last_change":"2024-01-15T17:58:18.998942+0000","last_active":"2024-01-15T17:58:21.158316+0000","last_peered":"2024-01-15T17:58:21.158316+0000","last_clean":"2024-01-15T17:58:21.158316+0000","last_became_active":"2024-01-15T17:58:18.998650+0000","last_became_peered":"2024-01-15T17:58:18.998650+0000","last_unstale":"2024-01-15T17:58:21.158316+0000","last_undegraded":"2024-01-15T17:58:21.158316+0000","last_fullsized":"2024-01-15T17:58:21.158316+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:39.127046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162172+0000","last_change":"2024-01-15T17:58:19.001928+0000","last_active":"2024-01-15T17:58:21.162172+0000","last_peered":"2024-01-15T17:58:21.162172+0000","last_clean":"2024-01-15T17:58:21.162172+0000","last_became_active":"2024-01-15T17:58:19.000617+0000","last_became_peered":"2024-01-15T17:58:19.000617+0000","last_unstale":"2024-01-15T17:58:21.162172+0000","last_undegraded":"2024-01-15T17:58:21.162172+0000","last_fullsized":"2024-01-15T17:58:21.162172+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:01.355486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159929+0000","last_change":"2024-01-15T17:58:18.987718+0000","last_active":"2024-01-15T17:58:21.159929+0000","last_peered":"2024-01-15T17:58:21.159929+0000","last_clean":"2024-01-15T17:58:21.159929+0000","last_became_active":"2024-01-15T17:58:18.987390+0000","last_became_peered":"2024-01-15T17:58:18.987390+0000","last_unstale":"2024-01-15T17:58:21.159929+0000","last_undegraded":"2024-01-15T17:58:21.159929+0000","last_fullsized":"2024-01-15T17:58:21.159929+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:05.757201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985536+0000","last_change":"2024-01-15T17:58:19.004638+0000","last_active":"2024-01-15T17:58:19.985536+0000","last_peered":"2024-01-15T17:58:19.985536+0000","last_clean":"2024-01-15T17:58:19.985536+0000","last_became_active":"2024-01-15T17:58:19.004325+0000","last_became_peered":"2024-01-15T17:58:19.004325+0000","last_unstale":"2024-01-15T17:58:19.985536+0000","last_undegraded":"2024-01-15T17:58:19.985536+0000","last_fullsized":"2024-01-15T17:58:19.985536+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:47.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346067+0000","last_change":"2024-01-15T17:58:18.987202+0000","last_active":"2024-01-15T17:58:20.346067+0000","last_peered":"2024-01-15T17:58:20.346067+0000","last_clean":"2024-01-15T17:58:20.346067+0000","last_became_active":"2024-01-15T17:58:18.986911+0000","last_became_peered":"2024-01-15T17:58:18.986911+0000","last_unstale":"2024-01-15T17:58:20.346067+0000","last_undegraded":"2024-01-15T17:58:20.346067+0000","last_fullsized":"2024-01-15T17:58:20.346067+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:01.355486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346145+0000","last_change":"2024-01-15T17:58:19.016790+0000","last_active":"2024-01-15T17:58:20.346145+0000","last_peered":"2024-01-15T17:58:20.346145+0000","last_clean":"2024-01-15T17:58:20.346145+0000","last_became_active":"2024-01-15T17:58:19.016452+0000","last_became_peered":"2024-01-15T17:58:19.016452+0000","last_unstale":"2024-01-15T17:58:20.346145+0000","last_undegraded":"2024-01-15T17:58:20.346145+0000","last_fullsized":"2024-01-15T17:58:20.346145+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:01:54.772028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987535+0000","last_change":"2024-01-15T17:58:19.012051+0000","last_active":"2024-01-15T17:58:19.987535+0000","last_peered":"2024-01-15T17:58:19.987535+0000","last_clean":"2024-01-15T17:58:19.987535+0000","last_became_active":"2024-01-15T17:58:19.011643+0000","last_became_peered":"2024-01-15T17:58:19.011643+0000","last_unstale":"2024-01-15T17:58:19.987535+0000","last_undegraded":"2024-01-15T17:58:19.987535+0000","last_fullsized":"2024-01-15T17:58:19.987535+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:28:46.759912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158121+0000","last_change":"2024-01-15T17:58:19.011350+0000","last_active":"2024-01-15T17:58:21.158121+0000","last_peered":"2024-01-15T17:58:21.158121+0000","last_clean":"2024-01-15T17:58:21.158121+0000","last_became_active":"2024-01-15T17:58:19.011046+0000","last_became_peered":"2024-01-15T17:58:19.011046+0000","last_unstale":"2024-01-15T17:58:21.158121+0000","last_undegraded":"2024-01-15T17:58:21.158121+0000","last_fullsized":"2024-01-15T17:58:21.158121+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:12:37.308652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161075+0000","last_change":"2024-01-15T17:58:18.996921+0000","last_active":"2024-01-15T17:58:21.161075+0000","last_peered":"2024-01-15T17:58:21.161075+0000","last_clean":"2024-01-15T17:58:21.161075+0000","last_became_active":"2024-01-15T17:58:18.994377+0000","last_became_peered":"2024-01-15T17:58:18.994377+0000","last_unstale":"2024-01-15T17:58:21.161075+0000","last_undegraded":"2024-01-15T17:58:21.161075+0000","last_fullsized":"2024-01-15T17:58:21.161075+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:54:37.664855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162353+0000","last_change":"2024-01-15T17:58:19.006417+0000","last_active":"2024-01-15T17:58:21.162353+0000","last_peered":"2024-01-15T17:58:21.162353+0000","last_clean":"2024-01-15T17:58:21.162353+0000","last_became_active":"2024-01-15T17:58:19.002993+0000","last_became_peered":"2024-01-15T17:58:19.002993+0000","last_unstale":"2024-01-15T17:58:21.162353+0000","last_undegraded":"2024-01-15T17:58:21.162353+0000","last_fullsized":"2024-01-15T17:58:21.162353+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:01:54.772028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986705+0000","last_change":"2024-01-15T17:58:18.995694+0000","last_active":"2024-01-15T17:58:19.986705+0000","last_peered":"2024-01-15T17:58:19.986705+0000","last_clean":"2024-01-15T17:58:19.986705+0000","last_became_active":"2024-01-15T17:58:18.993968+0000","last_became_peered":"2024-01-15T17:58:18.993968+0000","last_unstale":"2024-01-15T17:58:19.986705+0000","last_undegraded":"2024-01-15T17:58:19.986705+0000","last_fullsized":"2024-01-15T17:58:19.986705+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:25:46.220536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985649+0000","last_change":"2024-01-15T17:58:18.995548+0000","last_active":"2024-01-15T17:58:19.985649+0000","last_peered":"2024-01-15T17:58:19.985649+0000","last_clean":"2024-01-15T17:58:19.985649+0000","last_became_active":"2024-01-15T17:58:18.995281+0000","last_became_peered":"2024-01-15T17:58:18.995281+0000","last_unstale":"2024-01-15T17:58:19.985649+0000","last_undegraded":"2024-01-15T17:58:19.985649+0000","last_fullsized":"2024-01-15T17:58:19.985649+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:28:02.329866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986241+0000","last_change":"2024-01-15T17:58:18.992172+0000","last_active":"2024-01-15T17:58:19.986241+0000","last_peered":"2024-01-15T17:58:19.986241+0000","last_clean":"2024-01-15T17:58:19.986241+0000","last_became_active":"2024-01-15T17:58:18.991535+0000","last_became_peered":"2024-01-15T17:58:18.991535+0000","last_unstale":"2024-01-15T17:58:19.986241+0000","last_undegraded":"2024-01-15T17:58:19.986241+0000","last_fullsized":"2024-01-15T17:58:19.986241+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:07.285598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158273+0000","last_change":"2024-01-15T17:58:18.996294+0000","last_active":"2024-01-15T17:58:21.158273+0000","last_peered":"2024-01-15T17:58:21.158273+0000","last_clean":"2024-01-15T17:58:21.158273+0000","last_became_active":"2024-01-15T17:58:18.996025+0000","last_became_peered":"2024-01-15T17:58:18.996025+0000","last_unstale":"2024-01-15T17:58:21.158273+0000","last_undegraded":"2024-01-15T17:58:21.158273+0000","last_fullsized":"2024-01-15T17:58:21.158273+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:23:51.721700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346062+0000","last_change":"2024-01-15T17:58:18.984770+0000","last_active":"2024-01-15T17:58:20.346062+0000","last_peered":"2024-01-15T17:58:20.346062+0000","last_clean":"2024-01-15T17:58:20.346062+0000","last_became_active":"2024-01-15T17:58:18.984406+0000","last_became_peered":"2024-01-15T17:58:18.984406+0000","last_unstale":"2024-01-15T17:58:20.346062+0000","last_undegraded":"2024-01-15T17:58:20.346062+0000","last_fullsized":"2024-01-15T17:58:20.346062+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:10.953697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986646+0000","last_change":"2024-01-15T17:58:18.988385+0000","last_active":"2024-01-15T17:58:19.986646+0000","last_peered":"2024-01-15T17:58:19.986646+0000","last_clean":"2024-01-15T17:58:19.986646+0000","last_became_active":"2024-01-15T17:58:18.987296+0000","last_became_peered":"2024-01-15T17:58:18.987296+0000","last_unstale":"2024-01-15T17:58:19.986646+0000","last_undegraded":"2024-01-15T17:58:19.986646+0000","last_fullsized":"2024-01-15T17:58:19.986646+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:26.728833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986444+0000","last_change":"2024-01-15T17:58:18.987849+0000","last_active":"2024-01-15T17:58:19.986444+0000","last_peered":"2024-01-15T17:58:19.986444+0000","last_clean":"2024-01-15T17:58:19.986444+0000","last_became_active":"2024-01-15T17:58:18.987504+0000","last_became_peered":"2024-01-15T17:58:18.987504+0000","last_unstale":"2024-01-15T17:58:19.986444+0000","last_undegraded":"2024-01-15T17:58:19.986444+0000","last_fullsized":"2024-01-15T17:58:19.986444+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:43.781847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345606+0000","last_change":"2024-01-15T17:58:18.994812+0000","last_active":"2024-01-15T17:58:20.345606+0000","last_peered":"2024-01-15T17:58:20.345606+0000","last_clean":"2024-01-15T17:58:20.345606+0000","last_became_active":"2024-01-15T17:58:18.994420+0000","last_became_peered":"2024-01-15T17:58:18.994420+0000","last_unstale":"2024-01-15T17:58:20.345606+0000","last_undegraded":"2024-01-15T17:58:20.345606+0000","last_fullsized":"2024-01-15T17:58:20.345606+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:33.875326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160190+0000","last_change":"2024-01-15T17:58:18.987732+0000","last_active":"2024-01-15T17:58:21.160190+0000","last_peered":"2024-01-15T17:58:21.160190+0000","last_clean":"2024-01-15T17:58:21.160190+0000","last_became_active":"2024-01-15T17:58:18.986144+0000","last_became_peered":"2024-01-15T17:58:18.986144+0000","last_unstale":"2024-01-15T17:58:21.160190+0000","last_undegraded":"2024-01-15T17:58:21.160190+0000","last_fullsized":"2024-01-15T17:58:21.160190+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:55:52.164647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.157748+0000","last_change":"2024-01-15T17:58:18.983829+0000","last_active":"2024-01-15T17:58:21.157748+0000","last_peered":"2024-01-15T17:58:21.157748+0000","last_clean":"2024-01-15T17:58:21.157748+0000","last_became_active":"2024-01-15T17:58:18.983528+0000","last_became_peered":"2024-01-15T17:58:18.983528+0000","last_unstale":"2024-01-15T17:58:21.157748+0000","last_undegraded":"2024-01-15T17:58:21.157748+0000","last_fullsized":"2024-01-15T17:58:21.157748+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:53.234232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.979342+0000","last_change":"2024-01-15T17:58:18.985545+0000","last_active":"2024-01-15T17:58:19.979342+0000","last_peered":"2024-01-15T17:58:19.979342+0000","last_clean":"2024-01-15T17:58:19.979342+0000","last_became_active":"2024-01-15T17:58:18.981858+0000","last_became_peered":"2024-01-15T17:58:18.981858+0000","last_unstale":"2024-01-15T17:58:19.979342+0000","last_undegraded":"2024-01-15T17:58:19.979342+0000","last_fullsized":"2024-01-15T17:58:19.979342+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:43.781847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.018482+0000","last_change":"2024-01-15T17:58:17.980564+0000","last_active":"2024-01-15T17:58:19.018482+0000","last_peered":"2024-01-15T17:58:19.018482+0000","last_clean":"2024-01-15T17:58:19.018482+0000","last_became_active":"2024-01-15T17:58:17.979889+0000","last_became_peered":"2024-01-15T17:58:17.979889+0000","last_unstale":"2024-01-15T17:58:19.018482+0000","last_undegraded":"2024-01-15T17:58:19.018482+0000","last_fullsized":"2024-01-15T17:58:19.018482+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:14:34.561030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986677+0000","last_change":"2024-01-15T17:58:18.987759+0000","last_active":"2024-01-15T17:58:19.986677+0000","last_peered":"2024-01-15T17:58:19.986677+0000","last_clean":"2024-01-15T17:58:19.986677+0000","last_became_active":"2024-01-15T17:58:18.987413+0000","last_became_peered":"2024-01-15T17:58:18.987413+0000","last_unstale":"2024-01-15T17:58:19.986677+0000","last_undegraded":"2024-01-15T17:58:19.986677+0000","last_fullsized":"2024-01-15T17:58:19.986677+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:22.217018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019379+0000","last_change":"2024-01-15T17:58:17.981282+0000","last_active":"2024-01-15T17:58:19.019379+0000","last_peered":"2024-01-15T17:58:19.019379+0000","last_clean":"2024-01-15T17:58:19.019379+0000","last_became_active":"2024-01-15T17:58:17.980794+0000","last_became_peered":"2024-01-15T17:58:17.980794+0000","last_unstale":"2024-01-15T17:58:19.019379+0000","last_undegraded":"2024-01-15T17:58:19.019379+0000","last_fullsized":"2024-01-15T17:58:19.019379+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:15.887231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986915+0000","last_change":"2024-01-15T17:58:18.988826+0000","last_active":"2024-01-15T17:58:19.986915+0000","last_peered":"2024-01-15T17:58:19.986915+0000","last_clean":"2024-01-15T17:58:19.986915+0000","last_became_active":"2024-01-15T17:58:18.988087+0000","last_became_peered":"2024-01-15T17:58:18.988087+0000","last_unstale":"2024-01-15T17:58:19.986915+0000","last_undegraded":"2024-01-15T17:58:19.986915+0000","last_fullsized":"2024-01-15T17:58:19.986915+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:18.389244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.023715+0000","last_change":"2024-01-15T17:58:17.990072+0000","last_active":"2024-01-15T17:58:19.023715+0000","last_peered":"2024-01-15T17:58:19.023715+0000","last_clean":"2024-01-15T17:58:19.023715+0000","last_became_active":"2024-01-15T17:58:17.989788+0000","last_became_peered":"2024-01-15T17:58:17.989788+0000","last_unstale":"2024-01-15T17:58:19.023715+0000","last_undegraded":"2024-01-15T17:58:19.023715+0000","last_fullsized":"2024-01-15T17:58:19.023715+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:09:54.096692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.979523+0000","last_change":"2024-01-15T17:58:18.997500+0000","last_active":"2024-01-15T17:58:19.979523+0000","last_peered":"2024-01-15T17:58:19.979523+0000","last_clean":"2024-01-15T17:58:19.979523+0000","last_became_active":"2024-01-15T17:58:18.997234+0000","last_became_peered":"2024-01-15T17:58:18.997234+0000","last_unstale":"2024-01-15T17:58:19.979523+0000","last_undegraded":"2024-01-15T17:58:19.979523+0000","last_fullsized":"2024-01-15T17:58:19.979523+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:22.217018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021174+0000","last_change":"2024-01-15T17:58:17.989934+0000","last_active":"2024-01-15T17:58:19.021174+0000","last_peered":"2024-01-15T17:58:19.021174+0000","last_clean":"2024-01-15T17:58:19.021174+0000","last_became_active":"2024-01-15T17:58:17.989582+0000","last_became_peered":"2024-01-15T17:58:17.989582+0000","last_unstale":"2024-01-15T17:58:19.021174+0000","last_undegraded":"2024-01-15T17:58:19.021174+0000","last_fullsized":"2024-01-15T17:58:19.021174+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:04.365732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345619+0000","last_change":"2024-01-15T17:58:18.986303+0000","last_active":"2024-01-15T17:58:20.345619+0000","last_peered":"2024-01-15T17:58:20.345619+0000","last_clean":"2024-01-15T17:58:20.345619+0000","last_became_active":"2024-01-15T17:58:18.985470+0000","last_became_peered":"2024-01-15T17:58:18.985470+0000","last_unstale":"2024-01-15T17:58:20.345619+0000","last_undegraded":"2024-01-15T17:58:20.345619+0000","last_fullsized":"2024-01-15T17:58:20.345619+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:05.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.018703+0000","last_change":"2024-01-15T17:58:17.984103+0000","last_active":"2024-01-15T17:58:19.018703+0000","last_peered":"2024-01-15T17:58:19.018703+0000","last_clean":"2024-01-15T17:58:19.018703+0000","last_became_active":"2024-01-15T17:58:17.983807+0000","last_became_peered":"2024-01-15T17:58:17.983807+0000","last_unstale":"2024-01-15T17:58:19.018703+0000","last_undegraded":"2024-01-15T17:58:19.018703+0000","last_fullsized":"2024-01-15T17:58:19.018703+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:25.210788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163150+0000","last_change":"2024-01-15T17:58:19.002031+0000","last_active":"2024-01-15T17:58:21.163150+0000","last_peered":"2024-01-15T17:58:21.163150+0000","last_clean":"2024-01-15T17:58:21.163150+0000","last_became_active":"2024-01-15T17:58:19.001699+0000","last_became_peered":"2024-01-15T17:58:19.001699+0000","last_unstale":"2024-01-15T17:58:21.163150+0000","last_undegraded":"2024-01-15T17:58:21.163150+0000","last_fullsized":"2024-01-15T17:58:21.163150+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:48.531831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019861+0000","last_change":"2024-01-15T17:58:17.987855+0000","last_active":"2024-01-15T17:58:19.019861+0000","last_peered":"2024-01-15T17:58:19.019861+0000","last_clean":"2024-01-15T17:58:19.019861+0000","last_became_active":"2024-01-15T17:58:17.987507+0000","last_became_peered":"2024-01-15T17:58:17.987507+0000","last_unstale":"2024-01-15T17:58:19.019861+0000","last_undegraded":"2024-01-15T17:58:19.019861+0000","last_fullsized":"2024-01-15T17:58:19.019861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:32:42.768090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161051+0000","last_change":"2024-01-15T17:58:19.003102+0000","last_active":"2024-01-15T17:58:21.161051+0000","last_peered":"2024-01-15T17:58:21.161051+0000","last_clean":"2024-01-15T17:58:21.161051+0000","last_became_active":"2024-01-15T17:58:19.002788+0000","last_became_peered":"2024-01-15T17:58:19.002788+0000","last_unstale":"2024-01-15T17:58:21.161051+0000","last_undegraded":"2024-01-15T17:58:21.161051+0000","last_fullsized":"2024-01-15T17:58:21.161051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:05.379489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.975625+0000","last_change":"2024-01-15T17:58:17.991304+0000","last_active":"2024-01-15T17:58:18.975625+0000","last_peered":"2024-01-15T17:58:18.975625+0000","last_clean":"2024-01-15T17:58:18.975625+0000","last_became_active":"2024-01-15T17:58:17.990871+0000","last_became_peered":"2024-01-15T17:58:17.990871+0000","last_unstale":"2024-01-15T17:58:18.975625+0000","last_undegraded":"2024-01-15T17:58:18.975625+0000","last_fullsized":"2024-01-15T17:58:18.975625+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:09.939940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985537+0000","last_change":"2024-01-15T17:58:18.995504+0000","last_active":"2024-01-15T17:58:19.985537+0000","last_peered":"2024-01-15T17:58:19.985537+0000","last_clean":"2024-01-15T17:58:19.985537+0000","last_became_active":"2024-01-15T17:58:18.995235+0000","last_became_peered":"2024-01-15T17:58:18.995235+0000","last_unstale":"2024-01-15T17:58:19.985537+0000","last_undegraded":"2024-01-15T17:58:19.985537+0000","last_fullsized":"2024-01-15T17:58:19.985537+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:39.127046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.987590+0000","last_change":"2024-01-15T17:58:17.977262+0000","last_active":"2024-01-15T17:58:18.987590+0000","last_peered":"2024-01-15T17:58:18.987590+0000","last_clean":"2024-01-15T17:58:18.987590+0000","last_became_active":"2024-01-15T17:58:17.976960+0000","last_became_peered":"2024-01-15T17:58:17.976960+0000","last_unstale":"2024-01-15T17:58:18.987590+0000","last_undegraded":"2024-01-15T17:58:18.987590+0000","last_fullsized":"2024-01-15T17:58:18.987590+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:21.203261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.346224+0000","last_change":"2024-01-15T17:58:19.017344+0000","last_active":"2024-01-15T17:58:20.346224+0000","last_peered":"2024-01-15T17:58:20.346224+0000","last_clean":"2024-01-15T17:58:20.346224+0000","last_became_active":"2024-01-15T17:58:19.016961+0000","last_became_peered":"2024-01-15T17:58:19.016961+0000","last_unstale":"2024-01-15T17:58:20.346224+0000","last_undegraded":"2024-01-15T17:58:20.346224+0000","last_fullsized":"2024-01-15T17:58:20.346224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:48.531831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.967198+0000","last_change":"2024-01-15T17:58:17.984002+0000","last_active":"2024-01-15T17:58:18.967198+0000","last_peered":"2024-01-15T17:58:18.967198+0000","last_clean":"2024-01-15T17:58:18.967198+0000","last_became_active":"2024-01-15T17:58:17.983646+0000","last_became_peered":"2024-01-15T17:58:17.983646+0000","last_unstale":"2024-01-15T17:58:18.967198+0000","last_undegraded":"2024-01-15T17:58:18.967198+0000","last_fullsized":"2024-01-15T17:58:18.967198+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:39:43.632893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159543+0000","last_change":"2024-01-15T17:58:18.994672+0000","last_active":"2024-01-15T17:58:21.159543+0000","last_peered":"2024-01-15T17:58:21.159543+0000","last_clean":"2024-01-15T17:58:21.159543+0000","last_became_active":"2024-01-15T17:58:18.994367+0000","last_became_peered":"2024-01-15T17:58:18.994367+0000","last_unstale":"2024-01-15T17:58:21.159543+0000","last_undegraded":"2024-01-15T17:58:21.159543+0000","last_fullsized":"2024-01-15T17:58:21.159543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:53.669562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.997468+0000","last_change":"2024-01-15T17:58:17.988879+0000","last_active":"2024-01-15T17:58:18.997468+0000","last_peered":"2024-01-15T17:58:18.997468+0000","last_clean":"2024-01-15T17:58:18.997468+0000","last_became_active":"2024-01-15T17:58:17.988538+0000","last_became_peered":"2024-01-15T17:58:17.988538+0000","last_unstale":"2024-01-15T17:58:18.997468+0000","last_undegraded":"2024-01-15T17:58:18.997468+0000","last_fullsized":"2024-01-15T17:58:18.997468+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:06:27.277594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162839+0000","last_change":"2024-01-15T17:58:18.990318+0000","last_active":"2024-01-15T17:58:21.162839+0000","last_peered":"2024-01-15T17:58:21.162839+0000","last_clean":"2024-01-15T17:58:21.162839+0000","last_became_active":"2024-01-15T17:58:18.987210+0000","last_became_peered":"2024-01-15T17:58:18.987210+0000","last_unstale":"2024-01-15T17:58:21.162839+0000","last_undegraded":"2024-01-15T17:58:21.162839+0000","last_fullsized":"2024-01-15T17:58:21.162839+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:01:42.464858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021391+0000","last_change":"2024-01-15T17:58:17.990723+0000","last_active":"2024-01-15T17:58:19.021391+0000","last_peered":"2024-01-15T17:58:19.021391+0000","last_clean":"2024-01-15T17:58:19.021391+0000","last_became_active":"2024-01-15T17:58:17.989841+0000","last_became_peered":"2024-01-15T17:58:17.989841+0000","last_unstale":"2024-01-15T17:58:19.021391+0000","last_undegraded":"2024-01-15T17:58:19.021391+0000","last_fullsized":"2024-01-15T17:58:19.021391+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:47.518074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159377+0000","last_change":"2024-01-15T17:58:18.992357+0000","last_active":"2024-01-15T17:58:21.159377+0000","last_peered":"2024-01-15T17:58:21.159377+0000","last_clean":"2024-01-15T17:58:21.159377+0000","last_became_active":"2024-01-15T17:58:18.992039+0000","last_became_peered":"2024-01-15T17:58:18.992039+0000","last_unstale":"2024-01-15T17:58:21.159377+0000","last_undegraded":"2024-01-15T17:58:21.159377+0000","last_fullsized":"2024-01-15T17:58:21.159377+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:59:47.920817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.028902+0000","last_change":"2024-01-15T17:58:17.984206+0000","last_active":"2024-01-15T17:58:19.028902+0000","last_peered":"2024-01-15T17:58:19.028902+0000","last_clean":"2024-01-15T17:58:19.028902+0000","last_became_active":"2024-01-15T17:58:17.983944+0000","last_became_peered":"2024-01-15T17:58:17.983944+0000","last_unstale":"2024-01-15T17:58:19.028902+0000","last_undegraded":"2024-01-15T17:58:19.028902+0000","last_fullsized":"2024-01-15T17:58:19.028902+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:40:16.883155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987487+0000","last_change":"2024-01-15T17:58:19.004493+0000","last_active":"2024-01-15T17:58:19.987487+0000","last_peered":"2024-01-15T17:58:19.987487+0000","last_clean":"2024-01-15T17:58:19.987487+0000","last_became_active":"2024-01-15T17:58:19.004276+0000","last_became_peered":"2024-01-15T17:58:19.004276+0000","last_unstale":"2024-01-15T17:58:19.987487+0000","last_undegraded":"2024-01-15T17:58:19.987487+0000","last_fullsized":"2024-01-15T17:58:19.987487+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:12:41.726472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021349+0000","last_change":"2024-01-15T17:58:17.980792+0000","last_active":"2024-01-15T17:58:19.021349+0000","last_peered":"2024-01-15T17:58:19.021349+0000","last_clean":"2024-01-15T17:58:19.021349+0000","last_became_active":"2024-01-15T17:58:17.980499+0000","last_became_peered":"2024-01-15T17:58:17.980499+0000","last_unstale":"2024-01-15T17:58:19.021349+0000","last_undegraded":"2024-01-15T17:58:19.021349+0000","last_fullsized":"2024-01-15T17:58:19.021349+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:51:40.514469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163360+0000","last_change":"2024-01-15T17:58:19.001940+0000","last_active":"2024-01-15T17:58:21.163360+0000","last_peered":"2024-01-15T17:58:21.163360+0000","last_clean":"2024-01-15T17:58:21.163360+0000","last_became_active":"2024-01-15T17:58:19.000612+0000","last_became_peered":"2024-01-15T17:58:19.000612+0000","last_unstale":"2024-01-15T17:58:21.163360+0000","last_undegraded":"2024-01-15T17:58:21.163360+0000","last_fullsized":"2024-01-15T17:58:21.163360+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:28.109432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.986632+0000","last_change":"2024-01-15T17:58:17.988834+0000","last_active":"2024-01-15T17:58:18.986632+0000","last_peered":"2024-01-15T17:58:18.986632+0000","last_clean":"2024-01-15T17:58:18.986632+0000","last_became_active":"2024-01-15T17:58:17.988417+0000","last_became_peered":"2024-01-15T17:58:17.988417+0000","last_unstale":"2024-01-15T17:58:18.986632+0000","last_undegraded":"2024-01-15T17:58:18.986632+0000","last_fullsized":"2024-01-15T17:58:18.986632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:25.715076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159690+0000","last_change":"2024-01-15T17:58:18.987877+0000","last_active":"2024-01-15T17:58:21.159690+0000","last_peered":"2024-01-15T17:58:21.159690+0000","last_clean":"2024-01-15T17:58:21.159690+0000","last_became_active":"2024-01-15T17:58:18.987494+0000","last_became_peered":"2024-01-15T17:58:18.987494+0000","last_unstale":"2024-01-15T17:58:21.159690+0000","last_undegraded":"2024-01-15T17:58:21.159690+0000","last_fullsized":"2024-01-15T17:58:21.159690+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:56:15.494821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.963210+0000","last_change":"2024-01-15T17:58:17.978177+0000","last_active":"2024-01-15T17:58:18.963210+0000","last_peered":"2024-01-15T17:58:18.963210+0000","last_clean":"2024-01-15T17:58:18.963210+0000","last_became_active":"2024-01-15T17:58:17.977406+0000","last_became_peered":"2024-01-15T17:58:17.977406+0000","last_unstale":"2024-01-15T17:58:18.963210+0000","last_undegraded":"2024-01-15T17:58:18.963210+0000","last_fullsized":"2024-01-15T17:58:18.963210+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:33:14.070510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158955+0000","last_change":"2024-01-15T17:58:18.994744+0000","last_active":"2024-01-15T17:58:21.158955+0000","last_peered":"2024-01-15T17:58:21.158955+0000","last_clean":"2024-01-15T17:58:21.158955+0000","last_became_active":"2024-01-15T17:58:18.994405+0000","last_became_peered":"2024-01-15T17:58:18.994405+0000","last_unstale":"2024-01-15T17:58:21.158955+0000","last_undegraded":"2024-01-15T17:58:21.158955+0000","last_fullsized":"2024-01-15T17:58:21.158955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:54:09.274759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.026888+0000","last_change":"2024-01-15T17:58:17.987803+0000","last_active":"2024-01-15T17:58:19.026888+0000","last_peered":"2024-01-15T17:58:19.026888+0000","last_clean":"2024-01-15T17:58:19.026888+0000","last_became_active":"2024-01-15T17:58:17.987169+0000","last_became_peered":"2024-01-15T17:58:17.987169+0000","last_unstale":"2024-01-15T17:58:19.026888+0000","last_undegraded":"2024-01-15T17:58:19.026888+0000","last_fullsized":"2024-01-15T17:58:19.026888+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:25:45.206779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.158413+0000","last_change":"2024-01-15T17:58:18.987935+0000","last_active":"2024-01-15T17:58:21.158413+0000","last_peered":"2024-01-15T17:58:21.158413+0000","last_clean":"2024-01-15T17:58:21.158413+0000","last_became_active":"2024-01-15T17:58:18.987605+0000","last_became_peered":"2024-01-15T17:58:18.987605+0000","last_unstale":"2024-01-15T17:58:21.158413+0000","last_undegraded":"2024-01-15T17:58:21.158413+0000","last_fullsized":"2024-01-15T17:58:21.158413+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:47.706767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.020407+0000","last_change":"2024-01-15T17:58:17.983214+0000","last_active":"2024-01-15T17:58:19.020407+0000","last_peered":"2024-01-15T17:58:19.020407+0000","last_clean":"2024-01-15T17:58:19.020407+0000","last_became_active":"2024-01-15T17:58:17.981263+0000","last_became_peered":"2024-01-15T17:58:17.981263+0000","last_unstale":"2024-01-15T17:58:19.020407+0000","last_undegraded":"2024-01-15T17:58:19.020407+0000","last_fullsized":"2024-01-15T17:58:19.020407+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:40.241498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164081+0000","last_change":"2024-01-15T17:58:19.003039+0000","last_active":"2024-01-15T17:58:21.164081+0000","last_peered":"2024-01-15T17:58:21.164081+0000","last_clean":"2024-01-15T17:58:21.164081+0000","last_became_active":"2024-01-15T17:58:19.002748+0000","last_became_peered":"2024-01-15T17:58:19.002748+0000","last_unstale":"2024-01-15T17:58:21.164081+0000","last_undegraded":"2024-01-15T17:58:21.164081+0000","last_fullsized":"2024-01-15T17:58:21.164081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:35.488102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019305+0000","last_change":"2024-01-15T17:58:17.981090+0000","last_active":"2024-01-15T17:58:19.019305+0000","last_peered":"2024-01-15T17:58:19.019305+0000","last_clean":"2024-01-15T17:58:19.019305+0000","last_became_active":"2024-01-15T17:58:17.978330+0000","last_became_peered":"2024-01-15T17:58:17.978330+0000","last_unstale":"2024-01-15T17:58:19.019305+0000","last_undegraded":"2024-01-15T17:58:19.019305+0000","last_fullsized":"2024-01-15T17:58:19.019305+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:28:45.746156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.985724+0000","last_change":"2024-01-15T17:58:18.997258+0000","last_active":"2024-01-15T17:58:19.985724+0000","last_peered":"2024-01-15T17:58:19.985724+0000","last_clean":"2024-01-15T17:58:19.985724+0000","last_became_active":"2024-01-15T17:58:18.996811+0000","last_became_peered":"2024-01-15T17:58:18.996811+0000","last_unstale":"2024-01-15T17:58:19.985724+0000","last_undegraded":"2024-01-15T17:58:19.985724+0000","last_fullsized":"2024-01-15T17:58:19.985724+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:09:55.110449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.018745+0000","last_change":"2024-01-15T17:58:17.991610+0000","last_active":"2024-01-15T17:58:19.018745+0000","last_peered":"2024-01-15T17:58:19.018745+0000","last_clean":"2024-01-15T17:58:19.018745+0000","last_became_active":"2024-01-15T17:58:17.991339+0000","last_became_peered":"2024-01-15T17:58:17.991339+0000","last_unstale":"2024-01-15T17:58:19.018745+0000","last_undegraded":"2024-01-15T17:58:19.018745+0000","last_fullsized":"2024-01-15T17:58:19.018745+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:04.365732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987156+0000","last_change":"2024-01-15T17:58:18.989452+0000","last_active":"2024-01-15T17:58:19.987156+0000","last_peered":"2024-01-15T17:58:19.987156+0000","last_clean":"2024-01-15T17:58:19.987156+0000","last_became_active":"2024-01-15T17:58:18.989257+0000","last_became_peered":"2024-01-15T17:58:18.989257+0000","last_unstale":"2024-01-15T17:58:19.987156+0000","last_undegraded":"2024-01-15T17:58:19.987156+0000","last_fullsized":"2024-01-15T17:58:19.987156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:15:10.413760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.962717+0000","last_change":"2024-01-15T17:58:17.979360+0000","last_active":"2024-01-15T17:58:18.962717+0000","last_peered":"2024-01-15T17:58:18.962717+0000","last_clean":"2024-01-15T17:58:18.962717+0000","last_became_active":"2024-01-15T17:58:17.979089+0000","last_became_peered":"2024-01-15T17:58:17.979089+0000","last_unstale":"2024-01-15T17:58:18.962717+0000","last_undegraded":"2024-01-15T17:58:18.962717+0000","last_fullsized":"2024-01-15T17:58:18.962717+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:01:09.330325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161038+0000","last_change":"2024-01-15T17:58:18.989730+0000","last_active":"2024-01-15T17:58:21.161038+0000","last_peered":"2024-01-15T17:58:21.161038+0000","last_clean":"2024-01-15T17:58:21.161038+0000","last_became_active":"2024-01-15T17:58:18.989337+0000","last_became_peered":"2024-01-15T17:58:18.989337+0000","last_unstale":"2024-01-15T17:58:21.161038+0000","last_undegraded":"2024-01-15T17:58:21.161038+0000","last_fullsized":"2024-01-15T17:58:21.161038+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:32:02.467555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.021507+0000","last_change":"2024-01-15T17:58:17.984692+0000","last_active":"2024-01-15T17:58:19.021507+0000","last_peered":"2024-01-15T17:58:19.021507+0000","last_clean":"2024-01-15T17:58:19.021507+0000","last_became_active":"2024-01-15T17:58:17.983559+0000","last_became_peered":"2024-01-15T17:58:17.983559+0000","last_unstale":"2024-01-15T17:58:19.021507+0000","last_undegraded":"2024-01-15T17:58:19.021507+0000","last_fullsized":"2024-01-15T17:58:19.021507+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:10:46.123640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.979093+0000","last_change":"2024-01-15T17:58:18.996551+0000","last_active":"2024-01-15T17:58:19.979093+0000","last_peered":"2024-01-15T17:58:19.979093+0000","last_clean":"2024-01-15T17:58:19.979093+0000","last_became_active":"2024-01-15T17:58:18.996211+0000","last_became_peered":"2024-01-15T17:58:18.996211+0000","last_unstale":"2024-01-15T17:58:19.979093+0000","last_undegraded":"2024-01-15T17:58:19.979093+0000","last_fullsized":"2024-01-15T17:58:19.979093+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:07.285598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.979050+0000","last_change":"2024-01-15T17:58:17.984051+0000","last_active":"2024-01-15T17:58:18.979050+0000","last_peered":"2024-01-15T17:58:18.979050+0000","last_clean":"2024-01-15T17:58:18.979050+0000","last_became_active":"2024-01-15T17:58:17.983756+0000","last_became_peered":"2024-01-15T17:58:17.983756+0000","last_unstale":"2024-01-15T17:58:18.979050+0000","last_undegraded":"2024-01-15T17:58:18.979050+0000","last_fullsized":"2024-01-15T17:58:18.979050+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:27.095675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.157598+0000","last_change":"2024-01-15T17:58:18.995108+0000","last_active":"2024-01-15T17:58:21.157598+0000","last_peered":"2024-01-15T17:58:21.157598+0000","last_clean":"2024-01-15T17:58:21.157598+0000","last_became_active":"2024-01-15T17:58:18.994708+0000","last_became_peered":"2024-01-15T17:58:18.994708+0000","last_unstale":"2024-01-15T17:58:21.157598+0000","last_undegraded":"2024-01-15T17:58:21.157598+0000","last_fullsized":"2024-01-15T17:58:21.157598+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.885376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.990894+0000","last_change":"2024-01-15T17:58:17.983625+0000","last_active":"2024-01-15T17:58:18.990894+0000","last_peered":"2024-01-15T17:58:18.990894+0000","last_clean":"2024-01-15T17:58:18.990894+0000","last_became_active":"2024-01-15T17:58:17.983282+0000","last_became_peered":"2024-01-15T17:58:17.983282+0000","last_unstale":"2024-01-15T17:58:18.990894+0000","last_undegraded":"2024-01-15T17:58:18.990894+0000","last_fullsized":"2024-01-15T17:58:18.990894+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:15.887231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161926+0000","last_change":"2024-01-15T17:58:19.013008+0000","last_active":"2024-01-15T17:58:21.161926+0000","last_peered":"2024-01-15T17:58:21.161926+0000","last_clean":"2024-01-15T17:58:21.161926+0000","last_became_active":"2024-01-15T17:58:19.007927+0000","last_became_peered":"2024-01-15T17:58:19.007927+0000","last_unstale":"2024-01-15T17:58:21.161926+0000","last_undegraded":"2024-01-15T17:58:21.161926+0000","last_fullsized":"2024-01-15T17:58:21.161926+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:10.953697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.983895+0000","last_change":"2024-01-15T17:58:17.990404+0000","last_active":"2024-01-15T17:58:18.983895+0000","last_peered":"2024-01-15T17:58:18.983895+0000","last_clean":"2024-01-15T17:58:18.983895+0000","last_became_active":"2024-01-15T17:58:17.989721+0000","last_became_peered":"2024-01-15T17:58:17.989721+0000","last_unstale":"2024-01-15T17:58:18.983895+0000","last_undegraded":"2024-01-15T17:58:18.983895+0000","last_fullsized":"2024-01-15T17:58:18.983895+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:34.474345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.987041+0000","last_change":"2024-01-15T17:58:19.004632+0000","last_active":"2024-01-15T17:58:19.987041+0000","last_peered":"2024-01-15T17:58:19.987041+0000","last_clean":"2024-01-15T17:58:19.987041+0000","last_became_active":"2024-01-15T17:58:19.004331+0000","last_became_peered":"2024-01-15T17:58:19.004331+0000","last_unstale":"2024-01-15T17:58:19.987041+0000","last_undegraded":"2024-01-15T17:58:19.987041+0000","last_fullsized":"2024-01-15T17:58:19.987041+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:39:44.646650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.162647+0000","last_change":"2024-01-15T17:58:17.984411+0000","last_active":"2024-01-15T17:58:21.162647+0000","last_peered":"2024-01-15T17:58:21.162647+0000","last_clean":"2024-01-15T17:58:21.162647+0000","last_became_active":"2024-01-15T17:58:17.984086+0000","last_became_peered":"2024-01-15T17:58:17.984086+0000","last_unstale":"2024-01-15T17:58:21.162647+0000","last_undegraded":"2024-01-15T17:58:21.162647+0000","last_fullsized":"2024-01-15T17:58:21.162647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:54:03.255576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.163518+0000","last_change":"2024-01-15T17:58:19.015017+0000","last_active":"2024-01-15T17:58:21.163518+0000","last_peered":"2024-01-15T17:58:21.163518+0000","last_clean":"2024-01-15T17:58:21.163518+0000","last_became_active":"2024-01-15T17:58:19.008911+0000","last_became_peered":"2024-01-15T17:58:19.008911+0000","last_unstale":"2024-01-15T17:58:21.163518+0000","last_undegraded":"2024-01-15T17:58:21.163518+0000","last_fullsized":"2024-01-15T17:58:21.163518+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:30.537768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.979570+0000","last_change":"2024-01-15T17:58:17.990981+0000","last_active":"2024-01-15T17:58:18.979570+0000","last_peered":"2024-01-15T17:58:18.979570+0000","last_clean":"2024-01-15T17:58:18.979570+0000","last_became_active":"2024-01-15T17:58:17.990431+0000","last_became_peered":"2024-01-15T17:58:17.990431+0000","last_unstale":"2024-01-15T17:58:18.979570+0000","last_undegraded":"2024-01-15T17:58:18.979570+0000","last_fullsized":"2024-01-15T17:58:18.979570+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:27.095675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.159663+0000","last_change":"2024-01-15T17:58:18.991128+0000","last_active":"2024-01-15T17:58:21.159663+0000","last_peered":"2024-01-15T17:58:21.159663+0000","last_clean":"2024-01-15T17:58:21.159663+0000","last_became_active":"2024-01-15T17:58:18.990636+0000","last_became_peered":"2024-01-15T17:58:18.990636+0000","last_unstale":"2024-01-15T17:58:21.159663+0000","last_undegraded":"2024-01-15T17:58:21.159663+0000","last_fullsized":"2024-01-15T17:58:21.159663+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:11:25.096243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.982822+0000","last_change":"2024-01-15T17:58:17.988011+0000","last_active":"2024-01-15T17:58:18.982822+0000","last_peered":"2024-01-15T17:58:18.982822+0000","last_clean":"2024-01-15T17:58:18.982822+0000","last_became_active":"2024-01-15T17:58:17.987726+0000","last_became_peered":"2024-01-15T17:58:17.987726+0000","last_unstale":"2024-01-15T17:58:18.982822+0000","last_undegraded":"2024-01-15T17:58:18.982822+0000","last_fullsized":"2024-01-15T17:58:18.982822+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:28:01.316109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.160955+0000","last_change":"2024-01-15T17:58:18.989299+0000","last_active":"2024-01-15T17:58:21.160955+0000","last_peered":"2024-01-15T17:58:21.160955+0000","last_clean":"2024-01-15T17:58:21.160955+0000","last_became_active":"2024-01-15T17:58:18.988977+0000","last_became_peered":"2024-01-15T17:58:18.988977+0000","last_unstale":"2024-01-15T17:58:21.160955+0000","last_undegraded":"2024-01-15T17:58:21.160955+0000","last_fullsized":"2024-01-15T17:58:21.160955+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:14:36.844811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.977953+0000","last_change":"2024-01-15T17:58:17.991529+0000","last_active":"2024-01-15T17:58:18.977953+0000","last_peered":"2024-01-15T17:58:18.977953+0000","last_clean":"2024-01-15T17:58:18.977953+0000","last_became_active":"2024-01-15T17:58:17.990561+0000","last_became_peered":"2024-01-15T17:58:17.990561+0000","last_unstale":"2024-01-15T17:58:18.977953+0000","last_undegraded":"2024-01-15T17:58:18.977953+0000","last_fullsized":"2024-01-15T17:58:18.977953+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:00.341729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.161249+0000","last_change":"2024-01-15T17:58:18.992026+0000","last_active":"2024-01-15T17:58:21.161249+0000","last_peered":"2024-01-15T17:58:21.161249+0000","last_clean":"2024-01-15T17:58:21.161249+0000","last_became_active":"2024-01-15T17:58:18.991771+0000","last_became_peered":"2024-01-15T17:58:18.991771+0000","last_unstale":"2024-01-15T17:58:21.161249+0000","last_undegraded":"2024-01-15T17:58:21.161249+0000","last_fullsized":"2024-01-15T17:58:21.161249+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:36:10.367838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.964680+0000","last_change":"2024-01-15T17:58:17.984128+0000","last_active":"2024-01-15T17:58:18.964680+0000","last_peered":"2024-01-15T17:58:18.964680+0000","last_clean":"2024-01-15T17:58:18.964680+0000","last_became_active":"2024-01-15T17:58:17.983852+0000","last_became_peered":"2024-01-15T17:58:17.983852+0000","last_unstale":"2024-01-15T17:58:18.964680+0000","last_undegraded":"2024-01-15T17:58:18.964680+0000","last_fullsized":"2024-01-15T17:58:18.964680+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:06.271842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.986298+0000","last_change":"2024-01-15T17:58:18.987205+0000","last_active":"2024-01-15T17:58:19.986298+0000","last_peered":"2024-01-15T17:58:19.986298+0000","last_clean":"2024-01-15T17:58:19.986298+0000","last_became_active":"2024-01-15T17:58:18.986891+0000","last_became_peered":"2024-01-15T17:58:18.986891+0000","last_unstale":"2024-01-15T17:58:19.986298+0000","last_undegraded":"2024-01-15T17:58:19.986298+0000","last_fullsized":"2024-01-15T17:58:19.986298+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:59.015634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.982669+0000","last_change":"2024-01-15T17:58:17.983233+0000","last_active":"2024-01-15T17:58:18.982669+0000","last_peered":"2024-01-15T17:58:18.982669+0000","last_clean":"2024-01-15T17:58:18.982669+0000","last_became_active":"2024-01-15T17:58:17.981967+0000","last_became_peered":"2024-01-15T17:58:17.981967+0000","last_unstale":"2024-01-15T17:58:18.982669+0000","last_undegraded":"2024-01-15T17:58:18.982669+0000","last_fullsized":"2024-01-15T17:58:18.982669+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:46.693010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:20.345605+0000","last_change":"2024-01-15T17:58:18.979487+0000","last_active":"2024-01-15T17:58:20.345605+0000","last_peered":"2024-01-15T17:58:20.345605+0000","last_clean":"2024-01-15T17:58:20.345605+0000","last_became_active":"2024-01-15T17:58:18.977153+0000","last_became_peered":"2024-01-15T17:58:18.977153+0000","last_unstale":"2024-01-15T17:58:20.345605+0000","last_undegraded":"2024-01-15T17:58:20.345605+0000","last_fullsized":"2024-01-15T17:58:20.345605+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:10.697813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.987930+0000","last_change":"2024-01-15T17:58:17.976886+0000","last_active":"2024-01-15T17:58:18.987930+0000","last_peered":"2024-01-15T17:58:18.987930+0000","last_clean":"2024-01-15T17:58:18.987930+0000","last_became_active":"2024-01-15T17:58:17.976596+0000","last_became_peered":"2024-01-15T17:58:17.976596+0000","last_unstale":"2024-01-15T17:58:18.987930+0000","last_undegraded":"2024-01-15T17:58:18.987930+0000","last_fullsized":"2024-01-15T17:58:18.987930+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:39:43.632893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-01-15T17:58:21.164256+0000","last_change":"2024-01-15T17:58:19.013503+0000","last_active":"2024-01-15T17:58:21.164256+0000","last_peered":"2024-01-15T17:58:21.164256+0000","last_clean":"2024-01-15T17:58:21.164256+0000","last_became_active":"2024-01-15T17:58:19.008625+0000","last_became_peered":"2024-01-15T17:58:19.008625+0000","last_unstale":"2024-01-15T17:58:21.164256+0000","last_undegraded":"2024-01-15T17:58:21.164256+0000","last_fullsized":"2024-01-15T17:58:21.164256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:17.935043+0000","last_clean_scrub_stamp":"2024-01-15T17:58:17.935043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:18.389244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:19.019643+0000","last_change":"2024-01-15T17:58:17.989391+0000","last_active":"2024-01-15T17:58:19.019643+0000","last_peered":"2024-01-15T17:58:19.019643+0000","last_clean":"2024-01-15T17:58:19.019643+0000","last_became_active":"2024-01-15T17:58:17.989106+0000","last_became_peered":"2024-01-15T17:58:17.989106+0000","last_unstale":"2024-01-15T17:58:19.019643+0000","last_undegraded":"2024-01-15T17:58:19.019643+0000","last_fullsized":"2024-01-15T17:58:19.019643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:58:16.921286+0000","last_clean_scrub_stamp":"2024-01-15T17:58:16.921286+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:47.518074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":185,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-01-15T17:58:18.983999+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T17:58:18.983999+0000","last_peered":"2024-01-15T17:58:18.983999+0000","last_clean":"2024-01-15T17:58:18.983999+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T17:58:18.983999+0000","last_undegraded":"2024-01-15T17:58:18.983999+0000","last_fullsized":"2024-01-15T17:58:18.983999+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":815062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.1559999999999999,"15min":1.1539999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.5700000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.175,"15min":1.1679999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.6619999999999999,"5min":1.804,"15min":1.804},"last":1.286}]},{"osd":1,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1919999999999999,"15min":1.226},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.7,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.274,"15min":1.262},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.6319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.621}]},{"osd":2,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.1850000000000001,"15min":1.169},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.3460000000000001,"15min":1.3759999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.046}]},{"osd":3,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.294,"15min":1.298},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.5580000000000001,"5min":1.4890000000000001,"15min":1.4930000000000001},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":3.831,"5min":3.831,"15min":3.831},"last":1.2450000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4350000000000001,"15min":1.48},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.1890000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.71},{"interface":"front","average":{"1min":1.593,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.65000000000000002,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":4.4729999999999999,"5min":4.4729999999999999,"15min":4.4729999999999999},"last":1.3500000000000001}]},{"osd":5,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.6000000000000001,"15min":1.605},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":2.1589999999999998,"5min":7.3819999999999997,"15min":7.3819999999999997},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.548,"15min":1.5860000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":2.1230000000000002,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.768}]},{"osd":6,"last update":"Mon Jan 15 17:57:59 2024","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.649,"15min":1.6919999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9219999999999999,"5min":2.556,"15min":2.556},"last":1.8320000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.5880000000000001,"15min":1.625},"min":{"1min":0.75800000000000001,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1400000000000001,"5min":2.649,"15min":2.649},"last":1.5840000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542188,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":225276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.9279999999999999,"5min":1.409,"15min":1.3149999999999999},"min":{"1min":0.753,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":12.544,"5min":12.544,"15min":12.544},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.206,"15min":1.1759999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.599,"5min":1.734,"15min":1.734},"last":1.3620000000000001}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.8440000000000001,"5min":1.29,"15min":1.202},"min":{"1min":0.71799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":12.811999999999999,"5min":12.811999999999999,"15min":12.811999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.1870000000000001,"15min":1.179},"min":{"1min":0.83899999999999997,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4990000000000001,"5min":1.9139999999999999,"15min":1.9139999999999999},"last":1.0880000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.214,"15min":1.2},"min":{"1min":0.94299999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.5549999999999999,"5min":1.788,"15min":1.788},"last":1.2370000000000001},{"interface":"front","average":{"1min":2.0169999999999999,"5min":1.5680000000000001,"15min":1.5269999999999999},"min":{"1min":0.96899999999999997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":12.897,"5min":12.897,"15min":12.897},"last":1.421}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.169,"5min":1.597,"15min":1.5069999999999999},"min":{"1min":1.23,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":12.717000000000001,"5min":12.717000000000001,"15min":12.717000000000001},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.95399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.97,"5min":2.29,"15min":2.29},"last":1.4750000000000001}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.90700000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.6839999999999999,"5min":2.343,"15min":2.343},"last":0.95499999999999996},{"interface":"front","average":{"1min":2.3740000000000001,"5min":1.7,"15min":1.6279999999999999},"min":{"1min":0.92000000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":16.98,"5min":16.98,"15min":16.98},"last":1.752}]},{"osd":5,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.413,"15min":1.4390000000000001},"min":{"1min":0.93500000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"max":{"1min":2.101,"5min":2.548,"15min":2.548},"last":1.655},{"interface":"front","average":{"1min":1.583,"5min":1.6799999999999999,"15min":1.732},"min":{"1min":1.4339999999999999,"5min":1.163,"15min":1.163},"max":{"1min":1.8220000000000001,"5min":3.4830000000000001,"15min":3.4830000000000001},"last":1.337}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.5249999999999999,"15min":1.5629999999999999},"min":{"1min":1.1459999999999999,"5min":1.016,"15min":1.016},"max":{"1min":2.1930000000000001,"5min":2.4550000000000001,"15min":2.4550000000000001},"last":1.579},{"interface":"front","average":{"1min":1.575,"5min":1.6819999999999999,"15min":1.7370000000000001},"min":{"1min":1.3540000000000001,"5min":1.286,"15min":1.286},"max":{"1min":2.2480000000000002,"5min":3.141,"15min":3.141},"last":1.522}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":552960,"data_stored":224674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.206,"15min":1.2150000000000001},"min":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.681,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.26,"5min":1.3029999999999999,"15min":1.3600000000000001},"min":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.988,"5min":5.2290000000000001,"15min":5.2290000000000001},"last":0.82799999999999996}]},{"osd":1,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2130000000000001,"15min":1.1990000000000001},"min":{"1min":0.86499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2190000000000001,"15min":1.1970000000000001},"min":{"1min":0.91500000000000004,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.782,"5min":1.917,"15min":1.917},"last":0.91500000000000004}]},{"osd":2,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2150000000000001,"15min":1.222},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7130000000000001,"5min":2.835,"15min":2.835},"last":1.133},{"interface":"front","average":{"1min":1.272,"5min":1.272,"15min":1.256},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.2490000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":2.0670000000000002,"5min":1.48,"15min":1.3440000000000001},"min":{"1min":0.76000000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":14.914,"5min":14.914,"15min":14.914},"last":1.107},{"interface":"front","average":{"1min":1.46,"5min":1.407,"15min":1.427},"min":{"1min":1.0409999999999999,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.944,"5min":3.302,"15min":3.302},"last":1.3149999999999999}]},{"osd":4,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3839999999999999,"15min":1.407},"min":{"1min":1.03,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.8009999999999999,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.6559999999999999,"5min":1.476,"15min":1.48},"min":{"1min":0.96899999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":1.1899999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.577,"15min":1.625},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.0529999999999999,"5min":5.218,"15min":5.218},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.496,"5min":1.524,"15min":1.546},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":1.8260000000000001,"5min":2.1259999999999999,"15min":2.1259999999999999},"last":1.51}]},{"osd":7,"last update":"Mon Jan 15 17:58:00 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.607,"15min":1.6220000000000001},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":4.4059999999999997,"5min":4.4059999999999997,"15min":4.4059999999999997},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.6879999999999999,"15min":1.756},"min":{"1min":1.1379999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.2149999999999999,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.0549999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542188,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31728,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340112,"statfs":{"total":96636764160,"available":96604274688,"internally_reserved":0,"allocated":561152,"data_stored":226216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.2090000000000001,"15min":1.2110000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3160000000000001,"15min":1.3400000000000001},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.8100000000000001,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.3979999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.748,"5min":1.399,"15min":1.359},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":11.987,"5min":11.987,"15min":11.987},"last":1.165},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2929999999999999,"15min":1.331},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.9810000000000001,"5min":2.117,"15min":2.117},"last":0.86799999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.4450000000000001,"15min":1.4830000000000001},"min":{"1min":0.92800000000000005,"5min":0.745,"15min":0.745},"max":{"1min":2.6320000000000001,"5min":2.6320000000000001,"15min":2.6320000000000001},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.333,"15min":1.3360000000000001},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0009999999999999,"5min":2.141,"15min":2.141},"last":1.0900000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.528,"15min":1.5669999999999999},"min":{"1min":1.077,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.133},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.506,"15min":1.6040000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.931,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":1.296}]},{"osd":5,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.599,"15min":1.673},"min":{"1min":0.98299999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.3780000000000001,"5min":2.3780000000000001,"15min":2.3780000000000001},"last":1.345},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.5640000000000001,"15min":1.6699999999999999},"min":{"1min":0.57099999999999995,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1179999999999999,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.0549999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.5660000000000001,"15min":1.659},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":2.0419999999999998,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":1},{"interface":"front","average":{"1min":1.9870000000000001,"5min":1.821,"15min":1.837},"min":{"1min":0.92100000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":8.798,"5min":8.798,"15min":8.798},"last":1.2050000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:57:58 2024","interfaces":[{"interface":"back","average":{"1min":2.0489999999999999,"5min":1.8759999999999999,"15min":1.8979999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":11.894,"5min":11.894,"15min":11.894},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.7569999999999999,"5min":1.851,"15min":1.903},"min":{"1min":1.419,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":2.7200000000000002},"last":1.5189999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35888,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335952,"statfs":{"total":96636764160,"available":96600014848,"internally_reserved":0,"allocated":561152,"data_stored":225242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.98299999999999998,"15min":1},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3660000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.121,"5min":1.129,"15min":1.089},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5589999999999999,"5min":3.008,"15min":3.008},"last":1.1279999999999999}]},{"osd":1,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.772,"5min":1.1699999999999999,"15min":1.0609999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":12.391,"5min":12.391,"15min":12.391},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.079,"15min":1.0880000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.1539999999999999}]},{"osd":2,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.169,"15min":1.1559999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.9399999999999999,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":1},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.373,"15min":1.3720000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.925,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.96899999999999997}]},{"osd":4,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.381,"15min":1.377},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.6850000000000001,"5min":4.226,"15min":4.226},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.4670000000000001,"15min":1.4299999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":7.3600000000000003,"5min":7.3600000000000003,"15min":7.3600000000000003},"last":1.5629999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.516,"15min":1.526},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.3090000000000002,"5min":2.4159999999999999,"15min":2.4159999999999999},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.631,"15min":1.623},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.7360000000000002,"5min":4.3090000000000002,"15min":4.3090000000000002},"last":1.3300000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":2.052,"5min":1.7010000000000001,"15min":1.675},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":7.9630000000000001,"5min":7.9630000000000001,"15min":7.9630000000000001},"last":1.46},{"interface":"front","average":{"1min":1.788,"5min":1.671,"15min":1.6719999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":3.8679999999999999,"5min":3.8679999999999999,"15min":3.8679999999999999},"last":1.2370000000000001}]},{"osd":7,"last update":"Mon Jan 15 17:58:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.671,"15min":1.6890000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.621,"5min":4.4569999999999999,"15min":4.4569999999999999},"last":1.276},{"interface":"front","average":{"1min":1.9259999999999999,"5min":1.73,"15min":1.7210000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":7.407,"5min":7.407,"15min":7.407},"last":1.5209999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35892,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335948,"statfs":{"total":96636764160,"available":96600010752,"internally_reserved":0,"allocated":565248,"data_stored":226238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.6479999999999999,"5min":1.1040000000000001,"15min":1.0409999999999999},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":14.427,"5min":14.427,"15min":14.427},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0049999999999999,"15min":0.99299999999999999},"min":{"1min":0.437,"5min":0.432,"15min":0.432},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.88}]},{"osd":1,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.8859999999999999,"5min":1.371,"15min":1.3149999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":14.609,"5min":14.609,"15min":14.609},"last":1.169},{"interface":"front","average":{"1min":1.137,"5min":1.1819999999999999,"15min":1.2010000000000001},"min":{"1min":0.56100000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.722,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":0.79100000000000004}]},{"osd":3,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.286,"15min":1.208},"min":{"1min":0.54500000000000004,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":16.567,"5min":16.567,"15min":16.567},"last":0.748},{"interface":"front","average":{"1min":1.9470000000000001,"5min":1.5089999999999999,"15min":1.4730000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":14.528,"5min":14.528,"15min":14.528},"last":1.3919999999999999}]},{"osd":4,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.2789999999999999,"5min":1.716,"15min":1.6499999999999999},"min":{"1min":0.97099999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":14.779,"5min":14.779,"15min":14.779},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.4239999999999999,"15min":1.462},"min":{"1min":0.878,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6519999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.5569999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":2.1459999999999999,"5min":1.5669999999999999,"15min":1.508},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":17.167000000000002,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.1040000000000001},{"interface":"front","average":{"1min":2.2320000000000002,"5min":1.728,"15min":1.6599999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":14.702999999999999,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.5720000000000001}]},{"osd":6,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.47,"15min":1.486},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.7689999999999999,"5min":2.246,"15min":2.246},"last":1.266},{"interface":"front","average":{"1min":2.302,"5min":1.5880000000000001,"15min":1.4850000000000001},"min":{"1min":0.85399999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":17.07,"5min":17.07,"15min":17.07},"last":0.999}]},{"osd":7,"last update":"Mon Jan 15 17:58:01 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5760000000000001,"15min":1.611},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8129999999999999,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.484},{"interface":"front","average":{"1min":3.0960000000000001,"5min":1.806,"15min":1.6060000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":16.922000000000001,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.2170000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35884,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335956,"statfs":{"total":96636764160,"available":96600018944,"internally_reserved":0,"allocated":552960,"data_stored":224674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":0.94799999999999995,"15min":0.879},"min":{"1min":0.54100000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":3.302,"5min":3.302,"15min":3.302},"last":1.04},{"interface":"front","average":{"1min":1.103,"5min":0.998,"15min":0.98599999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.88600000000000001}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":0.93200000000000005,"15min":0.86799999999999999},"min":{"1min":0.48999999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.8079999999999998,"5min":2.8079999999999998,"15min":2.8079999999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.637,"5min":1.234,"15min":1.155},"min":{"1min":0.57499999999999996,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":3.6560000000000001,"5min":3.6560000000000001,"15min":3.6560000000000001},"last":1.7310000000000001}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.714,"5min":1.9079999999999999,"15min":2.1219999999999999},"min":{"1min":0.59599999999999997,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":6.5949999999999998,"5min":22.106999999999999,"15min":22.106999999999999},"last":1.2010000000000001},{"interface":"front","average":{"1min":2.4590000000000001,"5min":1.401,"15min":1.2370000000000001},"min":{"1min":0.47499999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":15.510999999999999,"5min":15.510999999999999,"15min":15.510999999999999},"last":1.276}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.6819999999999999,"15min":1.7529999999999999},"min":{"1min":0.98399999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":4.3230000000000004,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.744,"15min":1.9179999999999999},"min":{"1min":0.88500000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.9729999999999999,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.24}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.734,"15min":1.893},"min":{"1min":0.96999999999999997,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.9359999999999999,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.5880000000000001},{"interface":"front","average":{"1min":3.2770000000000001,"5min":1.8979999999999999,"15min":1.7},"min":{"1min":0.88100000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":20.823,"5min":20.823,"15min":20.823},"last":1.466}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.885,"15min":2.073},"min":{"1min":1.0469999999999999,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.8980000000000001,"5min":14.272,"15min":14.272},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.5049999999999999,"15min":1.4870000000000001},"min":{"1min":0.82599999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":4.3639999999999999,"5min":4.3639999999999999,"15min":4.3639999999999999},"last":1.4039999999999999}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.589,"15min":1.556},"min":{"1min":0.94999999999999996,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.615,"5min":1.546,"15min":1.554},"min":{"1min":1.111,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.8570000000000002,"5min":2.8570000000000002,"15min":2.8570000000000002},"last":1.349}]}]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36460,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335380,"statfs":{"total":96636764160,"available":96599429120,"internally_reserved":0,"allocated":1146880,"data_stored":815006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.89300000000000002,"15min":0.875},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.5580000000000001,"5min":1.756,"15min":1.756},"last":0.751},{"interface":"front","average":{"1min":1.109,"5min":1.085,"15min":1.0760000000000001},"min":{"1min":0.63300000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5509999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.97999999999999998}]},{"osd":2,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.0509999999999999,"15min":0.995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":2.0870000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.03,"5min":0.89800000000000002,"15min":0.84599999999999997},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":2.129,"5min":2.129,"15min":2.129},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.2669999999999999,"15min":1.2609999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.179,"5min":1.1040000000000001,"15min":1.056},"min":{"1min":0.42399999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":0.84199999999999997}]},{"osd":4,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.3049999999999999,"15min":1.27},"min":{"1min":0.85199999999999998,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":2.5779999999999998,"5min":2.5779999999999998,"15min":2.5779999999999998},"last":1.103},{"interface":"front","average":{"1min":1.6879999999999999,"5min":1.5189999999999999,"15min":1.5},"min":{"1min":1.119,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":4.0140000000000002,"5min":4.0140000000000002,"15min":4.0140000000000002},"last":1.1759999999999999}]},{"osd":5,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.7829999999999999,"5min":1.421,"15min":1.3460000000000001},"min":{"1min":0.88300000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":8.0120000000000005,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.4750000000000001,"5min":1.4710000000000001,"15min":1.48},"min":{"1min":0.93200000000000005,"5min":0.754,"15min":0.754},"max":{"1min":2.161,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.1419999999999999}]},{"osd":6,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.6890000000000001,"5min":1.5249999999999999,"15min":1.502},"min":{"1min":1.03,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":3.2669999999999999,"5min":3.2669999999999999,"15min":3.2669999999999999},"last":1.038},{"interface":"front","average":{"1min":1.474,"5min":1.464,"15min":1.4419999999999999},"min":{"1min":1.026,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":2.2730000000000001,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":0.95699999999999996}]},{"osd":7,"last update":"Mon Jan 15 17:58:04 2024","interfaces":[{"interface":"back","average":{"1min":1.778,"5min":1.653,"15min":1.645},"min":{"1min":0.97999999999999998,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":3.0259999999999998,"5min":3.8100000000000001,"15min":3.8100000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.724,"5min":1.581,"15min":1.55},"min":{"1min":0.94899999999999995,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":3.742,"5min":3.742,"15min":3.742},"last":1.21}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T17:58:29.041 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T17:58:29.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T17:58:29.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.312+0000 7f837ee00640 1 Processor -- start 2024-01-15T17:58:29.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.312+0000 7f837ee00640 1 -- start start 2024-01-15T17:58:29.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.313+0000 7f837ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 0x7f837806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837ee00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f837806a920 0x7f83780b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837ee00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837806ad80 con 0x7f83780b4020 2024-01-15T17:58:29.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837ee00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83780b47d0 con 0x7f837806a920 2024-01-15T17:58:29.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83780b4ab0 con 0x7f8378146970 2024-01-15T17:58:29.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f837806a920 0x7f83780b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f837806a920 0x7f83780b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57612/0 (socket says 172.21.15.26:57612) 2024-01-15T17:58:29.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837d000640 1 -- 172.21.15.26:0/4082275619 learned_addr learned my addr 172.21.15.26:0/4082275619 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837da00640 1 --2- 172.21.15.26:0/4082275619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 0x7f837806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837d000640 1 -- 172.21.15.26:0/4082275619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 msgr2=0x7f837806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837d000640 1 --2- 172.21.15.26:0/4082275619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 0x7f837806a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837d000640 1 -- 172.21.15.26:0/4082275619 >> v1:172.21.15.124:6789/0 conn(0x7f83780b4020 legacy=0x7f83780713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:29.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.314+0000 7f837d000640 1 -- 172.21.15.26:0/4082275619 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780b4d90 con 0x7f837806a920 2024-01-15T17:58:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.316+0000 7f837d000640 1 --2- 172.21.15.26:0/4082275619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f837806a920 0x7f83780b3ae0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8360009870 tx=0x7f836002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe5e0f4ba928352 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.316+0000 7f8377e00640 1 -- 172.21.15.26:0/4082275619 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360031640 con 0x7f837806a920 2024-01-15T17:58:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.316+0000 7f8377e00640 1 -- 172.21.15.26:0/4082275619 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83600317a0 con 0x7f837806a920 2024-01-15T17:58:29.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.316+0000 7f8377e00640 1 -- 172.21.15.26:0/4082275619 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360031b30 con 0x7f837806a920 2024-01-15T17:58:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.317+0000 7f837ee00640 1 -- 172.21.15.26:0/4082275619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f837806a920 msgr2=0x7f83780b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.317+0000 7f837ee00640 1 --2- 172.21.15.26:0/4082275619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f837806a920 0x7f83780b3ae0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8360009870 tx=0x7f836002b8f0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.317+0000 7f837ee00640 1 -- 172.21.15.26:0/4082275619 shutdown_connections 2024-01-15T17:58:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.317+0000 7f837ee00640 1 --2- 172.21.15.26:0/4082275619 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f837806a920 0x7f83780b3ae0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.317+0000 7f837ee00640 1 --2- 172.21.15.26:0/4082275619 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 0x7f837806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.317+0000 7f837ee00640 1 -- 172.21.15.26:0/4082275619 >> 172.21.15.26:0/4082275619 conn(0x7f83780b1dd0 msgr2=0x7f83780b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.317+0000 7f837ee00640 1 -- 172.21.15.26:0/4082275619 shutdown_connections 2024-01-15T17:58:29.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.317+0000 7f837ee00640 1 -- 172.21.15.26:0/4082275619 wait complete. 2024-01-15T17:58:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.319+0000 7f837ee00640 1 Processor -- start 2024-01-15T17:58:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.319+0000 7f837ee00640 1 -- start start 2024-01-15T17:58:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.320+0000 7f837ee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f837806a920 0x7f837815f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.320+0000 7f837ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83780b4020 0x7f83781371c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.320+0000 7f837ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 0x7f83781401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.320+0000 7f837ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837813c3c0 con 0x7f83780b4020 2024-01-15T17:58:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.320+0000 7f837ee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837813c6a0 con 0x7f837806a920 2024-01-15T17:58:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.320+0000 7f837ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837815fa90 con 0x7f8378146970 2024-01-15T17:58:29.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.320+0000 7f837d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83780b4020 0x7f83781371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83780b4020 0x7f83781371c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57618/0 (socket says 172.21.15.26:57618) 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837d000640 1 -- 172.21.15.26:0/2253448818 learned_addr learned my addr 172.21.15.26:0/2253448818 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837da00640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f837806a920 0x7f837815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837e400640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 0x7f83781401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837d000640 1 -- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 msgr2=0x7f83781401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837d000640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 0x7f83781401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837d000640 1 -- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f837806a920 msgr2=0x7f837815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837d000640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f837806a920 0x7f837815f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837d000640 1 -- 172.21.15.26:0/2253448818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83600094d0 con 0x7f83780b4020 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837e400640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 0x7f83781401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.321+0000 7f837d000640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83780b4020 0x7f83781371c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8360000c00 tx=0x7f83600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.322+0000 7f8375600640 1 -- 172.21.15.26:0/2253448818 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836003ac60 con 0x7f83780b4020 2024-01-15T17:58:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.322+0000 7f8375600640 1 -- 172.21.15.26:0/2253448818 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836003ade0 con 0x7f83780b4020 2024-01-15T17:58:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.322+0000 7f8375600640 1 -- 172.21.15.26:0/2253448818 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360044630 con 0x7f83780b4020 2024-01-15T17:58:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.322+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837815fd70 con 0x7f83780b4020 2024-01-15T17:58:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.322+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837815f790 con 0x7f83780b4020 2024-01-15T17:58:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.323+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8378067b60 con 0x7f83780b4020 2024-01-15T17:58:29.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.324+0000 7f8375600640 1 -- 172.21.15.26:0/2253448818 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f836002f050 con 0x7f83780b4020 2024-01-15T17:58:29.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.325+0000 7f8375600640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f835006b500 0x7f835006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.325+0000 7f8375600640 1 -- 172.21.15.26:0/2253448818 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f83600ac250 con 0x7f83780b4020 2024-01-15T17:58:29.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.325+0000 7f837da00640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f835006b500 0x7f835006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.326+0000 7f837da00640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f835006b500 0x7f835006d9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8368006fd0 tx=0x7f8368008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:29.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.329+0000 7f8375600640 1 -- 172.21.15.26:0/2253448818 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8360076c60 con 0x7f83780b4020 2024-01-15T17:58:29.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.591+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f83780b1100 con 0x7f83780b4020 2024-01-15T17:58:29.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.596+0000 7f8375600640 1 -- 172.21.15.26:0/2253448818 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8360032040 con 0x7f83780b4020 2024-01-15T17:58:29.600 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-15T17:58:29.600 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA== 2024-01-15T17:58:29.600 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-01-15T17:58:29.600 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-01-15T17:58:29.600 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T17:58:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-01-15T17:58:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f835006b500 msgr2=0x7f835006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f835006b500 0x7f835006d9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8368006fd0 tx=0x7f8368008040 comp rx=0 tx=0).stop 2024-01-15T17:58:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83780b4020 msgr2=0x7f83781371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83780b4020 0x7f83781371c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8360000c00 tx=0x7f83600046c0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 shutdown_connections 2024-01-15T17:58:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8378146970 0x7f83781401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f835006b500 0x7f835006d9c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83780b4020 0x7f83781371c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 --2- 172.21.15.26:0/2253448818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f837806a920 0x7f837815f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.599+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 >> 172.21.15.26:0/2253448818 conn(0x7f83780b1dd0 msgr2=0x7f837806d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.600+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 shutdown_connections 2024-01-15T17:58:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.600+0000 7f837ee00640 1 -- 172.21.15.26:0/2253448818 wait complete. 2024-01-15T17:58:29.623 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T17:58:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.836+0000 7fb33b400640 1 Processor -- start 2024-01-15T17:58:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.836+0000 7fb33b400640 1 -- start start 2024-01-15T17:58:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.837+0000 7fb33b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb334143c40 0x7fb334071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb33b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3340717d0 0x7fb3340cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb33b400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb334070e50 con 0x7fb3340cca40 2024-01-15T17:58:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb33b400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3340723c0 con 0x7fb3340717d0 2024-01-15T17:58:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb33b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340cce70 con 0x7fb334143c40 2024-01-15T17:58:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb339600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3340717d0 0x7fb3340cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb339600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb3340717d0 0x7fb3340cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57626/0 (socket says 172.21.15.26:57626) 2024-01-15T17:58:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb339600640 1 -- 172.21.15.26:0/3181564199 learned_addr learned my addr 172.21.15.26:0/3181564199 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb33a000640 1 --2- 172.21.15.26:0/3181564199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb334143c40 0x7fb334071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb339600640 1 -- 172.21.15.26:0/3181564199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb334143c40 msgr2=0x7fb334071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb339600640 1 --2- 172.21.15.26:0/3181564199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb334143c40 0x7fb334071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb339600640 1 -- 172.21.15.26:0/3181564199 >> v1:172.21.15.124:6789/0 conn(0x7fb3340cca40 legacy=0x7fb33415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.838+0000 7fb339600640 1 -- 172.21.15.26:0/3181564199 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33406e170 con 0x7fb3340717d0 2024-01-15T17:58:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.840+0000 7fb339600640 1 --2- 172.21.15.26:0/3181564199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3340717d0 0x7fb3340cc500 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009870 tx=0x7fb31c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b11e840fd1c0c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.840+0000 7fb338c00640 1 -- 172.21.15.26:0/3181564199 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c0317c0 con 0x7fb3340717d0 2024-01-15T17:58:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.840+0000 7fb338c00640 1 -- 172.21.15.26:0/3181564199 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb31c031920 con 0x7fb3340717d0 2024-01-15T17:58:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.840+0000 7fb338c00640 1 -- 172.21.15.26:0/3181564199 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb31c031cb0 con 0x7fb3340717d0 2024-01-15T17:58:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.841+0000 7fb33b400640 1 -- 172.21.15.26:0/3181564199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3340717d0 msgr2=0x7fb3340cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.841+0000 7fb33b400640 1 --2- 172.21.15.26:0/3181564199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3340717d0 0x7fb3340cc500 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009870 tx=0x7fb31c02b860 comp rx=0 tx=0).stop 2024-01-15T17:58:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.841+0000 7fb33b400640 1 -- 172.21.15.26:0/3181564199 shutdown_connections 2024-01-15T17:58:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.841+0000 7fb33b400640 1 --2- 172.21.15.26:0/3181564199 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb3340717d0 0x7fb3340cc500 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.841+0000 7fb33b400640 1 --2- 172.21.15.26:0/3181564199 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb334143c40 0x7fb334071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.841+0000 7fb33b400640 1 -- 172.21.15.26:0/3181564199 >> 172.21.15.26:0/3181564199 conn(0x7fb33406a800 msgr2=0x7fb33406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:29.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.841+0000 7fb33b400640 1 -- 172.21.15.26:0/3181564199 shutdown_connections 2024-01-15T17:58:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.841+0000 7fb33b400640 1 -- 172.21.15.26:0/3181564199 wait complete. 2024-01-15T17:58:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.843+0000 7fb33b400640 1 Processor -- start 2024-01-15T17:58:29.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.843+0000 7fb33b400640 1 -- start start 2024-01-15T17:58:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33b400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3340717d0 0x7fb33406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3340cca40 0x7fb334156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb334156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33413f470 con 0x7fb334143c40 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33b400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb334157060 con 0x7fb3340717d0 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb334157340 con 0x7fb3340cca40 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb334156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb334156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57640/0 (socket says 172.21.15.26:57640) 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33aa00640 1 -- 172.21.15.26:0/291941563 learned_addr learned my addr 172.21.15.26:0/291941563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb339600640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3340cca40 0x7fb334156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.844+0000 7fb33a000640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3340717d0 0x7fb33406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb33aa00640 1 -- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3340cca40 msgr2=0x7fb334156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb33aa00640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3340cca40 0x7fb334156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb33aa00640 1 -- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3340717d0 msgr2=0x7fb33406d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb33aa00640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3340717d0 0x7fb33406d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb33aa00640 1 -- 172.21.15.26:0/291941563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c0094d0 con 0x7fb334143c40 2024-01-15T17:58:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb33a000640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3340717d0 0x7fb33406d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb33aa00640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb334156970 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c780 tx=0x7fb32c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb332000640 1 -- 172.21.15.26:0/291941563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c004b60 con 0x7fb334143c40 2024-01-15T17:58:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb332000640 1 -- 172.21.15.26:0/291941563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c00ce90 con 0x7fb334143c40 2024-01-15T17:58:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3341543d0 con 0x7fb334143c40 2024-01-15T17:58:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb332000640 1 -- 172.21.15.26:0/291941563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c00a850 con 0x7fb334143c40 2024-01-15T17:58:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.845+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb334154990 con 0x7fb334143c40 2024-01-15T17:58:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.847+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb334154b80 con 0x7fb334143c40 2024-01-15T17:58:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.847+0000 7fb332000640 1 -- 172.21.15.26:0/291941563 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fb32c00a9b0 con 0x7fb334143c40 2024-01-15T17:58:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.848+0000 7fb332000640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb30c06b500 0x7fb30c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.848+0000 7fb332000640 1 -- 172.21.15.26:0/291941563 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb32c08a200 con 0x7fb334143c40 2024-01-15T17:58:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.849+0000 7fb33a000640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb30c06b500 0x7fb30c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.850+0000 7fb33a000640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb30c06b500 0x7fb30c06d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb324006fd0 tx=0x7fb324008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:29.853+0000 7fb332000640 1 -- 172.21.15.26:0/291941563 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb32c054ce0 con 0x7fb334143c40 2024-01-15T17:58:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.091+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb334067450 con 0x7fb334143c40 2024-01-15T17:58:30.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.093+0000 7fb332000640 1 -- 172.21.15.26:0/291941563 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fb32c058300 con 0x7fb334143c40 2024-01-15T17:58:30.096 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:58:30.096 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5218,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3293492366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3293492366},{"type":"v1","addr":"172.21.15.124:6836","nonce":3293492366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1049920857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":1049920857},{"type":"v1","addr":"172.21.15.124:6837","nonce":1049920857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2644667232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2644667232},{"type":"v1","addr":"172.21.15.26:6834","nonce":2644667232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T17:58:18.443247+0000","modified":"2024-01-15T17:58:21.537724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5802},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5802":{"gid":5802,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1948779240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1948779240},{"type":"v1","addr":"172.21.15.26:6835","nonce":1948779240}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-01-15T17:58:30.096 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 25 2024-01-15T17:58:30.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.095+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb30c06b500 msgr2=0x7fb30c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.095+0000 7fb33b400640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb30c06b500 0x7fb30c06d9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb324006fd0 tx=0x7fb324008040 comp rx=0 tx=0).stop 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.095+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 msgr2=0x7fb334156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.095+0000 7fb33b400640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb334156970 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c780 tx=0x7fb32c00cc40 comp rx=0 tx=0).stop 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.095+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 shutdown_connections 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.095+0000 7fb33b400640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb3340cca40 0x7fb334156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.095+0000 7fb33b400640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb30c06b500 0x7fb30c06d9c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.095+0000 7fb33b400640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb334143c40 0x7fb334156970 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.095+0000 7fb33b400640 1 --2- 172.21.15.26:0/291941563 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb3340717d0 0x7fb33406d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.096+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 >> 172.21.15.26:0/291941563 conn(0x7fb33406a800 msgr2=0x7fb33406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:30.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.096+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 shutdown_connections 2024-01-15T17:58:30.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.096+0000 7fb33b400640 1 -- 172.21.15.26:0/291941563 wait complete. 2024-01-15T17:58:30.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T17:58:18.443247+0000', 'modified': '2024-01-15T17:58:21.537724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5802}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5802': {'gid': 5802, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1948779240', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1948779240}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1948779240}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T17:58:30.522 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T17:58:30.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-15T17:58:30.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.744+0000 7f0b2b400640 1 Processor -- start 2024-01-15T17:58:30.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.744+0000 7f0b2b400640 1 -- start start 2024-01-15T17:58:30.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.745+0000 7f0b2b400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0b24160920 0x7f0b240b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.746+0000 7f0b2b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b24161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.746+0000 7f0b2b400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b240d0a10 con 0x7f0b240cf9f0 2024-01-15T17:58:30.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.746+0000 7f0b2b400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b240d0330 con 0x7f0b24160920 2024-01-15T17:58:30.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.746+0000 7f0b2b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240d1f50 con 0x7f0b240cfea0 2024-01-15T17:58:30.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.746+0000 7f0b29600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0b240cf9f0 0x7f0b24161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46716/0 (socket says 172.21.15.26:46716) 2024-01-15T17:58:30.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.746+0000 7f0b29600640 1 -- 172.21.15.26:0/3587965481 learned_addr learned my addr 172.21.15.26:0/3587965481 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.746+0000 7f0b2aa00640 1 --2- 172.21.15.26:0/3587965481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b24161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.746+0000 7f0b2a000640 1 --2- 172.21.15.26:0/3587965481 >> v2:172.21.15.124:3300/0 conn(0x7f0b24160920 0x7f0b240b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.747+0000 7f0b28c00640 1 -- 172.21.15.26:0/3587965481 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613905958 0 0) 0x7f0b240d0a10 con 0x7f0b240cf9f0 2024-01-15T17:58:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.747+0000 7f0b28c00640 1 -- 172.21.15.26:0/3587965481 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b04003520 con 0x7f0b240cf9f0 2024-01-15T17:58:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.747+0000 7f0b2aa00640 1 -- 172.21.15.26:0/3587965481 >> v2:172.21.15.124:3300/0 conn(0x7f0b24160920 msgr2=0x7f0b240b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:30.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.747+0000 7f0b2aa00640 1 --2- 172.21.15.26:0/3587965481 >> v2:172.21.15.124:3300/0 conn(0x7f0b24160920 0x7f0b240b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.747+0000 7f0b2aa00640 1 -- 172.21.15.26:0/3587965481 >> v1:172.21.15.26:6789/0 conn(0x7f0b240cf9f0 legacy=0x7f0b24161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.747+0000 7f0b2aa00640 1 -- 172.21.15.26:0/3587965481 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b240cf080 con 0x7f0b240cfea0 2024-01-15T17:58:30.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.749+0000 7f0b2aa00640 1 --2- 172.21.15.26:0/3587965481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b24161720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b140097a0 tx=0x7f0b1402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e34d7211d5c5a8d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:30.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.749+0000 7f0b28c00640 1 -- 172.21.15.26:0/3587965481 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14031830 con 0x7f0b240cfea0 2024-01-15T17:58:30.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.749+0000 7f0b28c00640 1 -- 172.21.15.26:0/3587965481 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b14031990 con 0x7f0b240cfea0 2024-01-15T17:58:30.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.749+0000 7f0b28c00640 1 -- 172.21.15.26:0/3587965481 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14031d00 con 0x7f0b240cfea0 2024-01-15T17:58:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.750+0000 7f0b2b400640 1 -- 172.21.15.26:0/3587965481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 msgr2=0x7f0b24161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.750+0000 7f0b2b400640 1 --2- 172.21.15.26:0/3587965481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b24161720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0b140097a0 tx=0x7f0b1402b8d0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.750+0000 7f0b2b400640 1 -- 172.21.15.26:0/3587965481 shutdown_connections 2024-01-15T17:58:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.750+0000 7f0b2b400640 1 --2- 172.21.15.26:0/3587965481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b24161720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.750+0000 7f0b2b400640 1 --2- 172.21.15.26:0/3587965481 >> v2:172.21.15.124:3300/0 conn(0x7f0b24160920 0x7f0b240b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.750+0000 7f0b2b400640 1 -- 172.21.15.26:0/3587965481 >> 172.21.15.26:0/3587965481 conn(0x7f0b240d3090 msgr2=0x7f0b240d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.750+0000 7f0b2b400640 1 -- 172.21.15.26:0/3587965481 shutdown_connections 2024-01-15T17:58:30.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.750+0000 7f0b2b400640 1 -- 172.21.15.26:0/3587965481 wait complete. 2024-01-15T17:58:30.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.752+0000 7f0b2b400640 1 Processor -- start 2024-01-15T17:58:30.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.752+0000 7f0b2b400640 1 -- start start 2024-01-15T17:58:30.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.753+0000 7f0b2b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b240cf9f0 0x7f0b2415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.753+0000 7f0b2b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b240b8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.753+0000 7f0b2b400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b24160920 0x7f0b2415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.753+0000 7f0b2b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240b7f00 con 0x7f0b240cf9f0 2024-01-15T17:58:30.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.753+0000 7f0b2b400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240cd460 con 0x7f0b24160920 2024-01-15T17:58:30.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.753+0000 7f0b2b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240b72f0 con 0x7f0b240cfea0 2024-01-15T17:58:30.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b240cf9f0 0x7f0b2415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b240cf9f0 0x7f0b2415b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57648/0 (socket says 172.21.15.26:57648) 2024-01-15T17:58:30.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2a000640 1 -- 172.21.15.26:0/4212466434 learned_addr learned my addr 172.21.15.26:0/4212466434 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2aa00640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b24160920 0x7f0b2415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b29600640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b240b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2a000640 1 -- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 msgr2=0x7f0b240b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2a000640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b240b8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2a000640 1 -- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b24160920 msgr2=0x7f0b2415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2a000640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b24160920 0x7f0b2415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2a000640 1 -- 172.21.15.26:0/4212466434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b14009400 con 0x7f0b240cf9f0 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b2aa00640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b24160920 0x7f0b2415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.754+0000 7f0b29600640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b240b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.755+0000 7f0b2a000640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b240cf9f0 0x7f0b2415b410 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00efc0 tx=0x7f0b1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.755+0000 7f0b22000640 1 -- 172.21.15.26:0/4212466434 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c014480 con 0x7f0b240cf9f0 2024-01-15T17:58:30.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.755+0000 7f0b2b400640 1 -- 172.21.15.26:0/4212466434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b240b75d0 con 0x7f0b240cf9f0 2024-01-15T17:58:30.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.755+0000 7f0b22000640 1 -- 172.21.15.26:0/4212466434 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1c00f040 con 0x7f0b240cf9f0 2024-01-15T17:58:30.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.755+0000 7f0b22000640 1 -- 172.21.15.26:0/4212466434 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c014800 con 0x7f0b240cf9f0 2024-01-15T17:58:30.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.755+0000 7f0b2b400640 1 -- 172.21.15.26:0/4212466434 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b240b7b90 con 0x7f0b240cf9f0 2024-01-15T17:58:30.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.757+0000 7f0b22000640 1 -- 172.21.15.26:0/4212466434 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f0b1c014960 con 0x7f0b240cf9f0 2024-01-15T17:58:30.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.758+0000 7f0b22000640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0afc06b500 0x7f0afc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.758+0000 7f0b22000640 1 -- 172.21.15.26:0/4212466434 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0b1c08adb0 con 0x7f0b240cf9f0 2024-01-15T17:58:30.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.759+0000 7f0b29600640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0afc06b500 0x7f0afc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.759+0000 7f0b2f5fe740 1 Processor -- start 2024-01-15T17:58:30.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.759+0000 7f0b2f5fe740 1 -- start start 2024-01-15T17:58:30.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.760+0000 7f0b29600640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0afc06b500 0x7f0afc06d9c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0b24161260 tx=0x7f0b0c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:30.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.760+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed376070 0x5622ed376450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.760+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed370d70 0x5622ed375270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.760+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3757b0 0x5622ed3b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.760+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ed2d4880 con 0x5622ed3757b0 2024-01-15T17:58:30.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.760+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ed2e3ae0 con 0x5622ed370d70 2024-01-15T17:58:30.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.760+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622ed2e11d0 con 0x5622ed376070 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b2aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3757b0 0x5622ed3b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b2aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3757b0 0x5622ed3b9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57658/0 (socket says 172.21.15.26:57658) 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b2aa00640 1 -- 172.21.15.26:0/4153647023 learned_addr learned my addr 172.21.15.26:0/4153647023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b29600640 1 --2- 172.21.15.26:0/4153647023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed370d70 0x5622ed375270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b2a000640 1 --2- 172.21.15.26:0/4153647023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed376070 0x5622ed376450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b2aa00640 1 -- 172.21.15.26:0/4153647023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed376070 msgr2=0x5622ed376450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b2aa00640 1 --2- 172.21.15.26:0/4153647023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed376070 0x5622ed376450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b2aa00640 1 -- 172.21.15.26:0/4153647023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed370d70 msgr2=0x5622ed375270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b2aa00640 1 --2- 172.21.15.26:0/4153647023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed370d70 0x5622ed375270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.761+0000 7f0b2aa00640 1 -- 172.21.15.26:0/4153647023 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622ed2a4860 con 0x5622ed3757b0 2024-01-15T17:58:30.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.762+0000 7f0b2aa00640 1 --2- 172.21.15.26:0/4153647023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3757b0 0x5622ed3b9730 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009890 tx=0x7f0b14030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7044efd83aaa97d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.762+0000 7f0b1aa00640 1 -- 172.21.15.26:0/4153647023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1403f070 con 0x5622ed3757b0 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.762+0000 7f0b1aa00640 1 -- 172.21.15.26:0/4153647023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b14031730 con 0x5622ed3757b0 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.762+0000 7f0b1aa00640 1 -- 172.21.15.26:0/4153647023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14031d00 con 0x5622ed3757b0 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.762+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4153647023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3757b0 msgr2=0x5622ed3b9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.762+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/4153647023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3757b0 0x5622ed3b9730 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009890 tx=0x7f0b14030000 comp rx=0 tx=0).stop 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.763+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4153647023 shutdown_connections 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.763+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/4153647023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3757b0 0x5622ed3b9730 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.763+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/4153647023 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed370d70 0x5622ed375270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.763+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/4153647023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed376070 0x5622ed376450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.763+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4153647023 >> 172.21.15.26:0/4153647023 conn(0x5622ed337c70 msgr2=0x5622ed3bbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.763+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4153647023 shutdown_connections 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.763+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4153647023 wait complete. 2024-01-15T17:58:30.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.764+0000 7f0b2f5fe740 1 -- start start 2024-01-15T17:58:30.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.764+0000 7f0b2f5fe740 1 Processor -- start 2024-01-15T17:58:30.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed370d70 0x5622ed39c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed3757b0 0x5622ed3a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed376070 0x5622ed3b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ed2dcac0 con 0x5622ed370d70 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ed2792f0 con 0x5622ed376070 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622ed296c70 con 0x5622ed3757b0 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2f5fe740 10 client.0 _ll_register_callbacks cb 0x5622ed29b3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed370d70 0x5622ed39c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed370d70 0x5622ed39c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57666/0 (socket says 172.21.15.26:57666) 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.765+0000 7f0b2a000640 1 -- 172.21.15.26:0/2166103277 learned_addr learned my addr 172.21.15.26:0/2166103277 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b29600640 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed3757b0 0x5622ed3a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b2aa00640 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed376070 0x5622ed3b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b2a000640 1 -- 172.21.15.26:0/2166103277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed3757b0 msgr2=0x5622ed3a42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b2a000640 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed3757b0 0x5622ed3a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b2a000640 1 -- 172.21.15.26:0/2166103277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed376070 msgr2=0x5622ed3b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b2a000640 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed376070 0x5622ed3b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b2a000640 1 -- 172.21.15.26:0/2166103277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b240cf080 con 0x5622ed370d70 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b2a000640 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed370d70 0x5622ed39c3d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0590c0 tx=0x7f0b1c05d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3203a0bdd04e6786 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b13400640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b13400640 1 -- 172.21.15.26:0/2166103277 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c061230 con 0x5622ed370d70 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b13400640 1 -- 172.21.15.26:0/2166103277 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0b1c061390 con 0x5622ed370d70 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2166103277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622ed2d5aa0 con 0x5622ed370d70 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b13400640 1 -- 172.21.15.26:0/2166103277 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b1c0890f0 con 0x5622ed370d70 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.766+0000 7f0b2f5fe740 10 client.5958 fetch_fsmap learned FSMap version 25 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2166103277 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5622ed2db140 con 0x5622ed370d70 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b13400640 1 -- 172.21.15.26:0/2166103277 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5622ed2d5aa0 con 0x5622ed370d70 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b13400640 1 -- 172.21.15.26:0/2166103277 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0b1c089250 con 0x5622ed370d70 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b2f5fe740 10 client.5958 fetch_fsmap finished waiting for FSMap version 25 2024-01-15T17:58:30.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b2f5fe740 10 client.5958 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T17:58:30.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b2f5fe740 20 client.5958 populate_metadata read hostname 'smithi026' 2024-01-15T17:58:30.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] conn(0x5622ed3c1e10 0x5622ed39e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b2f5fe740 4 client.5958 mds_command: new command op to 5802 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T17:58:30.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2166103277 --> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622ed2d57c0 con 0x5622ed3c1e10 2024-01-15T17:58:30.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.767+0000 7f0b29600640 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] conn(0x5622ed3c1e10 0x5622ed39e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.768+0000 7f0b29600640 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] conn(0x5622ed3c1e10 0x5622ed39e470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ca9f68d2284a86 server_cookie=dac5935ad7879a33 in_seq=0 out_seq=0 2024-01-15T17:58:30.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.768+0000 7f0b13400640 10 client.5958 ms_handle_connect on v2:172.21.15.26:6833/1948779240 2024-01-15T17:58:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b13400640 1 -- 172.21.15.26:0/2166103277 <== mds.0 v2:172.21.15.26:6833/1948779240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622ed2d57c0 con 0x5622ed3c1e10 2024-01-15T17:58:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b13400640 10 client.5958 handle_command_reply: tid=0 2024-01-15T17:58:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 client.5958 shutdown 2024-01-15T17:58:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2166103277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed370d70 msgr2=0x5622ed39c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed370d70 0x5622ed39c3d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0590c0 tx=0x7f0b1c05d5d0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2166103277 shutdown_connections 2024-01-15T17:58:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] conn(0x5622ed3c1e10 0x5622ed39e470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed3757b0 0x5622ed3a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed376070 0x5622ed3b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2166103277 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed370d70 0x5622ed39c3d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2166103277 >> 172.21.15.26:0/2166103277 conn(0x5622ed337c70 msgr2=0x5622ed36fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:30.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b13400640 0 client.5958 ms_handle_reset on v2:172.21.15.26:6833/1948779240 2024-01-15T17:58:30.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2166103277 shutdown_connections 2024-01-15T17:58:30.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.769+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2166103277 wait complete. 2024-01-15T17:58:30.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.770+0000 7f0b2f5fe740 20 client.5958 trim_cache size 0 max 16384 2024-01-15T17:58:30.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.787+0000 7f0b2f5fe740 1 Processor -- start 2024-01-15T17:58:30.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.787+0000 7f0b2f5fe740 1 -- start start 2024-01-15T17:58:30.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.788+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed3757b0 0x5622ed39b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.788+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed376070 0x5622ed3bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.788+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3713f0 0x5622ed39a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.788+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ed03ff20 con 0x5622ed376070 2024-01-15T17:58:30.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.788+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ed2dea40 con 0x5622ed3713f0 2024-01-15T17:58:30.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.788+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622ed3a1470 con 0x5622ed3757b0 2024-01-15T17:58:30.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.788+0000 7f0b29600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed376070 0x5622ed3bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.789+0000 7f0b29600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed376070 0x5622ed3bbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57668/0 (socket says 172.21.15.26:57668) 2024-01-15T17:58:30.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.789+0000 7f0b29600640 1 -- 172.21.15.26:0/4180780871 learned_addr learned my addr 172.21.15.26:0/4180780871 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:30.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.789+0000 7f0b29600640 1 -- 172.21.15.26:0/4180780871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed3757b0 msgr2=0x5622ed39b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:30.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.789+0000 7f0b29600640 1 --2- 172.21.15.26:0/4180780871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed3757b0 0x5622ed39b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.789+0000 7f0b29600640 1 -- 172.21.15.26:0/4180780871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3713f0 msgr2=0x5622ed39a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:30.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.789+0000 7f0b29600640 1 --2- 172.21.15.26:0/4180780871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3713f0 0x5622ed39a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.789+0000 7f0b29600640 1 -- 172.21.15.26:0/4180780871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622ed278b80 con 0x5622ed376070 2024-01-15T17:58:30.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.789+0000 7f0b29600640 1 --2- 172.21.15.26:0/4180780871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed376070 0x5622ed3bbcf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c000f60 tx=0x7f0b0c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5c949a3cd125a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:30.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.789+0000 7f0b1b400640 1 -- 172.21.15.26:0/4180780871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c005cb0 con 0x5622ed376070 2024-01-15T17:58:30.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b1b400640 1 -- 172.21.15.26:0/4180780871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c005e10 con 0x5622ed376070 2024-01-15T17:58:30.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b1b400640 1 -- 172.21.15.26:0/4180780871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c007760 con 0x5622ed376070 2024-01-15T17:58:30.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4180780871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed376070 msgr2=0x5622ed3bbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/4180780871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed376070 0x5622ed3bbcf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c000f60 tx=0x7f0b0c016040 comp rx=0 tx=0).stop 2024-01-15T17:58:30.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4180780871 shutdown_connections 2024-01-15T17:58:30.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/4180780871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3713f0 0x5622ed39a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/4180780871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed376070 0x5622ed3bbcf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/4180780871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed3757b0 0x5622ed39b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4180780871 >> 172.21.15.26:0/4180780871 conn(0x5622ed337c70 msgr2=0x5622ed3bac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:30.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4180780871 shutdown_connections 2024-01-15T17:58:30.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.790+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/4180780871 wait complete. 2024-01-15T17:58:30.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.791+0000 7f0b2f5fe740 1 -- start start 2024-01-15T17:58:30.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.791+0000 7f0b2f5fe740 1 Processor -- start 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.792+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3713f0 0x5622ed3a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.792+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3757b0 0x5622ed41bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.792+0000 7f0b2f5fe740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed376070 0x5622ed41d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.792+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ed3a3d70 con 0x5622ed3713f0 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.792+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622ed3de700 con 0x5622ed3757b0 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.792+0000 7f0b2f5fe740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622ed420400 con 0x5622ed376070 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.792+0000 7f0b2f5fe740 10 client.0 _ll_register_callbacks cb 0x7f0b1c059210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b2a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3713f0 0x5622ed3a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b2a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3713f0 0x5622ed3a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57676/0 (socket says 172.21.15.26:57676) 2024-01-15T17:58:30.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b2a000640 1 -- 172.21.15.26:0/2081129617 learned_addr learned my addr 172.21.15.26:0/2081129617 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b29600640 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3757b0 0x5622ed41bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b2a000640 1 -- 172.21.15.26:0/2081129617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed376070 msgr2=0x5622ed41d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b2a000640 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed376070 0x5622ed41d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b2a000640 1 -- 172.21.15.26:0/2081129617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3757b0 msgr2=0x5622ed41bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b2a000640 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3757b0 0x5622ed41bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b2a000640 1 -- 172.21.15.26:0/2081129617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b04003520 con 0x5622ed3713f0 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b29600640 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3757b0 0x5622ed41bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b2a000640 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3713f0 0x5622ed3a4190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00cfd0 tx=0x7f0b1c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570f22e4e93ca518 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.793+0000 7f0b13400640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b13400640 1 -- 172.21.15.26:0/2081129617 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0892f0 con 0x5622ed3713f0 2024-01-15T17:58:30.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b13400640 1 -- 172.21.15.26:0/2081129617 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0b1c0616c0 con 0x5622ed3713f0 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2081129617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b14031730 con 0x5622ed3713f0 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b13400640 1 -- 172.21.15.26:0/2081129617 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b1c058f00 con 0x5622ed3713f0 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b2f5fe740 10 client.5964 fetch_fsmap learned FSMap version 25 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2081129617 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5622ed29e170 con 0x5622ed3713f0 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b13400640 1 -- 172.21.15.26:0/2081129617 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b14031730 con 0x5622ed3713f0 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b13400640 1 -- 172.21.15.26:0/2081129617 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0b1c082b20 con 0x5622ed3713f0 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b2f5fe740 10 client.5964 fetch_fsmap finished waiting for FSMap version 25 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b2f5fe740 10 client.5964 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b2f5fe740 20 client.5964 populate_metadata read hostname 'smithi026' 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] conn(0x5622ed39a6c0 0x5622ed39aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b2f5fe740 4 client.5964 mds_command: new command op to 5802 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.794+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2081129617 --> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5622ecfdd6c0 con 0x5622ed39a6c0 2024-01-15T17:58:30.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.795+0000 7f0b29600640 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] conn(0x5622ed39a6c0 0x5622ed39aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.795+0000 7f0b29600640 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] conn(0x5622ed39a6c0 0x5622ed39aaa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4029edea00decc0e server_cookie=2f6c88cd3f612633 in_seq=0 out_seq=0 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.795+0000 7f0b13400640 10 client.5964 ms_handle_connect on v2:172.21.15.26:6833/1948779240 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.795+0000 7f0b13400640 1 -- 172.21.15.26:0/2081129617 <== mds.0 v2:172.21.15.26:6833/1948779240 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5622ecfdd6c0 con 0x5622ed39a6c0 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.795+0000 7f0b13400640 10 client.5964 handle_command_reply: tid=0 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 client.5964 shutdown 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2081129617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3713f0 msgr2=0x5622ed3a4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3713f0 0x5622ed3a4190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00cfd0 tx=0x7f0b1c000f40 comp rx=0 tx=0).stop 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2081129617 shutdown_connections 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.26:6833/1948779240,v1:172.21.15.26:6835/1948779240] conn(0x5622ed39a6c0 0x5622ed39aaa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x5622ed376070 0x5622ed41d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x5622ed3757b0 0x5622ed41bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 --2- 172.21.15.26:0/2081129617 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x5622ed3713f0 0x5622ed3a4190 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2081129617 >> 172.21.15.26:0/2081129617 conn(0x5622ed337c70 msgr2=0x5622ed39e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b13400640 0 client.5964 ms_handle_reset on v2:172.21.15.26:6833/1948779240 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2081129617 shutdown_connections 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 1 -- 172.21.15.26:0/2081129617 wait complete. 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.796+0000 7f0b2f5fe740 20 client.5964 trim_cache size 0 max 16384 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5802, 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-15T17:58:30.802 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-15T17:58:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 11.815510948, 2024-01-15T17:58:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 25, 2024-01-15T17:58:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 47, 2024-01-15T17:58:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 47, 2024-01-15T17:58:30.803 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 17.233523507000001 2024-01-15T17:58:30.803 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 -- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0afc06b500 msgr2=0x7f0afc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0afc06b500 0x7f0afc06d9c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0b24161260 tx=0x7f0b0c009040 comp rx=0 tx=0).stop 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 -- 172.21.15.26:0/4212466434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b240cf9f0 msgr2=0x7f0b2415b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b240cf9f0 0x7f0b2415b410 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00efc0 tx=0x7f0b1c00c490 comp rx=0 tx=0).stop 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 -- 172.21.15.26:0/4212466434 shutdown_connections 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b240cfea0 0x7f0b240b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0afc06b500 0x7f0afc06d9c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b24160920 0x7f0b2415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 --2- 172.21.15.26:0/4212466434 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b240cf9f0 0x7f0b2415b410 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 -- 172.21.15.26:0/4212466434 >> 172.21.15.26:0/4212466434 conn(0x7f0b240d3090 msgr2=0x7f0b241613f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 -- 172.21.15.26:0/4212466434 shutdown_connections 2024-01-15T17:58:30.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:30.797+0000 7f0b2b400640 1 -- 172.21.15.26:0/4212466434 wait complete. 2024-01-15T17:58:30.818 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T17:58:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.034+0000 7f4215c00640 1 Processor -- start 2024-01-15T17:58:31.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.034+0000 7f4215c00640 1 -- start start 2024-01-15T17:58:31.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.036+0000 7f4215c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f42100cd960 0x7f421015c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.036+0000 7f4215c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f421006f550 0x7f421006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.036+0000 7f4215c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421006eb00 con 0x7f421006a3e0 2024-01-15T17:58:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.036+0000 7f4215c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421006f0c0 con 0x7f42100cd960 2024-01-15T17:58:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.036+0000 7f4215c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421006ab80 con 0x7f421006f550 2024-01-15T17:58:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.037+0000 7f4207e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f421006a3e0 0x7f42100cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46722/0 (socket says 172.21.15.26:46722) 2024-01-15T17:58:31.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.037+0000 7f4207e00640 1 -- 172.21.15.26:0/2136627185 learned_addr learned my addr 172.21.15.26:0/2136627185 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.037+0000 7f420f400640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f421006f550 0x7f421006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.037+0000 7f420ea00640 1 --2- 172.21.15.26:0/2136627185 >> v2:172.21.15.124:3300/0 conn(0x7f42100cd960 0x7f421015c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.037+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554263931 0 0) 0x7f421006eb00 con 0x7f421006a3e0 2024-01-15T17:58:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.037+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f0003520 con 0x7f421006a3e0 2024-01-15T17:58:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1469680796 0 0) 0x7f41f0003520 con 0x7f421006a3e0 2024-01-15T17:58:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f421006f550 msgr2=0x7f421006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f421006f550 0x7f421006f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 >> v2:172.21.15.124:3300/0 conn(0x7f42100cd960 msgr2=0x7f421015c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 --2- 172.21.15.26:0/2136627185 >> v2:172.21.15.124:3300/0 conn(0x7f42100cd960 0x7f421015c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421006ae60 con 0x7f421006a3e0 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f41f8001670 con 0x7f421006a3e0 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 >> v1:172.21.15.26:6789/0 conn(0x7f421006a3e0 legacy=0x7f42100cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f421006f550 0x7f41f0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003ef0 0x7f41f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41f0004d00 0x7f41f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421006f0c0 con 0x7f41f0003ef0 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421006ab80 con 0x7f41f0004d00 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f000b9d0 con 0x7f421006f550 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f4207e00640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003ef0 0x7f41f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420ea00640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f421006f550 0x7f41f0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.038+0000 7f420f400640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41f0004d00 0x7f41f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:31.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.039+0000 7f4207e00640 1 -- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f421006f550 msgr2=0x7f41f0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.039+0000 7f4207e00640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f421006f550 0x7f41f0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.039+0000 7f4207e00640 1 -- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41f0004d00 msgr2=0x7f41f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.039+0000 7f4207e00640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41f0004d00 0x7f41f0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.039+0000 7f4207e00640 1 -- 172.21.15.26:0/2136627185 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f0003520 con 0x7f41f0003ef0 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.040+0000 7f4207e00640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003ef0 0x7f41f00047c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c270 tx=0x7f41f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d962d4386b6325 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.040+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f802cae0 con 0x7f41f0003ef0 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.040+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f802cc40 con 0x7f41f0003ef0 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.040+0000 7f420e000640 1 -- 172.21.15.26:0/2136627185 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f8005af0 con 0x7f41f0003ef0 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.040+0000 7f4215c00640 1 -- 172.21.15.26:0/2136627185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003ef0 msgr2=0x7f41f00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.040+0000 7f4215c00640 1 --2- 172.21.15.26:0/2136627185 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f41f0003ef0 0x7f41f00047c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f41f800c270 tx=0x7f41f00057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.040+0000 7f420f400640 1 -- 172.21.15.26:0/2136627185 reap_dead start 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.041+0000 7f4215c00640 1 -- 172.21.15.26:0/2136627185 shutdown_connections 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.041+0000 7f4215c00640 1 -- 172.21.15.26:0/2136627185 >> 172.21.15.26:0/2136627185 conn(0x7f42100b1dd0 msgr2=0x7f42100b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:31.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.041+0000 7f4215c00640 1 -- 172.21.15.26:0/2136627185 shutdown_connections 2024-01-15T17:58:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.041+0000 7f4215c00640 1 -- 172.21.15.26:0/2136627185 wait complete. 2024-01-15T17:58:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.042+0000 7f4215c00640 1 Processor -- start 2024-01-15T17:58:31.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.043+0000 7f4215c00640 1 -- start start 2024-01-15T17:58:31.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.043+0000 7f4215c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41f0003ef0 0x7f4210158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.043+0000 7f4215c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4210140820 0x7f42100ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f4215c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4210140ca0 0x7f42100ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f4215c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421013fb00 con 0x7f4210140ca0 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f4215c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421013fde0 con 0x7f41f0003ef0 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f4215c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42101400c0 con 0x7f4210140820 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f420f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4210140ca0 0x7f42100ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f420f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4210140ca0 0x7f42100ade60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57690/0 (socket says 172.21.15.26:57690) 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f420f400640 1 -- 172.21.15.26:0/2097100773 learned_addr learned my addr 172.21.15.26:0/2097100773 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f4207e00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4210140820 0x7f42100ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f420ea00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41f0003ef0 0x7f4210158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f420f400640 1 -- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4210140820 msgr2=0x7f42100ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f420f400640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4210140820 0x7f42100ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f420f400640 1 -- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41f0003ef0 msgr2=0x7f4210158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f420f400640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41f0003ef0 0x7f4210158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.044+0000 7f420f400640 1 -- 172.21.15.26:0/2097100773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42101403a0 con 0x7f4210140ca0 2024-01-15T17:58:31.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.045+0000 7f420f400640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4210140ca0 0x7f42100ade60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f421006c7c0 tx=0x7f420800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.045+0000 7f4206a00640 1 -- 172.21.15.26:0/2097100773 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208017400 con 0x7f4210140ca0 2024-01-15T17:58:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.045+0000 7f4206a00640 1 -- 172.21.15.26:0/2097100773 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4208011040 con 0x7f4210140ca0 2024-01-15T17:58:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.045+0000 7f4206a00640 1 -- 172.21.15.26:0/2097100773 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42080177c0 con 0x7f4210140ca0 2024-01-15T17:58:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.045+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42101418b0 con 0x7f4210140ca0 2024-01-15T17:58:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.045+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421013f730 con 0x7f4210140ca0 2024-01-15T17:58:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.046+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4210141aa0 con 0x7f4210140ca0 2024-01-15T17:58:31.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.047+0000 7f4206a00640 1 -- 172.21.15.26:0/2097100773 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f4208017920 con 0x7f4210140ca0 2024-01-15T17:58:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.048+0000 7f4206a00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f41e806b530 0x7f41e806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:31.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.048+0000 7f4206a00640 1 -- 172.21.15.26:0/2097100773 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f420808a550 con 0x7f4210140ca0 2024-01-15T17:58:31.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.049+0000 7f420ea00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f41e806b530 0x7f41e806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:31.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.050+0000 7f420ea00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f41e806b530 0x7f41e806d9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4200009a10 tx=0x7f4200008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:31.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.053+0000 7f4206a00640 1 -- 172.21.15.26:0/2097100773 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4208054f20 con 0x7f4210140ca0 2024-01-15T17:58:31.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.286+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f42100675d0 con 0x7f4210140ca0 2024-01-15T17:58:31.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.287+0000 7f4206a00640 1 -- 172.21.15.26:0/2097100773 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4208058540 con 0x7f4210140ca0 2024-01-15T17:58:31.290 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T17:58:31.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f41e806b530 msgr2=0x7f41e806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:31.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f41e806b530 0x7f41e806d9f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4200009a10 tx=0x7f4200008040 comp rx=0 tx=0).stop 2024-01-15T17:58:31.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4210140ca0 msgr2=0x7f42100ade60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:31.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4210140ca0 0x7f42100ade60 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f421006c7c0 tx=0x7f420800f4a0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 shutdown_connections 2024-01-15T17:58:31.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4210140820 0x7f42100ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f41e806b530 0x7f41e806d9f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4210140ca0 0x7f42100ade60 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 --2- 172.21.15.26:0/2097100773 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f41f0003ef0 0x7f4210158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:31.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.289+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 >> 172.21.15.26:0/2097100773 conn(0x7f42100b1dd0 msgr2=0x7f4210159800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:31.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.290+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 shutdown_connections 2024-01-15T17:58:31.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:31.290+0000 7f4215c00640 1 -- 172.21.15.26:0/2097100773 wait complete. 2024-01-15T17:58:31.313 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T17:58:31.313 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T17:58:31.313 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-15T17:58:31.313 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T17:58:31.313 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T17:58:31.313 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T17:58:31.313 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T17:58:31.313 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T17:58:31.314 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T17:58:31.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:31.314 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-15T17:58:31.358 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:58:31.358 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:58:31.358 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:58:31.358 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42406sec preferred_lft 42406sec 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-01-15T17:58:31.359 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::a4e5:3ff:fe91:fcad/64 scope link 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::f464:8fff:fecd:58d2/64 scope link 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:31.360 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:58:31.361 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether fe:82:c8:bc:3f:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:58:31.361 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::fc82:c8ff:febc:3fd2/64 scope link 2024-01-15T17:58:31.361 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:31.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:31.361 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T17:58:31.494 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-15T17:58:31.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:31.499 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-15T17:58:31.552 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-15T17:58:31.552 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T17:58:31.552 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-15T17:58:31.552 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-15T17:58:31.552 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T17:58:31.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:31.553 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T17:58:31.554 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T17:58:31.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-15T17:58:31.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-15T17:58:31.554 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-15T17:58:31.554 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T17:58:31.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:31.752 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T17:58:31.803 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T17:58:31.803 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T17:58:31.804 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T17:58:31.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:31.804 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-15T17:58:31.900 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T17:58:31.900 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:58:31.900 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T17:58:31.957 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T17:58:31.957 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:58:31.957 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T17:58:32.007 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:58:32.104 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:58:32.162 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:58:32.162 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:58:32.162 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:58:32.162 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:58:32.162 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:58:32.163 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:58:32.164 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:58:32.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:32.165 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:58:32.221 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T17:58:32.221 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T17:58:32.226 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T17:58:32.303 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:58:32.348 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T17:58:32.348 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T17:58:32.348 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T17:58:32.348 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T17:58:32.348 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:58:32.349 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:58:32.350 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T17:58:32.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:32.351 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T17:58:32.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:2024-01-15T17:58:32.406+0000 7fdb06648480 -1 init, newargv = 0x55ff3ec17bf0 newargc=15 2024-01-15T17:58:32.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[52830]: starting ceph client 2024-01-15T17:58:32.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[52830]: starting fuse 2024-01-15T17:58:32.436 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T17:58:32.437 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-01-15T17:58:32.437 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.0")) 2024-01-15T17:58:32.438 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-01-15T17:58:32.646 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.0.52830.asok 2024-01-15T17:58:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T17:58:32.663 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52830.asok 2024-01-15T17:58:32.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:32.664 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52830.asok status 2024-01-15T17:58:32.909 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:58:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-01-15T17:58:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T17:58:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T17:58:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "0", 2024-01-15T17:58:32.909 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "52830", 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5976, 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-01-15T17:58:32.910 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 5976 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1784315913 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1784315913 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.5976 192.168.0.1:0/1784315913", 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/1784315913", 2024-01-15T17:58:32.911 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-01-15T17:58:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 25, 2024-01-15T17:58:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 47, 2024-01-15T17:58:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-01-15T17:58:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-01-15T17:58:32.912 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-01-15T17:58:32.912 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:58:32.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:32.925 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-01-15T17:58:32.925 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T17:58:32.980 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-01-15T17:58:32.981 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T17:58:32.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:32.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T17:58:33.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T17:58:33.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.304+0000 7f34c9e00640 1 Processor -- start 2024-01-15T17:58:33.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.304+0000 7f34c9e00640 1 -- start start 2024-01-15T17:58:33.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.306+0000 7f34c9e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34c40b3ae0 0x7f34c415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:33.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.306+0000 7f34c9e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34c40cf290 0x7f34c40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:33.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.306+0000 7f34c9e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c4070c40 con 0x7f34c40ae5a0 2024-01-15T17:58:33.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.306+0000 7f34c9e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c4071450 con 0x7f34c40cf290 2024-01-15T17:58:33.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.306+0000 7f34c9e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c40cfc60 con 0x7f34c40b3ae0 2024-01-15T17:58:33.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.306+0000 7f34c3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34c40cf290 0x7f34c40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:33.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.306+0000 7f34c2a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f34c40ae5a0 0x7f34c406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42852/0 (socket says 172.21.15.26:42852) 2024-01-15T17:58:33.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.306+0000 7f34c2a00640 1 -- 172.21.15.26:0/968195694 learned_addr learned my addr 172.21.15.26:0/968195694 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:33.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.307+0000 7f34c2000640 1 --2- 172.21.15.26:0/968195694 >> v2:172.21.15.124:3300/0 conn(0x7f34c40b3ae0 0x7f34c415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:33.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.307+0000 7f34c3400640 1 -- 172.21.15.26:0/968195694 >> v2:172.21.15.124:3300/0 conn(0x7f34c40b3ae0 msgr2=0x7f34c415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:33.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.307+0000 7f34c3400640 1 --2- 172.21.15.26:0/968195694 >> v2:172.21.15.124:3300/0 conn(0x7f34c40b3ae0 0x7f34c415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.307+0000 7f34c3400640 1 -- 172.21.15.26:0/968195694 >> v1:172.21.15.26:6789/0 conn(0x7f34c40ae5a0 legacy=0x7f34c406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.307+0000 7f34c3400640 1 -- 172.21.15.26:0/968195694 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c406c470 con 0x7f34c40cf290 2024-01-15T17:58:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.307+0000 7f34c2000640 1 --2- 172.21.15.26:0/968195694 >> v2:172.21.15.124:3300/0 conn(0x7f34c40b3ae0 0x7f34c415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.308+0000 7f34c3400640 1 --2- 172.21.15.26:0/968195694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40cf290 0x7f34c40cf720 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f34c40cfea0 tx=0x7f34bc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8276e1e28bf68457 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:33.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.308+0000 7f34c1600640 1 -- 172.21.15.26:0/968195694 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc030610 con 0x7f34c40cf290 2024-01-15T17:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.308+0000 7f34c1600640 1 -- 172.21.15.26:0/968195694 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34bc030770 con 0x7f34c40cf290 2024-01-15T17:58:33.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.308+0000 7f34c1600640 1 -- 172.21.15.26:0/968195694 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc030b00 con 0x7f34c40cf290 2024-01-15T17:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.309+0000 7f34c9e00640 1 -- 172.21.15.26:0/968195694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40cf290 msgr2=0x7f34c40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.309+0000 7f34c9e00640 1 --2- 172.21.15.26:0/968195694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40cf290 0x7f34c40cf720 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f34c40cfea0 tx=0x7f34bc02b960 comp rx=0 tx=0).stop 2024-01-15T17:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.309+0000 7f34c9e00640 1 -- 172.21.15.26:0/968195694 shutdown_connections 2024-01-15T17:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.309+0000 7f34c9e00640 1 --2- 172.21.15.26:0/968195694 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40cf290 0x7f34c40cf720 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.309+0000 7f34c9e00640 1 --2- 172.21.15.26:0/968195694 >> v2:172.21.15.124:3300/0 conn(0x7f34c40b3ae0 0x7f34c415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.309+0000 7f34c9e00640 1 -- 172.21.15.26:0/968195694 >> 172.21.15.26:0/968195694 conn(0x7f34c406a800 msgr2=0x7f34c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.309+0000 7f34c9e00640 1 -- 172.21.15.26:0/968195694 shutdown_connections 2024-01-15T17:58:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.309+0000 7f34c9e00640 1 -- 172.21.15.26:0/968195694 wait complete. 2024-01-15T17:58:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.311+0000 7f34c9e00640 1 Processor -- start 2024-01-15T17:58:33.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.311+0000 7f34c9e00640 1 -- start start 2024-01-15T17:58:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.312+0000 7f34c9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40ae5a0 0x7f34c4156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.312+0000 7f34c9e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34c40b3ae0 0x7f34c41518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.312+0000 7f34c9e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34c40cf290 0x7f34c4157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.312+0000 7f34c9e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c4152990 con 0x7f34c40ae5a0 2024-01-15T17:58:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.312+0000 7f34c9e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c4159590 con 0x7f34c40b3ae0 2024-01-15T17:58:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.312+0000 7f34c9e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c41572e0 con 0x7f34c40cf290 2024-01-15T17:58:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.313+0000 7f34c2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40ae5a0 0x7f34c4156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.313+0000 7f34c2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40ae5a0 0x7f34c4156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46450/0 (socket says 172.21.15.26:46450) 2024-01-15T17:58:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.313+0000 7f34c2a00640 1 -- 172.21.15.26:0/1624916226 learned_addr learned my addr 172.21.15.26:0/1624916226 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.313+0000 7f34c2a00640 1 -- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34c40cf290 msgr2=0x7f34c4157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.313+0000 7f34c2a00640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34c40cf290 0x7f34c4157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.313+0000 7f34c2a00640 1 -- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34c40b3ae0 msgr2=0x7f34c41518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.313+0000 7f34c2a00640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34c40b3ae0 0x7f34c41518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.313+0000 7f34c2a00640 1 -- 172.21.15.26:0/1624916226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc009400 con 0x7f34c40ae5a0 2024-01-15T17:58:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.313+0000 7f34c2a00640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40ae5a0 0x7f34c4156da0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f34b400cba0 tx=0x7f34b400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.314+0000 7f34bb400640 1 -- 172.21.15.26:0/1624916226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b400ad40 con 0x7f34c40ae5a0 2024-01-15T17:58:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.314+0000 7f34bb400640 1 -- 172.21.15.26:0/1624916226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b4012400 con 0x7f34c40ae5a0 2024-01-15T17:58:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.314+0000 7f34bb400640 1 -- 172.21.15.26:0/1624916226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b400aea0 con 0x7f34c40ae5a0 2024-01-15T17:58:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.314+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c41597a0 con 0x7f34c40ae5a0 2024-01-15T17:58:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.314+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c4159d60 con 0x7f34c40ae5a0 2024-01-15T17:58:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.315+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c41574f0 con 0x7f34c40ae5a0 2024-01-15T17:58:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.316+0000 7f34bb400640 1 -- 172.21.15.26:0/1624916226 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f34b4007070 con 0x7f34c40ae5a0 2024-01-15T17:58:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.317+0000 7f34bb400640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f349c06b500 0x7f349c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.317+0000 7f34bb400640 1 -- 172.21.15.26:0/1624916226 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f34b4089f00 con 0x7f34c40ae5a0 2024-01-15T17:58:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.318+0000 7f34c2000640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f349c06b500 0x7f349c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.319+0000 7f34c2000640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f349c06b500 0x7f349c06d9c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f34ac006fd0 tx=0x7f34ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:33.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.322+0000 7f34bb400640 1 -- 172.21.15.26:0/1624916226 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34b4054a90 con 0x7f34c40ae5a0 2024-01-15T17:58:33.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.557+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f34c40675d0 con 0x7f34c40ae5a0 2024-01-15T17:58:33.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.558+0000 7f34bb400640 1 -- 172.21.15.26:0/1624916226 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f34b4017020 con 0x7f34c40ae5a0 2024-01-15T17:58:33.561 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f349c06b500 msgr2=0x7f349c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f349c06b500 0x7f349c06d9c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f34ac006fd0 tx=0x7f34ac008040 comp rx=0 tx=0).stop 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40ae5a0 msgr2=0x7f34c4156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40ae5a0 0x7f34c4156da0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f34b400cba0 tx=0x7f34b400a430 comp rx=0 tx=0).stop 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 shutdown_connections 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34c40cf290 0x7f34c4157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f349c06b500 0x7f349c06d9c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34c40b3ae0 0x7f34c41518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 --2- 172.21.15.26:0/1624916226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c40ae5a0 0x7f34c4156da0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:33.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.560+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 >> 172.21.15.26:0/1624916226 conn(0x7f34c406a800 msgr2=0x7f34c40b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.561+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 shutdown_connections 2024-01-15T17:58:33.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:33.561+0000 7f34c9e00640 1 -- 172.21.15.26:0/1624916226 wait complete. 2024-01-15T17:58:33.584 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T17:58:33.584 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T17:58:33.584 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2024-01-15T17:58:33.584 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T17:58:33.584 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T17:58:33.584 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T17:58:33.585 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T17:58:33.585 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T17:58:33.585 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T17:58:33.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:33.585 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2024-01-15T17:58:33.642 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T17:58:33.642 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T17:58:33.642 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T17:58:33.642 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:33.642 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T17:58:33.642 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:33.642 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:58:33.642 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42402sec preferred_lft 42402sec 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:33.643 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4c2d:94ff:fe47:f54c/64 scope link 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 26:4f:4f:5b:0b:31 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::244f:4fff:fe5b:b31/64 scope link 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::2070:16ff:fe92:4a4a/64 scope link 2024-01-15T17:58:33.644 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T17:58:33.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:33.645 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T17:58:33.735 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T17:58:33.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:33.740 DEBUG:teuthology.orchestra.run.smithi124:> route 2024-01-15T17:58:33.792 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2024-01-15T17:58:33.792 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T17:58:33.793 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T17:58:33.793 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T17:58:33.793 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T17:58:33.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:33.794 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T17:58:33.794 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T17:58:33.794 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T17:58:33.794 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T17:58:33.794 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T17:58:33.794 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T17:58:33.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:33.976 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T17:58:34.024 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T17:58:34.024 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T17:58:34.025 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T17:58:34.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:34.025 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2024-01-15T17:58:34.111 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T17:58:34.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:58:34.111 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T17:58:34.165 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T17:58:34.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T17:58:34.165 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T17:58:34.209 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T17:58:34.350 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:58:34.406 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T17:58:34.406 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T17:58:34.406 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T17:58:34.406 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T17:58:34.406 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T17:58:34.406 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:58:34.407 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:58:34.408 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:58:34.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:34.409 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T17:58:34.459 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T17:58:34.459 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T17:58:34.464 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T17:58:34.545 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T17:58:34.587 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T17:58:34.588 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T17:58:34.589 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T17:58:34.589 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T17:58:34.589 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T17:58:34.589 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T17:58:34.589 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:58:34.589 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T17:58:34.589 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:58:34.589 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T17:58:34.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:34.590 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T17:58:34.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:2024-01-15T17:58:34.639+0000 7ff1ccdd7480 -1 init, newargv = 0x5639e369c990 newargc=15 2024-01-15T17:58:34.642 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[43918]: starting ceph client 2024-01-15T17:58:34.654 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[43918]: starting fuse 2024-01-15T17:58:34.672 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T17:58:34.672 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T17:58:34.672 DEBUG:teuthology.orchestra.run.smithi124:> sudo stdin-killer -- python3 -c ' 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> import glob 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> import re 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> import os 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> import subprocess 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> def _find_admin_socket(client_name): 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> files = glob.glob(asok_path) 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> # Given a non-glob path, it better be there 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> if "*" not in asok_path: 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> assert(len(files) == 1) 2024-01-15T17:58:34.673 DEBUG:teuthology.orchestra.run.smithi124:> return files[0] 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> for f in files: 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> contents = proc_f.read() 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> if mountpoint in contents: 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> return f 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> print(_find_admin_socket("client.1")) 2024-01-15T17:58:34.674 DEBUG:teuthology.orchestra.run.smithi124:> ' 2024-01-15T17:58:34.872 INFO:teuthology.orchestra.run.smithi124.stdout:/var/run/ceph/ceph-client.1.43918.asok 2024-01-15T17:58:34.876 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T17:58:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T17:58:34.889 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.43918.asok 2024-01-15T17:58:34.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:34.889 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.43918.asok status 2024-01-15T17:58:35.117 INFO:teuthology.orchestra.run.smithi124.stdout:{ 2024-01-15T17:58:35.117 INFO:teuthology.orchestra.run.smithi124.stdout: "metadata": { 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "entity_id": "1", 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "hostname": "smithi124", 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "pid": "43918", 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "root": "/" 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_count": 0, 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_pinned_count": 0, 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "id": 5356, 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "inst": { 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "name": { 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "client", 2024-01-15T17:58:35.118 INFO:teuthology.orchestra.run.smithi124.stdout: "num": 5356 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 3139449247 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: } 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 3139449247 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "inst_str": "client.5356 192.168.0.1:0/3139449247", 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "addr_str": "192.168.0.1:0/3139449247", 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "inode_count": 1, 2024-01-15T17:58:35.119 INFO:teuthology.orchestra.run.smithi124.stdout: "mds_epoch": 25, 2024-01-15T17:58:35.120 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch": 47, 2024-01-15T17:58:35.120 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch_barrier": 0, 2024-01-15T17:58:35.120 INFO:teuthology.orchestra.run.smithi124.stdout: "blocklisted": false, 2024-01-15T17:58:35.120 INFO:teuthology.orchestra.run.smithi124.stdout: "fs_name": "cephfs" 2024-01-15T17:58:35.120 INFO:teuthology.orchestra.run.smithi124.stdout:} 2024-01-15T17:58:35.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:35.132 DEBUG:teuthology.orchestra.run.smithi124:> stat --file-system '--printf=%T 2024-01-15T17:58:35.132 DEBUG:teuthology.orchestra.run.smithi124:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T17:58:35.180 INFO:teuthology.orchestra.run.smithi124.stdout:fuseblk 2024-01-15T17:58:35.180 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T17:58:35.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:35.181 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T17:58:35.272 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T17:58:35.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.508+0000 7fdf96200640 1 Processor -- start 2024-01-15T17:58:35.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.509+0000 7fdf96200640 1 -- start start 2024-01-15T17:58:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.510+0000 7fdf96200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdf90143c40 0x7fdf90071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.510+0000 7fdf96200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf900ccc10 0x7fdf9006d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.510+0000 7fdf96200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf90070e50 con 0x7fdf900717d0 2024-01-15T17:58:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.510+0000 7fdf96200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf900723c0 con 0x7fdf90143c40 2024-01-15T17:58:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.510+0000 7fdf96200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90144020 con 0x7fdf900ccc10 2024-01-15T17:58:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.511+0000 7fdf8fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdf900717d0 0x7fdf900cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42866/0 (socket says 172.21.15.26:42866) 2024-01-15T17:58:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.511+0000 7fdf8fe00640 1 -- 172.21.15.26:0/280896836 learned_addr learned my addr 172.21.15.26:0/280896836 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:35.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.511+0000 7fdf95800640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf900ccc10 0x7fdf9006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:35.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.511+0000 7fdf94e00640 1 --2- 172.21.15.26:0/280896836 >> v2:172.21.15.124:3300/0 conn(0x7fdf90143c40 0x7fdf90071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:35.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.511+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2026320041 0 0) 0x7fdf90070e50 con 0x7fdf900717d0 2024-01-15T17:58:35.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.511+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf70003520 con 0x7fdf900717d0 2024-01-15T17:58:35.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3037867252 0 0) 0x7fdf70003520 con 0x7fdf900717d0 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf900ccc10 msgr2=0x7fdf9006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf900ccc10 0x7fdf9006d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 >> v2:172.21.15.124:3300/0 conn(0x7fdf90143c40 msgr2=0x7fdf90071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 --2- 172.21.15.26:0/280896836 >> v2:172.21.15.124:3300/0 conn(0x7fdf90143c40 0x7fdf90071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf90071e00 con 0x7fdf900717d0 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fdf78001670 con 0x7fdf900717d0 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 >> v1:172.21.15.26:6789/0 conn(0x7fdf900717d0 legacy=0x7fdf900cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf900ccc10 0x7fdf70003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf70003f40 0x7fdf700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf70004d00 0x7fdf70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900723c0 con 0x7fdf70003f40 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf90144020 con 0x7fdf70004d00 2024-01-15T17:58:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7000ba30 con 0x7fdf900ccc10 2024-01-15T17:58:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.512+0000 7fdf8fe00640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf70003f40 0x7fdf700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.513+0000 7fdf95800640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf70004d00 0x7fdf70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.513+0000 7fdf94e00640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf900ccc10 0x7fdf70003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.513+0000 7fdf8fe00640 1 -- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf900ccc10 msgr2=0x7fdf70003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.513+0000 7fdf8fe00640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf900ccc10 0x7fdf70003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.513+0000 7fdf8fe00640 1 -- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf70004d00 msgr2=0x7fdf70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.513+0000 7fdf8fe00640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf70004d00 0x7fdf70005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.513+0000 7fdf8fe00640 1 -- 172.21.15.26:0/280896836 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf70003520 con 0x7fdf70003f40 2024-01-15T17:58:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.513+0000 7fdf94e00640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf900ccc10 0x7fdf70003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.514+0000 7fdf8fe00640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf70003f40 0x7fdf700047c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c160 tx=0x7fdf700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da8763b07342731 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.514+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78005ad0 con 0x7fdf70003f40 2024-01-15T17:58:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.514+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf78005c30 con 0x7fdf70003f40 2024-01-15T17:58:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.514+0000 7fdf8f400640 1 -- 172.21.15.26:0/280896836 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7802c810 con 0x7fdf70003f40 2024-01-15T17:58:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.515+0000 7fdf96200640 1 -- 172.21.15.26:0/280896836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf70003f40 msgr2=0x7fdf700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.515+0000 7fdf96200640 1 --2- 172.21.15.26:0/280896836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf70003f40 0x7fdf700047c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800c160 tx=0x7fdf700057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.515+0000 7fdf95800640 1 -- 172.21.15.26:0/280896836 reap_dead start 2024-01-15T17:58:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.515+0000 7fdf96200640 1 -- 172.21.15.26:0/280896836 shutdown_connections 2024-01-15T17:58:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.515+0000 7fdf96200640 1 -- 172.21.15.26:0/280896836 >> 172.21.15.26:0/280896836 conn(0x7fdf9006a800 msgr2=0x7fdf9006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.515+0000 7fdf96200640 1 -- 172.21.15.26:0/280896836 shutdown_connections 2024-01-15T17:58:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.515+0000 7fdf96200640 1 -- 172.21.15.26:0/280896836 wait complete. 2024-01-15T17:58:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.517+0000 7fdf96200640 1 Processor -- start 2024-01-15T17:58:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.517+0000 7fdf96200640 1 -- start start 2024-01-15T17:58:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf96200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf70003f40 0x7fdf900ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf96200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900cb2a0 0x7fdf9013b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf96200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf9015a510 0x7fdf9015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf96200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9013c280 con 0x7fdf900cb2a0 2024-01-15T17:58:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf96200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9013c560 con 0x7fdf9015a510 2024-01-15T17:58:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf96200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9013c840 con 0x7fdf70003f40 2024-01-15T17:58:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf8fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900cb2a0 0x7fdf9013b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf8fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900cb2a0 0x7fdf9013b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46468/0 (socket says 172.21.15.26:46468) 2024-01-15T17:58:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf8fe00640 1 -- 172.21.15.26:0/3679266969 learned_addr learned my addr 172.21.15.26:0/3679266969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.518+0000 7fdf94e00640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf70003f40 0x7fdf900ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.519+0000 7fdf95800640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf9015a510 0x7fdf9015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.519+0000 7fdf8fe00640 1 -- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf70003f40 msgr2=0x7fdf900ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.519+0000 7fdf8fe00640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf70003f40 0x7fdf900ab150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.519+0000 7fdf8fe00640 1 -- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf9015a510 msgr2=0x7fdf9015a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.519+0000 7fdf8fe00640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf9015a510 0x7fdf9015a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.519+0000 7fdf8fe00640 1 -- 172.21.15.26:0/3679266969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf900a8560 con 0x7fdf900cb2a0 2024-01-15T17:58:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.519+0000 7fdf8fe00640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900cb2a0 0x7fdf9013b910 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdf78000fb0 tx=0x7fdf78000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.519+0000 7fdf8cc00640 1 -- 172.21.15.26:0/3679266969 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf78039070 con 0x7fdf900cb2a0 2024-01-15T17:58:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.520+0000 7fdf8cc00640 1 -- 172.21.15.26:0/3679266969 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7802dcb0 con 0x7fdf900cb2a0 2024-01-15T17:58:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.520+0000 7fdf8cc00640 1 -- 172.21.15.26:0/3679266969 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780072b0 con 0x7fdf900cb2a0 2024-01-15T17:58:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.520+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf900a8840 con 0x7fdf900cb2a0 2024-01-15T17:58:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.520+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf900ab690 con 0x7fdf900cb2a0 2024-01-15T17:58:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.521+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf900ab820 con 0x7fdf900cb2a0 2024-01-15T17:58:35.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.522+0000 7fdf8cc00640 1 -- 172.21.15.26:0/3679266969 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fdf78032070 con 0x7fdf900cb2a0 2024-01-15T17:58:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.523+0000 7fdf8cc00640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf6806b500 0x7fdf6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.523+0000 7fdf8cc00640 1 -- 172.21.15.26:0/3679266969 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdf7803c040 con 0x7fdf900cb2a0 2024-01-15T17:58:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.523+0000 7fdf94e00640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf6806b500 0x7fdf6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.525+0000 7fdf94e00640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf6806b500 0x7fdf6806d9c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdf80003be0 tx=0x7fdf8000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:35.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.528+0000 7fdf8cc00640 1 -- 172.21.15.26:0/3679266969 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf78075b60 con 0x7fdf900cb2a0 2024-01-15T17:58:35.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.762+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdf90067450 con 0x7fdf900cb2a0 2024-01-15T17:58:35.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.938+0000 7fdf8cc00640 1 -- 172.21.15.26:0/3679266969 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 136+0+0 (secure 0 0 0) 0x7fdf78079180 con 0x7fdf900cb2a0 2024-01-15T17:58:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.942+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf6806b500 msgr2=0x7fdf6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.942+0000 7fdf96200640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf6806b500 0x7fdf6806d9c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdf80003be0 tx=0x7fdf8000b040 comp rx=0 tx=0).stop 2024-01-15T17:58:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.942+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900cb2a0 msgr2=0x7fdf9013b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:35.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.942+0000 7fdf96200640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900cb2a0 0x7fdf9013b910 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdf78000fb0 tx=0x7fdf78000fe0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.942+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 shutdown_connections 2024-01-15T17:58:35.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.942+0000 7fdf96200640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf6806b500 0x7fdf6806d9c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.942+0000 7fdf96200640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf9015a510 0x7fdf9015a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.942+0000 7fdf96200640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf900cb2a0 0x7fdf9013b910 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.942+0000 7fdf96200640 1 --2- 172.21.15.26:0/3679266969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf70003f40 0x7fdf900ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:35.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.943+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 >> 172.21.15.26:0/3679266969 conn(0x7fdf9006a800 msgr2=0x7fdf901403d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:35.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.943+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 shutdown_connections 2024-01-15T17:58:35.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:35.943+0000 7fdf96200640 1 -- 172.21.15.26:0/3679266969 wait complete. 2024-01-15T17:58:35.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T17:58:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.190+0000 7f099a800640 1 Processor -- start 2024-01-15T17:58:36.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.191+0000 7f099a800640 1 -- start start 2024-01-15T17:58:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.192+0000 7f099a800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0994160920 0x7f09940b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.192+0000 7f099a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09940cf9f0 0x7f09940cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.192+0000 7f099a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0994161010 0x7f09941614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.192+0000 7f099a800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09940d03e0 con 0x7f09940cf9f0 2024-01-15T17:58:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.192+0000 7f099a800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09940d0a10 con 0x7f0994160920 2024-01-15T17:58:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.192+0000 7f099a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09940cf080 con 0x7f0994161010 2024-01-15T17:58:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0998a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09940cf9f0 0x7f09940cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0998a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09940cf9f0 0x7f09940cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46472/0 (socket says 172.21.15.26:46472) 2024-01-15T17:58:36.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0998a00640 1 -- 172.21.15.26:0/3887638163 learned_addr learned my addr 172.21.15.26:0/3887638163 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:36.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0999e00640 1 --2- 172.21.15.26:0/3887638163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0994161010 0x7f09941614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0999400640 1 --2- 172.21.15.26:0/3887638163 >> v2:172.21.15.124:3300/0 conn(0x7f0994160920 0x7f09940b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0998a00640 1 -- 172.21.15.26:0/3887638163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0994161010 msgr2=0x7f09941614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:36.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0998a00640 1 --2- 172.21.15.26:0/3887638163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0994161010 0x7f09941614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0998a00640 1 -- 172.21.15.26:0/3887638163 >> v2:172.21.15.124:3300/0 conn(0x7f0994160920 msgr2=0x7f09940b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:36.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0998a00640 1 --2- 172.21.15.26:0/3887638163 >> v2:172.21.15.124:3300/0 conn(0x7f0994160920 0x7f09940b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0998a00640 1 -- 172.21.15.26:0/3887638163 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09940c6be0 con 0x7f09940cf9f0 2024-01-15T17:58:36.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0999400640 1 --2- 172.21.15.26:0/3887638163 >> v2:172.21.15.124:3300/0 conn(0x7f0994160920 0x7f09940b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:36.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.193+0000 7f0999e00640 1 --2- 172.21.15.26:0/3887638163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0994161010 0x7f09941614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:36.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.195+0000 7f0998a00640 1 --2- 172.21.15.26:0/3887638163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09940cf9f0 0x7f09940cfea0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f099408c060 tx=0x7f098402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f43c81138fa31e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:36.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.195+0000 7f0993e00640 1 -- 172.21.15.26:0/3887638163 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098402ca70 con 0x7f09940cf9f0 2024-01-15T17:58:36.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.195+0000 7f0993e00640 1 -- 172.21.15.26:0/3887638163 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098402cbd0 con 0x7f09940cf9f0 2024-01-15T17:58:36.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.195+0000 7f0993e00640 1 -- 172.21.15.26:0/3887638163 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840343f0 con 0x7f09940cf9f0 2024-01-15T17:58:36.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.196+0000 7f099a800640 1 -- 172.21.15.26:0/3887638163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09940cf9f0 msgr2=0x7f09940cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.196+0000 7f099a800640 1 --2- 172.21.15.26:0/3887638163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09940cf9f0 0x7f09940cfea0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f099408c060 tx=0x7f098402bed0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.196+0000 7f099a800640 1 -- 172.21.15.26:0/3887638163 shutdown_connections 2024-01-15T17:58:36.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.196+0000 7f099a800640 1 --2- 172.21.15.26:0/3887638163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0994161010 0x7f09941614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.196+0000 7f099a800640 1 --2- 172.21.15.26:0/3887638163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09940cf9f0 0x7f09940cfea0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.196+0000 7f099a800640 1 --2- 172.21.15.26:0/3887638163 >> v2:172.21.15.124:3300/0 conn(0x7f0994160920 0x7f09940b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.196+0000 7f099a800640 1 -- 172.21.15.26:0/3887638163 >> 172.21.15.26:0/3887638163 conn(0x7f09940d3090 msgr2=0x7f09940d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:36.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.196+0000 7f099a800640 1 -- 172.21.15.26:0/3887638163 shutdown_connections 2024-01-15T17:58:36.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.196+0000 7f099a800640 1 -- 172.21.15.26:0/3887638163 wait complete. 2024-01-15T17:58:36.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.198+0000 7f099a800640 1 Processor -- start 2024-01-15T17:58:36.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.198+0000 7f099a800640 1 -- start start 2024-01-15T17:58:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f099a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09940cf9f0 0x7f099415d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f099a800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0994160920 0x7f09941f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f099a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0994161010 0x7f09940cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f099a800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099414f950 con 0x7f0994161010 2024-01-15T17:58:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f099a800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09940b5fc0 con 0x7f0994160920 2024-01-15T17:58:36.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f099a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09941f2a30 con 0x7f09940cf9f0 2024-01-15T17:58:36.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f0999e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0994161010 0x7f09940cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f0999e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0994161010 0x7f09940cb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46474/0 (socket says 172.21.15.26:46474) 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f0999e00640 1 -- 172.21.15.26:0/853071290 learned_addr learned my addr 172.21.15.26:0/853071290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f0999400640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09940cf9f0 0x7f099415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.200+0000 7f0998a00640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0994160920 0x7f09941f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f0999e00640 1 -- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09940cf9f0 msgr2=0x7f099415d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f0999e00640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09940cf9f0 0x7f099415d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f0999e00640 1 -- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0994160920 msgr2=0x7f09941f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f0999e00640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0994160920 0x7f09941f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f0999e00640 1 -- 172.21.15.26:0/853071290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0984009b30 con 0x7f0994161010 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f0999e00640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0994161010 0x7f09940cb6c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c7f0 tx=0x7f098c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f0991600640 1 -- 172.21.15.26:0/853071290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c007820 con 0x7f0994161010 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f0991600640 1 -- 172.21.15.26:0/853071290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c007980 con 0x7f0994161010 2024-01-15T17:58:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09940ccbd0 con 0x7f0994161010 2024-01-15T17:58:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.201+0000 7f0991600640 1 -- 172.21.15.26:0/853071290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c007d70 con 0x7f0994161010 2024-01-15T17:58:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.202+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09940b5c50 con 0x7f0994161010 2024-01-15T17:58:36.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.203+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09940b6130 con 0x7f0994161010 2024-01-15T17:58:36.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.204+0000 7f0991600640 1 -- 172.21.15.26:0/853071290 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f098c0113f0 con 0x7f0994161010 2024-01-15T17:58:36.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.205+0000 7f0991600640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096c06b500 0x7f096c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.205+0000 7f0991600640 1 -- 172.21.15.26:0/853071290 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f098c08a460 con 0x7f0994161010 2024-01-15T17:58:36.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.205+0000 7f0999400640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096c06b500 0x7f096c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.206+0000 7f0999400640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096c06b500 0x7f096c06d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f097c00a980 tx=0x7f097c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:36.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.209+0000 7f0991600640 1 -- 172.21.15.26:0/853071290 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f098c054e30 con 0x7f0994161010 2024-01-15T17:58:36.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.429+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f099408b1f0 con 0x7f096c06b500 2024-01-15T17:58:36.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.433+0000 7f0991600640 1 -- 172.21.15.26:0/853071290 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f099408b1f0 con 0x7f096c06b500 2024-01-15T17:58:36.436 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T17:58:36.437 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:58:36.437 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T17:58:36.437 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:58:36.437 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T17:58:36.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.435+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096c06b500 msgr2=0x7f096c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.435+0000 7f099a800640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096c06b500 0x7f096c06d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f097c00a980 tx=0x7f097c008040 comp rx=0 tx=0).stop 2024-01-15T17:58:36.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0994161010 msgr2=0x7f09940cb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0994161010 0x7f09940cb6c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c7f0 tx=0x7f098c00ccb0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 shutdown_connections 2024-01-15T17:58:36.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096c06b500 0x7f096c06d9c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0994161010 0x7f09940cb6c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0994160920 0x7f09941f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 --2- 172.21.15.26:0/853071290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09940cf9f0 0x7f099415d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 >> 172.21.15.26:0/853071290 conn(0x7f09940d3090 msgr2=0x7f099406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:36.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 shutdown_connections 2024-01-15T17:58:36.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.436+0000 7f099a800640 1 -- 172.21.15.26:0/853071290 wait complete. 2024-01-15T17:58:36.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T17:58:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.679+0000 7fd1a4e00640 1 Processor -- start 2024-01-15T17:58:36.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.680+0000 7fd1a4e00640 1 -- start start 2024-01-15T17:58:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.681+0000 7fd1a4e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1a0143ae0 0x7fd1a00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.681+0000 7fd1a4e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 0x7fd1a00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd1a4e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a00cfc30 con 0x7fd1a00cf7d0 2024-01-15T17:58:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd1a4e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a00b1ee0 con 0x7fd1a0143ae0 2024-01-15T17:58:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd1a4e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a00b2780 con 0x7fd1a015e3b0 2024-01-15T17:58:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd19de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1a0143ae0 0x7fd1a00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd19de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd1a0143ae0 0x7fd1a00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46482/0 (socket says 172.21.15.26:46482) 2024-01-15T17:58:36.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd19de00640 1 -- 172.21.15.26:0/1605583962 learned_addr learned my addr 172.21.15.26:0/1605583962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd19e800640 1 --2- 172.21.15.26:0/1605583962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 0x7fd1a00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd19de00640 1 -- 172.21.15.26:0/1605583962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 msgr2=0x7fd1a00702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd19de00640 1 --2- 172.21.15.26:0/1605583962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 0x7fd1a00702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd19de00640 1 -- 172.21.15.26:0/1605583962 >> v1:172.21.15.124:6789/0 conn(0x7fd1a00cf7d0 legacy=0x7fd1a015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:36.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd19de00640 1 -- 172.21.15.26:0/1605583962 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a015e990 con 0x7fd1a0143ae0 2024-01-15T17:58:36.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.682+0000 7fd19e800640 1 --2- 172.21.15.26:0/1605583962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 0x7fd1a00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:36.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.684+0000 7fd19de00640 1 --2- 172.21.15.26:0/1605583962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 0x7fd1a00cf290 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd19000bb10 tx=0x7fd19002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe29057075a9018 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:36.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.684+0000 7fd19d400640 1 -- 172.21.15.26:0/1605583962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19002ee70 con 0x7fd1a0143ae0 2024-01-15T17:58:36.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.684+0000 7fd19d400640 1 -- 172.21.15.26:0/1605583962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd190036440 con 0x7fd1a0143ae0 2024-01-15T17:58:36.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.684+0000 7fd19d400640 1 -- 172.21.15.26:0/1605583962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1900367d0 con 0x7fd1a0143ae0 2024-01-15T17:58:36.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.685+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1605583962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 msgr2=0x7fd1a00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.685+0000 7fd1a4e00640 1 --2- 172.21.15.26:0/1605583962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 0x7fd1a00cf290 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd19000bb10 tx=0x7fd19002e0c0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.685+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1605583962 shutdown_connections 2024-01-15T17:58:36.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.685+0000 7fd1a4e00640 1 --2- 172.21.15.26:0/1605583962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 0x7fd1a00702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.685+0000 7fd1a4e00640 1 --2- 172.21.15.26:0/1605583962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 0x7fd1a00cf290 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.685+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1605583962 >> 172.21.15.26:0/1605583962 conn(0x7fd1a006a800 msgr2=0x7fd1a006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:36.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.685+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1605583962 shutdown_connections 2024-01-15T17:58:36.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.685+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1605583962 wait complete. 2024-01-15T17:58:36.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.687+0000 7fd1a4e00640 1 Processor -- start 2024-01-15T17:58:36.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.687+0000 7fd1a4e00640 1 -- start start 2024-01-15T17:58:36.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.688+0000 7fd1a4e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1a00cf7d0 0x7fd1a01e6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.688+0000 7fd1a4e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 0x7fd1a01f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.688+0000 7fd1a4e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 0x7fd1a01e9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.688+0000 7fd1a4e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01f6830 con 0x7fd1a0143ae0 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.688+0000 7fd1a4e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01f6b10 con 0x7fd1a00cf7d0 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.688+0000 7fd1a4e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a01f6df0 con 0x7fd1a015e3b0 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd197e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 0x7fd1a01f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd197e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 0x7fd1a01f5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46496/0 (socket says 172.21.15.26:46496) 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd197e00640 1 -- 172.21.15.26:0/1350661422 learned_addr learned my addr 172.21.15.26:0/1350661422 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd19de00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1a00cf7d0 0x7fd1a01e6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd19e800640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 0x7fd1a01e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd197e00640 1 -- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 msgr2=0x7fd1a01e9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd197e00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 0x7fd1a01e9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd197e00640 1 -- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1a00cf7d0 msgr2=0x7fd1a01e6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd197e00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1a00cf7d0 0x7fd1a01e6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.689+0000 7fd197e00640 1 -- 172.21.15.26:0/1350661422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19000b770 con 0x7fd1a0143ae0 2024-01-15T17:58:36.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.690+0000 7fd197e00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 0x7fd1a01f5fe0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c430 tx=0x7fd18800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:36.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.690+0000 7fd196000640 1 -- 172.21.15.26:0/1350661422 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188011790 con 0x7fd1a0143ae0 2024-01-15T17:58:36.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.690+0000 7fd196000640 1 -- 172.21.15.26:0/1350661422 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1880118f0 con 0x7fd1a0143ae0 2024-01-15T17:58:36.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.690+0000 7fd196000640 1 -- 172.21.15.26:0/1350661422 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188011ce0 con 0x7fd1a0143ae0 2024-01-15T17:58:36.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.690+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a01f2b00 con 0x7fd1a0143ae0 2024-01-15T17:58:36.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.690+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a01f6520 con 0x7fd1a0143ae0 2024-01-15T17:58:36.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.691+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a01f3020 con 0x7fd1a0143ae0 2024-01-15T17:58:36.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.692+0000 7fd196000640 1 -- 172.21.15.26:0/1350661422 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fd18801c460 con 0x7fd1a0143ae0 2024-01-15T17:58:36.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.693+0000 7fd196000640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd17806b530 0x7fd17806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:36.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.693+0000 7fd196000640 1 -- 172.21.15.26:0/1350661422 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd188089f70 con 0x7fd1a0143ae0 2024-01-15T17:58:36.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.694+0000 7fd19de00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd17806b530 0x7fd17806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:36.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.695+0000 7fd19de00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd17806b530 0x7fd17806d9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd190038000 tx=0x7fd190008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:36.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.698+0000 7fd196000640 1 -- 172.21.15.26:0/1350661422 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1a01f3020 con 0x7fd1a0143ae0 2024-01-15T17:58:36.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.894+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd1a01f3020 con 0x7fd1a0143ae0 2024-01-15T17:58:36.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.895+0000 7fd196000640 1 -- 172.21.15.26:0/1350661422 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1a01f3020 con 0x7fd1a0143ae0 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd17806b530 msgr2=0x7fd17806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd17806b530 0x7fd17806d9f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd190038000 tx=0x7fd190008040 comp rx=0 tx=0).stop 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 msgr2=0x7fd1a01f5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 0x7fd1a01f5fe0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd18800c430 tx=0x7fd18800c8f0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 shutdown_connections 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd1a015e3b0 0x7fd1a01e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd17806b530 0x7fd17806d9f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd1a0143ae0 0x7fd1a01f5fe0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 --2- 172.21.15.26:0/1350661422 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd1a00cf7d0 0x7fd1a01e6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.898+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 >> 172.21.15.26:0/1350661422 conn(0x7fd1a006a800 msgr2=0x7fd1a015aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:36.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.899+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 shutdown_connections 2024-01-15T17:58:36.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:36.899+0000 7fd1a4e00640 1 -- 172.21.15.26:0/1350661422 wait complete. 2024-01-15T17:58:36.922 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T17:58:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.143+0000 7f637d600640 1 Processor -- start 2024-01-15T17:58:37.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.144+0000 7f637d600640 1 -- start start 2024-01-15T17:58:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.145+0000 7f637d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f637815afb0 0x7f637813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.146+0000 7f637d600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f637813fae0 0x7f63780b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.146+0000 7f637d600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637813ff30 con 0x7f637813cfa0 2024-01-15T17:58:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.146+0000 7f637d600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63780cdc40 con 0x7f637815afb0 2024-01-15T17:58:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.146+0000 7f637d600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63780cdec0 con 0x7f637813fae0 2024-01-15T17:58:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.146+0000 7f636fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f637815afb0 0x7f637813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.146+0000 7f636fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f637815afb0 0x7f637813f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46506/0 (socket says 172.21.15.26:46506) 2024-01-15T17:58:37.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.146+0000 7f636fe00640 1 -- 172.21.15.26:0/4277064699 learned_addr learned my addr 172.21.15.26:0/4277064699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:37.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.146+0000 7f637cc00640 1 --2- 172.21.15.26:0/4277064699 >> v2:172.21.15.124:3300/0 conn(0x7f637813fae0 0x7f63780b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.147+0000 7f636fe00640 1 -- 172.21.15.26:0/4277064699 >> v2:172.21.15.124:3300/0 conn(0x7f637813fae0 msgr2=0x7f63780b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:37.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.147+0000 7f636fe00640 1 --2- 172.21.15.26:0/4277064699 >> v2:172.21.15.124:3300/0 conn(0x7f637813fae0 0x7f63780b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.147+0000 7f636fe00640 1 -- 172.21.15.26:0/4277064699 >> v1:172.21.15.124:6789/0 conn(0x7f637813cfa0 legacy=0x7f63780cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.147+0000 7f636fe00640 1 -- 172.21.15.26:0/4277064699 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63780ce1a0 con 0x7f637815afb0 2024-01-15T17:58:37.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.147+0000 7f637cc00640 1 --2- 172.21.15.26:0/4277064699 >> v2:172.21.15.124:3300/0 conn(0x7f637813fae0 0x7f63780b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:58:37.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.148+0000 7f636fe00640 1 --2- 172.21.15.26:0/4277064699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637815afb0 0x7f637813f5a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6378154a90 tx=0x7f636002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84df71243f7e743e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:37.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.148+0000 7f6377400640 1 -- 172.21.15.26:0/4277064699 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360031870 con 0x7f637815afb0 2024-01-15T17:58:37.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.148+0000 7f6377400640 1 -- 172.21.15.26:0/4277064699 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63600319d0 con 0x7f637815afb0 2024-01-15T17:58:37.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.148+0000 7f6377400640 1 -- 172.21.15.26:0/4277064699 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360031d60 con 0x7f637815afb0 2024-01-15T17:58:37.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.149+0000 7f637d600640 1 -- 172.21.15.26:0/4277064699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637815afb0 msgr2=0x7f637813f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.149+0000 7f637d600640 1 --2- 172.21.15.26:0/4277064699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637815afb0 0x7f637813f5a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f6378154a90 tx=0x7f636002b910 comp rx=0 tx=0).stop 2024-01-15T17:58:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.149+0000 7f637d600640 1 -- 172.21.15.26:0/4277064699 shutdown_connections 2024-01-15T17:58:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.149+0000 7f637d600640 1 --2- 172.21.15.26:0/4277064699 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637815afb0 0x7f637813f5a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.149+0000 7f637d600640 1 --2- 172.21.15.26:0/4277064699 >> v2:172.21.15.124:3300/0 conn(0x7f637813fae0 0x7f63780b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.149+0000 7f637d600640 1 -- 172.21.15.26:0/4277064699 >> 172.21.15.26:0/4277064699 conn(0x7f637806a800 msgr2=0x7f637806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.149+0000 7f637d600640 1 -- 172.21.15.26:0/4277064699 shutdown_connections 2024-01-15T17:58:37.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.149+0000 7f637d600640 1 -- 172.21.15.26:0/4277064699 wait complete. 2024-01-15T17:58:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.151+0000 7f637d600640 1 Processor -- start 2024-01-15T17:58:37.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.151+0000 7f637d600640 1 -- start start 2024-01-15T17:58:37.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.152+0000 7f637d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637813cfa0 0x7f63781575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.152+0000 7f637d600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f637813fae0 0x7f6378153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.152+0000 7f637d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f637815afb0 0x7f6378154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.152+0000 7f637d600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780c9c30 con 0x7f637813cfa0 2024-01-15T17:58:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.152+0000 7f637d600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6378154140 con 0x7f637813fae0 2024-01-15T17:58:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.152+0000 7f637d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637815c850 con 0x7f637815afb0 2024-01-15T17:58:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.152+0000 7f6377e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637813cfa0 0x7f63781575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.152+0000 7f6377e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637813cfa0 0x7f63781575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46522/0 (socket says 172.21.15.26:46522) 2024-01-15T17:58:37.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.152+0000 7f6377e00640 1 -- 172.21.15.26:0/1438989802 learned_addr learned my addr 172.21.15.26:0/1438989802 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.153+0000 7f636fe00640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f637813fae0 0x7f6378153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.153+0000 7f637cc00640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f637815afb0 0x7f6378154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.153+0000 7f6377e00640 1 -- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f637815afb0 msgr2=0x7f6378154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.153+0000 7f6377e00640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f637815afb0 0x7f6378154a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.153+0000 7f6377e00640 1 -- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f637813fae0 msgr2=0x7f6378153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.153+0000 7f6377e00640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f637813fae0 0x7f6378153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.153+0000 7f6377e00640 1 -- 172.21.15.26:0/1438989802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600094d0 con 0x7f637813cfa0 2024-01-15T17:58:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.153+0000 7f6377e00640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637813cfa0 0x7f63781575d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f636800c450 tx=0x7f636800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.154+0000 7f6374c00640 1 -- 172.21.15.26:0/1438989802 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63680052d0 con 0x7f637813cfa0 2024-01-15T17:58:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.154+0000 7f6374c00640 1 -- 172.21.15.26:0/1438989802 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6368005430 con 0x7f637813cfa0 2024-01-15T17:58:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.154+0000 7f6374c00640 1 -- 172.21.15.26:0/1438989802 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368012870 con 0x7f637813cfa0 2024-01-15T17:58:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.154+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63780c9020 con 0x7f637813cfa0 2024-01-15T17:58:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.154+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63780c95e0 con 0x7f637813cfa0 2024-01-15T17:58:37.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.155+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6378067450 con 0x7f637813cfa0 2024-01-15T17:58:37.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.156+0000 7f6374c00640 1 -- 172.21.15.26:0/1438989802 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7f63680129d0 con 0x7f637813cfa0 2024-01-15T17:58:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.157+0000 7f6374c00640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f635006b530 0x7f635006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.157+0000 7f6374c00640 1 -- 172.21.15.26:0/1438989802 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6368089970 con 0x7f637813cfa0 2024-01-15T17:58:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.157+0000 7f636fe00640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f635006b530 0x7f635006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.158+0000 7f636fe00640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f635006b530 0x7f635006d9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6360036000 tx=0x7f636003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.161+0000 7f6374c00640 1 -- 172.21.15.26:0/1438989802 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6368054460 con 0x7f637813cfa0 2024-01-15T17:58:37.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.381+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f63780cea40 con 0x7f635006b530 2024-01-15T17:58:37.396 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:58:37.391+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:58:37.396 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:58:37.391+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:58:37.396 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:58:37.391+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:58:37.396 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:58:37.392+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:58:37.396 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:58:37.392+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:58:37.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.457+0000 7f6374c00640 1 -- 172.21.15.26:0/1438989802 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6360031810 con 0x7f635006b530 2024-01-15T17:58:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.460+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f635006b530 msgr2=0x7f635006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.460+0000 7f637d600640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f635006b530 0x7f635006d9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6360036000 tx=0x7f636003c040 comp rx=0 tx=0).stop 2024-01-15T17:58:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637813cfa0 msgr2=0x7f63781575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637813cfa0 0x7f63781575d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f636800c450 tx=0x7f636800c910 comp rx=0 tx=0).stop 2024-01-15T17:58:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 shutdown_connections 2024-01-15T17:58:37.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f637815afb0 0x7f6378154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f635006b530 0x7f635006d9f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f637813fae0 0x7f6378153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 --2- 172.21.15.26:0/1438989802 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f637813cfa0 0x7f63781575d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 >> 172.21.15.26:0/1438989802 conn(0x7f637806a800 msgr2=0x7f637806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:37.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 shutdown_connections 2024-01-15T17:58:37.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.461+0000 7f637d600640 1 -- 172.21.15.26:0/1438989802 wait complete. 2024-01-15T17:58:37.484 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T17:58:37.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.705+0000 7fb280a00640 1 Processor -- start 2024-01-15T17:58:37.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.706+0000 7fb280a00640 1 -- start start 2024-01-15T17:58:37.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.707+0000 7fb280a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb27c0adf20 0x7fb27c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.708+0000 7fb280a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27c15f4e0 con 0x7fb27c0ccb50 2024-01-15T17:58:37.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.708+0000 7fb280a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27c071130 con 0x7fb27c15bf10 2024-01-15T17:58:37.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.708+0000 7fb280a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27c15f970 con 0x7fb27c0adf20 2024-01-15T17:58:37.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.708+0000 7fb27b200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb27c0ccb50 0x7fb27c143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42870/0 (socket says 172.21.15.26:42870) 2024-01-15T17:58:37.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.708+0000 7fb27b200640 1 -- 172.21.15.26:0/3102488126 learned_addr learned my addr 172.21.15.26:0/3102488126 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:58:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.708+0000 7fb27a800640 1 --2- 172.21.15.26:0/3102488126 >> v2:172.21.15.124:3300/0 conn(0x7fb27c0adf20 0x7fb27c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.708+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809734473 0 0) 0x7fb27c15f4e0 con 0x7fb27c0ccb50 2024-01-15T17:58:37.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.708+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb25c003520 con 0x7fb27c0ccb50 2024-01-15T17:58:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1443404357 0 0) 0x7fb25c003520 con 0x7fb27c0ccb50 2024-01-15T17:58:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 >> v2:172.21.15.124:3300/0 conn(0x7fb27c0adf20 msgr2=0x7fb27c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 --2- 172.21.15.26:0/3102488126 >> v2:172.21.15.124:3300/0 conn(0x7fb27c0adf20 0x7fb27c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 >> v1:172.21.15.124:6789/0 conn(0x7fb27c15bf10 legacy=0x7fb27c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c15c390 con 0x7fb27c0ccb50 2024-01-15T17:58:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb274001670 con 0x7fb27c0ccb50 2024-01-15T17:58:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 >> v1:172.21.15.26:6789/0 conn(0x7fb27c0ccb50 legacy=0x7fb27c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb25c003970 0x7fb25c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25c0047c0 0x7fb25c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb25c005180 0x7fb25c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c15f970 con 0x7fb25c0047c0 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c00c120 con 0x7fb25c003970 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c00c2e0 con 0x7fb25c005180 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.709+0000 7fb279e00640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25c0047c0 0x7fb25c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.710+0000 7fb27a800640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb25c003970 0x7fb25c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.710+0000 7fb27b200640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb25c005180 0x7fb25c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.710+0000 7fb279e00640 1 -- 172.21.15.26:0/3102488126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb25c005180 msgr2=0x7fb25c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.710+0000 7fb279e00640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb25c005180 0x7fb25c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.710+0000 7fb279e00640 1 -- 172.21.15.26:0/3102488126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb25c003970 msgr2=0x7fb25c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.710+0000 7fb279e00640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb25c003970 0x7fb25c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.710+0000 7fb279e00640 1 -- 172.21.15.26:0/3102488126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c003520 con 0x7fb25c0047c0 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.710+0000 7fb27b200640 1 -- 172.21.15.26:0/3102488126 reap_dead start 2024-01-15T17:58:37.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.711+0000 7fb279e00640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25c0047c0 0x7fb25c004c40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb264001750 tx=0x7fb25c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb30a1dbfefbbbe server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:37.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.711+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2640052d0 con 0x7fb25c0047c0 2024-01-15T17:58:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.711+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26402e050 con 0x7fb25c0047c0 2024-01-15T17:58:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.711+0000 7fb279400640 1 -- 172.21.15.26:0/3102488126 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264003830 con 0x7fb25c0047c0 2024-01-15T17:58:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.712+0000 7fb280a00640 1 -- 172.21.15.26:0/3102488126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25c0047c0 msgr2=0x7fb25c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.712+0000 7fb280a00640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25c0047c0 0x7fb25c004c40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb264001750 tx=0x7fb25c0057f0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.712+0000 7fb280a00640 1 -- 172.21.15.26:0/3102488126 shutdown_connections 2024-01-15T17:58:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.712+0000 7fb280a00640 1 --2- 172.21.15.26:0/3102488126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb25c0047c0 0x7fb25c004c40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.712+0000 7fb280a00640 1 -- 172.21.15.26:0/3102488126 >> 172.21.15.26:0/3102488126 conn(0x7fb27c06a7d0 msgr2=0x7fb27c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.712+0000 7fb280a00640 1 -- 172.21.15.26:0/3102488126 shutdown_connections 2024-01-15T17:58:37.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.712+0000 7fb280a00640 1 -- 172.21.15.26:0/3102488126 wait complete. 2024-01-15T17:58:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.714+0000 7fb280a00640 1 Processor -- start 2024-01-15T17:58:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.714+0000 7fb280a00640 1 -- start start 2024-01-15T17:58:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.715+0000 7fb280a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb25c0047c0 0x7fb27c13c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.715+0000 7fb280a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb27c13aaa0 0x7fb27c15a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.715+0000 7fb280a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb27c13c770 0x7fb27c13cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.715+0000 7fb280a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c13d160 con 0x7fb27c13c770 2024-01-15T17:58:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.715+0000 7fb280a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c13bad0 con 0x7fb25c0047c0 2024-01-15T17:58:37.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.715+0000 7fb280a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c13d800 con 0x7fb27c13aaa0 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.715+0000 7fb27b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb27c13c770 0x7fb27c13cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.715+0000 7fb27b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb27c13c770 0x7fb27c13cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46548/0 (socket says 172.21.15.26:46548) 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.715+0000 7fb27b200640 1 -- 172.21.15.26:0/3786566838 learned_addr learned my addr 172.21.15.26:0/3786566838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb279e00640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb27c13aaa0 0x7fb27c15a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb27a800640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb25c0047c0 0x7fb27c13c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb27b200640 1 -- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb27c13aaa0 msgr2=0x7fb27c15a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb27b200640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb27c13aaa0 0x7fb27c15a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb27b200640 1 -- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb25c0047c0 msgr2=0x7fb27c13c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb27b200640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb25c0047c0 0x7fb27c13c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb27b200640 1 -- 172.21.15.26:0/3786566838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c13b230 con 0x7fb27c13c770 2024-01-15T17:58:37.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb27b200640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb27c13c770 0x7fb27c13cc20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0ae7d0 tx=0x7fb27400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:37.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb272a00640 1 -- 172.21.15.26:0/3786566838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27400eed0 con 0x7fb27c13c770 2024-01-15T17:58:37.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.716+0000 7fb272a00640 1 -- 172.21.15.26:0/3786566838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27400c5c0 con 0x7fb27c13c770 2024-01-15T17:58:37.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.717+0000 7fb272a00640 1 -- 172.21.15.26:0/3786566838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27400c950 con 0x7fb27c13c770 2024-01-15T17:58:37.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.717+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27c13b510 con 0x7fb27c13c770 2024-01-15T17:58:37.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.717+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27c13aee0 con 0x7fb27c13c770 2024-01-15T17:58:37.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.718+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27c067450 con 0x7fb27c13c770 2024-01-15T17:58:37.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.719+0000 7fb272a00640 1 -- 172.21.15.26:0/3786566838 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 9) v1 ==== 89743+0+0 (secure 0 0 0) 0x7fb27400cab0 con 0x7fb27c13c770 2024-01-15T17:58:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.720+0000 7fb272a00640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb25406b530 0x7fb25406d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:58:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.720+0000 7fb272a00640 1 -- 172.21.15.26:0/3786566838 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb274089210 con 0x7fb27c13c770 2024-01-15T17:58:37.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.720+0000 7fb27a800640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb25406b530 0x7fb25406d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:58:37.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.721+0000 7fb27a800640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb25406b530 0x7fb25406d9f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb26c007520 tx=0x7fb26c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:58:37.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.724+0000 7fb272a00640 1 -- 172.21.15.26:0/3786566838 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb274053dc0 con 0x7fb27c13c770 2024-01-15T17:58:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.947+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb27c15f170 con 0x7fb25406b530 2024-01-15T17:58:37.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.959+0000 7fb272a00640 1 -- 172.21.15.26:0/3786566838 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fb26c003990 con 0x7fb25406b530 2024-01-15T17:58:37.962 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd 2024-01-15T17:58:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb25406b530 msgr2=0x7fb25406d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb25406b530 0x7fb25406d9f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb26c007520 tx=0x7fb26c00b040 comp rx=0 tx=0).stop 2024-01-15T17:58:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb27c13c770 msgr2=0x7fb27c13cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:58:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb27c13c770 0x7fb27c13cc20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0ae7d0 tx=0x7fb27400eb90 comp rx=0 tx=0).stop 2024-01-15T17:58:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 shutdown_connections 2024-01-15T17:58:37.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb27c13aaa0 0x7fb27c15a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb25406b530 0x7fb25406d9f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb27c13c770 0x7fb27c13cc20 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 --2- 172.21.15.26:0/3786566838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb25c0047c0 0x7fb27c13c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:58:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 >> 172.21.15.26:0/3786566838 conn(0x7fb27c06a7d0 msgr2=0x7fb27c15d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:58:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 shutdown_connections 2024-01-15T17:58:37.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:58:37.962+0000 7fb280a00640 1 -- 172.21.15.26:0/3786566838 wait complete. 2024-01-15T17:58:37.985 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd 2024-01-15T17:58:37.985 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:58:37.985 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:38.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:38.039 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T17:58:53.406 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T17:58:53.406 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T17:58:53.406 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.2767 s, 70.3 MB/s 2024-01-15T17:58:53.408 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:58:53.408 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:58:53.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:58:53.466 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T17:58:56.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:58:56.730 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:59:08.100 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T17:59:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T17:59:08.101 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.5744 s, 73.7 MB/s 2024-01-15T17:59:08.103 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T17:59:08.103 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T17:59:08.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:59:08.161 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T17:59:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T17:59:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T17:59:22.532 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.3109 s, 75.0 MB/s 2024-01-15T17:59:22.534 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_delay 10 2024-01-15T17:59:22.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.760+0000 7fb1c8e00640 1 Processor -- start 2024-01-15T17:59:22.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.760+0000 7fb1c8e00640 1 -- start start 2024-01-15T17:59:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.761+0000 7fb1c8e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1c413cf10 0x7fb1c413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.761+0000 7fb1c8e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40ce440 con 0x7fb1c40cddf0 2024-01-15T17:59:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.761+0000 7fb1c8e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40ce720 con 0x7fb1c413fae0 2024-01-15T17:59:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.761+0000 7fb1c8e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c413ff60 con 0x7fb1c413cf10 2024-01-15T17:59:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.762+0000 7fb1c3e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb1c40cddf0 0x7fb1c40b2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33182/0 (socket says 172.21.15.26:33182) 2024-01-15T17:59:22.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.762+0000 7fb1c3e00640 1 -- 172.21.15.26:0/823275553 learned_addr learned my addr 172.21.15.26:0/823275553 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:59:22.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.762+0000 7fb1c3400640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1c413cf10 0x7fb1c413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.762+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395002366 0 0) 0x7fb1c40ce440 con 0x7fb1c40cddf0 2024-01-15T17:59:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.762+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a4003520 con 0x7fb1c40cddf0 2024-01-15T17:59:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1713226762 0 0) 0x7fb1a4003520 con 0x7fb1c40cddf0 2024-01-15T17:59:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1c413cf10 msgr2=0x7fb1c413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1c413cf10 0x7fb1c413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 >> v1:172.21.15.124:6789/0 conn(0x7fb1c413fae0 legacy=0x7fb1c40cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c40cea00 con 0x7fb1c40cddf0 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb1b4001670 con 0x7fb1c40cddf0 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 >> v1:172.21.15.26:6789/0 conn(0x7fb1c40cddf0 legacy=0x7fb1c40b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a40039a0 0x7fb1a4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1a40047c0 0x7fb1a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1c413cf10 0x7fb1a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c413ff60 con 0x7fb1a40039a0 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a400ba30 con 0x7fb1a40047c0 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a400bba0 con 0x7fb1c413cf10 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.763+0000 7fb1c3400640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a40039a0 0x7fb1a4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.764+0000 7fb1c2a00640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1a40047c0 0x7fb1a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.764+0000 7fb1c3400640 1 -- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1c413cf10 msgr2=0x7fb1a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.764+0000 7fb1c3400640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1c413cf10 0x7fb1a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.764+0000 7fb1c3400640 1 -- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1a40047c0 msgr2=0x7fb1a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:22.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.764+0000 7fb1c3400640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1a40047c0 0x7fb1a4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.764+0000 7fb1c3400640 1 -- 172.21.15.26:0/823275553 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4003520 con 0x7fb1a40039a0 2024-01-15T17:59:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.764+0000 7fb1c2a00640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1a40047c0 0x7fb1a4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:59:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.765+0000 7fb1c3400640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a40039a0 0x7fb1a4003d80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009bd0 tx=0x7fb1a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd466c56ffe7e7e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.765+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc02ee10 con 0x7fb1a40039a0 2024-01-15T17:59:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.765+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1bc00f040 con 0x7fb1a40039a0 2024-01-15T17:59:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.765+0000 7fb1c2000640 1 -- 172.21.15.26:0/823275553 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc036680 con 0x7fb1a40039a0 2024-01-15T17:59:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.766+0000 7fb1c8e00640 1 -- 172.21.15.26:0/823275553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a40039a0 msgr2=0x7fb1a4003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.766+0000 7fb1c8e00640 1 --2- 172.21.15.26:0/823275553 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1a40039a0 0x7fb1a4003d80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009bd0 tx=0x7fb1a40057f0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.766+0000 7fb1c3e00640 1 -- 172.21.15.26:0/823275553 reap_dead start 2024-01-15T17:59:22.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.766+0000 7fb1c8e00640 1 -- 172.21.15.26:0/823275553 shutdown_connections 2024-01-15T17:59:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.766+0000 7fb1c8e00640 1 -- 172.21.15.26:0/823275553 >> 172.21.15.26:0/823275553 conn(0x7fb1c406a7d0 msgr2=0x7fb1c406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.766+0000 7fb1c8e00640 1 -- 172.21.15.26:0/823275553 shutdown_connections 2024-01-15T17:59:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.766+0000 7fb1c8e00640 1 -- 172.21.15.26:0/823275553 wait complete. 2024-01-15T17:59:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.768+0000 7fb1c8e00640 1 Processor -- start 2024-01-15T17:59:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.769+0000 7fb1c8e00640 1 -- start start 2024-01-15T17:59:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.770+0000 7fb1c8e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1a40039a0 0x7fb1c4136180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.770+0000 7fb1c8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c4153370 0x7fb1c413bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.770+0000 7fb1c8e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1c415a870 0x7fb1c415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.770+0000 7fb1c8e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c415b230 con 0x7fb1c4153370 2024-01-15T17:59:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.770+0000 7fb1c8e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4070d00 con 0x7fb1c415a870 2024-01-15T17:59:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.770+0000 7fb1c8e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c41537f0 con 0x7fb1a40039a0 2024-01-15T17:59:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.770+0000 7fb1c2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c4153370 0x7fb1c413bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.770+0000 7fb1c2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c4153370 0x7fb1c413bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40948/0 (socket says 172.21.15.26:40948) 2024-01-15T17:59:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.770+0000 7fb1c2a00640 1 -- 172.21.15.26:0/313058722 learned_addr learned my addr 172.21.15.26:0/313058722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1c3400640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1a40039a0 0x7fb1c4136180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1c2a00640 1 -- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1a40039a0 msgr2=0x7fb1c4136180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1c2a00640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1a40039a0 0x7fb1c4136180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1c2a00640 1 -- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1c415a870 msgr2=0x7fb1c415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1c2a00640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1c415a870 0x7fb1c415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1c2a00640 1 -- 172.21.15.26:0/313058722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c413c4f0 con 0x7fb1c4153370 2024-01-15T17:59:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1c2a00640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c4153370 0x7fb1c413bfb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00d870 tx=0x7fb1ac00dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1bb400640 1 -- 172.21.15.26:0/313058722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac005480 con 0x7fb1c4153370 2024-01-15T17:59:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1bb400640 1 -- 172.21.15.26:0/313058722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac00de60 con 0x7fb1c4153370 2024-01-15T17:59:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1bb400640 1 -- 172.21.15.26:0/313058722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00b850 con 0x7fb1c4153370 2024-01-15T17:59:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.771+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c4153a50 con 0x7fb1c4153370 2024-01-15T17:59:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.772+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c4151a10 con 0x7fb1c4153370 2024-01-15T17:59:22.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.773+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c4067450 con 0x7fb1c4153370 2024-01-15T17:59:22.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.774+0000 7fb1bb400640 1 -- 172.21.15.26:0/313058722 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 89794+0+0 (secure 0 0 0) 0x7fb1ac00b9b0 con 0x7fb1c4153370 2024-01-15T17:59:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.775+0000 7fb1bb400640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb19c06b590 0x7fb19c06da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.775+0000 7fb1bb400640 1 -- 172.21.15.26:0/313058722 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb1ac08a190 con 0x7fb1c4153370 2024-01-15T17:59:22.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.775+0000 7fb1c3400640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb19c06b590 0x7fb19c06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:22.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.776+0000 7fb1c3400640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb19c06b590 0x7fb19c06da50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc038000 tx=0x7fb1bc030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:22.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.779+0000 7fb1bb400640 1 -- 172.21.15.26:0/313058722 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1ac058660 con 0x7fb1c4153370 2024-01-15T17:59:22.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.977+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}] v 0) v1 -- 0x7fb1c40cbf10 con 0x7fb1c4153370 2024-01-15T17:59:22.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.983+0000 7fb1bb400640 1 -- 172.21.15.26:0/313058722 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}]=0 v15)=0 v15) v1 ==== 131+0+0 (secure 0 0 0) 0x7fb1ac019310 con 0x7fb1c4153370 2024-01-15T17:59:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.986+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb19c06b590 msgr2=0x7fb19c06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.986+0000 7fb1c8e00640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb19c06b590 0x7fb19c06da50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc038000 tx=0x7fb1bc030000 comp rx=0 tx=0).stop 2024-01-15T17:59:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c4153370 msgr2=0x7fb1c413bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c4153370 0x7fb1c413bfb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00d870 tx=0x7fb1ac00dd30 comp rx=0 tx=0).stop 2024-01-15T17:59:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 shutdown_connections 2024-01-15T17:59:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb19c06b590 0x7fb19c06da50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1c415a870 0x7fb1c415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1c4153370 0x7fb1c413bfb0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 --2- 172.21.15.26:0/313058722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1a40039a0 0x7fb1c4136180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 >> 172.21.15.26:0/313058722 conn(0x7fb1c406a7d0 msgr2=0x7fb1c40afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 shutdown_connections 2024-01-15T17:59:22.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:22.987+0000 7fb1c8e00640 1 -- 172.21.15.26:0/313058722 wait complete. 2024-01-15T17:59:23.012 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-15T17:59:23.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.229+0000 7f0993000640 1 Processor -- start 2024-01-15T17:59:23.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.230+0000 7f0993000640 1 -- start start 2024-01-15T17:59:23.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.231+0000 7f0993000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f098c0cf9a0 0x7f098c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.231+0000 7f0993000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 0x7f098c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.231+0000 7f0993000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098c0d0320 con 0x7f098c15d4b0 2024-01-15T17:59:23.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.231+0000 7f0993000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098c0d1c60 con 0x7f098c0cf9a0 2024-01-15T17:59:23.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.231+0000 7f0993000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0b2730 con 0x7f098c0b31d0 2024-01-15T17:59:23.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.232+0000 7f0991c00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f098c15d4b0 0x7f098c0d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:33758/0 (socket says 172.21.15.26:33758) 2024-01-15T17:59:23.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.232+0000 7f0991c00640 1 -- 172.21.15.26:0/1066205858 learned_addr learned my addr 172.21.15.26:0/1066205858 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:59:23.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.232+0000 7f0992600640 1 --2- 172.21.15.26:0/1066205858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 0x7f098c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.232+0000 7f0991200640 1 --2- 172.21.15.26:0/1066205858 >> v2:172.21.15.124:3300/0 conn(0x7f098c0cf9a0 0x7f098c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.233+0000 7f0991200640 1 -- 172.21.15.26:0/1066205858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 msgr2=0x7f098c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:23.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.233+0000 7f0991200640 1 --2- 172.21.15.26:0/1066205858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 0x7f098c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.233+0000 7f0991200640 1 -- 172.21.15.26:0/1066205858 >> v1:172.21.15.124:6789/0 conn(0x7f098c15d4b0 legacy=0x7f098c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.233+0000 7f0991200640 1 -- 172.21.15.26:0/1066205858 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c0d1ec0 con 0x7f098c0cf9a0 2024-01-15T17:59:23.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.233+0000 7f098be00640 1 -- 172.21.15.26:0/1066205858 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957351954 0 0) 0x7f098c0d0320 con 0x7f098c15d4b0 2024-01-15T17:59:23.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.234+0000 7f0991200640 1 --2- 172.21.15.26:0/1066205858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 0x7f098c0cfde0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09740097a0 tx=0x7f097402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cc301ccba4c242 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.235+0000 7f098be00640 1 -- 172.21.15.26:0/1066205858 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097402d040 con 0x7f098c0cf9a0 2024-01-15T17:59:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.235+0000 7f098be00640 1 -- 172.21.15.26:0/1066205858 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974031810 con 0x7f098c0cf9a0 2024-01-15T17:59:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.235+0000 7f098be00640 1 -- 172.21.15.26:0/1066205858 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974031ba0 con 0x7f098c0cf9a0 2024-01-15T17:59:23.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.236+0000 7f0993000640 1 -- 172.21.15.26:0/1066205858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 msgr2=0x7f098c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.236+0000 7f0993000640 1 --2- 172.21.15.26:0/1066205858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 0x7f098c0cfde0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f09740097a0 tx=0x7f097402b8b0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.236+0000 7f0993000640 1 -- 172.21.15.26:0/1066205858 shutdown_connections 2024-01-15T17:59:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.236+0000 7f0993000640 1 --2- 172.21.15.26:0/1066205858 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 0x7f098c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.236+0000 7f0993000640 1 --2- 172.21.15.26:0/1066205858 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 0x7f098c0cfde0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.236+0000 7f0993000640 1 -- 172.21.15.26:0/1066205858 >> 172.21.15.26:0/1066205858 conn(0x7f098c0d3000 msgr2=0x7f098c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.236+0000 7f0993000640 1 -- 172.21.15.26:0/1066205858 shutdown_connections 2024-01-15T17:59:23.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.236+0000 7f0993000640 1 -- 172.21.15.26:0/1066205858 wait complete. 2024-01-15T17:59:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.238+0000 7f0993000640 1 Processor -- start 2024-01-15T17:59:23.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.238+0000 7f0993000640 1 -- start start 2024-01-15T17:59:23.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.239+0000 7f0993000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 0x7f098c1f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.239+0000 7f0993000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 0x7f098c09ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.239+0000 7f0993000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c15d4b0 0x7f098c09ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.239+0000 7f0993000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c15c4e0 con 0x7f098c15d4b0 2024-01-15T17:59:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.239+0000 7f0993000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c15c000 con 0x7f098c0cf9a0 2024-01-15T17:59:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.239+0000 7f0993000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c09d8f0 con 0x7f098c0b31d0 2024-01-15T17:59:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.239+0000 7f0992600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c15d4b0 0x7f098c09ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0992600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c15d4b0 0x7f098c09ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40954/0 (socket says 172.21.15.26:40954) 2024-01-15T17:59:23.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0992600640 1 -- 172.21.15.26:0/749710608 learned_addr learned my addr 172.21.15.26:0/749710608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:23.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0991c00640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 0x7f098c1f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0991200640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 0x7f098c09ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0992600640 1 -- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 msgr2=0x7f098c1f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0992600640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 0x7f098c1f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0992600640 1 -- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 msgr2=0x7f098c09ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0992600640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 0x7f098c09ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0992600640 1 -- 172.21.15.26:0/749710608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0974009400 con 0x7f098c15d4b0 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0991200640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 0x7f098c09ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.240+0000 7f0992600640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c15d4b0 0x7f098c09ef60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f098400efc0 tx=0x7f098400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.241+0000 7f0989600640 1 -- 172.21.15.26:0/749710608 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984009ba0 con 0x7f098c15d4b0 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.241+0000 7f0989600640 1 -- 172.21.15.26:0/749710608 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098400f040 con 0x7f098c15d4b0 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.241+0000 7f0989600640 1 -- 172.21.15.26:0/749710608 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840146c0 con 0x7f098c15d4b0 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.241+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c09dbd0 con 0x7f098c15d4b0 2024-01-15T17:59:23.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.241+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c09e190 con 0x7f098c15d4b0 2024-01-15T17:59:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.242+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c0d2710 con 0x7f098c15d4b0 2024-01-15T17:59:23.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.243+0000 7f0989600640 1 -- 172.21.15.26:0/749710608 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 89794+0+0 (secure 0 0 0) 0x7f0984014820 con 0x7f098c15d4b0 2024-01-15T17:59:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.244+0000 7f0989600640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096406b590 0x7f096406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.244+0000 7f0989600640 1 -- 172.21.15.26:0/749710608 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f098408ac70 con 0x7f098c15d4b0 2024-01-15T17:59:23.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.245+0000 7f0991c00640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096406b590 0x7f096406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.246+0000 7f0991c00640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096406b590 0x7f096406da50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f097c000f60 tx=0x7f097c007b90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:23.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.248+0000 7f0989600640 1 -- 172.21.15.26:0/749710608 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0984055620 con 0x7f098c15d4b0 2024-01-15T17:59:23.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.442+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f098c067450 con 0x7f098c15d4b0 2024-01-15T17:59:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.449+0000 7f0989600640 1 -- 172.21.15.26:0/749710608 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v16)=0 v16) v1 ==== 135+0+0 (secure 0 0 0) 0x7f0984018020 con 0x7f098c15d4b0 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096406b590 msgr2=0x7f096406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096406b590 0x7f096406da50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f097c000f60 tx=0x7f097c007b90 comp rx=0 tx=0).stop 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c15d4b0 msgr2=0x7f098c09ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c15d4b0 0x7f098c09ef60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f098400efc0 tx=0x7f098400c490 comp rx=0 tx=0).stop 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 shutdown_connections 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f096406b590 0x7f096406da50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f098c15d4b0 0x7f098c09ef60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f098c0cf9a0 0x7f098c09ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 --2- 172.21.15.26:0/749710608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f098c0b31d0 0x7f098c1f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 >> 172.21.15.26:0/749710608 conn(0x7f098c0d3000 msgr2=0x7f098c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 shutdown_connections 2024-01-15T17:59:23.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.453+0000 7f0993000640 1 -- 172.21.15.26:0/749710608 wait complete. 2024-01-15T17:59:23.476 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T17:59:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.697+0000 7fef76000640 1 Processor -- start 2024-01-15T17:59:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.697+0000 7fef76000640 1 -- start start 2024-01-15T17:59:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.698+0000 7fef76000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef7009a170 0x7fef700b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.698+0000 7fef76000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7006cea0 con 0x7fef7006c6b0 2024-01-15T17:59:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.699+0000 7fef76000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7006cbc0 con 0x7fef700cd240 2024-01-15T17:59:23.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.699+0000 7fef76000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7006f350 con 0x7fef7009a170 2024-01-15T17:59:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.699+0000 7fef6fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fef7006c6b0 0x7fef700cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33184/0 (socket says 172.21.15.26:33184) 2024-01-15T17:59:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.699+0000 7fef6fe00640 1 -- 172.21.15.26:0/3107586866 learned_addr learned my addr 172.21.15.26:0/3107586866 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:59:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.699+0000 7fef6ea00640 1 --2- 172.21.15.26:0/3107586866 >> v2:172.21.15.26:3300/0 conn(0x7fef7009a170 0x7fef700b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.699+0000 7fef6e000640 1 -- 172.21.15.26:0/3107586866 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879122451 0 0) 0x7fef7006cea0 con 0x7fef7006c6b0 2024-01-15T17:59:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.699+0000 7fef6e000640 1 -- 172.21.15.26:0/3107586866 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef50003520 con 0x7fef7006c6b0 2024-01-15T17:59:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.699+0000 7fef6ea00640 1 -- 172.21.15.26:0/3107586866 >> v1:172.21.15.124:6789/0 conn(0x7fef700cd240 legacy=0x7fef70099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.699+0000 7fef6ea00640 1 -- 172.21.15.26:0/3107586866 >> v1:172.21.15.26:6789/0 conn(0x7fef7006c6b0 legacy=0x7fef700cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.700+0000 7fef6ea00640 1 -- 172.21.15.26:0/3107586866 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7006f630 con 0x7fef7009a170 2024-01-15T17:59:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.701+0000 7fef6ea00640 1 --2- 172.21.15.26:0/3107586866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7009a170 0x7fef700b3820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fef58009d90 tx=0x7fef5802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f890f018076ac2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.701+0000 7fef6e000640 1 -- 172.21.15.26:0/3107586866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5802cd00 con 0x7fef7009a170 2024-01-15T17:59:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.701+0000 7fef6e000640 1 -- 172.21.15.26:0/3107586866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5802ce60 con 0x7fef7009a170 2024-01-15T17:59:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.701+0000 7fef6e000640 1 -- 172.21.15.26:0/3107586866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef58034660 con 0x7fef7009a170 2024-01-15T17:59:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.701+0000 7fef76000640 1 -- 172.21.15.26:0/3107586866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7009a170 msgr2=0x7fef700b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.701+0000 7fef76000640 1 --2- 172.21.15.26:0/3107586866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7009a170 0x7fef700b3820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fef58009d90 tx=0x7fef5802bf50 comp rx=0 tx=0).stop 2024-01-15T17:59:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.702+0000 7fef76000640 1 -- 172.21.15.26:0/3107586866 shutdown_connections 2024-01-15T17:59:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.702+0000 7fef76000640 1 --2- 172.21.15.26:0/3107586866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7009a170 0x7fef700b3820 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.702+0000 7fef76000640 1 -- 172.21.15.26:0/3107586866 >> 172.21.15.26:0/3107586866 conn(0x7fef700b2d00 msgr2=0x7fef700b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.702+0000 7fef76000640 1 -- 172.21.15.26:0/3107586866 shutdown_connections 2024-01-15T17:59:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.702+0000 7fef76000640 1 -- 172.21.15.26:0/3107586866 wait complete. 2024-01-15T17:59:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.703+0000 7fef76000640 1 Processor -- start 2024-01-15T17:59:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.704+0000 7fef76000640 1 -- start start 2024-01-15T17:59:23.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef76000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7006c6b0 0x7fef70154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef76000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef7009a170 0x7fef700c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef76000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef700cd240 0x7fef700c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef76000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef700ca300 con 0x7fef7006c6b0 2024-01-15T17:59:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef76000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef700c6340 con 0x7fef7009a170 2024-01-15T17:59:23.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef76000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef700c6620 con 0x7fef700cd240 2024-01-15T17:59:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef6f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7006c6b0 0x7fef70154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef6f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7006c6b0 0x7fef70154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40966/0 (socket says 172.21.15.26:40966) 2024-01-15T17:59:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef6f400640 1 -- 172.21.15.26:0/1860080722 learned_addr learned my addr 172.21.15.26:0/1860080722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef6ea00640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef7009a170 0x7fef700c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef6f400640 1 -- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef700cd240 msgr2=0x7fef700c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef6f400640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef700cd240 0x7fef700c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef6f400640 1 -- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef7009a170 msgr2=0x7fef700c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef6f400640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef7009a170 0x7fef700c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.705+0000 7fef6f400640 1 -- 172.21.15.26:0/1860080722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58009a70 con 0x7fef7006c6b0 2024-01-15T17:59:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.706+0000 7fef6f400640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7006c6b0 0x7fef70154850 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fef6000c450 tx=0x7fef6000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.706+0000 7fef67400640 1 -- 172.21.15.26:0/1860080722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef600052d0 con 0x7fef7006c6b0 2024-01-15T17:59:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.706+0000 7fef67400640 1 -- 172.21.15.26:0/1860080722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60005430 con 0x7fef7006c6b0 2024-01-15T17:59:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.706+0000 7fef67400640 1 -- 172.21.15.26:0/1860080722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60012870 con 0x7fef7006c6b0 2024-01-15T17:59:23.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.706+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef700c6900 con 0x7fef7006c6b0 2024-01-15T17:59:23.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.706+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef700b43f0 con 0x7fef7006c6b0 2024-01-15T17:59:23.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.707+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef700675d0 con 0x7fef7006c6b0 2024-01-15T17:59:23.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.708+0000 7fef67400640 1 -- 172.21.15.26:0/1860080722 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 89794+0+0 (secure 0 0 0) 0x7fef600129d0 con 0x7fef7006c6b0 2024-01-15T17:59:23.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.709+0000 7fef67400640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef4806b590 0x7fef4806da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:23.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.709+0000 7fef67400640 1 -- 172.21.15.26:0/1860080722 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fef60089b30 con 0x7fef7006c6b0 2024-01-15T17:59:23.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.710+0000 7fef6ea00640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef4806b590 0x7fef4806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:23.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.711+0000 7fef6ea00640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef4806b590 0x7fef4806da50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fef58036000 tx=0x7fef58030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:23.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.714+0000 7fef67400640 1 -- 172.21.15.26:0/1860080722 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef60054640 con 0x7fef7006c6b0 2024-01-15T17:59:23.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.939+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7fef700c64b0 con 0x7fef4806b590 2024-01-15T17:59:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.977+0000 7fef67400640 1 -- 172.21.15.26:0/1860080722 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fef700c64b0 con 0x7fef4806b590 2024-01-15T17:59:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.980+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef4806b590 msgr2=0x7fef4806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.980+0000 7fef76000640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef4806b590 0x7fef4806da50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fef58036000 tx=0x7fef58030000 comp rx=0 tx=0).stop 2024-01-15T17:59:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.980+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7006c6b0 msgr2=0x7fef70154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.980+0000 7fef76000640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7006c6b0 0x7fef70154850 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fef6000c450 tx=0x7fef6000c910 comp rx=0 tx=0).stop 2024-01-15T17:59:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.981+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 shutdown_connections 2024-01-15T17:59:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.981+0000 7fef76000640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef700cd240 0x7fef700c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.981+0000 7fef76000640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef4806b590 0x7fef4806da50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.981+0000 7fef76000640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef7009a170 0x7fef700c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.981+0000 7fef76000640 1 --2- 172.21.15.26:0/1860080722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef7006c6b0 0x7fef70154850 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.981+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 >> 172.21.15.26:0/1860080722 conn(0x7fef700b2d00 msgr2=0x7fef70099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.981+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 shutdown_connections 2024-01-15T17:59:23.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:23.981+0000 7fef76000640 1 -- 172.21.15.26:0/1860080722 wait complete. 2024-01-15T17:59:24.003 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T17:59:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.226+0000 7f1722600640 1 Processor -- start 2024-01-15T17:59:24.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.227+0000 7f1722600640 1 -- start start 2024-01-15T17:59:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.228+0000 7f1722600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c071e40 0x7f171c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.228+0000 7f1722600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f171c15d770 0x7f171c15dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.228+0000 7f1722600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c15e140 con 0x7f171c143810 2024-01-15T17:59:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.228+0000 7f1722600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c143bf0 con 0x7f171c15d770 2024-01-15T17:59:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.228+0000 7f1722600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0722c0 con 0x7f171c071e40 2024-01-15T17:59:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.229+0000 7f1721200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f171c143810 0x7f171c15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33192/0 (socket says 172.21.15.26:33192) 2024-01-15T17:59:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.229+0000 7f1721200640 1 -- 172.21.15.26:0/3675930298 learned_addr learned my addr 172.21.15.26:0/3675930298 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:59:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.229+0000 7f1721c00640 1 --2- 172.21.15.26:0/3675930298 >> v2:172.21.15.26:3300/0 conn(0x7f171c15d770 0x7f171c15dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.229+0000 7f171be00640 1 --2- 172.21.15.26:0/3675930298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c071e40 0x7f171c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.229+0000 7f1721c00640 1 -- 172.21.15.26:0/3675930298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c071e40 msgr2=0x7f171c0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.229+0000 7f1721c00640 1 --2- 172.21.15.26:0/3675930298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c071e40 0x7f171c0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.229+0000 7f171b400640 1 -- 172.21.15.26:0/3675930298 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218621485 0 0) 0x7f171c15e140 con 0x7f171c143810 2024-01-15T17:59:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.229+0000 7f1721c00640 1 -- 172.21.15.26:0/3675930298 >> v1:172.21.15.26:6789/0 conn(0x7f171c143810 legacy=0x7f171c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.229+0000 7f1721c00640 1 -- 172.21.15.26:0/3675930298 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c06c730 con 0x7f171c15d770 2024-01-15T17:59:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.231+0000 7f1721c00640 1 --2- 172.21.15.26:0/3675930298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 0x7f171c15dc00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f17140097a0 tx=0x7f171402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35621fcc24a34f06 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.231+0000 7f171b400640 1 -- 172.21.15.26:0/3675930298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171402f050 con 0x7f171c15d770 2024-01-15T17:59:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.231+0000 7f171b400640 1 -- 172.21.15.26:0/3675930298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714031700 con 0x7f171c15d770 2024-01-15T17:59:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.231+0000 7f171b400640 1 -- 172.21.15.26:0/3675930298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714031a90 con 0x7f171c15d770 2024-01-15T17:59:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.231+0000 7f1722600640 1 -- 172.21.15.26:0/3675930298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 msgr2=0x7f171c15dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.231+0000 7f1722600640 1 --2- 172.21.15.26:0/3675930298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 0x7f171c15dc00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f17140097a0 tx=0x7f171402b7a0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.232+0000 7f1722600640 1 -- 172.21.15.26:0/3675930298 shutdown_connections 2024-01-15T17:59:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.232+0000 7f1722600640 1 --2- 172.21.15.26:0/3675930298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 0x7f171c15dc00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.232+0000 7f1722600640 1 --2- 172.21.15.26:0/3675930298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c071e40 0x7f171c0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.232+0000 7f1722600640 1 -- 172.21.15.26:0/3675930298 >> 172.21.15.26:0/3675930298 conn(0x7f171c06a7d0 msgr2=0x7f171c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:24.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.232+0000 7f1722600640 1 -- 172.21.15.26:0/3675930298 shutdown_connections 2024-01-15T17:59:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.232+0000 7f1722600640 1 -- 172.21.15.26:0/3675930298 wait complete. 2024-01-15T17:59:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.234+0000 7f1722600640 1 Processor -- start 2024-01-15T17:59:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.234+0000 7f1722600640 1 -- start start 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1722600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c071e40 0x7f171c13e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1722600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c143810 0x7f171c145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1722600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 0x7f171c146280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1722600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c13ec00 con 0x7f171c15d770 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1722600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0af3f0 con 0x7f171c071e40 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1722600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0ce4e0 con 0x7f171c143810 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1721c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 0x7f171c146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1721c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 0x7f171c146280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40972/0 (socket says 172.21.15.26:40972) 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1721c00640 1 -- 172.21.15.26:0/4074930216 learned_addr learned my addr 172.21.15.26:0/4074930216 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f1721200640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c071e40 0x7f171c13e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.235+0000 7f171be00640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c143810 0x7f171c145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1721c00640 1 -- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c143810 msgr2=0x7f171c145d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1721c00640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c143810 0x7f171c145d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1721c00640 1 -- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c071e40 msgr2=0x7f171c13e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1721c00640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c071e40 0x7f171c13e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1721c00640 1 -- 172.21.15.26:0/4074930216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1714009400 con 0x7f171c15d770 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1721200640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c071e40 0x7f171c13e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1721c00640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 0x7f171c146280 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f17140097a0 tx=0x7f17140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1718c00640 1 -- 172.21.15.26:0/4074930216 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140339b0 con 0x7f171c15d770 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1718c00640 1 -- 172.21.15.26:0/4074930216 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714033b30 con 0x7f171c15d770 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c0ce7c0 con 0x7f171c15d770 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.236+0000 7f1718c00640 1 -- 172.21.15.26:0/4074930216 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171403c400 con 0x7f171c15d770 2024-01-15T17:59:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.237+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c0af920 con 0x7f171c15d770 2024-01-15T17:59:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.238+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c067450 con 0x7f171c15d770 2024-01-15T17:59:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.239+0000 7f1718c00640 1 -- 172.21.15.26:0/4074930216 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 89794+0+0 (secure 0 0 0) 0x7f171403a040 con 0x7f171c15d770 2024-01-15T17:59:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.239+0000 7f1718c00640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406b590 0x7f16f406da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.240+0000 7f1718c00640 1 -- 172.21.15.26:0/4074930216 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f17140ab9a0 con 0x7f171c15d770 2024-01-15T17:59:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.240+0000 7f1721200640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406b590 0x7f16f406da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.241+0000 7f1721200640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406b590 0x7f16f406da50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1704000f60 tx=0x7f1704007c10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:24.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.244+0000 7f1718c00640 1 -- 172.21.15.26:0/4074930216 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1714076450 con 0x7f171c15d770 2024-01-15T17:59:24.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.465+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f171c06c9f0 con 0x7f16f406b590 2024-01-15T17:59:24.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.476+0000 7f1718c00640 1 -- 172.21.15.26:0/4074930216 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f1704007270 con 0x7f16f406b590 2024-01-15T17:59:24.479 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd 2024-01-15T17:59:24.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.478+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406b590 msgr2=0x7f16f406da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:24.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.478+0000 7f1722600640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406b590 0x7f16f406da50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1704000f60 tx=0x7f1704007c10 comp rx=0 tx=0).stop 2024-01-15T17:59:24.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.478+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 msgr2=0x7f171c146280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.479+0000 7f1722600640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 0x7f171c146280 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f17140097a0 tx=0x7f17140046c0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.479+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 shutdown_connections 2024-01-15T17:59:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.479+0000 7f1722600640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c143810 0x7f171c145d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.479+0000 7f1722600640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406b590 0x7f16f406da50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.479+0000 7f1722600640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f171c15d770 0x7f171c146280 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.479+0000 7f1722600640 1 --2- 172.21.15.26:0/4074930216 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c071e40 0x7f171c13e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.479+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 >> 172.21.15.26:0/4074930216 conn(0x7f171c06a7d0 msgr2=0x7f171c0ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.479+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 shutdown_connections 2024-01-15T17:59:24.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:24.479+0000 7f1722600640 1 -- 172.21.15.26:0/4074930216 wait complete. 2024-01-15T17:59:24.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:59:24.500 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd) 2024-01-15T17:59:24.563 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd 2024-01-15T17:59:24.563 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T17:59:24.563 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:59:24.564 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T17:59:26.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:59:26.566 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd) 2024-01-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd 2024-01-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T17:59:26.629 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:59:26.630 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T17:59:26.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:59:26.878 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:59:28.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T17:59:28.631 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd) 2024-01-15T17:59:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/0e0b40da-5abd-45d6-bba8-add6f5944bbd 2024-01-15T17:59:28.700 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T17:59:28.700 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:59:28.701 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T17:59:28.702 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6931 2024-01-15T17:59:28.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.928+0000 7fcdeee00640 1 Processor -- start 2024-01-15T17:59:28.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.928+0000 7fcdeee00640 1 -- start start 2024-01-15T17:59:28.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.929+0000 7fcdeee00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcde8160920 0x7fcde80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:28.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.929+0000 7fcdeee00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcde80cf9f0 0x7fcde80cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:28.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.929+0000 7fcdeee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 0x7fcde81614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:28.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.929+0000 7fcdeee00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80d03e0 con 0x7fcde80cf9f0 2024-01-15T17:59:28.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.929+0000 7fcdeee00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde80d0a10 con 0x7fcde8160920 2024-01-15T17:59:28.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.929+0000 7fcdeee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80cf080 con 0x7fcde8161010 2024-01-15T17:59:28.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.930+0000 7fcdeda00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcde8160920 0x7fcde80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:28.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.930+0000 7fcded000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcde80cf9f0 0x7fcde80cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:28.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.930+0000 7fcdee400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 0x7fcde81614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:28.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.930+0000 7fcdeda00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcde8160920 0x7fcde80b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:57640/0 (socket says 172.21.15.26:57640) 2024-01-15T17:59:28.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.930+0000 7fcdeda00640 1 -- 172.21.15.26:0/2189261608 learned_addr learned my addr 172.21.15.26:0/2189261608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:28.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.931+0000 7fcdeda00640 1 -- 172.21.15.26:0/2189261608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 msgr2=0x7fcde81614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:28.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.931+0000 7fcdeda00640 1 --2- 172.21.15.26:0/2189261608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 0x7fcde81614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:28.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.931+0000 7fcdeda00640 1 -- 172.21.15.26:0/2189261608 >> v2:172.21.15.26:3300/0 conn(0x7fcde80cf9f0 msgr2=0x7fcde80cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:28.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.931+0000 7fcdeda00640 1 --2- 172.21.15.26:0/2189261608 >> v2:172.21.15.26:3300/0 conn(0x7fcde80cf9f0 0x7fcde80cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:28.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.931+0000 7fcdeda00640 1 -- 172.21.15.26:0/2189261608 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde80c6be0 con 0x7fcde8160920 2024-01-15T17:59:28.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.933+0000 7fcdeda00640 1 --2- 172.21.15.26:0/2189261608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcde8160920 0x7fcde80b3f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcde808c060 tx=0x7fcdd802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a70eb089ce981ccb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:28.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.933+0000 7fcde7e00640 1 -- 172.21.15.26:0/2189261608 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8031870 con 0x7fcde8160920 2024-01-15T17:59:28.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.933+0000 7fcde7e00640 1 -- 172.21.15.26:0/2189261608 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd80319d0 con 0x7fcde8160920 2024-01-15T17:59:28.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.934+0000 7fcde7e00640 1 -- 172.21.15.26:0/2189261608 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8031d60 con 0x7fcde8160920 2024-01-15T17:59:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.934+0000 7fcdeee00640 1 -- 172.21.15.26:0/2189261608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcde8160920 msgr2=0x7fcde80b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.934+0000 7fcdeee00640 1 --2- 172.21.15.26:0/2189261608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcde8160920 0x7fcde80b3f30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcde808c060 tx=0x7fcdd802b910 comp rx=0 tx=0).stop 2024-01-15T17:59:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.934+0000 7fcdeee00640 1 -- 172.21.15.26:0/2189261608 shutdown_connections 2024-01-15T17:59:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.934+0000 7fcdeee00640 1 --2- 172.21.15.26:0/2189261608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 0x7fcde81614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.934+0000 7fcdeee00640 1 --2- 172.21.15.26:0/2189261608 >> v2:172.21.15.26:3300/0 conn(0x7fcde80cf9f0 0x7fcde80cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.934+0000 7fcdeee00640 1 --2- 172.21.15.26:0/2189261608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcde8160920 0x7fcde80b3f30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.934+0000 7fcdeee00640 1 -- 172.21.15.26:0/2189261608 >> 172.21.15.26:0/2189261608 conn(0x7fcde80d3090 msgr2=0x7fcde80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.935+0000 7fcdeee00640 1 -- 172.21.15.26:0/2189261608 shutdown_connections 2024-01-15T17:59:28.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.935+0000 7fcdeee00640 1 -- 172.21.15.26:0/2189261608 wait complete. 2024-01-15T17:59:28.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.937+0000 7fcdeee00640 1 Processor -- start 2024-01-15T17:59:28.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.937+0000 7fcdeee00640 1 -- start start 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.938+0000 7fcdeee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcde80cf9f0 0x7fcde80b7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.938+0000 7fcdeee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8160920 0x7fcde80cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.938+0000 7fcdeee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 0x7fcde814a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.938+0000 7fcdeee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80cb240 con 0x7fcde8160920 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.938+0000 7fcdeee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80b86f0 con 0x7fcde80cf9f0 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.938+0000 7fcdeee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde80b7b70 con 0x7fcde8161010 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcded000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8160920 0x7fcde80cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcded000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8160920 0x7fcde80cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40992/0 (socket says 172.21.15.26:40992) 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcded000640 1 -- 172.21.15.26:0/747274287 learned_addr learned my addr 172.21.15.26:0/747274287 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcdeda00640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcde80cf9f0 0x7fcde80b7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:28.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcdee400640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 0x7fcde814a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcded000640 1 -- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 msgr2=0x7fcde814a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcded000640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 0x7fcde814a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcded000640 1 -- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcde80cf9f0 msgr2=0x7fcde80b7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcded000640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcde80cf9f0 0x7fcde80b7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.939+0000 7fcded000640 1 -- 172.21.15.26:0/747274287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd80094d0 con 0x7fcde8160920 2024-01-15T17:59:28.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.940+0000 7fcded000640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8160920 0x7fcde80cad00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000e870 tx=0x7fcdd000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.940+0000 7fcde5600640 1 -- 172.21.15.26:0/747274287 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000ee60 con 0x7fcde8160920 2024-01-15T17:59:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.940+0000 7fcde5600640 1 -- 172.21.15.26:0/747274287 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd000c940 con 0x7fcde8160920 2024-01-15T17:59:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.940+0000 7fcde5600640 1 -- 172.21.15.26:0/747274287 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000ccf0 con 0x7fcde8160920 2024-01-15T17:59:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.940+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde80b7e50 con 0x7fcde8160920 2024-01-15T17:59:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.940+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde80b8410 con 0x7fcde8160920 2024-01-15T17:59:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.941+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde8067450 con 0x7fcde8160920 2024-01-15T17:59:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.942+0000 7fcde5600640 1 -- 172.21.15.26:0/747274287 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 89794+0+0 (secure 0 0 0) 0x7fcdd0013470 con 0x7fcde8160920 2024-01-15T17:59:28.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.943+0000 7fcde5600640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdc006b590 0x7fcdc006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:28.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.943+0000 7fcde5600640 1 -- 172.21.15.26:0/747274287 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcdd008a450 con 0x7fcde8160920 2024-01-15T17:59:28.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.944+0000 7fcdeda00640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdc006b590 0x7fcdc006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:28.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.945+0000 7fcdeda00640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdc006b590 0x7fcdc006da50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8036000 tx=0x7fcdd803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:28.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:28.948+0000 7fcde5600640 1 -- 172.21.15.26:0/747274287 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcdd0054f60 con 0x7fcde8160920 2024-01-15T17:59:29.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.180+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6931", "target": ["mon-mgr", ""]}) v1 -- 0x7fcde80ceda0 con 0x7fcdc006b590 2024-01-15T17:59:29.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.376+0000 7fcde5600640 1 -- 172.21.15.26:0/747274287 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcdd803a420 con 0x7fcdc006b590 2024-01-15T17:59:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.380+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdc006b590 msgr2=0x7fcdc006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.380+0000 7fcdeee00640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdc006b590 0x7fcdc006da50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8036000 tx=0x7fcdd803c040 comp rx=0 tx=0).stop 2024-01-15T17:59:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.380+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8160920 msgr2=0x7fcde80cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.380+0000 7fcdeee00640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8160920 0x7fcde80cad00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000e870 tx=0x7fcdd000ed30 comp rx=0 tx=0).stop 2024-01-15T17:59:29.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.381+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 shutdown_connections 2024-01-15T17:59:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.381+0000 7fcdeee00640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcde8161010 0x7fcde814a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.381+0000 7fcdeee00640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdc006b590 0x7fcdc006da50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.381+0000 7fcdeee00640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcde8160920 0x7fcde80cad00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.381+0000 7fcdeee00640 1 --2- 172.21.15.26:0/747274287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcde80cf9f0 0x7fcde80b7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.381+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 >> 172.21.15.26:0/747274287 conn(0x7fcde80d3090 msgr2=0x7fcde806aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.381+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 shutdown_connections 2024-01-15T17:59:29.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.381+0000 7fcdeee00640 1 -- 172.21.15.26:0/747274287 wait complete. 2024-01-15T17:59:29.390 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.386+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.390 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.386+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.390 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.386+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.390 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.386+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.390 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.386+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.405 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6932 2024-01-15T17:59:29.421 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.418+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.421 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.418+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.421 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.418+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.422 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.418+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.422 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:29.418+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:29.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.621+0000 7fdbafe00640 1 Processor -- start 2024-01-15T17:59:29.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.621+0000 7fdbafe00640 1 -- start start 2024-01-15T17:59:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.623+0000 7fdbafe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdbb00cc500 0x7fdbb00cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.623+0000 7fdbafe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb0070e50 con 0x7fdbb006d160 2024-01-15T17:59:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.623+0000 7fdbafe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb0144020 con 0x7fdbb0143c40 2024-01-15T17:59:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.623+0000 7fdbafe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb00723c0 con 0x7fdbb00cc500 2024-01-15T17:59:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.623+0000 7fdbaf400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdbb006d160 0x7fdbb006d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33198/0 (socket says 172.21.15.26:33198) 2024-01-15T17:59:29.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.623+0000 7fdbaf400640 1 -- 172.21.15.26:0/1114946833 learned_addr learned my addr 172.21.15.26:0/1114946833 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:59:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.623+0000 7fdbae000640 1 --2- 172.21.15.26:0/1114946833 >> v2:172.21.15.124:3300/0 conn(0x7fdbb00cc500 0x7fdbb00cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:29.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684038035 0 0) 0x7fdbb0070e50 con 0x7fdbb006d160 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb90003520 con 0x7fdbb006d160 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1259999928 0 0) 0x7fdb90003520 con 0x7fdbb006d160 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 >> v2:172.21.15.124:3300/0 conn(0x7fdbb00cc500 msgr2=0x7fdbb00cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 --2- 172.21.15.26:0/1114946833 >> v2:172.21.15.124:3300/0 conn(0x7fdbb00cc500 0x7fdbb00cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 >> v1:172.21.15.124:6789/0 conn(0x7fdbb0143c40 legacy=0x7fdbb0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb0071e00 con 0x7fdbb006d160 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fdba80019f0 con 0x7fdbb006d160 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 >> v1:172.21.15.26:6789/0 conn(0x7fdbb006d160 legacy=0x7fdbb006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb90003970 0x7fdb90003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb900047c0 0x7fdb90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb90005180 0x7fdb9000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:29.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb00723c0 con 0x7fdb90005180 2024-01-15T17:59:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9000c120 con 0x7fdb900047c0 2024-01-15T17:59:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9000c2e0 con 0x7fdb90003970 2024-01-15T17:59:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.624+0000 7fdbaf400640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb90005180 0x7fdb9000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.625+0000 7fdbae000640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb900047c0 0x7fdb90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.625+0000 7fdbaf400640 1 -- 172.21.15.26:0/1114946833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb90003970 msgr2=0x7fdb90003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.625+0000 7fdbaf400640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb90003970 0x7fdb90003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.625+0000 7fdbaf400640 1 -- 172.21.15.26:0/1114946833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb900047c0 msgr2=0x7fdb90004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.625+0000 7fdbaf400640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb900047c0 0x7fdb90004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.625+0000 7fdbaf400640 1 -- 172.21.15.26:0/1114946833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb90003520 con 0x7fdb90005180 2024-01-15T17:59:29.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.626+0000 7fdbaf400640 1 -- 172.21.15.26:0/1114946833 reap_dead start 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.626+0000 7fdbaf400640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb90005180 0x7fdb9000b980 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdba800a830 tx=0x7fdb900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0dabac1acd43cb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.626+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba80072d0 con 0x7fdb90005180 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.626+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba802f070 con 0x7fdb90005180 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.626+0000 7fdbad600640 1 -- 172.21.15.26:0/1114946833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba80076b0 con 0x7fdb90005180 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.627+0000 7fdbafe00640 1 -- 172.21.15.26:0/1114946833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb90005180 msgr2=0x7fdb9000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.627+0000 7fdbafe00640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb90005180 0x7fdb9000b980 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdba800a830 tx=0x7fdb900057f0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.627+0000 7fdbafe00640 1 -- 172.21.15.26:0/1114946833 shutdown_connections 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.627+0000 7fdbafe00640 1 --2- 172.21.15.26:0/1114946833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb90005180 0x7fdb9000b980 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.627+0000 7fdbafe00640 1 -- 172.21.15.26:0/1114946833 >> 172.21.15.26:0/1114946833 conn(0x7fdbb006a800 msgr2=0x7fdbb006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.627+0000 7fdbafe00640 1 -- 172.21.15.26:0/1114946833 shutdown_connections 2024-01-15T17:59:29.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.627+0000 7fdbafe00640 1 -- 172.21.15.26:0/1114946833 wait complete. 2024-01-15T17:59:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.629+0000 7fdbafe00640 1 Processor -- start 2024-01-15T17:59:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.629+0000 7fdbafe00640 1 -- start start 2024-01-15T17:59:29.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.630+0000 7fdbafe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb90005180 0x7fdbb01375a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:29.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.630+0000 7fdbafe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbb015b860 0x7fdbb015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:29.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.630+0000 7fdbafe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0141ae0 0x7fdbb0141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:29.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.630+0000 7fdbafe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb01424a0 con 0x7fdbb0141ae0 2024-01-15T17:59:29.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.630+0000 7fdbafe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb015d7e0 con 0x7fdbb015b860 2024-01-15T17:59:29.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.630+0000 7fdbafe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb015c360 con 0x7fdb90005180 2024-01-15T17:59:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbaf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0141ae0 0x7fdbb0141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbaf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0141ae0 0x7fdbb0141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41008/0 (socket says 172.21.15.26:41008) 2024-01-15T17:59:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbaf400640 1 -- 172.21.15.26:0/750516601 learned_addr learned my addr 172.21.15.26:0/750516601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbae000640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbb015b860 0x7fdbb015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbaea00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb90005180 0x7fdbb01375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbaf400640 1 -- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb90005180 msgr2=0x7fdbb01375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbaf400640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb90005180 0x7fdbb01375a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbaf400640 1 -- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbb015b860 msgr2=0x7fdbb015bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:29.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbaf400640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbb015b860 0x7fdbb015bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbaf400640 1 -- 172.21.15.26:0/750516601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb015da60 con 0x7fdbb0141ae0 2024-01-15T17:59:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.631+0000 7fdbae000640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbb015b860 0x7fdbb015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:59:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.632+0000 7fdbaf400640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0141ae0 0x7fdbb0141f60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdba800a950 tx=0x7fdba8001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.632+0000 7fdba6a00640 1 -- 172.21.15.26:0/750516601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8044070 con 0x7fdbb0141ae0 2024-01-15T17:59:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.632+0000 7fdba6a00640 1 -- 172.21.15.26:0/750516601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba80019c0 con 0x7fdbb0141ae0 2024-01-15T17:59:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.632+0000 7fdba6a00640 1 -- 172.21.15.26:0/750516601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba8007650 con 0x7fdbb0141ae0 2024-01-15T17:59:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.632+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb015dd40 con 0x7fdbb0141ae0 2024-01-15T17:59:29.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.632+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb015e300 con 0x7fdbb0141ae0 2024-01-15T17:59:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.633+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb0067450 con 0x7fdbb0141ae0 2024-01-15T17:59:29.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.634+0000 7fdba6a00640 1 -- 172.21.15.26:0/750516601 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 89794+0+0 (secure 0 0 0) 0x7fdba8031030 con 0x7fdbb0141ae0 2024-01-15T17:59:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.636+0000 7fdba6a00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8806b5c0 0x7fdb8806da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.636+0000 7fdba6a00640 1 -- 172.21.15.26:0/750516601 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdba80ab6d0 con 0x7fdbb0141ae0 2024-01-15T17:59:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.636+0000 7fdbaea00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8806b5c0 0x7fdb8806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:29.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.637+0000 7fdbaea00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8806b5c0 0x7fdb8806da80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdba00016f0 tx=0x7fdba0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.640+0000 7fdba6a00640 1 -- 172.21.15.26:0/750516601 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdba8076160 con 0x7fdbb0141ae0 2024-01-15T17:59:29.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:29.875+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6932", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbb006cc40 con 0x7fdb8806b5c0 2024-01-15T17:59:30.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.047+0000 7fdba6a00640 1 -- 172.21.15.26:0/750516601 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdba00013b0 con 0x7fdb8806b5c0 2024-01-15T17:59:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8806b5c0 msgr2=0x7fdb8806da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8806b5c0 0x7fdb8806da80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdba00016f0 tx=0x7fdba0009040 comp rx=0 tx=0).stop 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0141ae0 msgr2=0x7fdbb0141f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0141ae0 0x7fdbb0141f60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdba800a950 tx=0x7fdba8001d80 comp rx=0 tx=0).stop 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 shutdown_connections 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb8806b5c0 0x7fdb8806da80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbb0141ae0 0x7fdbb0141f60 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbb015b860 0x7fdbb015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 --2- 172.21.15.26:0/750516601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb90005180 0x7fdbb01375a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 >> 172.21.15.26:0/750516601 conn(0x7fdbb006a800 msgr2=0x7fdbb00cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 shutdown_connections 2024-01-15T17:59:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.051+0000 7fdbafe00640 1 -- 172.21.15.26:0/750516601 wait complete. 2024-01-15T17:59:30.077 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6933 2024-01-15T17:59:30.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.298+0000 7f78e1a00640 1 Processor -- start 2024-01-15T17:59:30.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.299+0000 7f78e1a00640 1 -- start start 2024-01-15T17:59:30.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.300+0000 7f78e1a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f78dc0d0900 0x7f78dc0adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:30.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.300+0000 7f78e1a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78dc0b2170 con 0x7f78dc15b7f0 2024-01-15T17:59:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.300+0000 7f78e1a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78dc0b2450 con 0x7f78dc15bbd0 2024-01-15T17:59:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.300+0000 7f78e1a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78dc0b2730 con 0x7f78dc0d0900 2024-01-15T17:59:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.301+0000 7f78dbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f78dc15b7f0 0x7f78dc0d17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33208/0 (socket says 172.21.15.26:33208) 2024-01-15T17:59:30.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.301+0000 7f78dbe00640 1 -- 172.21.15.26:0/2671432881 learned_addr learned my addr 172.21.15.26:0/2671432881 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:59:30.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.301+0000 7f78d3e00640 1 --2- 172.21.15.26:0/2671432881 >> v2:172.21.15.124:3300/0 conn(0x7f78dc0d0900 0x7f78dc0adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:30.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.301+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195456563 0 0) 0x7f78dc0b2170 con 0x7f78dc15b7f0 2024-01-15T17:59:30.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.301+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78bc003520 con 0x7f78dc15b7f0 2024-01-15T17:59:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1123575592 0 0) 0x7f78bc003520 con 0x7f78dc15b7f0 2024-01-15T17:59:30.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 >> v2:172.21.15.124:3300/0 conn(0x7f78dc0d0900 msgr2=0x7f78dc0adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 --2- 172.21.15.26:0/2671432881 >> v2:172.21.15.124:3300/0 conn(0x7f78dc0d0900 0x7f78dc0adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 >> v1:172.21.15.124:6789/0 conn(0x7f78dc15bbd0 legacy=0x7f78dc15c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc0b08d0 con 0x7f78dc15b7f0 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f78cc001670 con 0x7f78dc15b7f0 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 >> v1:172.21.15.26:6789/0 conn(0x7f78dc15b7f0 legacy=0x7f78dc0d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78bc003970 0x7f78bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc0047c0 0x7f78bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78bc005180 0x7f78bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc0b2730 con 0x7f78bc0047c0 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc00c120 con 0x7f78bc005180 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc00c2e0 con 0x7f78bc003970 2024-01-15T17:59:30.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.302+0000 7f78d3e00640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc0047c0 0x7f78bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.303+0000 7f78dbe00640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78bc003970 0x7f78bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.303+0000 7f78e1000640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78bc005180 0x7f78bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.303+0000 7f78d3e00640 1 -- 172.21.15.26:0/2671432881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78bc003970 msgr2=0x7f78bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.303+0000 7f78d3e00640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78bc003970 0x7f78bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.303+0000 7f78d3e00640 1 -- 172.21.15.26:0/2671432881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78bc005180 msgr2=0x7f78bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.303+0000 7f78d3e00640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78bc005180 0x7f78bc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.303+0000 7f78d3e00640 1 -- 172.21.15.26:0/2671432881 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc003520 con 0x7f78bc0047c0 2024-01-15T17:59:30.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.303+0000 7f78e1000640 1 -- 172.21.15.26:0/2671432881 reap_dead start 2024-01-15T17:59:30.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.304+0000 7f78d3e00640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc0047c0 0x7f78bc004c40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009a90 tx=0x7f78bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20aba777487d0dc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:30.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.304+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c402c680 con 0x7f78bc0047c0 2024-01-15T17:59:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.305+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c402c7e0 con 0x7f78bc0047c0 2024-01-15T17:59:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.305+0000 7f78db400640 1 -- 172.21.15.26:0/2671432881 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c402cb90 con 0x7f78bc0047c0 2024-01-15T17:59:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.305+0000 7f78e1a00640 1 -- 172.21.15.26:0/2671432881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc0047c0 msgr2=0x7f78bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.305+0000 7f78e1a00640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc0047c0 0x7f78bc004c40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009a90 tx=0x7f78bc0057f0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.305+0000 7f78e1a00640 1 -- 172.21.15.26:0/2671432881 shutdown_connections 2024-01-15T17:59:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.305+0000 7f78e1a00640 1 --2- 172.21.15.26:0/2671432881 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78bc0047c0 0x7f78bc004c40 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.305+0000 7f78e1a00640 1 -- 172.21.15.26:0/2671432881 >> 172.21.15.26:0/2671432881 conn(0x7f78dc099a60 msgr2=0x7f78dc099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.305+0000 7f78e1a00640 1 -- 172.21.15.26:0/2671432881 shutdown_connections 2024-01-15T17:59:30.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.305+0000 7f78e1a00640 1 -- 172.21.15.26:0/2671432881 wait complete. 2024-01-15T17:59:30.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.307+0000 7f78e1a00640 1 Processor -- start 2024-01-15T17:59:30.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.307+0000 7f78e1a00640 1 -- start start 2024-01-15T17:59:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.308+0000 7f78e1a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78bc0047c0 0x7f78dc14c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.308+0000 7f78e1a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78dc06c4a0 0x7f78dc06c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.308+0000 7f78e1a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78dc06f270 0x7f78dc06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.308+0000 7f78e1a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc14ca50 con 0x7f78dc06c4a0 2024-01-15T17:59:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.308+0000 7f78e1a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc06ce40 con 0x7f78bc0047c0 2024-01-15T17:59:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.308+0000 7f78e1a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc14d560 con 0x7f78dc06f270 2024-01-15T17:59:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78dc06c4a0 0x7f78dc06c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:30.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78dc06c4a0 0x7f78dc06c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41024/0 (socket says 172.21.15.26:41024) 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78d3e00640 1 -- 172.21.15.26:0/3180796096 learned_addr learned my addr 172.21.15.26:0/3180796096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78e1000640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78dc06f270 0x7f78dc06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78dbe00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78bc0047c0 0x7f78dc14c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78d3e00640 1 -- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78dc06f270 msgr2=0x7f78dc06f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78d3e00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78dc06f270 0x7f78dc06f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78d3e00640 1 -- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78bc0047c0 msgr2=0x7f78dc14c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78d3e00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78bc0047c0 0x7f78dc14c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.309+0000 7f78d3e00640 1 -- 172.21.15.26:0/3180796096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc14cc30 con 0x7f78dc06c4a0 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.310+0000 7f78d3e00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78dc06c4a0 0x7f78dc06c900 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f78c402c2f0 tx=0x7f78c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.310+0000 7f78d8c00640 1 -- 172.21.15.26:0/3180796096 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4034a90 con 0x7f78dc06c4a0 2024-01-15T17:59:30.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.310+0000 7f78d8c00640 1 -- 172.21.15.26:0/3180796096 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c4034c10 con 0x7f78dc06c4a0 2024-01-15T17:59:30.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.310+0000 7f78d8c00640 1 -- 172.21.15.26:0/3180796096 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c403e400 con 0x7f78dc06c4a0 2024-01-15T17:59:30.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.310+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78dc14cf10 con 0x7f78dc06c4a0 2024-01-15T17:59:30.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.310+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc1557c0 con 0x7f78dc06c4a0 2024-01-15T17:59:30.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.312+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78dc067450 con 0x7f78dc06c4a0 2024-01-15T17:59:30.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.312+0000 7f78d8c00640 1 -- 172.21.15.26:0/3180796096 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 10) v1 ==== 89794+0+0 (secure 0 0 0) 0x7f78c403e560 con 0x7f78dc06c4a0 2024-01-15T17:59:30.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.314+0000 7f78d8c00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b406b5c0 0x7f78b406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:30.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.314+0000 7f78d8c00640 1 -- 172.21.15.26:0/3180796096 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f78c40abf70 con 0x7f78dc06c4a0 2024-01-15T17:59:30.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.314+0000 7f78dbe00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b406b5c0 0x7f78b406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:30.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.315+0000 7f78dbe00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b406b5c0 0x7f78b406da80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f78cc001c90 tx=0x7f78cc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:30.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.319+0000 7f78d8c00640 1 -- 172.21.15.26:0/3180796096 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78c4076b30 con 0x7f78dc06c4a0 2024-01-15T17:59:30.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.553+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6933", "target": ["mon-mgr", ""]}) v1 -- 0x7f78dc14d6a0 con 0x7f78b406b5c0 2024-01-15T17:59:30.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:30.943+0000 7f78d8c00640 1 -- 172.21.15.26:0/3180796096 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 11) v1 ==== 89896+0+0 (secure 0 0 0) 0x7f78c4076f00 con 0x7f78dc06c4a0 2024-01-15T17:59:33.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.779+0000 7f78d8c00640 1 -- 172.21.15.26:0/3180796096 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f78dc14d6a0 con 0x7f78b406b5c0 2024-01-15T17:59:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b406b5c0 msgr2=0x7f78b406da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b406b5c0 0x7f78b406da80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f78cc001c90 tx=0x7f78cc009040 comp rx=0 tx=0).stop 2024-01-15T17:59:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78dc06c4a0 msgr2=0x7f78dc06c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78dc06c4a0 0x7f78dc06c900 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f78c402c2f0 tx=0x7f78c4004730 comp rx=0 tx=0).stop 2024-01-15T17:59:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 shutdown_connections 2024-01-15T17:59:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78dc06f270 0x7f78dc06f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:33.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b406b5c0 0x7f78b406da80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78dc06c4a0 0x7f78dc06c900 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 --2- 172.21.15.26:0/3180796096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78bc0047c0 0x7f78dc14c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 >> 172.21.15.26:0/3180796096 conn(0x7f78dc099a60 msgr2=0x7f78dc15da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 shutdown_connections 2024-01-15T17:59:33.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:33.782+0000 7f78e1a00640 1 -- 172.21.15.26:0/3180796096 wait complete. 2024-01-15T17:59:33.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone6934 2024-01-15T17:59:34.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.037+0000 7fdf2be00640 1 Processor -- start 2024-01-15T17:59:34.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.037+0000 7fdf2be00640 1 -- start start 2024-01-15T17:59:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.039+0000 7fdf2be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf2c0cc500 0x7fdf2c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.039+0000 7fdf2be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdf2c06d160 0x7fdf2c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.039+0000 7fdf2be00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c070e50 con 0x7fdf2c143c40 2024-01-15T17:59:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.039+0000 7fdf2be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0723c0 con 0x7fdf2c0cc500 2024-01-15T17:59:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.039+0000 7fdf2be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cced0 con 0x7fdf2c06d160 2024-01-15T17:59:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.040+0000 7fdf21e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf2c0cc500 0x7fdf2c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:34.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.040+0000 7fdf21e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf2c0cc500 0x7fdf2c0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41552/0 (socket says 172.21.15.26:41552) 2024-01-15T17:59:34.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.040+0000 7fdf21e00640 1 -- 172.21.15.26:0/1237026233 learned_addr learned my addr 172.21.15.26:0/1237026233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:34.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.040+0000 7fdf2b400640 1 --2- 172.21.15.26:0/1237026233 >> v2:172.21.15.124:3300/0 conn(0x7fdf2c06d160 0x7fdf2c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:34.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.040+0000 7fdf21e00640 1 -- 172.21.15.26:0/1237026233 >> v2:172.21.15.124:3300/0 conn(0x7fdf2c06d160 msgr2=0x7fdf2c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:34.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.040+0000 7fdf21e00640 1 --2- 172.21.15.26:0/1237026233 >> v2:172.21.15.124:3300/0 conn(0x7fdf2c06d160 0x7fdf2c06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:34.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.040+0000 7fdf21e00640 1 -- 172.21.15.26:0/1237026233 >> v1:172.21.15.124:6789/0 conn(0x7fdf2c143c40 legacy=0x7fdf2c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:34.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.040+0000 7fdf21e00640 1 -- 172.21.15.26:0/1237026233 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c144020 con 0x7fdf2c0cc500 2024-01-15T17:59:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.041+0000 7fdf21e00640 1 --2- 172.21.15.26:0/1237026233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 0x7fdf2c0cc990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c070690 tx=0x7fdf1402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c55e4a824fb1b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.042+0000 7fdf2a000640 1 -- 172.21.15.26:0/1237026233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140317e0 con 0x7fdf2c0cc500 2024-01-15T17:59:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.042+0000 7fdf2a000640 1 -- 172.21.15.26:0/1237026233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf14031940 con 0x7fdf2c0cc500 2024-01-15T17:59:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.042+0000 7fdf2a000640 1 -- 172.21.15.26:0/1237026233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14031cd0 con 0x7fdf2c0cc500 2024-01-15T17:59:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.042+0000 7fdf2be00640 1 -- 172.21.15.26:0/1237026233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 msgr2=0x7fdf2c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.042+0000 7fdf2be00640 1 --2- 172.21.15.26:0/1237026233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 0x7fdf2c0cc990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c070690 tx=0x7fdf1402b880 comp rx=0 tx=0).stop 2024-01-15T17:59:34.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.043+0000 7fdf2be00640 1 -- 172.21.15.26:0/1237026233 shutdown_connections 2024-01-15T17:59:34.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.043+0000 7fdf2be00640 1 --2- 172.21.15.26:0/1237026233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 0x7fdf2c0cc990 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:34.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.043+0000 7fdf2be00640 1 --2- 172.21.15.26:0/1237026233 >> v2:172.21.15.124:3300/0 conn(0x7fdf2c06d160 0x7fdf2c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:34.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.043+0000 7fdf2be00640 1 -- 172.21.15.26:0/1237026233 >> 172.21.15.26:0/1237026233 conn(0x7fdf2c06a800 msgr2=0x7fdf2c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:34.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.043+0000 7fdf2be00640 1 -- 172.21.15.26:0/1237026233 shutdown_connections 2024-01-15T17:59:34.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.043+0000 7fdf2be00640 1 -- 172.21.15.26:0/1237026233 wait complete. 2024-01-15T17:59:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.045+0000 7fdf2be00640 1 Processor -- start 2024-01-15T17:59:34.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.045+0000 7fdf2be00640 1 -- start start 2024-01-15T17:59:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf2c06d160 0x7fdf2c13aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 0x7fdf2c1f0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf2be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c143c40 0x7fdf2c142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf2be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c13b3e0 con 0x7fdf2c0cc500 2024-01-15T17:59:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf2be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c139e00 con 0x7fdf2c06d160 2024-01-15T17:59:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf2be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c13a0e0 con 0x7fdf2c143c40 2024-01-15T17:59:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf21e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 0x7fdf2c1f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf21e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 0x7fdf2c1f0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41562/0 (socket says 172.21.15.26:41562) 2024-01-15T17:59:34.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf21e00640 1 -- 172.21.15.26:0/33486598 learned_addr learned my addr 172.21.15.26:0/33486598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.046+0000 7fdf2aa00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf2c06d160 0x7fdf2c13aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf21e00640 1 -- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c143c40 msgr2=0x7fdf2c142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf21e00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c143c40 0x7fdf2c142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf21e00640 1 -- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf2c06d160 msgr2=0x7fdf2c13aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf21e00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf2c06d160 0x7fdf2c13aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf21e00640 1 -- 172.21.15.26:0/33486598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf140094d0 con 0x7fdf2c0cc500 2024-01-15T17:59:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf21e00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 0x7fdf2c1f0980 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdf140090b0 tx=0x7fdf140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf23400640 1 -- 172.21.15.26:0/33486598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14032880 con 0x7fdf2c0cc500 2024-01-15T17:59:34.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf23400640 1 -- 172.21.15.26:0/33486598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf14032a00 con 0x7fdf2c0cc500 2024-01-15T17:59:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf23400640 1 -- 172.21.15.26:0/33486598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14032db0 con 0x7fdf2c0cc500 2024-01-15T17:59:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.047+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c13a3c0 con 0x7fdf2c0cc500 2024-01-15T17:59:34.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.048+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c13a980 con 0x7fdf2c0cc500 2024-01-15T17:59:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.049+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c13ab70 con 0x7fdf2c0cc500 2024-01-15T17:59:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.050+0000 7fdf23400640 1 -- 172.21.15.26:0/33486598 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 11) v1 ==== 89896+0+0 (secure 0 0 0) 0x7fdf1404d3f0 con 0x7fdf2c0cc500 2024-01-15T17:59:34.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.051+0000 7fdf23400640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf0406b6b0 0x7fdf0406db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.051+0000 7fdf23400640 1 -- 172.21.15.26:0/33486598 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdf140ac280 con 0x7fdf2c0cc500 2024-01-15T17:59:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.052+0000 7fdf2aa00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf0406b6b0 0x7fdf0406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:34.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.053+0000 7fdf2aa00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf0406b6b0 0x7fdf0406db70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009f20 tx=0x7fdf1c0094d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:34.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.056+0000 7fdf23400640 1 -- 172.21.15.26:0/33486598 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf14076a10 con 0x7fdf2c0cc500 2024-01-15T17:59:34.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:34.286+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone6934", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf2c06c960 con 0x7fdf0406b6b0 2024-01-15T17:59:38.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.763+0000 7fdf23400640 1 -- 172.21.15.26:0/33486598 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdf1c017400 con 0x7fdf0406b6b0 2024-01-15T17:59:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf0406b6b0 msgr2=0x7fdf0406db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf0406b6b0 0x7fdf0406db70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009f20 tx=0x7fdf1c0094d0 comp rx=0 tx=0).stop 2024-01-15T17:59:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 msgr2=0x7fdf2c1f0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 0x7fdf2c1f0980 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdf140090b0 tx=0x7fdf140046c0 comp rx=0 tx=0).stop 2024-01-15T17:59:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 shutdown_connections 2024-01-15T17:59:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf2c143c40 0x7fdf2c142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:38.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf0406b6b0 0x7fdf0406db70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf2c0cc500 0x7fdf2c1f0980 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 --2- 172.21.15.26:0/33486598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf2c06d160 0x7fdf2c13aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 >> 172.21.15.26:0/33486598 conn(0x7fdf2c06a800 msgr2=0x7fdf2c071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 shutdown_connections 2024-01-15T17:59:38.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:38.767+0000 7fdf2be00640 1 -- 172.21.15.26:0/33486598 wait complete. 2024-01-15T17:59:38.796 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T17:59:38.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone6931 2024-01-15T17:59:39.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.023+0000 7fef1bc00640 1 Processor -- start 2024-01-15T17:59:39.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.023+0000 7fef1bc00640 1 -- start start 2024-01-15T17:59:39.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.024+0000 7fef1bc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef140ce8f0 0x7fef140ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:39.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef1bc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 0x7fef140b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:39.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef1bc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef140d0de0 con 0x7fef14160f80 2024-01-15T17:59:39.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef1bc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef140d10c0 con 0x7fef140ce8f0 2024-01-15T17:59:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef1bc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef140d13a0 con 0x7fef140b2a00 2024-01-15T17:59:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef13e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef140ce8f0 0x7fef140ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef13e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef140ce8f0 0x7fef140ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41570/0 (socket says 172.21.15.26:41570) 2024-01-15T17:59:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef13e00640 1 -- 172.21.15.26:0/513034489 learned_addr learned my addr 172.21.15.26:0/513034489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef19200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fef14160f80 0x7fef14161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35786/0 (socket says 172.21.15.26:35786) 2024-01-15T17:59:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef19c00640 1 --2- 172.21.15.26:0/513034489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 0x7fef140b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef13e00640 1 -- 172.21.15.26:0/513034489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 msgr2=0x7fef140b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef13e00640 1 --2- 172.21.15.26:0/513034489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 0x7fef140b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:39.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.025+0000 7fef13e00640 1 -- 172.21.15.26:0/513034489 >> v1:172.21.15.26:6789/0 conn(0x7fef14160f80 legacy=0x7fef14161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:39.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.026+0000 7fef13e00640 1 -- 172.21.15.26:0/513034489 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef140d1680 con 0x7fef140ce8f0 2024-01-15T17:59:39.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.026+0000 7fef13400640 1 -- 172.21.15.26:0/513034489 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549255892 0 0) 0x7fef140d0de0 con 0x7fef14160f80 2024-01-15T17:59:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.027+0000 7fef13e00640 1 --2- 172.21.15.26:0/513034489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 0x7fef140ced30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feefc009870 tx=0x7feefc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e464c222147b403d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:39.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.027+0000 7fef13400640 1 -- 172.21.15.26:0/513034489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc030840 con 0x7fef140ce8f0 2024-01-15T17:59:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.027+0000 7fef13400640 1 -- 172.21.15.26:0/513034489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc0309a0 con 0x7fef140ce8f0 2024-01-15T17:59:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.027+0000 7fef13400640 1 -- 172.21.15.26:0/513034489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc030d30 con 0x7fef140ce8f0 2024-01-15T17:59:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.028+0000 7fef1bc00640 1 -- 172.21.15.26:0/513034489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 msgr2=0x7fef140ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.028+0000 7fef1bc00640 1 --2- 172.21.15.26:0/513034489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 0x7fef140ced30 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feefc009870 tx=0x7feefc02b980 comp rx=0 tx=0).stop 2024-01-15T17:59:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.028+0000 7fef1bc00640 1 -- 172.21.15.26:0/513034489 shutdown_connections 2024-01-15T17:59:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.028+0000 7fef1bc00640 1 --2- 172.21.15.26:0/513034489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 0x7fef140b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.028+0000 7fef1bc00640 1 --2- 172.21.15.26:0/513034489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 0x7fef140ced30 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.028+0000 7fef1bc00640 1 -- 172.21.15.26:0/513034489 >> 172.21.15.26:0/513034489 conn(0x7fef1409c550 msgr2=0x7fef140d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:39.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.028+0000 7fef1bc00640 1 -- 172.21.15.26:0/513034489 shutdown_connections 2024-01-15T17:59:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.028+0000 7fef1bc00640 1 -- 172.21.15.26:0/513034489 wait complete. 2024-01-15T17:59:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.030+0000 7fef1bc00640 1 Processor -- start 2024-01-15T17:59:39.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.030+0000 7fef1bc00640 1 -- start start 2024-01-15T17:59:39.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef1bc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 0x7fef140c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:39.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef1bc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 0x7fef141e7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:39.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef1bc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef14160f80 0x7fef140b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef1bc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef140c8740 con 0x7fef140ce8f0 2024-01-15T17:59:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef1bc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef140c8a20 con 0x7fef14160f80 2024-01-15T17:59:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef1bc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef140c8d00 con 0x7fef140b2a00 2024-01-15T17:59:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef13e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 0x7fef141e7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef13e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 0x7fef141e7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41584/0 (socket says 172.21.15.26:41584) 2024-01-15T17:59:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef13e00640 1 -- 172.21.15.26:0/3821521712 learned_addr learned my addr 172.21.15.26:0/3821521712 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef19c00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef14160f80 0x7fef140b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef19200640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 0x7fef140c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:39.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.032+0000 7fef13e00640 1 -- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 msgr2=0x7fef140c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef13e00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 0x7fef140c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef13e00640 1 -- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef14160f80 msgr2=0x7fef140b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef13e00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef14160f80 0x7fef140b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef13e00640 1 -- 172.21.15.26:0/3821521712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0094d0 con 0x7fef140ce8f0 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef19200640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 0x7fef140c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef19c00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef14160f80 0x7fef140b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef13e00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 0x7fef141e7fa0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef10c00640 1 -- 172.21.15.26:0/3821521712 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc047070 con 0x7fef140ce8f0 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef10c00640 1 -- 172.21.15.26:0/3821521712 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc039a90 con 0x7fef140ce8f0 2024-01-15T17:59:39.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef10c00640 1 -- 172.21.15.26:0/3821521712 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc039e20 con 0x7fef140ce8f0 2024-01-15T17:59:39.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef140c8fe0 con 0x7fef140ce8f0 2024-01-15T17:59:39.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.033+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef141e84e0 con 0x7fef140ce8f0 2024-01-15T17:59:39.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.035+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef140c9500 con 0x7fef140ce8f0 2024-01-15T17:59:39.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.036+0000 7fef10c00640 1 -- 172.21.15.26:0/3821521712 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 11) v1 ==== 89896+0+0 (secure 0 0 0) 0x7feefc04c410 con 0x7fef140ce8f0 2024-01-15T17:59:39.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.036+0000 7fef10c00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feeec06b6b0 0x7feeec06db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:39.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.037+0000 7fef10c00640 1 -- 172.21.15.26:0/3821521712 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7feefc03e070 con 0x7fef140ce8f0 2024-01-15T17:59:39.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.037+0000 7fef19200640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feeec06b6b0 0x7feeec06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:39.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.038+0000 7fef19200640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feeec06b6b0 0x7feeec06db70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fef04000f60 tx=0x7fef04009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:39.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.041+0000 7fef10c00640 1 -- 172.21.15.26:0/3821521712 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef140c9500 con 0x7fef140ce8f0 2024-01-15T17:59:39.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:39.274+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone6931", "target": ["mon-mgr", ""]}) v1 -- 0x7fef141e8670 con 0x7feeec06b6b0 2024-01-15T17:59:43.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.606+0000 7fef10c00640 1 -- 172.21.15.26:0/3821521712 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fef040012f0 con 0x7feeec06b6b0 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:43.610 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:59:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.609+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feeec06b6b0 msgr2=0x7feeec06db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:43.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.609+0000 7fef1bc00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feeec06b6b0 0x7feeec06db70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fef04000f60 tx=0x7fef04009040 comp rx=0 tx=0).stop 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.609+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 msgr2=0x7fef141e7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.609+0000 7fef1bc00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 0x7fef141e7fa0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7feefc000c00 tx=0x7feefc004730 comp rx=0 tx=0).stop 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.610+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 shutdown_connections 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.610+0000 7fef1bc00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feeec06b6b0 0x7feeec06db70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.610+0000 7fef1bc00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef14160f80 0x7fef140b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.610+0000 7fef1bc00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef140ce8f0 0x7fef141e7fa0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.610+0000 7fef1bc00640 1 --2- 172.21.15.26:0/3821521712 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef140b2a00 0x7fef140c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.610+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 >> 172.21.15.26:0/3821521712 conn(0x7fef1409c550 msgr2=0x7fef140b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.610+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 shutdown_connections 2024-01-15T17:59:43.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:43.610+0000 7fef1bc00640 1 -- 172.21.15.26:0/3821521712 wait complete. 2024-01-15T17:59:43.636 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T17:59:44.637 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone6931 2024-01-15T17:59:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.873+0000 7f4ec6000640 1 Processor -- start 2024-01-15T17:59:44.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.873+0000 7f4ec6000640 1 -- start start 2024-01-15T17:59:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.875+0000 7f4ec6000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ec013df80 0x7f4ec015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.875+0000 7f4ec6000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4ec013e360 0x7f4ec00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.875+0000 7f4ec6000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec006c470 con 0x7f4ec015f9c0 2024-01-15T17:59:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.875+0000 7f4ec6000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec006c750 con 0x7f4ec013df80 2024-01-15T17:59:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.875+0000 7f4ec6000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec006ca30 con 0x7f4ec013e360 2024-01-15T17:59:44.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.876+0000 7f4ebf400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ec013df80 0x7f4ec015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.876+0000 7f4ebf400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ec013df80 0x7f4ec015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44434/0 (socket says 172.21.15.26:44434) 2024-01-15T17:59:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.876+0000 7f4ebf400640 1 -- 172.21.15.26:0/2291887527 learned_addr learned my addr 172.21.15.26:0/2291887527 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.876+0000 7f4eb7e00640 1 --2- 172.21.15.26:0/2291887527 >> v2:172.21.15.124:3300/0 conn(0x7f4ec013e360 0x7f4ec00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.876+0000 7f4ebf400640 1 -- 172.21.15.26:0/2291887527 >> v2:172.21.15.124:3300/0 conn(0x7f4ec013e360 msgr2=0x7f4ec00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.876+0000 7f4ebf400640 1 --2- 172.21.15.26:0/2291887527 >> v2:172.21.15.124:3300/0 conn(0x7f4ec013e360 0x7f4ec00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.876+0000 7f4ebf400640 1 -- 172.21.15.26:0/2291887527 >> v1:172.21.15.124:6789/0 conn(0x7f4ec015f9c0 legacy=0x7f4ec01408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:44.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.876+0000 7f4ebf400640 1 -- 172.21.15.26:0/2291887527 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec006cd10 con 0x7f4ec013df80 2024-01-15T17:59:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.878+0000 7f4ebf400640 1 --2- 172.21.15.26:0/2291887527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec013df80 0x7f4ec015bee0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4ec006ded0 tx=0x7f4eb002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adea0550066038b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:44.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.878+0000 7f4ebea00640 1 -- 172.21.15.26:0/2291887527 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0030860 con 0x7f4ec013df80 2024-01-15T17:59:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.878+0000 7f4ebea00640 1 -- 172.21.15.26:0/2291887527 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb00309c0 con 0x7f4ec013df80 2024-01-15T17:59:44.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.878+0000 7f4ebea00640 1 -- 172.21.15.26:0/2291887527 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0030d50 con 0x7f4ec013df80 2024-01-15T17:59:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.879+0000 7f4ec6000640 1 -- 172.21.15.26:0/2291887527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec013df80 msgr2=0x7f4ec015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.879+0000 7f4ec6000640 1 --2- 172.21.15.26:0/2291887527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec013df80 0x7f4ec015bee0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4ec006ded0 tx=0x7f4eb002b9a0 comp rx=0 tx=0).stop 2024-01-15T17:59:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.879+0000 7f4ec6000640 1 -- 172.21.15.26:0/2291887527 shutdown_connections 2024-01-15T17:59:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.879+0000 7f4ec6000640 1 --2- 172.21.15.26:0/2291887527 >> v2:172.21.15.124:3300/0 conn(0x7f4ec013e360 0x7f4ec00cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.879+0000 7f4ec6000640 1 --2- 172.21.15.26:0/2291887527 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec013df80 0x7f4ec015bee0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.879+0000 7f4ec6000640 1 -- 172.21.15.26:0/2291887527 >> 172.21.15.26:0/2291887527 conn(0x7f4ec006a800 msgr2=0x7f4ec006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.879+0000 7f4ec6000640 1 -- 172.21.15.26:0/2291887527 shutdown_connections 2024-01-15T17:59:44.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.879+0000 7f4ec6000640 1 -- 172.21.15.26:0/2291887527 wait complete. 2024-01-15T17:59:44.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.881+0000 7f4ec6000640 1 Processor -- start 2024-01-15T17:59:44.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.881+0000 7f4ec6000640 1 -- start start 2024-01-15T17:59:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.882+0000 7f4ec6000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ec013df80 0x7f4ec0070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.882+0000 7f4ec6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ec013e360 0x7f4ec014aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.882+0000 7f4ec6000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec015f9c0 0x7f4ec00a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.882+0000 7f4ec6000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0071f70 con 0x7f4ec015f9c0 2024-01-15T17:59:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.882+0000 7f4ec6000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0071360 con 0x7f4ec013df80 2024-01-15T17:59:44.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.882+0000 7f4ec6000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0071640 con 0x7f4ec013e360 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec015f9c0 0x7f4ec00a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec015f9c0 0x7f4ec00a9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44440/0 (socket says 172.21.15.26:44440) 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebfe00640 1 -- 172.21.15.26:0/394981861 learned_addr learned my addr 172.21.15.26:0/394981861 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebf400640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ec013df80 0x7f4ec0070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4eb7e00640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ec013e360 0x7f4ec014aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebfe00640 1 -- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ec013e360 msgr2=0x7f4ec014aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebfe00640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ec013e360 0x7f4ec014aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebfe00640 1 -- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ec013df80 msgr2=0x7f4ec0070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebfe00640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ec013df80 0x7f4ec0070d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebfe00640 1 -- 172.21.15.26:0/394981861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb00094d0 con 0x7f4ec015f9c0 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.883+0000 7f4ebfe00640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec015f9c0 0x7f4ec00a9290 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c730 tx=0x7f4eb800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:44.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.884+0000 7f4eb7400640 1 -- 172.21.15.26:0/394981861 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8003b70 con 0x7f4ec015f9c0 2024-01-15T17:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.884+0000 7f4eb7400640 1 -- 172.21.15.26:0/394981861 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb8003cd0 con 0x7f4ec015f9c0 2024-01-15T17:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.884+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec0071920 con 0x7f4ec015f9c0 2024-01-15T17:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.884+0000 7f4eb7400640 1 -- 172.21.15.26:0/394981861 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800aa90 con 0x7f4ec015f9c0 2024-01-15T17:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.884+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec014b410 con 0x7f4ec015f9c0 2024-01-15T17:59:44.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.885+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec00675d0 con 0x7f4ec015f9c0 2024-01-15T17:59:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.886+0000 7f4eb7400640 1 -- 172.21.15.26:0/394981861 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 11) v1 ==== 89896+0+0 (secure 0 0 0) 0x7f4eb800abf0 con 0x7f4ec015f9c0 2024-01-15T17:59:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.887+0000 7f4eb7400640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e9806b6b0 0x7f4e9806db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:44.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.887+0000 7f4eb7400640 1 -- 172.21.15.26:0/394981861 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4eb808a150 con 0x7f4ec015f9c0 2024-01-15T17:59:44.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.888+0000 7f4ebf400640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e9806b6b0 0x7f4e9806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:44.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.889+0000 7f4ebf400640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e9806b6b0 0x7f4e9806db70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0036000 tx=0x7f4eb003b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:44.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:44.891+0000 7f4eb7400640 1 -- 172.21.15.26:0/394981861 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4eb80549d0 con 0x7f4ec015f9c0 2024-01-15T17:59:45.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.126+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone6931", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ec00ce850 con 0x7f4e9806b6b0 2024-01-15T17:59:45.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.150+0000 7f4eb7400640 1 -- 172.21.15.26:0/394981861 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f4eb0030c00 con 0x7f4e9806b6b0 2024-01-15T17:59:45.154 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:59:45.154 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T17:59:45.154 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T17:59:45.154 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T17:59:45.154 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T17:59:45.154 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T17:59:45.154 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T17:59:45.154 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:45.155 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T17:59:45.155 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 0.0, 2024-01-15T17:59:45.155 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "0 B/3 GB", 2024-01-15T17:59:45.155 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "0/3", 2024-01-15T17:59:45.155 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T17:59:45.155 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T17:59:45.155 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:45.155 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:45.155 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:59:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.153+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e9806b6b0 msgr2=0x7f4e9806db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.153+0000 7f4ec6000640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e9806b6b0 0x7f4e9806db70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0036000 tx=0x7f4eb003b040 comp rx=0 tx=0).stop 2024-01-15T17:59:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.153+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec015f9c0 msgr2=0x7f4ec00a9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:45.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.153+0000 7f4ec6000640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec015f9c0 0x7f4ec00a9290 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c730 tx=0x7f4eb800cbf0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.153+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 shutdown_connections 2024-01-15T17:59:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.153+0000 7f4ec6000640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ec013e360 0x7f4ec014aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.153+0000 7f4ec6000640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e9806b6b0 0x7f4e9806db70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.154+0000 7f4ec6000640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ec015f9c0 0x7f4ec00a9290 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.154+0000 7f4ec6000640 1 --2- 172.21.15.26:0/394981861 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ec013df80 0x7f4ec0070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.154+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 >> 172.21.15.26:0/394981861 conn(0x7f4ec006a800 msgr2=0x7f4ec0155620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.154+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 shutdown_connections 2024-01-15T17:59:45.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.154+0000 7f4ec6000640 1 -- 172.21.15.26:0/394981861 wait complete. 2024-01-15T17:59:45.176 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T17:59:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.389+0000 7f5754200640 1 Processor -- start 2024-01-15T17:59:45.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.389+0000 7f5754200640 1 -- start start 2024-01-15T17:59:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.392+0000 7f5754200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c15e3b0 0x7f574c164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.392+0000 7f5754200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0b1ee0 con 0x7f574c143ae0 2024-01-15T17:59:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.392+0000 7f5754200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574c0b2780 con 0x7f574c0cf9a0 2024-01-15T17:59:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.392+0000 7f5754200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c15e990 con 0x7f574c15e3b0 2024-01-15T17:59:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.392+0000 7f5751800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f574c143ae0 0x7f574c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41500/0 (socket says 172.21.15.26:41500) 2024-01-15T17:59:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.392+0000 7f5751800640 1 -- 172.21.15.26:0/1838455762 learned_addr learned my addr 172.21.15.26:0/1838455762 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:59:45.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.392+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99723340 0 0) 0x7f574c0b1ee0 con 0x7f574c143ae0 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.392+0000 7f5752200640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c15e3b0 0x7f574c164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.392+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f572c005730 con 0x7f574c143ae0 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4272631604 0 0) 0x7f572c005730 con 0x7f574c143ae0 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c15e3b0 msgr2=0x7f574c164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c15e3b0 0x7f574c164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 >> v1:172.21.15.124:6789/0 conn(0x7f574c0cf9a0 legacy=0x7f574c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c0b2130 con 0x7f574c143ae0 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f573c001670 con 0x7f574c143ae0 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 >> v1:172.21.15.26:6789/0 conn(0x7f574c143ae0 legacy=0x7f574c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c005bb0 0x7f572c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c15e3b0 0x7f572c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:45.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c006f10 0x7f572c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c15e990 con 0x7f572c006f10 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c00dc70 con 0x7f572c005bb0 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c00de30 con 0x7f574c15e3b0 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.393+0000 7f5752200640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c006f10 0x7f572c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.394+0000 7f5751800640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c005bb0 0x7f572c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.394+0000 7f5752200640 1 -- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c15e3b0 msgr2=0x7f572c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.394+0000 7f5750e00640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c15e3b0 0x7f572c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.394+0000 7f5752200640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c15e3b0 0x7f572c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.394+0000 7f5752200640 1 -- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c005bb0 msgr2=0x7f572c005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.394+0000 7f5752200640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c005bb0 0x7f572c005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.394+0000 7f5752200640 1 -- 172.21.15.26:0/1838455762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c005730 con 0x7f572c006f10 2024-01-15T17:59:45.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.394+0000 7f5750e00640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c15e3b0 0x7f572c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T17:59:45.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.395+0000 7f5752200640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c006f10 0x7f572c007390 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5744009f90 tx=0x7f572c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a4375b9fcf8f9d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.395+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574402cbd0 con 0x7f572c006f10 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.395+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574400f040 con 0x7f572c006f10 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.395+0000 7f574be00640 1 -- 172.21.15.26:0/1838455762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744034440 con 0x7f572c006f10 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.396+0000 7f5754200640 1 -- 172.21.15.26:0/1838455762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c006f10 msgr2=0x7f572c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.396+0000 7f5754200640 1 --2- 172.21.15.26:0/1838455762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c006f10 0x7f572c007390 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5744009f90 tx=0x7f572c007a00 comp rx=0 tx=0).stop 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.396+0000 7f5752200640 1 -- 172.21.15.26:0/1838455762 reap_dead start 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.396+0000 7f5754200640 1 -- 172.21.15.26:0/1838455762 shutdown_connections 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.396+0000 7f5754200640 1 -- 172.21.15.26:0/1838455762 >> 172.21.15.26:0/1838455762 conn(0x7f574c06a800 msgr2=0x7f574c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.396+0000 7f5754200640 1 -- 172.21.15.26:0/1838455762 shutdown_connections 2024-01-15T17:59:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.396+0000 7f5754200640 1 -- 172.21.15.26:0/1838455762 wait complete. 2024-01-15T17:59:45.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.399+0000 7f5754200640 1 Processor -- start 2024-01-15T17:59:45.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.399+0000 7f5754200640 1 -- start start 2024-01-15T17:59:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.400+0000 7f5754200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c006f10 0x7f574c1f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.400+0000 7f5754200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c156e80 0x7f574c0ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.400+0000 7f5754200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0b0cd0 0x7f574c0b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.400+0000 7f5754200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c139f30 con 0x7f574c0b0cd0 2024-01-15T17:59:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.400+0000 7f5754200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c0b1690 con 0x7f572c006f10 2024-01-15T17:59:45.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.400+0000 7f5754200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c1f73a0 con 0x7f574c156e80 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5752200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0b0cd0 0x7f574c0b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5752200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0b0cd0 0x7f574c0b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44466/0 (socket says 172.21.15.26:44466) 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5752200640 1 -- 172.21.15.26:0/1494334684 learned_addr learned my addr 172.21.15.26:0/1494334684 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5750e00640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c156e80 0x7f574c0ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5751800640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c006f10 0x7f574c1f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5752200640 1 -- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c156e80 msgr2=0x7f574c0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5752200640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c156e80 0x7f574c0ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5752200640 1 -- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c006f10 msgr2=0x7f574c1f6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5752200640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c006f10 0x7f574c1f6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.401+0000 7f5752200640 1 -- 172.21.15.26:0/1494334684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c1fabc0 con 0x7f574c0b0cd0 2024-01-15T17:59:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.402+0000 7f5752200640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0b0cd0 0x7f574c0b1150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5744009f90 tx=0x7f5744009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.402+0000 7f5749600640 1 -- 172.21.15.26:0/1494334684 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744034ad0 con 0x7f574c0b0cd0 2024-01-15T17:59:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.402+0000 7f5749600640 1 -- 172.21.15.26:0/1494334684 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5744034c30 con 0x7f574c0b0cd0 2024-01-15T17:59:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.402+0000 7f5749600640 1 -- 172.21.15.26:0/1494334684 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5744003990 con 0x7f574c0b0cd0 2024-01-15T17:59:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.402+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c1faea0 con 0x7f574c0b0cd0 2024-01-15T17:59:45.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.402+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c1fb460 con 0x7f574c0b0cd0 2024-01-15T17:59:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.403+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c06c3f0 con 0x7f574c0b0cd0 2024-01-15T17:59:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.404+0000 7f5749600640 1 -- 172.21.15.26:0/1494334684 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 11) v1 ==== 89896+0+0 (secure 0 0 0) 0x7f574403e440 con 0x7f574c0b0cd0 2024-01-15T17:59:45.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.405+0000 7f5749600640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f572406b6e0 0x7f572406dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.405+0000 7f5749600640 1 -- 172.21.15.26:0/1494334684 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f57440ac0c0 con 0x7f574c0b0cd0 2024-01-15T17:59:45.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.406+0000 7f5751800640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f572406b6e0 0x7f572406dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:45.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.407+0000 7f5751800640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f572406b6e0 0x7f572406dba0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f573c000c00 tx=0x7f573c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:45.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.410+0000 7f5749600640 1 -- 172.21.15.26:0/1494334684 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5744076a60 con 0x7f574c0b0cd0 2024-01-15T17:59:45.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.674+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f574c0675d0 con 0x7f574c0b0cd0 2024-01-15T17:59:45.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.675+0000 7f5749600640 1 -- 172.21.15.26:0/1494334684 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2093 (secure 0 0 0) 0x7f574407a080 con 0x7f574c0b0cd0 2024-01-15T17:59:45.678 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:59:45.678 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:59:45.678 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T17:59:45.679 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 299, 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T17:59:45.680 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 49, 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T17:59:45.681 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 800, 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3222211047, 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 6725820416, 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 766368292864, 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T17:59:45.682 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 511, 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 18169, 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 0, 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 5 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3, 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:45.683 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 3, 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 5802 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T17:59:45.684 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 12, 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:59:26.004851+0000", 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T17:59:45.685 INFO:teuthology.orchestra.run.smithi026.stdout: "075bcb90-4ab4-4203-8139-75a65f763f34": { 2024-01-15T17:59:45.686 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "2 ongoing cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T17:59:45.686 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0, 2024-01-15T17:59:45.686 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T17:59:45.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:45.686 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:45.686 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:59:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.677+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f572406b6e0 msgr2=0x7f572406dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.677+0000 7f5754200640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f572406b6e0 0x7f572406dba0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f573c000c00 tx=0x7f573c009040 comp rx=0 tx=0).stop 2024-01-15T17:59:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.677+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0b0cd0 msgr2=0x7f574c0b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.677+0000 7f5754200640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0b0cd0 0x7f574c0b1150 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5744009f90 tx=0x7f5744009750 comp rx=0 tx=0).stop 2024-01-15T17:59:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.678+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 shutdown_connections 2024-01-15T17:59:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.678+0000 7f5754200640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f574c156e80 0x7f574c0ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.678+0000 7f5754200640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f572406b6e0 0x7f572406dba0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.678+0000 7f5754200640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f574c0b0cd0 0x7f574c0b1150 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.678+0000 7f5754200640 1 --2- 172.21.15.26:0/1494334684 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c006f10 0x7f574c1f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:45.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.678+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 >> 172.21.15.26:0/1494334684 conn(0x7f574c06a800 msgr2=0x7f574c140be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:45.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.678+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 shutdown_connections 2024-01-15T17:59:45.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:45.678+0000 7f5754200640 1 -- 172.21.15.26:0/1494334684 wait complete. 2024-01-15T17:59:47.700 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T17:59:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.007+0000 7f8ec5200640 1 Processor -- start 2024-01-15T17:59:48.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.007+0000 7f8ec5200640 1 -- start start 2024-01-15T17:59:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.009+0000 7f8ec5200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8ec0150760 0x7f8ec0072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.009+0000 7f8ec5200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ec015d300 0x7f8ec015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.009+0000 7f8ec5200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 0x7f8ec00b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.009+0000 7f8ec5200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec015dc80 con 0x7f8ec015d300 2024-01-15T17:59:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.009+0000 7f8ec5200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec0071190 con 0x7f8ec0150760 2024-01-15T17:59:48.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.009+0000 7f8ec5200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec0071470 con 0x7f8ec00b2bd0 2024-01-15T17:59:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.010+0000 7f8eb7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ec015d300 0x7f8ec015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.010+0000 7f8eb7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ec015d300 0x7f8ec015d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44482/0 (socket says 172.21.15.26:44482) 2024-01-15T17:59:48.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.010+0000 7f8eb7e00640 1 -- 172.21.15.26:0/2962163064 learned_addr learned my addr 172.21.15.26:0/2962163064 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.010+0000 7f8ebde00640 1 --2- 172.21.15.26:0/2962163064 >> v2:172.21.15.124:3300/0 conn(0x7f8ec0150760 0x7f8ec0072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.010+0000 7f8ebe800640 1 --2- 172.21.15.26:0/2962163064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 0x7f8ec00b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.011+0000 7f8eb7e00640 1 -- 172.21.15.26:0/2962163064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 msgr2=0x7f8ec00b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.011+0000 7f8eb7e00640 1 --2- 172.21.15.26:0/2962163064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 0x7f8ec00b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.011+0000 7f8eb7e00640 1 -- 172.21.15.26:0/2962163064 >> v2:172.21.15.124:3300/0 conn(0x7f8ec0150760 msgr2=0x7f8ec0072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.011+0000 7f8eb7e00640 1 --2- 172.21.15.26:0/2962163064 >> v2:172.21.15.124:3300/0 conn(0x7f8ec0150760 0x7f8ec0072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.011+0000 7f8eb7e00640 1 -- 172.21.15.26:0/2962163064 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec0071750 con 0x7f8ec015d300 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.013+0000 7f8eb7e00640 1 --2- 172.21.15.26:0/2962163064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 0x7f8ec015d740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8ec006e420 tx=0x7f8ea802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bdbbb467e7f882 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.013+0000 7f8ebd400640 1 -- 172.21.15.26:0/2962163064 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8031870 con 0x7f8ec015d300 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.013+0000 7f8ebd400640 1 -- 172.21.15.26:0/2962163064 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea80319d0 con 0x7f8ec015d300 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.013+0000 7f8ebd400640 1 -- 172.21.15.26:0/2962163064 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8031d60 con 0x7f8ec015d300 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.014+0000 7f8ec5200640 1 -- 172.21.15.26:0/2962163064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 msgr2=0x7f8ec015d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.014+0000 7f8ec5200640 1 --2- 172.21.15.26:0/2962163064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 0x7f8ec015d740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8ec006e420 tx=0x7f8ea802b910 comp rx=0 tx=0).stop 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.014+0000 7f8ec5200640 1 -- 172.21.15.26:0/2962163064 shutdown_connections 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.014+0000 7f8ec5200640 1 --2- 172.21.15.26:0/2962163064 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 0x7f8ec00b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.014+0000 7f8ec5200640 1 --2- 172.21.15.26:0/2962163064 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 0x7f8ec015d740 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.014+0000 7f8ec5200640 1 --2- 172.21.15.26:0/2962163064 >> v2:172.21.15.124:3300/0 conn(0x7f8ec0150760 0x7f8ec0072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.014+0000 7f8ec5200640 1 -- 172.21.15.26:0/2962163064 >> 172.21.15.26:0/2962163064 conn(0x7f8ec006a800 msgr2=0x7f8ec006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.015+0000 7f8ec5200640 1 -- 172.21.15.26:0/2962163064 shutdown_connections 2024-01-15T17:59:48.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.015+0000 7f8ec5200640 1 -- 172.21.15.26:0/2962163064 wait complete. 2024-01-15T17:59:48.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.019+0000 7f8ec5200640 1 Processor -- start 2024-01-15T17:59:48.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.019+0000 7f8ec5200640 1 -- start start 2024-01-15T17:59:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ec5200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 0x7f8ec0146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ec5200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ec0150760 0x7f8ec00ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ec5200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 0x7f8ec01578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ec5200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec00ced00 con 0x7f8ec015d300 2024-01-15T17:59:48.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ec5200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec0152e40 con 0x7f8ec0150760 2024-01-15T17:59:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ec5200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec014d5e0 con 0x7f8ec00b2bd0 2024-01-15T17:59:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ebe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 0x7f8ec01578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ebde00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 0x7f8ec0146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ebde00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 0x7f8ec0146b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:60536/0 (socket says 172.21.15.26:60536) 2024-01-15T17:59:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ebde00640 1 -- 172.21.15.26:0/3939161251 learned_addr learned my addr 172.21.15.26:0/3939161251 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.021+0000 7f8ebe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 0x7f8ec01578a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44490/0 (socket says 172.21.15.26:44490) 2024-01-15T17:59:48.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.022+0000 7f8ebe800640 1 -- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 msgr2=0x7f8ec0146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.022+0000 7f8ebe800640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 0x7f8ec0146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.022+0000 7f8ebe800640 1 -- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ec0150760 msgr2=0x7f8ec00ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.022+0000 7f8ebe800640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ec0150760 0x7f8ec00ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.022+0000 7f8ebe800640 1 -- 172.21.15.26:0/3939161251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea80094d0 con 0x7f8ec015d300 2024-01-15T17:59:48.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.022+0000 7f8ebe800640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 0x7f8ec01578a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8007c60 tx=0x7f8eb800d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.023+0000 7f8eb6000640 1 -- 172.21.15.26:0/3939161251 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800f040 con 0x7f8ec015d300 2024-01-15T17:59:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.023+0000 7f8eb6000640 1 -- 172.21.15.26:0/3939161251 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb800ddc0 con 0x7f8ec015d300 2024-01-15T17:59:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.023+0000 7f8eb6000640 1 -- 172.21.15.26:0/3939161251 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8003bd0 con 0x7f8ec015d300 2024-01-15T17:59:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.023+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec014d300 con 0x7f8ec015d300 2024-01-15T17:59:48.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.024+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec0152ad0 con 0x7f8ec015d300 2024-01-15T17:59:48.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.025+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec00675d0 con 0x7f8ec015d300 2024-01-15T17:59:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.027+0000 7f8eb6000640 1 -- 172.21.15.26:0/3939161251 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 11) v1 ==== 89896+0+0 (secure 0 0 0) 0x7f8eb8003d30 con 0x7f8ec015d300 2024-01-15T17:59:48.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.028+0000 7f8eb6000640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9806b780 0x7f8e9806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:48.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.028+0000 7f8eb6000640 1 -- 172.21.15.26:0/3939161251 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8eb808a460 con 0x7f8ec015d300 2024-01-15T17:59:48.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.032+0000 7f8ebde00640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9806b780 0x7f8e9806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:48.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.038+0000 7f8eb6000640 1 -- 172.21.15.26:0/3939161251 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8eb8054c50 con 0x7f8ec015d300 2024-01-15T17:59:48.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.044+0000 7f8ebde00640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9806b780 0x7f8e9806dc40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006fd0 tx=0x7f8eb0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:48.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.560+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8ec0152f80 con 0x7f8ec015d300 2024-01-15T17:59:48.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.561+0000 7f8eb6000640 1 -- 172.21.15.26:0/3939161251 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2092 (secure 0 0 0) 0x7f8eb8058270 con 0x7f8ec015d300 2024-01-15T17:59:48.564 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:59:48.565 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:59:48.565 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:59:48.565 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T17:59:48.565 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T17:59:48.565 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T17:59:48.565 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T17:59:48.565 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:48.565 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:48.566 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 302, 2024-01-15T17:59:48.567 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T17:59:48.567 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T17:59:48.567 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T17:59:48.567 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T17:59:48.567 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:48.567 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T17:59:48.567 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 49, 2024-01-15T17:59:48.567 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T17:59:48.568 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T17:59:48.568 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T17:59:48.568 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T17:59:48.568 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T17:59:48.568 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T17:59:48.568 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:48.568 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T17:59:48.568 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T17:59:48.568 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:48.569 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T17:59:48.569 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T17:59:48.569 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:48.569 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:48.569 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T17:59:48.569 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T17:59:48.569 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 800, 2024-01-15T17:59:48.570 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3222211047, 2024-01-15T17:59:48.570 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 6725820416, 2024-01-15T17:59:48.570 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 766368292864, 2024-01-15T17:59:48.570 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T17:59:48.570 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 426, 2024-01-15T17:59:48.570 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 9809, 2024-01-15T17:59:48.570 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 0, 2024-01-15T17:59:48.571 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 3 2024-01-15T17:59:48.571 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:48.571 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T17:59:48.571 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-01-15T17:59:48.571 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3, 2024-01-15T17:59:48.571 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T17:59:48.572 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T17:59:48.572 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T17:59:48.572 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T17:59:48.572 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:48.572 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 3, 2024-01-15T17:59:48.572 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T17:59:48.572 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T17:59:48.573 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T17:59:48.573 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 5802 2024-01-15T17:59:48.573 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:48.573 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:48.573 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T17:59:48.573 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:48.574 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T17:59:48.574 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T17:59:48.574 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T17:59:48.574 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T17:59:48.574 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T17:59:48.575 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T17:59:48.575 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T17:59:48.575 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:48.575 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:48.575 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:48.576 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T17:59:48.576 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 12, 2024-01-15T17:59:48.576 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:59:26.004851+0000", 2024-01-15T17:59:48.576 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:48.576 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:48.577 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T17:59:48.577 INFO:teuthology.orchestra.run.smithi026.stdout: "075bcb90-4ab4-4203-8139-75a65f763f34": { 2024-01-15T17:59:48.577 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "2 ongoing cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T17:59:48.577 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0, 2024-01-15T17:59:48.577 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T17:59:48.577 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:48.578 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:48.578 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.564+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9806b780 msgr2=0x7f8e9806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.564+0000 7f8ec5200640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9806b780 0x7f8e9806dc40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0006fd0 tx=0x7f8eb0008040 comp rx=0 tx=0).stop 2024-01-15T17:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.564+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 msgr2=0x7f8ec01578a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:48.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.564+0000 7f8ec5200640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 0x7f8ec01578a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8007c60 tx=0x7f8eb800d490 comp rx=0 tx=0).stop 2024-01-15T17:59:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.565+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 shutdown_connections 2024-01-15T17:59:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.565+0000 7f8ec5200640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9806b780 0x7f8e9806dc40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.565+0000 7f8ec5200640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec015d300 0x7f8ec01578a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.565+0000 7f8ec5200640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ec0150760 0x7f8ec00ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.565+0000 7f8ec5200640 1 --2- 172.21.15.26:0/3939161251 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec00b2bd0 0x7f8ec0146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:48.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.565+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 >> 172.21.15.26:0/3939161251 conn(0x7f8ec006a800 msgr2=0x7f8ec00cf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.565+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 shutdown_connections 2024-01-15T17:59:48.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:48.565+0000 7f8ec5200640 1 -- 172.21.15.26:0/3939161251 wait complete. 2024-01-15T17:59:48.634 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:48.631+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:48.634 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:48.631+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:48.634 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:48.631+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:48.634 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:48.631+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:48.634 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:48.631+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:50.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T17:59:50.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.830+0000 7fb632000640 1 Processor -- start 2024-01-15T17:59:50.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.830+0000 7fb632000640 1 -- start start 2024-01-15T17:59:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.832+0000 7fb632000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c071f00 0x7fb62c146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.832+0000 7fb632000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb62c06c7c0 0x7fb62c166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.832+0000 7fb632000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62c0b2e10 con 0x7fb62c071410 2024-01-15T17:59:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.832+0000 7fb632000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62c06cec0 con 0x7fb62c06c7c0 2024-01-15T17:59:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.832+0000 7fb632000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c06d1a0 con 0x7fb62c071f00 2024-01-15T17:59:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.832+0000 7fb631600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb62c06c7c0 0x7fb62c166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.833+0000 7fb631600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb62c06c7c0 0x7fb62c166cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44502/0 (socket says 172.21.15.26:44502) 2024-01-15T17:59:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.833+0000 7fb631600640 1 -- 172.21.15.26:0/1391350789 learned_addr learned my addr 172.21.15.26:0/1391350789 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:50.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.833+0000 7fb623c00640 1 --2- 172.21.15.26:0/1391350789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c071f00 0x7fb62c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:50.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.833+0000 7fb631600640 1 -- 172.21.15.26:0/1391350789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c071f00 msgr2=0x7fb62c146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:50.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.833+0000 7fb631600640 1 --2- 172.21.15.26:0/1391350789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c071f00 0x7fb62c146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:50.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.833+0000 7fb631600640 1 -- 172.21.15.26:0/1391350789 >> v1:172.21.15.124:6789/0 conn(0x7fb62c071410 legacy=0x7fb62c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:50.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.833+0000 7fb631600640 1 -- 172.21.15.26:0/1391350789 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c06d480 con 0x7fb62c06c7c0 2024-01-15T17:59:50.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.834+0000 7fb631600640 1 --2- 172.21.15.26:0/1391350789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c06c7c0 0x7fb62c166cf0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb624009e50 tx=0x7fb62402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df34697e5a809 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:50.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.834+0000 7fb62be00640 1 -- 172.21.15.26:0/1391350789 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62402ca70 con 0x7fb62c06c7c0 2024-01-15T17:59:50.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.835+0000 7fb62be00640 1 -- 172.21.15.26:0/1391350789 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62402cbd0 con 0x7fb62c06c7c0 2024-01-15T17:59:50.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.835+0000 7fb62be00640 1 -- 172.21.15.26:0/1391350789 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240343f0 con 0x7fb62c06c7c0 2024-01-15T17:59:50.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.835+0000 7fb632000640 1 -- 172.21.15.26:0/1391350789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c06c7c0 msgr2=0x7fb62c166cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:50.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.835+0000 7fb632000640 1 --2- 172.21.15.26:0/1391350789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c06c7c0 0x7fb62c166cf0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb624009e50 tx=0x7fb62402bed0 comp rx=0 tx=0).stop 2024-01-15T17:59:50.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.835+0000 7fb632000640 1 -- 172.21.15.26:0/1391350789 shutdown_connections 2024-01-15T17:59:50.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.835+0000 7fb632000640 1 --2- 172.21.15.26:0/1391350789 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c06c7c0 0x7fb62c166cf0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:50.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.835+0000 7fb632000640 1 --2- 172.21.15.26:0/1391350789 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c071f00 0x7fb62c146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:50.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.836+0000 7fb632000640 1 -- 172.21.15.26:0/1391350789 >> 172.21.15.26:0/1391350789 conn(0x7fb62c06a800 msgr2=0x7fb62c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:50.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.836+0000 7fb632000640 1 -- 172.21.15.26:0/1391350789 shutdown_connections 2024-01-15T17:59:50.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.836+0000 7fb632000640 1 -- 172.21.15.26:0/1391350789 wait complete. 2024-01-15T17:59:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.837+0000 7fb632000640 1 Processor -- start 2024-01-15T17:59:50.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.838+0000 7fb632000640 1 -- start start 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb632000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c06c7c0 0x7fb62c13bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb632000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb62c071410 0x7fb62c15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb632000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c071f00 0x7fb62c1f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb632000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c13c350 con 0x7fb62c071f00 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb632000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c13c630 con 0x7fb62c071410 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb632000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62c13c910 con 0x7fb62c06c7c0 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb631600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c071f00 0x7fb62c1f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb631600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c071f00 0x7fb62c1f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44516/0 (socket says 172.21.15.26:44516) 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb631600640 1 -- 172.21.15.26:0/3500793883 learned_addr learned my addr 172.21.15.26:0/3500793883 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb630c00640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c06c7c0 0x7fb62c13bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:50.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.839+0000 7fb623c00640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb62c071410 0x7fb62c15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb631600640 1 -- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c06c7c0 msgr2=0x7fb62c13bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb631600640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c06c7c0 0x7fb62c13bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb631600640 1 -- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb62c071410 msgr2=0x7fb62c15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb631600640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb62c071410 0x7fb62c15fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb631600640 1 -- 172.21.15.26:0/3500793883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624009b30 con 0x7fb62c071f00 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb631600640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c071f00 0x7fb62c1f5d60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb624036040 tx=0x7fb624004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb629600640 1 -- 172.21.15.26:0/3500793883 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624009510 con 0x7fb62c071f00 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb629600640 1 -- 172.21.15.26:0/3500793883 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62403e450 con 0x7fb62c071f00 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb629600640 1 -- 172.21.15.26:0/3500793883 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62403e7e0 con 0x7fb62c071f00 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.840+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62c13cbf0 con 0x7fb62c071f00 2024-01-15T17:59:50.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.841+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62c1f62a0 con 0x7fb62c071f00 2024-01-15T17:59:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.842+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb62c13e610 con 0x7fb62c071f00 2024-01-15T17:59:50.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.843+0000 7fb629600640 1 -- 172.21.15.26:0/3500793883 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 11) v1 ==== 89896+0+0 (secure 0 0 0) 0x7fb62403e940 con 0x7fb62c071f00 2024-01-15T17:59:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.843+0000 7fb629600640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb60406b6b0 0x7fb60406db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.843+0000 7fb629600640 1 -- 172.21.15.26:0/3500793883 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb6240ad190 con 0x7fb62c071f00 2024-01-15T17:59:50.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.844+0000 7fb630c00640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb60406b6b0 0x7fb60406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:50.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.845+0000 7fb630c00640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb60406b6b0 0x7fb60406db70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb61c004ea0 tx=0x7fb61c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:50.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.848+0000 7fb629600640 1 -- 172.21.15.26:0/3500793883 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb624077980 con 0x7fb62c071f00 2024-01-15T17:59:50.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:50.941+0000 7fb629600640 1 -- 172.21.15.26:0/3500793883 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 12) v1 ==== 89947+0+0 (secure 0 0 0) 0x7fb624077d50 con 0x7fb62c071f00 2024-01-15T17:59:51.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.227+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb62c067450 con 0x7fb62c071f00 2024-01-15T17:59:51.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.228+0000 7fb629600640 1 -- 172.21.15.26:0/3500793883 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2139 (secure 0 0 0) 0x7fb624043870 con 0x7fb62c071f00 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T17:59:51.232 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 305, 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T17:59:51.233 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 49, 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:51.234 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 973, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3947917444, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 7548125184, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 765545988096, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 68477936, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 60455145, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 17, 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 19 2024-01-15T17:59:51.235 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3, 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 3, 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T17:59:51.236 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 5802 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:51.237 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 12, 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:59:26.004851+0000", 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "075bcb90-4ab4-4203-8139-75a65f763f34": { 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "3 ongoing cloning jobs; Avg progress = 22.22% (5s)\n [======......................] (remaining: 17s)", 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.22220000624656677, 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:51.238 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:59:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb60406b6b0 msgr2=0x7fb60406db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb60406b6b0 0x7fb60406db70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb61c004ea0 tx=0x7fb61c009040 comp rx=0 tx=0).stop 2024-01-15T17:59:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c071f00 msgr2=0x7fb62c1f5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:51.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c071f00 0x7fb62c1f5d60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb624036040 tx=0x7fb624004a40 comp rx=0 tx=0).stop 2024-01-15T17:59:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 shutdown_connections 2024-01-15T17:59:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb60406b6b0 0x7fb60406db70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb62c071f00 0x7fb62c1f5d60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb62c071410 0x7fb62c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 --2- 172.21.15.26:0/3500793883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb62c06c7c0 0x7fb62c13bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 >> 172.21.15.26:0/3500793883 conn(0x7fb62c06a800 msgr2=0x7fb62c142130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 shutdown_connections 2024-01-15T17:59:51.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:51.232+0000 7fb632000640 1 -- 172.21.15.26:0/3500793883 wait complete. 2024-01-15T17:59:53.270 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T17:59:53.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.560+0000 7fec7a000640 1 Processor -- start 2024-01-15T17:59:53.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.561+0000 7fec7a000640 1 -- start start 2024-01-15T17:59:53.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.562+0000 7fec7a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec740b16d0 0x7fec74153950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:53.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.563+0000 7fec7a000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec7406d480 0x7fec74159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:53.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.563+0000 7fec7a000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec740b2b70 con 0x7fec740cb0d0 2024-01-15T17:59:53.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.563+0000 7fec7a000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec740b2e50 con 0x7fec740b16d0 2024-01-15T17:59:53.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.563+0000 7fec7a000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec740b3130 con 0x7fec7406d480 2024-01-15T17:59:53.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.564+0000 7fec73400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec740b16d0 0x7fec74153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:53.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.564+0000 7fec73400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec740b16d0 0x7fec74153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60262/0 (socket says 172.21.15.26:60262) 2024-01-15T17:59:53.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.564+0000 7fec73400640 1 -- 172.21.15.26:0/3826956325 learned_addr learned my addr 172.21.15.26:0/3826956325 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:53.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.564+0000 7fec73400640 1 -- 172.21.15.26:0/3826956325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec7406d480 msgr2=0x7fec74159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:53.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.564+0000 7fec73400640 1 --2- 172.21.15.26:0/3826956325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec7406d480 0x7fec74159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:53.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.564+0000 7fec73400640 1 -- 172.21.15.26:0/3826956325 >> v1:172.21.15.26:6789/0 conn(0x7fec740cb0d0 legacy=0x7fec741598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:53.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.564+0000 7fec73400640 1 -- 172.21.15.26:0/3826956325 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec740b3410 con 0x7fec740b16d0 2024-01-15T17:59:53.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.566+0000 7fec73400640 1 --2- 172.21.15.26:0/3826956325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 0x7fec74153950 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009870 tx=0x7fec5c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6499b50bbc4d51f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.566+0000 7fec72a00640 1 -- 172.21.15.26:0/3826956325 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c031640 con 0x7fec740b16d0 2024-01-15T17:59:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.567+0000 7fec72a00640 1 -- 172.21.15.26:0/3826956325 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c0317a0 con 0x7fec740b16d0 2024-01-15T17:59:53.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.567+0000 7fec72a00640 1 -- 172.21.15.26:0/3826956325 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c031b30 con 0x7fec740b16d0 2024-01-15T17:59:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.568+0000 7fec7a000640 1 -- 172.21.15.26:0/3826956325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 msgr2=0x7fec74153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:53.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.568+0000 7fec7a000640 1 --2- 172.21.15.26:0/3826956325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 0x7fec74153950 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fec5c009870 tx=0x7fec5c02b8f0 comp rx=0 tx=0).stop 2024-01-15T17:59:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.568+0000 7fec7a000640 1 -- 172.21.15.26:0/3826956325 shutdown_connections 2024-01-15T17:59:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.568+0000 7fec7a000640 1 --2- 172.21.15.26:0/3826956325 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec7406d480 0x7fec74159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.568+0000 7fec7a000640 1 --2- 172.21.15.26:0/3826956325 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 0x7fec74153950 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:53.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.568+0000 7fec7a000640 1 -- 172.21.15.26:0/3826956325 >> 172.21.15.26:0/3826956325 conn(0x7fec740b0be0 msgr2=0x7fec740b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.570+0000 7fec7a000640 1 -- 172.21.15.26:0/3826956325 shutdown_connections 2024-01-15T17:59:53.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.570+0000 7fec7a000640 1 -- 172.21.15.26:0/3826956325 wait complete. 2024-01-15T17:59:53.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.573+0000 7fec7a000640 1 Processor -- start 2024-01-15T17:59:53.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.574+0000 7fec7a000640 1 -- start start 2024-01-15T17:59:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.575+0000 7fec7a000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec7406d480 0x7fec740ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.575+0000 7fec7a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 0x7fec74153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.575+0000 7fec7a000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec740cb0d0 0x7fec74155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.575+0000 7fec7a000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74152ee0 con 0x7fec740b16d0 2024-01-15T17:59:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.575+0000 7fec7a000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74156850 con 0x7fec7406d480 2024-01-15T17:59:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.575+0000 7fec7a000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec74156b30 con 0x7fec740cb0d0 2024-01-15T17:59:53.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.576+0000 7fec6a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 0x7fec74153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:53.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.576+0000 7fec6a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 0x7fec74153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60268/0 (socket says 172.21.15.26:60268) 2024-01-15T17:59:53.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.576+0000 7fec6a800640 1 -- 172.21.15.26:0/1478989257 learned_addr learned my addr 172.21.15.26:0/1478989257 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:53.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.577+0000 7fec6a800640 1 -- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec740cb0d0 msgr2=0x7fec74155cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-15T17:59:53.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.577+0000 7fec6a800640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec740cb0d0 0x7fec74155cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:53.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.577+0000 7fec6a800640 1 -- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec7406d480 msgr2=0x7fec740ce760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-15T17:59:53.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.577+0000 7fec6a800640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec7406d480 0x7fec740ce760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:53.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.577+0000 7fec6a800640 1 -- 172.21.15.26:0/1478989257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c0094d0 con 0x7fec740b16d0 2024-01-15T17:59:53.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.578+0000 7fec6a800640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 0x7fec74153f20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fec6400ba50 tx=0x7fec6400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:53.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.579+0000 7fec6be00640 1 -- 172.21.15.26:0/1478989257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6400cc10 con 0x7fec740b16d0 2024-01-15T17:59:53.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.579+0000 7fec6be00640 1 -- 172.21.15.26:0/1478989257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6400cd70 con 0x7fec740b16d0 2024-01-15T17:59:53.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.579+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec74156e10 con 0x7fec740b16d0 2024-01-15T17:59:53.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.579+0000 7fec6be00640 1 -- 172.21.15.26:0/1478989257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64014680 con 0x7fec740b16d0 2024-01-15T17:59:53.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.579+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec74156230 con 0x7fec740b16d0 2024-01-15T17:59:53.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.581+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec740675d0 con 0x7fec740b16d0 2024-01-15T17:59:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.588+0000 7fec6be00640 1 -- 172.21.15.26:0/1478989257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 12) v1 ==== 89947+0+0 (secure 0 0 0) 0x7fec6401d3b0 con 0x7fec740b16d0 2024-01-15T17:59:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.589+0000 7fec6be00640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec4c06b790 0x7fec4c06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.590+0000 7fec6be00640 1 -- 172.21.15.26:0/1478989257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fec6408a1f0 con 0x7fec740b16d0 2024-01-15T17:59:53.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.595+0000 7fec73400640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec4c06b790 0x7fec4c06dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:53.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.599+0000 7fec6be00640 1 -- 172.21.15.26:0/1478989257 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec6408f060 con 0x7fec740b16d0 2024-01-15T17:59:53.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:53.607+0000 7fec73400640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec4c06b790 0x7fec4c06dc50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fec5c036000 tx=0x7fec5c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:53.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:53.686+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:53.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:53.687+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:53.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:53.687+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:53.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:53.687+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:53.694 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T17:59:53.687+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T17:59:54.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.047+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fec740b43c0 con 0x7fec740b16d0 2024-01-15T17:59:54.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.047+0000 7fec6be00640 1 -- 172.21.15.26:0/1478989257 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2139 (secure 0 0 0) 0x7fec64019870 con 0x7fec740b16d0 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T17:59:54.051 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 308, 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T17:59:54.052 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 49, 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:54.053 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 973, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3947917444, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 7548125184, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 765545988096, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 68480713, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 60457596, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 17, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 19 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T17:59:54.054 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3, 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 3, 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 5802 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T17:59:54.055 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 12, 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:59:26.004851+0000", 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:54.056 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T17:59:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "075bcb90-4ab4-4203-8139-75a65f763f34": { 2024-01-15T17:59:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "3 ongoing cloning jobs; Avg progress = 22.22% (5s)\n [======......................] (remaining: 17s)", 2024-01-15T17:59:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.22220000624656677, 2024-01-15T17:59:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T17:59:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:54.057 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:54.057 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:59:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec4c06b790 msgr2=0x7fec4c06dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec4c06b790 0x7fec4c06dc50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fec5c036000 tx=0x7fec5c008030 comp rx=0 tx=0).stop 2024-01-15T17:59:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 msgr2=0x7fec74153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 0x7fec74153f20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fec6400ba50 tx=0x7fec6400bf10 comp rx=0 tx=0).stop 2024-01-15T17:59:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 shutdown_connections 2024-01-15T17:59:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec740cb0d0 0x7fec74155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec4c06b790 0x7fec4c06dc50 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fec5c036000 tx=0x7fec5c008030 comp rx=0 tx=0).stop 2024-01-15T17:59:54.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec740b16d0 0x7fec74153f20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:54.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 --2- 172.21.15.26:0/1478989257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec7406d480 0x7fec740ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:54.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.050+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 >> 172.21.15.26:0/1478989257 conn(0x7fec740b0be0 msgr2=0x7fec74147480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:54.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.051+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 shutdown_connections 2024-01-15T17:59:54.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:54.051+0000 7fec7a000640 1 -- 172.21.15.26:0/1478989257 wait complete. 2024-01-15T17:59:56.073 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T17:59:56.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.290+0000 7fd58ac00640 1 Processor -- start 2024-01-15T17:59:56.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.290+0000 7fd58ac00640 1 -- start start 2024-01-15T17:59:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.291+0000 7fd58ac00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd584143c40 0x7fd584071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.291+0000 7fd58ac00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840717d0 0x7fd5840cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.291+0000 7fd58ac00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd584070e50 con 0x7fd5840cca40 2024-01-15T17:59:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.291+0000 7fd58ac00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5840723c0 con 0x7fd584143c40 2024-01-15T17:59:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.291+0000 7fd58ac00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840cce70 con 0x7fd5840717d0 2024-01-15T17:59:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.292+0000 7fd58a200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd5840cca40 0x7fd58415eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53254/0 (socket says 172.21.15.26:53254) 2024-01-15T17:59:56.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.292+0000 7fd58a200640 1 -- 172.21.15.26:0/1841795480 learned_addr learned my addr 172.21.15.26:0/1841795480 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:59:56.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.292+0000 7fd588e00640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840717d0 0x7fd5840cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:56.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.292+0000 7fd589800640 1 --2- 172.21.15.26:0/1841795480 >> v2:172.21.15.124:3300/0 conn(0x7fd584143c40 0x7fd584071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:56.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.292+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390905634 0 0) 0x7fd584070e50 con 0x7fd5840cca40 2024-01-15T17:59:56.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.292+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd564003520 con 0x7fd5840cca40 2024-01-15T17:59:56.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669235467 0 0) 0x7fd564003520 con 0x7fd5840cca40 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840717d0 msgr2=0x7fd5840cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840717d0 0x7fd5840cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 >> v2:172.21.15.124:3300/0 conn(0x7fd584143c40 msgr2=0x7fd584071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 --2- 172.21.15.26:0/1841795480 >> v2:172.21.15.124:3300/0 conn(0x7fd584143c40 0x7fd584071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58406e170 con 0x7fd5840cca40 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd57c0019f0 con 0x7fd5840cca40 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 >> v1:172.21.15.26:6789/0 conn(0x7fd5840cca40 legacy=0x7fd58415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd564003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840717d0 0x7fd5640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd564004d00 0x7fd564005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840723c0 con 0x7fd564003970 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840cce70 con 0x7fd564004d00 2024-01-15T17:59:56.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56400ba00 con 0x7fd5840717d0 2024-01-15T17:59:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.293+0000 7fd589800640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd564003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.294+0000 7fd589800640 1 -- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840717d0 msgr2=0x7fd5640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.294+0000 7fd588e00640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840717d0 0x7fd5640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.294+0000 7fd589800640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840717d0 0x7fd5640047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.294+0000 7fd589800640 1 -- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd564004d00 msgr2=0x7fd564005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.294+0000 7fd589800640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd564004d00 0x7fd564005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.294+0000 7fd589800640 1 -- 172.21.15.26:0/1841795480 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564003520 con 0x7fd564003970 2024-01-15T17:59:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.295+0000 7fd589800640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd564003d90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd574009ac0 tx=0x7fd5640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79733881913a5ec1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.295+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57402c710 con 0x7fd564003970 2024-01-15T17:59:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.295+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57402c870 con 0x7fd564003970 2024-01-15T17:59:56.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.295+0000 7fd583e00640 1 -- 172.21.15.26:0/1841795480 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57402cc00 con 0x7fd564003970 2024-01-15T17:59:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.296+0000 7fd58ac00640 1 -- 172.21.15.26:0/1841795480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 msgr2=0x7fd564003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.296+0000 7fd58ac00640 1 --2- 172.21.15.26:0/1841795480 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd564003d90 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fd574009ac0 tx=0x7fd5640057f0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.296+0000 7fd58a200640 1 -- 172.21.15.26:0/1841795480 reap_dead start 2024-01-15T17:59:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.296+0000 7fd58ac00640 1 -- 172.21.15.26:0/1841795480 shutdown_connections 2024-01-15T17:59:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.296+0000 7fd58ac00640 1 -- 172.21.15.26:0/1841795480 >> 172.21.15.26:0/1841795480 conn(0x7fd58406a800 msgr2=0x7fd58406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.296+0000 7fd58ac00640 1 -- 172.21.15.26:0/1841795480 shutdown_connections 2024-01-15T17:59:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.296+0000 7fd58ac00640 1 -- 172.21.15.26:0/1841795480 wait complete. 2024-01-15T17:59:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.298+0000 7fd58ac00640 1 Processor -- start 2024-01-15T17:59:56.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.298+0000 7fd58ac00640 1 -- start start 2024-01-15T17:59:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.299+0000 7fd58ac00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd584150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:56.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.299+0000 7fd58ac00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5840a8960 0x7fd5841f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.299+0000 7fd58ac00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840a8ed0 0x7fd5840a9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.299+0000 7fd58ac00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5840a9890 con 0x7fd564003970 2024-01-15T17:59:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.299+0000 7fd58ac00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd584151680 con 0x7fd5840a8960 2024-01-15T17:59:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.299+0000 7fd58ac00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd584151960 con 0x7fd5840a8ed0 2024-01-15T17:59:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.299+0000 7fd589800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd584150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.299+0000 7fd589800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd584150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60288/0 (socket says 172.21.15.26:60288) 2024-01-15T17:59:56.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.299+0000 7fd589800640 1 -- 172.21.15.26:0/3326718349 learned_addr learned my addr 172.21.15.26:0/3326718349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd589800640 1 -- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840a8ed0 msgr2=0x7fd5840a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd589800640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840a8ed0 0x7fd5840a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd589800640 1 -- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5840a8960 msgr2=0x7fd5841f1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd589800640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5840a8960 0x7fd5841f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd589800640 1 -- 172.21.15.26:0/3326718349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd584151c40 con 0x7fd564003970 2024-01-15T17:59:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd589800640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd584150da0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd57402c2b0 tx=0x7fd574004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:56.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd581600640 1 -- 172.21.15.26:0/3326718349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574003810 con 0x7fd564003970 2024-01-15T17:59:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd581600640 1 -- 172.21.15.26:0/3326718349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57402de90 con 0x7fd564003970 2024-01-15T17:59:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd584151f20 con 0x7fd564003970 2024-01-15T17:59:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.300+0000 7fd581600640 1 -- 172.21.15.26:0/3326718349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574003b70 con 0x7fd564003970 2024-01-15T17:59:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.301+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5841512e0 con 0x7fd564003970 2024-01-15T17:59:56.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.302+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd584067450 con 0x7fd564003970 2024-01-15T17:59:56.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.303+0000 7fd581600640 1 -- 172.21.15.26:0/3326718349 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7fd574003cd0 con 0x7fd564003970 2024-01-15T17:59:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.304+0000 7fd581600640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd55c06b800 0x7fd55c06dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.304+0000 7fd581600640 1 -- 172.21.15.26:0/3326718349 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd5740ac370 con 0x7fd564003970 2024-01-15T17:59:56.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.305+0000 7fd588e00640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd55c06b800 0x7fd55c06dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:56.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.306+0000 7fd588e00640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd55c06b800 0x7fd55c06dcc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd58413f230 tx=0x7fd56c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:56.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.308+0000 7fd581600640 1 -- 172.21.15.26:0/3326718349 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd574076a60 con 0x7fd564003970 2024-01-15T17:59:56.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.627+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd5840ceb30 con 0x7fd564003970 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.628+0000 7fd581600640 1 -- 172.21.15.26:0/3326718349 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2145 (secure 0 0 0) 0x7fd57407a080 con 0x7fd564003970 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T17:59:56.631 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 310, 2024-01-15T17:59:56.632 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 49, 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:56.633 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1498, 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 6150035349, 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 11851661312, 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 761242451968, 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 254705137, 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 243891918, 2024-01-15T17:59:56.634 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 61, 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 64 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3, 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:56.635 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 3, 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 5802 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T17:59:56.636 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 12, 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:59:26.004851+0000", 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:56.637 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T17:59:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "075bcb90-4ab4-4203-8139-75a65f763f34": { 2024-01-15T17:59:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 41.668% (11s)\n [===========.................] (remaining: 15s)", 2024-01-15T17:59:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.41667500138282776, 2024-01-15T17:59:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T17:59:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:56.638 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:56.638 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:59:56.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd55c06b800 msgr2=0x7fd55c06dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:56.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd55c06b800 0x7fd55c06dcc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd58413f230 tx=0x7fd56c008040 comp rx=0 tx=0).stop 2024-01-15T17:59:56.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 msgr2=0x7fd584150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:56.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd584150da0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd57402c2b0 tx=0x7fd574004730 comp rx=0 tx=0).stop 2024-01-15T17:59:56.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 shutdown_connections 2024-01-15T17:59:56.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd5840a8ed0 0x7fd5840a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd55c06b800 0x7fd55c06dcc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5840a8960 0x7fd5841f1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 --2- 172.21.15.26:0/3326718349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd564003970 0x7fd584150da0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:56.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 >> 172.21.15.26:0/3326718349 conn(0x7fd58406a800 msgr2=0x7fd5840aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:56.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 shutdown_connections 2024-01-15T17:59:56.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:56.630+0000 7fd58ac00640 1 -- 172.21.15.26:0/3326718349 wait complete. 2024-01-15T17:59:57.010 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:59:57.016 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T17:59:58.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T17:59:58.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.958+0000 7fc75aa00640 1 Processor -- start 2024-01-15T17:59:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.958+0000 7fc75aa00640 1 -- start start 2024-01-15T17:59:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.960+0000 7fc75aa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc754099a60 0x7fc754099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.960+0000 7fc75aa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7540cfa70 con 0x7fc75406e7c0 2024-01-15T17:59:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.960+0000 7fc75aa00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75409a3e0 con 0x7fc75415e880 2024-01-15T17:59:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.960+0000 7fc75aa00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7540cff00 con 0x7fc754099a60 2024-01-15T17:59:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.961+0000 7fc758a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc75406e7c0 0x7fc75406ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53266/0 (socket says 172.21.15.26:53266) 2024-01-15T17:59:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.961+0000 7fc758a00640 1 -- 172.21.15.26:0/144259001 learned_addr learned my addr 172.21.15.26:0/144259001 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T17:59:58.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.961+0000 7fc753400640 1 --2- 172.21.15.26:0/144259001 >> v2:172.21.15.124:3300/0 conn(0x7fc754099a60 0x7fc754099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.962+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980356382 0 0) 0x7fc7540cfa70 con 0x7fc75406e7c0 2024-01-15T17:59:58.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.962+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc734003520 con 0x7fc75406e7c0 2024-01-15T17:59:58.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.965+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1873867223 0 0) 0x7fc734003520 con 0x7fc75406e7c0 2024-01-15T17:59:58.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 >> v2:172.21.15.124:3300/0 conn(0x7fc754099a60 msgr2=0x7fc754099ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 --2- 172.21.15.26:0/144259001 >> v2:172.21.15.124:3300/0 conn(0x7fc754099a60 0x7fc754099ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 >> v1:172.21.15.124:6789/0 conn(0x7fc75415e880 legacy=0x7fc75406ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75406fc70 con 0x7fc75406e7c0 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fc74c0019f0 con 0x7fc75406e7c0 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 >> v1:172.21.15.26:6789/0 conn(0x7fc75406e7c0 legacy=0x7fc75406ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340039c0 0x7fc734003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7340047c0 0x7fc734004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc734005180 0x7fc73400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540cff00 con 0x7fc7340039c0 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73400c060 con 0x7fc734005180 2024-01-15T17:59:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73400c220 con 0x7fc7340047c0 2024-01-15T17:59:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.966+0000 7fc758a00640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc734005180 0x7fc73400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.967+0000 7fc753400640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7340047c0 0x7fc734004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.967+0000 7fc753e00640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340039c0 0x7fc734003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.968+0000 7fc753400640 1 -- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc734005180 msgr2=0x7fc73400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:58.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.968+0000 7fc753400640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc734005180 0x7fc73400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.968+0000 7fc753400640 1 -- 172.21.15.26:0/144259001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340039c0 msgr2=0x7fc734003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T17:59:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.968+0000 7fc753400640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340039c0 0x7fc734003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.968+0000 7fc753400640 1 -- 172.21.15.26:0/144259001 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc734003520 con 0x7fc7340047c0 2024-01-15T17:59:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.968+0000 7fc758a00640 1 -- 172.21.15.26:0/144259001 reap_dead start 2024-01-15T17:59:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.968+0000 7fc753e00640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340039c0 0x7fc734003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T17:59:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.973+0000 7fc753400640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7340047c0 0x7fc734004c40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc73c010b00 tx=0x7fc7340057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46b8ea9111a6db2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.973+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c030960 con 0x7fc7340047c0 2024-01-15T17:59:58.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.974+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c030ac0 con 0x7fc7340047c0 2024-01-15T17:59:58.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.974+0000 7fc752a00640 1 -- 172.21.15.26:0/144259001 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c030e50 con 0x7fc7340047c0 2024-01-15T17:59:58.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.975+0000 7fc75aa00640 1 -- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7340047c0 msgr2=0x7fc734004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:58.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.975+0000 7fc75aa00640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7340047c0 0x7fc734004c40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc73c010b00 tx=0x7fc7340057f0 comp rx=0 tx=0).stop 2024-01-15T17:59:58.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.975+0000 7fc75aa00640 1 -- 172.21.15.26:0/144259001 shutdown_connections 2024-01-15T17:59:58.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.975+0000 7fc75aa00640 1 --2- 172.21.15.26:0/144259001 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc7340047c0 0x7fc734004c40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:58.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.975+0000 7fc75aa00640 1 -- 172.21.15.26:0/144259001 >> 172.21.15.26:0/144259001 conn(0x7fc7540b2d00 msgr2=0x7fc7540b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:58.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.975+0000 7fc75aa00640 1 -- 172.21.15.26:0/144259001 shutdown_connections 2024-01-15T17:59:58.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.975+0000 7fc75aa00640 1 -- 172.21.15.26:0/144259001 wait complete. 2024-01-15T17:59:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.977+0000 7fc75aa00640 1 Processor -- start 2024-01-15T17:59:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.977+0000 7fc75aa00640 1 -- start start 2024-01-15T17:59:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.978+0000 7fc75aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340047c0 0x7fc7540c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.978+0000 7fc75aa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc75414afd0 0x7fc7540cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.978+0000 7fc75aa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7540c7720 0x7fc75414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.978+0000 7fc75aa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541eea70 con 0x7fc7340047c0 2024-01-15T17:59:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.978+0000 7fc75aa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540cb760 con 0x7fc7540c7720 2024-01-15T17:59:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.978+0000 7fc75aa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7540cad90 con 0x7fc75414afd0 2024-01-15T17:59:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340047c0 0x7fc7540c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340047c0 0x7fc7540c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60306/0 (socket says 172.21.15.26:60306) 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753e00640 1 -- 172.21.15.26:0/278508707 learned_addr learned my addr 172.21.15.26:0/278508707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753400640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc75414afd0 0x7fc7540cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc758a00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7540c7720 0x7fc75414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753e00640 1 -- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc75414afd0 msgr2=0x7fc7540cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753e00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc75414afd0 0x7fc7540cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753e00640 1 -- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7540c7720 msgr2=0x7fc75414d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753e00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7540c7720 0x7fc75414d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753e00640 1 -- 172.21.15.26:0/278508707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75414b450 con 0x7fc7340047c0 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc758a00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7540c7720 0x7fc75414d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753400640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc75414afd0 0x7fc7540cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.979+0000 7fc753e00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340047c0 0x7fc7540c9ac0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc74400f750 tx=0x7fc74400fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.980+0000 7fc74be00640 1 -- 172.21.15.26:0/278508707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400fe60 con 0x7fc7340047c0 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.980+0000 7fc74be00640 1 -- 172.21.15.26:0/278508707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7440052e0 con 0x7fc7340047c0 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.980+0000 7fc74be00640 1 -- 172.21.15.26:0/278508707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400d8d0 con 0x7fc7340047c0 2024-01-15T17:59:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.980+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7540ca180 con 0x7fc7340047c0 2024-01-15T17:59:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.980+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7540ca740 con 0x7fc7340047c0 2024-01-15T17:59:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.981+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7540675d0 con 0x7fc7340047c0 2024-01-15T17:59:58.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.982+0000 7fc74be00640 1 -- 172.21.15.26:0/278508707 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7fc74400da30 con 0x7fc7340047c0 2024-01-15T17:59:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.984+0000 7fc74be00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc72c06b850 0x7fc72c06dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T17:59:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.984+0000 7fc74be00640 1 -- 172.21.15.26:0/278508707 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc744089e60 con 0x7fc7340047c0 2024-01-15T17:59:58.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.992+0000 7fc753400640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc72c06b850 0x7fc72c06dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T17:59:59.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:58.998+0000 7fc74be00640 1 -- 172.21.15.26:0/278508707 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc7440546a0 con 0x7fc7340047c0 2024-01-15T17:59:59.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.012+0000 7fc753400640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc72c06b850 0x7fc72c06dd10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc73c03a000 tx=0x7fc73c034000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T17:59:59.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.471+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc7540c4670 con 0x7fc7340047c0 2024-01-15T17:59:59.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.472+0000 7fc74be00640 1 -- 172.21.15.26:0/278508707 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2147 (secure 0 0 0) 0x7fc744001350 con 0x7fc7340047c0 2024-01-15T17:59:59.476 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:59.477 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 313, 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 49, 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T17:59:59.478 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T17:59:59.479 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 2146, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 8868043748, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 17682231296, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 755411881984, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 473781537, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 470314792, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 114, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 121 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T17:59:59.480 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 3, 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 5802 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T17:59:59.481 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 12, 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:59:26.004851+0000", 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "075bcb90-4ab4-4203-8139-75a65f763f34": { 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 41.668% (11s)\n [===========.................] (remaining: 15s)", 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.41667500138282776, 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T17:59:59.482 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:59.483 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T17:59:59.483 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T17:59:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.475+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc72c06b850 msgr2=0x7fc72c06dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.475+0000 7fc75aa00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc72c06b850 0x7fc72c06dd10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc73c03a000 tx=0x7fc73c034000 comp rx=0 tx=0).stop 2024-01-15T17:59:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.475+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340047c0 msgr2=0x7fc7540c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T17:59:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.475+0000 7fc75aa00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340047c0 0x7fc7540c9ac0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc74400f750 tx=0x7fc74400fc10 comp rx=0 tx=0).stop 2024-01-15T17:59:59.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.476+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 shutdown_connections 2024-01-15T17:59:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.476+0000 7fc75aa00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc75414afd0 0x7fc7540cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.476+0000 7fc75aa00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc72c06b850 0x7fc72c06dd10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.476+0000 7fc75aa00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc7540c7720 0x7fc75414d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.476+0000 7fc75aa00640 1 --2- 172.21.15.26:0/278508707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7340047c0 0x7fc7540c9ac0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T17:59:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.476+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 >> 172.21.15.26:0/278508707 conn(0x7fc7540b2d00 msgr2=0x7fc75415d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T17:59:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.476+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 shutdown_connections 2024-01-15T17:59:59.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T17:59:59.476+0000 7fc75aa00640 1 -- 172.21.15.26:0/278508707 wait complete. 2024-01-15T18:00:01.501 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:00:01.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.821+0000 7f3135c00640 1 Processor -- start 2024-01-15T18:00:01.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.821+0000 7f3135c00640 1 -- start start 2024-01-15T18:00:01.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.823+0000 7f3135c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f313013df80 0x7f313015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:01.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.823+0000 7f3135c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f313013e360 0x7f31300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:01.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.824+0000 7f3135c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313006c470 con 0x7f313015f9c0 2024-01-15T18:00:01.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.824+0000 7f3135c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313006c750 con 0x7f313013df80 2024-01-15T18:00:01.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.824+0000 7f3135c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313006ca30 con 0x7f313013e360 2024-01-15T18:00:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.824+0000 7f312e000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f313013e360 0x7f31300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.825+0000 7f312f400640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f313015f9c0 0x7f31301408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:51108/0 (socket says 172.21.15.26:51108) 2024-01-15T18:00:01.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.825+0000 7f312f400640 1 -- 172.21.15.26:0/1863841672 learned_addr learned my addr 172.21.15.26:0/1863841672 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.826+0000 7f312d600640 1 -- 172.21.15.26:0/1863841672 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2050800760 0 0) 0x7f313006c470 con 0x7f313015f9c0 2024-01-15T18:00:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.826+0000 7f312d600640 1 -- 172.21.15.26:0/1863841672 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3110003520 con 0x7f313015f9c0 2024-01-15T18:00:01.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.826+0000 7f312ea00640 1 --2- 172.21.15.26:0/1863841672 >> v2:172.21.15.26:3300/0 conn(0x7f313013df80 0x7f313015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.827+0000 7f312ea00640 1 -- 172.21.15.26:0/1863841672 >> v2:172.21.15.124:3300/0 conn(0x7f313013e360 msgr2=0x7f31300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.827+0000 7f312ea00640 1 --2- 172.21.15.26:0/1863841672 >> v2:172.21.15.124:3300/0 conn(0x7f313013e360 0x7f31300cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.827+0000 7f312ea00640 1 -- 172.21.15.26:0/1863841672 >> v1:172.21.15.124:6789/0 conn(0x7f313015f9c0 legacy=0x7f31301408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:01.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.827+0000 7f312ea00640 1 -- 172.21.15.26:0/1863841672 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313006cd10 con 0x7f313013df80 2024-01-15T18:00:01.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.830+0000 7f312ea00640 1 --2- 172.21.15.26:0/1863841672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f313013df80 0x7f313015bee0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f313006ded0 tx=0x7f312002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e996681cfcc4b11f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.830+0000 7f312d600640 1 -- 172.21.15.26:0/1863841672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120031830 con 0x7f313013df80 2024-01-15T18:00:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.831+0000 7f312d600640 1 -- 172.21.15.26:0/1863841672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3120031990 con 0x7f313013df80 2024-01-15T18:00:01.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.831+0000 7f312d600640 1 -- 172.21.15.26:0/1863841672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120031d20 con 0x7f313013df80 2024-01-15T18:00:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.832+0000 7f3135c00640 1 -- 172.21.15.26:0/1863841672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f313013df80 msgr2=0x7f313015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.832+0000 7f3135c00640 1 --2- 172.21.15.26:0/1863841672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f313013df80 0x7f313015bee0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f313006ded0 tx=0x7f312002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.832+0000 7f3135c00640 1 -- 172.21.15.26:0/1863841672 shutdown_connections 2024-01-15T18:00:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.832+0000 7f3135c00640 1 --2- 172.21.15.26:0/1863841672 >> v2:172.21.15.124:3300/0 conn(0x7f313013e360 0x7f31300cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.832+0000 7f3135c00640 1 --2- 172.21.15.26:0/1863841672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f313013df80 0x7f313015bee0 secure :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f313006ded0 tx=0x7f312002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:01.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.832+0000 7f3135c00640 1 -- 172.21.15.26:0/1863841672 >> 172.21.15.26:0/1863841672 conn(0x7f313006a800 msgr2=0x7f313006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.841+0000 7f3135c00640 1 -- 172.21.15.26:0/1863841672 shutdown_connections 2024-01-15T18:00:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.841+0000 7f3135c00640 1 -- 172.21.15.26:0/1863841672 wait complete. 2024-01-15T18:00:01.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.844+0000 7f3135c00640 1 Processor -- start 2024-01-15T18:00:01.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.845+0000 7f3135c00640 1 -- start start 2024-01-15T18:00:01.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.847+0000 7f3135c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313013e360 0x7f31301fccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:01.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.848+0000 7f3135c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f313015f9c0 0x7f3130070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.848+0000 7f3135c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3130070e90 0x7f31302030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.849+0000 7f3135c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3130203dd0 con 0x7f3130070e90 2024-01-15T18:00:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.849+0000 7f3135c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3130071320 con 0x7f313015f9c0 2024-01-15T18:00:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.849+0000 7f3135c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31301ff810 con 0x7f313013e360 2024-01-15T18:00:01.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.849+0000 7f312f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3130070e90 0x7f31302030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.849+0000 7f312f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3130070e90 0x7f31302030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60318/0 (socket says 172.21.15.26:60318) 2024-01-15T18:00:01.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.850+0000 7f312f400640 1 -- 172.21.15.26:0/3346762019 learned_addr learned my addr 172.21.15.26:0/3346762019 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.851+0000 7f312ea00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313013e360 0x7f31301fccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:01.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.851+0000 7f312e000640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f313015f9c0 0x7f3130070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.852+0000 7f312f400640 1 -- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313013e360 msgr2=0x7f31301fccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.852+0000 7f312f400640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313013e360 0x7f31301fccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.852+0000 7f312f400640 1 -- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f313015f9c0 msgr2=0x7f3130070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.852+0000 7f312f400640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f313015f9c0 0x7f3130070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:01.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.852+0000 7f312f400640 1 -- 172.21.15.26:0/3346762019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120009400 con 0x7f3130070e90 2024-01-15T18:00:01.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.853+0000 7f312f400640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3130070e90 0x7f31302030a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f312800cbb0 tx=0x7f312800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.856+0000 7f3126a00640 1 -- 172.21.15.26:0/3346762019 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312800ad40 con 0x7f3130070e90 2024-01-15T18:00:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.860+0000 7f3126a00640 1 -- 172.21.15.26:0/3346762019 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3128012400 con 0x7f3130070e90 2024-01-15T18:00:01.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.860+0000 7f3126a00640 1 -- 172.21.15.26:0/3346762019 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312800aea0 con 0x7f3130070e90 2024-01-15T18:00:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.862+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3130200350 con 0x7f3130070e90 2024-01-15T18:00:01.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.862+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31301ffd00 con 0x7f3130070e90 2024-01-15T18:00:01.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.866+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3100005250 con 0x7f3130070e90 2024-01-15T18:00:01.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.868+0000 7f3126a00640 1 -- 172.21.15.26:0/3346762019 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7f31280127b0 con 0x7f3130070e90 2024-01-15T18:00:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.869+0000 7f3126a00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f310806b850 0x7f310806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:01.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.869+0000 7f3126a00640 1 -- 172.21.15.26:0/3346762019 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f312808a170 con 0x7f3130070e90 2024-01-15T18:00:01.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.872+0000 7f312ea00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f310806b850 0x7f310806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:01.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.878+0000 7f3126a00640 1 -- 172.21.15.26:0/3346762019 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3128054ae0 con 0x7f3130070e90 2024-01-15T18:00:01.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:01.881+0000 7f312ea00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f310806b850 0x7f310806dd10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3120036000 tx=0x7f312003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:02.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.500+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f31000057d0 con 0x7f3130070e90 2024-01-15T18:00:02.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.501+0000 7f3126a00640 1 -- 172.21.15.26:0/3346762019 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2146 (secure 0 0 0) 0x7f3128001be0 con 0x7f3130070e90 2024-01-15T18:00:02.506 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:02.506 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:02.506 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:00:02.506 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:00:02.506 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:00:02.506 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 316, 2024-01-15T18:00:02.507 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 49, 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:02.508 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 2146, 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 8868043748, 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 17682231296, 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 755411881984, 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 405278533, 2024-01-15T18:00:02.509 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 409839548, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 97, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 104 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 3, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:00:02.510 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 5802 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:02.511 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 12, 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:59:26.004851+0000", 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: "075bcb90-4ab4-4203-8139-75a65f763f34": { 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 41.668% (11s)\n [===========.................] (remaining: 15s)", 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.41667500138282776, 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:02.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:02.513 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.505+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f310806b850 msgr2=0x7f310806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.505+0000 7f3135c00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f310806b850 0x7f310806dd10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3120036000 tx=0x7f312003c040 comp rx=0 tx=0).stop 2024-01-15T18:00:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.505+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3130070e90 msgr2=0x7f31302030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:02.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.505+0000 7f3135c00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3130070e90 0x7f31302030a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f312800cbb0 tx=0x7f312800a430 comp rx=0 tx=0).stop 2024-01-15T18:00:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.510+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 shutdown_connections 2024-01-15T18:00:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.510+0000 7f3135c00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f310806b850 0x7f310806dd10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.510+0000 7f3135c00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3130070e90 0x7f31302030a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.510+0000 7f3135c00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f313015f9c0 0x7f3130070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.510+0000 7f3135c00640 1 --2- 172.21.15.26:0/3346762019 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f313013e360 0x7f31301fccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.510+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 >> 172.21.15.26:0/3346762019 conn(0x7f313006a800 msgr2=0x7f313006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:02.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.510+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 shutdown_connections 2024-01-15T18:00:02.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:02.510+0000 7f3135c00640 1 -- 172.21.15.26:0/3346762019 wait complete. 2024-01-15T18:00:04.551 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:00:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.879+0000 7fea0d200640 1 Processor -- start 2024-01-15T18:00:04.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.879+0000 7fea0d200640 1 -- start start 2024-01-15T18:00:04.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.881+0000 7fea0d200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea08154020 0x7fea08066270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:04.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.881+0000 7fea0d200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea08070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:04.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.882+0000 7fea0d200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0815d550 con 0x7fea0806b690 2024-01-15T18:00:04.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.882+0000 7fea0d200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea081544d0 con 0x7fea08154020 2024-01-15T18:00:04.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.882+0000 7fea0d200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0815a7b0 con 0x7fea0815dd30 2024-01-15T18:00:04.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.883+0000 7fea05e00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fea0806b690 0x7fea08159f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:44790/0 (socket says 172.21.15.26:44790) 2024-01-15T18:00:04.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.883+0000 7fea05e00640 1 -- 172.21.15.26:0/2529701828 learned_addr learned my addr 172.21.15.26:0/2529701828 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:04.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.883+0000 7fea06800640 1 --2- 172.21.15.26:0/2529701828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea08070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.884+0000 7fea04a00640 1 -- 172.21.15.26:0/2529701828 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044437848 0 0) 0x7fea0815d550 con 0x7fea0806b690 2024-01-15T18:00:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.884+0000 7fea04a00640 1 -- 172.21.15.26:0/2529701828 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e8003520 con 0x7fea0806b690 2024-01-15T18:00:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.884+0000 7fea06800640 1 -- 172.21.15.26:0/2529701828 >> v2:172.21.15.26:3300/0 conn(0x7fea08154020 msgr2=0x7fea08066270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.885+0000 7fea06800640 1 --2- 172.21.15.26:0/2529701828 >> v2:172.21.15.26:3300/0 conn(0x7fea08154020 0x7fea08066270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:04.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.885+0000 7fea06800640 1 -- 172.21.15.26:0/2529701828 >> v1:172.21.15.124:6789/0 conn(0x7fea0806b690 legacy=0x7fea08159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:04.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.885+0000 7fea06800640 1 -- 172.21.15.26:0/2529701828 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0815aa90 con 0x7fea0815dd30 2024-01-15T18:00:04.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.888+0000 7fea06800640 1 --2- 172.21.15.26:0/2529701828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea08070af0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80097a0 tx=0x7fe9f802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77de53bb6b564e0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.889+0000 7fea04a00640 1 -- 172.21.15.26:0/2529701828 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8030610 con 0x7fea0815dd30 2024-01-15T18:00:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.889+0000 7fea04a00640 1 -- 172.21.15.26:0/2529701828 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f8030770 con 0x7fea0815dd30 2024-01-15T18:00:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.889+0000 7fea04a00640 1 -- 172.21.15.26:0/2529701828 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8030b40 con 0x7fea0815dd30 2024-01-15T18:00:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.890+0000 7fea0d200640 1 -- 172.21.15.26:0/2529701828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 msgr2=0x7fea08070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:04.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.890+0000 7fea0d200640 1 --2- 172.21.15.26:0/2529701828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea08070af0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe9f80097a0 tx=0x7fe9f802b960 comp rx=0 tx=0).stop 2024-01-15T18:00:04.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.891+0000 7fea0d200640 1 -- 172.21.15.26:0/2529701828 shutdown_connections 2024-01-15T18:00:04.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.891+0000 7fea0d200640 1 --2- 172.21.15.26:0/2529701828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea08070af0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:04.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.891+0000 7fea0d200640 1 --2- 172.21.15.26:0/2529701828 >> v2:172.21.15.26:3300/0 conn(0x7fea08154020 0x7fea08066270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:04.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.892+0000 7fea0d200640 1 -- 172.21.15.26:0/2529701828 >> 172.21.15.26:0/2529701828 conn(0x7fea080cf700 msgr2=0x7fea080cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.892+0000 7fea0d200640 1 -- 172.21.15.26:0/2529701828 shutdown_connections 2024-01-15T18:00:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.892+0000 7fea0d200640 1 -- 172.21.15.26:0/2529701828 wait complete. 2024-01-15T18:00:04.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.896+0000 7fea0d200640 1 Processor -- start 2024-01-15T18:00:04.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.896+0000 7fea0d200640 1 -- start start 2024-01-15T18:00:04.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.906+0000 7fea0d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea0806b690 0x7fea0814a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:04.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.909+0000 7fea0d200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea08154020 0x7fea080b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:04.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.909+0000 7fea05e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea0806b690 0x7fea0814a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:04.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.912+0000 7fea05e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea0806b690 0x7fea0814a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54018/0 (socket says 172.21.15.26:54018) 2024-01-15T18:00:04.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.912+0000 7fea05e00640 1 -- 172.21.15.26:0/627266838 learned_addr learned my addr 172.21.15.26:0/627266838 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:04.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.913+0000 7fea0d200640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea080b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:04.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.913+0000 7fea05400640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea08154020 0x7fea080b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.913+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea080b2a80 con 0x7fea0806b690 2024-01-15T18:00:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.915+0000 7fea06800640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea080b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.915+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea080cf2c0 con 0x7fea08154020 2024-01-15T18:00:04.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.915+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea080b2430 con 0x7fea0815dd30 2024-01-15T18:00:04.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.918+0000 7fea06800640 1 -- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea08154020 msgr2=0x7fea080b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.918+0000 7fea06800640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea08154020 0x7fea080b2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.918+0000 7fea06800640 1 -- 172.21.15.26:0/627266838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea0806b690 msgr2=0x7fea0814a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.918+0000 7fea06800640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea0806b690 0x7fea0814a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.919+0000 7fea06800640 1 -- 172.21.15.26:0/627266838 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f8009400 con 0x7fea0815dd30 2024-01-15T18:00:04.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.919+0000 7fea05400640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea08154020 0x7fea080b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:00:04.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.919+0000 7fea05e00640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea0806b690 0x7fea0814a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:00:04.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.921+0000 7fea06800640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea080b4230 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8033000 tx=0x7fe9f80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.937+0000 7fe9fe000640 1 -- 172.21.15.26:0/627266838 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f803f070 con 0x7fea0815dd30 2024-01-15T18:00:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.937+0000 7fe9fe000640 1 -- 172.21.15.26:0/627266838 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f8030e30 con 0x7fea0815dd30 2024-01-15T18:00:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.937+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea080b2710 con 0x7fea0815dd30 2024-01-15T18:00:04.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.938+0000 7fe9fe000640 1 -- 172.21.15.26:0/627266838 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f803a950 con 0x7fea0815dd30 2024-01-15T18:00:04.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.938+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea080b3450 con 0x7fea0815dd30 2024-01-15T18:00:04.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.940+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea080b3640 con 0x7fea0815dd30 2024-01-15T18:00:04.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.968+0000 7fe9fe000640 1 -- 172.21.15.26:0/627266838 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7fe9f8043070 con 0x7fea0815dd30 2024-01-15T18:00:04.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.969+0000 7fe9fe000640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe9e006b8a0 0x7fe9e006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:04.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.969+0000 7fe9fe000640 1 -- 172.21.15.26:0/627266838 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe9f80ac210 con 0x7fea0815dd30 2024-01-15T18:00:04.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.973+0000 7fea05e00640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe9e006b8a0 0x7fe9e006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:04.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.975+0000 7fe9fe000640 1 -- 172.21.15.26:0/627266838 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9f8076b60 con 0x7fea0815dd30 2024-01-15T18:00:04.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:04.984+0000 7fea05e00640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe9e006b8a0 0x7fe9e006dd60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fea00005c00 tx=0x7fea00009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:05.388 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:05.388 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:05.388 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:00:05.388 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:00:05.388 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:00:05.388 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:00:05.388 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:00:05.388 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:05.389 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:00:05.389 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:00:05.389 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:00:05.389 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:00:05.389 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:00:05.389 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:05.389 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:00:05.389 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:00:05.389 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 319, 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:00:05.390 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 49, 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:00:05.391 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:00:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:00:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:00:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:00:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 3010, 2024-01-15T18:00:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 12492035550, 2024-01-15T18:00:05.392 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 24565391360, 2024-01-15T18:00:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 748528721920, 2024-01-15T18:00:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:00:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 716970735, 2024-01-15T18:00:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 711758728, 2024-01-15T18:00:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 171, 2024-01-15T18:00:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 179 2024-01-15T18:00:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:00:05.393 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3, 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 3, 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:00:05.394 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:00:05.395 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 5802 2024-01-15T18:00:05.395 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:05.395 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:05.395 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:00:05.395 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:05.395 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:00:05.395 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:00:05.395 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:00:05.395 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:00:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:00:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:00:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:00:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:00:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:00:05.396 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 12, 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:59:26.004851+0000", 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "075bcb90-4ab4-4203-8139-75a65f763f34": { 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "2 ongoing cloning jobs; Avg progress = 50.0% (0s)\n [==============..............] ", 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.5, 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:05.397 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:05.398 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.382+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fea080b2570 con 0x7fea0815dd30 2024-01-15T18:00:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.383+0000 7fe9fe000640 1 -- 172.21.15.26:0/627266838 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2113 (secure 0 0 0) 0x7fe9f8034860 con 0x7fea0815dd30 2024-01-15T18:00:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.391+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe9e006b8a0 msgr2=0x7fe9e006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.391+0000 7fea0d200640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe9e006b8a0 0x7fe9e006dd60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fea00005c00 tx=0x7fea00009040 comp rx=0 tx=0).stop 2024-01-15T18:00:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.391+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 msgr2=0x7fea080b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.391+0000 7fea0d200640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea080b4230 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8033000 tx=0x7fe9f80046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.392+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 shutdown_connections 2024-01-15T18:00:05.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.392+0000 7fea0d200640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea0815dd30 0x7fea080b4230 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.392+0000 7fea0d200640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe9e006b8a0 0x7fe9e006dd60 secure :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fea00005c00 tx=0x7fea00009040 comp rx=0 tx=0).stop 2024-01-15T18:00:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.392+0000 7fea0d200640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea08154020 0x7fea080b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.392+0000 7fea0d200640 1 --2- 172.21.15.26:0/627266838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea0806b690 0x7fea0814a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:05.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.392+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 >> 172.21.15.26:0/627266838 conn(0x7fea080cf700 msgr2=0x7fea08152c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:05.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.397+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 shutdown_connections 2024-01-15T18:00:05.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:05.397+0000 7fea0d200640 1 -- 172.21.15.26:0/627266838 wait complete. 2024-01-15T18:00:07.424 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:00:07.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.692+0000 7f7276c00640 1 Processor -- start 2024-01-15T18:00:07.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.693+0000 7f7276c00640 1 -- start start 2024-01-15T18:00:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.694+0000 7f7276c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 0x7f72700b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.694+0000 7f7276c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f727006c6b0 0x7f7270099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.694+0000 7f7276c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727006cea0 con 0x7f7270099fa0 2024-01-15T18:00:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.694+0000 7f7276c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727009a430 con 0x7f727006c6b0 2024-01-15T18:00:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.694+0000 7f7276c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727006cbc0 con 0x7f72700cd240 2024-01-15T18:00:07.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.694+0000 7f7267200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f727006c6b0 0x7f7270099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.694+0000 7f7267200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f727006c6b0 0x7f7270099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54032/0 (socket says 172.21.15.26:54032) 2024-01-15T18:00:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.694+0000 7f7267200640 1 -- 172.21.15.26:0/1272563104 learned_addr learned my addr 172.21.15.26:0/1272563104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.694+0000 7f726fe00640 1 --2- 172.21.15.26:0/1272563104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 0x7f72700b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.695+0000 7f7267200640 1 -- 172.21.15.26:0/1272563104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 msgr2=0x7f72700b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.695+0000 7f7267200640 1 --2- 172.21.15.26:0/1272563104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 0x7f72700b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.695+0000 7f7267200640 1 -- 172.21.15.26:0/1272563104 >> v1:172.21.15.124:6789/0 conn(0x7f7270099fa0 legacy=0x7f727006b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.695+0000 7f7267200640 1 -- 172.21.15.26:0/1272563104 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727006f350 con 0x7f727006c6b0 2024-01-15T18:00:07.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.695+0000 7f726fe00640 1 --2- 172.21.15.26:0/1272563104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 0x7f72700b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.696+0000 7f7267200640 1 --2- 172.21.15.26:0/1272563104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 0x7f7270099a60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7258009870 tx=0x7f725802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35b5228f3747633 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.696+0000 7f726f400640 1 -- 172.21.15.26:0/1272563104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258030840 con 0x7f727006c6b0 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.696+0000 7f726f400640 1 -- 172.21.15.26:0/1272563104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72580309a0 con 0x7f727006c6b0 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.696+0000 7f726f400640 1 -- 172.21.15.26:0/1272563104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258030d30 con 0x7f727006c6b0 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.697+0000 7f7276c00640 1 -- 172.21.15.26:0/1272563104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 msgr2=0x7f7270099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.697+0000 7f7276c00640 1 --2- 172.21.15.26:0/1272563104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 0x7f7270099a60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f7258009870 tx=0x7f725802b980 comp rx=0 tx=0).stop 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.697+0000 7f7276c00640 1 -- 172.21.15.26:0/1272563104 shutdown_connections 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.697+0000 7f7276c00640 1 --2- 172.21.15.26:0/1272563104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 0x7f7270099a60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.697+0000 7f7276c00640 1 --2- 172.21.15.26:0/1272563104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 0x7f72700b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:07.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.697+0000 7f7276c00640 1 -- 172.21.15.26:0/1272563104 >> 172.21.15.26:0/1272563104 conn(0x7f72700b2d00 msgr2=0x7f72700b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.697+0000 7f7276c00640 1 -- 172.21.15.26:0/1272563104 shutdown_connections 2024-01-15T18:00:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.697+0000 7f7276c00640 1 -- 172.21.15.26:0/1272563104 wait complete. 2024-01-15T18:00:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.699+0000 7f7276c00640 1 Processor -- start 2024-01-15T18:00:07.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.699+0000 7f7276c00640 1 -- start start 2024-01-15T18:00:07.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f7276c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 0x7f72700b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:07.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f7276c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7270099fa0 0x7f72700c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:07.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f7276c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 0x7f72700c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:07.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f7276c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72700b54a0 con 0x7f727006c6b0 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f7276c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72700b45a0 con 0x7f7270099fa0 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f7276c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72700b4820 con 0x7f72700cd240 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f726fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 0x7f72700b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f726fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 0x7f72700b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54040/0 (socket says 172.21.15.26:54040) 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f7267200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7270099fa0 0x7f72700c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f726fe00640 1 -- 172.21.15.26:0/3651196661 learned_addr learned my addr 172.21.15.26:0/3651196661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.700+0000 7f7274c00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 0x7f72700c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.701+0000 7f726fe00640 1 -- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 msgr2=0x7f72700c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.701+0000 7f726fe00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 0x7f72700c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.701+0000 7f726fe00640 1 -- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7270099fa0 msgr2=0x7f72700c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.701+0000 7f726fe00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7270099fa0 0x7f72700c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:07.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.701+0000 7f726fe00640 1 -- 172.21.15.26:0/3651196661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72580094d0 con 0x7f727006c6b0 2024-01-15T18:00:07.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.701+0000 7f726fe00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 0x7f72700b4060 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7260009fc0 tx=0x7f726000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:07.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.702+0000 7f726cc00640 1 -- 172.21.15.26:0/3651196661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260003b90 con 0x7f727006c6b0 2024-01-15T18:00:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.702+0000 7f726cc00640 1 -- 172.21.15.26:0/3651196661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726000f040 con 0x7f727006c6b0 2024-01-15T18:00:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.702+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72700b4b00 con 0x7f727006c6b0 2024-01-15T18:00:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.702+0000 7f726cc00640 1 -- 172.21.15.26:0/3651196661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260004030 con 0x7f727006c6b0 2024-01-15T18:00:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.702+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72700b50c0 con 0x7f727006c6b0 2024-01-15T18:00:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.703+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72700c4e70 con 0x7f727006c6b0 2024-01-15T18:00:07.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.704+0000 7f726cc00640 1 -- 172.21.15.26:0/3651196661 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7f7260004190 con 0x7f727006c6b0 2024-01-15T18:00:07.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.705+0000 7f726cc00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f724806b7d0 0x7f724806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:07.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.705+0000 7f726cc00640 1 -- 172.21.15.26:0/3651196661 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f726008a810 con 0x7f727006c6b0 2024-01-15T18:00:07.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.706+0000 7f7267200640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f724806b7d0 0x7f724806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:07.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.706+0000 7f7267200640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f724806b7d0 0x7f724806dc90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7258036000 tx=0x7f7258008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:07.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:07.710+0000 7f726cc00640 1 -- 172.21.15.26:0/3651196661 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7260055000 con 0x7f727006c6b0 2024-01-15T18:00:08.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.184+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f72700675d0 con 0x7f727006c6b0 2024-01-15T18:00:08.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.185+0000 7f726cc00640 1 -- 172.21.15.26:0/3651196661 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1870 (secure 0 0 0) 0x7f726001a090 con 0x7f727006c6b0 2024-01-15T18:00:08.189 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:08.189 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:08.189 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:00:08.189 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:00:08.190 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 322, 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 49, 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:00:08.191 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 3622, 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 15046601982, 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 30261735424, 2024-01-15T18:00:08.192 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 742832377856, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 733687474, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 741065112, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 175, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 189 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 26, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 3, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 3, 2024-01-15T18:00:08.193 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 5802 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:00:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 12, 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T17:59:26.004851+0000", 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": {} 2024-01-15T18:00:08.195 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f724806b7d0 msgr2=0x7f724806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f724806b7d0 0x7f724806dc90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7258036000 tx=0x7f7258008030 comp rx=0 tx=0).stop 2024-01-15T18:00:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 msgr2=0x7f72700b4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 0x7f72700b4060 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7260009fc0 tx=0x7f726000d560 comp rx=0 tx=0).stop 2024-01-15T18:00:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 shutdown_connections 2024-01-15T18:00:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f72700cd240 0x7f72700c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f724806b7d0 0x7f724806dc90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7270099fa0 0x7f72700c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 --2- 172.21.15.26:0/3651196661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f727006c6b0 0x7f72700b4060 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 >> 172.21.15.26:0/3651196661 conn(0x7f72700b2d00 msgr2=0x7f72700cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 shutdown_connections 2024-01-15T18:00:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.188+0000 7f7276c00640 1 -- 172.21.15.26:0/3651196661 wait complete. 2024-01-15T18:00:08.217 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:00:08.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone6931 2024-01-15T18:00:08.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.499+0000 7f2725e00640 1 Processor -- start 2024-01-15T18:00:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.500+0000 7f2725e00640 1 -- start start 2024-01-15T18:00:08.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.501+0000 7f2725e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f27200cd170 0x7f2720071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:08.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.501+0000 7f2725e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2720071690 con 0x7f272014b2c0 2024-01-15T18:00:08.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.501+0000 7f2725e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2720071970 con 0x7f27200cf9a0 2024-01-15T18:00:08.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.501+0000 7f2725e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272014b6a0 con 0x7f27200cd170 2024-01-15T18:00:08.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.502+0000 7f271ea00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f272014b2c0 0x7f27200cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39454/0 (socket says 172.21.15.26:39454) 2024-01-15T18:00:08.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.502+0000 7f271ea00640 1 -- 172.21.15.26:0/2058351963 learned_addr learned my addr 172.21.15.26:0/2058351963 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166514583 0 0) 0x7f2720071690 con 0x7f272014b2c0 2024-01-15T18:00:08.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2700003520 con 0x7f272014b2c0 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182628003 0 0) 0x7f2720071970 con 0x7f27200cf9a0 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2720071690 con 0x7f27200cf9a0 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2270823262 0 0) 0x7f2700003520 con 0x7f272014b2c0 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 >> v2:172.21.15.124:3300/0 conn(0x7f27200cd170 msgr2=0x7f2720071070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 --2- 172.21.15.26:0/2058351963 >> v2:172.21.15.124:3300/0 conn(0x7f27200cd170 0x7f2720071070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 >> v1:172.21.15.124:6789/0 conn(0x7f27200cf9a0 legacy=0x7f2720070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2720071c50 con 0x7f272014b2c0 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f2710001670 con 0x7f272014b2c0 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 >> v1:172.21.15.26:6789/0 conn(0x7f272014b2c0 legacy=0x7f27200cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2700004bb0 0x7f2700004fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700003760 0x7f2700003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700004240 0x7f270000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272014b6a0 con 0x7f2700003760 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270000c1f0 con 0x7f2700004bb0 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.503+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270000c3b0 con 0x7f2700004240 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.504+0000 7f271e000640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700003760 0x7f2700003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.504+0000 7f271ea00640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2700004bb0 0x7f2700004fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.504+0000 7f271e000640 1 -- 172.21.15.26:0/2058351963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700004240 msgr2=0x7f270000b980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.504+0000 7f271e000640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700004240 0x7f270000b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.504+0000 7f271e000640 1 -- 172.21.15.26:0/2058351963 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2700004bb0 msgr2=0x7f2700004fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.504+0000 7f271e000640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2700004bb0 0x7f2700004fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.504+0000 7f271e000640 1 -- 172.21.15.26:0/2058351963 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2700003520 con 0x7f2700003760 2024-01-15T18:00:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.504+0000 7f271ea00640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2700004bb0 0x7f2700004fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.504+0000 7f271f400640 1 -- 172.21.15.26:0/2058351963 reap_dead start 2024-01-15T18:00:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.506+0000 7f271e000640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700003760 0x7f2700003be0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f27080018b0 tx=0x7f27000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f295d804d0a7e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.506+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27080052d0 con 0x7f2700003760 2024-01-15T18:00:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.506+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2708005450 con 0x7f2700003760 2024-01-15T18:00:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.506+0000 7f271d600640 1 -- 172.21.15.26:0/2058351963 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708007260 con 0x7f2700003760 2024-01-15T18:00:08.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.508+0000 7f2725e00640 1 -- 172.21.15.26:0/2058351963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700003760 msgr2=0x7f2700003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:08.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.508+0000 7f2725e00640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700003760 0x7f2700003be0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f27080018b0 tx=0x7f27000057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.509+0000 7f2725e00640 1 -- 172.21.15.26:0/2058351963 shutdown_connections 2024-01-15T18:00:08.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.509+0000 7f2725e00640 1 --2- 172.21.15.26:0/2058351963 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2700003760 0x7f2700003be0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.509+0000 7f2725e00640 1 -- 172.21.15.26:0/2058351963 >> 172.21.15.26:0/2058351963 conn(0x7f272006a800 msgr2=0x7f272006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:08.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.509+0000 7f2725e00640 1 -- 172.21.15.26:0/2058351963 shutdown_connections 2024-01-15T18:00:08.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.510+0000 7f2725e00640 1 -- 172.21.15.26:0/2058351963 wait complete. 2024-01-15T18:00:08.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.512+0000 7f2725e00640 1 Processor -- start 2024-01-15T18:00:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.512+0000 7f2725e00640 1 -- start start 2024-01-15T18:00:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.513+0000 7f2725e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700003760 0x7f2720162320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.513+0000 7f2725e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272015e210 0x7f2720162860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.514+0000 7f2725e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2720143e10 0x7f2720144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.514+0000 7f2725e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2720164df0 con 0x7f272015e210 2024-01-15T18:00:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.514+0000 7f2725e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2720164960 con 0x7f2720143e10 2024-01-15T18:00:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.514+0000 7f2725e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2720142350 con 0x7f2700003760 2024-01-15T18:00:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.514+0000 7f271ea00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700003760 0x7f2720162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.514+0000 7f271ea00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700003760 0x7f2720162320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:54908/0 (socket says 172.21.15.26:54908) 2024-01-15T18:00:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.514+0000 7f271ea00640 1 -- 172.21.15.26:0/3992847013 learned_addr learned my addr 172.21.15.26:0/3992847013 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:08.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.515+0000 7f271e000640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272015e210 0x7f2720162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:08.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.515+0000 7f271f400640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2720143e10 0x7f2720144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:08.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.515+0000 7f271ea00640 1 -- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2720143e10 msgr2=0x7f2720144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:08.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.515+0000 7f271ea00640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2720143e10 0x7f2720144290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.515+0000 7f271ea00640 1 -- 172.21.15.26:0/3992847013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272015e210 msgr2=0x7f2720162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:08.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.515+0000 7f271ea00640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272015e210 0x7f2720162860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.515+0000 7f271ea00640 1 -- 172.21.15.26:0/3992847013 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2720164030 con 0x7f2700003760 2024-01-15T18:00:08.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.515+0000 7f271ea00640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700003760 0x7f2720162320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f271000d800 tx=0x7f271000dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:08.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.517+0000 7f2717400640 1 -- 172.21.15.26:0/3992847013 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27100074e0 con 0x7f2700003760 2024-01-15T18:00:08.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.517+0000 7f2717400640 1 -- 172.21.15.26:0/3992847013 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2710007640 con 0x7f2700003760 2024-01-15T18:00:08.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.517+0000 7f2717400640 1 -- 172.21.15.26:0/3992847013 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710004a00 con 0x7f2700003760 2024-01-15T18:00:08.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.517+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2720164310 con 0x7f2700003760 2024-01-15T18:00:08.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.517+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27201630f0 con 0x7f2700003760 2024-01-15T18:00:08.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.518+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27200675d0 con 0x7f2700003760 2024-01-15T18:00:08.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.520+0000 7f2717400640 1 -- 172.21.15.26:0/3992847013 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7f2710004b60 con 0x7f2700003760 2024-01-15T18:00:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.522+0000 7f2717400640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f26f806b7d0 0x7f26f806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:08.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.522+0000 7f2717400640 1 -- 172.21.15.26:0/3992847013 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2710089ce0 con 0x7f2700003760 2024-01-15T18:00:08.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.526+0000 7f271e000640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f26f806b7d0 0x7f26f806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:08.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.529+0000 7f2717400640 1 -- 172.21.15.26:0/3992847013 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27100544d0 con 0x7f2700003760 2024-01-15T18:00:08.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.531+0000 7f271e000640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f26f806b7d0 0x7f26f806dc90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2708033000 tx=0x7f2708009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:08.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.889+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone6931", "target": ["mon-mgr", ""]}) v1 -- 0x7f27201621b0 con 0x7f26f806b7d0 2024-01-15T18:00:08.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.968+0000 7f2717400640 1 -- 172.21.15.26:0/3992847013 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f27201621b0 con 0x7f26f806b7d0 2024-01-15T18:00:08.973 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:08.973 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:00:08.974 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:00:08.974 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:08.974 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.970+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f26f806b7d0 msgr2=0x7f26f806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.970+0000 7f2725e00640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f26f806b7d0 0x7f26f806dc90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2708033000 tx=0x7f2708009030 comp rx=0 tx=0).stop 2024-01-15T18:00:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.970+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700003760 msgr2=0x7f2720162320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.970+0000 7f2725e00640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700003760 0x7f2720162320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f271000d800 tx=0x7f271000dcc0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.971+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 shutdown_connections 2024-01-15T18:00:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.971+0000 7f2725e00640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f26f806b7d0 0x7f26f806dc90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.971+0000 7f2725e00640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2720143e10 0x7f2720144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.971+0000 7f2725e00640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f272015e210 0x7f2720162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.971+0000 7f2725e00640 1 --2- 172.21.15.26:0/3992847013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2700003760 0x7f2720162320 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.971+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 >> 172.21.15.26:0/3992847013 conn(0x7f272006a800 msgr2=0x7f272006cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.971+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 shutdown_connections 2024-01-15T18:00:08.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:08.971+0000 7f2725e00640 1 -- 172.21.15.26:0/3992847013 wait complete. 2024-01-15T18:00:08.996 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:00:08.996 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone6932 2024-01-15T18:00:09.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.195+0000 7f1917e00640 1 Processor -- start 2024-01-15T18:00:09.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.195+0000 7f1917e00640 1 -- start start 2024-01-15T18:00:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.196+0000 7f1917e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f191815b060 0x7f19180ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.196+0000 7f1917e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f19180714a0 0x7f191806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.196+0000 7f1917e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19180cdbe0 con 0x7f19180ceb80 2024-01-15T18:00:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.196+0000 7f1917e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19180cf010 con 0x7f191815b060 2024-01-15T18:00:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.196+0000 7f1917e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191806e4c0 con 0x7f19180714a0 2024-01-15T18:00:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.197+0000 7f1916a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f191815b060 0x7f19180ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.197+0000 7f1916a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f191815b060 0x7f19180ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54054/0 (socket says 172.21.15.26:54054) 2024-01-15T18:00:09.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.197+0000 7f1916a00640 1 -- 172.21.15.26:0/3365625530 learned_addr learned my addr 172.21.15.26:0/3365625530 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:09.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.197+0000 7f1917400640 1 --2- 172.21.15.26:0/3365625530 >> v2:172.21.15.124:3300/0 conn(0x7f19180714a0 0x7f191806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.197+0000 7f1916a00640 1 -- 172.21.15.26:0/3365625530 >> v2:172.21.15.124:3300/0 conn(0x7f19180714a0 msgr2=0x7f191806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:09.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.197+0000 7f1916a00640 1 --2- 172.21.15.26:0/3365625530 >> v2:172.21.15.124:3300/0 conn(0x7f19180714a0 0x7f191806a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.197+0000 7f1916a00640 1 -- 172.21.15.26:0/3365625530 >> v1:172.21.15.26:6789/0 conn(0x7f19180ceb80 legacy=0x7f1918070d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:09.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.197+0000 7f1916a00640 1 -- 172.21.15.26:0/3365625530 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19180cde80 con 0x7f191815b060 2024-01-15T18:00:09.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.197+0000 7f1917400640 1 --2- 172.21.15.26:0/3365625530 >> v2:172.21.15.124:3300/0 conn(0x7f19180714a0 0x7f191806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:09.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.198+0000 7f1916a00640 1 --2- 172.21.15.26:0/3365625530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191815b060 0x7f19180ce640 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f191806bfb0 tx=0x7f191002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccd9e736b87859 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:09.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.199+0000 7f1915600640 1 -- 172.21.15.26:0/3365625530 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191002cb00 con 0x7f191815b060 2024-01-15T18:00:09.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.199+0000 7f1915600640 1 -- 172.21.15.26:0/3365625530 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191002cc60 con 0x7f191815b060 2024-01-15T18:00:09.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.199+0000 7f1915600640 1 -- 172.21.15.26:0/3365625530 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100343f0 con 0x7f191815b060 2024-01-15T18:00:09.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.200+0000 7f1917e00640 1 -- 172.21.15.26:0/3365625530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191815b060 msgr2=0x7f19180ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:09.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.200+0000 7f1917e00640 1 --2- 172.21.15.26:0/3365625530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191815b060 0x7f19180ce640 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f191806bfb0 tx=0x7f191002bf60 comp rx=0 tx=0).stop 2024-01-15T18:00:09.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.200+0000 7f1917e00640 1 -- 172.21.15.26:0/3365625530 shutdown_connections 2024-01-15T18:00:09.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.200+0000 7f1917e00640 1 --2- 172.21.15.26:0/3365625530 >> v2:172.21.15.124:3300/0 conn(0x7f19180714a0 0x7f191806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.200+0000 7f1917e00640 1 --2- 172.21.15.26:0/3365625530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f191815b060 0x7f19180ce640 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.200+0000 7f1917e00640 1 -- 172.21.15.26:0/3365625530 >> 172.21.15.26:0/3365625530 conn(0x7f19180b15b0 msgr2=0x7f19180b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:09.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.200+0000 7f1917e00640 1 -- 172.21.15.26:0/3365625530 shutdown_connections 2024-01-15T18:00:09.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.200+0000 7f1917e00640 1 -- 172.21.15.26:0/3365625530 wait complete. 2024-01-15T18:00:09.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.202+0000 7f1917e00640 1 Processor -- start 2024-01-15T18:00:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.202+0000 7f1917e00640 1 -- start start 2024-01-15T18:00:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1917e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19180714a0 0x7f19180a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1917e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19180ceb80 0x7f191806d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1917e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f191815b060 0x7f1918156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1917e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19181f3fd0 con 0x7f19180ceb80 2024-01-15T18:00:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1917e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19181f4220 con 0x7f191815b060 2024-01-15T18:00:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1917e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19181f4500 con 0x7f19180714a0 2024-01-15T18:00:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1916000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19180ceb80 0x7f191806d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1916000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19180ceb80 0x7f191806d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54058/0 (socket says 172.21.15.26:54058) 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1916000640 1 -- 172.21.15.26:0/3590208102 learned_addr learned my addr 172.21.15.26:0/3590208102 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1916a00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19180714a0 0x7f19180a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1917400640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f191815b060 0x7f1918156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1916000640 1 -- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19180714a0 msgr2=0x7f19180a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1916000640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19180714a0 0x7f19180a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1916000640 1 -- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f191815b060 msgr2=0x7f1918156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1916000640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f191815b060 0x7f1918156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.203+0000 7f1916000640 1 -- 172.21.15.26:0/3590208102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910009b30 con 0x7f19180ceb80 2024-01-15T18:00:09.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.204+0000 7f1916000640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19180ceb80 0x7f191806d490 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f190000c450 tx=0x7f190000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:09.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.204+0000 7f190ea00640 1 -- 172.21.15.26:0/3590208102 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000052d0 con 0x7f19180ceb80 2024-01-15T18:00:09.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.204+0000 7f190ea00640 1 -- 172.21.15.26:0/3590208102 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1900005430 con 0x7f19180ceb80 2024-01-15T18:00:09.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.204+0000 7f190ea00640 1 -- 172.21.15.26:0/3590208102 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900012870 con 0x7f19180ceb80 2024-01-15T18:00:09.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.204+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19181f47e0 con 0x7f19180ceb80 2024-01-15T18:00:09.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.204+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19180a8a00 con 0x7f19180ceb80 2024-01-15T18:00:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.206+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19181f4d00 con 0x7f19180ceb80 2024-01-15T18:00:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.206+0000 7f190ea00640 1 -- 172.21.15.26:0/3590208102 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7f19000129d0 con 0x7f19180ceb80 2024-01-15T18:00:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.207+0000 7f190ea00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18f006b7d0 0x7f18f006dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.207+0000 7f190ea00640 1 -- 172.21.15.26:0/3590208102 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1900089ee0 con 0x7f19180ceb80 2024-01-15T18:00:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.208+0000 7f1916a00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18f006b7d0 0x7f18f006dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.209+0000 7f1916a00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18f006b7d0 0x7f18f006dc90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1910035000 tx=0x7f1910030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:09.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.212+0000 7f190ea00640 1 -- 172.21.15.26:0/3590208102 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19181f4d00 con 0x7f19180ceb80 2024-01-15T18:00:09.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.445+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone6932", "target": ["mon-mgr", ""]}) v1 -- 0x7f191806b310 con 0x7f18f006b7d0 2024-01-15T18:00:09.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.469+0000 7f190ea00640 1 -- 172.21.15.26:0/3590208102 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f1910009510 con 0x7f18f006b7d0 2024-01-15T18:00:09.473 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:09.473 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:00:09.474 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:00:09.474 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:09.474 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18f006b7d0 msgr2=0x7f18f006dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18f006b7d0 0x7f18f006dc90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1910035000 tx=0x7f1910030000 comp rx=0 tx=0).stop 2024-01-15T18:00:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19180ceb80 msgr2=0x7f191806d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19180ceb80 0x7f191806d490 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f190000c450 tx=0x7f190000c910 comp rx=0 tx=0).stop 2024-01-15T18:00:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 shutdown_connections 2024-01-15T18:00:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18f006b7d0 0x7f18f006dc90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f191815b060 0x7f1918156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19180ceb80 0x7f191806d490 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 --2- 172.21.15.26:0/3590208102 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19180714a0 0x7f19180a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 >> 172.21.15.26:0/3590208102 conn(0x7f19180b15b0 msgr2=0x7f191815e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 shutdown_connections 2024-01-15T18:00:09.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.472+0000 7f1917e00640 1 -- 172.21.15.26:0/3590208102 wait complete. 2024-01-15T18:00:09.496 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:00:09.496 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone6933 2024-01-15T18:00:09.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.710+0000 7ffafa600640 1 Processor -- start 2024-01-15T18:00:09.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.710+0000 7ffafa600640 1 -- start start 2024-01-15T18:00:09.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.711+0000 7ffafa600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffaf4143ae0 0x7ffaf40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.711+0000 7ffafa600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ffaf415e3b0 0x7ffaf40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.711+0000 7ffafa600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf40cfc30 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.711+0000 7ffafa600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf40b1ee0 con 0x7ffaf4143ae0 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.711+0000 7ffafa600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf40b2780 con 0x7ffaf415e3b0 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.711+0000 7ffaf3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffaf4143ae0 0x7ffaf40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.712+0000 7ffaf3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffaf4143ae0 0x7ffaf40cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54068/0 (socket says 172.21.15.26:54068) 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.712+0000 7ffaf3400640 1 -- 172.21.15.26:0/2184974766 learned_addr learned my addr 172.21.15.26:0/2184974766 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.712+0000 7ffaf3400640 1 -- 172.21.15.26:0/2184974766 >> v2:172.21.15.124:3300/0 conn(0x7ffaf415e3b0 msgr2=0x7ffaf40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.712+0000 7ffaf3e00640 1 --2- 172.21.15.26:0/2184974766 >> v2:172.21.15.124:3300/0 conn(0x7ffaf415e3b0 0x7ffaf40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.712+0000 7ffaf3400640 1 --2- 172.21.15.26:0/2184974766 >> v2:172.21.15.124:3300/0 conn(0x7ffaf415e3b0 0x7ffaf40702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.712+0000 7ffaf3400640 1 -- 172.21.15.26:0/2184974766 >> v1:172.21.15.124:6789/0 conn(0x7ffaf40cf7d0 legacy=0x7ffaf415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:09.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.712+0000 7ffaf3400640 1 -- 172.21.15.26:0/2184974766 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf415e990 con 0x7ffaf4143ae0 2024-01-15T18:00:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.713+0000 7ffaf3400640 1 --2- 172.21.15.26:0/2184974766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf4143ae0 0x7ffaf40cf290 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ffaf406e4f0 tx=0x7ffae402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed4fd00270c3b2d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffaf2000640 1 -- 172.21.15.26:0/2184974766 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4031870 con 0x7ffaf4143ae0 2024-01-15T18:00:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffaf2000640 1 -- 172.21.15.26:0/2184974766 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae40319d0 con 0x7ffaf4143ae0 2024-01-15T18:00:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffaf2000640 1 -- 172.21.15.26:0/2184974766 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4031d60 con 0x7ffaf4143ae0 2024-01-15T18:00:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffafa600640 1 -- 172.21.15.26:0/2184974766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf4143ae0 msgr2=0x7ffaf40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffafa600640 1 --2- 172.21.15.26:0/2184974766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf4143ae0 0x7ffaf40cf290 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ffaf406e4f0 tx=0x7ffae402b910 comp rx=0 tx=0).stop 2024-01-15T18:00:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffafa600640 1 -- 172.21.15.26:0/2184974766 shutdown_connections 2024-01-15T18:00:09.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffafa600640 1 --2- 172.21.15.26:0/2184974766 >> v2:172.21.15.124:3300/0 conn(0x7ffaf415e3b0 0x7ffaf40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffafa600640 1 --2- 172.21.15.26:0/2184974766 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf4143ae0 0x7ffaf40cf290 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffafa600640 1 -- 172.21.15.26:0/2184974766 >> 172.21.15.26:0/2184974766 conn(0x7ffaf406a800 msgr2=0x7ffaf406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:09.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffafa600640 1 -- 172.21.15.26:0/2184974766 shutdown_connections 2024-01-15T18:00:09.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.714+0000 7ffafa600640 1 -- 172.21.15.26:0/2184974766 wait complete. 2024-01-15T18:00:09.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.716+0000 7ffafa600640 1 Processor -- start 2024-01-15T18:00:09.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.716+0000 7ffafa600640 1 -- start start 2024-01-15T18:00:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.717+0000 7ffafa600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40cf7d0 0x7ffaf413c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.717+0000 7ffafa600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffaf4143ae0 0x7ffaf4138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.717+0000 7ffafa600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffaf415e3b0 0x7ffaf4159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.717+0000 7ffafa600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf4158120 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.717+0000 7ffafa600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf40aff30 con 0x7ffaf415e3b0 2024-01-15T18:00:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.717+0000 7ffafa600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf40b0210 con 0x7ffaf4143ae0 2024-01-15T18:00:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.717+0000 7ffaf3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40cf7d0 0x7ffaf413c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40cf7d0 0x7ffaf413c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54082/0 (socket says 172.21.15.26:54082) 2024-01-15T18:00:09.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3400640 1 -- 172.21.15.26:0/1944787669 learned_addr learned my addr 172.21.15.26:0/1944787669 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3e00640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffaf415e3b0 0x7ffaf4159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf2a00640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffaf4143ae0 0x7ffaf4138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3400640 1 -- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffaf4143ae0 msgr2=0x7ffaf4138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3400640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffaf4143ae0 0x7ffaf4138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3400640 1 -- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffaf415e3b0 msgr2=0x7ffaf4159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:09.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3400640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffaf415e3b0 0x7ffaf4159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3400640 1 -- 172.21.15.26:0/1944787669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae40094d0 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3e00640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffaf415e3b0 0x7ffaf4159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf2a00640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffaf4143ae0 0x7ffaf4138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.718+0000 7ffaf3400640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40cf7d0 0x7ffaf413c7d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ffae402b950 tx=0x7ffae40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.719+0000 7ffaeb400640 1 -- 172.21.15.26:0/1944787669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4031860 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.719+0000 7ffaeb400640 1 -- 172.21.15.26:0/1944787669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae40443f0 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.719+0000 7ffaeb400640 1 -- 172.21.15.26:0/1944787669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae40447a0 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.719+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf40b04f0 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.719+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf413cd10 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.720+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf40675d0 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.721+0000 7ffaeb400640 1 -- 172.21.15.26:0/1944787669 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7ffae404d450 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.722+0000 7ffaeb400640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffacc06b7d0 0x7ffacc06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:09.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.722+0000 7ffaeb400640 1 -- 172.21.15.26:0/1944787669 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffae402d040 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.723+0000 7ffaf2a00640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffacc06b7d0 0x7ffacc06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.724+0000 7ffaf2a00640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffacc06b7d0 0x7ffacc06dc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffaf41f6880 tx=0x7ffadc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:09.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.726+0000 7ffaeb400640 1 -- 172.21.15.26:0/1944787669 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffae4077390 con 0x7ffaf40cf7d0 2024-01-15T18:00:09.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:09.962+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone6933", "target": ["mon-mgr", ""]}) v1 -- 0x7ffaf40b2410 con 0x7ffacc06b7d0 2024-01-15T18:00:10.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.092+0000 7ffaeb400640 1 -- 172.21.15.26:0/1944787669 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7ffadc005030 con 0x7ffacc06b7d0 2024-01-15T18:00:10.097 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:10.097 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:00:10.098 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:00:10.098 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:10.098 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffacc06b7d0 msgr2=0x7ffacc06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffacc06b7d0 0x7ffacc06dc90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffaf41f6880 tx=0x7ffadc009040 comp rx=0 tx=0).stop 2024-01-15T18:00:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40cf7d0 msgr2=0x7ffaf413c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40cf7d0 0x7ffaf413c7d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ffae402b950 tx=0x7ffae40046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 shutdown_connections 2024-01-15T18:00:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffaf4143ae0 0x7ffaf4138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffacc06b7d0 0x7ffacc06dc90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffaf415e3b0 0x7ffaf4159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 --2- 172.21.15.26:0/1944787669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffaf40cf7d0 0x7ffaf413c7d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 >> 172.21.15.26:0/1944787669 conn(0x7ffaf406a800 msgr2=0x7ffaf406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.096+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 shutdown_connections 2024-01-15T18:00:10.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.097+0000 7ffafa600640 1 -- 172.21.15.26:0/1944787669 wait complete. 2024-01-15T18:00:10.121 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:00:10.121 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone6934 2024-01-15T18:00:10.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.339+0000 7fb520a00640 1 Processor -- start 2024-01-15T18:00:10.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.339+0000 7fb520a00640 1 -- start start 2024-01-15T18:00:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.340+0000 7fb520a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb51c06bc00 0x7fb51c06c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:10.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.341+0000 7fb520a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c0cefe0 con 0x7fb51c0cd740 2024-01-15T18:00:10.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.341+0000 7fb520a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb51c06c930 con 0x7fb51c06d2f0 2024-01-15T18:00:10.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.341+0000 7fb520a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb51c06ef00 con 0x7fb51c06bc00 2024-01-15T18:00:10.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.341+0000 7fb51b400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb51c0cd740 0x7fb51c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:39476/0 (socket says 172.21.15.26:39476) 2024-01-15T18:00:10.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.341+0000 7fb51b400640 1 -- 172.21.15.26:0/4081304929 learned_addr learned my addr 172.21.15.26:0/4081304929 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:10.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.341+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461360932 0 0) 0x7fb51c0cefe0 con 0x7fb51c0cd740 2024-01-15T18:00:10.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.341+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4fc003520 con 0x7fb51c0cd740 2024-01-15T18:00:10.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.341+0000 7fb51be00640 1 --2- 172.21.15.26:0/4081304929 >> v2:172.21.15.124:3300/0 conn(0x7fb51c06bc00 0x7fb51c06c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:10.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3659335803 0 0) 0x7fb4fc003520 con 0x7fb51c0cd740 2024-01-15T18:00:10.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 >> v2:172.21.15.124:3300/0 conn(0x7fb51c06bc00 msgr2=0x7fb51c06c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 --2- 172.21.15.26:0/4081304929 >> v2:172.21.15.124:3300/0 conn(0x7fb51c06bc00 0x7fb51c06c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 >> v1:172.21.15.124:6789/0 conn(0x7fb51c06d2f0 legacy=0x7fb51c06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c06f1e0 con 0x7fb51c0cd740 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb50c001670 con 0x7fb51c0cd740 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 >> v1:172.21.15.26:6789/0 conn(0x7fb51c0cd740 legacy=0x7fb51c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4fc0039c0 0x7fb4fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4fc0047c0 0x7fb4fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb4fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c06ef00 con 0x7fb4fc0051d0 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc00c120 con 0x7fb4fc0047c0 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc00c2e0 con 0x7fb4fc0039c0 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.342+0000 7fb51be00640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb4fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.343+0000 7fb51b400640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4fc0039c0 0x7fb4fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:10.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.343+0000 7fb51aa00640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4fc0047c0 0x7fb4fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.343+0000 7fb51be00640 1 -- 172.21.15.26:0/4081304929 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4fc0039c0 msgr2=0x7fb4fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.343+0000 7fb51be00640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4fc0039c0 0x7fb4fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.343+0000 7fb51be00640 1 -- 172.21.15.26:0/4081304929 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4fc0047c0 msgr2=0x7fb4fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.343+0000 7fb51be00640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4fc0047c0 0x7fb4fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.343+0000 7fb51be00640 1 -- 172.21.15.26:0/4081304929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc003520 con 0x7fb4fc0051d0 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.344+0000 7fb51be00640 1 -- 172.21.15.26:0/4081304929 reap_dead start 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.344+0000 7fb51be00640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb4fc00b980 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb514007360 tx=0x7fb4fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cdc90b115e7e3c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.344+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51402ccf0 con 0x7fb4fc0051d0 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.344+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51402ce90 con 0x7fb4fc0051d0 2024-01-15T18:00:10.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.344+0000 7fb51a000640 1 -- 172.21.15.26:0/4081304929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb514003bd0 con 0x7fb4fc0051d0 2024-01-15T18:00:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.345+0000 7fb520a00640 1 -- 172.21.15.26:0/4081304929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 msgr2=0x7fb4fc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.345+0000 7fb520a00640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb4fc00b980 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb514007360 tx=0x7fb4fc0057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.345+0000 7fb520a00640 1 -- 172.21.15.26:0/4081304929 shutdown_connections 2024-01-15T18:00:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.345+0000 7fb520a00640 1 --2- 172.21.15.26:0/4081304929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb4fc00b980 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.345+0000 7fb520a00640 1 -- 172.21.15.26:0/4081304929 >> 172.21.15.26:0/4081304929 conn(0x7fb51c0d0ec0 msgr2=0x7fb51c0d12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.345+0000 7fb520a00640 1 -- 172.21.15.26:0/4081304929 shutdown_connections 2024-01-15T18:00:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.345+0000 7fb520a00640 1 -- 172.21.15.26:0/4081304929 wait complete. 2024-01-15T18:00:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.347+0000 7fb520a00640 1 Processor -- start 2024-01-15T18:00:10.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.347+0000 7fb520a00640 1 -- start start 2024-01-15T18:00:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.348+0000 7fb520a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb51c153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.348+0000 7fb520a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb51c0d12a0 0x7fb51c0d1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:10.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.348+0000 7fb520a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb51c1f62f0 0x7fb51c1f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.348+0000 7fb520a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0d1c60 con 0x7fb4fc0051d0 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.348+0000 7fb520a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c1f6cb0 con 0x7fb51c1f62f0 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.348+0000 7fb520a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c0c4d20 con 0x7fb51c0d12a0 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.348+0000 7fb51b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb51c153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb51c153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54106/0 (socket says 172.21.15.26:54106) 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51b400640 1 -- 172.21.15.26:0/1687832691 learned_addr learned my addr 172.21.15.26:0/1687832691 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51aa00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb51c0d12a0 0x7fb51c0d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51be00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb51c1f62f0 0x7fb51c1f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51b400640 1 -- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb51c0d12a0 msgr2=0x7fb51c0d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51b400640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb51c0d12a0 0x7fb51c0d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51b400640 1 -- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb51c1f62f0 msgr2=0x7fb51c1f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51b400640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb51c1f62f0 0x7fb51c1f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51b400640 1 -- 172.21.15.26:0/1687832691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c1f71b0 con 0x7fb4fc0051d0 2024-01-15T18:00:10.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51be00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb51c1f62f0 0x7fb51c1f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:10.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.349+0000 7fb51b400640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb51c153aa0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00e760 tx=0x7fb50c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:10.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.350+0000 7fb513400640 1 -- 172.21.15.26:0/1687832691 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00c660 con 0x7fb4fc0051d0 2024-01-15T18:00:10.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.350+0000 7fb513400640 1 -- 172.21.15.26:0/1687832691 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c00c7c0 con 0x7fb4fc0051d0 2024-01-15T18:00:10.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.350+0000 7fb513400640 1 -- 172.21.15.26:0/1687832691 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00cb70 con 0x7fb4fc0051d0 2024-01-15T18:00:10.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.350+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c1f6ed0 con 0x7fb4fc0051d0 2024-01-15T18:00:10.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.350+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c0c5fe0 con 0x7fb4fc0051d0 2024-01-15T18:00:10.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.351+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c067b60 con 0x7fb4fc0051d0 2024-01-15T18:00:10.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.352+0000 7fb513400640 1 -- 172.21.15.26:0/1687832691 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7fb50c00ccd0 con 0x7fb4fc0051d0 2024-01-15T18:00:10.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.353+0000 7fb513400640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb4f406b7d0 0x7fb4f406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:10.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.353+0000 7fb513400640 1 -- 172.21.15.26:0/1687832691 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb50c08a390 con 0x7fb4fc0051d0 2024-01-15T18:00:10.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.353+0000 7fb51aa00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb4f406b7d0 0x7fb4f406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:10.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.355+0000 7fb51aa00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb4f406b7d0 0x7fb4f406dc90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb51c15a3e0 tx=0x7fb504009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:10.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.357+0000 7fb513400640 1 -- 172.21.15.26:0/1687832691 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb50c054ba0 con 0x7fb4fc0051d0 2024-01-15T18:00:10.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:10.589+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone6934", "target": ["mon-mgr", ""]}) v1 -- 0x7fb51c0702a0 con 0x7fb4f406b7d0 2024-01-15T18:00:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.591+0000 7fb513400640 1 -- 172.21.15.26:0/1687832691 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fb51c0702a0 con 0x7fb4f406b7d0 2024-01-15T18:00:13.596 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:13.597 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:00:13.597 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:00:13.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:13.597 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb4f406b7d0 msgr2=0x7fb4f406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb4f406b7d0 0x7fb4f406dc90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb51c15a3e0 tx=0x7fb504009040 comp rx=0 tx=0).stop 2024-01-15T18:00:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 msgr2=0x7fb51c153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb51c153aa0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00e760 tx=0x7fb50c00ec20 comp rx=0 tx=0).stop 2024-01-15T18:00:13.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 shutdown_connections 2024-01-15T18:00:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb51c0d12a0 0x7fb51c0d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb4f406b7d0 0x7fb4f406dc90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb51c1f62f0 0x7fb51c1f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 --2- 172.21.15.26:0/1687832691 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4fc0051d0 0x7fb51c153aa0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 >> 172.21.15.26:0/1687832691 conn(0x7fb51c0d0ec0 msgr2=0x7fb51c0ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.596+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 shutdown_connections 2024-01-15T18:00:13.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.597+0000 7fb520a00640 1 -- 172.21.15.26:0/1687832691 wait complete. 2024-01-15T18:00:13.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:00:13.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.847+0000 7fd267800640 1 Processor -- start 2024-01-15T18:00:13.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.847+0000 7fd267800640 1 -- start start 2024-01-15T18:00:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd267800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd260157c30 0x7fd2600cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd267800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26015deb0 con 0x7fd2601618f0 2024-01-15T18:00:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd267800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd26015e190 con 0x7fd260071200 2024-01-15T18:00:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd267800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd26015ea30 con 0x7fd260157c30 2024-01-15T18:00:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd264e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd260157c30 0x7fd2600cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd25fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd2601618f0 0x7fd260070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49188/0 (socket says 172.21.15.26:49188) 2024-01-15T18:00:13.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd25fe00640 1 -- 172.21.15.26:0/3962703036 learned_addr learned my addr 172.21.15.26:0/3962703036 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:13.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd264e00640 1 -- 172.21.15.26:0/3962703036 >> v1:172.21.15.124:6789/0 conn(0x7fd260071200 legacy=0x7fd26006dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:13.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd264e00640 1 -- 172.21.15.26:0/3962703036 >> v1:172.21.15.26:6789/0 conn(0x7fd2601618f0 legacy=0x7fd260070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:13.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.849+0000 7fd264e00640 1 -- 172.21.15.26:0/3962703036 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26015e470 con 0x7fd260157c30 2024-01-15T18:00:13.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.850+0000 7fd264e00640 1 --2- 172.21.15.26:0/3962703036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd260157c30 0x7fd2600cd600 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd2500097a0 tx=0x7fd25002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa8541389b26229 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.851+0000 7fd25f400640 1 -- 172.21.15.26:0/3962703036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250031830 con 0x7fd260157c30 2024-01-15T18:00:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.851+0000 7fd25f400640 1 -- 172.21.15.26:0/3962703036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250031990 con 0x7fd260157c30 2024-01-15T18:00:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.851+0000 7fd25f400640 1 -- 172.21.15.26:0/3962703036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250031d20 con 0x7fd260157c30 2024-01-15T18:00:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.851+0000 7fd267800640 1 -- 172.21.15.26:0/3962703036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd260157c30 msgr2=0x7fd2600cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:13.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.851+0000 7fd267800640 1 --2- 172.21.15.26:0/3962703036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd260157c30 0x7fd2600cd600 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd2500097a0 tx=0x7fd25002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.852+0000 7fd267800640 1 -- 172.21.15.26:0/3962703036 shutdown_connections 2024-01-15T18:00:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.852+0000 7fd267800640 1 --2- 172.21.15.26:0/3962703036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd260157c30 0x7fd2600cd600 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.852+0000 7fd267800640 1 -- 172.21.15.26:0/3962703036 >> 172.21.15.26:0/3962703036 conn(0x7fd2600cf700 msgr2=0x7fd2600cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.852+0000 7fd267800640 1 -- 172.21.15.26:0/3962703036 shutdown_connections 2024-01-15T18:00:13.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.852+0000 7fd267800640 1 -- 172.21.15.26:0/3962703036 wait complete. 2024-01-15T18:00:13.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.854+0000 7fd267800640 1 Processor -- start 2024-01-15T18:00:13.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.854+0000 7fd267800640 1 -- start start 2024-01-15T18:00:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd267800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd260071200 0x7fd2600aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd267800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd260157c30 0x7fd26014a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:13.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd267800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2601618f0 0x7fd260158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd267800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600aea20 con 0x7fd2601618f0 2024-01-15T18:00:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd267800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260158ec0 con 0x7fd260157c30 2024-01-15T18:00:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd267800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600ae460 con 0x7fd260071200 2024-01-15T18:00:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd265800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2601618f0 0x7fd260158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd265800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2601618f0 0x7fd260158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54226/0 (socket says 172.21.15.26:54226) 2024-01-15T18:00:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd265800640 1 -- 172.21.15.26:0/2818401650 learned_addr learned my addr 172.21.15.26:0/2818401650 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd264e00640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd260071200 0x7fd2600aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd25fe00640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd260157c30 0x7fd26014a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:13.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd265800640 1 -- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd260071200 msgr2=0x7fd2600aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd265800640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd260071200 0x7fd2600aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd265800640 1 -- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd260157c30 msgr2=0x7fd26014a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd265800640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd260157c30 0x7fd26014a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd265800640 1 -- 172.21.15.26:0/2818401650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250009400 con 0x7fd2601618f0 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.855+0000 7fd25fe00640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd260157c30 0x7fd26014a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.856+0000 7fd265800640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2601618f0 0x7fd260158980 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd25800b970 tx=0x7fd25800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.856+0000 7fd25cc00640 1 -- 172.21.15.26:0/2818401650 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25800cb40 con 0x7fd2601618f0 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.856+0000 7fd25cc00640 1 -- 172.21.15.26:0/2818401650 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25800cca0 con 0x7fd2601618f0 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.856+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2600ae740 con 0x7fd2601618f0 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.856+0000 7fd25cc00640 1 -- 172.21.15.26:0/2818401650 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258014680 con 0x7fd2601618f0 2024-01-15T18:00:13.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.856+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2600afb80 con 0x7fd2601618f0 2024-01-15T18:00:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.857+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2600afda0 con 0x7fd2601618f0 2024-01-15T18:00:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.858+0000 7fd25cc00640 1 -- 172.21.15.26:0/2818401650 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7fd25801d3b0 con 0x7fd2601618f0 2024-01-15T18:00:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.859+0000 7fd25cc00640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd23806b7d0 0x7fd23806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.859+0000 7fd25cc00640 1 -- 172.21.15.26:0/2818401650 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd25808a190 con 0x7fd2601618f0 2024-01-15T18:00:13.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.860+0000 7fd264e00640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd23806b7d0 0x7fd23806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:13.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.861+0000 7fd264e00640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd23806b7d0 0x7fd23806dc90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd250036000 tx=0x7fd25003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:13.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:13.864+0000 7fd25cc00640 1 -- 172.21.15.26:0/2818401650 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2580549a0 con 0x7fd2601618f0 2024-01-15T18:00:14.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.083+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd260160dc0 con 0x7fd23806b7d0 2024-01-15T18:00:14.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.088+0000 7fd25cc00640 1 -- 172.21.15.26:0/2818401650 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd260160dc0 con 0x7fd23806b7d0 2024-01-15T18:00:14.091 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:00:14.092 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:14.092 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:00:14.092 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:14.092 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:00:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd23806b7d0 msgr2=0x7fd23806dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd23806b7d0 0x7fd23806dc90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd250036000 tx=0x7fd25003c040 comp rx=0 tx=0).stop 2024-01-15T18:00:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2601618f0 msgr2=0x7fd260158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2601618f0 0x7fd260158980 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd25800b970 tx=0x7fd25800be30 comp rx=0 tx=0).stop 2024-01-15T18:00:14.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 shutdown_connections 2024-01-15T18:00:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd23806b7d0 0x7fd23806dc90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2601618f0 0x7fd260158980 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd260157c30 0x7fd26014a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 --2- 172.21.15.26:0/2818401650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd260071200 0x7fd2600aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 >> 172.21.15.26:0/2818401650 conn(0x7fd2600cf700 msgr2=0x7fd26015f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 shutdown_connections 2024-01-15T18:00:14.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.090+0000 7fd267800640 1 -- 172.21.15.26:0/2818401650 wait complete. 2024-01-15T18:00:14.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:00:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.324+0000 7f248e200640 1 Processor -- start 2024-01-15T18:00:14.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.324+0000 7f248e200640 1 -- start start 2024-01-15T18:00:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.324+0000 7f248e200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248813df80 0x7f248815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.324+0000 7f248e200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f248813e360 0x7f24880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.324+0000 7f248e200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248806c470 con 0x7f248815f9c0 2024-01-15T18:00:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.324+0000 7f248e200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248806c750 con 0x7f248813e360 2024-01-15T18:00:14.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.324+0000 7f248e200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248806ca30 con 0x7f248813df80 2024-01-15T18:00:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2486a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f248813e360 0x7f24880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2487e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f248815f9c0 0x7f24881408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49202/0 (socket says 172.21.15.26:49202) 2024-01-15T18:00:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2487e00640 1 -- 172.21.15.26:0/2789627970 learned_addr learned my addr 172.21.15.26:0/2789627970 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2486a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f248813e360 0x7f24880cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54230/0 (socket says 172.21.15.26:54230) 2024-01-15T18:00:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2487400640 1 --2- 172.21.15.26:0/2789627970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248813df80 0x7f248815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2486a00640 1 -- 172.21.15.26:0/2789627970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248813df80 msgr2=0x7f248815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2486a00640 1 --2- 172.21.15.26:0/2789627970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248813df80 0x7f248815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2486000640 1 -- 172.21.15.26:0/2789627970 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3730754114 0 0) 0x7f248806c470 con 0x7f248815f9c0 2024-01-15T18:00:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2486a00640 1 -- 172.21.15.26:0/2789627970 >> v1:172.21.15.26:6789/0 conn(0x7f248815f9c0 legacy=0x7f24881408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2486a00640 1 -- 172.21.15.26:0/2789627970 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248806cd10 con 0x7f248813e360 2024-01-15T18:00:14.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.325+0000 7f2487400640 1 --2- 172.21.15.26:0/2789627970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248813df80 0x7f248815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f2486a00640 1 --2- 172.21.15.26:0/2789627970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813e360 0x7f24880cf290 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f24780097a0 tx=0x7f247802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c3fa6e7bcdadfe server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:14.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f2486000640 1 -- 172.21.15.26:0/2789627970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247802d040 con 0x7f248813e360 2024-01-15T18:00:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f2486000640 1 -- 172.21.15.26:0/2789627970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2478031810 con 0x7f248813e360 2024-01-15T18:00:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f2486000640 1 -- 172.21.15.26:0/2789627970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478031ba0 con 0x7f248813e360 2024-01-15T18:00:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f248e200640 1 -- 172.21.15.26:0/2789627970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813e360 msgr2=0x7f24880cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f248e200640 1 --2- 172.21.15.26:0/2789627970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813e360 0x7f24880cf290 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f24780097a0 tx=0x7f247802b8b0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f248e200640 1 -- 172.21.15.26:0/2789627970 shutdown_connections 2024-01-15T18:00:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f248e200640 1 --2- 172.21.15.26:0/2789627970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813e360 0x7f24880cf290 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f248e200640 1 --2- 172.21.15.26:0/2789627970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248813df80 0x7f248815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f248e200640 1 -- 172.21.15.26:0/2789627970 >> 172.21.15.26:0/2789627970 conn(0x7f248806a800 msgr2=0x7f248806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:14.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f248e200640 1 -- 172.21.15.26:0/2789627970 shutdown_connections 2024-01-15T18:00:14.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.328+0000 7f248e200640 1 -- 172.21.15.26:0/2789627970 wait complete. 2024-01-15T18:00:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.330+0000 7f248e200640 1 Processor -- start 2024-01-15T18:00:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.330+0000 7f248e200640 1 -- start start 2024-01-15T18:00:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f248e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813df80 0x7f24881f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f248e200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f248813e360 0x7f24881f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f248e200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248815f9c0 0x7f24881f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f248e200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24880b26f0 con 0x7f248813df80 2024-01-15T18:00:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f248e200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248814f430 con 0x7f248813e360 2024-01-15T18:00:14.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f248e200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248814eb90 con 0x7f248815f9c0 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2487400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813df80 0x7f24881f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2487400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813df80 0x7f24881f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54246/0 (socket says 172.21.15.26:54246) 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2487400640 1 -- 172.21.15.26:0/3360631192 learned_addr learned my addr 172.21.15.26:0/3360631192 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2486a00640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f248813e360 0x7f24881f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2487400640 1 -- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248815f9c0 msgr2=0x7f24881f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2487400640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248815f9c0 0x7f24881f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2487400640 1 -- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f248813e360 msgr2=0x7f24881f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2487400640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f248813e360 0x7f24881f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2487400640 1 -- 172.21.15.26:0/3360631192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2478009400 con 0x7f248813df80 2024-01-15T18:00:14.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.332+0000 7f2487400640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813df80 0x7f24881f3840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f247000c820 tx=0x7f247000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.333+0000 7f247f400640 1 -- 172.21.15.26:0/3360631192 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470007a30 con 0x7f248813df80 2024-01-15T18:00:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.333+0000 7f247f400640 1 -- 172.21.15.26:0/3360631192 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2470007b90 con 0x7f248813df80 2024-01-15T18:00:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.333+0000 7f247f400640 1 -- 172.21.15.26:0/3360631192 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000ce10 con 0x7f248813df80 2024-01-15T18:00:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.333+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248814ee70 con 0x7f248813df80 2024-01-15T18:00:14.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.333+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881f3d80 con 0x7f248813df80 2024-01-15T18:00:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.334+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24880b24e0 con 0x7f248813df80 2024-01-15T18:00:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.335+0000 7f247f400640 1 -- 172.21.15.26:0/3360631192 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7f24700113f0 con 0x7f248813df80 2024-01-15T18:00:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.336+0000 7f247f400640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f246006b800 0x7f246006dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.336+0000 7f247f400640 1 -- 172.21.15.26:0/3360631192 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f247008a6a0 con 0x7f248813df80 2024-01-15T18:00:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.338+0000 7f2486a00640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f246006b800 0x7f246006dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.338+0000 7f2486a00640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f246006b800 0x7f246006dcc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2478036000 tx=0x7f247803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:14.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.341+0000 7f247f400640 1 -- 172.21.15.26:0/3360631192 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2470054d10 con 0x7f248813df80 2024-01-15T18:00:14.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.562+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24880ce850 con 0x7f246006b800 2024-01-15T18:00:14.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.579+0000 7f247f400640 1 -- 172.21.15.26:0/3360631192 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+230 (secure 0 0 0) 0x7f247803a420 con 0x7f246006b800 2024-01-15T18:00:14.583 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:14.583 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sv1" 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone6934" 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone6933" 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:14.584 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:14.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone6931" 2024-01-15T18:00:14.585 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:14.585 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:14.585 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone6932" 2024-01-15T18:00:14.585 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:14.585 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:00:14.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f246006b800 msgr2=0x7f246006dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f246006b800 0x7f246006dcc0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2478036000 tx=0x7f247803c040 comp rx=0 tx=0).stop 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813df80 msgr2=0x7f24881f3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813df80 0x7f24881f3840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f247000c820 tx=0x7f247000cce0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 shutdown_connections 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f248815f9c0 0x7f24881f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f246006b800 0x7f246006dcc0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f248813e360 0x7f24881f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 --2- 172.21.15.26:0/3360631192 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f248813df80 0x7f24881f3840 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.581+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 >> 172.21.15.26:0/3360631192 conn(0x7f248806a800 msgr2=0x7f248815ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.582+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 shutdown_connections 2024-01-15T18:00:14.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.582+0000 7f248e200640 1 -- 172.21.15.26:0/3360631192 wait complete. 2024-01-15T18:00:14.606 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:00:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.823+0000 7fc62be00640 1 Processor -- start 2024-01-15T18:00:14.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.823+0000 7fc62be00640 1 -- start start 2024-01-15T18:00:14.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.825+0000 7fc62be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc62c0cc990 0x7fc62c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.825+0000 7fc62be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc62c070e50 con 0x7fc62c0cc500 2024-01-15T18:00:14.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.825+0000 7fc62be00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc62c144020 con 0x7fc62c143c40 2024-01-15T18:00:14.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.825+0000 7fc62be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc62c0723c0 con 0x7fc62c0cc990 2024-01-15T18:00:14.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.826+0000 7fc62a000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc62c0cc500 0x7fc62c06d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49210/0 (socket says 172.21.15.26:49210) 2024-01-15T18:00:14.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.826+0000 7fc62a000640 1 -- 172.21.15.26:0/4074282593 learned_addr learned my addr 172.21.15.26:0/4074282593 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:14.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.826+0000 7fc62b400640 1 --2- 172.21.15.26:0/4074282593 >> v2:172.21.15.26:3300/0 conn(0x7fc62c0cc990 0x7fc62c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.826+0000 7fc62b400640 1 -- 172.21.15.26:0/4074282593 >> v1:172.21.15.124:6789/0 conn(0x7fc62c143c40 legacy=0x7fc62c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.826+0000 7fc62b400640 1 -- 172.21.15.26:0/4074282593 >> v1:172.21.15.26:6789/0 conn(0x7fc62c0cc500 legacy=0x7fc62c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.826+0000 7fc62b400640 1 -- 172.21.15.26:0/4074282593 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c071e00 con 0x7fc62c0cc990 2024-01-15T18:00:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.828+0000 7fc62b400640 1 --2- 172.21.15.26:0/4074282593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 0x7fc62c06d870 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0097a0 tx=0x7fc61c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2b46cbffbe3f15 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.828+0000 7fc629600640 1 -- 172.21.15.26:0/4074282593 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c031830 con 0x7fc62c0cc990 2024-01-15T18:00:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.828+0000 7fc629600640 1 -- 172.21.15.26:0/4074282593 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc61c031990 con 0x7fc62c0cc990 2024-01-15T18:00:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.828+0000 7fc629600640 1 -- 172.21.15.26:0/4074282593 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c031d20 con 0x7fc62c0cc990 2024-01-15T18:00:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.829+0000 7fc62be00640 1 -- 172.21.15.26:0/4074282593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 msgr2=0x7fc62c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.829+0000 7fc62be00640 1 --2- 172.21.15.26:0/4074282593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 0x7fc62c06d870 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0097a0 tx=0x7fc61c02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.829+0000 7fc62be00640 1 -- 172.21.15.26:0/4074282593 shutdown_connections 2024-01-15T18:00:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.829+0000 7fc62be00640 1 --2- 172.21.15.26:0/4074282593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 0x7fc62c06d870 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.829+0000 7fc62be00640 1 -- 172.21.15.26:0/4074282593 >> 172.21.15.26:0/4074282593 conn(0x7fc62c06a800 msgr2=0x7fc62c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.829+0000 7fc62be00640 1 -- 172.21.15.26:0/4074282593 shutdown_connections 2024-01-15T18:00:14.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.829+0000 7fc62be00640 1 -- 172.21.15.26:0/4074282593 wait complete. 2024-01-15T18:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.831+0000 7fc62be00640 1 Processor -- start 2024-01-15T18:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.831+0000 7fc62be00640 1 -- start start 2024-01-15T18:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc62c0cc500 0x7fc62c0c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 0x7fc62c0c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc62c143c40 0x7fc62c0c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0c4350 con 0x7fc62c0cc990 2024-01-15T18:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0c4630 con 0x7fc62c143c40 2024-01-15T18:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c0c4910 con 0x7fc62c0cc500 2024-01-15T18:00:14.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 0x7fc62c0c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 0x7fc62c0c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54266/0 (socket says 172.21.15.26:54266) 2024-01-15T18:00:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62a000640 1 -- 172.21.15.26:0/2331570002 learned_addr learned my addr 172.21.15.26:0/2331570002 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62aa00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc62c0cc500 0x7fc62c0c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62b400640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc62c143c40 0x7fc62c0c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62a000640 1 -- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc62c0cc500 msgr2=0x7fc62c0c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62a000640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc62c0cc500 0x7fc62c0c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62a000640 1 -- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc62c143c40 msgr2=0x7fc62c0c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62a000640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc62c143c40 0x7fc62c0c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:14.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62a000640 1 -- 172.21.15.26:0/2331570002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c009400 con 0x7fc62c0cc990 2024-01-15T18:00:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62a000640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 0x7fc62c0c63e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc61400cba0 tx=0x7fc61400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc622a00640 1 -- 172.21.15.26:0/2331570002 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61400ad40 con 0x7fc62c0cc990 2024-01-15T18:00:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc622a00640 1 -- 172.21.15.26:0/2331570002 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc614012400 con 0x7fc62c0cc990 2024-01-15T18:00:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc622a00640 1 -- 172.21.15.26:0/2331570002 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61400aea0 con 0x7fc62c0cc990 2024-01-15T18:00:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62c0c4bf0 con 0x7fc62c0cc990 2024-01-15T18:00:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.833+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c139cb0 con 0x7fc62c0cc990 2024-01-15T18:00:14.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.834+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c0c5110 con 0x7fc62c0cc990 2024-01-15T18:00:14.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.836+0000 7fc622a00640 1 -- 172.21.15.26:0/2331570002 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7fc614007070 con 0x7fc62c0cc990 2024-01-15T18:00:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.836+0000 7fc622a00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc60406b7d0 0x7fc60406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.837+0000 7fc622a00640 1 -- 172.21.15.26:0/2331570002 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc61408a330 con 0x7fc62c0cc990 2024-01-15T18:00:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.837+0000 7fc62aa00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc60406b7d0 0x7fc60406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:14.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.838+0000 7fc62aa00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc60406b7d0 0x7fc60406dc90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc624000f60 tx=0x7fc624009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:14.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:14.841+0000 7fc622a00640 1 -- 172.21.15.26:0/2331570002 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc62c0c5110 con 0x7fc62c0cc990 2024-01-15T18:00:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:15.071+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc62c06cc40 con 0x7fc60406b7d0 2024-01-15T18:00:18.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.596+0000 7fc622a00640 1 -- 172.21.15.26:0/2331570002 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc624001240 con 0x7fc60406b7d0 2024-01-15T18:00:18.601 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:18.601 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:00:18.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:18.601 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1" 2024-01-15T18:00:18.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:18.602 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:00:18.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc60406b7d0 msgr2=0x7fc60406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:18.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc60406b7d0 0x7fc60406dc90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc624000f60 tx=0x7fc624009040 comp rx=0 tx=0).stop 2024-01-15T18:00:18.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 msgr2=0x7fc62c0c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:18.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 0x7fc62c0c63e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc61400cba0 tx=0x7fc61400a430 comp rx=0 tx=0).stop 2024-01-15T18:00:18.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 shutdown_connections 2024-01-15T18:00:18.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc60406b7d0 0x7fc60406dc90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:18.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc62c143c40 0x7fc62c0c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:18.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc62c0cc990 0x7fc62c0c63e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:18.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 --2- 172.21.15.26:0/2331570002 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc62c0cc500 0x7fc62c0c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:18.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.600+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 >> 172.21.15.26:0/2331570002 conn(0x7fc62c06a800 msgr2=0x7fc62c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:18.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.601+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 shutdown_connections 2024-01-15T18:00:18.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.601+0000 7fc62be00640 1 -- 172.21.15.26:0/2331570002 wait complete. 2024-01-15T18:00:18.630 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:00:18.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.861+0000 7fe520c00640 1 Processor -- start 2024-01-15T18:00:18.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.861+0000 7fe520c00640 1 -- start start 2024-01-15T18:00:18.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.863+0000 7fe520c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe51c0b3f30 0x7fe51c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.863+0000 7fe520c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51c0d0a10 con 0x7fe51c160920 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.863+0000 7fe520c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe51c0cf080 con 0x7fe51c0d0100 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.863+0000 7fe520c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe51c0c6be0 con 0x7fe51c0b3f30 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.863+0000 7fe51be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe51c0b3f30 0x7fe51c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.863+0000 7fe51be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe51c0b3f30 0x7fe51c0d1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54276/0 (socket says 172.21.15.26:54276) 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.863+0000 7fe51be00640 1 -- 172.21.15.26:0/3590482219 learned_addr learned my addr 172.21.15.26:0/3590482219 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.864+0000 7fe51be00640 1 -- 172.21.15.26:0/3590482219 >> v1:172.21.15.124:6789/0 conn(0x7fe51c0d0100 legacy=0x7fe51c0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.864+0000 7fe51be00640 1 -- 172.21.15.26:0/3590482219 >> v1:172.21.15.26:6789/0 conn(0x7fe51c160920 legacy=0x7fe51c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.864+0000 7fe51be00640 1 -- 172.21.15.26:0/3590482219 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c0ce7e0 con 0x7fe51c0b3f30 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.864+0000 7fe51a000640 1 -- 172.21.15.26:0/3590482219 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704998577 0 0) 0x7fe51c0d0a10 con 0x7fe51c160920 2024-01-15T18:00:18.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.865+0000 7fe51be00640 1 --2- 172.21.15.26:0/3590482219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 0x7fe51c0d1650 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe514009e50 tx=0x7fe51402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f25431d721ed6cf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.865+0000 7fe51a000640 1 -- 172.21.15.26:0/3590482219 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51402cb00 con 0x7fe51c0b3f30 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.865+0000 7fe51a000640 1 -- 172.21.15.26:0/3590482219 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51402cc60 con 0x7fe51c0b3f30 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.865+0000 7fe51a000640 1 -- 172.21.15.26:0/3590482219 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5140343f0 con 0x7fe51c0b3f30 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.866+0000 7fe520c00640 1 -- 172.21.15.26:0/3590482219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 msgr2=0x7fe51c0d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.866+0000 7fe520c00640 1 --2- 172.21.15.26:0/3590482219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 0x7fe51c0d1650 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe514009e50 tx=0x7fe51402bf60 comp rx=0 tx=0).stop 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.866+0000 7fe520c00640 1 -- 172.21.15.26:0/3590482219 shutdown_connections 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.866+0000 7fe520c00640 1 --2- 172.21.15.26:0/3590482219 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 0x7fe51c0d1650 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.866+0000 7fe520c00640 1 -- 172.21.15.26:0/3590482219 >> 172.21.15.26:0/3590482219 conn(0x7fe51c0d3090 msgr2=0x7fe51c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.866+0000 7fe520c00640 1 -- 172.21.15.26:0/3590482219 shutdown_connections 2024-01-15T18:00:18.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.866+0000 7fe520c00640 1 -- 172.21.15.26:0/3590482219 wait complete. 2024-01-15T18:00:18.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.868+0000 7fe520c00640 1 Processor -- start 2024-01-15T18:00:18.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.868+0000 7fe520c00640 1 -- start start 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.869+0000 7fe520c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 0x7fe51c0b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.869+0000 7fe520c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe51c0d0100 0x7fe51c14e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.869+0000 7fe520c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe51c160920 0x7fe51c0c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.869+0000 7fe520c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0b6cb0 con 0x7fe51c0b3f30 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.869+0000 7fe520c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c0b5c50 con 0x7fe51c0d0100 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.869+0000 7fe520c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c14ebb0 con 0x7fe51c160920 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.869+0000 7fe51b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 0x7fe51c0b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.869+0000 7fe51b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 0x7fe51c0b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54280/0 (socket says 172.21.15.26:54280) 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.869+0000 7fe51b400640 1 -- 172.21.15.26:0/420570616 learned_addr learned my addr 172.21.15.26:0/420570616 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51be00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe51c160920 0x7fe51c0c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:18.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51b400640 1 -- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe51c160920 msgr2=0x7fe51c0c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51b400640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe51c160920 0x7fe51c0c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51b400640 1 -- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe51c0d0100 msgr2=0x7fe51c14e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51aa00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe51c0d0100 0x7fe51c14e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51b400640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe51c0d0100 0x7fe51c14e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51b400640 1 -- 172.21.15.26:0/420570616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe514009b30 con 0x7fe51c0b3f30 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51be00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe51c160920 0x7fe51c0c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51aa00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe51c0d0100 0x7fe51c14e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe51b400640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 0x7fe51c0b7c40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe50400c410 tx=0x7fe50400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe513400640 1 -- 172.21.15.26:0/420570616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5040035f0 con 0x7fe51c0b3f30 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe513400640 1 -- 172.21.15.26:0/420570616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe504003770 con 0x7fe51c0b3f30 2024-01-15T18:00:18.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe513400640 1 -- 172.21.15.26:0/420570616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504003b20 con 0x7fe51c0b3f30 2024-01-15T18:00:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.870+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c0c7c70 con 0x7fe51c0b3f30 2024-01-15T18:00:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.871+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c0b61d0 con 0x7fe51c0b3f30 2024-01-15T18:00:18.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.872+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c067450 con 0x7fe51c0b3f30 2024-01-15T18:00:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.873+0000 7fe513400640 1 -- 172.21.15.26:0/420570616 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7fe504003c80 con 0x7fe51c0b3f30 2024-01-15T18:00:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.874+0000 7fe513400640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4f406b7d0 0x7fe4f406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.874+0000 7fe513400640 1 -- 172.21.15.26:0/420570616 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe504089a70 con 0x7fe51c0b3f30 2024-01-15T18:00:18.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.875+0000 7fe51aa00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4f406b7d0 0x7fe4f406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:18.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.876+0000 7fe51aa00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4f406b7d0 0x7fe4f406dc90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe50c003eb0 tx=0x7fe50c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:18.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:18.879+0000 7fe513400640 1 -- 172.21.15.26:0/420570616 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe504054230 con 0x7fe51c0b3f30 2024-01-15T18:00:19.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.114+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe51c160d00 con 0x7fe4f406b7d0 2024-01-15T18:00:19.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.180+0000 7fe513400640 1 -- 172.21.15.26:0/420570616 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe50c005030 con 0x7fe4f406b7d0 2024-01-15T18:00:19.183 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:19.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4f406b7d0 msgr2=0x7fe4f406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4f406b7d0 0x7fe4f406dc90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe50c003eb0 tx=0x7fe50c009040 comp rx=0 tx=0).stop 2024-01-15T18:00:19.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 msgr2=0x7fe51c0b7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 0x7fe51c0b7c40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe50400c410 tx=0x7fe50400c8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 shutdown_connections 2024-01-15T18:00:19.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe51c160920 0x7fe51c0c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4f406b7d0 0x7fe4f406dc90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe51c0d0100 0x7fe51c14e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 --2- 172.21.15.26:0/420570616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe51c0b3f30 0x7fe51c0b7c40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 >> 172.21.15.26:0/420570616 conn(0x7fe51c0d3090 msgr2=0x7fe51c15fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:19.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.183+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 shutdown_connections 2024-01-15T18:00:19.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.184+0000 7fe520c00640 1 -- 172.21.15.26:0/420570616 wait complete. 2024-01-15T18:00:19.208 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:00:19.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.433+0000 7f09a9200640 1 Processor -- start 2024-01-15T18:00:19.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.434+0000 7f09a9200640 1 -- start start 2024-01-15T18:00:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.435+0000 7f09a9200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f09a406c990 0x7f09a406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.435+0000 7f09a9200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a415eb60 con 0x7f09a406a6c0 2024-01-15T18:00:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.435+0000 7f09a9200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a415dfe0 con 0x7f09a415acc0 2024-01-15T18:00:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.435+0000 7f09a9200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a406d350 con 0x7f09a406c990 2024-01-15T18:00:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.436+0000 7f09a2800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f09a406a6c0 0x7f09a406ab50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49218/0 (socket says 172.21.15.26:49218) 2024-01-15T18:00:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.436+0000 7f09a2800640 1 -- 172.21.15.26:0/4062694395 learned_addr learned my addr 172.21.15.26:0/4062694395 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:19.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.436+0000 7f09a1400640 1 --2- 172.21.15.26:0/4062694395 >> v2:172.21.15.124:3300/0 conn(0x7f09a406c990 0x7f09a406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.436+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082951712 0 0) 0x7f09a415eb60 con 0x7f09a406a6c0 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.436+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0984003520 con 0x7f09a406a6c0 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2867519204 0 0) 0x7f0984003520 con 0x7f09a406a6c0 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 >> v2:172.21.15.124:3300/0 conn(0x7f09a406c990 msgr2=0x7f09a406ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 --2- 172.21.15.26:0/4062694395 >> v2:172.21.15.124:3300/0 conn(0x7f09a406c990 0x7f09a406ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 >> v1:172.21.15.124:6789/0 conn(0x7f09a415acc0 legacy=0x7f09a415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a415e2c0 con 0x7f09a406a6c0 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f099c0019f0 con 0x7f09a406a6c0 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 >> v1:172.21.15.26:6789/0 conn(0x7f09a406a6c0 legacy=0x7f09a406ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09840039c0 0x7f0984003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f0984004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0984005180 0x7f098400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a406d350 con 0x7f09840047c0 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098400e290 con 0x7f09840039c0 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098400e450 con 0x7f0984005180 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a1400640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f0984004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a1400640 1 -- 172.21.15.26:0/4062694395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0984005180 msgr2=0x7f098400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a1400640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0984005180 0x7f098400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a1400640 1 -- 172.21.15.26:0/4062694395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09840039c0 msgr2=0x7f0984003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:19.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.437+0000 7f09a1e00640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09840039c0 0x7f0984003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.438+0000 7f09a1400640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09840039c0 0x7f0984003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.438+0000 7f09a1400640 1 -- 172.21.15.26:0/4062694395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0984003520 con 0x7f09840047c0 2024-01-15T18:00:19.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.438+0000 7f09a2800640 1 -- 172.21.15.26:0/4062694395 reap_dead start 2024-01-15T18:00:19.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.439+0000 7f09a1400640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f0984004c40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f098c009ac0 tx=0x7f09840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdc6b24828e08ad server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.439+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c02c6b0 con 0x7f09840047c0 2024-01-15T18:00:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.439+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c02c810 con 0x7f09840047c0 2024-01-15T18:00:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.439+0000 7f09a0a00640 1 -- 172.21.15.26:0/4062694395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c02cba0 con 0x7f09840047c0 2024-01-15T18:00:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.440+0000 7f09a9200640 1 -- 172.21.15.26:0/4062694395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 msgr2=0x7f0984004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.440+0000 7f09a9200640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f0984004c40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f098c009ac0 tx=0x7f09840057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.440+0000 7f09a9200640 1 -- 172.21.15.26:0/4062694395 shutdown_connections 2024-01-15T18:00:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.440+0000 7f09a9200640 1 --2- 172.21.15.26:0/4062694395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f0984004c40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.440+0000 7f09a9200640 1 -- 172.21.15.26:0/4062694395 >> 172.21.15.26:0/4062694395 conn(0x7f09a40cf700 msgr2=0x7f09a40cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.440+0000 7f09a9200640 1 -- 172.21.15.26:0/4062694395 shutdown_connections 2024-01-15T18:00:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.440+0000 7f09a9200640 1 -- 172.21.15.26:0/4062694395 wait complete. 2024-01-15T18:00:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.442+0000 7f09a9200640 1 Processor -- start 2024-01-15T18:00:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.442+0000 7f09a9200640 1 -- start start 2024-01-15T18:00:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f09a40b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a9200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a4156e90 0x7f09a413e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a9200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09a41573c0 0x7f09a40ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a9200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40b1220 con 0x7f09840047c0 2024-01-15T18:00:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a9200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40afc40 con 0x7f09a41573c0 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a9200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40aff20 con 0x7f09a4156e90 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f09a40b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f09a40b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54310/0 (socket says 172.21.15.26:54310) 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a1e00640 1 -- 172.21.15.26:0/2629047052 learned_addr learned my addr 172.21.15.26:0/2629047052 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a1e00640 1 -- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a4156e90 msgr2=0x7f09a413e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a1e00640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a4156e90 0x7f09a413e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a1e00640 1 -- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09a41573c0 msgr2=0x7f09a40ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a1e00640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09a41573c0 0x7f09a40ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.443+0000 7f09a1e00640 1 -- 172.21.15.26:0/2629047052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40b0200 con 0x7f09840047c0 2024-01-15T18:00:19.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.444+0000 7f09a1e00640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f09a40b0ce0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f099400d480 tx=0x7f099400d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.444+0000 7f099a000640 1 -- 172.21.15.26:0/2629047052 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994005480 con 0x7f09840047c0 2024-01-15T18:00:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.444+0000 7f099a000640 1 -- 172.21.15.26:0/2629047052 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0994003610 con 0x7f09840047c0 2024-01-15T18:00:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.444+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a40b04e0 con 0x7f09840047c0 2024-01-15T18:00:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.444+0000 7f099a000640 1 -- 172.21.15.26:0/2629047052 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940039c0 con 0x7f09840047c0 2024-01-15T18:00:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.444+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a40af8d0 con 0x7f09840047c0 2024-01-15T18:00:19.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.445+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a40675d0 con 0x7f09840047c0 2024-01-15T18:00:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.446+0000 7f099a000640 1 -- 172.21.15.26:0/2629047052 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7f0994003b20 con 0x7f09840047c0 2024-01-15T18:00:19.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.447+0000 7f099a000640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f097c06b7d0 0x7f097c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.447+0000 7f099a000640 1 -- 172.21.15.26:0/2629047052 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f099408a3c0 con 0x7f09840047c0 2024-01-15T18:00:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.448+0000 7f09a1400640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f097c06b7d0 0x7f097c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.449+0000 7f09a1400640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f097c06b7d0 0x7f097c06dc90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f098c036000 tx=0x7f098c02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:19.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.451+0000 7f099a000640 1 -- 172.21.15.26:0/2629047052 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0994054b80 con 0x7f09840047c0 2024-01-15T18:00:19.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.672+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f09a415e880 con 0x7f097c06b7d0 2024-01-15T18:00:19.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.720+0000 7f099a000640 1 -- 172.21.15.26:0/2629047052 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f098c004210 con 0x7f097c06b7d0 2024-01-15T18:00:19.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.722+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f097c06b7d0 msgr2=0x7f097c06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.722+0000 7f09a9200640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f097c06b7d0 0x7f097c06dc90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f098c036000 tx=0x7f098c02e000 comp rx=0 tx=0).stop 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 msgr2=0x7f09a40b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f09a40b0ce0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f099400d480 tx=0x7f099400d940 comp rx=0 tx=0).stop 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 shutdown_connections 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f09a4156e90 0x7f09a413e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f097c06b7d0 0x7f097c06dc90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09a41573c0 0x7f09a40ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 --2- 172.21.15.26:0/2629047052 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09840047c0 0x7f09a40b0ce0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 >> 172.21.15.26:0/2629047052 conn(0x7f09a40cf700 msgr2=0x7f09a415b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 shutdown_connections 2024-01-15T18:00:19.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.723+0000 7f09a9200640 1 -- 172.21.15.26:0/2629047052 wait complete. 2024-01-15T18:00:19.732 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.729+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.732 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.729+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.732 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.729+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.732 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.729+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.732 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.729+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.749 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6934 2024-01-15T18:00:19.762 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.759+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.762 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.759+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.762 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.759+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.762 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.759+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.762 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:00:19.759+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:00:19.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.954+0000 7fe753800640 1 Processor -- start 2024-01-15T18:00:19.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.955+0000 7fe753800640 1 -- start start 2024-01-15T18:00:19.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.956+0000 7fe753800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe74c15e3b0 0x7fe74c164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.956+0000 7fe753800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74c0b1ee0 con 0x7fe74c143ae0 2024-01-15T18:00:19.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.956+0000 7fe753800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74c0b2780 con 0x7fe74c0cf9a0 2024-01-15T18:00:19.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.956+0000 7fe753800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74c15e990 con 0x7fe74c15e3b0 2024-01-15T18:00:19.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.957+0000 7fe751800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe74c15e3b0 0x7fe74c164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.957+0000 7fe750e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe74c143ae0 0x7fe74c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49232/0 (socket says 172.21.15.26:49232) 2024-01-15T18:00:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.957+0000 7fe750e00640 1 -- 172.21.15.26:0/4239940952 learned_addr learned my addr 172.21.15.26:0/4239940952 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:19.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.957+0000 7fe751800640 1 -- 172.21.15.26:0/4239940952 >> v1:172.21.15.124:6789/0 conn(0x7fe74c0cf9a0 legacy=0x7fe74c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.957+0000 7fe751800640 1 -- 172.21.15.26:0/4239940952 >> v1:172.21.15.26:6789/0 conn(0x7fe74c143ae0 legacy=0x7fe74c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.957+0000 7fe751800640 1 -- 172.21.15.26:0/4239940952 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c0b2130 con 0x7fe74c15e3b0 2024-01-15T18:00:19.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.958+0000 7fe751800640 1 --2- 172.21.15.26:0/4239940952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c15e3b0 0x7fe74c164de0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe744009d90 tx=0x7fe74402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9f91b32bb00f03 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.959+0000 7fe74be00640 1 -- 172.21.15.26:0/4239940952 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74402ee70 con 0x7fe74c15e3b0 2024-01-15T18:00:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.959+0000 7fe74be00640 1 -- 172.21.15.26:0/4239940952 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe744036440 con 0x7fe74c15e3b0 2024-01-15T18:00:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.959+0000 7fe74be00640 1 -- 172.21.15.26:0/4239940952 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7440367d0 con 0x7fe74c15e3b0 2024-01-15T18:00:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.959+0000 7fe753800640 1 -- 172.21.15.26:0/4239940952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c15e3b0 msgr2=0x7fe74c164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.959+0000 7fe753800640 1 --2- 172.21.15.26:0/4239940952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c15e3b0 0x7fe74c164de0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe744009d90 tx=0x7fe74402e0c0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.960+0000 7fe753800640 1 -- 172.21.15.26:0/4239940952 shutdown_connections 2024-01-15T18:00:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.960+0000 7fe753800640 1 --2- 172.21.15.26:0/4239940952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c15e3b0 0x7fe74c164de0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.960+0000 7fe753800640 1 -- 172.21.15.26:0/4239940952 >> 172.21.15.26:0/4239940952 conn(0x7fe74c06a800 msgr2=0x7fe74c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.960+0000 7fe753800640 1 -- 172.21.15.26:0/4239940952 shutdown_connections 2024-01-15T18:00:19.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.960+0000 7fe753800640 1 -- 172.21.15.26:0/4239940952 wait complete. 2024-01-15T18:00:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.961+0000 7fe753800640 1 Processor -- start 2024-01-15T18:00:19.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.962+0000 7fe753800640 1 -- start start 2024-01-15T18:00:19.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.962+0000 7fe753800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe74c0cf9a0 0x7fe74c0ada10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.962+0000 7fe753800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c143ae0 0x7fe74c15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.962+0000 7fe753800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe74c15e3b0 0x7fe74c15bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.962+0000 7fe753800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c133ef0 con 0x7fe74c143ae0 2024-01-15T18:00:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe753800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c133550 con 0x7fe74c0cf9a0 2024-01-15T18:00:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe753800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c133830 con 0x7fe74c15e3b0 2024-01-15T18:00:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe743e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c143ae0 0x7fe74c15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe743e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c143ae0 0x7fe74c15b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54328/0 (socket says 172.21.15.26:54328) 2024-01-15T18:00:19.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe743e00640 1 -- 172.21.15.26:0/3939856514 learned_addr learned my addr 172.21.15.26:0/3939856514 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe751800640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe74c15e3b0 0x7fe74c15bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe750e00640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe74c0cf9a0 0x7fe74c0ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe743e00640 1 -- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe74c15e3b0 msgr2=0x7fe74c15bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe743e00640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe74c15e3b0 0x7fe74c15bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe743e00640 1 -- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe74c0cf9a0 msgr2=0x7fe74c0ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe743e00640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe74c0cf9a0 0x7fe74c0ada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe743e00640 1 -- 172.21.15.26:0/3939856514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe744009a70 con 0x7fe74c143ae0 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.963+0000 7fe751800640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe74c15e3b0 0x7fe74c15bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.964+0000 7fe743e00640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c143ae0 0x7fe74c15b7e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe73400c430 tx=0x7fe73400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:19.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.964+0000 7fe749600640 1 -- 172.21.15.26:0/3939856514 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734011790 con 0x7fe74c143ae0 2024-01-15T18:00:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.964+0000 7fe749600640 1 -- 172.21.15.26:0/3939856514 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7340118f0 con 0x7fe74c143ae0 2024-01-15T18:00:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.964+0000 7fe749600640 1 -- 172.21.15.26:0/3939856514 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe734011ce0 con 0x7fe74c143ae0 2024-01-15T18:00:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.964+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c133b10 con 0x7fe74c143ae0 2024-01-15T18:00:19.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.964+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74c0adf50 con 0x7fe74c143ae0 2024-01-15T18:00:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.965+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74c06c3f0 con 0x7fe74c143ae0 2024-01-15T18:00:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.966+0000 7fe749600640 1 -- 172.21.15.26:0/3939856514 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7fe73401c460 con 0x7fe74c143ae0 2024-01-15T18:00:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.968+0000 7fe749600640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72406b7d0 0x7fe72406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:19.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.968+0000 7fe749600640 1 -- 172.21.15.26:0/3939856514 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe73408a330 con 0x7fe74c143ae0 2024-01-15T18:00:19.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.968+0000 7fe750e00640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72406b7d0 0x7fe72406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:19.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.970+0000 7fe750e00640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72406b7d0 0x7fe72406dc90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe73c005880 tx=0x7fe73c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:19.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:19.973+0000 7fe749600640 1 -- 172.21.15.26:0/3939856514 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe734054b40 con 0x7fe74c143ae0 2024-01-15T18:00:20.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.193+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6934", "target": ["mon-mgr", ""]}) v1 -- 0x7fe74c0675d0 con 0x7fe72406b7d0 2024-01-15T18:00:20.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.229+0000 7fe749600640 1 -- 172.21.15.26:0/3939856514 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe74c0675d0 con 0x7fe72406b7d0 2024-01-15T18:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72406b7d0 msgr2=0x7fe72406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72406b7d0 0x7fe72406dc90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe73c005880 tx=0x7fe73c009040 comp rx=0 tx=0).stop 2024-01-15T18:00:20.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c143ae0 msgr2=0x7fe74c15b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c143ae0 0x7fe74c15b7e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe73400c430 tx=0x7fe73400c8f0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 shutdown_connections 2024-01-15T18:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe74c15e3b0 0x7fe74c15bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72406b7d0 0x7fe72406dc90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe74c143ae0 0x7fe74c15b7e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 --2- 172.21.15.26:0/3939856514 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe74c0cf9a0 0x7fe74c0ada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 >> 172.21.15.26:0/3939856514 conn(0x7fe74c06a800 msgr2=0x7fe74c15b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.231+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 shutdown_connections 2024-01-15T18:00:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.232+0000 7fe753800640 1 -- 172.21.15.26:0/3939856514 wait complete. 2024-01-15T18:00:20.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6933 2024-01-15T18:00:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.437+0000 7f89c2e00640 1 Processor -- start 2024-01-15T18:00:20.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.437+0000 7f89c2e00640 1 -- start start 2024-01-15T18:00:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.439+0000 7f89c2e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89bc143c40 0x7f89bc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.439+0000 7f89c2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89bc0717d0 0x7f89bc0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.439+0000 7f89c2e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89bc070e50 con 0x7f89bc0cca40 2024-01-15T18:00:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.439+0000 7f89c2e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89bc0723c0 con 0x7f89bc143c40 2024-01-15T18:00:20.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.439+0000 7f89c2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc0cce70 con 0x7f89bc0717d0 2024-01-15T18:00:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.439+0000 7f89c2400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f89bc0cca40 0x7f89bc15eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49234/0 (socket says 172.21.15.26:49234) 2024-01-15T18:00:20.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.439+0000 7f89c2400640 1 -- 172.21.15.26:0/3957476576 learned_addr learned my addr 172.21.15.26:0/3957476576 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:20.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89c1a00640 1 --2- 172.21.15.26:0/3957476576 >> v2:172.21.15.124:3300/0 conn(0x7f89bc143c40 0x7f89bc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89c1000640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89bc0717d0 0x7f89bc0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268019458 0 0) 0x7f89bc070e50 con 0x7f89bc0cca40 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f899c003520 con 0x7f89bc0cca40 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1396611016 0 0) 0x7f899c003520 con 0x7f89bc0cca40 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89bc0717d0 msgr2=0x7f89bc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89bc0717d0 0x7f89bc0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 >> v2:172.21.15.124:3300/0 conn(0x7f89bc143c40 msgr2=0x7f89bc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 --2- 172.21.15.26:0/3957476576 >> v2:172.21.15.124:3300/0 conn(0x7f89bc143c40 0x7f89bc071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc06e170 con 0x7f89bc0cca40 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f89ac001670 con 0x7f89bc0cca40 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89c1a00640 1 --2- 172.21.15.26:0/3957476576 >> v2:172.21.15.124:3300/0 conn(0x7f89bc143c40 0x7f89bc071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 >> v1:172.21.15.26:6789/0 conn(0x7f89bc0cca40 legacy=0x7f89bc15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c003970 0x7f899c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f899c0047c0 0x7f899c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89bc0717d0 0x7f899c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc0723c0 con 0x7f899c0047c0 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc0cce70 con 0x7f899c003970 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c00bb10 con 0x7f89bc0717d0 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.440+0000 7f89c1000640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f899c0047c0 0x7f899c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.441+0000 7f89c1a00640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c003970 0x7f899c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.441+0000 7f89c1000640 1 -- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89bc0717d0 msgr2=0x7f899c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.441+0000 7f89c1000640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89bc0717d0 0x7f899c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.441+0000 7f89c1000640 1 -- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c003970 msgr2=0x7f899c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.441+0000 7f89c1000640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f899c003970 0x7f899c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.441+0000 7f89c1000640 1 -- 172.21.15.26:0/3957476576 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c003520 con 0x7f899c0047c0 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.442+0000 7f89c1000640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f899c0047c0 0x7f899c004c40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f89a400ba00 tx=0x7f899c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0b4f75f8efc12c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.442+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4032630 con 0x7f899c0047c0 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.442+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4032790 con 0x7f899c0047c0 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.442+0000 7f89bbe00640 1 -- 172.21.15.26:0/3957476576 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4032b60 con 0x7f899c0047c0 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.443+0000 7f89c2e00640 1 -- 172.21.15.26:0/3957476576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f899c0047c0 msgr2=0x7f899c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.443+0000 7f89c2e00640 1 --2- 172.21.15.26:0/3957476576 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f899c0047c0 0x7f899c004c40 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f89a400ba00 tx=0x7f899c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.443+0000 7f89c2400640 1 -- 172.21.15.26:0/3957476576 reap_dead start 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.443+0000 7f89c2e00640 1 -- 172.21.15.26:0/3957476576 shutdown_connections 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.443+0000 7f89c2e00640 1 -- 172.21.15.26:0/3957476576 >> 172.21.15.26:0/3957476576 conn(0x7f89bc06a800 msgr2=0x7f89bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.443+0000 7f89c2e00640 1 -- 172.21.15.26:0/3957476576 shutdown_connections 2024-01-15T18:00:20.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.443+0000 7f89c2e00640 1 -- 172.21.15.26:0/3957476576 wait complete. 2024-01-15T18:00:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.446+0000 7f89c2e00640 1 Processor -- start 2024-01-15T18:00:20.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.446+0000 7f89c2e00640 1 -- start start 2024-01-15T18:00:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.447+0000 7f89c2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0047c0 0x7f89bc135f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.447+0000 7f89c2e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1f6ee0 0x7f89bc1f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c2e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89bc142b50 0x7f89bc142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c2e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc143510 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c2e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc1f7360 con 0x7f89bc142b50 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc136e70 con 0x7f899c0047c0 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1f6ee0 0x7f89bc1f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1f6ee0 0x7f89bc1f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54354/0 (socket says 172.21.15.26:54354) 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c1000640 1 -- 172.21.15.26:0/4291831221 learned_addr learned my addr 172.21.15.26:0/4291831221 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c1a00640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0047c0 0x7f89bc135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c1000640 1 -- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0047c0 msgr2=0x7f89bc135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c2400640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89bc142b50 0x7f89bc142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c1000640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0047c0 0x7f89bc135f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c1000640 1 -- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89bc142b50 msgr2=0x7f89bc142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c1000640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89bc142b50 0x7f89bc142fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.448+0000 7f89c1000640 1 -- 172.21.15.26:0/4291831221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc1365d0 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.449+0000 7f89c1000640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1f6ee0 0x7f89bc1f2720 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f89a400bb20 tx=0x7f89a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.449+0000 7f89b9600640 1 -- 172.21.15.26:0/4291831221 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4004220 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.449+0000 7f89b9600640 1 -- 172.21.15.26:0/4291831221 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4004380 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.449+0000 7f89b9600640 1 -- 172.21.15.26:0/4291831221 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a403ba30 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.449+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89bc1368b0 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.449+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc0ac250 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.450+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89bc0ac410 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.451+0000 7f89b9600640 1 -- 172.21.15.26:0/4291831221 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 13) v1 ==== 89998+0+0 (secure 0 0 0) 0x7f89a403bb90 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.452+0000 7f89b9600640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899406b7d0 0x7f899406dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.452+0000 7f89b9600640 1 -- 172.21.15.26:0/4291831221 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f89a40acd90 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.453+0000 7f89c1a00640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899406b7d0 0x7f899406dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.454+0000 7f89c1a00640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899406b7d0 0x7f899406dc90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009f10 tx=0x7f89b400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:20.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.457+0000 7f89b9600640 1 -- 172.21.15.26:0/4291831221 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89a4077400 con 0x7f89bc1f6ee0 2024-01-15T18:00:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.670+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6933", "target": ["mon-mgr", ""]}) v1 -- 0x7f89bc0afa80 con 0x7f899406b7d0 2024-01-15T18:00:20.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.712+0000 7f89b9600640 1 -- 172.21.15.26:0/4291831221 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f89bc0afa80 con 0x7f899406b7d0 2024-01-15T18:00:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899406b7d0 msgr2=0x7f899406dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899406b7d0 0x7f899406dc90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89b4009f10 tx=0x7f89b400b040 comp rx=0 tx=0).stop 2024-01-15T18:00:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1f6ee0 msgr2=0x7f89bc1f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1f6ee0 0x7f89bc1f2720 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f89a400bb20 tx=0x7f89a4004730 comp rx=0 tx=0).stop 2024-01-15T18:00:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 shutdown_connections 2024-01-15T18:00:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899406b7d0 0x7f899406dc90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89bc142b50 0x7f89bc142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89bc1f6ee0 0x7f89bc1f2720 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 --2- 172.21.15.26:0/4291831221 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f899c0047c0 0x7f89bc135f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.714+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 >> 172.21.15.26:0/4291831221 conn(0x7f89bc06a800 msgr2=0x7f89bc0aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.715+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 shutdown_connections 2024-01-15T18:00:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.715+0000 7f89c2e00640 1 -- 172.21.15.26:0/4291831221 wait complete. 2024-01-15T18:00:20.737 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6931 2024-01-15T18:00:20.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.944+0000 7f4c11200640 1 Processor -- start 2024-01-15T18:00:20.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.944+0000 7f4c11200640 1 -- start start 2024-01-15T18:00:20.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.946+0000 7f4c11200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c0c0d06b0 0x7f4c0c15a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.946+0000 7f4c11200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0c15dbb0 con 0x7f4c0c157d30 2024-01-15T18:00:20.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.946+0000 7f4c11200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0c072330 con 0x7f4c0c157950 2024-01-15T18:00:20.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.946+0000 7f4c11200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0c15e4c0 con 0x7f4c0c0d06b0 2024-01-15T18:00:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.947+0000 7f4c09400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c0c0d06b0 0x7f4c0c15a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.947+0000 7f4c09400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4c0c0d06b0 0x7f4c0c15a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54356/0 (socket says 172.21.15.26:54356) 2024-01-15T18:00:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.947+0000 7f4c09400640 1 -- 172.21.15.26:0/3457069882 learned_addr learned my addr 172.21.15.26:0/3457069882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.947+0000 7f4c09400640 1 -- 172.21.15.26:0/3457069882 >> v1:172.21.15.124:6789/0 conn(0x7f4c0c157950 legacy=0x7f4c0c0cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.947+0000 7f4c09400640 1 -- 172.21.15.26:0/3457069882 >> v1:172.21.15.26:6789/0 conn(0x7f4c0c157d30 legacy=0x7f4c0c15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:20.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.948+0000 7f4c09400640 1 -- 172.21.15.26:0/3457069882 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0c15b360 con 0x7f4c0c0d06b0 2024-01-15T18:00:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.950+0000 7f4c09400640 1 --2- 172.21.15.26:0/3457069882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c0d06b0 0x7f4c0c15a5c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009870 tx=0x7f4bf402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68182726f305f94b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.951+0000 7f4c08a00640 1 -- 172.21.15.26:0/3457069882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40317e0 con 0x7f4c0c0d06b0 2024-01-15T18:00:20.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.951+0000 7f4c08a00640 1 -- 172.21.15.26:0/3457069882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf4031940 con 0x7f4c0c0d06b0 2024-01-15T18:00:20.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.951+0000 7f4c08a00640 1 -- 172.21.15.26:0/3457069882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4031cd0 con 0x7f4c0c0d06b0 2024-01-15T18:00:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.952+0000 7f4c11200640 1 -- 172.21.15.26:0/3457069882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c0d06b0 msgr2=0x7f4c0c15a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.952+0000 7f4c11200640 1 --2- 172.21.15.26:0/3457069882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c0d06b0 0x7f4c0c15a5c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009870 tx=0x7f4bf402b880 comp rx=0 tx=0).stop 2024-01-15T18:00:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.952+0000 7f4c11200640 1 -- 172.21.15.26:0/3457069882 shutdown_connections 2024-01-15T18:00:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.952+0000 7f4c11200640 1 --2- 172.21.15.26:0/3457069882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c0d06b0 0x7f4c0c15a5c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.952+0000 7f4c11200640 1 -- 172.21.15.26:0/3457069882 >> 172.21.15.26:0/3457069882 conn(0x7f4c0c06b4f0 msgr2=0x7f4c0c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.952+0000 7f4c11200640 1 -- 172.21.15.26:0/3457069882 shutdown_connections 2024-01-15T18:00:20.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.953+0000 7f4c11200640 1 -- 172.21.15.26:0/3457069882 wait complete. 2024-01-15T18:00:20.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.954+0000 7f4c11200640 1 Processor -- start 2024-01-15T18:00:20.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.954+0000 7f4c11200640 1 -- start start 2024-01-15T18:00:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.955+0000 7f4c11200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c0c0d06b0 0x7f4c0c159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.955+0000 7f4c11200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c157950 0x7f4c0c0b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.955+0000 7f4c11200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c0c157d30 0x7f4c0c1518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.955+0000 7f4c11200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c159260 con 0x7f4c0c157950 2024-01-15T18:00:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.955+0000 7f4c11200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c154ba0 con 0x7f4c0c157d30 2024-01-15T18:00:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.955+0000 7f4c11200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0c1585e0 con 0x7f4c0c0d06b0 2024-01-15T18:00:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c157950 0x7f4c0c0b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c157950 0x7f4c0c0b0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54358/0 (socket says 172.21.15.26:54358) 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09400640 1 -- 172.21.15.26:0/3758350322 learned_addr learned my addr 172.21.15.26:0/3758350322 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09400640 1 -- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c0c0d06b0 msgr2=0x7f4c0c159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09e00640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c0c0d06b0 0x7f4c0c159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09400640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c0c0d06b0 0x7f4c0c159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09400640 1 -- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c0c157d30 msgr2=0x7f4c0c1518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c0a800640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c0c157d30 0x7f4c0c1518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09400640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c0c157d30 0x7f4c0c1518d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09400640 1 -- 172.21.15.26:0/3758350322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40094d0 con 0x7f4c0c157950 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c09e00640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c0c0d06b0 0x7f4c0c159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.956+0000 7f4c0a800640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c0c157d30 0x7f4c0c1518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:20.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.957+0000 7f4c09400640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c157950 0x7f4c0c0b0f40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000c00 tx=0x7f4bf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:20.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.957+0000 7f4c02000640 1 -- 172.21.15.26:0/3758350322 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4032980 con 0x7f4c0c157950 2024-01-15T18:00:20.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.957+0000 7f4c02000640 1 -- 172.21.15.26:0/3758350322 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf4032b00 con 0x7f4c0c157950 2024-01-15T18:00:20.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.957+0000 7f4c02000640 1 -- 172.21.15.26:0/3758350322 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40453f0 con 0x7f4c0c157950 2024-01-15T18:00:20.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.957+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0c1588c0 con 0x7f4c0c157950 2024-01-15T18:00:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.957+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0c158e80 con 0x7f4c0c157950 2024-01-15T18:00:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.958+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0c1593a0 con 0x7f4c0c157950 2024-01-15T18:00:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.959+0000 7f4c02000640 1 -- 172.21.15.26:0/3758350322 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 90100+0+0 (secure 0 0 0) 0x7f4bf404d3f0 con 0x7f4c0c157950 2024-01-15T18:00:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.960+0000 7f4c02000640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4be406b8f0 0x7f4be406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.960+0000 7f4c02000640 1 -- 172.21.15.26:0/3758350322 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4bf40ac970 con 0x7f4c0c157950 2024-01-15T18:00:20.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.961+0000 7f4c09e00640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4be406b8f0 0x7f4be406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:20.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.962+0000 7f4c09e00640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4be406b8f0 0x7f4be406ddb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc007fd0 tx=0x7f4bfc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:20.965+0000 7f4c02000640 1 -- 172.21.15.26:0/3758350322 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c0c1593a0 con 0x7f4c0c157950 2024-01-15T18:00:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.192+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6931", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c0c15e0e0 con 0x7f4be406b8f0 2024-01-15T18:00:21.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.238+0000 7f4c02000640 1 -- 172.21.15.26:0/3758350322 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4bfc0012c0 con 0x7f4be406b8f0 2024-01-15T18:00:21.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.241+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4be406b8f0 msgr2=0x7f4be406ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:21.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.241+0000 7f4c11200640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4be406b8f0 0x7f4be406ddb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc007fd0 tx=0x7f4bfc009040 comp rx=0 tx=0).stop 2024-01-15T18:00:21.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.241+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c157950 msgr2=0x7f4c0c0b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:21.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.241+0000 7f4c11200640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c157950 0x7f4c0c0b0f40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000c00 tx=0x7f4bf40046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.242+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 shutdown_connections 2024-01-15T18:00:21.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.242+0000 7f4c11200640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4be406b8f0 0x7f4be406ddb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.242+0000 7f4c11200640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4c0c157d30 0x7f4c0c1518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.242+0000 7f4c11200640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4c0c157950 0x7f4c0c0b0f40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.242+0000 7f4c11200640 1 --2- 172.21.15.26:0/3758350322 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4c0c0d06b0 0x7f4c0c159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.242+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 >> 172.21.15.26:0/3758350322 conn(0x7f4c0c06b4f0 msgr2=0x7f4c0c164110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:21.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.242+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 shutdown_connections 2024-01-15T18:00:21.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.242+0000 7f4c11200640 1 -- 172.21.15.26:0/3758350322 wait complete. 2024-01-15T18:00:21.266 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone6932 2024-01-15T18:00:21.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.487+0000 7feb59200640 1 Processor -- start 2024-01-15T18:00:21.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.487+0000 7feb59200640 1 -- start start 2024-01-15T18:00:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.489+0000 7feb59200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feb540cf9a0 0x7feb5415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:21.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.489+0000 7feb59200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb54070c40 con 0x7feb540ae5a0 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.489+0000 7feb59200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb5406c470 con 0x7feb540b3ae0 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.489+0000 7feb59200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb54071450 con 0x7feb540cf9a0 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.489+0000 7feb51e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feb540ae5a0 0x7feb5406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49254/0 (socket says 172.21.15.26:49254) 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.489+0000 7feb51e00640 1 -- 172.21.15.26:0/1891891630 learned_addr learned my addr 172.21.15.26:0/1891891630 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.489+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585385422 0 0) 0x7feb54070c40 con 0x7feb540ae5a0 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.489+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb34003520 con 0x7feb540ae5a0 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.489+0000 7feb52800640 1 --2- 172.21.15.26:0/1891891630 >> v2:172.21.15.124:3300/0 conn(0x7feb540cf9a0 0x7feb5415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1037009687 0 0) 0x7feb34003520 con 0x7feb540ae5a0 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 >> v2:172.21.15.124:3300/0 conn(0x7feb540cf9a0 msgr2=0x7feb5415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 --2- 172.21.15.26:0/1891891630 >> v2:172.21.15.124:3300/0 conn(0x7feb540cf9a0 0x7feb5415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 >> v1:172.21.15.124:6789/0 conn(0x7feb540b3ae0 legacy=0x7feb540cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5406c750 con 0x7feb540ae5a0 2024-01-15T18:00:21.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7feb44001670 con 0x7feb540ae5a0 2024-01-15T18:00:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 >> v1:172.21.15.26:6789/0 conn(0x7feb540ae5a0 legacy=0x7feb5406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb340039c0 0x7feb34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340047c0 0x7feb34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb340051d0 0x7feb3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb54071450 con 0x7feb340047c0 2024-01-15T18:00:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3400c0e0 con 0x7feb340039c0 2024-01-15T18:00:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3400c2a0 con 0x7feb340051d0 2024-01-15T18:00:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb51400640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340047c0 0x7feb34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:21.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.490+0000 7feb51e00640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb340039c0 0x7feb34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.491+0000 7feb51400640 1 -- 172.21.15.26:0/1891891630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb340051d0 msgr2=0x7feb3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.491+0000 7feb51400640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb340051d0 0x7feb3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.491+0000 7feb51400640 1 -- 172.21.15.26:0/1891891630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb340039c0 msgr2=0x7feb34003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.491+0000 7feb51400640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb340039c0 0x7feb34003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.491+0000 7feb51400640 1 -- 172.21.15.26:0/1891891630 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34003520 con 0x7feb340047c0 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.491+0000 7feb52800640 1 -- 172.21.15.26:0/1891891630 reap_dead start 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.492+0000 7feb51400640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340047c0 0x7feb34004c40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feb3c003db0 tx=0x7feb340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e94f3e07016b2c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.492+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c032600 con 0x7feb340047c0 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.492+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c02e050 con 0x7feb340047c0 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.492+0000 7feb50a00640 1 -- 172.21.15.26:0/1891891630 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c032a20 con 0x7feb340047c0 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.493+0000 7feb59200640 1 -- 172.21.15.26:0/1891891630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340047c0 msgr2=0x7feb34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:21.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.493+0000 7feb59200640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340047c0 0x7feb34004c40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feb3c003db0 tx=0x7feb340057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.493+0000 7feb59200640 1 -- 172.21.15.26:0/1891891630 shutdown_connections 2024-01-15T18:00:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.493+0000 7feb59200640 1 --2- 172.21.15.26:0/1891891630 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb340047c0 0x7feb34004c40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.493+0000 7feb59200640 1 -- 172.21.15.26:0/1891891630 >> 172.21.15.26:0/1891891630 conn(0x7feb5406a800 msgr2=0x7feb5406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.493+0000 7feb59200640 1 -- 172.21.15.26:0/1891891630 shutdown_connections 2024-01-15T18:00:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.493+0000 7feb59200640 1 -- 172.21.15.26:0/1891891630 wait complete. 2024-01-15T18:00:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.495+0000 7feb59200640 1 Processor -- start 2024-01-15T18:00:21.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.495+0000 7feb59200640 1 -- start start 2024-01-15T18:00:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.496+0000 7feb59200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb340047c0 0x7feb540ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.496+0000 7feb59200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb540cb8a0 0x7feb5414a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:21.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.496+0000 7feb59200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb54147730 0x7feb540b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.496+0000 7feb59200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb54148b80 con 0x7feb540cb8a0 2024-01-15T18:00:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.496+0000 7feb59200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540cbce0 con 0x7feb54147730 2024-01-15T18:00:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.496+0000 7feb59200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb5414aed0 con 0x7feb340047c0 2024-01-15T18:00:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.496+0000 7feb51400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb540cb8a0 0x7feb5414a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.496+0000 7feb51400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb540cb8a0 0x7feb5414a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54378/0 (socket says 172.21.15.26:54378) 2024-01-15T18:00:21.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.496+0000 7feb51400640 1 -- 172.21.15.26:0/2336855995 learned_addr learned my addr 172.21.15.26:0/2336855995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb51400640 1 -- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb340047c0 msgr2=0x7feb540ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb52800640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb54147730 0x7feb540b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb51e00640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb340047c0 0x7feb540ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:21.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb51400640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb340047c0 0x7feb540ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb51400640 1 -- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb54147730 msgr2=0x7feb540b22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb51400640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb54147730 0x7feb540b22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb51400640 1 -- 172.21.15.26:0/2336855995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb540c4110 con 0x7feb540cb8a0 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb52800640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb54147730 0x7feb540b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb51400640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb540cb8a0 0x7feb5414a990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feb3c003ed0 tx=0x7feb3c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.497+0000 7feb4a000640 1 -- 172.21.15.26:0/2336855995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c03d420 con 0x7feb540cb8a0 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.498+0000 7feb4a000640 1 -- 172.21.15.26:0/2336855995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c03d580 con 0x7feb540cb8a0 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.498+0000 7feb4a000640 1 -- 172.21.15.26:0/2336855995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c03d930 con 0x7feb540cb8a0 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.498+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb54147f20 con 0x7feb540cb8a0 2024-01-15T18:00:21.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.498+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb541484c0 con 0x7feb540cb8a0 2024-01-15T18:00:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.499+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb540675d0 con 0x7feb540cb8a0 2024-01-15T18:00:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.500+0000 7feb4a000640 1 -- 172.21.15.26:0/2336855995 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 90100+0+0 (secure 0 0 0) 0x7feb3c03da90 con 0x7feb540cb8a0 2024-01-15T18:00:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.501+0000 7feb4a000640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb2c06b920 0x7feb2c06dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.501+0000 7feb4a000640 1 -- 172.21.15.26:0/2336855995 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7feb3c0ac470 con 0x7feb540cb8a0 2024-01-15T18:00:21.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.501+0000 7feb51e00640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb2c06b920 0x7feb2c06dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:21.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.502+0000 7feb51e00640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb2c06b920 0x7feb2c06dde0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feb44001050 tx=0x7feb44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:21.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.505+0000 7feb4a000640 1 -- 172.21.15.26:0/2336855995 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb3c076930 con 0x7feb540cb8a0 2024-01-15T18:00:21.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.727+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone6932", "target": ["mon-mgr", ""]}) v1 -- 0x7feb54070550 con 0x7feb2c06b920 2024-01-15T18:00:21.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.775+0000 7feb4a000640 1 -- 172.21.15.26:0/2336855995 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7feb44001550 con 0x7feb2c06b920 2024-01-15T18:00:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.778+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb2c06b920 msgr2=0x7feb2c06dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.778+0000 7feb59200640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb2c06b920 0x7feb2c06dde0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feb44001050 tx=0x7feb44009040 comp rx=0 tx=0).stop 2024-01-15T18:00:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.778+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb540cb8a0 msgr2=0x7feb5414a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.778+0000 7feb59200640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb540cb8a0 0x7feb5414a990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feb3c003ed0 tx=0x7feb3c0036f0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.779+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 shutdown_connections 2024-01-15T18:00:21.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.779+0000 7feb59200640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb2c06b920 0x7feb2c06dde0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.779+0000 7feb59200640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb54147730 0x7feb540b22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.779+0000 7feb59200640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb540cb8a0 0x7feb5414a990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.779+0000 7feb59200640 1 --2- 172.21.15.26:0/2336855995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb340047c0 0x7feb540ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:21.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.779+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 >> 172.21.15.26:0/2336855995 conn(0x7feb5406a800 msgr2=0x7feb540cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:21.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.779+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 shutdown_connections 2024-01-15T18:00:21.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:21.779+0000 7feb59200640 1 -- 172.21.15.26:0/2336855995 wait complete. 2024-01-15T18:00:21.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:21.803 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:00:21.869 INFO:teuthology.orchestra.run.smithi026.stdout:2 2024-01-15T18:00:21.870 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:00:22.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.085+0000 7fe1e4a00640 1 Processor -- start 2024-01-15T18:00:22.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.086+0000 7fe1e4a00640 1 -- start start 2024-01-15T18:00:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.087+0000 7fe1e4a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13df80 0x7fe1dc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.087+0000 7fe1e4a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe1dc13e360 0x7fe1dc0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.087+0000 7fe1e4a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1dc15f9c0 0x7fe1dc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.087+0000 7fe1e4a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc06c470 con 0x7fe1dc15f9c0 2024-01-15T18:00:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.087+0000 7fe1e4a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc06c750 con 0x7fe1dc13e360 2024-01-15T18:00:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.087+0000 7fe1e4a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc06ca30 con 0x7fe1dc13df80 2024-01-15T18:00:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e2a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1dc15f9c0 0x7fe1dc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e2a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe1dc15f9c0 0x7fe1dc0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53790/0 (socket says 172.21.15.26:53790) 2024-01-15T18:00:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e2a00640 1 -- 172.21.15.26:0/3887848707 learned_addr learned my addr 172.21.15.26:0/3887848707 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e2000640 1 --2- 172.21.15.26:0/3887848707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13df80 0x7fe1dc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e1600640 1 --2- 172.21.15.26:0/3887848707 >> v2:172.21.15.124:3300/0 conn(0x7fe1dc13e360 0x7fe1dc0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e2a00640 1 -- 172.21.15.26:0/3887848707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13df80 msgr2=0x7fe1dc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e2a00640 1 --2- 172.21.15.26:0/3887848707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13df80 0x7fe1dc15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e2a00640 1 -- 172.21.15.26:0/3887848707 >> v2:172.21.15.124:3300/0 conn(0x7fe1dc13e360 msgr2=0x7fe1dc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e2a00640 1 --2- 172.21.15.26:0/3887848707 >> v2:172.21.15.124:3300/0 conn(0x7fe1dc13e360 0x7fe1dc0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.088+0000 7fe1e2a00640 1 -- 172.21.15.26:0/3887848707 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc06cd10 con 0x7fe1dc15f9c0 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.089+0000 7fe1e2a00640 1 --2- 172.21.15.26:0/3887848707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15f9c0 0x7fe1dc0cf7d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0cfed0 tx=0x7fe1d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6e13f7400ebcb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.090+0000 7fe1e0c00640 1 -- 172.21.15.26:0/3887848707 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d402ca80 con 0x7fe1dc15f9c0 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.090+0000 7fe1e0c00640 1 -- 172.21.15.26:0/3887848707 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d402cbe0 con 0x7fe1dc15f9c0 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.090+0000 7fe1e0c00640 1 -- 172.21.15.26:0/3887848707 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d40343f0 con 0x7fe1dc15f9c0 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.090+0000 7fe1e4a00640 1 -- 172.21.15.26:0/3887848707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15f9c0 msgr2=0x7fe1dc0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.090+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/3887848707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15f9c0 0x7fe1dc0cf7d0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0cfed0 tx=0x7fe1d402bee0 comp rx=0 tx=0).stop 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.091+0000 7fe1e4a00640 1 -- 172.21.15.26:0/3887848707 shutdown_connections 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.091+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/3887848707 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc15f9c0 0x7fe1dc0cf7d0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.091+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/3887848707 >> v2:172.21.15.124:3300/0 conn(0x7fe1dc13e360 0x7fe1dc0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:22.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.091+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/3887848707 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13df80 0x7fe1dc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.091+0000 7fe1e4a00640 1 -- 172.21.15.26:0/3887848707 >> 172.21.15.26:0/3887848707 conn(0x7fe1dc06a800 msgr2=0x7fe1dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.091+0000 7fe1e4a00640 1 -- 172.21.15.26:0/3887848707 shutdown_connections 2024-01-15T18:00:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.091+0000 7fe1e4a00640 1 -- 172.21.15.26:0/3887848707 wait complete. 2024-01-15T18:00:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.092+0000 7fe1e4a00640 1 Processor -- start 2024-01-15T18:00:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.093+0000 7fe1e4a00640 1 -- start start 2024-01-15T18:00:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.093+0000 7fe1e4a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc13df80 0x7fe1dc154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.093+0000 7fe1e4a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13e360 0x7fe1dc0b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e4a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1dc15f9c0 0x7fe1dc0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e4a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc13bf70 con 0x7fe1dc13df80 2024-01-15T18:00:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e4a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc154c60 con 0x7fe1dc15f9c0 2024-01-15T18:00:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e4a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc155e60 con 0x7fe1dc13e360 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc13df80 0x7fe1dc154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc13df80 0x7fe1dc154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53804/0 (socket says 172.21.15.26:53804) 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e2000640 1 -- 172.21.15.26:0/1832739334 learned_addr learned my addr 172.21.15.26:0/1832739334 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e1600640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13e360 0x7fe1dc0b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e2000640 1 -- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13e360 msgr2=0x7fe1dc0b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e2000640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13e360 0x7fe1dc0b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e2000640 1 -- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1dc15f9c0 msgr2=0x7fe1dc0b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e2000640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1dc15f9c0 0x7fe1dc0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e2000640 1 -- 172.21.15.26:0/1832739334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4009b30 con 0x7fe1dc13df80 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.094+0000 7fe1e1600640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13e360 0x7fe1dc0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.095+0000 7fe1e2000640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc13df80 0x7fe1dc154720 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00d5f0 tx=0x7fe1cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.095+0000 7fe1da000640 1 -- 172.21.15.26:0/1832739334 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0125f0 con 0x7fe1dc13df80 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.095+0000 7fe1da000640 1 -- 172.21.15.26:0/1832739334 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc012750 con 0x7fe1dc13df80 2024-01-15T18:00:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.095+0000 7fe1da000640 1 -- 172.21.15.26:0/1832739334 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc012b00 con 0x7fe1dc13df80 2024-01-15T18:00:22.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.095+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc156140 con 0x7fe1dc13df80 2024-01-15T18:00:22.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.095+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc156700 con 0x7fe1dc13df80 2024-01-15T18:00:22.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.096+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc0675d0 con 0x7fe1dc13df80 2024-01-15T18:00:22.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.097+0000 7fe1da000640 1 -- 172.21.15.26:0/1832739334 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 14) v1 ==== 90100+0+0 (secure 0 0 0) 0x7fe1cc012c60 con 0x7fe1dc13df80 2024-01-15T18:00:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.098+0000 7fe1da000640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe1b406b8f0 0x7fe1b406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.098+0000 7fe1da000640 1 -- 172.21.15.26:0/1832739334 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe1cc08ac00 con 0x7fe1dc13df80 2024-01-15T18:00:22.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.099+0000 7fe1e1600640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe1b406b8f0 0x7fe1b406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:22.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.100+0000 7fe1e1600640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe1b406b8f0 0x7fe1b406ddb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc15ed90 tx=0x7fe1c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:22.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.102+0000 7fe1da000640 1 -- 172.21.15.26:0/1832739334 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1cc055120 con 0x7fe1dc13df80 2024-01-15T18:00:22.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:22.437+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1dc1408d0 con 0x7fe1b406b8f0 2024-01-15T18:00:22.474 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-01-15T18:00:22.470+0000 7f9819c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:00:22.582 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T18:00:24.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:24.979+0000 7fe1da000640 1 -- 172.21.15.26:0/1832739334 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fe1cc0554f0 con 0x7fe1dc13df80 2024-01-15T18:00:25.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.505+0000 7fe1da000640 1 -- 172.21.15.26:0/1832739334 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fe1c4009d90 con 0x7fe1b406b8f0 2024-01-15T18:00:25.512 INFO:teuthology.orchestra.run.smithi026.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:00:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe1b406b8f0 msgr2=0x7fe1b406ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe1b406b8f0 0x7fe1b406ddb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc15ed90 tx=0x7fe1c4008040 comp rx=0 tx=0).stop 2024-01-15T18:00:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc13df80 msgr2=0x7fe1dc154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc13df80 0x7fe1dc154720 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00d5f0 tx=0x7fe1cc00dab0 comp rx=0 tx=0).stop 2024-01-15T18:00:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 shutdown_connections 2024-01-15T18:00:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1dc13e360 0x7fe1dc0b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe1b406b8f0 0x7fe1b406ddb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1dc15f9c0 0x7fe1dc0b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 --2- 172.21.15.26:0/1832739334 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe1dc13df80 0x7fe1dc154720 secure :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00d5f0 tx=0x7fe1cc00dab0 comp rx=0 tx=0).stop 2024-01-15T18:00:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.513+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 >> 172.21.15.26:0/1832739334 conn(0x7fe1dc06a800 msgr2=0x7fe1dc13c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:25.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.516+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 shutdown_connections 2024-01-15T18:00:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:25.516+0000 7fe1e4a00640 1 -- 172.21.15.26:0/1832739334 wait complete. 2024-01-15T18:00:25.552 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:00:25.553 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:00:25.713 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:00:25.853 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:00:25.853 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:00:25.853 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:00:25.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:00:26.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.199+0000 7fe755c00640 1 Processor -- start 2024-01-15T18:00:26.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.199+0000 7fe755c00640 1 -- start start 2024-01-15T18:00:26.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.201+0000 7fe755c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe750140300 0x7fe75006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.201+0000 7fe755c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7500b5440 con 0x7fe75013ff20 2024-01-15T18:00:26.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.201+0000 7fe755c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe750161c80 con 0x7fe75006a920 2024-01-15T18:00:26.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.201+0000 7fe755c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75006adb0 con 0x7fe750140300 2024-01-15T18:00:26.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.201+0000 7fe74fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe75013ff20 0x7fe75015e1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49756/0 (socket says 172.21.15.26:49756) 2024-01-15T18:00:26.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.201+0000 7fe74fe00640 1 -- 172.21.15.26:0/2420954650 learned_addr learned my addr 172.21.15.26:0/2420954650 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:26.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.201+0000 7fe74f400640 1 --2- 172.21.15.26:0/2420954650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe750140300 0x7fe75006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.202+0000 7fe74ea00640 1 -- 172.21.15.26:0/2420954650 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880848549 0 0) 0x7fe750161c80 con 0x7fe75006a920 2024-01-15T18:00:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.202+0000 7fe74ea00640 1 -- 172.21.15.26:0/2420954650 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe730003520 con 0x7fe75006a920 2024-01-15T18:00:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.202+0000 7fe74f400640 1 -- 172.21.15.26:0/2420954650 >> v1:172.21.15.124:6789/0 conn(0x7fe75006a920 legacy=0x7fe750141c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.202+0000 7fe74f400640 1 -- 172.21.15.26:0/2420954650 >> v1:172.21.15.26:6789/0 conn(0x7fe75013ff20 legacy=0x7fe75015e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.202+0000 7fe74f400640 1 -- 172.21.15.26:0/2420954650 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7500b4830 con 0x7fe750140300 2024-01-15T18:00:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.203+0000 7fe74f400640 1 --2- 172.21.15.26:0/2420954650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe750140300 0x7fe75006a3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe7380097d0 tx=0x7fe73802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7908156e1f7a138 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:26.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.203+0000 7fe74ea00640 1 -- 172.21.15.26:0/2420954650 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738031780 con 0x7fe750140300 2024-01-15T18:00:26.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.203+0000 7fe74ea00640 1 -- 172.21.15.26:0/2420954650 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7380318e0 con 0x7fe750140300 2024-01-15T18:00:26.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.204+0000 7fe74ea00640 1 -- 172.21.15.26:0/2420954650 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738031c70 con 0x7fe750140300 2024-01-15T18:00:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.204+0000 7fe755c00640 1 -- 172.21.15.26:0/2420954650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe750140300 msgr2=0x7fe75006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.204+0000 7fe755c00640 1 --2- 172.21.15.26:0/2420954650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe750140300 0x7fe75006a3e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe7380097d0 tx=0x7fe73802b820 comp rx=0 tx=0).stop 2024-01-15T18:00:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.204+0000 7fe755c00640 1 -- 172.21.15.26:0/2420954650 shutdown_connections 2024-01-15T18:00:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.204+0000 7fe755c00640 1 --2- 172.21.15.26:0/2420954650 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe750140300 0x7fe75006a3e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.204+0000 7fe755c00640 1 -- 172.21.15.26:0/2420954650 >> 172.21.15.26:0/2420954650 conn(0x7fe7500b1cd0 msgr2=0x7fe7500b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.204+0000 7fe755c00640 1 -- 172.21.15.26:0/2420954650 shutdown_connections 2024-01-15T18:00:26.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.204+0000 7fe755c00640 1 -- 172.21.15.26:0/2420954650 wait complete. 2024-01-15T18:00:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.207+0000 7fe755c00640 1 Processor -- start 2024-01-15T18:00:26.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.207+0000 7fe755c00640 1 -- start start 2024-01-15T18:00:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe755c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe75006a920 0x7fe750157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe755c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe75013ff20 0x7fe7501f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe755c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe750140300 0x7fe7500ae510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe755c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7500ccd80 con 0x7fe75013ff20 2024-01-15T18:00:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe755c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7501577b0 con 0x7fe750140300 2024-01-15T18:00:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe755c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7501f71c0 con 0x7fe75006a920 2024-01-15T18:00:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe74f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe75013ff20 0x7fe7501f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe74f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe75013ff20 0x7fe7501f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53826/0 (socket says 172.21.15.26:53826) 2024-01-15T18:00:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe74f400640 1 -- 172.21.15.26:0/93580138 learned_addr learned my addr 172.21.15.26:0/93580138 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe74fe00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe75006a920 0x7fe750157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.209+0000 7fe755200640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe750140300 0x7fe7500ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.210+0000 7fe74f400640 1 -- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe75006a920 msgr2=0x7fe750157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.210+0000 7fe74f400640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe75006a920 0x7fe750157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.210+0000 7fe74f400640 1 -- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe750140300 msgr2=0x7fe7500ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.210+0000 7fe74f400640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe750140300 0x7fe7500ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.210+0000 7fe74f400640 1 -- 172.21.15.26:0/93580138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe738009400 con 0x7fe75013ff20 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.210+0000 7fe74fe00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe75006a920 0x7fe750157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.210+0000 7fe74f400640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe75013ff20 0x7fe7501f6300 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe738034040 tx=0x7fe7380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.210+0000 7fe747e00640 1 -- 172.21.15.26:0/93580138 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738004230 con 0x7fe75013ff20 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.210+0000 7fe747e00640 1 -- 172.21.15.26:0/93580138 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7380043b0 con 0x7fe75013ff20 2024-01-15T18:00:26.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.211+0000 7fe747e00640 1 -- 172.21.15.26:0/93580138 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe738004520 con 0x7fe75013ff20 2024-01-15T18:00:26.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.211+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7501f6c00 con 0x7fe75013ff20 2024-01-15T18:00:26.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.211+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7500cca10 con 0x7fe75013ff20 2024-01-15T18:00:26.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.213+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7500ccba0 con 0x7fe75013ff20 2024-01-15T18:00:26.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.214+0000 7fe747e00640 1 -- 172.21.15.26:0/93580138 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fe738032860 con 0x7fe75013ff20 2024-01-15T18:00:26.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.215+0000 7fe747e00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72806b860 0x7fe72806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.215+0000 7fe747e00640 1 -- 172.21.15.26:0/93580138 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fe7380ac810 con 0x7fe75013ff20 2024-01-15T18:00:26.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.215+0000 7fe74fe00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72806b860 0x7fe72806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.216+0000 7fe74fe00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72806b860 0x7fe72806dd20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe740000f60 tx=0x7fe740009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:26.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.224+0000 7fe747e00640 1 -- 172.21.15.26:0/93580138 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe738076fa0 con 0x7fe75013ff20 2024-01-15T18:00:26.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.450+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7500ccba0 con 0x7fe75013ff20 2024-01-15T18:00:26.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.451+0000 7fe747e00640 1 -- 172.21.15.26:0/93580138 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+13005 (secure 0 0 0) 0x7fe73807a5c0 con 0x7fe75013ff20 2024-01-15T18:00:26.455 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:26.456 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":52,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:00:25.482824+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1948779240":"2024-01-16T18:00:22.471234+0000","172.21.15.26:6833/1948779240":"2024-01-16T18:00:22.471234+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:00:26.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.454+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72806b860 msgr2=0x7fe72806dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.454+0000 7fe755c00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72806b860 0x7fe72806dd20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe740000f60 tx=0x7fe740009040 comp rx=0 tx=0).stop 2024-01-15T18:00:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.454+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe75013ff20 msgr2=0x7fe7501f6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.454+0000 7fe755c00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe75013ff20 0x7fe7501f6300 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe738034040 tx=0x7fe7380046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.455+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 shutdown_connections 2024-01-15T18:00:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.455+0000 7fe755c00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe72806b860 0x7fe72806dd20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.455+0000 7fe755c00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe750140300 0x7fe7500ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.455+0000 7fe755c00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe75013ff20 0x7fe7501f6300 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.455+0000 7fe755c00640 1 --2- 172.21.15.26:0/93580138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe75006a920 0x7fe750157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.455+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 >> 172.21.15.26:0/93580138 conn(0x7fe7500b1cd0 msgr2=0x7fe75015b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.455+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 shutdown_connections 2024-01-15T18:00:26.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.455+0000 7fe755c00640 1 -- 172.21.15.26:0/93580138 wait complete. 2024-01-15T18:00:26.486 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:00:26.486 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:00:26.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.733+0000 7feed7200640 1 Processor -- start 2024-01-15T18:00:26.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.733+0000 7feed7200640 1 -- start start 2024-01-15T18:00:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.734+0000 7feed7200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed013e360 0x7feed00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.734+0000 7feed7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed015f9c0 0x7feed00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.734+0000 7feed7200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed015c5f0 con 0x7feed013df80 2024-01-15T18:00:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.734+0000 7feed7200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed006c470 con 0x7feed013e360 2024-01-15T18:00:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.734+0000 7feed7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed006c750 con 0x7feed015f9c0 2024-01-15T18:00:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.734+0000 7feecf400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed013e360 0x7feed00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.734+0000 7feecf400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feed013e360 0x7feed00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53836/0 (socket says 172.21.15.26:53836) 2024-01-15T18:00:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.734+0000 7feecf400640 1 -- 172.21.15.26:0/1927130525 learned_addr learned my addr 172.21.15.26:0/1927130525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.734+0000 7feed5200640 1 --2- 172.21.15.26:0/1927130525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed015f9c0 0x7feed00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.735+0000 7feecf400640 1 -- 172.21.15.26:0/1927130525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed015f9c0 msgr2=0x7feed00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.735+0000 7feecf400640 1 --2- 172.21.15.26:0/1927130525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed015f9c0 0x7feed00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.735+0000 7feecf400640 1 -- 172.21.15.26:0/1927130525 >> v1:172.21.15.124:6789/0 conn(0x7feed013df80 legacy=0x7feed015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.735+0000 7feecf400640 1 -- 172.21.15.26:0/1927130525 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed006ca30 con 0x7feed013e360 2024-01-15T18:00:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.735+0000 7feed5200640 1 --2- 172.21.15.26:0/1927130525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed015f9c0 0x7feed00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:26.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.736+0000 7feecf400640 1 --2- 172.21.15.26:0/1927130525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 0x7feed00cf290 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009870 tx=0x7feeb802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ad82a96606e278 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.736+0000 7feecea00640 1 -- 172.21.15.26:0/1927130525 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8030840 con 0x7feed013e360 2024-01-15T18:00:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.736+0000 7feecea00640 1 -- 172.21.15.26:0/1927130525 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb80309a0 con 0x7feed013e360 2024-01-15T18:00:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.736+0000 7feecea00640 1 -- 172.21.15.26:0/1927130525 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8030d30 con 0x7feed013e360 2024-01-15T18:00:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.737+0000 7feed7200640 1 -- 172.21.15.26:0/1927130525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 msgr2=0x7feed00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.737+0000 7feed7200640 1 --2- 172.21.15.26:0/1927130525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 0x7feed00cf290 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009870 tx=0x7feeb802b980 comp rx=0 tx=0).stop 2024-01-15T18:00:26.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.737+0000 7feed7200640 1 -- 172.21.15.26:0/1927130525 shutdown_connections 2024-01-15T18:00:26.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.737+0000 7feed7200640 1 --2- 172.21.15.26:0/1927130525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed015f9c0 0x7feed00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.737+0000 7feed7200640 1 --2- 172.21.15.26:0/1927130525 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 0x7feed00cf290 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.737+0000 7feed7200640 1 -- 172.21.15.26:0/1927130525 >> 172.21.15.26:0/1927130525 conn(0x7feed006a800 msgr2=0x7feed006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:26.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.737+0000 7feed7200640 1 -- 172.21.15.26:0/1927130525 shutdown_connections 2024-01-15T18:00:26.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.737+0000 7feed7200640 1 -- 172.21.15.26:0/1927130525 wait complete. 2024-01-15T18:00:26.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.738+0000 7feed7200640 1 Processor -- start 2024-01-15T18:00:26.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.738+0000 7feed7200640 1 -- start start 2024-01-15T18:00:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.739+0000 7feed7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed013df80 0x7feed01315b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.739+0000 7feed7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 0x7feed0131f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.739+0000 7feed7200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feed015f9c0 0x7feed015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.739+0000 7feed7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed015e360 con 0x7feed013e360 2024-01-15T18:00:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.739+0000 7feed7200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00ae3d0 con 0x7feed015f9c0 2024-01-15T18:00:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.739+0000 7feed7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed015d6e0 con 0x7feed013df80 2024-01-15T18:00:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 0x7feed0131f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 0x7feed0131f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53838/0 (socket says 172.21.15.26:53838) 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecf400640 1 -- 172.21.15.26:0/4220387951 learned_addr learned my addr 172.21.15.26:0/4220387951 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecfe00640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed013df80 0x7feed01315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feed5200640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feed015f9c0 0x7feed015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecf400640 1 -- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed013df80 msgr2=0x7feed01315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecf400640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed013df80 0x7feed01315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecf400640 1 -- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feed015f9c0 msgr2=0x7feed015cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecf400640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feed015f9c0 0x7feed015cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecf400640 1 -- 172.21.15.26:0/4220387951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb80094d0 con 0x7feed013e360 2024-01-15T18:00:26.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feed5200640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feed015f9c0 0x7feed015cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.740+0000 7feecf400640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 0x7feed0131f90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000c00 tx=0x7feeb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.741+0000 7feec7e00640 1 -- 172.21.15.26:0/4220387951 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8039d50 con 0x7feed013e360 2024-01-15T18:00:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.741+0000 7feec7e00640 1 -- 172.21.15.26:0/4220387951 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8044480 con 0x7feed013e360 2024-01-15T18:00:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.741+0000 7feec7e00640 1 -- 172.21.15.26:0/4220387951 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8044830 con 0x7feed013e360 2024-01-15T18:00:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.741+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed015d9c0 con 0x7feed013e360 2024-01-15T18:00:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.741+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed015df80 con 0x7feed013e360 2024-01-15T18:00:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.742+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed015e4a0 con 0x7feed013e360 2024-01-15T18:00:26.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.743+0000 7feec7e00640 1 -- 172.21.15.26:0/4220387951 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7feeb802f390 con 0x7feed013e360 2024-01-15T18:00:26.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.743+0000 7feec7e00640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feea806b860 0x7feea806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:26.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.744+0000 7feec7e00640 1 -- 172.21.15.26:0/4220387951 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7feeb803e070 con 0x7feed013e360 2024-01-15T18:00:26.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.744+0000 7feecfe00640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feea806b860 0x7feea806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:26.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.745+0000 7feecfe00640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feea806b860 0x7feea806dd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feed012f470 tx=0x7feec0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:26.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.747+0000 7feec7e00640 1 -- 172.21.15.26:0/4220387951 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feed015e4a0 con 0x7feed013e360 2024-01-15T18:00:26.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.931+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feed015e4a0 con 0x7feed013e360 2024-01-15T18:00:26.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.932+0000 7feec7e00640 1 -- 172.21.15.26:0/4220387951 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7feeb8034020 con 0x7feed013e360 2024-01-15T18:00:26.936 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:00:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.934+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feea806b860 msgr2=0x7feea806dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.934+0000 7feed7200640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feea806b860 0x7feea806dd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7feed012f470 tx=0x7feec0009040 comp rx=0 tx=0).stop 2024-01-15T18:00:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.934+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 msgr2=0x7feed0131f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.934+0000 7feed7200640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 0x7feed0131f90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7feeb8000c00 tx=0x7feeb8004730 comp rx=0 tx=0).stop 2024-01-15T18:00:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.935+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 shutdown_connections 2024-01-15T18:00:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.935+0000 7feed7200640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feea806b860 0x7feea806dd20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.935+0000 7feed7200640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feed015f9c0 0x7feed015cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.935+0000 7feed7200640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feed013e360 0x7feed0131f90 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.935+0000 7feed7200640 1 --2- 172.21.15.26:0/4220387951 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feed013df80 0x7feed01315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.935+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 >> 172.21.15.26:0/4220387951 conn(0x7feed006a800 msgr2=0x7feed015eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.935+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 shutdown_connections 2024-01-15T18:00:26.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:26.935+0000 7feed7200640 1 -- 172.21.15.26:0/4220387951 wait complete. 2024-01-15T18:00:26.956 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:00:27.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.150+0000 7f2330800640 1 Processor -- start 2024-01-15T18:00:27.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.150+0000 7f2330800640 1 -- start start 2024-01-15T18:00:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.151+0000 7f2330800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23280cf9a0 0x7f232815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.151+0000 7f2330800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f232815e1e0 0x7f23280702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.152+0000 7f2330800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23280b1ee0 con 0x7f2328143ae0 2024-01-15T18:00:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.152+0000 7f2330800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23280b2780 con 0x7f232815e1e0 2024-01-15T18:00:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.152+0000 7f2330800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232815e990 con 0x7f23280cf9a0 2024-01-15T18:00:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.152+0000 7f232de00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2328143ae0 0x7f23280cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49762/0 (socket says 172.21.15.26:49762) 2024-01-15T18:00:27.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.152+0000 7f232de00640 1 -- 172.21.15.26:0/4244605066 learned_addr learned my addr 172.21.15.26:0/4244605066 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.152+0000 7f232d400640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23280cf9a0 0x7f232815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.152+0000 7f232e800640 1 --2- 172.21.15.26:0/4244605066 >> v2:172.21.15.124:3300/0 conn(0x7f232815e1e0 0x7f23280702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.152+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621520586 0 0) 0x7f23280b1ee0 con 0x7f2328143ae0 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.152+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2308003520 con 0x7f2328143ae0 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186261922 0 0) 0x7f2308003520 con 0x7f2328143ae0 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23280cf9a0 msgr2=0x7f232815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23280cf9a0 0x7f232815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 >> v2:172.21.15.124:3300/0 conn(0x7f232815e1e0 msgr2=0x7f23280702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 --2- 172.21.15.26:0/4244605066 >> v2:172.21.15.124:3300/0 conn(0x7f232815e1e0 0x7f23280702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23280b2130 con 0x7f2328143ae0 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f2320001670 con 0x7f2328143ae0 2024-01-15T18:00:27.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 >> v1:172.21.15.26:6789/0 conn(0x7f2328143ae0 legacy=0x7f23280cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23280cf9a0 0x7f2308003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2308003f40 0x7f23080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2308004d00 0x7f2308005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280b2780 con 0x7f2308003f40 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232815e990 con 0x7f2308004d00 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230800db90 con 0x7f23280cf9a0 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.153+0000 7f232d400640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2308003f40 0x7f23080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.154+0000 7f232d400640 1 -- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23280cf9a0 msgr2=0x7f2308003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.154+0000 7f232e800640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2308004d00 0x7f2308005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.154+0000 7f232d400640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23280cf9a0 0x7f2308003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.154+0000 7f232d400640 1 -- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2308004d00 msgr2=0x7f2308005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:27.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.154+0000 7f232d400640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2308004d00 0x7f2308005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.154+0000 7f232d400640 1 -- 172.21.15.26:0/4244605066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2308003520 con 0x7f2308003f40 2024-01-15T18:00:27.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.155+0000 7f232d400640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2308003f40 0x7f23080047c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2310009af0 tx=0x7f23080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf4e390a53dbbb7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:27.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.155+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231002c710 con 0x7f2308003f40 2024-01-15T18:00:27.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.155+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231002c870 con 0x7f2308003f40 2024-01-15T18:00:27.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.155+0000 7f232ca00640 1 -- 172.21.15.26:0/4244605066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231002cc00 con 0x7f2308003f40 2024-01-15T18:00:27.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.155+0000 7f2330800640 1 -- 172.21.15.26:0/4244605066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2308003f40 msgr2=0x7f23080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:27.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.155+0000 7f2330800640 1 --2- 172.21.15.26:0/4244605066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2308003f40 0x7f23080047c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2310009af0 tx=0x7f23080057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.155+0000 7f232e800640 1 -- 172.21.15.26:0/4244605066 reap_dead start 2024-01-15T18:00:27.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.155+0000 7f2330800640 1 -- 172.21.15.26:0/4244605066 shutdown_connections 2024-01-15T18:00:27.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.155+0000 7f2330800640 1 -- 172.21.15.26:0/4244605066 >> 172.21.15.26:0/4244605066 conn(0x7f232806a800 msgr2=0x7f232806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:27.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.156+0000 7f2330800640 1 -- 172.21.15.26:0/4244605066 shutdown_connections 2024-01-15T18:00:27.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.156+0000 7f2330800640 1 -- 172.21.15.26:0/4244605066 wait complete. 2024-01-15T18:00:27.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.157+0000 7f2330800640 1 Processor -- start 2024-01-15T18:00:27.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.157+0000 7f2330800640 1 -- start start 2024-01-15T18:00:27.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.158+0000 7f2330800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2308003f40 0x7f2328140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.158+0000 7f2330800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23280cdbb0 0x7f23280ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.158+0000 7f2330800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23281594d0 0x7f2328159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.158+0000 7f2330800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2328159e90 con 0x7f23281594d0 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.158+0000 7f2330800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280cc9f0 con 0x7f23280cdbb0 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.158+0000 7f2330800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23280cccd0 con 0x7f2308003f40 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23281594d0 0x7f2328159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23281594d0 0x7f2328159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53848/0 (socket says 172.21.15.26:53848) 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232e800640 1 -- 172.21.15.26:0/2272206489 learned_addr learned my addr 172.21.15.26:0/2272206489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232de00640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2308003f40 0x7f2328140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232d400640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23280cdbb0 0x7f23280ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232e800640 1 -- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2308003f40 msgr2=0x7f2328140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232e800640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2308003f40 0x7f2328140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232e800640 1 -- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23280cdbb0 msgr2=0x7f23280ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232e800640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23280cdbb0 0x7f23280ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232e800640 1 -- 172.21.15.26:0/2272206489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23280ccfb0 con 0x7f23281594d0 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.159+0000 7f232e800640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23281594d0 0x7f2328159950 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f23280cc3c0 tx=0x7f231800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.160+0000 7f2326000640 1 -- 172.21.15.26:0/2272206489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318016620 con 0x7f23281594d0 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.160+0000 7f2326000640 1 -- 172.21.15.26:0/2272206489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23180167a0 con 0x7f23281594d0 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.160+0000 7f2326000640 1 -- 172.21.15.26:0/2272206489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318016b50 con 0x7f23281594d0 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.160+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23280cd290 con 0x7f23281594d0 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.160+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23280cc610 con 0x7f23281594d0 2024-01-15T18:00:27.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.161+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23281f2e60 con 0x7f23281594d0 2024-01-15T18:00:27.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.162+0000 7f2326000640 1 -- 172.21.15.26:0/2272206489 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f2318016cb0 con 0x7f23281594d0 2024-01-15T18:00:27.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.163+0000 7f2326000640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f230006b860 0x7f230006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.163+0000 7f2326000640 1 -- 172.21.15.26:0/2272206489 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f231808b2d0 con 0x7f23281594d0 2024-01-15T18:00:27.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.163+0000 7f232de00640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f230006b860 0x7f230006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.164+0000 7f232de00640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f230006b860 0x7f230006dd20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2320000ff0 tx=0x7f23200016e0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:27.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.167+0000 7f2326000640 1 -- 172.21.15.26:0/2272206489 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23180594f0 con 0x7f23281594d0 2024-01-15T18:00:27.185 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:00:27.191 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:00:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.370+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23280675d0 con 0x7f23281594d0 2024-01-15T18:00:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.370+0000 7f2326000640 1 -- 172.21.15.26:0/2272206489 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v52) v1 ==== 80+0+122 (secure 0 0 0) 0x7f2318059080 con 0x7f23281594d0 2024-01-15T18:00:27.374 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6835/1948779240 2024-01-16T18:00:22.471234+0000 2024-01-15T18:00:27.374 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6833/1948779240 2024-01-16T18:00:22.471234+0000 2024-01-15T18:00:27.374 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:00:27.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f230006b860 msgr2=0x7f230006dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:27.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f230006b860 0x7f230006dd20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2320000ff0 tx=0x7f23200016e0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23281594d0 msgr2=0x7f2328159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:27.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23281594d0 0x7f2328159950 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f23280cc3c0 tx=0x7f231800f520 comp rx=0 tx=0).stop 2024-01-15T18:00:27.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 shutdown_connections 2024-01-15T18:00:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f230006b860 0x7f230006dd20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23281594d0 0x7f2328159950 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23280cdbb0 0x7f23280ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 --2- 172.21.15.26:0/2272206489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2308003f40 0x7f2328140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 >> 172.21.15.26:0/2272206489 conn(0x7f232806a800 msgr2=0x7f23280b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 shutdown_connections 2024-01-15T18:00:27.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.373+0000 7f2330800640 1 -- 172.21.15.26:0/2272206489 wait complete. 2024-01-15T18:00:27.394 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-01-15T18:00:27.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:27.394 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:00:27.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[52830]: fuse finished with error 0 and tester_r 0 2024-01-15T18:00:27.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:00:27.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:27.483 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:00:27.708 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:00:27.708 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:00:27.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:00:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.987+0000 7f5439000640 1 Processor -- start 2024-01-15T18:00:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.987+0000 7f5439000640 1 -- start start 2024-01-15T18:00:27.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.989+0000 7f5439000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5434162810 0x7f5434071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.989+0000 7f5439000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54340702e0 con 0x7f5434071780 2024-01-15T18:00:27.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.989+0000 7f5439000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54340705c0 con 0x7f5434162430 2024-01-15T18:00:27.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.989+0000 7f5439000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54340708a0 con 0x7f5434162810 2024-01-15T18:00:27.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.989+0000 7f5433e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5434071780 0x7f543406c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49770/0 (socket says 172.21.15.26:49770) 2024-01-15T18:00:27.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.989+0000 7f5433e00640 1 -- 172.21.15.26:0/1288570899 learned_addr learned my addr 172.21.15.26:0/1288570899 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f542be00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5434162810 0x7f5434071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257105111 0 0) 0x7f54340702e0 con 0x7f5434071780 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5414003520 con 0x7f5434071780 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3558870061 0 0) 0x7f5414003520 con 0x7f5434071780 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5434162810 msgr2=0x7f5434071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5434162810 0x7f5434071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 >> v1:172.21.15.124:6789/0 conn(0x7f5434162430 legacy=0x7f54340b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5434070b80 con 0x7f5434071780 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f542c0019f0 con 0x7f5434071780 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 >> v1:172.21.15.26:6789/0 conn(0x7f5434071780 legacy=0x7f543406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54140039a0 0x7f5414003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.990+0000 7f5432a00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54140047c0 0x7f5414004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5432a00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5414005140 0x7f541400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54340708a0 con 0x7f5414005140 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541400bec0 con 0x7f54140039a0 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541400c030 con 0x7f54140047c0 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5433e00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5414005140 0x7f541400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f542be00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54140047c0 0x7f5414004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5433400640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54140039a0 0x7f5414003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5433e00640 1 -- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54140047c0 msgr2=0x7f5414004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5433e00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54140047c0 0x7f5414004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5433e00640 1 -- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54140039a0 msgr2=0x7f5414003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5433e00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54140039a0 0x7f5414003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f5433e00640 1 -- 172.21.15.26:0/1288570899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414003520 con 0x7f5414005140 2024-01-15T18:00:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.991+0000 7f542be00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54140047c0 0x7f5414004c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.992+0000 7f5433e00640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5414005140 0x7f541400b980 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f542c00a760 tx=0x7f54140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfb8000c934e83d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c034630 con 0x7f5414005140 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c02f070 con 0x7f5414005140 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5432a00640 1 -- 172.21.15.26:0/1288570899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c034a10 con 0x7f5414005140 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5439000640 1 -- 172.21.15.26:0/1288570899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5414005140 msgr2=0x7f541400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5439000640 1 --2- 172.21.15.26:0/1288570899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5414005140 0x7f541400b980 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f542c00a760 tx=0x7f54140057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5433e00640 1 -- 172.21.15.26:0/1288570899 reap_dead start 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5439000640 1 -- 172.21.15.26:0/1288570899 shutdown_connections 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5439000640 1 -- 172.21.15.26:0/1288570899 >> 172.21.15.26:0/1288570899 conn(0x7f543406a800 msgr2=0x7f543406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5439000640 1 -- 172.21.15.26:0/1288570899 shutdown_connections 2024-01-15T18:00:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.993+0000 7f5439000640 1 -- 172.21.15.26:0/1288570899 wait complete. 2024-01-15T18:00:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.995+0000 7f5439000640 1 Processor -- start 2024-01-15T18:00:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.995+0000 7f5439000640 1 -- start start 2024-01-15T18:00:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.996+0000 7f5439000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5414005140 0x7f5434156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.996+0000 7f5439000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434160590 0x7f5434160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f5439000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54340a84c0 0x7f54340a8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f5439000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54340a8eb0 con 0x7f5434160590 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f5439000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5434160f50 con 0x7f54340a84c0 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f5439000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543415ff30 con 0x7f5414005140 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f542be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434160590 0x7f5434160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f542be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434160590 0x7f5434160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53870/0 (socket says 172.21.15.26:53870) 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f542be00640 1 -- 172.21.15.26:0/1571021973 learned_addr learned my addr 172.21.15.26:0/1571021973 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f5433400640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5414005140 0x7f5434156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f5433e00640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54340a84c0 0x7f54340a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f542be00640 1 -- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5414005140 msgr2=0x7f5434156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f542be00640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5414005140 0x7f5434156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f542be00640 1 -- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54340a84c0 msgr2=0x7f54340a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f542be00640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54340a84c0 0x7f54340a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.997+0000 7f542be00640 1 -- 172.21.15.26:0/1571021973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54340a90a0 con 0x7f5434160590 2024-01-15T18:00:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.998+0000 7f542be00640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434160590 0x7f5434160a10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f541c0116d0 tx=0x7f541c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.998+0000 7f542b400640 1 -- 172.21.15.26:0/1571021973 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c00f9c0 con 0x7f5434160590 2024-01-15T18:00:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.998+0000 7f542b400640 1 -- 172.21.15.26:0/1571021973 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c00fb20 con 0x7f5434160590 2024-01-15T18:00:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.998+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54340a9380 con 0x7f5434160590 2024-01-15T18:00:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.998+0000 7f542b400640 1 -- 172.21.15.26:0/1571021973 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c00fed0 con 0x7f5434160590 2024-01-15T18:00:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.998+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54340a9940 con 0x7f5434160590 2024-01-15T18:00:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:27.999+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5434067450 con 0x7f5434160590 2024-01-15T18:00:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.000+0000 7f542b400640 1 -- 172.21.15.26:0/1571021973 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f541c015460 con 0x7f5434160590 2024-01-15T18:00:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.001+0000 7f542b400640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f540c06b890 0x7f540c06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.001+0000 7f542b400640 1 -- 172.21.15.26:0/1571021973 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f541c08abe0 con 0x7f5434160590 2024-01-15T18:00:28.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.002+0000 7f5433400640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f540c06b890 0x7f540c06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.003+0000 7f5433400640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f540c06b890 0x7f540c06dd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5424007fd0 tx=0x7f5424009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:28.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.005+0000 7f542b400640 1 -- 172.21.15.26:0/1571021973 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f541c055480 con 0x7f5434160590 2024-01-15T18:00:28.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.200+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54340cdbf0 con 0x7f5434160590 2024-01-15T18:00:28.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.201+0000 7f542b400640 1 -- 172.21.15.26:0/1571021973 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+13005 (secure 0 0 0) 0x7f541c058aa0 con 0x7f5434160590 2024-01-15T18:00:28.205 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:28.205 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":52,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:00:25.482824+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1948779240":"2024-01-16T18:00:22.471234+0000","172.21.15.26:6833/1948779240":"2024-01-16T18:00:22.471234+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:00:28.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f540c06b890 msgr2=0x7f540c06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f540c06b890 0x7f540c06dd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5424007fd0 tx=0x7f5424009040 comp rx=0 tx=0).stop 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434160590 msgr2=0x7f5434160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434160590 0x7f5434160a10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f541c0116d0 tx=0x7f541c011b90 comp rx=0 tx=0).stop 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 shutdown_connections 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f540c06b890 0x7f540c06dd50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54340a84c0 0x7f54340a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5434160590 0x7f5434160a10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 --2- 172.21.15.26:0/1571021973 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5414005140 0x7f5434156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 >> 172.21.15.26:0/1571021973 conn(0x7f543406a800 msgr2=0x7f54340af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 shutdown_connections 2024-01-15T18:00:28.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.204+0000 7f5439000640 1 -- 172.21.15.26:0/1571021973 wait complete. 2024-01-15T18:00:28.225 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:00:28.226 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:00:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.448+0000 7fabc6400640 1 Processor -- start 2024-01-15T18:00:28.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.449+0000 7fabc6400640 1 -- start start 2024-01-15T18:00:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.450+0000 7fabc6400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc013fcb0 0x7fabc00cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.450+0000 7fabc6400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabc00cdc20 0x7fabc00b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.450+0000 7fabc6400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc00ce440 con 0x7fabc013cf10 2024-01-15T18:00:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.451+0000 7fabc6400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc00ce720 con 0x7fabc00cdc20 2024-01-15T18:00:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.451+0000 7fabc6400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc00cea00 con 0x7fabc013fcb0 2024-01-15T18:00:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.451+0000 7fabc5a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabc00cdc20 0x7fabc00b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.451+0000 7fabc5a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fabc00cdc20 0x7fabc00b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53886/0 (socket says 172.21.15.26:53886) 2024-01-15T18:00:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.451+0000 7fabc5a00640 1 -- 172.21.15.26:0/1485927813 learned_addr learned my addr 172.21.15.26:0/1485927813 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.451+0000 7fabb7e00640 1 --2- 172.21.15.26:0/1485927813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc013fcb0 0x7fabc00cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.452+0000 7fabc5a00640 1 -- 172.21.15.26:0/1485927813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc013fcb0 msgr2=0x7fabc00cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.452+0000 7fabc5a00640 1 --2- 172.21.15.26:0/1485927813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc013fcb0 0x7fabc00cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.452+0000 7fabc5a00640 1 -- 172.21.15.26:0/1485927813 >> v1:172.21.15.124:6789/0 conn(0x7fabc013cf10 legacy=0x7fabc013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.452+0000 7fabc5a00640 1 -- 172.21.15.26:0/1485927813 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc00b1d90 con 0x7fabc00cdc20 2024-01-15T18:00:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.453+0000 7fabc5a00640 1 --2- 172.21.15.26:0/1485927813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc00cdc20 0x7fabc00b2590 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009e50 tx=0x7fabb802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf22a985fcca77d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.453+0000 7fabbfe00640 1 -- 172.21.15.26:0/1485927813 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb802cb00 con 0x7fabc00cdc20 2024-01-15T18:00:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.453+0000 7fabbfe00640 1 -- 172.21.15.26:0/1485927813 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb802cc60 con 0x7fabc00cdc20 2024-01-15T18:00:28.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.453+0000 7fabbfe00640 1 -- 172.21.15.26:0/1485927813 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb80343f0 con 0x7fabc00cdc20 2024-01-15T18:00:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.454+0000 7fabc6400640 1 -- 172.21.15.26:0/1485927813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc00cdc20 msgr2=0x7fabc00b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.454+0000 7fabc6400640 1 --2- 172.21.15.26:0/1485927813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc00cdc20 0x7fabc00b2590 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009e50 tx=0x7fabb802bf60 comp rx=0 tx=0).stop 2024-01-15T18:00:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.454+0000 7fabc6400640 1 -- 172.21.15.26:0/1485927813 shutdown_connections 2024-01-15T18:00:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.454+0000 7fabc6400640 1 --2- 172.21.15.26:0/1485927813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc00cdc20 0x7fabc00b2590 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.454+0000 7fabc6400640 1 --2- 172.21.15.26:0/1485927813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc013fcb0 0x7fabc00cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.454+0000 7fabc6400640 1 -- 172.21.15.26:0/1485927813 >> 172.21.15.26:0/1485927813 conn(0x7fabc006a7d0 msgr2=0x7fabc006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.454+0000 7fabc6400640 1 -- 172.21.15.26:0/1485927813 shutdown_connections 2024-01-15T18:00:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.454+0000 7fabc6400640 1 -- 172.21.15.26:0/1485927813 wait complete. 2024-01-15T18:00:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.456+0000 7fabc6400640 1 Processor -- start 2024-01-15T18:00:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.456+0000 7fabc6400640 1 -- start start 2024-01-15T18:00:28.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.457+0000 7fabc6400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc00cdc20 0x7fabc00721f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.457+0000 7fabc6400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabc013cf10 0x7fabc0154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.457+0000 7fabc6400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc013fcb0 0x7fabc0151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.457+0000 7fabc6400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc013d2f0 con 0x7fabc013fcb0 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.457+0000 7fabc6400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc0157090 con 0x7fabc013cf10 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.457+0000 7fabc6400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc0157330 con 0x7fabc00cdc20 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc013fcb0 0x7fabc0151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc013fcb0 0x7fabc0151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53892/0 (socket says 172.21.15.26:53892) 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5a00640 1 -- 172.21.15.26:0/1312518026 learned_addr learned my addr 172.21.15.26:0/1312518026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabb7e00640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabc013cf10 0x7fabc0154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5000640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc00cdc20 0x7fabc00721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5a00640 1 -- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc00cdc20 msgr2=0x7fabc00721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5a00640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc00cdc20 0x7fabc00721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5a00640 1 -- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabc013cf10 msgr2=0x7fabc0154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5a00640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabc013cf10 0x7fabc0154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5a00640 1 -- 172.21.15.26:0/1312518026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb8009b30 con 0x7fabc013fcb0 2024-01-15T18:00:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.458+0000 7fabc5a00640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc013fcb0 0x7fabc0151760 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009e50 tx=0x7fabb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.459+0000 7fabbd600640 1 -- 172.21.15.26:0/1312518026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb8009510 con 0x7fabc013fcb0 2024-01-15T18:00:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.459+0000 7fabbd600640 1 -- 172.21.15.26:0/1312518026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb803e450 con 0x7fabc013fcb0 2024-01-15T18:00:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.459+0000 7fabbd600640 1 -- 172.21.15.26:0/1312518026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb803e800 con 0x7fabc013fcb0 2024-01-15T18:00:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.459+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc0157610 con 0x7fabc013fcb0 2024-01-15T18:00:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.459+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc0154fc0 con 0x7fabc013fcb0 2024-01-15T18:00:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.461+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc0067450 con 0x7fabc013fcb0 2024-01-15T18:00:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.461+0000 7fabbd600640 1 -- 172.21.15.26:0/1312518026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fabb803e960 con 0x7fabc013fcb0 2024-01-15T18:00:28.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.462+0000 7fabbd600640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9806b890 0x7fab9806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.462+0000 7fabbd600640 1 -- 172.21.15.26:0/1312518026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fabb80ad050 con 0x7fabc013fcb0 2024-01-15T18:00:28.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.463+0000 7fabc5000640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9806b890 0x7fab9806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.464+0000 7fabc5000640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9806b890 0x7fab9806dd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fabb0000f60 tx=0x7fabb0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:28.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.469+0000 7fabbd600640 1 -- 172.21.15.26:0/1312518026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fabb80778f0 con 0x7fabc013fcb0 2024-01-15T18:00:28.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.664+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabc00cbf10 con 0x7fabc013fcb0 2024-01-15T18:00:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.665+0000 7fabbd600640 1 -- 172.21.15.26:0/1312518026 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabb807af10 con 0x7fabc013fcb0 2024-01-15T18:00:28.669 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:00:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9806b890 msgr2=0x7fab9806dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9806b890 0x7fab9806dd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fabb0000f60 tx=0x7fabb0009040 comp rx=0 tx=0).stop 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc013fcb0 msgr2=0x7fabc0151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc013fcb0 0x7fabc0151760 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009e50 tx=0x7fabb8004a40 comp rx=0 tx=0).stop 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 shutdown_connections 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab9806b890 0x7fab9806dd50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabc013fcb0 0x7fabc0151760 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabc013cf10 0x7fabc0154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 --2- 172.21.15.26:0/1312518026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabc00cdc20 0x7fabc00721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 >> 172.21.15.26:0/1312518026 conn(0x7fabc006a7d0 msgr2=0x7fabc00af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 shutdown_connections 2024-01-15T18:00:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.668+0000 7fabc6400640 1 -- 172.21.15.26:0/1312518026 wait complete. 2024-01-15T18:00:28.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:00:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.909+0000 7f829ae00640 1 Processor -- start 2024-01-15T18:00:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.909+0000 7f829ae00640 1 -- start start 2024-01-15T18:00:28.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.910+0000 7f829ae00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f829415afb0 0x7f829413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.910+0000 7f829ae00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413fae0 0x7f82940b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.910+0000 7f829ae00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829413ff30 con 0x7f829413cfa0 2024-01-15T18:00:28.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.911+0000 7f829ae00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82940cdc40 con 0x7f829415afb0 2024-01-15T18:00:28.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.911+0000 7f829ae00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82940cdec0 con 0x7f829413fae0 2024-01-15T18:00:28.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.911+0000 7f8299000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f829415afb0 0x7f829413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.911+0000 7f8299000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f829415afb0 0x7f829413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:44494/0 (socket says 172.21.15.26:44494) 2024-01-15T18:00:28.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.911+0000 7f8299000640 1 -- 172.21.15.26:0/2985598561 learned_addr learned my addr 172.21.15.26:0/2985598561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:28.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.911+0000 7f829a400640 1 --2- 172.21.15.26:0/2985598561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413fae0 0x7f82940b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.912+0000 7f829a400640 1 -- 172.21.15.26:0/2985598561 >> v2:172.21.15.124:3300/0 conn(0x7f829415afb0 msgr2=0x7f829413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:28.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.912+0000 7f829a400640 1 --2- 172.21.15.26:0/2985598561 >> v2:172.21.15.124:3300/0 conn(0x7f829415afb0 0x7f829413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.912+0000 7f829a400640 1 -- 172.21.15.26:0/2985598561 >> v1:172.21.15.124:6789/0 conn(0x7f829413cfa0 legacy=0x7f82940cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.912+0000 7f829a400640 1 -- 172.21.15.26:0/2985598561 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82940ce1a0 con 0x7f829413fae0 2024-01-15T18:00:28.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.913+0000 7f829a400640 1 --2- 172.21.15.26:0/2985598561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413fae0 0x7f82940b2040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f828c0097a0 tx=0x7f828c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c808c3604e8887f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.914+0000 7f8293e00640 1 -- 172.21.15.26:0/2985598561 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c02e070 con 0x7f829413fae0 2024-01-15T18:00:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.914+0000 7f8293e00640 1 -- 172.21.15.26:0/2985598561 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c0315a0 con 0x7f829413fae0 2024-01-15T18:00:28.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.914+0000 7f8293e00640 1 -- 172.21.15.26:0/2985598561 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c031930 con 0x7f829413fae0 2024-01-15T18:00:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.914+0000 7f829ae00640 1 -- 172.21.15.26:0/2985598561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413fae0 msgr2=0x7f82940b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.914+0000 7f829ae00640 1 --2- 172.21.15.26:0/2985598561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413fae0 0x7f82940b2040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f828c0097a0 tx=0x7f828c02b840 comp rx=0 tx=0).stop 2024-01-15T18:00:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.915+0000 7f829ae00640 1 -- 172.21.15.26:0/2985598561 shutdown_connections 2024-01-15T18:00:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.915+0000 7f829ae00640 1 --2- 172.21.15.26:0/2985598561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413fae0 0x7f82940b2040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.915+0000 7f829ae00640 1 --2- 172.21.15.26:0/2985598561 >> v2:172.21.15.124:3300/0 conn(0x7f829415afb0 0x7f829413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.915+0000 7f829ae00640 1 -- 172.21.15.26:0/2985598561 >> 172.21.15.26:0/2985598561 conn(0x7f829406a800 msgr2=0x7f829406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.915+0000 7f829ae00640 1 -- 172.21.15.26:0/2985598561 shutdown_connections 2024-01-15T18:00:28.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.915+0000 7f829ae00640 1 -- 172.21.15.26:0/2985598561 wait complete. 2024-01-15T18:00:28.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.917+0000 7f829ae00640 1 Processor -- start 2024-01-15T18:00:28.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.917+0000 7f829ae00640 1 -- start start 2024-01-15T18:00:28.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f829ae00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413cfa0 0x7f82941506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f829ae00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829413fae0 0x7f8294155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f829ae00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f829415afb0 0x7f8294155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f829ae00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829413f000 con 0x7f829413fae0 2024-01-15T18:00:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f829ae00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8294150c20 con 0x7f829415afb0 2024-01-15T18:00:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f829ae00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8294154db0 con 0x7f829413cfa0 2024-01-15T18:00:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f8299000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829413fae0 0x7f8294155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f8299000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829413fae0 0x7f8294155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53902/0 (socket says 172.21.15.26:53902) 2024-01-15T18:00:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f8299000640 1 -- 172.21.15.26:0/2353213939 learned_addr learned my addr 172.21.15.26:0/2353213939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:28.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.918+0000 7f8299a00640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413cfa0 0x7f82941506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.919+0000 7f829a400640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f829415afb0 0x7f8294155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.919+0000 7f8299000640 1 -- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413cfa0 msgr2=0x7f82941506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.919+0000 7f8299000640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413cfa0 0x7f82941506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.919+0000 7f8299000640 1 -- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f829415afb0 msgr2=0x7f8294155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.919+0000 7f8299000640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f829415afb0 0x7f8294155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.919+0000 7f8299000640 1 -- 172.21.15.26:0/2353213939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c009400 con 0x7f829413fae0 2024-01-15T18:00:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.919+0000 7f8299000640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829413fae0 0x7f8294155590 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f827c00e840 tx=0x7f827c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.920+0000 7f8291600640 1 -- 172.21.15.26:0/2353213939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00c8e0 con 0x7f829413fae0 2024-01-15T18:00:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.920+0000 7f8291600640 1 -- 172.21.15.26:0/2353213939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c00ca40 con 0x7f829413fae0 2024-01-15T18:00:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.920+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829413f420 con 0x7f829413fae0 2024-01-15T18:00:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.920+0000 7f8291600640 1 -- 172.21.15.26:0/2353213939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c00cdf0 con 0x7f829413fae0 2024-01-15T18:00:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.920+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829413e660 con 0x7f829413fae0 2024-01-15T18:00:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.921+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8294067450 con 0x7f829413fae0 2024-01-15T18:00:28.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.922+0000 7f8291600640 1 -- 172.21.15.26:0/2353213939 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f827c013470 con 0x7f829413fae0 2024-01-15T18:00:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.923+0000 7f8291600640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f826c06b860 0x7f826c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.923+0000 7f8291600640 1 -- 172.21.15.26:0/2353213939 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f827c08a510 con 0x7f829413fae0 2024-01-15T18:00:28.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.924+0000 7f8299a00640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f826c06b860 0x7f826c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.925+0000 7f8299a00640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f826c06b860 0x7f826c06dd20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f82840012e0 tx=0x7f8284009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:28.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:28.927+0000 7f8291600640 1 -- 172.21.15.26:0/2353213939 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f827c054d50 con 0x7f829413fae0 2024-01-15T18:00:29.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.134+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f829406d7c0 con 0x7f829413fae0 2024-01-15T18:00:29.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.135+0000 7f8291600640 1 -- 172.21.15.26:0/2353213939 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v52) v1 ==== 80+0+122 (secure 0 0 0) 0x7f827c058370 con 0x7f829413fae0 2024-01-15T18:00:29.138 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6835/1948779240 2024-01-16T18:00:22.471234+0000 2024-01-15T18:00:29.138 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6833/1948779240 2024-01-16T18:00:22.471234+0000 2024-01-15T18:00:29.139 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:00:29.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.137+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f826c06b860 msgr2=0x7f826c06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:29.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.137+0000 7f829ae00640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f826c06b860 0x7f826c06dd20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f82840012e0 tx=0x7f8284009040 comp rx=0 tx=0).stop 2024-01-15T18:00:29.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.137+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829413fae0 msgr2=0x7f8294155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:29.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.137+0000 7f829ae00640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829413fae0 0x7f8294155590 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f827c00e840 tx=0x7f827c00ed00 comp rx=0 tx=0).stop 2024-01-15T18:00:29.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.138+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 shutdown_connections 2024-01-15T18:00:29.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.138+0000 7f829ae00640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f826c06b860 0x7f826c06dd20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.138+0000 7f829ae00640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f829415afb0 0x7f8294155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.138+0000 7f829ae00640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f829413fae0 0x7f8294155590 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.138+0000 7f829ae00640 1 --2- 172.21.15.26:0/2353213939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f829413cfa0 0x7f82941506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.138+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 >> 172.21.15.26:0/2353213939 conn(0x7f829406a800 msgr2=0x7f829406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.138+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 shutdown_connections 2024-01-15T18:00:29.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.138+0000 7f829ae00640 1 -- 172.21.15.26:0/2353213939 wait complete. 2024-01-15T18:00:29.161 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi124.front.sepia.ceph.com... 2024-01-15T18:00:29.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:29.161 DEBUG:teuthology.orchestra.run.smithi124:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:00:29.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[43918]: fuse finished with error 0 and tester_r 0 2024-01-15T18:00:29.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:00:29.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:29.254 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:00:29.435 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:00:29.435 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:00:29.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:00:29.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:00:29.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:29.494 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:00:29.548 INFO:teuthology.orchestra.run.smithi026.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:00:29.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:00:29.549 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:00:29.549 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:00:29.603 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:00:29.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:00:29.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:29.603 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:00:29.651 INFO:teuthology.orchestra.run.smithi124.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:00:29.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:00:29.653 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:00:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.877+0000 7f3d38a00640 1 Processor -- start 2024-01-15T18:00:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.878+0000 7f3d38a00640 1 -- start start 2024-01-15T18:00:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.879+0000 7f3d38a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3d34162430 0x7f3d340b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.879+0000 7f3d38a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d340702e0 con 0x7f3d34162810 2024-01-15T18:00:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.879+0000 7f3d38a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d340705c0 con 0x7f3d340b3650 2024-01-15T18:00:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.879+0000 7f3d38a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d340708a0 con 0x7f3d34162430 2024-01-15T18:00:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.880+0000 7f3d2a800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3d34162810 0x7f3d34071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49786/0 (socket says 172.21.15.26:49786) 2024-01-15T18:00:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.880+0000 7f3d2a800640 1 -- 172.21.15.26:0/1982988477 learned_addr learned my addr 172.21.15.26:0/1982988477 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.880+0000 7f3d33400640 1 --2- 172.21.15.26:0/1982988477 >> v2:172.21.15.124:3300/0 conn(0x7f3d34162430 0x7f3d340b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.880+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125681855 0 0) 0x7f3d340702e0 con 0x7f3d34162810 2024-01-15T18:00:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.880+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d14003520 con 0x7f3d34162810 2024-01-15T18:00:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4260649068 0 0) 0x7f3d14003520 con 0x7f3d34162810 2024-01-15T18:00:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 >> v2:172.21.15.124:3300/0 conn(0x7f3d34162430 msgr2=0x7f3d340b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 --2- 172.21.15.26:0/1982988477 >> v2:172.21.15.124:3300/0 conn(0x7f3d34162430 0x7f3d340b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 >> v1:172.21.15.124:6789/0 conn(0x7f3d340b3650 legacy=0x7f3d3406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d34070b80 con 0x7f3d34162810 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f3d1c001670 con 0x7f3d34162810 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 >> v1:172.21.15.26:6789/0 conn(0x7f3d34162810 legacy=0x7f3d34071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 --2- 172.21.15.26:0/1982988477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d140039c0 0x7f3d14003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 --2- 172.21.15.26:0/1982988477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d140047c0 0x7f3d14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 --2- 172.21.15.26:0/1982988477 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d14005180 0x7f3d1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340708a0 con 0x7f3d140039c0 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1400c080 con 0x7f3d14005180 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1400c240 con 0x7f3d140047c0 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.881+0000 7f3d33400640 1 --2- 172.21.15.26:0/1982988477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d140039c0 0x7f3d14003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.882+0000 7f3d33400640 1 -- 172.21.15.26:0/1982988477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d140047c0 msgr2=0x7f3d14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.882+0000 7f3d33400640 1 --2- 172.21.15.26:0/1982988477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d140047c0 0x7f3d14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.882+0000 7f3d33400640 1 -- 172.21.15.26:0/1982988477 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d14005180 msgr2=0x7f3d1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.882+0000 7f3d33400640 1 --2- 172.21.15.26:0/1982988477 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d14005180 0x7f3d1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.882+0000 7f3d33400640 1 -- 172.21.15.26:0/1982988477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d14003520 con 0x7f3d140039c0 2024-01-15T18:00:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.882+0000 7f3d33e00640 1 -- 172.21.15.26:0/1982988477 reap_dead start 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.883+0000 7f3d33400640 1 --2- 172.21.15.26:0/1982988477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d140039c0 0x7f3d14003de0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009ac0 tx=0x7f3d140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcc93574c8acb33 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.883+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2402c710 con 0x7f3d140039c0 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.883+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2402c870 con 0x7f3d140039c0 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.883+0000 7f3d32a00640 1 -- 172.21.15.26:0/1982988477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2402cc00 con 0x7f3d140039c0 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.884+0000 7f3d38a00640 1 -- 172.21.15.26:0/1982988477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d140039c0 msgr2=0x7f3d14003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.884+0000 7f3d38a00640 1 --2- 172.21.15.26:0/1982988477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d140039c0 0x7f3d14003de0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009ac0 tx=0x7f3d140057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.884+0000 7f3d38a00640 1 -- 172.21.15.26:0/1982988477 shutdown_connections 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.884+0000 7f3d38a00640 1 --2- 172.21.15.26:0/1982988477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d140039c0 0x7f3d14003de0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.884+0000 7f3d38a00640 1 -- 172.21.15.26:0/1982988477 >> 172.21.15.26:0/1982988477 conn(0x7f3d3406a800 msgr2=0x7f3d3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.884+0000 7f3d38a00640 1 -- 172.21.15.26:0/1982988477 shutdown_connections 2024-01-15T18:00:29.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.884+0000 7f3d38a00640 1 -- 172.21.15.26:0/1982988477 wait complete. 2024-01-15T18:00:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.886+0000 7f3d38a00640 1 Processor -- start 2024-01-15T18:00:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.886+0000 7f3d38a00640 1 -- start start 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d38a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d140039c0 0x7f3d340aa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d38a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d340ad6b0 0x7f3d341f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d38a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d340adb30 0x7f3d341fa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d38a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d341fa890 con 0x7f3d340adb30 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d38a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d341f8d70 con 0x7f3d140039c0 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d38a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d341f9050 con 0x7f3d340ad6b0 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d33e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d340adb30 0x7f3d341fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d33e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d340adb30 0x7f3d341fa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53930/0 (socket says 172.21.15.26:53930) 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d33e00640 1 -- 172.21.15.26:0/3304958667 learned_addr learned my addr 172.21.15.26:0/3304958667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d33400640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d140039c0 0x7f3d340aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.887+0000 7f3d2a800640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d340ad6b0 0x7f3d341f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d33e00640 1 -- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d340ad6b0 msgr2=0x7f3d341f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d33e00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d340ad6b0 0x7f3d341f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d33e00640 1 -- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d140039c0 msgr2=0x7f3d340aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d33e00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d140039c0 0x7f3d340aa860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d33e00640 1 -- 172.21.15.26:0/3304958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d341f9330 con 0x7f3d340adb30 2024-01-15T18:00:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d33e00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d340adb30 0x7f3d341fa350 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3d340726f0 tx=0x7f3d2c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d2be00640 1 -- 172.21.15.26:0/3304958667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0070d0 con 0x7f3d340adb30 2024-01-15T18:00:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d2be00640 1 -- 172.21.15.26:0/3304958667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c007250 con 0x7f3d340adb30 2024-01-15T18:00:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d2be00640 1 -- 172.21.15.26:0/3304958667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c007600 con 0x7f3d340adb30 2024-01-15T18:00:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d341f9610 con 0x7f3d340adb30 2024-01-15T18:00:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.888+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d340aada0 con 0x7f3d340adb30 2024-01-15T18:00:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.890+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d341f9b30 con 0x7f3d340adb30 2024-01-15T18:00:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.891+0000 7f3d2be00640 1 -- 172.21.15.26:0/3304958667 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f3d2c007760 con 0x7f3d340adb30 2024-01-15T18:00:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.892+0000 7f3d2be00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d0c06b890 0x7f3d0c06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.892+0000 7f3d2be00640 1 -- 172.21.15.26:0/3304958667 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f3d2c0898a0 con 0x7f3d340adb30 2024-01-15T18:00:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.892+0000 7f3d33400640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d0c06b890 0x7f3d0c06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:29.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.894+0000 7f3d33400640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d0c06b890 0x7f3d0c06dd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d24036000 tx=0x7f3d24030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:29.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:29.896+0000 7f3d2be00640 1 -- 172.21.15.26:0/3304958667 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d341f9b30 con 0x7f3d340adb30 2024-01-15T18:00:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.132+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d341f9b30 con 0x7f3d340adb30 2024-01-15T18:00:30.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.133+0000 7f3d2be00640 1 -- 172.21.15.26:0/3304958667 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f3d341f9b30 con 0x7f3d340adb30 2024-01-15T18:00:30.137 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:30.137 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5218,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3293492366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3293492366},{"type":"v1","addr":"172.21.15.124:6836","nonce":3293492366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1049920857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":1049920857},{"type":"v1","addr":"172.21.15.124:6837","nonce":1049920857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2644667232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2644667232},{"type":"v1","addr":"172.21.15.26:6834","nonce":2644667232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4255602003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4255602003},{"type":"v1","addr":"172.21.15.26:6835","nonce":4255602003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2024-01-15T18:00:30.137 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-01-15T18:00:30.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d0c06b890 msgr2=0x7f3d0c06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d0c06b890 0x7f3d0c06dd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d24036000 tx=0x7f3d24030000 comp rx=0 tx=0).stop 2024-01-15T18:00:30.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d340adb30 msgr2=0x7f3d341fa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d340adb30 0x7f3d341fa350 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3d340726f0 tx=0x7f3d2c00c960 comp rx=0 tx=0).stop 2024-01-15T18:00:30.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 shutdown_connections 2024-01-15T18:00:30.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d340ad6b0 0x7f3d341f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d0c06b890 0x7f3d0c06dd50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d340adb30 0x7f3d341fa350 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 --2- 172.21.15.26:0/3304958667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d140039c0 0x7f3d340aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 >> 172.21.15.26:0/3304958667 conn(0x7f3d3406a800 msgr2=0x7f3d3406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 shutdown_connections 2024-01-15T18:00:30.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.136+0000 7f3d38a00640 1 -- 172.21.15.26:0/3304958667 wait complete. 2024-01-15T18:00:30.160 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:00:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.376+0000 7f3622e00640 1 Processor -- start 2024-01-15T18:00:30.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.376+0000 7f3622e00640 1 -- start start 2024-01-15T18:00:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3622e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f361c0adf20 0x7f361c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3622e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f361c15bf10 0x7f361c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3622e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f361c0cc980 0x7f361c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3622e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c15f970 con 0x7f361c15bf10 2024-01-15T18:00:30.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3622e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c15f4e0 con 0x7f361c0cc980 2024-01-15T18:00:30.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3622e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0cce10 con 0x7f361c0adf20 2024-01-15T18:00:30.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3621000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f361c15bf10 0x7f361c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3621000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f361c15bf10 0x7f361c0cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53940/0 (socket says 172.21.15.26:53940) 2024-01-15T18:00:30.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3621000640 1 -- 172.21.15.26:0/930155587 learned_addr learned my addr 172.21.15.26:0/930155587 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:30.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3621a00640 1 --2- 172.21.15.26:0/930155587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f361c0adf20 0x7f361c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.378+0000 7f3622400640 1 --2- 172.21.15.26:0/930155587 >> v2:172.21.15.124:3300/0 conn(0x7f361c0cc980 0x7f361c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.379+0000 7f3621000640 1 -- 172.21.15.26:0/930155587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f361c0adf20 msgr2=0x7f361c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.379+0000 7f3621000640 1 --2- 172.21.15.26:0/930155587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f361c0adf20 0x7f361c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.379+0000 7f3621000640 1 -- 172.21.15.26:0/930155587 >> v2:172.21.15.124:3300/0 conn(0x7f361c0cc980 msgr2=0x7f361c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.379+0000 7f3621000640 1 --2- 172.21.15.26:0/930155587 >> v2:172.21.15.124:3300/0 conn(0x7f361c0cc980 0x7f361c143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.379+0000 7f3621000640 1 -- 172.21.15.26:0/930155587 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c071130 con 0x7f361c15bf10 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.380+0000 7f3621000640 1 --2- 172.21.15.26:0/930155587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c15bf10 0x7f361c0cc440 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f361c06e490 tx=0x7f360402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e49b385582d9296 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.380+0000 7f361be00640 1 -- 172.21.15.26:0/930155587 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36040317e0 con 0x7f361c15bf10 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.380+0000 7f361be00640 1 -- 172.21.15.26:0/930155587 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3604031940 con 0x7f361c15bf10 2024-01-15T18:00:30.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.380+0000 7f361be00640 1 -- 172.21.15.26:0/930155587 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604031cd0 con 0x7f361c15bf10 2024-01-15T18:00:30.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.381+0000 7f3622e00640 1 -- 172.21.15.26:0/930155587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c15bf10 msgr2=0x7f361c0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.381+0000 7f3622e00640 1 --2- 172.21.15.26:0/930155587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c15bf10 0x7f361c0cc440 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f361c06e490 tx=0x7f360402b880 comp rx=0 tx=0).stop 2024-01-15T18:00:30.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.381+0000 7f3622e00640 1 -- 172.21.15.26:0/930155587 shutdown_connections 2024-01-15T18:00:30.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.381+0000 7f3622e00640 1 --2- 172.21.15.26:0/930155587 >> v2:172.21.15.124:3300/0 conn(0x7f361c0cc980 0x7f361c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.381+0000 7f3622e00640 1 --2- 172.21.15.26:0/930155587 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c15bf10 0x7f361c0cc440 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.381+0000 7f3622e00640 1 --2- 172.21.15.26:0/930155587 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f361c0adf20 0x7f361c070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.381+0000 7f3622e00640 1 -- 172.21.15.26:0/930155587 >> 172.21.15.26:0/930155587 conn(0x7f361c06a7d0 msgr2=0x7f361c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:30.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.381+0000 7f3622e00640 1 -- 172.21.15.26:0/930155587 shutdown_connections 2024-01-15T18:00:30.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.381+0000 7f3622e00640 1 -- 172.21.15.26:0/930155587 wait complete. 2024-01-15T18:00:30.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.383+0000 7f3622e00640 1 Processor -- start 2024-01-15T18:00:30.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.383+0000 7f3622e00640 1 -- start start 2024-01-15T18:00:30.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.384+0000 7f3622e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f361c0adf20 0x7f361c13b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.384+0000 7f3622e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c0cc980 0x7f361c13f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.384+0000 7f3622e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f361c15bf10 0x7f361c071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.384+0000 7f3622e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c141a10 con 0x7f361c0cc980 2024-01-15T18:00:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.384+0000 7f3622e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c141cf0 con 0x7f361c15bf10 2024-01-15T18:00:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.384+0000 7f3622e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c141fd0 con 0x7f361c0adf20 2024-01-15T18:00:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.384+0000 7f3621000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c0cc980 0x7f361c13f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.384+0000 7f3621000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c0cc980 0x7f361c13f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53948/0 (socket says 172.21.15.26:53948) 2024-01-15T18:00:30.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.384+0000 7f3621000640 1 -- 172.21.15.26:0/3736254964 learned_addr learned my addr 172.21.15.26:0/3736254964 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3621000640 1 -- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f361c0adf20 msgr2=0x7f361c13b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3621000640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f361c0adf20 0x7f361c13b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3622400640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f361c15bf10 0x7f361c071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3621000640 1 -- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f361c15bf10 msgr2=0x7f361c071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3621000640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f361c15bf10 0x7f361c071b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3621000640 1 -- 172.21.15.26:0/3736254964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36040094d0 con 0x7f361c0cc980 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3621000640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c0cc980 0x7f361c13f9a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3604000c00 tx=0x7f36040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3619600640 1 -- 172.21.15.26:0/3736254964 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604032880 con 0x7f361c0cc980 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3619600640 1 -- 172.21.15.26:0/3736254964 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36040329e0 con 0x7f361c0cc980 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3619600640 1 -- 172.21.15.26:0/3736254964 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3604032d90 con 0x7f361c0cc980 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c1422b0 con 0x7f361c0cc980 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.385+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c141630 con 0x7f361c0cc980 2024-01-15T18:00:30.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.387+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c1424f0 con 0x7f361c0cc980 2024-01-15T18:00:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.388+0000 7f3619600640 1 -- 172.21.15.26:0/3736254964 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f360404e450 con 0x7f361c0cc980 2024-01-15T18:00:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.389+0000 7f3619600640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f35f406b860 0x7f35f406dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.389+0000 7f3619600640 1 -- 172.21.15.26:0/3736254964 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f36040ac720 con 0x7f361c0cc980 2024-01-15T18:00:30.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.390+0000 7f3621a00640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f35f406b860 0x7f35f406dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.391+0000 7f3621a00640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f35f406b860 0x7f35f406dd20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f360c00b7f0 tx=0x7f360c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:30.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.394+0000 7f3619600640 1 -- 172.21.15.26:0/3736254964 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f361c1424f0 con 0x7f361c0cc980 2024-01-15T18:00:30.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.588+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f361c1424f0 con 0x7f361c0cc980 2024-01-15T18:00:30.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.595+0000 7f3619600640 1 -- 172.21.15.26:0/3736254964 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f361c1424f0 con 0x7f361c0cc980 2024-01-15T18:00:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f35f406b860 msgr2=0x7f35f406dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f35f406b860 0x7f35f406dd20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f360c00b7f0 tx=0x7f360c008040 comp rx=0 tx=0).stop 2024-01-15T18:00:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c0cc980 msgr2=0x7f361c13f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c0cc980 0x7f361c13f9a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3604000c00 tx=0x7f36040046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 shutdown_connections 2024-01-15T18:00:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f35f406b860 0x7f35f406dd20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f361c15bf10 0x7f361c071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f361c0cc980 0x7f361c13f9a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 --2- 172.21.15.26:0/3736254964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f361c0adf20 0x7f361c13b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.598+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 >> 172.21.15.26:0/3736254964 conn(0x7f361c06a7d0 msgr2=0x7f361c15b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.599+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 shutdown_connections 2024-01-15T18:00:30.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.599+0000 7f3622e00640 1 -- 172.21.15.26:0/3736254964 wait complete. 2024-01-15T18:00:30.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_delay 2024-01-15T18:00:30.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.842+0000 7fba2b000640 1 Processor -- start 2024-01-15T18:00:30.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.842+0000 7fba2b000640 1 -- start start 2024-01-15T18:00:30.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.844+0000 7fba2b000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba240ccb50 0x7fba24070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.844+0000 7fba2b000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2415f4e0 con 0x7fba240adf20 2024-01-15T18:00:30.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.844+0000 7fba2b000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba24071130 con 0x7fba2415bf10 2024-01-15T18:00:30.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.844+0000 7fba2b000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2415f970 con 0x7fba240ccb50 2024-01-15T18:00:30.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.845+0000 7fba29c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fba240adf20 0x7fba240cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49794/0 (socket says 172.21.15.26:49794) 2024-01-15T18:00:30.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.845+0000 7fba29c00640 1 -- 172.21.15.26:0/1059231952 learned_addr learned my addr 172.21.15.26:0/1059231952 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:30.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.845+0000 7fba29200640 1 --2- 172.21.15.26:0/1059231952 >> v2:172.21.15.26:3300/0 conn(0x7fba240ccb50 0x7fba24070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.845+0000 7fba23e00640 1 -- 172.21.15.26:0/1059231952 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589068874 0 0) 0x7fba2415f4e0 con 0x7fba240adf20 2024-01-15T18:00:30.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.845+0000 7fba23e00640 1 -- 172.21.15.26:0/1059231952 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba04003520 con 0x7fba240adf20 2024-01-15T18:00:30.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.845+0000 7fba29200640 1 -- 172.21.15.26:0/1059231952 >> v1:172.21.15.124:6789/0 conn(0x7fba2415bf10 legacy=0x7fba24143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.845+0000 7fba29200640 1 -- 172.21.15.26:0/1059231952 >> v1:172.21.15.26:6789/0 conn(0x7fba240adf20 legacy=0x7fba240cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.845+0000 7fba29200640 1 -- 172.21.15.26:0/1059231952 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2415ebb0 con 0x7fba240ccb50 2024-01-15T18:00:30.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.847+0000 7fba29200640 1 --2- 172.21.15.26:0/1059231952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240ccb50 0x7fba24070a30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0097a0 tx=0x7fba0c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3efe393a4fb975 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:30.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.847+0000 7fba23e00640 1 -- 172.21.15.26:0/1059231952 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c031830 con 0x7fba240ccb50 2024-01-15T18:00:30.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.847+0000 7fba23e00640 1 -- 172.21.15.26:0/1059231952 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c031990 con 0x7fba240ccb50 2024-01-15T18:00:30.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.847+0000 7fba23e00640 1 -- 172.21.15.26:0/1059231952 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c031d20 con 0x7fba240ccb50 2024-01-15T18:00:30.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.847+0000 7fba2b000640 1 -- 172.21.15.26:0/1059231952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240ccb50 msgr2=0x7fba24070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.848+0000 7fba2b000640 1 --2- 172.21.15.26:0/1059231952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240ccb50 0x7fba24070a30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0097a0 tx=0x7fba0c02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.848+0000 7fba2b000640 1 -- 172.21.15.26:0/1059231952 shutdown_connections 2024-01-15T18:00:30.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.848+0000 7fba2b000640 1 --2- 172.21.15.26:0/1059231952 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240ccb50 0x7fba24070a30 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.848+0000 7fba2b000640 1 -- 172.21.15.26:0/1059231952 >> 172.21.15.26:0/1059231952 conn(0x7fba2406a7d0 msgr2=0x7fba2406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:30.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.848+0000 7fba2b000640 1 -- 172.21.15.26:0/1059231952 shutdown_connections 2024-01-15T18:00:30.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.848+0000 7fba2b000640 1 -- 172.21.15.26:0/1059231952 wait complete. 2024-01-15T18:00:30.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.850+0000 7fba2b000640 1 Processor -- start 2024-01-15T18:00:30.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.850+0000 7fba2b000640 1 -- start start 2024-01-15T18:00:30.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.851+0000 7fba2b000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240adf20 0x7fba241f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.851+0000 7fba2b000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba240ccb50 0x7fba240c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.851+0000 7fba2b000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba2415bf10 0x7fba2415c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.851+0000 7fba2b000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba240c4eb0 con 0x7fba240adf20 2024-01-15T18:00:30.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.851+0000 7fba2b000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba240c4c50 con 0x7fba240ccb50 2024-01-15T18:00:30.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.851+0000 7fba2b000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba241f1a50 con 0x7fba2415bf10 2024-01-15T18:00:30.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.851+0000 7fba29c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240adf20 0x7fba241f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.851+0000 7fba29c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240adf20 0x7fba241f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53962/0 (socket says 172.21.15.26:53962) 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.851+0000 7fba29c00640 1 -- 172.21.15.26:0/1844738550 learned_addr learned my addr 172.21.15.26:0/1844738550 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba29200640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba240ccb50 0x7fba240c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba29c00640 1 -- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba2415bf10 msgr2=0x7fba2415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba29c00640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba2415bf10 0x7fba2415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba29c00640 1 -- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba240ccb50 msgr2=0x7fba240c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba29c00640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba240ccb50 0x7fba240c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba29c00640 1 -- 172.21.15.26:0/1844738550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c009400 con 0x7fba240adf20 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba29200640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba240ccb50 0x7fba240c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:30.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba29c00640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240adf20 0x7fba241f4810 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c410 tx=0x7fba1400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:30.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba21600640 1 -- 172.21.15.26:0/1844738550 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14005ce0 con 0x7fba240adf20 2024-01-15T18:00:30.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba21600640 1 -- 172.21.15.26:0/1844738550 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba14005e40 con 0x7fba240adf20 2024-01-15T18:00:30.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba21600640 1 -- 172.21.15.26:0/1844738550 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba140072f0 con 0x7fba240adf20 2024-01-15T18:00:30.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.852+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba241f1d30 con 0x7fba240adf20 2024-01-15T18:00:30.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.853+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba241f22f0 con 0x7fba240adf20 2024-01-15T18:00:30.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.854+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba241f24e0 con 0x7fba240adf20 2024-01-15T18:00:30.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.855+0000 7fba21600640 1 -- 172.21.15.26:0/1844738550 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fba14007510 con 0x7fba240adf20 2024-01-15T18:00:30.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.856+0000 7fba21600640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9fc06b860 0x7fb9fc06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:30.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.856+0000 7fba21600640 1 -- 172.21.15.26:0/1844738550 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fba14089a50 con 0x7fba240adf20 2024-01-15T18:00:30.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.856+0000 7fba29200640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9fc06b860 0x7fb9fc06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.858+0000 7fba29200640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9fc06b860 0x7fb9fc06dd20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fba0c036000 tx=0x7fba0c03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:30.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:30.860+0000 7fba21600640 1 -- 172.21.15.26:0/1844738550 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba140543b0 con 0x7fba240adf20 2024-01-15T18:00:31.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.057+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"} v 0) v1 -- 0x7fba24067450 con 0x7fba240adf20 2024-01-15T18:00:31.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.064+0000 7fba21600640 1 -- 172.21.15.26:0/1844738550 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"}]=0 v18) v1 ==== 115+0+0 (secure 0 0 0) 0x7fba140579d0 con 0x7fba240adf20 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9fc06b860 msgr2=0x7fb9fc06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9fc06b860 0x7fb9fc06dd20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fba0c036000 tx=0x7fba0c03c040 comp rx=0 tx=0).stop 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240adf20 msgr2=0x7fba241f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240adf20 0x7fba241f4810 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fba1400c410 tx=0x7fba1400c8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 shutdown_connections 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba2415bf10 0x7fba2415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9fc06b860 0x7fb9fc06dd20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba240ccb50 0x7fba240c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 --2- 172.21.15.26:0/1844738550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba240adf20 0x7fba241f4810 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.067+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 >> 172.21.15.26:0/1844738550 conn(0x7fba2406a7d0 msgr2=0x7fba240cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:31.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.068+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 shutdown_connections 2024-01-15T18:00:31.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.068+0000 7fba2b000640 1 -- 172.21.15.26:0/1844738550 wait complete. 2024-01-15T18:00:31.091 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:00:31.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.309+0000 7f5879200640 1 Processor -- start 2024-01-15T18:00:31.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.310+0000 7f5879200640 1 -- start start 2024-01-15T18:00:31.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.311+0000 7f5879200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f58740cfde0 0x7f58740b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.311+0000 7f5879200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58740d1c60 con 0x7f58740cf9a0 2024-01-15T18:00:31.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.311+0000 7f5879200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58740b2730 con 0x7f587415d4b0 2024-01-15T18:00:31.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.311+0000 7f5879200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58740d1ec0 con 0x7f58740cfde0 2024-01-15T18:00:31.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.312+0000 7f5872a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f58740cf9a0 0x7f58740b31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49798/0 (socket says 172.21.15.26:49798) 2024-01-15T18:00:31.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.312+0000 7f5872a00640 1 -- 172.21.15.26:0/1494037003 learned_addr learned my addr 172.21.15.26:0/1494037003 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:31.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.312+0000 7f5873e00640 1 --2- 172.21.15.26:0/1494037003 >> v2:172.21.15.124:3300/0 conn(0x7f58740cfde0 0x7f58740b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.312+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529719457 0 0) 0x7f58740d1c60 con 0x7f58740cf9a0 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.312+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5854003520 con 0x7f58740cf9a0 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 564035974 0 0) 0x7f5854003520 con 0x7f58740cf9a0 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 >> v2:172.21.15.124:3300/0 conn(0x7f58740cfde0 msgr2=0x7f58740b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 --2- 172.21.15.26:0/1494037003 >> v2:172.21.15.124:3300/0 conn(0x7f58740cfde0 0x7f58740b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 >> v1:172.21.15.124:6789/0 conn(0x7f587415d4b0 legacy=0x7f58740d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740b29d0 con 0x7f58740cf9a0 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f585c001670 con 0x7f58740cf9a0 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 >> v1:172.21.15.26:6789/0 conn(0x7f58740cf9a0 legacy=0x7f58740b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5854003970 0x7f5854003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58540047c0 0x7f5854004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854005180 0x7f585400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58740d1ec0 con 0x7f5854003970 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585400bfd0 con 0x7f58540047c0 2024-01-15T18:00:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585400c190 con 0x7f5854005180 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.313+0000 7f5873400640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5854003970 0x7f5854003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.314+0000 7f5872a00640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58540047c0 0x7f5854004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.314+0000 7f5873e00640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854005180 0x7f585400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.314+0000 7f5873400640 1 -- 172.21.15.26:0/1494037003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854005180 msgr2=0x7f585400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.314+0000 7f5873400640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854005180 0x7f585400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.314+0000 7f5873400640 1 -- 172.21.15.26:0/1494037003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58540047c0 msgr2=0x7f5854004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.314+0000 7f5873400640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58540047c0 0x7f5854004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.314+0000 7f5873400640 1 -- 172.21.15.26:0/1494037003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5854003520 con 0x7f5854003970 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.314+0000 7f5873e00640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5854005180 0x7f585400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.314+0000 7f5873e00640 1 -- 172.21.15.26:0/1494037003 reap_dead start 2024-01-15T18:00:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.315+0000 7f5873400640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5854003970 0x7f5854003d50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5864001f60 tx=0x7f58540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f4685c4c0e37ce server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.315+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586402d6f0 con 0x7f5854003970 2024-01-15T18:00:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.316+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586402d870 con 0x7f5854003970 2024-01-15T18:00:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.316+0000 7f5872000640 1 -- 172.21.15.26:0/1494037003 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586402dc20 con 0x7f5854003970 2024-01-15T18:00:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.316+0000 7f5879200640 1 -- 172.21.15.26:0/1494037003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5854003970 msgr2=0x7f5854003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.316+0000 7f5879200640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5854003970 0x7f5854003d50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5864001f60 tx=0x7f58540057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.316+0000 7f5879200640 1 -- 172.21.15.26:0/1494037003 shutdown_connections 2024-01-15T18:00:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.316+0000 7f5879200640 1 --2- 172.21.15.26:0/1494037003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5854003970 0x7f5854003d50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.316+0000 7f5879200640 1 -- 172.21.15.26:0/1494037003 >> 172.21.15.26:0/1494037003 conn(0x7f58740d3000 msgr2=0x7f58740d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.316+0000 7f5879200640 1 -- 172.21.15.26:0/1494037003 shutdown_connections 2024-01-15T18:00:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.316+0000 7f5879200640 1 -- 172.21.15.26:0/1494037003 wait complete. 2024-01-15T18:00:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.318+0000 7f5879200640 1 Processor -- start 2024-01-15T18:00:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.318+0000 7f5879200640 1 -- start start 2024-01-15T18:00:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.319+0000 7f5879200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5854003970 0x7f58740b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.319+0000 7f5879200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58740caae0 0x7f58740caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.319+0000 7f5879200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58740b0970 0x7f58740b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.319+0000 7f5879200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58740b1330 con 0x7f58740caae0 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.319+0000 7f5879200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58740cc970 con 0x7f5854003970 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5879200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58740b7b90 con 0x7f58740b0970 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5872a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58740caae0 0x7f58740caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5872a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58740caae0 0x7f58740caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53976/0 (socket says 172.21.15.26:53976) 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5872a00640 1 -- 172.21.15.26:0/3246793800 learned_addr learned my addr 172.21.15.26:0/3246793800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5873400640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5854003970 0x7f58740b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5873e00640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58740b0970 0x7f58740b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5872a00640 1 -- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58740b0970 msgr2=0x7f58740b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5872a00640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58740b0970 0x7f58740b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5872a00640 1 -- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5854003970 msgr2=0x7f58740b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5872a00640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5854003970 0x7f58740b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5872a00640 1 -- 172.21.15.26:0/3246793800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740cbd60 con 0x7f58740caae0 2024-01-15T18:00:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.320+0000 7f5873400640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5854003970 0x7f58740b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.321+0000 7f5872a00640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58740caae0 0x7f58740caf40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e790 tx=0x7f585c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:31.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.321+0000 7f586b400640 1 -- 172.21.15.26:0/3246793800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00ee40 con 0x7f58740caae0 2024-01-15T18:00:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.321+0000 7f586b400640 1 -- 172.21.15.26:0/3246793800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c00c8d0 con 0x7f58740caae0 2024-01-15T18:00:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.321+0000 7f586b400640 1 -- 172.21.15.26:0/3246793800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00cc80 con 0x7f58740caae0 2024-01-15T18:00:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.321+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58740cc040 con 0x7f58740caae0 2024-01-15T18:00:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.321+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58740cc600 con 0x7f58740caae0 2024-01-15T18:00:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.322+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5874067450 con 0x7f58740caae0 2024-01-15T18:00:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.323+0000 7f586b400640 1 -- 172.21.15.26:0/3246793800 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f585c01d3b0 con 0x7f58740caae0 2024-01-15T18:00:31.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.324+0000 7f586b400640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f584c06b890 0x7f584c06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.324+0000 7f586b400640 1 -- 172.21.15.26:0/3246793800 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f585c089d10 con 0x7f58740caae0 2024-01-15T18:00:31.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.325+0000 7f5873400640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f584c06b890 0x7f584c06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.326+0000 7f5873400640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f584c06b890 0x7f584c06dd50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5864036000 tx=0x7f5864031000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:31.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.329+0000 7f586b400640 1 -- 172.21.15.26:0/3246793800 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f585c0545b0 con 0x7f58740caae0 2024-01-15T18:00:31.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.526+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f5874067d70 con 0x7f58740caae0 2024-01-15T18:00:31.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.533+0000 7f586b400640 1 -- 172.21.15.26:0/3246793800 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v19) v1 ==== 120+0+0 (secure 0 0 0) 0x7f585c019020 con 0x7f58740caae0 2024-01-15T18:00:31.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f584c06b890 msgr2=0x7f584c06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f584c06b890 0x7f584c06dd50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5864036000 tx=0x7f5864031000 comp rx=0 tx=0).stop 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58740caae0 msgr2=0x7f58740caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58740caae0 0x7f58740caf40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f585c00e790 tx=0x7f585c00ec50 comp rx=0 tx=0).stop 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 shutdown_connections 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58740b0970 0x7f58740b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f584c06b890 0x7f584c06dd50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58740caae0 0x7f58740caf40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 --2- 172.21.15.26:0/3246793800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5854003970 0x7f58740b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 >> 172.21.15.26:0/3246793800 conn(0x7f58740d3000 msgr2=0x7f587415c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 shutdown_connections 2024-01-15T18:00:31.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.536+0000 7f5879200640 1 -- 172.21.15.26:0/3246793800 wait complete. 2024-01-15T18:00:31.559 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones' 2024-01-15T18:00:31.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.796+0000 7f4d3ee00640 1 Processor -- start 2024-01-15T18:00:31.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.796+0000 7f4d3ee00640 1 -- start start 2024-01-15T18:00:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.798+0000 7f4d3ee00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4d380cc500 0x7f4d380cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.798+0000 7f4d3ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d3806d160 0x7f4d3806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.798+0000 7f4d3ee00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d38070e50 con 0x7f4d38143c40 2024-01-15T18:00:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.798+0000 7f4d3ee00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d380723c0 con 0x7f4d380cc500 2024-01-15T18:00:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.798+0000 7f4d3ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d380cced0 con 0x7f4d3806d160 2024-01-15T18:00:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.798+0000 7f4d3d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4d380cc500 0x7f4d380cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.798+0000 7f4d3d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4d380cc500 0x7f4d380cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53990/0 (socket says 172.21.15.26:53990) 2024-01-15T18:00:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.798+0000 7f4d3d000640 1 -- 172.21.15.26:0/79714364 learned_addr learned my addr 172.21.15.26:0/79714364 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:31.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.798+0000 7f4d3da00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4d38143c40 0x7f4d38071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49800/0 (socket says 172.21.15.26:49800) 2024-01-15T18:00:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.799+0000 7f4d3e400640 1 --2- 172.21.15.26:0/79714364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d3806d160 0x7f4d3806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.799+0000 7f4d3d000640 1 -- 172.21.15.26:0/79714364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d3806d160 msgr2=0x7f4d3806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.799+0000 7f4d3d000640 1 --2- 172.21.15.26:0/79714364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d3806d160 0x7f4d3806d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.799+0000 7f4d3d000640 1 -- 172.21.15.26:0/79714364 >> v1:172.21.15.26:6789/0 conn(0x7f4d38143c40 legacy=0x7f4d38071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.799+0000 7f4d3d000640 1 -- 172.21.15.26:0/79714364 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d38144020 con 0x7f4d380cc500 2024-01-15T18:00:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.799+0000 7f4d3e400640 1 --2- 172.21.15.26:0/79714364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d3806d160 0x7f4d3806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:31.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.800+0000 7f4d3d000640 1 --2- 172.21.15.26:0/79714364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 0x7f4d380cc990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009870 tx=0x7f4d2002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102642aebf2ee428 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.800+0000 7f4d37e00640 1 -- 172.21.15.26:0/79714364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20031640 con 0x7f4d380cc500 2024-01-15T18:00:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.800+0000 7f4d37e00640 1 -- 172.21.15.26:0/79714364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d200317a0 con 0x7f4d380cc500 2024-01-15T18:00:31.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.800+0000 7f4d37e00640 1 -- 172.21.15.26:0/79714364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20031b30 con 0x7f4d380cc500 2024-01-15T18:00:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.801+0000 7f4d3ee00640 1 -- 172.21.15.26:0/79714364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 msgr2=0x7f4d380cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.801+0000 7f4d3ee00640 1 --2- 172.21.15.26:0/79714364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 0x7f4d380cc990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009870 tx=0x7f4d2002b8f0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.801+0000 7f4d3ee00640 1 -- 172.21.15.26:0/79714364 shutdown_connections 2024-01-15T18:00:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.801+0000 7f4d3ee00640 1 --2- 172.21.15.26:0/79714364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d3806d160 0x7f4d3806d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.801+0000 7f4d3ee00640 1 --2- 172.21.15.26:0/79714364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 0x7f4d380cc990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.801+0000 7f4d3ee00640 1 -- 172.21.15.26:0/79714364 >> 172.21.15.26:0/79714364 conn(0x7f4d3806a800 msgr2=0x7f4d3806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.801+0000 7f4d3ee00640 1 -- 172.21.15.26:0/79714364 shutdown_connections 2024-01-15T18:00:31.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.801+0000 7f4d3ee00640 1 -- 172.21.15.26:0/79714364 wait complete. 2024-01-15T18:00:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.803+0000 7f4d3ee00640 1 Processor -- start 2024-01-15T18:00:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.803+0000 7f4d3ee00640 1 -- start start 2024-01-15T18:00:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3ee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d3806d160 0x7f4d380adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 0x7f4d3813e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d38143c40 0x7f4d38159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d380aca50 con 0x7f4d380cc500 2024-01-15T18:00:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3ee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d380acd30 con 0x7f4d3806d160 2024-01-15T18:00:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d380ad010 con 0x7f4d38143c40 2024-01-15T18:00:31.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 0x7f4d3813e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 0x7f4d3813e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53994/0 (socket says 172.21.15.26:53994) 2024-01-15T18:00:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3d000640 1 -- 172.21.15.26:0/4017776488 learned_addr learned my addr 172.21.15.26:0/4017776488 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3e400640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d38143c40 0x7f4d38159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3da00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d3806d160 0x7f4d380adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3d000640 1 -- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d38143c40 msgr2=0x7f4d38159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3d000640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d38143c40 0x7f4d38159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3d000640 1 -- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d3806d160 msgr2=0x7f4d380adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3d000640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d3806d160 0x7f4d380adf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3d000640 1 -- 172.21.15.26:0/4017776488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200094d0 con 0x7f4d380cc500 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3e400640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d38143c40 0x7f4d38159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.804+0000 7f4d3da00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d3806d160 0x7f4d380adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.805+0000 7f4d3d000640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 0x7f4d3813e2a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4d20000c00 tx=0x7f4d200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.805+0000 7f4d35600640 1 -- 172.21.15.26:0/4017776488 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2003ac60 con 0x7f4d380cc500 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.805+0000 7f4d35600640 1 -- 172.21.15.26:0/4017776488 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2003ade0 con 0x7f4d380cc500 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.805+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d380ad2f0 con 0x7f4d380cc500 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.805+0000 7f4d35600640 1 -- 172.21.15.26:0/4017776488 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20044630 con 0x7f4d380cc500 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.805+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d380adaf0 con 0x7f4d380cc500 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.806+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d380ad810 con 0x7f4d380cc500 2024-01-15T18:00:31.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.807+0000 7f4d35600640 1 -- 172.21.15.26:0/4017776488 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f4d2002f050 con 0x7f4d380cc500 2024-01-15T18:00:31.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.808+0000 7f4d35600640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4d1006b860 0x7f4d1006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:31.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.808+0000 7f4d35600640 1 -- 172.21.15.26:0/4017776488 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f4d200ac560 con 0x7f4d380cc500 2024-01-15T18:00:31.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.809+0000 7f4d3da00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4d1006b860 0x7f4d1006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:31.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.810+0000 7f4d3da00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4d1006b860 0x7f4d1006dd20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4d28000f60 tx=0x7f4d28009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:31.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:31.813+0000 7f4d35600640 1 -- 172.21.15.26:0/4017776488 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d380ad810 con 0x7f4d380cc500 2024-01-15T18:00:32.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.082+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones"]} v 0) v1 -- 0x7f4d380ad810 con 0x7f4d380cc500 2024-01-15T18:00:32.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.955+0000 7f4d35600640 1 -- 172.21.15.26:0/4017776488 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bar_for_4_clones"]}]=0 v265) v1 ==== 153+0+0 (secure 0 0 0) 0x7f4d380ad810 con 0x7f4d380cc500 2024-01-15T18:00:32.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.958+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4d1006b860 msgr2=0x7f4d1006dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:32.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.958+0000 7f4d3ee00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4d1006b860 0x7f4d1006dd20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4d28000f60 tx=0x7f4d28009040 comp rx=0 tx=0).stop 2024-01-15T18:00:32.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 msgr2=0x7f4d3813e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:32.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 0x7f4d3813e2a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4d20000c00 tx=0x7f4d200046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:32.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 shutdown_connections 2024-01-15T18:00:32.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4d38143c40 0x7f4d38159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:32.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4d1006b860 0x7f4d1006dd20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:32.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4d380cc500 0x7f4d3813e2a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:32.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 --2- 172.21.15.26:0/4017776488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4d3806d160 0x7f4d380adf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:32.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 >> 172.21.15.26:0/4017776488 conn(0x7f4d3806a800 msgr2=0x7f4d38071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:32.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 shutdown_connections 2024-01-15T18:00:32.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:32.959+0000 7f4d3ee00640 1 -- 172.21.15.26:0/4017776488 wait complete. 2024-01-15T18:00:32.988 INFO:tasks.cephfs_test_runner:X = number of clones issued. ... ok 2024-01-15T18:00:32.988 INFO:tasks.cephfs_test_runner:Starting test: test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgress) X = number of clones issued. 2024-01-15T18:00:32.988 INFO:tasks.cephfs_test_runner:test_progress_bars_for_7_clones (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:00:32.988 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:00:33.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.209+0000 7f5fff400640 1 Processor -- start 2024-01-15T18:00:33.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.209+0000 7f5fff400640 1 -- start start 2024-01-15T18:00:33.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.211+0000 7f5fff400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ff8070b20 0x7f5ff816c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.211+0000 7f5fff400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff806d9d0 con 0x7f5ff81619c0 2024-01-15T18:00:33.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.211+0000 7f5fff400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff806dcb0 con 0x7f5ff80b3e80 2024-01-15T18:00:33.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.211+0000 7f5fff400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff806df90 con 0x7f5ff8070b20 2024-01-15T18:00:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.211+0000 7f5ffca00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5ff81619c0 0x7f5ff806ce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55186/0 (socket says 172.21.15.26:55186) 2024-01-15T18:00:33.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.211+0000 7f5ffca00640 1 -- 172.21.15.26:0/254882429 learned_addr learned my addr 172.21.15.26:0/254882429 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ffd400640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ff8070b20 0x7f5ff816c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520533459 0 0) 0x7f5ff806d9d0 con 0x7f5ff81619c0 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd8003520 con 0x7f5ff81619c0 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4178635784 0 0) 0x7f5fd8003520 con 0x7f5ff81619c0 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ff8070b20 msgr2=0x7f5ff816c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ff8070b20 0x7f5ff816c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 >> v1:172.21.15.124:6789/0 conn(0x7f5ff80b3e80 legacy=0x7f5ff8070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80b2670 con 0x7f5ff81619c0 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f5fe8001670 con 0x7f5ff81619c0 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 >> v1:172.21.15.26:6789/0 conn(0x7f5ff81619c0 legacy=0x7f5ff806ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fd80039a0 0x7f5fd8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80047c0 0x7f5fd8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fd8005180 0x7f5fd800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff806df90 con 0x7f5fd80047c0 2024-01-15T18:00:33.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd800c050 con 0x7f5fd8005180 2024-01-15T18:00:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd800c210 con 0x7f5fd80039a0 2024-01-15T18:00:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.212+0000 7f5ffd400640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5ff8070b20 0x7f5ff816c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.213+0000 7f5ff7e00640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80047c0 0x7f5fd8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.213+0000 7f5ff7e00640 1 -- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fd80039a0 msgr2=0x7f5fd8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.213+0000 7f5ff7e00640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fd80039a0 0x7f5fd8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.213+0000 7f5ff7e00640 1 -- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fd8005180 msgr2=0x7f5fd800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.213+0000 7f5ff7e00640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5fd8005180 0x7f5fd800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.213+0000 7f5ff7e00640 1 -- 172.21.15.26:0/254882429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd8003520 con 0x7f5fd80047c0 2024-01-15T18:00:33.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.214+0000 7f5ff7e00640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80047c0 0x7f5fd8004c40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0001750 tx=0x7f5fd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7849dfd292e52dfa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:33.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.214+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00052d0 con 0x7f5fd80047c0 2024-01-15T18:00:33.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.215+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe002e050 con 0x7f5fd80047c0 2024-01-15T18:00:33.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.215+0000 7f5ff7400640 1 -- 172.21.15.26:0/254882429 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe00035f0 con 0x7f5fd80047c0 2024-01-15T18:00:33.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.215+0000 7f5fff400640 1 -- 172.21.15.26:0/254882429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80047c0 msgr2=0x7f5fd8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.215+0000 7f5fff400640 1 --2- 172.21.15.26:0/254882429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5fd80047c0 0x7f5fd8004c40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0001750 tx=0x7f5fd80057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.215+0000 7f5ffd400640 1 -- 172.21.15.26:0/254882429 reap_dead start 2024-01-15T18:00:33.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.215+0000 7f5fff400640 1 -- 172.21.15.26:0/254882429 shutdown_connections 2024-01-15T18:00:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.215+0000 7f5fff400640 1 -- 172.21.15.26:0/254882429 >> 172.21.15.26:0/254882429 conn(0x7f5ff80b0c00 msgr2=0x7f5ff80b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.215+0000 7f5fff400640 1 -- 172.21.15.26:0/254882429 shutdown_connections 2024-01-15T18:00:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.215+0000 7f5fff400640 1 -- 172.21.15.26:0/254882429 wait complete. 2024-01-15T18:00:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.217+0000 7f5fff400640 1 Processor -- start 2024-01-15T18:00:33.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.217+0000 7f5fff400640 1 -- start start 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.218+0000 7f5fff400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fd80047c0 0x7f5ff815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.218+0000 7f5fff400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ff8146630 0x7f5ff8146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.218+0000 7f5fff400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ff80cd160 0x7f5ff80cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.218+0000 7f5fff400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff8146ff0 con 0x7f5ff80cd160 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.218+0000 7f5fff400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80ce770 con 0x7f5ff8146630 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.218+0000 7f5fff400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80cded0 con 0x7f5fd80047c0 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.218+0000 7f5ffd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ff80cd160 0x7f5ff80cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ffd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ff80cd160 0x7f5ff80cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36744/0 (socket says 172.21.15.26:36744) 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ffd400640 1 -- 172.21.15.26:0/2808743621 learned_addr learned my addr 172.21.15.26:0/2808743621 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:33.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ffca00640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fd80047c0 0x7f5ff815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ff7e00640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ff8146630 0x7f5ff8146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ffd400640 1 -- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fd80047c0 msgr2=0x7f5ff815c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ffd400640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fd80047c0 0x7f5ff815c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ffd400640 1 -- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ff8146630 msgr2=0x7f5ff8146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ffd400640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ff8146630 0x7f5ff8146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ffd400640 1 -- 172.21.15.26:0/2808743621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff80ce1b0 con 0x7f5ff80cd160 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ff7e00640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ff8146630 0x7f5ff8146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.219+0000 7f5ffd400640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ff80cd160 0x7f5ff80cd610 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80703c0 tx=0x7f5ff000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.220+0000 7f5ff4c00640 1 -- 172.21.15.26:0/2808743621 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000de90 con 0x7f5ff80cd160 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.220+0000 7f5ff4c00640 1 -- 172.21.15.26:0/2808743621 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff000f040 con 0x7f5ff80cd160 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.220+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff80ce490 con 0x7f5ff80cd160 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.220+0000 7f5ff4c00640 1 -- 172.21.15.26:0/2808743621 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0003bd0 con 0x7f5ff80cd160 2024-01-15T18:00:33.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.220+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff80cdb50 con 0x7f5ff80cd160 2024-01-15T18:00:33.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.221+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff8099d40 con 0x7f5ff80cd160 2024-01-15T18:00:33.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.222+0000 7f5ff4c00640 1 -- 172.21.15.26:0/2808743621 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f5ff0003d30 con 0x7f5ff80cd160 2024-01-15T18:00:33.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.223+0000 7f5ff4c00640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5fd006b860 0x7f5fd006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.224+0000 7f5ff4c00640 1 -- 172.21.15.26:0/2808743621 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f5ff008a3b0 con 0x7f5ff80cd160 2024-01-15T18:00:33.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.224+0000 7f5ffca00640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5fd006b860 0x7f5fd006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.225+0000 7f5ffca00640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5fd006b860 0x7f5fd006dd20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80017a0 tx=0x7f5fe80015b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:33.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.228+0000 7f5ff4c00640 1 -- 172.21.15.26:0/2808743621 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ff8099d40 con 0x7f5ff80cd160 2024-01-15T18:00:33.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.421+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ff8099d40 con 0x7f5ff80cd160 2024-01-15T18:00:33.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.423+0000 7f5ff4c00640 1 -- 172.21.15.26:0/2808743621 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+13005 (secure 0 0 0) 0x7f5ff8099d40 con 0x7f5ff80cd160 2024-01-15T18:00:33.426 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:33.427 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":52,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:00:25.482824+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1948779240":"2024-01-16T18:00:22.471234+0000","172.21.15.26:6833/1948779240":"2024-01-16T18:00:22.471234+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:00:33.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5fd006b860 msgr2=0x7f5fd006dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5fd006b860 0x7f5fd006dd20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80017a0 tx=0x7f5fe80015b0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ff80cd160 msgr2=0x7f5ff80cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ff80cd160 0x7f5ff80cd610 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5ff80703c0 tx=0x7f5ff000d4a0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 shutdown_connections 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5fd006b860 0x7f5fd006dd20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5ff80cd160 0x7f5ff80cd610 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5ff8146630 0x7f5ff8146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 --2- 172.21.15.26:0/2808743621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5fd80047c0 0x7f5ff815c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 >> 172.21.15.26:0/2808743621 conn(0x7f5ff80b0c00 msgr2=0x7f5ff815e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 shutdown_connections 2024-01-15T18:00:33.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.426+0000 7f5fff400640 1 -- 172.21.15.26:0/2808743621 wait complete. 2024-01-15T18:00:33.448 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:00:33.448 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:00:33.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.665+0000 7f5374600640 1 Processor -- start 2024-01-15T18:00:33.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.665+0000 7f5374600640 1 -- start start 2024-01-15T18:00:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.667+0000 7f5374600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f536c0cf9a0 0x7f536c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.667+0000 7f5374600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c070c40 con 0x7f536c0ae5a0 2024-01-15T18:00:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.667+0000 7f5374600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f536c06c470 con 0x7f536c0b3ae0 2024-01-15T18:00:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.667+0000 7f5374600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f536c071450 con 0x7f536c0cf9a0 2024-01-15T18:00:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.668+0000 7f5371c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f536c0ae5a0 0x7f536c06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55192/0 (socket says 172.21.15.26:55192) 2024-01-15T18:00:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.668+0000 7f5371c00640 1 -- 172.21.15.26:0/2154672688 learned_addr learned my addr 172.21.15.26:0/2154672688 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.668+0000 7f5372600640 1 --2- 172.21.15.26:0/2154672688 >> v2:172.21.15.124:3300/0 conn(0x7f536c0cf9a0 0x7f536c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.668+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696817126 0 0) 0x7f536c070c40 con 0x7f536c0ae5a0 2024-01-15T18:00:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.668+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f534c003520 con 0x7f536c0ae5a0 2024-01-15T18:00:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2287678007 0 0) 0x7f534c003520 con 0x7f536c0ae5a0 2024-01-15T18:00:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 >> v2:172.21.15.124:3300/0 conn(0x7f536c0cf9a0 msgr2=0x7f536c15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 --2- 172.21.15.26:0/2154672688 >> v2:172.21.15.124:3300/0 conn(0x7f536c0cf9a0 0x7f536c15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 >> v1:172.21.15.124:6789/0 conn(0x7f536c0b3ae0 legacy=0x7f536c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c06c750 con 0x7f536c0ae5a0 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f535c001670 con 0x7f536c0ae5a0 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 >> v1:172.21.15.26:6789/0 conn(0x7f536c0ae5a0 legacy=0x7f536c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f534c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f534c0047c0 0x7f534c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f534c005180 0x7f534c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c071450 con 0x7f534c003970 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c00c030 con 0x7f534c0047c0 2024-01-15T18:00:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534c00c1f0 con 0x7f534c005180 2024-01-15T18:00:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f5371c00640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f534c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.669+0000 7f5371200640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f534c0047c0 0x7f534c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.670+0000 7f5372600640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f534c005180 0x7f534c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.670+0000 7f5371c00640 1 -- 172.21.15.26:0/2154672688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f534c005180 msgr2=0x7f534c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.670+0000 7f5371c00640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f534c005180 0x7f534c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.670+0000 7f5371c00640 1 -- 172.21.15.26:0/2154672688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f534c0047c0 msgr2=0x7f534c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.670+0000 7f5371c00640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f534c0047c0 0x7f534c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.670+0000 7f5371c00640 1 -- 172.21.15.26:0/2154672688 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c003520 con 0x7f534c003970 2024-01-15T18:00:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.670+0000 7f5372600640 1 -- 172.21.15.26:0/2154672688 reap_dead start 2024-01-15T18:00:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.671+0000 7f5371c00640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f534c003d90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c190 tx=0x7f534c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f362a591d552093 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c005ad0 con 0x7f534c003970 2024-01-15T18:00:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c005c30 con 0x7f534c003970 2024-01-15T18:00:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f536be00640 1 -- 172.21.15.26:0/2154672688 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c02c830 con 0x7f534c003970 2024-01-15T18:00:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f5374600640 1 -- 172.21.15.26:0/2154672688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 msgr2=0x7f534c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f5374600640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f534c003d90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c190 tx=0x7f534c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f5374600640 1 -- 172.21.15.26:0/2154672688 shutdown_connections 2024-01-15T18:00:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f5374600640 1 --2- 172.21.15.26:0/2154672688 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f534c003d90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f5374600640 1 -- 172.21.15.26:0/2154672688 >> 172.21.15.26:0/2154672688 conn(0x7f536c06a800 msgr2=0x7f536c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f5374600640 1 -- 172.21.15.26:0/2154672688 shutdown_connections 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.672+0000 7f5374600640 1 -- 172.21.15.26:0/2154672688 wait complete. 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.674+0000 7f5374600640 1 Processor -- start 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.674+0000 7f5374600640 1 -- start start 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.675+0000 7f5374600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f536c0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.675+0000 7f5374600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f536c0cb8a0 0x7f536c1f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.675+0000 7f5374600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f536c0cbd20 0x7f536c1f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.675+0000 7f5374600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c0cc840 con 0x7f534c003970 2024-01-15T18:00:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.675+0000 7f5374600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536c156220 con 0x7f536c0cbd20 2024-01-15T18:00:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.675+0000 7f5374600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536c1567a0 con 0x7f536c0cb8a0 2024-01-15T18:00:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.675+0000 7f5371c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f536c0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5371c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f536c0b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36772/0 (socket says 172.21.15.26:36772) 2024-01-15T18:00:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5371c00640 1 -- 172.21.15.26:0/2426372332 learned_addr learned my addr 172.21.15.26:0/2426372332 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5371c00640 1 -- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f536c0cb8a0 msgr2=0x7f536c1f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5372600640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f536c0cbd20 0x7f536c1f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5371200640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f536c0cb8a0 0x7f536c1f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5371c00640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f536c0cb8a0 0x7f536c1f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5371c00640 1 -- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f536c0cbd20 msgr2=0x7f536c1f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5371c00640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f536c0cbd20 0x7f536c1f25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5371c00640 1 -- 172.21.15.26:0/2426372332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536c1f2b20 con 0x7f534c003970 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.676+0000 7f5371c00640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f536c0b2af0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c190 tx=0x7f536c06c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.677+0000 7f5369600640 1 -- 172.21.15.26:0/2426372332 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c03c070 con 0x7f534c003970 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.677+0000 7f5369600640 1 -- 172.21.15.26:0/2426372332 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f535c02ca40 con 0x7f534c003970 2024-01-15T18:00:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.677+0000 7f5369600640 1 -- 172.21.15.26:0/2426372332 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535c02cdb0 con 0x7f534c003970 2024-01-15T18:00:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.677+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536c0cc280 con 0x7f534c003970 2024-01-15T18:00:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.677+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536c156430 con 0x7f534c003970 2024-01-15T18:00:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.678+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f536c0675d0 con 0x7f534c003970 2024-01-15T18:00:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.679+0000 7f5369600640 1 -- 172.21.15.26:0/2426372332 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f535c032070 con 0x7f534c003970 2024-01-15T18:00:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.680+0000 7f5369600640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f534406b860 0x7f534406dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.680+0000 7f5369600640 1 -- 172.21.15.26:0/2426372332 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f535c0ab0c0 con 0x7f534c003970 2024-01-15T18:00:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.681+0000 7f5371200640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f534406b860 0x7f534406dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:33.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.682+0000 7f5371200640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f534406b860 0x7f534406dd20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5354007fd0 tx=0x7f5354009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.686+0000 7f5369600640 1 -- 172.21.15.26:0/2426372332 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f535c0dde90 con 0x7f534c003970 2024-01-15T18:00:33.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.871+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f536c15e400 con 0x7f534c003970 2024-01-15T18:00:33.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.873+0000 7f5369600640 1 -- 172.21.15.26:0/2426372332 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f535c045020 con 0x7f534c003970 2024-01-15T18:00:33.876 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:00:33.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f534406b860 msgr2=0x7f534406dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f534406b860 0x7f534406dd20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5354007fd0 tx=0x7f5354009040 comp rx=0 tx=0).stop 2024-01-15T18:00:33.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 msgr2=0x7f536c0b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:33.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f536c0b2af0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f535c00c190 tx=0x7f536c06c6c0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 shutdown_connections 2024-01-15T18:00:33.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f536c0cb8a0 0x7f536c1f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f534406b860 0x7f534406dd20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f536c0cbd20 0x7f536c1f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 --2- 172.21.15.26:0/2426372332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f534c003970 0x7f536c0b2af0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:33.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 >> 172.21.15.26:0/2426372332 conn(0x7f536c06a800 msgr2=0x7f536c0ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:33.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 shutdown_connections 2024-01-15T18:00:33.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:33.875+0000 7f5374600640 1 -- 172.21.15.26:0/2426372332 wait complete. 2024-01-15T18:00:33.897 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones' 2024-01-15T18:00:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.109+0000 7f1f0d800640 1 Processor -- start 2024-01-15T18:00:34.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.110+0000 7f1f0d800640 1 -- start start 2024-01-15T18:00:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.111+0000 7f1f0d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f080d1650 0x7f1f080d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.111+0000 7f1f0d800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f080cec40 con 0x7f1f080c6be0 2024-01-15T18:00:34.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.111+0000 7f1f0d800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f080d0420 con 0x7f1f080b34e0 2024-01-15T18:00:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.111+0000 7f1f0d800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f080d2010 con 0x7f1f080d1650 2024-01-15T18:00:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.111+0000 7f1f07400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f080d1650 0x7f1f080d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.112+0000 7f1f0ce00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f080c6be0 0x7f1f080c7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55202/0 (socket says 172.21.15.26:55202) 2024-01-15T18:00:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.112+0000 7f1f07400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f080d1650 0x7f1f080d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36776/0 (socket says 172.21.15.26:36776) 2024-01-15T18:00:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.112+0000 7f1f0ce00640 1 -- 172.21.15.26:0/627155285 learned_addr learned my addr 172.21.15.26:0/627155285 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.112+0000 7f1f07400640 1 -- 172.21.15.26:0/627155285 >> v1:172.21.15.124:6789/0 conn(0x7f1f080b34e0 legacy=0x7f1f080b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.112+0000 7f1f07400640 1 -- 172.21.15.26:0/627155285 >> v1:172.21.15.26:6789/0 conn(0x7f1f080c6be0 legacy=0x7f1f080c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.112+0000 7f1f07400640 1 -- 172.21.15.26:0/627155285 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080d0e70 con 0x7f1f080d1650 2024-01-15T18:00:34.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.112+0000 7f1f06a00640 1 -- 172.21.15.26:0/627155285 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538878759 0 0) 0x7f1f080cec40 con 0x7f1f080c6be0 2024-01-15T18:00:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.114+0000 7f1f07400640 1 --2- 172.21.15.26:0/627155285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080d1650 0x7f1f080d1ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00097a0 tx=0x7f1ef002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d8e9fc1c72f039 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:34.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.114+0000 7f1f06a00640 1 -- 172.21.15.26:0/627155285 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0030610 con 0x7f1f080d1650 2024-01-15T18:00:34.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.114+0000 7f1f06a00640 1 -- 172.21.15.26:0/627155285 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0030770 con 0x7f1f080d1650 2024-01-15T18:00:34.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.114+0000 7f1f06a00640 1 -- 172.21.15.26:0/627155285 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0030b00 con 0x7f1f080d1650 2024-01-15T18:00:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.115+0000 7f1f0d800640 1 -- 172.21.15.26:0/627155285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080d1650 msgr2=0x7f1f080d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.115+0000 7f1f0d800640 1 --2- 172.21.15.26:0/627155285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080d1650 0x7f1f080d1ad0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1ef00097a0 tx=0x7f1ef002b960 comp rx=0 tx=0).stop 2024-01-15T18:00:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.115+0000 7f1f0d800640 1 -- 172.21.15.26:0/627155285 shutdown_connections 2024-01-15T18:00:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.115+0000 7f1f0d800640 1 --2- 172.21.15.26:0/627155285 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080d1650 0x7f1f080d1ad0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.115+0000 7f1f0d800640 1 -- 172.21.15.26:0/627155285 >> 172.21.15.26:0/627155285 conn(0x7f1f080d3090 msgr2=0x7f1f080d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.115+0000 7f1f0d800640 1 -- 172.21.15.26:0/627155285 shutdown_connections 2024-01-15T18:00:34.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.115+0000 7f1f0d800640 1 -- 172.21.15.26:0/627155285 wait complete. 2024-01-15T18:00:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.116+0000 7f1f0d800640 1 Processor -- start 2024-01-15T18:00:34.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.117+0000 7f1f0d800640 1 -- start start 2024-01-15T18:00:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f0d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080b34e0 0x7f1f080b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f0d800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f080c6be0 0x7f1f080b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f0d800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f080d1650 0x7f1f080cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:34.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f0d800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080b2690 con 0x7f1f080b34e0 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f0d800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080b8c10 con 0x7f1f080c6be0 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f0d800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080b7d30 con 0x7f1f080d1650 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f07e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080b34e0 0x7f1f080b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f07e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080b34e0 0x7f1f080b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36788/0 (socket says 172.21.15.26:36788) 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f07e00640 1 -- 172.21.15.26:0/1485881668 learned_addr learned my addr 172.21.15.26:0/1485881668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f07e00640 1 -- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f080d1650 msgr2=0x7f1f080cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f07e00640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f080d1650 0x7f1f080cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f07e00640 1 -- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f080c6be0 msgr2=0x7f1f080b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f07e00640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f080c6be0 0x7f1f080b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.118+0000 7f1f07e00640 1 -- 172.21.15.26:0/1485881668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0009400 con 0x7f1f080b34e0 2024-01-15T18:00:34.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.119+0000 7f1f07e00640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080b34e0 0x7f1f080b2150 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c450 tx=0x7f1ef800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.119+0000 7f1effe00640 1 -- 172.21.15.26:0/1485881668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80052d0 con 0x7f1f080b34e0 2024-01-15T18:00:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.119+0000 7f1effe00640 1 -- 172.21.15.26:0/1485881668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef8005430 con 0x7f1f080b34e0 2024-01-15T18:00:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.119+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f080b8690 con 0x7f1f080b34e0 2024-01-15T18:00:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.119+0000 7f1effe00640 1 -- 172.21.15.26:0/1485881668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8012870 con 0x7f1f080b34e0 2024-01-15T18:00:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.119+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f080b79c0 con 0x7f1f080b34e0 2024-01-15T18:00:34.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.121+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f080b7820 con 0x7f1f080b34e0 2024-01-15T18:00:34.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.122+0000 7f1effe00640 1 -- 172.21.15.26:0/1485881668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f1ef80129d0 con 0x7f1f080b34e0 2024-01-15T18:00:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.123+0000 7f1effe00640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ee006b890 0x7f1ee006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:34.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.123+0000 7f1effe00640 1 -- 172.21.15.26:0/1485881668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f1ef8089e90 con 0x7f1f080b34e0 2024-01-15T18:00:34.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.123+0000 7f1f07400640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ee006b890 0x7f1ee006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:34.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.125+0000 7f1f07400640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ee006b890 0x7f1ee006dd50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0036000 tx=0x7f1ef0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:34.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.127+0000 7f1effe00640 1 -- 172.21.15.26:0/1485881668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f080b7820 con 0x7f1f080b34e0 2024-01-15T18:00:34.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.390+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones"]} v 0) v1 -- 0x7f1f080b7820 con 0x7f1f080b34e0 2024-01-15T18:00:34.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.969+0000 7f1effe00640 1 -- 172.21.15.26:0/1485881668 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones"]}]=0 v267) v1 ==== 157+0+0 (secure 0 0 0) 0x7f1f080b7820 con 0x7f1f080b34e0 2024-01-15T18:00:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ee006b890 msgr2=0x7f1ee006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ee006b890 0x7f1ee006dd50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0036000 tx=0x7f1ef0030000 comp rx=0 tx=0).stop 2024-01-15T18:00:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080b34e0 msgr2=0x7f1f080b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080b34e0 0x7f1f080b2150 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c450 tx=0x7f1ef800c910 comp rx=0 tx=0).stop 2024-01-15T18:00:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 shutdown_connections 2024-01-15T18:00:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f080d1650 0x7f1f080cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1ee006b890 0x7f1ee006dd50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:34.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f080c6be0 0x7f1f080b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:34.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 --2- 172.21.15.26:0/1485881668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f080b34e0 0x7f1f080b2150 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:34.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.972+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 >> 172.21.15.26:0/1485881668 conn(0x7f1f080d3090 msgr2=0x7f1f080cdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:34.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.973+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 shutdown_connections 2024-01-15T18:00:34.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:34.973+0000 7f1f0d800640 1 -- 172.21.15.26:0/1485881668 wait complete. 2024-01-15T18:00:34.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:00:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.217+0000 7f3aa7000640 1 Processor -- start 2024-01-15T18:00:35.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.218+0000 7f3aa7000640 1 -- start start 2024-01-15T18:00:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.219+0000 7f3aa7000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00ae5a0 0x7f3aa015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.219+0000 7f3aa7000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3aa00cf290 0x7f3aa00cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.219+0000 7f3aa7000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa0070c40 con 0x7f3aa00b3ae0 2024-01-15T18:00:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.219+0000 7f3aa7000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa0071450 con 0x7f3aa00cf290 2024-01-15T18:00:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.219+0000 7f3aa7000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa00cfc60 con 0x7f3aa00ae5a0 2024-01-15T18:00:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.220+0000 7f3aa5000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3aa00cf290 0x7f3aa00cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.220+0000 7f3aa5000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3aa00cf290 0x7f3aa00cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36802/0 (socket says 172.21.15.26:36802) 2024-01-15T18:00:35.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.220+0000 7f3aa5000640 1 -- 172.21.15.26:0/2979204154 learned_addr learned my addr 172.21.15.26:0/2979204154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.220+0000 7f3a9fe00640 1 --2- 172.21.15.26:0/2979204154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00ae5a0 0x7f3aa015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.220+0000 7f3aa5000640 1 -- 172.21.15.26:0/2979204154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00ae5a0 msgr2=0x7f3aa015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.220+0000 7f3aa5000640 1 --2- 172.21.15.26:0/2979204154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00ae5a0 0x7f3aa015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.220+0000 7f3aa5000640 1 -- 172.21.15.26:0/2979204154 >> v1:172.21.15.26:6789/0 conn(0x7f3aa00b3ae0 legacy=0x7f3aa006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:35.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.220+0000 7f3aa5000640 1 -- 172.21.15.26:0/2979204154 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa006c470 con 0x7f3aa00cf290 2024-01-15T18:00:35.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.221+0000 7f3aa5000640 1 --2- 172.21.15.26:0/2979204154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 0x7f3aa00cf720 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009e50 tx=0x7f3a9802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b2cd832cdb7797 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:35.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.222+0000 7f3a9ea00640 1 -- 172.21.15.26:0/2979204154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9802cb90 con 0x7f3aa00cf290 2024-01-15T18:00:35.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.222+0000 7f3a9ea00640 1 -- 172.21.15.26:0/2979204154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9802ccf0 con 0x7f3aa00cf290 2024-01-15T18:00:35.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.222+0000 7f3a9ea00640 1 -- 172.21.15.26:0/2979204154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98034610 con 0x7f3aa00cf290 2024-01-15T18:00:35.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.222+0000 7f3aa7000640 1 -- 172.21.15.26:0/2979204154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 msgr2=0x7f3aa00cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:35.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.222+0000 7f3aa7000640 1 --2- 172.21.15.26:0/2979204154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 0x7f3aa00cf720 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009e50 tx=0x7f3a9802bff0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.222+0000 7f3aa7000640 1 -- 172.21.15.26:0/2979204154 shutdown_connections 2024-01-15T18:00:35.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.222+0000 7f3aa7000640 1 --2- 172.21.15.26:0/2979204154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 0x7f3aa00cf720 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.222+0000 7f3aa7000640 1 --2- 172.21.15.26:0/2979204154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00ae5a0 0x7f3aa015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.222+0000 7f3aa7000640 1 -- 172.21.15.26:0/2979204154 >> 172.21.15.26:0/2979204154 conn(0x7f3aa006a800 msgr2=0x7f3aa006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.223+0000 7f3aa7000640 1 -- 172.21.15.26:0/2979204154 shutdown_connections 2024-01-15T18:00:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.223+0000 7f3aa7000640 1 -- 172.21.15.26:0/2979204154 wait complete. 2024-01-15T18:00:35.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.224+0000 7f3aa7000640 1 Processor -- start 2024-01-15T18:00:35.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.225+0000 7f3aa7000640 1 -- start start 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa7000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aa00ae5a0 0x7f3aa0160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa7000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00b3ae0 0x7f3aa01503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa7000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 0x7f3aa015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa7000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa015c600 con 0x7f3aa00cf290 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa7000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa015b560 con 0x7f3aa00ae5a0 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa7000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa015b840 con 0x7f3aa00b3ae0 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa5000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 0x7f3aa015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa5000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 0x7f3aa015ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36812/0 (socket says 172.21.15.26:36812) 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa5000640 1 -- 172.21.15.26:0/288448847 learned_addr learned my addr 172.21.15.26:0/288448847 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3a9fe00640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aa00ae5a0 0x7f3aa0160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3a9f400640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00b3ae0 0x7f3aa01503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.226+0000 7f3aa5000640 1 -- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00b3ae0 msgr2=0x7f3aa01503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3aa5000640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00b3ae0 0x7f3aa01503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3aa5000640 1 -- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aa00ae5a0 msgr2=0x7f3aa0160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3aa5000640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aa00ae5a0 0x7f3aa0160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3aa5000640 1 -- 172.21.15.26:0/288448847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a98009b30 con 0x7f3aa00cf290 2024-01-15T18:00:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3a9f400640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00b3ae0 0x7f3aa01503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3aa5000640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 0x7f3aa015ce00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009e50 tx=0x7f3a98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3a97e00640 1 -- 172.21.15.26:0/288448847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98009510 con 0x7f3aa00cf290 2024-01-15T18:00:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3a97e00640 1 -- 172.21.15.26:0/288448847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9803e450 con 0x7f3aa00cf290 2024-01-15T18:00:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3a97e00640 1 -- 172.21.15.26:0/288448847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9803e7e0 con 0x7f3aa00cf290 2024-01-15T18:00:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa015bb20 con 0x7f3aa00cf290 2024-01-15T18:00:35.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.227+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa015c0e0 con 0x7f3aa00cf290 2024-01-15T18:00:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.228+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa015c2d0 con 0x7f3aa00cf290 2024-01-15T18:00:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.229+0000 7f3a97e00640 1 -- 172.21.15.26:0/288448847 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f3a9803e940 con 0x7f3aa00cf290 2024-01-15T18:00:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.230+0000 7f3a97e00640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a7806b860 0x7f3a7806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:35.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.230+0000 7f3a97e00640 1 -- 172.21.15.26:0/288448847 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f3a980ad050 con 0x7f3aa00cf290 2024-01-15T18:00:35.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.231+0000 7f3a9fe00640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a7806b860 0x7f3a7806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:35.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.232+0000 7f3a9fe00640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a7806b860 0x7f3a7806dd20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3a90006fd0 tx=0x7f3a90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:35.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.235+0000 7f3a97e00640 1 -- 172.21.15.26:0/288448847 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a980778a0 con 0x7f3aa00cf290 2024-01-15T18:00:35.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.428+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3aa00675d0 con 0x7f3aa00cf290 2024-01-15T18:00:35.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.434+0000 7f3a97e00640 1 -- 172.21.15.26:0/288448847 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3a98043020 con 0x7f3aa00cf290 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a7806b860 msgr2=0x7f3a7806dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a7806b860 0x7f3a7806dd20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3a90006fd0 tx=0x7f3a90008040 comp rx=0 tx=0).stop 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 msgr2=0x7f3aa015ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 0x7f3aa015ce00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009e50 tx=0x7f3a98004a40 comp rx=0 tx=0).stop 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 shutdown_connections 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3aa00b3ae0 0x7f3aa01503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a7806b860 0x7f3a7806dd20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3aa00cf290 0x7f3aa015ce00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 --2- 172.21.15.26:0/288448847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3aa00ae5a0 0x7f3aa0160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:35.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 >> 172.21.15.26:0/288448847 conn(0x7f3aa006a800 msgr2=0x7f3aa015a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:35.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 shutdown_connections 2024-01-15T18:00:35.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:35.437+0000 7f3aa7000640 1 -- 172.21.15.26:0/288448847 wait complete. 2024-01-15T18:00:35.459 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:00:35.459 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:00:35.574 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:00:35.687 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:00:35.687 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:00:35.687 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:00:35.748 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:00:35.748 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:00:35.748 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:00:35.805 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:00:35.805 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:00:35.805 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:00:35.865 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:00:35.865 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:00:35.865 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:00:35.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:00:35.921 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:00:35.921 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:00:36.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.160+0000 7fa8b3200640 1 Processor -- start 2024-01-15T18:00:36.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.160+0000 7fa8b3200640 1 -- start start 2024-01-15T18:00:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.162+0000 7fa8b3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 0x7fa8b4146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.162+0000 7fa8b3200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8b406c7c0 0x7fa8b40717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.162+0000 7fa8b3200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b406cc20 con 0x7fa8b4071d30 2024-01-15T18:00:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.162+0000 7fa8b3200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b406cec0 con 0x7fa8b406c7c0 2024-01-15T18:00:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.162+0000 7fa8b3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b406d1a0 con 0x7fa8b4071410 2024-01-15T18:00:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.163+0000 7fa8b1400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8b406c7c0 0x7fa8b40717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.163+0000 7fa8b1400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa8b406c7c0 0x7fa8b40717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36824/0 (socket says 172.21.15.26:36824) 2024-01-15T18:00:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.163+0000 7fa8b1400640 1 -- 172.21.15.26:0/1558293665 learned_addr learned my addr 172.21.15.26:0/1558293665 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.163+0000 7fa8b1e00640 1 --2- 172.21.15.26:0/1558293665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 0x7fa8b4146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.163+0000 7fa8b1400640 1 -- 172.21.15.26:0/1558293665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 msgr2=0x7fa8b4146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:36.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.163+0000 7fa8b1400640 1 --2- 172.21.15.26:0/1558293665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 0x7fa8b4146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.163+0000 7fa8b1400640 1 -- 172.21.15.26:0/1558293665 >> v1:172.21.15.124:6789/0 conn(0x7fa8b4071d30 legacy=0x7fa8b40b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:36.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.163+0000 7fa8b1400640 1 -- 172.21.15.26:0/1558293665 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b406d480 con 0x7fa8b406c7c0 2024-01-15T18:00:36.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.164+0000 7fa8b1400640 1 --2- 172.21.15.26:0/1558293665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b406c7c0 0x7fa8b40717f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009870 tx=0x7fa89c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50445db12fb60118 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.165+0000 7fa8b0a00640 1 -- 172.21.15.26:0/1558293665 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c030840 con 0x7fa8b406c7c0 2024-01-15T18:00:36.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.165+0000 7fa8b0a00640 1 -- 172.21.15.26:0/1558293665 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c0309a0 con 0x7fa8b406c7c0 2024-01-15T18:00:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.165+0000 7fa8b0a00640 1 -- 172.21.15.26:0/1558293665 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c030d30 con 0x7fa8b406c7c0 2024-01-15T18:00:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.165+0000 7fa8b3200640 1 -- 172.21.15.26:0/1558293665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b406c7c0 msgr2=0x7fa8b40717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.165+0000 7fa8b3200640 1 --2- 172.21.15.26:0/1558293665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b406c7c0 0x7fa8b40717f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009870 tx=0x7fa89c02b980 comp rx=0 tx=0).stop 2024-01-15T18:00:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.165+0000 7fa8b3200640 1 -- 172.21.15.26:0/1558293665 shutdown_connections 2024-01-15T18:00:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.165+0000 7fa8b3200640 1 --2- 172.21.15.26:0/1558293665 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b406c7c0 0x7fa8b40717f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.165+0000 7fa8b3200640 1 --2- 172.21.15.26:0/1558293665 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 0x7fa8b4146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.165+0000 7fa8b3200640 1 -- 172.21.15.26:0/1558293665 >> 172.21.15.26:0/1558293665 conn(0x7fa8b406a800 msgr2=0x7fa8b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.166+0000 7fa8b3200640 1 -- 172.21.15.26:0/1558293665 shutdown_connections 2024-01-15T18:00:36.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.166+0000 7fa8b3200640 1 -- 172.21.15.26:0/1558293665 wait complete. 2024-01-15T18:00:36.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.167+0000 7fa8b3200640 1 Processor -- start 2024-01-15T18:00:36.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.168+0000 7fa8b3200640 1 -- start start 2024-01-15T18:00:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.168+0000 7fa8b3200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8b406c7c0 0x7fa8b415c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.168+0000 7fa8b3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 0x7fa8b40c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.168+0000 7fa8b3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b4071d30 0x7fa8b41f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.169+0000 7fa8b3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b415cf10 con 0x7fa8b4071d30 2024-01-15T18:00:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.169+0000 7fa8b3200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b40c63e0 con 0x7fa8b406c7c0 2024-01-15T18:00:36.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.169+0000 7fa8b3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b40c66c0 con 0x7fa8b4071410 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.169+0000 7fa8b2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b4071d30 0x7fa8b41f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.169+0000 7fa8b2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b4071d30 0x7fa8b41f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36836/0 (socket says 172.21.15.26:36836) 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.169+0000 7fa8b2800640 1 -- 172.21.15.26:0/608107759 learned_addr learned my addr 172.21.15.26:0/608107759 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.169+0000 7fa8b1e00640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8b406c7c0 0x7fa8b415c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.169+0000 7fa8b1400640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 0x7fa8b40c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8b2800640 1 -- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 msgr2=0x7fa8b40c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8b2800640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 0x7fa8b40c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8b2800640 1 -- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8b406c7c0 msgr2=0x7fa8b415c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8b2800640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8b406c7c0 0x7fa8b415c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8b2800640 1 -- 172.21.15.26:0/608107759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c0094d0 con 0x7fa8b4071d30 2024-01-15T18:00:36.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8b2800640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b4071d30 0x7fa8b41f2af0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00c6b0 tx=0x7fa8ac00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8aa000640 1 -- 172.21.15.26:0/608107759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac003b70 con 0x7fa8b4071d30 2024-01-15T18:00:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8aa000640 1 -- 172.21.15.26:0/608107759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac003cf0 con 0x7fa8b4071d30 2024-01-15T18:00:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8aa000640 1 -- 172.21.15.26:0/608107759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac003ea0 con 0x7fa8b4071d30 2024-01-15T18:00:36.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b40c3d70 con 0x7fa8b4071d30 2024-01-15T18:00:36.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.170+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b40c4330 con 0x7fa8b4071d30 2024-01-15T18:00:36.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.172+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b40724e0 con 0x7fa8b4071d30 2024-01-15T18:00:36.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.173+0000 7fa8aa000640 1 -- 172.21.15.26:0/608107759 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fa8ac005310 con 0x7fa8b4071d30 2024-01-15T18:00:36.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.174+0000 7fa8aa000640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa88c06b860 0x7fa88c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.174+0000 7fa8aa000640 1 -- 172.21.15.26:0/608107759 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7fa8ac08a0b0 con 0x7fa8b4071d30 2024-01-15T18:00:36.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.175+0000 7fa8b1e00640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa88c06b860 0x7fa88c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.176+0000 7fa8b1e00640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa88c06b860 0x7fa88c06dd20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4006fd0 tx=0x7fa8a4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:36.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.178+0000 7fa8aa000640 1 -- 172.21.15.26:0/608107759 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8ac054950 con 0x7fa8b4071d30 2024-01-15T18:00:36.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.409+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8b4067450 con 0x7fa8b4071d30 2024-01-15T18:00:36.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.410+0000 7fa8aa000640 1 -- 172.21.15.26:0/608107759 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fa8ac057f70 con 0x7fa8b4071d30 2024-01-15T18:00:36.414 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:36.414 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5218,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3293492366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3293492366},{"type":"v1","addr":"172.21.15.124:6836","nonce":3293492366}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5221,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/1049920857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":1049920857},{"type":"v1","addr":"172.21.15.124:6837","nonce":1049920857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5793,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/2644667232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2644667232},{"type":"v1","addr":"172.21.15.26:6834","nonce":2644667232}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4255602003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4255602003},{"type":"v1","addr":"172.21.15.26:6835","nonce":4255602003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2024-01-15T18:00:36.415 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 28 2024-01-15T18:00:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa88c06b860 msgr2=0x7fa88c06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa88c06b860 0x7fa88c06dd20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4006fd0 tx=0x7fa8a4008040 comp rx=0 tx=0).stop 2024-01-15T18:00:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b4071d30 msgr2=0x7fa8b41f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:36.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b4071d30 0x7fa8b41f2af0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00c6b0 tx=0x7fa8ac00cb70 comp rx=0 tx=0).stop 2024-01-15T18:00:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 shutdown_connections 2024-01-15T18:00:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8b4071410 0x7fa8b40c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa88c06b860 0x7fa88c06dd20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa8b4071d30 0x7fa8b41f2af0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 --2- 172.21.15.26:0/608107759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa8b406c7c0 0x7fa8b415c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 >> 172.21.15.26:0/608107759 conn(0x7fa8b406a800 msgr2=0x7fa8b40b0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:36.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 shutdown_connections 2024-01-15T18:00:36.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.413+0000 7fa8b3200640 1 -- 172.21.15.26:0/608107759 wait complete. 2024-01-15T18:00:36.437 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:00:36.437 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:00:36.437 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:00:36.437 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:00:36.438 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:00:36.438 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:00:36.438 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:00:36.438 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:00:36.438 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:00:36.438 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:00:36.439 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:00:36.439 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:00:36.439 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:00:36.439 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:00:36.440 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:00:36.440 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:00:36.457 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:00:36.457 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:00:36.461 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:00:36.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:00:36.468 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:00:36.468 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:00:36.469 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:00:36.469 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:00:36.508 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:00:36.508 DEBUG:teuthology.parallel:result is None 2024-01-15T18:00:36.515 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:00:36.515 DEBUG:teuthology.parallel:result is None 2024-01-15T18:00:36.517 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:00:36.518 DEBUG:teuthology.parallel:result is None 2024-01-15T18:00:36.523 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:00:36.523 DEBUG:teuthology.parallel:result is None 2024-01-15T18:00:36.523 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:00:36.666 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-15T18:00:36.671 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T18:00:36.672 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T18:00:36.672 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T18:00:36.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.706+0000 7f1f9cc00640 1 Processor -- start 2024-01-15T18:00:36.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.706+0000 7f1f9cc00640 1 -- start start 2024-01-15T18:00:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.707+0000 7f1f9cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980cc1e0 0x7f1f980cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.707+0000 7f1f9cc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f98074520 0x7f1f980ac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.707+0000 7f1f9cc00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f980ae750 0x7f1f980ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.707+0000 7f1f9cc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f98162100 con 0x7f1f98074520 2024-01-15T18:00:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.707+0000 7f1f9cc00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980ccf20 con 0x7f1f980ae750 2024-01-15T18:00:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.707+0000 7f1f9cc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980cd200 con 0x7f1f980cc1e0 2024-01-15T18:00:36.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f95400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f98074520 0x7f1f980ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f95400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f98074520 0x7f1f980ac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36874/0 (socket says 172.21.15.26:36874) 2024-01-15T18:00:36.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f95400640 1 -- 172.21.15.26:0/4225772233 learned_addr learned my addr 172.21.15.26:0/4225772233 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:36.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f96800640 1 --2- 172.21.15.26:0/4225772233 >> v2:172.21.15.124:3300/0 conn(0x7f1f980ae750 0x7f1f980ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f95e00640 1 --2- 172.21.15.26:0/4225772233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980cc1e0 0x7f1f980cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f95400640 1 -- 172.21.15.26:0/4225772233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980cc1e0 msgr2=0x7f1f980cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:36.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f95400640 1 --2- 172.21.15.26:0/4225772233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980cc1e0 0x7f1f980cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f95400640 1 -- 172.21.15.26:0/4225772233 >> v2:172.21.15.124:3300/0 conn(0x7f1f980ae750 msgr2=0x7f1f980ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:36.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f95400640 1 --2- 172.21.15.26:0/4225772233 >> v2:172.21.15.124:3300/0 conn(0x7f1f980ae750 0x7f1f980ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.708+0000 7f1f95400640 1 -- 172.21.15.26:0/4225772233 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9806daa0 con 0x7f1f98074520 2024-01-15T18:00:36.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.709+0000 7f1f95400640 1 --2- 172.21.15.26:0/4225772233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 0x7f1f980ac050 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1f980716e0 tx=0x7f1f8002b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe51fbb177ba543 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:36.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.709+0000 7f1f94a00640 1 -- 172.21.15.26:0/4225772233 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80031740 con 0x7f1f98074520 2024-01-15T18:00:36.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.709+0000 7f1f94a00640 1 -- 172.21.15.26:0/4225772233 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f800318a0 con 0x7f1f98074520 2024-01-15T18:00:36.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.709+0000 7f1f94a00640 1 -- 172.21.15.26:0/4225772233 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80031c30 con 0x7f1f98074520 2024-01-15T18:00:36.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.710+0000 7f1f9cc00640 1 -- 172.21.15.26:0/4225772233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 msgr2=0x7f1f980ac050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:36.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.710+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/4225772233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 0x7f1f980ac050 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1f980716e0 tx=0x7f1f8002b7e0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.710+0000 7f1f9cc00640 1 -- 172.21.15.26:0/4225772233 shutdown_connections 2024-01-15T18:00:36.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.710+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/4225772233 >> v2:172.21.15.124:3300/0 conn(0x7f1f980ae750 0x7f1f980ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.710+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/4225772233 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 0x7f1f980ac050 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.710+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/4225772233 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980cc1e0 0x7f1f980cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.710+0000 7f1f9cc00640 1 -- 172.21.15.26:0/4225772233 >> 172.21.15.26:0/4225772233 conn(0x7f1f980b0d70 msgr2=0x7f1f980b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:36.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.710+0000 7f1f9cc00640 1 -- 172.21.15.26:0/4225772233 shutdown_connections 2024-01-15T18:00:36.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.710+0000 7f1f9cc00640 1 -- 172.21.15.26:0/4225772233 wait complete. 2024-01-15T18:00:36.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.712+0000 7f1f9cc00640 1 Processor -- start 2024-01-15T18:00:36.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.712+0000 7f1f9cc00640 1 -- start start 2024-01-15T18:00:36.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f9cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 0x7f1f981f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f9cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980ae750 0x7f1f980c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f9cc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980cc1e0 0x7f1f9815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f9cc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f981f67f0 con 0x7f1f98074520 2024-01-15T18:00:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f9cc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980c76e0 con 0x7f1f980cc1e0 2024-01-15T18:00:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f9cc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980c79c0 con 0x7f1f980ae750 2024-01-15T18:00:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f95e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 0x7f1f981f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f95e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 0x7f1f981f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36880/0 (socket says 172.21.15.26:36880) 2024-01-15T18:00:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f95e00640 1 -- 172.21.15.26:0/3326677372 learned_addr learned my addr 172.21.15.26:0/3326677372 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:36.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f95400640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980ae750 0x7f1f980c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.713+0000 7f1f96800640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980cc1e0 0x7f1f9815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f95e00640 1 -- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980ae750 msgr2=0x7f1f980c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f95e00640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980ae750 0x7f1f980c2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f95e00640 1 -- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980cc1e0 msgr2=0x7f1f9815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f95e00640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980cc1e0 0x7f1f9815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f95e00640 1 -- 172.21.15.26:0/3326677372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f800094d0 con 0x7f1f98074520 2024-01-15T18:00:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f95e00640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 0x7f1f981f62b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800e870 tx=0x7f1f8800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f8e000640 1 -- 172.21.15.26:0/3326677372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880094f0 con 0x7f1f98074520 2024-01-15T18:00:36.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f8e000640 1 -- 172.21.15.26:0/3326677372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8800c940 con 0x7f1f98074520 2024-01-15T18:00:36.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f980c7ca0 con 0x7f1f98074520 2024-01-15T18:00:36.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f8e000640 1 -- 172.21.15.26:0/3326677372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800ccf0 con 0x7f1f98074520 2024-01-15T18:00:36.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.714+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f980c8260 con 0x7f1f98074520 2024-01-15T18:00:36.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.716+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f98066da0 con 0x7f1f98074520 2024-01-15T18:00:36.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.716+0000 7f1f8e000640 1 -- 172.21.15.26:0/3326677372 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f1f88013470 con 0x7f1f98074520 2024-01-15T18:00:36.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.717+0000 7f1f8e000640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006b860 0x7f1f7006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:36.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.717+0000 7f1f8e000640 1 -- 172.21.15.26:0/3326677372 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4971+0+0 (secure 0 0 0) 0x7f1f8808a370 con 0x7f1f98074520 2024-01-15T18:00:36.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.718+0000 7f1f95400640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006b860 0x7f1f7006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:36.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.719+0000 7f1f95400640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006b860 0x7f1f7006dd20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1f80035000 tx=0x7f1f80008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:36.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.721+0000 7f1f8e000640 1 -- 172.21.15.26:0/3326677372 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f88054d60 con 0x7f1f98074520 2024-01-15T18:00:36.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.919+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1f9806e7a0 con 0x7f1f98074520 2024-01-15T18:00:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.993+0000 7f1f8e000640 1 -- 172.21.15.26:0/3326677372 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1f88058380 con 0x7f1f98074520 2024-01-15T18:00:36.997 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-01-15T18:00:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006b860 msgr2=0x7f1f7006dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006b860 0x7f1f7006dd20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1f80035000 tx=0x7f1f80008030 comp rx=0 tx=0).stop 2024-01-15T18:00:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 msgr2=0x7f1f981f62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 0x7f1f981f62b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1f8800e870 tx=0x7f1f8800ed30 comp rx=0 tx=0).stop 2024-01-15T18:00:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 shutdown_connections 2024-01-15T18:00:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980ae750 0x7f1f980c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006b860 0x7f1f7006dd20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980cc1e0 0x7f1f9815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 --2- 172.21.15.26:0/3326677372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f98074520 0x7f1f981f62b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 >> 172.21.15.26:0/3326677372 conn(0x7f1f980b0d70 msgr2=0x7f1f980b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 shutdown_connections 2024-01-15T18:00:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:36.996+0000 7f1f9cc00640 1 -- 172.21.15.26:0/3326677372 wait complete. 2024-01-15T18:00:37.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:00:37.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.234+0000 7f70c4000640 1 Processor -- start 2024-01-15T18:00:37.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.234+0000 7f70c4000640 1 -- start start 2024-01-15T18:00:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.236+0000 7f70c4000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f70bc0cf9a0 0x7f70bc15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.236+0000 7f70c4000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc070c40 con 0x7f70bc0b3ae0 2024-01-15T18:00:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.236+0000 7f70c4000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70bc06c470 con 0x7f70bc0ae5a0 2024-01-15T18:00:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.236+0000 7f70c4000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70bc071450 con 0x7f70bc0cf9a0 2024-01-15T18:00:37.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.236+0000 7f70c0c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f70bc0b3ae0 0x7f70bc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55246/0 (socket says 172.21.15.26:55246) 2024-01-15T18:00:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.236+0000 7f70c0c00640 1 -- 172.21.15.26:0/1959836367 learned_addr learned my addr 172.21.15.26:0/1959836367 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70c2000640 1 --2- 172.21.15.26:0/1959836367 >> v2:172.21.15.124:3300/0 conn(0x7f70bc0cf9a0 0x7f70bc15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470050907 0 0) 0x7f70bc070c40 con 0x7f70bc0b3ae0 2024-01-15T18:00:37.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f709c003520 con 0x7f70bc0b3ae0 2024-01-15T18:00:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3908304299 0 0) 0x7f709c003520 con 0x7f70bc0b3ae0 2024-01-15T18:00:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 >> v2:172.21.15.124:3300/0 conn(0x7f70bc0cf9a0 msgr2=0x7f70bc15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70bbe00640 1 --2- 172.21.15.26:0/1959836367 >> v2:172.21.15.124:3300/0 conn(0x7f70bc0cf9a0 0x7f70bc15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 >> v1:172.21.15.124:6789/0 conn(0x7f70bc0ae5a0 legacy=0x7f70bc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc06c750 con 0x7f70bc0b3ae0 2024-01-15T18:00:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f70a4001670 con 0x7f70bc0b3ae0 2024-01-15T18:00:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.237+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 >> v1:172.21.15.26:6789/0 conn(0x7f70bc0b3ae0 legacy=0x7f70bc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70bbe00640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f709c0039c0 0x7f709c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70bbe00640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f709c0047c0 0x7f709c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70bbe00640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f709c005180 0x7f709c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc071450 con 0x7f709c0039c0 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c00c060 con 0x7f709c005180 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c00c220 con 0x7f709c0047c0 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c1600640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f709c0039c0 0x7f709c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c0c00640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f709c0047c0 0x7f709c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c2000640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f709c005180 0x7f709c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c1600640 1 -- 172.21.15.26:0/1959836367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f709c0047c0 msgr2=0x7f709c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c1600640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f709c0047c0 0x7f709c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c1600640 1 -- 172.21.15.26:0/1959836367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f709c005180 msgr2=0x7f709c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c1600640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f709c005180 0x7f709c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c1600640 1 -- 172.21.15.26:0/1959836367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c003520 con 0x7f709c0039c0 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c2000640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f709c005180 0x7f709c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.238+0000 7f70c2000640 1 -- 172.21.15.26:0/1959836367 reap_dead start 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.239+0000 7f70c1600640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f709c0039c0 0x7f709c003de0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f70ac001750 tx=0x7f709c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5656e7d36de9dfa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.240+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0052d0 con 0x7f709c0039c0 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.240+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac02e050 con 0x7f709c0039c0 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.240+0000 7f70bbe00640 1 -- 172.21.15.26:0/1959836367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac003830 con 0x7f709c0039c0 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.240+0000 7f70c4000640 1 -- 172.21.15.26:0/1959836367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f709c0039c0 msgr2=0x7f709c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.240+0000 7f70c4000640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f709c0039c0 0x7f709c003de0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f70ac001750 tx=0x7f709c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.240+0000 7f70c4000640 1 -- 172.21.15.26:0/1959836367 shutdown_connections 2024-01-15T18:00:37.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.240+0000 7f70c4000640 1 --2- 172.21.15.26:0/1959836367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f709c0039c0 0x7f709c003de0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.240+0000 7f70c4000640 1 -- 172.21.15.26:0/1959836367 >> 172.21.15.26:0/1959836367 conn(0x7f70bc06a800 msgr2=0x7f70bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:37.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.240+0000 7f70c4000640 1 -- 172.21.15.26:0/1959836367 shutdown_connections 2024-01-15T18:00:37.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.241+0000 7f70c4000640 1 -- 172.21.15.26:0/1959836367 wait complete. 2024-01-15T18:00:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.242+0000 7f70c4000640 1 Processor -- start 2024-01-15T18:00:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.242+0000 7f70c4000640 1 -- start start 2024-01-15T18:00:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.243+0000 7f70c4000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f709c0039c0 0x7f70bc156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.243+0000 7f70c4000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70bc0cc2d0 0x7f70bc1581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.243+0000 7f70c4000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70bc0b23c0 0x7f70bc14b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.243+0000 7f70c4000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc158700 con 0x7f70bc0b23c0 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.243+0000 7f70c4000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc156200 con 0x7f709c0039c0 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.243+0000 7f70c4000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc157140 con 0x7f70bc0cc2d0 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.244+0000 7f70c2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70bc0b23c0 0x7f70bc14b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.244+0000 7f70c2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70bc0b23c0 0x7f70bc14b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36892/0 (socket says 172.21.15.26:36892) 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.244+0000 7f70c2000640 1 -- 172.21.15.26:0/2153811115 learned_addr learned my addr 172.21.15.26:0/2153811115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.244+0000 7f70c2000640 1 -- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70bc0cc2d0 msgr2=0x7f70bc1581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.244+0000 7f70c2000640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70bc0cc2d0 0x7f70bc1581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.244+0000 7f70c2000640 1 -- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f709c0039c0 msgr2=0x7f70bc156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.244+0000 7f70c2000640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f709c0039c0 0x7f70bc156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.244+0000 7f70c2000640 1 -- 172.21.15.26:0/2153811115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70bc157420 con 0x7f70bc0b23c0 2024-01-15T18:00:37.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.244+0000 7f70c2000640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70bc0b23c0 0x7f70bc14b280 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f70bc06dfa0 tx=0x7f70b400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:37.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.245+0000 7f70b9600640 1 -- 172.21.15.26:0/2153811115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4016620 con 0x7f70bc0b23c0 2024-01-15T18:00:37.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.245+0000 7f70b9600640 1 -- 172.21.15.26:0/2153811115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b4016780 con 0x7f70bc0b23c0 2024-01-15T18:00:37.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.245+0000 7f70b9600640 1 -- 172.21.15.26:0/2153811115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4016af0 con 0x7f70bc0b23c0 2024-01-15T18:00:37.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.245+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70bc157700 con 0x7f70bc0b23c0 2024-01-15T18:00:37.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.245+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc157cc0 con 0x7f70bc0b23c0 2024-01-15T18:00:37.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.246+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70bc157e80 con 0x7f70bc0b23c0 2024-01-15T18:00:37.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.247+0000 7f70b9600640 1 -- 172.21.15.26:0/2153811115 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f70b4016c50 con 0x7f70bc0b23c0 2024-01-15T18:00:37.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.248+0000 7f70b9600640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f709406b860 0x7f709406dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.248+0000 7f70b9600640 1 -- 172.21.15.26:0/2153811115 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f70b408b230 con 0x7f70bc0b23c0 2024-01-15T18:00:37.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.248+0000 7f70c1600640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f709406b860 0x7f709406dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.249+0000 7f70c1600640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f709406b860 0x7f709406dd20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f70ac033000 tx=0x7f70ac02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:37.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.252+0000 7f70b9600640 1 -- 172.21.15.26:0/2153811115 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70b4055a10 con 0x7f70bc0b23c0 2024-01-15T18:00:37.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.503+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f70bc15e400 con 0x7f70bc0b23c0 2024-01-15T18:00:37.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.504+0000 7f70b9600640 1 -- 172.21.15.26:0/2153811115 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f70b4059030 con 0x7f70bc0b23c0 2024-01-15T18:00:37.508 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:37.508 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:37.508 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-01-15T18:00:37.508 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.508 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-01-15T18:00:37.508 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVl3erNJhAANxPK+39yVM5neq9iKC1HkA==", 2024-01-15T18:00:37.508 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.509 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:00:37.509 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:00:37.509 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:00:37.509 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.509 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.509 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.509 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.509 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-01-15T18:00:37.509 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlLaUlDRAAeKaen5hLlwDDWY6OAMRsPQ==", 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-01-15T18:00:37.510 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlaIAVOBAAA8LEFM1HtzRtNooQCCjS/w==", 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlmpTBHRAAUwbZeVpysMsn56QRLRJCoA==", 2024-01-15T18:00:37.511 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.512 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:00:37.512 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:00:37.512 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:00:37.512 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.512 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.512 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.512 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.512 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-01-15T18:00:37.512 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA7caVlCTmgDhAAPWb0qEh134xxyF8lM9DoNg==", 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA+caVlaxF5JxAA23Z9j21ATy3GVlhgndDIxw==", 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.513 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBCcaVlcRtjBRAAOFN9Xf5kN5MXL6yQfRI27g==", 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:00:37.514 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBFcaVlvAzEHhAACDoNh7BZY9gbKTdGwIrhlg==", 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.515 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBMcaVl8yiOMBAAdI4NTeSB1d/nSHRaIDMcFw==", 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.516 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBQcaVltf8sARAAfPaJAfoTgQoEivw90jRz/Q==", 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.517 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBTcaVltuKiGhAAT0cIJiRkQ38QXT/h6Gyggg==", 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-01-15T18:00:37.518 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBWcaVlbJqVMhAAKjpl6pAHbytjZWMYeRbRjA==", 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA==", 2024-01-15T18:00:37.519 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVlrVpqBxAAxOL0iuAlOvfR+KgSsin3vQ==", 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.520 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlO8vdNxAAlUE/TFCDQoXuxvwLPwUvtg==", 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.521 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVl5qgCEhAALgt7d3OjEmq5bc4nAD2rTw==", 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.522 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA2caVlyg4eHBAAwTCXVk8hO9+zcZ1tUA3gfA==", 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.523 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlvuBcGhAAGqdCMSffJwnMKlFh3CtyCw==", 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.524 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVle8Z+DhAA9cWxzc4puvrHAedqupPG3Q==", 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:00:37.525 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:00:37.526 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:00:37.526 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.526 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:37.526 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-01-15T18:00:37.526 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f709406b860 msgr2=0x7f709406dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f709406b860 0x7f709406dd20 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f70ac033000 tx=0x7f70ac02f000 comp rx=0 tx=0).stop 2024-01-15T18:00:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70bc0b23c0 msgr2=0x7f70bc14b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70bc0b23c0 0x7f70bc14b280 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f70bc06dfa0 tx=0x7f70b400f520 comp rx=0 tx=0).stop 2024-01-15T18:00:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 shutdown_connections 2024-01-15T18:00:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70bc0cc2d0 0x7f70bc1581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f709406b860 0x7f709406dd20 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70bc0b23c0 0x7f70bc14b280 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 --2- 172.21.15.26:0/2153811115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f709c0039c0 0x7f70bc156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.506+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 >> 172.21.15.26:0/2153811115 conn(0x7f70bc06a800 msgr2=0x7f70bc1617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.507+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 shutdown_connections 2024-01-15T18:00:37.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.507+0000 7f70c4000640 1 -- 172.21.15.26:0/2153811115 wait complete. 2024-01-15T18:00:37.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:00:37.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.739+0000 7f9083e00640 1 Processor -- start 2024-01-15T18:00:37.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.739+0000 7f9083e00640 1 -- start start 2024-01-15T18:00:37.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.741+0000 7f9083e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9084143c40 0x7f9084071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.741+0000 7f9083e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90840717d0 0x7f90840cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.741+0000 7f9083e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9084070e50 con 0x7f90840cca40 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.741+0000 7f9083e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90840723c0 con 0x7f90840717d0 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.741+0000 7f9083e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90840cce70 con 0x7f9084143c40 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.741+0000 7f907be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90840717d0 0x7f90840cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.741+0000 7f907be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90840717d0 0x7f90840cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36906/0 (socket says 172.21.15.26:36906) 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.741+0000 7f907be00640 1 -- 172.21.15.26:0/159911613 learned_addr learned my addr 172.21.15.26:0/159911613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.741+0000 7f9082a00640 1 --2- 172.21.15.26:0/159911613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9084143c40 0x7f9084071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.742+0000 7f907be00640 1 -- 172.21.15.26:0/159911613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9084143c40 msgr2=0x7f9084071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.742+0000 7f907be00640 1 --2- 172.21.15.26:0/159911613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9084143c40 0x7f9084071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.742+0000 7f907be00640 1 -- 172.21.15.26:0/159911613 >> v1:172.21.15.124:6789/0 conn(0x7f90840cca40 legacy=0x7f908415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.742+0000 7f907be00640 1 -- 172.21.15.26:0/159911613 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908406e170 con 0x7f90840717d0 2024-01-15T18:00:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.743+0000 7f907be00640 1 --2- 172.21.15.26:0/159911613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 0x7f90840cc500 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f906c009870 tx=0x7f906c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e91f7a5e7ede5c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.743+0000 7f9082000640 1 -- 172.21.15.26:0/159911613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c0317c0 con 0x7f90840717d0 2024-01-15T18:00:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.743+0000 7f9082000640 1 -- 172.21.15.26:0/159911613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c031920 con 0x7f90840717d0 2024-01-15T18:00:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.743+0000 7f9082000640 1 -- 172.21.15.26:0/159911613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c031cb0 con 0x7f90840717d0 2024-01-15T18:00:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.744+0000 7f9083e00640 1 -- 172.21.15.26:0/159911613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 msgr2=0x7f90840cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.744+0000 7f9083e00640 1 --2- 172.21.15.26:0/159911613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 0x7f90840cc500 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f906c009870 tx=0x7f906c02b860 comp rx=0 tx=0).stop 2024-01-15T18:00:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.744+0000 7f9083e00640 1 -- 172.21.15.26:0/159911613 shutdown_connections 2024-01-15T18:00:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.744+0000 7f9083e00640 1 --2- 172.21.15.26:0/159911613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 0x7f90840cc500 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.744+0000 7f9083e00640 1 --2- 172.21.15.26:0/159911613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9084143c40 0x7f9084071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.744+0000 7f9083e00640 1 -- 172.21.15.26:0/159911613 >> 172.21.15.26:0/159911613 conn(0x7f908406a800 msgr2=0x7f908406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.744+0000 7f9083e00640 1 -- 172.21.15.26:0/159911613 shutdown_connections 2024-01-15T18:00:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.744+0000 7f9083e00640 1 -- 172.21.15.26:0/159911613 wait complete. 2024-01-15T18:00:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.746+0000 7f9083e00640 1 Processor -- start 2024-01-15T18:00:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.746+0000 7f9083e00640 1 -- start start 2024-01-15T18:00:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9083e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 0x7f908412cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9083e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90840cca40 0x7f90840ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9083e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9084143c40 0x7f90840ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9083e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9084159060 con 0x7f90840717d0 2024-01-15T18:00:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9083e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f908412d4a0 con 0x7f9084143c40 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9083e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908413ed80 con 0x7f90840cca40 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9082a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 0x7f908412cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9082a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 0x7f908412cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36912/0 (socket says 172.21.15.26:36912) 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9082a00640 1 -- 172.21.15.26:0/2856068506 learned_addr learned my addr 172.21.15.26:0/2856068506 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f907be00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90840cca40 0x7f90840ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9083400640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9084143c40 0x7f90840ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9082a00640 1 -- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90840cca40 msgr2=0x7f90840ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9082a00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90840cca40 0x7f90840ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9082a00640 1 -- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9084143c40 msgr2=0x7f90840ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9082a00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9084143c40 0x7f90840ad7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9082a00640 1 -- 172.21.15.26:0/2856068506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c0094d0 con 0x7f90840717d0 2024-01-15T18:00:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f907be00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90840cca40 0x7f90840ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.747+0000 7f9083400640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9084143c40 0x7f90840ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.748+0000 7f9082a00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 0x7f908412cf60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f907400d5f0 tx=0x7f907400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.748+0000 7f907b400640 1 -- 172.21.15.26:0/2856068506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90740125f0 con 0x7f90840717d0 2024-01-15T18:00:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.748+0000 7f907b400640 1 -- 172.21.15.26:0/2856068506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9074012750 con 0x7f90840717d0 2024-01-15T18:00:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.748+0000 7f907b400640 1 -- 172.21.15.26:0/2856068506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074012b00 con 0x7f90840717d0 2024-01-15T18:00:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.748+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9084142240 con 0x7f90840717d0 2024-01-15T18:00:37.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.748+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908413c5e0 con 0x7f90840717d0 2024-01-15T18:00:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.749+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9084067450 con 0x7f90840717d0 2024-01-15T18:00:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.750+0000 7f907b400640 1 -- 172.21.15.26:0/2856068506 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f9074012c60 con 0x7f90840717d0 2024-01-15T18:00:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.751+0000 7f907b400640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f905c06b860 0x7f905c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.751+0000 7f907b400640 1 -- 172.21.15.26:0/2856068506 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f907408a760 con 0x7f90840717d0 2024-01-15T18:00:37.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.752+0000 7f907be00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f905c06b860 0x7f905c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:37.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.753+0000 7f907be00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f905c06b860 0x7f905c06dd20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f906c036000 tx=0x7f906c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:37.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.755+0000 7f907b400640 1 -- 172.21.15.26:0/2856068506 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90740550e0 con 0x7f90840717d0 2024-01-15T18:00:37.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.941+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90840ceb30 con 0x7f90840717d0 2024-01-15T18:00:37.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.943+0000 7f907b400640 1 -- 172.21.15.26:0/2856068506 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+12876 (secure 0 0 0) 0x7f9074058700 con 0x7f90840717d0 2024-01-15T18:00:37.947 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:37.947 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":56,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:00:37.007375+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:00:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f905c06b860 msgr2=0x7f905c06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f905c06b860 0x7f905c06dd20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f906c036000 tx=0x7f906c03d040 comp rx=0 tx=0).stop 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 msgr2=0x7f908412cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 0x7f908412cf60 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f907400d5f0 tx=0x7f907400dab0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 shutdown_connections 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90840cca40 0x7f90840ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f905c06b860 0x7f905c06dd20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9084143c40 0x7f90840ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 --2- 172.21.15.26:0/2856068506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90840717d0 0x7f908412cf60 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 >> 172.21.15.26:0/2856068506 conn(0x7f908406a800 msgr2=0x7f9084140090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:37.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 shutdown_connections 2024-01-15T18:00:37.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:37.946+0000 7f9083e00640 1 -- 172.21.15.26:0/2856068506 wait complete. 2024-01-15T18:00:37.969 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:00:37.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:00:38.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.179+0000 7fe175a00640 1 Processor -- start 2024-01-15T18:00:38.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.179+0000 7fe175a00640 1 -- start start 2024-01-15T18:00:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.181+0000 7fe175a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17015d610 0x7fe1700ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.181+0000 7fe175a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe1700d1ef0 0x7fe1700b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.181+0000 7fe175a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1700b3400 con 0x7fe1700ced60 2024-01-15T18:00:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.181+0000 7fe175a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1700d1390 con 0x7fe1700d1ef0 2024-01-15T18:00:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.181+0000 7fe175a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700b27f0 con 0x7fe17015d610 2024-01-15T18:00:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.181+0000 7fe16e000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe1700ced60 0x7fe1700d17e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55262/0 (socket says 172.21.15.26:55262) 2024-01-15T18:00:38.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.181+0000 7fe16e000640 1 -- 172.21.15.26:0/381764218 learned_addr learned my addr 172.21.15.26:0/381764218 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.181+0000 7fe16ea00640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17015d610 0x7fe1700ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.181+0000 7fe16f400640 1 --2- 172.21.15.26:0/381764218 >> v2:172.21.15.124:3300/0 conn(0x7fe1700d1ef0 0x7fe1700b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029257852 0 0) 0x7fe1700b3400 con 0x7fe1700ced60 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe150003520 con 0x7fe1700ced60 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1824765486 0 0) 0x7fe150003520 con 0x7fe1700ced60 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17015d610 msgr2=0x7fe1700ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17015d610 0x7fe1700ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 >> v2:172.21.15.124:3300/0 conn(0x7fe1700d1ef0 msgr2=0x7fe1700b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 --2- 172.21.15.26:0/381764218 >> v2:172.21.15.124:3300/0 conn(0x7fe1700d1ef0 0x7fe1700b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1700b2ad0 con 0x7fe1700ced60 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fe158001670 con 0x7fe1700ced60 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 >> v1:172.21.15.26:6789/0 conn(0x7fe1700ced60 legacy=0x7fe1700d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17015d610 0x7fe150003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe150003ff0 0x7fe1500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe150005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700d1390 con 0x7fe150004d00 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700b27f0 con 0x7fe150003ff0 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15000bae0 con 0x7fe17015d610 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.182+0000 7fe16f400640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe150005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.183+0000 7fe16ea00640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17015d610 0x7fe150003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.183+0000 7fe16f400640 1 -- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17015d610 msgr2=0x7fe150003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.183+0000 7fe16e000640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe150003ff0 0x7fe1500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.183+0000 7fe16f400640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17015d610 0x7fe150003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.183+0000 7fe16f400640 1 -- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe150003ff0 msgr2=0x7fe1500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.183+0000 7fe16f400640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe150003ff0 0x7fe1500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.183+0000 7fe16f400640 1 -- 172.21.15.26:0/381764218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150003520 con 0x7fe150004d00 2024-01-15T18:00:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.183+0000 7fe16e000640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe150003ff0 0x7fe1500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.183+0000 7fe16ea00640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe17015d610 0x7fe150003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.184+0000 7fe16f400640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe150005180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe16800ba00 tx=0x7fe1500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45912432e17b9c0d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.184+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168032660 con 0x7fe150004d00 2024-01-15T18:00:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.184+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1680327c0 con 0x7fe150004d00 2024-01-15T18:00:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.184+0000 7fe16d600640 1 -- 172.21.15.26:0/381764218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168032b50 con 0x7fe150004d00 2024-01-15T18:00:38.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.185+0000 7fe175a00640 1 -- 172.21.15.26:0/381764218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 msgr2=0x7fe150005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.185+0000 7fe175a00640 1 --2- 172.21.15.26:0/381764218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe150005180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe16800ba00 tx=0x7fe1500057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.185+0000 7fe16f400640 1 -- 172.21.15.26:0/381764218 reap_dead start 2024-01-15T18:00:38.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.185+0000 7fe175a00640 1 -- 172.21.15.26:0/381764218 shutdown_connections 2024-01-15T18:00:38.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.185+0000 7fe175a00640 1 -- 172.21.15.26:0/381764218 >> 172.21.15.26:0/381764218 conn(0x7fe17009c540 msgr2=0x7fe1700d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:38.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.185+0000 7fe175a00640 1 -- 172.21.15.26:0/381764218 shutdown_connections 2024-01-15T18:00:38.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.185+0000 7fe175a00640 1 -- 172.21.15.26:0/381764218 wait complete. 2024-01-15T18:00:38.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.187+0000 7fe175a00640 1 Processor -- start 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.187+0000 7fe175a00640 1 -- start start 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.188+0000 7fe175a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe1701595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.188+0000 7fe175a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1700cb530 0x7fe17015e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.188+0000 7fe175a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1700cb9b0 0x7fe17015fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.188+0000 7fe175a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe17015f040 con 0x7fe150004d00 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.188+0000 7fe175a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700b6050 con 0x7fe1700cb9b0 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.188+0000 7fe175a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1700b6330 con 0x7fe1700cb530 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.188+0000 7fe16ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe1701595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.188+0000 7fe16ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe1701595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36938/0 (socket says 172.21.15.26:36938) 2024-01-15T18:00:38.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.188+0000 7fe16ea00640 1 -- 172.21.15.26:0/2981505485 learned_addr learned my addr 172.21.15.26:0/2981505485 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe16e000640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1700cb530 0x7fe17015e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe16f400640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1700cb9b0 0x7fe17015fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe16ea00640 1 -- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1700cb530 msgr2=0x7fe17015e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe16ea00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1700cb530 0x7fe17015e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe16ea00640 1 -- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1700cb9b0 msgr2=0x7fe17015fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe16ea00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1700cb9b0 0x7fe17015fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe16ea00640 1 -- 172.21.15.26:0/2981505485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1700b6610 con 0x7fe150004d00 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe16ea00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe1701595d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe160009fd0 tx=0x7fe16000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe166a00640 1 -- 172.21.15.26:0/2981505485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1600145f0 con 0x7fe150004d00 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe166a00640 1 -- 172.21.15.26:0/2981505485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe160014750 con 0x7fe150004d00 2024-01-15T18:00:38.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe166a00640 1 -- 172.21.15.26:0/2981505485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe160014ae0 con 0x7fe150004d00 2024-01-15T18:00:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1700b68f0 con 0x7fe150004d00 2024-01-15T18:00:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.189+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe17015eac0 con 0x7fe150004d00 2024-01-15T18:00:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.191+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1700675d0 con 0x7fe150004d00 2024-01-15T18:00:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.192+0000 7fe166a00640 1 -- 172.21.15.26:0/2981505485 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fe160014c40 con 0x7fe150004d00 2024-01-15T18:00:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.192+0000 7fe166a00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe14806b890 0x7fe14806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.193+0000 7fe166a00640 1 -- 172.21.15.26:0/2981505485 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fe16008b2c0 con 0x7fe150004d00 2024-01-15T18:00:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.193+0000 7fe16e000640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe14806b890 0x7fe14806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.194+0000 7fe16e000640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe14806b890 0x7fe14806dd50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe158001eb0 tx=0x7fe158009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:38.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.197+0000 7fe166a00640 1 -- 172.21.15.26:0/2981505485 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe160055a10 con 0x7fe150004d00 2024-01-15T18:00:38.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.390+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe17006c350 con 0x7fe150004d00 2024-01-15T18:00:38.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.391+0000 7fe166a00640 1 -- 172.21.15.26:0/2981505485 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe160059030 con 0x7fe150004d00 2024-01-15T18:00:38.395 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:00:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe14806b890 msgr2=0x7fe14806dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe14806b890 0x7fe14806dd50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe158001eb0 tx=0x7fe158009040 comp rx=0 tx=0).stop 2024-01-15T18:00:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 msgr2=0x7fe1701595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe1701595d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe160009fd0 tx=0x7fe16000f5e0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 shutdown_connections 2024-01-15T18:00:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe1700cb530 0x7fe17015e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe14806b890 0x7fe14806dd50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe1700cb9b0 0x7fe17015fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 --2- 172.21.15.26:0/2981505485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe150004d00 0x7fe1701595d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 >> 172.21.15.26:0/2981505485 conn(0x7fe17009c540 msgr2=0x7fe17006b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:38.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 shutdown_connections 2024-01-15T18:00:38.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.394+0000 7fe175a00640 1 -- 172.21.15.26:0/2981505485 wait complete. 2024-01-15T18:00:38.417 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:00:38.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.628+0000 7fec05800640 1 Processor -- start 2024-01-15T18:00:38.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.628+0000 7fec05800640 1 -- start start 2024-01-15T18:00:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.629+0000 7fec05800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fec00160920 0x7fec000b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.629+0000 7fec05800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec000cf9f0 0x7fec000cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.630+0000 7fec05800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec000d0a10 con 0x7fec00161010 2024-01-15T18:00:38.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.630+0000 7fec05800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec000d03b0 con 0x7fec00160920 2024-01-15T18:00:38.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.630+0000 7fec05800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec000d1f50 con 0x7fec000cf9f0 2024-01-15T18:00:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.630+0000 7febffe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fec00160920 0x7fec000b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.630+0000 7fec04e00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fec00161010 0x7fec001614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:34110/0 (socket says 172.21.15.26:34110) 2024-01-15T18:00:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.630+0000 7fec04e00640 1 -- 172.21.15.26:0/3086424139 learned_addr learned my addr 172.21.15.26:0/3086424139 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.631+0000 7febff400640 1 --2- 172.21.15.26:0/3086424139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec000cf9f0 0x7fec000cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.631+0000 7febffe00640 1 -- 172.21.15.26:0/3086424139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec000cf9f0 msgr2=0x7fec000cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.631+0000 7febffe00640 1 --2- 172.21.15.26:0/3086424139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec000cf9f0 0x7fec000cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.631+0000 7febffe00640 1 -- 172.21.15.26:0/3086424139 >> v1:172.21.15.124:6789/0 conn(0x7fec00161010 legacy=0x7fec001614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.631+0000 7febffe00640 1 -- 172.21.15.26:0/3086424139 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec000cf080 con 0x7fec00160920 2024-01-15T18:00:38.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.633+0000 7febffe00640 1 --2- 172.21.15.26:0/3086424139 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec00160920 0x7fec000b3f30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7febf00097a0 tx=0x7febf002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b14e7178652585 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:38.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.633+0000 7febfea00640 1 -- 172.21.15.26:0/3086424139 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0031830 con 0x7fec00160920 2024-01-15T18:00:38.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.633+0000 7febfea00640 1 -- 172.21.15.26:0/3086424139 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf0031990 con 0x7fec00160920 2024-01-15T18:00:38.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.633+0000 7febfea00640 1 -- 172.21.15.26:0/3086424139 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0031d20 con 0x7fec00160920 2024-01-15T18:00:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.634+0000 7fec05800640 1 -- 172.21.15.26:0/3086424139 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec00160920 msgr2=0x7fec000b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.634+0000 7fec05800640 1 --2- 172.21.15.26:0/3086424139 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec00160920 0x7fec000b3f30 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7febf00097a0 tx=0x7febf002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.634+0000 7fec05800640 1 -- 172.21.15.26:0/3086424139 shutdown_connections 2024-01-15T18:00:38.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.634+0000 7fec05800640 1 --2- 172.21.15.26:0/3086424139 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec000cf9f0 0x7fec000cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.634+0000 7fec05800640 1 --2- 172.21.15.26:0/3086424139 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec00160920 0x7fec000b3f30 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.634+0000 7fec05800640 1 -- 172.21.15.26:0/3086424139 >> 172.21.15.26:0/3086424139 conn(0x7fec000d3090 msgr2=0x7fec000d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.634+0000 7fec05800640 1 -- 172.21.15.26:0/3086424139 shutdown_connections 2024-01-15T18:00:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.634+0000 7fec05800640 1 -- 172.21.15.26:0/3086424139 wait complete. 2024-01-15T18:00:38.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.636+0000 7fec05800640 1 Processor -- start 2024-01-15T18:00:38.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.636+0000 7fec05800640 1 -- start start 2024-01-15T18:00:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.637+0000 7fec05800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec000cf9f0 0x7fec0014d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.637+0000 7fec05800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec00160920 0x7fec000b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.637+0000 7fec05800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec00161010 0x7fec000b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.637+0000 7fec05800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec00157950 con 0x7fec000cf9f0 2024-01-15T18:00:38.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.637+0000 7fec05800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec000cb310 con 0x7fec00161010 2024-01-15T18:00:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.637+0000 7fec05800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec000cb5f0 con 0x7fec00160920 2024-01-15T18:00:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.637+0000 7febffe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec000cf9f0 0x7fec0014d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.637+0000 7febffe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec000cf9f0 0x7fec0014d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36946/0 (socket says 172.21.15.26:36946) 2024-01-15T18:00:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.637+0000 7febffe00640 1 -- 172.21.15.26:0/1530741502 learned_addr learned my addr 172.21.15.26:0/1530741502 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febff400640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec00160920 0x7fec000b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febffe00640 1 -- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec00160920 msgr2=0x7fec000b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febffe00640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec00160920 0x7fec000b6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7fec04e00640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec00161010 0x7fec000b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febffe00640 1 -- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec00161010 msgr2=0x7fec000b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febffe00640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec00161010 0x7fec000b69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febffe00640 1 -- 172.21.15.26:0/1530741502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf0009400 con 0x7fec000cf9f0 2024-01-15T18:00:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febff400640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec00160920 0x7fec000b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febffe00640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec000cf9f0 0x7fec0014d2a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7febf00098c0 tx=0x7febf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febf7e00640 1 -- 172.21.15.26:0/1530741502 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf003a6f0 con 0x7fec000cf9f0 2024-01-15T18:00:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febf7e00640 1 -- 172.21.15.26:0/1530741502 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf003a870 con 0x7fec000cf9f0 2024-01-15T18:00:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7febf7e00640 1 -- 172.21.15.26:0/1530741502 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf003ac20 con 0x7fec000cf9f0 2024-01-15T18:00:38.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.638+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec000cb8d0 con 0x7fec000cf9f0 2024-01-15T18:00:38.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.639+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0014d7e0 con 0x7fec000cf9f0 2024-01-15T18:00:38.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.640+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec000cbdf0 con 0x7fec000cf9f0 2024-01-15T18:00:38.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.641+0000 7febf7e00640 1 -- 172.21.15.26:0/1530741502 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7febf0030380 con 0x7fec000cf9f0 2024-01-15T18:00:38.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.642+0000 7febf7e00640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febd806b890 0x7febd806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:38.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.642+0000 7febf7e00640 1 -- 172.21.15.26:0/1530741502 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7febf00abdd0 con 0x7fec000cf9f0 2024-01-15T18:00:38.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.642+0000 7febff400640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febd806b890 0x7febd806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:38.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.643+0000 7febff400640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febd806b890 0x7febd806dd50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec00157550 tx=0x7febe8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:38.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.646+0000 7febf7e00640 1 -- 172.21.15.26:0/1530741502 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec000cbdf0 con 0x7fec000cf9f0 2024-01-15T18:00:38.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.845+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fec000cbdf0 con 0x7fec000cf9f0 2024-01-15T18:00:38.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.845+0000 7febf7e00640 1 -- 172.21.15.26:0/1530741502 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+65 (secure 0 0 0) 0x7fec000cbdf0 con 0x7fec000cf9f0 2024-01-15T18:00:38.849 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:38.849 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:00:38.849 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:38.849 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-15T18:00:38.850 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-15T18:00:38.850 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:38.850 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:00:38.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.848+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febd806b890 msgr2=0x7febd806dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.848+0000 7fec05800640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febd806b890 0x7febd806dd50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fec00157550 tx=0x7febe8008040 comp rx=0 tx=0).stop 2024-01-15T18:00:38.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec000cf9f0 msgr2=0x7fec0014d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:38.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec000cf9f0 0x7fec0014d2a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7febf00098c0 tx=0x7febf00046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 shutdown_connections 2024-01-15T18:00:38.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec00160920 0x7fec000b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febd806b890 0x7febd806dd50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec00161010 0x7fec000b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 --2- 172.21.15.26:0/1530741502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec000cf9f0 0x7fec0014d2a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 >> 172.21.15.26:0/1530741502 conn(0x7fec000d3090 msgr2=0x7fec0006c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 shutdown_connections 2024-01-15T18:00:38.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:38.849+0000 7fec05800640 1 -- 172.21.15.26:0/1530741502 wait complete. 2024-01-15T18:00:38.872 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:00:38.872 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:00:39.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.096+0000 7f8eca800640 1 Processor -- start 2024-01-15T18:00:39.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.097+0000 7f8eca800640 1 -- start start 2024-01-15T18:00:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.098+0000 7f8eca800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413df80 0x7f8ec415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.099+0000 7f8eca800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec406c470 con 0x7f8ec413e360 2024-01-15T18:00:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.099+0000 7f8eca800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec406c750 con 0x7f8ec40cf9a0 2024-01-15T18:00:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.099+0000 7f8eca800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec406ca30 con 0x7f8ec413df80 2024-01-15T18:00:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.099+0000 7f8ec2a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8ec413e360 0x7f8ec40cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55268/0 (socket says 172.21.15.26:55268) 2024-01-15T18:00:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.099+0000 7f8ec2a00640 1 -- 172.21.15.26:0/3722160267 learned_addr learned my addr 172.21.15.26:0/3722160267 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:39.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.099+0000 7f8ec3400640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413df80 0x7f8ec415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.099+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359710010 0 0) 0x7f8ec406c470 con 0x7f8ec413e360 2024-01-15T18:00:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea4003520 con 0x7f8ec413e360 2024-01-15T18:00:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177249858 0 0) 0x7f8ea4003520 con 0x7f8ec413e360 2024-01-15T18:00:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413df80 msgr2=0x7f8ec415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413df80 0x7f8ec415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 >> v1:172.21.15.124:6789/0 conn(0x7f8ec40cf9a0 legacy=0x7f8ec40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec406cd10 con 0x7f8ec413e360 2024-01-15T18:00:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8eac001670 con 0x7f8ec413e360 2024-01-15T18:00:39.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 >> v1:172.21.15.26:6789/0 conn(0x7f8ec413e360 legacy=0x7f8ec40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40039a0 0x7f8ea4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea40047c0 0x7f8ea4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413df80 0x7f8ea4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec406ca30 con 0x7f8ea40047c0 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea400ba60 con 0x7f8ea40039a0 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea400bc20 con 0x7f8ec413df80 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.100+0000 7f8ec2a00640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea40047c0 0x7f8ea4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.101+0000 7f8ec3400640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40039a0 0x7f8ea4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.101+0000 7f8ec3e00640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413df80 0x7f8ea4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:39.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.101+0000 7f8ec2a00640 1 -- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413df80 msgr2=0x7f8ea4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.101+0000 7f8ec2a00640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413df80 0x7f8ea4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.101+0000 7f8ec2a00640 1 -- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40039a0 msgr2=0x7f8ea4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.101+0000 7f8ec2a00640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40039a0 0x7f8ea4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:39.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.101+0000 7f8ec2a00640 1 -- 172.21.15.26:0/3722160267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4003520 con 0x7f8ea40047c0 2024-01-15T18:00:39.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.102+0000 7f8ec2a00640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea40047c0 0x7f8ea4004c40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8eac001eb0 tx=0x7f8ea40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fe632cc8496cb1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.102+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac005ad0 con 0x7f8ea40047c0 2024-01-15T18:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.102+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eac005c30 con 0x7f8ea40047c0 2024-01-15T18:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.102+0000 7f8ec2000640 1 -- 172.21.15.26:0/3722160267 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eac02c7c0 con 0x7f8ea40047c0 2024-01-15T18:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.103+0000 7f8eca800640 1 -- 172.21.15.26:0/3722160267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea40047c0 msgr2=0x7f8ea4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.103+0000 7f8eca800640 1 --2- 172.21.15.26:0/3722160267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea40047c0 0x7f8ea4004c40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8eac001eb0 tx=0x7f8ea40057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.103+0000 7f8ec3e00640 1 -- 172.21.15.26:0/3722160267 reap_dead start 2024-01-15T18:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.103+0000 7f8eca800640 1 -- 172.21.15.26:0/3722160267 shutdown_connections 2024-01-15T18:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.103+0000 7f8eca800640 1 -- 172.21.15.26:0/3722160267 >> 172.21.15.26:0/3722160267 conn(0x7f8ec406a800 msgr2=0x7f8ec406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.103+0000 7f8eca800640 1 -- 172.21.15.26:0/3722160267 shutdown_connections 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.103+0000 7f8eca800640 1 -- 172.21.15.26:0/3722160267 wait complete. 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.105+0000 7f8eca800640 1 Processor -- start 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.105+0000 7f8eca800640 1 -- start start 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.106+0000 7f8eca800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40047c0 0x7f8ec4153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.106+0000 7f8eca800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413d670 0x7f8ec413cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.106+0000 7f8eca800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec4152f40 0x7f8ec40b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.106+0000 7f8eca800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4156f90 con 0x7f8ec4152f40 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.106+0000 7f8eca800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4151ec0 con 0x7f8ea40047c0 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.106+0000 7f8eca800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec41521a0 con 0x7f8ec413d670 2024-01-15T18:00:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.106+0000 7f8ec3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec4152f40 0x7f8ec40b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.106+0000 7f8ec3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec4152f40 0x7f8ec40b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36972/0 (socket says 172.21.15.26:36972) 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.106+0000 7f8ec3e00640 1 -- 172.21.15.26:0/562050868 learned_addr learned my addr 172.21.15.26:0/562050868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.107+0000 7f8ec3400640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40047c0 0x7f8ec4153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.107+0000 7f8ec2a00640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413d670 0x7f8ec413cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.107+0000 7f8ec3e00640 1 -- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413d670 msgr2=0x7f8ec413cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.107+0000 7f8ec3e00640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413d670 0x7f8ec413cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.107+0000 7f8ec3e00640 1 -- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40047c0 msgr2=0x7f8ec4153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.107+0000 7f8ec3e00640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40047c0 0x7f8ec4153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.107+0000 7f8ec3e00640 1 -- 172.21.15.26:0/562050868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec4152480 con 0x7f8ec4152f40 2024-01-15T18:00:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.107+0000 7f8ec3400640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40047c0 0x7f8ec4153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.107+0000 7f8ec3e00640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec4152f40 0x7f8ec40b2cf0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40cfea0 tx=0x7f8ebc00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.108+0000 7f8ebb400640 1 -- 172.21.15.26:0/562050868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00b070 con 0x7f8ec4152f40 2024-01-15T18:00:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.108+0000 7f8ebb400640 1 -- 172.21.15.26:0/562050868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ebc0054e0 con 0x7f8ec4152f40 2024-01-15T18:00:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.108+0000 7f8ebb400640 1 -- 172.21.15.26:0/562050868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc003810 con 0x7f8ec4152f40 2024-01-15T18:00:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.108+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec4152760 con 0x7f8ec4152f40 2024-01-15T18:00:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.108+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec413d2f0 con 0x7f8ec4152f40 2024-01-15T18:00:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.109+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec413d4b0 con 0x7f8ec4152f40 2024-01-15T18:00:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.110+0000 7f8ebb400640 1 -- 172.21.15.26:0/562050868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f8ebc003970 con 0x7f8ec4152f40 2024-01-15T18:00:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.111+0000 7f8ebb400640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9c06b860 0x7f8e9c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.111+0000 7f8ebb400640 1 -- 172.21.15.26:0/562050868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f8ebc089a80 con 0x7f8ec4152f40 2024-01-15T18:00:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.111+0000 7f8ec3400640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9c06b860 0x7f8e9c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.112+0000 7f8ec3400640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9c06b860 0x7f8e9c06dd20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4151d20 tx=0x7f8eb400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.115+0000 7f8ebb400640 1 -- 172.21.15.26:0/562050868 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ec413d4b0 con 0x7f8ec4152f40 2024-01-15T18:00:39.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:39.310+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8ec413d480 con 0x7f8ec4152f40 2024-01-15T18:00:40.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.018+0000 7f8ebb400640 1 -- 172.21.15.26:0/562050868 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8ebc0180b0 con 0x7f8ec4152f40 2024-01-15T18:00:40.023 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-15T18:00:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9c06b860 msgr2=0x7f8e9c06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9c06b860 0x7f8e9c06dd20 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4151d20 tx=0x7f8eb400b040 comp rx=0 tx=0).stop 2024-01-15T18:00:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec4152f40 msgr2=0x7f8ec40b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec4152f40 0x7f8ec40b2cf0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8ec40cfea0 tx=0x7f8ebc00d890 comp rx=0 tx=0).stop 2024-01-15T18:00:40.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 shutdown_connections 2024-01-15T18:00:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ec413d670 0x7f8ec413cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e9c06b860 0x7f8e9c06dd20 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ec4152f40 0x7f8ec40b2cf0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 --2- 172.21.15.26:0/562050868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40047c0 0x7f8ec4153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 >> 172.21.15.26:0/562050868 conn(0x7f8ec406a800 msgr2=0x7f8ec406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:40.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.023+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 shutdown_connections 2024-01-15T18:00:40.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.025+0000 7f8eca800640 1 -- 172.21.15.26:0/562050868 wait complete. 2024-01-15T18:00:40.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:00:40.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.274+0000 7f42a9a00640 1 Processor -- start 2024-01-15T18:00:40.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.274+0000 7f42a9a00640 1 -- start start 2024-01-15T18:00:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.276+0000 7f42a9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42a413fcb0 0x7f42a4164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.276+0000 7f42a9a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40cdc40 con 0x7f42a415afb0 2024-01-15T18:00:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.276+0000 7f42a9a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a40cdec0 con 0x7f42a413cfa0 2024-01-15T18:00:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.276+0000 7f42a9a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a40ce1a0 con 0x7f42a413fcb0 2024-01-15T18:00:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.276+0000 7f42a9000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f42a413fcb0 0x7f42a4164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.276+0000 7f42a3400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f42a415afb0 0x7f42a413f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55274/0 (socket says 172.21.15.26:55274) 2024-01-15T18:00:40.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.276+0000 7f42a3400640 1 -- 172.21.15.26:0/2073934906 learned_addr learned my addr 172.21.15.26:0/2073934906 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.277+0000 7f42a9000640 1 -- 172.21.15.26:0/2073934906 >> v1:172.21.15.124:6789/0 conn(0x7f42a413cfa0 legacy=0x7f42a40cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.277+0000 7f42a9000640 1 -- 172.21.15.26:0/2073934906 >> v1:172.21.15.26:6789/0 conn(0x7f42a415afb0 legacy=0x7f42a413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.277+0000 7f42a9000640 1 -- 172.21.15.26:0/2073934906 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40ce480 con 0x7f42a413fcb0 2024-01-15T18:00:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.278+0000 7f42a9000640 1 --2- 172.21.15.26:0/2073934906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a413fcb0 0x7f42a4164d80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f429c0097a0 tx=0x7f429c02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6b224c1d857078 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.278+0000 7f42a2a00640 1 -- 172.21.15.26:0/2073934906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c02f8a0 con 0x7f42a413fcb0 2024-01-15T18:00:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.279+0000 7f42a2a00640 1 -- 172.21.15.26:0/2073934906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c02fa00 con 0x7f42a413fcb0 2024-01-15T18:00:40.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.279+0000 7f42a2a00640 1 -- 172.21.15.26:0/2073934906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c02fd90 con 0x7f42a413fcb0 2024-01-15T18:00:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.279+0000 7f42a9a00640 1 -- 172.21.15.26:0/2073934906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a413fcb0 msgr2=0x7f42a4164d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.279+0000 7f42a9a00640 1 --2- 172.21.15.26:0/2073934906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a413fcb0 0x7f42a4164d80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f429c0097a0 tx=0x7f429c02dae0 comp rx=0 tx=0).stop 2024-01-15T18:00:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.279+0000 7f42a9a00640 1 -- 172.21.15.26:0/2073934906 shutdown_connections 2024-01-15T18:00:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.279+0000 7f42a9a00640 1 --2- 172.21.15.26:0/2073934906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a413fcb0 0x7f42a4164d80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.279+0000 7f42a9a00640 1 -- 172.21.15.26:0/2073934906 >> 172.21.15.26:0/2073934906 conn(0x7f42a406a800 msgr2=0x7f42a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.279+0000 7f42a9a00640 1 -- 172.21.15.26:0/2073934906 shutdown_connections 2024-01-15T18:00:40.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.279+0000 7f42a9a00640 1 -- 172.21.15.26:0/2073934906 wait complete. 2024-01-15T18:00:40.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.281+0000 7f42a9a00640 1 Processor -- start 2024-01-15T18:00:40.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.281+0000 7f42a9a00640 1 -- start start 2024-01-15T18:00:40.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.282+0000 7f42a9a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42a413cfa0 0x7f42a4153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:40.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.282+0000 7f42a9a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42a413fcb0 0x7f42a41548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:40.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.282+0000 7f42a9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a415afb0 0x7f42a4156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:40.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.282+0000 7f42a9a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a4155320 con 0x7f42a415afb0 2024-01-15T18:00:40.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.282+0000 7f42a9a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a4155600 con 0x7f42a413fcb0 2024-01-15T18:00:40.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.282+0000 7f42a9a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a41558e0 con 0x7f42a413cfa0 2024-01-15T18:00:40.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a9000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a415afb0 0x7f42a4156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:40.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a9000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a415afb0 0x7f42a4156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36990/0 (socket says 172.21.15.26:36990) 2024-01-15T18:00:40.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a9000640 1 -- 172.21.15.26:0/2751316200 learned_addr learned my addr 172.21.15.26:0/2751316200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:40.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a3e00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42a413cfa0 0x7f42a4153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:40.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a3400640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42a413fcb0 0x7f42a41548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:40.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a9000640 1 -- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42a413cfa0 msgr2=0x7f42a4153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:40.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a9000640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42a413cfa0 0x7f42a4153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:40.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a9000640 1 -- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42a413fcb0 msgr2=0x7f42a41548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:40.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a9000640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42a413fcb0 0x7f42a41548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:40.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a9000640 1 -- 172.21.15.26:0/2751316200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c009400 con 0x7f42a415afb0 2024-01-15T18:00:40.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a3400640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42a413fcb0 0x7f42a41548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:40.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.283+0000 7f42a9000640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a415afb0 0x7f42a4156e40 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f429c0098c0 tx=0x7f429c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:40.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.284+0000 7f429be00640 1 -- 172.21.15.26:0/2751316200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c030b00 con 0x7f42a415afb0 2024-01-15T18:00:40.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.284+0000 7f429be00640 1 -- 172.21.15.26:0/2751316200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c030c60 con 0x7f42a415afb0 2024-01-15T18:00:40.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.284+0000 7f429be00640 1 -- 172.21.15.26:0/2751316200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c042410 con 0x7f42a415afb0 2024-01-15T18:00:40.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.284+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a4155bc0 con 0x7f42a415afb0 2024-01-15T18:00:40.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.284+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a4154120 con 0x7f42a415afb0 2024-01-15T18:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.285+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a4067450 con 0x7f42a415afb0 2024-01-15T18:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.287+0000 7f429be00640 1 -- 172.21.15.26:0/2751316200 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f429c039400 con 0x7f42a415afb0 2024-01-15T18:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.288+0000 7f429be00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f427c06b860 0x7f427c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:40.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.288+0000 7f429be00640 1 -- 172.21.15.26:0/2751316200 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f429c0ac960 con 0x7f42a415afb0 2024-01-15T18:00:40.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.288+0000 7f42a3e00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f427c06b860 0x7f427c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:40.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.289+0000 7f42a3e00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f427c06b860 0x7f427c06dd20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4294007fd0 tx=0x7f4294009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:40.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.292+0000 7f429be00640 1 -- 172.21.15.26:0/2751316200 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f429c077050 con 0x7f42a415afb0 2024-01-15T18:00:40.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:40.488+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f42a40cbff0 con 0x7f42a415afb0 2024-01-15T18:00:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.035+0000 7f429be00640 1 -- 172.21.15.26:0/2751316200 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7f429c07a670 con 0x7f42a415afb0 2024-01-15T18:00:41.039 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-15T18:00:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.038+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f427c06b860 msgr2=0x7f427c06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.038+0000 7f42a9a00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f427c06b860 0x7f427c06dd20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4294007fd0 tx=0x7f4294009040 comp rx=0 tx=0).stop 2024-01-15T18:00:41.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.038+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a415afb0 msgr2=0x7f42a4156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.038+0000 7f42a9a00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a415afb0 0x7f42a4156e40 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f429c0098c0 tx=0x7f429c0046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.039+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 shutdown_connections 2024-01-15T18:00:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.039+0000 7f42a9a00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f427c06b860 0x7f427c06dd20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.039+0000 7f42a9a00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42a415afb0 0x7f42a4156e40 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.039+0000 7f42a9a00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42a413fcb0 0x7f42a41548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.039+0000 7f42a9a00640 1 --2- 172.21.15.26:0/2751316200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42a413cfa0 0x7f42a4153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.039+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 >> 172.21.15.26:0/2751316200 conn(0x7f42a406a800 msgr2=0x7f42a40aef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.039+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 shutdown_connections 2024-01-15T18:00:41.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.039+0000 7f42a9a00640 1 -- 172.21.15.26:0/2751316200 wait complete. 2024-01-15T18:00:41.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:00:41.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.262+0000 7f37b7200640 1 Processor -- start 2024-01-15T18:00:41.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.262+0000 7f37b7200640 1 -- start start 2024-01-15T18:00:41.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.263+0000 7f37b7200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f37b015d580 0x7f37b00cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:41.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.263+0000 7f37b7200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b00ce760 con 0x7f37b00b20d0 2024-01-15T18:00:41.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.264+0000 7f37b7200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b00cf250 con 0x7f37b015d960 2024-01-15T18:00:41.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.264+0000 7f37b7200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b00d0a00 con 0x7f37b015d580 2024-01-15T18:00:41.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.264+0000 7f37b5e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f37b015d580 0x7f37b00cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:41.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.264+0000 7f37b6800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f37b00b20d0 0x7f37b00b0000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55284/0 (socket says 172.21.15.26:55284) 2024-01-15T18:00:41.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.264+0000 7f37b6800640 1 -- 172.21.15.26:0/660173282 learned_addr learned my addr 172.21.15.26:0/660173282 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:41.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.264+0000 7f37b5e00640 1 -- 172.21.15.26:0/660173282 >> v1:172.21.15.124:6789/0 conn(0x7f37b015d960 legacy=0x7f37b00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:41.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.264+0000 7f37b5e00640 1 -- 172.21.15.26:0/660173282 >> v1:172.21.15.26:6789/0 conn(0x7f37b00b20d0 legacy=0x7f37b00b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:41.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.264+0000 7f37b5e00640 1 -- 172.21.15.26:0/660173282 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00d1f50 con 0x7f37b015d580 2024-01-15T18:00:41.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.264+0000 7f37b4a00640 1 -- 172.21.15.26:0/660173282 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304480940 0 0) 0x7f37b00ce760 con 0x7f37b00b20d0 2024-01-15T18:00:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.266+0000 7f37b5e00640 1 --2- 172.21.15.26:0/660173282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 0x7f37b00cf650 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f37a00097a0 tx=0x7f37a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad620f404c29292f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.266+0000 7f37b4a00640 1 -- 172.21.15.26:0/660173282 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a002e070 con 0x7f37b015d580 2024-01-15T18:00:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.266+0000 7f37b4a00640 1 -- 172.21.15.26:0/660173282 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a00315a0 con 0x7f37b015d580 2024-01-15T18:00:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.266+0000 7f37b4a00640 1 -- 172.21.15.26:0/660173282 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0031930 con 0x7f37b015d580 2024-01-15T18:00:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.267+0000 7f37b7200640 1 -- 172.21.15.26:0/660173282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 msgr2=0x7f37b00cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.267+0000 7f37b7200640 1 --2- 172.21.15.26:0/660173282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 0x7f37b00cf650 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f37a00097a0 tx=0x7f37a002b840 comp rx=0 tx=0).stop 2024-01-15T18:00:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.267+0000 7f37b7200640 1 -- 172.21.15.26:0/660173282 shutdown_connections 2024-01-15T18:00:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.267+0000 7f37b7200640 1 --2- 172.21.15.26:0/660173282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 0x7f37b00cf650 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:41.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.267+0000 7f37b7200640 1 -- 172.21.15.26:0/660173282 >> 172.21.15.26:0/660173282 conn(0x7f37b00d3040 msgr2=0x7f37b00d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:41.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.267+0000 7f37b7200640 1 -- 172.21.15.26:0/660173282 shutdown_connections 2024-01-15T18:00:41.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.267+0000 7f37b7200640 1 -- 172.21.15.26:0/660173282 wait complete. 2024-01-15T18:00:41.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.269+0000 7f37b7200640 1 Processor -- start 2024-01-15T18:00:41.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.269+0000 7f37b7200640 1 -- start start 2024-01-15T18:00:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.270+0000 7f37b7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37b00b20d0 0x7f37b015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.270+0000 7f37b7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 0x7f37b015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:41.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.270+0000 7f37b7200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37b015d960 0x7f37b0160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.270+0000 7f37b7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00c72f0 con 0x7f37b015d580 2024-01-15T18:00:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.270+0000 7f37b7200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b015ecd0 con 0x7f37b015d960 2024-01-15T18:00:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.270+0000 7f37b7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00c7540 con 0x7f37b00b20d0 2024-01-15T18:00:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.270+0000 7f37b5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 0x7f37b015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.270+0000 7f37b5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 0x7f37b015ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36996/0 (socket says 172.21.15.26:36996) 2024-01-15T18:00:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.270+0000 7f37b5400640 1 -- 172.21.15.26:0/1968234483 learned_addr learned my addr 172.21.15.26:0/1968234483 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37b5e00640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37b00b20d0 0x7f37b015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37b5400640 1 -- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37b00b20d0 msgr2=0x7f37b015a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:41.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37b5400640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37b00b20d0 0x7f37b015a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37b6800640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37b015d960 0x7f37b0160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37b5400640 1 -- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37b015d960 msgr2=0x7f37b0160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37b5400640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37b015d960 0x7f37b0160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37b5400640 1 -- 172.21.15.26:0/1968234483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a0009400 con 0x7f37b015d580 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37b5400640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 0x7f37b015ddf0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f379800b3f0 tx=0x7f379800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37ae000640 1 -- 172.21.15.26:0/1968234483 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798011830 con 0x7f37b015d580 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37ae000640 1 -- 172.21.15.26:0/1968234483 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3798011990 con 0x7f37b015d580 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37ae000640 1 -- 172.21.15.26:0/1968234483 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798011d40 con 0x7f37b015d580 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.271+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b00c7820 con 0x7f37b015d580 2024-01-15T18:00:41.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.272+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b015e330 con 0x7f37b015d580 2024-01-15T18:00:41.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.273+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b00d2b60 con 0x7f37b015d580 2024-01-15T18:00:41.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.274+0000 7f37ae000640 1 -- 172.21.15.26:0/1968234483 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f379801a400 con 0x7f37b015d580 2024-01-15T18:00:41.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.275+0000 7f37ae000640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f378806b860 0x7f378806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:41.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.275+0000 7f37ae000640 1 -- 172.21.15.26:0/1968234483 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f379808a370 con 0x7f37b015d580 2024-01-15T18:00:41.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.276+0000 7f37b5e00640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f378806b860 0x7f378806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:41.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.277+0000 7f37b5e00640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f378806b860 0x7f378806dd20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f37a0036000 tx=0x7f37a003d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:41.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.280+0000 7f37ae000640 1 -- 172.21.15.26:0/1968234483 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3798054a40 con 0x7f37b015d580 2024-01-15T18:00:41.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:41.516+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f37b00c7d40 con 0x7f37b015d580 2024-01-15T18:00:42.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.052+0000 7f37ae000640 1 -- 172.21.15.26:0/1968234483 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v31) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3798058060 con 0x7f37b015d580 2024-01-15T18:00:42.062 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 8 and data pool 9 2024-01-15T18:00:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.064+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f378806b860 msgr2=0x7f378806dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.064+0000 7f37b7200640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f378806b860 0x7f378806dd20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f37a0036000 tx=0x7f37a003d040 comp rx=0 tx=0).stop 2024-01-15T18:00:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.064+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 msgr2=0x7f37b015ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.064+0000 7f37b7200640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 0x7f37b015ddf0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f379800b3f0 tx=0x7f379800b8b0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.067+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 shutdown_connections 2024-01-15T18:00:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.067+0000 7f37b7200640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f378806b860 0x7f378806dd20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.067+0000 7f37b7200640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37b015d960 0x7f37b0160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.067+0000 7f37b7200640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37b015d580 0x7f37b015ddf0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.067+0000 7f37b7200640 1 --2- 172.21.15.26:0/1968234483 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37b00b20d0 0x7f37b015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.067+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 >> 172.21.15.26:0/1968234483 conn(0x7f37b00d3040 msgr2=0x7f37b015b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.068+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 shutdown_connections 2024-01-15T18:00:42.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.069+0000 7f37b7200640 1 -- 172.21.15.26:0/1968234483 wait complete. 2024-01-15T18:00:42.101 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:00:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.279+0000 7fcb2f200640 1 Processor -- start 2024-01-15T18:00:42.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.279+0000 7fcb2f200640 1 -- start start 2024-01-15T18:00:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.280+0000 7fcb2f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb30160920 0x7fcb300b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.280+0000 7fcb2f200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcb300cf9f0 0x7fcb300cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.280+0000 7fcb2f200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb300d0a10 con 0x7fcb30161010 2024-01-15T18:00:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.281+0000 7fcb2f200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb300d03b0 con 0x7fcb300cf9f0 2024-01-15T18:00:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.281+0000 7fcb2f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb300d1f50 con 0x7fcb30160920 2024-01-15T18:00:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.281+0000 7fcb2de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb30160920 0x7fcb300b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.281+0000 7fcb2de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb30160920 0x7fcb300b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:35634/0 (socket says 172.21.15.26:35634) 2024-01-15T18:00:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.281+0000 7fcb2de00640 1 -- 172.21.15.26:0/728475761 learned_addr learned my addr 172.21.15.26:0/728475761 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.281+0000 7fcb2d400640 1 --2- 172.21.15.26:0/728475761 >> v2:172.21.15.124:3300/0 conn(0x7fcb300cf9f0 0x7fcb300cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.282+0000 7fcb2de00640 1 -- 172.21.15.26:0/728475761 >> v2:172.21.15.124:3300/0 conn(0x7fcb300cf9f0 msgr2=0x7fcb300cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.282+0000 7fcb2de00640 1 --2- 172.21.15.26:0/728475761 >> v2:172.21.15.124:3300/0 conn(0x7fcb300cf9f0 0x7fcb300cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.282+0000 7fcb2de00640 1 -- 172.21.15.26:0/728475761 >> v1:172.21.15.124:6789/0 conn(0x7fcb30161010 legacy=0x7fcb301614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.282+0000 7fcb2de00640 1 -- 172.21.15.26:0/728475761 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb300cf080 con 0x7fcb30160920 2024-01-15T18:00:42.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.283+0000 7fcb2de00640 1 --2- 172.21.15.26:0/728475761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb30160920 0x7fcb300b3f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009870 tx=0x7fcb2002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40f699aaf7ebf404 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.284+0000 7fcb2ca00640 1 -- 172.21.15.26:0/728475761 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20030840 con 0x7fcb30160920 2024-01-15T18:00:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.284+0000 7fcb2ca00640 1 -- 172.21.15.26:0/728475761 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb200309a0 con 0x7fcb30160920 2024-01-15T18:00:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.284+0000 7fcb2ca00640 1 -- 172.21.15.26:0/728475761 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb20030d30 con 0x7fcb30160920 2024-01-15T18:00:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.285+0000 7fcb2f200640 1 -- 172.21.15.26:0/728475761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb30160920 msgr2=0x7fcb300b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.285+0000 7fcb2f200640 1 --2- 172.21.15.26:0/728475761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb30160920 0x7fcb300b3f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009870 tx=0x7fcb2002b980 comp rx=0 tx=0).stop 2024-01-15T18:00:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.285+0000 7fcb2f200640 1 -- 172.21.15.26:0/728475761 shutdown_connections 2024-01-15T18:00:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.285+0000 7fcb2f200640 1 --2- 172.21.15.26:0/728475761 >> v2:172.21.15.124:3300/0 conn(0x7fcb300cf9f0 0x7fcb300cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.285+0000 7fcb2f200640 1 --2- 172.21.15.26:0/728475761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb30160920 0x7fcb300b3f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.285+0000 7fcb2f200640 1 -- 172.21.15.26:0/728475761 >> 172.21.15.26:0/728475761 conn(0x7fcb300d3090 msgr2=0x7fcb300d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:42.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.285+0000 7fcb2f200640 1 -- 172.21.15.26:0/728475761 shutdown_connections 2024-01-15T18:00:42.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.285+0000 7fcb2f200640 1 -- 172.21.15.26:0/728475761 wait complete. 2024-01-15T18:00:42.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.287+0000 7fcb2f200640 1 Processor -- start 2024-01-15T18:00:42.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.287+0000 7fcb2f200640 1 -- start start 2024-01-15T18:00:42.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.288+0000 7fcb2f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb300cf9f0 0x7fcb30149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.288+0000 7fcb2f200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb30160920 0x7fcb300cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.288+0000 7fcb2f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb30161010 0x7fcb3015f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.288+0000 7fcb2f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3015f510 con 0x7fcb30161010 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.288+0000 7fcb2f200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3015e660 con 0x7fcb30160920 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.288+0000 7fcb2f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3015e900 con 0x7fcb300cf9f0 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.288+0000 7fcb2e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb30161010 0x7fcb3015f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.288+0000 7fcb2e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb30161010 0x7fcb3015f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36856/0 (socket says 172.21.15.26:36856) 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.288+0000 7fcb2e800640 1 -- 172.21.15.26:0/3415500541 learned_addr learned my addr 172.21.15.26:0/3415500541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb2de00640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb300cf9f0 0x7fcb30149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb2d400640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb30160920 0x7fcb300cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb2e800640 1 -- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb300cf9f0 msgr2=0x7fcb30149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb2e800640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb300cf9f0 0x7fcb30149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb2e800640 1 -- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb30160920 msgr2=0x7fcb300cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb2e800640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb30160920 0x7fcb300cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb2e800640 1 -- 172.21.15.26:0/3415500541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200094d0 con 0x7fcb30161010 2024-01-15T18:00:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb2e800640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb30161010 0x7fcb3015f9a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800c7a0 tx=0x7fcb2800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb26000640 1 -- 172.21.15.26:0/3415500541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28003b70 con 0x7fcb30161010 2024-01-15T18:00:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb26000640 1 -- 172.21.15.26:0/3415500541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb28003cf0 con 0x7fcb30161010 2024-01-15T18:00:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.289+0000 7fcb26000640 1 -- 172.21.15.26:0/3415500541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28003ea0 con 0x7fcb30161010 2024-01-15T18:00:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.290+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3015ebe0 con 0x7fcb30161010 2024-01-15T18:00:42.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.290+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3015f1a0 con 0x7fcb30161010 2024-01-15T18:00:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.291+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3015f360 con 0x7fcb30161010 2024-01-15T18:00:42.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.292+0000 7fcb26000640 1 -- 172.21.15.26:0/3415500541 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fcb2800aa90 con 0x7fcb30161010 2024-01-15T18:00:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.293+0000 7fcb26000640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb0806b860 0x7fcb0806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.293+0000 7fcb26000640 1 -- 172.21.15.26:0/3415500541 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcb2808a500 con 0x7fcb30161010 2024-01-15T18:00:42.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.293+0000 7fcb2de00640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb0806b860 0x7fcb0806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.295+0000 7fcb2de00640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb0806b860 0x7fcb0806dd20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcb20036000 tx=0x7fcb20008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:42.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.297+0000 7fcb26000640 1 -- 172.21.15.26:0/3415500541 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb28054aa0 con 0x7fcb30161010 2024-01-15T18:00:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.483+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb30067450 con 0x7fcb30161010 2024-01-15T18:00:42.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.485+0000 7fcb26000640 1 -- 172.21.15.26:0/3415500541 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fcb280580c0 con 0x7fcb30161010 2024-01-15T18:00:42.489 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:42.489 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":59,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:00:41.517826+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:00:39.364932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-15T18:00:40.543640+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:00:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb0806b860 msgr2=0x7fcb0806dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb0806b860 0x7fcb0806dd20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcb20036000 tx=0x7fcb20008030 comp rx=0 tx=0).stop 2024-01-15T18:00:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb30161010 msgr2=0x7fcb3015f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb30161010 0x7fcb3015f9a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800c7a0 tx=0x7fcb2800cc60 comp rx=0 tx=0).stop 2024-01-15T18:00:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 shutdown_connections 2024-01-15T18:00:42.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb0806b860 0x7fcb0806dd20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb30161010 0x7fcb3015f9a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb30160920 0x7fcb300cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 --2- 172.21.15.26:0/3415500541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb300cf9f0 0x7fcb30149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 >> 172.21.15.26:0/3415500541 conn(0x7fcb300d3090 msgr2=0x7fcb300b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 shutdown_connections 2024-01-15T18:00:42.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.487+0000 7fcb2f200640 1 -- 172.21.15.26:0/3415500541 wait complete. 2024-01-15T18:00:42.509 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:00:42.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.724+0000 7fab4f200640 1 Processor -- start 2024-01-15T18:00:42.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.724+0000 7fab4f200640 1 -- start start 2024-01-15T18:00:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.726+0000 7fab4f200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab50160920 0x7fab500b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.726+0000 7fab4f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 0x7fab50161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.726+0000 7fab4f200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab500d0a10 con 0x7fab500cf9f0 2024-01-15T18:00:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.726+0000 7fab4f200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab500d0330 con 0x7fab50160920 2024-01-15T18:00:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.726+0000 7fab4f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab500d1f50 con 0x7fab500cfea0 2024-01-15T18:00:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.726+0000 7fab4de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab50160920 0x7fab500b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.726+0000 7fab4de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab50160920 0x7fab500b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36860/0 (socket says 172.21.15.26:36860) 2024-01-15T18:00:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.726+0000 7fab4de00640 1 -- 172.21.15.26:0/3815238903 learned_addr learned my addr 172.21.15.26:0/3815238903 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.727+0000 7fab4e800640 1 --2- 172.21.15.26:0/3815238903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 0x7fab50161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.727+0000 7fab4de00640 1 -- 172.21.15.26:0/3815238903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 msgr2=0x7fab50161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:42.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.727+0000 7fab4de00640 1 --2- 172.21.15.26:0/3815238903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 0x7fab50161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.727+0000 7fab4de00640 1 -- 172.21.15.26:0/3815238903 >> v1:172.21.15.124:6789/0 conn(0x7fab500cf9f0 legacy=0x7fab50161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.727+0000 7fab4de00640 1 -- 172.21.15.26:0/3815238903 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab500cf080 con 0x7fab50160920 2024-01-15T18:00:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.728+0000 7fab4de00640 1 --2- 172.21.15.26:0/3815238903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 0x7fab500b3f30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fab40009870 tx=0x7fab4002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d577a09aebc72942 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:42.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.728+0000 7fab4ca00640 1 -- 172.21.15.26:0/3815238903 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40030840 con 0x7fab50160920 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.729+0000 7fab4ca00640 1 -- 172.21.15.26:0/3815238903 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab400309a0 con 0x7fab50160920 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.729+0000 7fab4ca00640 1 -- 172.21.15.26:0/3815238903 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40030d30 con 0x7fab50160920 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.729+0000 7fab4f200640 1 -- 172.21.15.26:0/3815238903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 msgr2=0x7fab500b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.729+0000 7fab4f200640 1 --2- 172.21.15.26:0/3815238903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 0x7fab500b3f30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fab40009870 tx=0x7fab4002b980 comp rx=0 tx=0).stop 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.729+0000 7fab4f200640 1 -- 172.21.15.26:0/3815238903 shutdown_connections 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.729+0000 7fab4f200640 1 --2- 172.21.15.26:0/3815238903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 0x7fab50161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.729+0000 7fab4f200640 1 --2- 172.21.15.26:0/3815238903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 0x7fab500b3f30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.729+0000 7fab4f200640 1 -- 172.21.15.26:0/3815238903 >> 172.21.15.26:0/3815238903 conn(0x7fab500d3090 msgr2=0x7fab500d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.730+0000 7fab4f200640 1 -- 172.21.15.26:0/3815238903 shutdown_connections 2024-01-15T18:00:42.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.730+0000 7fab4f200640 1 -- 172.21.15.26:0/3815238903 wait complete. 2024-01-15T18:00:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.731+0000 7fab4f200640 1 Processor -- start 2024-01-15T18:00:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.731+0000 7fab4f200640 1 -- start start 2024-01-15T18:00:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.732+0000 7fab4f200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab500cf9f0 0x7fab500cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.732+0000 7fab4f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 0x7fab500c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.732+0000 7fab4f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 0x7fab500c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.732+0000 7fab4f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab500ca160 con 0x7fab50160920 2024-01-15T18:00:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.732+0000 7fab4f200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab500ca720 con 0x7fab500cf9f0 2024-01-15T18:00:42.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.732+0000 7fab4f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab501f38d0 con 0x7fab500cfea0 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 0x7fab500c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 0x7fab500c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36866/0 (socket says 172.21.15.26:36866) 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4e800640 1 -- 172.21.15.26:0/1249524386 learned_addr learned my addr 172.21.15.26:0/1249524386 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4de00640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab500cf9f0 0x7fab500cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4d400640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 0x7fab500c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4e800640 1 -- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 msgr2=0x7fab500c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4e800640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 0x7fab500c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4e800640 1 -- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab500cf9f0 msgr2=0x7fab500cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4e800640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab500cf9f0 0x7fab500cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.733+0000 7fab4e800640 1 -- 172.21.15.26:0/1249524386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400094d0 con 0x7fab50160920 2024-01-15T18:00:42.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.734+0000 7fab4e800640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 0x7fab500c8b00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fab48007e70 tx=0x7fab4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:42.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.734+0000 7fab46000640 1 -- 172.21.15.26:0/1249524386 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48014620 con 0x7fab50160920 2024-01-15T18:00:42.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.734+0000 7fab46000640 1 -- 172.21.15.26:0/1249524386 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab480147a0 con 0x7fab50160920 2024-01-15T18:00:42.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.734+0000 7fab46000640 1 -- 172.21.15.26:0/1249524386 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48014b50 con 0x7fab50160920 2024-01-15T18:00:42.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.734+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab501f3bb0 con 0x7fab50160920 2024-01-15T18:00:42.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.734+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab500cb0f0 con 0x7fab50160920 2024-01-15T18:00:42.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.735+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab50067450 con 0x7fab50160920 2024-01-15T18:00:42.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.736+0000 7fab46000640 1 -- 172.21.15.26:0/1249524386 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fab48014cb0 con 0x7fab50160920 2024-01-15T18:00:42.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.737+0000 7fab46000640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab2806b860 0x7fab2806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:42.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.738+0000 7fab46000640 1 -- 172.21.15.26:0/1249524386 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fab4808afe0 con 0x7fab50160920 2024-01-15T18:00:42.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.738+0000 7fab4de00640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab2806b860 0x7fab2806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:42.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.739+0000 7fab4de00640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab2806b860 0x7fab2806dd20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fab40036000 tx=0x7fab40008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:42.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.742+0000 7fab46000640 1 -- 172.21.15.26:0/1249524386 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab480555d0 con 0x7fab50160920 2024-01-15T18:00:42.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.933+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab500ca860 con 0x7fab50160920 2024-01-15T18:00:42.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.936+0000 7fab46000640 1 -- 172.21.15.26:0/1249524386 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16597 (secure 0 0 0) 0x7fab48058bf0 con 0x7fab50160920 2024-01-15T18:00:42.939 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:42.940 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":59,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:00:41.517826+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:00:39.364932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-15T18:00:40.543640+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:00:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab2806b860 msgr2=0x7fab2806dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab2806b860 0x7fab2806dd20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fab40036000 tx=0x7fab40008030 comp rx=0 tx=0).stop 2024-01-15T18:00:42.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 msgr2=0x7fab500c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 0x7fab500c8b00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fab48007e70 tx=0x7fab4800e4b0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 shutdown_connections 2024-01-15T18:00:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab500cfea0 0x7fab500c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fab2806b860 0x7fab2806dd20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab50160920 0x7fab500c8b00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 --2- 172.21.15.26:0/1249524386 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab500cf9f0 0x7fab500cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 >> 172.21.15.26:0/1249524386 conn(0x7fab500d3090 msgr2=0x7fab5006c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 shutdown_connections 2024-01-15T18:00:42.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:42.939+0000 7fab4f200640 1 -- 172.21.15.26:0/1249524386 wait complete. 2024-01-15T18:00:42.963 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.185+0000 7f8b32e00640 1 Processor -- start 2024-01-15T18:00:43.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.185+0000 7f8b32e00640 1 -- start start 2024-01-15T18:00:43.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.187+0000 7f8b32e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8b2c15d4b0 0x7f8b2c0d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:43.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.187+0000 7f8b32e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b2c0cf9a0 0x7f8b2c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.187+0000 7f8b32e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b2c0d0320 con 0x7f8b2c0cddf0 2024-01-15T18:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.187+0000 7f8b32e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c0b2730 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.187+0000 7f8b32e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b2c0d1ec0 con 0x7f8b2c15d4b0 2024-01-15T18:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.187+0000 7f8b31000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b2c0cf9a0 0x7f8b2c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.187+0000 7f8b31000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b2c0cf9a0 0x7f8b2c0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36876/0 (socket says 172.21.15.26:36876) 2024-01-15T18:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.187+0000 7f8b31000640 1 -- 172.21.15.26:0/213156363 learned_addr learned my addr 172.21.15.26:0/213156363 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.188+0000 7f8b31a00640 1 --2- 172.21.15.26:0/213156363 >> v2:172.21.15.124:3300/0 conn(0x7f8b2c15d4b0 0x7f8b2c0d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.188+0000 7f8b31000640 1 -- 172.21.15.26:0/213156363 >> v2:172.21.15.124:3300/0 conn(0x7f8b2c15d4b0 msgr2=0x7f8b2c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:43.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.188+0000 7f8b31000640 1 --2- 172.21.15.26:0/213156363 >> v2:172.21.15.124:3300/0 conn(0x7f8b2c15d4b0 0x7f8b2c0d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:43.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.188+0000 7f8b31000640 1 -- 172.21.15.26:0/213156363 >> v1:172.21.15.26:6789/0 conn(0x7f8b2c0cddf0 legacy=0x7f8b2c0ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:43.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.188+0000 7f8b31000640 1 -- 172.21.15.26:0/213156363 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c0b29d0 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.188+0000 7f8b31a00640 1 --2- 172.21.15.26:0/213156363 >> v2:172.21.15.124:3300/0 conn(0x7f8b2c15d4b0 0x7f8b2c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.189+0000 7f8b31000640 1 --2- 172.21.15.26:0/213156363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 0x7f8b2c0cfde0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c08b5b0 tx=0x7f8b1402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958c7250991aa1b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.189+0000 7f8b2be00640 1 -- 172.21.15.26:0/213156363 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14030860 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.189+0000 7f8b2be00640 1 -- 172.21.15.26:0/213156363 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b140309c0 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.189+0000 7f8b2be00640 1 -- 172.21.15.26:0/213156363 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14030d50 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.190+0000 7f8b32e00640 1 -- 172.21.15.26:0/213156363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 msgr2=0x7f8b2c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.190+0000 7f8b32e00640 1 --2- 172.21.15.26:0/213156363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 0x7f8b2c0cfde0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c08b5b0 tx=0x7f8b1402b9a0 comp rx=0 tx=0).stop 2024-01-15T18:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.190+0000 7f8b32e00640 1 -- 172.21.15.26:0/213156363 shutdown_connections 2024-01-15T18:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.190+0000 7f8b32e00640 1 --2- 172.21.15.26:0/213156363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 0x7f8b2c0cfde0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.190+0000 7f8b32e00640 1 --2- 172.21.15.26:0/213156363 >> v2:172.21.15.124:3300/0 conn(0x7f8b2c15d4b0 0x7f8b2c0d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.190+0000 7f8b32e00640 1 -- 172.21.15.26:0/213156363 >> 172.21.15.26:0/213156363 conn(0x7f8b2c0d3000 msgr2=0x7f8b2c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.190+0000 7f8b32e00640 1 -- 172.21.15.26:0/213156363 shutdown_connections 2024-01-15T18:00:43.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.190+0000 7f8b32e00640 1 -- 172.21.15.26:0/213156363 wait complete. 2024-01-15T18:00:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.192+0000 7f8b32e00640 1 Processor -- start 2024-01-15T18:00:43.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.192+0000 7f8b32e00640 1 -- start start 2024-01-15T18:00:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.193+0000 7f8b32e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b2c0cddf0 0x7f8b2c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.193+0000 7f8b32e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 0x7f8b2c0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.193+0000 7f8b32e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2c15d4b0 0x7f8b2c1f2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.193+0000 7f8b32e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c0cd3e0 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.193+0000 7f8b32e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c0cd160 con 0x7f8b2c0cddf0 2024-01-15T18:00:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.193+0000 7f8b32e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c09e2f0 con 0x7f8b2c15d4b0 2024-01-15T18:00:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.193+0000 7f8b31000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 0x7f8b2c0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.193+0000 7f8b31000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 0x7f8b2c0ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36878/0 (socket says 172.21.15.26:36878) 2024-01-15T18:00:43.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.193+0000 7f8b31000640 1 -- 172.21.15.26:0/3407908674 learned_addr learned my addr 172.21.15.26:0/3407908674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:43.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.194+0000 7f8b31a00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b2c0cddf0 0x7f8b2c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:43.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.194+0000 7f8b31000640 1 -- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2c15d4b0 msgr2=0x7f8b2c1f2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:43.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.194+0000 7f8b32400640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2c15d4b0 0x7f8b2c1f2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:43.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.194+0000 7f8b31000640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2c15d4b0 0x7f8b2c1f2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:43.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.194+0000 7f8b31000640 1 -- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b2c0cddf0 msgr2=0x7f8b2c15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:43.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.194+0000 7f8b31000640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b2c0cddf0 0x7f8b2c15bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:43.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.194+0000 7f8b31000640 1 -- 172.21.15.26:0/3407908674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b140094d0 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.194+0000 7f8b31a00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b2c0cddf0 0x7f8b2c15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:43.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.194+0000 7f8b31000640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 0x7f8b2c0ccc20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8b1402b9e0 tx=0x7f8b14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:43.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.195+0000 7f8b29600640 1 -- 172.21.15.26:0/3407908674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14030830 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.195+0000 7f8b29600640 1 -- 172.21.15.26:0/3407908674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14043400 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.195+0000 7f8b29600640 1 -- 172.21.15.26:0/3407908674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b140437b0 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.195+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b2c15da10 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.195+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b2c09e8b0 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.196+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2c067450 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.197+0000 7f8b29600640 1 -- 172.21.15.26:0/3407908674 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f8b14043910 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.198+0000 7f8b29600640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0406b860 0x7f8b0406dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:43.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.198+0000 7f8b29600640 1 -- 172.21.15.26:0/3407908674 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8b1403e070 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.199+0000 7f8b31a00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0406b860 0x7f8b0406dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:43.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.200+0000 7f8b31a00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0406b860 0x7f8b0406dd20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c006fd0 tx=0x7f8b1c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:43.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.202+0000 7f8b29600640 1 -- 172.21.15.26:0/3407908674 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b14078120 con 0x7f8b2c0cf9a0 2024-01-15T18:00:43.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:43.434+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8b2c0b3d80 con 0x7f8b2c0cf9a0 2024-01-15T18:00:44.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.072+0000 7f8b29600640 1 -- 172.21.15.26:0/3407908674 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8b1407b740 con 0x7f8b2c0cf9a0 2024-01-15T18:00:44.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0406b860 msgr2=0x7f8b0406dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:44.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0406b860 0x7f8b0406dd20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c006fd0 tx=0x7f8b1c008040 comp rx=0 tx=0).stop 2024-01-15T18:00:44.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 msgr2=0x7f8b2c0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:44.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 0x7f8b2c0ccc20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8b1402b9e0 tx=0x7f8b14004730 comp rx=0 tx=0).stop 2024-01-15T18:00:44.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 shutdown_connections 2024-01-15T18:00:44.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b2c15d4b0 0x7f8b2c1f2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0406b860 0x7f8b0406dd20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b2c0cf9a0 0x7f8b2c0ccc20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 --2- 172.21.15.26:0/3407908674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b2c0cddf0 0x7f8b2c15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.075+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 >> 172.21.15.26:0/3407908674 conn(0x7f8b2c0d3000 msgr2=0x7f8b2c0b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:44.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.076+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 shutdown_connections 2024-01-15T18:00:44.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.076+0000 7f8b32e00640 1 -- 172.21.15.26:0/3407908674 wait complete. 2024-01-15T18:00:44.100 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:00:44.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:00:44.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.324+0000 7fc56da00640 1 Processor -- start 2024-01-15T18:00:44.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.324+0000 7fc56da00640 1 -- start start 2024-01-15T18:00:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.326+0000 7fc56da00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5680ccb50 0x7fc568070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.326+0000 7fc56da00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56815f4e0 con 0x7fc56815bf10 2024-01-15T18:00:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.326+0000 7fc56da00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56815c3c0 con 0x7fc5680adf20 2024-01-15T18:00:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.326+0000 7fc56da00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56815f970 con 0x7fc5680ccb50 2024-01-15T18:00:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.326+0000 7fc56d000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc56815bf10 0x7fc568143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54266/0 (socket says 172.21.15.26:54266) 2024-01-15T18:00:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.326+0000 7fc56d000640 1 -- 172.21.15.26:0/3458586818 learned_addr learned my addr 172.21.15.26:0/3458586818 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:44.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.327+0000 7fc567400640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5680ccb50 0x7fc568070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.327+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585551390 0 0) 0x7fc56815f4e0 con 0x7fc56815bf10 2024-01-15T18:00:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.327+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc548003520 con 0x7fc56815bf10 2024-01-15T18:00:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.327+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2011411607 0 0) 0x7fc548003520 con 0x7fc56815bf10 2024-01-15T18:00:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5680ccb50 msgr2=0x7fc568070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5680ccb50 0x7fc568070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 >> v1:172.21.15.124:6789/0 conn(0x7fc5680adf20 legacy=0x7fc5680cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc568071130 con 0x7fc56815bf10 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fc560001670 con 0x7fc56815bf10 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 >> v1:172.21.15.26:6789/0 conn(0x7fc56815bf10 legacy=0x7fc568143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5480039a0 0x7fc548003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5680ccb50 0x7fc5480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc548005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56815f970 con 0x7fc548004d00 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54800bab0 con 0x7fc5480039a0 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54800bc70 con 0x7fc5680ccb50 2024-01-15T18:00:44.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc56d000640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc548005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc567400640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5680ccb50 0x7fc5480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.328+0000 7fc567e00640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5480039a0 0x7fc548003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.329+0000 7fc56d000640 1 -- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5680ccb50 msgr2=0x7fc5480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.329+0000 7fc56d000640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5680ccb50 0x7fc5480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.329+0000 7fc56d000640 1 -- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5480039a0 msgr2=0x7fc548003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.329+0000 7fc56d000640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5480039a0 0x7fc548003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.329+0000 7fc56d000640 1 -- 172.21.15.26:0/3458586818 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc548003520 con 0x7fc548004d00 2024-01-15T18:00:44.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.330+0000 7fc56d000640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc548005180 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc56000c210 tx=0x7fc5480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3520f0f136d1f41b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.330+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc560005ad0 con 0x7fc548004d00 2024-01-15T18:00:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.330+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc560005c30 con 0x7fc548004d00 2024-01-15T18:00:44.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.330+0000 7fc566a00640 1 -- 172.21.15.26:0/3458586818 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56002cad0 con 0x7fc548004d00 2024-01-15T18:00:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.331+0000 7fc56da00640 1 -- 172.21.15.26:0/3458586818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 msgr2=0x7fc548005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.331+0000 7fc56da00640 1 --2- 172.21.15.26:0/3458586818 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc548005180 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc56000c210 tx=0x7fc5480057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.331+0000 7fc56d000640 1 -- 172.21.15.26:0/3458586818 reap_dead start 2024-01-15T18:00:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.331+0000 7fc56da00640 1 -- 172.21.15.26:0/3458586818 shutdown_connections 2024-01-15T18:00:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.331+0000 7fc56da00640 1 -- 172.21.15.26:0/3458586818 >> 172.21.15.26:0/3458586818 conn(0x7fc56806a7d0 msgr2=0x7fc56806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.331+0000 7fc56da00640 1 -- 172.21.15.26:0/3458586818 shutdown_connections 2024-01-15T18:00:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.331+0000 7fc56da00640 1 -- 172.21.15.26:0/3458586818 wait complete. 2024-01-15T18:00:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.333+0000 7fc56da00640 1 Processor -- start 2024-01-15T18:00:44.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.333+0000 7fc56da00640 1 -- start start 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.334+0000 7fc56da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc5680a9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.334+0000 7fc56da00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc568155990 0x7fc5681492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.334+0000 7fc56da00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc568155df0 0x7fc568149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.334+0000 7fc56da00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5680a9580 con 0x7fc548004d00 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.334+0000 7fc56da00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5680aff10 con 0x7fc568155df0 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.334+0000 7fc56da00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5680afc30 con 0x7fc568155990 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.334+0000 7fc567e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc5680a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.334+0000 7fc567e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc5680a9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36900/0 (socket says 172.21.15.26:36900) 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.334+0000 7fc567e00640 1 -- 172.21.15.26:0/1332094661 learned_addr learned my addr 172.21.15.26:0/1332094661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:44.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc567400640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc568155990 0x7fc5681492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc56d000640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc568155df0 0x7fc568149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc567e00640 1 -- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc568155990 msgr2=0x7fc5681492d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc567e00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc568155990 0x7fc5681492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc567e00640 1 -- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc568155df0 msgr2=0x7fc568149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc567e00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc568155df0 0x7fc568149810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc567e00640 1 -- 172.21.15.26:0/1332094661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5680a8560 con 0x7fc548004d00 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc56d000640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc568155df0 0x7fc568149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc567e00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc5680a9040 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc55800e8f0 tx=0x7fc55800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc55fe00640 1 -- 172.21.15.26:0/1332094661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558003b80 con 0x7fc548004d00 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc55fe00640 1 -- 172.21.15.26:0/1332094661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc558003ce0 con 0x7fc548004d00 2024-01-15T18:00:44.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc55fe00640 1 -- 172.21.15.26:0/1332094661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5580052f0 con 0x7fc548004d00 2024-01-15T18:00:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.335+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5680a8840 con 0x7fc548004d00 2024-01-15T18:00:44.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.336+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5680aef20 con 0x7fc548004d00 2024-01-15T18:00:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.337+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc568070eb0 con 0x7fc548004d00 2024-01-15T18:00:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.337+0000 7fc55fe00640 1 -- 172.21.15.26:0/1332094661 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fc558005450 con 0x7fc548004d00 2024-01-15T18:00:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.338+0000 7fc55fe00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc54006b860 0x7fc54006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.339+0000 7fc55fe00640 1 -- 172.21.15.26:0/1332094661 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc55808a7f0 con 0x7fc548004d00 2024-01-15T18:00:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.339+0000 7fc567400640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc54006b860 0x7fc54006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:44.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.340+0000 7fc567400640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc54006b860 0x7fc54006dd20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc5681540e0 tx=0x7fc550008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:44.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.343+0000 7fc55fe00640 1 -- 172.21.15.26:0/1332094661 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc558054d40 con 0x7fc548004d00 2024-01-15T18:00:44.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:44.575+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc568067450 con 0x7fc548004d00 2024-01-15T18:00:45.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.075+0000 7fc55fe00640 1 -- 172.21.15.26:0/1332094661 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc558058360 con 0x7fc548004d00 2024-01-15T18:00:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.078+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc54006b860 msgr2=0x7fc54006dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.078+0000 7fc56da00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc54006b860 0x7fc54006dd20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc5681540e0 tx=0x7fc550008040 comp rx=0 tx=0).stop 2024-01-15T18:00:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.078+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 msgr2=0x7fc5680a9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.078+0000 7fc56da00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc5680a9040 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc55800e8f0 tx=0x7fc55800edb0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.078+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 shutdown_connections 2024-01-15T18:00:45.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.078+0000 7fc56da00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc568155990 0x7fc5681492d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.079+0000 7fc56da00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc54006b860 0x7fc54006dd20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.079+0000 7fc56da00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc568155df0 0x7fc568149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.079+0000 7fc56da00640 1 --2- 172.21.15.26:0/1332094661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc548004d00 0x7fc5680a9040 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.079+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 >> 172.21.15.26:0/1332094661 conn(0x7fc56806a7d0 msgr2=0x7fc56806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.079+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 shutdown_connections 2024-01-15T18:00:45.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.079+0000 7fc56da00640 1 -- 172.21.15.26:0/1332094661 wait complete. 2024-01-15T18:00:45.103 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:00:45.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:00:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.320+0000 7f50c4c00640 1 Processor -- start 2024-01-15T18:00:45.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.320+0000 7f50c4c00640 1 -- start start 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.322+0000 7f50c4c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f50c0146650 0x7f50c00b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.322+0000 7f50c4c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50c00726f0 0x7f50c00cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.322+0000 7f50c4c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c015e030 0x7f50c015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.322+0000 7f50c4c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c015dba0 con 0x7f50c00726f0 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.322+0000 7f50c4c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c006f800 con 0x7f50c0146650 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.322+0000 7f50c4c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c006f240 con 0x7f50c015e030 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50c00726f0 0x7f50c00cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50c00726f0 0x7f50c00cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36914/0 (socket says 172.21.15.26:36914) 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bea00640 1 -- 172.21.15.26:0/2954416760 learned_addr learned my addr 172.21.15.26:0/2954416760 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bfe00640 1 --2- 172.21.15.26:0/2954416760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c015e030 0x7f50c015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bf400640 1 --2- 172.21.15.26:0/2954416760 >> v2:172.21.15.124:3300/0 conn(0x7f50c0146650 0x7f50c00b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bea00640 1 -- 172.21.15.26:0/2954416760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c015e030 msgr2=0x7f50c015e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bea00640 1 --2- 172.21.15.26:0/2954416760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c015e030 0x7f50c015e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bea00640 1 -- 172.21.15.26:0/2954416760 >> v2:172.21.15.124:3300/0 conn(0x7f50c0146650 msgr2=0x7f50c00b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bea00640 1 --2- 172.21.15.26:0/2954416760 >> v2:172.21.15.124:3300/0 conn(0x7f50c0146650 0x7f50c00b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.323+0000 7f50bea00640 1 -- 172.21.15.26:0/2954416760 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c006f520 con 0x7f50c00726f0 2024-01-15T18:00:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.324+0000 7f50bea00640 1 --2- 172.21.15.26:0/2954416760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c00726f0 0x7f50c00cca20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f50c0072c70 tx=0x7f50a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1773de2b1268461 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.325+0000 7f50be000640 1 -- 172.21.15.26:0/2954416760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8031870 con 0x7f50c00726f0 2024-01-15T18:00:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.325+0000 7f50be000640 1 -- 172.21.15.26:0/2954416760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a80319d0 con 0x7f50c00726f0 2024-01-15T18:00:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.325+0000 7f50be000640 1 -- 172.21.15.26:0/2954416760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8031d60 con 0x7f50c00726f0 2024-01-15T18:00:45.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.325+0000 7f50c4c00640 1 -- 172.21.15.26:0/2954416760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c00726f0 msgr2=0x7f50c00cca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.325+0000 7f50c4c00640 1 --2- 172.21.15.26:0/2954416760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c00726f0 0x7f50c00cca20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f50c0072c70 tx=0x7f50a802b910 comp rx=0 tx=0).stop 2024-01-15T18:00:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.326+0000 7f50c4c00640 1 -- 172.21.15.26:0/2954416760 shutdown_connections 2024-01-15T18:00:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.326+0000 7f50c4c00640 1 --2- 172.21.15.26:0/2954416760 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c015e030 0x7f50c015e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.326+0000 7f50c4c00640 1 --2- 172.21.15.26:0/2954416760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c00726f0 0x7f50c00cca20 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.326+0000 7f50c4c00640 1 --2- 172.21.15.26:0/2954416760 >> v2:172.21.15.124:3300/0 conn(0x7f50c0146650 0x7f50c00b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.326+0000 7f50c4c00640 1 -- 172.21.15.26:0/2954416760 >> 172.21.15.26:0/2954416760 conn(0x7f50c00b1d40 msgr2=0x7f50c00b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.326+0000 7f50c4c00640 1 -- 172.21.15.26:0/2954416760 shutdown_connections 2024-01-15T18:00:45.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.326+0000 7f50c4c00640 1 -- 172.21.15.26:0/2954416760 wait complete. 2024-01-15T18:00:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.328+0000 7f50c4c00640 1 Processor -- start 2024-01-15T18:00:45.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.328+0000 7f50c4c00640 1 -- start start 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.330+0000 7f50c4c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c00726f0 0x7f50c01f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.330+0000 7f50c4c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50c0146650 0x7f50c00b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.330+0000 7f50c4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c015e030 0x7f50c00aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.330+0000 7f50c4c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c00ae730 con 0x7f50c015e030 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.330+0000 7f50c4c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01f3530 con 0x7f50c0146650 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.330+0000 7f50c4c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c01f3810 con 0x7f50c00726f0 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.330+0000 7f50bfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c015e030 0x7f50c00aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.330+0000 7f50bfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c015e030 0x7f50c00aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36926/0 (socket says 172.21.15.26:36926) 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.330+0000 7f50bfe00640 1 -- 172.21.15.26:0/3278236047 learned_addr learned my addr 172.21.15.26:0/3278236047 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:45.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50bea00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50c0146650 0x7f50c00b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50bf400640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c00726f0 0x7f50c01f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50bfe00640 1 -- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c00726f0 msgr2=0x7f50c01f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50bfe00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c00726f0 0x7f50c01f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50bfe00640 1 -- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50c0146650 msgr2=0x7f50c00b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50bfe00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50c0146650 0x7f50c00b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50bfe00640 1 -- 172.21.15.26:0/3278236047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a80094d0 con 0x7f50c015e030 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50bf400640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c00726f0 0x7f50c01f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50bfe00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c015e030 0x7f50c00aeb20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f50b800d5d0 tx=0x7f50b800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50b7400640 1 -- 172.21.15.26:0/3278236047 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b80125f0 con 0x7f50c015e030 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50b7400640 1 -- 172.21.15.26:0/3278236047 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b8012770 con 0x7f50c015e030 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50b7400640 1 -- 172.21.15.26:0/3278236047 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8012b20 con 0x7f50c015e030 2024-01-15T18:00:45.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.331+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c01f3af0 con 0x7f50c015e030 2024-01-15T18:00:45.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.332+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c01f3dd0 con 0x7f50c015e030 2024-01-15T18:00:45.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.333+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c01f67d0 con 0x7f50c015e030 2024-01-15T18:00:45.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.334+0000 7f50b7400640 1 -- 172.21.15.26:0/3278236047 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f50b8012c80 con 0x7f50c015e030 2024-01-15T18:00:45.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.334+0000 7f50b7400640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f509806b860 0x7f509806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.335+0000 7f50b7400640 1 -- 172.21.15.26:0/3278236047 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f50b808ab70 con 0x7f50c015e030 2024-01-15T18:00:45.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.335+0000 7f50bf400640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f509806b860 0x7f509806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.336+0000 7f50bf400640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f509806b860 0x7f509806dd20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009f70 tx=0x7f50b000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:45.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.339+0000 7f50b7400640 1 -- 172.21.15.26:0/3278236047 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50c01f67d0 con 0x7f50c015e030 2024-01-15T18:00:45.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.573+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50c01f67d0 con 0x7f50c015e030 2024-01-15T18:00:45.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.574+0000 7f50b7400640 1 -- 172.21.15.26:0/3278236047 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f50c01f67d0 con 0x7f50c015e030 2024-01-15T18:00:45.578 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:45.578 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1159022909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1159022909},{"type":"v1","addr":"172.21.15.26:6835","nonce":1159022909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/248571798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":248571798},{"type":"v1","addr":"172.21.15.124:6837","nonce":248571798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3357264964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3357264964},{"type":"v1","addr":"172.21.15.26:6833","nonce":3357264964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:00:41.518292+0000","modified":"2024-01-15T18:00:44.577344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35186":{"gid":35186,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/3403655022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3403655022},{"type":"v1","addr":"172.21.15.124:6836","nonce":3403655022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-01-15T18:00:45.578 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 35 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f509806b860 msgr2=0x7f509806dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f509806b860 0x7f509806dd20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f50b0009f70 tx=0x7f50b000b040 comp rx=0 tx=0).stop 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c015e030 msgr2=0x7f50c00aeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c015e030 0x7f50c00aeb20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f50b800d5d0 tx=0x7f50b800da90 comp rx=0 tx=0).stop 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 shutdown_connections 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f509806b860 0x7f509806dd20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50c015e030 0x7f50c00aeb20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50c0146650 0x7f50c00b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 --2- 172.21.15.26:0/3278236047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50c00726f0 0x7f50c01f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 >> 172.21.15.26:0/3278236047 conn(0x7f50c00b1d40 msgr2=0x7f50c00afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 shutdown_connections 2024-01-15T18:00:45.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.577+0000 7f50c4c00640 1 -- 172.21.15.26:0/3278236047 wait complete. 2024-01-15T18:00:45.601 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:00:45.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.819+0000 7f83bbe00640 1 Processor -- start 2024-01-15T18:00:45.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.819+0000 7f83bbe00640 1 -- start start 2024-01-15T18:00:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.821+0000 7f83bbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83bc143c40 0x7f83bc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.821+0000 7f83bbe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f83bc0717d0 0x7f83bc0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.821+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 0x7f83bc15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.821+0000 7f83bbe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83bc0723c0 con 0x7f83bc143c40 2024-01-15T18:00:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.821+0000 7f83bbe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83bc070e50 con 0x7f83bc0717d0 2024-01-15T18:00:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.821+0000 7f83bbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc0cce40 con 0x7f83bc0cca40 2024-01-15T18:00:45.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.821+0000 7f83baa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83bc143c40 0x7f83bc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.821+0000 7f83baa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f83bc143c40 0x7f83bc071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36940/0 (socket says 172.21.15.26:36940) 2024-01-15T18:00:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.821+0000 7f83baa00640 1 -- 172.21.15.26:0/2941579501 learned_addr learned my addr 172.21.15.26:0/2941579501 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.822+0000 7f83b3e00640 1 --2- 172.21.15.26:0/2941579501 >> v2:172.21.15.124:3300/0 conn(0x7f83bc0717d0 0x7f83bc0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.822+0000 7f83bb400640 1 --2- 172.21.15.26:0/2941579501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 0x7f83bc15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.822+0000 7f83baa00640 1 -- 172.21.15.26:0/2941579501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 msgr2=0x7f83bc15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.822+0000 7f83baa00640 1 --2- 172.21.15.26:0/2941579501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 0x7f83bc15eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.822+0000 7f83baa00640 1 -- 172.21.15.26:0/2941579501 >> v2:172.21.15.124:3300/0 conn(0x7f83bc0717d0 msgr2=0x7f83bc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.822+0000 7f83baa00640 1 --2- 172.21.15.26:0/2941579501 >> v2:172.21.15.124:3300/0 conn(0x7f83bc0717d0 0x7f83bc0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.822+0000 7f83baa00640 1 -- 172.21.15.26:0/2941579501 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83bc144020 con 0x7f83bc143c40 2024-01-15T18:00:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.823+0000 7f83baa00640 1 --2- 172.21.15.26:0/2941579501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 0x7f83bc071290 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0b44a0 tx=0x7f83ac02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f321abf082bb908c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83ba000640 1 -- 172.21.15.26:0/2941579501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0317d0 con 0x7f83bc143c40 2024-01-15T18:00:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83ba000640 1 -- 172.21.15.26:0/2941579501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac031930 con 0x7f83bc143c40 2024-01-15T18:00:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83ba000640 1 -- 172.21.15.26:0/2941579501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac031cc0 con 0x7f83bc143c40 2024-01-15T18:00:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83bbe00640 1 -- 172.21.15.26:0/2941579501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 msgr2=0x7f83bc071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83bbe00640 1 --2- 172.21.15.26:0/2941579501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 0x7f83bc071290 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f83bc0b44a0 tx=0x7f83ac02b870 comp rx=0 tx=0).stop 2024-01-15T18:00:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83bbe00640 1 -- 172.21.15.26:0/2941579501 shutdown_connections 2024-01-15T18:00:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83bbe00640 1 --2- 172.21.15.26:0/2941579501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 0x7f83bc15eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83bbe00640 1 --2- 172.21.15.26:0/2941579501 >> v2:172.21.15.124:3300/0 conn(0x7f83bc0717d0 0x7f83bc0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83bbe00640 1 --2- 172.21.15.26:0/2941579501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 0x7f83bc071290 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83bbe00640 1 -- 172.21.15.26:0/2941579501 >> 172.21.15.26:0/2941579501 conn(0x7f83bc06a800 msgr2=0x7f83bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83bbe00640 1 -- 172.21.15.26:0/2941579501 shutdown_connections 2024-01-15T18:00:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.824+0000 7f83bbe00640 1 -- 172.21.15.26:0/2941579501 wait complete. 2024-01-15T18:00:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.826+0000 7f83bbe00640 1 Processor -- start 2024-01-15T18:00:45.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.826+0000 7f83bbe00640 1 -- start start 2024-01-15T18:00:45.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.827+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83bc0717d0 0x7f83bc0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.827+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 0x7f83bc1e0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.827+0000 7f83bbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 0x7f83bc1e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.827+0000 7f83bbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1e7170 con 0x7f83bc143c40 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.827+0000 7f83bbe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1e65c0 con 0x7f83bc0717d0 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.827+0000 7f83bbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1e1480 con 0x7f83bc0cca40 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.827+0000 7f83bb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 0x7f83bc1e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83bb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 0x7f83bc1e3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36948/0 (socket says 172.21.15.26:36948) 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83bb400640 1 -- 172.21.15.26:0/3324618656 learned_addr learned my addr 172.21.15.26:0/3324618656 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83b3e00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 0x7f83bc1e0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83baa00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83bc0717d0 0x7f83bc0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83bb400640 1 -- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 msgr2=0x7f83bc1e0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83bb400640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 0x7f83bc1e0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83bb400640 1 -- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83bc0717d0 msgr2=0x7f83bc0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83bb400640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83bc0717d0 0x7f83bc0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83bb400640 1 -- 172.21.15.26:0/3324618656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac0094d0 con 0x7f83bc143c40 2024-01-15T18:00:45.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83baa00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83bc0717d0 0x7f83bc0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.828+0000 7f83bb400640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 0x7f83bc1e3c10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f83b4007e40 tx=0x7f83b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.829+0000 7f83b2a00640 1 -- 172.21.15.26:0/3324618656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4014620 con 0x7f83bc143c40 2024-01-15T18:00:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.829+0000 7f83b2a00640 1 -- 172.21.15.26:0/3324618656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b400ee90 con 0x7f83bc143c40 2024-01-15T18:00:45.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.829+0000 7f83b2a00640 1 -- 172.21.15.26:0/3324618656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b40149e0 con 0x7f83bc143c40 2024-01-15T18:00:45.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.829+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc13f730 con 0x7f83bc143c40 2024-01-15T18:00:45.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.829+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc1e6ab0 con 0x7f83bc143c40 2024-01-15T18:00:45.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.830+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc1e72b0 con 0x7f83bc143c40 2024-01-15T18:00:45.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.831+0000 7f83b2a00640 1 -- 172.21.15.26:0/3324618656 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f83b4014b40 con 0x7f83bc143c40 2024-01-15T18:00:45.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.832+0000 7f83b2a00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f839406b860 0x7f839406dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:45.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.832+0000 7f83b2a00640 1 -- 172.21.15.26:0/3324618656 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f83b408af80 con 0x7f83bc143c40 2024-01-15T18:00:45.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.832+0000 7f83baa00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f839406b860 0x7f839406dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:45.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.834+0000 7f83baa00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f839406b860 0x7f839406dd20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f83ac036000 tx=0x7f83ac03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:45.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:45.836+0000 7f83b2a00640 1 -- 172.21.15.26:0/3324618656 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83bc1e72b0 con 0x7f83bc143c40 2024-01-15T18:00:46.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.034+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83bc1e72b0 con 0x7f83bc143c40 2024-01-15T18:00:46.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.036+0000 7f83b2a00640 1 -- 172.21.15.26:0/3324618656 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16584 (secure 0 0 0) 0x7f83bc1e72b0 con 0x7f83bc143c40 2024-01-15T18:00:46.040 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:46.040 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":60,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:00:43.051198+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:00:39.364932+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-01-15T18:00:40.543640+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:00:46.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f839406b860 msgr2=0x7f839406dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f839406b860 0x7f839406dd20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f83ac036000 tx=0x7f83ac03d040 comp rx=0 tx=0).stop 2024-01-15T18:00:46.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 msgr2=0x7f83bc1e3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 0x7f83bc1e3c10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f83b4007e40 tx=0x7f83b400e4b0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 shutdown_connections 2024-01-15T18:00:46.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f83bc0cca40 0x7f83bc1e0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f839406b860 0x7f839406dd20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f83bc143c40 0x7f83bc1e3c10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 --2- 172.21.15.26:0/3324618656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f83bc0717d0 0x7f83bc0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 >> 172.21.15.26:0/3324618656 conn(0x7f83bc06a800 msgr2=0x7f83bc0aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:46.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 shutdown_connections 2024-01-15T18:00:46.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.039+0000 7f83bbe00640 1 -- 172.21.15.26:0/3324618656 wait complete. 2024-01-15T18:00:46.062 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:00:46.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:00:46.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.272+0000 7f94a3e00640 1 Processor -- start 2024-01-15T18:00:46.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.273+0000 7f94a3e00640 1 -- start start 2024-01-15T18:00:46.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.274+0000 7f94a3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94a415d3c0 0x7f94a40d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.274+0000 7f94a3e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f94a40d0e90 0x7f94a40afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.274+0000 7f94a3e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a40d12f0 con 0x7f94a40b01b0 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.274+0000 7f94a3e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a40ce5d0 con 0x7f94a415d3c0 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.274+0000 7f94a3e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a40b23c0 con 0x7f94a40d0e90 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.274+0000 7f94a2a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94a415d3c0 0x7f94a40d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.274+0000 7f94a2a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f94a415d3c0 0x7f94a40d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36958/0 (socket says 172.21.15.26:36958) 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.274+0000 7f94a2a00640 1 -- 172.21.15.26:0/4003313322 learned_addr learned my addr 172.21.15.26:0/4003313322 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.275+0000 7f94a2000640 1 --2- 172.21.15.26:0/4003313322 >> v2:172.21.15.124:3300/0 conn(0x7f94a40d0e90 0x7f94a40afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.275+0000 7f94a2a00640 1 -- 172.21.15.26:0/4003313322 >> v2:172.21.15.124:3300/0 conn(0x7f94a40d0e90 msgr2=0x7f94a40afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.275+0000 7f94a2a00640 1 --2- 172.21.15.26:0/4003313322 >> v2:172.21.15.124:3300/0 conn(0x7f94a40d0e90 0x7f94a40afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.275+0000 7f94a2a00640 1 -- 172.21.15.26:0/4003313322 >> v1:172.21.15.124:6789/0 conn(0x7f94a40b01b0 legacy=0x7f94a40ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:46.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.275+0000 7f94a2a00640 1 -- 172.21.15.26:0/4003313322 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a40d0800 con 0x7f94a415d3c0 2024-01-15T18:00:46.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.276+0000 7f94a2a00640 1 --2- 172.21.15.26:0/4003313322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 0x7f94a40d0220 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f94a40a0fc0 tx=0x7f948c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5e39ce88d26723 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:46.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.277+0000 7f94a1600640 1 -- 172.21.15.26:0/4003313322 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0317e0 con 0x7f94a415d3c0 2024-01-15T18:00:46.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.277+0000 7f94a1600640 1 -- 172.21.15.26:0/4003313322 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c031940 con 0x7f94a415d3c0 2024-01-15T18:00:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.277+0000 7f94a1600640 1 -- 172.21.15.26:0/4003313322 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c031cd0 con 0x7f94a415d3c0 2024-01-15T18:00:46.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.277+0000 7f94a3e00640 1 -- 172.21.15.26:0/4003313322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 msgr2=0x7f94a40d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.277+0000 7f94a3e00640 1 --2- 172.21.15.26:0/4003313322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 0x7f94a40d0220 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f94a40a0fc0 tx=0x7f948c02b880 comp rx=0 tx=0).stop 2024-01-15T18:00:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.278+0000 7f94a3e00640 1 -- 172.21.15.26:0/4003313322 shutdown_connections 2024-01-15T18:00:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.278+0000 7f94a3e00640 1 --2- 172.21.15.26:0/4003313322 >> v2:172.21.15.124:3300/0 conn(0x7f94a40d0e90 0x7f94a40afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.278+0000 7f94a3e00640 1 --2- 172.21.15.26:0/4003313322 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 0x7f94a40d0220 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.278+0000 7f94a3e00640 1 -- 172.21.15.26:0/4003313322 >> 172.21.15.26:0/4003313322 conn(0x7f94a40d30b0 msgr2=0x7f94a40d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.278+0000 7f94a3e00640 1 -- 172.21.15.26:0/4003313322 shutdown_connections 2024-01-15T18:00:46.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.278+0000 7f94a3e00640 1 -- 172.21.15.26:0/4003313322 wait complete. 2024-01-15T18:00:46.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.279+0000 7f94a3e00640 1 Processor -- start 2024-01-15T18:00:46.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.280+0000 7f94a3e00640 1 -- start start 2024-01-15T18:00:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94a40b01b0 0x7f94a414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94a40d0e90 0x7f94a40b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 0x7f94a40cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a40b3cf0 con 0x7f94a415d3c0 2024-01-15T18:00:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a414fa50 con 0x7f94a40d0e90 2024-01-15T18:00:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a40b46c0 con 0x7f94a40b01b0 2024-01-15T18:00:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 0x7f94a40cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 0x7f94a40cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36974/0 (socket says 172.21.15.26:36974) 2024-01-15T18:00:46.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3400640 1 -- 172.21.15.26:0/918679872 learned_addr learned my addr 172.21.15.26:0/918679872 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a2a00640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94a40b01b0 0x7f94a414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a2000640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94a40d0e90 0x7f94a40b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3400640 1 -- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94a40b01b0 msgr2=0x7f94a414f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3400640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94a40b01b0 0x7f94a414f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3400640 1 -- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94a40d0e90 msgr2=0x7f94a40b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3400640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94a40d0e90 0x7f94a40b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.281+0000 7f94a3400640 1 -- 172.21.15.26:0/918679872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c0094d0 con 0x7f94a415d3c0 2024-01-15T18:00:46.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.282+0000 7f94a3400640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 0x7f94a40cc350 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c410 tx=0x7f949c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.282+0000 7f949b400640 1 -- 172.21.15.26:0/918679872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c007230 con 0x7f94a415d3c0 2024-01-15T18:00:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.282+0000 7f949b400640 1 -- 172.21.15.26:0/918679872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c007390 con 0x7f94a415d3c0 2024-01-15T18:00:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.282+0000 7f949b400640 1 -- 172.21.15.26:0/918679872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c0067d0 con 0x7f94a415d3c0 2024-01-15T18:00:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.282+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a415bcc0 con 0x7f94a415d3c0 2024-01-15T18:00:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.282+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a40b33c0 con 0x7f94a415d3c0 2024-01-15T18:00:46.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.283+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a40b3ea0 con 0x7f94a415d3c0 2024-01-15T18:00:46.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.284+0000 7f949b400640 1 -- 172.21.15.26:0/918679872 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f949c006930 con 0x7f94a415d3c0 2024-01-15T18:00:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.286+0000 7f949b400640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f947c06b890 0x7f947c06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.286+0000 7f949b400640 1 -- 172.21.15.26:0/918679872 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f949c089bd0 con 0x7f94a415d3c0 2024-01-15T18:00:46.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.286+0000 7f94a2a00640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f947c06b890 0x7f947c06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.287+0000 7f94a2a00640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f947c06b890 0x7f947c06dd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f948c036000 tx=0x7f948c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:46.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.291+0000 7f949b400640 1 -- 172.21.15.26:0/918679872 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94a40b3ea0 con 0x7f94a415d3c0 2024-01-15T18:00:46.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.489+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94a40cdbc0 con 0x7f947c06b890 2024-01-15T18:00:46.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.517+0000 7f949b400640 1 -- 172.21.15.26:0/918679872 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286126 (secure 0 0 0) 0x7f948c031b30 con 0x7f947c06b890 2024-01-15T18:00:46.526 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:46.528 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:00:46.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f947c06b890 msgr2=0x7f947c06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f947c06b890 0x7f947c06dd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f948c036000 tx=0x7f948c03d040 comp rx=0 tx=0).stop 2024-01-15T18:00:46.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 msgr2=0x7f94a40cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 0x7f94a40cc350 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c410 tx=0x7f949c00c8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 shutdown_connections 2024-01-15T18:00:46.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f947c06b890 0x7f947c06dd50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f94a415d3c0 0x7f94a40cc350 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f94a40d0e90 0x7f94a40b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 --2- 172.21.15.26:0/918679872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94a40b01b0 0x7f94a414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 >> 172.21.15.26:0/918679872 conn(0x7f94a40d30b0 msgr2=0x7f94a40b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:46.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.524+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 shutdown_connections 2024-01-15T18:00:46.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.525+0000 7f94a3e00640 1 -- 172.21.15.26:0/918679872 wait complete. 2024-01-15T18:00:46.549 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-01-15T18:00:46.034438+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":194,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":310116,"kb_used_data":6568,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754664604,"statfs":{"total":773094113280,"available":772776554496,"internally_reserved":0,"allocated":6725632,"data_stored":3478232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002328"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.051825+0000","last_change":"2024-01-15T18:00:41.051825+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.051825+0000","last_undegraded":"2024-01-15T18:00:41.051825+0000","last_fullsized":"2024-01-15T18:00:41.051825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063316+0000","last_change":"2024-01-15T18:00:42.060637+0000","last_active":"2024-01-15T18:00:42.063316+0000","last_peered":"2024-01-15T18:00:42.063316+0000","last_clean":"2024-01-15T18:00:42.063316+0000","last_became_active":"2024-01-15T18:00:42.060376+0000","last_became_peered":"2024-01-15T18:00:42.060376+0000","last_unstale":"2024-01-15T18:00:42.063316+0000","last_undegraded":"2024-01-15T18:00:42.063316+0000","last_fullsized":"2024-01-15T18:00:42.063316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:58:12.883647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082872+0000","last_change":"2024-01-15T18:00:42.077438+0000","last_active":"2024-01-15T18:00:42.082872+0000","last_peered":"2024-01-15T18:00:42.082872+0000","last_clean":"2024-01-15T18:00:42.082872+0000","last_became_active":"2024-01-15T18:00:42.077041+0000","last_became_peered":"2024-01-15T18:00:42.077041+0000","last_unstale":"2024-01-15T18:00:42.082872+0000","last_undegraded":"2024-01-15T18:00:42.082872+0000","last_fullsized":"2024-01-15T18:00:42.082872+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.072305+0000","last_change":"2024-01-15T18:00:42.067513+0000","last_active":"2024-01-15T18:00:42.072305+0000","last_peered":"2024-01-15T18:00:42.072305+0000","last_clean":"2024-01-15T18:00:42.072305+0000","last_became_active":"2024-01-15T18:00:42.066627+0000","last_became_peered":"2024-01-15T18:00:42.066627+0000","last_unstale":"2024-01-15T18:00:42.072305+0000","last_undegraded":"2024-01-15T18:00:42.072305+0000","last_fullsized":"2024-01-15T18:00:42.072305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:13.509938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.043365+0000","last_change":"2024-01-15T18:00:41.043365+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.043365+0000","last_undegraded":"2024-01-15T18:00:41.043365+0000","last_fullsized":"2024-01-15T18:00:41.043365+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076462+0000","last_change":"2024-01-15T18:00:42.067558+0000","last_active":"2024-01-15T18:00:42.076462+0000","last_peered":"2024-01-15T18:00:42.076462+0000","last_clean":"2024-01-15T18:00:42.076462+0000","last_became_active":"2024-01-15T18:00:42.067182+0000","last_became_peered":"2024-01-15T18:00:42.067182+0000","last_unstale":"2024-01-15T18:00:42.076462+0000","last_undegraded":"2024-01-15T18:00:42.076462+0000","last_fullsized":"2024-01-15T18:00:42.076462+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:13.509938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064214+0000","last_change":"2024-01-15T18:00:42.064214+0000","last_active":"2024-01-15T18:00:42.064214+0000","last_peered":"2024-01-15T18:00:42.064214+0000","last_clean":"2024-01-15T18:00:42.064214+0000","last_became_active":"2024-01-15T18:00:42.063930+0000","last_became_peered":"2024-01-15T18:00:42.063930+0000","last_unstale":"2024-01-15T18:00:42.064214+0000","last_undegraded":"2024-01-15T18:00:42.064214+0000","last_fullsized":"2024-01-15T18:00:42.064214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:57:42.271743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.079933+0000","last_change":"2024-01-15T18:00:42.072265+0000","last_active":"2024-01-15T18:00:42.079933+0000","last_peered":"2024-01-15T18:00:42.079933+0000","last_clean":"2024-01-15T18:00:42.079933+0000","last_became_active":"2024-01-15T18:00:42.071980+0000","last_became_peered":"2024-01-15T18:00:42.071980+0000","last_unstale":"2024-01-15T18:00:42.079933+0000","last_undegraded":"2024-01-15T18:00:42.079933+0000","last_fullsized":"2024-01-15T18:00:42.079933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:41.900799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":3,"reported_epoch":59,"state":"creating+activating","last_fresh":"2024-01-15T18:00:42.043232+0000","last_change":"2024-01-15T18:00:42.043232+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:42.043232+0000","last_undegraded":"2024-01-15T18:00:42.043232+0000","last_fullsized":"2024-01-15T18:00:42.043232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:25:02.833433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.058936+0000","last_change":"2024-01-15T18:00:42.055688+0000","last_active":"2024-01-15T18:00:42.058936+0000","last_peered":"2024-01-15T18:00:42.058936+0000","last_clean":"2024-01-15T18:00:42.058936+0000","last_became_active":"2024-01-15T18:00:42.055302+0000","last_became_peered":"2024-01-15T18:00:42.055302+0000","last_unstale":"2024-01-15T18:00:42.058936+0000","last_undegraded":"2024-01-15T18:00:42.058936+0000","last_fullsized":"2024-01-15T18:00:42.058936+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:22:28.043841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.066509+0000","last_change":"2024-01-15T18:00:42.066509+0000","last_active":"2024-01-15T18:00:42.066509+0000","last_peered":"2024-01-15T18:00:42.066509+0000","last_clean":"2024-01-15T18:00:42.066509+0000","last_became_active":"2024-01-15T18:00:42.066232+0000","last_became_peered":"2024-01-15T18:00:42.066232+0000","last_unstale":"2024-01-15T18:00:42.066509+0000","last_undegraded":"2024-01-15T18:00:42.066509+0000","last_fullsized":"2024-01-15T18:00:42.066509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:39:13.515709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.072325+0000","last_change":"2024-01-15T18:00:42.068083+0000","last_active":"2024-01-15T18:00:42.072325+0000","last_peered":"2024-01-15T18:00:42.072325+0000","last_clean":"2024-01-15T18:00:42.072325+0000","last_became_active":"2024-01-15T18:00:42.067826+0000","last_became_peered":"2024-01-15T18:00:42.067826+0000","last_unstale":"2024-01-15T18:00:42.072325+0000","last_undegraded":"2024-01-15T18:00:42.072325+0000","last_fullsized":"2024-01-15T18:00:42.072325+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082480+0000","last_change":"2024-01-15T18:00:42.080948+0000","last_active":"2024-01-15T18:00:42.082480+0000","last_peered":"2024-01-15T18:00:42.082480+0000","last_clean":"2024-01-15T18:00:42.082480+0000","last_became_active":"2024-01-15T18:00:42.079522+0000","last_became_peered":"2024-01-15T18:00:42.079522+0000","last_unstale":"2024-01-15T18:00:42.082480+0000","last_undegraded":"2024-01-15T18:00:42.082480+0000","last_fullsized":"2024-01-15T18:00:42.082480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:44.959104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":3,"reported_epoch":59,"state":"creating+activating","last_fresh":"2024-01-15T18:00:42.042710+0000","last_change":"2024-01-15T18:00:42.042710+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:42.042710+0000","last_undegraded":"2024-01-15T18:00:42.042710+0000","last_fullsized":"2024-01-15T18:00:42.042710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:16:20.555824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063338+0000","last_change":"2024-01-15T18:00:42.063338+0000","last_active":"2024-01-15T18:00:42.063338+0000","last_peered":"2024-01-15T18:00:42.063338+0000","last_clean":"2024-01-15T18:00:42.063338+0000","last_became_active":"2024-01-15T18:00:42.062711+0000","last_became_peered":"2024-01-15T18:00:42.062711+0000","last_unstale":"2024-01-15T18:00:42.063338+0000","last_undegraded":"2024-01-15T18:00:42.063338+0000","last_fullsized":"2024-01-15T18:00:42.063338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:23.443661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081709+0000","last_change":"2024-01-15T18:00:42.073413+0000","last_active":"2024-01-15T18:00:42.081709+0000","last_peered":"2024-01-15T18:00:42.081709+0000","last_clean":"2024-01-15T18:00:42.081709+0000","last_became_active":"2024-01-15T18:00:42.072167+0000","last_became_peered":"2024-01-15T18:00:42.072167+0000","last_unstale":"2024-01-15T18:00:42.081709+0000","last_undegraded":"2024-01-15T18:00:42.081709+0000","last_fullsized":"2024-01-15T18:00:42.081709+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:24.550976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.062432+0000","last_change":"2024-01-15T18:00:42.058452+0000","last_active":"2024-01-15T18:00:42.062432+0000","last_peered":"2024-01-15T18:00:42.062432+0000","last_clean":"2024-01-15T18:00:42.062432+0000","last_became_active":"2024-01-15T18:00:42.057885+0000","last_became_peered":"2024-01-15T18:00:42.057885+0000","last_unstale":"2024-01-15T18:00:42.062432+0000","last_undegraded":"2024-01-15T18:00:42.062432+0000","last_fullsized":"2024-01-15T18:00:42.062432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:56.491932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083323+0000","last_change":"2024-01-15T18:00:42.079987+0000","last_active":"2024-01-15T18:00:42.083323+0000","last_peered":"2024-01-15T18:00:42.083323+0000","last_clean":"2024-01-15T18:00:42.083323+0000","last_became_active":"2024-01-15T18:00:42.079525+0000","last_became_peered":"2024-01-15T18:00:42.079525+0000","last_unstale":"2024-01-15T18:00:42.083323+0000","last_undegraded":"2024-01-15T18:00:42.083323+0000","last_fullsized":"2024-01-15T18:00:42.083323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:24.550976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.046537+0000","last_change":"2024-01-15T18:00:41.046537+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.046537+0000","last_undegraded":"2024-01-15T18:00:41.046537+0000","last_fullsized":"2024-01-15T18:00:41.046537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083287+0000","last_change":"2024-01-15T18:00:42.080315+0000","last_active":"2024-01-15T18:00:42.083287+0000","last_peered":"2024-01-15T18:00:42.083287+0000","last_clean":"2024-01-15T18:00:42.083287+0000","last_became_active":"2024-01-15T18:00:42.079623+0000","last_became_peered":"2024-01-15T18:00:42.079623+0000","last_unstale":"2024-01-15T18:00:42.083287+0000","last_undegraded":"2024-01-15T18:00:42.083287+0000","last_fullsized":"2024-01-15T18:00:42.083287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:56:47.128531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.078789+0000","last_change":"2024-01-15T18:00:42.071297+0000","last_active":"2024-01-15T18:00:42.078789+0000","last_peered":"2024-01-15T18:00:42.078789+0000","last_clean":"2024-01-15T18:00:42.078789+0000","last_became_active":"2024-01-15T18:00:42.071046+0000","last_became_peered":"2024-01-15T18:00:42.071046+0000","last_unstale":"2024-01-15T18:00:42.078789+0000","last_undegraded":"2024-01-15T18:00:42.078789+0000","last_fullsized":"2024-01-15T18:00:42.078789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.077554+0000","last_change":"2024-01-15T18:00:42.066557+0000","last_active":"2024-01-15T18:00:42.077554+0000","last_peered":"2024-01-15T18:00:42.077554+0000","last_clean":"2024-01-15T18:00:42.077554+0000","last_became_active":"2024-01-15T18:00:42.066269+0000","last_became_peered":"2024-01-15T18:00:42.066269+0000","last_unstale":"2024-01-15T18:00:42.077554+0000","last_undegraded":"2024-01-15T18:00:42.077554+0000","last_fullsized":"2024-01-15T18:00:42.077554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:40:52.391655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076381+0000","last_change":"2024-01-15T18:00:42.071412+0000","last_active":"2024-01-15T18:00:42.076381+0000","last_peered":"2024-01-15T18:00:42.076381+0000","last_clean":"2024-01-15T18:00:42.076381+0000","last_became_active":"2024-01-15T18:00:42.070884+0000","last_became_peered":"2024-01-15T18:00:42.070884+0000","last_unstale":"2024-01-15T18:00:42.076381+0000","last_undegraded":"2024-01-15T18:00:42.076381+0000","last_fullsized":"2024-01-15T18:00:42.076381+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:17.543171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081314+0000","last_change":"2024-01-15T18:00:42.078634+0000","last_active":"2024-01-15T18:00:42.081314+0000","last_peered":"2024-01-15T18:00:42.081314+0000","last_clean":"2024-01-15T18:00:42.081314+0000","last_became_active":"2024-01-15T18:00:42.078165+0000","last_became_peered":"2024-01-15T18:00:42.078165+0000","last_unstale":"2024-01-15T18:00:42.081314+0000","last_undegraded":"2024-01-15T18:00:42.081314+0000","last_fullsized":"2024-01-15T18:00:42.081314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:41:43.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.051170+0000","last_change":"2024-01-15T18:00:41.051170+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.051170+0000","last_undegraded":"2024-01-15T18:00:41.051170+0000","last_fullsized":"2024-01-15T18:00:41.051170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.079608+0000","last_change":"2024-01-15T18:00:42.076259+0000","last_active":"2024-01-15T18:00:42.079608+0000","last_peered":"2024-01-15T18:00:42.079608+0000","last_clean":"2024-01-15T18:00:42.079608+0000","last_became_active":"2024-01-15T18:00:42.076003+0000","last_became_peered":"2024-01-15T18:00:42.076003+0000","last_unstale":"2024-01-15T18:00:42.079608+0000","last_undegraded":"2024-01-15T18:00:42.079608+0000","last_fullsized":"2024-01-15T18:00:42.079608+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:00.533678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080247+0000","last_change":"2024-01-15T18:00:42.072997+0000","last_active":"2024-01-15T18:00:42.080247+0000","last_peered":"2024-01-15T18:00:42.080247+0000","last_clean":"2024-01-15T18:00:42.080247+0000","last_became_active":"2024-01-15T18:00:42.070411+0000","last_became_peered":"2024-01-15T18:00:42.070411+0000","last_unstale":"2024-01-15T18:00:42.080247+0000","last_undegraded":"2024-01-15T18:00:42.080247+0000","last_fullsized":"2024-01-15T18:00:42.080247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:44.207537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.045517+0000","last_change":"2024-01-15T18:00:41.045517+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.045517+0000","last_undegraded":"2024-01-15T18:00:41.045517+0000","last_fullsized":"2024-01-15T18:00:41.045517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.077697+0000","last_change":"2024-01-15T18:00:42.067598+0000","last_active":"2024-01-15T18:00:42.077697+0000","last_peered":"2024-01-15T18:00:42.077697+0000","last_clean":"2024-01-15T18:00:42.077697+0000","last_became_active":"2024-01-15T18:00:42.067297+0000","last_became_peered":"2024-01-15T18:00:42.067297+0000","last_unstale":"2024-01-15T18:00:42.077697+0000","last_undegraded":"2024-01-15T18:00:42.077697+0000","last_fullsized":"2024-01-15T18:00:42.077697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:42.520698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081309+0000","last_change":"2024-01-15T18:00:42.074513+0000","last_active":"2024-01-15T18:00:42.081309+0000","last_peered":"2024-01-15T18:00:42.081309+0000","last_clean":"2024-01-15T18:00:42.081309+0000","last_became_active":"2024-01-15T18:00:42.074280+0000","last_became_peered":"2024-01-15T18:00:42.074280+0000","last_unstale":"2024-01-15T18:00:42.081309+0000","last_undegraded":"2024-01-15T18:00:42.081309+0000","last_fullsized":"2024-01-15T18:00:42.081309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.043336+0000","last_change":"2024-01-15T18:00:41.043336+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.043336+0000","last_undegraded":"2024-01-15T18:00:41.043336+0000","last_fullsized":"2024-01-15T18:00:41.043336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.088984+0000","last_change":"2024-01-15T18:00:42.086123+0000","last_active":"2024-01-15T18:00:42.088984+0000","last_peered":"2024-01-15T18:00:42.088984+0000","last_clean":"2024-01-15T18:00:42.088984+0000","last_became_active":"2024-01-15T18:00:42.085208+0000","last_became_peered":"2024-01-15T18:00:42.085208+0000","last_unstale":"2024-01-15T18:00:42.088984+0000","last_undegraded":"2024-01-15T18:00:42.088984+0000","last_fullsized":"2024-01-15T18:00:42.088984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:00.533678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069251+0000","last_change":"2024-01-15T18:00:42.065563+0000","last_active":"2024-01-15T18:00:42.069251+0000","last_peered":"2024-01-15T18:00:42.069251+0000","last_clean":"2024-01-15T18:00:42.069251+0000","last_became_active":"2024-01-15T18:00:42.065279+0000","last_became_peered":"2024-01-15T18:00:42.065279+0000","last_unstale":"2024-01-15T18:00:42.069251+0000","last_undegraded":"2024-01-15T18:00:42.069251+0000","last_fullsized":"2024-01-15T18:00:42.069251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:52.055388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.092341+0000","last_change":"2024-01-15T18:00:41.072898+0000","last_active":"2024-01-15T18:00:42.092341+0000","last_peered":"2024-01-15T18:00:42.092341+0000","last_clean":"2024-01-15T18:00:42.092341+0000","last_became_active":"2024-01-15T18:00:41.071115+0000","last_became_peered":"2024-01-15T18:00:41.071115+0000","last_unstale":"2024-01-15T18:00:42.092341+0000","last_undegraded":"2024-01-15T18:00:42.092341+0000","last_fullsized":"2024-01-15T18:00:42.092341+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063277+0000","last_change":"2024-01-15T18:00:42.060261+0000","last_active":"2024-01-15T18:00:42.063277+0000","last_peered":"2024-01-15T18:00:42.063277+0000","last_clean":"2024-01-15T18:00:42.063277+0000","last_became_active":"2024-01-15T18:00:42.059975+0000","last_became_peered":"2024-01-15T18:00:42.059975+0000","last_unstale":"2024-01-15T18:00:42.063277+0000","last_undegraded":"2024-01-15T18:00:42.063277+0000","last_fullsized":"2024-01-15T18:00:42.063277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:44.959104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082453+0000","last_change":"2024-01-15T18:00:41.067396+0000","last_active":"2024-01-15T18:00:42.082453+0000","last_peered":"2024-01-15T18:00:42.082453+0000","last_clean":"2024-01-15T18:00:42.082453+0000","last_became_active":"2024-01-15T18:00:41.067056+0000","last_became_peered":"2024-01-15T18:00:41.067056+0000","last_unstale":"2024-01-15T18:00:42.082453+0000","last_undegraded":"2024-01-15T18:00:42.082453+0000","last_fullsized":"2024-01-15T18:00:42.082453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:24.577110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076694+0000","last_change":"2024-01-15T18:00:42.065885+0000","last_active":"2024-01-15T18:00:42.076694+0000","last_peered":"2024-01-15T18:00:42.076694+0000","last_clean":"2024-01-15T18:00:42.076694+0000","last_became_active":"2024-01-15T18:00:42.065563+0000","last_became_peered":"2024-01-15T18:00:42.065563+0000","last_unstale":"2024-01-15T18:00:42.076694+0000","last_undegraded":"2024-01-15T18:00:42.076694+0000","last_fullsized":"2024-01-15T18:00:42.076694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:53:36.562001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.090834+0000","last_change":"2024-01-15T18:00:41.053014+0000","last_active":"2024-01-15T18:00:42.090834+0000","last_peered":"2024-01-15T18:00:42.090834+0000","last_clean":"2024-01-15T18:00:42.090834+0000","last_became_active":"2024-01-15T18:00:41.052718+0000","last_became_peered":"2024-01-15T18:00:41.052718+0000","last_unstale":"2024-01-15T18:00:42.090834+0000","last_undegraded":"2024-01-15T18:00:42.090834+0000","last_fullsized":"2024-01-15T18:00:42.090834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080283+0000","last_change":"2024-01-15T18:00:42.077700+0000","last_active":"2024-01-15T18:00:42.080283+0000","last_peered":"2024-01-15T18:00:42.080283+0000","last_clean":"2024-01-15T18:00:42.080283+0000","last_became_active":"2024-01-15T18:00:42.077465+0000","last_became_peered":"2024-01-15T18:00:42.077465+0000","last_unstale":"2024-01-15T18:00:42.080283+0000","last_undegraded":"2024-01-15T18:00:42.080283+0000","last_fullsized":"2024-01-15T18:00:42.080283+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070270+0000","last_change":"2024-01-15T18:00:41.057990+0000","last_active":"2024-01-15T18:00:42.070270+0000","last_peered":"2024-01-15T18:00:42.070270+0000","last_clean":"2024-01-15T18:00:42.070270+0000","last_became_active":"2024-01-15T18:00:41.057666+0000","last_became_peered":"2024-01-15T18:00:41.057666+0000","last_unstale":"2024-01-15T18:00:42.070270+0000","last_undegraded":"2024-01-15T18:00:42.070270+0000","last_fullsized":"2024-01-15T18:00:42.070270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.077504+0000","last_change":"2024-01-15T18:00:42.066199+0000","last_active":"2024-01-15T18:00:42.077504+0000","last_peered":"2024-01-15T18:00:42.077504+0000","last_clean":"2024-01-15T18:00:42.077504+0000","last_became_active":"2024-01-15T18:00:42.065918+0000","last_became_peered":"2024-01-15T18:00:42.065918+0000","last_unstale":"2024-01-15T18:00:42.077504+0000","last_undegraded":"2024-01-15T18:00:42.077504+0000","last_fullsized":"2024-01-15T18:00:42.077504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:50:20.962431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.044061+0000","last_change":"2024-01-15T18:00:41.063462+0000","last_active":"2024-01-15T18:00:42.044061+0000","last_peered":"2024-01-15T18:00:42.044061+0000","last_clean":"2024-01-15T18:00:42.044061+0000","last_became_active":"2024-01-15T18:00:41.063125+0000","last_became_peered":"2024-01-15T18:00:41.063125+0000","last_unstale":"2024-01-15T18:00:42.044061+0000","last_undegraded":"2024-01-15T18:00:42.044061+0000","last_fullsized":"2024-01-15T18:00:42.044061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:26:23.829753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081534+0000","last_change":"2024-01-15T18:00:42.079309+0000","last_active":"2024-01-15T18:00:42.081534+0000","last_peered":"2024-01-15T18:00:42.081534+0000","last_clean":"2024-01-15T18:00:42.081534+0000","last_became_active":"2024-01-15T18:00:42.078911+0000","last_became_peered":"2024-01-15T18:00:42.078911+0000","last_unstale":"2024-01-15T18:00:42.081534+0000","last_undegraded":"2024-01-15T18:00:42.081534+0000","last_fullsized":"2024-01-15T18:00:42.081534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:05:36.094656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.090879+0000","last_change":"2024-01-15T18:00:41.069799+0000","last_active":"2024-01-15T18:00:42.090879+0000","last_peered":"2024-01-15T18:00:42.090879+0000","last_clean":"2024-01-15T18:00:42.090879+0000","last_became_active":"2024-01-15T18:00:41.069467+0000","last_became_peered":"2024-01-15T18:00:41.069467+0000","last_unstale":"2024-01-15T18:00:42.090879+0000","last_undegraded":"2024-01-15T18:00:42.090879+0000","last_fullsized":"2024-01-15T18:00:42.090879+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.066827+0000","last_change":"2024-01-15T18:00:42.064990+0000","last_active":"2024-01-15T18:00:42.066827+0000","last_peered":"2024-01-15T18:00:42.066827+0000","last_clean":"2024-01-15T18:00:42.066827+0000","last_became_active":"2024-01-15T18:00:42.064698+0000","last_became_peered":"2024-01-15T18:00:42.064698+0000","last_unstale":"2024-01-15T18:00:42.066827+0000","last_undegraded":"2024-01-15T18:00:42.066827+0000","last_fullsized":"2024-01-15T18:00:42.066827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:23:58.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069728+0000","last_change":"2024-01-15T18:00:41.057228+0000","last_active":"2024-01-15T18:00:42.069728+0000","last_peered":"2024-01-15T18:00:42.069728+0000","last_clean":"2024-01-15T18:00:42.069728+0000","last_became_active":"2024-01-15T18:00:41.056929+0000","last_became_peered":"2024-01-15T18:00:41.056929+0000","last_unstale":"2024-01-15T18:00:42.069728+0000","last_undegraded":"2024-01-15T18:00:42.069728+0000","last_fullsized":"2024-01-15T18:00:42.069728+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.065721+0000","last_change":"2024-01-15T18:00:42.057749+0000","last_active":"2024-01-15T18:00:42.065721+0000","last_peered":"2024-01-15T18:00:42.065721+0000","last_clean":"2024-01-15T18:00:42.065721+0000","last_became_active":"2024-01-15T18:00:42.054149+0000","last_became_peered":"2024-01-15T18:00:42.054149+0000","last_unstale":"2024-01-15T18:00:42.065721+0000","last_undegraded":"2024-01-15T18:00:42.065721+0000","last_fullsized":"2024-01-15T18:00:42.065721+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:51.141215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.047331+0000","last_change":"2024-01-15T18:00:41.067706+0000","last_active":"2024-01-15T18:00:42.047331+0000","last_peered":"2024-01-15T18:00:42.047331+0000","last_clean":"2024-01-15T18:00:42.047331+0000","last_became_active":"2024-01-15T18:00:41.067325+0000","last_became_peered":"2024-01-15T18:00:41.067325+0000","last_unstale":"2024-01-15T18:00:42.047331+0000","last_undegraded":"2024-01-15T18:00:42.047331+0000","last_fullsized":"2024-01-15T18:00:42.047331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:50:53.105108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063135+0000","last_change":"2024-01-15T18:00:42.063135+0000","last_active":"2024-01-15T18:00:42.063135+0000","last_peered":"2024-01-15T18:00:42.063135+0000","last_clean":"2024-01-15T18:00:42.063135+0000","last_became_active":"2024-01-15T18:00:42.062863+0000","last_became_peered":"2024-01-15T18:00:42.062863+0000","last_unstale":"2024-01-15T18:00:42.063135+0000","last_undegraded":"2024-01-15T18:00:42.063135+0000","last_fullsized":"2024-01-15T18:00:42.063135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:35.983660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.052982+0000","last_change":"2024-01-15T18:00:41.073770+0000","last_active":"2024-01-15T18:00:42.052982+0000","last_peered":"2024-01-15T18:00:42.052982+0000","last_clean":"2024-01-15T18:00:42.052982+0000","last_became_active":"2024-01-15T18:00:41.073486+0000","last_became_peered":"2024-01-15T18:00:41.073486+0000","last_unstale":"2024-01-15T18:00:42.052982+0000","last_undegraded":"2024-01-15T18:00:42.052982+0000","last_fullsized":"2024-01-15T18:00:42.052982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069498+0000","last_change":"2024-01-15T18:00:42.064514+0000","last_active":"2024-01-15T18:00:42.069498+0000","last_peered":"2024-01-15T18:00:42.069498+0000","last_clean":"2024-01-15T18:00:42.069498+0000","last_became_active":"2024-01-15T18:00:42.064248+0000","last_became_peered":"2024-01-15T18:00:42.064248+0000","last_unstale":"2024-01-15T18:00:42.069498+0000","last_undegraded":"2024-01-15T18:00:42.069498+0000","last_fullsized":"2024-01-15T18:00:42.069498+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:00.533678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.036548+0000","last_change":"2024-01-15T18:00:41.064452+0000","last_active":"2024-01-15T18:00:42.036548+0000","last_peered":"2024-01-15T18:00:42.036548+0000","last_clean":"2024-01-15T18:00:42.036548+0000","last_became_active":"2024-01-15T18:00:41.064118+0000","last_became_peered":"2024-01-15T18:00:41.064118+0000","last_unstale":"2024-01-15T18:00:42.036548+0000","last_undegraded":"2024-01-15T18:00:42.036548+0000","last_fullsized":"2024-01-15T18:00:42.036548+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:31.276292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.068233+0000","last_change":"2024-01-15T18:00:42.068233+0000","last_active":"2024-01-15T18:00:42.068233+0000","last_peered":"2024-01-15T18:00:42.068233+0000","last_clean":"2024-01-15T18:00:42.068233+0000","last_became_active":"2024-01-15T18:00:42.066136+0000","last_became_peered":"2024-01-15T18:00:42.066136+0000","last_unstale":"2024-01-15T18:00:42.068233+0000","last_undegraded":"2024-01-15T18:00:42.068233+0000","last_fullsized":"2024-01-15T18:00:42.068233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:03:53.619388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085512+0000","last_change":"2024-01-15T18:00:41.069415+0000","last_active":"2024-01-15T18:00:42.085512+0000","last_peered":"2024-01-15T18:00:42.085512+0000","last_clean":"2024-01-15T18:00:42.085512+0000","last_became_active":"2024-01-15T18:00:41.069194+0000","last_became_peered":"2024-01-15T18:00:41.069194+0000","last_unstale":"2024-01-15T18:00:42.085512+0000","last_undegraded":"2024-01-15T18:00:42.085512+0000","last_fullsized":"2024-01-15T18:00:42.085512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:08.061519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076472+0000","last_change":"2024-01-15T18:00:42.072986+0000","last_active":"2024-01-15T18:00:42.076472+0000","last_peered":"2024-01-15T18:00:42.076472+0000","last_clean":"2024-01-15T18:00:42.076472+0000","last_became_active":"2024-01-15T18:00:42.072700+0000","last_became_peered":"2024-01-15T18:00:42.072700+0000","last_unstale":"2024-01-15T18:00:42.076472+0000","last_undegraded":"2024-01-15T18:00:42.076472+0000","last_fullsized":"2024-01-15T18:00:42.076472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:48.146540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.044213+0000","last_change":"2024-01-15T18:00:41.056524+0000","last_active":"2024-01-15T18:00:42.044213+0000","last_peered":"2024-01-15T18:00:42.044213+0000","last_clean":"2024-01-15T18:00:42.044213+0000","last_became_active":"2024-01-15T18:00:41.056086+0000","last_became_peered":"2024-01-15T18:00:41.056086+0000","last_unstale":"2024-01-15T18:00:42.044213+0000","last_undegraded":"2024-01-15T18:00:42.044213+0000","last_fullsized":"2024-01-15T18:00:42.044213+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:36.552797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.075686+0000","last_change":"2024-01-15T18:00:42.072006+0000","last_active":"2024-01-15T18:00:42.075686+0000","last_peered":"2024-01-15T18:00:42.075686+0000","last_clean":"2024-01-15T18:00:42.075686+0000","last_became_active":"2024-01-15T18:00:42.071671+0000","last_became_peered":"2024-01-15T18:00:42.071671+0000","last_unstale":"2024-01-15T18:00:42.075686+0000","last_undegraded":"2024-01-15T18:00:42.075686+0000","last_fullsized":"2024-01-15T18:00:42.075686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085054+0000","last_change":"2024-01-15T18:00:41.067157+0000","last_active":"2024-01-15T18:00:42.085054+0000","last_peered":"2024-01-15T18:00:42.085054+0000","last_clean":"2024-01-15T18:00:42.085054+0000","last_became_active":"2024-01-15T18:00:41.066820+0000","last_became_peered":"2024-01-15T18:00:41.066820+0000","last_unstale":"2024-01-15T18:00:42.085054+0000","last_undegraded":"2024-01-15T18:00:42.085054+0000","last_fullsized":"2024-01-15T18:00:42.085054+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:03:48.949979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.068749+0000","last_change":"2024-01-15T18:00:42.063066+0000","last_active":"2024-01-15T18:00:42.068749+0000","last_peered":"2024-01-15T18:00:42.068749+0000","last_clean":"2024-01-15T18:00:42.068749+0000","last_became_active":"2024-01-15T18:00:42.062649+0000","last_became_peered":"2024-01-15T18:00:42.062649+0000","last_unstale":"2024-01-15T18:00:42.068749+0000","last_undegraded":"2024-01-15T18:00:42.068749+0000","last_fullsized":"2024-01-15T18:00:42.068749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:52.055388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.040717+0000","last_change":"2024-01-15T18:00:41.061164+0000","last_active":"2024-01-15T18:00:42.040717+0000","last_peered":"2024-01-15T18:00:42.040717+0000","last_clean":"2024-01-15T18:00:42.040717+0000","last_became_active":"2024-01-15T18:00:41.060847+0000","last_became_peered":"2024-01-15T18:00:41.060847+0000","last_unstale":"2024-01-15T18:00:42.040717+0000","last_undegraded":"2024-01-15T18:00:42.040717+0000","last_fullsized":"2024-01-15T18:00:42.040717+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":3,"reported_epoch":59,"state":"creating+activating","last_fresh":"2024-01-15T18:00:42.046824+0000","last_change":"2024-01-15T18:00:42.046824+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:42.046824+0000","last_undegraded":"2024-01-15T18:00:42.046824+0000","last_fullsized":"2024-01-15T18:00:42.046824+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:22:14.891183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069884+0000","last_change":"2024-01-15T18:00:41.060084+0000","last_active":"2024-01-15T18:00:42.069884+0000","last_peered":"2024-01-15T18:00:42.069884+0000","last_clean":"2024-01-15T18:00:42.069884+0000","last_became_active":"2024-01-15T18:00:41.057544+0000","last_became_peered":"2024-01-15T18:00:41.057544+0000","last_unstale":"2024-01-15T18:00:42.069884+0000","last_undegraded":"2024-01-15T18:00:42.069884+0000","last_fullsized":"2024-01-15T18:00:42.069884+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076361+0000","last_change":"2024-01-15T18:00:42.071201+0000","last_active":"2024-01-15T18:00:42.076361+0000","last_peered":"2024-01-15T18:00:42.076361+0000","last_clean":"2024-01-15T18:00:42.076361+0000","last_became_active":"2024-01-15T18:00:42.070498+0000","last_became_peered":"2024-01-15T18:00:42.070498+0000","last_unstale":"2024-01-15T18:00:42.076361+0000","last_undegraded":"2024-01-15T18:00:42.076361+0000","last_fullsized":"2024-01-15T18:00:42.076361+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:27.820063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.087649+0000","last_change":"2024-01-15T18:00:41.064043+0000","last_active":"2024-01-15T18:00:42.087649+0000","last_peered":"2024-01-15T18:00:42.087649+0000","last_clean":"2024-01-15T18:00:42.087649+0000","last_became_active":"2024-01-15T18:00:41.063128+0000","last_became_peered":"2024-01-15T18:00:41.063128+0000","last_unstale":"2024-01-15T18:00:42.087649+0000","last_undegraded":"2024-01-15T18:00:42.087649+0000","last_fullsized":"2024-01-15T18:00:42.087649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:12.496547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083296+0000","last_change":"2024-01-15T18:00:42.079795+0000","last_active":"2024-01-15T18:00:42.083296+0000","last_peered":"2024-01-15T18:00:42.083296+0000","last_clean":"2024-01-15T18:00:42.083296+0000","last_became_active":"2024-01-15T18:00:42.079150+0000","last_became_peered":"2024-01-15T18:00:42.079150+0000","last_unstale":"2024-01-15T18:00:42.083296+0000","last_undegraded":"2024-01-15T18:00:42.083296+0000","last_fullsized":"2024-01-15T18:00:42.083296+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:41.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.084348+0000","last_change":"2024-01-15T18:00:41.071443+0000","last_active":"2024-01-15T18:00:42.084348+0000","last_peered":"2024-01-15T18:00:42.084348+0000","last_clean":"2024-01-15T18:00:42.084348+0000","last_became_active":"2024-01-15T18:00:41.071219+0000","last_became_peered":"2024-01-15T18:00:41.071219+0000","last_unstale":"2024-01-15T18:00:42.084348+0000","last_undegraded":"2024-01-15T18:00:42.084348+0000","last_fullsized":"2024-01-15T18:00:42.084348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:56.788064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081207+0000","last_change":"2024-01-15T18:00:42.079222+0000","last_active":"2024-01-15T18:00:42.081207+0000","last_peered":"2024-01-15T18:00:42.081207+0000","last_clean":"2024-01-15T18:00:42.081207+0000","last_became_active":"2024-01-15T18:00:42.078933+0000","last_became_peered":"2024-01-15T18:00:42.078933+0000","last_unstale":"2024-01-15T18:00:42.081207+0000","last_undegraded":"2024-01-15T18:00:42.081207+0000","last_fullsized":"2024-01-15T18:00:42.081207+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:44.207537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.091309+0000","last_change":"2024-01-15T18:00:41.071043+0000","last_active":"2024-01-15T18:00:42.091309+0000","last_peered":"2024-01-15T18:00:42.091309+0000","last_clean":"2024-01-15T18:00:42.091309+0000","last_became_active":"2024-01-15T18:00:41.070768+0000","last_became_peered":"2024-01-15T18:00:41.070768+0000","last_unstale":"2024-01-15T18:00:42.091309+0000","last_undegraded":"2024-01-15T18:00:42.091309+0000","last_fullsized":"2024-01-15T18:00:42.091309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080200+0000","last_change":"2024-01-15T18:00:42.076070+0000","last_active":"2024-01-15T18:00:42.080200+0000","last_peered":"2024-01-15T18:00:42.080200+0000","last_clean":"2024-01-15T18:00:42.080200+0000","last_became_active":"2024-01-15T18:00:42.068998+0000","last_became_peered":"2024-01-15T18:00:42.068998+0000","last_unstale":"2024-01-15T18:00:42.080200+0000","last_undegraded":"2024-01-15T18:00:42.080200+0000","last_fullsized":"2024-01-15T18:00:42.080200+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:08:12.230431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.072591+0000","last_change":"2024-01-15T18:00:41.061292+0000","last_active":"2024-01-15T18:00:42.072591+0000","last_peered":"2024-01-15T18:00:42.072591+0000","last_clean":"2024-01-15T18:00:42.072591+0000","last_became_active":"2024-01-15T18:00:41.061006+0000","last_became_peered":"2024-01-15T18:00:41.061006+0000","last_unstale":"2024-01-15T18:00:42.072591+0000","last_undegraded":"2024-01-15T18:00:42.072591+0000","last_fullsized":"2024-01-15T18:00:42.072591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.073395+0000","last_change":"2024-01-15T18:00:42.067416+0000","last_active":"2024-01-15T18:00:42.073395+0000","last_peered":"2024-01-15T18:00:42.073395+0000","last_clean":"2024-01-15T18:00:42.073395+0000","last_became_active":"2024-01-15T18:00:42.067129+0000","last_became_peered":"2024-01-15T18:00:42.067129+0000","last_unstale":"2024-01-15T18:00:42.073395+0000","last_undegraded":"2024-01-15T18:00:42.073395+0000","last_fullsized":"2024-01-15T18:00:42.073395+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:49.570465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070835+0000","last_change":"2024-01-15T18:00:41.071456+0000","last_active":"2024-01-15T18:00:42.070835+0000","last_peered":"2024-01-15T18:00:42.070835+0000","last_clean":"2024-01-15T18:00:42.070835+0000","last_became_active":"2024-01-15T18:00:41.071189+0000","last_became_peered":"2024-01-15T18:00:41.071189+0000","last_unstale":"2024-01-15T18:00:42.070835+0000","last_undegraded":"2024-01-15T18:00:42.070835+0000","last_fullsized":"2024-01-15T18:00:42.070835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070175+0000","last_change":"2024-01-15T18:00:42.065297+0000","last_active":"2024-01-15T18:00:42.070175+0000","last_peered":"2024-01-15T18:00:42.070175+0000","last_clean":"2024-01-15T18:00:42.070175+0000","last_became_active":"2024-01-15T18:00:42.064969+0000","last_became_peered":"2024-01-15T18:00:42.064969+0000","last_unstale":"2024-01-15T18:00:42.070175+0000","last_undegraded":"2024-01-15T18:00:42.070175+0000","last_fullsized":"2024-01-15T18:00:42.070175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.084074+0000","last_change":"2024-01-15T18:00:41.068101+0000","last_active":"2024-01-15T18:00:42.084074+0000","last_peered":"2024-01-15T18:00:42.084074+0000","last_clean":"2024-01-15T18:00:42.084074+0000","last_became_active":"2024-01-15T18:00:41.065928+0000","last_became_peered":"2024-01-15T18:00:41.065928+0000","last_unstale":"2024-01-15T18:00:42.084074+0000","last_undegraded":"2024-01-15T18:00:42.084074+0000","last_fullsized":"2024-01-15T18:00:42.084074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:19:23.525421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.075413+0000","last_change":"2024-01-15T18:00:42.067592+0000","last_active":"2024-01-15T18:00:42.075413+0000","last_peered":"2024-01-15T18:00:42.075413+0000","last_clean":"2024-01-15T18:00:42.075413+0000","last_became_active":"2024-01-15T18:00:42.067309+0000","last_became_peered":"2024-01-15T18:00:42.067309+0000","last_unstale":"2024-01-15T18:00:42.075413+0000","last_undegraded":"2024-01-15T18:00:42.075413+0000","last_fullsized":"2024-01-15T18:00:42.075413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.046741+0000","last_change":"2024-01-15T18:00:41.050813+0000","last_active":"2024-01-15T18:00:42.046741+0000","last_peered":"2024-01-15T18:00:42.046741+0000","last_clean":"2024-01-15T18:00:42.046741+0000","last_became_active":"2024-01-15T18:00:41.050320+0000","last_became_peered":"2024-01-15T18:00:41.050320+0000","last_unstale":"2024-01-15T18:00:42.046741+0000","last_undegraded":"2024-01-15T18:00:42.046741+0000","last_fullsized":"2024-01-15T18:00:42.046741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.075305+0000","last_change":"2024-01-15T18:00:42.070090+0000","last_active":"2024-01-15T18:00:42.075305+0000","last_peered":"2024-01-15T18:00:42.075305+0000","last_clean":"2024-01-15T18:00:42.075305+0000","last_became_active":"2024-01-15T18:00:42.068897+0000","last_became_peered":"2024-01-15T18:00:42.068897+0000","last_unstale":"2024-01-15T18:00:42.075305+0000","last_undegraded":"2024-01-15T18:00:42.075305+0000","last_fullsized":"2024-01-15T18:00:42.075305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070801+0000","last_change":"2024-01-15T18:00:41.065251+0000","last_active":"2024-01-15T18:00:42.070801+0000","last_peered":"2024-01-15T18:00:42.070801+0000","last_clean":"2024-01-15T18:00:42.070801+0000","last_became_active":"2024-01-15T18:00:41.065006+0000","last_became_peered":"2024-01-15T18:00:41.065006+0000","last_unstale":"2024-01-15T18:00:42.070801+0000","last_undegraded":"2024-01-15T18:00:42.070801+0000","last_fullsized":"2024-01-15T18:00:42.070801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083131+0000","last_change":"2024-01-15T18:00:42.083131+0000","last_active":"2024-01-15T18:00:42.083131+0000","last_peered":"2024-01-15T18:00:42.083131+0000","last_clean":"2024-01-15T18:00:42.083131+0000","last_became_active":"2024-01-15T18:00:42.082546+0000","last_became_peered":"2024-01-15T18:00:42.082546+0000","last_unstale":"2024-01-15T18:00:42.083131+0000","last_undegraded":"2024-01-15T18:00:42.083131+0000","last_fullsized":"2024-01-15T18:00:42.083131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:24.550976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.046483+0000","last_change":"2024-01-15T18:00:41.066193+0000","last_active":"2024-01-15T18:00:42.046483+0000","last_peered":"2024-01-15T18:00:42.046483+0000","last_clean":"2024-01-15T18:00:42.046483+0000","last_became_active":"2024-01-15T18:00:41.065913+0000","last_became_peered":"2024-01-15T18:00:41.065913+0000","last_unstale":"2024-01-15T18:00:42.046483+0000","last_undegraded":"2024-01-15T18:00:42.046483+0000","last_fullsized":"2024-01-15T18:00:42.046483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.079328+0000","last_change":"2024-01-15T18:00:42.076020+0000","last_active":"2024-01-15T18:00:42.079328+0000","last_peered":"2024-01-15T18:00:42.079328+0000","last_clean":"2024-01-15T18:00:42.079328+0000","last_became_active":"2024-01-15T18:00:42.075751+0000","last_became_peered":"2024-01-15T18:00:42.075751+0000","last_unstale":"2024-01-15T18:00:42.079328+0000","last_undegraded":"2024-01-15T18:00:42.079328+0000","last_fullsized":"2024-01-15T18:00:42.079328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:52.055388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076414+0000","last_change":"2024-01-15T18:00:41.070458+0000","last_active":"2024-01-15T18:00:42.076414+0000","last_peered":"2024-01-15T18:00:42.076414+0000","last_clean":"2024-01-15T18:00:42.076414+0000","last_became_active":"2024-01-15T18:00:41.066322+0000","last_became_peered":"2024-01-15T18:00:41.066322+0000","last_unstale":"2024-01-15T18:00:42.076414+0000","last_undegraded":"2024-01-15T18:00:42.076414+0000","last_fullsized":"2024-01-15T18:00:42.076414+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.066727+0000","last_change":"2024-01-15T18:00:42.066727+0000","last_active":"2024-01-15T18:00:42.066727+0000","last_peered":"2024-01-15T18:00:42.066727+0000","last_clean":"2024-01-15T18:00:42.066727+0000","last_became_active":"2024-01-15T18:00:42.065788+0000","last_became_peered":"2024-01-15T18:00:42.065788+0000","last_unstale":"2024-01-15T18:00:42.066727+0000","last_undegraded":"2024-01-15T18:00:42.066727+0000","last_fullsized":"2024-01-15T18:00:42.066727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:01:13.061488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.040831+0000","last_change":"2024-01-15T18:00:41.063919+0000","last_active":"2024-01-15T18:00:42.040831+0000","last_peered":"2024-01-15T18:00:42.040831+0000","last_clean":"2024-01-15T18:00:42.040831+0000","last_became_active":"2024-01-15T18:00:41.062115+0000","last_became_peered":"2024-01-15T18:00:41.062115+0000","last_unstale":"2024-01-15T18:00:42.040831+0000","last_undegraded":"2024-01-15T18:00:42.040831+0000","last_fullsized":"2024-01-15T18:00:42.040831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080268+0000","last_change":"2024-01-15T18:00:42.073589+0000","last_active":"2024-01-15T18:00:42.080268+0000","last_peered":"2024-01-15T18:00:42.080268+0000","last_clean":"2024-01-15T18:00:42.080268+0000","last_became_active":"2024-01-15T18:00:42.073254+0000","last_became_peered":"2024-01-15T18:00:42.073254+0000","last_unstale":"2024-01-15T18:00:42.080268+0000","last_undegraded":"2024-01-15T18:00:42.080268+0000","last_fullsized":"2024-01-15T18:00:42.080268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:29.053165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070418+0000","last_change":"2024-01-15T18:00:41.057399+0000","last_active":"2024-01-15T18:00:42.070418+0000","last_peered":"2024-01-15T18:00:42.070418+0000","last_clean":"2024-01-15T18:00:42.070418+0000","last_became_active":"2024-01-15T18:00:41.056174+0000","last_became_peered":"2024-01-15T18:00:41.056174+0000","last_unstale":"2024-01-15T18:00:42.070418+0000","last_undegraded":"2024-01-15T18:00:42.070418+0000","last_fullsized":"2024-01-15T18:00:42.070418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.061539+0000","last_change":"2024-01-15T18:00:42.052005+0000","last_active":"2024-01-15T18:00:42.061539+0000","last_peered":"2024-01-15T18:00:42.061539+0000","last_clean":"2024-01-15T18:00:42.061539+0000","last_became_active":"2024-01-15T18:00:42.051122+0000","last_became_peered":"2024-01-15T18:00:42.051122+0000","last_unstale":"2024-01-15T18:00:42.061539+0000","last_undegraded":"2024-01-15T18:00:42.061539+0000","last_fullsized":"2024-01-15T18:00:42.061539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:59:19.158591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.046278+0000","last_change":"2024-01-15T18:00:41.064076+0000","last_active":"2024-01-15T18:00:42.046278+0000","last_peered":"2024-01-15T18:00:42.046278+0000","last_clean":"2024-01-15T18:00:42.046278+0000","last_became_active":"2024-01-15T18:00:41.062980+0000","last_became_peered":"2024-01-15T18:00:41.062980+0000","last_unstale":"2024-01-15T18:00:42.046278+0000","last_undegraded":"2024-01-15T18:00:42.046278+0000","last_fullsized":"2024-01-15T18:00:42.046278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:41:42.728224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.078756+0000","last_change":"2024-01-15T18:00:42.070329+0000","last_active":"2024-01-15T18:00:42.078756+0000","last_peered":"2024-01-15T18:00:42.078756+0000","last_clean":"2024-01-15T18:00:42.078756+0000","last_became_active":"2024-01-15T18:00:42.068798+0000","last_became_peered":"2024-01-15T18:00:42.068798+0000","last_unstale":"2024-01-15T18:00:42.078756+0000","last_undegraded":"2024-01-15T18:00:42.078756+0000","last_fullsized":"2024-01-15T18:00:42.078756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:44.207537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.043282+0000","last_change":"2024-01-15T18:00:41.054271+0000","last_active":"2024-01-15T18:00:42.043282+0000","last_peered":"2024-01-15T18:00:42.043282+0000","last_clean":"2024-01-15T18:00:42.043282+0000","last_became_active":"2024-01-15T18:00:41.053925+0000","last_became_peered":"2024-01-15T18:00:41.053925+0000","last_unstale":"2024-01-15T18:00:42.043282+0000","last_undegraded":"2024-01-15T18:00:42.043282+0000","last_fullsized":"2024-01-15T18:00:42.043282+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:18:46.672266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064312+0000","last_change":"2024-01-15T18:00:42.057721+0000","last_active":"2024-01-15T18:00:42.064312+0000","last_peered":"2024-01-15T18:00:42.064312+0000","last_clean":"2024-01-15T18:00:42.064312+0000","last_became_active":"2024-01-15T18:00:42.057091+0000","last_became_peered":"2024-01-15T18:00:42.057091+0000","last_unstale":"2024-01-15T18:00:42.064312+0000","last_undegraded":"2024-01-15T18:00:42.064312+0000","last_fullsized":"2024-01-15T18:00:42.064312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:33:19.250071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.045742+0000","last_change":"2024-01-15T18:00:41.061944+0000","last_active":"2024-01-15T18:00:42.045742+0000","last_peered":"2024-01-15T18:00:42.045742+0000","last_clean":"2024-01-15T18:00:42.045742+0000","last_became_active":"2024-01-15T18:00:41.061646+0000","last_became_peered":"2024-01-15T18:00:41.061646+0000","last_unstale":"2024-01-15T18:00:42.045742+0000","last_undegraded":"2024-01-15T18:00:42.045742+0000","last_fullsized":"2024-01-15T18:00:42.045742+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:23:57.016225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.065689+0000","last_change":"2024-01-15T18:00:42.055286+0000","last_active":"2024-01-15T18:00:42.065689+0000","last_peered":"2024-01-15T18:00:42.065689+0000","last_clean":"2024-01-15T18:00:42.065689+0000","last_became_active":"2024-01-15T18:00:42.054746+0000","last_became_peered":"2024-01-15T18:00:42.054746+0000","last_unstale":"2024-01-15T18:00:42.065689+0000","last_undegraded":"2024-01-15T18:00:42.065689+0000","last_fullsized":"2024-01-15T18:00:42.065689+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:14.526164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.036713+0000","last_change":"2024-01-15T18:00:41.068287+0000","last_active":"2024-01-15T18:00:42.036713+0000","last_peered":"2024-01-15T18:00:42.036713+0000","last_clean":"2024-01-15T18:00:42.036713+0000","last_became_active":"2024-01-15T18:00:41.066120+0000","last_became_peered":"2024-01-15T18:00:41.066120+0000","last_unstale":"2024-01-15T18:00:42.036713+0000","last_undegraded":"2024-01-15T18:00:42.036713+0000","last_fullsized":"2024-01-15T18:00:42.036713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:39:48.165240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083708+0000","last_change":"2024-01-15T18:00:42.080052+0000","last_active":"2024-01-15T18:00:42.083708+0000","last_peered":"2024-01-15T18:00:42.083708+0000","last_clean":"2024-01-15T18:00:42.083708+0000","last_became_active":"2024-01-15T18:00:42.079699+0000","last_became_peered":"2024-01-15T18:00:42.079699+0000","last_unstale":"2024-01-15T18:00:42.083708+0000","last_undegraded":"2024-01-15T18:00:42.083708+0000","last_fullsized":"2024-01-15T18:00:42.083708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:27.598423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.040946+0000","last_change":"2024-01-15T18:00:41.061168+0000","last_active":"2024-01-15T18:00:42.040946+0000","last_peered":"2024-01-15T18:00:42.040946+0000","last_clean":"2024-01-15T18:00:42.040946+0000","last_became_active":"2024-01-15T18:00:41.060793+0000","last_became_peered":"2024-01-15T18:00:41.060793+0000","last_unstale":"2024-01-15T18:00:42.040946+0000","last_undegraded":"2024-01-15T18:00:42.040946+0000","last_fullsized":"2024-01-15T18:00:42.040946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:02.304705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":195,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-01-15T18:00:41.055451+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:00:41.055451+0000","last_peered":"2024-01-15T18:00:41.055451+0000","last_clean":"2024-01-15T18:00:41.055451+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:00:41.055451+0000","last_undegraded":"2024-01-15T18:00:41.055451+0000","last_fullsized":"2024-01-15T18:00:41.055451+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36600,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335240,"statfs":{"total":96636764160,"available":96599285760,"internally_reserved":0,"allocated":1286144,"data_stored":877555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.2230000000000001,"15min":1.177},"min":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":5.556,"5min":5.556,"15min":5.556},"last":1.179},{"interface":"front","average":{"1min":3.444,"5min":1.649,"15min":1.3260000000000001},"min":{"1min":0.747,"5min":0.623,"15min":0.623},"max":{"1min":38.137,"5min":38.137,"15min":38.137},"last":1.3420000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.266,"15min":1.25},"min":{"1min":0.72999999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":4.194,"5min":1.861,"15min":1.4570000000000001},"min":{"1min":0.72699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":51.843000000000004,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.0009999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.921,"5min":1.361,"15min":1.2270000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":9.3369999999999997,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":1.288},{"interface":"front","average":{"1min":1.643,"5min":1.397,"15min":1.393},"min":{"1min":0.66000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.2889999999999997,"5min":6.2889999999999997,"15min":6.2889999999999997},"last":1.3959999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":1.49,"15min":1.363},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":9.1440000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5029999999999999,"15min":1.4970000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":1.534}]},{"osd":4,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":3.3300000000000001,"5min":1.788,"15min":1.5980000000000001},"min":{"1min":0.95799999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":22.359999999999999,"5min":22.359999999999999,"15min":22.359999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":2.2130000000000001,"5min":1.597,"15min":1.496},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":12.667999999999999,"5min":12.667999999999999,"15min":12.667999999999999},"last":1.599}]},{"osd":5,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.0220000000000002,"5min":2.0569999999999999,"15min":1.7569999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":25.006,"5min":25.006,"15min":25.006},"last":1.649},{"interface":"front","average":{"1min":3.4950000000000001,"5min":1.9239999999999999,"15min":1.712},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":22.693999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.2330000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.9530000000000003,"5min":2.2770000000000001,"15min":1.901},"min":{"1min":1.0580000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":23.834,"5min":23.834,"15min":23.834},"last":1.4359999999999999},{"interface":"front","average":{"1min":5.9169999999999998,"5min":2.4729999999999999,"15min":1.9199999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":63.932000000000002,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.7010000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542215,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.413,"15min":1.3169999999999999},"min":{"1min":0.71699999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.8769999999999998,"5min":12.544,"15min":12.544},"last":1.345},{"interface":"front","average":{"1min":2.1840000000000002,"5min":1.423,"15min":1.248},"min":{"1min":0.81899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":13.98,"5min":13.98,"15min":13.98},"last":1.0880000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.357,"15min":1.2250000000000001},"min":{"1min":0.82599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.5659999999999998,"5min":12.811999999999999,"15min":12.811999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":2.0600000000000001,"5min":1.349,"15min":1.2330000000000001},"min":{"1min":0.86499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":8.8130000000000006,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.1399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2970000000000002,"5min":1.456,"15min":1.28},"min":{"1min":0.90300000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4890000000000001,"15min":1.5009999999999999},"min":{"1min":0.86899999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":4.0419999999999998,"5min":12.897,"15min":12.897},"last":1.466}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.988,"5min":1.7010000000000001,"15min":1.542},"min":{"1min":1.0880000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":9.5,"5min":12.717000000000001,"15min":12.717000000000001},"last":1.5629999999999999},{"interface":"front","average":{"1min":2.52,"5min":1.5840000000000001,"15min":1.4410000000000001},"min":{"1min":0.93799999999999994,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":14.066000000000001,"5min":14.066000000000001,"15min":14.066000000000001},"last":1.835}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5009999999999999,"15min":1.454},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":9.2829999999999995,"5min":9.2829999999999995,"15min":9.2829999999999995},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.6359999999999999,"15min":1.607},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.6989999999999998,"5min":16.98,"15min":16.98},"last":1.6539999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":1.548,"15min":1.484},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":14.176,"5min":14.176,"15min":14.176},"last":1.6919999999999999},{"interface":"front","average":{"1min":2.3079999999999998,"5min":1.756,"15min":1.7569999999999999},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"last":1.3879999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":1.6399999999999999,"15min":1.601},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":14.109999999999999,"5min":14.109999999999999,"15min":14.109999999999999},"last":1.966},{"interface":"front","average":{"1min":2.5870000000000002,"5min":1.8009999999999999,"15min":1.7769999999999999},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":14.478,"5min":14.478,"15min":14.478},"last":1.7529999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3149999999999999,"15min":1.2509999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.7490000000000001,"5min":16.178000000000001,"15min":16.178000000000001},"last":1.046},{"interface":"front","average":{"1min":2.4980000000000002,"5min":1.4750000000000001,"15min":1.417},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":17.184999999999999,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.5349999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2929999999999999,"15min":1.2250000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":5.6219999999999999,"5min":5.6219999999999999,"15min":5.6219999999999999},"last":1.274},{"interface":"front","average":{"1min":1.423,"5min":1.2629999999999999,"15min":1.2110000000000001},"min":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.1379999999999999,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":1.1399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2050000000000001,"15min":1.2190000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.681,"5min":2.835,"15min":2.835},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.367,"15min":1.288},"min":{"1min":1.0760000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.9740000000000002,"5min":5.9740000000000002,"15min":5.9740000000000002},"last":1.327}]},{"osd":3,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.3520000000000001},"min":{"1min":0.73099999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8029999999999999,"5min":14.914,"15min":14.914},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.365,"15min":1.413},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.726,"5min":3.302,"15min":3.302},"last":1.2929999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.395,"15min":1.411},"min":{"1min":0.99299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.7759999999999998,"5min":4.7759999999999998,"15min":4.7759999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.75,"15min":1.571},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":17.991,"5min":17.991,"15min":17.991},"last":0.92100000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.506,"15min":1.6020000000000001},"min":{"1min":0.91900000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":6.4630000000000001,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.792,"5min":1.5369999999999999,"15min":1.55},"min":{"1min":0.88100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":7.7759999999999998,"5min":7.7759999999999998,"15min":7.7759999999999998},"last":1.4159999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5580000000000001,"15min":1.6060000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":6.3230000000000004,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.603,"15min":1.728},"min":{"1min":0.90500000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.2030000000000003,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.109}]}]},{"osd":4,"up_from":14,"seq":60129542215,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.9989999999999997,"5min":11.413,"15min":11.413},"last":1.492},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.4350000000000001,"15min":1.3799999999999999},"min":{"1min":0.77500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":9.7439999999999998,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":1.367}]},{"osd":1,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.4670000000000001,"15min":1.3819999999999999},"min":{"1min":0.84299999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":9.7010000000000005,"5min":11.987,"15min":11.987},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.27,"15min":1.323},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":6.149,"5min":6.149,"15min":6.149},"last":1.2210000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.708,"15min":1.5700000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":7.032,"5min":13.865,"15min":13.865},"last":1.343},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.514,"15min":1.3959999999999999},"min":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.028,"5min":13.903,"15min":13.903},"last":1.4139999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.6040000000000001,"15min":1.5920000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":6.8819999999999997,"5min":11.611000000000001,"15min":11.611000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":2.0800000000000001,"5min":1.546,"15min":1.6180000000000001},"min":{"1min":0.69199999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":13.25,"5min":13.25,"15min":13.25},"last":1.5589999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.6359999999999999,"15min":1.6850000000000001},"min":{"1min":0.61899999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":6.6820000000000004,"5min":11.561,"15min":11.561},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.8999999999999999,"5min":1.5509999999999999,"15min":1.665},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":9.6229999999999993,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.28}]},{"osd":6,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.468,"15min":1.6259999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.698,"5min":1.8500000000000001,"15min":1.8460000000000001},"min":{"1min":0.85799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":6.9329999999999998,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.6439999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":2.359,"5min":1.8979999999999999,"15min":1.9059999999999999},"min":{"1min":1.375,"5min":0.503,"15min":0.503},"max":{"1min":10.157999999999999,"5min":11.894,"15min":11.894},"last":1.456},{"interface":"front","average":{"1min":2.5169999999999999,"5min":2.048,"15min":1.968},"min":{"1min":1.325,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":8.4169999999999998,"5min":11.714,"15min":11.714},"last":1.702}]}]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44340,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327500,"statfs":{"total":96636764160,"available":96591360000,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":1.1990000000000001,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.435,"15min":0.435},"max":{"1min":9.9789999999999992,"5min":9.9789999999999992,"15min":9.9789999999999992},"last":0.76800000000000002},{"interface":"front","average":{"1min":2.9420000000000002,"5min":1.5149999999999999,"15min":1.218},"min":{"1min":0.85199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":21.047000000000001,"5min":21.047000000000001,"15min":21.047000000000001},"last":1.242}]},{"osd":1,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.8919999999999999,"5min":1.524,"15min":1.179},"min":{"1min":0.79700000000000004,"5min":0.375,"15min":0.375},"max":{"1min":17.777999999999999,"5min":17.777999999999999,"15min":17.777999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.786,"5min":1.2210000000000001,"15min":1.135},"min":{"1min":0.69299999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":9.2360000000000007,"5min":9.2360000000000007,"15min":9.2360000000000007},"last":0.94999999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.27,"15min":1.1899999999999999},"min":{"1min":0.79000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.4930000000000003,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.331},{"interface":"front","average":{"1min":2.0510000000000002,"5min":1.492,"15min":1.4119999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":9.3040000000000003,"5min":9.3040000000000003,"15min":9.3040000000000003},"last":1.2889999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":1.51,"15min":1.4199999999999999},"min":{"1min":1.1120000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":1.1299999999999999},{"interface":"front","average":{"1min":2.367,"5min":1.643,"15min":1.4890000000000001},"min":{"1min":0.92300000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":10.313000000000001,"5min":10.313000000000001,"15min":10.313000000000001},"last":1.077}]},{"osd":5,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.2229999999999999,"5min":1.635,"15min":1.5649999999999999},"min":{"1min":1.046,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.243,"5min":10.243,"15min":10.243},"last":1.5},{"interface":"front","average":{"1min":2.2330000000000001,"5min":1.742,"15min":1.6599999999999999},"min":{"1min":0.90500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":6.2000000000000002,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.544}]},{"osd":6,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.5510000000000002,"5min":1.851,"15min":1.7250000000000001},"min":{"1min":0.98399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":10.012,"5min":10.012,"15min":10.012},"last":1.6120000000000001},{"interface":"front","average":{"1min":2.6190000000000002,"5min":1.8220000000000001,"15min":1.722},"min":{"1min":1.0840000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":10.291,"5min":10.291,"15min":10.291},"last":1.5760000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.802,"5min":1.8819999999999999,"15min":1.7589999999999999},"min":{"1min":1.02,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":12.627000000000001,"5min":12.627000000000001,"15min":12.627000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":2.9710000000000001,"5min":1.946,"15min":1.7929999999999999},"min":{"1min":1.1240000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":10.268000000000001,"5min":10.268000000000001,"15min":10.268000000000001},"last":1.2150000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542217,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40180,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331660,"statfs":{"total":96636764160,"available":96595619840,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.042},"min":{"1min":0.48999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.466,"5min":14.427,"15min":14.427},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.0860000000000001,"15min":1.02},"min":{"1min":0.64100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.7290000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3129999999999999,"15min":1.2949999999999999},"min":{"1min":0.53000000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7490000000000001,"5min":14.609,"15min":14.609},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.29,"5min":1.1830000000000001,"15min":1.2010000000000001},"min":{"1min":0.748,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5059999999999998,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.2789999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2609999999999999,"15min":1.1990000000000001},"min":{"1min":0.64700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.772,"5min":16.567,"15min":16.567},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.48,"15min":1.4630000000000001},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8180000000000001,"5min":14.528,"15min":14.528},"last":1.383}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.708,"15min":1.647},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.7690000000000001,"5min":14.779,"15min":14.779},"last":1.4750000000000001},{"interface":"front","average":{"1min":2.073,"5min":1.546,"15min":1.502},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":10.75,"5min":10.75,"15min":10.75},"last":1.5089999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.5609999999999999,"15min":1.506},"min":{"1min":0.85199999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.258,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.653,"5min":1.726,"15min":1.659},"min":{"1min":1,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.4750000000000001,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.5600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.889,"5min":1.5580000000000001,"15min":1.5149999999999999},"min":{"1min":1.131,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.6339999999999999,"15min":1.5},"min":{"1min":1.171,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.415,"5min":17.07,"15min":17.07},"last":1.2390000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.839,"5min":1.621,"15min":1.6259999999999999},"min":{"1min":1.333,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.2170000000000001,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.6040000000000001},{"interface":"front","average":{"1min":2.9049999999999998,"5min":2.1070000000000002,"15min":1.7070000000000001},"min":{"1min":1.2070000000000001,"5min":0.752,"15min":0.752},"max":{"1min":15.135,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.4419999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542216,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40176,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331664,"statfs":{"total":96636764160,"available":96595623936,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.095,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":8.1140000000000008,"5min":8.1140000000000008,"15min":8.1140000000000008},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0169999999999999,"15min":0.99299999999999999},"min":{"1min":0.64000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.456,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.73799999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.4289999999999998,"5min":1.3109999999999999,"15min":0.99399999999999999},"min":{"1min":0.51000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":21.126000000000001,"5min":21.126000000000001,"15min":21.126000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":2.427,"5min":1.5309999999999999,"15min":1.254},"min":{"1min":0.62,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":11.5,"5min":11.5,"15min":11.5},"last":1.0700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.573,"15min":2.0110000000000001},"min":{"1min":0.76400000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":8.3209999999999997,"5min":22.106999999999999,"15min":22.106999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.4239999999999999,"15min":1.244},"min":{"1min":0.58699999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6060000000000001,"5min":15.510999999999999,"15min":15.510999999999999},"last":0.96399999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.841,"5min":1.6419999999999999,"15min":1.74},"min":{"1min":0.88800000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":6.3109999999999999,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.355},{"interface":"front","average":{"1min":1.734,"5min":1.5800000000000001,"15min":1.863},"min":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.8239999999999998,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.246}]},{"osd":5,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0179999999999998,"5min":1.6339999999999999,"15min":1.8600000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":11.552,"5min":13.228999999999999,"15min":13.228999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.851,"5min":1.9159999999999999,"15min":1.706},"min":{"1min":0.85399999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":8.2319999999999993,"5min":20.823,"15min":20.823},"last":1.593}]},{"osd":6,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.282,"5min":1.7749999999999999,"15min":2.0369999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":10.696,"5min":14.272,"15min":14.272},"last":1.657},{"interface":"front","average":{"1min":2.1150000000000002,"5min":1.637,"15min":1.5309999999999999},"min":{"1min":1.0629999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":8.8320000000000007,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.143}]},{"osd":7,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.6459999999999999,"15min":1.575},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":8.2769999999999992,"5min":8.2769999999999992,"15min":8.2769999999999992},"last":1.8580000000000001},{"interface":"front","average":{"1min":3.2530000000000001,"5min":1.881,"15min":1.665},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":24.289999999999999,"5min":24.289999999999999,"15min":24.289999999999999},"last":1.468}]}]},{"osd":0,"up_from":14,"seq":60129542217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40760,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331080,"statfs":{"total":96636764160,"available":96595025920,"internally_reserved":0,"allocated":1286144,"data_stored":877555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.92100000000000004,"15min":0.88500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.1559999999999999,"15min":1.1000000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.2709999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.012},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.0009999999999999,"15min":0.88},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7490000000000001,"5min":2.129,"15min":2.129},"last":0.80700000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.8440000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.381},{"interface":"front","average":{"1min":1.792,"5min":1.3049999999999999,"15min":1.123},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":14.74,"5min":14.74,"15min":14.74},"last":1.0049999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8109999999999999,"5min":1.4179999999999999,"15min":1.3080000000000001},"min":{"1min":1.004,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":8.6959999999999997,"5min":8.6959999999999997,"15min":8.6959999999999997},"last":1.3100000000000001},{"interface":"front","average":{"1min":2.3540000000000001,"5min":1.6839999999999999,"15min":1.5549999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":15.975,"5min":15.975,"15min":15.975},"last":1.5429999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":1.502,"15min":1.373},"min":{"1min":1.0860000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":3.133,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.1639999999999999},{"interface":"front","average":{"1min":2.1000000000000001,"5min":1.571,"15min":1.5129999999999999},"min":{"1min":1.0669999999999999,"5min":0.754,"15min":0.754},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.835}]},{"osd":6,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.524},"min":{"1min":0.96999999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":2.0190000000000001},{"interface":"front","average":{"1min":1.732,"5min":1.526,"15min":1.462},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":1.4399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0409999999999999,"5min":1.71,"15min":1.6639999999999999},"min":{"1min":1.214,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":2.1970000000000001,"5min":1.7090000000000001,"15min":1.593},"min":{"1min":1.012,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":6.2619999999999996,"5min":6.2619999999999996,"15min":6.2619999999999996},"last":1.954}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:00:46.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:00:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.815+0000 7fb034600640 1 Processor -- start 2024-01-15T18:00:46.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.815+0000 7fb034600640 1 -- start start 2024-01-15T18:00:46.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.817+0000 7fb034600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb02c0cf9a0 0x7fb02c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.817+0000 7fb034600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02c070c40 con 0x7fb02c0ae5a0 2024-01-15T18:00:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.817+0000 7fb034600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb02c06c470 con 0x7fb02c0b3ae0 2024-01-15T18:00:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.817+0000 7fb034600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c071450 con 0x7fb02c0cf9a0 2024-01-15T18:00:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.817+0000 7fb031c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb02c0ae5a0 0x7fb02c06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54276/0 (socket says 172.21.15.26:54276) 2024-01-15T18:00:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.817+0000 7fb031c00640 1 -- 172.21.15.26:0/1143058373 learned_addr learned my addr 172.21.15.26:0/1143058373 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.817+0000 7fb032600640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb02c0cf9a0 0x7fb02c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.817+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473822509 0 0) 0x7fb02c070c40 con 0x7fb02c0ae5a0 2024-01-15T18:00:46.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb00c003520 con 0x7fb02c0ae5a0 2024-01-15T18:00:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1855948301 0 0) 0x7fb00c003520 con 0x7fb02c0ae5a0 2024-01-15T18:00:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb02c0cf9a0 msgr2=0x7fb02c15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb02c0cf9a0 0x7fb02c15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 >> v1:172.21.15.124:6789/0 conn(0x7fb02c0b3ae0 legacy=0x7fb02c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c06c750 con 0x7fb02c0ae5a0 2024-01-15T18:00:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb01c001670 con 0x7fb02c0ae5a0 2024-01-15T18:00:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 >> v1:172.21.15.26:6789/0 conn(0x7fb02c0ae5a0 legacy=0x7fb02c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb02c0cf9a0 0x7fb00c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb00c003f20 0x7fb00c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00c004d00 0x7fb00c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c071450 con 0x7fb00c004d00 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c00baf0 con 0x7fb00c003f20 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.818+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c00bcb0 con 0x7fb02c0cf9a0 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.819+0000 7fb032600640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00c004d00 0x7fb00c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.819+0000 7fb031200640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb00c003f20 0x7fb00c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.819+0000 7fb031c00640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb02c0cf9a0 0x7fb00c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.819+0000 7fb032600640 1 -- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb02c0cf9a0 msgr2=0x7fb00c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.819+0000 7fb032600640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb02c0cf9a0 0x7fb00c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.819+0000 7fb032600640 1 -- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb00c003f20 msgr2=0x7fb00c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:46.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.819+0000 7fb032600640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb00c003f20 0x7fb00c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.819+0000 7fb032600640 1 -- 172.21.15.26:0/1143058373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c003520 con 0x7fb00c004d00 2024-01-15T18:00:46.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.820+0000 7fb032600640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00c004d00 0x7fb00c005180 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb024009fc0 tx=0x7fb00c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5003042eac409d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:46.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.820+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02402cc20 con 0x7fb00c004d00 2024-01-15T18:00:46.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.820+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02400f040 con 0x7fb00c004d00 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.820+0000 7fb02be00640 1 -- 172.21.15.26:0/1143058373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024034440 con 0x7fb00c004d00 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.821+0000 7fb034600640 1 -- 172.21.15.26:0/1143058373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00c004d00 msgr2=0x7fb00c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.821+0000 7fb034600640 1 --2- 172.21.15.26:0/1143058373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb00c004d00 0x7fb00c005180 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb024009fc0 tx=0x7fb00c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.821+0000 7fb032600640 1 -- 172.21.15.26:0/1143058373 reap_dead start 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.821+0000 7fb034600640 1 -- 172.21.15.26:0/1143058373 shutdown_connections 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.821+0000 7fb034600640 1 -- 172.21.15.26:0/1143058373 >> 172.21.15.26:0/1143058373 conn(0x7fb02c06a800 msgr2=0x7fb02c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.821+0000 7fb034600640 1 -- 172.21.15.26:0/1143058373 shutdown_connections 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.821+0000 7fb034600640 1 -- 172.21.15.26:0/1143058373 wait complete. 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.823+0000 7fb034600640 1 Processor -- start 2024-01-15T18:00:46.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.823+0000 7fb034600640 1 -- start start 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.824+0000 7fb034600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb00c004d00 0x7fb02c0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.824+0000 7fb034600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb02c1602f0 0x7fb02c1f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.824+0000 7fb034600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb02c160750 0x7fb02c1f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.824+0000 7fb034600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c1f6e60 con 0x7fb02c160750 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.824+0000 7fb034600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c1f6b80 con 0x7fb02c1602f0 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.824+0000 7fb034600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb02c0cb9d0 con 0x7fb00c004d00 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb032600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb02c160750 0x7fb02c1f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb032600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb02c160750 0x7fb02c1f6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36988/0 (socket says 172.21.15.26:36988) 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb032600640 1 -- 172.21.15.26:0/1470518511 learned_addr learned my addr 172.21.15.26:0/1470518511 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb031200640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb02c1602f0 0x7fb02c1f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb031c00640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb00c004d00 0x7fb02c0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb032600640 1 -- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb00c004d00 msgr2=0x7fb02c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb032600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb00c004d00 0x7fb02c0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb032600640 1 -- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb02c1602f0 msgr2=0x7fb02c1f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb032600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb02c1602f0 0x7fb02c1f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb032600640 1 -- 172.21.15.26:0/1470518511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c0cbcb0 con 0x7fb02c160750 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb031200640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb02c1602f0 0x7fb02c1f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.825+0000 7fb032600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb02c160750 0x7fb02c1f6490 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb024002230 tx=0x7fb024009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.826+0000 7fb029600640 1 -- 172.21.15.26:0/1470518511 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024034ad0 con 0x7fb02c160750 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.826+0000 7fb029600640 1 -- 172.21.15.26:0/1470518511 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb024034c30 con 0x7fb02c160750 2024-01-15T18:00:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.826+0000 7fb029600640 1 -- 172.21.15.26:0/1470518511 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024003990 con 0x7fb02c160750 2024-01-15T18:00:46.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.826+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02c0cbf90 con 0x7fb02c160750 2024-01-15T18:00:46.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.826+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02c0c5310 con 0x7fb02c160750 2024-01-15T18:00:46.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.827+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02c0cc180 con 0x7fb02c160750 2024-01-15T18:00:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.828+0000 7fb029600640 1 -- 172.21.15.26:0/1470518511 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fb02403e440 con 0x7fb02c160750 2024-01-15T18:00:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.829+0000 7fb029600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb00406b860 0x7fb00406dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.829+0000 7fb029600640 1 -- 172.21.15.26:0/1470518511 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb0240ac3f0 con 0x7fb02c160750 2024-01-15T18:00:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.830+0000 7fb031c00640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb00406b860 0x7fb00406dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.831+0000 7fb031c00640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb00406b860 0x7fb00406dd20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb01c001c80 tx=0x7fb01c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:46.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:46.833+0000 7fb029600640 1 -- 172.21.15.26:0/1470518511 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb024076a20 con 0x7fb02c160750 2024-01-15T18:00:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.022+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb02c070550 con 0x7fb00406b860 2024-01-15T18:00:47.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.046+0000 7fb029600640 1 -- 172.21.15.26:0/1470518511 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286126 (secure 0 0 0) 0x7fb01c010920 con 0x7fb00406b860 2024-01-15T18:00:47.055 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:00:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.051+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb00406b860 msgr2=0x7fb00406dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.051+0000 7fb034600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb00406b860 0x7fb00406dd20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb01c001c80 tx=0x7fb01c009040 comp rx=0 tx=0).stop 2024-01-15T18:00:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.051+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb02c160750 msgr2=0x7fb02c1f6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.051+0000 7fb034600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb02c160750 0x7fb02c1f6490 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb024002230 tx=0x7fb024009750 comp rx=0 tx=0).stop 2024-01-15T18:00:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.052+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 shutdown_connections 2024-01-15T18:00:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.052+0000 7fb034600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb00406b860 0x7fb00406dd20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.052+0000 7fb034600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb02c160750 0x7fb02c1f6490 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.052+0000 7fb034600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb02c1602f0 0x7fb02c1f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.052+0000 7fb034600640 1 --2- 172.21.15.26:0/1470518511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb00c004d00 0x7fb02c0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.052+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 >> 172.21.15.26:0/1470518511 conn(0x7fb02c06a800 msgr2=0x7fb02c0acd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:47.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.052+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 shutdown_connections 2024-01-15T18:00:47.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.052+0000 7fb034600640 1 -- 172.21.15.26:0/1470518511 wait complete. 2024-01-15T18:00:47.075 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-01-15T18:00:46.034438+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":194,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":310116,"kb_used_data":6568,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754664604,"statfs":{"total":773094113280,"available":772776554496,"internally_reserved":0,"allocated":6725632,"data_stored":3478232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002328"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.051825+0000","last_change":"2024-01-15T18:00:41.051825+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.051825+0000","last_undegraded":"2024-01-15T18:00:41.051825+0000","last_fullsized":"2024-01-15T18:00:41.051825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063316+0000","last_change":"2024-01-15T18:00:42.060637+0000","last_active":"2024-01-15T18:00:42.063316+0000","last_peered":"2024-01-15T18:00:42.063316+0000","last_clean":"2024-01-15T18:00:42.063316+0000","last_became_active":"2024-01-15T18:00:42.060376+0000","last_became_peered":"2024-01-15T18:00:42.060376+0000","last_unstale":"2024-01-15T18:00:42.063316+0000","last_undegraded":"2024-01-15T18:00:42.063316+0000","last_fullsized":"2024-01-15T18:00:42.063316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:58:12.883647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082872+0000","last_change":"2024-01-15T18:00:42.077438+0000","last_active":"2024-01-15T18:00:42.082872+0000","last_peered":"2024-01-15T18:00:42.082872+0000","last_clean":"2024-01-15T18:00:42.082872+0000","last_became_active":"2024-01-15T18:00:42.077041+0000","last_became_peered":"2024-01-15T18:00:42.077041+0000","last_unstale":"2024-01-15T18:00:42.082872+0000","last_undegraded":"2024-01-15T18:00:42.082872+0000","last_fullsized":"2024-01-15T18:00:42.082872+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.072305+0000","last_change":"2024-01-15T18:00:42.067513+0000","last_active":"2024-01-15T18:00:42.072305+0000","last_peered":"2024-01-15T18:00:42.072305+0000","last_clean":"2024-01-15T18:00:42.072305+0000","last_became_active":"2024-01-15T18:00:42.066627+0000","last_became_peered":"2024-01-15T18:00:42.066627+0000","last_unstale":"2024-01-15T18:00:42.072305+0000","last_undegraded":"2024-01-15T18:00:42.072305+0000","last_fullsized":"2024-01-15T18:00:42.072305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:13.509938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.043365+0000","last_change":"2024-01-15T18:00:41.043365+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.043365+0000","last_undegraded":"2024-01-15T18:00:41.043365+0000","last_fullsized":"2024-01-15T18:00:41.043365+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076462+0000","last_change":"2024-01-15T18:00:42.067558+0000","last_active":"2024-01-15T18:00:42.076462+0000","last_peered":"2024-01-15T18:00:42.076462+0000","last_clean":"2024-01-15T18:00:42.076462+0000","last_became_active":"2024-01-15T18:00:42.067182+0000","last_became_peered":"2024-01-15T18:00:42.067182+0000","last_unstale":"2024-01-15T18:00:42.076462+0000","last_undegraded":"2024-01-15T18:00:42.076462+0000","last_fullsized":"2024-01-15T18:00:42.076462+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:13.509938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064214+0000","last_change":"2024-01-15T18:00:42.064214+0000","last_active":"2024-01-15T18:00:42.064214+0000","last_peered":"2024-01-15T18:00:42.064214+0000","last_clean":"2024-01-15T18:00:42.064214+0000","last_became_active":"2024-01-15T18:00:42.063930+0000","last_became_peered":"2024-01-15T18:00:42.063930+0000","last_unstale":"2024-01-15T18:00:42.064214+0000","last_undegraded":"2024-01-15T18:00:42.064214+0000","last_fullsized":"2024-01-15T18:00:42.064214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:57:42.271743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.079933+0000","last_change":"2024-01-15T18:00:42.072265+0000","last_active":"2024-01-15T18:00:42.079933+0000","last_peered":"2024-01-15T18:00:42.079933+0000","last_clean":"2024-01-15T18:00:42.079933+0000","last_became_active":"2024-01-15T18:00:42.071980+0000","last_became_peered":"2024-01-15T18:00:42.071980+0000","last_unstale":"2024-01-15T18:00:42.079933+0000","last_undegraded":"2024-01-15T18:00:42.079933+0000","last_fullsized":"2024-01-15T18:00:42.079933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:41.900799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":3,"reported_epoch":59,"state":"creating+activating","last_fresh":"2024-01-15T18:00:42.043232+0000","last_change":"2024-01-15T18:00:42.043232+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:42.043232+0000","last_undegraded":"2024-01-15T18:00:42.043232+0000","last_fullsized":"2024-01-15T18:00:42.043232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:25:02.833433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.058936+0000","last_change":"2024-01-15T18:00:42.055688+0000","last_active":"2024-01-15T18:00:42.058936+0000","last_peered":"2024-01-15T18:00:42.058936+0000","last_clean":"2024-01-15T18:00:42.058936+0000","last_became_active":"2024-01-15T18:00:42.055302+0000","last_became_peered":"2024-01-15T18:00:42.055302+0000","last_unstale":"2024-01-15T18:00:42.058936+0000","last_undegraded":"2024-01-15T18:00:42.058936+0000","last_fullsized":"2024-01-15T18:00:42.058936+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:22:28.043841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.066509+0000","last_change":"2024-01-15T18:00:42.066509+0000","last_active":"2024-01-15T18:00:42.066509+0000","last_peered":"2024-01-15T18:00:42.066509+0000","last_clean":"2024-01-15T18:00:42.066509+0000","last_became_active":"2024-01-15T18:00:42.066232+0000","last_became_peered":"2024-01-15T18:00:42.066232+0000","last_unstale":"2024-01-15T18:00:42.066509+0000","last_undegraded":"2024-01-15T18:00:42.066509+0000","last_fullsized":"2024-01-15T18:00:42.066509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:39:13.515709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.072325+0000","last_change":"2024-01-15T18:00:42.068083+0000","last_active":"2024-01-15T18:00:42.072325+0000","last_peered":"2024-01-15T18:00:42.072325+0000","last_clean":"2024-01-15T18:00:42.072325+0000","last_became_active":"2024-01-15T18:00:42.067826+0000","last_became_peered":"2024-01-15T18:00:42.067826+0000","last_unstale":"2024-01-15T18:00:42.072325+0000","last_undegraded":"2024-01-15T18:00:42.072325+0000","last_fullsized":"2024-01-15T18:00:42.072325+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082480+0000","last_change":"2024-01-15T18:00:42.080948+0000","last_active":"2024-01-15T18:00:42.082480+0000","last_peered":"2024-01-15T18:00:42.082480+0000","last_clean":"2024-01-15T18:00:42.082480+0000","last_became_active":"2024-01-15T18:00:42.079522+0000","last_became_peered":"2024-01-15T18:00:42.079522+0000","last_unstale":"2024-01-15T18:00:42.082480+0000","last_undegraded":"2024-01-15T18:00:42.082480+0000","last_fullsized":"2024-01-15T18:00:42.082480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:44.959104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":3,"reported_epoch":59,"state":"creating+activating","last_fresh":"2024-01-15T18:00:42.042710+0000","last_change":"2024-01-15T18:00:42.042710+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:42.042710+0000","last_undegraded":"2024-01-15T18:00:42.042710+0000","last_fullsized":"2024-01-15T18:00:42.042710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:16:20.555824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063338+0000","last_change":"2024-01-15T18:00:42.063338+0000","last_active":"2024-01-15T18:00:42.063338+0000","last_peered":"2024-01-15T18:00:42.063338+0000","last_clean":"2024-01-15T18:00:42.063338+0000","last_became_active":"2024-01-15T18:00:42.062711+0000","last_became_peered":"2024-01-15T18:00:42.062711+0000","last_unstale":"2024-01-15T18:00:42.063338+0000","last_undegraded":"2024-01-15T18:00:42.063338+0000","last_fullsized":"2024-01-15T18:00:42.063338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:23.443661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081709+0000","last_change":"2024-01-15T18:00:42.073413+0000","last_active":"2024-01-15T18:00:42.081709+0000","last_peered":"2024-01-15T18:00:42.081709+0000","last_clean":"2024-01-15T18:00:42.081709+0000","last_became_active":"2024-01-15T18:00:42.072167+0000","last_became_peered":"2024-01-15T18:00:42.072167+0000","last_unstale":"2024-01-15T18:00:42.081709+0000","last_undegraded":"2024-01-15T18:00:42.081709+0000","last_fullsized":"2024-01-15T18:00:42.081709+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:24.550976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.062432+0000","last_change":"2024-01-15T18:00:42.058452+0000","last_active":"2024-01-15T18:00:42.062432+0000","last_peered":"2024-01-15T18:00:42.062432+0000","last_clean":"2024-01-15T18:00:42.062432+0000","last_became_active":"2024-01-15T18:00:42.057885+0000","last_became_peered":"2024-01-15T18:00:42.057885+0000","last_unstale":"2024-01-15T18:00:42.062432+0000","last_undegraded":"2024-01-15T18:00:42.062432+0000","last_fullsized":"2024-01-15T18:00:42.062432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:56.491932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083323+0000","last_change":"2024-01-15T18:00:42.079987+0000","last_active":"2024-01-15T18:00:42.083323+0000","last_peered":"2024-01-15T18:00:42.083323+0000","last_clean":"2024-01-15T18:00:42.083323+0000","last_became_active":"2024-01-15T18:00:42.079525+0000","last_became_peered":"2024-01-15T18:00:42.079525+0000","last_unstale":"2024-01-15T18:00:42.083323+0000","last_undegraded":"2024-01-15T18:00:42.083323+0000","last_fullsized":"2024-01-15T18:00:42.083323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:24.550976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.046537+0000","last_change":"2024-01-15T18:00:41.046537+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.046537+0000","last_undegraded":"2024-01-15T18:00:41.046537+0000","last_fullsized":"2024-01-15T18:00:41.046537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083287+0000","last_change":"2024-01-15T18:00:42.080315+0000","last_active":"2024-01-15T18:00:42.083287+0000","last_peered":"2024-01-15T18:00:42.083287+0000","last_clean":"2024-01-15T18:00:42.083287+0000","last_became_active":"2024-01-15T18:00:42.079623+0000","last_became_peered":"2024-01-15T18:00:42.079623+0000","last_unstale":"2024-01-15T18:00:42.083287+0000","last_undegraded":"2024-01-15T18:00:42.083287+0000","last_fullsized":"2024-01-15T18:00:42.083287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:56:47.128531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.078789+0000","last_change":"2024-01-15T18:00:42.071297+0000","last_active":"2024-01-15T18:00:42.078789+0000","last_peered":"2024-01-15T18:00:42.078789+0000","last_clean":"2024-01-15T18:00:42.078789+0000","last_became_active":"2024-01-15T18:00:42.071046+0000","last_became_peered":"2024-01-15T18:00:42.071046+0000","last_unstale":"2024-01-15T18:00:42.078789+0000","last_undegraded":"2024-01-15T18:00:42.078789+0000","last_fullsized":"2024-01-15T18:00:42.078789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.077554+0000","last_change":"2024-01-15T18:00:42.066557+0000","last_active":"2024-01-15T18:00:42.077554+0000","last_peered":"2024-01-15T18:00:42.077554+0000","last_clean":"2024-01-15T18:00:42.077554+0000","last_became_active":"2024-01-15T18:00:42.066269+0000","last_became_peered":"2024-01-15T18:00:42.066269+0000","last_unstale":"2024-01-15T18:00:42.077554+0000","last_undegraded":"2024-01-15T18:00:42.077554+0000","last_fullsized":"2024-01-15T18:00:42.077554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:40:52.391655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076381+0000","last_change":"2024-01-15T18:00:42.071412+0000","last_active":"2024-01-15T18:00:42.076381+0000","last_peered":"2024-01-15T18:00:42.076381+0000","last_clean":"2024-01-15T18:00:42.076381+0000","last_became_active":"2024-01-15T18:00:42.070884+0000","last_became_peered":"2024-01-15T18:00:42.070884+0000","last_unstale":"2024-01-15T18:00:42.076381+0000","last_undegraded":"2024-01-15T18:00:42.076381+0000","last_fullsized":"2024-01-15T18:00:42.076381+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:17.543171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081314+0000","last_change":"2024-01-15T18:00:42.078634+0000","last_active":"2024-01-15T18:00:42.081314+0000","last_peered":"2024-01-15T18:00:42.081314+0000","last_clean":"2024-01-15T18:00:42.081314+0000","last_became_active":"2024-01-15T18:00:42.078165+0000","last_became_peered":"2024-01-15T18:00:42.078165+0000","last_unstale":"2024-01-15T18:00:42.081314+0000","last_undegraded":"2024-01-15T18:00:42.081314+0000","last_fullsized":"2024-01-15T18:00:42.081314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:41:43.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.051170+0000","last_change":"2024-01-15T18:00:41.051170+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.051170+0000","last_undegraded":"2024-01-15T18:00:41.051170+0000","last_fullsized":"2024-01-15T18:00:41.051170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.079608+0000","last_change":"2024-01-15T18:00:42.076259+0000","last_active":"2024-01-15T18:00:42.079608+0000","last_peered":"2024-01-15T18:00:42.079608+0000","last_clean":"2024-01-15T18:00:42.079608+0000","last_became_active":"2024-01-15T18:00:42.076003+0000","last_became_peered":"2024-01-15T18:00:42.076003+0000","last_unstale":"2024-01-15T18:00:42.079608+0000","last_undegraded":"2024-01-15T18:00:42.079608+0000","last_fullsized":"2024-01-15T18:00:42.079608+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:00.533678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080247+0000","last_change":"2024-01-15T18:00:42.072997+0000","last_active":"2024-01-15T18:00:42.080247+0000","last_peered":"2024-01-15T18:00:42.080247+0000","last_clean":"2024-01-15T18:00:42.080247+0000","last_became_active":"2024-01-15T18:00:42.070411+0000","last_became_peered":"2024-01-15T18:00:42.070411+0000","last_unstale":"2024-01-15T18:00:42.080247+0000","last_undegraded":"2024-01-15T18:00:42.080247+0000","last_fullsized":"2024-01-15T18:00:42.080247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:44.207537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.045517+0000","last_change":"2024-01-15T18:00:41.045517+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.045517+0000","last_undegraded":"2024-01-15T18:00:41.045517+0000","last_fullsized":"2024-01-15T18:00:41.045517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.077697+0000","last_change":"2024-01-15T18:00:42.067598+0000","last_active":"2024-01-15T18:00:42.077697+0000","last_peered":"2024-01-15T18:00:42.077697+0000","last_clean":"2024-01-15T18:00:42.077697+0000","last_became_active":"2024-01-15T18:00:42.067297+0000","last_became_peered":"2024-01-15T18:00:42.067297+0000","last_unstale":"2024-01-15T18:00:42.077697+0000","last_undegraded":"2024-01-15T18:00:42.077697+0000","last_fullsized":"2024-01-15T18:00:42.077697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:42.520698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081309+0000","last_change":"2024-01-15T18:00:42.074513+0000","last_active":"2024-01-15T18:00:42.081309+0000","last_peered":"2024-01-15T18:00:42.081309+0000","last_clean":"2024-01-15T18:00:42.081309+0000","last_became_active":"2024-01-15T18:00:42.074280+0000","last_became_peered":"2024-01-15T18:00:42.074280+0000","last_unstale":"2024-01-15T18:00:42.081309+0000","last_undegraded":"2024-01-15T18:00:42.081309+0000","last_fullsized":"2024-01-15T18:00:42.081309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.043336+0000","last_change":"2024-01-15T18:00:41.043336+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.043336+0000","last_undegraded":"2024-01-15T18:00:41.043336+0000","last_fullsized":"2024-01-15T18:00:41.043336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.088984+0000","last_change":"2024-01-15T18:00:42.086123+0000","last_active":"2024-01-15T18:00:42.088984+0000","last_peered":"2024-01-15T18:00:42.088984+0000","last_clean":"2024-01-15T18:00:42.088984+0000","last_became_active":"2024-01-15T18:00:42.085208+0000","last_became_peered":"2024-01-15T18:00:42.085208+0000","last_unstale":"2024-01-15T18:00:42.088984+0000","last_undegraded":"2024-01-15T18:00:42.088984+0000","last_fullsized":"2024-01-15T18:00:42.088984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:00.533678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069251+0000","last_change":"2024-01-15T18:00:42.065563+0000","last_active":"2024-01-15T18:00:42.069251+0000","last_peered":"2024-01-15T18:00:42.069251+0000","last_clean":"2024-01-15T18:00:42.069251+0000","last_became_active":"2024-01-15T18:00:42.065279+0000","last_became_peered":"2024-01-15T18:00:42.065279+0000","last_unstale":"2024-01-15T18:00:42.069251+0000","last_undegraded":"2024-01-15T18:00:42.069251+0000","last_fullsized":"2024-01-15T18:00:42.069251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:52.055388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.092341+0000","last_change":"2024-01-15T18:00:41.072898+0000","last_active":"2024-01-15T18:00:42.092341+0000","last_peered":"2024-01-15T18:00:42.092341+0000","last_clean":"2024-01-15T18:00:42.092341+0000","last_became_active":"2024-01-15T18:00:41.071115+0000","last_became_peered":"2024-01-15T18:00:41.071115+0000","last_unstale":"2024-01-15T18:00:42.092341+0000","last_undegraded":"2024-01-15T18:00:42.092341+0000","last_fullsized":"2024-01-15T18:00:42.092341+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063277+0000","last_change":"2024-01-15T18:00:42.060261+0000","last_active":"2024-01-15T18:00:42.063277+0000","last_peered":"2024-01-15T18:00:42.063277+0000","last_clean":"2024-01-15T18:00:42.063277+0000","last_became_active":"2024-01-15T18:00:42.059975+0000","last_became_peered":"2024-01-15T18:00:42.059975+0000","last_unstale":"2024-01-15T18:00:42.063277+0000","last_undegraded":"2024-01-15T18:00:42.063277+0000","last_fullsized":"2024-01-15T18:00:42.063277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:44.959104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082453+0000","last_change":"2024-01-15T18:00:41.067396+0000","last_active":"2024-01-15T18:00:42.082453+0000","last_peered":"2024-01-15T18:00:42.082453+0000","last_clean":"2024-01-15T18:00:42.082453+0000","last_became_active":"2024-01-15T18:00:41.067056+0000","last_became_peered":"2024-01-15T18:00:41.067056+0000","last_unstale":"2024-01-15T18:00:42.082453+0000","last_undegraded":"2024-01-15T18:00:42.082453+0000","last_fullsized":"2024-01-15T18:00:42.082453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:24.577110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076694+0000","last_change":"2024-01-15T18:00:42.065885+0000","last_active":"2024-01-15T18:00:42.076694+0000","last_peered":"2024-01-15T18:00:42.076694+0000","last_clean":"2024-01-15T18:00:42.076694+0000","last_became_active":"2024-01-15T18:00:42.065563+0000","last_became_peered":"2024-01-15T18:00:42.065563+0000","last_unstale":"2024-01-15T18:00:42.076694+0000","last_undegraded":"2024-01-15T18:00:42.076694+0000","last_fullsized":"2024-01-15T18:00:42.076694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:53:36.562001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.090834+0000","last_change":"2024-01-15T18:00:41.053014+0000","last_active":"2024-01-15T18:00:42.090834+0000","last_peered":"2024-01-15T18:00:42.090834+0000","last_clean":"2024-01-15T18:00:42.090834+0000","last_became_active":"2024-01-15T18:00:41.052718+0000","last_became_peered":"2024-01-15T18:00:41.052718+0000","last_unstale":"2024-01-15T18:00:42.090834+0000","last_undegraded":"2024-01-15T18:00:42.090834+0000","last_fullsized":"2024-01-15T18:00:42.090834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080283+0000","last_change":"2024-01-15T18:00:42.077700+0000","last_active":"2024-01-15T18:00:42.080283+0000","last_peered":"2024-01-15T18:00:42.080283+0000","last_clean":"2024-01-15T18:00:42.080283+0000","last_became_active":"2024-01-15T18:00:42.077465+0000","last_became_peered":"2024-01-15T18:00:42.077465+0000","last_unstale":"2024-01-15T18:00:42.080283+0000","last_undegraded":"2024-01-15T18:00:42.080283+0000","last_fullsized":"2024-01-15T18:00:42.080283+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070270+0000","last_change":"2024-01-15T18:00:41.057990+0000","last_active":"2024-01-15T18:00:42.070270+0000","last_peered":"2024-01-15T18:00:42.070270+0000","last_clean":"2024-01-15T18:00:42.070270+0000","last_became_active":"2024-01-15T18:00:41.057666+0000","last_became_peered":"2024-01-15T18:00:41.057666+0000","last_unstale":"2024-01-15T18:00:42.070270+0000","last_undegraded":"2024-01-15T18:00:42.070270+0000","last_fullsized":"2024-01-15T18:00:42.070270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.077504+0000","last_change":"2024-01-15T18:00:42.066199+0000","last_active":"2024-01-15T18:00:42.077504+0000","last_peered":"2024-01-15T18:00:42.077504+0000","last_clean":"2024-01-15T18:00:42.077504+0000","last_became_active":"2024-01-15T18:00:42.065918+0000","last_became_peered":"2024-01-15T18:00:42.065918+0000","last_unstale":"2024-01-15T18:00:42.077504+0000","last_undegraded":"2024-01-15T18:00:42.077504+0000","last_fullsized":"2024-01-15T18:00:42.077504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:50:20.962431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.044061+0000","last_change":"2024-01-15T18:00:41.063462+0000","last_active":"2024-01-15T18:00:42.044061+0000","last_peered":"2024-01-15T18:00:42.044061+0000","last_clean":"2024-01-15T18:00:42.044061+0000","last_became_active":"2024-01-15T18:00:41.063125+0000","last_became_peered":"2024-01-15T18:00:41.063125+0000","last_unstale":"2024-01-15T18:00:42.044061+0000","last_undegraded":"2024-01-15T18:00:42.044061+0000","last_fullsized":"2024-01-15T18:00:42.044061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:26:23.829753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081534+0000","last_change":"2024-01-15T18:00:42.079309+0000","last_active":"2024-01-15T18:00:42.081534+0000","last_peered":"2024-01-15T18:00:42.081534+0000","last_clean":"2024-01-15T18:00:42.081534+0000","last_became_active":"2024-01-15T18:00:42.078911+0000","last_became_peered":"2024-01-15T18:00:42.078911+0000","last_unstale":"2024-01-15T18:00:42.081534+0000","last_undegraded":"2024-01-15T18:00:42.081534+0000","last_fullsized":"2024-01-15T18:00:42.081534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:05:36.094656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.090879+0000","last_change":"2024-01-15T18:00:41.069799+0000","last_active":"2024-01-15T18:00:42.090879+0000","last_peered":"2024-01-15T18:00:42.090879+0000","last_clean":"2024-01-15T18:00:42.090879+0000","last_became_active":"2024-01-15T18:00:41.069467+0000","last_became_peered":"2024-01-15T18:00:41.069467+0000","last_unstale":"2024-01-15T18:00:42.090879+0000","last_undegraded":"2024-01-15T18:00:42.090879+0000","last_fullsized":"2024-01-15T18:00:42.090879+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.066827+0000","last_change":"2024-01-15T18:00:42.064990+0000","last_active":"2024-01-15T18:00:42.066827+0000","last_peered":"2024-01-15T18:00:42.066827+0000","last_clean":"2024-01-15T18:00:42.066827+0000","last_became_active":"2024-01-15T18:00:42.064698+0000","last_became_peered":"2024-01-15T18:00:42.064698+0000","last_unstale":"2024-01-15T18:00:42.066827+0000","last_undegraded":"2024-01-15T18:00:42.066827+0000","last_fullsized":"2024-01-15T18:00:42.066827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:23:58.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069728+0000","last_change":"2024-01-15T18:00:41.057228+0000","last_active":"2024-01-15T18:00:42.069728+0000","last_peered":"2024-01-15T18:00:42.069728+0000","last_clean":"2024-01-15T18:00:42.069728+0000","last_became_active":"2024-01-15T18:00:41.056929+0000","last_became_peered":"2024-01-15T18:00:41.056929+0000","last_unstale":"2024-01-15T18:00:42.069728+0000","last_undegraded":"2024-01-15T18:00:42.069728+0000","last_fullsized":"2024-01-15T18:00:42.069728+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.065721+0000","last_change":"2024-01-15T18:00:42.057749+0000","last_active":"2024-01-15T18:00:42.065721+0000","last_peered":"2024-01-15T18:00:42.065721+0000","last_clean":"2024-01-15T18:00:42.065721+0000","last_became_active":"2024-01-15T18:00:42.054149+0000","last_became_peered":"2024-01-15T18:00:42.054149+0000","last_unstale":"2024-01-15T18:00:42.065721+0000","last_undegraded":"2024-01-15T18:00:42.065721+0000","last_fullsized":"2024-01-15T18:00:42.065721+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:51.141215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.047331+0000","last_change":"2024-01-15T18:00:41.067706+0000","last_active":"2024-01-15T18:00:42.047331+0000","last_peered":"2024-01-15T18:00:42.047331+0000","last_clean":"2024-01-15T18:00:42.047331+0000","last_became_active":"2024-01-15T18:00:41.067325+0000","last_became_peered":"2024-01-15T18:00:41.067325+0000","last_unstale":"2024-01-15T18:00:42.047331+0000","last_undegraded":"2024-01-15T18:00:42.047331+0000","last_fullsized":"2024-01-15T18:00:42.047331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:50:53.105108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063135+0000","last_change":"2024-01-15T18:00:42.063135+0000","last_active":"2024-01-15T18:00:42.063135+0000","last_peered":"2024-01-15T18:00:42.063135+0000","last_clean":"2024-01-15T18:00:42.063135+0000","last_became_active":"2024-01-15T18:00:42.062863+0000","last_became_peered":"2024-01-15T18:00:42.062863+0000","last_unstale":"2024-01-15T18:00:42.063135+0000","last_undegraded":"2024-01-15T18:00:42.063135+0000","last_fullsized":"2024-01-15T18:00:42.063135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:35.983660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.052982+0000","last_change":"2024-01-15T18:00:41.073770+0000","last_active":"2024-01-15T18:00:42.052982+0000","last_peered":"2024-01-15T18:00:42.052982+0000","last_clean":"2024-01-15T18:00:42.052982+0000","last_became_active":"2024-01-15T18:00:41.073486+0000","last_became_peered":"2024-01-15T18:00:41.073486+0000","last_unstale":"2024-01-15T18:00:42.052982+0000","last_undegraded":"2024-01-15T18:00:42.052982+0000","last_fullsized":"2024-01-15T18:00:42.052982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069498+0000","last_change":"2024-01-15T18:00:42.064514+0000","last_active":"2024-01-15T18:00:42.069498+0000","last_peered":"2024-01-15T18:00:42.069498+0000","last_clean":"2024-01-15T18:00:42.069498+0000","last_became_active":"2024-01-15T18:00:42.064248+0000","last_became_peered":"2024-01-15T18:00:42.064248+0000","last_unstale":"2024-01-15T18:00:42.069498+0000","last_undegraded":"2024-01-15T18:00:42.069498+0000","last_fullsized":"2024-01-15T18:00:42.069498+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:00.533678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.036548+0000","last_change":"2024-01-15T18:00:41.064452+0000","last_active":"2024-01-15T18:00:42.036548+0000","last_peered":"2024-01-15T18:00:42.036548+0000","last_clean":"2024-01-15T18:00:42.036548+0000","last_became_active":"2024-01-15T18:00:41.064118+0000","last_became_peered":"2024-01-15T18:00:41.064118+0000","last_unstale":"2024-01-15T18:00:42.036548+0000","last_undegraded":"2024-01-15T18:00:42.036548+0000","last_fullsized":"2024-01-15T18:00:42.036548+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:31.276292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.068233+0000","last_change":"2024-01-15T18:00:42.068233+0000","last_active":"2024-01-15T18:00:42.068233+0000","last_peered":"2024-01-15T18:00:42.068233+0000","last_clean":"2024-01-15T18:00:42.068233+0000","last_became_active":"2024-01-15T18:00:42.066136+0000","last_became_peered":"2024-01-15T18:00:42.066136+0000","last_unstale":"2024-01-15T18:00:42.068233+0000","last_undegraded":"2024-01-15T18:00:42.068233+0000","last_fullsized":"2024-01-15T18:00:42.068233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:03:53.619388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085512+0000","last_change":"2024-01-15T18:00:41.069415+0000","last_active":"2024-01-15T18:00:42.085512+0000","last_peered":"2024-01-15T18:00:42.085512+0000","last_clean":"2024-01-15T18:00:42.085512+0000","last_became_active":"2024-01-15T18:00:41.069194+0000","last_became_peered":"2024-01-15T18:00:41.069194+0000","last_unstale":"2024-01-15T18:00:42.085512+0000","last_undegraded":"2024-01-15T18:00:42.085512+0000","last_fullsized":"2024-01-15T18:00:42.085512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:08.061519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076472+0000","last_change":"2024-01-15T18:00:42.072986+0000","last_active":"2024-01-15T18:00:42.076472+0000","last_peered":"2024-01-15T18:00:42.076472+0000","last_clean":"2024-01-15T18:00:42.076472+0000","last_became_active":"2024-01-15T18:00:42.072700+0000","last_became_peered":"2024-01-15T18:00:42.072700+0000","last_unstale":"2024-01-15T18:00:42.076472+0000","last_undegraded":"2024-01-15T18:00:42.076472+0000","last_fullsized":"2024-01-15T18:00:42.076472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:48.146540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.044213+0000","last_change":"2024-01-15T18:00:41.056524+0000","last_active":"2024-01-15T18:00:42.044213+0000","last_peered":"2024-01-15T18:00:42.044213+0000","last_clean":"2024-01-15T18:00:42.044213+0000","last_became_active":"2024-01-15T18:00:41.056086+0000","last_became_peered":"2024-01-15T18:00:41.056086+0000","last_unstale":"2024-01-15T18:00:42.044213+0000","last_undegraded":"2024-01-15T18:00:42.044213+0000","last_fullsized":"2024-01-15T18:00:42.044213+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:36.552797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.075686+0000","last_change":"2024-01-15T18:00:42.072006+0000","last_active":"2024-01-15T18:00:42.075686+0000","last_peered":"2024-01-15T18:00:42.075686+0000","last_clean":"2024-01-15T18:00:42.075686+0000","last_became_active":"2024-01-15T18:00:42.071671+0000","last_became_peered":"2024-01-15T18:00:42.071671+0000","last_unstale":"2024-01-15T18:00:42.075686+0000","last_undegraded":"2024-01-15T18:00:42.075686+0000","last_fullsized":"2024-01-15T18:00:42.075686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085054+0000","last_change":"2024-01-15T18:00:41.067157+0000","last_active":"2024-01-15T18:00:42.085054+0000","last_peered":"2024-01-15T18:00:42.085054+0000","last_clean":"2024-01-15T18:00:42.085054+0000","last_became_active":"2024-01-15T18:00:41.066820+0000","last_became_peered":"2024-01-15T18:00:41.066820+0000","last_unstale":"2024-01-15T18:00:42.085054+0000","last_undegraded":"2024-01-15T18:00:42.085054+0000","last_fullsized":"2024-01-15T18:00:42.085054+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:03:48.949979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.068749+0000","last_change":"2024-01-15T18:00:42.063066+0000","last_active":"2024-01-15T18:00:42.068749+0000","last_peered":"2024-01-15T18:00:42.068749+0000","last_clean":"2024-01-15T18:00:42.068749+0000","last_became_active":"2024-01-15T18:00:42.062649+0000","last_became_peered":"2024-01-15T18:00:42.062649+0000","last_unstale":"2024-01-15T18:00:42.068749+0000","last_undegraded":"2024-01-15T18:00:42.068749+0000","last_fullsized":"2024-01-15T18:00:42.068749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:52.055388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.040717+0000","last_change":"2024-01-15T18:00:41.061164+0000","last_active":"2024-01-15T18:00:42.040717+0000","last_peered":"2024-01-15T18:00:42.040717+0000","last_clean":"2024-01-15T18:00:42.040717+0000","last_became_active":"2024-01-15T18:00:41.060847+0000","last_became_peered":"2024-01-15T18:00:41.060847+0000","last_unstale":"2024-01-15T18:00:42.040717+0000","last_undegraded":"2024-01-15T18:00:42.040717+0000","last_fullsized":"2024-01-15T18:00:42.040717+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":3,"reported_epoch":59,"state":"creating+activating","last_fresh":"2024-01-15T18:00:42.046824+0000","last_change":"2024-01-15T18:00:42.046824+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:42.046824+0000","last_undegraded":"2024-01-15T18:00:42.046824+0000","last_fullsized":"2024-01-15T18:00:42.046824+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:22:14.891183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069884+0000","last_change":"2024-01-15T18:00:41.060084+0000","last_active":"2024-01-15T18:00:42.069884+0000","last_peered":"2024-01-15T18:00:42.069884+0000","last_clean":"2024-01-15T18:00:42.069884+0000","last_became_active":"2024-01-15T18:00:41.057544+0000","last_became_peered":"2024-01-15T18:00:41.057544+0000","last_unstale":"2024-01-15T18:00:42.069884+0000","last_undegraded":"2024-01-15T18:00:42.069884+0000","last_fullsized":"2024-01-15T18:00:42.069884+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076361+0000","last_change":"2024-01-15T18:00:42.071201+0000","last_active":"2024-01-15T18:00:42.076361+0000","last_peered":"2024-01-15T18:00:42.076361+0000","last_clean":"2024-01-15T18:00:42.076361+0000","last_became_active":"2024-01-15T18:00:42.070498+0000","last_became_peered":"2024-01-15T18:00:42.070498+0000","last_unstale":"2024-01-15T18:00:42.076361+0000","last_undegraded":"2024-01-15T18:00:42.076361+0000","last_fullsized":"2024-01-15T18:00:42.076361+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:27.820063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.087649+0000","last_change":"2024-01-15T18:00:41.064043+0000","last_active":"2024-01-15T18:00:42.087649+0000","last_peered":"2024-01-15T18:00:42.087649+0000","last_clean":"2024-01-15T18:00:42.087649+0000","last_became_active":"2024-01-15T18:00:41.063128+0000","last_became_peered":"2024-01-15T18:00:41.063128+0000","last_unstale":"2024-01-15T18:00:42.087649+0000","last_undegraded":"2024-01-15T18:00:42.087649+0000","last_fullsized":"2024-01-15T18:00:42.087649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:12.496547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083296+0000","last_change":"2024-01-15T18:00:42.079795+0000","last_active":"2024-01-15T18:00:42.083296+0000","last_peered":"2024-01-15T18:00:42.083296+0000","last_clean":"2024-01-15T18:00:42.083296+0000","last_became_active":"2024-01-15T18:00:42.079150+0000","last_became_peered":"2024-01-15T18:00:42.079150+0000","last_unstale":"2024-01-15T18:00:42.083296+0000","last_undegraded":"2024-01-15T18:00:42.083296+0000","last_fullsized":"2024-01-15T18:00:42.083296+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:41.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.084348+0000","last_change":"2024-01-15T18:00:41.071443+0000","last_active":"2024-01-15T18:00:42.084348+0000","last_peered":"2024-01-15T18:00:42.084348+0000","last_clean":"2024-01-15T18:00:42.084348+0000","last_became_active":"2024-01-15T18:00:41.071219+0000","last_became_peered":"2024-01-15T18:00:41.071219+0000","last_unstale":"2024-01-15T18:00:42.084348+0000","last_undegraded":"2024-01-15T18:00:42.084348+0000","last_fullsized":"2024-01-15T18:00:42.084348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:56.788064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081207+0000","last_change":"2024-01-15T18:00:42.079222+0000","last_active":"2024-01-15T18:00:42.081207+0000","last_peered":"2024-01-15T18:00:42.081207+0000","last_clean":"2024-01-15T18:00:42.081207+0000","last_became_active":"2024-01-15T18:00:42.078933+0000","last_became_peered":"2024-01-15T18:00:42.078933+0000","last_unstale":"2024-01-15T18:00:42.081207+0000","last_undegraded":"2024-01-15T18:00:42.081207+0000","last_fullsized":"2024-01-15T18:00:42.081207+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:44.207537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.091309+0000","last_change":"2024-01-15T18:00:41.071043+0000","last_active":"2024-01-15T18:00:42.091309+0000","last_peered":"2024-01-15T18:00:42.091309+0000","last_clean":"2024-01-15T18:00:42.091309+0000","last_became_active":"2024-01-15T18:00:41.070768+0000","last_became_peered":"2024-01-15T18:00:41.070768+0000","last_unstale":"2024-01-15T18:00:42.091309+0000","last_undegraded":"2024-01-15T18:00:42.091309+0000","last_fullsized":"2024-01-15T18:00:42.091309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080200+0000","last_change":"2024-01-15T18:00:42.076070+0000","last_active":"2024-01-15T18:00:42.080200+0000","last_peered":"2024-01-15T18:00:42.080200+0000","last_clean":"2024-01-15T18:00:42.080200+0000","last_became_active":"2024-01-15T18:00:42.068998+0000","last_became_peered":"2024-01-15T18:00:42.068998+0000","last_unstale":"2024-01-15T18:00:42.080200+0000","last_undegraded":"2024-01-15T18:00:42.080200+0000","last_fullsized":"2024-01-15T18:00:42.080200+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:08:12.230431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.072591+0000","last_change":"2024-01-15T18:00:41.061292+0000","last_active":"2024-01-15T18:00:42.072591+0000","last_peered":"2024-01-15T18:00:42.072591+0000","last_clean":"2024-01-15T18:00:42.072591+0000","last_became_active":"2024-01-15T18:00:41.061006+0000","last_became_peered":"2024-01-15T18:00:41.061006+0000","last_unstale":"2024-01-15T18:00:42.072591+0000","last_undegraded":"2024-01-15T18:00:42.072591+0000","last_fullsized":"2024-01-15T18:00:42.072591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.073395+0000","last_change":"2024-01-15T18:00:42.067416+0000","last_active":"2024-01-15T18:00:42.073395+0000","last_peered":"2024-01-15T18:00:42.073395+0000","last_clean":"2024-01-15T18:00:42.073395+0000","last_became_active":"2024-01-15T18:00:42.067129+0000","last_became_peered":"2024-01-15T18:00:42.067129+0000","last_unstale":"2024-01-15T18:00:42.073395+0000","last_undegraded":"2024-01-15T18:00:42.073395+0000","last_fullsized":"2024-01-15T18:00:42.073395+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:49.570465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070835+0000","last_change":"2024-01-15T18:00:41.071456+0000","last_active":"2024-01-15T18:00:42.070835+0000","last_peered":"2024-01-15T18:00:42.070835+0000","last_clean":"2024-01-15T18:00:42.070835+0000","last_became_active":"2024-01-15T18:00:41.071189+0000","last_became_peered":"2024-01-15T18:00:41.071189+0000","last_unstale":"2024-01-15T18:00:42.070835+0000","last_undegraded":"2024-01-15T18:00:42.070835+0000","last_fullsized":"2024-01-15T18:00:42.070835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070175+0000","last_change":"2024-01-15T18:00:42.065297+0000","last_active":"2024-01-15T18:00:42.070175+0000","last_peered":"2024-01-15T18:00:42.070175+0000","last_clean":"2024-01-15T18:00:42.070175+0000","last_became_active":"2024-01-15T18:00:42.064969+0000","last_became_peered":"2024-01-15T18:00:42.064969+0000","last_unstale":"2024-01-15T18:00:42.070175+0000","last_undegraded":"2024-01-15T18:00:42.070175+0000","last_fullsized":"2024-01-15T18:00:42.070175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.084074+0000","last_change":"2024-01-15T18:00:41.068101+0000","last_active":"2024-01-15T18:00:42.084074+0000","last_peered":"2024-01-15T18:00:42.084074+0000","last_clean":"2024-01-15T18:00:42.084074+0000","last_became_active":"2024-01-15T18:00:41.065928+0000","last_became_peered":"2024-01-15T18:00:41.065928+0000","last_unstale":"2024-01-15T18:00:42.084074+0000","last_undegraded":"2024-01-15T18:00:42.084074+0000","last_fullsized":"2024-01-15T18:00:42.084074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:19:23.525421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.075413+0000","last_change":"2024-01-15T18:00:42.067592+0000","last_active":"2024-01-15T18:00:42.075413+0000","last_peered":"2024-01-15T18:00:42.075413+0000","last_clean":"2024-01-15T18:00:42.075413+0000","last_became_active":"2024-01-15T18:00:42.067309+0000","last_became_peered":"2024-01-15T18:00:42.067309+0000","last_unstale":"2024-01-15T18:00:42.075413+0000","last_undegraded":"2024-01-15T18:00:42.075413+0000","last_fullsized":"2024-01-15T18:00:42.075413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.046741+0000","last_change":"2024-01-15T18:00:41.050813+0000","last_active":"2024-01-15T18:00:42.046741+0000","last_peered":"2024-01-15T18:00:42.046741+0000","last_clean":"2024-01-15T18:00:42.046741+0000","last_became_active":"2024-01-15T18:00:41.050320+0000","last_became_peered":"2024-01-15T18:00:41.050320+0000","last_unstale":"2024-01-15T18:00:42.046741+0000","last_undegraded":"2024-01-15T18:00:42.046741+0000","last_fullsized":"2024-01-15T18:00:42.046741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.075305+0000","last_change":"2024-01-15T18:00:42.070090+0000","last_active":"2024-01-15T18:00:42.075305+0000","last_peered":"2024-01-15T18:00:42.075305+0000","last_clean":"2024-01-15T18:00:42.075305+0000","last_became_active":"2024-01-15T18:00:42.068897+0000","last_became_peered":"2024-01-15T18:00:42.068897+0000","last_unstale":"2024-01-15T18:00:42.075305+0000","last_undegraded":"2024-01-15T18:00:42.075305+0000","last_fullsized":"2024-01-15T18:00:42.075305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070801+0000","last_change":"2024-01-15T18:00:41.065251+0000","last_active":"2024-01-15T18:00:42.070801+0000","last_peered":"2024-01-15T18:00:42.070801+0000","last_clean":"2024-01-15T18:00:42.070801+0000","last_became_active":"2024-01-15T18:00:41.065006+0000","last_became_peered":"2024-01-15T18:00:41.065006+0000","last_unstale":"2024-01-15T18:00:42.070801+0000","last_undegraded":"2024-01-15T18:00:42.070801+0000","last_fullsized":"2024-01-15T18:00:42.070801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083131+0000","last_change":"2024-01-15T18:00:42.083131+0000","last_active":"2024-01-15T18:00:42.083131+0000","last_peered":"2024-01-15T18:00:42.083131+0000","last_clean":"2024-01-15T18:00:42.083131+0000","last_became_active":"2024-01-15T18:00:42.082546+0000","last_became_peered":"2024-01-15T18:00:42.082546+0000","last_unstale":"2024-01-15T18:00:42.083131+0000","last_undegraded":"2024-01-15T18:00:42.083131+0000","last_fullsized":"2024-01-15T18:00:42.083131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:24.550976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.046483+0000","last_change":"2024-01-15T18:00:41.066193+0000","last_active":"2024-01-15T18:00:42.046483+0000","last_peered":"2024-01-15T18:00:42.046483+0000","last_clean":"2024-01-15T18:00:42.046483+0000","last_became_active":"2024-01-15T18:00:41.065913+0000","last_became_peered":"2024-01-15T18:00:41.065913+0000","last_unstale":"2024-01-15T18:00:42.046483+0000","last_undegraded":"2024-01-15T18:00:42.046483+0000","last_fullsized":"2024-01-15T18:00:42.046483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.079328+0000","last_change":"2024-01-15T18:00:42.076020+0000","last_active":"2024-01-15T18:00:42.079328+0000","last_peered":"2024-01-15T18:00:42.079328+0000","last_clean":"2024-01-15T18:00:42.079328+0000","last_became_active":"2024-01-15T18:00:42.075751+0000","last_became_peered":"2024-01-15T18:00:42.075751+0000","last_unstale":"2024-01-15T18:00:42.079328+0000","last_undegraded":"2024-01-15T18:00:42.079328+0000","last_fullsized":"2024-01-15T18:00:42.079328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:52.055388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076414+0000","last_change":"2024-01-15T18:00:41.070458+0000","last_active":"2024-01-15T18:00:42.076414+0000","last_peered":"2024-01-15T18:00:42.076414+0000","last_clean":"2024-01-15T18:00:42.076414+0000","last_became_active":"2024-01-15T18:00:41.066322+0000","last_became_peered":"2024-01-15T18:00:41.066322+0000","last_unstale":"2024-01-15T18:00:42.076414+0000","last_undegraded":"2024-01-15T18:00:42.076414+0000","last_fullsized":"2024-01-15T18:00:42.076414+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.066727+0000","last_change":"2024-01-15T18:00:42.066727+0000","last_active":"2024-01-15T18:00:42.066727+0000","last_peered":"2024-01-15T18:00:42.066727+0000","last_clean":"2024-01-15T18:00:42.066727+0000","last_became_active":"2024-01-15T18:00:42.065788+0000","last_became_peered":"2024-01-15T18:00:42.065788+0000","last_unstale":"2024-01-15T18:00:42.066727+0000","last_undegraded":"2024-01-15T18:00:42.066727+0000","last_fullsized":"2024-01-15T18:00:42.066727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:01:13.061488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.040831+0000","last_change":"2024-01-15T18:00:41.063919+0000","last_active":"2024-01-15T18:00:42.040831+0000","last_peered":"2024-01-15T18:00:42.040831+0000","last_clean":"2024-01-15T18:00:42.040831+0000","last_became_active":"2024-01-15T18:00:41.062115+0000","last_became_peered":"2024-01-15T18:00:41.062115+0000","last_unstale":"2024-01-15T18:00:42.040831+0000","last_undegraded":"2024-01-15T18:00:42.040831+0000","last_fullsized":"2024-01-15T18:00:42.040831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080268+0000","last_change":"2024-01-15T18:00:42.073589+0000","last_active":"2024-01-15T18:00:42.080268+0000","last_peered":"2024-01-15T18:00:42.080268+0000","last_clean":"2024-01-15T18:00:42.080268+0000","last_became_active":"2024-01-15T18:00:42.073254+0000","last_became_peered":"2024-01-15T18:00:42.073254+0000","last_unstale":"2024-01-15T18:00:42.080268+0000","last_undegraded":"2024-01-15T18:00:42.080268+0000","last_fullsized":"2024-01-15T18:00:42.080268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:29.053165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070418+0000","last_change":"2024-01-15T18:00:41.057399+0000","last_active":"2024-01-15T18:00:42.070418+0000","last_peered":"2024-01-15T18:00:42.070418+0000","last_clean":"2024-01-15T18:00:42.070418+0000","last_became_active":"2024-01-15T18:00:41.056174+0000","last_became_peered":"2024-01-15T18:00:41.056174+0000","last_unstale":"2024-01-15T18:00:42.070418+0000","last_undegraded":"2024-01-15T18:00:42.070418+0000","last_fullsized":"2024-01-15T18:00:42.070418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.061539+0000","last_change":"2024-01-15T18:00:42.052005+0000","last_active":"2024-01-15T18:00:42.061539+0000","last_peered":"2024-01-15T18:00:42.061539+0000","last_clean":"2024-01-15T18:00:42.061539+0000","last_became_active":"2024-01-15T18:00:42.051122+0000","last_became_peered":"2024-01-15T18:00:42.051122+0000","last_unstale":"2024-01-15T18:00:42.061539+0000","last_undegraded":"2024-01-15T18:00:42.061539+0000","last_fullsized":"2024-01-15T18:00:42.061539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:59:19.158591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.046278+0000","last_change":"2024-01-15T18:00:41.064076+0000","last_active":"2024-01-15T18:00:42.046278+0000","last_peered":"2024-01-15T18:00:42.046278+0000","last_clean":"2024-01-15T18:00:42.046278+0000","last_became_active":"2024-01-15T18:00:41.062980+0000","last_became_peered":"2024-01-15T18:00:41.062980+0000","last_unstale":"2024-01-15T18:00:42.046278+0000","last_undegraded":"2024-01-15T18:00:42.046278+0000","last_fullsized":"2024-01-15T18:00:42.046278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:41:42.728224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.078756+0000","last_change":"2024-01-15T18:00:42.070329+0000","last_active":"2024-01-15T18:00:42.078756+0000","last_peered":"2024-01-15T18:00:42.078756+0000","last_clean":"2024-01-15T18:00:42.078756+0000","last_became_active":"2024-01-15T18:00:42.068798+0000","last_became_peered":"2024-01-15T18:00:42.068798+0000","last_unstale":"2024-01-15T18:00:42.078756+0000","last_undegraded":"2024-01-15T18:00:42.078756+0000","last_fullsized":"2024-01-15T18:00:42.078756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:44.207537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.043282+0000","last_change":"2024-01-15T18:00:41.054271+0000","last_active":"2024-01-15T18:00:42.043282+0000","last_peered":"2024-01-15T18:00:42.043282+0000","last_clean":"2024-01-15T18:00:42.043282+0000","last_became_active":"2024-01-15T18:00:41.053925+0000","last_became_peered":"2024-01-15T18:00:41.053925+0000","last_unstale":"2024-01-15T18:00:42.043282+0000","last_undegraded":"2024-01-15T18:00:42.043282+0000","last_fullsized":"2024-01-15T18:00:42.043282+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:18:46.672266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064312+0000","last_change":"2024-01-15T18:00:42.057721+0000","last_active":"2024-01-15T18:00:42.064312+0000","last_peered":"2024-01-15T18:00:42.064312+0000","last_clean":"2024-01-15T18:00:42.064312+0000","last_became_active":"2024-01-15T18:00:42.057091+0000","last_became_peered":"2024-01-15T18:00:42.057091+0000","last_unstale":"2024-01-15T18:00:42.064312+0000","last_undegraded":"2024-01-15T18:00:42.064312+0000","last_fullsized":"2024-01-15T18:00:42.064312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:33:19.250071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.045742+0000","last_change":"2024-01-15T18:00:41.061944+0000","last_active":"2024-01-15T18:00:42.045742+0000","last_peered":"2024-01-15T18:00:42.045742+0000","last_clean":"2024-01-15T18:00:42.045742+0000","last_became_active":"2024-01-15T18:00:41.061646+0000","last_became_peered":"2024-01-15T18:00:41.061646+0000","last_unstale":"2024-01-15T18:00:42.045742+0000","last_undegraded":"2024-01-15T18:00:42.045742+0000","last_fullsized":"2024-01-15T18:00:42.045742+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:23:57.016225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.065689+0000","last_change":"2024-01-15T18:00:42.055286+0000","last_active":"2024-01-15T18:00:42.065689+0000","last_peered":"2024-01-15T18:00:42.065689+0000","last_clean":"2024-01-15T18:00:42.065689+0000","last_became_active":"2024-01-15T18:00:42.054746+0000","last_became_peered":"2024-01-15T18:00:42.054746+0000","last_unstale":"2024-01-15T18:00:42.065689+0000","last_undegraded":"2024-01-15T18:00:42.065689+0000","last_fullsized":"2024-01-15T18:00:42.065689+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:14.526164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.036713+0000","last_change":"2024-01-15T18:00:41.068287+0000","last_active":"2024-01-15T18:00:42.036713+0000","last_peered":"2024-01-15T18:00:42.036713+0000","last_clean":"2024-01-15T18:00:42.036713+0000","last_became_active":"2024-01-15T18:00:41.066120+0000","last_became_peered":"2024-01-15T18:00:41.066120+0000","last_unstale":"2024-01-15T18:00:42.036713+0000","last_undegraded":"2024-01-15T18:00:42.036713+0000","last_fullsized":"2024-01-15T18:00:42.036713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:39:48.165240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083708+0000","last_change":"2024-01-15T18:00:42.080052+0000","last_active":"2024-01-15T18:00:42.083708+0000","last_peered":"2024-01-15T18:00:42.083708+0000","last_clean":"2024-01-15T18:00:42.083708+0000","last_became_active":"2024-01-15T18:00:42.079699+0000","last_became_peered":"2024-01-15T18:00:42.079699+0000","last_unstale":"2024-01-15T18:00:42.083708+0000","last_undegraded":"2024-01-15T18:00:42.083708+0000","last_fullsized":"2024-01-15T18:00:42.083708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:27.598423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.040946+0000","last_change":"2024-01-15T18:00:41.061168+0000","last_active":"2024-01-15T18:00:42.040946+0000","last_peered":"2024-01-15T18:00:42.040946+0000","last_clean":"2024-01-15T18:00:42.040946+0000","last_became_active":"2024-01-15T18:00:41.060793+0000","last_became_peered":"2024-01-15T18:00:41.060793+0000","last_unstale":"2024-01-15T18:00:42.040946+0000","last_undegraded":"2024-01-15T18:00:42.040946+0000","last_fullsized":"2024-01-15T18:00:42.040946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:02.304705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":195,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-01-15T18:00:41.055451+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:00:41.055451+0000","last_peered":"2024-01-15T18:00:41.055451+0000","last_clean":"2024-01-15T18:00:41.055451+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:00:41.055451+0000","last_undegraded":"2024-01-15T18:00:41.055451+0000","last_fullsized":"2024-01-15T18:00:41.055451+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36600,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335240,"statfs":{"total":96636764160,"available":96599285760,"internally_reserved":0,"allocated":1286144,"data_stored":877555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.2230000000000001,"15min":1.177},"min":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":5.556,"5min":5.556,"15min":5.556},"last":1.179},{"interface":"front","average":{"1min":3.444,"5min":1.649,"15min":1.3260000000000001},"min":{"1min":0.747,"5min":0.623,"15min":0.623},"max":{"1min":38.137,"5min":38.137,"15min":38.137},"last":1.3420000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.266,"15min":1.25},"min":{"1min":0.72999999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":4.194,"5min":1.861,"15min":1.4570000000000001},"min":{"1min":0.72699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":51.843000000000004,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.0009999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.921,"5min":1.361,"15min":1.2270000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":9.3369999999999997,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":1.288},{"interface":"front","average":{"1min":1.643,"5min":1.397,"15min":1.393},"min":{"1min":0.66000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.2889999999999997,"5min":6.2889999999999997,"15min":6.2889999999999997},"last":1.3959999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":1.49,"15min":1.363},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":9.1440000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5029999999999999,"15min":1.4970000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":1.534}]},{"osd":4,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":3.3300000000000001,"5min":1.788,"15min":1.5980000000000001},"min":{"1min":0.95799999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":22.359999999999999,"5min":22.359999999999999,"15min":22.359999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":2.2130000000000001,"5min":1.597,"15min":1.496},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":12.667999999999999,"5min":12.667999999999999,"15min":12.667999999999999},"last":1.599}]},{"osd":5,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.0220000000000002,"5min":2.0569999999999999,"15min":1.7569999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":25.006,"5min":25.006,"15min":25.006},"last":1.649},{"interface":"front","average":{"1min":3.4950000000000001,"5min":1.9239999999999999,"15min":1.712},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":22.693999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.2330000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.9530000000000003,"5min":2.2770000000000001,"15min":1.901},"min":{"1min":1.0580000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":23.834,"5min":23.834,"15min":23.834},"last":1.4359999999999999},{"interface":"front","average":{"1min":5.9169999999999998,"5min":2.4729999999999999,"15min":1.9199999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":63.932000000000002,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.7010000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542215,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.413,"15min":1.3169999999999999},"min":{"1min":0.71699999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.8769999999999998,"5min":12.544,"15min":12.544},"last":1.345},{"interface":"front","average":{"1min":2.1840000000000002,"5min":1.423,"15min":1.248},"min":{"1min":0.81899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":13.98,"5min":13.98,"15min":13.98},"last":1.0880000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.357,"15min":1.2250000000000001},"min":{"1min":0.82599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.5659999999999998,"5min":12.811999999999999,"15min":12.811999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":2.0600000000000001,"5min":1.349,"15min":1.2330000000000001},"min":{"1min":0.86499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":8.8130000000000006,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.1399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2970000000000002,"5min":1.456,"15min":1.28},"min":{"1min":0.90300000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4890000000000001,"15min":1.5009999999999999},"min":{"1min":0.86899999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":4.0419999999999998,"5min":12.897,"15min":12.897},"last":1.466}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.988,"5min":1.7010000000000001,"15min":1.542},"min":{"1min":1.0880000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":9.5,"5min":12.717000000000001,"15min":12.717000000000001},"last":1.5629999999999999},{"interface":"front","average":{"1min":2.52,"5min":1.5840000000000001,"15min":1.4410000000000001},"min":{"1min":0.93799999999999994,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":14.066000000000001,"5min":14.066000000000001,"15min":14.066000000000001},"last":1.835}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5009999999999999,"15min":1.454},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":9.2829999999999995,"5min":9.2829999999999995,"15min":9.2829999999999995},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.6359999999999999,"15min":1.607},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.6989999999999998,"5min":16.98,"15min":16.98},"last":1.6539999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":1.548,"15min":1.484},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":14.176,"5min":14.176,"15min":14.176},"last":1.6919999999999999},{"interface":"front","average":{"1min":2.3079999999999998,"5min":1.756,"15min":1.7569999999999999},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"last":1.3879999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":1.6399999999999999,"15min":1.601},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":14.109999999999999,"5min":14.109999999999999,"15min":14.109999999999999},"last":1.966},{"interface":"front","average":{"1min":2.5870000000000002,"5min":1.8009999999999999,"15min":1.7769999999999999},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":14.478,"5min":14.478,"15min":14.478},"last":1.7529999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3149999999999999,"15min":1.2509999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.7490000000000001,"5min":16.178000000000001,"15min":16.178000000000001},"last":1.046},{"interface":"front","average":{"1min":2.4980000000000002,"5min":1.4750000000000001,"15min":1.417},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":17.184999999999999,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.5349999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2929999999999999,"15min":1.2250000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":5.6219999999999999,"5min":5.6219999999999999,"15min":5.6219999999999999},"last":1.274},{"interface":"front","average":{"1min":1.423,"5min":1.2629999999999999,"15min":1.2110000000000001},"min":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.1379999999999999,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":1.1399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2050000000000001,"15min":1.2190000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.681,"5min":2.835,"15min":2.835},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.367,"15min":1.288},"min":{"1min":1.0760000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.9740000000000002,"5min":5.9740000000000002,"15min":5.9740000000000002},"last":1.327}]},{"osd":3,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.3520000000000001},"min":{"1min":0.73099999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8029999999999999,"5min":14.914,"15min":14.914},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.365,"15min":1.413},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.726,"5min":3.302,"15min":3.302},"last":1.2929999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.395,"15min":1.411},"min":{"1min":0.99299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.7759999999999998,"5min":4.7759999999999998,"15min":4.7759999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.75,"15min":1.571},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":17.991,"5min":17.991,"15min":17.991},"last":0.92100000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.506,"15min":1.6020000000000001},"min":{"1min":0.91900000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":6.4630000000000001,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.792,"5min":1.5369999999999999,"15min":1.55},"min":{"1min":0.88100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":7.7759999999999998,"5min":7.7759999999999998,"15min":7.7759999999999998},"last":1.4159999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5580000000000001,"15min":1.6060000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":6.3230000000000004,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.603,"15min":1.728},"min":{"1min":0.90500000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.2030000000000003,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.109}]}]},{"osd":4,"up_from":14,"seq":60129542215,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.9989999999999997,"5min":11.413,"15min":11.413},"last":1.492},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.4350000000000001,"15min":1.3799999999999999},"min":{"1min":0.77500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":9.7439999999999998,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":1.367}]},{"osd":1,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.4670000000000001,"15min":1.3819999999999999},"min":{"1min":0.84299999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":9.7010000000000005,"5min":11.987,"15min":11.987},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.27,"15min":1.323},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":6.149,"5min":6.149,"15min":6.149},"last":1.2210000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.708,"15min":1.5700000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":7.032,"5min":13.865,"15min":13.865},"last":1.343},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.514,"15min":1.3959999999999999},"min":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.028,"5min":13.903,"15min":13.903},"last":1.4139999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.6040000000000001,"15min":1.5920000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":6.8819999999999997,"5min":11.611000000000001,"15min":11.611000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":2.0800000000000001,"5min":1.546,"15min":1.6180000000000001},"min":{"1min":0.69199999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":13.25,"5min":13.25,"15min":13.25},"last":1.5589999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.6359999999999999,"15min":1.6850000000000001},"min":{"1min":0.61899999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":6.6820000000000004,"5min":11.561,"15min":11.561},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.8999999999999999,"5min":1.5509999999999999,"15min":1.665},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":9.6229999999999993,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.28}]},{"osd":6,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.468,"15min":1.6259999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.698,"5min":1.8500000000000001,"15min":1.8460000000000001},"min":{"1min":0.85799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":6.9329999999999998,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.6439999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":2.359,"5min":1.8979999999999999,"15min":1.9059999999999999},"min":{"1min":1.375,"5min":0.503,"15min":0.503},"max":{"1min":10.157999999999999,"5min":11.894,"15min":11.894},"last":1.456},{"interface":"front","average":{"1min":2.5169999999999999,"5min":2.048,"15min":1.968},"min":{"1min":1.325,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":8.4169999999999998,"5min":11.714,"15min":11.714},"last":1.702}]}]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44340,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327500,"statfs":{"total":96636764160,"available":96591360000,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":1.1990000000000001,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.435,"15min":0.435},"max":{"1min":9.9789999999999992,"5min":9.9789999999999992,"15min":9.9789999999999992},"last":0.76800000000000002},{"interface":"front","average":{"1min":2.9420000000000002,"5min":1.5149999999999999,"15min":1.218},"min":{"1min":0.85199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":21.047000000000001,"5min":21.047000000000001,"15min":21.047000000000001},"last":1.242}]},{"osd":1,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.8919999999999999,"5min":1.524,"15min":1.179},"min":{"1min":0.79700000000000004,"5min":0.375,"15min":0.375},"max":{"1min":17.777999999999999,"5min":17.777999999999999,"15min":17.777999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.786,"5min":1.2210000000000001,"15min":1.135},"min":{"1min":0.69299999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":9.2360000000000007,"5min":9.2360000000000007,"15min":9.2360000000000007},"last":0.94999999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.27,"15min":1.1899999999999999},"min":{"1min":0.79000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.4930000000000003,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.331},{"interface":"front","average":{"1min":2.0510000000000002,"5min":1.492,"15min":1.4119999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":9.3040000000000003,"5min":9.3040000000000003,"15min":9.3040000000000003},"last":1.2889999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":1.51,"15min":1.4199999999999999},"min":{"1min":1.1120000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":1.1299999999999999},{"interface":"front","average":{"1min":2.367,"5min":1.643,"15min":1.4890000000000001},"min":{"1min":0.92300000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":10.313000000000001,"5min":10.313000000000001,"15min":10.313000000000001},"last":1.077}]},{"osd":5,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.2229999999999999,"5min":1.635,"15min":1.5649999999999999},"min":{"1min":1.046,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.243,"5min":10.243,"15min":10.243},"last":1.5},{"interface":"front","average":{"1min":2.2330000000000001,"5min":1.742,"15min":1.6599999999999999},"min":{"1min":0.90500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":6.2000000000000002,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.544}]},{"osd":6,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.5510000000000002,"5min":1.851,"15min":1.7250000000000001},"min":{"1min":0.98399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":10.012,"5min":10.012,"15min":10.012},"last":1.6120000000000001},{"interface":"front","average":{"1min":2.6190000000000002,"5min":1.8220000000000001,"15min":1.722},"min":{"1min":1.0840000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":10.291,"5min":10.291,"15min":10.291},"last":1.5760000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.802,"5min":1.8819999999999999,"15min":1.7589999999999999},"min":{"1min":1.02,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":12.627000000000001,"5min":12.627000000000001,"15min":12.627000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":2.9710000000000001,"5min":1.946,"15min":1.7929999999999999},"min":{"1min":1.1240000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":10.268000000000001,"5min":10.268000000000001,"15min":10.268000000000001},"last":1.2150000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542217,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40180,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331660,"statfs":{"total":96636764160,"available":96595619840,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.042},"min":{"1min":0.48999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.466,"5min":14.427,"15min":14.427},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.0860000000000001,"15min":1.02},"min":{"1min":0.64100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.7290000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3129999999999999,"15min":1.2949999999999999},"min":{"1min":0.53000000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7490000000000001,"5min":14.609,"15min":14.609},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.29,"5min":1.1830000000000001,"15min":1.2010000000000001},"min":{"1min":0.748,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5059999999999998,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.2789999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2609999999999999,"15min":1.1990000000000001},"min":{"1min":0.64700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.772,"5min":16.567,"15min":16.567},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.48,"15min":1.4630000000000001},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8180000000000001,"5min":14.528,"15min":14.528},"last":1.383}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.708,"15min":1.647},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.7690000000000001,"5min":14.779,"15min":14.779},"last":1.4750000000000001},{"interface":"front","average":{"1min":2.073,"5min":1.546,"15min":1.502},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":10.75,"5min":10.75,"15min":10.75},"last":1.5089999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.5609999999999999,"15min":1.506},"min":{"1min":0.85199999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.258,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.653,"5min":1.726,"15min":1.659},"min":{"1min":1,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.4750000000000001,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.5600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.889,"5min":1.5580000000000001,"15min":1.5149999999999999},"min":{"1min":1.131,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.6339999999999999,"15min":1.5},"min":{"1min":1.171,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.415,"5min":17.07,"15min":17.07},"last":1.2390000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.839,"5min":1.621,"15min":1.6259999999999999},"min":{"1min":1.333,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.2170000000000001,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.6040000000000001},{"interface":"front","average":{"1min":2.9049999999999998,"5min":2.1070000000000002,"15min":1.7070000000000001},"min":{"1min":1.2070000000000001,"5min":0.752,"15min":0.752},"max":{"1min":15.135,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.4419999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542216,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40176,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331664,"statfs":{"total":96636764160,"available":96595623936,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.095,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":8.1140000000000008,"5min":8.1140000000000008,"15min":8.1140000000000008},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0169999999999999,"15min":0.99299999999999999},"min":{"1min":0.64000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.456,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.73799999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.4289999999999998,"5min":1.3109999999999999,"15min":0.99399999999999999},"min":{"1min":0.51000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":21.126000000000001,"5min":21.126000000000001,"15min":21.126000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":2.427,"5min":1.5309999999999999,"15min":1.254},"min":{"1min":0.62,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":11.5,"5min":11.5,"15min":11.5},"last":1.0700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.573,"15min":2.0110000000000001},"min":{"1min":0.76400000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":8.3209999999999997,"5min":22.106999999999999,"15min":22.106999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.4239999999999999,"15min":1.244},"min":{"1min":0.58699999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6060000000000001,"5min":15.510999999999999,"15min":15.510999999999999},"last":0.96399999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.841,"5min":1.6419999999999999,"15min":1.74},"min":{"1min":0.88800000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":6.3109999999999999,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.355},{"interface":"front","average":{"1min":1.734,"5min":1.5800000000000001,"15min":1.863},"min":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.8239999999999998,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.246}]},{"osd":5,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0179999999999998,"5min":1.6339999999999999,"15min":1.8600000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":11.552,"5min":13.228999999999999,"15min":13.228999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.851,"5min":1.9159999999999999,"15min":1.706},"min":{"1min":0.85399999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":8.2319999999999993,"5min":20.823,"15min":20.823},"last":1.593}]},{"osd":6,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.282,"5min":1.7749999999999999,"15min":2.0369999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":10.696,"5min":14.272,"15min":14.272},"last":1.657},{"interface":"front","average":{"1min":2.1150000000000002,"5min":1.637,"15min":1.5309999999999999},"min":{"1min":1.0629999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":8.8320000000000007,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.143}]},{"osd":7,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.6459999999999999,"15min":1.575},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":8.2769999999999992,"5min":8.2769999999999992,"15min":8.2769999999999992},"last":1.8580000000000001},{"interface":"front","average":{"1min":3.2530000000000001,"5min":1.881,"15min":1.665},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":24.289999999999999,"5min":24.289999999999999,"15min":24.289999999999999},"last":1.468}]}]},{"osd":0,"up_from":14,"seq":60129542217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40760,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331080,"statfs":{"total":96636764160,"available":96595025920,"internally_reserved":0,"allocated":1286144,"data_stored":877555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.92100000000000004,"15min":0.88500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.1559999999999999,"15min":1.1000000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.2709999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.012},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.0009999999999999,"15min":0.88},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7490000000000001,"5min":2.129,"15min":2.129},"last":0.80700000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.8440000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.381},{"interface":"front","average":{"1min":1.792,"5min":1.3049999999999999,"15min":1.123},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":14.74,"5min":14.74,"15min":14.74},"last":1.0049999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8109999999999999,"5min":1.4179999999999999,"15min":1.3080000000000001},"min":{"1min":1.004,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":8.6959999999999997,"5min":8.6959999999999997,"15min":8.6959999999999997},"last":1.3100000000000001},{"interface":"front","average":{"1min":2.3540000000000001,"5min":1.6839999999999999,"15min":1.5549999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":15.975,"5min":15.975,"15min":15.975},"last":1.5429999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":1.502,"15min":1.373},"min":{"1min":1.0860000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":3.133,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.1639999999999999},{"interface":"front","average":{"1min":2.1000000000000001,"5min":1.571,"15min":1.5129999999999999},"min":{"1min":1.0669999999999999,"5min":0.754,"15min":0.754},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.835}]},{"osd":6,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.524},"min":{"1min":0.96999999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":2.0190000000000001},{"interface":"front","average":{"1min":1.732,"5min":1.526,"15min":1.462},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":1.4399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0409999999999999,"5min":1.71,"15min":1.6639999999999999},"min":{"1min":1.214,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":2.1970000000000001,"5min":1.7090000000000001,"15min":1.593},"min":{"1min":1.012,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":6.2619999999999996,"5min":6.2619999999999996,"15min":6.2619999999999996},"last":1.954}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:00:47.079 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:00:47.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.340+0000 7f882d400640 1 Processor -- start 2024-01-15T18:00:47.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.340+0000 7f882d400640 1 -- start start 2024-01-15T18:00:47.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.342+0000 7f882d400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8828071410 0x7f8828146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.342+0000 7f882d400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88280b2e10 con 0x7f8828071f00 2024-01-15T18:00:47.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.342+0000 7f882d400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882806cc20 con 0x7f882806c7c0 2024-01-15T18:00:47.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.342+0000 7f882d400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882806cec0 con 0x7f8828071410 2024-01-15T18:00:47.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.342+0000 7f882ca00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8828071f00 0x7f8828166cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54288/0 (socket says 172.21.15.26:54288) 2024-01-15T18:00:47.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.342+0000 7f882ca00640 1 -- 172.21.15.26:0/1449069017 learned_addr learned my addr 172.21.15.26:0/1449069017 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827e00640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8828071410 0x7f8828146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722723293 0 0) 0x7f88280b2e10 con 0x7f8828071f00 2024-01-15T18:00:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8808003520 con 0x7f8828071f00 2024-01-15T18:00:47.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2859715648 0 0) 0x7f8808003520 con 0x7f8828071f00 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8828071410 msgr2=0x7f8828146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827400640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8828071410 0x7f8828146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 >> v1:172.21.15.124:6789/0 conn(0x7f882806c7c0 legacy=0x7f88280717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882806d1a0 con 0x7f8828071f00 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f88200019f0 con 0x7f8828071f00 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.343+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 >> v1:172.21.15.26:6789/0 conn(0x7f8828071f00 legacy=0x7f8828166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.344+0000 7f8827400640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88080039a0 0x7f8808003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.344+0000 7f8827400640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f8808004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.344+0000 7f8827400640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8828071410 0x7f8808005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.344+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882806cec0 con 0x7f88080047c0 2024-01-15T18:00:47.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.344+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880800ba60 con 0x7f88080039a0 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.344+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880800bc20 con 0x7f8828071410 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.344+0000 7f881f200640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f8808004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.344+0000 7f8827e00640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88080039a0 0x7f8808003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.344+0000 7f882ca00640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8828071410 0x7f8808005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.345+0000 7f881f200640 1 -- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8828071410 msgr2=0x7f8808005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.345+0000 7f881f200640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8828071410 0x7f8808005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.345+0000 7f881f200640 1 -- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88080039a0 msgr2=0x7f8808003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.345+0000 7f881f200640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88080039a0 0x7f8808003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.345+0000 7f881f200640 1 -- 172.21.15.26:0/1449069017 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8808003520 con 0x7f88080047c0 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.345+0000 7f882ca00640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8828071410 0x7f8808005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:47.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.345+0000 7f8827e00640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88080039a0 0x7f8808003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:47.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.346+0000 7f881f200640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f8808004c40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8810001720 tx=0x7f88080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5b7e1a61fd78c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.346+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88100052d0 con 0x7f88080047c0 2024-01-15T18:00:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.347+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881002e050 con 0x7f88080047c0 2024-01-15T18:00:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.347+0000 7f8827400640 1 -- 172.21.15.26:0/1449069017 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88100035f0 con 0x7f88080047c0 2024-01-15T18:00:47.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.347+0000 7f882d400640 1 -- 172.21.15.26:0/1449069017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 msgr2=0x7f8808004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.347+0000 7f882d400640 1 --2- 172.21.15.26:0/1449069017 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f8808004c40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8810001720 tx=0x7f88080057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.347+0000 7f882ca00640 1 -- 172.21.15.26:0/1449069017 reap_dead start 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.347+0000 7f882d400640 1 -- 172.21.15.26:0/1449069017 shutdown_connections 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.347+0000 7f882d400640 1 -- 172.21.15.26:0/1449069017 >> 172.21.15.26:0/1449069017 conn(0x7f882806a800 msgr2=0x7f882806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.347+0000 7f882d400640 1 -- 172.21.15.26:0/1449069017 shutdown_connections 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.347+0000 7f882d400640 1 -- 172.21.15.26:0/1449069017 wait complete. 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.349+0000 7f882d400640 1 Processor -- start 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.349+0000 7f882d400640 1 -- start start 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.350+0000 7f882d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f882813c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.350+0000 7f882d400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88281f2c50 0x7f882813c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.350+0000 7f882d400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88281f63e0 0x7f88281f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.350+0000 7f882d400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882813cc60 con 0x7f88080047c0 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.350+0000 7f882d400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88281f30e0 con 0x7f88281f2c50 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.350+0000 7f882d400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88281f33c0 con 0x7f88281f63e0 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f8827e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f882813c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f8827e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f882813c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37004/0 (socket says 172.21.15.26:37004) 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f8827e00640 1 -- 172.21.15.26:0/1396413905 learned_addr learned my addr 172.21.15.26:0/1396413905 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f881f200640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88281f2c50 0x7f882813c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f882ca00640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88281f63e0 0x7f88281f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f8827e00640 1 -- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88281f63e0 msgr2=0x7f88281f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f8827e00640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88281f63e0 0x7f88281f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f8827e00640 1 -- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88281f2c50 msgr2=0x7f882813c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f8827e00640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88281f2c50 0x7f882813c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.351+0000 7f8827e00640 1 -- 172.21.15.26:0/1396413905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88281f36a0 con 0x7f88080047c0 2024-01-15T18:00:47.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.352+0000 7f8827e00640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f882813c1e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f88180116d0 tx=0x7f8818011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:47.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.352+0000 7f8824c00640 1 -- 172.21.15.26:0/1396413905 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800f9c0 con 0x7f88080047c0 2024-01-15T18:00:47.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.352+0000 7f8824c00640 1 -- 172.21.15.26:0/1396413905 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881800fb20 con 0x7f88080047c0 2024-01-15T18:00:47.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.352+0000 7f8824c00640 1 -- 172.21.15.26:0/1396413905 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800fed0 con 0x7f88080047c0 2024-01-15T18:00:47.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.352+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88281f3980 con 0x7f88080047c0 2024-01-15T18:00:47.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.352+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882815ce40 con 0x7f88080047c0 2024-01-15T18:00:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.353+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8828067450 con 0x7f88080047c0 2024-01-15T18:00:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.354+0000 7f8824c00640 1 -- 172.21.15.26:0/1396413905 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f8818015460 con 0x7f88080047c0 2024-01-15T18:00:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.355+0000 7f8824c00640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f880006b860 0x7f880006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.355+0000 7f8824c00640 1 -- 172.21.15.26:0/1396413905 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f881808af00 con 0x7f88080047c0 2024-01-15T18:00:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.356+0000 7f881f200640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f880006b860 0x7f880006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.357+0000 7f881f200640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f880006b860 0x7f880006dd20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8810033000 tx=0x7f8810009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:47.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.360+0000 7f8824c00640 1 -- 172.21.15.26:0/1396413905 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8818055530 con 0x7f88080047c0 2024-01-15T18:00:47.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.550+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f882806da40 con 0x7f880006b860 2024-01-15T18:00:47.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.573+0000 7f8824c00640 1 -- 172.21.15.26:0/1396413905 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286126 (secure 0 0 0) 0x7f8810005410 con 0x7f880006b860 2024-01-15T18:00:47.581 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:47.583 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:00:47.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f880006b860 msgr2=0x7f880006dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f880006b860 0x7f880006dd20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f8810033000 tx=0x7f8810009030 comp rx=0 tx=0).stop 2024-01-15T18:00:47.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 msgr2=0x7f882813c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f882813c1e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f88180116d0 tx=0x7f8818011b90 comp rx=0 tx=0).stop 2024-01-15T18:00:47.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 shutdown_connections 2024-01-15T18:00:47.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88281f63e0 0x7f88281f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f880006b860 0x7f880006dd20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88281f2c50 0x7f882813c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 --2- 172.21.15.26:0/1396413905 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88080047c0 0x7f882813c1e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 >> 172.21.15.26:0/1396413905 conn(0x7f882806a800 msgr2=0x7f88280b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:47.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.579+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 shutdown_connections 2024-01-15T18:00:47.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.580+0000 7f882d400640 1 -- 172.21.15.26:0/1396413905 wait complete. 2024-01-15T18:00:47.603 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-01-15T18:00:46.034438+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":194,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":88,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":310116,"kb_used_data":6568,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754664604,"statfs":{"total":773094113280,"available":772776554496,"internally_reserved":0,"allocated":6725632,"data_stored":3478232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002328"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.051825+0000","last_change":"2024-01-15T18:00:41.051825+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.051825+0000","last_undegraded":"2024-01-15T18:00:41.051825+0000","last_fullsized":"2024-01-15T18:00:41.051825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063316+0000","last_change":"2024-01-15T18:00:42.060637+0000","last_active":"2024-01-15T18:00:42.063316+0000","last_peered":"2024-01-15T18:00:42.063316+0000","last_clean":"2024-01-15T18:00:42.063316+0000","last_became_active":"2024-01-15T18:00:42.060376+0000","last_became_peered":"2024-01-15T18:00:42.060376+0000","last_unstale":"2024-01-15T18:00:42.063316+0000","last_undegraded":"2024-01-15T18:00:42.063316+0000","last_fullsized":"2024-01-15T18:00:42.063316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:58:12.883647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082872+0000","last_change":"2024-01-15T18:00:42.077438+0000","last_active":"2024-01-15T18:00:42.082872+0000","last_peered":"2024-01-15T18:00:42.082872+0000","last_clean":"2024-01-15T18:00:42.082872+0000","last_became_active":"2024-01-15T18:00:42.077041+0000","last_became_peered":"2024-01-15T18:00:42.077041+0000","last_unstale":"2024-01-15T18:00:42.082872+0000","last_undegraded":"2024-01-15T18:00:42.082872+0000","last_fullsized":"2024-01-15T18:00:42.082872+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.072305+0000","last_change":"2024-01-15T18:00:42.067513+0000","last_active":"2024-01-15T18:00:42.072305+0000","last_peered":"2024-01-15T18:00:42.072305+0000","last_clean":"2024-01-15T18:00:42.072305+0000","last_became_active":"2024-01-15T18:00:42.066627+0000","last_became_peered":"2024-01-15T18:00:42.066627+0000","last_unstale":"2024-01-15T18:00:42.072305+0000","last_undegraded":"2024-01-15T18:00:42.072305+0000","last_fullsized":"2024-01-15T18:00:42.072305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:13.509938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.043365+0000","last_change":"2024-01-15T18:00:41.043365+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.043365+0000","last_undegraded":"2024-01-15T18:00:41.043365+0000","last_fullsized":"2024-01-15T18:00:41.043365+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076462+0000","last_change":"2024-01-15T18:00:42.067558+0000","last_active":"2024-01-15T18:00:42.076462+0000","last_peered":"2024-01-15T18:00:42.076462+0000","last_clean":"2024-01-15T18:00:42.076462+0000","last_became_active":"2024-01-15T18:00:42.067182+0000","last_became_peered":"2024-01-15T18:00:42.067182+0000","last_unstale":"2024-01-15T18:00:42.076462+0000","last_undegraded":"2024-01-15T18:00:42.076462+0000","last_fullsized":"2024-01-15T18:00:42.076462+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:13.509938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064214+0000","last_change":"2024-01-15T18:00:42.064214+0000","last_active":"2024-01-15T18:00:42.064214+0000","last_peered":"2024-01-15T18:00:42.064214+0000","last_clean":"2024-01-15T18:00:42.064214+0000","last_became_active":"2024-01-15T18:00:42.063930+0000","last_became_peered":"2024-01-15T18:00:42.063930+0000","last_unstale":"2024-01-15T18:00:42.064214+0000","last_undegraded":"2024-01-15T18:00:42.064214+0000","last_fullsized":"2024-01-15T18:00:42.064214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:57:42.271743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.079933+0000","last_change":"2024-01-15T18:00:42.072265+0000","last_active":"2024-01-15T18:00:42.079933+0000","last_peered":"2024-01-15T18:00:42.079933+0000","last_clean":"2024-01-15T18:00:42.079933+0000","last_became_active":"2024-01-15T18:00:42.071980+0000","last_became_peered":"2024-01-15T18:00:42.071980+0000","last_unstale":"2024-01-15T18:00:42.079933+0000","last_undegraded":"2024-01-15T18:00:42.079933+0000","last_fullsized":"2024-01-15T18:00:42.079933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:02:41.900799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":3,"reported_epoch":59,"state":"creating+activating","last_fresh":"2024-01-15T18:00:42.043232+0000","last_change":"2024-01-15T18:00:42.043232+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:42.043232+0000","last_undegraded":"2024-01-15T18:00:42.043232+0000","last_fullsized":"2024-01-15T18:00:42.043232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:25:02.833433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.058936+0000","last_change":"2024-01-15T18:00:42.055688+0000","last_active":"2024-01-15T18:00:42.058936+0000","last_peered":"2024-01-15T18:00:42.058936+0000","last_clean":"2024-01-15T18:00:42.058936+0000","last_became_active":"2024-01-15T18:00:42.055302+0000","last_became_peered":"2024-01-15T18:00:42.055302+0000","last_unstale":"2024-01-15T18:00:42.058936+0000","last_undegraded":"2024-01-15T18:00:42.058936+0000","last_fullsized":"2024-01-15T18:00:42.058936+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:22:28.043841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.066509+0000","last_change":"2024-01-15T18:00:42.066509+0000","last_active":"2024-01-15T18:00:42.066509+0000","last_peered":"2024-01-15T18:00:42.066509+0000","last_clean":"2024-01-15T18:00:42.066509+0000","last_became_active":"2024-01-15T18:00:42.066232+0000","last_became_peered":"2024-01-15T18:00:42.066232+0000","last_unstale":"2024-01-15T18:00:42.066509+0000","last_undegraded":"2024-01-15T18:00:42.066509+0000","last_fullsized":"2024-01-15T18:00:42.066509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:39:13.515709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.072325+0000","last_change":"2024-01-15T18:00:42.068083+0000","last_active":"2024-01-15T18:00:42.072325+0000","last_peered":"2024-01-15T18:00:42.072325+0000","last_clean":"2024-01-15T18:00:42.072325+0000","last_became_active":"2024-01-15T18:00:42.067826+0000","last_became_peered":"2024-01-15T18:00:42.067826+0000","last_unstale":"2024-01-15T18:00:42.072325+0000","last_undegraded":"2024-01-15T18:00:42.072325+0000","last_fullsized":"2024-01-15T18:00:42.072325+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082480+0000","last_change":"2024-01-15T18:00:42.080948+0000","last_active":"2024-01-15T18:00:42.082480+0000","last_peered":"2024-01-15T18:00:42.082480+0000","last_clean":"2024-01-15T18:00:42.082480+0000","last_became_active":"2024-01-15T18:00:42.079522+0000","last_became_peered":"2024-01-15T18:00:42.079522+0000","last_unstale":"2024-01-15T18:00:42.082480+0000","last_undegraded":"2024-01-15T18:00:42.082480+0000","last_fullsized":"2024-01-15T18:00:42.082480+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:44.959104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":3,"reported_epoch":59,"state":"creating+activating","last_fresh":"2024-01-15T18:00:42.042710+0000","last_change":"2024-01-15T18:00:42.042710+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:42.042710+0000","last_undegraded":"2024-01-15T18:00:42.042710+0000","last_fullsized":"2024-01-15T18:00:42.042710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:16:20.555824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063338+0000","last_change":"2024-01-15T18:00:42.063338+0000","last_active":"2024-01-15T18:00:42.063338+0000","last_peered":"2024-01-15T18:00:42.063338+0000","last_clean":"2024-01-15T18:00:42.063338+0000","last_became_active":"2024-01-15T18:00:42.062711+0000","last_became_peered":"2024-01-15T18:00:42.062711+0000","last_unstale":"2024-01-15T18:00:42.063338+0000","last_undegraded":"2024-01-15T18:00:42.063338+0000","last_fullsized":"2024-01-15T18:00:42.063338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:30:23.443661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081709+0000","last_change":"2024-01-15T18:00:42.073413+0000","last_active":"2024-01-15T18:00:42.081709+0000","last_peered":"2024-01-15T18:00:42.081709+0000","last_clean":"2024-01-15T18:00:42.081709+0000","last_became_active":"2024-01-15T18:00:42.072167+0000","last_became_peered":"2024-01-15T18:00:42.072167+0000","last_unstale":"2024-01-15T18:00:42.081709+0000","last_undegraded":"2024-01-15T18:00:42.081709+0000","last_fullsized":"2024-01-15T18:00:42.081709+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:24.550976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.062432+0000","last_change":"2024-01-15T18:00:42.058452+0000","last_active":"2024-01-15T18:00:42.062432+0000","last_peered":"2024-01-15T18:00:42.062432+0000","last_clean":"2024-01-15T18:00:42.062432+0000","last_became_active":"2024-01-15T18:00:42.057885+0000","last_became_peered":"2024-01-15T18:00:42.057885+0000","last_unstale":"2024-01-15T18:00:42.062432+0000","last_undegraded":"2024-01-15T18:00:42.062432+0000","last_fullsized":"2024-01-15T18:00:42.062432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:56.491932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083323+0000","last_change":"2024-01-15T18:00:42.079987+0000","last_active":"2024-01-15T18:00:42.083323+0000","last_peered":"2024-01-15T18:00:42.083323+0000","last_clean":"2024-01-15T18:00:42.083323+0000","last_became_active":"2024-01-15T18:00:42.079525+0000","last_became_peered":"2024-01-15T18:00:42.079525+0000","last_unstale":"2024-01-15T18:00:42.083323+0000","last_undegraded":"2024-01-15T18:00:42.083323+0000","last_fullsized":"2024-01-15T18:00:42.083323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:24.550976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.046537+0000","last_change":"2024-01-15T18:00:41.046537+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.046537+0000","last_undegraded":"2024-01-15T18:00:41.046537+0000","last_fullsized":"2024-01-15T18:00:41.046537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083287+0000","last_change":"2024-01-15T18:00:42.080315+0000","last_active":"2024-01-15T18:00:42.083287+0000","last_peered":"2024-01-15T18:00:42.083287+0000","last_clean":"2024-01-15T18:00:42.083287+0000","last_became_active":"2024-01-15T18:00:42.079623+0000","last_became_peered":"2024-01-15T18:00:42.079623+0000","last_unstale":"2024-01-15T18:00:42.083287+0000","last_undegraded":"2024-01-15T18:00:42.083287+0000","last_fullsized":"2024-01-15T18:00:42.083287+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:56:47.128531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.078789+0000","last_change":"2024-01-15T18:00:42.071297+0000","last_active":"2024-01-15T18:00:42.078789+0000","last_peered":"2024-01-15T18:00:42.078789+0000","last_clean":"2024-01-15T18:00:42.078789+0000","last_became_active":"2024-01-15T18:00:42.071046+0000","last_became_peered":"2024-01-15T18:00:42.071046+0000","last_unstale":"2024-01-15T18:00:42.078789+0000","last_undegraded":"2024-01-15T18:00:42.078789+0000","last_fullsized":"2024-01-15T18:00:42.078789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.077554+0000","last_change":"2024-01-15T18:00:42.066557+0000","last_active":"2024-01-15T18:00:42.077554+0000","last_peered":"2024-01-15T18:00:42.077554+0000","last_clean":"2024-01-15T18:00:42.077554+0000","last_became_active":"2024-01-15T18:00:42.066269+0000","last_became_peered":"2024-01-15T18:00:42.066269+0000","last_unstale":"2024-01-15T18:00:42.077554+0000","last_undegraded":"2024-01-15T18:00:42.077554+0000","last_fullsized":"2024-01-15T18:00:42.077554+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:40:52.391655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076381+0000","last_change":"2024-01-15T18:00:42.071412+0000","last_active":"2024-01-15T18:00:42.076381+0000","last_peered":"2024-01-15T18:00:42.076381+0000","last_clean":"2024-01-15T18:00:42.076381+0000","last_became_active":"2024-01-15T18:00:42.070884+0000","last_became_peered":"2024-01-15T18:00:42.070884+0000","last_unstale":"2024-01-15T18:00:42.076381+0000","last_undegraded":"2024-01-15T18:00:42.076381+0000","last_fullsized":"2024-01-15T18:00:42.076381+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:17.543171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081314+0000","last_change":"2024-01-15T18:00:42.078634+0000","last_active":"2024-01-15T18:00:42.081314+0000","last_peered":"2024-01-15T18:00:42.081314+0000","last_clean":"2024-01-15T18:00:42.081314+0000","last_became_active":"2024-01-15T18:00:42.078165+0000","last_became_peered":"2024-01-15T18:00:42.078165+0000","last_unstale":"2024-01-15T18:00:42.081314+0000","last_undegraded":"2024-01-15T18:00:42.081314+0000","last_fullsized":"2024-01-15T18:00:42.081314+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:41:43.741615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.051170+0000","last_change":"2024-01-15T18:00:41.051170+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.051170+0000","last_undegraded":"2024-01-15T18:00:41.051170+0000","last_fullsized":"2024-01-15T18:00:41.051170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.079608+0000","last_change":"2024-01-15T18:00:42.076259+0000","last_active":"2024-01-15T18:00:42.079608+0000","last_peered":"2024-01-15T18:00:42.079608+0000","last_clean":"2024-01-15T18:00:42.079608+0000","last_became_active":"2024-01-15T18:00:42.076003+0000","last_became_peered":"2024-01-15T18:00:42.076003+0000","last_unstale":"2024-01-15T18:00:42.079608+0000","last_undegraded":"2024-01-15T18:00:42.079608+0000","last_fullsized":"2024-01-15T18:00:42.079608+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:00.533678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080247+0000","last_change":"2024-01-15T18:00:42.072997+0000","last_active":"2024-01-15T18:00:42.080247+0000","last_peered":"2024-01-15T18:00:42.080247+0000","last_clean":"2024-01-15T18:00:42.080247+0000","last_became_active":"2024-01-15T18:00:42.070411+0000","last_became_peered":"2024-01-15T18:00:42.070411+0000","last_unstale":"2024-01-15T18:00:42.080247+0000","last_undegraded":"2024-01-15T18:00:42.080247+0000","last_fullsized":"2024-01-15T18:00:42.080247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:44.207537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.045517+0000","last_change":"2024-01-15T18:00:41.045517+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.045517+0000","last_undegraded":"2024-01-15T18:00:41.045517+0000","last_fullsized":"2024-01-15T18:00:41.045517+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.077697+0000","last_change":"2024-01-15T18:00:42.067598+0000","last_active":"2024-01-15T18:00:42.077697+0000","last_peered":"2024-01-15T18:00:42.077697+0000","last_clean":"2024-01-15T18:00:42.077697+0000","last_became_active":"2024-01-15T18:00:42.067297+0000","last_became_peered":"2024-01-15T18:00:42.067297+0000","last_unstale":"2024-01-15T18:00:42.077697+0000","last_undegraded":"2024-01-15T18:00:42.077697+0000","last_fullsized":"2024-01-15T18:00:42.077697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:42.520698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081309+0000","last_change":"2024-01-15T18:00:42.074513+0000","last_active":"2024-01-15T18:00:42.081309+0000","last_peered":"2024-01-15T18:00:42.081309+0000","last_clean":"2024-01-15T18:00:42.081309+0000","last_became_active":"2024-01-15T18:00:42.074280+0000","last_became_peered":"2024-01-15T18:00:42.074280+0000","last_unstale":"2024-01-15T18:00:42.081309+0000","last_undegraded":"2024-01-15T18:00:42.081309+0000","last_fullsized":"2024-01-15T18:00:42.081309+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-01-15T18:00:41.043336+0000","last_change":"2024-01-15T18:00:41.043336+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:41.043336+0000","last_undegraded":"2024-01-15T18:00:41.043336+0000","last_fullsized":"2024-01-15T18:00:41.043336+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.088984+0000","last_change":"2024-01-15T18:00:42.086123+0000","last_active":"2024-01-15T18:00:42.088984+0000","last_peered":"2024-01-15T18:00:42.088984+0000","last_clean":"2024-01-15T18:00:42.088984+0000","last_became_active":"2024-01-15T18:00:42.085208+0000","last_became_peered":"2024-01-15T18:00:42.085208+0000","last_unstale":"2024-01-15T18:00:42.088984+0000","last_undegraded":"2024-01-15T18:00:42.088984+0000","last_fullsized":"2024-01-15T18:00:42.088984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:00.533678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069251+0000","last_change":"2024-01-15T18:00:42.065563+0000","last_active":"2024-01-15T18:00:42.069251+0000","last_peered":"2024-01-15T18:00:42.069251+0000","last_clean":"2024-01-15T18:00:42.069251+0000","last_became_active":"2024-01-15T18:00:42.065279+0000","last_became_peered":"2024-01-15T18:00:42.065279+0000","last_unstale":"2024-01-15T18:00:42.069251+0000","last_undegraded":"2024-01-15T18:00:42.069251+0000","last_fullsized":"2024-01-15T18:00:42.069251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:52.055388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.092341+0000","last_change":"2024-01-15T18:00:41.072898+0000","last_active":"2024-01-15T18:00:42.092341+0000","last_peered":"2024-01-15T18:00:42.092341+0000","last_clean":"2024-01-15T18:00:42.092341+0000","last_became_active":"2024-01-15T18:00:41.071115+0000","last_became_peered":"2024-01-15T18:00:41.071115+0000","last_unstale":"2024-01-15T18:00:42.092341+0000","last_undegraded":"2024-01-15T18:00:42.092341+0000","last_fullsized":"2024-01-15T18:00:42.092341+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063277+0000","last_change":"2024-01-15T18:00:42.060261+0000","last_active":"2024-01-15T18:00:42.063277+0000","last_peered":"2024-01-15T18:00:42.063277+0000","last_clean":"2024-01-15T18:00:42.063277+0000","last_became_active":"2024-01-15T18:00:42.059975+0000","last_became_peered":"2024-01-15T18:00:42.059975+0000","last_unstale":"2024-01-15T18:00:42.063277+0000","last_undegraded":"2024-01-15T18:00:42.063277+0000","last_fullsized":"2024-01-15T18:00:42.063277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:44.959104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082453+0000","last_change":"2024-01-15T18:00:41.067396+0000","last_active":"2024-01-15T18:00:42.082453+0000","last_peered":"2024-01-15T18:00:42.082453+0000","last_clean":"2024-01-15T18:00:42.082453+0000","last_became_active":"2024-01-15T18:00:41.067056+0000","last_became_peered":"2024-01-15T18:00:41.067056+0000","last_unstale":"2024-01-15T18:00:42.082453+0000","last_undegraded":"2024-01-15T18:00:42.082453+0000","last_fullsized":"2024-01-15T18:00:42.082453+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:24.577110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076694+0000","last_change":"2024-01-15T18:00:42.065885+0000","last_active":"2024-01-15T18:00:42.076694+0000","last_peered":"2024-01-15T18:00:42.076694+0000","last_clean":"2024-01-15T18:00:42.076694+0000","last_became_active":"2024-01-15T18:00:42.065563+0000","last_became_peered":"2024-01-15T18:00:42.065563+0000","last_unstale":"2024-01-15T18:00:42.076694+0000","last_undegraded":"2024-01-15T18:00:42.076694+0000","last_fullsized":"2024-01-15T18:00:42.076694+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:53:36.562001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.090834+0000","last_change":"2024-01-15T18:00:41.053014+0000","last_active":"2024-01-15T18:00:42.090834+0000","last_peered":"2024-01-15T18:00:42.090834+0000","last_clean":"2024-01-15T18:00:42.090834+0000","last_became_active":"2024-01-15T18:00:41.052718+0000","last_became_peered":"2024-01-15T18:00:41.052718+0000","last_unstale":"2024-01-15T18:00:42.090834+0000","last_undegraded":"2024-01-15T18:00:42.090834+0000","last_fullsized":"2024-01-15T18:00:42.090834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080283+0000","last_change":"2024-01-15T18:00:42.077700+0000","last_active":"2024-01-15T18:00:42.080283+0000","last_peered":"2024-01-15T18:00:42.080283+0000","last_clean":"2024-01-15T18:00:42.080283+0000","last_became_active":"2024-01-15T18:00:42.077465+0000","last_became_peered":"2024-01-15T18:00:42.077465+0000","last_unstale":"2024-01-15T18:00:42.080283+0000","last_undegraded":"2024-01-15T18:00:42.080283+0000","last_fullsized":"2024-01-15T18:00:42.080283+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070270+0000","last_change":"2024-01-15T18:00:41.057990+0000","last_active":"2024-01-15T18:00:42.070270+0000","last_peered":"2024-01-15T18:00:42.070270+0000","last_clean":"2024-01-15T18:00:42.070270+0000","last_became_active":"2024-01-15T18:00:41.057666+0000","last_became_peered":"2024-01-15T18:00:41.057666+0000","last_unstale":"2024-01-15T18:00:42.070270+0000","last_undegraded":"2024-01-15T18:00:42.070270+0000","last_fullsized":"2024-01-15T18:00:42.070270+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.077504+0000","last_change":"2024-01-15T18:00:42.066199+0000","last_active":"2024-01-15T18:00:42.077504+0000","last_peered":"2024-01-15T18:00:42.077504+0000","last_clean":"2024-01-15T18:00:42.077504+0000","last_became_active":"2024-01-15T18:00:42.065918+0000","last_became_peered":"2024-01-15T18:00:42.065918+0000","last_unstale":"2024-01-15T18:00:42.077504+0000","last_undegraded":"2024-01-15T18:00:42.077504+0000","last_fullsized":"2024-01-15T18:00:42.077504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:50:20.962431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.044061+0000","last_change":"2024-01-15T18:00:41.063462+0000","last_active":"2024-01-15T18:00:42.044061+0000","last_peered":"2024-01-15T18:00:42.044061+0000","last_clean":"2024-01-15T18:00:42.044061+0000","last_became_active":"2024-01-15T18:00:41.063125+0000","last_became_peered":"2024-01-15T18:00:41.063125+0000","last_unstale":"2024-01-15T18:00:42.044061+0000","last_undegraded":"2024-01-15T18:00:42.044061+0000","last_fullsized":"2024-01-15T18:00:42.044061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:26:23.829753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081534+0000","last_change":"2024-01-15T18:00:42.079309+0000","last_active":"2024-01-15T18:00:42.081534+0000","last_peered":"2024-01-15T18:00:42.081534+0000","last_clean":"2024-01-15T18:00:42.081534+0000","last_became_active":"2024-01-15T18:00:42.078911+0000","last_became_peered":"2024-01-15T18:00:42.078911+0000","last_unstale":"2024-01-15T18:00:42.081534+0000","last_undegraded":"2024-01-15T18:00:42.081534+0000","last_fullsized":"2024-01-15T18:00:42.081534+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:05:36.094656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.090879+0000","last_change":"2024-01-15T18:00:41.069799+0000","last_active":"2024-01-15T18:00:42.090879+0000","last_peered":"2024-01-15T18:00:42.090879+0000","last_clean":"2024-01-15T18:00:42.090879+0000","last_became_active":"2024-01-15T18:00:41.069467+0000","last_became_peered":"2024-01-15T18:00:41.069467+0000","last_unstale":"2024-01-15T18:00:42.090879+0000","last_undegraded":"2024-01-15T18:00:42.090879+0000","last_fullsized":"2024-01-15T18:00:42.090879+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.066827+0000","last_change":"2024-01-15T18:00:42.064990+0000","last_active":"2024-01-15T18:00:42.066827+0000","last_peered":"2024-01-15T18:00:42.066827+0000","last_clean":"2024-01-15T18:00:42.066827+0000","last_became_active":"2024-01-15T18:00:42.064698+0000","last_became_peered":"2024-01-15T18:00:42.064698+0000","last_unstale":"2024-01-15T18:00:42.066827+0000","last_undegraded":"2024-01-15T18:00:42.066827+0000","last_fullsized":"2024-01-15T18:00:42.066827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:23:58.029615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069728+0000","last_change":"2024-01-15T18:00:41.057228+0000","last_active":"2024-01-15T18:00:42.069728+0000","last_peered":"2024-01-15T18:00:42.069728+0000","last_clean":"2024-01-15T18:00:42.069728+0000","last_became_active":"2024-01-15T18:00:41.056929+0000","last_became_peered":"2024-01-15T18:00:41.056929+0000","last_unstale":"2024-01-15T18:00:42.069728+0000","last_undegraded":"2024-01-15T18:00:42.069728+0000","last_fullsized":"2024-01-15T18:00:42.069728+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.065721+0000","last_change":"2024-01-15T18:00:42.057749+0000","last_active":"2024-01-15T18:00:42.065721+0000","last_peered":"2024-01-15T18:00:42.065721+0000","last_clean":"2024-01-15T18:00:42.065721+0000","last_became_active":"2024-01-15T18:00:42.054149+0000","last_became_peered":"2024-01-15T18:00:42.054149+0000","last_unstale":"2024-01-15T18:00:42.065721+0000","last_undegraded":"2024-01-15T18:00:42.065721+0000","last_fullsized":"2024-01-15T18:00:42.065721+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:51.141215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.047331+0000","last_change":"2024-01-15T18:00:41.067706+0000","last_active":"2024-01-15T18:00:42.047331+0000","last_peered":"2024-01-15T18:00:42.047331+0000","last_clean":"2024-01-15T18:00:42.047331+0000","last_became_active":"2024-01-15T18:00:41.067325+0000","last_became_peered":"2024-01-15T18:00:41.067325+0000","last_unstale":"2024-01-15T18:00:42.047331+0000","last_undegraded":"2024-01-15T18:00:42.047331+0000","last_fullsized":"2024-01-15T18:00:42.047331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:50:53.105108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.063135+0000","last_change":"2024-01-15T18:00:42.063135+0000","last_active":"2024-01-15T18:00:42.063135+0000","last_peered":"2024-01-15T18:00:42.063135+0000","last_clean":"2024-01-15T18:00:42.063135+0000","last_became_active":"2024-01-15T18:00:42.062863+0000","last_became_peered":"2024-01-15T18:00:42.062863+0000","last_unstale":"2024-01-15T18:00:42.063135+0000","last_undegraded":"2024-01-15T18:00:42.063135+0000","last_fullsized":"2024-01-15T18:00:42.063135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:11:35.983660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.052982+0000","last_change":"2024-01-15T18:00:41.073770+0000","last_active":"2024-01-15T18:00:42.052982+0000","last_peered":"2024-01-15T18:00:42.052982+0000","last_clean":"2024-01-15T18:00:42.052982+0000","last_became_active":"2024-01-15T18:00:41.073486+0000","last_became_peered":"2024-01-15T18:00:41.073486+0000","last_unstale":"2024-01-15T18:00:42.052982+0000","last_undegraded":"2024-01-15T18:00:42.052982+0000","last_fullsized":"2024-01-15T18:00:42.052982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069498+0000","last_change":"2024-01-15T18:00:42.064514+0000","last_active":"2024-01-15T18:00:42.069498+0000","last_peered":"2024-01-15T18:00:42.069498+0000","last_clean":"2024-01-15T18:00:42.069498+0000","last_became_active":"2024-01-15T18:00:42.064248+0000","last_became_peered":"2024-01-15T18:00:42.064248+0000","last_unstale":"2024-01-15T18:00:42.069498+0000","last_undegraded":"2024-01-15T18:00:42.069498+0000","last_fullsized":"2024-01-15T18:00:42.069498+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:00.533678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.036548+0000","last_change":"2024-01-15T18:00:41.064452+0000","last_active":"2024-01-15T18:00:42.036548+0000","last_peered":"2024-01-15T18:00:42.036548+0000","last_clean":"2024-01-15T18:00:42.036548+0000","last_became_active":"2024-01-15T18:00:41.064118+0000","last_became_peered":"2024-01-15T18:00:41.064118+0000","last_unstale":"2024-01-15T18:00:42.036548+0000","last_undegraded":"2024-01-15T18:00:42.036548+0000","last_fullsized":"2024-01-15T18:00:42.036548+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:31.276292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.068233+0000","last_change":"2024-01-15T18:00:42.068233+0000","last_active":"2024-01-15T18:00:42.068233+0000","last_peered":"2024-01-15T18:00:42.068233+0000","last_clean":"2024-01-15T18:00:42.068233+0000","last_became_active":"2024-01-15T18:00:42.066136+0000","last_became_peered":"2024-01-15T18:00:42.066136+0000","last_unstale":"2024-01-15T18:00:42.068233+0000","last_undegraded":"2024-01-15T18:00:42.068233+0000","last_fullsized":"2024-01-15T18:00:42.068233+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:03:53.619388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085512+0000","last_change":"2024-01-15T18:00:41.069415+0000","last_active":"2024-01-15T18:00:42.085512+0000","last_peered":"2024-01-15T18:00:42.085512+0000","last_clean":"2024-01-15T18:00:42.085512+0000","last_became_active":"2024-01-15T18:00:41.069194+0000","last_became_peered":"2024-01-15T18:00:41.069194+0000","last_unstale":"2024-01-15T18:00:42.085512+0000","last_undegraded":"2024-01-15T18:00:42.085512+0000","last_fullsized":"2024-01-15T18:00:42.085512+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:08.061519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076472+0000","last_change":"2024-01-15T18:00:42.072986+0000","last_active":"2024-01-15T18:00:42.076472+0000","last_peered":"2024-01-15T18:00:42.076472+0000","last_clean":"2024-01-15T18:00:42.076472+0000","last_became_active":"2024-01-15T18:00:42.072700+0000","last_became_peered":"2024-01-15T18:00:42.072700+0000","last_unstale":"2024-01-15T18:00:42.076472+0000","last_undegraded":"2024-01-15T18:00:42.076472+0000","last_fullsized":"2024-01-15T18:00:42.076472+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:48.146540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.044213+0000","last_change":"2024-01-15T18:00:41.056524+0000","last_active":"2024-01-15T18:00:42.044213+0000","last_peered":"2024-01-15T18:00:42.044213+0000","last_clean":"2024-01-15T18:00:42.044213+0000","last_became_active":"2024-01-15T18:00:41.056086+0000","last_became_peered":"2024-01-15T18:00:41.056086+0000","last_unstale":"2024-01-15T18:00:42.044213+0000","last_undegraded":"2024-01-15T18:00:42.044213+0000","last_fullsized":"2024-01-15T18:00:42.044213+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:36.552797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.075686+0000","last_change":"2024-01-15T18:00:42.072006+0000","last_active":"2024-01-15T18:00:42.075686+0000","last_peered":"2024-01-15T18:00:42.075686+0000","last_clean":"2024-01-15T18:00:42.075686+0000","last_became_active":"2024-01-15T18:00:42.071671+0000","last_became_peered":"2024-01-15T18:00:42.071671+0000","last_unstale":"2024-01-15T18:00:42.075686+0000","last_undegraded":"2024-01-15T18:00:42.075686+0000","last_fullsized":"2024-01-15T18:00:42.075686+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085054+0000","last_change":"2024-01-15T18:00:41.067157+0000","last_active":"2024-01-15T18:00:42.085054+0000","last_peered":"2024-01-15T18:00:42.085054+0000","last_clean":"2024-01-15T18:00:42.085054+0000","last_became_active":"2024-01-15T18:00:41.066820+0000","last_became_peered":"2024-01-15T18:00:41.066820+0000","last_unstale":"2024-01-15T18:00:42.085054+0000","last_undegraded":"2024-01-15T18:00:42.085054+0000","last_fullsized":"2024-01-15T18:00:42.085054+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:03:48.949979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.068749+0000","last_change":"2024-01-15T18:00:42.063066+0000","last_active":"2024-01-15T18:00:42.068749+0000","last_peered":"2024-01-15T18:00:42.068749+0000","last_clean":"2024-01-15T18:00:42.068749+0000","last_became_active":"2024-01-15T18:00:42.062649+0000","last_became_peered":"2024-01-15T18:00:42.062649+0000","last_unstale":"2024-01-15T18:00:42.068749+0000","last_undegraded":"2024-01-15T18:00:42.068749+0000","last_fullsized":"2024-01-15T18:00:42.068749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:52.055388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.040717+0000","last_change":"2024-01-15T18:00:41.061164+0000","last_active":"2024-01-15T18:00:42.040717+0000","last_peered":"2024-01-15T18:00:42.040717+0000","last_clean":"2024-01-15T18:00:42.040717+0000","last_became_active":"2024-01-15T18:00:41.060847+0000","last_became_peered":"2024-01-15T18:00:41.060847+0000","last_unstale":"2024-01-15T18:00:42.040717+0000","last_undegraded":"2024-01-15T18:00:42.040717+0000","last_fullsized":"2024-01-15T18:00:42.040717+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":3,"reported_epoch":59,"state":"creating+activating","last_fresh":"2024-01-15T18:00:42.046824+0000","last_change":"2024-01-15T18:00:42.046824+0000","last_active":"2024-01-15T18:00:41.020101+0000","last_peered":"2024-01-15T18:00:41.020101+0000","last_clean":"2024-01-15T18:00:41.020101+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:00:42.046824+0000","last_undegraded":"2024-01-15T18:00:42.046824+0000","last_fullsized":"2024-01-15T18:00:42.046824+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:22:14.891183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":["4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069884+0000","last_change":"2024-01-15T18:00:41.060084+0000","last_active":"2024-01-15T18:00:42.069884+0000","last_peered":"2024-01-15T18:00:42.069884+0000","last_clean":"2024-01-15T18:00:42.069884+0000","last_became_active":"2024-01-15T18:00:41.057544+0000","last_became_peered":"2024-01-15T18:00:41.057544+0000","last_unstale":"2024-01-15T18:00:42.069884+0000","last_undegraded":"2024-01-15T18:00:42.069884+0000","last_fullsized":"2024-01-15T18:00:42.069884+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076361+0000","last_change":"2024-01-15T18:00:42.071201+0000","last_active":"2024-01-15T18:00:42.076361+0000","last_peered":"2024-01-15T18:00:42.076361+0000","last_clean":"2024-01-15T18:00:42.076361+0000","last_became_active":"2024-01-15T18:00:42.070498+0000","last_became_peered":"2024-01-15T18:00:42.070498+0000","last_unstale":"2024-01-15T18:00:42.076361+0000","last_undegraded":"2024-01-15T18:00:42.076361+0000","last_fullsized":"2024-01-15T18:00:42.076361+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:27.820063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.087649+0000","last_change":"2024-01-15T18:00:41.064043+0000","last_active":"2024-01-15T18:00:42.087649+0000","last_peered":"2024-01-15T18:00:42.087649+0000","last_clean":"2024-01-15T18:00:42.087649+0000","last_became_active":"2024-01-15T18:00:41.063128+0000","last_became_peered":"2024-01-15T18:00:41.063128+0000","last_unstale":"2024-01-15T18:00:42.087649+0000","last_undegraded":"2024-01-15T18:00:42.087649+0000","last_fullsized":"2024-01-15T18:00:42.087649+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:12.496547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083296+0000","last_change":"2024-01-15T18:00:42.079795+0000","last_active":"2024-01-15T18:00:42.083296+0000","last_peered":"2024-01-15T18:00:42.083296+0000","last_clean":"2024-01-15T18:00:42.083296+0000","last_became_active":"2024-01-15T18:00:42.079150+0000","last_became_peered":"2024-01-15T18:00:42.079150+0000","last_unstale":"2024-01-15T18:00:42.083296+0000","last_undegraded":"2024-01-15T18:00:42.083296+0000","last_fullsized":"2024-01-15T18:00:42.083296+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:41.713866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.084348+0000","last_change":"2024-01-15T18:00:41.071443+0000","last_active":"2024-01-15T18:00:42.084348+0000","last_peered":"2024-01-15T18:00:42.084348+0000","last_clean":"2024-01-15T18:00:42.084348+0000","last_became_active":"2024-01-15T18:00:41.071219+0000","last_became_peered":"2024-01-15T18:00:41.071219+0000","last_unstale":"2024-01-15T18:00:42.084348+0000","last_undegraded":"2024-01-15T18:00:42.084348+0000","last_fullsized":"2024-01-15T18:00:42.084348+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:56.788064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.081207+0000","last_change":"2024-01-15T18:00:42.079222+0000","last_active":"2024-01-15T18:00:42.081207+0000","last_peered":"2024-01-15T18:00:42.081207+0000","last_clean":"2024-01-15T18:00:42.081207+0000","last_became_active":"2024-01-15T18:00:42.078933+0000","last_became_peered":"2024-01-15T18:00:42.078933+0000","last_unstale":"2024-01-15T18:00:42.081207+0000","last_undegraded":"2024-01-15T18:00:42.081207+0000","last_fullsized":"2024-01-15T18:00:42.081207+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:44.207537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.091309+0000","last_change":"2024-01-15T18:00:41.071043+0000","last_active":"2024-01-15T18:00:42.091309+0000","last_peered":"2024-01-15T18:00:42.091309+0000","last_clean":"2024-01-15T18:00:42.091309+0000","last_became_active":"2024-01-15T18:00:41.070768+0000","last_became_peered":"2024-01-15T18:00:41.070768+0000","last_unstale":"2024-01-15T18:00:42.091309+0000","last_undegraded":"2024-01-15T18:00:42.091309+0000","last_fullsized":"2024-01-15T18:00:42.091309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080200+0000","last_change":"2024-01-15T18:00:42.076070+0000","last_active":"2024-01-15T18:00:42.080200+0000","last_peered":"2024-01-15T18:00:42.080200+0000","last_clean":"2024-01-15T18:00:42.080200+0000","last_became_active":"2024-01-15T18:00:42.068998+0000","last_became_peered":"2024-01-15T18:00:42.068998+0000","last_unstale":"2024-01-15T18:00:42.080200+0000","last_undegraded":"2024-01-15T18:00:42.080200+0000","last_fullsized":"2024-01-15T18:00:42.080200+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:08:12.230431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.072591+0000","last_change":"2024-01-15T18:00:41.061292+0000","last_active":"2024-01-15T18:00:42.072591+0000","last_peered":"2024-01-15T18:00:42.072591+0000","last_clean":"2024-01-15T18:00:42.072591+0000","last_became_active":"2024-01-15T18:00:41.061006+0000","last_became_peered":"2024-01-15T18:00:41.061006+0000","last_unstale":"2024-01-15T18:00:42.072591+0000","last_undegraded":"2024-01-15T18:00:42.072591+0000","last_fullsized":"2024-01-15T18:00:42.072591+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.073395+0000","last_change":"2024-01-15T18:00:42.067416+0000","last_active":"2024-01-15T18:00:42.073395+0000","last_peered":"2024-01-15T18:00:42.073395+0000","last_clean":"2024-01-15T18:00:42.073395+0000","last_became_active":"2024-01-15T18:00:42.067129+0000","last_became_peered":"2024-01-15T18:00:42.067129+0000","last_unstale":"2024-01-15T18:00:42.073395+0000","last_undegraded":"2024-01-15T18:00:42.073395+0000","last_fullsized":"2024-01-15T18:00:42.073395+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:49.570465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070835+0000","last_change":"2024-01-15T18:00:41.071456+0000","last_active":"2024-01-15T18:00:42.070835+0000","last_peered":"2024-01-15T18:00:42.070835+0000","last_clean":"2024-01-15T18:00:42.070835+0000","last_became_active":"2024-01-15T18:00:41.071189+0000","last_became_peered":"2024-01-15T18:00:41.071189+0000","last_unstale":"2024-01-15T18:00:42.070835+0000","last_undegraded":"2024-01-15T18:00:42.070835+0000","last_fullsized":"2024-01-15T18:00:42.070835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070175+0000","last_change":"2024-01-15T18:00:42.065297+0000","last_active":"2024-01-15T18:00:42.070175+0000","last_peered":"2024-01-15T18:00:42.070175+0000","last_clean":"2024-01-15T18:00:42.070175+0000","last_became_active":"2024-01-15T18:00:42.064969+0000","last_became_peered":"2024-01-15T18:00:42.064969+0000","last_unstale":"2024-01-15T18:00:42.070175+0000","last_undegraded":"2024-01-15T18:00:42.070175+0000","last_fullsized":"2024-01-15T18:00:42.070175+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.084074+0000","last_change":"2024-01-15T18:00:41.068101+0000","last_active":"2024-01-15T18:00:42.084074+0000","last_peered":"2024-01-15T18:00:42.084074+0000","last_clean":"2024-01-15T18:00:42.084074+0000","last_became_active":"2024-01-15T18:00:41.065928+0000","last_became_peered":"2024-01-15T18:00:41.065928+0000","last_unstale":"2024-01-15T18:00:42.084074+0000","last_undegraded":"2024-01-15T18:00:42.084074+0000","last_fullsized":"2024-01-15T18:00:42.084074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:19:23.525421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.075413+0000","last_change":"2024-01-15T18:00:42.067592+0000","last_active":"2024-01-15T18:00:42.075413+0000","last_peered":"2024-01-15T18:00:42.075413+0000","last_clean":"2024-01-15T18:00:42.075413+0000","last_became_active":"2024-01-15T18:00:42.067309+0000","last_became_peered":"2024-01-15T18:00:42.067309+0000","last_unstale":"2024-01-15T18:00:42.075413+0000","last_undegraded":"2024-01-15T18:00:42.075413+0000","last_fullsized":"2024-01-15T18:00:42.075413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.046741+0000","last_change":"2024-01-15T18:00:41.050813+0000","last_active":"2024-01-15T18:00:42.046741+0000","last_peered":"2024-01-15T18:00:42.046741+0000","last_clean":"2024-01-15T18:00:42.046741+0000","last_became_active":"2024-01-15T18:00:41.050320+0000","last_became_peered":"2024-01-15T18:00:41.050320+0000","last_unstale":"2024-01-15T18:00:42.046741+0000","last_undegraded":"2024-01-15T18:00:42.046741+0000","last_fullsized":"2024-01-15T18:00:42.046741+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.075305+0000","last_change":"2024-01-15T18:00:42.070090+0000","last_active":"2024-01-15T18:00:42.075305+0000","last_peered":"2024-01-15T18:00:42.075305+0000","last_clean":"2024-01-15T18:00:42.075305+0000","last_became_active":"2024-01-15T18:00:42.068897+0000","last_became_peered":"2024-01-15T18:00:42.068897+0000","last_unstale":"2024-01-15T18:00:42.075305+0000","last_undegraded":"2024-01-15T18:00:42.075305+0000","last_fullsized":"2024-01-15T18:00:42.075305+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070801+0000","last_change":"2024-01-15T18:00:41.065251+0000","last_active":"2024-01-15T18:00:42.070801+0000","last_peered":"2024-01-15T18:00:42.070801+0000","last_clean":"2024-01-15T18:00:42.070801+0000","last_became_active":"2024-01-15T18:00:41.065006+0000","last_became_peered":"2024-01-15T18:00:41.065006+0000","last_unstale":"2024-01-15T18:00:42.070801+0000","last_undegraded":"2024-01-15T18:00:42.070801+0000","last_fullsized":"2024-01-15T18:00:42.070801+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083131+0000","last_change":"2024-01-15T18:00:42.083131+0000","last_active":"2024-01-15T18:00:42.083131+0000","last_peered":"2024-01-15T18:00:42.083131+0000","last_clean":"2024-01-15T18:00:42.083131+0000","last_became_active":"2024-01-15T18:00:42.082546+0000","last_became_peered":"2024-01-15T18:00:42.082546+0000","last_unstale":"2024-01-15T18:00:42.083131+0000","last_undegraded":"2024-01-15T18:00:42.083131+0000","last_fullsized":"2024-01-15T18:00:42.083131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:24.550976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.046483+0000","last_change":"2024-01-15T18:00:41.066193+0000","last_active":"2024-01-15T18:00:42.046483+0000","last_peered":"2024-01-15T18:00:42.046483+0000","last_clean":"2024-01-15T18:00:42.046483+0000","last_became_active":"2024-01-15T18:00:41.065913+0000","last_became_peered":"2024-01-15T18:00:41.065913+0000","last_unstale":"2024-01-15T18:00:42.046483+0000","last_undegraded":"2024-01-15T18:00:42.046483+0000","last_fullsized":"2024-01-15T18:00:42.046483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.079328+0000","last_change":"2024-01-15T18:00:42.076020+0000","last_active":"2024-01-15T18:00:42.079328+0000","last_peered":"2024-01-15T18:00:42.079328+0000","last_clean":"2024-01-15T18:00:42.079328+0000","last_became_active":"2024-01-15T18:00:42.075751+0000","last_became_peered":"2024-01-15T18:00:42.075751+0000","last_unstale":"2024-01-15T18:00:42.079328+0000","last_undegraded":"2024-01-15T18:00:42.079328+0000","last_fullsized":"2024-01-15T18:00:42.079328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:52.055388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.076414+0000","last_change":"2024-01-15T18:00:41.070458+0000","last_active":"2024-01-15T18:00:42.076414+0000","last_peered":"2024-01-15T18:00:42.076414+0000","last_clean":"2024-01-15T18:00:42.076414+0000","last_became_active":"2024-01-15T18:00:41.066322+0000","last_became_peered":"2024-01-15T18:00:41.066322+0000","last_unstale":"2024-01-15T18:00:42.076414+0000","last_undegraded":"2024-01-15T18:00:42.076414+0000","last_fullsized":"2024-01-15T18:00:42.076414+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":8,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.066727+0000","last_change":"2024-01-15T18:00:42.066727+0000","last_active":"2024-01-15T18:00:42.066727+0000","last_peered":"2024-01-15T18:00:42.066727+0000","last_clean":"2024-01-15T18:00:42.066727+0000","last_became_active":"2024-01-15T18:00:42.065788+0000","last_became_peered":"2024-01-15T18:00:42.065788+0000","last_unstale":"2024-01-15T18:00:42.066727+0000","last_undegraded":"2024-01-15T18:00:42.066727+0000","last_fullsized":"2024-01-15T18:00:42.066727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:01:13.061488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.040831+0000","last_change":"2024-01-15T18:00:41.063919+0000","last_active":"2024-01-15T18:00:42.040831+0000","last_peered":"2024-01-15T18:00:42.040831+0000","last_clean":"2024-01-15T18:00:42.040831+0000","last_became_active":"2024-01-15T18:00:41.062115+0000","last_became_peered":"2024-01-15T18:00:41.062115+0000","last_unstale":"2024-01-15T18:00:42.040831+0000","last_undegraded":"2024-01-15T18:00:42.040831+0000","last_fullsized":"2024-01-15T18:00:42.040831+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.080268+0000","last_change":"2024-01-15T18:00:42.073589+0000","last_active":"2024-01-15T18:00:42.080268+0000","last_peered":"2024-01-15T18:00:42.080268+0000","last_clean":"2024-01-15T18:00:42.080268+0000","last_became_active":"2024-01-15T18:00:42.073254+0000","last_became_peered":"2024-01-15T18:00:42.073254+0000","last_unstale":"2024-01-15T18:00:42.080268+0000","last_undegraded":"2024-01-15T18:00:42.080268+0000","last_fullsized":"2024-01-15T18:00:42.080268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:29.053165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.070418+0000","last_change":"2024-01-15T18:00:41.057399+0000","last_active":"2024-01-15T18:00:42.070418+0000","last_peered":"2024-01-15T18:00:42.070418+0000","last_clean":"2024-01-15T18:00:42.070418+0000","last_became_active":"2024-01-15T18:00:41.056174+0000","last_became_peered":"2024-01-15T18:00:41.056174+0000","last_unstale":"2024-01-15T18:00:42.070418+0000","last_undegraded":"2024-01-15T18:00:42.070418+0000","last_fullsized":"2024-01-15T18:00:42.070418+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.061539+0000","last_change":"2024-01-15T18:00:42.052005+0000","last_active":"2024-01-15T18:00:42.061539+0000","last_peered":"2024-01-15T18:00:42.061539+0000","last_clean":"2024-01-15T18:00:42.061539+0000","last_became_active":"2024-01-15T18:00:42.051122+0000","last_became_peered":"2024-01-15T18:00:42.051122+0000","last_unstale":"2024-01-15T18:00:42.061539+0000","last_undegraded":"2024-01-15T18:00:42.061539+0000","last_fullsized":"2024-01-15T18:00:42.061539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:59:19.158591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.046278+0000","last_change":"2024-01-15T18:00:41.064076+0000","last_active":"2024-01-15T18:00:42.046278+0000","last_peered":"2024-01-15T18:00:42.046278+0000","last_clean":"2024-01-15T18:00:42.046278+0000","last_became_active":"2024-01-15T18:00:41.062980+0000","last_became_peered":"2024-01-15T18:00:41.062980+0000","last_unstale":"2024-01-15T18:00:42.046278+0000","last_undegraded":"2024-01-15T18:00:42.046278+0000","last_fullsized":"2024-01-15T18:00:42.046278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:41:42.728224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.078756+0000","last_change":"2024-01-15T18:00:42.070329+0000","last_active":"2024-01-15T18:00:42.078756+0000","last_peered":"2024-01-15T18:00:42.078756+0000","last_clean":"2024-01-15T18:00:42.078756+0000","last_became_active":"2024-01-15T18:00:42.068798+0000","last_became_peered":"2024-01-15T18:00:42.068798+0000","last_unstale":"2024-01-15T18:00:42.078756+0000","last_undegraded":"2024-01-15T18:00:42.078756+0000","last_fullsized":"2024-01-15T18:00:42.078756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:44.207537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.043282+0000","last_change":"2024-01-15T18:00:41.054271+0000","last_active":"2024-01-15T18:00:42.043282+0000","last_peered":"2024-01-15T18:00:42.043282+0000","last_clean":"2024-01-15T18:00:42.043282+0000","last_became_active":"2024-01-15T18:00:41.053925+0000","last_became_peered":"2024-01-15T18:00:41.053925+0000","last_unstale":"2024-01-15T18:00:42.043282+0000","last_undegraded":"2024-01-15T18:00:42.043282+0000","last_fullsized":"2024-01-15T18:00:42.043282+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:18:46.672266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064312+0000","last_change":"2024-01-15T18:00:42.057721+0000","last_active":"2024-01-15T18:00:42.064312+0000","last_peered":"2024-01-15T18:00:42.064312+0000","last_clean":"2024-01-15T18:00:42.064312+0000","last_became_active":"2024-01-15T18:00:42.057091+0000","last_became_peered":"2024-01-15T18:00:42.057091+0000","last_unstale":"2024-01-15T18:00:42.064312+0000","last_undegraded":"2024-01-15T18:00:42.064312+0000","last_fullsized":"2024-01-15T18:00:42.064312+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:33:19.250071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.045742+0000","last_change":"2024-01-15T18:00:41.061944+0000","last_active":"2024-01-15T18:00:42.045742+0000","last_peered":"2024-01-15T18:00:42.045742+0000","last_clean":"2024-01-15T18:00:42.045742+0000","last_became_active":"2024-01-15T18:00:41.061646+0000","last_became_peered":"2024-01-15T18:00:41.061646+0000","last_unstale":"2024-01-15T18:00:42.045742+0000","last_undegraded":"2024-01-15T18:00:42.045742+0000","last_fullsized":"2024-01-15T18:00:42.045742+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:23:57.016225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.065689+0000","last_change":"2024-01-15T18:00:42.055286+0000","last_active":"2024-01-15T18:00:42.065689+0000","last_peered":"2024-01-15T18:00:42.065689+0000","last_clean":"2024-01-15T18:00:42.065689+0000","last_became_active":"2024-01-15T18:00:42.054746+0000","last_became_peered":"2024-01-15T18:00:42.054746+0000","last_unstale":"2024-01-15T18:00:42.065689+0000","last_undegraded":"2024-01-15T18:00:42.065689+0000","last_fullsized":"2024-01-15T18:00:42.065689+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:14.526164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.036713+0000","last_change":"2024-01-15T18:00:41.068287+0000","last_active":"2024-01-15T18:00:42.036713+0000","last_peered":"2024-01-15T18:00:42.036713+0000","last_clean":"2024-01-15T18:00:42.036713+0000","last_became_active":"2024-01-15T18:00:41.066120+0000","last_became_peered":"2024-01-15T18:00:41.066120+0000","last_unstale":"2024-01-15T18:00:42.036713+0000","last_undegraded":"2024-01-15T18:00:42.036713+0000","last_fullsized":"2024-01-15T18:00:42.036713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:39:48.165240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083708+0000","last_change":"2024-01-15T18:00:42.080052+0000","last_active":"2024-01-15T18:00:42.083708+0000","last_peered":"2024-01-15T18:00:42.083708+0000","last_clean":"2024-01-15T18:00:42.083708+0000","last_became_active":"2024-01-15T18:00:42.079699+0000","last_became_peered":"2024-01-15T18:00:42.079699+0000","last_unstale":"2024-01-15T18:00:42.083708+0000","last_undegraded":"2024-01-15T18:00:42.083708+0000","last_fullsized":"2024-01-15T18:00:42.083708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:32:27.598423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.040946+0000","last_change":"2024-01-15T18:00:41.061168+0000","last_active":"2024-01-15T18:00:42.040946+0000","last_peered":"2024-01-15T18:00:42.040946+0000","last_clean":"2024-01-15T18:00:42.040946+0000","last_became_active":"2024-01-15T18:00:41.060793+0000","last_became_peered":"2024-01-15T18:00:41.060793+0000","last_unstale":"2024-01-15T18:00:42.040946+0000","last_undegraded":"2024-01-15T18:00:42.040946+0000","last_fullsized":"2024-01-15T18:00:42.040946+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:02.304705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":195,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-01-15T18:00:41.055451+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:00:41.055451+0000","last_peered":"2024-01-15T18:00:41.055451+0000","last_clean":"2024-01-15T18:00:41.055451+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:00:41.055451+0000","last_undegraded":"2024-01-15T18:00:41.055451+0000","last_fullsized":"2024-01-15T18:00:41.055451+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":29,"ondisk_log_size":29,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542216,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36600,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335240,"statfs":{"total":96636764160,"available":96599285760,"internally_reserved":0,"allocated":1286144,"data_stored":877555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.2230000000000001,"15min":1.177},"min":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":5.556,"5min":5.556,"15min":5.556},"last":1.179},{"interface":"front","average":{"1min":3.444,"5min":1.649,"15min":1.3260000000000001},"min":{"1min":0.747,"5min":0.623,"15min":0.623},"max":{"1min":38.137,"5min":38.137,"15min":38.137},"last":1.3420000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.266,"15min":1.25},"min":{"1min":0.72999999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":4.194,"5min":1.861,"15min":1.4570000000000001},"min":{"1min":0.72699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":51.843000000000004,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.0009999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.921,"5min":1.361,"15min":1.2270000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":9.3369999999999997,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":1.288},{"interface":"front","average":{"1min":1.643,"5min":1.397,"15min":1.393},"min":{"1min":0.66000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.2889999999999997,"5min":6.2889999999999997,"15min":6.2889999999999997},"last":1.3959999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":1.49,"15min":1.363},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":9.1440000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5029999999999999,"15min":1.4970000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":1.534}]},{"osd":4,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":3.3300000000000001,"5min":1.788,"15min":1.5980000000000001},"min":{"1min":0.95799999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":22.359999999999999,"5min":22.359999999999999,"15min":22.359999999999999},"last":1.1240000000000001},{"interface":"front","average":{"1min":2.2130000000000001,"5min":1.597,"15min":1.496},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":12.667999999999999,"5min":12.667999999999999,"15min":12.667999999999999},"last":1.599}]},{"osd":5,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.0220000000000002,"5min":2.0569999999999999,"15min":1.7569999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":25.006,"5min":25.006,"15min":25.006},"last":1.649},{"interface":"front","average":{"1min":3.4950000000000001,"5min":1.9239999999999999,"15min":1.712},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":22.693999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.2330000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.9530000000000003,"5min":2.2770000000000001,"15min":1.901},"min":{"1min":1.0580000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":23.834,"5min":23.834,"15min":23.834},"last":1.4359999999999999},{"interface":"front","average":{"1min":5.9169999999999998,"5min":2.4729999999999999,"15min":1.9199999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":63.932000000000002,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.7010000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542215,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.413,"15min":1.3169999999999999},"min":{"1min":0.71699999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.8769999999999998,"5min":12.544,"15min":12.544},"last":1.345},{"interface":"front","average":{"1min":2.1840000000000002,"5min":1.423,"15min":1.248},"min":{"1min":0.81899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":13.98,"5min":13.98,"15min":13.98},"last":1.0880000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.357,"15min":1.2250000000000001},"min":{"1min":0.82599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.5659999999999998,"5min":12.811999999999999,"15min":12.811999999999999},"last":1.1910000000000001},{"interface":"front","average":{"1min":2.0600000000000001,"5min":1.349,"15min":1.2330000000000001},"min":{"1min":0.86499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":8.8130000000000006,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.1399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2970000000000002,"5min":1.456,"15min":1.28},"min":{"1min":0.90300000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4890000000000001,"15min":1.5009999999999999},"min":{"1min":0.86899999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":4.0419999999999998,"5min":12.897,"15min":12.897},"last":1.466}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.988,"5min":1.7010000000000001,"15min":1.542},"min":{"1min":1.0880000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":9.5,"5min":12.717000000000001,"15min":12.717000000000001},"last":1.5629999999999999},{"interface":"front","average":{"1min":2.52,"5min":1.5840000000000001,"15min":1.4410000000000001},"min":{"1min":0.93799999999999994,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":14.066000000000001,"5min":14.066000000000001,"15min":14.066000000000001},"last":1.835}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5009999999999999,"15min":1.454},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":9.2829999999999995,"5min":9.2829999999999995,"15min":9.2829999999999995},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.6359999999999999,"15min":1.607},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.6989999999999998,"5min":16.98,"15min":16.98},"last":1.6539999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":1.548,"15min":1.484},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":14.176,"5min":14.176,"15min":14.176},"last":1.6919999999999999},{"interface":"front","average":{"1min":2.3079999999999998,"5min":1.756,"15min":1.7569999999999999},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"last":1.3879999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":1.6399999999999999,"15min":1.601},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":14.109999999999999,"5min":14.109999999999999,"15min":14.109999999999999},"last":1.966},{"interface":"front","average":{"1min":2.5870000000000002,"5min":1.8009999999999999,"15min":1.7769999999999999},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":14.478,"5min":14.478,"15min":14.478},"last":1.7529999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3149999999999999,"15min":1.2509999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.7490000000000001,"5min":16.178000000000001,"15min":16.178000000000001},"last":1.046},{"interface":"front","average":{"1min":2.4980000000000002,"5min":1.4750000000000001,"15min":1.417},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":17.184999999999999,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.5349999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2929999999999999,"15min":1.2250000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":5.6219999999999999,"5min":5.6219999999999999,"15min":5.6219999999999999},"last":1.274},{"interface":"front","average":{"1min":1.423,"5min":1.2629999999999999,"15min":1.2110000000000001},"min":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.1379999999999999,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":1.1399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2050000000000001,"15min":1.2190000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.681,"5min":2.835,"15min":2.835},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.367,"15min":1.288},"min":{"1min":1.0760000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.9740000000000002,"5min":5.9740000000000002,"15min":5.9740000000000002},"last":1.327}]},{"osd":3,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.3520000000000001},"min":{"1min":0.73099999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8029999999999999,"5min":14.914,"15min":14.914},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.365,"15min":1.413},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.726,"5min":3.302,"15min":3.302},"last":1.2929999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.395,"15min":1.411},"min":{"1min":0.99299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.7759999999999998,"5min":4.7759999999999998,"15min":4.7759999999999998},"last":1.1819999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.75,"15min":1.571},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":17.991,"5min":17.991,"15min":17.991},"last":0.92100000000000004}]},{"osd":6,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.506,"15min":1.6020000000000001},"min":{"1min":0.91900000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":6.4630000000000001,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.792,"5min":1.5369999999999999,"15min":1.55},"min":{"1min":0.88100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":7.7759999999999998,"5min":7.7759999999999998,"15min":7.7759999999999998},"last":1.4159999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5580000000000001,"15min":1.6060000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":6.3230000000000004,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.603,"15min":1.728},"min":{"1min":0.90500000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.2030000000000003,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.109}]}]},{"osd":4,"up_from":14,"seq":60129542215,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36020,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335820,"statfs":{"total":96636764160,"available":96599879680,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.9989999999999997,"5min":11.413,"15min":11.413},"last":1.492},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.4350000000000001,"15min":1.3799999999999999},"min":{"1min":0.77500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":9.7439999999999998,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":1.367}]},{"osd":1,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.4670000000000001,"15min":1.3819999999999999},"min":{"1min":0.84299999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":9.7010000000000005,"5min":11.987,"15min":11.987},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.27,"15min":1.323},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":6.149,"5min":6.149,"15min":6.149},"last":1.2210000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.708,"15min":1.5700000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":7.032,"5min":13.865,"15min":13.865},"last":1.343},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.514,"15min":1.3959999999999999},"min":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.028,"5min":13.903,"15min":13.903},"last":1.4139999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.6040000000000001,"15min":1.5920000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":6.8819999999999997,"5min":11.611000000000001,"15min":11.611000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":2.0800000000000001,"5min":1.546,"15min":1.6180000000000001},"min":{"1min":0.69199999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":13.25,"5min":13.25,"15min":13.25},"last":1.5589999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.6359999999999999,"15min":1.6850000000000001},"min":{"1min":0.61899999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":6.6820000000000004,"5min":11.561,"15min":11.561},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.8999999999999999,"5min":1.5509999999999999,"15min":1.665},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":9.6229999999999993,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.28}]},{"osd":6,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.468,"15min":1.6259999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.698,"5min":1.8500000000000001,"15min":1.8460000000000001},"min":{"1min":0.85799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":6.9329999999999998,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.6439999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":2.359,"5min":1.8979999999999999,"15min":1.9059999999999999},"min":{"1min":1.375,"5min":0.503,"15min":0.503},"max":{"1min":10.157999999999999,"5min":11.894,"15min":11.894},"last":1.456},{"interface":"front","average":{"1min":2.5169999999999999,"5min":2.048,"15min":1.968},"min":{"1min":1.325,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":8.4169999999999998,"5min":11.714,"15min":11.714},"last":1.702}]}]},{"osd":3,"up_from":14,"seq":60129542217,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44340,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327500,"statfs":{"total":96636764160,"available":96591360000,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":1.1990000000000001,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.435,"15min":0.435},"max":{"1min":9.9789999999999992,"5min":9.9789999999999992,"15min":9.9789999999999992},"last":0.76800000000000002},{"interface":"front","average":{"1min":2.9420000000000002,"5min":1.5149999999999999,"15min":1.218},"min":{"1min":0.85199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":21.047000000000001,"5min":21.047000000000001,"15min":21.047000000000001},"last":1.242}]},{"osd":1,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.8919999999999999,"5min":1.524,"15min":1.179},"min":{"1min":0.79700000000000004,"5min":0.375,"15min":0.375},"max":{"1min":17.777999999999999,"5min":17.777999999999999,"15min":17.777999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.786,"5min":1.2210000000000001,"15min":1.135},"min":{"1min":0.69299999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":9.2360000000000007,"5min":9.2360000000000007,"15min":9.2360000000000007},"last":0.94999999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.27,"15min":1.1899999999999999},"min":{"1min":0.79000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.4930000000000003,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":1.331},{"interface":"front","average":{"1min":2.0510000000000002,"5min":1.492,"15min":1.4119999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":9.3040000000000003,"5min":9.3040000000000003,"15min":9.3040000000000003},"last":1.2889999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":1.51,"15min":1.4199999999999999},"min":{"1min":1.1120000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":1.1299999999999999},{"interface":"front","average":{"1min":2.367,"5min":1.643,"15min":1.4890000000000001},"min":{"1min":0.92300000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":10.313000000000001,"5min":10.313000000000001,"15min":10.313000000000001},"last":1.077}]},{"osd":5,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.2229999999999999,"5min":1.635,"15min":1.5649999999999999},"min":{"1min":1.046,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.243,"5min":10.243,"15min":10.243},"last":1.5},{"interface":"front","average":{"1min":2.2330000000000001,"5min":1.742,"15min":1.6599999999999999},"min":{"1min":0.90500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":6.2000000000000002,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.544}]},{"osd":6,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.5510000000000002,"5min":1.851,"15min":1.7250000000000001},"min":{"1min":0.98399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":10.012,"5min":10.012,"15min":10.012},"last":1.6120000000000001},{"interface":"front","average":{"1min":2.6190000000000002,"5min":1.8220000000000001,"15min":1.722},"min":{"1min":1.0840000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":10.291,"5min":10.291,"15min":10.291},"last":1.5760000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.802,"5min":1.8819999999999999,"15min":1.7589999999999999},"min":{"1min":1.02,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":12.627000000000001,"5min":12.627000000000001,"15min":12.627000000000001},"last":1.4510000000000001},{"interface":"front","average":{"1min":2.9710000000000001,"5min":1.946,"15min":1.7929999999999999},"min":{"1min":1.1240000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":10.268000000000001,"5min":10.268000000000001,"15min":10.268000000000001},"last":1.2150000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542217,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40180,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331660,"statfs":{"total":96636764160,"available":96595619840,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.042},"min":{"1min":0.48999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.466,"5min":14.427,"15min":14.427},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.0860000000000001,"15min":1.02},"min":{"1min":0.64100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":1.7290000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3129999999999999,"15min":1.2949999999999999},"min":{"1min":0.53000000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7490000000000001,"5min":14.609,"15min":14.609},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.29,"5min":1.1830000000000001,"15min":1.2010000000000001},"min":{"1min":0.748,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5059999999999998,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.2789999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2609999999999999,"15min":1.1990000000000001},"min":{"1min":0.64700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.772,"5min":16.567,"15min":16.567},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.48,"15min":1.4630000000000001},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8180000000000001,"5min":14.528,"15min":14.528},"last":1.383}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.708,"15min":1.647},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.7690000000000001,"5min":14.779,"15min":14.779},"last":1.4750000000000001},{"interface":"front","average":{"1min":2.073,"5min":1.546,"15min":1.502},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":10.75,"5min":10.75,"15min":10.75},"last":1.5089999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.5609999999999999,"15min":1.506},"min":{"1min":0.85199999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.258,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.653,"5min":1.726,"15min":1.659},"min":{"1min":1,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.4750000000000001,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.5600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.889,"5min":1.5580000000000001,"15min":1.5149999999999999},"min":{"1min":1.131,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.6339999999999999,"15min":1.5},"min":{"1min":1.171,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.415,"5min":17.07,"15min":17.07},"last":1.2390000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.839,"5min":1.621,"15min":1.6259999999999999},"min":{"1min":1.333,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.2170000000000001,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.6040000000000001},{"interface":"front","average":{"1min":2.9049999999999998,"5min":2.1070000000000002,"15min":1.7070000000000001},"min":{"1min":1.2070000000000001,"5min":0.752,"15min":0.752},"max":{"1min":15.135,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.4419999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542216,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40176,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331664,"statfs":{"total":96636764160,"available":96595623936,"internally_reserved":0,"allocated":692224,"data_stored":287187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.095,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":8.1140000000000008,"5min":8.1140000000000008,"15min":8.1140000000000008},"last":1.1930000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0169999999999999,"15min":0.99299999999999999},"min":{"1min":0.64000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.456,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.73799999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.4289999999999998,"5min":1.3109999999999999,"15min":0.99399999999999999},"min":{"1min":0.51000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":21.126000000000001,"5min":21.126000000000001,"15min":21.126000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":2.427,"5min":1.5309999999999999,"15min":1.254},"min":{"1min":0.62,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":11.5,"5min":11.5,"15min":11.5},"last":1.0700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.573,"15min":2.0110000000000001},"min":{"1min":0.76400000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":8.3209999999999997,"5min":22.106999999999999,"15min":22.106999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.4239999999999999,"15min":1.244},"min":{"1min":0.58699999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6060000000000001,"5min":15.510999999999999,"15min":15.510999999999999},"last":0.96399999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.841,"5min":1.6419999999999999,"15min":1.74},"min":{"1min":0.88800000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":6.3109999999999999,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.355},{"interface":"front","average":{"1min":1.734,"5min":1.5800000000000001,"15min":1.863},"min":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.8239999999999998,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.246}]},{"osd":5,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0179999999999998,"5min":1.6339999999999999,"15min":1.8600000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":11.552,"5min":13.228999999999999,"15min":13.228999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.851,"5min":1.9159999999999999,"15min":1.706},"min":{"1min":0.85399999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":8.2319999999999993,"5min":20.823,"15min":20.823},"last":1.593}]},{"osd":6,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.282,"5min":1.7749999999999999,"15min":2.0369999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":10.696,"5min":14.272,"15min":14.272},"last":1.657},{"interface":"front","average":{"1min":2.1150000000000002,"5min":1.637,"15min":1.5309999999999999},"min":{"1min":1.0629999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":8.8320000000000007,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.143}]},{"osd":7,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.6459999999999999,"15min":1.575},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":8.2769999999999992,"5min":8.2769999999999992,"15min":8.2769999999999992},"last":1.8580000000000001},{"interface":"front","average":{"1min":3.2530000000000001,"5min":1.881,"15min":1.665},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":24.289999999999999,"5min":24.289999999999999,"15min":24.289999999999999},"last":1.468}]}]},{"osd":0,"up_from":14,"seq":60129542217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40760,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331080,"statfs":{"total":96636764160,"available":96595025920,"internally_reserved":0,"allocated":1286144,"data_stored":877555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.92100000000000004,"15min":0.88500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.1559999999999999,"15min":1.1000000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.2709999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.012},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.0009999999999999,"15min":0.88},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7490000000000001,"5min":2.129,"15min":2.129},"last":0.80700000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.8440000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.381},{"interface":"front","average":{"1min":1.792,"5min":1.3049999999999999,"15min":1.123},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":14.74,"5min":14.74,"15min":14.74},"last":1.0049999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8109999999999999,"5min":1.4179999999999999,"15min":1.3080000000000001},"min":{"1min":1.004,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":8.6959999999999997,"5min":8.6959999999999997,"15min":8.6959999999999997},"last":1.3100000000000001},{"interface":"front","average":{"1min":2.3540000000000001,"5min":1.6839999999999999,"15min":1.5549999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":15.975,"5min":15.975,"15min":15.975},"last":1.5429999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":1.502,"15min":1.373},"min":{"1min":1.0860000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":3.133,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.1639999999999999},{"interface":"front","average":{"1min":2.1000000000000001,"5min":1.571,"15min":1.5129999999999999},"min":{"1min":1.0669999999999999,"5min":0.754,"15min":0.754},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.835}]},{"osd":6,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.524},"min":{"1min":0.96999999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":2.0190000000000001},{"interface":"front","average":{"1min":1.732,"5min":1.526,"15min":1.462},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":1.4399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0409999999999999,"5min":1.71,"15min":1.6639999999999999},"min":{"1min":1.214,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":2.1970000000000001,"5min":1.7090000000000001,"15min":1.593},"min":{"1min":1.012,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":6.2619999999999996,"5min":6.2619999999999996,"15min":6.2619999999999996},"last":1.954}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:00:47.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2024-01-15T18:00:47.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:00:41.051825+0000', 'last_change': '2024-01-15T18:00:41.051825+0000', 'last_active': '2024-01-15T18:00:41.020101+0000', 'last_peered': '2024-01-15T18:00:41.020101+0000', 'last_clean': '2024-01-15T18:00:41.020101+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:00:41.051825+0000', 'last_undegraded': '2024-01-15T18:00:41.051825+0000', 'last_fullsized': '2024-01-15T18:00:41.051825+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_clean_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:00:47.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2024-01-15T18:00:47.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:00:41.043365+0000', 'last_change': '2024-01-15T18:00:41.043365+0000', 'last_active': '2024-01-15T18:00:41.020101+0000', 'last_peered': '2024-01-15T18:00:41.020101+0000', 'last_clean': '2024-01-15T18:00:41.020101+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:00:41.043365+0000', 'last_undegraded': '2024-01-15T18:00:41.043365+0000', 'last_fullsized': '2024-01-15T18:00:41.043365+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_clean_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:00:47.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2024-01-15T18:00:47.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 59, 'state': 'creating+activating', 'last_fresh': '2024-01-15T18:00:42.043232+0000', 'last_change': '2024-01-15T18:00:42.043232+0000', 'last_active': '2024-01-15T18:00:41.020101+0000', 'last_peered': '2024-01-15T18:00:41.020101+0000', 'last_clean': '2024-01-15T18:00:41.020101+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:00:42.043232+0000', 'last_undegraded': '2024-01-15T18:00:42.043232+0000', 'last_fullsized': '2024-01-15T18:00:42.043232+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_clean_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-16T20:25:02.833433+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': ['4', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:00:47.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2024-01-15T18:00:47.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 59, 'state': 'creating+activating', 'last_fresh': '2024-01-15T18:00:42.042710+0000', 'last_change': '2024-01-15T18:00:42.042710+0000', 'last_active': '2024-01-15T18:00:41.020101+0000', 'last_peered': '2024-01-15T18:00:41.020101+0000', 'last_clean': '2024-01-15T18:00:41.020101+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:00:42.042710+0000', 'last_undegraded': '2024-01-15T18:00:42.042710+0000', 'last_fullsized': '2024-01-15T18:00:42.042710+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_clean_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-16T23:16:20.555824+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': ['4', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:00:47.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2024-01-15T18:00:47.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:00:41.046537+0000', 'last_change': '2024-01-15T18:00:41.046537+0000', 'last_active': '2024-01-15T18:00:41.020101+0000', 'last_peered': '2024-01-15T18:00:41.020101+0000', 'last_clean': '2024-01-15T18:00:41.020101+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:00:41.046537+0000', 'last_undegraded': '2024-01-15T18:00:41.046537+0000', 'last_fullsized': '2024-01-15T18:00:41.046537+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_clean_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:00:47.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2024-01-15T18:00:47.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:00:41.051170+0000', 'last_change': '2024-01-15T18:00:41.051170+0000', 'last_active': '2024-01-15T18:00:41.020101+0000', 'last_peered': '2024-01-15T18:00:41.020101+0000', 'last_clean': '2024-01-15T18:00:41.020101+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:00:41.051170+0000', 'last_undegraded': '2024-01-15T18:00:41.051170+0000', 'last_fullsized': '2024-01-15T18:00:41.051170+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_clean_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:00:47.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-01-15T18:00:47.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:00:41.045517+0000', 'last_change': '2024-01-15T18:00:41.045517+0000', 'last_active': '2024-01-15T18:00:41.020101+0000', 'last_peered': '2024-01-15T18:00:41.020101+0000', 'last_clean': '2024-01-15T18:00:41.020101+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:00:41.045517+0000', 'last_undegraded': '2024-01-15T18:00:41.045517+0000', 'last_fullsized': '2024-01-15T18:00:41.045517+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_clean_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:00:47.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2024-01-15T18:00:47.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 58, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:00:41.043336+0000', 'last_change': '2024-01-15T18:00:41.043336+0000', 'last_active': '2024-01-15T18:00:41.020101+0000', 'last_peered': '2024-01-15T18:00:41.020101+0000', 'last_clean': '2024-01-15T18:00:41.020101+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:00:41.043336+0000', 'last_undegraded': '2024-01-15T18:00:41.043336+0000', 'last_fullsized': '2024-01-15T18:00:41.043336+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_clean_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:00:47.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2024-01-15T18:00:47.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 59, 'state': 'creating+activating', 'last_fresh': '2024-01-15T18:00:42.046824+0000', 'last_change': '2024-01-15T18:00:42.046824+0000', 'last_active': '2024-01-15T18:00:41.020101+0000', 'last_peered': '2024-01-15T18:00:41.020101+0000', 'last_clean': '2024-01-15T18:00:41.020101+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:00:42.046824+0000', 'last_undegraded': '2024-01-15T18:00:42.046824+0000', 'last_fullsized': '2024-01-15T18:00:42.046824+0000', 'mapping_epoch': 58, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 58, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'last_clean_scrub_stamp': '2024-01-15T18:00:41.020101+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-01-17T02:22:14.891183+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': ['4', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:00:47.611 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-15T18:00:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.869+0000 7f4a45800640 1 Processor -- start 2024-01-15T18:00:47.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.869+0000 7f4a45800640 1 -- start start 2024-01-15T18:00:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.870+0000 7f4a45800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4a400cf9a0 0x7f4a4015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.870+0000 7f4a45800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 0x7f4a400702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.870+0000 7f4a45800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a400b1ee0 con 0x7f4a40143ae0 2024-01-15T18:00:47.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.870+0000 7f4a45800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a400b2780 con 0x7f4a400cf9a0 2024-01-15T18:00:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.870+0000 7f4a45800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4015e990 con 0x7f4a4015e1e0 2024-01-15T18:00:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a3ea00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4a40143ae0 0x7f4a400cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54292/0 (socket says 172.21.15.26:54292) 2024-01-15T18:00:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a3ea00640 1 -- 172.21.15.26:0/3627266120 learned_addr learned my addr 172.21.15.26:0/3627266120 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a35e00640 1 --2- 172.21.15.26:0/3627266120 >> v2:172.21.15.26:3300/0 conn(0x7f4a400cf9a0 0x7f4a4015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a3e000640 1 -- 172.21.15.26:0/3627266120 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246652339 0 0) 0x7f4a400b1ee0 con 0x7f4a40143ae0 2024-01-15T18:00:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a3f400640 1 --2- 172.21.15.26:0/3627266120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 0x7f4a400702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a3e000640 1 -- 172.21.15.26:0/3627266120 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a20003520 con 0x7f4a40143ae0 2024-01-15T18:00:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a35e00640 1 -- 172.21.15.26:0/3627266120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 msgr2=0x7f4a400702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:47.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a35e00640 1 --2- 172.21.15.26:0/3627266120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 0x7f4a400702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a35e00640 1 -- 172.21.15.26:0/3627266120 >> v1:172.21.15.26:6789/0 conn(0x7f4a40143ae0 legacy=0x7f4a400cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a35e00640 1 -- 172.21.15.26:0/3627266120 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a400b2130 con 0x7f4a400cf9a0 2024-01-15T18:00:47.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.871+0000 7f4a3f400640 1 --2- 172.21.15.26:0/3627266120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 0x7f4a400702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.873+0000 7f4a35e00640 1 --2- 172.21.15.26:0/3627266120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 0x7f4a4015dca0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4a280097a0 tx=0x7f4a2802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b89f31b8d70be2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.873+0000 7f4a3e000640 1 -- 172.21.15.26:0/3627266120 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2802d040 con 0x7f4a400cf9a0 2024-01-15T18:00:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.873+0000 7f4a3e000640 1 -- 172.21.15.26:0/3627266120 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28031810 con 0x7f4a400cf9a0 2024-01-15T18:00:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.873+0000 7f4a3e000640 1 -- 172.21.15.26:0/3627266120 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28031ba0 con 0x7f4a400cf9a0 2024-01-15T18:00:47.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.874+0000 7f4a45800640 1 -- 172.21.15.26:0/3627266120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 msgr2=0x7f4a4015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.874+0000 7f4a45800640 1 --2- 172.21.15.26:0/3627266120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 0x7f4a4015dca0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4a280097a0 tx=0x7f4a2802b8b0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.874+0000 7f4a45800640 1 -- 172.21.15.26:0/3627266120 shutdown_connections 2024-01-15T18:00:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.874+0000 7f4a45800640 1 --2- 172.21.15.26:0/3627266120 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 0x7f4a400702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.874+0000 7f4a45800640 1 --2- 172.21.15.26:0/3627266120 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 0x7f4a4015dca0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.874+0000 7f4a45800640 1 -- 172.21.15.26:0/3627266120 >> 172.21.15.26:0/3627266120 conn(0x7f4a4006a800 msgr2=0x7f4a4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.874+0000 7f4a45800640 1 -- 172.21.15.26:0/3627266120 shutdown_connections 2024-01-15T18:00:47.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.874+0000 7f4a45800640 1 -- 172.21.15.26:0/3627266120 wait complete. 2024-01-15T18:00:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.876+0000 7f4a45800640 1 Processor -- start 2024-01-15T18:00:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.876+0000 7f4a45800640 1 -- start start 2024-01-15T18:00:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a45800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 0x7f4a400cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a45800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a40143ae0 0x7f4a400c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a45800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 0x7f4a400ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a45800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4015a1b0 con 0x7f4a400cf9a0 2024-01-15T18:00:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a45800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40131630 con 0x7f4a40143ae0 2024-01-15T18:00:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a45800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a401318a0 con 0x7f4a4015e1e0 2024-01-15T18:00:47.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a3ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 0x7f4a400cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a3ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 0x7f4a400cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37030/0 (socket says 172.21.15.26:37030) 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a3ea00640 1 -- 172.21.15.26:0/2673371220 learned_addr learned my addr 172.21.15.26:0/2673371220 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a3ea00640 1 -- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 msgr2=0x7f4a400ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a3ea00640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 0x7f4a400ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a3ea00640 1 -- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a40143ae0 msgr2=0x7f4a400c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a3ea00640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a40143ae0 0x7f4a400c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.877+0000 7f4a3ea00640 1 -- 172.21.15.26:0/2673371220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a28009400 con 0x7f4a400cf9a0 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.878+0000 7f4a3ea00640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 0x7f4a400cb350 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c530 tx=0x7f4a3000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.878+0000 7f4a37400640 1 -- 172.21.15.26:0/2673371220 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30005ce0 con 0x7f4a400cf9a0 2024-01-15T18:00:47.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.878+0000 7f4a37400640 1 -- 172.21.15.26:0/2673371220 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30005e40 con 0x7f4a400cf9a0 2024-01-15T18:00:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.878+0000 7f4a37400640 1 -- 172.21.15.26:0/2673371220 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a300072b0 con 0x7f4a400cf9a0 2024-01-15T18:00:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.878+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a40131b80 con 0x7f4a400cf9a0 2024-01-15T18:00:47.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.878+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4013f410 con 0x7f4a400cf9a0 2024-01-15T18:00:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.880+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a400675d0 con 0x7f4a400cf9a0 2024-01-15T18:00:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.880+0000 7f4a37400640 1 -- 172.21.15.26:0/2673371220 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f4a30007410 con 0x7f4a400cf9a0 2024-01-15T18:00:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.881+0000 7f4a37400640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4a1806b8b0 0x7f4a1806dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.881+0000 7f4a37400640 1 -- 172.21.15.26:0/2673371220 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4a30089c80 con 0x7f4a400cf9a0 2024-01-15T18:00:47.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.882+0000 7f4a35e00640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4a1806b8b0 0x7f4a1806dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.883+0000 7f4a35e00640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4a1806b8b0 0x7f4a1806dd70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4a28036000 tx=0x7f4a28030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:47.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:47.886+0000 7f4a37400640 1 -- 172.21.15.26:0/2673371220 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a30054390 con 0x7f4a400cf9a0 2024-01-15T18:00:48.011 INFO:tasks.ceph.mds.d.smithi124.stderr:2024-01-15T18:00:48.008+0000 7fe42ea00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:00:48.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.150+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4a4015dca0 con 0x7f4a400cf9a0 2024-01-15T18:00:48.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.151+0000 7f4a37400640 1 -- 172.21.15.26:0/2673371220 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1014 (secure 0 0 0) 0x7f4a30017080 con 0x7f4a400cf9a0 2024-01-15T18:00:48.154 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:48.154 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":362,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":60,"num_osds":8,"num_up_osds":8,"osd_up_since":1705341296,"num_in_osds":8,"osd_in_since":1705341288,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":317820928,"bytes_avail":772776292352,"bytes_total":773094113280,"write_bytes_sec":2048,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":35,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":35186}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-01-15T17:59:26.004851+0000","services":{}},"progress_events":{}} 2024-01-15T18:00:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.153+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4a1806b8b0 msgr2=0x7f4a1806dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.153+0000 7f4a45800640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4a1806b8b0 0x7f4a1806dd70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4a28036000 tx=0x7f4a28030000 comp rx=0 tx=0).stop 2024-01-15T18:00:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.153+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 msgr2=0x7f4a400cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:48.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.153+0000 7f4a45800640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 0x7f4a400cb350 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000c530 tx=0x7f4a3000c9f0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.153+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 shutdown_connections 2024-01-15T18:00:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.153+0000 7f4a45800640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4a4015e1e0 0x7f4a400ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.153+0000 7f4a45800640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4a1806b8b0 0x7f4a1806dd70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.153+0000 7f4a45800640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4a40143ae0 0x7f4a400c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.153+0000 7f4a45800640 1 --2- 172.21.15.26:0/2673371220 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4a400cf9a0 0x7f4a400cb350 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.154+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 >> 172.21.15.26:0/2673371220 conn(0x7f4a4006a800 msgr2=0x7f4a4006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.154+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 shutdown_connections 2024-01-15T18:00:48.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.154+0000 7f4a45800640 1 -- 172.21.15.26:0/2673371220 wait complete. 2024-01-15T18:00:48.173 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-15T18:00:48.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:00:48.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.383+0000 7f8aa7e00640 1 Processor -- start 2024-01-15T18:00:48.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.383+0000 7f8aa7e00640 1 -- start start 2024-01-15T18:00:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.385+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa815afb0 0x7f8aa813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.385+0000 7f8aa7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8aa813fae0 0x7f8aa80b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:48.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.385+0000 7f8aa7e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa813ff00 con 0x7f8aa813cfa0 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.385+0000 7f8aa7e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa80cdc40 con 0x7f8aa813fae0 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.385+0000 7f8aa7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa80cdec0 con 0x7f8aa815afb0 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.385+0000 7f8aa7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8aa813fae0 0x7f8aa80b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.385+0000 7f8aa7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8aa813fae0 0x7f8aa80b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37046/0 (socket says 172.21.15.26:37046) 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.385+0000 7f8aa7400640 1 -- 172.21.15.26:0/2959169899 learned_addr learned my addr 172.21.15.26:0/2959169899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.386+0000 7f8aa6000640 1 --2- 172.21.15.26:0/2959169899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa815afb0 0x7f8aa813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.386+0000 7f8aa7400640 1 -- 172.21.15.26:0/2959169899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa815afb0 msgr2=0x7f8aa813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.386+0000 7f8aa7400640 1 --2- 172.21.15.26:0/2959169899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa815afb0 0x7f8aa813f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.386+0000 7f8aa7400640 1 -- 172.21.15.26:0/2959169899 >> v1:172.21.15.124:6789/0 conn(0x7f8aa813cfa0 legacy=0x7f8aa80cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:48.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.386+0000 7f8aa7400640 1 -- 172.21.15.26:0/2959169899 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa80ce1a0 con 0x7f8aa813fae0 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.387+0000 7f8aa7400640 1 --2- 172.21.15.26:0/2959169899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa813fae0 0x7f8aa80b2040 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009e50 tx=0x7f8aa002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8202d64508f466 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.387+0000 7f8aa5600640 1 -- 172.21.15.26:0/2959169899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa002ca80 con 0x7f8aa813fae0 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.387+0000 7f8aa5600640 1 -- 172.21.15.26:0/2959169899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa002cbe0 con 0x7f8aa813fae0 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.387+0000 7f8aa5600640 1 -- 172.21.15.26:0/2959169899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00343f0 con 0x7f8aa813fae0 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.388+0000 7f8aa7e00640 1 -- 172.21.15.26:0/2959169899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa813fae0 msgr2=0x7f8aa80b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.388+0000 7f8aa7e00640 1 --2- 172.21.15.26:0/2959169899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa813fae0 0x7f8aa80b2040 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009e50 tx=0x7f8aa002bee0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.388+0000 7f8aa7e00640 1 -- 172.21.15.26:0/2959169899 shutdown_connections 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.388+0000 7f8aa7e00640 1 --2- 172.21.15.26:0/2959169899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa813fae0 0x7f8aa80b2040 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.388+0000 7f8aa7e00640 1 --2- 172.21.15.26:0/2959169899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa815afb0 0x7f8aa813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.388+0000 7f8aa7e00640 1 -- 172.21.15.26:0/2959169899 >> 172.21.15.26:0/2959169899 conn(0x7f8aa806a800 msgr2=0x7f8aa806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.388+0000 7f8aa7e00640 1 -- 172.21.15.26:0/2959169899 shutdown_connections 2024-01-15T18:00:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.388+0000 7f8aa7e00640 1 -- 172.21.15.26:0/2959169899 wait complete. 2024-01-15T18:00:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.390+0000 7f8aa7e00640 1 Processor -- start 2024-01-15T18:00:48.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.390+0000 7f8aa7e00640 1 -- start start 2024-01-15T18:00:48.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa813cfa0 0x7f8aa815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aa813fae0 0x7f8aa8150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa815afb0 0x7f8aa8150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa7e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa81304b0 con 0x7f8aa815afb0 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa7e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa8133c20 con 0x7f8aa813fae0 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa813e8d0 con 0x7f8aa813cfa0 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa815afb0 0x7f8aa8150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa815afb0 0x7f8aa8150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37050/0 (socket says 172.21.15.26:37050) 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa7400640 1 -- 172.21.15.26:0/3547815631 learned_addr learned my addr 172.21.15.26:0/3547815631 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa6a00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa813cfa0 0x7f8aa815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.391+0000 7f8aa6000640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aa813fae0 0x7f8aa8150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:48.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8aa7400640 1 -- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa813cfa0 msgr2=0x7f8aa815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8aa7400640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa813cfa0 0x7f8aa815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8aa7400640 1 -- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aa813fae0 msgr2=0x7f8aa8150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8aa7400640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aa813fae0 0x7f8aa8150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8aa7400640 1 -- 172.21.15.26:0/3547815631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa0009b30 con 0x7f8aa815afb0 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8aa6000640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aa813fae0 0x7f8aa8150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8aa7400640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa815afb0 0x7f8aa8150940 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009e50 tx=0x7f8aa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8a9ea00640 1 -- 172.21.15.26:0/3547815631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0009510 con 0x7f8aa815afb0 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8a9ea00640 1 -- 172.21.15.26:0/3547815631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa003e450 con 0x7f8aa815afb0 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8a9ea00640 1 -- 172.21.15.26:0/3547815631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa003e800 con 0x7f8aa815afb0 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa813ebb0 con 0x7f8aa815afb0 2024-01-15T18:00:48.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.392+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa8150e80 con 0x7f8aa815afb0 2024-01-15T18:00:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.393+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa8133dd0 con 0x7f8aa815afb0 2024-01-15T18:00:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.394+0000 7f8a9ea00640 1 -- 172.21.15.26:0/3547815631 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f8aa002cd50 con 0x7f8aa815afb0 2024-01-15T18:00:48.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.395+0000 7f8a9ea00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a8006b860 0x7f8a8006dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.395+0000 7f8a9ea00640 1 -- 172.21.15.26:0/3547815631 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8aa00ad0b0 con 0x7f8aa815afb0 2024-01-15T18:00:48.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.396+0000 7f8aa6a00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a8006b860 0x7f8a8006dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:48.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.397+0000 7f8aa6a00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a8006b860 0x7f8a8006dd20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8a98000f60 tx=0x7f8a98009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:48.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.400+0000 7f8a9ea00640 1 -- 172.21.15.26:0/3547815631 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8aa8133dd0 con 0x7f8aa815afb0 2024-01-15T18:00:48.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.589+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8aa815a530 con 0x7f8a8006b860 2024-01-15T18:00:48.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.612+0000 7f8a9ea00640 1 -- 172.21.15.26:0/3547815631 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286683 (secure 0 0 0) 0x7f8aa815a530 con 0x7f8a8006b860 2024-01-15T18:00:48.620 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:48.622 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:00:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a8006b860 msgr2=0x7f8a8006dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a8006b860 0x7f8a8006dd20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8a98000f60 tx=0x7f8a98009040 comp rx=0 tx=0).stop 2024-01-15T18:00:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa815afb0 msgr2=0x7f8aa8150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa815afb0 0x7f8aa8150940 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009e50 tx=0x7f8aa0004a40 comp rx=0 tx=0).stop 2024-01-15T18:00:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 shutdown_connections 2024-01-15T18:00:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8a8006b860 0x7f8a8006dd20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8aa815afb0 0x7f8aa8150940 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8aa813fae0 0x7f8aa8150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 --2- 172.21.15.26:0/3547815631 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8aa813cfa0 0x7f8aa815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 >> 172.21.15.26:0/3547815631 conn(0x7f8aa806a800 msgr2=0x7f8aa80cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 shutdown_connections 2024-01-15T18:00:48.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:48.618+0000 7f8aa7e00640 1 -- 172.21.15.26:0/3547815631 wait complete. 2024-01-15T18:00:48.641 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-01-15T18:00:48.035091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":310372,"kb_used_data":6824,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754664348,"statfs":{"total":773094113280,"available":772776292352,"internally_reserved":0,"allocated":6987776,"data_stored":3588100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.996721"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106217+0000","last_change":"2024-01-15T18:00:42.075316+0000","last_active":"2024-01-15T18:00:43.106217+0000","last_peered":"2024-01-15T18:00:43.106217+0000","last_clean":"2024-01-15T18:00:43.106217+0000","last_became_active":"2024-01-15T18:00:42.075016+0000","last_became_peered":"2024-01-15T18:00:42.075016+0000","last_unstale":"2024-01-15T18:00:43.106217+0000","last_undegraded":"2024-01-15T18:00:43.106217+0000","last_fullsized":"2024-01-15T18:00:43.106217+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:36.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106550+0000","last_change":"2024-01-15T18:00:42.060637+0000","last_active":"2024-01-15T18:00:43.106550+0000","last_peered":"2024-01-15T18:00:43.106550+0000","last_clean":"2024-01-15T18:00:43.106550+0000","last_became_active":"2024-01-15T18:00:42.060376+0000","last_became_peered":"2024-01-15T18:00:42.060376+0000","last_unstale":"2024-01-15T18:00:43.106550+0000","last_undegraded":"2024-01-15T18:00:43.106550+0000","last_fullsized":"2024-01-15T18:00:43.106550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:39:13.052279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104921+0000","last_change":"2024-01-15T18:00:42.077438+0000","last_active":"2024-01-15T18:00:43.104921+0000","last_peered":"2024-01-15T18:00:43.104921+0000","last_clean":"2024-01-15T18:00:43.104921+0000","last_became_active":"2024-01-15T18:00:42.077041+0000","last_became_peered":"2024-01-15T18:00:42.077041+0000","last_unstale":"2024-01-15T18:00:43.104921+0000","last_undegraded":"2024-01-15T18:00:43.104921+0000","last_fullsized":"2024-01-15T18:00:43.104921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:25.288201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105004+0000","last_change":"2024-01-15T18:00:42.067513+0000","last_active":"2024-01-15T18:00:43.105004+0000","last_peered":"2024-01-15T18:00:43.105004+0000","last_clean":"2024-01-15T18:00:43.105004+0000","last_became_active":"2024-01-15T18:00:42.066627+0000","last_became_peered":"2024-01-15T18:00:42.066627+0000","last_unstale":"2024-01-15T18:00:43.105004+0000","last_undegraded":"2024-01-15T18:00:43.105004+0000","last_fullsized":"2024-01-15T18:00:43.105004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:24:18.622665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105449+0000","last_change":"2024-01-15T18:00:42.062431+0000","last_active":"2024-01-15T18:00:43.105449+0000","last_peered":"2024-01-15T18:00:43.105449+0000","last_clean":"2024-01-15T18:00:43.105449+0000","last_became_active":"2024-01-15T18:00:42.062153+0000","last_became_peered":"2024-01-15T18:00:42.062153+0000","last_unstale":"2024-01-15T18:00:43.105449+0000","last_undegraded":"2024-01-15T18:00:43.105449+0000","last_fullsized":"2024-01-15T18:00:43.105449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:03:23.463624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.064844+0000","last_change":"2024-01-15T18:00:42.067558+0000","last_active":"2024-01-15T18:00:43.064844+0000","last_peered":"2024-01-15T18:00:43.064844+0000","last_clean":"2024-01-15T18:00:43.064844+0000","last_became_active":"2024-01-15T18:00:42.067182+0000","last_became_peered":"2024-01-15T18:00:42.067182+0000","last_unstale":"2024-01-15T18:00:43.064844+0000","last_undegraded":"2024-01-15T18:00:43.064844+0000","last_fullsized":"2024-01-15T18:00:43.064844+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:59.730413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.067178+0000","last_change":"2024-01-15T18:00:42.064214+0000","last_active":"2024-01-15T18:00:43.067178+0000","last_peered":"2024-01-15T18:00:43.067178+0000","last_clean":"2024-01-15T18:00:43.067178+0000","last_became_active":"2024-01-15T18:00:42.063930+0000","last_became_peered":"2024-01-15T18:00:42.063930+0000","last_unstale":"2024-01-15T18:00:43.067178+0000","last_undegraded":"2024-01-15T18:00:43.067178+0000","last_fullsized":"2024-01-15T18:00:43.067178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:29.053165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103502+0000","last_change":"2024-01-15T18:00:42.072265+0000","last_active":"2024-01-15T18:00:43.103502+0000","last_peered":"2024-01-15T18:00:43.103502+0000","last_clean":"2024-01-15T18:00:43.103502+0000","last_became_active":"2024-01-15T18:00:42.071980+0000","last_became_peered":"2024-01-15T18:00:42.071980+0000","last_unstale":"2024-01-15T18:00:43.103502+0000","last_undegraded":"2024-01-15T18:00:43.103502+0000","last_fullsized":"2024-01-15T18:00:43.103502+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:53.751752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.068181+0000","last_change":"2024-01-15T18:00:42.081161+0000","last_active":"2024-01-15T18:00:43.068181+0000","last_peered":"2024-01-15T18:00:43.068181+0000","last_clean":"2024-01-15T18:00:43.068181+0000","last_became_active":"2024-01-15T18:00:42.080853+0000","last_became_peered":"2024-01-15T18:00:42.080853+0000","last_unstale":"2024-01-15T18:00:43.068181+0000","last_undegraded":"2024-01-15T18:00:43.068181+0000","last_fullsized":"2024-01-15T18:00:43.068181+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103753+0000","last_change":"2024-01-15T18:00:42.055688+0000","last_active":"2024-01-15T18:00:43.103753+0000","last_peered":"2024-01-15T18:00:43.103753+0000","last_clean":"2024-01-15T18:00:43.103753+0000","last_became_active":"2024-01-15T18:00:42.055302+0000","last_became_peered":"2024-01-15T18:00:42.055302+0000","last_unstale":"2024-01-15T18:00:43.103753+0000","last_undegraded":"2024-01-15T18:00:43.103753+0000","last_fullsized":"2024-01-15T18:00:43.103753+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:22.274947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104110+0000","last_change":"2024-01-15T18:00:42.066509+0000","last_active":"2024-01-15T18:00:43.104110+0000","last_peered":"2024-01-15T18:00:43.104110+0000","last_clean":"2024-01-15T18:00:43.104110+0000","last_became_active":"2024-01-15T18:00:42.066232+0000","last_became_peered":"2024-01-15T18:00:42.066232+0000","last_unstale":"2024-01-15T18:00:43.104110+0000","last_undegraded":"2024-01-15T18:00:43.104110+0000","last_fullsized":"2024-01-15T18:00:43.104110+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:16.633951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105338+0000","last_change":"2024-01-15T18:00:42.068083+0000","last_active":"2024-01-15T18:00:43.105338+0000","last_peered":"2024-01-15T18:00:43.105338+0000","last_clean":"2024-01-15T18:00:43.105338+0000","last_became_active":"2024-01-15T18:00:42.067826+0000","last_became_peered":"2024-01-15T18:00:42.067826+0000","last_unstale":"2024-01-15T18:00:43.105338+0000","last_undegraded":"2024-01-15T18:00:43.105338+0000","last_fullsized":"2024-01-15T18:00:43.105338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:25.288201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.066253+0000","last_change":"2024-01-15T18:00:42.080948+0000","last_active":"2024-01-15T18:00:43.066253+0000","last_peered":"2024-01-15T18:00:43.066253+0000","last_clean":"2024-01-15T18:00:43.066253+0000","last_became_active":"2024-01-15T18:00:42.079522+0000","last_became_peered":"2024-01-15T18:00:42.079522+0000","last_unstale":"2024-01-15T18:00:43.066253+0000","last_undegraded":"2024-01-15T18:00:43.066253+0000","last_fullsized":"2024-01-15T18:00:43.066253+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:16.633951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.067883+0000","last_change":"2024-01-15T18:00:42.069509+0000","last_active":"2024-01-15T18:00:43.067883+0000","last_peered":"2024-01-15T18:00:43.067883+0000","last_clean":"2024-01-15T18:00:43.067883+0000","last_became_active":"2024-01-15T18:00:42.068025+0000","last_became_peered":"2024-01-15T18:00:42.068025+0000","last_unstale":"2024-01-15T18:00:43.067883+0000","last_undegraded":"2024-01-15T18:00:43.067883+0000","last_fullsized":"2024-01-15T18:00:43.067883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.067445+0000","last_change":"2024-01-15T18:00:42.063338+0000","last_active":"2024-01-15T18:00:43.067445+0000","last_peered":"2024-01-15T18:00:43.067445+0000","last_clean":"2024-01-15T18:00:43.067445+0000","last_became_active":"2024-01-15T18:00:42.062711+0000","last_became_peered":"2024-01-15T18:00:42.062711+0000","last_unstale":"2024-01-15T18:00:43.067445+0000","last_undegraded":"2024-01-15T18:00:43.067445+0000","last_fullsized":"2024-01-15T18:00:43.067445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:53:36.562001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.064521+0000","last_change":"2024-01-15T18:00:42.073413+0000","last_active":"2024-01-15T18:00:43.064521+0000","last_peered":"2024-01-15T18:00:43.064521+0000","last_clean":"2024-01-15T18:00:43.064521+0000","last_became_active":"2024-01-15T18:00:42.072167+0000","last_became_peered":"2024-01-15T18:00:42.072167+0000","last_unstale":"2024-01-15T18:00:43.064521+0000","last_undegraded":"2024-01-15T18:00:43.064521+0000","last_fullsized":"2024-01-15T18:00:43.064521+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:09.074909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.107553+0000","last_change":"2024-01-15T18:00:42.058452+0000","last_active":"2024-01-15T18:00:43.107553+0000","last_peered":"2024-01-15T18:00:43.107553+0000","last_clean":"2024-01-15T18:00:43.107553+0000","last_became_active":"2024-01-15T18:00:42.057885+0000","last_became_peered":"2024-01-15T18:00:42.057885+0000","last_unstale":"2024-01-15T18:00:43.107553+0000","last_undegraded":"2024-01-15T18:00:43.107553+0000","last_fullsized":"2024-01-15T18:00:43.107553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:34.368288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105339+0000","last_change":"2024-01-15T18:00:42.079987+0000","last_active":"2024-01-15T18:00:43.105339+0000","last_peered":"2024-01-15T18:00:43.105339+0000","last_clean":"2024-01-15T18:00:43.105339+0000","last_became_active":"2024-01-15T18:00:42.079525+0000","last_became_peered":"2024-01-15T18:00:42.079525+0000","last_unstale":"2024-01-15T18:00:43.105339+0000","last_undegraded":"2024-01-15T18:00:43.105339+0000","last_fullsized":"2024-01-15T18:00:43.105339+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:21.834287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105577+0000","last_change":"2024-01-15T18:00:42.078544+0000","last_active":"2024-01-15T18:00:43.105577+0000","last_peered":"2024-01-15T18:00:43.105577+0000","last_clean":"2024-01-15T18:00:43.105577+0000","last_became_active":"2024-01-15T18:00:42.078114+0000","last_became_peered":"2024-01-15T18:00:42.078114+0000","last_unstale":"2024-01-15T18:00:43.105577+0000","last_undegraded":"2024-01-15T18:00:43.105577+0000","last_fullsized":"2024-01-15T18:00:43.105577+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:39:45.755470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106676+0000","last_change":"2024-01-15T18:00:42.080315+0000","last_active":"2024-01-15T18:00:43.106676+0000","last_peered":"2024-01-15T18:00:43.106676+0000","last_clean":"2024-01-15T18:00:43.106676+0000","last_became_active":"2024-01-15T18:00:42.079623+0000","last_became_peered":"2024-01-15T18:00:42.079623+0000","last_unstale":"2024-01-15T18:00:43.106676+0000","last_undegraded":"2024-01-15T18:00:43.106676+0000","last_fullsized":"2024-01-15T18:00:43.106676+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:36.763406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103244+0000","last_change":"2024-01-15T18:00:42.071297+0000","last_active":"2024-01-15T18:00:43.103244+0000","last_peered":"2024-01-15T18:00:43.103244+0000","last_clean":"2024-01-15T18:00:43.103244+0000","last_became_active":"2024-01-15T18:00:42.071046+0000","last_became_peered":"2024-01-15T18:00:42.071046+0000","last_unstale":"2024-01-15T18:00:43.103244+0000","last_undegraded":"2024-01-15T18:00:43.103244+0000","last_fullsized":"2024-01-15T18:00:43.103244+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:08:56.863439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106121+0000","last_change":"2024-01-15T18:00:42.066557+0000","last_active":"2024-01-15T18:00:43.106121+0000","last_peered":"2024-01-15T18:00:43.106121+0000","last_clean":"2024-01-15T18:00:43.106121+0000","last_became_active":"2024-01-15T18:00:42.066269+0000","last_became_peered":"2024-01-15T18:00:42.066269+0000","last_unstale":"2024-01-15T18:00:43.106121+0000","last_undegraded":"2024-01-15T18:00:43.106121+0000","last_fullsized":"2024-01-15T18:00:43.106121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:14:22.776341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103266+0000","last_change":"2024-01-15T18:00:42.071412+0000","last_active":"2024-01-15T18:00:43.103266+0000","last_peered":"2024-01-15T18:00:43.103266+0000","last_clean":"2024-01-15T18:00:43.103266+0000","last_became_active":"2024-01-15T18:00:42.070884+0000","last_became_peered":"2024-01-15T18:00:42.070884+0000","last_unstale":"2024-01-15T18:00:43.103266+0000","last_undegraded":"2024-01-15T18:00:43.103266+0000","last_fullsized":"2024-01-15T18:00:43.103266+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:59.730413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065896+0000","last_change":"2024-01-15T18:00:42.078634+0000","last_active":"2024-01-15T18:00:43.065896+0000","last_peered":"2024-01-15T18:00:43.065896+0000","last_clean":"2024-01-15T18:00:43.065896+0000","last_became_active":"2024-01-15T18:00:42.078165+0000","last_became_peered":"2024-01-15T18:00:42.078165+0000","last_unstale":"2024-01-15T18:00:43.065896+0000","last_undegraded":"2024-01-15T18:00:43.065896+0000","last_fullsized":"2024-01-15T18:00:43.065896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:22.274947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106751+0000","last_change":"2024-01-15T18:00:42.077511+0000","last_active":"2024-01-15T18:00:43.106751+0000","last_peered":"2024-01-15T18:00:43.106751+0000","last_clean":"2024-01-15T18:00:43.106751+0000","last_became_active":"2024-01-15T18:00:42.077238+0000","last_became_peered":"2024-01-15T18:00:42.077238+0000","last_unstale":"2024-01-15T18:00:43.106751+0000","last_undegraded":"2024-01-15T18:00:43.106751+0000","last_fullsized":"2024-01-15T18:00:43.106751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:33:19.250071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104711+0000","last_change":"2024-01-15T18:00:42.076259+0000","last_active":"2024-01-15T18:00:43.104711+0000","last_peered":"2024-01-15T18:00:43.104711+0000","last_clean":"2024-01-15T18:00:43.104711+0000","last_became_active":"2024-01-15T18:00:42.076003+0000","last_became_peered":"2024-01-15T18:00:42.076003+0000","last_unstale":"2024-01-15T18:00:43.104711+0000","last_undegraded":"2024-01-15T18:00:43.104711+0000","last_fullsized":"2024-01-15T18:00:43.104711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:59.730413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065378+0000","last_change":"2024-01-15T18:00:42.072997+0000","last_active":"2024-01-15T18:00:43.065378+0000","last_peered":"2024-01-15T18:00:43.065378+0000","last_clean":"2024-01-15T18:00:43.065378+0000","last_became_active":"2024-01-15T18:00:42.070411+0000","last_became_peered":"2024-01-15T18:00:42.070411+0000","last_unstale":"2024-01-15T18:00:43.065378+0000","last_undegraded":"2024-01-15T18:00:43.065378+0000","last_fullsized":"2024-01-15T18:00:43.065378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:39.286999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105677+0000","last_change":"2024-01-15T18:00:42.069370+0000","last_active":"2024-01-15T18:00:43.105677+0000","last_peered":"2024-01-15T18:00:43.105677+0000","last_clean":"2024-01-15T18:00:43.105677+0000","last_became_active":"2024-01-15T18:00:42.069077+0000","last_became_peered":"2024-01-15T18:00:42.069077+0000","last_unstale":"2024-01-15T18:00:43.105677+0000","last_undegraded":"2024-01-15T18:00:43.105677+0000","last_fullsized":"2024-01-15T18:00:43.105677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:37:31.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106414+0000","last_change":"2024-01-15T18:00:42.067598+0000","last_active":"2024-01-15T18:00:43.106414+0000","last_peered":"2024-01-15T18:00:43.106414+0000","last_clean":"2024-01-15T18:00:43.106414+0000","last_became_active":"2024-01-15T18:00:42.067297+0000","last_became_peered":"2024-01-15T18:00:42.067297+0000","last_unstale":"2024-01-15T18:00:43.106414+0000","last_undegraded":"2024-01-15T18:00:43.106414+0000","last_fullsized":"2024-01-15T18:00:43.106414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:11:42.077219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103249+0000","last_change":"2024-01-15T18:00:42.074513+0000","last_active":"2024-01-15T18:00:43.103249+0000","last_peered":"2024-01-15T18:00:43.103249+0000","last_clean":"2024-01-15T18:00:43.103249+0000","last_became_active":"2024-01-15T18:00:42.074280+0000","last_became_peered":"2024-01-15T18:00:42.074280+0000","last_unstale":"2024-01-15T18:00:43.103249+0000","last_undegraded":"2024-01-15T18:00:43.103249+0000","last_fullsized":"2024-01-15T18:00:43.103249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:17.327716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105227+0000","last_change":"2024-01-15T18:00:42.064236+0000","last_active":"2024-01-15T18:00:43.105227+0000","last_peered":"2024-01-15T18:00:43.105227+0000","last_clean":"2024-01-15T18:00:43.105227+0000","last_became_active":"2024-01-15T18:00:42.063972+0000","last_became_peered":"2024-01-15T18:00:42.063972+0000","last_unstale":"2024-01-15T18:00:43.105227+0000","last_undegraded":"2024-01-15T18:00:43.105227+0000","last_fullsized":"2024-01-15T18:00:43.105227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:51:55.182082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.064609+0000","last_change":"2024-01-15T18:00:42.086123+0000","last_active":"2024-01-15T18:00:43.064609+0000","last_peered":"2024-01-15T18:00:43.064609+0000","last_clean":"2024-01-15T18:00:43.064609+0000","last_became_active":"2024-01-15T18:00:42.085208+0000","last_became_peered":"2024-01-15T18:00:42.085208+0000","last_unstale":"2024-01-15T18:00:43.064609+0000","last_undegraded":"2024-01-15T18:00:43.064609+0000","last_fullsized":"2024-01-15T18:00:43.064609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:17.327716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065154+0000","last_change":"2024-01-15T18:00:42.065563+0000","last_active":"2024-01-15T18:00:43.065154+0000","last_peered":"2024-01-15T18:00:43.065154+0000","last_clean":"2024-01-15T18:00:43.065154+0000","last_became_active":"2024-01-15T18:00:42.065279+0000","last_became_peered":"2024-01-15T18:00:42.065279+0000","last_unstale":"2024-01-15T18:00:43.065154+0000","last_undegraded":"2024-01-15T18:00:43.065154+0000","last_fullsized":"2024-01-15T18:00:43.065154+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:58:21.543745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168562+0000","last_change":"2024-01-15T18:00:41.072898+0000","last_active":"2024-01-15T18:00:42.168562+0000","last_peered":"2024-01-15T18:00:42.168562+0000","last_clean":"2024-01-15T18:00:42.168562+0000","last_became_active":"2024-01-15T18:00:41.071115+0000","last_became_peered":"2024-01-15T18:00:41.071115+0000","last_unstale":"2024-01-15T18:00:42.168562+0000","last_undegraded":"2024-01-15T18:00:42.168562+0000","last_fullsized":"2024-01-15T18:00:42.168562+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104209+0000","last_change":"2024-01-15T18:00:42.060261+0000","last_active":"2024-01-15T18:00:43.104209+0000","last_peered":"2024-01-15T18:00:43.104209+0000","last_clean":"2024-01-15T18:00:43.104209+0000","last_became_active":"2024-01-15T18:00:42.059975+0000","last_became_peered":"2024-01-15T18:00:42.059975+0000","last_unstale":"2024-01-15T18:00:43.104209+0000","last_undegraded":"2024-01-15T18:00:43.104209+0000","last_fullsized":"2024-01-15T18:00:43.104209+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:31:51.180049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130455+0000","last_change":"2024-01-15T18:00:41.067396+0000","last_active":"2024-01-15T18:00:42.130455+0000","last_peered":"2024-01-15T18:00:42.130455+0000","last_clean":"2024-01-15T18:00:42.130455+0000","last_became_active":"2024-01-15T18:00:41.067056+0000","last_became_peered":"2024-01-15T18:00:41.067056+0000","last_unstale":"2024-01-15T18:00:42.130455+0000","last_undegraded":"2024-01-15T18:00:42.130455+0000","last_fullsized":"2024-01-15T18:00:42.130455+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:24.577110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065213+0000","last_change":"2024-01-15T18:00:42.065885+0000","last_active":"2024-01-15T18:00:43.065213+0000","last_peered":"2024-01-15T18:00:43.065213+0000","last_clean":"2024-01-15T18:00:43.065213+0000","last_became_active":"2024-01-15T18:00:42.065563+0000","last_became_peered":"2024-01-15T18:00:42.065563+0000","last_unstale":"2024-01-15T18:00:43.065213+0000","last_undegraded":"2024-01-15T18:00:43.065213+0000","last_fullsized":"2024-01-15T18:00:43.065213+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:25.288201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168495+0000","last_change":"2024-01-15T18:00:41.053014+0000","last_active":"2024-01-15T18:00:42.168495+0000","last_peered":"2024-01-15T18:00:42.168495+0000","last_clean":"2024-01-15T18:00:42.168495+0000","last_became_active":"2024-01-15T18:00:41.052718+0000","last_became_peered":"2024-01-15T18:00:41.052718+0000","last_unstale":"2024-01-15T18:00:42.168495+0000","last_undegraded":"2024-01-15T18:00:42.168495+0000","last_fullsized":"2024-01-15T18:00:42.168495+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065578+0000","last_change":"2024-01-15T18:00:42.077700+0000","last_active":"2024-01-15T18:00:43.065578+0000","last_peered":"2024-01-15T18:00:43.065578+0000","last_clean":"2024-01-15T18:00:43.065578+0000","last_became_active":"2024-01-15T18:00:42.077465+0000","last_became_peered":"2024-01-15T18:00:42.077465+0000","last_unstale":"2024-01-15T18:00:43.065578+0000","last_undegraded":"2024-01-15T18:00:43.065578+0000","last_fullsized":"2024-01-15T18:00:43.065578+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:24:05.366767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085365+0000","last_change":"2024-01-15T18:00:41.057990+0000","last_active":"2024-01-15T18:00:42.085365+0000","last_peered":"2024-01-15T18:00:42.085365+0000","last_clean":"2024-01-15T18:00:42.085365+0000","last_became_active":"2024-01-15T18:00:41.057666+0000","last_became_peered":"2024-01-15T18:00:41.057666+0000","last_unstale":"2024-01-15T18:00:42.085365+0000","last_undegraded":"2024-01-15T18:00:42.085365+0000","last_fullsized":"2024-01-15T18:00:42.085365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106520+0000","last_change":"2024-01-15T18:00:42.066199+0000","last_active":"2024-01-15T18:00:43.106520+0000","last_peered":"2024-01-15T18:00:43.106520+0000","last_clean":"2024-01-15T18:00:43.106520+0000","last_became_active":"2024-01-15T18:00:42.065918+0000","last_became_peered":"2024-01-15T18:00:42.065918+0000","last_unstale":"2024-01-15T18:00:43.106520+0000","last_undegraded":"2024-01-15T18:00:43.106520+0000","last_fullsized":"2024-01-15T18:00:43.106520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:07.672520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082029+0000","last_change":"2024-01-15T18:00:41.063462+0000","last_active":"2024-01-15T18:00:42.082029+0000","last_peered":"2024-01-15T18:00:42.082029+0000","last_clean":"2024-01-15T18:00:42.082029+0000","last_became_active":"2024-01-15T18:00:41.063125+0000","last_became_peered":"2024-01-15T18:00:41.063125+0000","last_unstale":"2024-01-15T18:00:42.082029+0000","last_undegraded":"2024-01-15T18:00:42.082029+0000","last_fullsized":"2024-01-15T18:00:42.082029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:26:23.829753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106167+0000","last_change":"2024-01-15T18:00:42.079309+0000","last_active":"2024-01-15T18:00:43.106167+0000","last_peered":"2024-01-15T18:00:43.106167+0000","last_clean":"2024-01-15T18:00:43.106167+0000","last_became_active":"2024-01-15T18:00:42.078911+0000","last_became_peered":"2024-01-15T18:00:42.078911+0000","last_unstale":"2024-01-15T18:00:43.106167+0000","last_undegraded":"2024-01-15T18:00:43.106167+0000","last_fullsized":"2024-01-15T18:00:43.106167+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:47.720642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168371+0000","last_change":"2024-01-15T18:00:41.069799+0000","last_active":"2024-01-15T18:00:42.168371+0000","last_peered":"2024-01-15T18:00:42.168371+0000","last_clean":"2024-01-15T18:00:42.168371+0000","last_became_active":"2024-01-15T18:00:41.069467+0000","last_became_peered":"2024-01-15T18:00:41.069467+0000","last_unstale":"2024-01-15T18:00:42.168371+0000","last_undegraded":"2024-01-15T18:00:42.168371+0000","last_fullsized":"2024-01-15T18:00:42.168371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.066339+0000","last_change":"2024-01-15T18:00:42.064990+0000","last_active":"2024-01-15T18:00:43.066339+0000","last_peered":"2024-01-15T18:00:43.066339+0000","last_clean":"2024-01-15T18:00:43.066339+0000","last_became_active":"2024-01-15T18:00:42.064698+0000","last_became_peered":"2024-01-15T18:00:42.064698+0000","last_unstale":"2024-01-15T18:00:43.066339+0000","last_undegraded":"2024-01-15T18:00:43.066339+0000","last_fullsized":"2024-01-15T18:00:43.066339+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:31:51.180049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083341+0000","last_change":"2024-01-15T18:00:41.057228+0000","last_active":"2024-01-15T18:00:42.083341+0000","last_peered":"2024-01-15T18:00:42.083341+0000","last_clean":"2024-01-15T18:00:42.083341+0000","last_became_active":"2024-01-15T18:00:41.056929+0000","last_became_peered":"2024-01-15T18:00:41.056929+0000","last_unstale":"2024-01-15T18:00:42.083341+0000","last_undegraded":"2024-01-15T18:00:42.083341+0000","last_fullsized":"2024-01-15T18:00:42.083341+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105935+0000","last_change":"2024-01-15T18:00:42.057749+0000","last_active":"2024-01-15T18:00:43.105935+0000","last_peered":"2024-01-15T18:00:43.105935+0000","last_clean":"2024-01-15T18:00:43.105935+0000","last_became_active":"2024-01-15T18:00:42.054149+0000","last_became_peered":"2024-01-15T18:00:42.054149+0000","last_unstale":"2024-01-15T18:00:43.105935+0000","last_undegraded":"2024-01-15T18:00:43.105935+0000","last_fullsized":"2024-01-15T18:00:43.105935+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:08:28.978781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064484+0000","last_change":"2024-01-15T18:00:41.067706+0000","last_active":"2024-01-15T18:00:42.064484+0000","last_peered":"2024-01-15T18:00:42.064484+0000","last_clean":"2024-01-15T18:00:42.064484+0000","last_became_active":"2024-01-15T18:00:41.067325+0000","last_became_peered":"2024-01-15T18:00:41.067325+0000","last_unstale":"2024-01-15T18:00:42.064484+0000","last_undegraded":"2024-01-15T18:00:42.064484+0000","last_fullsized":"2024-01-15T18:00:42.064484+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:50:53.105108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.068294+0000","last_change":"2024-01-15T18:00:42.063135+0000","last_active":"2024-01-15T18:00:43.068294+0000","last_peered":"2024-01-15T18:00:43.068294+0000","last_clean":"2024-01-15T18:00:43.068294+0000","last_became_active":"2024-01-15T18:00:42.062863+0000","last_became_peered":"2024-01-15T18:00:42.062863+0000","last_unstale":"2024-01-15T18:00:43.068294+0000","last_undegraded":"2024-01-15T18:00:43.068294+0000","last_fullsized":"2024-01-15T18:00:43.068294+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.095223+0000","last_change":"2024-01-15T18:00:41.073770+0000","last_active":"2024-01-15T18:00:42.095223+0000","last_peered":"2024-01-15T18:00:42.095223+0000","last_clean":"2024-01-15T18:00:42.095223+0000","last_became_active":"2024-01-15T18:00:41.073486+0000","last_became_peered":"2024-01-15T18:00:41.073486+0000","last_unstale":"2024-01-15T18:00:42.095223+0000","last_undegraded":"2024-01-15T18:00:42.095223+0000","last_fullsized":"2024-01-15T18:00:42.095223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105140+0000","last_change":"2024-01-15T18:00:42.064514+0000","last_active":"2024-01-15T18:00:43.105140+0000","last_peered":"2024-01-15T18:00:43.105140+0000","last_clean":"2024-01-15T18:00:43.105140+0000","last_became_active":"2024-01-15T18:00:42.064248+0000","last_became_peered":"2024-01-15T18:00:42.064248+0000","last_unstale":"2024-01-15T18:00:43.105140+0000","last_undegraded":"2024-01-15T18:00:43.105140+0000","last_fullsized":"2024-01-15T18:00:43.105140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:39.286999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130465+0000","last_change":"2024-01-15T18:00:41.064452+0000","last_active":"2024-01-15T18:00:42.130465+0000","last_peered":"2024-01-15T18:00:42.130465+0000","last_clean":"2024-01-15T18:00:42.130465+0000","last_became_active":"2024-01-15T18:00:41.064118+0000","last_became_peered":"2024-01-15T18:00:41.064118+0000","last_unstale":"2024-01-15T18:00:42.130465+0000","last_undegraded":"2024-01-15T18:00:42.130465+0000","last_fullsized":"2024-01-15T18:00:42.130465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:31.276292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104433+0000","last_change":"2024-01-15T18:00:42.068233+0000","last_active":"2024-01-15T18:00:43.104433+0000","last_peered":"2024-01-15T18:00:43.104433+0000","last_clean":"2024-01-15T18:00:43.104433+0000","last_became_active":"2024-01-15T18:00:42.066136+0000","last_became_peered":"2024-01-15T18:00:42.066136+0000","last_unstale":"2024-01-15T18:00:43.104433+0000","last_undegraded":"2024-01-15T18:00:43.104433+0000","last_fullsized":"2024-01-15T18:00:43.104433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:06:50.567877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.095182+0000","last_change":"2024-01-15T18:00:41.069415+0000","last_active":"2024-01-15T18:00:42.095182+0000","last_peered":"2024-01-15T18:00:42.095182+0000","last_clean":"2024-01-15T18:00:42.095182+0000","last_became_active":"2024-01-15T18:00:41.069194+0000","last_became_peered":"2024-01-15T18:00:41.069194+0000","last_unstale":"2024-01-15T18:00:42.095182+0000","last_undegraded":"2024-01-15T18:00:42.095182+0000","last_fullsized":"2024-01-15T18:00:42.095182+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:08.061519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105203+0000","last_change":"2024-01-15T18:00:42.072986+0000","last_active":"2024-01-15T18:00:43.105203+0000","last_peered":"2024-01-15T18:00:43.105203+0000","last_clean":"2024-01-15T18:00:43.105203+0000","last_became_active":"2024-01-15T18:00:42.072700+0000","last_became_peered":"2024-01-15T18:00:42.072700+0000","last_unstale":"2024-01-15T18:00:43.105203+0000","last_undegraded":"2024-01-15T18:00:43.105203+0000","last_fullsized":"2024-01-15T18:00:43.105203+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:24:05.366767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.092266+0000","last_change":"2024-01-15T18:00:41.056524+0000","last_active":"2024-01-15T18:00:42.092266+0000","last_peered":"2024-01-15T18:00:42.092266+0000","last_clean":"2024-01-15T18:00:42.092266+0000","last_became_active":"2024-01-15T18:00:41.056086+0000","last_became_peered":"2024-01-15T18:00:41.056086+0000","last_unstale":"2024-01-15T18:00:42.092266+0000","last_undegraded":"2024-01-15T18:00:42.092266+0000","last_fullsized":"2024-01-15T18:00:42.092266+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:36.552797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103073+0000","last_change":"2024-01-15T18:00:42.072006+0000","last_active":"2024-01-15T18:00:43.103073+0000","last_peered":"2024-01-15T18:00:43.103073+0000","last_clean":"2024-01-15T18:00:43.103073+0000","last_became_active":"2024-01-15T18:00:42.071671+0000","last_became_peered":"2024-01-15T18:00:42.071671+0000","last_unstale":"2024-01-15T18:00:43.103073+0000","last_undegraded":"2024-01-15T18:00:43.103073+0000","last_fullsized":"2024-01-15T18:00:43.103073+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:09.074909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130393+0000","last_change":"2024-01-15T18:00:41.067157+0000","last_active":"2024-01-15T18:00:42.130393+0000","last_peered":"2024-01-15T18:00:42.130393+0000","last_clean":"2024-01-15T18:00:42.130393+0000","last_became_active":"2024-01-15T18:00:41.066820+0000","last_became_peered":"2024-01-15T18:00:41.066820+0000","last_unstale":"2024-01-15T18:00:42.130393+0000","last_undegraded":"2024-01-15T18:00:42.130393+0000","last_fullsized":"2024-01-15T18:00:42.130393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:03:48.949979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104038+0000","last_change":"2024-01-15T18:00:42.063066+0000","last_active":"2024-01-15T18:00:43.104038+0000","last_peered":"2024-01-15T18:00:43.104038+0000","last_clean":"2024-01-15T18:00:43.104038+0000","last_became_active":"2024-01-15T18:00:42.062649+0000","last_became_peered":"2024-01-15T18:00:42.062649+0000","last_unstale":"2024-01-15T18:00:43.104038+0000","last_undegraded":"2024-01-15T18:00:43.104038+0000","last_fullsized":"2024-01-15T18:00:43.104038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:58:21.543745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.364757+0000","last_change":"2024-01-15T18:00:41.061164+0000","last_active":"2024-01-15T18:00:42.364757+0000","last_peered":"2024-01-15T18:00:42.364757+0000","last_clean":"2024-01-15T18:00:42.364757+0000","last_became_active":"2024-01-15T18:00:41.060847+0000","last_became_peered":"2024-01-15T18:00:41.060847+0000","last_unstale":"2024-01-15T18:00:42.364757+0000","last_undegraded":"2024-01-15T18:00:42.364757+0000","last_fullsized":"2024-01-15T18:00:42.364757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.067095+0000","last_change":"2024-01-15T18:00:42.071612+0000","last_active":"2024-01-15T18:00:43.067095+0000","last_peered":"2024-01-15T18:00:43.067095+0000","last_clean":"2024-01-15T18:00:43.067095+0000","last_became_active":"2024-01-15T18:00:42.071343+0000","last_became_peered":"2024-01-15T18:00:42.071343+0000","last_unstale":"2024-01-15T18:00:43.067095+0000","last_undegraded":"2024-01-15T18:00:43.067095+0000","last_fullsized":"2024-01-15T18:00:43.067095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:02:30.544639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083367+0000","last_change":"2024-01-15T18:00:41.060084+0000","last_active":"2024-01-15T18:00:42.083367+0000","last_peered":"2024-01-15T18:00:42.083367+0000","last_clean":"2024-01-15T18:00:42.083367+0000","last_became_active":"2024-01-15T18:00:41.057544+0000","last_became_peered":"2024-01-15T18:00:41.057544+0000","last_unstale":"2024-01-15T18:00:42.083367+0000","last_undegraded":"2024-01-15T18:00:42.083367+0000","last_fullsized":"2024-01-15T18:00:42.083367+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103615+0000","last_change":"2024-01-15T18:00:42.071201+0000","last_active":"2024-01-15T18:00:43.103615+0000","last_peered":"2024-01-15T18:00:43.103615+0000","last_clean":"2024-01-15T18:00:43.103615+0000","last_became_active":"2024-01-15T18:00:42.070498+0000","last_became_peered":"2024-01-15T18:00:42.070498+0000","last_unstale":"2024-01-15T18:00:43.103615+0000","last_undegraded":"2024-01-15T18:00:43.103615+0000","last_fullsized":"2024-01-15T18:00:43.103615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:24:18.622665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.095073+0000","last_change":"2024-01-15T18:00:41.064043+0000","last_active":"2024-01-15T18:00:42.095073+0000","last_peered":"2024-01-15T18:00:42.095073+0000","last_clean":"2024-01-15T18:00:42.095073+0000","last_became_active":"2024-01-15T18:00:41.063128+0000","last_became_peered":"2024-01-15T18:00:41.063128+0000","last_unstale":"2024-01-15T18:00:42.095073+0000","last_undegraded":"2024-01-15T18:00:42.095073+0000","last_fullsized":"2024-01-15T18:00:42.095073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:12.496547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105537+0000","last_change":"2024-01-15T18:00:42.079795+0000","last_active":"2024-01-15T18:00:43.105537+0000","last_peered":"2024-01-15T18:00:43.105537+0000","last_clean":"2024-01-15T18:00:43.105537+0000","last_became_active":"2024-01-15T18:00:42.079150+0000","last_became_peered":"2024-01-15T18:00:42.079150+0000","last_unstale":"2024-01-15T18:00:43.105537+0000","last_undegraded":"2024-01-15T18:00:43.105537+0000","last_fullsized":"2024-01-15T18:00:43.105537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:35:00.330346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130486+0000","last_change":"2024-01-15T18:00:41.071443+0000","last_active":"2024-01-15T18:00:42.130486+0000","last_peered":"2024-01-15T18:00:42.130486+0000","last_clean":"2024-01-15T18:00:42.130486+0000","last_became_active":"2024-01-15T18:00:41.071219+0000","last_became_peered":"2024-01-15T18:00:41.071219+0000","last_unstale":"2024-01-15T18:00:42.130486+0000","last_undegraded":"2024-01-15T18:00:42.130486+0000","last_fullsized":"2024-01-15T18:00:42.130486+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:56.788064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105180+0000","last_change":"2024-01-15T18:00:42.079222+0000","last_active":"2024-01-15T18:00:43.105180+0000","last_peered":"2024-01-15T18:00:43.105180+0000","last_clean":"2024-01-15T18:00:43.105180+0000","last_became_active":"2024-01-15T18:00:42.078933+0000","last_became_peered":"2024-01-15T18:00:42.078933+0000","last_unstale":"2024-01-15T18:00:43.105180+0000","last_undegraded":"2024-01-15T18:00:43.105180+0000","last_fullsized":"2024-01-15T18:00:43.105180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:58:21.543745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168527+0000","last_change":"2024-01-15T18:00:41.071043+0000","last_active":"2024-01-15T18:00:42.168527+0000","last_peered":"2024-01-15T18:00:42.168527+0000","last_clean":"2024-01-15T18:00:42.168527+0000","last_became_active":"2024-01-15T18:00:41.070768+0000","last_became_peered":"2024-01-15T18:00:41.070768+0000","last_unstale":"2024-01-15T18:00:42.168527+0000","last_undegraded":"2024-01-15T18:00:42.168527+0000","last_fullsized":"2024-01-15T18:00:42.168527+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106831+0000","last_change":"2024-01-15T18:00:42.076070+0000","last_active":"2024-01-15T18:00:43.106831+0000","last_peered":"2024-01-15T18:00:43.106831+0000","last_clean":"2024-01-15T18:00:43.106831+0000","last_became_active":"2024-01-15T18:00:42.068998+0000","last_became_peered":"2024-01-15T18:00:42.068998+0000","last_unstale":"2024-01-15T18:00:43.106831+0000","last_undegraded":"2024-01-15T18:00:43.106831+0000","last_fullsized":"2024-01-15T18:00:43.106831+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:48:33.487610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085417+0000","last_change":"2024-01-15T18:00:41.061292+0000","last_active":"2024-01-15T18:00:42.085417+0000","last_peered":"2024-01-15T18:00:42.085417+0000","last_clean":"2024-01-15T18:00:42.085417+0000","last_became_active":"2024-01-15T18:00:41.061006+0000","last_became_peered":"2024-01-15T18:00:41.061006+0000","last_unstale":"2024-01-15T18:00:42.085417+0000","last_undegraded":"2024-01-15T18:00:42.085417+0000","last_fullsized":"2024-01-15T18:00:42.085417+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105474+0000","last_change":"2024-01-15T18:00:42.067416+0000","last_active":"2024-01-15T18:00:43.105474+0000","last_peered":"2024-01-15T18:00:43.105474+0000","last_clean":"2024-01-15T18:00:43.105474+0000","last_became_active":"2024-01-15T18:00:42.067129+0000","last_became_peered":"2024-01-15T18:00:42.067129+0000","last_unstale":"2024-01-15T18:00:43.105474+0000","last_undegraded":"2024-01-15T18:00:43.105474+0000","last_fullsized":"2024-01-15T18:00:43.105474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:35:40.538621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168424+0000","last_change":"2024-01-15T18:00:41.071456+0000","last_active":"2024-01-15T18:00:42.168424+0000","last_peered":"2024-01-15T18:00:42.168424+0000","last_clean":"2024-01-15T18:00:42.168424+0000","last_became_active":"2024-01-15T18:00:41.071189+0000","last_became_peered":"2024-01-15T18:00:41.071189+0000","last_unstale":"2024-01-15T18:00:42.168424+0000","last_undegraded":"2024-01-15T18:00:42.168424+0000","last_fullsized":"2024-01-15T18:00:42.168424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103570+0000","last_change":"2024-01-15T18:00:42.065297+0000","last_active":"2024-01-15T18:00:43.103570+0000","last_peered":"2024-01-15T18:00:43.103570+0000","last_clean":"2024-01-15T18:00:43.103570+0000","last_became_active":"2024-01-15T18:00:42.064969+0000","last_became_peered":"2024-01-15T18:00:42.064969+0000","last_unstale":"2024-01-15T18:00:43.103570+0000","last_undegraded":"2024-01-15T18:00:43.103570+0000","last_fullsized":"2024-01-15T18:00:43.103570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:17.327716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130413+0000","last_change":"2024-01-15T18:00:41.068101+0000","last_active":"2024-01-15T18:00:42.130413+0000","last_peered":"2024-01-15T18:00:42.130413+0000","last_clean":"2024-01-15T18:00:42.130413+0000","last_became_active":"2024-01-15T18:00:41.065928+0000","last_became_peered":"2024-01-15T18:00:41.065928+0000","last_unstale":"2024-01-15T18:00:42.130413+0000","last_undegraded":"2024-01-15T18:00:42.130413+0000","last_fullsized":"2024-01-15T18:00:42.130413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:19:23.525421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.064980+0000","last_change":"2024-01-15T18:00:42.067592+0000","last_active":"2024-01-15T18:00:43.064980+0000","last_peered":"2024-01-15T18:00:43.064980+0000","last_clean":"2024-01-15T18:00:43.064980+0000","last_became_active":"2024-01-15T18:00:42.067309+0000","last_became_peered":"2024-01-15T18:00:42.067309+0000","last_unstale":"2024-01-15T18:00:43.064980+0000","last_undegraded":"2024-01-15T18:00:43.064980+0000","last_fullsized":"2024-01-15T18:00:43.064980+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:24:18.622665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069730+0000","last_change":"2024-01-15T18:00:41.050813+0000","last_active":"2024-01-15T18:00:42.069730+0000","last_peered":"2024-01-15T18:00:42.069730+0000","last_clean":"2024-01-15T18:00:42.069730+0000","last_became_active":"2024-01-15T18:00:41.050320+0000","last_became_peered":"2024-01-15T18:00:41.050320+0000","last_unstale":"2024-01-15T18:00:42.069730+0000","last_undegraded":"2024-01-15T18:00:42.069730+0000","last_fullsized":"2024-01-15T18:00:42.069730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065150+0000","last_change":"2024-01-15T18:00:42.070090+0000","last_active":"2024-01-15T18:00:43.065150+0000","last_peered":"2024-01-15T18:00:43.065150+0000","last_clean":"2024-01-15T18:00:43.065150+0000","last_became_active":"2024-01-15T18:00:42.068897+0000","last_became_peered":"2024-01-15T18:00:42.068897+0000","last_unstale":"2024-01-15T18:00:43.065150+0000","last_undegraded":"2024-01-15T18:00:43.065150+0000","last_fullsized":"2024-01-15T18:00:43.065150+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:08:56.863439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083292+0000","last_change":"2024-01-15T18:00:41.065251+0000","last_active":"2024-01-15T18:00:42.083292+0000","last_peered":"2024-01-15T18:00:42.083292+0000","last_clean":"2024-01-15T18:00:42.083292+0000","last_became_active":"2024-01-15T18:00:41.065006+0000","last_became_peered":"2024-01-15T18:00:41.065006+0000","last_unstale":"2024-01-15T18:00:42.083292+0000","last_undegraded":"2024-01-15T18:00:42.083292+0000","last_fullsized":"2024-01-15T18:00:42.083292+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105082+0000","last_change":"2024-01-15T18:00:42.083131+0000","last_active":"2024-01-15T18:00:43.105082+0000","last_peered":"2024-01-15T18:00:43.105082+0000","last_clean":"2024-01-15T18:00:43.105082+0000","last_became_active":"2024-01-15T18:00:42.082546+0000","last_became_peered":"2024-01-15T18:00:42.082546+0000","last_unstale":"2024-01-15T18:00:43.105082+0000","last_undegraded":"2024-01-15T18:00:43.105082+0000","last_fullsized":"2024-01-15T18:00:43.105082+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:08:56.863439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.087697+0000","last_change":"2024-01-15T18:00:41.066193+0000","last_active":"2024-01-15T18:00:42.087697+0000","last_peered":"2024-01-15T18:00:42.087697+0000","last_clean":"2024-01-15T18:00:42.087697+0000","last_became_active":"2024-01-15T18:00:41.065913+0000","last_became_peered":"2024-01-15T18:00:41.065913+0000","last_unstale":"2024-01-15T18:00:42.087697+0000","last_undegraded":"2024-01-15T18:00:42.087697+0000","last_fullsized":"2024-01-15T18:00:42.087697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105504+0000","last_change":"2024-01-15T18:00:42.076020+0000","last_active":"2024-01-15T18:00:43.105504+0000","last_peered":"2024-01-15T18:00:43.105504+0000","last_clean":"2024-01-15T18:00:43.105504+0000","last_became_active":"2024-01-15T18:00:42.075751+0000","last_became_peered":"2024-01-15T18:00:42.075751+0000","last_unstale":"2024-01-15T18:00:43.105504+0000","last_undegraded":"2024-01-15T18:00:43.105504+0000","last_fullsized":"2024-01-15T18:00:43.105504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:39.286999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065784+0000","last_change":"2024-01-15T18:00:41.070458+0000","last_active":"2024-01-15T18:00:43.065784+0000","last_peered":"2024-01-15T18:00:43.065784+0000","last_clean":"2024-01-15T18:00:43.065784+0000","last_became_active":"2024-01-15T18:00:41.066322+0000","last_became_peered":"2024-01-15T18:00:41.066322+0000","last_unstale":"2024-01-15T18:00:43.065784+0000","last_undegraded":"2024-01-15T18:00:43.065784+0000","last_fullsized":"2024-01-15T18:00:43.065784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104807+0000","last_change":"2024-01-15T18:00:42.066727+0000","last_active":"2024-01-15T18:00:43.104807+0000","last_peered":"2024-01-15T18:00:43.104807+0000","last_clean":"2024-01-15T18:00:43.104807+0000","last_became_active":"2024-01-15T18:00:42.065788+0000","last_became_peered":"2024-01-15T18:00:42.065788+0000","last_unstale":"2024-01-15T18:00:43.104807+0000","last_undegraded":"2024-01-15T18:00:43.104807+0000","last_fullsized":"2024-01-15T18:00:43.104807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:16:20.555824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.364842+0000","last_change":"2024-01-15T18:00:41.063919+0000","last_active":"2024-01-15T18:00:42.364842+0000","last_peered":"2024-01-15T18:00:42.364842+0000","last_clean":"2024-01-15T18:00:42.364842+0000","last_became_active":"2024-01-15T18:00:41.062115+0000","last_became_peered":"2024-01-15T18:00:41.062115+0000","last_unstale":"2024-01-15T18:00:42.364842+0000","last_undegraded":"2024-01-15T18:00:42.364842+0000","last_fullsized":"2024-01-15T18:00:42.364842+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065646+0000","last_change":"2024-01-15T18:00:42.073589+0000","last_active":"2024-01-15T18:00:43.065646+0000","last_peered":"2024-01-15T18:00:43.065646+0000","last_clean":"2024-01-15T18:00:43.065646+0000","last_became_active":"2024-01-15T18:00:42.073254+0000","last_became_peered":"2024-01-15T18:00:42.073254+0000","last_unstale":"2024-01-15T18:00:43.065646+0000","last_undegraded":"2024-01-15T18:00:43.065646+0000","last_fullsized":"2024-01-15T18:00:43.065646+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:21.834287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083478+0000","last_change":"2024-01-15T18:00:41.057399+0000","last_active":"2024-01-15T18:00:42.083478+0000","last_peered":"2024-01-15T18:00:42.083478+0000","last_clean":"2024-01-15T18:00:42.083478+0000","last_became_active":"2024-01-15T18:00:41.056174+0000","last_became_peered":"2024-01-15T18:00:41.056174+0000","last_unstale":"2024-01-15T18:00:42.083478+0000","last_undegraded":"2024-01-15T18:00:42.083478+0000","last_fullsized":"2024-01-15T18:00:42.083478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106964+0000","last_change":"2024-01-15T18:00:42.052005+0000","last_active":"2024-01-15T18:00:43.106964+0000","last_peered":"2024-01-15T18:00:43.106964+0000","last_clean":"2024-01-15T18:00:43.106964+0000","last_became_active":"2024-01-15T18:00:42.051122+0000","last_became_peered":"2024-01-15T18:00:42.051122+0000","last_unstale":"2024-01-15T18:00:43.106964+0000","last_undegraded":"2024-01-15T18:00:43.106964+0000","last_fullsized":"2024-01-15T18:00:43.106964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:22.659547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069747+0000","last_change":"2024-01-15T18:00:41.064076+0000","last_active":"2024-01-15T18:00:42.069747+0000","last_peered":"2024-01-15T18:00:42.069747+0000","last_clean":"2024-01-15T18:00:42.069747+0000","last_became_active":"2024-01-15T18:00:41.062980+0000","last_became_peered":"2024-01-15T18:00:41.062980+0000","last_unstale":"2024-01-15T18:00:42.069747+0000","last_undegraded":"2024-01-15T18:00:42.069747+0000","last_fullsized":"2024-01-15T18:00:42.069747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:41:42.728224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104591+0000","last_change":"2024-01-15T18:00:42.070329+0000","last_active":"2024-01-15T18:00:43.104591+0000","last_peered":"2024-01-15T18:00:43.104591+0000","last_clean":"2024-01-15T18:00:43.104591+0000","last_became_active":"2024-01-15T18:00:42.068798+0000","last_became_peered":"2024-01-15T18:00:42.068798+0000","last_unstale":"2024-01-15T18:00:43.104591+0000","last_undegraded":"2024-01-15T18:00:43.104591+0000","last_fullsized":"2024-01-15T18:00:43.104591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:55.705369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064582+0000","last_change":"2024-01-15T18:00:41.054271+0000","last_active":"2024-01-15T18:00:42.064582+0000","last_peered":"2024-01-15T18:00:42.064582+0000","last_clean":"2024-01-15T18:00:42.064582+0000","last_became_active":"2024-01-15T18:00:41.053925+0000","last_became_peered":"2024-01-15T18:00:41.053925+0000","last_unstale":"2024-01-15T18:00:42.064582+0000","last_undegraded":"2024-01-15T18:00:42.064582+0000","last_fullsized":"2024-01-15T18:00:42.064582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:18:46.672266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103301+0000","last_change":"2024-01-15T18:00:42.057721+0000","last_active":"2024-01-15T18:00:43.103301+0000","last_peered":"2024-01-15T18:00:43.103301+0000","last_clean":"2024-01-15T18:00:43.103301+0000","last_became_active":"2024-01-15T18:00:42.057091+0000","last_became_peered":"2024-01-15T18:00:42.057091+0000","last_unstale":"2024-01-15T18:00:43.103301+0000","last_undegraded":"2024-01-15T18:00:43.103301+0000","last_fullsized":"2024-01-15T18:00:43.103301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:03.318096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069705+0000","last_change":"2024-01-15T18:00:41.061944+0000","last_active":"2024-01-15T18:00:42.069705+0000","last_peered":"2024-01-15T18:00:42.069705+0000","last_clean":"2024-01-15T18:00:42.069705+0000","last_became_active":"2024-01-15T18:00:41.061646+0000","last_became_peered":"2024-01-15T18:00:41.061646+0000","last_unstale":"2024-01-15T18:00:42.069705+0000","last_undegraded":"2024-01-15T18:00:42.069705+0000","last_fullsized":"2024-01-15T18:00:42.069705+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:23:57.016225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.107069+0000","last_change":"2024-01-15T18:00:42.055286+0000","last_active":"2024-01-15T18:00:43.107069+0000","last_peered":"2024-01-15T18:00:43.107069+0000","last_clean":"2024-01-15T18:00:43.107069+0000","last_became_active":"2024-01-15T18:00:42.054746+0000","last_became_peered":"2024-01-15T18:00:42.054746+0000","last_unstale":"2024-01-15T18:00:43.107069+0000","last_undegraded":"2024-01-15T18:00:43.107069+0000","last_fullsized":"2024-01-15T18:00:43.107069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:59.371652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130314+0000","last_change":"2024-01-15T18:00:41.068287+0000","last_active":"2024-01-15T18:00:42.130314+0000","last_peered":"2024-01-15T18:00:42.130314+0000","last_clean":"2024-01-15T18:00:42.130314+0000","last_became_active":"2024-01-15T18:00:41.066120+0000","last_became_peered":"2024-01-15T18:00:41.066120+0000","last_unstale":"2024-01-15T18:00:42.130314+0000","last_undegraded":"2024-01-15T18:00:42.130314+0000","last_fullsized":"2024-01-15T18:00:42.130314+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:39:48.165240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103877+0000","last_change":"2024-01-15T18:00:42.080052+0000","last_active":"2024-01-15T18:00:43.103877+0000","last_peered":"2024-01-15T18:00:43.103877+0000","last_clean":"2024-01-15T18:00:43.103877+0000","last_became_active":"2024-01-15T18:00:42.079699+0000","last_became_peered":"2024-01-15T18:00:42.079699+0000","last_unstale":"2024-01-15T18:00:43.103877+0000","last_undegraded":"2024-01-15T18:00:43.103877+0000","last_fullsized":"2024-01-15T18:00:43.103877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:51:28.238550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.364734+0000","last_change":"2024-01-15T18:00:41.061168+0000","last_active":"2024-01-15T18:00:42.364734+0000","last_peered":"2024-01-15T18:00:42.364734+0000","last_clean":"2024-01-15T18:00:42.364734+0000","last_became_active":"2024-01-15T18:00:41.060793+0000","last_became_peered":"2024-01-15T18:00:41.060793+0000","last_unstale":"2024-01-15T18:00:42.364734+0000","last_undegraded":"2024-01-15T18:00:42.364734+0000","last_fullsized":"2024-01-15T18:00:42.364734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:02.304705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":196,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.050457+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:00:42.050457+0000","last_peered":"2024-01-15T18:00:42.050457+0000","last_clean":"2024-01-15T18:00:42.050457+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:00:42.050457+0000","last_undegraded":"2024-01-15T18:00:42.050457+0000","last_fullsized":"2024-01-15T18:00:42.050457+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542217,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1318912,"data_stored":890846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.2230000000000001,"15min":1.177},"min":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":5.556,"5min":5.556,"15min":5.556},"last":0.70599999999999996},{"interface":"front","average":{"1min":3.444,"5min":1.649,"15min":1.3260000000000001},"min":{"1min":0.747,"5min":0.623,"15min":0.623},"max":{"1min":38.137,"5min":38.137,"15min":38.137},"last":1.5309999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.266,"15min":1.25},"min":{"1min":0.72999999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":4.194,"5min":1.861,"15min":1.4570000000000001},"min":{"1min":0.72699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":51.843000000000004,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.3979999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.921,"5min":1.361,"15min":1.2270000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":9.3369999999999997,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.643,"5min":1.397,"15min":1.393},"min":{"1min":0.66000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.2889999999999997,"5min":6.2889999999999997,"15min":6.2889999999999997},"last":1.1439999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":1.49,"15min":1.363},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":9.1440000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.462},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5029999999999999,"15min":1.4970000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":1.367}]},{"osd":4,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":3.3300000000000001,"5min":1.788,"15min":1.5980000000000001},"min":{"1min":0.95799999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":22.359999999999999,"5min":22.359999999999999,"15min":22.359999999999999},"last":1.645},{"interface":"front","average":{"1min":2.2130000000000001,"5min":1.597,"15min":1.496},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":12.667999999999999,"5min":12.667999999999999,"15min":12.667999999999999},"last":1.2450000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.0220000000000002,"5min":2.0569999999999999,"15min":1.7569999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":25.006,"5min":25.006,"15min":25.006},"last":1.5129999999999999},{"interface":"front","average":{"1min":3.4950000000000001,"5min":1.9239999999999999,"15min":1.712},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":22.693999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.734}]},{"osd":6,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.9530000000000003,"5min":2.2770000000000001,"15min":1.901},"min":{"1min":1.0580000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":23.834,"5min":23.834,"15min":23.834},"last":1.8120000000000001},{"interface":"front","average":{"1min":5.9169999999999998,"5min":2.4729999999999999,"15min":1.9199999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":63.932000000000002,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.5780000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542216,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36044,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335796,"statfs":{"total":96636764160,"available":96599855104,"internally_reserved":0,"allocated":716800,"data_stored":300318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.413,"15min":1.3169999999999999},"min":{"1min":0.71699999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.8769999999999998,"5min":12.544,"15min":12.544},"last":1.2110000000000001},{"interface":"front","average":{"1min":2.1840000000000002,"5min":1.423,"15min":1.248},"min":{"1min":0.81899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":13.98,"5min":13.98,"15min":13.98},"last":0.88100000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.357,"15min":1.2250000000000001},"min":{"1min":0.82599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.5659999999999998,"5min":12.811999999999999,"15min":12.811999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":2.0600000000000001,"5min":1.349,"15min":1.2330000000000001},"min":{"1min":0.86499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":8.8130000000000006,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.048}]},{"osd":2,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2970000000000002,"5min":1.456,"15min":1.28},"min":{"1min":0.90300000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4890000000000001,"15min":1.5009999999999999},"min":{"1min":0.86899999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":4.0419999999999998,"5min":12.897,"15min":12.897},"last":1.4710000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.988,"5min":1.7010000000000001,"15min":1.542},"min":{"1min":1.0880000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":9.5,"5min":12.717000000000001,"15min":12.717000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":2.52,"5min":1.5840000000000001,"15min":1.4410000000000001},"min":{"1min":0.93799999999999994,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":14.066000000000001,"5min":14.066000000000001,"15min":14.066000000000001},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5009999999999999,"15min":1.454},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":9.2829999999999995,"5min":9.2829999999999995,"15min":9.2829999999999995},"last":1.163},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.6359999999999999,"15min":1.607},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.6989999999999998,"5min":16.98,"15min":16.98},"last":1.415}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":1.548,"15min":1.484},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":14.176,"5min":14.176,"15min":14.176},"last":0.98599999999999999},{"interface":"front","average":{"1min":2.3079999999999998,"5min":1.756,"15min":1.7569999999999999},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"last":1.391}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":1.6399999999999999,"15min":1.601},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":14.109999999999999,"5min":14.109999999999999,"15min":14.109999999999999},"last":1.087},{"interface":"front","average":{"1min":2.5870000000000002,"5min":1.8009999999999999,"15min":1.7769999999999999},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":14.478,"5min":14.478,"15min":14.478},"last":1.335}]}]},{"osd":5,"up_from":14,"seq":60129542216,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":724992,"data_stored":300970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3149999999999999,"15min":1.2509999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.7490000000000001,"5min":16.178000000000001,"15min":16.178000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":2.4980000000000002,"5min":1.4750000000000001,"15min":1.417},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":17.184999999999999,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.1539999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2929999999999999,"15min":1.2250000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":5.6219999999999999,"5min":5.6219999999999999,"15min":5.6219999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.2629999999999999,"15min":1.2110000000000001},"min":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.1379999999999999,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":1.26}]},{"osd":2,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2050000000000001,"15min":1.2190000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.681,"5min":2.835,"15min":2.835},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.367,"15min":1.288},"min":{"1min":1.0760000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.9740000000000002,"5min":5.9740000000000002,"15min":5.9740000000000002},"last":1.522}]},{"osd":3,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.3520000000000001},"min":{"1min":0.73099999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8029999999999999,"5min":14.914,"15min":14.914},"last":1.121},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.365,"15min":1.413},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.726,"5min":3.302,"15min":3.302},"last":1.5609999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.395,"15min":1.411},"min":{"1min":0.99299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.7759999999999998,"5min":4.7759999999999998,"15min":4.7759999999999998},"last":1.609},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.75,"15min":1.571},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":17.991,"5min":17.991,"15min":17.991},"last":1.4790000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.506,"15min":1.6020000000000001},"min":{"1min":0.91900000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":6.4630000000000001,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.792,"5min":1.5369999999999999,"15min":1.55},"min":{"1min":0.88100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":7.7759999999999998,"5min":7.7759999999999998,"15min":7.7759999999999998},"last":1.6930000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5580000000000001,"15min":1.6060000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":6.3230000000000004,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.603,"15min":1.728},"min":{"1min":0.90500000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.2030000000000003,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.4370000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542216,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":729088,"data_stored":301930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.9989999999999997,"5min":11.413,"15min":11.413},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.4350000000000001,"15min":1.3799999999999999},"min":{"1min":0.77500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":9.7439999999999998,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":1.494}]},{"osd":1,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.4670000000000001,"15min":1.3819999999999999},"min":{"1min":0.84299999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":9.7010000000000005,"5min":11.987,"15min":11.987},"last":1.268},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.27,"15min":1.323},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":6.149,"5min":6.149,"15min":6.149},"last":1.107}]},{"osd":2,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.708,"15min":1.5700000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":7.032,"5min":13.865,"15min":13.865},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.514,"15min":1.3959999999999999},"min":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.028,"5min":13.903,"15min":13.903},"last":1.329}]},{"osd":3,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.6040000000000001,"15min":1.5920000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":6.8819999999999997,"5min":11.611000000000001,"15min":11.611000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":2.0800000000000001,"5min":1.546,"15min":1.6180000000000001},"min":{"1min":0.69199999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":13.25,"5min":13.25,"15min":13.25},"last":1.159}]},{"osd":5,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.6359999999999999,"15min":1.6850000000000001},"min":{"1min":0.61899999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":6.6820000000000004,"5min":11.561,"15min":11.561},"last":1.532},{"interface":"front","average":{"1min":1.8999999999999999,"5min":1.5509999999999999,"15min":1.665},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":9.6229999999999993,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.4370000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.468,"15min":1.6259999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.698,"5min":1.8500000000000001,"15min":1.8460000000000001},"min":{"1min":0.85799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":6.9329999999999998,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.6559999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":2.359,"5min":1.8979999999999999,"15min":1.9059999999999999},"min":{"1min":1.375,"5min":0.503,"15min":0.503},"max":{"1min":10.157999999999999,"5min":11.894,"15min":11.894},"last":1.585},{"interface":"front","average":{"1min":2.5169999999999999,"5min":2.048,"15min":1.968},"min":{"1min":1.325,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":8.4169999999999998,"5min":11.714,"15min":11.714},"last":1.748}]}]},{"osd":3,"up_from":14,"seq":60129542218,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44368,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327472,"statfs":{"total":96636764160,"available":96591331328,"internally_reserved":0,"allocated":724992,"data_stored":300374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":1.1990000000000001,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.435,"15min":0.435},"max":{"1min":9.9789999999999992,"5min":9.9789999999999992,"15min":9.9789999999999992},"last":0.76000000000000001},{"interface":"front","average":{"1min":2.9420000000000002,"5min":1.5149999999999999,"15min":1.218},"min":{"1min":0.85199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":21.047000000000001,"5min":21.047000000000001,"15min":21.047000000000001},"last":0.93000000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.8919999999999999,"5min":1.524,"15min":1.179},"min":{"1min":0.79700000000000004,"5min":0.375,"15min":0.375},"max":{"1min":17.777999999999999,"5min":17.777999999999999,"15min":17.777999999999999},"last":1.087},{"interface":"front","average":{"1min":1.786,"5min":1.2210000000000001,"15min":1.135},"min":{"1min":0.69299999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":9.2360000000000007,"5min":9.2360000000000007,"15min":9.2360000000000007},"last":0.86399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.27,"15min":1.1899999999999999},"min":{"1min":0.79000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.4930000000000003,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":2.0510000000000002,"5min":1.492,"15min":1.4119999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":9.3040000000000003,"5min":9.3040000000000003,"15min":9.3040000000000003},"last":0.71299999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":1.51,"15min":1.4199999999999999},"min":{"1min":1.1120000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":1.1299999999999999},{"interface":"front","average":{"1min":2.367,"5min":1.643,"15min":1.4890000000000001},"min":{"1min":0.92300000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":10.313000000000001,"5min":10.313000000000001,"15min":10.313000000000001},"last":1.4370000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.2229999999999999,"5min":1.635,"15min":1.5649999999999999},"min":{"1min":1.046,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.243,"5min":10.243,"15min":10.243},"last":1.492},{"interface":"front","average":{"1min":2.2330000000000001,"5min":1.742,"15min":1.6599999999999999},"min":{"1min":0.90500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":6.2000000000000002,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.2210000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.5510000000000002,"5min":1.851,"15min":1.7250000000000001},"min":{"1min":0.98399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":10.012,"5min":10.012,"15min":10.012},"last":1.25},{"interface":"front","average":{"1min":2.6190000000000002,"5min":1.8220000000000001,"15min":1.722},"min":{"1min":1.0840000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":10.291,"5min":10.291,"15min":10.291},"last":1.335}]},{"osd":7,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.802,"5min":1.8819999999999999,"15min":1.7589999999999999},"min":{"1min":1.02,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":12.627000000000001,"5min":12.627000000000001,"15min":12.627000000000001},"last":1.284},{"interface":"front","average":{"1min":2.9710000000000001,"5min":1.946,"15min":1.7929999999999999},"min":{"1min":1.1240000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":10.268000000000001,"5min":10.268000000000001,"15min":10.268000000000001},"last":1.1819999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":724992,"data_stored":300374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.042},"min":{"1min":0.48999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.466,"5min":14.427,"15min":14.427},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.0860000000000001,"15min":1.02},"min":{"1min":0.64100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3129999999999999,"15min":1.2949999999999999},"min":{"1min":0.53000000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7490000000000001,"5min":14.609,"15min":14.609},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.1830000000000001,"15min":1.2010000000000001},"min":{"1min":0.748,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5059999999999998,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.2569999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2609999999999999,"15min":1.1990000000000001},"min":{"1min":0.64700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.772,"5min":16.567,"15min":16.567},"last":1.734},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.48,"15min":1.4630000000000001},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8180000000000001,"5min":14.528,"15min":14.528},"last":1.631}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.708,"15min":1.647},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.7690000000000001,"5min":14.779,"15min":14.779},"last":1.5049999999999999},{"interface":"front","average":{"1min":2.073,"5min":1.546,"15min":1.502},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":10.75,"5min":10.75,"15min":10.75},"last":1.0820000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.5609999999999999,"15min":1.506},"min":{"1min":0.85199999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.258,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.653,"5min":1.726,"15min":1.659},"min":{"1min":1,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.4750000000000001,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.7849999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.889,"5min":1.5580000000000001,"15min":1.5149999999999999},"min":{"1min":1.131,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.6799999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.6339999999999999,"15min":1.5},"min":{"1min":1.171,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.415,"5min":17.07,"15min":17.07},"last":1.294}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.839,"5min":1.621,"15min":1.6259999999999999},"min":{"1min":1.333,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.2170000000000001,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.4530000000000001},{"interface":"front","average":{"1min":2.9049999999999998,"5min":2.1070000000000002,"15min":1.7070000000000001},"min":{"1min":1.2070000000000001,"5min":0.752,"15min":0.752},"max":{"1min":15.135,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.552}]}]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40212,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331628,"statfs":{"total":96636764160,"available":96595587072,"internally_reserved":0,"allocated":720896,"data_stored":300408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.095,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":8.1140000000000008,"5min":8.1140000000000008,"15min":8.1140000000000008},"last":0.876},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0169999999999999,"15min":0.99299999999999999},"min":{"1min":0.64000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.456,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":1.5600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.4289999999999998,"5min":1.3109999999999999,"15min":0.99399999999999999},"min":{"1min":0.51000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":21.126000000000001,"5min":21.126000000000001,"15min":21.126000000000001},"last":1.014},{"interface":"front","average":{"1min":2.427,"5min":1.5309999999999999,"15min":1.254},"min":{"1min":0.62,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":11.5,"5min":11.5,"15min":11.5},"last":1.605}]},{"osd":3,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.573,"15min":2.0110000000000001},"min":{"1min":0.76400000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":8.3209999999999997,"5min":22.106999999999999,"15min":22.106999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.4239999999999999,"15min":1.244},"min":{"1min":0.58699999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6060000000000001,"5min":15.510999999999999,"15min":15.510999999999999},"last":0.93100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.841,"5min":1.6419999999999999,"15min":1.74},"min":{"1min":0.88800000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":6.3109999999999999,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.236},{"interface":"front","average":{"1min":1.734,"5min":1.5800000000000001,"15min":1.863},"min":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.8239999999999998,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.506}]},{"osd":5,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0179999999999998,"5min":1.6339999999999999,"15min":1.8600000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":11.552,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.851,"5min":1.9159999999999999,"15min":1.706},"min":{"1min":0.85399999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":8.2319999999999993,"5min":20.823,"15min":20.823},"last":1.714}]},{"osd":6,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.282,"5min":1.7749999999999999,"15min":2.0369999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":10.696,"5min":14.272,"15min":14.272},"last":1.8380000000000001},{"interface":"front","average":{"1min":2.1150000000000002,"5min":1.637,"15min":1.5309999999999999},"min":{"1min":1.0629999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":8.8320000000000007,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.649}]},{"osd":7,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.6459999999999999,"15min":1.575},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":8.2769999999999992,"5min":8.2769999999999992,"15min":8.2769999999999992},"last":1.806},{"interface":"front","average":{"1min":3.2530000000000001,"5min":1.881,"15min":1.665},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":24.289999999999999,"5min":24.289999999999999,"15min":24.289999999999999},"last":1.776}]}]},{"osd":0,"up_from":14,"seq":60129542218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40800,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331040,"statfs":{"total":96636764160,"available":96594984960,"internally_reserved":0,"allocated":1327104,"data_stored":892880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.92100000000000004,"15min":0.88500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.1559999999999999,"15min":1.1000000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.2709999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.012},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.0009999999999999,"15min":0.88},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7490000000000001,"5min":2.129,"15min":2.129},"last":0.80700000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.8440000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.381},{"interface":"front","average":{"1min":1.792,"5min":1.3049999999999999,"15min":1.123},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":14.74,"5min":14.74,"15min":14.74},"last":1.0049999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8109999999999999,"5min":1.4179999999999999,"15min":1.3080000000000001},"min":{"1min":1.004,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":8.6959999999999997,"5min":8.6959999999999997,"15min":8.6959999999999997},"last":1.3100000000000001},{"interface":"front","average":{"1min":2.3540000000000001,"5min":1.6839999999999999,"15min":1.5549999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":15.975,"5min":15.975,"15min":15.975},"last":1.5429999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":1.502,"15min":1.373},"min":{"1min":1.0860000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":3.133,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.1639999999999999},{"interface":"front","average":{"1min":2.1000000000000001,"5min":1.571,"15min":1.5129999999999999},"min":{"1min":1.0669999999999999,"5min":0.754,"15min":0.754},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.835}]},{"osd":6,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.524},"min":{"1min":0.96999999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":2.0190000000000001},{"interface":"front","average":{"1min":1.732,"5min":1.526,"15min":1.462},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":1.4399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0409999999999999,"5min":1.71,"15min":1.6639999999999999},"min":{"1min":1.214,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":2.1970000000000001,"5min":1.7090000000000001,"15min":1.593},"min":{"1min":1.012,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":6.2619999999999996,"5min":6.2619999999999996,"15min":6.2619999999999996},"last":1.954}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:00:51.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:00:51.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.877+0000 7fc56b200640 1 Processor -- start 2024-01-15T18:00:51.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.877+0000 7fc56b200640 1 -- start start 2024-01-15T18:00:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.879+0000 7fc56b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5640cc500 0x7fc5640cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.879+0000 7fc56b200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc56406d160 0x7fc56406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.879+0000 7fc56b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc564070e50 con 0x7fc564143c40 2024-01-15T18:00:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.879+0000 7fc56b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5640723c0 con 0x7fc5640cc500 2024-01-15T18:00:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.879+0000 7fc56b200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5640cced0 con 0x7fc56406d160 2024-01-15T18:00:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.879+0000 7fc569400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5640cc500 0x7fc5640cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:51.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.879+0000 7fc569400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc5640cc500 0x7fc5640cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37058/0 (socket says 172.21.15.26:37058) 2024-01-15T18:00:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.879+0000 7fc569400640 1 -- 172.21.15.26:0/285740989 learned_addr learned my addr 172.21.15.26:0/285740989 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.880+0000 7fc56a800640 1 --2- 172.21.15.26:0/285740989 >> v2:172.21.15.124:3300/0 conn(0x7fc56406d160 0x7fc56406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.880+0000 7fc569400640 1 -- 172.21.15.26:0/285740989 >> v2:172.21.15.124:3300/0 conn(0x7fc56406d160 msgr2=0x7fc56406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.880+0000 7fc569400640 1 --2- 172.21.15.26:0/285740989 >> v2:172.21.15.124:3300/0 conn(0x7fc56406d160 0x7fc56406d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.880+0000 7fc569400640 1 -- 172.21.15.26:0/285740989 >> v1:172.21.15.124:6789/0 conn(0x7fc564143c40 legacy=0x7fc564071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:51.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.880+0000 7fc569400640 1 -- 172.21.15.26:0/285740989 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc564144020 con 0x7fc5640cc500 2024-01-15T18:00:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.881+0000 7fc569400640 1 --2- 172.21.15.26:0/285740989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5640cc500 0x7fc5640cc990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc564070690 tx=0x7fc54c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21aa31bbd2d900a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:51.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.881+0000 7fc568a00640 1 -- 172.21.15.26:0/285740989 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c031800 con 0x7fc5640cc500 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.881+0000 7fc568a00640 1 -- 172.21.15.26:0/285740989 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c031960 con 0x7fc5640cc500 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.881+0000 7fc568a00640 1 -- 172.21.15.26:0/285740989 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c031cf0 con 0x7fc5640cc500 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.882+0000 7fc56b200640 1 -- 172.21.15.26:0/285740989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5640cc500 msgr2=0x7fc5640cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.882+0000 7fc56b200640 1 --2- 172.21.15.26:0/285740989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5640cc500 0x7fc5640cc990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fc564070690 tx=0x7fc54c02b8a0 comp rx=0 tx=0).stop 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.882+0000 7fc56b200640 1 -- 172.21.15.26:0/285740989 shutdown_connections 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.882+0000 7fc56b200640 1 --2- 172.21.15.26:0/285740989 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5640cc500 0x7fc5640cc990 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.882+0000 7fc56b200640 1 --2- 172.21.15.26:0/285740989 >> v2:172.21.15.124:3300/0 conn(0x7fc56406d160 0x7fc56406d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.882+0000 7fc56b200640 1 -- 172.21.15.26:0/285740989 >> 172.21.15.26:0/285740989 conn(0x7fc56406a800 msgr2=0x7fc56406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.882+0000 7fc56b200640 1 -- 172.21.15.26:0/285740989 shutdown_connections 2024-01-15T18:00:51.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.882+0000 7fc56b200640 1 -- 172.21.15.26:0/285740989 wait complete. 2024-01-15T18:00:51.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.884+0000 7fc56b200640 1 Processor -- start 2024-01-15T18:00:51.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.884+0000 7fc56b200640 1 -- start start 2024-01-15T18:00:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc56b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56406d160 0x7fc564138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc56b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5640cc500 0x7fc564132670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:51.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc56b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc564143c40 0x7fc564142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc56b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5641391c0 con 0x7fc56406d160 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc56b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5640b2bb0 con 0x7fc564143c40 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc56b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc564132bb0 con 0x7fc5640cc500 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc569e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56406d160 0x7fc564138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc569e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56406d160 0x7fc564138c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37068/0 (socket says 172.21.15.26:37068) 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc569e00640 1 -- 172.21.15.26:0/1125368436 learned_addr learned my addr 172.21.15.26:0/1125368436 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc56a800640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc564143c40 0x7fc564142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc569e00640 1 -- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5640cc500 msgr2=0x7fc564132670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc569e00640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5640cc500 0x7fc564132670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc569e00640 1 -- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc564143c40 msgr2=0x7fc564142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc569e00640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc564143c40 0x7fc564142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:51.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc569e00640 1 -- 172.21.15.26:0/1125368436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c0094d0 con 0x7fc56406d160 2024-01-15T18:00:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.886+0000 7fc56a800640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc564143c40 0x7fc564142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.887+0000 7fc569e00640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56406d160 0x7fc564138c80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc55400b3f0 tx=0x7fc55400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.887+0000 7fc562000640 1 -- 172.21.15.26:0/1125368436 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554011830 con 0x7fc56406d160 2024-01-15T18:00:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.887+0000 7fc562000640 1 -- 172.21.15.26:0/1125368436 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc554011990 con 0x7fc56406d160 2024-01-15T18:00:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.887+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5640b25a0 con 0x7fc56406d160 2024-01-15T18:00:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.887+0000 7fc562000640 1 -- 172.21.15.26:0/1125368436 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc554011d40 con 0x7fc56406d160 2024-01-15T18:00:51.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.887+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc564156860 con 0x7fc56406d160 2024-01-15T18:00:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.888+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc564067450 con 0x7fc56406d160 2024-01-15T18:00:51.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.890+0000 7fc562000640 1 -- 172.21.15.26:0/1125368436 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fc55401a400 con 0x7fc56406d160 2024-01-15T18:00:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.890+0000 7fc562000640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53c06b860 0x7fc53c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.891+0000 7fc562000640 1 -- 172.21.15.26:0/1125368436 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc55408a480 con 0x7fc56406d160 2024-01-15T18:00:51.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.891+0000 7fc569400640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53c06b860 0x7fc53c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:51.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.892+0000 7fc569400640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53c06b860 0x7fc53c06dd20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc54c036000 tx=0x7fc54c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:51.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:51.895+0000 7fc562000640 1 -- 172.21.15.26:0/1125368436 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc554054ab0 con 0x7fc56406d160 2024-01-15T18:00:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.086+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc56406c960 con 0x7fc53c06b860 2024-01-15T18:00:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.110+0000 7fc562000640 1 -- 172.21.15.26:0/1125368436 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286632 (secure 0 0 0) 0x7fc54c031bd0 con 0x7fc53c06b860 2024-01-15T18:00:52.118 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:00:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.115+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53c06b860 msgr2=0x7fc53c06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53c06b860 0x7fc53c06dd20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc54c036000 tx=0x7fc54c030000 comp rx=0 tx=0).stop 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56406d160 msgr2=0x7fc564138c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56406d160 0x7fc564138c80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc55400b3f0 tx=0x7fc55400b8b0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 shutdown_connections 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5640cc500 0x7fc564132670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53c06b860 0x7fc53c06dd20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc564143c40 0x7fc564142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 --2- 172.21.15.26:0/1125368436 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56406d160 0x7fc564138c80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 >> 172.21.15.26:0/1125368436 conn(0x7fc56406a800 msgr2=0x7fc564135200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 shutdown_connections 2024-01-15T18:00:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.116+0000 7fc56b200640 1 -- 172.21.15.26:0/1125368436 wait complete. 2024-01-15T18:00:52.141 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-01-15T18:00:52.036663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":310372,"kb_used_data":6824,"kb_used_omap":12,"kb_used_meta":303411,"kb_avail":754664348,"statfs":{"total":773094113280,"available":772776292352,"internally_reserved":0,"allocated":6987776,"data_stored":3588100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":310693424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001192"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106217+0000","last_change":"2024-01-15T18:00:42.075316+0000","last_active":"2024-01-15T18:00:43.106217+0000","last_peered":"2024-01-15T18:00:43.106217+0000","last_clean":"2024-01-15T18:00:43.106217+0000","last_became_active":"2024-01-15T18:00:42.075016+0000","last_became_peered":"2024-01-15T18:00:42.075016+0000","last_unstale":"2024-01-15T18:00:43.106217+0000","last_undegraded":"2024-01-15T18:00:43.106217+0000","last_fullsized":"2024-01-15T18:00:43.106217+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:36.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106550+0000","last_change":"2024-01-15T18:00:42.060637+0000","last_active":"2024-01-15T18:00:43.106550+0000","last_peered":"2024-01-15T18:00:43.106550+0000","last_clean":"2024-01-15T18:00:43.106550+0000","last_became_active":"2024-01-15T18:00:42.060376+0000","last_became_peered":"2024-01-15T18:00:42.060376+0000","last_unstale":"2024-01-15T18:00:43.106550+0000","last_undegraded":"2024-01-15T18:00:43.106550+0000","last_fullsized":"2024-01-15T18:00:43.106550+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:39:13.052279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104921+0000","last_change":"2024-01-15T18:00:42.077438+0000","last_active":"2024-01-15T18:00:43.104921+0000","last_peered":"2024-01-15T18:00:43.104921+0000","last_clean":"2024-01-15T18:00:43.104921+0000","last_became_active":"2024-01-15T18:00:42.077041+0000","last_became_peered":"2024-01-15T18:00:42.077041+0000","last_unstale":"2024-01-15T18:00:43.104921+0000","last_undegraded":"2024-01-15T18:00:43.104921+0000","last_fullsized":"2024-01-15T18:00:43.104921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:25.288201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105004+0000","last_change":"2024-01-15T18:00:42.067513+0000","last_active":"2024-01-15T18:00:43.105004+0000","last_peered":"2024-01-15T18:00:43.105004+0000","last_clean":"2024-01-15T18:00:43.105004+0000","last_became_active":"2024-01-15T18:00:42.066627+0000","last_became_peered":"2024-01-15T18:00:42.066627+0000","last_unstale":"2024-01-15T18:00:43.105004+0000","last_undegraded":"2024-01-15T18:00:43.105004+0000","last_fullsized":"2024-01-15T18:00:43.105004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:24:18.622665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105449+0000","last_change":"2024-01-15T18:00:42.062431+0000","last_active":"2024-01-15T18:00:43.105449+0000","last_peered":"2024-01-15T18:00:43.105449+0000","last_clean":"2024-01-15T18:00:43.105449+0000","last_became_active":"2024-01-15T18:00:42.062153+0000","last_became_peered":"2024-01-15T18:00:42.062153+0000","last_unstale":"2024-01-15T18:00:43.105449+0000","last_undegraded":"2024-01-15T18:00:43.105449+0000","last_fullsized":"2024-01-15T18:00:43.105449+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:03:23.463624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.064844+0000","last_change":"2024-01-15T18:00:42.067558+0000","last_active":"2024-01-15T18:00:43.064844+0000","last_peered":"2024-01-15T18:00:43.064844+0000","last_clean":"2024-01-15T18:00:43.064844+0000","last_became_active":"2024-01-15T18:00:42.067182+0000","last_became_peered":"2024-01-15T18:00:42.067182+0000","last_unstale":"2024-01-15T18:00:43.064844+0000","last_undegraded":"2024-01-15T18:00:43.064844+0000","last_fullsized":"2024-01-15T18:00:43.064844+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:59.730413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.067178+0000","last_change":"2024-01-15T18:00:42.064214+0000","last_active":"2024-01-15T18:00:43.067178+0000","last_peered":"2024-01-15T18:00:43.067178+0000","last_clean":"2024-01-15T18:00:43.067178+0000","last_became_active":"2024-01-15T18:00:42.063930+0000","last_became_peered":"2024-01-15T18:00:42.063930+0000","last_unstale":"2024-01-15T18:00:43.067178+0000","last_undegraded":"2024-01-15T18:00:43.067178+0000","last_fullsized":"2024-01-15T18:00:43.067178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:29.053165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103502+0000","last_change":"2024-01-15T18:00:42.072265+0000","last_active":"2024-01-15T18:00:43.103502+0000","last_peered":"2024-01-15T18:00:43.103502+0000","last_clean":"2024-01-15T18:00:43.103502+0000","last_became_active":"2024-01-15T18:00:42.071980+0000","last_became_peered":"2024-01-15T18:00:42.071980+0000","last_unstale":"2024-01-15T18:00:43.103502+0000","last_undegraded":"2024-01-15T18:00:43.103502+0000","last_fullsized":"2024-01-15T18:00:43.103502+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:54:53.751752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.068181+0000","last_change":"2024-01-15T18:00:42.081161+0000","last_active":"2024-01-15T18:00:43.068181+0000","last_peered":"2024-01-15T18:00:43.068181+0000","last_clean":"2024-01-15T18:00:43.068181+0000","last_became_active":"2024-01-15T18:00:42.080853+0000","last_became_peered":"2024-01-15T18:00:42.080853+0000","last_unstale":"2024-01-15T18:00:43.068181+0000","last_undegraded":"2024-01-15T18:00:43.068181+0000","last_fullsized":"2024-01-15T18:00:43.068181+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:20.013981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103753+0000","last_change":"2024-01-15T18:00:42.055688+0000","last_active":"2024-01-15T18:00:43.103753+0000","last_peered":"2024-01-15T18:00:43.103753+0000","last_clean":"2024-01-15T18:00:43.103753+0000","last_became_active":"2024-01-15T18:00:42.055302+0000","last_became_peered":"2024-01-15T18:00:42.055302+0000","last_unstale":"2024-01-15T18:00:43.103753+0000","last_undegraded":"2024-01-15T18:00:43.103753+0000","last_fullsized":"2024-01-15T18:00:43.103753+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:22.274947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104110+0000","last_change":"2024-01-15T18:00:42.066509+0000","last_active":"2024-01-15T18:00:43.104110+0000","last_peered":"2024-01-15T18:00:43.104110+0000","last_clean":"2024-01-15T18:00:43.104110+0000","last_became_active":"2024-01-15T18:00:42.066232+0000","last_became_peered":"2024-01-15T18:00:42.066232+0000","last_unstale":"2024-01-15T18:00:43.104110+0000","last_undegraded":"2024-01-15T18:00:43.104110+0000","last_fullsized":"2024-01-15T18:00:43.104110+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:16.633951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105338+0000","last_change":"2024-01-15T18:00:42.068083+0000","last_active":"2024-01-15T18:00:43.105338+0000","last_peered":"2024-01-15T18:00:43.105338+0000","last_clean":"2024-01-15T18:00:43.105338+0000","last_became_active":"2024-01-15T18:00:42.067826+0000","last_became_peered":"2024-01-15T18:00:42.067826+0000","last_unstale":"2024-01-15T18:00:43.105338+0000","last_undegraded":"2024-01-15T18:00:43.105338+0000","last_fullsized":"2024-01-15T18:00:43.105338+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:25.288201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.066253+0000","last_change":"2024-01-15T18:00:42.080948+0000","last_active":"2024-01-15T18:00:43.066253+0000","last_peered":"2024-01-15T18:00:43.066253+0000","last_clean":"2024-01-15T18:00:43.066253+0000","last_became_active":"2024-01-15T18:00:42.079522+0000","last_became_peered":"2024-01-15T18:00:42.079522+0000","last_unstale":"2024-01-15T18:00:43.066253+0000","last_undegraded":"2024-01-15T18:00:43.066253+0000","last_fullsized":"2024-01-15T18:00:43.066253+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:16.633951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.067883+0000","last_change":"2024-01-15T18:00:42.069509+0000","last_active":"2024-01-15T18:00:43.067883+0000","last_peered":"2024-01-15T18:00:43.067883+0000","last_clean":"2024-01-15T18:00:43.067883+0000","last_became_active":"2024-01-15T18:00:42.068025+0000","last_became_peered":"2024-01-15T18:00:42.068025+0000","last_unstale":"2024-01-15T18:00:43.067883+0000","last_undegraded":"2024-01-15T18:00:43.067883+0000","last_fullsized":"2024-01-15T18:00:43.067883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:35.279988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.067445+0000","last_change":"2024-01-15T18:00:42.063338+0000","last_active":"2024-01-15T18:00:43.067445+0000","last_peered":"2024-01-15T18:00:43.067445+0000","last_clean":"2024-01-15T18:00:43.067445+0000","last_became_active":"2024-01-15T18:00:42.062711+0000","last_became_peered":"2024-01-15T18:00:42.062711+0000","last_unstale":"2024-01-15T18:00:43.067445+0000","last_undegraded":"2024-01-15T18:00:43.067445+0000","last_fullsized":"2024-01-15T18:00:43.067445+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:53:36.562001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.064521+0000","last_change":"2024-01-15T18:00:42.073413+0000","last_active":"2024-01-15T18:00:43.064521+0000","last_peered":"2024-01-15T18:00:43.064521+0000","last_clean":"2024-01-15T18:00:43.064521+0000","last_became_active":"2024-01-15T18:00:42.072167+0000","last_became_peered":"2024-01-15T18:00:42.072167+0000","last_unstale":"2024-01-15T18:00:43.064521+0000","last_undegraded":"2024-01-15T18:00:43.064521+0000","last_fullsized":"2024-01-15T18:00:43.064521+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:09.074909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.107553+0000","last_change":"2024-01-15T18:00:42.058452+0000","last_active":"2024-01-15T18:00:43.107553+0000","last_peered":"2024-01-15T18:00:43.107553+0000","last_clean":"2024-01-15T18:00:43.107553+0000","last_became_active":"2024-01-15T18:00:42.057885+0000","last_became_peered":"2024-01-15T18:00:42.057885+0000","last_unstale":"2024-01-15T18:00:43.107553+0000","last_undegraded":"2024-01-15T18:00:43.107553+0000","last_fullsized":"2024-01-15T18:00:43.107553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:50:34.368288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105339+0000","last_change":"2024-01-15T18:00:42.079987+0000","last_active":"2024-01-15T18:00:43.105339+0000","last_peered":"2024-01-15T18:00:43.105339+0000","last_clean":"2024-01-15T18:00:43.105339+0000","last_became_active":"2024-01-15T18:00:42.079525+0000","last_became_peered":"2024-01-15T18:00:42.079525+0000","last_unstale":"2024-01-15T18:00:43.105339+0000","last_undegraded":"2024-01-15T18:00:43.105339+0000","last_fullsized":"2024-01-15T18:00:43.105339+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:21.834287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105577+0000","last_change":"2024-01-15T18:00:42.078544+0000","last_active":"2024-01-15T18:00:43.105577+0000","last_peered":"2024-01-15T18:00:43.105577+0000","last_clean":"2024-01-15T18:00:43.105577+0000","last_became_active":"2024-01-15T18:00:42.078114+0000","last_became_peered":"2024-01-15T18:00:42.078114+0000","last_unstale":"2024-01-15T18:00:43.105577+0000","last_undegraded":"2024-01-15T18:00:43.105577+0000","last_fullsized":"2024-01-15T18:00:43.105577+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:39:45.755470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106676+0000","last_change":"2024-01-15T18:00:42.080315+0000","last_active":"2024-01-15T18:00:43.106676+0000","last_peered":"2024-01-15T18:00:43.106676+0000","last_clean":"2024-01-15T18:00:43.106676+0000","last_became_active":"2024-01-15T18:00:42.079623+0000","last_became_peered":"2024-01-15T18:00:42.079623+0000","last_unstale":"2024-01-15T18:00:43.106676+0000","last_undegraded":"2024-01-15T18:00:43.106676+0000","last_fullsized":"2024-01-15T18:00:43.106676+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:36.763406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103244+0000","last_change":"2024-01-15T18:00:42.071297+0000","last_active":"2024-01-15T18:00:43.103244+0000","last_peered":"2024-01-15T18:00:43.103244+0000","last_clean":"2024-01-15T18:00:43.103244+0000","last_became_active":"2024-01-15T18:00:42.071046+0000","last_became_peered":"2024-01-15T18:00:42.071046+0000","last_unstale":"2024-01-15T18:00:43.103244+0000","last_undegraded":"2024-01-15T18:00:43.103244+0000","last_fullsized":"2024-01-15T18:00:43.103244+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:08:56.863439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106121+0000","last_change":"2024-01-15T18:00:42.066557+0000","last_active":"2024-01-15T18:00:43.106121+0000","last_peered":"2024-01-15T18:00:43.106121+0000","last_clean":"2024-01-15T18:00:43.106121+0000","last_became_active":"2024-01-15T18:00:42.066269+0000","last_became_peered":"2024-01-15T18:00:42.066269+0000","last_unstale":"2024-01-15T18:00:43.106121+0000","last_undegraded":"2024-01-15T18:00:43.106121+0000","last_fullsized":"2024-01-15T18:00:43.106121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:14:22.776341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103266+0000","last_change":"2024-01-15T18:00:42.071412+0000","last_active":"2024-01-15T18:00:43.103266+0000","last_peered":"2024-01-15T18:00:43.103266+0000","last_clean":"2024-01-15T18:00:43.103266+0000","last_became_active":"2024-01-15T18:00:42.070884+0000","last_became_peered":"2024-01-15T18:00:42.070884+0000","last_unstale":"2024-01-15T18:00:43.103266+0000","last_undegraded":"2024-01-15T18:00:43.103266+0000","last_fullsized":"2024-01-15T18:00:43.103266+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:59.730413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065896+0000","last_change":"2024-01-15T18:00:42.078634+0000","last_active":"2024-01-15T18:00:43.065896+0000","last_peered":"2024-01-15T18:00:43.065896+0000","last_clean":"2024-01-15T18:00:43.065896+0000","last_became_active":"2024-01-15T18:00:42.078165+0000","last_became_peered":"2024-01-15T18:00:42.078165+0000","last_unstale":"2024-01-15T18:00:43.065896+0000","last_undegraded":"2024-01-15T18:00:43.065896+0000","last_fullsized":"2024-01-15T18:00:43.065896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:22.274947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106751+0000","last_change":"2024-01-15T18:00:42.077511+0000","last_active":"2024-01-15T18:00:43.106751+0000","last_peered":"2024-01-15T18:00:43.106751+0000","last_clean":"2024-01-15T18:00:43.106751+0000","last_became_active":"2024-01-15T18:00:42.077238+0000","last_became_peered":"2024-01-15T18:00:42.077238+0000","last_unstale":"2024-01-15T18:00:43.106751+0000","last_undegraded":"2024-01-15T18:00:43.106751+0000","last_fullsized":"2024-01-15T18:00:43.106751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:33:19.250071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104711+0000","last_change":"2024-01-15T18:00:42.076259+0000","last_active":"2024-01-15T18:00:43.104711+0000","last_peered":"2024-01-15T18:00:43.104711+0000","last_clean":"2024-01-15T18:00:43.104711+0000","last_became_active":"2024-01-15T18:00:42.076003+0000","last_became_peered":"2024-01-15T18:00:42.076003+0000","last_unstale":"2024-01-15T18:00:43.104711+0000","last_undegraded":"2024-01-15T18:00:43.104711+0000","last_fullsized":"2024-01-15T18:00:43.104711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:33:59.730413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065378+0000","last_change":"2024-01-15T18:00:42.072997+0000","last_active":"2024-01-15T18:00:43.065378+0000","last_peered":"2024-01-15T18:00:43.065378+0000","last_clean":"2024-01-15T18:00:43.065378+0000","last_became_active":"2024-01-15T18:00:42.070411+0000","last_became_peered":"2024-01-15T18:00:42.070411+0000","last_unstale":"2024-01-15T18:00:43.065378+0000","last_undegraded":"2024-01-15T18:00:43.065378+0000","last_fullsized":"2024-01-15T18:00:43.065378+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:39.286999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105677+0000","last_change":"2024-01-15T18:00:42.069370+0000","last_active":"2024-01-15T18:00:43.105677+0000","last_peered":"2024-01-15T18:00:43.105677+0000","last_clean":"2024-01-15T18:00:43.105677+0000","last_became_active":"2024-01-15T18:00:42.069077+0000","last_became_peered":"2024-01-15T18:00:42.069077+0000","last_unstale":"2024-01-15T18:00:43.105677+0000","last_undegraded":"2024-01-15T18:00:43.105677+0000","last_fullsized":"2024-01-15T18:00:43.105677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:37:31.980917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106414+0000","last_change":"2024-01-15T18:00:42.067598+0000","last_active":"2024-01-15T18:00:43.106414+0000","last_peered":"2024-01-15T18:00:43.106414+0000","last_clean":"2024-01-15T18:00:43.106414+0000","last_became_active":"2024-01-15T18:00:42.067297+0000","last_became_peered":"2024-01-15T18:00:42.067297+0000","last_unstale":"2024-01-15T18:00:43.106414+0000","last_undegraded":"2024-01-15T18:00:43.106414+0000","last_fullsized":"2024-01-15T18:00:43.106414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:11:42.077219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103249+0000","last_change":"2024-01-15T18:00:42.074513+0000","last_active":"2024-01-15T18:00:43.103249+0000","last_peered":"2024-01-15T18:00:43.103249+0000","last_clean":"2024-01-15T18:00:43.103249+0000","last_became_active":"2024-01-15T18:00:42.074280+0000","last_became_peered":"2024-01-15T18:00:42.074280+0000","last_unstale":"2024-01-15T18:00:43.103249+0000","last_undegraded":"2024-01-15T18:00:43.103249+0000","last_fullsized":"2024-01-15T18:00:43.103249+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:17.327716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105227+0000","last_change":"2024-01-15T18:00:42.064236+0000","last_active":"2024-01-15T18:00:43.105227+0000","last_peered":"2024-01-15T18:00:43.105227+0000","last_clean":"2024-01-15T18:00:43.105227+0000","last_became_active":"2024-01-15T18:00:42.063972+0000","last_became_peered":"2024-01-15T18:00:42.063972+0000","last_unstale":"2024-01-15T18:00:43.105227+0000","last_undegraded":"2024-01-15T18:00:43.105227+0000","last_fullsized":"2024-01-15T18:00:43.105227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:51:55.182082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.064609+0000","last_change":"2024-01-15T18:00:42.086123+0000","last_active":"2024-01-15T18:00:43.064609+0000","last_peered":"2024-01-15T18:00:43.064609+0000","last_clean":"2024-01-15T18:00:43.064609+0000","last_became_active":"2024-01-15T18:00:42.085208+0000","last_became_peered":"2024-01-15T18:00:42.085208+0000","last_unstale":"2024-01-15T18:00:43.064609+0000","last_undegraded":"2024-01-15T18:00:43.064609+0000","last_fullsized":"2024-01-15T18:00:43.064609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:17.327716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065154+0000","last_change":"2024-01-15T18:00:42.065563+0000","last_active":"2024-01-15T18:00:43.065154+0000","last_peered":"2024-01-15T18:00:43.065154+0000","last_clean":"2024-01-15T18:00:43.065154+0000","last_became_active":"2024-01-15T18:00:42.065279+0000","last_became_peered":"2024-01-15T18:00:42.065279+0000","last_unstale":"2024-01-15T18:00:43.065154+0000","last_undegraded":"2024-01-15T18:00:43.065154+0000","last_fullsized":"2024-01-15T18:00:43.065154+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:58:21.543745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":16,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168562+0000","last_change":"2024-01-15T18:00:41.072898+0000","last_active":"2024-01-15T18:00:42.168562+0000","last_peered":"2024-01-15T18:00:42.168562+0000","last_clean":"2024-01-15T18:00:42.168562+0000","last_became_active":"2024-01-15T18:00:41.071115+0000","last_became_peered":"2024-01-15T18:00:41.071115+0000","last_unstale":"2024-01-15T18:00:42.168562+0000","last_undegraded":"2024-01-15T18:00:42.168562+0000","last_fullsized":"2024-01-15T18:00:42.168562+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104209+0000","last_change":"2024-01-15T18:00:42.060261+0000","last_active":"2024-01-15T18:00:43.104209+0000","last_peered":"2024-01-15T18:00:43.104209+0000","last_clean":"2024-01-15T18:00:43.104209+0000","last_became_active":"2024-01-15T18:00:42.059975+0000","last_became_peered":"2024-01-15T18:00:42.059975+0000","last_unstale":"2024-01-15T18:00:43.104209+0000","last_undegraded":"2024-01-15T18:00:43.104209+0000","last_fullsized":"2024-01-15T18:00:43.104209+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:31:51.180049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130455+0000","last_change":"2024-01-15T18:00:41.067396+0000","last_active":"2024-01-15T18:00:42.130455+0000","last_peered":"2024-01-15T18:00:42.130455+0000","last_clean":"2024-01-15T18:00:42.130455+0000","last_became_active":"2024-01-15T18:00:41.067056+0000","last_became_peered":"2024-01-15T18:00:41.067056+0000","last_unstale":"2024-01-15T18:00:42.130455+0000","last_undegraded":"2024-01-15T18:00:42.130455+0000","last_fullsized":"2024-01-15T18:00:42.130455+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:13:24.577110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065213+0000","last_change":"2024-01-15T18:00:42.065885+0000","last_active":"2024-01-15T18:00:43.065213+0000","last_peered":"2024-01-15T18:00:43.065213+0000","last_clean":"2024-01-15T18:00:43.065213+0000","last_became_active":"2024-01-15T18:00:42.065563+0000","last_became_peered":"2024-01-15T18:00:42.065563+0000","last_unstale":"2024-01-15T18:00:43.065213+0000","last_undegraded":"2024-01-15T18:00:43.065213+0000","last_fullsized":"2024-01-15T18:00:43.065213+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:25.288201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168495+0000","last_change":"2024-01-15T18:00:41.053014+0000","last_active":"2024-01-15T18:00:42.168495+0000","last_peered":"2024-01-15T18:00:42.168495+0000","last_clean":"2024-01-15T18:00:42.168495+0000","last_became_active":"2024-01-15T18:00:41.052718+0000","last_became_peered":"2024-01-15T18:00:41.052718+0000","last_unstale":"2024-01-15T18:00:42.168495+0000","last_undegraded":"2024-01-15T18:00:42.168495+0000","last_fullsized":"2024-01-15T18:00:42.168495+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065578+0000","last_change":"2024-01-15T18:00:42.077700+0000","last_active":"2024-01-15T18:00:43.065578+0000","last_peered":"2024-01-15T18:00:43.065578+0000","last_clean":"2024-01-15T18:00:43.065578+0000","last_became_active":"2024-01-15T18:00:42.077465+0000","last_became_peered":"2024-01-15T18:00:42.077465+0000","last_unstale":"2024-01-15T18:00:43.065578+0000","last_undegraded":"2024-01-15T18:00:43.065578+0000","last_fullsized":"2024-01-15T18:00:43.065578+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:24:05.366767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085365+0000","last_change":"2024-01-15T18:00:41.057990+0000","last_active":"2024-01-15T18:00:42.085365+0000","last_peered":"2024-01-15T18:00:42.085365+0000","last_clean":"2024-01-15T18:00:42.085365+0000","last_became_active":"2024-01-15T18:00:41.057666+0000","last_became_peered":"2024-01-15T18:00:41.057666+0000","last_unstale":"2024-01-15T18:00:42.085365+0000","last_undegraded":"2024-01-15T18:00:42.085365+0000","last_fullsized":"2024-01-15T18:00:42.085365+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106520+0000","last_change":"2024-01-15T18:00:42.066199+0000","last_active":"2024-01-15T18:00:43.106520+0000","last_peered":"2024-01-15T18:00:43.106520+0000","last_clean":"2024-01-15T18:00:43.106520+0000","last_became_active":"2024-01-15T18:00:42.065918+0000","last_became_peered":"2024-01-15T18:00:42.065918+0000","last_unstale":"2024-01-15T18:00:43.106520+0000","last_undegraded":"2024-01-15T18:00:43.106520+0000","last_fullsized":"2024-01-15T18:00:43.106520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:07.672520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.082029+0000","last_change":"2024-01-15T18:00:41.063462+0000","last_active":"2024-01-15T18:00:42.082029+0000","last_peered":"2024-01-15T18:00:42.082029+0000","last_clean":"2024-01-15T18:00:42.082029+0000","last_became_active":"2024-01-15T18:00:41.063125+0000","last_became_peered":"2024-01-15T18:00:41.063125+0000","last_unstale":"2024-01-15T18:00:42.082029+0000","last_undegraded":"2024-01-15T18:00:42.082029+0000","last_fullsized":"2024-01-15T18:00:42.082029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:26:23.829753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106167+0000","last_change":"2024-01-15T18:00:42.079309+0000","last_active":"2024-01-15T18:00:43.106167+0000","last_peered":"2024-01-15T18:00:43.106167+0000","last_clean":"2024-01-15T18:00:43.106167+0000","last_became_active":"2024-01-15T18:00:42.078911+0000","last_became_peered":"2024-01-15T18:00:42.078911+0000","last_unstale":"2024-01-15T18:00:43.106167+0000","last_undegraded":"2024-01-15T18:00:43.106167+0000","last_fullsized":"2024-01-15T18:00:43.106167+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:30:47.720642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168371+0000","last_change":"2024-01-15T18:00:41.069799+0000","last_active":"2024-01-15T18:00:42.168371+0000","last_peered":"2024-01-15T18:00:42.168371+0000","last_clean":"2024-01-15T18:00:42.168371+0000","last_became_active":"2024-01-15T18:00:41.069467+0000","last_became_peered":"2024-01-15T18:00:41.069467+0000","last_unstale":"2024-01-15T18:00:42.168371+0000","last_undegraded":"2024-01-15T18:00:42.168371+0000","last_fullsized":"2024-01-15T18:00:42.168371+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.066339+0000","last_change":"2024-01-15T18:00:42.064990+0000","last_active":"2024-01-15T18:00:43.066339+0000","last_peered":"2024-01-15T18:00:43.066339+0000","last_clean":"2024-01-15T18:00:43.066339+0000","last_became_active":"2024-01-15T18:00:42.064698+0000","last_became_peered":"2024-01-15T18:00:42.064698+0000","last_unstale":"2024-01-15T18:00:43.066339+0000","last_undegraded":"2024-01-15T18:00:43.066339+0000","last_fullsized":"2024-01-15T18:00:43.066339+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:31:51.180049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083341+0000","last_change":"2024-01-15T18:00:41.057228+0000","last_active":"2024-01-15T18:00:42.083341+0000","last_peered":"2024-01-15T18:00:42.083341+0000","last_clean":"2024-01-15T18:00:42.083341+0000","last_became_active":"2024-01-15T18:00:41.056929+0000","last_became_peered":"2024-01-15T18:00:41.056929+0000","last_unstale":"2024-01-15T18:00:42.083341+0000","last_undegraded":"2024-01-15T18:00:42.083341+0000","last_fullsized":"2024-01-15T18:00:42.083341+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:28:47.133150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105935+0000","last_change":"2024-01-15T18:00:42.057749+0000","last_active":"2024-01-15T18:00:43.105935+0000","last_peered":"2024-01-15T18:00:43.105935+0000","last_clean":"2024-01-15T18:00:43.105935+0000","last_became_active":"2024-01-15T18:00:42.054149+0000","last_became_peered":"2024-01-15T18:00:42.054149+0000","last_unstale":"2024-01-15T18:00:43.105935+0000","last_undegraded":"2024-01-15T18:00:43.105935+0000","last_fullsized":"2024-01-15T18:00:43.105935+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:08:28.978781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064484+0000","last_change":"2024-01-15T18:00:41.067706+0000","last_active":"2024-01-15T18:00:42.064484+0000","last_peered":"2024-01-15T18:00:42.064484+0000","last_clean":"2024-01-15T18:00:42.064484+0000","last_became_active":"2024-01-15T18:00:41.067325+0000","last_became_peered":"2024-01-15T18:00:41.067325+0000","last_unstale":"2024-01-15T18:00:42.064484+0000","last_undegraded":"2024-01-15T18:00:42.064484+0000","last_fullsized":"2024-01-15T18:00:42.064484+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:50:53.105108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.068294+0000","last_change":"2024-01-15T18:00:42.063135+0000","last_active":"2024-01-15T18:00:43.068294+0000","last_peered":"2024-01-15T18:00:43.068294+0000","last_clean":"2024-01-15T18:00:43.068294+0000","last_became_active":"2024-01-15T18:00:42.062863+0000","last_became_peered":"2024-01-15T18:00:42.062863+0000","last_unstale":"2024-01-15T18:00:43.068294+0000","last_undegraded":"2024-01-15T18:00:43.068294+0000","last_fullsized":"2024-01-15T18:00:43.068294+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:26:31.608249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.095223+0000","last_change":"2024-01-15T18:00:41.073770+0000","last_active":"2024-01-15T18:00:42.095223+0000","last_peered":"2024-01-15T18:00:42.095223+0000","last_clean":"2024-01-15T18:00:42.095223+0000","last_became_active":"2024-01-15T18:00:41.073486+0000","last_became_peered":"2024-01-15T18:00:41.073486+0000","last_unstale":"2024-01-15T18:00:42.095223+0000","last_undegraded":"2024-01-15T18:00:42.095223+0000","last_fullsized":"2024-01-15T18:00:42.095223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105140+0000","last_change":"2024-01-15T18:00:42.064514+0000","last_active":"2024-01-15T18:00:43.105140+0000","last_peered":"2024-01-15T18:00:43.105140+0000","last_clean":"2024-01-15T18:00:43.105140+0000","last_became_active":"2024-01-15T18:00:42.064248+0000","last_became_peered":"2024-01-15T18:00:42.064248+0000","last_unstale":"2024-01-15T18:00:43.105140+0000","last_undegraded":"2024-01-15T18:00:43.105140+0000","last_fullsized":"2024-01-15T18:00:43.105140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:39.286999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130465+0000","last_change":"2024-01-15T18:00:41.064452+0000","last_active":"2024-01-15T18:00:42.130465+0000","last_peered":"2024-01-15T18:00:42.130465+0000","last_clean":"2024-01-15T18:00:42.130465+0000","last_became_active":"2024-01-15T18:00:41.064118+0000","last_became_peered":"2024-01-15T18:00:41.064118+0000","last_unstale":"2024-01-15T18:00:42.130465+0000","last_undegraded":"2024-01-15T18:00:42.130465+0000","last_fullsized":"2024-01-15T18:00:42.130465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:31.276292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104433+0000","last_change":"2024-01-15T18:00:42.068233+0000","last_active":"2024-01-15T18:00:43.104433+0000","last_peered":"2024-01-15T18:00:43.104433+0000","last_clean":"2024-01-15T18:00:43.104433+0000","last_became_active":"2024-01-15T18:00:42.066136+0000","last_became_peered":"2024-01-15T18:00:42.066136+0000","last_unstale":"2024-01-15T18:00:43.104433+0000","last_undegraded":"2024-01-15T18:00:43.104433+0000","last_fullsized":"2024-01-15T18:00:43.104433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:06:50.567877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.095182+0000","last_change":"2024-01-15T18:00:41.069415+0000","last_active":"2024-01-15T18:00:42.095182+0000","last_peered":"2024-01-15T18:00:42.095182+0000","last_clean":"2024-01-15T18:00:42.095182+0000","last_became_active":"2024-01-15T18:00:41.069194+0000","last_became_peered":"2024-01-15T18:00:41.069194+0000","last_unstale":"2024-01-15T18:00:42.095182+0000","last_undegraded":"2024-01-15T18:00:42.095182+0000","last_fullsized":"2024-01-15T18:00:42.095182+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:08.061519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105203+0000","last_change":"2024-01-15T18:00:42.072986+0000","last_active":"2024-01-15T18:00:43.105203+0000","last_peered":"2024-01-15T18:00:43.105203+0000","last_clean":"2024-01-15T18:00:43.105203+0000","last_became_active":"2024-01-15T18:00:42.072700+0000","last_became_peered":"2024-01-15T18:00:42.072700+0000","last_unstale":"2024-01-15T18:00:43.105203+0000","last_undegraded":"2024-01-15T18:00:43.105203+0000","last_fullsized":"2024-01-15T18:00:43.105203+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:24:05.366767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.092266+0000","last_change":"2024-01-15T18:00:41.056524+0000","last_active":"2024-01-15T18:00:42.092266+0000","last_peered":"2024-01-15T18:00:42.092266+0000","last_clean":"2024-01-15T18:00:42.092266+0000","last_became_active":"2024-01-15T18:00:41.056086+0000","last_became_peered":"2024-01-15T18:00:41.056086+0000","last_unstale":"2024-01-15T18:00:42.092266+0000","last_undegraded":"2024-01-15T18:00:42.092266+0000","last_fullsized":"2024-01-15T18:00:42.092266+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:34:36.552797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103073+0000","last_change":"2024-01-15T18:00:42.072006+0000","last_active":"2024-01-15T18:00:43.103073+0000","last_peered":"2024-01-15T18:00:43.103073+0000","last_clean":"2024-01-15T18:00:43.103073+0000","last_became_active":"2024-01-15T18:00:42.071671+0000","last_became_peered":"2024-01-15T18:00:42.071671+0000","last_unstale":"2024-01-15T18:00:43.103073+0000","last_undegraded":"2024-01-15T18:00:43.103073+0000","last_fullsized":"2024-01-15T18:00:43.103073+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:09.074909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130393+0000","last_change":"2024-01-15T18:00:41.067157+0000","last_active":"2024-01-15T18:00:42.130393+0000","last_peered":"2024-01-15T18:00:42.130393+0000","last_clean":"2024-01-15T18:00:42.130393+0000","last_became_active":"2024-01-15T18:00:41.066820+0000","last_became_peered":"2024-01-15T18:00:41.066820+0000","last_unstale":"2024-01-15T18:00:42.130393+0000","last_undegraded":"2024-01-15T18:00:42.130393+0000","last_fullsized":"2024-01-15T18:00:42.130393+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:03:48.949979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104038+0000","last_change":"2024-01-15T18:00:42.063066+0000","last_active":"2024-01-15T18:00:43.104038+0000","last_peered":"2024-01-15T18:00:43.104038+0000","last_clean":"2024-01-15T18:00:43.104038+0000","last_became_active":"2024-01-15T18:00:42.062649+0000","last_became_peered":"2024-01-15T18:00:42.062649+0000","last_unstale":"2024-01-15T18:00:43.104038+0000","last_undegraded":"2024-01-15T18:00:43.104038+0000","last_fullsized":"2024-01-15T18:00:43.104038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:58:21.543745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.364757+0000","last_change":"2024-01-15T18:00:41.061164+0000","last_active":"2024-01-15T18:00:42.364757+0000","last_peered":"2024-01-15T18:00:42.364757+0000","last_clean":"2024-01-15T18:00:42.364757+0000","last_became_active":"2024-01-15T18:00:41.060847+0000","last_became_peered":"2024-01-15T18:00:41.060847+0000","last_unstale":"2024-01-15T18:00:42.364757+0000","last_undegraded":"2024-01-15T18:00:42.364757+0000","last_fullsized":"2024-01-15T18:00:42.364757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.067095+0000","last_change":"2024-01-15T18:00:42.071612+0000","last_active":"2024-01-15T18:00:43.067095+0000","last_peered":"2024-01-15T18:00:43.067095+0000","last_clean":"2024-01-15T18:00:43.067095+0000","last_became_active":"2024-01-15T18:00:42.071343+0000","last_became_peered":"2024-01-15T18:00:42.071343+0000","last_unstale":"2024-01-15T18:00:43.067095+0000","last_undegraded":"2024-01-15T18:00:43.067095+0000","last_fullsized":"2024-01-15T18:00:43.067095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:02:30.544639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083367+0000","last_change":"2024-01-15T18:00:41.060084+0000","last_active":"2024-01-15T18:00:42.083367+0000","last_peered":"2024-01-15T18:00:42.083367+0000","last_clean":"2024-01-15T18:00:42.083367+0000","last_became_active":"2024-01-15T18:00:41.057544+0000","last_became_peered":"2024-01-15T18:00:41.057544+0000","last_unstale":"2024-01-15T18:00:42.083367+0000","last_undegraded":"2024-01-15T18:00:42.083367+0000","last_fullsized":"2024-01-15T18:00:42.083367+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103615+0000","last_change":"2024-01-15T18:00:42.071201+0000","last_active":"2024-01-15T18:00:43.103615+0000","last_peered":"2024-01-15T18:00:43.103615+0000","last_clean":"2024-01-15T18:00:43.103615+0000","last_became_active":"2024-01-15T18:00:42.070498+0000","last_became_peered":"2024-01-15T18:00:42.070498+0000","last_unstale":"2024-01-15T18:00:43.103615+0000","last_undegraded":"2024-01-15T18:00:43.103615+0000","last_fullsized":"2024-01-15T18:00:43.103615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:24:18.622665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":20,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.095073+0000","last_change":"2024-01-15T18:00:41.064043+0000","last_active":"2024-01-15T18:00:42.095073+0000","last_peered":"2024-01-15T18:00:42.095073+0000","last_clean":"2024-01-15T18:00:42.095073+0000","last_became_active":"2024-01-15T18:00:41.063128+0000","last_became_peered":"2024-01-15T18:00:41.063128+0000","last_unstale":"2024-01-15T18:00:42.095073+0000","last_undegraded":"2024-01-15T18:00:42.095073+0000","last_fullsized":"2024-01-15T18:00:42.095073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:12.496547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105537+0000","last_change":"2024-01-15T18:00:42.079795+0000","last_active":"2024-01-15T18:00:43.105537+0000","last_peered":"2024-01-15T18:00:43.105537+0000","last_clean":"2024-01-15T18:00:43.105537+0000","last_became_active":"2024-01-15T18:00:42.079150+0000","last_became_peered":"2024-01-15T18:00:42.079150+0000","last_unstale":"2024-01-15T18:00:43.105537+0000","last_undegraded":"2024-01-15T18:00:43.105537+0000","last_fullsized":"2024-01-15T18:00:43.105537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:35:00.330346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130486+0000","last_change":"2024-01-15T18:00:41.071443+0000","last_active":"2024-01-15T18:00:42.130486+0000","last_peered":"2024-01-15T18:00:42.130486+0000","last_clean":"2024-01-15T18:00:42.130486+0000","last_became_active":"2024-01-15T18:00:41.071219+0000","last_became_peered":"2024-01-15T18:00:41.071219+0000","last_unstale":"2024-01-15T18:00:42.130486+0000","last_undegraded":"2024-01-15T18:00:42.130486+0000","last_fullsized":"2024-01-15T18:00:42.130486+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:56.788064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105180+0000","last_change":"2024-01-15T18:00:42.079222+0000","last_active":"2024-01-15T18:00:43.105180+0000","last_peered":"2024-01-15T18:00:43.105180+0000","last_clean":"2024-01-15T18:00:43.105180+0000","last_became_active":"2024-01-15T18:00:42.078933+0000","last_became_peered":"2024-01-15T18:00:42.078933+0000","last_unstale":"2024-01-15T18:00:43.105180+0000","last_undegraded":"2024-01-15T18:00:43.105180+0000","last_fullsized":"2024-01-15T18:00:43.105180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:58:21.543745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168527+0000","last_change":"2024-01-15T18:00:41.071043+0000","last_active":"2024-01-15T18:00:42.168527+0000","last_peered":"2024-01-15T18:00:42.168527+0000","last_clean":"2024-01-15T18:00:42.168527+0000","last_became_active":"2024-01-15T18:00:41.070768+0000","last_became_peered":"2024-01-15T18:00:41.070768+0000","last_unstale":"2024-01-15T18:00:42.168527+0000","last_undegraded":"2024-01-15T18:00:42.168527+0000","last_fullsized":"2024-01-15T18:00:42.168527+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:26.259100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106831+0000","last_change":"2024-01-15T18:00:42.076070+0000","last_active":"2024-01-15T18:00:43.106831+0000","last_peered":"2024-01-15T18:00:43.106831+0000","last_clean":"2024-01-15T18:00:43.106831+0000","last_became_active":"2024-01-15T18:00:42.068998+0000","last_became_peered":"2024-01-15T18:00:42.068998+0000","last_unstale":"2024-01-15T18:00:43.106831+0000","last_undegraded":"2024-01-15T18:00:43.106831+0000","last_fullsized":"2024-01-15T18:00:43.106831+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:48:33.487610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.085417+0000","last_change":"2024-01-15T18:00:41.061292+0000","last_active":"2024-01-15T18:00:42.085417+0000","last_peered":"2024-01-15T18:00:42.085417+0000","last_clean":"2024-01-15T18:00:42.085417+0000","last_became_active":"2024-01-15T18:00:41.061006+0000","last_became_peered":"2024-01-15T18:00:41.061006+0000","last_unstale":"2024-01-15T18:00:42.085417+0000","last_undegraded":"2024-01-15T18:00:42.085417+0000","last_fullsized":"2024-01-15T18:00:42.085417+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:02:48.557075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105474+0000","last_change":"2024-01-15T18:00:42.067416+0000","last_active":"2024-01-15T18:00:43.105474+0000","last_peered":"2024-01-15T18:00:43.105474+0000","last_clean":"2024-01-15T18:00:43.105474+0000","last_became_active":"2024-01-15T18:00:42.067129+0000","last_became_peered":"2024-01-15T18:00:42.067129+0000","last_unstale":"2024-01-15T18:00:43.105474+0000","last_undegraded":"2024-01-15T18:00:43.105474+0000","last_fullsized":"2024-01-15T18:00:43.105474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:35:40.538621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.168424+0000","last_change":"2024-01-15T18:00:41.071456+0000","last_active":"2024-01-15T18:00:42.168424+0000","last_peered":"2024-01-15T18:00:42.168424+0000","last_clean":"2024-01-15T18:00:42.168424+0000","last_became_active":"2024-01-15T18:00:41.071189+0000","last_became_peered":"2024-01-15T18:00:41.071189+0000","last_unstale":"2024-01-15T18:00:42.168424+0000","last_undegraded":"2024-01-15T18:00:42.168424+0000","last_fullsized":"2024-01-15T18:00:42.168424+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:16.529781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103570+0000","last_change":"2024-01-15T18:00:42.065297+0000","last_active":"2024-01-15T18:00:43.103570+0000","last_peered":"2024-01-15T18:00:43.103570+0000","last_clean":"2024-01-15T18:00:43.103570+0000","last_became_active":"2024-01-15T18:00:42.064969+0000","last_became_peered":"2024-01-15T18:00:42.064969+0000","last_unstale":"2024-01-15T18:00:43.103570+0000","last_undegraded":"2024-01-15T18:00:43.103570+0000","last_fullsized":"2024-01-15T18:00:43.103570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:17.327716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130413+0000","last_change":"2024-01-15T18:00:41.068101+0000","last_active":"2024-01-15T18:00:42.130413+0000","last_peered":"2024-01-15T18:00:42.130413+0000","last_clean":"2024-01-15T18:00:42.130413+0000","last_became_active":"2024-01-15T18:00:41.065928+0000","last_became_peered":"2024-01-15T18:00:41.065928+0000","last_unstale":"2024-01-15T18:00:42.130413+0000","last_undegraded":"2024-01-15T18:00:42.130413+0000","last_fullsized":"2024-01-15T18:00:42.130413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:19:23.525421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.064980+0000","last_change":"2024-01-15T18:00:42.067592+0000","last_active":"2024-01-15T18:00:43.064980+0000","last_peered":"2024-01-15T18:00:43.064980+0000","last_clean":"2024-01-15T18:00:43.064980+0000","last_became_active":"2024-01-15T18:00:42.067309+0000","last_became_peered":"2024-01-15T18:00:42.067309+0000","last_unstale":"2024-01-15T18:00:43.064980+0000","last_undegraded":"2024-01-15T18:00:43.064980+0000","last_fullsized":"2024-01-15T18:00:43.064980+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:24:18.622665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069730+0000","last_change":"2024-01-15T18:00:41.050813+0000","last_active":"2024-01-15T18:00:42.069730+0000","last_peered":"2024-01-15T18:00:42.069730+0000","last_clean":"2024-01-15T18:00:42.069730+0000","last_became_active":"2024-01-15T18:00:41.050320+0000","last_became_peered":"2024-01-15T18:00:41.050320+0000","last_unstale":"2024-01-15T18:00:42.069730+0000","last_undegraded":"2024-01-15T18:00:42.069730+0000","last_fullsized":"2024-01-15T18:00:42.069730+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065150+0000","last_change":"2024-01-15T18:00:42.070090+0000","last_active":"2024-01-15T18:00:43.065150+0000","last_peered":"2024-01-15T18:00:43.065150+0000","last_clean":"2024-01-15T18:00:43.065150+0000","last_became_active":"2024-01-15T18:00:42.068897+0000","last_became_peered":"2024-01-15T18:00:42.068897+0000","last_unstale":"2024-01-15T18:00:43.065150+0000","last_undegraded":"2024-01-15T18:00:43.065150+0000","last_fullsized":"2024-01-15T18:00:43.065150+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:08:56.863439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083292+0000","last_change":"2024-01-15T18:00:41.065251+0000","last_active":"2024-01-15T18:00:42.083292+0000","last_peered":"2024-01-15T18:00:42.083292+0000","last_clean":"2024-01-15T18:00:42.083292+0000","last_became_active":"2024-01-15T18:00:41.065006+0000","last_became_peered":"2024-01-15T18:00:41.065006+0000","last_unstale":"2024-01-15T18:00:42.083292+0000","last_undegraded":"2024-01-15T18:00:42.083292+0000","last_fullsized":"2024-01-15T18:00:42.083292+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105082+0000","last_change":"2024-01-15T18:00:42.083131+0000","last_active":"2024-01-15T18:00:43.105082+0000","last_peered":"2024-01-15T18:00:43.105082+0000","last_clean":"2024-01-15T18:00:43.105082+0000","last_became_active":"2024-01-15T18:00:42.082546+0000","last_became_peered":"2024-01-15T18:00:42.082546+0000","last_unstale":"2024-01-15T18:00:43.105082+0000","last_undegraded":"2024-01-15T18:00:43.105082+0000","last_fullsized":"2024-01-15T18:00:43.105082+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:08:56.863439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.087697+0000","last_change":"2024-01-15T18:00:41.066193+0000","last_active":"2024-01-15T18:00:42.087697+0000","last_peered":"2024-01-15T18:00:42.087697+0000","last_clean":"2024-01-15T18:00:42.087697+0000","last_became_active":"2024-01-15T18:00:41.065913+0000","last_became_peered":"2024-01-15T18:00:41.065913+0000","last_unstale":"2024-01-15T18:00:42.087697+0000","last_undegraded":"2024-01-15T18:00:42.087697+0000","last_fullsized":"2024-01-15T18:00:42.087697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:19.680254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.105504+0000","last_change":"2024-01-15T18:00:42.076020+0000","last_active":"2024-01-15T18:00:43.105504+0000","last_peered":"2024-01-15T18:00:43.105504+0000","last_clean":"2024-01-15T18:00:43.105504+0000","last_became_active":"2024-01-15T18:00:42.075751+0000","last_became_peered":"2024-01-15T18:00:42.075751+0000","last_unstale":"2024-01-15T18:00:43.105504+0000","last_undegraded":"2024-01-15T18:00:43.105504+0000","last_fullsized":"2024-01-15T18:00:43.105504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:38:39.286999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065784+0000","last_change":"2024-01-15T18:00:41.070458+0000","last_active":"2024-01-15T18:00:43.065784+0000","last_peered":"2024-01-15T18:00:43.065784+0000","last_clean":"2024-01-15T18:00:43.065784+0000","last_became_active":"2024-01-15T18:00:41.066322+0000","last_became_peered":"2024-01-15T18:00:41.066322+0000","last_unstale":"2024-01-15T18:00:43.065784+0000","last_undegraded":"2024-01-15T18:00:43.065784+0000","last_fullsized":"2024-01-15T18:00:43.065784+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:35:40.700476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104807+0000","last_change":"2024-01-15T18:00:42.066727+0000","last_active":"2024-01-15T18:00:43.104807+0000","last_peered":"2024-01-15T18:00:43.104807+0000","last_clean":"2024-01-15T18:00:43.104807+0000","last_became_active":"2024-01-15T18:00:42.065788+0000","last_became_peered":"2024-01-15T18:00:42.065788+0000","last_unstale":"2024-01-15T18:00:43.104807+0000","last_undegraded":"2024-01-15T18:00:43.104807+0000","last_fullsized":"2024-01-15T18:00:43.104807+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:16:20.555824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.364842+0000","last_change":"2024-01-15T18:00:41.063919+0000","last_active":"2024-01-15T18:00:42.364842+0000","last_peered":"2024-01-15T18:00:42.364842+0000","last_clean":"2024-01-15T18:00:42.364842+0000","last_became_active":"2024-01-15T18:00:41.062115+0000","last_became_peered":"2024-01-15T18:00:41.062115+0000","last_unstale":"2024-01-15T18:00:42.364842+0000","last_undegraded":"2024-01-15T18:00:42.364842+0000","last_fullsized":"2024-01-15T18:00:42.364842+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:04.177405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.065646+0000","last_change":"2024-01-15T18:00:42.073589+0000","last_active":"2024-01-15T18:00:43.065646+0000","last_peered":"2024-01-15T18:00:43.065646+0000","last_clean":"2024-01-15T18:00:43.065646+0000","last_became_active":"2024-01-15T18:00:42.073254+0000","last_became_peered":"2024-01-15T18:00:42.073254+0000","last_unstale":"2024-01-15T18:00:43.065646+0000","last_undegraded":"2024-01-15T18:00:43.065646+0000","last_fullsized":"2024-01-15T18:00:43.065646+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:21.834287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.083478+0000","last_change":"2024-01-15T18:00:41.057399+0000","last_active":"2024-01-15T18:00:42.083478+0000","last_peered":"2024-01-15T18:00:42.083478+0000","last_clean":"2024-01-15T18:00:42.083478+0000","last_became_active":"2024-01-15T18:00:41.056174+0000","last_became_peered":"2024-01-15T18:00:41.056174+0000","last_unstale":"2024-01-15T18:00:42.083478+0000","last_undegraded":"2024-01-15T18:00:42.083478+0000","last_fullsized":"2024-01-15T18:00:42.083478+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:26.806673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.106964+0000","last_change":"2024-01-15T18:00:42.052005+0000","last_active":"2024-01-15T18:00:43.106964+0000","last_peered":"2024-01-15T18:00:43.106964+0000","last_clean":"2024-01-15T18:00:43.106964+0000","last_became_active":"2024-01-15T18:00:42.051122+0000","last_became_peered":"2024-01-15T18:00:42.051122+0000","last_unstale":"2024-01-15T18:00:43.106964+0000","last_undegraded":"2024-01-15T18:00:43.106964+0000","last_fullsized":"2024-01-15T18:00:43.106964+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:59:22.659547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069747+0000","last_change":"2024-01-15T18:00:41.064076+0000","last_active":"2024-01-15T18:00:42.069747+0000","last_peered":"2024-01-15T18:00:42.069747+0000","last_clean":"2024-01-15T18:00:42.069747+0000","last_became_active":"2024-01-15T18:00:41.062980+0000","last_became_peered":"2024-01-15T18:00:41.062980+0000","last_unstale":"2024-01-15T18:00:42.069747+0000","last_undegraded":"2024-01-15T18:00:42.069747+0000","last_fullsized":"2024-01-15T18:00:42.069747+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:41:42.728224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.104591+0000","last_change":"2024-01-15T18:00:42.070329+0000","last_active":"2024-01-15T18:00:43.104591+0000","last_peered":"2024-01-15T18:00:43.104591+0000","last_clean":"2024-01-15T18:00:43.104591+0000","last_became_active":"2024-01-15T18:00:42.068798+0000","last_became_peered":"2024-01-15T18:00:42.068798+0000","last_unstale":"2024-01-15T18:00:43.104591+0000","last_undegraded":"2024-01-15T18:00:43.104591+0000","last_fullsized":"2024-01-15T18:00:43.104591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:37:55.705369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.064582+0000","last_change":"2024-01-15T18:00:41.054271+0000","last_active":"2024-01-15T18:00:42.064582+0000","last_peered":"2024-01-15T18:00:42.064582+0000","last_clean":"2024-01-15T18:00:42.064582+0000","last_became_active":"2024-01-15T18:00:41.053925+0000","last_became_peered":"2024-01-15T18:00:41.053925+0000","last_unstale":"2024-01-15T18:00:42.064582+0000","last_undegraded":"2024-01-15T18:00:42.064582+0000","last_fullsized":"2024-01-15T18:00:42.064582+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:18:46.672266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103301+0000","last_change":"2024-01-15T18:00:42.057721+0000","last_active":"2024-01-15T18:00:43.103301+0000","last_peered":"2024-01-15T18:00:43.103301+0000","last_clean":"2024-01-15T18:00:43.103301+0000","last_became_active":"2024-01-15T18:00:42.057091+0000","last_became_peered":"2024-01-15T18:00:42.057091+0000","last_unstale":"2024-01-15T18:00:43.103301+0000","last_undegraded":"2024-01-15T18:00:43.103301+0000","last_fullsized":"2024-01-15T18:00:43.103301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:03.318096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.069705+0000","last_change":"2024-01-15T18:00:41.061944+0000","last_active":"2024-01-15T18:00:42.069705+0000","last_peered":"2024-01-15T18:00:42.069705+0000","last_clean":"2024-01-15T18:00:42.069705+0000","last_became_active":"2024-01-15T18:00:41.061646+0000","last_became_peered":"2024-01-15T18:00:41.061646+0000","last_unstale":"2024-01-15T18:00:42.069705+0000","last_undegraded":"2024-01-15T18:00:42.069705+0000","last_fullsized":"2024-01-15T18:00:42.069705+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:23:57.016225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.107069+0000","last_change":"2024-01-15T18:00:42.055286+0000","last_active":"2024-01-15T18:00:43.107069+0000","last_peered":"2024-01-15T18:00:43.107069+0000","last_clean":"2024-01-15T18:00:43.107069+0000","last_became_active":"2024-01-15T18:00:42.054746+0000","last_became_peered":"2024-01-15T18:00:42.054746+0000","last_unstale":"2024-01-15T18:00:43.107069+0000","last_undegraded":"2024-01-15T18:00:43.107069+0000","last_fullsized":"2024-01-15T18:00:43.107069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:59.371652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.130314+0000","last_change":"2024-01-15T18:00:41.068287+0000","last_active":"2024-01-15T18:00:42.130314+0000","last_peered":"2024-01-15T18:00:42.130314+0000","last_clean":"2024-01-15T18:00:42.130314+0000","last_became_active":"2024-01-15T18:00:41.066120+0000","last_became_peered":"2024-01-15T18:00:41.066120+0000","last_unstale":"2024-01-15T18:00:42.130314+0000","last_undegraded":"2024-01-15T18:00:42.130314+0000","last_fullsized":"2024-01-15T18:00:42.130314+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:39:48.165240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-01-15T18:00:43.103877+0000","last_change":"2024-01-15T18:00:42.080052+0000","last_active":"2024-01-15T18:00:43.103877+0000","last_peered":"2024-01-15T18:00:43.103877+0000","last_clean":"2024-01-15T18:00:43.103877+0000","last_became_active":"2024-01-15T18:00:42.079699+0000","last_became_peered":"2024-01-15T18:00:42.079699+0000","last_unstale":"2024-01-15T18:00:43.103877+0000","last_undegraded":"2024-01-15T18:00:43.103877+0000","last_fullsized":"2024-01-15T18:00:43.103877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:41.020101+0000","last_clean_scrub_stamp":"2024-01-15T18:00:41.020101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:51:28.238550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.364734+0000","last_change":"2024-01-15T18:00:41.061168+0000","last_active":"2024-01-15T18:00:42.364734+0000","last_peered":"2024-01-15T18:00:42.364734+0000","last_clean":"2024-01-15T18:00:42.364734+0000","last_became_active":"2024-01-15T18:00:41.060793+0000","last_became_peered":"2024-01-15T18:00:41.060793+0000","last_unstale":"2024-01-15T18:00:42.364734+0000","last_undegraded":"2024-01-15T18:00:42.364734+0000","last_fullsized":"2024-01-15T18:00:42.364734+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:00:40.006711+0000","last_clean_scrub_stamp":"2024-01-15T18:00:40.006711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:02.304705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":196,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-01-15T18:00:42.050457+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:00:42.050457+0000","last_peered":"2024-01-15T18:00:42.050457+0000","last_clean":"2024-01-15T18:00:42.050457+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:00:42.050457+0000","last_undegraded":"2024-01-15T18:00:42.050457+0000","last_fullsized":"2024-01-15T18:00:42.050457+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542217,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1318912,"data_stored":890846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.2230000000000001,"15min":1.177},"min":{"1min":0.69799999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":5.556,"5min":5.556,"15min":5.556},"last":0.70599999999999996},{"interface":"front","average":{"1min":3.444,"5min":1.649,"15min":1.3260000000000001},"min":{"1min":0.747,"5min":0.623,"15min":0.623},"max":{"1min":38.137,"5min":38.137,"15min":38.137},"last":1.5309999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.266,"15min":1.25},"min":{"1min":0.72999999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":5.0449999999999999,"5min":5.0449999999999999,"15min":5.0449999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":4.194,"5min":1.861,"15min":1.4570000000000001},"min":{"1min":0.72699999999999998,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":51.843000000000004,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.3979999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":1.921,"5min":1.361,"15min":1.2270000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":9.3369999999999997,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.643,"5min":1.397,"15min":1.393},"min":{"1min":0.66000000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.2889999999999997,"5min":6.2889999999999997,"15min":6.2889999999999997},"last":1.1439999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":2.2589999999999999,"5min":1.49,"15min":1.363},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":9.1440000000000001,"5min":9.1440000000000001,"15min":9.1440000000000001},"last":1.462},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5029999999999999,"15min":1.4970000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":6.2489999999999997,"5min":6.2489999999999997,"15min":6.2489999999999997},"last":1.367}]},{"osd":4,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":3.3300000000000001,"5min":1.788,"15min":1.5980000000000001},"min":{"1min":0.95799999999999996,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":22.359999999999999,"5min":22.359999999999999,"15min":22.359999999999999},"last":1.645},{"interface":"front","average":{"1min":2.2130000000000001,"5min":1.597,"15min":1.496},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":12.667999999999999,"5min":12.667999999999999,"15min":12.667999999999999},"last":1.2450000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.0220000000000002,"5min":2.0569999999999999,"15min":1.7569999999999999},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":25.006,"5min":25.006,"15min":25.006},"last":1.5129999999999999},{"interface":"front","average":{"1min":3.4950000000000001,"5min":1.9239999999999999,"15min":1.712},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":22.693999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.734}]},{"osd":6,"last update":"Mon Jan 15 18:00:06 2024","interfaces":[{"interface":"back","average":{"1min":4.9530000000000003,"5min":2.2770000000000001,"15min":1.901},"min":{"1min":1.0580000000000001,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":23.834,"5min":23.834,"15min":23.834},"last":1.8120000000000001},{"interface":"front","average":{"1min":5.9169999999999998,"5min":2.4729999999999999,"15min":1.9199999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":63.932000000000002,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.5780000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542216,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36044,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335796,"statfs":{"total":96636764160,"available":96599855104,"internally_reserved":0,"allocated":716800,"data_stored":300318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.413,"15min":1.3169999999999999},"min":{"1min":0.71699999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.8769999999999998,"5min":12.544,"15min":12.544},"last":1.2110000000000001},{"interface":"front","average":{"1min":2.1840000000000002,"5min":1.423,"15min":1.248},"min":{"1min":0.81899999999999995,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":13.98,"5min":13.98,"15min":13.98},"last":0.88100000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.484,"5min":1.357,"15min":1.2250000000000001},"min":{"1min":0.82599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.5659999999999998,"5min":12.811999999999999,"15min":12.811999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":2.0600000000000001,"5min":1.349,"15min":1.2330000000000001},"min":{"1min":0.86499999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":8.8130000000000006,"5min":8.8130000000000006,"15min":8.8130000000000006},"last":1.048}]},{"osd":2,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2970000000000002,"5min":1.456,"15min":1.28},"min":{"1min":0.90300000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":8.7080000000000002,"5min":8.7080000000000002,"15min":8.7080000000000002},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4890000000000001,"15min":1.5009999999999999},"min":{"1min":0.86899999999999999,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":4.0419999999999998,"5min":12.897,"15min":12.897},"last":1.4710000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.988,"5min":1.7010000000000001,"15min":1.542},"min":{"1min":1.0880000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":9.5,"5min":12.717000000000001,"15min":12.717000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":2.52,"5min":1.5840000000000001,"15min":1.4410000000000001},"min":{"1min":0.93799999999999994,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":14.066000000000001,"5min":14.066000000000001,"15min":14.066000000000001},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5009999999999999,"15min":1.454},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":9.2829999999999995,"5min":9.2829999999999995,"15min":9.2829999999999995},"last":1.163},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.6359999999999999,"15min":1.607},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":3.6989999999999998,"5min":16.98,"15min":16.98},"last":1.415}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.1659999999999999,"5min":1.548,"15min":1.484},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":14.176,"5min":14.176,"15min":14.176},"last":0.98599999999999999},{"interface":"front","average":{"1min":2.3079999999999998,"5min":1.756,"15min":1.7569999999999999},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":8.8689999999999998,"5min":8.8689999999999998,"15min":8.8689999999999998},"last":1.391}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":2.29,"5min":1.6399999999999999,"15min":1.601},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":14.109999999999999,"5min":14.109999999999999,"15min":14.109999999999999},"last":1.087},{"interface":"front","average":{"1min":2.5870000000000002,"5min":1.8009999999999999,"15min":1.7769999999999999},"min":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"max":{"1min":14.478,"5min":14.478,"15min":14.478},"last":1.335}]}]},{"osd":5,"up_from":14,"seq":60129542216,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36052,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335788,"statfs":{"total":96636764160,"available":96599846912,"internally_reserved":0,"allocated":724992,"data_stored":300970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.3149999999999999,"15min":1.2509999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":2.7490000000000001,"5min":16.178000000000001,"15min":16.178000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":2.4980000000000002,"5min":1.4750000000000001,"15min":1.417},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":17.184999999999999,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.1539999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.2929999999999999,"15min":1.2250000000000001},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":5.6219999999999999,"5min":5.6219999999999999,"15min":5.6219999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.423,"5min":1.2629999999999999,"15min":1.2110000000000001},"min":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.1379999999999999,"5min":4.1379999999999999,"15min":4.1379999999999999},"last":1.26}]},{"osd":2,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2050000000000001,"15min":1.2190000000000001},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.681,"5min":2.835,"15min":2.835},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.367,"15min":1.288},"min":{"1min":1.0760000000000001,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":5.9740000000000002,"5min":5.9740000000000002,"15min":5.9740000000000002},"last":1.522}]},{"osd":3,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.506,"15min":1.3520000000000001},"min":{"1min":0.73099999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.8029999999999999,"5min":14.914,"15min":14.914},"last":1.121},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.365,"15min":1.413},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.726,"5min":3.302,"15min":3.302},"last":1.5609999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.395,"15min":1.411},"min":{"1min":0.99299999999999999,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.7759999999999998,"5min":4.7759999999999998,"15min":4.7759999999999998},"last":1.609},{"interface":"front","average":{"1min":2.1709999999999998,"5min":1.75,"15min":1.571},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":17.991,"5min":17.991,"15min":17.991},"last":1.4790000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5589999999999999,"5min":1.506,"15min":1.6020000000000001},"min":{"1min":0.91900000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":6.4630000000000001,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.792,"5min":1.5369999999999999,"15min":1.55},"min":{"1min":0.88100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":7.7759999999999998,"5min":7.7759999999999998,"15min":7.7759999999999998},"last":1.6930000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:00:04 2024","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5580000000000001,"15min":1.6060000000000001},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":6.3230000000000004,"5min":6.3230000000000004,"15min":6.3230000000000004},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.603,"15min":1.728},"min":{"1min":0.90500000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":5.2030000000000003,"5min":5.3259999999999996,"15min":5.3259999999999996},"last":1.4370000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542216,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36056,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335784,"statfs":{"total":96636764160,"available":96599842816,"internally_reserved":0,"allocated":729088,"data_stored":301930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.544,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":6.9989999999999997,"5min":11.413,"15min":11.413},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.8979999999999999,"5min":1.4350000000000001,"15min":1.3799999999999999},"min":{"1min":0.77500000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":9.7439999999999998,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":1.494}]},{"osd":1,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.8089999999999999,"5min":1.4670000000000001,"15min":1.3819999999999999},"min":{"1min":0.84299999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":9.7010000000000005,"5min":11.987,"15min":11.987},"last":1.268},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.27,"15min":1.323},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":6.149,"5min":6.149,"15min":6.149},"last":1.107}]},{"osd":2,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.708,"15min":1.5700000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":7.032,"5min":13.865,"15min":13.865},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.514,"15min":1.3959999999999999},"min":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":3.028,"5min":13.903,"15min":13.903},"last":1.329}]},{"osd":3,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5760000000000001,"5min":1.6040000000000001,"15min":1.5920000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":6.8819999999999997,"5min":11.611000000000001,"15min":11.611000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":2.0800000000000001,"5min":1.546,"15min":1.6180000000000001},"min":{"1min":0.69199999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":13.25,"5min":13.25,"15min":13.25},"last":1.159}]},{"osd":5,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.6359999999999999,"15min":1.6850000000000001},"min":{"1min":0.61899999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":6.6820000000000004,"5min":11.561,"15min":11.561},"last":1.532},{"interface":"front","average":{"1min":1.8999999999999999,"5min":1.5509999999999999,"15min":1.665},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":9.6229999999999993,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.4370000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.468,"15min":1.6259999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":6.1699999999999999,"5min":6.1699999999999999,"15min":6.1699999999999999},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.698,"5min":1.8500000000000001,"15min":1.8460000000000001},"min":{"1min":0.85799999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":6.9329999999999998,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.6559999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:02 2024","interfaces":[{"interface":"back","average":{"1min":2.359,"5min":1.8979999999999999,"15min":1.9059999999999999},"min":{"1min":1.375,"5min":0.503,"15min":0.503},"max":{"1min":10.157999999999999,"5min":11.894,"15min":11.894},"last":1.585},{"interface":"front","average":{"1min":2.5169999999999999,"5min":2.048,"15min":1.968},"min":{"1min":1.325,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":8.4169999999999998,"5min":11.714,"15min":11.714},"last":1.748}]}]},{"osd":3,"up_from":14,"seq":60129542218,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44368,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327472,"statfs":{"total":96636764160,"available":96591331328,"internally_reserved":0,"allocated":724992,"data_stored":300374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.1120000000000001,"5min":1.1990000000000001,"15min":1.0720000000000001},"min":{"1min":0.62,"5min":0.435,"15min":0.435},"max":{"1min":9.9789999999999992,"5min":9.9789999999999992,"15min":9.9789999999999992},"last":0.76000000000000001},{"interface":"front","average":{"1min":2.9420000000000002,"5min":1.5149999999999999,"15min":1.218},"min":{"1min":0.85199999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":21.047000000000001,"5min":21.047000000000001,"15min":21.047000000000001},"last":0.93000000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.8919999999999999,"5min":1.524,"15min":1.179},"min":{"1min":0.79700000000000004,"5min":0.375,"15min":0.375},"max":{"1min":17.777999999999999,"5min":17.777999999999999,"15min":17.777999999999999},"last":1.087},{"interface":"front","average":{"1min":1.786,"5min":1.2210000000000001,"15min":1.135},"min":{"1min":0.69299999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":9.2360000000000007,"5min":9.2360000000000007,"15min":9.2360000000000007},"last":0.86399999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.27,"15min":1.1899999999999999},"min":{"1min":0.79000000000000004,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":5.4930000000000003,"5min":5.4930000000000003,"15min":5.4930000000000003},"last":0.78400000000000003},{"interface":"front","average":{"1min":2.0510000000000002,"5min":1.492,"15min":1.4119999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":9.3040000000000003,"5min":9.3040000000000003,"15min":9.3040000000000003},"last":0.71299999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.0630000000000002,"5min":1.51,"15min":1.4199999999999999},"min":{"1min":1.1120000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":1.1299999999999999},{"interface":"front","average":{"1min":2.367,"5min":1.643,"15min":1.4890000000000001},"min":{"1min":0.92300000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":10.313000000000001,"5min":10.313000000000001,"15min":10.313000000000001},"last":1.4370000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.2229999999999999,"5min":1.635,"15min":1.5649999999999999},"min":{"1min":1.046,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":10.243,"5min":10.243,"15min":10.243},"last":1.492},{"interface":"front","average":{"1min":2.2330000000000001,"5min":1.742,"15min":1.6599999999999999},"min":{"1min":0.90500000000000003,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":6.2000000000000002,"5min":6.2000000000000002,"15min":6.2000000000000002},"last":1.2210000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.5510000000000002,"5min":1.851,"15min":1.7250000000000001},"min":{"1min":0.98399999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":10.012,"5min":10.012,"15min":10.012},"last":1.25},{"interface":"front","average":{"1min":2.6190000000000002,"5min":1.8220000000000001,"15min":1.722},"min":{"1min":1.0840000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":10.291,"5min":10.291,"15min":10.291},"last":1.335}]},{"osd":7,"last update":"Mon Jan 15 18:00:05 2024","interfaces":[{"interface":"back","average":{"1min":2.802,"5min":1.8819999999999999,"15min":1.7589999999999999},"min":{"1min":1.02,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":12.627000000000001,"5min":12.627000000000001,"15min":12.627000000000001},"last":1.284},{"interface":"front","average":{"1min":2.9710000000000001,"5min":1.946,"15min":1.7929999999999999},"min":{"1min":1.1240000000000001,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":10.268000000000001,"5min":10.268000000000001,"15min":10.268000000000001},"last":1.1819999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40208,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331632,"statfs":{"total":96636764160,"available":96595591168,"internally_reserved":0,"allocated":724992,"data_stored":300374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.107,"15min":1.042},"min":{"1min":0.48999999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.466,"5min":14.427,"15min":14.427},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.0860000000000001,"15min":1.02},"min":{"1min":0.64100000000000001,"5min":0.432,"15min":0.432},"max":{"1min":2.3900000000000001,"5min":2.3900000000000001,"15min":2.3900000000000001},"last":0.86599999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.3129999999999999,"15min":1.2949999999999999},"min":{"1min":0.53000000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.7490000000000001,"5min":14.609,"15min":14.609},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.1830000000000001,"15min":1.2010000000000001},"min":{"1min":0.748,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.5059999999999998,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.2869999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2609999999999999,"15min":1.1990000000000001},"min":{"1min":0.64700000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.772,"5min":16.567,"15min":16.567},"last":1.532},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.48,"15min":1.4630000000000001},"min":{"1min":0.53700000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8180000000000001,"5min":14.528,"15min":14.528},"last":1.385}]},{"osd":4,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.6559999999999999,"5min":1.708,"15min":1.647},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.7690000000000001,"5min":14.779,"15min":14.779},"last":1.607},{"interface":"front","average":{"1min":2.073,"5min":1.546,"15min":1.502},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":10.75,"5min":10.75,"15min":10.75},"last":1.492}]},{"osd":5,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.5609999999999999,"15min":1.506},"min":{"1min":0.85199999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.258,"5min":17.167000000000002,"15min":17.167000000000002},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.653,"5min":1.726,"15min":1.659},"min":{"1min":1,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.4750000000000001,"5min":14.702999999999999,"15min":14.702999999999999},"last":1.6739999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.889,"5min":1.5580000000000001,"15min":1.5149999999999999},"min":{"1min":1.131,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":6.5830000000000002,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.742},{"interface":"front","average":{"1min":1.605,"5min":1.6339999999999999,"15min":1.5},"min":{"1min":1.171,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.415,"5min":17.07,"15min":17.07},"last":1.776}]},{"osd":7,"last update":"Mon Jan 15 18:00:03 2024","interfaces":[{"interface":"back","average":{"1min":1.839,"5min":1.621,"15min":1.6259999999999999},"min":{"1min":1.333,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":3.2170000000000001,"5min":3.6070000000000002,"15min":3.6070000000000002},"last":1.6479999999999999},{"interface":"front","average":{"1min":2.9049999999999998,"5min":2.1070000000000002,"15min":1.7070000000000001},"min":{"1min":1.2070000000000001,"5min":0.752,"15min":0.752},"max":{"1min":15.135,"5min":16.922000000000001,"15min":16.922000000000001},"last":1.702}]}]},{"osd":1,"up_from":14,"seq":60129542217,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40212,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331628,"statfs":{"total":96636764160,"available":96595587072,"internally_reserved":0,"allocated":720896,"data_stored":300408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.095,"15min":0.92800000000000005},"min":{"1min":0.49199999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":8.1140000000000008,"5min":8.1140000000000008,"15min":8.1140000000000008},"last":0.876},{"interface":"front","average":{"1min":0.92600000000000005,"5min":1.0169999999999999,"15min":0.99299999999999999},"min":{"1min":0.64000000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.456,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":1.5600000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.4289999999999998,"5min":1.3109999999999999,"15min":0.99399999999999999},"min":{"1min":0.51000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":21.126000000000001,"5min":21.126000000000001,"15min":21.126000000000001},"last":1.014},{"interface":"front","average":{"1min":2.427,"5min":1.5309999999999999,"15min":1.254},"min":{"1min":0.62,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":11.5,"5min":11.5,"15min":11.5},"last":1.605}]},{"osd":3,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.573,"15min":2.0110000000000001},"min":{"1min":0.76400000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":8.3209999999999997,"5min":22.106999999999999,"15min":22.106999999999999},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.4239999999999999,"15min":1.244},"min":{"1min":0.58699999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6060000000000001,"5min":15.510999999999999,"15min":15.510999999999999},"last":0.93100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.841,"5min":1.6419999999999999,"15min":1.74},"min":{"1min":0.88800000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":6.3109999999999999,"5min":9.8510000000000009,"15min":9.8510000000000009},"last":1.236},{"interface":"front","average":{"1min":1.734,"5min":1.5800000000000001,"15min":1.863},"min":{"1min":0.80500000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":4.8239999999999998,"5min":14.359999999999999,"15min":14.359999999999999},"last":1.506}]},{"osd":5,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.0179999999999998,"5min":1.6339999999999999,"15min":1.8600000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":11.552,"5min":13.228999999999999,"15min":13.228999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.851,"5min":1.9159999999999999,"15min":1.706},"min":{"1min":0.85399999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":8.2319999999999993,"5min":20.823,"15min":20.823},"last":1.714}]},{"osd":6,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":2.282,"5min":1.7749999999999999,"15min":2.0369999999999999},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":10.696,"5min":14.272,"15min":14.272},"last":1.8380000000000001},{"interface":"front","average":{"1min":2.1150000000000002,"5min":1.637,"15min":1.5309999999999999},"min":{"1min":1.0629999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":8.8320000000000007,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.649}]},{"osd":7,"last update":"Mon Jan 15 18:00:07 2024","interfaces":[{"interface":"back","average":{"1min":1.8120000000000001,"5min":1.6459999999999999,"15min":1.575},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":8.2769999999999992,"5min":8.2769999999999992,"15min":8.2769999999999992},"last":1.806},{"interface":"front","average":{"1min":3.2530000000000001,"5min":1.881,"15min":1.665},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":24.289999999999999,"5min":24.289999999999999,"15min":24.289999999999999},"last":1.776}]}]},{"osd":0,"up_from":14,"seq":60129542218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40800,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331040,"statfs":{"total":96636764160,"available":96594984960,"internally_reserved":0,"allocated":1327104,"data_stored":892880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.92100000000000004,"15min":0.88500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":2.3650000000000002,"5min":2.3650000000000002,"15min":2.3650000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.363,"5min":1.1559999999999999,"15min":1.1000000000000001},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0979999999999999,"5min":5.0979999999999999,"15min":5.0979999999999999},"last":1.2709999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.012},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.6000000000000001,"5min":2.6000000000000001,"15min":2.6000000000000001},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.0009999999999999,"15min":0.88},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7490000000000001,"5min":2.129,"15min":2.129},"last":0.80700000000000005}]},{"osd":3,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2709999999999999,"15min":1.262},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.8440000000000001,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":1.381},{"interface":"front","average":{"1min":1.792,"5min":1.3049999999999999,"15min":1.123},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":14.74,"5min":14.74,"15min":14.74},"last":1.0049999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8109999999999999,"5min":1.4179999999999999,"15min":1.3080000000000001},"min":{"1min":1.004,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":8.6959999999999997,"5min":8.6959999999999997,"15min":8.6959999999999997},"last":1.3100000000000001},{"interface":"front","average":{"1min":2.3540000000000001,"5min":1.6839999999999999,"15min":1.5549999999999999},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":15.975,"5min":15.975,"15min":15.975},"last":1.5429999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.641,"5min":1.502,"15min":1.373},"min":{"1min":1.0860000000000001,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":3.133,"5min":8.0120000000000005,"15min":8.0120000000000005},"last":1.1639999999999999},{"interface":"front","average":{"1min":2.1000000000000001,"5min":1.571,"15min":1.5129999999999999},"min":{"1min":1.0669999999999999,"5min":0.754,"15min":0.754},"max":{"1min":6.6829999999999998,"5min":6.6829999999999998,"15min":6.6829999999999998},"last":1.835}]},{"osd":6,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":1.8360000000000001,"5min":1.5900000000000001,"15min":1.524},"min":{"1min":0.96999999999999997,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":5.2560000000000002,"5min":5.2560000000000002,"15min":5.2560000000000002},"last":2.0190000000000001},{"interface":"front","average":{"1min":1.732,"5min":1.526,"15min":1.462},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.181,"5min":3.181,"15min":3.181},"last":1.4399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:00:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0409999999999999,"5min":1.71,"15min":1.6639999999999999},"min":{"1min":1.214,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":7.6760000000000002,"5min":7.6760000000000002,"15min":7.6760000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":2.1970000000000001,"5min":1.7090000000000001,"15min":1.593},"min":{"1min":1.012,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":6.2619999999999996,"5min":6.2619999999999996,"15min":6.2619999999999996},"last":1.954}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:00:52.146 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:00:52.146 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:00:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.417+0000 7f7521800640 1 Processor -- start 2024-01-15T18:00:52.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.417+0000 7f7521800640 1 -- start start 2024-01-15T18:00:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.418+0000 7f7521800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f751c155950 0x7f751c0ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.418+0000 7f7521800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f751c0b2ce0 0x7f751c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.418+0000 7f7521800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f751c15edb0 con 0x7f751c15c440 2024-01-15T18:00:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.418+0000 7f7521800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751c0b36d0 con 0x7f751c155950 2024-01-15T18:00:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.418+0000 7f7521800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f751c0cd300 con 0x7f751c0b2ce0 2024-01-15T18:00:52.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.418+0000 7f751a000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f751c15c440 0x7f751c15f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50636/0 (socket says 172.21.15.26:50636) 2024-01-15T18:00:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.418+0000 7f751a000640 1 -- 172.21.15.26:0/753932229 learned_addr learned my addr 172.21.15.26:0/753932229 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.419+0000 7f751aa00640 1 --2- 172.21.15.26:0/753932229 >> v2:172.21.15.26:3300/0 conn(0x7f751c155950 0x7f751c0ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.419+0000 7f751b400640 1 --2- 172.21.15.26:0/753932229 >> v2:172.21.15.124:3300/0 conn(0x7f751c0b2ce0 0x7f751c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.419+0000 7f7519600640 1 -- 172.21.15.26:0/753932229 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533590792 0 0) 0x7f751c15edb0 con 0x7f751c15c440 2024-01-15T18:00:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.419+0000 7f7519600640 1 -- 172.21.15.26:0/753932229 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc003520 con 0x7f751c15c440 2024-01-15T18:00:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.419+0000 7f751aa00640 1 -- 172.21.15.26:0/753932229 >> v2:172.21.15.124:3300/0 conn(0x7f751c0b2ce0 msgr2=0x7f751c0b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.419+0000 7f751aa00640 1 --2- 172.21.15.26:0/753932229 >> v2:172.21.15.124:3300/0 conn(0x7f751c0b2ce0 0x7f751c0b3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.419+0000 7f751aa00640 1 -- 172.21.15.26:0/753932229 >> v1:172.21.15.26:6789/0 conn(0x7f751c15c440 legacy=0x7f751c15f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.419+0000 7f751aa00640 1 -- 172.21.15.26:0/753932229 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c0cd5e0 con 0x7f751c155950 2024-01-15T18:00:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.420+0000 7f751aa00640 1 --2- 172.21.15.26:0/753932229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c155950 0x7f751c0ce340 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f751c0d0690 tx=0x7f750402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac2a4c3b6970b7b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:52.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.420+0000 7f7519600640 1 -- 172.21.15.26:0/753932229 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504031830 con 0x7f751c155950 2024-01-15T18:00:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7519600640 1 -- 172.21.15.26:0/753932229 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7504031990 con 0x7f751c155950 2024-01-15T18:00:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7519600640 1 -- 172.21.15.26:0/753932229 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504031d00 con 0x7f751c155950 2024-01-15T18:00:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7521800640 1 -- 172.21.15.26:0/753932229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c155950 msgr2=0x7f751c0ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7521800640 1 --2- 172.21.15.26:0/753932229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c155950 0x7f751c0ce340 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f751c0d0690 tx=0x7f750402b8d0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7521800640 1 -- 172.21.15.26:0/753932229 shutdown_connections 2024-01-15T18:00:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7521800640 1 --2- 172.21.15.26:0/753932229 >> v2:172.21.15.124:3300/0 conn(0x7f751c0b2ce0 0x7f751c0b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7521800640 1 --2- 172.21.15.26:0/753932229 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c155950 0x7f751c0ce340 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7521800640 1 -- 172.21.15.26:0/753932229 >> 172.21.15.26:0/753932229 conn(0x7f751c066450 msgr2=0x7f751c06a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:52.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7521800640 1 -- 172.21.15.26:0/753932229 shutdown_connections 2024-01-15T18:00:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.421+0000 7f7521800640 1 -- 172.21.15.26:0/753932229 wait complete. 2024-01-15T18:00:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.423+0000 7f7521800640 1 Processor -- start 2024-01-15T18:00:52.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.423+0000 7f7521800640 1 -- start start 2024-01-15T18:00:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.424+0000 7f7521800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f751c0b2ce0 0x7f751c0c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.424+0000 7f7521800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f751c155950 0x7f751c0b6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.424+0000 7f7521800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c15c440 0x7f751c06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.424+0000 7f7521800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c0c7540 con 0x7f751c15c440 2024-01-15T18:00:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.424+0000 7f7521800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751c0c7820 con 0x7f751c0b2ce0 2024-01-15T18:00:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.424+0000 7f7521800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751c0c7b00 con 0x7f751c155950 2024-01-15T18:00:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.424+0000 7f751b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c15c440 0x7f751c06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.424+0000 7f751b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c15c440 0x7f751c06cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46338/0 (socket says 172.21.15.26:46338) 2024-01-15T18:00:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.424+0000 7f751b400640 1 -- 172.21.15.26:0/3529873664 learned_addr learned my addr 172.21.15.26:0/3529873664 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.425+0000 7f751aa00640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f751c0b2ce0 0x7f751c0c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.425+0000 7f751b400640 1 -- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f751c155950 msgr2=0x7f751c0b6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.425+0000 7f751b400640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f751c155950 0x7f751c0b6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.425+0000 7f751b400640 1 -- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f751c0b2ce0 msgr2=0x7f751c0c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.425+0000 7f751b400640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f751c0b2ce0 0x7f751c0c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.425+0000 7f751b400640 1 -- 172.21.15.26:0/3529873664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7504009400 con 0x7f751c15c440 2024-01-15T18:00:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.425+0000 7f751b400640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c15c440 0x7f751c06cb40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f751400d800 tx=0x7f751400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.426+0000 7f7512a00640 1 -- 172.21.15.26:0/3529873664 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751400b6c0 con 0x7f751c15c440 2024-01-15T18:00:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.426+0000 7f7512a00640 1 -- 172.21.15.26:0/3529873664 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751400b840 con 0x7f751c15c440 2024-01-15T18:00:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.426+0000 7f7512a00640 1 -- 172.21.15.26:0/3529873664 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751400bbf0 con 0x7f751c15c440 2024-01-15T18:00:52.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.426+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751c0c7de0 con 0x7f751c15c440 2024-01-15T18:00:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.426+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751c0b6b90 con 0x7f751c15c440 2024-01-15T18:00:52.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.427+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751c0c8300 con 0x7f751c15c440 2024-01-15T18:00:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.428+0000 7f7512a00640 1 -- 172.21.15.26:0/3529873664 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f751400bd50 con 0x7f751c15c440 2024-01-15T18:00:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.429+0000 7f7512a00640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f74f406b860 0x7f74f406dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.429+0000 7f7512a00640 1 -- 172.21.15.26:0/3529873664 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f751408ac30 con 0x7f751c15c440 2024-01-15T18:00:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.430+0000 7f751aa00640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f74f406b860 0x7f74f406dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.431+0000 7f751aa00640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f74f406b860 0x7f74f406dd20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7504036000 tx=0x7f7504008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:52.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.433+0000 7f7512a00640 1 -- 172.21.15.26:0/3529873664 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75140551d0 con 0x7f751c15c440 2024-01-15T18:00:52.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.696+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f751c067670 con 0x7f751c15c440 2024-01-15T18:00:52.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.702+0000 7f7512a00640 1 -- 172.21.15.26:0/3529873664 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f75140587f0 con 0x7f751c15c440 2024-01-15T18:00:52.705 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-15T18:00:52.705 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA== 2024-01-15T18:00:52.706 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-01-15T18:00:52.706 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-01-15T18:00:52.706 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:00:52.706 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-01-15T18:00:52.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.704+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f74f406b860 msgr2=0x7f74f406dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.704+0000 7f7521800640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f74f406b860 0x7f74f406dd20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7504036000 tx=0x7f7504008030 comp rx=0 tx=0).stop 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.704+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c15c440 msgr2=0x7f751c06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.704+0000 7f7521800640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c15c440 0x7f751c06cb40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f751400d800 tx=0x7f751400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.704+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 shutdown_connections 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.704+0000 7f7521800640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f751c155950 0x7f751c0b6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.704+0000 7f7521800640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f74f406b860 0x7f74f406dd20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.704+0000 7f7521800640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f751c15c440 0x7f751c06cb40 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.704+0000 7f7521800640 1 --2- 172.21.15.26:0/3529873664 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f751c0b2ce0 0x7f751c0c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.705+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 >> 172.21.15.26:0/3529873664 conn(0x7f751c066450 msgr2=0x7f751c0cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.705+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 shutdown_connections 2024-01-15T18:00:52.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.705+0000 7f7521800640 1 -- 172.21.15.26:0/3529873664 wait complete. 2024-01-15T18:00:52.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:00:52.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.937+0000 7f1a37800640 1 Processor -- start 2024-01-15T18:00:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.937+0000 7f1a37800640 1 -- start start 2024-01-15T18:00:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.939+0000 7f1a37800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a3013df80 0x7f1a3015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.939+0000 7f1a37800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1a300cf9a0 0x7f1a300cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.939+0000 7f1a37800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a3006c470 con 0x7f1a3013e360 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.939+0000 7f1a37800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a3006c750 con 0x7f1a300cf9a0 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.939+0000 7f1a37800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3006ca30 con 0x7f1a3013df80 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.939+0000 7f1a2fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1a3013e360 0x7f1a300cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50650/0 (socket says 172.21.15.26:50650) 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.939+0000 7f1a2fe00640 1 -- 172.21.15.26:0/3508894887 learned_addr learned my addr 172.21.15.26:0/3508894887 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a34e00640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a3013df80 0x7f1a3015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a35800640 1 --2- 172.21.15.26:0/3508894887 >> v2:172.21.15.124:3300/0 conn(0x7f1a300cf9a0 0x7f1a300cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3077325269 0 0) 0x7f1a3006c470 con 0x7f1a3013e360 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a10003520 con 0x7f1a3013e360 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3101862798 0 0) 0x7f1a10003520 con 0x7f1a3013e360 2024-01-15T18:00:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a3013df80 msgr2=0x7f1a3015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a3013df80 0x7f1a3015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 >> v2:172.21.15.124:3300/0 conn(0x7f1a300cf9a0 msgr2=0x7f1a300cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 --2- 172.21.15.26:0/3508894887 >> v2:172.21.15.124:3300/0 conn(0x7f1a300cf9a0 0x7f1a300cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3006cd10 con 0x7f1a3013e360 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f1a18001670 con 0x7f1a3013e360 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 >> v1:172.21.15.26:6789/0 conn(0x7f1a3013e360 legacy=0x7f1a300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a10003970 0x7f1a10003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a3013df80 0x7f1a100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a10004d00 0x7f1a10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3006c750 con 0x7f1a10003970 2024-01-15T18:00:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3006ca30 con 0x7f1a10004d00 2024-01-15T18:00:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.940+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1000fd90 con 0x7f1a3013df80 2024-01-15T18:00:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.941+0000 7f1a34e00640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a10003970 0x7f1a10003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.941+0000 7f1a2fe00640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a3013df80 0x7f1a100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.941+0000 7f1a34e00640 1 -- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a3013df80 msgr2=0x7f1a100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.941+0000 7f1a34e00640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a3013df80 0x7f1a100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.941+0000 7f1a34e00640 1 -- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a10004d00 msgr2=0x7f1a10005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.941+0000 7f1a35800640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a10004d00 0x7f1a10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.941+0000 7f1a34e00640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a10004d00 0x7f1a10005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.941+0000 7f1a34e00640 1 -- 172.21.15.26:0/3508894887 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10003520 con 0x7f1a10003970 2024-01-15T18:00:52.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.941+0000 7f1a2fe00640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a3013df80 0x7f1a100047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:52.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.942+0000 7f1a34e00640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a10003970 0x7f1a10003d90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009af0 tx=0x7f1a100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe25e492db2644c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:52.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.943+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2002c6b0 con 0x7f1a10003970 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.943+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2002c810 con 0x7f1a10003970 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.943+0000 7f1a2f400640 1 -- 172.21.15.26:0/3508894887 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2002cba0 con 0x7f1a10003970 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.943+0000 7f1a37800640 1 -- 172.21.15.26:0/3508894887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a10003970 msgr2=0x7f1a10003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.943+0000 7f1a37800640 1 --2- 172.21.15.26:0/3508894887 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a10003970 0x7f1a10003d90 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1a20009af0 tx=0x7f1a100057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.943+0000 7f1a35800640 1 -- 172.21.15.26:0/3508894887 reap_dead start 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.943+0000 7f1a37800640 1 -- 172.21.15.26:0/3508894887 shutdown_connections 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.943+0000 7f1a37800640 1 -- 172.21.15.26:0/3508894887 >> 172.21.15.26:0/3508894887 conn(0x7f1a3006a800 msgr2=0x7f1a3006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.943+0000 7f1a37800640 1 -- 172.21.15.26:0/3508894887 shutdown_connections 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.944+0000 7f1a37800640 1 -- 172.21.15.26:0/3508894887 wait complete. 2024-01-15T18:00:52.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.945+0000 7f1a37800640 1 Processor -- start 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.946+0000 7f1a37800640 1 -- start start 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.946+0000 7f1a37800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a10003970 0x7f1a300b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.946+0000 7f1a37800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a300ae080 0x7f1a301537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.946+0000 7f1a37800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a3013af50 0x7f1a300afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.946+0000 7f1a37800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30152410 con 0x7f1a3013af50 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.946+0000 7f1a37800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30158400 con 0x7f1a300ae080 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.946+0000 7f1a37800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a30154120 con 0x7f1a10003970 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a35800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a3013af50 0x7f1a300afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a35800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a3013af50 0x7f1a300afb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46352/0 (socket says 172.21.15.26:46352) 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a35800640 1 -- 172.21.15.26:0/3474023854 learned_addr learned my addr 172.21.15.26:0/3474023854 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:52.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a2fe00640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a300ae080 0x7f1a301537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a34e00640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a10003970 0x7f1a300b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a35800640 1 -- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a10003970 msgr2=0x7f1a300b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a35800640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a10003970 0x7f1a300b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a35800640 1 -- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a300ae080 msgr2=0x7f1a301537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a35800640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a300ae080 0x7f1a301537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a35800640 1 -- 172.21.15.26:0/3474023854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a30154400 con 0x7f1a3013af50 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a2fe00640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a300ae080 0x7f1a301537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.947+0000 7f1a35800640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a3013af50 0x7f1a300afb80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1a300cc770 tx=0x7f1a28009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.948+0000 7f1a2cc00640 1 -- 172.21.15.26:0/3474023854 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28007cf0 con 0x7f1a3013af50 2024-01-15T18:00:52.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.948+0000 7f1a2cc00640 1 -- 172.21.15.26:0/3474023854 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a28007e70 con 0x7f1a3013af50 2024-01-15T18:00:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.948+0000 7f1a2cc00640 1 -- 172.21.15.26:0/3474023854 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28014620 con 0x7f1a3013af50 2024-01-15T18:00:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.948+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a301546e0 con 0x7f1a3013af50 2024-01-15T18:00:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.948+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a30153d10 con 0x7f1a3013af50 2024-01-15T18:00:52.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.949+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a301585c0 con 0x7f1a3013af50 2024-01-15T18:00:52.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.950+0000 7f1a2cc00640 1 -- 172.21.15.26:0/3474023854 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f1a2801e440 con 0x7f1a3013af50 2024-01-15T18:00:52.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.951+0000 7f1a2cc00640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a0806b890 0x7f1a0806dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:52.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.951+0000 7f1a2cc00640 1 -- 172.21.15.26:0/3474023854 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1a2808a250 con 0x7f1a3013af50 2024-01-15T18:00:52.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.952+0000 7f1a34e00640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a0806b890 0x7f1a0806dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:52.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.953+0000 7f1a34e00640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a0806b890 0x7f1a0806dd50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1a20036000 tx=0x7f1a20030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:52.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:52.956+0000 7f1a2cc00640 1 -- 172.21.15.26:0/3474023854 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a280547f0 con 0x7f1a3013af50 2024-01-15T18:00:53.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.188+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a300675d0 con 0x7f1a3013af50 2024-01-15T18:00:53.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.189+0000 7f1a2cc00640 1 -- 172.21.15.26:0/3474023854 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f1a28003c40 con 0x7f1a3013af50 2024-01-15T18:00:53.193 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:00:53.194 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1159022909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1159022909},{"type":"v1","addr":"172.21.15.26:6835","nonce":1159022909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/248571798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":248571798},{"type":"v1","addr":"172.21.15.124:6837","nonce":248571798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3357264964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3357264964},{"type":"v1","addr":"172.21.15.26:6833","nonce":3357264964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:00:41.518292+0000","modified":"2024-01-15T18:00:44.577344+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":35186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_35186":{"gid":35186,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6836/3403655022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3403655022},{"type":"v1","addr":"172.21.15.124:6836","nonce":3403655022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-01-15T18:00:53.194 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 35 2024-01-15T18:00:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.192+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a0806b890 msgr2=0x7f1a0806dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.192+0000 7f1a37800640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a0806b890 0x7f1a0806dd50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1a20036000 tx=0x7f1a20030000 comp rx=0 tx=0).stop 2024-01-15T18:00:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.192+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a3013af50 msgr2=0x7f1a300afb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.192+0000 7f1a37800640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a3013af50 0x7f1a300afb80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1a300cc770 tx=0x7f1a28009cb0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.192+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 shutdown_connections 2024-01-15T18:00:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.192+0000 7f1a37800640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a0806b890 0x7f1a0806dd50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.192+0000 7f1a37800640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a3013af50 0x7f1a300afb80 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.192+0000 7f1a37800640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a300ae080 0x7f1a301537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.192+0000 7f1a37800640 1 --2- 172.21.15.26:0/3474023854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a10003970 0x7f1a300b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.193+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 >> 172.21.15.26:0/3474023854 conn(0x7f1a3006a800 msgr2=0x7f1a300ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.193+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 shutdown_connections 2024-01-15T18:00:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.193+0000 7f1a37800640 1 -- 172.21.15.26:0/3474023854 wait complete. 2024-01-15T18:00:53.619 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:00:41.518292+0000', 'modified': '2024-01-15T18:00:44.577344+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 35186}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_35186': {'gid': 35186, 'name': 'd', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6836/3403655022', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6834', 'nonce': 3403655022}, {'type': 'v1', 'addr': '172.21.15.124:6836', 'nonce': 3403655022}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:00:53.619 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:00:53.619 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-15T18:00:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.846+0000 7f88d9400640 1 Processor -- start 2024-01-15T18:00:53.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.847+0000 7f88d9400640 1 -- start start 2024-01-15T18:00:53.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.848+0000 7f88d9400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d4143c40 0x7f88d4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.848+0000 7f88d9400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f88d40717d0 0x7f88d40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.848+0000 7f88d9400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88d40cca40 0x7f88d415eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.848+0000 7f88d9400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d40723c0 con 0x7f88d40cca40 2024-01-15T18:00:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.848+0000 7f88d9400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d4070e50 con 0x7f88d40717d0 2024-01-15T18:00:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.848+0000 7f88d9400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40cce70 con 0x7f88d4143c40 2024-01-15T18:00:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d8a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88d40cca40 0x7f88d415eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d8a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f88d40cca40 0x7f88d415eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46362/0 (socket says 172.21.15.26:46362) 2024-01-15T18:00:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d8a00640 1 -- 172.21.15.26:0/3263931610 learned_addr learned my addr 172.21.15.26:0/3263931610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:53.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d3e00640 1 --2- 172.21.15.26:0/3263931610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d4143c40 0x7f88d4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d3400640 1 --2- 172.21.15.26:0/3263931610 >> v2:172.21.15.124:3300/0 conn(0x7f88d40717d0 0x7f88d40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d8a00640 1 -- 172.21.15.26:0/3263931610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d4143c40 msgr2=0x7f88d4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d8a00640 1 --2- 172.21.15.26:0/3263931610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d4143c40 0x7f88d4071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d8a00640 1 -- 172.21.15.26:0/3263931610 >> v2:172.21.15.124:3300/0 conn(0x7f88d40717d0 msgr2=0x7f88d40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d8a00640 1 --2- 172.21.15.26:0/3263931610 >> v2:172.21.15.124:3300/0 conn(0x7f88d40717d0 0x7f88d40cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d8a00640 1 -- 172.21.15.26:0/3263931610 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d4144020 con 0x7f88d40cca40 2024-01-15T18:00:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d3e00640 1 --2- 172.21.15.26:0/3263931610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d4143c40 0x7f88d4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.849+0000 7f88d3400640 1 --2- 172.21.15.26:0/3263931610 >> v2:172.21.15.124:3300/0 conn(0x7f88d40717d0 0x7f88d40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:53.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.850+0000 7f88d8a00640 1 --2- 172.21.15.26:0/3263931610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d40cca40 0x7f88d415eab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f88d406e440 tx=0x7f88cc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de82b2bf3345f0c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:53.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d2a00640 1 -- 172.21.15.26:0/3263931610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0317d0 con 0x7f88d40cca40 2024-01-15T18:00:53.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d2a00640 1 -- 172.21.15.26:0/3263931610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc031930 con 0x7f88d40cca40 2024-01-15T18:00:53.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d2a00640 1 -- 172.21.15.26:0/3263931610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc031cc0 con 0x7f88d40cca40 2024-01-15T18:00:53.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d9400640 1 -- 172.21.15.26:0/3263931610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d40cca40 msgr2=0x7f88d415eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d9400640 1 --2- 172.21.15.26:0/3263931610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d40cca40 0x7f88d415eab0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f88d406e440 tx=0x7f88cc02b870 comp rx=0 tx=0).stop 2024-01-15T18:00:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d9400640 1 -- 172.21.15.26:0/3263931610 shutdown_connections 2024-01-15T18:00:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d9400640 1 --2- 172.21.15.26:0/3263931610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d40cca40 0x7f88d415eab0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d9400640 1 --2- 172.21.15.26:0/3263931610 >> v2:172.21.15.124:3300/0 conn(0x7f88d40717d0 0x7f88d40cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d9400640 1 --2- 172.21.15.26:0/3263931610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d4143c40 0x7f88d4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d9400640 1 -- 172.21.15.26:0/3263931610 >> 172.21.15.26:0/3263931610 conn(0x7f88d406a800 msgr2=0x7f88d406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.851+0000 7f88d9400640 1 -- 172.21.15.26:0/3263931610 shutdown_connections 2024-01-15T18:00:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.852+0000 7f88d9400640 1 -- 172.21.15.26:0/3263931610 wait complete. 2024-01-15T18:00:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.853+0000 7f88d9400640 1 Processor -- start 2024-01-15T18:00:53.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.853+0000 7f88d9400640 1 -- start start 2024-01-15T18:00:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.854+0000 7f88d9400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88d40717d0 0x7f88d40a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.854+0000 7f88d9400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d40cca40 0x7f88d41e3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.854+0000 7f88d9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4143c40 0x7f88d40ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.854+0000 7f88d9400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40aad50 con 0x7f88d4143c40 2024-01-15T18:00:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.854+0000 7f88d9400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40a85f0 con 0x7f88d40717d0 2024-01-15T18:00:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.854+0000 7f88d9400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40a88d0 con 0x7f88d40cca40 2024-01-15T18:00:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4143c40 0x7f88d40ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4143c40 0x7f88d40ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46376/0 (socket says 172.21.15.26:46376) 2024-01-15T18:00:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d8a00640 1 -- 172.21.15.26:0/1041411444 learned_addr learned my addr 172.21.15.26:0/1041411444 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d8a00640 1 -- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d40cca40 msgr2=0x7f88d41e3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d3400640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d40cca40 0x7f88d41e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d3e00640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88d40717d0 0x7f88d40a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d8a00640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d40cca40 0x7f88d41e3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d8a00640 1 -- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88d40717d0 msgr2=0x7f88d40a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d8a00640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88d40717d0 0x7f88d40a93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d8a00640 1 -- 172.21.15.26:0/1041411444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc0094d0 con 0x7f88d4143c40 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.855+0000 7f88d3400640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d40cca40 0x7f88d41e3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.856+0000 7f88d8a00640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4143c40 0x7f88d40ada30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0090b0 tx=0x7f88cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.856+0000 7f88cbe00640 1 -- 172.21.15.26:0/1041411444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0329b0 con 0x7f88d4143c40 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.856+0000 7f88cbe00640 1 -- 172.21.15.26:0/1041411444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc032b10 con 0x7f88d4143c40 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.856+0000 7f88cbe00640 1 -- 172.21.15.26:0/1041411444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0453f0 con 0x7f88d4143c40 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.856+0000 7f88d9400640 1 -- 172.21.15.26:0/1041411444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d40a8bb0 con 0x7f88d4143c40 2024-01-15T18:00:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.856+0000 7f88d9400640 1 -- 172.21.15.26:0/1041411444 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d40adf70 con 0x7f88d4143c40 2024-01-15T18:00:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.859+0000 7f88cbe00640 1 -- 172.21.15.26:0/1041411444 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f88cc030070 con 0x7f88d4143c40 2024-01-15T18:00:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.859+0000 7f88cbe00640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f88ac06b860 0x7f88ac06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.860+0000 7f88cbe00640 1 -- 172.21.15.26:0/1041411444 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f88cc0ac670 con 0x7f88d4143c40 2024-01-15T18:00:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.860+0000 7f88dd7f3740 1 Processor -- start 2024-01-15T18:00:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.860+0000 7f88d3e00640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f88ac06b860 0x7f88ac06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.860+0000 7f88dd7f3740 1 -- start start 2024-01-15T18:00:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.861+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c40d070 0x563b9c40d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.861+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c407d70 0x563b9c40c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.861+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c450730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.861+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9c36b880 con 0x563b9c40c7b0 2024-01-15T18:00:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.861+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9c37aae0 con 0x563b9c407d70 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.861+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b9c3781d0 con 0x563b9c40d070 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c450730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c450730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46382/0 (socket says 172.21.15.26:46382) 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d8a00640 1 -- 172.21.15.26:0/3616591575 learned_addr learned my addr 172.21.15.26:0/3616591575 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d3e00640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f88ac06b860 0x7f88ac06dd20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f88c4008f10 tx=0x7f88c4008460 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d3400640 1 --2- 172.21.15.26:0/3616591575 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c407d70 0x563b9c40c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d8a00640 1 -- 172.21.15.26:0/3616591575 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c40d070 msgr2=0x563b9c40d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d8a00640 1 --2- 172.21.15.26:0/3616591575 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c40d070 0x563b9c40d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d8a00640 1 -- 172.21.15.26:0/3616591575 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c407d70 msgr2=0x563b9c40c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d8a00640 1 --2- 172.21.15.26:0/3616591575 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c407d70 0x563b9c40c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d8a00640 1 -- 172.21.15.26:0/3616591575 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b9c33b860 con 0x563b9c40c7b0 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88d8a00640 1 --2- 172.21.15.26:0/3616591575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c450730 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f88cc07a6b0 tx=0x7f88cc07a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6147367e8f7a2818 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88c8c00640 1 -- 172.21.15.26:0/3616591575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0aa1d0 con 0x563b9c40c7b0 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88c8c00640 1 -- 172.21.15.26:0/3616591575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc078d50 con 0x563b9c40c7b0 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.862+0000 7f88c8c00640 1 -- 172.21.15.26:0/3616591575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc079ea0 con 0x563b9c40c7b0 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.863+0000 7f88dd7f3740 1 -- 172.21.15.26:0/3616591575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 msgr2=0x563b9c450730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.863+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/3616591575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c450730 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f88cc07a6b0 tx=0x7f88cc07a8b0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.863+0000 7f88dd7f3740 1 -- 172.21.15.26:0/3616591575 shutdown_connections 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.863+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/3616591575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c450730 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.863+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/3616591575 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c407d70 0x563b9c40c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.863+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/3616591575 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c40d070 0x563b9c40d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.863+0000 7f88dd7f3740 1 -- 172.21.15.26:0/3616591575 >> 172.21.15.26:0/3616591575 conn(0x563b9c3cec70 msgr2=0x563b9c452b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:53.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.863+0000 7f88dd7f3740 1 -- 172.21.15.26:0/3616591575 shutdown_connections 2024-01-15T18:00:53.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.863+0000 7f88dd7f3740 1 -- 172.21.15.26:0/3616591575 wait complete. 2024-01-15T18:00:53.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.864+0000 7f88dd7f3740 1 -- start start 2024-01-15T18:00:53.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.864+0000 7f88dd7f3740 1 Processor -- start 2024-01-15T18:00:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.865+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 0x563b9c435470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.865+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c433470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.865+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c4659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.865+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9c373ac0 con 0x563b9c40c7b0 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.865+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9c3102f0 con 0x563b9c40d070 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.865+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b9c32dc70 con 0x563b9c407d70 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.865+0000 7f88dd7f3740 10 client.0 _ll_register_callbacks cb 0x563b9c3323b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.865+0000 7f88d3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c433470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.865+0000 7f88d3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c433470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46384/0 (socket says 172.21.15.26:46384) 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88d3400640 1 -- 172.21.15.26:0/4169635524 learned_addr learned my addr 172.21.15.26:0/4169635524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88d3e00640 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 0x563b9c435470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88d3400640 1 -- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 msgr2=0x563b9c435470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88d3400640 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 0x563b9c435470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88d3400640 1 -- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 msgr2=0x563b9c4659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88d3400640 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c4659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88d3400640 1 -- 172.21.15.26:0/4169635524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d40adf70 con 0x563b9c40c7b0 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88d3e00640 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 0x563b9c435470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88d3400640 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c433470 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e3b0 tx=0x7f88bc00e870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d862ec512bab0a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.866+0000 7f88c1600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88c1600640 1 -- 172.21.15.26:0/4169635524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc012840 con 0x563b9c40c7b0 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88dd7f3740 1 -- 172.21.15.26:0/4169635524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b9c36caa0 con 0x563b9c40c7b0 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88c1600640 1 -- 172.21.15.26:0/4169635524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f88bc00b030 con 0x563b9c40c7b0 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88c1600640 1 -- 172.21.15.26:0/4169635524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88bc0129a0 con 0x563b9c40c7b0 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88dd7f3740 10 client.6567 fetch_fsmap learned FSMap version 35 2024-01-15T18:00:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88dd7f3740 1 -- 172.21.15.26:0/4169635524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x563b9c372140 con 0x563b9c40c7b0 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88c1600640 1 -- 172.21.15.26:0/4169635524 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563b9c36caa0 con 0x563b9c40c7b0 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88c1600640 1 -- 172.21.15.26:0/4169635524 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f88bc018020 con 0x563b9c40c7b0 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88dd7f3740 10 client.6567 fetch_fsmap finished waiting for FSMap version 35 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88dd7f3740 10 client.6567 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88dd7f3740 20 client.6567 populate_metadata read hostname 'smithi026' 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] conn(0x563b9c459460 0x563b9c46fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88dd7f3740 4 client.6567 mds_command: new command op to 35186 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.867+0000 7f88dd7f3740 1 -- 172.21.15.26:0/4169635524 --> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b9c36c7c0 con 0x563b9c459460 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.868+0000 7f88d3e00640 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] conn(0x563b9c459460 0x563b9c46fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.869+0000 7f88d3e00640 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] conn(0x563b9c459460 0x563b9c46fe10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1939ad404e5471ac server_cookie=2bc35e161f9f248c in_seq=0 out_seq=0 2024-01-15T18:00:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.869+0000 7f88c1600640 10 client.6567 ms_handle_connect on v2:172.21.15.124:6834/3403655022 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88c1600640 1 -- 172.21.15.26:0/4169635524 <== mds.0 v2:172.21.15.124:6834/3403655022 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b9c36c7c0 con 0x563b9c459460 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88c1600640 10 client.6567 handle_command_reply: tid=0 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 client.6567 shutdown 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 -- 172.21.15.26:0/4169635524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 msgr2=0x563b9c433470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c433470 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e3b0 tx=0x7f88bc00e870 comp rx=0 tx=0).stop 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 -- 172.21.15.26:0/4169635524 shutdown_connections 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] conn(0x563b9c459460 0x563b9c46fe10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c4659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c433470 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/4169635524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 0x563b9c435470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88c1600640 0 client.6567 ms_handle_reset on v2:172.21.15.124:6834/3403655022 2024-01-15T18:00:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 -- 172.21.15.26:0/4169635524 >> 172.21.15.26:0/4169635524 conn(0x563b9c3cec70 msgr2=0x563b9c406dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 -- 172.21.15.26:0/4169635524 shutdown_connections 2024-01-15T18:00:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.871+0000 7f88dd7f3740 1 -- 172.21.15.26:0/4169635524 wait complete. 2024-01-15T18:00:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.872+0000 7f88dd7f3740 20 client.6567 trim_cache size 0 max 16384 2024-01-15T18:00:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.889+0000 7f88dd7f3740 1 Processor -- start 2024-01-15T18:00:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.889+0000 7f88dd7f3740 1 -- start start 2024-01-15T18:00:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c407d70 0x563b9c406f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c40c7b0 0x563b9c435d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c4362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9c0d6f20 con 0x563b9c407d70 2024-01-15T18:00:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9c403bb0 con 0x563b9c40d070 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b9c408330 con 0x563b9c40c7b0 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c407d70 0x563b9c406f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c407d70 0x563b9c406f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46386/0 (socket says 172.21.15.26:46386) 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88d3e00640 1 -- 172.21.15.26:0/2702906567 learned_addr learned my addr 172.21.15.26:0/2702906567 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88d3e00640 1 -- 172.21.15.26:0/2702906567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c40c7b0 msgr2=0x563b9c435d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88d3e00640 1 --2- 172.21.15.26:0/2702906567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c40c7b0 0x563b9c435d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88d3e00640 1 -- 172.21.15.26:0/2702906567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 msgr2=0x563b9c4362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88d3e00640 1 --2- 172.21.15.26:0/2702906567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c4362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.890+0000 7f88d3e00640 1 -- 172.21.15.26:0/2702906567 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b9c30fb80 con 0x563b9c407d70 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.891+0000 7f88d3e00640 1 --2- 172.21.15.26:0/2702906567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c407d70 0x563b9c406f80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f88c400b560 tx=0x7f88c4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f096130657219c4c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.891+0000 7f88c9600640 1 -- 172.21.15.26:0/2702906567 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c401f040 con 0x563b9c407d70 2024-01-15T18:00:53.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.891+0000 7f88c9600640 1 -- 172.21.15.26:0/2702906567 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c40212b0 con 0x563b9c407d70 2024-01-15T18:00:53.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.891+0000 7f88c9600640 1 -- 172.21.15.26:0/2702906567 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4021620 con 0x563b9c407d70 2024-01-15T18:00:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.891+0000 7f88dd7f3740 1 -- 172.21.15.26:0/2702906567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c407d70 msgr2=0x563b9c406f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.891+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/2702906567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c407d70 0x563b9c406f80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f88c400b560 tx=0x7f88c4010000 comp rx=0 tx=0).stop 2024-01-15T18:00:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.892+0000 7f88dd7f3740 1 -- 172.21.15.26:0/2702906567 shutdown_connections 2024-01-15T18:00:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.892+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/2702906567 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c40c7b0 0x563b9c435d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.892+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/2702906567 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c4362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.892+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/2702906567 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c407d70 0x563b9c406f80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.892+0000 7f88dd7f3740 1 -- 172.21.15.26:0/2702906567 >> 172.21.15.26:0/2702906567 conn(0x563b9c3cec70 msgr2=0x563b9c434280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.892+0000 7f88dd7f3740 1 -- 172.21.15.26:0/2702906567 shutdown_connections 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.892+0000 7f88dd7f3740 1 -- 172.21.15.26:0/2702906567 wait complete. 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.892+0000 7f88dd7f3740 1 -- start start 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.893+0000 7f88dd7f3740 1 Processor -- start 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 0x563b9c43b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c473e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88dd7f3740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c451970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9c44e520 con 0x563b9c40c7b0 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b9c433de0 con 0x563b9c40d070 2024-01-15T18:00:53.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88dd7f3740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b9c44e980 con 0x563b9c407d70 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88dd7f3740 10 client.0 _ll_register_callbacks cb 0x7f88bc00cca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88d3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c473e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88d3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c473e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46392/0 (socket says 172.21.15.26:46392) 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88d3400640 1 -- 172.21.15.26:0/186887638 learned_addr learned my addr 172.21.15.26:0/186887638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88d3400640 1 -- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 msgr2=0x563b9c43b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88d3400640 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 0x563b9c43b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88d3400640 1 -- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 msgr2=0x563b9c451970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88d8a00640 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c451970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88d3400640 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c451970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.894+0000 7f88d3400640 1 -- 172.21.15.26:0/186887638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c4013070 con 0x563b9c40c7b0 2024-01-15T18:00:53.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88d3400640 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c473e30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e3b0 tx=0x7f88bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5b380e8c0e5ed9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88c1600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88c1600640 1 -- 172.21.15.26:0/186887638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc028620 con 0x563b9c40c7b0 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88c1600640 1 -- 172.21.15.26:0/186887638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f88bc00b040 con 0x563b9c40c7b0 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88dd7f3740 1 -- 172.21.15.26:0/186887638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88cc078d50 con 0x563b9c40c7b0 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88c1600640 1 -- 172.21.15.26:0/186887638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88bc012e80 con 0x563b9c40c7b0 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88dd7f3740 10 client.6573 fetch_fsmap learned FSMap version 35 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88dd7f3740 1 -- 172.21.15.26:0/186887638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x563b9c335170 con 0x563b9c40c7b0 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88c1600640 1 -- 172.21.15.26:0/186887638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc078d50 con 0x563b9c40c7b0 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.895+0000 7f88c1600640 1 -- 172.21.15.26:0/186887638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f88bc032070 con 0x563b9c40c7b0 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.896+0000 7f88dd7f3740 10 client.6573 fetch_fsmap finished waiting for FSMap version 35 2024-01-15T18:00:53.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.896+0000 7f88dd7f3740 10 client.6573 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:00:53.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.896+0000 7f88dd7f3740 20 client.6573 populate_metadata read hostname 'smithi026' 2024-01-15T18:00:53.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.896+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] conn(0x563b9c4bc530 0x563b9c451eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:53.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.896+0000 7f88dd7f3740 4 client.6573 mds_command: new command op to 35186 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:00:53.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.896+0000 7f88dd7f3740 1 -- 172.21.15.26:0/186887638 --> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563b9c0746c0 con 0x563b9c4bc530 2024-01-15T18:00:53.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.896+0000 7f88d3e00640 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] conn(0x563b9c4bc530 0x563b9c451eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:53.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.897+0000 7f88d3e00640 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] conn(0x563b9c4bc530 0x563b9c451eb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=175ffee6612ffbc2 server_cookie=5f548876d045bfc8 in_seq=0 out_seq=0 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.897+0000 7f88c1600640 10 client.6573 ms_handle_connect on v2:172.21.15.124:6834/3403655022 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88c1600640 1 -- 172.21.15.26:0/186887638 <== mds.0 v2:172.21.15.124:6834/3403655022 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x563b9c0746c0 con 0x563b9c4bc530 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88c1600640 10 client.6573 handle_command_reply: tid=0 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88dd7f3740 1 client.6573 shutdown 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88dd7f3740 1 -- 172.21.15.26:0/186887638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 msgr2=0x563b9c473e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c473e30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00e3b0 tx=0x7f88bc004730 comp rx=0 tx=0).stop 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88dd7f3740 1 -- 172.21.15.26:0/186887638 shutdown_connections 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:6834/3403655022,v1:172.21.15.124:6836/3403655022] conn(0x563b9c4bc530 0x563b9c451eb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x563b9c40d070 0x563b9c451970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x563b9c40c7b0 0x563b9c473e30 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88dd7f3740 1 --2- 172.21.15.26:0/186887638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x563b9c407d70 0x563b9c43b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88c1600640 0 client.6573 ms_handle_reset on v2:172.21.15.124:6834/3403655022 2024-01-15T18:00:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.898+0000 7f88dd7f3740 1 -- 172.21.15.26:0/186887638 >> 172.21.15.26:0/186887638 conn(0x563b9c3cec70 msgr2=0x563b9c434260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88dd7f3740 1 -- 172.21.15.26:0/186887638 shutdown_connections 2024-01-15T18:00:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88dd7f3740 1 -- 172.21.15.26:0/186887638 wait complete. 2024-01-15T18:00:53.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88dd7f3740 20 client.6573 trim_cache size 0 max 16384 2024-01-15T18:00:53.907 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:00:53.907 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 35186, 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 11.840762826000001, 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 35, 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 60, 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 60, 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 17.225399662000001 2024-01-15T18:00:53.908 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 -- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f88ac06b860 msgr2=0x7f88ac06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f88ac06b860 0x7f88ac06dd20 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f88c4008f10 tx=0x7f88c4008460 comp rx=0 tx=0).stop 2024-01-15T18:00:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 -- 172.21.15.26:0/1041411444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4143c40 msgr2=0x7f88d40ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4143c40 0x7f88d40ada30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0090b0 tx=0x7f88cc0046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 -- 172.21.15.26:0/1041411444 shutdown_connections 2024-01-15T18:00:53.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88d40cca40 0x7f88d41e3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f88ac06b860 0x7f88ac06dd20 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f88d4143c40 0x7f88d40ada30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 --2- 172.21.15.26:0/1041411444 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f88d40717d0 0x7f88d40a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.899+0000 7f88d9400640 1 -- 172.21.15.26:0/1041411444 >> 172.21.15.26:0/1041411444 conn(0x7f88d406a800 msgr2=0x7f88d406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.900+0000 7f88d9400640 1 -- 172.21.15.26:0/1041411444 shutdown_connections 2024-01-15T18:00:53.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:53.900+0000 7f88d9400640 1 -- 172.21.15.26:0/1041411444 wait complete. 2024-01-15T18:00:53.922 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:00:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.142+0000 7f9a1a600640 1 Processor -- start 2024-01-15T18:00:54.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.143+0000 7f9a1a600640 1 -- start start 2024-01-15T18:00:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.144+0000 7f9a1a600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 0x7f9a1415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.144+0000 7f9a1a600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a140cf290 0x7f9a140cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.144+0000 7f9a1a600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a14070c40 con 0x7f9a140ae5a0 2024-01-15T18:00:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.144+0000 7f9a1a600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a14071450 con 0x7f9a140cf290 2024-01-15T18:00:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.144+0000 7f9a1a600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140cfc60 con 0x7f9a140b3ae0 2024-01-15T18:00:54.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.145+0000 7f9a13400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9a140ae5a0 0x7f9a1406cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50662/0 (socket says 172.21.15.26:50662) 2024-01-15T18:00:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.145+0000 7f9a13400640 1 -- 172.21.15.26:0/808390051 learned_addr learned my addr 172.21.15.26:0/808390051 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.145+0000 7f9a13e00640 1 --2- 172.21.15.26:0/808390051 >> v2:172.21.15.26:3300/0 conn(0x7f9a140cf290 0x7f9a140cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.145+0000 7f9a12a00640 1 --2- 172.21.15.26:0/808390051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 0x7f9a1415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.145+0000 7f9a12000640 1 -- 172.21.15.26:0/808390051 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065030526 0 0) 0x7f9a14070c40 con 0x7f9a140ae5a0 2024-01-15T18:00:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.145+0000 7f9a12000640 1 -- 172.21.15.26:0/808390051 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f4003520 con 0x7f9a140ae5a0 2024-01-15T18:00:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.146+0000 7f9a13e00640 1 -- 172.21.15.26:0/808390051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 msgr2=0x7f9a1415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.146+0000 7f9a13e00640 1 --2- 172.21.15.26:0/808390051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 0x7f9a1415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.146+0000 7f9a13e00640 1 -- 172.21.15.26:0/808390051 >> v1:172.21.15.26:6789/0 conn(0x7f9a140ae5a0 legacy=0x7f9a1406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:54.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.146+0000 7f9a13e00640 1 -- 172.21.15.26:0/808390051 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1406c470 con 0x7f9a140cf290 2024-01-15T18:00:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.147+0000 7f9a13e00640 1 --2- 172.21.15.26:0/808390051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 0x7f9a140cf720 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0097a0 tx=0x7f9a0c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637d799e7782f19d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.147+0000 7f9a12000640 1 -- 172.21.15.26:0/808390051 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c030610 con 0x7f9a140cf290 2024-01-15T18:00:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.147+0000 7f9a12000640 1 -- 172.21.15.26:0/808390051 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c030770 con 0x7f9a140cf290 2024-01-15T18:00:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.147+0000 7f9a12000640 1 -- 172.21.15.26:0/808390051 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c030ae0 con 0x7f9a140cf290 2024-01-15T18:00:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.148+0000 7f9a1a600640 1 -- 172.21.15.26:0/808390051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 msgr2=0x7f9a140cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:54.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.148+0000 7f9a1a600640 1 --2- 172.21.15.26:0/808390051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 0x7f9a140cf720 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0097a0 tx=0x7f9a0c02b960 comp rx=0 tx=0).stop 2024-01-15T18:00:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.148+0000 7f9a1a600640 1 -- 172.21.15.26:0/808390051 shutdown_connections 2024-01-15T18:00:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.148+0000 7f9a1a600640 1 --2- 172.21.15.26:0/808390051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 0x7f9a140cf720 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.148+0000 7f9a1a600640 1 --2- 172.21.15.26:0/808390051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 0x7f9a1415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.148+0000 7f9a1a600640 1 -- 172.21.15.26:0/808390051 >> 172.21.15.26:0/808390051 conn(0x7f9a1406a800 msgr2=0x7f9a1406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.148+0000 7f9a1a600640 1 -- 172.21.15.26:0/808390051 shutdown_connections 2024-01-15T18:00:54.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.148+0000 7f9a1a600640 1 -- 172.21.15.26:0/808390051 wait complete. 2024-01-15T18:00:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.150+0000 7f9a1a600640 1 Processor -- start 2024-01-15T18:00:54.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.150+0000 7f9a1a600640 1 -- start start 2024-01-15T18:00:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.151+0000 7f9a1a600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a140ae5a0 0x7f9a141536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.151+0000 7f9a1a600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 0x7f9a14152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.151+0000 7f9a1a600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 0x7f9a1414f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.151+0000 7f9a1a600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a14153bf0 con 0x7f9a140cf290 2024-01-15T18:00:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.151+0000 7f9a1a600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1415ef50 con 0x7f9a140ae5a0 2024-01-15T18:00:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.151+0000 7f9a1a600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1415f1f0 con 0x7f9a140b3ae0 2024-01-15T18:00:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a13e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 0x7f9a1414f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:54.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a13e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 0x7f9a1414f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46406/0 (socket says 172.21.15.26:46406) 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a13e00640 1 -- 172.21.15.26:0/406967536 learned_addr learned my addr 172.21.15.26:0/406967536 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a13400640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a140ae5a0 0x7f9a141536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a12a00640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 0x7f9a14152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a13e00640 1 -- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 msgr2=0x7f9a14152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a13e00640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 0x7f9a14152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a13e00640 1 -- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a140ae5a0 msgr2=0x7f9a141536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a13e00640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a140ae5a0 0x7f9a141536b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.152+0000 7f9a13e00640 1 -- 172.21.15.26:0/406967536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c009400 con 0x7f9a140cf290 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.153+0000 7f9a13e00640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 0x7f9a1414f5c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0097a0 tx=0x7f9a0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.153+0000 7f9a0b400640 1 -- 172.21.15.26:0/406967536 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c03e070 con 0x7f9a140cf290 2024-01-15T18:00:54.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.153+0000 7f9a0b400640 1 -- 172.21.15.26:0/406967536 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c0394d0 con 0x7f9a140cf290 2024-01-15T18:00:54.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.153+0000 7f9a0b400640 1 -- 172.21.15.26:0/406967536 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c039860 con 0x7f9a140cf290 2024-01-15T18:00:54.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.153+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1415f4d0 con 0x7f9a140cf290 2024-01-15T18:00:54.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.153+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a14153410 con 0x7f9a140cf290 2024-01-15T18:00:54.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.154+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1415f9f0 con 0x7f9a140cf290 2024-01-15T18:00:54.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.155+0000 7f9a0b400640 1 -- 172.21.15.26:0/406967536 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f9a0c042070 con 0x7f9a140cf290 2024-01-15T18:00:54.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.156+0000 7f9a0b400640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99ec06b890 0x7f99ec06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:54.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.156+0000 7f9a0b400640 1 -- 172.21.15.26:0/406967536 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9a0c0ac3b0 con 0x7f9a140cf290 2024-01-15T18:00:54.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.157+0000 7f9a13400640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99ec06b890 0x7f99ec06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:54.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.158+0000 7f9a13400640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99ec06b890 0x7f99ec06dd50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9a04005880 tx=0x7f9a04009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:54.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.161+0000 7f9a0b400640 1 -- 172.21.15.26:0/406967536 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a1415f9f0 con 0x7f9a140cf290 2024-01-15T18:00:54.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.396+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9a1415f9f0 con 0x7f9a140cf290 2024-01-15T18:00:54.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.397+0000 7f9a0b400640 1 -- 172.21.15.26:0/406967536 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9a1415f9f0 con 0x7f9a140cf290 2024-01-15T18:00:54.400 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:00:54.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99ec06b890 msgr2=0x7f99ec06dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:54.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99ec06b890 0x7f99ec06dd50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9a04005880 tx=0x7f9a04009040 comp rx=0 tx=0).stop 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 msgr2=0x7f9a1414f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 0x7f9a1414f5c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c0097a0 tx=0x7f9a0c0046c0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 shutdown_connections 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a140b3ae0 0x7f9a14152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99ec06b890 0x7f99ec06dd50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a140cf290 0x7f9a1414f5c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 --2- 172.21.15.26:0/406967536 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a140ae5a0 0x7f9a141536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.399+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 >> 172.21.15.26:0/406967536 conn(0x7f9a1406a800 msgr2=0x7f9a140b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.400+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 shutdown_connections 2024-01-15T18:00:54.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:54.400+0000 7f9a1a600640 1 -- 172.21.15.26:0/406967536 wait complete. 2024-01-15T18:00:54.423 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:00:54.423 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:00:54.423 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-15T18:00:54.423 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:00:54.423 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:00:54.423 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:00:54.423 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:00:54.423 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:00:54.423 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:00:54.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:54.424 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-15T18:00:54.479 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:00:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:00:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:00:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T18:00:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:54.479 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:00:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:54.479 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42263sec preferred_lft 42263sec 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-01-15T18:00:54.480 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::a4e5:3ff:fe91:fcad/64 scope link 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:00:54.481 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:00:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::f464:8fff:fecd:58d2/64 scope link 2024-01-15T18:00:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:54.482 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:00:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether fe:82:c8:bc:3f:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:00:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::fc82:c8ff:febc:3fd2/64 scope link 2024-01-15T18:00:54.482 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:54.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:54.483 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:00:54.582 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-15T18:00:54.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:54.588 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-15T18:00:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-15T18:00:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:00:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-15T18:00:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-15T18:00:54.646 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:00:54.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:54.647 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T18:00:54.647 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T18:00:54.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-15T18:00:54.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-15T18:00:54.647 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-15T18:00:54.647 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T18:00:54.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:54.847 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T18:00:54.903 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:00:54.903 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:00:54.904 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:00:54.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:54.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-15T18:00:55.006 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:00:55.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:00:55.006 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:00:55.063 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:00:55.064 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:00:55.064 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:00:55.120 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:00:55.220 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:00:55.278 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:00:55.278 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:00:55.278 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:00:55.279 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:00:55.279 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:00:55.279 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:00:55.279 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:00:55.279 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:00:55.279 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:00:55.279 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:00:55.279 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:00:55.280 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:00:55.280 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:00:55.280 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:00:55.280 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:00:55.280 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:00:55.281 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:00:55.281 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:00:55.281 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:00:55.281 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:00:55.281 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:00:55.281 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:00:55.281 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:00:55.281 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:00:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:00:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:00:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:00:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:00:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:00:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:00:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:00:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:00:55.282 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:00:55.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:55.284 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:00:55.339 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:00:55.340 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:00:55.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:00:55.425 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:00:55.468 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:00:55.468 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:00:55.469 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:00:55.469 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:00:55.469 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:00:55.469 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:00:55.469 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:00:55.469 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:00:55.469 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:00:55.469 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:00:55.469 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:00:55.470 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:00:55.470 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:00:55.470 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:00:55.470 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:00:55.470 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:00:55.470 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:00:55.470 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:00:55.470 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:00:55.471 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:00:55.471 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:00:55.471 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:00:55.471 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:00:55.471 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:00:55.471 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:00:55.471 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:00:55.471 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:00:55.471 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:00:55.472 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:00:55.472 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:00:55.472 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:00:55.472 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:00:55.472 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:00:55.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:55.473 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:00:55.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:2024-01-15T18:00:55.535+0000 7fd317840480 -1 init, newargv = 0x55608922c740 newargc=15 2024-01-15T18:00:55.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[57105]: starting ceph client 2024-01-15T18:00:55.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[57105]: starting fuse 2024-01-15T18:00:55.561 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T18:00:55.562 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:00:55.562 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:00:55.562 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-01-15T18:00:55.562 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-01-15T18:00:55.562 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-01-15T18:00:55.562 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-01-15T18:00:55.562 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:00:55.563 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-01-15T18:00:55.563 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:00:55.563 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-01-15T18:00:55.563 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:00:55.563 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:00:55.563 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-01-15T18:00:55.563 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-01-15T18:00:55.563 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-01-15T18:00:55.564 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-01-15T18:00:55.564 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:00:55.564 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-01-15T18:00:55.564 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:00:55.564 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:00:55.564 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:00:55.564 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-01-15T18:00:55.564 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-01-15T18:00:55.565 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-01-15T18:00:55.565 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:00:55.565 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:00:55.565 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.0")) 2024-01-15T18:00:55.565 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-01-15T18:00:55.775 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.0.57105.asok 2024-01-15T18:00:55.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:00:55.793 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57105.asok 2024-01-15T18:00:55.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:55.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57105.asok status 2024-01-15T18:00:56.038 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:00:56.038 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "0", 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "57105", 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-01-15T18:00:56.039 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 6591, 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 6591 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1518907366 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:56.040 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 1518907366 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.6591 192.168.0.1:0/1518907366", 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/1518907366", 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 35, 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 60, 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-01-15T18:00:56.041 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:00:56.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:56.054 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-01-15T18:00:56.054 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:00:56.108 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-01-15T18:00:56.108 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:00:56.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:56.109 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:00:56.206 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:00:56.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.413+0000 7f8769c00640 1 Processor -- start 2024-01-15T18:00:56.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.414+0000 7f8769c00640 1 -- start start 2024-01-15T18:00:56.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.415+0000 7f8769c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8764162430 0x7f87640b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:56.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.415+0000 7f8769c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8764162810 0x7f8764071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:56.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.415+0000 7f8769c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87640702e0 con 0x7f87640b3650 2024-01-15T18:00:56.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.415+0000 7f8769c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640705c0 con 0x7f8764162430 2024-01-15T18:00:56.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.415+0000 7f8769c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87640708a0 con 0x7f8764162810 2024-01-15T18:00:56.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.415+0000 7f8763e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8764162430 0x7f87640b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:56.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.415+0000 7f8763e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8764162430 0x7f87640b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46414/0 (socket says 172.21.15.26:46414) 2024-01-15T18:00:56.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.415+0000 7f8763e00640 1 -- 172.21.15.26:0/2374851806 learned_addr learned my addr 172.21.15.26:0/2374851806 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:56.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.416+0000 7f8763e00640 1 -- 172.21.15.26:0/2374851806 >> v2:172.21.15.124:3300/0 conn(0x7f8764162810 msgr2=0x7f8764071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:56.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.416+0000 7f8763400640 1 --2- 172.21.15.26:0/2374851806 >> v2:172.21.15.124:3300/0 conn(0x7f8764162810 0x7f8764071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:56.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.416+0000 7f8763e00640 1 --2- 172.21.15.26:0/2374851806 >> v2:172.21.15.124:3300/0 conn(0x7f8764162810 0x7f8764071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.416+0000 7f8763e00640 1 -- 172.21.15.26:0/2374851806 >> v1:172.21.15.124:6789/0 conn(0x7f87640b3650 legacy=0x7f87641650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:56.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.416+0000 7f8763e00640 1 -- 172.21.15.26:0/2374851806 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764070b80 con 0x7f8764162430 2024-01-15T18:00:56.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.417+0000 7f8763e00640 1 --2- 172.21.15.26:0/2374851806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 0x7f87640b3110 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f876415c090 tx=0x7f875402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc518e24ab7eadf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:56.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.417+0000 7f8762a00640 1 -- 172.21.15.26:0/2374851806 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754031870 con 0x7f8764162430 2024-01-15T18:00:56.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.417+0000 7f8762a00640 1 -- 172.21.15.26:0/2374851806 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87540319d0 con 0x7f8764162430 2024-01-15T18:00:56.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.417+0000 7f8762a00640 1 -- 172.21.15.26:0/2374851806 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754031d60 con 0x7f8764162430 2024-01-15T18:00:56.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.418+0000 7f8769c00640 1 -- 172.21.15.26:0/2374851806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 msgr2=0x7f87640b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:56.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.418+0000 7f8769c00640 1 --2- 172.21.15.26:0/2374851806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 0x7f87640b3110 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f876415c090 tx=0x7f875402b910 comp rx=0 tx=0).stop 2024-01-15T18:00:56.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.418+0000 7f8769c00640 1 -- 172.21.15.26:0/2374851806 shutdown_connections 2024-01-15T18:00:56.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.418+0000 7f8769c00640 1 --2- 172.21.15.26:0/2374851806 >> v2:172.21.15.124:3300/0 conn(0x7f8764162810 0x7f8764071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.418+0000 7f8769c00640 1 --2- 172.21.15.26:0/2374851806 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 0x7f87640b3110 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.418+0000 7f8769c00640 1 -- 172.21.15.26:0/2374851806 >> 172.21.15.26:0/2374851806 conn(0x7f876406a800 msgr2=0x7f876406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:56.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.418+0000 7f8769c00640 1 -- 172.21.15.26:0/2374851806 shutdown_connections 2024-01-15T18:00:56.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.418+0000 7f8769c00640 1 -- 172.21.15.26:0/2374851806 wait complete. 2024-01-15T18:00:56.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.420+0000 7f8769c00640 1 Processor -- start 2024-01-15T18:00:56.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.420+0000 7f8769c00640 1 -- start start 2024-01-15T18:00:56.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8769c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87640b3650 0x7f8764159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:56.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8769c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 0x7f876415ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:56.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8769c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8764143790 0x7f876415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:56.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8769c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8764143ff0 con 0x7f8764162430 2024-01-15T18:00:56.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8769c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876415c640 con 0x7f8764143790 2024-01-15T18:00:56.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8769c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8764143c20 con 0x7f87640b3650 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8763400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 0x7f876415ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8763400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 0x7f876415ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46428/0 (socket says 172.21.15.26:46428) 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8763400640 1 -- 172.21.15.26:0/3755625263 learned_addr learned my addr 172.21.15.26:0/3755625263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8763e00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87640b3650 0x7f8764159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8769200640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8764143790 0x7f876415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8763400640 1 -- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87640b3650 msgr2=0x7f8764159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8763400640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87640b3650 0x7f8764159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8763400640 1 -- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8764143790 msgr2=0x7f876415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8763400640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8764143790 0x7f876415c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.421+0000 7f8763400640 1 -- 172.21.15.26:0/3755625263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540094d0 con 0x7f8764162430 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.422+0000 7f8763400640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 0x7f876415ba20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c7f0 tx=0x7f874c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.422+0000 7f875be00640 1 -- 172.21.15.26:0/3755625263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c004230 con 0x7f8764162430 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.422+0000 7f875be00640 1 -- 172.21.15.26:0/3755625263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c004390 con 0x7f8764162430 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.422+0000 7f875be00640 1 -- 172.21.15.26:0/3755625263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c007860 con 0x7f8764162430 2024-01-15T18:00:56.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.422+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8764144290 con 0x7f8764162430 2024-01-15T18:00:56.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.422+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8764144850 con 0x7f8764162430 2024-01-15T18:00:56.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.424+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8764067450 con 0x7f8764162430 2024-01-15T18:00:56.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.425+0000 7f875be00640 1 -- 172.21.15.26:0/3755625263 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f874c007bd0 con 0x7f8764162430 2024-01-15T18:00:56.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.426+0000 7f875be00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f873c06b860 0x7f873c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:56.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.426+0000 7f875be00640 1 -- 172.21.15.26:0/3755625263 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f874c08a6d0 con 0x7f8764162430 2024-01-15T18:00:56.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.427+0000 7f8763e00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f873c06b860 0x7f873c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:56.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.428+0000 7f8763e00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f873c06b860 0x7f873c06dd20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8754036000 tx=0x7f875403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:56.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.431+0000 7f875be00640 1 -- 172.21.15.26:0/3755625263 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f874c054cd0 con 0x7f8764162430 2024-01-15T18:00:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.666+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f87640b2ef0 con 0x7f8764162430 2024-01-15T18:00:56.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.667+0000 7f875be00640 1 -- 172.21.15.26:0/3755625263 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f874c0582f0 con 0x7f8764162430 2024-01-15T18:00:56.671 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:00:56.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f873c06b860 msgr2=0x7f873c06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:56.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f873c06b860 0x7f873c06dd20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8754036000 tx=0x7f875403c040 comp rx=0 tx=0).stop 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 msgr2=0x7f876415ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 0x7f876415ba20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f874c00c7f0 tx=0x7f874c00ccb0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 shutdown_connections 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f873c06b860 0x7f873c06dd20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8764143790 0x7f876415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8764162430 0x7f876415ba20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 --2- 172.21.15.26:0/3755625263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87640b3650 0x7f8764159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 >> 172.21.15.26:0/3755625263 conn(0x7f876406a800 msgr2=0x7f876406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 shutdown_connections 2024-01-15T18:00:56.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:56.670+0000 7f8769c00640 1 -- 172.21.15.26:0/3755625263 wait complete. 2024-01-15T18:00:56.692 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:00:56.692 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:00:56.692 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2024-01-15T18:00:56.693 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:00:56.693 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:00:56.693 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:00:56.693 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:00:56.693 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:00:56.693 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:00:56.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:56.693 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2024-01-15T18:00:56.751 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:00:56.751 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:00:56.751 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:00:56.751 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:56.751 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T18:00:56.751 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:56.751 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:00:56.751 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:56.751 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42259sec preferred_lft 42259sec 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:00:56.752 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4c2d:94ff:fe47:f54c/64 scope link 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 26:4f:4f:5b:0b:31 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::244f:4fff:fe5b:b31/64 scope link 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::2070:16ff:fe92:4a4a/64 scope link 2024-01-15T18:00:56.753 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:00:56.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:56.754 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:00:56.841 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T18:00:56.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:56.846 DEBUG:teuthology.orchestra.run.smithi124:> route 2024-01-15T18:00:56.901 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2024-01-15T18:00:56.901 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:00:56.901 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:00:56.902 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:00:56.902 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:00:56.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:56.902 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T18:00:56.902 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T18:00:56.902 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:00:56.902 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:00:56.902 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:00:56.902 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T18:00:57.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:57.088 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T18:00:57.137 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:00:57.137 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:00:57.138 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:00:57.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:57.138 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2024-01-15T18:00:57.225 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:00:57.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:00:57.226 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:00:57.280 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:00:57.280 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:00:57.280 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:00:57.329 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:00:57.331 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:00:57.339 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:00:57.419 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:00:57.509 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:00:57.510 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:00:57.510 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:00:57.510 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:00:57.510 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:00:57.510 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:00:57.510 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:00:57.511 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:00:57.512 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:00:57.512 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:00:57.512 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:00:57.512 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:00:57.512 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:00:57.512 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:00:57.512 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:00:57.512 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:00:57.513 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:00:57.513 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:00:57.513 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:00:57.513 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:00:57.513 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:00:57.513 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:00:57.513 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:00:57.513 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:00:57.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:57.515 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:00:57.611 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:00:57.611 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:00:57.618 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:00:57.695 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:00:57.740 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:00:57.740 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:00:57.740 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:00:57.740 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:00:57.740 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:00:57.740 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:00:57.740 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:00:57.741 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:00:57.742 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:00:57.743 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:00:57.743 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:00:57.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:57.744 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:00:57.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:2024-01-15T18:00:57.806+0000 7f019935e480 -1 init, newargv = 0x55d5f869c8a0 newargc=15 2024-01-15T18:00:57.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[44855]: starting ceph client 2024-01-15T18:00:57.818 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[44855]: starting fuse 2024-01-15T18:00:57.830 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T18:00:57.830 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:00:57.830 DEBUG:teuthology.orchestra.run.smithi124:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:00:57.830 DEBUG:teuthology.orchestra.run.smithi124:> import glob 2024-01-15T18:00:57.830 DEBUG:teuthology.orchestra.run.smithi124:> import re 2024-01-15T18:00:57.830 DEBUG:teuthology.orchestra.run.smithi124:> import os 2024-01-15T18:00:57.830 DEBUG:teuthology.orchestra.run.smithi124:> import subprocess 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> def _find_admin_socket(client_name): 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> files = glob.glob(asok_path) 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> # Given a non-glob path, it better be there 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> if "*" not in asok_path: 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> assert(len(files) == 1) 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> return files[0] 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:00:57.831 DEBUG:teuthology.orchestra.run.smithi124:> for f in files: 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> contents = proc_f.read() 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> if mountpoint in contents: 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> return f 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> print(_find_admin_socket("client.1")) 2024-01-15T18:00:57.832 DEBUG:teuthology.orchestra.run.smithi124:> ' 2024-01-15T18:00:58.020 INFO:teuthology.orchestra.run.smithi124.stdout:/var/run/ceph/ceph-client.1.44855.asok 2024-01-15T18:00:58.025 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T18:00:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:00:58.037 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44855.asok 2024-01-15T18:00:58.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:58.038 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44855.asok status 2024-01-15T18:00:58.264 INFO:teuthology.orchestra.run.smithi124.stdout:{ 2024-01-15T18:00:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: "metadata": { 2024-01-15T18:00:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:00:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:00:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: "entity_id": "1", 2024-01-15T18:00:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: "hostname": "smithi124", 2024-01-15T18:00:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:00:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: "pid": "44855", 2024-01-15T18:00:58.265 INFO:teuthology.orchestra.run.smithi124.stdout: "root": "/" 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_count": 0, 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_pinned_count": 0, 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: "id": 35273, 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: "inst": { 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: "name": { 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "client", 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: "num": 35273 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:00:58.266 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 4074370877 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: } 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 4074370877 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:00:58.267 INFO:teuthology.orchestra.run.smithi124.stdout: "inst_str": "client.35273 192.168.0.1:0/4074370877", 2024-01-15T18:00:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: "addr_str": "192.168.0.1:0/4074370877", 2024-01-15T18:00:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: "inode_count": 1, 2024-01-15T18:00:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: "mds_epoch": 35, 2024-01-15T18:00:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch": 60, 2024-01-15T18:00:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:00:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: "blocklisted": false, 2024-01-15T18:00:58.268 INFO:teuthology.orchestra.run.smithi124.stdout: "fs_name": "cephfs" 2024-01-15T18:00:58.268 INFO:teuthology.orchestra.run.smithi124.stdout:} 2024-01-15T18:00:58.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:58.280 DEBUG:teuthology.orchestra.run.smithi124:> stat --file-system '--printf=%T 2024-01-15T18:00:58.280 DEBUG:teuthology.orchestra.run.smithi124:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:00:58.328 INFO:teuthology.orchestra.run.smithi124.stdout:fuseblk 2024-01-15T18:00:58.328 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:00:58.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:00:58.328 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:00:58.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:00:58.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.635+0000 7f12da400640 1 Processor -- start 2024-01-15T18:00:58.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.635+0000 7f12da400640 1 -- start start 2024-01-15T18:00:58.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.637+0000 7f12da400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d40b3f30 0x7f12d40d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:58.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.637+0000 7f12da400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d40d0a10 con 0x7f12d4160920 2024-01-15T18:00:58.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.637+0000 7f12da400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d40cf080 con 0x7f12d40d0100 2024-01-15T18:00:58.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.637+0000 7f12da400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40c6be0 con 0x7f12d40b3f30 2024-01-15T18:00:58.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.637+0000 7f12d9000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f12d4160920 0x7f12d40cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50664/0 (socket says 172.21.15.26:50664) 2024-01-15T18:00:58.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.637+0000 7f12d9000640 1 -- 172.21.15.26:0/1803538987 learned_addr learned my addr 172.21.15.26:0/1803538987 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:58.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819168381 0 0) 0x7f12d40d0a10 con 0x7f12d4160920 2024-01-15T18:00:58.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b4003520 con 0x7f12d4160920 2024-01-15T18:00:58.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d9a00640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d40b3f30 0x7f12d40d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:58.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941315917 0 0) 0x7f12b4003520 con 0x7f12d4160920 2024-01-15T18:00:58.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d40b3f30 msgr2=0x7f12d40d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:58.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d40b3f30 0x7f12d40d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 >> v1:172.21.15.124:6789/0 conn(0x7f12d40d0100 legacy=0x7f12d40d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40ce7e0 con 0x7f12d4160920 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f12c4001670 con 0x7f12d4160920 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 >> v1:172.21.15.26:6789/0 conn(0x7f12d4160920 legacy=0x7f12d40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b40039a0 0x7f12b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.638+0000 7f12d3e00640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b40047c0 0x7f12b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12d3e00640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d40b3f30 0x7f12b4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40c6be0 con 0x7f12b40047c0 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b400bb10 con 0x7f12b40039a0 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b400bcd0 con 0x7f12d40b3f30 2024-01-15T18:00:58.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12cbc00640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b40047c0 0x7f12b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:58.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12cbc00640 1 -- 172.21.15.26:0/1803538987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d40b3f30 msgr2=0x7f12b4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:58.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12cbc00640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d40b3f30 0x7f12b4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:58.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12cbc00640 1 -- 172.21.15.26:0/1803538987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b40039a0 msgr2=0x7f12b4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:58.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12cbc00640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b40039a0 0x7f12b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:58.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.639+0000 7f12cbc00640 1 -- 172.21.15.26:0/1803538987 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4003520 con 0x7f12b40047c0 2024-01-15T18:00:58.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.640+0000 7f12cbc00640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b40047c0 0x7f12b4004c40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f12bc001750 tx=0x7f12b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1294bd8afbfd3fe4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:58.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.640+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc0052d0 con 0x7f12b40047c0 2024-01-15T18:00:58.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.640+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc02e050 con 0x7f12b40047c0 2024-01-15T18:00:58.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.640+0000 7f12d3e00640 1 -- 172.21.15.26:0/1803538987 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc0035f0 con 0x7f12b40047c0 2024-01-15T18:00:58.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.641+0000 7f12da400640 1 -- 172.21.15.26:0/1803538987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b40047c0 msgr2=0x7f12b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:58.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.641+0000 7f12da400640 1 --2- 172.21.15.26:0/1803538987 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b40047c0 0x7f12b4004c40 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f12bc001750 tx=0x7f12b40057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:58.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.641+0000 7f12d9a00640 1 -- 172.21.15.26:0/1803538987 reap_dead start 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.641+0000 7f12da400640 1 -- 172.21.15.26:0/1803538987 shutdown_connections 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.641+0000 7f12da400640 1 -- 172.21.15.26:0/1803538987 >> 172.21.15.26:0/1803538987 conn(0x7f12d40d3090 msgr2=0x7f12d40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.641+0000 7f12da400640 1 -- 172.21.15.26:0/1803538987 shutdown_connections 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.641+0000 7f12da400640 1 -- 172.21.15.26:0/1803538987 wait complete. 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.643+0000 7f12da400640 1 Processor -- start 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.643+0000 7f12da400640 1 -- start start 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.644+0000 7f12da400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b40047c0 0x7f12d41f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.644+0000 7f12da400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d41f5d30 0x7f12d40ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.644+0000 7f12da400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12d40ca6a0 0x7f12d40cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.644+0000 7f12da400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40cad90 con 0x7f12d40ca6a0 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.644+0000 7f12da400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40c6d20 con 0x7f12b40047c0 2024-01-15T18:00:58.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.644+0000 7f12da400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40cab20 con 0x7f12d41f5d30 2024-01-15T18:00:58.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.644+0000 7f12d9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12d40ca6a0 0x7f12d40cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:58.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.644+0000 7f12d9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12d40ca6a0 0x7f12d40cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46434/0 (socket says 172.21.15.26:46434) 2024-01-15T18:00:58.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.644+0000 7f12d9a00640 1 -- 172.21.15.26:0/3602669298 learned_addr learned my addr 172.21.15.26:0/3602669298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:58.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12d9a00640 1 -- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d41f5d30 msgr2=0x7f12d40ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:58.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12d9a00640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d41f5d30 0x7f12d40ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:58.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12d9a00640 1 -- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b40047c0 msgr2=0x7f12d41f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:58.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12d9a00640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b40047c0 0x7f12d41f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:58.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12d9a00640 1 -- 172.21.15.26:0/3602669298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40cd290 con 0x7f12d40ca6a0 2024-01-15T18:00:58.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12d9a00640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12d40ca6a0 0x7f12d40cc560 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f12d40d3060 tx=0x7f12cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:58.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12d1600640 1 -- 172.21.15.26:0/3602669298 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc018070 con 0x7f12d40ca6a0 2024-01-15T18:00:58.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12d1600640 1 -- 172.21.15.26:0/3602669298 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc00eb90 con 0x7f12d40ca6a0 2024-01-15T18:00:58.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12d1600640 1 -- 172.21.15.26:0/3602669298 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc003990 con 0x7f12d40ca6a0 2024-01-15T18:00:58.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d40c6f30 con 0x7f12d40ca6a0 2024-01-15T18:00:58.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.645+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d40c74f0 con 0x7f12d40ca6a0 2024-01-15T18:00:58.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.646+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d4067450 con 0x7f12d40ca6a0 2024-01-15T18:00:58.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.647+0000 7f12d1600640 1 -- 172.21.15.26:0/3602669298 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f12cc014420 con 0x7f12d40ca6a0 2024-01-15T18:00:58.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.648+0000 7f12d1600640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12ac06b860 0x7f12ac06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:58.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.648+0000 7f12d1600640 1 -- 172.21.15.26:0/3602669298 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f12cc08a570 con 0x7f12d40ca6a0 2024-01-15T18:00:58.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.649+0000 7f12d9000640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12ac06b860 0x7f12ac06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:58.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.650+0000 7f12d9000640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12ac06b860 0x7f12ac06dd20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f12d40d1d70 tx=0x7f12c4001530 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:58.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.653+0000 7f12d1600640 1 -- 172.21.15.26:0/3602669298 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12cc054b10 con 0x7f12d40ca6a0 2024-01-15T18:00:58.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:58.882+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f12d408c7e0 con 0x7f12d40ca6a0 2024-01-15T18:00:59.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.112+0000 7f12d1600640 1 -- 172.21.15.26:0/3602669298 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 136+0+0 (secure 0 0 0) 0x7f12cc058130 con 0x7f12d40ca6a0 2024-01-15T18:00:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12ac06b860 msgr2=0x7f12ac06dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12ac06b860 0x7f12ac06dd20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f12d40d1d70 tx=0x7f12c4001530 comp rx=0 tx=0).stop 2024-01-15T18:00:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12d40ca6a0 msgr2=0x7f12d40cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12d40ca6a0 0x7f12d40cc560 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f12d40d3060 tx=0x7f12cc00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 shutdown_connections 2024-01-15T18:00:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12d41f5d30 0x7f12d40ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12ac06b860 0x7f12ac06dd20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12d40ca6a0 0x7f12d40cc560 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 --2- 172.21.15.26:0/3602669298 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b40047c0 0x7f12d41f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 >> 172.21.15.26:0/3602669298 conn(0x7f12d40d3090 msgr2=0x7f12d40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 shutdown_connections 2024-01-15T18:00:59.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.115+0000 7f12da400640 1 -- 172.21.15.26:0/3602669298 wait complete. 2024-01-15T18:00:59.139 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:00:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.352+0000 7fb5a0400640 1 Processor -- start 2024-01-15T18:00:59.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.352+0000 7fb5a0400640 1 -- start start 2024-01-15T18:00:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.354+0000 7fb5a0400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb59815e3b0 0x7fb598164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.354+0000 7fb5a0400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5980b1ee0 con 0x7fb598143ae0 2024-01-15T18:00:59.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.354+0000 7fb5a0400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5980b2780 con 0x7fb5980cf9a0 2024-01-15T18:00:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.354+0000 7fb5a0400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb59815e990 con 0x7fb59815e3b0 2024-01-15T18:00:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.354+0000 7fb59da00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb598143ae0 0x7fb5980cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50676/0 (socket says 172.21.15.26:50676) 2024-01-15T18:00:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.354+0000 7fb59da00640 1 -- 172.21.15.26:0/1013700022 learned_addr learned my addr 172.21.15.26:0/1013700022 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.354+0000 7fb59e400640 1 --2- 172.21.15.26:0/1013700022 >> v2:172.21.15.26:3300/0 conn(0x7fb59815e3b0 0x7fb598164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.354+0000 7fb597e00640 1 -- 172.21.15.26:0/1013700022 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034759961 0 0) 0x7fb5980b1ee0 con 0x7fb598143ae0 2024-01-15T18:00:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.355+0000 7fb597e00640 1 -- 172.21.15.26:0/1013700022 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb578003520 con 0x7fb598143ae0 2024-01-15T18:00:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.355+0000 7fb59e400640 1 -- 172.21.15.26:0/1013700022 >> v1:172.21.15.124:6789/0 conn(0x7fb5980cf9a0 legacy=0x7fb59815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:59.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.355+0000 7fb59e400640 1 -- 172.21.15.26:0/1013700022 >> v1:172.21.15.26:6789/0 conn(0x7fb598143ae0 legacy=0x7fb5980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.355+0000 7fb59e400640 1 -- 172.21.15.26:0/1013700022 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5980b2130 con 0x7fb59815e3b0 2024-01-15T18:00:59.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.356+0000 7fb59e400640 1 --2- 172.21.15.26:0/1013700022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 0x7fb598164de0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb590009d90 tx=0x7fb59002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48b67f5a682078f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:59.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.356+0000 7fb597e00640 1 -- 172.21.15.26:0/1013700022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59002cc60 con 0x7fb59815e3b0 2024-01-15T18:00:59.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.356+0000 7fb597e00640 1 -- 172.21.15.26:0/1013700022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59002cdc0 con 0x7fb59815e3b0 2024-01-15T18:00:59.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.356+0000 7fb597e00640 1 -- 172.21.15.26:0/1013700022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb590034660 con 0x7fb59815e3b0 2024-01-15T18:00:59.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.357+0000 7fb5a0400640 1 -- 172.21.15.26:0/1013700022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 msgr2=0x7fb598164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.357+0000 7fb5a0400640 1 --2- 172.21.15.26:0/1013700022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 0x7fb598164de0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb590009d90 tx=0x7fb59002beb0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.357+0000 7fb5a0400640 1 -- 172.21.15.26:0/1013700022 shutdown_connections 2024-01-15T18:00:59.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.357+0000 7fb5a0400640 1 --2- 172.21.15.26:0/1013700022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 0x7fb598164de0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.357+0000 7fb5a0400640 1 -- 172.21.15.26:0/1013700022 >> 172.21.15.26:0/1013700022 conn(0x7fb59806a800 msgr2=0x7fb59806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:59.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.357+0000 7fb5a0400640 1 -- 172.21.15.26:0/1013700022 shutdown_connections 2024-01-15T18:00:59.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.357+0000 7fb5a0400640 1 -- 172.21.15.26:0/1013700022 wait complete. 2024-01-15T18:00:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.358+0000 7fb5a0400640 1 Processor -- start 2024-01-15T18:00:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.359+0000 7fb5a0400640 1 -- start start 2024-01-15T18:00:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.359+0000 7fb5a0400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5980cf9a0 0x7fb598157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.359+0000 7fb5a0400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb598143ae0 0x7fb5980afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.359+0000 7fb5a0400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 0x7fb5980a86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.359+0000 7fb5a0400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb598159f90 con 0x7fb59815e3b0 2024-01-15T18:00:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.359+0000 7fb5a0400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb598159380 con 0x7fb5980cf9a0 2024-01-15T18:00:59.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.359+0000 7fb5a0400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb598159660 con 0x7fb598143ae0 2024-01-15T18:00:59.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 0x7fb5980a86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 0x7fb5980a86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46460/0 (socket says 172.21.15.26:46460) 2024-01-15T18:00:59.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59e400640 1 -- 172.21.15.26:0/4204607177 learned_addr learned my addr 172.21.15.26:0/4204607177 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:59.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59da00640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5980cf9a0 0x7fb598157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59d000640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb598143ae0 0x7fb5980afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59e400640 1 -- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb598143ae0 msgr2=0x7fb5980afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59e400640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb598143ae0 0x7fb5980afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59e400640 1 -- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5980cf9a0 msgr2=0x7fb598157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59e400640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5980cf9a0 0x7fb598157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59e400640 1 -- 172.21.15.26:0/4204607177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb590009a70 con 0x7fb59815e3b0 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59d000640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb598143ae0 0x7fb5980afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.360+0000 7fb59da00640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5980cf9a0 0x7fb598157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.361+0000 7fb59e400640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 0x7fb5980a86d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb590009eb0 tx=0x7fb590004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.361+0000 7fb595600640 1 -- 172.21.15.26:0/4204607177 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59003e610 con 0x7fb59815e3b0 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.361+0000 7fb595600640 1 -- 172.21.15.26:0/4204607177 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59003e770 con 0x7fb59815e3b0 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.361+0000 7fb595600640 1 -- 172.21.15.26:0/4204607177 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59003eb20 con 0x7fb59815e3b0 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.361+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb598159940 con 0x7fb59815e3b0 2024-01-15T18:00:59.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.361+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5980affd0 con 0x7fb59815e3b0 2024-01-15T18:00:59.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.362+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59815a140 con 0x7fb59815e3b0 2024-01-15T18:00:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.363+0000 7fb595600640 1 -- 172.21.15.26:0/4204607177 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fb59003b050 con 0x7fb59815e3b0 2024-01-15T18:00:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.364+0000 7fb595600640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb57006b890 0x7fb57006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.364+0000 7fb595600640 1 -- 172.21.15.26:0/4204607177 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb5900acbf0 con 0x7fb59815e3b0 2024-01-15T18:00:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.364+0000 7fb59da00640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb57006b890 0x7fb57006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.365+0000 7fb59da00640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb57006b890 0x7fb57006dd50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb588005880 tx=0x7fb588009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:59.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.368+0000 7fb595600640 1 -- 172.21.15.26:0/4204607177 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb59815a140 con 0x7fb59815e3b0 2024-01-15T18:00:59.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.585+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb59806c3f0 con 0x7fb57006b890 2024-01-15T18:00:59.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.589+0000 7fb595600640 1 -- 172.21.15.26:0/4204607177 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb59806c3f0 con 0x7fb57006b890 2024-01-15T18:00:59.592 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:00:59.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:00:59.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:00:59.592 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:00:59.592 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb57006b890 msgr2=0x7fb57006dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb57006b890 0x7fb57006dd50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb588005880 tx=0x7fb588009040 comp rx=0 tx=0).stop 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 msgr2=0x7fb5980a86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 0x7fb5980a86d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb590009eb0 tx=0x7fb590004a40 comp rx=0 tx=0).stop 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 shutdown_connections 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb598143ae0 0x7fb5980afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb57006b890 0x7fb57006dd50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb59815e3b0 0x7fb5980a86d0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 --2- 172.21.15.26:0/4204607177 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5980cf9a0 0x7fb598157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 >> 172.21.15.26:0/4204607177 conn(0x7fb59806a800 msgr2=0x7fb59815c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:59.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 shutdown_connections 2024-01-15T18:00:59.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.591+0000 7fb5a0400640 1 -- 172.21.15.26:0/4204607177 wait complete. 2024-01-15T18:00:59.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:00:59.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.827+0000 7f0051e00640 1 Processor -- start 2024-01-15T18:00:59.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.827+0000 7f0051e00640 1 -- start start 2024-01-15T18:00:59.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.828+0000 7f0051e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f004c0d0100 0x7f004c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.828+0000 7f0051e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0d0f40 0x7f004c0d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.828+0000 7f0051e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c0d0a10 con 0x7f004c160920 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.828+0000 7f0051e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c0cf080 con 0x7f004c0d0100 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.828+0000 7f0051e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0c6be0 con 0x7f004c0d0f40 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.829+0000 7f0050a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f004c160920 0x7f004c0cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50682/0 (socket says 172.21.15.26:50682) 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.829+0000 7f0050a00640 1 -- 172.21.15.26:0/1385295955 learned_addr learned my addr 172.21.15.26:0/1385295955 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.829+0000 7f0043e00640 1 --2- 172.21.15.26:0/1385295955 >> v2:172.21.15.124:3300/0 conn(0x7f004c0d0100 0x7f004c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.829+0000 7f0051400640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0d0f40 0x7f004c0d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.829+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507083969 0 0) 0x7f004c0d0a10 con 0x7f004c160920 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.829+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f002c003520 con 0x7f004c160920 2024-01-15T18:00:59.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 813420181 0 0) 0x7f002c003520 con 0x7f004c160920 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0d0f40 msgr2=0x7f004c0d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0d0f40 0x7f004c0d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 >> v2:172.21.15.124:3300/0 conn(0x7f004c0d0100 msgr2=0x7f004c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 --2- 172.21.15.26:0/1385295955 >> v2:172.21.15.124:3300/0 conn(0x7f004c0d0100 0x7f004c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c0ce7e0 con 0x7f004c160920 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f003c001670 con 0x7f004c160920 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 >> v1:172.21.15.26:6789/0 conn(0x7f004c160920 legacy=0x7f004c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f002c003970 0x7f002c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0d0f40 0x7f002c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f002c004d00 0x7f002c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0cf080 con 0x7f002c003970 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0c6be0 con 0x7f002c004d00 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c00ba00 con 0x7f004c0d0f40 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f0043e00640 1 --2- 172.21.15.26:0/1385295955 >> v2:172.21.15.124:3300/0 conn(0x7f004c0d0100 0x7f004c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.830+0000 7f0050a00640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f002c003970 0x7f002c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.831+0000 7f0051400640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f002c004d00 0x7f002c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.831+0000 7f0050a00640 1 -- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0d0f40 msgr2=0x7f002c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.831+0000 7f0050a00640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f004c0d0f40 0x7f002c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.831+0000 7f0050a00640 1 -- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f002c004d00 msgr2=0x7f002c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:00:59.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.831+0000 7f0050a00640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f002c004d00 0x7f002c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.831+0000 7f0050a00640 1 -- 172.21.15.26:0/1385295955 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c003520 con 0x7f002c003970 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.832+0000 7f0050a00640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f002c003970 0x7f002c003d90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c190 tx=0x7f002c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4965bda747769928 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.832+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c005ad0 con 0x7f002c003970 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.832+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c005c50 con 0x7f002c003970 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.832+0000 7f004be00640 1 -- 172.21.15.26:0/1385295955 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c02c850 con 0x7f002c003970 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.833+0000 7f0051e00640 1 -- 172.21.15.26:0/1385295955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f002c003970 msgr2=0x7f002c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.833+0000 7f0051e00640 1 --2- 172.21.15.26:0/1385295955 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f002c003970 0x7f002c003d90 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c190 tx=0x7f002c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.833+0000 7f0051400640 1 -- 172.21.15.26:0/1385295955 reap_dead start 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.833+0000 7f0051e00640 1 -- 172.21.15.26:0/1385295955 shutdown_connections 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.833+0000 7f0051e00640 1 -- 172.21.15.26:0/1385295955 >> 172.21.15.26:0/1385295955 conn(0x7f004c0d3090 msgr2=0x7f004c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:00:59.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.833+0000 7f0051e00640 1 -- 172.21.15.26:0/1385295955 shutdown_connections 2024-01-15T18:00:59.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.833+0000 7f0051e00640 1 -- 172.21.15.26:0/1385295955 wait complete. 2024-01-15T18:00:59.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.835+0000 7f0051e00640 1 Processor -- start 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.835+0000 7f0051e00640 1 -- start start 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0051e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f002c003970 0x7f004c0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0051e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c157300 0x7f004c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0051e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c0cc850 0x7f004c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0051e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0c92c0 con 0x7f004c157300 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0051e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c157cc0 con 0x7f004c0cc850 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0051e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c1f3070 con 0x7f002c003970 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0043e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c157300 0x7f004c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0043e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c157300 0x7f004c157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46472/0 (socket says 172.21.15.26:46472) 2024-01-15T18:00:59.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0043e00640 1 -- 172.21.15.26:0/3415319486 learned_addr learned my addr 172.21.15.26:0/3415319486 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0050a00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f002c003970 0x7f004c0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0051400640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c0cc850 0x7f004c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0043e00640 1 -- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f002c003970 msgr2=0x7f004c0c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0043e00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f002c003970 0x7f004c0c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0043e00640 1 -- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c0cc850 msgr2=0x7f004c0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0043e00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c0cc850 0x7f004c0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.836+0000 7f0043e00640 1 -- 172.21.15.26:0/3415319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c1f3350 con 0x7f004c157300 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.837+0000 7f0043e00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c157300 0x7f004c157780 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f003400e7e0 tx=0x7f003400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.837+0000 7f0049600640 1 -- 172.21.15.26:0/3415319486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034004030 con 0x7f004c157300 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.837+0000 7f0049600640 1 -- 172.21.15.26:0/3415319486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0034004190 con 0x7f004c157300 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.837+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c1f3630 con 0x7f004c157300 2024-01-15T18:00:59.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.837+0000 7f0049600640 1 -- 172.21.15.26:0/3415319486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003400c6d0 con 0x7f004c157300 2024-01-15T18:00:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.837+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c1f3bf0 con 0x7f004c157300 2024-01-15T18:00:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.838+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c1f3db0 con 0x7f004c157300 2024-01-15T18:00:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.839+0000 7f0049600640 1 -- 172.21.15.26:0/3415319486 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f003400c830 con 0x7f004c157300 2024-01-15T18:00:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.840+0000 7f0049600640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b860 0x7f002406dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:00:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.840+0000 7f0049600640 1 -- 172.21.15.26:0/3415319486 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f003408a5f0 con 0x7f004c157300 2024-01-15T18:00:59.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.841+0000 7f0050a00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b860 0x7f002406dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:00:59.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.842+0000 7f0050a00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b860 0x7f002406dd20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c160 tx=0x7f003c009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:00:59.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:00:59.845+0000 7f0049600640 1 -- 172.21.15.26:0/3415319486 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0034054b10 con 0x7f004c157300 2024-01-15T18:01:00.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.040+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f004c067450 con 0x7f004c157300 2024-01-15T18:01:00.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.041+0000 7f0049600640 1 -- 172.21.15.26:0/3415319486 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0034058130 con 0x7f004c157300 2024-01-15T18:01:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.043+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b860 msgr2=0x7f002406dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.043+0000 7f0051e00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b860 0x7f002406dd20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f003c00c160 tx=0x7f003c009030 comp rx=0 tx=0).stop 2024-01-15T18:01:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.043+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c157300 msgr2=0x7f004c157780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.043+0000 7f0051e00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c157300 0x7f004c157780 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f003400e7e0 tx=0x7f003400eca0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.044+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 shutdown_connections 2024-01-15T18:01:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.044+0000 7f0051e00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f002406b860 0x7f002406dd20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.044+0000 7f0051e00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f004c0cc850 0x7f004c0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.044+0000 7f0051e00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004c157300 0x7f004c157780 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.044+0000 7f0051e00640 1 --2- 172.21.15.26:0/3415319486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f002c003970 0x7f004c0c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.044+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 >> 172.21.15.26:0/3415319486 conn(0x7f004c0d3090 msgr2=0x7f004c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.044+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 shutdown_connections 2024-01-15T18:01:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.044+0000 7f0051e00640 1 -- 172.21.15.26:0/3415319486 wait complete. 2024-01-15T18:01:00.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:01:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.286+0000 7fec93200640 1 Processor -- start 2024-01-15T18:01:00.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.286+0000 7fec93200640 1 -- start start 2024-01-15T18:01:00.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.288+0000 7fec93200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fec8c0b3f30 0x7fec8c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.288+0000 7fec93200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c0d0a10 con 0x7fec8c160920 2024-01-15T18:01:00.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.288+0000 7fec93200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c0cf080 con 0x7fec8c0d0100 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.288+0000 7fec93200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8c0c6be0 con 0x7fec8c0b3f30 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.288+0000 7fec91e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fec8c160920 0x7fec8c0cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50686/0 (socket says 172.21.15.26:50686) 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.288+0000 7fec91e00640 1 -- 172.21.15.26:0/3893111509 learned_addr learned my addr 172.21.15.26:0/3893111509 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.288+0000 7fec92800640 1 --2- 172.21.15.26:0/3893111509 >> v2:172.21.15.124:3300/0 conn(0x7fec8c0b3f30 0x7fec8c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.288+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604505315 0 0) 0x7fec8c0d0a10 con 0x7fec8c160920 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.288+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec6c003520 con 0x7fec8c160920 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2454521759 0 0) 0x7fec6c003520 con 0x7fec8c160920 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 >> v2:172.21.15.124:3300/0 conn(0x7fec8c0b3f30 msgr2=0x7fec8c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 --2- 172.21.15.26:0/3893111509 >> v2:172.21.15.124:3300/0 conn(0x7fec8c0b3f30 0x7fec8c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 >> v1:172.21.15.124:6789/0 conn(0x7fec8c0d0100 legacy=0x7fec8c0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c0ce7e0 con 0x7fec8c160920 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fec7c001670 con 0x7fec8c160920 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 >> v1:172.21.15.26:6789/0 conn(0x7fec8c160920 legacy=0x7fec8c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec6c0047c0 0x7fec6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec6c005180 0x7fec6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0c6be0 con 0x7fec6c0039c0 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c00c080 con 0x7fec6c0047c0 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c00c240 con 0x7fec6c005180 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.289+0000 7fec91e00640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.290+0000 7fec91e00640 1 -- 172.21.15.26:0/3893111509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec6c005180 msgr2=0x7fec6c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.290+0000 7fec91e00640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec6c005180 0x7fec6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.290+0000 7fec91e00640 1 -- 172.21.15.26:0/3893111509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec6c0047c0 msgr2=0x7fec6c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.290+0000 7fec91400640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec6c0047c0 0x7fec6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.290+0000 7fec91e00640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec6c0047c0 0x7fec6c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.290+0000 7fec92800640 1 -- 172.21.15.26:0/3893111509 reap_dead start 2024-01-15T18:01:00.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.290+0000 7fec91e00640 1 -- 172.21.15.26:0/3893111509 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c003520 con 0x7fec6c0039c0 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.291+0000 7fec91e00640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c0e0 tx=0x7fec6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79fe5eb46e376a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.291+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c005ad0 con 0x7fec6c0039c0 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.291+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c005c50 con 0x7fec6c0039c0 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.291+0000 7fec90a00640 1 -- 172.21.15.26:0/3893111509 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c02c790 con 0x7fec6c0039c0 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.292+0000 7fec93200640 1 -- 172.21.15.26:0/3893111509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 msgr2=0x7fec6c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.292+0000 7fec93200640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c0e0 tx=0x7fec6c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.292+0000 7fec93200640 1 -- 172.21.15.26:0/3893111509 shutdown_connections 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.292+0000 7fec93200640 1 --2- 172.21.15.26:0/3893111509 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec6c003de0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.292+0000 7fec93200640 1 -- 172.21.15.26:0/3893111509 >> 172.21.15.26:0/3893111509 conn(0x7fec8c0d3090 msgr2=0x7fec8c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.292+0000 7fec93200640 1 -- 172.21.15.26:0/3893111509 shutdown_connections 2024-01-15T18:01:00.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.292+0000 7fec93200640 1 -- 172.21.15.26:0/3893111509 wait complete. 2024-01-15T18:01:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.294+0000 7fec93200640 1 Processor -- start 2024-01-15T18:01:00.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.294+0000 7fec93200640 1 -- start start 2024-01-15T18:01:00.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec93200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec8c0b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec93200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec8c0b5c50 0x7fec8c1515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec93200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8c0b8080 0x7fec8c0b8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec93200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0b8a40 con 0x7fec6c0039c0 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec93200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0ca370 con 0x7fec8c0b5c50 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec93200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0b6410 con 0x7fec8c0b8080 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec91e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec8c0b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec91e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec8c0b79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46488/0 (socket says 172.21.15.26:46488) 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec91e00640 1 -- 172.21.15.26:0/1642947765 learned_addr learned my addr 172.21.15.26:0/1642947765 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec92800640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8c0b8080 0x7fec8c0b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec91e00640 1 -- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8c0b8080 msgr2=0x7fec8c0b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec91400640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec8c0b5c50 0x7fec8c1515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec91e00640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8c0b8080 0x7fec8c0b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec91e00640 1 -- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec8c0b5c50 msgr2=0x7fec8c1515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec91e00640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec8c0b5c50 0x7fec8c1515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec91e00640 1 -- 172.21.15.26:0/1642947765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c0b66f0 con 0x7fec6c0039c0 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.295+0000 7fec92800640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8c0b8080 0x7fec8c0b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.296+0000 7fec91e00640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec8c0b79b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c200 tx=0x7fec7c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.296+0000 7fec8a000640 1 -- 172.21.15.26:0/1642947765 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c03c040 con 0x7fec6c0039c0 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.296+0000 7fec8a000640 1 -- 172.21.15.26:0/1642947765 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c02cc60 con 0x7fec6c0039c0 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.296+0000 7fec8a000640 1 -- 172.21.15.26:0/1642947765 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c007070 con 0x7fec6c0039c0 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.296+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8c0b69d0 con 0x7fec6c0039c0 2024-01-15T18:01:00.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.296+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c0b60b0 con 0x7fec6c0039c0 2024-01-15T18:01:00.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.297+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c0b7ef0 con 0x7fec6c0039c0 2024-01-15T18:01:00.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.298+0000 7fec8a000640 1 -- 172.21.15.26:0/1642947765 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7fec7c032070 con 0x7fec6c0039c0 2024-01-15T18:01:00.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.299+0000 7fec8a000640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec6406b890 0x7fec6406dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.299+0000 7fec8a000640 1 -- 172.21.15.26:0/1642947765 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fec7c0ab940 con 0x7fec6c0039c0 2024-01-15T18:01:00.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.300+0000 7fec91400640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec6406b890 0x7fec6406dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.301+0000 7fec91400640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec6406b890 0x7fec6406dd50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0b7660 tx=0x7fec74009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:00.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.304+0000 7fec8a000640 1 -- 172.21.15.26:0/1642947765 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec7c075ee0 con 0x7fec6c0039c0 2024-01-15T18:01:00.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.524+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7fec8c160d00 con 0x7fec6406b890 2024-01-15T18:01:00.537 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:00.534+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:00.537 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:00.534+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:00.537 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:00.534+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:00.537 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:00.534+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:00.537 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:00.534+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:00.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.591+0000 7fec8a000640 1 -- 172.21.15.26:0/1642947765 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fec7400d040 con 0x7fec6406b890 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.593+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec6406b890 msgr2=0x7fec6406dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.593+0000 7fec93200640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec6406b890 0x7fec6406dd50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0b7660 tx=0x7fec74009040 comp rx=0 tx=0).stop 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.593+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 msgr2=0x7fec8c0b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.593+0000 7fec93200640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec8c0b79b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c200 tx=0x7fec7c02e070 comp rx=0 tx=0).stop 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.593+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 shutdown_connections 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.593+0000 7fec93200640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec8c0b8080 0x7fec8c0b8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.593+0000 7fec93200640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec6406b890 0x7fec6406dd50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.593+0000 7fec93200640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec8c0b5c50 0x7fec8c1515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.593+0000 7fec93200640 1 --2- 172.21.15.26:0/1642947765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec6c0039c0 0x7fec8c0b79b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.594+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 >> 172.21.15.26:0/1642947765 conn(0x7fec8c0d3090 msgr2=0x7fec8c0d0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:00.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.594+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 shutdown_connections 2024-01-15T18:01:00.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.594+0000 7fec93200640 1 -- 172.21.15.26:0/1642947765 wait complete. 2024-01-15T18:01:00.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:01:00.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.833+0000 7f887f200640 1 Processor -- start 2024-01-15T18:01:00.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.834+0000 7f887f200640 1 -- start start 2024-01-15T18:01:00.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.835+0000 7f887f200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f888015d4b0 0x7f88800d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.835+0000 7f887f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88800cf9a0 0x7f88800cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.835+0000 7f887f200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88800d0320 con 0x7f88800cddf0 2024-01-15T18:01:00.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.835+0000 7f887f200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88800b2730 con 0x7f888015d4b0 2024-01-15T18:01:00.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.835+0000 7f887f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88800d1ec0 con 0x7f88800cf9a0 2024-01-15T18:01:00.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.835+0000 7f887e800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f88800cddf0 0x7f88800ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50688/0 (socket says 172.21.15.26:50688) 2024-01-15T18:01:00.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.835+0000 7f887e800640 1 -- 172.21.15.26:0/3532898352 learned_addr learned my addr 172.21.15.26:0/3532898352 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:01:00.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887de00640 1 --2- 172.21.15.26:0/3532898352 >> v2:172.21.15.124:3300/0 conn(0x7f888015d4b0 0x7f88800d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887d400640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88800cf9a0 0x7f88800cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131572014 0 0) 0x7f88800d0320 con 0x7f88800cddf0 2024-01-15T18:01:00.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8860003520 con 0x7f88800cddf0 2024-01-15T18:01:00.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4089090637 0 0) 0x7f8860003520 con 0x7f88800cddf0 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88800cf9a0 msgr2=0x7f88800cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88800cf9a0 0x7f88800cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 >> v2:172.21.15.124:3300/0 conn(0x7f888015d4b0 msgr2=0x7f88800d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 --2- 172.21.15.26:0/3532898352 >> v2:172.21.15.124:3300/0 conn(0x7f888015d4b0 0x7f88800d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88800b29d0 con 0x7f88800cddf0 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8878001670 con 0x7f88800cddf0 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 >> v1:172.21.15.26:6789/0 conn(0x7f88800cddf0 legacy=0x7f88800ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88800cf9a0 0x7f8860003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8860003ef0 0x7f88600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f8860005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.836+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88800b2730 con 0x7f8860004d00 2024-01-15T18:01:00.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88800d1ec0 con 0x7f8860003ef0 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886000b980 con 0x7f88800cf9a0 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887e800640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f8860005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887de00640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88800cf9a0 0x7f8860003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887d400640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8860003ef0 0x7f88600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887e800640 1 -- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88800cf9a0 msgr2=0x7f8860003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887e800640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f88800cf9a0 0x7f8860003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887e800640 1 -- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8860003ef0 msgr2=0x7f88600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887e800640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8860003ef0 0x7f88600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.837+0000 7f887e800640 1 -- 172.21.15.26:0/3532898352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8860003520 con 0x7f8860004d00 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.839+0000 7f887e800640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f8860005180 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f887800c240 tx=0x7f88600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32537bdd44b1bdf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.839+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8878005ad0 con 0x7f8860004d00 2024-01-15T18:01:00.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.839+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8878005c30 con 0x7f8860004d00 2024-01-15T18:01:00.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.839+0000 7f887ca00640 1 -- 172.21.15.26:0/3532898352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887802cae0 con 0x7f8860004d00 2024-01-15T18:01:00.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.840+0000 7f887f200640 1 -- 172.21.15.26:0/3532898352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 msgr2=0x7f8860005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.840+0000 7f887f200640 1 --2- 172.21.15.26:0/3532898352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f8860005180 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f887800c240 tx=0x7f88600057f0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.840+0000 7f887e800640 1 -- 172.21.15.26:0/3532898352 reap_dead start 2024-01-15T18:01:00.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.840+0000 7f887f200640 1 -- 172.21.15.26:0/3532898352 shutdown_connections 2024-01-15T18:01:00.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.840+0000 7f887f200640 1 -- 172.21.15.26:0/3532898352 >> 172.21.15.26:0/3532898352 conn(0x7f88800d3000 msgr2=0x7f88800d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:00.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.840+0000 7f887f200640 1 -- 172.21.15.26:0/3532898352 shutdown_connections 2024-01-15T18:01:00.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.840+0000 7f887f200640 1 -- 172.21.15.26:0/3532898352 wait complete. 2024-01-15T18:01:00.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.843+0000 7f887f200640 1 Processor -- start 2024-01-15T18:01:00.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.843+0000 7f887f200640 1 -- start start 2024-01-15T18:01:00.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.844+0000 7f887f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f88800a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.844+0000 7f887f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f888009db90 0x7f888015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.844+0000 7f887f200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f888015ad50 0x7f888015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.844+0000 7f887f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88800a0540 con 0x7f8860004d00 2024-01-15T18:01:00.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.844+0000 7f887f200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888015b6f0 con 0x7f888015ad50 2024-01-15T18:01:00.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.844+0000 7f887f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88800b7380 con 0x7f888009db90 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.844+0000 7f887de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f88800a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f88800a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46496/0 (socket says 172.21.15.26:46496) 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887de00640 1 -- 172.21.15.26:0/2745441762 learned_addr learned my addr 172.21.15.26:0/2745441762 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887d400640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f888009db90 0x7f888015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887de00640 1 -- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f888009db90 msgr2=0x7f888015a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887de00640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f888009db90 0x7f888015a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887de00640 1 -- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f888015ad50 msgr2=0x7f888015b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887de00640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f888015ad50 0x7f888015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887de00640 1 -- 172.21.15.26:0/2745441762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888015da10 con 0x7f8860004d00 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887d400640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f888009db90 0x7f888015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.845+0000 7f887de00640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f88800a0000 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8870011aa0 tx=0x7f8870011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.846+0000 7f8876000640 1 -- 172.21.15.26:0/2745441762 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8870009680 con 0x7f8860004d00 2024-01-15T18:01:00.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.846+0000 7f8876000640 1 -- 172.21.15.26:0/2745441762 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887000fb00 con 0x7f8860004d00 2024-01-15T18:01:00.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.846+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88800a0d30 con 0x7f8860004d00 2024-01-15T18:01:00.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.846+0000 7f8876000640 1 -- 172.21.15.26:0/2745441762 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887000fe90 con 0x7f8860004d00 2024-01-15T18:01:00.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.846+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88800a0a50 con 0x7f8860004d00 2024-01-15T18:01:00.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.847+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8880067450 con 0x7f8860004d00 2024-01-15T18:01:00.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.848+0000 7f8876000640 1 -- 172.21.15.26:0/2745441762 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 15) v1 ==== 90049+0+0 (secure 0 0 0) 0x7f8870014440 con 0x7f8860004d00 2024-01-15T18:01:00.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.849+0000 7f8876000640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f885806b860 0x7f885806dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:00.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.849+0000 7f8876000640 1 -- 172.21.15.26:0/2745441762 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f887008aec0 con 0x7f8860004d00 2024-01-15T18:01:00.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.850+0000 7f887d400640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f885806b860 0x7f885806dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:00.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.851+0000 7f887d400640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f885806b860 0x7f885806dd20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8880071ae0 tx=0x7f886800b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:00.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:00.853+0000 7f8876000640 1 -- 172.21.15.26:0/2745441762 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88700554f0 con 0x7f8860004d00 2024-01-15T18:01:01.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.077+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f88800b4270 con 0x7f885806b860 2024-01-15T18:01:01.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.083+0000 7f8876000640 1 -- 172.21.15.26:0/2745441762 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f88800b4270 con 0x7f885806b860 2024-01-15T18:01:01.087 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f885806b860 msgr2=0x7f885806dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f885806b860 0x7f885806dd20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8880071ae0 tx=0x7f886800b040 comp rx=0 tx=0).stop 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 msgr2=0x7f88800a0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f88800a0000 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8870011aa0 tx=0x7f8870011f60 comp rx=0 tx=0).stop 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 shutdown_connections 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f888009db90 0x7f888015a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f885806b860 0x7f885806dd20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f888015ad50 0x7f888015b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 --2- 172.21.15.26:0/2745441762 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8860004d00 0x7f88800a0000 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 >> 172.21.15.26:0/2745441762 conn(0x7f88800d3000 msgr2=0x7f888006b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:01.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 shutdown_connections 2024-01-15T18:01:01.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:01.086+0000 7f887f200640 1 -- 172.21.15.26:0/2745441762 wait complete. 2024-01-15T18:01:01.110 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10 2024-01-15T18:01:01.110 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:01:01.110 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:01:01.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:01:01.163 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:01:17.762 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:01:17.763 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:01:17.763 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.5147 s, 65.0 MB/s 2024-01-15T18:01:17.765 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:01:17.765 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:01:17.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:01:17.821 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:01:27.477 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:01:27.486 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:01:31.352 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:01:31.352 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:01:31.352 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 13.4645 s, 79.7 MB/s 2024-01-15T18:01:31.354 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:01:31.354 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:01:31.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:01:31.411 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:01:46.135 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:01:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:01:46.136 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.6566 s, 73.3 MB/s 2024-01-15T18:01:46.138 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_delay 10 2024-01-15T18:01:46.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.364+0000 7fd2b6000640 1 Processor -- start 2024-01-15T18:01:46.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.365+0000 7fd2b6000640 1 -- start start 2024-01-15T18:01:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.366+0000 7fd2b6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 0x7fd2b0070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.366+0000 7fd2b6000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2b015bf10 0x7fd2b0143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.366+0000 7fd2b6000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b015f4e0 con 0x7fd2b00adf20 2024-01-15T18:01:46.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.366+0000 7fd2b6000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b015f970 con 0x7fd2b015bf10 2024-01-15T18:01:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.366+0000 7fd2b6000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b015c3c0 con 0x7fd2b00ccb50 2024-01-15T18:01:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.367+0000 7fd2b5600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2b015bf10 0x7fd2b0143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.367+0000 7fd2b5600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2b015bf10 0x7fd2b0143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40488/0 (socket says 172.21.15.26:40488) 2024-01-15T18:01:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.367+0000 7fd2b5600640 1 -- 172.21.15.26:0/2461077421 learned_addr learned my addr 172.21.15.26:0/2461077421 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.367+0000 7fd2afe00640 1 --2- 172.21.15.26:0/2461077421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 0x7fd2b0070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.367+0000 7fd2b5600640 1 -- 172.21.15.26:0/2461077421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 msgr2=0x7fd2b0070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.367+0000 7fd2b5600640 1 --2- 172.21.15.26:0/2461077421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 0x7fd2b0070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.367+0000 7fd2b5600640 1 -- 172.21.15.26:0/2461077421 >> v1:172.21.15.124:6789/0 conn(0x7fd2b00adf20 legacy=0x7fd2b00cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.367+0000 7fd2b5600640 1 -- 172.21.15.26:0/2461077421 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b0071130 con 0x7fd2b015bf10 2024-01-15T18:01:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.369+0000 7fd2b5600640 1 --2- 172.21.15.26:0/2461077421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 0x7fd2b0143d30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009e50 tx=0x7fd2a802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bcd7b6a2f9c6b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.369+0000 7fd2af400640 1 -- 172.21.15.26:0/2461077421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a802ca70 con 0x7fd2b015bf10 2024-01-15T18:01:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.369+0000 7fd2af400640 1 -- 172.21.15.26:0/2461077421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a802cbd0 con 0x7fd2b015bf10 2024-01-15T18:01:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.369+0000 7fd2af400640 1 -- 172.21.15.26:0/2461077421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80343f0 con 0x7fd2b015bf10 2024-01-15T18:01:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.370+0000 7fd2b6000640 1 -- 172.21.15.26:0/2461077421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 msgr2=0x7fd2b0143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.370+0000 7fd2b6000640 1 --2- 172.21.15.26:0/2461077421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 0x7fd2b0143d30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009e50 tx=0x7fd2a802bed0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.370+0000 7fd2b6000640 1 -- 172.21.15.26:0/2461077421 shutdown_connections 2024-01-15T18:01:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.370+0000 7fd2b6000640 1 --2- 172.21.15.26:0/2461077421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 0x7fd2b0143d30 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.370+0000 7fd2b6000640 1 --2- 172.21.15.26:0/2461077421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 0x7fd2b0070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.370+0000 7fd2b6000640 1 -- 172.21.15.26:0/2461077421 >> 172.21.15.26:0/2461077421 conn(0x7fd2b006a7d0 msgr2=0x7fd2b006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.370+0000 7fd2b6000640 1 -- 172.21.15.26:0/2461077421 shutdown_connections 2024-01-15T18:01:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.370+0000 7fd2b6000640 1 -- 172.21.15.26:0/2461077421 wait complete. 2024-01-15T18:01:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.372+0000 7fd2b6000640 1 Processor -- start 2024-01-15T18:01:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.372+0000 7fd2b6000640 1 -- start start 2024-01-15T18:01:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.373+0000 7fd2b6000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2b00adf20 0x7fd2b015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.373+0000 7fd2b6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 0x7fd2b013e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.373+0000 7fd2b6000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 0x7fd2b013e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.373+0000 7fd2b6000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b00b0110 con 0x7fd2b015bf10 2024-01-15T18:01:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.373+0000 7fd2b6000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b015ae30 con 0x7fd2b00adf20 2024-01-15T18:01:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.373+0000 7fd2b6000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b00af490 con 0x7fd2b00ccb50 2024-01-15T18:01:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.373+0000 7fd2b5600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 0x7fd2b013e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.373+0000 7fd2b5600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 0x7fd2b013e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40504/0 (socket says 172.21.15.26:40504) 2024-01-15T18:01:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.373+0000 7fd2b5600640 1 -- 172.21.15.26:0/2904233661 learned_addr learned my addr 172.21.15.26:0/2904233661 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2b4c00640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2b00adf20 0x7fd2b015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2afe00640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 0x7fd2b013e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2b5600640 1 -- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 msgr2=0x7fd2b013e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2b5600640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 0x7fd2b013e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2b5600640 1 -- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2b00adf20 msgr2=0x7fd2b015a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2b5600640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2b00adf20 0x7fd2b015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2b5600640 1 -- 172.21.15.26:0/2904233661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8009b30 con 0x7fd2b015bf10 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2b5600640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 0x7fd2b013e600 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009e50 tx=0x7fd2a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2acc00640 1 -- 172.21.15.26:0/2904233661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a803e450 con 0x7fd2b015bf10 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2acc00640 1 -- 172.21.15.26:0/2904233661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a803e5b0 con 0x7fd2b015bf10 2024-01-15T18:01:46.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2acc00640 1 -- 172.21.15.26:0/2904233661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a803e9a0 con 0x7fd2b015bf10 2024-01-15T18:01:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.374+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b00af770 con 0x7fd2b015bf10 2024-01-15T18:01:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.375+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b00afd30 con 0x7fd2b015bf10 2024-01-15T18:01:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.376+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b00b0250 con 0x7fd2b015bf10 2024-01-15T18:01:46.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.377+0000 7fd2acc00640 1 -- 172.21.15.26:0/2904233661 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 90100+0+0 (secure 0 0 0) 0x7fd2a803eb00 con 0x7fd2b015bf10 2024-01-15T18:01:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.378+0000 7fd2acc00640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd28806b920 0x7fd28806dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.378+0000 7fd2acc00640 1 -- 172.21.15.26:0/2904233661 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd2a80ad260 con 0x7fd2b015bf10 2024-01-15T18:01:46.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.378+0000 7fd2b4c00640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd28806b920 0x7fd28806dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.379+0000 7fd2b4c00640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd28806b920 0x7fd28806dde0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00015a0 tx=0x7fd2a0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:46.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.382+0000 7fd2acc00640 1 -- 172.21.15.26:0/2904233661 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2b00b0250 con 0x7fd2b015bf10 2024-01-15T18:01:46.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.578+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}] v 0) v1 -- 0x7fd2b00b0250 con 0x7fd2b015bf10 2024-01-15T18:01:46.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.583+0000 7fd2acc00640 1 -- 172.21.15.26:0/2904233661 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}]=0 v21)=0 v21) v1 ==== 131+0+0 (secure 0 0 0) 0x7fd2b00b0250 con 0x7fd2b015bf10 2024-01-15T18:01:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd28806b920 msgr2=0x7fd28806dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd28806b920 0x7fd28806dde0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00015a0 tx=0x7fd2a0009040 comp rx=0 tx=0).stop 2024-01-15T18:01:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 msgr2=0x7fd2b013e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 0x7fd2b013e600 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009e50 tx=0x7fd2a8004a40 comp rx=0 tx=0).stop 2024-01-15T18:01:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 shutdown_connections 2024-01-15T18:01:46.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2b00ccb50 0x7fd2b013e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd28806b920 0x7fd28806dde0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2b015bf10 0x7fd2b013e600 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 --2- 172.21.15.26:0/2904233661 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2b00adf20 0x7fd2b015a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 >> 172.21.15.26:0/2904233661 conn(0x7fd2b006a7d0 msgr2=0x7fd2b006abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.586+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 shutdown_connections 2024-01-15T18:01:46.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.587+0000 7fd2b6000640 1 -- 172.21.15.26:0/2904233661 wait complete. 2024-01-15T18:01:46.607 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-15T18:01:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.828+0000 7f9fa8000640 1 Processor -- start 2024-01-15T18:01:46.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.828+0000 7f9fa8000640 1 -- start start 2024-01-15T18:01:46.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.830+0000 7f9fa8000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9fa013df80 0x7f9fa015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.830+0000 7f9fa8000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9fa013e360 0x7f9fa00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.830+0000 7f9fa8000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.830+0000 7f9fa8000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa006c470 con 0x7f9fa013e360 2024-01-15T18:01:46.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.830+0000 7f9fa8000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa006c750 con 0x7f9fa013df80 2024-01-15T18:01:46.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.830+0000 7f9fa8000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa006ca30 con 0x7f9fa015f9c0 2024-01-15T18:01:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa4c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9fa013e360 0x7f9fa00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa5600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9fa013df80 0x7f9fa015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa00cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:58700/0 (socket says 172.21.15.26:58700) 2024-01-15T18:01:46.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa5600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9fa013df80 0x7f9fa015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:48306/0 (socket says 172.21.15.26:48306) 2024-01-15T18:01:46.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa6000640 1 -- 172.21.15.26:0/327253298 learned_addr learned my addr 172.21.15.26:0/327253298 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:46.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa6000640 1 -- 172.21.15.26:0/327253298 >> v2:172.21.15.124:3300/0 conn(0x7f9fa013df80 msgr2=0x7f9fa015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:46.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa6000640 1 --2- 172.21.15.26:0/327253298 >> v2:172.21.15.124:3300/0 conn(0x7f9fa013df80 0x7f9fa015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa6000640 1 -- 172.21.15.26:0/327253298 >> v2:172.21.15.26:3300/0 conn(0x7f9fa013e360 msgr2=0x7f9fa00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:46.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa6000640 1 --2- 172.21.15.26:0/327253298 >> v2:172.21.15.26:3300/0 conn(0x7f9fa013e360 0x7f9fa00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.831+0000 7f9fa6000640 1 -- 172.21.15.26:0/327253298 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa006cd10 con 0x7f9fa015f9c0 2024-01-15T18:01:46.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.833+0000 7f9fa6000640 1 --2- 172.21.15.26:0/327253298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa00cf7d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0072770 tx=0x7f9f9802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb286f45cac8b6d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:46.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9f9fe00640 1 -- 172.21.15.26:0/327253298 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9802ca80 con 0x7f9fa015f9c0 2024-01-15T18:01:46.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9f9fe00640 1 -- 172.21.15.26:0/327253298 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9802cbe0 con 0x7f9fa015f9c0 2024-01-15T18:01:46.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9f9fe00640 1 -- 172.21.15.26:0/327253298 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980343f0 con 0x7f9fa015f9c0 2024-01-15T18:01:46.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9fa8000640 1 -- 172.21.15.26:0/327253298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 msgr2=0x7f9fa00cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:46.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9fa8000640 1 --2- 172.21.15.26:0/327253298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa00cf7d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0072770 tx=0x7f9f9802bee0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9fa8000640 1 -- 172.21.15.26:0/327253298 shutdown_connections 2024-01-15T18:01:46.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9fa8000640 1 --2- 172.21.15.26:0/327253298 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa00cf7d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9fa8000640 1 --2- 172.21.15.26:0/327253298 >> v2:172.21.15.26:3300/0 conn(0x7f9fa013e360 0x7f9fa00cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9fa8000640 1 --2- 172.21.15.26:0/327253298 >> v2:172.21.15.124:3300/0 conn(0x7f9fa013df80 0x7f9fa015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.834+0000 7f9fa8000640 1 -- 172.21.15.26:0/327253298 >> 172.21.15.26:0/327253298 conn(0x7f9fa006a800 msgr2=0x7f9fa006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:46.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.835+0000 7f9fa8000640 1 -- 172.21.15.26:0/327253298 shutdown_connections 2024-01-15T18:01:46.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.835+0000 7f9fa8000640 1 -- 172.21.15.26:0/327253298 wait complete. 2024-01-15T18:01:46.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.836+0000 7f9fa8000640 1 Processor -- start 2024-01-15T18:01:46.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.837+0000 7f9fa8000640 1 -- start start 2024-01-15T18:01:46.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.837+0000 7f9fa8000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa013df80 0x7f9fa0154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.837+0000 7f9fa8000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fa013e360 0x7f9fa00a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.837+0000 7f9fa8000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa015de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.837+0000 7f9fa8000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00a8cb0 con 0x7f9fa013e360 2024-01-15T18:01:46.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.837+0000 7f9fa8000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00a9610 con 0x7f9fa013df80 2024-01-15T18:01:46.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.837+0000 7f9fa8000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa00b2cb0 con 0x7f9fa015f9c0 2024-01-15T18:01:46.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fa013e360 0x7f9fa00a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fa013e360 0x7f9fa00a90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40510/0 (socket says 172.21.15.26:40510) 2024-01-15T18:01:46.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa4c00640 1 -- 172.21.15.26:0/206593765 learned_addr learned my addr 172.21.15.26:0/206593765 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:46.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa6000640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa5600640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa013df80 0x7f9fa0154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa4c00640 1 -- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 msgr2=0x7f9fa015de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:46.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa4c00640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa015de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa4c00640 1 -- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa013df80 msgr2=0x7f9fa0154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:46.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa4c00640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa013df80 0x7f9fa0154860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa4c00640 1 -- 172.21.15.26:0/206593765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98009b30 con 0x7f9fa013e360 2024-01-15T18:01:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.838+0000 7f9fa6000640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa015de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:01:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.839+0000 7f9fa4c00640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fa013e360 0x7f9fa00a90d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800e9b0 tx=0x7f9f8800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.839+0000 7f9f9d600640 1 -- 172.21.15.26:0/206593765 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88009670 con 0x7f9fa013e360 2024-01-15T18:01:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.839+0000 7f9f9d600640 1 -- 172.21.15.26:0/206593765 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8800c9a0 con 0x7f9fa013e360 2024-01-15T18:01:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.839+0000 7f9f9d600640 1 -- 172.21.15.26:0/206593765 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800cd30 con 0x7f9fa013e360 2024-01-15T18:01:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.839+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa00b29d0 con 0x7f9fa013e360 2024-01-15T18:01:46.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.839+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa00a88d0 con 0x7f9fa013e360 2024-01-15T18:01:46.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.840+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa00a8df0 con 0x7f9fa013e360 2024-01-15T18:01:46.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.841+0000 7f9f9d600640 1 -- 172.21.15.26:0/206593765 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 90100+0+0 (secure 0 0 0) 0x7f9f88013470 con 0x7f9fa013e360 2024-01-15T18:01:46.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.842+0000 7f9f9d600640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f7806b8f0 0x7f9f7806ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:46.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.842+0000 7f9f9d600640 1 -- 172.21.15.26:0/206593765 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9f8808a8c0 con 0x7f9fa013e360 2024-01-15T18:01:46.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.843+0000 7f9fa5600640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f7806b8f0 0x7f9f7806ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:46.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.844+0000 7f9fa5600640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f7806b8f0 0x7f9f7806ddb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009970 tx=0x7f9f90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:46.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:46.847+0000 7f9f9d600640 1 -- 172.21.15.26:0/206593765 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fa00a8df0 con 0x7f9fa013e360 2024-01-15T18:01:47.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.041+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f9fa00a8df0 con 0x7f9fa013e360 2024-01-15T18:01:47.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.048+0000 7f9f9d600640 1 -- 172.21.15.26:0/206593765 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v22)=0 v22) v1 ==== 135+0+0 (secure 0 0 0) 0x7f9fa00a8df0 con 0x7f9fa013e360 2024-01-15T18:01:47.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f7806b8f0 msgr2=0x7f9f7806ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f7806b8f0 0x7f9f7806ddb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009970 tx=0x7f9f90008040 comp rx=0 tx=0).stop 2024-01-15T18:01:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fa013e360 msgr2=0x7f9fa00a90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fa013e360 0x7f9fa00a90d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800e9b0 tx=0x7f9f8800ee70 comp rx=0 tx=0).stop 2024-01-15T18:01:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 shutdown_connections 2024-01-15T18:01:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9fa015f9c0 0x7f9fa015de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f7806b8f0 0x7f9f7806ddb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9fa013e360 0x7f9fa00a90d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 --2- 172.21.15.26:0/206593765 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9fa013df80 0x7f9fa0154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.051+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 >> 172.21.15.26:0/206593765 conn(0x7f9fa006a800 msgr2=0x7f9fa00afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.052+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 shutdown_connections 2024-01-15T18:01:47.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.052+0000 7f9fa8000640 1 -- 172.21.15.26:0/206593765 wait complete. 2024-01-15T18:01:47.074 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:01:47.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.291+0000 7f587d800640 1 Processor -- start 2024-01-15T18:01:47.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.291+0000 7f587d800640 1 -- start start 2024-01-15T18:01:47.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.293+0000 7f587d800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 0x7f58780b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.293+0000 7f587d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58780ad760 0x7f587815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.293+0000 7f587d800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587815ea30 con 0x7f587815b930 2024-01-15T18:01:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.293+0000 7f587d800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58780adc40 con 0x7f58780ad760 2024-01-15T18:01:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.293+0000 7f587d800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58780adf20 con 0x7f58780cd4a0 2024-01-15T18:01:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.293+0000 7f5877400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58780ad760 0x7f587815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.293+0000 7f5877400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58780ad760 0x7f587815b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40522/0 (socket says 172.21.15.26:40522) 2024-01-15T18:01:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.293+0000 7f5877400640 1 -- 172.21.15.26:0/2237554418 learned_addr learned my addr 172.21.15.26:0/2237554418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:47.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.294+0000 7f5877e00640 1 --2- 172.21.15.26:0/2237554418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 0x7f58780b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.294+0000 7f5877400640 1 -- 172.21.15.26:0/2237554418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 msgr2=0x7f58780b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.294+0000 7f5877400640 1 --2- 172.21.15.26:0/2237554418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 0x7f58780b3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.294+0000 7f5877400640 1 -- 172.21.15.26:0/2237554418 >> v1:172.21.15.124:6789/0 conn(0x7f587815b930 legacy=0x7f58780cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:47.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.294+0000 7f5877400640 1 -- 172.21.15.26:0/2237554418 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58780ae200 con 0x7f58780ad760 2024-01-15T18:01:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.295+0000 7f5877400640 1 --2- 172.21.15.26:0/2237554418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58780ad760 0x7f587815b3f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5860009870 tx=0x7f586002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf606b1708e975e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.295+0000 7f5876a00640 1 -- 172.21.15.26:0/2237554418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860031640 con 0x7f58780ad760 2024-01-15T18:01:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.295+0000 7f5876a00640 1 -- 172.21.15.26:0/2237554418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58600317a0 con 0x7f58780ad760 2024-01-15T18:01:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.295+0000 7f5876a00640 1 -- 172.21.15.26:0/2237554418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860031b30 con 0x7f58780ad760 2024-01-15T18:01:47.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.296+0000 7f587d800640 1 -- 172.21.15.26:0/2237554418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58780ad760 msgr2=0x7f587815b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.296+0000 7f587d800640 1 --2- 172.21.15.26:0/2237554418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58780ad760 0x7f587815b3f0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5860009870 tx=0x7f586002b8f0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.296+0000 7f587d800640 1 -- 172.21.15.26:0/2237554418 shutdown_connections 2024-01-15T18:01:47.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.296+0000 7f587d800640 1 --2- 172.21.15.26:0/2237554418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58780ad760 0x7f587815b3f0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.296+0000 7f587d800640 1 --2- 172.21.15.26:0/2237554418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 0x7f58780b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.297+0000 7f587d800640 1 -- 172.21.15.26:0/2237554418 >> 172.21.15.26:0/2237554418 conn(0x7f587806a800 msgr2=0x7f587806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:47.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.297+0000 7f587d800640 1 -- 172.21.15.26:0/2237554418 shutdown_connections 2024-01-15T18:01:47.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.297+0000 7f587d800640 1 -- 172.21.15.26:0/2237554418 wait complete. 2024-01-15T18:01:47.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.299+0000 7f587d800640 1 Processor -- start 2024-01-15T18:01:47.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.299+0000 7f587d800640 1 -- start start 2024-01-15T18:01:47.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f587d800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58780ad760 0x7f58781f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f587d800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 0x7f58781425f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f587d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f587815b930 0x7f5878152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f587d800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58781f23c0 con 0x7f587815b930 2024-01-15T18:01:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f587d800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5878155e40 con 0x7f58780ad760 2024-01-15T18:01:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f587d800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58780afa40 con 0x7f58780cd4a0 2024-01-15T18:01:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f587ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f587815b930 0x7f5878152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f587ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f587815b930 0x7f5878152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40524/0 (socket says 172.21.15.26:40524) 2024-01-15T18:01:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f587ce00640 1 -- 172.21.15.26:0/828406494 learned_addr learned my addr 172.21.15.26:0/828406494 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f5877e00640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58780ad760 0x7f58781f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.301+0000 7f5877400640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 0x7f58781425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f587ce00640 1 -- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 msgr2=0x7f58781425f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f587ce00640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 0x7f58781425f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f587ce00640 1 -- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58780ad760 msgr2=0x7f58781f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f587ce00640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58780ad760 0x7f58781f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f587ce00640 1 -- 172.21.15.26:0/828406494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58600094d0 con 0x7f587815b930 2024-01-15T18:01:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f587ce00640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f587815b930 0x7f5878152cf0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f587000b970 tx=0x7f587000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f586fe00640 1 -- 172.21.15.26:0/828406494 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587000cb40 con 0x7f587815b930 2024-01-15T18:01:47.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f586fe00640 1 -- 172.21.15.26:0/828406494 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587000cca0 con 0x7f587815b930 2024-01-15T18:01:47.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58780afd20 con 0x7f587815b930 2024-01-15T18:01:47.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.302+0000 7f586fe00640 1 -- 172.21.15.26:0/828406494 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870014680 con 0x7f587815b930 2024-01-15T18:01:47.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.303+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5878142b30 con 0x7f587815b930 2024-01-15T18:01:47.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.304+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58780af8a0 con 0x7f587815b930 2024-01-15T18:01:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.305+0000 7f586fe00640 1 -- 172.21.15.26:0/828406494 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 90100+0+0 (secure 0 0 0) 0x7f587001d3b0 con 0x7f587815b930 2024-01-15T18:01:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.306+0000 7f586fe00640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f585006b8f0 0x7f585006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.306+0000 7f586fe00640 1 -- 172.21.15.26:0/828406494 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f587008a500 con 0x7f587815b930 2024-01-15T18:01:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.306+0000 7f5877e00640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f585006b8f0 0x7f585006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.307+0000 7f5877e00640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f585006b8f0 0x7f585006ddb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5868006fd0 tx=0x7f5868008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:47.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.310+0000 7f586fe00640 1 -- 172.21.15.26:0/828406494 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58780af8a0 con 0x7f587815b930 2024-01-15T18:01:47.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.534+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7f587806e390 con 0x7f585006b8f0 2024-01-15T18:01:47.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.568+0000 7f586fe00640 1 -- 172.21.15.26:0/828406494 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f587806e390 con 0x7f585006b8f0 2024-01-15T18:01:47.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.570+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f585006b8f0 msgr2=0x7f585006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.570+0000 7f587d800640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f585006b8f0 0x7f585006ddb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5868006fd0 tx=0x7f5868008040 comp rx=0 tx=0).stop 2024-01-15T18:01:47.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.570+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f587815b930 msgr2=0x7f5878152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.570+0000 7f587d800640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f587815b930 0x7f5878152cf0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f587000b970 tx=0x7f587000be30 comp rx=0 tx=0).stop 2024-01-15T18:01:47.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.571+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 shutdown_connections 2024-01-15T18:01:47.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.571+0000 7f587d800640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58780cd4a0 0x7f58781425f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.571+0000 7f587d800640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f585006b8f0 0x7f585006ddb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.571+0000 7f587d800640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f587815b930 0x7f5878152cf0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.571+0000 7f587d800640 1 --2- 172.21.15.26:0/828406494 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f58780ad760 0x7f58781f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.571+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 >> 172.21.15.26:0/828406494 conn(0x7f587806a800 msgr2=0x7f5878159060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:47.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.571+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 shutdown_connections 2024-01-15T18:01:47.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.571+0000 7f587d800640 1 -- 172.21.15.26:0/828406494 wait complete. 2024-01-15T18:01:47.595 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:01:47.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.811+0000 7fc19de00640 1 Processor -- start 2024-01-15T18:01:47.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.812+0000 7fc19de00640 1 -- start start 2024-01-15T18:01:47.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.813+0000 7fc19de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc19815d4b0 0x7fc1980d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.813+0000 7fc19de00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1980b2730 con 0x7fc1980cf9a0 2024-01-15T18:01:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.813+0000 7fc19de00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1980d1ec0 con 0x7fc1980b41e0 2024-01-15T18:01:47.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.813+0000 7fc19de00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980b29d0 con 0x7fc19815d4b0 2024-01-15T18:01:47.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.814+0000 7fc18fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc1980cf9a0 0x7fc1980cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58214/0 (socket says 172.21.15.26:58214) 2024-01-15T18:01:47.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.814+0000 7fc18fe00640 1 -- 172.21.15.26:0/2507449916 learned_addr learned my addr 172.21.15.26:0/2507449916 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:01:47.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.814+0000 7fc19ca00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc19815d4b0 0x7fc1980d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.814+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982707753 0 0) 0x7fc1980b2730 con 0x7fc1980cf9a0 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.814+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc178003520 con 0x7fc1980cf9a0 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 672239364 0 0) 0x7fc178003520 con 0x7fc1980cf9a0 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc19815d4b0 msgr2=0x7fc1980d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc19815d4b0 0x7fc1980d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 >> v1:172.21.15.124:6789/0 conn(0x7fc1980b41e0 legacy=0x7fc1980ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980b2cb0 con 0x7fc1980cf9a0 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fc180001670 con 0x7fc1980cf9a0 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 >> v1:172.21.15.26:6789/0 conn(0x7fc1980cf9a0 legacy=0x7fc1980cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc19815d4b0 0x7fc1780039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc1780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc178004d00 0x7fc178005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980b29d0 con 0x7fc178003ee0 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17800b9e0 con 0x7fc178004d00 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17800bba0 con 0x7fc19815d4b0 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.815+0000 7fc18fe00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc1780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.816+0000 7fc19ca00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc19815d4b0 0x7fc1780039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.816+0000 7fc18fe00640 1 -- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc19815d4b0 msgr2=0x7fc1780039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.816+0000 7fc18fe00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc19815d4b0 0x7fc1780039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.816+0000 7fc18fe00640 1 -- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc178004d00 msgr2=0x7fc178005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.816+0000 7fc18fe00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc178004d00 0x7fc178005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.816+0000 7fc18fe00640 1 -- 172.21.15.26:0/2507449916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178003520 con 0x7fc178003ee0 2024-01-15T18:01:47.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.816+0000 7fc19ca00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc19815d4b0 0x7fc1780039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:01:47.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.817+0000 7fc18fe00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc1780047c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc180004f60 tx=0x7fc1780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e27be18edeea01 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:47.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.817+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180005ad0 con 0x7fc178003ee0 2024-01-15T18:01:47.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.817+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18002f070 con 0x7fc178003ee0 2024-01-15T18:01:47.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.817+0000 7fc197e00640 1 -- 172.21.15.26:0/2507449916 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800080d0 con 0x7fc178003ee0 2024-01-15T18:01:47.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.818+0000 7fc19de00640 1 -- 172.21.15.26:0/2507449916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 msgr2=0x7fc1780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.818+0000 7fc19de00640 1 --2- 172.21.15.26:0/2507449916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc1780047c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc180004f60 tx=0x7fc1780057f0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.818+0000 7fc19d400640 1 -- 172.21.15.26:0/2507449916 reap_dead start 2024-01-15T18:01:47.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.818+0000 7fc19de00640 1 -- 172.21.15.26:0/2507449916 shutdown_connections 2024-01-15T18:01:47.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.818+0000 7fc19de00640 1 -- 172.21.15.26:0/2507449916 >> 172.21.15.26:0/2507449916 conn(0x7fc1980d3000 msgr2=0x7fc1980d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:47.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.818+0000 7fc19de00640 1 -- 172.21.15.26:0/2507449916 shutdown_connections 2024-01-15T18:01:47.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.818+0000 7fc19de00640 1 -- 172.21.15.26:0/2507449916 wait complete. 2024-01-15T18:01:47.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.820+0000 7fc19de00640 1 Processor -- start 2024-01-15T18:01:47.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.820+0000 7fc19de00640 1 -- start start 2024-01-15T18:01:47.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.821+0000 7fc19de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc198151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.821+0000 7fc19de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1980c87c0 0x7fc1980c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.821+0000 7fc19de00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1980afbd0 0x7fc1980b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.821+0000 7fc19de00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1980b0590 con 0x7fc178003ee0 2024-01-15T18:01:47.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.821+0000 7fc19de00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981f7060 con 0x7fc1980afbd0 2024-01-15T18:01:47.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.821+0000 7fc19de00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1981f6450 con 0x7fc1980c87c0 2024-01-15T18:01:47.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.821+0000 7fc19ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc198151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.821+0000 7fc19ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc198151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40532/0 (socket says 172.21.15.26:40532) 2024-01-15T18:01:47.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.821+0000 7fc19ca00640 1 -- 172.21.15.26:0/1288114677 learned_addr learned my addr 172.21.15.26:0/1288114677 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:47.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc19d400640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1980afbd0 0x7fc1980b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc18fe00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1980c87c0 0x7fc1980c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc19ca00640 1 -- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1980c87c0 msgr2=0x7fc1980c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc19ca00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1980c87c0 0x7fc1980c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc19ca00640 1 -- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1980afbd0 msgr2=0x7fc1980b0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc19ca00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1980afbd0 0x7fc1980b0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc19ca00640 1 -- 172.21.15.26:0/1288114677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19815da10 con 0x7fc178003ee0 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc18fe00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1980c87c0 0x7fc1980c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc19ca00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc198151410 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc1880108e0 tx=0x7fc188010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc195600640 1 -- 172.21.15.26:0/1288114677 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18800ec20 con 0x7fc178003ee0 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc195600640 1 -- 172.21.15.26:0/1288114677 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18800ed80 con 0x7fc178003ee0 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc195600640 1 -- 172.21.15.26:0/1288114677 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18801c600 con 0x7fc178003ee0 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.822+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1981f6730 con 0x7fc178003ee0 2024-01-15T18:01:47.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.823+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1981f6cf0 con 0x7fc178003ee0 2024-01-15T18:01:47.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.824+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc198067450 con 0x7fc178003ee0 2024-01-15T18:01:47.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.825+0000 7fc195600640 1 -- 172.21.15.26:0/1288114677 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 90100+0+0 (secure 0 0 0) 0x7fc18801c760 con 0x7fc178003ee0 2024-01-15T18:01:47.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.826+0000 7fc195600640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc17006b920 0x7fc17006dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:47.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.826+0000 7fc195600640 1 -- 172.21.15.26:0/1288114677 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc18808ade0 con 0x7fc178003ee0 2024-01-15T18:01:47.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.827+0000 7fc18fe00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc17006b920 0x7fc17006dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:47.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.828+0000 7fc18fe00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc17006b920 0x7fc17006dde0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc180034000 tx=0x7fc180038040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:47.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:47.830+0000 7fc195600640 1 -- 172.21.15.26:0/1288114677 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc188091080 con 0x7fc178003ee0 2024-01-15T18:01:48.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.060+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc19815d890 con 0x7fc17006b920 2024-01-15T18:01:48.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.072+0000 7fc195600640 1 -- 172.21.15.26:0/1288114677 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fc18003c040 con 0x7fc17006b920 2024-01-15T18:01:48.076 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10 2024-01-15T18:01:48.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc17006b920 msgr2=0x7fc17006dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:48.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc17006b920 0x7fc17006dde0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc180034000 tx=0x7fc180038040 comp rx=0 tx=0).stop 2024-01-15T18:01:48.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 msgr2=0x7fc198151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:48.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc198151410 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc1880108e0 tx=0x7fc188010da0 comp rx=0 tx=0).stop 2024-01-15T18:01:48.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 shutdown_connections 2024-01-15T18:01:48.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1980c87c0 0x7fc1980c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:48.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc17006b920 0x7fc17006dde0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:48.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1980afbd0 0x7fc1980b0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:48.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 --2- 172.21.15.26:0/1288114677 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc178003ee0 0x7fc198151410 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:48.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 >> 172.21.15.26:0/1288114677 conn(0x7fc1980d3000 msgr2=0x7fc198160800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:48.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 shutdown_connections 2024-01-15T18:01:48.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:48.075+0000 7fc19de00640 1 -- 172.21.15.26:0/1288114677 wait complete. 2024-01-15T18:01:48.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:01:48.099 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10) 2024-01-15T18:01:48.157 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10 2024-01-15T18:01:48.158 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:01:48.158 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:01:48.159 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:01:50.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:01:50.160 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10) 2024-01-15T18:01:50.231 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10 2024-01-15T18:01:50.231 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:01:50.231 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:01:50.232 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:01:52.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:01:52.235 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10) 2024-01-15T18:01:52.305 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/bb493521-c94d-4448-9e2e-6e0794999a10 2024-01-15T18:01:52.305 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:01:52.305 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:01:52.306 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:01:52.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone20831 2024-01-15T18:01:52.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.540+0000 7fcfc1600640 1 Processor -- start 2024-01-15T18:01:52.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.540+0000 7fcfc1600640 1 -- start start 2024-01-15T18:01:52.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.542+0000 7fcfc1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc0cc500 0x7fcfbc0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.542+0000 7fcfc1600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcfbc06d160 0x7fcfbc06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.542+0000 7fcfc1600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfbc070e50 con 0x7fcfbc143c40 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.542+0000 7fcfc1600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc0723c0 con 0x7fcfbc06d160 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.542+0000 7fcfc1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc144020 con 0x7fcfbc0cc500 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.542+0000 7fcfc0c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcfbc06d160 0x7fcfbc06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.542+0000 7fcfc0c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcfbc06d160 0x7fcfbc06d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57984/0 (socket says 172.21.15.26:57984) 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.542+0000 7fcfc0c00640 1 -- 172.21.15.26:0/1988872163 learned_addr learned my addr 172.21.15.26:0/1988872163 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.543+0000 7fcfb3200640 1 --2- 172.21.15.26:0/1988872163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc0cc500 0x7fcfbc0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.543+0000 7fcfc0c00640 1 -- 172.21.15.26:0/1988872163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc0cc500 msgr2=0x7fcfbc0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:52.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.543+0000 7fcfc0c00640 1 --2- 172.21.15.26:0/1988872163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc0cc500 0x7fcfbc0cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.543+0000 7fcfc0c00640 1 -- 172.21.15.26:0/1988872163 >> v1:172.21.15.26:6789/0 conn(0x7fcfbc143c40 legacy=0x7fcfbc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:52.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.543+0000 7fcfc0c00640 1 -- 172.21.15.26:0/1988872163 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0ccea0 con 0x7fcfbc06d160 2024-01-15T18:01:52.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.543+0000 7fcfb3200640 1 --2- 172.21.15.26:0/1988872163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc0cc500 0x7fcfbc0cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:01:52.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.544+0000 7fcfc0c00640 1 --2- 172.21.15.26:0/1988872163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 0x7fcfbc06d590 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009870 tx=0x7fcfb402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95aa32e1db3ec064 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:52.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.544+0000 7fcfbb400640 1 -- 172.21.15.26:0/1988872163 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4031640 con 0x7fcfbc06d160 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfbb400640 1 -- 172.21.15.26:0/1988872163 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb40317a0 con 0x7fcfbc06d160 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfbb400640 1 -- 172.21.15.26:0/1988872163 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4031b30 con 0x7fcfbc06d160 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfc1600640 1 -- 172.21.15.26:0/1988872163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 msgr2=0x7fcfbc06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfc1600640 1 --2- 172.21.15.26:0/1988872163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 0x7fcfbc06d590 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009870 tx=0x7fcfb402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfc1600640 1 -- 172.21.15.26:0/1988872163 shutdown_connections 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfc1600640 1 --2- 172.21.15.26:0/1988872163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 0x7fcfbc06d590 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfc1600640 1 --2- 172.21.15.26:0/1988872163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc0cc500 0x7fcfbc0cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfc1600640 1 -- 172.21.15.26:0/1988872163 >> 172.21.15.26:0/1988872163 conn(0x7fcfbc06a800 msgr2=0x7fcfbc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfc1600640 1 -- 172.21.15.26:0/1988872163 shutdown_connections 2024-01-15T18:01:52.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.545+0000 7fcfc1600640 1 -- 172.21.15.26:0/1988872163 wait complete. 2024-01-15T18:01:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.547+0000 7fcfc1600640 1 Processor -- start 2024-01-15T18:01:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.547+0000 7fcfc1600640 1 -- start start 2024-01-15T18:01:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.548+0000 7fcfc1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 0x7fcfbc12d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.548+0000 7fcfc1600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfbc0cc500 0x7fcfbc1e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.548+0000 7fcfc1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc143c40 0x7fcfbc1e85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:52.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.548+0000 7fcfc1600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc1e7000 con 0x7fcfbc06d160 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.548+0000 7fcfc1600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc1e72e0 con 0x7fcfbc0cc500 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.548+0000 7fcfc1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc1e75c0 con 0x7fcfbc143c40 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.548+0000 7fcfbbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 0x7fcfbc12d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.548+0000 7fcfbbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 0x7fcfbc12d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57990/0 (socket says 172.21.15.26:57990) 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.548+0000 7fcfbbe00640 1 -- 172.21.15.26:0/513485655 learned_addr learned my addr 172.21.15.26:0/513485655 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.549+0000 7fcfbbe00640 1 -- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc143c40 msgr2=0x7fcfbc1e85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.549+0000 7fcfbbe00640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc143c40 0x7fcfbc1e85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.549+0000 7fcfb3200640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfbc0cc500 0x7fcfbc1e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.549+0000 7fcfbbe00640 1 -- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfbc0cc500 msgr2=0x7fcfbc1e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:52.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.549+0000 7fcfbbe00640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfbc0cc500 0x7fcfbc1e80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.549+0000 7fcfbbe00640 1 -- 172.21.15.26:0/513485655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40094d0 con 0x7fcfbc06d160 2024-01-15T18:01:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.549+0000 7fcfbbe00640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 0x7fcfbc12d460 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00c730 tx=0x7fcfac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.549+0000 7fcfb8c00640 1 -- 172.21.15.26:0/513485655 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac003b70 con 0x7fcfbc06d160 2024-01-15T18:01:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.550+0000 7fcfb8c00640 1 -- 172.21.15.26:0/513485655 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac003cf0 con 0x7fcfbc06d160 2024-01-15T18:01:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.550+0000 7fcfb8c00640 1 -- 172.21.15.26:0/513485655 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac003ea0 con 0x7fcfbc06d160 2024-01-15T18:01:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.550+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc1e78a0 con 0x7fcfbc06d160 2024-01-15T18:01:52.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.550+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc1e0920 con 0x7fcfbc06d160 2024-01-15T18:01:52.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.551+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc0cdf30 con 0x7fcfbc06d160 2024-01-15T18:01:52.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.552+0000 7fcfb8c00640 1 -- 172.21.15.26:0/513485655 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 90100+0+0 (secure 0 0 0) 0x7fcfac00aa90 con 0x7fcfbc06d160 2024-01-15T18:01:52.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.553+0000 7fcfb8c00640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9406b8f0 0x7fcf9406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:52.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.553+0000 7fcfb8c00640 1 -- 172.21.15.26:0/513485655 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fcfac08a5d0 con 0x7fcfbc06d160 2024-01-15T18:01:52.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.554+0000 7fcfb3200640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9406b8f0 0x7fcf9406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:52.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.555+0000 7fcfb3200640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9406b8f0 0x7fcf9406ddb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc1305f0 tx=0x7fcfa4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:52.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.558+0000 7fcfb8c00640 1 -- 172.21.15.26:0/513485655 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfac054aa0 con 0x7fcfbc06d160 2024-01-15T18:01:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.793+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone20831", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfbc1e7450 con 0x7fcf9406b8f0 2024-01-15T18:01:52.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.938+0000 7fcfb8c00640 1 -- 172.21.15.26:0/513485655 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcfbc1e7450 con 0x7fcf9406b8f0 2024-01-15T18:01:52.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.940+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9406b8f0 msgr2=0x7fcf9406ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.940+0000 7fcfc1600640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9406b8f0 0x7fcf9406ddb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc1305f0 tx=0x7fcfa4008040 comp rx=0 tx=0).stop 2024-01-15T18:01:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.940+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 msgr2=0x7fcfbc12d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.940+0000 7fcfc1600640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 0x7fcfbc12d460 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00c730 tx=0x7fcfac00cbf0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.941+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 shutdown_connections 2024-01-15T18:01:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.941+0000 7fcfc1600640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfbc143c40 0x7fcfbc1e85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.941+0000 7fcfc1600640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9406b8f0 0x7fcf9406ddb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.941+0000 7fcfc1600640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfbc0cc500 0x7fcfbc1e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.941+0000 7fcfc1600640 1 --2- 172.21.15.26:0/513485655 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfbc06d160 0x7fcfbc12d460 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:52.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.941+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 >> 172.21.15.26:0/513485655 conn(0x7fcfbc06a800 msgr2=0x7fcfbc06d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.941+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 shutdown_connections 2024-01-15T18:01:52.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:52.941+0000 7fcfc1600640 1 -- 172.21.15.26:0/513485655 wait complete. 2024-01-15T18:01:52.951 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.947+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:52.951 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.947+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:52.951 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.947+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:52.951 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.947+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:52.951 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.947+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:52.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone20832 2024-01-15T18:01:52.973 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.970+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:52.973 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.970+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:52.974 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.970+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:52.974 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.970+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:52.974 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:01:52.970+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:01:53.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.188+0000 7f85a2c00640 1 Processor -- start 2024-01-15T18:01:53.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.189+0000 7f85a2c00640 1 -- start start 2024-01-15T18:01:53.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.190+0000 7f85a2c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f859c13df80 0x7f859c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.190+0000 7f85a2c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f859c0cf9a0 0x7f859c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.190+0000 7f85a2c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859c06c470 con 0x7f859c13e360 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.190+0000 7f85a2c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c06c750 con 0x7f859c13df80 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.190+0000 7f85a2c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c06ca30 con 0x7f859c0cf9a0 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.190+0000 7f859be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f859c13df80 0x7f859c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.191+0000 7f859be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f859c13df80 0x7f859c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58004/0 (socket says 172.21.15.26:58004) 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.191+0000 7f859be00640 1 -- 172.21.15.26:0/109666440 learned_addr learned my addr 172.21.15.26:0/109666440 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.191+0000 7f859be00640 1 -- 172.21.15.26:0/109666440 >> v2:172.21.15.124:3300/0 conn(0x7f859c0cf9a0 msgr2=0x7f859c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.191+0000 7f859be00640 1 --2- 172.21.15.26:0/109666440 >> v2:172.21.15.124:3300/0 conn(0x7f859c0cf9a0 0x7f859c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.191+0000 7f859be00640 1 -- 172.21.15.26:0/109666440 >> v1:172.21.15.124:6789/0 conn(0x7f859c13e360 legacy=0x7f859c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:53.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.191+0000 7f859be00640 1 -- 172.21.15.26:0/109666440 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c06cd10 con 0x7f859c13df80 2024-01-15T18:01:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.192+0000 7f859be00640 1 --2- 172.21.15.26:0/109666440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13df80 0x7f859c15bee0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f859c0ccb10 tx=0x7f858c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f409be8d8b1d0a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.192+0000 7f859b400640 1 -- 172.21.15.26:0/109666440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c031870 con 0x7f859c13df80 2024-01-15T18:01:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.192+0000 7f859b400640 1 -- 172.21.15.26:0/109666440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c0319d0 con 0x7f859c13df80 2024-01-15T18:01:53.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.192+0000 7f859b400640 1 -- 172.21.15.26:0/109666440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c031d60 con 0x7f859c13df80 2024-01-15T18:01:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.193+0000 7f85a2c00640 1 -- 172.21.15.26:0/109666440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13df80 msgr2=0x7f859c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.193+0000 7f85a2c00640 1 --2- 172.21.15.26:0/109666440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13df80 0x7f859c15bee0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f859c0ccb10 tx=0x7f858c02b910 comp rx=0 tx=0).stop 2024-01-15T18:01:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.193+0000 7f85a2c00640 1 -- 172.21.15.26:0/109666440 shutdown_connections 2024-01-15T18:01:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.193+0000 7f85a2c00640 1 --2- 172.21.15.26:0/109666440 >> v2:172.21.15.124:3300/0 conn(0x7f859c0cf9a0 0x7f859c0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.193+0000 7f85a2c00640 1 --2- 172.21.15.26:0/109666440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13df80 0x7f859c15bee0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.193+0000 7f85a2c00640 1 -- 172.21.15.26:0/109666440 >> 172.21.15.26:0/109666440 conn(0x7f859c06a800 msgr2=0x7f859c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.193+0000 7f85a2c00640 1 -- 172.21.15.26:0/109666440 shutdown_connections 2024-01-15T18:01:53.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.193+0000 7f85a2c00640 1 -- 172.21.15.26:0/109666440 wait complete. 2024-01-15T18:01:53.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.196+0000 7f85a2c00640 1 Processor -- start 2024-01-15T18:01:53.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.196+0000 7f85a2c00640 1 -- start start 2024-01-15T18:01:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.197+0000 7f85a2c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f859c0cf9a0 0x7f859c13d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.197+0000 7f85a2c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f859c13df80 0x7f859c152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.197+0000 7f85a2c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13e360 0x7f859c153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.197+0000 7f85a2c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c1f3170 con 0x7f859c13e360 2024-01-15T18:01:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.197+0000 7f85a2c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c153610 con 0x7f859c0cf9a0 2024-01-15T18:01:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.197+0000 7f85a2c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c1538f0 con 0x7f859c13df80 2024-01-15T18:01:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.197+0000 7f85a0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13e360 0x7f859c153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.197+0000 7f85a0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13e360 0x7f859c153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58014/0 (socket says 172.21.15.26:58014) 2024-01-15T18:01:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.197+0000 7f85a0c00640 1 -- 172.21.15.26:0/1948440028 learned_addr learned my addr 172.21.15.26:0/1948440028 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:53.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f8593e00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f859c13df80 0x7f859c152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f859be00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f859c0cf9a0 0x7f859c13d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f85a0c00640 1 -- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f859c13df80 msgr2=0x7f859c152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f85a0c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f859c13df80 0x7f859c152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f85a0c00640 1 -- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f859c0cf9a0 msgr2=0x7f859c13d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f85a0c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f859c0cf9a0 0x7f859c13d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f85a0c00640 1 -- 172.21.15.26:0/1948440028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0094d0 con 0x7f859c13e360 2024-01-15T18:01:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f859be00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f859c0cf9a0 0x7f859c13d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:01:53.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f8593e00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f859c13df80 0x7f859c152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:01:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.198+0000 7f85a0c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13e360 0x7f859c153020 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f859400b9e0 tx=0x7f859400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.199+0000 7f8598c00640 1 -- 172.21.15.26:0/1948440028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859400cb20 con 0x7f859c13e360 2024-01-15T18:01:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.199+0000 7f8598c00640 1 -- 172.21.15.26:0/1948440028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859400cca0 con 0x7f859c13e360 2024-01-15T18:01:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.199+0000 7f8598c00640 1 -- 172.21.15.26:0/1948440028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594014680 con 0x7f859c13e360 2024-01-15T18:01:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.199+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859c153bd0 con 0x7f859c13e360 2024-01-15T18:01:53.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.199+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859c1e5400 con 0x7f859c13e360 2024-01-15T18:01:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.200+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859c0675d0 con 0x7f859c13e360 2024-01-15T18:01:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.201+0000 7f8598c00640 1 -- 172.21.15.26:0/1948440028 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 90100+0+0 (secure 0 0 0) 0x7f859401d3b0 con 0x7f859c13e360 2024-01-15T18:01:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.202+0000 7f8598c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f857406b8f0 0x7f857406ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.202+0000 7f8598c00640 1 -- 172.21.15.26:0/1948440028 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f859408a6c0 con 0x7f859c13e360 2024-01-15T18:01:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.203+0000 7f859be00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f857406b8f0 0x7f857406ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.204+0000 7f859be00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f857406b8f0 0x7f857406ddb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f858c036000 tx=0x7f858c03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:53.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.207+0000 7f8598c00640 1 -- 172.21.15.26:0/1948440028 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8594054bd0 con 0x7f859c13e360 2024-01-15T18:01:53.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.432+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone20832", "target": ["mon-mgr", ""]}) v1 -- 0x7f859c15f9c0 con 0x7f857406b8f0 2024-01-15T18:01:53.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.615+0000 7f8598c00640 1 -- 172.21.15.26:0/1948440028 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f858c031d10 con 0x7f857406b8f0 2024-01-15T18:01:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f857406b8f0 msgr2=0x7f857406ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f857406b8f0 0x7f857406ddb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f858c036000 tx=0x7f858c03c040 comp rx=0 tx=0).stop 2024-01-15T18:01:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13e360 msgr2=0x7f859c153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:53.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13e360 0x7f859c153020 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f859400b9e0 tx=0x7f859400bea0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 shutdown_connections 2024-01-15T18:01:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f859c13df80 0x7f859c152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f857406b8f0 0x7f857406ddb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f859c13e360 0x7f859c153020 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 --2- 172.21.15.26:0/1948440028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f859c0cf9a0 0x7f859c13d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 >> 172.21.15.26:0/1948440028 conn(0x7f859c06a800 msgr2=0x7f859c070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.618+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 shutdown_connections 2024-01-15T18:01:53.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.619+0000 7f85a2c00640 1 -- 172.21.15.26:0/1948440028 wait complete. 2024-01-15T18:01:53.643 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone20833 2024-01-15T18:01:53.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.876+0000 7f1343200640 1 Processor -- start 2024-01-15T18:01:53.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.876+0000 7f1343200640 1 -- start start 2024-01-15T18:01:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.878+0000 7f1343200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f134415afb0 0x7f134413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.878+0000 7f1343200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134413fae0 0x7f13440b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.878+0000 7f1343200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f134413ff30 con 0x7f134413cfa0 2024-01-15T18:01:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.878+0000 7f1343200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13440cdc40 con 0x7f134415afb0 2024-01-15T18:01:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.878+0000 7f1343200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13440cdec0 con 0x7f134413fae0 2024-01-15T18:01:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.878+0000 7f1341e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f134413cfa0 0x7f13440cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57596/0 (socket says 172.21.15.26:57596) 2024-01-15T18:01:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.878+0000 7f1341e00640 1 -- 172.21.15.26:0/2669642003 learned_addr learned my addr 172.21.15.26:0/2669642003 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:01:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457841505 0 0) 0x7f1334001110 con 0x7f134413cfa0 2024-01-15T18:01:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1324003520 con 0x7f134413cfa0 2024-01-15T18:01:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1342800640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134413fae0 0x7f13440b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1341400640 1 --2- 172.21.15.26:0/2669642003 >> v2:172.21.15.124:3300/0 conn(0x7f134415afb0 0x7f134413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1381080192 0 0) 0x7f1324003520 con 0x7f134413cfa0 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134413fae0 msgr2=0x7f13440b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134413fae0 0x7f13440b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 >> v2:172.21.15.124:3300/0 conn(0x7f134415afb0 msgr2=0x7f134413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 --2- 172.21.15.26:0/2669642003 >> v2:172.21.15.124:3300/0 conn(0x7f134415afb0 0x7f134413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13440ce1a0 con 0x7f134413cfa0 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f1334001800 con 0x7f134413cfa0 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1341400640 1 --2- 172.21.15.26:0/2669642003 >> v2:172.21.15.124:3300/0 conn(0x7f134415afb0 0x7f134413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 >> v1:172.21.15.26:6789/0 conn(0x7f134413cfa0 legacy=0x7f13440cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:53.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1324003970 0x7f1324003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240047c0 0x7f1324004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134413fae0 0x7f1324005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440cdc40 con 0x7f13240047c0 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13440cdec0 con 0x7f1324003970 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.879+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132400bab0 con 0x7f134413fae0 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.880+0000 7f1341400640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240047c0 0x7f1324004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.880+0000 7f1341e00640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1324003970 0x7f1324003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.880+0000 7f1342800640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134413fae0 0x7f1324005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.880+0000 7f1341400640 1 -- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134413fae0 msgr2=0x7f1324005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.880+0000 7f1341400640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134413fae0 0x7f1324005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.880+0000 7f1341400640 1 -- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1324003970 msgr2=0x7f1324003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.880+0000 7f1341400640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1324003970 0x7f1324003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.880+0000 7f1341400640 1 -- 172.21.15.26:0/2669642003 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1324003520 con 0x7f13240047c0 2024-01-15T18:01:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.880+0000 7f1342800640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f134413fae0 0x7f1324005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:01:53.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.881+0000 7f1341400640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240047c0 0x7f1324004c40 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f132c006fd0 tx=0x7f13240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1dad1d242df9d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:53.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.882+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c004230 con 0x7f13240047c0 2024-01-15T18:01:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.882+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c004390 con 0x7f13240047c0 2024-01-15T18:01:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.882+0000 7f1340a00640 1 -- 172.21.15.26:0/2669642003 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c003830 con 0x7f13240047c0 2024-01-15T18:01:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.882+0000 7f1343200640 1 -- 172.21.15.26:0/2669642003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240047c0 msgr2=0x7f1324004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.882+0000 7f1343200640 1 --2- 172.21.15.26:0/2669642003 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f13240047c0 0x7f1324004c40 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f132c006fd0 tx=0x7f13240057f0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.882+0000 7f1342800640 1 -- 172.21.15.26:0/2669642003 reap_dead start 2024-01-15T18:01:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.882+0000 7f1343200640 1 -- 172.21.15.26:0/2669642003 shutdown_connections 2024-01-15T18:01:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.882+0000 7f1343200640 1 -- 172.21.15.26:0/2669642003 >> 172.21.15.26:0/2669642003 conn(0x7f134406a800 msgr2=0x7f134406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.883+0000 7f1343200640 1 -- 172.21.15.26:0/2669642003 shutdown_connections 2024-01-15T18:01:53.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.883+0000 7f1343200640 1 -- 172.21.15.26:0/2669642003 wait complete. 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.885+0000 7f1343200640 1 Processor -- start 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.885+0000 7f1343200640 1 -- start start 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1343200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13240047c0 0x7f1344151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1343200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1344071430 0x7f13440718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1343200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1344131ca0 0x7f1344132120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1343200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1344071df0 con 0x7f1344071430 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1343200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1344132fd0 con 0x7f1344131ca0 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1343200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1344132a10 con 0x7f13240047c0 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1341400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1344071430 0x7f13440718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1341400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1344071430 0x7f13440718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58038/0 (socket says 172.21.15.26:58038) 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1341400640 1 -- 172.21.15.26:0/2430725373 learned_addr learned my addr 172.21.15.26:0/2430725373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:53.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.886+0000 7f1342800640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1344131ca0 0x7f1344132120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f1341e00640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13240047c0 0x7f1344151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f1341400640 1 -- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13240047c0 msgr2=0x7f1344151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f1341400640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13240047c0 0x7f1344151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f1341400640 1 -- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1344131ca0 msgr2=0x7f1344132120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f1341400640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1344131ca0 0x7f1344132120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f1341400640 1 -- 172.21.15.26:0/2430725373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1344132cf0 con 0x7f1344071430 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f1341400640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1344071430 0x7f13440718b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f132c02bfc0 tx=0x7f132c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f133a000640 1 -- 172.21.15.26:0/2430725373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c030660 con 0x7f1344071430 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f133a000640 1 -- 172.21.15.26:0/2430725373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c0307c0 con 0x7f1344071430 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1344155b40 con 0x7f1344071430 2024-01-15T18:01:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.887+0000 7f133a000640 1 -- 172.21.15.26:0/2430725373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c030b30 con 0x7f1344071430 2024-01-15T18:01:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.888+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1344132660 con 0x7f1344071430 2024-01-15T18:01:53.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.889+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13441559a0 con 0x7f1344071430 2024-01-15T18:01:53.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.890+0000 7f133a000640 1 -- 172.21.15.26:0/2430725373 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 16) v1 ==== 90100+0+0 (secure 0 0 0) 0x7f132c030c90 con 0x7f1344071430 2024-01-15T18:01:53.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.891+0000 7f133a000640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f131c06b8f0 0x7f131c06ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:53.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.891+0000 7f133a000640 1 -- 172.21.15.26:0/2430725373 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f132c0ac550 con 0x7f1344071430 2024-01-15T18:01:53.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.892+0000 7f1341e00640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f131c06b8f0 0x7f131c06ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:53.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.893+0000 7f1341e00640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f131c06b8f0 0x7f131c06ddb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1334000c00 tx=0x7f1334009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:53.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:53.896+0000 7f133a000640 1 -- 172.21.15.26:0/2430725373 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13441559a0 con 0x7f1344071430 2024-01-15T18:01:54.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:54.124+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone20833", "target": ["mon-mgr", ""]}) v1 -- 0x7f1344071df0 con 0x7f131c06b8f0 2024-01-15T18:01:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:55.008+0000 7f133a000640 1 -- 172.21.15.26:0/2430725373 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 17) v1 ==== 90202+0+0 (secure 0 0 0) 0x7f132c076eb0 con 0x7f1344071430 2024-01-15T18:01:56.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.873+0000 7f133a000640 1 -- 172.21.15.26:0/2430725373 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1344071df0 con 0x7f131c06b8f0 2024-01-15T18:01:56.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f131c06b8f0 msgr2=0x7f131c06ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:56.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f131c06b8f0 0x7f131c06ddb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1334000c00 tx=0x7f1334009040 comp rx=0 tx=0).stop 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1344071430 msgr2=0x7f13440718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1344071430 0x7f13440718b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f132c02bfc0 tx=0x7f132c004730 comp rx=0 tx=0).stop 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 shutdown_connections 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f131c06b8f0 0x7f131c06ddb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1344131ca0 0x7f1344132120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1344071430 0x7f13440718b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 --2- 172.21.15.26:0/2430725373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f13240047c0 0x7f1344151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 >> 172.21.15.26:0/2430725373 conn(0x7f134406a800 msgr2=0x7f134413bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 shutdown_connections 2024-01-15T18:01:56.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:56.876+0000 7f1343200640 1 -- 172.21.15.26:0/2430725373 wait complete. 2024-01-15T18:01:56.903 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone20834 2024-01-15T18:01:57.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.136+0000 7f00d3600640 1 Processor -- start 2024-01-15T18:01:57.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.136+0000 7f00d3600640 1 -- start start 2024-01-15T18:01:57.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.138+0000 7f00d3600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00cc0ce8f0 0x7f00cc0ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.138+0000 7f00d3600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc0b2a00 0x7f00cc0b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.138+0000 7f00d3600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc0d0de0 con 0x7f00cc160f80 2024-01-15T18:01:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.138+0000 7f00d3600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00cc0d10c0 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.138+0000 7f00d3600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0d13a0 con 0x7f00cc0b2a00 2024-01-15T18:01:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.139+0000 7f00c3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00cc0ce8f0 0x7f00cc0ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.139+0000 7f00d1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc0b2a00 0x7f00cc0b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.139+0000 7f00d0c00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f00cc160f80 0x7f00cc161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:55368/0 (socket says 172.21.15.26:55368) 2024-01-15T18:01:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.139+0000 7f00d0c00640 1 -- 172.21.15.26:0/2166699403 learned_addr learned my addr 172.21.15.26:0/2166699403 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:01:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.139+0000 7f00cbe00640 1 -- 172.21.15.26:0/2166699403 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894198494 0 0) 0x7f00cc0d0de0 con 0x7f00cc160f80 2024-01-15T18:01:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.139+0000 7f00cbe00640 1 -- 172.21.15.26:0/2166699403 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00ac003520 con 0x7f00cc160f80 2024-01-15T18:01:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.140+0000 7f00d1600640 1 -- 172.21.15.26:0/2166699403 >> v2:172.21.15.26:3300/0 conn(0x7f00cc0ce8f0 msgr2=0x7f00cc0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:01:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.140+0000 7f00d1600640 1 --2- 172.21.15.26:0/2166699403 >> v2:172.21.15.26:3300/0 conn(0x7f00cc0ce8f0 0x7f00cc0ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.140+0000 7f00d1600640 1 -- 172.21.15.26:0/2166699403 >> v1:172.21.15.124:6789/0 conn(0x7f00cc160f80 legacy=0x7f00cc161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.140+0000 7f00d1600640 1 -- 172.21.15.26:0/2166699403 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc0d1680 con 0x7f00cc0b2a00 2024-01-15T18:01:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.140+0000 7f00c3e00640 1 --2- 172.21.15.26:0/2166699403 >> v2:172.21.15.26:3300/0 conn(0x7f00cc0ce8f0 0x7f00cc0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:01:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.141+0000 7f00d1600640 1 --2- 172.21.15.26:0/2166699403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc0b2a00 0x7f00cc0b2e90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009d90 tx=0x7f00c402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce62eac246bcc7ea server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00cbe00640 1 -- 172.21.15.26:0/2166699403 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c402cc60 con 0x7f00cc0b2a00 2024-01-15T18:01:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00cbe00640 1 -- 172.21.15.26:0/2166699403 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c402cdc0 con 0x7f00cc0b2a00 2024-01-15T18:01:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00cbe00640 1 -- 172.21.15.26:0/2166699403 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4034660 con 0x7f00cc0b2a00 2024-01-15T18:01:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00d3600640 1 -- 172.21.15.26:0/2166699403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc0b2a00 msgr2=0x7f00cc0b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00d3600640 1 --2- 172.21.15.26:0/2166699403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc0b2a00 0x7f00cc0b2e90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009d90 tx=0x7f00c402beb0 comp rx=0 tx=0).stop 2024-01-15T18:01:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00d3600640 1 -- 172.21.15.26:0/2166699403 shutdown_connections 2024-01-15T18:01:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00d3600640 1 --2- 172.21.15.26:0/2166699403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc0b2a00 0x7f00cc0b2e90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00d3600640 1 --2- 172.21.15.26:0/2166699403 >> v2:172.21.15.26:3300/0 conn(0x7f00cc0ce8f0 0x7f00cc0ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00d3600640 1 -- 172.21.15.26:0/2166699403 >> 172.21.15.26:0/2166699403 conn(0x7f00cc09c550 msgr2=0x7f00cc0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:01:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00d3600640 1 -- 172.21.15.26:0/2166699403 shutdown_connections 2024-01-15T18:01:57.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.142+0000 7f00d3600640 1 -- 172.21.15.26:0/2166699403 wait complete. 2024-01-15T18:01:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.144+0000 7f00d3600640 1 Processor -- start 2024-01-15T18:01:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.144+0000 7f00d3600640 1 -- start start 2024-01-15T18:01:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.145+0000 7f00d3600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00cc0b2a00 0x7f00cc0b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.145+0000 7f00d3600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00cc0ce8f0 0x7f00cc0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.145+0000 7f00d3600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc160f80 0x7f00cc0b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.145+0000 7f00d3600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0b7f10 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.145+0000 7f00d3600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0b8180 con 0x7f00cc0b2a00 2024-01-15T18:01:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.145+0000 7f00d3600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0b8460 con 0x7f00cc160f80 2024-01-15T18:01:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00cc0ce8f0 0x7f00cc0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00cc0ce8f0 0x7f00cc0b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58060/0 (socket says 172.21.15.26:58060) 2024-01-15T18:01:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00c3e00640 1 -- 172.21.15.26:0/2131142373 learned_addr learned my addr 172.21.15.26:0/2131142373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:01:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00d0c00640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00cc0b2a00 0x7f00cc0b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00d1600640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc160f80 0x7f00cc0b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00c3e00640 1 -- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc160f80 msgr2=0x7f00cc0b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00c3e00640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc160f80 0x7f00cc0b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00c3e00640 1 -- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00cc0b2a00 msgr2=0x7f00cc0b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:01:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00c3e00640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00cc0b2a00 0x7f00cc0b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:01:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.146+0000 7f00c3e00640 1 -- 172.21.15.26:0/2131142373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b4009400 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.147+0000 7f00c3e00640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00cc0ce8f0 0x7f00cc0b6060 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f00b400ebe0 tx=0x7f00b400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.147+0000 7f00c9600640 1 -- 172.21.15.26:0/2131142373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4009760 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.147+0000 7f00c9600640 1 -- 172.21.15.26:0/2131142373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b400cc30 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.147+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c4009a70 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.147+0000 7f00c9600640 1 -- 172.21.15.26:0/2131142373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b400eee0 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.147+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00cc1470c0 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.148+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00cc0675d0 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.149+0000 7f00c9600640 1 -- 172.21.15.26:0/2131142373 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 90202+0+0 (secure 0 0 0) 0x7f00b4013470 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.150+0000 7f00c9600640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f00a406ba10 0x7f00a406ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:01:57.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.151+0000 7f00c9600640 1 -- 172.21.15.26:0/2131142373 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f00b4088860 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.151+0000 7f00d0c00640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f00a406ba10 0x7f00a406ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:01:57.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.152+0000 7f00d0c00640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f00a406ba10 0x7f00a406ded0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f00bc005880 tx=0x7f00bc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:01:57.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.155+0000 7f00c9600640 1 -- 172.21.15.26:0/2131142373 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00b4088570 con 0x7f00cc0ce8f0 2024-01-15T18:01:57.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:01:57.396+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone20834", "target": ["mon-mgr", ""]}) v1 -- 0x7f00cc0d1c40 con 0x7f00a406ba10 2024-01-15T18:01:57.616 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:01:57.624 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:02:01.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.880+0000 7f00c9600640 1 -- 172.21.15.26:0/2131142373 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f00bc001350 con 0x7f00a406ba10 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f00a406ba10 msgr2=0x7f00a406ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f00a406ba10 0x7f00a406ded0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f00bc005880 tx=0x7f00bc009040 comp rx=0 tx=0).stop 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00cc0ce8f0 msgr2=0x7f00cc0b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00cc0ce8f0 0x7f00cc0b6060 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f00b400ebe0 tx=0x7f00b400c520 comp rx=0 tx=0).stop 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 shutdown_connections 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00cc160f80 0x7f00cc0b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f00a406ba10 0x7f00a406ded0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00cc0ce8f0 0x7f00cc0b6060 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 --2- 172.21.15.26:0/2131142373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00cc0b2a00 0x7f00cc0b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 >> 172.21.15.26:0/2131142373 conn(0x7f00cc09c550 msgr2=0x7f00cc0b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:01.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 shutdown_connections 2024-01-15T18:02:01.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:01.883+0000 7f00d3600640 1 -- 172.21.15.26:0/2131142373 wait complete. 2024-01-15T18:02:01.907 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone20835 2024-01-15T18:02:02.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.138+0000 7fdf44400640 1 Processor -- start 2024-01-15T18:02:02.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.138+0000 7fdf44400640 1 -- start start 2024-01-15T18:02:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.140+0000 7fdf44400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdf3c15b2a0 0x7fdf3c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.140+0000 7fdf44400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf3c0cfaf0 0x7fdf3c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.140+0000 7fdf44400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c0cdea0 0x7fdf3c0cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.140+0000 7fdf44400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c0ce330 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.140+0000 7fdf44400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c0d00a0 con 0x7fdf3c15b2a0 2024-01-15T18:02:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.140+0000 7fdf44400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0b4010 con 0x7fdf3c0cdea0 2024-01-15T18:02:02.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.140+0000 7fdf41000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf3c0cfaf0 0x7fdf3c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.140+0000 7fdf41000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdf3c0cfaf0 0x7fdf3c0cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47196/0 (socket says 172.21.15.26:47196) 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.140+0000 7fdf41000640 1 -- 172.21.15.26:0/97434937 learned_addr learned my addr 172.21.15.26:0/97434937 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.141+0000 7fdf41a00640 1 --2- 172.21.15.26:0/97434937 >> v2:172.21.15.124:3300/0 conn(0x7fdf3c15b2a0 0x7fdf3c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.141+0000 7fdf41000640 1 -- 172.21.15.26:0/97434937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c0cdea0 msgr2=0x7fdf3c0cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.141+0000 7fdf42400640 1 --2- 172.21.15.26:0/97434937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c0cdea0 0x7fdf3c0cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.141+0000 7fdf41000640 1 --2- 172.21.15.26:0/97434937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c0cdea0 0x7fdf3c0cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.141+0000 7fdf41000640 1 -- 172.21.15.26:0/97434937 >> v2:172.21.15.124:3300/0 conn(0x7fdf3c15b2a0 msgr2=0x7fdf3c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.141+0000 7fdf41000640 1 --2- 172.21.15.26:0/97434937 >> v2:172.21.15.124:3300/0 conn(0x7fdf3c15b2a0 0x7fdf3c0cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.141+0000 7fdf41000640 1 -- 172.21.15.26:0/97434937 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c0b42f0 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.141+0000 7fdf41a00640 1 --2- 172.21.15.26:0/97434937 >> v2:172.21.15.124:3300/0 conn(0x7fdf3c15b2a0 0x7fdf3c0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:02.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.141+0000 7fdf42400640 1 --2- 172.21.15.26:0/97434937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c0cdea0 0x7fdf3c0cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:02.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.142+0000 7fdf41000640 1 --2- 172.21.15.26:0/97434937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 0x7fdf3c0cd960 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0d0f80 tx=0x7fdf2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d715b7a84be8f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.142+0000 7fdf3be00640 1 -- 172.21.15.26:0/97434937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24031870 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.142+0000 7fdf3be00640 1 -- 172.21.15.26:0/97434937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf240319d0 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.142+0000 7fdf3be00640 1 -- 172.21.15.26:0/97434937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24031d60 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.143+0000 7fdf44400640 1 -- 172.21.15.26:0/97434937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 msgr2=0x7fdf3c0cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.143+0000 7fdf44400640 1 --2- 172.21.15.26:0/97434937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 0x7fdf3c0cd960 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c0d0f80 tx=0x7fdf2402b910 comp rx=0 tx=0).stop 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.143+0000 7fdf44400640 1 -- 172.21.15.26:0/97434937 shutdown_connections 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.143+0000 7fdf44400640 1 --2- 172.21.15.26:0/97434937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c0cdea0 0x7fdf3c0cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.143+0000 7fdf44400640 1 --2- 172.21.15.26:0/97434937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 0x7fdf3c0cd960 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.143+0000 7fdf44400640 1 --2- 172.21.15.26:0/97434937 >> v2:172.21.15.124:3300/0 conn(0x7fdf3c15b2a0 0x7fdf3c0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.143+0000 7fdf44400640 1 -- 172.21.15.26:0/97434937 >> 172.21.15.26:0/97434937 conn(0x7fdf3c0d1020 msgr2=0x7fdf3c0d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.143+0000 7fdf44400640 1 -- 172.21.15.26:0/97434937 shutdown_connections 2024-01-15T18:02:02.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.143+0000 7fdf44400640 1 -- 172.21.15.26:0/97434937 wait complete. 2024-01-15T18:02:02.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.145+0000 7fdf44400640 1 Processor -- start 2024-01-15T18:02:02.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.145+0000 7fdf44400640 1 -- start start 2024-01-15T18:02:02.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.146+0000 7fdf44400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3c0cdea0 0x7fdf3c09da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:02.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.146+0000 7fdf44400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 0x7fdf3c14f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:02.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.146+0000 7fdf44400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c15b2a0 0x7fdf3c09ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:02.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.146+0000 7fdf44400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c09ed00 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.146+0000 7fdf44400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c09e080 con 0x7fdf3c0cdea0 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.146+0000 7fdf44400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c09e360 con 0x7fdf3c15b2a0 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.146+0000 7fdf41000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 0x7fdf3c14f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf41000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 0x7fdf3c14f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47206/0 (socket says 172.21.15.26:47206) 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf41000640 1 -- 172.21.15.26:0/1777515883 learned_addr learned my addr 172.21.15.26:0/1777515883 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf41a00640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3c0cdea0 0x7fdf3c09da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf42400640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c15b2a0 0x7fdf3c09ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf41000640 1 -- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c15b2a0 msgr2=0x7fdf3c09ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf41000640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c15b2a0 0x7fdf3c09ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf41000640 1 -- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3c0cdea0 msgr2=0x7fdf3c09da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf41000640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3c0cdea0 0x7fdf3c09da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf41000640 1 -- 172.21.15.26:0/1777515883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf240094d0 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.147+0000 7fdf41a00640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3c0cdea0 0x7fdf3c09da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:02:02.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.148+0000 7fdf41000640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 0x7fdf3c14f410 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdf2402b950 tx=0x7fdf240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:02.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.148+0000 7fdf39600640 1 -- 172.21.15.26:0/1777515883 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24031860 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.148+0000 7fdf39600640 1 -- 172.21.15.26:0/1777515883 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf240443f0 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.148+0000 7fdf39600640 1 -- 172.21.15.26:0/1777515883 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf240447a0 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.148+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c09e640 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.148+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c14f950 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.150+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c09ee40 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.150+0000 7fdf39600640 1 -- 172.21.15.26:0/1777515883 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 90202+0+0 (secure 0 0 0) 0x7fdf2404d450 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.151+0000 7fdf39600640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf1406ba10 0x7fdf1406ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:02.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.151+0000 7fdf39600640 1 -- 172.21.15.26:0/1777515883 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdf2402d040 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.152+0000 7fdf41a00640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf1406ba10 0x7fdf1406ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:02.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.153+0000 7fdf41a00640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf1406ba10 0x7fdf1406ded0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00a910 tx=0x7fdf2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:02.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.156+0000 7fdf39600640 1 -- 172.21.15.26:0/1777515883 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf3c09ee40 con 0x7fdf3c0cfaf0 2024-01-15T18:02:02.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:02.390+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone20835", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf3c0b6de0 con 0x7fdf1406ba10 2024-01-15T18:02:06.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.872+0000 7fdf39600640 1 -- 172.21.15.26:0/1777515883 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdf2c00a170 con 0x7fdf1406ba10 2024-01-15T18:02:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf1406ba10 msgr2=0x7fdf1406ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf1406ba10 0x7fdf1406ded0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00a910 tx=0x7fdf2c008040 comp rx=0 tx=0).stop 2024-01-15T18:02:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 msgr2=0x7fdf3c14f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 0x7fdf3c14f410 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdf2402b950 tx=0x7fdf240046c0 comp rx=0 tx=0).stop 2024-01-15T18:02:06.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 shutdown_connections 2024-01-15T18:02:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf3c15b2a0 0x7fdf3c09ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf1406ba10 0x7fdf1406ded0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf3c0cfaf0 0x7fdf3c14f410 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 --2- 172.21.15.26:0/1777515883 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf3c0cdea0 0x7fdf3c09da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 >> 172.21.15.26:0/1777515883 conn(0x7fdf3c0d1020 msgr2=0x7fdf3c0cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 shutdown_connections 2024-01-15T18:02:06.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:06.875+0000 7fdf44400640 1 -- 172.21.15.26:0/1777515883 wait complete. 2024-01-15T18:02:06.899 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone20836 2024-01-15T18:02:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.125+0000 7f26c8a00640 1 Processor -- start 2024-01-15T18:02:07.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.125+0000 7f26c8a00640 1 -- start start 2024-01-15T18:02:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.126+0000 7f26c8a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4143c40 0x7f26c4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.126+0000 7f26c8a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f26c40ccc10 0x7f26c406d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.126+0000 7f26c8a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c4070e50 con 0x7f26c40717d0 2024-01-15T18:02:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.127+0000 7f26c8a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c40723c0 con 0x7f26c40ccc10 2024-01-15T18:02:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.127+0000 7f26c8a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4144020 con 0x7f26c4143c40 2024-01-15T18:02:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.127+0000 7f26bbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f26c40717d0 0x7f26c40cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58978/0 (socket says 172.21.15.26:58978) 2024-01-15T18:02:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.127+0000 7f26bbe00640 1 -- 172.21.15.26:0/780190435 learned_addr learned my addr 172.21.15.26:0/780190435 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.127+0000 7f26c3200640 1 --2- 172.21.15.26:0/780190435 >> v2:172.21.15.124:3300/0 conn(0x7f26c40ccc10 0x7f26c406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.127+0000 7f26c2800640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4143c40 0x7f26c4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176049416 0 0) 0x7f26c4070e50 con 0x7f26c40717d0 2024-01-15T18:02:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26a4003520 con 0x7f26c40717d0 2024-01-15T18:02:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440810912 0 0) 0x7f26a4003520 con 0x7f26c40717d0 2024-01-15T18:02:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4143c40 msgr2=0x7f26c4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4143c40 0x7f26c4071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 >> v2:172.21.15.124:3300/0 conn(0x7f26c40ccc10 msgr2=0x7f26c406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 --2- 172.21.15.26:0/780190435 >> v2:172.21.15.124:3300/0 conn(0x7f26c40ccc10 0x7f26c406d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c4071e00 con 0x7f26c40717d0 2024-01-15T18:02:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f26ac001670 con 0x7f26c40717d0 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 >> v1:172.21.15.26:6789/0 conn(0x7f26c40717d0 legacy=0x7f26c40cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4143c40 0x7f26a4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26a4003f40 0x7f26a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c40723c0 con 0x7f26a4004d00 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4144020 con 0x7f26a4003f40 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.128+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a400ba30 con 0x7f26c4143c40 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.129+0000 7f26c3200640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.129+0000 7f26bbe00640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26a4003f40 0x7f26a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.129+0000 7f26c2800640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4143c40 0x7f26a4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.129+0000 7f26c3200640 1 -- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4143c40 msgr2=0x7f26a4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.129+0000 7f26c3200640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4143c40 0x7f26a4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.129+0000 7f26c3200640 1 -- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26a4003f40 msgr2=0x7f26a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.129+0000 7f26c3200640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26a4003f40 0x7f26a40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.129+0000 7f26c3200640 1 -- 172.21.15.26:0/780190435 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4003520 con 0x7f26a4004d00 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.129+0000 7f26c2800640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4143c40 0x7f26a4003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.130+0000 7f26c3200640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26a4005180 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009fc0 tx=0x7f26a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bb87cc2cb4f48f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.130+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc02cc30 con 0x7f26a4004d00 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.130+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc00f040 con 0x7f26a4004d00 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.130+0000 7f26c1e00640 1 -- 172.21.15.26:0/780190435 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc034440 con 0x7f26a4004d00 2024-01-15T18:02:07.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.131+0000 7f26c8a00640 1 -- 172.21.15.26:0/780190435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 msgr2=0x7f26a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.131+0000 7f26c8a00640 1 --2- 172.21.15.26:0/780190435 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26a4005180 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009fc0 tx=0x7f26a40057f0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.131+0000 7f26c3200640 1 -- 172.21.15.26:0/780190435 reap_dead start 2024-01-15T18:02:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.131+0000 7f26c8a00640 1 -- 172.21.15.26:0/780190435 shutdown_connections 2024-01-15T18:02:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.131+0000 7f26c8a00640 1 -- 172.21.15.26:0/780190435 >> 172.21.15.26:0/780190435 conn(0x7f26c406a800 msgr2=0x7f26c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.131+0000 7f26c8a00640 1 -- 172.21.15.26:0/780190435 shutdown_connections 2024-01-15T18:02:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.131+0000 7f26c8a00640 1 -- 172.21.15.26:0/780190435 wait complete. 2024-01-15T18:02:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.133+0000 7f26c8a00640 1 Processor -- start 2024-01-15T18:02:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.133+0000 7f26c8a00640 1 -- start start 2024-01-15T18:02:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.134+0000 7f26c8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26c41321d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.134+0000 7f26c8a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4132710 0x7f26c415e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.134+0000 7f26c8a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c415ea90 0x7f26c41420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.134+0000 7f26c8a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c413f690 con 0x7f26a4004d00 2024-01-15T18:02:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.134+0000 7f26c8a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c415ca50 con 0x7f26c415ea90 2024-01-15T18:02:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.134+0000 7f26c8a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c415cd30 con 0x7f26c4132710 2024-01-15T18:02:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.134+0000 7f26c2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26c41321d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26c41321d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47220/0 (socket says 172.21.15.26:47220) 2024-01-15T18:02:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c2800640 1 -- 172.21.15.26:0/2601054169 learned_addr learned my addr 172.21.15.26:0/2601054169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c3200640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c415ea90 0x7f26c41420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26bbe00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4132710 0x7f26c415e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c2800640 1 -- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4132710 msgr2=0x7f26c415e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c2800640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4132710 0x7f26c415e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c2800640 1 -- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c415ea90 msgr2=0x7f26c41420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c2800640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c415ea90 0x7f26c41420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c2800640 1 -- 172.21.15.26:0/2601054169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c415d010 con 0x7f26a4004d00 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c3200640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c415ea90 0x7f26c41420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26bbe00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4132710 0x7f26c415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.135+0000 7f26c2800640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26c41321d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f26b40108b0 tx=0x7f26b4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.136+0000 7f26baa00640 1 -- 172.21.15.26:0/2601054169 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b400eb60 con 0x7f26a4004d00 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.136+0000 7f26baa00640 1 -- 172.21.15.26:0/2601054169 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b400ecc0 con 0x7f26a4004d00 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.136+0000 7f26baa00640 1 -- 172.21.15.26:0/2601054169 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b401c400 con 0x7f26a4004d00 2024-01-15T18:02:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.136+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c415d2f0 con 0x7f26a4004d00 2024-01-15T18:02:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.136+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c41425e0 con 0x7f26a4004d00 2024-01-15T18:02:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.137+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c4067450 con 0x7f26a4004d00 2024-01-15T18:02:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.138+0000 7f26baa00640 1 -- 172.21.15.26:0/2601054169 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 90202+0+0 (secure 0 0 0) 0x7f26b401c560 con 0x7f26a4004d00 2024-01-15T18:02:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.139+0000 7f26baa00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f269c06ba10 0x7f269c06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.139+0000 7f26baa00640 1 -- 172.21.15.26:0/2601054169 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f26b408b3c0 con 0x7f26a4004d00 2024-01-15T18:02:07.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.140+0000 7f26bbe00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f269c06ba10 0x7f269c06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.141+0000 7f26bbe00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f269c06ba10 0x7f269c06ded0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f26ac001eb0 tx=0x7f26ac001b80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.144+0000 7f26baa00640 1 -- 172.21.15.26:0/2601054169 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26b4088860 con 0x7f26a4004d00 2024-01-15T18:02:07.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.379+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone20836", "target": ["mon-mgr", ""]}) v1 -- 0x7f26c406cc40 con 0x7f269c06ba10 2024-01-15T18:02:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.546+0000 7f26baa00640 1 -- 172.21.15.26:0/2601054169 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f26ac001cd0 con 0x7f269c06ba10 2024-01-15T18:02:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.549+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f269c06ba10 msgr2=0x7f269c06ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.549+0000 7f26c8a00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f269c06ba10 0x7f269c06ded0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f26ac001eb0 tx=0x7f26ac001b80 comp rx=0 tx=0).stop 2024-01-15T18:02:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 msgr2=0x7f26c41321d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26c41321d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f26b40108b0 tx=0x7f26b4010d70 comp rx=0 tx=0).stop 2024-01-15T18:02:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 shutdown_connections 2024-01-15T18:02:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f26c4132710 0x7f26c415e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f269c06ba10 0x7f269c06ded0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f26c415ea90 0x7f26c41420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 --2- 172.21.15.26:0/2601054169 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f26a4004d00 0x7f26c41321d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 >> 172.21.15.26:0/2601054169 conn(0x7f26c406a800 msgr2=0x7f26c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 shutdown_connections 2024-01-15T18:02:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.550+0000 7f26c8a00640 1 -- 172.21.15.26:0/2601054169 wait complete. 2024-01-15T18:02:07.577 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone20837 2024-01-15T18:02:07.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.799+0000 7f7119800640 1 Processor -- start 2024-01-15T18:02:07.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.800+0000 7f7119800640 1 -- start start 2024-01-15T18:02:07.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.801+0000 7f7119800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f711413df80 0x7f711415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.801+0000 7f7119800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711413e360 0x7f71140cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.801+0000 7f7119800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711406c470 con 0x7f711415f9c0 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.801+0000 7f7119800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711406c750 con 0x7f711413df80 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.801+0000 7f7119800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711406ca30 con 0x7f711413e360 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.802+0000 7f7112a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f711413df80 0x7f711415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.802+0000 7f7112a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f711413df80 0x7f711415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47236/0 (socket says 172.21.15.26:47236) 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.802+0000 7f7112a00640 1 -- 172.21.15.26:0/2242314303 learned_addr learned my addr 172.21.15.26:0/2242314303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.802+0000 7f7109e00640 1 --2- 172.21.15.26:0/2242314303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711413e360 0x7f71140cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.802+0000 7f7112a00640 1 -- 172.21.15.26:0/2242314303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711413e360 msgr2=0x7f71140cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.802+0000 7f7112a00640 1 --2- 172.21.15.26:0/2242314303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711413e360 0x7f71140cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.802+0000 7f7112a00640 1 -- 172.21.15.26:0/2242314303 >> v1:172.21.15.124:6789/0 conn(0x7f711415f9c0 legacy=0x7f71141408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.802+0000 7f7112a00640 1 -- 172.21.15.26:0/2242314303 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711406cd10 con 0x7f711413df80 2024-01-15T18:02:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.803+0000 7f7112a00640 1 --2- 172.21.15.26:0/2242314303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413df80 0x7f711415bee0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7104009e50 tx=0x7f710402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f94b26f3880df0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.804+0000 7f7112000640 1 -- 172.21.15.26:0/2242314303 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710402ca80 con 0x7f711413df80 2024-01-15T18:02:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.804+0000 7f7112000640 1 -- 172.21.15.26:0/2242314303 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710402cbe0 con 0x7f711413df80 2024-01-15T18:02:07.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.804+0000 7f7112000640 1 -- 172.21.15.26:0/2242314303 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040343f0 con 0x7f711413df80 2024-01-15T18:02:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.804+0000 7f7119800640 1 -- 172.21.15.26:0/2242314303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413df80 msgr2=0x7f711415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:07.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.804+0000 7f7119800640 1 --2- 172.21.15.26:0/2242314303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413df80 0x7f711415bee0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7104009e50 tx=0x7f710402bee0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.804+0000 7f7119800640 1 -- 172.21.15.26:0/2242314303 shutdown_connections 2024-01-15T18:02:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.804+0000 7f7119800640 1 --2- 172.21.15.26:0/2242314303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711413e360 0x7f71140cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.804+0000 7f7119800640 1 --2- 172.21.15.26:0/2242314303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413df80 0x7f711415bee0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.804+0000 7f7119800640 1 -- 172.21.15.26:0/2242314303 >> 172.21.15.26:0/2242314303 conn(0x7f711406a800 msgr2=0x7f711406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.805+0000 7f7119800640 1 -- 172.21.15.26:0/2242314303 shutdown_connections 2024-01-15T18:02:07.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.805+0000 7f7119800640 1 -- 172.21.15.26:0/2242314303 wait complete. 2024-01-15T18:02:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.806+0000 7f7119800640 1 Processor -- start 2024-01-15T18:02:07.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.806+0000 7f7119800640 1 -- start start 2024-01-15T18:02:07.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.807+0000 7f7119800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711413df80 0x7f7114135700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.807+0000 7f7119800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413e360 0x7f711415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.807+0000 7f7119800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711415f9c0 0x7f711415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.807+0000 7f7119800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7114135c40 con 0x7f711413e360 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.807+0000 7f7119800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71140722c0 con 0x7f711413df80 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.807+0000 7f7119800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7114071220 con 0x7f711415f9c0 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7109e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413e360 0x7f711415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7109e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413e360 0x7f711415a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47252/0 (socket says 172.21.15.26:47252) 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7109e00640 1 -- 172.21.15.26:0/148579714 learned_addr learned my addr 172.21.15.26:0/148579714 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7112a00640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711413df80 0x7f7114135700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7113400640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711415f9c0 0x7f711415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7109e00640 1 -- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711415f9c0 msgr2=0x7f711415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7109e00640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711415f9c0 0x7f711415a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7109e00640 1 -- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711413df80 msgr2=0x7f7114135700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7109e00640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711413df80 0x7f7114135700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7109e00640 1 -- 172.21.15.26:0/148579714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104009b30 con 0x7f711413e360 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.808+0000 7f7112a00640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711413df80 0x7f7114135700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.809+0000 7f7109e00640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413e360 0x7f711415a140 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00c7f0 tx=0x7f70fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.809+0000 7f710b400640 1 -- 172.21.15.26:0/148579714 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc004220 con 0x7f711413e360 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.809+0000 7f710b400640 1 -- 172.21.15.26:0/148579714 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc0043a0 con 0x7f711413e360 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.809+0000 7f710b400640 1 -- 172.21.15.26:0/148579714 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc007860 con 0x7f711413e360 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.809+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7114071500 con 0x7f711413e360 2024-01-15T18:02:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.809+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7114071ac0 con 0x7f711413e360 2024-01-15T18:02:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.810+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71140675d0 con 0x7f711413e360 2024-01-15T18:02:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.811+0000 7f710b400640 1 -- 172.21.15.26:0/148579714 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 90202+0+0 (secure 0 0 0) 0x7f70fc007bd0 con 0x7f711413e360 2024-01-15T18:02:07.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.812+0000 7f710b400640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70ec06ba10 0x7f70ec06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:07.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.812+0000 7f710b400640 1 -- 172.21.15.26:0/148579714 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f70fc08adc0 con 0x7f711413e360 2024-01-15T18:02:07.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.813+0000 7f7112a00640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70ec06ba10 0x7f70ec06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:07.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.814+0000 7f7112a00640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70ec06ba10 0x7f70ec06ded0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7104035000 tx=0x7f7104030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:07.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:07.817+0000 7f710b400640 1 -- 172.21.15.26:0/148579714 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f70fc0881e0 con 0x7f711413e360 2024-01-15T18:02:08.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.044+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone20837", "target": ["mon-mgr", ""]}) v1 -- 0x7f71140ce850 con 0x7f70ec06ba10 2024-01-15T18:02:08.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.413+0000 7f710b400640 1 -- 172.21.15.26:0/148579714 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7104044100 con 0x7f70ec06ba10 2024-01-15T18:02:08.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70ec06ba10 msgr2=0x7f70ec06ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:08.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70ec06ba10 0x7f70ec06ded0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7104035000 tx=0x7f7104030000 comp rx=0 tx=0).stop 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413e360 msgr2=0x7f711415a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413e360 0x7f711415a140 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00c7f0 tx=0x7f70fc00ccb0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 shutdown_connections 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f711415f9c0 0x7f711415a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70ec06ba10 0x7f70ec06ded0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f711413e360 0x7f711415a140 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 --2- 172.21.15.26:0/148579714 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f711413df80 0x7f7114135700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 >> 172.21.15.26:0/148579714 conn(0x7f711406a800 msgr2=0x7f71140b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.416+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 shutdown_connections 2024-01-15T18:02:08.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.417+0000 7f7119800640 1 -- 172.21.15.26:0/148579714 wait complete. 2024-01-15T18:02:08.441 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T18:02:08.441 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20831 2024-01-15T18:02:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.666+0000 7fba57e00640 1 Processor -- start 2024-01-15T18:02:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.666+0000 7fba57e00640 1 -- start start 2024-01-15T18:02:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.667+0000 7fba57e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba5815b210 0x7fba580d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.667+0000 7fba57e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fba580cdf00 0x7fba580b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.667+0000 7fba57e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba5815b5f0 con 0x7fba580b01e0 2024-01-15T18:02:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.667+0000 7fba57e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba580cce60 con 0x7fba580cdf00 2024-01-15T18:02:08.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.667+0000 7fba57e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba580cf510 con 0x7fba5815b210 2024-01-15T18:02:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.668+0000 7fba56a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba5815b210 0x7fba580d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.668+0000 7fba56a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba5815b210 0x7fba580d1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:36058/0 (socket says 172.21.15.26:36058) 2024-01-15T18:02:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.668+0000 7fba56a00640 1 -- 172.21.15.26:0/168229145 learned_addr learned my addr 172.21.15.26:0/168229145 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.668+0000 7fba57400640 1 --2- 172.21.15.26:0/168229145 >> v2:172.21.15.124:3300/0 conn(0x7fba580cdf00 0x7fba580b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.669+0000 7fba56a00640 1 -- 172.21.15.26:0/168229145 >> v2:172.21.15.124:3300/0 conn(0x7fba580cdf00 msgr2=0x7fba580b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.669+0000 7fba56a00640 1 --2- 172.21.15.26:0/168229145 >> v2:172.21.15.124:3300/0 conn(0x7fba580cdf00 0x7fba580b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.669+0000 7fba56a00640 1 -- 172.21.15.26:0/168229145 >> v1:172.21.15.124:6789/0 conn(0x7fba580b01e0 legacy=0x7fba580cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.669+0000 7fba56a00640 1 -- 172.21.15.26:0/168229145 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba580cf7f0 con 0x7fba5815b210 2024-01-15T18:02:08.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.670+0000 7fba56a00640 1 --2- 172.21.15.26:0/168229145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba5815b210 0x7fba580d1920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba48009870 tx=0x7fba4802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d1f8673acebb010 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:08.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.671+0000 7fba55600640 1 -- 172.21.15.26:0/168229145 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48031640 con 0x7fba5815b210 2024-01-15T18:02:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.671+0000 7fba55600640 1 -- 172.21.15.26:0/168229145 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba480317a0 con 0x7fba5815b210 2024-01-15T18:02:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.671+0000 7fba55600640 1 -- 172.21.15.26:0/168229145 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba48031b30 con 0x7fba5815b210 2024-01-15T18:02:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.672+0000 7fba57e00640 1 -- 172.21.15.26:0/168229145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba5815b210 msgr2=0x7fba580d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.672+0000 7fba57e00640 1 --2- 172.21.15.26:0/168229145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba5815b210 0x7fba580d1920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba48009870 tx=0x7fba4802b8f0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.672+0000 7fba57e00640 1 -- 172.21.15.26:0/168229145 shutdown_connections 2024-01-15T18:02:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.672+0000 7fba57e00640 1 --2- 172.21.15.26:0/168229145 >> v2:172.21.15.124:3300/0 conn(0x7fba580cdf00 0x7fba580b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.672+0000 7fba57e00640 1 --2- 172.21.15.26:0/168229145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba5815b210 0x7fba580d1920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.672+0000 7fba57e00640 1 -- 172.21.15.26:0/168229145 >> 172.21.15.26:0/168229145 conn(0x7fba580d0e00 msgr2=0x7fba580d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:08.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.672+0000 7fba57e00640 1 -- 172.21.15.26:0/168229145 shutdown_connections 2024-01-15T18:02:08.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.672+0000 7fba57e00640 1 -- 172.21.15.26:0/168229145 wait complete. 2024-01-15T18:02:08.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.674+0000 7fba57e00640 1 Processor -- start 2024-01-15T18:02:08.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.674+0000 7fba57e00640 1 -- start start 2024-01-15T18:02:08.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba57e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba580b01e0 0x7fba580c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:08.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba57e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba580cdf00 0x7fba5809c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba57e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba5815b210 0x7fba58146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba57e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba58145290 con 0x7fba5815b210 2024-01-15T18:02:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba57e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba580c3f00 con 0x7fba580cdf00 2024-01-15T18:02:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba57e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba581454a0 con 0x7fba580b01e0 2024-01-15T18:02:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba57400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba5815b210 0x7fba58146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba57400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba5815b210 0x7fba58146540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47264/0 (socket says 172.21.15.26:47264) 2024-01-15T18:02:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba57400640 1 -- 172.21.15.26:0/3290574919 learned_addr learned my addr 172.21.15.26:0/3290574919 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba56a00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba580b01e0 0x7fba580c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:08.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.675+0000 7fba56000640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba580cdf00 0x7fba5809c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba57400640 1 -- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba580b01e0 msgr2=0x7fba580c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba57400640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba580b01e0 0x7fba580c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba57400640 1 -- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba580cdf00 msgr2=0x7fba5809c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba57400640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba580cdf00 0x7fba5809c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba57400640 1 -- 172.21.15.26:0/3290574919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba480094d0 con 0x7fba5815b210 2024-01-15T18:02:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba57400640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba5815b210 0x7fba58146540 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fba5000ef90 tx=0x7fba5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba4ea00640 1 -- 172.21.15.26:0/3290574919 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba50009ca0 con 0x7fba5815b210 2024-01-15T18:02:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba4ea00640 1 -- 172.21.15.26:0/3290574919 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5000f040 con 0x7fba5815b210 2024-01-15T18:02:08.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba4ea00640 1 -- 172.21.15.26:0/3290574919 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba500146c0 con 0x7fba5815b210 2024-01-15T18:02:08.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba58145780 con 0x7fba5815b210 2024-01-15T18:02:08.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.676+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba58145d40 con 0x7fba5815b210 2024-01-15T18:02:08.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.678+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba58067b60 con 0x7fba5815b210 2024-01-15T18:02:08.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.679+0000 7fba4ea00640 1 -- 172.21.15.26:0/3290574919 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 90202+0+0 (secure 0 0 0) 0x7fba50014820 con 0x7fba5815b210 2024-01-15T18:02:08.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.679+0000 7fba4ea00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba3006ba10 0x7fba3006ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:08.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.680+0000 7fba4ea00640 1 -- 172.21.15.26:0/3290574919 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fba50089410 con 0x7fba5815b210 2024-01-15T18:02:08.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.680+0000 7fba56a00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba3006ba10 0x7fba3006ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:08.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.681+0000 7fba56a00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba3006ba10 0x7fba3006ded0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fba48036000 tx=0x7fba4803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:08.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.685+0000 7fba4ea00640 1 -- 172.21.15.26:0/3290574919 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba50089100 con 0x7fba5815b210 2024-01-15T18:02:08.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.916+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20831", "target": ["mon-mgr", ""]}) v1 -- 0x7fba580b0730 con 0x7fba3006ba10 2024-01-15T18:02:08.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.940+0000 7fba4ea00640 1 -- 172.21.15.26:0/3290574919 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7fba48031990 con 0x7fba3006ba10 2024-01-15T18:02:08.944 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:02:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:02:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:02:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:02:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:02:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:02:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 0.0, 2024-01-15T18:02:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:02:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "0/3", 2024-01-15T18:02:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:02:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:02:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:08.946 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:08.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba3006ba10 msgr2=0x7fba3006ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba3006ba10 0x7fba3006ded0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fba48036000 tx=0x7fba4803c040 comp rx=0 tx=0).stop 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba5815b210 msgr2=0x7fba58146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba5815b210 0x7fba58146540 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fba5000ef90 tx=0x7fba5000c490 comp rx=0 tx=0).stop 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 shutdown_connections 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba3006ba10 0x7fba3006ded0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba5815b210 0x7fba58146540 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba580cdf00 0x7fba5809c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 --2- 172.21.15.26:0/3290574919 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba580b01e0 0x7fba580c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 >> 172.21.15.26:0/3290574919 conn(0x7fba580d0e00 msgr2=0x7fba5806e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:08.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 shutdown_connections 2024-01-15T18:02:08.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:08.942+0000 7fba57e00640 1 -- 172.21.15.26:0/3290574919 wait complete. 2024-01-15T18:02:08.964 INFO:tasks.cephfs.test_volumes:expected clone state = ('pending', 'in-progress') 2024-01-15T18:02:08.964 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20837 2024-01-15T18:02:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.205+0000 7f0895a00640 1 Processor -- start 2024-01-15T18:02:09.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.206+0000 7f0895a00640 1 -- start start 2024-01-15T18:02:09.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.207+0000 7f0895a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0890160a20 0x7f08900cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:09.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.207+0000 7f0895a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f08900b3b20 0x7f08900cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:09.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.207+0000 7f0895a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08900d1e00 con 0x7f0890160e00 2024-01-15T18:02:09.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.207+0000 7f0895a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08900d3c40 con 0x7f0890160a20 2024-01-15T18:02:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.207+0000 7f0895a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08900d3e50 con 0x7f08900b3b20 2024-01-15T18:02:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.207+0000 7f088fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0890160a20 0x7f08900cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.208+0000 7f088f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0890160e00 0x7f08900b2a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58994/0 (socket says 172.21.15.26:58994) 2024-01-15T18:02:09.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.208+0000 7f088f400640 1 -- 172.21.15.26:0/1663856433 learned_addr learned my addr 172.21.15.26:0/1663856433 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:09.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.208+0000 7f0895000640 1 --2- 172.21.15.26:0/1663856433 >> v2:172.21.15.124:3300/0 conn(0x7f08900b3b20 0x7f08900cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:09.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.208+0000 7f088fe00640 1 -- 172.21.15.26:0/1663856433 >> v2:172.21.15.124:3300/0 conn(0x7f08900b3b20 msgr2=0x7f08900cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:09.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.208+0000 7f088fe00640 1 --2- 172.21.15.26:0/1663856433 >> v2:172.21.15.124:3300/0 conn(0x7f08900b3b20 0x7f08900cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.208+0000 7f088fe00640 1 -- 172.21.15.26:0/1663856433 >> v1:172.21.15.26:6789/0 conn(0x7f0890160e00 legacy=0x7f08900b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:09.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.208+0000 7f088ea00640 1 -- 172.21.15.26:0/1663856433 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955517782 0 0) 0x7f08900d1e00 con 0x7f0890160e00 2024-01-15T18:02:09.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.208+0000 7f088fe00640 1 -- 172.21.15.26:0/1663856433 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08900d1a90 con 0x7f0890160a20 2024-01-15T18:02:09.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.209+0000 7f088fe00640 1 --2- 172.21.15.26:0/1663856433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160a20 0x7f08900cf850 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f089008d550 tx=0x7f088002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1768b9a75b46ba95 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:09.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.210+0000 7f088ea00640 1 -- 172.21.15.26:0/1663856433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880031830 con 0x7f0890160a20 2024-01-15T18:02:09.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.210+0000 7f088ea00640 1 -- 172.21.15.26:0/1663856433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0880031990 con 0x7f0890160a20 2024-01-15T18:02:09.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.210+0000 7f088ea00640 1 -- 172.21.15.26:0/1663856433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880031d20 con 0x7f0890160a20 2024-01-15T18:02:09.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.211+0000 7f0895a00640 1 -- 172.21.15.26:0/1663856433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160a20 msgr2=0x7f08900cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:09.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.211+0000 7f0895a00640 1 --2- 172.21.15.26:0/1663856433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160a20 0x7f08900cf850 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f089008d550 tx=0x7f088002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.211+0000 7f0895a00640 1 -- 172.21.15.26:0/1663856433 shutdown_connections 2024-01-15T18:02:09.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.211+0000 7f0895a00640 1 --2- 172.21.15.26:0/1663856433 >> v2:172.21.15.124:3300/0 conn(0x7f08900b3b20 0x7f08900cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.211+0000 7f0895a00640 1 --2- 172.21.15.26:0/1663856433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160a20 0x7f08900cf850 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.211+0000 7f0895a00640 1 -- 172.21.15.26:0/1663856433 >> 172.21.15.26:0/1663856433 conn(0x7f08900d3120 msgr2=0x7f08900d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:09.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.211+0000 7f0895a00640 1 -- 172.21.15.26:0/1663856433 shutdown_connections 2024-01-15T18:02:09.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.211+0000 7f0895a00640 1 -- 172.21.15.26:0/1663856433 wait complete. 2024-01-15T18:02:09.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.213+0000 7f0895a00640 1 Processor -- start 2024-01-15T18:02:09.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.214+0000 7f0895a00640 1 -- start start 2024-01-15T18:02:09.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f0895a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08900b3b20 0x7f089009e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:09.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f0895a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0890160a20 0x7f0890152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:09.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f0895a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160e00 0x7f0890152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:09.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f0895a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089015cc40 con 0x7f0890160e00 2024-01-15T18:02:09.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f0895a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08901520d0 con 0x7f0890160a20 2024-01-15T18:02:09.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f0895a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089009ea00 con 0x7f08900b3b20 2024-01-15T18:02:09.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f0895000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160e00 0x7f0890152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:09.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f088f400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0890160a20 0x7f0890152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f088f400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0890160a20 0x7f0890152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:53402/0 (socket says 172.21.15.26:53402) 2024-01-15T18:02:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.215+0000 7f088f400640 1 -- 172.21.15.26:0/1785815564 learned_addr learned my addr 172.21.15.26:0/1785815564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.216+0000 7f088fe00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08900b3b20 0x7f089009e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.216+0000 7f0895000640 1 -- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08900b3b20 msgr2=0x7f089009e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.216+0000 7f0895000640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08900b3b20 0x7f089009e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.216+0000 7f0895000640 1 -- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0890160a20 msgr2=0x7f0890152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.216+0000 7f0895000640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0890160a20 0x7f0890152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.216+0000 7f0895000640 1 -- 172.21.15.26:0/1785815564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0880009400 con 0x7f0890160e00 2024-01-15T18:02:09.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.216+0000 7f088f400640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0890160a20 0x7f0890152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:02:09.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.217+0000 7f0895000640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160e00 0x7f0890152aa0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0888007cf0 tx=0x7f088800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:09.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.217+0000 7f0887e00640 1 -- 172.21.15.26:0/1785815564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888018070 con 0x7f0890160e00 2024-01-15T18:02:09.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.217+0000 7f0887e00640 1 -- 172.21.15.26:0/1785815564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088800edd0 con 0x7f0890160e00 2024-01-15T18:02:09.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.217+0000 7f0887e00640 1 -- 172.21.15.26:0/1785815564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888003bb0 con 0x7f0890160e00 2024-01-15T18:02:09.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.218+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08901514c0 con 0x7f0890160e00 2024-01-15T18:02:09.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.218+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0890151a80 con 0x7f0890160e00 2024-01-15T18:02:09.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.219+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0890067b60 con 0x7f0890160e00 2024-01-15T18:02:09.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.223+0000 7f0887e00640 1 -- 172.21.15.26:0/1785815564 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 90202+0+0 (secure 0 0 0) 0x7f0888014420 con 0x7f0890160e00 2024-01-15T18:02:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.223+0000 7f0887e00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f086806ba10 0x7f086806ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.224+0000 7f0887e00640 1 -- 172.21.15.26:0/1785815564 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0888089500 con 0x7f0890160e00 2024-01-15T18:02:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.224+0000 7f088fe00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f086806ba10 0x7f086806ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:09.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.225+0000 7f088fe00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f086806ba10 0x7f086806ded0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0880036000 tx=0x7f088003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:09.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.228+0000 7f0887e00640 1 -- 172.21.15.26:0/1785815564 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08880891f0 con 0x7f0890160e00 2024-01-15T18:02:09.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.638+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20837", "target": ["mon-mgr", ""]}) v1 -- 0x7f089015efa0 con 0x7f086806ba10 2024-01-15T18:02:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.683+0000 7f0887e00640 1 -- 172.21.15.26:0/1785815564 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f089015efa0 con 0x7f086806ba10 2024-01-15T18:02:09.687 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:09.687 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:09.687 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:02:09.687 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:02:09.687 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:02:09.687 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:02:09.687 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:02:09.687 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:09.688 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:09.688 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f086806ba10 msgr2=0x7f086806ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f086806ba10 0x7f086806ded0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0880036000 tx=0x7f088003c040 comp rx=0 tx=0).stop 2024-01-15T18:02:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160e00 msgr2=0x7f0890152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160e00 0x7f0890152aa0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0888007cf0 tx=0x7f088800e4b0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 shutdown_connections 2024-01-15T18:02:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f086806ba10 0x7f086806ded0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0890160e00 0x7f0890152aa0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0890160a20 0x7f0890152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 --2- 172.21.15.26:0/1785815564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08900b3b20 0x7f089009e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 >> 172.21.15.26:0/1785815564 conn(0x7f08900d3120 msgr2=0x7f089015d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 shutdown_connections 2024-01-15T18:02:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:09.686+0000 7f0895a00640 1 -- 172.21.15.26:0/1785815564 wait complete. 2024-01-15T18:02:09.718 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.019+0000 7f7b7da00640 1 Processor -- start 2024-01-15T18:02:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.019+0000 7f7b7da00640 1 -- start start 2024-01-15T18:02:10.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.022+0000 7f7b7da00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7b78160f80 0x7f7b78161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.023+0000 7f7b7da00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780ce8f0 0x7f7b780ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:10.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.024+0000 7f7b7da00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b780cfbe0 0x7f7b780d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.024+0000 7f7b7da00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b781618a0 con 0x7f7b780cfbe0 2024-01-15T18:02:10.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.024+0000 7f7b7da00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780d0de0 con 0x7f7b78160f80 2024-01-15T18:02:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.025+0000 7f7b7da00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780d10c0 con 0x7f7b780ce8f0 2024-01-15T18:02:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.025+0000 7f7b76a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7b78160f80 0x7f7b78161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.026+0000 7f7b76000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780ce8f0 0x7f7b780ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.026+0000 7f7b77400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b780cfbe0 0x7f7b780d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.026+0000 7f7b77400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b780cfbe0 0x7f7b780d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47286/0 (socket says 172.21.15.26:47286) 2024-01-15T18:02:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.026+0000 7f7b77400640 1 -- 172.21.15.26:0/1186296381 learned_addr learned my addr 172.21.15.26:0/1186296381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.027+0000 7f7b77400640 1 -- 172.21.15.26:0/1186296381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780ce8f0 msgr2=0x7f7b780ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.027+0000 7f7b77400640 1 --2- 172.21.15.26:0/1186296381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780ce8f0 0x7f7b780ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.027+0000 7f7b77400640 1 -- 172.21.15.26:0/1186296381 >> v2:172.21.15.124:3300/0 conn(0x7f7b78160f80 msgr2=0x7f7b78161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.027+0000 7f7b77400640 1 --2- 172.21.15.26:0/1186296381 >> v2:172.21.15.124:3300/0 conn(0x7f7b78160f80 0x7f7b78161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.027+0000 7f7b77400640 1 -- 172.21.15.26:0/1186296381 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b780d13a0 con 0x7f7b780cfbe0 2024-01-15T18:02:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.029+0000 7f7b77400640 1 --2- 172.21.15.26:0/1186296381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b780cfbe0 0x7f7b780d0040 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7b7808d490 tx=0x7f7b7002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74f568c763ca984 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:10.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.031+0000 7f7b75600640 1 -- 172.21.15.26:0/1186296381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7002ca80 con 0x7f7b780cfbe0 2024-01-15T18:02:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.031+0000 7f7b75600640 1 -- 172.21.15.26:0/1186296381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7002cbe0 con 0x7f7b780cfbe0 2024-01-15T18:02:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.031+0000 7f7b75600640 1 -- 172.21.15.26:0/1186296381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b700343f0 con 0x7f7b780cfbe0 2024-01-15T18:02:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.032+0000 7f7b7da00640 1 -- 172.21.15.26:0/1186296381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b780cfbe0 msgr2=0x7f7b780d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.032+0000 7f7b7da00640 1 --2- 172.21.15.26:0/1186296381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b780cfbe0 0x7f7b780d0040 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7b7808d490 tx=0x7f7b7002bee0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.033+0000 7f7b7da00640 1 -- 172.21.15.26:0/1186296381 shutdown_connections 2024-01-15T18:02:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.033+0000 7f7b7da00640 1 --2- 172.21.15.26:0/1186296381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b780cfbe0 0x7f7b780d0040 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.033+0000 7f7b7da00640 1 --2- 172.21.15.26:0/1186296381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780ce8f0 0x7f7b780ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.033+0000 7f7b7da00640 1 --2- 172.21.15.26:0/1186296381 >> v2:172.21.15.124:3300/0 conn(0x7f7b78160f80 0x7f7b78161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.033+0000 7f7b7da00640 1 -- 172.21.15.26:0/1186296381 >> 172.21.15.26:0/1186296381 conn(0x7f7b7809c550 msgr2=0x7f7b780d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.033+0000 7f7b7da00640 1 -- 172.21.15.26:0/1186296381 shutdown_connections 2024-01-15T18:02:10.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.033+0000 7f7b7da00640 1 -- 172.21.15.26:0/1186296381 wait complete. 2024-01-15T18:02:10.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.038+0000 7f7b7da00640 1 Processor -- start 2024-01-15T18:02:10.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.038+0000 7f7b7da00640 1 -- start start 2024-01-15T18:02:10.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.041+0000 7f7b7da00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b780ce8f0 0x7f7b780b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.042+0000 7f7b7da00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780cfbe0 0x7f7b7815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:10.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.042+0000 7f7b7da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b78160f80 0x7f7b7815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:10.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.047+0000 7f7b7da00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780b8b30 con 0x7f7b78160f80 2024-01-15T18:02:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.047+0000 7f7b7da00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b781508b0 con 0x7f7b780ce8f0 2024-01-15T18:02:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.047+0000 7f7b7da00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7815fd00 con 0x7f7b780cfbe0 2024-01-15T18:02:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.045+0000 7f7b76a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b780ce8f0 0x7f7b780b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.047+0000 7f7b76a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b780ce8f0 0x7f7b780b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:53406/0 (socket says 172.21.15.26:53406) 2024-01-15T18:02:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.047+0000 7f7b76a00640 1 -- 172.21.15.26:0/3544797499 learned_addr learned my addr 172.21.15.26:0/3544797499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:10.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.047+0000 7f7b76000640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780cfbe0 0x7f7b7815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.049+0000 7f7b76a00640 1 -- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780cfbe0 msgr2=0x7f7b7815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.049+0000 7f7b76a00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780cfbe0 0x7f7b7815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.049+0000 7f7b76a00640 1 -- 172.21.15.26:0/3544797499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b78160f80 msgr2=0x7f7b7815f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:10.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.049+0000 7f7b77400640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b78160f80 0x7f7b7815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.050+0000 7f7b76a00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b78160f80 0x7f7b7815f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.050+0000 7f7b76a00640 1 -- 172.21.15.26:0/3544797499 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70009b30 con 0x7f7b780ce8f0 2024-01-15T18:02:10.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.050+0000 7f7b77400640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b78160f80 0x7f7b7815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:10.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.054+0000 7f7b76a00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b780ce8f0 0x7f7b780b5e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000d800 tx=0x7f7b6000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.055+0000 7f7b6ea00640 1 -- 172.21.15.26:0/3544797499 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000b6c0 con 0x7f7b780ce8f0 2024-01-15T18:02:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.056+0000 7f7b6ea00640 1 -- 172.21.15.26:0/3544797499 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6000b820 con 0x7f7b780ce8f0 2024-01-15T18:02:10.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.056+0000 7f7b6ea00640 1 -- 172.21.15.26:0/3544797499 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6000bbb0 con 0x7f7b780ce8f0 2024-01-15T18:02:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.057+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b78160510 con 0x7f7b780ce8f0 2024-01-15T18:02:10.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.057+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b78160230 con 0x7f7b780ce8f0 2024-01-15T18:02:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.060+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b780b8da0 con 0x7f7b780ce8f0 2024-01-15T18:02:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.062+0000 7f7b6ea00640 1 -- 172.21.15.26:0/3544797499 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 17) v1 ==== 90202+0+0 (secure 0 0 0) 0x7f7b6000bd10 con 0x7f7b780ce8f0 2024-01-15T18:02:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.064+0000 7f7b6ea00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b5006bae0 0x7f7b5006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.065+0000 7f7b6ea00640 1 -- 172.21.15.26:0/3544797499 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7b6008b1b0 con 0x7f7b780ce8f0 2024-01-15T18:02:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.066+0000 7f7b76000640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b5006bae0 0x7f7b5006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.068+0000 7f7b76000640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b5006bae0 0x7f7b5006dfa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7b7815d0d0 tx=0x7f7b68008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.072+0000 7f7b6ea00640 1 -- 172.21.15.26:0/3544797499 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b60088760 con 0x7f7b780ce8f0 2024-01-15T18:02:10.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.516+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7b780675d0 con 0x7f7b780ce8f0 2024-01-15T18:02:10.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.518+0000 7f7b6ea00640 1 -- 172.21.15.26:0/3544797499 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2338 (secure 0 0 0) 0x7f7b60038940 con 0x7f7b780ce8f0 2024-01-15T18:02:10.523 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:10.523 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:10.523 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:10.523 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:10.523 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:10.524 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:10.524 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:10.524 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:10.524 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:10.524 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:10.524 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:10.524 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:10.524 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:10.524 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 444, 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:10.525 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:10.526 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:10.526 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:10.526 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:10.526 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:10.526 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:10.526 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:10.526 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:10.526 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:10.526 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:10.527 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:10.527 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:10.527 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:10.527 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:10.527 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:10.527 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:10.527 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:10.527 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:10.527 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:10.528 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:10.528 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 804, 2024-01-15T18:02:10.528 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3222300906, 2024-01-15T18:02:10.528 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 6748307456, 2024-01-15T18:02:10.528 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 766345805824, 2024-01-15T18:02:10.528 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:10.528 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 938, 2024-01-15T18:02:10.528 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 25163, 2024-01-15T18:02:10.529 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 0, 2024-01-15T18:02:10.529 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 7 2024-01-15T18:02:10.529 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:10.529 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:10.529 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:10.529 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:10.529 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:10.529 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:10.529 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:10.530 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:10.530 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:10.530 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:10.530 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:10.530 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:10.530 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:10.530 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:10.530 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:10.530 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:10.531 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:10.532 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "2 ongoing cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0, 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "5 ongoing+pending cloning jobs; Avg progress = 0.0% (0s)\n [............................] ", 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0, 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:10.533 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.523+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b5006bae0 msgr2=0x7f7b5006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.523+0000 7f7b7da00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b5006bae0 0x7f7b5006dfa0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7b7815d0d0 tx=0x7f7b68008040 comp rx=0 tx=0).stop 2024-01-15T18:02:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b780ce8f0 msgr2=0x7f7b780b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b780ce8f0 0x7f7b780b5e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b6000d800 tx=0x7f7b6000dcc0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 shutdown_connections 2024-01-15T18:02:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b780cfbe0 0x7f7b7815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b5006bae0 0x7f7b5006dfa0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b78160f80 0x7f7b7815f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 --2- 172.21.15.26:0/3544797499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b780ce8f0 0x7f7b780b5e30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 >> 172.21.15.26:0/3544797499 conn(0x7f7b7809c550 msgr2=0x7f7b780b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 shutdown_connections 2024-01-15T18:02:10.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:10.524+0000 7f7b7da00640 1 -- 172.21.15.26:0/3544797499 wait complete. 2024-01-15T18:02:11.560 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:11.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:11.740+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:11.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:11.740+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:11.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:11.740+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:11.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:11.740+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:11.746 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:11.740+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.847+0000 7f06ede00640 1 Processor -- start 2024-01-15T18:02:11.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.848+0000 7f06ede00640 1 -- start start 2024-01-15T18:02:11.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.850+0000 7f06ede00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06e8160f80 0x7f06e8161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:11.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.851+0000 7f06ede00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e80cf1b0 con 0x7f06e80ced50 2024-01-15T18:02:11.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.851+0000 7f06ede00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e81618a0 con 0x7f06e80ce8f0 2024-01-15T18:02:11.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.851+0000 7f06ede00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80d0a00 con 0x7f06e8160f80 2024-01-15T18:02:11.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.851+0000 7f06e7400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f06e80ced50 0x7f06e80d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58998/0 (socket says 172.21.15.26:58998) 2024-01-15T18:02:11.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.851+0000 7f06e7400640 1 -- 172.21.15.26:0/2850943577 learned_addr learned my addr 172.21.15.26:0/2850943577 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:11.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.852+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175997119 0 0) 0x7f06e80cf1b0 con 0x7f06e80ced50 2024-01-15T18:02:11.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.852+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06c8003520 con 0x7f06e80ced50 2024-01-15T18:02:11.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041616588 0 0) 0x7f06c8003520 con 0x7f06e80ced50 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06e8160f80 msgr2=0x7f06e8161360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06e8160f80 0x7f06e8161360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 >> v1:172.21.15.124:6789/0 conn(0x7f06e80ce8f0 legacy=0x7f06e80cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80d0de0 con 0x7f06e80ced50 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f06e0001670 con 0x7f06e80ced50 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 >> v1:172.21.15.26:6789/0 conn(0x7f06e80ced50 legacy=0x7f06e80d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06c80039a0 0x7f06c8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06c80047c0 0x7f06c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c80051d0 0x7f06c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:11.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80d0a00 con 0x7f06c80051d0 2024-01-15T18:02:11.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c800c100 con 0x7f06c80039a0 2024-01-15T18:02:11.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.853+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c800c2c0 con 0x7f06c80047c0 2024-01-15T18:02:11.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.854+0000 7f06e7400640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c80051d0 0x7f06c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:11.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.855+0000 7f06e6a00640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06c80039a0 0x7f06c8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:11.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.855+0000 7f06e7400640 1 -- 172.21.15.26:0/2850943577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06c80047c0 msgr2=0x7f06c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:11.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.855+0000 7f06e7400640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06c80047c0 0x7f06c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:11.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.855+0000 7f06e7400640 1 -- 172.21.15.26:0/2850943577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06c80039a0 msgr2=0x7f06c8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:11.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.855+0000 7f06e7400640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06c80039a0 0x7f06c8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:11.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.855+0000 7f06e7400640 1 -- 172.21.15.26:0/2850943577 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80d0de0 con 0x7f06c80051d0 2024-01-15T18:02:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.857+0000 7f06e7400640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c80051d0 0x7f06c800b980 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f06e000c290 tx=0x7f06c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b393eeb59beca65b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.857+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e002cb40 con 0x7f06c80051d0 2024-01-15T18:02:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.857+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e002cca0 con 0x7f06c80051d0 2024-01-15T18:02:11.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.857+0000 7f06e6000640 1 -- 172.21.15.26:0/2850943577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0005af0 con 0x7f06c80051d0 2024-01-15T18:02:11.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.858+0000 7f06ede00640 1 -- 172.21.15.26:0/2850943577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c80051d0 msgr2=0x7f06c800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.858+0000 7f06ede00640 1 --2- 172.21.15.26:0/2850943577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06c80051d0 0x7f06c800b980 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f06e000c290 tx=0x7f06c80057f0 comp rx=0 tx=0).stop 2024-01-15T18:02:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.858+0000 7f06e7400640 1 -- 172.21.15.26:0/2850943577 reap_dead start 2024-01-15T18:02:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.859+0000 7f06ede00640 1 -- 172.21.15.26:0/2850943577 shutdown_connections 2024-01-15T18:02:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.859+0000 7f06ede00640 1 -- 172.21.15.26:0/2850943577 >> 172.21.15.26:0/2850943577 conn(0x7f06e809c550 msgr2=0x7f06e80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.859+0000 7f06ede00640 1 -- 172.21.15.26:0/2850943577 shutdown_connections 2024-01-15T18:02:11.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.859+0000 7f06ede00640 1 -- 172.21.15.26:0/2850943577 wait complete. 2024-01-15T18:02:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.861+0000 7f06ede00640 1 Processor -- start 2024-01-15T18:02:11.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.862+0000 7f06ede00640 1 -- start start 2024-01-15T18:02:11.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.863+0000 7f06ede00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06c80051d0 0x7f06e81f37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:11.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.863+0000 7f06ede00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80b1d70 0x7f06e81f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:11.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.863+0000 7f06ede00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06e80c6d40 0x7f06e81f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:11.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.863+0000 7f06ede00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e81f4e20 con 0x7f06e80b1d70 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.863+0000 7f06ede00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e81f4210 con 0x7f06c80051d0 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.863+0000 7f06ede00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e81f44f0 con 0x7f06e80c6d40 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.864+0000 7f06e7400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06e80c6d40 0x7f06e81f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.864+0000 7f06dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80b1d70 0x7f06e81f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.864+0000 7f06dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80b1d70 0x7f06e81f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47298/0 (socket says 172.21.15.26:47298) 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.864+0000 7f06dfe00640 1 -- 172.21.15.26:0/30285848 learned_addr learned my addr 172.21.15.26:0/30285848 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.864+0000 7f06e6a00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06c80051d0 0x7f06e81f37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.865+0000 7f06dfe00640 1 -- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06e80c6d40 msgr2=0x7f06e81f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:11.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.865+0000 7f06dfe00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06e80c6d40 0x7f06e81f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.865+0000 7f06dfe00640 1 -- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06c80051d0 msgr2=0x7f06e81f37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.865+0000 7f06dfe00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06c80051d0 0x7f06e81f37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.865+0000 7f06dfe00640 1 -- 172.21.15.26:0/30285848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e81f47d0 con 0x7f06e80b1d70 2024-01-15T18:02:11.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.865+0000 7f06dfe00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80b1d70 0x7f06e81f2590 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f06d000c390 tx=0x7f06d000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.866+0000 7f06df400640 1 -- 172.21.15.26:0/30285848 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d00035f0 con 0x7f06e80b1d70 2024-01-15T18:02:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.866+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e81f4ab0 con 0x7f06e80b1d70 2024-01-15T18:02:11.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.866+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e81f2ad0 con 0x7f06e80b1d70 2024-01-15T18:02:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.867+0000 7f06df400640 1 -- 172.21.15.26:0/30285848 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d0003750 con 0x7f06e80b1d70 2024-01-15T18:02:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.867+0000 7f06df400640 1 -- 172.21.15.26:0/30285848 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d0003b00 con 0x7f06e80b1d70 2024-01-15T18:02:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.868+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b8005250 con 0x7f06e80b1d70 2024-01-15T18:02:11.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.869+0000 7f06df400640 1 -- 172.21.15.26:0/30285848 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 17) v1 ==== 90202+0+0 (secure 0 0 0) 0x7f06d0003c60 con 0x7f06e80b1d70 2024-01-15T18:02:11.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.871+0000 7f06df400640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f06c006ba10 0x7f06c006ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:11.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.871+0000 7f06df400640 1 -- 172.21.15.26:0/30285848 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f06d0087b20 con 0x7f06e80b1d70 2024-01-15T18:02:11.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.879+0000 7f06df400640 1 -- 172.21.15.26:0/30285848 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06d0087830 con 0x7f06e80b1d70 2024-01-15T18:02:11.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.895+0000 7f06e6a00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f06c006ba10 0x7f06c006ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:11.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:11.898+0000 7f06e6a00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f06c006ba10 0x7f06c006ded0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f06e808d490 tx=0x7f06d8009590 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:12.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.162+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f06b80057d0 con 0x7f06e80b1d70 2024-01-15T18:02:12.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.163+0000 7f06df400640 1 -- 172.21.15.26:0/30285848 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2418 (secure 0 0 0) 0x7f06d0017990 con 0x7f06e80b1d70 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:12.167 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:12.168 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:12.168 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:12.168 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:12.168 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:12.168 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:12.168 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 446, 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:12.169 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:12.170 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 849, 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3411044586, 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 6991577088, 2024-01-15T18:02:12.171 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 766102536192, 2024-01-15T18:02:12.172 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:12.172 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 16771466, 2024-01-15T18:02:12.172 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 15735320, 2024-01-15T18:02:12.172 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 4, 2024-01-15T18:02:12.172 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 7 2024-01-15T18:02:12.172 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:12.172 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:12.172 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:12.172 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:12.173 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:12.174 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:12.175 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:12.175 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:12.175 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:12.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:12.175 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:12.175 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:12.175 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:12.175 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:12.175 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "2 ongoing cloning jobs; Avg progress = 33.33% (4s)\n [=========...................] (remaining: 9s)", 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.33329999446868896, 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (4s)\n [==..........................] (remaining: 46s)", 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.095228567719459534, 2024-01-15T18:02:12.176 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:12.177 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:12.177 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:12.177 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:12.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f06c006ba10 msgr2=0x7f06c006ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:12.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f06c006ba10 0x7f06c006ded0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f06e808d490 tx=0x7f06d8009590 comp rx=0 tx=0).stop 2024-01-15T18:02:12.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80b1d70 msgr2=0x7f06e81f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:12.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80b1d70 0x7f06e81f2590 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f06d000c390 tx=0x7f06d000c850 comp rx=0 tx=0).stop 2024-01-15T18:02:12.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 shutdown_connections 2024-01-15T18:02:12.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06e80c6d40 0x7f06e81f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:12.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f06c006ba10 0x7f06c006ded0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:12.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f06e80b1d70 0x7f06e81f2590 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:12.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 --2- 172.21.15.26:0/30285848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06c80051d0 0x7f06e81f37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:12.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.165+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 >> 172.21.15.26:0/30285848 conn(0x7f06e809c550 msgr2=0x7f06e80cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:12.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.166+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 shutdown_connections 2024-01-15T18:02:12.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:12.166+0000 7f06ede00640 1 -- 172.21.15.26:0/30285848 wait complete. 2024-01-15T18:02:14.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:14.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.514+0000 7f70a6600640 1 Processor -- start 2024-01-15T18:02:14.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.514+0000 7f70a6600640 1 -- start start 2024-01-15T18:02:14.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.516+0000 7f70a6600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f70a00b30b0 0x7f70a00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:14.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.516+0000 7f70a6600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a006e380 0x7f70a006d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:14.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.516+0000 7f70a6600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a015d840 con 0x7f70a015d460 2024-01-15T18:02:14.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.516+0000 7f70a6600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a0071490 con 0x7f70a00b30b0 2024-01-15T18:02:14.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.516+0000 7f70a6600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a00b3790 con 0x7f70a006e380 2024-01-15T18:02:14.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.517+0000 7f709f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f70a015d460 0x7f70a00b29a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48790/0 (socket says 172.21.15.26:48790) 2024-01-15T18:02:14.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.517+0000 7f709f400640 1 -- 172.21.15.26:0/276046403 learned_addr learned my addr 172.21.15.26:0/276046403 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:14.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.517+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433857875 0 0) 0x7f70a015d840 con 0x7f70a015d460 2024-01-15T18:02:14.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.517+0000 7f709fe00640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a006e380 0x7f70a006d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:14.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7080003520 con 0x7f70a015d460 2024-01-15T18:02:14.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1638912231 0 0) 0x7f7080003520 con 0x7f70a015d460 2024-01-15T18:02:14.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a006e380 msgr2=0x7f70a006d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:14.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a006e380 0x7f70a006d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 >> v2:172.21.15.124:3300/0 conn(0x7f70a00b30b0 msgr2=0x7f70a00ac8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:02:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 --2- 172.21.15.26:0/276046403 >> v2:172.21.15.124:3300/0 conn(0x7f70a00b30b0 0x7f70a00ac8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a0072190 con 0x7f70a015d460 2024-01-15T18:02:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f70900019f0 con 0x7f70a015d460 2024-01-15T18:02:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 >> v1:172.21.15.26:6789/0 conn(0x7f70a015d460 legacy=0x7f70a00b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7080003970 0x7f7080003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70800047c0 0x7f7080004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7080005180 0x7f708000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:14.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0071490 con 0x7f7080005180 2024-01-15T18:02:14.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708000bff0 con 0x7f7080003970 2024-01-15T18:02:14.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.518+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708000c1b0 con 0x7f70800047c0 2024-01-15T18:02:14.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.519+0000 7f709fe00640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7080005180 0x7f708000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:14.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.520+0000 7f709fe00640 1 -- 172.21.15.26:0/276046403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70800047c0 msgr2=0x7f7080004c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:02:14.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.520+0000 7f709fe00640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70800047c0 0x7f7080004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:14.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.520+0000 7f709fe00640 1 -- 172.21.15.26:0/276046403 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7080003970 msgr2=0x7f7080003d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:02:14.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.520+0000 7f709fe00640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7080003970 0x7f7080003d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:14.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.520+0000 7f709fe00640 1 -- 172.21.15.26:0/276046403 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a0072190 con 0x7f7080005180 2024-01-15T18:02:14.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.522+0000 7f709fe00640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7080005180 0x7f708000b980 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f70a006e760 tx=0x7f70800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1194d2aa3cd27879 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:14.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.523+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098034610 con 0x7f7080005180 2024-01-15T18:02:14.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.523+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709802e070 con 0x7f7080005180 2024-01-15T18:02:14.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.523+0000 7f709ea00640 1 -- 172.21.15.26:0/276046403 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980349b0 con 0x7f7080005180 2024-01-15T18:02:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.525+0000 7f70a6600640 1 -- 172.21.15.26:0/276046403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7080005180 msgr2=0x7f708000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:14.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.525+0000 7f70a6600640 1 --2- 172.21.15.26:0/276046403 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7080005180 0x7f708000b980 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f70a006e760 tx=0x7f70800057f0 comp rx=0 tx=0).stop 2024-01-15T18:02:14.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.525+0000 7f709fe00640 1 -- 172.21.15.26:0/276046403 reap_dead start 2024-01-15T18:02:14.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.528+0000 7f70a6600640 1 -- 172.21.15.26:0/276046403 shutdown_connections 2024-01-15T18:02:14.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.528+0000 7f70a6600640 1 -- 172.21.15.26:0/276046403 >> 172.21.15.26:0/276046403 conn(0x7f70a00b0be0 msgr2=0x7f70a00b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:14.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.529+0000 7f70a6600640 1 -- 172.21.15.26:0/276046403 shutdown_connections 2024-01-15T18:02:14.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.529+0000 7f70a6600640 1 -- 172.21.15.26:0/276046403 wait complete. 2024-01-15T18:02:14.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.532+0000 7f70a6600640 1 Processor -- start 2024-01-15T18:02:14.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.533+0000 7f70a6600640 1 -- start start 2024-01-15T18:02:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.538+0000 7f70a6600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a01476b0 0x7f70a01524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.539+0000 7f70a6600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70a0147a90 0x7f70a0154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.539+0000 7f70a6600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a0160380 0x7f70a0154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.539+0000 7f70a6600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0154a80 con 0x7f70a0160380 2024-01-15T18:02:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.539+0000 7f70a6600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0154cd0 con 0x7f70a0147a90 2024-01-15T18:02:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.539+0000 7f70a6600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0154fb0 con 0x7f70a01476b0 2024-01-15T18:02:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.539+0000 7f7096800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70a0147a90 0x7f70a0154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.540+0000 7f7096800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70a0147a90 0x7f70a0154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58406/0 (socket says 172.21.15.26:58406) 2024-01-15T18:02:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.540+0000 7f7096800640 1 -- 172.21.15.26:0/3749427118 learned_addr learned my addr 172.21.15.26:0/3749427118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.540+0000 7f709f400640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a01476b0 0x7f70a01524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.540+0000 7f709fe00640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a0160380 0x7f70a0154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.541+0000 7f709f400640 1 -- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70a0147a90 msgr2=0x7f70a0154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.541+0000 7f709f400640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70a0147a90 0x7f70a0154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.541+0000 7f709f400640 1 -- 172.21.15.26:0/3749427118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a0160380 msgr2=0x7f70a0154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.541+0000 7f709f400640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a0160380 0x7f70a0154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.541+0000 7f709f400640 1 -- 172.21.15.26:0/3749427118 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a0155290 con 0x7f70a01476b0 2024-01-15T18:02:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.541+0000 7f709fe00640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a0160380 0x7f70a0154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:02:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.543+0000 7f709f400640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a01476b0 0x7f70a01524a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f709000cb40 tx=0x7f709000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.545+0000 7f7097e00640 1 -- 172.21.15.26:0/3749427118 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000aa90 con 0x7f70a01476b0 2024-01-15T18:02:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.545+0000 7f7097e00640 1 -- 172.21.15.26:0/3749427118 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709000abf0 con 0x7f70a01476b0 2024-01-15T18:02:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.545+0000 7f7097e00640 1 -- 172.21.15.26:0/3749427118 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7090004e30 con 0x7f70a01476b0 2024-01-15T18:02:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.545+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a0155570 con 0x7f70a01476b0 2024-01-15T18:02:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.546+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a0152bc0 con 0x7f70a01476b0 2024-01-15T18:02:14.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.548+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a015d9d0 con 0x7f70a01476b0 2024-01-15T18:02:14.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.556+0000 7f7097e00640 1 -- 172.21.15.26:0/3749427118 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 18) v1 ==== 90253+0+0 (secure 0 0 0) 0x7f70900070d0 con 0x7f70a01476b0 2024-01-15T18:02:14.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.557+0000 7f7097e00640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f707806bad0 0x7f707806df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:14.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.558+0000 7f7097e00640 1 -- 172.21.15.26:0/3749427118 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7090087bd0 con 0x7f70a01476b0 2024-01-15T18:02:14.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.572+0000 7f7097e00640 1 -- 172.21.15.26:0/3749427118 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7090054640 con 0x7f70a01476b0 2024-01-15T18:02:14.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.573+0000 7f7096800640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f707806bad0 0x7f707806df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:14.575+0000 7f7096800640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f707806bad0 0x7f707806df90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7088006fd0 tx=0x7f7088008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.036+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f70a015d9d0 con 0x7f70a01476b0 2024-01-15T18:02:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.038+0000 7f7097e00640 1 -- 172.21.15.26:0/3749427118 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2425 (secure 0 0 0) 0x7f70a015d9d0 con 0x7f70a01476b0 2024-01-15T18:02:15.042 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:15.042 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:15.043 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:15.044 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:15.044 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:15.044 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:15.044 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:15.044 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:15.044 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:15.044 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:15.044 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 449, 2024-01-15T18:02:15.044 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:15.045 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:15.046 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1316, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 5369951341, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 10514591744, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 762579521536, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 178890254, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 178918147, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 43, 2024-01-15T18:02:15.047 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 51 2024-01-15T18:02:15.048 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:15.048 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:15.048 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:15.048 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:15.048 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:15.048 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:15.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:15.048 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:15.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:15.049 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:15.050 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "3 ongoing cloning jobs; Avg progress = 22.22% (7s)\n [======......................] (remaining: 24s)", 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.22220000624656677, 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:15.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:15.052 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:15.052 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (7s)\n [==..........................] (remaining: 66s)", 2024-01-15T18:02:15.052 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.095228567719459534, 2024-01-15T18:02:15.052 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:15.052 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:15.052 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:15.052 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:15.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f707806bad0 msgr2=0x7f707806df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:15.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f707806bad0 0x7f707806df90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7088006fd0 tx=0x7f7088008040 comp rx=0 tx=0).stop 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a01476b0 msgr2=0x7f70a01524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a01476b0 0x7f70a01524a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f709000cb40 tx=0x7f709000a060 comp rx=0 tx=0).stop 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 shutdown_connections 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f707806bad0 0x7f707806df90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70a0160380 0x7f70a0154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70a0147a90 0x7f70a0154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 --2- 172.21.15.26:0/3749427118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70a01476b0 0x7f70a01524a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 >> 172.21.15.26:0/3749427118 conn(0x7f70a00b0be0 msgr2=0x7f70a0161580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 shutdown_connections 2024-01-15T18:02:15.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:15.040+0000 7f70a6600640 1 -- 172.21.15.26:0/3749427118 wait complete. 2024-01-15T18:02:16.844 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:16.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:16.845 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:16.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:16.845 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:16.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:16.845 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:16.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:16.845 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:16.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:17.066 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.347+0000 7f1429a00640 1 Processor -- start 2024-01-15T18:02:17.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.347+0000 7f1429a00640 1 -- start start 2024-01-15T18:02:17.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.349+0000 7f1429a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f14240cfcf0 0x7f1424160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.349+0000 7f1429a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d3910 0x7f1424160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.349+0000 7f1429a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14240d4160 con 0x7f14240d00d0 2024-01-15T18:02:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.349+0000 7f1429a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14240ceff0 con 0x7f14240cfcf0 2024-01-15T18:02:17.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.349+0000 7f1429a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14240cf410 con 0x7f14240d3910 2024-01-15T18:02:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.350+0000 7f141be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d3910 0x7f1424160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.350+0000 7f1422a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f14240cfcf0 0x7f1424160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.350+0000 7f1423400640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f14240d00d0 0x7f14240b2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:55918/0 (socket says 172.21.15.26:55918) 2024-01-15T18:02:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.350+0000 7f141be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d3910 0x7f1424160c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:54500/0 (socket says 172.21.15.26:54500) 2024-01-15T18:02:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.350+0000 7f1422a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f14240cfcf0 0x7f1424160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:58408/0 (socket says 172.21.15.26:58408) 2024-01-15T18:02:17.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.350+0000 7f1423400640 1 -- 172.21.15.26:0/3345713855 learned_addr learned my addr 172.21.15.26:0/3345713855 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:17.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.351+0000 7f1422000640 1 -- 172.21.15.26:0/3345713855 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509441224 0 0) 0x7f14240d4160 con 0x7f14240d00d0 2024-01-15T18:02:17.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.351+0000 7f1422000640 1 -- 172.21.15.26:0/3345713855 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1404003520 con 0x7f14240d00d0 2024-01-15T18:02:17.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.351+0000 7f141be00640 1 -- 172.21.15.26:0/3345713855 >> v2:172.21.15.124:3300/0 conn(0x7f14240cfcf0 msgr2=0x7f1424160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.351+0000 7f141be00640 1 --2- 172.21.15.26:0/3345713855 >> v2:172.21.15.124:3300/0 conn(0x7f14240cfcf0 0x7f1424160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.351+0000 7f141be00640 1 -- 172.21.15.26:0/3345713855 >> v1:172.21.15.124:6789/0 conn(0x7f14240d00d0 legacy=0x7f14240b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:17.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.351+0000 7f141be00640 1 -- 172.21.15.26:0/3345713855 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14240b2420 con 0x7f14240d3910 2024-01-15T18:02:17.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.354+0000 7f141be00640 1 --2- 172.21.15.26:0/3345713855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d3910 0x7f1424160c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f140c0097a0 tx=0x7f140c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78a7a011275efd91 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.356+0000 7f1422000640 1 -- 172.21.15.26:0/3345713855 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c02e070 con 0x7f14240d3910 2024-01-15T18:02:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.356+0000 7f1422000640 1 -- 172.21.15.26:0/3345713855 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c0315a0 con 0x7f14240d3910 2024-01-15T18:02:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.356+0000 7f1422000640 1 -- 172.21.15.26:0/3345713855 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c031930 con 0x7f14240d3910 2024-01-15T18:02:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.357+0000 7f1429a00640 1 -- 172.21.15.26:0/3345713855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d3910 msgr2=0x7f1424160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.357+0000 7f1429a00640 1 --2- 172.21.15.26:0/3345713855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d3910 0x7f1424160c40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f140c0097a0 tx=0x7f140c02b840 comp rx=0 tx=0).stop 2024-01-15T18:02:17.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.357+0000 7f1429a00640 1 -- 172.21.15.26:0/3345713855 shutdown_connections 2024-01-15T18:02:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.357+0000 7f1429a00640 1 --2- 172.21.15.26:0/3345713855 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d3910 0x7f1424160c40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.357+0000 7f1429a00640 1 --2- 172.21.15.26:0/3345713855 >> v2:172.21.15.124:3300/0 conn(0x7f14240cfcf0 0x7f1424160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.357+0000 7f1429a00640 1 -- 172.21.15.26:0/3345713855 >> 172.21.15.26:0/3345713855 conn(0x7f142409c530 msgr2=0x7f14240d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.357+0000 7f1429a00640 1 -- 172.21.15.26:0/3345713855 shutdown_connections 2024-01-15T18:02:17.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.357+0000 7f1429a00640 1 -- 172.21.15.26:0/3345713855 wait complete. 2024-01-15T18:02:17.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.359+0000 7f1429a00640 1 Processor -- start 2024-01-15T18:02:17.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.359+0000 7f1429a00640 1 -- start start 2024-01-15T18:02:17.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.360+0000 7f1429a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14240cfcf0 0x7f14240cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:17.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.360+0000 7f1429a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d00d0 0x7f14240cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.360+0000 7f1429a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14240d3910 0x7f14240cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.360+0000 7f1429a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14241f6470 con 0x7f14240cfcf0 2024-01-15T18:02:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.360+0000 7f1429a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14241491f0 con 0x7f14240d3910 2024-01-15T18:02:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.360+0000 7f1429a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1424149400 con 0x7f14240d00d0 2024-01-15T18:02:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f1422a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14240cfcf0 0x7f14240cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f1422a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14240cfcf0 0x7f14240cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34522/0 (socket says 172.21.15.26:34522) 2024-01-15T18:02:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f1422a00640 1 -- 172.21.15.26:0/2389817974 learned_addr learned my addr 172.21.15.26:0/2389817974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f1423400640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14240d3910 0x7f14240cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:17.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f141be00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d00d0 0x7f14240cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f1422a00640 1 -- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d00d0 msgr2=0x7f14240cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f1422a00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d00d0 0x7f14240cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f1422a00640 1 -- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14240d3910 msgr2=0x7f14240cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f1422a00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14240d3910 0x7f14240cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f1422a00640 1 -- 172.21.15.26:0/2389817974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140c009400 con 0x7f14240cfcf0 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.361+0000 7f141be00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d00d0 0x7f14240cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.362+0000 7f1422a00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14240cfcf0 0x7f14240cbf70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f141400e870 tx=0x7f141400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.362+0000 7f141aa00640 1 -- 172.21.15.26:0/2389817974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14140094f0 con 0x7f14240cfcf0 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.362+0000 7f141aa00640 1 -- 172.21.15.26:0/2389817974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141400c940 con 0x7f14240cfcf0 2024-01-15T18:02:17.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.362+0000 7f141aa00640 1 -- 172.21.15.26:0/2389817974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141400ccf0 con 0x7f14240cfcf0 2024-01-15T18:02:17.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.362+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14241496e0 con 0x7f14240cfcf0 2024-01-15T18:02:17.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.362+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1424149ca0 con 0x7f14240cfcf0 2024-01-15T18:02:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.364+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14240675d0 con 0x7f14240cfcf0 2024-01-15T18:02:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.364+0000 7f141aa00640 1 -- 172.21.15.26:0/2389817974 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 18) v1 ==== 90253+0+0 (secure 0 0 0) 0x7f1414013470 con 0x7f14240cfcf0 2024-01-15T18:02:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.365+0000 7f141aa00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13fc06baa0 0x7f13fc06df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.365+0000 7f141aa00640 1 -- 172.21.15.26:0/2389817974 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f14140883b0 con 0x7f14240cfcf0 2024-01-15T18:02:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.366+0000 7f141be00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13fc06baa0 0x7f13fc06df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:17.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.367+0000 7f141be00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13fc06baa0 0x7f13fc06df60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f140c0097a0 tx=0x7f140c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:17.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.371+0000 7f141aa00640 1 -- 172.21.15.26:0/2389817974 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1414054e20 con 0x7f14240cfcf0 2024-01-15T18:02:17.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.820+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f142415eec0 con 0x7f14240cfcf0 2024-01-15T18:02:17.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.822+0000 7f141aa00640 1 -- 172.21.15.26:0/2389817974 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2425 (secure 0 0 0) 0x7f1414021020 con 0x7f14240cfcf0 2024-01-15T18:02:17.827 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:17.827 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:17.827 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:17.827 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:17.827 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:17.827 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:17.827 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:17.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:17.828 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 452, 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:17.829 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:17.830 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1316, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 5369951341, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 10514591744, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 762579521536, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 178874542, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 178897657, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 43, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 49 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:17.831 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:17.832 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:17.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "3 ongoing cloning jobs; Avg progress = 22.22% (7s)\n [======......................] (remaining: 24s)", 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.22220000624656677, 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:17.834 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (7s)\n [==..........................] (remaining: 66s)", 2024-01-15T18:02:17.835 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.095228567719459534, 2024-01-15T18:02:17.835 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:17.835 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:17.835 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:17.835 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13fc06baa0 msgr2=0x7f13fc06df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13fc06baa0 0x7f13fc06df60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f140c0097a0 tx=0x7f140c03d040 comp rx=0 tx=0).stop 2024-01-15T18:02:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14240cfcf0 msgr2=0x7f14240cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14240cfcf0 0x7f14240cbf70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f141400e870 tx=0x7f141400ed30 comp rx=0 tx=0).stop 2024-01-15T18:02:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 shutdown_connections 2024-01-15T18:02:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f14240d00d0 0x7f14240cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f13fc06baa0 0x7f13fc06df60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:17.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f14240d3910 0x7f14240cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:17.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 --2- 172.21.15.26:0/2389817974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f14240cfcf0 0x7f14240cbf70 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:17.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 >> 172.21.15.26:0/2389817974 conn(0x7f142409c530 msgr2=0x7f142415f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:17.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 shutdown_connections 2024-01-15T18:02:17.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:17.825+0000 7f1429a00640 1 -- 172.21.15.26:0/2389817974 wait complete. 2024-01-15T18:02:19.854 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:20.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.186+0000 7f7833600640 1 Processor -- start 2024-01-15T18:02:20.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.187+0000 7f7833600640 1 -- start start 2024-01-15T18:02:20.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.189+0000 7f7833600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:20.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.189+0000 7f7833600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c0cf1c0 con 0x7f782c0ce8f0 2024-01-15T18:02:20.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.189+0000 7f7833600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c0d0de0 con 0x7f782c160f80 2024-01-15T18:02:20.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.189+0000 7f7833600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0d10c0 con 0x7f782c0ced30 2024-01-15T18:02:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.193+0000 7f7831600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.193+0000 7f7831600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:54512/0 (socket says 172.21.15.26:54512) 2024-01-15T18:02:20.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.193+0000 7f7831600640 1 -- 172.21.15.26:0/671217969 learned_addr learned my addr 172.21.15.26:0/671217969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:20.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.195+0000 7f782b400640 1 -- 172.21.15.26:0/671217969 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555051724 0 0) 0x7f7814001110 con 0x7f782c0ce8f0 2024-01-15T18:02:20.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.195+0000 7f782b400640 1 -- 172.21.15.26:0/671217969 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f780c003520 con 0x7f782c0ce8f0 2024-01-15T18:02:20.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.195+0000 7f782b400640 1 -- 172.21.15.26:0/671217969 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941381755 0 0) 0x7f782c0d0de0 con 0x7f782c160f80 2024-01-15T18:02:20.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.195+0000 7f7831600640 1 -- 172.21.15.26:0/671217969 >> v1:172.21.15.124:6789/0 conn(0x7f782c160f80 legacy=0x7f782c161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.195+0000 7f7831600640 1 -- 172.21.15.26:0/671217969 >> v1:172.21.15.26:6789/0 conn(0x7f782c0ce8f0 legacy=0x7f782c0b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:20.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.195+0000 7f7831600640 1 -- 172.21.15.26:0/671217969 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c0d13a0 con 0x7f782c0ced30 2024-01-15T18:02:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.200+0000 7f7831600640 1 --2- 172.21.15.26:0/671217969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0b3110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7824009e80 tx=0x7f782402bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a756a2a6b448f077 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.201+0000 7f782b400640 1 -- 172.21.15.26:0/671217969 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782402cb70 con 0x7f782c0ced30 2024-01-15T18:02:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.201+0000 7f782b400640 1 -- 172.21.15.26:0/671217969 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782402ccd0 con 0x7f782c0ced30 2024-01-15T18:02:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.201+0000 7f782b400640 1 -- 172.21.15.26:0/671217969 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7824034610 con 0x7f782c0ced30 2024-01-15T18:02:20.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.202+0000 7f7833600640 1 -- 172.21.15.26:0/671217969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 msgr2=0x7f782c0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:20.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.202+0000 7f7833600640 1 --2- 172.21.15.26:0/671217969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0b3110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7824009e80 tx=0x7f782402bfd0 comp rx=0 tx=0).stop 2024-01-15T18:02:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.204+0000 7f7833600640 1 -- 172.21.15.26:0/671217969 shutdown_connections 2024-01-15T18:02:20.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.204+0000 7f7833600640 1 --2- 172.21.15.26:0/671217969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0b3110 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:20.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.204+0000 7f7833600640 1 -- 172.21.15.26:0/671217969 >> 172.21.15.26:0/671217969 conn(0x7f782c09c550 msgr2=0x7f782c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:20.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.206+0000 7f7833600640 1 -- 172.21.15.26:0/671217969 shutdown_connections 2024-01-15T18:02:20.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.206+0000 7f7833600640 1 -- 172.21.15.26:0/671217969 wait complete. 2024-01-15T18:02:20.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.212+0000 7f7833600640 1 Processor -- start 2024-01-15T18:02:20.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.213+0000 7f7833600640 1 -- start start 2024-01-15T18:02:20.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.215+0000 7f7833600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f782c0ce8f0 0x7f782c15aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:20.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.215+0000 7f7833600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:20.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.216+0000 7f7833600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f782c160f80 0x7f782c0b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:20.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.216+0000 7f782be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.216+0000 7f7830c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f782c0ce8f0 0x7f782c15aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.216+0000 7f782be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:54522/0 (socket says 172.21.15.26:54522) 2024-01-15T18:02:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.216+0000 7f782be00640 1 -- 172.21.15.26:0/2615914644 learned_addr learned my addr 172.21.15.26:0/2615914644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.216+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0c96d0 con 0x7f782c0ce8f0 2024-01-15T18:02:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.217+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c15a820 con 0x7f782c160f80 2024-01-15T18:02:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.217+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0c8e30 con 0x7f782c0ced30 2024-01-15T18:02:20.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.217+0000 7f7831600640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f782c160f80 0x7f782c0b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:20.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.218+0000 7f7830c00640 1 -- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 msgr2=0x7f782c0c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:20.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.218+0000 7f7830c00640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.218+0000 7f7830c00640 1 -- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f782c160f80 msgr2=0x7f782c0b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.218+0000 7f7830c00640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f782c160f80 0x7f782c0b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.218+0000 7f7830c00640 1 -- 172.21.15.26:0/2615914644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7824009b30 con 0x7f782c0ce8f0 2024-01-15T18:02:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.219+0000 7f7830c00640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f782c0ce8f0 0x7f782c15aa50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f781c00c3e0 tx=0x7f781c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:20.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.220+0000 7f7828c00640 1 -- 172.21.15.26:0/2615914644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0035f0 con 0x7f782c0ce8f0 2024-01-15T18:02:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.220+0000 7f7828c00640 1 -- 172.21.15.26:0/2615914644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c003770 con 0x7f782c0ce8f0 2024-01-15T18:02:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.220+0000 7f7828c00640 1 -- 172.21.15.26:0/2615914644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c003b60 con 0x7f782c0ce8f0 2024-01-15T18:02:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.220+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c0c9110 con 0x7f782c0ce8f0 2024-01-15T18:02:20.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.220+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c15e260 con 0x7f782c0ce8f0 2024-01-15T18:02:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.225+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77fc005250 con 0x7f782c0ce8f0 2024-01-15T18:02:20.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.226+0000 7f7828c00640 1 -- 172.21.15.26:0/2615914644 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f781c003cc0 con 0x7f782c0ce8f0 2024-01-15T18:02:20.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.227+0000 7f7828c00640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f780406bb80 0x7f780406e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:20.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.228+0000 7f7828c00640 1 -- 172.21.15.26:0/2615914644 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f781c087b50 con 0x7f782c0ce8f0 2024-01-15T18:02:20.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.239+0000 7f7828c00640 1 -- 172.21.15.26:0/2615914644 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f781c0542f0 con 0x7f782c0ce8f0 2024-01-15T18:02:20.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.246+0000 7f782be00640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f780406bb80 0x7f780406e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:20.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.311+0000 7f782be00640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f780406bb80 0x7f780406e040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f782c0a1190 tx=0x7f7814009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.745+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f77fc0057d0 con 0x7f782c0ce8f0 2024-01-15T18:02:20.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.746+0000 7f7828c00640 1 -- 172.21.15.26:0/2615914644 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2429 (secure 0 0 0) 0x7f781c017990 con 0x7f782c0ce8f0 2024-01-15T18:02:20.751 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:20.751 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:20.751 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:20.751 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:20.751 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:20.751 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:20.751 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:20.751 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:20.752 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 454, 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:20.753 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:20.754 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1877, 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 7723083287, 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 14980587520, 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 758113525760, 2024-01-15T18:02:20.755 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:20.756 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 378730420, 2024-01-15T18:02:20.756 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 374922114, 2024-01-15T18:02:20.756 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 91, 2024-01-15T18:02:20.756 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 100 2024-01-15T18:02:20.756 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:20.756 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:20.756 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:20.756 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:20.756 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:20.757 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:20.758 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:20.759 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 33.335% (10s)\n [=========...................] (remaining: 20s)", 2024-01-15T18:02:20.760 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.33335000276565552, 2024-01-15T18:02:20.760 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:20.760 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:20.760 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:20.760 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 19.049% (10s)\n [=====.......................] (remaining: 43s)", 2024-01-15T18:02:20.760 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.19048571586608887, 2024-01-15T18:02:20.760 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:20.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:20.760 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:20.761 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f780406bb80 msgr2=0x7f780406e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f780406bb80 0x7f780406e040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f782c0a1190 tx=0x7f7814009040 comp rx=0 tx=0).stop 2024-01-15T18:02:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f782c0ce8f0 msgr2=0x7f782c15aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f782c0ce8f0 0x7f782c15aa50 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f781c00c3e0 tx=0x7f781c00c8a0 comp rx=0 tx=0).stop 2024-01-15T18:02:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 shutdown_connections 2024-01-15T18:02:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f782c0ced30 0x7f782c0c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f780406bb80 0x7f780406e040 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:20.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f782c160f80 0x7f782c0b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 --2- 172.21.15.26:0/2615914644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f782c0ce8f0 0x7f782c15aa50 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 >> 172.21.15.26:0/2615914644 conn(0x7f782c09c550 msgr2=0x7f782c0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 shutdown_connections 2024-01-15T18:02:20.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:20.749+0000 7f7833600640 1 -- 172.21.15.26:0/2615914644 wait complete. 2024-01-15T18:02:22.773 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:23.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.093+0000 7f75b4400640 1 Processor -- start 2024-01-15T18:02:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.097+0000 7f75b4400640 1 -- start start 2024-01-15T18:02:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.100+0000 7f75b4400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac0d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.100+0000 7f75b4400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f75ac15efc0 0x7f75ac15f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.101+0000 7f75b4400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ac0d0f90 con 0x7f75ac158db0 2024-01-15T18:02:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.101+0000 7f75b4400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ac0d0630 con 0x7f75ac15efc0 2024-01-15T18:02:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.101+0000 7f75b4400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac15fd30 con 0x7f75ac06ee90 2024-01-15T18:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.102+0000 7f75b1a00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f75ac158db0 0x7f75ac06e780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:58396/0 (socket says 172.21.15.26:58396) 2024-01-15T18:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.102+0000 7f75b1a00640 1 -- 172.21.15.26:0/3015757794 learned_addr learned my addr 172.21.15.26:0/3015757794 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.102+0000 7f75b2400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f75ac15efc0 0x7f75ac15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.103+0000 7f75b1000640 1 --2- 172.21.15.26:0/3015757794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac0d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.106+0000 7f75b1000640 1 -- 172.21.15.26:0/3015757794 >> v2:172.21.15.124:3300/0 conn(0x7f75ac15efc0 msgr2=0x7f75ac15f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.106+0000 7f75b1000640 1 --2- 172.21.15.26:0/3015757794 >> v2:172.21.15.124:3300/0 conn(0x7f75ac15efc0 0x7f75ac15f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.106+0000 7f75b1000640 1 -- 172.21.15.26:0/3015757794 >> v1:172.21.15.124:6789/0 conn(0x7f75ac158db0 legacy=0x7f75ac06e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.106+0000 7f75b1000640 1 -- 172.21.15.26:0/3015757794 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac0cfc90 con 0x7f75ac06ee90 2024-01-15T18:02:23.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.107+0000 7f75b2400640 1 --2- 172.21.15.26:0/3015757794 >> v2:172.21.15.124:3300/0 conn(0x7f75ac15efc0 0x7f75ac15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:02:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.108+0000 7f75b1000640 1 --2- 172.21.15.26:0/3015757794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac0d0a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75940097a0 tx=0x7f759402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21527c9dd79d4381 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.109+0000 7f75abe00640 1 -- 172.21.15.26:0/3015757794 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759402e070 con 0x7f75ac06ee90 2024-01-15T18:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.109+0000 7f75abe00640 1 -- 172.21.15.26:0/3015757794 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75940315a0 con 0x7f75ac06ee90 2024-01-15T18:02:23.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.109+0000 7f75abe00640 1 -- 172.21.15.26:0/3015757794 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594031970 con 0x7f75ac06ee90 2024-01-15T18:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.110+0000 7f75b4400640 1 -- 172.21.15.26:0/3015757794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 msgr2=0x7f75ac0d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.110+0000 7f75b4400640 1 --2- 172.21.15.26:0/3015757794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac0d0a50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f75940097a0 tx=0x7f759402b840 comp rx=0 tx=0).stop 2024-01-15T18:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.111+0000 7f75b4400640 1 -- 172.21.15.26:0/3015757794 shutdown_connections 2024-01-15T18:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.111+0000 7f75b4400640 1 --2- 172.21.15.26:0/3015757794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac0d0a50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.111+0000 7f75b4400640 1 --2- 172.21.15.26:0/3015757794 >> v2:172.21.15.124:3300/0 conn(0x7f75ac15efc0 0x7f75ac15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.111+0000 7f75b4400640 1 -- 172.21.15.26:0/3015757794 >> 172.21.15.26:0/3015757794 conn(0x7f75ac0b6500 msgr2=0x7f75ac0b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.112+0000 7f75b4400640 1 -- 172.21.15.26:0/3015757794 shutdown_connections 2024-01-15T18:02:23.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.112+0000 7f75b4400640 1 -- 172.21.15.26:0/3015757794 wait complete. 2024-01-15T18:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.116+0000 7f75b4400640 1 Processor -- start 2024-01-15T18:02:23.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.117+0000 7f75b4400640 1 -- start start 2024-01-15T18:02:23.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.119+0000 7f75b4400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:23.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.120+0000 7f75b4400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ac158db0 0x7f75ac152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:23.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.120+0000 7f75b4400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ac15efc0 0x7f75ac14e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.121+0000 7f75b4400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac153be0 con 0x7f75ac15efc0 2024-01-15T18:02:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.121+0000 7f75b4400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac153ec0 con 0x7f75ac158db0 2024-01-15T18:02:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.121+0000 7f75b4400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac1541a0 con 0x7f75ac06ee90 2024-01-15T18:02:23.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.121+0000 7f75b1000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ac158db0 0x7f75ac152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.121+0000 7f75b1000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ac158db0 0x7f75ac152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:38038/0 (socket says 172.21.15.26:38038) 2024-01-15T18:02:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.122+0000 7f75b2400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ac15efc0 0x7f75ac14e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.122+0000 7f75b1a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.122+0000 7f75b1a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:56742/0 (socket says 172.21.15.26:56742) 2024-01-15T18:02:23.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.122+0000 7f75b1000640 1 -- 172.21.15.26:0/2809585915 learned_addr learned my addr 172.21.15.26:0/2809585915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:23.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.124+0000 7f75b1000640 1 -- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 msgr2=0x7f75ac150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:23.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.124+0000 7f75b1000640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.124+0000 7f75b1000640 1 -- 172.21.15.26:0/2809585915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ac15efc0 msgr2=0x7f75ac14e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.124+0000 7f75b1000640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ac15efc0 0x7f75ac14e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.124+0000 7f75b1000640 1 -- 172.21.15.26:0/2809585915 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7594009400 con 0x7f75ac158db0 2024-01-15T18:02:23.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.130+0000 7f75b2400640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ac15efc0 0x7f75ac14e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:02:23.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.132+0000 7f75b1000640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ac158db0 0x7f75ac152560 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7594034040 tx=0x7f75940046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:23.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.137+0000 7f75a9600640 1 -- 172.21.15.26:0/2809585915 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594032840 con 0x7f75ac158db0 2024-01-15T18:02:23.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.137+0000 7f75a9600640 1 -- 172.21.15.26:0/2809585915 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75940329a0 con 0x7f75ac158db0 2024-01-15T18:02:23.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.137+0000 7f75a9600640 1 -- 172.21.15.26:0/2809585915 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594032d50 con 0x7f75ac158db0 2024-01-15T18:02:23.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.138+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ac154480 con 0x7f75ac158db0 2024-01-15T18:02:23.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.138+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ac152aa0 con 0x7f75ac158db0 2024-01-15T18:02:23.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.142+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c005250 con 0x7f75ac158db0 2024-01-15T18:02:23.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.146+0000 7f75a9600640 1 -- 172.21.15.26:0/2809585915 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f759404e440 con 0x7f75ac158db0 2024-01-15T18:02:23.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.148+0000 7f75a9600640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f758406bc00 0x7f758406e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:23.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.149+0000 7f75a9600640 1 -- 172.21.15.26:0/2809585915 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f75940aa640 con 0x7f75ac158db0 2024-01-15T18:02:23.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.160+0000 7f75a9600640 1 -- 172.21.15.26:0/2809585915 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7594076f00 con 0x7f75ac158db0 2024-01-15T18:02:23.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.197+0000 7f75b1a00640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f758406bc00 0x7f758406e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:23.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.223+0000 7f75b1a00640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f758406bc00 0x7f758406e0c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f759c005880 tx=0x7f759c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:23.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.598+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f757c0057d0 con 0x7f75ac158db0 2024-01-15T18:02:23.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.599+0000 7f75a9600640 1 -- 172.21.15.26:0/2809585915 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2427 (secure 0 0 0) 0x7f759407a520 con 0x7f75ac158db0 2024-01-15T18:02:23.604 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:23.605 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:23.605 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:23.605 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:23.605 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:23.605 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:23.605 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:23.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:23.605 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:23.605 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:23.606 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:23.606 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:23.606 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:23.606 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:23.606 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:23.606 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:23.606 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:23.606 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:23.606 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 457, 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:23.607 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:23.608 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:23.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:23.608 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:23.608 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:23.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:23.608 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:23.608 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:23.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:23.609 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:23.609 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:23.609 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:23.609 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:23.609 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:23.609 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:23.609 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1877, 2024-01-15T18:02:23.609 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 7723083287, 2024-01-15T18:02:23.609 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 14980587520, 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 758113525760, 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 378733334, 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 374917573, 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 91, 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 98 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:23.610 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:23.611 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:23.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:23.612 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:23.612 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:23.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:23.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:23.612 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:23.612 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:23.612 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:23.613 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 41.668% (11s)\n [===========.................] (remaining: 16s)", 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.41667500138282776, 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 23.81% (11s)\n [======......................] (remaining: 36s)", 2024-01-15T18:02:23.614 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.23810000717639923, 2024-01-15T18:02:23.615 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:23.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:23.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:23.615 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f758406bc00 msgr2=0x7f758406e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f758406bc00 0x7f758406e0c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f759c005880 tx=0x7f759c009040 comp rx=0 tx=0).stop 2024-01-15T18:02:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ac158db0 msgr2=0x7f75ac152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ac158db0 0x7f75ac152560 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7594034040 tx=0x7f75940046c0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 shutdown_connections 2024-01-15T18:02:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f758406bc00 0x7f758406e0c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ac15efc0 0x7f75ac14e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ac158db0 0x7f75ac152560 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 --2- 172.21.15.26:0/2809585915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ac06ee90 0x7f75ac150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 >> 172.21.15.26:0/2809585915 conn(0x7f75ac0b6500 msgr2=0x7f75ac15b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 shutdown_connections 2024-01-15T18:02:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:23.602+0000 7f75b4400640 1 -- 172.21.15.26:0/2809585915 wait complete. 2024-01-15T18:02:25.631 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.854+0000 7f5a4d200640 1 Processor -- start 2024-01-15T18:02:25.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.854+0000 7f5a4d200640 1 -- start start 2024-01-15T18:02:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.856+0000 7f5a4d200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5a4813df80 0x7f5a4815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.856+0000 7f5a4d200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4813e360 0x7f5a480cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.856+0000 7f5a4d200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4806c470 con 0x7f5a4815f9c0 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.856+0000 7f5a4d200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4806c750 con 0x7f5a4813df80 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.856+0000 7f5a4d200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4806ca30 con 0x7f5a4813e360 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.856+0000 7f5a46800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5a4815f9c0 0x7f5a481408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35860/0 (socket says 172.21.15.26:35860) 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.856+0000 7f5a45e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5a4813df80 0x7f5a4815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.856+0000 7f5a46800640 1 -- 172.21.15.26:0/4223292008 learned_addr learned my addr 172.21.15.26:0/4223292008 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.856+0000 7f5a45400640 1 --2- 172.21.15.26:0/4223292008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4813e360 0x7f5a480cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.857+0000 7f5a44a00640 1 -- 172.21.15.26:0/4223292008 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357752631 0 0) 0x7f5a4806c470 con 0x7f5a4815f9c0 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.857+0000 7f5a44a00640 1 -- 172.21.15.26:0/4223292008 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a28003520 con 0x7f5a4815f9c0 2024-01-15T18:02:25.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.857+0000 7f5a45e00640 1 -- 172.21.15.26:0/4223292008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4813e360 msgr2=0x7f5a480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:25.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.857+0000 7f5a45e00640 1 --2- 172.21.15.26:0/4223292008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4813e360 0x7f5a480cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:25.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.857+0000 7f5a45e00640 1 -- 172.21.15.26:0/4223292008 >> v1:172.21.15.26:6789/0 conn(0x7f5a4815f9c0 legacy=0x7f5a481408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:25.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.857+0000 7f5a45e00640 1 -- 172.21.15.26:0/4223292008 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4806cd10 con 0x7f5a4813df80 2024-01-15T18:02:25.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.859+0000 7f5a45e00640 1 --2- 172.21.15.26:0/4223292008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813df80 0x7f5a4815bee0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5a380097a0 tx=0x7f5a3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73354b5912f81125 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:25.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.859+0000 7f5a44a00640 1 -- 172.21.15.26:0/4223292008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38031830 con 0x7f5a4813df80 2024-01-15T18:02:25.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.859+0000 7f5a44a00640 1 -- 172.21.15.26:0/4223292008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38031990 con 0x7f5a4813df80 2024-01-15T18:02:25.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.859+0000 7f5a44a00640 1 -- 172.21.15.26:0/4223292008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38031d20 con 0x7f5a4813df80 2024-01-15T18:02:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.860+0000 7f5a4d200640 1 -- 172.21.15.26:0/4223292008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813df80 msgr2=0x7f5a4815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.860+0000 7f5a4d200640 1 --2- 172.21.15.26:0/4223292008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813df80 0x7f5a4815bee0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5a380097a0 tx=0x7f5a3802b8d0 comp rx=0 tx=0).stop 2024-01-15T18:02:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.860+0000 7f5a4d200640 1 -- 172.21.15.26:0/4223292008 shutdown_connections 2024-01-15T18:02:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.860+0000 7f5a4d200640 1 --2- 172.21.15.26:0/4223292008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4813e360 0x7f5a480cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:25.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.860+0000 7f5a4d200640 1 --2- 172.21.15.26:0/4223292008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813df80 0x7f5a4815bee0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.860+0000 7f5a4d200640 1 -- 172.21.15.26:0/4223292008 >> 172.21.15.26:0/4223292008 conn(0x7f5a4806a800 msgr2=0x7f5a4806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.860+0000 7f5a4d200640 1 -- 172.21.15.26:0/4223292008 shutdown_connections 2024-01-15T18:02:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.860+0000 7f5a4d200640 1 -- 172.21.15.26:0/4223292008 wait complete. 2024-01-15T18:02:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.862+0000 7f5a4d200640 1 Processor -- start 2024-01-15T18:02:25.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.862+0000 7f5a4d200640 1 -- start start 2024-01-15T18:02:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.863+0000 7f5a4d200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4813df80 0x7f5a481509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.863+0000 7f5a4d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813e360 0x7f5a480b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.863+0000 7f5a4d200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815f9c0 0x7f5a48071220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.863+0000 7f5a4d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48070d90 con 0x7f5a4813e360 2024-01-15T18:02:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.863+0000 7f5a4d200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4814f1d0 con 0x7f5a4813df80 2024-01-15T18:02:25.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.863+0000 7f5a4d200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48070460 con 0x7f5a4815f9c0 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.863+0000 7f5a45400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813e360 0x7f5a480b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.863+0000 7f5a45400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813e360 0x7f5a480b2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48304/0 (socket says 172.21.15.26:48304) 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.863+0000 7f5a45400640 1 -- 172.21.15.26:0/872152301 learned_addr learned my addr 172.21.15.26:0/872152301 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a45400640 1 -- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815f9c0 msgr2=0x7f5a48071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a45400640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815f9c0 0x7f5a48071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a45400640 1 -- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4813df80 msgr2=0x7f5a481509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a45e00640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4813df80 0x7f5a481509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a45400640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4813df80 0x7f5a481509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a45400640 1 -- 172.21.15.26:0/872152301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a38009400 con 0x7f5a4813e360 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a45400640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813e360 0x7f5a480b2690 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000efc0 tx=0x7f5a3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:25.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a3e000640 1 -- 172.21.15.26:0/872152301 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a30014480 con 0x7f5a4813e360 2024-01-15T18:02:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a3e000640 1 -- 172.21.15.26:0/872152301 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3000f040 con 0x7f5a4813e360 2024-01-15T18:02:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a3e000640 1 -- 172.21.15.26:0/872152301 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a30014800 con 0x7f5a4813e360 2024-01-15T18:02:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.864+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a48070740 con 0x7f5a4813e360 2024-01-15T18:02:25.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.865+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a48150ee0 con 0x7f5a4813e360 2024-01-15T18:02:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.866+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a48070f40 con 0x7f5a4813e360 2024-01-15T18:02:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.867+0000 7f5a3e000640 1 -- 172.21.15.26:0/872152301 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f5a30014960 con 0x7f5a4813e360 2024-01-15T18:02:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.868+0000 7f5a3e000640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5a2006bb30 0x7f5a2006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:25.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.868+0000 7f5a3e000640 1 -- 172.21.15.26:0/872152301 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5a30089520 con 0x7f5a4813e360 2024-01-15T18:02:25.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.868+0000 7f5a45e00640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5a2006bb30 0x7f5a2006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:25.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.870+0000 7f5a45e00640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5a2006bb30 0x7f5a2006dff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5a38036000 tx=0x7f5a3803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:25.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:25.872+0000 7f5a3e000640 1 -- 172.21.15.26:0/872152301 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a48070f40 con 0x7f5a4813e360 2024-01-15T18:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.273+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5a480675d0 con 0x7f5a4813e360 2024-01-15T18:02:26.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.274+0000 7f5a3e000640 1 -- 172.21.15.26:0/872152301 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2412 (secure 0 0 0) 0x7f5a300596a0 con 0x7f5a4813e360 2024-01-15T18:02:26.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:26.279 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:26.279 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:26.279 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:26.280 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 460, 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:26.281 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 2636, 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 10910882637, 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 21355511808, 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 751738601472, 2024-01-15T18:02:26.282 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 477588834, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 461540856, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 114, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 118 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:26.283 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:26.284 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "2 ongoing cloning jobs; Avg progress = 50.0% (17s)\n [==============..............] (remaining: 17s)", 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.5, 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:26.285 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 42.857% (17s)\n [============................] (remaining: 23s)", 2024-01-15T18:02:26.286 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.4285714328289032, 2024-01-15T18:02:26.286 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:26.286 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:26.286 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:26.286 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.277+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5a2006bb30 msgr2=0x7f5a2006dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.277+0000 7f5a4d200640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5a2006bb30 0x7f5a2006dff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5a38036000 tx=0x7f5a3803c040 comp rx=0 tx=0).stop 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.277+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813e360 msgr2=0x7f5a480b2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.277+0000 7f5a4d200640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813e360 0x7f5a480b2690 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000efc0 tx=0x7f5a3000c490 comp rx=0 tx=0).stop 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.277+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 shutdown_connections 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.278+0000 7f5a4d200640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5a4815f9c0 0x7f5a48071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.278+0000 7f5a4d200640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5a2006bb30 0x7f5a2006dff0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.278+0000 7f5a4d200640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5a4813e360 0x7f5a480b2690 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.278+0000 7f5a4d200640 1 --2- 172.21.15.26:0/872152301 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5a4813df80 0x7f5a481509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.278+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 >> 172.21.15.26:0/872152301 conn(0x7f5a4806a800 msgr2=0x7f5a480b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.278+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 shutdown_connections 2024-01-15T18:02:26.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:26.278+0000 7f5a4d200640 1 -- 172.21.15.26:0/872152301 wait complete. 2024-01-15T18:02:28.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:02:28.123 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:02:28.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.614+0000 7fbcfc000640 1 Processor -- start 2024-01-15T18:02:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.615+0000 7fbcfc000640 1 -- start start 2024-01-15T18:02:28.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.617+0000 7fbcfc000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40cfcf0 0x7fbcf4160700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:28.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.617+0000 7fbcfc000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbcf40d00d0 0x7fbcf4160c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:28.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.617+0000 7fbcfc000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf40d4160 con 0x7fbcf40d3910 2024-01-15T18:02:28.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.617+0000 7fbcfc000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf40ceff0 con 0x7fbcf40d00d0 2024-01-15T18:02:28.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.617+0000 7fbcfc000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40cf410 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.618+0000 7fbcf9600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40cfcf0 0x7fbcf4160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:28.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.618+0000 7fbcf8c00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fbcf40d3910 0x7fbcf40b2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:58398/0 (socket says 172.21.15.26:58398) 2024-01-15T18:02:28.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.618+0000 7fbcf8c00640 1 -- 172.21.15.26:0/1173580178 learned_addr learned my addr 172.21.15.26:0/1173580178 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:28.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.618+0000 7fbcfa000640 1 --2- 172.21.15.26:0/1173580178 >> v2:172.21.15.26:3300/0 conn(0x7fbcf40d00d0 0x7fbcf4160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.619+0000 7fbcfa000640 1 -- 172.21.15.26:0/1173580178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40cfcf0 msgr2=0x7fbcf4160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.619+0000 7fbcfa000640 1 --2- 172.21.15.26:0/1173580178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40cfcf0 0x7fbcf4160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.619+0000 7fbcfa000640 1 -- 172.21.15.26:0/1173580178 >> v1:172.21.15.124:6789/0 conn(0x7fbcf40d3910 legacy=0x7fbcf40b2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:28.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.619+0000 7fbcfa000640 1 -- 172.21.15.26:0/1173580178 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf40b2420 con 0x7fbcf40d00d0 2024-01-15T18:02:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.620+0000 7fbcfa000640 1 --2- 172.21.15.26:0/1173580178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40d00d0 0x7fbcf4160c40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009dc0 tx=0x7fbcec02be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c69fa4111c6ce9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.621+0000 7fbcf3e00640 1 -- 172.21.15.26:0/1173580178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec02cbb0 con 0x7fbcf40d00d0 2024-01-15T18:02:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.621+0000 7fbcf3e00640 1 -- 172.21.15.26:0/1173580178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec02cd10 con 0x7fbcf40d00d0 2024-01-15T18:02:28.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.621+0000 7fbcf3e00640 1 -- 172.21.15.26:0/1173580178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec034660 con 0x7fbcf40d00d0 2024-01-15T18:02:28.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.622+0000 7fbcfc000640 1 -- 172.21.15.26:0/1173580178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40d00d0 msgr2=0x7fbcf4160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:28.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.622+0000 7fbcfc000640 1 --2- 172.21.15.26:0/1173580178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40d00d0 0x7fbcf4160c40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009dc0 tx=0x7fbcec02be00 comp rx=0 tx=0).stop 2024-01-15T18:02:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.622+0000 7fbcfc000640 1 -- 172.21.15.26:0/1173580178 shutdown_connections 2024-01-15T18:02:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.622+0000 7fbcfc000640 1 --2- 172.21.15.26:0/1173580178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40d00d0 0x7fbcf4160c40 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.622+0000 7fbcfc000640 1 --2- 172.21.15.26:0/1173580178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40cfcf0 0x7fbcf4160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.622+0000 7fbcfc000640 1 -- 172.21.15.26:0/1173580178 >> 172.21.15.26:0/1173580178 conn(0x7fbcf409c530 msgr2=0x7fbcf40d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.622+0000 7fbcfc000640 1 -- 172.21.15.26:0/1173580178 shutdown_connections 2024-01-15T18:02:28.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.623+0000 7fbcfc000640 1 -- 172.21.15.26:0/1173580178 wait complete. 2024-01-15T18:02:28.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.625+0000 7fbcfc000640 1 Processor -- start 2024-01-15T18:02:28.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.625+0000 7fbcfc000640 1 -- start start 2024-01-15T18:02:28.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcfc000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40cfcf0 0x7fbcf4154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:28.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcfc000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbcf40d00d0 0x7fbcf40b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:28.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcfc000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40d3910 0x7fbcf415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:28.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcfc000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40b82b0 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcfc000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40b8590 con 0x7fbcf40d00d0 2024-01-15T18:02:28.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcfc000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf40b8870 con 0x7fbcf40d3910 2024-01-15T18:02:28.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcf9600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40cfcf0 0x7fbcf4154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:28.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcf9600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40cfcf0 0x7fbcf4154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48316/0 (socket says 172.21.15.26:48316) 2024-01-15T18:02:28.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcf9600640 1 -- 172.21.15.26:0/542408163 learned_addr learned my addr 172.21.15.26:0/542408163 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:28.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.626+0000 7fbcf8c00640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbcf40d00d0 0x7fbcf40b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:28.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.627+0000 7fbcfa000640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40d3910 0x7fbcf415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:28.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.627+0000 7fbcf9600640 1 -- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40d3910 msgr2=0x7fbcf415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:28.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.627+0000 7fbcf9600640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40d3910 0x7fbcf415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:28.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.627+0000 7fbcf9600640 1 -- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbcf40d00d0 msgr2=0x7fbcf40b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:28.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.627+0000 7fbcf9600640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbcf40d00d0 0x7fbcf40b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:28.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.627+0000 7fbcf9600640 1 -- 172.21.15.26:0/542408163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec009a70 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.629+0000 7fbcf9600640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40cfcf0 0x7fbcf4154640 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbce400e870 tx=0x7fbce400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:28.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.630+0000 7fbcf1600640 1 -- 172.21.15.26:0/542408163 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce40094f0 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.630+0000 7fbcf1600640 1 -- 172.21.15.26:0/542408163 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce400c940 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.630+0000 7fbcf1600640 1 -- 172.21.15.26:0/542408163 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce400ccf0 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.630+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf40b8b50 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.631+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf4154b80 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.633+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf40675d0 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.637+0000 7fbcf1600640 1 -- 172.21.15.26:0/542408163 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7fbce4013470 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.639+0000 7fbcf1600640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbccc06bb80 0x7fbccc06e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:28.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.639+0000 7fbcf1600640 1 -- 172.21.15.26:0/542408163 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbce4088e10 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.641+0000 7fbcf8c00640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbccc06bb80 0x7fbccc06e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.643+0000 7fbcf1600640 1 -- 172.21.15.26:0/542408163 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbce40555f0 con 0x7fbcf40cfcf0 2024-01-15T18:02:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:28.648+0000 7fbcf8c00640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbccc06bb80 0x7fbccc06e040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40ca280 tx=0x7fbcdc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:29.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.096+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbcf415f920 con 0x7fbcf40cfcf0 2024-01-15T18:02:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.096+0000 7fbcf1600640 1 -- 172.21.15.26:0/542408163 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2412 (secure 0 0 0) 0x7fbce4021090 con 0x7fbcf40cfcf0 2024-01-15T18:02:29.100 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:29.100 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:29.100 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:29.100 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:29.100 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:29.100 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:29.100 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:29.101 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 463, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:29.102 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 3274, 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 13587051461, 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 26877362176, 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 746216751104, 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 693573328, 2024-01-15T18:02:29.103 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 684530213, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 166, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 176 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:29.104 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:29.105 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "2 ongoing cloning jobs; Avg progress = 50.0% (17s)\n [==============..............] (remaining: 17s)", 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.5, 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 42.857% (17s)\n [============................] (remaining: 23s)", 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.4285714328289032, 2024-01-15T18:02:29.106 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:29.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:29.107 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:29.107 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:29.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbccc06bb80 msgr2=0x7fbccc06e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbccc06bb80 0x7fbccc06e040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbcf40ca280 tx=0x7fbcdc009040 comp rx=0 tx=0).stop 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40cfcf0 msgr2=0x7fbcf4154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40cfcf0 0x7fbcf4154640 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fbce400e870 tx=0x7fbce400ed30 comp rx=0 tx=0).stop 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 shutdown_connections 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbcf40d3910 0x7fbcf415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbccc06bb80 0x7fbccc06e040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbcf40d00d0 0x7fbcf40b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 --2- 172.21.15.26:0/542408163 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbcf40cfcf0 0x7fbcf4154640 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 >> 172.21.15.26:0/542408163 conn(0x7fbcf409c530 msgr2=0x7fbcf40b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 shutdown_connections 2024-01-15T18:02:29.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:29.098+0000 7fbcfc000640 1 -- 172.21.15.26:0/542408163 wait complete. 2024-01-15T18:02:31.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:31.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.443+0000 7f9477600640 1 Processor -- start 2024-01-15T18:02:31.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.444+0000 7f9477600640 1 -- start start 2024-01-15T18:02:31.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.447+0000 7f9477600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9470150760 0x7f9470072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:31.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.447+0000 7f9477600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f947015d300 0x7f947015d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.447+0000 7f9477600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947015dc80 con 0x7f94700b2bd0 2024-01-15T18:02:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.447+0000 7f9477600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94700b3770 con 0x7f947015d300 2024-01-15T18:02:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.447+0000 7f9477600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9470071190 con 0x7f9470150760 2024-01-15T18:02:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.448+0000 7f9474c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9470150760 0x7f9470072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:31.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.448+0000 7f9474c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9470150760 0x7f9470072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:56762/0 (socket says 172.21.15.26:56762) 2024-01-15T18:02:31.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.448+0000 7f9474c00640 1 -- 172.21.15.26:0/3483216663 learned_addr learned my addr 172.21.15.26:0/3483216663 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:31.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.448+0000 7f9467e00640 1 --2- 172.21.15.26:0/3483216663 >> v2:172.21.15.124:3300/0 conn(0x7f947015d300 0x7f947015d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:31.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.448+0000 7f9474c00640 1 -- 172.21.15.26:0/3483216663 >> v2:172.21.15.124:3300/0 conn(0x7f947015d300 msgr2=0x7f947015d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:31.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.448+0000 7f9474c00640 1 --2- 172.21.15.26:0/3483216663 >> v2:172.21.15.124:3300/0 conn(0x7f947015d300 0x7f947015d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:31.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.448+0000 7f9474c00640 1 -- 172.21.15.26:0/3483216663 >> v1:172.21.15.124:6789/0 conn(0x7f94700b2bd0 legacy=0x7f94700b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:31.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.448+0000 7f9474c00640 1 -- 172.21.15.26:0/3483216663 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9470071470 con 0x7f9470150760 2024-01-15T18:02:31.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.450+0000 7f9474c00640 1 --2- 172.21.15.26:0/3483216663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9470150760 0x7f9470072230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f946000aa10 tx=0x7f946002ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d8d8add58fad2ad server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:31.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.450+0000 7f946fe00640 1 -- 172.21.15.26:0/3483216663 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946002dc10 con 0x7f9470150760 2024-01-15T18:02:31.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.450+0000 7f946fe00640 1 -- 172.21.15.26:0/3483216663 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946002dd70 con 0x7f9470150760 2024-01-15T18:02:31.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.450+0000 7f946fe00640 1 -- 172.21.15.26:0/3483216663 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460035660 con 0x7f9470150760 2024-01-15T18:02:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.451+0000 7f9477600640 1 -- 172.21.15.26:0/3483216663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9470150760 msgr2=0x7f9470072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.451+0000 7f9477600640 1 --2- 172.21.15.26:0/3483216663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9470150760 0x7f9470072230 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f946000aa10 tx=0x7f946002ce60 comp rx=0 tx=0).stop 2024-01-15T18:02:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.451+0000 7f9477600640 1 -- 172.21.15.26:0/3483216663 shutdown_connections 2024-01-15T18:02:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.451+0000 7f9477600640 1 --2- 172.21.15.26:0/3483216663 >> v2:172.21.15.124:3300/0 conn(0x7f947015d300 0x7f947015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.451+0000 7f9477600640 1 --2- 172.21.15.26:0/3483216663 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9470150760 0x7f9470072230 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.451+0000 7f9477600640 1 -- 172.21.15.26:0/3483216663 >> 172.21.15.26:0/3483216663 conn(0x7f947006a800 msgr2=0x7f947006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.451+0000 7f9477600640 1 -- 172.21.15.26:0/3483216663 shutdown_connections 2024-01-15T18:02:31.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.451+0000 7f9477600640 1 -- 172.21.15.26:0/3483216663 wait complete. 2024-01-15T18:02:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.455+0000 7f9477600640 1 Processor -- start 2024-01-15T18:02:31.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.455+0000 7f9477600640 1 -- start start 2024-01-15T18:02:31.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9477600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94700b2bd0 0x7f94701fa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:31.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9477600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9470150760 0x7f9470165450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:31.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9477600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947015d300 0x7f9470165990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9477600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9470139a30 con 0x7f947015d300 2024-01-15T18:02:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9477600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9470139750 con 0x7f9470150760 2024-01-15T18:02:31.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9477600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947013b230 con 0x7f94700b2bd0 2024-01-15T18:02:31.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9474c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94700b2bd0 0x7f94701fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:31.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9467e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9470150760 0x7f9470165450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9474c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94700b2bd0 0x7f94701fa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:56776/0 (socket says 172.21.15.26:56776) 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9474c00640 1 -- 172.21.15.26:0/2883531452 learned_addr learned my addr 172.21.15.26:0/2883531452 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9467e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9470150760 0x7f9470165450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:38078/0 (socket says 172.21.15.26:38078) 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.456+0000 7f9475600640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947015d300 0x7f9470165990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.457+0000 7f9474c00640 1 -- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9470150760 msgr2=0x7f9470165450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.457+0000 7f9474c00640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9470150760 0x7f9470165450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.457+0000 7f9474c00640 1 -- 172.21.15.26:0/2883531452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947015d300 msgr2=0x7f9470165990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.457+0000 7f9474c00640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947015d300 0x7f9470165990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.457+0000 7f9474c00640 1 -- 172.21.15.26:0/2883531452 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946000a670 con 0x7f94700b2bd0 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.457+0000 7f9475600640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947015d300 0x7f9470165990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:02:31.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.457+0000 7f9474c00640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94700b2bd0 0x7f94701fa450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f946000a1c0 tx=0x7f9460004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.458+0000 7f946d600640 1 -- 172.21.15.26:0/2883531452 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460040650 con 0x7f94700b2bd0 2024-01-15T18:02:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.458+0000 7f946d600640 1 -- 172.21.15.26:0/2883531452 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94600407f0 con 0x7f94700b2bd0 2024-01-15T18:02:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.458+0000 7f946d600640 1 -- 172.21.15.26:0/2883531452 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460040ba0 con 0x7f94700b2bd0 2024-01-15T18:02:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.458+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947013b510 con 0x7f94700b2bd0 2024-01-15T18:02:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.458+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94701fa990 con 0x7f94700b2bd0 2024-01-15T18:02:31.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.459+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947013ba30 con 0x7f94700b2bd0 2024-01-15T18:02:31.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.460+0000 7f946d600640 1 -- 172.21.15.26:0/2883531452 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f946003c070 con 0x7f94700b2bd0 2024-01-15T18:02:31.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.461+0000 7f946d600640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f944806bb60 0x7f944806e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:31.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.462+0000 7f946d600640 1 -- 172.21.15.26:0/2883531452 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f94600aad60 con 0x7f94700b2bd0 2024-01-15T18:02:31.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.462+0000 7f9467e00640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f944806bb60 0x7f944806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:31.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.463+0000 7f9467e00640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f944806bb60 0x7f944806e020 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9470137ae0 tx=0x7f9458009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:31.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.469+0000 7f946d600640 1 -- 172.21.15.26:0/2883531452 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9460077450 con 0x7f94700b2bd0 2024-01-15T18:02:31.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.925+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f947013b370 con 0x7f94700b2bd0 2024-01-15T18:02:31.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.926+0000 7f946d600640 1 -- 172.21.15.26:0/2883531452 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2430 (secure 0 0 0) 0x7f946003a990 con 0x7f94700b2bd0 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:31.931 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 466, 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:31.932 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:31.933 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 3274, 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 13587051461, 2024-01-15T18:02:31.934 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 26877362176, 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 746216751104, 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 493690229, 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 488479050, 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 118, 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 126 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:31.935 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:31.936 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:31.937 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "2 ongoing cloning jobs; Avg progress = 83.335% (22s)\n [=======================.....] (remaining: 4s)", 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.83335000276565552, 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:31.938 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:31.939 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:31.939 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 52.381% (22s)\n [==============..............] (remaining: 20s)", 2024-01-15T18:02:31.939 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.52381426095962524, 2024-01-15T18:02:31.939 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:31.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:31.939 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:31.939 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.933+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f944806bb60 msgr2=0x7f944806e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.933+0000 7f9477600640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f944806bb60 0x7f944806e020 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9470137ae0 tx=0x7f9458009190 comp rx=0 tx=0).stop 2024-01-15T18:02:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.933+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94700b2bd0 msgr2=0x7f94701fa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.933+0000 7f9477600640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94700b2bd0 0x7f94701fa450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f946000a1c0 tx=0x7f9460004730 comp rx=0 tx=0).stop 2024-01-15T18:02:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.934+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 shutdown_connections 2024-01-15T18:02:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.934+0000 7f9477600640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f944806bb60 0x7f944806e020 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:31.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.934+0000 7f9477600640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f947015d300 0x7f9470165990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.934+0000 7f9477600640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9470150760 0x7f9470165450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.934+0000 7f9477600640 1 --2- 172.21.15.26:0/2883531452 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94700b2bd0 0x7f94701fa450 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f946000a1c0 tx=0x7f9460004730 comp rx=0 tx=0).stop 2024-01-15T18:02:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.934+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 >> 172.21.15.26:0/2883531452 conn(0x7f947006a800 msgr2=0x7f947006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.934+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 shutdown_connections 2024-01-15T18:02:31.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:31.935+0000 7f9477600640 1 -- 172.21.15.26:0/2883531452 wait complete. 2024-01-15T18:02:33.967 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:02:34.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.193+0000 7fa757e00640 1 Processor -- start 2024-01-15T18:02:34.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.194+0000 7fa757e00640 1 -- start start 2024-01-15T18:02:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.195+0000 7fa757e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa758143c40 0x7fa758071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.195+0000 7fa757e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7580717d0 0x7fa7580cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:34.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.195+0000 7fa757e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa758070e50 con 0x7fa7580cca40 2024-01-15T18:02:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.195+0000 7fa757e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7580723c0 con 0x7fa7580717d0 2024-01-15T18:02:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.195+0000 7fa757e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7580cce40 con 0x7fa758143c40 2024-01-15T18:02:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.196+0000 7fa74fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7580717d0 0x7fa7580cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.196+0000 7fa74fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa7580717d0 0x7fa7580cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51222/0 (socket says 172.21.15.26:51222) 2024-01-15T18:02:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.196+0000 7fa74fe00640 1 -- 172.21.15.26:0/888541859 learned_addr learned my addr 172.21.15.26:0/888541859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.196+0000 7fa756a00640 1 --2- 172.21.15.26:0/888541859 >> v2:172.21.15.124:3300/0 conn(0x7fa758143c40 0x7fa758071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.196+0000 7fa74fe00640 1 -- 172.21.15.26:0/888541859 >> v2:172.21.15.124:3300/0 conn(0x7fa758143c40 msgr2=0x7fa758071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.196+0000 7fa74fe00640 1 --2- 172.21.15.26:0/888541859 >> v2:172.21.15.124:3300/0 conn(0x7fa758143c40 0x7fa758071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.196+0000 7fa74fe00640 1 -- 172.21.15.26:0/888541859 >> v1:172.21.15.124:6789/0 conn(0x7fa7580cca40 legacy=0x7fa75815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:34.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.196+0000 7fa74fe00640 1 -- 172.21.15.26:0/888541859 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75806e170 con 0x7fa7580717d0 2024-01-15T18:02:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.197+0000 7fa74fe00640 1 --2- 172.21.15.26:0/888541859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 0x7fa7580cc500 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa75806e410 tx=0x7fa74002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1325c7425de2f604 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:34.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.198+0000 7fa756000640 1 -- 172.21.15.26:0/888541859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7400317e0 con 0x7fa7580717d0 2024-01-15T18:02:34.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.198+0000 7fa756000640 1 -- 172.21.15.26:0/888541859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa740031940 con 0x7fa7580717d0 2024-01-15T18:02:34.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.198+0000 7fa756000640 1 -- 172.21.15.26:0/888541859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740031cd0 con 0x7fa7580717d0 2024-01-15T18:02:34.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.198+0000 7fa757e00640 1 -- 172.21.15.26:0/888541859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 msgr2=0x7fa7580cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:34.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.198+0000 7fa757e00640 1 --2- 172.21.15.26:0/888541859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 0x7fa7580cc500 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa75806e410 tx=0x7fa74002b880 comp rx=0 tx=0).stop 2024-01-15T18:02:34.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.199+0000 7fa757e00640 1 -- 172.21.15.26:0/888541859 shutdown_connections 2024-01-15T18:02:34.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.199+0000 7fa757e00640 1 --2- 172.21.15.26:0/888541859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 0x7fa7580cc500 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.199+0000 7fa757e00640 1 --2- 172.21.15.26:0/888541859 >> v2:172.21.15.124:3300/0 conn(0x7fa758143c40 0x7fa758071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.199+0000 7fa757e00640 1 -- 172.21.15.26:0/888541859 >> 172.21.15.26:0/888541859 conn(0x7fa75806a800 msgr2=0x7fa75806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:34.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.199+0000 7fa757e00640 1 -- 172.21.15.26:0/888541859 shutdown_connections 2024-01-15T18:02:34.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.199+0000 7fa757e00640 1 -- 172.21.15.26:0/888541859 wait complete. 2024-01-15T18:02:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.201+0000 7fa757e00640 1 Processor -- start 2024-01-15T18:02:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.201+0000 7fa757e00640 1 -- start start 2024-01-15T18:02:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.202+0000 7fa757e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 0x7fa75813a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.202+0000 7fa757e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7580cca40 0x7fa7581f0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.202+0000 7fa757e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa758143c40 0x7fa758137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.202+0000 7fa757e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7581373c0 con 0x7fa7580717d0 2024-01-15T18:02:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.202+0000 7fa757e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75813a680 con 0x7fa758143c40 2024-01-15T18:02:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.202+0000 7fa757e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75813a960 con 0x7fa7580cca40 2024-01-15T18:02:34.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa756a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 0x7fa75813a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa756a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 0x7fa75813a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51226/0 (socket says 172.21.15.26:51226) 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa756a00640 1 -- 172.21.15.26:0/685882561 learned_addr learned my addr 172.21.15.26:0/685882561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa74fe00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7580cca40 0x7fa7581f0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa757400640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa758143c40 0x7fa758137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa756a00640 1 -- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7580cca40 msgr2=0x7fa7581f0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa756a00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7580cca40 0x7fa7581f0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa756a00640 1 -- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa758143c40 msgr2=0x7fa758137bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa756a00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa758143c40 0x7fa758137bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.203+0000 7fa756a00640 1 -- 172.21.15.26:0/685882561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7400094d0 con 0x7fa7580717d0 2024-01-15T18:02:34.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.204+0000 7fa756a00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 0x7fa75813a0a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa74800d5f0 tx=0x7fa74800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:34.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.204+0000 7fa74ea00640 1 -- 172.21.15.26:0/685882561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7480125f0 con 0x7fa7580717d0 2024-01-15T18:02:34.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.204+0000 7fa74ea00640 1 -- 172.21.15.26:0/685882561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa748012750 con 0x7fa7580717d0 2024-01-15T18:02:34.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.204+0000 7fa74ea00640 1 -- 172.21.15.26:0/685882561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748012b00 con 0x7fa7580717d0 2024-01-15T18:02:34.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.204+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75813ac40 con 0x7fa7580717d0 2024-01-15T18:02:34.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.204+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75813b200 con 0x7fa7580717d0 2024-01-15T18:02:34.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.205+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa758067450 con 0x7fa7580717d0 2024-01-15T18:02:34.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.207+0000 7fa74ea00640 1 -- 172.21.15.26:0/685882561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7fa748012c60 con 0x7fa7580717d0 2024-01-15T18:02:34.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.208+0000 7fa74ea00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa73006bb30 0x7fa73006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:34.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.208+0000 7fa74ea00640 1 -- 172.21.15.26:0/685882561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa748088af0 con 0x7fa7580717d0 2024-01-15T18:02:34.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.208+0000 7fa74fe00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa73006bb30 0x7fa73006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:34.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.210+0000 7fa74fe00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa73006bb30 0x7fa73006dff0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa740036000 tx=0x7fa74003d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:34.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.213+0000 7fa74ea00640 1 -- 172.21.15.26:0/685882561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7480551e0 con 0x7fa7580717d0 2024-01-15T18:02:34.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.484+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa7580ceb30 con 0x7fa7580717d0 2024-01-15T18:02:34.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.485+0000 7fa74ea00640 1 -- 172.21.15.26:0/685882561 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2405 (secure 0 0 0) 0x7fa748058800 con 0x7fa7580717d0 2024-01-15T18:02:34.489 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:34.489 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:34.489 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:02:34.489 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:02:34.490 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 468, 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 60, 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:02:34.491 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:02:34.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 3833, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 15927589640, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 31585476608, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 741508636672, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 681697147, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 683484453, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 163, 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 175 2024-01-15T18:02:34.493 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 36, 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 4, 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 4, 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:02:34.494 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 35186 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:34.495 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 14, 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:01:48.060374+0000", 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "10822bda-5aed-4ea6-9297-38dffc358d6e": { 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "Subvolume \"subvol_clone20837\" has been 100.0% cloned (25s)\n [============================] ", 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 1, 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "119b3b9f-7af4-492c-a5f5-c83938b3d70c": { 2024-01-15T18:02:34.496 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 57.143% (25s)\n [================............] (remaining: 19s)", 2024-01-15T18:02:34.497 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.57142859697341919, 2024-01-15T18:02:34.497 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:02:34.497 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:34.497 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:34.497 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa73006bb30 msgr2=0x7fa73006dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa73006bb30 0x7fa73006dff0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa740036000 tx=0x7fa74003d040 comp rx=0 tx=0).stop 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 msgr2=0x7fa75813a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 0x7fa75813a0a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa74800d5f0 tx=0x7fa74800dab0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 shutdown_connections 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7580cca40 0x7fa7581f0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa73006bb30 0x7fa73006dff0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa758143c40 0x7fa758137bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 --2- 172.21.15.26:0/685882561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7580717d0 0x7fa75813a0a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:34.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 >> 172.21.15.26:0/685882561 conn(0x7fa75806a800 msgr2=0x7fa7581406e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 shutdown_connections 2024-01-15T18:02:34.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:34.488+0000 7fa757e00640 1 -- 172.21.15.26:0/685882561 wait complete. 2024-01-15T18:02:36.512 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:02:36.513 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20831 2024-01-15T18:02:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.744+0000 7faa1e400640 1 Processor -- start 2024-01-15T18:02:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.744+0000 7faa1e400640 1 -- start start 2024-01-15T18:02:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa1e400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa18143ae0 0x7faa180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa1e400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faa180cf7d0 0x7faa1815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa1e400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa180cfc30 con 0x7faa1815e1e0 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa1e400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa180b1ee0 con 0x7faa18143ae0 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa1e400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa180b2780 con 0x7faa180cf7d0 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa17400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa18143ae0 0x7faa180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa17400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faa18143ae0 0x7faa180cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51232/0 (socket says 172.21.15.26:51232) 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa17400640 1 -- 172.21.15.26:0/832085126 learned_addr learned my addr 172.21.15.26:0/832085126 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa0fe00640 1 --2- 172.21.15.26:0/832085126 >> v2:172.21.15.124:3300/0 conn(0x7faa180cf7d0 0x7faa1815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa17400640 1 -- 172.21.15.26:0/832085126 >> v2:172.21.15.124:3300/0 conn(0x7faa180cf7d0 msgr2=0x7faa1815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.746+0000 7faa17400640 1 --2- 172.21.15.26:0/832085126 >> v2:172.21.15.124:3300/0 conn(0x7faa180cf7d0 0x7faa1815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.747+0000 7faa17400640 1 -- 172.21.15.26:0/832085126 >> v1:172.21.15.124:6789/0 conn(0x7faa1815e1e0 legacy=0x7faa1815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.747+0000 7faa17400640 1 -- 172.21.15.26:0/832085126 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1815e990 con 0x7faa18143ae0 2024-01-15T18:02:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.748+0000 7faa17400640 1 --2- 172.21.15.26:0/832085126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa18143ae0 0x7faa180cf290 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7faa180cb830 tx=0x7faa0802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27aca76ad1c9224 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.748+0000 7faa16a00640 1 -- 172.21.15.26:0/832085126 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0802ee90 con 0x7faa18143ae0 2024-01-15T18:02:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.748+0000 7faa16a00640 1 -- 172.21.15.26:0/832085126 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa08036440 con 0x7faa18143ae0 2024-01-15T18:02:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.748+0000 7faa16a00640 1 -- 172.21.15.26:0/832085126 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa080367d0 con 0x7faa18143ae0 2024-01-15T18:02:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.749+0000 7faa1e400640 1 -- 172.21.15.26:0/832085126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa18143ae0 msgr2=0x7faa180cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.749+0000 7faa1e400640 1 --2- 172.21.15.26:0/832085126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa18143ae0 0x7faa180cf290 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7faa180cb830 tx=0x7faa0802e0e0 comp rx=0 tx=0).stop 2024-01-15T18:02:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.749+0000 7faa1e400640 1 -- 172.21.15.26:0/832085126 shutdown_connections 2024-01-15T18:02:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.749+0000 7faa1e400640 1 --2- 172.21.15.26:0/832085126 >> v2:172.21.15.124:3300/0 conn(0x7faa180cf7d0 0x7faa1815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.749+0000 7faa1e400640 1 --2- 172.21.15.26:0/832085126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa18143ae0 0x7faa180cf290 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.749+0000 7faa1e400640 1 -- 172.21.15.26:0/832085126 >> 172.21.15.26:0/832085126 conn(0x7faa1806a800 msgr2=0x7faa1806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.749+0000 7faa1e400640 1 -- 172.21.15.26:0/832085126 shutdown_connections 2024-01-15T18:02:36.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.749+0000 7faa1e400640 1 -- 172.21.15.26:0/832085126 wait complete. 2024-01-15T18:02:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.751+0000 7faa1e400640 1 Processor -- start 2024-01-15T18:02:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.751+0000 7faa1e400640 1 -- start start 2024-01-15T18:02:36.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa1e400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa180cf7d0 0x7faa180c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa1e400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa18143ae0 0x7faa180b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa1e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1815e1e0 0x7faa180b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa1e400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa180c6340 con 0x7faa1815e1e0 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa1e400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa180c6620 con 0x7faa18143ae0 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa1e400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa180c3ef0 con 0x7faa180cf7d0 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa17e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1815e1e0 0x7faa180b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa17e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1815e1e0 0x7faa180b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51236/0 (socket says 172.21.15.26:51236) 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa17e00640 1 -- 172.21.15.26:0/458947698 learned_addr learned my addr 172.21.15.26:0/458947698 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa17400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa180cf7d0 0x7faa180c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:36.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.752+0000 7faa0fe00640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa18143ae0 0x7faa180b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa17e00640 1 -- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa180cf7d0 msgr2=0x7faa180c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa17e00640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa180cf7d0 0x7faa180c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa17e00640 1 -- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa18143ae0 msgr2=0x7faa180b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa17e00640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa18143ae0 0x7faa180b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa17e00640 1 -- 172.21.15.26:0/458947698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0800b770 con 0x7faa1815e1e0 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa17e00640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1815e1e0 0x7faa180b1310 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c780 tx=0x7faa1000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa0f400640 1 -- 172.21.15.26:0/458947698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa10003d80 con 0x7faa1815e1e0 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa0f400640 1 -- 172.21.15.26:0/458947698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1000ce90 con 0x7faa1815e1e0 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa0f400640 1 -- 172.21.15.26:0/458947698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa100052b0 con 0x7faa1815e1e0 2024-01-15T18:02:36.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa180c41d0 con 0x7faa1815e1e0 2024-01-15T18:02:36.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.753+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa180c6050 con 0x7faa1815e1e0 2024-01-15T18:02:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.755+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa180b2bf0 con 0x7faa1815e1e0 2024-01-15T18:02:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.756+0000 7faa0f400640 1 -- 172.21.15.26:0/458947698 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7faa10005410 con 0x7faa1815e1e0 2024-01-15T18:02:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.757+0000 7faa0f400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f006bb80 0x7fa9f006e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.757+0000 7faa0f400640 1 -- 172.21.15.26:0/458947698 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7faa10087ec0 con 0x7faa1815e1e0 2024-01-15T18:02:36.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.757+0000 7faa17400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f006bb80 0x7fa9f006e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:36.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.758+0000 7faa17400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f006bb80 0x7fa9f006e040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faa08038000 tx=0x7faa08008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:36.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.762+0000 7faa0f400640 1 -- 172.21.15.26:0/458947698 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa180b2bf0 con 0x7faa1815e1e0 2024-01-15T18:02:36.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:36.995+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20831", "target": ["mon-mgr", ""]}) v1 -- 0x7faa1815e670 con 0x7fa9f006bb80 2024-01-15T18:02:37.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.238+0000 7faa0f400640 1 -- 172.21.15.26:0/458947698 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7faa08036ab0 con 0x7fa9f006bb80 2024-01-15T18:02:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:37.242 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:37.242 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:02:37.242 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:37.242 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:37.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f006bb80 msgr2=0x7fa9f006e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:37.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f006bb80 0x7fa9f006e040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faa08038000 tx=0x7faa08008040 comp rx=0 tx=0).stop 2024-01-15T18:02:37.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1815e1e0 msgr2=0x7faa180b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1815e1e0 0x7faa180b1310 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7faa1000c780 tx=0x7faa1000cc40 comp rx=0 tx=0).stop 2024-01-15T18:02:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 shutdown_connections 2024-01-15T18:02:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa9f006bb80 0x7fa9f006e040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faa1815e1e0 0x7faa180b1310 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faa18143ae0 0x7faa180b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 --2- 172.21.15.26:0/458947698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faa180cf7d0 0x7faa180c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.244+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 >> 172.21.15.26:0/458947698 conn(0x7faa1806a800 msgr2=0x7faa1815ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.245+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 shutdown_connections 2024-01-15T18:02:37.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.245+0000 7faa1e400640 1 -- 172.21.15.26:0/458947698 wait complete. 2024-01-15T18:02:37.274 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:02:37.274 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20832 2024-01-15T18:02:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.533+0000 7f9103e00640 1 Processor -- start 2024-01-15T18:02:37.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.533+0000 7f9103e00640 1 -- start start 2024-01-15T18:02:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.534+0000 7f9103e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f90fc099a60 0x7f90fc099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.534+0000 7f9103e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f90fc06e7c0 0x7f90fc06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.534+0000 7f9103e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90fc0cfa70 con 0x7f90fc15e880 2024-01-15T18:02:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.534+0000 7f9103e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc09a3e0 con 0x7f90fc099a60 2024-01-15T18:02:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.534+0000 7f9103e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90fc0cff00 con 0x7f90fc06e7c0 2024-01-15T18:02:37.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.534+0000 7f9101400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f90fc15e880 0x7f90fc06ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58744/0 (socket says 172.21.15.26:58744) 2024-01-15T18:02:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.535+0000 7f9101400640 1 -- 172.21.15.26:0/2218851495 learned_addr learned my addr 172.21.15.26:0/2218851495 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.535+0000 7f9100a00640 1 --2- 172.21.15.26:0/2218851495 >> v2:172.21.15.26:3300/0 conn(0x7f90fc099a60 0x7f90fc099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.535+0000 7f9101e00640 1 --2- 172.21.15.26:0/2218851495 >> v2:172.21.15.124:3300/0 conn(0x7f90fc06e7c0 0x7f90fc06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.535+0000 7f9100a00640 1 -- 172.21.15.26:0/2218851495 >> v2:172.21.15.124:3300/0 conn(0x7f90fc06e7c0 msgr2=0x7f90fc06ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.535+0000 7f9100a00640 1 --2- 172.21.15.26:0/2218851495 >> v2:172.21.15.124:3300/0 conn(0x7f90fc06e7c0 0x7f90fc06ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.535+0000 7f9100a00640 1 -- 172.21.15.26:0/2218851495 >> v1:172.21.15.26:6789/0 conn(0x7f90fc15e880 legacy=0x7f90fc06ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.535+0000 7f90fbe00640 1 -- 172.21.15.26:0/2218851495 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477957308 0 0) 0x7f90fc0cfa70 con 0x7f90fc15e880 2024-01-15T18:02:37.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.535+0000 7f9100a00640 1 -- 172.21.15.26:0/2218851495 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90fc06fc70 con 0x7f90fc099a60 2024-01-15T18:02:37.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.537+0000 7f9100a00640 1 --2- 172.21.15.26:0/2218851495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc099a60 0x7f90fc099ea0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0b72c0 tx=0x7f90e402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5f6ae9254e3c15 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.538+0000 7f90fbe00640 1 -- 172.21.15.26:0/2218851495 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4030610 con 0x7f90fc099a60 2024-01-15T18:02:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.538+0000 7f90fbe00640 1 -- 172.21.15.26:0/2218851495 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4030770 con 0x7f90fc099a60 2024-01-15T18:02:37.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.538+0000 7f90fbe00640 1 -- 172.21.15.26:0/2218851495 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4030b00 con 0x7f90fc099a60 2024-01-15T18:02:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.539+0000 7f9103e00640 1 -- 172.21.15.26:0/2218851495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc099a60 msgr2=0x7f90fc099ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.539+0000 7f9103e00640 1 --2- 172.21.15.26:0/2218851495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc099a60 0x7f90fc099ea0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0b72c0 tx=0x7f90e402b960 comp rx=0 tx=0).stop 2024-01-15T18:02:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.539+0000 7f9103e00640 1 -- 172.21.15.26:0/2218851495 shutdown_connections 2024-01-15T18:02:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.539+0000 7f9103e00640 1 --2- 172.21.15.26:0/2218851495 >> v2:172.21.15.124:3300/0 conn(0x7f90fc06e7c0 0x7f90fc06ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:37.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.539+0000 7f9103e00640 1 --2- 172.21.15.26:0/2218851495 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc099a60 0x7f90fc099ea0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.539+0000 7f9103e00640 1 -- 172.21.15.26:0/2218851495 >> 172.21.15.26:0/2218851495 conn(0x7f90fc0b2d00 msgr2=0x7f90fc0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.539+0000 7f9103e00640 1 -- 172.21.15.26:0/2218851495 shutdown_connections 2024-01-15T18:02:37.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.539+0000 7f9103e00640 1 -- 172.21.15.26:0/2218851495 wait complete. 2024-01-15T18:02:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.542+0000 7f9103e00640 1 Processor -- start 2024-01-15T18:02:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.542+0000 7f9103e00640 1 -- start start 2024-01-15T18:02:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.543+0000 7f9103e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc06e7c0 0x7f90fc0b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.543+0000 7f9103e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90fc099a60 0x7f90fc152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.543+0000 7f9103e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90fc15e880 0x7f90fc06ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.543+0000 7f9103e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc0b7110 con 0x7f90fc06e7c0 2024-01-15T18:02:37.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.543+0000 7f9103e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc1501a0 con 0x7f90fc099a60 2024-01-15T18:02:37.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.543+0000 7f9103e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc1533f0 con 0x7f90fc15e880 2024-01-15T18:02:37.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.544+0000 7f9101400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc06e7c0 0x7f90fc0b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:37.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.544+0000 7f9101400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc06e7c0 0x7f90fc0b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51258/0 (socket says 172.21.15.26:51258) 2024-01-15T18:02:37.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.544+0000 7f9101400640 1 -- 172.21.15.26:0/2729871591 learned_addr learned my addr 172.21.15.26:0/2729871591 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.545+0000 7f9101400640 1 -- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90fc15e880 msgr2=0x7f90fc06ba90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-15T18:02:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.545+0000 7f9101400640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90fc15e880 0x7f90fc06ba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.545+0000 7f9101400640 1 -- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90fc099a60 msgr2=0x7f90fc152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.545+0000 7f9101400640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90fc099a60 0x7f90fc152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.545+0000 7f9101400640 1 -- 172.21.15.26:0/2729871591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e4009400 con 0x7f90fc06e7c0 2024-01-15T18:02:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.545+0000 7f9101400640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc06e7c0 0x7f90fc0b6bd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00c460 tx=0x7f90ec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:37.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.546+0000 7f90f9600640 1 -- 172.21.15.26:0/2729871591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec0074e0 con 0x7f90fc06e7c0 2024-01-15T18:02:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.546+0000 7f90f9600640 1 -- 172.21.15.26:0/2729871591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec007640 con 0x7f90fc06e7c0 2024-01-15T18:02:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.546+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc14f590 con 0x7f90fc06e7c0 2024-01-15T18:02:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.546+0000 7f90f9600640 1 -- 172.21.15.26:0/2729871591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec004a20 con 0x7f90fc06e7c0 2024-01-15T18:02:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.546+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc14fb50 con 0x7f90fc06e7c0 2024-01-15T18:02:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.548+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90fc0675d0 con 0x7f90fc06e7c0 2024-01-15T18:02:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.551+0000 7f90f9600640 1 -- 172.21.15.26:0/2729871591 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f90ec004b80 con 0x7f90fc06e7c0 2024-01-15T18:02:37.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.555+0000 7f90f9600640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f90d406bb30 0x7f90d406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.555+0000 7f90f9600640 1 -- 172.21.15.26:0/2729871591 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f90ec087bb0 con 0x7f90fc06e7c0 2024-01-15T18:02:37.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.559+0000 7f90f9600640 1 -- 172.21.15.26:0/2729871591 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90ec0542a0 con 0x7f90fc06e7c0 2024-01-15T18:02:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.567+0000 7f9100a00640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f90d406bb30 0x7f90d406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:37.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.570+0000 7f9100a00640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f90d406bb30 0x7f90d406dff0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f90e4036000 tx=0x7f90e4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:37.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:37.953+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20832", "target": ["mon-mgr", ""]}) v1 -- 0x7f90fc0cf700 con 0x7f90d406bb30 2024-01-15T18:02:38.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.045+0000 7f90f9600640 1 -- 172.21.15.26:0/2729871591 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f90e403f040 con 0x7f90d406bb30 2024-01-15T18:02:38.050 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:38.050 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:38.050 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:02:38.050 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:38.050 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:38.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.049+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f90d406bb30 msgr2=0x7f90d406dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:38.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.049+0000 7f9103e00640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f90d406bb30 0x7f90d406dff0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f90e4036000 tx=0x7f90e4030000 comp rx=0 tx=0).stop 2024-01-15T18:02:38.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.049+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc06e7c0 msgr2=0x7f90fc0b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:38.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.049+0000 7f9103e00640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc06e7c0 0x7f90fc0b6bd0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00c460 tx=0x7f90ec00c920 comp rx=0 tx=0).stop 2024-01-15T18:02:38.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.050+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 shutdown_connections 2024-01-15T18:02:38.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.050+0000 7f9103e00640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90fc15e880 0x7f90fc06ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.050+0000 7f9103e00640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f90d406bb30 0x7f90d406dff0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.050+0000 7f9103e00640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90fc099a60 0x7f90fc152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.050+0000 7f9103e00640 1 --2- 172.21.15.26:0/2729871591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90fc06e7c0 0x7f90fc0b6bd0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.050+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 >> 172.21.15.26:0/2729871591 conn(0x7f90fc0b2d00 msgr2=0x7f90fc0cc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:38.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.051+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 shutdown_connections 2024-01-15T18:02:38.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.051+0000 7f9103e00640 1 -- 172.21.15.26:0/2729871591 wait complete. 2024-01-15T18:02:38.081 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:02:38.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20833 2024-01-15T18:02:38.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.380+0000 7f12a3a00640 1 Processor -- start 2024-01-15T18:02:38.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.380+0000 7f12a3a00640 1 -- start start 2024-01-15T18:02:38.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.382+0000 7f12a3a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f129c13e360 0x7f129c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:38.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.382+0000 7f12a3a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c15c5f0 con 0x7f129c15f9c0 2024-01-15T18:02:38.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.382+0000 7f12a3a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c06c470 con 0x7f129c13df80 2024-01-15T18:02:38.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.382+0000 7f12a3a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c06c750 con 0x7f129c13e360 2024-01-15T18:02:38.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.382+0000 7f1293e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f129c13e360 0x7f129c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:38.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.382+0000 7f1293e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f129c13e360 0x7f129c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51274/0 (socket says 172.21.15.26:51274) 2024-01-15T18:02:38.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.383+0000 7f1293e00640 1 -- 172.21.15.26:0/3152016749 learned_addr learned my addr 172.21.15.26:0/3152016749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:38.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.383+0000 7f1293e00640 1 -- 172.21.15.26:0/3152016749 >> v1:172.21.15.124:6789/0 conn(0x7f129c13df80 legacy=0x7f129c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:38.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.383+0000 7f1293e00640 1 -- 172.21.15.26:0/3152016749 >> v1:172.21.15.26:6789/0 conn(0x7f129c15f9c0 legacy=0x7f129c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:38.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.383+0000 7f1293e00640 1 -- 172.21.15.26:0/3152016749 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c06ca30 con 0x7f129c13e360 2024-01-15T18:02:38.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.385+0000 7f1293e00640 1 --2- 172.21.15.26:0/3152016749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 0x7f129c0cf290 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1284009870 tx=0x7f128402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7d8eaf9a52207d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:38.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.385+0000 7f129be00640 1 -- 172.21.15.26:0/3152016749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284030860 con 0x7f129c13e360 2024-01-15T18:02:38.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.386+0000 7f129be00640 1 -- 172.21.15.26:0/3152016749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12840309c0 con 0x7f129c13e360 2024-01-15T18:02:38.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.386+0000 7f129be00640 1 -- 172.21.15.26:0/3152016749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284030d50 con 0x7f129c13e360 2024-01-15T18:02:38.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.387+0000 7f12a3a00640 1 -- 172.21.15.26:0/3152016749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 msgr2=0x7f129c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.387+0000 7f12a3a00640 1 --2- 172.21.15.26:0/3152016749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 0x7f129c0cf290 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1284009870 tx=0x7f128402b9a0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.387+0000 7f12a3a00640 1 -- 172.21.15.26:0/3152016749 shutdown_connections 2024-01-15T18:02:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.387+0000 7f12a3a00640 1 --2- 172.21.15.26:0/3152016749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 0x7f129c0cf290 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.387+0000 7f12a3a00640 1 -- 172.21.15.26:0/3152016749 >> 172.21.15.26:0/3152016749 conn(0x7f129c06a800 msgr2=0x7f129c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.387+0000 7f12a3a00640 1 -- 172.21.15.26:0/3152016749 shutdown_connections 2024-01-15T18:02:38.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.387+0000 7f12a3a00640 1 -- 172.21.15.26:0/3152016749 wait complete. 2024-01-15T18:02:38.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.390+0000 7f12a3a00640 1 Processor -- start 2024-01-15T18:02:38.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.391+0000 7f12a3a00640 1 -- start start 2024-01-15T18:02:38.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.393+0000 7f12a3a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f129c13df80 0x7f129c13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:38.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.393+0000 7f12a3a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 0x7f129c1e5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:38.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.393+0000 7f12a3a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f129c15f9c0 0x7f129c15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:38.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.393+0000 7f12a3a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c13d720 con 0x7f129c13e360 2024-01-15T18:02:38.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.393+0000 7f12a3a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1f3780 con 0x7f129c13df80 2024-01-15T18:02:38.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.393+0000 7f12a3a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1f3a60 con 0x7f129c15f9c0 2024-01-15T18:02:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.394+0000 7f1293e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 0x7f129c1e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.394+0000 7f1293e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 0x7f129c1e5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51280/0 (socket says 172.21.15.26:51280) 2024-01-15T18:02:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.394+0000 7f1293e00640 1 -- 172.21.15.26:0/1366299880 learned_addr learned my addr 172.21.15.26:0/1366299880 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.394+0000 7f12a1000640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f129c13df80 0x7f129c13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:38.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.394+0000 7f12a1a00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f129c15f9c0 0x7f129c15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:38.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.394+0000 7f1293e00640 1 -- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f129c15f9c0 msgr2=0x7f129c15d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:38.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.395+0000 7f1293e00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f129c15f9c0 0x7f129c15d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.395+0000 7f1293e00640 1 -- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f129c13df80 msgr2=0x7f129c13d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:38.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.395+0000 7f1293e00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f129c13df80 0x7f129c13d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.395+0000 7f1293e00640 1 -- 172.21.15.26:0/1366299880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12840094d0 con 0x7f129c13e360 2024-01-15T18:02:38.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.397+0000 7f12a1a00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f129c15f9c0 0x7f129c15d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:38.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.397+0000 7f12a1000640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f129c13df80 0x7f129c13d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:02:38.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.399+0000 7f1293e00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 0x7f129c1e5510 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1284009870 tx=0x7f1284004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:38.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.399+0000 7f1299600640 1 -- 172.21.15.26:0/1366299880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284039e20 con 0x7f129c13e360 2024-01-15T18:02:38.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.399+0000 7f1299600640 1 -- 172.21.15.26:0/1366299880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1284043400 con 0x7f129c13e360 2024-01-15T18:02:38.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.399+0000 7f1299600640 1 -- 172.21.15.26:0/1366299880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12840437b0 con 0x7f129c13e360 2024-01-15T18:02:38.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.399+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c1f3d40 con 0x7f129c13e360 2024-01-15T18:02:38.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.399+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c1f4300 con 0x7f129c13e360 2024-01-15T18:02:38.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.401+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129c06b160 con 0x7f129c13e360 2024-01-15T18:02:38.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.404+0000 7f1299600640 1 -- 172.21.15.26:0/1366299880 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f1284043910 con 0x7f129c13e360 2024-01-15T18:02:38.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.410+0000 7f1299600640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f127406bb80 0x7f127406e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:38.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.410+0000 7f1299600640 1 -- 172.21.15.26:0/1366299880 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f128403e070 con 0x7f129c13e360 2024-01-15T18:02:38.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.412+0000 7f12a1000640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f127406bb80 0x7f127406e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:38.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.414+0000 7f1299600640 1 -- 172.21.15.26:0/1366299880 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1284078030 con 0x7f129c13e360 2024-01-15T18:02:38.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.415+0000 7f12a1000640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f127406bb80 0x7f127406e040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f128c006fd0 tx=0x7f128c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:38.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.830+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20833", "target": ["mon-mgr", ""]}) v1 -- 0x7f129c06cff0 con 0x7f127406bb80 2024-01-15T18:02:38.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.926+0000 7f1299600640 1 -- 172.21.15.26:0/1366299880 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f128c009e20 con 0x7f127406bb80 2024-01-15T18:02:38.930 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:38.930 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:38.930 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:02:38.930 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:38.931 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.930+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f127406bb80 msgr2=0x7f127406e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.930+0000 7f12a3a00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f127406bb80 0x7f127406e040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f128c006fd0 tx=0x7f128c008040 comp rx=0 tx=0).stop 2024-01-15T18:02:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.930+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 msgr2=0x7f129c1e5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.930+0000 7f12a3a00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 0x7f129c1e5510 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1284009870 tx=0x7f1284004730 comp rx=0 tx=0).stop 2024-01-15T18:02:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.930+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 shutdown_connections 2024-01-15T18:02:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.930+0000 7f12a3a00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f129c15f9c0 0x7f129c15d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.930+0000 7f12a3a00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f127406bb80 0x7f127406e040 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.930+0000 7f12a3a00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f129c13e360 0x7f129c1e5510 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.930+0000 7f12a3a00640 1 --2- 172.21.15.26:0/1366299880 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f129c13df80 0x7f129c13d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.931+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 >> 172.21.15.26:0/1366299880 conn(0x7f129c06a800 msgr2=0x7f129c15be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.931+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 shutdown_connections 2024-01-15T18:02:38.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:38.931+0000 7f12a3a00640 1 -- 172.21.15.26:0/1366299880 wait complete. 2024-01-15T18:02:38.961 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:02:38.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20834 2024-01-15T18:02:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.290+0000 7fb58f800640 1 Processor -- start 2024-01-15T18:02:39.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.291+0000 7fb58f800640 1 -- start start 2024-01-15T18:02:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.294+0000 7fb58f800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb588143ae0 0x7fb5880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.294+0000 7fb58f800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb58815e3b0 0x7fb58806c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.294+0000 7fb58f800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5880cfc30 con 0x7fb5880cf7d0 2024-01-15T18:02:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.294+0000 7fb58f800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5880b1ee0 con 0x7fb58815e3b0 2024-01-15T18:02:39.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.294+0000 7fb58f800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5880b2780 con 0x7fb588143ae0 2024-01-15T18:02:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.295+0000 7fb58d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb58815e3b0 0x7fb58806c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.295+0000 7fb58d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb58815e3b0 0x7fb58806c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51288/0 (socket says 172.21.15.26:51288) 2024-01-15T18:02:39.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.295+0000 7fb58d800640 1 -- 172.21.15.26:0/4190786657 learned_addr learned my addr 172.21.15.26:0/4190786657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.296+0000 7fb58d800640 1 -- 172.21.15.26:0/4190786657 >> v2:172.21.15.124:3300/0 conn(0x7fb588143ae0 msgr2=0x7fb5880cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.296+0000 7fb58d800640 1 --2- 172.21.15.26:0/4190786657 >> v2:172.21.15.124:3300/0 conn(0x7fb588143ae0 0x7fb5880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.296+0000 7fb58d800640 1 -- 172.21.15.26:0/4190786657 >> v1:172.21.15.124:6789/0 conn(0x7fb5880cf7d0 legacy=0x7fb58815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:39.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.296+0000 7fb58d800640 1 -- 172.21.15.26:0/4190786657 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58815e990 con 0x7fb58815e3b0 2024-01-15T18:02:39.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.298+0000 7fb58d800640 1 --2- 172.21.15.26:0/4190786657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb58815e3b0 0x7fb58806c470 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb5880cfea0 tx=0x7fb57802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a68b31095ea1e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.298+0000 7fb587400640 1 -- 172.21.15.26:0/4190786657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578031870 con 0x7fb58815e3b0 2024-01-15T18:02:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.299+0000 7fb587400640 1 -- 172.21.15.26:0/4190786657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5780319d0 con 0x7fb58815e3b0 2024-01-15T18:02:39.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.299+0000 7fb587400640 1 -- 172.21.15.26:0/4190786657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578031d60 con 0x7fb58815e3b0 2024-01-15T18:02:39.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.299+0000 7fb58f800640 1 -- 172.21.15.26:0/4190786657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb58815e3b0 msgr2=0x7fb58806c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.299+0000 7fb58f800640 1 --2- 172.21.15.26:0/4190786657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb58815e3b0 0x7fb58806c470 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fb5880cfea0 tx=0x7fb57802b910 comp rx=0 tx=0).stop 2024-01-15T18:02:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.300+0000 7fb58f800640 1 -- 172.21.15.26:0/4190786657 shutdown_connections 2024-01-15T18:02:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.300+0000 7fb58f800640 1 --2- 172.21.15.26:0/4190786657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb58815e3b0 0x7fb58806c470 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.300+0000 7fb58f800640 1 --2- 172.21.15.26:0/4190786657 >> v2:172.21.15.124:3300/0 conn(0x7fb588143ae0 0x7fb5880cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.300+0000 7fb58f800640 1 -- 172.21.15.26:0/4190786657 >> 172.21.15.26:0/4190786657 conn(0x7fb58806a800 msgr2=0x7fb58806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.300+0000 7fb58f800640 1 -- 172.21.15.26:0/4190786657 shutdown_connections 2024-01-15T18:02:39.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.300+0000 7fb58f800640 1 -- 172.21.15.26:0/4190786657 wait complete. 2024-01-15T18:02:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.302+0000 7fb58f800640 1 Processor -- start 2024-01-15T18:02:39.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.302+0000 7fb58f800640 1 -- start start 2024-01-15T18:02:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.303+0000 7fb58f800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5880cf7d0 0x7fb588159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.303+0000 7fb58f800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb588143ae0 0x7fb58813fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.303+0000 7fb58f800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb58815e3b0 0x7fb58813dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.303+0000 7fb58f800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58813dad0 con 0x7fb588143ae0 2024-01-15T18:02:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.303+0000 7fb58f800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880cd590 con 0x7fb58815e3b0 2024-01-15T18:02:39.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.303+0000 7fb58f800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880cc980 con 0x7fb5880cf7d0 2024-01-15T18:02:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.304+0000 7fb587e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb588143ae0 0x7fb58813fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.304+0000 7fb58ce00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5880cf7d0 0x7fb588159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:39.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.304+0000 7fb587e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb588143ae0 0x7fb58813fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51300/0 (socket says 172.21.15.26:51300) 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.304+0000 7fb587e00640 1 -- 172.21.15.26:0/223981091 learned_addr learned my addr 172.21.15.26:0/223981091 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.304+0000 7fb58d800640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb58815e3b0 0x7fb58813dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb587e00640 1 -- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5880cf7d0 msgr2=0x7fb588159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb587e00640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5880cf7d0 0x7fb588159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb587e00640 1 -- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb58815e3b0 msgr2=0x7fb58813dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb587e00640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb58815e3b0 0x7fb58813dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb587e00640 1 -- 172.21.15.26:0/223981091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5780094d0 con 0x7fb588143ae0 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb587e00640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb588143ae0 0x7fb58813fc80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c410 tx=0x7fb57000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb584c00640 1 -- 172.21.15.26:0/223981091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5700035f0 con 0x7fb588143ae0 2024-01-15T18:02:39.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb584c00640 1 -- 172.21.15.26:0/223981091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb570003750 con 0x7fb588143ae0 2024-01-15T18:02:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb584c00640 1 -- 172.21.15.26:0/223981091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570003b40 con 0x7fb588143ae0 2024-01-15T18:02:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.305+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5880ccc60 con 0x7fb588143ae0 2024-01-15T18:02:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.306+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5880cd220 con 0x7fb588143ae0 2024-01-15T18:02:39.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.307+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5880675d0 con 0x7fb588143ae0 2024-01-15T18:02:39.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.309+0000 7fb584c00640 1 -- 172.21.15.26:0/223981091 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7fb570003ca0 con 0x7fb588143ae0 2024-01-15T18:02:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.310+0000 7fb584c00640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb56006bb60 0x7fb56006e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:39.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.310+0000 7fb584c00640 1 -- 172.21.15.26:0/223981091 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb570087680 con 0x7fb588143ae0 2024-01-15T18:02:39.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.315+0000 7fb58ce00640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb56006bb60 0x7fb56006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:39.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.318+0000 7fb584c00640 1 -- 172.21.15.26:0/223981091 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb570053e80 con 0x7fb588143ae0 2024-01-15T18:02:39.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.324+0000 7fb58ce00640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb56006bb60 0x7fb56006e020 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb58000a400 tx=0x7fb58000d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:39.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.711+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20834", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5880b2410 con 0x7fb56006bb60 2024-01-15T18:02:39.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.876+0000 7fb584c00640 1 -- 172.21.15.26:0/223981091 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fb580009c30 con 0x7fb56006bb60 2024-01-15T18:02:39.880 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:39.881 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:39.881 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:02:39.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:39.881 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.881+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb56006bb60 msgr2=0x7fb56006e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.881+0000 7fb58f800640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb56006bb60 0x7fb56006e020 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb58000a400 tx=0x7fb58000d040 comp rx=0 tx=0).stop 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.881+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb588143ae0 msgr2=0x7fb58813fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.881+0000 7fb58f800640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb588143ae0 0x7fb58813fc80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c410 tx=0x7fb57000c8d0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.882+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 shutdown_connections 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.882+0000 7fb58f800640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb56006bb60 0x7fb56006e020 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.882+0000 7fb58f800640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb58815e3b0 0x7fb58813dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.882+0000 7fb58f800640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb588143ae0 0x7fb58813fc80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.882+0000 7fb58f800640 1 --2- 172.21.15.26:0/223981091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5880cf7d0 0x7fb588159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:39.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.882+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 >> 172.21.15.26:0/223981091 conn(0x7fb58806a800 msgr2=0x7fb58806ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.882+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 shutdown_connections 2024-01-15T18:02:39.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:39.882+0000 7fb58f800640 1 -- 172.21.15.26:0/223981091 wait complete. 2024-01-15T18:02:39.908 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:02:39.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20835 2024-01-15T18:02:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.200+0000 7f754be00640 1 Processor -- start 2024-01-15T18:02:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.200+0000 7f754be00640 1 -- start start 2024-01-15T18:02:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.201+0000 7f754be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f754c0cf9a0 0x7f754c0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.201+0000 7f754be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c0b31d0 0x7f754c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.201+0000 7f754be00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f754c0d0320 con 0x7f754c15d4b0 2024-01-15T18:02:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.201+0000 7f754be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754c0d1c60 con 0x7f754c0cf9a0 2024-01-15T18:02:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.201+0000 7f754be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c0b2730 con 0x7f754c0b31d0 2024-01-15T18:02:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.202+0000 7f754b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c0b31d0 0x7f754c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.202+0000 7f7543e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f754c0cf9a0 0x7f754c0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.202+0000 7f754b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c0b31d0 0x7f754c0b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:43642/0 (socket says 172.21.15.26:43642) 2024-01-15T18:02:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.202+0000 7f7543e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f754c0cf9a0 0x7f754c0cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:49052/0 (socket says 172.21.15.26:49052) 2024-01-15T18:02:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.202+0000 7f754b400640 1 -- 172.21.15.26:0/1329293907 learned_addr learned my addr 172.21.15.26:0/1329293907 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.203+0000 7f7543e00640 1 -- 172.21.15.26:0/1329293907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c0b31d0 msgr2=0x7f754c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.203+0000 7f7543e00640 1 --2- 172.21.15.26:0/1329293907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c0b31d0 0x7f754c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.203+0000 7f7543e00640 1 -- 172.21.15.26:0/1329293907 >> v1:172.21.15.124:6789/0 conn(0x7f754c15d4b0 legacy=0x7f754c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.203+0000 7f7543e00640 1 -- 172.21.15.26:0/1329293907 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c0d1ec0 con 0x7f754c0cf9a0 2024-01-15T18:02:40.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.205+0000 7f7543e00640 1 --2- 172.21.15.26:0/1329293907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f754c0cf9a0 0x7f754c0cfde0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f75340097a0 tx=0x7f753402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf52798b220ca854 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:40.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.205+0000 7f754a000640 1 -- 172.21.15.26:0/1329293907 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753402d040 con 0x7f754c0cf9a0 2024-01-15T18:02:40.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.205+0000 7f754a000640 1 -- 172.21.15.26:0/1329293907 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7534031810 con 0x7f754c0cf9a0 2024-01-15T18:02:40.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.205+0000 7f754a000640 1 -- 172.21.15.26:0/1329293907 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534031ba0 con 0x7f754c0cf9a0 2024-01-15T18:02:40.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.206+0000 7f754be00640 1 -- 172.21.15.26:0/1329293907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f754c0cf9a0 msgr2=0x7f754c0cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:40.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.206+0000 7f754be00640 1 --2- 172.21.15.26:0/1329293907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f754c0cf9a0 0x7f754c0cfde0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f75340097a0 tx=0x7f753402b8b0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.206+0000 7f754be00640 1 -- 172.21.15.26:0/1329293907 shutdown_connections 2024-01-15T18:02:40.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.206+0000 7f754be00640 1 --2- 172.21.15.26:0/1329293907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c0b31d0 0x7f754c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.206+0000 7f754be00640 1 --2- 172.21.15.26:0/1329293907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f754c0cf9a0 0x7f754c0cfde0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.206+0000 7f754be00640 1 -- 172.21.15.26:0/1329293907 >> 172.21.15.26:0/1329293907 conn(0x7f754c0d3000 msgr2=0x7f754c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:40.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.206+0000 7f754be00640 1 -- 172.21.15.26:0/1329293907 shutdown_connections 2024-01-15T18:02:40.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.206+0000 7f754be00640 1 -- 172.21.15.26:0/1329293907 wait complete. 2024-01-15T18:02:40.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.208+0000 7f754be00640 1 Processor -- start 2024-01-15T18:02:40.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.208+0000 7f754be00640 1 -- start start 2024-01-15T18:02:40.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.209+0000 7f754be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c0b31d0 0x7f754c0cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:40.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.210+0000 7f754be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f754c0cf9a0 0x7f754c14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.210+0000 7f754be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c15d4b0 0x7f754c0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.210+0000 7f754be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c09f3d0 con 0x7f754c0b31d0 2024-01-15T18:02:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.210+0000 7f754be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f754c09fc10 con 0x7f754c0cf9a0 2024-01-15T18:02:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.210+0000 7f754be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754c09f600 con 0x7f754c15d4b0 2024-01-15T18:02:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.210+0000 7f754aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c0b31d0 0x7f754c0cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.210+0000 7f754aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c0b31d0 0x7f754c0cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51314/0 (socket says 172.21.15.26:51314) 2024-01-15T18:02:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.210+0000 7f754aa00640 1 -- 172.21.15.26:0/289962290 learned_addr learned my addr 172.21.15.26:0/289962290 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:40.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.210+0000 7f7543e00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f754c0cf9a0 0x7f754c14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f754aa00640 1 -- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c15d4b0 msgr2=0x7f754c0b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f754aa00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c15d4b0 0x7f754c0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f754aa00640 1 -- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f754c0cf9a0 msgr2=0x7f754c14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f754aa00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f754c0cf9a0 0x7f754c14d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f754aa00640 1 -- 172.21.15.26:0/289962290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534009400 con 0x7f754c0b31d0 2024-01-15T18:02:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f754aa00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c0b31d0 0x7f754c0cb2d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f753c00c400 tx=0x7f753c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f7542a00640 1 -- 172.21.15.26:0/289962290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c0052d0 con 0x7f754c0b31d0 2024-01-15T18:02:40.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f7542a00640 1 -- 172.21.15.26:0/289962290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753c005450 con 0x7f754c0b31d0 2024-01-15T18:02:40.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f754c0cb070 con 0x7f754c0b31d0 2024-01-15T18:02:40.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.211+0000 7f7542a00640 1 -- 172.21.15.26:0/289962290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753c007260 con 0x7f754c0b31d0 2024-01-15T18:02:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.212+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f754c09f8a0 con 0x7f754c0b31d0 2024-01-15T18:02:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.213+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f754c067450 con 0x7f754c0b31d0 2024-01-15T18:02:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.214+0000 7f7542a00640 1 -- 172.21.15.26:0/289962290 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f753c0073c0 con 0x7f754c0b31d0 2024-01-15T18:02:40.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.215+0000 7f7542a00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f752406bb30 0x7f752406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.216+0000 7f7542a00640 1 -- 172.21.15.26:0/289962290 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f753c087a10 con 0x7f754c0b31d0 2024-01-15T18:02:40.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.216+0000 7f7543e00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f752406bb30 0x7f752406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:40.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.218+0000 7f7543e00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f752406bb30 0x7f752406dff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7534036000 tx=0x7f753403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:40.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.220+0000 7f7542a00640 1 -- 172.21.15.26:0/289962290 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f753c054100 con 0x7f754c0b31d0 2024-01-15T18:02:40.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.572+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20835", "target": ["mon-mgr", ""]}) v1 -- 0x7f754c0d08e0 con 0x7f752406bb30 2024-01-15T18:02:40.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.586+0000 7f7542a00640 1 -- 172.21.15.26:0/289962290 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f754c0d08e0 con 0x7f752406bb30 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:02:40.590 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:02:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-15T18:02:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:02:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:02:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:40.591 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:40.591 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:40.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.589+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f752406bb30 msgr2=0x7f752406dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:40.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.589+0000 7f754be00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f752406bb30 0x7f752406dff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7534036000 tx=0x7f753403c040 comp rx=0 tx=0).stop 2024-01-15T18:02:40.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.589+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c0b31d0 msgr2=0x7f754c0cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:40.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.589+0000 7f754be00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c0b31d0 0x7f754c0cb2d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f753c00c400 tx=0x7f753c00c8c0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.590+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 shutdown_connections 2024-01-15T18:02:40.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.590+0000 7f754be00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f754c15d4b0 0x7f754c0b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.590+0000 7f754be00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f752406bb30 0x7f752406dff0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.590+0000 7f754be00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f754c0cf9a0 0x7f754c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.590+0000 7f754be00640 1 --2- 172.21.15.26:0/289962290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f754c0b31d0 0x7f754c0cb2d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:40.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.590+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 >> 172.21.15.26:0/289962290 conn(0x7f754c0d3000 msgr2=0x7f754c0d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:40.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.591+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 shutdown_connections 2024-01-15T18:02:40.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:40.591+0000 7f754be00640 1 -- 172.21.15.26:0/289962290 wait complete. 2024-01-15T18:02:40.617 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:02:41.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20835 2024-01-15T18:02:41.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.890+0000 7fad60800640 1 Processor -- start 2024-01-15T18:02:41.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.890+0000 7fad60800640 1 -- start start 2024-01-15T18:02:41.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad60800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad5813df80 0x7fad5815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:41.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad60800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fad5813e360 0x7fad580cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:41.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad60800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5806c470 con 0x7fad5815f9c0 2024-01-15T18:02:41.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad60800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5806c750 con 0x7fad5813df80 2024-01-15T18:02:41.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad60800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5806ca30 con 0x7fad5813e360 2024-01-15T18:02:41.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad5de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad5813df80 0x7fad5815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:41.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad5de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad5813df80 0x7fad5815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51318/0 (socket says 172.21.15.26:51318) 2024-01-15T18:02:41.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad5de00640 1 -- 172.21.15.26:0/2870974126 learned_addr learned my addr 172.21.15.26:0/2870974126 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:41.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad5de00640 1 -- 172.21.15.26:0/2870974126 >> v2:172.21.15.124:3300/0 conn(0x7fad5813e360 msgr2=0x7fad580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:41.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.892+0000 7fad5de00640 1 --2- 172.21.15.26:0/2870974126 >> v2:172.21.15.124:3300/0 conn(0x7fad5813e360 0x7fad580cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:41.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.893+0000 7fad5de00640 1 -- 172.21.15.26:0/2870974126 >> v1:172.21.15.124:6789/0 conn(0x7fad5815f9c0 legacy=0x7fad581408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:41.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.893+0000 7fad5de00640 1 -- 172.21.15.26:0/2870974126 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5806cd10 con 0x7fad5813df80 2024-01-15T18:02:41.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.894+0000 7fad5de00640 1 --2- 172.21.15.26:0/2870974126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813df80 0x7fad5815bee0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fad5806ded0 tx=0x7fad4802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f275e52eea5f176d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:41.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.894+0000 7fad5ca00640 1 -- 172.21.15.26:0/2870974126 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48031890 con 0x7fad5813df80 2024-01-15T18:02:41.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.895+0000 7fad5ca00640 1 -- 172.21.15.26:0/2870974126 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad480319f0 con 0x7fad5813df80 2024-01-15T18:02:41.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.895+0000 7fad5ca00640 1 -- 172.21.15.26:0/2870974126 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad48031d80 con 0x7fad5813df80 2024-01-15T18:02:41.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.895+0000 7fad60800640 1 -- 172.21.15.26:0/2870974126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813df80 msgr2=0x7fad5815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:41.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.895+0000 7fad60800640 1 --2- 172.21.15.26:0/2870974126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813df80 0x7fad5815bee0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fad5806ded0 tx=0x7fad4802b930 comp rx=0 tx=0).stop 2024-01-15T18:02:41.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.896+0000 7fad60800640 1 -- 172.21.15.26:0/2870974126 shutdown_connections 2024-01-15T18:02:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.896+0000 7fad60800640 1 --2- 172.21.15.26:0/2870974126 >> v2:172.21.15.124:3300/0 conn(0x7fad5813e360 0x7fad580cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.896+0000 7fad60800640 1 --2- 172.21.15.26:0/2870974126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813df80 0x7fad5815bee0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.896+0000 7fad60800640 1 -- 172.21.15.26:0/2870974126 >> 172.21.15.26:0/2870974126 conn(0x7fad5806a800 msgr2=0x7fad5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.896+0000 7fad60800640 1 -- 172.21.15.26:0/2870974126 shutdown_connections 2024-01-15T18:02:41.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.896+0000 7fad60800640 1 -- 172.21.15.26:0/2870974126 wait complete. 2024-01-15T18:02:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.899+0000 7fad60800640 1 Processor -- start 2024-01-15T18:02:41.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.900+0000 7fad60800640 1 -- start start 2024-01-15T18:02:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.902+0000 7fad60800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5813df80 0x7fad581f0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.902+0000 7fad60800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813e360 0x7fad5814f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.902+0000 7fad60800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad5815f9c0 0x7fad5814f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.902+0000 7fad60800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5814f040 con 0x7fad5813e360 2024-01-15T18:02:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.902+0000 7fad60800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5814e430 con 0x7fad5813df80 2024-01-15T18:02:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.902+0000 7fad60800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5814e710 con 0x7fad5815f9c0 2024-01-15T18:02:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.903+0000 7fad5d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813e360 0x7fad5814f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:41.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.904+0000 7fad5e800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad5815f9c0 0x7fad5814f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.904+0000 7fad5de00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5813df80 0x7fad581f0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.904+0000 7fad5d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813e360 0x7fad5814f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51334/0 (socket says 172.21.15.26:51334) 2024-01-15T18:02:41.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.904+0000 7fad5d400640 1 -- 172.21.15.26:0/2481963007 learned_addr learned my addr 172.21.15.26:0/2481963007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.905+0000 7fad5d400640 1 -- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad5815f9c0 msgr2=0x7fad5814f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.905+0000 7fad5d400640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad5815f9c0 0x7fad5814f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.905+0000 7fad5d400640 1 -- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5813df80 msgr2=0x7fad581f0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.905+0000 7fad5d400640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5813df80 0x7fad581f0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:41.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.905+0000 7fad5d400640 1 -- 172.21.15.26:0/2481963007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad480094d0 con 0x7fad5813e360 2024-01-15T18:02:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.905+0000 7fad5d400640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813e360 0x7fad5814f1a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fad4000b450 tx=0x7fad4000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.906+0000 7fad56000640 1 -- 172.21.15.26:0/2481963007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40011890 con 0x7fad5813e360 2024-01-15T18:02:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.906+0000 7fad56000640 1 -- 172.21.15.26:0/2481963007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad400119f0 con 0x7fad5813e360 2024-01-15T18:02:41.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.906+0000 7fad56000640 1 -- 172.21.15.26:0/2481963007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40011d80 con 0x7fad5813e360 2024-01-15T18:02:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.906+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5814e9f0 con 0x7fad5813e360 2024-01-15T18:02:41.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.907+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad581f0ec0 con 0x7fad5813e360 2024-01-15T18:02:41.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.908+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5814aa20 con 0x7fad5813e360 2024-01-15T18:02:41.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.911+0000 7fad56000640 1 -- 172.21.15.26:0/2481963007 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7fad4001a400 con 0x7fad5813e360 2024-01-15T18:02:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.912+0000 7fad56000640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad3006bb60 0x7fad3006e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:41.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.913+0000 7fad56000640 1 -- 172.21.15.26:0/2481963007 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fad400888f0 con 0x7fad5813e360 2024-01-15T18:02:41.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.913+0000 7fad5de00640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad3006bb60 0x7fad3006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:41.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.915+0000 7fad5de00640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad3006bb60 0x7fad3006e020 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fad48036000 tx=0x7fad4803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:41.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:41.919+0000 7fad56000640 1 -- 172.21.15.26:0/2481963007 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad5814aa20 con 0x7fad5813e360 2024-01-15T18:02:42.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.341+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20835", "target": ["mon-mgr", ""]}) v1 -- 0x7fad5814e880 con 0x7fad3006bb60 2024-01-15T18:02:42.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.371+0000 7fad56000640 1 -- 172.21.15.26:0/2481963007 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fad5814e880 con 0x7fad3006bb60 2024-01-15T18:02:42.375 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:42.375 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:42.375 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:02:42.375 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:02:42.375 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:02:42.375 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:02:42.375 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:02:42.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:02:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:02:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:02:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-15T18:02:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:02:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:02:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:42.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:42.376 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.373+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad3006bb60 msgr2=0x7fad3006e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.373+0000 7fad60800640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad3006bb60 0x7fad3006e020 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fad48036000 tx=0x7fad4803c040 comp rx=0 tx=0).stop 2024-01-15T18:02:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813e360 msgr2=0x7fad5814f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813e360 0x7fad5814f1a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fad4000b450 tx=0x7fad4000b910 comp rx=0 tx=0).stop 2024-01-15T18:02:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 shutdown_connections 2024-01-15T18:02:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad5815f9c0 0x7fad5814f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad3006bb60 0x7fad3006e020 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:42.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad5813e360 0x7fad5814f1a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 --2- 172.21.15.26:0/2481963007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad5813df80 0x7fad581f0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 >> 172.21.15.26:0/2481963007 conn(0x7fad5806a800 msgr2=0x7fad5815df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 shutdown_connections 2024-01-15T18:02:42.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:42.374+0000 7fad60800640 1 -- 172.21.15.26:0/2481963007 wait complete. 2024-01-15T18:02:42.398 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:02:43.400 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20835 2024-01-15T18:02:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.736+0000 7f041a800640 1 Processor -- start 2024-01-15T18:02:43.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.736+0000 7f041a800640 1 -- start start 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.737+0000 7f041a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0414143ae0 0x7f04140cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.737+0000 7f041a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f041415e3b0 0x7f04140702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.737+0000 7f041a800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04140cfc30 con 0x7f04140cf7d0 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.737+0000 7f041a800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140b1ee0 con 0x7f0414143ae0 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.737+0000 7f041a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140b2780 con 0x7f041415e3b0 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.738+0000 7f0413400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0414143ae0 0x7f04140cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.738+0000 7f0413400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0414143ae0 0x7f04140cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51746/0 (socket says 172.21.15.26:51746) 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.738+0000 7f0413400640 1 -- 172.21.15.26:0/556445485 learned_addr learned my addr 172.21.15.26:0/556445485 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.738+0000 7f040be00640 1 --1- 172.21.15.26:0/556445485 >> v1:172.21.15.26:6789/0 conn(0x7f04140cf7d0 0x7f041415dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51464/0 (socket says 172.21.15.26:51464) 2024-01-15T18:02:43.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.738+0000 7f0413400640 1 -- 172.21.15.26:0/556445485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f041415e3b0 msgr2=0x7f04140702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:43.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.738+0000 7f0413400640 1 --2- 172.21.15.26:0/556445485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f041415e3b0 0x7f04140702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:43.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.738+0000 7f0413400640 1 -- 172.21.15.26:0/556445485 >> v1:172.21.15.26:6789/0 conn(0x7f04140cf7d0 legacy=0x7f041415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:43.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.738+0000 7f0413400640 1 -- 172.21.15.26:0/556445485 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041415e990 con 0x7f0414143ae0 2024-01-15T18:02:43.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.738+0000 7f0412a00640 1 -- 172.21.15.26:0/556445485 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491460963 0 0) 0x7f04140cfc30 con 0x7f04140cf7d0 2024-01-15T18:02:43.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.740+0000 7f0413400640 1 --2- 172.21.15.26:0/556445485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 0x7f04140cf290 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f04040098a0 tx=0x7f040402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd207548c78fcb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.740+0000 7f0412a00640 1 -- 172.21.15.26:0/556445485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404030840 con 0x7f0414143ae0 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.740+0000 7f0412a00640 1 -- 172.21.15.26:0/556445485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04040309a0 con 0x7f0414143ae0 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.740+0000 7f0412a00640 1 -- 172.21.15.26:0/556445485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404030d30 con 0x7f0414143ae0 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.740+0000 7f041a800640 1 -- 172.21.15.26:0/556445485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 msgr2=0x7f04140cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.740+0000 7f041a800640 1 --2- 172.21.15.26:0/556445485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 0x7f04140cf290 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f04040098a0 tx=0x7f040402b980 comp rx=0 tx=0).stop 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.741+0000 7f041a800640 1 -- 172.21.15.26:0/556445485 shutdown_connections 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.741+0000 7f041a800640 1 --2- 172.21.15.26:0/556445485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f041415e3b0 0x7f04140702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.741+0000 7f041a800640 1 --2- 172.21.15.26:0/556445485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 0x7f04140cf290 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.741+0000 7f041a800640 1 -- 172.21.15.26:0/556445485 >> 172.21.15.26:0/556445485 conn(0x7f041406a800 msgr2=0x7f041406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.741+0000 7f041a800640 1 -- 172.21.15.26:0/556445485 shutdown_connections 2024-01-15T18:02:43.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.741+0000 7f041a800640 1 -- 172.21.15.26:0/556445485 wait complete. 2024-01-15T18:02:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.743+0000 7f041a800640 1 Processor -- start 2024-01-15T18:02:43.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.743+0000 7f041a800640 1 -- start start 2024-01-15T18:02:43.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f041a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04140cf7d0 0x7f0414139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:43.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f041a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 0x7f041413b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:43.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f041a800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f041415e3b0 0x7f0414139580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:43.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f041a800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041413b260 con 0x7f0414143ae0 2024-01-15T18:02:43.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f041a800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041413a380 con 0x7f041415e3b0 2024-01-15T18:02:43.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f041a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041413bc30 con 0x7f04140cf7d0 2024-01-15T18:02:43.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f040be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 0x7f041413b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:43.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f040be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 0x7f041413b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51758/0 (socket says 172.21.15.26:51758) 2024-01-15T18:02:43.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f040be00640 1 -- 172.21.15.26:0/4054342971 learned_addr learned my addr 172.21.15.26:0/4054342971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:43.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f040be00640 1 -- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04140cf7d0 msgr2=0x7f0414139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f040be00640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04140cf7d0 0x7f0414139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f040be00640 1 -- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f041415e3b0 msgr2=0x7f0414139580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.744+0000 7f040be00640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f041415e3b0 0x7f0414139580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.745+0000 7f040be00640 1 -- 172.21.15.26:0/4054342971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04040094d0 con 0x7f0414143ae0 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.745+0000 7f040be00640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 0x7f041413b6f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00c4d0 tx=0x7f03fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.745+0000 7f040b400640 1 -- 172.21.15.26:0/4054342971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0074e0 con 0x7f0414143ae0 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.745+0000 7f040b400640 1 -- 172.21.15.26:0/4054342971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc007660 con 0x7f0414143ae0 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.745+0000 7f040b400640 1 -- 172.21.15.26:0/4054342971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc004a20 con 0x7f0414143ae0 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.745+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041413a650 con 0x7f0414143ae0 2024-01-15T18:02:43.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.745+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041413ac10 con 0x7f0414143ae0 2024-01-15T18:02:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.747+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04140675d0 con 0x7f0414143ae0 2024-01-15T18:02:43.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.748+0000 7f040b400640 1 -- 172.21.15.26:0/4054342971 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f03fc004b80 con 0x7f0414143ae0 2024-01-15T18:02:43.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.749+0000 7f040b400640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f03ec06bb30 0x7f03ec06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:43.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.750+0000 7f040b400640 1 -- 172.21.15.26:0/4054342971 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f03fc087790 con 0x7f0414143ae0 2024-01-15T18:02:43.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.751+0000 7f0413400640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f03ec06bb30 0x7f03ec06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:43.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.753+0000 7f0413400640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f03ec06bb30 0x7f03ec06dff0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0404036000 tx=0x7f040403b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:43.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:43.754+0000 7f040b400640 1 -- 172.21.15.26:0/4054342971 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03fc053e80 con 0x7f0414143ae0 2024-01-15T18:02:44.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.161+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20835", "target": ["mon-mgr", ""]}) v1 -- 0x7f04140b2410 con 0x7f03ec06bb30 2024-01-15T18:02:44.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.246+0000 7f040b400640 1 -- 172.21.15.26:0/4054342971 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f0404030b90 con 0x7f03ec06bb30 2024-01-15T18:02:44.250 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:44.250 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:44.250 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:02:44.250 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:02:44.251 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:02:44.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:44.252 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:44.252 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:44.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f03ec06bb30 msgr2=0x7f03ec06dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:44.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f03ec06bb30 0x7f03ec06dff0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0404036000 tx=0x7f040403b040 comp rx=0 tx=0).stop 2024-01-15T18:02:44.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 msgr2=0x7f041413b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:44.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 0x7f041413b6f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f03fc00c4d0 tx=0x7f03fc00c990 comp rx=0 tx=0).stop 2024-01-15T18:02:44.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 shutdown_connections 2024-01-15T18:02:44.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f03ec06bb30 0x7f03ec06dff0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:44.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f041415e3b0 0x7f0414139580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:44.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0414143ae0 0x7f041413b6f0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:44.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 --2- 172.21.15.26:0/4054342971 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04140cf7d0 0x7f0414139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:44.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 >> 172.21.15.26:0/4054342971 conn(0x7f041406a800 msgr2=0x7f04140cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:44.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 shutdown_connections 2024-01-15T18:02:44.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:44.250+0000 7f041a800640 1 -- 172.21.15.26:0/4054342971 wait complete. 2024-01-15T18:02:44.292 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:02:45.293 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20835 2024-01-15T18:02:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.645+0000 7ffb75600640 1 Processor -- start 2024-01-15T18:02:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.645+0000 7ffb75600640 1 -- start start 2024-01-15T18:02:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.646+0000 7ffb75600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb70160f80 0x7ffb70161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.646+0000 7ffb75600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb700ced50 0x7ffb700d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.646+0000 7ffb75600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb700cf1b0 con 0x7ffb700ce8f0 2024-01-15T18:02:45.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.646+0000 7ffb75600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb701618a0 con 0x7ffb700ced50 2024-01-15T18:02:45.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.646+0000 7ffb75600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700d0de0 con 0x7ffb70160f80 2024-01-15T18:02:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.647+0000 7ffb6ea00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb70160f80 0x7ffb70161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.648+0000 7ffb6f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffb700ced50 0x7ffb700d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.648+0000 7ffb6ea00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb70160f80 0x7ffb70161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:42908/0 (socket says 172.21.15.26:42908) 2024-01-15T18:02:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.648+0000 7ffb6ea00640 1 -- 172.21.15.26:0/4245964098 learned_addr learned my addr 172.21.15.26:0/4245964098 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.648+0000 7ffb6f400640 1 -- 172.21.15.26:0/4245964098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb70160f80 msgr2=0x7ffb70161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.648+0000 7ffb6f400640 1 --2- 172.21.15.26:0/4245964098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb70160f80 0x7ffb70161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:45.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.648+0000 7ffb6f400640 1 -- 172.21.15.26:0/4245964098 >> v1:172.21.15.124:6789/0 conn(0x7ffb700ce8f0 legacy=0x7ffb700cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.648+0000 7ffb6f400640 1 -- 172.21.15.26:0/4245964098 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb700d10c0 con 0x7ffb700ced50 2024-01-15T18:02:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.650+0000 7ffb6f400640 1 --2- 172.21.15.26:0/4245964098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb700ced50 0x7ffb700d02f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009d90 tx=0x7ffb6802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397150d4171e9780 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.651+0000 7ffb6e000640 1 -- 172.21.15.26:0/4245964098 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6802cbd0 con 0x7ffb700ced50 2024-01-15T18:02:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.651+0000 7ffb6e000640 1 -- 172.21.15.26:0/4245964098 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6802cd30 con 0x7ffb700ced50 2024-01-15T18:02:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.651+0000 7ffb6e000640 1 -- 172.21.15.26:0/4245964098 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6802cea0 con 0x7ffb700ced50 2024-01-15T18:02:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.651+0000 7ffb75600640 1 -- 172.21.15.26:0/4245964098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb700ced50 msgr2=0x7ffb700d02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.651+0000 7ffb75600640 1 --2- 172.21.15.26:0/4245964098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb700ced50 0x7ffb700d02f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009d90 tx=0x7ffb6802be20 comp rx=0 tx=0).stop 2024-01-15T18:02:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.652+0000 7ffb75600640 1 -- 172.21.15.26:0/4245964098 shutdown_connections 2024-01-15T18:02:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.652+0000 7ffb75600640 1 --2- 172.21.15.26:0/4245964098 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb700ced50 0x7ffb700d02f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.652+0000 7ffb75600640 1 --2- 172.21.15.26:0/4245964098 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb70160f80 0x7ffb70161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.652+0000 7ffb75600640 1 -- 172.21.15.26:0/4245964098 >> 172.21.15.26:0/4245964098 conn(0x7ffb7009c550 msgr2=0x7ffb700d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.652+0000 7ffb75600640 1 -- 172.21.15.26:0/4245964098 shutdown_connections 2024-01-15T18:02:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.652+0000 7ffb75600640 1 -- 172.21.15.26:0/4245964098 wait complete. 2024-01-15T18:02:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.654+0000 7ffb75600640 1 Processor -- start 2024-01-15T18:02:45.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.655+0000 7ffb75600640 1 -- start start 2024-01-15T18:02:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.656+0000 7ffb75600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb700ce8f0 0x7ffb7015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.656+0000 7ffb75600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb700ced50 0x7ffb701f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.656+0000 7ffb75600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb70160f80 0x7ffb701f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.656+0000 7ffb75600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700b7310 con 0x7ffb70160f80 2024-01-15T18:02:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.656+0000 7ffb75600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700b77a0 con 0x7ffb700ce8f0 2024-01-15T18:02:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.656+0000 7ffb75600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7015c390 con 0x7ffb700ced50 2024-01-15T18:02:45.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.657+0000 7ffb6f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb70160f80 0x7ffb701f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.657+0000 7ffb6ea00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb700ce8f0 0x7ffb7015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.657+0000 7ffb6f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb70160f80 0x7ffb701f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51772/0 (socket says 172.21.15.26:51772) 2024-01-15T18:02:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.657+0000 7ffb6f400640 1 -- 172.21.15.26:0/2738927166 learned_addr learned my addr 172.21.15.26:0/2738927166 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.658+0000 7ffb6f400640 1 -- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb700ced50 msgr2=0x7ffb701f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.658+0000 7ffb6f400640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb700ced50 0x7ffb701f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.658+0000 7ffb6f400640 1 -- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb700ce8f0 msgr2=0x7ffb7015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:45.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.658+0000 7ffb6f400640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb700ce8f0 0x7ffb7015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:45.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.658+0000 7ffb6f400640 1 -- 172.21.15.26:0/2738927166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb68009a70 con 0x7ffb70160f80 2024-01-15T18:02:45.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.658+0000 7ffb6f400640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb70160f80 0x7ffb701f6ea0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ffb6802c430 tx=0x7ffb68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:45.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.659+0000 7ffb66a00640 1 -- 172.21.15.26:0/2738927166 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6802cea0 con 0x7ffb70160f80 2024-01-15T18:02:45.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.659+0000 7ffb66a00640 1 -- 172.21.15.26:0/2738927166 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6803e630 con 0x7ffb70160f80 2024-01-15T18:02:45.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.659+0000 7ffb66a00640 1 -- 172.21.15.26:0/2738927166 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6803e9e0 con 0x7ffb70160f80 2024-01-15T18:02:45.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.660+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb700b69e0 con 0x7ffb70160f80 2024-01-15T18:02:45.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.660+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb700b6fa0 con 0x7ffb70160f80 2024-01-15T18:02:45.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.662+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb700b7480 con 0x7ffb70160f80 2024-01-15T18:02:45.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.665+0000 7ffb66a00640 1 -- 172.21.15.26:0/2738927166 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7ffb6803b050 con 0x7ffb70160f80 2024-01-15T18:02:45.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.666+0000 7ffb66a00640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb4806bc00 0x7ffb4806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:45.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.666+0000 7ffb66a00640 1 -- 172.21.15.26:0/2738927166 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ffb680aaf80 con 0x7ffb70160f80 2024-01-15T18:02:45.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.669+0000 7ffb6ea00640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb4806bc00 0x7ffb4806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:45.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.679+0000 7ffb6ea00640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb4806bc00 0x7ffb4806e0c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c900 tx=0x7ffb60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:45.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:45.681+0000 7ffb66a00640 1 -- 172.21.15.26:0/2738927166 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb68077790 con 0x7ffb70160f80 2024-01-15T18:02:46.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.088+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20835", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb7008d2e0 con 0x7ffb4806bc00 2024-01-15T18:02:46.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.095+0000 7ffb66a00640 1 -- 172.21.15.26:0/2738927166 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7ffb7008d2e0 con 0x7ffb4806bc00 2024-01-15T18:02:46.098 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:46.099 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:46.099 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:02:46.099 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:02:46.099 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:02:46.099 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:02:46.099 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:02:46.099 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:46.099 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:02:46.099 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:02:46.100 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:02:46.100 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "2/3", 2024-01-15T18:02:46.100 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:02:46.100 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:02:46.100 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:46.100 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:46.100 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb4806bc00 msgr2=0x7ffb4806e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb4806bc00 0x7ffb4806e0c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000c900 tx=0x7ffb60008040 comp rx=0 tx=0).stop 2024-01-15T18:02:46.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb70160f80 msgr2=0x7ffb701f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb70160f80 0x7ffb701f6ea0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ffb6802c430 tx=0x7ffb68004a40 comp rx=0 tx=0).stop 2024-01-15T18:02:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 shutdown_connections 2024-01-15T18:02:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffb700ced50 0x7ffb701f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffb4806bc00 0x7ffb4806e0c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffb70160f80 0x7ffb701f6ea0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 --2- 172.21.15.26:0/2738927166 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffb700ce8f0 0x7ffb7015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 >> 172.21.15.26:0/2738927166 conn(0x7ffb7009c550 msgr2=0x7ffb700b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 shutdown_connections 2024-01-15T18:02:46.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:46.097+0000 7ffb75600640 1 -- 172.21.15.26:0/2738927166 wait complete. 2024-01-15T18:02:46.118 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:02:47.119 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20835 2024-01-15T18:02:47.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.439+0000 7fe587e00640 1 Processor -- start 2024-01-15T18:02:47.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.439+0000 7fe587e00640 1 -- start start 2024-01-15T18:02:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.441+0000 7fe587e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe58014b2c0 0x7fe5800cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.441+0000 7fe587e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5800cf7d0 0x7fe580070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.441+0000 7fe587e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5800cfbf0 con 0x7fe580070ea0 2024-01-15T18:02:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.441+0000 7fe587e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58014b6a0 con 0x7fe5800cf7d0 2024-01-15T18:02:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.441+0000 7fe587e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe580071690 con 0x7fe58014b2c0 2024-01-15T18:02:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.442+0000 7fe584a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5800cf7d0 0x7fe580070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.442+0000 7fe585400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe58014b2c0 0x7fe5800cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.442+0000 7fe584a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5800cf7d0 0x7fe580070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51776/0 (socket says 172.21.15.26:51776) 2024-01-15T18:02:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.442+0000 7fe584a00640 1 -- 172.21.15.26:0/2121475628 learned_addr learned my addr 172.21.15.26:0/2121475628 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.443+0000 7fe584a00640 1 -- 172.21.15.26:0/2121475628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe58014b2c0 msgr2=0x7fe5800cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.443+0000 7fe584a00640 1 --2- 172.21.15.26:0/2121475628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe58014b2c0 0x7fe5800cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.443+0000 7fe584a00640 1 -- 172.21.15.26:0/2121475628 >> v1:172.21.15.26:6789/0 conn(0x7fe580070ea0 legacy=0x7fe5800b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.443+0000 7fe584a00640 1 -- 172.21.15.26:0/2121475628 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580071970 con 0x7fe5800cf7d0 2024-01-15T18:02:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.445+0000 7fe584a00640 1 --2- 172.21.15.26:0/2121475628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5800cf7d0 0x7fe580070960 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe568009870 tx=0x7fe56802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9eda44799dddfd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.445+0000 7fe57fe00640 1 -- 172.21.15.26:0/2121475628 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568030840 con 0x7fe5800cf7d0 2024-01-15T18:02:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.446+0000 7fe57fe00640 1 -- 172.21.15.26:0/2121475628 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5680309a0 con 0x7fe5800cf7d0 2024-01-15T18:02:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.446+0000 7fe57fe00640 1 -- 172.21.15.26:0/2121475628 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568030d30 con 0x7fe5800cf7d0 2024-01-15T18:02:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.447+0000 7fe587e00640 1 -- 172.21.15.26:0/2121475628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5800cf7d0 msgr2=0x7fe580070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:47.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.447+0000 7fe587e00640 1 --2- 172.21.15.26:0/2121475628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5800cf7d0 0x7fe580070960 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe568009870 tx=0x7fe56802b980 comp rx=0 tx=0).stop 2024-01-15T18:02:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.447+0000 7fe587e00640 1 -- 172.21.15.26:0/2121475628 shutdown_connections 2024-01-15T18:02:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.447+0000 7fe587e00640 1 --2- 172.21.15.26:0/2121475628 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5800cf7d0 0x7fe580070960 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.447+0000 7fe587e00640 1 --2- 172.21.15.26:0/2121475628 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe58014b2c0 0x7fe5800cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.447+0000 7fe587e00640 1 -- 172.21.15.26:0/2121475628 >> 172.21.15.26:0/2121475628 conn(0x7fe58006a800 msgr2=0x7fe58006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:47.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.448+0000 7fe587e00640 1 -- 172.21.15.26:0/2121475628 shutdown_connections 2024-01-15T18:02:47.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.448+0000 7fe587e00640 1 -- 172.21.15.26:0/2121475628 wait complete. 2024-01-15T18:02:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.450+0000 7fe587e00640 1 Processor -- start 2024-01-15T18:02:47.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.450+0000 7fe587e00640 1 -- start start 2024-01-15T18:02:47.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.451+0000 7fe587e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe580070ea0 0x7fe580144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:47.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.452+0000 7fe587e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5800cf7d0 0x7fe5801605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:47.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.452+0000 7fe587e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe58014b2c0 0x7fe580142dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:47.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.452+0000 7fe587e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5801e9f20 con 0x7fe580070ea0 2024-01-15T18:02:47.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.452+0000 7fe587e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe580161640 con 0x7fe58014b2c0 2024-01-15T18:02:47.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.452+0000 7fe587e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe580143a10 con 0x7fe5800cf7d0 2024-01-15T18:02:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.455+0000 7fe585400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe580070ea0 0x7fe580144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.456+0000 7fe584a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5800cf7d0 0x7fe5801605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:47.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.456+0000 7fe585e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe58014b2c0 0x7fe580142dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.456+0000 7fe585400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe580070ea0 0x7fe580144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51778/0 (socket says 172.21.15.26:51778) 2024-01-15T18:02:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.456+0000 7fe585400640 1 -- 172.21.15.26:0/1911729730 learned_addr learned my addr 172.21.15.26:0/1911729730 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:47.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.456+0000 7fe585400640 1 -- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5800cf7d0 msgr2=0x7fe5801605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.456+0000 7fe585400640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5800cf7d0 0x7fe5801605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.456+0000 7fe585400640 1 -- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe58014b2c0 msgr2=0x7fe580142dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.456+0000 7fe585400640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe58014b2c0 0x7fe580142dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.456+0000 7fe585400640 1 -- 172.21.15.26:0/1911729730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680094d0 con 0x7fe580070ea0 2024-01-15T18:02:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.457+0000 7fe584a00640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5800cf7d0 0x7fe5801605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:02:47.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.457+0000 7fe585400640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe580070ea0 0x7fe580144ab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe57000d5f0 tx=0x7fe57000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.458+0000 7fe57d600640 1 -- 172.21.15.26:0/1911729730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5700125f0 con 0x7fe580070ea0 2024-01-15T18:02:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.458+0000 7fe57d600640 1 -- 172.21.15.26:0/1911729730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe570012750 con 0x7fe580070ea0 2024-01-15T18:02:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.458+0000 7fe57d600640 1 -- 172.21.15.26:0/1911729730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570012b00 con 0x7fe580070ea0 2024-01-15T18:02:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.458+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe580143cf0 con 0x7fe580070ea0 2024-01-15T18:02:47.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.458+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5801442b0 con 0x7fe580070ea0 2024-01-15T18:02:47.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.460+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe550005250 con 0x7fe580070ea0 2024-01-15T18:02:47.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.462+0000 7fe57d600640 1 -- 172.21.15.26:0/1911729730 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7fe570012c60 con 0x7fe580070ea0 2024-01-15T18:02:47.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.464+0000 7fe57d600640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe55806bb80 0x7fe55806e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:47.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.464+0000 7fe57d600640 1 -- 172.21.15.26:0/1911729730 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe570088b20 con 0x7fe580070ea0 2024-01-15T18:02:47.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.467+0000 7fe584a00640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe55806bb80 0x7fe55806e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:47.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.469+0000 7fe584a00640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe55806bb80 0x7fe55806e040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe568036000 tx=0x7fe568008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:47.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.478+0000 7fe57d600640 1 -- 172.21.15.26:0/1911729730 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5700551c0 con 0x7fe580070ea0 2024-01-15T18:02:47.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.863+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20835", "target": ["mon-mgr", ""]}) v1 -- 0x7fe550000bc0 con 0x7fe55806bb80 2024-01-15T18:02:47.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.883+0000 7fe57d600640 1 -- 172.21.15.26:0/1911729730 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7fe550000bc0 con 0x7fe55806bb80 2024-01-15T18:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:47.887 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "2/3", 2024-01-15T18:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:47.888 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:47.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.887+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe55806bb80 msgr2=0x7fe55806e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:47.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.887+0000 7fe587e00640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe55806bb80 0x7fe55806e040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe568036000 tx=0x7fe568008030 comp rx=0 tx=0).stop 2024-01-15T18:02:47.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.887+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe580070ea0 msgr2=0x7fe580144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:47.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.887+0000 7fe587e00640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe580070ea0 0x7fe580144ab0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe57000d5f0 tx=0x7fe57000dab0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.887+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 shutdown_connections 2024-01-15T18:02:47.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.887+0000 7fe587e00640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5800cf7d0 0x7fe5801605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.887+0000 7fe587e00640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe55806bb80 0x7fe55806e040 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.887+0000 7fe587e00640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe58014b2c0 0x7fe580142dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.887+0000 7fe587e00640 1 --2- 172.21.15.26:0/1911729730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe580070ea0 0x7fe580144ab0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:47.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.888+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 >> 172.21.15.26:0/1911729730 conn(0x7fe58006a800 msgr2=0x7fe580149bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:47.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.888+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 shutdown_connections 2024-01-15T18:02:47.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:47.888+0000 7fe587e00640 1 -- 172.21.15.26:0/1911729730 wait complete. 2024-01-15T18:02:47.923 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:02:48.924 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20835 2024-01-15T18:02:49.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.191+0000 7fdf53600640 1 Processor -- start 2024-01-15T18:02:49.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.191+0000 7fdf53600640 1 -- start start 2024-01-15T18:02:49.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.194+0000 7fdf53600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c160f80 0x7fdf4c161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:49.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.194+0000 7fdf53600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4c0cf1b0 con 0x7fdf4c0ced50 2024-01-15T18:02:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.194+0000 7fdf53600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4c1618a0 con 0x7fdf4c0ce8f0 2024-01-15T18:02:49.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.194+0000 7fdf53600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0d0a00 con 0x7fdf4c160f80 2024-01-15T18:02:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.196+0000 7fdf51600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdf4c0ced50 0x7fdf4c0d02f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51482/0 (socket says 172.21.15.26:51482) 2024-01-15T18:02:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.196+0000 7fdf51600640 1 -- 172.21.15.26:0/2841786305 learned_addr learned my addr 172.21.15.26:0/2841786305 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:49.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.196+0000 7fdf50c00640 1 --2- 172.21.15.26:0/2841786305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c160f80 0x7fdf4c161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.197+0000 7fdf4b400640 1 -- 172.21.15.26:0/2841786305 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415725401 0 0) 0x7fdf4c0cf1b0 con 0x7fdf4c0ced50 2024-01-15T18:02:49.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.197+0000 7fdf4b400640 1 -- 172.21.15.26:0/2841786305 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf2c003520 con 0x7fdf4c0ced50 2024-01-15T18:02:49.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.198+0000 7fdf50c00640 1 -- 172.21.15.26:0/2841786305 >> v1:172.21.15.124:6789/0 conn(0x7fdf4c0ce8f0 legacy=0x7fdf4c0cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:49.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.198+0000 7fdf50c00640 1 -- 172.21.15.26:0/2841786305 >> v1:172.21.15.26:6789/0 conn(0x7fdf4c0ced50 legacy=0x7fdf4c0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:49.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.198+0000 7fdf4b400640 1 -- 172.21.15.26:0/2841786305 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1539077556 0 0) 0x7fdf2c003520 con 0x7fdf4c0ced50 2024-01-15T18:02:49.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.198+0000 7fdf50c00640 1 -- 172.21.15.26:0/2841786305 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c0d0de0 con 0x7fdf4c160f80 2024-01-15T18:02:49.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.210+0000 7fdf50c00640 1 --2- 172.21.15.26:0/2841786305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c160f80 0x7fdf4c161360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009dc0 tx=0x7fdf4402bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e145e6cb635250e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:49.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.212+0000 7fdf4b400640 1 -- 172.21.15.26:0/2841786305 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c001670 con 0x7fdf4c160f80 2024-01-15T18:02:49.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.212+0000 7fdf4b400640 1 -- 172.21.15.26:0/2841786305 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4402c880 con 0x7fdf4c160f80 2024-01-15T18:02:49.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.212+0000 7fdf4b400640 1 -- 172.21.15.26:0/2841786305 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4402cc10 con 0x7fdf4c160f80 2024-01-15T18:02:49.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.213+0000 7fdf53600640 1 -- 172.21.15.26:0/2841786305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c160f80 msgr2=0x7fdf4c161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:49.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.213+0000 7fdf53600640 1 --2- 172.21.15.26:0/2841786305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c160f80 0x7fdf4c161360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf44009dc0 tx=0x7fdf4402bd50 comp rx=0 tx=0).stop 2024-01-15T18:02:49.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.213+0000 7fdf53600640 1 -- 172.21.15.26:0/2841786305 shutdown_connections 2024-01-15T18:02:49.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.213+0000 7fdf53600640 1 --2- 172.21.15.26:0/2841786305 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c160f80 0x7fdf4c161360 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:49.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.213+0000 7fdf53600640 1 -- 172.21.15.26:0/2841786305 >> 172.21.15.26:0/2841786305 conn(0x7fdf4c09c550 msgr2=0x7fdf4c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:49.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.214+0000 7fdf53600640 1 -- 172.21.15.26:0/2841786305 shutdown_connections 2024-01-15T18:02:49.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.214+0000 7fdf53600640 1 -- 172.21.15.26:0/2841786305 wait complete. 2024-01-15T18:02:49.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.217+0000 7fdf53600640 1 Processor -- start 2024-01-15T18:02:49.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.218+0000 7fdf53600640 1 -- start start 2024-01-15T18:02:49.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.220+0000 7fdf53600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c0ce8f0 0x7fdf4c0c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:49.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.220+0000 7fdf53600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf4c0ced50 0x7fdf4c149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:49.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.220+0000 7fdf53600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf4c160f80 0x7fdf4c14a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.220+0000 7fdf53600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0cc650 con 0x7fdf4c160f80 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.220+0000 7fdf53600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c15ad10 con 0x7fdf4c0ced50 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.220+0000 7fdf53600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c0cbd40 con 0x7fdf4c0ce8f0 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.221+0000 7fdf51600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf4c160f80 0x7fdf4c14a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.221+0000 7fdf51600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf4c160f80 0x7fdf4c14a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51792/0 (socket says 172.21.15.26:51792) 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.221+0000 7fdf51600640 1 -- 172.21.15.26:0/188435947 learned_addr learned my addr 172.21.15.26:0/188435947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.222+0000 7fdf4be00640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf4c0ced50 0x7fdf4c149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.222+0000 7fdf51600640 1 -- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c0ce8f0 msgr2=0x7fdf4c0c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.222+0000 7fdf51600640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c0ce8f0 0x7fdf4c0c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.222+0000 7fdf51600640 1 -- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf4c0ced50 msgr2=0x7fdf4c149fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:49.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.222+0000 7fdf51600640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf4c0ced50 0x7fdf4c149fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:49.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.223+0000 7fdf51600640 1 -- 172.21.15.26:0/188435947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44009a70 con 0x7fdf4c160f80 2024-01-15T18:02:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.225+0000 7fdf51600640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf4c160f80 0x7fdf4c14a910 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00c5b0 tx=0x7fdf3c00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:49.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.226+0000 7fdf48c00640 1 -- 172.21.15.26:0/188435947 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c005ce0 con 0x7fdf4c160f80 2024-01-15T18:02:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.226+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c0c7390 con 0x7fdf4c160f80 2024-01-15T18:02:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.226+0000 7fdf48c00640 1 -- 172.21.15.26:0/188435947 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c005e40 con 0x7fdf4c160f80 2024-01-15T18:02:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.226+0000 7fdf48c00640 1 -- 172.21.15.26:0/188435947 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0072b0 con 0x7fdf4c160f80 2024-01-15T18:02:49.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.227+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c0cc270 con 0x7fdf4c160f80 2024-01-15T18:02:49.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.228+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c0675d0 con 0x7fdf4c160f80 2024-01-15T18:02:49.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.230+0000 7fdf48c00640 1 -- 172.21.15.26:0/188435947 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7fdf3c007410 con 0x7fdf4c160f80 2024-01-15T18:02:49.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.231+0000 7fdf48c00640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf2406bb30 0x7fdf2406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:49.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.231+0000 7fdf48c00640 1 -- 172.21.15.26:0/188435947 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdf3c087bb0 con 0x7fdf4c160f80 2024-01-15T18:02:49.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.232+0000 7fdf50c00640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf2406bb30 0x7fdf2406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:49.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.237+0000 7fdf48c00640 1 -- 172.21.15.26:0/188435947 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf3c057e40 con 0x7fdf4c160f80 2024-01-15T18:02:49.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.239+0000 7fdf50c00640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf2406bb30 0x7fdf2406dff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf44036000 tx=0x7fdf4402e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:49.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:49.746+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20835", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf4c0d1680 con 0x7fdf2406bb30 2024-01-15T18:02:50.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.287+0000 7fdf48c00640 1 -- 172.21.15.26:0/188435947 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7fdf4402cc70 con 0x7fdf2406bb30 2024-01-15T18:02:50.291 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:02:50.291 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:50.291 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:50.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf2406bb30 msgr2=0x7fdf2406dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:50.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf2406bb30 0x7fdf2406dff0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdf44036000 tx=0x7fdf4402e000 comp rx=0 tx=0).stop 2024-01-15T18:02:50.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf4c160f80 msgr2=0x7fdf4c14a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:50.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf4c160f80 0x7fdf4c14a910 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c00c5b0 tx=0x7fdf3c00ca70 comp rx=0 tx=0).stop 2024-01-15T18:02:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 shutdown_connections 2024-01-15T18:02:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf2406bb30 0x7fdf2406dff0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf4c160f80 0x7fdf4c14a910 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf4c0ced50 0x7fdf4c149fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 --2- 172.21.15.26:0/188435947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf4c0ce8f0 0x7fdf4c0c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 >> 172.21.15.26:0/188435947 conn(0x7fdf4c09c550 msgr2=0x7fdf4c0ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 shutdown_connections 2024-01-15T18:02:50.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.290+0000 7fdf53600640 1 -- 172.21.15.26:0/188435947 wait complete. 2024-01-15T18:02:50.327 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:02:50.327 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20836 2024-01-15T18:02:50.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.644+0000 7f497e200640 1 Processor -- start 2024-01-15T18:02:50.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.644+0000 7f497e200640 1 -- start start 2024-01-15T18:02:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.646+0000 7f497e200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f497815de80 0x7f49780ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.646+0000 7f497e200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497806c730 con 0x7f4978071e40 2024-01-15T18:02:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.646+0000 7f497e200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49780724d0 con 0x7f4978143810 2024-01-15T18:02:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.646+0000 7f497e200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4978143bf0 con 0x7f497815de80 2024-01-15T18:02:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.647+0000 7f4977e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4978071e40 0x7f497815d770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51484/0 (socket says 172.21.15.26:51484) 2024-01-15T18:02:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.647+0000 7f4977e00640 1 -- 172.21.15.26:0/2057325544 learned_addr learned my addr 172.21.15.26:0/2057325544 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:50.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.648+0000 7f497d800640 1 --2- 172.21.15.26:0/2057325544 >> v2:172.21.15.124:3300/0 conn(0x7f497815de80 0x7f49780ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.648+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808601478 0 0) 0x7f497806c730 con 0x7f4978071e40 2024-01-15T18:02:50.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.649+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4958003520 con 0x7f4978071e40 2024-01-15T18:02:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.649+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 390537678 0 0) 0x7f4958003520 con 0x7f4978071e40 2024-01-15T18:02:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.649+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 >> v2:172.21.15.124:3300/0 conn(0x7f497815de80 msgr2=0x7f49780ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.649+0000 7f4977400640 1 --2- 172.21.15.26:0/2057325544 >> v2:172.21.15.124:3300/0 conn(0x7f497815de80 0x7f49780ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.649+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 >> v1:172.21.15.124:6789/0 conn(0x7f4978143810 legacy=0x7f497815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.650+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497806c9f0 con 0x7f4978071e40 2024-01-15T18:02:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.650+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f4968001670 con 0x7f4978071e40 2024-01-15T18:02:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.650+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 >> v1:172.21.15.26:6789/0 conn(0x7f4978071e40 legacy=0x7f497815d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.650+0000 7f4977400640 1 --2- 172.21.15.26:0/2057325544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4958003970 0x7f4958003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.650+0000 7f4977400640 1 --2- 172.21.15.26:0/2057325544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49580047c0 0x7f4958004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:50.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.650+0000 7f4977400640 1 --2- 172.21.15.26:0/2057325544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49580052a0 0x7f495800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.650+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4978143bf0 con 0x7f4958003970 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.650+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495800c170 con 0x7f49580047c0 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.650+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495800c330 con 0x7f49580052a0 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.651+0000 7f497ce00640 1 --2- 172.21.15.26:0/2057325544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4958003970 0x7f4958003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.652+0000 7f497ce00640 1 -- 172.21.15.26:0/2057325544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49580052a0 msgr2=0x7f495800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.652+0000 7f497ce00640 1 --2- 172.21.15.26:0/2057325544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49580052a0 0x7f495800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.652+0000 7f497ce00640 1 -- 172.21.15.26:0/2057325544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49580047c0 msgr2=0x7f4958004c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.652+0000 7f497ce00640 1 --2- 172.21.15.26:0/2057325544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49580047c0 0x7f4958004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.652+0000 7f497ce00640 1 -- 172.21.15.26:0/2057325544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497806c9f0 con 0x7f4958003970 2024-01-15T18:02:50.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.652+0000 7f497d800640 1 -- 172.21.15.26:0/2057325544 reap_dead start 2024-01-15T18:02:50.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.654+0000 7f497ce00640 1 --2- 172.21.15.26:0/2057325544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4958003970 0x7f4958003d50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4960001860 tx=0x7f49580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fddcaa714d161cb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:50.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.655+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960033410 con 0x7f4958003970 2024-01-15T18:02:50.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.655+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4960033590 con 0x7f4958003970 2024-01-15T18:02:50.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.655+0000 7f4977400640 1 -- 172.21.15.26:0/2057325544 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960033980 con 0x7f4958003970 2024-01-15T18:02:50.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.656+0000 7f497e200640 1 -- 172.21.15.26:0/2057325544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4958003970 msgr2=0x7f4958003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:50.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.656+0000 7f497e200640 1 --2- 172.21.15.26:0/2057325544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4958003970 0x7f4958003d50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4960001860 tx=0x7f49580057f0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.656+0000 7f497e200640 1 -- 172.21.15.26:0/2057325544 shutdown_connections 2024-01-15T18:02:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.656+0000 7f497e200640 1 --2- 172.21.15.26:0/2057325544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4958003970 0x7f4958003d50 secure :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f4960001860 tx=0x7f49580057f0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.656+0000 7f497e200640 1 -- 172.21.15.26:0/2057325544 >> 172.21.15.26:0/2057325544 conn(0x7f497806a7d0 msgr2=0x7f497806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:50.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.657+0000 7f497e200640 1 -- 172.21.15.26:0/2057325544 shutdown_connections 2024-01-15T18:02:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.658+0000 7f497e200640 1 -- 172.21.15.26:0/2057325544 wait complete. 2024-01-15T18:02:50.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.661+0000 7f497e200640 1 Processor -- start 2024-01-15T18:02:50.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.661+0000 7f497e200640 1 -- start start 2024-01-15T18:02:50.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.663+0000 7f497e200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497815c9d0 0x7f4978144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.663+0000 7f497e200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497815cdb0 0x7f49780b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.663+0000 7f497e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f497813b960 0x7f49780cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.663+0000 7f497e200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49780722f0 con 0x7f497813b960 2024-01-15T18:02:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.663+0000 7f497e200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49780b17c0 con 0x7f497815cdb0 2024-01-15T18:02:50.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.663+0000 7f497e200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4978145370 con 0x7f497815c9d0 2024-01-15T18:02:50.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.664+0000 7f497d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f497813b960 0x7f49780cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:50.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.664+0000 7f497d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f497813b960 0x7f49780cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:51802/0 (socket says 172.21.15.26:51802) 2024-01-15T18:02:50.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.664+0000 7f497d800640 1 -- 172.21.15.26:0/2309891116 learned_addr learned my addr 172.21.15.26:0/2309891116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:50.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.664+0000 7f4977e00640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497815cdb0 0x7f49780b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:50.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.664+0000 7f497ce00640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497815c9d0 0x7f4978144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:50.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.665+0000 7f497d800640 1 -- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497815c9d0 msgr2=0x7f4978144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:50.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.665+0000 7f497d800640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497815c9d0 0x7f4978144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.665+0000 7f497d800640 1 -- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497815cdb0 msgr2=0x7f49780b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:50.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.665+0000 7f497d800640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497815cdb0 0x7f49780b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:50.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.665+0000 7f497d800640 1 -- 172.21.15.26:0/2309891116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4978145650 con 0x7f497813b960 2024-01-15T18:02:50.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.665+0000 7f497d800640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f497813b960 0x7f49780cbd00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f497815e260 tx=0x7f497000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:50.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.666+0000 7f4974c00640 1 -- 172.21.15.26:0/2309891116 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497000ed90 con 0x7f497813b960 2024-01-15T18:02:50.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.666+0000 7f4974c00640 1 -- 172.21.15.26:0/2309891116 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4970014420 con 0x7f497813b960 2024-01-15T18:02:50.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.666+0000 7f4974c00640 1 -- 172.21.15.26:0/2309891116 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700147d0 con 0x7f497813b960 2024-01-15T18:02:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.667+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4978145930 con 0x7f497813b960 2024-01-15T18:02:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.667+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4978145ef0 con 0x7f497813b960 2024-01-15T18:02:50.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.669+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4978146110 con 0x7f497813b960 2024-01-15T18:02:50.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.677+0000 7f4974c00640 1 -- 172.21.15.26:0/2309891116 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f4970014930 con 0x7f497813b960 2024-01-15T18:02:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.679+0000 7f4974c00640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f495006bb30 0x7f495006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:50.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.679+0000 7f4974c00640 1 -- 172.21.15.26:0/2309891116 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4970089a70 con 0x7f497813b960 2024-01-15T18:02:50.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.682+0000 7f4974c00640 1 -- 172.21.15.26:0/2309891116 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4970056160 con 0x7f497813b960 2024-01-15T18:02:50.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.698+0000 7f497ce00640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f495006bb30 0x7f495006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:50.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:50.699+0000 7f497ce00640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f495006bb30 0x7f495006dff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4960035000 tx=0x7f4960009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:51.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.060+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20836", "target": ["mon-mgr", ""]}) v1 -- 0x7f497806cfb0 con 0x7f495006bb30 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 100.0, 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "3 GB/3 GB", 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "3/3", 2024-01-15T18:02:51.077 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:02:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:02:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:51.078 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:51.078 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:51.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.072+0000 7f4974c00640 1 -- 172.21.15.26:0/2309891116 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+347 (secure 0 0 0) 0x7f49600072e0 con 0x7f495006bb30 2024-01-15T18:02:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f495006bb30 msgr2=0x7f495006dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f495006bb30 0x7f495006dff0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4960035000 tx=0x7f4960009030 comp rx=0 tx=0).stop 2024-01-15T18:02:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f497813b960 msgr2=0x7f49780cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f497813b960 0x7f49780cbd00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f497815e260 tx=0x7f497000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:02:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 shutdown_connections 2024-01-15T18:02:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f495006bb30 0x7f495006dff0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f497813b960 0x7f49780cbd00 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f497815cdb0 0x7f49780b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:51.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 --2- 172.21.15.26:0/2309891116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f497815c9d0 0x7f4978144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 >> 172.21.15.26:0/2309891116 conn(0x7f497806a7d0 msgr2=0x7f497806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.076+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 shutdown_connections 2024-01-15T18:02:51.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:51.077+0000 7f497e200640 1 -- 172.21.15.26:0/2309891116 wait complete. 2024-01-15T18:02:51.110 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:02:52.111 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20836 2024-01-15T18:02:52.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.340+0000 7f9334400640 1 Processor -- start 2024-01-15T18:02:52.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.340+0000 7f9334400640 1 -- start start 2024-01-15T18:02:52.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.341+0000 7f9334400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f932c15d610 0x7f932c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:52.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.341+0000 7f9334400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f932c0ced60 0x7f932c0d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:52.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.341+0000 7f9334400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f932c0d2170 con 0x7f932c0d1d20 2024-01-15T18:02:52.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.341+0000 7f9334400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c0d1390 con 0x7f932c15d610 2024-01-15T18:02:52.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.341+0000 7f9334400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f932c0b3400 con 0x7f932c0ced60 2024-01-15T18:02:52.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.342+0000 7f9331a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f932c15d610 0x7f932c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:52.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.342+0000 7f9331a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f932c15d610 0x7f932c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34862/0 (socket says 172.21.15.26:34862) 2024-01-15T18:02:52.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.342+0000 7f9331a00640 1 -- 172.21.15.26:0/3916836478 learned_addr learned my addr 172.21.15.26:0/3916836478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:52.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.342+0000 7f9331000640 1 --2- 172.21.15.26:0/3916836478 >> v2:172.21.15.124:3300/0 conn(0x7f932c0ced60 0x7f932c0d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:52.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.342+0000 7f9331a00640 1 -- 172.21.15.26:0/3916836478 >> v2:172.21.15.124:3300/0 conn(0x7f932c0ced60 msgr2=0x7f932c0d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:52.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.342+0000 7f9331a00640 1 --2- 172.21.15.26:0/3916836478 >> v2:172.21.15.124:3300/0 conn(0x7f932c0ced60 0x7f932c0d17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:52.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.342+0000 7f9331a00640 1 -- 172.21.15.26:0/3916836478 >> v1:172.21.15.124:6789/0 conn(0x7f932c0d1d20 legacy=0x7f932c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:52.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.342+0000 7f9331a00640 1 -- 172.21.15.26:0/3916836478 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c0b27f0 con 0x7f932c15d610 2024-01-15T18:02:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.343+0000 7f9331a00640 1 --2- 172.21.15.26:0/3916836478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c15d610 0x7f932c0ce820 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f932c08d4b0 tx=0x7f931c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67b8186b2fef285 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.344+0000 7f932be00640 1 -- 172.21.15.26:0/3916836478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c02ca80 con 0x7f932c15d610 2024-01-15T18:02:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.344+0000 7f932be00640 1 -- 172.21.15.26:0/3916836478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c02cbe0 con 0x7f932c15d610 2024-01-15T18:02:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.344+0000 7f932be00640 1 -- 172.21.15.26:0/3916836478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0343f0 con 0x7f932c15d610 2024-01-15T18:02:52.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.344+0000 7f9334400640 1 -- 172.21.15.26:0/3916836478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c15d610 msgr2=0x7f932c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:52.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.344+0000 7f9334400640 1 --2- 172.21.15.26:0/3916836478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c15d610 0x7f932c0ce820 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f932c08d4b0 tx=0x7f931c02bee0 comp rx=0 tx=0).stop 2024-01-15T18:02:52.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.345+0000 7f9334400640 1 -- 172.21.15.26:0/3916836478 shutdown_connections 2024-01-15T18:02:52.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.345+0000 7f9334400640 1 --2- 172.21.15.26:0/3916836478 >> v2:172.21.15.124:3300/0 conn(0x7f932c0ced60 0x7f932c0d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:52.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.345+0000 7f9334400640 1 --2- 172.21.15.26:0/3916836478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c15d610 0x7f932c0ce820 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:52.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.345+0000 7f9334400640 1 -- 172.21.15.26:0/3916836478 >> 172.21.15.26:0/3916836478 conn(0x7f932c09c540 msgr2=0x7f932c0d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:52.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.345+0000 7f9334400640 1 -- 172.21.15.26:0/3916836478 shutdown_connections 2024-01-15T18:02:52.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.345+0000 7f9334400640 1 -- 172.21.15.26:0/3916836478 wait complete. 2024-01-15T18:02:52.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.346+0000 7f9334400640 1 Processor -- start 2024-01-15T18:02:52.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.346+0000 7f9334400640 1 -- start start 2024-01-15T18:02:52.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.347+0000 7f9334400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f932c0ced60 0x7f932c160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:52.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.347+0000 7f9334400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0d1d20 0x7f932c0b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:52.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.347+0000 7f9334400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f932c15d610 0x7f932c0b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:52.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.347+0000 7f9334400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c160ad0 con 0x7f932c0d1d20 2024-01-15T18:02:52.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.347+0000 7f9334400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c159930 con 0x7f932c15d610 2024-01-15T18:02:52.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.347+0000 7f9334400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932c0cc910 con 0x7f932c0ced60 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0d1d20 0x7f932c0b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0d1d20 0x7f932c0b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34868/0 (socket says 172.21.15.26:34868) 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331000640 1 -- 172.21.15.26:0/682629518 learned_addr learned my addr 172.21.15.26:0/682629518 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9332400640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f932c15d610 0x7f932c0b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331a00640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f932c0ced60 0x7f932c160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331000640 1 -- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f932c0ced60 msgr2=0x7f932c160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331000640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f932c0ced60 0x7f932c160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331000640 1 -- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f932c15d610 msgr2=0x7f932c0b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331000640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f932c15d610 0x7f932c0b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331000640 1 -- 172.21.15.26:0/682629518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c009b30 con 0x7f932c0d1d20 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.348+0000 7f9331000640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0d1d20 0x7f932c0b6050 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f931400d800 tx=0x7f931400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.349+0000 7f9329600640 1 -- 172.21.15.26:0/682629518 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400b6c0 con 0x7f932c0d1d20 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.349+0000 7f9329600640 1 -- 172.21.15.26:0/682629518 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931400b820 con 0x7f932c0d1d20 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.349+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c0ccbf0 con 0x7f932c0d1d20 2024-01-15T18:02:52.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.349+0000 7f9329600640 1 -- 172.21.15.26:0/682629518 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931400bbb0 con 0x7f932c0d1d20 2024-01-15T18:02:52.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.349+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c15e420 con 0x7f932c0d1d20 2024-01-15T18:02:52.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.350+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932c0675d0 con 0x7f932c0d1d20 2024-01-15T18:02:52.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.351+0000 7f9329600640 1 -- 172.21.15.26:0/682629518 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f931400bd10 con 0x7f932c0d1d20 2024-01-15T18:02:52.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.352+0000 7f9329600640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f930406bb30 0x7f930406dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:52.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.352+0000 7f9329600640 1 -- 172.21.15.26:0/682629518 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9314088bd0 con 0x7f932c0d1d20 2024-01-15T18:02:52.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.352+0000 7f9331a00640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f930406bb30 0x7f930406dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:52.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.353+0000 7f9331a00640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f930406bb30 0x7f930406dff0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f931c035000 tx=0x7f931c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:52.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.356+0000 7f9329600640 1 -- 172.21.15.26:0/682629518 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93140552c0 con 0x7f932c0d1d20 2024-01-15T18:02:52.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:52.576+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20836", "target": ["mon-mgr", ""]}) v1 -- 0x7f932c0b3090 con 0x7f930406bb30 2024-01-15T18:02:53.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.374+0000 7f9329600640 1 -- 172.21.15.26:0/682629518 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f931c02ca50 con 0x7f930406bb30 2024-01-15T18:02:53.379 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:02:53.379 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:53.379 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:53.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f930406bb30 msgr2=0x7f930406dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:53.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f930406bb30 0x7f930406dff0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f931c035000 tx=0x7f931c030000 comp rx=0 tx=0).stop 2024-01-15T18:02:53.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0d1d20 msgr2=0x7f932c0b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:53.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0d1d20 0x7f932c0b6050 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f931400d800 tx=0x7f931400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 shutdown_connections 2024-01-15T18:02:53.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f930406bb30 0x7f930406dff0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f932c15d610 0x7f932c0b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f932c0d1d20 0x7f932c0b6050 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 --2- 172.21.15.26:0/682629518 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f932c0ced60 0x7f932c160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.378+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 >> 172.21.15.26:0/682629518 conn(0x7f932c09c540 msgr2=0x7f932c0b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.379+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 shutdown_connections 2024-01-15T18:02:53.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.379+0000 7f9334400640 1 -- 172.21.15.26:0/682629518 wait complete. 2024-01-15T18:02:53.403 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:02:53.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20837 2024-01-15T18:02:53.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.673+0000 7f4f1b600640 1 Processor -- start 2024-01-15T18:02:53.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.673+0000 7f4f1b600640 1 -- start start 2024-01-15T18:02:53.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.674+0000 7f4f1b600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4f140cf9a0 0x7f4f1415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:53.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.674+0000 7f4f1b600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f1415e1e0 0x7f4f140702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:53.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.674+0000 7f4f1b600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f140b1ee0 con 0x7f4f14143ae0 2024-01-15T18:02:53.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.674+0000 7f4f1b600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f140b2780 con 0x7f4f140cf9a0 2024-01-15T18:02:53.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.674+0000 7f4f1b600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1415e990 con 0x7f4f1415e1e0 2024-01-15T18:02:53.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.675+0000 7f4f18c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4f14143ae0 0x7f4f140cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54010/0 (socket says 172.21.15.26:54010) 2024-01-15T18:02:53.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.675+0000 7f4f18c00640 1 -- 172.21.15.26:0/2773218640 learned_addr learned my addr 172.21.15.26:0/2773218640 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:53.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.675+0000 7f4f0be00640 1 --2- 172.21.15.26:0/2773218640 >> v2:172.21.15.26:3300/0 conn(0x7f4f140cf9a0 0x7f4f1415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:53.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.675+0000 7f4f19600640 1 --2- 172.21.15.26:0/2773218640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f1415e1e0 0x7f4f140702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:53.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.675+0000 7f4f13e00640 1 -- 172.21.15.26:0/2773218640 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220226270 0 0) 0x7f4f140b1ee0 con 0x7f4f14143ae0 2024-01-15T18:02:53.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.675+0000 7f4f0be00640 1 -- 172.21.15.26:0/2773218640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f1415e1e0 msgr2=0x7f4f140702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:53.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.675+0000 7f4f0be00640 1 --2- 172.21.15.26:0/2773218640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f1415e1e0 0x7f4f140702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.675+0000 7f4f0be00640 1 -- 172.21.15.26:0/2773218640 >> v1:172.21.15.26:6789/0 conn(0x7f4f14143ae0 legacy=0x7f4f140cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:53.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.675+0000 7f4f0be00640 1 -- 172.21.15.26:0/2773218640 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f140b2130 con 0x7f4f140cf9a0 2024-01-15T18:02:53.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.677+0000 7f4f0be00640 1 --2- 172.21.15.26:0/2773218640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f140cf9a0 0x7f4f1415dca0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0097a0 tx=0x7f4efc02fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c34e0ab564164b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:53.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.677+0000 7f4f13e00640 1 -- 172.21.15.26:0/2773218640 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc030a90 con 0x7f4f140cf9a0 2024-01-15T18:02:53.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.677+0000 7f4f13e00640 1 -- 172.21.15.26:0/2773218640 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc030bf0 con 0x7f4f140cf9a0 2024-01-15T18:02:53.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.677+0000 7f4f13e00640 1 -- 172.21.15.26:0/2773218640 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc038440 con 0x7f4f140cf9a0 2024-01-15T18:02:53.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.678+0000 7f4f1b600640 1 -- 172.21.15.26:0/2773218640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f140cf9a0 msgr2=0x7f4f1415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:53.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.678+0000 7f4f1b600640 1 --2- 172.21.15.26:0/2773218640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f140cf9a0 0x7f4f1415dca0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0097a0 tx=0x7f4efc02fce0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.678+0000 7f4f1b600640 1 -- 172.21.15.26:0/2773218640 shutdown_connections 2024-01-15T18:02:53.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.678+0000 7f4f1b600640 1 --2- 172.21.15.26:0/2773218640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f1415e1e0 0x7f4f140702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.678+0000 7f4f1b600640 1 --2- 172.21.15.26:0/2773218640 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f140cf9a0 0x7f4f1415dca0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.678+0000 7f4f1b600640 1 -- 172.21.15.26:0/2773218640 >> 172.21.15.26:0/2773218640 conn(0x7f4f1406a800 msgr2=0x7f4f1406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:53.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.678+0000 7f4f1b600640 1 -- 172.21.15.26:0/2773218640 shutdown_connections 2024-01-15T18:02:53.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.678+0000 7f4f1b600640 1 -- 172.21.15.26:0/2773218640 wait complete. 2024-01-15T18:02:53.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.680+0000 7f4f1b600640 1 Processor -- start 2024-01-15T18:02:53.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.680+0000 7f4f1b600640 1 -- start start 2024-01-15T18:02:53.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.681+0000 7f4f1b600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f140cf9a0 0x7f4f1413e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:53.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.681+0000 7f4f1b600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f14143ae0 0x7f4f140b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:53.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.681+0000 7f4f1b600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f1415e1e0 0x7f4f140b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:53.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.681+0000 7f4f1b600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1413f800 con 0x7f4f1415e1e0 2024-01-15T18:02:53.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.681+0000 7f4f1b600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1413eba0 con 0x7f4f140cf9a0 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.681+0000 7f4f1b600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1413ee60 con 0x7f4f14143ae0 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.681+0000 7f4f19600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f1415e1e0 0x7f4f140b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.681+0000 7f4f19600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f1415e1e0 0x7f4f140b1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34896/0 (socket says 172.21.15.26:34896) 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.681+0000 7f4f19600640 1 -- 172.21.15.26:0/3613814095 learned_addr learned my addr 172.21.15.26:0/3613814095 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.682+0000 7f4f19600640 1 -- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f14143ae0 msgr2=0x7f4f140b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.682+0000 7f4f19600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f14143ae0 0x7f4f140b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.682+0000 7f4f19600640 1 -- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f140cf9a0 msgr2=0x7f4f1413e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.682+0000 7f4f19600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f140cf9a0 0x7f4f1413e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.682+0000 7f4f19600640 1 -- 172.21.15.26:0/3613814095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc009400 con 0x7f4f1415e1e0 2024-01-15T18:02:53.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.682+0000 7f4f19600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f1415e1e0 0x7f4f140b1100 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c007e60 tx=0x7f4f0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:53.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.682+0000 7f4f11600640 1 -- 172.21.15.26:0/3613814095 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c014420 con 0x7f4f1415e1e0 2024-01-15T18:02:53.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.682+0000 7f4f11600640 1 -- 172.21.15.26:0/3613814095 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c0145a0 con 0x7f4f1415e1e0 2024-01-15T18:02:53.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.683+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1413f140 con 0x7f4f1415e1e0 2024-01-15T18:02:53.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.683+0000 7f4f11600640 1 -- 172.21.15.26:0/3613814095 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c014950 con 0x7f4f1415e1e0 2024-01-15T18:02:53.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.683+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1413fc20 con 0x7f4f1415e1e0 2024-01-15T18:02:53.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.684+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1413f940 con 0x7f4f1415e1e0 2024-01-15T18:02:53.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.685+0000 7f4f11600640 1 -- 172.21.15.26:0/3613814095 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f4f0c014ab0 con 0x7f4f1415e1e0 2024-01-15T18:02:53.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.686+0000 7f4f11600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4eec06bb80 0x7f4eec06e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:53.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.686+0000 7f4f11600640 1 -- 172.21.15.26:0/3613814095 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4f0c088f10 con 0x7f4f1415e1e0 2024-01-15T18:02:53.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.692+0000 7f4f11600640 1 -- 172.21.15.26:0/3613814095 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f1413f940 con 0x7f4f1415e1e0 2024-01-15T18:02:53.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.694+0000 7f4f18c00640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4eec06bb80 0x7f4eec06e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:53.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:53.695+0000 7f4f18c00640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4eec06bb80 0x7f4eec06e040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f04007fd0 tx=0x7f4f04007920 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:54.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.077+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20837", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f1413efd0 con 0x7f4eec06bb80 2024-01-15T18:02:54.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.125+0000 7f4f11600640 1 -- 172.21.15.26:0/3613814095 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f4f1413efd0 con 0x7f4eec06bb80 2024-01-15T18:02:54.131 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:02:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:02:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:02:54.131 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "2/3", 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:54.132 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:54.133 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4eec06bb80 msgr2=0x7f4eec06e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:54.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4eec06bb80 0x7f4eec06e040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4f04007fd0 tx=0x7f4f04007920 comp rx=0 tx=0).stop 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f1415e1e0 msgr2=0x7f4f140b1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f1415e1e0 0x7f4f140b1100 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c007e60 tx=0x7f4f0c00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 shutdown_connections 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f14143ae0 0x7f4f140b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4eec06bb80 0x7f4eec06e040 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f1415e1e0 0x7f4f140b1100 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 --2- 172.21.15.26:0/3613814095 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f140cf9a0 0x7f4f1413e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.130+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 >> 172.21.15.26:0/3613814095 conn(0x7f4f1406a800 msgr2=0x7f4f1406cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.131+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 shutdown_connections 2024-01-15T18:02:54.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:54.131+0000 7f4f1b600640 1 -- 172.21.15.26:0/3613814095 wait complete. 2024-01-15T18:02:54.164 INFO:tasks.cephfs.test_volumes:current clone state = in-progress 2024-01-15T18:02:55.165 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone20837 2024-01-15T18:02:55.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.352+0000 7f8b70800640 1 Processor -- start 2024-01-15T18:02:55.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.352+0000 7f8b70800640 1 -- start start 2024-01-15T18:02:55.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.354+0000 7f8b70800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b680cf9a0 0x7f8b6815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:55.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.354+0000 7f8b70800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b680b1ee0 con 0x7f8b68143ae0 2024-01-15T18:02:55.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.354+0000 7f8b70800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b680b2780 con 0x7f8b6815e1e0 2024-01-15T18:02:55.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.354+0000 7f8b70800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6815e990 con 0x7f8b680cf9a0 2024-01-15T18:02:55.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.354+0000 7f8b6de00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8b68143ae0 0x7f8b680cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54022/0 (socket says 172.21.15.26:54022) 2024-01-15T18:02:55.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.354+0000 7f8b6de00640 1 -- 172.21.15.26:0/844759402 learned_addr learned my addr 172.21.15.26:0/844759402 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:55.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.354+0000 7f8b6d400640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b680cf9a0 0x7f8b6815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.354+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760165976 0 0) 0x7f8b680b1ee0 con 0x7f8b68143ae0 2024-01-15T18:02:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b48005730 con 0x7f8b68143ae0 2024-01-15T18:02:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3087891800 0 0) 0x7f8b48005730 con 0x7f8b68143ae0 2024-01-15T18:02:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b680cf9a0 msgr2=0x7f8b6815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b680cf9a0 0x7f8b6815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 >> v1:172.21.15.124:6789/0 conn(0x7f8b6815e1e0 legacy=0x7f8b680702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680b2130 con 0x7f8b68143ae0 2024-01-15T18:02:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8b58001670 con 0x7f8b68143ae0 2024-01-15T18:02:55.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 >> v1:172.21.15.26:6789/0 conn(0x7f8b68143ae0 legacy=0x7f8b680cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b48005bb0 0x7f8b48005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b480069d0 0x7f8b48006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b680cf9a0 0x7f8b48007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6815e990 con 0x7f8b480069d0 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4800dc40 con 0x7f8b48005bb0 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4800de00 con 0x7f8b680cf9a0 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.355+0000 7f8b6d400640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b480069d0 0x7f8b48006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.356+0000 7f8b6de00640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b48005bb0 0x7f8b48005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.356+0000 7f8b6e800640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b680cf9a0 0x7f8b48007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.356+0000 7f8b6d400640 1 -- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b680cf9a0 msgr2=0x7f8b48007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.356+0000 7f8b6d400640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b680cf9a0 0x7f8b48007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.356+0000 7f8b6d400640 1 -- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b48005bb0 msgr2=0x7f8b48005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.356+0000 7f8b6d400640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b48005bb0 0x7f8b48005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.356+0000 7f8b6d400640 1 -- 172.21.15.26:0/844759402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b48005730 con 0x7f8b480069d0 2024-01-15T18:02:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.357+0000 7f8b6d400640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b480069d0 0x7f8b48006e50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009a90 tx=0x7f8b48007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2142e05ad64d188 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.357+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5002c680 con 0x7f8b480069d0 2024-01-15T18:02:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.357+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5002c7e0 con 0x7f8b480069d0 2024-01-15T18:02:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.357+0000 7f8b6ca00640 1 -- 172.21.15.26:0/844759402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5002cb70 con 0x7f8b480069d0 2024-01-15T18:02:55.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.358+0000 7f8b70800640 1 -- 172.21.15.26:0/844759402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b480069d0 msgr2=0x7f8b48006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:55.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.358+0000 7f8b70800640 1 --2- 172.21.15.26:0/844759402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b480069d0 0x7f8b48006e50 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009a90 tx=0x7f8b48007a00 comp rx=0 tx=0).stop 2024-01-15T18:02:55.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.358+0000 7f8b6e800640 1 -- 172.21.15.26:0/844759402 reap_dead start 2024-01-15T18:02:55.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.358+0000 7f8b70800640 1 -- 172.21.15.26:0/844759402 shutdown_connections 2024-01-15T18:02:55.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.358+0000 7f8b70800640 1 -- 172.21.15.26:0/844759402 >> 172.21.15.26:0/844759402 conn(0x7f8b6806a800 msgr2=0x7f8b6806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.358+0000 7f8b70800640 1 -- 172.21.15.26:0/844759402 shutdown_connections 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.358+0000 7f8b70800640 1 -- 172.21.15.26:0/844759402 wait complete. 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.360+0000 7f8b70800640 1 Processor -- start 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.360+0000 7f8b70800640 1 -- start start 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.361+0000 7f8b70800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b480069d0 0x7f8b6813ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.361+0000 7f8b70800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b68158ec0 0x7f8b681edd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.361+0000 7f8b70800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b68159340 0x7f8b681ee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.361+0000 7f8b70800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b681ee810 con 0x7f8b68159340 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.361+0000 7f8b70800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6813de70 con 0x7f8b480069d0 2024-01-15T18:02:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.361+0000 7f8b70800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b681598a0 con 0x7f8b68158ec0 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.361+0000 7f8b6e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b68159340 0x7f8b681ee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.361+0000 7f8b6e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b68159340 0x7f8b681ee2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34920/0 (socket says 172.21.15.26:34920) 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.361+0000 7f8b6e800640 1 -- 172.21.15.26:0/1174714551 learned_addr learned my addr 172.21.15.26:0/1174714551 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b6de00640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b480069d0 0x7f8b6813ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b6d400640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b68158ec0 0x7f8b681edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b6e800640 1 -- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b68158ec0 msgr2=0x7f8b681edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b6e800640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b68158ec0 0x7f8b681edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b6e800640 1 -- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b480069d0 msgr2=0x7f8b6813ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b6e800640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b480069d0 0x7f8b6813ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b6e800640 1 -- 172.21.15.26:0/1174714551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68159b80 con 0x7f8b68159340 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b6e800640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b68159340 0x7f8b681ee2d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8b48005b80 tx=0x7f8b6000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b66000640 1 -- 172.21.15.26:0/1174714551 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60003970 con 0x7f8b68159340 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b66000640 1 -- 172.21.15.26:0/1174714551 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b60003ad0 con 0x7f8b68159340 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b66000640 1 -- 172.21.15.26:0/1174714551 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60003e80 con 0x7f8b68159340 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.362+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b68159e60 con 0x7f8b68159340 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.363+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6813b3b0 con 0x7f8b68159340 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.364+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6813b5a0 con 0x7f8b68159340 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.365+0000 7f8b66000640 1 -- 172.21.15.26:0/1174714551 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f8b600050d0 con 0x7f8b68159340 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.366+0000 7f8b66000640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b4006bb30 0x7f8b4006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.366+0000 7f8b66000640 1 -- 172.21.15.26:0/1174714551 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8b60088410 con 0x7f8b68159340 2024-01-15T18:02:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.366+0000 7f8b6de00640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b4006bb30 0x7f8b4006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:55.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.367+0000 7f8b6de00640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b4006bb30 0x7f8b4006dff0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8b58001eb0 tx=0x7f8b58009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:55.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.371+0000 7f8b66000640 1 -- 172.21.15.26:0/1174714551 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b60054b00 con 0x7f8b68159340 2024-01-15T18:02:55.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:55.606+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone20837", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b6815e640 con 0x7f8b4006bb30 2024-01-15T18:02:56.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.330+0000 7f8b66000640 1 -- 172.21.15.26:0/1174714551 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f8b58005040 con 0x7f8b4006bb30 2024-01-15T18:02:56.335 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:02:56.335 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:02:56.335 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:02:56.335 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:56.335 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:02:56.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b4006bb30 msgr2=0x7f8b4006dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:56.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b4006bb30 0x7f8b4006dff0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8b58001eb0 tx=0x7f8b58009040 comp rx=0 tx=0).stop 2024-01-15T18:02:56.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b68159340 msgr2=0x7f8b681ee2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:56.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b68159340 0x7f8b681ee2d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8b48005b80 tx=0x7f8b6000ec10 comp rx=0 tx=0).stop 2024-01-15T18:02:56.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 shutdown_connections 2024-01-15T18:02:56.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b68158ec0 0x7f8b681edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b4006bb30 0x7f8b4006dff0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b68159340 0x7f8b681ee2d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 --2- 172.21.15.26:0/1174714551 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b480069d0 0x7f8b6813ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 >> 172.21.15.26:0/1174714551 conn(0x7f8b6806a800 msgr2=0x7f8b6815a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:56.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 shutdown_connections 2024-01-15T18:02:56.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.334+0000 7f8b70800640 1 -- 172.21.15.26:0/1174714551 wait complete. 2024-01-15T18:02:56.358 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:02:56.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.575+0000 7f49df400640 1 Processor -- start 2024-01-15T18:02:56.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.575+0000 7f49df400640 1 -- start start 2024-01-15T18:02:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.577+0000 7f49df400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d8143c40 0x7f49d8071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.577+0000 7f49df400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f49d80ccc10 0x7f49d806d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.577+0000 7f49df400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d8070e50 con 0x7f49d80717d0 2024-01-15T18:02:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.577+0000 7f49df400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d80723c0 con 0x7f49d80ccc10 2024-01-15T18:02:56.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.577+0000 7f49df400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8144020 con 0x7f49d8143c40 2024-01-15T18:02:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.578+0000 7f49dea00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f49d80ccc10 0x7f49d806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:56.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.578+0000 7f49de000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d8143c40 0x7f49d8071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.578+0000 7f49dea00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f49d80ccc10 0x7f49d806d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:42148/0 (socket says 172.21.15.26:42148) 2024-01-15T18:02:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.578+0000 7f49de000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d8143c40 0x7f49d8071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:44500/0 (socket says 172.21.15.26:44500) 2024-01-15T18:02:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.578+0000 7f49dea00640 1 -- 172.21.15.26:0/3793060149 learned_addr learned my addr 172.21.15.26:0/3793060149 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.578+0000 7f49de000640 1 -- 172.21.15.26:0/3793060149 >> v2:172.21.15.124:3300/0 conn(0x7f49d80ccc10 msgr2=0x7f49d806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.578+0000 7f49de000640 1 --2- 172.21.15.26:0/3793060149 >> v2:172.21.15.124:3300/0 conn(0x7f49d80ccc10 0x7f49d806d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.578+0000 7f49de000640 1 -- 172.21.15.26:0/3793060149 >> v1:172.21.15.124:6789/0 conn(0x7f49d80717d0 legacy=0x7f49d80cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:56.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.578+0000 7f49de000640 1 -- 172.21.15.26:0/3793060149 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d8071e00 con 0x7f49d8143c40 2024-01-15T18:02:56.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.580+0000 7f49de000640 1 --2- 172.21.15.26:0/3793060149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d8143c40 0x7f49d8071290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009d90 tx=0x7f49d002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af9ac347f4ad0c21 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:56.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.580+0000 7f49dcc00640 1 -- 172.21.15.26:0/3793060149 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d002cbc0 con 0x7f49d8143c40 2024-01-15T18:02:56.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.580+0000 7f49dcc00640 1 -- 172.21.15.26:0/3793060149 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d002cd20 con 0x7f49d8143c40 2024-01-15T18:02:56.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.580+0000 7f49dcc00640 1 -- 172.21.15.26:0/3793060149 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0034660 con 0x7f49d8143c40 2024-01-15T18:02:56.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.581+0000 7f49df400640 1 -- 172.21.15.26:0/3793060149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d8143c40 msgr2=0x7f49d8071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:56.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.581+0000 7f49df400640 1 --2- 172.21.15.26:0/3793060149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d8143c40 0x7f49d8071290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009d90 tx=0x7f49d002be10 comp rx=0 tx=0).stop 2024-01-15T18:02:56.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.581+0000 7f49df400640 1 -- 172.21.15.26:0/3793060149 shutdown_connections 2024-01-15T18:02:56.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.581+0000 7f49df400640 1 --2- 172.21.15.26:0/3793060149 >> v2:172.21.15.124:3300/0 conn(0x7f49d80ccc10 0x7f49d806d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.581+0000 7f49df400640 1 --2- 172.21.15.26:0/3793060149 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d8143c40 0x7f49d8071290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.581+0000 7f49df400640 1 -- 172.21.15.26:0/3793060149 >> 172.21.15.26:0/3793060149 conn(0x7f49d806a800 msgr2=0x7f49d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:56.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.581+0000 7f49df400640 1 -- 172.21.15.26:0/3793060149 shutdown_connections 2024-01-15T18:02:56.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.581+0000 7f49df400640 1 -- 172.21.15.26:0/3793060149 wait complete. 2024-01-15T18:02:56.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.584+0000 7f49df400640 1 Processor -- start 2024-01-15T18:02:56.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.584+0000 7f49df400640 1 -- start start 2024-01-15T18:02:56.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.585+0000 7f49df400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d80717d0 0x7f49d815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:56.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.585+0000 7f49df400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d80ccc10 0x7f49d81e0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:56.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.585+0000 7f49df400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d8143c40 0x7f49d80ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:56.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.585+0000 7f49df400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80ab930 con 0x7f49d80717d0 2024-01-15T18:02:56.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.585+0000 7f49df400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80acae0 con 0x7f49d8143c40 2024-01-15T18:02:56.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.585+0000 7f49df400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d80acdc0 con 0x7f49d80ccc10 2024-01-15T18:02:56.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.585+0000 7f49de000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d80717d0 0x7f49d815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:56.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.585+0000 7f49de000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d80717d0 0x7f49d815a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34930/0 (socket says 172.21.15.26:34930) 2024-01-15T18:02:56.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.585+0000 7f49de000640 1 -- 172.21.15.26:0/4225956732 learned_addr learned my addr 172.21.15.26:0/4225956732 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:56.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.586+0000 7f49de000640 1 -- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d80ccc10 msgr2=0x7f49d81e0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:56.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.586+0000 7f49dea00640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d8143c40 0x7f49d80ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:56.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.586+0000 7f49de000640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d80ccc10 0x7f49d81e0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.586+0000 7f49de000640 1 -- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d8143c40 msgr2=0x7f49d80ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:56.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.586+0000 7f49de000640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d8143c40 0x7f49d80ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.586+0000 7f49de000640 1 -- 172.21.15.26:0/4225956732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009a70 con 0x7f49d80717d0 2024-01-15T18:02:56.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.586+0000 7f49dea00640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d8143c40 0x7f49d80ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:56.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.586+0000 7f49de000640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d80717d0 0x7f49d815a130 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009d90 tx=0x7f49d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:56.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.587+0000 7f49d6000640 1 -- 172.21.15.26:0/4225956732 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0040650 con 0x7f49d80717d0 2024-01-15T18:02:56.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.587+0000 7f49d6000640 1 -- 172.21.15.26:0/4225956732 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d002ce90 con 0x7f49d80717d0 2024-01-15T18:02:56.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.587+0000 7f49d6000640 1 -- 172.21.15.26:0/4225956732 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00409f0 con 0x7f49d80717d0 2024-01-15T18:02:56.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.587+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d80ad0a0 con 0x7f49d80717d0 2024-01-15T18:02:56.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.587+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d80ab5c0 con 0x7f49d80717d0 2024-01-15T18:02:56.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.588+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d80ad2c0 con 0x7f49d80717d0 2024-01-15T18:02:56.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.589+0000 7f49d6000640 1 -- 172.21.15.26:0/4225956732 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f49d0040b50 con 0x7f49d80717d0 2024-01-15T18:02:56.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.590+0000 7f49d6000640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f49b006bb30 0x7f49b006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:56.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.590+0000 7f49d6000640 1 -- 172.21.15.26:0/4225956732 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f49d00ab290 con 0x7f49d80717d0 2024-01-15T18:02:56.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.591+0000 7f49dd600640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f49b006bb30 0x7f49b006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:56.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.592+0000 7f49dd600640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f49b006bb30 0x7f49b006dff0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f49c0000f60 tx=0x7f49c00079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:56.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.595+0000 7f49d6000640 1 -- 172.21.15.26:0/4225956732 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49d00779e0 con 0x7f49d80717d0 2024-01-15T18:02:56.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.817+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f49d80acc50 con 0x7f49b006bb30 2024-01-15T18:02:56.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.820+0000 7f49d6000640 1 -- 172.21.15.26:0/4225956732 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f49d80acc50 con 0x7f49b006bb30 2024-01-15T18:02:56.824 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:02:56.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:02:56.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:56.825 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:02:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f49b006bb30 msgr2=0x7f49b006dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f49b006bb30 0x7f49b006dff0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f49c0000f60 tx=0x7f49c00079b0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d80717d0 msgr2=0x7f49d815a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d80717d0 0x7f49d815a130 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009d90 tx=0x7f49d0004a40 comp rx=0 tx=0).stop 2024-01-15T18:02:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 shutdown_connections 2024-01-15T18:02:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f49d80ccc10 0x7f49d81e0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f49b006bb30 0x7f49b006dff0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f49d8143c40 0x7f49d80ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 --2- 172.21.15.26:0/4225956732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f49d80717d0 0x7f49d815a130 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 >> 172.21.15.26:0/4225956732 conn(0x7f49d806a800 msgr2=0x7f49d80b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 shutdown_connections 2024-01-15T18:02:56.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:56.823+0000 7f49df400640 1 -- 172.21.15.26:0/4225956732 wait complete. 2024-01-15T18:02:56.847 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:02:57.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.070+0000 7f4af5400640 1 Processor -- start 2024-01-15T18:02:57.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.070+0000 7f4af5400640 1 -- start start 2024-01-15T18:02:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.072+0000 7f4af5400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4af015d4b0 0x7f4af00d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.072+0000 7f4af5400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4af00cf9a0 0x7f4af00cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.072+0000 7f4af5400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 0x7f4af00ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.072+0000 7f4af5400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af00d0320 con 0x7f4af015d4b0 2024-01-15T18:02:57.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.072+0000 7f4af5400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af00b2730 con 0x7f4af00cf9a0 2024-01-15T18:02:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.072+0000 7f4af5400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00d1ec0 con 0x7f4af00cddf0 2024-01-15T18:02:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.072+0000 7f4aefe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4af015d4b0 0x7f4af00d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.073+0000 7f4aefe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4af015d4b0 0x7f4af00d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34946/0 (socket says 172.21.15.26:34946) 2024-01-15T18:02:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.073+0000 7f4aefe00640 1 -- 172.21.15.26:0/2018244846 learned_addr learned my addr 172.21.15.26:0/2018244846 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.073+0000 7f4aef400640 1 --2- 172.21.15.26:0/2018244846 >> v2:172.21.15.124:3300/0 conn(0x7f4af00cf9a0 0x7f4af00cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.073+0000 7f4af4a00640 1 --2- 172.21.15.26:0/2018244846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 0x7f4af00ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.073+0000 7f4aefe00640 1 -- 172.21.15.26:0/2018244846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 msgr2=0x7f4af00ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.073+0000 7f4aefe00640 1 --2- 172.21.15.26:0/2018244846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 0x7f4af00ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.073+0000 7f4aefe00640 1 -- 172.21.15.26:0/2018244846 >> v2:172.21.15.124:3300/0 conn(0x7f4af00cf9a0 msgr2=0x7f4af00cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:57.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.073+0000 7f4aefe00640 1 --2- 172.21.15.26:0/2018244846 >> v2:172.21.15.124:3300/0 conn(0x7f4af00cf9a0 0x7f4af00cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.073+0000 7f4aefe00640 1 -- 172.21.15.26:0/2018244846 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00b29d0 con 0x7f4af015d4b0 2024-01-15T18:02:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.074+0000 7f4aefe00640 1 --2- 172.21.15.26:0/2018244846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af015d4b0 0x7f4af00d1550 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4af008b580 tx=0x7f4ae002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b61a2d4a08dd760 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.074+0000 7f4aeea00640 1 -- 172.21.15.26:0/2018244846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0031870 con 0x7f4af015d4b0 2024-01-15T18:02:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4aeea00640 1 -- 172.21.15.26:0/2018244846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae00319d0 con 0x7f4af015d4b0 2024-01-15T18:02:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4aeea00640 1 -- 172.21.15.26:0/2018244846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0031d60 con 0x7f4af015d4b0 2024-01-15T18:02:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4af5400640 1 -- 172.21.15.26:0/2018244846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af015d4b0 msgr2=0x7f4af00d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4af5400640 1 --2- 172.21.15.26:0/2018244846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af015d4b0 0x7f4af00d1550 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4af008b580 tx=0x7f4ae002b910 comp rx=0 tx=0).stop 2024-01-15T18:02:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4af5400640 1 -- 172.21.15.26:0/2018244846 shutdown_connections 2024-01-15T18:02:57.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4af5400640 1 --2- 172.21.15.26:0/2018244846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 0x7f4af00ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4af5400640 1 --2- 172.21.15.26:0/2018244846 >> v2:172.21.15.124:3300/0 conn(0x7f4af00cf9a0 0x7f4af00cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4af5400640 1 --2- 172.21.15.26:0/2018244846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af015d4b0 0x7f4af00d1550 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4af5400640 1 -- 172.21.15.26:0/2018244846 >> 172.21.15.26:0/2018244846 conn(0x7f4af00d3000 msgr2=0x7f4af00d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.075+0000 7f4af5400640 1 -- 172.21.15.26:0/2018244846 shutdown_connections 2024-01-15T18:02:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.076+0000 7f4af5400640 1 -- 172.21.15.26:0/2018244846 wait complete. 2024-01-15T18:02:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.077+0000 7f4af5400640 1 Processor -- start 2024-01-15T18:02:57.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.077+0000 7f4af5400640 1 -- start start 2024-01-15T18:02:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.078+0000 7f4af5400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 0x7f4af009cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4af5400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af00cf9a0 0x7f4af00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4af5400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4af015d4b0 0x7f4af01f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4af5400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af009d2c0 con 0x7f4af00cf9a0 2024-01-15T18:02:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4af5400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af009d5a0 con 0x7f4af015d4b0 2024-01-15T18:02:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4af5400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af009d880 con 0x7f4af00cddf0 2024-01-15T18:02:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af00cf9a0 0x7f4af00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af00cf9a0 0x7f4af00ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34954/0 (socket says 172.21.15.26:34954) 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aef400640 1 -- 172.21.15.26:0/4001791224 learned_addr learned my addr 172.21.15.26:0/4001791224 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4af4a00640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4af015d4b0 0x7f4af01f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aef400640 1 -- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 msgr2=0x7f4af009cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aefe00640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 0x7f4af009cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aef400640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 0x7f4af009cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aef400640 1 -- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4af015d4b0 msgr2=0x7f4af01f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aef400640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4af015d4b0 0x7f4af01f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aef400640 1 -- 172.21.15.26:0/4001791224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae00094d0 con 0x7f4af00cf9a0 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4af4a00640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4af015d4b0 0x7f4af01f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:57.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.079+0000 7f4aefe00640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 0x7f4af009cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.080+0000 7f4aef400640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af00cf9a0 0x7f4af00ccd50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800d800 tx=0x7f4ad800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.080+0000 7f4ae7e00640 1 -- 172.21.15.26:0/4001791224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800b6c0 con 0x7f4af00cf9a0 2024-01-15T18:02:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.080+0000 7f4ae7e00640 1 -- 172.21.15.26:0/4001791224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad800b820 con 0x7f4af00cf9a0 2024-01-15T18:02:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.080+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af015da10 con 0x7f4af00cf9a0 2024-01-15T18:02:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.080+0000 7f4ae7e00640 1 -- 172.21.15.26:0/4001791224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800bbb0 con 0x7f4af00cf9a0 2024-01-15T18:02:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.080+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af009de40 con 0x7f4af00cf9a0 2024-01-15T18:02:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.081+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af0067450 con 0x7f4af00cf9a0 2024-01-15T18:02:57.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.082+0000 7f4ae7e00640 1 -- 172.21.15.26:0/4001791224 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f4ad800bd10 con 0x7f4af00cf9a0 2024-01-15T18:02:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.083+0000 7f4ae7e00640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ac806bb30 0x7f4ac806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.083+0000 7f4ae7e00640 1 -- 172.21.15.26:0/4001791224 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4ad8088b70 con 0x7f4af00cf9a0 2024-01-15T18:02:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.084+0000 7f4aefe00640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ac806bb30 0x7f4ac806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.086+0000 7f4aefe00640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ac806bb30 0x7f4ac806dff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0036000 tx=0x7f4ae003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:57.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.088+0000 7f4ae7e00640 1 -- 172.21.15.26:0/4001791224 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ad80552c0 con 0x7f4af00cf9a0 2024-01-15T18:02:57.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.312+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4af015d890 con 0x7f4ac806bb30 2024-01-15T18:02:57.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.336+0000 7f4ae7e00640 1 -- 172.21.15.26:0/4001791224 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+381 (secure 0 0 0) 0x7f4ae0031bc0 con 0x7f4ac806bb30 2024-01-15T18:02:57.340 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:02:57.340 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone20834" 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sv1" 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone20837" 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:57.341 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone20836" 2024-01-15T18:02:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone20832" 2024-01-15T18:02:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone20835" 2024-01-15T18:02:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:57.342 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone20831" 2024-01-15T18:02:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:02:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:02:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone20833" 2024-01-15T18:02:57.343 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:02:57.343 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:02:57.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ac806bb30 msgr2=0x7f4ac806dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ac806bb30 0x7f4ac806dff0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0036000 tx=0x7f4ae003c040 comp rx=0 tx=0).stop 2024-01-15T18:02:57.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af00cf9a0 msgr2=0x7f4af00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af00cf9a0 0x7f4af00ccd50 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800d800 tx=0x7f4ad800dcc0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 shutdown_connections 2024-01-15T18:02:57.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4ac806bb30 0x7f4ac806dff0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4af015d4b0 0x7f4af01f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4af00cf9a0 0x7f4af00ccd50 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 --2- 172.21.15.26:0/4001791224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4af00cddf0 0x7f4af009cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 >> 172.21.15.26:0/4001791224 conn(0x7f4af00d3000 msgr2=0x7f4af00b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:57.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 shutdown_connections 2024-01-15T18:02:57.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.339+0000 7f4af5400640 1 -- 172.21.15.26:0/4001791224 wait complete. 2024-01-15T18:02:57.364 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone20834 2024-01-15T18:02:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.587+0000 7f8f3b600640 1 Processor -- start 2024-01-15T18:02:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.588+0000 7f8f3b600640 1 -- start start 2024-01-15T18:02:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.589+0000 7f8f3b600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8f340cf9a0 0x7f8f34166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.589+0000 7f8f3b600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3415c5f0 con 0x7f8f3413e360 2024-01-15T18:02:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.589+0000 7f8f3b600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3406c470 con 0x7f8f3413df80 2024-01-15T18:02:57.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.589+0000 7f8f3b600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3406c750 con 0x7f8f340cf9a0 2024-01-15T18:02:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.590+0000 7f8f33e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8f3413e360 0x7f8f340cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54026/0 (socket says 172.21.15.26:54026) 2024-01-15T18:02:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.590+0000 7f8f33e00640 1 -- 172.21.15.26:0/3808503804 learned_addr learned my addr 172.21.15.26:0/3808503804 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.590+0000 7f8f39600640 1 --2- 172.21.15.26:0/3808503804 >> v2:172.21.15.124:3300/0 conn(0x7f8f340cf9a0 0x7f8f34166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.590+0000 7f8f33400640 1 -- 172.21.15.26:0/3808503804 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874813643 0 0) 0x7f8f3415c5f0 con 0x7f8f3413e360 2024-01-15T18:02:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.590+0000 7f8f33400640 1 -- 172.21.15.26:0/3808503804 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f14003520 con 0x7f8f3413e360 2024-01-15T18:02:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.591+0000 7f8f33400640 1 -- 172.21.15.26:0/3808503804 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1093307213 0 0) 0x7f8f14003520 con 0x7f8f3413e360 2024-01-15T18:02:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.591+0000 7f8f39600640 1 -- 172.21.15.26:0/3808503804 >> v1:172.21.15.124:6789/0 conn(0x7f8f3413df80 legacy=0x7f8f3415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.591+0000 7f8f39600640 1 -- 172.21.15.26:0/3808503804 >> v1:172.21.15.26:6789/0 conn(0x7f8f3413e360 legacy=0x7f8f340cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.591+0000 7f8f39600640 1 -- 172.21.15.26:0/3808503804 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3406ca30 con 0x7f8f340cf9a0 2024-01-15T18:02:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.592+0000 7f8f39600640 1 --2- 172.21.15.26:0/3808503804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 0x7f8f34166030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0097a0 tx=0x7f8f2c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6777043fcf987f1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.593+0000 7f8f33400640 1 -- 172.21.15.26:0/3808503804 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c02f050 con 0x7f8f340cf9a0 2024-01-15T18:02:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.593+0000 7f8f33400640 1 -- 172.21.15.26:0/3808503804 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c031500 con 0x7f8f340cf9a0 2024-01-15T18:02:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.593+0000 7f8f33400640 1 -- 172.21.15.26:0/3808503804 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f240019f0 con 0x7f8f340cf9a0 2024-01-15T18:02:57.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.593+0000 7f8f3b600640 1 -- 172.21.15.26:0/3808503804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 msgr2=0x7f8f34166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.593+0000 7f8f3b600640 1 --2- 172.21.15.26:0/3808503804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 0x7f8f34166030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c0097a0 tx=0x7f8f2c02b820 comp rx=0 tx=0).stop 2024-01-15T18:02:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.593+0000 7f8f3b600640 1 -- 172.21.15.26:0/3808503804 shutdown_connections 2024-01-15T18:02:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.593+0000 7f8f3b600640 1 --2- 172.21.15.26:0/3808503804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 0x7f8f34166030 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.594+0000 7f8f3b600640 1 -- 172.21.15.26:0/3808503804 >> 172.21.15.26:0/3808503804 conn(0x7f8f3406a800 msgr2=0x7f8f3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.594+0000 7f8f3b600640 1 -- 172.21.15.26:0/3808503804 shutdown_connections 2024-01-15T18:02:57.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.594+0000 7f8f3b600640 1 -- 172.21.15.26:0/3808503804 wait complete. 2024-01-15T18:02:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.595+0000 7f8f3b600640 1 Processor -- start 2024-01-15T18:02:57.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.596+0000 7f8f3b600640 1 -- start start 2024-01-15T18:02:57.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f3b600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 0x7f8f34070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f3b600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3413df80 0x7f8f341f0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f3b600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f3413e360 0x7f8f34159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f3b600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f34071980 con 0x7f8f3413df80 2024-01-15T18:02:57.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f3b600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f341f0880 con 0x7f8f340cf9a0 2024-01-15T18:02:57.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f3b600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f34070e00 con 0x7f8f3413e360 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3413df80 0x7f8f341f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3413df80 0x7f8f341f0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34960/0 (socket says 172.21.15.26:34960) 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f33e00640 1 -- 172.21.15.26:0/3464304346 learned_addr learned my addr 172.21.15.26:0/3464304346 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f39600640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f3413e360 0x7f8f34159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f38c00640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 0x7f8f34070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f33e00640 1 -- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f3413e360 msgr2=0x7f8f34159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f33e00640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f3413e360 0x7f8f34159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f33e00640 1 -- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 msgr2=0x7f8f34070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f33e00640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 0x7f8f34070770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f33e00640 1 -- 172.21.15.26:0/3464304346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f2c009400 con 0x7f8f3413df80 2024-01-15T18:02:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f38c00640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 0x7f8f34070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.597+0000 7f8f39600640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f3413e360 0x7f8f34159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.598+0000 7f8f33e00640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3413df80 0x7f8f341f0340 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8f240018d0 tx=0x7f8f2400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.598+0000 7f8f30c00640 1 -- 172.21.15.26:0/3464304346 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24012400 con 0x7f8f3413df80 2024-01-15T18:02:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.598+0000 7f8f30c00640 1 -- 172.21.15.26:0/3464304346 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f24012560 con 0x7f8f3413df80 2024-01-15T18:02:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.598+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f340710e0 con 0x7f8f3413df80 2024-01-15T18:02:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.598+0000 7f8f30c00640 1 -- 172.21.15.26:0/3464304346 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24012970 con 0x7f8f3413df80 2024-01-15T18:02:57.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.598+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f340716a0 con 0x7f8f3413df80 2024-01-15T18:02:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.599+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f340675d0 con 0x7f8f3413df80 2024-01-15T18:02:57.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.600+0000 7f8f30c00640 1 -- 172.21.15.26:0/3464304346 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f8f24012ad0 con 0x7f8f3413df80 2024-01-15T18:02:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.602+0000 7f8f30c00640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f0c06bb60 0x7f8f0c06e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.602+0000 7f8f30c00640 1 -- 172.21.15.26:0/3464304346 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8f24088710 con 0x7f8f3413df80 2024-01-15T18:02:57.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.603+0000 7f8f38c00640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f0c06bb60 0x7f8f0c06e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:57.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.604+0000 7f8f38c00640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f0c06bb60 0x7f8f0c06e020 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c000f60 tx=0x7f8f1c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:57.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.606+0000 7f8f30c00640 1 -- 172.21.15.26:0/3464304346 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f24054eb0 con 0x7f8f3413df80 2024-01-15T18:02:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.829+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone20834", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f3406cff0 con 0x7f8f0c06bb60 2024-01-15T18:02:57.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.874+0000 7f8f30c00640 1 -- 172.21.15.26:0/3464304346 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8f1c001340 con 0x7f8f0c06bb60 2024-01-15T18:02:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.876+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f0c06bb60 msgr2=0x7f8f0c06e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.876+0000 7f8f3b600640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f0c06bb60 0x7f8f0c06e020 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c000f60 tx=0x7f8f1c009040 comp rx=0 tx=0).stop 2024-01-15T18:02:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.876+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3413df80 msgr2=0x7f8f341f0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.876+0000 7f8f3b600640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3413df80 0x7f8f341f0340 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8f240018d0 tx=0x7f8f2400a430 comp rx=0 tx=0).stop 2024-01-15T18:02:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.876+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 shutdown_connections 2024-01-15T18:02:57.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.876+0000 7f8f3b600640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8f3413e360 0x7f8f34159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.876+0000 7f8f3b600640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8f0c06bb60 0x7f8f0c06e020 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.876+0000 7f8f3b600640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8f3413df80 0x7f8f341f0340 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.876+0000 7f8f3b600640 1 --2- 172.21.15.26:0/3464304346 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8f340cf9a0 0x7f8f34070770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.877+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 >> 172.21.15.26:0/3464304346 conn(0x7f8f3406a800 msgr2=0x7f8f34153a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.877+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 shutdown_connections 2024-01-15T18:02:57.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:57.877+0000 7f8f3b600640 1 -- 172.21.15.26:0/3464304346 wait complete. 2024-01-15T18:02:57.884 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.881+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:57.884 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.881+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:57.884 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.881+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:57.884 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.881+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:57.884 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.881+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:57.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:02:57.908 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.905+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:57.908 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.905+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:57.908 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.905+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:57.908 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.905+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:57.908 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:02:57.905+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:02:58.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.127+0000 7f318a200640 1 Processor -- start 2024-01-15T18:02:58.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.127+0000 7f318a200640 1 -- start start 2024-01-15T18:02:58.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.129+0000 7f318a200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f318406d8d0 0x7f318416c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:58.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.129+0000 7f318a200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840702e0 con 0x7f3184162430 2024-01-15T18:02:58.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.129+0000 7f318a200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840705c0 con 0x7f3184162810 2024-01-15T18:02:58.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.129+0000 7f318a200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31840708a0 con 0x7f318406d8d0 2024-01-15T18:02:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.129+0000 7f3188e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3184162430 0x7f31840b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:54040/0 (socket says 172.21.15.26:54040) 2024-01-15T18:02:58.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.129+0000 7f3188e00640 1 -- 172.21.15.26:0/2833900181 learned_addr learned my addr 172.21.15.26:0/2833900181 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:02:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.129+0000 7f3189800640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f318406d8d0 0x7f318416c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172494069 0 0) 0x7f31840702e0 con 0x7f3184162430 2024-01-15T18:02:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3164003520 con 0x7f3184162430 2024-01-15T18:02:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1006003478 0 0) 0x7f3164003520 con 0x7f3184162430 2024-01-15T18:02:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f318406d8d0 msgr2=0x7f318416c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f318406d8d0 0x7f318416c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 >> v1:172.21.15.124:6789/0 conn(0x7f3184162810 legacy=0x7f3184071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:58.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184070b80 con 0x7f3184162430 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f3174001670 con 0x7f3184162430 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 >> v1:172.21.15.26:6789/0 conn(0x7f3184162430 legacy=0x7f31840b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f318406d8d0 0x7f31640039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3164003ee0 0x7f31640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3164004d00 0x7f3164005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31840708a0 con 0x7f3164003ee0 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316400b9e0 con 0x7f3164004d00 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.130+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316400bba0 con 0x7f318406d8d0 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.131+0000 7f3183e00640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3164003ee0 0x7f31640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.131+0000 7f3188e00640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f318406d8d0 0x7f31640039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.131+0000 7f3189800640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3164004d00 0x7f3164005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:58.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.131+0000 7f3183e00640 1 -- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f318406d8d0 msgr2=0x7f31640039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.131+0000 7f3183e00640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f318406d8d0 0x7f31640039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.131+0000 7f3183e00640 1 -- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3164004d00 msgr2=0x7f3164005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:02:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.131+0000 7f3183e00640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3164004d00 0x7f3164005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.131+0000 7f3183e00640 1 -- 172.21.15.26:0/2833900181 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3164003520 con 0x7f3164003ee0 2024-01-15T18:02:58.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.131+0000 7f3189800640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3164004d00 0x7f3164005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:02:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.132+0000 7f3183e00640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3164003ee0 0x7f31640047c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f316c001830 tx=0x7f31640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc67907b74e82de server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.133+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c032600 con 0x7f3164003ee0 2024-01-15T18:02:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.133+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c02e050 con 0x7f3164003ee0 2024-01-15T18:02:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.133+0000 7f3183400640 1 -- 172.21.15.26:0/2833900181 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c0329c0 con 0x7f3164003ee0 2024-01-15T18:02:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.133+0000 7f318a200640 1 -- 172.21.15.26:0/2833900181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3164003ee0 msgr2=0x7f31640047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.133+0000 7f318a200640 1 --2- 172.21.15.26:0/2833900181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3164003ee0 0x7f31640047c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f316c001830 tx=0x7f31640057f0 comp rx=0 tx=0).stop 2024-01-15T18:02:58.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.133+0000 7f3189800640 1 -- 172.21.15.26:0/2833900181 reap_dead start 2024-01-15T18:02:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.133+0000 7f318a200640 1 -- 172.21.15.26:0/2833900181 shutdown_connections 2024-01-15T18:02:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.133+0000 7f318a200640 1 -- 172.21.15.26:0/2833900181 >> 172.21.15.26:0/2833900181 conn(0x7f318406a800 msgr2=0x7f318406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:02:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.133+0000 7f318a200640 1 -- 172.21.15.26:0/2833900181 shutdown_connections 2024-01-15T18:02:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.134+0000 7f318a200640 1 -- 172.21.15.26:0/2833900181 wait complete. 2024-01-15T18:02:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.135+0000 7f318a200640 1 Processor -- start 2024-01-15T18:02:58.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.135+0000 7f318a200640 1 -- start start 2024-01-15T18:02:58.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.136+0000 7f318a200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3164003ee0 0x7f31841f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.136+0000 7f318a200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3184147ee0 0x7f3184148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.136+0000 7f318a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3184148e70 0x7f3184164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.136+0000 7f318a200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318414a120 con 0x7f3184148e70 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.136+0000 7f318a200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318414b790 con 0x7f3184147ee0 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.136+0000 7f318a200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31841494a0 con 0x7f3164003ee0 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.137+0000 7f3189800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3184148e70 0x7f3184164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.137+0000 7f3189800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3184148e70 0x7f3184164dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34972/0 (socket says 172.21.15.26:34972) 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.137+0000 7f3189800640 1 -- 172.21.15.26:0/2145310711 learned_addr learned my addr 172.21.15.26:0/2145310711 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.137+0000 7f3189800640 1 -- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3164003ee0 msgr2=0x7f31841f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.137+0000 7f3189800640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3164003ee0 0x7f31841f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.137+0000 7f3189800640 1 -- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3184147ee0 msgr2=0x7f3184148930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.137+0000 7f3189800640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3184147ee0 0x7f3184148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.137+0000 7f3189800640 1 -- 172.21.15.26:0/2145310711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184149780 con 0x7f3184148e70 2024-01-15T18:02:58.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.137+0000 7f3189800640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3184148e70 0x7f3184164dd0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3184072810 tx=0x7f317c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.138+0000 7f3180c00640 1 -- 172.21.15.26:0/2145310711 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c017400 con 0x7f3184148e70 2024-01-15T18:02:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.138+0000 7f3180c00640 1 -- 172.21.15.26:0/2145310711 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c011040 con 0x7f3184148e70 2024-01-15T18:02:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.138+0000 7f3180c00640 1 -- 172.21.15.26:0/2145310711 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c017780 con 0x7f3184148e70 2024-01-15T18:02:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.138+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3184149a60 con 0x7f3184148e70 2024-01-15T18:02:58.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.138+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3184148360 con 0x7f3184148e70 2024-01-15T18:02:58.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.139+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3184067450 con 0x7f3184148e70 2024-01-15T18:02:58.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.140+0000 7f3180c00640 1 -- 172.21.15.26:0/2145310711 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 19) v1 ==== 90304+0+0 (secure 0 0 0) 0x7f317c0203b0 con 0x7f3184148e70 2024-01-15T18:02:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.141+0000 7f3180c00640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f315c06bb30 0x7f315c06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:02:58.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.141+0000 7f3180c00640 1 -- 172.21.15.26:0/2145310711 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f317c088850 con 0x7f3184148e70 2024-01-15T18:02:58.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.142+0000 7f3188e00640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f315c06bb30 0x7f315c06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:02:58.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.143+0000 7f3188e00640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f315c06bb30 0x7f315c06dff0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3174005990 tx=0x7f31740011a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:02:58.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.145+0000 7f3180c00640 1 -- 172.21.15.26:0/2145310711 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f317c054fa0 con 0x7f3184148e70 2024-01-15T18:02:58.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.342+0000 7f3180c00640 1 -- 172.21.15.26:0/2145310711 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f317c055370 con 0x7f3184148e70 2024-01-15T18:02:58.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:02:58.380+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f31840b3820 con 0x7f315c06bb30 2024-01-15T18:02:58.398 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:02:58.405 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:03:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.709+0000 7f3180c00640 1 -- 172.21.15.26:0/2145310711 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f317400e780 con 0x7f315c06bb30 2024-01-15T18:03:01.714 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:01.714 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:03:01.714 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:01.714 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1" 2024-01-15T18:03:01.714 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:01.714 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:03:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f315c06bb30 msgr2=0x7f315c06dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f315c06bb30 0x7f315c06dff0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3174005990 tx=0x7f31740011a0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3184148e70 msgr2=0x7f3184164dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3184148e70 0x7f3184164dd0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3184072810 tx=0x7f317c00f4a0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 shutdown_connections 2024-01-15T18:03:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f315c06bb30 0x7f315c06dff0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3184148e70 0x7f3184164dd0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3184147ee0 0x7f3184148930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 --2- 172.21.15.26:0/2145310711 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3164003ee0 0x7f31841f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 >> 172.21.15.26:0/2145310711 conn(0x7f318406a800 msgr2=0x7f3184158190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.712+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 shutdown_connections 2024-01-15T18:03:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.713+0000 7f318a200640 1 -- 172.21.15.26:0/2145310711 wait complete. 2024-01-15T18:03:01.738 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:03:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.969+0000 7f236d000640 1 Processor -- start 2024-01-15T18:03:01.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.969+0000 7f236d000640 1 -- start start 2024-01-15T18:03:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.971+0000 7f236d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f236806c470 0x7f23680cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.971+0000 7f236d000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f236806dab0 0x7f23680ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.971+0000 7f236d000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f236806ca10 con 0x7f23681447f0 2024-01-15T18:03:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.971+0000 7f236d000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f236806d690 con 0x7f236806c470 2024-01-15T18:03:01.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.971+0000 7f236d000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23680cebb0 con 0x7f236806dab0 2024-01-15T18:03:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.971+0000 7f235fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f236806c470 0x7f23680cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.971+0000 7f235fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f236806c470 0x7f23680cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60914/0 (socket says 172.21.15.26:60914) 2024-01-15T18:03:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.971+0000 7f235fe00640 1 -- 172.21.15.26:0/275395355 learned_addr learned my addr 172.21.15.26:0/275395355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.972+0000 7f2367e00640 1 --2- 172.21.15.26:0/275395355 >> v2:172.21.15.124:3300/0 conn(0x7f236806dab0 0x7f23680ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.972+0000 7f235fe00640 1 -- 172.21.15.26:0/275395355 >> v2:172.21.15.124:3300/0 conn(0x7f236806dab0 msgr2=0x7f23680ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.972+0000 7f235fe00640 1 --2- 172.21.15.26:0/275395355 >> v2:172.21.15.124:3300/0 conn(0x7f236806dab0 0x7f23680ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.972+0000 7f235fe00640 1 -- 172.21.15.26:0/275395355 >> v1:172.21.15.26:6789/0 conn(0x7f23681447f0 legacy=0x7f236815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.972+0000 7f235fe00640 1 -- 172.21.15.26:0/275395355 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236806ccf0 con 0x7f236806c470 2024-01-15T18:03:01.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.972+0000 7f2366a00640 1 -- 172.21.15.26:0/275395355 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406821142 0 0) 0x7f236806ca10 con 0x7f23681447f0 2024-01-15T18:03:01.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.973+0000 7f235fe00640 1 --2- 172.21.15.26:0/275395355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236806c470 0x7f23680cc5e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f23680aea60 tx=0x7f235002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe8f23d4d4ee408 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.973+0000 7f2366a00640 1 -- 172.21.15.26:0/275395355 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350031870 con 0x7f236806c470 2024-01-15T18:03:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f2366a00640 1 -- 172.21.15.26:0/275395355 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23500319d0 con 0x7f236806c470 2024-01-15T18:03:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f2366a00640 1 -- 172.21.15.26:0/275395355 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350031d60 con 0x7f236806c470 2024-01-15T18:03:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f236d000640 1 -- 172.21.15.26:0/275395355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236806c470 msgr2=0x7f23680cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f236d000640 1 --2- 172.21.15.26:0/275395355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236806c470 0x7f23680cc5e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f23680aea60 tx=0x7f235002b910 comp rx=0 tx=0).stop 2024-01-15T18:03:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f236d000640 1 -- 172.21.15.26:0/275395355 shutdown_connections 2024-01-15T18:03:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f236d000640 1 --2- 172.21.15.26:0/275395355 >> v2:172.21.15.124:3300/0 conn(0x7f236806dab0 0x7f23680ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f236d000640 1 --2- 172.21.15.26:0/275395355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f236806c470 0x7f23680cc5e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f236d000640 1 -- 172.21.15.26:0/275395355 >> 172.21.15.26:0/275395355 conn(0x7f236806a3e0 msgr2=0x7f236806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f236d000640 1 -- 172.21.15.26:0/275395355 shutdown_connections 2024-01-15T18:03:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.974+0000 7f236d000640 1 -- 172.21.15.26:0/275395355 wait complete. 2024-01-15T18:03:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.976+0000 7f236d000640 1 Processor -- start 2024-01-15T18:03:01.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.976+0000 7f236d000640 1 -- start start 2024-01-15T18:03:01.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.977+0000 7f236d000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236806c470 0x7f23680700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.977+0000 7f236d000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236806dab0 0x7f2368138ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.977+0000 7f236d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23681447f0 0x7f2368139410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.977+0000 7f236d000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368071390 con 0x7f23681447f0 2024-01-15T18:03:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.977+0000 7f236d000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2368070780 con 0x7f236806c470 2024-01-15T18:03:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.977+0000 7f236d000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2368070a60 con 0x7f236806dab0 2024-01-15T18:03:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.977+0000 7f2367e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23681447f0 0x7f2368139410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.977+0000 7f2367e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23681447f0 0x7f2368139410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60928/0 (socket says 172.21.15.26:60928) 2024-01-15T18:03:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.977+0000 7f2367e00640 1 -- 172.21.15.26:0/408419690 learned_addr learned my addr 172.21.15.26:0/408419690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:01.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f235fe00640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236806dab0 0x7f2368138ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f2367400640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236806c470 0x7f23680700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f2367e00640 1 -- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236806dab0 msgr2=0x7f2368138ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f2367e00640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236806dab0 0x7f2368138ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f2367e00640 1 -- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236806c470 msgr2=0x7f23680700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f2367e00640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236806c470 0x7f23680700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f2367e00640 1 -- 172.21.15.26:0/408419690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23500094d0 con 0x7f23681447f0 2024-01-15T18:03:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f235fe00640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236806dab0 0x7f2368138ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f2367400640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236806c470 0x7f23680700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.978+0000 7f2367e00640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23681447f0 0x7f2368139410 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2360007e40 tx=0x7f236000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.979+0000 7f235f400640 1 -- 172.21.15.26:0/408419690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360014620 con 0x7f23681447f0 2024-01-15T18:03:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.979+0000 7f235f400640 1 -- 172.21.15.26:0/408419690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236000ee90 con 0x7f23681447f0 2024-01-15T18:03:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.979+0000 7f235f400640 1 -- 172.21.15.26:0/408419690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600149e0 con 0x7f23681447f0 2024-01-15T18:03:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.979+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2368070d40 con 0x7f23681447f0 2024-01-15T18:03:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.979+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2368071820 con 0x7f23681447f0 2024-01-15T18:03:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.980+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2368071500 con 0x7f23681447f0 2024-01-15T18:03:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.981+0000 7f235f400640 1 -- 172.21.15.26:0/408419690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f2360014b40 con 0x7f23681447f0 2024-01-15T18:03:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.982+0000 7f235f400640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234006bc50 0x7f234006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.982+0000 7f235f400640 1 -- 172.21.15.26:0/408419690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2360089100 con 0x7f23681447f0 2024-01-15T18:03:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.982+0000 7f2367400640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234006bc50 0x7f234006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.984+0000 7f2367400640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234006bc50 0x7f234006e110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2358000f60 tx=0x7f2358009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:01.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:01.986+0000 7f235f400640 1 -- 172.21.15.26:0/408419690 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23600557c0 con 0x7f23681447f0 2024-01-15T18:03:02.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.217+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23680cea40 con 0x7f234006bc50 2024-01-15T18:03:02.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.286+0000 7f235f400640 1 -- 172.21.15.26:0/408419690 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f23680cea40 con 0x7f234006bc50 2024-01-15T18:03:02.290 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.289+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234006bc50 msgr2=0x7f234006e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.289+0000 7f236d000640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234006bc50 0x7f234006e110 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2358000f60 tx=0x7f2358009040 comp rx=0 tx=0).stop 2024-01-15T18:03:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.289+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23681447f0 msgr2=0x7f2368139410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.289+0000 7f236d000640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23681447f0 0x7f2368139410 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2360007e40 tx=0x7f236000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.290+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 shutdown_connections 2024-01-15T18:03:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.290+0000 7f236d000640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f236806dab0 0x7f2368138ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.290+0000 7f236d000640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234006bc50 0x7f234006e110 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.290+0000 7f236d000640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23681447f0 0x7f2368139410 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.290+0000 7f236d000640 1 --2- 172.21.15.26:0/408419690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f236806c470 0x7f23680700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.290+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 >> 172.21.15.26:0/408419690 conn(0x7f236806a3e0 msgr2=0x7f2368142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.290+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 shutdown_connections 2024-01-15T18:03:02.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.290+0000 7f236d000640 1 -- 172.21.15.26:0/408419690 wait complete. 2024-01-15T18:03:02.314 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:03:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.539+0000 7f7124a00640 1 Processor -- start 2024-01-15T18:03:02.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.540+0000 7f7124a00640 1 -- start start 2024-01-15T18:03:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.542+0000 7f7124a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f71200adf20 0x7f7120070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.542+0000 7f7124a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712015f4e0 con 0x7f712015bf10 2024-01-15T18:03:02.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.542+0000 7f7124a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7120071130 con 0x7f71200ccb50 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.542+0000 7f7124a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712015f970 con 0x7f71200adf20 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.542+0000 7f7117e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f712015bf10 0x7f71200cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51076/0 (socket says 172.21.15.26:51076) 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.542+0000 7f7117e00640 1 -- 172.21.15.26:0/591509777 learned_addr learned my addr 172.21.15.26:0/591509777 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711e800640 1 --2- 172.21.15.26:0/591509777 >> v2:172.21.15.124:3300/0 conn(0x7f71200adf20 0x7f7120070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605948605 0 0) 0x7f712015f4e0 con 0x7f712015bf10 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7100003520 con 0x7f712015bf10 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 209979900 0 0) 0x7f7100003520 con 0x7f712015bf10 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 >> v2:172.21.15.124:3300/0 conn(0x7f71200adf20 msgr2=0x7f7120070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 --2- 172.21.15.26:0/591509777 >> v2:172.21.15.124:3300/0 conn(0x7f71200adf20 0x7f7120070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 >> v1:172.21.15.124:6789/0 conn(0x7f71200ccb50 legacy=0x7f7120143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712015c390 con 0x7f712015bf10 2024-01-15T18:03:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f7108001670 con 0x7f712015bf10 2024-01-15T18:03:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 >> v1:172.21.15.26:6789/0 conn(0x7f712015bf10 legacy=0x7f71200cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7100003970 0x7f7100003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.543+0000 7f711de00640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f7100004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f711de00640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7100005180 0x7f710000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712015f970 con 0x7f71000047c0 2024-01-15T18:03:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710000c080 con 0x7f7100003970 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710000c240 con 0x7f7100005180 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f7117e00640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f7100004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f711e800640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7100003970 0x7f7100003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f711f200640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7100005180 0x7f710000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f7117e00640 1 -- 172.21.15.26:0/591509777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7100005180 msgr2=0x7f710000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f7117e00640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7100005180 0x7f710000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f7117e00640 1 -- 172.21.15.26:0/591509777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7100003970 msgr2=0x7f7100003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f7117e00640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7100003970 0x7f7100003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f7117e00640 1 -- 172.21.15.26:0/591509777 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100003520 con 0x7f71000047c0 2024-01-15T18:03:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.544+0000 7f711f200640 1 -- 172.21.15.26:0/591509777 reap_dead start 2024-01-15T18:03:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.546+0000 7f7117e00640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f7100004c40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f710800c0e0 tx=0x7f71000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f958b80079f31d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.546+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108005ad0 con 0x7f71000047c0 2024-01-15T18:03:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.546+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108005c30 con 0x7f71000047c0 2024-01-15T18:03:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.546+0000 7f711de00640 1 -- 172.21.15.26:0/591509777 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710802c7d0 con 0x7f71000047c0 2024-01-15T18:03:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.546+0000 7f7124a00640 1 -- 172.21.15.26:0/591509777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 msgr2=0x7f7100004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.546+0000 7f7124a00640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f7100004c40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f710800c0e0 tx=0x7f71000057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.547+0000 7f7124a00640 1 -- 172.21.15.26:0/591509777 shutdown_connections 2024-01-15T18:03:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.547+0000 7f7124a00640 1 --2- 172.21.15.26:0/591509777 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f7100004c40 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.547+0000 7f7124a00640 1 -- 172.21.15.26:0/591509777 >> 172.21.15.26:0/591509777 conn(0x7f712006a7d0 msgr2=0x7f712006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.547+0000 7f7124a00640 1 -- 172.21.15.26:0/591509777 shutdown_connections 2024-01-15T18:03:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.547+0000 7f7124a00640 1 -- 172.21.15.26:0/591509777 wait complete. 2024-01-15T18:03:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.548+0000 7f7124a00640 1 Processor -- start 2024-01-15T18:03:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.549+0000 7f7124a00640 1 -- start start 2024-01-15T18:03:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.549+0000 7f7124a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f71200acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.549+0000 7f7124a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71200afcf0 0x7f71201f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.549+0000 7f7124a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71200b0170 0x7f71200c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f7124a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71200ac4d0 con 0x7f71000047c0 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f7124a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71200ad420 con 0x7f71200b0170 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f7124a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71201f46a0 con 0x7f71200afcf0 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f71200acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f71200acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60932/0 (socket says 172.21.15.26:60932) 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711e800640 1 -- 172.21.15.26:0/3980512649 learned_addr learned my addr 172.21.15.26:0/3980512649 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f7117e00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71200afcf0 0x7f71201f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711f200640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71200b0170 0x7f71200c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711e800640 1 -- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71200afcf0 msgr2=0x7f71201f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711e800640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71200afcf0 0x7f71201f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711e800640 1 -- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71200b0170 msgr2=0x7f71200c5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711e800640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71200b0170 0x7f71200c5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711e800640 1 -- 172.21.15.26:0/3980512649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71201f4980 con 0x7f71000047c0 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.550+0000 7f711f200640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71200b0170 0x7f71200c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.551+0000 7f711e800640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f71200acee0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f71100114e0 tx=0x7f71100119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.551+0000 7f7116a00640 1 -- 172.21.15.26:0/3980512649 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110015870 con 0x7f71000047c0 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.551+0000 7f7116a00640 1 -- 172.21.15.26:0/3980512649 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71100159d0 con 0x7f71000047c0 2024-01-15T18:03:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.551+0000 7f7116a00640 1 -- 172.21.15.26:0/3980512649 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110015d40 con 0x7f71000047c0 2024-01-15T18:03:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.551+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71200ac6e0 con 0x7f71000047c0 2024-01-15T18:03:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.551+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7120138bf0 con 0x7f71000047c0 2024-01-15T18:03:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.552+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71201f4b70 con 0x7f71000047c0 2024-01-15T18:03:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.553+0000 7f7116a00640 1 -- 172.21.15.26:0/3980512649 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f7110013400 con 0x7f71000047c0 2024-01-15T18:03:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.554+0000 7f7116a00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70f806bc50 0x7f70f806e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.554+0000 7f7116a00640 1 -- 172.21.15.26:0/3980512649 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7110088b90 con 0x7f71000047c0 2024-01-15T18:03:02.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.555+0000 7f7117e00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70f806bc50 0x7f70f806e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.556+0000 7f7117e00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70f806bc50 0x7f70f806e110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7108034000 tx=0x7f7108009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.559+0000 7f7116a00640 1 -- 172.21.15.26:0/3980512649 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71100bde90 con 0x7f71000047c0 2024-01-15T18:03:02.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.780+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f712015f170 con 0x7f70f806bc50 2024-01-15T18:03:02.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.824+0000 7f7116a00640 1 -- 172.21.15.26:0/3980512649 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f710803d040 con 0x7f70f806bc50 2024-01-15T18:03:02.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70f806bc50 msgr2=0x7f70f806e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:02.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70f806bc50 0x7f70f806e110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7108034000 tx=0x7f7108009030 comp rx=0 tx=0).stop 2024-01-15T18:03:02.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 msgr2=0x7f71200acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:02.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f71200acee0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f71100114e0 tx=0x7f71100119a0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 shutdown_connections 2024-01-15T18:03:02.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71200afcf0 0x7f71201f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f70f806bc50 0x7f70f806e110 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71200b0170 0x7f71200c5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 --2- 172.21.15.26:0/3980512649 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71000047c0 0x7f71200acee0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:02.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 >> 172.21.15.26:0/3980512649 conn(0x7f712006a7d0 msgr2=0x7f712015cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:02.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 shutdown_connections 2024-01-15T18:03:02.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:02.826+0000 7f7124a00640 1 -- 172.21.15.26:0/3980512649 wait complete. 2024-01-15T18:03:02.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone20837 2024-01-15T18:03:03.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.072+0000 7fdbfec00640 1 Processor -- start 2024-01-15T18:03:03.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.072+0000 7fdbfec00640 1 -- start start 2024-01-15T18:03:03.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.074+0000 7fdbfec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80ce8f0 0x7fdbf80ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.074+0000 7fdbfec00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf80d0de0 con 0x7fdbf8160f80 2024-01-15T18:03:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.074+0000 7fdbfec00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf80d10c0 con 0x7fdbf80b2a00 2024-01-15T18:03:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.074+0000 7fdbfec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80d13a0 con 0x7fdbf80ce8f0 2024-01-15T18:03:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.074+0000 7fdbf7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdbf8160f80 0x7fdbf8161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51086/0 (socket says 172.21.15.26:51086) 2024-01-15T18:03:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.074+0000 7fdbf7e00640 1 -- 172.21.15.26:0/904450516 learned_addr learned my addr 172.21.15.26:0/904450516 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbefe00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80ce8f0 0x7fdbf80ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783597279 0 0) 0x7fdbf80d0de0 con 0x7fdbf8160f80 2024-01-15T18:03:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbd8003520 con 0x7fdbf8160f80 2024-01-15T18:03:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 257138940 0 0) 0x7fdbd8003520 con 0x7fdbf8160f80 2024-01-15T18:03:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80ce8f0 msgr2=0x7fdbf80ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80ce8f0 0x7fdbf80ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 >> v1:172.21.15.124:6789/0 conn(0x7fdbf80b2a00 legacy=0x7fdbf80b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf80d1680 con 0x7fdbf8160f80 2024-01-15T18:03:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fdbe8001670 con 0x7fdbf8160f80 2024-01-15T18:03:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 >> v1:172.21.15.26:6789/0 conn(0x7fdbf8160f80 legacy=0x7fdbf8161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80ce8f0 0x7fdbd80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbd8004d00 0x7fdbd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80d13a0 con 0x7fdbd8003f20 2024-01-15T18:03:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd800ba60 con 0x7fdbd8004d00 2024-01-15T18:03:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.075+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd800bc20 con 0x7fdbf80ce8f0 2024-01-15T18:03:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbefe00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbfcc00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbd8004d00 0x7fdbd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbefe00640 1 -- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80ce8f0 msgr2=0x7fdbd80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbf7e00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80ce8f0 0x7fdbd80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbefe00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80ce8f0 0x7fdbd80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbefe00640 1 -- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbd8004d00 msgr2=0x7fdbd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbefe00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbd8004d00 0x7fdbd8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbefe00640 1 -- 172.21.15.26:0/904450516 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd8003520 con 0x7fdbd8003f20 2024-01-15T18:03:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbf7e00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80ce8f0 0x7fdbd80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:03.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.076+0000 7fdbfcc00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbd8004d00 0x7fdbd8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.078+0000 7fdbefe00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbd80047c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009ac0 tx=0x7fdbd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebb72fb4e0756f8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.078+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe002c6b0 con 0x7fdbd8003f20 2024-01-15T18:03:03.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.078+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe002c810 con 0x7fdbd8003f20 2024-01-15T18:03:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.078+0000 7fdbf7400640 1 -- 172.21.15.26:0/904450516 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe002cba0 con 0x7fdbd8003f20 2024-01-15T18:03:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.078+0000 7fdbfec00640 1 -- 172.21.15.26:0/904450516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 msgr2=0x7fdbd80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.078+0000 7fdbfec00640 1 --2- 172.21.15.26:0/904450516 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbd80047c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009ac0 tx=0x7fdbd80057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.079+0000 7fdbfcc00640 1 -- 172.21.15.26:0/904450516 reap_dead start 2024-01-15T18:03:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.079+0000 7fdbfec00640 1 -- 172.21.15.26:0/904450516 shutdown_connections 2024-01-15T18:03:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.079+0000 7fdbfec00640 1 -- 172.21.15.26:0/904450516 >> 172.21.15.26:0/904450516 conn(0x7fdbf809c550 msgr2=0x7fdbf80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.079+0000 7fdbfec00640 1 -- 172.21.15.26:0/904450516 shutdown_connections 2024-01-15T18:03:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.079+0000 7fdbfec00640 1 -- 172.21.15.26:0/904450516 wait complete. 2024-01-15T18:03:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.081+0000 7fdbfec00640 1 Processor -- start 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.081+0000 7fdbfec00640 1 -- start start 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.082+0000 7fdbfec00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbf80c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.082+0000 7fdbfec00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf80ca450 0x7fdbf815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.082+0000 7fdbfec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80c6fb0 0x7fdbf80b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.082+0000 7fdbfec00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80caca0 con 0x7fdbd8003f20 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.082+0000 7fdbfec00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80b0d50 con 0x7fdbf80ca450 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.082+0000 7fdbfec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80b0420 con 0x7fdbf80c6fb0 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.082+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbf80c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.082+0000 7fdbf7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbf80c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60944/0 (socket says 172.21.15.26:60944) 2024-01-15T18:03:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.082+0000 7fdbf7e00640 1 -- 172.21.15.26:0/2089865697 learned_addr learned my addr 172.21.15.26:0/2089865697 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbf7e00640 1 -- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80c6fb0 msgr2=0x7fdbf80b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbf7e00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80c6fb0 0x7fdbf80b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbf7e00640 1 -- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf80ca450 msgr2=0x7fdbf815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbf7e00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf80ca450 0x7fdbf815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbf7e00640 1 -- 172.21.15.26:0/2089865697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf80b0700 con 0x7fdbd8003f20 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbf7e00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbf80c7c30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800d8b0 tx=0x7fdbe800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbf4c00640 1 -- 172.21.15.26:0/2089865697 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80074e0 con 0x7fdbd8003f20 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbf4c00640 1 -- 172.21.15.26:0/2089865697 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe8007640 con 0x7fdbd8003f20 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf80b09e0 con 0x7fdbd8003f20 2024-01-15T18:03:03.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.083+0000 7fdbf4c00640 1 -- 172.21.15.26:0/2089865697 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80049e0 con 0x7fdbd8003f20 2024-01-15T18:03:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.084+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf815dc00 con 0x7fdbd8003f20 2024-01-15T18:03:03.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.085+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf80675d0 con 0x7fdbd8003f20 2024-01-15T18:03:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.086+0000 7fdbf4c00640 1 -- 172.21.15.26:0/2089865697 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7fdbe8004b40 con 0x7fdbd8003f20 2024-01-15T18:03:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.088+0000 7fdbf4c00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006bc80 0x7fdbd006e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.088+0000 7fdbf4c00640 1 -- 172.21.15.26:0/2089865697 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdbe8088510 con 0x7fdbd8003f20 2024-01-15T18:03:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.088+0000 7fdbefe00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006bc80 0x7fdbd006e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.090+0000 7fdbefe00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006bc80 0x7fdbd006e140 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0036000 tx=0x7fdbe002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:03.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.092+0000 7fdbf4c00640 1 -- 172.21.15.26:0/2089865697 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdbe80586f0 con 0x7fdbd8003f20 2024-01-15T18:03:03.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.316+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone20837", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbf80caa90 con 0x7fdbd006bc80 2024-01-15T18:03:03.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.363+0000 7fdbf4c00640 1 -- 172.21.15.26:0/2089865697 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdbf80caa90 con 0x7fdbd006bc80 2024-01-15T18:03:03.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.366+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006bc80 msgr2=0x7fdbd006e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.366+0000 7fdbfec00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006bc80 0x7fdbd006e140 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0036000 tx=0x7fdbe002e000 comp rx=0 tx=0).stop 2024-01-15T18:03:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.366+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 msgr2=0x7fdbf80c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.366+0000 7fdbfec00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbf80c7c30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800d8b0 tx=0x7fdbe800dd70 comp rx=0 tx=0).stop 2024-01-15T18:03:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.367+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 shutdown_connections 2024-01-15T18:03:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.367+0000 7fdbfec00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdbf80c6fb0 0x7fdbf80b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.367+0000 7fdbfec00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdbd006bc80 0x7fdbd006e140 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.367+0000 7fdbfec00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdbf80ca450 0x7fdbf815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.367+0000 7fdbfec00640 1 --2- 172.21.15.26:0/2089865697 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdbd8003f20 0x7fdbf80c7c30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.367+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 >> 172.21.15.26:0/2089865697 conn(0x7fdbf809c550 msgr2=0x7fdbf806b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:03.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.367+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 shutdown_connections 2024-01-15T18:03:03.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.367+0000 7fdbfec00640 1 -- 172.21.15.26:0/2089865697 wait complete. 2024-01-15T18:03:03.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone20836 2024-01-15T18:03:03.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.595+0000 7f93dfe00640 1 Processor -- start 2024-01-15T18:03:03.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.596+0000 7f93dfe00640 1 -- start start 2024-01-15T18:03:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.597+0000 7f93dfe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d815d610 0x7f93d80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.597+0000 7f93dfe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f93d80d1ef0 0x7f93d80b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.597+0000 7f93dfe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d80b3400 con 0x7f93d80ced60 2024-01-15T18:03:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.597+0000 7f93dfe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d80d1390 con 0x7f93d80d1ef0 2024-01-15T18:03:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.597+0000 7f93dfe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80b27f0 con 0x7f93d815d610 2024-01-15T18:03:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.597+0000 7f93dca00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f93d80ced60 0x7f93d80d17e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51094/0 (socket says 172.21.15.26:51094) 2024-01-15T18:03:03.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.597+0000 7f93dca00640 1 -- 172.21.15.26:0/2276142377 learned_addr learned my addr 172.21.15.26:0/2276142377 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93dd400640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d815d610 0x7f93d80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93dde00640 1 --2- 172.21.15.26:0/2276142377 >> v2:172.21.15.124:3300/0 conn(0x7f93d80d1ef0 0x7f93d80b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604871011 0 0) 0x7f93d80b3400 con 0x7f93d80ced60 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93b8003520 con 0x7f93d80ced60 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3562132961 0 0) 0x7f93b8003520 con 0x7f93d80ced60 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d815d610 msgr2=0x7f93d80ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d815d610 0x7f93d80ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 >> v2:172.21.15.124:3300/0 conn(0x7f93d80d1ef0 msgr2=0x7f93d80b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 --2- 172.21.15.26:0/2276142377 >> v2:172.21.15.124:3300/0 conn(0x7f93d80d1ef0 0x7f93d80b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80b2ad0 con 0x7f93d80ced60 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f93c8001670 con 0x7f93d80ced60 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 >> v1:172.21.15.26:6789/0 conn(0x7f93d80ced60 legacy=0x7f93d80d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93b8003970 0x7f93b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d815d610 0x7f93b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93b8004d00 0x7f93b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80d1390 con 0x7f93b8004d00 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80b27f0 con 0x7f93b8003970 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.598+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b800ba00 con 0x7f93d815d610 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dde00640 1 --2- 172.21.15.26:0/2276142377 >> v2:172.21.15.124:3300/0 conn(0x7f93d80d1ef0 0x7f93d80b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dde00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93b8004d00 0x7f93b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dca00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d815d610 0x7f93b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dd400640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93b8003970 0x7f93b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dde00640 1 -- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d815d610 msgr2=0x7f93b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dde00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d815d610 0x7f93b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dde00640 1 -- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93b8003970 msgr2=0x7f93b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dde00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93b8003970 0x7f93b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dde00640 1 -- 172.21.15.26:0/2276142377 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b8003520 con 0x7f93b8004d00 2024-01-15T18:03:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.599+0000 7f93dca00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93d815d610 0x7f93b80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:03:03.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.600+0000 7f93dde00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93b8004d00 0x7f93b8005180 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f93d00073b0 tx=0x7f93b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04e376ebcfcf799 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:03.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.601+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d002cd90 con 0x7f93b8004d00 2024-01-15T18:03:03.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.601+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d0034440 con 0x7f93b8004d00 2024-01-15T18:03:03.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.601+0000 7f93d7e00640 1 -- 172.21.15.26:0/2276142377 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d00347b0 con 0x7f93b8004d00 2024-01-15T18:03:03.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.601+0000 7f93dfe00640 1 -- 172.21.15.26:0/2276142377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93b8004d00 msgr2=0x7f93b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.601+0000 7f93dfe00640 1 --2- 172.21.15.26:0/2276142377 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93b8004d00 0x7f93b8005180 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f93d00073b0 tx=0x7f93b80057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.601+0000 7f93dde00640 1 -- 172.21.15.26:0/2276142377 reap_dead start 2024-01-15T18:03:03.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.601+0000 7f93dfe00640 1 -- 172.21.15.26:0/2276142377 shutdown_connections 2024-01-15T18:03:03.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.601+0000 7f93dfe00640 1 -- 172.21.15.26:0/2276142377 >> 172.21.15.26:0/2276142377 conn(0x7f93d809c540 msgr2=0x7f93d80d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.601+0000 7f93dfe00640 1 -- 172.21.15.26:0/2276142377 shutdown_connections 2024-01-15T18:03:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.602+0000 7f93dfe00640 1 -- 172.21.15.26:0/2276142377 wait complete. 2024-01-15T18:03:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.603+0000 7f93dfe00640 1 Processor -- start 2024-01-15T18:03:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.603+0000 7f93dfe00640 1 -- start start 2024-01-15T18:03:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.604+0000 7f93dfe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93b8004d00 0x7f93d80b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.604+0000 7f93dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93d815b730 0x7f93d80cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.604+0000 7f93dfe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81528b0 0x7f93d80cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.604+0000 7f93dfe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80b75a0 con 0x7f93d815b730 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.604+0000 7f93dfe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80cb630 con 0x7f93d81528b0 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.604+0000 7f93dfe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d80b8bb0 con 0x7f93b8004d00 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93d815b730 0x7f93d80cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93d815b730 0x7f93d80cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60968/0 (socket says 172.21.15.26:60968) 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dca00640 1 -- 172.21.15.26:0/1242964756 learned_addr learned my addr 172.21.15.26:0/1242964756 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dd400640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93b8004d00 0x7f93d80b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dca00640 1 -- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93b8004d00 msgr2=0x7f93d80b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dde00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81528b0 0x7f93d80cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dca00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93b8004d00 0x7f93d80b84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dca00640 1 -- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81528b0 msgr2=0x7f93d80cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dca00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81528b0 0x7f93d80cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dca00640 1 -- 172.21.15.26:0/1242964756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80b6c00 con 0x7f93d815b730 2024-01-15T18:03:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.605+0000 7f93dca00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93d815b730 0x7f93d80cbd80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f93c800e8f0 tx=0x7f93c800edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.606+0000 7f93d5600640 1 -- 172.21.15.26:0/1242964756 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c800c8d0 con 0x7f93d815b730 2024-01-15T18:03:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.606+0000 7f93d5600640 1 -- 172.21.15.26:0/1242964756 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c800ca30 con 0x7f93d815b730 2024-01-15T18:03:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.606+0000 7f93d5600640 1 -- 172.21.15.26:0/1242964756 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c800cda0 con 0x7f93d815b730 2024-01-15T18:03:03.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.606+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d80b6ee0 con 0x7f93d815b730 2024-01-15T18:03:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.606+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d80b79c0 con 0x7f93d815b730 2024-01-15T18:03:03.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.607+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d80b76e0 con 0x7f93d815b730 2024-01-15T18:03:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.608+0000 7f93d5600640 1 -- 172.21.15.26:0/1242964756 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f93c801d460 con 0x7f93d815b730 2024-01-15T18:03:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.609+0000 7f93d5600640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f93b006bc50 0x7f93b006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.609+0000 7f93d5600640 1 -- 172.21.15.26:0/1242964756 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f93c80888b0 con 0x7f93d815b730 2024-01-15T18:03:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.610+0000 7f93dd400640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f93b006bc50 0x7f93b006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:03.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.611+0000 7f93dd400640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f93b006bc50 0x7f93b006e110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93c00098d0 tx=0x7f93c0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:03.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.613+0000 7f93d5600640 1 -- 172.21.15.26:0/1242964756 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93d80b76e0 con 0x7f93d815b730 2024-01-15T18:03:03.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.834+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone20836", "target": ["mon-mgr", ""]}) v1 -- 0x7f93d815d9f0 con 0x7f93b006bc50 2024-01-15T18:03:03.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.883+0000 7f93d5600640 1 -- 172.21.15.26:0/1242964756 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f93c0004020 con 0x7f93b006bc50 2024-01-15T18:03:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.885+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f93b006bc50 msgr2=0x7f93b006e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.885+0000 7f93dfe00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f93b006bc50 0x7f93b006e110 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f93c00098d0 tx=0x7f93c0008040 comp rx=0 tx=0).stop 2024-01-15T18:03:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.885+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93d815b730 msgr2=0x7f93d80cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.885+0000 7f93dfe00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93d815b730 0x7f93d80cbd80 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f93c800e8f0 tx=0x7f93c800edb0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.886+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 shutdown_connections 2024-01-15T18:03:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.886+0000 7f93dfe00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f93b006bc50 0x7f93b006e110 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.886+0000 7f93dfe00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f93d81528b0 0x7f93d80cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.886+0000 7f93dfe00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f93d815b730 0x7f93d80cbd80 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.886+0000 7f93dfe00640 1 --2- 172.21.15.26:0/1242964756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f93b8004d00 0x7f93d80b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:03.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.886+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 >> 172.21.15.26:0/1242964756 conn(0x7f93d809c540 msgr2=0x7f93d80b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:03.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.886+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 shutdown_connections 2024-01-15T18:03:03.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:03.886+0000 7f93dfe00640 1 -- 172.21.15.26:0/1242964756 wait complete. 2024-01-15T18:03:03.908 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone20832 2024-01-15T18:03:04.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.126+0000 7f7229a00640 1 Processor -- start 2024-01-15T18:03:04.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.126+0000 7f7229a00640 1 -- start start 2024-01-15T18:03:04.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.128+0000 7f7229a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f722413df80 0x7f722415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.128+0000 7f7229a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f722413e360 0x7f72240cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.128+0000 7f7229a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722406c470 con 0x7f722415f9c0 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.128+0000 7f7229a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722406c750 con 0x7f722413e360 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.128+0000 7f7229a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722406ca30 con 0x7f722413df80 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.128+0000 7f7222000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f722413e360 0x7f72240cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.128+0000 7f7222000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f722413e360 0x7f72240cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60976/0 (socket says 172.21.15.26:60976) 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.128+0000 7f7222000640 1 -- 172.21.15.26:0/137862206 learned_addr learned my addr 172.21.15.26:0/137862206 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.129+0000 7f7222a00640 1 --2- 172.21.15.26:0/137862206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f722413df80 0x7f722415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.129+0000 7f7222000640 1 -- 172.21.15.26:0/137862206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f722413df80 msgr2=0x7f722415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.129+0000 7f7222000640 1 --2- 172.21.15.26:0/137862206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f722413df80 0x7f722415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.129+0000 7f7222000640 1 -- 172.21.15.26:0/137862206 >> v1:172.21.15.124:6789/0 conn(0x7f722415f9c0 legacy=0x7f72241408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:04.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.129+0000 7f7222000640 1 -- 172.21.15.26:0/137862206 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722406cd10 con 0x7f722413e360 2024-01-15T18:03:04.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.130+0000 7f7222000640 1 --2- 172.21.15.26:0/137862206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413e360 0x7f72240cf290 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f720c009870 tx=0x7f720c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64d05021bd7475b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:04.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.130+0000 7f7221600640 1 -- 172.21.15.26:0/137862206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c031640 con 0x7f722413e360 2024-01-15T18:03:04.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.130+0000 7f7221600640 1 -- 172.21.15.26:0/137862206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c0317a0 con 0x7f722413e360 2024-01-15T18:03:04.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.130+0000 7f7221600640 1 -- 172.21.15.26:0/137862206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c031b30 con 0x7f722413e360 2024-01-15T18:03:04.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.131+0000 7f7229a00640 1 -- 172.21.15.26:0/137862206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413e360 msgr2=0x7f72240cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.131+0000 7f7229a00640 1 --2- 172.21.15.26:0/137862206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413e360 0x7f72240cf290 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f720c009870 tx=0x7f720c02b8f0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.131+0000 7f7229a00640 1 -- 172.21.15.26:0/137862206 shutdown_connections 2024-01-15T18:03:04.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.131+0000 7f7229a00640 1 --2- 172.21.15.26:0/137862206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413e360 0x7f72240cf290 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.131+0000 7f7229a00640 1 --2- 172.21.15.26:0/137862206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f722413df80 0x7f722415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.131+0000 7f7229a00640 1 -- 172.21.15.26:0/137862206 >> 172.21.15.26:0/137862206 conn(0x7f722406a800 msgr2=0x7f722406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:04.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.131+0000 7f7229a00640 1 -- 172.21.15.26:0/137862206 shutdown_connections 2024-01-15T18:03:04.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.131+0000 7f7229a00640 1 -- 172.21.15.26:0/137862206 wait complete. 2024-01-15T18:03:04.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.133+0000 7f7229a00640 1 Processor -- start 2024-01-15T18:03:04.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.133+0000 7f7229a00640 1 -- start start 2024-01-15T18:03:04.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.134+0000 7f7229a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413df80 0x7f722413d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.134+0000 7f7229a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f722413e360 0x7f7224151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.134+0000 7f7229a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f722415f9c0 0x7f7224151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.134+0000 7f7229a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72241f7670 con 0x7f722413df80 2024-01-15T18:03:04.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.134+0000 7f7229a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72241f7950 con 0x7f722415f9c0 2024-01-15T18:03:04.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.134+0000 7f7229a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72241f7c30 con 0x7f722413e360 2024-01-15T18:03:04.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.134+0000 7f7222a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413df80 0x7f722413d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:04.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7222a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413df80 0x7f722413d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60988/0 (socket says 172.21.15.26:60988) 2024-01-15T18:03:04.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7222a00640 1 -- 172.21.15.26:0/1632686414 learned_addr learned my addr 172.21.15.26:0/1632686414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:04.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7222a00640 1 -- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f722413e360 msgr2=0x7f7224151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7222a00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f722413e360 0x7f7224151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7222a00640 1 -- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f722415f9c0 msgr2=0x7f7224151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7223400640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f722415f9c0 0x7f7224151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7222a00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f722415f9c0 0x7f7224151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7222a00640 1 -- 172.21.15.26:0/1632686414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0094d0 con 0x7f722413df80 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7223400640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f722415f9c0 0x7f7224151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.135+0000 7f7222a00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413df80 0x7f722413d960 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7214007e40 tx=0x7f721400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.136+0000 7f721aa00640 1 -- 172.21.15.26:0/1632686414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214014620 con 0x7f722413df80 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.136+0000 7f721aa00640 1 -- 172.21.15.26:0/1632686414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721400ee90 con 0x7f722413df80 2024-01-15T18:03:04.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.136+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72241f7f10 con 0x7f722413df80 2024-01-15T18:03:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.136+0000 7f721aa00640 1 -- 172.21.15.26:0/1632686414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140149c0 con 0x7f722413df80 2024-01-15T18:03:04.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.136+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722414b0a0 con 0x7f722413df80 2024-01-15T18:03:04.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.137+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72240675d0 con 0x7f722413df80 2024-01-15T18:03:04.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.138+0000 7f721aa00640 1 -- 172.21.15.26:0/1632686414 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f7214014b20 con 0x7f722413df80 2024-01-15T18:03:04.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.139+0000 7f721aa00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71fc06bc50 0x7f71fc06e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.139+0000 7f721aa00640 1 -- 172.21.15.26:0/1632686414 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7214089030 con 0x7f722413df80 2024-01-15T18:03:04.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.140+0000 7f7222000640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71fc06bc50 0x7f71fc06e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:04.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.141+0000 7f7222000640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71fc06bc50 0x7f71fc06e110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f720c036000 tx=0x7f720c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:04.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.144+0000 7f721aa00640 1 -- 172.21.15.26:0/1632686414 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7214055750 con 0x7f722413df80 2024-01-15T18:03:04.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.363+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone20832", "target": ["mon-mgr", ""]}) v1 -- 0x7f72240ce850 con 0x7f71fc06bc50 2024-01-15T18:03:04.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.407+0000 7f721aa00640 1 -- 172.21.15.26:0/1632686414 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f720c031990 con 0x7f71fc06bc50 2024-01-15T18:03:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71fc06bc50 msgr2=0x7f71fc06e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71fc06bc50 0x7f71fc06e110 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f720c036000 tx=0x7f720c030000 comp rx=0 tx=0).stop 2024-01-15T18:03:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413df80 msgr2=0x7f722413d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413df80 0x7f722413d960 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7214007e40 tx=0x7f721400e4b0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 shutdown_connections 2024-01-15T18:03:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f722413e360 0x7f7224151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71fc06bc50 0x7f71fc06e110 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f722415f9c0 0x7f7224151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 --2- 172.21.15.26:0/1632686414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f722413df80 0x7f722413d960 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 >> 172.21.15.26:0/1632686414 conn(0x7f722406a800 msgr2=0x7f72240b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:04.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 shutdown_connections 2024-01-15T18:03:04.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.410+0000 7f7229a00640 1 -- 172.21.15.26:0/1632686414 wait complete. 2024-01-15T18:03:04.435 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone20835 2024-01-15T18:03:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.658+0000 7fcd0a800640 1 Processor -- start 2024-01-15T18:03:04.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.659+0000 7fcd0a800640 1 -- start start 2024-01-15T18:03:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.660+0000 7fcd0a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd04159590 0x7fcd04070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.661+0000 7fcd0a800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd04071790 con 0x7fcd0406d050 2024-01-15T18:03:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.661+0000 7fcd0a800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd04071100 con 0x7fcd040b3c20 2024-01-15T18:03:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.661+0000 7fcd0a800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd04070860 con 0x7fcd04159590 2024-01-15T18:03:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.661+0000 7fcd09400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd04159590 0x7fcd04070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:04.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.661+0000 7fcd09400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd04159590 0x7fcd04070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32770/0 (socket says 172.21.15.26:32770) 2024-01-15T18:03:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.661+0000 7fcd09400640 1 -- 172.21.15.26:0/2846133443 learned_addr learned my addr 172.21.15.26:0/2846133443 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.662+0000 7fcd09400640 1 -- 172.21.15.26:0/2846133443 >> v1:172.21.15.124:6789/0 conn(0x7fcd040b3c20 legacy=0x7fcd040ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.662+0000 7fcd09400640 1 -- 172.21.15.26:0/2846133443 >> v1:172.21.15.26:6789/0 conn(0x7fcd0406d050 legacy=0x7fcd0406d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.662+0000 7fcd09400640 1 -- 172.21.15.26:0/2846133443 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd04070b40 con 0x7fcd04159590 2024-01-15T18:03:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.663+0000 7fcd09400640 1 --2- 172.21.15.26:0/2846133443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 0x7fcd04070160 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fccf400bb10 tx=0x7fccf402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f11d5c4dbb39a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.663+0000 7fcd03e00640 1 -- 172.21.15.26:0/2846133443 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf402ee90 con 0x7fcd04159590 2024-01-15T18:03:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.663+0000 7fcd03e00640 1 -- 172.21.15.26:0/2846133443 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf4036440 con 0x7fcd04159590 2024-01-15T18:03:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.663+0000 7fcd03e00640 1 -- 172.21.15.26:0/2846133443 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf40367d0 con 0x7fcd04159590 2024-01-15T18:03:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.664+0000 7fcd0a800640 1 -- 172.21.15.26:0/2846133443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 msgr2=0x7fcd04070160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.664+0000 7fcd0a800640 1 --2- 172.21.15.26:0/2846133443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 0x7fcd04070160 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fccf400bb10 tx=0x7fccf402e0e0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.664+0000 7fcd0a800640 1 -- 172.21.15.26:0/2846133443 shutdown_connections 2024-01-15T18:03:04.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.664+0000 7fcd0a800640 1 --2- 172.21.15.26:0/2846133443 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 0x7fcd04070160 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.664+0000 7fcd0a800640 1 -- 172.21.15.26:0/2846133443 >> 172.21.15.26:0/2846133443 conn(0x7fcd0406a800 msgr2=0x7fcd0406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.664+0000 7fcd0a800640 1 -- 172.21.15.26:0/2846133443 shutdown_connections 2024-01-15T18:03:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.664+0000 7fcd0a800640 1 -- 172.21.15.26:0/2846133443 wait complete. 2024-01-15T18:03:04.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.666+0000 7fcd0a800640 1 Processor -- start 2024-01-15T18:03:04.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.666+0000 7fcd0a800640 1 -- start start 2024-01-15T18:03:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.667+0000 7fcd0a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd0406d050 0x7fcd041f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.667+0000 7fcd0a800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd040b3c20 0x7fcd04147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.667+0000 7fcd0a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 0x7fcd04143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.667+0000 7fcd0a800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041f2d80 con 0x7fcd04159590 2024-01-15T18:03:04.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.667+0000 7fcd0a800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041482f0 con 0x7fcd040b3c20 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.667+0000 7fcd0a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd041487e0 con 0x7fcd0406d050 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.667+0000 7fcd09e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 0x7fcd04143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.667+0000 7fcd09e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 0x7fcd04143850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32782/0 (socket says 172.21.15.26:32782) 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.667+0000 7fcd09e00640 1 -- 172.21.15.26:0/4120671041 learned_addr learned my addr 172.21.15.26:0/4120671041 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd08a00640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd040b3c20 0x7fcd04147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd09e00640 1 -- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd0406d050 msgr2=0x7fcd041f2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd09e00640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd0406d050 0x7fcd041f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd09e00640 1 -- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd040b3c20 msgr2=0x7fcd04147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd09e00640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd040b3c20 0x7fcd04147db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd09e00640 1 -- 172.21.15.26:0/4120671041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf400b770 con 0x7fcd04159590 2024-01-15T18:03:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd09e00640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 0x7fcd04143850 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c760 tx=0x7fccfc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd01600640 1 -- 172.21.15.26:0/4120671041 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc003b90 con 0x7fcd04159590 2024-01-15T18:03:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd01600640 1 -- 172.21.15.26:0/4120671041 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc003cf0 con 0x7fcd04159590 2024-01-15T18:03:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd01600640 1 -- 172.21.15.26:0/4120671041 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00aa90 con 0x7fcd04159590 2024-01-15T18:03:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd041433c0 con 0x7fcd04159590 2024-01-15T18:03:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.668+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd04142a90 con 0x7fcd04159590 2024-01-15T18:03:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.670+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd04067450 con 0x7fcd04159590 2024-01-15T18:03:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.671+0000 7fcd01600640 1 -- 172.21.15.26:0/4120671041 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7fccfc00abf0 con 0x7fcd04159590 2024-01-15T18:03:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.672+0000 7fcd01600640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fccdc06bc50 0x7fccdc06e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.672+0000 7fcd01600640 1 -- 172.21.15.26:0/4120671041 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fccfc0883d0 con 0x7fcd04159590 2024-01-15T18:03:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.673+0000 7fcd09400640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fccdc06bc50 0x7fccdc06e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.674+0000 7fcd09400640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fccdc06bc50 0x7fccdc06e110 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fccf4038000 tx=0x7fccf4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.677+0000 7fcd01600640 1 -- 172.21.15.26:0/4120671041 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fccfc054ba0 con 0x7fcd04159590 2024-01-15T18:03:04.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.903+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone20835", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd04159970 con 0x7fccdc06bc50 2024-01-15T18:03:04.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.950+0000 7fcd01600640 1 -- 172.21.15.26:0/4120671041 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fccf4036980 con 0x7fccdc06bc50 2024-01-15T18:03:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fccdc06bc50 msgr2=0x7fccdc06e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fccdc06bc50 0x7fccdc06e110 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fccf4038000 tx=0x7fccf4030000 comp rx=0 tx=0).stop 2024-01-15T18:03:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 msgr2=0x7fcd04143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 0x7fcd04143850 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c760 tx=0x7fccfc00cc20 comp rx=0 tx=0).stop 2024-01-15T18:03:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 shutdown_connections 2024-01-15T18:03:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fccdc06bc50 0x7fccdc06e110 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd04159590 0x7fcd04143850 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd040b3c20 0x7fcd04147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 --2- 172.21.15.26:0/4120671041 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd0406d050 0x7fcd041f2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.953+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 >> 172.21.15.26:0/4120671041 conn(0x7fcd0406a800 msgr2=0x7fcd040aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.954+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 shutdown_connections 2024-01-15T18:03:04.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:04.954+0000 7fcd0a800640 1 -- 172.21.15.26:0/4120671041 wait complete. 2024-01-15T18:03:04.978 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone20831 2024-01-15T18:03:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.207+0000 7f73a3200640 1 Processor -- start 2024-01-15T18:03:05.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.207+0000 7f73a3200640 1 -- start start 2024-01-15T18:03:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.209+0000 7f73a3200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f739c0cc990 0x7f739c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.209+0000 7f73a3200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c070e50 con 0x7f739c143c40 2024-01-15T18:03:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.209+0000 7f73a3200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739c144020 con 0x7f739c0cc500 2024-01-15T18:03:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.209+0000 7f73a3200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739c0723c0 con 0x7f739c0cc990 2024-01-15T18:03:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.209+0000 7f73a1e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f739c143c40 0x7f739c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51122/0 (socket says 172.21.15.26:51122) 2024-01-15T18:03:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.209+0000 7f73a1e00640 1 -- 172.21.15.26:0/2499777384 learned_addr learned my addr 172.21.15.26:0/2499777384 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811950427 0 0) 0x7f739c070e50 con 0x7f739c143c40 2024-01-15T18:03:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f737c003520 con 0x7f739c143c40 2024-01-15T18:03:05.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a2800640 1 --2- 172.21.15.26:0/2499777384 >> v2:172.21.15.124:3300/0 conn(0x7f739c0cc990 0x7f739c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2522881029 0 0) 0x7f737c003520 con 0x7f739c143c40 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 >> v2:172.21.15.124:3300/0 conn(0x7f739c0cc990 msgr2=0x7f739c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 --2- 172.21.15.26:0/2499777384 >> v2:172.21.15.124:3300/0 conn(0x7f739c0cc990 0x7f739c06d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 >> v1:172.21.15.124:6789/0 conn(0x7f739c0cc500 legacy=0x7f739c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c071e00 con 0x7f739c143c40 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f738c001670 con 0x7f739c143c40 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a2800640 1 --2- 172.21.15.26:0/2499777384 >> v2:172.21.15.124:3300/0 conn(0x7f739c0cc990 0x7f739c06d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 >> v1:172.21.15.26:6789/0 conn(0x7f739c143c40 legacy=0x7f739c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f737c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f737c0047c0 0x7f737c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f737c0052a0 0x7f737c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c0723c0 con 0x7f737c0039c0 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c00c1c0 con 0x7f737c0047c0 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.210+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c00c380 con 0x7f737c0052a0 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.211+0000 7f73a1e00640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f737c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.211+0000 7f73a1400640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f737c0047c0 0x7f737c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.211+0000 7f73a2800640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f737c0052a0 0x7f737c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.211+0000 7f73a1e00640 1 -- 172.21.15.26:0/2499777384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f737c0052a0 msgr2=0x7f737c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.211+0000 7f73a1e00640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f737c0052a0 0x7f737c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.211+0000 7f73a1e00640 1 -- 172.21.15.26:0/2499777384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f737c0047c0 msgr2=0x7f737c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.211+0000 7f73a1e00640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f737c0047c0 0x7f737c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.211+0000 7f73a1e00640 1 -- 172.21.15.26:0/2499777384 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c003520 con 0x7f737c0039c0 2024-01-15T18:03:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.211+0000 7f73a2800640 1 -- 172.21.15.26:0/2499777384 reap_dead start 2024-01-15T18:03:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.212+0000 7f73a1e00640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f737c003de0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f738c00c0e0 tx=0x7f737c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae2720ce5d25e75 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.213+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c005ad0 con 0x7f737c0039c0 2024-01-15T18:03:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.213+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c005c30 con 0x7f737c0039c0 2024-01-15T18:03:05.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.213+0000 7f73a0a00640 1 -- 172.21.15.26:0/2499777384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c0072b0 con 0x7f737c0039c0 2024-01-15T18:03:05.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.213+0000 7f73a3200640 1 -- 172.21.15.26:0/2499777384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 msgr2=0x7f737c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.213+0000 7f73a3200640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f737c003de0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f738c00c0e0 tx=0x7f737c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.214+0000 7f73a3200640 1 -- 172.21.15.26:0/2499777384 shutdown_connections 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.214+0000 7f73a3200640 1 --2- 172.21.15.26:0/2499777384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f737c003de0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.214+0000 7f73a3200640 1 -- 172.21.15.26:0/2499777384 >> 172.21.15.26:0/2499777384 conn(0x7f739c06a800 msgr2=0x7f739c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.214+0000 7f73a3200640 1 -- 172.21.15.26:0/2499777384 shutdown_connections 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.214+0000 7f73a3200640 1 -- 172.21.15.26:0/2499777384 wait complete. 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.216+0000 7f73a3200640 1 Processor -- start 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.216+0000 7f73a3200640 1 -- start start 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f739c15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a3200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f739c1eebd0 0x7f739c1ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f739c15df60 0x7f739c15e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c15e920 con 0x7f737c0039c0 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a3200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c15ceb0 con 0x7f739c1eebd0 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739c15c2a0 con 0x7f739c15df60 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f739c15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f739c15a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32800/0 (socket says 172.21.15.26:32800) 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a1e00640 1 -- 172.21.15.26:0/829704657 learned_addr learned my addr 172.21.15.26:0/829704657 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a2800640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f739c15df60 0x7f739c15e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a1400640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f739c1eebd0 0x7f739c1ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a1e00640 1 -- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f739c15df60 msgr2=0x7f739c15e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a1e00640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f739c15df60 0x7f739c15e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.217+0000 7f73a1e00640 1 -- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f739c1eebd0 msgr2=0x7f739c1ef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f73a1e00640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f739c1eebd0 0x7f739c1ef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f73a1e00640 1 -- 172.21.15.26:0/829704657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c15c580 con 0x7f737c0039c0 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f73a2800640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f739c15df60 0x7f739c15e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f73a1400640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f739c1eebd0 0x7f739c1ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f73a1e00640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f739c15a1d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f738c000c00 tx=0x7f738c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f739a000640 1 -- 172.21.15.26:0/829704657 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c03c040 con 0x7f737c0039c0 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f739a000640 1 -- 172.21.15.26:0/829704657 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738c007560 con 0x7f737c0039c0 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f739a000640 1 -- 172.21.15.26:0/829704657 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c02c770 con 0x7f737c0039c0 2024-01-15T18:03:05.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739c15c860 con 0x7f737c0039c0 2024-01-15T18:03:05.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.218+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c15a710 con 0x7f737c0039c0 2024-01-15T18:03:05.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.220+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739c067450 con 0x7f737c0039c0 2024-01-15T18:03:05.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.221+0000 7f739a000640 1 -- 172.21.15.26:0/829704657 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f738c032070 con 0x7f737c0039c0 2024-01-15T18:03:05.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.221+0000 7f739a000640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f737406bc80 0x7f737406e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.221+0000 7f739a000640 1 -- 172.21.15.26:0/829704657 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f738c0a99f0 con 0x7f737c0039c0 2024-01-15T18:03:05.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.222+0000 7f73a1400640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f737406bc80 0x7f737406e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.223+0000 7f73a1400640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f737406bc80 0x7f737406e140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f739c0b0a30 tx=0x7f7384009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:05.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.226+0000 7f739a000640 1 -- 172.21.15.26:0/829704657 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f738c0760b0 con 0x7f737c0039c0 2024-01-15T18:03:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.452+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone20831", "target": ["mon-mgr", ""]}) v1 -- 0x7f739c06cc40 con 0x7f737406bc80 2024-01-15T18:03:05.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.508+0000 7f739a000640 1 -- 172.21.15.26:0/829704657 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f738400d050 con 0x7f737406bc80 2024-01-15T18:03:05.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.511+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f737406bc80 msgr2=0x7f737406e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.511+0000 7f73a3200640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f737406bc80 0x7f737406e140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f739c0b0a30 tx=0x7f7384009040 comp rx=0 tx=0).stop 2024-01-15T18:03:05.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.511+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 msgr2=0x7f739c15a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.511+0000 7f73a3200640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f739c15a1d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f738c000c00 tx=0x7f738c02e070 comp rx=0 tx=0).stop 2024-01-15T18:03:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.512+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 shutdown_connections 2024-01-15T18:03:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.512+0000 7f73a3200640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f739c15df60 0x7f739c15e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.512+0000 7f73a3200640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f737406bc80 0x7f737406e140 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.512+0000 7f73a3200640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f739c1eebd0 0x7f739c1ef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.512+0000 7f73a3200640 1 --2- 172.21.15.26:0/829704657 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f737c0039c0 0x7f739c15a1d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.512+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 >> 172.21.15.26:0/829704657 conn(0x7f739c06a800 msgr2=0x7f739c06d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.512+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 shutdown_connections 2024-01-15T18:03:05.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.512+0000 7f73a3200640 1 -- 172.21.15.26:0/829704657 wait complete. 2024-01-15T18:03:05.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone20833 2024-01-15T18:03:05.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.843+0000 7f3528400640 1 Processor -- start 2024-01-15T18:03:05.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.843+0000 7f3528400640 1 -- start start 2024-01-15T18:03:05.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.845+0000 7f3528400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f352014b2c0 0x7f35200cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.845+0000 7f3528400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35200cf7d0 0x7f3520070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.845+0000 7f3528400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3520070ea0 0x7f35200b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.845+0000 7f3528400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352014b6a0 con 0x7f352014b2c0 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.846+0000 7f3528400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35200cfbf0 con 0x7f3520070ea0 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.846+0000 7f3528400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3520071690 con 0x7f35200cf7d0 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.846+0000 7f3525000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35200cf7d0 0x7f3520070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.846+0000 7f3525000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35200cf7d0 0x7f3520070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:60108/0 (socket says 172.21.15.26:60108) 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.846+0000 7f3525000640 1 -- 172.21.15.26:0/2550555302 learned_addr learned my addr 172.21.15.26:0/2550555302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.846+0000 7f3525a00640 1 --2- 172.21.15.26:0/2550555302 >> v2:172.21.15.26:3300/0 conn(0x7f352014b2c0 0x7f35200cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.847+0000 7f3526400640 1 --2- 172.21.15.26:0/2550555302 >> v2:172.21.15.124:3300/0 conn(0x7f3520070ea0 0x7f35200b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.847+0000 7f3525000640 1 -- 172.21.15.26:0/2550555302 >> v2:172.21.15.124:3300/0 conn(0x7f3520070ea0 msgr2=0x7f35200b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.847+0000 7f3525000640 1 --2- 172.21.15.26:0/2550555302 >> v2:172.21.15.124:3300/0 conn(0x7f3520070ea0 0x7f35200b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.847+0000 7f3525000640 1 -- 172.21.15.26:0/2550555302 >> v2:172.21.15.26:3300/0 conn(0x7f352014b2c0 msgr2=0x7f35200cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.847+0000 7f3525000640 1 --2- 172.21.15.26:0/2550555302 >> v2:172.21.15.26:3300/0 conn(0x7f352014b2c0 0x7f35200cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.847+0000 7f3525000640 1 -- 172.21.15.26:0/2550555302 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3520071970 con 0x7f35200cf7d0 2024-01-15T18:03:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.847+0000 7f3525a00640 1 --2- 172.21.15.26:0/2550555302 >> v2:172.21.15.26:3300/0 conn(0x7f352014b2c0 0x7f35200cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:03:05.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.849+0000 7f3525000640 1 --2- 172.21.15.26:0/2550555302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35200cf7d0 0x7f3520070960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f352006e2a0 tx=0x7f350802b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17faf434d9821092 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.850+0000 7f351fe00640 1 -- 172.21.15.26:0/2550555302 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508031880 con 0x7f35200cf7d0 2024-01-15T18:03:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.850+0000 7f351fe00640 1 -- 172.21.15.26:0/2550555302 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35080319e0 con 0x7f35200cf7d0 2024-01-15T18:03:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.851+0000 7f351fe00640 1 -- 172.21.15.26:0/2550555302 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508031d70 con 0x7f35200cf7d0 2024-01-15T18:03:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.852+0000 7f3528400640 1 -- 172.21.15.26:0/2550555302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35200cf7d0 msgr2=0x7f3520070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.852+0000 7f3528400640 1 --2- 172.21.15.26:0/2550555302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35200cf7d0 0x7f3520070960 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f352006e2a0 tx=0x7f350802b920 comp rx=0 tx=0).stop 2024-01-15T18:03:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.852+0000 7f3528400640 1 -- 172.21.15.26:0/2550555302 shutdown_connections 2024-01-15T18:03:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.852+0000 7f3528400640 1 --2- 172.21.15.26:0/2550555302 >> v2:172.21.15.124:3300/0 conn(0x7f3520070ea0 0x7f35200b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.852+0000 7f3528400640 1 --2- 172.21.15.26:0/2550555302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f35200cf7d0 0x7f3520070960 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.852+0000 7f3528400640 1 --2- 172.21.15.26:0/2550555302 >> v2:172.21.15.26:3300/0 conn(0x7f352014b2c0 0x7f35200cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.852+0000 7f3528400640 1 -- 172.21.15.26:0/2550555302 >> 172.21.15.26:0/2550555302 conn(0x7f352006a800 msgr2=0x7f352006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.852+0000 7f3528400640 1 -- 172.21.15.26:0/2550555302 shutdown_connections 2024-01-15T18:03:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.852+0000 7f3528400640 1 -- 172.21.15.26:0/2550555302 wait complete. 2024-01-15T18:03:05.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.854+0000 7f3528400640 1 Processor -- start 2024-01-15T18:03:05.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.854+0000 7f3528400640 1 -- start start 2024-01-15T18:03:05.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.855+0000 7f3528400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3520070ea0 0x7f3520161190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3528400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35200cf7d0 0x7f3520162ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3528400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352014b2c0 0x7f3520163410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3528400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35201eaec0 con 0x7f352014b2c0 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3528400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35201eb1a0 con 0x7f35200cf7d0 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3528400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35201eb480 con 0x7f3520070ea0 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3525000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35200cf7d0 0x7f3520162ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3526400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352014b2c0 0x7f3520163410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3525000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35200cf7d0 0x7f3520162ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:53546/0 (socket says 172.21.15.26:53546) 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3525000640 1 -- 172.21.15.26:0/1501343833 learned_addr learned my addr 172.21.15.26:0/1501343833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.856+0000 7f3525a00640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3520070ea0 0x7f3520161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f3526400640 1 -- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3520070ea0 msgr2=0x7f3520161190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f3526400640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3520070ea0 0x7f3520161190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f3526400640 1 -- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35200cf7d0 msgr2=0x7f3520162ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f3526400640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35200cf7d0 0x7f3520162ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f3526400640 1 -- 172.21.15.26:0/1501343833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080094d0 con 0x7f352014b2c0 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f3525000640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35200cf7d0 0x7f3520162ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f3526400640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352014b2c0 0x7f3520163410 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3518007d40 tx=0x7f351800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f351d600640 1 -- 172.21.15.26:0/1501343833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518018070 con 0x7f352014b2c0 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f351d600640 1 -- 172.21.15.26:0/1501343833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351800edd0 con 0x7f352014b2c0 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f351d600640 1 -- 172.21.15.26:0/1501343833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518003bd0 con 0x7f352014b2c0 2024-01-15T18:03:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35201eb760 con 0x7f352014b2c0 2024-01-15T18:03:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.857+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35201616d0 con 0x7f352014b2c0 2024-01-15T18:03:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.860+0000 7f351d600640 1 -- 172.21.15.26:0/1501343833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f3518014420 con 0x7f352014b2c0 2024-01-15T18:03:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.860+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35201ebc80 con 0x7f352014b2c0 2024-01-15T18:03:05.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.861+0000 7f351d600640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34f806bc50 0x7f34f806e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:05.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.861+0000 7f351d600640 1 -- 172.21.15.26:0/1501343833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3518088750 con 0x7f352014b2c0 2024-01-15T18:03:05.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.861+0000 7f3525a00640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34f806bc50 0x7f34f806e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:05.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.862+0000 7f3525a00640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34f806bc50 0x7f34f806e110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f352006b2f0 tx=0x7f3510008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:05.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:05.871+0000 7f351d600640 1 -- 172.21.15.26:0/1501343833 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35201ebc80 con 0x7f352014b2c0 2024-01-15T18:03:06.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.264+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone20833", "target": ["mon-mgr", ""]}) v1 -- 0x7f35200b3970 con 0x7f34f806bc50 2024-01-15T18:03:06.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.290+0000 7f351d600640 1 -- 172.21.15.26:0/1501343833 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f35200b3970 con 0x7f34f806bc50 2024-01-15T18:03:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34f806bc50 msgr2=0x7f34f806e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:06.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34f806bc50 0x7f34f806e110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f352006b2f0 tx=0x7f3510008040 comp rx=0 tx=0).stop 2024-01-15T18:03:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352014b2c0 msgr2=0x7f3520163410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352014b2c0 0x7f3520163410 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3518007d40 tx=0x7f351800e4b0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 shutdown_connections 2024-01-15T18:03:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34f806bc50 0x7f34f806e110 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f352014b2c0 0x7f3520163410 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f35200cf7d0 0x7f3520162ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 --2- 172.21.15.26:0/1501343833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3520070ea0 0x7f3520161190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.294+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 >> 172.21.15.26:0/1501343833 conn(0x7f352006a800 msgr2=0x7f35200ce470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.295+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 shutdown_connections 2024-01-15T18:03:06.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.295+0000 7f3528400640 1 -- 172.21.15.26:0/1501343833 wait complete. 2024-01-15T18:03:06.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:06.323 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:03:06.396 INFO:teuthology.orchestra.run.smithi026.stdout:2 2024-01-15T18:03:06.398 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:03:06.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.650+0000 7ff92d000640 1 Processor -- start 2024-01-15T18:03:06.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.650+0000 7ff92d000640 1 -- start start 2024-01-15T18:03:06.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.654+0000 7ff92d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff928099a60 0x7ff928099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:06.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.654+0000 7ff92d000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff92806e7c0 0x7ff92806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:06.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.654+0000 7ff92d000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9280cfa70 con 0x7ff92815e880 2024-01-15T18:03:06.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.654+0000 7ff92d000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92809a3e0 con 0x7ff928099a60 2024-01-15T18:03:06.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.654+0000 7ff92d000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9280cff00 con 0x7ff92806e7c0 2024-01-15T18:03:06.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.654+0000 7ff925400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff928099a60 0x7ff928099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:06.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.654+0000 7ff925e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff92815e880 0x7ff92806ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51128/0 (socket says 172.21.15.26:51128) 2024-01-15T18:03:06.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.654+0000 7ff925e00640 1 -- 172.21.15.26:0/3566488295 learned_addr learned my addr 172.21.15.26:0/3566488295 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:06.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.655+0000 7ff925400640 1 -- 172.21.15.26:0/3566488295 >> v2:172.21.15.124:3300/0 conn(0x7ff92806e7c0 msgr2=0x7ff92806ec10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:03:06.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.655+0000 7ff925400640 1 --2- 172.21.15.26:0/3566488295 >> v2:172.21.15.124:3300/0 conn(0x7ff92806e7c0 0x7ff92806ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.655+0000 7ff925400640 1 -- 172.21.15.26:0/3566488295 >> v1:172.21.15.26:6789/0 conn(0x7ff92815e880 legacy=0x7ff92806ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:06.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.655+0000 7ff925400640 1 -- 172.21.15.26:0/3566488295 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92806fc70 con 0x7ff928099a60 2024-01-15T18:03:06.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.656+0000 7ff925400640 1 --2- 172.21.15.26:0/3566488295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff928099a60 0x7ff928099ea0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff9280b72c0 tx=0x7ff91002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e69c0a208264e54 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:06.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.656+0000 7ff924a00640 1 -- 172.21.15.26:0/3566488295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910030840 con 0x7ff928099a60 2024-01-15T18:03:06.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.657+0000 7ff924a00640 1 -- 172.21.15.26:0/3566488295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9100309a0 con 0x7ff928099a60 2024-01-15T18:03:06.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.657+0000 7ff924a00640 1 -- 172.21.15.26:0/3566488295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910030d30 con 0x7ff928099a60 2024-01-15T18:03:06.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.657+0000 7ff92d000640 1 -- 172.21.15.26:0/3566488295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff928099a60 msgr2=0x7ff928099ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:06.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.658+0000 7ff92d000640 1 --2- 172.21.15.26:0/3566488295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff928099a60 0x7ff928099ea0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ff9280b72c0 tx=0x7ff91002b980 comp rx=0 tx=0).stop 2024-01-15T18:03:06.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.659+0000 7ff92d000640 1 -- 172.21.15.26:0/3566488295 shutdown_connections 2024-01-15T18:03:06.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.659+0000 7ff92d000640 1 --2- 172.21.15.26:0/3566488295 >> v2:172.21.15.124:3300/0 conn(0x7ff92806e7c0 0x7ff92806ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.659+0000 7ff92d000640 1 --2- 172.21.15.26:0/3566488295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff928099a60 0x7ff928099ea0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.659+0000 7ff92d000640 1 -- 172.21.15.26:0/3566488295 >> 172.21.15.26:0/3566488295 conn(0x7ff9280b2d00 msgr2=0x7ff9280b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:06.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.659+0000 7ff92d000640 1 -- 172.21.15.26:0/3566488295 shutdown_connections 2024-01-15T18:03:06.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.659+0000 7ff92d000640 1 -- 172.21.15.26:0/3566488295 wait complete. 2024-01-15T18:03:06.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.661+0000 7ff92d000640 1 Processor -- start 2024-01-15T18:03:06.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.662+0000 7ff92d000640 1 -- start start 2024-01-15T18:03:06.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.663+0000 7ff92d000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff92806e7c0 0x7ff9280ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:06.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.663+0000 7ff92d000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff928099a60 0x7ff9280ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:06.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.663+0000 7ff92d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92815e880 0x7ff9281fae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:06.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.663+0000 7ff92d000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280c9c90 con 0x7ff92815e880 2024-01-15T18:03:06.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.663+0000 7ff92d000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280c8da0 con 0x7ff92806e7c0 2024-01-15T18:03:06.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.663+0000 7ff92d000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9280c9080 con 0x7ff928099a60 2024-01-15T18:03:06.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.664+0000 7ff925400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff928099a60 0x7ff9280ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:06.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.664+0000 7ff925400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff928099a60 0x7ff9280ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:60122/0 (socket says 172.21.15.26:60122) 2024-01-15T18:03:06.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.664+0000 7ff925400640 1 -- 172.21.15.26:0/2488084353 learned_addr learned my addr 172.21.15.26:0/2488084353 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:06.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.664+0000 7ff926800640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92815e880 0x7ff9281fae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:06.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.665+0000 7ff925400640 1 -- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff92806e7c0 msgr2=0x7ff9280ca3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-01-15T18:03:06.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.665+0000 7ff925400640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff92806e7c0 0x7ff9280ca3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.665+0000 7ff925400640 1 -- 172.21.15.26:0/2488084353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92815e880 msgr2=0x7ff9281fae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:06.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.665+0000 7ff925400640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92815e880 0x7ff9281fae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:06.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.665+0000 7ff925400640 1 -- 172.21.15.26:0/2488084353 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff910009400 con 0x7ff928099a60 2024-01-15T18:03:06.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.666+0000 7ff925400640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff928099a60 0x7ff9280ca930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff910009890 tx=0x7ff910004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:06.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.668+0000 7ff91e000640 1 -- 172.21.15.26:0/2488084353 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91003e040 con 0x7ff928099a60 2024-01-15T18:03:06.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.668+0000 7ff91e000640 1 -- 172.21.15.26:0/2488084353 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9100443f0 con 0x7ff928099a60 2024-01-15T18:03:06.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.668+0000 7ff91e000640 1 -- 172.21.15.26:0/2488084353 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910044780 con 0x7ff928099a60 2024-01-15T18:03:06.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.668+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9280c9360 con 0x7ff928099a60 2024-01-15T18:03:06.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.668+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9280c9920 con 0x7ff928099a60 2024-01-15T18:03:06.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.670+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9280675d0 con 0x7ff928099a60 2024-01-15T18:03:06.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.672+0000 7ff91e000640 1 -- 172.21.15.26:0/2488084353 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 20) v1 ==== 90406+0+0 (secure 0 0 0) 0x7ff91004d3b0 con 0x7ff928099a60 2024-01-15T18:03:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.673+0000 7ff91e000640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff90006bc80 0x7ff90006e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.673+0000 7ff91e000640 1 -- 172.21.15.26:0/2488084353 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff9100aafe0 con 0x7ff928099a60 2024-01-15T18:03:06.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.674+0000 7ff925e00640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff90006bc80 0x7ff90006e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:06.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.675+0000 7ff925e00640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff90006bc80 0x7ff90006e140 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff918000f60 tx=0x7ff918007a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:06.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:06.681+0000 7ff91e000640 1 -- 172.21.15.26:0/2488084353 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff910077700 con 0x7ff928099a60 2024-01-15T18:03:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:07.066+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9280cf700 con 0x7ff90006bc80 2024-01-15T18:03:07.109 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-01-15T18:03:07.105+0000 7f9819c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:03:07.222 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T18:03:09.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:09.174+0000 7ff91e000640 1 -- 172.21.15.26:0/2488084353 <== mon.2 v2:172.21.15.124:3301/0 7 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7ff910077ad0 con 0x7ff928099a60 2024-01-15T18:03:10.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.135+0000 7ff91e000640 1 -- 172.21.15.26:0/2488084353 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7ff9180058b0 con 0x7ff90006bc80 2024-01-15T18:03:10.140 INFO:teuthology.orchestra.run.smithi026.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:03:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff90006bc80 msgr2=0x7ff90006e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff90006bc80 0x7ff90006e140 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff918000f60 tx=0x7ff918007a00 comp rx=0 tx=0).stop 2024-01-15T18:03:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff928099a60 msgr2=0x7ff9280ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff928099a60 0x7ff9280ca930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff910009890 tx=0x7ff910004730 comp rx=0 tx=0).stop 2024-01-15T18:03:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 shutdown_connections 2024-01-15T18:03:10.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff928099a60 0x7ff9280ca930 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff90006bc80 0x7ff90006e140 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff92815e880 0x7ff9281fae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 --2- 172.21.15.26:0/2488084353 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff92806e7c0 0x7ff9280ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 >> 172.21.15.26:0/2488084353 conn(0x7ff9280b2d00 msgr2=0x7ff9281537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:10.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 shutdown_connections 2024-01-15T18:03:10.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.140+0000 7ff92d000640 1 -- 172.21.15.26:0/2488084353 wait complete. 2024-01-15T18:03:10.175 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:03:10.176 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:03:10.317 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:03:10.444 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:03:10.445 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:03:10.445 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:03:10.507 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:03:10.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.689+0000 7f3818a00640 1 Processor -- start 2024-01-15T18:03:10.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.690+0000 7f3818a00640 1 -- start start 2024-01-15T18:03:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.691+0000 7f3818a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381413fcb0 0x7f3814164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.691+0000 7f3818a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140cdc40 con 0x7f381415afb0 2024-01-15T18:03:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.691+0000 7f3818a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140cdec0 con 0x7f381413cfa0 2024-01-15T18:03:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.691+0000 7f3818a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140ce1a0 con 0x7f381413fcb0 2024-01-15T18:03:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.692+0000 7f3811e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f381415afb0 0x7f381413f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51140/0 (socket says 172.21.15.26:51140) 2024-01-15T18:03:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.692+0000 7f3811e00640 1 -- 172.21.15.26:0/186653020 learned_addr learned my addr 172.21.15.26:0/186653020 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.692+0000 7f3813200640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381413fcb0 0x7f3814164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.692+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361102082 0 0) 0x7f38140cdc40 con 0x7f381415afb0 2024-01-15T18:03:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.692+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f4003520 con 0x7f381415afb0 2024-01-15T18:03:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.692+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359668657 0 0) 0x7f37f4003520 con 0x7f381415afb0 2024-01-15T18:03:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381413fcb0 msgr2=0x7f3814164d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381413fcb0 0x7f3814164d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 >> v1:172.21.15.124:6789/0 conn(0x7f381413cfa0 legacy=0x7f38140cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140ce480 con 0x7f381415afb0 2024-01-15T18:03:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f37fc001670 con 0x7f381415afb0 2024-01-15T18:03:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 >> v1:172.21.15.26:6789/0 conn(0x7f381415afb0 legacy=0x7f381413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40039a0 0x7f37f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381413fcb0 0x7f37f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37f4004d00 0x7f37f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:10.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140ce1a0 con 0x7f37f40039a0 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400dcf0 con 0x7f37f4004d00 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400deb0 con 0x7f381413fcb0 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3812800640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40039a0 0x7f37f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3813200640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37f4004d00 0x7f37f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3811e00640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381413fcb0 0x7f37f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.693+0000 7f3812800640 1 -- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381413fcb0 msgr2=0x7f37f40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.694+0000 7f3812800640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f381413fcb0 0x7f37f40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.694+0000 7f3812800640 1 -- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37f4004d00 msgr2=0x7f37f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.694+0000 7f3812800640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37f4004d00 0x7f37f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.694+0000 7f3812800640 1 -- 172.21.15.26:0/186653020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f4003520 con 0x7f37f40039a0 2024-01-15T18:03:10.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3812800640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40039a0 0x7f37f4003dc0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3804001750 tx=0x7f37f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbe68615c74d55e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:10.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38040052d0 con 0x7f37f40039a0 2024-01-15T18:03:10.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380402e050 con 0x7f37f40039a0 2024-01-15T18:03:10.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3811400640 1 -- 172.21.15.26:0/186653020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38040035f0 con 0x7f37f40039a0 2024-01-15T18:03:10.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3818a00640 1 -- 172.21.15.26:0/186653020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40039a0 msgr2=0x7f37f4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:10.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3818a00640 1 --2- 172.21.15.26:0/186653020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37f40039a0 0x7f37f4003dc0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3804001750 tx=0x7f37f40057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3813200640 1 -- 172.21.15.26:0/186653020 reap_dead start 2024-01-15T18:03:10.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3818a00640 1 -- 172.21.15.26:0/186653020 shutdown_connections 2024-01-15T18:03:10.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3818a00640 1 -- 172.21.15.26:0/186653020 >> 172.21.15.26:0/186653020 conn(0x7f381406a800 msgr2=0x7f381406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:10.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.695+0000 7f3818a00640 1 -- 172.21.15.26:0/186653020 shutdown_connections 2024-01-15T18:03:10.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.696+0000 7f3818a00640 1 -- 172.21.15.26:0/186653020 wait complete. 2024-01-15T18:03:10.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.697+0000 7f3818a00640 1 Processor -- start 2024-01-15T18:03:10.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.697+0000 7f3818a00640 1 -- start start 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.698+0000 7f3818a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37f40039a0 0x7f38141592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.698+0000 7f3818a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140ade10 0x7f3814153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.698+0000 7f3818a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ae290 0x7f38141543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.698+0000 7f3818a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140ad7f0 con 0x7f38140ae290 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.698+0000 7f3818a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814156310 con 0x7f37f40039a0 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.698+0000 7f3818a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140acc70 con 0x7f38140ade10 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.698+0000 7f3813200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ae290 0x7f38141543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.698+0000 7f3813200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ae290 0x7f38141543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32832/0 (socket says 172.21.15.26:32832) 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.698+0000 7f3813200640 1 -- 172.21.15.26:0/1583342860 learned_addr learned my addr 172.21.15.26:0/1583342860 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f3812800640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37f40039a0 0x7f38141592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:10.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f3813200640 1 -- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140ade10 msgr2=0x7f3814153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:10.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f3813200640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140ade10 0x7f3814153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f3813200640 1 -- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37f40039a0 msgr2=0x7f38141592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:10.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f3813200640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37f40039a0 0x7f38141592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f3813200640 1 -- 172.21.15.26:0/1583342860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140acf50 con 0x7f38140ae290 2024-01-15T18:03:10.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f3813200640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ae290 0x7f38141543b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f38140b3fa0 tx=0x7f380c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:10.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f380aa00640 1 -- 172.21.15.26:0/1583342860 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c00e070 con 0x7f38140ae290 2024-01-15T18:03:10.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f380aa00640 1 -- 172.21.15.26:0/1583342860 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c011e10 con 0x7f38140ae290 2024-01-15T18:03:10.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f380aa00640 1 -- 172.21.15.26:0/1583342860 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c003990 con 0x7f38140ae290 2024-01-15T18:03:10.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.699+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38140ad230 con 0x7f38140ae290 2024-01-15T18:03:10.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.700+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38140ac8d0 con 0x7f38140ae290 2024-01-15T18:03:10.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.701+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3814067450 con 0x7f38140ae290 2024-01-15T18:03:10.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.701+0000 7f380aa00640 1 -- 172.21.15.26:0/1583342860 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f380c003af0 con 0x7f38140ae290 2024-01-15T18:03:10.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.702+0000 7f380aa00640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06bbc0 0x7f37ec06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:10.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.702+0000 7f380aa00640 1 -- 172.21.15.26:0/1583342860 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f380c08b390 con 0x7f38140ae290 2024-01-15T18:03:10.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.703+0000 7f3812800640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06bbc0 0x7f37ec06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:10.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.704+0000 7f3812800640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06bbc0 0x7f37ec06e080 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3804033000 tx=0x7f3804009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:10.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.707+0000 7f380aa00640 1 -- 172.21.15.26:0/1583342860 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f380c055330 con 0x7f38140ae290 2024-01-15T18:03:10.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.896+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3814156510 con 0x7f38140ae290 2024-01-15T18:03:10.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.898+0000 7f380aa00640 1 -- 172.21.15.26:0/1583342860 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+13051 (secure 0 0 0) 0x7f380c058950 con 0x7f38140ae290 2024-01-15T18:03:10.902 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:10.902 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":63,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:03:10.120035+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6836/3403655022":"2024-01-16T18:03:07.106284+0000","172.21.15.124:6834/3403655022":"2024-01-16T18:03:07.106284+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:03:10.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.900+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06bbc0 msgr2=0x7f37ec06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:10.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.900+0000 7f3818a00640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06bbc0 0x7f37ec06e080 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3804033000 tx=0x7f3804009030 comp rx=0 tx=0).stop 2024-01-15T18:03:10.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.900+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ae290 msgr2=0x7f38141543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:10.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.900+0000 7f3818a00640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ae290 0x7f38141543b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f38140b3fa0 tx=0x7f380c0115a0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.901+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 shutdown_connections 2024-01-15T18:03:10.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.901+0000 7f3818a00640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38140ade10 0x7f3814153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.901+0000 7f3818a00640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f37ec06bbc0 0x7f37ec06e080 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.901+0000 7f3818a00640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38140ae290 0x7f38141543b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.901+0000 7f3818a00640 1 --2- 172.21.15.26:0/1583342860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37f40039a0 0x7f38141592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:10.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.901+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 >> 172.21.15.26:0/1583342860 conn(0x7f381406a800 msgr2=0x7f381413f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:10.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.901+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 shutdown_connections 2024-01-15T18:03:10.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:10.901+0000 7f3818a00640 1 -- 172.21.15.26:0/1583342860 wait complete. 2024-01-15T18:03:10.923 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:03:10.923 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:03:11.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.173+0000 7f9a94800640 1 Processor -- start 2024-01-15T18:03:11.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.173+0000 7f9a94800640 1 -- start start 2024-01-15T18:03:11.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.175+0000 7f9a94800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9a8c0cd170 0x7f9a8c071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.175+0000 7f9a94800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8c071690 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.175+0000 7f9a94800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a8c071970 con 0x7f9a8c14b2c0 2024-01-15T18:03:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.175+0000 7f9a94800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8c14b6a0 con 0x7f9a8c0cd170 2024-01-15T18:03:11.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.176+0000 7f9a91e00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f9a8c14b2c0 0x7f9a8c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:44546/0 (socket says 172.21.15.26:44546) 2024-01-15T18:03:11.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.176+0000 7f9a91400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9a8c0cf9a0 0x7f9a8c070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51152/0 (socket says 172.21.15.26:51152) 2024-01-15T18:03:11.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.176+0000 7f9a91e00640 1 -- 172.21.15.26:0/1301324387 learned_addr learned my addr 172.21.15.26:0/1301324387 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:11.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.176+0000 7f9a92800640 1 --2- 172.21.15.26:0/1301324387 >> v2:172.21.15.124:3300/0 conn(0x7f9a8c0cd170 0x7f9a8c071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.177+0000 7f9a90a00640 1 -- 172.21.15.26:0/1301324387 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321989427 0 0) 0x7f9a8c071690 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.177+0000 7f9a90a00640 1 -- 172.21.15.26:0/1301324387 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a6c003520 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.177+0000 7f9a90a00640 1 -- 172.21.15.26:0/1301324387 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789972041 0 0) 0x7f9a8c071970 con 0x7f9a8c14b2c0 2024-01-15T18:03:11.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.177+0000 7f9a90a00640 1 -- 172.21.15.26:0/1301324387 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a8c071690 con 0x7f9a8c14b2c0 2024-01-15T18:03:11.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.178+0000 7f9a92800640 1 -- 172.21.15.26:0/1301324387 >> v1:172.21.15.124:6789/0 conn(0x7f9a8c14b2c0 legacy=0x7f9a8c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.178+0000 7f9a92800640 1 -- 172.21.15.26:0/1301324387 >> v1:172.21.15.26:6789/0 conn(0x7f9a8c0cf9a0 legacy=0x7f9a8c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.178+0000 7f9a92800640 1 -- 172.21.15.26:0/1301324387 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c071c50 con 0x7f9a8c0cd170 2024-01-15T18:03:11.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.179+0000 7f9a92800640 1 --2- 172.21.15.26:0/1301324387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cd170 0x7f9a8c071070 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009d90 tx=0x7f9a8402bd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fcc5471e0477348 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:11.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.180+0000 7f9a90a00640 1 -- 172.21.15.26:0/1301324387 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a74001670 con 0x7f9a8c0cd170 2024-01-15T18:03:11.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.180+0000 7f9a90a00640 1 -- 172.21.15.26:0/1301324387 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8402c870 con 0x7f9a8c0cd170 2024-01-15T18:03:11.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.180+0000 7f9a90a00640 1 -- 172.21.15.26:0/1301324387 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8402cbe0 con 0x7f9a8c0cd170 2024-01-15T18:03:11.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.181+0000 7f9a94800640 1 -- 172.21.15.26:0/1301324387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cd170 msgr2=0x7f9a8c071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.181+0000 7f9a94800640 1 --2- 172.21.15.26:0/1301324387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cd170 0x7f9a8c071070 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9a84009d90 tx=0x7f9a8402bd40 comp rx=0 tx=0).stop 2024-01-15T18:03:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.181+0000 7f9a94800640 1 -- 172.21.15.26:0/1301324387 shutdown_connections 2024-01-15T18:03:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.181+0000 7f9a94800640 1 --2- 172.21.15.26:0/1301324387 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c0cd170 0x7f9a8c071070 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.181+0000 7f9a94800640 1 -- 172.21.15.26:0/1301324387 >> 172.21.15.26:0/1301324387 conn(0x7f9a8c06a800 msgr2=0x7f9a8c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:11.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.181+0000 7f9a94800640 1 -- 172.21.15.26:0/1301324387 shutdown_connections 2024-01-15T18:03:11.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.182+0000 7f9a94800640 1 -- 172.21.15.26:0/1301324387 wait complete. 2024-01-15T18:03:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.184+0000 7f9a94800640 1 Processor -- start 2024-01-15T18:03:11.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.184+0000 7f9a94800640 1 -- start start 2024-01-15T18:03:11.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.186+0000 7f9a94800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c0cd170 0x7f9a8c13ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.186+0000 7f9a94800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a8c0cf9a0 0x7f9a8c160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.186+0000 7f9a94800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c14b2c0 0x7f9a8c144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.186+0000 7f9a91400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a8c0cf9a0 0x7f9a8c160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.186+0000 7f9a91400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a8c0cf9a0 0x7f9a8c160d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32834/0 (socket says 172.21.15.26:32834) 2024-01-15T18:03:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.186+0000 7f9a91400640 1 -- 172.21.15.26:0/923744086 learned_addr learned my addr 172.21.15.26:0/923744086 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:11.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.187+0000 7f9a91e00640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c0cd170 0x7f9a8c13ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.187+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c15fca0 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.187+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c15ff80 con 0x7f9a8c14b2c0 2024-01-15T18:03:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.187+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c160260 con 0x7f9a8c0cd170 2024-01-15T18:03:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.187+0000 7f9a91400640 1 -- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c0cd170 msgr2=0x7f9a8c13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.187+0000 7f9a91400640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c0cd170 0x7f9a8c13ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.188+0000 7f9a92800640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c14b2c0 0x7f9a8c144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.188+0000 7f9a91400640 1 -- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c14b2c0 msgr2=0x7f9a8c144680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.188+0000 7f9a91400640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c14b2c0 0x7f9a8c144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.188+0000 7f9a91400640 1 -- 172.21.15.26:0/923744086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84009a70 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.188+0000 7f9a91400640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a8c0cf9a0 0x7f9a8c160d40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c8c0 tx=0x7f9a7400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.189+0000 7f9a8a000640 1 -- 172.21.15.26:0/923744086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a740074e0 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.189+0000 7f9a8a000640 1 -- 172.21.15.26:0/923744086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a74007640 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.190+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c160540 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.190+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c13f500 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.190+0000 7f9a8a000640 1 -- 172.21.15.26:0/923744086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a740074e0 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.192+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c160a60 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.194+0000 7f9a8a000640 1 -- 172.21.15.26:0/923744086 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f9a740047e0 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.195+0000 7f9a8a000640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a6406bc20 0x7f9a6406e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.196+0000 7f9a8a000640 1 -- 172.21.15.26:0/923744086 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f9a74087640 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.196+0000 7f9a91e00640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a6406bc20 0x7f9a6406e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.198+0000 7f9a91e00640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a6406bc20 0x7f9a6406e0e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0011e0 tx=0x7f9a7c007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:11.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.206+0000 7f9a8a000640 1 -- 172.21.15.26:0/923744086 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a8c160a60 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.584+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a8c160a60 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.585+0000 7f9a8a000640 1 -- 172.21.15.26:0/923744086 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a8c160a60 con 0x7f9a8c0cf9a0 2024-01-15T18:03:11.588 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:03:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a6406bc20 msgr2=0x7f9a6406e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a6406bc20 0x7f9a6406e0e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0011e0 tx=0x7f9a7c007d30 comp rx=0 tx=0).stop 2024-01-15T18:03:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a8c0cf9a0 msgr2=0x7f9a8c160d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a8c0cf9a0 0x7f9a8c160d40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f9a7400c8c0 tx=0x7f9a7400cd80 comp rx=0 tx=0).stop 2024-01-15T18:03:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 shutdown_connections 2024-01-15T18:03:11.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a6406bc20 0x7f9a6406e0e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a8c14b2c0 0x7f9a8c144680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a8c0cf9a0 0x7f9a8c160d40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 --2- 172.21.15.26:0/923744086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a8c0cd170 0x7f9a8c13ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 >> 172.21.15.26:0/923744086 conn(0x7f9a8c06a800 msgr2=0x7f9a8c161410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 shutdown_connections 2024-01-15T18:03:11.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.587+0000 7f9a94800640 1 -- 172.21.15.26:0/923744086 wait complete. 2024-01-15T18:03:11.608 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:03:11.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.795+0000 7f04b3000640 1 Processor -- start 2024-01-15T18:03:11.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.795+0000 7f04b3000640 1 -- start start 2024-01-15T18:03:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.796+0000 7f04b3000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 0x7f04ac0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.796+0000 7f04b3000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04ac0cf9f0 0x7f04ac0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.796+0000 7f04b3000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ac0d0a10 con 0x7f04ac161010 2024-01-15T18:03:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.796+0000 7f04b3000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ac0d03b0 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.796+0000 7f04b3000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0d1f50 con 0x7f04ac160920 2024-01-15T18:03:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.797+0000 7f04b1200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f04ac0cf9f0 0x7f04ac0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.797+0000 7f04b2600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f04ac161010 0x7f04ac1614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51164/0 (socket says 172.21.15.26:51164) 2024-01-15T18:03:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.797+0000 7f04b2600640 1 -- 172.21.15.26:0/190214727 learned_addr learned my addr 172.21.15.26:0/190214727 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.797+0000 7f04b1c00640 1 --2- 172.21.15.26:0/190214727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 0x7f04ac0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.797+0000 7f04b1200640 1 -- 172.21.15.26:0/190214727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 msgr2=0x7f04ac0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.797+0000 7f04b1200640 1 --2- 172.21.15.26:0/190214727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 0x7f04ac0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.797+0000 7f04b1200640 1 -- 172.21.15.26:0/190214727 >> v1:172.21.15.26:6789/0 conn(0x7f04ac161010 legacy=0x7f04ac1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.797+0000 7f04b1200640 1 -- 172.21.15.26:0/190214727 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0cf080 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.798+0000 7f04b1200640 1 --2- 172.21.15.26:0/190214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 0x7f04ac0cfe70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f049c009d90 tx=0x7f049c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d71c888fcc689 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04abe00640 1 -- 172.21.15.26:0/190214727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c02cc60 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04abe00640 1 -- 172.21.15.26:0/190214727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049c02cdc0 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04abe00640 1 -- 172.21.15.26:0/190214727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049c034660 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04b3000640 1 -- 172.21.15.26:0/190214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 msgr2=0x7f04ac0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04b3000640 1 --2- 172.21.15.26:0/190214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 0x7f04ac0cfe70 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f049c009d90 tx=0x7f049c02beb0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04b3000640 1 -- 172.21.15.26:0/190214727 shutdown_connections 2024-01-15T18:03:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04b3000640 1 --2- 172.21.15.26:0/190214727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 0x7f04ac0cfe70 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04b3000640 1 --2- 172.21.15.26:0/190214727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 0x7f04ac0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04b3000640 1 -- 172.21.15.26:0/190214727 >> 172.21.15.26:0/190214727 conn(0x7f04ac0d3090 msgr2=0x7f04ac0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04b3000640 1 -- 172.21.15.26:0/190214727 shutdown_connections 2024-01-15T18:03:11.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.799+0000 7f04b3000640 1 -- 172.21.15.26:0/190214727 wait complete. 2024-01-15T18:03:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.801+0000 7f04b3000640 1 Processor -- start 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.801+0000 7f04b3000640 1 -- start start 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.802+0000 7f04b3000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 0x7f04ac151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.802+0000 7f04b3000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 0x7f04ac0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.802+0000 7f04b3000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04ac161010 0x7f04ac15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.802+0000 7f04b3000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0afe60 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.802+0000 7f04b3000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0cbb70 con 0x7f04ac161010 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.802+0000 7f04b3000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac151b50 con 0x7f04ac160920 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.802+0000 7f04b1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 0x7f04ac151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.802+0000 7f04b1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 0x7f04ac151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32838/0 (socket says 172.21.15.26:32838) 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.802+0000 7f04b1c00640 1 -- 172.21.15.26:0/906874907 learned_addr learned my addr 172.21.15.26:0/906874907 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:11.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.803+0000 7f04b1200640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 0x7f04ac0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.803+0000 7f04b1c00640 1 -- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 msgr2=0x7f04ac0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.803+0000 7f04b2600640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04ac161010 0x7f04ac15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.803+0000 7f04b1c00640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 0x7f04ac0ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.803+0000 7f04b1c00640 1 -- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04ac161010 msgr2=0x7f04ac15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.803+0000 7f04b1c00640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04ac161010 0x7f04ac15e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.803+0000 7f04b1c00640 1 -- 172.21.15.26:0/906874907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c009a70 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.803+0000 7f04b1c00640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 0x7f04ac151610 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f049400d800 tx=0x7f049400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.803+0000 7f04a9600640 1 -- 172.21.15.26:0/906874907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049400b6c0 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.804+0000 7f04a9600640 1 -- 172.21.15.26:0/906874907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049400b820 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.804+0000 7f04a9600640 1 -- 172.21.15.26:0/906874907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049400bbb0 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.804+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac0caad0 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.804+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac0cb090 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.805+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ac067450 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.806+0000 7f04a9600640 1 -- 172.21.15.26:0/906874907 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f049400bd10 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.806+0000 7f04a9600640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f048406bc10 0x7f048406e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.807+0000 7f04a9600640 1 -- 172.21.15.26:0/906874907 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f0494088c40 con 0x7f04ac0cf9f0 2024-01-15T18:03:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.807+0000 7f04b1200640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f048406bc10 0x7f048406e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:11.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.808+0000 7f04b1200640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f048406bc10 0x7f048406e0d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f049c036000 tx=0x7f049c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:11.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:11.811+0000 7f04a9600640 1 -- 172.21.15.26:0/906874907 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0494055310 con 0x7f04ac0cf9f0 2024-01-15T18:03:12.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.007+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f04ac0c98f0 con 0x7f04ac0cf9f0 2024-01-15T18:03:12.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.008+0000 7f04a9600640 1 -- 172.21.15.26:0/906874907 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v63) v1 ==== 80+0+124 (secure 0 0 0) 0x7f049405de60 con 0x7f04ac0cf9f0 2024-01-15T18:03:12.012 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6836/3403655022 2024-01-16T18:03:07.106284+0000 2024-01-15T18:03:12.013 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6834/3403655022 2024-01-16T18:03:07.106284+0000 2024-01-15T18:03:12.013 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:03:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f048406bc10 msgr2=0x7f048406e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f048406bc10 0x7f048406e0d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f049c036000 tx=0x7f049c030000 comp rx=0 tx=0).stop 2024-01-15T18:03:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 msgr2=0x7f04ac151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 0x7f04ac151610 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f049400d800 tx=0x7f049400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 shutdown_connections 2024-01-15T18:03:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f04ac160920 0x7f04ac0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f048406bc10 0x7f048406e0d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f04ac161010 0x7f04ac15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 --2- 172.21.15.26:0/906874907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04ac0cf9f0 0x7f04ac151610 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 >> 172.21.15.26:0/906874907 conn(0x7f04ac0d3090 msgr2=0x7f04ac0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 shutdown_connections 2024-01-15T18:03:12.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.011+0000 7f04b3000640 1 -- 172.21.15.26:0/906874907 wait complete. 2024-01-15T18:03:12.034 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-01-15T18:03:12.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:12.034 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:03:12.130 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:03:12.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:12.131 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:03:12.140 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[57105]: fuse finished with error 0 and tester_r 0 2024-01-15T18:03:12.361 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:03:12.361 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:03:12.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:03:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.648+0000 7fa690600640 1 Processor -- start 2024-01-15T18:03:12.652 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.649+0000 7fa690600640 1 -- start start 2024-01-15T18:03:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.650+0000 7fa690600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6880ae5a0 0x7fa68815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.650+0000 7fa690600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6880cf290 0x7fa6880cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.650+0000 7fa690600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa688070c40 con 0x7fa6880b3ae0 2024-01-15T18:03:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.650+0000 7fa690600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa688071450 con 0x7fa6880ae5a0 2024-01-15T18:03:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.650+0000 7fa690600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6880cfc60 con 0x7fa6880cf290 2024-01-15T18:03:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.651+0000 7fa68dc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6880ae5a0 0x7fa68815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.651+0000 7fa68dc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa6880ae5a0 0x7fa68815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34948/0 (socket says 172.21.15.26:34948) 2024-01-15T18:03:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.651+0000 7fa68dc00640 1 -- 172.21.15.26:0/3840222835 learned_addr learned my addr 172.21.15.26:0/3840222835 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:12.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.651+0000 7fa68dc00640 1 -- 172.21.15.26:0/3840222835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6880cf290 msgr2=0x7fa6880cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.651+0000 7fa68dc00640 1 --2- 172.21.15.26:0/3840222835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6880cf290 0x7fa6880cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.651+0000 7fa68dc00640 1 -- 172.21.15.26:0/3840222835 >> v1:172.21.15.124:6789/0 conn(0x7fa6880b3ae0 legacy=0x7fa68806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:12.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.651+0000 7fa68dc00640 1 -- 172.21.15.26:0/3840222835 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68806c470 con 0x7fa6880ae5a0 2024-01-15T18:03:12.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.652+0000 7fa68dc00640 1 --2- 172.21.15.26:0/3840222835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880ae5a0 0x7fa68815e430 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa678009e50 tx=0x7fa67802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c7ec2b07a79baa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.653+0000 7fa687e00640 1 -- 172.21.15.26:0/3840222835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67802cb00 con 0x7fa6880ae5a0 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.653+0000 7fa687e00640 1 -- 172.21.15.26:0/3840222835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67802cc60 con 0x7fa6880ae5a0 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.653+0000 7fa687e00640 1 -- 172.21.15.26:0/3840222835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780343f0 con 0x7fa6880ae5a0 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.654+0000 7fa690600640 1 -- 172.21.15.26:0/3840222835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880ae5a0 msgr2=0x7fa68815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.654+0000 7fa690600640 1 --2- 172.21.15.26:0/3840222835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880ae5a0 0x7fa68815e430 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa678009e50 tx=0x7fa67802bf60 comp rx=0 tx=0).stop 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.654+0000 7fa690600640 1 -- 172.21.15.26:0/3840222835 shutdown_connections 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.654+0000 7fa690600640 1 --2- 172.21.15.26:0/3840222835 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6880cf290 0x7fa6880cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.654+0000 7fa690600640 1 --2- 172.21.15.26:0/3840222835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880ae5a0 0x7fa68815e430 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.654+0000 7fa690600640 1 -- 172.21.15.26:0/3840222835 >> 172.21.15.26:0/3840222835 conn(0x7fa68806a800 msgr2=0x7fa68806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:12.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.654+0000 7fa690600640 1 -- 172.21.15.26:0/3840222835 shutdown_connections 2024-01-15T18:03:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.654+0000 7fa690600640 1 -- 172.21.15.26:0/3840222835 wait complete. 2024-01-15T18:03:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.656+0000 7fa690600640 1 Processor -- start 2024-01-15T18:03:12.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.656+0000 7fa690600640 1 -- start start 2024-01-15T18:03:12.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.657+0000 7fa690600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6880ae5a0 0x7fa68814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:12.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.657+0000 7fa690600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880b3ae0 0x7fa6881518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.657+0000 7fa690600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6880cf290 0x7fa688155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.657+0000 7fa690600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68814f7f0 con 0x7fa6880b3ae0 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.657+0000 7fa690600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68814f510 con 0x7fa6880cf290 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.657+0000 7fa690600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881507b0 con 0x7fa6880ae5a0 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.657+0000 7fa68d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880b3ae0 0x7fa6881518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.657+0000 7fa68d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880b3ae0 0x7fa6881518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34956/0 (socket says 172.21.15.26:34956) 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.657+0000 7fa68d200640 1 -- 172.21.15.26:0/2591624266 learned_addr learned my addr 172.21.15.26:0/2591624266 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa68dc00640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6880ae5a0 0x7fa68814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa68e600640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6880cf290 0x7fa688155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:12.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa68d200640 1 -- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6880ae5a0 msgr2=0x7fa68814eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa68d200640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6880ae5a0 0x7fa68814eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa68d200640 1 -- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6880cf290 msgr2=0x7fa688155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa68d200640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6880cf290 0x7fa688155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa68d200640 1 -- 172.21.15.26:0/2591624266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678009b30 con 0x7fa6880b3ae0 2024-01-15T18:03:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa68d200640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880b3ae0 0x7fa6881518a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa67000c410 tx=0x7fa67000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa685600640 1 -- 172.21.15.26:0/2591624266 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6700035f0 con 0x7fa6880b3ae0 2024-01-15T18:03:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa685600640 1 -- 172.21.15.26:0/2591624266 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670003750 con 0x7fa6880b3ae0 2024-01-15T18:03:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa688151380 con 0x7fa6880b3ae0 2024-01-15T18:03:12.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa685600640 1 -- 172.21.15.26:0/2591624266 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670003b40 con 0x7fa6880b3ae0 2024-01-15T18:03:12.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.658+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa688150a90 con 0x7fa6880b3ae0 2024-01-15T18:03:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.660+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6880675d0 con 0x7fa6880b3ae0 2024-01-15T18:03:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.661+0000 7fa685600640 1 -- 172.21.15.26:0/2591624266 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fa670003ca0 con 0x7fa6880b3ae0 2024-01-15T18:03:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.661+0000 7fa685600640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66006bbf0 0x7fa66006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.662+0000 7fa685600640 1 -- 172.21.15.26:0/2591624266 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fa6700876a0 con 0x7fa6880b3ae0 2024-01-15T18:03:12.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.662+0000 7fa68dc00640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66006bbf0 0x7fa66006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:12.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.663+0000 7fa68dc00640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66006bbf0 0x7fa66006e0b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa678035000 tx=0x7fa678030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:12.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.666+0000 7fa685600640 1 -- 172.21.15.26:0/2591624266 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa670053f60 con 0x7fa6880b3ae0 2024-01-15T18:03:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.860+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa688161ae0 con 0x7fa6880b3ae0 2024-01-15T18:03:12.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.861+0000 7fa685600640 1 -- 172.21.15.26:0/2591624266 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+13051 (secure 0 0 0) 0x7fa670057580 con 0x7fa6880b3ae0 2024-01-15T18:03:12.865 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:12.865 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":63,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:03:10.120035+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6836/3403655022":"2024-01-16T18:03:07.106284+0000","172.21.15.124:6834/3403655022":"2024-01-16T18:03:07.106284+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:03:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66006bbf0 msgr2=0x7fa66006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66006bbf0 0x7fa66006e0b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa678035000 tx=0x7fa678030000 comp rx=0 tx=0).stop 2024-01-15T18:03:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880b3ae0 msgr2=0x7fa6881518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880b3ae0 0x7fa6881518a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa67000c410 tx=0x7fa67000c8d0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 shutdown_connections 2024-01-15T18:03:12.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66006bbf0 0x7fa66006e0b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa6880cf290 0x7fa688155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa6880b3ae0 0x7fa6881518a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 --2- 172.21.15.26:0/2591624266 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa6880ae5a0 0x7fa68814eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 >> 172.21.15.26:0/2591624266 conn(0x7fa68806a800 msgr2=0x7fa6880b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 shutdown_connections 2024-01-15T18:03:12.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:12.864+0000 7fa690600640 1 -- 172.21.15.26:0/2591624266 wait complete. 2024-01-15T18:03:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:03:12.888 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:03:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.115+0000 7f8b69800640 1 Processor -- start 2024-01-15T18:03:13.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.115+0000 7f8b69800640 1 -- start start 2024-01-15T18:03:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.116+0000 7f8b69800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8b640cf9a0 0x7f8b6415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.116+0000 7f8b69800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b6415e1e0 0x7f8b640702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.117+0000 7f8b69800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b640b1ee0 con 0x7f8b64143ae0 2024-01-15T18:03:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.117+0000 7f8b69800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b640b2780 con 0x7f8b640cf9a0 2024-01-15T18:03:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.117+0000 7f8b69800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6415e990 con 0x7f8b6415e1e0 2024-01-15T18:03:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.117+0000 7f8b62a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8b64143ae0 0x7f8b640cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46138/0 (socket says 172.21.15.26:46138) 2024-01-15T18:03:13.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.117+0000 7f8b62a00640 1 -- 172.21.15.26:0/3187538939 learned_addr learned my addr 172.21.15.26:0/3187538939 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.117+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724059760 0 0) 0x7f8b640b1ee0 con 0x7f8b64143ae0 2024-01-15T18:03:13.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.117+0000 7f8b63400640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b6415e1e0 0x7f8b640702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.117+0000 7f8b62000640 1 --2- 172.21.15.26:0/3187538939 >> v2:172.21.15.124:3300/0 conn(0x7f8b640cf9a0 0x7f8b6415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.117+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b44003520 con 0x7f8b64143ae0 2024-01-15T18:03:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3462221443 0 0) 0x7f8b44003520 con 0x7f8b64143ae0 2024-01-15T18:03:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b6415e1e0 msgr2=0x7f8b640702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b6415e1e0 0x7f8b640702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 >> v2:172.21.15.124:3300/0 conn(0x7f8b640cf9a0 msgr2=0x7f8b6415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 --2- 172.21.15.26:0/3187538939 >> v2:172.21.15.124:3300/0 conn(0x7f8b640cf9a0 0x7f8b6415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b640b2130 con 0x7f8b64143ae0 2024-01-15T18:03:13.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8b54001670 con 0x7f8b64143ae0 2024-01-15T18:03:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 >> v1:172.21.15.26:6789/0 conn(0x7f8b64143ae0 legacy=0x7f8b640cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b44003970 0x7f8b44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b440047c0 0x7f8b44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b6415e1e0 0x7f8b44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640b2780 con 0x7f8b440047c0 2024-01-15T18:03:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6415e990 con 0x7f8b44003970 2024-01-15T18:03:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4400ba60 con 0x7f8b6415e1e0 2024-01-15T18:03:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.118+0000 7f8b62000640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b440047c0 0x7f8b44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.119+0000 7f8b62a00640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b44003970 0x7f8b44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.119+0000 7f8b62000640 1 -- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b6415e1e0 msgr2=0x7f8b44005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.119+0000 7f8b62000640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b6415e1e0 0x7f8b44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.119+0000 7f8b62000640 1 -- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b44003970 msgr2=0x7f8b44003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.119+0000 7f8b62000640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b44003970 0x7f8b44003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.119+0000 7f8b62000640 1 -- 172.21.15.26:0/3187538939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b44003520 con 0x7f8b440047c0 2024-01-15T18:03:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.119+0000 7f8b62a00640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b44003970 0x7f8b44003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.120+0000 7f8b62000640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b440047c0 0x7f8b44004c40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00ba00 tx=0x7f8b440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e36937d2947ab4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:13.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.120+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c032630 con 0x7f8b440047c0 2024-01-15T18:03:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.120+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c032790 con 0x7f8b440047c0 2024-01-15T18:03:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.120+0000 7f8b61600640 1 -- 172.21.15.26:0/3187538939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c032b20 con 0x7f8b440047c0 2024-01-15T18:03:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.121+0000 7f8b69800640 1 -- 172.21.15.26:0/3187538939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b440047c0 msgr2=0x7f8b44004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.121+0000 7f8b69800640 1 --2- 172.21.15.26:0/3187538939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b440047c0 0x7f8b44004c40 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00ba00 tx=0x7f8b440057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.121+0000 7f8b63400640 1 -- 172.21.15.26:0/3187538939 reap_dead start 2024-01-15T18:03:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.121+0000 7f8b69800640 1 -- 172.21.15.26:0/3187538939 shutdown_connections 2024-01-15T18:03:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.121+0000 7f8b69800640 1 -- 172.21.15.26:0/3187538939 >> 172.21.15.26:0/3187538939 conn(0x7f8b6406a800 msgr2=0x7f8b6406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.121+0000 7f8b69800640 1 -- 172.21.15.26:0/3187538939 shutdown_connections 2024-01-15T18:03:13.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.121+0000 7f8b69800640 1 -- 172.21.15.26:0/3187538939 wait complete. 2024-01-15T18:03:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.123+0000 7f8b69800640 1 Processor -- start 2024-01-15T18:03:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.123+0000 7f8b69800640 1 -- start start 2024-01-15T18:03:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.124+0000 7f8b69800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b440047c0 0x7f8b640c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.124+0000 7f8b69800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640c4760 0x7f8b6415ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.124+0000 7f8b69800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b64159560 0x7f8b64159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.124+0000 7f8b69800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b64159f50 con 0x7f8b640c4760 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.124+0000 7f8b69800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640c5220 con 0x7f8b440047c0 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.124+0000 7f8b69800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640aca50 con 0x7f8b64159560 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.124+0000 7f8b62000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640c4760 0x7f8b6415ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.124+0000 7f8b62000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640c4760 0x7f8b6415ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34966/0 (socket says 172.21.15.26:34966) 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.124+0000 7f8b62000640 1 -- 172.21.15.26:0/2029469759 learned_addr learned my addr 172.21.15.26:0/2029469759 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.125+0000 7f8b63400640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b64159560 0x7f8b64159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.125+0000 7f8b62a00640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b440047c0 0x7f8b640c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.125+0000 7f8b62000640 1 -- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b64159560 msgr2=0x7f8b64159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.125+0000 7f8b62000640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b64159560 0x7f8b64159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.125+0000 7f8b62000640 1 -- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b440047c0 msgr2=0x7f8b640c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.125+0000 7f8b62000640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b440047c0 0x7f8b640c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.125+0000 7f8b62000640 1 -- 172.21.15.26:0/2029469759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b640c4f40 con 0x7f8b640c4760 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.125+0000 7f8b62000640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640c4760 0x7f8b6415ca00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c02beb0 tx=0x7f8b4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.126+0000 7f8b5aa00640 1 -- 172.21.15.26:0/2029469759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c004220 con 0x7f8b640c4760 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.126+0000 7f8b5aa00640 1 -- 172.21.15.26:0/2029469759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c0043a0 con 0x7f8b640c4760 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.126+0000 7f8b5aa00640 1 -- 172.21.15.26:0/2029469759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c03ba70 con 0x7f8b640c4760 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.126+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b640acd10 con 0x7f8b640c4760 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.126+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b640c4be0 con 0x7f8b640c4760 2024-01-15T18:03:13.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.127+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b640712d0 con 0x7f8b640c4760 2024-01-15T18:03:13.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.128+0000 7f8b5aa00640 1 -- 172.21.15.26:0/2029469759 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f8b4c03bbd0 con 0x7f8b640c4760 2024-01-15T18:03:13.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.129+0000 7f8b5aa00640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b3c06bb80 0x7f8b3c06e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.129+0000 7f8b5aa00640 1 -- 172.21.15.26:0/2029469759 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f8b4c0aadb0 con 0x7f8b640c4760 2024-01-15T18:03:13.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.130+0000 7f8b62a00640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b3c06bb80 0x7f8b3c06e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.131+0000 7f8b62a00640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b3c06bb80 0x7f8b3c06e040 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8b54001eb0 tx=0x7f8b54009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:13.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.133+0000 7f8b5aa00640 1 -- 172.21.15.26:0/2029469759 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b4c077670 con 0x7f8b640c4760 2024-01-15T18:03:13.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.329+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b640712d0 con 0x7f8b640c4760 2024-01-15T18:03:13.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.330+0000 7f8b5aa00640 1 -- 172.21.15.26:0/2029469759 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b4c036020 con 0x7f8b640c4760 2024-01-15T18:03:13.333 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:03:13.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b3c06bb80 msgr2=0x7f8b3c06e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b3c06bb80 0x7f8b3c06e040 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8b54001eb0 tx=0x7f8b54009040 comp rx=0 tx=0).stop 2024-01-15T18:03:13.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640c4760 msgr2=0x7f8b6415ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640c4760 0x7f8b6415ca00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c02beb0 tx=0x7f8b4c004730 comp rx=0 tx=0).stop 2024-01-15T18:03:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 shutdown_connections 2024-01-15T18:03:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b64159560 0x7f8b64159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b3c06bb80 0x7f8b3c06e040 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b640c4760 0x7f8b6415ca00 secure :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c02beb0 tx=0x7f8b4c004730 comp rx=0 tx=0).stop 2024-01-15T18:03:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 --2- 172.21.15.26:0/2029469759 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b440047c0 0x7f8b640c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 >> 172.21.15.26:0/2029469759 conn(0x7f8b6406a800 msgr2=0x7f8b6406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.332+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 shutdown_connections 2024-01-15T18:03:13.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.333+0000 7f8b69800640 1 -- 172.21.15.26:0/2029469759 wait complete. 2024-01-15T18:03:13.357 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:03:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.578+0000 7f46fb000640 1 Processor -- start 2024-01-15T18:03:13.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.578+0000 7f46fb000640 1 -- start start 2024-01-15T18:03:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.580+0000 7f46fb000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f46f40d0100 0x7f46f40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.580+0000 7f46fb000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f46f40d0f40 0x7f46f40d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.580+0000 7f46fb000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f40d0a10 con 0x7f46f4160920 2024-01-15T18:03:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.580+0000 7f46fb000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f40cf080 con 0x7f46f40d0100 2024-01-15T18:03:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.580+0000 7f46fb000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f40c6be0 con 0x7f46f40d0f40 2024-01-15T18:03:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.580+0000 7f46f9c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f46f4160920 0x7f46f40cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46144/0 (socket says 172.21.15.26:46144) 2024-01-15T18:03:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.580+0000 7f46f9c00640 1 -- 172.21.15.26:0/2674243831 learned_addr learned my addr 172.21.15.26:0/2674243831 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:13.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.580+0000 7f46f9200640 1 --2- 172.21.15.26:0/2674243831 >> v2:172.21.15.26:3300/0 conn(0x7f46f40d0100 0x7f46f40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.581+0000 7f46fa600640 1 --2- 172.21.15.26:0/2674243831 >> v2:172.21.15.124:3300/0 conn(0x7f46f40d0f40 0x7f46f40d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.581+0000 7f46f9200640 1 -- 172.21.15.26:0/2674243831 >> v2:172.21.15.124:3300/0 conn(0x7f46f40d0f40 msgr2=0x7f46f40d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.581+0000 7f46f9200640 1 --2- 172.21.15.26:0/2674243831 >> v2:172.21.15.124:3300/0 conn(0x7f46f40d0f40 0x7f46f40d1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.581+0000 7f46f9200640 1 -- 172.21.15.26:0/2674243831 >> v1:172.21.15.26:6789/0 conn(0x7f46f4160920 legacy=0x7f46f40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.581+0000 7f46f9200640 1 -- 172.21.15.26:0/2674243831 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f40ce7e0 con 0x7f46f40d0100 2024-01-15T18:03:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.582+0000 7f46f9200640 1 --2- 172.21.15.26:0/2674243831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f40d0100 0x7f46f40b3f30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f46f41620b0 tx=0x7f46dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a282ef69fa5f2687 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46f3e00640 1 -- 172.21.15.26:0/2674243831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc031830 con 0x7f46f40d0100 2024-01-15T18:03:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46f3e00640 1 -- 172.21.15.26:0/2674243831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc031990 con 0x7f46f40d0100 2024-01-15T18:03:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46f3e00640 1 -- 172.21.15.26:0/2674243831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc031d20 con 0x7f46f40d0100 2024-01-15T18:03:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46fb000640 1 -- 172.21.15.26:0/2674243831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f40d0100 msgr2=0x7f46f40b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46fb000640 1 --2- 172.21.15.26:0/2674243831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f40d0100 0x7f46f40b3f30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f46f41620b0 tx=0x7f46dc02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46fb000640 1 -- 172.21.15.26:0/2674243831 shutdown_connections 2024-01-15T18:03:13.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46fb000640 1 --2- 172.21.15.26:0/2674243831 >> v2:172.21.15.124:3300/0 conn(0x7f46f40d0f40 0x7f46f40d1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46fb000640 1 --2- 172.21.15.26:0/2674243831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f40d0100 0x7f46f40b3f30 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46fb000640 1 -- 172.21.15.26:0/2674243831 >> 172.21.15.26:0/2674243831 conn(0x7f46f40d3090 msgr2=0x7f46f40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46fb000640 1 -- 172.21.15.26:0/2674243831 shutdown_connections 2024-01-15T18:03:13.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.583+0000 7f46fb000640 1 -- 172.21.15.26:0/2674243831 wait complete. 2024-01-15T18:03:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.585+0000 7f46fb000640 1 Processor -- start 2024-01-15T18:03:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.585+0000 7f46fb000640 1 -- start start 2024-01-15T18:03:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.586+0000 7f46fb000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46f40d0100 0x7f46f40ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.586+0000 7f46fb000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46f40d0f40 0x7f46f414c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.586+0000 7f46fb000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f4160920 0x7f46f40cca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.586+0000 7f46fb000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f414ca90 con 0x7f46f4160920 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.586+0000 7f46fb000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40ccfc0 con 0x7f46f40d0f40 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.586+0000 7f46fb000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40ca7f0 con 0x7f46f40d0100 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46fa600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f4160920 0x7f46f40cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46fa600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f4160920 0x7f46f40cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34992/0 (socket says 172.21.15.26:34992) 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46fa600640 1 -- 172.21.15.26:0/2232576284 learned_addr learned my addr 172.21.15.26:0/2232576284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46f9200640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46f40d0f40 0x7f46f414c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46f9c00640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46f40d0100 0x7f46f40ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46fa600640 1 -- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46f40d0100 msgr2=0x7f46f40ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46fa600640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46f40d0100 0x7f46f40ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46fa600640 1 -- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46f40d0f40 msgr2=0x7f46f414c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46fa600640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46f40d0f40 0x7f46f414c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46fa600640 1 -- 172.21.15.26:0/2232576284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc009400 con 0x7f46f4160920 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.587+0000 7f46f9c00640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46f40d0100 0x7f46f40ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.588+0000 7f46fa600640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f4160920 0x7f46f40cca80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f46ec007e40 tx=0x7f46ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.588+0000 7f46f1600640 1 -- 172.21.15.26:0/2232576284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec014620 con 0x7f46f4160920 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.588+0000 7f46f1600640 1 -- 172.21.15.26:0/2232576284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec00ee90 con 0x7f46f4160920 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.588+0000 7f46f1600640 1 -- 172.21.15.26:0/2232576284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0149e0 con 0x7f46f4160920 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.588+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f40b8240 con 0x7f46f4160920 2024-01-15T18:03:13.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.588+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f40b87c0 con 0x7f46f4160920 2024-01-15T18:03:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.589+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f4067450 con 0x7f46f4160920 2024-01-15T18:03:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.591+0000 7f46f1600640 1 -- 172.21.15.26:0/2232576284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f46ec014b40 con 0x7f46f4160920 2024-01-15T18:03:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.591+0000 7f46f1600640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f46cc06bbc0 0x7f46cc06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.592+0000 7f46f1600640 1 -- 172.21.15.26:0/2232576284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f46ec088ed0 con 0x7f46f4160920 2024-01-15T18:03:13.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.592+0000 7f46f9c00640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f46cc06bbc0 0x7f46cc06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:13.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.593+0000 7f46f9c00640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f46cc06bbc0 0x7f46cc06e080 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f46e4005880 tx=0x7f46e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:13.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.596+0000 7f46f1600640 1 -- 172.21.15.26:0/2232576284 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46ec055790 con 0x7f46f4160920 2024-01-15T18:03:13.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.806+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f46f408c7e0 con 0x7f46f4160920 2024-01-15T18:03:13.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.806+0000 7f46f1600640 1 -- 172.21.15.26:0/2232576284 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v63) v1 ==== 80+0+124 (secure 0 0 0) 0x7f46ec058db0 con 0x7f46f4160920 2024-01-15T18:03:13.810 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6836/3403655022 2024-01-16T18:03:07.106284+0000 2024-01-15T18:03:13.810 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6834/3403655022 2024-01-16T18:03:07.106284+0000 2024-01-15T18:03:13.811 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:03:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f46cc06bbc0 msgr2=0x7f46cc06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f46cc06bbc0 0x7f46cc06e080 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f46e4005880 tx=0x7f46e4009040 comp rx=0 tx=0).stop 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f4160920 msgr2=0x7f46f40cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f4160920 0x7f46f40cca80 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f46ec007e40 tx=0x7f46ec00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 shutdown_connections 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f46cc06bbc0 0x7f46cc06e080 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46f4160920 0x7f46f40cca80 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f46f40d0f40 0x7f46f414c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 --2- 172.21.15.26:0/2232576284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f46f40d0100 0x7f46f40ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.809+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 >> 172.21.15.26:0/2232576284 conn(0x7f46f40d3090 msgr2=0x7f46f415df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.810+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 shutdown_connections 2024-01-15T18:03:13.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:13.810+0000 7f46fb000640 1 -- 172.21.15.26:0/2232576284 wait complete. 2024-01-15T18:03:13.832 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi124.front.sepia.ceph.com... 2024-01-15T18:03:13.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:13.833 DEBUG:teuthology.orchestra.run.smithi124:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:03:13.930 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[44855]: fuse finished with error 0 and tester_r 0 2024-01-15T18:03:13.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:03:13.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:13.936 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:03:14.107 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:03:14.107 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:03:14.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:03:14.168 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:03:14.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:14.168 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:03:14.223 INFO:teuthology.orchestra.run.smithi026.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:03:14.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:03:14.225 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:03:14.225 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:03:14.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:03:14.281 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:03:14.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:14.282 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:03:14.333 INFO:teuthology.orchestra.run.smithi124.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:03:14.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:03:14.334 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:03:14.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.564+0000 7f0009a00640 1 Processor -- start 2024-01-15T18:03:14.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.564+0000 7f0009a00640 1 -- start start 2024-01-15T18:03:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.565+0000 7f0009a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00040cc500 0x7f00040cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.565+0000 7f0009a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f000406d160 0x7f000406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.565+0000 7f0009a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0004070e50 con 0x7f0004143c40 2024-01-15T18:03:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.565+0000 7f0009a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00040723c0 con 0x7f000406d160 2024-01-15T18:03:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.565+0000 7f0009a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0004144020 con 0x7f00040cc500 2024-01-15T18:03:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.566+0000 7f0003400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00040cc500 0x7f00040cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.566+0000 7f0009000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f000406d160 0x7f000406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:14.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.566+0000 7f0003400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00040cc500 0x7f00040cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:47302/0 (socket says 172.21.15.26:47302) 2024-01-15T18:03:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.566+0000 7f0003400640 1 -- 172.21.15.26:0/1502816640 learned_addr learned my addr 172.21.15.26:0/1502816640 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.567+0000 7f0003400640 1 -- 172.21.15.26:0/1502816640 >> v2:172.21.15.124:3300/0 conn(0x7f000406d160 msgr2=0x7f000406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:14.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.567+0000 7f0003400640 1 --2- 172.21.15.26:0/1502816640 >> v2:172.21.15.124:3300/0 conn(0x7f000406d160 0x7f000406d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.567+0000 7f0003400640 1 -- 172.21.15.26:0/1502816640 >> v1:172.21.15.124:6789/0 conn(0x7f0004143c40 legacy=0x7f0004071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:14.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.567+0000 7f0003400640 1 -- 172.21.15.26:0/1502816640 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00040ccea0 con 0x7f00040cc500 2024-01-15T18:03:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.568+0000 7f0003400640 1 --2- 172.21.15.26:0/1502816640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00040cc500 0x7f00040cc960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7effec009870 tx=0x7effec02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa4287de62c5f4a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.569+0000 7f0002a00640 1 -- 172.21.15.26:0/1502816640 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec031640 con 0x7f00040cc500 2024-01-15T18:03:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.569+0000 7f0002a00640 1 -- 172.21.15.26:0/1502816640 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec0317a0 con 0x7f00040cc500 2024-01-15T18:03:14.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.569+0000 7f0002a00640 1 -- 172.21.15.26:0/1502816640 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec031b30 con 0x7f00040cc500 2024-01-15T18:03:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.570+0000 7f0009a00640 1 -- 172.21.15.26:0/1502816640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00040cc500 msgr2=0x7f00040cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:14.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.570+0000 7f0009a00640 1 --2- 172.21.15.26:0/1502816640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00040cc500 0x7f00040cc960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7effec009870 tx=0x7effec02b8f0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.570+0000 7f0009a00640 1 -- 172.21.15.26:0/1502816640 shutdown_connections 2024-01-15T18:03:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.570+0000 7f0009a00640 1 --2- 172.21.15.26:0/1502816640 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f00040cc500 0x7f00040cc960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.570+0000 7f0009a00640 1 --2- 172.21.15.26:0/1502816640 >> v2:172.21.15.124:3300/0 conn(0x7f000406d160 0x7f000406d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.570+0000 7f0009a00640 1 -- 172.21.15.26:0/1502816640 >> 172.21.15.26:0/1502816640 conn(0x7f000406a800 msgr2=0x7f000406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.570+0000 7f0009a00640 1 -- 172.21.15.26:0/1502816640 shutdown_connections 2024-01-15T18:03:14.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.570+0000 7f0009a00640 1 -- 172.21.15.26:0/1502816640 wait complete. 2024-01-15T18:03:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.572+0000 7f0009a00640 1 Processor -- start 2024-01-15T18:03:14.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.572+0000 7f0009a00640 1 -- start start 2024-01-15T18:03:14.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.573+0000 7f0009a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000406d160 0x7f0004139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.573+0000 7f0009a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00040cc500 0x7f0004151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.573+0000 7f0009a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0004143c40 0x7f0004157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.573+0000 7f0009a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004139670 con 0x7f00040cc500 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.573+0000 7f0009a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0004139390 con 0x7f000406d160 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.573+0000 7f0009a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0004157390 con 0x7f0004143c40 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.573+0000 7f0003400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00040cc500 0x7f0004151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.573+0000 7f0003400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00040cc500 0x7f0004151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35002/0 (socket says 172.21.15.26:35002) 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.573+0000 7f0003400640 1 -- 172.21.15.26:0/648896705 learned_addr learned my addr 172.21.15.26:0/648896705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.574+0000 7f0003e00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000406d160 0x7f0004139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:14.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.574+0000 7f0009000640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0004143c40 0x7f0004157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.574+0000 7f0003400640 1 -- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0004143c40 msgr2=0x7f0004157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.574+0000 7f0003400640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0004143c40 0x7f0004157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.574+0000 7f0003400640 1 -- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000406d160 msgr2=0x7f0004139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.574+0000 7f0003400640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000406d160 0x7f0004139ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.574+0000 7f0003400640 1 -- 172.21.15.26:0/648896705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec0094d0 con 0x7f00040cc500 2024-01-15T18:03:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.574+0000 7f0009000640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0004143c40 0x7f0004157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.574+0000 7f0003400640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00040cc500 0x7f0004151a30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7effec000c00 tx=0x7effec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.575+0000 7efffbe00640 1 -- 172.21.15.26:0/648896705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec03a9f0 con 0x7f00040cc500 2024-01-15T18:03:14.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.575+0000 7efffbe00640 1 -- 172.21.15.26:0/648896705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effec03ab90 con 0x7f00040cc500 2024-01-15T18:03:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.575+0000 7efffbe00640 1 -- 172.21.15.26:0/648896705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effec0443f0 con 0x7f00040cc500 2024-01-15T18:03:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.575+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0004157670 con 0x7f00040cc500 2024-01-15T18:03:14.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.575+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0004139020 con 0x7f00040cc500 2024-01-15T18:03:14.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.576+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00041571f0 con 0x7f00040cc500 2024-01-15T18:03:14.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.577+0000 7efffbe00640 1 -- 172.21.15.26:0/648896705 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7effec02f050 con 0x7f00040cc500 2024-01-15T18:03:14.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.578+0000 7efffbe00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effdc06bbc0 0x7effdc06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:14.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.578+0000 7efffbe00640 1 -- 172.21.15.26:0/648896705 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7effec0aa440 con 0x7f00040cc500 2024-01-15T18:03:14.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.579+0000 7f0003e00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effdc06bbc0 0x7effdc06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:14.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.580+0000 7f0003e00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effdc06bbc0 0x7effdc06e080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efff4007fd0 tx=0x7efff4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:14.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.582+0000 7efffbe00640 1 -- 172.21.15.26:0/648896705 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00041571f0 con 0x7f00040cc500 2024-01-15T18:03:14.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.816+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0004157b90 con 0x7f00040cc500 2024-01-15T18:03:14.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.818+0000 7efffbe00640 1 -- 172.21.15.26:0/648896705 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f0004157b90 con 0x7f00040cc500 2024-01-15T18:03:14.822 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:14.822 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1159022909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1159022909},{"type":"v1","addr":"172.21.15.26:6835","nonce":1159022909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/248571798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":248571798},{"type":"v1","addr":"172.21.15.124:6837","nonce":248571798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3357264964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3357264964},{"type":"v1","addr":"172.21.15.26:6833","nonce":3357264964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3611655863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3611655863},{"type":"v1","addr":"172.21.15.124:6836","nonce":3611655863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[]} 2024-01-15T18:03:14.822 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 38 2024-01-15T18:03:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.820+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effdc06bbc0 msgr2=0x7effdc06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.820+0000 7f0009a00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effdc06bbc0 0x7effdc06e080 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7efff4007fd0 tx=0x7efff4009040 comp rx=0 tx=0).stop 2024-01-15T18:03:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00040cc500 msgr2=0x7f0004151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00040cc500 0x7f0004151a30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7effec000c00 tx=0x7effec0046c0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 shutdown_connections 2024-01-15T18:03:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0004143c40 0x7f0004157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7effdc06bbc0 0x7effdc06e080 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00040cc500 0x7f0004151a30 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 --2- 172.21.15.26:0/648896705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f000406d160 0x7f0004139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 >> 172.21.15.26:0/648896705 conn(0x7f000406a800 msgr2=0x7f000406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 shutdown_connections 2024-01-15T18:03:14.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:14.821+0000 7f0009a00640 1 -- 172.21.15.26:0/648896705 wait complete. 2024-01-15T18:03:14.844 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:03:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.063+0000 7f9a15c00640 1 Processor -- start 2024-01-15T18:03:15.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.063+0000 7f9a15c00640 1 -- start start 2024-01-15T18:03:15.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.065+0000 7f9a15c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a1015d580 0x7f9a100cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.065+0000 7f9a15c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9a1015d960 0x7f9a100cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.065+0000 7f9a15c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a100ce760 con 0x7f9a100b20d0 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.065+0000 7f9a15c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a100d0a00 con 0x7f9a1015d580 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.065+0000 7f9a15c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a100cf250 con 0x7f9a1015d960 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.065+0000 7f9a0fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a1015d580 0x7f9a100cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.065+0000 7f9a0fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9a1015d580 0x7f9a100cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35008/0 (socket says 172.21.15.26:35008) 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.065+0000 7f9a0fe00640 1 -- 172.21.15.26:0/2451739122 learned_addr learned my addr 172.21.15.26:0/2451739122 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.065+0000 7f9a0f400640 1 --2- 172.21.15.26:0/2451739122 >> v2:172.21.15.124:3300/0 conn(0x7f9a1015d960 0x7f9a100cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.066+0000 7f9a0fe00640 1 -- 172.21.15.26:0/2451739122 >> v2:172.21.15.124:3300/0 conn(0x7f9a1015d960 msgr2=0x7f9a100cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:15.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.066+0000 7f9a0fe00640 1 --2- 172.21.15.26:0/2451739122 >> v2:172.21.15.124:3300/0 conn(0x7f9a1015d960 0x7f9a100cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.066+0000 7f9a0fe00640 1 -- 172.21.15.26:0/2451739122 >> v1:172.21.15.26:6789/0 conn(0x7f9a100b20d0 legacy=0x7f9a100d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.066+0000 7f9a0fe00640 1 -- 172.21.15.26:0/2451739122 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a100ce9b0 con 0x7f9a1015d580 2024-01-15T18:03:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.067+0000 7f9a0fe00640 1 --2- 172.21.15.26:0/2451739122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1015d580 0x7f9a100cf650 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9a100c6b40 tx=0x7f9a0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6a9e938ff9ffcc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:15.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.068+0000 7f9a0ea00640 1 -- 172.21.15.26:0/2451739122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00031870 con 0x7f9a1015d580 2024-01-15T18:03:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.068+0000 7f9a0ea00640 1 -- 172.21.15.26:0/2451739122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a000319d0 con 0x7f9a1015d580 2024-01-15T18:03:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.068+0000 7f9a0ea00640 1 -- 172.21.15.26:0/2451739122 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00031d60 con 0x7f9a1015d580 2024-01-15T18:03:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.068+0000 7f9a15c00640 1 -- 172.21.15.26:0/2451739122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1015d580 msgr2=0x7f9a100cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.068+0000 7f9a15c00640 1 --2- 172.21.15.26:0/2451739122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1015d580 0x7f9a100cf650 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f9a100c6b40 tx=0x7f9a0002b910 comp rx=0 tx=0).stop 2024-01-15T18:03:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.069+0000 7f9a15c00640 1 -- 172.21.15.26:0/2451739122 shutdown_connections 2024-01-15T18:03:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.069+0000 7f9a15c00640 1 --2- 172.21.15.26:0/2451739122 >> v2:172.21.15.124:3300/0 conn(0x7f9a1015d960 0x7f9a100cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.069+0000 7f9a15c00640 1 --2- 172.21.15.26:0/2451739122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a1015d580 0x7f9a100cf650 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.069+0000 7f9a15c00640 1 -- 172.21.15.26:0/2451739122 >> 172.21.15.26:0/2451739122 conn(0x7f9a100d3040 msgr2=0x7f9a100d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.069+0000 7f9a15c00640 1 -- 172.21.15.26:0/2451739122 shutdown_connections 2024-01-15T18:03:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.069+0000 7f9a15c00640 1 -- 172.21.15.26:0/2451739122 wait complete. 2024-01-15T18:03:15.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.070+0000 7f9a15c00640 1 Processor -- start 2024-01-15T18:03:15.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.071+0000 7f9a15c00640 1 -- start start 2024-01-15T18:03:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.071+0000 7f9a15c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a100b20d0 0x7f9a1014b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.071+0000 7f9a15c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a1015d580 0x7f9a100ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a15c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a1015d960 0x7f9a1015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a15c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a100cabd0 con 0x7f9a100b20d0 2024-01-15T18:03:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a15c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a100caeb0 con 0x7f9a1015d580 2024-01-15T18:03:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a15c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a100cb190 con 0x7f9a1015d960 2024-01-15T18:03:15.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a100b20d0 0x7f9a1014b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a100b20d0 0x7f9a1014b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35012/0 (socket says 172.21.15.26:35012) 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a0fe00640 1 -- 172.21.15.26:0/1944158474 learned_addr learned my addr 172.21.15.26:0/1944158474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a0fe00640 1 -- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a1015d960 msgr2=0x7f9a1015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a15200640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a1015d960 0x7f9a1015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a0f400640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a1015d580 0x7f9a100ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a0fe00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a1015d960 0x7f9a1015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a0fe00640 1 -- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a1015d580 msgr2=0x7f9a100ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a0fe00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a1015d580 0x7f9a100ca110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a0fe00640 1 -- 172.21.15.26:0/1944158474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000094d0 con 0x7f9a100b20d0 2024-01-15T18:03:15.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.072+0000 7f9a15200640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a1015d960 0x7f9a1015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.073+0000 7f9a0fe00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a100b20d0 0x7f9a1014b330 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9a00000c00 tx=0x7f9a000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.073+0000 7f9a07e00640 1 -- 172.21.15.26:0/1944158474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0003ad30 con 0x7f9a100b20d0 2024-01-15T18:03:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.073+0000 7f9a07e00640 1 -- 172.21.15.26:0/1944158474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0003ae90 con 0x7f9a100b20d0 2024-01-15T18:03:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.073+0000 7f9a07e00640 1 -- 172.21.15.26:0/1944158474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00044630 con 0x7f9a100b20d0 2024-01-15T18:03:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.073+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a100cb470 con 0x7f9a100b20d0 2024-01-15T18:03:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.073+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a100b7850 con 0x7f9a100b20d0 2024-01-15T18:03:15.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.074+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a10067450 con 0x7f9a100b20d0 2024-01-15T18:03:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.076+0000 7f9a07e00640 1 -- 172.21.15.26:0/1944158474 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f9a0004d450 con 0x7f9a100b20d0 2024-01-15T18:03:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.077+0000 7f9a07e00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99e806bbf0 0x7f99e806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.077+0000 7f9a07e00640 1 -- 172.21.15.26:0/1944158474 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f9a0002d040 con 0x7f9a100b20d0 2024-01-15T18:03:15.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.078+0000 7f9a0f400640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99e806bbf0 0x7f99e806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.079+0000 7f9a0f400640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99e806bbf0 0x7f99e806e0b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9a101611c0 tx=0x7f99f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:15.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.082+0000 7f9a07e00640 1 -- 172.21.15.26:0/1944158474 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a00077360 con 0x7f9a100b20d0 2024-01-15T18:03:15.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.281+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9a100cb990 con 0x7f9a100b20d0 2024-01-15T18:03:15.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.289+0000 7f9a07e00640 1 -- 172.21.15.26:0/1944158474 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9a0007a980 con 0x7f9a100b20d0 2024-01-15T18:03:15.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99e806bbf0 msgr2=0x7f99e806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99e806bbf0 0x7f99e806e0b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9a101611c0 tx=0x7f99f8008040 comp rx=0 tx=0).stop 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a100b20d0 msgr2=0x7f9a1014b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a100b20d0 0x7f9a1014b330 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f9a00000c00 tx=0x7f9a000046c0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 shutdown_connections 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9a1015d960 0x7f9a1015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f99e806bbf0 0x7f99e806e0b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9a1015d580 0x7f9a100ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 --2- 172.21.15.26:0/1944158474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9a100b20d0 0x7f9a1014b330 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 >> 172.21.15.26:0/1944158474 conn(0x7f9a100d3040 msgr2=0x7f9a100cdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:15.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 shutdown_connections 2024-01-15T18:03:15.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.292+0000 7f9a15c00640 1 -- 172.21.15.26:0/1944158474 wait complete. 2024-01-15T18:03:15.316 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_delay 2024-01-15T18:03:15.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.529+0000 7ff84e800640 1 Processor -- start 2024-01-15T18:03:15.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.529+0000 7ff84e800640 1 -- start start 2024-01-15T18:03:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.530+0000 7ff84e800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff84813e360 0x7ff8480cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.530+0000 7ff84e800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff84815f9c0 0x7ff8480cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.530+0000 7ff84e800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff84815c5f0 con 0x7ff84813df80 2024-01-15T18:03:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.530+0000 7ff84e800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff84806c470 con 0x7ff84815f9c0 2024-01-15T18:03:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.530+0000 7ff84e800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff84806c750 con 0x7ff84813e360 2024-01-15T18:03:15.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.530+0000 7ff847e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff84815f9c0 0x7ff8480cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.531+0000 7ff847e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff84815f9c0 0x7ff8480cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35026/0 (socket says 172.21.15.26:35026) 2024-01-15T18:03:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.531+0000 7ff847e00640 1 -- 172.21.15.26:0/4123622361 learned_addr learned my addr 172.21.15.26:0/4123622361 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.531+0000 7ff847400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff84813df80 0x7ff84815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46168/0 (socket says 172.21.15.26:46168) 2024-01-15T18:03:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.531+0000 7ff847e00640 1 -- 172.21.15.26:0/4123622361 >> v2:172.21.15.124:3300/0 conn(0x7ff84813e360 msgr2=0x7ff8480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.531+0000 7ff83fe00640 1 --2- 172.21.15.26:0/4123622361 >> v2:172.21.15.124:3300/0 conn(0x7ff84813e360 0x7ff8480cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.531+0000 7ff847e00640 1 --2- 172.21.15.26:0/4123622361 >> v2:172.21.15.124:3300/0 conn(0x7ff84813e360 0x7ff8480cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.531+0000 7ff847e00640 1 -- 172.21.15.26:0/4123622361 >> v1:172.21.15.26:6789/0 conn(0x7ff84813df80 legacy=0x7ff84815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.531+0000 7ff847e00640 1 -- 172.21.15.26:0/4123622361 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff84806ca30 con 0x7ff84815f9c0 2024-01-15T18:03:15.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.531+0000 7ff83fe00640 1 --2- 172.21.15.26:0/4123622361 >> v2:172.21.15.124:3300/0 conn(0x7ff84813e360 0x7ff8480cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:15.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.532+0000 7ff847e00640 1 --2- 172.21.15.26:0/4123622361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 0x7ff8480cf7d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff84806ded0 tx=0x7ff84002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ff16602631458a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:15.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.532+0000 7ff846a00640 1 -- 172.21.15.26:0/4123622361 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84002cb20 con 0x7ff84815f9c0 2024-01-15T18:03:15.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff846a00640 1 -- 172.21.15.26:0/4123622361 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84002cc80 con 0x7ff84815f9c0 2024-01-15T18:03:15.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff846a00640 1 -- 172.21.15.26:0/4123622361 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8400343f0 con 0x7ff84815f9c0 2024-01-15T18:03:15.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff84e800640 1 -- 172.21.15.26:0/4123622361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 msgr2=0x7ff8480cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff84e800640 1 --2- 172.21.15.26:0/4123622361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 0x7ff8480cf7d0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff84806ded0 tx=0x7ff84002bf80 comp rx=0 tx=0).stop 2024-01-15T18:03:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff84e800640 1 -- 172.21.15.26:0/4123622361 shutdown_connections 2024-01-15T18:03:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff84e800640 1 --2- 172.21.15.26:0/4123622361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 0x7ff8480cf7d0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff84e800640 1 --2- 172.21.15.26:0/4123622361 >> v2:172.21.15.124:3300/0 conn(0x7ff84813e360 0x7ff8480cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff84e800640 1 -- 172.21.15.26:0/4123622361 >> 172.21.15.26:0/4123622361 conn(0x7ff84806a800 msgr2=0x7ff84806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff84e800640 1 -- 172.21.15.26:0/4123622361 shutdown_connections 2024-01-15T18:03:15.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.533+0000 7ff84e800640 1 -- 172.21.15.26:0/4123622361 wait complete. 2024-01-15T18:03:15.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.535+0000 7ff84e800640 1 Processor -- start 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.535+0000 7ff84e800640 1 -- start start 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.536+0000 7ff84e800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff84813df80 0x7ff84813cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.536+0000 7ff84e800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff84813e360 0x7ff8480b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.536+0000 7ff84e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 0x7ff8480aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.536+0000 7ff84e800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8481f3ca0 con 0x7ff84815f9c0 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.536+0000 7ff84e800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8480a97b0 con 0x7ff84813df80 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.536+0000 7ff84e800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8480a84c0 con 0x7ff84813e360 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 0x7ff8480aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 0x7ff8480aa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35034/0 (socket says 172.21.15.26:35034) 2024-01-15T18:03:15.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847e00640 1 -- 172.21.15.26:0/3871301117 learned_addr learned my addr 172.21.15.26:0/3871301117 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847400640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff84813df80 0x7ff84813cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff83fe00640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff84813e360 0x7ff8480b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847e00640 1 -- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff84813e360 msgr2=0x7ff8480b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847e00640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff84813e360 0x7ff8480b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847e00640 1 -- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff84813df80 msgr2=0x7ff84813cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847e00640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff84813df80 0x7ff84813cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847e00640 1 -- 172.21.15.26:0/3871301117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff840009b30 con 0x7ff84815f9c0 2024-01-15T18:03:15.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847400640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff84813df80 0x7ff84813cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:15.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.537+0000 7ff847e00640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 0x7ff8480aa310 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff840036040 tx=0x7ff840004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:15.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.538+0000 7ff83f400640 1 -- 172.21.15.26:0/3871301117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840007e70 con 0x7ff84815f9c0 2024-01-15T18:03:15.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.538+0000 7ff83f400640 1 -- 172.21.15.26:0/3871301117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff840009510 con 0x7ff84815f9c0 2024-01-15T18:03:15.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.538+0000 7ff83f400640 1 -- 172.21.15.26:0/3871301117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84003e690 con 0x7ff84815f9c0 2024-01-15T18:03:15.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.538+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8480a9320 con 0x7ff84815f9c0 2024-01-15T18:03:15.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.538+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84813d0b0 con 0x7ff84815f9c0 2024-01-15T18:03:15.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.539+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8480a94b0 con 0x7ff84815f9c0 2024-01-15T18:03:15.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.540+0000 7ff83f400640 1 -- 172.21.15.26:0/3871301117 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7ff84003e7f0 con 0x7ff84815f9c0 2024-01-15T18:03:15.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.541+0000 7ff83f400640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff82006bbf0 0x7ff82006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:15.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.541+0000 7ff83f400640 1 -- 172.21.15.26:0/3871301117 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7ff8400ab0b0 con 0x7ff84815f9c0 2024-01-15T18:03:15.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.541+0000 7ff847400640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff82006bbf0 0x7ff82006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:15.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.543+0000 7ff847400640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff82006bbf0 0x7ff82006e0b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff838000f60 tx=0x7ff838009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:15.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.546+0000 7ff83f400640 1 -- 172.21.15.26:0/3871301117 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff840077970 con 0x7ff84815f9c0 2024-01-15T18:03:15.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.745+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"} v 0) v1 -- 0x7ff8480675d0 con 0x7ff84815f9c0 2024-01-15T18:03:15.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.751+0000 7ff83f400640 1 -- 172.21.15.26:0/3871301117 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"}]=0 v24) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff84007af90 con 0x7ff84815f9c0 2024-01-15T18:03:15.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff82006bbf0 msgr2=0x7ff82006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff82006bbf0 0x7ff82006e0b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff838000f60 tx=0x7ff838009040 comp rx=0 tx=0).stop 2024-01-15T18:03:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 msgr2=0x7ff8480aa310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 0x7ff8480aa310 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff840036040 tx=0x7ff840004a40 comp rx=0 tx=0).stop 2024-01-15T18:03:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 shutdown_connections 2024-01-15T18:03:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff84813e360 0x7ff8480b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff82006bbf0 0x7ff82006e0b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff84815f9c0 0x7ff8480aa310 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 --2- 172.21.15.26:0/3871301117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff84813df80 0x7ff84813cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 >> 172.21.15.26:0/3871301117 conn(0x7ff84806a800 msgr2=0x7ff8480cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.754+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 shutdown_connections 2024-01-15T18:03:15.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.755+0000 7ff84e800640 1 -- 172.21.15.26:0/3871301117 wait complete. 2024-01-15T18:03:15.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:03:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.998+0000 7f5c53400640 1 Processor -- start 2024-01-15T18:03:16.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:15.999+0000 7f5c53400640 1 -- start start 2024-01-15T18:03:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.000+0000 7f5c53400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5c4c13df80 0x7f5c4c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.000+0000 7f5c53400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13e360 0x7f5c4c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.000+0000 7f5c53400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c06c470 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.000+0000 7f5c53400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4c06c750 con 0x7f5c4c13df80 2024-01-15T18:03:16.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.000+0000 7f5c53400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c06ca30 con 0x7f5c4c13e360 2024-01-15T18:03:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.001+0000 7f5c4be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13e360 0x7f5c4c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.001+0000 7f5c50a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5c4c13df80 0x7f5c4c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.001+0000 7f5c4be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13e360 0x7f5c4c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:47330/0 (socket says 172.21.15.26:47330) 2024-01-15T18:03:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.001+0000 7f5c4be00640 1 -- 172.21.15.26:0/2774233970 learned_addr learned my addr 172.21.15.26:0/2774233970 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.001+0000 7f5c50a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5c4c13df80 0x7f5c4c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:59928/0 (socket says 172.21.15.26:59928) 2024-01-15T18:03:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.002+0000 7f5c4be00640 1 -- 172.21.15.26:0/2774233970 >> v2:172.21.15.124:3300/0 conn(0x7f5c4c13df80 msgr2=0x7f5c4c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:16.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.002+0000 7f5c4be00640 1 --2- 172.21.15.26:0/2774233970 >> v2:172.21.15.124:3300/0 conn(0x7f5c4c13df80 0x7f5c4c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.002+0000 7f5c4be00640 1 -- 172.21.15.26:0/2774233970 >> v1:172.21.15.124:6789/0 conn(0x7f5c4c15f9c0 legacy=0x7f5c4c1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.002+0000 7f5c4be00640 1 -- 172.21.15.26:0/2774233970 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c06cd10 con 0x7f5c4c13e360 2024-01-15T18:03:16.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.003+0000 7f5c4be00640 1 --2- 172.21.15.26:0/2774233970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13e360 0x7f5c4c0cf290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009870 tx=0x7f5c3402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8c578c055e1e1ce server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.004+0000 7f5c4b400640 1 -- 172.21.15.26:0/2774233970 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34030840 con 0x7f5c4c13e360 2024-01-15T18:03:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.004+0000 7f5c4b400640 1 -- 172.21.15.26:0/2774233970 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c340309a0 con 0x7f5c4c13e360 2024-01-15T18:03:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.004+0000 7f5c4b400640 1 -- 172.21.15.26:0/2774233970 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34030d30 con 0x7f5c4c13e360 2024-01-15T18:03:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.004+0000 7f5c53400640 1 -- 172.21.15.26:0/2774233970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13e360 msgr2=0x7f5c4c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.004+0000 7f5c53400640 1 --2- 172.21.15.26:0/2774233970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13e360 0x7f5c4c0cf290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009870 tx=0x7f5c3402b980 comp rx=0 tx=0).stop 2024-01-15T18:03:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.005+0000 7f5c53400640 1 -- 172.21.15.26:0/2774233970 shutdown_connections 2024-01-15T18:03:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.005+0000 7f5c53400640 1 --2- 172.21.15.26:0/2774233970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13e360 0x7f5c4c0cf290 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.005+0000 7f5c53400640 1 --2- 172.21.15.26:0/2774233970 >> v2:172.21.15.124:3300/0 conn(0x7f5c4c13df80 0x7f5c4c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.005+0000 7f5c53400640 1 -- 172.21.15.26:0/2774233970 >> 172.21.15.26:0/2774233970 conn(0x7f5c4c06a800 msgr2=0x7f5c4c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.005+0000 7f5c53400640 1 -- 172.21.15.26:0/2774233970 shutdown_connections 2024-01-15T18:03:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.005+0000 7f5c53400640 1 -- 172.21.15.26:0/2774233970 wait complete. 2024-01-15T18:03:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.007+0000 7f5c53400640 1 Processor -- start 2024-01-15T18:03:16.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.007+0000 7f5c53400640 1 -- start start 2024-01-15T18:03:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.008+0000 7f5c53400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13df80 0x7f5c4c135790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.008+0000 7f5c53400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c4c13e360 0x7f5c4c071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.008+0000 7f5c53400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c4c15f9c0 0x7f5c4c0adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.008+0000 7f5c53400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c135cd0 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.008+0000 7f5c53400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c071c80 con 0x7f5c4c13e360 2024-01-15T18:03:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.008+0000 7f5c53400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0706a0 con 0x7f5c4c13df80 2024-01-15T18:03:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c51400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c4c15f9c0 0x7f5c4c0adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c51400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c4c15f9c0 0x7f5c4c0adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35050/0 (socket says 172.21.15.26:35050) 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c51400640 1 -- 172.21.15.26:0/2652240016 learned_addr learned my addr 172.21.15.26:0/2652240016 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c4be00640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c4c13e360 0x7f5c4c071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c50a00640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13df80 0x7f5c4c135790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c51400640 1 -- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13df80 msgr2=0x7f5c4c135790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c51400640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13df80 0x7f5c4c135790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c51400640 1 -- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c4c13e360 msgr2=0x7f5c4c071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c51400640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c4c13e360 0x7f5c4c071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c51400640 1 -- 172.21.15.26:0/2652240016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c340094d0 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.009+0000 7f5c50a00640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13df80 0x7f5c4c135790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:03:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.010+0000 7f5c51400640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c4c15f9c0 0x7f5c4c0adb40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c7a0 tx=0x7f5c4400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:16.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.010+0000 7f5c48c00640 1 -- 172.21.15.26:0/2652240016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44003b70 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.010+0000 7f5c48c00640 1 -- 172.21.15.26:0/2652240016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c44003cf0 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.010+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c070980 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.010+0000 7f5c48c00640 1 -- 172.21.15.26:0/2652240016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44003ea0 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.010+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c070f40 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.012+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c0675d0 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.013+0000 7f5c48c00640 1 -- 172.21.15.26:0/2652240016 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f5c4400aa90 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.013+0000 7f5c48c00640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c2406bbc0 0x7f5c2406e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.014+0000 7f5c48c00640 1 -- 172.21.15.26:0/2652240016 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f5c440882b0 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.014+0000 7f5c50a00640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c2406bbc0 0x7f5c2406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.015+0000 7f5c50a00640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c2406bbc0 0x7f5c2406e080 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009af0 tx=0x7f5c3c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.019+0000 7f5c48c00640 1 -- 172.21.15.26:0/2652240016 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c44054bd0 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.216+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f5c4c0b43c0 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.223+0000 7f5c48c00640 1 -- 172.21.15.26:0/2652240016 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v25) v1 ==== 120+0+0 (secure 0 0 0) 0x7f5c440581f0 con 0x7f5c4c15f9c0 2024-01-15T18:03:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c2406bbc0 msgr2=0x7f5c2406e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c2406bbc0 0x7f5c2406e080 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009af0 tx=0x7f5c3c008040 comp rx=0 tx=0).stop 2024-01-15T18:03:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c4c15f9c0 msgr2=0x7f5c4c0adb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c4c15f9c0 0x7f5c4c0adb40 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400c7a0 tx=0x7f5c4400cc60 comp rx=0 tx=0).stop 2024-01-15T18:03:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 shutdown_connections 2024-01-15T18:03:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c2406bbc0 0x7f5c2406e080 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c4c15f9c0 0x7f5c4c0adb40 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c4c13e360 0x7f5c4c071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 --2- 172.21.15.26:0/2652240016 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c4c13df80 0x7f5c4c135790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 >> 172.21.15.26:0/2652240016 conn(0x7f5c4c06a800 msgr2=0x7f5c4c0b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 shutdown_connections 2024-01-15T18:03:16.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.226+0000 7f5c53400640 1 -- 172.21.15.26:0/2652240016 wait complete. 2024-01-15T18:03:16.248 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones' 2024-01-15T18:03:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.464+0000 7f89fda00640 1 Processor -- start 2024-01-15T18:03:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.465+0000 7f89fda00640 1 -- start start 2024-01-15T18:03:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.466+0000 7f89fda00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f80b3eb0 0x7f89f80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.466+0000 7f89fda00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89f8160b80 0x7f89f81617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.466+0000 7f89fda00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f80b2b90 con 0x7f89f81607a0 2024-01-15T18:03:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.466+0000 7f89fda00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f80d21d0 con 0x7f89f8160b80 2024-01-15T18:03:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.466+0000 7f89fda00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80b2e30 con 0x7f89f80b3eb0 2024-01-15T18:03:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.467+0000 7f89fd000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89f8160b80 0x7f89f81617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.467+0000 7f89fd000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89f8160b80 0x7f89f81617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:59940/0 (socket says 172.21.15.26:59940) 2024-01-15T18:03:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.467+0000 7f89fd000640 1 -- 172.21.15.26:0/2261662496 learned_addr learned my addr 172.21.15.26:0/2261662496 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.467+0000 7f89f7400640 1 --2- 172.21.15.26:0/2261662496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f80b3eb0 0x7f89f80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.467+0000 7f89fd000640 1 -- 172.21.15.26:0/2261662496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f80b3eb0 msgr2=0x7f89f80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.467+0000 7f89fd000640 1 --2- 172.21.15.26:0/2261662496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f80b3eb0 0x7f89f80b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.467+0000 7f89fd000640 1 -- 172.21.15.26:0/2261662496 >> v1:172.21.15.124:6789/0 conn(0x7f89f81607a0 legacy=0x7f89f81610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.467+0000 7f89fd000640 1 -- 172.21.15.26:0/2261662496 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f80b3110 con 0x7f89f8160b80 2024-01-15T18:03:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.469+0000 7f89fd000640 1 --2- 172.21.15.26:0/2261662496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 0x7f89f81617d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009e50 tx=0x7f89f002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f987d95b2582790b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.469+0000 7f89f6a00640 1 -- 172.21.15.26:0/2261662496 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f002cb00 con 0x7f89f8160b80 2024-01-15T18:03:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.469+0000 7f89f6a00640 1 -- 172.21.15.26:0/2261662496 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f002cc60 con 0x7f89f8160b80 2024-01-15T18:03:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.469+0000 7f89f6a00640 1 -- 172.21.15.26:0/2261662496 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00343f0 con 0x7f89f8160b80 2024-01-15T18:03:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.470+0000 7f89fda00640 1 -- 172.21.15.26:0/2261662496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 msgr2=0x7f89f81617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.470+0000 7f89fda00640 1 --2- 172.21.15.26:0/2261662496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 0x7f89f81617d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009e50 tx=0x7f89f002bf60 comp rx=0 tx=0).stop 2024-01-15T18:03:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.470+0000 7f89fda00640 1 -- 172.21.15.26:0/2261662496 shutdown_connections 2024-01-15T18:03:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.470+0000 7f89fda00640 1 --2- 172.21.15.26:0/2261662496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f80b3eb0 0x7f89f80b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.470+0000 7f89fda00640 1 --2- 172.21.15.26:0/2261662496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 0x7f89f81617d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.470+0000 7f89fda00640 1 -- 172.21.15.26:0/2261662496 >> 172.21.15.26:0/2261662496 conn(0x7f89f809c590 msgr2=0x7f89f80d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.470+0000 7f89fda00640 1 -- 172.21.15.26:0/2261662496 shutdown_connections 2024-01-15T18:03:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.470+0000 7f89fda00640 1 -- 172.21.15.26:0/2261662496 wait complete. 2024-01-15T18:03:16.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.472+0000 7f89fda00640 1 Processor -- start 2024-01-15T18:03:16.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.472+0000 7f89fda00640 1 -- start start 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.473+0000 7f89fda00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f80b3eb0 0x7f89f80c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.473+0000 7f89fda00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f81607a0 0x7f89f81516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.473+0000 7f89fda00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 0x7f89f80b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.473+0000 7f89fda00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80c6d80 con 0x7f89f80b3eb0 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.473+0000 7f89fda00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f8151c10 con 0x7f89f8160b80 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.473+0000 7f89fda00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f8154d40 con 0x7f89f81607a0 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f80b3eb0 0x7f89f80c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f80b3eb0 0x7f89f80c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35062/0 (socket says 172.21.15.26:35062) 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7e00640 1 -- 172.21.15.26:0/1721647200 learned_addr learned my addr 172.21.15.26:0/1721647200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89fd000640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 0x7f89f80b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7400640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f81607a0 0x7f89f81516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7e00640 1 -- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f81607a0 msgr2=0x7f89f81516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7e00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f81607a0 0x7f89f81516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7e00640 1 -- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 msgr2=0x7f89f80b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7e00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 0x7f89f80b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7e00640 1 -- 172.21.15.26:0/1721647200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0009b30 con 0x7f89f80b3eb0 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89fd000640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 0x7f89f80b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.474+0000 7f89f7e00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f80b3eb0 0x7f89f80c8070 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f89e800c400 tx=0x7f89e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.475+0000 7f89efe00640 1 -- 172.21.15.26:0/1721647200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80052d0 con 0x7f89f80b3eb0 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.475+0000 7f89efe00640 1 -- 172.21.15.26:0/1721647200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8005450 con 0x7f89f80b3eb0 2024-01-15T18:03:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.475+0000 7f89efe00640 1 -- 172.21.15.26:0/1721647200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8007260 con 0x7f89f80b3eb0 2024-01-15T18:03:16.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.475+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f80c6fd0 con 0x7f89f80b3eb0 2024-01-15T18:03:16.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.475+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f80c7590 con 0x7f89f80b3eb0 2024-01-15T18:03:16.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.476+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f8067450 con 0x7f89f80b3eb0 2024-01-15T18:03:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.477+0000 7f89efe00640 1 -- 172.21.15.26:0/1721647200 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f89e80073c0 con 0x7f89f80b3eb0 2024-01-15T18:03:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.478+0000 7f89efe00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f89d006bbc0 0x7f89d006e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.478+0000 7f89efe00640 1 -- 172.21.15.26:0/1721647200 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f89e80878e0 con 0x7f89f80b3eb0 2024-01-15T18:03:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.479+0000 7f89f7400640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f89d006bbc0 0x7f89d006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.480+0000 7f89f7400640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f89d006bbc0 0x7f89d006e080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f89f80b72f0 tx=0x7f89e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:16.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.482+0000 7f89efe00640 1 -- 172.21.15.26:0/1721647200 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89e808e070 con 0x7f89f80b3eb0 2024-01-15T18:03:16.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:16.749+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones"]} v 0) v1 -- 0x7f89f80c66d0 con 0x7f89f80b3eb0 2024-01-15T18:03:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.199+0000 7f89efe00640 1 -- 172.21.15.26:0/1721647200 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_bars_for_7_clones"]}]=0 v382) v1 ==== 154+0+0 (secure 0 0 0) 0x7f89e80541a0 con 0x7f89f80b3eb0 2024-01-15T18:03:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f89d006bbc0 msgr2=0x7f89d006e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f89d006bbc0 0x7f89d006e080 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f89f80b72f0 tx=0x7f89e0008040 comp rx=0 tx=0).stop 2024-01-15T18:03:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f80b3eb0 msgr2=0x7f89f80c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f80b3eb0 0x7f89f80c8070 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f89e800c400 tx=0x7f89e800c8c0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 shutdown_connections 2024-01-15T18:03:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89f81607a0 0x7f89f81516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f89d006bbc0 0x7f89d006e080 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89f8160b80 0x7f89f80b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 --2- 172.21.15.26:0/1721647200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89f80b3eb0 0x7f89f80c8070 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 >> 172.21.15.26:0/1721647200 conn(0x7f89f809c590 msgr2=0x7f89f80b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 shutdown_connections 2024-01-15T18:03:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.203+0000 7f89fda00640 1 -- 172.21.15.26:0/1721647200 wait complete. 2024-01-15T18:03:17.229 INFO:tasks.cephfs_test_runner:X = number of clones issued. ... ok 2024-01-15T18:03:17.230 INFO:tasks.cephfs_test_runner:Starting test: test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgress) X = number of clones issued. 2024-01-15T18:03:17.230 INFO:tasks.cephfs_test_runner:test_progress_drops_when_new_jobs_are_added (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:03:17.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:03:17.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.449+0000 7f0c62400640 1 Processor -- start 2024-01-15T18:03:17.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.449+0000 7f0c62400640 1 -- start start 2024-01-15T18:03:17.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.450+0000 7f0c62400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0c5c15d3c0 0x7f0c5c0d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.450+0000 7f0c62400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c0d0e90 0x7f0c5c0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.450+0000 7f0c62400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c0d12f0 con 0x7f0c5c0b01b0 2024-01-15T18:03:17.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.450+0000 7f0c62400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5c0ce5d0 con 0x7f0c5c15d3c0 2024-01-15T18:03:17.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.450+0000 7f0c62400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b23c0 con 0x7f0c5c0d0e90 2024-01-15T18:03:17.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.450+0000 7f0c61a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0c5c0b01b0 0x7f0c5c0ce9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46172/0 (socket says 172.21.15.26:46172) 2024-01-15T18:03:17.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.450+0000 7f0c61a00640 1 -- 172.21.15.26:0/4160304715 learned_addr learned my addr 172.21.15.26:0/4160304715 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c61000640 1 --2- 172.21.15.26:0/4160304715 >> v2:172.21.15.124:3300/0 conn(0x7f0c5c15d3c0 0x7f0c5c0d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c5be00640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c0d0e90 0x7f0c5c0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3791102724 0 0) 0x7f0c54001110 con 0x7f0c5c0b01b0 2024-01-15T18:03:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c3c003520 con 0x7f0c5c0b01b0 2024-01-15T18:03:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1025040902 0 0) 0x7f0c3c003520 con 0x7f0c5c0b01b0 2024-01-15T18:03:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c0d0e90 msgr2=0x7f0c5c0afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c5b400640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c0d0e90 0x7f0c5c0afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 >> v2:172.21.15.124:3300/0 conn(0x7f0c5c15d3c0 msgr2=0x7f0c5c0d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c5b400640 1 --2- 172.21.15.26:0/4160304715 >> v2:172.21.15.124:3300/0 conn(0x7f0c5c15d3c0 0x7f0c5c0d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.451+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0d0800 con 0x7f0c5c0b01b0 2024-01-15T18:03:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f0c54001800 con 0x7f0c5c0b01b0 2024-01-15T18:03:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 >> v1:172.21.15.26:6789/0 conn(0x7f0c5c0b01b0 legacy=0x7f0c5c0ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5b400640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c0d0e90 0x7f0c3c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5b400640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c3c003ef0 0x7f0c3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5b400640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c004d00 0x7f0c3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0ce5d0 con 0x7f0c3c003ef0 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b23c0 con 0x7f0c3c004d00 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c00b9d0 con 0x7f0c5c0d0e90 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5be00640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c3c003ef0 0x7f0c3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c61000640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c0d0e90 0x7f0c3c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c61a00640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c004d00 0x7f0c3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5be00640 1 -- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c0d0e90 msgr2=0x7f0c3c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5be00640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c0d0e90 0x7f0c3c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5be00640 1 -- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c004d00 msgr2=0x7f0c3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5be00640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c004d00 0x7f0c3c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c5be00640 1 -- 172.21.15.26:0/4160304715 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c003520 con 0x7f0c3c003ef0 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c61a00640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c004d00 0x7f0c3c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.452+0000 7f0c61000640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c0d0e90 0x7f0c3c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.454+0000 7f0c5be00640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c3c003ef0 0x7f0c3c0047c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400e170 tx=0x7f0c3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92c20cb9b8db0b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.454+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4402cbf0 con 0x7f0c3c003ef0 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.454+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4400f040 con 0x7f0c3c003ef0 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.454+0000 7f0c5b400640 1 -- 172.21.15.26:0/4160304715 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44034440 con 0x7f0c3c003ef0 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.454+0000 7f0c62400640 1 -- 172.21.15.26:0/4160304715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c3c003ef0 msgr2=0x7f0c3c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.454+0000 7f0c62400640 1 --2- 172.21.15.26:0/4160304715 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c3c003ef0 0x7f0c3c0047c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400e170 tx=0x7f0c3c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.454+0000 7f0c61a00640 1 -- 172.21.15.26:0/4160304715 reap_dead start 2024-01-15T18:03:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.455+0000 7f0c62400640 1 -- 172.21.15.26:0/4160304715 shutdown_connections 2024-01-15T18:03:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.455+0000 7f0c62400640 1 -- 172.21.15.26:0/4160304715 >> 172.21.15.26:0/4160304715 conn(0x7f0c5c0d30b0 msgr2=0x7f0c5c0d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.455+0000 7f0c62400640 1 -- 172.21.15.26:0/4160304715 shutdown_connections 2024-01-15T18:03:17.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.455+0000 7f0c62400640 1 -- 172.21.15.26:0/4160304715 wait complete. 2024-01-15T18:03:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.456+0000 7f0c62400640 1 Processor -- start 2024-01-15T18:03:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.457+0000 7f0c62400640 1 -- start start 2024-01-15T18:03:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c62400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c003ef0 0x7f0c5c1f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c62400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c1521e0 0x7f0c5c1526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c62400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c0b3fc0 0x7f0c5c0b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c62400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c151de0 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c62400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b4980 con 0x7f0c3c003ef0 2024-01-15T18:03:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c62400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b7f00 con 0x7f0c5c1521e0 2024-01-15T18:03:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c61a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c0b3fc0 0x7f0c5c0b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c61a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c0b3fc0 0x7f0c5c0b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35080/0 (socket says 172.21.15.26:35080) 2024-01-15T18:03:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c61a00640 1 -- 172.21.15.26:0/2830069121 learned_addr learned my addr 172.21.15.26:0/2830069121 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c5be00640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c1521e0 0x7f0c5c1526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c61000640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c003ef0 0x7f0c5c1f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c61a00640 1 -- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c1521e0 msgr2=0x7f0c5c1526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c61a00640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c1521e0 0x7f0c5c1526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c61a00640 1 -- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c003ef0 msgr2=0x7f0c5c1f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c61a00640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c003ef0 0x7f0c5c1f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.458+0000 7f0c61a00640 1 -- 172.21.15.26:0/2830069121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0b2ef0 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.459+0000 7f0c61a00640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c0b3fc0 0x7f0c5c0b4440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c06ada0 tx=0x7f0c5400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.459+0000 7f0c58c00640 1 -- 172.21.15.26:0/2830069121 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c540072d0 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.459+0000 7f0c58c00640 1 -- 172.21.15.26:0/2830069121 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54007430 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.459+0000 7f0c58c00640 1 -- 172.21.15.26:0/2830069121 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c540047e0 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.459+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c0b8150 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.459+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c0b3190 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.460+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c067b60 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.461+0000 7f0c58c00640 1 -- 172.21.15.26:0/2830069121 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f0c54004940 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.462+0000 7f0c58c00640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0c3406bbf0 0x7f0c3406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.462+0000 7f0c58c00640 1 -- 172.21.15.26:0/2830069121 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f0c5408a390 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.463+0000 7f0c61000640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0c3406bbf0 0x7f0c3406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.464+0000 7f0c61000640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0c3406bbf0 0x7f0c3406e0b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009b20 tx=0x7f0c4c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.466+0000 7f0c58c00640 1 -- 172.21.15.26:0/2830069121 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c54054930 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.666+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c5c0d2470 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.668+0000 7f0c58c00640 1 -- 172.21.15.26:0/2830069121 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+13051 (secure 0 0 0) 0x7f0c5c0d2470 con 0x7f0c5c0b3fc0 2024-01-15T18:03:17.672 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:17.672 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":63,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:03:10.120035+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6836/3403655022":"2024-01-16T18:03:07.106284+0000","172.21.15.124:6834/3403655022":"2024-01-16T18:03:07.106284+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:03:17.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0c3406bbf0 msgr2=0x7f0c3406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0c3406bbf0 0x7f0c3406e0b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009b20 tx=0x7f0c4c008040 comp rx=0 tx=0).stop 2024-01-15T18:03:17.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c0b3fc0 msgr2=0x7f0c5c0b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c0b3fc0 0x7f0c5c0b4440 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c06ada0 tx=0x7f0c5400dc00 comp rx=0 tx=0).stop 2024-01-15T18:03:17.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 shutdown_connections 2024-01-15T18:03:17.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c5c1521e0 0x7f0c5c1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0c3406bbf0 0x7f0c3406e0b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c5c0b3fc0 0x7f0c5c0b4440 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 --2- 172.21.15.26:0/2830069121 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c3c003ef0 0x7f0c5c1f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 >> 172.21.15.26:0/2830069121 conn(0x7f0c5c0d30b0 msgr2=0x7f0c5c0cff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:17.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 shutdown_connections 2024-01-15T18:03:17.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.671+0000 7f0c62400640 1 -- 172.21.15.26:0/2830069121 wait complete. 2024-01-15T18:03:17.694 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:03:17.695 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:03:17.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.915+0000 7f9ed7c00640 1 Processor -- start 2024-01-15T18:03:17.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.915+0000 7f9ed7c00640 1 -- start start 2024-01-15T18:03:17.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.916+0000 7f9ed7c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9ed013df80 0x7f9ed015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.917+0000 7f9ed7c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed013e360 0x7f9ed00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.917+0000 7f9ed7c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed006c470 con 0x7f9ed015f9c0 2024-01-15T18:03:17.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.917+0000 7f9ed7c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed006c750 con 0x7f9ed013df80 2024-01-15T18:03:17.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.917+0000 7f9ed7c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed006ca30 con 0x7f9ed013e360 2024-01-15T18:03:17.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.917+0000 7f9ed5c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9ed015f9c0 0x7f9ed01408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46186/0 (socket says 172.21.15.26:46186) 2024-01-15T18:03:17.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.917+0000 7f9ed5c00640 1 -- 172.21.15.26:0/2667435324 learned_addr learned my addr 172.21.15.26:0/2667435324 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ed5200640 1 --2- 172.21.15.26:0/2667435324 >> v2:172.21.15.124:3300/0 conn(0x7f9ed013df80 0x7f9ed015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ec7c00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed013e360 0x7f9ed00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099744020 0 0) 0x7f9ed006c470 con 0x7f9ed015f9c0 2024-01-15T18:03:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb0003520 con 0x7f9ed015f9c0 2024-01-15T18:03:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874375550 0 0) 0x7f9eb0003520 con 0x7f9ed015f9c0 2024-01-15T18:03:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed013e360 msgr2=0x7f9ed00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed013e360 0x7f9ed00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 >> v2:172.21.15.124:3300/0 conn(0x7f9ed013df80 msgr2=0x7f9ed015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 --2- 172.21.15.26:0/2667435324 >> v2:172.21.15.124:3300/0 conn(0x7f9ed013df80 0x7f9ed015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed006cd10 con 0x7f9ed015f9c0 2024-01-15T18:03:17.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9ec80019f0 con 0x7f9ed015f9c0 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 >> v1:172.21.15.26:6789/0 conn(0x7f9ed015f9c0 legacy=0x7f9ed01408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0003970 0x7f9eb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed013e360 0x7f9eb00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eb0004d00 0x7f9eb0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed006c750 con 0x7f9eb0004d00 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed006ca30 con 0x7f9eb0003970 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.918+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb000b980 con 0x7f9ed013e360 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.919+0000 7f9ed5c00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eb0004d00 0x7f9eb0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.919+0000 7f9ec7c00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed013e360 0x7f9eb00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.919+0000 7f9ed5c00640 1 -- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed013e360 msgr2=0x7f9eb00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.919+0000 7f9ed5c00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed013e360 0x7f9eb00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.919+0000 7f9ed5c00640 1 -- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0003970 msgr2=0x7f9eb0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.919+0000 7f9ed5200640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0003970 0x7f9eb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.919+0000 7f9ed5c00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0003970 0x7f9eb0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.919+0000 7f9ed5c00640 1 -- 172.21.15.26:0/2667435324 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb0003520 con 0x7f9eb0004d00 2024-01-15T18:03:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.919+0000 7f9ed5200640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0003970 0x7f9eb0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.920+0000 7f9ed5c00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eb0004d00 0x7f9eb0005180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800a890 tx=0x7f9eb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb17e240cdb27d7c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:17.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8034630 con 0x7f9eb0004d00 2024-01-15T18:03:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec802f070 con 0x7f9eb0004d00 2024-01-15T18:03:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ecfe00640 1 -- 172.21.15.26:0/2667435324 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80349f0 con 0x7f9eb0004d00 2024-01-15T18:03:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ed7c00640 1 -- 172.21.15.26:0/2667435324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eb0004d00 msgr2=0x7f9eb0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ed7c00640 1 --2- 172.21.15.26:0/2667435324 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9eb0004d00 0x7f9eb0005180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800a890 tx=0x7f9eb00057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ed5c00640 1 -- 172.21.15.26:0/2667435324 reap_dead start 2024-01-15T18:03:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ed7c00640 1 -- 172.21.15.26:0/2667435324 shutdown_connections 2024-01-15T18:03:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ed7c00640 1 -- 172.21.15.26:0/2667435324 >> 172.21.15.26:0/2667435324 conn(0x7f9ed006a800 msgr2=0x7f9ed006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ed7c00640 1 -- 172.21.15.26:0/2667435324 shutdown_connections 2024-01-15T18:03:17.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.921+0000 7f9ed7c00640 1 -- 172.21.15.26:0/2667435324 wait complete. 2024-01-15T18:03:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.923+0000 7f9ed7c00640 1 Processor -- start 2024-01-15T18:03:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.924+0000 7f9ed7c00640 1 -- start start 2024-01-15T18:03:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ed7c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0004d00 0x7f9ed014dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ed7c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed01541e0 0x7f9ed013a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ed7c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed0151400 0x7f9ed014f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ed7c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0150040 con 0x7f9ed01541e0 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ed7c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed014d640 con 0x7f9eb0004d00 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ed7c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed014ca30 con 0x7f9ed0151400 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ec7c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed01541e0 0x7f9ed013a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ec7c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed01541e0 0x7f9ed013a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35104/0 (socket says 172.21.15.26:35104) 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ec7c00640 1 -- 172.21.15.26:0/3613516874 learned_addr learned my addr 172.21.15.26:0/3613516874 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ed5200640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0004d00 0x7f9ed014dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ed5c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed0151400 0x7f9ed014f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ec7c00640 1 -- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed0151400 msgr2=0x7f9ed014f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ec7c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed0151400 0x7f9ed014f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ec7c00640 1 -- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0004d00 msgr2=0x7f9ed014dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ec7c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0004d00 0x7f9ed014dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.925+0000 7f9ec7c00640 1 -- 172.21.15.26:0/3613516874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed014cd10 con 0x7f9ed01541e0 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.926+0000 7f9ec7c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed01541e0 0x7f9ed013a1b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8010860 tx=0x7f9eb8010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.926+0000 7f9ecd600640 1 -- 172.21.15.26:0/3613516874 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8009610 con 0x7f9ed01541e0 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.926+0000 7f9ecd600640 1 -- 172.21.15.26:0/3613516874 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb800eb60 con 0x7f9ed01541e0 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.926+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed014cff0 con 0x7f9ed01541e0 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.926+0000 7f9ecd600640 1 -- 172.21.15.26:0/3613516874 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb801c400 con 0x7f9ed01541e0 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.926+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed014f080 con 0x7f9ed01541e0 2024-01-15T18:03:17.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.927+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed014d7f0 con 0x7f9ed01541e0 2024-01-15T18:03:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.928+0000 7f9ecd600640 1 -- 172.21.15.26:0/3613516874 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f9eb801c560 con 0x7f9ed01541e0 2024-01-15T18:03:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.929+0000 7f9ecd600640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806bbc0 0x7f9ea806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.929+0000 7f9ecd600640 1 -- 172.21.15.26:0/3613516874 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f9eb8088a10 con 0x7f9ed01541e0 2024-01-15T18:03:17.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.930+0000 7f9ed5200640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806bbc0 0x7f9ea806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.931+0000 7f9ed5200640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806bbc0 0x7f9ea806e080 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009b20 tx=0x7f9ec0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:17.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:17.934+0000 7f9ecd600640 1 -- 172.21.15.26:0/3613516874 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ed014d7f0 con 0x7f9ed01541e0 2024-01-15T18:03:18.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.126+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ed01517e0 con 0x7f9ed01541e0 2024-01-15T18:03:18.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.128+0000 7f9ecd600640 1 -- 172.21.15.26:0/3613516874 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9eb8055380 con 0x7f9ed01541e0 2024-01-15T18:03:18.131 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:03:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806bbc0 msgr2=0x7f9ea806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806bbc0 0x7f9ea806e080 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009b20 tx=0x7f9ec0008040 comp rx=0 tx=0).stop 2024-01-15T18:03:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed01541e0 msgr2=0x7f9ed013a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed01541e0 0x7f9ed013a1b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8010860 tx=0x7f9eb8010d20 comp rx=0 tx=0).stop 2024-01-15T18:03:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 shutdown_connections 2024-01-15T18:03:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ed0151400 0x7f9ed014f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ea806bbc0 0x7f9ea806e080 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ed01541e0 0x7f9ed013a1b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 --2- 172.21.15.26:0/3613516874 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9eb0004d00 0x7f9ed014dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 >> 172.21.15.26:0/3613516874 conn(0x7f9ed006a800 msgr2=0x7f9ed006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:18.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 shutdown_connections 2024-01-15T18:03:18.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.130+0000 7f9ed7c00640 1 -- 172.21.15.26:0/3613516874 wait complete. 2024-01-15T18:03:18.154 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added' 2024-01-15T18:03:18.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.375+0000 7f1fc0000640 1 Processor -- start 2024-01-15T18:03:18.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.376+0000 7f1fc0000640 1 -- start start 2024-01-15T18:03:18.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.377+0000 7f1fc0000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1fb813e360 0x7f1fb80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:18.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.377+0000 7f1fc0000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb815c5f0 con 0x7f1fb813df80 2024-01-15T18:03:18.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.377+0000 7f1fc0000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb806c470 con 0x7f1fb815f9c0 2024-01-15T18:03:18.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.377+0000 7f1fc0000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb806c750 con 0x7f1fb813e360 2024-01-15T18:03:18.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.378+0000 7f1fbd600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1fb813df80 0x7f1fb815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46188/0 (socket says 172.21.15.26:46188) 2024-01-15T18:03:18.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.378+0000 7f1fbd600640 1 -- 172.21.15.26:0/2013844057 learned_addr learned my addr 172.21.15.26:0/2013844057 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:18.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.378+0000 7f1fbcc00640 1 --2- 172.21.15.26:0/2013844057 >> v2:172.21.15.124:3300/0 conn(0x7f1fb813e360 0x7f1fb80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.378+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229762812 0 0) 0x7f1fb815c5f0 con 0x7f1fb813df80 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.378+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f98003520 con 0x7f1fb813df80 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061315707 0 0) 0x7f1f98003520 con 0x7f1fb813df80 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 >> v2:172.21.15.124:3300/0 conn(0x7f1fb813e360 msgr2=0x7f1fb80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 --2- 172.21.15.26:0/2013844057 >> v2:172.21.15.124:3300/0 conn(0x7f1fb813e360 0x7f1fb80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 >> v1:172.21.15.124:6789/0 conn(0x7f1fb815f9c0 legacy=0x7f1fb80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb806ca30 con 0x7f1fb813df80 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f1fa8001690 con 0x7f1fb813df80 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 >> v1:172.21.15.26:6789/0 conn(0x7f1fb813df80 legacy=0x7f1fb815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:18.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f98003970 0x7f1f98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f98005180 0x7f1f9800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb806c750 con 0x7f1f980047c0 2024-01-15T18:03:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9800c0d0 con 0x7f1f98003970 2024-01-15T18:03:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9800c290 con 0x7f1f98005180 2024-01-15T18:03:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.379+0000 7f1fbcc00640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.380+0000 7f1fbcc00640 1 -- 172.21.15.26:0/2013844057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f98005180 msgr2=0x7f1f9800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.380+0000 7f1fbcc00640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f98005180 0x7f1f9800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.380+0000 7f1fbd600640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f98003970 0x7f1f98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.380+0000 7f1fbcc00640 1 -- 172.21.15.26:0/2013844057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f98003970 msgr2=0x7f1f98003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.380+0000 7f1fbcc00640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f98003970 0x7f1f98003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.380+0000 7f1fbcc00640 1 -- 172.21.15.26:0/2013844057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98003520 con 0x7f1f980047c0 2024-01-15T18:03:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.380+0000 7f1fbe000640 1 -- 172.21.15.26:0/2013844057 reap_dead start 2024-01-15T18:03:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.380+0000 7f1fbd600640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f98003970 0x7f1f98003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.381+0000 7f1fbcc00640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000ba10 tx=0x7f1f980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfc8bdbc694e22 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.381+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0032640 con 0x7f1f980047c0 2024-01-15T18:03:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.381+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa00327a0 con 0x7f1f980047c0 2024-01-15T18:03:18.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.381+0000 7f1fb7e00640 1 -- 172.21.15.26:0/2013844057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0032b30 con 0x7f1f980047c0 2024-01-15T18:03:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.382+0000 7f1fc0000640 1 -- 172.21.15.26:0/2013844057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f980047c0 msgr2=0x7f1f98004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.382+0000 7f1fc0000640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000ba10 tx=0x7f1f980057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.382+0000 7f1fc0000640 1 -- 172.21.15.26:0/2013844057 shutdown_connections 2024-01-15T18:03:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.382+0000 7f1fc0000640 1 --2- 172.21.15.26:0/2013844057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f980047c0 0x7f1f98004c40 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.382+0000 7f1fc0000640 1 -- 172.21.15.26:0/2013844057 >> 172.21.15.26:0/2013844057 conn(0x7f1fb806a800 msgr2=0x7f1fb806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.382+0000 7f1fc0000640 1 -- 172.21.15.26:0/2013844057 shutdown_connections 2024-01-15T18:03:18.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.382+0000 7f1fc0000640 1 -- 172.21.15.26:0/2013844057 wait complete. 2024-01-15T18:03:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.384+0000 7f1fc0000640 1 Processor -- start 2024-01-15T18:03:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.384+0000 7f1fc0000640 1 -- start start 2024-01-15T18:03:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.385+0000 7f1fc0000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980047c0 0x7f1fb813d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:18.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.385+0000 7f1fc0000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fb81f2e00 0x7f1fb81f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.385+0000 7f1fc0000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb81e7970 0x7f1fb81e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.385+0000 7f1fc0000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81e74e0 con 0x7f1fb81f2e00 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.385+0000 7f1fc0000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81e68d0 con 0x7f1f980047c0 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.385+0000 7f1fc0000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81e6bb0 con 0x7f1fb81e7970 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.385+0000 7f1fbcc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fb81f2e00 0x7f1fb81f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.385+0000 7f1fbcc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fb81f2e00 0x7f1fb81f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35112/0 (socket says 172.21.15.26:35112) 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.385+0000 7f1fbcc00640 1 -- 172.21.15.26:0/2209389626 learned_addr learned my addr 172.21.15.26:0/2209389626 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fbd600640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980047c0 0x7f1fb813d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fbe000640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb81e7970 0x7f1fb81e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:18.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fbcc00640 1 -- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb81e7970 msgr2=0x7f1fb81e7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fbcc00640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb81e7970 0x7f1fb81e7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fbcc00640 1 -- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980047c0 msgr2=0x7f1fb813d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fbcc00640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980047c0 0x7f1fb813d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fbcc00640 1 -- 172.21.15.26:0/2209389626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb81e6e90 con 0x7f1fb81f2e00 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fbcc00640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fb81f2e00 0x7f1fb81f3280 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1fa002bf90 tx=0x7f1fa0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fb5600640 1 -- 172.21.15.26:0/2209389626 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa003ca40 con 0x7f1fb81f2e00 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fb5600640 1 -- 172.21.15.26:0/2209389626 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa003cba0 con 0x7f1fb81f2e00 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.386+0000 7f1fb5600640 1 -- 172.21.15.26:0/2209389626 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0030410 con 0x7f1fb81f2e00 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.387+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb81e7170 con 0x7f1fb81f2e00 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.387+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb80ce000 con 0x7f1fb81f2e00 2024-01-15T18:03:18.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.388+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb80675d0 con 0x7f1fb81f2e00 2024-01-15T18:03:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.389+0000 7f1fb5600640 1 -- 172.21.15.26:0/2209389626 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f1fa0030570 con 0x7f1fb81f2e00 2024-01-15T18:03:18.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.389+0000 7f1fb5600640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f9006bbf0 0x7f1f9006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:18.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.390+0000 7f1fb5600640 1 -- 172.21.15.26:0/2209389626 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f1fa00aaf00 con 0x7f1fb81f2e00 2024-01-15T18:03:18.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.390+0000 7f1fbd600640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f9006bbf0 0x7f1f9006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:18.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.391+0000 7f1fbd600640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f9006bbf0 0x7f1f9006e0b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8154860 tx=0x7f1fa8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:18.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.394+0000 7f1fb5600640 1 -- 172.21.15.26:0/2209389626 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1fa00778e0 con 0x7f1fb81f2e00 2024-01-15T18:03:18.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:18.662+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added"]} v 0) v1 -- 0x7f1fb806df00 con 0x7f1fb81f2e00 2024-01-15T18:03:19.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.211+0000 7f1fb5600640 1 -- 172.21.15.26:0/2209389626 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added"]}]=0 v384) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1fa007af00 con 0x7f1fb81f2e00 2024-01-15T18:03:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.214+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f9006bbf0 msgr2=0x7f1f9006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f9006bbf0 0x7f1f9006e0b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8154860 tx=0x7f1fa8009040 comp rx=0 tx=0).stop 2024-01-15T18:03:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fb81f2e00 msgr2=0x7f1fb81f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fb81f2e00 0x7f1fb81f3280 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1fa002bf90 tx=0x7f1fa0004730 comp rx=0 tx=0).stop 2024-01-15T18:03:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 shutdown_connections 2024-01-15T18:03:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1fb81e7970 0x7f1fb81e7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f9006bbf0 0x7f1f9006e0b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1fb81f2e00 0x7f1fb81f3280 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 --2- 172.21.15.26:0/2209389626 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f980047c0 0x7f1fb813d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 >> 172.21.15.26:0/2209389626 conn(0x7f1fb806a800 msgr2=0x7f1fb80b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:19.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 shutdown_connections 2024-01-15T18:03:19.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.215+0000 7f1fc0000640 1 -- 172.21.15.26:0/2209389626 wait complete. 2024-01-15T18:03:19.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:03:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.460+0000 7f1d96800640 1 Processor -- start 2024-01-15T18:03:19.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.461+0000 7f1d96800640 1 -- start start 2024-01-15T18:03:19.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.462+0000 7f1d96800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d90160920 0x7f1d900b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:19.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.462+0000 7f1d96800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1d900cfe70 0x7f1d90161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:19.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.462+0000 7f1d96800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900d0a10 con 0x7f1d900cf9f0 2024-01-15T18:03:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.463+0000 7f1d96800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900d1f50 con 0x7f1d900cfe70 2024-01-15T18:03:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.463+0000 7f1d96800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900cf080 con 0x7f1d90160920 2024-01-15T18:03:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.463+0000 7f1d94a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1d900cf9f0 0x7f1d90161010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46198/0 (socket says 172.21.15.26:46198) 2024-01-15T18:03:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.463+0000 7f1d94a00640 1 -- 172.21.15.26:0/3759045501 learned_addr learned my addr 172.21.15.26:0/3759045501 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.463+0000 7f1d95e00640 1 --2- 172.21.15.26:0/3759045501 >> v2:172.21.15.26:3300/0 conn(0x7f1d900cfe70 0x7f1d90161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.463+0000 7f1d95400640 1 --2- 172.21.15.26:0/3759045501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d90160920 0x7f1d900b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.463+0000 7f1d8fe00640 1 -- 172.21.15.26:0/3759045501 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542841585 0 0) 0x7f1d900d0a10 con 0x7f1d900cf9f0 2024-01-15T18:03:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.463+0000 7f1d8fe00640 1 -- 172.21.15.26:0/3759045501 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d70003520 con 0x7f1d900cf9f0 2024-01-15T18:03:19.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.464+0000 7f1d95e00640 1 -- 172.21.15.26:0/3759045501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d90160920 msgr2=0x7f1d900b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:19.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.464+0000 7f1d95e00640 1 --2- 172.21.15.26:0/3759045501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d90160920 0x7f1d900b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.464+0000 7f1d95e00640 1 -- 172.21.15.26:0/3759045501 >> v1:172.21.15.26:6789/0 conn(0x7f1d900cf9f0 legacy=0x7f1d90161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:19.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.464+0000 7f1d95e00640 1 -- 172.21.15.26:0/3759045501 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900c6be0 con 0x7f1d900cfe70 2024-01-15T18:03:19.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.464+0000 7f1d95400640 1 --2- 172.21.15.26:0/3759045501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d90160920 0x7f1d900b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:03:19.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.465+0000 7f1d95e00640 1 --2- 172.21.15.26:0/3759045501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d900cfe70 0x7f1d90161720 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1d880097a0 tx=0x7f1d8802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65fa67729d80d4e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:19.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.465+0000 7f1d8fe00640 1 -- 172.21.15.26:0/3759045501 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8802d040 con 0x7f1d900cfe70 2024-01-15T18:03:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.465+0000 7f1d8fe00640 1 -- 172.21.15.26:0/3759045501 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d88031810 con 0x7f1d900cfe70 2024-01-15T18:03:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.465+0000 7f1d8fe00640 1 -- 172.21.15.26:0/3759045501 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88031b80 con 0x7f1d900cfe70 2024-01-15T18:03:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.466+0000 7f1d96800640 1 -- 172.21.15.26:0/3759045501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d900cfe70 msgr2=0x7f1d90161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.466+0000 7f1d96800640 1 --2- 172.21.15.26:0/3759045501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d900cfe70 0x7f1d90161720 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1d880097a0 tx=0x7f1d8802b8b0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.466+0000 7f1d96800640 1 -- 172.21.15.26:0/3759045501 shutdown_connections 2024-01-15T18:03:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.466+0000 7f1d96800640 1 --2- 172.21.15.26:0/3759045501 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d900cfe70 0x7f1d90161720 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.466+0000 7f1d96800640 1 --2- 172.21.15.26:0/3759045501 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d90160920 0x7f1d900b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.466+0000 7f1d96800640 1 -- 172.21.15.26:0/3759045501 >> 172.21.15.26:0/3759045501 conn(0x7f1d900d3090 msgr2=0x7f1d900d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:19.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.466+0000 7f1d96800640 1 -- 172.21.15.26:0/3759045501 shutdown_connections 2024-01-15T18:03:19.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.466+0000 7f1d96800640 1 -- 172.21.15.26:0/3759045501 wait complete. 2024-01-15T18:03:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.468+0000 7f1d96800640 1 Processor -- start 2024-01-15T18:03:19.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.468+0000 7f1d96800640 1 -- start start 2024-01-15T18:03:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d96800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d900cf9f0 0x7f1d90151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d96800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d900cfe70 0x7f1d90154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d96800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90160920 0x7f1d900c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d96800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90153610 con 0x7f1d90160920 2024-01-15T18:03:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d96800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90153860 con 0x7f1d900cfe70 2024-01-15T18:03:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d96800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90153b40 con 0x7f1d900cf9f0 2024-01-15T18:03:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d95e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90160920 0x7f1d900c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d95e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90160920 0x7f1d900c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35124/0 (socket says 172.21.15.26:35124) 2024-01-15T18:03:19.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d95e00640 1 -- 172.21.15.26:0/752522488 learned_addr learned my addr 172.21.15.26:0/752522488 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d94a00640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d900cfe70 0x7f1d90154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.469+0000 7f1d95400640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d900cf9f0 0x7f1d90151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d95e00640 1 -- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d900cf9f0 msgr2=0x7f1d90151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d95e00640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d900cf9f0 0x7f1d90151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d95e00640 1 -- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d900cfe70 msgr2=0x7f1d90154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d95e00640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d900cfe70 0x7f1d90154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d95e00640 1 -- 172.21.15.26:0/752522488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d88009400 con 0x7f1d90160920 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d95e00640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90160920 0x7f1d900c8b60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1d8802b8f0 tx=0x7f1d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d8d600640 1 -- 172.21.15.26:0/752522488 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88006e10 con 0x7f1d90160920 2024-01-15T18:03:19.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d8d600640 1 -- 172.21.15.26:0/752522488 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8803ad60 con 0x7f1d90160920 2024-01-15T18:03:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d90153e20 con 0x7f1d90160920 2024-01-15T18:03:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.470+0000 7f1d8d600640 1 -- 172.21.15.26:0/752522488 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d88044630 con 0x7f1d90160920 2024-01-15T18:03:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.471+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d90154b60 con 0x7f1d90160920 2024-01-15T18:03:19.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.472+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d90154340 con 0x7f1d90160920 2024-01-15T18:03:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.472+0000 7f1d8d600640 1 -- 172.21.15.26:0/752522488 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f1d88044790 con 0x7f1d90160920 2024-01-15T18:03:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.473+0000 7f1d8d600640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1d6806bbc0 0x7f1d6806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.473+0000 7f1d8d600640 1 -- 172.21.15.26:0/752522488 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f1d880aa790 con 0x7f1d90160920 2024-01-15T18:03:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.474+0000 7f1d95400640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1d6806bbc0 0x7f1d6806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:19.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.475+0000 7f1d95400640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1d6806bbc0 0x7f1d6806e080 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1d80003fd0 tx=0x7f1d800039f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:19.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.478+0000 7f1d8d600640 1 -- 172.21.15.26:0/752522488 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d90154340 con 0x7f1d90160920 2024-01-15T18:03:19.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.670+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1d90154340 con 0x7f1d90160920 2024-01-15T18:03:19.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.675+0000 7f1d8d600640 1 -- 172.21.15.26:0/752522488 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1d90154340 con 0x7f1d90160920 2024-01-15T18:03:19.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1d6806bbc0 msgr2=0x7f1d6806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:19.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1d6806bbc0 0x7f1d6806e080 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1d80003fd0 tx=0x7f1d800039f0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90160920 msgr2=0x7f1d900c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:19.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90160920 0x7f1d900c8b60 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f1d8802b8f0 tx=0x7f1d880046c0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 shutdown_connections 2024-01-15T18:03:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1d6806bbc0 0x7f1d6806e080 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1d90160920 0x7f1d900c8b60 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1d900cfe70 0x7f1d90154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 --2- 172.21.15.26:0/752522488 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1d900cf9f0 0x7f1d90151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 >> 172.21.15.26:0/752522488 conn(0x7f1d900d3090 msgr2=0x7f1d9006c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 shutdown_connections 2024-01-15T18:03:19.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:19.678+0000 7f1d96800640 1 -- 172.21.15.26:0/752522488 wait complete. 2024-01-15T18:03:19.701 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:03:19.701 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:03:19.821 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:03:19.949 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:03:19.950 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:03:19.950 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:03:20.010 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:03:20.011 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:03:20.011 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:03:20.069 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:03:20.070 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:03:20.070 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:03:20.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:03:20.130 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:03:20.130 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:03:20.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:03:20.187 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:03:20.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:03:20.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.415+0000 7f069a000640 1 Processor -- start 2024-01-15T18:03:20.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.415+0000 7f069a000640 1 -- start start 2024-01-15T18:03:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.416+0000 7f069a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0694143ae0 0x7f06940cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.416+0000 7f069a000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06940cf7d0 0x7f069415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.416+0000 7f069a000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06940cfc30 con 0x7f069415e1e0 2024-01-15T18:03:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.416+0000 7f069a000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06940b1ee0 con 0x7f0694143ae0 2024-01-15T18:03:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.416+0000 7f069a000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06940b2780 con 0x7f06940cf7d0 2024-01-15T18:03:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.417+0000 7f0693400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0694143ae0 0x7f06940cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.417+0000 7f0693400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0694143ae0 0x7f06940cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35134/0 (socket says 172.21.15.26:35134) 2024-01-15T18:03:20.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.417+0000 7f0693400640 1 -- 172.21.15.26:0/3946177788 learned_addr learned my addr 172.21.15.26:0/3946177788 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:20.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.417+0000 7f0692a00640 1 --2- 172.21.15.26:0/3946177788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06940cf7d0 0x7f069415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.417+0000 7f0693400640 1 -- 172.21.15.26:0/3946177788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06940cf7d0 msgr2=0x7f069415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:20.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.417+0000 7f0693400640 1 --2- 172.21.15.26:0/3946177788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06940cf7d0 0x7f069415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.417+0000 7f0693400640 1 -- 172.21.15.26:0/3946177788 >> v1:172.21.15.124:6789/0 conn(0x7f069415e1e0 legacy=0x7f069415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:20.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.417+0000 7f0693400640 1 -- 172.21.15.26:0/3946177788 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069415e990 con 0x7f0694143ae0 2024-01-15T18:03:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.418+0000 7f0693400640 1 --2- 172.21.15.26:0/3946177788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0694143ae0 0x7f06940cf290 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0684009870 tx=0x7f068402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a4a93fd8366f78 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.419+0000 7f0692000640 1 -- 172.21.15.26:0/3946177788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684031640 con 0x7f0694143ae0 2024-01-15T18:03:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.419+0000 7f0692000640 1 -- 172.21.15.26:0/3946177788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06840317a0 con 0x7f0694143ae0 2024-01-15T18:03:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.419+0000 7f0692000640 1 -- 172.21.15.26:0/3946177788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684031b30 con 0x7f0694143ae0 2024-01-15T18:03:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.420+0000 7f069a000640 1 -- 172.21.15.26:0/3946177788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0694143ae0 msgr2=0x7f06940cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.420+0000 7f069a000640 1 --2- 172.21.15.26:0/3946177788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0694143ae0 0x7f06940cf290 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0684009870 tx=0x7f068402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.420+0000 7f069a000640 1 -- 172.21.15.26:0/3946177788 shutdown_connections 2024-01-15T18:03:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.420+0000 7f069a000640 1 --2- 172.21.15.26:0/3946177788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f06940cf7d0 0x7f069415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.420+0000 7f069a000640 1 --2- 172.21.15.26:0/3946177788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0694143ae0 0x7f06940cf290 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.420+0000 7f069a000640 1 -- 172.21.15.26:0/3946177788 >> 172.21.15.26:0/3946177788 conn(0x7f069406a800 msgr2=0x7f069406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:20.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.420+0000 7f069a000640 1 -- 172.21.15.26:0/3946177788 shutdown_connections 2024-01-15T18:03:20.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.420+0000 7f069a000640 1 -- 172.21.15.26:0/3946177788 wait complete. 2024-01-15T18:03:20.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.422+0000 7f069a000640 1 Processor -- start 2024-01-15T18:03:20.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.422+0000 7f069a000640 1 -- start start 2024-01-15T18:03:20.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f069a000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06940cf7d0 0x7f069415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f069a000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0694143ae0 0x7f06940adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f069a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f069415e1e0 0x7f06941597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f069a000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06940a8580 con 0x7f069415e1e0 2024-01-15T18:03:20.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f069a000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06940ae400 con 0x7f06940cf7d0 2024-01-15T18:03:20.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f069a000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069415c520 con 0x7f0694143ae0 2024-01-15T18:03:20.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f0693e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f069415e1e0 0x7f06941597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f0693e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f069415e1e0 0x7f06941597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35144/0 (socket says 172.21.15.26:35144) 2024-01-15T18:03:20.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f0693e00640 1 -- 172.21.15.26:0/1713425528 learned_addr learned my addr 172.21.15.26:0/1713425528 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:20.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f0692a00640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0694143ae0 0x7f06940adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f0693400640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06940cf7d0 0x7f069415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f0693e00640 1 -- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0694143ae0 msgr2=0x7f06940adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:20.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f0693e00640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0694143ae0 0x7f06940adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f0693e00640 1 -- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06940cf7d0 msgr2=0x7f069415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.423+0000 7f0693e00640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06940cf7d0 0x7f069415bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.424+0000 7f0693e00640 1 -- 172.21.15.26:0/1713425528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840094d0 con 0x7f069415e1e0 2024-01-15T18:03:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.424+0000 7f0692a00640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0694143ae0 0x7f06940adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.424+0000 7f0693e00640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f069415e1e0 0x7f06941597d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f068c00b970 tx=0x7f068c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.424+0000 7f068b400640 1 -- 172.21.15.26:0/1713425528 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00cb40 con 0x7f069415e1e0 2024-01-15T18:03:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.424+0000 7f068b400640 1 -- 172.21.15.26:0/1713425528 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c00cca0 con 0x7f069415e1e0 2024-01-15T18:03:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.424+0000 7f068b400640 1 -- 172.21.15.26:0/1713425528 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c014680 con 0x7f069415e1e0 2024-01-15T18:03:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.424+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069415c800 con 0x7f069415e1e0 2024-01-15T18:03:20.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.424+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069415cdc0 con 0x7f069415e1e0 2024-01-15T18:03:20.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.425+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06940b2bf0 con 0x7f069415e1e0 2024-01-15T18:03:20.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.426+0000 7f068b400640 1 -- 172.21.15.26:0/1713425528 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f068c01d3b0 con 0x7f069415e1e0 2024-01-15T18:03:20.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.427+0000 7f068b400640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f066c06bbc0 0x7f066c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.427+0000 7f068b400640 1 -- 172.21.15.26:0/1713425528 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f068c088490 con 0x7f069415e1e0 2024-01-15T18:03:20.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.428+0000 7f0693400640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f066c06bbc0 0x7f066c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.429+0000 7f0693400640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f066c06bbc0 0x7f066c06e080 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0684036000 tx=0x7f068403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:20.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.433+0000 7f068b400640 1 -- 172.21.15.26:0/1713425528 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06940b2bf0 con 0x7f069415e1e0 2024-01-15T18:03:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.663+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06940b2bf0 con 0x7f069415e1e0 2024-01-15T18:03:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.664+0000 7f068b400640 1 -- 172.21.15.26:0/1713425528 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f06940b2bf0 con 0x7f069415e1e0 2024-01-15T18:03:20.669 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:20.670 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5641,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/1159022909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":1159022909},{"type":"v1","addr":"172.21.15.26:6835","nonce":1159022909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5644,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/248571798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":248571798},{"type":"v1","addr":"172.21.15.124:6837","nonce":248571798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6405,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/3357264964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":3357264964},{"type":"v1","addr":"172.21.15.26:6833","nonce":3357264964}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3611655863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3611655863},{"type":"v1","addr":"172.21.15.124:6836","nonce":3611655863}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[]} 2024-01-15T18:03:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 38 2024-01-15T18:03:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f066c06bbc0 msgr2=0x7f066c06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f066c06bbc0 0x7f066c06e080 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0684036000 tx=0x7f068403c040 comp rx=0 tx=0).stop 2024-01-15T18:03:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f069415e1e0 msgr2=0x7f06941597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f069415e1e0 0x7f06941597d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f068c00b970 tx=0x7f068c00be30 comp rx=0 tx=0).stop 2024-01-15T18:03:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 shutdown_connections 2024-01-15T18:03:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0694143ae0 0x7f06940adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f066c06bbc0 0x7f066c06e080 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f069415e1e0 0x7f06941597d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 --2- 172.21.15.26:0/1713425528 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f06940cf7d0 0x7f069415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 >> 172.21.15.26:0/1713425528 conn(0x7f069406a800 msgr2=0x7f069406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 shutdown_connections 2024-01-15T18:03:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.667+0000 7f069a000640 1 -- 172.21.15.26:0/1713425528 wait complete. 2024-01-15T18:03:20.691 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:03:20.691 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:03:20.692 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:03:20.692 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:03:20.692 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:03:20.692 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:03:20.692 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:03:20.692 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:03:20.693 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:03:20.693 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:03:20.693 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:03:20.693 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:03:20.693 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:03:20.693 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:03:20.694 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:03:20.694 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:03:20.711 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:03:20.711 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:03:20.715 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:03:20.715 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:03:20.718 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:03:20.718 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:03:20.719 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:03:20.719 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:03:20.762 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:03:20.763 DEBUG:teuthology.parallel:result is None 2024-01-15T18:03:20.766 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:03:20.767 DEBUG:teuthology.parallel:result is None 2024-01-15T18:03:20.769 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:03:20.769 DEBUG:teuthology.parallel:result is None 2024-01-15T18:03:20.772 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:03:20.772 DEBUG:teuthology.parallel:result is None 2024-01-15T18:03:20.773 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:03:20.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-01-15T18:03:20.922 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-01-15T18:03:20.922 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-01-15T18:03:20.923 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-15T18:03:20.924 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T18:03:20.928 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T18:03:20.928 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T18:03:20.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.961+0000 7fc135200640 1 Processor -- start 2024-01-15T18:03:20.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.961+0000 7fc135200640 1 -- start start 2024-01-15T18:03:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.962+0000 7fc135200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc13013df80 0x7fc13015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.962+0000 7fc135200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc1300cf9a0 0x7fc1300cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.962+0000 7fc135200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13006c470 con 0x7fc13013e360 2024-01-15T18:03:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.962+0000 7fc135200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13006c750 con 0x7fc1300cf9a0 2024-01-15T18:03:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.962+0000 7fc135200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13006ca30 con 0x7fc13013df80 2024-01-15T18:03:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.963+0000 7fc12d400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc13013e360 0x7fc1300cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:46220/0 (socket says 172.21.15.26:46220) 2024-01-15T18:03:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.963+0000 7fc12d400640 1 -- 172.21.15.26:0/187420616 learned_addr learned my addr 172.21.15.26:0/187420616 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.963+0000 7fc12e800640 1 --2- 172.21.15.26:0/187420616 >> v2:172.21.15.26:3300/0 conn(0x7fc1300cf9a0 0x7fc1300cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.963+0000 7fc12de00640 1 --2- 172.21.15.26:0/187420616 >> v2:172.21.15.124:3300/0 conn(0x7fc13013df80 0x7fc13015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.963+0000 7fc12e800640 1 -- 172.21.15.26:0/187420616 >> v2:172.21.15.124:3300/0 conn(0x7fc13013df80 msgr2=0x7fc13015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.963+0000 7fc12e800640 1 --2- 172.21.15.26:0/187420616 >> v2:172.21.15.124:3300/0 conn(0x7fc13013df80 0x7fc13015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.963+0000 7fc12e800640 1 -- 172.21.15.26:0/187420616 >> v1:172.21.15.26:6789/0 conn(0x7fc13013e360 legacy=0x7fc1300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.963+0000 7fc12e800640 1 -- 172.21.15.26:0/187420616 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13006cd10 con 0x7fc1300cf9a0 2024-01-15T18:03:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.964+0000 7fc12e800640 1 --2- 172.21.15.26:0/187420616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1300cf9a0 0x7fc1300cd840 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc130072770 tx=0x7fc12802bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6dc90f1b9f6ac5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc12ca00640 1 -- 172.21.15.26:0/187420616 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12802ca30 con 0x7fc1300cf9a0 2024-01-15T18:03:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc12ca00640 1 -- 172.21.15.26:0/187420616 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12802cb90 con 0x7fc1300cf9a0 2024-01-15T18:03:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc12ca00640 1 -- 172.21.15.26:0/187420616 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc128034440 con 0x7fc1300cf9a0 2024-01-15T18:03:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc135200640 1 -- 172.21.15.26:0/187420616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1300cf9a0 msgr2=0x7fc1300cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc135200640 1 --2- 172.21.15.26:0/187420616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1300cf9a0 0x7fc1300cd840 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc130072770 tx=0x7fc12802bd00 comp rx=0 tx=0).stop 2024-01-15T18:03:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc135200640 1 -- 172.21.15.26:0/187420616 shutdown_connections 2024-01-15T18:03:20.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc135200640 1 --2- 172.21.15.26:0/187420616 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1300cf9a0 0x7fc1300cd840 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc135200640 1 --2- 172.21.15.26:0/187420616 >> v2:172.21.15.124:3300/0 conn(0x7fc13013df80 0x7fc13015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc135200640 1 -- 172.21.15.26:0/187420616 >> 172.21.15.26:0/187420616 conn(0x7fc13006a800 msgr2=0x7fc13006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:20.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc135200640 1 -- 172.21.15.26:0/187420616 shutdown_connections 2024-01-15T18:03:20.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.965+0000 7fc135200640 1 -- 172.21.15.26:0/187420616 wait complete. 2024-01-15T18:03:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.967+0000 7fc135200640 1 Processor -- start 2024-01-15T18:03:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.967+0000 7fc135200640 1 -- start start 2024-01-15T18:03:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.968+0000 7fc135200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1300cf9a0 0x7fc1300b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.968+0000 7fc135200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc13013df80 0x7fc13013a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.968+0000 7fc135200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc13013e360 0x7fc13014e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.968+0000 7fc135200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300b2ed0 con 0x7fc13013df80 2024-01-15T18:03:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.968+0000 7fc135200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc130139bf0 con 0x7fc13013e360 2024-01-15T18:03:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.968+0000 7fc135200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300b22c0 con 0x7fc1300cf9a0 2024-01-15T18:03:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.968+0000 7fc12d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc13013df80 0x7fc13013a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.968+0000 7fc12d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc13013df80 0x7fc13013a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35196/0 (socket says 172.21.15.26:35196) 2024-01-15T18:03:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.968+0000 7fc12d400640 1 -- 172.21.15.26:0/2444316796 learned_addr learned my addr 172.21.15.26:0/2444316796 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.969+0000 7fc12de00640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1300cf9a0 0x7fc1300b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.969+0000 7fc12e800640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc13013e360 0x7fc13014e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.969+0000 7fc12d400640 1 -- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1300cf9a0 msgr2=0x7fc1300b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.969+0000 7fc12d400640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1300cf9a0 0x7fc1300b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.969+0000 7fc12d400640 1 -- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc13013e360 msgr2=0x7fc13014e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.969+0000 7fc12d400640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc13013e360 0x7fc13014e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.969+0000 7fc12d400640 1 -- 172.21.15.26:0/2444316796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc128009a70 con 0x7fc13013df80 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.969+0000 7fc12d400640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc13013df80 0x7fc13013a260 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c780 tx=0x7fc11800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.970+0000 7fc126000640 1 -- 172.21.15.26:0/2444316796 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180072d0 con 0x7fc13013df80 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.970+0000 7fc126000640 1 -- 172.21.15.26:0/2444316796 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc118007450 con 0x7fc13013df80 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.970+0000 7fc126000640 1 -- 172.21.15.26:0/2444316796 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180047e0 con 0x7fc13013df80 2024-01-15T18:03:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.970+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1300b25a0 con 0x7fc13013df80 2024-01-15T18:03:20.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.970+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1300b2b60 con 0x7fc13013df80 2024-01-15T18:03:20.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.971+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1300b3080 con 0x7fc13013df80 2024-01-15T18:03:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.972+0000 7fc126000640 1 -- 172.21.15.26:0/2444316796 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fc118004940 con 0x7fc13013df80 2024-01-15T18:03:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.973+0000 7fc126000640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc10806bbc0 0x7fc10806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.973+0000 7fc126000640 1 -- 172.21.15.26:0/2444316796 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fc118087920 con 0x7fc13013df80 2024-01-15T18:03:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.973+0000 7fc12de00640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc10806bbc0 0x7fc10806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.974+0000 7fc12de00640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc10806bbc0 0x7fc10806e080 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc120009b40 tx=0x7fc120009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:20.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:20.977+0000 7fc126000640 1 -- 172.21.15.26:0/2444316796 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc1300b3080 con 0x7fc13013df80 2024-01-15T18:03:21.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:21.190+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc1300675d0 con 0x7fc13013df80 2024-01-15T18:03:22.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.050+0000 7fc126000640 1 -- 172.21.15.26:0/2444316796 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc118018080 con 0x7fc13013df80 2024-01-15T18:03:22.055 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-01-15T18:03:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc10806bbc0 msgr2=0x7fc10806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc10806bbc0 0x7fc10806e080 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc120009b40 tx=0x7fc120009190 comp rx=0 tx=0).stop 2024-01-15T18:03:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc13013df80 msgr2=0x7fc13013a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc13013df80 0x7fc13013a260 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc11800c780 tx=0x7fc11800cc40 comp rx=0 tx=0).stop 2024-01-15T18:03:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 shutdown_connections 2024-01-15T18:03:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc10806bbc0 0x7fc10806e080 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc13013e360 0x7fc13014e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc13013df80 0x7fc13013a260 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 --2- 172.21.15.26:0/2444316796 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1300cf9a0 0x7fc1300b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 >> 172.21.15.26:0/2444316796 conn(0x7fc13006a800 msgr2=0x7fc1300b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:22.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 shutdown_connections 2024-01-15T18:03:22.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.054+0000 7fc135200640 1 -- 172.21.15.26:0/2444316796 wait complete. 2024-01-15T18:03:22.082 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:03:22.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.300+0000 7fea42200640 1 Processor -- start 2024-01-15T18:03:22.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.301+0000 7fea42200640 1 -- start start 2024-01-15T18:03:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.302+0000 7fea42200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06c580 0x7fea3c0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.302+0000 7fea42200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c0cfe30 con 0x7fea3c0cf560 2024-01-15T18:03:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.302+0000 7fea42200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c0d0110 con 0x7fea3c15d560 2024-01-15T18:03:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.302+0000 7fea42200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0d03f0 con 0x7fea3c06c580 2024-01-15T18:03:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.303+0000 7fea41800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fea3c0cf560 0x7fea3c0b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45302/0 (socket says 172.21.15.26:45302) 2024-01-15T18:03:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.303+0000 7fea41800640 1 -- 172.21.15.26:0/3544849904 learned_addr learned my addr 172.21.15.26:0/3544849904 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.303+0000 7fea3be00640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06c580 0x7fea3c0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.303+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191821519 0 0) 0x7fea3c0cfe30 con 0x7fea3c0cf560 2024-01-15T18:03:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.303+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea1c003520 con 0x7fea3c0cf560 2024-01-15T18:03:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1516494328 0 0) 0x7fea1c003520 con 0x7fea3c0cf560 2024-01-15T18:03:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06c580 msgr2=0x7fea3c0cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06c580 0x7fea3c0cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 >> v1:172.21.15.124:6789/0 conn(0x7fea3c15d560 legacy=0x7fea3c06be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0d06d0 con 0x7fea3c0cf560 2024-01-15T18:03:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fea34001670 con 0x7fea3c0cf560 2024-01-15T18:03:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 >> v1:172.21.15.26:6789/0 conn(0x7fea3c0cf560 legacy=0x7fea3c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea1c0039a0 0x7fea1c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06c580 0x7fea1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea1c004d00 0x7fea1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0d03f0 con 0x7fea1c004d00 2024-01-15T18:03:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c00ba30 con 0x7fea1c0039a0 2024-01-15T18:03:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c00bbf0 con 0x7fea3c06c580 2024-01-15T18:03:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea41800640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea1c004d00 0x7fea1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.304+0000 7fea40e00640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea1c0039a0 0x7fea1c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.305+0000 7fea3be00640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06c580 0x7fea1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.305+0000 7fea41800640 1 -- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06c580 msgr2=0x7fea1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.305+0000 7fea41800640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06c580 0x7fea1c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.305+0000 7fea41800640 1 -- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea1c0039a0 msgr2=0x7fea1c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.305+0000 7fea41800640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea1c0039a0 0x7fea1c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.305+0000 7fea41800640 1 -- 172.21.15.26:0/3544849904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c003520 con 0x7fea1c004d00 2024-01-15T18:03:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.306+0000 7fea41800640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea1c004d00 0x7fea1c005180 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c210 tx=0x7fea1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae83d95b6a596726 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.306+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34005ad0 con 0x7fea1c004d00 2024-01-15T18:03:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.306+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea34005c30 con 0x7fea1c004d00 2024-01-15T18:03:22.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.306+0000 7fea3b400640 1 -- 172.21.15.26:0/3544849904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3402c890 con 0x7fea1c004d00 2024-01-15T18:03:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.307+0000 7fea42200640 1 -- 172.21.15.26:0/3544849904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea1c004d00 msgr2=0x7fea1c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.307+0000 7fea42200640 1 --2- 172.21.15.26:0/3544849904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea1c004d00 0x7fea1c005180 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fea3400c210 tx=0x7fea1c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.307+0000 7fea41800640 1 -- 172.21.15.26:0/3544849904 reap_dead start 2024-01-15T18:03:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.307+0000 7fea42200640 1 -- 172.21.15.26:0/3544849904 shutdown_connections 2024-01-15T18:03:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.307+0000 7fea42200640 1 -- 172.21.15.26:0/3544849904 >> 172.21.15.26:0/3544849904 conn(0x7fea3c099a60 msgr2=0x7fea3c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.307+0000 7fea42200640 1 -- 172.21.15.26:0/3544849904 shutdown_connections 2024-01-15T18:03:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.307+0000 7fea42200640 1 -- 172.21.15.26:0/3544849904 wait complete. 2024-01-15T18:03:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.309+0000 7fea42200640 1 Processor -- start 2024-01-15T18:03:22.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.309+0000 7fea42200640 1 -- start start 2024-01-15T18:03:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.310+0000 7fea42200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea1c004d00 0x7fea3c152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.310+0000 7fea42200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c15e360 0x7fea3c0c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.310+0000 7fea42200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06f380 0x7fea3c06f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.310+0000 7fea42200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c15f1a0 con 0x7fea3c15e360 2024-01-15T18:03:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.310+0000 7fea42200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c0c9590 con 0x7fea1c004d00 2024-01-15T18:03:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.310+0000 7fea42200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c1528b0 con 0x7fea3c06f380 2024-01-15T18:03:22.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.310+0000 7fea3be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c15e360 0x7fea3c0c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.310+0000 7fea3be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c15e360 0x7fea3c0c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46126/0 (socket says 172.21.15.26:46126) 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.310+0000 7fea3be00640 1 -- 172.21.15.26:0/1241538978 learned_addr learned my addr 172.21.15.26:0/1241538978 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea3be00640 1 -- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06f380 msgr2=0x7fea3c06f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea41800640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06f380 0x7fea3c06f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea40e00640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea1c004d00 0x7fea3c152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea3be00640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06f380 0x7fea3c06f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea3be00640 1 -- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea1c004d00 msgr2=0x7fea3c152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea3be00640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea1c004d00 0x7fea3c152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea3be00640 1 -- 172.21.15.26:0/1241538978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0c9830 con 0x7fea3c15e360 2024-01-15T18:03:22.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea3be00640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c15e360 0x7fea3c0c88e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fea2c007f30 tx=0x7fea2c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea38c00640 1 -- 172.21.15.26:0/1241538978 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00e030 con 0x7fea3c15e360 2024-01-15T18:03:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea38c00640 1 -- 172.21.15.26:0/1241538978 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c016470 con 0x7fea3c15e360 2024-01-15T18:03:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea38c00640 1 -- 172.21.15.26:0/1241538978 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c016820 con 0x7fea3c15e360 2024-01-15T18:03:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c15e900 con 0x7fea3c15e360 2024-01-15T18:03:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.311+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c15eec0 con 0x7fea3c15e360 2024-01-15T18:03:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.312+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c067b60 con 0x7fea3c15e360 2024-01-15T18:03:22.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.313+0000 7fea38c00640 1 -- 172.21.15.26:0/1241538978 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fea2c016980 con 0x7fea3c15e360 2024-01-15T18:03:22.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.315+0000 7fea38c00640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1406bbc0 0x7fea1406e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.315+0000 7fea38c00640 1 -- 172.21.15.26:0/1241538978 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fea2c00f070 con 0x7fea3c15e360 2024-01-15T18:03:22.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.316+0000 7fea40e00640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1406bbc0 0x7fea1406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.317+0000 7fea40e00640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1406bbc0 0x7fea1406e080 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fea24007fd0 tx=0x7fea24009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:22.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.319+0000 7fea38c00640 1 -- 172.21.15.26:0/1241538978 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea2c055d60 con 0x7fea3c15e360 2024-01-15T18:03:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.589+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fea3c06cb40 con 0x7fea3c15e360 2024-01-15T18:03:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.590+0000 7fea38c00640 1 -- 172.21.15.26:0/1241538978 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fea2c059380 con 0x7fea3c15e360 2024-01-15T18:03:22.595 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:22.595 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:03:22.596 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-01-15T18:03:22.596 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.596 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-01-15T18:03:22.596 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVl3erNJhAANxPK+39yVM5neq9iKC1HkA==", 2024-01-15T18:03:22.596 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.596 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:03:22.596 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:03:22.596 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:03:22.596 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.597 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.597 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.597 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.597 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-01-15T18:03:22.597 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlLaUlDRAAeKaen5hLlwDDWY6OAMRsPQ==", 2024-01-15T18:03:22.597 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:03:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:03:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:03:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-01-15T18:03:22.598 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlaIAVOBAAA8LEFM1HtzRtNooQCCjS/w==", 2024-01-15T18:03:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:03:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:03:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:03:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.599 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-01-15T18:03:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlmpTBHRAAUwbZeVpysMsn56QRLRJCoA==", 2024-01-15T18:03:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:03:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:03:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:03:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.600 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-01-15T18:03:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA7caVlCTmgDhAAPWb0qEh134xxyF8lM9DoNg==", 2024-01-15T18:03:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:03:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:03:22.601 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-01-15T18:03:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA+caVlaxF5JxAA23Z9j21ATy3GVlhgndDIxw==", 2024-01-15T18:03:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:03:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:03:22.602 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-01-15T18:03:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBCcaVlcRtjBRAAOFN9Xf5kN5MXL6yQfRI27g==", 2024-01-15T18:03:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:03:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:03:22.603 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-01-15T18:03:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBFcaVlvAzEHhAACDoNh7BZY9gbKTdGwIrhlg==", 2024-01-15T18:03:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:03:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:03:22.604 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-01-15T18:03:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBMcaVl8yiOMBAAdI4NTeSB1d/nSHRaIDMcFw==", 2024-01-15T18:03:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:03:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:03:22.605 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-01-15T18:03:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBQcaVltf8sARAAfPaJAfoTgQoEivw90jRz/Q==", 2024-01-15T18:03:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:03:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:03:22.606 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBTcaVltuKiGhAAT0cIJiRkQ38QXT/h6Gyggg==", 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.607 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBWcaVlbJqVMhAAKjpl6pAHbytjZWMYeRbRjA==", 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.608 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA==", 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.609 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-01-15T18:03:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVlrVpqBxAAxOL0iuAlOvfR+KgSsin3vQ==", 2024-01-15T18:03:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:03:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:03:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:03:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:03:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.610 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlO8vdNxAAlUE/TFCDQoXuxvwLPwUvtg==", 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.611 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-01-15T18:03:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVl5qgCEhAALgt7d3OjEmq5bc4nAD2rTw==", 2024-01-15T18:03:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:03:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:03:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:03:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:03:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.612 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-01-15T18:03:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA2caVlyg4eHBAAwTCXVk8hO9+zcZ1tUA3gfA==", 2024-01-15T18:03:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:03:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:03:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:03:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.613 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlvuBcGhAAGqdCMSffJwnMKlFh3CtyCw==", 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.614 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVle8Z+DhAA9cWxzc4puvrHAedqupPG3Q==", 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:22.615 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-01-15T18:03:22.616 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:03:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1406bbc0 msgr2=0x7fea1406e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1406bbc0 0x7fea1406e080 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fea24007fd0 tx=0x7fea24009040 comp rx=0 tx=0).stop 2024-01-15T18:03:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c15e360 msgr2=0x7fea3c0c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c15e360 0x7fea3c0c88e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fea2c007f30 tx=0x7fea2c011660 comp rx=0 tx=0).stop 2024-01-15T18:03:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 shutdown_connections 2024-01-15T18:03:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea3c06f380 0x7fea3c06f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1406bbc0 0x7fea1406e080 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea3c15e360 0x7fea3c0c88e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 --2- 172.21.15.26:0/1241538978 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea1c004d00 0x7fea3c152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.593+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 >> 172.21.15.26:0/1241538978 conn(0x7fea3c099a60 msgr2=0x7fea3c0b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.594+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 shutdown_connections 2024-01-15T18:03:22.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.594+0000 7fea42200640 1 -- 172.21.15.26:0/1241538978 wait complete. 2024-01-15T18:03:22.619 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:03:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.837+0000 7f5cafe00640 1 Processor -- start 2024-01-15T18:03:22.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.838+0000 7f5cafe00640 1 -- start start 2024-01-15T18:03:22.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.839+0000 7f5cafe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5cb00cc500 0x7f5cb00cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.839+0000 7f5cafe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb0070e50 con 0x7f5cb006d160 2024-01-15T18:03:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.839+0000 7f5cafe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb0144020 con 0x7f5cb0143c40 2024-01-15T18:03:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.839+0000 7f5cafe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb00723c0 con 0x7f5cb00cc500 2024-01-15T18:03:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.840+0000 7f5caf400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5cb006d160 0x7f5cb006d560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45310/0 (socket says 172.21.15.26:45310) 2024-01-15T18:03:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.840+0000 7f5caf400640 1 -- 172.21.15.26:0/2905301501 learned_addr learned my addr 172.21.15.26:0/2905301501 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.840+0000 7f5cae000640 1 --2- 172.21.15.26:0/2905301501 >> v2:172.21.15.124:3300/0 conn(0x7f5cb00cc500 0x7f5cb00cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.840+0000 7f5cad600640 1 -- 172.21.15.26:0/2905301501 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410374298 0 0) 0x7f5cb0070e50 con 0x7f5cb006d160 2024-01-15T18:03:22.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.840+0000 7f5cad600640 1 -- 172.21.15.26:0/2905301501 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c90003520 con 0x7f5cb006d160 2024-01-15T18:03:22.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.841+0000 7f5cae000640 1 -- 172.21.15.26:0/2905301501 >> v1:172.21.15.124:6789/0 conn(0x7f5cb0143c40 legacy=0x7f5cb0071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.841+0000 7f5cae000640 1 -- 172.21.15.26:0/2905301501 >> v1:172.21.15.26:6789/0 conn(0x7f5cb006d160 legacy=0x7f5cb006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.841+0000 7f5cae000640 1 -- 172.21.15.26:0/2905301501 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb0071e00 con 0x7f5cb00cc500 2024-01-15T18:03:22.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.842+0000 7f5cae000640 1 --2- 172.21.15.26:0/2905301501 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cb00cc500 0x7f5cb00cc990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5c980097a0 tx=0x7f5c9802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab4093c38c5b291 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.843+0000 7f5cad600640 1 -- 172.21.15.26:0/2905301501 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98031830 con 0x7f5cb00cc500 2024-01-15T18:03:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.843+0000 7f5cad600640 1 -- 172.21.15.26:0/2905301501 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c98031990 con 0x7f5cb00cc500 2024-01-15T18:03:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.843+0000 7f5cad600640 1 -- 172.21.15.26:0/2905301501 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98031d20 con 0x7f5cb00cc500 2024-01-15T18:03:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.844+0000 7f5cafe00640 1 -- 172.21.15.26:0/2905301501 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cb00cc500 msgr2=0x7f5cb00cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.844+0000 7f5cafe00640 1 --2- 172.21.15.26:0/2905301501 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cb00cc500 0x7f5cb00cc990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5c980097a0 tx=0x7f5c9802b8d0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.844+0000 7f5cafe00640 1 -- 172.21.15.26:0/2905301501 shutdown_connections 2024-01-15T18:03:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.844+0000 7f5cafe00640 1 --2- 172.21.15.26:0/2905301501 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cb00cc500 0x7f5cb00cc990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.844+0000 7f5cafe00640 1 -- 172.21.15.26:0/2905301501 >> 172.21.15.26:0/2905301501 conn(0x7f5cb006a800 msgr2=0x7f5cb006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:22.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.844+0000 7f5cafe00640 1 -- 172.21.15.26:0/2905301501 shutdown_connections 2024-01-15T18:03:22.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.844+0000 7f5cafe00640 1 -- 172.21.15.26:0/2905301501 wait complete. 2024-01-15T18:03:22.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.846+0000 7f5cafe00640 1 Processor -- start 2024-01-15T18:03:22.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.846+0000 7f5cafe00640 1 -- start start 2024-01-15T18:03:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.847+0000 7f5cafe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cb006d160 0x7f5cb015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5cafe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cb00cc500 0x7f5cb00c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5cafe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cb0143c40 0x7f5cb00c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5cafe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb00c7fe0 con 0x7f5cb006d160 2024-01-15T18:03:22.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5cafe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb00c82c0 con 0x7f5cb00cc500 2024-01-15T18:03:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5cafe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb00c85a0 con 0x7f5cb0143c40 2024-01-15T18:03:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5caea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cb006d160 0x7f5cb015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5caea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cb006d160 0x7f5cb015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46140/0 (socket says 172.21.15.26:46140) 2024-01-15T18:03:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5caea00640 1 -- 172.21.15.26:0/1734797273 learned_addr learned my addr 172.21.15.26:0/1734797273 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5cae000640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cb00cc500 0x7f5cb00c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5caea00640 1 -- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cb0143c40 msgr2=0x7f5cb00c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5caea00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cb0143c40 0x7f5cb00c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5caea00640 1 -- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cb00cc500 msgr2=0x7f5cb00c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:22.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5caea00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cb00cc500 0x7f5cb00c75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.848+0000 7f5caea00640 1 -- 172.21.15.26:0/1734797273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c98009400 con 0x7f5cb006d160 2024-01-15T18:03:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.849+0000 7f5caea00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cb006d160 0x7f5cb015a8f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c400 tx=0x7f5ca000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.849+0000 7f5ca6a00640 1 -- 172.21.15.26:0/1734797273 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca00052d0 con 0x7f5cb006d160 2024-01-15T18:03:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.849+0000 7f5ca6a00640 1 -- 172.21.15.26:0/1734797273 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca0005450 con 0x7f5cb006d160 2024-01-15T18:03:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.849+0000 7f5ca6a00640 1 -- 172.21.15.26:0/1734797273 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0007260 con 0x7f5cb006d160 2024-01-15T18:03:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.849+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb00c8880 con 0x7f5cb006d160 2024-01-15T18:03:22.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.849+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb015ae30 con 0x7f5cb006d160 2024-01-15T18:03:22.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.850+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb0067450 con 0x7f5cb006d160 2024-01-15T18:03:22.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.851+0000 7f5ca6a00640 1 -- 172.21.15.26:0/1734797273 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f5ca00073c0 con 0x7f5cb006d160 2024-01-15T18:03:22.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.852+0000 7f5ca6a00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c8806bbc0 0x7f5c8806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:22.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.852+0000 7f5ca6a00640 1 -- 172.21.15.26:0/1734797273 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f5ca0087930 con 0x7f5cb006d160 2024-01-15T18:03:22.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.853+0000 7f5cae000640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c8806bbc0 0x7f5c8806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:22.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.854+0000 7f5cae000640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c8806bbc0 0x7f5c8806e080 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5c98036000 tx=0x7f5c98008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:22.857+0000 7f5ca6a00640 1 -- 172.21.15.26:0/1734797273 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ca0054160 con 0x7f5cb006d160 2024-01-15T18:03:23.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.052+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cb0071810 con 0x7f5cb006d160 2024-01-15T18:03:23.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.053+0000 7f5ca6a00640 1 -- 172.21.15.26:0/1734797273 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+12920 (secure 0 0 0) 0x7f5ca0057780 con 0x7f5cb006d160 2024-01-15T18:03:23.058 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:23.058 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":68,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:03:22.047901+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:03:23.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c8806bbc0 msgr2=0x7f5c8806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c8806bbc0 0x7f5c8806e080 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5c98036000 tx=0x7f5c98008030 comp rx=0 tx=0).stop 2024-01-15T18:03:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cb006d160 msgr2=0x7f5cb015a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cb006d160 0x7f5cb015a8f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000c400 tx=0x7f5ca000c8c0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 shutdown_connections 2024-01-15T18:03:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5cb0143c40 0x7f5cb00c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c8806bbc0 0x7f5c8806e080 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5cb00cc500 0x7f5cb00c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 --2- 172.21.15.26:0/1734797273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5cb006d160 0x7f5cb015a8f0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 >> 172.21.15.26:0/1734797273 conn(0x7f5cb006a800 msgr2=0x7f5cb00aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 shutdown_connections 2024-01-15T18:03:23.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.056+0000 7f5cafe00640 1 -- 172.21.15.26:0/1734797273 wait complete. 2024-01-15T18:03:23.080 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:03:23.080 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:03:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.292+0000 7ff794c00640 1 Processor -- start 2024-01-15T18:03:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.293+0000 7ff794c00640 1 -- start start 2024-01-15T18:03:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.294+0000 7ff794c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff79015d4b0 0x7ff7900d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.294+0000 7ff794c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff7900cf9a0 0x7ff7900cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.294+0000 7ff794c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cddf0 0x7ff7900ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.294+0000 7ff794c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7900d0320 con 0x7ff7900cf9a0 2024-01-15T18:03:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.294+0000 7ff794c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7900b2730 con 0x7ff79015d4b0 2024-01-15T18:03:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.294+0000 7ff794c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900d1ec0 con 0x7ff7900cddf0 2024-01-15T18:03:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.294+0000 7ff787e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff7900cf9a0 0x7ff7900cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff787e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff7900cf9a0 0x7ff7900cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46152/0 (socket says 172.21.15.26:46152) 2024-01-15T18:03:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff787e00640 1 -- 172.21.15.26:0/2014323701 learned_addr learned my addr 172.21.15.26:0/2014323701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff78f400640 1 --2- 172.21.15.26:0/2014323701 >> v2:172.21.15.124:3300/0 conn(0x7ff79015d4b0 0x7ff7900d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff78fe00640 1 --2- 172.21.15.26:0/2014323701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cddf0 0x7ff7900ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff787e00640 1 -- 172.21.15.26:0/2014323701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cddf0 msgr2=0x7ff7900ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff787e00640 1 --2- 172.21.15.26:0/2014323701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cddf0 0x7ff7900ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff787e00640 1 -- 172.21.15.26:0/2014323701 >> v2:172.21.15.124:3300/0 conn(0x7ff79015d4b0 msgr2=0x7ff7900d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff787e00640 1 --2- 172.21.15.26:0/2014323701 >> v2:172.21.15.124:3300/0 conn(0x7ff79015d4b0 0x7ff7900d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff787e00640 1 -- 172.21.15.26:0/2014323701 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900b29d0 con 0x7ff7900cf9a0 2024-01-15T18:03:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.295+0000 7ff78fe00640 1 --2- 172.21.15.26:0/2014323701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cddf0 0x7ff7900ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.296+0000 7ff787e00640 1 --2- 172.21.15.26:0/2014323701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7900cf9a0 0x7ff7900cfde0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff79008b5b0 tx=0x7ff77802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1857802b4fe1a9b2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff78ea00640 1 -- 172.21.15.26:0/2014323701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778031860 con 0x7ff7900cf9a0 2024-01-15T18:03:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff78ea00640 1 -- 172.21.15.26:0/2014323701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7780319c0 con 0x7ff7900cf9a0 2024-01-15T18:03:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff78ea00640 1 -- 172.21.15.26:0/2014323701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff778031d50 con 0x7ff7900cf9a0 2024-01-15T18:03:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff794c00640 1 -- 172.21.15.26:0/2014323701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7900cf9a0 msgr2=0x7ff7900cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff794c00640 1 --2- 172.21.15.26:0/2014323701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7900cf9a0 0x7ff7900cfde0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff79008b5b0 tx=0x7ff77802b900 comp rx=0 tx=0).stop 2024-01-15T18:03:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff794c00640 1 -- 172.21.15.26:0/2014323701 shutdown_connections 2024-01-15T18:03:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff794c00640 1 --2- 172.21.15.26:0/2014323701 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cddf0 0x7ff7900ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff794c00640 1 --2- 172.21.15.26:0/2014323701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff7900cf9a0 0x7ff7900cfde0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff794c00640 1 --2- 172.21.15.26:0/2014323701 >> v2:172.21.15.124:3300/0 conn(0x7ff79015d4b0 0x7ff7900d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff794c00640 1 -- 172.21.15.26:0/2014323701 >> 172.21.15.26:0/2014323701 conn(0x7ff7900d3000 msgr2=0x7ff7900d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.297+0000 7ff794c00640 1 -- 172.21.15.26:0/2014323701 shutdown_connections 2024-01-15T18:03:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.298+0000 7ff794c00640 1 -- 172.21.15.26:0/2014323701 wait complete. 2024-01-15T18:03:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.299+0000 7ff794c00640 1 Processor -- start 2024-01-15T18:03:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.299+0000 7ff794c00640 1 -- start start 2024-01-15T18:03:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.300+0000 7ff794c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7900cddf0 0x7ff79015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.300+0000 7ff794c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cf9a0 0x7ff7900cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.300+0000 7ff794c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79015d4b0 0x7ff7901f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.300+0000 7ff794c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900cba90 con 0x7ff79015d4b0 2024-01-15T18:03:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.300+0000 7ff794c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900ca4b0 con 0x7ff7900cddf0 2024-01-15T18:03:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.300+0000 7ff794c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900ca790 con 0x7ff7900cf9a0 2024-01-15T18:03:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79015d4b0 0x7ff7901f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79015d4b0 0x7ff7901f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46166/0 (socket says 172.21.15.26:46166) 2024-01-15T18:03:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78fe00640 1 -- 172.21.15.26:0/3265726195 learned_addr learned my addr 172.21.15.26:0/3265726195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff787e00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cf9a0 0x7ff7900cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78f400640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7900cddf0 0x7ff79015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78fe00640 1 -- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cf9a0 msgr2=0x7ff7900cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78fe00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cf9a0 0x7ff7900cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78fe00640 1 -- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7900cddf0 msgr2=0x7ff79015b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78fe00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7900cddf0 0x7ff79015b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78fe00640 1 -- 172.21.15.26:0/3265726195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7780094d0 con 0x7ff79015d4b0 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78f400640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7900cddf0 0x7ff79015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff787e00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cf9a0 0x7ff7900cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.301+0000 7ff78fe00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79015d4b0 0x7ff7901f2d60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff78800cef0 tx=0x7ff78800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.302+0000 7ff787400640 1 -- 172.21.15.26:0/3265726195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800d840 con 0x7ff79015d4b0 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.302+0000 7ff787400640 1 -- 172.21.15.26:0/3265726195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800d9a0 con 0x7ff79015d4b0 2024-01-15T18:03:23.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.302+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79015da10 con 0x7ff79015d4b0 2024-01-15T18:03:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.302+0000 7ff787400640 1 -- 172.21.15.26:0/3265726195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800dd50 con 0x7ff79015d4b0 2024-01-15T18:03:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.302+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7900cad50 con 0x7ff79015d4b0 2024-01-15T18:03:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.303+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff790067450 con 0x7ff79015d4b0 2024-01-15T18:03:23.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.304+0000 7ff787400640 1 -- 172.21.15.26:0/3265726195 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7ff78801e3b0 con 0x7ff79015d4b0 2024-01-15T18:03:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.305+0000 7ff787400640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff76806bbf0 0x7ff76806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.305+0000 7ff787400640 1 -- 172.21.15.26:0/3265726195 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4913+0+0 (secure 0 0 0) 0x7ff788089070 con 0x7ff79015d4b0 2024-01-15T18:03:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.305+0000 7ff78f400640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff76806bbf0 0x7ff76806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.307+0000 7ff78f400640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff76806bbf0 0x7ff76806e0b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff780006fd0 tx=0x7ff780008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:23.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.310+0000 7ff787400640 1 -- 172.21.15.26:0/3265726195 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff788055740 con 0x7ff79015d4b0 2024-01-15T18:03:23.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.503+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7900b2340 con 0x7ff79015d4b0 2024-01-15T18:03:23.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.504+0000 7ff787400640 1 -- 172.21.15.26:0/3265726195 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff78801a0a0 con 0x7ff79015d4b0 2024-01-15T18:03:23.509 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:03:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff76806bbf0 msgr2=0x7ff76806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff76806bbf0 0x7ff76806e0b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff780006fd0 tx=0x7ff780008040 comp rx=0 tx=0).stop 2024-01-15T18:03:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79015d4b0 msgr2=0x7ff7901f2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79015d4b0 0x7ff7901f2d60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7ff78800cef0 tx=0x7ff78800a710 comp rx=0 tx=0).stop 2024-01-15T18:03:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 shutdown_connections 2024-01-15T18:03:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff7900cf9a0 0x7ff7900cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff76806bbf0 0x7ff76806e0b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff79015d4b0 0x7ff7901f2d60 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 --2- 172.21.15.26:0/3265726195 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff7900cddf0 0x7ff79015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.507+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 >> 172.21.15.26:0/3265726195 conn(0x7ff7900d3000 msgr2=0x7ff79006bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.508+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 shutdown_connections 2024-01-15T18:03:23.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.508+0000 7ff794c00640 1 -- 172.21.15.26:0/3265726195 wait complete. 2024-01-15T18:03:23.532 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:03:23.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.749+0000 7fb731a00640 1 Processor -- start 2024-01-15T18:03:23.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.749+0000 7fb731a00640 1 -- start start 2024-01-15T18:03:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.750+0000 7fb731a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c15b2a0 0x7fb72c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.750+0000 7fb731a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb72c0ce070 0x7fb72c164f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.750+0000 7fb731a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72c0d00a0 con 0x7fb72c0cfaf0 2024-01-15T18:03:23.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.750+0000 7fb731a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72c0b4010 con 0x7fb72c0ce070 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.750+0000 7fb731a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0b42f0 con 0x7fb72c15b2a0 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.751+0000 7fb72a000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb72c0cfaf0 0x7fb72c0cd960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45326/0 (socket says 172.21.15.26:45326) 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.751+0000 7fb72a000640 1 -- 172.21.15.26:0/509941171 learned_addr learned my addr 172.21.15.26:0/509941171 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.751+0000 7fb72aa00640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c15b2a0 0x7fb72c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.751+0000 7fb72b400640 1 --2- 172.21.15.26:0/509941171 >> v2:172.21.15.124:3300/0 conn(0x7fb72c0ce070 0x7fb72c164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.751+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724686047 0 0) 0x7fb72c0d00a0 con 0x7fb72c0cfaf0 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.751+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb70c003520 con 0x7fb72c0cfaf0 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2097343259 0 0) 0x7fb70c003520 con 0x7fb72c0cfaf0 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c15b2a0 msgr2=0x7fb72c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c15b2a0 0x7fb72c0cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 >> v2:172.21.15.124:3300/0 conn(0x7fb72c0ce070 msgr2=0x7fb72c164f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 --2- 172.21.15.26:0/509941171 >> v2:172.21.15.124:3300/0 conn(0x7fb72c0ce070 0x7fb72c164f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c0b45d0 con 0x7fb72c0cfaf0 2024-01-15T18:03:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb714001670 con 0x7fb72c0cfaf0 2024-01-15T18:03:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 >> v1:172.21.15.26:6789/0 conn(0x7fb72c0cfaf0 legacy=0x7fb72c0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c15b2a0 0x7fb70c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb70c003ef0 0x7fb70c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb70c004d00 0x7fb70c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0b4010 con 0x7fb70c003ef0 2024-01-15T18:03:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0b42f0 con 0x7fb70c004d00 2024-01-15T18:03:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c00b980 con 0x7fb72c15b2a0 2024-01-15T18:03:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb72a000640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb70c003ef0 0x7fb70c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb72aa00640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c15b2a0 0x7fb70c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.752+0000 7fb72b400640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb70c004d00 0x7fb70c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.753+0000 7fb72a000640 1 -- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c15b2a0 msgr2=0x7fb70c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.753+0000 7fb72a000640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c15b2a0 0x7fb70c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.753+0000 7fb72a000640 1 -- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb70c004d00 msgr2=0x7fb70c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.753+0000 7fb72a000640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb70c004d00 0x7fb70c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.753+0000 7fb72a000640 1 -- 172.21.15.26:0/509941171 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c003520 con 0x7fb70c003ef0 2024-01-15T18:03:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.754+0000 7fb72a000640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb70c003ef0 0x7fb70c0047c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb71400c240 tx=0x7fb70c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece2beca01304983 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.754+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71402cae0 con 0x7fb70c003ef0 2024-01-15T18:03:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.754+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71402cc40 con 0x7fb70c003ef0 2024-01-15T18:03:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.754+0000 7fb729600640 1 -- 172.21.15.26:0/509941171 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb714005af0 con 0x7fb70c003ef0 2024-01-15T18:03:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.755+0000 7fb731a00640 1 -- 172.21.15.26:0/509941171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb70c003ef0 msgr2=0x7fb70c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.755+0000 7fb731a00640 1 --2- 172.21.15.26:0/509941171 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb70c003ef0 0x7fb70c0047c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb71400c240 tx=0x7fb70c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.755+0000 7fb72b400640 1 -- 172.21.15.26:0/509941171 reap_dead start 2024-01-15T18:03:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.755+0000 7fb731a00640 1 -- 172.21.15.26:0/509941171 shutdown_connections 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.755+0000 7fb731a00640 1 -- 172.21.15.26:0/509941171 >> 172.21.15.26:0/509941171 conn(0x7fb72c0d1020 msgr2=0x7fb72c0d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.755+0000 7fb731a00640 1 -- 172.21.15.26:0/509941171 shutdown_connections 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.755+0000 7fb731a00640 1 -- 172.21.15.26:0/509941171 wait complete. 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.757+0000 7fb731a00640 1 Processor -- start 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.757+0000 7fb731a00640 1 -- start start 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb731a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb70c003ef0 0x7fb72c151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb731a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0c8c20 0x7fb72c06c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb731a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb72c0ca8c0 0x7fb72c0cad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb731a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c9da0 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb731a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0cb2b0 con 0x7fb70c003ef0 2024-01-15T18:03:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb731a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c1518d0 con 0x7fb72c0c8c20 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb72b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb72c0ca8c0 0x7fb72c0cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb72b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb72c0ca8c0 0x7fb72c0cad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46174/0 (socket says 172.21.15.26:46174) 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb72b400640 1 -- 172.21.15.26:0/2701350431 learned_addr learned my addr 172.21.15.26:0/2701350431 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb72aa00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb70c003ef0 0x7fb72c151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb72a000640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0c8c20 0x7fb72c06c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb72b400640 1 -- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0c8c20 msgr2=0x7fb72c06c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb72b400640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0c8c20 0x7fb72c06c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb72b400640 1 -- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb70c003ef0 msgr2=0x7fb72c151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.758+0000 7fb72b400640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb70c003ef0 0x7fb72c151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.759+0000 7fb72b400640 1 -- 172.21.15.26:0/2701350431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c0c9120 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.759+0000 7fb72b400640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb72c0ca8c0 0x7fb72c0cad70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0718f0 tx=0x7fb724011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.759+0000 7fb722a00640 1 -- 172.21.15.26:0/2701350431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724004500 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.759+0000 7fb722a00640 1 -- 172.21.15.26:0/2701350431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb724003610 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.759+0000 7fb722a00640 1 -- 172.21.15.26:0/2701350431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240039c0 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.759+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72c0c9400 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.759+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c0c99c0 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.760+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c0c9ee0 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.761+0000 7fb722a00640 1 -- 172.21.15.26:0/2701350431 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fb724003b20 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.763+0000 7fb722a00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb70406bbf0 0x7fb70406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.763+0000 7fb722a00640 1 -- 172.21.15.26:0/2701350431 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fb724088220 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.764+0000 7fb72aa00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb70406bbf0 0x7fb70406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.764+0000 7fb72aa00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb70406bbf0 0x7fb70406e0b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb71c012350 tx=0x7fb71c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.768+0000 7fb722a00640 1 -- 172.21.15.26:0/2701350431 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb72c0c9ee0 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.966+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb72c0c9ee0 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.967+0000 7fb722a00640 1 -- 172.21.15.26:0/2701350431 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v68) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb72c0c9ee0 con 0x7fb72c0ca8c0 2024-01-15T18:03:23.972 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:23.972 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:03:23.972 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:23.972 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-15T18:03:23.972 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-15T18:03:23.973 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:23.973 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:03:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb70406bbf0 msgr2=0x7fb70406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb70406bbf0 0x7fb70406e0b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb71c012350 tx=0x7fb71c00b040 comp rx=0 tx=0).stop 2024-01-15T18:03:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb72c0ca8c0 msgr2=0x7fb72c0cad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb72c0ca8c0 0x7fb72c0cad70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0718f0 tx=0x7fb724011920 comp rx=0 tx=0).stop 2024-01-15T18:03:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 shutdown_connections 2024-01-15T18:03:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb72c0c8c20 0x7fb72c06c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb70406bbf0 0x7fb70406e0b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb72c0ca8c0 0x7fb72c0cad70 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 --2- 172.21.15.26:0/2701350431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb70c003ef0 0x7fb72c151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.970+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 >> 172.21.15.26:0/2701350431 conn(0x7fb72c0d1020 msgr2=0x7fb72c15a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.971+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 shutdown_connections 2024-01-15T18:03:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:23.971+0000 7fb731a00640 1 -- 172.21.15.26:0/2701350431 wait complete. 2024-01-15T18:03:23.996 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:03:23.996 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:03:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.224+0000 7f7ac2a00640 1 Processor -- start 2024-01-15T18:03:24.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.225+0000 7f7ac2a00640 1 -- start start 2024-01-15T18:03:24.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac2a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7abc160920 0x7f7abc0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac2a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7abc0cf9f0 0x7f7abc0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac2a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 0x7f7abc1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac2a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7abc0d03e0 con 0x7f7abc160920 2024-01-15T18:03:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac2a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7abc0d0a10 con 0x7f7abc0cf9f0 2024-01-15T18:03:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac2a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc0cf080 con 0x7f7abc161010 2024-01-15T18:03:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac1600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7abc160920 0x7f7abc0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac1600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7abc160920 0x7f7abc0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46188/0 (socket says 172.21.15.26:46188) 2024-01-15T18:03:24.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac1600640 1 -- 172.21.15.26:0/352136313 learned_addr learned my addr 172.21.15.26:0/352136313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac0c00640 1 --2- 172.21.15.26:0/352136313 >> v2:172.21.15.124:3300/0 conn(0x7f7abc0cf9f0 0x7f7abc0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac2000640 1 --2- 172.21.15.26:0/352136313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 0x7f7abc1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac1600640 1 -- 172.21.15.26:0/352136313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 msgr2=0x7f7abc1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac1600640 1 --2- 172.21.15.26:0/352136313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 0x7f7abc1614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac1600640 1 -- 172.21.15.26:0/352136313 >> v2:172.21.15.124:3300/0 conn(0x7f7abc0cf9f0 msgr2=0x7f7abc0cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.227+0000 7f7ac1600640 1 --2- 172.21.15.26:0/352136313 >> v2:172.21.15.124:3300/0 conn(0x7f7abc0cf9f0 0x7f7abc0cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.228+0000 7f7ac1600640 1 -- 172.21.15.26:0/352136313 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7abc0c6be0 con 0x7f7abc160920 2024-01-15T18:03:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.229+0000 7f7ac1600640 1 --2- 172.21.15.26:0/352136313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 0x7f7abc0b3f30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7abc08c1f0 tx=0x7f7aac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78cc66d784172b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.229+0000 7f7abbe00640 1 -- 172.21.15.26:0/352136313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac031870 con 0x7f7abc160920 2024-01-15T18:03:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.229+0000 7f7abbe00640 1 -- 172.21.15.26:0/352136313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac0319d0 con 0x7f7abc160920 2024-01-15T18:03:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.229+0000 7f7abbe00640 1 -- 172.21.15.26:0/352136313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac031d60 con 0x7f7abc160920 2024-01-15T18:03:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.230+0000 7f7ac2a00640 1 -- 172.21.15.26:0/352136313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 msgr2=0x7f7abc0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:24.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.230+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/352136313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 0x7f7abc0b3f30 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7abc08c1f0 tx=0x7f7aac02b910 comp rx=0 tx=0).stop 2024-01-15T18:03:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.230+0000 7f7ac2a00640 1 -- 172.21.15.26:0/352136313 shutdown_connections 2024-01-15T18:03:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.230+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/352136313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 0x7f7abc1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.230+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/352136313 >> v2:172.21.15.124:3300/0 conn(0x7f7abc0cf9f0 0x7f7abc0cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.230+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/352136313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 0x7f7abc0b3f30 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.230+0000 7f7ac2a00640 1 -- 172.21.15.26:0/352136313 >> 172.21.15.26:0/352136313 conn(0x7f7abc0d3090 msgr2=0x7f7abc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.230+0000 7f7ac2a00640 1 -- 172.21.15.26:0/352136313 shutdown_connections 2024-01-15T18:03:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.230+0000 7f7ac2a00640 1 -- 172.21.15.26:0/352136313 wait complete. 2024-01-15T18:03:24.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.232+0000 7f7ac2a00640 1 Processor -- start 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.232+0000 7f7ac2a00640 1 -- start start 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.233+0000 7f7ac2a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7abc0cf9f0 0x7f7abc14cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 0x7f7abc14c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac2a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 0x7f7abc0b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac2a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc15d7e0 con 0x7f7abc160920 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac2a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc15cda0 con 0x7f7abc0cf9f0 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac2a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc15c800 con 0x7f7abc161010 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 0x7f7abc14c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 0x7f7abc14c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46198/0 (socket says 172.21.15.26:46198) 2024-01-15T18:03:24.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac0c00640 1 -- 172.21.15.26:0/3093424788 learned_addr learned my addr 172.21.15.26:0/3093424788 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac1600640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7abc0cf9f0 0x7f7abc14cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac0c00640 1 -- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 msgr2=0x7f7abc0b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.234+0000 7f7ac2000640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 0x7f7abc0b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.235+0000 7f7ac0c00640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 0x7f7abc0b7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.235+0000 7f7ac0c00640 1 -- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7abc0cf9f0 msgr2=0x7f7abc14cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.235+0000 7f7ac0c00640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7abc0cf9f0 0x7f7abc14cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.235+0000 7f7ac0c00640 1 -- 172.21.15.26:0/3093424788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac0094d0 con 0x7f7abc160920 2024-01-15T18:03:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.235+0000 7f7ac1600640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7abc0cf9f0 0x7f7abc14cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:03:24.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.235+0000 7f7ac0c00640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 0x7f7abc14c450 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400d800 tx=0x7f7aa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.237+0000 7f7ab9600640 1 -- 172.21.15.26:0/3093424788 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40096a0 con 0x7f7abc160920 2024-01-15T18:03:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.237+0000 7f7ab9600640 1 -- 172.21.15.26:0/3093424788 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa4009800 con 0x7f7abc160920 2024-01-15T18:03:24.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.237+0000 7f7ab9600640 1 -- 172.21.15.26:0/3093424788 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4009990 con 0x7f7abc160920 2024-01-15T18:03:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.237+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc15cac0 con 0x7f7abc160920 2024-01-15T18:03:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.237+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc15d500 con 0x7f7abc160920 2024-01-15T18:03:24.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.238+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7abc15d080 con 0x7f7abc160920 2024-01-15T18:03:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.239+0000 7f7ab9600640 1 -- 172.21.15.26:0/3093424788 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f7aa4009af0 con 0x7f7abc160920 2024-01-15T18:03:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.240+0000 7f7ab9600640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a9406bbf0 0x7f7a9406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:24.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.240+0000 7f7ab9600640 1 -- 172.21.15.26:0/3093424788 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f7aa4088aa0 con 0x7f7abc160920 2024-01-15T18:03:24.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.241+0000 7f7ac1600640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a9406bbf0 0x7f7a9406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:24.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.242+0000 7f7ac1600640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a9406bbf0 0x7f7a9406e0b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7aac036000 tx=0x7f7aac03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:24.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.244+0000 7f7ab9600640 1 -- 172.21.15.26:0/3093424788 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7abc15d080 con 0x7f7abc160920 2024-01-15T18:03:24.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:24.446+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7abc15d080 con 0x7f7abc160920 2024-01-15T18:03:25.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.257+0000 7f7ab9600640 1 -- 172.21.15.26:0/3093424788 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v69) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7abc15d080 con 0x7f7abc160920 2024-01-15T18:03:25.264 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-15T18:03:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a9406bbf0 msgr2=0x7f7a9406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a9406bbf0 0x7f7a9406e0b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7aac036000 tx=0x7f7aac03c040 comp rx=0 tx=0).stop 2024-01-15T18:03:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 msgr2=0x7f7abc14c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 0x7f7abc14c450 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400d800 tx=0x7f7aa400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 shutdown_connections 2024-01-15T18:03:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7abc161010 0x7f7abc0b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a9406bbf0 0x7f7a9406e0b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7abc160920 0x7f7abc14c450 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 --2- 172.21.15.26:0/3093424788 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7abc0cf9f0 0x7f7abc14cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.262+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 >> 172.21.15.26:0/3093424788 conn(0x7f7abc0d3090 msgr2=0x7f7abc15b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.263+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 shutdown_connections 2024-01-15T18:03:25.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.263+0000 7f7ac2a00640 1 -- 172.21.15.26:0/3093424788 wait complete. 2024-01-15T18:03:25.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:03:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.510+0000 7f0ea7e00640 1 Processor -- start 2024-01-15T18:03:25.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.510+0000 7f0ea7e00640 1 -- start start 2024-01-15T18:03:25.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.512+0000 7f0ea7e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0ea8160920 0x7f0ea80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.512+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cfea0 0x7f0ea8161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.512+0000 7f0ea7e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea80d0a10 con 0x7f0ea80cf9f0 2024-01-15T18:03:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.512+0000 7f0ea7e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea80d0330 con 0x7f0ea8160920 2024-01-15T18:03:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.512+0000 7f0ea7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea80d1f50 con 0x7f0ea80cfea0 2024-01-15T18:03:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.513+0000 7f0ea7400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cfea0 0x7f0ea8161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.513+0000 7f0ea7400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cfea0 0x7f0ea8161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:41554/0 (socket says 172.21.15.26:41554) 2024-01-15T18:03:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.513+0000 7f0ea7400640 1 -- 172.21.15.26:0/3629094719 learned_addr learned my addr 172.21.15.26:0/3629094719 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:25.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.513+0000 7f0ea6a00640 1 --2- 172.21.15.26:0/3629094719 >> v2:172.21.15.124:3300/0 conn(0x7f0ea8160920 0x7f0ea80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:25.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.514+0000 7f0ea7400640 1 -- 172.21.15.26:0/3629094719 >> v2:172.21.15.124:3300/0 conn(0x7f0ea8160920 msgr2=0x7f0ea80b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:25.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.514+0000 7f0ea7400640 1 --2- 172.21.15.26:0/3629094719 >> v2:172.21.15.124:3300/0 conn(0x7f0ea8160920 0x7f0ea80b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.514+0000 7f0ea7400640 1 -- 172.21.15.26:0/3629094719 >> v1:172.21.15.124:6789/0 conn(0x7f0ea80cf9f0 legacy=0x7f0ea8161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:25.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.514+0000 7f0ea7400640 1 -- 172.21.15.26:0/3629094719 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80cf080 con 0x7f0ea80cfea0 2024-01-15T18:03:25.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.515+0000 7f0ea7400640 1 --2- 172.21.15.26:0/3629094719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cfea0 0x7f0ea8161720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009e50 tx=0x7f0ea002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ce818505f88fa8b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.515+0000 7f0ea6000640 1 -- 172.21.15.26:0/3629094719 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea002ca80 con 0x7f0ea80cfea0 2024-01-15T18:03:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.516+0000 7f0ea6000640 1 -- 172.21.15.26:0/3629094719 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea002cbe0 con 0x7f0ea80cfea0 2024-01-15T18:03:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.516+0000 7f0ea6000640 1 -- 172.21.15.26:0/3629094719 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00343f0 con 0x7f0ea80cfea0 2024-01-15T18:03:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.516+0000 7f0ea7e00640 1 -- 172.21.15.26:0/3629094719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cfea0 msgr2=0x7f0ea8161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.516+0000 7f0ea7e00640 1 --2- 172.21.15.26:0/3629094719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cfea0 0x7f0ea8161720 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009e50 tx=0x7f0ea002bee0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.516+0000 7f0ea7e00640 1 -- 172.21.15.26:0/3629094719 shutdown_connections 2024-01-15T18:03:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.516+0000 7f0ea7e00640 1 --2- 172.21.15.26:0/3629094719 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cfea0 0x7f0ea8161720 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.516+0000 7f0ea7e00640 1 --2- 172.21.15.26:0/3629094719 >> v2:172.21.15.124:3300/0 conn(0x7f0ea8160920 0x7f0ea80b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.516+0000 7f0ea7e00640 1 -- 172.21.15.26:0/3629094719 >> 172.21.15.26:0/3629094719 conn(0x7f0ea80d3090 msgr2=0x7f0ea80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.517+0000 7f0ea7e00640 1 -- 172.21.15.26:0/3629094719 shutdown_connections 2024-01-15T18:03:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.517+0000 7f0ea7e00640 1 -- 172.21.15.26:0/3629094719 wait complete. 2024-01-15T18:03:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.519+0000 7f0ea7e00640 1 Processor -- start 2024-01-15T18:03:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.519+0000 7f0ea7e00640 1 -- start start 2024-01-15T18:03:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.520+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cf9f0 0x7f0ea81f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.520+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ea80cfea0 0x7f0ea80b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.520+0000 7f0ea7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8160920 0x7f0ea81f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.520+0000 7f0ea7e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81f4110 con 0x7f0ea8160920 2024-01-15T18:03:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.520+0000 7f0ea7e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81f35f0 con 0x7f0ea80cfea0 2024-01-15T18:03:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.520+0000 7f0ea7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea81f3870 con 0x7f0ea80cf9f0 2024-01-15T18:03:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0ea7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8160920 0x7f0ea81f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0ea7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8160920 0x7f0ea81f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46208/0 (socket says 172.21.15.26:46208) 2024-01-15T18:03:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0ea7400640 1 -- 172.21.15.26:0/1868816187 learned_addr learned my addr 172.21.15.26:0/1868816187 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0ea6a00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cf9f0 0x7f0ea81f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0e9fe00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ea80cfea0 0x7f0ea80b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:25.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0ea7400640 1 -- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cf9f0 msgr2=0x7f0ea81f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:25.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0ea7400640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cf9f0 0x7f0ea81f4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0ea7400640 1 -- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ea80cfea0 msgr2=0x7f0ea80b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:25.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0ea7400640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ea80cfea0 0x7f0ea80b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:25.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.521+0000 7f0ea7400640 1 -- 172.21.15.26:0/1868816187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0009b30 con 0x7f0ea8160920 2024-01-15T18:03:25.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.522+0000 7f0ea7400640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8160920 0x7f0ea81f30b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009e50 tx=0x7f0ea0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:25.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.522+0000 7f0e9ea00640 1 -- 172.21.15.26:0/1868816187 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0009510 con 0x7f0ea8160920 2024-01-15T18:03:25.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.522+0000 7f0e9ea00640 1 -- 172.21.15.26:0/1868816187 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea003e450 con 0x7f0ea8160920 2024-01-15T18:03:25.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.522+0000 7f0e9ea00640 1 -- 172.21.15.26:0/1868816187 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea003e800 con 0x7f0ea8160920 2024-01-15T18:03:25.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.522+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea81f3b50 con 0x7f0ea8160920 2024-01-15T18:03:25.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.522+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea80b5df0 con 0x7f0ea8160920 2024-01-15T18:03:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.523+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea8067450 con 0x7f0ea8160920 2024-01-15T18:03:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.525+0000 7f0e9ea00640 1 -- 172.21.15.26:0/1868816187 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f0ea002cd50 con 0x7f0ea8160920 2024-01-15T18:03:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.526+0000 7f0e9ea00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e8006bbc0 0x7f0e8006e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.526+0000 7f0e9ea00640 1 -- 172.21.15.26:0/1868816187 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f0ea00ab640 con 0x7f0ea8160920 2024-01-15T18:03:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.527+0000 7f0ea6a00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e8006bbc0 0x7f0e8006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.528+0000 7f0ea6a00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e8006bbc0 0x7f0e8006e080 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009ad0 tx=0x7f0e98008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:25.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.529+0000 7f0e9ea00640 1 -- 172.21.15.26:0/1868816187 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0ea0077f90 con 0x7f0ea8160920 2024-01-15T18:03:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:25.723+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0ea808c7e0 con 0x7f0ea8160920 2024-01-15T18:03:26.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.264+0000 7f0e9ea00640 1 -- 172.21.15.26:0/1868816187 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v70) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0ea0033e80 con 0x7f0ea8160920 2024-01-15T18:03:26.270 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-15T18:03:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.268+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e8006bbc0 msgr2=0x7f0e8006e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.268+0000 7f0ea7e00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e8006bbc0 0x7f0e8006e080 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009ad0 tx=0x7f0e98008040 comp rx=0 tx=0).stop 2024-01-15T18:03:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.268+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8160920 msgr2=0x7f0ea81f30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.268+0000 7f0ea7e00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8160920 0x7f0ea81f30b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009e50 tx=0x7f0ea0004a40 comp rx=0 tx=0).stop 2024-01-15T18:03:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.269+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 shutdown_connections 2024-01-15T18:03:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.269+0000 7f0ea7e00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e8006bbc0 0x7f0e8006e080 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.269+0000 7f0ea7e00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0ea8160920 0x7f0ea81f30b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.269+0000 7f0ea7e00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0ea80cfea0 0x7f0ea80b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:26.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.269+0000 7f0ea7e00640 1 --2- 172.21.15.26:0/1868816187 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0ea80cf9f0 0x7f0ea81f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.269+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 >> 172.21.15.26:0/1868816187 conn(0x7f0ea80d3090 msgr2=0x7f0ea815c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.269+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 shutdown_connections 2024-01-15T18:03:26.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.269+0000 7f0ea7e00640 1 -- 172.21.15.26:0/1868816187 wait complete. 2024-01-15T18:03:26.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:03:26.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.489+0000 7f6f00a00640 1 Processor -- start 2024-01-15T18:03:26.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.489+0000 7f6f00a00640 1 -- start start 2024-01-15T18:03:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.490+0000 7f6f00a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6ef80b3ae0 0x7f6ef815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.491+0000 7f6f00a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 0x7f6ef80cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.491+0000 7f6f00a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef8070c40 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.491+0000 7f6f00a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef8071450 con 0x7f6ef80b3ae0 2024-01-15T18:03:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.491+0000 7f6f00a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef80cfc60 con 0x7f6ef80cf290 2024-01-15T18:03:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.491+0000 7f6efd600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6ef80b3ae0 0x7f6ef815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:26.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.491+0000 7f6efea00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 0x7f6ef80cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.491+0000 7f6efd600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6ef80b3ae0 0x7f6ef815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:32902/0 (socket says 172.21.15.26:32902) 2024-01-15T18:03:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.491+0000 7f6efd600640 1 -- 172.21.15.26:0/3262181583 learned_addr learned my addr 172.21.15.26:0/3262181583 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.492+0000 7f6efd600640 1 -- 172.21.15.26:0/3262181583 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 msgr2=0x7f6ef80cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:26.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.492+0000 7f6efd600640 1 --2- 172.21.15.26:0/3262181583 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 0x7f6ef80cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.492+0000 7f6efd600640 1 -- 172.21.15.26:0/3262181583 >> v1:172.21.15.124:6789/0 conn(0x7f6ef80ae5a0 legacy=0x7f6ef806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.492+0000 7f6efd600640 1 -- 172.21.15.26:0/3262181583 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef806c470 con 0x7f6ef80b3ae0 2024-01-15T18:03:26.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.494+0000 7f6efd600640 1 --2- 172.21.15.26:0/3262181583 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ef80b3ae0 0x7f6ef815e430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009870 tx=0x7f6ee002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=591560d530bca031 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.494+0000 7f6efcc00640 1 -- 172.21.15.26:0/3262181583 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee002f830 con 0x7f6ef80b3ae0 2024-01-15T18:03:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.494+0000 7f6efcc00640 1 -- 172.21.15.26:0/3262181583 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee002f990 con 0x7f6ef80b3ae0 2024-01-15T18:03:26.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.494+0000 7f6efcc00640 1 -- 172.21.15.26:0/3262181583 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee002fd20 con 0x7f6ef80b3ae0 2024-01-15T18:03:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.495+0000 7f6f00a00640 1 -- 172.21.15.26:0/3262181583 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ef80b3ae0 msgr2=0x7f6ef815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.495+0000 7f6f00a00640 1 --2- 172.21.15.26:0/3262181583 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ef80b3ae0 0x7f6ef815e430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009870 tx=0x7f6ee002ba10 comp rx=0 tx=0).stop 2024-01-15T18:03:26.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.495+0000 7f6f00a00640 1 -- 172.21.15.26:0/3262181583 shutdown_connections 2024-01-15T18:03:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.495+0000 7f6f00a00640 1 --2- 172.21.15.26:0/3262181583 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 0x7f6ef80cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.495+0000 7f6f00a00640 1 --2- 172.21.15.26:0/3262181583 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ef80b3ae0 0x7f6ef815e430 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.495+0000 7f6f00a00640 1 -- 172.21.15.26:0/3262181583 >> 172.21.15.26:0/3262181583 conn(0x7f6ef806a800 msgr2=0x7f6ef806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.495+0000 7f6f00a00640 1 -- 172.21.15.26:0/3262181583 shutdown_connections 2024-01-15T18:03:26.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.495+0000 7f6f00a00640 1 -- 172.21.15.26:0/3262181583 wait complete. 2024-01-15T18:03:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.497+0000 7f6f00a00640 1 Processor -- start 2024-01-15T18:03:26.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.497+0000 7f6f00a00640 1 -- start start 2024-01-15T18:03:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.498+0000 7f6f00a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ef80ae5a0 0x7f6ef81514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.498+0000 7f6f00a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ef80b3ae0 0x7f6ef815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.498+0000 7f6f00a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 0x7f6ef815f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.498+0000 7f6f00a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef815cce0 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.498+0000 7f6f00a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef815c060 con 0x7f6ef80b3ae0 2024-01-15T18:03:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.498+0000 7f6f00a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef815c340 con 0x7f6ef80cf290 2024-01-15T18:03:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efe000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ef80ae5a0 0x7f6ef81514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:26.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efe000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ef80ae5a0 0x7f6ef81514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46216/0 (socket says 172.21.15.26:46216) 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efe000640 1 -- 172.21.15.26:0/30396918 learned_addr learned my addr 172.21.15.26:0/30396918 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efea00640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 0x7f6ef815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efd600640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ef80b3ae0 0x7f6ef815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efe000640 1 -- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 msgr2=0x7f6ef815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efe000640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 0x7f6ef815f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efe000640 1 -- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ef80b3ae0 msgr2=0x7f6ef815b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efe000640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ef80b3ae0 0x7f6ef815b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.499+0000 7f6efe000640 1 -- 172.21.15.26:0/30396918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee00094d0 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.500+0000 7f6efe000640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ef80ae5a0 0x7f6ef81514c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c400 tx=0x7f6ee800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:26.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.500+0000 7f6ef6000640 1 -- 172.21.15.26:0/30396918 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee80052d0 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.500+0000 7f6ef6000640 1 -- 172.21.15.26:0/30396918 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee8005450 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.500+0000 7f6ef6000640 1 -- 172.21.15.26:0/30396918 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8007260 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.500+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef815c620 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.500+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef81506f0 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.501+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef815ce20 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.502+0000 7f6ef6000640 1 -- 172.21.15.26:0/30396918 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f6ee80073c0 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.504+0000 7f6ef6000640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ed006bbc0 0x7f6ed006e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.504+0000 7f6ef6000640 1 -- 172.21.15.26:0/30396918 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5665+0+0 (secure 0 0 0) 0x7f6ee80878b0 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.504+0000 7f6efd600640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ed006bbc0 0x7f6ed006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:26.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.505+0000 7f6efd600640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ed006bbc0 0x7f6ed006e080 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0036000 tx=0x7f6ee0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:26.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.509+0000 7f6ef6000640 1 -- 172.21.15.26:0/30396918 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ef815ce20 con 0x7f6ef80ae5a0 2024-01-15T18:03:26.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:26.742+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6ef815ce20 con 0x7f6ef80ae5a0 2024-01-15T18:03:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.290+0000 7f6ef6000640 1 -- 172.21.15.26:0/30396918 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6ee8054170 con 0x7f6ef80ae5a0 2024-01-15T18:03:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 10 and data pool 11 2024-01-15T18:03:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ed006bbc0 msgr2=0x7f6ed006e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ed006bbc0 0x7f6ed006e080 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0036000 tx=0x7f6ee0008030 comp rx=0 tx=0).stop 2024-01-15T18:03:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ef80ae5a0 msgr2=0x7f6ef81514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ef80ae5a0 0x7f6ef81514c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6ee800c400 tx=0x7f6ee800c8c0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 shutdown_connections 2024-01-15T18:03:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ef80cf290 0x7f6ef815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ed006bbc0 0x7f6ed006e080 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ef80b3ae0 0x7f6ef815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 --2- 172.21.15.26:0/30396918 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ef80ae5a0 0x7f6ef81514c0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.296+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 >> 172.21.15.26:0/30396918 conn(0x7f6ef806a800 msgr2=0x7f6ef815d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.297+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 shutdown_connections 2024-01-15T18:03:27.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.298+0000 7f6f00a00640 1 -- 172.21.15.26:0/30396918 wait complete. 2024-01-15T18:03:27.347 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:03:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.518+0000 7ff474000640 1 Processor -- start 2024-01-15T18:03:27.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.518+0000 7ff474000640 1 -- start start 2024-01-15T18:03:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.520+0000 7ff474000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff46c13df80 0x7ff46c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.520+0000 7ff474000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46c06c470 con 0x7ff46c0cf9a0 2024-01-15T18:03:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.520+0000 7ff474000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff46c06c750 con 0x7ff46c13e360 2024-01-15T18:03:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.520+0000 7ff474000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff46c06ca30 con 0x7ff46c13df80 2024-01-15T18:03:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.520+0000 7ff471600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff46c13df80 0x7ff46c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.520+0000 7ff472000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff46c0cf9a0 0x7ff46c0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45338/0 (socket says 172.21.15.26:45338) 2024-01-15T18:03:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.520+0000 7ff472000640 1 -- 172.21.15.26:0/1815828353 learned_addr learned my addr 172.21.15.26:0/1815828353 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.521+0000 7ff471600640 1 -- 172.21.15.26:0/1815828353 >> v1:172.21.15.124:6789/0 conn(0x7ff46c13e360 legacy=0x7ff46c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.521+0000 7ff471600640 1 -- 172.21.15.26:0/1815828353 >> v1:172.21.15.26:6789/0 conn(0x7ff46c0cf9a0 legacy=0x7ff46c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.521+0000 7ff471600640 1 -- 172.21.15.26:0/1815828353 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c06cd10 con 0x7ff46c13df80 2024-01-15T18:03:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.522+0000 7ff471600640 1 --2- 172.21.15.26:0/1815828353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13df80 0x7ff46c15bee0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0097a0 tx=0x7ff45c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afa9296a57318fb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.522+0000 7ff46be00640 1 -- 172.21.15.26:0/1815828353 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c031830 con 0x7ff46c13df80 2024-01-15T18:03:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.523+0000 7ff46be00640 1 -- 172.21.15.26:0/1815828353 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c031990 con 0x7ff46c13df80 2024-01-15T18:03:27.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.523+0000 7ff46be00640 1 -- 172.21.15.26:0/1815828353 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c031d20 con 0x7ff46c13df80 2024-01-15T18:03:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.523+0000 7ff474000640 1 -- 172.21.15.26:0/1815828353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13df80 msgr2=0x7ff46c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.523+0000 7ff474000640 1 --2- 172.21.15.26:0/1815828353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13df80 0x7ff46c15bee0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0097a0 tx=0x7ff45c02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.523+0000 7ff474000640 1 -- 172.21.15.26:0/1815828353 shutdown_connections 2024-01-15T18:03:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.523+0000 7ff474000640 1 --2- 172.21.15.26:0/1815828353 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13df80 0x7ff46c15bee0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.523+0000 7ff474000640 1 -- 172.21.15.26:0/1815828353 >> 172.21.15.26:0/1815828353 conn(0x7ff46c06a800 msgr2=0x7ff46c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.524+0000 7ff474000640 1 -- 172.21.15.26:0/1815828353 shutdown_connections 2024-01-15T18:03:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.524+0000 7ff474000640 1 -- 172.21.15.26:0/1815828353 wait complete. 2024-01-15T18:03:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.526+0000 7ff474000640 1 Processor -- start 2024-01-15T18:03:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.526+0000 7ff474000640 1 -- start start 2024-01-15T18:03:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.527+0000 7ff474000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c0cf9a0 0x7ff46c13a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.527+0000 7ff474000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c13df80 0x7ff46c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.527+0000 7ff474000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13e360 0x7ff46c0b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.527+0000 7ff474000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c13b770 con 0x7ff46c13e360 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.527+0000 7ff474000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c13c5d0 con 0x7ff46c0cf9a0 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.527+0000 7ff474000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c13b9c0 con 0x7ff46c13df80 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff472000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13e360 0x7ff46c0b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff472000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13e360 0x7ff46c0b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46230/0 (socket says 172.21.15.26:46230) 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff472000640 1 -- 172.21.15.26:0/1973523794 learned_addr learned my addr 172.21.15.26:0/1973523794 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff471600640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c0cf9a0 0x7ff46c13a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff472000640 1 -- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c13df80 msgr2=0x7ff46c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff472000640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c13df80 0x7ff46c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff472000640 1 -- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c0cf9a0 msgr2=0x7ff46c13a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff472000640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c0cf9a0 0x7ff46c13a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff472000640 1 -- 172.21.15.26:0/1973523794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c009400 con 0x7ff46c13e360 2024-01-15T18:03:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff472000640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13e360 0x7ff46c0b2cf0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff46400cad0 tx=0x7ff46400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.528+0000 7ff469600640 1 -- 172.21.15.26:0/1973523794 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400ac30 con 0x7ff46c13e360 2024-01-15T18:03:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.529+0000 7ff469600640 1 -- 172.21.15.26:0/1973523794 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46400ad90 con 0x7ff46c13e360 2024-01-15T18:03:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.529+0000 7ff469600640 1 -- 172.21.15.26:0/1973523794 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff464012600 con 0x7ff46c13e360 2024-01-15T18:03:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.529+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c13bca0 con 0x7ff46c13e360 2024-01-15T18:03:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.529+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c13c260 con 0x7ff46c13e360 2024-01-15T18:03:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.530+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46c13c760 con 0x7ff46c13e360 2024-01-15T18:03:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.531+0000 7ff469600640 1 -- 172.21.15.26:0/1973523794 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7ff464007070 con 0x7ff46c13e360 2024-01-15T18:03:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.532+0000 7ff469600640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff44406bbc0 0x7ff44406e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.532+0000 7ff469600640 1 -- 172.21.15.26:0/1973523794 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff464088220 con 0x7ff46c13e360 2024-01-15T18:03:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.532+0000 7ff471600640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff44406bbc0 0x7ff44406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.533+0000 7ff471600640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff44406bbc0 0x7ff44406e080 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff45c036000 tx=0x7ff45c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.536+0000 7ff469600640 1 -- 172.21.15.26:0/1973523794 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff464054b90 con 0x7ff46c13e360 2024-01-15T18:03:27.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-01-15T18:03:27.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-01-15T18:03:27.636 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-01-15T18:03:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.732+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff46c0675d0 con 0x7ff46c13e360 2024-01-15T18:03:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.735+0000 7ff469600640 1 -- 172.21.15.26:0/1973523794 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16644 (secure 0 0 0) 0x7ff464017020 con 0x7ff46c13e360 2024-01-15T18:03:27.741 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:27.742 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:03:26.744515+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:03:24.501738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-15T18:03:25.778629+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:03:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff44406bbc0 msgr2=0x7ff44406e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff44406bbc0 0x7ff44406e080 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff45c036000 tx=0x7ff45c008030 comp rx=0 tx=0).stop 2024-01-15T18:03:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13e360 msgr2=0x7ff46c0b2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13e360 0x7ff46c0b2cf0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff46400cad0 tx=0x7ff46400cf90 comp rx=0 tx=0).stop 2024-01-15T18:03:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 shutdown_connections 2024-01-15T18:03:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff46c13df80 0x7ff46c0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff44406bbc0 0x7ff44406e080 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff46c13e360 0x7ff46c0b2cf0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 --2- 172.21.15.26:0/1973523794 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff46c0cf9a0 0x7ff46c13a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 >> 172.21.15.26:0/1973523794 conn(0x7ff46c06a800 msgr2=0x7ff46c153b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:27.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.738+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 shutdown_connections 2024-01-15T18:03:27.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.739+0000 7ff474000640 1 -- 172.21.15.26:0/1973523794 wait complete. 2024-01-15T18:03:27.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:03:27.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.980+0000 7fd0bc600640 1 Processor -- start 2024-01-15T18:03:27.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.980+0000 7fd0bc600640 1 -- start start 2024-01-15T18:03:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.981+0000 7fd0bc600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd0b413e360 0x7fd0b40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0bc600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd0b415f9c0 0x7fd0b40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0bc600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b415c5f0 con 0x7fd0b413df80 2024-01-15T18:03:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0bc600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b406c470 con 0x7fd0b415f9c0 2024-01-15T18:03:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0bc600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b406c750 con 0x7fd0b413e360 2024-01-15T18:03:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0ba600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd0b415f9c0 0x7fd0b40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0ba600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd0b415f9c0 0x7fd0b40cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46232/0 (socket says 172.21.15.26:46232) 2024-01-15T18:03:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0ba600640 1 -- 172.21.15.26:0/4158866562 learned_addr learned my addr 172.21.15.26:0/4158866562 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0ba600640 1 -- 172.21.15.26:0/4158866562 >> v2:172.21.15.124:3300/0 conn(0x7fd0b413e360 msgr2=0x7fd0b40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:27.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0b9200640 1 --2- 172.21.15.26:0/4158866562 >> v2:172.21.15.124:3300/0 conn(0x7fd0b413e360 0x7fd0b40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0ba600640 1 --2- 172.21.15.26:0/4158866562 >> v2:172.21.15.124:3300/0 conn(0x7fd0b413e360 0x7fd0b40cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.982+0000 7fd0ba600640 1 -- 172.21.15.26:0/4158866562 >> v1:172.21.15.124:6789/0 conn(0x7fd0b413df80 legacy=0x7fd0b415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.983+0000 7fd0ba600640 1 -- 172.21.15.26:0/4158866562 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b406ca30 con 0x7fd0b415f9c0 2024-01-15T18:03:27.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.984+0000 7fd0ba600640 1 --2- 172.21.15.26:0/4158866562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b415f9c0 0x7fd0b40cf7d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fd0b406ded0 tx=0x7fd0ac02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a706ed06d49380 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:27.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.984+0000 7fd0b3e00640 1 -- 172.21.15.26:0/4158866562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac02cb20 con 0x7fd0b415f9c0 2024-01-15T18:03:27.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.984+0000 7fd0b3e00640 1 -- 172.21.15.26:0/4158866562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac02cc80 con 0x7fd0b415f9c0 2024-01-15T18:03:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.984+0000 7fd0b3e00640 1 -- 172.21.15.26:0/4158866562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0343f0 con 0x7fd0b415f9c0 2024-01-15T18:03:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.985+0000 7fd0bc600640 1 -- 172.21.15.26:0/4158866562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b415f9c0 msgr2=0x7fd0b40cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.985+0000 7fd0bc600640 1 --2- 172.21.15.26:0/4158866562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b415f9c0 0x7fd0b40cf7d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fd0b406ded0 tx=0x7fd0ac02bf80 comp rx=0 tx=0).stop 2024-01-15T18:03:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.985+0000 7fd0bc600640 1 -- 172.21.15.26:0/4158866562 shutdown_connections 2024-01-15T18:03:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.985+0000 7fd0bc600640 1 --2- 172.21.15.26:0/4158866562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b415f9c0 0x7fd0b40cf7d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.985+0000 7fd0bc600640 1 --2- 172.21.15.26:0/4158866562 >> v2:172.21.15.124:3300/0 conn(0x7fd0b413e360 0x7fd0b40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.985+0000 7fd0bc600640 1 -- 172.21.15.26:0/4158866562 >> 172.21.15.26:0/4158866562 conn(0x7fd0b406a800 msgr2=0x7fd0b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.985+0000 7fd0bc600640 1 -- 172.21.15.26:0/4158866562 shutdown_connections 2024-01-15T18:03:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.985+0000 7fd0bc600640 1 -- 172.21.15.26:0/4158866562 wait complete. 2024-01-15T18:03:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.986+0000 7fd0bc600640 1 Processor -- start 2024-01-15T18:03:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.987+0000 7fd0bc600640 1 -- start start 2024-01-15T18:03:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0bc600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b413df80 0x7fd0b4071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0bc600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0b413e360 0x7fd0b414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0bc600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b415f9c0 0x7fd0b415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0bc600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b414dec0 con 0x7fd0b413df80 2024-01-15T18:03:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0bc600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4070de0 con 0x7fd0b415f9c0 2024-01-15T18:03:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0bc600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40710c0 con 0x7fd0b413e360 2024-01-15T18:03:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0b9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b413df80 0x7fd0b4071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0b9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b413df80 0x7fd0b4071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46234/0 (socket says 172.21.15.26:46234) 2024-01-15T18:03:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0b9c00640 1 -- 172.21.15.26:0/75053466 learned_addr learned my addr 172.21.15.26:0/75053466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0ba600640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b415f9c0 0x7fd0b415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0b9200640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0b413e360 0x7fd0b414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0b9c00640 1 -- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0b413e360 msgr2=0x7fd0b414d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0b9c00640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0b413e360 0x7fd0b414d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.988+0000 7fd0b9c00640 1 -- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b415f9c0 msgr2=0x7fd0b415cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.989+0000 7fd0b9c00640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b415f9c0 0x7fd0b415cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.989+0000 7fd0b9c00640 1 -- 172.21.15.26:0/75053466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac009b30 con 0x7fd0b413df80 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.989+0000 7fd0b9c00640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b413df80 0x7fd0b4071e80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400c340 tx=0x7fd0a400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.989+0000 7fd0b1600640 1 -- 172.21.15.26:0/75053466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a40052d0 con 0x7fd0b413df80 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.989+0000 7fd0b1600640 1 -- 172.21.15.26:0/75053466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a4005430 con 0x7fd0b413df80 2024-01-15T18:03:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.989+0000 7fd0b1600640 1 -- 172.21.15.26:0/75053466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4003850 con 0x7fd0b413df80 2024-01-15T18:03:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.989+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b40713a0 con 0x7fd0b413df80 2024-01-15T18:03:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.989+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b4071960 con 0x7fd0b413df80 2024-01-15T18:03:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.991+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b4071b70 con 0x7fd0b413df80 2024-01-15T18:03:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.991+0000 7fd0b1600640 1 -- 172.21.15.26:0/75053466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fd0a40039b0 con 0x7fd0b413df80 2024-01-15T18:03:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.992+0000 7fd0b1600640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd08c06bbc0 0x7fd08c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.992+0000 7fd0b1600640 1 -- 172.21.15.26:0/75053466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd0a40879f0 con 0x7fd0b413df80 2024-01-15T18:03:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.993+0000 7fd0b9200640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd08c06bbc0 0x7fd08c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.994+0000 7fd0b9200640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd08c06bbc0 0x7fd08c06e080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd09c006fd0 tx=0x7fd09c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:27.997+0000 7fd0b1600640 1 -- 172.21.15.26:0/75053466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0a40542b0 con 0x7fd0b413df80 2024-01-15T18:03:28.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.198+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0b40675d0 con 0x7fd0b413df80 2024-01-15T18:03:28.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.202+0000 7fd0b1600640 1 -- 172.21.15.26:0/75053466 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16644 (secure 0 0 0) 0x7fd0a40578d0 con 0x7fd0b413df80 2024-01-15T18:03:28.208 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:28.208 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":71,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:03:26.744515+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:03:24.501738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-15T18:03:25.778629+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:03:28.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd08c06bbc0 msgr2=0x7fd08c06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd08c06bbc0 0x7fd08c06e080 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd09c006fd0 tx=0x7fd09c008040 comp rx=0 tx=0).stop 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b413df80 msgr2=0x7fd0b4071e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b413df80 0x7fd0b4071e80 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400c340 tx=0x7fd0a400c800 comp rx=0 tx=0).stop 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 shutdown_connections 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0b413e360 0x7fd0b414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd08c06bbc0 0x7fd08c06e080 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b415f9c0 0x7fd0b415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 --2- 172.21.15.26:0/75053466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b413df80 0x7fd0b4071e80 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.205+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 >> 172.21.15.26:0/75053466 conn(0x7fd0b406a800 msgr2=0x7fd0b41540f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:28.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.206+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 shutdown_connections 2024-01-15T18:03:28.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.206+0000 7fd0bc600640 1 -- 172.21.15.26:0/75053466 wait complete. 2024-01-15T18:03:28.230 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:03:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.451+0000 7ff6ac400640 1 Processor -- start 2024-01-15T18:03:28.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.451+0000 7ff6ac400640 1 -- start start 2024-01-15T18:03:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6ac400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6a40cf9a0 0x7ff6a415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6ac400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff6a415e1e0 0x7ff6a40702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6ac400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a40b1ee0 con 0x7ff6a4143ae0 2024-01-15T18:03:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6ac400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a40b2780 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6ac400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a415e990 con 0x7ff6a415e1e0 2024-01-15T18:03:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6a9a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff6a4143ae0 0x7ff6a40cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45342/0 (socket says 172.21.15.26:45342) 2024-01-15T18:03:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6a9a00640 1 -- 172.21.15.26:0/3232861366 learned_addr learned my addr 172.21.15.26:0/3232861366 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6a9000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6a40cf9a0 0x7ff6a415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6aa400640 1 --2- 172.21.15.26:0/3232861366 >> v2:172.21.15.124:3300/0 conn(0x7ff6a415e1e0 0x7ff6a40702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.453+0000 7ff6a9000640 1 -- 172.21.15.26:0/3232861366 >> v2:172.21.15.124:3300/0 conn(0x7ff6a415e1e0 msgr2=0x7ff6a40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.454+0000 7ff6a9000640 1 --2- 172.21.15.26:0/3232861366 >> v2:172.21.15.124:3300/0 conn(0x7ff6a415e1e0 0x7ff6a40702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.454+0000 7ff6a9000640 1 -- 172.21.15.26:0/3232861366 >> v1:172.21.15.26:6789/0 conn(0x7ff6a4143ae0 legacy=0x7ff6a40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.454+0000 7ff6a9000640 1 -- 172.21.15.26:0/3232861366 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a40b2130 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.454+0000 7ff6a3e00640 1 -- 172.21.15.26:0/3232861366 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783703735 0 0) 0x7ff6a40b1ee0 con 0x7ff6a4143ae0 2024-01-15T18:03:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.454+0000 7ff6aa400640 1 --2- 172.21.15.26:0/3232861366 >> v2:172.21.15.124:3300/0 conn(0x7ff6a415e1e0 0x7ff6a40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.455+0000 7ff6a9000640 1 --2- 172.21.15.26:0/3232861366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 0x7ff6a415dca0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40ae9d0 tx=0x7ff68c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368895d82829d552 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.455+0000 7ff6a3e00640 1 -- 172.21.15.26:0/3232861366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c031830 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.455+0000 7ff6a3e00640 1 -- 172.21.15.26:0/3232861366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68c031990 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.455+0000 7ff6a3e00640 1 -- 172.21.15.26:0/3232861366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c031d20 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.456+0000 7ff6ac400640 1 -- 172.21.15.26:0/3232861366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 msgr2=0x7ff6a415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.456+0000 7ff6ac400640 1 --2- 172.21.15.26:0/3232861366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 0x7ff6a415dca0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff6a40ae9d0 tx=0x7ff68c02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.456+0000 7ff6ac400640 1 -- 172.21.15.26:0/3232861366 shutdown_connections 2024-01-15T18:03:28.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.456+0000 7ff6ac400640 1 --2- 172.21.15.26:0/3232861366 >> v2:172.21.15.124:3300/0 conn(0x7ff6a415e1e0 0x7ff6a40702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.456+0000 7ff6ac400640 1 --2- 172.21.15.26:0/3232861366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 0x7ff6a415dca0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.456+0000 7ff6ac400640 1 -- 172.21.15.26:0/3232861366 >> 172.21.15.26:0/3232861366 conn(0x7ff6a406a800 msgr2=0x7ff6a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.456+0000 7ff6ac400640 1 -- 172.21.15.26:0/3232861366 shutdown_connections 2024-01-15T18:03:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.456+0000 7ff6ac400640 1 -- 172.21.15.26:0/3232861366 wait complete. 2024-01-15T18:03:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.458+0000 7ff6ac400640 1 Processor -- start 2024-01-15T18:03:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.458+0000 7ff6ac400640 1 -- start start 2024-01-15T18:03:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.459+0000 7ff6ac400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 0x7ff6a40cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.459+0000 7ff6ac400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6a4143ae0 0x7ff6a40c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.459+0000 7ff6ac400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6a415e1e0 0x7ff6a40c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.459+0000 7ff6ac400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40c61e0 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.459+0000 7ff6ac400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40c4c00 con 0x7ff6a4143ae0 2024-01-15T18:03:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.459+0000 7ff6ac400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a40c4ee0 con 0x7ff6a415e1e0 2024-01-15T18:03:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.459+0000 7ff6a9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 0x7ff6a40cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.459+0000 7ff6a9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 0x7ff6a40cb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46260/0 (socket says 172.21.15.26:46260) 2024-01-15T18:03:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.459+0000 7ff6a9a00640 1 -- 172.21.15.26:0/1624886414 learned_addr learned my addr 172.21.15.26:0/1624886414 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a9a00640 1 -- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6a415e1e0 msgr2=0x7ff6a40c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a9a00640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6a415e1e0 0x7ff6a40c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a9a00640 1 -- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6a4143ae0 msgr2=0x7ff6a40c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a9000640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6a4143ae0 0x7ff6a40c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a9a00640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6a4143ae0 0x7ff6a40c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a9a00640 1 -- 172.21.15.26:0/1624886414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c009400 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a9a00640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 0x7ff6a40cb340 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff69400c4d0 tx=0x7ff69400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a1600640 1 -- 172.21.15.26:0/1624886414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6940074e0 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a1600640 1 -- 172.21.15.26:0/1624886414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff694007660 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6a1600640 1 -- 172.21.15.26:0/1624886414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694004a20 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.460+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a40c51c0 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.461+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a40c5780 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.462+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a40719e0 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.463+0000 7ff6a1600640 1 -- 172.21.15.26:0/1624886414 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7ff694004b80 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.464+0000 7ff6a1600640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff67c06bc10 0x7ff67c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:28.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.464+0000 7ff6a1600640 1 -- 172.21.15.26:0/1624886414 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff694087b70 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.464+0000 7ff6a9000640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff67c06bc10 0x7ff67c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:28.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.466+0000 7ff6a9000640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff67c06bc10 0x7ff67c06e0d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff68c036000 tx=0x7ff68c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:28.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.468+0000 7ff6a1600640 1 -- 172.21.15.26:0/1624886414 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff694054430 con 0x7ff6a40cf9a0 2024-01-15T18:03:28.536 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:03:28.544 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:03:28.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:28.706+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff6a40675d0 con 0x7ff6a40cf9a0 2024-01-15T18:03:29.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.333+0000 7ff6a1600640 1 -- 172.21.15.26:0/1624886414 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff6940180b0 con 0x7ff6a40cf9a0 2024-01-15T18:03:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.337+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff67c06bc10 msgr2=0x7ff67c06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.337+0000 7ff6ac400640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff67c06bc10 0x7ff67c06e0d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff68c036000 tx=0x7ff68c008030 comp rx=0 tx=0).stop 2024-01-15T18:03:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 msgr2=0x7ff6a40cb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 0x7ff6a40cb340 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff69400c4d0 tx=0x7ff69400c990 comp rx=0 tx=0).stop 2024-01-15T18:03:29.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 shutdown_connections 2024-01-15T18:03:29.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6a415e1e0 0x7ff6a40c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:29.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff67c06bc10 0x7ff67c06e0d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:29.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6a4143ae0 0x7ff6a40c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:29.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 --2- 172.21.15.26:0/1624886414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6a40cf9a0 0x7ff6a40cb340 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:29.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 >> 172.21.15.26:0/1624886414 conn(0x7ff6a406a800 msgr2=0x7ff6a406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:29.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 shutdown_connections 2024-01-15T18:03:29.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.338+0000 7ff6ac400640 1 -- 172.21.15.26:0/1624886414 wait complete. 2024-01-15T18:03:29.366 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:03:29.367 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:03:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.595+0000 7f2d73800640 1 Processor -- start 2024-01-15T18:03:29.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.595+0000 7f2d73800640 1 -- start start 2024-01-15T18:03:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.597+0000 7f2d73800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d6c14b2c0 0x7f2d6c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.597+0000 7f2d73800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c0cfbf0 con 0x7f2d6c071070 2024-01-15T18:03:29.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.597+0000 7f2d73800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c071690 con 0x7f2d6c0cf7d0 2024-01-15T18:03:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.597+0000 7f2d73800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c14b6a0 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.598+0000 7f2d70e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2d6c14b2c0 0x7f2d6c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.598+0000 7f2d71800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2d6c071070 0x7f2d6c0cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45356/0 (socket says 172.21.15.26:45356) 2024-01-15T18:03:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.598+0000 7f2d71800640 1 -- 172.21.15.26:0/1892932880 learned_addr learned my addr 172.21.15.26:0/1892932880 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.598+0000 7f2d6b400640 1 -- 172.21.15.26:0/1892932880 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879461635 0 0) 0x7f2d6c0cfbf0 con 0x7f2d6c071070 2024-01-15T18:03:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.598+0000 7f2d6b400640 1 -- 172.21.15.26:0/1892932880 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d4c003520 con 0x7f2d6c071070 2024-01-15T18:03:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.598+0000 7f2d70e00640 1 -- 172.21.15.26:0/1892932880 >> v1:172.21.15.124:6789/0 conn(0x7f2d6c0cf7d0 legacy=0x7f2d6c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.598+0000 7f2d70e00640 1 -- 172.21.15.26:0/1892932880 >> v1:172.21.15.26:6789/0 conn(0x7f2d6c071070 legacy=0x7f2d6c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:29.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.598+0000 7f2d70e00640 1 -- 172.21.15.26:0/1892932880 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c071970 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.600+0000 7f2d70e00640 1 --2- 172.21.15.26:0/1892932880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 0x7f2d6c0cf290 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0097a0 tx=0x7f2d5c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ce3de7d550230 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.601+0000 7f2d6b400640 1 -- 172.21.15.26:0/1892932880 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c030610 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.601+0000 7f2d6b400640 1 -- 172.21.15.26:0/1892932880 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c030770 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.601+0000 7f2d6b400640 1 -- 172.21.15.26:0/1892932880 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c030ae0 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.601+0000 7f2d73800640 1 -- 172.21.15.26:0/1892932880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 msgr2=0x7f2d6c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.601+0000 7f2d73800640 1 --2- 172.21.15.26:0/1892932880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 0x7f2d6c0cf290 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c0097a0 tx=0x7f2d5c02b960 comp rx=0 tx=0).stop 2024-01-15T18:03:29.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.602+0000 7f2d73800640 1 -- 172.21.15.26:0/1892932880 shutdown_connections 2024-01-15T18:03:29.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.602+0000 7f2d73800640 1 --2- 172.21.15.26:0/1892932880 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 0x7f2d6c0cf290 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:29.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.602+0000 7f2d73800640 1 -- 172.21.15.26:0/1892932880 >> 172.21.15.26:0/1892932880 conn(0x7f2d6c06a800 msgr2=0x7f2d6c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:29.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.602+0000 7f2d73800640 1 -- 172.21.15.26:0/1892932880 shutdown_connections 2024-01-15T18:03:29.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.602+0000 7f2d73800640 1 -- 172.21.15.26:0/1892932880 wait complete. 2024-01-15T18:03:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.604+0000 7f2d73800640 1 Processor -- start 2024-01-15T18:03:29.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.604+0000 7f2d73800640 1 -- start start 2024-01-15T18:03:29.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.605+0000 7f2d73800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d6c071070 0x7f2d6c141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d73800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6c0cf7d0 0x7f2d6c0c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d73800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 0x7f2d6c0c4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d73800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c1447b0 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d73800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c143ba0 con 0x7f2d6c071070 2024-01-15T18:03:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d73800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c143e80 con 0x7f2d6c0cf7d0 2024-01-15T18:03:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d71800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 0x7f2d6c0c4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d71800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 0x7f2d6c0c4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46276/0 (socket says 172.21.15.26:46276) 2024-01-15T18:03:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d71800640 1 -- 172.21.15.26:0/851981744 learned_addr learned my addr 172.21.15.26:0/851981744 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:29.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d70e00640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d6c071070 0x7f2d6c141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.606+0000 7f2d6be00640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6c0cf7d0 0x7f2d6c0c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d71800640 1 -- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6c0cf7d0 msgr2=0x7f2d6c0c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d71800640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6c0cf7d0 0x7f2d6c0c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d71800640 1 -- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d6c071070 msgr2=0x7f2d6c141f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d71800640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d6c071070 0x7f2d6c141f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d71800640 1 -- 172.21.15.26:0/851981744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c009400 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d71800640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 0x7f2d6c0c4390 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c430 tx=0x7f2d6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d68c00640 1 -- 172.21.15.26:0/851981744 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64005ce0 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d68c00640 1 -- 172.21.15.26:0/851981744 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d64005e40 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d68c00640 1 -- 172.21.15.26:0/851981744 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64007310 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.607+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c144160 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.608+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c142440 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.609+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c06de40 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.610+0000 7f2d68c00640 1 -- 172.21.15.26:0/851981744 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f2d64007470 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.611+0000 7f2d68c00640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4406bbc0 0x7f2d4406e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:29.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.611+0000 7f2d68c00640 1 -- 172.21.15.26:0/851981744 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2d64087aa0 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.612+0000 7f2d70e00640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4406bbc0 0x7f2d4406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:29.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.613+0000 7f2d70e00640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4406bbc0 0x7f2d4406e080 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c036000 tx=0x7f2d5c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:29.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.615+0000 7f2d68c00640 1 -- 172.21.15.26:0/851981744 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d640543c0 con 0x7f2d6c14b2c0 2024-01-15T18:03:29.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:29.850+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2d6c0675d0 con 0x7f2d6c14b2c0 2024-01-15T18:03:30.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.348+0000 7f2d68c00640 1 -- 172.21.15.26:0/851981744 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2d640579e0 con 0x7f2d6c14b2c0 2024-01-15T18:03:30.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4406bbc0 msgr2=0x7f2d4406e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4406bbc0 0x7f2d4406e080 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c036000 tx=0x7f2d5c008030 comp rx=0 tx=0).stop 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 msgr2=0x7f2d6c0c4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 0x7f2d6c0c4390 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c430 tx=0x7f2d6400c8f0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 shutdown_connections 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2d6c0cf7d0 0x7f2d6c0c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2d4406bbc0 0x7f2d4406e080 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2d6c14b2c0 0x7f2d6c0c4390 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 --2- 172.21.15.26:0/851981744 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2d6c071070 0x7f2d6c141f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 >> 172.21.15.26:0/851981744 conn(0x7f2d6c06a800 msgr2=0x7f2d6c148c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 shutdown_connections 2024-01-15T18:03:30.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.353+0000 7f2d73800640 1 -- 172.21.15.26:0/851981744 wait complete. 2024-01-15T18:03:30.381 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:03:30.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:03:30.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.604+0000 7f8bfdc00640 1 Processor -- start 2024-01-15T18:03:30.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.604+0000 7f8bfdc00640 1 -- start start 2024-01-15T18:03:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.606+0000 7f8bfdc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8bf8160920 0x7f8bf80b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.606+0000 7f8bfdc00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8bf80cf9f0 0x7f8bf80cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.606+0000 7f8bfdc00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf80d0a10 con 0x7f8bf8161010 2024-01-15T18:03:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.606+0000 7f8bfdc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80d03e0 con 0x7f8bf8160920 2024-01-15T18:03:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.606+0000 7f8bfdc00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf80d1f50 con 0x7f8bf80cf9f0 2024-01-15T18:03:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.606+0000 7f8bf7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8bf8160920 0x7f8bf80b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.606+0000 7f8bf7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8bf8160920 0x7f8bf80b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46280/0 (socket says 172.21.15.26:46280) 2024-01-15T18:03:30.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.606+0000 7f8bf7e00640 1 -- 172.21.15.26:0/99134572 learned_addr learned my addr 172.21.15.26:0/99134572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:30.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.607+0000 7f8befe00640 1 --2- 172.21.15.26:0/99134572 >> v2:172.21.15.124:3300/0 conn(0x7f8bf80cf9f0 0x7f8bf80cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:30.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.607+0000 7f8bf7e00640 1 -- 172.21.15.26:0/99134572 >> v2:172.21.15.124:3300/0 conn(0x7f8bf80cf9f0 msgr2=0x7f8bf80cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:30.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.607+0000 7f8bf7e00640 1 --2- 172.21.15.26:0/99134572 >> v2:172.21.15.124:3300/0 conn(0x7f8bf80cf9f0 0x7f8bf80cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.607+0000 7f8bf7e00640 1 -- 172.21.15.26:0/99134572 >> v1:172.21.15.124:6789/0 conn(0x7f8bf8161010 legacy=0x7f8bf8161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:30.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.607+0000 7f8bf7e00640 1 -- 172.21.15.26:0/99134572 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80cf080 con 0x7f8bf8160920 2024-01-15T18:03:30.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.608+0000 7f8bf7e00640 1 --2- 172.21.15.26:0/99134572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8160920 0x7f8bf80b3f30 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8bf808c1f0 tx=0x7f8be802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5831404e32262341 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:30.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.608+0000 7f8bf7400640 1 -- 172.21.15.26:0/99134572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8031870 con 0x7f8bf8160920 2024-01-15T18:03:30.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.608+0000 7f8bf7400640 1 -- 172.21.15.26:0/99134572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be80319d0 con 0x7f8bf8160920 2024-01-15T18:03:30.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.608+0000 7f8bf7400640 1 -- 172.21.15.26:0/99134572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8031d60 con 0x7f8bf8160920 2024-01-15T18:03:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.609+0000 7f8bfdc00640 1 -- 172.21.15.26:0/99134572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8160920 msgr2=0x7f8bf80b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.609+0000 7f8bfdc00640 1 --2- 172.21.15.26:0/99134572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8160920 0x7f8bf80b3f30 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8bf808c1f0 tx=0x7f8be802b910 comp rx=0 tx=0).stop 2024-01-15T18:03:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.609+0000 7f8bfdc00640 1 -- 172.21.15.26:0/99134572 shutdown_connections 2024-01-15T18:03:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.609+0000 7f8bfdc00640 1 --2- 172.21.15.26:0/99134572 >> v2:172.21.15.124:3300/0 conn(0x7f8bf80cf9f0 0x7f8bf80cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.609+0000 7f8bfdc00640 1 --2- 172.21.15.26:0/99134572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8160920 0x7f8bf80b3f30 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.609+0000 7f8bfdc00640 1 -- 172.21.15.26:0/99134572 >> 172.21.15.26:0/99134572 conn(0x7f8bf80d3090 msgr2=0x7f8bf80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.609+0000 7f8bfdc00640 1 -- 172.21.15.26:0/99134572 shutdown_connections 2024-01-15T18:03:30.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.609+0000 7f8bfdc00640 1 -- 172.21.15.26:0/99134572 wait complete. 2024-01-15T18:03:30.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.611+0000 7f8bfdc00640 1 Processor -- start 2024-01-15T18:03:30.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.612+0000 7f8bfdc00640 1 -- start start 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.613+0000 7f8bfdc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cf9f0 0x7f8bf8157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.613+0000 7f8bfdc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf8160920 0x7f8bf815b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.613+0000 7f8bfdc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8161010 0x7f8bf815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.613+0000 7f8bfdc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80b6860 con 0x7f8bf8161010 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.613+0000 7f8bfdc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf81584c0 con 0x7f8bf8160920 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.613+0000 7f8bfdc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80b5c50 con 0x7f8bf80cf9f0 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.613+0000 7f8bfd200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8161010 0x7f8bf815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.613+0000 7f8bfd200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8161010 0x7f8bf815bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46290/0 (socket says 172.21.15.26:46290) 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.613+0000 7f8bfd200640 1 -- 172.21.15.26:0/3089584967 learned_addr learned my addr 172.21.15.26:0/3089584967 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8befe00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf8160920 0x7f8bf815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bf7e00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cf9f0 0x7f8bf8157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:30.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bfd200640 1 -- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cf9f0 msgr2=0x7f8bf8157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bfd200640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cf9f0 0x7f8bf8157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bfd200640 1 -- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf8160920 msgr2=0x7f8bf815b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bfd200640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf8160920 0x7f8bf815b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bfd200640 1 -- 172.21.15.26:0/3089584967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be80094d0 con 0x7f8bf8161010 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bf7e00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cf9f0 0x7f8bf8157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8befe00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf8160920 0x7f8bf815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bfd200640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8161010 0x7f8bf815bcd0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c780 tx=0x7f8bf000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bf4c00640 1 -- 172.21.15.26:0/3089584967 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0003d80 con 0x7f8bf8161010 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bf4c00640 1 -- 172.21.15.26:0/3089584967 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf000ce90 con 0x7f8bf8161010 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bf4c00640 1 -- 172.21.15.26:0/3089584967 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00052b0 con 0x7f8bf8161010 2024-01-15T18:03:30.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.614+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf80b5f30 con 0x7f8bf8161010 2024-01-15T18:03:30.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.615+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf80b64f0 con 0x7f8bf8161010 2024-01-15T18:03:30.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.616+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf80b6a10 con 0x7f8bf8161010 2024-01-15T18:03:30.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.617+0000 7f8bf4c00640 1 -- 172.21.15.26:0/3089584967 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f8bf0005410 con 0x7f8bf8161010 2024-01-15T18:03:30.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.618+0000 7f8bf4c00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8bd006bbf0 0x7f8bd006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:30.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.618+0000 7f8bf4c00640 1 -- 172.21.15.26:0/3089584967 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8bf00877f0 con 0x7f8bf8161010 2024-01-15T18:03:30.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.618+0000 7f8bf7e00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8bd006bbf0 0x7f8bd006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:30.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.620+0000 7f8bf7e00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8bd006bbf0 0x7f8bd006e0b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8be8036000 tx=0x7f8be803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:30.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.623+0000 7f8bf4c00640 1 -- 172.21.15.26:0/3089584967 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8bf00bce90 con 0x7f8bf8161010 2024-01-15T18:03:30.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.857+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bf8067450 con 0x7f8bf8161010 2024-01-15T18:03:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.859+0000 7f8bf4c00640 1 -- 172.21.15.26:0/3089584967 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f8bf80b6a10 con 0x7f8bf8161010 2024-01-15T18:03:30.864 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:30.864 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1986791067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1986791067},{"type":"v1","addr":"172.21.15.26:6834","nonce":1986791067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6082,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3285856051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3285856051},{"type":"v1","addr":"172.21.15.124:6836","nonce":3285856051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6085,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/198642905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":198642905},{"type":"v1","addr":"172.21.15.124:6837","nonce":198642905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:03:26.744869+0000","modified":"2024-01-15T18:03:29.852734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1813593645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1813593645},{"type":"v1","addr":"172.21.15.26:6835","nonce":1813593645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-01-15T18:03:30.864 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 44 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8bd006bbf0 msgr2=0x7f8bd006e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8bd006bbf0 0x7f8bd006e0b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8be8036000 tx=0x7f8be803c040 comp rx=0 tx=0).stop 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8161010 msgr2=0x7f8bf815bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8161010 0x7f8bf815bcd0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c780 tx=0x7f8bf000cc40 comp rx=0 tx=0).stop 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 shutdown_connections 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8bd006bbf0 0x7f8bd006e0b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8bf8161010 0x7f8bf815bcd0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8bf8160920 0x7f8bf815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 --2- 172.21.15.26:0/3089584967 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8bf80cf9f0 0x7f8bf8157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 >> 172.21.15.26:0/3089584967 conn(0x7f8bf80d3090 msgr2=0x7f8bf806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 shutdown_connections 2024-01-15T18:03:30.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:30.862+0000 7f8bfdc00640 1 -- 172.21.15.26:0/3089584967 wait complete. 2024-01-15T18:03:30.887 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:03:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.098+0000 7f6c3b400640 1 Processor -- start 2024-01-15T18:03:31.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.098+0000 7f6c3b400640 1 -- start start 2024-01-15T18:03:31.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.099+0000 7f6c3b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c340ce8f0 0x7f6c340ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.099+0000 7f6c3b400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c340d0de0 con 0x7f6c34160f80 2024-01-15T18:03:31.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.099+0000 7f6c3b400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c340d10c0 con 0x7f6c340b2a00 2024-01-15T18:03:31.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.099+0000 7f6c3b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340d13a0 con 0x7f6c340ce8f0 2024-01-15T18:03:31.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c38a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6c34160f80 0x7f6c34161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45372/0 (socket says 172.21.15.26:45372) 2024-01-15T18:03:31.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c38a00640 1 -- 172.21.15.26:0/3264806236 learned_addr learned my addr 172.21.15.26:0/3264806236 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:31.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520064997 0 0) 0x7f6c340d0de0 con 0x7f6c34160f80 2024-01-15T18:03:31.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c14003520 con 0x7f6c34160f80 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c33e00640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c340ce8f0 0x7f6c340ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3520024883 0 0) 0x7f6c14003520 con 0x7f6c34160f80 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c340ce8f0 msgr2=0x7f6c340ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c33400640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c340ce8f0 0x7f6c340ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 >> v1:172.21.15.124:6789/0 conn(0x7f6c340b2a00 legacy=0x7f6c340b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.100+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340d1680 con 0x7f6c34160f80 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f6c24001670 con 0x7f6c34160f80 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 >> v1:172.21.15.26:6789/0 conn(0x7f6c34160f80 legacy=0x7f6c34161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c33400640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c340ce8f0 0x7f6c140039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c33400640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c14004020 0x7f6c140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c33400640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c340d13a0 con 0x7f6c14004d00 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1400bbc0 con 0x7f6c14004020 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1400bd80 con 0x7f6c340ce8f0 2024-01-15T18:03:31.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c39400640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c33e00640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c14004020 0x7f6c140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c38a00640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c340ce8f0 0x7f6c140039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c39400640 1 -- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c340ce8f0 msgr2=0x7f6c140039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c39400640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c340ce8f0 0x7f6c140039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c39400640 1 -- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c14004020 msgr2=0x7f6c140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c39400640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c14004020 0x7f6c140047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c39400640 1 -- 172.21.15.26:0/3264806236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c14003520 con 0x7f6c14004d00 2024-01-15T18:03:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.101+0000 7f6c38a00640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c340ce8f0 0x7f6c140039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:31.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.102+0000 7f6c39400640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c14005180 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c001750 tx=0x7f6c140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51b1a1918330cc6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:31.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.103+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0052d0 con 0x7f6c14004d00 2024-01-15T18:03:31.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.103+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c02e050 con 0x7f6c14004d00 2024-01-15T18:03:31.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.103+0000 7f6c33400640 1 -- 172.21.15.26:0/3264806236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c003830 con 0x7f6c14004d00 2024-01-15T18:03:31.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.103+0000 7f6c3b400640 1 -- 172.21.15.26:0/3264806236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 msgr2=0x7f6c14005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.103+0000 7f6c3b400640 1 --2- 172.21.15.26:0/3264806236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c14005180 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c001750 tx=0x7f6c140057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.103+0000 7f6c39400640 1 -- 172.21.15.26:0/3264806236 reap_dead start 2024-01-15T18:03:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.103+0000 7f6c3b400640 1 -- 172.21.15.26:0/3264806236 shutdown_connections 2024-01-15T18:03:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.103+0000 7f6c3b400640 1 -- 172.21.15.26:0/3264806236 >> 172.21.15.26:0/3264806236 conn(0x7f6c3409c550 msgr2=0x7f6c340d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.103+0000 7f6c3b400640 1 -- 172.21.15.26:0/3264806236 shutdown_connections 2024-01-15T18:03:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.104+0000 7f6c3b400640 1 -- 172.21.15.26:0/3264806236 wait complete. 2024-01-15T18:03:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.105+0000 7f6c3b400640 1 Processor -- start 2024-01-15T18:03:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.105+0000 7f6c3b400640 1 -- start start 2024-01-15T18:03:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.106+0000 7f6c3b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c341e87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.106+0000 7f6c3b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c34158800 0x7f6c340b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.106+0000 7f6c3b400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c3415df40 0x7f6c3415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.106+0000 7f6c3b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341e9a00 con 0x7f6c14004d00 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.106+0000 7f6c3b400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3415e900 con 0x7f6c3415df40 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.106+0000 7f6c3b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c34158c80 con 0x7f6c34158800 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c38a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c341e87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c38a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c341e87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46308/0 (socket says 172.21.15.26:46308) 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c38a00640 1 -- 172.21.15.26:0/666518608 learned_addr learned my addr 172.21.15.26:0/666518608 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c33e00640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c34158800 0x7f6c340b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c39400640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c3415df40 0x7f6c3415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c38a00640 1 -- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c34158800 msgr2=0x7f6c340b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c38a00640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c34158800 0x7f6c340b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c38a00640 1 -- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c3415df40 msgr2=0x7f6c3415e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c38a00640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c3415df40 0x7f6c3415e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c38a00640 1 -- 172.21.15.26:0/666518608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340b4260 con 0x7f6c14004d00 2024-01-15T18:03:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c33e00640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c34158800 0x7f6c340b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.107+0000 7f6c39400640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c3415df40 0x7f6c3415e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.108+0000 7f6c38a00640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c341e87a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6c24000c00 tx=0x7f6c2400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.108+0000 7f6c30c00640 1 -- 172.21.15.26:0/666518608 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240074e0 con 0x7f6c14004d00 2024-01-15T18:03:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.108+0000 7f6c30c00640 1 -- 172.21.15.26:0/666518608 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24007640 con 0x7f6c14004d00 2024-01-15T18:03:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.108+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c341e8d80 con 0x7f6c14004d00 2024-01-15T18:03:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.108+0000 7f6c30c00640 1 -- 172.21.15.26:0/666518608 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24004a20 con 0x7f6c14004d00 2024-01-15T18:03:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.108+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c341e9340 con 0x7f6c14004d00 2024-01-15T18:03:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.109+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c340675d0 con 0x7f6c14004d00 2024-01-15T18:03:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.110+0000 7f6c30c00640 1 -- 172.21.15.26:0/666518608 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f6c24004b80 con 0x7f6c14004d00 2024-01-15T18:03:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.111+0000 7f6c30c00640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6c0c06bbc0 0x7f6c0c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.111+0000 7f6c30c00640 1 -- 172.21.15.26:0/666518608 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6c24087fe0 con 0x7f6c14004d00 2024-01-15T18:03:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.112+0000 7f6c33e00640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6c0c06bbc0 0x7f6c0c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.113+0000 7f6c33e00640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6c0c06bbc0 0x7f6c0c06e080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6c340b48b0 tx=0x7f6c1c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.115+0000 7f6c30c00640 1 -- 172.21.15.26:0/666518608 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c24054790 con 0x7f6c14004d00 2024-01-15T18:03:31.235 INFO:tasks.ceph.mds.c.smithi026.stderr:2024-01-15T18:03:31.230+0000 7fc915800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:03:31.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.312+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c340d3920 con 0x7f6c14004d00 2024-01-15T18:03:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.315+0000 7f6c30c00640 1 -- 172.21.15.26:0/666518608 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16631 (secure 0 0 0) 0x7f6c24019090 con 0x7f6c14004d00 2024-01-15T18:03:31.321 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:31.321 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":72,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:03:28.291817+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:03:24.501738+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-01-15T18:03:25.778629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6c0c06bbc0 msgr2=0x7f6c0c06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6c0c06bbc0 0x7f6c0c06e080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f6c340b48b0 tx=0x7f6c1c00b040 comp rx=0 tx=0).stop 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 msgr2=0x7f6c341e87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c341e87a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6c24000c00 tx=0x7f6c2400dd40 comp rx=0 tx=0).stop 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 shutdown_connections 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6c34158800 0x7f6c340b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6c0c06bbc0 0x7f6c0c06e080 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6c3415df40 0x7f6c3415e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 --2- 172.21.15.26:0/666518608 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6c14004d00 0x7f6c341e87a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 >> 172.21.15.26:0/666518608 conn(0x7f6c3409c550 msgr2=0x7f6c34161360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 shutdown_connections 2024-01-15T18:03:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.318+0000 7f6c3b400640 1 -- 172.21.15.26:0/666518608 wait complete. 2024-01-15T18:03:31.342 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:03:31.343 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:03:31.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.559+0000 7fdfc1400640 1 Processor -- start 2024-01-15T18:03:31.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.559+0000 7fdfc1400640 1 -- start start 2024-01-15T18:03:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.560+0000 7fdfc1400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdfbc0cc960 0x7fdfbc168080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.560+0000 7fdfc1400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfbc0d1d10 con 0x7fdfbc15d440 2024-01-15T18:03:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.561+0000 7fdfc1400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfbc06c520 con 0x7fdfbc0d1900 2024-01-15T18:03:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.561+0000 7fdfc1400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc0cfaf0 con 0x7fdfbc0cc960 2024-01-15T18:03:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.561+0000 7fdfbbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fdfbc15d440 0x7fdfbc06b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45378/0 (socket says 172.21.15.26:45378) 2024-01-15T18:03:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.561+0000 7fdfbbe00640 1 -- 172.21.15.26:0/3078988212 learned_addr learned my addr 172.21.15.26:0/3078988212 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.561+0000 7fdfc0a00640 1 --2- 172.21.15.26:0/3078988212 >> v2:172.21.15.124:3300/0 conn(0x7fdfbc0cc960 0x7fdfbc168080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.561+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291515907 0 0) 0x7fdfbc0d1d10 con 0x7fdfbc15d440 2024-01-15T18:03:31.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.561+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf9c003520 con 0x7fdfbc15d440 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 579758827 0 0) 0x7fdf9c003520 con 0x7fdfbc15d440 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 >> v2:172.21.15.124:3300/0 conn(0x7fdfbc0cc960 msgr2=0x7fdfbc168080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 --2- 172.21.15.26:0/3078988212 >> v2:172.21.15.124:3300/0 conn(0x7fdfbc0cc960 0x7fdfbc168080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 >> v1:172.21.15.124:6789/0 conn(0x7fdfbc0d1900 legacy=0x7fdfbc06be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc0cfd00 con 0x7fdfbc15d440 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fdfac001670 con 0x7fdfbc15d440 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 >> v1:172.21.15.26:6789/0 conn(0x7fdfbc15d440 legacy=0x7fdfbc06b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9c003970 0x7fdf9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf9c0047c0 0x7fdf9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf9c005180 0x7fdf9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0cfaf0 con 0x7fdf9c003970 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c00c020 con 0x7fdf9c0047c0 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c00c1e0 con 0x7fdf9c005180 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.562+0000 7fdfbbe00640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9c003970 0x7fdf9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.563+0000 7fdfbb400640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf9c0047c0 0x7fdf9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.563+0000 7fdfbbe00640 1 -- 172.21.15.26:0/3078988212 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf9c005180 msgr2=0x7fdf9c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.563+0000 7fdfbbe00640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf9c005180 0x7fdf9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.563+0000 7fdfbbe00640 1 -- 172.21.15.26:0/3078988212 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf9c0047c0 msgr2=0x7fdf9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.563+0000 7fdfbbe00640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdf9c0047c0 0x7fdf9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.563+0000 7fdfbbe00640 1 -- 172.21.15.26:0/3078988212 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c003520 con 0x7fdf9c003970 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.563+0000 7fdfc0a00640 1 -- 172.21.15.26:0/3078988212 reap_dead start 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.564+0000 7fdfbbe00640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9c003970 0x7fdf9c003d90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c240 tx=0x7fdf9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c44378c544df81 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:31.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.564+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac005ad0 con 0x7fdf9c003970 2024-01-15T18:03:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.564+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac005c30 con 0x7fdf9c003970 2024-01-15T18:03:31.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.564+0000 7fdfbaa00640 1 -- 172.21.15.26:0/3078988212 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac02c870 con 0x7fdf9c003970 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.565+0000 7fdfc1400640 1 -- 172.21.15.26:0/3078988212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9c003970 msgr2=0x7fdf9c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.565+0000 7fdfc1400640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9c003970 0x7fdf9c003d90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c240 tx=0x7fdf9c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.565+0000 7fdfc1400640 1 -- 172.21.15.26:0/3078988212 shutdown_connections 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.565+0000 7fdfc1400640 1 --2- 172.21.15.26:0/3078988212 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdf9c003970 0x7fdf9c003d90 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.565+0000 7fdfc1400640 1 -- 172.21.15.26:0/3078988212 >> 172.21.15.26:0/3078988212 conn(0x7fdfbc0d0de0 msgr2=0x7fdfbc0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.565+0000 7fdfc1400640 1 -- 172.21.15.26:0/3078988212 shutdown_connections 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.565+0000 7fdfc1400640 1 -- 172.21.15.26:0/3078988212 wait complete. 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.567+0000 7fdfc1400640 1 Processor -- start 2024-01-15T18:03:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.567+0000 7fdfc1400640 1 -- start start 2024-01-15T18:03:31.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.568+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf9c003970 0x7fdfbc1f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.568+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfbc0c6ae0 0x7fdfbc14bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.568+0000 7fdfc1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15dc70 0x7fdfbc0b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.568+0000 7fdfc1400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc14ce30 con 0x7fdfbc15dc70 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.568+0000 7fdfc1400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc14c870 con 0x7fdfbc0c6ae0 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.568+0000 7fdfc1400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc14cb50 con 0x7fdf9c003970 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.568+0000 7fdfc0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15dc70 0x7fdfbc0b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.568+0000 7fdfc0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15dc70 0x7fdfbc0b5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46328/0 (socket says 172.21.15.26:46328) 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.568+0000 7fdfc0a00640 1 -- 172.21.15.26:0/3546162790 learned_addr learned my addr 172.21.15.26:0/3546162790 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfbb400640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfbc0c6ae0 0x7fdfbc14bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfc0a00640 1 -- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf9c003970 msgr2=0x7fdfbc1f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfbbe00640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf9c003970 0x7fdfbc1f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfc0a00640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf9c003970 0x7fdfbc1f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfc0a00640 1 -- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfbc0c6ae0 msgr2=0x7fdfbc14bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfc0a00640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfbc0c6ae0 0x7fdfbc14bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfc0a00640 1 -- 172.21.15.26:0/3546162790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc1f6320 con 0x7fdfbc15dc70 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfbbe00640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf9c003970 0x7fdfbc1f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfbb400640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfbc0c6ae0 0x7fdfbc14bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfc0a00640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15dc70 0x7fdfbc0b5d70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc070fe0 tx=0x7fdfb400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.569+0000 7fdfb3e00640 1 -- 172.21.15.26:0/3546162790 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb400c680 con 0x7fdfbc15dc70 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.570+0000 7fdfb3e00640 1 -- 172.21.15.26:0/3546162790 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb400c7e0 con 0x7fdfbc15dc70 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.570+0000 7fdfb3e00640 1 -- 172.21.15.26:0/3546162790 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb400cbd0 con 0x7fdfbc15dc70 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.570+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc1f6600 con 0x7fdfbc15dc70 2024-01-15T18:03:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.570+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc14c460 con 0x7fdfbc15dc70 2024-01-15T18:03:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.571+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc1f67f0 con 0x7fdfbc15dc70 2024-01-15T18:03:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.572+0000 7fdfb3e00640 1 -- 172.21.15.26:0/3546162790 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fdfb400cd30 con 0x7fdfbc15dc70 2024-01-15T18:03:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.573+0000 7fdfb3e00640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf9406bbc0 0x7fdf9406e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.573+0000 7fdfb3e00640 1 -- 172.21.15.26:0/3546162790 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fdfb4087d60 con 0x7fdfbc15dc70 2024-01-15T18:03:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.574+0000 7fdfbbe00640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf9406bbc0 0x7fdf9406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:31.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.575+0000 7fdfbbe00640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf9406bbc0 0x7fdf9406e080 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdfac034000 tx=0x7fdfac009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:31.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.577+0000 7fdfb3e00640 1 -- 172.21.15.26:0/3546162790 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdfb4054490 con 0x7fdfbc15dc70 2024-01-15T18:03:31.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.767+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfbc1f7240 con 0x7fdf9406bbc0 2024-01-15T18:03:31.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.795+0000 7fdfb3e00640 1 -- 172.21.15.26:0/3546162790 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282435 (secure 0 0 0) 0x7fdfbc1f7240 con 0x7fdf9406bbc0 2024-01-15T18:03:31.804 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:31.806 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf9406bbc0 msgr2=0x7fdf9406e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf9406bbc0 0x7fdf9406e080 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdfac034000 tx=0x7fdfac009030 comp rx=0 tx=0).stop 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15dc70 msgr2=0x7fdfbc0b5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15dc70 0x7fdfbc0b5d70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc070fe0 tx=0x7fdfb400ec00 comp rx=0 tx=0).stop 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 shutdown_connections 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdf9406bbc0 0x7fdf9406e080 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdfbc15dc70 0x7fdfbc0b5d70 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdfbc0c6ae0 0x7fdfbc14bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 --2- 172.21.15.26:0/3546162790 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdf9c003970 0x7fdfbc1f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:31.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 >> 172.21.15.26:0/3546162790 conn(0x7fdfbc0d0de0 msgr2=0x7fdfbc15c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 shutdown_connections 2024-01-15T18:03:31.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:31.802+0000 7fdfc1400640 1 -- 172.21.15.26:0/3546162790 wait complete. 2024-01-15T18:03:31.829 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-01-15T18:03:30.104180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":168,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409936,"kb_used_data":7772,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754564784,"statfs":{"total":773094113280,"available":772674338816,"internally_reserved":0,"allocated":7958528,"data_stored":4028525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.841676"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281242+0000","last_change":"2024-01-15T18:03:26.281242+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281242+0000","last_undegraded":"2024-01-15T18:03:26.281242+0000","last_fullsized":"2024-01-15T18:03:26.281242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342379+0000","last_change":"2024-01-15T18:03:27.341616+0000","last_active":"2024-01-15T18:03:27.342379+0000","last_peered":"2024-01-15T18:03:27.342379+0000","last_clean":"2024-01-15T18:03:27.342379+0000","last_became_active":"2024-01-15T18:03:27.341248+0000","last_became_peered":"2024-01-15T18:03:27.341248+0000","last_unstale":"2024-01-15T18:03:27.342379+0000","last_undegraded":"2024-01-15T18:03:27.342379+0000","last_fullsized":"2024-01-15T18:03:27.342379+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:51.975291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283811+0000","last_change":"2024-01-15T18:03:26.283811+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283811+0000","last_undegraded":"2024-01-15T18:03:26.283811+0000","last_fullsized":"2024-01-15T18:03:26.283811+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.288618+0000","last_change":"2024-01-15T18:03:26.288618+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.288618+0000","last_undegraded":"2024-01-15T18:03:26.288618+0000","last_fullsized":"2024-01-15T18:03:26.288618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.272956+0000","last_change":"2024-01-15T18:03:26.272956+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.272956+0000","last_undegraded":"2024-01-15T18:03:26.272956+0000","last_fullsized":"2024-01-15T18:03:26.272956+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.321499+0000","last_change":"2024-01-15T18:03:27.319124+0000","last_active":"2024-01-15T18:03:27.321499+0000","last_peered":"2024-01-15T18:03:27.321499+0000","last_clean":"2024-01-15T18:03:27.321499+0000","last_became_active":"2024-01-15T18:03:27.318855+0000","last_became_peered":"2024-01-15T18:03:27.318855+0000","last_unstale":"2024-01-15T18:03:27.321499+0000","last_undegraded":"2024-01-15T18:03:27.321499+0000","last_fullsized":"2024-01-15T18:03:27.321499+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:40.231033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284505+0000","last_change":"2024-01-15T18:03:26.284505+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284505+0000","last_undegraded":"2024-01-15T18:03:26.284505+0000","last_fullsized":"2024-01-15T18:03:26.284505+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.321938+0000","last_change":"2024-01-15T18:03:27.316226+0000","last_active":"2024-01-15T18:03:27.321938+0000","last_peered":"2024-01-15T18:03:27.321938+0000","last_clean":"2024-01-15T18:03:27.321938+0000","last_became_active":"2024-01-15T18:03:27.315889+0000","last_became_peered":"2024-01-15T18:03:27.315889+0000","last_unstale":"2024-01-15T18:03:27.321938+0000","last_undegraded":"2024-01-15T18:03:27.321938+0000","last_fullsized":"2024-01-15T18:03:27.321938+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:56.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.277643+0000","last_change":"2024-01-15T18:03:26.277643+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.277643+0000","last_undegraded":"2024-01-15T18:03:26.277643+0000","last_fullsized":"2024-01-15T18:03:26.277643+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.275264+0000","last_change":"2024-01-15T18:03:26.275264+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.275264+0000","last_undegraded":"2024-01-15T18:03:26.275264+0000","last_fullsized":"2024-01-15T18:03:26.275264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282234+0000","last_change":"2024-01-15T18:03:26.282234+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282234+0000","last_undegraded":"2024-01-15T18:03:26.282234+0000","last_fullsized":"2024-01-15T18:03:26.282234+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.288002+0000","last_change":"2024-01-15T18:03:26.288002+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.288002+0000","last_undegraded":"2024-01-15T18:03:26.288002+0000","last_fullsized":"2024-01-15T18:03:26.288002+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282326+0000","last_change":"2024-01-15T18:03:26.282326+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282326+0000","last_undegraded":"2024-01-15T18:03:26.282326+0000","last_fullsized":"2024-01-15T18:03:26.282326+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274577+0000","last_change":"2024-01-15T18:03:26.274577+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274577+0000","last_undegraded":"2024-01-15T18:03:26.274577+0000","last_fullsized":"2024-01-15T18:03:26.274577+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284462+0000","last_change":"2024-01-15T18:03:26.284462+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284462+0000","last_undegraded":"2024-01-15T18:03:26.284462+0000","last_fullsized":"2024-01-15T18:03:26.284462+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.275307+0000","last_change":"2024-01-15T18:03:26.275307+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.275307+0000","last_undegraded":"2024-01-15T18:03:26.275307+0000","last_fullsized":"2024-01-15T18:03:26.275307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.276377+0000","last_change":"2024-01-15T18:03:26.276377+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.276377+0000","last_undegraded":"2024-01-15T18:03:26.276377+0000","last_fullsized":"2024-01-15T18:03:26.276377+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.286446+0000","last_change":"2024-01-15T18:03:26.286446+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.286446+0000","last_undegraded":"2024-01-15T18:03:26.286446+0000","last_fullsized":"2024-01-15T18:03:26.286446+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341755+0000","last_change":"2024-01-15T18:03:27.338146+0000","last_active":"2024-01-15T18:03:27.341755+0000","last_peered":"2024-01-15T18:03:27.341755+0000","last_clean":"2024-01-15T18:03:27.341755+0000","last_became_active":"2024-01-15T18:03:27.337850+0000","last_became_peered":"2024-01-15T18:03:27.337850+0000","last_unstale":"2024-01-15T18:03:27.341755+0000","last_undegraded":"2024-01-15T18:03:27.341755+0000","last_fullsized":"2024-01-15T18:03:27.341755+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:06:27.744895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282114+0000","last_change":"2024-01-15T18:03:26.282114+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282114+0000","last_undegraded":"2024-01-15T18:03:26.282114+0000","last_fullsized":"2024-01-15T18:03:26.282114+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.276325+0000","last_change":"2024-01-15T18:03:26.276325+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.276325+0000","last_undegraded":"2024-01-15T18:03:26.276325+0000","last_fullsized":"2024-01-15T18:03:26.276325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274415+0000","last_change":"2024-01-15T18:03:26.274415+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274415+0000","last_undegraded":"2024-01-15T18:03:26.274415+0000","last_fullsized":"2024-01-15T18:03:26.274415+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279090+0000","last_change":"2024-01-15T18:03:26.279090+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279090+0000","last_undegraded":"2024-01-15T18:03:26.279090+0000","last_fullsized":"2024-01-15T18:03:26.279090+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279397+0000","last_change":"2024-01-15T18:03:26.279397+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279397+0000","last_undegraded":"2024-01-15T18:03:26.279397+0000","last_fullsized":"2024-01-15T18:03:26.279397+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.276036+0000","last_change":"2024-01-15T18:03:26.276036+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.276036+0000","last_undegraded":"2024-01-15T18:03:26.276036+0000","last_fullsized":"2024-01-15T18:03:26.276036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284633+0000","last_change":"2024-01-15T18:03:26.284633+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284633+0000","last_undegraded":"2024-01-15T18:03:26.284633+0000","last_fullsized":"2024-01-15T18:03:26.284633+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283351+0000","last_change":"2024-01-15T18:03:26.283351+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283351+0000","last_undegraded":"2024-01-15T18:03:26.283351+0000","last_fullsized":"2024-01-15T18:03:26.283351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274031+0000","last_change":"2024-01-15T18:03:26.274031+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274031+0000","last_undegraded":"2024-01-15T18:03:26.274031+0000","last_fullsized":"2024-01-15T18:03:26.274031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282955+0000","last_change":"2024-01-15T18:03:26.282955+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282955+0000","last_undegraded":"2024-01-15T18:03:26.282955+0000","last_fullsized":"2024-01-15T18:03:26.282955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283969+0000","last_change":"2024-01-15T18:03:26.283969+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283969+0000","last_undegraded":"2024-01-15T18:03:26.283969+0000","last_fullsized":"2024-01-15T18:03:26.283969+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283167+0000","last_change":"2024-01-15T18:03:26.283167+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283167+0000","last_undegraded":"2024-01-15T18:03:26.283167+0000","last_fullsized":"2024-01-15T18:03:26.283167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.324636+0000","last_change":"2024-01-15T18:03:27.321263+0000","last_active":"2024-01-15T18:03:27.324636+0000","last_peered":"2024-01-15T18:03:27.324636+0000","last_clean":"2024-01-15T18:03:27.324636+0000","last_became_active":"2024-01-15T18:03:27.320958+0000","last_became_peered":"2024-01-15T18:03:27.320958+0000","last_unstale":"2024-01-15T18:03:27.324636+0000","last_undegraded":"2024-01-15T18:03:27.324636+0000","last_fullsized":"2024-01-15T18:03:27.324636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:43:50.724930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279714+0000","last_change":"2024-01-15T18:03:26.279714+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279714+0000","last_undegraded":"2024-01-15T18:03:26.279714+0000","last_fullsized":"2024-01-15T18:03:26.279714+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299516+0000","last_change":"2024-01-15T18:03:26.298227+0000","last_active":"2024-01-15T18:03:26.299516+0000","last_peered":"2024-01-15T18:03:26.299516+0000","last_clean":"2024-01-15T18:03:26.299516+0000","last_became_active":"2024-01-15T18:03:26.297015+0000","last_became_peered":"2024-01-15T18:03:26.297015+0000","last_unstale":"2024-01-15T18:03:26.299516+0000","last_undegraded":"2024-01-15T18:03:26.299516+0000","last_fullsized":"2024-01-15T18:03:26.299516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:43.508315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274116+0000","last_change":"2024-01-15T18:03:26.274116+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274116+0000","last_undegraded":"2024-01-15T18:03:26.274116+0000","last_fullsized":"2024-01-15T18:03:26.274116+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295089+0000","last_change":"2024-01-15T18:03:26.292579+0000","last_active":"2024-01-15T18:03:26.295089+0000","last_peered":"2024-01-15T18:03:26.295089+0000","last_clean":"2024-01-15T18:03:26.295089+0000","last_became_active":"2024-01-15T18:03:26.291631+0000","last_became_peered":"2024-01-15T18:03:26.291631+0000","last_unstale":"2024-01-15T18:03:26.295089+0000","last_undegraded":"2024-01-15T18:03:26.295089+0000","last_fullsized":"2024-01-15T18:03:26.295089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:32.983343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341535+0000","last_change":"2024-01-15T18:03:27.336001+0000","last_active":"2024-01-15T18:03:27.341535+0000","last_peered":"2024-01-15T18:03:27.341535+0000","last_clean":"2024-01-15T18:03:27.341535+0000","last_became_active":"2024-01-15T18:03:27.335265+0000","last_became_peered":"2024-01-15T18:03:27.335265+0000","last_unstale":"2024-01-15T18:03:27.341535+0000","last_undegraded":"2024-01-15T18:03:27.341535+0000","last_fullsized":"2024-01-15T18:03:27.341535+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:52.859519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.304785+0000","last_change":"2024-01-15T18:03:26.303968+0000","last_active":"2024-01-15T18:03:26.304785+0000","last_peered":"2024-01-15T18:03:26.304785+0000","last_clean":"2024-01-15T18:03:26.304785+0000","last_became_active":"2024-01-15T18:03:26.303669+0000","last_became_peered":"2024-01-15T18:03:26.303669+0000","last_unstale":"2024-01-15T18:03:26.304785+0000","last_undegraded":"2024-01-15T18:03:26.304785+0000","last_fullsized":"2024-01-15T18:03:26.304785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:43.710985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281710+0000","last_change":"2024-01-15T18:03:26.281710+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281710+0000","last_undegraded":"2024-01-15T18:03:26.281710+0000","last_fullsized":"2024-01-15T18:03:26.281710+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342311+0000","last_change":"2024-01-15T18:03:26.300990+0000","last_active":"2024-01-15T18:03:27.342311+0000","last_peered":"2024-01-15T18:03:27.342311+0000","last_clean":"2024-01-15T18:03:27.342311+0000","last_became_active":"2024-01-15T18:03:26.300735+0000","last_became_peered":"2024-01-15T18:03:26.300735+0000","last_unstale":"2024-01-15T18:03:27.342311+0000","last_undegraded":"2024-01-15T18:03:27.342311+0000","last_fullsized":"2024-01-15T18:03:27.342311+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:53:06.533003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283325+0000","last_change":"2024-01-15T18:03:26.283325+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283325+0000","last_undegraded":"2024-01-15T18:03:26.283325+0000","last_fullsized":"2024-01-15T18:03:26.283325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297810+0000","last_change":"2024-01-15T18:03:26.296481+0000","last_active":"2024-01-15T18:03:26.297810+0000","last_peered":"2024-01-15T18:03:26.297810+0000","last_clean":"2024-01-15T18:03:26.297810+0000","last_became_active":"2024-01-15T18:03:26.293509+0000","last_became_peered":"2024-01-15T18:03:26.293509+0000","last_unstale":"2024-01-15T18:03:26.297810+0000","last_undegraded":"2024-01-15T18:03:26.297810+0000","last_fullsized":"2024-01-15T18:03:26.297810+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:48.220096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274045+0000","last_change":"2024-01-15T18:03:26.274045+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274045+0000","last_undegraded":"2024-01-15T18:03:26.274045+0000","last_fullsized":"2024-01-15T18:03:26.274045+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.340684+0000","last_change":"2024-01-15T18:03:26.302584+0000","last_active":"2024-01-15T18:03:27.340684+0000","last_peered":"2024-01-15T18:03:27.340684+0000","last_clean":"2024-01-15T18:03:27.340684+0000","last_became_active":"2024-01-15T18:03:26.301356+0000","last_became_peered":"2024-01-15T18:03:26.301356+0000","last_unstale":"2024-01-15T18:03:27.340684+0000","last_undegraded":"2024-01-15T18:03:27.340684+0000","last_fullsized":"2024-01-15T18:03:27.340684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:22:00.115460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.272409+0000","last_change":"2024-01-15T18:03:26.272409+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.272409+0000","last_undegraded":"2024-01-15T18:03:26.272409+0000","last_fullsized":"2024-01-15T18:03:26.272409+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.293022+0000","last_change":"2024-01-15T18:03:26.282542+0000","last_active":"2024-01-15T18:03:26.293022+0000","last_peered":"2024-01-15T18:03:26.293022+0000","last_clean":"2024-01-15T18:03:26.293022+0000","last_became_active":"2024-01-15T18:03:26.282226+0000","last_became_peered":"2024-01-15T18:03:26.282226+0000","last_unstale":"2024-01-15T18:03:26.293022+0000","last_undegraded":"2024-01-15T18:03:26.293022+0000","last_fullsized":"2024-01-15T18:03:26.293022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:57.562173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281723+0000","last_change":"2024-01-15T18:03:26.281723+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281723+0000","last_undegraded":"2024-01-15T18:03:26.281723+0000","last_fullsized":"2024-01-15T18:03:26.281723+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.289032+0000","last_change":"2024-01-15T18:03:26.285864+0000","last_active":"2024-01-15T18:03:26.289032+0000","last_peered":"2024-01-15T18:03:26.289032+0000","last_clean":"2024-01-15T18:03:26.289032+0000","last_became_active":"2024-01-15T18:03:26.285507+0000","last_became_peered":"2024-01-15T18:03:26.285507+0000","last_unstale":"2024-01-15T18:03:26.289032+0000","last_undegraded":"2024-01-15T18:03:26.289032+0000","last_fullsized":"2024-01-15T18:03:26.289032+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:28:37.864230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283831+0000","last_change":"2024-01-15T18:03:26.283831+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283831+0000","last_undegraded":"2024-01-15T18:03:26.283831+0000","last_fullsized":"2024-01-15T18:03:26.283831+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295148+0000","last_change":"2024-01-15T18:03:26.294412+0000","last_active":"2024-01-15T18:03:26.295148+0000","last_peered":"2024-01-15T18:03:26.295148+0000","last_clean":"2024-01-15T18:03:26.295148+0000","last_became_active":"2024-01-15T18:03:26.294070+0000","last_became_peered":"2024-01-15T18:03:26.294070+0000","last_unstale":"2024-01-15T18:03:26.295148+0000","last_undegraded":"2024-01-15T18:03:26.295148+0000","last_fullsized":"2024-01-15T18:03:26.295148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:06:26.733111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.280249+0000","last_change":"2024-01-15T18:03:26.280249+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.280249+0000","last_undegraded":"2024-01-15T18:03:26.280249+0000","last_fullsized":"2024-01-15T18:03:26.280249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.312561+0000","last_change":"2024-01-15T18:03:26.302162+0000","last_active":"2024-01-15T18:03:27.312561+0000","last_peered":"2024-01-15T18:03:27.312561+0000","last_clean":"2024-01-15T18:03:27.312561+0000","last_became_active":"2024-01-15T18:03:26.301821+0000","last_became_peered":"2024-01-15T18:03:26.301821+0000","last_unstale":"2024-01-15T18:03:27.312561+0000","last_undegraded":"2024-01-15T18:03:27.312561+0000","last_fullsized":"2024-01-15T18:03:27.312561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:24:07.021223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.340063+0000","last_change":"2024-01-15T18:03:27.329589+0000","last_active":"2024-01-15T18:03:27.340063+0000","last_peered":"2024-01-15T18:03:27.340063+0000","last_clean":"2024-01-15T18:03:27.340063+0000","last_became_active":"2024-01-15T18:03:27.328118+0000","last_became_peered":"2024-01-15T18:03:27.328118+0000","last_unstale":"2024-01-15T18:03:27.340063+0000","last_undegraded":"2024-01-15T18:03:27.340063+0000","last_fullsized":"2024-01-15T18:03:27.340063+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:48.991808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.296636+0000","last_change":"2024-01-15T18:03:26.295197+0000","last_active":"2024-01-15T18:03:26.296636+0000","last_peered":"2024-01-15T18:03:26.296636+0000","last_clean":"2024-01-15T18:03:26.296636+0000","last_became_active":"2024-01-15T18:03:26.294932+0000","last_became_peered":"2024-01-15T18:03:26.294932+0000","last_unstale":"2024-01-15T18:03:26.296636+0000","last_undegraded":"2024-01-15T18:03:26.296636+0000","last_fullsized":"2024-01-15T18:03:26.296636+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:20.869224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.273058+0000","last_change":"2024-01-15T18:03:26.273058+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.273058+0000","last_undegraded":"2024-01-15T18:03:26.273058+0000","last_fullsized":"2024-01-15T18:03:26.273058+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294412+0000","last_change":"2024-01-15T18:03:26.283744+0000","last_active":"2024-01-15T18:03:26.294412+0000","last_peered":"2024-01-15T18:03:26.294412+0000","last_clean":"2024-01-15T18:03:26.294412+0000","last_became_active":"2024-01-15T18:03:26.283382+0000","last_became_peered":"2024-01-15T18:03:26.283382+0000","last_unstale":"2024-01-15T18:03:26.294412+0000","last_undegraded":"2024-01-15T18:03:26.294412+0000","last_fullsized":"2024-01-15T18:03:26.294412+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:13:56.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278423+0000","last_change":"2024-01-15T18:03:26.278423+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278423+0000","last_undegraded":"2024-01-15T18:03:26.278423+0000","last_fullsized":"2024-01-15T18:03:26.278423+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295393+0000","last_change":"2024-01-15T18:03:26.290460+0000","last_active":"2024-01-15T18:03:26.295393+0000","last_peered":"2024-01-15T18:03:26.295393+0000","last_clean":"2024-01-15T18:03:26.295393+0000","last_became_active":"2024-01-15T18:03:26.290114+0000","last_became_peered":"2024-01-15T18:03:26.290114+0000","last_unstale":"2024-01-15T18:03:26.295393+0000","last_undegraded":"2024-01-15T18:03:26.295393+0000","last_fullsized":"2024-01-15T18:03:26.295393+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:49.162905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.286454+0000","last_change":"2024-01-15T18:03:26.286454+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.286454+0000","last_undegraded":"2024-01-15T18:03:26.286454+0000","last_fullsized":"2024-01-15T18:03:26.286454+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.306852+0000","last_change":"2024-01-15T18:03:26.302531+0000","last_active":"2024-01-15T18:03:27.306852+0000","last_peered":"2024-01-15T18:03:27.306852+0000","last_clean":"2024-01-15T18:03:27.306852+0000","last_became_active":"2024-01-15T18:03:26.301660+0000","last_became_peered":"2024-01-15T18:03:26.301660+0000","last_unstale":"2024-01-15T18:03:27.306852+0000","last_undegraded":"2024-01-15T18:03:27.306852+0000","last_fullsized":"2024-01-15T18:03:27.306852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:18:48.053993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281591+0000","last_change":"2024-01-15T18:03:26.281591+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281591+0000","last_undegraded":"2024-01-15T18:03:26.281591+0000","last_fullsized":"2024-01-15T18:03:26.281591+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299435+0000","last_change":"2024-01-15T18:03:26.297958+0000","last_active":"2024-01-15T18:03:26.299435+0000","last_peered":"2024-01-15T18:03:26.299435+0000","last_clean":"2024-01-15T18:03:26.299435+0000","last_became_active":"2024-01-15T18:03:26.297417+0000","last_became_peered":"2024-01-15T18:03:26.297417+0000","last_unstale":"2024-01-15T18:03:26.299435+0000","last_undegraded":"2024-01-15T18:03:26.299435+0000","last_fullsized":"2024-01-15T18:03:26.299435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:20:43.839295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.272601+0000","last_change":"2024-01-15T18:03:26.272601+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.272601+0000","last_undegraded":"2024-01-15T18:03:26.272601+0000","last_fullsized":"2024-01-15T18:03:26.272601+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.293724+0000","last_change":"2024-01-15T18:03:26.287860+0000","last_active":"2024-01-15T18:03:26.293724+0000","last_peered":"2024-01-15T18:03:26.293724+0000","last_clean":"2024-01-15T18:03:26.293724+0000","last_became_active":"2024-01-15T18:03:26.287555+0000","last_became_peered":"2024-01-15T18:03:26.287555+0000","last_unstale":"2024-01-15T18:03:26.293724+0000","last_undegraded":"2024-01-15T18:03:26.293724+0000","last_fullsized":"2024-01-15T18:03:26.293724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:50.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274473+0000","last_change":"2024-01-15T18:03:26.274473+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274473+0000","last_undegraded":"2024-01-15T18:03:26.274473+0000","last_fullsized":"2024-01-15T18:03:26.274473+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341894+0000","last_change":"2024-01-15T18:03:26.300342+0000","last_active":"2024-01-15T18:03:27.341894+0000","last_peered":"2024-01-15T18:03:27.341894+0000","last_clean":"2024-01-15T18:03:27.341894+0000","last_became_active":"2024-01-15T18:03:26.300100+0000","last_became_peered":"2024-01-15T18:03:26.300100+0000","last_unstale":"2024-01-15T18:03:27.341894+0000","last_undegraded":"2024-01-15T18:03:27.341894+0000","last_fullsized":"2024-01-15T18:03:27.341894+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:08.274277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279678+0000","last_change":"2024-01-15T18:03:26.279678+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279678+0000","last_undegraded":"2024-01-15T18:03:26.279678+0000","last_fullsized":"2024-01-15T18:03:26.279678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.303643+0000","last_change":"2024-01-15T18:03:26.298853+0000","last_active":"2024-01-15T18:03:26.303643+0000","last_peered":"2024-01-15T18:03:26.303643+0000","last_clean":"2024-01-15T18:03:26.303643+0000","last_became_active":"2024-01-15T18:03:26.298530+0000","last_became_peered":"2024-01-15T18:03:26.298530+0000","last_unstale":"2024-01-15T18:03:26.303643+0000","last_undegraded":"2024-01-15T18:03:26.303643+0000","last_fullsized":"2024-01-15T18:03:26.303643+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:51.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284222+0000","last_change":"2024-01-15T18:03:26.284222+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284222+0000","last_undegraded":"2024-01-15T18:03:26.284222+0000","last_fullsized":"2024-01-15T18:03:26.284222+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.343468+0000","last_change":"2024-01-15T18:03:26.301874+0000","last_active":"2024-01-15T18:03:27.343468+0000","last_peered":"2024-01-15T18:03:27.343468+0000","last_clean":"2024-01-15T18:03:27.343468+0000","last_became_active":"2024-01-15T18:03:26.301425+0000","last_became_peered":"2024-01-15T18:03:26.301425+0000","last_unstale":"2024-01-15T18:03:27.343468+0000","last_undegraded":"2024-01-15T18:03:27.343468+0000","last_fullsized":"2024-01-15T18:03:27.343468+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:43.710985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.272442+0000","last_change":"2024-01-15T18:03:26.272442+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.272442+0000","last_undegraded":"2024-01-15T18:03:26.272442+0000","last_fullsized":"2024-01-15T18:03:26.272442+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299247+0000","last_change":"2024-01-15T18:03:26.295610+0000","last_active":"2024-01-15T18:03:26.299247+0000","last_peered":"2024-01-15T18:03:26.299247+0000","last_clean":"2024-01-15T18:03:26.299247+0000","last_became_active":"2024-01-15T18:03:26.295164+0000","last_became_peered":"2024-01-15T18:03:26.295164+0000","last_unstale":"2024-01-15T18:03:26.299247+0000","last_undegraded":"2024-01-15T18:03:26.299247+0000","last_fullsized":"2024-01-15T18:03:26.299247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:42.703538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279636+0000","last_change":"2024-01-15T18:03:26.279636+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279636+0000","last_undegraded":"2024-01-15T18:03:26.279636+0000","last_fullsized":"2024-01-15T18:03:26.279636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.316666+0000","last_change":"2024-01-15T18:03:26.301741+0000","last_active":"2024-01-15T18:03:27.316666+0000","last_peered":"2024-01-15T18:03:27.316666+0000","last_clean":"2024-01-15T18:03:27.316666+0000","last_became_active":"2024-01-15T18:03:26.301228+0000","last_became_peered":"2024-01-15T18:03:26.301228+0000","last_unstale":"2024-01-15T18:03:27.316666+0000","last_undegraded":"2024-01-15T18:03:27.316666+0000","last_fullsized":"2024-01-15T18:03:27.316666+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:08.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278650+0000","last_change":"2024-01-15T18:03:26.278650+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278650+0000","last_undegraded":"2024-01-15T18:03:26.278650+0000","last_fullsized":"2024-01-15T18:03:26.278650+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294795+0000","last_change":"2024-01-15T18:03:26.293020+0000","last_active":"2024-01-15T18:03:26.294795+0000","last_peered":"2024-01-15T18:03:26.294795+0000","last_clean":"2024-01-15T18:03:26.294795+0000","last_became_active":"2024-01-15T18:03:26.292663+0000","last_became_peered":"2024-01-15T18:03:26.292663+0000","last_unstale":"2024-01-15T18:03:26.294795+0000","last_undegraded":"2024-01-15T18:03:26.294795+0000","last_fullsized":"2024-01-15T18:03:26.294795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:51.697653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278555+0000","last_change":"2024-01-15T18:03:26.278555+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278555+0000","last_undegraded":"2024-01-15T18:03:26.278555+0000","last_fullsized":"2024-01-15T18:03:26.278555+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295435+0000","last_change":"2024-01-15T18:03:26.292741+0000","last_active":"2024-01-15T18:03:26.295435+0000","last_peered":"2024-01-15T18:03:26.295435+0000","last_clean":"2024-01-15T18:03:26.295435+0000","last_became_active":"2024-01-15T18:03:26.292349+0000","last_became_peered":"2024-01-15T18:03:26.292349+0000","last_unstale":"2024-01-15T18:03:26.295435+0000","last_undegraded":"2024-01-15T18:03:26.295435+0000","last_fullsized":"2024-01-15T18:03:26.295435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:19:42.341080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282316+0000","last_change":"2024-01-15T18:03:26.282316+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282316+0000","last_undegraded":"2024-01-15T18:03:26.282316+0000","last_fullsized":"2024-01-15T18:03:26.282316+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341991+0000","last_change":"2024-01-15T18:03:26.299863+0000","last_active":"2024-01-15T18:03:27.341991+0000","last_peered":"2024-01-15T18:03:27.341991+0000","last_clean":"2024-01-15T18:03:27.341991+0000","last_became_active":"2024-01-15T18:03:26.299525+0000","last_became_peered":"2024-01-15T18:03:26.299525+0000","last_unstale":"2024-01-15T18:03:27.341991+0000","last_undegraded":"2024-01-15T18:03:27.341991+0000","last_fullsized":"2024-01-15T18:03:27.341991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:51.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284466+0000","last_change":"2024-01-15T18:03:26.284466+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284466+0000","last_undegraded":"2024-01-15T18:03:26.284466+0000","last_fullsized":"2024-01-15T18:03:26.284466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299404+0000","last_change":"2024-01-15T18:03:26.297682+0000","last_active":"2024-01-15T18:03:26.299404+0000","last_peered":"2024-01-15T18:03:26.299404+0000","last_clean":"2024-01-15T18:03:26.299404+0000","last_became_active":"2024-01-15T18:03:26.297319+0000","last_became_peered":"2024-01-15T18:03:26.297319+0000","last_unstale":"2024-01-15T18:03:26.299404+0000","last_undegraded":"2024-01-15T18:03:26.299404+0000","last_fullsized":"2024-01-15T18:03:26.299404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:40:28.369164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278628+0000","last_change":"2024-01-15T18:03:26.278628+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278628+0000","last_undegraded":"2024-01-15T18:03:26.278628+0000","last_fullsized":"2024-01-15T18:03:26.278628+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.292204+0000","last_change":"2024-01-15T18:03:26.290405+0000","last_active":"2024-01-15T18:03:26.292204+0000","last_peered":"2024-01-15T18:03:26.292204+0000","last_clean":"2024-01-15T18:03:26.292204+0000","last_became_active":"2024-01-15T18:03:26.288657+0000","last_became_peered":"2024-01-15T18:03:26.288657+0000","last_unstale":"2024-01-15T18:03:26.292204+0000","last_undegraded":"2024-01-15T18:03:26.292204+0000","last_fullsized":"2024-01-15T18:03:26.292204+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:44.557559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281044+0000","last_change":"2024-01-15T18:03:26.281044+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281044+0000","last_undegraded":"2024-01-15T18:03:26.281044+0000","last_fullsized":"2024-01-15T18:03:26.281044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297486+0000","last_change":"2024-01-15T18:03:26.296161+0000","last_active":"2024-01-15T18:03:26.297486+0000","last_peered":"2024-01-15T18:03:26.297486+0000","last_clean":"2024-01-15T18:03:26.297486+0000","last_became_active":"2024-01-15T18:03:26.295339+0000","last_became_peered":"2024-01-15T18:03:26.295339+0000","last_unstale":"2024-01-15T18:03:26.297486+0000","last_undegraded":"2024-01-15T18:03:26.297486+0000","last_fullsized":"2024-01-15T18:03:26.297486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:20:43.839295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.272331+0000","last_change":"2024-01-15T18:03:26.272331+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.272331+0000","last_undegraded":"2024-01-15T18:03:26.272331+0000","last_fullsized":"2024-01-15T18:03:26.272331+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299430+0000","last_change":"2024-01-15T18:03:26.296244+0000","last_active":"2024-01-15T18:03:26.299430+0000","last_peered":"2024-01-15T18:03:26.299430+0000","last_clean":"2024-01-15T18:03:26.299430+0000","last_became_active":"2024-01-15T18:03:26.295955+0000","last_became_peered":"2024-01-15T18:03:26.295955+0000","last_unstale":"2024-01-15T18:03:26.299430+0000","last_undegraded":"2024-01-15T18:03:26.299430+0000","last_fullsized":"2024-01-15T18:03:26.299430+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:39.932581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.331003+0000","last_change":"2024-01-15T18:03:27.323628+0000","last_active":"2024-01-15T18:03:27.331003+0000","last_peered":"2024-01-15T18:03:27.331003+0000","last_clean":"2024-01-15T18:03:27.331003+0000","last_became_active":"2024-01-15T18:03:27.323353+0000","last_became_peered":"2024-01-15T18:03:27.323353+0000","last_unstale":"2024-01-15T18:03:27.331003+0000","last_undegraded":"2024-01-15T18:03:27.331003+0000","last_fullsized":"2024-01-15T18:03:27.331003+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:12.045333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297615+0000","last_change":"2024-01-15T18:03:26.295494+0000","last_active":"2024-01-15T18:03:26.297615+0000","last_peered":"2024-01-15T18:03:26.297615+0000","last_clean":"2024-01-15T18:03:26.297615+0000","last_became_active":"2024-01-15T18:03:26.295221+0000","last_became_peered":"2024-01-15T18:03:26.295221+0000","last_unstale":"2024-01-15T18:03:26.297615+0000","last_undegraded":"2024-01-15T18:03:26.297615+0000","last_fullsized":"2024-01-15T18:03:26.297615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:33.869808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.285274+0000","last_change":"2024-01-15T18:03:26.285274+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.285274+0000","last_undegraded":"2024-01-15T18:03:26.285274+0000","last_fullsized":"2024-01-15T18:03:26.285274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294031+0000","last_change":"2024-01-15T18:03:26.291361+0000","last_active":"2024-01-15T18:03:26.294031+0000","last_peered":"2024-01-15T18:03:26.294031+0000","last_clean":"2024-01-15T18:03:26.294031+0000","last_became_active":"2024-01-15T18:03:26.290863+0000","last_became_peered":"2024-01-15T18:03:26.290863+0000","last_unstale":"2024-01-15T18:03:26.294031+0000","last_undegraded":"2024-01-15T18:03:26.294031+0000","last_fullsized":"2024-01-15T18:03:26.294031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:24:07.021223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.332585+0000","last_change":"2024-01-15T18:03:27.325192+0000","last_active":"2024-01-15T18:03:27.332585+0000","last_peered":"2024-01-15T18:03:27.332585+0000","last_clean":"2024-01-15T18:03:27.332585+0000","last_became_active":"2024-01-15T18:03:27.324697+0000","last_became_peered":"2024-01-15T18:03:27.324697+0000","last_unstale":"2024-01-15T18:03:27.332585+0000","last_undegraded":"2024-01-15T18:03:27.332585+0000","last_fullsized":"2024-01-15T18:03:27.332585+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:20.920788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.308092+0000","last_change":"2024-01-15T18:03:26.298754+0000","last_active":"2024-01-15T18:03:27.308092+0000","last_peered":"2024-01-15T18:03:27.308092+0000","last_clean":"2024-01-15T18:03:27.308092+0000","last_became_active":"2024-01-15T18:03:26.298412+0000","last_became_peered":"2024-01-15T18:03:26.298412+0000","last_unstale":"2024-01-15T18:03:27.308092+0000","last_undegraded":"2024-01-15T18:03:27.308092+0000","last_fullsized":"2024-01-15T18:03:27.308092+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:43:09.251315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278407+0000","last_change":"2024-01-15T18:03:26.278407+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278407+0000","last_undegraded":"2024-01-15T18:03:26.278407+0000","last_fullsized":"2024-01-15T18:03:26.278407+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342463+0000","last_change":"2024-01-15T18:03:26.300437+0000","last_active":"2024-01-15T18:03:27.342463+0000","last_peered":"2024-01-15T18:03:27.342463+0000","last_clean":"2024-01-15T18:03:27.342463+0000","last_became_active":"2024-01-15T18:03:26.300210+0000","last_became_peered":"2024-01-15T18:03:26.300210+0000","last_unstale":"2024-01-15T18:03:27.342463+0000","last_undegraded":"2024-01-15T18:03:27.342463+0000","last_fullsized":"2024-01-15T18:03:27.342463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:47.498523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":206,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-01-15T18:03:25.275709+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:03:25.275709+0000","last_peered":"2024-01-15T18:03:25.275709+0000","last_clean":"2024-01-15T18:03:25.275709+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:03:25.275709+0000","last_undegraded":"2024-01-15T18:03:25.275709+0000","last_fullsized":"2024-01-15T18:03:25.275709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1441792,"data_stored":947133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.04,"5min":1.6439999999999999,"15min":1.3180000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":19.843,"5min":19.843,"15min":19.843},"last":0.878},{"interface":"front","average":{"1min":4.2229999999999999,"5min":2.3180000000000001,"15min":1.552},"min":{"1min":0.75700000000000001,"5min":0.629,"15min":0.623},"max":{"1min":30.498000000000001,"5min":38.137,"15min":38.137},"last":1.0149999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.129,"5min":1.716,"15min":1.383},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":19.917000000000002,"5min":19.917000000000002,"15min":19.917000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":2.2839999999999998,"5min":2.109,"15min":1.546},"min":{"1min":0.71199999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":10.361000000000001,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.143}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":1.6910000000000001,"15min":1.345},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":8.1289999999999996,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":0.94899999999999995},{"interface":"front","average":{"1min":3.0760000000000001,"5min":1.8149999999999999,"15min":1.5169999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":13.478999999999999,"5min":13.478999999999999,"15min":13.478999999999999},"last":1.202}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.3359999999999999,"5min":1.9450000000000001,"15min":1.5129999999999999},"min":{"1min":0.56000000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":16.632000000000001,"5min":16.632000000000001,"15min":16.632000000000001},"last":1.284},{"interface":"front","average":{"1min":3.177,"5min":1.867,"15min":1.617},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":13.439,"5min":13.439,"15min":13.439},"last":1.4279999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":2.145,"15min":1.694},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50700000000000001},"max":{"1min":10.881,"5min":22.359999999999999,"15min":22.359999999999999},"last":1.254},{"interface":"front","average":{"1min":4.4820000000000002,"5min":2.2149999999999999,"15min":1.702},"min":{"1min":0.52700000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":38.682000000000002,"5min":38.682000000000002,"15min":38.682000000000002},"last":1.633}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.8140000000000001,"5min":2.286,"15min":1.831},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":13.379,"5min":25.006,"15min":25.006},"last":1.4950000000000001},{"interface":"front","average":{"1min":2.9169999999999998,"5min":2.2669999999999999,"15min":1.8069999999999999},"min":{"1min":0.80200000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":15.664999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.661}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.4729999999999999,"5min":2.5,"15min":1.954},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.59299999999999997},"max":{"1min":11.366,"5min":23.834,"15min":23.834},"last":1.794},{"interface":"front","average":{"1min":2.8380000000000001,"5min":2.8050000000000002,"15min":2.012},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":9.2599999999999998,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.7310000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45900,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325940,"statfs":{"total":96636764160,"available":96589762560,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.325,"15min":1.3340000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":7.79,"5min":7.79,"15min":12.544},"last":0.88500000000000001},{"interface":"front","average":{"1min":2.1379999999999999,"5min":1.579,"15min":1.3149999999999999},"min":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":14.013,"5min":14.013,"15min":14.013},"last":1.5009999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.403,"15min":1.284},"min":{"1min":0.67400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":7.6200000000000001,"5min":7.6200000000000001,"15min":12.811999999999999},"last":1.107},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.8999999999999999,"15min":1.4199999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":20.523,"5min":20.523,"15min":20.523},"last":1.4199999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":1.877,"15min":1.4279999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":17.713000000000001,"5min":17.713000000000001,"15min":17.713000000000001},"last":1.871},{"interface":"front","average":{"1min":1.788,"5min":1.3799999999999999,"15min":1.4850000000000001},"min":{"1min":0.59899999999999998,"5min":0.504,"15min":0.504},"max":{"1min":7.702,"5min":7.702,"15min":12.897},"last":1.006}]},{"osd":3,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0840000000000001,"5min":1.74,"15min":1.6000000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.7480000000000002,"5min":9.5,"15min":12.717000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":2.3410000000000002,"5min":1.825,"15min":1.518},"min":{"1min":0.876,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":13.795,"5min":14.066000000000001,"15min":14.066000000000001},"last":1.607}]},{"osd":4,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":4.0739999999999998,"5min":2.0659999999999998,"15min":1.6200000000000001},"min":{"1min":0.59599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.5760000000000001},{"interface":"front","average":{"1min":2.2650000000000001,"5min":1.774,"15min":1.6890000000000001},"min":{"1min":0.52000000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":7.8319999999999999,"5min":17.289000000000001,"15min":17.289000000000001},"last":1.248}]},{"osd":5,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.9529999999999998,"5min":1.819,"15min":1.5620000000000001},"min":{"1min":0.66900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":13.593,"5min":14.176,"15min":14.176},"last":1.8440000000000001},{"interface":"front","average":{"1min":3.4950000000000001,"5min":2.1110000000000002,"15min":1.849},"min":{"1min":0.621,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":17.949000000000002,"5min":17.949000000000002,"15min":17.949000000000002},"last":1.752}]},{"osd":7,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.4359999999999999,"5min":2.0190000000000001,"15min":1.708},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":21.265000000000001,"5min":21.265000000000001,"15min":21.265000000000001},"last":1.71},{"interface":"front","average":{"1min":4.3949999999999996,"5min":2.3439999999999999,"15min":1.9299999999999999},"min":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":19.571000000000002,"5min":19.571000000000002,"15min":19.571000000000002},"last":1.665}]}]},{"osd":5,"up_from":14,"seq":60129542248,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45900,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325940,"statfs":{"total":96636764160,"available":96589762560,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9890000000000001,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":19.266999999999999,"5min":19.266999999999999,"15min":19.266999999999999},"last":1.236},{"interface":"front","average":{"1min":1.292,"5min":1.607,"15min":1.4330000000000001},"min":{"1min":0.69399999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.1760000000000002,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.55}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":1.5469999999999999,"15min":1.3169999999999999},"min":{"1min":0.71299999999999997,"5min":0.499,"15min":0.499},"max":{"1min":9.1259999999999994,"5min":16.297000000000001,"15min":16.297000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.8899999999999999,"5min":1.3859999999999999,"15min":1.2629999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2379999999999995,"5min":9.2379999999999995,"15min":9.2379999999999995},"last":1.0569999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.345,"15min":1.262},"min":{"1min":0.88600000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.0369999999999999,"5min":8.1820000000000004,"15min":8.1820000000000004},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.8220000000000001,"5min":1.6479999999999999,"15min":1.3899999999999999},"min":{"1min":0.88500000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2159999999999993,"5min":16.158999999999999,"15min":16.158999999999999},"last":0.996}]},{"osd":3,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.5549999999999999,"15min":1.4370000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.56100000000000005},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":14.914},"last":1.341},{"interface":"front","average":{"1min":1.9930000000000001,"5min":1.526,"15min":1.4570000000000001},"min":{"1min":0.76300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":7.0110000000000001,"5min":7.0110000000000001,"15min":7.0110000000000001},"last":1.3879999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9850000000000001,"5min":1.6579999999999999,"15min":1.4870000000000001},"min":{"1min":0.76100000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":9.0410000000000004,"5min":16.414999999999999,"15min":16.414999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.925,"15min":1.627},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":9.1519999999999992,"5min":17.991,"15min":17.991},"last":1.51}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.492,"15min":1.573},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.669,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":2.0609999999999999,"5min":1.788,"15min":1.623},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":9.1649999999999991,"5min":16.364000000000001,"15min":16.364000000000001},"last":1.468}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9370000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"min":{"1min":0.66300000000000003,"5min":0.628,"15min":0.628},"max":{"1min":9.0600000000000005,"5min":9.0600000000000005,"15min":9.0600000000000005},"last":1.423},{"interface":"front","average":{"1min":3.0600000000000001,"5min":1.891,"15min":1.79},"min":{"1min":0.83999999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":19.356999999999999,"5min":19.356999999999999,"15min":19.356999999999999},"last":1.669}]}]},{"osd":4,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50060,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321780,"statfs":{"total":96636764160,"available":96585502720,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.5229999999999999,"15min":1.3149999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":7.8869999999999996,"5min":11.413,"15min":11.413},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.8180000000000001,"5min":1.534,"15min":1.401},"min":{"1min":0.63,"5min":0.63,"15min":0.53700000000000003},"max":{"1min":7.9160000000000004,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":0.84899999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.367,"15min":1.369},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.45000000000000001},"max":{"1min":3.5750000000000002,"5min":9.7010000000000005,"15min":11.987},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.444,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.6109999999999998,"5min":16.058,"15min":16.058},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.7789999999999999,"15min":1.575},"min":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.6890000000000001,"5min":13.865,"15min":13.865},"last":1.052},{"interface":"front","average":{"1min":1.788,"5min":1.833,"15min":1.5009999999999999},"min":{"1min":0.89100000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":5.7050000000000001,"5min":13.903,"15min":13.903},"last":1.3879999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.919,"5min":1.9099999999999999,"15min":1.675},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":11.773,"5min":26.055,"15min":26.055},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.673,"15min":1.611},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":4.0419999999999998,"5min":13.25,"15min":13.25},"last":1.004}]},{"osd":5,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":2.9729999999999999,"5min":1.853,"15min":1.72},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":29.422000000000001,"5min":29.422000000000001,"15min":29.422000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.5720000000000001,"15min":1.619},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.7160000000000002,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.216}]},{"osd":6,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.6439999999999999,"15min":1.6379999999999999},"min":{"1min":0.80100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":5.7919999999999998,"5min":18.260000000000002,"15min":18.260000000000002},"last":1.579},{"interface":"front","average":{"1min":1.597,"5min":1.6890000000000001,"15min":1.7849999999999999},"min":{"1min":0.59999999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.6459999999999999,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.7410000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.982,"5min":1.881,"15min":1.889},"min":{"1min":0.60499999999999998,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":10.664,"5min":10.664,"15min":11.894},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.851,"5min":2.0379999999999998,"15min":1.9390000000000001},"min":{"1min":0.61399999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.3319999999999999,"5min":11.714,"15min":11.714},"last":1.6679999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58376,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313464,"statfs":{"total":96636764160,"available":96576987136,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5390000000000001,"5min":1.5580000000000001,"15min":1.1830000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":18.399999999999999,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":2.7730000000000001,"5min":1.8500000000000001,"15min":1.349},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.795999999999999,"5min":21.047000000000001,"15min":21.047000000000001},"last":0.91000000000000003}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5499999999999998,"5min":1.736,"15min":1.304},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":12.372,"5min":17.777999999999999,"15min":17.777999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.367,"15min":1.179},"min":{"1min":0.498,"5min":0.48099999999999998,"15min":0.39500000000000002},"max":{"1min":11.002000000000001,"5min":11.002000000000001,"15min":11.002000000000001},"last":1.081}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.413,"15min":1.244},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":1.0309999999999999},{"interface":"front","average":{"1min":2.2599999999999998,"5min":1.72,"15min":1.488},"min":{"1min":0.63300000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":20.495000000000001,"5min":20.495000000000001,"15min":20.495000000000001},"last":0.98899999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4380000000000002,"5min":1.7310000000000001,"15min":1.4950000000000001},"min":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.63500000000000001},"max":{"1min":11.826000000000001,"5min":11.826000000000001,"15min":11.826000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":3.1379999999999999,"5min":1.9950000000000001,"15min":1.625},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.54600000000000004},"max":{"1min":12.439,"5min":12.439,"15min":12.439},"last":1.3500000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.867,"15min":1.6379999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.57299999999999995},"max":{"1min":20.606000000000002,"5min":20.606000000000002,"15min":20.606000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.161,"5min":1.8120000000000001,"15min":1.6870000000000001},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.68000000000000005},"max":{"1min":11.603,"5min":11.603,"15min":11.603},"last":1.9019999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.0129999999999999,"5min":2.1059999999999999,"15min":1.823},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":12.416,"5min":12.416,"15min":12.416},"last":1.8540000000000001},{"interface":"front","average":{"1min":2.391,"5min":1.962,"15min":1.768},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":20.556000000000001,"5min":20.556000000000001,"15min":20.556000000000001},"last":1.8280000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.8460000000000001,"5min":2.331,"15min":1.8999999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.70699999999999996},"max":{"1min":28.843,"5min":28.843,"15min":28.843},"last":2.0019999999999998},{"interface":"front","average":{"1min":3.4940000000000002,"5min":2.3199999999999998,"15min":1.9219999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":15.143000000000001,"5min":15.143000000000001,"15min":15.143000000000001},"last":1.8029999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50044,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321796,"statfs":{"total":96636764160,"available":96585519104,"internally_reserved":0,"allocated":835584,"data_stored":350434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.3889999999999998,"5min":8.3279999999999994,"15min":14.427},"last":0.35899999999999999},{"interface":"front","average":{"1min":1.893,"5min":1.3979999999999999,"15min":1.1299999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.432},"max":{"1min":11.243,"5min":11.243,"15min":11.243},"last":0.39300000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.4139999999999999,"15min":1.357},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40300000000000002},"max":{"1min":4.2779999999999996,"5min":11.718,"15min":14.609},"last":0.51300000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.278,"15min":1.224},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":4.3449999999999998,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":0.46000000000000002}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6749999999999998,"5min":1.7050000000000001,"15min":1.3859999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.34799999999999998},"max":{"1min":12.93,"5min":19.535,"15min":19.535},"last":0.54500000000000004},{"interface":"front","average":{"1min":1.657,"5min":1.589,"15min":1.518},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.2830000000000004,"5min":13.406000000000001,"15min":14.528},"last":0.66300000000000003}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.6320000000000001,"5min":2.0880000000000001,"15min":1.8069999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":29.986999999999998,"5min":29.986999999999998,"15min":29.986999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":2.9140000000000001,"5min":1.9530000000000001,"15min":1.619},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":21.199999999999999,"5min":21.199999999999999,"15min":21.199999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.1960000000000002,"5min":1.8200000000000001,"15min":1.6220000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":12.227,"5min":19.681999999999999,"15min":19.681999999999999},"last":1.01},{"interface":"front","average":{"1min":1.752,"5min":1.7410000000000001,"15min":1.698},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":4.0519999999999996,"5min":10.108000000000001,"15min":14.702999999999999},"last":1.1160000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.726,"15min":1.5629999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.5839999999999996,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":0.94599999999999995},{"interface":"front","average":{"1min":2.0640000000000001,"5min":1.8720000000000001,"15min":1.631},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":12.176,"5min":19.783000000000001,"15min":19.783000000000001},"last":0.75700000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":5.6159999999999997,"5min":2.5179999999999998,"15min":1.907},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":57.006999999999998,"5min":57.006999999999998,"15min":57.006999999999998},"last":0.89600000000000002},{"interface":"front","average":{"1min":3.8799999999999999,"5min":2.5950000000000002,"15min":1.9690000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":21.795999999999999,"5min":21.795999999999999,"15min":21.795999999999999},"last":0.72599999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54220,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317620,"statfs":{"total":96636764160,"available":96581242880,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.27,"15min":1.0209999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42799999999999999},"max":{"1min":13.749000000000001,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.751},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.157,"15min":1.0449999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":18.658000000000001,"5min":18.658000000000001,"15min":18.658000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.331,"15min":1.0329999999999999},"min":{"1min":0.48599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.3029999999999999,"5min":21.126000000000001,"15min":21.126000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.585,"15min":1.3109999999999999},"min":{"1min":0.52700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":9.7279999999999998,"5min":11.5,"15min":11.5},"last":1.0389999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7350000000000001,"5min":1.4279999999999999,"15min":1.855},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":13.593,"5min":13.593,"15min":22.106999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.2250000000000001,"15min":1.26},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.9610000000000003,"5min":7.9610000000000003,"15min":15.510999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.708,"15min":1.726},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72499999999999998},"max":{"1min":3.27,"5min":14.827,"15min":14.827},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.6839999999999999,"15min":1.8109999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3809999999999998,"5min":14.851000000000001,"15min":14.851000000000001},"last":1.4299999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":1.643,"15min":1.7829999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.0700000000000003,"5min":11.552,"15min":13.228999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":3.1400000000000001,"5min":1.929,"15min":1.8089999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.81399999999999995},"max":{"1min":18.452000000000002,"5min":18.452000000000002,"15min":20.823},"last":0.83699999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.0910000000000002,"5min":1.9890000000000001,"15min":2.0139999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":34.158000000000001,"5min":34.158000000000001,"15min":34.158000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.6200000000000001,"15min":1.534},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.52,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.1950000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6859999999999999,"5min":1.865,"15min":1.6639999999999999},"min":{"1min":0.84599999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":17.928999999999998,"5min":17.928999999999998,"15min":17.928999999999998},"last":1.109},{"interface":"front","average":{"1min":1.673,"5min":1.9470000000000001,"15min":1.6839999999999999},"min":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.9139999999999997,"5min":24.289999999999999,"15min":24.289999999999999},"last":1.7150000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542250,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54796,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317044,"statfs":{"total":96636764160,"available":96580653056,"internally_reserved":0,"allocated":1441792,"data_stored":947133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.0649999999999999,"15min":0.94199999999999995},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":6.1890000000000001,"5min":6.1890000000000001,"15min":6.1890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.806,"5min":1.413,"15min":1.1899999999999999},"min":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":11.470000000000001,"5min":11.470000000000001,"15min":11.470000000000001},"last":1.3959999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.333,"15min":1.117},"min":{"1min":0.45500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":11.59,"5min":11.59,"15min":11.59},"last":1.323},{"interface":"front","average":{"1min":2.7040000000000002,"5min":1.377,"15min":1.0309999999999999},"min":{"1min":0.41399999999999998,"5min":0.38200000000000001,"15min":0.35899999999999999},"max":{"1min":14.512,"5min":14.512,"15min":14.512},"last":1.2110000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5920000000000001,"15min":1.3720000000000001},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":11.544,"5min":11.544,"15min":11.544},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.446,"15min":1.194},"min":{"1min":0.41999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.1539999999999999,"5min":14.74,"15min":14.74},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.452,"5min":2.0379999999999998,"15min":1.532},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":32.115000000000002,"5min":32.115000000000002,"15min":32.115000000000002},"last":1.536},{"interface":"front","average":{"1min":2.2280000000000002,"5min":1.903,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":11.992000000000001,"5min":15.975,"15min":15.975},"last":1.4570000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.4870000000000001,"5min":2.1560000000000001,"15min":1.629},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":32.174999999999997,"5min":32.174999999999997,"15min":32.174999999999997},"last":1.369},{"interface":"front","average":{"1min":2.871,"5min":1.915,"15min":1.623},"min":{"1min":0.875,"5min":0.875,"15min":0.754},"max":{"1min":17.225000000000001,"5min":17.225000000000001,"15min":17.225000000000001},"last":1.4830000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.097,"5min":1.746,"15min":1.587},"min":{"1min":0.99199999999999999,"5min":0.96999999999999997,"15min":0.78000000000000003},"max":{"1min":6.1189999999999998,"5min":6.1189999999999998,"15min":6.1189999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":4.3369999999999997,"5min":2.1120000000000001,"15min":1.669},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":32.216999999999999,"5min":32.216999999999999,"15min":32.216999999999999},"last":1.609}]},{"osd":7,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5350000000000001,"5min":2.5710000000000002,"15min":1.9550000000000001},"min":{"1min":1.036,"5min":0.98899999999999999,"15min":0.81299999999999994},"max":{"1min":34.131,"5min":34.131,"15min":34.131},"last":1.508},{"interface":"front","average":{"1min":4.6580000000000004,"5min":2.3799999999999999,"15min":1.8320000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.70999999999999996},"max":{"1min":21.884,"5min":21.884,"15min":21.884},"last":1.266}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:03:31.834 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:03:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.091+0000 7fb436a00640 1 Processor -- start 2024-01-15T18:03:32.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.091+0000 7fb436a00640 1 -- start start 2024-01-15T18:03:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.093+0000 7fb436a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4300b3ae0 0x7fb43015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.093+0000 7fb436a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb430070c40 con 0x7fb4300cf290 2024-01-15T18:03:32.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.093+0000 7fb436a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43006c470 con 0x7fb4300ae5a0 2024-01-15T18:03:32.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.093+0000 7fb436a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb430071450 con 0x7fb4300b3ae0 2024-01-15T18:03:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.093+0000 7fb42f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb4300b3ae0 0x7fb43015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.093+0000 7fb434a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb4300cf290 0x7fb4300b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43370/0 (socket says 172.21.15.26:43370) 2024-01-15T18:03:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.093+0000 7fb434a00640 1 -- 172.21.15.26:0/2244340205 learned_addr learned my addr 172.21.15.26:0/2244340205 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.094+0000 7fb42f400640 1 -- 172.21.15.26:0/2244340205 >> v1:172.21.15.124:6789/0 conn(0x7fb4300ae5a0 legacy=0x7fb43006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.094+0000 7fb42ea00640 1 -- 172.21.15.26:0/2244340205 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741568818 0 0) 0x7fb430070c40 con 0x7fb4300cf290 2024-01-15T18:03:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.094+0000 7fb42f400640 1 -- 172.21.15.26:0/2244340205 >> v1:172.21.15.26:6789/0 conn(0x7fb4300cf290 legacy=0x7fb4300b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.094+0000 7fb42f400640 1 -- 172.21.15.26:0/2244340205 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43006c750 con 0x7fb4300b3ae0 2024-01-15T18:03:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.095+0000 7fb42f400640 1 --2- 172.21.15.26:0/2244340205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300b3ae0 0x7fb43015e430 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb4180097a0 tx=0x7fb41802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb981d0110eacde server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.095+0000 7fb42ea00640 1 -- 172.21.15.26:0/2244340205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41802f810 con 0x7fb4300b3ae0 2024-01-15T18:03:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.095+0000 7fb42ea00640 1 -- 172.21.15.26:0/2244340205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41802f970 con 0x7fb4300b3ae0 2024-01-15T18:03:32.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.096+0000 7fb42ea00640 1 -- 172.21.15.26:0/2244340205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41802fd00 con 0x7fb4300b3ae0 2024-01-15T18:03:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.096+0000 7fb436a00640 1 -- 172.21.15.26:0/2244340205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300b3ae0 msgr2=0x7fb43015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.096+0000 7fb436a00640 1 --2- 172.21.15.26:0/2244340205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300b3ae0 0x7fb43015e430 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb4180097a0 tx=0x7fb41802b9f0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.096+0000 7fb436a00640 1 -- 172.21.15.26:0/2244340205 shutdown_connections 2024-01-15T18:03:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.096+0000 7fb436a00640 1 --2- 172.21.15.26:0/2244340205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300b3ae0 0x7fb43015e430 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.096+0000 7fb436a00640 1 -- 172.21.15.26:0/2244340205 >> 172.21.15.26:0/2244340205 conn(0x7fb43006a800 msgr2=0x7fb43006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.096+0000 7fb436a00640 1 -- 172.21.15.26:0/2244340205 shutdown_connections 2024-01-15T18:03:32.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.096+0000 7fb436a00640 1 -- 172.21.15.26:0/2244340205 wait complete. 2024-01-15T18:03:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.098+0000 7fb436a00640 1 Processor -- start 2024-01-15T18:03:32.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.098+0000 7fb436a00640 1 -- start start 2024-01-15T18:03:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.099+0000 7fb436a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300ae5a0 0x7fb4300ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.099+0000 7fb436a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4300b3ae0 0x7fb430155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.099+0000 7fb436a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4300cf290 0x7fb4301f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.099+0000 7fb436a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb430156560 con 0x7fb4300ae5a0 2024-01-15T18:03:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.099+0000 7fb436a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300ce660 con 0x7fb4300cf290 2024-01-15T18:03:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.099+0000 7fb436a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300ccb50 con 0x7fb4300b3ae0 2024-01-15T18:03:32.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300ae5a0 0x7fb4300ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb42fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300ae5a0 0x7fb4300ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50528/0 (socket says 172.21.15.26:50528) 2024-01-15T18:03:32.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb42fe00640 1 -- 172.21.15.26:0/2487298672 learned_addr learned my addr 172.21.15.26:0/2487298672 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb42f400640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4300b3ae0 0x7fb430155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb42fe00640 1 -- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4300b3ae0 msgr2=0x7fb430155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb434a00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4300cf290 0x7fb4301f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb42fe00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4300b3ae0 0x7fb430155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb42fe00640 1 -- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4300cf290 msgr2=0x7fb4301f64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb42fe00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4300cf290 0x7fb4301f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.100+0000 7fb42fe00640 1 -- 172.21.15.26:0/2487298672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb418009400 con 0x7fb4300ae5a0 2024-01-15T18:03:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.101+0000 7fb42fe00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300ae5a0 0x7fb4300ce120 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb42000c450 tx=0x7fb42000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:32.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.101+0000 7fb427e00640 1 -- 172.21.15.26:0/2487298672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4200052d0 con 0x7fb4300ae5a0 2024-01-15T18:03:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.101+0000 7fb427e00640 1 -- 172.21.15.26:0/2487298672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb420005430 con 0x7fb4300ae5a0 2024-01-15T18:03:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.101+0000 7fb427e00640 1 -- 172.21.15.26:0/2487298672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420012870 con 0x7fb4300ae5a0 2024-01-15T18:03:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.101+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb43015c4b0 con 0x7fb4300ae5a0 2024-01-15T18:03:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.101+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb43015ca70 con 0x7fb4300ae5a0 2024-01-15T18:03:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.102+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4300675d0 con 0x7fb4300ae5a0 2024-01-15T18:03:32.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.103+0000 7fb427e00640 1 -- 172.21.15.26:0/2487298672 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fb4200129d0 con 0x7fb4300ae5a0 2024-01-15T18:03:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.104+0000 7fb427e00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40806bbc0 0x7fb40806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.104+0000 7fb427e00640 1 -- 172.21.15.26:0/2487298672 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb420087f90 con 0x7fb4300ae5a0 2024-01-15T18:03:32.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.105+0000 7fb42f400640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40806bbc0 0x7fb40806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.106+0000 7fb42f400640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40806bbc0 0x7fb40806e080 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb418036000 tx=0x7fb418008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:32.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.109+0000 7fb427e00640 1 -- 172.21.15.26:0/2487298672 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4200548b0 con 0x7fb4300ae5a0 2024-01-15T18:03:32.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.306+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb430070550 con 0x7fb40806bbc0 2024-01-15T18:03:32.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.331+0000 7fb427e00640 1 -- 172.21.15.26:0/2487298672 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283196 (secure 0 0 0) 0x7fb41802fb60 con 0x7fb40806bbc0 2024-01-15T18:03:32.341 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:32.343 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:03:32.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.338+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40806bbc0 msgr2=0x7fb40806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40806bbc0 0x7fb40806e080 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb418036000 tx=0x7fb418008030 comp rx=0 tx=0).stop 2024-01-15T18:03:32.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300ae5a0 msgr2=0x7fb4300ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300ae5a0 0x7fb4300ce120 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb42000c450 tx=0x7fb42000c910 comp rx=0 tx=0).stop 2024-01-15T18:03:32.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 shutdown_connections 2024-01-15T18:03:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4300b3ae0 0x7fb430155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40806bbc0 0x7fb40806e080 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4300cf290 0x7fb4301f64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 --2- 172.21.15.26:0/2487298672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb4300ae5a0 0x7fb4300ce120 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 >> 172.21.15.26:0/2487298672 conn(0x7fb43006a800 msgr2=0x7fb4300b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 shutdown_connections 2024-01-15T18:03:32.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.339+0000 7fb436a00640 1 -- 172.21.15.26:0/2487298672 wait complete. 2024-01-15T18:03:32.364 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-01-15T18:03:32.104844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":168,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409992,"kb_used_data":7820,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754564728,"statfs":{"total":773094113280,"available":772674281472,"internally_reserved":0,"allocated":8007680,"data_stored":4049655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001932"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281242+0000","last_change":"2024-01-15T18:03:26.281242+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281242+0000","last_undegraded":"2024-01-15T18:03:26.281242+0000","last_fullsized":"2024-01-15T18:03:26.281242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342379+0000","last_change":"2024-01-15T18:03:27.341616+0000","last_active":"2024-01-15T18:03:27.342379+0000","last_peered":"2024-01-15T18:03:27.342379+0000","last_clean":"2024-01-15T18:03:27.342379+0000","last_became_active":"2024-01-15T18:03:27.341248+0000","last_became_peered":"2024-01-15T18:03:27.341248+0000","last_unstale":"2024-01-15T18:03:27.342379+0000","last_undegraded":"2024-01-15T18:03:27.342379+0000","last_fullsized":"2024-01-15T18:03:27.342379+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:51.975291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283811+0000","last_change":"2024-01-15T18:03:26.283811+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283811+0000","last_undegraded":"2024-01-15T18:03:26.283811+0000","last_fullsized":"2024-01-15T18:03:26.283811+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.288618+0000","last_change":"2024-01-15T18:03:26.288618+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.288618+0000","last_undegraded":"2024-01-15T18:03:26.288618+0000","last_fullsized":"2024-01-15T18:03:26.288618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302106+0000","last_change":"2024-01-15T18:03:27.319087+0000","last_active":"2024-01-15T18:03:28.302106+0000","last_peered":"2024-01-15T18:03:28.302106+0000","last_clean":"2024-01-15T18:03:28.302106+0000","last_became_active":"2024-01-15T18:03:27.318702+0000","last_became_peered":"2024-01-15T18:03:27.318702+0000","last_unstale":"2024-01-15T18:03:28.302106+0000","last_undegraded":"2024-01-15T18:03:28.302106+0000","last_fullsized":"2024-01-15T18:03:28.302106+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:13:57.340165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.321499+0000","last_change":"2024-01-15T18:03:27.319124+0000","last_active":"2024-01-15T18:03:27.321499+0000","last_peered":"2024-01-15T18:03:27.321499+0000","last_clean":"2024-01-15T18:03:27.321499+0000","last_became_active":"2024-01-15T18:03:27.318855+0000","last_became_peered":"2024-01-15T18:03:27.318855+0000","last_unstale":"2024-01-15T18:03:27.321499+0000","last_undegraded":"2024-01-15T18:03:27.321499+0000","last_fullsized":"2024-01-15T18:03:27.321499+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:40.231033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284505+0000","last_change":"2024-01-15T18:03:26.284505+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284505+0000","last_undegraded":"2024-01-15T18:03:26.284505+0000","last_fullsized":"2024-01-15T18:03:26.284505+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.321938+0000","last_change":"2024-01-15T18:03:27.316226+0000","last_active":"2024-01-15T18:03:27.321938+0000","last_peered":"2024-01-15T18:03:27.321938+0000","last_clean":"2024-01-15T18:03:27.321938+0000","last_became_active":"2024-01-15T18:03:27.315889+0000","last_became_peered":"2024-01-15T18:03:27.315889+0000","last_unstale":"2024-01-15T18:03:27.321938+0000","last_undegraded":"2024-01-15T18:03:27.321938+0000","last_fullsized":"2024-01-15T18:03:27.321938+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:56.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302728+0000","last_change":"2024-01-15T18:03:27.331363+0000","last_active":"2024-01-15T18:03:28.302728+0000","last_peered":"2024-01-15T18:03:28.302728+0000","last_clean":"2024-01-15T18:03:28.302728+0000","last_became_active":"2024-01-15T18:03:27.330530+0000","last_became_peered":"2024-01-15T18:03:27.330530+0000","last_unstale":"2024-01-15T18:03:28.302728+0000","last_undegraded":"2024-01-15T18:03:28.302728+0000","last_fullsized":"2024-01-15T18:03:28.302728+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:43:50.724930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.275264+0000","last_change":"2024-01-15T18:03:26.275264+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.275264+0000","last_undegraded":"2024-01-15T18:03:26.275264+0000","last_fullsized":"2024-01-15T18:03:26.275264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282234+0000","last_change":"2024-01-15T18:03:26.282234+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282234+0000","last_undegraded":"2024-01-15T18:03:26.282234+0000","last_fullsized":"2024-01-15T18:03:26.282234+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.288002+0000","last_change":"2024-01-15T18:03:26.288002+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.288002+0000","last_undegraded":"2024-01-15T18:03:26.288002+0000","last_fullsized":"2024-01-15T18:03:26.288002+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282326+0000","last_change":"2024-01-15T18:03:26.282326+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282326+0000","last_undegraded":"2024-01-15T18:03:26.282326+0000","last_fullsized":"2024-01-15T18:03:26.282326+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302207+0000","last_change":"2024-01-15T18:03:27.329433+0000","last_active":"2024-01-15T18:03:28.302207+0000","last_peered":"2024-01-15T18:03:28.302207+0000","last_clean":"2024-01-15T18:03:28.302207+0000","last_became_active":"2024-01-15T18:03:27.329186+0000","last_became_peered":"2024-01-15T18:03:27.329186+0000","last_unstale":"2024-01-15T18:03:28.302207+0000","last_undegraded":"2024-01-15T18:03:28.302207+0000","last_fullsized":"2024-01-15T18:03:28.302207+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:34.881592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284462+0000","last_change":"2024-01-15T18:03:26.284462+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284462+0000","last_undegraded":"2024-01-15T18:03:26.284462+0000","last_fullsized":"2024-01-15T18:03:26.284462+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.275307+0000","last_change":"2024-01-15T18:03:26.275307+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.275307+0000","last_undegraded":"2024-01-15T18:03:26.275307+0000","last_fullsized":"2024-01-15T18:03:26.275307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302977+0000","last_change":"2024-01-15T18:03:27.330530+0000","last_active":"2024-01-15T18:03:28.302977+0000","last_peered":"2024-01-15T18:03:28.302977+0000","last_clean":"2024-01-15T18:03:28.302977+0000","last_became_active":"2024-01-15T18:03:27.330149+0000","last_became_peered":"2024-01-15T18:03:27.330149+0000","last_unstale":"2024-01-15T18:03:28.302977+0000","last_undegraded":"2024-01-15T18:03:28.302977+0000","last_fullsized":"2024-01-15T18:03:28.302977+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:12.045333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.286446+0000","last_change":"2024-01-15T18:03:26.286446+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.286446+0000","last_undegraded":"2024-01-15T18:03:26.286446+0000","last_fullsized":"2024-01-15T18:03:26.286446+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341755+0000","last_change":"2024-01-15T18:03:27.338146+0000","last_active":"2024-01-15T18:03:27.341755+0000","last_peered":"2024-01-15T18:03:27.341755+0000","last_clean":"2024-01-15T18:03:27.341755+0000","last_became_active":"2024-01-15T18:03:27.337850+0000","last_became_peered":"2024-01-15T18:03:27.337850+0000","last_unstale":"2024-01-15T18:03:27.341755+0000","last_undegraded":"2024-01-15T18:03:27.341755+0000","last_fullsized":"2024-01-15T18:03:27.341755+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:06:27.744895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282114+0000","last_change":"2024-01-15T18:03:26.282114+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282114+0000","last_undegraded":"2024-01-15T18:03:26.282114+0000","last_fullsized":"2024-01-15T18:03:26.282114+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.276325+0000","last_change":"2024-01-15T18:03:26.276325+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.276325+0000","last_undegraded":"2024-01-15T18:03:26.276325+0000","last_fullsized":"2024-01-15T18:03:26.276325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274415+0000","last_change":"2024-01-15T18:03:26.274415+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274415+0000","last_undegraded":"2024-01-15T18:03:26.274415+0000","last_fullsized":"2024-01-15T18:03:26.274415+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279090+0000","last_change":"2024-01-15T18:03:26.279090+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279090+0000","last_undegraded":"2024-01-15T18:03:26.279090+0000","last_fullsized":"2024-01-15T18:03:26.279090+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279397+0000","last_change":"2024-01-15T18:03:26.279397+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279397+0000","last_undegraded":"2024-01-15T18:03:26.279397+0000","last_fullsized":"2024-01-15T18:03:26.279397+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.276036+0000","last_change":"2024-01-15T18:03:26.276036+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.276036+0000","last_undegraded":"2024-01-15T18:03:26.276036+0000","last_fullsized":"2024-01-15T18:03:26.276036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284633+0000","last_change":"2024-01-15T18:03:26.284633+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284633+0000","last_undegraded":"2024-01-15T18:03:26.284633+0000","last_fullsized":"2024-01-15T18:03:26.284633+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283351+0000","last_change":"2024-01-15T18:03:26.283351+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283351+0000","last_undegraded":"2024-01-15T18:03:26.283351+0000","last_fullsized":"2024-01-15T18:03:26.283351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274031+0000","last_change":"2024-01-15T18:03:26.274031+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274031+0000","last_undegraded":"2024-01-15T18:03:26.274031+0000","last_fullsized":"2024-01-15T18:03:26.274031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282955+0000","last_change":"2024-01-15T18:03:26.282955+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282955+0000","last_undegraded":"2024-01-15T18:03:26.282955+0000","last_fullsized":"2024-01-15T18:03:26.282955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283969+0000","last_change":"2024-01-15T18:03:26.283969+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283969+0000","last_undegraded":"2024-01-15T18:03:26.283969+0000","last_fullsized":"2024-01-15T18:03:26.283969+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283167+0000","last_change":"2024-01-15T18:03:26.283167+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283167+0000","last_undegraded":"2024-01-15T18:03:26.283167+0000","last_fullsized":"2024-01-15T18:03:26.283167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.324636+0000","last_change":"2024-01-15T18:03:27.321263+0000","last_active":"2024-01-15T18:03:27.324636+0000","last_peered":"2024-01-15T18:03:27.324636+0000","last_clean":"2024-01-15T18:03:27.324636+0000","last_became_active":"2024-01-15T18:03:27.320958+0000","last_became_peered":"2024-01-15T18:03:27.320958+0000","last_unstale":"2024-01-15T18:03:27.324636+0000","last_undegraded":"2024-01-15T18:03:27.324636+0000","last_fullsized":"2024-01-15T18:03:27.324636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:43:50.724930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279714+0000","last_change":"2024-01-15T18:03:26.279714+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279714+0000","last_undegraded":"2024-01-15T18:03:26.279714+0000","last_fullsized":"2024-01-15T18:03:26.279714+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299516+0000","last_change":"2024-01-15T18:03:26.298227+0000","last_active":"2024-01-15T18:03:26.299516+0000","last_peered":"2024-01-15T18:03:26.299516+0000","last_clean":"2024-01-15T18:03:26.299516+0000","last_became_active":"2024-01-15T18:03:26.297015+0000","last_became_peered":"2024-01-15T18:03:26.297015+0000","last_unstale":"2024-01-15T18:03:26.299516+0000","last_undegraded":"2024-01-15T18:03:26.299516+0000","last_fullsized":"2024-01-15T18:03:26.299516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:43.508315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302905+0000","last_change":"2024-01-15T18:03:27.330723+0000","last_active":"2024-01-15T18:03:28.302905+0000","last_peered":"2024-01-15T18:03:28.302905+0000","last_clean":"2024-01-15T18:03:28.302905+0000","last_became_active":"2024-01-15T18:03:27.330236+0000","last_became_peered":"2024-01-15T18:03:27.330236+0000","last_unstale":"2024-01-15T18:03:28.302905+0000","last_undegraded":"2024-01-15T18:03:28.302905+0000","last_fullsized":"2024-01-15T18:03:28.302905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:56.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295089+0000","last_change":"2024-01-15T18:03:26.292579+0000","last_active":"2024-01-15T18:03:26.295089+0000","last_peered":"2024-01-15T18:03:26.295089+0000","last_clean":"2024-01-15T18:03:26.295089+0000","last_became_active":"2024-01-15T18:03:26.291631+0000","last_became_peered":"2024-01-15T18:03:26.291631+0000","last_unstale":"2024-01-15T18:03:26.295089+0000","last_undegraded":"2024-01-15T18:03:26.295089+0000","last_fullsized":"2024-01-15T18:03:26.295089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:32.983343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341535+0000","last_change":"2024-01-15T18:03:27.336001+0000","last_active":"2024-01-15T18:03:27.341535+0000","last_peered":"2024-01-15T18:03:27.341535+0000","last_clean":"2024-01-15T18:03:27.341535+0000","last_became_active":"2024-01-15T18:03:27.335265+0000","last_became_peered":"2024-01-15T18:03:27.335265+0000","last_unstale":"2024-01-15T18:03:27.341535+0000","last_undegraded":"2024-01-15T18:03:27.341535+0000","last_fullsized":"2024-01-15T18:03:27.341535+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:52.859519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.304785+0000","last_change":"2024-01-15T18:03:26.303968+0000","last_active":"2024-01-15T18:03:26.304785+0000","last_peered":"2024-01-15T18:03:26.304785+0000","last_clean":"2024-01-15T18:03:26.304785+0000","last_became_active":"2024-01-15T18:03:26.303669+0000","last_became_peered":"2024-01-15T18:03:26.303669+0000","last_unstale":"2024-01-15T18:03:26.304785+0000","last_undegraded":"2024-01-15T18:03:26.304785+0000","last_fullsized":"2024-01-15T18:03:26.304785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:43.710985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281710+0000","last_change":"2024-01-15T18:03:26.281710+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281710+0000","last_undegraded":"2024-01-15T18:03:26.281710+0000","last_fullsized":"2024-01-15T18:03:26.281710+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342311+0000","last_change":"2024-01-15T18:03:26.300990+0000","last_active":"2024-01-15T18:03:27.342311+0000","last_peered":"2024-01-15T18:03:27.342311+0000","last_clean":"2024-01-15T18:03:27.342311+0000","last_became_active":"2024-01-15T18:03:26.300735+0000","last_became_peered":"2024-01-15T18:03:26.300735+0000","last_unstale":"2024-01-15T18:03:27.342311+0000","last_undegraded":"2024-01-15T18:03:27.342311+0000","last_fullsized":"2024-01-15T18:03:27.342311+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:53:06.533003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283325+0000","last_change":"2024-01-15T18:03:26.283325+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283325+0000","last_undegraded":"2024-01-15T18:03:26.283325+0000","last_fullsized":"2024-01-15T18:03:26.283325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297810+0000","last_change":"2024-01-15T18:03:26.296481+0000","last_active":"2024-01-15T18:03:26.297810+0000","last_peered":"2024-01-15T18:03:26.297810+0000","last_clean":"2024-01-15T18:03:26.297810+0000","last_became_active":"2024-01-15T18:03:26.293509+0000","last_became_peered":"2024-01-15T18:03:26.293509+0000","last_unstale":"2024-01-15T18:03:26.297810+0000","last_undegraded":"2024-01-15T18:03:26.297810+0000","last_fullsized":"2024-01-15T18:03:26.297810+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:48.220096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274045+0000","last_change":"2024-01-15T18:03:26.274045+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274045+0000","last_undegraded":"2024-01-15T18:03:26.274045+0000","last_fullsized":"2024-01-15T18:03:26.274045+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.340684+0000","last_change":"2024-01-15T18:03:26.302584+0000","last_active":"2024-01-15T18:03:27.340684+0000","last_peered":"2024-01-15T18:03:27.340684+0000","last_clean":"2024-01-15T18:03:27.340684+0000","last_became_active":"2024-01-15T18:03:26.301356+0000","last_became_peered":"2024-01-15T18:03:26.301356+0000","last_unstale":"2024-01-15T18:03:27.340684+0000","last_undegraded":"2024-01-15T18:03:27.340684+0000","last_fullsized":"2024-01-15T18:03:27.340684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:22:00.115460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.301672+0000","last_change":"2024-01-15T18:03:27.326042+0000","last_active":"2024-01-15T18:03:28.301672+0000","last_peered":"2024-01-15T18:03:28.301672+0000","last_clean":"2024-01-15T18:03:28.301672+0000","last_became_active":"2024-01-15T18:03:27.325476+0000","last_became_peered":"2024-01-15T18:03:27.325476+0000","last_unstale":"2024-01-15T18:03:28.301672+0000","last_undegraded":"2024-01-15T18:03:28.301672+0000","last_fullsized":"2024-01-15T18:03:28.301672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:37:22.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.293022+0000","last_change":"2024-01-15T18:03:26.282542+0000","last_active":"2024-01-15T18:03:26.293022+0000","last_peered":"2024-01-15T18:03:26.293022+0000","last_clean":"2024-01-15T18:03:26.293022+0000","last_became_active":"2024-01-15T18:03:26.282226+0000","last_became_peered":"2024-01-15T18:03:26.282226+0000","last_unstale":"2024-01-15T18:03:26.293022+0000","last_undegraded":"2024-01-15T18:03:26.293022+0000","last_fullsized":"2024-01-15T18:03:26.293022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:57.562173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281723+0000","last_change":"2024-01-15T18:03:26.281723+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281723+0000","last_undegraded":"2024-01-15T18:03:26.281723+0000","last_fullsized":"2024-01-15T18:03:26.281723+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.312237+0000","last_change":"2024-01-15T18:03:26.285864+0000","last_active":"2024-01-15T18:03:27.312237+0000","last_peered":"2024-01-15T18:03:27.312237+0000","last_clean":"2024-01-15T18:03:27.312237+0000","last_became_active":"2024-01-15T18:03:26.285507+0000","last_became_peered":"2024-01-15T18:03:26.285507+0000","last_unstale":"2024-01-15T18:03:27.312237+0000","last_undegraded":"2024-01-15T18:03:27.312237+0000","last_fullsized":"2024-01-15T18:03:27.312237+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:47.778037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283831+0000","last_change":"2024-01-15T18:03:26.283831+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283831+0000","last_undegraded":"2024-01-15T18:03:26.283831+0000","last_fullsized":"2024-01-15T18:03:26.283831+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295148+0000","last_change":"2024-01-15T18:03:26.294412+0000","last_active":"2024-01-15T18:03:26.295148+0000","last_peered":"2024-01-15T18:03:26.295148+0000","last_clean":"2024-01-15T18:03:26.295148+0000","last_became_active":"2024-01-15T18:03:26.294070+0000","last_became_peered":"2024-01-15T18:03:26.294070+0000","last_unstale":"2024-01-15T18:03:26.295148+0000","last_undegraded":"2024-01-15T18:03:26.295148+0000","last_fullsized":"2024-01-15T18:03:26.295148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:06:26.733111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.280249+0000","last_change":"2024-01-15T18:03:26.280249+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.280249+0000","last_undegraded":"2024-01-15T18:03:26.280249+0000","last_fullsized":"2024-01-15T18:03:26.280249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.312561+0000","last_change":"2024-01-15T18:03:26.302162+0000","last_active":"2024-01-15T18:03:27.312561+0000","last_peered":"2024-01-15T18:03:27.312561+0000","last_clean":"2024-01-15T18:03:27.312561+0000","last_became_active":"2024-01-15T18:03:26.301821+0000","last_became_peered":"2024-01-15T18:03:26.301821+0000","last_unstale":"2024-01-15T18:03:27.312561+0000","last_undegraded":"2024-01-15T18:03:27.312561+0000","last_fullsized":"2024-01-15T18:03:27.312561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:24:07.021223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.340063+0000","last_change":"2024-01-15T18:03:27.329589+0000","last_active":"2024-01-15T18:03:27.340063+0000","last_peered":"2024-01-15T18:03:27.340063+0000","last_clean":"2024-01-15T18:03:27.340063+0000","last_became_active":"2024-01-15T18:03:27.328118+0000","last_became_peered":"2024-01-15T18:03:27.328118+0000","last_unstale":"2024-01-15T18:03:27.340063+0000","last_undegraded":"2024-01-15T18:03:27.340063+0000","last_fullsized":"2024-01-15T18:03:27.340063+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:48.991808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.296636+0000","last_change":"2024-01-15T18:03:26.295197+0000","last_active":"2024-01-15T18:03:26.296636+0000","last_peered":"2024-01-15T18:03:26.296636+0000","last_clean":"2024-01-15T18:03:26.296636+0000","last_became_active":"2024-01-15T18:03:26.294932+0000","last_became_peered":"2024-01-15T18:03:26.294932+0000","last_unstale":"2024-01-15T18:03:26.296636+0000","last_undegraded":"2024-01-15T18:03:26.296636+0000","last_fullsized":"2024-01-15T18:03:26.296636+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:20.869224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302274+0000","last_change":"2024-01-15T18:03:27.315984+0000","last_active":"2024-01-15T18:03:28.302274+0000","last_peered":"2024-01-15T18:03:28.302274+0000","last_clean":"2024-01-15T18:03:28.302274+0000","last_became_active":"2024-01-15T18:03:27.315700+0000","last_became_peered":"2024-01-15T18:03:27.315700+0000","last_unstale":"2024-01-15T18:03:28.302274+0000","last_undegraded":"2024-01-15T18:03:28.302274+0000","last_fullsized":"2024-01-15T18:03:28.302274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:16.050477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294412+0000","last_change":"2024-01-15T18:03:26.283744+0000","last_active":"2024-01-15T18:03:26.294412+0000","last_peered":"2024-01-15T18:03:26.294412+0000","last_clean":"2024-01-15T18:03:26.294412+0000","last_became_active":"2024-01-15T18:03:26.283382+0000","last_became_peered":"2024-01-15T18:03:26.283382+0000","last_unstale":"2024-01-15T18:03:26.294412+0000","last_undegraded":"2024-01-15T18:03:26.294412+0000","last_fullsized":"2024-01-15T18:03:26.294412+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:13:56.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278423+0000","last_change":"2024-01-15T18:03:26.278423+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278423+0000","last_undegraded":"2024-01-15T18:03:26.278423+0000","last_fullsized":"2024-01-15T18:03:26.278423+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295393+0000","last_change":"2024-01-15T18:03:26.290460+0000","last_active":"2024-01-15T18:03:26.295393+0000","last_peered":"2024-01-15T18:03:26.295393+0000","last_clean":"2024-01-15T18:03:26.295393+0000","last_became_active":"2024-01-15T18:03:26.290114+0000","last_became_peered":"2024-01-15T18:03:26.290114+0000","last_unstale":"2024-01-15T18:03:26.295393+0000","last_undegraded":"2024-01-15T18:03:26.295393+0000","last_fullsized":"2024-01-15T18:03:26.295393+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:49.162905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.286454+0000","last_change":"2024-01-15T18:03:26.286454+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.286454+0000","last_undegraded":"2024-01-15T18:03:26.286454+0000","last_fullsized":"2024-01-15T18:03:26.286454+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.306852+0000","last_change":"2024-01-15T18:03:26.302531+0000","last_active":"2024-01-15T18:03:27.306852+0000","last_peered":"2024-01-15T18:03:27.306852+0000","last_clean":"2024-01-15T18:03:27.306852+0000","last_became_active":"2024-01-15T18:03:26.301660+0000","last_became_peered":"2024-01-15T18:03:26.301660+0000","last_unstale":"2024-01-15T18:03:27.306852+0000","last_undegraded":"2024-01-15T18:03:27.306852+0000","last_fullsized":"2024-01-15T18:03:27.306852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:18:48.053993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281591+0000","last_change":"2024-01-15T18:03:26.281591+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281591+0000","last_undegraded":"2024-01-15T18:03:26.281591+0000","last_fullsized":"2024-01-15T18:03:26.281591+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299435+0000","last_change":"2024-01-15T18:03:26.297958+0000","last_active":"2024-01-15T18:03:26.299435+0000","last_peered":"2024-01-15T18:03:26.299435+0000","last_clean":"2024-01-15T18:03:26.299435+0000","last_became_active":"2024-01-15T18:03:26.297417+0000","last_became_peered":"2024-01-15T18:03:26.297417+0000","last_unstale":"2024-01-15T18:03:26.299435+0000","last_undegraded":"2024-01-15T18:03:26.299435+0000","last_fullsized":"2024-01-15T18:03:26.299435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:20:43.839295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.301949+0000","last_change":"2024-01-15T18:03:27.330251+0000","last_active":"2024-01-15T18:03:28.301949+0000","last_peered":"2024-01-15T18:03:28.301949+0000","last_clean":"2024-01-15T18:03:28.301949+0000","last_became_active":"2024-01-15T18:03:27.329948+0000","last_became_peered":"2024-01-15T18:03:27.329948+0000","last_unstale":"2024-01-15T18:03:28.301949+0000","last_undegraded":"2024-01-15T18:03:28.301949+0000","last_fullsized":"2024-01-15T18:03:28.301949+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:58.573957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.293724+0000","last_change":"2024-01-15T18:03:26.287860+0000","last_active":"2024-01-15T18:03:26.293724+0000","last_peered":"2024-01-15T18:03:26.293724+0000","last_clean":"2024-01-15T18:03:26.293724+0000","last_became_active":"2024-01-15T18:03:26.287555+0000","last_became_peered":"2024-01-15T18:03:26.287555+0000","last_unstale":"2024-01-15T18:03:26.293724+0000","last_undegraded":"2024-01-15T18:03:26.293724+0000","last_fullsized":"2024-01-15T18:03:26.293724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:50.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274473+0000","last_change":"2024-01-15T18:03:26.274473+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274473+0000","last_undegraded":"2024-01-15T18:03:26.274473+0000","last_fullsized":"2024-01-15T18:03:26.274473+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341894+0000","last_change":"2024-01-15T18:03:26.300342+0000","last_active":"2024-01-15T18:03:27.341894+0000","last_peered":"2024-01-15T18:03:27.341894+0000","last_clean":"2024-01-15T18:03:27.341894+0000","last_became_active":"2024-01-15T18:03:26.300100+0000","last_became_peered":"2024-01-15T18:03:26.300100+0000","last_unstale":"2024-01-15T18:03:27.341894+0000","last_undegraded":"2024-01-15T18:03:27.341894+0000","last_fullsized":"2024-01-15T18:03:27.341894+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:08.274277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279678+0000","last_change":"2024-01-15T18:03:26.279678+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279678+0000","last_undegraded":"2024-01-15T18:03:26.279678+0000","last_fullsized":"2024-01-15T18:03:26.279678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.303643+0000","last_change":"2024-01-15T18:03:26.298853+0000","last_active":"2024-01-15T18:03:26.303643+0000","last_peered":"2024-01-15T18:03:26.303643+0000","last_clean":"2024-01-15T18:03:26.303643+0000","last_became_active":"2024-01-15T18:03:26.298530+0000","last_became_peered":"2024-01-15T18:03:26.298530+0000","last_unstale":"2024-01-15T18:03:26.303643+0000","last_undegraded":"2024-01-15T18:03:26.303643+0000","last_fullsized":"2024-01-15T18:03:26.303643+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:51.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284222+0000","last_change":"2024-01-15T18:03:26.284222+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284222+0000","last_undegraded":"2024-01-15T18:03:26.284222+0000","last_fullsized":"2024-01-15T18:03:26.284222+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.343468+0000","last_change":"2024-01-15T18:03:26.301874+0000","last_active":"2024-01-15T18:03:27.343468+0000","last_peered":"2024-01-15T18:03:27.343468+0000","last_clean":"2024-01-15T18:03:27.343468+0000","last_became_active":"2024-01-15T18:03:26.301425+0000","last_became_peered":"2024-01-15T18:03:26.301425+0000","last_unstale":"2024-01-15T18:03:27.343468+0000","last_undegraded":"2024-01-15T18:03:27.343468+0000","last_fullsized":"2024-01-15T18:03:27.343468+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:43.710985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302611+0000","last_change":"2024-01-15T18:03:27.325587+0000","last_active":"2024-01-15T18:03:28.302611+0000","last_peered":"2024-01-15T18:03:28.302611+0000","last_clean":"2024-01-15T18:03:28.302611+0000","last_became_active":"2024-01-15T18:03:27.325317+0000","last_became_peered":"2024-01-15T18:03:27.325317+0000","last_unstale":"2024-01-15T18:03:28.302611+0000","last_undegraded":"2024-01-15T18:03:28.302611+0000","last_fullsized":"2024-01-15T18:03:28.302611+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:18:27.777253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299247+0000","last_change":"2024-01-15T18:03:26.295610+0000","last_active":"2024-01-15T18:03:26.299247+0000","last_peered":"2024-01-15T18:03:26.299247+0000","last_clean":"2024-01-15T18:03:26.299247+0000","last_became_active":"2024-01-15T18:03:26.295164+0000","last_became_peered":"2024-01-15T18:03:26.295164+0000","last_unstale":"2024-01-15T18:03:26.299247+0000","last_undegraded":"2024-01-15T18:03:26.299247+0000","last_fullsized":"2024-01-15T18:03:26.299247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:42.703538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279636+0000","last_change":"2024-01-15T18:03:26.279636+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279636+0000","last_undegraded":"2024-01-15T18:03:26.279636+0000","last_fullsized":"2024-01-15T18:03:26.279636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.316666+0000","last_change":"2024-01-15T18:03:26.301741+0000","last_active":"2024-01-15T18:03:27.316666+0000","last_peered":"2024-01-15T18:03:27.316666+0000","last_clean":"2024-01-15T18:03:27.316666+0000","last_became_active":"2024-01-15T18:03:26.301228+0000","last_became_peered":"2024-01-15T18:03:26.301228+0000","last_unstale":"2024-01-15T18:03:27.316666+0000","last_undegraded":"2024-01-15T18:03:27.316666+0000","last_fullsized":"2024-01-15T18:03:27.316666+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:08.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278650+0000","last_change":"2024-01-15T18:03:26.278650+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278650+0000","last_undegraded":"2024-01-15T18:03:26.278650+0000","last_fullsized":"2024-01-15T18:03:26.278650+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294795+0000","last_change":"2024-01-15T18:03:26.293020+0000","last_active":"2024-01-15T18:03:26.294795+0000","last_peered":"2024-01-15T18:03:26.294795+0000","last_clean":"2024-01-15T18:03:26.294795+0000","last_became_active":"2024-01-15T18:03:26.292663+0000","last_became_peered":"2024-01-15T18:03:26.292663+0000","last_unstale":"2024-01-15T18:03:26.294795+0000","last_undegraded":"2024-01-15T18:03:26.294795+0000","last_fullsized":"2024-01-15T18:03:26.294795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:51.697653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278555+0000","last_change":"2024-01-15T18:03:26.278555+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278555+0000","last_undegraded":"2024-01-15T18:03:26.278555+0000","last_fullsized":"2024-01-15T18:03:26.278555+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295435+0000","last_change":"2024-01-15T18:03:26.292741+0000","last_active":"2024-01-15T18:03:26.295435+0000","last_peered":"2024-01-15T18:03:26.295435+0000","last_clean":"2024-01-15T18:03:26.295435+0000","last_became_active":"2024-01-15T18:03:26.292349+0000","last_became_peered":"2024-01-15T18:03:26.292349+0000","last_unstale":"2024-01-15T18:03:26.295435+0000","last_undegraded":"2024-01-15T18:03:26.295435+0000","last_fullsized":"2024-01-15T18:03:26.295435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:19:42.341080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282316+0000","last_change":"2024-01-15T18:03:26.282316+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282316+0000","last_undegraded":"2024-01-15T18:03:26.282316+0000","last_fullsized":"2024-01-15T18:03:26.282316+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341991+0000","last_change":"2024-01-15T18:03:26.299863+0000","last_active":"2024-01-15T18:03:27.341991+0000","last_peered":"2024-01-15T18:03:27.341991+0000","last_clean":"2024-01-15T18:03:27.341991+0000","last_became_active":"2024-01-15T18:03:26.299525+0000","last_became_peered":"2024-01-15T18:03:26.299525+0000","last_unstale":"2024-01-15T18:03:27.341991+0000","last_undegraded":"2024-01-15T18:03:27.341991+0000","last_fullsized":"2024-01-15T18:03:27.341991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:51.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284466+0000","last_change":"2024-01-15T18:03:26.284466+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284466+0000","last_undegraded":"2024-01-15T18:03:26.284466+0000","last_fullsized":"2024-01-15T18:03:26.284466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299404+0000","last_change":"2024-01-15T18:03:26.297682+0000","last_active":"2024-01-15T18:03:26.299404+0000","last_peered":"2024-01-15T18:03:26.299404+0000","last_clean":"2024-01-15T18:03:26.299404+0000","last_became_active":"2024-01-15T18:03:26.297319+0000","last_became_peered":"2024-01-15T18:03:26.297319+0000","last_unstale":"2024-01-15T18:03:26.299404+0000","last_undegraded":"2024-01-15T18:03:26.299404+0000","last_fullsized":"2024-01-15T18:03:26.299404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:40:28.369164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278628+0000","last_change":"2024-01-15T18:03:26.278628+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278628+0000","last_undegraded":"2024-01-15T18:03:26.278628+0000","last_fullsized":"2024-01-15T18:03:26.278628+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.311963+0000","last_change":"2024-01-15T18:03:26.290405+0000","last_active":"2024-01-15T18:03:27.311963+0000","last_peered":"2024-01-15T18:03:27.311963+0000","last_clean":"2024-01-15T18:03:27.311963+0000","last_became_active":"2024-01-15T18:03:26.288657+0000","last_became_peered":"2024-01-15T18:03:26.288657+0000","last_unstale":"2024-01-15T18:03:27.311963+0000","last_undegraded":"2024-01-15T18:03:27.311963+0000","last_fullsized":"2024-01-15T18:03:27.311963+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:56.868408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281044+0000","last_change":"2024-01-15T18:03:26.281044+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281044+0000","last_undegraded":"2024-01-15T18:03:26.281044+0000","last_fullsized":"2024-01-15T18:03:26.281044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297486+0000","last_change":"2024-01-15T18:03:26.296161+0000","last_active":"2024-01-15T18:03:26.297486+0000","last_peered":"2024-01-15T18:03:26.297486+0000","last_clean":"2024-01-15T18:03:26.297486+0000","last_became_active":"2024-01-15T18:03:26.295339+0000","last_became_peered":"2024-01-15T18:03:26.295339+0000","last_unstale":"2024-01-15T18:03:26.297486+0000","last_undegraded":"2024-01-15T18:03:26.297486+0000","last_fullsized":"2024-01-15T18:03:26.297486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:20:43.839295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302217+0000","last_change":"2024-01-15T18:03:27.330393+0000","last_active":"2024-01-15T18:03:28.302217+0000","last_peered":"2024-01-15T18:03:28.302217+0000","last_clean":"2024-01-15T18:03:28.302217+0000","last_became_active":"2024-01-15T18:03:27.329266+0000","last_became_peered":"2024-01-15T18:03:27.329266+0000","last_unstale":"2024-01-15T18:03:28.302217+0000","last_undegraded":"2024-01-15T18:03:28.302217+0000","last_fullsized":"2024-01-15T18:03:28.302217+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:50.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.311753+0000","last_change":"2024-01-15T18:03:26.296244+0000","last_active":"2024-01-15T18:03:27.311753+0000","last_peered":"2024-01-15T18:03:27.311753+0000","last_clean":"2024-01-15T18:03:27.311753+0000","last_became_active":"2024-01-15T18:03:26.295955+0000","last_became_peered":"2024-01-15T18:03:26.295955+0000","last_unstale":"2024-01-15T18:03:27.311753+0000","last_undegraded":"2024-01-15T18:03:27.311753+0000","last_fullsized":"2024-01-15T18:03:27.311753+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:44:34.308154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.331003+0000","last_change":"2024-01-15T18:03:27.323628+0000","last_active":"2024-01-15T18:03:27.331003+0000","last_peered":"2024-01-15T18:03:27.331003+0000","last_clean":"2024-01-15T18:03:27.331003+0000","last_became_active":"2024-01-15T18:03:27.323353+0000","last_became_peered":"2024-01-15T18:03:27.323353+0000","last_unstale":"2024-01-15T18:03:27.331003+0000","last_undegraded":"2024-01-15T18:03:27.331003+0000","last_fullsized":"2024-01-15T18:03:27.331003+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:12.045333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297615+0000","last_change":"2024-01-15T18:03:26.295494+0000","last_active":"2024-01-15T18:03:26.297615+0000","last_peered":"2024-01-15T18:03:26.297615+0000","last_clean":"2024-01-15T18:03:26.297615+0000","last_became_active":"2024-01-15T18:03:26.295221+0000","last_became_peered":"2024-01-15T18:03:26.295221+0000","last_unstale":"2024-01-15T18:03:26.297615+0000","last_undegraded":"2024-01-15T18:03:26.297615+0000","last_fullsized":"2024-01-15T18:03:26.297615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:33.869808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.285274+0000","last_change":"2024-01-15T18:03:26.285274+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.285274+0000","last_undegraded":"2024-01-15T18:03:26.285274+0000","last_fullsized":"2024-01-15T18:03:26.285274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294031+0000","last_change":"2024-01-15T18:03:26.291361+0000","last_active":"2024-01-15T18:03:26.294031+0000","last_peered":"2024-01-15T18:03:26.294031+0000","last_clean":"2024-01-15T18:03:26.294031+0000","last_became_active":"2024-01-15T18:03:26.290863+0000","last_became_peered":"2024-01-15T18:03:26.290863+0000","last_unstale":"2024-01-15T18:03:26.294031+0000","last_undegraded":"2024-01-15T18:03:26.294031+0000","last_fullsized":"2024-01-15T18:03:26.294031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:24:07.021223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.332585+0000","last_change":"2024-01-15T18:03:27.325192+0000","last_active":"2024-01-15T18:03:27.332585+0000","last_peered":"2024-01-15T18:03:27.332585+0000","last_clean":"2024-01-15T18:03:27.332585+0000","last_became_active":"2024-01-15T18:03:27.324697+0000","last_became_peered":"2024-01-15T18:03:27.324697+0000","last_unstale":"2024-01-15T18:03:27.332585+0000","last_undegraded":"2024-01-15T18:03:27.332585+0000","last_fullsized":"2024-01-15T18:03:27.332585+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:20.920788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.308092+0000","last_change":"2024-01-15T18:03:26.298754+0000","last_active":"2024-01-15T18:03:27.308092+0000","last_peered":"2024-01-15T18:03:27.308092+0000","last_clean":"2024-01-15T18:03:27.308092+0000","last_became_active":"2024-01-15T18:03:26.298412+0000","last_became_peered":"2024-01-15T18:03:26.298412+0000","last_unstale":"2024-01-15T18:03:27.308092+0000","last_undegraded":"2024-01-15T18:03:27.308092+0000","last_fullsized":"2024-01-15T18:03:27.308092+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:43:09.251315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278407+0000","last_change":"2024-01-15T18:03:26.278407+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278407+0000","last_undegraded":"2024-01-15T18:03:26.278407+0000","last_fullsized":"2024-01-15T18:03:26.278407+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342463+0000","last_change":"2024-01-15T18:03:26.300437+0000","last_active":"2024-01-15T18:03:27.342463+0000","last_peered":"2024-01-15T18:03:27.342463+0000","last_clean":"2024-01-15T18:03:27.342463+0000","last_became_active":"2024-01-15T18:03:26.300210+0000","last_became_peered":"2024-01-15T18:03:26.300210+0000","last_unstale":"2024-01-15T18:03:27.342463+0000","last_undegraded":"2024-01-15T18:03:27.342463+0000","last_fullsized":"2024-01-15T18:03:27.342463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:47.498523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":206,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-01-15T18:03:25.275709+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:03:25.275709+0000","last_peered":"2024-01-15T18:03:25.275709+0000","last_clean":"2024-01-15T18:03:25.275709+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:03:25.275709+0000","last_undegraded":"2024-01-15T18:03:25.275709+0000","last_fullsized":"2024-01-15T18:03:25.275709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1441792,"data_stored":947133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.04,"5min":1.6439999999999999,"15min":1.3180000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":19.843,"5min":19.843,"15min":19.843},"last":0.878},{"interface":"front","average":{"1min":4.2229999999999999,"5min":2.3180000000000001,"15min":1.552},"min":{"1min":0.75700000000000001,"5min":0.629,"15min":0.623},"max":{"1min":30.498000000000001,"5min":38.137,"15min":38.137},"last":1.0149999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.129,"5min":1.716,"15min":1.383},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":19.917000000000002,"5min":19.917000000000002,"15min":19.917000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":2.2839999999999998,"5min":2.109,"15min":1.546},"min":{"1min":0.71199999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":10.361000000000001,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.143}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":1.6910000000000001,"15min":1.345},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":8.1289999999999996,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":0.94899999999999995},{"interface":"front","average":{"1min":3.0760000000000001,"5min":1.8149999999999999,"15min":1.5169999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":13.478999999999999,"5min":13.478999999999999,"15min":13.478999999999999},"last":1.202}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.3359999999999999,"5min":1.9450000000000001,"15min":1.5129999999999999},"min":{"1min":0.56000000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":16.632000000000001,"5min":16.632000000000001,"15min":16.632000000000001},"last":1.284},{"interface":"front","average":{"1min":3.177,"5min":1.867,"15min":1.617},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":13.439,"5min":13.439,"15min":13.439},"last":1.4279999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":2.145,"15min":1.694},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50700000000000001},"max":{"1min":10.881,"5min":22.359999999999999,"15min":22.359999999999999},"last":1.254},{"interface":"front","average":{"1min":4.4820000000000002,"5min":2.2149999999999999,"15min":1.702},"min":{"1min":0.52700000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":38.682000000000002,"5min":38.682000000000002,"15min":38.682000000000002},"last":1.633}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.8140000000000001,"5min":2.286,"15min":1.831},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":13.379,"5min":25.006,"15min":25.006},"last":1.4950000000000001},{"interface":"front","average":{"1min":2.9169999999999998,"5min":2.2669999999999999,"15min":1.8069999999999999},"min":{"1min":0.80200000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":15.664999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.661}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.4729999999999999,"5min":2.5,"15min":1.954},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.59299999999999997},"max":{"1min":11.366,"5min":23.834,"15min":23.834},"last":1.794},{"interface":"front","average":{"1min":2.8380000000000001,"5min":2.8050000000000002,"15min":2.012},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":9.2599999999999998,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.7310000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45900,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325940,"statfs":{"total":96636764160,"available":96589762560,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.325,"15min":1.3340000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":7.79,"5min":7.79,"15min":12.544},"last":0.88500000000000001},{"interface":"front","average":{"1min":2.1379999999999999,"5min":1.579,"15min":1.3149999999999999},"min":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":14.013,"5min":14.013,"15min":14.013},"last":1.5009999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.403,"15min":1.284},"min":{"1min":0.67400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":7.6200000000000001,"5min":7.6200000000000001,"15min":12.811999999999999},"last":1.107},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.8999999999999999,"15min":1.4199999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":20.523,"5min":20.523,"15min":20.523},"last":1.4199999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":1.877,"15min":1.4279999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":17.713000000000001,"5min":17.713000000000001,"15min":17.713000000000001},"last":1.871},{"interface":"front","average":{"1min":1.788,"5min":1.3799999999999999,"15min":1.4850000000000001},"min":{"1min":0.59899999999999998,"5min":0.504,"15min":0.504},"max":{"1min":7.702,"5min":7.702,"15min":12.897},"last":1.006}]},{"osd":3,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0840000000000001,"5min":1.74,"15min":1.6000000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.7480000000000002,"5min":9.5,"15min":12.717000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":2.3410000000000002,"5min":1.825,"15min":1.518},"min":{"1min":0.876,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":13.795,"5min":14.066000000000001,"15min":14.066000000000001},"last":1.607}]},{"osd":4,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":4.0739999999999998,"5min":2.0659999999999998,"15min":1.6200000000000001},"min":{"1min":0.59599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.5760000000000001},{"interface":"front","average":{"1min":2.2650000000000001,"5min":1.774,"15min":1.6890000000000001},"min":{"1min":0.52000000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":7.8319999999999999,"5min":17.289000000000001,"15min":17.289000000000001},"last":1.248}]},{"osd":5,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.9529999999999998,"5min":1.819,"15min":1.5620000000000001},"min":{"1min":0.66900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":13.593,"5min":14.176,"15min":14.176},"last":1.8440000000000001},{"interface":"front","average":{"1min":3.4950000000000001,"5min":2.1110000000000002,"15min":1.849},"min":{"1min":0.621,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":17.949000000000002,"5min":17.949000000000002,"15min":17.949000000000002},"last":1.752}]},{"osd":7,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.4359999999999999,"5min":2.0190000000000001,"15min":1.708},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":21.265000000000001,"5min":21.265000000000001,"15min":21.265000000000001},"last":1.71},{"interface":"front","average":{"1min":4.3949999999999996,"5min":2.3439999999999999,"15min":1.9299999999999999},"min":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":19.571000000000002,"5min":19.571000000000002,"15min":19.571000000000002},"last":1.665}]}]},{"osd":5,"up_from":14,"seq":60129542248,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45900,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325940,"statfs":{"total":96636764160,"available":96589762560,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9890000000000001,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":19.266999999999999,"5min":19.266999999999999,"15min":19.266999999999999},"last":1.236},{"interface":"front","average":{"1min":1.292,"5min":1.607,"15min":1.4330000000000001},"min":{"1min":0.69399999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.1760000000000002,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.55}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":1.5469999999999999,"15min":1.3169999999999999},"min":{"1min":0.71299999999999997,"5min":0.499,"15min":0.499},"max":{"1min":9.1259999999999994,"5min":16.297000000000001,"15min":16.297000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.8899999999999999,"5min":1.3859999999999999,"15min":1.2629999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2379999999999995,"5min":9.2379999999999995,"15min":9.2379999999999995},"last":1.0569999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.345,"15min":1.262},"min":{"1min":0.88600000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.0369999999999999,"5min":8.1820000000000004,"15min":8.1820000000000004},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.8220000000000001,"5min":1.6479999999999999,"15min":1.3899999999999999},"min":{"1min":0.88500000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2159999999999993,"5min":16.158999999999999,"15min":16.158999999999999},"last":0.996}]},{"osd":3,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.5549999999999999,"15min":1.4370000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.56100000000000005},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":14.914},"last":1.341},{"interface":"front","average":{"1min":1.9930000000000001,"5min":1.526,"15min":1.4570000000000001},"min":{"1min":0.76300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":7.0110000000000001,"5min":7.0110000000000001,"15min":7.0110000000000001},"last":1.3879999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9850000000000001,"5min":1.6579999999999999,"15min":1.4870000000000001},"min":{"1min":0.76100000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":9.0410000000000004,"5min":16.414999999999999,"15min":16.414999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.925,"15min":1.627},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":9.1519999999999992,"5min":17.991,"15min":17.991},"last":1.51}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.492,"15min":1.573},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.669,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":2.0609999999999999,"5min":1.788,"15min":1.623},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":9.1649999999999991,"5min":16.364000000000001,"15min":16.364000000000001},"last":1.468}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9370000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"min":{"1min":0.66300000000000003,"5min":0.628,"15min":0.628},"max":{"1min":9.0600000000000005,"5min":9.0600000000000005,"15min":9.0600000000000005},"last":1.423},{"interface":"front","average":{"1min":3.0600000000000001,"5min":1.891,"15min":1.79},"min":{"1min":0.83999999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":19.356999999999999,"5min":19.356999999999999,"15min":19.356999999999999},"last":1.669}]}]},{"osd":4,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50060,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321780,"statfs":{"total":96636764160,"available":96585502720,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.5229999999999999,"15min":1.3149999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":7.8869999999999996,"5min":11.413,"15min":11.413},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.8180000000000001,"5min":1.534,"15min":1.401},"min":{"1min":0.63,"5min":0.63,"15min":0.53700000000000003},"max":{"1min":7.9160000000000004,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":0.84899999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.367,"15min":1.369},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.45000000000000001},"max":{"1min":3.5750000000000002,"5min":9.7010000000000005,"15min":11.987},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.444,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.6109999999999998,"5min":16.058,"15min":16.058},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.7789999999999999,"15min":1.575},"min":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.6890000000000001,"5min":13.865,"15min":13.865},"last":1.052},{"interface":"front","average":{"1min":1.788,"5min":1.833,"15min":1.5009999999999999},"min":{"1min":0.89100000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":5.7050000000000001,"5min":13.903,"15min":13.903},"last":1.3879999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.919,"5min":1.9099999999999999,"15min":1.675},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":11.773,"5min":26.055,"15min":26.055},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.673,"15min":1.611},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":4.0419999999999998,"5min":13.25,"15min":13.25},"last":1.004}]},{"osd":5,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":2.9729999999999999,"5min":1.853,"15min":1.72},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":29.422000000000001,"5min":29.422000000000001,"15min":29.422000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.5720000000000001,"15min":1.619},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.7160000000000002,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.216}]},{"osd":6,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.6439999999999999,"15min":1.6379999999999999},"min":{"1min":0.80100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":5.7919999999999998,"5min":18.260000000000002,"15min":18.260000000000002},"last":1.579},{"interface":"front","average":{"1min":1.597,"5min":1.6890000000000001,"15min":1.7849999999999999},"min":{"1min":0.59999999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.6459999999999999,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.7410000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.982,"5min":1.881,"15min":1.889},"min":{"1min":0.60499999999999998,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":10.664,"5min":10.664,"15min":11.894},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.851,"5min":2.0379999999999998,"15min":1.9390000000000001},"min":{"1min":0.61399999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.3319999999999999,"5min":11.714,"15min":11.714},"last":1.6679999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58376,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313464,"statfs":{"total":96636764160,"available":96576987136,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5390000000000001,"5min":1.5580000000000001,"15min":1.1830000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":18.399999999999999,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":2.7730000000000001,"5min":1.8500000000000001,"15min":1.349},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.795999999999999,"5min":21.047000000000001,"15min":21.047000000000001},"last":0.91000000000000003}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5499999999999998,"5min":1.736,"15min":1.304},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":12.372,"5min":17.777999999999999,"15min":17.777999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.367,"15min":1.179},"min":{"1min":0.498,"5min":0.48099999999999998,"15min":0.39500000000000002},"max":{"1min":11.002000000000001,"5min":11.002000000000001,"15min":11.002000000000001},"last":1.081}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.413,"15min":1.244},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":1.0309999999999999},{"interface":"front","average":{"1min":2.2599999999999998,"5min":1.72,"15min":1.488},"min":{"1min":0.63300000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":20.495000000000001,"5min":20.495000000000001,"15min":20.495000000000001},"last":0.98899999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4380000000000002,"5min":1.7310000000000001,"15min":1.4950000000000001},"min":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.63500000000000001},"max":{"1min":11.826000000000001,"5min":11.826000000000001,"15min":11.826000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":3.1379999999999999,"5min":1.9950000000000001,"15min":1.625},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.54600000000000004},"max":{"1min":12.439,"5min":12.439,"15min":12.439},"last":1.3500000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.867,"15min":1.6379999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.57299999999999995},"max":{"1min":20.606000000000002,"5min":20.606000000000002,"15min":20.606000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.161,"5min":1.8120000000000001,"15min":1.6870000000000001},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.68000000000000005},"max":{"1min":11.603,"5min":11.603,"15min":11.603},"last":1.9019999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.0129999999999999,"5min":2.1059999999999999,"15min":1.823},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":12.416,"5min":12.416,"15min":12.416},"last":1.8540000000000001},{"interface":"front","average":{"1min":2.391,"5min":1.962,"15min":1.768},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":20.556000000000001,"5min":20.556000000000001,"15min":20.556000000000001},"last":1.8280000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.8460000000000001,"5min":2.331,"15min":1.8999999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.70699999999999996},"max":{"1min":28.843,"5min":28.843,"15min":28.843},"last":2.0019999999999998},{"interface":"front","average":{"1min":3.4940000000000002,"5min":2.3199999999999998,"15min":1.9219999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":15.143000000000001,"5min":15.143000000000001,"15min":15.143000000000001},"last":1.8029999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542251,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":884736,"data_stored":371564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.3889999999999998,"5min":8.3279999999999994,"15min":14.427},"last":0.748},{"interface":"front","average":{"1min":1.893,"5min":1.3979999999999999,"15min":1.1299999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.432},"max":{"1min":11.243,"5min":11.243,"15min":11.243},"last":1.46}]},{"osd":1,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.4139999999999999,"15min":1.357},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40300000000000002},"max":{"1min":4.2779999999999996,"5min":11.718,"15min":14.609},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.278,"15min":1.224},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":4.3449999999999998,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":1.0629999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6749999999999998,"5min":1.7050000000000001,"15min":1.3859999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.34799999999999998},"max":{"1min":12.93,"5min":19.535,"15min":19.535},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.657,"5min":1.589,"15min":1.518},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.2830000000000004,"5min":13.406000000000001,"15min":14.528},"last":0.999}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.6320000000000001,"5min":2.0880000000000001,"15min":1.8069999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":29.986999999999998,"5min":29.986999999999998,"15min":29.986999999999998},"last":1.347},{"interface":"front","average":{"1min":2.9140000000000001,"5min":1.9530000000000001,"15min":1.619},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":21.199999999999999,"5min":21.199999999999999,"15min":21.199999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.1960000000000002,"5min":1.8200000000000001,"15min":1.6220000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":12.227,"5min":19.681999999999999,"15min":19.681999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.7410000000000001,"15min":1.698},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":4.0519999999999996,"5min":10.108000000000001,"15min":14.702999999999999},"last":1.6200000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.726,"15min":1.5629999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.5839999999999996,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":2.0640000000000001,"5min":1.8720000000000001,"15min":1.631},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":12.176,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.6599999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":5.6159999999999997,"5min":2.5179999999999998,"15min":1.907},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":57.006999999999998,"5min":57.006999999999998,"15min":57.006999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":3.8799999999999999,"5min":2.5950000000000002,"15min":1.9690000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":21.795999999999999,"5min":21.795999999999999,"15min":21.795999999999999},"last":1.796}]}]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54220,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317620,"statfs":{"total":96636764160,"available":96581242880,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.27,"15min":1.0209999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42799999999999999},"max":{"1min":13.749000000000001,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.751},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.157,"15min":1.0449999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":18.658000000000001,"5min":18.658000000000001,"15min":18.658000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.331,"15min":1.0329999999999999},"min":{"1min":0.48599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.3029999999999999,"5min":21.126000000000001,"15min":21.126000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.585,"15min":1.3109999999999999},"min":{"1min":0.52700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":9.7279999999999998,"5min":11.5,"15min":11.5},"last":1.0389999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7350000000000001,"5min":1.4279999999999999,"15min":1.855},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":13.593,"5min":13.593,"15min":22.106999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.2250000000000001,"15min":1.26},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.9610000000000003,"5min":7.9610000000000003,"15min":15.510999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.708,"15min":1.726},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72499999999999998},"max":{"1min":3.27,"5min":14.827,"15min":14.827},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.6839999999999999,"15min":1.8109999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3809999999999998,"5min":14.851000000000001,"15min":14.851000000000001},"last":1.4299999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":1.643,"15min":1.7829999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.0700000000000003,"5min":11.552,"15min":13.228999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":3.1400000000000001,"5min":1.929,"15min":1.8089999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.81399999999999995},"max":{"1min":18.452000000000002,"5min":18.452000000000002,"15min":20.823},"last":0.83699999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.0910000000000002,"5min":1.9890000000000001,"15min":2.0139999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":34.158000000000001,"5min":34.158000000000001,"15min":34.158000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.6200000000000001,"15min":1.534},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.52,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.1950000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6859999999999999,"5min":1.865,"15min":1.6639999999999999},"min":{"1min":0.84599999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":17.928999999999998,"5min":17.928999999999998,"15min":17.928999999999998},"last":1.109},{"interface":"front","average":{"1min":1.673,"5min":1.9470000000000001,"15min":1.6839999999999999},"min":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.9139999999999997,"5min":24.289999999999999,"15min":24.289999999999999},"last":1.7150000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542250,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54796,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317044,"statfs":{"total":96636764160,"available":96580653056,"internally_reserved":0,"allocated":1441792,"data_stored":947133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.0649999999999999,"15min":0.94199999999999995},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":6.1890000000000001,"5min":6.1890000000000001,"15min":6.1890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.806,"5min":1.413,"15min":1.1899999999999999},"min":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":11.470000000000001,"5min":11.470000000000001,"15min":11.470000000000001},"last":1.3959999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.333,"15min":1.117},"min":{"1min":0.45500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":11.59,"5min":11.59,"15min":11.59},"last":1.323},{"interface":"front","average":{"1min":2.7040000000000002,"5min":1.377,"15min":1.0309999999999999},"min":{"1min":0.41399999999999998,"5min":0.38200000000000001,"15min":0.35899999999999999},"max":{"1min":14.512,"5min":14.512,"15min":14.512},"last":1.2110000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5920000000000001,"15min":1.3720000000000001},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":11.544,"5min":11.544,"15min":11.544},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.446,"15min":1.194},"min":{"1min":0.41999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.1539999999999999,"5min":14.74,"15min":14.74},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.452,"5min":2.0379999999999998,"15min":1.532},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":32.115000000000002,"5min":32.115000000000002,"15min":32.115000000000002},"last":1.536},{"interface":"front","average":{"1min":2.2280000000000002,"5min":1.903,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":11.992000000000001,"5min":15.975,"15min":15.975},"last":1.4570000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.4870000000000001,"5min":2.1560000000000001,"15min":1.629},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":32.174999999999997,"5min":32.174999999999997,"15min":32.174999999999997},"last":1.369},{"interface":"front","average":{"1min":2.871,"5min":1.915,"15min":1.623},"min":{"1min":0.875,"5min":0.875,"15min":0.754},"max":{"1min":17.225000000000001,"5min":17.225000000000001,"15min":17.225000000000001},"last":1.4830000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.097,"5min":1.746,"15min":1.587},"min":{"1min":0.99199999999999999,"5min":0.96999999999999997,"15min":0.78000000000000003},"max":{"1min":6.1189999999999998,"5min":6.1189999999999998,"15min":6.1189999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":4.3369999999999997,"5min":2.1120000000000001,"15min":1.669},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":32.216999999999999,"5min":32.216999999999999,"15min":32.216999999999999},"last":1.609}]},{"osd":7,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5350000000000001,"5min":2.5710000000000002,"15min":1.9550000000000001},"min":{"1min":1.036,"5min":0.98899999999999999,"15min":0.81299999999999994},"max":{"1min":34.131,"5min":34.131,"15min":34.131},"last":1.508},{"interface":"front","average":{"1min":4.6580000000000004,"5min":2.3799999999999999,"15min":1.8320000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.70999999999999996},"max":{"1min":21.884,"5min":21.884,"15min":21.884},"last":1.266}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:03:32.368 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:03:32.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.628+0000 7fe891e00640 1 Processor -- start 2024-01-15T18:03:32.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.628+0000 7fe891e00640 1 -- start start 2024-01-15T18:03:32.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.629+0000 7fe891e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe88c0cf9a0 0x7fe88c166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.629+0000 7fe891e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c15c5f0 con 0x7fe88c13df80 2024-01-15T18:03:32.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.630+0000 7fe891e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88c06c470 con 0x7fe88c13e360 2024-01-15T18:03:32.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.630+0000 7fe891e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88c06c750 con 0x7fe88c0cf9a0 2024-01-15T18:03:32.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.630+0000 7fe88aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe88c13df80 0x7fe88c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43382/0 (socket says 172.21.15.26:43382) 2024-01-15T18:03:32.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.630+0000 7fe88aa00640 1 -- 172.21.15.26:0/764215050 learned_addr learned my addr 172.21.15.26:0/764215050 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:32.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.630+0000 7fe88b400640 1 --2- 172.21.15.26:0/764215050 >> v2:172.21.15.124:3300/0 conn(0x7fe88c0cf9a0 0x7fe88c166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848878376 0 0) 0x7fe88c15c5f0 con 0x7fe88c13df80 2024-01-15T18:03:32.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe86c003520 con 0x7fe88c13df80 2024-01-15T18:03:32.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1063368699 0 0) 0x7fe86c003520 con 0x7fe88c13df80 2024-01-15T18:03:32.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 >> v2:172.21.15.124:3300/0 conn(0x7fe88c0cf9a0 msgr2=0x7fe88c166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:32.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 --2- 172.21.15.26:0/764215050 >> v2:172.21.15.124:3300/0 conn(0x7fe88c0cf9a0 0x7fe88c166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 >> v1:172.21.15.124:6789/0 conn(0x7fe88c13e360 legacy=0x7fe88c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c06ca30 con 0x7fe88c13df80 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fe87c001670 con 0x7fe88c13df80 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 >> v1:172.21.15.26:6789/0 conn(0x7fe88c13df80 legacy=0x7fe88c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c003970 0x7fe86c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe86c0047c0 0x7fe86c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe86c005180 0x7fe86c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c06c750 con 0x7fe86c0047c0 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c00c020 con 0x7fe86c005180 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.631+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c00c1e0 con 0x7fe86c003970 2024-01-15T18:03:32.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88a000640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe86c0047c0 0x7fe86c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88aa00640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c003970 0x7fe86c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88b400640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe86c005180 0x7fe86c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88a000640 1 -- 172.21.15.26:0/764215050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c003970 msgr2=0x7fe86c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:32.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88a000640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c003970 0x7fe86c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88a000640 1 -- 172.21.15.26:0/764215050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe86c005180 msgr2=0x7fe86c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:32.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88a000640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe86c005180 0x7fe86c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88a000640 1 -- 172.21.15.26:0/764215050 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c003520 con 0x7fe86c0047c0 2024-01-15T18:03:32.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88b400640 1 -- 172.21.15.26:0/764215050 reap_dead start 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.632+0000 7fe88aa00640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c003970 0x7fe86c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.633+0000 7fe88a000640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe86c0047c0 0x7fe86c004c40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe874001e20 tx=0x7fe86c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a175c635707b1d20 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.633+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874003b70 con 0x7fe86c0047c0 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.633+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe874003cd0 con 0x7fe86c0047c0 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.633+0000 7fe889600640 1 -- 172.21.15.26:0/764215050 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874005310 con 0x7fe86c0047c0 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.634+0000 7fe891e00640 1 -- 172.21.15.26:0/764215050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe86c0047c0 msgr2=0x7fe86c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.634+0000 7fe891e00640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe86c0047c0 0x7fe86c004c40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe874001e20 tx=0x7fe86c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.634+0000 7fe891e00640 1 -- 172.21.15.26:0/764215050 shutdown_connections 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.634+0000 7fe891e00640 1 --2- 172.21.15.26:0/764215050 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe86c0047c0 0x7fe86c004c40 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.634+0000 7fe891e00640 1 -- 172.21.15.26:0/764215050 >> 172.21.15.26:0/764215050 conn(0x7fe88c06a800 msgr2=0x7fe88c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:32.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.634+0000 7fe891e00640 1 -- 172.21.15.26:0/764215050 shutdown_connections 2024-01-15T18:03:32.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.634+0000 7fe891e00640 1 -- 172.21.15.26:0/764215050 wait complete. 2024-01-15T18:03:32.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.636+0000 7fe891e00640 1 Processor -- start 2024-01-15T18:03:32.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.636+0000 7fe891e00640 1 -- start start 2024-01-15T18:03:32.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.637+0000 7fe891e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c0047c0 0x7fe88c134b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.637+0000 7fe891e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88c14ab20 0x7fe88c152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.637+0000 7fe891e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88c14afa0 0x7fe88c153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.637+0000 7fe891e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c1350c0 con 0x7fe88c14afa0 2024-01-15T18:03:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.637+0000 7fe891e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c14bb30 con 0x7fe88c14ab20 2024-01-15T18:03:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.637+0000 7fe891e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c14b570 con 0x7fe86c0047c0 2024-01-15T18:03:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88c14afa0 0x7fe88c153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88c14afa0 0x7fe88c153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50548/0 (socket says 172.21.15.26:50548) 2024-01-15T18:03:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88b400640 1 -- 172.21.15.26:0/2702982349 learned_addr learned my addr 172.21.15.26:0/2702982349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88a000640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88c14ab20 0x7fe88c152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88aa00640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c0047c0 0x7fe88c134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88b400640 1 -- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c0047c0 msgr2=0x7fe88c134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88b400640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c0047c0 0x7fe88c134b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88b400640 1 -- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88c14ab20 msgr2=0x7fe88c152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88b400640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88c14ab20 0x7fe88c152c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88b400640 1 -- 172.21.15.26:0/2702982349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c14b850 con 0x7fe88c14afa0 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.638+0000 7fe88b400640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88c14afa0 0x7fe88c153150 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0ccae0 tx=0x7fe884010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.639+0000 7fe883400640 1 -- 172.21.15.26:0/2702982349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe884004230 con 0x7fe88c14afa0 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.639+0000 7fe883400640 1 -- 172.21.15.26:0/2702982349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe884004390 con 0x7fe88c14afa0 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.639+0000 7fe883400640 1 -- 172.21.15.26:0/2702982349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88400e8e0 con 0x7fe88c14afa0 2024-01-15T18:03:32.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.639+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c14c310 con 0x7fe88c14afa0 2024-01-15T18:03:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.639+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c150eb0 con 0x7fe88c14afa0 2024-01-15T18:03:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.640+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88c0675d0 con 0x7fe88c14afa0 2024-01-15T18:03:32.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.641+0000 7fe883400640 1 -- 172.21.15.26:0/2702982349 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fe88400ea40 con 0x7fe88c14afa0 2024-01-15T18:03:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.642+0000 7fe883400640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe86406bbf0 0x7fe86406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.642+0000 7fe883400640 1 -- 172.21.15.26:0/2702982349 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe884088360 con 0x7fe88c14afa0 2024-01-15T18:03:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.643+0000 7fe88aa00640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe86406bbf0 0x7fe86406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:32.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.644+0000 7fe88aa00640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe86406bbf0 0x7fe86406e0b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe88c14c170 tx=0x7fe87c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:32.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.647+0000 7fe883400640 1 -- 172.21.15.26:0/2702982349 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe884054a30 con 0x7fe88c14afa0 2024-01-15T18:03:32.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.836+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe88c06cff0 con 0x7fe86406bbf0 2024-01-15T18:03:32.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.858+0000 7fe883400640 1 -- 172.21.15.26:0/2702982349 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283196 (secure 0 0 0) 0x7fe87c00d070 con 0x7fe86406bbf0 2024-01-15T18:03:32.868 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:32.870 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:03:32.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe86406bbf0 msgr2=0x7fe86406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe86406bbf0 0x7fe86406e0b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe88c14c170 tx=0x7fe87c009040 comp rx=0 tx=0).stop 2024-01-15T18:03:32.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88c14afa0 msgr2=0x7fe88c153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:32.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88c14afa0 0x7fe88c153150 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0ccae0 tx=0x7fe884010c20 comp rx=0 tx=0).stop 2024-01-15T18:03:32.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 shutdown_connections 2024-01-15T18:03:32.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe86406bbf0 0x7fe86406e0b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe88c14afa0 0x7fe88c153150 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe88c14ab20 0x7fe88c152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 --2- 172.21.15.26:0/2702982349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe86c0047c0 0x7fe88c134b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 >> 172.21.15.26:0/2702982349 conn(0x7fe88c06a800 msgr2=0x7fe88c15cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 shutdown_connections 2024-01-15T18:03:32.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:32.864+0000 7fe891e00640 1 -- 172.21.15.26:0/2702982349 wait complete. 2024-01-15T18:03:32.886 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-01-15T18:03:32.104844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":168,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409992,"kb_used_data":7820,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754564728,"statfs":{"total":773094113280,"available":772674281472,"internally_reserved":0,"allocated":8007680,"data_stored":4049655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001932"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281242+0000","last_change":"2024-01-15T18:03:26.281242+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281242+0000","last_undegraded":"2024-01-15T18:03:26.281242+0000","last_fullsized":"2024-01-15T18:03:26.281242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342379+0000","last_change":"2024-01-15T18:03:27.341616+0000","last_active":"2024-01-15T18:03:27.342379+0000","last_peered":"2024-01-15T18:03:27.342379+0000","last_clean":"2024-01-15T18:03:27.342379+0000","last_became_active":"2024-01-15T18:03:27.341248+0000","last_became_peered":"2024-01-15T18:03:27.341248+0000","last_unstale":"2024-01-15T18:03:27.342379+0000","last_undegraded":"2024-01-15T18:03:27.342379+0000","last_fullsized":"2024-01-15T18:03:27.342379+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:51.975291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283811+0000","last_change":"2024-01-15T18:03:26.283811+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283811+0000","last_undegraded":"2024-01-15T18:03:26.283811+0000","last_fullsized":"2024-01-15T18:03:26.283811+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.288618+0000","last_change":"2024-01-15T18:03:26.288618+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.288618+0000","last_undegraded":"2024-01-15T18:03:26.288618+0000","last_fullsized":"2024-01-15T18:03:26.288618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302106+0000","last_change":"2024-01-15T18:03:27.319087+0000","last_active":"2024-01-15T18:03:28.302106+0000","last_peered":"2024-01-15T18:03:28.302106+0000","last_clean":"2024-01-15T18:03:28.302106+0000","last_became_active":"2024-01-15T18:03:27.318702+0000","last_became_peered":"2024-01-15T18:03:27.318702+0000","last_unstale":"2024-01-15T18:03:28.302106+0000","last_undegraded":"2024-01-15T18:03:28.302106+0000","last_fullsized":"2024-01-15T18:03:28.302106+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:13:57.340165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.321499+0000","last_change":"2024-01-15T18:03:27.319124+0000","last_active":"2024-01-15T18:03:27.321499+0000","last_peered":"2024-01-15T18:03:27.321499+0000","last_clean":"2024-01-15T18:03:27.321499+0000","last_became_active":"2024-01-15T18:03:27.318855+0000","last_became_peered":"2024-01-15T18:03:27.318855+0000","last_unstale":"2024-01-15T18:03:27.321499+0000","last_undegraded":"2024-01-15T18:03:27.321499+0000","last_fullsized":"2024-01-15T18:03:27.321499+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:40.231033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284505+0000","last_change":"2024-01-15T18:03:26.284505+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284505+0000","last_undegraded":"2024-01-15T18:03:26.284505+0000","last_fullsized":"2024-01-15T18:03:26.284505+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.321938+0000","last_change":"2024-01-15T18:03:27.316226+0000","last_active":"2024-01-15T18:03:27.321938+0000","last_peered":"2024-01-15T18:03:27.321938+0000","last_clean":"2024-01-15T18:03:27.321938+0000","last_became_active":"2024-01-15T18:03:27.315889+0000","last_became_peered":"2024-01-15T18:03:27.315889+0000","last_unstale":"2024-01-15T18:03:27.321938+0000","last_undegraded":"2024-01-15T18:03:27.321938+0000","last_fullsized":"2024-01-15T18:03:27.321938+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:56.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302728+0000","last_change":"2024-01-15T18:03:27.331363+0000","last_active":"2024-01-15T18:03:28.302728+0000","last_peered":"2024-01-15T18:03:28.302728+0000","last_clean":"2024-01-15T18:03:28.302728+0000","last_became_active":"2024-01-15T18:03:27.330530+0000","last_became_peered":"2024-01-15T18:03:27.330530+0000","last_unstale":"2024-01-15T18:03:28.302728+0000","last_undegraded":"2024-01-15T18:03:28.302728+0000","last_fullsized":"2024-01-15T18:03:28.302728+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:43:50.724930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.275264+0000","last_change":"2024-01-15T18:03:26.275264+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.275264+0000","last_undegraded":"2024-01-15T18:03:26.275264+0000","last_fullsized":"2024-01-15T18:03:26.275264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282234+0000","last_change":"2024-01-15T18:03:26.282234+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282234+0000","last_undegraded":"2024-01-15T18:03:26.282234+0000","last_fullsized":"2024-01-15T18:03:26.282234+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.288002+0000","last_change":"2024-01-15T18:03:26.288002+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.288002+0000","last_undegraded":"2024-01-15T18:03:26.288002+0000","last_fullsized":"2024-01-15T18:03:26.288002+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282326+0000","last_change":"2024-01-15T18:03:26.282326+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282326+0000","last_undegraded":"2024-01-15T18:03:26.282326+0000","last_fullsized":"2024-01-15T18:03:26.282326+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302207+0000","last_change":"2024-01-15T18:03:27.329433+0000","last_active":"2024-01-15T18:03:28.302207+0000","last_peered":"2024-01-15T18:03:28.302207+0000","last_clean":"2024-01-15T18:03:28.302207+0000","last_became_active":"2024-01-15T18:03:27.329186+0000","last_became_peered":"2024-01-15T18:03:27.329186+0000","last_unstale":"2024-01-15T18:03:28.302207+0000","last_undegraded":"2024-01-15T18:03:28.302207+0000","last_fullsized":"2024-01-15T18:03:28.302207+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:34.881592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284462+0000","last_change":"2024-01-15T18:03:26.284462+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284462+0000","last_undegraded":"2024-01-15T18:03:26.284462+0000","last_fullsized":"2024-01-15T18:03:26.284462+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.275307+0000","last_change":"2024-01-15T18:03:26.275307+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.275307+0000","last_undegraded":"2024-01-15T18:03:26.275307+0000","last_fullsized":"2024-01-15T18:03:26.275307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302977+0000","last_change":"2024-01-15T18:03:27.330530+0000","last_active":"2024-01-15T18:03:28.302977+0000","last_peered":"2024-01-15T18:03:28.302977+0000","last_clean":"2024-01-15T18:03:28.302977+0000","last_became_active":"2024-01-15T18:03:27.330149+0000","last_became_peered":"2024-01-15T18:03:27.330149+0000","last_unstale":"2024-01-15T18:03:28.302977+0000","last_undegraded":"2024-01-15T18:03:28.302977+0000","last_fullsized":"2024-01-15T18:03:28.302977+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:12.045333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.286446+0000","last_change":"2024-01-15T18:03:26.286446+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.286446+0000","last_undegraded":"2024-01-15T18:03:26.286446+0000","last_fullsized":"2024-01-15T18:03:26.286446+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341755+0000","last_change":"2024-01-15T18:03:27.338146+0000","last_active":"2024-01-15T18:03:27.341755+0000","last_peered":"2024-01-15T18:03:27.341755+0000","last_clean":"2024-01-15T18:03:27.341755+0000","last_became_active":"2024-01-15T18:03:27.337850+0000","last_became_peered":"2024-01-15T18:03:27.337850+0000","last_unstale":"2024-01-15T18:03:27.341755+0000","last_undegraded":"2024-01-15T18:03:27.341755+0000","last_fullsized":"2024-01-15T18:03:27.341755+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:06:27.744895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282114+0000","last_change":"2024-01-15T18:03:26.282114+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282114+0000","last_undegraded":"2024-01-15T18:03:26.282114+0000","last_fullsized":"2024-01-15T18:03:26.282114+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.276325+0000","last_change":"2024-01-15T18:03:26.276325+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.276325+0000","last_undegraded":"2024-01-15T18:03:26.276325+0000","last_fullsized":"2024-01-15T18:03:26.276325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274415+0000","last_change":"2024-01-15T18:03:26.274415+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274415+0000","last_undegraded":"2024-01-15T18:03:26.274415+0000","last_fullsized":"2024-01-15T18:03:26.274415+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279090+0000","last_change":"2024-01-15T18:03:26.279090+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279090+0000","last_undegraded":"2024-01-15T18:03:26.279090+0000","last_fullsized":"2024-01-15T18:03:26.279090+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279397+0000","last_change":"2024-01-15T18:03:26.279397+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279397+0000","last_undegraded":"2024-01-15T18:03:26.279397+0000","last_fullsized":"2024-01-15T18:03:26.279397+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.276036+0000","last_change":"2024-01-15T18:03:26.276036+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.276036+0000","last_undegraded":"2024-01-15T18:03:26.276036+0000","last_fullsized":"2024-01-15T18:03:26.276036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284633+0000","last_change":"2024-01-15T18:03:26.284633+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284633+0000","last_undegraded":"2024-01-15T18:03:26.284633+0000","last_fullsized":"2024-01-15T18:03:26.284633+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283351+0000","last_change":"2024-01-15T18:03:26.283351+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283351+0000","last_undegraded":"2024-01-15T18:03:26.283351+0000","last_fullsized":"2024-01-15T18:03:26.283351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274031+0000","last_change":"2024-01-15T18:03:26.274031+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274031+0000","last_undegraded":"2024-01-15T18:03:26.274031+0000","last_fullsized":"2024-01-15T18:03:26.274031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282955+0000","last_change":"2024-01-15T18:03:26.282955+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282955+0000","last_undegraded":"2024-01-15T18:03:26.282955+0000","last_fullsized":"2024-01-15T18:03:26.282955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283969+0000","last_change":"2024-01-15T18:03:26.283969+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283969+0000","last_undegraded":"2024-01-15T18:03:26.283969+0000","last_fullsized":"2024-01-15T18:03:26.283969+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283167+0000","last_change":"2024-01-15T18:03:26.283167+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283167+0000","last_undegraded":"2024-01-15T18:03:26.283167+0000","last_fullsized":"2024-01-15T18:03:26.283167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.324636+0000","last_change":"2024-01-15T18:03:27.321263+0000","last_active":"2024-01-15T18:03:27.324636+0000","last_peered":"2024-01-15T18:03:27.324636+0000","last_clean":"2024-01-15T18:03:27.324636+0000","last_became_active":"2024-01-15T18:03:27.320958+0000","last_became_peered":"2024-01-15T18:03:27.320958+0000","last_unstale":"2024-01-15T18:03:27.324636+0000","last_undegraded":"2024-01-15T18:03:27.324636+0000","last_fullsized":"2024-01-15T18:03:27.324636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:43:50.724930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279714+0000","last_change":"2024-01-15T18:03:26.279714+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279714+0000","last_undegraded":"2024-01-15T18:03:26.279714+0000","last_fullsized":"2024-01-15T18:03:26.279714+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299516+0000","last_change":"2024-01-15T18:03:26.298227+0000","last_active":"2024-01-15T18:03:26.299516+0000","last_peered":"2024-01-15T18:03:26.299516+0000","last_clean":"2024-01-15T18:03:26.299516+0000","last_became_active":"2024-01-15T18:03:26.297015+0000","last_became_peered":"2024-01-15T18:03:26.297015+0000","last_unstale":"2024-01-15T18:03:26.299516+0000","last_undegraded":"2024-01-15T18:03:26.299516+0000","last_fullsized":"2024-01-15T18:03:26.299516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:43.508315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302905+0000","last_change":"2024-01-15T18:03:27.330723+0000","last_active":"2024-01-15T18:03:28.302905+0000","last_peered":"2024-01-15T18:03:28.302905+0000","last_clean":"2024-01-15T18:03:28.302905+0000","last_became_active":"2024-01-15T18:03:27.330236+0000","last_became_peered":"2024-01-15T18:03:27.330236+0000","last_unstale":"2024-01-15T18:03:28.302905+0000","last_undegraded":"2024-01-15T18:03:28.302905+0000","last_fullsized":"2024-01-15T18:03:28.302905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:56.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295089+0000","last_change":"2024-01-15T18:03:26.292579+0000","last_active":"2024-01-15T18:03:26.295089+0000","last_peered":"2024-01-15T18:03:26.295089+0000","last_clean":"2024-01-15T18:03:26.295089+0000","last_became_active":"2024-01-15T18:03:26.291631+0000","last_became_peered":"2024-01-15T18:03:26.291631+0000","last_unstale":"2024-01-15T18:03:26.295089+0000","last_undegraded":"2024-01-15T18:03:26.295089+0000","last_fullsized":"2024-01-15T18:03:26.295089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:32.983343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341535+0000","last_change":"2024-01-15T18:03:27.336001+0000","last_active":"2024-01-15T18:03:27.341535+0000","last_peered":"2024-01-15T18:03:27.341535+0000","last_clean":"2024-01-15T18:03:27.341535+0000","last_became_active":"2024-01-15T18:03:27.335265+0000","last_became_peered":"2024-01-15T18:03:27.335265+0000","last_unstale":"2024-01-15T18:03:27.341535+0000","last_undegraded":"2024-01-15T18:03:27.341535+0000","last_fullsized":"2024-01-15T18:03:27.341535+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:52.859519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.304785+0000","last_change":"2024-01-15T18:03:26.303968+0000","last_active":"2024-01-15T18:03:26.304785+0000","last_peered":"2024-01-15T18:03:26.304785+0000","last_clean":"2024-01-15T18:03:26.304785+0000","last_became_active":"2024-01-15T18:03:26.303669+0000","last_became_peered":"2024-01-15T18:03:26.303669+0000","last_unstale":"2024-01-15T18:03:26.304785+0000","last_undegraded":"2024-01-15T18:03:26.304785+0000","last_fullsized":"2024-01-15T18:03:26.304785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:43.710985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281710+0000","last_change":"2024-01-15T18:03:26.281710+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281710+0000","last_undegraded":"2024-01-15T18:03:26.281710+0000","last_fullsized":"2024-01-15T18:03:26.281710+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342311+0000","last_change":"2024-01-15T18:03:26.300990+0000","last_active":"2024-01-15T18:03:27.342311+0000","last_peered":"2024-01-15T18:03:27.342311+0000","last_clean":"2024-01-15T18:03:27.342311+0000","last_became_active":"2024-01-15T18:03:26.300735+0000","last_became_peered":"2024-01-15T18:03:26.300735+0000","last_unstale":"2024-01-15T18:03:27.342311+0000","last_undegraded":"2024-01-15T18:03:27.342311+0000","last_fullsized":"2024-01-15T18:03:27.342311+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:53:06.533003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283325+0000","last_change":"2024-01-15T18:03:26.283325+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283325+0000","last_undegraded":"2024-01-15T18:03:26.283325+0000","last_fullsized":"2024-01-15T18:03:26.283325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297810+0000","last_change":"2024-01-15T18:03:26.296481+0000","last_active":"2024-01-15T18:03:26.297810+0000","last_peered":"2024-01-15T18:03:26.297810+0000","last_clean":"2024-01-15T18:03:26.297810+0000","last_became_active":"2024-01-15T18:03:26.293509+0000","last_became_peered":"2024-01-15T18:03:26.293509+0000","last_unstale":"2024-01-15T18:03:26.297810+0000","last_undegraded":"2024-01-15T18:03:26.297810+0000","last_fullsized":"2024-01-15T18:03:26.297810+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:48.220096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274045+0000","last_change":"2024-01-15T18:03:26.274045+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274045+0000","last_undegraded":"2024-01-15T18:03:26.274045+0000","last_fullsized":"2024-01-15T18:03:26.274045+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.340684+0000","last_change":"2024-01-15T18:03:26.302584+0000","last_active":"2024-01-15T18:03:27.340684+0000","last_peered":"2024-01-15T18:03:27.340684+0000","last_clean":"2024-01-15T18:03:27.340684+0000","last_became_active":"2024-01-15T18:03:26.301356+0000","last_became_peered":"2024-01-15T18:03:26.301356+0000","last_unstale":"2024-01-15T18:03:27.340684+0000","last_undegraded":"2024-01-15T18:03:27.340684+0000","last_fullsized":"2024-01-15T18:03:27.340684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:22:00.115460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.301672+0000","last_change":"2024-01-15T18:03:27.326042+0000","last_active":"2024-01-15T18:03:28.301672+0000","last_peered":"2024-01-15T18:03:28.301672+0000","last_clean":"2024-01-15T18:03:28.301672+0000","last_became_active":"2024-01-15T18:03:27.325476+0000","last_became_peered":"2024-01-15T18:03:27.325476+0000","last_unstale":"2024-01-15T18:03:28.301672+0000","last_undegraded":"2024-01-15T18:03:28.301672+0000","last_fullsized":"2024-01-15T18:03:28.301672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:37:22.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.293022+0000","last_change":"2024-01-15T18:03:26.282542+0000","last_active":"2024-01-15T18:03:26.293022+0000","last_peered":"2024-01-15T18:03:26.293022+0000","last_clean":"2024-01-15T18:03:26.293022+0000","last_became_active":"2024-01-15T18:03:26.282226+0000","last_became_peered":"2024-01-15T18:03:26.282226+0000","last_unstale":"2024-01-15T18:03:26.293022+0000","last_undegraded":"2024-01-15T18:03:26.293022+0000","last_fullsized":"2024-01-15T18:03:26.293022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:57.562173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281723+0000","last_change":"2024-01-15T18:03:26.281723+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281723+0000","last_undegraded":"2024-01-15T18:03:26.281723+0000","last_fullsized":"2024-01-15T18:03:26.281723+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.312237+0000","last_change":"2024-01-15T18:03:26.285864+0000","last_active":"2024-01-15T18:03:27.312237+0000","last_peered":"2024-01-15T18:03:27.312237+0000","last_clean":"2024-01-15T18:03:27.312237+0000","last_became_active":"2024-01-15T18:03:26.285507+0000","last_became_peered":"2024-01-15T18:03:26.285507+0000","last_unstale":"2024-01-15T18:03:27.312237+0000","last_undegraded":"2024-01-15T18:03:27.312237+0000","last_fullsized":"2024-01-15T18:03:27.312237+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:47.778037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283831+0000","last_change":"2024-01-15T18:03:26.283831+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283831+0000","last_undegraded":"2024-01-15T18:03:26.283831+0000","last_fullsized":"2024-01-15T18:03:26.283831+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295148+0000","last_change":"2024-01-15T18:03:26.294412+0000","last_active":"2024-01-15T18:03:26.295148+0000","last_peered":"2024-01-15T18:03:26.295148+0000","last_clean":"2024-01-15T18:03:26.295148+0000","last_became_active":"2024-01-15T18:03:26.294070+0000","last_became_peered":"2024-01-15T18:03:26.294070+0000","last_unstale":"2024-01-15T18:03:26.295148+0000","last_undegraded":"2024-01-15T18:03:26.295148+0000","last_fullsized":"2024-01-15T18:03:26.295148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:06:26.733111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.280249+0000","last_change":"2024-01-15T18:03:26.280249+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.280249+0000","last_undegraded":"2024-01-15T18:03:26.280249+0000","last_fullsized":"2024-01-15T18:03:26.280249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.312561+0000","last_change":"2024-01-15T18:03:26.302162+0000","last_active":"2024-01-15T18:03:27.312561+0000","last_peered":"2024-01-15T18:03:27.312561+0000","last_clean":"2024-01-15T18:03:27.312561+0000","last_became_active":"2024-01-15T18:03:26.301821+0000","last_became_peered":"2024-01-15T18:03:26.301821+0000","last_unstale":"2024-01-15T18:03:27.312561+0000","last_undegraded":"2024-01-15T18:03:27.312561+0000","last_fullsized":"2024-01-15T18:03:27.312561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:24:07.021223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.340063+0000","last_change":"2024-01-15T18:03:27.329589+0000","last_active":"2024-01-15T18:03:27.340063+0000","last_peered":"2024-01-15T18:03:27.340063+0000","last_clean":"2024-01-15T18:03:27.340063+0000","last_became_active":"2024-01-15T18:03:27.328118+0000","last_became_peered":"2024-01-15T18:03:27.328118+0000","last_unstale":"2024-01-15T18:03:27.340063+0000","last_undegraded":"2024-01-15T18:03:27.340063+0000","last_fullsized":"2024-01-15T18:03:27.340063+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:48.991808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.296636+0000","last_change":"2024-01-15T18:03:26.295197+0000","last_active":"2024-01-15T18:03:26.296636+0000","last_peered":"2024-01-15T18:03:26.296636+0000","last_clean":"2024-01-15T18:03:26.296636+0000","last_became_active":"2024-01-15T18:03:26.294932+0000","last_became_peered":"2024-01-15T18:03:26.294932+0000","last_unstale":"2024-01-15T18:03:26.296636+0000","last_undegraded":"2024-01-15T18:03:26.296636+0000","last_fullsized":"2024-01-15T18:03:26.296636+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:20.869224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302274+0000","last_change":"2024-01-15T18:03:27.315984+0000","last_active":"2024-01-15T18:03:28.302274+0000","last_peered":"2024-01-15T18:03:28.302274+0000","last_clean":"2024-01-15T18:03:28.302274+0000","last_became_active":"2024-01-15T18:03:27.315700+0000","last_became_peered":"2024-01-15T18:03:27.315700+0000","last_unstale":"2024-01-15T18:03:28.302274+0000","last_undegraded":"2024-01-15T18:03:28.302274+0000","last_fullsized":"2024-01-15T18:03:28.302274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:16.050477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294412+0000","last_change":"2024-01-15T18:03:26.283744+0000","last_active":"2024-01-15T18:03:26.294412+0000","last_peered":"2024-01-15T18:03:26.294412+0000","last_clean":"2024-01-15T18:03:26.294412+0000","last_became_active":"2024-01-15T18:03:26.283382+0000","last_became_peered":"2024-01-15T18:03:26.283382+0000","last_unstale":"2024-01-15T18:03:26.294412+0000","last_undegraded":"2024-01-15T18:03:26.294412+0000","last_fullsized":"2024-01-15T18:03:26.294412+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:13:56.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278423+0000","last_change":"2024-01-15T18:03:26.278423+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278423+0000","last_undegraded":"2024-01-15T18:03:26.278423+0000","last_fullsized":"2024-01-15T18:03:26.278423+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295393+0000","last_change":"2024-01-15T18:03:26.290460+0000","last_active":"2024-01-15T18:03:26.295393+0000","last_peered":"2024-01-15T18:03:26.295393+0000","last_clean":"2024-01-15T18:03:26.295393+0000","last_became_active":"2024-01-15T18:03:26.290114+0000","last_became_peered":"2024-01-15T18:03:26.290114+0000","last_unstale":"2024-01-15T18:03:26.295393+0000","last_undegraded":"2024-01-15T18:03:26.295393+0000","last_fullsized":"2024-01-15T18:03:26.295393+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:49.162905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.286454+0000","last_change":"2024-01-15T18:03:26.286454+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.286454+0000","last_undegraded":"2024-01-15T18:03:26.286454+0000","last_fullsized":"2024-01-15T18:03:26.286454+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.306852+0000","last_change":"2024-01-15T18:03:26.302531+0000","last_active":"2024-01-15T18:03:27.306852+0000","last_peered":"2024-01-15T18:03:27.306852+0000","last_clean":"2024-01-15T18:03:27.306852+0000","last_became_active":"2024-01-15T18:03:26.301660+0000","last_became_peered":"2024-01-15T18:03:26.301660+0000","last_unstale":"2024-01-15T18:03:27.306852+0000","last_undegraded":"2024-01-15T18:03:27.306852+0000","last_fullsized":"2024-01-15T18:03:27.306852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:18:48.053993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281591+0000","last_change":"2024-01-15T18:03:26.281591+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281591+0000","last_undegraded":"2024-01-15T18:03:26.281591+0000","last_fullsized":"2024-01-15T18:03:26.281591+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299435+0000","last_change":"2024-01-15T18:03:26.297958+0000","last_active":"2024-01-15T18:03:26.299435+0000","last_peered":"2024-01-15T18:03:26.299435+0000","last_clean":"2024-01-15T18:03:26.299435+0000","last_became_active":"2024-01-15T18:03:26.297417+0000","last_became_peered":"2024-01-15T18:03:26.297417+0000","last_unstale":"2024-01-15T18:03:26.299435+0000","last_undegraded":"2024-01-15T18:03:26.299435+0000","last_fullsized":"2024-01-15T18:03:26.299435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:20:43.839295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.301949+0000","last_change":"2024-01-15T18:03:27.330251+0000","last_active":"2024-01-15T18:03:28.301949+0000","last_peered":"2024-01-15T18:03:28.301949+0000","last_clean":"2024-01-15T18:03:28.301949+0000","last_became_active":"2024-01-15T18:03:27.329948+0000","last_became_peered":"2024-01-15T18:03:27.329948+0000","last_unstale":"2024-01-15T18:03:28.301949+0000","last_undegraded":"2024-01-15T18:03:28.301949+0000","last_fullsized":"2024-01-15T18:03:28.301949+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:58.573957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.293724+0000","last_change":"2024-01-15T18:03:26.287860+0000","last_active":"2024-01-15T18:03:26.293724+0000","last_peered":"2024-01-15T18:03:26.293724+0000","last_clean":"2024-01-15T18:03:26.293724+0000","last_became_active":"2024-01-15T18:03:26.287555+0000","last_became_peered":"2024-01-15T18:03:26.287555+0000","last_unstale":"2024-01-15T18:03:26.293724+0000","last_undegraded":"2024-01-15T18:03:26.293724+0000","last_fullsized":"2024-01-15T18:03:26.293724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:50.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274473+0000","last_change":"2024-01-15T18:03:26.274473+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274473+0000","last_undegraded":"2024-01-15T18:03:26.274473+0000","last_fullsized":"2024-01-15T18:03:26.274473+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341894+0000","last_change":"2024-01-15T18:03:26.300342+0000","last_active":"2024-01-15T18:03:27.341894+0000","last_peered":"2024-01-15T18:03:27.341894+0000","last_clean":"2024-01-15T18:03:27.341894+0000","last_became_active":"2024-01-15T18:03:26.300100+0000","last_became_peered":"2024-01-15T18:03:26.300100+0000","last_unstale":"2024-01-15T18:03:27.341894+0000","last_undegraded":"2024-01-15T18:03:27.341894+0000","last_fullsized":"2024-01-15T18:03:27.341894+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:08.274277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279678+0000","last_change":"2024-01-15T18:03:26.279678+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279678+0000","last_undegraded":"2024-01-15T18:03:26.279678+0000","last_fullsized":"2024-01-15T18:03:26.279678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.303643+0000","last_change":"2024-01-15T18:03:26.298853+0000","last_active":"2024-01-15T18:03:26.303643+0000","last_peered":"2024-01-15T18:03:26.303643+0000","last_clean":"2024-01-15T18:03:26.303643+0000","last_became_active":"2024-01-15T18:03:26.298530+0000","last_became_peered":"2024-01-15T18:03:26.298530+0000","last_unstale":"2024-01-15T18:03:26.303643+0000","last_undegraded":"2024-01-15T18:03:26.303643+0000","last_fullsized":"2024-01-15T18:03:26.303643+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:51.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284222+0000","last_change":"2024-01-15T18:03:26.284222+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284222+0000","last_undegraded":"2024-01-15T18:03:26.284222+0000","last_fullsized":"2024-01-15T18:03:26.284222+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.343468+0000","last_change":"2024-01-15T18:03:26.301874+0000","last_active":"2024-01-15T18:03:27.343468+0000","last_peered":"2024-01-15T18:03:27.343468+0000","last_clean":"2024-01-15T18:03:27.343468+0000","last_became_active":"2024-01-15T18:03:26.301425+0000","last_became_peered":"2024-01-15T18:03:26.301425+0000","last_unstale":"2024-01-15T18:03:27.343468+0000","last_undegraded":"2024-01-15T18:03:27.343468+0000","last_fullsized":"2024-01-15T18:03:27.343468+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:43.710985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302611+0000","last_change":"2024-01-15T18:03:27.325587+0000","last_active":"2024-01-15T18:03:28.302611+0000","last_peered":"2024-01-15T18:03:28.302611+0000","last_clean":"2024-01-15T18:03:28.302611+0000","last_became_active":"2024-01-15T18:03:27.325317+0000","last_became_peered":"2024-01-15T18:03:27.325317+0000","last_unstale":"2024-01-15T18:03:28.302611+0000","last_undegraded":"2024-01-15T18:03:28.302611+0000","last_fullsized":"2024-01-15T18:03:28.302611+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:18:27.777253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299247+0000","last_change":"2024-01-15T18:03:26.295610+0000","last_active":"2024-01-15T18:03:26.299247+0000","last_peered":"2024-01-15T18:03:26.299247+0000","last_clean":"2024-01-15T18:03:26.299247+0000","last_became_active":"2024-01-15T18:03:26.295164+0000","last_became_peered":"2024-01-15T18:03:26.295164+0000","last_unstale":"2024-01-15T18:03:26.299247+0000","last_undegraded":"2024-01-15T18:03:26.299247+0000","last_fullsized":"2024-01-15T18:03:26.299247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:42.703538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279636+0000","last_change":"2024-01-15T18:03:26.279636+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279636+0000","last_undegraded":"2024-01-15T18:03:26.279636+0000","last_fullsized":"2024-01-15T18:03:26.279636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.316666+0000","last_change":"2024-01-15T18:03:26.301741+0000","last_active":"2024-01-15T18:03:27.316666+0000","last_peered":"2024-01-15T18:03:27.316666+0000","last_clean":"2024-01-15T18:03:27.316666+0000","last_became_active":"2024-01-15T18:03:26.301228+0000","last_became_peered":"2024-01-15T18:03:26.301228+0000","last_unstale":"2024-01-15T18:03:27.316666+0000","last_undegraded":"2024-01-15T18:03:27.316666+0000","last_fullsized":"2024-01-15T18:03:27.316666+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:08.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278650+0000","last_change":"2024-01-15T18:03:26.278650+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278650+0000","last_undegraded":"2024-01-15T18:03:26.278650+0000","last_fullsized":"2024-01-15T18:03:26.278650+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294795+0000","last_change":"2024-01-15T18:03:26.293020+0000","last_active":"2024-01-15T18:03:26.294795+0000","last_peered":"2024-01-15T18:03:26.294795+0000","last_clean":"2024-01-15T18:03:26.294795+0000","last_became_active":"2024-01-15T18:03:26.292663+0000","last_became_peered":"2024-01-15T18:03:26.292663+0000","last_unstale":"2024-01-15T18:03:26.294795+0000","last_undegraded":"2024-01-15T18:03:26.294795+0000","last_fullsized":"2024-01-15T18:03:26.294795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:51.697653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278555+0000","last_change":"2024-01-15T18:03:26.278555+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278555+0000","last_undegraded":"2024-01-15T18:03:26.278555+0000","last_fullsized":"2024-01-15T18:03:26.278555+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295435+0000","last_change":"2024-01-15T18:03:26.292741+0000","last_active":"2024-01-15T18:03:26.295435+0000","last_peered":"2024-01-15T18:03:26.295435+0000","last_clean":"2024-01-15T18:03:26.295435+0000","last_became_active":"2024-01-15T18:03:26.292349+0000","last_became_peered":"2024-01-15T18:03:26.292349+0000","last_unstale":"2024-01-15T18:03:26.295435+0000","last_undegraded":"2024-01-15T18:03:26.295435+0000","last_fullsized":"2024-01-15T18:03:26.295435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:19:42.341080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282316+0000","last_change":"2024-01-15T18:03:26.282316+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282316+0000","last_undegraded":"2024-01-15T18:03:26.282316+0000","last_fullsized":"2024-01-15T18:03:26.282316+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341991+0000","last_change":"2024-01-15T18:03:26.299863+0000","last_active":"2024-01-15T18:03:27.341991+0000","last_peered":"2024-01-15T18:03:27.341991+0000","last_clean":"2024-01-15T18:03:27.341991+0000","last_became_active":"2024-01-15T18:03:26.299525+0000","last_became_peered":"2024-01-15T18:03:26.299525+0000","last_unstale":"2024-01-15T18:03:27.341991+0000","last_undegraded":"2024-01-15T18:03:27.341991+0000","last_fullsized":"2024-01-15T18:03:27.341991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:51.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284466+0000","last_change":"2024-01-15T18:03:26.284466+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284466+0000","last_undegraded":"2024-01-15T18:03:26.284466+0000","last_fullsized":"2024-01-15T18:03:26.284466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299404+0000","last_change":"2024-01-15T18:03:26.297682+0000","last_active":"2024-01-15T18:03:26.299404+0000","last_peered":"2024-01-15T18:03:26.299404+0000","last_clean":"2024-01-15T18:03:26.299404+0000","last_became_active":"2024-01-15T18:03:26.297319+0000","last_became_peered":"2024-01-15T18:03:26.297319+0000","last_unstale":"2024-01-15T18:03:26.299404+0000","last_undegraded":"2024-01-15T18:03:26.299404+0000","last_fullsized":"2024-01-15T18:03:26.299404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:40:28.369164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278628+0000","last_change":"2024-01-15T18:03:26.278628+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278628+0000","last_undegraded":"2024-01-15T18:03:26.278628+0000","last_fullsized":"2024-01-15T18:03:26.278628+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.311963+0000","last_change":"2024-01-15T18:03:26.290405+0000","last_active":"2024-01-15T18:03:27.311963+0000","last_peered":"2024-01-15T18:03:27.311963+0000","last_clean":"2024-01-15T18:03:27.311963+0000","last_became_active":"2024-01-15T18:03:26.288657+0000","last_became_peered":"2024-01-15T18:03:26.288657+0000","last_unstale":"2024-01-15T18:03:27.311963+0000","last_undegraded":"2024-01-15T18:03:27.311963+0000","last_fullsized":"2024-01-15T18:03:27.311963+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:56.868408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281044+0000","last_change":"2024-01-15T18:03:26.281044+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281044+0000","last_undegraded":"2024-01-15T18:03:26.281044+0000","last_fullsized":"2024-01-15T18:03:26.281044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297486+0000","last_change":"2024-01-15T18:03:26.296161+0000","last_active":"2024-01-15T18:03:26.297486+0000","last_peered":"2024-01-15T18:03:26.297486+0000","last_clean":"2024-01-15T18:03:26.297486+0000","last_became_active":"2024-01-15T18:03:26.295339+0000","last_became_peered":"2024-01-15T18:03:26.295339+0000","last_unstale":"2024-01-15T18:03:26.297486+0000","last_undegraded":"2024-01-15T18:03:26.297486+0000","last_fullsized":"2024-01-15T18:03:26.297486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:20:43.839295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302217+0000","last_change":"2024-01-15T18:03:27.330393+0000","last_active":"2024-01-15T18:03:28.302217+0000","last_peered":"2024-01-15T18:03:28.302217+0000","last_clean":"2024-01-15T18:03:28.302217+0000","last_became_active":"2024-01-15T18:03:27.329266+0000","last_became_peered":"2024-01-15T18:03:27.329266+0000","last_unstale":"2024-01-15T18:03:28.302217+0000","last_undegraded":"2024-01-15T18:03:28.302217+0000","last_fullsized":"2024-01-15T18:03:28.302217+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:50.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.311753+0000","last_change":"2024-01-15T18:03:26.296244+0000","last_active":"2024-01-15T18:03:27.311753+0000","last_peered":"2024-01-15T18:03:27.311753+0000","last_clean":"2024-01-15T18:03:27.311753+0000","last_became_active":"2024-01-15T18:03:26.295955+0000","last_became_peered":"2024-01-15T18:03:26.295955+0000","last_unstale":"2024-01-15T18:03:27.311753+0000","last_undegraded":"2024-01-15T18:03:27.311753+0000","last_fullsized":"2024-01-15T18:03:27.311753+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:44:34.308154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.331003+0000","last_change":"2024-01-15T18:03:27.323628+0000","last_active":"2024-01-15T18:03:27.331003+0000","last_peered":"2024-01-15T18:03:27.331003+0000","last_clean":"2024-01-15T18:03:27.331003+0000","last_became_active":"2024-01-15T18:03:27.323353+0000","last_became_peered":"2024-01-15T18:03:27.323353+0000","last_unstale":"2024-01-15T18:03:27.331003+0000","last_undegraded":"2024-01-15T18:03:27.331003+0000","last_fullsized":"2024-01-15T18:03:27.331003+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:12.045333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297615+0000","last_change":"2024-01-15T18:03:26.295494+0000","last_active":"2024-01-15T18:03:26.297615+0000","last_peered":"2024-01-15T18:03:26.297615+0000","last_clean":"2024-01-15T18:03:26.297615+0000","last_became_active":"2024-01-15T18:03:26.295221+0000","last_became_peered":"2024-01-15T18:03:26.295221+0000","last_unstale":"2024-01-15T18:03:26.297615+0000","last_undegraded":"2024-01-15T18:03:26.297615+0000","last_fullsized":"2024-01-15T18:03:26.297615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:33.869808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.285274+0000","last_change":"2024-01-15T18:03:26.285274+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.285274+0000","last_undegraded":"2024-01-15T18:03:26.285274+0000","last_fullsized":"2024-01-15T18:03:26.285274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294031+0000","last_change":"2024-01-15T18:03:26.291361+0000","last_active":"2024-01-15T18:03:26.294031+0000","last_peered":"2024-01-15T18:03:26.294031+0000","last_clean":"2024-01-15T18:03:26.294031+0000","last_became_active":"2024-01-15T18:03:26.290863+0000","last_became_peered":"2024-01-15T18:03:26.290863+0000","last_unstale":"2024-01-15T18:03:26.294031+0000","last_undegraded":"2024-01-15T18:03:26.294031+0000","last_fullsized":"2024-01-15T18:03:26.294031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:24:07.021223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.332585+0000","last_change":"2024-01-15T18:03:27.325192+0000","last_active":"2024-01-15T18:03:27.332585+0000","last_peered":"2024-01-15T18:03:27.332585+0000","last_clean":"2024-01-15T18:03:27.332585+0000","last_became_active":"2024-01-15T18:03:27.324697+0000","last_became_peered":"2024-01-15T18:03:27.324697+0000","last_unstale":"2024-01-15T18:03:27.332585+0000","last_undegraded":"2024-01-15T18:03:27.332585+0000","last_fullsized":"2024-01-15T18:03:27.332585+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:20.920788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.308092+0000","last_change":"2024-01-15T18:03:26.298754+0000","last_active":"2024-01-15T18:03:27.308092+0000","last_peered":"2024-01-15T18:03:27.308092+0000","last_clean":"2024-01-15T18:03:27.308092+0000","last_became_active":"2024-01-15T18:03:26.298412+0000","last_became_peered":"2024-01-15T18:03:26.298412+0000","last_unstale":"2024-01-15T18:03:27.308092+0000","last_undegraded":"2024-01-15T18:03:27.308092+0000","last_fullsized":"2024-01-15T18:03:27.308092+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:43:09.251315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278407+0000","last_change":"2024-01-15T18:03:26.278407+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278407+0000","last_undegraded":"2024-01-15T18:03:26.278407+0000","last_fullsized":"2024-01-15T18:03:26.278407+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342463+0000","last_change":"2024-01-15T18:03:26.300437+0000","last_active":"2024-01-15T18:03:27.342463+0000","last_peered":"2024-01-15T18:03:27.342463+0000","last_clean":"2024-01-15T18:03:27.342463+0000","last_became_active":"2024-01-15T18:03:26.300210+0000","last_became_peered":"2024-01-15T18:03:26.300210+0000","last_unstale":"2024-01-15T18:03:27.342463+0000","last_undegraded":"2024-01-15T18:03:27.342463+0000","last_fullsized":"2024-01-15T18:03:27.342463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:47.498523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":206,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-01-15T18:03:25.275709+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:03:25.275709+0000","last_peered":"2024-01-15T18:03:25.275709+0000","last_clean":"2024-01-15T18:03:25.275709+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:03:25.275709+0000","last_undegraded":"2024-01-15T18:03:25.275709+0000","last_fullsized":"2024-01-15T18:03:25.275709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1441792,"data_stored":947133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.04,"5min":1.6439999999999999,"15min":1.3180000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":19.843,"5min":19.843,"15min":19.843},"last":0.878},{"interface":"front","average":{"1min":4.2229999999999999,"5min":2.3180000000000001,"15min":1.552},"min":{"1min":0.75700000000000001,"5min":0.629,"15min":0.623},"max":{"1min":30.498000000000001,"5min":38.137,"15min":38.137},"last":1.0149999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.129,"5min":1.716,"15min":1.383},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":19.917000000000002,"5min":19.917000000000002,"15min":19.917000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":2.2839999999999998,"5min":2.109,"15min":1.546},"min":{"1min":0.71199999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":10.361000000000001,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.143}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":1.6910000000000001,"15min":1.345},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":8.1289999999999996,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":0.94899999999999995},{"interface":"front","average":{"1min":3.0760000000000001,"5min":1.8149999999999999,"15min":1.5169999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":13.478999999999999,"5min":13.478999999999999,"15min":13.478999999999999},"last":1.202}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.3359999999999999,"5min":1.9450000000000001,"15min":1.5129999999999999},"min":{"1min":0.56000000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":16.632000000000001,"5min":16.632000000000001,"15min":16.632000000000001},"last":1.284},{"interface":"front","average":{"1min":3.177,"5min":1.867,"15min":1.617},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":13.439,"5min":13.439,"15min":13.439},"last":1.4279999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":2.145,"15min":1.694},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50700000000000001},"max":{"1min":10.881,"5min":22.359999999999999,"15min":22.359999999999999},"last":1.254},{"interface":"front","average":{"1min":4.4820000000000002,"5min":2.2149999999999999,"15min":1.702},"min":{"1min":0.52700000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":38.682000000000002,"5min":38.682000000000002,"15min":38.682000000000002},"last":1.633}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.8140000000000001,"5min":2.286,"15min":1.831},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":13.379,"5min":25.006,"15min":25.006},"last":1.4950000000000001},{"interface":"front","average":{"1min":2.9169999999999998,"5min":2.2669999999999999,"15min":1.8069999999999999},"min":{"1min":0.80200000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":15.664999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.661}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.4729999999999999,"5min":2.5,"15min":1.954},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.59299999999999997},"max":{"1min":11.366,"5min":23.834,"15min":23.834},"last":1.794},{"interface":"front","average":{"1min":2.8380000000000001,"5min":2.8050000000000002,"15min":2.012},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":9.2599999999999998,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.7310000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45900,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325940,"statfs":{"total":96636764160,"available":96589762560,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.325,"15min":1.3340000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":7.79,"5min":7.79,"15min":12.544},"last":0.88500000000000001},{"interface":"front","average":{"1min":2.1379999999999999,"5min":1.579,"15min":1.3149999999999999},"min":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":14.013,"5min":14.013,"15min":14.013},"last":1.5009999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.403,"15min":1.284},"min":{"1min":0.67400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":7.6200000000000001,"5min":7.6200000000000001,"15min":12.811999999999999},"last":1.107},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.8999999999999999,"15min":1.4199999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":20.523,"5min":20.523,"15min":20.523},"last":1.4199999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":1.877,"15min":1.4279999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":17.713000000000001,"5min":17.713000000000001,"15min":17.713000000000001},"last":1.871},{"interface":"front","average":{"1min":1.788,"5min":1.3799999999999999,"15min":1.4850000000000001},"min":{"1min":0.59899999999999998,"5min":0.504,"15min":0.504},"max":{"1min":7.702,"5min":7.702,"15min":12.897},"last":1.006}]},{"osd":3,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0840000000000001,"5min":1.74,"15min":1.6000000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.7480000000000002,"5min":9.5,"15min":12.717000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":2.3410000000000002,"5min":1.825,"15min":1.518},"min":{"1min":0.876,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":13.795,"5min":14.066000000000001,"15min":14.066000000000001},"last":1.607}]},{"osd":4,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":4.0739999999999998,"5min":2.0659999999999998,"15min":1.6200000000000001},"min":{"1min":0.59599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.5760000000000001},{"interface":"front","average":{"1min":2.2650000000000001,"5min":1.774,"15min":1.6890000000000001},"min":{"1min":0.52000000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":7.8319999999999999,"5min":17.289000000000001,"15min":17.289000000000001},"last":1.248}]},{"osd":5,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.9529999999999998,"5min":1.819,"15min":1.5620000000000001},"min":{"1min":0.66900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":13.593,"5min":14.176,"15min":14.176},"last":1.8440000000000001},{"interface":"front","average":{"1min":3.4950000000000001,"5min":2.1110000000000002,"15min":1.849},"min":{"1min":0.621,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":17.949000000000002,"5min":17.949000000000002,"15min":17.949000000000002},"last":1.752}]},{"osd":7,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.4359999999999999,"5min":2.0190000000000001,"15min":1.708},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":21.265000000000001,"5min":21.265000000000001,"15min":21.265000000000001},"last":1.71},{"interface":"front","average":{"1min":4.3949999999999996,"5min":2.3439999999999999,"15min":1.9299999999999999},"min":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":19.571000000000002,"5min":19.571000000000002,"15min":19.571000000000002},"last":1.665}]}]},{"osd":5,"up_from":14,"seq":60129542248,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45900,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325940,"statfs":{"total":96636764160,"available":96589762560,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9890000000000001,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":19.266999999999999,"5min":19.266999999999999,"15min":19.266999999999999},"last":1.236},{"interface":"front","average":{"1min":1.292,"5min":1.607,"15min":1.4330000000000001},"min":{"1min":0.69399999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.1760000000000002,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.55}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":1.5469999999999999,"15min":1.3169999999999999},"min":{"1min":0.71299999999999997,"5min":0.499,"15min":0.499},"max":{"1min":9.1259999999999994,"5min":16.297000000000001,"15min":16.297000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.8899999999999999,"5min":1.3859999999999999,"15min":1.2629999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2379999999999995,"5min":9.2379999999999995,"15min":9.2379999999999995},"last":1.0569999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.345,"15min":1.262},"min":{"1min":0.88600000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.0369999999999999,"5min":8.1820000000000004,"15min":8.1820000000000004},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.8220000000000001,"5min":1.6479999999999999,"15min":1.3899999999999999},"min":{"1min":0.88500000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2159999999999993,"5min":16.158999999999999,"15min":16.158999999999999},"last":0.996}]},{"osd":3,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.5549999999999999,"15min":1.4370000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.56100000000000005},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":14.914},"last":1.341},{"interface":"front","average":{"1min":1.9930000000000001,"5min":1.526,"15min":1.4570000000000001},"min":{"1min":0.76300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":7.0110000000000001,"5min":7.0110000000000001,"15min":7.0110000000000001},"last":1.3879999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9850000000000001,"5min":1.6579999999999999,"15min":1.4870000000000001},"min":{"1min":0.76100000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":9.0410000000000004,"5min":16.414999999999999,"15min":16.414999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.925,"15min":1.627},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":9.1519999999999992,"5min":17.991,"15min":17.991},"last":1.51}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.492,"15min":1.573},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.669,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":2.0609999999999999,"5min":1.788,"15min":1.623},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":9.1649999999999991,"5min":16.364000000000001,"15min":16.364000000000001},"last":1.468}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9370000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"min":{"1min":0.66300000000000003,"5min":0.628,"15min":0.628},"max":{"1min":9.0600000000000005,"5min":9.0600000000000005,"15min":9.0600000000000005},"last":1.423},{"interface":"front","average":{"1min":3.0600000000000001,"5min":1.891,"15min":1.79},"min":{"1min":0.83999999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":19.356999999999999,"5min":19.356999999999999,"15min":19.356999999999999},"last":1.669}]}]},{"osd":4,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50060,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321780,"statfs":{"total":96636764160,"available":96585502720,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.5229999999999999,"15min":1.3149999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":7.8869999999999996,"5min":11.413,"15min":11.413},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.8180000000000001,"5min":1.534,"15min":1.401},"min":{"1min":0.63,"5min":0.63,"15min":0.53700000000000003},"max":{"1min":7.9160000000000004,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":0.84899999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.367,"15min":1.369},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.45000000000000001},"max":{"1min":3.5750000000000002,"5min":9.7010000000000005,"15min":11.987},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.444,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.6109999999999998,"5min":16.058,"15min":16.058},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.7789999999999999,"15min":1.575},"min":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.6890000000000001,"5min":13.865,"15min":13.865},"last":1.052},{"interface":"front","average":{"1min":1.788,"5min":1.833,"15min":1.5009999999999999},"min":{"1min":0.89100000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":5.7050000000000001,"5min":13.903,"15min":13.903},"last":1.3879999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.919,"5min":1.9099999999999999,"15min":1.675},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":11.773,"5min":26.055,"15min":26.055},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.673,"15min":1.611},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":4.0419999999999998,"5min":13.25,"15min":13.25},"last":1.004}]},{"osd":5,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":2.9729999999999999,"5min":1.853,"15min":1.72},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":29.422000000000001,"5min":29.422000000000001,"15min":29.422000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.5720000000000001,"15min":1.619},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.7160000000000002,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.216}]},{"osd":6,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.6439999999999999,"15min":1.6379999999999999},"min":{"1min":0.80100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":5.7919999999999998,"5min":18.260000000000002,"15min":18.260000000000002},"last":1.579},{"interface":"front","average":{"1min":1.597,"5min":1.6890000000000001,"15min":1.7849999999999999},"min":{"1min":0.59999999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.6459999999999999,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.7410000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.982,"5min":1.881,"15min":1.889},"min":{"1min":0.60499999999999998,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":10.664,"5min":10.664,"15min":11.894},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.851,"5min":2.0379999999999998,"15min":1.9390000000000001},"min":{"1min":0.61399999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.3319999999999999,"5min":11.714,"15min":11.714},"last":1.6679999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58376,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313464,"statfs":{"total":96636764160,"available":96576987136,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5390000000000001,"5min":1.5580000000000001,"15min":1.1830000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":18.399999999999999,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":2.7730000000000001,"5min":1.8500000000000001,"15min":1.349},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.795999999999999,"5min":21.047000000000001,"15min":21.047000000000001},"last":0.91000000000000003}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5499999999999998,"5min":1.736,"15min":1.304},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":12.372,"5min":17.777999999999999,"15min":17.777999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.367,"15min":1.179},"min":{"1min":0.498,"5min":0.48099999999999998,"15min":0.39500000000000002},"max":{"1min":11.002000000000001,"5min":11.002000000000001,"15min":11.002000000000001},"last":1.081}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.413,"15min":1.244},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":1.0309999999999999},{"interface":"front","average":{"1min":2.2599999999999998,"5min":1.72,"15min":1.488},"min":{"1min":0.63300000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":20.495000000000001,"5min":20.495000000000001,"15min":20.495000000000001},"last":0.98899999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4380000000000002,"5min":1.7310000000000001,"15min":1.4950000000000001},"min":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.63500000000000001},"max":{"1min":11.826000000000001,"5min":11.826000000000001,"15min":11.826000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":3.1379999999999999,"5min":1.9950000000000001,"15min":1.625},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.54600000000000004},"max":{"1min":12.439,"5min":12.439,"15min":12.439},"last":1.3500000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.867,"15min":1.6379999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.57299999999999995},"max":{"1min":20.606000000000002,"5min":20.606000000000002,"15min":20.606000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.161,"5min":1.8120000000000001,"15min":1.6870000000000001},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.68000000000000005},"max":{"1min":11.603,"5min":11.603,"15min":11.603},"last":1.9019999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.0129999999999999,"5min":2.1059999999999999,"15min":1.823},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":12.416,"5min":12.416,"15min":12.416},"last":1.8540000000000001},{"interface":"front","average":{"1min":2.391,"5min":1.962,"15min":1.768},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":20.556000000000001,"5min":20.556000000000001,"15min":20.556000000000001},"last":1.8280000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.8460000000000001,"5min":2.331,"15min":1.8999999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.70699999999999996},"max":{"1min":28.843,"5min":28.843,"15min":28.843},"last":2.0019999999999998},{"interface":"front","average":{"1min":3.4940000000000002,"5min":2.3199999999999998,"15min":1.9219999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":15.143000000000001,"5min":15.143000000000001,"15min":15.143000000000001},"last":1.8029999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542251,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":884736,"data_stored":371564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.3889999999999998,"5min":8.3279999999999994,"15min":14.427},"last":0.748},{"interface":"front","average":{"1min":1.893,"5min":1.3979999999999999,"15min":1.1299999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.432},"max":{"1min":11.243,"5min":11.243,"15min":11.243},"last":1.46}]},{"osd":1,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.4139999999999999,"15min":1.357},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40300000000000002},"max":{"1min":4.2779999999999996,"5min":11.718,"15min":14.609},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.278,"15min":1.224},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":4.3449999999999998,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":1.0629999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6749999999999998,"5min":1.7050000000000001,"15min":1.3859999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.34799999999999998},"max":{"1min":12.93,"5min":19.535,"15min":19.535},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.657,"5min":1.589,"15min":1.518},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.2830000000000004,"5min":13.406000000000001,"15min":14.528},"last":0.999}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.6320000000000001,"5min":2.0880000000000001,"15min":1.8069999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":29.986999999999998,"5min":29.986999999999998,"15min":29.986999999999998},"last":1.347},{"interface":"front","average":{"1min":2.9140000000000001,"5min":1.9530000000000001,"15min":1.619},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":21.199999999999999,"5min":21.199999999999999,"15min":21.199999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.1960000000000002,"5min":1.8200000000000001,"15min":1.6220000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":12.227,"5min":19.681999999999999,"15min":19.681999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.7410000000000001,"15min":1.698},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":4.0519999999999996,"5min":10.108000000000001,"15min":14.702999999999999},"last":1.6200000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.726,"15min":1.5629999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.5839999999999996,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":2.0640000000000001,"5min":1.8720000000000001,"15min":1.631},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":12.176,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.6599999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":5.6159999999999997,"5min":2.5179999999999998,"15min":1.907},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":57.006999999999998,"5min":57.006999999999998,"15min":57.006999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":3.8799999999999999,"5min":2.5950000000000002,"15min":1.9690000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":21.795999999999999,"5min":21.795999999999999,"15min":21.795999999999999},"last":1.796}]}]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54220,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317620,"statfs":{"total":96636764160,"available":96581242880,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.27,"15min":1.0209999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42799999999999999},"max":{"1min":13.749000000000001,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.751},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.157,"15min":1.0449999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":18.658000000000001,"5min":18.658000000000001,"15min":18.658000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.331,"15min":1.0329999999999999},"min":{"1min":0.48599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.3029999999999999,"5min":21.126000000000001,"15min":21.126000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.585,"15min":1.3109999999999999},"min":{"1min":0.52700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":9.7279999999999998,"5min":11.5,"15min":11.5},"last":1.0389999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7350000000000001,"5min":1.4279999999999999,"15min":1.855},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":13.593,"5min":13.593,"15min":22.106999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.2250000000000001,"15min":1.26},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.9610000000000003,"5min":7.9610000000000003,"15min":15.510999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.708,"15min":1.726},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72499999999999998},"max":{"1min":3.27,"5min":14.827,"15min":14.827},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.6839999999999999,"15min":1.8109999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3809999999999998,"5min":14.851000000000001,"15min":14.851000000000001},"last":1.4299999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":1.643,"15min":1.7829999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.0700000000000003,"5min":11.552,"15min":13.228999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":3.1400000000000001,"5min":1.929,"15min":1.8089999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.81399999999999995},"max":{"1min":18.452000000000002,"5min":18.452000000000002,"15min":20.823},"last":0.83699999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.0910000000000002,"5min":1.9890000000000001,"15min":2.0139999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":34.158000000000001,"5min":34.158000000000001,"15min":34.158000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.6200000000000001,"15min":1.534},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.52,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.1950000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6859999999999999,"5min":1.865,"15min":1.6639999999999999},"min":{"1min":0.84599999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":17.928999999999998,"5min":17.928999999999998,"15min":17.928999999999998},"last":1.109},{"interface":"front","average":{"1min":1.673,"5min":1.9470000000000001,"15min":1.6839999999999999},"min":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.9139999999999997,"5min":24.289999999999999,"15min":24.289999999999999},"last":1.7150000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542250,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54796,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317044,"statfs":{"total":96636764160,"available":96580653056,"internally_reserved":0,"allocated":1441792,"data_stored":947133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.0649999999999999,"15min":0.94199999999999995},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":6.1890000000000001,"5min":6.1890000000000001,"15min":6.1890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.806,"5min":1.413,"15min":1.1899999999999999},"min":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":11.470000000000001,"5min":11.470000000000001,"15min":11.470000000000001},"last":1.3959999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.333,"15min":1.117},"min":{"1min":0.45500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":11.59,"5min":11.59,"15min":11.59},"last":1.323},{"interface":"front","average":{"1min":2.7040000000000002,"5min":1.377,"15min":1.0309999999999999},"min":{"1min":0.41399999999999998,"5min":0.38200000000000001,"15min":0.35899999999999999},"max":{"1min":14.512,"5min":14.512,"15min":14.512},"last":1.2110000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5920000000000001,"15min":1.3720000000000001},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":11.544,"5min":11.544,"15min":11.544},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.446,"15min":1.194},"min":{"1min":0.41999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.1539999999999999,"5min":14.74,"15min":14.74},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.452,"5min":2.0379999999999998,"15min":1.532},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":32.115000000000002,"5min":32.115000000000002,"15min":32.115000000000002},"last":1.536},{"interface":"front","average":{"1min":2.2280000000000002,"5min":1.903,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":11.992000000000001,"5min":15.975,"15min":15.975},"last":1.4570000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.4870000000000001,"5min":2.1560000000000001,"15min":1.629},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":32.174999999999997,"5min":32.174999999999997,"15min":32.174999999999997},"last":1.369},{"interface":"front","average":{"1min":2.871,"5min":1.915,"15min":1.623},"min":{"1min":0.875,"5min":0.875,"15min":0.754},"max":{"1min":17.225000000000001,"5min":17.225000000000001,"15min":17.225000000000001},"last":1.4830000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.097,"5min":1.746,"15min":1.587},"min":{"1min":0.99199999999999999,"5min":0.96999999999999997,"15min":0.78000000000000003},"max":{"1min":6.1189999999999998,"5min":6.1189999999999998,"15min":6.1189999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":4.3369999999999997,"5min":2.1120000000000001,"15min":1.669},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":32.216999999999999,"5min":32.216999999999999,"15min":32.216999999999999},"last":1.609}]},{"osd":7,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5350000000000001,"5min":2.5710000000000002,"15min":1.9550000000000001},"min":{"1min":1.036,"5min":0.98899999999999999,"15min":0.81299999999999994},"max":{"1min":34.131,"5min":34.131,"15min":34.131},"last":1.508},{"interface":"front","average":{"1min":4.6580000000000004,"5min":2.3799999999999999,"15min":1.8320000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.70999999999999996},"max":{"1min":21.884,"5min":21.884,"15min":21.884},"last":1.266}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:03:32.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2024-01-15T18:03:32.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.281242+0000', 'last_change': '2024-01-15T18:03:26.281242+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.281242+0000', 'last_undegraded': '2024-01-15T18:03:26.281242+0000', 'last_fullsized': '2024-01-15T18:03:26.281242+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.892 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2024-01-15T18:03:32.892 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.283811+0000', 'last_change': '2024-01-15T18:03:26.283811+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.283811+0000', 'last_undegraded': '2024-01-15T18:03:26.283811+0000', 'last_fullsized': '2024-01-15T18:03:26.283811+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:03:32.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2024-01-15T18:03:32.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.288618+0000', 'last_change': '2024-01-15T18:03:26.288618+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.288618+0000', 'last_undegraded': '2024-01-15T18:03:26.288618+0000', 'last_fullsized': '2024-01-15T18:03:26.288618+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:03:32.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2024-01-15T18:03:32.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.284505+0000', 'last_change': '2024-01-15T18:03:26.284505+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.284505+0000', 'last_undegraded': '2024-01-15T18:03:26.284505+0000', 'last_fullsized': '2024-01-15T18:03:26.284505+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:03:32.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2024-01-15T18:03:32.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.275264+0000', 'last_change': '2024-01-15T18:03:26.275264+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.275264+0000', 'last_undegraded': '2024-01-15T18:03:26.275264+0000', 'last_fullsized': '2024-01-15T18:03:26.275264+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:03:32.893 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2024-01-15T18:03:32.893 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.282234+0000', 'last_change': '2024-01-15T18:03:26.282234+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.282234+0000', 'last_undegraded': '2024-01-15T18:03:26.282234+0000', 'last_fullsized': '2024-01-15T18:03:26.282234+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2024-01-15T18:03:32.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.288002+0000', 'last_change': '2024-01-15T18:03:26.288002+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.288002+0000', 'last_undegraded': '2024-01-15T18:03:26.288002+0000', 'last_fullsized': '2024-01-15T18:03:26.288002+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:03:32.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2024-01-15T18:03:32.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.282326+0000', 'last_change': '2024-01-15T18:03:26.282326+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.282326+0000', 'last_undegraded': '2024-01-15T18:03:26.282326+0000', 'last_fullsized': '2024-01-15T18:03:26.282326+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:03:32.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2024-01-15T18:03:32.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.284462+0000', 'last_change': '2024-01-15T18:03:26.284462+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.284462+0000', 'last_undegraded': '2024-01-15T18:03:26.284462+0000', 'last_fullsized': '2024-01-15T18:03:26.284462+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:03:32.894 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2024-01-15T18:03:32.894 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.275307+0000', 'last_change': '2024-01-15T18:03:26.275307+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.275307+0000', 'last_undegraded': '2024-01-15T18:03:26.275307+0000', 'last_fullsized': '2024-01-15T18:03:26.275307+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:03:32.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2024-01-15T18:03:32.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.286446+0000', 'last_change': '2024-01-15T18:03:26.286446+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.286446+0000', 'last_undegraded': '2024-01-15T18:03:26.286446+0000', 'last_fullsized': '2024-01-15T18:03:26.286446+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:03:32.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2024-01-15T18:03:32.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.282114+0000', 'last_change': '2024-01-15T18:03:26.282114+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.282114+0000', 'last_undegraded': '2024-01-15T18:03:26.282114+0000', 'last_fullsized': '2024-01-15T18:03:26.282114+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:03:32.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2024-01-15T18:03:32.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.276325+0000', 'last_change': '2024-01-15T18:03:26.276325+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.276325+0000', 'last_undegraded': '2024-01-15T18:03:26.276325+0000', 'last_fullsized': '2024-01-15T18:03:26.276325+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:03:32.895 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2024-01-15T18:03:32.895 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.274415+0000', 'last_change': '2024-01-15T18:03:26.274415+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.274415+0000', 'last_undegraded': '2024-01-15T18:03:26.274415+0000', 'last_fullsized': '2024-01-15T18:03:26.274415+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:03:32.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2024-01-15T18:03:32.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.279090+0000', 'last_change': '2024-01-15T18:03:26.279090+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.279090+0000', 'last_undegraded': '2024-01-15T18:03:26.279090+0000', 'last_fullsized': '2024-01-15T18:03:26.279090+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2024-01-15T18:03:32.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.279397+0000', 'last_change': '2024-01-15T18:03:26.279397+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.279397+0000', 'last_undegraded': '2024-01-15T18:03:26.279397+0000', 'last_fullsized': '2024-01-15T18:03:26.279397+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2024-01-15T18:03:32.896 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.276036+0000', 'last_change': '2024-01-15T18:03:26.276036+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.276036+0000', 'last_undegraded': '2024-01-15T18:03:26.276036+0000', 'last_fullsized': '2024-01-15T18:03:26.276036+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:03:32.896 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2024-01-15T18:03:32.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.284633+0000', 'last_change': '2024-01-15T18:03:26.284633+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.284633+0000', 'last_undegraded': '2024-01-15T18:03:26.284633+0000', 'last_fullsized': '2024-01-15T18:03:26.284633+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:03:32.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2024-01-15T18:03:32.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.283351+0000', 'last_change': '2024-01-15T18:03:26.283351+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.283351+0000', 'last_undegraded': '2024-01-15T18:03:26.283351+0000', 'last_fullsized': '2024-01-15T18:03:26.283351+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:03:32.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2024-01-15T18:03:32.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.274031+0000', 'last_change': '2024-01-15T18:03:26.274031+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.274031+0000', 'last_undegraded': '2024-01-15T18:03:26.274031+0000', 'last_fullsized': '2024-01-15T18:03:26.274031+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:03:32.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2024-01-15T18:03:32.897 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.282955+0000', 'last_change': '2024-01-15T18:03:26.282955+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.282955+0000', 'last_undegraded': '2024-01-15T18:03:26.282955+0000', 'last_fullsized': '2024-01-15T18:03:26.282955+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:03:32.897 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2024-01-15T18:03:32.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.283969+0000', 'last_change': '2024-01-15T18:03:26.283969+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.283969+0000', 'last_undegraded': '2024-01-15T18:03:26.283969+0000', 'last_fullsized': '2024-01-15T18:03:26.283969+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:03:32.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2024-01-15T18:03:32.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.283167+0000', 'last_change': '2024-01-15T18:03:26.283167+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.283167+0000', 'last_undegraded': '2024-01-15T18:03:26.283167+0000', 'last_fullsized': '2024-01-15T18:03:26.283167+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:03:32.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2024-01-15T18:03:32.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.279714+0000', 'last_change': '2024-01-15T18:03:26.279714+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.279714+0000', 'last_undegraded': '2024-01-15T18:03:26.279714+0000', 'last_fullsized': '2024-01-15T18:03:26.279714+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2024-01-15T18:03:32.898 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.281710+0000', 'last_change': '2024-01-15T18:03:26.281710+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.281710+0000', 'last_undegraded': '2024-01-15T18:03:26.281710+0000', 'last_fullsized': '2024-01-15T18:03:26.281710+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.898 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2024-01-15T18:03:32.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.283325+0000', 'last_change': '2024-01-15T18:03:26.283325+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.283325+0000', 'last_undegraded': '2024-01-15T18:03:26.283325+0000', 'last_fullsized': '2024-01-15T18:03:26.283325+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:03:32.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2024-01-15T18:03:32.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.274045+0000', 'last_change': '2024-01-15T18:03:26.274045+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.274045+0000', 'last_undegraded': '2024-01-15T18:03:26.274045+0000', 'last_fullsized': '2024-01-15T18:03:26.274045+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:03:32.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2024-01-15T18:03:32.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.281723+0000', 'last_change': '2024-01-15T18:03:26.281723+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.281723+0000', 'last_undegraded': '2024-01-15T18:03:26.281723+0000', 'last_fullsized': '2024-01-15T18:03:26.281723+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:03:32.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2024-01-15T18:03:32.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.283831+0000', 'last_change': '2024-01-15T18:03:26.283831+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.283831+0000', 'last_undegraded': '2024-01-15T18:03:26.283831+0000', 'last_fullsized': '2024-01-15T18:03:26.283831+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:03:32.899 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2024-01-15T18:03:32.899 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.280249+0000', 'last_change': '2024-01-15T18:03:26.280249+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.280249+0000', 'last_undegraded': '2024-01-15T18:03:26.280249+0000', 'last_fullsized': '2024-01-15T18:03:26.280249+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2024-01-15T18:03:32.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.278423+0000', 'last_change': '2024-01-15T18:03:26.278423+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.278423+0000', 'last_undegraded': '2024-01-15T18:03:26.278423+0000', 'last_fullsized': '2024-01-15T18:03:26.278423+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2024-01-15T18:03:32.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.286454+0000', 'last_change': '2024-01-15T18:03:26.286454+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.286454+0000', 'last_undegraded': '2024-01-15T18:03:26.286454+0000', 'last_fullsized': '2024-01-15T18:03:26.286454+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:03:32.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2024-01-15T18:03:32.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.281591+0000', 'last_change': '2024-01-15T18:03:26.281591+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.281591+0000', 'last_undegraded': '2024-01-15T18:03:26.281591+0000', 'last_fullsized': '2024-01-15T18:03:26.281591+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:03:32.900 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2024-01-15T18:03:32.900 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.274473+0000', 'last_change': '2024-01-15T18:03:26.274473+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.274473+0000', 'last_undegraded': '2024-01-15T18:03:26.274473+0000', 'last_fullsized': '2024-01-15T18:03:26.274473+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:03:32.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2024-01-15T18:03:32.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.279678+0000', 'last_change': '2024-01-15T18:03:26.279678+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.279678+0000', 'last_undegraded': '2024-01-15T18:03:26.279678+0000', 'last_fullsized': '2024-01-15T18:03:26.279678+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2024-01-15T18:03:32.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.284222+0000', 'last_change': '2024-01-15T18:03:26.284222+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.284222+0000', 'last_undegraded': '2024-01-15T18:03:26.284222+0000', 'last_fullsized': '2024-01-15T18:03:26.284222+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:03:32.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2024-01-15T18:03:32.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.279636+0000', 'last_change': '2024-01-15T18:03:26.279636+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.279636+0000', 'last_undegraded': '2024-01-15T18:03:26.279636+0000', 'last_fullsized': '2024-01-15T18:03:26.279636+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:03:32.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2024-01-15T18:03:32.901 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.278650+0000', 'last_change': '2024-01-15T18:03:26.278650+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.278650+0000', 'last_undegraded': '2024-01-15T18:03:26.278650+0000', 'last_fullsized': '2024-01-15T18:03:26.278650+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:03:32.901 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2024-01-15T18:03:32.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.278555+0000', 'last_change': '2024-01-15T18:03:26.278555+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.278555+0000', 'last_undegraded': '2024-01-15T18:03:26.278555+0000', 'last_fullsized': '2024-01-15T18:03:26.278555+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:03:32.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2024-01-15T18:03:32.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.282316+0000', 'last_change': '2024-01-15T18:03:26.282316+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.282316+0000', 'last_undegraded': '2024-01-15T18:03:26.282316+0000', 'last_fullsized': '2024-01-15T18:03:26.282316+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:03:32.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2024-01-15T18:03:32.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.284466+0000', 'last_change': '2024-01-15T18:03:26.284466+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.284466+0000', 'last_undegraded': '2024-01-15T18:03:26.284466+0000', 'last_fullsized': '2024-01-15T18:03:26.284466+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:03:32.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2024-01-15T18:03:32.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.278628+0000', 'last_change': '2024-01-15T18:03:26.278628+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.278628+0000', 'last_undegraded': '2024-01-15T18:03:26.278628+0000', 'last_fullsized': '2024-01-15T18:03:26.278628+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2024-01-15T18:03:32.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.281044+0000', 'last_change': '2024-01-15T18:03:26.281044+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.281044+0000', 'last_undegraded': '2024-01-15T18:03:26.281044+0000', 'last_fullsized': '2024-01-15T18:03:26.281044+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:03:32.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2024-01-15T18:03:32.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.285274+0000', 'last_change': '2024-01-15T18:03:26.285274+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.285274+0000', 'last_undegraded': '2024-01-15T18:03:26.285274+0000', 'last_fullsized': '2024-01-15T18:03:26.285274+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:03:32.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2024-01-15T18:03:32.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 70, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:03:26.278407+0000', 'last_change': '2024-01-15T18:03:26.278407+0000', 'last_active': '2024-01-15T18:03:26.259097+0000', 'last_peered': '2024-01-15T18:03:26.259097+0000', 'last_clean': '2024-01-15T18:03:26.259097+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:03:26.278407+0000', 'last_undegraded': '2024-01-15T18:03:26.278407+0000', 'last_fullsized': '2024-01-15T18:03:26.278407+0000', 'mapping_epoch': 70, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 70, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'last_clean_scrub_stamp': '2024-01-15T18:03:26.259097+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:03:32.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-15T18:03:33.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.146+0000 7f3fcb400640 1 Processor -- start 2024-01-15T18:03:33.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.147+0000 7f3fcb400640 1 -- start start 2024-01-15T18:03:33.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.148+0000 7f3fcb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fc4160890 0x7f3fc40d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.148+0000 7f3fcb400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3fc40c6be0 0x7f3fc40cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.148+0000 7f3fcb400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc40cf6a0 con 0x7f3fc4160c70 2024-01-15T18:03:33.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.148+0000 7f3fcb400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc40d1430 con 0x7f3fc40c6be0 2024-01-15T18:03:33.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.148+0000 7f3fcb400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40b4370 con 0x7f3fc4160890 2024-01-15T18:03:33.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.149+0000 7f3fc9600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3fc4160c70 0x7f3fc40d1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43394/0 (socket says 172.21.15.26:43394) 2024-01-15T18:03:33.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.149+0000 7f3fc9600640 1 -- 172.21.15.26:0/933259203 learned_addr learned my addr 172.21.15.26:0/933259203 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.149+0000 7f3fcaa00640 1 --2- 172.21.15.26:0/933259203 >> v2:172.21.15.124:3300/0 conn(0x7f3fc40c6be0 0x7f3fc40cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.149+0000 7f3fca000640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fc4160890 0x7f3fc40d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.149+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457986686 0 0) 0x7f3fc40cf6a0 con 0x7f3fc4160c70 2024-01-15T18:03:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.149+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fa4003520 con 0x7f3fc4160c70 2024-01-15T18:03:33.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1229967945 0 0) 0x7f3fa4003520 con 0x7f3fc4160c70 2024-01-15T18:03:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fc4160890 msgr2=0x7f3fc40d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fc4160890 0x7f3fc40d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 >> v2:172.21.15.124:3300/0 conn(0x7f3fc40c6be0 msgr2=0x7f3fc40cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 --2- 172.21.15.26:0/933259203 >> v2:172.21.15.124:3300/0 conn(0x7f3fc40c6be0 0x7f3fc40cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40ce7c0 con 0x7f3fc4160c70 2024-01-15T18:03:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f3fac001670 con 0x7f3fc4160c70 2024-01-15T18:03:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 >> v1:172.21.15.26:6789/0 conn(0x7f3fc4160c70 legacy=0x7f3fc40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa4003970 0x7f3fa4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fa40047c0 0x7f3fa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fc4160890 0x7f3fa4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40d1430 con 0x7f3fa4003970 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40b4370 con 0x7f3fa40047c0 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa400ba60 con 0x7f3fc4160890 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fca000640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa4003970 0x7f3fa4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.150+0000 7f3fc9600640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fa40047c0 0x7f3fa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.151+0000 7f3fca000640 1 -- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fc4160890 msgr2=0x7f3fa4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.151+0000 7f3fca000640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fc4160890 0x7f3fa4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.151+0000 7f3fca000640 1 -- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fa40047c0 msgr2=0x7f3fa4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:33.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.151+0000 7f3fca000640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fa40047c0 0x7f3fa4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.151+0000 7f3fca000640 1 -- 172.21.15.26:0/933259203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa4003520 con 0x7f3fa4003970 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.152+0000 7f3fca000640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa4003970 0x7f3fa4003d90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009a90 tx=0x7f3fa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feaccd3e8afe6895 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb402c6b0 con 0x7f3fa4003970 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb402c810 con 0x7f3fa4003970 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fc8c00640 1 -- 172.21.15.26:0/933259203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb402cba0 con 0x7f3fa4003970 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fcb400640 1 -- 172.21.15.26:0/933259203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa4003970 msgr2=0x7f3fa4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fcb400640 1 --2- 172.21.15.26:0/933259203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fa4003970 0x7f3fa4003d90 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009a90 tx=0x7f3fa40057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fcaa00640 1 -- 172.21.15.26:0/933259203 reap_dead start 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fcb400640 1 -- 172.21.15.26:0/933259203 shutdown_connections 2024-01-15T18:03:33.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fcb400640 1 -- 172.21.15.26:0/933259203 >> 172.21.15.26:0/933259203 conn(0x7f3fc40d3090 msgr2=0x7f3fc40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:33.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fcb400640 1 -- 172.21.15.26:0/933259203 shutdown_connections 2024-01-15T18:03:33.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.153+0000 7f3fcb400640 1 -- 172.21.15.26:0/933259203 wait complete. 2024-01-15T18:03:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.155+0000 7f3fcb400640 1 Processor -- start 2024-01-15T18:03:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.155+0000 7f3fcb400640 1 -- start start 2024-01-15T18:03:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.156+0000 7f3fcb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fa4003970 0x7f3fc4152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.156+0000 7f3fcb400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fc40b6fa0 0x7f3fc4155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.156+0000 7f3fcb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fc4155c40 0x7f3fc4153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.156+0000 7f3fcb400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40cb0b0 con 0x7f3fc4155c40 2024-01-15T18:03:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.156+0000 7f3fcb400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40cb390 con 0x7f3fc40b6fa0 2024-01-15T18:03:33.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.156+0000 7f3fcb400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40cb670 con 0x7f3fa4003970 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fcaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fc4155c40 0x7f3fc4153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fcaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fc4155c40 0x7f3fc4153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50570/0 (socket says 172.21.15.26:50570) 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fcaa00640 1 -- 172.21.15.26:0/4190654859 learned_addr learned my addr 172.21.15.26:0/4190654859 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fca000640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fa4003970 0x7f3fc4152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fc9600640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fc40b6fa0 0x7f3fc4155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fcaa00640 1 -- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fa4003970 msgr2=0x7f3fc4152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fcaa00640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fa4003970 0x7f3fc4152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fcaa00640 1 -- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fc40b6fa0 msgr2=0x7f3fc4155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fcaa00640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fc40b6fa0 0x7f3fc4155700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fcaa00640 1 -- 172.21.15.26:0/4190654859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40cb950 con 0x7f3fc4155c40 2024-01-15T18:03:33.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.157+0000 7f3fcaa00640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fc4155c40 0x7f3fc4153400 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40b9120 tx=0x7f3fbc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.158+0000 7f3fc2000640 1 -- 172.21.15.26:0/4190654859 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc010040 con 0x7f3fc4155c40 2024-01-15T18:03:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.158+0000 7f3fc2000640 1 -- 172.21.15.26:0/4190654859 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc00bce0 con 0x7f3fc4155c40 2024-01-15T18:03:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.158+0000 7f3fc2000640 1 -- 172.21.15.26:0/4190654859 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc014640 con 0x7f3fc4155c40 2024-01-15T18:03:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.158+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc41560d0 con 0x7f3fc4155c40 2024-01-15T18:03:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.158+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc40b7420 con 0x7f3fc4155c40 2024-01-15T18:03:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.159+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc40cb1f0 con 0x7f3fc4155c40 2024-01-15T18:03:33.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.160+0000 7f3fc2000640 1 -- 172.21.15.26:0/4190654859 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f3fbc0147a0 con 0x7f3fc4155c40 2024-01-15T18:03:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.161+0000 7f3fc2000640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3f9c06bbc0 0x7f3f9c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.161+0000 7f3fc2000640 1 -- 172.21.15.26:0/4190654859 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3fbc088a10 con 0x7f3fc4155c40 2024-01-15T18:03:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.162+0000 7f3fca000640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3f9c06bbc0 0x7f3f9c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.163+0000 7f3fca000640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3f9c06bbc0 0x7f3f9c06e080 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4036000 tx=0x7f3fb4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:33.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.165+0000 7f3fc2000640 1 -- 172.21.15.26:0/4190654859 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fc40cb1f0 con 0x7f3fc4155c40 2024-01-15T18:03:33.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.427+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3fc40cb1f0 con 0x7f3fc4155c40 2024-01-15T18:03:33.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.428+0000 7f3fc2000640 1 -- 172.21.15.26:0/4190654859 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1097 (secure 0 0 0) 0x7f3fc40cb1f0 con 0x7f3fc4155c40 2024-01-15T18:03:33.433 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:33.433 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":527,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":72,"num_osds":8,"num_up_osds":8,"osd_up_since":1705341296,"num_in_osds":8,"osd_in_since":1705341288,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":52},{"state_name":"creating+peering","count":45}],"num_pgs":97,"num_pools":3,"num_objects":7,"data_bytes":590460,"bytes_used":419831808,"bytes_avail":772674281472,"bytes_total":773094113280,"inactive_pgs_ratio":0.46391752362251282,"write_bytes_sec":341,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":44,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":7095}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-01-15T18:01:48.060374+0000","services":{}},"progress_events":{}} 2024-01-15T18:03:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.430+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3f9c06bbc0 msgr2=0x7f3f9c06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.430+0000 7f3fcb400640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3f9c06bbc0 0x7f3f9c06e080 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4036000 tx=0x7f3fb4030000 comp rx=0 tx=0).stop 2024-01-15T18:03:33.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fc4155c40 msgr2=0x7f3fc4153400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fc4155c40 0x7f3fc4153400 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40b9120 tx=0x7f3fbc00b480 comp rx=0 tx=0).stop 2024-01-15T18:03:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 shutdown_connections 2024-01-15T18:03:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3f9c06bbc0 0x7f3f9c06e080 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3fc4155c40 0x7f3fc4153400 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3fc40b6fa0 0x7f3fc4155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 --2- 172.21.15.26:0/4190654859 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3fa4003970 0x7f3fc4152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 >> 172.21.15.26:0/4190654859 conn(0x7f3fc40d3090 msgr2=0x7f3fc40c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 shutdown_connections 2024-01-15T18:03:33.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.431+0000 7f3fcb400640 1 -- 172.21.15.26:0/4190654859 wait complete. 2024-01-15T18:03:33.452 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-15T18:03:33.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:03:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.663+0000 7f9ca7e00640 1 Processor -- start 2024-01-15T18:03:33.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.663+0000 7f9ca7e00640 1 -- start start 2024-01-15T18:03:33.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.665+0000 7f9ca7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca8159590 0x7f9ca8070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.665+0000 7f9ca7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ca80b3c20 0x7f9ca80ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.665+0000 7f9ca7e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8071790 con 0x7f9ca80cd220 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.665+0000 7f9ca7e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca8071100 con 0x7f9ca80b3c20 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.665+0000 7f9ca7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca8070860 con 0x7f9ca8159590 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.665+0000 7f9ca6000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ca80b3c20 0x7f9ca80ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.665+0000 7f9ca6000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ca80b3c20 0x7f9ca80ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50572/0 (socket says 172.21.15.26:50572) 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.665+0000 7f9ca6000640 1 -- 172.21.15.26:0/4086646073 learned_addr learned my addr 172.21.15.26:0/4086646073 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.666+0000 7f9ca6a00640 1 --2- 172.21.15.26:0/4086646073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca8159590 0x7f9ca8070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.666+0000 7f9ca6000640 1 -- 172.21.15.26:0/4086646073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca8159590 msgr2=0x7f9ca8070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:33.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.666+0000 7f9ca6000640 1 --2- 172.21.15.26:0/4086646073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca8159590 0x7f9ca8070160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.666+0000 7f9ca6000640 1 -- 172.21.15.26:0/4086646073 >> v1:172.21.15.124:6789/0 conn(0x7f9ca80cd220 legacy=0x7f9ca806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:33.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.666+0000 7f9ca6000640 1 -- 172.21.15.26:0/4086646073 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8070b40 con 0x7f9ca80b3c20 2024-01-15T18:03:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.667+0000 7f9ca6000640 1 --2- 172.21.15.26:0/4086646073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 0x7f9ca80ccce0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000bb00 tx=0x7f9c9002de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af4fb24a754e500 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.667+0000 7f9ca5600640 1 -- 172.21.15.26:0/4086646073 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9002ebc0 con 0x7f9ca80b3c20 2024-01-15T18:03:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.667+0000 7f9ca5600640 1 -- 172.21.15.26:0/4086646073 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c90032070 con 0x7f9ca80b3c20 2024-01-15T18:03:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.667+0000 7f9ca5600640 1 -- 172.21.15.26:0/4086646073 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90038400 con 0x7f9ca80b3c20 2024-01-15T18:03:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.668+0000 7f9ca7e00640 1 -- 172.21.15.26:0/4086646073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 msgr2=0x7f9ca80ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.668+0000 7f9ca7e00640 1 --2- 172.21.15.26:0/4086646073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 0x7f9ca80ccce0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000bb00 tx=0x7f9c9002de10 comp rx=0 tx=0).stop 2024-01-15T18:03:33.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.668+0000 7f9ca7e00640 1 -- 172.21.15.26:0/4086646073 shutdown_connections 2024-01-15T18:03:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.668+0000 7f9ca7e00640 1 --2- 172.21.15.26:0/4086646073 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 0x7f9ca80ccce0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.668+0000 7f9ca7e00640 1 --2- 172.21.15.26:0/4086646073 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca8159590 0x7f9ca8070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.668+0000 7f9ca7e00640 1 -- 172.21.15.26:0/4086646073 >> 172.21.15.26:0/4086646073 conn(0x7f9ca806a800 msgr2=0x7f9ca806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.668+0000 7f9ca7e00640 1 -- 172.21.15.26:0/4086646073 shutdown_connections 2024-01-15T18:03:33.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.668+0000 7f9ca7e00640 1 -- 172.21.15.26:0/4086646073 wait complete. 2024-01-15T18:03:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.670+0000 7f9ca7e00640 1 Processor -- start 2024-01-15T18:03:33.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.671+0000 7f9ca7e00640 1 -- start start 2024-01-15T18:03:33.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.672+0000 7f9ca7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 0x7f9ca81540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.672+0000 7f9ca7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca80cd220 0x7f9ca81f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.672+0000 7f9ca7e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ca8159590 0x7f9ca814deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.672+0000 7f9ca7e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca80aca00 con 0x7f9ca80b3c20 2024-01-15T18:03:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.672+0000 7f9ca7e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca81548f0 con 0x7f9ca8159590 2024-01-15T18:03:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.672+0000 7f9ca7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca80acca0 con 0x7f9ca80cd220 2024-01-15T18:03:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.672+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 0x7f9ca81540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.672+0000 7f9ca6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 0x7f9ca81540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50580/0 (socket says 172.21.15.26:50580) 2024-01-15T18:03:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.672+0000 7f9ca6a00640 1 -- 172.21.15.26:0/2137172115 learned_addr learned my addr 172.21.15.26:0/2137172115 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9ca7400640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ca8159590 0x7f9ca814deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9ca6000640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca80cd220 0x7f9ca81f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9ca6a00640 1 -- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca80cd220 msgr2=0x7f9ca81f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9ca6a00640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca80cd220 0x7f9ca81f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9ca6a00640 1 -- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ca8159590 msgr2=0x7f9ca814deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9ca6a00640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ca8159590 0x7f9ca814deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9ca6a00640 1 -- 172.21.15.26:0/2137172115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9000b760 con 0x7f9ca80b3c20 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9ca6a00640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 0x7f9ca81540c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c7f0 tx=0x7f9c9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9c9f400640 1 -- 172.21.15.26:0/2137172115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98004220 con 0x7f9ca80b3c20 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9c9f400640 1 -- 172.21.15.26:0/2137172115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c980043a0 con 0x7f9ca80b3c20 2024-01-15T18:03:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9c9f400640 1 -- 172.21.15.26:0/2137172115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98007860 con 0x7f9ca80b3c20 2024-01-15T18:03:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.673+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca80acf80 con 0x7f9ca80b3c20 2024-01-15T18:03:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.674+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca81f6170 con 0x7f9ca80b3c20 2024-01-15T18:03:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.675+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca8067450 con 0x7f9ca80b3c20 2024-01-15T18:03:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.676+0000 7f9c9f400640 1 -- 172.21.15.26:0/2137172115 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f9c98007bd0 con 0x7f9ca80b3c20 2024-01-15T18:03:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.677+0000 7f9c9f400640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c8006bc10 0x7f9c8006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.677+0000 7f9c9f400640 1 -- 172.21.15.26:0/2137172115 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9c98088450 con 0x7f9ca80b3c20 2024-01-15T18:03:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.677+0000 7f9ca6000640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c8006bc10 0x7f9c8006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.679+0000 7f9ca6000640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c8006bc10 0x7f9c8006e0d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9c90039000 tx=0x7f9c90034000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.681+0000 7f9c9f400640 1 -- 172.21.15.26:0/2137172115 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c98054d10 con 0x7f9ca80b3c20 2024-01-15T18:03:33.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.862+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ca8159970 con 0x7f9c8006bc10 2024-01-15T18:03:33.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.885+0000 7f9c9f400640 1 -- 172.21.15.26:0/2137172115 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283196 (secure 0 0 0) 0x7f9c90006cd0 con 0x7f9c8006bc10 2024-01-15T18:03:33.895 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:03:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c8006bc10 msgr2=0x7f9c8006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c8006bc10 0x7f9c8006e0d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9c90039000 tx=0x7f9c90034000 comp rx=0 tx=0).stop 2024-01-15T18:03:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 msgr2=0x7f9ca81540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 0x7f9ca81540c0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c7f0 tx=0x7f9c9800ccb0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 shutdown_connections 2024-01-15T18:03:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ca80cd220 0x7f9ca81f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c8006bc10 0x7f9c8006e0d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ca8159590 0x7f9ca814deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 --2- 172.21.15.26:0/2137172115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ca80b3c20 0x7f9ca81540c0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 >> 172.21.15.26:0/2137172115 conn(0x7f9ca806a800 msgr2=0x7f9ca80cdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 shutdown_connections 2024-01-15T18:03:33.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:33.891+0000 7f9ca7e00640 1 -- 172.21.15.26:0/2137172115 wait complete. 2024-01-15T18:03:33.914 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-01-15T18:03:32.104844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590460,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":168,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126,"ondisk_log_size":126,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":47,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":409992,"kb_used_data":7820,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754564728,"statfs":{"total":773094113280,"available":772674281472,"internally_reserved":0,"allocated":8007680,"data_stored":4049655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001932"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281242+0000","last_change":"2024-01-15T18:03:26.281242+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281242+0000","last_undegraded":"2024-01-15T18:03:26.281242+0000","last_fullsized":"2024-01-15T18:03:26.281242+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342379+0000","last_change":"2024-01-15T18:03:27.341616+0000","last_active":"2024-01-15T18:03:27.342379+0000","last_peered":"2024-01-15T18:03:27.342379+0000","last_clean":"2024-01-15T18:03:27.342379+0000","last_became_active":"2024-01-15T18:03:27.341248+0000","last_became_peered":"2024-01-15T18:03:27.341248+0000","last_unstale":"2024-01-15T18:03:27.342379+0000","last_undegraded":"2024-01-15T18:03:27.342379+0000","last_fullsized":"2024-01-15T18:03:27.342379+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:51.975291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283811+0000","last_change":"2024-01-15T18:03:26.283811+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283811+0000","last_undegraded":"2024-01-15T18:03:26.283811+0000","last_fullsized":"2024-01-15T18:03:26.283811+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.288618+0000","last_change":"2024-01-15T18:03:26.288618+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.288618+0000","last_undegraded":"2024-01-15T18:03:26.288618+0000","last_fullsized":"2024-01-15T18:03:26.288618+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302106+0000","last_change":"2024-01-15T18:03:27.319087+0000","last_active":"2024-01-15T18:03:28.302106+0000","last_peered":"2024-01-15T18:03:28.302106+0000","last_clean":"2024-01-15T18:03:28.302106+0000","last_became_active":"2024-01-15T18:03:27.318702+0000","last_became_peered":"2024-01-15T18:03:27.318702+0000","last_unstale":"2024-01-15T18:03:28.302106+0000","last_undegraded":"2024-01-15T18:03:28.302106+0000","last_fullsized":"2024-01-15T18:03:28.302106+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:13:57.340165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.321499+0000","last_change":"2024-01-15T18:03:27.319124+0000","last_active":"2024-01-15T18:03:27.321499+0000","last_peered":"2024-01-15T18:03:27.321499+0000","last_clean":"2024-01-15T18:03:27.321499+0000","last_became_active":"2024-01-15T18:03:27.318855+0000","last_became_peered":"2024-01-15T18:03:27.318855+0000","last_unstale":"2024-01-15T18:03:27.321499+0000","last_undegraded":"2024-01-15T18:03:27.321499+0000","last_fullsized":"2024-01-15T18:03:27.321499+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:40.231033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284505+0000","last_change":"2024-01-15T18:03:26.284505+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284505+0000","last_undegraded":"2024-01-15T18:03:26.284505+0000","last_fullsized":"2024-01-15T18:03:26.284505+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.321938+0000","last_change":"2024-01-15T18:03:27.316226+0000","last_active":"2024-01-15T18:03:27.321938+0000","last_peered":"2024-01-15T18:03:27.321938+0000","last_clean":"2024-01-15T18:03:27.321938+0000","last_became_active":"2024-01-15T18:03:27.315889+0000","last_became_peered":"2024-01-15T18:03:27.315889+0000","last_unstale":"2024-01-15T18:03:27.321938+0000","last_undegraded":"2024-01-15T18:03:27.321938+0000","last_fullsized":"2024-01-15T18:03:27.321938+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:56.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302728+0000","last_change":"2024-01-15T18:03:27.331363+0000","last_active":"2024-01-15T18:03:28.302728+0000","last_peered":"2024-01-15T18:03:28.302728+0000","last_clean":"2024-01-15T18:03:28.302728+0000","last_became_active":"2024-01-15T18:03:27.330530+0000","last_became_peered":"2024-01-15T18:03:27.330530+0000","last_unstale":"2024-01-15T18:03:28.302728+0000","last_undegraded":"2024-01-15T18:03:28.302728+0000","last_fullsized":"2024-01-15T18:03:28.302728+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:43:50.724930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.275264+0000","last_change":"2024-01-15T18:03:26.275264+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.275264+0000","last_undegraded":"2024-01-15T18:03:26.275264+0000","last_fullsized":"2024-01-15T18:03:26.275264+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282234+0000","last_change":"2024-01-15T18:03:26.282234+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282234+0000","last_undegraded":"2024-01-15T18:03:26.282234+0000","last_fullsized":"2024-01-15T18:03:26.282234+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.288002+0000","last_change":"2024-01-15T18:03:26.288002+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.288002+0000","last_undegraded":"2024-01-15T18:03:26.288002+0000","last_fullsized":"2024-01-15T18:03:26.288002+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282326+0000","last_change":"2024-01-15T18:03:26.282326+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282326+0000","last_undegraded":"2024-01-15T18:03:26.282326+0000","last_fullsized":"2024-01-15T18:03:26.282326+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302207+0000","last_change":"2024-01-15T18:03:27.329433+0000","last_active":"2024-01-15T18:03:28.302207+0000","last_peered":"2024-01-15T18:03:28.302207+0000","last_clean":"2024-01-15T18:03:28.302207+0000","last_became_active":"2024-01-15T18:03:27.329186+0000","last_became_peered":"2024-01-15T18:03:27.329186+0000","last_unstale":"2024-01-15T18:03:28.302207+0000","last_undegraded":"2024-01-15T18:03:28.302207+0000","last_fullsized":"2024-01-15T18:03:28.302207+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:34.881592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284462+0000","last_change":"2024-01-15T18:03:26.284462+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284462+0000","last_undegraded":"2024-01-15T18:03:26.284462+0000","last_fullsized":"2024-01-15T18:03:26.284462+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.275307+0000","last_change":"2024-01-15T18:03:26.275307+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.275307+0000","last_undegraded":"2024-01-15T18:03:26.275307+0000","last_fullsized":"2024-01-15T18:03:26.275307+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302977+0000","last_change":"2024-01-15T18:03:27.330530+0000","last_active":"2024-01-15T18:03:28.302977+0000","last_peered":"2024-01-15T18:03:28.302977+0000","last_clean":"2024-01-15T18:03:28.302977+0000","last_became_active":"2024-01-15T18:03:27.330149+0000","last_became_peered":"2024-01-15T18:03:27.330149+0000","last_unstale":"2024-01-15T18:03:28.302977+0000","last_undegraded":"2024-01-15T18:03:28.302977+0000","last_fullsized":"2024-01-15T18:03:28.302977+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:12.045333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.286446+0000","last_change":"2024-01-15T18:03:26.286446+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.286446+0000","last_undegraded":"2024-01-15T18:03:26.286446+0000","last_fullsized":"2024-01-15T18:03:26.286446+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341755+0000","last_change":"2024-01-15T18:03:27.338146+0000","last_active":"2024-01-15T18:03:27.341755+0000","last_peered":"2024-01-15T18:03:27.341755+0000","last_clean":"2024-01-15T18:03:27.341755+0000","last_became_active":"2024-01-15T18:03:27.337850+0000","last_became_peered":"2024-01-15T18:03:27.337850+0000","last_unstale":"2024-01-15T18:03:27.341755+0000","last_undegraded":"2024-01-15T18:03:27.341755+0000","last_fullsized":"2024-01-15T18:03:27.341755+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:06:27.744895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282114+0000","last_change":"2024-01-15T18:03:26.282114+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282114+0000","last_undegraded":"2024-01-15T18:03:26.282114+0000","last_fullsized":"2024-01-15T18:03:26.282114+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.276325+0000","last_change":"2024-01-15T18:03:26.276325+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.276325+0000","last_undegraded":"2024-01-15T18:03:26.276325+0000","last_fullsized":"2024-01-15T18:03:26.276325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274415+0000","last_change":"2024-01-15T18:03:26.274415+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274415+0000","last_undegraded":"2024-01-15T18:03:26.274415+0000","last_fullsized":"2024-01-15T18:03:26.274415+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279090+0000","last_change":"2024-01-15T18:03:26.279090+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279090+0000","last_undegraded":"2024-01-15T18:03:26.279090+0000","last_fullsized":"2024-01-15T18:03:26.279090+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279397+0000","last_change":"2024-01-15T18:03:26.279397+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279397+0000","last_undegraded":"2024-01-15T18:03:26.279397+0000","last_fullsized":"2024-01-15T18:03:26.279397+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.276036+0000","last_change":"2024-01-15T18:03:26.276036+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.276036+0000","last_undegraded":"2024-01-15T18:03:26.276036+0000","last_fullsized":"2024-01-15T18:03:26.276036+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284633+0000","last_change":"2024-01-15T18:03:26.284633+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284633+0000","last_undegraded":"2024-01-15T18:03:26.284633+0000","last_fullsized":"2024-01-15T18:03:26.284633+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283351+0000","last_change":"2024-01-15T18:03:26.283351+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283351+0000","last_undegraded":"2024-01-15T18:03:26.283351+0000","last_fullsized":"2024-01-15T18:03:26.283351+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274031+0000","last_change":"2024-01-15T18:03:26.274031+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274031+0000","last_undegraded":"2024-01-15T18:03:26.274031+0000","last_fullsized":"2024-01-15T18:03:26.274031+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282955+0000","last_change":"2024-01-15T18:03:26.282955+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282955+0000","last_undegraded":"2024-01-15T18:03:26.282955+0000","last_fullsized":"2024-01-15T18:03:26.282955+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283969+0000","last_change":"2024-01-15T18:03:26.283969+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283969+0000","last_undegraded":"2024-01-15T18:03:26.283969+0000","last_fullsized":"2024-01-15T18:03:26.283969+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283167+0000","last_change":"2024-01-15T18:03:26.283167+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283167+0000","last_undegraded":"2024-01-15T18:03:26.283167+0000","last_fullsized":"2024-01-15T18:03:26.283167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.324636+0000","last_change":"2024-01-15T18:03:27.321263+0000","last_active":"2024-01-15T18:03:27.324636+0000","last_peered":"2024-01-15T18:03:27.324636+0000","last_clean":"2024-01-15T18:03:27.324636+0000","last_became_active":"2024-01-15T18:03:27.320958+0000","last_became_peered":"2024-01-15T18:03:27.320958+0000","last_unstale":"2024-01-15T18:03:27.324636+0000","last_undegraded":"2024-01-15T18:03:27.324636+0000","last_fullsized":"2024-01-15T18:03:27.324636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:43:50.724930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279714+0000","last_change":"2024-01-15T18:03:26.279714+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279714+0000","last_undegraded":"2024-01-15T18:03:26.279714+0000","last_fullsized":"2024-01-15T18:03:26.279714+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299516+0000","last_change":"2024-01-15T18:03:26.298227+0000","last_active":"2024-01-15T18:03:26.299516+0000","last_peered":"2024-01-15T18:03:26.299516+0000","last_clean":"2024-01-15T18:03:26.299516+0000","last_became_active":"2024-01-15T18:03:26.297015+0000","last_became_peered":"2024-01-15T18:03:26.297015+0000","last_unstale":"2024-01-15T18:03:26.299516+0000","last_undegraded":"2024-01-15T18:03:26.299516+0000","last_fullsized":"2024-01-15T18:03:26.299516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:43.508315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302905+0000","last_change":"2024-01-15T18:03:27.330723+0000","last_active":"2024-01-15T18:03:28.302905+0000","last_peered":"2024-01-15T18:03:28.302905+0000","last_clean":"2024-01-15T18:03:28.302905+0000","last_became_active":"2024-01-15T18:03:27.330236+0000","last_became_peered":"2024-01-15T18:03:27.330236+0000","last_unstale":"2024-01-15T18:03:28.302905+0000","last_undegraded":"2024-01-15T18:03:28.302905+0000","last_fullsized":"2024-01-15T18:03:28.302905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:56.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295089+0000","last_change":"2024-01-15T18:03:26.292579+0000","last_active":"2024-01-15T18:03:26.295089+0000","last_peered":"2024-01-15T18:03:26.295089+0000","last_clean":"2024-01-15T18:03:26.295089+0000","last_became_active":"2024-01-15T18:03:26.291631+0000","last_became_peered":"2024-01-15T18:03:26.291631+0000","last_unstale":"2024-01-15T18:03:26.295089+0000","last_undegraded":"2024-01-15T18:03:26.295089+0000","last_fullsized":"2024-01-15T18:03:26.295089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:32.983343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341535+0000","last_change":"2024-01-15T18:03:27.336001+0000","last_active":"2024-01-15T18:03:27.341535+0000","last_peered":"2024-01-15T18:03:27.341535+0000","last_clean":"2024-01-15T18:03:27.341535+0000","last_became_active":"2024-01-15T18:03:27.335265+0000","last_became_peered":"2024-01-15T18:03:27.335265+0000","last_unstale":"2024-01-15T18:03:27.341535+0000","last_undegraded":"2024-01-15T18:03:27.341535+0000","last_fullsized":"2024-01-15T18:03:27.341535+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:52.859519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.304785+0000","last_change":"2024-01-15T18:03:26.303968+0000","last_active":"2024-01-15T18:03:26.304785+0000","last_peered":"2024-01-15T18:03:26.304785+0000","last_clean":"2024-01-15T18:03:26.304785+0000","last_became_active":"2024-01-15T18:03:26.303669+0000","last_became_peered":"2024-01-15T18:03:26.303669+0000","last_unstale":"2024-01-15T18:03:26.304785+0000","last_undegraded":"2024-01-15T18:03:26.304785+0000","last_fullsized":"2024-01-15T18:03:26.304785+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:43.710985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281710+0000","last_change":"2024-01-15T18:03:26.281710+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281710+0000","last_undegraded":"2024-01-15T18:03:26.281710+0000","last_fullsized":"2024-01-15T18:03:26.281710+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342311+0000","last_change":"2024-01-15T18:03:26.300990+0000","last_active":"2024-01-15T18:03:27.342311+0000","last_peered":"2024-01-15T18:03:27.342311+0000","last_clean":"2024-01-15T18:03:27.342311+0000","last_became_active":"2024-01-15T18:03:26.300735+0000","last_became_peered":"2024-01-15T18:03:26.300735+0000","last_unstale":"2024-01-15T18:03:27.342311+0000","last_undegraded":"2024-01-15T18:03:27.342311+0000","last_fullsized":"2024-01-15T18:03:27.342311+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:53:06.533003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283325+0000","last_change":"2024-01-15T18:03:26.283325+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283325+0000","last_undegraded":"2024-01-15T18:03:26.283325+0000","last_fullsized":"2024-01-15T18:03:26.283325+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297810+0000","last_change":"2024-01-15T18:03:26.296481+0000","last_active":"2024-01-15T18:03:26.297810+0000","last_peered":"2024-01-15T18:03:26.297810+0000","last_clean":"2024-01-15T18:03:26.297810+0000","last_became_active":"2024-01-15T18:03:26.293509+0000","last_became_peered":"2024-01-15T18:03:26.293509+0000","last_unstale":"2024-01-15T18:03:26.297810+0000","last_undegraded":"2024-01-15T18:03:26.297810+0000","last_fullsized":"2024-01-15T18:03:26.297810+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:48.220096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274045+0000","last_change":"2024-01-15T18:03:26.274045+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274045+0000","last_undegraded":"2024-01-15T18:03:26.274045+0000","last_fullsized":"2024-01-15T18:03:26.274045+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.340684+0000","last_change":"2024-01-15T18:03:26.302584+0000","last_active":"2024-01-15T18:03:27.340684+0000","last_peered":"2024-01-15T18:03:27.340684+0000","last_clean":"2024-01-15T18:03:27.340684+0000","last_became_active":"2024-01-15T18:03:26.301356+0000","last_became_peered":"2024-01-15T18:03:26.301356+0000","last_unstale":"2024-01-15T18:03:27.340684+0000","last_undegraded":"2024-01-15T18:03:27.340684+0000","last_fullsized":"2024-01-15T18:03:27.340684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:22:00.115460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.301672+0000","last_change":"2024-01-15T18:03:27.326042+0000","last_active":"2024-01-15T18:03:28.301672+0000","last_peered":"2024-01-15T18:03:28.301672+0000","last_clean":"2024-01-15T18:03:28.301672+0000","last_became_active":"2024-01-15T18:03:27.325476+0000","last_became_peered":"2024-01-15T18:03:27.325476+0000","last_unstale":"2024-01-15T18:03:28.301672+0000","last_undegraded":"2024-01-15T18:03:28.301672+0000","last_fullsized":"2024-01-15T18:03:28.301672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:37:22.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.293022+0000","last_change":"2024-01-15T18:03:26.282542+0000","last_active":"2024-01-15T18:03:26.293022+0000","last_peered":"2024-01-15T18:03:26.293022+0000","last_clean":"2024-01-15T18:03:26.293022+0000","last_became_active":"2024-01-15T18:03:26.282226+0000","last_became_peered":"2024-01-15T18:03:26.282226+0000","last_unstale":"2024-01-15T18:03:26.293022+0000","last_undegraded":"2024-01-15T18:03:26.293022+0000","last_fullsized":"2024-01-15T18:03:26.293022+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:57.562173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281723+0000","last_change":"2024-01-15T18:03:26.281723+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281723+0000","last_undegraded":"2024-01-15T18:03:26.281723+0000","last_fullsized":"2024-01-15T18:03:26.281723+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.312237+0000","last_change":"2024-01-15T18:03:26.285864+0000","last_active":"2024-01-15T18:03:27.312237+0000","last_peered":"2024-01-15T18:03:27.312237+0000","last_clean":"2024-01-15T18:03:27.312237+0000","last_became_active":"2024-01-15T18:03:26.285507+0000","last_became_peered":"2024-01-15T18:03:26.285507+0000","last_unstale":"2024-01-15T18:03:27.312237+0000","last_undegraded":"2024-01-15T18:03:27.312237+0000","last_fullsized":"2024-01-15T18:03:27.312237+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:47.778037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.283831+0000","last_change":"2024-01-15T18:03:26.283831+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.283831+0000","last_undegraded":"2024-01-15T18:03:26.283831+0000","last_fullsized":"2024-01-15T18:03:26.283831+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295148+0000","last_change":"2024-01-15T18:03:26.294412+0000","last_active":"2024-01-15T18:03:26.295148+0000","last_peered":"2024-01-15T18:03:26.295148+0000","last_clean":"2024-01-15T18:03:26.295148+0000","last_became_active":"2024-01-15T18:03:26.294070+0000","last_became_peered":"2024-01-15T18:03:26.294070+0000","last_unstale":"2024-01-15T18:03:26.295148+0000","last_undegraded":"2024-01-15T18:03:26.295148+0000","last_fullsized":"2024-01-15T18:03:26.295148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:06:26.733111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.280249+0000","last_change":"2024-01-15T18:03:26.280249+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.280249+0000","last_undegraded":"2024-01-15T18:03:26.280249+0000","last_fullsized":"2024-01-15T18:03:26.280249+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.312561+0000","last_change":"2024-01-15T18:03:26.302162+0000","last_active":"2024-01-15T18:03:27.312561+0000","last_peered":"2024-01-15T18:03:27.312561+0000","last_clean":"2024-01-15T18:03:27.312561+0000","last_became_active":"2024-01-15T18:03:26.301821+0000","last_became_peered":"2024-01-15T18:03:26.301821+0000","last_unstale":"2024-01-15T18:03:27.312561+0000","last_undegraded":"2024-01-15T18:03:27.312561+0000","last_fullsized":"2024-01-15T18:03:27.312561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:24:07.021223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.340063+0000","last_change":"2024-01-15T18:03:27.329589+0000","last_active":"2024-01-15T18:03:27.340063+0000","last_peered":"2024-01-15T18:03:27.340063+0000","last_clean":"2024-01-15T18:03:27.340063+0000","last_became_active":"2024-01-15T18:03:27.328118+0000","last_became_peered":"2024-01-15T18:03:27.328118+0000","last_unstale":"2024-01-15T18:03:27.340063+0000","last_undegraded":"2024-01-15T18:03:27.340063+0000","last_fullsized":"2024-01-15T18:03:27.340063+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:48.991808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.296636+0000","last_change":"2024-01-15T18:03:26.295197+0000","last_active":"2024-01-15T18:03:26.296636+0000","last_peered":"2024-01-15T18:03:26.296636+0000","last_clean":"2024-01-15T18:03:26.296636+0000","last_became_active":"2024-01-15T18:03:26.294932+0000","last_became_peered":"2024-01-15T18:03:26.294932+0000","last_unstale":"2024-01-15T18:03:26.296636+0000","last_undegraded":"2024-01-15T18:03:26.296636+0000","last_fullsized":"2024-01-15T18:03:26.296636+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:20.869224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302274+0000","last_change":"2024-01-15T18:03:27.315984+0000","last_active":"2024-01-15T18:03:28.302274+0000","last_peered":"2024-01-15T18:03:28.302274+0000","last_clean":"2024-01-15T18:03:28.302274+0000","last_became_active":"2024-01-15T18:03:27.315700+0000","last_became_peered":"2024-01-15T18:03:27.315700+0000","last_unstale":"2024-01-15T18:03:28.302274+0000","last_undegraded":"2024-01-15T18:03:28.302274+0000","last_fullsized":"2024-01-15T18:03:28.302274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:16.050477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294412+0000","last_change":"2024-01-15T18:03:26.283744+0000","last_active":"2024-01-15T18:03:26.294412+0000","last_peered":"2024-01-15T18:03:26.294412+0000","last_clean":"2024-01-15T18:03:26.294412+0000","last_became_active":"2024-01-15T18:03:26.283382+0000","last_became_peered":"2024-01-15T18:03:26.283382+0000","last_unstale":"2024-01-15T18:03:26.294412+0000","last_undegraded":"2024-01-15T18:03:26.294412+0000","last_fullsized":"2024-01-15T18:03:26.294412+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:13:56.328381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278423+0000","last_change":"2024-01-15T18:03:26.278423+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278423+0000","last_undegraded":"2024-01-15T18:03:26.278423+0000","last_fullsized":"2024-01-15T18:03:26.278423+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295393+0000","last_change":"2024-01-15T18:03:26.290460+0000","last_active":"2024-01-15T18:03:26.295393+0000","last_peered":"2024-01-15T18:03:26.295393+0000","last_clean":"2024-01-15T18:03:26.295393+0000","last_became_active":"2024-01-15T18:03:26.290114+0000","last_became_peered":"2024-01-15T18:03:26.290114+0000","last_unstale":"2024-01-15T18:03:26.295393+0000","last_undegraded":"2024-01-15T18:03:26.295393+0000","last_fullsized":"2024-01-15T18:03:26.295393+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:49.162905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.286454+0000","last_change":"2024-01-15T18:03:26.286454+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.286454+0000","last_undegraded":"2024-01-15T18:03:26.286454+0000","last_fullsized":"2024-01-15T18:03:26.286454+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.306852+0000","last_change":"2024-01-15T18:03:26.302531+0000","last_active":"2024-01-15T18:03:27.306852+0000","last_peered":"2024-01-15T18:03:27.306852+0000","last_clean":"2024-01-15T18:03:27.306852+0000","last_became_active":"2024-01-15T18:03:26.301660+0000","last_became_peered":"2024-01-15T18:03:26.301660+0000","last_unstale":"2024-01-15T18:03:27.306852+0000","last_undegraded":"2024-01-15T18:03:27.306852+0000","last_fullsized":"2024-01-15T18:03:27.306852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:18:48.053993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281591+0000","last_change":"2024-01-15T18:03:26.281591+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281591+0000","last_undegraded":"2024-01-15T18:03:26.281591+0000","last_fullsized":"2024-01-15T18:03:26.281591+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299435+0000","last_change":"2024-01-15T18:03:26.297958+0000","last_active":"2024-01-15T18:03:26.299435+0000","last_peered":"2024-01-15T18:03:26.299435+0000","last_clean":"2024-01-15T18:03:26.299435+0000","last_became_active":"2024-01-15T18:03:26.297417+0000","last_became_peered":"2024-01-15T18:03:26.297417+0000","last_unstale":"2024-01-15T18:03:26.299435+0000","last_undegraded":"2024-01-15T18:03:26.299435+0000","last_fullsized":"2024-01-15T18:03:26.299435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:20:43.839295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.301949+0000","last_change":"2024-01-15T18:03:27.330251+0000","last_active":"2024-01-15T18:03:28.301949+0000","last_peered":"2024-01-15T18:03:28.301949+0000","last_clean":"2024-01-15T18:03:28.301949+0000","last_became_active":"2024-01-15T18:03:27.329948+0000","last_became_peered":"2024-01-15T18:03:27.329948+0000","last_unstale":"2024-01-15T18:03:28.301949+0000","last_undegraded":"2024-01-15T18:03:28.301949+0000","last_fullsized":"2024-01-15T18:03:28.301949+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:58.573957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.293724+0000","last_change":"2024-01-15T18:03:26.287860+0000","last_active":"2024-01-15T18:03:26.293724+0000","last_peered":"2024-01-15T18:03:26.293724+0000","last_clean":"2024-01-15T18:03:26.293724+0000","last_became_active":"2024-01-15T18:03:26.287555+0000","last_became_peered":"2024-01-15T18:03:26.287555+0000","last_unstale":"2024-01-15T18:03:26.293724+0000","last_undegraded":"2024-01-15T18:03:26.293724+0000","last_fullsized":"2024-01-15T18:03:26.293724+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:50.963506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.274473+0000","last_change":"2024-01-15T18:03:26.274473+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.274473+0000","last_undegraded":"2024-01-15T18:03:26.274473+0000","last_fullsized":"2024-01-15T18:03:26.274473+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341894+0000","last_change":"2024-01-15T18:03:26.300342+0000","last_active":"2024-01-15T18:03:27.341894+0000","last_peered":"2024-01-15T18:03:27.341894+0000","last_clean":"2024-01-15T18:03:27.341894+0000","last_became_active":"2024-01-15T18:03:26.300100+0000","last_became_peered":"2024-01-15T18:03:26.300100+0000","last_unstale":"2024-01-15T18:03:27.341894+0000","last_undegraded":"2024-01-15T18:03:27.341894+0000","last_fullsized":"2024-01-15T18:03:27.341894+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:08.274277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279678+0000","last_change":"2024-01-15T18:03:26.279678+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279678+0000","last_undegraded":"2024-01-15T18:03:26.279678+0000","last_fullsized":"2024-01-15T18:03:26.279678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.303643+0000","last_change":"2024-01-15T18:03:26.298853+0000","last_active":"2024-01-15T18:03:26.303643+0000","last_peered":"2024-01-15T18:03:26.303643+0000","last_clean":"2024-01-15T18:03:26.303643+0000","last_became_active":"2024-01-15T18:03:26.298530+0000","last_became_peered":"2024-01-15T18:03:26.298530+0000","last_unstale":"2024-01-15T18:03:26.303643+0000","last_undegraded":"2024-01-15T18:03:26.303643+0000","last_fullsized":"2024-01-15T18:03:26.303643+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:51.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284222+0000","last_change":"2024-01-15T18:03:26.284222+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284222+0000","last_undegraded":"2024-01-15T18:03:26.284222+0000","last_fullsized":"2024-01-15T18:03:26.284222+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.343468+0000","last_change":"2024-01-15T18:03:26.301874+0000","last_active":"2024-01-15T18:03:27.343468+0000","last_peered":"2024-01-15T18:03:27.343468+0000","last_clean":"2024-01-15T18:03:27.343468+0000","last_became_active":"2024-01-15T18:03:26.301425+0000","last_became_peered":"2024-01-15T18:03:26.301425+0000","last_unstale":"2024-01-15T18:03:27.343468+0000","last_undegraded":"2024-01-15T18:03:27.343468+0000","last_fullsized":"2024-01-15T18:03:27.343468+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:43.710985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302611+0000","last_change":"2024-01-15T18:03:27.325587+0000","last_active":"2024-01-15T18:03:28.302611+0000","last_peered":"2024-01-15T18:03:28.302611+0000","last_clean":"2024-01-15T18:03:28.302611+0000","last_became_active":"2024-01-15T18:03:27.325317+0000","last_became_peered":"2024-01-15T18:03:27.325317+0000","last_unstale":"2024-01-15T18:03:28.302611+0000","last_undegraded":"2024-01-15T18:03:28.302611+0000","last_fullsized":"2024-01-15T18:03:28.302611+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:18:27.777253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299247+0000","last_change":"2024-01-15T18:03:26.295610+0000","last_active":"2024-01-15T18:03:26.299247+0000","last_peered":"2024-01-15T18:03:26.299247+0000","last_clean":"2024-01-15T18:03:26.299247+0000","last_became_active":"2024-01-15T18:03:26.295164+0000","last_became_peered":"2024-01-15T18:03:26.295164+0000","last_unstale":"2024-01-15T18:03:26.299247+0000","last_undegraded":"2024-01-15T18:03:26.299247+0000","last_fullsized":"2024-01-15T18:03:26.299247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:42.703538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.279636+0000","last_change":"2024-01-15T18:03:26.279636+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.279636+0000","last_undegraded":"2024-01-15T18:03:26.279636+0000","last_fullsized":"2024-01-15T18:03:26.279636+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.316666+0000","last_change":"2024-01-15T18:03:26.301741+0000","last_active":"2024-01-15T18:03:27.316666+0000","last_peered":"2024-01-15T18:03:27.316666+0000","last_clean":"2024-01-15T18:03:27.316666+0000","last_became_active":"2024-01-15T18:03:26.301228+0000","last_became_peered":"2024-01-15T18:03:26.301228+0000","last_unstale":"2024-01-15T18:03:27.316666+0000","last_undegraded":"2024-01-15T18:03:27.316666+0000","last_fullsized":"2024-01-15T18:03:27.316666+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:08.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278650+0000","last_change":"2024-01-15T18:03:26.278650+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278650+0000","last_undegraded":"2024-01-15T18:03:26.278650+0000","last_fullsized":"2024-01-15T18:03:26.278650+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294795+0000","last_change":"2024-01-15T18:03:26.293020+0000","last_active":"2024-01-15T18:03:26.294795+0000","last_peered":"2024-01-15T18:03:26.294795+0000","last_clean":"2024-01-15T18:03:26.294795+0000","last_became_active":"2024-01-15T18:03:26.292663+0000","last_became_peered":"2024-01-15T18:03:26.292663+0000","last_unstale":"2024-01-15T18:03:26.294795+0000","last_undegraded":"2024-01-15T18:03:26.294795+0000","last_fullsized":"2024-01-15T18:03:26.294795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:51.697653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278555+0000","last_change":"2024-01-15T18:03:26.278555+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278555+0000","last_undegraded":"2024-01-15T18:03:26.278555+0000","last_fullsized":"2024-01-15T18:03:26.278555+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.295435+0000","last_change":"2024-01-15T18:03:26.292741+0000","last_active":"2024-01-15T18:03:26.295435+0000","last_peered":"2024-01-15T18:03:26.295435+0000","last_clean":"2024-01-15T18:03:26.295435+0000","last_became_active":"2024-01-15T18:03:26.292349+0000","last_became_peered":"2024-01-15T18:03:26.292349+0000","last_unstale":"2024-01-15T18:03:26.295435+0000","last_undegraded":"2024-01-15T18:03:26.295435+0000","last_fullsized":"2024-01-15T18:03:26.295435+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:19:42.341080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.282316+0000","last_change":"2024-01-15T18:03:26.282316+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.282316+0000","last_undegraded":"2024-01-15T18:03:26.282316+0000","last_fullsized":"2024-01-15T18:03:26.282316+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341991+0000","last_change":"2024-01-15T18:03:26.299863+0000","last_active":"2024-01-15T18:03:27.341991+0000","last_peered":"2024-01-15T18:03:27.341991+0000","last_clean":"2024-01-15T18:03:27.341991+0000","last_became_active":"2024-01-15T18:03:26.299525+0000","last_became_peered":"2024-01-15T18:03:26.299525+0000","last_unstale":"2024-01-15T18:03:27.341991+0000","last_undegraded":"2024-01-15T18:03:27.341991+0000","last_fullsized":"2024-01-15T18:03:27.341991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:51.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.284466+0000","last_change":"2024-01-15T18:03:26.284466+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.284466+0000","last_undegraded":"2024-01-15T18:03:26.284466+0000","last_fullsized":"2024-01-15T18:03:26.284466+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.299404+0000","last_change":"2024-01-15T18:03:26.297682+0000","last_active":"2024-01-15T18:03:26.299404+0000","last_peered":"2024-01-15T18:03:26.299404+0000","last_clean":"2024-01-15T18:03:26.299404+0000","last_became_active":"2024-01-15T18:03:26.297319+0000","last_became_peered":"2024-01-15T18:03:26.297319+0000","last_unstale":"2024-01-15T18:03:26.299404+0000","last_undegraded":"2024-01-15T18:03:26.299404+0000","last_fullsized":"2024-01-15T18:03:26.299404+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:40:28.369164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278628+0000","last_change":"2024-01-15T18:03:26.278628+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278628+0000","last_undegraded":"2024-01-15T18:03:26.278628+0000","last_fullsized":"2024-01-15T18:03:26.278628+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.311963+0000","last_change":"2024-01-15T18:03:26.290405+0000","last_active":"2024-01-15T18:03:27.311963+0000","last_peered":"2024-01-15T18:03:27.311963+0000","last_clean":"2024-01-15T18:03:27.311963+0000","last_became_active":"2024-01-15T18:03:26.288657+0000","last_became_peered":"2024-01-15T18:03:26.288657+0000","last_unstale":"2024-01-15T18:03:27.311963+0000","last_undegraded":"2024-01-15T18:03:27.311963+0000","last_fullsized":"2024-01-15T18:03:27.311963+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:56.868408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.281044+0000","last_change":"2024-01-15T18:03:26.281044+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.281044+0000","last_undegraded":"2024-01-15T18:03:26.281044+0000","last_fullsized":"2024-01-15T18:03:26.281044+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297486+0000","last_change":"2024-01-15T18:03:26.296161+0000","last_active":"2024-01-15T18:03:26.297486+0000","last_peered":"2024-01-15T18:03:26.297486+0000","last_clean":"2024-01-15T18:03:26.297486+0000","last_became_active":"2024-01-15T18:03:26.295339+0000","last_became_peered":"2024-01-15T18:03:26.295339+0000","last_unstale":"2024-01-15T18:03:26.297486+0000","last_undegraded":"2024-01-15T18:03:26.297486+0000","last_fullsized":"2024-01-15T18:03:26.297486+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:20:43.839295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302217+0000","last_change":"2024-01-15T18:03:27.330393+0000","last_active":"2024-01-15T18:03:28.302217+0000","last_peered":"2024-01-15T18:03:28.302217+0000","last_clean":"2024-01-15T18:03:28.302217+0000","last_became_active":"2024-01-15T18:03:27.329266+0000","last_became_peered":"2024-01-15T18:03:27.329266+0000","last_unstale":"2024-01-15T18:03:28.302217+0000","last_undegraded":"2024-01-15T18:03:28.302217+0000","last_fullsized":"2024-01-15T18:03:28.302217+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:50.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.311753+0000","last_change":"2024-01-15T18:03:26.296244+0000","last_active":"2024-01-15T18:03:27.311753+0000","last_peered":"2024-01-15T18:03:27.311753+0000","last_clean":"2024-01-15T18:03:27.311753+0000","last_became_active":"2024-01-15T18:03:26.295955+0000","last_became_peered":"2024-01-15T18:03:26.295955+0000","last_unstale":"2024-01-15T18:03:27.311753+0000","last_undegraded":"2024-01-15T18:03:27.311753+0000","last_fullsized":"2024-01-15T18:03:27.311753+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:44:34.308154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.331003+0000","last_change":"2024-01-15T18:03:27.323628+0000","last_active":"2024-01-15T18:03:27.331003+0000","last_peered":"2024-01-15T18:03:27.331003+0000","last_clean":"2024-01-15T18:03:27.331003+0000","last_became_active":"2024-01-15T18:03:27.323353+0000","last_became_peered":"2024-01-15T18:03:27.323353+0000","last_unstale":"2024-01-15T18:03:27.331003+0000","last_undegraded":"2024-01-15T18:03:27.331003+0000","last_fullsized":"2024-01-15T18:03:27.331003+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:12.045333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.297615+0000","last_change":"2024-01-15T18:03:26.295494+0000","last_active":"2024-01-15T18:03:26.297615+0000","last_peered":"2024-01-15T18:03:26.297615+0000","last_clean":"2024-01-15T18:03:26.297615+0000","last_became_active":"2024-01-15T18:03:26.295221+0000","last_became_peered":"2024-01-15T18:03:26.295221+0000","last_unstale":"2024-01-15T18:03:26.297615+0000","last_undegraded":"2024-01-15T18:03:26.297615+0000","last_fullsized":"2024-01-15T18:03:26.297615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:33.869808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.285274+0000","last_change":"2024-01-15T18:03:26.285274+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.285274+0000","last_undegraded":"2024-01-15T18:03:26.285274+0000","last_fullsized":"2024-01-15T18:03:26.285274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-01-15T18:03:26.294031+0000","last_change":"2024-01-15T18:03:26.291361+0000","last_active":"2024-01-15T18:03:26.294031+0000","last_peered":"2024-01-15T18:03:26.294031+0000","last_clean":"2024-01-15T18:03:26.294031+0000","last_became_active":"2024-01-15T18:03:26.290863+0000","last_became_peered":"2024-01-15T18:03:26.290863+0000","last_unstale":"2024-01-15T18:03:26.294031+0000","last_undegraded":"2024-01-15T18:03:26.294031+0000","last_fullsized":"2024-01-15T18:03:26.294031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:24:07.021223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.332585+0000","last_change":"2024-01-15T18:03:27.325192+0000","last_active":"2024-01-15T18:03:27.332585+0000","last_peered":"2024-01-15T18:03:27.332585+0000","last_clean":"2024-01-15T18:03:27.332585+0000","last_became_active":"2024-01-15T18:03:27.324697+0000","last_became_peered":"2024-01-15T18:03:27.324697+0000","last_unstale":"2024-01-15T18:03:27.332585+0000","last_undegraded":"2024-01-15T18:03:27.332585+0000","last_fullsized":"2024-01-15T18:03:27.332585+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:20.920788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.308092+0000","last_change":"2024-01-15T18:03:26.298754+0000","last_active":"2024-01-15T18:03:27.308092+0000","last_peered":"2024-01-15T18:03:27.308092+0000","last_clean":"2024-01-15T18:03:27.308092+0000","last_became_active":"2024-01-15T18:03:26.298412+0000","last_became_peered":"2024-01-15T18:03:26.298412+0000","last_unstale":"2024-01-15T18:03:27.308092+0000","last_undegraded":"2024-01-15T18:03:27.308092+0000","last_fullsized":"2024-01-15T18:03:27.308092+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:43:09.251315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":70,"state":"creating+peering","last_fresh":"2024-01-15T18:03:26.278407+0000","last_change":"2024-01-15T18:03:26.278407+0000","last_active":"2024-01-15T18:03:26.259097+0000","last_peered":"2024-01-15T18:03:26.259097+0000","last_clean":"2024-01-15T18:03:26.259097+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:03:26.278407+0000","last_undegraded":"2024-01-15T18:03:26.278407+0000","last_fullsized":"2024-01-15T18:03:26.278407+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.342463+0000","last_change":"2024-01-15T18:03:26.300437+0000","last_active":"2024-01-15T18:03:27.342463+0000","last_peered":"2024-01-15T18:03:27.342463+0000","last_clean":"2024-01-15T18:03:27.342463+0000","last_became_active":"2024-01-15T18:03:26.300210+0000","last_became_peered":"2024-01-15T18:03:26.300210+0000","last_unstale":"2024-01-15T18:03:27.342463+0000","last_undegraded":"2024-01-15T18:03:27.342463+0000","last_fullsized":"2024-01-15T18:03:27.342463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:47.498523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":206,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-01-15T18:03:25.275709+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:03:25.275709+0000","last_peered":"2024-01-15T18:03:25.275709+0000","last_clean":"2024-01-15T18:03:25.275709+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:03:25.275709+0000","last_undegraded":"2024-01-15T18:03:25.275709+0000","last_fullsized":"2024-01-15T18:03:25.275709+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":92,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50640,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321200,"statfs":{"total":96636764160,"available":96584908800,"internally_reserved":0,"allocated":1441792,"data_stored":947133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.04,"5min":1.6439999999999999,"15min":1.3180000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":19.843,"5min":19.843,"15min":19.843},"last":0.878},{"interface":"front","average":{"1min":4.2229999999999999,"5min":2.3180000000000001,"15min":1.552},"min":{"1min":0.75700000000000001,"5min":0.629,"15min":0.623},"max":{"1min":30.498000000000001,"5min":38.137,"15min":38.137},"last":1.0149999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.129,"5min":1.716,"15min":1.383},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":19.917000000000002,"5min":19.917000000000002,"15min":19.917000000000002},"last":1.3260000000000001},{"interface":"front","average":{"1min":2.2839999999999998,"5min":2.109,"15min":1.546},"min":{"1min":0.71199999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":10.361000000000001,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.143}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":1.6910000000000001,"15min":1.345},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":8.1289999999999996,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":0.94899999999999995},{"interface":"front","average":{"1min":3.0760000000000001,"5min":1.8149999999999999,"15min":1.5169999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":13.478999999999999,"5min":13.478999999999999,"15min":13.478999999999999},"last":1.202}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.3359999999999999,"5min":1.9450000000000001,"15min":1.5129999999999999},"min":{"1min":0.56000000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":16.632000000000001,"5min":16.632000000000001,"15min":16.632000000000001},"last":1.284},{"interface":"front","average":{"1min":3.177,"5min":1.867,"15min":1.617},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":13.439,"5min":13.439,"15min":13.439},"last":1.4279999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":2.145,"15min":1.694},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50700000000000001},"max":{"1min":10.881,"5min":22.359999999999999,"15min":22.359999999999999},"last":1.254},{"interface":"front","average":{"1min":4.4820000000000002,"5min":2.2149999999999999,"15min":1.702},"min":{"1min":0.52700000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":38.682000000000002,"5min":38.682000000000002,"15min":38.682000000000002},"last":1.633}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.8140000000000001,"5min":2.286,"15min":1.831},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":13.379,"5min":25.006,"15min":25.006},"last":1.4950000000000001},{"interface":"front","average":{"1min":2.9169999999999998,"5min":2.2669999999999999,"15min":1.8069999999999999},"min":{"1min":0.80200000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":15.664999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.661}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.4729999999999999,"5min":2.5,"15min":1.954},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.59299999999999997},"max":{"1min":11.366,"5min":23.834,"15min":23.834},"last":1.794},{"interface":"front","average":{"1min":2.8380000000000001,"5min":2.8050000000000002,"15min":2.012},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":9.2599999999999998,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.7310000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45900,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325940,"statfs":{"total":96636764160,"available":96589762560,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.325,"15min":1.3340000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":7.79,"5min":7.79,"15min":12.544},"last":0.88500000000000001},{"interface":"front","average":{"1min":2.1379999999999999,"5min":1.579,"15min":1.3149999999999999},"min":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":14.013,"5min":14.013,"15min":14.013},"last":1.5009999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.403,"15min":1.284},"min":{"1min":0.67400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":7.6200000000000001,"5min":7.6200000000000001,"15min":12.811999999999999},"last":1.107},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.8999999999999999,"15min":1.4199999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":20.523,"5min":20.523,"15min":20.523},"last":1.4199999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":1.877,"15min":1.4279999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":17.713000000000001,"5min":17.713000000000001,"15min":17.713000000000001},"last":1.871},{"interface":"front","average":{"1min":1.788,"5min":1.3799999999999999,"15min":1.4850000000000001},"min":{"1min":0.59899999999999998,"5min":0.504,"15min":0.504},"max":{"1min":7.702,"5min":7.702,"15min":12.897},"last":1.006}]},{"osd":3,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0840000000000001,"5min":1.74,"15min":1.6000000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.7480000000000002,"5min":9.5,"15min":12.717000000000001},"last":1.3320000000000001},{"interface":"front","average":{"1min":2.3410000000000002,"5min":1.825,"15min":1.518},"min":{"1min":0.876,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":13.795,"5min":14.066000000000001,"15min":14.066000000000001},"last":1.607}]},{"osd":4,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":4.0739999999999998,"5min":2.0659999999999998,"15min":1.6200000000000001},"min":{"1min":0.59599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.5760000000000001},{"interface":"front","average":{"1min":2.2650000000000001,"5min":1.774,"15min":1.6890000000000001},"min":{"1min":0.52000000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":7.8319999999999999,"5min":17.289000000000001,"15min":17.289000000000001},"last":1.248}]},{"osd":5,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.9529999999999998,"5min":1.819,"15min":1.5620000000000001},"min":{"1min":0.66900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":13.593,"5min":14.176,"15min":14.176},"last":1.8440000000000001},{"interface":"front","average":{"1min":3.4950000000000001,"5min":2.1110000000000002,"15min":1.849},"min":{"1min":0.621,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":17.949000000000002,"5min":17.949000000000002,"15min":17.949000000000002},"last":1.752}]},{"osd":7,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.4359999999999999,"5min":2.0190000000000001,"15min":1.708},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":21.265000000000001,"5min":21.265000000000001,"15min":21.265000000000001},"last":1.71},{"interface":"front","average":{"1min":4.3949999999999996,"5min":2.3439999999999999,"15min":1.9299999999999999},"min":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":19.571000000000002,"5min":19.571000000000002,"15min":19.571000000000002},"last":1.665}]}]},{"osd":5,"up_from":14,"seq":60129542248,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45900,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325940,"statfs":{"total":96636764160,"available":96589762560,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9890000000000001,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":19.266999999999999,"5min":19.266999999999999,"15min":19.266999999999999},"last":1.236},{"interface":"front","average":{"1min":1.292,"5min":1.607,"15min":1.4330000000000001},"min":{"1min":0.69399999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.1760000000000002,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.55}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":1.5469999999999999,"15min":1.3169999999999999},"min":{"1min":0.71299999999999997,"5min":0.499,"15min":0.499},"max":{"1min":9.1259999999999994,"5min":16.297000000000001,"15min":16.297000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.8899999999999999,"5min":1.3859999999999999,"15min":1.2629999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2379999999999995,"5min":9.2379999999999995,"15min":9.2379999999999995},"last":1.0569999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.345,"15min":1.262},"min":{"1min":0.88600000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.0369999999999999,"5min":8.1820000000000004,"15min":8.1820000000000004},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.8220000000000001,"5min":1.6479999999999999,"15min":1.3899999999999999},"min":{"1min":0.88500000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2159999999999993,"5min":16.158999999999999,"15min":16.158999999999999},"last":0.996}]},{"osd":3,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.5549999999999999,"15min":1.4370000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.56100000000000005},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":14.914},"last":1.341},{"interface":"front","average":{"1min":1.9930000000000001,"5min":1.526,"15min":1.4570000000000001},"min":{"1min":0.76300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":7.0110000000000001,"5min":7.0110000000000001,"15min":7.0110000000000001},"last":1.3879999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9850000000000001,"5min":1.6579999999999999,"15min":1.4870000000000001},"min":{"1min":0.76100000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":9.0410000000000004,"5min":16.414999999999999,"15min":16.414999999999999},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.925,"15min":1.627},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":9.1519999999999992,"5min":17.991,"15min":17.991},"last":1.51}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.492,"15min":1.573},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.669,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":1.6120000000000001},{"interface":"front","average":{"1min":2.0609999999999999,"5min":1.788,"15min":1.623},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":9.1649999999999991,"5min":16.364000000000001,"15min":16.364000000000001},"last":1.468}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9370000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"min":{"1min":0.66300000000000003,"5min":0.628,"15min":0.628},"max":{"1min":9.0600000000000005,"5min":9.0600000000000005,"15min":9.0600000000000005},"last":1.423},{"interface":"front","average":{"1min":3.0600000000000001,"5min":1.891,"15min":1.79},"min":{"1min":0.83999999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":19.356999999999999,"5min":19.356999999999999,"15min":19.356999999999999},"last":1.669}]}]},{"osd":4,"up_from":14,"seq":60129542248,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50060,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321780,"statfs":{"total":96636764160,"available":96585502720,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.5229999999999999,"15min":1.3149999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":7.8869999999999996,"5min":11.413,"15min":11.413},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.8180000000000001,"5min":1.534,"15min":1.401},"min":{"1min":0.63,"5min":0.63,"15min":0.53700000000000003},"max":{"1min":7.9160000000000004,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":0.84899999999999998}]},{"osd":1,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.367,"15min":1.369},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.45000000000000001},"max":{"1min":3.5750000000000002,"5min":9.7010000000000005,"15min":11.987},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.444,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.6109999999999998,"5min":16.058,"15min":16.058},"last":0.72899999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.7789999999999999,"15min":1.575},"min":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.6890000000000001,"5min":13.865,"15min":13.865},"last":1.052},{"interface":"front","average":{"1min":1.788,"5min":1.833,"15min":1.5009999999999999},"min":{"1min":0.89100000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":5.7050000000000001,"5min":13.903,"15min":13.903},"last":1.3879999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.919,"5min":1.9099999999999999,"15min":1.675},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":11.773,"5min":26.055,"15min":26.055},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.673,"15min":1.611},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":4.0419999999999998,"5min":13.25,"15min":13.25},"last":1.004}]},{"osd":5,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":2.9729999999999999,"5min":1.853,"15min":1.72},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":29.422000000000001,"5min":29.422000000000001,"15min":29.422000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.5720000000000001,"15min":1.619},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.7160000000000002,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.216}]},{"osd":6,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.6439999999999999,"15min":1.6379999999999999},"min":{"1min":0.80100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":5.7919999999999998,"5min":18.260000000000002,"15min":18.260000000000002},"last":1.579},{"interface":"front","average":{"1min":1.597,"5min":1.6890000000000001,"15min":1.7849999999999999},"min":{"1min":0.59999999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.6459999999999999,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.7410000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.982,"5min":1.881,"15min":1.889},"min":{"1min":0.60499999999999998,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":10.664,"5min":10.664,"15min":11.894},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.851,"5min":2.0379999999999998,"15min":1.9390000000000001},"min":{"1min":0.61399999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.3319999999999999,"5min":11.714,"15min":11.714},"last":1.6679999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58376,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313464,"statfs":{"total":96636764160,"available":96576987136,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5390000000000001,"5min":1.5580000000000001,"15min":1.1830000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":18.399999999999999,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":2.7730000000000001,"5min":1.8500000000000001,"15min":1.349},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.795999999999999,"5min":21.047000000000001,"15min":21.047000000000001},"last":0.91000000000000003}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5499999999999998,"5min":1.736,"15min":1.304},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":12.372,"5min":17.777999999999999,"15min":17.777999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.367,"15min":1.179},"min":{"1min":0.498,"5min":0.48099999999999998,"15min":0.39500000000000002},"max":{"1min":11.002000000000001,"5min":11.002000000000001,"15min":11.002000000000001},"last":1.081}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.413,"15min":1.244},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":1.0309999999999999},{"interface":"front","average":{"1min":2.2599999999999998,"5min":1.72,"15min":1.488},"min":{"1min":0.63300000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":20.495000000000001,"5min":20.495000000000001,"15min":20.495000000000001},"last":0.98899999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4380000000000002,"5min":1.7310000000000001,"15min":1.4950000000000001},"min":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.63500000000000001},"max":{"1min":11.826000000000001,"5min":11.826000000000001,"15min":11.826000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":3.1379999999999999,"5min":1.9950000000000001,"15min":1.625},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.54600000000000004},"max":{"1min":12.439,"5min":12.439,"15min":12.439},"last":1.3500000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.867,"15min":1.6379999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.57299999999999995},"max":{"1min":20.606000000000002,"5min":20.606000000000002,"15min":20.606000000000002},"last":1.3740000000000001},{"interface":"front","average":{"1min":2.161,"5min":1.8120000000000001,"15min":1.6870000000000001},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.68000000000000005},"max":{"1min":11.603,"5min":11.603,"15min":11.603},"last":1.9019999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.0129999999999999,"5min":2.1059999999999999,"15min":1.823},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":12.416,"5min":12.416,"15min":12.416},"last":1.8540000000000001},{"interface":"front","average":{"1min":2.391,"5min":1.962,"15min":1.768},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":20.556000000000001,"5min":20.556000000000001,"15min":20.556000000000001},"last":1.8280000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.8460000000000001,"5min":2.331,"15min":1.8999999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.70699999999999996},"max":{"1min":28.843,"5min":28.843,"15min":28.843},"last":2.0019999999999998},{"interface":"front","average":{"1min":3.4940000000000002,"5min":2.3199999999999998,"15min":1.9219999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":15.143000000000001,"5min":15.143000000000001,"15min":15.143000000000001},"last":1.8029999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542251,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":884736,"data_stored":371564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.3889999999999998,"5min":8.3279999999999994,"15min":14.427},"last":0.748},{"interface":"front","average":{"1min":1.893,"5min":1.3979999999999999,"15min":1.1299999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.432},"max":{"1min":11.243,"5min":11.243,"15min":11.243},"last":1.46}]},{"osd":1,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.4139999999999999,"15min":1.357},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40300000000000002},"max":{"1min":4.2779999999999996,"5min":11.718,"15min":14.609},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.278,"15min":1.224},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":4.3449999999999998,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":1.0629999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6749999999999998,"5min":1.7050000000000001,"15min":1.3859999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.34799999999999998},"max":{"1min":12.93,"5min":19.535,"15min":19.535},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.657,"5min":1.589,"15min":1.518},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.2830000000000004,"5min":13.406000000000001,"15min":14.528},"last":0.999}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.6320000000000001,"5min":2.0880000000000001,"15min":1.8069999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":29.986999999999998,"5min":29.986999999999998,"15min":29.986999999999998},"last":1.347},{"interface":"front","average":{"1min":2.9140000000000001,"5min":1.9530000000000001,"15min":1.619},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":21.199999999999999,"5min":21.199999999999999,"15min":21.199999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.1960000000000002,"5min":1.8200000000000001,"15min":1.6220000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":12.227,"5min":19.681999999999999,"15min":19.681999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.7410000000000001,"15min":1.698},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":4.0519999999999996,"5min":10.108000000000001,"15min":14.702999999999999},"last":1.6200000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.726,"15min":1.5629999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.5839999999999996,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":2.0640000000000001,"5min":1.8720000000000001,"15min":1.631},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":12.176,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.6599999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":5.6159999999999997,"5min":2.5179999999999998,"15min":1.907},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":57.006999999999998,"5min":57.006999999999998,"15min":57.006999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":3.8799999999999999,"5min":2.5950000000000002,"15min":1.9690000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":21.795999999999999,"5min":21.795999999999999,"15min":21.795999999999999},"last":1.796}]}]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54220,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317620,"statfs":{"total":96636764160,"available":96581242880,"internally_reserved":0,"allocated":847872,"data_stored":356765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.27,"15min":1.0209999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42799999999999999},"max":{"1min":13.749000000000001,"5min":13.749000000000001,"15min":13.749000000000001},"last":0.751},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.157,"15min":1.0449999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":18.658000000000001,"5min":18.658000000000001,"15min":18.658000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.331,"15min":1.0329999999999999},"min":{"1min":0.48599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.3029999999999999,"5min":21.126000000000001,"15min":21.126000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.585,"15min":1.3109999999999999},"min":{"1min":0.52700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":9.7279999999999998,"5min":11.5,"15min":11.5},"last":1.0389999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7350000000000001,"5min":1.4279999999999999,"15min":1.855},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":13.593,"5min":13.593,"15min":22.106999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.2250000000000001,"15min":1.26},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.9610000000000003,"5min":7.9610000000000003,"15min":15.510999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.708,"15min":1.726},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72499999999999998},"max":{"1min":3.27,"5min":14.827,"15min":14.827},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.6839999999999999,"15min":1.8109999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3809999999999998,"5min":14.851000000000001,"15min":14.851000000000001},"last":1.4299999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":1.643,"15min":1.7829999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.0700000000000003,"5min":11.552,"15min":13.228999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":3.1400000000000001,"5min":1.929,"15min":1.8089999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.81399999999999995},"max":{"1min":18.452000000000002,"5min":18.452000000000002,"15min":20.823},"last":0.83699999999999997}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.0910000000000002,"5min":1.9890000000000001,"15min":2.0139999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":34.158000000000001,"5min":34.158000000000001,"15min":34.158000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.6200000000000001,"15min":1.534},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.52,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":1.1950000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6859999999999999,"5min":1.865,"15min":1.6639999999999999},"min":{"1min":0.84599999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":17.928999999999998,"5min":17.928999999999998,"15min":17.928999999999998},"last":1.109},{"interface":"front","average":{"1min":1.673,"5min":1.9470000000000001,"15min":1.6839999999999999},"min":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.9139999999999997,"5min":24.289999999999999,"15min":24.289999999999999},"last":1.7150000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542250,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54796,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317044,"statfs":{"total":96636764160,"available":96580653056,"internally_reserved":0,"allocated":1441792,"data_stored":947133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.0649999999999999,"15min":0.94199999999999995},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":6.1890000000000001,"5min":6.1890000000000001,"15min":6.1890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":1.806,"5min":1.413,"15min":1.1899999999999999},"min":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":11.470000000000001,"5min":11.470000000000001,"15min":11.470000000000001},"last":1.3959999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.333,"15min":1.117},"min":{"1min":0.45500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":11.59,"5min":11.59,"15min":11.59},"last":1.323},{"interface":"front","average":{"1min":2.7040000000000002,"5min":1.377,"15min":1.0309999999999999},"min":{"1min":0.41399999999999998,"5min":0.38200000000000001,"15min":0.35899999999999999},"max":{"1min":14.512,"5min":14.512,"15min":14.512},"last":1.2110000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5920000000000001,"15min":1.3720000000000001},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":11.544,"5min":11.544,"15min":11.544},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.518,"5min":1.446,"15min":1.194},"min":{"1min":0.41999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.1539999999999999,"5min":14.74,"15min":14.74},"last":0.96099999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.452,"5min":2.0379999999999998,"15min":1.532},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":32.115000000000002,"5min":32.115000000000002,"15min":32.115000000000002},"last":1.536},{"interface":"front","average":{"1min":2.2280000000000002,"5min":1.903,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":11.992000000000001,"5min":15.975,"15min":15.975},"last":1.4570000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.4870000000000001,"5min":2.1560000000000001,"15min":1.629},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":32.174999999999997,"5min":32.174999999999997,"15min":32.174999999999997},"last":1.369},{"interface":"front","average":{"1min":2.871,"5min":1.915,"15min":1.623},"min":{"1min":0.875,"5min":0.875,"15min":0.754},"max":{"1min":17.225000000000001,"5min":17.225000000000001,"15min":17.225000000000001},"last":1.4830000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.097,"5min":1.746,"15min":1.587},"min":{"1min":0.99199999999999999,"5min":0.96999999999999997,"15min":0.78000000000000003},"max":{"1min":6.1189999999999998,"5min":6.1189999999999998,"15min":6.1189999999999998},"last":1.4259999999999999},{"interface":"front","average":{"1min":4.3369999999999997,"5min":2.1120000000000001,"15min":1.669},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":32.216999999999999,"5min":32.216999999999999,"15min":32.216999999999999},"last":1.609}]},{"osd":7,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5350000000000001,"5min":2.5710000000000002,"15min":1.9550000000000001},"min":{"1min":1.036,"5min":0.98899999999999999,"15min":0.81299999999999994},"max":{"1min":34.131,"5min":34.131,"15min":34.131},"last":1.508},{"interface":"front","average":{"1min":4.6580000000000004,"5min":2.3799999999999999,"15min":1.8320000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.70999999999999996},"max":{"1min":21.884,"5min":21.884,"15min":21.884},"last":1.266}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:03:36.965 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:03:37.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.192+0000 7fcc52400640 1 Processor -- start 2024-01-15T18:03:37.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.192+0000 7fcc52400640 1 -- start start 2024-01-15T18:03:37.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.194+0000 7fcc52400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc4c160890 0x7fcc4c0d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.194+0000 7fcc52400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c160c70 0x7fcc4c0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.194+0000 7fcc52400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc4c0cf6a0 con 0x7fcc4c0c6be0 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.194+0000 7fcc52400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc4c0d1430 con 0x7fcc4c160890 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.194+0000 7fcc52400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0b4370 con 0x7fcc4c160c70 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.194+0000 7fcc51a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcc4c0c6be0 0x7fcc4c161110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43400/0 (socket says 172.21.15.26:43400) 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.194+0000 7fcc51a00640 1 -- 172.21.15.26:0/2356119606 learned_addr learned my addr 172.21.15.26:0/2356119606 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.194+0000 7fcc51000640 1 --2- 172.21.15.26:0/2356119606 >> v2:172.21.15.26:3300/0 conn(0x7fcc4c160890 0x7fcc4c0d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.195+0000 7fcc4be00640 1 --2- 172.21.15.26:0/2356119606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c160c70 0x7fcc4c0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.195+0000 7fcc51000640 1 -- 172.21.15.26:0/2356119606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c160c70 msgr2=0x7fcc4c0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.195+0000 7fcc51000640 1 --2- 172.21.15.26:0/2356119606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c160c70 0x7fcc4c0d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.195+0000 7fcc51000640 1 -- 172.21.15.26:0/2356119606 >> v1:172.21.15.26:6789/0 conn(0x7fcc4c0c6be0 legacy=0x7fcc4c161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.195+0000 7fcc51000640 1 -- 172.21.15.26:0/2356119606 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0ce7c0 con 0x7fcc4c160890 2024-01-15T18:03:37.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.195+0000 7fcc4b400640 1 -- 172.21.15.26:0/2356119606 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102640784 0 0) 0x7fcc4c0cf6a0 con 0x7fcc4c0c6be0 2024-01-15T18:03:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.196+0000 7fcc51000640 1 --2- 172.21.15.26:0/2356119606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 0x7fcc4c0d3bb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0097a0 tx=0x7fcc3c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb54e5ed1836d318 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.196+0000 7fcc4b400640 1 -- 172.21.15.26:0/2356119606 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0316f0 con 0x7fcc4c160890 2024-01-15T18:03:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.196+0000 7fcc4b400640 1 -- 172.21.15.26:0/2356119606 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3c031850 con 0x7fcc4c160890 2024-01-15T18:03:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.196+0000 7fcc4b400640 1 -- 172.21.15.26:0/2356119606 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c031be0 con 0x7fcc4c160890 2024-01-15T18:03:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.197+0000 7fcc52400640 1 -- 172.21.15.26:0/2356119606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 msgr2=0x7fcc4c0d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.197+0000 7fcc52400640 1 --2- 172.21.15.26:0/2356119606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 0x7fcc4c0d3bb0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0097a0 tx=0x7fcc3c02b790 comp rx=0 tx=0).stop 2024-01-15T18:03:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.197+0000 7fcc52400640 1 -- 172.21.15.26:0/2356119606 shutdown_connections 2024-01-15T18:03:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.197+0000 7fcc52400640 1 --2- 172.21.15.26:0/2356119606 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c160c70 0x7fcc4c0d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.197+0000 7fcc52400640 1 --2- 172.21.15.26:0/2356119606 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 0x7fcc4c0d3bb0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.197+0000 7fcc52400640 1 -- 172.21.15.26:0/2356119606 >> 172.21.15.26:0/2356119606 conn(0x7fcc4c0d3090 msgr2=0x7fcc4c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.197+0000 7fcc52400640 1 -- 172.21.15.26:0/2356119606 shutdown_connections 2024-01-15T18:03:37.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.197+0000 7fcc52400640 1 -- 172.21.15.26:0/2356119606 wait complete. 2024-01-15T18:03:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.199+0000 7fcc52400640 1 Processor -- start 2024-01-15T18:03:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.199+0000 7fcc52400640 1 -- start start 2024-01-15T18:03:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.200+0000 7fcc52400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c0c6be0 0x7fcc4c0b8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.200+0000 7fcc52400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 0x7fcc4c151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.200+0000 7fcc52400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc4c160c70 0x7fcc4c152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.200+0000 7fcc52400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c0cafb0 con 0x7fcc4c160890 2024-01-15T18:03:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.200+0000 7fcc52400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c152970 con 0x7fcc4c160c70 2024-01-15T18:03:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.200+0000 7fcc52400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c1f29b0 con 0x7fcc4c0c6be0 2024-01-15T18:03:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.200+0000 7fcc4be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 0x7fcc4c151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.200+0000 7fcc4be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 0x7fcc4c151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50592/0 (socket says 172.21.15.26:50592) 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.200+0000 7fcc4be00640 1 -- 172.21.15.26:0/3720359713 learned_addr learned my addr 172.21.15.26:0/3720359713 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.201+0000 7fcc51000640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c0c6be0 0x7fcc4c0b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.201+0000 7fcc51a00640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc4c160c70 0x7fcc4c152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.201+0000 7fcc4be00640 1 -- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c0c6be0 msgr2=0x7fcc4c0b8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.201+0000 7fcc4be00640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c0c6be0 0x7fcc4c0b8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.201+0000 7fcc4be00640 1 -- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc4c160c70 msgr2=0x7fcc4c152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.201+0000 7fcc4be00640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc4c160c70 0x7fcc4c152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.201+0000 7fcc4be00640 1 -- 172.21.15.26:0/3720359713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3c009400 con 0x7fcc4c160890 2024-01-15T18:03:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.201+0000 7fcc51a00640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc4c160c70 0x7fcc4c152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.201+0000 7fcc4be00640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 0x7fcc4c151ea0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400d800 tx=0x7fcc3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.202+0000 7fcc48c00640 1 -- 172.21.15.26:0/3720359713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400b6c0 con 0x7fcc4c160890 2024-01-15T18:03:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.202+0000 7fcc48c00640 1 -- 172.21.15.26:0/3720359713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3400b820 con 0x7fcc4c160890 2024-01-15T18:03:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.202+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc4c0b7d90 con 0x7fcc4c160890 2024-01-15T18:03:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.202+0000 7fcc48c00640 1 -- 172.21.15.26:0/3720359713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400bbb0 con 0x7fcc4c160890 2024-01-15T18:03:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.202+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4c0b77d0 con 0x7fcc4c160890 2024-01-15T18:03:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.203+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc4c067450 con 0x7fcc4c160890 2024-01-15T18:03:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.204+0000 7fcc48c00640 1 -- 172.21.15.26:0/3720359713 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fcc3400bd10 con 0x7fcc4c160890 2024-01-15T18:03:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.205+0000 7fcc48c00640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc2406bbc0 0x7fcc2406e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.205+0000 7fcc48c00640 1 -- 172.21.15.26:0/3720359713 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcc34088c20 con 0x7fcc4c160890 2024-01-15T18:03:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.206+0000 7fcc51000640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc2406bbc0 0x7fcc2406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.207+0000 7fcc51000640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc2406bbc0 0x7fcc2406e080 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c036000 tx=0x7fcc3c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:37.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.210+0000 7fcc48c00640 1 -- 172.21.15.26:0/3720359713 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc340552f0 con 0x7fcc4c160890 2024-01-15T18:03:37.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.401+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc4c0ced80 con 0x7fcc2406bbc0 2024-01-15T18:03:37.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.425+0000 7fcc48c00640 1 -- 172.21.15.26:0/3720359713 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286963 (secure 0 0 0) 0x7fcc3c0319c0 con 0x7fcc2406bbc0 2024-01-15T18:03:37.434 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:37.436 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:03:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc2406bbc0 msgr2=0x7fcc2406e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc2406bbc0 0x7fcc2406e080 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c036000 tx=0x7fcc3c008030 comp rx=0 tx=0).stop 2024-01-15T18:03:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 msgr2=0x7fcc4c151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 0x7fcc4c151ea0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcc3400d800 tx=0x7fcc3400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 shutdown_connections 2024-01-15T18:03:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc2406bbc0 0x7fcc2406e080 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc4c160c70 0x7fcc4c152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc4c160890 0x7fcc4c151ea0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 --2- 172.21.15.26:0/3720359713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc4c0c6be0 0x7fcc4c0b8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.431+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 >> 172.21.15.26:0/3720359713 conn(0x7fcc4c0d3090 msgr2=0x7fcc4c0c7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:37.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.432+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 shutdown_connections 2024-01-15T18:03:37.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.432+0000 7fcc52400640 1 -- 172.21.15.26:0/3720359713 wait complete. 2024-01-15T18:03:37.457 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-01-15T18:03:36.106646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410216,"kb_used_data":8040,"kb_used_omap":12,"kb_used_meta":402035,"kb_avail":754564504,"statfs":{"total":773094113280,"available":772674052096,"internally_reserved":0,"allocated":8232960,"data_stored":4145172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":411684400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.811445"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.530695+0000","last_change":"2024-01-15T18:03:27.336969+0000","last_active":"2024-01-15T18:03:28.530695+0000","last_peered":"2024-01-15T18:03:28.530695+0000","last_clean":"2024-01-15T18:03:28.530695+0000","last_became_active":"2024-01-15T18:03:27.334516+0000","last_became_peered":"2024-01-15T18:03:27.334516+0000","last_unstale":"2024-01-15T18:03:28.530695+0000","last_undegraded":"2024-01-15T18:03:28.530695+0000","last_fullsized":"2024-01-15T18:03:28.530695+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:41:36.086607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.535285+0000","last_change":"2024-01-15T18:03:27.341616+0000","last_active":"2024-01-15T18:03:28.535285+0000","last_peered":"2024-01-15T18:03:28.535285+0000","last_clean":"2024-01-15T18:03:28.535285+0000","last_became_active":"2024-01-15T18:03:27.341248+0000","last_became_peered":"2024-01-15T18:03:27.341248+0000","last_unstale":"2024-01-15T18:03:28.535285+0000","last_undegraded":"2024-01-15T18:03:28.535285+0000","last_fullsized":"2024-01-15T18:03:28.535285+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:28:35.401367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531788+0000","last_change":"2024-01-15T18:03:27.309873+0000","last_active":"2024-01-15T18:03:28.531788+0000","last_peered":"2024-01-15T18:03:28.531788+0000","last_clean":"2024-01-15T18:03:28.531788+0000","last_became_active":"2024-01-15T18:03:27.307909+0000","last_became_peered":"2024-01-15T18:03:27.307909+0000","last_unstale":"2024-01-15T18:03:28.531788+0000","last_undegraded":"2024-01-15T18:03:28.531788+0000","last_fullsized":"2024-01-15T18:03:28.531788+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:55:54.602826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.299189+0000","last_change":"2024-01-15T18:03:27.323946+0000","last_active":"2024-01-15T18:03:28.299189+0000","last_peered":"2024-01-15T18:03:28.299189+0000","last_clean":"2024-01-15T18:03:28.299189+0000","last_became_active":"2024-01-15T18:03:27.322492+0000","last_became_peered":"2024-01-15T18:03:27.322492+0000","last_unstale":"2024-01-15T18:03:28.299189+0000","last_undegraded":"2024-01-15T18:03:28.299189+0000","last_fullsized":"2024-01-15T18:03:28.299189+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:53:07.544787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302106+0000","last_change":"2024-01-15T18:03:27.319087+0000","last_active":"2024-01-15T18:03:28.302106+0000","last_peered":"2024-01-15T18:03:28.302106+0000","last_clean":"2024-01-15T18:03:28.302106+0000","last_became_active":"2024-01-15T18:03:27.318702+0000","last_became_peered":"2024-01-15T18:03:27.318702+0000","last_unstale":"2024-01-15T18:03:28.302106+0000","last_undegraded":"2024-01-15T18:03:28.302106+0000","last_fullsized":"2024-01-15T18:03:28.302106+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:13:57.340165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.533425+0000","last_change":"2024-01-15T18:03:27.319124+0000","last_active":"2024-01-15T18:03:28.533425+0000","last_peered":"2024-01-15T18:03:28.533425+0000","last_clean":"2024-01-15T18:03:28.533425+0000","last_became_active":"2024-01-15T18:03:27.318855+0000","last_became_peered":"2024-01-15T18:03:27.318855+0000","last_unstale":"2024-01-15T18:03:28.533425+0000","last_undegraded":"2024-01-15T18:03:28.533425+0000","last_fullsized":"2024-01-15T18:03:28.533425+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:43.531807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.534082+0000","last_change":"2024-01-15T18:03:27.332303+0000","last_active":"2024-01-15T18:03:28.534082+0000","last_peered":"2024-01-15T18:03:28.534082+0000","last_clean":"2024-01-15T18:03:28.534082+0000","last_became_active":"2024-01-15T18:03:27.330616+0000","last_became_peered":"2024-01-15T18:03:27.330616+0000","last_unstale":"2024-01-15T18:03:28.534082+0000","last_undegraded":"2024-01-15T18:03:28.534082+0000","last_fullsized":"2024-01-15T18:03:28.534082+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:58.855143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.532695+0000","last_change":"2024-01-15T18:03:27.316226+0000","last_active":"2024-01-15T18:03:28.532695+0000","last_peered":"2024-01-15T18:03:28.532695+0000","last_clean":"2024-01-15T18:03:28.532695+0000","last_became_active":"2024-01-15T18:03:27.315889+0000","last_became_peered":"2024-01-15T18:03:27.315889+0000","last_unstale":"2024-01-15T18:03:28.532695+0000","last_undegraded":"2024-01-15T18:03:28.532695+0000","last_fullsized":"2024-01-15T18:03:28.532695+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:51:21.001103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302728+0000","last_change":"2024-01-15T18:03:27.331363+0000","last_active":"2024-01-15T18:03:28.302728+0000","last_peered":"2024-01-15T18:03:28.302728+0000","last_clean":"2024-01-15T18:03:28.302728+0000","last_became_active":"2024-01-15T18:03:27.330530+0000","last_became_peered":"2024-01-15T18:03:27.330530+0000","last_unstale":"2024-01-15T18:03:28.302728+0000","last_undegraded":"2024-01-15T18:03:28.302728+0000","last_fullsized":"2024-01-15T18:03:28.302728+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:43:50.724930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.306485+0000","last_change":"2024-01-15T18:03:27.319885+0000","last_active":"2024-01-15T18:03:28.306485+0000","last_peered":"2024-01-15T18:03:28.306485+0000","last_clean":"2024-01-15T18:03:28.306485+0000","last_became_active":"2024-01-15T18:03:27.319578+0000","last_became_peered":"2024-01-15T18:03:27.319578+0000","last_unstale":"2024-01-15T18:03:28.306485+0000","last_undegraded":"2024-01-15T18:03:28.306485+0000","last_fullsized":"2024-01-15T18:03:28.306485+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:20:44.851079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.530707+0000","last_change":"2024-01-15T18:03:27.337891+0000","last_active":"2024-01-15T18:03:28.530707+0000","last_peered":"2024-01-15T18:03:28.530707+0000","last_clean":"2024-01-15T18:03:28.530707+0000","last_became_active":"2024-01-15T18:03:27.337229+0000","last_became_peered":"2024-01-15T18:03:27.337229+0000","last_unstale":"2024-01-15T18:03:28.530707+0000","last_undegraded":"2024-01-15T18:03:28.530707+0000","last_fullsized":"2024-01-15T18:03:28.530707+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:20:37.946245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.299399+0000","last_change":"2024-01-15T18:03:27.325360+0000","last_active":"2024-01-15T18:03:28.299399+0000","last_peered":"2024-01-15T18:03:28.299399+0000","last_clean":"2024-01-15T18:03:28.299399+0000","last_became_active":"2024-01-15T18:03:27.322920+0000","last_became_peered":"2024-01-15T18:03:27.322920+0000","last_unstale":"2024-01-15T18:03:28.299399+0000","last_undegraded":"2024-01-15T18:03:28.299399+0000","last_fullsized":"2024-01-15T18:03:28.299399+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:18:49.065777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.530445+0000","last_change":"2024-01-15T18:03:27.324610+0000","last_active":"2024-01-15T18:03:28.530445+0000","last_peered":"2024-01-15T18:03:28.530445+0000","last_clean":"2024-01-15T18:03:28.530445+0000","last_became_active":"2024-01-15T18:03:27.323109+0000","last_became_peered":"2024-01-15T18:03:27.323109+0000","last_unstale":"2024-01-15T18:03:28.530445+0000","last_undegraded":"2024-01-15T18:03:28.530445+0000","last_fullsized":"2024-01-15T18:03:28.530445+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:09.627478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302207+0000","last_change":"2024-01-15T18:03:27.329433+0000","last_active":"2024-01-15T18:03:28.302207+0000","last_peered":"2024-01-15T18:03:28.302207+0000","last_clean":"2024-01-15T18:03:28.302207+0000","last_became_active":"2024-01-15T18:03:27.329186+0000","last_became_peered":"2024-01-15T18:03:27.329186+0000","last_unstale":"2024-01-15T18:03:28.302207+0000","last_undegraded":"2024-01-15T18:03:28.302207+0000","last_fullsized":"2024-01-15T18:03:28.302207+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:34.881592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.529598+0000","last_change":"2024-01-15T18:03:27.318982+0000","last_active":"2024-01-15T18:03:28.529598+0000","last_peered":"2024-01-15T18:03:28.529598+0000","last_clean":"2024-01-15T18:03:28.529598+0000","last_became_active":"2024-01-15T18:03:27.318686+0000","last_became_peered":"2024-01-15T18:03:27.318686+0000","last_unstale":"2024-01-15T18:03:28.529598+0000","last_undegraded":"2024-01-15T18:03:28.529598+0000","last_fullsized":"2024-01-15T18:03:28.529598+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:25:20.997563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.305250+0000","last_change":"2024-01-15T18:03:27.322064+0000","last_active":"2024-01-15T18:03:28.305250+0000","last_peered":"2024-01-15T18:03:28.305250+0000","last_clean":"2024-01-15T18:03:28.305250+0000","last_became_active":"2024-01-15T18:03:27.321790+0000","last_became_peered":"2024-01-15T18:03:27.321790+0000","last_unstale":"2024-01-15T18:03:28.305250+0000","last_undegraded":"2024-01-15T18:03:28.305250+0000","last_fullsized":"2024-01-15T18:03:28.305250+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:35:34.719258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302977+0000","last_change":"2024-01-15T18:03:27.330530+0000","last_active":"2024-01-15T18:03:28.302977+0000","last_peered":"2024-01-15T18:03:28.302977+0000","last_clean":"2024-01-15T18:03:28.302977+0000","last_became_active":"2024-01-15T18:03:27.330149+0000","last_became_peered":"2024-01-15T18:03:27.330149+0000","last_unstale":"2024-01-15T18:03:28.302977+0000","last_undegraded":"2024-01-15T18:03:28.302977+0000","last_fullsized":"2024-01-15T18:03:28.302977+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:00:12.045333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.299741+0000","last_change":"2024-01-15T18:03:27.324694+0000","last_active":"2024-01-15T18:03:28.299741+0000","last_peered":"2024-01-15T18:03:28.299741+0000","last_clean":"2024-01-15T18:03:28.299741+0000","last_became_active":"2024-01-15T18:03:27.324359+0000","last_became_peered":"2024-01-15T18:03:27.324359+0000","last_unstale":"2024-01-15T18:03:28.299741+0000","last_undegraded":"2024-01-15T18:03:28.299741+0000","last_fullsized":"2024-01-15T18:03:28.299741+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:43:10.263099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.533968+0000","last_change":"2024-01-15T18:03:27.338146+0000","last_active":"2024-01-15T18:03:28.533968+0000","last_peered":"2024-01-15T18:03:28.533968+0000","last_clean":"2024-01-15T18:03:28.533968+0000","last_became_active":"2024-01-15T18:03:27.337850+0000","last_became_peered":"2024-01-15T18:03:27.337850+0000","last_unstale":"2024-01-15T18:03:28.533968+0000","last_undegraded":"2024-01-15T18:03:28.533968+0000","last_fullsized":"2024-01-15T18:03:28.533968+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:52:14.889033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.530433+0000","last_change":"2024-01-15T18:03:27.320637+0000","last_active":"2024-01-15T18:03:28.530433+0000","last_peered":"2024-01-15T18:03:28.530433+0000","last_clean":"2024-01-15T18:03:28.530433+0000","last_became_active":"2024-01-15T18:03:27.320332+0000","last_became_peered":"2024-01-15T18:03:27.320332+0000","last_unstale":"2024-01-15T18:03:28.530433+0000","last_undegraded":"2024-01-15T18:03:28.530433+0000","last_fullsized":"2024-01-15T18:03:28.530433+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:30:14.193710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.306216+0000","last_change":"2024-01-15T18:03:27.320193+0000","last_active":"2024-01-15T18:03:28.306216+0000","last_peered":"2024-01-15T18:03:28.306216+0000","last_clean":"2024-01-15T18:03:28.306216+0000","last_became_active":"2024-01-15T18:03:27.317523+0000","last_became_peered":"2024-01-15T18:03:27.317523+0000","last_unstale":"2024-01-15T18:03:28.306216+0000","last_undegraded":"2024-01-15T18:03:28.306216+0000","last_fullsized":"2024-01-15T18:03:28.306216+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:07:52.709438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.305508+0000","last_change":"2024-01-15T18:03:27.320752+0000","last_active":"2024-01-15T18:03:28.305508+0000","last_peered":"2024-01-15T18:03:28.305508+0000","last_clean":"2024-01-15T18:03:28.305508+0000","last_became_active":"2024-01-15T18:03:27.320440+0000","last_became_peered":"2024-01-15T18:03:27.320440+0000","last_unstale":"2024-01-15T18:03:28.305508+0000","last_undegraded":"2024-01-15T18:03:28.305508+0000","last_fullsized":"2024-01-15T18:03:28.305508+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:55:26.109036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.529675+0000","last_change":"2024-01-15T18:03:27.337221+0000","last_active":"2024-01-15T18:03:28.529675+0000","last_peered":"2024-01-15T18:03:28.529675+0000","last_clean":"2024-01-15T18:03:28.529675+0000","last_became_active":"2024-01-15T18:03:27.335855+0000","last_became_peered":"2024-01-15T18:03:27.335855+0000","last_unstale":"2024-01-15T18:03:28.529675+0000","last_undegraded":"2024-01-15T18:03:28.529675+0000","last_fullsized":"2024-01-15T18:03:28.529675+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:49:36.236688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.529398+0000","last_change":"2024-01-15T18:03:27.336987+0000","last_active":"2024-01-15T18:03:28.529398+0000","last_peered":"2024-01-15T18:03:28.529398+0000","last_clean":"2024-01-15T18:03:28.529398+0000","last_became_active":"2024-01-15T18:03:27.336189+0000","last_became_peered":"2024-01-15T18:03:27.336189+0000","last_unstale":"2024-01-15T18:03:28.529398+0000","last_undegraded":"2024-01-15T18:03:28.529398+0000","last_fullsized":"2024-01-15T18:03:28.529398+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:27:40.692623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.305939+0000","last_change":"2024-01-15T18:03:27.321164+0000","last_active":"2024-01-15T18:03:28.305939+0000","last_peered":"2024-01-15T18:03:28.305939+0000","last_clean":"2024-01-15T18:03:28.305939+0000","last_became_active":"2024-01-15T18:03:27.319400+0000","last_became_peered":"2024-01-15T18:03:27.319400+0000","last_unstale":"2024-01-15T18:03:28.305939+0000","last_undegraded":"2024-01-15T18:03:28.305939+0000","last_fullsized":"2024-01-15T18:03:28.305939+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:27:22.599869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.532428+0000","last_change":"2024-01-15T18:03:27.326655+0000","last_active":"2024-01-15T18:03:28.532428+0000","last_peered":"2024-01-15T18:03:28.532428+0000","last_clean":"2024-01-15T18:03:28.532428+0000","last_became_active":"2024-01-15T18:03:27.326363+0000","last_became_peered":"2024-01-15T18:03:27.326363+0000","last_unstale":"2024-01-15T18:03:28.532428+0000","last_undegraded":"2024-01-15T18:03:28.532428+0000","last_fullsized":"2024-01-15T18:03:28.532428+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:33:56.921000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531998+0000","last_change":"2024-01-15T18:03:27.320983+0000","last_active":"2024-01-15T18:03:28.531998+0000","last_peered":"2024-01-15T18:03:28.531998+0000","last_clean":"2024-01-15T18:03:28.531998+0000","last_became_active":"2024-01-15T18:03:27.320676+0000","last_became_peered":"2024-01-15T18:03:27.320676+0000","last_unstale":"2024-01-15T18:03:28.531998+0000","last_undegraded":"2024-01-15T18:03:28.531998+0000","last_fullsized":"2024-01-15T18:03:28.531998+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:46:02.254138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.306107+0000","last_change":"2024-01-15T18:03:27.318203+0000","last_active":"2024-01-15T18:03:28.306107+0000","last_peered":"2024-01-15T18:03:28.306107+0000","last_clean":"2024-01-15T18:03:28.306107+0000","last_became_active":"2024-01-15T18:03:27.317517+0000","last_became_peered":"2024-01-15T18:03:27.317517+0000","last_unstale":"2024-01-15T18:03:28.306107+0000","last_undegraded":"2024-01-15T18:03:28.306107+0000","last_fullsized":"2024-01-15T18:03:28.306107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:40:29.380948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531133+0000","last_change":"2024-01-15T18:03:27.318237+0000","last_active":"2024-01-15T18:03:28.531133+0000","last_peered":"2024-01-15T18:03:28.531133+0000","last_clean":"2024-01-15T18:03:28.531133+0000","last_became_active":"2024-01-15T18:03:27.317905+0000","last_became_peered":"2024-01-15T18:03:27.317905+0000","last_unstale":"2024-01-15T18:03:28.531133+0000","last_undegraded":"2024-01-15T18:03:28.531133+0000","last_fullsized":"2024-01-15T18:03:28.531133+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:44:16.008179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.533406+0000","last_change":"2024-01-15T18:03:27.324217+0000","last_active":"2024-01-15T18:03:28.533406+0000","last_peered":"2024-01-15T18:03:28.533406+0000","last_clean":"2024-01-15T18:03:28.533406+0000","last_became_active":"2024-01-15T18:03:27.323556+0000","last_became_peered":"2024-01-15T18:03:27.323556+0000","last_unstale":"2024-01-15T18:03:28.533406+0000","last_undegraded":"2024-01-15T18:03:28.533406+0000","last_fullsized":"2024-01-15T18:03:28.533406+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:58.335368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531834+0000","last_change":"2024-01-15T18:03:27.329407+0000","last_active":"2024-01-15T18:03:28.531834+0000","last_peered":"2024-01-15T18:03:28.531834+0000","last_clean":"2024-01-15T18:03:28.531834+0000","last_became_active":"2024-01-15T18:03:27.329060+0000","last_became_peered":"2024-01-15T18:03:27.329060+0000","last_unstale":"2024-01-15T18:03:28.531834+0000","last_undegraded":"2024-01-15T18:03:28.531834+0000","last_fullsized":"2024-01-15T18:03:28.531834+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:19:30.130452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.533097+0000","last_change":"2024-01-15T18:03:27.321263+0000","last_active":"2024-01-15T18:03:28.533097+0000","last_peered":"2024-01-15T18:03:28.533097+0000","last_clean":"2024-01-15T18:03:28.533097+0000","last_became_active":"2024-01-15T18:03:27.320958+0000","last_became_peered":"2024-01-15T18:03:27.320958+0000","last_unstale":"2024-01-15T18:03:28.533097+0000","last_undegraded":"2024-01-15T18:03:28.533097+0000","last_fullsized":"2024-01-15T18:03:28.533097+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:16:54.464013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.530586+0000","last_change":"2024-01-15T18:03:27.342749+0000","last_active":"2024-01-15T18:03:28.530586+0000","last_peered":"2024-01-15T18:03:28.530586+0000","last_clean":"2024-01-15T18:03:28.530586+0000","last_became_active":"2024-01-15T18:03:27.342393+0000","last_became_peered":"2024-01-15T18:03:27.342393+0000","last_unstale":"2024-01-15T18:03:28.530586+0000","last_undegraded":"2024-01-15T18:03:28.530586+0000","last_fullsized":"2024-01-15T18:03:28.530586+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:59:49.152783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"71'4","reported_seq":16,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.340079+0000","last_change":"2024-01-15T18:03:26.298227+0000","last_active":"2024-01-15T18:03:27.340079+0000","last_peered":"2024-01-15T18:03:27.340079+0000","last_clean":"2024-01-15T18:03:27.340079+0000","last_became_active":"2024-01-15T18:03:26.297015+0000","last_became_peered":"2024-01-15T18:03:26.297015+0000","last_unstale":"2024-01-15T18:03:27.340079+0000","last_undegraded":"2024-01-15T18:03:27.340079+0000","last_fullsized":"2024-01-15T18:03:27.340079+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:08.615694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302905+0000","last_change":"2024-01-15T18:03:27.330723+0000","last_active":"2024-01-15T18:03:28.302905+0000","last_peered":"2024-01-15T18:03:28.302905+0000","last_clean":"2024-01-15T18:03:28.302905+0000","last_became_active":"2024-01-15T18:03:27.330236+0000","last_became_peered":"2024-01-15T18:03:27.330236+0000","last_unstale":"2024-01-15T18:03:28.302905+0000","last_undegraded":"2024-01-15T18:03:28.302905+0000","last_fullsized":"2024-01-15T18:03:28.302905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:56.574101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.332054+0000","last_change":"2024-01-15T18:03:26.292579+0000","last_active":"2024-01-15T18:03:27.332054+0000","last_peered":"2024-01-15T18:03:27.332054+0000","last_clean":"2024-01-15T18:03:27.332054+0000","last_became_active":"2024-01-15T18:03:26.291631+0000","last_became_peered":"2024-01-15T18:03:26.291631+0000","last_unstale":"2024-01-15T18:03:27.332054+0000","last_undegraded":"2024-01-15T18:03:27.332054+0000","last_fullsized":"2024-01-15T18:03:27.332054+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:42.703538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.534089+0000","last_change":"2024-01-15T18:03:27.336001+0000","last_active":"2024-01-15T18:03:28.534089+0000","last_peered":"2024-01-15T18:03:28.534089+0000","last_clean":"2024-01-15T18:03:28.534089+0000","last_became_active":"2024-01-15T18:03:27.335265+0000","last_became_peered":"2024-01-15T18:03:27.335265+0000","last_unstale":"2024-01-15T18:03:28.534089+0000","last_undegraded":"2024-01-15T18:03:28.534089+0000","last_fullsized":"2024-01-15T18:03:28.534089+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:56:26.778892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.343044+0000","last_change":"2024-01-15T18:03:26.303968+0000","last_active":"2024-01-15T18:03:27.343044+0000","last_peered":"2024-01-15T18:03:27.343044+0000","last_clean":"2024-01-15T18:03:27.343044+0000","last_became_active":"2024-01-15T18:03:26.303669+0000","last_became_peered":"2024-01-15T18:03:26.303669+0000","last_unstale":"2024-01-15T18:03:27.343044+0000","last_undegraded":"2024-01-15T18:03:27.343044+0000","last_fullsized":"2024-01-15T18:03:27.343044+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:28:34.389583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531107+0000","last_change":"2024-01-15T18:03:27.338100+0000","last_active":"2024-01-15T18:03:28.531107+0000","last_peered":"2024-01-15T18:03:28.531107+0000","last_clean":"2024-01-15T18:03:28.531107+0000","last_became_active":"2024-01-15T18:03:27.337541+0000","last_became_peered":"2024-01-15T18:03:27.337541+0000","last_unstale":"2024-01-15T18:03:28.531107+0000","last_undegraded":"2024-01-15T18:03:28.531107+0000","last_fullsized":"2024-01-15T18:03:28.531107+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:37:56.538247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393150+0000","last_change":"2024-01-15T18:03:26.300990+0000","last_active":"2024-01-15T18:03:27.393150+0000","last_peered":"2024-01-15T18:03:27.393150+0000","last_clean":"2024-01-15T18:03:27.393150+0000","last_became_active":"2024-01-15T18:03:26.300735+0000","last_became_peered":"2024-01-15T18:03:26.300735+0000","last_unstale":"2024-01-15T18:03:27.393150+0000","last_undegraded":"2024-01-15T18:03:27.393150+0000","last_fullsized":"2024-01-15T18:03:27.393150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:53:06.533003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.533709+0000","last_change":"2024-01-15T18:03:27.333213+0000","last_active":"2024-01-15T18:03:28.533709+0000","last_peered":"2024-01-15T18:03:28.533709+0000","last_clean":"2024-01-15T18:03:28.533709+0000","last_became_active":"2024-01-15T18:03:27.332685+0000","last_became_peered":"2024-01-15T18:03:27.332685+0000","last_unstale":"2024-01-15T18:03:28.533709+0000","last_undegraded":"2024-01-15T18:03:28.533709+0000","last_fullsized":"2024-01-15T18:03:28.533709+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:30:14.193710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.334977+0000","last_change":"2024-01-15T18:03:26.296481+0000","last_active":"2024-01-15T18:03:27.334977+0000","last_peered":"2024-01-15T18:03:27.334977+0000","last_clean":"2024-01-15T18:03:27.334977+0000","last_became_active":"2024-01-15T18:03:26.293509+0000","last_became_peered":"2024-01-15T18:03:26.293509+0000","last_unstale":"2024-01-15T18:03:27.334977+0000","last_undegraded":"2024-01-15T18:03:27.334977+0000","last_fullsized":"2024-01-15T18:03:27.334977+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:11.811869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.306528+0000","last_change":"2024-01-15T18:03:27.321876+0000","last_active":"2024-01-15T18:03:28.306528+0000","last_peered":"2024-01-15T18:03:28.306528+0000","last_clean":"2024-01-15T18:03:28.306528+0000","last_became_active":"2024-01-15T18:03:27.321564+0000","last_became_peered":"2024-01-15T18:03:27.321564+0000","last_unstale":"2024-01-15T18:03:28.306528+0000","last_undegraded":"2024-01-15T18:03:28.306528+0000","last_fullsized":"2024-01-15T18:03:28.306528+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:19:43.352864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393068+0000","last_change":"2024-01-15T18:03:26.302584+0000","last_active":"2024-01-15T18:03:27.393068+0000","last_peered":"2024-01-15T18:03:27.393068+0000","last_clean":"2024-01-15T18:03:27.393068+0000","last_became_active":"2024-01-15T18:03:26.301356+0000","last_became_peered":"2024-01-15T18:03:26.301356+0000","last_unstale":"2024-01-15T18:03:27.393068+0000","last_undegraded":"2024-01-15T18:03:27.393068+0000","last_fullsized":"2024-01-15T18:03:27.393068+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:22:00.115460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.301672+0000","last_change":"2024-01-15T18:03:27.326042+0000","last_active":"2024-01-15T18:03:28.301672+0000","last_peered":"2024-01-15T18:03:28.301672+0000","last_clean":"2024-01-15T18:03:28.301672+0000","last_became_active":"2024-01-15T18:03:27.325476+0000","last_became_peered":"2024-01-15T18:03:27.325476+0000","last_unstale":"2024-01-15T18:03:28.301672+0000","last_undegraded":"2024-01-15T18:03:28.301672+0000","last_fullsized":"2024-01-15T18:03:28.301672+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:37:22.119881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.308394+0000","last_change":"2024-01-15T18:03:26.282542+0000","last_active":"2024-01-15T18:03:27.308394+0000","last_peered":"2024-01-15T18:03:27.308394+0000","last_clean":"2024-01-15T18:03:27.308394+0000","last_became_active":"2024-01-15T18:03:26.282226+0000","last_became_peered":"2024-01-15T18:03:26.282226+0000","last_unstale":"2024-01-15T18:03:27.308394+0000","last_undegraded":"2024-01-15T18:03:27.308394+0000","last_fullsized":"2024-01-15T18:03:27.308394+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:08:19.909004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.529805+0000","last_change":"2024-01-15T18:03:27.321337+0000","last_active":"2024-01-15T18:03:28.529805+0000","last_peered":"2024-01-15T18:03:28.529805+0000","last_clean":"2024-01-15T18:03:28.529805+0000","last_became_active":"2024-01-15T18:03:27.321050+0000","last_became_peered":"2024-01-15T18:03:27.321050+0000","last_unstale":"2024-01-15T18:03:28.529805+0000","last_undegraded":"2024-01-15T18:03:28.529805+0000","last_fullsized":"2024-01-15T18:03:28.529805+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:58.335368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.312237+0000","last_change":"2024-01-15T18:03:26.285864+0000","last_active":"2024-01-15T18:03:27.312237+0000","last_peered":"2024-01-15T18:03:27.312237+0000","last_clean":"2024-01-15T18:03:27.312237+0000","last_became_active":"2024-01-15T18:03:26.285507+0000","last_became_peered":"2024-01-15T18:03:26.285507+0000","last_unstale":"2024-01-15T18:03:27.312237+0000","last_undegraded":"2024-01-15T18:03:27.312237+0000","last_fullsized":"2024-01-15T18:03:27.312237+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:11:47.778037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.532751+0000","last_change":"2024-01-15T18:03:27.324261+0000","last_active":"2024-01-15T18:03:28.532751+0000","last_peered":"2024-01-15T18:03:28.532751+0000","last_clean":"2024-01-15T18:03:28.532751+0000","last_became_active":"2024-01-15T18:03:27.323709+0000","last_became_peered":"2024-01-15T18:03:27.323709+0000","last_unstale":"2024-01-15T18:03:28.532751+0000","last_undegraded":"2024-01-15T18:03:28.532751+0000","last_fullsized":"2024-01-15T18:03:28.532751+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:25:20.997563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.310100+0000","last_change":"2024-01-15T18:03:26.294412+0000","last_active":"2024-01-15T18:03:27.310100+0000","last_peered":"2024-01-15T18:03:27.310100+0000","last_clean":"2024-01-15T18:03:27.310100+0000","last_became_active":"2024-01-15T18:03:26.294070+0000","last_became_peered":"2024-01-15T18:03:26.294070+0000","last_unstale":"2024-01-15T18:03:27.310100+0000","last_undegraded":"2024-01-15T18:03:27.310100+0000","last_fullsized":"2024-01-15T18:03:27.310100+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:58:39.310996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531054+0000","last_change":"2024-01-15T18:03:27.343855+0000","last_active":"2024-01-15T18:03:28.531054+0000","last_peered":"2024-01-15T18:03:28.531054+0000","last_clean":"2024-01-15T18:03:28.531054+0000","last_became_active":"2024-01-15T18:03:27.341968+0000","last_became_peered":"2024-01-15T18:03:27.341968+0000","last_unstale":"2024-01-15T18:03:28.531054+0000","last_undegraded":"2024-01-15T18:03:28.531054+0000","last_fullsized":"2024-01-15T18:03:28.531054+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:52.274614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393192+0000","last_change":"2024-01-15T18:03:26.302162+0000","last_active":"2024-01-15T18:03:27.393192+0000","last_peered":"2024-01-15T18:03:27.393192+0000","last_clean":"2024-01-15T18:03:27.393192+0000","last_became_active":"2024-01-15T18:03:26.301821+0000","last_became_peered":"2024-01-15T18:03:26.301821+0000","last_unstale":"2024-01-15T18:03:27.393192+0000","last_undegraded":"2024-01-15T18:03:27.393192+0000","last_fullsized":"2024-01-15T18:03:27.393192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:24:07.021223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.534479+0000","last_change":"2024-01-15T18:03:27.329589+0000","last_active":"2024-01-15T18:03:28.534479+0000","last_peered":"2024-01-15T18:03:28.534479+0000","last_clean":"2024-01-15T18:03:28.534479+0000","last_became_active":"2024-01-15T18:03:27.328118+0000","last_became_peered":"2024-01-15T18:03:27.328118+0000","last_unstale":"2024-01-15T18:03:28.534479+0000","last_undegraded":"2024-01-15T18:03:28.534479+0000","last_fullsized":"2024-01-15T18:03:28.534479+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:43:38.804644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"71'3","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.336115+0000","last_change":"2024-01-15T18:03:26.295197+0000","last_active":"2024-01-15T18:03:27.336115+0000","last_peered":"2024-01-15T18:03:27.336115+0000","last_clean":"2024-01-15T18:03:27.336115+0000","last_became_active":"2024-01-15T18:03:26.294932+0000","last_became_peered":"2024-01-15T18:03:26.294932+0000","last_unstale":"2024-01-15T18:03:27.336115+0000","last_undegraded":"2024-01-15T18:03:27.336115+0000","last_fullsized":"2024-01-15T18:03:27.336115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:51:39.367823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302274+0000","last_change":"2024-01-15T18:03:27.315984+0000","last_active":"2024-01-15T18:03:28.302274+0000","last_peered":"2024-01-15T18:03:28.302274+0000","last_clean":"2024-01-15T18:03:28.302274+0000","last_became_active":"2024-01-15T18:03:27.315700+0000","last_became_peered":"2024-01-15T18:03:27.315700+0000","last_unstale":"2024-01-15T18:03:28.302274+0000","last_undegraded":"2024-01-15T18:03:28.302274+0000","last_fullsized":"2024-01-15T18:03:28.302274+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:47:16.050477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"71'6","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.337269+0000","last_change":"2024-01-15T18:03:26.283744+0000","last_active":"2024-01-15T18:03:27.337269+0000","last_peered":"2024-01-15T18:03:27.337269+0000","last_clean":"2024-01-15T18:03:27.337269+0000","last_became_active":"2024-01-15T18:03:26.283382+0000","last_became_peered":"2024-01-15T18:03:26.283382+0000","last_unstale":"2024-01-15T18:03:27.337269+0000","last_undegraded":"2024-01-15T18:03:27.337269+0000","last_fullsized":"2024-01-15T18:03:27.337269+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:47.498523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.529929+0000","last_change":"2024-01-15T18:03:27.335705+0000","last_active":"2024-01-15T18:03:28.529929+0000","last_peered":"2024-01-15T18:03:28.529929+0000","last_clean":"2024-01-15T18:03:28.529929+0000","last_became_active":"2024-01-15T18:03:27.333591+0000","last_became_peered":"2024-01-15T18:03:27.333591+0000","last_unstale":"2024-01-15T18:03:28.529929+0000","last_undegraded":"2024-01-15T18:03:28.529929+0000","last_fullsized":"2024-01-15T18:03:28.529929+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:41.605473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.331630+0000","last_change":"2024-01-15T18:03:26.290460+0000","last_active":"2024-01-15T18:03:27.331630+0000","last_peered":"2024-01-15T18:03:27.331630+0000","last_clean":"2024-01-15T18:03:27.331630+0000","last_became_active":"2024-01-15T18:03:26.290114+0000","last_became_peered":"2024-01-15T18:03:26.290114+0000","last_unstale":"2024-01-15T18:03:27.331630+0000","last_undegraded":"2024-01-15T18:03:27.331630+0000","last_fullsized":"2024-01-15T18:03:27.331630+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:15:51.847735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.533820+0000","last_change":"2024-01-15T18:03:27.326970+0000","last_active":"2024-01-15T18:03:28.533820+0000","last_peered":"2024-01-15T18:03:28.533820+0000","last_clean":"2024-01-15T18:03:28.533820+0000","last_became_active":"2024-01-15T18:03:27.326652+0000","last_became_peered":"2024-01-15T18:03:27.326652+0000","last_unstale":"2024-01-15T18:03:28.533820+0000","last_undegraded":"2024-01-15T18:03:28.533820+0000","last_fullsized":"2024-01-15T18:03:28.533820+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:14:09.627478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393079+0000","last_change":"2024-01-15T18:03:26.302531+0000","last_active":"2024-01-15T18:03:27.393079+0000","last_peered":"2024-01-15T18:03:27.393079+0000","last_clean":"2024-01-15T18:03:27.393079+0000","last_became_active":"2024-01-15T18:03:26.301660+0000","last_became_peered":"2024-01-15T18:03:26.301660+0000","last_unstale":"2024-01-15T18:03:27.393079+0000","last_undegraded":"2024-01-15T18:03:27.393079+0000","last_fullsized":"2024-01-15T18:03:27.393079+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:18:48.053993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.529544+0000","last_change":"2024-01-15T18:03:27.324641+0000","last_active":"2024-01-15T18:03:28.529544+0000","last_peered":"2024-01-15T18:03:28.529544+0000","last_clean":"2024-01-15T18:03:28.529544+0000","last_became_active":"2024-01-15T18:03:27.324343+0000","last_became_peered":"2024-01-15T18:03:27.324343+0000","last_unstale":"2024-01-15T18:03:28.529544+0000","last_undegraded":"2024-01-15T18:03:28.529544+0000","last_fullsized":"2024-01-15T18:03:28.529544+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:33:56.921000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.313678+0000","last_change":"2024-01-15T18:03:26.297958+0000","last_active":"2024-01-15T18:03:27.313678+0000","last_peered":"2024-01-15T18:03:27.313678+0000","last_clean":"2024-01-15T18:03:27.313678+0000","last_became_active":"2024-01-15T18:03:26.297417+0000","last_became_peered":"2024-01-15T18:03:26.297417+0000","last_unstale":"2024-01-15T18:03:27.313678+0000","last_undegraded":"2024-01-15T18:03:27.313678+0000","last_fullsized":"2024-01-15T18:03:27.313678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:36:43.508315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.301949+0000","last_change":"2024-01-15T18:03:27.330251+0000","last_active":"2024-01-15T18:03:28.301949+0000","last_peered":"2024-01-15T18:03:28.301949+0000","last_clean":"2024-01-15T18:03:28.301949+0000","last_became_active":"2024-01-15T18:03:27.329948+0000","last_became_peered":"2024-01-15T18:03:27.329948+0000","last_unstale":"2024-01-15T18:03:28.301949+0000","last_undegraded":"2024-01-15T18:03:28.301949+0000","last_fullsized":"2024-01-15T18:03:28.301949+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:42:58.573957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"71'8","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.341605+0000","last_change":"2024-01-15T18:03:26.287860+0000","last_active":"2024-01-15T18:03:27.341605+0000","last_peered":"2024-01-15T18:03:27.341605+0000","last_clean":"2024-01-15T18:03:27.341605+0000","last_became_active":"2024-01-15T18:03:26.287555+0000","last_became_peered":"2024-01-15T18:03:26.287555+0000","last_unstale":"2024-01-15T18:03:27.341605+0000","last_undegraded":"2024-01-15T18:03:27.341605+0000","last_fullsized":"2024-01-15T18:03:27.341605+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:18:18.016465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.306518+0000","last_change":"2024-01-15T18:03:27.318746+0000","last_active":"2024-01-15T18:03:28.306518+0000","last_peered":"2024-01-15T18:03:28.306518+0000","last_clean":"2024-01-15T18:03:28.306518+0000","last_became_active":"2024-01-15T18:03:27.317123+0000","last_became_peered":"2024-01-15T18:03:27.317123+0000","last_unstale":"2024-01-15T18:03:28.306518+0000","last_undegraded":"2024-01-15T18:03:28.306518+0000","last_fullsized":"2024-01-15T18:03:28.306518+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:33.995128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393204+0000","last_change":"2024-01-15T18:03:26.300342+0000","last_active":"2024-01-15T18:03:27.393204+0000","last_peered":"2024-01-15T18:03:27.393204+0000","last_clean":"2024-01-15T18:03:27.393204+0000","last_became_active":"2024-01-15T18:03:26.300100+0000","last_became_peered":"2024-01-15T18:03:26.300100+0000","last_unstale":"2024-01-15T18:03:27.393204+0000","last_undegraded":"2024-01-15T18:03:27.393204+0000","last_fullsized":"2024-01-15T18:03:27.393204+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:40:08.274277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531096+0000","last_change":"2024-01-15T18:03:27.339162+0000","last_active":"2024-01-15T18:03:28.531096+0000","last_peered":"2024-01-15T18:03:28.531096+0000","last_clean":"2024-01-15T18:03:28.531096+0000","last_became_active":"2024-01-15T18:03:27.338832+0000","last_became_peered":"2024-01-15T18:03:27.338832+0000","last_unstale":"2024-01-15T18:03:28.531096+0000","last_undegraded":"2024-01-15T18:03:28.531096+0000","last_fullsized":"2024-01-15T18:03:28.531096+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:46:02.536947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.343756+0000","last_change":"2024-01-15T18:03:26.298853+0000","last_active":"2024-01-15T18:03:27.343756+0000","last_peered":"2024-01-15T18:03:27.343756+0000","last_clean":"2024-01-15T18:03:27.343756+0000","last_became_active":"2024-01-15T18:03:26.298530+0000","last_became_peered":"2024-01-15T18:03:26.298530+0000","last_unstale":"2024-01-15T18:03:27.343756+0000","last_undegraded":"2024-01-15T18:03:27.343756+0000","last_fullsized":"2024-01-15T18:03:27.343756+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:43:37.792860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.299379+0000","last_change":"2024-01-15T18:03:27.324162+0000","last_active":"2024-01-15T18:03:28.299379+0000","last_peered":"2024-01-15T18:03:28.299379+0000","last_clean":"2024-01-15T18:03:28.299379+0000","last_became_active":"2024-01-15T18:03:27.323205+0000","last_became_peered":"2024-01-15T18:03:27.323205+0000","last_unstale":"2024-01-15T18:03:28.299379+0000","last_undegraded":"2024-01-15T18:03:28.299379+0000","last_fullsized":"2024-01-15T18:03:28.299379+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:22:01.127244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"71'1","reported_seq":13,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393169+0000","last_change":"2024-01-15T18:03:26.301874+0000","last_active":"2024-01-15T18:03:27.393169+0000","last_peered":"2024-01-15T18:03:27.393169+0000","last_clean":"2024-01-15T18:03:27.393169+0000","last_became_active":"2024-01-15T18:03:26.301425+0000","last_became_peered":"2024-01-15T18:03:26.301425+0000","last_unstale":"2024-01-15T18:03:27.393169+0000","last_undegraded":"2024-01-15T18:03:27.393169+0000","last_fullsized":"2024-01-15T18:03:27.393169+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:43.710985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302611+0000","last_change":"2024-01-15T18:03:27.325587+0000","last_active":"2024-01-15T18:03:28.302611+0000","last_peered":"2024-01-15T18:03:28.302611+0000","last_clean":"2024-01-15T18:03:28.302611+0000","last_became_active":"2024-01-15T18:03:27.325317+0000","last_became_peered":"2024-01-15T18:03:27.325317+0000","last_unstale":"2024-01-15T18:03:28.302611+0000","last_undegraded":"2024-01-15T18:03:28.302611+0000","last_fullsized":"2024-01-15T18:03:28.302611+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:18:27.777253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.326644+0000","last_change":"2024-01-15T18:03:26.295610+0000","last_active":"2024-01-15T18:03:27.326644+0000","last_peered":"2024-01-15T18:03:27.326644+0000","last_clean":"2024-01-15T18:03:27.326644+0000","last_became_active":"2024-01-15T18:03:26.295164+0000","last_became_peered":"2024-01-15T18:03:26.295164+0000","last_unstale":"2024-01-15T18:03:27.326644+0000","last_undegraded":"2024-01-15T18:03:27.326644+0000","last_fullsized":"2024-01-15T18:03:27.326644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:57.323584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531491+0000","last_change":"2024-01-15T18:03:27.309590+0000","last_active":"2024-01-15T18:03:28.531491+0000","last_peered":"2024-01-15T18:03:28.531491+0000","last_clean":"2024-01-15T18:03:28.531491+0000","last_became_active":"2024-01-15T18:03:27.309287+0000","last_became_peered":"2024-01-15T18:03:27.309287+0000","last_unstale":"2024-01-15T18:03:28.531491+0000","last_undegraded":"2024-01-15T18:03:28.531491+0000","last_fullsized":"2024-01-15T18:03:28.531491+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:57.291147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393095+0000","last_change":"2024-01-15T18:03:26.301741+0000","last_active":"2024-01-15T18:03:27.393095+0000","last_peered":"2024-01-15T18:03:27.393095+0000","last_clean":"2024-01-15T18:03:27.393095+0000","last_became_active":"2024-01-15T18:03:26.301228+0000","last_became_peered":"2024-01-15T18:03:26.301228+0000","last_unstale":"2024-01-15T18:03:27.393095+0000","last_undegraded":"2024-01-15T18:03:27.393095+0000","last_fullsized":"2024-01-15T18:03:27.393095+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:08.955644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.530215+0000","last_change":"2024-01-15T18:03:27.324124+0000","last_active":"2024-01-15T18:03:28.530215+0000","last_peered":"2024-01-15T18:03:28.530215+0000","last_clean":"2024-01-15T18:03:28.530215+0000","last_became_active":"2024-01-15T18:03:27.323813+0000","last_became_peered":"2024-01-15T18:03:27.323813+0000","last_unstale":"2024-01-15T18:03:28.530215+0000","last_undegraded":"2024-01-15T18:03:28.530215+0000","last_fullsized":"2024-01-15T18:03:28.530215+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:18:39.883710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.314006+0000","last_change":"2024-01-15T18:03:26.293020+0000","last_active":"2024-01-15T18:03:27.314006+0000","last_peered":"2024-01-15T18:03:27.314006+0000","last_clean":"2024-01-15T18:03:27.314006+0000","last_became_active":"2024-01-15T18:03:26.292663+0000","last_became_peered":"2024-01-15T18:03:26.292663+0000","last_unstale":"2024-01-15T18:03:27.314006+0000","last_undegraded":"2024-01-15T18:03:27.314006+0000","last_fullsized":"2024-01-15T18:03:27.314006+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:10:32.534847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531070+0000","last_change":"2024-01-15T18:03:27.323037+0000","last_active":"2024-01-15T18:03:28.531070+0000","last_peered":"2024-01-15T18:03:28.531070+0000","last_clean":"2024-01-15T18:03:28.531070+0000","last_became_active":"2024-01-15T18:03:27.322747+0000","last_became_peered":"2024-01-15T18:03:27.322747+0000","last_unstale":"2024-01-15T18:03:28.531070+0000","last_undegraded":"2024-01-15T18:03:28.531070+0000","last_fullsized":"2024-01-15T18:03:28.531070+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:43:30.009187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.299212+0000","last_change":"2024-01-15T18:03:26.292741+0000","last_active":"2024-01-15T18:03:27.299212+0000","last_peered":"2024-01-15T18:03:27.299212+0000","last_clean":"2024-01-15T18:03:27.299212+0000","last_became_active":"2024-01-15T18:03:26.292349+0000","last_became_peered":"2024-01-15T18:03:26.292349+0000","last_unstale":"2024-01-15T18:03:27.299212+0000","last_undegraded":"2024-01-15T18:03:27.299212+0000","last_fullsized":"2024-01-15T18:03:27.299212+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:19:29.118668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531014+0000","last_change":"2024-01-15T18:03:27.318860+0000","last_active":"2024-01-15T18:03:28.531014+0000","last_peered":"2024-01-15T18:03:28.531014+0000","last_clean":"2024-01-15T18:03:28.531014+0000","last_became_active":"2024-01-15T18:03:27.318508+0000","last_became_peered":"2024-01-15T18:03:27.318508+0000","last_unstale":"2024-01-15T18:03:28.531014+0000","last_undegraded":"2024-01-15T18:03:28.531014+0000","last_fullsized":"2024-01-15T18:03:28.531014+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:25:58.855143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393214+0000","last_change":"2024-01-15T18:03:26.299863+0000","last_active":"2024-01-15T18:03:27.393214+0000","last_peered":"2024-01-15T18:03:27.393214+0000","last_clean":"2024-01-15T18:03:27.393214+0000","last_became_active":"2024-01-15T18:03:26.299525+0000","last_became_peered":"2024-01-15T18:03:26.299525+0000","last_unstale":"2024-01-15T18:03:27.393214+0000","last_undegraded":"2024-01-15T18:03:27.393214+0000","last_fullsized":"2024-01-15T18:03:27.393214+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:35:51.148643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531615+0000","last_change":"2024-01-15T18:03:27.329884+0000","last_active":"2024-01-15T18:03:28.531615+0000","last_peered":"2024-01-15T18:03:28.531615+0000","last_clean":"2024-01-15T18:03:28.531615+0000","last_became_active":"2024-01-15T18:03:27.329221+0000","last_became_peered":"2024-01-15T18:03:27.329221+0000","last_unstale":"2024-01-15T18:03:28.531615+0000","last_undegraded":"2024-01-15T18:03:28.531615+0000","last_fullsized":"2024-01-15T18:03:28.531615+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:28.228411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"72'1","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.533065+0000","last_change":"2024-01-15T18:03:26.297682+0000","last_active":"2024-01-15T18:03:28.533065+0000","last_peered":"2024-01-15T18:03:28.533065+0000","last_clean":"2024-01-15T18:03:28.533065+0000","last_became_active":"2024-01-15T18:03:26.297319+0000","last_became_peered":"2024-01-15T18:03:26.297319+0000","last_unstale":"2024-01-15T18:03:28.533065+0000","last_undegraded":"2024-01-15T18:03:28.533065+0000","last_fullsized":"2024-01-15T18:03:28.533065+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:46:01.167230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531415+0000","last_change":"2024-01-15T18:03:27.341632+0000","last_active":"2024-01-15T18:03:28.531415+0000","last_peered":"2024-01-15T18:03:28.531415+0000","last_clean":"2024-01-15T18:03:28.531415+0000","last_became_active":"2024-01-15T18:03:27.341353+0000","last_became_peered":"2024-01-15T18:03:27.341353+0000","last_unstale":"2024-01-15T18:03:28.531415+0000","last_undegraded":"2024-01-15T18:03:28.531415+0000","last_fullsized":"2024-01-15T18:03:28.531415+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:53:09.368361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.311963+0000","last_change":"2024-01-15T18:03:26.290405+0000","last_active":"2024-01-15T18:03:27.311963+0000","last_peered":"2024-01-15T18:03:27.311963+0000","last_clean":"2024-01-15T18:03:27.311963+0000","last_became_active":"2024-01-15T18:03:26.288657+0000","last_became_peered":"2024-01-15T18:03:26.288657+0000","last_unstale":"2024-01-15T18:03:27.311963+0000","last_undegraded":"2024-01-15T18:03:27.311963+0000","last_fullsized":"2024-01-15T18:03:27.311963+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:56.868408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.531452+0000","last_change":"2024-01-15T18:03:27.338193+0000","last_active":"2024-01-15T18:03:28.531452+0000","last_peered":"2024-01-15T18:03:28.531452+0000","last_clean":"2024-01-15T18:03:28.531452+0000","last_became_active":"2024-01-15T18:03:27.337901+0000","last_became_peered":"2024-01-15T18:03:27.337901+0000","last_unstale":"2024-01-15T18:03:28.531452+0000","last_undegraded":"2024-01-15T18:03:28.531452+0000","last_fullsized":"2024-01-15T18:03:28.531452+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:40:10.272997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.298865+0000","last_change":"2024-01-15T18:03:26.296161+0000","last_active":"2024-01-15T18:03:27.298865+0000","last_peered":"2024-01-15T18:03:27.298865+0000","last_clean":"2024-01-15T18:03:27.298865+0000","last_became_active":"2024-01-15T18:03:26.295339+0000","last_became_peered":"2024-01-15T18:03:26.295339+0000","last_unstale":"2024-01-15T18:03:27.298865+0000","last_undegraded":"2024-01-15T18:03:27.298865+0000","last_fullsized":"2024-01-15T18:03:27.298865+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:27.216627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.302217+0000","last_change":"2024-01-15T18:03:27.330393+0000","last_active":"2024-01-15T18:03:28.302217+0000","last_peered":"2024-01-15T18:03:28.302217+0000","last_clean":"2024-01-15T18:03:28.302217+0000","last_became_active":"2024-01-15T18:03:27.329266+0000","last_became_peered":"2024-01-15T18:03:27.329266+0000","last_unstale":"2024-01-15T18:03:28.302217+0000","last_undegraded":"2024-01-15T18:03:28.302217+0000","last_fullsized":"2024-01-15T18:03:28.302217+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:54:50.174689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.311753+0000","last_change":"2024-01-15T18:03:26.296244+0000","last_active":"2024-01-15T18:03:27.311753+0000","last_peered":"2024-01-15T18:03:27.311753+0000","last_clean":"2024-01-15T18:03:27.311753+0000","last_became_active":"2024-01-15T18:03:26.295955+0000","last_became_peered":"2024-01-15T18:03:26.295955+0000","last_unstale":"2024-01-15T18:03:27.311753+0000","last_undegraded":"2024-01-15T18:03:27.311753+0000","last_fullsized":"2024-01-15T18:03:27.311753+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:44:34.308154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.534270+0000","last_change":"2024-01-15T18:03:27.323628+0000","last_active":"2024-01-15T18:03:28.534270+0000","last_peered":"2024-01-15T18:03:28.534270+0000","last_clean":"2024-01-15T18:03:28.534270+0000","last_became_active":"2024-01-15T18:03:27.323353+0000","last_became_peered":"2024-01-15T18:03:27.323353+0000","last_unstale":"2024-01-15T18:03:28.534270+0000","last_undegraded":"2024-01-15T18:03:28.534270+0000","last_fullsized":"2024-01-15T18:03:28.534270+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:14.612874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.308393+0000","last_change":"2024-01-15T18:03:26.295494+0000","last_active":"2024-01-15T18:03:27.308393+0000","last_peered":"2024-01-15T18:03:27.308393+0000","last_clean":"2024-01-15T18:03:27.308393+0000","last_became_active":"2024-01-15T18:03:26.295221+0000","last_became_peered":"2024-01-15T18:03:26.295221+0000","last_unstale":"2024-01-15T18:03:27.308393+0000","last_undegraded":"2024-01-15T18:03:27.308393+0000","last_fullsized":"2024-01-15T18:03:27.308393+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:21:47.980023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.298771+0000","last_change":"2024-01-15T18:03:27.313147+0000","last_active":"2024-01-15T18:03:28.298771+0000","last_peered":"2024-01-15T18:03:28.298771+0000","last_clean":"2024-01-15T18:03:28.298771+0000","last_became_active":"2024-01-15T18:03:27.312821+0000","last_became_peered":"2024-01-15T18:03:27.312821+0000","last_unstale":"2024-01-15T18:03:28.298771+0000","last_undegraded":"2024-01-15T18:03:28.298771+0000","last_fullsized":"2024-01-15T18:03:28.298771+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:42:40.231033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.302881+0000","last_change":"2024-01-15T18:03:26.291361+0000","last_active":"2024-01-15T18:03:27.302881+0000","last_peered":"2024-01-15T18:03:27.302881+0000","last_clean":"2024-01-15T18:03:27.302881+0000","last_became_active":"2024-01-15T18:03:26.290863+0000","last_became_peered":"2024-01-15T18:03:26.290863+0000","last_unstale":"2024-01-15T18:03:27.302881+0000","last_undegraded":"2024-01-15T18:03:27.302881+0000","last_fullsized":"2024-01-15T18:03:27.302881+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:59:42.520023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.532876+0000","last_change":"2024-01-15T18:03:27.325192+0000","last_active":"2024-01-15T18:03:28.532876+0000","last_peered":"2024-01-15T18:03:28.532876+0000","last_clean":"2024-01-15T18:03:28.532876+0000","last_became_active":"2024-01-15T18:03:27.324697+0000","last_became_peered":"2024-01-15T18:03:27.324697+0000","last_unstale":"2024-01-15T18:03:28.532876+0000","last_undegraded":"2024-01-15T18:03:28.532876+0000","last_fullsized":"2024-01-15T18:03:28.532876+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:18:19.028250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393035+0000","last_change":"2024-01-15T18:03:26.298754+0000","last_active":"2024-01-15T18:03:27.393035+0000","last_peered":"2024-01-15T18:03:27.393035+0000","last_clean":"2024-01-15T18:03:27.393035+0000","last_became_active":"2024-01-15T18:03:26.298412+0000","last_became_peered":"2024-01-15T18:03:26.298412+0000","last_unstale":"2024-01-15T18:03:27.393035+0000","last_undegraded":"2024-01-15T18:03:27.393035+0000","last_fullsized":"2024-01-15T18:03:27.393035+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:43:09.251315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-01-15T18:03:28.530929+0000","last_change":"2024-01-15T18:03:27.323613+0000","last_active":"2024-01-15T18:03:28.530929+0000","last_peered":"2024-01-15T18:03:28.530929+0000","last_clean":"2024-01-15T18:03:28.530929+0000","last_became_active":"2024-01-15T18:03:27.323339+0000","last_became_peered":"2024-01-15T18:03:27.323339+0000","last_unstale":"2024-01-15T18:03:28.530929+0000","last_undegraded":"2024-01-15T18:03:28.530929+0000","last_fullsized":"2024-01-15T18:03:28.530929+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:26.259097+0000","last_clean_scrub_stamp":"2024-01-15T18:03:26.259097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:18:06.032156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"71'2","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.393115+0000","last_change":"2024-01-15T18:03:26.300437+0000","last_active":"2024-01-15T18:03:27.393115+0000","last_peered":"2024-01-15T18:03:27.393115+0000","last_clean":"2024-01-15T18:03:27.393115+0000","last_became_active":"2024-01-15T18:03:26.300210+0000","last_became_peered":"2024-01-15T18:03:26.300210+0000","last_unstale":"2024-01-15T18:03:27.393115+0000","last_undegraded":"2024-01-15T18:03:27.393115+0000","last_fullsized":"2024-01-15T18:03:27.393115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:03:25.247313+0000","last_clean_scrub_stamp":"2024-01-15T18:03:25.247313+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:56:47.498523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":208,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-01-15T18:03:27.319985+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:03:27.319985+0000","last_peered":"2024-01-15T18:03:27.319985+0000","last_clean":"2024-01-15T18:03:27.319985+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:03:27.319985+0000","last_undegraded":"2024-01-15T18:03:27.319985+0000","last_fullsized":"2024-01-15T18:03:27.319985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542250,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50668,"kb_used_data":1440,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321172,"statfs":{"total":96636764160,"available":96584880128,"internally_reserved":0,"allocated":1474560,"data_stored":960936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.04,"5min":1.6439999999999999,"15min":1.3180000000000001},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":19.843,"5min":19.843,"15min":19.843},"last":0.83599999999999997},{"interface":"front","average":{"1min":4.2229999999999999,"5min":2.3180000000000001,"15min":1.552},"min":{"1min":0.75700000000000001,"5min":0.629,"15min":0.623},"max":{"1min":30.498000000000001,"5min":38.137,"15min":38.137},"last":0.53400000000000003}]},{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.129,"5min":1.716,"15min":1.383},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":19.917000000000002,"5min":19.917000000000002,"15min":19.917000000000002},"last":1.351},{"interface":"front","average":{"1min":2.2839999999999998,"5min":2.109,"15min":1.546},"min":{"1min":0.71199999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":10.361000000000001,"5min":51.843000000000004,"15min":51.843000000000004},"last":1.7989999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.3149999999999999,"5min":1.6910000000000001,"15min":1.345},"min":{"1min":0.50600000000000001,"5min":0.437,"15min":0.437},"max":{"1min":8.1289999999999996,"5min":9.3369999999999997,"15min":9.3369999999999997},"last":0.92800000000000005},{"interface":"front","average":{"1min":3.0760000000000001,"5min":1.8149999999999999,"15min":1.5169999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":13.478999999999999,"5min":13.478999999999999,"15min":13.478999999999999},"last":0.67300000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":3.3359999999999999,"5min":1.9450000000000001,"15min":1.5129999999999999},"min":{"1min":0.56000000000000005,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":16.632000000000001,"5min":16.632000000000001,"15min":16.632000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":3.177,"5min":1.867,"15min":1.617},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":13.439,"5min":13.439,"15min":13.439},"last":0.72499999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.5859999999999999,"5min":2.145,"15min":1.694},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.50700000000000001},"max":{"1min":10.881,"5min":22.359999999999999,"15min":22.359999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":4.4820000000000002,"5min":2.2149999999999999,"15min":1.702},"min":{"1min":0.52700000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":38.682000000000002,"5min":38.682000000000002,"15min":38.682000000000002},"last":1.1870000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.8140000000000001,"5min":2.286,"15min":1.831},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":13.379,"5min":25.006,"15min":25.006},"last":0.89000000000000001},{"interface":"front","average":{"1min":2.9169999999999998,"5min":2.2669999999999999,"15min":1.8069999999999999},"min":{"1min":0.80200000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":15.664999999999999,"5min":22.693999999999999,"15min":22.693999999999999},"last":1.71}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.4729999999999999,"5min":2.5,"15min":1.954},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.59299999999999997},"max":{"1min":11.366,"5min":23.834,"15min":23.834},"last":1.3},{"interface":"front","average":{"1min":2.8380000000000001,"5min":2.8050000000000002,"15min":2.012},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":9.2599999999999998,"5min":63.932000000000002,"15min":63.932000000000002},"last":1.1240000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542249,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45928,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325912,"statfs":{"total":96636764160,"available":96589733888,"internally_reserved":0,"allocated":880640,"data_stored":370044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.667,"5min":1.325,"15min":1.3340000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":7.79,"5min":7.79,"15min":12.544},"last":0.72299999999999998},{"interface":"front","average":{"1min":2.1379999999999999,"5min":1.579,"15min":1.3149999999999999},"min":{"1min":0.73899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":14.013,"5min":14.013,"15min":14.013},"last":0.77000000000000002}]},{"osd":1,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":1.881,"5min":1.403,"15min":1.284},"min":{"1min":0.67400000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":7.6200000000000001,"5min":7.6200000000000001,"15min":12.811999999999999},"last":1.123},{"interface":"front","average":{"1min":2.9430000000000001,"5min":1.8999999999999999,"15min":1.4199999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":20.523,"5min":20.523,"15min":20.523},"last":1.179}]},{"osd":2,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.0470000000000002,"5min":1.877,"15min":1.4279999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":17.713000000000001,"5min":17.713000000000001,"15min":17.713000000000001},"last":1.319},{"interface":"front","average":{"1min":1.788,"5min":1.3799999999999999,"15min":1.4850000000000001},"min":{"1min":0.59899999999999998,"5min":0.504,"15min":0.504},"max":{"1min":7.702,"5min":7.702,"15min":12.897},"last":0.97199999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.0840000000000001,"5min":1.74,"15min":1.6000000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":7.7480000000000002,"5min":9.5,"15min":12.717000000000001},"last":1.23},{"interface":"front","average":{"1min":2.3410000000000002,"5min":1.825,"15min":1.518},"min":{"1min":0.876,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":13.795,"5min":14.066000000000001,"15min":14.066000000000001},"last":1.2749999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":4.0739999999999998,"5min":2.0659999999999998,"15min":1.6200000000000001},"min":{"1min":0.59599999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":22.16,"5min":22.16,"15min":22.16},"last":1.4330000000000001},{"interface":"front","average":{"1min":2.2650000000000001,"5min":1.774,"15min":1.6890000000000001},"min":{"1min":0.52000000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":7.8319999999999999,"5min":17.289000000000001,"15min":17.289000000000001},"last":1.569}]},{"osd":5,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":2.9529999999999998,"5min":1.819,"15min":1.5620000000000001},"min":{"1min":0.66900000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":13.593,"5min":14.176,"15min":14.176},"last":1.393},{"interface":"front","average":{"1min":3.4950000000000001,"5min":2.1110000000000002,"15min":1.849},"min":{"1min":0.621,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":17.949000000000002,"5min":17.949000000000002,"15min":17.949000000000002},"last":1.518}]},{"osd":7,"last update":"Mon Jan 15 18:03:09 2024","interfaces":[{"interface":"back","average":{"1min":3.4359999999999999,"5min":2.0190000000000001,"15min":1.708},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":21.265000000000001,"5min":21.265000000000001,"15min":21.265000000000001},"last":1.482},{"interface":"front","average":{"1min":4.3949999999999996,"5min":2.3439999999999999,"15min":1.9299999999999999},"min":{"1min":0.65300000000000002,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":19.571000000000002,"5min":19.571000000000002,"15min":19.571000000000002},"last":1.629}]}]},{"osd":5,"up_from":14,"seq":60129542249,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45924,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325916,"statfs":{"total":96636764160,"available":96589737984,"internally_reserved":0,"allocated":876544,"data_stored":370042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9890000000000001,"5min":1.4279999999999999,"15min":1.284},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":19.266999999999999,"5min":19.266999999999999,"15min":19.266999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.607,"15min":1.4330000000000001},"min":{"1min":0.69399999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":3.1760000000000002,"5min":17.184999999999999,"15min":17.184999999999999},"last":1.0960000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":1.5469999999999999,"15min":1.3169999999999999},"min":{"1min":0.71299999999999997,"5min":0.499,"15min":0.499},"max":{"1min":9.1259999999999994,"5min":16.297000000000001,"15min":16.297000000000001},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.8899999999999999,"5min":1.3859999999999999,"15min":1.2629999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2379999999999995,"5min":9.2379999999999995,"15min":9.2379999999999995},"last":1.335}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.345,"15min":1.262},"min":{"1min":0.88600000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":4.0369999999999999,"5min":8.1820000000000004,"15min":8.1820000000000004},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.8220000000000001,"5min":1.6479999999999999,"15min":1.3899999999999999},"min":{"1min":0.88500000000000001,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":9.2159999999999993,"5min":16.158999999999999,"15min":16.158999999999999},"last":1.2849999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9299999999999999,"5min":1.5549999999999999,"15min":1.4370000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.56100000000000005},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":14.914},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.9930000000000001,"5min":1.526,"15min":1.4570000000000001},"min":{"1min":0.76300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":7.0110000000000001,"5min":7.0110000000000001,"15min":7.0110000000000001},"last":1.002}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9850000000000001,"5min":1.6579999999999999,"15min":1.4870000000000001},"min":{"1min":0.76100000000000001,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":9.0410000000000004,"5min":16.414999999999999,"15min":16.414999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.7350000000000001,"5min":1.925,"15min":1.627},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":9.1519999999999992,"5min":17.991,"15min":17.991},"last":1.2150000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6719999999999999,"5min":1.492,"15min":1.573},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":3.669,"5min":6.4630000000000001,"15min":6.4630000000000001},"last":1.254},{"interface":"front","average":{"1min":2.0609999999999999,"5min":1.788,"15min":1.623},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":9.1649999999999991,"5min":16.364000000000001,"15min":16.364000000000001},"last":1.429}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.9370000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"min":{"1min":0.66300000000000003,"5min":0.628,"15min":0.628},"max":{"1min":9.0600000000000005,"5min":9.0600000000000005,"15min":9.0600000000000005},"last":1.448},{"interface":"front","average":{"1min":3.0600000000000001,"5min":1.891,"15min":1.79},"min":{"1min":0.83999999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":19.356999999999999,"5min":19.356999999999999,"15min":19.356999999999999},"last":1.387}]}]},{"osd":4,"up_from":14,"seq":60129542249,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50088,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321752,"statfs":{"total":96636764160,"available":96585474048,"internally_reserved":0,"allocated":876544,"data_stored":370022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.5229999999999999,"15min":1.3149999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":7.8869999999999996,"5min":11.413,"15min":11.413},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.8180000000000001,"5min":1.534,"15min":1.401},"min":{"1min":0.63,"5min":0.63,"15min":0.53700000000000003},"max":{"1min":7.9160000000000004,"5min":9.7439999999999998,"15min":9.7439999999999998},"last":0.91500000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.367,"15min":1.369},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.45000000000000001},"max":{"1min":3.5750000000000002,"5min":9.7010000000000005,"15min":11.987},"last":1.081},{"interface":"front","average":{"1min":1.476,"5min":1.444,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":5.6109999999999998,"5min":16.058,"15min":16.058},"last":0.71899999999999997}]},{"osd":2,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.7789999999999999,"15min":1.575},"min":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.6890000000000001,"5min":13.865,"15min":13.865},"last":1.28},{"interface":"front","average":{"1min":1.788,"5min":1.833,"15min":1.5009999999999999},"min":{"1min":0.89100000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":5.7050000000000001,"5min":13.903,"15min":13.903},"last":0.86599999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.919,"5min":1.9099999999999999,"15min":1.675},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":11.773,"5min":26.055,"15min":26.055},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.6719999999999999,"5min":1.673,"15min":1.611},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":4.0419999999999998,"5min":13.25,"15min":13.25},"last":1.333}]},{"osd":5,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":2.9729999999999999,"5min":1.853,"15min":1.72},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":29.422000000000001,"5min":29.422000000000001,"15min":29.422000000000001},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.6679999999999999,"5min":1.5720000000000001,"15min":1.619},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":5.7160000000000002,"5min":9.6229999999999993,"15min":9.6229999999999993},"last":1.3979999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.623,"5min":1.6439999999999999,"15min":1.6379999999999999},"min":{"1min":0.80100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":5.7919999999999998,"5min":18.260000000000002,"15min":18.260000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.597,"5min":1.6890000000000001,"15min":1.7849999999999999},"min":{"1min":0.59999999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":3.6459999999999999,"5min":11.657999999999999,"15min":11.657999999999999},"last":1.55}]},{"osd":7,"last update":"Mon Jan 15 18:03:07 2024","interfaces":[{"interface":"back","average":{"1min":1.982,"5min":1.881,"15min":1.889},"min":{"1min":0.60499999999999998,"5min":0.51000000000000001,"15min":0.503},"max":{"1min":10.664,"5min":10.664,"15min":11.894},"last":1.8089999999999999},{"interface":"front","average":{"1min":1.851,"5min":2.0379999999999998,"15min":1.9390000000000001},"min":{"1min":0.61399999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":6.3319999999999999,"5min":11.714,"15min":11.714},"last":1.6519999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542251,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58416,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313424,"statfs":{"total":96636764160,"available":96576946176,"internally_reserved":0,"allocated":880640,"data_stored":370064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5390000000000001,"5min":1.5580000000000001,"15min":1.1830000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":18.399999999999999,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":2.7730000000000001,"5min":1.8500000000000001,"15min":1.349},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":18.795999999999999,"5min":21.047000000000001,"15min":21.047000000000001},"last":1.0980000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.5499999999999998,"5min":1.736,"15min":1.304},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":12.372,"5min":17.777999999999999,"15min":17.777999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.367,"15min":1.179},"min":{"1min":0.498,"5min":0.48099999999999998,"15min":0.39500000000000002},"max":{"1min":11.002000000000001,"5min":11.002000000000001,"15min":11.002000000000001},"last":1.006}]},{"osd":2,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":1.413,"15min":1.244},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":1.298},{"interface":"front","average":{"1min":2.2599999999999998,"5min":1.72,"15min":1.488},"min":{"1min":0.63300000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":20.495000000000001,"5min":20.495000000000001,"15min":20.495000000000001},"last":1.6140000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4380000000000002,"5min":1.7310000000000001,"15min":1.4950000000000001},"min":{"1min":0.84199999999999997,"5min":0.82799999999999996,"15min":0.63500000000000001},"max":{"1min":11.826000000000001,"5min":11.826000000000001,"15min":11.826000000000001},"last":1.1859999999999999},{"interface":"front","average":{"1min":3.1379999999999999,"5min":1.9950000000000001,"15min":1.625},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.54600000000000004},"max":{"1min":12.439,"5min":12.439,"15min":12.439},"last":1.3169999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":2.4940000000000002,"5min":1.867,"15min":1.6379999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.57299999999999995},"max":{"1min":20.606000000000002,"5min":20.606000000000002,"15min":20.606000000000002},"last":1.7190000000000001},{"interface":"front","average":{"1min":2.161,"5min":1.8120000000000001,"15min":1.6870000000000001},"min":{"1min":0.88800000000000001,"5min":0.88300000000000001,"15min":0.68000000000000005},"max":{"1min":11.603,"5min":11.603,"15min":11.603},"last":1.5189999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.0129999999999999,"5min":2.1059999999999999,"15min":1.823},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":12.416,"5min":12.416,"15min":12.416},"last":1.462},{"interface":"front","average":{"1min":2.391,"5min":1.962,"15min":1.768},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":20.556000000000001,"5min":20.556000000000001,"15min":20.556000000000001},"last":1.79}]},{"osd":7,"last update":"Mon Jan 15 18:03:11 2024","interfaces":[{"interface":"back","average":{"1min":3.8460000000000001,"5min":2.331,"15min":1.8999999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.70699999999999996},"max":{"1min":28.843,"5min":28.843,"15min":28.843},"last":1.355},{"interface":"front","average":{"1min":3.4940000000000002,"5min":2.3199999999999998,"15min":1.9219999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":15.143000000000001,"5min":15.143000000000001,"15min":15.143000000000001},"last":1.6830000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542251,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50100,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321740,"statfs":{"total":96636764160,"available":96585461760,"internally_reserved":0,"allocated":884736,"data_stored":371564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":1.0720000000000001,"15min":1.0620000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":2.3889999999999998,"5min":8.3279999999999994,"15min":14.427},"last":0.748},{"interface":"front","average":{"1min":1.893,"5min":1.3979999999999999,"15min":1.1299999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.432},"max":{"1min":11.243,"5min":11.243,"15min":11.243},"last":1.46}]},{"osd":1,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.4139999999999999,"15min":1.357},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40300000000000002},"max":{"1min":4.2779999999999996,"5min":11.718,"15min":14.609},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.278,"15min":1.224},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":4.3449999999999998,"5min":4.3449999999999998,"15min":4.3449999999999998},"last":1.0629999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6749999999999998,"5min":1.7050000000000001,"15min":1.3859999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.34799999999999998},"max":{"1min":12.93,"5min":19.535,"15min":19.535},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.657,"5min":1.589,"15min":1.518},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":4.2830000000000004,"5min":13.406000000000001,"15min":14.528},"last":0.999}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.6320000000000001,"5min":2.0880000000000001,"15min":1.8069999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":29.986999999999998,"5min":29.986999999999998,"15min":29.986999999999998},"last":1.347},{"interface":"front","average":{"1min":2.9140000000000001,"5min":1.9530000000000001,"15min":1.619},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":21.199999999999999,"5min":21.199999999999999,"15min":21.199999999999999},"last":1.2070000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.1960000000000002,"5min":1.8200000000000001,"15min":1.6220000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":12.227,"5min":19.681999999999999,"15min":19.681999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.7410000000000001,"15min":1.698},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":4.0519999999999996,"5min":10.108000000000001,"15min":14.702999999999999},"last":1.6200000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7210000000000001,"5min":1.726,"15min":1.5629999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":4.5839999999999996,"5min":6.5830000000000002,"15min":6.5830000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":2.0640000000000001,"5min":1.8720000000000001,"15min":1.631},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":12.176,"5min":19.783000000000001,"15min":19.783000000000001},"last":1.6599999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":5.6159999999999997,"5min":2.5179999999999998,"15min":1.907},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":57.006999999999998,"5min":57.006999999999998,"15min":57.006999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":3.8799999999999999,"5min":2.5950000000000002,"15min":1.9690000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":21.795999999999999,"5min":21.795999999999999,"15min":21.795999999999999},"last":1.796}]}]},{"osd":1,"up_from":14,"seq":60129542250,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54264,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317576,"statfs":{"total":96636764160,"available":96581197824,"internally_reserved":0,"allocated":892928,"data_stored":372180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.0110000000000001,"5min":1.27,"15min":1.0209999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42799999999999999},"max":{"1min":13.749000000000001,"5min":13.749000000000001,"15min":13.749000000000001},"last":1.208},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.157,"15min":1.0449999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":18.658000000000001,"5min":18.658000000000001,"15min":18.658000000000001},"last":1.6379999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.331,"15min":1.0329999999999999},"min":{"1min":0.48599999999999999,"5min":0.376,"15min":0.376},"max":{"1min":1.3029999999999999,"5min":21.126000000000001,"15min":21.126000000000001},"last":1.49},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.585,"15min":1.3109999999999999},"min":{"1min":0.52700000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":9.7279999999999998,"5min":11.5,"15min":11.5},"last":1.728}]},{"osd":3,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.7350000000000001,"5min":1.4279999999999999,"15min":1.855},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":13.593,"5min":13.593,"15min":22.106999999999999},"last":2.0939999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.2250000000000001,"15min":1.26},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":7.9610000000000003,"5min":7.9610000000000003,"15min":15.510999999999999},"last":2.069}]},{"osd":4,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.708,"15min":1.726},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72499999999999998},"max":{"1min":3.27,"5min":14.827,"15min":14.827},"last":2.2330000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.6839999999999999,"15min":1.8109999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.3809999999999998,"5min":14.851000000000001,"15min":14.851000000000001},"last":2.2589999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":1.643,"15min":1.7829999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":6.0700000000000003,"5min":11.552,"15min":13.228999999999999},"last":2.4700000000000002},{"interface":"front","average":{"1min":3.1400000000000001,"5min":1.929,"15min":1.8089999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.81399999999999995},"max":{"1min":18.452000000000002,"5min":18.452000000000002,"15min":20.823},"last":2.5179999999999998}]},{"osd":6,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":3.0910000000000002,"5min":1.9890000000000001,"15min":2.0139999999999998},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":34.158000000000001,"5min":34.158000000000001,"15min":34.158000000000001},"last":2.363},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.6200000000000001,"15min":1.534},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.52,"5min":8.8320000000000007,"15min":8.8320000000000007},"last":2.6680000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:03:12 2024","interfaces":[{"interface":"back","average":{"1min":2.6859999999999999,"5min":1.865,"15min":1.6639999999999999},"min":{"1min":0.84599999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":17.928999999999998,"5min":17.928999999999998,"15min":17.928999999999998},"last":2.4159999999999999},{"interface":"front","average":{"1min":1.673,"5min":1.9470000000000001,"15min":1.6839999999999999},"min":{"1min":0.70299999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":4.9139999999999997,"5min":24.289999999999999,"15min":24.289999999999999},"last":2.7069999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54828,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317012,"statfs":{"total":96636764160,"available":96580620288,"internally_reserved":0,"allocated":1466368,"data_stored":960320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.0649999999999999,"15min":0.94199999999999995},"min":{"1min":0.32400000000000001,"5min":0.312,"15min":0.312},"max":{"1min":6.1890000000000001,"5min":6.1890000000000001,"15min":6.1890000000000001},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.806,"5min":1.413,"15min":1.1899999999999999},"min":{"1min":0.40000000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":11.470000000000001,"5min":11.470000000000001,"15min":11.470000000000001},"last":0.80300000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.333,"15min":1.117},"min":{"1min":0.45500000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":11.59,"5min":11.59,"15min":11.59},"last":1.4119999999999999},{"interface":"front","average":{"1min":2.7040000000000002,"5min":1.377,"15min":1.0309999999999999},"min":{"1min":0.41399999999999998,"5min":0.38200000000000001,"15min":0.35899999999999999},"max":{"1min":14.512,"5min":14.512,"15min":14.512},"last":0.73499999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.2799999999999998,"5min":1.5920000000000001,"15min":1.3720000000000001},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":11.544,"5min":11.544,"15min":11.544},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.518,"5min":1.446,"15min":1.194},"min":{"1min":0.41999999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":6.1539999999999999,"5min":14.74,"15min":14.74},"last":1.0649999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.452,"5min":2.0379999999999998,"15min":1.532},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":32.115000000000002,"5min":32.115000000000002,"15min":32.115000000000002},"last":1.141},{"interface":"front","average":{"1min":2.2280000000000002,"5min":1.903,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":11.992000000000001,"5min":15.975,"15min":15.975},"last":1.534}]},{"osd":5,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":4.4870000000000001,"5min":2.1560000000000001,"15min":1.629},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":32.174999999999997,"5min":32.174999999999997,"15min":32.174999999999997},"last":1.0229999999999999},{"interface":"front","average":{"1min":2.871,"5min":1.915,"15min":1.623},"min":{"1min":0.875,"5min":0.875,"15min":0.754},"max":{"1min":17.225000000000001,"5min":17.225000000000001,"15min":17.225000000000001},"last":1.4830000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":2.097,"5min":1.746,"15min":1.587},"min":{"1min":0.99199999999999999,"5min":0.96999999999999997,"15min":0.78000000000000003},"max":{"1min":6.1189999999999998,"5min":6.1189999999999998,"15min":6.1189999999999998},"last":1.593},{"interface":"front","average":{"1min":4.3369999999999997,"5min":2.1120000000000001,"15min":1.669},"min":{"1min":0.67800000000000005,"5min":0.625,"15min":0.625},"max":{"1min":32.216999999999999,"5min":32.216999999999999,"15min":32.216999999999999},"last":1.2789999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:03:13 2024","interfaces":[{"interface":"back","average":{"1min":5.5350000000000001,"5min":2.5710000000000002,"15min":1.9550000000000001},"min":{"1min":1.036,"5min":0.98899999999999999,"15min":0.81299999999999994},"max":{"1min":34.131,"5min":34.131,"15min":34.131},"last":1.6419999999999999},{"interface":"front","average":{"1min":4.6580000000000004,"5min":2.3799999999999999,"15min":1.8320000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.70999999999999996},"max":{"1min":21.884,"5min":21.884,"15min":21.884},"last":1.7090000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:03:37.462 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:03:37.462 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:03:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.730+0000 7f0e80800640 1 Processor -- start 2024-01-15T18:03:37.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.730+0000 7f0e80800640 1 -- start start 2024-01-15T18:03:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.732+0000 7f0e80800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e78143960 0x7f0e78070760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.732+0000 7f0e80800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e7806daf0 0x7f0e780ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.732+0000 7f0e80800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e78143d40 con 0x7f0e780b3080 2024-01-15T18:03:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.733+0000 7f0e80800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e780b3a20 con 0x7f0e78143960 2024-01-15T18:03:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.733+0000 7f0e80800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780b3d00 con 0x7f0e7806daf0 2024-01-15T18:03:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.733+0000 7f0e7d400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0e780b3080 0x7f0e7806d3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43402/0 (socket says 172.21.15.26:43402) 2024-01-15T18:03:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.733+0000 7f0e7d400640 1 -- 172.21.15.26:0/3976358115 learned_addr learned my addr 172.21.15.26:0/3976358115 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:37.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.733+0000 7f0e7de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e78143960 0x7f0e78070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.733+0000 7f0e7ca00640 1 -- 172.21.15.26:0/3976358115 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243617038 0 0) 0x7f0e78143d40 con 0x7f0e780b3080 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.733+0000 7f0e7e800640 1 --2- 172.21.15.26:0/3976358115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e7806daf0 0x7f0e780ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.733+0000 7f0e7ca00640 1 -- 172.21.15.26:0/3976358115 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e58003520 con 0x7f0e780b3080 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.734+0000 7f0e7de00640 1 -- 172.21.15.26:0/3976358115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e7806daf0 msgr2=0x7f0e780ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.734+0000 7f0e7de00640 1 --2- 172.21.15.26:0/3976358115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e7806daf0 0x7f0e780ce030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.734+0000 7f0e7ca00640 1 -- 172.21.15.26:0/3976358115 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 549483125 0 0) 0x7f0e58003520 con 0x7f0e780b3080 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.734+0000 7f0e7de00640 1 -- 172.21.15.26:0/3976358115 >> v1:172.21.15.26:6789/0 conn(0x7f0e780b3080 legacy=0x7f0e7806d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.734+0000 7f0e7de00640 1 -- 172.21.15.26:0/3976358115 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780b3fe0 con 0x7f0e78143960 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.735+0000 7f0e7de00640 1 --2- 172.21.15.26:0/3976358115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e78143960 0x7f0e78070760 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097a0 tx=0x7f0e6802b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6340f35aed4f55b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.735+0000 7f0e7ca00640 1 -- 172.21.15.26:0/3976358115 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60001670 con 0x7f0e78143960 2024-01-15T18:03:37.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.735+0000 7f0e7ca00640 1 -- 172.21.15.26:0/3976358115 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6802e050 con 0x7f0e78143960 2024-01-15T18:03:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.735+0000 7f0e7ca00640 1 -- 172.21.15.26:0/3976358115 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68030730 con 0x7f0e78143960 2024-01-15T18:03:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.736+0000 7f0e80800640 1 -- 172.21.15.26:0/3976358115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e78143960 msgr2=0x7f0e78070760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.736+0000 7f0e80800640 1 --2- 172.21.15.26:0/3976358115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e78143960 0x7f0e78070760 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f0e680097a0 tx=0x7f0e6802b710 comp rx=0 tx=0).stop 2024-01-15T18:03:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.736+0000 7f0e80800640 1 -- 172.21.15.26:0/3976358115 shutdown_connections 2024-01-15T18:03:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.736+0000 7f0e80800640 1 --2- 172.21.15.26:0/3976358115 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e7806daf0 0x7f0e780ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.736+0000 7f0e80800640 1 --2- 172.21.15.26:0/3976358115 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e78143960 0x7f0e78070760 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.736+0000 7f0e80800640 1 -- 172.21.15.26:0/3976358115 >> 172.21.15.26:0/3976358115 conn(0x7f0e780b0b70 msgr2=0x7f0e780b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.736+0000 7f0e80800640 1 -- 172.21.15.26:0/3976358115 shutdown_connections 2024-01-15T18:03:37.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.736+0000 7f0e80800640 1 -- 172.21.15.26:0/3976358115 wait complete. 2024-01-15T18:03:37.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.738+0000 7f0e80800640 1 Processor -- start 2024-01-15T18:03:37.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.738+0000 7f0e80800640 1 -- start start 2024-01-15T18:03:37.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.739+0000 7f0e80800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7806daf0 0x7f0e78141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.739+0000 7f0e80800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e780b3080 0x7f0e7813b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.739+0000 7f0e80800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e78143960 0x7f0e781576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.739+0000 7f0e80800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e781421c0 con 0x7f0e7806daf0 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.739+0000 7f0e80800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7813b740 con 0x7f0e780b3080 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.739+0000 7f0e80800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e781423d0 con 0x7f0e78143960 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.739+0000 7f0e7de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7806daf0 0x7f0e78141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.739+0000 7f0e7de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7806daf0 0x7f0e78141c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50608/0 (socket says 172.21.15.26:50608) 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.739+0000 7f0e7de00640 1 -- 172.21.15.26:0/3489636668 learned_addr learned my addr 172.21.15.26:0/3489636668 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e7d400640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e780b3080 0x7f0e7813b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e7e800640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e78143960 0x7f0e781576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e7de00640 1 -- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e78143960 msgr2=0x7f0e781576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e7de00640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e78143960 0x7f0e781576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e7de00640 1 -- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e780b3080 msgr2=0x7f0e7813b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e7de00640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e780b3080 0x7f0e7813b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e7de00640 1 -- 172.21.15.26:0/3489636668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e68009400 con 0x7f0e7806daf0 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e7e800640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e78143960 0x7f0e781576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e7de00640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7806daf0 0x7f0e78141c80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0e680098c0 tx=0x7f0e680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e76000640 1 -- 172.21.15.26:0/3489636668 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6803b8d0 con 0x7f0e7806daf0 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e76000640 1 -- 172.21.15.26:0/3489636668 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6803ba70 con 0x7f0e7806daf0 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e76000640 1 -- 172.21.15.26:0/3489636668 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6803be20 con 0x7f0e7806daf0 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.740+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e781426b0 con 0x7f0e7806daf0 2024-01-15T18:03:37.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.741+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e78142a40 con 0x7f0e7806daf0 2024-01-15T18:03:37.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.742+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e780675d0 con 0x7f0e7806daf0 2024-01-15T18:03:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.743+0000 7f0e76000640 1 -- 172.21.15.26:0/3489636668 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f0e68039040 con 0x7f0e7806daf0 2024-01-15T18:03:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.744+0000 7f0e76000640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e5006bbc0 0x7f0e5006e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.744+0000 7f0e76000640 1 -- 172.21.15.26:0/3489636668 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0e680ac6a0 con 0x7f0e7806daf0 2024-01-15T18:03:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.744+0000 7f0e7d400640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e5006bbc0 0x7f0e5006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:37.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.746+0000 7f0e7d400640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e5006bbc0 0x7f0e5006e080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001eb0 tx=0x7f0e60009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:37.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:37.748+0000 7f0e76000640 1 -- 172.21.15.26:0/3489636668 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e680765b0 con 0x7f0e7806daf0 2024-01-15T18:03:38.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.013+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0e780cd730 con 0x7f0e7806daf0 2024-01-15T18:03:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.018+0000 7f0e76000640 1 -- 172.21.15.26:0/3489636668 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0e68037020 con 0x7f0e7806daf0 2024-01-15T18:03:38.022 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-15T18:03:38.022 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA== 2024-01-15T18:03:38.023 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-01-15T18:03:38.023 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-01-15T18:03:38.023 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:03:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-01-15T18:03:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e5006bbc0 msgr2=0x7f0e5006e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e5006bbc0 0x7f0e5006e080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0e60001eb0 tx=0x7f0e60009040 comp rx=0 tx=0).stop 2024-01-15T18:03:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7806daf0 msgr2=0x7f0e78141c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7806daf0 0x7f0e78141c80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0e680098c0 tx=0x7f0e680046c0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 shutdown_connections 2024-01-15T18:03:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e78143960 0x7f0e781576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e5006bbc0 0x7f0e5006e080 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e780b3080 0x7f0e7813b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 --2- 172.21.15.26:0/3489636668 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e7806daf0 0x7f0e78141c80 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 >> 172.21.15.26:0/3489636668 conn(0x7f0e780b0b70 msgr2=0x7f0e7806cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 shutdown_connections 2024-01-15T18:03:38.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.021+0000 7f0e80800640 1 -- 172.21.15.26:0/3489636668 wait complete. 2024-01-15T18:03:38.044 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:03:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.265+0000 7f9ac0a00640 1 Processor -- start 2024-01-15T18:03:38.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.265+0000 7f9ac0a00640 1 -- start start 2024-01-15T18:03:38.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9ac0a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9abc162430 0x7f9abc0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9ac0a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9abc162810 0x7f9abc071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9ac0a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9abc0702e0 con 0x7f9abc0b3650 2024-01-15T18:03:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9ac0a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9abc0705c0 con 0x7f9abc162430 2024-01-15T18:03:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9ac0a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9abc0708a0 con 0x7f9abc162810 2024-01-15T18:03:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9abb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9abc162430 0x7f9abc0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9abbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9abc0b3650 0x7f9abc1650c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43414/0 (socket says 172.21.15.26:43414) 2024-01-15T18:03:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9abb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9abc162430 0x7f9abc0b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50624/0 (socket says 172.21.15.26:50624) 2024-01-15T18:03:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9abbe00640 1 -- 172.21.15.26:0/3360035446 learned_addr learned my addr 172.21.15.26:0/3360035446 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:38.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.267+0000 7f9ab2800640 1 --2- 172.21.15.26:0/3360035446 >> v2:172.21.15.124:3300/0 conn(0x7f9abc162810 0x7f9abc071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.268+0000 7f9abb400640 1 -- 172.21.15.26:0/3360035446 >> v2:172.21.15.124:3300/0 conn(0x7f9abc162810 msgr2=0x7f9abc071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.268+0000 7f9abb400640 1 --2- 172.21.15.26:0/3360035446 >> v2:172.21.15.124:3300/0 conn(0x7f9abc162810 0x7f9abc071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.268+0000 7f9abb400640 1 -- 172.21.15.26:0/3360035446 >> v1:172.21.15.26:6789/0 conn(0x7f9abc0b3650 legacy=0x7f9abc1650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:38.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.268+0000 7f9abb400640 1 -- 172.21.15.26:0/3360035446 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc070b80 con 0x7f9abc162430 2024-01-15T18:03:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.269+0000 7f9abb400640 1 --2- 172.21.15.26:0/3360035446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc162430 0x7f9abc0b3110 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9abc15c090 tx=0x7f9aac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e884c49329a298 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.269+0000 7f9abaa00640 1 -- 172.21.15.26:0/3360035446 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac031830 con 0x7f9abc162430 2024-01-15T18:03:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.269+0000 7f9abaa00640 1 -- 172.21.15.26:0/3360035446 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac031990 con 0x7f9abc162430 2024-01-15T18:03:38.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.269+0000 7f9abaa00640 1 -- 172.21.15.26:0/3360035446 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac031d20 con 0x7f9abc162430 2024-01-15T18:03:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.270+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3360035446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc162430 msgr2=0x7f9abc0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.270+0000 7f9ac0a00640 1 --2- 172.21.15.26:0/3360035446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc162430 0x7f9abc0b3110 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f9abc15c090 tx=0x7f9aac02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.270+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3360035446 shutdown_connections 2024-01-15T18:03:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.270+0000 7f9ac0a00640 1 --2- 172.21.15.26:0/3360035446 >> v2:172.21.15.124:3300/0 conn(0x7f9abc162810 0x7f9abc071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.270+0000 7f9ac0a00640 1 --2- 172.21.15.26:0/3360035446 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc162430 0x7f9abc0b3110 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.270+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3360035446 >> 172.21.15.26:0/3360035446 conn(0x7f9abc06a800 msgr2=0x7f9abc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.270+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3360035446 shutdown_connections 2024-01-15T18:03:38.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.270+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3360035446 wait complete. 2024-01-15T18:03:38.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.272+0000 7f9ac0a00640 1 Processor -- start 2024-01-15T18:03:38.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.272+0000 7f9ac0a00640 1 -- start start 2024-01-15T18:03:38.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.273+0000 7f9ac0a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9abc0b3650 0x7f9abc146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.273+0000 7f9ac0a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9abc162430 0x7f9abc164630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.273+0000 7f9ac0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc153e80 0x7f9abc158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.273+0000 7f9ac0a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc165590 con 0x7f9abc153e80 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.273+0000 7f9ac0a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc1ebcc0 con 0x7f9abc0b3650 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.273+0000 7f9ac0a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9abc164fd0 con 0x7f9abc162430 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.273+0000 7f9abbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc153e80 0x7f9abc158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.273+0000 7f9abbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc153e80 0x7f9abc158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50634/0 (socket says 172.21.15.26:50634) 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.273+0000 7f9abbe00640 1 -- 172.21.15.26:0/3539289479 learned_addr learned my addr 172.21.15.26:0/3539289479 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9abb400640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9abc0b3650 0x7f9abc146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:38.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9ab2800640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9abc162430 0x7f9abc164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9abbe00640 1 -- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9abc162430 msgr2=0x7f9abc164630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9abbe00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9abc162430 0x7f9abc164630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9abbe00640 1 -- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9abc0b3650 msgr2=0x7f9abc146dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9abbe00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9abc0b3650 0x7f9abc146dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9abbe00640 1 -- 172.21.15.26:0/3539289479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac009400 con 0x7f9abc153e80 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9abb400640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9abc0b3650 0x7f9abc146dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9ab2800640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9abc162430 0x7f9abc164630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9abbe00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc153e80 0x7f9abc158b30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400cb20 tx=0x7f9ab400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9ab3e00640 1 -- 172.21.15.26:0/3539289479 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400ac40 con 0x7f9abc153e80 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9ab3e00640 1 -- 172.21.15.26:0/3539289479 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab400ada0 con 0x7f9abc153e80 2024-01-15T18:03:38.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9ab3e00640 1 -- 172.21.15.26:0/3539289479 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4012600 con 0x7f9abc153e80 2024-01-15T18:03:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.274+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9abc1652b0 con 0x7f9abc153e80 2024-01-15T18:03:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.275+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9abc147310 con 0x7f9abc153e80 2024-01-15T18:03:38.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.276+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9abc067450 con 0x7f9abc153e80 2024-01-15T18:03:38.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.277+0000 7f9ab3e00640 1 -- 172.21.15.26:0/3539289479 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f9ab4012760 con 0x7f9abc153e80 2024-01-15T18:03:38.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.278+0000 7f9ab3e00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9406bbf0 0x7f9a9406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:38.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.278+0000 7f9ab3e00640 1 -- 172.21.15.26:0/3539289479 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9ab40882c0 con 0x7f9abc153e80 2024-01-15T18:03:38.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.279+0000 7f9abb400640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9406bbf0 0x7f9a9406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:38.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.280+0000 7f9abb400640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9406bbf0 0x7f9a9406e0b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9aac036000 tx=0x7f9aac008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:38.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.283+0000 7f9ab3e00640 1 -- 172.21.15.26:0/3539289479 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ab4054c30 con 0x7f9abc153e80 2024-01-15T18:03:38.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.518+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9abc1ebe00 con 0x7f9abc153e80 2024-01-15T18:03:38.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.520+0000 7f9ab3e00640 1 -- 172.21.15.26:0/3539289479 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4918 (secure 0 0 0) 0x7f9ab4017090 con 0x7f9abc153e80 2024-01-15T18:03:38.524 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:03:38.524 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1986791067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1986791067},{"type":"v1","addr":"172.21.15.26:6834","nonce":1986791067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6082,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3285856051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3285856051},{"type":"v1","addr":"172.21.15.124:6836","nonce":3285856051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6085,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/198642905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":198642905},{"type":"v1","addr":"172.21.15.124:6837","nonce":198642905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:03:26.744869+0000","modified":"2024-01-15T18:03:29.852734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7095},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7095":{"gid":7095,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/1813593645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":1813593645},{"type":"v1","addr":"172.21.15.26:6835","nonce":1813593645}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-01-15T18:03:38.524 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 44 2024-01-15T18:03:38.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.522+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9406bbf0 msgr2=0x7f9a9406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.522+0000 7f9ac0a00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9406bbf0 0x7f9a9406e0b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9aac036000 tx=0x7f9aac008030 comp rx=0 tx=0).stop 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.522+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc153e80 msgr2=0x7f9abc158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.522+0000 7f9ac0a00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc153e80 0x7f9abc158b30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400cb20 tx=0x7f9ab400a060 comp rx=0 tx=0).stop 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.523+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 shutdown_connections 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.523+0000 7f9ac0a00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9abc162430 0x7f9abc164630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.523+0000 7f9ac0a00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9406bbf0 0x7f9a9406e0b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.523+0000 7f9ac0a00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9abc153e80 0x7f9abc158b30 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.523+0000 7f9ac0a00640 1 --2- 172.21.15.26:0/3539289479 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9abc0b3650 0x7f9abc146dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.523+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 >> 172.21.15.26:0/3539289479 conn(0x7f9abc06a800 msgr2=0x7f9abc0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:38.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.523+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 shutdown_connections 2024-01-15T18:03:38.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:38.523+0000 7f9ac0a00640 1 -- 172.21.15.26:0/3539289479 wait complete. 2024-01-15T18:03:38.949 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:03:26.744869+0000', 'modified': '2024-01-15T18:03:29.852734+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7095}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7095': {'gid': 7095, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/1813593645', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6833', 'nonce': 1813593645}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 1813593645}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:03:38.949 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:03:38.949 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-01-15T18:03:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.176+0000 7fd080e00640 1 Processor -- start 2024-01-15T18:03:39.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.177+0000 7fd080e00640 1 -- start start 2024-01-15T18:03:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.178+0000 7fd080e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c0cf9a0 0x7fd07c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.178+0000 7fd080e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07c0b1ee0 con 0x7fd07c15e1e0 2024-01-15T18:03:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.178+0000 7fd080e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07c0b2780 con 0x7fd07c143ae0 2024-01-15T18:03:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.178+0000 7fd080e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c15e990 con 0x7fd07c0cf9a0 2024-01-15T18:03:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.179+0000 7fd07a800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd07c15e1e0 0x7fd07c0702c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43428/0 (socket says 172.21.15.26:43428) 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.179+0000 7fd07a800640 1 -- 172.21.15.26:0/1020370703 learned_addr learned my addr 172.21.15.26:0/1020370703 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.179+0000 7fd079400640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c0cf9a0 0x7fd07c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.179+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704934130 0 0) 0x7fd07c0b1ee0 con 0x7fd07c15e1e0 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.179+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd05c003520 con 0x7fd07c15e1e0 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1126957036 0 0) 0x7fd05c003520 con 0x7fd07c15e1e0 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c0cf9a0 msgr2=0x7fd07c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c0cf9a0 0x7fd07c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 >> v1:172.21.15.124:6789/0 conn(0x7fd07c143ae0 legacy=0x7fd07c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c0b2130 con 0x7fd07c15e1e0 2024-01-15T18:03:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd074001670 con 0x7fd07c15e1e0 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 >> v1:172.21.15.26:6789/0 conn(0x7fd07c15e1e0 legacy=0x7fd07c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c0cf9a0 0x7fd05c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd05c003f70 0x7fd05c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd05c004d00 0x7fd05c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c15e990 con 0x7fd05c004d00 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c00bac0 con 0x7fd05c003f70 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05c00bc80 con 0x7fd07c0cf9a0 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd07a800640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd05c004d00 0x7fd05c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.180+0000 7fd079e00640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c0cf9a0 0x7fd05c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.181+0000 7fd079400640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd05c003f70 0x7fd05c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.181+0000 7fd07a800640 1 -- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c0cf9a0 msgr2=0x7fd05c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.181+0000 7fd07a800640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c0cf9a0 0x7fd05c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.181+0000 7fd07a800640 1 -- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd05c003f70 msgr2=0x7fd05c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.181+0000 7fd07a800640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd05c003f70 0x7fd05c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.181+0000 7fd07a800640 1 -- 172.21.15.26:0/1020370703 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd05c003520 con 0x7fd05c004d00 2024-01-15T18:03:39.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.182+0000 7fd07a800640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd05c004d00 0x7fd05c005180 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c0e0 tx=0x7fd05c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f98f73368b73173 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.182+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd074005ad0 con 0x7fd05c004d00 2024-01-15T18:03:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.182+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd074005c30 con 0x7fd05c004d00 2024-01-15T18:03:39.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.182+0000 7fd078a00640 1 -- 172.21.15.26:0/1020370703 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07402c7d0 con 0x7fd05c004d00 2024-01-15T18:03:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.183+0000 7fd080e00640 1 -- 172.21.15.26:0/1020370703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd05c004d00 msgr2=0x7fd05c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.183+0000 7fd080e00640 1 --2- 172.21.15.26:0/1020370703 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd05c004d00 0x7fd05c005180 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c0e0 tx=0x7fd05c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.183+0000 7fd07a800640 1 -- 172.21.15.26:0/1020370703 reap_dead start 2024-01-15T18:03:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.183+0000 7fd080e00640 1 -- 172.21.15.26:0/1020370703 shutdown_connections 2024-01-15T18:03:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.183+0000 7fd080e00640 1 -- 172.21.15.26:0/1020370703 >> 172.21.15.26:0/1020370703 conn(0x7fd07c06a800 msgr2=0x7fd07c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.183+0000 7fd080e00640 1 -- 172.21.15.26:0/1020370703 shutdown_connections 2024-01-15T18:03:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.183+0000 7fd080e00640 1 -- 172.21.15.26:0/1020370703 wait complete. 2024-01-15T18:03:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.185+0000 7fd080e00640 1 Processor -- start 2024-01-15T18:03:39.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.185+0000 7fd080e00640 1 -- start start 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd080e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd05c004d00 0x7fd07c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd080e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c1591d0 0x7fd07c13ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd080e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c13b280 0x7fd07c1398f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd080e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c0b19e0 con 0x7fd07c13b280 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd080e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c159eb0 con 0x7fd05c004d00 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd080e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c0aff30 con 0x7fd07c1591d0 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd07a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c13b280 0x7fd07c1398f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd07a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c13b280 0x7fd07c1398f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50652/0 (socket says 172.21.15.26:50652) 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd07a800640 1 -- 172.21.15.26:0/1094036197 learned_addr learned my addr 172.21.15.26:0/1094036197 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:39.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd079e00640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd05c004d00 0x7fd07c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd07a800640 1 -- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c1591d0 msgr2=0x7fd07c13ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd07a800640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c1591d0 0x7fd07c13ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd07a800640 1 -- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd05c004d00 msgr2=0x7fd07c0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd07a800640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd05c004d00 0x7fd07c0b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.186+0000 7fd07a800640 1 -- 172.21.15.26:0/1094036197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07c0af690 con 0x7fd07c13b280 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.187+0000 7fd07a800640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c13b280 0x7fd07c1398f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c200 tx=0x7fd07c0b26f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.187+0000 7fd072000640 1 -- 172.21.15.26:0/1094036197 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07403c070 con 0x7fd07c13b280 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.187+0000 7fd072000640 1 -- 172.21.15.26:0/1094036197 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07402c7d0 con 0x7fd07c13b280 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.187+0000 7fd072000640 1 -- 172.21.15.26:0/1094036197 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07402cb60 con 0x7fd07c13b280 2024-01-15T18:03:39.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.187+0000 7fd080e00640 1 -- 172.21.15.26:0/1094036197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c0af970 con 0x7fd07c13b280 2024-01-15T18:03:39.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.187+0000 7fd080e00640 1 -- 172.21.15.26:0/1094036197 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07c0cca30 con 0x7fd07c13b280 2024-01-15T18:03:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.189+0000 7fd072000640 1 -- 172.21.15.26:0/1094036197 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fd074032070 con 0x7fd07c13b280 2024-01-15T18:03:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.190+0000 7fd072000640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd05406bbf0 0x7fd05406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.190+0000 7fd072000640 1 -- 172.21.15.26:0/1094036197 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd0740ab670 con 0x7fd07c13b280 2024-01-15T18:03:39.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.191+0000 7fd079e00640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd05406bbf0 0x7fd05406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.191+0000 7fd083a3c740 1 Processor -- start 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.191+0000 7fd083a3c740 1 -- start start 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.192+0000 7fd079e00640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd05406bbf0 0x7fd05406e0b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd06c007fd0 tx=0x7fd06c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.192+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9aeea0 0x564c2e9af280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.192+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.192+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2e9f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.192+0000 7fd083a3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2e90d880 con 0x564c2e9aeea0 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.192+0000 7fd083a3c740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2e91cae0 con 0x564c2e9a9ba0 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.192+0000 7fd083a3c740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c2e91a1d0 con 0x564c2e9ae5e0 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9aeea0 0x564c2e9af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9aeea0 0x564c2e9af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50662/0 (socket says 172.21.15.26:50662) 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079e00640 1 -- 172.21.15.26:0/4000679549 learned_addr learned my addr 172.21.15.26:0/4000679549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079400640 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd07a800640 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2e9f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079e00640 1 -- 172.21.15.26:0/4000679549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 msgr2=0x564c2e9f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079e00640 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2e9f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079e00640 1 -- 172.21.15.26:0/4000679549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9a9ba0 msgr2=0x564c2e9ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079e00640 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079e00640 1 -- 172.21.15.26:0/4000679549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2e8dd860 con 0x564c2e9aeea0 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd07a800640 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2e9f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:39.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.193+0000 7fd079e00640 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9aeea0 0x564c2e9af280 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00f770 tx=0x7fd06c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a134a4f489a37eb9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd06aa00640 1 -- 172.21.15.26:0/4000679549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c014850 con 0x564c2e9aeea0 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd06aa00640 1 -- 172.21.15.26:0/4000679549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c00d050 con 0x564c2e9aeea0 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd06aa00640 1 -- 172.21.15.26:0/4000679549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c014bb0 con 0x564c2e9aeea0 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd083a3c740 1 -- 172.21.15.26:0/4000679549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9aeea0 msgr2=0x564c2e9af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd083a3c740 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9aeea0 0x564c2e9af280 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00f770 tx=0x7fd06c012000 comp rx=0 tx=0).stop 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd083a3c740 1 -- 172.21.15.26:0/4000679549 shutdown_connections 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd083a3c740 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2e9f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd083a3c740 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd083a3c740 1 --2- 172.21.15.26:0/4000679549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9aeea0 0x564c2e9af280 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd083a3c740 1 -- 172.21.15.26:0/4000679549 >> 172.21.15.26:0/4000679549 conn(0x564c2e970c70 msgr2=0x564c2e9f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:39.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd083a3c740 1 -- 172.21.15.26:0/4000679549 shutdown_connections 2024-01-15T18:03:39.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.194+0000 7fd083a3c740 1 -- 172.21.15.26:0/4000679549 wait complete. 2024-01-15T18:03:39.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.195+0000 7fd083a3c740 1 -- start start 2024-01-15T18:03:39.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.195+0000 7fd083a3c740 1 Processor -- start 2024-01-15T18:03:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.196+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.196+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2e9ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.196+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2e9d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.196+0000 7fd083a3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2e915ac0 con 0x564c2e9a9ba0 2024-01-15T18:03:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.196+0000 7fd083a3c740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2e8b22f0 con 0x564c2e9aeea0 2024-01-15T18:03:39.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.196+0000 7fd083a3c740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c2e8cfc70 con 0x564c2e9ae5e0 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.196+0000 7fd083a3c740 10 client.0 _ll_register_callbacks cb 0x564c2e8d43b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50672/0 (socket says 172.21.15.26:50672) 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079e00640 1 -- 172.21.15.26:0/1141473613 learned_addr learned my addr 172.21.15.26:0/1141473613 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079400640 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2e9ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd07a800640 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2e9d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079e00640 1 -- 172.21.15.26:0/1141473613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 msgr2=0x564c2e9ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079e00640 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2e9ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079e00640 1 -- 172.21.15.26:0/1141473613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 msgr2=0x564c2e9d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079e00640 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2e9d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079e00640 1 -- 172.21.15.26:0/1141473613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07c0b2780 con 0x564c2e9a9ba0 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd07a800640 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2e9d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd079e00640 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ce560 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd06c001460 tx=0x7fd06c001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2beab1bdce704 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.197+0000 7fd063400640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.198+0000 7fd063400640 1 -- 172.21.15.26:0/1141473613 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c020760 con 0x564c2e9a9ba0 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.198+0000 7fd063400640 1 -- 172.21.15.26:0/1141473613 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd06c01c040 con 0x564c2e9a9ba0 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.198+0000 7fd083a3c740 1 -- 172.21.15.26:0/1141473613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c2e90eaa0 con 0x564c2e9a9ba0 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.198+0000 7fd063400640 1 -- 172.21.15.26:0/1141473613 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd06c0208c0 con 0x564c2e9a9ba0 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.198+0000 7fd083a3c740 10 client.7263 fetch_fsmap learned FSMap version 44 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.198+0000 7fd083a3c740 1 -- 172.21.15.26:0/1141473613 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x564c2e914140 con 0x564c2e9a9ba0 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.198+0000 7fd063400640 1 -- 172.21.15.26:0/1141473613 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564c2e90eaa0 con 0x564c2e9a9ba0 2024-01-15T18:03:39.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.198+0000 7fd063400640 1 -- 172.21.15.26:0/1141473613 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd06c012070 con 0x564c2e9a9ba0 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.198+0000 7fd083a3c740 10 client.7263 fetch_fsmap finished waiting for FSMap version 44 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.199+0000 7fd083a3c740 10 client.7263 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.199+0000 7fd083a3c740 20 client.7263 populate_metadata read hostname 'smithi026' 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.199+0000 7fd083a3c740 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] conn(0x564c2e9f6f40 0x564c2e9ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.199+0000 7fd083a3c740 4 client.7263 mds_command: new command op to 7095 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.199+0000 7fd083a3c740 1 -- 172.21.15.26:0/1141473613 --> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c2e90e7c0 con 0x564c2e9f6f40 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.199+0000 7fd079400640 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] conn(0x564c2e9f6f40 0x564c2e9ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.199+0000 7fd063400640 10 client.7263 ms_handle_connect on v2:172.21.15.26:6833/1813593645 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.199+0000 7fd079400640 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] conn(0x564c2e9f6f40 0x564c2e9ceb90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320a8cb9dd109265 server_cookie=f6c89cd57e42657d in_seq=0 out_seq=0 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.200+0000 7fd063400640 1 -- 172.21.15.26:0/1141473613 <== mds.0 v2:172.21.15.26:6833/1813593645 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564c2e90e7c0 con 0x564c2e9f6f40 2024-01-15T18:03:39.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.200+0000 7fd063400640 10 client.7263 handle_command_reply: tid=0 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.200+0000 7fd083a3c740 1 client.7263 shutdown 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 -- 172.21.15.26:0/1141473613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 msgr2=0x564c2e9ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ce560 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd06c001460 tx=0x7fd06c001190 comp rx=0 tx=0).stop 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 -- 172.21.15.26:0/1141473613 shutdown_connections 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] conn(0x564c2e9f6f40 0x564c2e9ceb90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2e9ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2e9d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 --2- 172.21.15.26:0/1141473613 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9ce560 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd063400640 0 client.7263 ms_handle_reset on v2:172.21.15.26:6833/1813593645 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 -- 172.21.15.26:0/1141473613 >> 172.21.15.26:0/1141473613 conn(0x564c2e970c70 msgr2=0x564c2e9f1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 -- 172.21.15.26:0/1141473613 shutdown_connections 2024-01-15T18:03:39.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 1 -- 172.21.15.26:0/1141473613 wait complete. 2024-01-15T18:03:39.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.201+0000 7fd083a3c740 20 client.7263 trim_cache size 0 max 16384 2024-01-15T18:03:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.219+0000 7fd083a3c740 1 Processor -- start 2024-01-15T18:03:39.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.219+0000 7fd083a3c740 1 -- start start 2024-01-15T18:03:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.220+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.220+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2ea06d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.220+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2ea07240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.220+0000 7fd083a3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2e678f20 con 0x564c2e9a9ba0 2024-01-15T18:03:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.220+0000 7fd083a3c740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2e917a40 con 0x564c2e9aeea0 2024-01-15T18:03:39.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.220+0000 7fd083a3c740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c2ea18420 con 0x564c2e9ae5e0 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50684/0 (socket says 172.21.15.26:50684) 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079e00640 1 -- 172.21.15.26:0/2802733489 learned_addr learned my addr 172.21.15.26:0/2802733489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079400640 1 --2- 172.21.15.26:0/2802733489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2ea06d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd07a800640 1 --2- 172.21.15.26:0/2802733489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2ea07240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079e00640 1 -- 172.21.15.26:0/2802733489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 msgr2=0x564c2ea06d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079e00640 1 --2- 172.21.15.26:0/2802733489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2ea06d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079e00640 1 -- 172.21.15.26:0/2802733489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 msgr2=0x564c2ea07240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079e00640 1 --2- 172.21.15.26:0/2802733489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2ea07240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079e00640 1 -- 172.21.15.26:0/2802733489 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c2e8b1b80 con 0x564c2e9a9ba0 2024-01-15T18:03:39.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.221+0000 7fd079e00640 1 --2- 172.21.15.26:0/2802733489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9cca50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd06c014610 tx=0x7fd06c02b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83dbe91da609661 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd06b400640 1 -- 172.21.15.26:0/2802733489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c014e80 con 0x564c2e9a9ba0 2024-01-15T18:03:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd06b400640 1 -- 172.21.15.26:0/2802733489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c00f610 con 0x564c2e9a9ba0 2024-01-15T18:03:39.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd06b400640 1 -- 172.21.15.26:0/2802733489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c020760 con 0x564c2e9a9ba0 2024-01-15T18:03:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd083a3c740 1 -- 172.21.15.26:0/2802733489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 msgr2=0x564c2e9cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd083a3c740 1 --2- 172.21.15.26:0/2802733489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9cca50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd06c014610 tx=0x7fd06c02b000 comp rx=0 tx=0).stop 2024-01-15T18:03:39.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd083a3c740 1 -- 172.21.15.26:0/2802733489 shutdown_connections 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd083a3c740 1 --2- 172.21.15.26:0/2802733489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2ea06d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd083a3c740 1 --2- 172.21.15.26:0/2802733489 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2ea07240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd083a3c740 1 --2- 172.21.15.26:0/2802733489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9cca50 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.222+0000 7fd083a3c740 1 -- 172.21.15.26:0/2802733489 >> 172.21.15.26:0/2802733489 conn(0x564c2e970c70 msgr2=0x564c2e9a52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.223+0000 7fd083a3c740 1 -- 172.21.15.26:0/2802733489 shutdown_connections 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.223+0000 7fd083a3c740 1 -- 172.21.15.26:0/2802733489 wait complete. 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.223+0000 7fd083a3c740 1 -- start start 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.223+0000 7fd083a3c740 1 Processor -- start 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.224+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.224+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2ea585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.224+0000 7fd083a3c740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2ea0ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.224+0000 7fd083a3c740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ea563a0 con 0x564c2e9a9ba0 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.224+0000 7fd083a3c740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c2ea58d60 con 0x564c2e9aeea0 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.224+0000 7fd083a3c740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c2e9a94c0 con 0x564c2e9ae5e0 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.224+0000 7fd083a3c740 10 client.0 _ll_register_callbacks cb 0x564c2e9ae530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9a8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50700/0 (socket says 172.21.15.26:50700) 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079e00640 1 -- 172.21.15.26:0/3588436748 learned_addr learned my addr 172.21.15.26:0/3588436748 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079400640 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2ea585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd07a800640 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2ea0ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079e00640 1 -- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 msgr2=0x564c2ea585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079e00640 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2ea585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079e00640 1 -- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 msgr2=0x564c2ea0ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079e00640 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2ea0ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079e00640 1 -- 172.21.15.26:0/3588436748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c2e914140 con 0x564c2e9a9ba0 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd07a800640 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2ea0ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.225+0000 7fd079400640 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2ea585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.226+0000 7fd079e00640 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9a8f80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd06c041040 tx=0x7fd06c00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fda2798d7542625 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.226+0000 7fd063400640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.226+0000 7fd063400640 1 -- 172.21.15.26:0/3588436748 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c0074f0 con 0x564c2e9a9ba0 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.226+0000 7fd063400640 1 -- 172.21.15.26:0/3588436748 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd06c007650 con 0x564c2e9a9ba0 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.226+0000 7fd083a3c740 1 -- 172.21.15.26:0/3588436748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd06c00d050 con 0x564c2e9a9ba0 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.226+0000 7fd063400640 1 -- 172.21.15.26:0/3588436748 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd06c0067d0 con 0x564c2e9a9ba0 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.226+0000 7fd083a3c740 10 client.7269 fetch_fsmap learned FSMap version 44 2024-01-15T18:03:39.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.226+0000 7fd083a3c740 1 -- 172.21.15.26:0/3588436748 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x564c2e8d7170 con 0x564c2e9a9ba0 2024-01-15T18:03:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.227+0000 7fd063400640 1 -- 172.21.15.26:0/3588436748 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c056020 con 0x564c2e9a9ba0 2024-01-15T18:03:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.227+0000 7fd063400640 1 -- 172.21.15.26:0/3588436748 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd06c012070 con 0x564c2e9a9ba0 2024-01-15T18:03:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.227+0000 7fd083a3c740 10 client.7269 fetch_fsmap finished waiting for FSMap version 44 2024-01-15T18:03:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.227+0000 7fd083a3c740 10 client.7269 resolve_mds: resolved name 'c' to daemon mds.c 2024-01-15T18:03:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.227+0000 7fd083a3c740 20 client.7269 populate_metadata read hostname 'smithi026' 2024-01-15T18:03:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.227+0000 7fd083a3c740 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] conn(0x564c2e9a9f80 0x564c2ea62530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.227+0000 7fd083a3c740 4 client.7269 mds_command: new command op to 7095 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:03:39.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.227+0000 7fd083a3c740 1 -- 172.21.15.26:0/3588436748 --> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564c2e6166c0 con 0x564c2e9a9f80 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.227+0000 7fd079400640 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] conn(0x564c2e9a9f80 0x564c2ea62530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd063400640 10 client.7269 ms_handle_connect on v2:172.21.15.26:6833/1813593645 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd079400640 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] conn(0x564c2e9a9f80 0x564c2ea62530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0fba6b2cf0e396a server_cookie=af65a21fb559bcb5 in_seq=0 out_seq=0 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd063400640 1 -- 172.21.15.26:0/3588436748 <== mds.0 v2:172.21.15.26:6833/1813593645 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564c2e6166c0 con 0x564c2e9a9f80 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd063400640 10 client.7269 handle_command_reply: tid=0 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 client.7269 shutdown 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 -- 172.21.15.26:0/3588436748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 msgr2=0x564c2e9a8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9a8f80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fd06c041040 tx=0x7fd06c00a000 comp rx=0 tx=0).stop 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 -- 172.21.15.26:0/3588436748 shutdown_connections 2024-01-15T18:03:39.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.26:6833/1813593645,v1:172.21.15.26:6835/1813593645] conn(0x564c2e9a9f80 0x564c2ea62530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x564c2e9ae5e0 0x564c2ea585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x564c2e9aeea0 0x564c2ea0ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 --2- 172.21.15.26:0/3588436748 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x564c2e9a9ba0 0x564c2e9a8f80 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 -- 172.21.15.26:0/3588436748 >> 172.21.15.26:0/3588436748 conn(0x564c2e970c70 msgr2=0x564c2e9ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd063400640 0 client.7269 ms_handle_reset on v2:172.21.15.26:6833/1813593645 2024-01-15T18:03:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 -- 172.21.15.26:0/3588436748 shutdown_connections 2024-01-15T18:03:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.228+0000 7fd083a3c740 1 -- 172.21.15.26:0/3588436748 wait complete. 2024-01-15T18:03:39.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd083a3c740 20 client.7269 trim_cache size 0 max 16384 2024-01-15T18:03:39.238 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7095, 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 11.934952990999999, 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 44, 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 72, 2024-01-15T18:03:39.239 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 72, 2024-01-15T18:03:39.240 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 18.301463011999999 2024-01-15T18:03:39.240 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:03:39.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 -- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd05406bbf0 msgr2=0x7fd05406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd05406bbf0 0x7fd05406e0b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd06c007fd0 tx=0x7fd06c009040 comp rx=0 tx=0).stop 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 -- 172.21.15.26:0/1094036197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c13b280 msgr2=0x7fd07c1398f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c13b280 0x7fd07c1398f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd07400c200 tx=0x7fd07c0b26f0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 -- 172.21.15.26:0/1094036197 shutdown_connections 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd07c1591d0 0x7fd07c13ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd05406bbf0 0x7fd05406e0b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd07c13b280 0x7fd07c1398f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 --2- 172.21.15.26:0/1094036197 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd05c004d00 0x7fd07c0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 -- 172.21.15.26:0/1094036197 >> 172.21.15.26:0/1094036197 conn(0x7fd07c06a800 msgr2=0x7fd07c15caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.229+0000 7fd080e00640 1 -- 172.21.15.26:0/1094036197 shutdown_connections 2024-01-15T18:03:39.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.230+0000 7fd080e00640 1 -- 172.21.15.26:0/1094036197 wait complete. 2024-01-15T18:03:39.251 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:03:39.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.464+0000 7fd535e00640 1 Processor -- start 2024-01-15T18:03:39.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.464+0000 7fd535e00640 1 -- start start 2024-01-15T18:03:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.466+0000 7fd535e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd5300cc500 0x7fd5300cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.466+0000 7fd535e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd53006d160 0x7fd53006d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.466+0000 7fd535e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd530070e50 con 0x7fd530143c40 2024-01-15T18:03:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.466+0000 7fd535e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300723c0 con 0x7fd5300cc500 2024-01-15T18:03:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.466+0000 7fd535e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5300cced0 con 0x7fd53006d160 2024-01-15T18:03:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd534a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd530143c40 0x7fd530071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:43430/0 (socket says 172.21.15.26:43430) 2024-01-15T18:03:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd534a00640 1 -- 172.21.15.26:0/3802456522 learned_addr learned my addr 172.21.15.26:0/3802456522 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd52fe00640 1 --2- 172.21.15.26:0/3802456522 >> v2:172.21.15.26:3300/0 conn(0x7fd5300cc500 0x7fd5300cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd535400640 1 --2- 172.21.15.26:0/3802456522 >> v2:172.21.15.124:3300/0 conn(0x7fd53006d160 0x7fd53006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd52f400640 1 -- 172.21.15.26:0/3802456522 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371013115 0 0) 0x7fd530070e50 con 0x7fd530143c40 2024-01-15T18:03:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd52f400640 1 -- 172.21.15.26:0/3802456522 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd510003520 con 0x7fd530143c40 2024-01-15T18:03:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd52fe00640 1 -- 172.21.15.26:0/3802456522 >> v2:172.21.15.124:3300/0 conn(0x7fd53006d160 msgr2=0x7fd53006d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd52fe00640 1 --2- 172.21.15.26:0/3802456522 >> v2:172.21.15.124:3300/0 conn(0x7fd53006d160 0x7fd53006d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd52fe00640 1 -- 172.21.15.26:0/3802456522 >> v1:172.21.15.26:6789/0 conn(0x7fd530143c40 legacy=0x7fd530071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.467+0000 7fd52fe00640 1 -- 172.21.15.26:0/3802456522 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd530144020 con 0x7fd5300cc500 2024-01-15T18:03:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.469+0000 7fd52fe00640 1 --2- 172.21.15.26:0/3802456522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5300cc500 0x7fd5300cc990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd530070690 tx=0x7fd51802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a2d2a3fadc946b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.469+0000 7fd52f400640 1 -- 172.21.15.26:0/3802456522 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51802e070 con 0x7fd5300cc500 2024-01-15T18:03:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.469+0000 7fd52f400640 1 -- 172.21.15.26:0/3802456522 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5180315a0 con 0x7fd5300cc500 2024-01-15T18:03:39.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.469+0000 7fd52f400640 1 -- 172.21.15.26:0/3802456522 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd518031930 con 0x7fd5300cc500 2024-01-15T18:03:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.470+0000 7fd535e00640 1 -- 172.21.15.26:0/3802456522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5300cc500 msgr2=0x7fd5300cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.470+0000 7fd535e00640 1 --2- 172.21.15.26:0/3802456522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5300cc500 0x7fd5300cc990 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd530070690 tx=0x7fd51802b840 comp rx=0 tx=0).stop 2024-01-15T18:03:39.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.470+0000 7fd535e00640 1 -- 172.21.15.26:0/3802456522 shutdown_connections 2024-01-15T18:03:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.470+0000 7fd535e00640 1 --2- 172.21.15.26:0/3802456522 >> v2:172.21.15.124:3300/0 conn(0x7fd53006d160 0x7fd53006d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.470+0000 7fd535e00640 1 --2- 172.21.15.26:0/3802456522 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd5300cc500 0x7fd5300cc990 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.470+0000 7fd535e00640 1 -- 172.21.15.26:0/3802456522 >> 172.21.15.26:0/3802456522 conn(0x7fd53006a800 msgr2=0x7fd53006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.470+0000 7fd535e00640 1 -- 172.21.15.26:0/3802456522 shutdown_connections 2024-01-15T18:03:39.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.470+0000 7fd535e00640 1 -- 172.21.15.26:0/3802456522 wait complete. 2024-01-15T18:03:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.472+0000 7fd535e00640 1 Processor -- start 2024-01-15T18:03:39.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.472+0000 7fd535e00640 1 -- start start 2024-01-15T18:03:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd535e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd53006d160 0x7fd5301f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd535e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5300cc500 0x7fd5300b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd535e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd530143c40 0x7fd5300b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd535e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300c5530 con 0x7fd53006d160 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd535e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300cb030 con 0x7fd5300cc500 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd535e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300cb310 con 0x7fd530143c40 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd534a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd53006d160 0x7fd5301f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd534a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd53006d160 0x7fd5301f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50714/0 (socket says 172.21.15.26:50714) 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd534a00640 1 -- 172.21.15.26:0/3994192757 learned_addr learned my addr 172.21.15.26:0/3994192757 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.473+0000 7fd52fe00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5300cc500 0x7fd5300b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd535400640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd530143c40 0x7fd5300b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd534a00640 1 -- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd530143c40 msgr2=0x7fd5300b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd534a00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd530143c40 0x7fd5300b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd534a00640 1 -- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5300cc500 msgr2=0x7fd5300b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd534a00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5300cc500 0x7fd5300b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd534a00640 1 -- 172.21.15.26:0/3994192757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd518009400 con 0x7fd53006d160 2024-01-15T18:03:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd534a00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd53006d160 0x7fd5301f5b80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c410 tx=0x7fd52000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd52cc00640 1 -- 172.21.15.26:0/3994192757 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd520005ce0 con 0x7fd53006d160 2024-01-15T18:03:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd52cc00640 1 -- 172.21.15.26:0/3994192757 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd520005e40 con 0x7fd53006d160 2024-01-15T18:03:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd52cc00640 1 -- 172.21.15.26:0/3994192757 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5200072f0 con 0x7fd53006d160 2024-01-15T18:03:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5300c5740 con 0x7fd53006d160 2024-01-15T18:03:39.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.474+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5301f60c0 con 0x7fd53006d160 2024-01-15T18:03:39.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.476+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5300cb4d0 con 0x7fd53006d160 2024-01-15T18:03:39.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.477+0000 7fd52cc00640 1 -- 172.21.15.26:0/3994192757 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fd520007450 con 0x7fd53006d160 2024-01-15T18:03:39.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.477+0000 7fd52cc00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd50806bbf0 0x7fd50806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:39.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.478+0000 7fd52cc00640 1 -- 172.21.15.26:0/3994192757 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd520087a90 con 0x7fd53006d160 2024-01-15T18:03:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.478+0000 7fd52fe00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd50806bbf0 0x7fd50806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:39.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.479+0000 7fd52fe00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd50806bbf0 0x7fd50806e0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd518036000 tx=0x7fd51803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:39.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.482+0000 7fd52cc00640 1 -- 172.21.15.26:0/3994192757 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd520057f00 con 0x7fd53006d160 2024-01-15T18:03:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.714+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd530067450 con 0x7fd53006d160 2024-01-15T18:03:39.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.715+0000 7fd52cc00640 1 -- 172.21.15.26:0/3994192757 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd520057a90 con 0x7fd53006d160 2024-01-15T18:03:39.719 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:03:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd50806bbf0 msgr2=0x7fd50806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd50806bbf0 0x7fd50806e0b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd518036000 tx=0x7fd51803d040 comp rx=0 tx=0).stop 2024-01-15T18:03:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd53006d160 msgr2=0x7fd5301f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd53006d160 0x7fd5301f5b80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c410 tx=0x7fd52000c8d0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 shutdown_connections 2024-01-15T18:03:39.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd530143c40 0x7fd5300b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd50806bbf0 0x7fd50806e0b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd5300cc500 0x7fd5300b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 --2- 172.21.15.26:0/3994192757 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd53006d160 0x7fd5301f5b80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.717+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 >> 172.21.15.26:0/3994192757 conn(0x7fd53006a800 msgr2=0x7fd53006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.718+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 shutdown_connections 2024-01-15T18:03:39.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:39.718+0000 7fd535e00640 1 -- 172.21.15.26:0/3994192757 wait complete. 2024-01-15T18:03:39.740 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:03:39.740 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:03:39.741 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-15T18:03:39.741 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:03:39.741 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:03:39.741 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:03:39.741 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:03:39.741 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:03:39.741 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:03:39.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:39.742 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-15T18:03:39.795 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:03:39.795 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:39.796 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-15T18:03:39.797 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:03:39.797 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:39.797 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-15T18:03:39.797 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-15T18:03:39.797 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 42098sec preferred_lft 42098sec 2024-01-15T18:03:39.797 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-01-15T18:03:39.797 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:39.797 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::a4e5:3ff:fe91:fcad/64 scope link 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:03:39.798 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:03:39.799 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::f464:8fff:fecd:58d2/64 scope link 2024-01-15T18:03:39.799 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:39.799 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:03:39.799 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether fe:82:c8:bc:3f:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:03:39.799 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::fc82:c8ff:febc:3fd2/64 scope link 2024-01-15T18:03:39.799 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:39.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:39.800 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:03:39.892 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-15T18:03:39.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:39.897 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-15T18:03:39.954 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-15T18:03:39.954 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:03:39.954 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-15T18:03:39.954 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-15T18:03:39.954 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:03:39.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:39.955 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T18:03:39.955 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T18:03:39.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-15T18:03:39.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-15T18:03:39.955 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-15T18:03:39.955 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T18:03:40.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:40.158 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T18:03:40.209 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:03:40.209 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:03:40.211 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:03:40.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:40.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-15T18:03:40.310 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:03:40.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:03:40.310 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:03:40.367 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:03:40.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:03:40.367 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:03:40.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:03:40.525 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:03:40.582 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:03:40.582 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:03:40.582 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:03:40.583 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:03:40.584 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:03:40.584 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:03:40.584 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:03:40.584 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:03:40.584 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:03:40.584 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:03:40.584 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:03:40.584 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:03:40.584 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:03:40.585 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:03:40.586 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:03:40.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:40.587 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:03:40.644 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:03:40.645 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:03:40.650 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:03:40.737 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:03:40.780 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:03:40.781 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:03:40.782 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:03:40.783 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:03:40.784 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:03:40.784 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:03:40.784 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:03:40.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:40.785 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:03:40.851 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:2024-01-15T18:03:40.848+0000 7f4b4ba47480 -1 init, newargv = 0x55f54a9fd890 newargc=15 2024-01-15T18:03:40.851 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[62099]: starting ceph client 2024-01-15T18:03:40.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[62099]: starting fuse 2024-01-15T18:03:40.874 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T18:03:40.875 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:03:40.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:03:40.875 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-01-15T18:03:40.875 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-01-15T18:03:40.875 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-01-15T18:03:40.875 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-01-15T18:03:40.875 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:03:40.875 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:03:40.876 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.0")) 2024-01-15T18:03:40.877 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-01-15T18:03:41.085 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.0.62099.asok 2024-01-15T18:03:41.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:03:41.102 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.62099.asok 2024-01-15T18:03:41.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:41.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.62099.asok status 2024-01-15T18:03:41.350 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:03:41.350 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-01-15T18:03:41.350 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:03:41.350 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:03:41.350 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "0", 2024-01-15T18:03:41.350 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-01-15T18:03:41.350 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:03:41.350 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "62099", 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7287, 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 7287 2024-01-15T18:03:41.351 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2355936207 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:03:41.352 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 2355936207 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.7287 192.168.0.1:0/2355936207", 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/2355936207", 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 44, 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 72, 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-01-15T18:03:41.353 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:03:41.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:41.366 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-01-15T18:03:41.366 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:03:41.419 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-01-15T18:03:41.419 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:03:41.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:41.419 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:03:41.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:03:41.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.729+0000 7fe87b200640 1 Processor -- start 2024-01-15T18:03:41.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.730+0000 7fe87b200640 1 -- start start 2024-01-15T18:03:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.731+0000 7fe87b200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe8740ae5a0 0x7fe87415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.731+0000 7fe87b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 0x7fe87406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.731+0000 7fe87b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe87406d120 0x7fe8740cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.731+0000 7fe87b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe874071450 con 0x7fe87406d120 2024-01-15T18:03:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.731+0000 7fe87b200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe874070c40 con 0x7fe8740ae5a0 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.731+0000 7fe87b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87406c470 con 0x7fe8740b3ae0 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe879200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe87406d120 0x7fe8740cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe879200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe87406d120 0x7fe8740cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50722/0 (socket says 172.21.15.26:50722) 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe879200640 1 -- 172.21.15.26:0/3070051092 learned_addr learned my addr 172.21.15.26:0/3070051092 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe86be00640 1 --2- 172.21.15.26:0/3070051092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 0x7fe87406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe873e00640 1 --2- 172.21.15.26:0/3070051092 >> v2:172.21.15.124:3300/0 conn(0x7fe8740ae5a0 0x7fe87415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe879200640 1 -- 172.21.15.26:0/3070051092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 msgr2=0x7fe87406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe879200640 1 --2- 172.21.15.26:0/3070051092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 0x7fe87406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe879200640 1 -- 172.21.15.26:0/3070051092 >> v2:172.21.15.124:3300/0 conn(0x7fe8740ae5a0 msgr2=0x7fe87415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe879200640 1 --2- 172.21.15.26:0/3070051092 >> v2:172.21.15.124:3300/0 conn(0x7fe8740ae5a0 0x7fe87415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.732+0000 7fe879200640 1 -- 172.21.15.26:0/3070051092 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87406c750 con 0x7fe87406d120 2024-01-15T18:03:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.733+0000 7fe879200640 1 --2- 172.21.15.26:0/3070051092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 0x7fe8740cf290 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe8740cfe20 tx=0x7fe86c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81241ce44f3b55ea server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe873400640 1 -- 172.21.15.26:0/3070051092 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c02ca70 con 0x7fe87406d120 2024-01-15T18:03:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe873400640 1 -- 172.21.15.26:0/3070051092 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c02cbd0 con 0x7fe87406d120 2024-01-15T18:03:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe873400640 1 -- 172.21.15.26:0/3070051092 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0343f0 con 0x7fe87406d120 2024-01-15T18:03:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe87b200640 1 -- 172.21.15.26:0/3070051092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 msgr2=0x7fe8740cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe87b200640 1 --2- 172.21.15.26:0/3070051092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 0x7fe8740cf290 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fe8740cfe20 tx=0x7fe86c02bed0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe87b200640 1 -- 172.21.15.26:0/3070051092 shutdown_connections 2024-01-15T18:03:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe87b200640 1 --2- 172.21.15.26:0/3070051092 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 0x7fe8740cf290 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe87b200640 1 --2- 172.21.15.26:0/3070051092 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 0x7fe87406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe87b200640 1 --2- 172.21.15.26:0/3070051092 >> v2:172.21.15.124:3300/0 conn(0x7fe8740ae5a0 0x7fe87415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe87b200640 1 -- 172.21.15.26:0/3070051092 >> 172.21.15.26:0/3070051092 conn(0x7fe87406a800 msgr2=0x7fe87406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.734+0000 7fe87b200640 1 -- 172.21.15.26:0/3070051092 shutdown_connections 2024-01-15T18:03:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.735+0000 7fe87b200640 1 -- 172.21.15.26:0/3070051092 wait complete. 2024-01-15T18:03:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.736+0000 7fe87b200640 1 Processor -- start 2024-01-15T18:03:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.736+0000 7fe87b200640 1 -- start start 2024-01-15T18:03:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.737+0000 7fe87b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 0x7fe8741f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.737+0000 7fe87b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8740ae5a0 0x7fe874156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.737+0000 7fe87b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 0x7fe874158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.737+0000 7fe87b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe874156cd0 con 0x7fe87406d120 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.737+0000 7fe87b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741f5f10 con 0x7fe8740ae5a0 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.737+0000 7fe87b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8741f6190 con 0x7fe8740b3ae0 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe873e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 0x7fe8741f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe873e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 0x7fe8741f2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50730/0 (socket says 172.21.15.26:50730) 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe873e00640 1 -- 172.21.15.26:0/1081155610 learned_addr learned my addr 172.21.15.26:0/1081155610 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe879200640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 0x7fe874158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe86be00640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8740ae5a0 0x7fe874156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe873e00640 1 -- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 msgr2=0x7fe874158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe873e00640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 0x7fe874158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe873e00640 1 -- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8740ae5a0 msgr2=0x7fe874156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe873e00640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8740ae5a0 0x7fe874156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe873e00640 1 -- 172.21.15.26:0/1081155610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c009b30 con 0x7fe87406d120 2024-01-15T18:03:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.738+0000 7fe879200640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 0x7fe874158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.739+0000 7fe873e00640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 0x7fe8741f2980 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d800 tx=0x7fe86400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.739+0000 7fe870c00640 1 -- 172.21.15.26:0/1081155610 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8640096a0 con 0x7fe87406d120 2024-01-15T18:03:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.739+0000 7fe870c00640 1 -- 172.21.15.26:0/1081155610 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864009800 con 0x7fe87406d120 2024-01-15T18:03:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.739+0000 7fe870c00640 1 -- 172.21.15.26:0/1081155610 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864009b90 con 0x7fe87406d120 2024-01-15T18:03:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.739+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8741f6470 con 0x7fe87406d120 2024-01-15T18:03:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.739+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8741f6a30 con 0x7fe87406d120 2024-01-15T18:03:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.740+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8740675d0 con 0x7fe87406d120 2024-01-15T18:03:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.741+0000 7fe870c00640 1 -- 172.21.15.26:0/1081155610 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fe864009cf0 con 0x7fe87406d120 2024-01-15T18:03:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.742+0000 7fe870c00640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe84c06bbc0 0x7fe84c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.742+0000 7fe870c00640 1 -- 172.21.15.26:0/1081155610 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe864088970 con 0x7fe87406d120 2024-01-15T18:03:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.743+0000 7fe86be00640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe84c06bbc0 0x7fe84c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.744+0000 7fe86be00640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe84c06bbc0 0x7fe84c06e080 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe85c006fd0 tx=0x7fe85c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:41.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.747+0000 7fe870c00640 1 -- 172.21.15.26:0/1081155610 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe864055230 con 0x7fe87406d120 2024-01-15T18:03:41.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.980+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe87415ee80 con 0x7fe87406d120 2024-01-15T18:03:41.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.980+0000 7fe870c00640 1 -- 172.21.15.26:0/1081155610 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe86400b440 con 0x7fe87406d120 2024-01-15T18:03:41.984 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:03:41.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe84c06bbc0 msgr2=0x7fe84c06e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:41.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe84c06bbc0 0x7fe84c06e080 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe85c006fd0 tx=0x7fe85c008040 comp rx=0 tx=0).stop 2024-01-15T18:03:41.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 msgr2=0x7fe8741f2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:41.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 0x7fe8741f2980 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d800 tx=0x7fe86400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 shutdown_connections 2024-01-15T18:03:41.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe8740b3ae0 0x7fe874158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe84c06bbc0 0x7fe84c06e080 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe8740ae5a0 0x7fe874156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 --2- 172.21.15.26:0/1081155610 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe87406d120 0x7fe8741f2980 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:41.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.983+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 >> 172.21.15.26:0/1081155610 conn(0x7fe87406a800 msgr2=0x7fe8740b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:41.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.984+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 shutdown_connections 2024-01-15T18:03:41.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:41.984+0000 7fe87b200640 1 -- 172.21.15.26:0/1081155610 wait complete. 2024-01-15T18:03:42.008 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:03:42.009 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:03:42.009 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2024-01-15T18:03:42.009 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:03:42.009 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:03:42.009 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:03:42.009 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:03:42.009 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:03:42.009 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:03:42.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:42.010 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2024-01-15T18:03:42.069 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:03:42.069 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:03:42.070 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:03:42.070 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:42.070 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T18:03:42.070 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:42.070 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:03:42.070 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:42.070 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2024-01-15T18:03:42.070 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:03:42.070 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 42094sec preferred_lft 42094sec 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:03:42.071 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4c2d:94ff:fe47:f54c/64 scope link 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 26:4f:4f:5b:0b:31 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::244f:4fff:fe5b:b31/64 scope link 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:03:42.072 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:03:42.073 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::2070:16ff:fe92:4a4a/64 scope link 2024-01-15T18:03:42.073 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:03:42.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:42.073 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:03:42.160 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T18:03:42.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:42.164 DEBUG:teuthology.orchestra.run.smithi124:> route 2024-01-15T18:03:42.211 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2024-01-15T18:03:42.211 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:03:42.211 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:03:42.211 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:03:42.212 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:03:42.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:42.212 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T18:03:42.212 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T18:03:42.212 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:03:42.212 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:03:42.212 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:03:42.212 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T18:03:42.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:42.441 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T18:03:42.488 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:03:42.489 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:03:42.490 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:03:42.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:42.490 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2024-01-15T18:03:42.580 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:03:42.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:03:42.580 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:03:42.635 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:03:42.635 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:03:42.635 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:03:42.688 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:03:42.776 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:03:42.829 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:03:42.829 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:03:42.829 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:03:42.829 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:03:42.829 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:03:42.829 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:03:42.829 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:03:42.830 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:03:42.830 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:03:42.830 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:03:42.830 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:03:42.830 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:03:42.830 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:03:42.830 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:03:42.830 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:03:42.830 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:03:42.831 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:03:42.832 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:03:42.832 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:03:42.832 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:03:42.832 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:03:42.832 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:03:42.832 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:03:42.832 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:03:42.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:42.834 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:03:42.884 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:03:42.884 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:03:42.888 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:03:42.961 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:03:43.000 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:03:43.000 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:03:43.000 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:03:43.000 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:03:43.001 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:03:43.001 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:03:43.001 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:03:43.001 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:03:43.001 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:03:43.001 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:03:43.001 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:03:43.001 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:03:43.001 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:03:43.002 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:03:43.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:03:43.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:43.005 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:03:43.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:2024-01-15T18:03:43.066+0000 7f41681ff480 -1 init, newargv = 0x559f8dce0ad0 newargc=15 2024-01-15T18:03:43.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[45691]: starting ceph client 2024-01-15T18:03:43.082 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[45691]: starting fuse 2024-01-15T18:03:43.089 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T18:03:43.090 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:03:43.090 DEBUG:teuthology.orchestra.run.smithi124:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:03:43.090 DEBUG:teuthology.orchestra.run.smithi124:> import glob 2024-01-15T18:03:43.090 DEBUG:teuthology.orchestra.run.smithi124:> import re 2024-01-15T18:03:43.090 DEBUG:teuthology.orchestra.run.smithi124:> import os 2024-01-15T18:03:43.090 DEBUG:teuthology.orchestra.run.smithi124:> import subprocess 2024-01-15T18:03:43.090 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> def _find_admin_socket(client_name): 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> files = glob.glob(asok_path) 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> # Given a non-glob path, it better be there 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> if "*" not in asok_path: 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> assert(len(files) == 1) 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> return files[0] 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:03:43.091 DEBUG:teuthology.orchestra.run.smithi124:> for f in files: 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> contents = proc_f.read() 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> if mountpoint in contents: 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> return f 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> print(_find_admin_socket("client.1")) 2024-01-15T18:03:43.092 DEBUG:teuthology.orchestra.run.smithi124:> ' 2024-01-15T18:03:43.289 INFO:teuthology.orchestra.run.smithi124.stdout:/var/run/ceph/ceph-client.1.45691.asok 2024-01-15T18:03:43.293 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T18:03:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:03:43.306 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.45691.asok 2024-01-15T18:03:43.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:43.307 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.45691.asok status 2024-01-15T18:03:43.537 INFO:teuthology.orchestra.run.smithi124.stdout:{ 2024-01-15T18:03:43.538 INFO:teuthology.orchestra.run.smithi124.stdout: "metadata": { 2024-01-15T18:03:43.538 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:03:43.538 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:03:43.538 INFO:teuthology.orchestra.run.smithi124.stdout: "entity_id": "1", 2024-01-15T18:03:43.538 INFO:teuthology.orchestra.run.smithi124.stdout: "hostname": "smithi124", 2024-01-15T18:03:43.538 INFO:teuthology.orchestra.run.smithi124.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:03:43.538 INFO:teuthology.orchestra.run.smithi124.stdout: "pid": "45691", 2024-01-15T18:03:43.538 INFO:teuthology.orchestra.run.smithi124.stdout: "root": "/" 2024-01-15T18:03:43.538 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_count": 0, 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_pinned_count": 0, 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: "id": 6187, 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: "inst": { 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: "name": { 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "client", 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: "num": 6187 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:03:43.539 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:03:43.540 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:03:43.540 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 1412679440 2024-01-15T18:03:43.540 INFO:teuthology.orchestra.run.smithi124.stdout: } 2024-01-15T18:03:43.540 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:03:43.540 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:03:43.540 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:03:43.540 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:03:43.540 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 1412679440 2024-01-15T18:03:43.540 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:03:43.541 INFO:teuthology.orchestra.run.smithi124.stdout: "inst_str": "client.6187 192.168.0.1:0/1412679440", 2024-01-15T18:03:43.541 INFO:teuthology.orchestra.run.smithi124.stdout: "addr_str": "192.168.0.1:0/1412679440", 2024-01-15T18:03:43.541 INFO:teuthology.orchestra.run.smithi124.stdout: "inode_count": 1, 2024-01-15T18:03:43.541 INFO:teuthology.orchestra.run.smithi124.stdout: "mds_epoch": 44, 2024-01-15T18:03:43.541 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch": 72, 2024-01-15T18:03:43.541 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:03:43.541 INFO:teuthology.orchestra.run.smithi124.stdout: "blocklisted": false, 2024-01-15T18:03:43.541 INFO:teuthology.orchestra.run.smithi124.stdout: "fs_name": "cephfs" 2024-01-15T18:03:43.541 INFO:teuthology.orchestra.run.smithi124.stdout:} 2024-01-15T18:03:43.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:43.553 DEBUG:teuthology.orchestra.run.smithi124:> stat --file-system '--printf=%T 2024-01-15T18:03:43.553 DEBUG:teuthology.orchestra.run.smithi124:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:03:43.608 INFO:teuthology.orchestra.run.smithi124.stdout:fuseblk 2024-01-15T18:03:43.608 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:03:43.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:43.609 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:03:43.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:03:43.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.932+0000 7fb105800640 1 Processor -- start 2024-01-15T18:03:43.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.933+0000 7fb105800640 1 -- start start 2024-01-15T18:03:43.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.934+0000 7fb105800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10013cfa0 0x7fb10015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:43.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.935+0000 7fb105800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1000cdec0 con 0x7fb10013fcb0 2024-01-15T18:03:43.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.935+0000 7fb105800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1000ce1a0 con 0x7fb1000cd530 2024-01-15T18:03:43.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.935+0000 7fb105800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000ce480 con 0x7fb10013cfa0 2024-01-15T18:03:43.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.935+0000 7fb104e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb10013fcb0 0x7fb1000b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42238/0 (socket says 172.21.15.26:42238) 2024-01-15T18:03:43.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.935+0000 7fb104e00640 1 -- 172.21.15.26:0/4095507520 learned_addr learned my addr 172.21.15.26:0/4095507520 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:43.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.935+0000 7fb0ffe00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10013cfa0 0x7fb10015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:43.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3303518114 0 0) 0x7fb1000cdec0 con 0x7fb10013fcb0 2024-01-15T18:03:43.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e0005730 con 0x7fb10013fcb0 2024-01-15T18:03:43.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127216635 0 0) 0x7fb0e0005730 con 0x7fb10013fcb0 2024-01-15T18:03:43.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10013cfa0 msgr2=0x7fb10015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:43.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10013cfa0 0x7fb10015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:43.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 >> v1:172.21.15.124:6789/0 conn(0x7fb1000cd530 legacy=0x7fb10013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:43.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000ce760 con 0x7fb10013fcb0 2024-01-15T18:03:43.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb0f8001670 con 0x7fb10013fcb0 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 >> v1:172.21.15.26:6789/0 conn(0x7fb10013fcb0 legacy=0x7fb1000b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0e0005bb0 0x7fb0e0005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10013cfa0 0x7fb0e00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e0006f10 0x7fb0e0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000ce480 con 0x7fb0e0006f10 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e000dd20 con 0x7fb0e0005bb0 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.936+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e000dee0 con 0x7fb10013cfa0 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.937+0000 7fb104e00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e0006f10 0x7fb0e0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.937+0000 7fb0ffe00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0e0005bb0 0x7fb0e0005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:43.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.937+0000 7fb0ff400640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10013cfa0 0x7fb0e00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.937+0000 7fb104e00640 1 -- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10013cfa0 msgr2=0x7fb0e00069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.937+0000 7fb104e00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10013cfa0 0x7fb0e00069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.937+0000 7fb104e00640 1 -- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0e0005bb0 msgr2=0x7fb0e0005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.937+0000 7fb104e00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0e0005bb0 0x7fb0e0005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.937+0000 7fb104e00640 1 -- 172.21.15.26:0/4095507520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0005730 con 0x7fb0e0006f10 2024-01-15T18:03:43.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.937+0000 7fb0ff400640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb10013cfa0 0x7fb0e00069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:43.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.938+0000 7fb104e00640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e0006f10 0x7fb0e0007390 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c0e0 tx=0x7fb0e0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d910d018595c1fba server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:43.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.938+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8005ad0 con 0x7fb0e0006f10 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.938+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f8005c30 con 0x7fb0e0006f10 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.938+0000 7fb0fea00640 1 -- 172.21.15.26:0/4095507520 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f802c810 con 0x7fb0e0006f10 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.939+0000 7fb105800640 1 -- 172.21.15.26:0/4095507520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e0006f10 msgr2=0x7fb0e0007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.939+0000 7fb105800640 1 --2- 172.21.15.26:0/4095507520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0e0006f10 0x7fb0e0007390 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c0e0 tx=0x7fb0e0007a00 comp rx=0 tx=0).stop 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.939+0000 7fb104e00640 1 -- 172.21.15.26:0/4095507520 reap_dead start 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.939+0000 7fb105800640 1 -- 172.21.15.26:0/4095507520 shutdown_connections 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.939+0000 7fb105800640 1 -- 172.21.15.26:0/4095507520 >> 172.21.15.26:0/4095507520 conn(0x7fb10006a800 msgr2=0x7fb10006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.939+0000 7fb105800640 1 -- 172.21.15.26:0/4095507520 shutdown_connections 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.939+0000 7fb105800640 1 -- 172.21.15.26:0/4095507520 wait complete. 2024-01-15T18:03:43.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.941+0000 7fb105800640 1 Processor -- start 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.941+0000 7fb105800640 1 -- start start 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.942+0000 7fb105800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0e0006f10 0x7fb10013ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.942+0000 7fb105800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb100131430 0x7fb10013d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.942+0000 7fb105800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10013d990 0x7fb1000a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.942+0000 7fb105800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10013eb20 con 0x7fb10013d990 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.942+0000 7fb105800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10013dea0 con 0x7fb100131430 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.942+0000 7fb105800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10013e180 con 0x7fb0e0006f10 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb104e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10013d990 0x7fb1000a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb104e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10013d990 0x7fb1000a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40678/0 (socket says 172.21.15.26:40678) 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb104e00640 1 -- 172.21.15.26:0/4059008122 learned_addr learned my addr 172.21.15.26:0/4059008122 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:43.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb0ffe00640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0e0006f10 0x7fb10013ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb104e00640 1 -- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0e0006f10 msgr2=0x7fb10013ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb104e00640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0e0006f10 0x7fb10013ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb104e00640 1 -- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb100131430 msgr2=0x7fb10013d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb0ff400640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb100131430 0x7fb10013d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb104e00640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb100131430 0x7fb10013d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb104e00640 1 -- 172.21.15.26:0/4059008122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10013e460 con 0x7fb10013d990 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.943+0000 7fb0ff400640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb100131430 0x7fb10013d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.944+0000 7fb104e00640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10013d990 0x7fb1000a84c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c200 tx=0x7fb0f802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.944+0000 7fb0f7e00640 1 -- 172.21.15.26:0/4059008122 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f803c040 con 0x7fb10013d990 2024-01-15T18:03:43.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.944+0000 7fb0f7e00640 1 -- 172.21.15.26:0/4059008122 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f802ca10 con 0x7fb10013d990 2024-01-15T18:03:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.944+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10013e740 con 0x7fb10013d990 2024-01-15T18:03:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.944+0000 7fb0f7e00640 1 -- 172.21.15.26:0/4059008122 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f802cda0 con 0x7fb10013d990 2024-01-15T18:03:43.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.944+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10013c7e0 con 0x7fb10013d990 2024-01-15T18:03:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.945+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb100067450 con 0x7fb10013d990 2024-01-15T18:03:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.946+0000 7fb0f7e00640 1 -- 172.21.15.26:0/4059008122 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fb0f8032070 con 0x7fb10013d990 2024-01-15T18:03:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.948+0000 7fb0f7e00640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0d806bbf0 0x7fb0d806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.948+0000 7fb0f7e00640 1 -- 172.21.15.26:0/4059008122 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb0f80a9690 con 0x7fb10013d990 2024-01-15T18:03:43.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.948+0000 7fb0ffe00640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0d806bbf0 0x7fb0d806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:43.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.949+0000 7fb0ffe00640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0d806bbf0 0x7fb0d806e0b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0007380 tx=0x7fb0f000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:43.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:43.952+0000 7fb0f7e00640 1 -- 172.21.15.26:0/4059008122 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb0f8075d30 con 0x7fb10013d990 2024-01-15T18:03:44.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.189+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb1000b4000 con 0x7fb10013d990 2024-01-15T18:03:44.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.421+0000 7fb0f7e00640 1 -- 172.21.15.26:0/4059008122 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 136+0+0 (secure 0 0 0) 0x7fb0f8079350 con 0x7fb10013d990 2024-01-15T18:03:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.425+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0d806bbf0 msgr2=0x7fb0d806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.425+0000 7fb105800640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0d806bbf0 0x7fb0d806e0b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0007380 tx=0x7fb0f000b040 comp rx=0 tx=0).stop 2024-01-15T18:03:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.425+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10013d990 msgr2=0x7fb1000a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.425+0000 7fb105800640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10013d990 0x7fb1000a84c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800c200 tx=0x7fb0f802e070 comp rx=0 tx=0).stop 2024-01-15T18:03:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.426+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 shutdown_connections 2024-01-15T18:03:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.426+0000 7fb105800640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0d806bbf0 0x7fb0d806e0b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.426+0000 7fb105800640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb10013d990 0x7fb1000a84c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.426+0000 7fb105800640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb100131430 0x7fb10013d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.426+0000 7fb105800640 1 --2- 172.21.15.26:0/4059008122 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0e0006f10 0x7fb10013ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.426+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 >> 172.21.15.26:0/4059008122 conn(0x7fb10006a800 msgr2=0x7fb100155290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:44.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.426+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 shutdown_connections 2024-01-15T18:03:44.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.426+0000 7fb105800640 1 -- 172.21.15.26:0/4059008122 wait complete. 2024-01-15T18:03:44.452 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:03:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.670+0000 7fac52800640 1 Processor -- start 2024-01-15T18:03:44.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.670+0000 7fac52800640 1 -- start start 2024-01-15T18:03:44.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.672+0000 7fac52800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac4c143c40 0x7fac4c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:44.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.672+0000 7fac52800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fac4c0717d0 0x7fac4c0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:44.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.672+0000 7fac52800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c070e50 con 0x7fac4c0cca40 2024-01-15T18:03:44.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.672+0000 7fac52800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0723c0 con 0x7fac4c143c40 2024-01-15T18:03:44.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.672+0000 7fac52800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c0cce40 con 0x7fac4c0717d0 2024-01-15T18:03:44.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.672+0000 7fac51400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac4c143c40 0x7fac4c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:44.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.672+0000 7fac51400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac4c143c40 0x7fac4c071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40682/0 (socket says 172.21.15.26:40682) 2024-01-15T18:03:44.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.672+0000 7fac51400640 1 -- 172.21.15.26:0/3417749930 learned_addr learned my addr 172.21.15.26:0/3417749930 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:44.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.673+0000 7fac51400640 1 -- 172.21.15.26:0/3417749930 >> v2:172.21.15.124:3300/0 conn(0x7fac4c0717d0 msgr2=0x7fac4c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.673+0000 7fac51400640 1 --2- 172.21.15.26:0/3417749930 >> v2:172.21.15.124:3300/0 conn(0x7fac4c0717d0 0x7fac4c0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.673+0000 7fac51400640 1 -- 172.21.15.26:0/3417749930 >> v1:172.21.15.124:6789/0 conn(0x7fac4c0cca40 legacy=0x7fac4c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:44.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.673+0000 7fac51400640 1 -- 172.21.15.26:0/3417749930 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c06e170 con 0x7fac4c143c40 2024-01-15T18:03:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.674+0000 7fac51400640 1 --2- 172.21.15.26:0/3417749930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c143c40 0x7fac4c071290 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0b44a0 tx=0x7fac3c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7db30c42f8ac3c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:44.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.674+0000 7fac4be00640 1 -- 172.21.15.26:0/3417749930 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0317e0 con 0x7fac4c143c40 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.674+0000 7fac4be00640 1 -- 172.21.15.26:0/3417749930 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c031940 con 0x7fac4c143c40 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.674+0000 7fac4be00640 1 -- 172.21.15.26:0/3417749930 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c031cd0 con 0x7fac4c143c40 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.675+0000 7fac52800640 1 -- 172.21.15.26:0/3417749930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c143c40 msgr2=0x7fac4c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.675+0000 7fac52800640 1 --2- 172.21.15.26:0/3417749930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c143c40 0x7fac4c071290 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fac4c0b44a0 tx=0x7fac3c02b880 comp rx=0 tx=0).stop 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.675+0000 7fac52800640 1 -- 172.21.15.26:0/3417749930 shutdown_connections 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.675+0000 7fac52800640 1 --2- 172.21.15.26:0/3417749930 >> v2:172.21.15.124:3300/0 conn(0x7fac4c0717d0 0x7fac4c0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.675+0000 7fac52800640 1 --2- 172.21.15.26:0/3417749930 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c143c40 0x7fac4c071290 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.675+0000 7fac52800640 1 -- 172.21.15.26:0/3417749930 >> 172.21.15.26:0/3417749930 conn(0x7fac4c06a800 msgr2=0x7fac4c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.675+0000 7fac52800640 1 -- 172.21.15.26:0/3417749930 shutdown_connections 2024-01-15T18:03:44.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.675+0000 7fac52800640 1 -- 172.21.15.26:0/3417749930 wait complete. 2024-01-15T18:03:44.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.677+0000 7fac52800640 1 Processor -- start 2024-01-15T18:03:44.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.677+0000 7fac52800640 1 -- start start 2024-01-15T18:03:44.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.678+0000 7fac52800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac4c0717d0 0x7fac4c0a8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:44.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.678+0000 7fac52800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c0cca40 0x7fac4c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:44.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.678+0000 7fac52800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac4c143c40 0x7fac4c0b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:44.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.678+0000 7fac52800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0aa380 con 0x7fac4c0cca40 2024-01-15T18:03:44.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.678+0000 7fac52800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0aa660 con 0x7fac4c0717d0 2024-01-15T18:03:44.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.678+0000 7fac52800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c0a9700 con 0x7fac4c143c40 2024-01-15T18:03:44.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.678+0000 7fac50a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c0cca40 0x7fac4c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:44.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.678+0000 7fac50a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c0cca40 0x7fac4c0afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40690/0 (socket says 172.21.15.26:40690) 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.678+0000 7fac50a00640 1 -- 172.21.15.26:0/2088295556 learned_addr learned my addr 172.21.15.26:0/2088295556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.679+0000 7fac51e00640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac4c143c40 0x7fac4c0b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.679+0000 7fac51400640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac4c0717d0 0x7fac4c0a8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.679+0000 7fac50a00640 1 -- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac4c143c40 msgr2=0x7fac4c0b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.679+0000 7fac50a00640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac4c143c40 0x7fac4c0b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.679+0000 7fac50a00640 1 -- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac4c0717d0 msgr2=0x7fac4c0a8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.679+0000 7fac50a00640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac4c0717d0 0x7fac4c0a8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.679+0000 7fac50a00640 1 -- 172.21.15.26:0/2088295556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c0094d0 con 0x7fac4c0cca40 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.679+0000 7fac50a00640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c0cca40 0x7fac4c0afac0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fac3400b4d0 tx=0x7fac3400b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.680+0000 7fac49600640 1 -- 172.21.15.26:0/2088295556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34010870 con 0x7fac4c0cca40 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.680+0000 7fac49600640 1 -- 172.21.15.26:0/2088295556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac340109d0 con 0x7fac4c0cca40 2024-01-15T18:03:44.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.680+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c0a99e0 con 0x7fac4c0cca40 2024-01-15T18:03:44.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.680+0000 7fac49600640 1 -- 172.21.15.26:0/2088295556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34010d80 con 0x7fac4c0cca40 2024-01-15T18:03:44.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.680+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c0a9fa0 con 0x7fac4c0cca40 2024-01-15T18:03:44.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.681+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4c0cb6d0 con 0x7fac4c0cca40 2024-01-15T18:03:44.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.682+0000 7fac49600640 1 -- 172.21.15.26:0/2088295556 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fac3400f3c0 con 0x7fac4c0cca40 2024-01-15T18:03:44.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.683+0000 7fac49600640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fac2406bbf0 0x7fac2406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:44.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.683+0000 7fac49600640 1 -- 172.21.15.26:0/2088295556 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fac34088610 con 0x7fac4c0cca40 2024-01-15T18:03:44.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.684+0000 7fac51400640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fac2406bbf0 0x7fac2406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:44.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.685+0000 7fac51400640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fac2406bbf0 0x7fac2406e0b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fac3c036000 tx=0x7fac3c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:44.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.687+0000 7fac49600640 1 -- 172.21.15.26:0/2088295556 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac34054ce0 con 0x7fac4c0cca40 2024-01-15T18:03:44.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.905+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fac4c15afb0 con 0x7fac2406bbf0 2024-01-15T18:03:44.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.909+0000 7fac49600640 1 -- 172.21.15.26:0/2088295556 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fac4c15afb0 con 0x7fac2406bbf0 2024-01-15T18:03:44.913 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:03:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:03:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:03:44.913 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:03:44.913 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:03:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fac2406bbf0 msgr2=0x7fac2406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fac2406bbf0 0x7fac2406e0b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fac3c036000 tx=0x7fac3c03d040 comp rx=0 tx=0).stop 2024-01-15T18:03:44.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c0cca40 msgr2=0x7fac4c0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c0cca40 0x7fac4c0afac0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fac3400b4d0 tx=0x7fac3400b990 comp rx=0 tx=0).stop 2024-01-15T18:03:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 shutdown_connections 2024-01-15T18:03:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac4c143c40 0x7fac4c0b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fac2406bbf0 0x7fac2406e0b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac4c0cca40 0x7fac4c0afac0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 --2- 172.21.15.26:0/2088295556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac4c0717d0 0x7fac4c0a8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 >> 172.21.15.26:0/2088295556 conn(0x7fac4c06a800 msgr2=0x7fac4c15a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 shutdown_connections 2024-01-15T18:03:44.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:44.911+0000 7fac52800640 1 -- 172.21.15.26:0/2088295556 wait complete. 2024-01-15T18:03:44.932 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:03:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.153+0000 7f7a45200640 1 Processor -- start 2024-01-15T18:03:45.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.153+0000 7f7a45200640 1 -- start start 2024-01-15T18:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.154+0000 7f7a45200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7a4006c470 0x7f7a400cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.154+0000 7f7a45200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4006ca10 con 0x7f7a4006dab0 2024-01-15T18:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.154+0000 7f7a45200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4006ccf0 con 0x7f7a401447f0 2024-01-15T18:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.154+0000 7f7a45200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4006d690 con 0x7f7a4006c470 2024-01-15T18:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.155+0000 7f7a3fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7a4006dab0 0x7f7a4015c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42242/0 (socket says 172.21.15.26:42242) 2024-01-15T18:03:45.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.155+0000 7f7a3fe00640 1 -- 172.21.15.26:0/563816721 learned_addr learned my addr 172.21.15.26:0/563816721 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.155+0000 7f7a37e00640 1 --2- 172.21.15.26:0/563816721 >> v2:172.21.15.26:3300/0 conn(0x7f7a4006c470 0x7f7a400cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.155+0000 7f7a3ea00640 1 -- 172.21.15.26:0/563816721 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583793321 0 0) 0x7f7a4006ca10 con 0x7f7a4006dab0 2024-01-15T18:03:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.155+0000 7f7a3ea00640 1 -- 172.21.15.26:0/563816721 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a20003520 con 0x7f7a4006dab0 2024-01-15T18:03:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.155+0000 7f7a37e00640 1 -- 172.21.15.26:0/563816721 >> v1:172.21.15.124:6789/0 conn(0x7f7a401447f0 legacy=0x7f7a4015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.155+0000 7f7a37e00640 1 -- 172.21.15.26:0/563816721 >> v1:172.21.15.26:6789/0 conn(0x7f7a4006dab0 legacy=0x7f7a4015c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.155+0000 7f7a37e00640 1 -- 172.21.15.26:0/563816721 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4006cfd0 con 0x7f7a4006c470 2024-01-15T18:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.157+0000 7f7a37e00640 1 --2- 172.21.15.26:0/563816721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 0x7f7a400cc5e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7a280097a0 tx=0x7f7a2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4f8ca58dfabb6d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.157+0000 7f7a3ea00640 1 -- 172.21.15.26:0/563816721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28031830 con 0x7f7a4006c470 2024-01-15T18:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.157+0000 7f7a3ea00640 1 -- 172.21.15.26:0/563816721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a28031990 con 0x7f7a4006c470 2024-01-15T18:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.157+0000 7f7a3ea00640 1 -- 172.21.15.26:0/563816721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28031d20 con 0x7f7a4006c470 2024-01-15T18:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.158+0000 7f7a45200640 1 -- 172.21.15.26:0/563816721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 msgr2=0x7f7a400cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.158+0000 7f7a45200640 1 --2- 172.21.15.26:0/563816721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 0x7f7a400cc5e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f7a280097a0 tx=0x7f7a2802b8d0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.158+0000 7f7a45200640 1 -- 172.21.15.26:0/563816721 shutdown_connections 2024-01-15T18:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.158+0000 7f7a45200640 1 --2- 172.21.15.26:0/563816721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 0x7f7a400cc5e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.158+0000 7f7a45200640 1 -- 172.21.15.26:0/563816721 >> 172.21.15.26:0/563816721 conn(0x7f7a4006a3e0 msgr2=0x7f7a4006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.158+0000 7f7a45200640 1 -- 172.21.15.26:0/563816721 shutdown_connections 2024-01-15T18:03:45.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.158+0000 7f7a45200640 1 -- 172.21.15.26:0/563816721 wait complete. 2024-01-15T18:03:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.160+0000 7f7a45200640 1 Processor -- start 2024-01-15T18:03:45.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.160+0000 7f7a45200640 1 -- start start 2024-01-15T18:03:45.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.161+0000 7f7a45200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 0x7f7a401f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.161+0000 7f7a45200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4006dab0 0x7f7a4015da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.161+0000 7f7a45200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a401447f0 0x7f7a4015dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.161+0000 7f7a45200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4013b560 con 0x7f7a4006c470 2024-01-15T18:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.161+0000 7f7a45200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a40071130 con 0x7f7a401447f0 2024-01-15T18:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.161+0000 7f7a45200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a40071410 con 0x7f7a4006dab0 2024-01-15T18:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.161+0000 7f7a3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 0x7f7a401f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.161+0000 7f7a3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 0x7f7a401f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40698/0 (socket says 172.21.15.26:40698) 2024-01-15T18:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.161+0000 7f7a3f400640 1 -- 172.21.15.26:0/659885164 learned_addr learned my addr 172.21.15.26:0/659885164 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:45.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a3fe00640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a401447f0 0x7f7a4015dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a3f400640 1 -- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4006dab0 msgr2=0x7f7a4015da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a37e00640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4006dab0 0x7f7a4015da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a3f400640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4006dab0 0x7f7a4015da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a3f400640 1 -- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a401447f0 msgr2=0x7f7a4015dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a3f400640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a401447f0 0x7f7a4015dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a3f400640 1 -- 172.21.15.26:0/659885164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a28009400 con 0x7f7a4006c470 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a37e00640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4006dab0 0x7f7a4015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a3fe00640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a401447f0 0x7f7a4015dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.162+0000 7f7a3f400640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 0x7f7a401f5770 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000c400 tx=0x7f7a3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:45.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.163+0000 7f7a37400640 1 -- 172.21.15.26:0/659885164 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a300052d0 con 0x7f7a4006c470 2024-01-15T18:03:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.163+0000 7f7a37400640 1 -- 172.21.15.26:0/659885164 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a30005430 con 0x7f7a4006c470 2024-01-15T18:03:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.163+0000 7f7a37400640 1 -- 172.21.15.26:0/659885164 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30003830 con 0x7f7a4006c470 2024-01-15T18:03:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.163+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a400716f0 con 0x7f7a4006c470 2024-01-15T18:03:45.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.163+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a40071cb0 con 0x7f7a4006c470 2024-01-15T18:03:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.164+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4006ca10 con 0x7f7a4006c470 2024-01-15T18:03:45.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.165+0000 7f7a37400640 1 -- 172.21.15.26:0/659885164 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f7a30003990 con 0x7f7a4006c470 2024-01-15T18:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.166+0000 7f7a37400640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a1806bc10 0x7f7a1806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.166+0000 7f7a37400640 1 -- 172.21.15.26:0/659885164 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7a30087a70 con 0x7f7a4006c470 2024-01-15T18:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.167+0000 7f7a37e00640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a1806bc10 0x7f7a1806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.168+0000 7f7a37e00640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a1806bc10 0x7f7a1806e0d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7a28036000 tx=0x7f7a28008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:45.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.170+0000 7f7a37400640 1 -- 172.21.15.26:0/659885164 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a30054370 con 0x7f7a4006c470 2024-01-15T18:03:45.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.364+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7a40067450 con 0x7f7a4006c470 2024-01-15T18:03:45.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.364+0000 7f7a37400640 1 -- 172.21.15.26:0/659885164 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a30057990 con 0x7f7a4006c470 2024-01-15T18:03:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.367+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a1806bc10 msgr2=0x7f7a1806e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.367+0000 7f7a45200640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a1806bc10 0x7f7a1806e0d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7a28036000 tx=0x7f7a28008030 comp rx=0 tx=0).stop 2024-01-15T18:03:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 msgr2=0x7f7a401f5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 0x7f7a401f5770 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000c400 tx=0x7f7a3000c8c0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 shutdown_connections 2024-01-15T18:03:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7a4006dab0 0x7f7a4015da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7a1806bc10 0x7f7a1806e0d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7a401447f0 0x7f7a4015dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 --2- 172.21.15.26:0/659885164 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7a4006c470 0x7f7a401f5770 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 >> 172.21.15.26:0/659885164 conn(0x7f7a4006a3e0 msgr2=0x7f7a40159560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 shutdown_connections 2024-01-15T18:03:45.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.368+0000 7f7a45200640 1 -- 172.21.15.26:0/659885164 wait complete. 2024-01-15T18:03:45.391 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:03:45.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.622+0000 7f50f9600640 1 Processor -- start 2024-01-15T18:03:45.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.622+0000 7f50f9600640 1 -- start start 2024-01-15T18:03:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.624+0000 7f50f9600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f4071410 0x7f50f4146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.624+0000 7f50f9600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f50f4071f00 0x7f50f4166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.624+0000 7f50f9600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f40b2e10 con 0x7f50f406c7c0 2024-01-15T18:03:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.624+0000 7f50f9600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f406cc20 con 0x7f50f4071f00 2024-01-15T18:03:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.624+0000 7f50f9600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f406cec0 con 0x7f50f4071410 2024-01-15T18:03:45.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.624+0000 7f50f3400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f50f406c7c0 0x7f50f40717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42258/0 (socket says 172.21.15.26:42258) 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.624+0000 7f50f3400640 1 -- 172.21.15.26:0/2760651983 learned_addr learned my addr 172.21.15.26:0/2760651983 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f8c00640 1 --2- 172.21.15.26:0/2760651983 >> v2:172.21.15.124:3300/0 conn(0x7f50f4071f00 0x7f50f4166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883446094 0 0) 0x7f50f40b2e10 con 0x7f50f406c7c0 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50d4003520 con 0x7f50f406c7c0 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f3e00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f4071410 0x7f50f4146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 728150051 0 0) 0x7f50d4003520 con 0x7f50f406c7c0 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f4071410 msgr2=0x7f50f4146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f4071410 0x7f50f4146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 >> v2:172.21.15.124:3300/0 conn(0x7f50f4071f00 msgr2=0x7f50f4166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:45.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 --2- 172.21.15.26:0/2760651983 >> v2:172.21.15.124:3300/0 conn(0x7f50f4071f00 0x7f50f4166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f406d1a0 con 0x7f50f406c7c0 2024-01-15T18:03:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f50dc001670 con 0x7f50f406c7c0 2024-01-15T18:03:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 >> v1:172.21.15.26:6789/0 conn(0x7f50f406c7c0 legacy=0x7f50f40717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50d4003970 0x7f50d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f4071410 0x7f50d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d4004d00 0x7f50d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f406cc20 con 0x7f50d4004d00 2024-01-15T18:03:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f406cec0 con 0x7f50d4003970 2024-01-15T18:03:45.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.625+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d400ba00 con 0x7f50f4071410 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.626+0000 7f50f8c00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d4004d00 0x7f50d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.626+0000 7f50f3400640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f4071410 0x7f50d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.626+0000 7f50f8c00640 1 -- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f4071410 msgr2=0x7f50d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.626+0000 7f50f8c00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f4071410 0x7f50d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.626+0000 7f50f3e00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50d4003970 0x7f50d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.626+0000 7f50f8c00640 1 -- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50d4003970 msgr2=0x7f50d4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.626+0000 7f50f8c00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50d4003970 0x7f50d4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.626+0000 7f50f8c00640 1 -- 172.21.15.26:0/2760651983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d4003520 con 0x7f50d4004d00 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.627+0000 7f50f8c00640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d4004d00 0x7f50d4005180 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009fc0 tx=0x7f50d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa068e9700df383e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:45.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.627+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec02cc60 con 0x7f50d4004d00 2024-01-15T18:03:45.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.627+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec00f040 con 0x7f50d4004d00 2024-01-15T18:03:45.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.628+0000 7f50f2a00640 1 -- 172.21.15.26:0/2760651983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec034440 con 0x7f50d4004d00 2024-01-15T18:03:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.628+0000 7f50f9600640 1 -- 172.21.15.26:0/2760651983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d4004d00 msgr2=0x7f50d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.628+0000 7f50f9600640 1 --2- 172.21.15.26:0/2760651983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50d4004d00 0x7f50d4005180 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009fc0 tx=0x7f50d40057f0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.628+0000 7f50f8c00640 1 -- 172.21.15.26:0/2760651983 reap_dead start 2024-01-15T18:03:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.628+0000 7f50f9600640 1 -- 172.21.15.26:0/2760651983 shutdown_connections 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.628+0000 7f50f9600640 1 -- 172.21.15.26:0/2760651983 >> 172.21.15.26:0/2760651983 conn(0x7f50f406a800 msgr2=0x7f50f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.628+0000 7f50f9600640 1 -- 172.21.15.26:0/2760651983 shutdown_connections 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.628+0000 7f50f9600640 1 -- 172.21.15.26:0/2760651983 wait complete. 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.630+0000 7f50f9600640 1 Processor -- start 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.630+0000 7f50f9600640 1 -- start start 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.631+0000 7f50f9600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50d4004d00 0x7f50f41e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.631+0000 7f50f9600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f40ca850 0x7f50f41e2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.631+0000 7f50f9600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f41444a0 0x7f50f4144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.631+0000 7f50f9600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f41e34e0 con 0x7f50f41444a0 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.631+0000 7f50f9600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f4144e60 con 0x7f50d4004d00 2024-01-15T18:03:45.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.631+0000 7f50f9600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f412e450 con 0x7f50f40ca850 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f41444a0 0x7f50f4144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f41444a0 0x7f50f4144920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40710/0 (socket says 172.21.15.26:40710) 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f8c00640 1 -- 172.21.15.26:0/2181432203 learned_addr learned my addr 172.21.15.26:0/2181432203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f3e00640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50d4004d00 0x7f50f41e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f3400640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f40ca850 0x7f50f41e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f8c00640 1 -- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f40ca850 msgr2=0x7f50f41e2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f8c00640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f40ca850 0x7f50f41e2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f8c00640 1 -- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50d4004d00 msgr2=0x7f50f41e3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f8c00640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50d4004d00 0x7f50f41e3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f8c00640 1 -- 172.21.15.26:0/2181432203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f41e2970 con 0x7f50f41444a0 2024-01-15T18:03:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.632+0000 7f50f8c00640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f41444a0 0x7f50f4144920 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f50ec037040 tx=0x7f50ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.633+0000 7f50ebe00640 1 -- 172.21.15.26:0/2181432203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec043070 con 0x7f50f41444a0 2024-01-15T18:03:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.633+0000 7f50ebe00640 1 -- 172.21.15.26:0/2181432203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ec034b50 con 0x7f50f41444a0 2024-01-15T18:03:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.633+0000 7f50ebe00640 1 -- 172.21.15.26:0/2181432203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec003990 con 0x7f50f41444a0 2024-01-15T18:03:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.633+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f41e2c40 con 0x7f50f41444a0 2024-01-15T18:03:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.633+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f41e3200 con 0x7f50f41444a0 2024-01-15T18:03:45.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.634+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f412e220 con 0x7f50f41444a0 2024-01-15T18:03:45.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.635+0000 7f50ebe00640 1 -- 172.21.15.26:0/2181432203 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7f50ec03e440 con 0x7f50f41444a0 2024-01-15T18:03:45.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.636+0000 7f50ebe00640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50cc06bbf0 0x7f50cc06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:45.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.636+0000 7f50ebe00640 1 -- 172.21.15.26:0/2181432203 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f50ec0a9e80 con 0x7f50f41444a0 2024-01-15T18:03:45.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.637+0000 7f50f3e00640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50cc06bbf0 0x7f50cc06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.638+0000 7f50f3e00640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50cc06bbf0 0x7f50cc06e0b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f50e4006fd0 tx=0x7f50e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.640+0000 7f50ebe00640 1 -- 172.21.15.26:0/2181432203 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50f412e220 con 0x7f50f41444a0 2024-01-15T18:03:45.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.862+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f50f406da40 con 0x7f50cc06bbf0 2024-01-15T18:03:45.879 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:03:45.875+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:03:45.879 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:03:45.875+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:03:45.880 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:03:45.875+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:03:45.880 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:03:45.875+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:03:45.880 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:03:45.875+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:03:45.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.950+0000 7f50ebe00640 1 -- 172.21.15.26:0/2181432203 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f50e400c4b0 con 0x7f50cc06bbf0 2024-01-15T18:03:45.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50cc06bbf0 msgr2=0x7f50cc06e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50cc06bbf0 0x7f50cc06e0b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f50e4006fd0 tx=0x7f50e4008040 comp rx=0 tx=0).stop 2024-01-15T18:03:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f41444a0 msgr2=0x7f50f4144920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f41444a0 0x7f50f4144920 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f50ec037040 tx=0x7f50ec009750 comp rx=0 tx=0).stop 2024-01-15T18:03:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 shutdown_connections 2024-01-15T18:03:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50f40ca850 0x7f50f41e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50cc06bbf0 0x7f50cc06e0b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50f41444a0 0x7f50f4144920 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 --2- 172.21.15.26:0/2181432203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50d4004d00 0x7f50f41e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:45.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 >> 172.21.15.26:0/2181432203 conn(0x7f50f406a800 msgr2=0x7f50f40b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:45.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.953+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 shutdown_connections 2024-01-15T18:03:45.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:45.954+0000 7f50f9600640 1 -- 172.21.15.26:0/2181432203 wait complete. 2024-01-15T18:03:45.981 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:03:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.205+0000 7fd552000640 1 Processor -- start 2024-01-15T18:03:46.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.206+0000 7fd552000640 1 -- start start 2024-01-15T18:03:46.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.207+0000 7fd552000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd54c0cf9a0 0x7fd54c166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:46.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.207+0000 7fd552000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54c15c5f0 con 0x7fd54c13df80 2024-01-15T18:03:46.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.207+0000 7fd552000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54c06c470 con 0x7fd54c13e360 2024-01-15T18:03:46.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.207+0000 7fd552000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c06c750 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.208+0000 7fd54b400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd54c13df80 0x7fd54c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42268/0 (socket says 172.21.15.26:42268) 2024-01-15T18:03:46.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.208+0000 7fd54b400640 1 -- 172.21.15.26:0/717903974 learned_addr learned my addr 172.21.15.26:0/717903974 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:03:46.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.208+0000 7fd54be00640 1 --2- 172.21.15.26:0/717903974 >> v2:172.21.15.26:3300/0 conn(0x7fd54c0cf9a0 0x7fd54c166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:46.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.208+0000 7fd54a000640 1 -- 172.21.15.26:0/717903974 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491133194 0 0) 0x7fd54c15c5f0 con 0x7fd54c13df80 2024-01-15T18:03:46.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.208+0000 7fd54a000640 1 -- 172.21.15.26:0/717903974 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd52c003520 con 0x7fd54c13df80 2024-01-15T18:03:46.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.209+0000 7fd54be00640 1 -- 172.21.15.26:0/717903974 >> v1:172.21.15.124:6789/0 conn(0x7fd54c13e360 legacy=0x7fd54c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.209+0000 7fd54be00640 1 -- 172.21.15.26:0/717903974 >> v1:172.21.15.26:6789/0 conn(0x7fd54c13df80 legacy=0x7fd54c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.209+0000 7fd54a000640 1 -- 172.21.15.26:0/717903974 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291934818 0 0) 0x7fd52c003520 con 0x7fd54c13df80 2024-01-15T18:03:46.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.209+0000 7fd54be00640 1 -- 172.21.15.26:0/717903974 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c06ca30 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.210+0000 7fd54be00640 1 --2- 172.21.15.26:0/717903974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 0x7fd54c166030 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd544009d90 tx=0x7fd54402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee788f38a02e44c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:46.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.211+0000 7fd54a000640 1 -- 172.21.15.26:0/717903974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534001670 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.211+0000 7fd54a000640 1 -- 172.21.15.26:0/717903974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54402c940 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.211+0000 7fd54a000640 1 -- 172.21.15.26:0/717903974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54402ccb0 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.211+0000 7fd552000640 1 -- 172.21.15.26:0/717903974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 msgr2=0x7fd54c166030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.211+0000 7fd552000640 1 --2- 172.21.15.26:0/717903974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 0x7fd54c166030 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd544009d90 tx=0x7fd54402be10 comp rx=0 tx=0).stop 2024-01-15T18:03:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.211+0000 7fd552000640 1 -- 172.21.15.26:0/717903974 shutdown_connections 2024-01-15T18:03:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.211+0000 7fd552000640 1 --2- 172.21.15.26:0/717903974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 0x7fd54c166030 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.211+0000 7fd552000640 1 -- 172.21.15.26:0/717903974 >> 172.21.15.26:0/717903974 conn(0x7fd54c06a800 msgr2=0x7fd54c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.212+0000 7fd552000640 1 -- 172.21.15.26:0/717903974 shutdown_connections 2024-01-15T18:03:46.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.212+0000 7fd552000640 1 -- 172.21.15.26:0/717903974 wait complete. 2024-01-15T18:03:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.213+0000 7fd552000640 1 Processor -- start 2024-01-15T18:03:46.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.213+0000 7fd552000640 1 -- start start 2024-01-15T18:03:46.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.214+0000 7fd552000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 0x7fd54c158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:46.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.214+0000 7fd552000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd54c13df80 0x7fd54c0a8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:46.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.214+0000 7fd552000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd54c13e360 0x7fd54c1301d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:46.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.214+0000 7fd552000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c159dc0 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.214+0000 7fd552000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c1591b0 con 0x7fd54c13e360 2024-01-15T18:03:46.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.214+0000 7fd552000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c159490 con 0x7fd54c13df80 2024-01-15T18:03:46.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.215+0000 7fd54b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 0x7fd54c158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:46.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.215+0000 7fd54b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 0x7fd54c158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40728/0 (socket says 172.21.15.26:40728) 2024-01-15T18:03:46.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.215+0000 7fd54b400640 1 -- 172.21.15.26:0/3776552588 learned_addr learned my addr 172.21.15.26:0/3776552588 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.215+0000 7fd54b400640 1 -- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd54c13df80 msgr2=0x7fd54c0a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.215+0000 7fd54b400640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd54c13df80 0x7fd54c0a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.215+0000 7fd54b400640 1 -- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd54c13e360 msgr2=0x7fd54c1301d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.215+0000 7fd54b400640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd54c13e360 0x7fd54c1301d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.215+0000 7fd54b400640 1 -- 172.21.15.26:0/3776552588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd544009a70 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.215+0000 7fd54b400640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 0x7fd54c158ad0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd53400c780 tx=0x7fd53400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.216+0000 7fd543400640 1 -- 172.21.15.26:0/3776552588 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534005ad0 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.216+0000 7fd543400640 1 -- 172.21.15.26:0/3776552588 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534005c30 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.216+0000 7fd543400640 1 -- 172.21.15.26:0/3776552588 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534007070 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.216+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c159770 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.216+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c130710 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.217+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c130930 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.218+0000 7fd543400640 1 -- 172.21.15.26:0/3776552588 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 21) v1 ==== 90355+0+0 (secure 0 0 0) 0x7fd5340071d0 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.219+0000 7fd543400640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd52406bbc0 0x7fd52406e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:03:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.219+0000 7fd543400640 1 -- 172.21.15.26:0/3776552588 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd534088170 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.220+0000 7fd54aa00640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd52406bbc0 0x7fd52406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:03:46.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.221+0000 7fd54aa00640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd52406bbc0 0x7fd52406e080 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd53c000f60 tx=0x7fd53c0079d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:03:46.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.224+0000 7fd543400640 1 -- 172.21.15.26:0/3776552588 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd54c130930 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.445+0000 7fd543400640 1 -- 172.21.15.26:0/3776552588 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 22) v1 ==== 90406+0+0 (secure 0 0 0) 0x7fd534054e00 con 0x7fd54c0cf9a0 2024-01-15T18:03:46.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.455+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd54c06cff0 con 0x7fd52406bbc0 2024-01-15T18:03:46.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.470+0000 7fd543400640 1 -- 172.21.15.26:0/3776552588 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fd53c005080 con 0x7fd52406bbc0 2024-01-15T18:03:46.475 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c 2024-01-15T18:03:46.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd52406bbc0 msgr2=0x7fd52406e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:46.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd52406bbc0 0x7fd52406e080 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd53c000f60 tx=0x7fd53c0079d0 comp rx=0 tx=0).stop 2024-01-15T18:03:46.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 msgr2=0x7fd54c158ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:03:46.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 0x7fd54c158ad0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fd53400c780 tx=0x7fd53400cc40 comp rx=0 tx=0).stop 2024-01-15T18:03:46.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 shutdown_connections 2024-01-15T18:03:46.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd54c13df80 0x7fd54c0a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:46.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd52406bbc0 0x7fd52406e080 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:46.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd54c13e360 0x7fd54c1301d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:46.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 --2- 172.21.15.26:0/3776552588 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd54c0cf9a0 0x7fd54c158ad0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:03:46.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.473+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 >> 172.21.15.26:0/3776552588 conn(0x7fd54c06a800 msgr2=0x7fd54c0aeb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:03:46.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.474+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 shutdown_connections 2024-01-15T18:03:46.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:03:46.474+0000 7fd552000640 1 -- 172.21.15.26:0/3776552588 wait complete. 2024-01-15T18:03:46.498 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c 2024-01-15T18:03:46.498 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:03:46.498 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:03:46.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:03:46.548 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:03:58.676 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:03:58.685 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:04:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:04:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:04:02.794 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.1186 s, 66.6 MB/s 2024-01-15T18:04:02.796 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:04:02.796 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:04:02.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:04:02.855 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:04:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:04:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:04:17.495 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.5747 s, 73.7 MB/s 2024-01-15T18:04:17.497 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:04:17.497 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:04:17.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:04:17.554 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:04:28.828 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:04:28.838 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:04:32.613 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:04:32.613 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:04:32.613 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.9963 s, 71.6 MB/s 2024-01-15T18:04:32.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 10 2024-01-15T18:04:32.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.834+0000 7f7da7000640 1 Processor -- start 2024-01-15T18:04:32.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.834+0000 7f7da7000640 1 -- start start 2024-01-15T18:04:32.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.836+0000 7f7da7000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7da015e3b0 0x7f7da0164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.836+0000 7f7da7000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da00b1ee0 con 0x7f7da0143ae0 2024-01-15T18:04:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.836+0000 7f7da7000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da00b2780 con 0x7f7da00cf9a0 2024-01-15T18:04:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.836+0000 7f7da7000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da015e990 con 0x7f7da015e3b0 2024-01-15T18:04:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.836+0000 7f7d9fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7da0143ae0 0x7f7da00cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49280/0 (socket says 172.21.15.26:49280) 2024-01-15T18:04:32.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.836+0000 7f7d9fe00640 1 -- 172.21.15.26:0/3182384854 learned_addr learned my addr 172.21.15.26:0/3182384854 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:04:32.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7da5000640 1 --2- 172.21.15.26:0/3182384854 >> v2:172.21.15.124:3300/0 conn(0x7f7da015e3b0 0x7f7da0164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:32.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609348168 0 0) 0x7f7da00b1ee0 con 0x7f7da0143ae0 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d80005730 con 0x7f7da0143ae0 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 273344495 0 0) 0x7f7d80005730 con 0x7f7da0143ae0 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 >> v2:172.21.15.124:3300/0 conn(0x7f7da015e3b0 msgr2=0x7f7da0164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 --2- 172.21.15.26:0/3182384854 >> v2:172.21.15.124:3300/0 conn(0x7f7da015e3b0 0x7f7da0164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 >> v1:172.21.15.124:6789/0 conn(0x7f7da00cf9a0 legacy=0x7f7da015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da00b2130 con 0x7f7da0143ae0 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f7d90001670 con 0x7f7da0143ae0 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 >> v1:172.21.15.26:6789/0 conn(0x7f7da0143ae0 legacy=0x7f7da00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d80005b80 0x7f7d80005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:32.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d800069d0 0x7f7d80006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.837+0000 7f7d9ea00640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d80007390 0x7f7d8000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da015e990 con 0x7f7d800069d0 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8000e230 con 0x7f7d80005b80 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8000e3f0 con 0x7f7d80007390 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9f400640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d800069d0 0x7f7d80006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9fe00640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d80005b80 0x7f7d80005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9f400640 1 -- 172.21.15.26:0/3182384854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d80007390 msgr2=0x7f7d8000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9f400640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d80007390 0x7f7d8000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9f400640 1 -- 172.21.15.26:0/3182384854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d80005b80 msgr2=0x7f7d80005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9f400640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d80005b80 0x7f7d80005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7d9f400640 1 -- 172.21.15.26:0/3182384854 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80005730 con 0x7f7d800069d0 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.838+0000 7f7da5000640 1 -- 172.21.15.26:0/3182384854 reap_dead start 2024-01-15T18:04:32.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.839+0000 7f7d9f400640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d800069d0 0x7f7d80006e50 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7d88001750 tx=0x7f7d80007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bf8c65cc3732ef server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d880052d0 con 0x7f7d800069d0 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8802e050 con 0x7f7d800069d0 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7d9ea00640 1 -- 172.21.15.26:0/3182384854 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88003830 con 0x7f7d800069d0 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7da7000640 1 -- 172.21.15.26:0/3182384854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d800069d0 msgr2=0x7f7d80006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7da7000640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d800069d0 0x7f7d80006e50 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f7d88001750 tx=0x7f7d80007a00 comp rx=0 tx=0).stop 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7da7000640 1 -- 172.21.15.26:0/3182384854 shutdown_connections 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7da7000640 1 --2- 172.21.15.26:0/3182384854 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d800069d0 0x7f7d80006e50 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7da7000640 1 -- 172.21.15.26:0/3182384854 >> 172.21.15.26:0/3182384854 conn(0x7f7da006a800 msgr2=0x7f7da006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7da7000640 1 -- 172.21.15.26:0/3182384854 shutdown_connections 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.840+0000 7f7da7000640 1 -- 172.21.15.26:0/3182384854 wait complete. 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.842+0000 7f7da7000640 1 Processor -- start 2024-01-15T18:04:32.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.842+0000 7f7da7000640 1 -- start start 2024-01-15T18:04:32.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.843+0000 7f7da7000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d800069d0 0x7f7da0159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.843+0000 7f7da7000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7da00cdcf0 0x7f7da015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.843+0000 7f7da7000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da00ae270 0x7f7da01403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da7000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da015b090 con 0x7f7da00ae270 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da7000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da015b330 con 0x7f7da00cdcf0 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da7000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da015b610 con 0x7f7d800069d0 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da5000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da00ae270 0x7f7da01403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da5000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da00ae270 0x7f7da01403d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47846/0 (socket says 172.21.15.26:47846) 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da5000640 1 -- 172.21.15.26:0/3529615373 learned_addr learned my addr 172.21.15.26:0/3529615373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7d9f400640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7da00cdcf0 0x7f7da015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7d9fe00640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d800069d0 0x7f7da0159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da5000640 1 -- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d800069d0 msgr2=0x7f7da0159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da5000640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d800069d0 0x7f7da0159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da5000640 1 -- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7da00cdcf0 msgr2=0x7f7da015c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:32.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da5000640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7da00cdcf0 0x7f7da015c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:32.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.844+0000 7f7da5000640 1 -- 172.21.15.26:0/3529615373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da015b8f0 con 0x7f7da00ae270 2024-01-15T18:04:32.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.845+0000 7f7da5000640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da00ae270 0x7f7da01403d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7da0071f10 tx=0x7f7d9800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:32.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.845+0000 7f7d97e00640 1 -- 172.21.15.26:0/3529615373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98010040 con 0x7f7da00ae270 2024-01-15T18:04:32.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.845+0000 7f7d97e00640 1 -- 172.21.15.26:0/3529615373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9800bb70 con 0x7f7da00ae270 2024-01-15T18:04:32.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.845+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da015bbd0 con 0x7f7da00ae270 2024-01-15T18:04:32.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.845+0000 7f7d97e00640 1 -- 172.21.15.26:0/3529615373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98014400 con 0x7f7da00ae270 2024-01-15T18:04:32.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.845+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da0140910 con 0x7f7da00ae270 2024-01-15T18:04:32.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.846+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da00675d0 con 0x7f7da00ae270 2024-01-15T18:04:32.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.847+0000 7f7d97e00640 1 -- 172.21.15.26:0/3529615373 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f7d98014560 con 0x7f7da00ae270 2024-01-15T18:04:32.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.848+0000 7f7d97e00640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d7806bc80 0x7f7d7806e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:32.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.848+0000 7f7d97e00640 1 -- 172.21.15.26:0/3529615373 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f7d980889f0 con 0x7f7da00ae270 2024-01-15T18:04:32.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.849+0000 7f7d9fe00640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d7806bc80 0x7f7d7806e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:32.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.850+0000 7f7d9fe00640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d7806bc80 0x7f7d7806e140 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7d90001050 tx=0x7f7d90009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:32.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:32.853+0000 7f7d97e00640 1 -- 172.21.15.26:0/3529615373 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d98055110 con 0x7f7da00ae270 2024-01-15T18:04:33.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.048+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f7da00cf540 con 0x7f7da00ae270 2024-01-15T18:04:33.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.055+0000 7f7d97e00640 1 -- 172.21.15.26:0/3529615373 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v27)=0 v27) v1 ==== 136+0+0 (secure 0 0 0) 0x7f7d98058730 con 0x7f7da00ae270 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d7806bc80 msgr2=0x7f7d7806e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d7806bc80 0x7f7d7806e140 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7d90001050 tx=0x7f7d90009040 comp rx=0 tx=0).stop 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da00ae270 msgr2=0x7f7da01403d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da00ae270 0x7f7da01403d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7da0071f10 tx=0x7f7d9800ef30 comp rx=0 tx=0).stop 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 shutdown_connections 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d7806bc80 0x7f7d7806e140 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7da00ae270 0x7f7da01403d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7da00cdcf0 0x7f7da015c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 --2- 172.21.15.26:0/3529615373 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d800069d0 0x7f7da0159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 >> 172.21.15.26:0/3529615373 conn(0x7f7da006a800 msgr2=0x7f7da013fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:33.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 shutdown_connections 2024-01-15T18:04:33.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.058+0000 7f7da7000640 1 -- 172.21.15.26:0/3529615373 wait complete. 2024-01-15T18:04:33.084 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:04:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.308+0000 7f47e1600640 1 Processor -- start 2024-01-15T18:04:33.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.308+0000 7f47e1600640 1 -- start start 2024-01-15T18:04:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.310+0000 7f47e1600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47dc150760 0x7f47dc072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.310+0000 7f47e1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47dc15d300 0x7f47dc15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.310+0000 7f47e1600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc15dc80 con 0x7f47dc0b2bd0 2024-01-15T18:04:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.310+0000 7f47e1600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc0b3770 con 0x7f47dc150760 2024-01-15T18:04:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.310+0000 7f47e1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc071190 con 0x7f47dc15d300 2024-01-15T18:04:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.310+0000 7f47daa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47dc150760 0x7f47dc072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.311+0000 7f47daa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f47dc150760 0x7f47dc072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47856/0 (socket says 172.21.15.26:47856) 2024-01-15T18:04:33.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.311+0000 7f47daa00640 1 -- 172.21.15.26:0/3799467332 learned_addr learned my addr 172.21.15.26:0/3799467332 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.311+0000 7f47d3e00640 1 --2- 172.21.15.26:0/3799467332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47dc15d300 0x7f47dc15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.311+0000 7f47daa00640 1 -- 172.21.15.26:0/3799467332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47dc15d300 msgr2=0x7f47dc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.311+0000 7f47daa00640 1 --2- 172.21.15.26:0/3799467332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47dc15d300 0x7f47dc15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.311+0000 7f47daa00640 1 -- 172.21.15.26:0/3799467332 >> v1:172.21.15.124:6789/0 conn(0x7f47dc0b2bd0 legacy=0x7f47dc0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:33.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.311+0000 7f47daa00640 1 -- 172.21.15.26:0/3799467332 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc071470 con 0x7f47dc150760 2024-01-15T18:04:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.312+0000 7f47daa00640 1 --2- 172.21.15.26:0/3799467332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 0x7f47dc072230 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00aa10 tx=0x7f47cc02ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e72216bd46d41 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.313+0000 7f47da000640 1 -- 172.21.15.26:0/3799467332 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc02ec80 con 0x7f47dc150760 2024-01-15T18:04:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.313+0000 7f47da000640 1 -- 172.21.15.26:0/3799467332 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc02ede0 con 0x7f47dc150760 2024-01-15T18:04:33.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.313+0000 7f47da000640 1 -- 172.21.15.26:0/3799467332 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc036660 con 0x7f47dc150760 2024-01-15T18:04:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.313+0000 7f47e1600640 1 -- 172.21.15.26:0/3799467332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 msgr2=0x7f47dc072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.313+0000 7f47e1600640 1 --2- 172.21.15.26:0/3799467332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 0x7f47dc072230 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00aa10 tx=0x7f47cc02ded0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.314+0000 7f47e1600640 1 -- 172.21.15.26:0/3799467332 shutdown_connections 2024-01-15T18:04:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.314+0000 7f47e1600640 1 --2- 172.21.15.26:0/3799467332 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47dc15d300 0x7f47dc15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.314+0000 7f47e1600640 1 --2- 172.21.15.26:0/3799467332 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 0x7f47dc072230 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.314+0000 7f47e1600640 1 -- 172.21.15.26:0/3799467332 >> 172.21.15.26:0/3799467332 conn(0x7f47dc06a800 msgr2=0x7f47dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.314+0000 7f47e1600640 1 -- 172.21.15.26:0/3799467332 shutdown_connections 2024-01-15T18:04:33.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.314+0000 7f47e1600640 1 -- 172.21.15.26:0/3799467332 wait complete. 2024-01-15T18:04:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.315+0000 7f47e1600640 1 Processor -- start 2024-01-15T18:04:33.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.315+0000 7f47e1600640 1 -- start start 2024-01-15T18:04:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.316+0000 7f47e1600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47dc0b2bd0 0x7f47dc137790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.316+0000 7f47e1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 0x7f47dc1482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.316+0000 7f47e1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47dc15d300 0x7f47dc0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.316+0000 7f47e1600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0cdfc0 con 0x7f47dc150760 2024-01-15T18:04:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.316+0000 7f47e1600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0cd340 con 0x7f47dc0b2bd0 2024-01-15T18:04:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.316+0000 7f47e1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0cd620 con 0x7f47dc15d300 2024-01-15T18:04:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 0x7f47dc1482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 0x7f47dc1482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47872/0 (socket says 172.21.15.26:47872) 2024-01-15T18:04:33.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47d3e00640 1 -- 172.21.15.26:0/3168994647 learned_addr learned my addr 172.21.15.26:0/3168994647 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47daa00640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47dc0b2bd0 0x7f47dc137790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47d3e00640 1 -- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47dc15d300 msgr2=0x7f47dc0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47d3e00640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47dc15d300 0x7f47dc0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47d3e00640 1 -- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47dc0b2bd0 msgr2=0x7f47dc137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47d3e00640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47dc0b2bd0 0x7f47dc137790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47d3e00640 1 -- 172.21.15.26:0/3168994647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc00a670 con 0x7f47dc150760 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.317+0000 7f47d3e00640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 0x7f47dc1482b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f47c400d5f0 tx=0x7f47c400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.318+0000 7f47d2a00640 1 -- 172.21.15.26:0/3168994647 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c40125f0 con 0x7f47dc150760 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.318+0000 7f47d2a00640 1 -- 172.21.15.26:0/3168994647 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c4012750 con 0x7f47dc150760 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.318+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc0cd900 con 0x7f47dc150760 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.318+0000 7f47d2a00640 1 -- 172.21.15.26:0/3168994647 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4012b00 con 0x7f47dc150760 2024-01-15T18:04:33.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.318+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc137cd0 con 0x7f47dc150760 2024-01-15T18:04:33.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.319+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc072230 con 0x7f47dc150760 2024-01-15T18:04:33.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.320+0000 7f47d2a00640 1 -- 172.21.15.26:0/3168994647 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f47c4012c60 con 0x7f47dc150760 2024-01-15T18:04:33.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.321+0000 7f47d2a00640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47b406bc50 0x7f47b406e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.321+0000 7f47d2a00640 1 -- 172.21.15.26:0/3168994647 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f47c4088bb0 con 0x7f47dc150760 2024-01-15T18:04:33.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.322+0000 7f47daa00640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47b406bc50 0x7f47b406e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.323+0000 7f47daa00640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47b406bc50 0x7f47b406e110 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f47cc038000 tx=0x7f47cc032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:33.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.326+0000 7f47d2a00640 1 -- 172.21.15.26:0/3168994647 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47c4055270 con 0x7f47dc150760 2024-01-15T18:04:33.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.555+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7f47dc071d10 con 0x7f47b406bc50 2024-01-15T18:04:33.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.590+0000 7f47d2a00640 1 -- 172.21.15.26:0/3168994647 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f47cc0365b0 con 0x7f47b406bc50 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47b406bc50 msgr2=0x7f47b406e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47b406bc50 0x7f47b406e110 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f47cc038000 tx=0x7f47cc032000 comp rx=0 tx=0).stop 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 msgr2=0x7f47dc1482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 0x7f47dc1482b0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f47c400d5f0 tx=0x7f47c400dab0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 shutdown_connections 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f47dc15d300 0x7f47dc0ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47b406bc50 0x7f47b406e110 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f47dc150760 0x7f47dc1482b0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 --2- 172.21.15.26:0/3168994647 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f47dc0b2bd0 0x7f47dc137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 >> 172.21.15.26:0/3168994647 conn(0x7f47dc06a800 msgr2=0x7f47dc0cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:33.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 shutdown_connections 2024-01-15T18:04:33.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.593+0000 7f47e1600640 1 -- 172.21.15.26:0/3168994647 wait complete. 2024-01-15T18:04:33.618 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:04:33.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.837+0000 7fd3cdc00640 1 Processor -- start 2024-01-15T18:04:33.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.837+0000 7fd3cdc00640 1 -- start start 2024-01-15T18:04:33.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.839+0000 7fd3cdc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3c80ccb50 0x7fd3c8070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.839+0000 7fd3cdc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c815f4e0 con 0x7fd3c80adf20 2024-01-15T18:04:33.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.839+0000 7fd3cdc00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c8071130 con 0x7fd3c815bf10 2024-01-15T18:04:33.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.839+0000 7fd3cdc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c815f970 con 0x7fd3c80ccb50 2024-01-15T18:04:33.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.839+0000 7fd3c7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3c80ccb50 0x7fd3c8070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.839+0000 7fd3c7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd3c80ccb50 0x7fd3c8070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47880/0 (socket says 172.21.15.26:47880) 2024-01-15T18:04:33.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.839+0000 7fd3c7400640 1 -- 172.21.15.26:0/2181561634 learned_addr learned my addr 172.21.15.26:0/2181561634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:33.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.840+0000 7fd3c7400640 1 -- 172.21.15.26:0/2181561634 >> v1:172.21.15.124:6789/0 conn(0x7fd3c815bf10 legacy=0x7fd3c8143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:33.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.840+0000 7fd3c7400640 1 -- 172.21.15.26:0/2181561634 >> v1:172.21.15.26:6789/0 conn(0x7fd3c80adf20 legacy=0x7fd3c80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.840+0000 7fd3c7400640 1 -- 172.21.15.26:0/2181561634 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c815ebb0 con 0x7fd3c80ccb50 2024-01-15T18:04:33.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.841+0000 7fd3c7400640 1 --2- 172.21.15.26:0/2181561634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c80ccb50 0x7fd3c8070a30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009870 tx=0x7fd3b002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3a7f3d8e0706ec server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:33.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.842+0000 7fd3c6a00640 1 -- 172.21.15.26:0/2181561634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0031860 con 0x7fd3c80ccb50 2024-01-15T18:04:33.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.842+0000 7fd3c6a00640 1 -- 172.21.15.26:0/2181561634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b00319c0 con 0x7fd3c80ccb50 2024-01-15T18:04:33.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.842+0000 7fd3c6a00640 1 -- 172.21.15.26:0/2181561634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0031d50 con 0x7fd3c80ccb50 2024-01-15T18:04:33.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.842+0000 7fd3cdc00640 1 -- 172.21.15.26:0/2181561634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c80ccb50 msgr2=0x7fd3c8070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.842+0000 7fd3cdc00640 1 --2- 172.21.15.26:0/2181561634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c80ccb50 0x7fd3c8070a30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0009870 tx=0x7fd3b002b900 comp rx=0 tx=0).stop 2024-01-15T18:04:33.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.842+0000 7fd3cdc00640 1 -- 172.21.15.26:0/2181561634 shutdown_connections 2024-01-15T18:04:33.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.842+0000 7fd3cdc00640 1 --2- 172.21.15.26:0/2181561634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c80ccb50 0x7fd3c8070a30 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.843+0000 7fd3cdc00640 1 -- 172.21.15.26:0/2181561634 >> 172.21.15.26:0/2181561634 conn(0x7fd3c806a7d0 msgr2=0x7fd3c806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:33.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.843+0000 7fd3cdc00640 1 -- 172.21.15.26:0/2181561634 shutdown_connections 2024-01-15T18:04:33.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.843+0000 7fd3cdc00640 1 -- 172.21.15.26:0/2181561634 wait complete. 2024-01-15T18:04:33.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.845+0000 7fd3cdc00640 1 Processor -- start 2024-01-15T18:04:33.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.845+0000 7fd3cdc00640 1 -- start start 2024-01-15T18:04:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.846+0000 7fd3cdc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c80adf20 0x7fd3c80aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.846+0000 7fd3cdc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c80ccb50 0x7fd3c813f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cdc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c815bf10 0x7fd3c813f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cdc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c80ab920 con 0x7fd3c815bf10 2024-01-15T18:04:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cdc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c813fcf0 con 0x7fd3c80ccb50 2024-01-15T18:04:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cdc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c813ffc0 con 0x7fd3c80adf20 2024-01-15T18:04:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cd200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c815bf10 0x7fd3c813f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cd200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c815bf10 0x7fd3c813f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47892/0 (socket says 172.21.15.26:47892) 2024-01-15T18:04:33.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cd200640 1 -- 172.21.15.26:0/75665758 learned_addr learned my addr 172.21.15.26:0/75665758 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3c7e00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c80adf20 0x7fd3c80aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3c7400640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c80ccb50 0x7fd3c813f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cd200640 1 -- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c80adf20 msgr2=0x7fd3c80aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cd200640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c80adf20 0x7fd3c80aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cd200640 1 -- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c80ccb50 msgr2=0x7fd3c813f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cd200640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c80ccb50 0x7fd3c813f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.847+0000 7fd3cd200640 1 -- 172.21.15.26:0/75665758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b00094d0 con 0x7fd3c815bf10 2024-01-15T18:04:33.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.848+0000 7fd3c7e00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c80adf20 0x7fd3c80aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:04:33.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.848+0000 7fd3cd200640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c815bf10 0x7fd3c813f7b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c780 tx=0x7fd3c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:33.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.848+0000 7fd3bfe00640 1 -- 172.21.15.26:0/75665758 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0004b60 con 0x7fd3c815bf10 2024-01-15T18:04:33.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.848+0000 7fd3bfe00640 1 -- 172.21.15.26:0/75665758 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c000ce90 con 0x7fd3c815bf10 2024-01-15T18:04:33.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.848+0000 7fd3bfe00640 1 -- 172.21.15.26:0/75665758 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c000a850 con 0x7fd3c815bf10 2024-01-15T18:04:33.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.848+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c81402a0 con 0x7fd3c815bf10 2024-01-15T18:04:33.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.848+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c8140860 con 0x7fd3c815bf10 2024-01-15T18:04:33.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.850+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c8067450 con 0x7fd3c815bf10 2024-01-15T18:04:33.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.851+0000 7fd3bfe00640 1 -- 172.21.15.26:0/75665758 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) v1 ==== 90406+0+0 (secure 0 0 0) 0x7fd3c000a9b0 con 0x7fd3c815bf10 2024-01-15T18:04:33.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.853+0000 7fd3bfe00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd3a006bc50 0x7fd3a006e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:33.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.853+0000 7fd3bfe00640 1 -- 172.21.15.26:0/75665758 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fd3c00888c0 con 0x7fd3c815bf10 2024-01-15T18:04:33.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.854+0000 7fd3c7e00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd3a006bc50 0x7fd3a006e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:33.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.855+0000 7fd3c7e00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd3a006bc50 0x7fd3a006e110 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8007fd0 tx=0x7fd3b8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:33.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:33.859+0000 7fd3bfe00640 1 -- 172.21.15.26:0/75665758 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3c0054f80 con 0x7fd3c815bf10 2024-01-15T18:04:34.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.082+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3c815f170 con 0x7fd3a006bc50 2024-01-15T18:04:34.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.095+0000 7fd3bfe00640 1 -- 172.21.15.26:0/75665758 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fd3b8001c90 con 0x7fd3a006bc50 2024-01-15T18:04:34.099 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd3a006bc50 msgr2=0x7fd3a006e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd3a006bc50 0x7fd3a006e110 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8007fd0 tx=0x7fd3b8009040 comp rx=0 tx=0).stop 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c815bf10 msgr2=0x7fd3c813f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c815bf10 0x7fd3c813f7b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000c780 tx=0x7fd3c000cc40 comp rx=0 tx=0).stop 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 shutdown_connections 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd3a006bc50 0x7fd3a006e110 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd3c815bf10 0x7fd3c813f7b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd3c80ccb50 0x7fd3c813f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 --2- 172.21.15.26:0/75665758 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd3c80adf20 0x7fd3c80aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:34.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 >> 172.21.15.26:0/75665758 conn(0x7fd3c806a7d0 msgr2=0x7fd3c815b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 shutdown_connections 2024-01-15T18:04:34.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:34.097+0000 7fd3cdc00640 1 -- 172.21.15.26:0/75665758 wait complete. 2024-01-15T18:04:34.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:04:34.121 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c) 2024-01-15T18:04:34.184 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c 2024-01-15T18:04:34.184 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:04:34.184 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:04:34.185 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:04:36.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:04:36.186 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c) 2024-01-15T18:04:36.259 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/9f63cf19-dd36-4699-b64e-578fe5009c1c 2024-01-15T18:04:36.260 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:04:36.260 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:04:36.261 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:04:36.261 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone4181 2024-01-15T18:04:36.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.489+0000 7ff884400640 1 Processor -- start 2024-01-15T18:04:36.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.489+0000 7ff884400640 1 -- start start 2024-01-15T18:04:36.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.491+0000 7ff884400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff87c13df80 0x7ff87c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.491+0000 7ff884400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff87c13e360 0x7ff87c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.491+0000 7ff884400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c06c470 con 0x7ff87c15f9c0 2024-01-15T18:04:36.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.491+0000 7ff884400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c06c750 con 0x7ff87c13e360 2024-01-15T18:04:36.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.491+0000 7ff884400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c06ca30 con 0x7ff87c13df80 2024-01-15T18:04:36.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.492+0000 7ff882400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff87c15f9c0 0x7ff87c1408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49290/0 (socket says 172.21.15.26:49290) 2024-01-15T18:04:36.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.492+0000 7ff882400640 1 -- 172.21.15.26:0/2838922824 learned_addr learned my addr 172.21.15.26:0/2838922824 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:04:36.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.492+0000 7ff881000640 1 --2- 172.21.15.26:0/2838922824 >> v2:172.21.15.26:3300/0 conn(0x7ff87c13e360 0x7ff87c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.492+0000 7ff881a00640 1 --2- 172.21.15.26:0/2838922824 >> v2:172.21.15.124:3300/0 conn(0x7ff87c13df80 0x7ff87c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.492+0000 7ff881000640 1 -- 172.21.15.26:0/2838922824 >> v2:172.21.15.124:3300/0 conn(0x7ff87c13df80 msgr2=0x7ff87c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:36.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.492+0000 7ff881000640 1 --2- 172.21.15.26:0/2838922824 >> v2:172.21.15.124:3300/0 conn(0x7ff87c13df80 0x7ff87c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.492+0000 7ff881000640 1 -- 172.21.15.26:0/2838922824 >> v1:172.21.15.26:6789/0 conn(0x7ff87c15f9c0 legacy=0x7ff87c1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.492+0000 7ff881000640 1 -- 172.21.15.26:0/2838922824 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c06cd10 con 0x7ff87c13e360 2024-01-15T18:04:36.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.493+0000 7ff881000640 1 --2- 172.21.15.26:0/2838922824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13e360 0x7ff87c0cf290 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff87c072770 tx=0x7ff86402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55aa9afb10c9208 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:36.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff87be00640 1 -- 172.21.15.26:0/2838922824 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff864031830 con 0x7ff87c13e360 2024-01-15T18:04:36.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff87be00640 1 -- 172.21.15.26:0/2838922824 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff864031990 con 0x7ff87c13e360 2024-01-15T18:04:36.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff87be00640 1 -- 172.21.15.26:0/2838922824 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff864031d20 con 0x7ff87c13e360 2024-01-15T18:04:36.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff884400640 1 -- 172.21.15.26:0/2838922824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13e360 msgr2=0x7ff87c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff884400640 1 --2- 172.21.15.26:0/2838922824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13e360 0x7ff87c0cf290 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff87c072770 tx=0x7ff86402b8d0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff884400640 1 -- 172.21.15.26:0/2838922824 shutdown_connections 2024-01-15T18:04:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff884400640 1 --2- 172.21.15.26:0/2838922824 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13e360 0x7ff87c0cf290 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff884400640 1 --2- 172.21.15.26:0/2838922824 >> v2:172.21.15.124:3300/0 conn(0x7ff87c13df80 0x7ff87c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff884400640 1 -- 172.21.15.26:0/2838922824 >> 172.21.15.26:0/2838922824 conn(0x7ff87c06a800 msgr2=0x7ff87c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff884400640 1 -- 172.21.15.26:0/2838922824 shutdown_connections 2024-01-15T18:04:36.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.494+0000 7ff884400640 1 -- 172.21.15.26:0/2838922824 wait complete. 2024-01-15T18:04:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.496+0000 7ff884400640 1 Processor -- start 2024-01-15T18:04:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.496+0000 7ff884400640 1 -- start start 2024-01-15T18:04:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.497+0000 7ff884400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13df80 0x7ff87c152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.497+0000 7ff884400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff87c13e360 0x7ff87c1e5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.497+0000 7ff884400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff87c15f9c0 0x7ff87c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.497+0000 7ff884400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1e5a50 con 0x7ff87c13df80 2024-01-15T18:04:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff884400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1e7dc0 con 0x7ff87c15f9c0 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff884400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1e80a0 con 0x7ff87c13e360 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff881a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13df80 0x7ff87c152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff881a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13df80 0x7ff87c152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47910/0 (socket says 172.21.15.26:47910) 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff881a00640 1 -- 172.21.15.26:0/1201088439 learned_addr learned my addr 172.21.15.26:0/1201088439 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff881a00640 1 -- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff87c13e360 msgr2=0x7ff87c1e5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff881a00640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff87c13e360 0x7ff87c1e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff881a00640 1 -- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff87c15f9c0 msgr2=0x7ff87c15a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff882400640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff87c15f9c0 0x7ff87c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff881a00640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff87c15f9c0 0x7ff87c15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.498+0000 7ff881a00640 1 -- 172.21.15.26:0/1201088439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff864009400 con 0x7ff87c13df80 2024-01-15T18:04:36.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.499+0000 7ff881a00640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13df80 0x7ff87c152640 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d5f0 tx=0x7ff86c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:36.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.499+0000 7ff879600640 1 -- 172.21.15.26:0/1201088439 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0125f0 con 0x7ff87c13df80 2024-01-15T18:04:36.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.499+0000 7ff879600640 1 -- 172.21.15.26:0/1201088439 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c012750 con 0x7ff87c13df80 2024-01-15T18:04:36.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.499+0000 7ff879600640 1 -- 172.21.15.26:0/1201088439 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c012b00 con 0x7ff87c13df80 2024-01-15T18:04:36.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.499+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c1e8380 con 0x7ff87c13df80 2024-01-15T18:04:36.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.499+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c1e5cd0 con 0x7ff87c13df80 2024-01-15T18:04:36.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.500+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87c0675d0 con 0x7ff87c13df80 2024-01-15T18:04:36.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.501+0000 7ff879600640 1 -- 172.21.15.26:0/1201088439 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) v1 ==== 90406+0+0 (secure 0 0 0) 0x7ff86c012c60 con 0x7ff87c13df80 2024-01-15T18:04:36.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.502+0000 7ff879600640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff85406bc50 0x7ff85406e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:36.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.502+0000 7ff879600640 1 -- 172.21.15.26:0/1201088439 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7ff86c088bb0 con 0x7ff87c13df80 2024-01-15T18:04:36.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.503+0000 7ff881000640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff85406bc50 0x7ff85406e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:36.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.504+0000 7ff881000640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff85406bc50 0x7ff85406e110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff864036000 tx=0x7ff864030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:36.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.507+0000 7ff879600640 1 -- 172.21.15.26:0/1201088439 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff86c055270 con 0x7ff87c13df80 2024-01-15T18:04:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.735+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone4181", "target": ["mon-mgr", ""]}) v1 -- 0x7ff87c1e7f30 con 0x7ff85406bc50 2024-01-15T18:04:36.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.898+0000 7ff879600640 1 -- 172.21.15.26:0/1201088439 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff87c1e7f30 con 0x7ff85406bc50 2024-01-15T18:04:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff85406bc50 msgr2=0x7ff85406e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff85406bc50 0x7ff85406e110 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff864036000 tx=0x7ff864030000 comp rx=0 tx=0).stop 2024-01-15T18:04:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13df80 msgr2=0x7ff87c152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13df80 0x7ff87c152640 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d5f0 tx=0x7ff86c00dab0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 shutdown_connections 2024-01-15T18:04:36.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff87c13e360 0x7ff87c1e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff85406bc50 0x7ff85406e110 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff87c15f9c0 0x7ff87c15a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 --2- 172.21.15.26:0/1201088439 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff87c13df80 0x7ff87c152640 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:36.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.901+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 >> 172.21.15.26:0/1201088439 conn(0x7ff87c06a800 msgr2=0x7ff87c0cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:36.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.902+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 shutdown_connections 2024-01-15T18:04:36.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:36.902+0000 7ff884400640 1 -- 172.21.15.26:0/1201088439 wait complete. 2024-01-15T18:04:36.909 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.906+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:36.909 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.906+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:36.909 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.906+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:36.909 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.906+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:36.910 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.906+0000 7f8f5aa00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:36.930 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone4182 2024-01-15T18:04:36.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.939+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:36.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.940+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:36.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.940+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:36.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.940+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:36.943 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:36.940+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.156+0000 7fc02f200640 1 Processor -- start 2024-01-15T18:04:37.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.156+0000 7fc02f200640 1 -- start start 2024-01-15T18:04:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.157+0000 7fc02f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0300d0100 0x7fc0300b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.157+0000 7fc02f200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0300d0f40 0x7fc0300d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.157+0000 7fc02f200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0300d0a10 con 0x7fc030160920 2024-01-15T18:04:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.157+0000 7fc02f200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0300cf080 con 0x7fc0300d0f40 2024-01-15T18:04:37.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.157+0000 7fc02f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300c6be0 con 0x7fc0300d0100 2024-01-15T18:04:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.158+0000 7fc02e800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0300d0f40 0x7fc0300d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.158+0000 7fc02d400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0300d0100 0x7fc0300b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.158+0000 7fc02e800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0300d0f40 0x7fc0300d13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:32786/0 (socket says 172.21.15.26:32786) 2024-01-15T18:04:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.158+0000 7fc02e800640 1 -- 172.21.15.26:0/3026677703 learned_addr learned my addr 172.21.15.26:0/3026677703 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:37.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.158+0000 7fc02de00640 1 --1- 172.21.15.26:0/3026677703 >> v1:172.21.15.124:6789/0 conn(0x7fc030160920 0x7fc0300cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:50768/0 (socket says 172.21.15.26:50768) 2024-01-15T18:04:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.159+0000 7fc02e800640 1 -- 172.21.15.26:0/3026677703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0300d0100 msgr2=0x7fc0300b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.159+0000 7fc02e800640 1 --2- 172.21.15.26:0/3026677703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0300d0100 0x7fc0300b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.159+0000 7fc02e800640 1 -- 172.21.15.26:0/3026677703 >> v1:172.21.15.124:6789/0 conn(0x7fc030160920 legacy=0x7fc0300cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.159+0000 7fc02ca00640 1 -- 172.21.15.26:0/3026677703 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430089387 0 0) 0x7fc0300d0a10 con 0x7fc030160920 2024-01-15T18:04:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.159+0000 7fc02e800640 1 -- 172.21.15.26:0/3026677703 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0300ce7e0 con 0x7fc0300d0f40 2024-01-15T18:04:37.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.159+0000 7fc02d400640 1 --2- 172.21.15.26:0/3026677703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0300d0100 0x7fc0300b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:04:37.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.161+0000 7fc02e800640 1 --2- 172.21.15.26:0/3026677703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0300d0f40 0x7fc0300d13f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc028009e50 tx=0x7fc02802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c958d3fba82a5838 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:37.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.161+0000 7fc02ca00640 1 -- 172.21.15.26:0/3026677703 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02802cb00 con 0x7fc0300d0f40 2024-01-15T18:04:37.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.161+0000 7fc02ca00640 1 -- 172.21.15.26:0/3026677703 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02802cc60 con 0x7fc0300d0f40 2024-01-15T18:04:37.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.161+0000 7fc02ca00640 1 -- 172.21.15.26:0/3026677703 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0280343f0 con 0x7fc0300d0f40 2024-01-15T18:04:37.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.162+0000 7fc02f200640 1 -- 172.21.15.26:0/3026677703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0300d0f40 msgr2=0x7fc0300d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.162+0000 7fc02f200640 1 --2- 172.21.15.26:0/3026677703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0300d0f40 0x7fc0300d13f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc028009e50 tx=0x7fc02802bf60 comp rx=0 tx=0).stop 2024-01-15T18:04:37.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.162+0000 7fc02f200640 1 -- 172.21.15.26:0/3026677703 shutdown_connections 2024-01-15T18:04:37.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.162+0000 7fc02f200640 1 --2- 172.21.15.26:0/3026677703 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0300d0100 0x7fc0300b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.162+0000 7fc02f200640 1 --2- 172.21.15.26:0/3026677703 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0300d0f40 0x7fc0300d13f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.162+0000 7fc02f200640 1 -- 172.21.15.26:0/3026677703 >> 172.21.15.26:0/3026677703 conn(0x7fc0300d3090 msgr2=0x7fc0300d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:37.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.162+0000 7fc02f200640 1 -- 172.21.15.26:0/3026677703 shutdown_connections 2024-01-15T18:04:37.167 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.163+0000 7fc02f200640 1 -- 172.21.15.26:0/3026677703 wait complete. 2024-01-15T18:04:37.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.164+0000 7fc02f200640 1 Processor -- start 2024-01-15T18:04:37.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.165+0000 7fc02f200640 1 -- start start 2024-01-15T18:04:37.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.165+0000 7fc02f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0300d0100 0x7fc0301f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02f200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0300d0f40 0x7fc030158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc030160920 0x7fc0300b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0301f4040 con 0x7fc0300d0100 2024-01-15T18:04:37.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02f200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300b76c0 con 0x7fc0300d0f40 2024-01-15T18:04:37.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0301f3430 con 0x7fc030160920 2024-01-15T18:04:37.171 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0300d0100 0x7fc0301f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0300d0100 0x7fc0301f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47912/0 (socket says 172.21.15.26:47912) 2024-01-15T18:04:37.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02de00640 1 -- 172.21.15.26:0/3054164470 learned_addr learned my addr 172.21.15.26:0/3054164470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:37.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02d400640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0300d0f40 0x7fc030158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02de00640 1 -- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc030160920 msgr2=0x7fc0300b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02de00640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc030160920 0x7fc0300b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02de00640 1 -- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0300d0f40 msgr2=0x7fc030158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02de00640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0300d0f40 0x7fc030158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.166+0000 7fc02de00640 1 -- 172.21.15.26:0/3054164470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc018009400 con 0x7fc0300d0100 2024-01-15T18:04:37.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.167+0000 7fc02de00640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0300d0100 0x7fc0301f4230 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c430 tx=0x7fc02000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:37.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.167+0000 7fc026000640 1 -- 172.21.15.26:0/3054164470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0200074e0 con 0x7fc0300d0100 2024-01-15T18:04:37.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.167+0000 7fc026000640 1 -- 172.21.15.26:0/3054164470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc020007640 con 0x7fc0300d0100 2024-01-15T18:04:37.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.167+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc028009b30 con 0x7fc0300d0100 2024-01-15T18:04:37.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.167+0000 7fc026000640 1 -- 172.21.15.26:0/3054164470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020004a20 con 0x7fc0300d0100 2024-01-15T18:04:37.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.167+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0301f39f0 con 0x7fc0300d0100 2024-01-15T18:04:37.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.168+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc030067450 con 0x7fc0300d0100 2024-01-15T18:04:37.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.169+0000 7fc026000640 1 -- 172.21.15.26:0/3054164470 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) v1 ==== 90406+0+0 (secure 0 0 0) 0x7fc020004b80 con 0x7fc0300d0100 2024-01-15T18:04:37.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.170+0000 7fc026000640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc00806bc50 0x7fc00806e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.170+0000 7fc026000640 1 -- 172.21.15.26:0/3054164470 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fc020088500 con 0x7fc0300d0100 2024-01-15T18:04:37.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.171+0000 7fc02d400640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc00806bc50 0x7fc00806e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.172+0000 7fc02d400640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc00806bc50 0x7fc00806e110 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc03015b250 tx=0x7fc018008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:37.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.175+0000 7fc026000640 1 -- 172.21.15.26:0/3054164470 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc020054bc0 con 0x7fc0300d0100 2024-01-15T18:04:37.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.408+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone4182", "target": ["mon-mgr", ""]}) v1 -- 0x7fc030160d00 con 0x7fc00806bc50 2024-01-15T18:04:37.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.559+0000 7fc026000640 1 -- 172.21.15.26:0/3054164470 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc0180098b0 con 0x7fc00806bc50 2024-01-15T18:04:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.562+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc00806bc50 msgr2=0x7fc00806e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.562+0000 7fc02f200640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc00806bc50 0x7fc00806e110 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc03015b250 tx=0x7fc018008040 comp rx=0 tx=0).stop 2024-01-15T18:04:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.562+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0300d0100 msgr2=0x7fc0301f4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.562+0000 7fc02f200640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0300d0100 0x7fc0301f4230 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc02000c430 tx=0x7fc02000c8f0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.563+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 shutdown_connections 2024-01-15T18:04:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.563+0000 7fc02f200640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc030160920 0x7fc0300b7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.563+0000 7fc02f200640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc00806bc50 0x7fc00806e110 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.563+0000 7fc02f200640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0300d0f40 0x7fc030158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.563+0000 7fc02f200640 1 --2- 172.21.15.26:0/3054164470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0300d0100 0x7fc0301f4230 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.563+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 >> 172.21.15.26:0/3054164470 conn(0x7fc0300d3090 msgr2=0x7fc0300cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.563+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 shutdown_connections 2024-01-15T18:04:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.563+0000 7fc02f200640 1 -- 172.21.15.26:0/3054164470 wait complete. 2024-01-15T18:04:37.591 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone4183 2024-01-15T18:04:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.808+0000 7f7223e00640 1 Processor -- start 2024-01-15T18:04:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.809+0000 7f7223e00640 1 -- start start 2024-01-15T18:04:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.810+0000 7f7223e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f721c0b3ae0 0x7f721c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.810+0000 7f7223e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c070c40 con 0x7f721c0cf290 2024-01-15T18:04:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.810+0000 7f7223e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c06c470 con 0x7f721c0ae5a0 2024-01-15T18:04:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.810+0000 7f7223e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c071450 con 0x7f721c0b3ae0 2024-01-15T18:04:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.811+0000 7f7221e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f721c0cf290 0x7f721c0b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49306/0 (socket says 172.21.15.26:49306) 2024-01-15T18:04:37.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.811+0000 7f7221e00640 1 -- 172.21.15.26:0/759719232 learned_addr learned my addr 172.21.15.26:0/759719232 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:04:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.811+0000 7f7220a00640 1 --2- 172.21.15.26:0/759719232 >> v2:172.21.15.124:3300/0 conn(0x7f721c0b3ae0 0x7f721c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.811+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137849181 0 0) 0x7f721c070c40 con 0x7f721c0cf290 2024-01-15T18:04:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.811+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71fc003520 con 0x7f721c0cf290 2024-01-15T18:04:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1857419160 0 0) 0x7f71fc003520 con 0x7f721c0cf290 2024-01-15T18:04:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 >> v2:172.21.15.124:3300/0 conn(0x7f721c0b3ae0 msgr2=0x7f721c15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 --2- 172.21.15.26:0/759719232 >> v2:172.21.15.124:3300/0 conn(0x7f721c0b3ae0 0x7f721c15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 >> v1:172.21.15.124:6789/0 conn(0x7f721c0ae5a0 legacy=0x7f721c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c06c750 con 0x7f721c0cf290 2024-01-15T18:04:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f72140019f0 con 0x7f721c0cf290 2024-01-15T18:04:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 >> v1:172.21.15.26:6789/0 conn(0x7f721c0cf290 legacy=0x7f721c0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71fc0039c0 0x7f71fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71fc0047c0 0x7f71fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71fc005180 0x7f71fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c071450 con 0x7f71fc0047c0 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc00c060 con 0x7f71fc0039c0 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc00c220 con 0x7f71fc005180 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f7220a00640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71fc0047c0 0x7f71fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.812+0000 7f7221400640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71fc0039c0 0x7f71fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.813+0000 7f7220a00640 1 -- 172.21.15.26:0/759719232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71fc005180 msgr2=0x7f71fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.813+0000 7f7220a00640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71fc005180 0x7f71fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.813+0000 7f7220a00640 1 -- 172.21.15.26:0/759719232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71fc0039c0 msgr2=0x7f71fc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.813+0000 7f7220a00640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71fc0039c0 0x7f71fc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.813+0000 7f7220a00640 1 -- 172.21.15.26:0/759719232 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc003520 con 0x7f71fc0047c0 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.813+0000 7f7221400640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f71fc0039c0 0x7f71fc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.813+0000 7f7221e00640 1 -- 172.21.15.26:0/759719232 reap_dead start 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.814+0000 7f7220a00640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71fc0047c0 0x7f71fc004c40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7204009ac0 tx=0x7f71fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b33bf9bf80a983 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.814+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720402c6b0 con 0x7f71fc0047c0 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.814+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720402c810 con 0x7f71fc0047c0 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.814+0000 7f721be00640 1 -- 172.21.15.26:0/759719232 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720402cba0 con 0x7f71fc0047c0 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.815+0000 7f7223e00640 1 -- 172.21.15.26:0/759719232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71fc0047c0 msgr2=0x7f71fc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.815+0000 7f7223e00640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71fc0047c0 0x7f71fc004c40 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f7204009ac0 tx=0x7f71fc0057f0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.815+0000 7f7223e00640 1 -- 172.21.15.26:0/759719232 shutdown_connections 2024-01-15T18:04:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.815+0000 7f7223e00640 1 --2- 172.21.15.26:0/759719232 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71fc0047c0 0x7f71fc004c40 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.815+0000 7f7223e00640 1 -- 172.21.15.26:0/759719232 >> 172.21.15.26:0/759719232 conn(0x7f721c06a800 msgr2=0x7f721c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.815+0000 7f7223e00640 1 -- 172.21.15.26:0/759719232 shutdown_connections 2024-01-15T18:04:37.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.815+0000 7f7223e00640 1 -- 172.21.15.26:0/759719232 wait complete. 2024-01-15T18:04:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.817+0000 7f7223e00640 1 Processor -- start 2024-01-15T18:04:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.817+0000 7f7223e00640 1 -- start start 2024-01-15T18:04:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.818+0000 7f7223e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71fc0047c0 0x7f721c14fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.818+0000 7f7223e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c1f63d0 0x7f721c1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.818+0000 7f7223e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f721c15b700 0x7f721c1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.818+0000 7f7223e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1f2f80 con 0x7f721c1f63d0 2024-01-15T18:04:37.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.818+0000 7f7223e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1f6970 con 0x7f721c15b700 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.818+0000 7f7223e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1f6c50 con 0x7f71fc0047c0 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.818+0000 7f7220a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c1f63d0 0x7f721c1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.818+0000 7f7220a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c1f63d0 0x7f721c1531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47928/0 (socket says 172.21.15.26:47928) 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.818+0000 7f7220a00640 1 -- 172.21.15.26:0/1587083545 learned_addr learned my addr 172.21.15.26:0/1587083545 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7220a00640 1 -- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71fc0047c0 msgr2=0x7f721c14fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7220a00640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71fc0047c0 0x7f721c14fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7220a00640 1 -- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f721c15b700 msgr2=0x7f721c1593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7220a00640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f721c15b700 0x7f721c1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7220a00640 1 -- 172.21.15.26:0/1587083545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c1f6f30 con 0x7f721c1f63d0 2024-01-15T18:04:37.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7220a00640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c1f63d0 0x7f721c1531d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f720402c250 tx=0x7f7204004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7219600640 1 -- 172.21.15.26:0/1587083545 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204003810 con 0x7f721c1f63d0 2024-01-15T18:04:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7219600640 1 -- 172.21.15.26:0/1587083545 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7204003970 con 0x7f721c1f63d0 2024-01-15T18:04:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7219600640 1 -- 172.21.15.26:0/1587083545 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204003d00 con 0x7f721c1f63d0 2024-01-15T18:04:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c1f7210 con 0x7f721c1f63d0 2024-01-15T18:04:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.819+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c15bb80 con 0x7f721c1f63d0 2024-01-15T18:04:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.821+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c1f2db0 con 0x7f721c1f63d0 2024-01-15T18:04:37.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.821+0000 7f7219600640 1 -- 172.21.15.26:0/1587083545 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 22) v1 ==== 90406+0+0 (secure 0 0 0) 0x7f7204034440 con 0x7f721c1f63d0 2024-01-15T18:04:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.822+0000 7f7219600640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71f406bc50 0x7f71f406e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.822+0000 7f7219600640 1 -- 172.21.15.26:0/1587083545 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f72040aa9f0 con 0x7f721c1f63d0 2024-01-15T18:04:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.823+0000 7f7221400640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71f406bc50 0x7f71f406e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:37.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.824+0000 7f7221400640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71f406bc50 0x7f71f406e110 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f721c0695b0 tx=0x7f720c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:37.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:37.826+0000 7f7219600640 1 -- 172.21.15.26:0/1587083545 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7204077160 con 0x7f721c1f63d0 2024-01-15T18:04:38.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:38.059+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone4183", "target": ["mon-mgr", ""]}) v1 -- 0x7f721c070550 con 0x7f71f406bc50 2024-01-15T18:04:39.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:39.241+0000 7f7219600640 1 -- 172.21.15.26:0/1587083545 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 23) v1 ==== 90508+0+0 (secure 0 0 0) 0x7f7204077530 con 0x7f721c1f63d0 2024-01-15T18:04:41.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.135+0000 7f7219600640 1 -- 172.21.15.26:0/1587083545 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f720c014420 con 0x7f71f406bc50 2024-01-15T18:04:41.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71f406bc50 msgr2=0x7f71f406e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:41.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71f406bc50 0x7f71f406e110 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f721c0695b0 tx=0x7f720c009040 comp rx=0 tx=0).stop 2024-01-15T18:04:41.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c1f63d0 msgr2=0x7f721c1531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:41.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c1f63d0 0x7f721c1531d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f720402c250 tx=0x7f7204004730 comp rx=0 tx=0).stop 2024-01-15T18:04:41.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 shutdown_connections 2024-01-15T18:04:41.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f71f406bc50 0x7f71f406e110 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:41.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f721c15b700 0x7f721c1593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:41.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f721c1f63d0 0x7f721c1531d0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:41.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 --2- 172.21.15.26:0/1587083545 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71fc0047c0 0x7f721c14fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:41.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.138+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 >> 172.21.15.26:0/1587083545 conn(0x7f721c06a800 msgr2=0x7f721c0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:41.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.139+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 shutdown_connections 2024-01-15T18:04:41.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.139+0000 7f7223e00640 1 -- 172.21.15.26:0/1587083545 wait complete. 2024-01-15T18:04:41.168 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone4184 2024-01-15T18:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.387+0000 7fef8ee00640 1 Processor -- start 2024-01-15T18:04:41.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.387+0000 7fef8ee00640 1 -- start start 2024-01-15T18:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8ee00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fef8815d3c0 0x7fef880d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:41.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8ee00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef880ce9f0 0x7fef880cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8ee00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef880d12f0 con 0x7fef880d0e90 2024-01-15T18:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8ee00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef880ce5d0 con 0x7fef880ce9f0 2024-01-15T18:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8ee00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef880b23c0 con 0x7fef8815d3c0 2024-01-15T18:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8e400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef880ce9f0 0x7fef880cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8d000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fef880d0e90 0x7fef880afc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49310/0 (socket says 172.21.15.26:49310) 2024-01-15T18:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8d000640 1 -- 172.21.15.26:0/2582067235 learned_addr learned my addr 172.21.15.26:0/2582067235 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8e400640 1 --2- 172.21.15.26:0/2582067235 >> v2:172.21.15.26:3300/0 conn(0x7fef880ce9f0 0x7fef880cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47940/0 (socket says 172.21.15.26:47940) 2024-01-15T18:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.389+0000 7fef8da00640 1 --2- 172.21.15.26:0/2582067235 >> v2:172.21.15.124:3300/0 conn(0x7fef8815d3c0 0x7fef880d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:41.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.390+0000 7fef8e400640 1 -- 172.21.15.26:0/2582067235 >> v2:172.21.15.124:3300/0 conn(0x7fef8815d3c0 msgr2=0x7fef880d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:41.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.390+0000 7fef8e400640 1 --2- 172.21.15.26:0/2582067235 >> v2:172.21.15.124:3300/0 conn(0x7fef8815d3c0 0x7fef880d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:41.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.390+0000 7fef8e400640 1 -- 172.21.15.26:0/2582067235 >> v1:172.21.15.26:6789/0 conn(0x7fef880d0e90 legacy=0x7fef880afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:41.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.390+0000 7fef8e400640 1 -- 172.21.15.26:0/2582067235 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef880d0800 con 0x7fef880ce9f0 2024-01-15T18:04:41.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.390+0000 7fef87e00640 1 -- 172.21.15.26:0/2582067235 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105621268 0 0) 0x7fef70001110 con 0x7fef880d0e90 2024-01-15T18:04:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.391+0000 7fef8e400640 1 --2- 172.21.15.26:0/2582067235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 0x7fef880cee80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fef880b5bb0 tx=0x7fef8002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94488d51fd79972 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.391+0000 7fef87e00640 1 -- 172.21.15.26:0/2582067235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8002cbf0 con 0x7fef880ce9f0 2024-01-15T18:04:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.391+0000 7fef87e00640 1 -- 172.21.15.26:0/2582067235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8002cd50 con 0x7fef880ce9f0 2024-01-15T18:04:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.391+0000 7fef87e00640 1 -- 172.21.15.26:0/2582067235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80034660 con 0x7fef880ce9f0 2024-01-15T18:04:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.392+0000 7fef8ee00640 1 -- 172.21.15.26:0/2582067235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 msgr2=0x7fef880cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.392+0000 7fef8ee00640 1 --2- 172.21.15.26:0/2582067235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 0x7fef880cee80 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fef880b5bb0 tx=0x7fef8002be40 comp rx=0 tx=0).stop 2024-01-15T18:04:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.392+0000 7fef8ee00640 1 -- 172.21.15.26:0/2582067235 shutdown_connections 2024-01-15T18:04:41.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.392+0000 7fef8ee00640 1 --2- 172.21.15.26:0/2582067235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 0x7fef880cee80 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.392+0000 7fef8ee00640 1 --2- 172.21.15.26:0/2582067235 >> v2:172.21.15.124:3300/0 conn(0x7fef8815d3c0 0x7fef880d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.392+0000 7fef8ee00640 1 -- 172.21.15.26:0/2582067235 >> 172.21.15.26:0/2582067235 conn(0x7fef880d30b0 msgr2=0x7fef880d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.392+0000 7fef8ee00640 1 -- 172.21.15.26:0/2582067235 shutdown_connections 2024-01-15T18:04:41.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.392+0000 7fef8ee00640 1 -- 172.21.15.26:0/2582067235 wait complete. 2024-01-15T18:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.394+0000 7fef8ee00640 1 Processor -- start 2024-01-15T18:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.394+0000 7fef8ee00640 1 -- start start 2024-01-15T18:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.395+0000 7fef8ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 0x7fef880b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.395+0000 7fef8ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef880d0e90 0x7fef8814e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.395+0000 7fef8ee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef8815d3c0 0x7fef8815e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:41.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.395+0000 7fef8ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8815ece0 con 0x7fef880ce9f0 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.395+0000 7fef8ee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef880b4190 con 0x7fef8815d3c0 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.395+0000 7fef8ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef880b4470 con 0x7fef880d0e90 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.395+0000 7fef8da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 0x7fef880b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.396+0000 7fef8da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 0x7fef880b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47944/0 (socket says 172.21.15.26:47944) 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.396+0000 7fef8da00640 1 -- 172.21.15.26:0/1593811517 learned_addr learned my addr 172.21.15.26:0/1593811517 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.396+0000 7fef8e400640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef8815d3c0 0x7fef8815e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.396+0000 7fef8da00640 1 -- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef880d0e90 msgr2=0x7fef8814e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.396+0000 7fef8da00640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef880d0e90 0x7fef8814e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.396+0000 7fef8da00640 1 -- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef8815d3c0 msgr2=0x7fef8815e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:41.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.396+0000 7fef8da00640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef8815d3c0 0x7fef8815e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.396+0000 7fef8da00640 1 -- 172.21.15.26:0/1593811517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef80009a70 con 0x7fef880ce9f0 2024-01-15T18:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.396+0000 7fef8da00640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 0x7fef880b4f50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fef7800d5f0 tx=0x7fef7800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.397+0000 7fef85600640 1 -- 172.21.15.26:0/1593811517 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef780125f0 con 0x7fef880ce9f0 2024-01-15T18:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.397+0000 7fef85600640 1 -- 172.21.15.26:0/1593811517 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef78012750 con 0x7fef880ce9f0 2024-01-15T18:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.397+0000 7fef85600640 1 -- 172.21.15.26:0/1593811517 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78012b00 con 0x7fef880ce9f0 2024-01-15T18:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.397+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef880b4750 con 0x7fef880ce9f0 2024-01-15T18:04:41.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.397+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8815d9c0 con 0x7fef880ce9f0 2024-01-15T18:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.398+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef880cdba0 con 0x7fef880ce9f0 2024-01-15T18:04:41.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.399+0000 7fef85600640 1 -- 172.21.15.26:0/1593811517 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) v1 ==== 90508+0+0 (secure 0 0 0) 0x7fef78012c60 con 0x7fef880ce9f0 2024-01-15T18:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.400+0000 7fef85600640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef6006bd70 0x7fef6006e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.400+0000 7fef85600640 1 -- 172.21.15.26:0/1593811517 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fef78088a30 con 0x7fef880ce9f0 2024-01-15T18:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.401+0000 7fef8d000640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef6006bd70 0x7fef6006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.402+0000 7fef8d000640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef6006bd70 0x7fef6006e230 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fef70007fd0 tx=0x7fef70007960 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:41.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.405+0000 7fef85600640 1 -- 172.21.15.26:0/1593811517 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef880cdba0 con 0x7fef880ce9f0 2024-01-15T18:04:41.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:41.635+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone4184", "target": ["mon-mgr", ""]}) v1 -- 0x7fef880ce1f0 con 0x7fef6006bd70 2024-01-15T18:04:46.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.120+0000 7fef85600640 1 -- 172.21.15.26:0/1593811517 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fef70001220 con 0x7fef6006bd70 2024-01-15T18:04:46.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef6006bd70 msgr2=0x7fef6006e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef6006bd70 0x7fef6006e230 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fef70007fd0 tx=0x7fef70007960 comp rx=0 tx=0).stop 2024-01-15T18:04:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 msgr2=0x7fef880b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 0x7fef880b4f50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fef7800d5f0 tx=0x7fef7800dab0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 shutdown_connections 2024-01-15T18:04:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef880d0e90 0x7fef8814e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef6006bd70 0x7fef6006e230 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef8815d3c0 0x7fef8815e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 --2- 172.21.15.26:0/1593811517 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef880ce9f0 0x7fef880b4f50 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 >> 172.21.15.26:0/1593811517 conn(0x7fef880d30b0 msgr2=0x7fef880d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 shutdown_connections 2024-01-15T18:04:46.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.124+0000 7fef8ee00640 1 -- 172.21.15.26:0/1593811517 wait complete. 2024-01-15T18:04:46.150 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone4185 2024-01-15T18:04:46.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.365+0000 7fba13a00640 1 Processor -- start 2024-01-15T18:04:46.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.366+0000 7fba13a00640 1 -- start start 2024-01-15T18:04:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.367+0000 7fba13a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.367+0000 7fba13a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fba0c13e360 0x7fba0c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.367+0000 7fba13a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0c06c470 con 0x7fba0c15f9c0 2024-01-15T18:04:46.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.367+0000 7fba13a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c06c750 con 0x7fba0c13e360 2024-01-15T18:04:46.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.367+0000 7fba13a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c06ca30 con 0x7fba0c13df80 2024-01-15T18:04:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.368+0000 7fba03e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fba0c13e360 0x7fba0c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.368+0000 7fba11000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.368+0000 7fba11000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:54934/0 (socket says 172.21.15.26:54934) 2024-01-15T18:04:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.368+0000 7fba11000640 1 -- 172.21.15.26:0/1313693023 learned_addr learned my addr 172.21.15.26:0/1313693023 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.368+0000 7fba03e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fba0c13e360 0x7fba0c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:33828/0 (socket says 172.21.15.26:33828) 2024-01-15T18:04:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.369+0000 7fba11000640 1 -- 172.21.15.26:0/1313693023 >> v2:172.21.15.124:3300/0 conn(0x7fba0c13e360 msgr2=0x7fba0c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:46.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.369+0000 7fba11000640 1 --2- 172.21.15.26:0/1313693023 >> v2:172.21.15.124:3300/0 conn(0x7fba0c13e360 0x7fba0c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.369+0000 7fba11000640 1 -- 172.21.15.26:0/1313693023 >> v1:172.21.15.124:6789/0 conn(0x7fba0c15f9c0 legacy=0x7fba0c1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:46.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.369+0000 7fba11000640 1 -- 172.21.15.26:0/1313693023 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c06cd10 con 0x7fba0c13df80 2024-01-15T18:04:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.370+0000 7fba11000640 1 --2- 172.21.15.26:0/1313693023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c15bee0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009870 tx=0x7fb9fc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d072cac57ef513e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.371+0000 7fba0be00640 1 -- 172.21.15.26:0/1313693023 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc030840 con 0x7fba0c13df80 2024-01-15T18:04:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.371+0000 7fba0be00640 1 -- 172.21.15.26:0/1313693023 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc0309a0 con 0x7fba0c13df80 2024-01-15T18:04:46.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.371+0000 7fba0be00640 1 -- 172.21.15.26:0/1313693023 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc030d30 con 0x7fba0c13df80 2024-01-15T18:04:46.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.372+0000 7fba13a00640 1 -- 172.21.15.26:0/1313693023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 msgr2=0x7fba0c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:46.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.372+0000 7fba13a00640 1 --2- 172.21.15.26:0/1313693023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c15bee0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009870 tx=0x7fb9fc02b980 comp rx=0 tx=0).stop 2024-01-15T18:04:46.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.372+0000 7fba13a00640 1 -- 172.21.15.26:0/1313693023 shutdown_connections 2024-01-15T18:04:46.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.372+0000 7fba13a00640 1 --2- 172.21.15.26:0/1313693023 >> v2:172.21.15.124:3300/0 conn(0x7fba0c13e360 0x7fba0c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.372+0000 7fba13a00640 1 --2- 172.21.15.26:0/1313693023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c15bee0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.372+0000 7fba13a00640 1 -- 172.21.15.26:0/1313693023 >> 172.21.15.26:0/1313693023 conn(0x7fba0c06a800 msgr2=0x7fba0c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:46.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.372+0000 7fba13a00640 1 -- 172.21.15.26:0/1313693023 shutdown_connections 2024-01-15T18:04:46.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.372+0000 7fba13a00640 1 -- 172.21.15.26:0/1313693023 wait complete. 2024-01-15T18:04:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.374+0000 7fba13a00640 1 Processor -- start 2024-01-15T18:04:46.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.375+0000 7fba13a00640 1 -- start start 2024-01-15T18:04:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba13a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c071d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba13a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba0c13e360 0x7fba0c14d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba13a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0c15f9c0 0x7fba0c0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba13a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c071880 con 0x7fba0c15f9c0 2024-01-15T18:04:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba13a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c070990 con 0x7fba0c13e360 2024-01-15T18:04:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba13a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c070c70 con 0x7fba0c13df80 2024-01-15T18:04:46.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba11a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0c15f9c0 0x7fba0c0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba11a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0c15f9c0 0x7fba0c0b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42272/0 (socket says 172.21.15.26:42272) 2024-01-15T18:04:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba11a00640 1 -- 172.21.15.26:0/1379685730 learned_addr learned my addr 172.21.15.26:0/1379685730 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.376+0000 7fba11000640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c071d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.377+0000 7fba03e00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba0c13e360 0x7fba0c14d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.377+0000 7fba11a00640 1 -- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 msgr2=0x7fba0c071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.377+0000 7fba11a00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c071d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.377+0000 7fba11a00640 1 -- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba0c13e360 msgr2=0x7fba0c14d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.377+0000 7fba11a00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba0c13e360 0x7fba0c14d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:46.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.377+0000 7fba11a00640 1 -- 172.21.15.26:0/1379685730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc0094d0 con 0x7fba0c15f9c0 2024-01-15T18:04:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.377+0000 7fba03e00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba0c13e360 0x7fba0c14d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:04:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.377+0000 7fba11a00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0c15f9c0 0x7fba0c0b2f60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c7a0 tx=0x7fba0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.377+0000 7fba09600640 1 -- 172.21.15.26:0/1379685730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04003b70 con 0x7fba0c15f9c0 2024-01-15T18:04:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.378+0000 7fba09600640 1 -- 172.21.15.26:0/1379685730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04003cf0 con 0x7fba0c15f9c0 2024-01-15T18:04:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.378+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0c070f50 con 0x7fba0c15f9c0 2024-01-15T18:04:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.378+0000 7fba09600640 1 -- 172.21.15.26:0/1379685730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04003ea0 con 0x7fba0c15f9c0 2024-01-15T18:04:46.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.378+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0c071510 con 0x7fba0c15f9c0 2024-01-15T18:04:46.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.379+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0c0675d0 con 0x7fba0c15f9c0 2024-01-15T18:04:46.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.380+0000 7fba09600640 1 -- 172.21.15.26:0/1379685730 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) v1 ==== 90508+0+0 (secure 0 0 0) 0x7fba0400aa90 con 0x7fba0c15f9c0 2024-01-15T18:04:46.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.381+0000 7fba09600640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9e406bda0 0x7fb9e406e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:46.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.381+0000 7fba09600640 1 -- 172.21.15.26:0/1379685730 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fba040882e0 con 0x7fba0c15f9c0 2024-01-15T18:04:46.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.382+0000 7fba11000640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9e406bda0 0x7fb9e406e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:46.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.383+0000 7fba11000640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9e406bda0 0x7fb9e406e260 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc036000 tx=0x7fb9fc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:46.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.386+0000 7fba09600640 1 -- 172.21.15.26:0/1379685730 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba04091080 con 0x7fba0c15f9c0 2024-01-15T18:04:46.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:46.615+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone4185", "target": ["mon-mgr", ""]}) v1 -- 0x7fba0c15eac0 con 0x7fb9e406bda0 2024-01-15T18:04:51.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.141+0000 7fba09600640 1 -- 172.21.15.26:0/1379685730 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fba0c15eac0 con 0x7fb9e406bda0 2024-01-15T18:04:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9e406bda0 msgr2=0x7fb9e406e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9e406bda0 0x7fb9e406e260 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc036000 tx=0x7fb9fc008030 comp rx=0 tx=0).stop 2024-01-15T18:04:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0c15f9c0 msgr2=0x7fba0c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0c15f9c0 0x7fba0c0b2f60 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c7a0 tx=0x7fba0400cc60 comp rx=0 tx=0).stop 2024-01-15T18:04:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 shutdown_connections 2024-01-15T18:04:51.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9e406bda0 0x7fb9e406e260 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0c15f9c0 0x7fba0c0b2f60 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba0c13e360 0x7fba0c14d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 --2- 172.21.15.26:0/1379685730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba0c13df80 0x7fba0c071d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 >> 172.21.15.26:0/1379685730 conn(0x7fba0c06a800 msgr2=0x7fba0c15d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:51.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.144+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 shutdown_connections 2024-01-15T18:04:51.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.145+0000 7fba13a00640 1 -- 172.21.15.26:0/1379685730 wait complete. 2024-01-15T18:04:51.169 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone4186 2024-01-15T18:04:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.384+0000 7fa86c400640 1 Processor -- start 2024-01-15T18:04:51.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.384+0000 7fa86c400640 1 -- start start 2024-01-15T18:04:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.386+0000 7fa86c400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa86406a3e0 0x7fa8640ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.386+0000 7fa86c400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa864071de0 con 0x7fa8640aede0 2024-01-15T18:04:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.386+0000 7fa86c400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8640af270 con 0x7fa8640d0b60 2024-01-15T18:04:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.386+0000 7fa86c400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa864072030 con 0x7fa86406a3e0 2024-01-15T18:04:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.387+0000 7fa86a400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa8640aede0 0x7fa86406f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45962/0 (socket says 172.21.15.26:45962) 2024-01-15T18:04:51.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.387+0000 7fa86a400640 1 -- 172.21.15.26:0/2878517485 learned_addr learned my addr 172.21.15.26:0/2878517485 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:04:51.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.387+0000 7fa869a00640 1 --2- 172.21.15.26:0/2878517485 >> v2:172.21.15.124:3300/0 conn(0x7fa86406a3e0 0x7fa8640ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.387+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985925364 0 0) 0x7fa864071de0 con 0x7fa8640aede0 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.387+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa844003520 con 0x7fa8640aede0 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101189259 0 0) 0x7fa844003520 con 0x7fa8640aede0 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 >> v2:172.21.15.124:3300/0 conn(0x7fa86406a3e0 msgr2=0x7fa8640ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 --2- 172.21.15.26:0/2878517485 >> v2:172.21.15.124:3300/0 conn(0x7fa86406a3e0 0x7fa8640ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 >> v1:172.21.15.124:6789/0 conn(0x7fa8640d0b60 legacy=0x7fa86406a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864072310 con 0x7fa8640aede0 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa85c001670 con 0x7fa8640aede0 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 >> v1:172.21.15.26:6789/0 conn(0x7fa8640aede0 legacy=0x7fa86406f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa844003970 0x7fa844003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8440047c0 0x7fa844004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa84400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa864072030 con 0x7fa844005180 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84400c020 con 0x7fa844003970 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84400c1e0 con 0x7fa8440047c0 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa86a400640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa84400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.388+0000 7fa869a00640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa844003970 0x7fa844003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.389+0000 7fa869000640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8440047c0 0x7fa844004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.389+0000 7fa86a400640 1 -- 172.21.15.26:0/2878517485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8440047c0 msgr2=0x7fa844004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.389+0000 7fa86a400640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa8440047c0 0x7fa844004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.389+0000 7fa86a400640 1 -- 172.21.15.26:0/2878517485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa844003970 msgr2=0x7fa844003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.389+0000 7fa86a400640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa844003970 0x7fa844003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.389+0000 7fa86a400640 1 -- 172.21.15.26:0/2878517485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa844003520 con 0x7fa844005180 2024-01-15T18:04:51.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.390+0000 7fa86a400640 1 -- 172.21.15.26:0/2878517485 reap_dead start 2024-01-15T18:04:51.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.390+0000 7fa86a400640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa84400b980 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c240 tx=0x7fa8440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32afac978c1a2195 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:51.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.390+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c005ad0 con 0x7fa844005180 2024-01-15T18:04:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.390+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c005c70 con 0x7fa844005180 2024-01-15T18:04:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.390+0000 7fa863e00640 1 -- 172.21.15.26:0/2878517485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c02c8b0 con 0x7fa844005180 2024-01-15T18:04:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.391+0000 7fa86c400640 1 -- 172.21.15.26:0/2878517485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 msgr2=0x7fa84400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.391+0000 7fa86c400640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa84400b980 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00c240 tx=0x7fa8440057f0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.391+0000 7fa86c400640 1 -- 172.21.15.26:0/2878517485 shutdown_connections 2024-01-15T18:04:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.391+0000 7fa86c400640 1 --2- 172.21.15.26:0/2878517485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa84400b980 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.391+0000 7fa86c400640 1 -- 172.21.15.26:0/2878517485 >> 172.21.15.26:0/2878517485 conn(0x7fa8640cfef0 msgr2=0x7fa8640d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.391+0000 7fa86c400640 1 -- 172.21.15.26:0/2878517485 shutdown_connections 2024-01-15T18:04:51.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.391+0000 7fa86c400640 1 -- 172.21.15.26:0/2878517485 wait complete. 2024-01-15T18:04:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.393+0000 7fa86c400640 1 Processor -- start 2024-01-15T18:04:51.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.393+0000 7fa86c400640 1 -- start start 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.394+0000 7fa86c400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa864150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.394+0000 7fa86c400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa864160480 0x7fa8640cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.394+0000 7fa86c400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa86415c180 0x7fa86415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.394+0000 7fa86c400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86415cb70 con 0x7fa844005180 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.394+0000 7fa86c400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa864155550 con 0x7fa86415c180 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.394+0000 7fa86c400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640cda30 con 0x7fa864160480 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.394+0000 7fa869a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa864150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.394+0000 7fa869a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa864150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42300/0 (socket says 172.21.15.26:42300) 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.394+0000 7fa869a00640 1 -- 172.21.15.26:0/3072288381 learned_addr learned my addr 172.21.15.26:0/3072288381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa869a00640 1 -- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa864160480 msgr2=0x7fa8640cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa869a00640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa864160480 0x7fa8640cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa869a00640 1 -- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa86415c180 msgr2=0x7fa86415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa869a00640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa86415c180 0x7fa86415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa869a00640 1 -- 172.21.15.26:0/3072288381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864151360 con 0x7fa844005180 2024-01-15T18:04:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa869a00640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa864150e20 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa854007f30 tx=0x7fa854011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa861600640 1 -- 172.21.15.26:0/3072288381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400e030 con 0x7fa844005180 2024-01-15T18:04:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa861600640 1 -- 172.21.15.26:0/3072288381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854016470 con 0x7fa844005180 2024-01-15T18:04:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa861600640 1 -- 172.21.15.26:0/3072288381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854016820 con 0x7fa844005180 2024-01-15T18:04:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8640cd190 con 0x7fa844005180 2024-01-15T18:04:51.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.395+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8640cd750 con 0x7fa844005180 2024-01-15T18:04:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.397+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8640675d0 con 0x7fa844005180 2024-01-15T18:04:51.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.398+0000 7fa861600640 1 -- 172.21.15.26:0/3072288381 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) v1 ==== 90508+0+0 (secure 0 0 0) 0x7fa854016980 con 0x7fa844005180 2024-01-15T18:04:51.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.399+0000 7fa861600640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa83c06bda0 0x7fa83c06e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:51.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.399+0000 7fa861600640 1 -- 172.21.15.26:0/3072288381 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fa85400f070 con 0x7fa844005180 2024-01-15T18:04:51.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.399+0000 7fa869000640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa83c06bda0 0x7fa83c06e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:51.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.400+0000 7fa869000640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa83c06bda0 0x7fa83c06e260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa84c000f60 tx=0x7fa84c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:51.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.404+0000 7fa861600640 1 -- 172.21.15.26:0/3072288381 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa854055cd0 con 0x7fa844005180 2024-01-15T18:04:51.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.634+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone4186", "target": ["mon-mgr", ""]}) v1 -- 0x7fa86406eb60 con 0x7fa83c06bda0 2024-01-15T18:04:51.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.831+0000 7fa861600640 1 -- 172.21.15.26:0/3072288381 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa84c014420 con 0x7fa83c06bda0 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa83c06bda0 msgr2=0x7fa83c06e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa83c06bda0 0x7fa83c06e260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa84c000f60 tx=0x7fa84c009040 comp rx=0 tx=0).stop 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 msgr2=0x7fa864150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa864150e20 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fa854007f30 tx=0x7fa854011660 comp rx=0 tx=0).stop 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 shutdown_connections 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa864160480 0x7fa8640cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa83c06bda0 0x7fa83c06e260 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa86415c180 0x7fa86415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 --2- 172.21.15.26:0/3072288381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa844005180 0x7fa864150e20 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 >> 172.21.15.26:0/3072288381 conn(0x7fa8640cfef0 msgr2=0x7fa86406d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 shutdown_connections 2024-01-15T18:04:51.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:51.835+0000 7fa86c400640 1 -- 172.21.15.26:0/3072288381 wait complete. 2024-01-15T18:04:51.859 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone4187 2024-01-15T18:04:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.078+0000 7f37ab800640 1 Processor -- start 2024-01-15T18:04:52.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.078+0000 7f37ab800640 1 -- start start 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.080+0000 7f37ab800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a413df80 0x7f37a415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.080+0000 7f37ab800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f37a40cf9a0 0x7f37a40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.080+0000 7f37ab800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a406c470 con 0x7f37a413e360 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.080+0000 7f37ab800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a406c750 con 0x7f37a40cf9a0 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.080+0000 7f37ab800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a406ca30 con 0x7f37a413df80 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.081+0000 7f37a9800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f37a40cf9a0 0x7f37a40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.081+0000 7f37a9800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f37a40cf9a0 0x7f37a40cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48806/0 (socket says 172.21.15.26:48806) 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.081+0000 7f37a9800640 1 -- 172.21.15.26:0/544362599 learned_addr learned my addr 172.21.15.26:0/544362599 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.081+0000 7f37a8e00640 1 --2- 172.21.15.26:0/544362599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a413df80 0x7f37a415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.081+0000 7f37a9800640 1 -- 172.21.15.26:0/544362599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a413df80 msgr2=0x7f37a415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.081+0000 7f37a9800640 1 --2- 172.21.15.26:0/544362599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a413df80 0x7f37a415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.081+0000 7f37a9800640 1 -- 172.21.15.26:0/544362599 >> v1:172.21.15.26:6789/0 conn(0x7f37a413e360 legacy=0x7f37a40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.081+0000 7f37a9800640 1 -- 172.21.15.26:0/544362599 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a406cd10 con 0x7f37a40cf9a0 2024-01-15T18:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.081+0000 7f37a3400640 1 -- 172.21.15.26:0/544362599 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2210073241 0 0) 0x7f37a406c470 con 0x7f37a413e360 2024-01-15T18:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.082+0000 7f37a9800640 1 --2- 172.21.15.26:0/544362599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a40cf9a0 0x7f37a40cd840 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f379c009e50 tx=0x7f379c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3cf92e5328726 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.083+0000 7f37a3400640 1 -- 172.21.15.26:0/544362599 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c02cb00 con 0x7f37a40cf9a0 2024-01-15T18:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.083+0000 7f37a3400640 1 -- 172.21.15.26:0/544362599 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c02cc60 con 0x7f37a40cf9a0 2024-01-15T18:04:52.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.083+0000 7f37a3400640 1 -- 172.21.15.26:0/544362599 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c0343f0 con 0x7f37a40cf9a0 2024-01-15T18:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.083+0000 7f37ab800640 1 -- 172.21.15.26:0/544362599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a40cf9a0 msgr2=0x7f37a40cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.083+0000 7f37ab800640 1 --2- 172.21.15.26:0/544362599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a40cf9a0 0x7f37a40cd840 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f379c009e50 tx=0x7f379c02bf60 comp rx=0 tx=0).stop 2024-01-15T18:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.083+0000 7f37ab800640 1 -- 172.21.15.26:0/544362599 shutdown_connections 2024-01-15T18:04:52.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.083+0000 7f37ab800640 1 --2- 172.21.15.26:0/544362599 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a40cf9a0 0x7f37a40cd840 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.084+0000 7f37ab800640 1 --2- 172.21.15.26:0/544362599 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a413df80 0x7f37a415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.084+0000 7f37ab800640 1 -- 172.21.15.26:0/544362599 >> 172.21.15.26:0/544362599 conn(0x7f37a406a800 msgr2=0x7f37a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.084+0000 7f37ab800640 1 -- 172.21.15.26:0/544362599 shutdown_connections 2024-01-15T18:04:52.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.084+0000 7f37ab800640 1 -- 172.21.15.26:0/544362599 wait complete. 2024-01-15T18:04:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.085+0000 7f37ab800640 1 Processor -- start 2024-01-15T18:04:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.086+0000 7f37ab800640 1 -- start start 2024-01-15T18:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.086+0000 7f37ab800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a40cf9a0 0x7f37a4159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37ab800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a413df80 0x7f37a415a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37ab800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37a413e360 0x7f37a4071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37ab800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a40a9210 con 0x7f37a413df80 2024-01-15T18:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37ab800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a40706a0 con 0x7f37a413e360 2024-01-15T18:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37ab800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a4070980 con 0x7f37a40cf9a0 2024-01-15T18:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a413df80 0x7f37a415a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a413df80 0x7f37a415a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48808/0 (socket says 172.21.15.26:48808) 2024-01-15T18:04:52.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a3e00640 1 -- 172.21.15.26:0/446403381 learned_addr learned my addr 172.21.15.26:0/446403381 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a8e00640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a40cf9a0 0x7f37a4159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a3e00640 1 -- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a40cf9a0 msgr2=0x7f37a4159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a9800640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37a413e360 0x7f37a4071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a3e00640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a40cf9a0 0x7f37a4159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a3e00640 1 -- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37a413e360 msgr2=0x7f37a4071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a3e00640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37a413e360 0x7f37a4071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.087+0000 7f37a3e00640 1 -- 172.21.15.26:0/446403381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c009b30 con 0x7f37a413df80 2024-01-15T18:04:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.088+0000 7f37a3e00640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a413df80 0x7f37a415a4c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c4b0 tx=0x7f378c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.088+0000 7f37a0c00640 1 -- 172.21.15.26:0/446403381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c005480 con 0x7f37a413df80 2024-01-15T18:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.088+0000 7f37a0c00640 1 -- 172.21.15.26:0/446403381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378c0035f0 con 0x7f37a413df80 2024-01-15T18:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.088+0000 7f37a0c00640 1 -- 172.21.15.26:0/446403381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c0039e0 con 0x7f37a413df80 2024-01-15T18:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.088+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a4070c60 con 0x7f37a413df80 2024-01-15T18:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.088+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a4071220 con 0x7f37a413df80 2024-01-15T18:04:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.090+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a4071440 con 0x7f37a413df80 2024-01-15T18:04:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.090+0000 7f37a0c00640 1 -- 172.21.15.26:0/446403381 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) v1 ==== 90508+0+0 (secure 0 0 0) 0x7f378c003b40 con 0x7f37a413df80 2024-01-15T18:04:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.091+0000 7f37a0c00640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f377c06bd70 0x7f377c06e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.091+0000 7f37a0c00640 1 -- 172.21.15.26:0/446403381 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f378c087d00 con 0x7f37a413df80 2024-01-15T18:04:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.092+0000 7f37a8e00640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f377c06bd70 0x7f377c06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.093+0000 7f37a8e00640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f377c06bd70 0x7f377c06e230 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3794006fd0 tx=0x7f3794008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.096+0000 7f37a0c00640 1 -- 172.21.15.26:0/446403381 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f378c0543f0 con 0x7f37a413df80 2024-01-15T18:04:52.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.325+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone4187", "target": ["mon-mgr", ""]}) v1 -- 0x7f37a415f9c0 con 0x7f377c06bd70 2024-01-15T18:04:52.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.508+0000 7f37a0c00640 1 -- 172.21.15.26:0/446403381 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f379400b200 con 0x7f377c06bd70 2024-01-15T18:04:52.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.511+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f377c06bd70 msgr2=0x7f377c06e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.511+0000 7f37ab800640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f377c06bd70 0x7f377c06e230 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3794006fd0 tx=0x7f3794008040 comp rx=0 tx=0).stop 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.511+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a413df80 msgr2=0x7f37a415a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.511+0000 7f37ab800640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a413df80 0x7f37a415a4c0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f378c00c4b0 tx=0x7f378c00c970 comp rx=0 tx=0).stop 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.512+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 shutdown_connections 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.512+0000 7f37ab800640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f377c06bd70 0x7f377c06e230 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.512+0000 7f37ab800640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f37a413e360 0x7f37a4071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.512+0000 7f37ab800640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37a413df80 0x7f37a415a4c0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.512+0000 7f37ab800640 1 --2- 172.21.15.26:0/446403381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37a40cf9a0 0x7f37a4159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.512+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 >> 172.21.15.26:0/446403381 conn(0x7f37a406a800 msgr2=0x7f37a415d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:52.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.512+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 shutdown_connections 2024-01-15T18:04:52.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.512+0000 7f37ab800640 1 -- 172.21.15.26:0/446403381 wait complete. 2024-01-15T18:04:52.534 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T18:04:52.535 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4184 2024-01-15T18:04:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.750+0000 7f6f1ca00640 1 Processor -- start 2024-01-15T18:04:52.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.750+0000 7f6f1ca00640 1 -- start start 2024-01-15T18:04:52.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.752+0000 7f6f1ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f140c9870 0x7f6f140ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.752+0000 7f6f1ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f14161740 0x7f6f140c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.752+0000 7f6f1ca00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f140cf4c0 con 0x7f6f140d0560 2024-01-15T18:04:52.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.752+0000 7f6f1ca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f140d2ef0 con 0x7f6f14161740 2024-01-15T18:04:52.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.752+0000 7f6f1ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140c9c50 con 0x7f6f140c9870 2024-01-15T18:04:52.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.752+0000 7f6f19600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6f140d0560 0x7f6f14161030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:48960/0 (socket says 172.21.15.26:48960) 2024-01-15T18:04:52.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.752+0000 7f6f19600640 1 -- 172.21.15.26:0/86081481 learned_addr learned my addr 172.21.15.26:0/86081481 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:04:52.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.752+0000 7f6f1aa00640 1 --2- 172.21.15.26:0/86081481 >> v2:172.21.15.26:3300/0 conn(0x7f6f14161740 0x7f6f140c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.752+0000 7f6f1a000640 1 --2- 172.21.15.26:0/86081481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f140c9870 0x7f6f140ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.753+0000 7f6f1aa00640 1 -- 172.21.15.26:0/86081481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f140c9870 msgr2=0x7f6f140ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:52.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.753+0000 7f6f1aa00640 1 --2- 172.21.15.26:0/86081481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f140c9870 0x7f6f140ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.753+0000 7f6f1aa00640 1 -- 172.21.15.26:0/86081481 >> v1:172.21.15.26:6789/0 conn(0x7f6f140d0560 legacy=0x7f6f14161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.753+0000 7f6f1aa00640 1 -- 172.21.15.26:0/86081481 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140cf7a0 con 0x7f6f14161740 2024-01-15T18:04:52.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.753+0000 7f6f18c00640 1 -- 172.21.15.26:0/86081481 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524914861 0 0) 0x7f6f140cf4c0 con 0x7f6f140d0560 2024-01-15T18:04:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.754+0000 7f6f1aa00640 1 --2- 172.21.15.26:0/86081481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f14161740 0x7f6f140c6da0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009dc0 tx=0x7f6f0402be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f647d7a3f421102a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.754+0000 7f6f18c00640 1 -- 172.21.15.26:0/86081481 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0402cc30 con 0x7f6f14161740 2024-01-15T18:04:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.754+0000 7f6f18c00640 1 -- 172.21.15.26:0/86081481 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0402cd90 con 0x7f6f14161740 2024-01-15T18:04:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.754+0000 7f6f18c00640 1 -- 172.21.15.26:0/86081481 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04034660 con 0x7f6f14161740 2024-01-15T18:04:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.755+0000 7f6f1ca00640 1 -- 172.21.15.26:0/86081481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f14161740 msgr2=0x7f6f140c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.755+0000 7f6f1ca00640 1 --2- 172.21.15.26:0/86081481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f14161740 0x7f6f140c6da0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009dc0 tx=0x7f6f0402be80 comp rx=0 tx=0).stop 2024-01-15T18:04:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.755+0000 7f6f1ca00640 1 -- 172.21.15.26:0/86081481 shutdown_connections 2024-01-15T18:04:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.755+0000 7f6f1ca00640 1 --2- 172.21.15.26:0/86081481 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f14161740 0x7f6f140c6da0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.755+0000 7f6f1ca00640 1 --2- 172.21.15.26:0/86081481 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f140c9870 0x7f6f140ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.755+0000 7f6f1ca00640 1 -- 172.21.15.26:0/86081481 >> 172.21.15.26:0/86081481 conn(0x7f6f1409c560 msgr2=0x7f6f140d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.755+0000 7f6f1ca00640 1 -- 172.21.15.26:0/86081481 shutdown_connections 2024-01-15T18:04:52.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.755+0000 7f6f1ca00640 1 -- 172.21.15.26:0/86081481 wait complete. 2024-01-15T18:04:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.757+0000 7f6f1ca00640 1 Processor -- start 2024-01-15T18:04:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.757+0000 7f6f1ca00640 1 -- start start 2024-01-15T18:04:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.758+0000 7f6f1ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f140c9870 0x7f6f140b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.758+0000 7f6f1ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f140d0560 0x7f6f141f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.758+0000 7f6f1ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f14161740 0x7f6f140b8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.758+0000 7f6f1ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140b7360 con 0x7f6f140c9870 2024-01-15T18:04:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.758+0000 7f6f1ca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140b66e0 con 0x7f6f140d0560 2024-01-15T18:04:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.758+0000 7f6f1ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140b69c0 con 0x7f6f14161740 2024-01-15T18:04:52.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.758+0000 7f6f1a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f140c9870 0x7f6f140b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f1a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f140c9870 0x7f6f140b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48814/0 (socket says 172.21.15.26:48814) 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f1a000640 1 -- 172.21.15.26:0/876181975 learned_addr learned my addr 172.21.15.26:0/876181975 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f1a000640 1 -- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f14161740 msgr2=0x7f6f140b8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f19600640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f140d0560 0x7f6f141f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f1aa00640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f14161740 0x7f6f140b8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f1a000640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f14161740 0x7f6f140b8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f1a000640 1 -- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f140d0560 msgr2=0x7f6f141f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f1a000640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f140d0560 0x7f6f141f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f1a000640 1 -- 172.21.15.26:0/876181975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f04009a70 con 0x7f6f140c9870 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f19600640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f140d0560 0x7f6f141f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:04:52.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.759+0000 7f6f1a000640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f140c9870 0x7f6f140b5de0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d800 tx=0x7f6f0c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.760+0000 7f6f12000640 1 -- 172.21.15.26:0/876181975 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00b6c0 con 0x7f6f140c9870 2024-01-15T18:04:52.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.760+0000 7f6f12000640 1 -- 172.21.15.26:0/876181975 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c00b820 con 0x7f6f140c9870 2024-01-15T18:04:52.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.760+0000 7f6f12000640 1 -- 172.21.15.26:0/876181975 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00bbb0 con 0x7f6f140c9870 2024-01-15T18:04:52.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.760+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f140b6ca0 con 0x7f6f140c9870 2024-01-15T18:04:52.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.760+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f140b6320 con 0x7f6f140c9870 2024-01-15T18:04:52.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.761+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f140675d0 con 0x7f6f140c9870 2024-01-15T18:04:52.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.762+0000 7f6f12000640 1 -- 172.21.15.26:0/876181975 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) v1 ==== 90508+0+0 (secure 0 0 0) 0x7f6f0c00bd10 con 0x7f6f140c9870 2024-01-15T18:04:52.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.763+0000 7f6f12000640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6eec06bd70 0x7f6eec06e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:52.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.763+0000 7f6f12000640 1 -- 172.21.15.26:0/876181975 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f6f0c088a80 con 0x7f6f140c9870 2024-01-15T18:04:52.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.764+0000 7f6f19600640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6eec06bd70 0x7f6eec06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:52.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.765+0000 7f6f19600640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6eec06bd70 0x7f6eec06e230 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f140b7de0 tx=0x7f6efc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:52.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:52.767+0000 7f6f12000640 1 -- 172.21.15.26:0/876181975 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f0c0553a0 con 0x7f6f140c9870 2024-01-15T18:04:53.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.001+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4184", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f140d0040 con 0x7f6eec06bd70 2024-01-15T18:04:53.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.017+0000 7f6f12000640 1 -- 172.21.15.26:0/876181975 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f6efc001d50 con 0x7f6eec06bd70 2024-01-15T18:04:53.021 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:04:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:04:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:04:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:04:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:04:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:04:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:04:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:04:53.022 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:04:53.022 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:04:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6eec06bd70 msgr2=0x7f6eec06e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6eec06bd70 0x7f6eec06e230 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f6f140b7de0 tx=0x7f6efc009040 comp rx=0 tx=0).stop 2024-01-15T18:04:53.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f140c9870 msgr2=0x7f6f140b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f140c9870 0x7f6f140b5de0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00d800 tx=0x7f6f0c00dcc0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 shutdown_connections 2024-01-15T18:04:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f14161740 0x7f6f140b8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6eec06bd70 0x7f6eec06e230 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f140d0560 0x7f6f141f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 --2- 172.21.15.26:0/876181975 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f140c9870 0x7f6f140b5de0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 >> 172.21.15.26:0/876181975 conn(0x7f6f1409c560 msgr2=0x7f6f1415f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.020+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 shutdown_connections 2024-01-15T18:04:53.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:53.021+0000 7f6f1ca00640 1 -- 172.21.15.26:0/876181975 wait complete. 2024-01-15T18:04:53.044 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:04:54.045 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4184 2024-01-15T18:04:54.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.277+0000 7f9c4a400640 1 Processor -- start 2024-01-15T18:04:54.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.278+0000 7f9c4a400640 1 -- start start 2024-01-15T18:04:54.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.279+0000 7f9c4a400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c440adf20 0x7f9c44070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:54.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.279+0000 7f9c4a400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c4415bf10 0x7f9c440cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:54.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.279+0000 7f9c4a400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4415f4e0 con 0x7f9c440cc980 2024-01-15T18:04:54.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.279+0000 7f9c4a400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4415f970 con 0x7f9c440adf20 2024-01-15T18:04:54.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.279+0000 7f9c4a400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440cce10 con 0x7f9c4415bf10 2024-01-15T18:04:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.280+0000 7f9c49000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c440adf20 0x7f9c44070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.280+0000 7f9c49000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9c440adf20 0x7f9c44070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48828/0 (socket says 172.21.15.26:48828) 2024-01-15T18:04:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.280+0000 7f9c49000640 1 -- 172.21.15.26:0/1358496074 learned_addr learned my addr 172.21.15.26:0/1358496074 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.280+0000 7f9c3be00640 1 --2- 172.21.15.26:0/1358496074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c4415bf10 0x7f9c440cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.280+0000 7f9c49000640 1 -- 172.21.15.26:0/1358496074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c4415bf10 msgr2=0x7f9c440cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.280+0000 7f9c49000640 1 --2- 172.21.15.26:0/1358496074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c4415bf10 0x7f9c440cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.280+0000 7f9c49000640 1 -- 172.21.15.26:0/1358496074 >> v1:172.21.15.124:6789/0 conn(0x7f9c440cc980 legacy=0x7f9c44143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.280+0000 7f9c49000640 1 -- 172.21.15.26:0/1358496074 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c44071130 con 0x7f9c440adf20 2024-01-15T18:04:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.281+0000 7f9c49000640 1 --2- 172.21.15.26:0/1358496074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c440adf20 0x7f9c44070a30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009870 tx=0x7f9c3402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb04709354e7d8c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.282+0000 7f9c43e00640 1 -- 172.21.15.26:0/1358496074 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c340317c0 con 0x7f9c440adf20 2024-01-15T18:04:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.282+0000 7f9c43e00640 1 -- 172.21.15.26:0/1358496074 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c34031920 con 0x7f9c440adf20 2024-01-15T18:04:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.282+0000 7f9c43e00640 1 -- 172.21.15.26:0/1358496074 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34031cb0 con 0x7f9c440adf20 2024-01-15T18:04:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.283+0000 7f9c4a400640 1 -- 172.21.15.26:0/1358496074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c440adf20 msgr2=0x7f9c44070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.283+0000 7f9c4a400640 1 --2- 172.21.15.26:0/1358496074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c440adf20 0x7f9c44070a30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009870 tx=0x7f9c3402b860 comp rx=0 tx=0).stop 2024-01-15T18:04:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.283+0000 7f9c4a400640 1 -- 172.21.15.26:0/1358496074 shutdown_connections 2024-01-15T18:04:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.283+0000 7f9c4a400640 1 --2- 172.21.15.26:0/1358496074 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c4415bf10 0x7f9c440cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:54.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.283+0000 7f9c4a400640 1 --2- 172.21.15.26:0/1358496074 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c440adf20 0x7f9c44070a30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:54.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.283+0000 7f9c4a400640 1 -- 172.21.15.26:0/1358496074 >> 172.21.15.26:0/1358496074 conn(0x7f9c4406a7d0 msgr2=0x7f9c4406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:54.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.283+0000 7f9c4a400640 1 -- 172.21.15.26:0/1358496074 shutdown_connections 2024-01-15T18:04:54.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.283+0000 7f9c4a400640 1 -- 172.21.15.26:0/1358496074 wait complete. 2024-01-15T18:04:54.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.285+0000 7f9c4a400640 1 Processor -- start 2024-01-15T18:04:54.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.285+0000 7f9c4a400640 1 -- start start 2024-01-15T18:04:54.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c4a400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c440adf20 0x7f9c440abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:54.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c4a400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c440cc980 0x7f9c440cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:54.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c4a400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c4415bf10 0x7f9c44139020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:54.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c4a400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c44157520 con 0x7f9c4415bf10 2024-01-15T18:04:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c4a400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440ad4a0 con 0x7f9c440cc980 2024-01-15T18:04:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c4a400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c440ac400 con 0x7f9c440adf20 2024-01-15T18:04:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c49a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c4415bf10 0x7f9c44139020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c49a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c4415bf10 0x7f9c44139020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48840/0 (socket says 172.21.15.26:48840) 2024-01-15T18:04:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c49a00640 1 -- 172.21.15.26:0/2446852490 learned_addr learned my addr 172.21.15.26:0/2446852490 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.286+0000 7f9c49000640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c440adf20 0x7f9c440abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.287+0000 7f9c3be00640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c440cc980 0x7f9c440cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.287+0000 7f9c49a00640 1 -- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c440adf20 msgr2=0x7f9c440abb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:54.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.287+0000 7f9c49a00640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c440adf20 0x7f9c440abb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.287+0000 7f9c49a00640 1 -- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c440cc980 msgr2=0x7f9c440cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.287+0000 7f9c49a00640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c440cc980 0x7f9c440cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.287+0000 7f9c49a00640 1 -- 172.21.15.26:0/2446852490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c340094d0 con 0x7f9c4415bf10 2024-01-15T18:04:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.287+0000 7f9c49a00640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c4415bf10 0x7f9c44139020 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c780 tx=0x7f9c3c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.287+0000 7f9c41600640 1 -- 172.21.15.26:0/2446852490 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c004b60 con 0x7f9c4415bf10 2024-01-15T18:04:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.287+0000 7f9c41600640 1 -- 172.21.15.26:0/2446852490 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c00ce90 con 0x7f9c4415bf10 2024-01-15T18:04:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.288+0000 7f9c41600640 1 -- 172.21.15.26:0/2446852490 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00a850 con 0x7f9c4415bf10 2024-01-15T18:04:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.288+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c440ac6e0 con 0x7f9c4415bf10 2024-01-15T18:04:54.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.288+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c440acca0 con 0x7f9c4415bf10 2024-01-15T18:04:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.289+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c44067450 con 0x7f9c4415bf10 2024-01-15T18:04:54.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.290+0000 7f9c41600640 1 -- 172.21.15.26:0/2446852490 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) v1 ==== 90508+0+0 (secure 0 0 0) 0x7f9c3c00a9b0 con 0x7f9c4415bf10 2024-01-15T18:04:54.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.291+0000 7f9c41600640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c1c06bd70 0x7f9c1c06e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:54.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.291+0000 7f9c41600640 1 -- 172.21.15.26:0/2446852490 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f9c3c088930 con 0x7f9c4415bf10 2024-01-15T18:04:54.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.292+0000 7f9c49000640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c1c06bd70 0x7f9c1c06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:54.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.293+0000 7f9c49000640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c1c06bd70 0x7f9c1c06e230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9c34036000 tx=0x7f9c3403d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:54.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.297+0000 7f9c41600640 1 -- 172.21.15.26:0/2446852490 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c3c054fc0 con 0x7f9c4415bf10 2024-01-15T18:04:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.530+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4184", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c4415f170 con 0x7f9c1c06bd70 2024-01-15T18:04:54.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.544+0000 7f9c41600640 1 -- 172.21.15.26:0/2446852490 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f9c34031b10 con 0x7f9c1c06bd70 2024-01-15T18:04:54.548 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:04:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:04:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:04:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:04:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:04:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:04:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:04:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:04:54.549 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:04:54.550 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:04:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c1c06bd70 msgr2=0x7f9c1c06e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:54.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c1c06bd70 0x7f9c1c06e230 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9c34036000 tx=0x7f9c3403d040 comp rx=0 tx=0).stop 2024-01-15T18:04:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c4415bf10 msgr2=0x7f9c44139020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c4415bf10 0x7f9c44139020 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c780 tx=0x7f9c3c00cc40 comp rx=0 tx=0).stop 2024-01-15T18:04:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 shutdown_connections 2024-01-15T18:04:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c1c06bd70 0x7f9c1c06e230 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c4415bf10 0x7f9c44139020 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c440cc980 0x7f9c440cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 --2- 172.21.15.26:0/2446852490 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c440adf20 0x7f9c440abb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 >> 172.21.15.26:0/2446852490 conn(0x7f9c4406a7d0 msgr2=0x7f9c440af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:54.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 shutdown_connections 2024-01-15T18:04:54.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:54.547+0000 7f9c4a400640 1 -- 172.21.15.26:0/2446852490 wait complete. 2024-01-15T18:04:54.572 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:04:55.573 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4184 2024-01-15T18:04:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.804+0000 7fa4b9800640 1 Processor -- start 2024-01-15T18:04:55.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.804+0000 7fa4b9800640 1 -- start start 2024-01-15T18:04:55.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.806+0000 7fa4b9800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa4b406c730 0x7fa4b40b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.806+0000 7fa4b9800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 0x7fa4b4071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.806+0000 7fa4b9800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa4b406cb10 0x7fa4b406cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.806+0000 7fa4b9800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b40708d0 con 0x7fa4b406c730 2024-01-15T18:04:55.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.806+0000 7fa4b9800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b4070bb0 con 0x7fa4b406cb10 2024-01-15T18:04:55.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.806+0000 7fa4b9800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b4070e90 con 0x7fa4b4070160 2024-01-15T18:04:55.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa4b406c730 0x7fa4b40b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa4b406c730 0x7fa4b40b2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48852/0 (socket says 172.21.15.26:48852) 2024-01-15T18:04:55.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b3e00640 1 -- 172.21.15.26:0/2241665772 learned_addr learned my addr 172.21.15.26:0/2241665772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:55.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4ab200640 1 --2- 172.21.15.26:0/2241665772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 0x7fa4b4071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b8e00640 1 --2- 172.21.15.26:0/2241665772 >> v2:172.21.15.124:3300/0 conn(0x7fa4b406cb10 0x7fa4b406cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b3e00640 1 -- 172.21.15.26:0/2241665772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 msgr2=0x7fa4b4071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:55.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b3e00640 1 --2- 172.21.15.26:0/2241665772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 0x7fa4b4071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b3e00640 1 -- 172.21.15.26:0/2241665772 >> v2:172.21.15.124:3300/0 conn(0x7fa4b406cb10 msgr2=0x7fa4b406cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:04:55.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b3e00640 1 --2- 172.21.15.26:0/2241665772 >> v2:172.21.15.124:3300/0 conn(0x7fa4b406cb10 0x7fa4b406cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b3e00640 1 -- 172.21.15.26:0/2241665772 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b4071170 con 0x7fa4b406c730 2024-01-15T18:04:55.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4b8e00640 1 --2- 172.21.15.26:0/2241665772 >> v2:172.21.15.124:3300/0 conn(0x7fa4b406cb10 0x7fa4b406cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:04:55.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.807+0000 7fa4ab200640 1 --2- 172.21.15.26:0/2241665772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 0x7fa4b4071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:04:55.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.808+0000 7fa4b3e00640 1 --2- 172.21.15.26:0/2241665772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 0x7fa4b40b2e00 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa4b406e500 tx=0x7fa4a402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448cd137dfc52d74 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.809+0000 7fa4b3400640 1 -- 172.21.15.26:0/2241665772 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a402ca80 con 0x7fa4b406c730 2024-01-15T18:04:55.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.809+0000 7fa4b3400640 1 -- 172.21.15.26:0/2241665772 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a402cbe0 con 0x7fa4b406c730 2024-01-15T18:04:55.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.809+0000 7fa4b3400640 1 -- 172.21.15.26:0/2241665772 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a40343f0 con 0x7fa4b406c730 2024-01-15T18:04:55.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.809+0000 7fa4b9800640 1 -- 172.21.15.26:0/2241665772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 msgr2=0x7fa4b40b2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.809+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2241665772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 0x7fa4b40b2e00 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa4b406e500 tx=0x7fa4a402bee0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.810+0000 7fa4b9800640 1 -- 172.21.15.26:0/2241665772 shutdown_connections 2024-01-15T18:04:55.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.810+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2241665772 >> v2:172.21.15.124:3300/0 conn(0x7fa4b406cb10 0x7fa4b406cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.810+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2241665772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 0x7fa4b4071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.810+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2241665772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 0x7fa4b40b2e00 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.810+0000 7fa4b9800640 1 -- 172.21.15.26:0/2241665772 >> 172.21.15.26:0/2241665772 conn(0x7fa4b406a7d0 msgr2=0x7fa4b406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:04:55.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.810+0000 7fa4b9800640 1 -- 172.21.15.26:0/2241665772 shutdown_connections 2024-01-15T18:04:55.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.810+0000 7fa4b9800640 1 -- 172.21.15.26:0/2241665772 wait complete. 2024-01-15T18:04:55.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.811+0000 7fa4b9800640 1 Processor -- start 2024-01-15T18:04:55.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.812+0000 7fa4b9800640 1 -- start start 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b9800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 0x7fa4b4160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b9800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b406cb10 0x7fa4b40a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b9800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 0x7fa4b40aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b9800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b4143580 con 0x7fa4b406c730 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b9800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b4160ab0 con 0x7fa4b406cb10 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b9800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b40ad950 con 0x7fa4b4070160 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 0x7fa4b4160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 0x7fa4b4160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48862/0 (socket says 172.21.15.26:48862) 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b3e00640 1 -- 172.21.15.26:0/2994509725 learned_addr learned my addr 172.21.15.26:0/2994509725 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4ab200640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b406cb10 0x7fa4b40a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.813+0000 7fa4b8e00640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 0x7fa4b40aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b3e00640 1 -- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 msgr2=0x7fa4b40aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b3e00640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 0x7fa4b40aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b3e00640 1 -- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b406cb10 msgr2=0x7fa4b40a84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b3e00640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b406cb10 0x7fa4b40a84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b3e00640 1 -- 172.21.15.26:0/2994509725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a4009b30 con 0x7fa4b406c730 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4ab200640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b406cb10 0x7fa4b40a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b3e00640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 0x7fa4b4160570 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4036040 tx=0x7fa4a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b0c00640 1 -- 172.21.15.26:0/2994509725 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a403e450 con 0x7fa4b406c730 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b0c00640 1 -- 172.21.15.26:0/2994509725 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a403e5f0 con 0x7fa4b406c730 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b0c00640 1 -- 172.21.15.26:0/2994509725 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a403e9a0 con 0x7fa4b406c730 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.814+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b40adc30 con 0x7fa4b406c730 2024-01-15T18:04:55.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.815+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b40ae1f0 con 0x7fa4b406c730 2024-01-15T18:04:55.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.816+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b40ae3b0 con 0x7fa4b406c730 2024-01-15T18:04:55.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.817+0000 7fa4b0c00640 1 -- 172.21.15.26:0/2994509725 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 23) v1 ==== 90508+0+0 (secure 0 0 0) 0x7fa4a402cd50 con 0x7fa4b406c730 2024-01-15T18:04:55.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.817+0000 7fa4b0c00640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa48c06bd70 0x7fa48c06e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:04:55.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.818+0000 7fa4b0c00640 1 -- 172.21.15.26:0/2994509725 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fa4a40ab140 con 0x7fa4b406c730 2024-01-15T18:04:55.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.818+0000 7fa4ab200640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa48c06bd70 0x7fa48c06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:04:55.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.819+0000 7fa4ab200640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa48c06bd70 0x7fa48c06e230 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa49c006fd0 tx=0x7fa49c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:04:55.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:55.823+0000 7fa4b0c00640 1 -- 172.21.15.26:0/2994509725 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa4a4077a00 con 0x7fa4b406c730 2024-01-15T18:04:55.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:55.975+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:55.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:55.975+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:55.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:55.975+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:55.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:55.975+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:55.979 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:04:55.975+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:04:56.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:56.062+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4184", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4b4071eb0 con 0x7fa48c06bd70 2024-01-15T18:04:57.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:04:57.224+0000 7fa4b0c00640 1 -- 172.21.15.26:0/2994509725 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 24) v1 ==== 90559+0+0 (secure 0 0 0) 0x7fa4a4077dd0 con 0x7fa4b406c730 2024-01-15T18:04:58.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:04:58.971 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:05:00.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.955+0000 7fa4b0c00640 1 -- 172.21.15.26:0/2994509725 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fa49c00c040 con 0x7fa48c06bd70 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:00.959 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa48c06bd70 msgr2=0x7fa48c06e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa48c06bd70 0x7fa48c06e230 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa49c006fd0 tx=0x7fa49c008040 comp rx=0 tx=0).stop 2024-01-15T18:05:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 msgr2=0x7fa4b4160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 0x7fa4b4160570 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4036040 tx=0x7fa4a4004a40 comp rx=0 tx=0).stop 2024-01-15T18:05:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 shutdown_connections 2024-01-15T18:05:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa4b4070160 0x7fa4b40aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa48c06bd70 0x7fa48c06e230 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa4b406cb10 0x7fa4b40a84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 --2- 172.21.15.26:0/2994509725 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa4b406c730 0x7fa4b4160570 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 >> 172.21.15.26:0/2994509725 conn(0x7fa4b406a7d0 msgr2=0x7fa4b4145ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.958+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 shutdown_connections 2024-01-15T18:05:00.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:00.959+0000 7fa4b9800640 1 -- 172.21.15.26:0/2994509725 wait complete. 2024-01-15T18:05:00.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:05:00.975+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:05:00.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:05:00.975+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:05:00.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:05:00.975+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:05:00.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:05:00.975+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:05:00.978 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:05:00.975+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:05:00.984 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:05:01.985 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4184 2024-01-15T18:05:02.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.318+0000 7f4fa6200640 1 Processor -- start 2024-01-15T18:05:02.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.319+0000 7f4fa6200640 1 -- start start 2024-01-15T18:05:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.321+0000 7f4fa6200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4fa00cd170 0x7f4fa0071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.321+0000 7f4fa6200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa0071690 con 0x7f4fa00cf9a0 2024-01-15T18:05:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.321+0000 7f4fa6200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa0071970 con 0x7f4fa014b2c0 2024-01-15T18:05:02.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.321+0000 7f4fa6200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa014b6a0 con 0x7f4fa00cd170 2024-01-15T18:05:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.323+0000 7f4f9f400640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f4fa014b2c0 0x7f4fa00cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:60638/0 (socket says 172.21.15.26:60638) 2024-01-15T18:05:02.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.323+0000 7f4f9f400640 1 -- 172.21.15.26:0/1125264985 learned_addr learned my addr 172.21.15.26:0/1125264985 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.324+0000 7f4f9fe00640 1 --2- 172.21.15.26:0/1125264985 >> v2:172.21.15.26:3300/0 conn(0x7f4fa00cd170 0x7f4fa0071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.324+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953991985 0 0) 0x7f4fa0071970 con 0x7f4fa014b2c0 2024-01-15T18:05:02.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.325+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f80003520 con 0x7f4fa014b2c0 2024-01-15T18:05:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 <== mon.1 v1:172.21.15.124:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3052765808 0 0) 0x7f4f80003520 con 0x7f4fa014b2c0 2024-01-15T18:05:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 >> v2:172.21.15.26:3300/0 conn(0x7f4fa00cd170 msgr2=0x7f4fa0071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 --2- 172.21.15.26:0/1125264985 >> v2:172.21.15.26:3300/0 conn(0x7f4fa00cd170 0x7f4fa0071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 >> v1:172.21.15.26:6789/0 conn(0x7f4fa00cf9a0 legacy=0x7f4fa0070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 --> v1:172.21.15.124:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa0071c50 con 0x7f4fa014b2c0 2024-01-15T18:05:02.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 <== mon.1 v1:172.21.15.124:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f4f90001670 con 0x7f4fa014b2c0 2024-01-15T18:05:02.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 >> v1:172.21.15.124:6789/0 conn(0x7f4fa014b2c0 legacy=0x7f4fa00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 0x7f4f80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f800047c0 0x7f4f80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f800051d0 0x7f4f8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa014b6a0 con 0x7f4f800051d0 2024-01-15T18:05:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8000c120 con 0x7f4f800047c0 2024-01-15T18:05:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.326+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8000c2e0 con 0x7f4f800039c0 2024-01-15T18:05:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.327+0000 7f4f9fe00640 1 --2- 172.21.15.26:0/1125264985 >> v2:172.21.15.26:3300/0 conn(0x7f4fa00cd170 0x7f4fa0071070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:02.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.327+0000 7f4f9fe00640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f800051d0 0x7f4f8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.327+0000 7f4f9f400640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 0x7f4f80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.327+0000 7f4f9ea00640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f800047c0 0x7f4f80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.328+0000 7f4f9f400640 1 -- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f800047c0 msgr2=0x7f4f80004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.328+0000 7f4f9f400640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4f800047c0 0x7f4f80004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.328+0000 7f4f9f400640 1 -- 172.21.15.26:0/1125264985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f800051d0 msgr2=0x7f4f8000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.328+0000 7f4f9f400640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f800051d0 0x7f4f8000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.328+0000 7f4f9f400640 1 -- 172.21.15.26:0/1125264985 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80003520 con 0x7f4f800039c0 2024-01-15T18:05:02.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.328+0000 7f4f9fe00640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4f800051d0 0x7f4f8000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:02.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.328+0000 7f4f9fe00640 1 -- 172.21.15.26:0/1125264985 reap_dead start 2024-01-15T18:05:02.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.330+0000 7f4f9f400640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 0x7f4f80003de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000c0e0 tx=0x7f4f800057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2ec658ff079f8f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:02.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.332+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90005ad0 con 0x7f4f800039c0 2024-01-15T18:05:02.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.332+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f90005c30 con 0x7f4f800039c0 2024-01-15T18:05:02.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.332+0000 7f4f9e000640 1 -- 172.21.15.26:0/1125264985 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f9002c790 con 0x7f4f800039c0 2024-01-15T18:05:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.335+0000 7f4fa6200640 1 -- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 msgr2=0x7f4f80003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.335+0000 7f4fa6200640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 0x7f4f80003de0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000c0e0 tx=0x7f4f800057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.335+0000 7f4fa6200640 1 -- 172.21.15.26:0/1125264985 shutdown_connections 2024-01-15T18:05:02.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.335+0000 7f4fa6200640 1 --2- 172.21.15.26:0/1125264985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 0x7f4f80003de0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.335+0000 7f4fa6200640 1 -- 172.21.15.26:0/1125264985 >> 172.21.15.26:0/1125264985 conn(0x7f4fa006a800 msgr2=0x7f4fa006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:02.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.335+0000 7f4fa6200640 1 -- 172.21.15.26:0/1125264985 shutdown_connections 2024-01-15T18:05:02.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.335+0000 7f4fa6200640 1 -- 172.21.15.26:0/1125264985 wait complete. 2024-01-15T18:05:02.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.338+0000 7f4fa6200640 1 Processor -- start 2024-01-15T18:05:02.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.338+0000 7f4fa6200640 1 -- start start 2024-01-15T18:05:02.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.340+0000 7f4fa6200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 0x7f4fa0158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.340+0000 7f4fa6200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa015ffb0 0x7f4fa01facf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.340+0000 7f4fa6200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fa01fb230 0x7f4fa0162270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.340+0000 7f4fa6200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa0139470 con 0x7f4fa015ffb0 2024-01-15T18:05:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.340+0000 7f4fa6200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa01fb6b0 con 0x7f4fa01fb230 2024-01-15T18:05:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.340+0000 7f4fa6200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa0159290 con 0x7f4f800039c0 2024-01-15T18:05:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.340+0000 7f4f9ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa015ffb0 0x7f4fa01facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.340+0000 7f4f9ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa015ffb0 0x7f4fa01facf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34360/0 (socket says 172.21.15.26:34360) 2024-01-15T18:05:02.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.340+0000 7f4f9ea00640 1 -- 172.21.15.26:0/1696745969 learned_addr learned my addr 172.21.15.26:0/1696745969 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.341+0000 7f4f9f400640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 0x7f4fa0158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.341+0000 7f4f9fe00640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fa01fb230 0x7f4fa0162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.341+0000 7f4f9ea00640 1 -- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 msgr2=0x7f4fa0158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.341+0000 7f4f9ea00640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 0x7f4fa0158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.341+0000 7f4f9ea00640 1 -- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fa01fb230 msgr2=0x7f4fa0162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.341+0000 7f4f9ea00640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fa01fb230 0x7f4fa0162270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.341+0000 7f4f9ea00640 1 -- 172.21.15.26:0/1696745969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa015ef10 con 0x7f4fa015ffb0 2024-01-15T18:05:02.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.342+0000 7f4f9ea00640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa015ffb0 0x7f4fa01facf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e480 tx=0x7f4f8800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.342+0000 7f4f97400640 1 -- 172.21.15.26:0/1696745969 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f880052e0 con 0x7f4fa015ffb0 2024-01-15T18:05:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.342+0000 7f4f97400640 1 -- 172.21.15.26:0/1696745969 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f88005440 con 0x7f4fa015ffb0 2024-01-15T18:05:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.342+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa015f1f0 con 0x7f4fa015ffb0 2024-01-15T18:05:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.342+0000 7f4f97400640 1 -- 172.21.15.26:0/1696745969 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88003810 con 0x7f4fa015ffb0 2024-01-15T18:05:02.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.343+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa015f7b0 con 0x7f4fa015ffb0 2024-01-15T18:05:02.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.344+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa0072610 con 0x7f4fa015ffb0 2024-01-15T18:05:02.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.346+0000 7f4f97400640 1 -- 172.21.15.26:0/1696745969 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 24) v1 ==== 90559+0+0 (secure 0 0 0) 0x7f4f88003970 con 0x7f4fa015ffb0 2024-01-15T18:05:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.348+0000 7f4f97400640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4f7806be50 0x7f4f7806e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:02.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.349+0000 7f4f97400640 1 -- 172.21.15.26:0/1696745969 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f4f880882e0 con 0x7f4fa015ffb0 2024-01-15T18:05:02.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.356+0000 7f4f97400640 1 -- 172.21.15.26:0/1696745969 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4f88090020 con 0x7f4fa015ffb0 2024-01-15T18:05:02.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.367+0000 7f4f9f400640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4f7806be50 0x7f4f7806e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:02.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.370+0000 7f4f9f400640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4f7806be50 0x7f4f7806e310 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4f90034000 tx=0x7f4f90009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:02.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.711+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4184", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fa0071970 con 0x7f4f7806be50 2024-01-15T18:05:02.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.742+0000 7f4f97400640 1 -- 172.21.15.26:0/1696745969 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7f4fa0071970 con 0x7f4f7806be50 2024-01-15T18:05:02.747 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:02.747 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:05:02.747 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:05:02.747 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:05:02.747 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:05:02.747 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:05:02.747 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 0.0, 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "0/3", 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:02.748 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:02.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.745+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4f7806be50 msgr2=0x7f4f7806e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.745+0000 7f4fa6200640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4f7806be50 0x7f4f7806e310 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4f90034000 tx=0x7f4f90009030 comp rx=0 tx=0).stop 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa015ffb0 msgr2=0x7f4fa01facf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa015ffb0 0x7f4fa01facf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800e480 tx=0x7f4f8800e940 comp rx=0 tx=0).stop 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 shutdown_connections 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4f7806be50 0x7f4f7806e310 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fa01fb230 0x7f4fa0162270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fa015ffb0 0x7f4fa01facf0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 --2- 172.21.15.26:0/1696745969 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4f800039c0 0x7f4fa0158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 >> 172.21.15.26:0/1696745969 conn(0x7f4fa006a800 msgr2=0x7f4fa00ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 shutdown_connections 2024-01-15T18:05:02.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:02.746+0000 7f4fa6200640 1 -- 172.21.15.26:0/1696745969 wait complete. 2024-01-15T18:05:02.769 INFO:tasks.cephfs.test_volumes:expected clone state = ('pending', 'in-progress') 2024-01-15T18:05:02.770 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4187 2024-01-15T18:05:03.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.079+0000 7f25ba400640 1 Processor -- start 2024-01-15T18:05:03.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.079+0000 7f25ba400640 1 -- start start 2024-01-15T18:05:03.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.082+0000 7f25ba400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b40cf9a0 0x7f25b4070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.082+0000 7f25ba400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f25b4070ea0 0x7f25b40cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.082+0000 7f25ba400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b4071690 con 0x7f25b414b2c0 2024-01-15T18:05:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.082+0000 7f25ba400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b414b6a0 con 0x7f25b4070ea0 2024-01-15T18:05:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.082+0000 7f25ba400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b4071970 con 0x7f25b40cf9a0 2024-01-15T18:05:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.082+0000 7f25b3400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f25b414b2c0 0x7f25b40cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41764/0 (socket says 172.21.15.26:41764) 2024-01-15T18:05:03.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.083+0000 7f25b3400640 1 -- 172.21.15.26:0/2740544023 learned_addr learned my addr 172.21.15.26:0/2740544023 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.084+0000 7f25b3e00640 1 --2- 172.21.15.26:0/2740544023 >> v2:172.21.15.26:3300/0 conn(0x7f25b4070ea0 0x7f25b40cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.085+0000 7f25b2a00640 1 --2- 172.21.15.26:0/2740544023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b40cf9a0 0x7f25b4070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.085+0000 7f25b3e00640 1 -- 172.21.15.26:0/2740544023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b40cf9a0 msgr2=0x7f25b4070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.085+0000 7f25b3e00640 1 --2- 172.21.15.26:0/2740544023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b40cf9a0 0x7f25b4070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.085+0000 7f25b3e00640 1 -- 172.21.15.26:0/2740544023 >> v1:172.21.15.26:6789/0 conn(0x7f25b414b2c0 legacy=0x7f25b40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.086+0000 7f25b3e00640 1 -- 172.21.15.26:0/2740544023 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b4071c50 con 0x7f25b4070ea0 2024-01-15T18:05:03.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.092+0000 7f25b3e00640 1 --2- 172.21.15.26:0/2740544023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b4070ea0 0x7f25b40cc160 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0097a0 tx=0x7f25ac02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9662ce8788416f78 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.094+0000 7f25b2000640 1 -- 172.21.15.26:0/2740544023 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac02d040 con 0x7f25b4070ea0 2024-01-15T18:05:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.094+0000 7f25b2000640 1 -- 172.21.15.26:0/2740544023 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac031810 con 0x7f25b4070ea0 2024-01-15T18:05:03.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.094+0000 7f25b2000640 1 -- 172.21.15.26:0/2740544023 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac031ba0 con 0x7f25b4070ea0 2024-01-15T18:05:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.095+0000 7f25ba400640 1 -- 172.21.15.26:0/2740544023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b4070ea0 msgr2=0x7f25b40cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.095+0000 7f25ba400640 1 --2- 172.21.15.26:0/2740544023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b4070ea0 0x7f25b40cc160 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0097a0 tx=0x7f25ac02b8b0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.096+0000 7f25ba400640 1 -- 172.21.15.26:0/2740544023 shutdown_connections 2024-01-15T18:05:03.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.096+0000 7f25ba400640 1 --2- 172.21.15.26:0/2740544023 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b4070ea0 0x7f25b40cc160 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.096+0000 7f25ba400640 1 --2- 172.21.15.26:0/2740544023 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b40cf9a0 0x7f25b4070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.096+0000 7f25ba400640 1 -- 172.21.15.26:0/2740544023 >> 172.21.15.26:0/2740544023 conn(0x7f25b406a800 msgr2=0x7f25b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:03.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.096+0000 7f25ba400640 1 -- 172.21.15.26:0/2740544023 shutdown_connections 2024-01-15T18:05:03.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.097+0000 7f25ba400640 1 -- 172.21.15.26:0/2740544023 wait complete. 2024-01-15T18:05:03.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.100+0000 7f25ba400640 1 Processor -- start 2024-01-15T18:05:03.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.100+0000 7f25ba400640 1 -- start start 2024-01-15T18:05:03.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.102+0000 7f25ba400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b4070ea0 0x7f25b4161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.102+0000 7f25ba400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b40cf9a0 0x7f25b41fb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.102+0000 7f25ba400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25b414b2c0 0x7f25b4142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.102+0000 7f25ba400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b41635a0 con 0x7f25b40cf9a0 2024-01-15T18:05:03.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.102+0000 7f25ba400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b4163180 con 0x7f25b414b2c0 2024-01-15T18:05:03.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.102+0000 7f25ba400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b41fb580 con 0x7f25b4070ea0 2024-01-15T18:05:03.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.103+0000 7f25b2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b40cf9a0 0x7f25b41fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.103+0000 7f25b2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b40cf9a0 0x7f25b41fb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34382/0 (socket says 172.21.15.26:34382) 2024-01-15T18:05:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.103+0000 7f25b2a00640 1 -- 172.21.15.26:0/2479286459 learned_addr learned my addr 172.21.15.26:0/2479286459 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.104+0000 7f25b3e00640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25b414b2c0 0x7f25b4142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.104+0000 7f25b3400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b4070ea0 0x7f25b4161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.106+0000 7f25b3e00640 1 -- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b4070ea0 msgr2=0x7f25b4161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.106+0000 7f25b3e00640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b4070ea0 0x7f25b4161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.106+0000 7f25b3e00640 1 -- 172.21.15.26:0/2479286459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b40cf9a0 msgr2=0x7f25b41fb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.106+0000 7f25b3e00640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b40cf9a0 0x7f25b41fb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.106+0000 7f25b3e00640 1 -- 172.21.15.26:0/2479286459 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac009400 con 0x7f25b414b2c0 2024-01-15T18:05:03.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.106+0000 7f25b2a00640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b40cf9a0 0x7f25b41fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:03.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.110+0000 7f25b3e00640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25b414b2c0 0x7f25b4142130 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f25ac033000 tx=0x7f25ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.112+0000 7f25ab400640 1 -- 172.21.15.26:0/2479286459 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac006e10 con 0x7f25b414b2c0 2024-01-15T18:05:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.113+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b4142670 con 0x7f25b414b2c0 2024-01-15T18:05:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.113+0000 7f25ab400640 1 -- 172.21.15.26:0/2479286459 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac03ace0 con 0x7f25b414b2c0 2024-01-15T18:05:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.113+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b41627e0 con 0x7f25b414b2c0 2024-01-15T18:05:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.114+0000 7f25ab400640 1 -- 172.21.15.26:0/2479286459 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac044620 con 0x7f25b414b2c0 2024-01-15T18:05:03.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.115+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b41632c0 con 0x7f25b414b2c0 2024-01-15T18:05:03.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.123+0000 7f25ab400640 1 -- 172.21.15.26:0/2479286459 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 24) v1 ==== 90559+0+0 (secure 0 0 0) 0x7f25ac04d440 con 0x7f25b414b2c0 2024-01-15T18:05:03.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.124+0000 7f25ab400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f258c06be30 0x7f258c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.124+0000 7f25ab400640 1 -- 172.21.15.26:0/2479286459 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f25ac0aa9d0 con 0x7f25b414b2c0 2024-01-15T18:05:03.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.130+0000 7f25ab400640 1 -- 172.21.15.26:0/2479286459 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25ac077300 con 0x7f25b414b2c0 2024-01-15T18:05:03.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.142+0000 7f25b3400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f258c06be30 0x7f258c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.144+0000 7f25b3400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f258c06be30 0x7f258c06e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f25a4005c00 tx=0x7f25a4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.277+0000 7f25ab400640 1 -- 172.21.15.26:0/2479286459 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f25ac0776d0 con 0x7f25b414b2c0 2024-01-15T18:05:03.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.511+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4187", "target": ["mon-mgr", ""]}) v1 -- 0x7f25b4071330 con 0x7f258c06be30 2024-01-15T18:05:03.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.585+0000 7f25ab400640 1 -- 172.21.15.26:0/2479286459 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f25a4001ee0 con 0x7f258c06be30 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:03.589 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f258c06be30 msgr2=0x7f258c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f258c06be30 0x7f258c06e2f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f25a4005c00 tx=0x7f25a4009040 comp rx=0 tx=0).stop 2024-01-15T18:05:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25b414b2c0 msgr2=0x7f25b4142130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25b414b2c0 0x7f25b4142130 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f25ac033000 tx=0x7f25ac0046c0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 shutdown_connections 2024-01-15T18:05:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f258c06be30 0x7f258c06e2f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25b414b2c0 0x7f25b4142130 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25b40cf9a0 0x7f25b41fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 --2- 172.21.15.26:0/2479286459 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25b4070ea0 0x7f25b4161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 >> 172.21.15.26:0/2479286459 conn(0x7f25b406a800 msgr2=0x7f25b406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.588+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 shutdown_connections 2024-01-15T18:05:03.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.589+0000 7f25ba400640 1 -- 172.21.15.26:0/2479286459 wait complete. 2024-01-15T18:05:03.632 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:05:03.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.893+0000 7f9bb9600640 1 Processor -- start 2024-01-15T18:05:03.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.893+0000 7f9bb9600640 1 -- start start 2024-01-15T18:05:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.894+0000 7f9bb9600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9bb40cf9a0 0x7f9bb415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.895+0000 7f9bb9600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb40b1ee0 con 0x7f9bb4143ae0 2024-01-15T18:05:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.895+0000 7f9bb9600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb40b2780 con 0x7f9bb415e1e0 2024-01-15T18:05:03.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.895+0000 7f9bb9600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb415e990 con 0x7f9bb40cf9a0 2024-01-15T18:05:03.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.895+0000 7f9bb2a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9bb4143ae0 0x7f9bb40cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41766/0 (socket says 172.21.15.26:41766) 2024-01-15T18:05:03.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.895+0000 7f9bb2a00640 1 -- 172.21.15.26:0/3601885643 learned_addr learned my addr 172.21.15.26:0/3601885643 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:03.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461244938 0 0) 0x7f9bb40b1ee0 con 0x7f9bb4143ae0 2024-01-15T18:05:03.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b94003520 con 0x7f9bb4143ae0 2024-01-15T18:05:03.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1997591947 0 0) 0x7f9b94003520 con 0x7f9bb4143ae0 2024-01-15T18:05:03.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 >> v2:172.21.15.124:3300/0 conn(0x7f9bb40cf9a0 msgr2=0x7f9bb415dca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:05:03.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 --2- 172.21.15.26:0/3601885643 >> v2:172.21.15.124:3300/0 conn(0x7f9bb40cf9a0 0x7f9bb415dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 >> v1:172.21.15.124:6789/0 conn(0x7f9bb415e1e0 legacy=0x7f9bb40702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb40b2130 con 0x7f9bb4143ae0 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9ba4001670 con 0x7f9bb4143ae0 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 >> v1:172.21.15.26:6789/0 conn(0x7f9bb4143ae0 legacy=0x7f9bb40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 --2- 172.21.15.26:0/3601885643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b94003970 0x7f9b94003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 --2- 172.21.15.26:0/3601885643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b940047c0 0x7f9b94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 --2- 172.21.15.26:0/3601885643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b940051f0 0x7f9b9400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb415e990 con 0x7f9b940051f0 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9400e360 con 0x7f9b94003970 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.896+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9400e520 con 0x7f9b940047c0 2024-01-15T18:05:03.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.897+0000 7f9bb3400640 1 --2- 172.21.15.26:0/3601885643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b940051f0 0x7f9b9400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.897+0000 7f9bb3400640 1 -- 172.21.15.26:0/3601885643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b940047c0 msgr2=0x7f9b94004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:03.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.897+0000 7f9bb3400640 1 --2- 172.21.15.26:0/3601885643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b940047c0 0x7f9b94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.897+0000 7f9bb3400640 1 -- 172.21.15.26:0/3601885643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b94003970 msgr2=0x7f9b94003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:03.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.897+0000 7f9bb3400640 1 --2- 172.21.15.26:0/3601885643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b94003970 0x7f9b94003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.897+0000 7f9bb3400640 1 -- 172.21.15.26:0/3601885643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb40b2130 con 0x7f9b940051f0 2024-01-15T18:05:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.899+0000 7f9bb3400640 1 -- 172.21.15.26:0/3601885643 reap_dead start 2024-01-15T18:05:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.899+0000 7f9bb3400640 1 --2- 172.21.15.26:0/3601885643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b940051f0 0x7f9b9400db90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f9bac004040 tx=0x7f9b940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bcc2ced841a7b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.899+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac030920 con 0x7f9b940051f0 2024-01-15T18:05:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.899+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac030a80 con 0x7f9b940051f0 2024-01-15T18:05:03.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.899+0000 7f9bb1600640 1 -- 172.21.15.26:0/3601885643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac030e10 con 0x7f9b940051f0 2024-01-15T18:05:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.900+0000 7f9bb9600640 1 -- 172.21.15.26:0/3601885643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b940051f0 msgr2=0x7f9b9400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.900+0000 7f9bb9600640 1 --2- 172.21.15.26:0/3601885643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b940051f0 0x7f9b9400db90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f9bac004040 tx=0x7f9b940057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.901+0000 7f9bb9600640 1 -- 172.21.15.26:0/3601885643 shutdown_connections 2024-01-15T18:05:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.901+0000 7f9bb9600640 1 --2- 172.21.15.26:0/3601885643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b940051f0 0x7f9b9400db90 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.901+0000 7f9bb9600640 1 -- 172.21.15.26:0/3601885643 >> 172.21.15.26:0/3601885643 conn(0x7f9bb406a800 msgr2=0x7f9bb406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.901+0000 7f9bb9600640 1 -- 172.21.15.26:0/3601885643 shutdown_connections 2024-01-15T18:05:03.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.901+0000 7f9bb9600640 1 -- 172.21.15.26:0/3601885643 wait complete. 2024-01-15T18:05:03.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.905+0000 7f9bb9600640 1 Processor -- start 2024-01-15T18:05:03.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.905+0000 7f9bb9600640 1 -- start start 2024-01-15T18:05:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.906+0000 7f9bb9600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b940051f0 0x7f9bb4070530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.906+0000 7f9bb9600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb40cce90 0x7f9bb412ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.906+0000 7f9bb9600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bb413ebf0 0x7f9bb40ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.906+0000 7f9bb9600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb40cd320 con 0x7f9bb40cce90 2024-01-15T18:05:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.906+0000 7f9bb9600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb40cd600 con 0x7f9bb413ebf0 2024-01-15T18:05:03.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.906+0000 7f9bb9600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb40cd8e0 con 0x7f9b940051f0 2024-01-15T18:05:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.908+0000 7f9bb2a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b940051f0 0x7f9bb4070530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.908+0000 7f9bb2a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b940051f0 0x7f9bb4070530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:42718/0 (socket says 172.21.15.26:42718) 2024-01-15T18:05:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.908+0000 7f9bb2a00640 1 -- 172.21.15.26:0/1024394555 learned_addr learned my addr 172.21.15.26:0/1024394555 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.910+0000 7f9bb3400640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bb413ebf0 0x7f9bb40ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.910+0000 7f9bb2000640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb40cce90 0x7f9bb412ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.911+0000 7f9bb2000640 1 -- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b940051f0 msgr2=0x7f9bb4070530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.911+0000 7f9bb2000640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b940051f0 0x7f9bb4070530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.911+0000 7f9bb2000640 1 -- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bb413ebf0 msgr2=0x7f9bb40ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:03.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.911+0000 7f9bb2000640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bb413ebf0 0x7f9bb40ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:03.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.911+0000 7f9bb2000640 1 -- 172.21.15.26:0/1024394555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb40cdbc0 con 0x7f9bb40cce90 2024-01-15T18:05:03.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.912+0000 7f9bb2000640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb40cce90 0x7f9bb412ffa0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00b410 tx=0x7f9b9c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:03.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.912+0000 7f9baaa00640 1 -- 172.21.15.26:0/1024394555 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c011830 con 0x7f9bb40cce90 2024-01-15T18:05:03.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.913+0000 7f9baaa00640 1 -- 172.21.15.26:0/1024394555 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c0119b0 con 0x7f9bb40cce90 2024-01-15T18:05:03.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.913+0000 7f9baaa00640 1 -- 172.21.15.26:0/1024394555 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c011d60 con 0x7f9bb40cce90 2024-01-15T18:05:03.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.913+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb40cdea0 con 0x7f9bb40cce90 2024-01-15T18:05:03.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.914+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb4070c50 con 0x7f9bb40cce90 2024-01-15T18:05:03.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.916+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb40ce090 con 0x7f9bb40cce90 2024-01-15T18:05:03.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.917+0000 7f9baaa00640 1 -- 172.21.15.26:0/1024394555 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f9b9c0193b0 con 0x7f9bb40cce90 2024-01-15T18:05:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.918+0000 7f9baaa00640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b8c06bee0 0x7f9b8c06e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:03.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.919+0000 7f9baaa00640 1 -- 172.21.15.26:0/1024394555 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f9b9c088390 con 0x7f9bb40cce90 2024-01-15T18:05:03.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.924+0000 7f9baaa00640 1 -- 172.21.15.26:0/1024394555 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b9c054a30 con 0x7f9bb40cce90 2024-01-15T18:05:03.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.928+0000 7f9bb2a00640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b8c06bee0 0x7f9b8c06e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:03.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:03.931+0000 7f9bb2a00640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b8c06bee0 0x7f9b8c06e3a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001820 tx=0x7f9ba4001770 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:04.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.337+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9bb4070c50 con 0x7f9bb40cce90 2024-01-15T18:05:04.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.337+0000 7f9baaa00640 1 -- 172.21.15.26:0/1024394555 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2347 (secure 0 0 0) 0x7f9bb4070c50 con 0x7f9bb40cce90 2024-01-15T18:05:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:04.341 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:04.341 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 618, 2024-01-15T18:05:04.342 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:04.343 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 925, 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3729988380, 2024-01-15T18:05:04.344 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 7296094208, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 765798019072, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 49615324, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 42302637, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 12, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 17 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:04.345 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:04.346 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 0.0% (9s)\n [............................] ", 2024-01-15T18:05:04.347 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0, 2024-01-15T18:05:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 0.0% (9s)\n [............................] ", 2024-01-15T18:05:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0, 2024-01-15T18:05:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:04.348 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:04.348 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b8c06bee0 msgr2=0x7f9b8c06e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b8c06bee0 0x7f9b8c06e3a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001820 tx=0x7f9ba4001770 comp rx=0 tx=0).stop 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb40cce90 msgr2=0x7f9bb412ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb40cce90 0x7f9bb412ffa0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00b410 tx=0x7f9b9c009f90 comp rx=0 tx=0).stop 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 shutdown_connections 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b8c06bee0 0x7f9b8c06e3a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9bb413ebf0 0x7f9bb40ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9bb40cce90 0x7f9bb412ffa0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 --2- 172.21.15.26:0/1024394555 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b940051f0 0x7f9bb4070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 >> 172.21.15.26:0/1024394555 conn(0x7f9bb406a800 msgr2=0x7f9bb406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 shutdown_connections 2024-01-15T18:05:04.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:04.340+0000 7f9bb9600640 1 -- 172.21.15.26:0/1024394555 wait complete. 2024-01-15T18:05:05.362 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:05.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.592+0000 7f2b5b200640 1 Processor -- start 2024-01-15T18:05:05.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.592+0000 7f2b5b200640 1 -- start start 2024-01-15T18:05:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.593+0000 7f2b5b200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2b5c160920 0x7f2b5c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.593+0000 7f2b5b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b5c0cf9f0 0x7f2b5c0cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.594+0000 7f2b5b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5c0d0a10 con 0x7f2b5c161010 2024-01-15T18:05:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.594+0000 7f2b5b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5c0d03e0 con 0x7f2b5c0cf9f0 2024-01-15T18:05:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.594+0000 7f2b5b200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5c0d1f50 con 0x7f2b5c160920 2024-01-15T18:05:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.594+0000 7f2b59400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b5c0cf9f0 0x7f2b5c0cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.594+0000 7f2b59400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2b5c0cf9f0 0x7f2b5c0cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34414/0 (socket says 172.21.15.26:34414) 2024-01-15T18:05:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.594+0000 7f2b59400640 1 -- 172.21.15.26:0/3359700676 learned_addr learned my addr 172.21.15.26:0/3359700676 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:05.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.594+0000 7f2b59e00640 1 --2- 172.21.15.26:0/3359700676 >> v2:172.21.15.124:3300/0 conn(0x7f2b5c160920 0x7f2b5c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.595+0000 7f2b59400640 1 -- 172.21.15.26:0/3359700676 >> v2:172.21.15.124:3300/0 conn(0x7f2b5c160920 msgr2=0x7f2b5c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.595+0000 7f2b59400640 1 --2- 172.21.15.26:0/3359700676 >> v2:172.21.15.124:3300/0 conn(0x7f2b5c160920 0x7f2b5c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.595+0000 7f2b59400640 1 -- 172.21.15.26:0/3359700676 >> v1:172.21.15.124:6789/0 conn(0x7f2b5c161010 legacy=0x7f2b5c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:05.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.595+0000 7f2b59400640 1 -- 172.21.15.26:0/3359700676 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c0cf080 con 0x7f2b5c0cf9f0 2024-01-15T18:05:05.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.596+0000 7f2b59400640 1 --2- 172.21.15.26:0/3359700676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c0cf9f0 0x7f2b5c0cfea0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c08c060 tx=0x7f2b4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168d52e7d64707d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.596+0000 7f2b58a00640 1 -- 172.21.15.26:0/3359700676 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44031870 con 0x7f2b5c0cf9f0 2024-01-15T18:05:05.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.596+0000 7f2b58a00640 1 -- 172.21.15.26:0/3359700676 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b440319d0 con 0x7f2b5c0cf9f0 2024-01-15T18:05:05.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.596+0000 7f2b58a00640 1 -- 172.21.15.26:0/3359700676 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44031d60 con 0x7f2b5c0cf9f0 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.597+0000 7f2b5b200640 1 -- 172.21.15.26:0/3359700676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c0cf9f0 msgr2=0x7f2b5c0cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.597+0000 7f2b5b200640 1 --2- 172.21.15.26:0/3359700676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c0cf9f0 0x7f2b5c0cfea0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c08c060 tx=0x7f2b4402b910 comp rx=0 tx=0).stop 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.597+0000 7f2b5b200640 1 -- 172.21.15.26:0/3359700676 shutdown_connections 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.597+0000 7f2b5b200640 1 --2- 172.21.15.26:0/3359700676 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c0cf9f0 0x7f2b5c0cfea0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.597+0000 7f2b5b200640 1 --2- 172.21.15.26:0/3359700676 >> v2:172.21.15.124:3300/0 conn(0x7f2b5c160920 0x7f2b5c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.597+0000 7f2b5b200640 1 -- 172.21.15.26:0/3359700676 >> 172.21.15.26:0/3359700676 conn(0x7f2b5c0d3090 msgr2=0x7f2b5c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.597+0000 7f2b5b200640 1 -- 172.21.15.26:0/3359700676 shutdown_connections 2024-01-15T18:05:05.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.597+0000 7f2b5b200640 1 -- 172.21.15.26:0/3359700676 wait complete. 2024-01-15T18:05:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.599+0000 7f2b5b200640 1 Processor -- start 2024-01-15T18:05:05.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.599+0000 7f2b5b200640 1 -- start start 2024-01-15T18:05:05.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.600+0000 7f2b5b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c0cf9f0 0x7f2b5c1f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.600+0000 7f2b5b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c160920 0x7f2b5c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.600+0000 7f2b5b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b5c161010 0x7f2b5c0cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.600+0000 7f2b5b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c1f7310 con 0x7f2b5c160920 2024-01-15T18:05:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.600+0000 7f2b5b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c0b6d00 con 0x7f2b5c161010 2024-01-15T18:05:05.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.600+0000 7f2b5b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c1f3360 con 0x7f2b5c0cf9f0 2024-01-15T18:05:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.600+0000 7f2b59400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c160920 0x7f2b5c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.601+0000 7f2b59400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c160920 0x7f2b5c157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34426/0 (socket says 172.21.15.26:34426) 2024-01-15T18:05:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.601+0000 7f2b59400640 1 -- 172.21.15.26:0/931516367 learned_addr learned my addr 172.21.15.26:0/931516367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.601+0000 7f2b59e00640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c0cf9f0 0x7f2b5c1f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.601+0000 7f2b5a800640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b5c161010 0x7f2b5c0cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.601+0000 7f2b59400640 1 -- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c0cf9f0 msgr2=0x7f2b5c1f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.601+0000 7f2b59400640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c0cf9f0 0x7f2b5c1f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.601+0000 7f2b59400640 1 -- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b5c161010 msgr2=0x7f2b5c0cbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.601+0000 7f2b59400640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b5c161010 0x7f2b5c0cbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.601+0000 7f2b59400640 1 -- 172.21.15.26:0/931516367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b440094d0 con 0x7f2b5c160920 2024-01-15T18:05:05.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.602+0000 7f2b59400640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c160920 0x7f2b5c157f00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2b4402b950 tx=0x7f2b440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.602+0000 7f2b52000640 1 -- 172.21.15.26:0/931516367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4403ad30 con 0x7f2b5c160920 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.602+0000 7f2b52000640 1 -- 172.21.15.26:0/931516367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b440443f0 con 0x7f2b5c160920 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.602+0000 7f2b52000640 1 -- 172.21.15.26:0/931516367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b440447a0 con 0x7f2b5c160920 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.602+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c158440 con 0x7f2b5c160920 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.602+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c0b63d0 con 0x7f2b5c160920 2024-01-15T18:05:05.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.604+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c0b6eb0 con 0x7f2b5c160920 2024-01-15T18:05:05.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.605+0000 7f2b52000640 1 -- 172.21.15.26:0/931516367 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f2b4404d450 con 0x7f2b5c160920 2024-01-15T18:05:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.606+0000 7f2b52000640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bec0 0x7f2b3406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:05.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.606+0000 7f2b52000640 1 -- 172.21.15.26:0/931516367 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f2b4402d040 con 0x7f2b5c160920 2024-01-15T18:05:05.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.607+0000 7f2b59e00640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bec0 0x7f2b3406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:05.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.608+0000 7f2b59e00640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bec0 0x7f2b3406e380 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c006fd0 tx=0x7f2b4c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:05.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.612+0000 7f2b52000640 1 -- 172.21.15.26:0/931516367 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b5c0b6eb0 con 0x7f2b5c160920 2024-01-15T18:05:05.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.877+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2b5c0b6eb0 con 0x7f2b5c160920 2024-01-15T18:05:05.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.878+0000 7f2b52000640 1 -- 172.21.15.26:0/931516367 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2347 (secure 0 0 0) 0x7f2b5c0b6eb0 con 0x7f2b5c160920 2024-01-15T18:05:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:05.882 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:05.882 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 620, 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:05.883 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:05.884 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 925, 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3729988380, 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 7296094208, 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 765798019072, 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 49615324, 2024-01-15T18:05:05.885 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 42302637, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 12, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 17 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:05.886 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:05.887 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 0.0% (9s)\n [............................] ", 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0, 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 0.0% (9s)\n [............................] ", 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0, 2024-01-15T18:05:05.888 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:05.889 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:05.889 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:05.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bec0 msgr2=0x7f2b3406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bec0 0x7f2b3406e380 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c006fd0 tx=0x7f2b4c008040 comp rx=0 tx=0).stop 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c160920 msgr2=0x7f2b5c157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c160920 0x7f2b5c157f00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f2b4402b950 tx=0x7f2b440046c0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 shutdown_connections 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bec0 0x7f2b3406e380 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b5c161010 0x7f2b5c0cbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c160920 0x7f2b5c157f00 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 --2- 172.21.15.26:0/931516367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c0cf9f0 0x7f2b5c1f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 >> 172.21.15.26:0/931516367 conn(0x7f2b5c0d3090 msgr2=0x7f2b5c06af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.881+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 shutdown_connections 2024-01-15T18:05:05.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:05.882+0000 7f2b5b200640 1 -- 172.21.15.26:0/931516367 wait complete. 2024-01-15T18:05:06.905 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:07.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.127+0000 7fa622a00640 1 Processor -- start 2024-01-15T18:05:07.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.128+0000 7fa622a00640 1 -- start start 2024-01-15T18:05:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.129+0000 7fa622a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa61c146770 0x7fa61c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.129+0000 7fa622a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61c0b2e10 con 0x7fa61c071f00 2024-01-15T18:05:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.129+0000 7fa622a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61c146bc0 con 0x7fa61c071410 2024-01-15T18:05:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.129+0000 7fa622a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c06cec0 con 0x7fa61c146770 2024-01-15T18:05:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.130+0000 7fa620c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa61c071f00 0x7fa61c164ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41776/0 (socket says 172.21.15.26:41776) 2024-01-15T18:05:07.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.130+0000 7fa620c00640 1 -- 172.21.15.26:0/1205304133 learned_addr learned my addr 172.21.15.26:0/1205304133 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.130+0000 7fa622000640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa61c146770 0x7fa61c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.130+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13314895 0 0) 0x7fa61c0b2e10 con 0x7fa61c071f00 2024-01-15T18:05:07.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.130+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5fc003520 con 0x7fa61c071f00 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.130+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291716218 0 0) 0x7fa5fc003520 con 0x7fa61c071f00 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa61c146770 msgr2=0x7fa61c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa61c146770 0x7fa61c06c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 >> v1:172.21.15.124:6789/0 conn(0x7fa61c071410 legacy=0x7fa61c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c06d1a0 con 0x7fa61c071f00 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa604001670 con 0x7fa61c071f00 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 >> v1:172.21.15.26:6789/0 conn(0x7fa61c071f00 legacy=0x7fa61c164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5fc0039a0 0x7fa5fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5fc0047c0 0x7fa5fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa61c146770 0x7fa5fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c06cec0 con 0x7fa5fc0039a0 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc0054e0 con 0x7fa5fc0047c0 2024-01-15T18:05:07.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5fc00bec0 con 0x7fa61c146770 2024-01-15T18:05:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa621600640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5fc0039a0 0x7fa5fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa620c00640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5fc0047c0 0x7fa5fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa622000640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa61c146770 0x7fa5fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa621600640 1 -- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa61c146770 msgr2=0x7fa5fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa621600640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa61c146770 0x7fa5fc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa621600640 1 -- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5fc0047c0 msgr2=0x7fa5fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa621600640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5fc0047c0 0x7fa5fc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.131+0000 7fa621600640 1 -- 172.21.15.26:0/1205304133 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5fc003520 con 0x7fa5fc0039a0 2024-01-15T18:05:07.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.132+0000 7fa620c00640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa5fc0047c0 0x7fa5fc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.133+0000 7fa621600640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5fc0039a0 0x7fa5fc003dc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa60c001750 tx=0x7fa5fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519be089daf6e033 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.133+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0052d0 con 0x7fa5fc0039a0 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.133+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c02e050 con 0x7fa5fc0039a0 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.133+0000 7fa61be00640 1 -- 172.21.15.26:0/1205304133 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0035f0 con 0x7fa5fc0039a0 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.133+0000 7fa622a00640 1 -- 172.21.15.26:0/1205304133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5fc0039a0 msgr2=0x7fa5fc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.133+0000 7fa622a00640 1 --2- 172.21.15.26:0/1205304133 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa5fc0039a0 0x7fa5fc003dc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa60c001750 tx=0x7fa5fc0057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.133+0000 7fa622000640 1 -- 172.21.15.26:0/1205304133 reap_dead start 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.133+0000 7fa622a00640 1 -- 172.21.15.26:0/1205304133 shutdown_connections 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.133+0000 7fa622a00640 1 -- 172.21.15.26:0/1205304133 >> 172.21.15.26:0/1205304133 conn(0x7fa61c06a800 msgr2=0x7fa61c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.134+0000 7fa622a00640 1 -- 172.21.15.26:0/1205304133 shutdown_connections 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.134+0000 7fa622a00640 1 -- 172.21.15.26:0/1205304133 wait complete. 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.135+0000 7fa622a00640 1 Processor -- start 2024-01-15T18:05:07.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.135+0000 7fa622a00640 1 -- start start 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.136+0000 7fa622a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5fc0039a0 0x7fa61c1e6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.136+0000 7fa622a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa61c141a10 0x7fa61c1e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.136+0000 7fa622a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c1e78e0 0x7fa61c1e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.136+0000 7fa622a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0c6a00 con 0x7fa61c1e78e0 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.136+0000 7fa622a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c1e71f0 con 0x7fa61c141a10 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.136+0000 7fa622a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c1e74d0 con 0x7fa5fc0039a0 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa622000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c1e78e0 0x7fa61c1e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa622000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c1e78e0 0x7fa61c1e7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34450/0 (socket says 172.21.15.26:34450) 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa622000640 1 -- 172.21.15.26:0/2057337184 learned_addr learned my addr 172.21.15.26:0/2057337184 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa621600640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5fc0039a0 0x7fa61c1e6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa620c00640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa61c141a10 0x7fa61c1e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa622000640 1 -- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5fc0039a0 msgr2=0x7fa61c1e6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa622000640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5fc0039a0 0x7fa61c1e6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa622000640 1 -- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa61c141a10 msgr2=0x7fa61c1e69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa622000640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa61c141a10 0x7fa61c1e69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.137+0000 7fa622000640 1 -- 172.21.15.26:0/2057337184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0c6620 con 0x7fa61c1e78e0 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.138+0000 7fa622000640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c1e78e0 0x7fa61c1e7d90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa61c146ec0 tx=0x7fa614009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.138+0000 7fa619600640 1 -- 172.21.15.26:0/2057337184 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614007cf0 con 0x7fa61c1e78e0 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.138+0000 7fa619600640 1 -- 172.21.15.26:0/2057337184 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa614007e70 con 0x7fa61c1e78e0 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.138+0000 7fa619600640 1 -- 172.21.15.26:0/2057337184 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614014620 con 0x7fa61c1e78e0 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.138+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c1e6f00 con 0x7fa61c1e78e0 2024-01-15T18:05:07.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.138+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c1e84b0 con 0x7fa61c1e78e0 2024-01-15T18:05:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.140+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c1e8730 con 0x7fa61c1e78e0 2024-01-15T18:05:07.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.140+0000 7fa619600640 1 -- 172.21.15.26:0/2057337184 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fa61401d3b0 con 0x7fa61c1e78e0 2024-01-15T18:05:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.141+0000 7fa619600640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa5f406be90 0x7fa5f406e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.141+0000 7fa619600640 1 -- 172.21.15.26:0/2057337184 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fa6140881e0 con 0x7fa61c1e78e0 2024-01-15T18:05:07.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.142+0000 7fa621600640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa5f406be90 0x7fa5f406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:07.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.143+0000 7fa621600640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa5f406be90 0x7fa5f406e350 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa60c033000 tx=0x7fa60c009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:07.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.145+0000 7fa619600640 1 -- 172.21.15.26:0/2057337184 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa614054aa0 con 0x7fa61c1e78e0 2024-01-15T18:05:07.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.409+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa61c0c6460 con 0x7fa61c1e78e0 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.410+0000 7fa619600640 1 -- 172.21.15.26:0/2057337184 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2422 (secure 0 0 0) 0x7fa614019090 con 0x7fa61c1e78e0 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:07.414 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 621, 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:07.415 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:07.416 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 925, 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3729988380, 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 7296094208, 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 765798019072, 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 49614190, 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 42291751, 2024-01-15T18:05:07.417 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 12, 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 15 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:07.418 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:07.419 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.095228567719459534, 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:05:07.420 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.16664999723434448, 2024-01-15T18:05:07.421 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:07.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:07.421 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:07.421 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.412+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa5f406be90 msgr2=0x7fa5f406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.412+0000 7fa622a00640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa5f406be90 0x7fa5f406e350 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa60c033000 tx=0x7fa60c009030 comp rx=0 tx=0).stop 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.412+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c1e78e0 msgr2=0x7fa61c1e7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.412+0000 7fa622a00640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c1e78e0 0x7fa61c1e7d90 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa61c146ec0 tx=0x7fa614009d50 comp rx=0 tx=0).stop 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.413+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 shutdown_connections 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.413+0000 7fa622a00640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa5f406be90 0x7fa5f406e350 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.413+0000 7fa622a00640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa61c1e78e0 0x7fa61c1e7d90 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.413+0000 7fa622a00640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa61c141a10 0x7fa61c1e69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.413+0000 7fa622a00640 1 --2- 172.21.15.26:0/2057337184 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa5fc0039a0 0x7fa61c1e6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:07.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.413+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 >> 172.21.15.26:0/2057337184 conn(0x7fa61c06a800 msgr2=0x7fa61c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:07.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.413+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 shutdown_connections 2024-01-15T18:05:07.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:07.413+0000 7fa622a00640 1 -- 172.21.15.26:0/2057337184 wait complete. 2024-01-15T18:05:08.434 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:08.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.657+0000 7f7872000640 1 Processor -- start 2024-01-15T18:05:08.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.657+0000 7f7872000640 1 -- start start 2024-01-15T18:05:08.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.658+0000 7f7872000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f786c1607a0 0x7f786c0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.658+0000 7f7872000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f786c160b80 0x7f786c1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.658+0000 7f7872000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0b2b90 con 0x7f786c0b3eb0 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.658+0000 7f7872000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c0d21d0 con 0x7f786c160b80 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.658+0000 7f7872000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0b2e30 con 0x7f786c1607a0 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.659+0000 7f7871600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f786c160b80 0x7f786c1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.659+0000 7f7871600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f786c160b80 0x7f786c1617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34466/0 (socket says 172.21.15.26:34466) 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.659+0000 7f7871600640 1 -- 172.21.15.26:0/450774867 learned_addr learned my addr 172.21.15.26:0/450774867 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.659+0000 7f7871600640 1 -- 172.21.15.26:0/450774867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f786c1607a0 msgr2=0x7f786c0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.659+0000 7f7871600640 1 --2- 172.21.15.26:0/450774867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f786c1607a0 0x7f786c0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.659+0000 7f7871600640 1 -- 172.21.15.26:0/450774867 >> v1:172.21.15.124:6789/0 conn(0x7f786c0b3eb0 legacy=0x7f786c1610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:08.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.659+0000 7f7871600640 1 -- 172.21.15.26:0/450774867 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c0b3110 con 0x7f786c160b80 2024-01-15T18:05:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.660+0000 7f7871600640 1 --2- 172.21.15.26:0/450774867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 0x7f786c1617d0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f7864009e50 tx=0x7f786402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fa46a9ea821879 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f786b400640 1 -- 172.21.15.26:0/450774867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786402ca70 con 0x7f786c160b80 2024-01-15T18:05:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f786b400640 1 -- 172.21.15.26:0/450774867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786402cbd0 con 0x7f786c160b80 2024-01-15T18:05:08.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f786b400640 1 -- 172.21.15.26:0/450774867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78640343f0 con 0x7f786c160b80 2024-01-15T18:05:08.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f7872000640 1 -- 172.21.15.26:0/450774867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 msgr2=0x7f786c1617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f7872000640 1 --2- 172.21.15.26:0/450774867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 0x7f786c1617d0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f7864009e50 tx=0x7f786402bed0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f7872000640 1 -- 172.21.15.26:0/450774867 shutdown_connections 2024-01-15T18:05:08.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f7872000640 1 --2- 172.21.15.26:0/450774867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 0x7f786c1617d0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f7872000640 1 --2- 172.21.15.26:0/450774867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f786c1607a0 0x7f786c0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f7872000640 1 -- 172.21.15.26:0/450774867 >> 172.21.15.26:0/450774867 conn(0x7f786c09c590 msgr2=0x7f786c0d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f7872000640 1 -- 172.21.15.26:0/450774867 shutdown_connections 2024-01-15T18:05:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.661+0000 7f7872000640 1 -- 172.21.15.26:0/450774867 wait complete. 2024-01-15T18:05:08.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.663+0000 7f7872000640 1 Processor -- start 2024-01-15T18:05:08.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.663+0000 7f7872000640 1 -- start start 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.664+0000 7f7872000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f786c0b3eb0 0x7f786c158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.664+0000 7f7872000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f786c1607a0 0x7f786c0cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.664+0000 7f7872000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 0x7f786c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.664+0000 7f7872000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0b5e70 con 0x7f786c160b80 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.664+0000 7f7872000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0b63f0 con 0x7f786c1607a0 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.664+0000 7f7872000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0cbbf0 con 0x7f786c0b3eb0 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7871600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 0x7f786c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7871600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 0x7f786c0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34478/0 (socket says 172.21.15.26:34478) 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7871600640 1 -- 172.21.15.26:0/2262719273 learned_addr learned my addr 172.21.15.26:0/2262719273 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7870c00640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f786c0b3eb0 0x7f786c158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f786be00640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f786c1607a0 0x7f786c0cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7871600640 1 -- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f786c0b3eb0 msgr2=0x7f786c158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7871600640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f786c0b3eb0 0x7f786c158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7871600640 1 -- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f786c1607a0 msgr2=0x7f786c0cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7871600640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f786c1607a0 0x7f786c0cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7871600640 1 -- 172.21.15.26:0/2262719273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864009b30 con 0x7f786c160b80 2024-01-15T18:05:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.665+0000 7f7871600640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 0x7f786c0ccca0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f7864009e50 tx=0x7f7864004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.666+0000 7f7868c00640 1 -- 172.21.15.26:0/2262719273 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786403e450 con 0x7f786c160b80 2024-01-15T18:05:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.666+0000 7f7868c00640 1 -- 172.21.15.26:0/2262719273 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786403e5d0 con 0x7f786c160b80 2024-01-15T18:05:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.666+0000 7f7868c00640 1 -- 172.21.15.26:0/2262719273 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786403e980 con 0x7f786c160b80 2024-01-15T18:05:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.666+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c0cbed0 con 0x7f786c160b80 2024-01-15T18:05:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.666+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c1593a0 con 0x7f786c160b80 2024-01-15T18:05:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.667+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c0cc0f0 con 0x7f786c160b80 2024-01-15T18:05:08.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.668+0000 7f7868c00640 1 -- 172.21.15.26:0/2262719273 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f786403eae0 con 0x7f786c160b80 2024-01-15T18:05:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.670+0000 7f7868c00640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f784406bee0 0x7f784406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.670+0000 7f7868c00640 1 -- 172.21.15.26:0/2262719273 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f78640ab4d0 con 0x7f786c160b80 2024-01-15T18:05:08.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.670+0000 7f7870c00640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f784406bee0 0x7f784406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:08.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.671+0000 7f7870c00640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f784406bee0 0x7f784406e3a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b1c70 tx=0x7f785c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:08.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.674+0000 7f7868c00640 1 -- 172.21.15.26:0/2262719273 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7864077d90 con 0x7f786c160b80 2024-01-15T18:05:08.941 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.937+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f786c161cc0 con 0x7f786c160b80 2024-01-15T18:05:08.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.938+0000 7f7868c00640 1 -- 172.21.15.26:0/2262719273 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7f7864043090 con 0x7f786c160b80 2024-01-15T18:05:08.942 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:08.942 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:08.942 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:08.942 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:08.942 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:08.942 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:08.943 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 623, 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:08.944 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:08.945 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1315, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 5369988979, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 11176099840, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 761918013440, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 178891253, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 178908569, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 43, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 48 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:08.946 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:08.947 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:08.948 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.095228567719459534, 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.16664999723434448, 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:08.949 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f784406bee0 msgr2=0x7f784406e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f784406bee0 0x7f784406e3a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f786c0b1c70 tx=0x7f785c008040 comp rx=0 tx=0).stop 2024-01-15T18:05:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 msgr2=0x7f786c0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:08.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 0x7f786c0ccca0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f7864009e50 tx=0x7f7864004a40 comp rx=0 tx=0).stop 2024-01-15T18:05:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 shutdown_connections 2024-01-15T18:05:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f784406bee0 0x7f784406e3a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f786c160b80 0x7f786c0ccca0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f786c1607a0 0x7f786c0cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 --2- 172.21.15.26:0/2262719273 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f786c0b3eb0 0x7f786c158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 >> 172.21.15.26:0/2262719273 conn(0x7f786c09c590 msgr2=0x7f786c0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 shutdown_connections 2024-01-15T18:05:08.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:08.941+0000 7f7872000640 1 -- 172.21.15.26:0/2262719273 wait complete. 2024-01-15T18:05:09.966 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.190+0000 7f9b57200640 1 Processor -- start 2024-01-15T18:05:10.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.191+0000 7f9b57200640 1 -- start start 2024-01-15T18:05:10.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.192+0000 7f9b57200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9b580d0100 0x7f9b580b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.192+0000 7f9b57200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b580d0a10 con 0x7f9b580d0f40 2024-01-15T18:05:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.192+0000 7f9b57200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b580cf080 con 0x7f9b58160920 2024-01-15T18:05:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.192+0000 7f9b57200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b580c6be0 con 0x7f9b580d0100 2024-01-15T18:05:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.193+0000 7f9b56800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9b580d0f40 0x7f9b580d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41778/0 (socket says 172.21.15.26:41778) 2024-01-15T18:05:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.193+0000 7f9b56800640 1 -- 172.21.15.26:0/1840842371 learned_addr learned my addr 172.21.15.26:0/1840842371 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.193+0000 7f9b55400640 1 --2- 172.21.15.26:0/1840842371 >> v2:172.21.15.124:3300/0 conn(0x7f9b580d0100 0x7f9b580b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.193+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625605866 0 0) 0x7f9b580d0a10 con 0x7f9b580d0f40 2024-01-15T18:05:10.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.193+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b38003520 con 0x7f9b580d0f40 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 87298022 0 0) 0x7f9b38003520 con 0x7f9b580d0f40 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 >> v2:172.21.15.124:3300/0 conn(0x7f9b580d0100 msgr2=0x7f9b580b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 --2- 172.21.15.26:0/1840842371 >> v2:172.21.15.124:3300/0 conn(0x7f9b580d0100 0x7f9b580b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 >> v1:172.21.15.124:6789/0 conn(0x7f9b58160920 legacy=0x7f9b580cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b580ce7e0 con 0x7f9b580d0f40 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9b50001670 con 0x7f9b580d0f40 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 >> v1:172.21.15.26:6789/0 conn(0x7f9b580d0f40 legacy=0x7f9b580d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b38003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b380047c0 0x7f9b38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b38005180 0x7f9b3800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580c6be0 con 0x7f9b380039c0 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3800c080 con 0x7f9b380047c0 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3800c240 con 0x7f9b38005180 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b55e00640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b38003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.194+0000 7f9b55400640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b380047c0 0x7f9b38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.195+0000 7f9b55e00640 1 -- 172.21.15.26:0/1840842371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b38005180 msgr2=0x7f9b3800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.195+0000 7f9b55e00640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b38005180 0x7f9b3800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.195+0000 7f9b55e00640 1 -- 172.21.15.26:0/1840842371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b380047c0 msgr2=0x7f9b38004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.195+0000 7f9b55e00640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b380047c0 0x7f9b38004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.195+0000 7f9b55e00640 1 -- 172.21.15.26:0/1840842371 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b38003520 con 0x7f9b380039c0 2024-01-15T18:05:10.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.195+0000 7f9b55400640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b380047c0 0x7f9b38004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.195+0000 7f9b56800640 1 -- 172.21.15.26:0/1840842371 reap_dead start 2024-01-15T18:05:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.196+0000 7f9b55e00640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b38003de0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9b48001750 tx=0x7f9b380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca6a62fcf375b63 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.196+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480052d0 con 0x7f9b380039c0 2024-01-15T18:05:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.196+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4802e050 con 0x7f9b380039c0 2024-01-15T18:05:10.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.196+0000 7f9b54a00640 1 -- 172.21.15.26:0/1840842371 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48003830 con 0x7f9b380039c0 2024-01-15T18:05:10.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.197+0000 7f9b57200640 1 -- 172.21.15.26:0/1840842371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 msgr2=0x7f9b38003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.197+0000 7f9b57200640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b38003de0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f9b48001750 tx=0x7f9b380057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.197+0000 7f9b57200640 1 -- 172.21.15.26:0/1840842371 shutdown_connections 2024-01-15T18:05:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.197+0000 7f9b57200640 1 --2- 172.21.15.26:0/1840842371 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b38003de0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.197+0000 7f9b57200640 1 -- 172.21.15.26:0/1840842371 >> 172.21.15.26:0/1840842371 conn(0x7f9b580d3090 msgr2=0x7f9b580d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.197+0000 7f9b57200640 1 -- 172.21.15.26:0/1840842371 shutdown_connections 2024-01-15T18:05:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.197+0000 7f9b57200640 1 -- 172.21.15.26:0/1840842371 wait complete. 2024-01-15T18:05:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.199+0000 7f9b57200640 1 Processor -- start 2024-01-15T18:05:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.199+0000 7f9b57200640 1 -- start start 2024-01-15T18:05:10.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b57200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b580ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b57200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b581546f0 0x7f9b58153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b57200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b580c8bc0 0x7f9b580c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b57200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580c9580 con 0x7f9b380039c0 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b57200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580ca990 con 0x7f9b580c8bc0 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b57200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580cac70 con 0x7f9b581546f0 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b55e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b580ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b55e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b580ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34496/0 (socket says 172.21.15.26:34496) 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b55e00640 1 -- 172.21.15.26:0/670732317 learned_addr learned my addr 172.21.15.26:0/670732317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b56800640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b580c8bc0 0x7f9b580c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b55400640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b581546f0 0x7f9b58153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b55e00640 1 -- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b581546f0 msgr2=0x7f9b58153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b55e00640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b581546f0 0x7f9b58153ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b55e00640 1 -- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b580c8bc0 msgr2=0x7f9b580c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b55e00640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b580c8bc0 0x7f9b580c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b55e00640 1 -- 172.21.15.26:0/670732317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b580caf50 con 0x7f9b380039c0 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.200+0000 7f9b56800640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b580c8bc0 0x7f9b580c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.201+0000 7f9b55e00640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b580ca160 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9b48000c00 tx=0x7f9b48001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.201+0000 7f9b4e000640 1 -- 172.21.15.26:0/670732317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b480076c0 con 0x7f9b380039c0 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.201+0000 7f9b4e000640 1 -- 172.21.15.26:0/670732317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b48007a30 con 0x7f9b380039c0 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.201+0000 7f9b4e000640 1 -- 172.21.15.26:0/670732317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b48006be0 con 0x7f9b380039c0 2024-01-15T18:05:10.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.201+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b580cb230 con 0x7f9b380039c0 2024-01-15T18:05:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.201+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b58154410 con 0x7f9b380039c0 2024-01-15T18:05:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.202+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b58067450 con 0x7f9b380039c0 2024-01-15T18:05:10.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.203+0000 7f9b4e000640 1 -- 172.21.15.26:0/670732317 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f9b48006d40 con 0x7f9b380039c0 2024-01-15T18:05:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.204+0000 7f9b4e000640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b3006bec0 0x7f9b3006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.204+0000 7f9b4e000640 1 -- 172.21.15.26:0/670732317 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f9b480aa3c0 con 0x7f9b380039c0 2024-01-15T18:05:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.205+0000 7f9b55400640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b3006bec0 0x7f9b3006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:10.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.206+0000 7f9b55400640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b3006bec0 0x7f9b3006e380 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9b5815afc0 tx=0x7f9b4000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:10.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.208+0000 7f9b4e000640 1 -- 172.21.15.26:0/670732317 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b48076a20 con 0x7f9b380039c0 2024-01-15T18:05:10.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.478+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9b580c6710 con 0x7f9b380039c0 2024-01-15T18:05:10.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.479+0000 7f9b4e000640 1 -- 172.21.15.26:0/670732317 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7f9b48041020 con 0x7f9b380039c0 2024-01-15T18:05:10.482 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:10.483 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:10.484 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:10.484 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:10.484 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:10.484 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:10.484 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:10.484 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:10.485 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:10.485 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:10.485 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 624, 2024-01-15T18:05:10.485 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:10.485 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:10.485 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:10.485 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:10.485 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:10.486 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1315, 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 5369988979, 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 11176099840, 2024-01-15T18:05:10.487 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 761918013440, 2024-01-15T18:05:10.488 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:10.488 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 178891253, 2024-01-15T18:05:10.488 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 178908569, 2024-01-15T18:05:10.488 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 43, 2024-01-15T18:05:10.488 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 48 2024-01-15T18:05:10.488 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:10.488 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:10.488 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:10.488 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:10.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:10.490 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:05:10.491 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.095228567719459534, 2024-01-15T18:05:10.492 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:10.492 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:10.492 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:10.492 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:05:10.492 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.16664999723434448, 2024-01-15T18:05:10.492 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:10.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:10.492 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:10.493 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:10.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.481+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b3006bec0 msgr2=0x7f9b3006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.481+0000 7f9b57200640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b3006bec0 0x7f9b3006e380 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9b5815afc0 tx=0x7f9b4000b040 comp rx=0 tx=0).stop 2024-01-15T18:05:10.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.481+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 msgr2=0x7f9b580ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:10.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.481+0000 7f9b57200640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b580ca160 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9b48000c00 tx=0x7f9b48001120 comp rx=0 tx=0).stop 2024-01-15T18:05:10.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.481+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 shutdown_connections 2024-01-15T18:05:10.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.481+0000 7f9b57200640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b581546f0 0x7f9b58153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.481+0000 7f9b57200640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b3006bec0 0x7f9b3006e380 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.481+0000 7f9b57200640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b580c8bc0 0x7f9b580c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.481+0000 7f9b57200640 1 --2- 172.21.15.26:0/670732317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b380039c0 0x7f9b580ca160 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:10.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.482+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 >> 172.21.15.26:0/670732317 conn(0x7f9b580d3090 msgr2=0x7f9b5815b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:10.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.482+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 shutdown_connections 2024-01-15T18:05:10.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:10.482+0000 7f9b57200640 1 -- 172.21.15.26:0/670732317 wait complete. 2024-01-15T18:05:11.507 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:11.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.786+0000 7fa34ca00640 1 Processor -- start 2024-01-15T18:05:11.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.786+0000 7fa34ca00640 1 -- start start 2024-01-15T18:05:11.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.788+0000 7fa34ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa344150760 0x7fa344072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.788+0000 7fa34ca00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa34415d300 0x7fa34415d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.788+0000 7fa34ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3440b2bd0 0x7fa3440b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.788+0000 7fa34ca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa34415dc80 con 0x7fa3440b2bd0 2024-01-15T18:05:11.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.788+0000 7fa34ca00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa344071190 con 0x7fa34415d300 2024-01-15T18:05:11.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.788+0000 7fa34ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa344071470 con 0x7fa344150760 2024-01-15T18:05:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.789+0000 7fa34aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3440b2bd0 0x7fa3440b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.789+0000 7fa34aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa3440b2bd0 0x7fa3440b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34502/0 (socket says 172.21.15.26:34502) 2024-01-15T18:05:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.789+0000 7fa34aa00640 1 -- 172.21.15.26:0/1484971313 learned_addr learned my addr 172.21.15.26:0/1484971313 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:11.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.789+0000 7fa34a000640 1 --2- 172.21.15.26:0/1484971313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa344150760 0x7fa344072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.789+0000 7fa34aa00640 1 -- 172.21.15.26:0/1484971313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa344150760 msgr2=0x7fa344072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.789+0000 7fa34aa00640 1 --2- 172.21.15.26:0/1484971313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa344150760 0x7fa344072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.789+0000 7fa34aa00640 1 -- 172.21.15.26:0/1484971313 >> v2:172.21.15.124:3300/0 conn(0x7fa34415d300 msgr2=0x7fa34415d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.789+0000 7fa34aa00640 1 --2- 172.21.15.26:0/1484971313 >> v2:172.21.15.124:3300/0 conn(0x7fa34415d300 0x7fa34415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.789+0000 7fa34aa00640 1 -- 172.21.15.26:0/1484971313 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa344071750 con 0x7fa3440b2bd0 2024-01-15T18:05:11.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.791+0000 7fa34aa00640 1 --2- 172.21.15.26:0/1484971313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 0x7fa3440b3060 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa34406e0c0 tx=0x7fa33c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae39acd3e01f312f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.791+0000 7fa348c00640 1 -- 172.21.15.26:0/1484971313 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c02ca70 con 0x7fa3440b2bd0 2024-01-15T18:05:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.792+0000 7fa348c00640 1 -- 172.21.15.26:0/1484971313 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33c02cbd0 con 0x7fa3440b2bd0 2024-01-15T18:05:11.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.792+0000 7fa348c00640 1 -- 172.21.15.26:0/1484971313 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0343f0 con 0x7fa3440b2bd0 2024-01-15T18:05:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.793+0000 7fa34ca00640 1 -- 172.21.15.26:0/1484971313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 msgr2=0x7fa3440b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.793+0000 7fa34ca00640 1 --2- 172.21.15.26:0/1484971313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 0x7fa3440b3060 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa34406e0c0 tx=0x7fa33c02bed0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.793+0000 7fa34ca00640 1 -- 172.21.15.26:0/1484971313 shutdown_connections 2024-01-15T18:05:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.793+0000 7fa34ca00640 1 --2- 172.21.15.26:0/1484971313 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 0x7fa3440b3060 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.793+0000 7fa34ca00640 1 --2- 172.21.15.26:0/1484971313 >> v2:172.21.15.124:3300/0 conn(0x7fa34415d300 0x7fa34415d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.793+0000 7fa34ca00640 1 --2- 172.21.15.26:0/1484971313 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa344150760 0x7fa344072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.793+0000 7fa34ca00640 1 -- 172.21.15.26:0/1484971313 >> 172.21.15.26:0/1484971313 conn(0x7fa34406a800 msgr2=0x7fa34406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.793+0000 7fa34ca00640 1 -- 172.21.15.26:0/1484971313 shutdown_connections 2024-01-15T18:05:11.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.793+0000 7fa34ca00640 1 -- 172.21.15.26:0/1484971313 wait complete. 2024-01-15T18:05:11.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.796+0000 7fa34ca00640 1 Processor -- start 2024-01-15T18:05:11.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.796+0000 7fa34ca00640 1 -- start start 2024-01-15T18:05:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.797+0000 7fa34ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 0x7fa344146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.797+0000 7fa34ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa344150760 0x7fa3440cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.797+0000 7fa34ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa34415d300 0x7fa3440b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.797+0000 7fa34ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3441467e0 con 0x7fa3440b2bd0 2024-01-15T18:05:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.797+0000 7fa34ca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa344158b70 con 0x7fa34415d300 2024-01-15T18:05:11.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.797+0000 7fa34ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa344158e50 con 0x7fa344150760 2024-01-15T18:05:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.798+0000 7fa34a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 0x7fa344146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.798+0000 7fa34a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 0x7fa344146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34510/0 (socket says 172.21.15.26:34510) 2024-01-15T18:05:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.798+0000 7fa34a000640 1 -- 172.21.15.26:0/258791705 learned_addr learned my addr 172.21.15.26:0/258791705 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.798+0000 7fa34aa00640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa34415d300 0x7fa3440b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.799+0000 7fa34a000640 1 -- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa344150760 msgr2=0x7fa3440cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.799+0000 7fa34a000640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa344150760 0x7fa3440cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.799+0000 7fa34a000640 1 -- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa34415d300 msgr2=0x7fa3440b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.799+0000 7fa34a000640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa34415d300 0x7fa3440b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.799+0000 7fa34a000640 1 -- 172.21.15.26:0/258791705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c009b30 con 0x7fa3440b2bd0 2024-01-15T18:05:11.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.799+0000 7fa34a000640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 0x7fa344146100 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa33400d800 tx=0x7fa33400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.799+0000 7fa342000640 1 -- 172.21.15.26:0/258791705 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400b6c0 con 0x7fa3440b2bd0 2024-01-15T18:05:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.799+0000 7fa342000640 1 -- 172.21.15.26:0/258791705 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33400b820 con 0x7fa3440b2bd0 2024-01-15T18:05:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.799+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa344159130 con 0x7fa3440b2bd0 2024-01-15T18:05:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.800+0000 7fa342000640 1 -- 172.21.15.26:0/258791705 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400bbb0 con 0x7fa3440b2bd0 2024-01-15T18:05:11.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.800+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa344143a50 con 0x7fa3440b2bd0 2024-01-15T18:05:11.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.802+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3440675d0 con 0x7fa3440b2bd0 2024-01-15T18:05:11.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.803+0000 7fa342000640 1 -- 172.21.15.26:0/258791705 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fa33400bd10 con 0x7fa3440b2bd0 2024-01-15T18:05:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.806+0000 7fa342000640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa31c06be90 0x7fa31c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:11.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.806+0000 7fa342000640 1 -- 172.21.15.26:0/258791705 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fa334088d70 con 0x7fa3440b2bd0 2024-01-15T18:05:11.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.811+0000 7fa342000640 1 -- 172.21.15.26:0/258791705 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3340553d0 con 0x7fa3440b2bd0 2024-01-15T18:05:11.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.813+0000 7fa349600640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa31c06be90 0x7fa31c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:11.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:11.817+0000 7fa349600640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa31c06be90 0x7fa31c06e350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa34415c610 tx=0x7fa32c006bf0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:12.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.280+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa34415aa30 con 0x7fa3440b2bd0 2024-01-15T18:05:12.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.280+0000 7fa342000640 1 -- 172.21.15.26:0/258791705 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7fa3340589f0 con 0x7fa3440b2bd0 2024-01-15T18:05:12.285 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:12.285 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:12.285 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:12.285 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:12.285 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:12.285 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:12.285 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:12.285 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:12.285 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:12.286 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:12.286 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:12.286 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:12.286 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:12.286 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:12.286 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:12.286 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:12.286 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:12.286 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 626, 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:12.287 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:12.288 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:12.288 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:12.288 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:12.288 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:12.288 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:12.288 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:12.288 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:12.289 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:12.289 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:12.289 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:12.289 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:12.289 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:12.289 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:12.289 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:12.289 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1348, 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 5508401011, 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 11356585984, 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 761737527296, 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 190770682, 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 190430669, 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 46, 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 49 2024-01-15T18:05:12.290 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:12.291 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:12.292 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:12.292 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:12.292 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:12.292 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:12.292 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:12.292 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:12.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:12.292 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:12.292 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:12.293 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.095228567719459534, 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:12.294 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:12.295 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:05:12.295 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.16664999723434448, 2024-01-15T18:05:12.295 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:12.295 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:12.295 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:12.295 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:12.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa31c06be90 msgr2=0x7fa31c06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa31c06be90 0x7fa31c06e350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa34415c610 tx=0x7fa32c006bf0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 msgr2=0x7fa344146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:12.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 0x7fa344146100 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa33400d800 tx=0x7fa33400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 shutdown_connections 2024-01-15T18:05:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa344150760 0x7fa3440cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa31c06be90 0x7fa31c06e350 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa34415d300 0x7fa3440b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 --2- 172.21.15.26:0/258791705 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa3440b2bd0 0x7fa344146100 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 >> 172.21.15.26:0/258791705 conn(0x7fa34406a800 msgr2=0x7fa34415b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 shutdown_connections 2024-01-15T18:05:12.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:12.284+0000 7fa34ca00640 1 -- 172.21.15.26:0/258791705 wait complete. 2024-01-15T18:05:13.323 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:13.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.621+0000 7fa7e0c00640 1 Processor -- start 2024-01-15T18:05:13.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.621+0000 7fa7e0c00640 1 -- start start 2024-01-15T18:05:13.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.623+0000 7fa7e0c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc15e3b0 0x7fa7dc164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.623+0000 7fa7e0c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0b1ee0 con 0x7fa7dc143ae0 2024-01-15T18:05:13.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.623+0000 7fa7e0c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0b2780 con 0x7fa7dc0cf9a0 2024-01-15T18:05:13.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.623+0000 7fa7e0c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc15e990 con 0x7fa7dc15e3b0 2024-01-15T18:05:13.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.623+0000 7fa7d9e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa7dc143ae0 0x7fa7dc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50714/0 (socket says 172.21.15.26:50714) 2024-01-15T18:05:13.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.623+0000 7fa7d9e00640 1 -- 172.21.15.26:0/1828607879 learned_addr learned my addr 172.21.15.26:0/1828607879 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:13.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.624+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654627451 0 0) 0x7fa7dc0b1ee0 con 0x7fa7dc143ae0 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.624+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7bc005730 con 0x7fa7dc143ae0 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.624+0000 7fa7da800640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc15e3b0 0x7fa7dc164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.624+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2726432904 0 0) 0x7fa7bc005730 con 0x7fa7dc143ae0 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.624+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc15e3b0 msgr2=0x7fa7dc164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.624+0000 7fa7d8a00640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc15e3b0 0x7fa7dc164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.624+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 >> v1:172.21.15.124:6789/0 conn(0x7fa7dc0cf9a0 legacy=0x7fa7dc15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.624+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc0b2130 con 0x7fa7dc143ae0 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.624+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa7cc001670 con 0x7fa7dc143ae0 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.625+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 >> v1:172.21.15.26:6789/0 conn(0x7fa7dc143ae0 legacy=0x7fa7dc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.625+0000 7fa7d8a00640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc15e3b0 0x7fa7bc005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.625+0000 7fa7d8a00640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7bc0060f0 0x7fa7bc0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.625+0000 7fa7d8a00640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7bc007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.625+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc15e990 con 0x7fa7bc006f10 2024-01-15T18:05:13.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.625+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc00ff90 con 0x7fa7bc0060f0 2024-01-15T18:05:13.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.625+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc010120 con 0x7fa7dc15e3b0 2024-01-15T18:05:13.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.625+0000 7fa7da800640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7bc007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.626+0000 7fa7d9e00640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc15e3b0 0x7fa7bc005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.626+0000 7fa7da800640 1 -- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc15e3b0 msgr2=0x7fa7bc005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.626+0000 7fa7da800640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc15e3b0 0x7fa7bc005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.626+0000 7fa7da800640 1 -- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7bc0060f0 msgr2=0x7fa7bc0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:13.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.626+0000 7fa7da800640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7bc0060f0 0x7fa7bc0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.626+0000 7fa7da800640 1 -- 172.21.15.26:0/1828607879 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc005730 con 0x7fa7bc006f10 2024-01-15T18:05:13.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.628+0000 7fa7da800640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7bc007390 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4007390 tx=0x7fa7bc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625907fffcd8cf5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.628+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d402ccb0 con 0x7fa7bc006f10 2024-01-15T18:05:13.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.628+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d402ce10 con 0x7fa7bc006f10 2024-01-15T18:05:13.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.628+0000 7fa7d8a00640 1 -- 172.21.15.26:0/1828607879 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4034660 con 0x7fa7bc006f10 2024-01-15T18:05:13.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.630+0000 7fa7e0c00640 1 -- 172.21.15.26:0/1828607879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 msgr2=0x7fa7bc007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.630+0000 7fa7e0c00640 1 --2- 172.21.15.26:0/1828607879 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7bc007390 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4007390 tx=0x7fa7bc007a00 comp rx=0 tx=0).stop 2024-01-15T18:05:13.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.630+0000 7fa7da800640 1 -- 172.21.15.26:0/1828607879 reap_dead start 2024-01-15T18:05:13.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.630+0000 7fa7e0c00640 1 -- 172.21.15.26:0/1828607879 shutdown_connections 2024-01-15T18:05:13.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.630+0000 7fa7e0c00640 1 -- 172.21.15.26:0/1828607879 >> 172.21.15.26:0/1828607879 conn(0x7fa7dc06a800 msgr2=0x7fa7dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:13.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.631+0000 7fa7e0c00640 1 -- 172.21.15.26:0/1828607879 shutdown_connections 2024-01-15T18:05:13.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.631+0000 7fa7e0c00640 1 -- 172.21.15.26:0/1828607879 wait complete. 2024-01-15T18:05:13.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.634+0000 7fa7e0c00640 1 Processor -- start 2024-01-15T18:05:13.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.634+0000 7fa7e0c00640 1 -- start start 2024-01-15T18:05:13.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.636+0000 7fa7e0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7dc0ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.636+0000 7fa7e0c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7dc0b0bc0 0x7fa7dc0b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.636+0000 7fa7e0c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc158d30 0x7fa7dc1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.636+0000 7fa7e0c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0b1580 con 0x7fa7bc006f10 2024-01-15T18:05:13.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.636+0000 7fa7e0c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0cd180 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.636+0000 7fa7e0c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc159ab0 con 0x7fa7dc158d30 2024-01-15T18:05:13.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.637+0000 7fa7d9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7dc0ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.637+0000 7fa7d9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7dc0ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55972/0 (socket says 172.21.15.26:55972) 2024-01-15T18:05:13.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.637+0000 7fa7d9e00640 1 -- 172.21.15.26:0/3320471188 learned_addr learned my addr 172.21.15.26:0/3320471188 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:13.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.637+0000 7fa7d9400640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7dc0b0bc0 0x7fa7dc0b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.638+0000 7fa7d9400640 1 -- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc158d30 msgr2=0x7fa7dc1591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.638+0000 7fa7d9400640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc158d30 0x7fa7dc1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.638+0000 7fa7d9400640 1 -- 172.21.15.26:0/3320471188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 msgr2=0x7fa7dc0ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.638+0000 7fa7d9400640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7dc0ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.638+0000 7fa7d9400640 1 -- 172.21.15.26:0/3320471188 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc159d90 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.638+0000 7fa7d9e00640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7dc0ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.639+0000 7fa7d9400640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7dc0b0bc0 0x7fa7dc0b1040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c410 tx=0x7fa7c400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:13.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.639+0000 7fa7d2000640 1 -- 172.21.15.26:0/3320471188 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4011790 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.639+0000 7fa7d2000640 1 -- 172.21.15.26:0/3320471188 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c4011910 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.639+0000 7fa7d2000640 1 -- 172.21.15.26:0/3320471188 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4011cc0 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.639+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc0ccb30 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.641+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc0ae6e0 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.644+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7dc06c3f0 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.646+0000 7fa7d2000640 1 -- 172.21.15.26:0/3320471188 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fa7c401c460 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.648+0000 7fa7d2000640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7b406bec0 0x7fa7b406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:13.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.648+0000 7fa7d2000640 1 -- 172.21.15.26:0/3320471188 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fa7c40885d0 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.663+0000 7fa7d9e00640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7b406bec0 0x7fa7b406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:13.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.665+0000 7fa7d2000640 1 -- 172.21.15.26:0/3320471188 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7c4054d50 con 0x7fa7dc0b0bc0 2024-01-15T18:05:13.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:13.670+0000 7fa7d9e00640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7b406bec0 0x7fa7b406e380 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc001050 tx=0x7fa7cc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:14.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.035+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa7dc0675d0 con 0x7fa7dc0b0bc0 2024-01-15T18:05:14.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.036+0000 7fa7d2000640 1 -- 172.21.15.26:0/3320471188 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7fa7c4058370 con 0x7fa7dc0b0bc0 2024-01-15T18:05:14.040 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:14.040 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:14.040 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:14.041 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:14.041 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:14.041 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:14.041 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:14.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:14.041 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:14.041 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:14.041 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:14.042 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:14.042 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:14.042 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:14.042 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:14.042 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:14.042 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:14.042 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:14.043 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:14.043 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 628, 2024-01-15T18:05:14.043 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:14.043 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:14.043 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:14.043 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:14.043 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:14.043 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:14.044 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:14.044 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:14.044 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:14.044 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:14.044 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:14.044 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:14.044 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:14.044 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:14.045 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:14.045 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:14.045 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:14.045 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:14.045 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:14.045 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:14.045 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:14.046 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:14.046 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:14.046 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1348, 2024-01-15T18:05:14.046 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 5508401011, 2024-01-15T18:05:14.046 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 11356585984, 2024-01-15T18:05:14.046 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 761737527296, 2024-01-15T18:05:14.046 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:14.046 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 190770682, 2024-01-15T18:05:14.047 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 190430669, 2024-01-15T18:05:14.047 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 46, 2024-01-15T18:05:14.047 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 49 2024-01-15T18:05:14.047 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:14.047 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:14.047 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:14.047 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:14.047 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:14.048 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:14.048 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:14.048 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:14.048 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:14.048 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:14.048 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:14.048 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:14.048 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:14.049 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:14.049 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:14.049 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:14.049 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:14.049 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:14.049 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:14.049 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:14.049 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:14.050 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:14.050 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:14.050 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:14.050 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:14.050 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:14.050 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:14.050 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:14.050 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:14.051 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:14.051 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:14.051 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:14.051 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:14.051 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:14.051 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:14.051 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 9.523% (13s)\n [==..........................] (remaining: 2m)", 2024-01-15T18:05:14.051 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.095228567719459534, 2024-01-15T18:05:14.051 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:14.052 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:14.052 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:14.052 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 16.665% (13s)\n [====........................] (remaining: 65s)", 2024-01-15T18:05:14.052 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.16664999723434448, 2024-01-15T18:05:14.052 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:14.052 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:14.052 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:14.052 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:14.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.038+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7b406bec0 msgr2=0x7fa7b406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:14.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.038+0000 7fa7e0c00640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7b406bec0 0x7fa7b406e380 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc001050 tx=0x7fa7cc009040 comp rx=0 tx=0).stop 2024-01-15T18:05:14.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.038+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7dc0b0bc0 msgr2=0x7fa7dc0b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:14.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.038+0000 7fa7e0c00640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7dc0b0bc0 0x7fa7dc0b1040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400c410 tx=0x7fa7c400c8d0 comp rx=0 tx=0).stop 2024-01-15T18:05:14.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.039+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 shutdown_connections 2024-01-15T18:05:14.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.039+0000 7fa7e0c00640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7dc158d30 0x7fa7dc1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:14.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.039+0000 7fa7e0c00640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7b406bec0 0x7fa7b406e380 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:14.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.039+0000 7fa7e0c00640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7dc0b0bc0 0x7fa7dc0b1040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:14.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.039+0000 7fa7e0c00640 1 --2- 172.21.15.26:0/3320471188 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7bc006f10 0x7fa7dc0ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:14.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.039+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 >> 172.21.15.26:0/3320471188 conn(0x7fa7dc06a800 msgr2=0x7fa7dc070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:14.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.039+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 shutdown_connections 2024-01-15T18:05:14.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:14.039+0000 7fa7e0c00640 1 -- 172.21.15.26:0/3320471188 wait complete. 2024-01-15T18:05:15.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.351+0000 7f6bb0c00640 1 Processor -- start 2024-01-15T18:05:15.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.351+0000 7f6bb0c00640 1 -- start start 2024-01-15T18:05:15.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.353+0000 7f6bb0c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6bac160f80 0x7f6bac161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.353+0000 7f6bb0c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac0ced50 0x7f6bac0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.353+0000 7f6bb0c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac1618a0 con 0x7f6bac0ce8f0 2024-01-15T18:05:15.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.353+0000 7f6bb0c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac0cf1e0 con 0x7f6bac160f80 2024-01-15T18:05:15.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.353+0000 7f6bb0c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0d0de0 con 0x7f6bac0ced50 2024-01-15T18:05:15.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.353+0000 7f6ba9400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6bac0ce8f0 0x7f6bac0cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50716/0 (socket says 172.21.15.26:50716) 2024-01-15T18:05:15.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.353+0000 7f6ba9400640 1 -- 172.21.15.26:0/493273377 learned_addr learned my addr 172.21.15.26:0/493273377 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.354+0000 7f6ba9e00640 1 --2- 172.21.15.26:0/493273377 >> v2:172.21.15.124:3300/0 conn(0x7f6bac160f80 0x7f6bac161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.354+0000 7f6baa800640 1 --2- 172.21.15.26:0/493273377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac0ced50 0x7f6bac0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.354+0000 7f6ba8a00640 1 -- 172.21.15.26:0/493273377 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979858350 0 0) 0x7f6bac1618a0 con 0x7f6bac0ce8f0 2024-01-15T18:05:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.354+0000 7f6ba8a00640 1 -- 172.21.15.26:0/493273377 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b8c003520 con 0x7f6bac0ce8f0 2024-01-15T18:05:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.354+0000 7f6ba8a00640 1 -- 172.21.15.26:0/493273377 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3881252973 0 0) 0x7f6b8c003520 con 0x7f6bac0ce8f0 2024-01-15T18:05:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.354+0000 7f6ba9e00640 1 -- 172.21.15.26:0/493273377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac0ced50 msgr2=0x7f6bac0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.354+0000 7f6ba9e00640 1 --2- 172.21.15.26:0/493273377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac0ced50 0x7f6bac0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.354+0000 7f6ba9e00640 1 -- 172.21.15.26:0/493273377 >> v1:172.21.15.26:6789/0 conn(0x7f6bac0ce8f0 legacy=0x7f6bac0cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.354+0000 7f6ba9e00640 1 -- 172.21.15.26:0/493273377 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0d10c0 con 0x7f6bac160f80 2024-01-15T18:05:15.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.355+0000 7f6baa800640 1 --2- 172.21.15.26:0/493273377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac0ced50 0x7f6bac0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:15.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.356+0000 7f6ba9e00640 1 --2- 172.21.15.26:0/493273377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac160f80 0x7f6bac161360 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6b940097a0 tx=0x7f6b9402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5216917b46f99b00 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:15.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.356+0000 7f6ba8a00640 1 -- 172.21.15.26:0/493273377 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c001670 con 0x7f6bac160f80 2024-01-15T18:05:15.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.356+0000 7f6ba8a00640 1 -- 172.21.15.26:0/493273377 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9402e050 con 0x7f6bac160f80 2024-01-15T18:05:15.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.356+0000 7f6ba8a00640 1 -- 172.21.15.26:0/493273377 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94030710 con 0x7f6bac160f80 2024-01-15T18:05:15.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.357+0000 7f6bb0c00640 1 -- 172.21.15.26:0/493273377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac160f80 msgr2=0x7f6bac161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.357+0000 7f6bb0c00640 1 --2- 172.21.15.26:0/493273377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac160f80 0x7f6bac161360 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6b940097a0 tx=0x7f6b9402b790 comp rx=0 tx=0).stop 2024-01-15T18:05:15.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.358+0000 7f6bb0c00640 1 -- 172.21.15.26:0/493273377 shutdown_connections 2024-01-15T18:05:15.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.358+0000 7f6bb0c00640 1 --2- 172.21.15.26:0/493273377 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac0ced50 0x7f6bac0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.358+0000 7f6bb0c00640 1 --2- 172.21.15.26:0/493273377 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac160f80 0x7f6bac161360 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.358+0000 7f6bb0c00640 1 -- 172.21.15.26:0/493273377 >> 172.21.15.26:0/493273377 conn(0x7f6bac09c550 msgr2=0x7f6bac0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:15.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.358+0000 7f6bb0c00640 1 -- 172.21.15.26:0/493273377 shutdown_connections 2024-01-15T18:05:15.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.358+0000 7f6bb0c00640 1 -- 172.21.15.26:0/493273377 wait complete. 2024-01-15T18:05:15.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.360+0000 7f6bb0c00640 1 Processor -- start 2024-01-15T18:05:15.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.361+0000 7f6bb0c00640 1 -- start start 2024-01-15T18:05:15.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.362+0000 7f6bb0c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac0ce8f0 0x7f6bac0cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.362+0000 7f6bb0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac0ced50 0x7f6bac151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.362+0000 7f6bb0c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac160f80 0x7f6bac0caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.362+0000 7f6bb0c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0caa40 con 0x7f6bac0ced50 2024-01-15T18:05:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.362+0000 7f6bb0c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0cd240 con 0x7f6bac0ce8f0 2024-01-15T18:05:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.362+0000 7f6bb0c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac151b70 con 0x7f6bac160f80 2024-01-15T18:05:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.363+0000 7f6ba9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac0ced50 0x7f6bac151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.363+0000 7f6ba9e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac0ce8f0 0x7f6bac0cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.363+0000 7f6ba9e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac0ce8f0 0x7f6bac0cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:43916/0 (socket says 172.21.15.26:43916) 2024-01-15T18:05:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.363+0000 7f6ba9e00640 1 -- 172.21.15.26:0/2534654510 learned_addr learned my addr 172.21.15.26:0/2534654510 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.364+0000 7f6ba9400640 1 -- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac160f80 msgr2=0x7f6bac0caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.364+0000 7f6ba9400640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac160f80 0x7f6bac0caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.364+0000 7f6ba9400640 1 -- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac0ce8f0 msgr2=0x7f6bac0cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.364+0000 7f6ba9400640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac0ce8f0 0x7f6bac0cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.364+0000 7f6ba9400640 1 -- 172.21.15.26:0/2534654510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba4009a70 con 0x7f6bac0ced50 2024-01-15T18:05:15.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.365+0000 7f6ba9400640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac0ced50 0x7f6bac151240 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c005880 tx=0x7f6b9c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:15.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.365+0000 7f6ba2000640 1 -- 172.21.15.26:0/2534654510 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c005ce0 con 0x7f6bac0ced50 2024-01-15T18:05:15.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.366+0000 7f6ba2000640 1 -- 172.21.15.26:0/2534654510 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c005e60 con 0x7f6bac0ced50 2024-01-15T18:05:15.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.366+0000 7f6ba2000640 1 -- 172.21.15.26:0/2534654510 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0072b0 con 0x7f6bac0ced50 2024-01-15T18:05:15.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.366+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b94009400 con 0x7f6bac0ced50 2024-01-15T18:05:15.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.367+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac0ca3f0 con 0x7f6bac0ced50 2024-01-15T18:05:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.371+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac0cabf0 con 0x7f6bac0ced50 2024-01-15T18:05:15.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.372+0000 7f6ba2000640 1 -- 172.21.15.26:0/2534654510 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f6b9c007410 con 0x7f6bac0ced50 2024-01-15T18:05:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.374+0000 7f6ba2000640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b8406bee0 0x7f6b8406e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:15.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.374+0000 7f6ba2000640 1 -- 172.21.15.26:0/2534654510 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f6b9c087c90 con 0x7f6bac0ced50 2024-01-15T18:05:15.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.378+0000 7f6ba9e00640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b8406bee0 0x7f6b8406e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:15.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.383+0000 7f6ba2000640 1 -- 172.21.15.26:0/2534654510 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bac0cabf0 con 0x7f6bac0ced50 2024-01-15T18:05:15.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.398+0000 7f6ba9e00640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b8406bee0 0x7f6b8406e3a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6b940097a0 tx=0x7f6b9402f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:15.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.866+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f6bac0cabf0 con 0x7f6bac0ced50 2024-01-15T18:05:15.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.866+0000 7f6ba2000640 1 -- 172.21.15.26:0/2534654510 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7f6bac0cabf0 con 0x7f6bac0ced50 2024-01-15T18:05:15.871 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:15.871 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:15.871 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:15.871 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:15.871 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:15.871 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:15.871 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:15.871 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:15.871 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:15.872 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:15.872 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:15.872 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:15.872 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:15.872 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:15.872 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:15.872 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 630, 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:15.873 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:15.874 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1496, 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 6129232725, 2024-01-15T18:05:15.875 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 11755102208, 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 761339011072, 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 245626828, 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 236558790, 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 59, 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 62 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:15.876 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:15.877 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:15.878 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:15.879 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 19.046% (22s)\n [=====.......................] (remaining: 96s)", 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.19045713543891907, 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 33.33% (22s)\n [=========...................] (remaining: 45s)", 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.33329999446868896, 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:15.880 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:15.881 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:15.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b8406bee0 msgr2=0x7f6b8406e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b8406bee0 0x7f6b8406e3a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6b940097a0 tx=0x7f6b9402f000 comp rx=0 tx=0).stop 2024-01-15T18:05:15.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac0ced50 msgr2=0x7f6bac151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:15.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac0ced50 0x7f6bac151240 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c005880 tx=0x7f6b9c00c9b0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 shutdown_connections 2024-01-15T18:05:15.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bac160f80 0x7f6bac0caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b8406bee0 0x7f6b8406e3a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bac0ced50 0x7f6bac151240 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 --2- 172.21.15.26:0/2534654510 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bac0ce8f0 0x7f6bac0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:15.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 >> 172.21.15.26:0/2534654510 conn(0x7f6bac09c550 msgr2=0x7f6bac0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:15.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 shutdown_connections 2024-01-15T18:05:15.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:15.869+0000 7f6bb0c00640 1 -- 172.21.15.26:0/2534654510 wait complete. 2024-01-15T18:05:16.891 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.204+0000 7ffa53e00640 1 Processor -- start 2024-01-15T18:05:17.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.204+0000 7ffa53e00640 1 -- start start 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.206+0000 7ffa53e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa540d1900 0x7ffa5406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.206+0000 7ffa53e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ffa540cb950 0x7ffa540cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.206+0000 7ffa53e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa540d1d10 con 0x7ffa5415d440 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.206+0000 7ffa53e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa540cfaf0 con 0x7ffa540d1900 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.206+0000 7ffa53e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa540cfd00 con 0x7ffa540cb950 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.206+0000 7ffa52a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffa5415d440 0x7ffa5406b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50724/0 (socket says 172.21.15.26:50724) 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.206+0000 7ffa52a00640 1 -- 172.21.15.26:0/3658397907 learned_addr learned my addr 172.21.15.26:0/3658397907 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.206+0000 7ffa52000640 1 --2- 172.21.15.26:0/3658397907 >> v2:172.21.15.26:3300/0 conn(0x7ffa540d1900 0x7ffa5406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.206+0000 7ffa53400640 1 --2- 172.21.15.26:0/3658397907 >> v2:172.21.15.124:3300/0 conn(0x7ffa540cb950 0x7ffa540cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.207+0000 7ffa53400640 1 -- 172.21.15.26:0/3658397907 >> v2:172.21.15.26:3300/0 conn(0x7ffa540d1900 msgr2=0x7ffa5406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.207+0000 7ffa53400640 1 --2- 172.21.15.26:0/3658397907 >> v2:172.21.15.26:3300/0 conn(0x7ffa540d1900 0x7ffa5406be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.207+0000 7ffa53400640 1 -- 172.21.15.26:0/3658397907 >> v1:172.21.15.26:6789/0 conn(0x7ffa5415d440 legacy=0x7ffa5406b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.207+0000 7ffa53400640 1 -- 172.21.15.26:0/3658397907 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa540cffe0 con 0x7ffa540cb950 2024-01-15T18:05:17.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.207+0000 7ffa52000640 1 --2- 172.21.15.26:0/3658397907 >> v2:172.21.15.26:3300/0 conn(0x7ffa540d1900 0x7ffa5406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.208+0000 7ffa53400640 1 --2- 172.21.15.26:0/3658397907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa540cb950 0x7ffa540cbd60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa5409ed80 tx=0x7ffa4c02bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=853dd912a9f78b10 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.208+0000 7ffa51600640 1 -- 172.21.15.26:0/3658397907 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c02cc70 con 0x7ffa540cb950 2024-01-15T18:05:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.209+0000 7ffa51600640 1 -- 172.21.15.26:0/3658397907 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c02cdd0 con 0x7ffa540cb950 2024-01-15T18:05:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.209+0000 7ffa51600640 1 -- 172.21.15.26:0/3658397907 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c034660 con 0x7ffa540cb950 2024-01-15T18:05:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.209+0000 7ffa53e00640 1 -- 172.21.15.26:0/3658397907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa540cb950 msgr2=0x7ffa540cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.209+0000 7ffa53e00640 1 --2- 172.21.15.26:0/3658397907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa540cb950 0x7ffa540cbd60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ffa5409ed80 tx=0x7ffa4c02bec0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.209+0000 7ffa53e00640 1 -- 172.21.15.26:0/3658397907 shutdown_connections 2024-01-15T18:05:17.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.209+0000 7ffa53e00640 1 --2- 172.21.15.26:0/3658397907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa540cb950 0x7ffa540cbd60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.209+0000 7ffa53e00640 1 --2- 172.21.15.26:0/3658397907 >> v2:172.21.15.26:3300/0 conn(0x7ffa540d1900 0x7ffa5406be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.209+0000 7ffa53e00640 1 -- 172.21.15.26:0/3658397907 >> 172.21.15.26:0/3658397907 conn(0x7ffa540d0de0 msgr2=0x7ffa540d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:17.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.209+0000 7ffa53e00640 1 -- 172.21.15.26:0/3658397907 shutdown_connections 2024-01-15T18:05:17.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.210+0000 7ffa53e00640 1 -- 172.21.15.26:0/3658397907 wait complete. 2024-01-15T18:05:17.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.211+0000 7ffa53e00640 1 Processor -- start 2024-01-15T18:05:17.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.211+0000 7ffa53e00640 1 -- start start 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.212+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa540cb950 0x7ffa5415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.212+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa540d1900 0x7ffa541f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.212+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa5415d440 0x7ffa541e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.212+0000 7ffa53e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540c5220 con 0x7ffa5415d440 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.212+0000 7ffa53e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5415bf60 con 0x7ffa540d1900 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.212+0000 7ffa53e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5415b4a0 con 0x7ffa540cb950 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa53400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa5415d440 0x7ffa541e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa53400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa5415d440 0x7ffa541e5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56004/0 (socket says 172.21.15.26:56004) 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa53400640 1 -- 172.21.15.26:0/411302448 learned_addr learned my addr 172.21.15.26:0/411302448 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa52a00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa540cb950 0x7ffa5415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa52000640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa540d1900 0x7ffa541f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa53400640 1 -- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa540cb950 msgr2=0x7ffa5415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa53400640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa540cb950 0x7ffa5415af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa53400640 1 -- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa540d1900 msgr2=0x7ffa541f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa53400640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa540d1900 0x7ffa541f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa53400640 1 -- 172.21.15.26:0/411302448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c009400 con 0x7ffa5415d440 2024-01-15T18:05:17.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.213+0000 7ffa52a00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa540cb950 0x7ffa5415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:17.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.214+0000 7ffa53400640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa5415d440 0x7ffa541e5a90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009e80 tx=0x7ffa4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:17.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.214+0000 7ffa4aa00640 1 -- 172.21.15.26:0/411302448 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c040650 con 0x7ffa5415d440 2024-01-15T18:05:17.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.214+0000 7ffa4aa00640 1 -- 172.21.15.26:0/411302448 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c0407b0 con 0x7ffa5415d440 2024-01-15T18:05:17.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.214+0000 7ffa4aa00640 1 -- 172.21.15.26:0/411302448 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c040b60 con 0x7ffa5415d440 2024-01-15T18:05:17.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.214+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c009a70 con 0x7ffa5415d440 2024-01-15T18:05:17.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.214+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa540c48f0 con 0x7ffa5415d440 2024-01-15T18:05:17.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.215+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa540c53d0 con 0x7ffa5415d440 2024-01-15T18:05:17.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.216+0000 7ffa4aa00640 1 -- 172.21.15.26:0/411302448 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7ffa4c040cc0 con 0x7ffa5415d440 2024-01-15T18:05:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.217+0000 7ffa4aa00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa2c06bee0 0x7ffa2c06e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.217+0000 7ffa4aa00640 1 -- 172.21.15.26:0/411302448 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7ffa4c0ab230 con 0x7ffa5415d440 2024-01-15T18:05:17.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.218+0000 7ffa52a00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa2c06bee0 0x7ffa2c06e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:17.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.219+0000 7ffa52a00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa2c06bee0 0x7ffa2c06e3a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffa44005880 tx=0x7ffa44009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:17.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.222+0000 7ffa4aa00640 1 -- 172.21.15.26:0/411302448 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa540c53d0 con 0x7ffa5415d440 2024-01-15T18:05:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.492+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ffa54067450 con 0x7ffa5415d440 2024-01-15T18:05:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.493+0000 7ffa4aa00640 1 -- 172.21.15.26:0/411302448 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7ffa4c044990 con 0x7ffa5415d440 2024-01-15T18:05:17.497 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:17.497 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:17.497 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:17.497 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:17.497 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:17.497 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:17.498 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 631, 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:17.499 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:17.500 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 1496, 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 6129232725, 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 11755102208, 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 761339011072, 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 203690507, 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 199856676, 2024-01-15T18:05:17.501 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 48, 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 50 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:17.502 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:17.503 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:17.504 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 19.046% (22s)\n [=====.......................] (remaining: 96s)", 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.19045713543891907, 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 33.33% (22s)\n [=========...................] (remaining: 45s)", 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.33329999446868896, 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:17.505 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:17.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa2c06bee0 msgr2=0x7ffa2c06e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa2c06bee0 0x7ffa2c06e3a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffa44005880 tx=0x7ffa44009040 comp rx=0 tx=0).stop 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa5415d440 msgr2=0x7ffa541e5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa5415d440 0x7ffa541e5a90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009e80 tx=0x7ffa4c004a40 comp rx=0 tx=0).stop 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 shutdown_connections 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa2c06bee0 0x7ffa2c06e3a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa5415d440 0x7ffa541e5a90 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa540d1900 0x7ffa541f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 --2- 172.21.15.26:0/411302448 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa540cb950 0x7ffa5415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 >> 172.21.15.26:0/411302448 conn(0x7ffa540d0de0 msgr2=0x7ffa540cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:17.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 shutdown_connections 2024-01-15T18:05:17.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:17.496+0000 7ffa53e00640 1 -- 172.21.15.26:0/411302448 wait complete. 2024-01-15T18:05:18.519 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:18.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.745+0000 7f4873e00640 1 Processor -- start 2024-01-15T18:05:18.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.745+0000 7f4873e00640 1 -- start start 2024-01-15T18:05:18.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.747+0000 7f4873e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 0x7f48740cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:18.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.747+0000 7f4873e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f487406d160 0x7f487406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:18.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.747+0000 7f4873e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4874070e50 con 0x7f4874143c40 2024-01-15T18:05:18.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.747+0000 7f4873e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48740723c0 con 0x7f487406d160 2024-01-15T18:05:18.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.747+0000 7f4873e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4874144020 con 0x7f48740cc500 2024-01-15T18:05:18.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.748+0000 7f4872a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4874143c40 0x7f4874071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50736/0 (socket says 172.21.15.26:50736) 2024-01-15T18:05:18.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.748+0000 7f4872a00640 1 -- 172.21.15.26:0/775999282 learned_addr learned my addr 172.21.15.26:0/775999282 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:18.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.748+0000 7f4873400640 1 --2- 172.21.15.26:0/775999282 >> v2:172.21.15.26:3300/0 conn(0x7f487406d160 0x7f487406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:18.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.748+0000 7f486be00640 1 --2- 172.21.15.26:0/775999282 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 0x7f48740cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:18.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.748+0000 7f4873400640 1 -- 172.21.15.26:0/775999282 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 msgr2=0x7f48740cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:18.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.748+0000 7f4873400640 1 --2- 172.21.15.26:0/775999282 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 0x7f48740cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:18.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.748+0000 7f4873400640 1 -- 172.21.15.26:0/775999282 >> v1:172.21.15.26:6789/0 conn(0x7f4874143c40 legacy=0x7f4874071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:18.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.748+0000 7f4873400640 1 -- 172.21.15.26:0/775999282 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740ccea0 con 0x7f487406d160 2024-01-15T18:05:18.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.749+0000 7f4873400640 1 --2- 172.21.15.26:0/775999282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 0x7f487406d590 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f486c009d90 tx=0x7f486c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00dcd60741987c4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:18.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.750+0000 7f4872000640 1 -- 172.21.15.26:0/775999282 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c02cbd0 con 0x7f487406d160 2024-01-15T18:05:18.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.750+0000 7f4872000640 1 -- 172.21.15.26:0/775999282 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c02cd30 con 0x7f487406d160 2024-01-15T18:05:18.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.750+0000 7f4872000640 1 -- 172.21.15.26:0/775999282 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c02cea0 con 0x7f487406d160 2024-01-15T18:05:18.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.750+0000 7f4873e00640 1 -- 172.21.15.26:0/775999282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 msgr2=0x7f487406d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:18.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.750+0000 7f4873e00640 1 --2- 172.21.15.26:0/775999282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 0x7f487406d590 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f486c009d90 tx=0x7f486c02be20 comp rx=0 tx=0).stop 2024-01-15T18:05:18.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.751+0000 7f4873e00640 1 -- 172.21.15.26:0/775999282 shutdown_connections 2024-01-15T18:05:18.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.751+0000 7f4873e00640 1 --2- 172.21.15.26:0/775999282 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 0x7f487406d590 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:18.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.751+0000 7f4873e00640 1 --2- 172.21.15.26:0/775999282 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 0x7f48740cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:18.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.751+0000 7f4873e00640 1 -- 172.21.15.26:0/775999282 >> 172.21.15.26:0/775999282 conn(0x7f487406a800 msgr2=0x7f487406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:18.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.751+0000 7f4873e00640 1 -- 172.21.15.26:0/775999282 shutdown_connections 2024-01-15T18:05:18.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.751+0000 7f4873e00640 1 -- 172.21.15.26:0/775999282 wait complete. 2024-01-15T18:05:18.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.752+0000 7f4873e00640 1 Processor -- start 2024-01-15T18:05:18.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.753+0000 7f4873e00640 1 -- start start 2024-01-15T18:05:18.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f4873e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 0x7f487415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f4873e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 0x7f487413e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f4873e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4874143c40 0x7f487413ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f4873e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740b1f00 con 0x7f487406d160 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f4873e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740b27a0 con 0x7f4874143c40 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f4873e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487415abc0 con 0x7f48740cc500 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f4872a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 0x7f487415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f4872a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 0x7f487415a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56014/0 (socket says 172.21.15.26:56014) 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f4872a00640 1 -- 172.21.15.26:0/4123705355 learned_addr learned my addr 172.21.15.26:0/4123705355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.754+0000 7f486be00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 0x7f487413e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:18.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f4873400640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4874143c40 0x7f487413ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f4872a00640 1 -- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 msgr2=0x7f487413e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f4872a00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 0x7f487413e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f4872a00640 1 -- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4874143c40 msgr2=0x7f487413ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f4872a00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4874143c40 0x7f487413ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f4872a00640 1 -- 172.21.15.26:0/4123705355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c009a70 con 0x7f487406d160 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f4872a00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 0x7f487415a680 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f486400c4b0 tx=0x7f486400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f486aa00640 1 -- 172.21.15.26:0/4123705355 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864005ce0 con 0x7f487406d160 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f486aa00640 1 -- 172.21.15.26:0/4123705355 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864005e40 con 0x7f487406d160 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48741f75a0 con 0x7f487406d160 2024-01-15T18:05:18.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f486aa00640 1 -- 172.21.15.26:0/4123705355 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864007270 con 0x7f487406d160 2024-01-15T18:05:18.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.755+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48740b2430 con 0x7f487406d160 2024-01-15T18:05:18.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.756+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4874067450 con 0x7f487406d160 2024-01-15T18:05:18.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.757+0000 7f486aa00640 1 -- 172.21.15.26:0/4123705355 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f48640073d0 con 0x7f487406d160 2024-01-15T18:05:18.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.758+0000 7f486aa00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f484c06be90 0x7f484c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:18.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.758+0000 7f486aa00640 1 -- 172.21.15.26:0/4123705355 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f4864087c30 con 0x7f487406d160 2024-01-15T18:05:18.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.759+0000 7f486be00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f484c06be90 0x7f484c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:18.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.760+0000 7f486be00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f484c06be90 0x7f484c06e350 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f487413e290 tx=0x7f485c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:18.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:18.763+0000 7f486aa00640 1 -- 172.21.15.26:0/4123705355 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4864054340 con 0x7f487406d160 2024-01-15T18:05:19.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.030+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f48740ceb30 con 0x7f487406d160 2024-01-15T18:05:19.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.031+0000 7f486aa00640 1 -- 172.21.15.26:0/4123705355 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2411 (secure 0 0 0) 0x7f4864017090 con 0x7f487406d160 2024-01-15T18:05:19.034 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:19.035 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 633, 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:19.036 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:19.037 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 2331, 2024-01-15T18:05:19.038 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 9631566068, 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 18948521984, 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 754145591296, 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 487031282, 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 491590409, 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 116, 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 123 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:19.039 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:19.040 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:19.041 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "7 ongoing+pending cloning jobs; Avg progress = 28.571% (25s)\n [========....................] (remaining: 64s)", 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.28571429848670959, 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:19.042 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:19.043 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:19.043 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 50.0% (25s)\n [==============..............] (remaining: 25s)", 2024-01-15T18:05:19.043 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.5, 2024-01-15T18:05:19.043 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:19.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:19.043 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:19.043 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.033+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f484c06be90 msgr2=0x7f484c06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.033+0000 7f4873e00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f484c06be90 0x7f484c06e350 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f487413e290 tx=0x7f485c008040 comp rx=0 tx=0).stop 2024-01-15T18:05:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.033+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 msgr2=0x7f487415a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.033+0000 7f4873e00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 0x7f487415a680 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f486400c4b0 tx=0x7f486400c970 comp rx=0 tx=0).stop 2024-01-15T18:05:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.034+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 shutdown_connections 2024-01-15T18:05:19.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.034+0000 7f4873e00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48740cc500 0x7f487413e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.034+0000 7f4873e00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f484c06be90 0x7f484c06e350 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.034+0000 7f4873e00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4874143c40 0x7f487413ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.034+0000 7f4873e00640 1 --2- 172.21.15.26:0/4123705355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f487406d160 0x7f487415a680 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.034+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 >> 172.21.15.26:0/4123705355 conn(0x7f487406a800 msgr2=0x7f4874141230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.034+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 shutdown_connections 2024-01-15T18:05:19.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:19.034+0000 7f4873e00640 1 -- 172.21.15.26:0/4123705355 wait complete. 2024-01-15T18:05:20.057 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone4188 2024-01-15T18:05:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.287+0000 7f3677c00640 1 Processor -- start 2024-01-15T18:05:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.288+0000 7f3677c00640 1 -- start start 2024-01-15T18:05:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.289+0000 7f3677c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f36700ae5a0 0x7f367015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.289+0000 7f3677c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f36700b3ae0 0x7f367006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.289+0000 7f3677c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3670070c40 con 0x7f367006d120 2024-01-15T18:05:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.289+0000 7f3677c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3670071450 con 0x7f36700b3ae0 2024-01-15T18:05:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.289+0000 7f3677c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f367006c470 con 0x7f36700ae5a0 2024-01-15T18:05:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.290+0000 7f3675c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f367006d120 0x7f36700cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50748/0 (socket says 172.21.15.26:50748) 2024-01-15T18:05:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.290+0000 7f3675c00640 1 -- 172.21.15.26:0/755051645 learned_addr learned my addr 172.21.15.26:0/755051645 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.290+0000 7f3667e00640 1 --2- 172.21.15.26:0/755051645 >> v2:172.21.15.26:3300/0 conn(0x7f36700b3ae0 0x7f367006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.290+0000 7f3675200640 1 --2- 172.21.15.26:0/755051645 >> v2:172.21.15.124:3300/0 conn(0x7f36700ae5a0 0x7f367015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.290+0000 7f3667e00640 1 -- 172.21.15.26:0/755051645 >> v2:172.21.15.124:3300/0 conn(0x7f36700ae5a0 msgr2=0x7f367015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.290+0000 7f3667e00640 1 --2- 172.21.15.26:0/755051645 >> v2:172.21.15.124:3300/0 conn(0x7f36700ae5a0 0x7f367015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.290+0000 7f3667e00640 1 -- 172.21.15.26:0/755051645 >> v1:172.21.15.26:6789/0 conn(0x7f367006d120 legacy=0x7f36700cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.290+0000 7f3667e00640 1 -- 172.21.15.26:0/755051645 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367006c750 con 0x7f36700b3ae0 2024-01-15T18:05:20.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.291+0000 7f3667e00640 1 --2- 172.21.15.26:0/755051645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 0x7f367006cbe0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f36700cfe20 tx=0x7f365802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67746042f4e7cebb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.292+0000 7f366fe00640 1 -- 172.21.15.26:0/755051645 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658031830 con 0x7f36700b3ae0 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.292+0000 7f366fe00640 1 -- 172.21.15.26:0/755051645 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3658031990 con 0x7f36700b3ae0 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.292+0000 7f366fe00640 1 -- 172.21.15.26:0/755051645 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658031d20 con 0x7f36700b3ae0 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.293+0000 7f3677c00640 1 -- 172.21.15.26:0/755051645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 msgr2=0x7f367006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.293+0000 7f3677c00640 1 --2- 172.21.15.26:0/755051645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 0x7f367006cbe0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f36700cfe20 tx=0x7f365802b8d0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.293+0000 7f3677c00640 1 -- 172.21.15.26:0/755051645 shutdown_connections 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.293+0000 7f3677c00640 1 --2- 172.21.15.26:0/755051645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 0x7f367006cbe0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.293+0000 7f3677c00640 1 --2- 172.21.15.26:0/755051645 >> v2:172.21.15.124:3300/0 conn(0x7f36700ae5a0 0x7f367015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.293+0000 7f3677c00640 1 -- 172.21.15.26:0/755051645 >> 172.21.15.26:0/755051645 conn(0x7f367006a800 msgr2=0x7f367006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:20.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.293+0000 7f3677c00640 1 -- 172.21.15.26:0/755051645 shutdown_connections 2024-01-15T18:05:20.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.293+0000 7f3677c00640 1 -- 172.21.15.26:0/755051645 wait complete. 2024-01-15T18:05:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.295+0000 7f3677c00640 1 Processor -- start 2024-01-15T18:05:20.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.296+0000 7f3677c00640 1 -- start start 2024-01-15T18:05:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.297+0000 7f3677c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f367006d120 0x7f3670152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.297+0000 7f3677c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36700ae5a0 0x7f3670157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.297+0000 7f3677c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 0x7f3670158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.297+0000 7f3677c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36700cc240 con 0x7f36700b3ae0 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.297+0000 7f3677c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367006af50 con 0x7f367006d120 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.297+0000 7f3677c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36700cc490 con 0x7f36700ae5a0 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.297+0000 7f3675c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 0x7f3670158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.297+0000 7f3675c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 0x7f3670158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56032/0 (socket says 172.21.15.26:56032) 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.297+0000 7f3675c00640 1 -- 172.21.15.26:0/391821944 learned_addr learned my addr 172.21.15.26:0/391821944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.298+0000 7f3675200640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f367006d120 0x7f3670152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.298+0000 7f3667e00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36700ae5a0 0x7f3670157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.298+0000 7f3675c00640 1 -- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36700ae5a0 msgr2=0x7f3670157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.298+0000 7f3675c00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36700ae5a0 0x7f3670157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.298+0000 7f3675c00640 1 -- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f367006d120 msgr2=0x7f3670152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.298+0000 7f3675c00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f367006d120 0x7f3670152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.298+0000 7f3675c00640 1 -- 172.21.15.26:0/391821944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658009400 con 0x7f36700b3ae0 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.298+0000 7f3675200640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f367006d120 0x7f3670152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.298+0000 7f3675c00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 0x7f3670158350 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f366800cbc0 tx=0x7f366800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.299+0000 7f366d600640 1 -- 172.21.15.26:0/391821944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366800ad40 con 0x7f36700b3ae0 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.299+0000 7f366d600640 1 -- 172.21.15.26:0/391821944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3668012400 con 0x7f36700b3ae0 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.299+0000 7f366d600640 1 -- 172.21.15.26:0/391821944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366800aea0 con 0x7f36700b3ae0 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.299+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36700cc770 con 0x7f36700b3ae0 2024-01-15T18:05:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.299+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367006abe0 con 0x7f36700b3ae0 2024-01-15T18:05:20.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.300+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36700ccc90 con 0x7f36700b3ae0 2024-01-15T18:05:20.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.301+0000 7f366d600640 1 -- 172.21.15.26:0/391821944 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f3668007070 con 0x7f36700b3ae0 2024-01-15T18:05:20.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.302+0000 7f366d600640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f364806be90 0x7f364806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.302+0000 7f366d600640 1 -- 172.21.15.26:0/391821944 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f36680886f0 con 0x7f36700b3ae0 2024-01-15T18:05:20.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.303+0000 7f3675200640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f364806be90 0x7f364806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.304+0000 7f3675200640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f364806be90 0x7f364806e350 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3660006fd0 tx=0x7f3660008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:20.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.307+0000 7f366d600640 1 -- 172.21.15.26:0/391821944 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36700ccc90 con 0x7f36700b3ae0 2024-01-15T18:05:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.542+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone4188", "target": ["mon-mgr", ""]}) v1 -- 0x7f36700cc600 con 0x7f364806be90 2024-01-15T18:05:20.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.710+0000 7f366d600640 1 -- 172.21.15.26:0/391821944 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f36700cc600 con 0x7f364806be90 2024-01-15T18:05:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f364806be90 msgr2=0x7f364806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f364806be90 0x7f364806e350 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3660006fd0 tx=0x7f3660008040 comp rx=0 tx=0).stop 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 msgr2=0x7f3670158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 0x7f3670158350 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f366800cbc0 tx=0x7f366800a430 comp rx=0 tx=0).stop 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 shutdown_connections 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f36700ae5a0 0x7f3670157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f364806be90 0x7f364806e350 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f36700b3ae0 0x7f3670158350 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 --2- 172.21.15.26:0/391821944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f367006d120 0x7f3670152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 >> 172.21.15.26:0/391821944 conn(0x7f367006a800 msgr2=0x7f36700cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 shutdown_connections 2024-01-15T18:05:20.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.714+0000 7f3677c00640 1 -- 172.21.15.26:0/391821944 wait complete. 2024-01-15T18:05:20.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone4189 2024-01-15T18:05:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.970+0000 7f0818400640 1 Processor -- start 2024-01-15T18:05:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.970+0000 7f0818400640 1 -- start start 2024-01-15T18:05:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.971+0000 7f0818400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f081015d6e0 0x7f081006b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.971+0000 7f0818400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100d0530 con 0x7f08100b02b0 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.971+0000 7f0818400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100b0070 con 0x7f08100cb640 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.971+0000 7f0818400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08100b3a80 con 0x7f081015d6e0 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.971+0000 7f0815a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f081015d6e0 0x7f081006b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.971+0000 7f0815000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f08100b02b0 0x7f081006be60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50754/0 (socket says 172.21.15.26:50754) 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.971+0000 7f0815000640 1 -- 172.21.15.26:0/2476456530 learned_addr learned my addr 172.21.15.26:0/2476456530 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.972+0000 7f0815a00640 1 -- 172.21.15.26:0/2476456530 >> v1:172.21.15.124:6789/0 conn(0x7f08100cb640 legacy=0x7f08100cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.972+0000 7f0815a00640 1 -- 172.21.15.26:0/2476456530 >> v1:172.21.15.26:6789/0 conn(0x7f08100b02b0 legacy=0x7f081006be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.972+0000 7f0815a00640 1 -- 172.21.15.26:0/2476456530 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08100b37a0 con 0x7f081015d6e0 2024-01-15T18:05:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.972+0000 7f080fe00640 1 -- 172.21.15.26:0/2476456530 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459317671 0 0) 0x7f08100d0530 con 0x7f08100b02b0 2024-01-15T18:05:20.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.973+0000 7f0815a00640 1 --2- 172.21.15.26:0/2476456530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f081015d6e0 0x7f081006b920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f08000097a0 tx=0x7f080002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c1b557e1cbde4f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:20.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f080fe00640 1 -- 172.21.15.26:0/2476456530 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800031830 con 0x7f081015d6e0 2024-01-15T18:05:20.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f080fe00640 1 -- 172.21.15.26:0/2476456530 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0800031990 con 0x7f081015d6e0 2024-01-15T18:05:20.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f080fe00640 1 -- 172.21.15.26:0/2476456530 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800031d20 con 0x7f081015d6e0 2024-01-15T18:05:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f0818400640 1 -- 172.21.15.26:0/2476456530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f081015d6e0 msgr2=0x7f081006b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f0818400640 1 --2- 172.21.15.26:0/2476456530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f081015d6e0 0x7f081006b920 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f08000097a0 tx=0x7f080002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f0818400640 1 -- 172.21.15.26:0/2476456530 shutdown_connections 2024-01-15T18:05:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f0818400640 1 --2- 172.21.15.26:0/2476456530 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f081015d6e0 0x7f081006b920 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f0818400640 1 -- 172.21.15.26:0/2476456530 >> 172.21.15.26:0/2476456530 conn(0x7f08100b2ae0 msgr2=0x7f08100b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f0818400640 1 -- 172.21.15.26:0/2476456530 shutdown_connections 2024-01-15T18:05:20.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.974+0000 7f0818400640 1 -- 172.21.15.26:0/2476456530 wait complete. 2024-01-15T18:05:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.976+0000 7f0818400640 1 Processor -- start 2024-01-15T18:05:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.976+0000 7f0818400640 1 -- start start 2024-01-15T18:05:20.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.977+0000 7f0818400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08100b02b0 0x7f08101527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.977+0000 7f0818400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08100cb640 0x7f08100af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.977+0000 7f0818400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081015d6e0 0x7f08100c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.977+0000 7f0818400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100aec10 con 0x7f08100b02b0 2024-01-15T18:05:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.977+0000 7f0818400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100c8e90 con 0x7f081015d6e0 2024-01-15T18:05:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.977+0000 7f0818400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08100af570 con 0x7f08100cb640 2024-01-15T18:05:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.977+0000 7f0815a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08100b02b0 0x7f08101527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.977+0000 7f0815a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08100b02b0 0x7f08101527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56056/0 (socket says 172.21.15.26:56056) 2024-01-15T18:05:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.977+0000 7f0815a00640 1 -- 172.21.15.26:0/1495917245 learned_addr learned my addr 172.21.15.26:0/1495917245 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.978+0000 7f0815a00640 1 -- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08100cb640 msgr2=0x7f08100af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.978+0000 7f0815a00640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08100cb640 0x7f08100af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.978+0000 7f0815a00640 1 -- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081015d6e0 msgr2=0x7f08100c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.978+0000 7f0816400640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081015d6e0 0x7f08100c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.978+0000 7f0815a00640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081015d6e0 0x7f08100c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.978+0000 7f0815a00640 1 -- 172.21.15.26:0/1495917245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800009400 con 0x7f08100b02b0 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.978+0000 7f0816400640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081015d6e0 0x7f08100c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.978+0000 7f0815a00640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08100b02b0 0x7f08101527c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f0800000c00 tx=0x7f08000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.979+0000 7f080d600640 1 -- 172.21.15.26:0/1495917245 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080003aa60 con 0x7f08100b02b0 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.979+0000 7f080d600640 1 -- 172.21.15.26:0/1495917245 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080003abc0 con 0x7f08100b02b0 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.979+0000 7f080d600640 1 -- 172.21.15.26:0/1495917245 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000443f0 con 0x7f08100b02b0 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.979+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08100adf90 con 0x7f08100b02b0 2024-01-15T18:05:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.979+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08100ae550 con 0x7f08100b02b0 2024-01-15T18:05:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.980+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08100675d0 con 0x7f08100b02b0 2024-01-15T18:05:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.981+0000 7f080d600640 1 -- 172.21.15.26:0/1495917245 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f080004d450 con 0x7f08100b02b0 2024-01-15T18:05:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.982+0000 7f080d600640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07e806be90 0x7f07e806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.982+0000 7f080d600640 1 -- 172.21.15.26:0/1495917245 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f08000aaa20 con 0x7f08100b02b0 2024-01-15T18:05:20.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.983+0000 7f0815000640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07e806be90 0x7f07e806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.984+0000 7f0815000640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07e806be90 0x7f07e806e350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f07f8007fd0 tx=0x7f07f8007960 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:20.986+0000 7f080d600640 1 -- 172.21.15.26:0/1495917245 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0800077080 con 0x7f08100b02b0 2024-01-15T18:05:21.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.219+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone4189", "target": ["mon-mgr", ""]}) v1 -- 0x7f08100d01c0 con 0x7f07e806be90 2024-01-15T18:05:21.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.408+0000 7f080d600640 1 -- 172.21.15.26:0/1495917245 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f07f8001220 con 0x7f07e806be90 2024-01-15T18:05:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07e806be90 msgr2=0x7f07e806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07e806be90 0x7f07e806e350 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f07f8007fd0 tx=0x7f07f8007960 comp rx=0 tx=0).stop 2024-01-15T18:05:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08100b02b0 msgr2=0x7f08101527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:21.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08100b02b0 0x7f08101527c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f0800000c00 tx=0x7f08000046c0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 shutdown_connections 2024-01-15T18:05:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08100cb640 0x7f08100af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07e806be90 0x7f07e806e350 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f081015d6e0 0x7f08100c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 --2- 172.21.15.26:0/1495917245 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08100b02b0 0x7f08101527c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.411+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 >> 172.21.15.26:0/1495917245 conn(0x7f08100b2ae0 msgr2=0x7f08100cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.412+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 shutdown_connections 2024-01-15T18:05:21.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.412+0000 7f0818400640 1 -- 172.21.15.26:0/1495917245 wait complete. 2024-01-15T18:05:21.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41810 2024-01-15T18:05:21.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.667+0000 7f8aea800640 1 Processor -- start 2024-01-15T18:05:21.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.667+0000 7f8aea800640 1 -- start start 2024-01-15T18:05:21.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.668+0000 7f8aea800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8ae40d0100 0x7f8ae40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:21.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.668+0000 7f8aea800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae40d0a10 con 0x7f8ae4160920 2024-01-15T18:05:21.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.668+0000 7f8aea800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae40cf080 con 0x7f8ae40d0f40 2024-01-15T18:05:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.668+0000 7f8aea800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae40c6be0 con 0x7f8ae40d0100 2024-01-15T18:05:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.669+0000 7f8ae9400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8ae4160920 0x7f8ae40cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50770/0 (socket says 172.21.15.26:50770) 2024-01-15T18:05:21.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.669+0000 7f8ae9400640 1 -- 172.21.15.26:0/4249999440 learned_addr learned my addr 172.21.15.26:0/4249999440 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:21.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.669+0000 7f8ae8a00640 1 --2- 172.21.15.26:0/4249999440 >> v2:172.21.15.124:3300/0 conn(0x7f8ae40d0100 0x7f8ae40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:21.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.669+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884786936 0 0) 0x7f8ae40d0a10 con 0x7f8ae4160920 2024-01-15T18:05:21.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.669+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ac4003520 con 0x7f8ae4160920 2024-01-15T18:05:21.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3711429231 0 0) 0x7f8ac4003520 con 0x7f8ae4160920 2024-01-15T18:05:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 >> v2:172.21.15.124:3300/0 conn(0x7f8ae40d0100 msgr2=0x7f8ae40b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 --2- 172.21.15.26:0/4249999440 >> v2:172.21.15.124:3300/0 conn(0x7f8ae40d0100 0x7f8ae40b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 >> v1:172.21.15.124:6789/0 conn(0x7f8ae40d0f40 legacy=0x7f8ae40d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae40ce7e0 con 0x7f8ae4160920 2024-01-15T18:05:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8ad4001670 con 0x7f8ae4160920 2024-01-15T18:05:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 >> v1:172.21.15.26:6789/0 conn(0x7f8ae4160920 legacy=0x7f8ae40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ac4003970 0x7f8ac4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:21.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40047c0 0x7f8ac4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:21.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac4005180 0x7f8ac400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:21.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40c6be0 con 0x7f8ac40047c0 2024-01-15T18:05:21.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac400c020 con 0x7f8ac4005180 2024-01-15T18:05:21.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac400c1e0 con 0x7f8ac4003970 2024-01-15T18:05:21.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.670+0000 7f8ae8a00640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40047c0 0x7f8ac4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:21.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.671+0000 7f8ae9e00640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac4005180 0x7f8ac400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:21.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.671+0000 7f8ae8a00640 1 -- 172.21.15.26:0/4249999440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ac4003970 msgr2=0x7f8ac4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.671+0000 7f8ae8a00640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ac4003970 0x7f8ac4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.671+0000 7f8ae8a00640 1 -- 172.21.15.26:0/4249999440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac4005180 msgr2=0x7f8ac400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.671+0000 7f8ae8a00640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac4005180 0x7f8ac400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.671+0000 7f8ae8a00640 1 -- 172.21.15.26:0/4249999440 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4003520 con 0x7f8ac40047c0 2024-01-15T18:05:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.671+0000 7f8ae9e00640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac4005180 0x7f8ac400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.671+0000 7f8ae9e00640 1 -- 172.21.15.26:0/4249999440 reap_dead start 2024-01-15T18:05:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.672+0000 7f8ae8a00640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40047c0 0x7f8ac4004c40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009fc0 tx=0x7f8ac40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e97b35f18da871 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.672+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc02cc20 con 0x7f8ac40047c0 2024-01-15T18:05:21.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.672+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc00f040 con 0x7f8ac40047c0 2024-01-15T18:05:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.672+0000 7f8ae3e00640 1 -- 172.21.15.26:0/4249999440 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc034440 con 0x7f8ac40047c0 2024-01-15T18:05:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.673+0000 7f8aea800640 1 -- 172.21.15.26:0/4249999440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40047c0 msgr2=0x7f8ac4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.673+0000 7f8aea800640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40047c0 0x7f8ac4004c40 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009fc0 tx=0x7f8ac40057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.673+0000 7f8aea800640 1 -- 172.21.15.26:0/4249999440 shutdown_connections 2024-01-15T18:05:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.673+0000 7f8aea800640 1 --2- 172.21.15.26:0/4249999440 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ac40047c0 0x7f8ac4004c40 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.673+0000 7f8aea800640 1 -- 172.21.15.26:0/4249999440 >> 172.21.15.26:0/4249999440 conn(0x7f8ae40d3090 msgr2=0x7f8ae40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.673+0000 7f8aea800640 1 -- 172.21.15.26:0/4249999440 shutdown_connections 2024-01-15T18:05:21.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.673+0000 7f8aea800640 1 -- 172.21.15.26:0/4249999440 wait complete. 2024-01-15T18:05:21.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.675+0000 7f8aea800640 1 Processor -- start 2024-01-15T18:05:21.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.675+0000 7f8aea800640 1 -- start start 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.676+0000 7f8aea800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac40047c0 0x7f8ae40afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.676+0000 7f8aea800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae40cbb90 0x7f8ae40cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.676+0000 7f8aea800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ae414af60 0x7f8ae414b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.676+0000 7f8aea800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae4149630 con 0x7f8ae414af60 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.676+0000 7f8aea800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae40b1ac0 con 0x7f8ac40047c0 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.676+0000 7f8aea800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae415c850 con 0x7f8ae40cbb90 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.676+0000 7f8ae9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ae414af60 0x7f8ae414b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.676+0000 7f8ae9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ae414af60 0x7f8ae414b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56076/0 (socket says 172.21.15.26:56076) 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.676+0000 7f8ae9e00640 1 -- 172.21.15.26:0/359680964 learned_addr learned my addr 172.21.15.26:0/359680964 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:21.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae9400640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac40047c0 0x7f8ae40afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae9e00640 1 -- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae40cbb90 msgr2=0x7f8ae40cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae9e00640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae40cbb90 0x7f8ae40cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae9e00640 1 -- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac40047c0 msgr2=0x7f8ae40afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae9e00640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac40047c0 0x7f8ae40afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae9e00640 1 -- 172.21.15.26:0/359680964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae40cc550 con 0x7f8ae414af60 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae9400640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac40047c0 0x7f8ae40afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae9e00640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ae414af60 0x7f8ae414b3e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8ae408c2c0 tx=0x7f8adc00d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae1600640 1 -- 172.21.15.26:0/359680964 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00b070 con 0x7f8ae414af60 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae1600640 1 -- 172.21.15.26:0/359680964 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc0052d0 con 0x7f8ae414af60 2024-01-15T18:05:21.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae40cc7f0 con 0x7f8ae414af60 2024-01-15T18:05:21.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8ae1600640 1 -- 172.21.15.26:0/359680964 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc0035f0 con 0x7f8ae414af60 2024-01-15T18:05:21.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.677+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae4148f70 con 0x7f8ae414af60 2024-01-15T18:05:21.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.679+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae4149770 con 0x7f8ae414af60 2024-01-15T18:05:21.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.680+0000 7f8ae1600640 1 -- 172.21.15.26:0/359680964 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f8adc003750 con 0x7f8ae414af60 2024-01-15T18:05:21.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.681+0000 7f8ae1600640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8abc06bec0 0x7f8abc06e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:21.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.681+0000 7f8ae1600640 1 -- 172.21.15.26:0/359680964 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f8adc087ad0 con 0x7f8ae414af60 2024-01-15T18:05:21.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.681+0000 7f8ae9400640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8abc06bec0 0x7f8abc06e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:21.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.682+0000 7f8ae9400640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8abc06bec0 0x7f8abc06e380 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4001820 tx=0x7f8ad4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:21.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.685+0000 7f8ae1600640 1 -- 172.21.15.26:0/359680964 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ae4149770 con 0x7f8ae414af60 2024-01-15T18:05:21.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:21.919+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41810", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ae40b3ea0 con 0x7f8abc06bec0 2024-01-15T18:05:22.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.101+0000 7f8ae1600640 1 -- 172.21.15.26:0/359680964 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8ae40b3ea0 con 0x7f8abc06bec0 2024-01-15T18:05:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8abc06bec0 msgr2=0x7f8abc06e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8abc06bec0 0x7f8abc06e380 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4001820 tx=0x7f8ad4009040 comp rx=0 tx=0).stop 2024-01-15T18:05:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ae414af60 msgr2=0x7f8ae414b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ae414af60 0x7f8ae414b3e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f8ae408c2c0 tx=0x7f8adc00d5e0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 shutdown_connections 2024-01-15T18:05:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ae40cbb90 0x7f8ae40cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8abc06bec0 0x7f8abc06e380 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ae414af60 0x7f8ae414b3e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 --2- 172.21.15.26:0/359680964 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ac40047c0 0x7f8ae40afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 >> 172.21.15.26:0/359680964 conn(0x7f8ae40d3090 msgr2=0x7f8ae40d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 shutdown_connections 2024-01-15T18:05:22.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.105+0000 7f8aea800640 1 -- 172.21.15.26:0/359680964 wait complete. 2024-01-15T18:05:22.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41811 2024-01-15T18:05:22.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.361+0000 7f1809600640 1 Processor -- start 2024-01-15T18:05:22.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.362+0000 7f1809600640 1 -- start start 2024-01-15T18:05:22.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.363+0000 7f1809600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f180409a170 0x7f18040b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.363+0000 7f1809600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180406cea0 con 0x7f180406c6b0 2024-01-15T18:05:22.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.363+0000 7f1809600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180406cbc0 con 0x7f18040cd240 2024-01-15T18:05:22.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.363+0000 7f1809600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180406f350 con 0x7f180409a170 2024-01-15T18:05:22.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.364+0000 7f1803400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f180406c6b0 0x7f18040cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53868/0 (socket says 172.21.15.26:53868) 2024-01-15T18:05:22.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.364+0000 7f1803400640 1 -- 172.21.15.26:0/2949093263 learned_addr learned my addr 172.21.15.26:0/2949093263 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:22.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.364+0000 7f1802000640 1 --2- 172.21.15.26:0/2949093263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f180409a170 0x7f18040b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.364+0000 7f1801600640 1 -- 172.21.15.26:0/2949093263 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311762656 0 0) 0x7f180406cea0 con 0x7f180406c6b0 2024-01-15T18:05:22.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.365+0000 7f1801600640 1 -- 172.21.15.26:0/2949093263 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17e4003520 con 0x7f180406c6b0 2024-01-15T18:05:22.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.365+0000 7f1801600640 1 -- 172.21.15.26:0/2949093263 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679226332 0 0) 0x7f180406cbc0 con 0x7f18040cd240 2024-01-15T18:05:22.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.365+0000 7f1801600640 1 -- 172.21.15.26:0/2949093263 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f180406cea0 con 0x7f18040cd240 2024-01-15T18:05:22.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.365+0000 7f1802000640 1 -- 172.21.15.26:0/2949093263 >> v1:172.21.15.124:6789/0 conn(0x7f18040cd240 legacy=0x7f1804099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.365+0000 7f1802000640 1 -- 172.21.15.26:0/2949093263 >> v1:172.21.15.26:6789/0 conn(0x7f180406c6b0 legacy=0x7f18040cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.365+0000 7f1802000640 1 -- 172.21.15.26:0/2949093263 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180406f630 con 0x7f180409a170 2024-01-15T18:05:22.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.366+0000 7f1802000640 1 --2- 172.21.15.26:0/2949093263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f180409a170 0x7f18040b3820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009dc0 tx=0x7f17ec02bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14b9c0018af29bd8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:22.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.367+0000 7f1801600640 1 -- 172.21.15.26:0/2949093263 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec02ccc0 con 0x7f180409a170 2024-01-15T18:05:22.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.367+0000 7f1801600640 1 -- 172.21.15.26:0/2949093263 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec02ce20 con 0x7f180409a170 2024-01-15T18:05:22.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.367+0000 7f1801600640 1 -- 172.21.15.26:0/2949093263 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec034660 con 0x7f180409a170 2024-01-15T18:05:22.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.367+0000 7f1809600640 1 -- 172.21.15.26:0/2949093263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f180409a170 msgr2=0x7f18040b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.367+0000 7f1809600640 1 --2- 172.21.15.26:0/2949093263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f180409a170 0x7f18040b3820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009dc0 tx=0x7f17ec02bf10 comp rx=0 tx=0).stop 2024-01-15T18:05:22.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.368+0000 7f1809600640 1 -- 172.21.15.26:0/2949093263 shutdown_connections 2024-01-15T18:05:22.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.368+0000 7f1809600640 1 --2- 172.21.15.26:0/2949093263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f180409a170 0x7f18040b3820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.368+0000 7f1809600640 1 -- 172.21.15.26:0/2949093263 >> 172.21.15.26:0/2949093263 conn(0x7f18040b2d00 msgr2=0x7f18040b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:22.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.368+0000 7f1809600640 1 -- 172.21.15.26:0/2949093263 shutdown_connections 2024-01-15T18:05:22.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.368+0000 7f1809600640 1 -- 172.21.15.26:0/2949093263 wait complete. 2024-01-15T18:05:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.370+0000 7f1809600640 1 Processor -- start 2024-01-15T18:05:22.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.370+0000 7f1809600640 1 -- start start 2024-01-15T18:05:22.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1809600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180406c6b0 0x7f18040c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1809600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f180409a170 0x7f180415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1809600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18040cd240 0x7f18040b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1809600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18040b4e80 con 0x7f180406c6b0 2024-01-15T18:05:22.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1809600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18040c7020 con 0x7f180409a170 2024-01-15T18:05:22.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1809600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180415f1a0 con 0x7f18040cd240 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1802a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180406c6b0 0x7f18040c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1802a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180406c6b0 0x7f18040c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47208/0 (socket says 172.21.15.26:47208) 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1802a00640 1 -- 172.21.15.26:0/4043882674 learned_addr learned my addr 172.21.15.26:0/4043882674 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1802000640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f180409a170 0x7f180415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1803400640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18040cd240 0x7f18040b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1802a00640 1 -- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18040cd240 msgr2=0x7f18040b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1802a00640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18040cd240 0x7f18040b6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1802a00640 1 -- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f180409a170 msgr2=0x7f180415ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.371+0000 7f1802a00640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f180409a170 0x7f180415ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.372+0000 7f1802a00640 1 -- 172.21.15.26:0/4043882674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec009a70 con 0x7f180406c6b0 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.372+0000 7f1802000640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f180409a170 0x7f180415ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:22.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.372+0000 7f1802a00640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180406c6b0 0x7f18040c6ae0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f17f400c500 tx=0x7f17f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.372+0000 7f17faa00640 1 -- 172.21.15.26:0/4043882674 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f40052d0 con 0x7f180406c6b0 2024-01-15T18:05:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.372+0000 7f17faa00640 1 -- 172.21.15.26:0/4043882674 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f4005450 con 0x7f180406c6b0 2024-01-15T18:05:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.372+0000 7f17faa00640 1 -- 172.21.15.26:0/4043882674 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4012870 con 0x7f180406c6b0 2024-01-15T18:05:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.372+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18040b4200 con 0x7f180406c6b0 2024-01-15T18:05:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.372+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18040b47c0 con 0x7f180406c6b0 2024-01-15T18:05:22.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.373+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1804073d90 con 0x7f180406c6b0 2024-01-15T18:05:22.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.375+0000 7f17faa00640 1 -- 172.21.15.26:0/4043882674 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f17f40129d0 con 0x7f180406c6b0 2024-01-15T18:05:22.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.375+0000 7f17faa00640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f17dc06be90 0x7f17dc06e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:22.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.376+0000 7f17faa00640 1 -- 172.21.15.26:0/4043882674 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f17f4088490 con 0x7f180406c6b0 2024-01-15T18:05:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.376+0000 7f1802000640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f17dc06be90 0x7f17dc06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:22.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.377+0000 7f1802000640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f17dc06be90 0x7f17dc06e350 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17ec036000 tx=0x7f17ec030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:22.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.381+0000 7f17faa00640 1 -- 172.21.15.26:0/4043882674 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17f4054a70 con 0x7f180406c6b0 2024-01-15T18:05:22.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.612+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41811", "target": ["mon-mgr", ""]}) v1 -- 0x7f18040675d0 con 0x7f17dc06be90 2024-01-15T18:05:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.768+0000 7f17faa00640 1 -- 172.21.15.26:0/4043882674 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f18040675d0 con 0x7f17dc06be90 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.771+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f17dc06be90 msgr2=0x7f17dc06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.771+0000 7f1809600640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f17dc06be90 0x7f17dc06e350 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17ec036000 tx=0x7f17ec030000 comp rx=0 tx=0).stop 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180406c6b0 msgr2=0x7f18040c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180406c6b0 0x7f18040c6ae0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f17f400c500 tx=0x7f17f400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 shutdown_connections 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18040cd240 0x7f18040b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f17dc06be90 0x7f17dc06e350 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f180409a170 0x7f180415ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 --2- 172.21.15.26:0/4043882674 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f180406c6b0 0x7f18040c6ae0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 >> 172.21.15.26:0/4043882674 conn(0x7f18040b2d00 msgr2=0x7f1804099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 shutdown_connections 2024-01-15T18:05:22.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:22.772+0000 7f1809600640 1 -- 172.21.15.26:0/4043882674 wait complete. 2024-01-15T18:05:22.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41812 2024-01-15T18:05:23.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.019+0000 7f7b29000640 1 Processor -- start 2024-01-15T18:05:23.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.020+0000 7f7b29000640 1 -- start start 2024-01-15T18:05:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.021+0000 7f7b29000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24160920 0x7f7b240b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.021+0000 7f7b29000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7b240cf9f0 0x7f7b240cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.021+0000 7f7b29000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b24161010 0x7f7b241614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.021+0000 7f7b29000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b240d03b0 con 0x7f7b24161010 2024-01-15T18:05:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.021+0000 7f7b29000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b240d0a10 con 0x7f7b240cf9f0 2024-01-15T18:05:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.021+0000 7f7b29000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240cf080 con 0x7f7b24160920 2024-01-15T18:05:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b24161010 0x7f7b241614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b24161010 0x7f7b241614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47220/0 (socket says 172.21.15.26:47220) 2024-01-15T18:05:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23e00640 1 -- 172.21.15.26:0/266878470 learned_addr learned my addr 172.21.15.26:0/266878470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:23.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23400640 1 --2- 172.21.15.26:0/266878470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24160920 0x7f7b240b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b22a00640 1 --2- 172.21.15.26:0/266878470 >> v2:172.21.15.124:3300/0 conn(0x7f7b240cf9f0 0x7f7b240cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23e00640 1 -- 172.21.15.26:0/266878470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24160920 msgr2=0x7f7b240b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23e00640 1 --2- 172.21.15.26:0/266878470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24160920 0x7f7b240b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23e00640 1 -- 172.21.15.26:0/266878470 >> v2:172.21.15.124:3300/0 conn(0x7f7b240cf9f0 msgr2=0x7f7b240cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23e00640 1 --2- 172.21.15.26:0/266878470 >> v2:172.21.15.124:3300/0 conn(0x7f7b240cf9f0 0x7f7b240cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23e00640 1 -- 172.21.15.26:0/266878470 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b240c6be0 con 0x7f7b24161010 2024-01-15T18:05:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b23400640 1 --2- 172.21.15.26:0/266878470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24160920 0x7f7b240b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:23.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.022+0000 7f7b22a00640 1 --2- 172.21.15.26:0/266878470 >> v2:172.21.15.124:3300/0 conn(0x7f7b240cf9f0 0x7f7b240cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.023+0000 7f7b23e00640 1 --2- 172.21.15.26:0/266878470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24161010 0x7f7b241614a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f7b2408d300 tx=0x7f7b1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee4fceffba144e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.024+0000 7f7b22000640 1 -- 172.21.15.26:0/266878470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c031870 con 0x7f7b24161010 2024-01-15T18:05:23.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.024+0000 7f7b22000640 1 -- 172.21.15.26:0/266878470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c0319d0 con 0x7f7b24161010 2024-01-15T18:05:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.024+0000 7f7b22000640 1 -- 172.21.15.26:0/266878470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c031d60 con 0x7f7b24161010 2024-01-15T18:05:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.024+0000 7f7b29000640 1 -- 172.21.15.26:0/266878470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24161010 msgr2=0x7f7b241614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.024+0000 7f7b29000640 1 --2- 172.21.15.26:0/266878470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24161010 0x7f7b241614a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f7b2408d300 tx=0x7f7b1c02b910 comp rx=0 tx=0).stop 2024-01-15T18:05:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.025+0000 7f7b29000640 1 -- 172.21.15.26:0/266878470 shutdown_connections 2024-01-15T18:05:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.025+0000 7f7b29000640 1 --2- 172.21.15.26:0/266878470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b24161010 0x7f7b241614a0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.025+0000 7f7b29000640 1 --2- 172.21.15.26:0/266878470 >> v2:172.21.15.124:3300/0 conn(0x7f7b240cf9f0 0x7f7b240cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.025+0000 7f7b29000640 1 --2- 172.21.15.26:0/266878470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24160920 0x7f7b240b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.025+0000 7f7b29000640 1 -- 172.21.15.26:0/266878470 >> 172.21.15.26:0/266878470 conn(0x7f7b240d3090 msgr2=0x7f7b240d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:23.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.025+0000 7f7b29000640 1 -- 172.21.15.26:0/266878470 shutdown_connections 2024-01-15T18:05:23.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.025+0000 7f7b29000640 1 -- 172.21.15.26:0/266878470 wait complete. 2024-01-15T18:05:23.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.027+0000 7f7b29000640 1 Processor -- start 2024-01-15T18:05:23.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.027+0000 7f7b29000640 1 -- start start 2024-01-15T18:05:23.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b29000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b240cf9f0 0x7f7b2415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b29000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b24160920 0x7f7b2414c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b29000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24161010 0x7f7b2415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b29000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240cb840 con 0x7f7b240cf9f0 2024-01-15T18:05:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b29000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b240b89b0 con 0x7f7b24160920 2024-01-15T18:05:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b29000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2415d850 con 0x7f7b24161010 2024-01-15T18:05:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b23400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b240cf9f0 0x7f7b2415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b23400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b240cf9f0 0x7f7b2415d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47232/0 (socket says 172.21.15.26:47232) 2024-01-15T18:05:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b23400640 1 -- 172.21.15.26:0/1718109040 learned_addr learned my addr 172.21.15.26:0/1718109040 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:23.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b23e00640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24161010 0x7f7b2415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b22a00640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b24160920 0x7f7b2414c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b23400640 1 -- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24161010 msgr2=0x7f7b2415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b23400640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24161010 0x7f7b2415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b23400640 1 -- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b24160920 msgr2=0x7f7b2414c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b23400640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b24160920 0x7f7b2414c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b23400640 1 -- 172.21.15.26:0/1718109040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0094d0 con 0x7f7b240cf9f0 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.029+0000 7f7b22a00640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b24160920 0x7f7b2414c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.030+0000 7f7b23400640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b240cf9f0 0x7f7b2415d310 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400cf90 tx=0x7f7b1400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.030+0000 7f7b1b400640 1 -- 172.21.15.26:0/1718109040 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400d840 con 0x7f7b240cf9f0 2024-01-15T18:05:23.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.030+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b240cb1c0 con 0x7f7b240cf9f0 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.030+0000 7f7b1b400640 1 -- 172.21.15.26:0/1718109040 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1400d9a0 con 0x7f7b240cf9f0 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.030+0000 7f7b1b400640 1 -- 172.21.15.26:0/1718109040 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1400dd30 con 0x7f7b240cf9f0 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.030+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b240cb460 con 0x7f7b240cf9f0 2024-01-15T18:05:23.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.032+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b240b5a60 con 0x7f7b240cf9f0 2024-01-15T18:05:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.032+0000 7f7b1b400640 1 -- 172.21.15.26:0/1718109040 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f7b14015440 con 0x7f7b240cf9f0 2024-01-15T18:05:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.034+0000 7f7b1b400640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7afc06be90 0x7f7afc06e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.035+0000 7f7b1b400640 1 -- 172.21.15.26:0/1718109040 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f7b14088f80 con 0x7f7b240cf9f0 2024-01-15T18:05:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.035+0000 7f7b22a00640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7afc06be90 0x7f7afc06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.036+0000 7f7b22a00640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7afc06be90 0x7f7afc06e350 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7b241f68e0 tx=0x7f7b0c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.041+0000 7f7b1b400640 1 -- 172.21.15.26:0/1718109040 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b140557e0 con 0x7f7b240cf9f0 2024-01-15T18:05:23.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.273+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41812", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b2406bf90 con 0x7f7afc06be90 2024-01-15T18:05:23.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.444+0000 7f7b1b400640 1 -- 172.21.15.26:0/1718109040 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7b2406bf90 con 0x7f7afc06be90 2024-01-15T18:05:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7afc06be90 msgr2=0x7f7afc06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7afc06be90 0x7f7afc06e350 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7b241f68e0 tx=0x7f7b0c008040 comp rx=0 tx=0).stop 2024-01-15T18:05:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b240cf9f0 msgr2=0x7f7b2415d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b240cf9f0 0x7f7b2415d310 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400cf90 tx=0x7f7b1400a7d0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 shutdown_connections 2024-01-15T18:05:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b24161010 0x7f7b2415ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7afc06be90 0x7f7afc06e350 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b24160920 0x7f7b2414c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 --2- 172.21.15.26:0/1718109040 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b240cf9f0 0x7f7b2415d310 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 >> 172.21.15.26:0/1718109040 conn(0x7f7b240d3090 msgr2=0x7f7b240d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 shutdown_connections 2024-01-15T18:05:23.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.448+0000 7f7b29000640 1 -- 172.21.15.26:0/1718109040 wait complete. 2024-01-15T18:05:23.472 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41813 2024-01-15T18:05:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.691+0000 7fa80be00640 1 Processor -- start 2024-01-15T18:05:23.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.691+0000 7fa80be00640 1 -- start start 2024-01-15T18:05:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.693+0000 7fa80be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa80c0cc990 0x7fa80c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.693+0000 7fa80be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa80c070e50 con 0x7fa80c143c40 2024-01-15T18:05:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.693+0000 7fa80be00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa80c144020 con 0x7fa80c0cc500 2024-01-15T18:05:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.693+0000 7fa80be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa80c0723c0 con 0x7fa80c0cc990 2024-01-15T18:05:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.693+0000 7fa80aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa80c143c40 0x7fa80c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53882/0 (socket says 172.21.15.26:53882) 2024-01-15T18:05:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.693+0000 7fa80aa00640 1 -- 172.21.15.26:0/864993559 learned_addr learned my addr 172.21.15.26:0/864993559 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.694+0000 7fa80b400640 1 --2- 172.21.15.26:0/864993559 >> v2:172.21.15.124:3300/0 conn(0x7fa80c0cc990 0x7fa80c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.694+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281795518 0 0) 0x7fa80c070e50 con 0x7fa80c143c40 2024-01-15T18:05:23.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.694+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7ec003520 con 0x7fa80c143c40 2024-01-15T18:05:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.694+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 717493126 0 0) 0x7fa7ec003520 con 0x7fa80c143c40 2024-01-15T18:05:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.694+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 >> v2:172.21.15.124:3300/0 conn(0x7fa80c0cc990 msgr2=0x7fa80c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.694+0000 7fa809600640 1 --2- 172.21.15.26:0/864993559 >> v2:172.21.15.124:3300/0 conn(0x7fa80c0cc990 0x7fa80c06d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.694+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 >> v1:172.21.15.124:6789/0 conn(0x7fa80c0cc500 legacy=0x7fa80c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.694+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c071e00 con 0x7fa80c143c40 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.694+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa7fc001670 con 0x7fa80c143c40 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 >> v1:172.21.15.26:6789/0 conn(0x7fa80c143c40 legacy=0x7fa80c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa809600640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7ec0039c0 0x7fa7ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa809600640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7ec0047c0 0x7fa7ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa809600640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7ec0051d0 0x7fa7ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c0723c0 con 0x7fa7ec0039c0 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec00c120 con 0x7fa7ec0051d0 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec00c2e0 con 0x7fa7ec0047c0 2024-01-15T18:05:23.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa80aa00640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7ec0039c0 0x7fa7ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa80a000640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7ec0047c0 0x7fa7ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa80b400640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7ec0051d0 0x7fa7ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa80aa00640 1 -- 172.21.15.26:0/864993559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7ec0047c0 msgr2=0x7fa7ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa80aa00640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa7ec0047c0 0x7fa7ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa80aa00640 1 -- 172.21.15.26:0/864993559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7ec0051d0 msgr2=0x7fa7ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa80aa00640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7ec0051d0 0x7fa7ec00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa80b400640 1 -- 172.21.15.26:0/864993559 reap_dead start 2024-01-15T18:05:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.695+0000 7fa80aa00640 1 -- 172.21.15.26:0/864993559 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec003520 con 0x7fa7ec0039c0 2024-01-15T18:05:23.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.697+0000 7fa80aa00640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7ec0039c0 0x7fa7ec003de0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc001eb0 tx=0x7fa7ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70b195cb73306e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.697+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc005ad0 con 0x7fa7ec0039c0 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.697+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc005c50 con 0x7fa7ec0039c0 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.697+0000 7fa809600640 1 -- 172.21.15.26:0/864993559 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc02c7a0 con 0x7fa7ec0039c0 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.698+0000 7fa80be00640 1 -- 172.21.15.26:0/864993559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7ec0039c0 msgr2=0x7fa7ec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.698+0000 7fa80be00640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7ec0039c0 0x7fa7ec003de0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc001eb0 tx=0x7fa7ec0057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.698+0000 7fa80be00640 1 -- 172.21.15.26:0/864993559 shutdown_connections 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.698+0000 7fa80be00640 1 --2- 172.21.15.26:0/864993559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa7ec0039c0 0x7fa7ec003de0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.698+0000 7fa80be00640 1 -- 172.21.15.26:0/864993559 >> 172.21.15.26:0/864993559 conn(0x7fa80c06a800 msgr2=0x7fa80c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.698+0000 7fa80be00640 1 -- 172.21.15.26:0/864993559 shutdown_connections 2024-01-15T18:05:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.698+0000 7fa80be00640 1 -- 172.21.15.26:0/864993559 wait complete. 2024-01-15T18:05:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.700+0000 7fa80be00640 1 Processor -- start 2024-01-15T18:05:23.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.700+0000 7fa80be00640 1 -- start start 2024-01-15T18:05:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.701+0000 7fa80be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7ec0039c0 0x7fa80c141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.701+0000 7fa80be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa80c1e73c0 0x7fa80c15f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.701+0000 7fa80be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15f690 0x7fa80c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.701+0000 7fa80be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c1e6040 con 0x7fa80c15f690 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.701+0000 7fa80be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c1e6320 con 0x7fa7ec0039c0 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.701+0000 7fa80be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c1e6600 con 0x7fa80c1e73c0 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15f690 0x7fa80c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15f690 0x7fa80c159870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47238/0 (socket says 172.21.15.26:47238) 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80b400640 1 -- 172.21.15.26:0/2815762418 learned_addr learned my addr 172.21.15.26:0/2815762418 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80a000640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa80c1e73c0 0x7fa80c15f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80aa00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7ec0039c0 0x7fa80c141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80b400640 1 -- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa80c1e73c0 msgr2=0x7fa80c15f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80b400640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa80c1e73c0 0x7fa80c15f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80b400640 1 -- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7ec0039c0 msgr2=0x7fa80c141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:23.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80b400640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7ec0039c0 0x7fa80c141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80b400640 1 -- 172.21.15.26:0/2815762418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c1e68e0 con 0x7fa80c15f690 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.702+0000 7fa80b400640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15f690 0x7fa80c159870 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0ce1a0 tx=0x7fa80400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.703+0000 7fa802a00640 1 -- 172.21.15.26:0/2815762418 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804016420 con 0x7fa80c15f690 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.703+0000 7fa802a00640 1 -- 172.21.15.26:0/2815762418 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8040165a0 con 0x7fa80c15f690 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.703+0000 7fa802a00640 1 -- 172.21.15.26:0/2815762418 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804016950 con 0x7fa80c15f690 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.703+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c1e6bc0 con 0x7fa80c15f690 2024-01-15T18:05:23.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.703+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c142330 con 0x7fa80c15f690 2024-01-15T18:05:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.704+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80c1e70e0 con 0x7fa80c15f690 2024-01-15T18:05:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.705+0000 7fa802a00640 1 -- 172.21.15.26:0/2815762418 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fa804016ab0 con 0x7fa80c15f690 2024-01-15T18:05:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.706+0000 7fa802a00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7e406be90 0x7fa7e406e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:23.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.706+0000 7fa802a00640 1 -- 172.21.15.26:0/2815762418 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fa804089460 con 0x7fa80c15f690 2024-01-15T18:05:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.707+0000 7fa80aa00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7e406be90 0x7fa7e406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:23.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.708+0000 7fa80aa00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7e406be90 0x7fa7e406e350 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc034000 tx=0x7fa7fc009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:23.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.710+0000 7fa802a00640 1 -- 172.21.15.26:0/2815762418 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa80c1e70e0 con 0x7fa80c15f690 2024-01-15T18:05:23.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:23.956+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41813", "target": ["mon-mgr", ""]}) v1 -- 0x7fa80c06cc40 con 0x7fa7e406be90 2024-01-15T18:05:24.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.687+0000 7fa802a00640 1 -- 172.21.15.26:0/2815762418 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa7fc03d040 con 0x7fa7e406be90 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.703+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7e406be90 msgr2=0x7fa7e406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.703+0000 7fa80be00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7e406be90 0x7fa7e406e350 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc034000 tx=0x7fa7fc009030 comp rx=0 tx=0).stop 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.703+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15f690 msgr2=0x7fa80c159870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.703+0000 7fa80be00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15f690 0x7fa80c159870 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0ce1a0 tx=0x7fa80400f520 comp rx=0 tx=0).stop 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.704+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 shutdown_connections 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.704+0000 7fa80be00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa80c1e73c0 0x7fa80c15f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.704+0000 7fa80be00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa7e406be90 0x7fa7e406e350 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.704+0000 7fa80be00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa80c15f690 0x7fa80c159870 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.704+0000 7fa80be00640 1 --2- 172.21.15.26:0/2815762418 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa7ec0039c0 0x7fa80c141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:24.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.704+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 >> 172.21.15.26:0/2815762418 conn(0x7fa80c06a800 msgr2=0x7fa80c0b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.704+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 shutdown_connections 2024-01-15T18:05:24.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:24.704+0000 7fa80be00640 1 -- 172.21.15.26:0/2815762418 wait complete. 2024-01-15T18:05:24.749 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41814 2024-01-15T18:05:25.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.042+0000 7f3725c00640 1 Processor -- start 2024-01-15T18:05:25.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.043+0000 7f3725c00640 1 -- start start 2024-01-15T18:05:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.044+0000 7f3725c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f37200cf9a0 0x7f3720070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.044+0000 7f3725c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3720070ea0 0x7f37200cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.044+0000 7f3725c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3720071690 con 0x7f372014b2c0 2024-01-15T18:05:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.044+0000 7f3725c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f372014b6a0 con 0x7f37200cf9a0 2024-01-15T18:05:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.044+0000 7f3725c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3720071970 con 0x7f3720070ea0 2024-01-15T18:05:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.045+0000 7f371ea00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f372014b2c0 0x7f37200cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53894/0 (socket says 172.21.15.26:53894) 2024-01-15T18:05:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.045+0000 7f371ea00640 1 -- 172.21.15.26:0/548214970 learned_addr learned my addr 172.21.15.26:0/548214970 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.045+0000 7f371e000640 1 --2- 172.21.15.26:0/548214970 >> v2:172.21.15.124:3300/0 conn(0x7f37200cf9a0 0x7f3720070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.045+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914918496 0 0) 0x7f3720071690 con 0x7f372014b2c0 2024-01-15T18:05:25.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.045+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3700003520 con 0x7f372014b2c0 2024-01-15T18:05:25.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.045+0000 7f371f400640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3720070ea0 0x7f37200cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2413184525 0 0) 0x7f3700003520 con 0x7f372014b2c0 2024-01-15T18:05:25.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3720070ea0 msgr2=0x7f37200cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3720070ea0 0x7f37200cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 >> v2:172.21.15.124:3300/0 conn(0x7f37200cf9a0 msgr2=0x7f3720070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 --2- 172.21.15.26:0/548214970 >> v2:172.21.15.124:3300/0 conn(0x7f37200cf9a0 0x7f3720070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720071c50 con 0x7f372014b2c0 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f3710001670 con 0x7f372014b2c0 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 >> v1:172.21.15.26:6789/0 conn(0x7f372014b2c0 legacy=0x7f37200cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3720070ea0 0x7f3700003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3700003ef0 0x7f37000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3700004d00 0x7f3700005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372014b6a0 con 0x7f3700003ef0 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3720071970 con 0x7f3700004d00 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.046+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370000b9d0 con 0x7f3720070ea0 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.047+0000 7f371e000640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3700003ef0 0x7f37000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.047+0000 7f371ea00640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3720070ea0 0x7f3700003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.047+0000 7f371e000640 1 -- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3720070ea0 msgr2=0x7f3700003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.048+0000 7f371e000640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3720070ea0 0x7f3700003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.048+0000 7f371e000640 1 -- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3700004d00 msgr2=0x7f3700005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.048+0000 7f371e000640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3700004d00 0x7f3700005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.048+0000 7f371e000640 1 -- 172.21.15.26:0/548214970 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720071c50 con 0x7f3700003ef0 2024-01-15T18:05:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.049+0000 7f371e000640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3700003ef0 0x7f37000047c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f370800ba30 tx=0x7f37000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3ef99d57f21d7b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.050+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708032680 con 0x7f3700003ef0 2024-01-15T18:05:25.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.050+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37080327e0 con 0x7f3700003ef0 2024-01-15T18:05:25.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.050+0000 7f371d600640 1 -- 172.21.15.26:0/548214970 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708032b70 con 0x7f3700003ef0 2024-01-15T18:05:25.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.051+0000 7f3725c00640 1 -- 172.21.15.26:0/548214970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3700003ef0 msgr2=0x7f37000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.051+0000 7f3725c00640 1 --2- 172.21.15.26:0/548214970 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3700003ef0 0x7f37000047c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f370800ba30 tx=0x7f37000057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.051+0000 7f371f400640 1 -- 172.21.15.26:0/548214970 reap_dead start 2024-01-15T18:05:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.052+0000 7f3725c00640 1 -- 172.21.15.26:0/548214970 shutdown_connections 2024-01-15T18:05:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.052+0000 7f3725c00640 1 -- 172.21.15.26:0/548214970 >> 172.21.15.26:0/548214970 conn(0x7f372006a800 msgr2=0x7f372006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.052+0000 7f3725c00640 1 -- 172.21.15.26:0/548214970 shutdown_connections 2024-01-15T18:05:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.052+0000 7f3725c00640 1 -- 172.21.15.26:0/548214970 wait complete. 2024-01-15T18:05:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.055+0000 7f3725c00640 1 Processor -- start 2024-01-15T18:05:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.056+0000 7f3725c00640 1 -- start start 2024-01-15T18:05:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.057+0000 7f3725c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3700003ef0 0x7f3720143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.057+0000 7f3725c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720142dc0 0x7f37200adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.057+0000 7f3725c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37200cd710 0x7f37200cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.057+0000 7f3725c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37201f6fc0 con 0x7f3720142dc0 2024-01-15T18:05:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.058+0000 7f3725c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37200ce0d0 con 0x7f3700003ef0 2024-01-15T18:05:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.058+0000 7f3725c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3720144530 con 0x7f37200cd710 2024-01-15T18:05:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.058+0000 7f371e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720142dc0 0x7f37200adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.058+0000 7f371e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720142dc0 0x7f37200adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47248/0 (socket says 172.21.15.26:47248) 2024-01-15T18:05:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.058+0000 7f371e000640 1 -- 172.21.15.26:0/3280882217 learned_addr learned my addr 172.21.15.26:0/3280882217 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.059+0000 7f371ea00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3700003ef0 0x7f3720143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.059+0000 7f371e000640 1 -- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37200cd710 msgr2=0x7f37200cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.059+0000 7f371e000640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37200cd710 0x7f37200cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.059+0000 7f371e000640 1 -- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3700003ef0 msgr2=0x7f3720143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.059+0000 7f371e000640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3700003ef0 0x7f3720143ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.059+0000 7f371e000640 1 -- 172.21.15.26:0/3280882217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3720144810 con 0x7f3720142dc0 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.059+0000 7f371e000640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720142dc0 0x7f37200adf40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f370802bf00 tx=0x7f3708004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.060+0000 7f3716a00640 1 -- 172.21.15.26:0/3280882217 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708004220 con 0x7f3720142dc0 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.060+0000 7f3716a00640 1 -- 172.21.15.26:0/3280882217 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37080043a0 con 0x7f3720142dc0 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.060+0000 7f3716a00640 1 -- 172.21.15.26:0/3280882217 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3708030a60 con 0x7f3720142dc0 2024-01-15T18:05:25.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.060+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3720144af0 con 0x7f3720142dc0 2024-01-15T18:05:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.061+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3720144220 con 0x7f3720142dc0 2024-01-15T18:05:25.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.063+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37201f7130 con 0x7f3720142dc0 2024-01-15T18:05:25.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.064+0000 7f3716a00640 1 -- 172.21.15.26:0/3280882217 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f3708030bc0 con 0x7f3720142dc0 2024-01-15T18:05:25.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.066+0000 7f3716a00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f36f806bec0 0x7f36f806e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:25.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.066+0000 7f3716a00640 1 -- 172.21.15.26:0/3280882217 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f37080ab1c0 con 0x7f3720142dc0 2024-01-15T18:05:25.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.073+0000 7f3716a00640 1 -- 172.21.15.26:0/3280882217 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37201f7130 con 0x7f3720142dc0 2024-01-15T18:05:25.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.077+0000 7f371ea00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f36f806bec0 0x7f36f806e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:25.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.084+0000 7f371ea00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f36f806bec0 0x7f36f806e380 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3710000c00 tx=0x7f3710009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:25.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.421+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41814", "target": ["mon-mgr", ""]}) v1 -- 0x7f3720071330 con 0x7f36f806bec0 2024-01-15T18:05:25.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.941+0000 7f3716a00640 1 -- 172.21.15.26:0/3280882217 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f37100109b0 con 0x7f36f806bec0 2024-01-15T18:05:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f36f806bec0 msgr2=0x7f36f806e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f36f806bec0 0x7f36f806e380 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3710000c00 tx=0x7f3710009040 comp rx=0 tx=0).stop 2024-01-15T18:05:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720142dc0 msgr2=0x7f37200adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720142dc0 0x7f37200adf40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f370802bf00 tx=0x7f3708004730 comp rx=0 tx=0).stop 2024-01-15T18:05:25.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 shutdown_connections 2024-01-15T18:05:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37200cd710 0x7f37200cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f36f806bec0 0x7f36f806e380 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3720142dc0 0x7f37200adf40 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 --2- 172.21.15.26:0/3280882217 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3700003ef0 0x7f3720143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 >> 172.21.15.26:0/3280882217 conn(0x7f372006a800 msgr2=0x7f37200cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 shutdown_connections 2024-01-15T18:05:25.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:25.944+0000 7f3725c00640 1 -- 172.21.15.26:0/3280882217 wait complete. 2024-01-15T18:05:25.969 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41815 2024-01-15T18:05:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.232+0000 7fe246000640 1 Processor -- start 2024-01-15T18:05:26.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.232+0000 7fe246000640 1 -- start start 2024-01-15T18:05:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.233+0000 7fe246000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe240070c00 0x7fe2400d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.233+0000 7fe246000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe2400c5f50 0x7fe2400c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.233+0000 7fe246000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2400cf430 con 0x7fe24015b130 2024-01-15T18:05:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.233+0000 7fe246000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2400cec60 con 0x7fe240070c00 2024-01-15T18:05:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.233+0000 7fe246000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2400d0890 con 0x7fe2400c5f50 2024-01-15T18:05:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.234+0000 7fe23f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe24015b130 0x7fe24015f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53896/0 (socket says 172.21.15.26:53896) 2024-01-15T18:05:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.234+0000 7fe23f400640 1 -- 172.21.15.26:0/3899299838 learned_addr learned my addr 172.21.15.26:0/3899299838 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.234+0000 7fe23ea00640 1 --2- 172.21.15.26:0/3899299838 >> v2:172.21.15.26:3300/0 conn(0x7fe240070c00 0x7fe2400d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.234+0000 7fe23e000640 1 -- 172.21.15.26:0/3899299838 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573532288 0 0) 0x7fe2400cf430 con 0x7fe24015b130 2024-01-15T18:05:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.234+0000 7fe23e000640 1 -- 172.21.15.26:0/3899299838 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe220003520 con 0x7fe24015b130 2024-01-15T18:05:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.234+0000 7fe23ea00640 1 -- 172.21.15.26:0/3899299838 >> v2:172.21.15.124:3300/0 conn(0x7fe2400c5f50 msgr2=0x7fe2400c63c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:05:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.234+0000 7fe23ea00640 1 --2- 172.21.15.26:0/3899299838 >> v2:172.21.15.124:3300/0 conn(0x7fe2400c5f50 0x7fe2400c63c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.234+0000 7fe23ea00640 1 -- 172.21.15.26:0/3899299838 >> v1:172.21.15.26:6789/0 conn(0x7fe24015b130 legacy=0x7fe24015f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.235+0000 7fe23ea00640 1 -- 172.21.15.26:0/3899299838 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400cee70 con 0x7fe240070c00 2024-01-15T18:05:26.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.236+0000 7fe23ea00640 1 --2- 172.21.15.26:0/3899299838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe240070c00 0x7fe2400d1330 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fe2400b39e0 tx=0x7fe22802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b50e5882adb590 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.236+0000 7fe23e000640 1 -- 172.21.15.26:0/3899299838 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228031640 con 0x7fe240070c00 2024-01-15T18:05:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.236+0000 7fe23e000640 1 -- 172.21.15.26:0/3899299838 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2280317a0 con 0x7fe240070c00 2024-01-15T18:05:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.236+0000 7fe23e000640 1 -- 172.21.15.26:0/3899299838 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228031b10 con 0x7fe240070c00 2024-01-15T18:05:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.237+0000 7fe246000640 1 -- 172.21.15.26:0/3899299838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe240070c00 msgr2=0x7fe2400d1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.237+0000 7fe246000640 1 --2- 172.21.15.26:0/3899299838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe240070c00 0x7fe2400d1330 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fe2400b39e0 tx=0x7fe22802b8f0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.237+0000 7fe246000640 1 -- 172.21.15.26:0/3899299838 shutdown_connections 2024-01-15T18:05:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.237+0000 7fe246000640 1 --2- 172.21.15.26:0/3899299838 >> v2:172.21.15.124:3300/0 conn(0x7fe2400c5f50 0x7fe2400c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.237+0000 7fe246000640 1 --2- 172.21.15.26:0/3899299838 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe240070c00 0x7fe2400d1330 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.237+0000 7fe246000640 1 -- 172.21.15.26:0/3899299838 >> 172.21.15.26:0/3899299838 conn(0x7fe2400b2ce0 msgr2=0x7fe2400b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:26.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.237+0000 7fe246000640 1 -- 172.21.15.26:0/3899299838 shutdown_connections 2024-01-15T18:05:26.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.237+0000 7fe246000640 1 -- 172.21.15.26:0/3899299838 wait complete. 2024-01-15T18:05:26.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.239+0000 7fe246000640 1 Processor -- start 2024-01-15T18:05:26.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.239+0000 7fe246000640 1 -- start start 2024-01-15T18:05:26.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe246000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe240070c00 0x7fe24015e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe246000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2400c5f50 0x7fe24014a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe246000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe24015b130 0x7fe24015b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe246000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400ccf20 con 0x7fe24015b130 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe246000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2400cd7c0 con 0x7fe2400c5f50 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe246000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24014a8a0 con 0x7fe240070c00 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe23fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe24015b130 0x7fe24015b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe23fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe24015b130 0x7fe24015b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47264/0 (socket says 172.21.15.26:47264) 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe23fe00640 1 -- 172.21.15.26:0/2395243827 learned_addr learned my addr 172.21.15.26:0/2395243827 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe23f400640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe240070c00 0x7fe24015e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.241+0000 7fe23ea00640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2400c5f50 0x7fe24014a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe23fe00640 1 -- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe240070c00 msgr2=0x7fe24015e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe23fe00640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe240070c00 0x7fe24015e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe23fe00640 1 -- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2400c5f50 msgr2=0x7fe24014a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe23fe00640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2400c5f50 0x7fe24014a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe23fe00640 1 -- 172.21.15.26:0/2395243827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe228009400 con 0x7fe24015b130 2024-01-15T18:05:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe23ea00640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2400c5f50 0x7fe24014a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:05:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe23fe00640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe24015b130 0x7fe24015b630 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe23800b9f0 tx=0x7fe23800beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe237400640 1 -- 172.21.15.26:0/2395243827 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800cbd0 con 0x7fe24015b130 2024-01-15T18:05:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe237400640 1 -- 172.21.15.26:0/2395243827 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23800cd30 con 0x7fe24015b130 2024-01-15T18:05:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe237400640 1 -- 172.21.15.26:0/2395243827 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23800cea0 con 0x7fe24015b130 2024-01-15T18:05:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.242+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24015bb70 con 0x7fe24015b130 2024-01-15T18:05:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.243+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2400cd170 con 0x7fe24015b130 2024-01-15T18:05:26.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.244+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2400675d0 con 0x7fe24015b130 2024-01-15T18:05:26.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.245+0000 7fe237400640 1 -- 172.21.15.26:0/2395243827 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fe23801d3b0 con 0x7fe24015b130 2024-01-15T18:05:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.247+0000 7fe237400640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe21806be90 0x7fe21806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.247+0000 7fe237400640 1 -- 172.21.15.26:0/2395243827 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fe238088740 con 0x7fe24015b130 2024-01-15T18:05:26.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.250+0000 7fe23f400640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe21806be90 0x7fe21806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.253+0000 7fe237400640 1 -- 172.21.15.26:0/2395243827 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe238054e50 con 0x7fe24015b130 2024-01-15T18:05:26.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.259+0000 7fe23f400640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe21806be90 0x7fe21806e350 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe230005880 tx=0x7fe230009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:26.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.634+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41815", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2400d0e10 con 0x7fe21806be90 2024-01-15T18:05:26.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.807+0000 7fe237400640 1 -- 172.21.15.26:0/2395243827 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe23000f820 con 0x7fe21806be90 2024-01-15T18:05:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe21806be90 msgr2=0x7fe21806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe21806be90 0x7fe21806e350 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe230005880 tx=0x7fe230009040 comp rx=0 tx=0).stop 2024-01-15T18:05:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe24015b130 msgr2=0x7fe24015b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe24015b130 0x7fe24015b630 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe23800b9f0 tx=0x7fe23800beb0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 shutdown_connections 2024-01-15T18:05:26.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe21806be90 0x7fe21806e350 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe24015b130 0x7fe24015b630 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2400c5f50 0x7fe24014a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 --2- 172.21.15.26:0/2395243827 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe240070c00 0x7fe24015e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:26.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 >> 172.21.15.26:0/2395243827 conn(0x7fe2400b2ce0 msgr2=0x7fe2400d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:26.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 shutdown_connections 2024-01-15T18:05:26.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:26.810+0000 7fe246000640 1 -- 172.21.15.26:0/2395243827 wait complete. 2024-01-15T18:05:26.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41816 2024-01-15T18:05:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.024+0000 7f3e5f200640 1 Processor -- start 2024-01-15T18:05:27.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.025+0000 7f3e5f200640 1 -- start start 2024-01-15T18:05:27.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.026+0000 7f3e5f200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3e600d0100 0x7f3e600b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.026+0000 7f3e5f200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3e600d0f40 0x7f3e600d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.026+0000 7f3e5f200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e600d0a10 con 0x7f3e60160920 2024-01-15T18:05:27.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.026+0000 7f3e5f200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e600cf080 con 0x7f3e600d0f40 2024-01-15T18:05:27.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.026+0000 7f3e5f200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e600c6be0 con 0x7f3e600d0100 2024-01-15T18:05:27.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e5de00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3e60160920 0x7f3e600cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53898/0 (socket says 172.21.15.26:53898) 2024-01-15T18:05:27.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e5de00640 1 -- 172.21.15.26:0/435221498 learned_addr learned my addr 172.21.15.26:0/435221498 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:27.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e5e800640 1 --2- 172.21.15.26:0/435221498 >> v2:172.21.15.26:3300/0 conn(0x7f3e600d0f40 0x7f3e600d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e5d400640 1 -- 172.21.15.26:0/435221498 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625625778 0 0) 0x7f3e600d0a10 con 0x7f3e60160920 2024-01-15T18:05:27.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e57e00640 1 --2- 172.21.15.26:0/435221498 >> v2:172.21.15.124:3300/0 conn(0x7f3e600d0100 0x7f3e600b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e5d400640 1 -- 172.21.15.26:0/435221498 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e40003520 con 0x7f3e60160920 2024-01-15T18:05:27.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e5e800640 1 -- 172.21.15.26:0/435221498 >> v2:172.21.15.124:3300/0 conn(0x7f3e600d0100 msgr2=0x7f3e600b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:27.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e5e800640 1 --2- 172.21.15.26:0/435221498 >> v2:172.21.15.124:3300/0 conn(0x7f3e600d0100 0x7f3e600b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e5e800640 1 -- 172.21.15.26:0/435221498 >> v1:172.21.15.26:6789/0 conn(0x7f3e60160920 legacy=0x7f3e600cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e5e800640 1 -- 172.21.15.26:0/435221498 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e600ce7e0 con 0x7f3e600d0f40 2024-01-15T18:05:27.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.027+0000 7f3e57e00640 1 --2- 172.21.15.26:0/435221498 >> v2:172.21.15.124:3300/0 conn(0x7f3e600d0100 0x7f3e600b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:27.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.028+0000 7f3e5e800640 1 --2- 172.21.15.26:0/435221498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0f40 0x7f3e600d1390 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f3e6008c1f0 tx=0x7f3e5802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4195ff10b99591 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.028+0000 7f3e5d400640 1 -- 172.21.15.26:0/435221498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5802cc70 con 0x7f3e600d0f40 2024-01-15T18:05:27.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5d400640 1 -- 172.21.15.26:0/435221498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5802cdd0 con 0x7f3e600d0f40 2024-01-15T18:05:27.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5d400640 1 -- 172.21.15.26:0/435221498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e58034660 con 0x7f3e600d0f40 2024-01-15T18:05:27.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5f200640 1 -- 172.21.15.26:0/435221498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0f40 msgr2=0x7f3e600d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5f200640 1 --2- 172.21.15.26:0/435221498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0f40 0x7f3e600d1390 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f3e6008c1f0 tx=0x7f3e5802bec0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5f200640 1 -- 172.21.15.26:0/435221498 shutdown_connections 2024-01-15T18:05:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5f200640 1 --2- 172.21.15.26:0/435221498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0f40 0x7f3e600d1390 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5f200640 1 --2- 172.21.15.26:0/435221498 >> v2:172.21.15.124:3300/0 conn(0x7f3e600d0100 0x7f3e600b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5f200640 1 -- 172.21.15.26:0/435221498 >> 172.21.15.26:0/435221498 conn(0x7f3e600d3090 msgr2=0x7f3e600d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5f200640 1 -- 172.21.15.26:0/435221498 shutdown_connections 2024-01-15T18:05:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.029+0000 7f3e5f200640 1 -- 172.21.15.26:0/435221498 wait complete. 2024-01-15T18:05:27.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.031+0000 7f3e5f200640 1 Processor -- start 2024-01-15T18:05:27.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.031+0000 7f3e5f200640 1 -- start start 2024-01-15T18:05:27.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0100 0x7f3e6014e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e600d0f40 0x7f3e6014c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e60160920 0x7f3e6014cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6014de60 con 0x7f3e600d0100 2024-01-15T18:05:27.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5f200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6014d250 con 0x7f3e600d0f40 2024-01-15T18:05:27.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6014d530 con 0x7f3e60160920 2024-01-15T18:05:27.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0100 0x7f3e6014e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0100 0x7f3e6014e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47294/0 (socket says 172.21.15.26:47294) 2024-01-15T18:05:27.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5de00640 1 -- 172.21.15.26:0/3476945026 learned_addr learned my addr 172.21.15.26:0/3476945026 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e57e00640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e600d0f40 0x7f3e6014c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5de00640 1 -- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e60160920 msgr2=0x7f3e6014cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5de00640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e60160920 0x7f3e6014cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5de00640 1 -- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e600d0f40 msgr2=0x7f3e6014c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.032+0000 7f3e5de00640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e600d0f40 0x7f3e6014c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.033+0000 7f3e5de00640 1 -- 172.21.15.26:0/3476945026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e58009a70 con 0x7f3e600d0100 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.033+0000 7f3e57e00640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e600d0f40 0x7f3e6014c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.033+0000 7f3e5de00640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0100 0x7f3e6014e900 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000c410 tx=0x7f3e5000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.033+0000 7f3e56000640 1 -- 172.21.15.26:0/3476945026 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e50005ce0 con 0x7f3e600d0100 2024-01-15T18:05:27.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.033+0000 7f3e56000640 1 -- 172.21.15.26:0/3476945026 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e50005e40 con 0x7f3e600d0100 2024-01-15T18:05:27.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.033+0000 7f3e56000640 1 -- 172.21.15.26:0/3476945026 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e500072f0 con 0x7f3e600d0100 2024-01-15T18:05:27.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.033+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6014d810 con 0x7f3e600d0100 2024-01-15T18:05:27.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.033+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6014e2f0 con 0x7f3e600d0100 2024-01-15T18:05:27.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.034+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e600b5ac0 con 0x7f3e600d0100 2024-01-15T18:05:27.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.035+0000 7f3e56000640 1 -- 172.21.15.26:0/3476945026 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f3e50007510 con 0x7f3e600d0100 2024-01-15T18:05:27.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.036+0000 7f3e56000640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e3806be90 0x7f3e3806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.036+0000 7f3e56000640 1 -- 172.21.15.26:0/3476945026 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f3e50087ec0 con 0x7f3e600d0100 2024-01-15T18:05:27.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.037+0000 7f3e57e00640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e3806be90 0x7f3e3806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.038+0000 7f3e57e00640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e3806be90 0x7f3e3806e350 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3e48006fd0 tx=0x7f3e48008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:27.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.040+0000 7f3e56000640 1 -- 172.21.15.26:0/3476945026 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e500545e0 con 0x7f3e600d0100 2024-01-15T18:05:27.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.263+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41816", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e600d3f00 con 0x7f3e3806be90 2024-01-15T18:05:27.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.613+0000 7f3e56000640 1 -- 172.21.15.26:0/3476945026 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3e600d3f00 con 0x7f3e3806be90 2024-01-15T18:05:27.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.618+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e3806be90 msgr2=0x7f3e3806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.618+0000 7f3e5f200640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e3806be90 0x7f3e3806e350 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3e48006fd0 tx=0x7f3e48008040 comp rx=0 tx=0).stop 2024-01-15T18:05:27.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.618+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0100 msgr2=0x7f3e6014e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:27.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.618+0000 7f3e5f200640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0100 0x7f3e6014e900 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000c410 tx=0x7f3e5000c8d0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.618+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 shutdown_connections 2024-01-15T18:05:27.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.618+0000 7f3e5f200640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e60160920 0x7f3e6014cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.618+0000 7f3e5f200640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e3806be90 0x7f3e3806e350 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.618+0000 7f3e5f200640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e600d0f40 0x7f3e6014c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.619+0000 7f3e5f200640 1 --2- 172.21.15.26:0/3476945026 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e600d0100 0x7f3e6014e900 secure :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000c410 tx=0x7f3e5000c8d0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.619+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 >> 172.21.15.26:0/3476945026 conn(0x7f3e600d3090 msgr2=0x7f3e600cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:27.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.620+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 shutdown_connections 2024-01-15T18:05:27.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.620+0000 7f3e5f200640 1 -- 172.21.15.26:0/3476945026 wait complete. 2024-01-15T18:05:27.659 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41817 2024-01-15T18:05:27.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.989+0000 7f21b1800640 1 Processor -- start 2024-01-15T18:05:27.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.989+0000 7f21b1800640 1 -- start start 2024-01-15T18:05:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.991+0000 7f21b1800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21ac143ae0 0x7f21ac0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.991+0000 7f21b1800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21ac0cf7d0 0x7f21ac15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.991+0000 7f21b1800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f21ac15e1e0 0x7f21ac15a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.991+0000 7f21b1800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac0cfc30 con 0x7f21ac143ae0 2024-01-15T18:05:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.991+0000 7f21b1800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac0b1ee0 con 0x7f21ac15e1e0 2024-01-15T18:05:27.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.991+0000 7f21b1800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0b2780 con 0x7f21ac0cf7d0 2024-01-15T18:05:27.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.993+0000 7f21ab400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f21ac15e1e0 0x7f21ac15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.993+0000 7f21ab400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f21ac15e1e0 0x7f21ac15a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:60646/0 (socket says 172.21.15.26:60646) 2024-01-15T18:05:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.993+0000 7f21ab400640 1 -- 172.21.15.26:0/625235759 learned_addr learned my addr 172.21.15.26:0/625235759 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.992+0000 7f21aaa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f21ac143ae0 0x7f21ac0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.994+0000 7f21aaa00640 1 -- 172.21.15.26:0/625235759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21ac0cf7d0 msgr2=0x7f21ac15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.994+0000 7f21aaa00640 1 --2- 172.21.15.26:0/625235759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21ac0cf7d0 0x7f21ac15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.994+0000 7f21aaa00640 1 -- 172.21.15.26:0/625235759 >> v2:172.21.15.124:3300/0 conn(0x7f21ac15e1e0 msgr2=0x7f21ac15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.994+0000 7f21aaa00640 1 --2- 172.21.15.26:0/625235759 >> v2:172.21.15.124:3300/0 conn(0x7f21ac15e1e0 0x7f21ac15a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:27.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.994+0000 7f21aaa00640 1 -- 172.21.15.26:0/625235759 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac15e990 con 0x7f21ac143ae0 2024-01-15T18:05:27.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.994+0000 7f21ab400640 1 --2- 172.21.15.26:0/625235759 >> v2:172.21.15.124:3300/0 conn(0x7f21ac15e1e0 0x7f21ac15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.996+0000 7f21aaa00640 1 --2- 172.21.15.26:0/625235759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac143ae0 0x7f21ac0cf290 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f21ac072160 tx=0x7f219402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b89067e542c232 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.997+0000 7f21a9600640 1 -- 172.21.15.26:0/625235759 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194031780 con 0x7f21ac143ae0 2024-01-15T18:05:28.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.998+0000 7f21a9600640 1 -- 172.21.15.26:0/625235759 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21940318e0 con 0x7f21ac143ae0 2024-01-15T18:05:28.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.998+0000 7f21a9600640 1 -- 172.21.15.26:0/625235759 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194031c70 con 0x7f21ac143ae0 2024-01-15T18:05:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.999+0000 7f21b1800640 1 -- 172.21.15.26:0/625235759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac143ae0 msgr2=0x7f21ac0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:27.999+0000 7f21b1800640 1 --2- 172.21.15.26:0/625235759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac143ae0 0x7f21ac0cf290 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f21ac072160 tx=0x7f219402b820 comp rx=0 tx=0).stop 2024-01-15T18:05:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.000+0000 7f21b1800640 1 -- 172.21.15.26:0/625235759 shutdown_connections 2024-01-15T18:05:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.000+0000 7f21b1800640 1 --2- 172.21.15.26:0/625235759 >> v2:172.21.15.124:3300/0 conn(0x7f21ac15e1e0 0x7f21ac15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.000+0000 7f21b1800640 1 --2- 172.21.15.26:0/625235759 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21ac0cf7d0 0x7f21ac15dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.000+0000 7f21b1800640 1 --2- 172.21.15.26:0/625235759 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac143ae0 0x7f21ac0cf290 secure :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f21ac072160 tx=0x7f219402b820 comp rx=0 tx=0).stop 2024-01-15T18:05:28.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.000+0000 7f21b1800640 1 -- 172.21.15.26:0/625235759 >> 172.21.15.26:0/625235759 conn(0x7f21ac06a800 msgr2=0x7f21ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.002+0000 7f21b1800640 1 -- 172.21.15.26:0/625235759 shutdown_connections 2024-01-15T18:05:28.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.003+0000 7f21b1800640 1 -- 172.21.15.26:0/625235759 wait complete. 2024-01-15T18:05:28.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.007+0000 7f21b1800640 1 Processor -- start 2024-01-15T18:05:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.007+0000 7f21b1800640 1 -- start start 2024-01-15T18:05:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.009+0000 7f21b1800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21ac0cf7d0 0x7f21ac1429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.009+0000 7f21b1800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac15e1e0 0x7f21ac06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.009+0000 7f21b1800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21ac0a84c0 0x7f21ac06cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.009+0000 7f21b1800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac142ef0 con 0x7f21ac15e1e0 2024-01-15T18:05:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.009+0000 7f21b1800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac15fef0 con 0x7f21ac0cf7d0 2024-01-15T18:05:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.009+0000 7f21b1800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac1601d0 con 0x7f21ac0a84c0 2024-01-15T18:05:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.009+0000 7f21aa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac15e1e0 0x7f21ac06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.009+0000 7f21aa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac15e1e0 0x7f21ac06ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47306/0 (socket says 172.21.15.26:47306) 2024-01-15T18:05:28.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.010+0000 7f21aa000640 1 -- 172.21.15.26:0/3994824638 learned_addr learned my addr 172.21.15.26:0/3994824638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.010+0000 7f21aaa00640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21ac0cf7d0 0x7f21ac1429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.010+0000 7f21aa000640 1 -- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21ac0a84c0 msgr2=0x7f21ac06cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.010+0000 7f21aa000640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21ac0a84c0 0x7f21ac06cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.010+0000 7f21aa000640 1 -- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21ac0cf7d0 msgr2=0x7f21ac1429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.010+0000 7f21aa000640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21ac0cf7d0 0x7f21ac1429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.010+0000 7f21aa000640 1 -- 172.21.15.26:0/3994824638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a4009b30 con 0x7f21ac15e1e0 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.010+0000 7f21aaa00640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21ac0cf7d0 0x7f21ac1429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.011+0000 7f21aa000640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac15e1e0 0x7f21ac06ca10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f219c00b420 tx=0x7f219c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.011+0000 7f21a2a00640 1 -- 172.21.15.26:0/3994824638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c011830 con 0x7f21ac15e1e0 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.011+0000 7f21a2a00640 1 -- 172.21.15.26:0/3994824638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219c011990 con 0x7f21ac15e1e0 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.011+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2194009400 con 0x7f21ac15e1e0 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.011+0000 7f21a2a00640 1 -- 172.21.15.26:0/3994824638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219c011d40 con 0x7f21ac15e1e0 2024-01-15T18:05:28.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.011+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac160790 con 0x7f21ac15e1e0 2024-01-15T18:05:28.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.013+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac0b2bf0 con 0x7f21ac15e1e0 2024-01-15T18:05:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.014+0000 7f21a2a00640 1 -- 172.21.15.26:0/3994824638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f219c01b460 con 0x7f21ac15e1e0 2024-01-15T18:05:28.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.015+0000 7f21a2a00640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f218406be90 0x7f218406e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:28.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.015+0000 7f21a2a00640 1 -- 172.21.15.26:0/3994824638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f219c088610 con 0x7f21ac15e1e0 2024-01-15T18:05:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.020+0000 7f21aaa00640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f218406be90 0x7f218406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.021+0000 7f21a2a00640 1 -- 172.21.15.26:0/3994824638 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21ac0b2bf0 con 0x7f21ac15e1e0 2024-01-15T18:05:28.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.029+0000 7f21aaa00640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f218406be90 0x7f218406e350 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2194036000 tx=0x7f219403d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.382+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41817", "target": ["mon-mgr", ""]}) v1 -- 0x7f21ac160340 con 0x7f218406be90 2024-01-15T18:05:28.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.804+0000 7f21a2a00640 1 -- 172.21.15.26:0/3994824638 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f21ac160340 con 0x7f218406be90 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f218406be90 msgr2=0x7f218406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f218406be90 0x7f218406e350 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2194036000 tx=0x7f219403d040 comp rx=0 tx=0).stop 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac15e1e0 msgr2=0x7f21ac06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac15e1e0 0x7f21ac06ca10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f219c00b420 tx=0x7f219c00b8e0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 shutdown_connections 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f21ac0a84c0 0x7f21ac06cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f218406be90 0x7f218406e350 secure :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2194036000 tx=0x7f219403d040 comp rx=0 tx=0).stop 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f21ac15e1e0 0x7f21ac06ca10 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 --2- 172.21.15.26:0/3994824638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f21ac0cf7d0 0x7f21ac1429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.807+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 >> 172.21.15.26:0/3994824638 conn(0x7f21ac06a800 msgr2=0x7f21ac0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:28.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.808+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 shutdown_connections 2024-01-15T18:05:28.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:28.808+0000 7f21b1800640 1 -- 172.21.15.26:0/3994824638 wait complete. 2024-01-15T18:05:28.842 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41818 2024-01-15T18:05:29.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:05:29.109 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:05:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.116+0000 7ff0f8a00640 1 Processor -- start 2024-01-15T18:05:29.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.117+0000 7ff0f8a00640 1 -- start start 2024-01-15T18:05:29.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.118+0000 7ff0f8a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff0f014b2c0 0x7ff0f00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.118+0000 7ff0f8a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff0f0071070 0x7ff0f00cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.118+0000 7ff0f8a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f00cfbf0 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.118+0000 7ff0f8a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f014b6a0 con 0x7ff0f0071070 2024-01-15T18:05:29.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.118+0000 7ff0f8a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f0071690 con 0x7ff0f014b2c0 2024-01-15T18:05:29.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.119+0000 7ff0f6a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff0f0071070 0x7ff0f00cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.119+0000 7ff0f5600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff0f00cf7d0 0x7ff0f0070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53914/0 (socket says 172.21.15.26:53914) 2024-01-15T18:05:29.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.119+0000 7ff0f5600640 1 -- 172.21.15.26:0/108672290 learned_addr learned my addr 172.21.15.26:0/108672290 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:29.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.120+0000 7ff0f6000640 1 --2- 172.21.15.26:0/108672290 >> v2:172.21.15.124:3300/0 conn(0x7ff0f014b2c0 0x7ff0f00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.120+0000 7ff0f4c00640 1 -- 172.21.15.26:0/108672290 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862574054 0 0) 0x7ff0f00cfbf0 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.120+0000 7ff0f4c00640 1 -- 172.21.15.26:0/108672290 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0d0003520 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.120+0000 7ff0f6a00640 1 -- 172.21.15.26:0/108672290 >> v2:172.21.15.124:3300/0 conn(0x7ff0f014b2c0 msgr2=0x7ff0f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.120+0000 7ff0f6a00640 1 --2- 172.21.15.26:0/108672290 >> v2:172.21.15.124:3300/0 conn(0x7ff0f014b2c0 0x7ff0f00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.120+0000 7ff0f6a00640 1 -- 172.21.15.26:0/108672290 >> v1:172.21.15.26:6789/0 conn(0x7ff0f00cf7d0 legacy=0x7ff0f0070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.120+0000 7ff0f6a00640 1 -- 172.21.15.26:0/108672290 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f0071970 con 0x7ff0f0071070 2024-01-15T18:05:29.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.122+0000 7ff0f6a00640 1 --2- 172.21.15.26:0/108672290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f0071070 0x7ff0f00cc160 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff0f006e650 tx=0x7ff0e802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b507698120dde0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.122+0000 7ff0f4c00640 1 -- 172.21.15.26:0/108672290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e802cc70 con 0x7ff0f0071070 2024-01-15T18:05:29.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.123+0000 7ff0f4c00640 1 -- 172.21.15.26:0/108672290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e802cdd0 con 0x7ff0f0071070 2024-01-15T18:05:29.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.123+0000 7ff0f4c00640 1 -- 172.21.15.26:0/108672290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8034660 con 0x7ff0f0071070 2024-01-15T18:05:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.124+0000 7ff0f8a00640 1 -- 172.21.15.26:0/108672290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f0071070 msgr2=0x7ff0f00cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.124+0000 7ff0f8a00640 1 --2- 172.21.15.26:0/108672290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f0071070 0x7ff0f00cc160 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7ff0f006e650 tx=0x7ff0e802bec0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.124+0000 7ff0f8a00640 1 -- 172.21.15.26:0/108672290 shutdown_connections 2024-01-15T18:05:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.124+0000 7ff0f8a00640 1 --2- 172.21.15.26:0/108672290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f0071070 0x7ff0f00cc160 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.124+0000 7ff0f8a00640 1 --2- 172.21.15.26:0/108672290 >> v2:172.21.15.124:3300/0 conn(0x7ff0f014b2c0 0x7ff0f00cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.124+0000 7ff0f8a00640 1 -- 172.21.15.26:0/108672290 >> 172.21.15.26:0/108672290 conn(0x7ff0f006a800 msgr2=0x7ff0f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.124+0000 7ff0f8a00640 1 -- 172.21.15.26:0/108672290 shutdown_connections 2024-01-15T18:05:29.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.124+0000 7ff0f8a00640 1 -- 172.21.15.26:0/108672290 wait complete. 2024-01-15T18:05:29.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.126+0000 7ff0f8a00640 1 Processor -- start 2024-01-15T18:05:29.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.126+0000 7ff0f8a00640 1 -- start start 2024-01-15T18:05:29.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.128+0000 7ff0f8a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0f0071070 0x7ff0f015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.128+0000 7ff0f8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f00cf7d0 0x7ff0f01f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.128+0000 7ff0f8a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0f014b2c0 0x7ff0f015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.128+0000 7ff0f8a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f015e7f0 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.128+0000 7ff0f8a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f01f8380 con 0x7ff0f0071070 2024-01-15T18:05:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.128+0000 7ff0f8a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f015de90 con 0x7ff0f014b2c0 2024-01-15T18:05:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.128+0000 7ff0f5600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f00cf7d0 0x7ff0f01f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.129+0000 7ff0f5600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f00cf7d0 0x7ff0f01f8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47318/0 (socket says 172.21.15.26:47318) 2024-01-15T18:05:29.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.129+0000 7ff0f5600640 1 -- 172.21.15.26:0/307238063 learned_addr learned my addr 172.21.15.26:0/307238063 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.129+0000 7ff0f6a00640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0f014b2c0 0x7ff0f015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.129+0000 7ff0f5600640 1 -- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0f014b2c0 msgr2=0x7ff0f015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.129+0000 7ff0f5600640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0f014b2c0 0x7ff0f015ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.129+0000 7ff0f5600640 1 -- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0f0071070 msgr2=0x7ff0f015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.129+0000 7ff0f5600640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0f0071070 0x7ff0f015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.129+0000 7ff0f5600640 1 -- 172.21.15.26:0/307238063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e8009a70 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.130+0000 7ff0f5600640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f00cf7d0 0x7ff0f01f8500 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c500 tx=0x7ff0d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.131+0000 7ff0ee000640 1 -- 172.21.15.26:0/307238063 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8005ce0 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.131+0000 7ff0ee000640 1 -- 172.21.15.26:0/307238063 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8005e40 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.131+0000 7ff0ee000640 1 -- 172.21.15.26:0/307238063 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8007330 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.131+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f01f7dc0 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.131+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f015e130 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.133+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f015ea60 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.134+0000 7ff0ee000640 1 -- 172.21.15.26:0/307238063 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7ff0d8007550 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.135+0000 7ff0ee000640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff0c806bf60 0x7ff0c806e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:29.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.135+0000 7ff0ee000640 1 -- 172.21.15.26:0/307238063 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7ff0d8087f00 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.142+0000 7ff0ee000640 1 -- 172.21.15.26:0/307238063 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0d8054620 con 0x7ff0f00cf7d0 2024-01-15T18:05:29.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.150+0000 7ff0f6000640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff0c806bf60 0x7ff0c806e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:29.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.156+0000 7ff0f6000640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff0c806bf60 0x7ff0c806e420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0006fd0 tx=0x7ff0e0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:29.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.535+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41818", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0f00cbc40 con 0x7ff0c806bf60 2024-01-15T18:05:29.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.921+0000 7ff0ee000640 1 -- 172.21.15.26:0/307238063 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff0e000be20 con 0x7ff0c806bf60 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff0c806bf60 msgr2=0x7ff0c806e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff0c806bf60 0x7ff0c806e420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0006fd0 tx=0x7ff0e0008040 comp rx=0 tx=0).stop 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f00cf7d0 msgr2=0x7ff0f01f8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f00cf7d0 0x7ff0f01f8500 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c500 tx=0x7ff0d800c9c0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 shutdown_connections 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff0f014b2c0 0x7ff0f015ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff0c806bf60 0x7ff0c806e420 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff0f00cf7d0 0x7ff0f01f8500 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 --2- 172.21.15.26:0/307238063 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff0f0071070 0x7ff0f015c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:29.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 >> 172.21.15.26:0/307238063 conn(0x7ff0f006a800 msgr2=0x7ff0f00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 shutdown_connections 2024-01-15T18:05:29.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:29.924+0000 7ff0f8a00640 1 -- 172.21.15.26:0/307238063 wait complete. 2024-01-15T18:05:29.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41819 2024-01-15T18:05:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.166+0000 7ff779a00640 1 Processor -- start 2024-01-15T18:05:30.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.166+0000 7ff779a00640 1 -- start start 2024-01-15T18:05:30.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.168+0000 7ff779a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff77415afb0 0x7ff77413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.168+0000 7ff779a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413fae0 0x7ff7740b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.168+0000 7ff779a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77413ff30 con 0x7ff77413cfa0 2024-01-15T18:05:30.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.168+0000 7ff779a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7740cdc40 con 0x7ff77415afb0 2024-01-15T18:05:30.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.168+0000 7ff779a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740cdec0 con 0x7ff77413fae0 2024-01-15T18:05:30.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.168+0000 7ff76be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff77415afb0 0x7ff77413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.168+0000 7ff76be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff77415afb0 0x7ff77413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47332/0 (socket says 172.21.15.26:47332) 2024-01-15T18:05:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.168+0000 7ff76be00640 1 -- 172.21.15.26:0/4276214678 learned_addr learned my addr 172.21.15.26:0/4276214678 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.169+0000 7ff779000640 1 --2- 172.21.15.26:0/4276214678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413fae0 0x7ff7740b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.169+0000 7ff76be00640 1 -- 172.21.15.26:0/4276214678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413fae0 msgr2=0x7ff7740b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.169+0000 7ff76be00640 1 --2- 172.21.15.26:0/4276214678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413fae0 0x7ff7740b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.169+0000 7ff76be00640 1 -- 172.21.15.26:0/4276214678 >> v1:172.21.15.124:6789/0 conn(0x7ff77413cfa0 legacy=0x7ff7740cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:30.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.169+0000 7ff76be00640 1 -- 172.21.15.26:0/4276214678 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7740ce1a0 con 0x7ff77415afb0 2024-01-15T18:05:30.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.170+0000 7ff76be00640 1 --2- 172.21.15.26:0/4276214678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 0x7ff77413f5a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009870 tx=0x7ff75c02da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416ef6d8b2656dcc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.170+0000 7ff773400640 1 -- 172.21.15.26:0/4276214678 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c02f830 con 0x7ff77415afb0 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.170+0000 7ff773400640 1 -- 172.21.15.26:0/4276214678 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c02f990 con 0x7ff77415afb0 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.170+0000 7ff773400640 1 -- 172.21.15.26:0/4276214678 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c02fd20 con 0x7ff77415afb0 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.170+0000 7ff779a00640 1 -- 172.21.15.26:0/4276214678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 msgr2=0x7ff77413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.170+0000 7ff779a00640 1 --2- 172.21.15.26:0/4276214678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 0x7ff77413f5a0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009870 tx=0x7ff75c02da70 comp rx=0 tx=0).stop 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.171+0000 7ff779a00640 1 -- 172.21.15.26:0/4276214678 shutdown_connections 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.171+0000 7ff779a00640 1 --2- 172.21.15.26:0/4276214678 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413fae0 0x7ff7740b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.171+0000 7ff779a00640 1 --2- 172.21.15.26:0/4276214678 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 0x7ff77413f5a0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.171+0000 7ff779a00640 1 -- 172.21.15.26:0/4276214678 >> 172.21.15.26:0/4276214678 conn(0x7ff77406a800 msgr2=0x7ff77406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.171+0000 7ff779a00640 1 -- 172.21.15.26:0/4276214678 shutdown_connections 2024-01-15T18:05:30.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.171+0000 7ff779a00640 1 -- 172.21.15.26:0/4276214678 wait complete. 2024-01-15T18:05:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.172+0000 7ff779a00640 1 Processor -- start 2024-01-15T18:05:30.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.172+0000 7ff779a00640 1 -- start start 2024-01-15T18:05:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.173+0000 7ff779a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413cfa0 0x7ff774159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.173+0000 7ff779a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77413fae0 0x7ff774153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.173+0000 7ff779a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 0x7ff77406cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.173+0000 7ff779a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741f7590 con 0x7ff77415afb0 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.173+0000 7ff779a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741f6910 con 0x7ff77413fae0 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.173+0000 7ff779a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741f6bf0 con 0x7ff77413cfa0 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff779000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 0x7ff77406cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff779000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 0x7ff77406cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47338/0 (socket says 172.21.15.26:47338) 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff779000640 1 -- 172.21.15.26:0/501784737 learned_addr learned my addr 172.21.15.26:0/501784737 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff76be00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77413fae0 0x7ff774153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff773e00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413cfa0 0x7ff774159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff779000640 1 -- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413cfa0 msgr2=0x7ff774159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff779000640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413cfa0 0x7ff774159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff779000640 1 -- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77413fae0 msgr2=0x7ff774153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff779000640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77413fae0 0x7ff774153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff779000640 1 -- 172.21.15.26:0/501784737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0094d0 con 0x7ff77415afb0 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff76be00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77413fae0 0x7ff774153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:30.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.174+0000 7ff779000640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 0x7ff77406cf30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00cd70 tx=0x7ff76c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:30.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.175+0000 7ff770c00640 1 -- 172.21.15.26:0/501784737 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c003b90 con 0x7ff77415afb0 2024-01-15T18:05:30.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.175+0000 7ff770c00640 1 -- 172.21.15.26:0/501784737 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c003cf0 con 0x7ff77415afb0 2024-01-15T18:05:30.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.175+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7741f6ed0 con 0x7ff77415afb0 2024-01-15T18:05:30.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.175+0000 7ff770c00640 1 -- 172.21.15.26:0/501784737 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c004240 con 0x7ff77415afb0 2024-01-15T18:05:30.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.175+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77414fe90 con 0x7ff77415afb0 2024-01-15T18:05:30.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.176+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7741f76d0 con 0x7ff77415afb0 2024-01-15T18:05:30.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.177+0000 7ff770c00640 1 -- 172.21.15.26:0/501784737 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7ff76c0043a0 con 0x7ff77415afb0 2024-01-15T18:05:30.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.178+0000 7ff770c00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff74c06be90 0x7ff74c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:30.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.178+0000 7ff770c00640 1 -- 172.21.15.26:0/501784737 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7ff76c012070 con 0x7ff77415afb0 2024-01-15T18:05:30.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.178+0000 7ff773e00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff74c06be90 0x7ff74c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:30.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.179+0000 7ff773e00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff74c06be90 0x7ff74c06e350 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff764003d90 tx=0x7ff764009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:30.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.182+0000 7ff770c00640 1 -- 172.21.15.26:0/501784737 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff7741f76d0 con 0x7ff77415afb0 2024-01-15T18:05:30.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:30.411+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41819", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7740cea40 con 0x7ff74c06be90 2024-01-15T18:05:37.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.190+0000 7ff770c00640 1 -- 172.21.15.26:0/501784737 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff764003980 con 0x7ff74c06be90 2024-01-15T18:05:37.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.193+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff74c06be90 msgr2=0x7ff74c06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.193+0000 7ff779a00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff74c06be90 0x7ff74c06e350 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff764003d90 tx=0x7ff764009040 comp rx=0 tx=0).stop 2024-01-15T18:05:37.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.193+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 msgr2=0x7ff77406cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.193+0000 7ff779a00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 0x7ff77406cf30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00cd70 tx=0x7ff76c00a660 comp rx=0 tx=0).stop 2024-01-15T18:05:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.193+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 shutdown_connections 2024-01-15T18:05:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.193+0000 7ff779a00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff74c06be90 0x7ff74c06e350 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.193+0000 7ff779a00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff77415afb0 0x7ff77406cf30 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.193+0000 7ff779a00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff77413fae0 0x7ff774153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.193+0000 7ff779a00640 1 --2- 172.21.15.26:0/501784737 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff77413cfa0 0x7ff774159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.194+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 >> 172.21.15.26:0/501784737 conn(0x7ff77406a800 msgr2=0x7ff77406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.194+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 shutdown_connections 2024-01-15T18:05:37.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.194+0000 7ff779a00640 1 -- 172.21.15.26:0/501784737 wait complete. 2024-01-15T18:05:37.232 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone41820 2024-01-15T18:05:37.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.540+0000 7f2c96200640 1 Processor -- start 2024-01-15T18:05:37.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.541+0000 7f2c96200640 1 -- start start 2024-01-15T18:05:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.543+0000 7f2c96200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2c9014b2c0 0x7f2c900cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.543+0000 7f2c96200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 0x7f2c900cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.543+0000 7f2c96200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c900cfbf0 con 0x7f2c900cf7d0 2024-01-15T18:05:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.543+0000 7f2c96200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9014b6a0 con 0x7f2c9014b2c0 2024-01-15T18:05:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.543+0000 7f2c96200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90071690 con 0x7f2c90071070 2024-01-15T18:05:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.545+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 0x7f2c900cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.545+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 0x7f2c900cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:53498/0 (socket says 172.21.15.26:53498) 2024-01-15T18:05:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.545+0000 7f2c8fe00640 1 -- 172.21.15.26:0/2114674183 learned_addr learned my addr 172.21.15.26:0/2114674183 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.546+0000 7f2c8f400640 1 --2- 172.21.15.26:0/2114674183 >> v2:172.21.15.26:3300/0 conn(0x7f2c9014b2c0 0x7f2c900cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.546+0000 7f2c8fe00640 1 -- 172.21.15.26:0/2114674183 >> v2:172.21.15.26:3300/0 conn(0x7f2c9014b2c0 msgr2=0x7f2c900cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.546+0000 7f2c8fe00640 1 --2- 172.21.15.26:0/2114674183 >> v2:172.21.15.26:3300/0 conn(0x7f2c9014b2c0 0x7f2c900cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.546+0000 7f2c8fe00640 1 -- 172.21.15.26:0/2114674183 >> v1:172.21.15.124:6789/0 conn(0x7f2c900cf7d0 legacy=0x7f2c90070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:37.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.546+0000 7f2c8fe00640 1 -- 172.21.15.26:0/2114674183 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c90071970 con 0x7f2c90071070 2024-01-15T18:05:37.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.549+0000 7f2c8fe00640 1 --2- 172.21.15.26:0/2114674183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 0x7f2c900cc160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c8002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f3a6280c749d428 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:37.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.553+0000 7f2c8e000640 1 -- 172.21.15.26:0/2114674183 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80031640 con 0x7f2c90071070 2024-01-15T18:05:37.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.553+0000 7f2c8e000640 1 -- 172.21.15.26:0/2114674183 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c800317a0 con 0x7f2c90071070 2024-01-15T18:05:37.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.553+0000 7f2c8e000640 1 -- 172.21.15.26:0/2114674183 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80031b30 con 0x7f2c90071070 2024-01-15T18:05:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.555+0000 7f2c96200640 1 -- 172.21.15.26:0/2114674183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 msgr2=0x7f2c900cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.555+0000 7f2c96200640 1 --2- 172.21.15.26:0/2114674183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 0x7f2c900cc160 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c8002b8f0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.555+0000 7f2c96200640 1 -- 172.21.15.26:0/2114674183 shutdown_connections 2024-01-15T18:05:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.555+0000 7f2c96200640 1 --2- 172.21.15.26:0/2114674183 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 0x7f2c900cc160 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.555+0000 7f2c96200640 1 --2- 172.21.15.26:0/2114674183 >> v2:172.21.15.26:3300/0 conn(0x7f2c9014b2c0 0x7f2c900cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.555+0000 7f2c96200640 1 -- 172.21.15.26:0/2114674183 >> 172.21.15.26:0/2114674183 conn(0x7f2c9006a800 msgr2=0x7f2c9006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.556+0000 7f2c96200640 1 -- 172.21.15.26:0/2114674183 shutdown_connections 2024-01-15T18:05:37.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.557+0000 7f2c96200640 1 -- 172.21.15.26:0/2114674183 wait complete. 2024-01-15T18:05:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.559+0000 7f2c96200640 1 Processor -- start 2024-01-15T18:05:37.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.560+0000 7f2c96200640 1 -- start start 2024-01-15T18:05:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.561+0000 7f2c96200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 0x7f2c90160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.561+0000 7f2c96200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c900cf7d0 0x7f2c90164a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.561+0000 7f2c96200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9014b2c0 0x7f2c9015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.561+0000 7f2c96200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9015cb10 con 0x7f2c9014b2c0 2024-01-15T18:05:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.561+0000 7f2c96200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9015f480 con 0x7f2c900cf7d0 2024-01-15T18:05:37.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.561+0000 7f2c96200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9015f760 con 0x7f2c90071070 2024-01-15T18:05:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.562+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9014b2c0 0x7f2c9015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.562+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9014b2c0 0x7f2c9015c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34332/0 (socket says 172.21.15.26:34332) 2024-01-15T18:05:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.562+0000 7f2c8fe00640 1 -- 172.21.15.26:0/3923840111 learned_addr learned my addr 172.21.15.26:0/3923840111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:37.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.562+0000 7f2c8ea00640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c900cf7d0 0x7f2c90164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.563+0000 7f2c8ea00640 1 -- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 msgr2=0x7f2c90160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.563+0000 7f2c8ea00640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 0x7f2c90160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.564+0000 7f2c8ea00640 1 -- 172.21.15.26:0/3923840111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9014b2c0 msgr2=0x7f2c9015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.564+0000 7f2c8ea00640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9014b2c0 0x7f2c9015c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.564+0000 7f2c8ea00640 1 -- 172.21.15.26:0/3923840111 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800094d0 con 0x7f2c900cf7d0 2024-01-15T18:05:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.564+0000 7f2c8fe00640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9014b2c0 0x7f2c9015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:37.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.564+0000 7f2c8ea00640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c900cf7d0 0x7f2c90164a30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c6d0 tx=0x7f2c8800cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:37.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.565+0000 7f2c87400640 1 -- 172.21.15.26:0/3923840111 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88003b70 con 0x7f2c900cf7d0 2024-01-15T18:05:37.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.565+0000 7f2c87400640 1 -- 172.21.15.26:0/3923840111 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c88003cf0 con 0x7f2c900cf7d0 2024-01-15T18:05:37.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.565+0000 7f2c87400640 1 -- 172.21.15.26:0/3923840111 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88003ea0 con 0x7f2c900cf7d0 2024-01-15T18:05:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.566+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9015fa40 con 0x7f2c900cf7d0 2024-01-15T18:05:37.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.566+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c90160000 con 0x7f2c900cf7d0 2024-01-15T18:05:37.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.567+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c90160220 con 0x7f2c900cf7d0 2024-01-15T18:05:37.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.571+0000 7f2c87400640 1 -- 172.21.15.26:0/3923840111 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f2c880054c0 con 0x7f2c900cf7d0 2024-01-15T18:05:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.572+0000 7f2c87400640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c6806bee0 0x7f2c6806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:37.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.573+0000 7f2c87400640 1 -- 172.21.15.26:0/3923840111 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f2c88088630 con 0x7f2c900cf7d0 2024-01-15T18:05:37.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.582+0000 7f2c8f400640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c6806bee0 0x7f2c6806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:37.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.583+0000 7f2c87400640 1 -- 172.21.15.26:0/3923840111 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c88054d80 con 0x7f2c900cf7d0 2024-01-15T18:05:37.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.589+0000 7f2c8f400640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c6806bee0 0x7f2c6806e3a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006a90 tx=0x7f2c78006a20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:37.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:37.942+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone41820", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c9006d480 con 0x7f2c6806bee0 2024-01-15T18:05:38.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.321+0000 7f2c87400640 1 -- 172.21.15.26:0/3923840111 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2c9006d480 con 0x7f2c6806bee0 2024-01-15T18:05:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.325+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c6806bee0 msgr2=0x7f2c6806e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.325+0000 7f2c96200640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c6806bee0 0x7f2c6806e3a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006a90 tx=0x7f2c78006a20 comp rx=0 tx=0).stop 2024-01-15T18:05:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.326+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c900cf7d0 msgr2=0x7f2c90164a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.326+0000 7f2c96200640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c900cf7d0 0x7f2c90164a30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800c6d0 tx=0x7f2c8800cb90 comp rx=0 tx=0).stop 2024-01-15T18:05:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.326+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 shutdown_connections 2024-01-15T18:05:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.326+0000 7f2c96200640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c6806bee0 0x7f2c6806e3a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.326+0000 7f2c96200640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c9014b2c0 0x7f2c9015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.326+0000 7f2c96200640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c900cf7d0 0x7f2c90164a30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.326+0000 7f2c96200640 1 --2- 172.21.15.26:0/3923840111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c90071070 0x7f2c90160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.326+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 >> 172.21.15.26:0/3923840111 conn(0x7f2c9006a800 msgr2=0x7f2c90148660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:38.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.328+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 shutdown_connections 2024-01-15T18:05:38.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.330+0000 7f2c96200640 1 -- 172.21.15.26:0/3923840111 wait complete. 2024-01-15T18:05:38.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json-pretty 2024-01-15T18:05:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.653+0000 7f44c9e00640 1 Processor -- start 2024-01-15T18:05:38.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.654+0000 7f44c9e00640 1 -- start start 2024-01-15T18:05:38.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.656+0000 7f44c9e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c40ce8f0 0x7f44c40ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.656+0000 7f44c9e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c40d0de0 con 0x7f44c4160f80 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.656+0000 7f44c9e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c40d10c0 con 0x7f44c40b2a00 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.656+0000 7f44c9e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40d13a0 con 0x7f44c40ce8f0 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.656+0000 7f44c2a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f44c4160f80 0x7f44c4161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42850/0 (socket says 172.21.15.26:42850) 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.656+0000 7f44c2a00640 1 -- 172.21.15.26:0/2126732485 learned_addr learned my addr 172.21.15.26:0/2126732485 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125467196 0 0) 0x7f44c40d0de0 con 0x7f44c4160f80 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a4003520 con 0x7f44c4160f80 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c2000640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c40ce8f0 0x7f44c40ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2874993394 0 0) 0x7f44a4003520 con 0x7f44c4160f80 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c40ce8f0 msgr2=0x7f44c40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c40ce8f0 0x7f44c40ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 >> v1:172.21.15.124:6789/0 conn(0x7f44c40b2a00 legacy=0x7f44c40b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40d1680 con 0x7f44c4160f80 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f44b4001670 con 0x7f44c4160f80 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 >> v1:172.21.15.26:6789/0 conn(0x7f44c4160f80 legacy=0x7f44c4161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 0x7f44a4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44a40047c0 0x7f44a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c40ce8f0 0x7f44a4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40d13a0 con 0x7f44a40039a0 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a400bb10 con 0x7f44a40047c0 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.657+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a400bca0 con 0x7f44c40ce8f0 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.658+0000 7f44c2a00640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 0x7f44a4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.658+0000 7f44c2a00640 1 -- 172.21.15.26:0/2126732485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c40ce8f0 msgr2=0x7f44a4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:38.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.658+0000 7f44c2a00640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c40ce8f0 0x7f44a4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.658+0000 7f44c2a00640 1 -- 172.21.15.26:0/2126732485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44a40047c0 msgr2=0x7f44a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.658+0000 7f44c2a00640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44a40047c0 0x7f44a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.658+0000 7f44c2a00640 1 -- 172.21.15.26:0/2126732485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4003520 con 0x7f44a40039a0 2024-01-15T18:05:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.659+0000 7f44c2a00640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 0x7f44a4003da0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c190 tx=0x7f44a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7ca5fadafa28a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.659+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4005ad0 con 0x7f44a40039a0 2024-01-15T18:05:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.659+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b4005c30 con 0x7f44a40039a0 2024-01-15T18:05:38.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.659+0000 7f44c1600640 1 -- 172.21.15.26:0/2126732485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b402c810 con 0x7f44a40039a0 2024-01-15T18:05:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.660+0000 7f44c9e00640 1 -- 172.21.15.26:0/2126732485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 msgr2=0x7f44a4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.660+0000 7f44c9e00640 1 --2- 172.21.15.26:0/2126732485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 0x7f44a4003da0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f44b400c190 tx=0x7f44a40057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.660+0000 7f44c3400640 1 -- 172.21.15.26:0/2126732485 reap_dead start 2024-01-15T18:05:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.661+0000 7f44c9e00640 1 -- 172.21.15.26:0/2126732485 shutdown_connections 2024-01-15T18:05:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.661+0000 7f44c9e00640 1 -- 172.21.15.26:0/2126732485 >> 172.21.15.26:0/2126732485 conn(0x7f44c409c550 msgr2=0x7f44c40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.661+0000 7f44c9e00640 1 -- 172.21.15.26:0/2126732485 shutdown_connections 2024-01-15T18:05:38.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.661+0000 7f44c9e00640 1 -- 172.21.15.26:0/2126732485 wait complete. 2024-01-15T18:05:38.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.664+0000 7f44c9e00640 1 Processor -- start 2024-01-15T18:05:38.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.665+0000 7f44c9e00640 1 -- start start 2024-01-15T18:05:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.666+0000 7f44c9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 0x7f44c4158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.666+0000 7f44c9e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44c40b79d0 0x7f44c415f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.666+0000 7f44c9e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c415f570 0x7f44c40b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.666+0000 7f44c9e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40b6970 con 0x7f44a40039a0 2024-01-15T18:05:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.666+0000 7f44c9e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40b6c10 con 0x7f44c40b79d0 2024-01-15T18:05:38.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.666+0000 7f44c9e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c40b6ef0 con 0x7f44c415f570 2024-01-15T18:05:38.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.667+0000 7f44c3400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c415f570 0x7f44c40b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.667+0000 7f44c3400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c415f570 0x7f44c40b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:53526/0 (socket says 172.21.15.26:53526) 2024-01-15T18:05:38.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.667+0000 7f44c2000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44c40b79d0 0x7f44c415f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.667+0000 7f44c3400640 1 -- 172.21.15.26:0/3121616890 learned_addr learned my addr 172.21.15.26:0/3121616890 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:38.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.667+0000 7f44c2a00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 0x7f44c4158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.668+0000 7f44c2a00640 1 -- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c415f570 msgr2=0x7f44c40b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.668+0000 7f44c2a00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c415f570 0x7f44c40b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.668+0000 7f44c2a00640 1 -- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44c40b79d0 msgr2=0x7f44c415f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:38.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.668+0000 7f44c2a00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44c40b79d0 0x7f44c415f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:38.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.668+0000 7f44c2a00640 1 -- 172.21.15.26:0/3121616890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40b71d0 con 0x7f44a40039a0 2024-01-15T18:05:38.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.668+0000 7f44c3400640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c415f570 0x7f44c40b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:05:38.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.669+0000 7f44c2a00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 0x7f44c4158670 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f44b4031000 tx=0x7f44b4001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:38.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.669+0000 7f44bb400640 1 -- 172.21.15.26:0/3121616890 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b403c040 con 0x7f44a40039a0 2024-01-15T18:05:38.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.669+0000 7f44bb400640 1 -- 172.21.15.26:0/3121616890 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b402cc40 con 0x7f44a40039a0 2024-01-15T18:05:38.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.669+0000 7f44bb400640 1 -- 172.21.15.26:0/3121616890 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b4007070 con 0x7f44a40039a0 2024-01-15T18:05:38.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.669+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c40b74b0 con 0x7f44a40039a0 2024-01-15T18:05:38.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.670+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c4158bb0 con 0x7f44a40039a0 2024-01-15T18:05:38.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.671+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c40675d0 con 0x7f44a40039a0 2024-01-15T18:05:38.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.673+0000 7f44bb400640 1 -- 172.21.15.26:0/3121616890 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f44b4032070 con 0x7f44a40039a0 2024-01-15T18:05:38.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.674+0000 7f44bb400640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f449c06be90 0x7f449c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:38.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.674+0000 7f44bb400640 1 -- 172.21.15.26:0/3121616890 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f44b40a9610 con 0x7f44a40039a0 2024-01-15T18:05:38.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.681+0000 7f44c2000640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f449c06be90 0x7f449c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:38.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.683+0000 7f44bb400640 1 -- 172.21.15.26:0/3121616890 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44b4075d60 con 0x7f44a40039a0 2024-01-15T18:05:38.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:38.685+0000 7f44c2000640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f449c06be90 0x7f449c06e350 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f44c40b1a90 tx=0x7f44ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.172+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f44c40d3920 con 0x7f44a40039a0 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.173+0000 7f44bb400640 1 -- 172.21.15.26:0/3121616890 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+2426 (secure 0 0 0) 0x7f44b4079380 con 0x7f44a40039a0 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:05:39.178 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 653, 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:39.179 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 74, 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:39.180 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 3422, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 14137054625, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 28393373696, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 744700739584, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 382515482, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 372447317, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 92, 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 105 2024-01-15T18:05:39.181 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 45, 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 5, 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 5, 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "c", 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:05:39.182 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 7095 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:39.183 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 16, 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:04:32.130685+0000", 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": { 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "98f8b045-8d38-4d4c-b08c-e2a9fdf0f30e": { 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "18 ongoing+pending cloning jobs; Avg progress = 18.519% (38s)\n [=====.......................] (remaining: 2m)", 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.185188889503479, 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "d053d221-147b-4046-8d92-99e89e0f4670": { 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "message": "4 ongoing cloning jobs; Avg progress = 83.335% (38s)\n [=======================.....] (remaining: 7s)", 2024-01-15T18:05:39.184 INFO:teuthology.orchestra.run.smithi026.stdout: "progress": 0.83335000276565552, 2024-01-15T18:05:39.185 INFO:teuthology.orchestra.run.smithi026.stdout: "add_to_ceph_s": true 2024-01-15T18:05:39.185 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:39.185 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:05:39.185 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:05:39.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.178+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f449c06be90 msgr2=0x7f449c06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.178+0000 7f44c9e00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f449c06be90 0x7f449c06e350 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f44c40b1a90 tx=0x7f44ac008040 comp rx=0 tx=0).stop 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.178+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 msgr2=0x7f44c4158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.178+0000 7f44c9e00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 0x7f44c4158670 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f44b4031000 tx=0x7f44b4001930 comp rx=0 tx=0).stop 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.179+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 shutdown_connections 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.179+0000 7f44c9e00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44c415f570 0x7f44c40b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.179+0000 7f44c9e00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f449c06be90 0x7f449c06e350 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.179+0000 7f44c9e00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44c40b79d0 0x7f44c415f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.179+0000 7f44c9e00640 1 --2- 172.21.15.26:0/3121616890 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44a40039a0 0x7f44c4158670 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.179+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 >> 172.21.15.26:0/3121616890 conn(0x7f44c409c550 msgr2=0x7f44c415c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.181+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 shutdown_connections 2024-01-15T18:05:39.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.181+0000 7f44c9e00640 1 -- 172.21.15.26:0/3121616890 wait complete. 2024-01-15T18:05:39.221 INFO:tasks.cephfs.test_volumes:new_onpen_p is less than onpen_p. 2024-01-15T18:05:39.221 INFO:tasks.cephfs.test_volumes:new_onpen_p = 0.185188889503479; onpen_p = 0.2857142984867096 2024-01-15T18:05:39.222 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone4181 2024-01-15T18:05:39.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.482+0000 7f20dca00640 1 Processor -- start 2024-01-15T18:05:39.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.482+0000 7f20dca00640 1 -- start start 2024-01-15T18:05:39.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.483+0000 7f20dca00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f20d40ced30 0x7f20d40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.483+0000 7f20dca00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d40cf1c0 con 0x7f20d4160f80 2024-01-15T18:05:39.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.483+0000 7f20dca00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d40d0de0 con 0x7f20d40ce8f0 2024-01-15T18:05:39.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.483+0000 7f20dca00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d40d10c0 con 0x7f20d40ced30 2024-01-15T18:05:39.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.484+0000 7f20da000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f20d4160f80 0x7f20d4161360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42860/0 (socket says 172.21.15.26:42860) 2024-01-15T18:05:39.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.484+0000 7f20da000640 1 -- 172.21.15.26:0/2013790258 learned_addr learned my addr 172.21.15.26:0/2013790258 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:39.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.484+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768163729 0 0) 0x7f20c4001110 con 0x7f20d4160f80 2024-01-15T18:05:39.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.484+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20b4003520 con 0x7f20d4160f80 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.484+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4016667294 0 0) 0x7f20b4003520 con 0x7f20d4160f80 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.484+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 >> v2:172.21.15.124:3300/0 conn(0x7f20d40ced30 msgr2=0x7f20d40b3110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 --2- 172.21.15.26:0/2013790258 >> v2:172.21.15.124:3300/0 conn(0x7f20d40ced30 0x7f20d40b3110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 >> v1:172.21.15.124:6789/0 conn(0x7f20d40ce8f0 legacy=0x7f20d40b2a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d40d13a0 con 0x7f20d4160f80 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f20c4001800 con 0x7f20d4160f80 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 >> v1:172.21.15.26:6789/0 conn(0x7f20d4160f80 legacy=0x7f20d4161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 --2- 172.21.15.26:0/2013790258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20b40039c0 0x7f20b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 --2- 172.21.15.26:0/2013790258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b40047c0 0x7f20b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 --2- 172.21.15.26:0/2013790258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20b40052a0 0x7f20b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40d10c0 con 0x7f20b40047c0 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b400c1f0 con 0x7f20b40052a0 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b400c3b0 con 0x7f20b40039c0 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d9600640 1 --2- 172.21.15.26:0/2013790258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b40047c0 0x7f20b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d9600640 1 -- 172.21.15.26:0/2013790258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20b40039c0 msgr2=0x7f20b4003dc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:05:39.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d9600640 1 --2- 172.21.15.26:0/2013790258 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20b40039c0 0x7f20b4003dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d9600640 1 -- 172.21.15.26:0/2013790258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20b40052a0 msgr2=0x7f20b400b980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-01-15T18:05:39.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d9600640 1 --2- 172.21.15.26:0/2013790258 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20b40052a0 0x7f20b400b980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20d9600640 1 -- 172.21.15.26:0/2013790258 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b4003520 con 0x7f20b40047c0 2024-01-15T18:05:39.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.485+0000 7f20daa00640 1 -- 172.21.15.26:0/2013790258 reap_dead start 2024-01-15T18:05:39.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.487+0000 7f20d9600640 1 --2- 172.21.15.26:0/2013790258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b40047c0 0x7f20b4004c40 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009830 tx=0x7f20b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545f95444d049072 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.487+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc032600 con 0x7f20b40047c0 2024-01-15T18:05:39.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.487+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc02e050 con 0x7f20b40047c0 2024-01-15T18:05:39.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.487+0000 7f20d8c00640 1 -- 172.21.15.26:0/2013790258 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0329c0 con 0x7f20b40047c0 2024-01-15T18:05:39.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.488+0000 7f20dca00640 1 -- 172.21.15.26:0/2013790258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b40047c0 msgr2=0x7f20b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.488+0000 7f20dca00640 1 --2- 172.21.15.26:0/2013790258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b40047c0 0x7f20b4004c40 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009830 tx=0x7f20b40057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.489+0000 7f20dca00640 1 -- 172.21.15.26:0/2013790258 shutdown_connections 2024-01-15T18:05:39.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.489+0000 7f20dca00640 1 --2- 172.21.15.26:0/2013790258 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20b40047c0 0x7f20b4004c40 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.489+0000 7f20dca00640 1 -- 172.21.15.26:0/2013790258 >> 172.21.15.26:0/2013790258 conn(0x7f20d409c550 msgr2=0x7f20d40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:39.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.489+0000 7f20dca00640 1 -- 172.21.15.26:0/2013790258 shutdown_connections 2024-01-15T18:05:39.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.489+0000 7f20dca00640 1 -- 172.21.15.26:0/2013790258 wait complete. 2024-01-15T18:05:39.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.492+0000 7f20dca00640 1 Processor -- start 2024-01-15T18:05:39.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.493+0000 7f20dca00640 1 -- start start 2024-01-15T18:05:39.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.495+0000 7f20dca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20b40047c0 0x7f20d41587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.496+0000 7f20dca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20d41f2ed0 0x7f20d41f3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.496+0000 7f20dca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d406b7f0 0x7f20d406bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.496+0000 7f20dca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d4158d20 con 0x7f20d406b7f0 2024-01-15T18:05:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.496+0000 7f20dca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d406c1b0 con 0x7f20b40047c0 2024-01-15T18:05:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.496+0000 7f20dca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d40c7fd0 con 0x7f20d41f2ed0 2024-01-15T18:05:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.496+0000 7f20daa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d406b7f0 0x7f20d406bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.496+0000 7f20daa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d406b7f0 0x7f20d406bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34370/0 (socket says 172.21.15.26:34370) 2024-01-15T18:05:39.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.496+0000 7f20daa00640 1 -- 172.21.15.26:0/945047075 learned_addr learned my addr 172.21.15.26:0/945047075 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.497+0000 7f20da000640 1 --2- 172.21.15.26:0/945047075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20b40047c0 0x7f20d41587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.497+0000 7f20daa00640 1 -- 172.21.15.26:0/945047075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20d41f2ed0 msgr2=0x7f20d41f3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.497+0000 7f20daa00640 1 --2- 172.21.15.26:0/945047075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f20d41f2ed0 0x7f20d41f3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.497+0000 7f20daa00640 1 -- 172.21.15.26:0/945047075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20b40047c0 msgr2=0x7f20d41587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.497+0000 7f20daa00640 1 --2- 172.21.15.26:0/945047075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20b40047c0 0x7f20d41587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:39.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.498+0000 7f20daa00640 1 -- 172.21.15.26:0/945047075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d41f39e0 con 0x7f20d406b7f0 2024-01-15T18:05:39.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.498+0000 7f20da000640 1 --2- 172.21.15.26:0/945047075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f20b40047c0 0x7f20d41587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.498+0000 7f20daa00640 1 --2- 172.21.15.26:0/945047075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f20d406b7f0 0x7f20d406bc70 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f20d408b4b0 tx=0x7f20cc00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.499+0000 7f20d2000640 1 -- 172.21.15.26:0/945047075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00cb50 con 0x7f20d406b7f0 2024-01-15T18:05:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.500+0000 7f20d2000640 1 -- 172.21.15.26:0/945047075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc00ccb0 con 0x7f20d406b7f0 2024-01-15T18:05:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.500+0000 7f20dca00640 1 -- 172.21.15.26:0/945047075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d41f3cc0 con 0x7f20d406b7f0 2024-01-15T18:05:39.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.500+0000 7f20d2000640 1 -- 172.21.15.26:0/945047075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc0052b0 con 0x7f20d406b7f0 2024-01-15T18:05:39.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.500+0000 7f20dca00640 1 -- 172.21.15.26:0/945047075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d40c8500 con 0x7f20d406b7f0 2024-01-15T18:05:39.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.502+0000 7f20dca00640 1 -- 172.21.15.26:0/945047075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a4005250 con 0x7f20d406b7f0 2024-01-15T18:05:39.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.505+0000 7f20d2000640 1 -- 172.21.15.26:0/945047075 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f20cc012020 con 0x7f20d406b7f0 2024-01-15T18:05:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.507+0000 7f20d2000640 1 --2- 172.21.15.26:0/945047075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f20ac06bf90 0x7f20ac06e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:39.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.507+0000 7f20d2000640 1 -- 172.21.15.26:0/945047075 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f20cc0882e0 con 0x7f20d406b7f0 2024-01-15T18:05:39.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.511+0000 7f20da000640 1 --2- 172.21.15.26:0/945047075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f20ac06bf90 0x7f20ac06e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:39.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.513+0000 7f20da000640 1 --2- 172.21.15.26:0/945047075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f20ac06bf90 0x7f20ac06e450 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f20c4001eb0 tx=0x7f20c4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:39.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.514+0000 7f20d2000640 1 -- 172.21.15.26:0/945047075 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20cc054940 con 0x7f20d406b7f0 2024-01-15T18:05:39.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:39.818+0000 7f20dca00640 1 -- 172.21.15.26:0/945047075 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone4181", "target": ["mon-mgr", ""]}) v1 -- 0x7f20a4000bc0 con 0x7f20ac06bf90 2024-01-15T18:05:41.176 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:05:41.172+0000 7f8f54600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-01-15T18:05:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.172+0000 7f20d2000640 1 -- 172.21.15.26:0/945047075 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f20a4000bc0 con 0x7f20ac06bf90 2024-01-15T18:05:41.177 INFO:teuthology.orchestra.run.smithi026.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-01-15T18:05:41.196 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-01-15T18:05:41.197 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone4182 2024-01-15T18:05:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.402+0000 7f2376200640 1 Processor -- start 2024-01-15T18:05:41.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.402+0000 7f2376200640 1 -- start start 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.404+0000 7f2376200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f237015b0c0 0x7f237006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.404+0000 7f2376200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f237006fab0 0x7f237006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.404+0000 7f2376200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f237006e800 0x7f237006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.404+0000 7f2376200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23700cebb0 con 0x7f237006fab0 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.404+0000 7f2376200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23700cee90 con 0x7f237006e800 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.404+0000 7f2376200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23700cf170 con 0x7f237015b0c0 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.404+0000 7f236fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f237006fab0 0x7f237006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.405+0000 7f236fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f237006fab0 0x7f237006a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34372/0 (socket says 172.21.15.26:34372) 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.405+0000 7f236fe00640 1 -- 172.21.15.26:0/3447923741 learned_addr learned my addr 172.21.15.26:0/3447923741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.405+0000 7f2374e00640 1 --2- 172.21.15.26:0/3447923741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f237015b0c0 0x7f237006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.405+0000 7f2375800640 1 --2- 172.21.15.26:0/3447923741 >> v2:172.21.15.124:3300/0 conn(0x7f237006e800 0x7f237006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.405+0000 7f236fe00640 1 -- 172.21.15.26:0/3447923741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f237015b0c0 msgr2=0x7f237006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.405+0000 7f236fe00640 1 --2- 172.21.15.26:0/3447923741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f237015b0c0 0x7f237006a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.405+0000 7f236fe00640 1 -- 172.21.15.26:0/3447923741 >> v2:172.21.15.124:3300/0 conn(0x7f237006e800 msgr2=0x7f237006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.405+0000 7f236fe00640 1 --2- 172.21.15.26:0/3447923741 >> v2:172.21.15.124:3300/0 conn(0x7f237006e800 0x7f237006d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.405+0000 7f236fe00640 1 -- 172.21.15.26:0/3447923741 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23700cf450 con 0x7f237006fab0 2024-01-15T18:05:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.406+0000 7f236fe00640 1 --2- 172.21.15.26:0/3447923741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237006fab0 0x7f237006a920 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f237015ea40 tx=0x7f235802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64757dcaa956ab53 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.407+0000 7f236f400640 1 -- 172.21.15.26:0/3447923741 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358031870 con 0x7f237006fab0 2024-01-15T18:05:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.407+0000 7f236f400640 1 -- 172.21.15.26:0/3447923741 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23580319d0 con 0x7f237006fab0 2024-01-15T18:05:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.407+0000 7f236f400640 1 -- 172.21.15.26:0/3447923741 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358031d60 con 0x7f237006fab0 2024-01-15T18:05:41.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.408+0000 7f2376200640 1 -- 172.21.15.26:0/3447923741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237006fab0 msgr2=0x7f237006a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.408+0000 7f2376200640 1 --2- 172.21.15.26:0/3447923741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237006fab0 0x7f237006a920 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f237015ea40 tx=0x7f235802b910 comp rx=0 tx=0).stop 2024-01-15T18:05:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.408+0000 7f2376200640 1 -- 172.21.15.26:0/3447923741 shutdown_connections 2024-01-15T18:05:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.408+0000 7f2376200640 1 --2- 172.21.15.26:0/3447923741 >> v2:172.21.15.124:3300/0 conn(0x7f237006e800 0x7f237006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.408+0000 7f2376200640 1 --2- 172.21.15.26:0/3447923741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237006fab0 0x7f237006a920 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.408+0000 7f2376200640 1 --2- 172.21.15.26:0/3447923741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f237015b0c0 0x7f237006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.408+0000 7f2376200640 1 -- 172.21.15.26:0/3447923741 >> 172.21.15.26:0/3447923741 conn(0x7f237006bb20 msgr2=0x7f23700cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.408+0000 7f2376200640 1 -- 172.21.15.26:0/3447923741 shutdown_connections 2024-01-15T18:05:41.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.408+0000 7f2376200640 1 -- 172.21.15.26:0/3447923741 wait complete. 2024-01-15T18:05:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.410+0000 7f2376200640 1 Processor -- start 2024-01-15T18:05:41.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.410+0000 7f2376200640 1 -- start start 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.411+0000 7f2376200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237006e800 0x7f237013a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.411+0000 7f2376200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f237006fab0 0x7f23701e01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.411+0000 7f2376200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f237015b0c0 0x7f23701e06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.411+0000 7f2376200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23701e1e00 con 0x7f237006e800 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.411+0000 7f2376200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23701e2320 con 0x7f237006fab0 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.411+0000 7f2376200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23701e2600 con 0x7f237015b0c0 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.412+0000 7f2374e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237006e800 0x7f237013a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.412+0000 7f2374e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237006e800 0x7f237013a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34376/0 (socket says 172.21.15.26:34376) 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.412+0000 7f2374e00640 1 -- 172.21.15.26:0/4199175210 learned_addr learned my addr 172.21.15.26:0/4199175210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.412+0000 7f2374e00640 1 -- 172.21.15.26:0/4199175210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f237015b0c0 msgr2=0x7f23701e06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.412+0000 7f2374e00640 1 --2- 172.21.15.26:0/4199175210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f237015b0c0 0x7f23701e06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.412+0000 7f2374e00640 1 -- 172.21.15.26:0/4199175210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f237006fab0 msgr2=0x7f23701e01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.412+0000 7f2374e00640 1 --2- 172.21.15.26:0/4199175210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f237006fab0 0x7f23701e01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.412+0000 7f2374e00640 1 -- 172.21.15.26:0/4199175210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580094d0 con 0x7f237006e800 2024-01-15T18:05:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.412+0000 7f2374e00640 1 --2- 172.21.15.26:0/4199175210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f237006e800 0x7f237013a310 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f236000d5f0 tx=0x7f236000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.413+0000 7f236cc00640 1 -- 172.21.15.26:0/4199175210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600125f0 con 0x7f237006e800 2024-01-15T18:05:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.413+0000 7f236cc00640 1 -- 172.21.15.26:0/4199175210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2360012750 con 0x7f237006e800 2024-01-15T18:05:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.413+0000 7f236cc00640 1 -- 172.21.15.26:0/4199175210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2360012b00 con 0x7f237006e800 2024-01-15T18:05:41.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.413+0000 7f2376200640 1 -- 172.21.15.26:0/4199175210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237013aa40 con 0x7f237006e800 2024-01-15T18:05:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.413+0000 7f2376200640 1 -- 172.21.15.26:0/4199175210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237013b000 con 0x7f237006e800 2024-01-15T18:05:41.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.415+0000 7f2376200640 1 -- 172.21.15.26:0/4199175210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2370067b60 con 0x7f237006e800 2024-01-15T18:05:41.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.415+0000 7f236cc00640 1 -- 172.21.15.26:0/4199175210 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f2360012c60 con 0x7f237006e800 2024-01-15T18:05:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.417+0000 7f236cc00640 1 --2- 172.21.15.26:0/4199175210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234806be90 0x7f234806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.417+0000 7f236cc00640 1 -- 172.21.15.26:0/4199175210 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f2360088d00 con 0x7f237006e800 2024-01-15T18:05:41.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.417+0000 7f236fe00640 1 --2- 172.21.15.26:0/4199175210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234806be90 0x7f234806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.419+0000 7f236fe00640 1 --2- 172.21.15.26:0/4199175210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f234806be90 0x7f234806e350 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2358036000 tx=0x7f2358030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.421+0000 7f236cc00640 1 -- 172.21.15.26:0/4199175210 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23600552d0 con 0x7f237006e800 2024-01-15T18:05:41.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.672+0000 7f2376200640 1 -- 172.21.15.26:0/4199175210 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone4182", "target": ["mon-mgr", ""]}) v1 -- 0x7f237006da20 con 0x7f234806be90 2024-01-15T18:05:41.698 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:05:41.694+0000 7f8f54600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-01-15T18:05:41.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.694+0000 7f236cc00640 1 -- 172.21.15.26:0/4199175210 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f2358031b40 con 0x7f234806be90 2024-01-15T18:05:41.699 INFO:teuthology.orchestra.run.smithi026.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-01-15T18:05:41.714 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-01-15T18:05:41.714 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone4183 2024-01-15T18:05:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.942+0000 7fee7e000640 1 Processor -- start 2024-01-15T18:05:41.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.942+0000 7fee7e000640 1 -- start start 2024-01-15T18:05:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.944+0000 7fee7e000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee7813fcb0 0x7fee780cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.944+0000 7fee7e000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fee780cdc20 0x7fee780b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.944+0000 7fee7e000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780ce440 con 0x7fee7813cf10 2024-01-15T18:05:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.944+0000 7fee7e000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780ce720 con 0x7fee780cdc20 2024-01-15T18:05:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.944+0000 7fee7e000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee780cea00 con 0x7fee7813fcb0 2024-01-15T18:05:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.944+0000 7fee7cc00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fee7813cf10 0x7fee7813f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34336/0 (socket says 172.21.15.26:34336) 2024-01-15T18:05:41.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.944+0000 7fee7cc00640 1 -- 172.21.15.26:0/3273609165 learned_addr learned my addr 172.21.15.26:0/3273609165 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.944+0000 7fee77e00640 1 --2- 172.21.15.26:0/3273609165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee7813fcb0 0x7fee780cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.944+0000 7fee7d600640 1 --2- 172.21.15.26:0/3273609165 >> v2:172.21.15.124:3300/0 conn(0x7fee780cdc20 0x7fee780b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.945+0000 7fee77400640 1 -- 172.21.15.26:0/3273609165 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247597741 0 0) 0x7fee780ce440 con 0x7fee7813cf10 2024-01-15T18:05:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.945+0000 7fee77400640 1 -- 172.21.15.26:0/3273609165 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee58003520 con 0x7fee7813cf10 2024-01-15T18:05:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.945+0000 7fee77400640 1 -- 172.21.15.26:0/3273609165 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2671778968 0 0) 0x7fee58003520 con 0x7fee7813cf10 2024-01-15T18:05:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.945+0000 7fee77e00640 1 -- 172.21.15.26:0/3273609165 >> v2:172.21.15.124:3300/0 conn(0x7fee780cdc20 msgr2=0x7fee780b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.945+0000 7fee77e00640 1 --2- 172.21.15.26:0/3273609165 >> v2:172.21.15.124:3300/0 conn(0x7fee780cdc20 0x7fee780b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.945+0000 7fee77e00640 1 -- 172.21.15.26:0/3273609165 >> v1:172.21.15.26:6789/0 conn(0x7fee7813cf10 legacy=0x7fee7813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.945+0000 7fee77e00640 1 -- 172.21.15.26:0/3273609165 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780b1d90 con 0x7fee7813fcb0 2024-01-15T18:05:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.947+0000 7fee77e00640 1 --2- 172.21.15.26:0/3273609165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee7813fcb0 0x7fee780cd6e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fee600097a0 tx=0x7fee6002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e467619becd885de server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.947+0000 7fee77400640 1 -- 172.21.15.26:0/3273609165 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680019f0 con 0x7fee7813fcb0 2024-01-15T18:05:41.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.947+0000 7fee77400640 1 -- 172.21.15.26:0/3273609165 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6002e050 con 0x7fee7813fcb0 2024-01-15T18:05:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.947+0000 7fee77400640 1 -- 172.21.15.26:0/3273609165 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60030710 con 0x7fee7813fcb0 2024-01-15T18:05:41.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.948+0000 7fee7e000640 1 -- 172.21.15.26:0/3273609165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee7813fcb0 msgr2=0x7fee780cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.948+0000 7fee7e000640 1 --2- 172.21.15.26:0/3273609165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee7813fcb0 0x7fee780cd6e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fee600097a0 tx=0x7fee6002b790 comp rx=0 tx=0).stop 2024-01-15T18:05:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.948+0000 7fee7e000640 1 -- 172.21.15.26:0/3273609165 shutdown_connections 2024-01-15T18:05:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.948+0000 7fee7e000640 1 --2- 172.21.15.26:0/3273609165 >> v2:172.21.15.124:3300/0 conn(0x7fee780cdc20 0x7fee780b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.948+0000 7fee7e000640 1 --2- 172.21.15.26:0/3273609165 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee7813fcb0 0x7fee780cd6e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.948+0000 7fee7e000640 1 -- 172.21.15.26:0/3273609165 >> 172.21.15.26:0/3273609165 conn(0x7fee7806a7d0 msgr2=0x7fee7806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.948+0000 7fee7e000640 1 -- 172.21.15.26:0/3273609165 shutdown_connections 2024-01-15T18:05:41.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.948+0000 7fee7e000640 1 -- 172.21.15.26:0/3273609165 wait complete. 2024-01-15T18:05:41.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.950+0000 7fee7e000640 1 Processor -- start 2024-01-15T18:05:41.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.950+0000 7fee7e000640 1 -- start start 2024-01-15T18:05:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.951+0000 7fee7e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee780cdc20 0x7fee781f9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.951+0000 7fee7e000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee7813cf10 0x7fee7813d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.951+0000 7fee7e000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee7813fcb0 0x7fee7813d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.951+0000 7fee7e000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee781fa510 con 0x7fee780cdc20 2024-01-15T18:05:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.951+0000 7fee7e000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee78155cf0 con 0x7fee7813cf10 2024-01-15T18:05:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.951+0000 7fee7e000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7813e140 con 0x7fee7813fcb0 2024-01-15T18:05:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.951+0000 7fee7cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee780cdc20 0x7fee781f9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.951+0000 7fee7cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee780cdc20 0x7fee781f9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36810/0 (socket says 172.21.15.26:36810) 2024-01-15T18:05:41.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.951+0000 7fee7cc00640 1 -- 172.21.15.26:0/2604252317 learned_addr learned my addr 172.21.15.26:0/2604252317 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee77e00640 1 --2- 172.21.15.26:0/2604252317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee7813cf10 0x7fee7813d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee7cc00640 1 -- 172.21.15.26:0/2604252317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee7813fcb0 msgr2=0x7fee7813d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee7cc00640 1 --2- 172.21.15.26:0/2604252317 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee7813fcb0 0x7fee7813d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee7cc00640 1 -- 172.21.15.26:0/2604252317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee7813cf10 msgr2=0x7fee7813d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee7cc00640 1 --2- 172.21.15.26:0/2604252317 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee7813cf10 0x7fee7813d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee7cc00640 1 -- 172.21.15.26:0/2604252317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee60009400 con 0x7fee780cdc20 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee7cc00640 1 --2- 172.21.15.26:0/2604252317 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee780cdc20 0x7fee781f9fd0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fee680018d0 tx=0x7fee6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee74c00640 1 -- 172.21.15.26:0/2604252317 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68012400 con 0x7fee780cdc20 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee74c00640 1 -- 172.21.15.26:0/2604252317 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee68012560 con 0x7fee780cdc20 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.952+0000 7fee74c00640 1 -- 172.21.15.26:0/2604252317 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68012910 con 0x7fee780cdc20 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.953+0000 7fee7e000640 1 -- 172.21.15.26:0/2604252317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee78153370 con 0x7fee780cdc20 2024-01-15T18:05:41.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.953+0000 7fee7e000640 1 -- 172.21.15.26:0/2604252317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee781fa7f0 con 0x7fee780cdc20 2024-01-15T18:05:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.954+0000 7fee7e000640 1 -- 172.21.15.26:0/2604252317 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee78067450 con 0x7fee780cdc20 2024-01-15T18:05:41.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.955+0000 7fee74c00640 1 -- 172.21.15.26:0/2604252317 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fee68012a70 con 0x7fee780cdc20 2024-01-15T18:05:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.956+0000 7fee74c00640 1 --2- 172.21.15.26:0/2604252317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fee5006be90 0x7fee5006e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:41.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.956+0000 7fee74c00640 1 -- 172.21.15.26:0/2604252317 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fee68089010 con 0x7fee780cdc20 2024-01-15T18:05:41.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.956+0000 7fee77e00640 1 --2- 172.21.15.26:0/2604252317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fee5006be90 0x7fee5006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:41.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.957+0000 7fee77e00640 1 --2- 172.21.15.26:0/2604252317 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fee5006be90 0x7fee5006e350 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fee60034000 tx=0x7fee6002f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:41.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:41.960+0000 7fee74c00640 1 -- 172.21.15.26:0/2604252317 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee680556d0 con 0x7fee780cdc20 2024-01-15T18:05:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.198+0000 7fee7e000640 1 -- 172.21.15.26:0/2604252317 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone4183", "target": ["mon-mgr", ""]}) v1 -- 0x7fee780ce0b0 con 0x7fee5006be90 2024-01-15T18:05:42.224 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:05:42.221+0000 7f8f54600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-01-15T18:05:42.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.220+0000 7fee74c00640 1 -- 172.21.15.26:0/2604252317 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) v1 ==== 60+0+0 (secure 0 0 0) 0x7fee60030870 con 0x7fee5006be90 2024-01-15T18:05:42.225 INFO:teuthology.orchestra.run.smithi026.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-01-15T18:05:42.243 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-01-15T18:05:42.243 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone4184 2024-01-15T18:05:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.465+0000 7f12b9a00640 1 Processor -- start 2024-01-15T18:05:42.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.466+0000 7f12b9a00640 1 -- start start 2024-01-15T18:05:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.467+0000 7f12b9a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12b40b35a0 0x7f12b40cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.467+0000 7f12b9a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b40ce440 con 0x7f12b413fcb0 2024-01-15T18:05:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.467+0000 7f12b9a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b40ce720 con 0x7f12b413cf10 2024-01-15T18:05:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.467+0000 7f12b9a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40cea00 con 0x7f12b40b35a0 2024-01-15T18:05:42.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.468+0000 7f12b3400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f12b413fcb0 0x7f12b40cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34350/0 (socket says 172.21.15.26:34350) 2024-01-15T18:05:42.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.468+0000 7f12b3400640 1 -- 172.21.15.26:0/2952098892 learned_addr learned my addr 172.21.15.26:0/2952098892 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.468+0000 7f12b9000640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12b40b35a0 0x7f12b40cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.468+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428677407 0 0) 0x7f12b40ce440 con 0x7f12b413fcb0 2024-01-15T18:05:42.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1294005730 con 0x7f12b413fcb0 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3322342192 0 0) 0x7f1294005730 con 0x7f12b413fcb0 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12b40b35a0 msgr2=0x7f12b40cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12b40b35a0 0x7f12b40cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 >> v1:172.21.15.124:6789/0 conn(0x7f12b413cf10 legacy=0x7f12b413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b40b1d90 con 0x7f12b413fcb0 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f129c001670 con 0x7f12b413fcb0 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 >> v1:172.21.15.26:6789/0 conn(0x7f12b413fcb0 legacy=0x7f12b40cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12b40b35a0 0x7f1294005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12940060f0 0x7f12940069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1294006f10 0x7f1294007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b40cea00 con 0x7f1294006f10 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129400dbf0 con 0x7f12940060f0 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.469+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129400ddb0 con 0x7f12b40b35a0 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.470+0000 7f12b9000640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1294006f10 0x7f1294007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.470+0000 7f12b3400640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12940060f0 0x7f12940069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.470+0000 7f12b3e00640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12b40b35a0 0x7f1294005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.470+0000 7f12b9000640 1 -- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12b40b35a0 msgr2=0x7f1294005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.470+0000 7f12b9000640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12b40b35a0 0x7f1294005bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.470+0000 7f12b9000640 1 -- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12940060f0 msgr2=0x7f12940069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.470+0000 7f12b9000640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12940060f0 0x7f12940069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.470+0000 7f12b9000640 1 -- 172.21.15.26:0/2952098892 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294005730 con 0x7f1294006f10 2024-01-15T18:05:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.471+0000 7f12b9000640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1294006f10 0x7f1294007390 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009fc0 tx=0x7f1294007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbed5ebd4a51037 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.471+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac02cbf0 con 0x7f1294006f10 2024-01-15T18:05:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.471+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac00f040 con 0x7f1294006f10 2024-01-15T18:05:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.471+0000 7f12b2a00640 1 -- 172.21.15.26:0/2952098892 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac034440 con 0x7f1294006f10 2024-01-15T18:05:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.472+0000 7f12b9a00640 1 -- 172.21.15.26:0/2952098892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1294006f10 msgr2=0x7f1294007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.472+0000 7f12b9a00640 1 --2- 172.21.15.26:0/2952098892 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1294006f10 0x7f1294007390 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f12ac009fc0 tx=0x7f1294007a00 comp rx=0 tx=0).stop 2024-01-15T18:05:42.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.472+0000 7f12b9000640 1 -- 172.21.15.26:0/2952098892 reap_dead start 2024-01-15T18:05:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.472+0000 7f12b9a00640 1 -- 172.21.15.26:0/2952098892 shutdown_connections 2024-01-15T18:05:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.472+0000 7f12b9a00640 1 -- 172.21.15.26:0/2952098892 >> 172.21.15.26:0/2952098892 conn(0x7f12b406a7d0 msgr2=0x7f12b406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.472+0000 7f12b9a00640 1 -- 172.21.15.26:0/2952098892 shutdown_connections 2024-01-15T18:05:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.472+0000 7f12b9a00640 1 -- 172.21.15.26:0/2952098892 wait complete. 2024-01-15T18:05:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.474+0000 7f12b9a00640 1 Processor -- start 2024-01-15T18:05:42.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.474+0000 7f12b9a00640 1 -- start start 2024-01-15T18:05:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.475+0000 7f12b9a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1294006f10 0x7f12b41eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.475+0000 7f12b9a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b4139480 0x7f12b41f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.475+0000 7f12b9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b41f1c30 0x7f12b413d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.475+0000 7f12b9a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b413aae0 con 0x7f12b41f1c30 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.475+0000 7f12b9a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b41ef0f0 con 0x7f12b4139480 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.475+0000 7f12b9a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b4139e60 con 0x7f1294006f10 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.475+0000 7f12b9000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b41f1c30 0x7f12b413d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.475+0000 7f12b9000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b41f1c30 0x7f12b413d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36828/0 (socket says 172.21.15.26:36828) 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.475+0000 7f12b9000640 1 -- 172.21.15.26:0/851301222 learned_addr learned my addr 172.21.15.26:0/851301222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.476+0000 7f12b3e00640 1 --2- 172.21.15.26:0/851301222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1294006f10 0x7f12b41eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.476+0000 7f12b3400640 1 --2- 172.21.15.26:0/851301222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b4139480 0x7f12b41f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.476+0000 7f12b9000640 1 -- 172.21.15.26:0/851301222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1294006f10 msgr2=0x7f12b41eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.476+0000 7f12b9000640 1 --2- 172.21.15.26:0/851301222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1294006f10 0x7f12b41eebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.476+0000 7f12b9000640 1 -- 172.21.15.26:0/851301222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b4139480 msgr2=0x7f12b41f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.476+0000 7f12b9000640 1 --2- 172.21.15.26:0/851301222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12b4139480 0x7f12b41f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.476+0000 7f12b9000640 1 -- 172.21.15.26:0/851301222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b413a140 con 0x7f12b41f1c30 2024-01-15T18:05:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.476+0000 7f12b9000640 1 --2- 172.21.15.26:0/851301222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12b41f1c30 0x7f12b413d3d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f12ac037040 tx=0x7f12ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.477+0000 7f12abe00640 1 -- 172.21.15.26:0/851301222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac034ad0 con 0x7f12b41f1c30 2024-01-15T18:05:42.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.477+0000 7f12abe00640 1 -- 172.21.15.26:0/851301222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ac034c30 con 0x7f12b41f1c30 2024-01-15T18:05:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.477+0000 7f12abe00640 1 -- 172.21.15.26:0/851301222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ac003990 con 0x7f12b41f1c30 2024-01-15T18:05:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.477+0000 7f12b9a00640 1 -- 172.21.15.26:0/851301222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b413a420 con 0x7f12b41f1c30 2024-01-15T18:05:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.477+0000 7f12b9a00640 1 -- 172.21.15.26:0/851301222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b4138460 con 0x7f12b41f1c30 2024-01-15T18:05:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.478+0000 7f12b9a00640 1 -- 172.21.15.26:0/851301222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b4067450 con 0x7f12b41f1c30 2024-01-15T18:05:42.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.479+0000 7f12abe00640 1 -- 172.21.15.26:0/851301222 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f12ac047460 con 0x7f12b41f1c30 2024-01-15T18:05:42.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.480+0000 7f12abe00640 1 --2- 172.21.15.26:0/851301222 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f128c06be90 0x7f128c06e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.480+0000 7f12abe00640 1 -- 172.21.15.26:0/851301222 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f12ac0aa490 con 0x7f12b41f1c30 2024-01-15T18:05:42.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.481+0000 7f12b3e00640 1 --2- 172.21.15.26:0/851301222 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f128c06be90 0x7f128c06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.482+0000 7f12b3e00640 1 --2- 172.21.15.26:0/851301222 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f128c06be90 0x7f128c06e350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f12a4000f60 tx=0x7f12a4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.484+0000 7f12abe00640 1 -- 172.21.15.26:0/851301222 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12ac076d50 con 0x7f12b41f1c30 2024-01-15T18:05:42.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.717+0000 7f12b9a00640 1 -- 172.21.15.26:0/851301222 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone4184", "target": ["mon-mgr", ""]}) v1 -- 0x7f12b41386b0 con 0x7f128c06be90 2024-01-15T18:05:42.740 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:05:42.736+0000 7f8f54600640 -1 mgr.server reply reply (22) Invalid argument cannot cancel -- clone finished (check clone status) 2024-01-15T18:05:42.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.737+0000 7f12abe00640 1 -- 172.21.15.26:0/851301222 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -22 cannot cancel -- clone finished (check clone status)) v1 ==== 60+0+0 (secure 0 0 0) 0x7f12b41386b0 con 0x7f128c06be90 2024-01-15T18:05:42.741 INFO:teuthology.orchestra.run.smithi026.stderr:Error EINVAL: cannot cancel -- clone finished (check clone status) 2024-01-15T18:05:42.754 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-01-15T18:05:42.754 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone4185 2024-01-15T18:05:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.977+0000 7f57fa200640 1 Processor -- start 2024-01-15T18:05:42.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.977+0000 7f57fa200640 1 -- start start 2024-01-15T18:05:42.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.979+0000 7f57fa200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f41505b0 0x7f57f40b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.979+0000 7f57fa200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f57f406d6d0 0x7f57f40ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.979+0000 7f57fa200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f40b1e30 con 0x7f57f415a690 2024-01-15T18:05:42.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.979+0000 7f57fa200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f40b20a0 con 0x7f57f406d6d0 2024-01-15T18:05:42.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.979+0000 7f57fa200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40b2380 con 0x7f57f41505b0 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.979+0000 7f57f3e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f57f415a690 0x7f57f406cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34354/0 (socket says 172.21.15.26:34354) 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.979+0000 7f57f3e00640 1 -- 172.21.15.26:0/3167796909 learned_addr learned my addr 172.21.15.26:0/3167796909 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f9800640 1 --2- 172.21.15.26:0/3167796909 >> v2:172.21.15.124:3300/0 conn(0x7f57f406d6d0 0x7f57f40ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f8e00640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f41505b0 0x7f57f40b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455381102 0 0) 0x7f57f40b1e30 con 0x7f57f415a690 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57d4003520 con 0x7f57f415a690 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 440798938 0 0) 0x7f57d4003520 con 0x7f57f415a690 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f41505b0 msgr2=0x7f57f40b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f41505b0 0x7f57f40b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 >> v2:172.21.15.124:3300/0 conn(0x7f57f406d6d0 msgr2=0x7f57f40ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 --2- 172.21.15.26:0/3167796909 >> v2:172.21.15.124:3300/0 conn(0x7f57f406d6d0 0x7f57f40ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40b2660 con 0x7f57f415a690 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f57dc0019f0 con 0x7f57f415a690 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 >> v1:172.21.15.26:6789/0 conn(0x7f57f415a690 legacy=0x7f57f406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f41505b0 0x7f57d4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57d4004d00 0x7f57d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40b20a0 con 0x7f57d4003ef0 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40b2380 con 0x7f57d4004d00 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.980+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d400b980 con 0x7f57f41505b0 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f3e00640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f8e00640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f41505b0 0x7f57d4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f3e00640 1 -- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f41505b0 msgr2=0x7f57d4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f9800640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57d4004d00 0x7f57d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f3e00640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f41505b0 0x7f57d4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f3e00640 1 -- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57d4004d00 msgr2=0x7f57d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f3e00640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57d4004d00 0x7f57d4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f3e00640 1 -- 172.21.15.26:0/3167796909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4003520 con 0x7f57d4003ef0 2024-01-15T18:05:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f8e00640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f41505b0 0x7f57d4003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:42.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.981+0000 7f57f9800640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57d4004d00 0x7f57d4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.982+0000 7f57f3e00640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57d40047c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00a860 tx=0x7f57d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c110afa958c5e78 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.983+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc034630 con 0x7f57d4003ef0 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.983+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc02f070 con 0x7f57d4003ef0 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.983+0000 7f57f3400640 1 -- 172.21.15.26:0/3167796909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc034a30 con 0x7f57d4003ef0 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.983+0000 7f57fa200640 1 -- 172.21.15.26:0/3167796909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 msgr2=0x7f57d40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.983+0000 7f57fa200640 1 --2- 172.21.15.26:0/3167796909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57d40047c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00a860 tx=0x7f57d40057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.983+0000 7f57f9800640 1 -- 172.21.15.26:0/3167796909 reap_dead start 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.984+0000 7f57fa200640 1 -- 172.21.15.26:0/3167796909 shutdown_connections 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.984+0000 7f57fa200640 1 -- 172.21.15.26:0/3167796909 >> 172.21.15.26:0/3167796909 conn(0x7f57f406a7d0 msgr2=0x7f57f406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.984+0000 7f57fa200640 1 -- 172.21.15.26:0/3167796909 shutdown_connections 2024-01-15T18:05:42.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.984+0000 7f57fa200640 1 -- 172.21.15.26:0/3167796909 wait complete. 2024-01-15T18:05:42.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.985+0000 7f57fa200640 1 Processor -- start 2024-01-15T18:05:42.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.986+0000 7f57fa200640 1 -- start start 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57fa200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57f415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57fa200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f413b720 0x7f57f414b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57fa200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57f40703d0 0x7f57f4070850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57fa200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4070d90 con 0x7f57d4003ef0 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57fa200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4142f20 con 0x7f57f40703d0 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57fa200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f414b610 con 0x7f57f413b720 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57f415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57f415ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36844/0 (socket says 172.21.15.26:36844) 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f8e00640 1 -- 172.21.15.26:0/3831407399 learned_addr learned my addr 172.21.15.26:0/3831407399 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f9800640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57f40703d0 0x7f57f4070850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f3e00640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f413b720 0x7f57f414b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f8e00640 1 -- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f413b720 msgr2=0x7f57f414b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f8e00640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f413b720 0x7f57f414b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f8e00640 1 -- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57f40703d0 msgr2=0x7f57f4070850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f8e00640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57f40703d0 0x7f57f4070850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f8e00640 1 -- 172.21.15.26:0/3831407399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f413bba0 con 0x7f57d4003ef0 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.987+0000 7f57f3e00640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f413b720 0x7f57f414b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.988+0000 7f57f8e00640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57f415ca80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f57e40108b0 tx=0x7f57e4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.988+0000 7f57f0c00640 1 -- 172.21.15.26:0/3831407399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e400eb60 con 0x7f57d4003ef0 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.988+0000 7f57f0c00640 1 -- 172.21.15.26:0/3831407399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e400ecc0 con 0x7f57d4003ef0 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.988+0000 7f57f0c00640 1 -- 172.21.15.26:0/3831407399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e401c400 con 0x7f57d4003ef0 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.988+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f414b890 con 0x7f57d4003ef0 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.988+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f41428d0 con 0x7f57d4003ef0 2024-01-15T18:05:42.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.990+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f4067450 con 0x7f57d4003ef0 2024-01-15T18:05:42.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.991+0000 7f57f0c00640 1 -- 172.21.15.26:0/3831407399 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f57e401c560 con 0x7f57d4003ef0 2024-01-15T18:05:42.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.992+0000 7f57f0c00640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57cc06be90 0x7f57cc06e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:42.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.992+0000 7f57f0c00640 1 -- 172.21.15.26:0/3831407399 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f57e4088d30 con 0x7f57d4003ef0 2024-01-15T18:05:42.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.992+0000 7f57f3e00640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57cc06be90 0x7f57cc06e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:42.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.993+0000 7f57f3e00640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57cc06be90 0x7f57cc06e350 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f57dc034000 tx=0x7f57dc009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:43.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:42.996+0000 7f57f0c00640 1 -- 172.21.15.26:0/3831407399 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57e40554a0 con 0x7f57d4003ef0 2024-01-15T18:05:43.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:43.226+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone4185", "target": ["mon-mgr", ""]}) v1 -- 0x7f57f40b41e0 con 0x7f57cc06be90 2024-01-15T18:05:51.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.155+0000 7f57f0c00640 1 -- 172.21.15.26:0/3831407399 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f57dc0018b0 con 0x7f57cc06be90 2024-01-15T18:05:51.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.158+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57cc06be90 msgr2=0x7f57cc06e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.158+0000 7f57fa200640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57cc06be90 0x7f57cc06e350 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f57dc034000 tx=0x7f57dc009030 comp rx=0 tx=0).stop 2024-01-15T18:05:51.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.158+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 msgr2=0x7f57f415ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.158+0000 7f57fa200640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57f415ca80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f57e40108b0 tx=0x7f57e4010d70 comp rx=0 tx=0).stop 2024-01-15T18:05:51.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.160+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 shutdown_connections 2024-01-15T18:05:51.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.160+0000 7f57fa200640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f57f413b720 0x7f57f414b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.160+0000 7f57fa200640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57cc06be90 0x7f57cc06e350 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.160+0000 7f57fa200640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f57f40703d0 0x7f57f4070850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.160+0000 7f57fa200640 1 --2- 172.21.15.26:0/3831407399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f57d4003ef0 0x7f57f415ca80 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.160+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 >> 172.21.15.26:0/3831407399 conn(0x7f57f406a7d0 msgr2=0x7f57f4158f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:51.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.160+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 shutdown_connections 2024-01-15T18:05:51.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.160+0000 7f57fa200640 1 -- 172.21.15.26:0/3831407399 wait complete. 2024-01-15T18:05:51.188 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone4186 2024-01-15T18:05:51.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.409+0000 7ff132a00640 1 Processor -- start 2024-01-15T18:05:51.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.409+0000 7ff132a00640 1 -- start start 2024-01-15T18:05:51.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.411+0000 7ff132a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c14b2c0 0x7ff12c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.411+0000 7ff132a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12c0cfbf0 con 0x7ff12c071070 2024-01-15T18:05:51.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.411+0000 7ff132a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12c071690 con 0x7ff12c0cf7d0 2024-01-15T18:05:51.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.411+0000 7ff132a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c14b6a0 con 0x7ff12c14b2c0 2024-01-15T18:05:51.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.411+0000 7ff130a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff12c071070 0x7ff12c0cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:34366/0 (socket says 172.21.15.26:34366) 2024-01-15T18:05:51.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.411+0000 7ff130a00640 1 -- 172.21.15.26:0/637741248 learned_addr learned my addr 172.21.15.26:0/637741248 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:05:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.411+0000 7ff12be00640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c14b2c0 0x7ff12c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710309323 0 0) 0x7ff12c0cfbf0 con 0x7ff12c071070 2024-01-15T18:05:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff10c003520 con 0x7ff12c071070 2024-01-15T18:05:51.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3992374843 0 0) 0x7ff10c003520 con 0x7ff12c071070 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c14b2c0 msgr2=0x7ff12c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c14b2c0 0x7ff12c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 >> v1:172.21.15.124:6789/0 conn(0x7ff12c0cf7d0 legacy=0x7ff12c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c071970 con 0x7ff12c071070 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7ff124001670 con 0x7ff12c071070 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 >> v1:172.21.15.26:6789/0 conn(0x7ff12c071070 legacy=0x7ff12c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c14b2c0 0x7ff10c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff10c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10c004d00 0x7ff10c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c14b6a0 con 0x7ff10c003f70 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c00bac0 con 0x7ff10c004d00 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.412+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10c00bc80 con 0x7ff12c14b2c0 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.413+0000 7ff12b400640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff10c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.413+0000 7ff12be00640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c14b2c0 0x7ff10c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.413+0000 7ff130a00640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10c004d00 0x7ff10c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.413+0000 7ff12b400640 1 -- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c14b2c0 msgr2=0x7ff10c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.413+0000 7ff12b400640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c14b2c0 0x7ff10c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.413+0000 7ff12b400640 1 -- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10c004d00 msgr2=0x7ff10c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.413+0000 7ff12b400640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10c004d00 0x7ff10c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.413+0000 7ff12b400640 1 -- 172.21.15.26:0/637741248 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10c003520 con 0x7ff10c003f70 2024-01-15T18:05:51.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.413+0000 7ff130a00640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff10c004d00 0x7ff10c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:05:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.414+0000 7ff12b400640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff10c0047c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff114001750 tx=0x7ff10c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dfb939a274f561 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.414+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1140052d0 con 0x7ff10c003f70 2024-01-15T18:05:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.415+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11402e050 con 0x7ff10c003f70 2024-01-15T18:05:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.415+0000 7ff12aa00640 1 -- 172.21.15.26:0/637741248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114003830 con 0x7ff10c003f70 2024-01-15T18:05:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.415+0000 7ff132a00640 1 -- 172.21.15.26:0/637741248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 msgr2=0x7ff10c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.415+0000 7ff132a00640 1 --2- 172.21.15.26:0/637741248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff10c0047c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff114001750 tx=0x7ff10c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.415+0000 7ff130a00640 1 -- 172.21.15.26:0/637741248 reap_dead start 2024-01-15T18:05:51.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.415+0000 7ff132a00640 1 -- 172.21.15.26:0/637741248 shutdown_connections 2024-01-15T18:05:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.415+0000 7ff132a00640 1 -- 172.21.15.26:0/637741248 >> 172.21.15.26:0/637741248 conn(0x7ff12c06a800 msgr2=0x7ff12c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:05:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.415+0000 7ff132a00640 1 -- 172.21.15.26:0/637741248 shutdown_connections 2024-01-15T18:05:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.415+0000 7ff132a00640 1 -- 172.21.15.26:0/637741248 wait complete. 2024-01-15T18:05:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.417+0000 7ff132a00640 1 Processor -- start 2024-01-15T18:05:51.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.417+0000 7ff132a00640 1 -- start start 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.418+0000 7ff132a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff12c1647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.418+0000 7ff132a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c148d90 0x7ff12c149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.418+0000 7ff132a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12c13f2b0 0x7ff12c13f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.418+0000 7ff132a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c149750 con 0x7ff10c003f70 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.418+0000 7ff132a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1f6bf0 con 0x7ff12c13f2b0 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.418+0000 7ff132a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c162930 con 0x7ff12c148d90 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.418+0000 7ff12be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff12c1647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.418+0000 7ff12be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff12c1647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:36864/0 (socket says 172.21.15.26:36864) 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.418+0000 7ff12be00640 1 -- 172.21.15.26:0/3422290242 learned_addr learned my addr 172.21.15.26:0/3422290242 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:05:51.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.419+0000 7ff12b400640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c148d90 0x7ff12c149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.419+0000 7ff130a00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12c13f2b0 0x7ff12c13f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.419+0000 7ff12be00640 1 -- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c148d90 msgr2=0x7ff12c149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.419+0000 7ff12be00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c148d90 0x7ff12c149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.419+0000 7ff12be00640 1 -- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12c13f2b0 msgr2=0x7ff12c13f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.419+0000 7ff12be00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12c13f2b0 0x7ff12c13f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.419+0000 7ff12be00640 1 -- 172.21.15.26:0/3422290242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c161d60 con 0x7ff10c003f70 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.419+0000 7ff12be00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff12c1647c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff11c007f60 tx=0x7ff11c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.419+0000 7ff123e00640 1 -- 172.21.15.26:0/3422290242 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00e030 con 0x7ff10c003f70 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.420+0000 7ff123e00640 1 -- 172.21.15.26:0/3422290242 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c016470 con 0x7ff10c003f70 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.420+0000 7ff123e00640 1 -- 172.21.15.26:0/3422290242 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c016820 con 0x7ff10c003f70 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.420+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c162000 con 0x7ff10c003f70 2024-01-15T18:05:51.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.420+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c1625c0 con 0x7ff10c003f70 2024-01-15T18:05:51.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.421+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c0675d0 con 0x7ff10c003f70 2024-01-15T18:05:51.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.422+0000 7ff123e00640 1 -- 172.21.15.26:0/3422290242 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7ff11c016b90 con 0x7ff10c003f70 2024-01-15T18:05:51.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.423+0000 7ff123e00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10406bec0 0x7ff10406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:05:51.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.423+0000 7ff123e00640 1 -- 172.21.15.26:0/3422290242 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7ff11c00f070 con 0x7ff10c003f70 2024-01-15T18:05:51.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.423+0000 7ff12b400640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10406bec0 0x7ff10406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:05:51.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.425+0000 7ff12b400640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10406bec0 0x7ff10406e380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff114033000 tx=0x7ff114009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:05:51.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.427+0000 7ff123e00640 1 -- 172.21.15.26:0/3422290242 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff11c056040 con 0x7ff10c003f70 2024-01-15T18:05:51.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:05:51.659+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone4186", "target": ["mon-mgr", ""]}) v1 -- 0x7ff12c0cbc40 con 0x7ff10406bec0 2024-01-15T18:05:59.311 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:05:59.320 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:06:01.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.251+0000 7ff123e00640 1 -- 172.21.15.26:0/3422290242 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff1140054c0 con 0x7ff10406bec0 2024-01-15T18:06:01.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10406bec0 msgr2=0x7ff10406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10406bec0 0x7ff10406e380 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff114033000 tx=0x7ff114009030 comp rx=0 tx=0).stop 2024-01-15T18:06:01.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 msgr2=0x7ff12c1647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff12c1647c0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff11c007f60 tx=0x7ff11c007f90 comp rx=0 tx=0).stop 2024-01-15T18:06:01.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 shutdown_connections 2024-01-15T18:06:01.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff12c148d90 0x7ff12c149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10406bec0 0x7ff10406e380 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12c13f2b0 0x7ff12c13f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 --2- 172.21.15.26:0/3422290242 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff10c003f70 0x7ff12c1647c0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 >> 172.21.15.26:0/3422290242 conn(0x7ff12c06a800 msgr2=0x7ff12c0afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:01.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 shutdown_connections 2024-01-15T18:06:01.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.255+0000 7ff132a00640 1 -- 172.21.15.26:0/3422290242 wait complete. 2024-01-15T18:06:01.281 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone4187 2024-01-15T18:06:01.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.499+0000 7fa197e00640 1 Processor -- start 2024-01-15T18:06:01.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.499+0000 7fa197e00640 1 -- start start 2024-01-15T18:06:01.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.501+0000 7fa197e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980ccb50 0x7fa198070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.501+0000 7fa197e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19815f4e0 con 0x7fa19815bf10 2024-01-15T18:06:01.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.501+0000 7fa197e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19815c3c0 con 0x7fa1980adf20 2024-01-15T18:06:01.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.501+0000 7fa197e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19815f970 con 0x7fa1980ccb50 2024-01-15T18:06:01.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.501+0000 7fa197400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa19815bf10 0x7fa198143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56844/0 (socket says 172.21.15.26:56844) 2024-01-15T18:06:01.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.501+0000 7fa197400640 1 -- 172.21.15.26:0/892292847 learned_addr learned my addr 172.21.15.26:0/892292847 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:06:01.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa18fe00640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980ccb50 0x7fa198070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547186425 0 0) 0x7fa19815f4e0 con 0x7fa19815bf10 2024-01-15T18:06:01.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa178003520 con 0x7fa19815bf10 2024-01-15T18:06:01.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1240874545 0 0) 0x7fa178003520 con 0x7fa19815bf10 2024-01-15T18:06:01.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980ccb50 msgr2=0x7fa198070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:01.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980ccb50 0x7fa198070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 >> v1:172.21.15.124:6789/0 conn(0x7fa1980adf20 legacy=0x7fa1980cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198071130 con 0x7fa19815bf10 2024-01-15T18:06:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa1900019f0 con 0x7fa19815bf10 2024-01-15T18:06:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 >> v1:172.21.15.26:6789/0 conn(0x7fa19815bf10 legacy=0x7fa198143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1780039a0 0x7fa178003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980ccb50 0x7fa1780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa178004d00 0x7fa178005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19815f970 con 0x7fa1780039a0 2024-01-15T18:06:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17800bab0 con 0x7fa178004d00 2024-01-15T18:06:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.502+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17800bc70 con 0x7fa1980ccb50 2024-01-15T18:06:01.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.503+0000 7fa196a00640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1780039a0 0x7fa178003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.503+0000 7fa18fe00640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980ccb50 0x7fa1780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.503+0000 7fa196a00640 1 -- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980ccb50 msgr2=0x7fa1780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:01.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.503+0000 7fa197400640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa178004d00 0x7fa178005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.503+0000 7fa196a00640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980ccb50 0x7fa1780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.503+0000 7fa196a00640 1 -- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa178004d00 msgr2=0x7fa178005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:01.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.503+0000 7fa196a00640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa178004d00 0x7fa178005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.503+0000 7fa196a00640 1 -- 172.21.15.26:0/892292847 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa178003520 con 0x7fa1780039a0 2024-01-15T18:06:01.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.503+0000 7fa197400640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa178004d00 0x7fa178005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:01.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.504+0000 7fa196a00640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1780039a0 0x7fa178003dc0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa188001750 tx=0x7fa1780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3196723651efdf2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:01.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880052d0 con 0x7fa1780039a0 2024-01-15T18:06:01.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18802e050 con 0x7fa1780039a0 2024-01-15T18:06:01.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa196000640 1 -- 172.21.15.26:0/892292847 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880035f0 con 0x7fa1780039a0 2024-01-15T18:06:01.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa197e00640 1 -- 172.21.15.26:0/892292847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1780039a0 msgr2=0x7fa178003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa197e00640 1 --2- 172.21.15.26:0/892292847 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1780039a0 0x7fa178003dc0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fa188001750 tx=0x7fa1780057f0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa197400640 1 -- 172.21.15.26:0/892292847 reap_dead start 2024-01-15T18:06:01.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa197e00640 1 -- 172.21.15.26:0/892292847 shutdown_connections 2024-01-15T18:06:01.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa197e00640 1 -- 172.21.15.26:0/892292847 >> 172.21.15.26:0/892292847 conn(0x7fa19806a7d0 msgr2=0x7fa19806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:01.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa197e00640 1 -- 172.21.15.26:0/892292847 shutdown_connections 2024-01-15T18:06:01.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.505+0000 7fa197e00640 1 -- 172.21.15.26:0/892292847 wait complete. 2024-01-15T18:06:01.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.507+0000 7fa197e00640 1 Processor -- start 2024-01-15T18:06:01.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.507+0000 7fa197e00640 1 -- start start 2024-01-15T18:06:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.508+0000 7fa197e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1780039a0 0x7fa198140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.508+0000 7fa197e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980caa20 0x7fa19813eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.508+0000 7fa197e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1980c48c0 0x7fa1980713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.508+0000 7fa197e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980c5480 con 0x7fa1980c48c0 2024-01-15T18:06:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.508+0000 7fa197e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980c51a0 con 0x7fa1780039a0 2024-01-15T18:06:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.508+0000 7fa197e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1981401e0 con 0x7fa1980caa20 2024-01-15T18:06:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa197400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1980c48c0 0x7fa1980713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa197400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1980c48c0 0x7fa1980713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33536/0 (socket says 172.21.15.26:33536) 2024-01-15T18:06:01.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa197400640 1 -- 172.21.15.26:0/1371271053 learned_addr learned my addr 172.21.15.26:0/1371271053 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa18fe00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980caa20 0x7fa19813eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa197400640 1 -- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980caa20 msgr2=0x7fa19813eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa196a00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1780039a0 0x7fa198140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa197400640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980caa20 0x7fa19813eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa197400640 1 -- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1780039a0 msgr2=0x7fa198140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa197400640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1780039a0 0x7fa198140cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa197400640 1 -- 172.21.15.26:0/1371271053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1981404c0 con 0x7fa1980c48c0 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa18fe00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980caa20 0x7fa19813eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa196a00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1780039a0 0x7fa198140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.509+0000 7fa197400640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1980c48c0 0x7fa1980713c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa19815c2f0 tx=0x7fa19000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:01.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.510+0000 7fa18ea00640 1 -- 172.21.15.26:0/1371271053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19000de70 con 0x7fa1980c48c0 2024-01-15T18:06:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.510+0000 7fa18ea00640 1 -- 172.21.15.26:0/1371271053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19000bb20 con 0x7fa1980c48c0 2024-01-15T18:06:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.510+0000 7fa18ea00640 1 -- 172.21.15.26:0/1371271053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19000bed0 con 0x7fa1980c48c0 2024-01-15T18:06:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.510+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1981407a0 con 0x7fa1980c48c0 2024-01-15T18:06:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.510+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1980c4d40 con 0x7fa1980c48c0 2024-01-15T18:06:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.511+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198140040 con 0x7fa1980c48c0 2024-01-15T18:06:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.512+0000 7fa18ea00640 1 -- 172.21.15.26:0/1371271053 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fa190007070 con 0x7fa1980c48c0 2024-01-15T18:06:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.513+0000 7fa18ea00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa17006bec0 0x7fa17006e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.513+0000 7fa18ea00640 1 -- 172.21.15.26:0/1371271053 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fa190088900 con 0x7fa1980c48c0 2024-01-15T18:06:01.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.514+0000 7fa196a00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa17006bec0 0x7fa17006e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.515+0000 7fa196a00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa17006bec0 0x7fa17006e380 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa188033000 tx=0x7fa188009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:01.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.518+0000 7fa18ea00640 1 -- 172.21.15.26:0/1371271053 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa198140040 con 0x7fa1980c48c0 2024-01-15T18:06:01.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:01.749+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone4187", "target": ["mon-mgr", ""]}) v1 -- 0x7fa19815f170 con 0x7fa17006bec0 2024-01-15T18:06:02.264 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:06:02.260+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:02.264 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:06:02.260+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:02.264 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:06:02.260+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:02.264 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:06:02.260+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:02.264 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:06:02.260+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:06:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.545+0000 7fa18ea00640 1 -- 172.21.15.26:0/1371271053 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa1880054a0 con 0x7fa17006bec0 2024-01-15T18:06:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa17006bec0 msgr2=0x7fa17006e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa17006bec0 0x7fa17006e380 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa188033000 tx=0x7fa188009030 comp rx=0 tx=0).stop 2024-01-15T18:06:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1980c48c0 msgr2=0x7fa1980713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1980c48c0 0x7fa1980713c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fa19815c2f0 tx=0x7fa19000b4a0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 shutdown_connections 2024-01-15T18:06:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa1980caa20 0x7fa19813eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa17006bec0 0x7fa17006e380 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa1980c48c0 0x7fa1980713c0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 --2- 172.21.15.26:0/1371271053 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa1780039a0 0x7fa198140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 >> 172.21.15.26:0/1371271053 conn(0x7fa19806a7d0 msgr2=0x7fa19815df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.548+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 shutdown_connections 2024-01-15T18:06:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.549+0000 7fa197e00640 1 -- 172.21.15.26:0/1371271053 wait complete. 2024-01-15T18:06:02.589 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone4188 2024-01-15T18:06:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.855+0000 7fa692200640 1 Processor -- start 2024-01-15T18:06:02.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.856+0000 7fa692200640 1 -- start start 2024-01-15T18:06:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.857+0000 7fa692200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c143ae0 0x7fa68c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.857+0000 7fa692200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa68c15e3b0 0x7fa68c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.858+0000 7fa692200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68c0cfc30 con 0x7fa68c0cf7d0 2024-01-15T18:06:02.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.858+0000 7fa692200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68c0b1ee0 con 0x7fa68c15e3b0 2024-01-15T18:06:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.858+0000 7fa692200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0b2780 con 0x7fa68c143ae0 2024-01-15T18:06:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.858+0000 7fa683e00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7fa68c0cf7d0 0x7fa68c15dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:46736/0 (socket says 172.21.15.26:46736) 2024-01-15T18:06:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.858+0000 7fa683e00640 1 -- 172.21.15.26:0/1555375832 learned_addr learned my addr 172.21.15.26:0/1555375832 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:06:02.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.858+0000 7fa68b400640 1 --2- 172.21.15.26:0/1555375832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c143ae0 0x7fa68c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.859+0000 7fa68be00640 1 --2- 172.21.15.26:0/1555375832 >> v2:172.21.15.124:3300/0 conn(0x7fa68c15e3b0 0x7fa68c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.860+0000 7fa68be00640 1 -- 172.21.15.26:0/1555375832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c143ae0 msgr2=0x7fa68c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.860+0000 7fa68be00640 1 --2- 172.21.15.26:0/1555375832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c143ae0 0x7fa68c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.860+0000 7fa68be00640 1 -- 172.21.15.26:0/1555375832 >> v1:172.21.15.124:6789/0 conn(0x7fa68c0cf7d0 legacy=0x7fa68c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.860+0000 7fa68be00640 1 -- 172.21.15.26:0/1555375832 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c15e990 con 0x7fa68c15e3b0 2024-01-15T18:06:02.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.863+0000 7fa68be00640 1 --2- 172.21.15.26:0/1555375832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 0x7fa68c0702c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa684009d90 tx=0x7fa68402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb522984b101dcc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.863+0000 7fa68aa00640 1 -- 172.21.15.26:0/1555375832 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68402cc60 con 0x7fa68c15e3b0 2024-01-15T18:06:02.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.864+0000 7fa68aa00640 1 -- 172.21.15.26:0/1555375832 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68402cdc0 con 0x7fa68c15e3b0 2024-01-15T18:06:02.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.864+0000 7fa68aa00640 1 -- 172.21.15.26:0/1555375832 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684034660 con 0x7fa68c15e3b0 2024-01-15T18:06:02.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.865+0000 7fa692200640 1 -- 172.21.15.26:0/1555375832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 msgr2=0x7fa68c0702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.865+0000 7fa692200640 1 --2- 172.21.15.26:0/1555375832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 0x7fa68c0702c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa684009d90 tx=0x7fa68402beb0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.865+0000 7fa692200640 1 -- 172.21.15.26:0/1555375832 shutdown_connections 2024-01-15T18:06:02.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.865+0000 7fa692200640 1 --2- 172.21.15.26:0/1555375832 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 0x7fa68c0702c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.865+0000 7fa692200640 1 --2- 172.21.15.26:0/1555375832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c143ae0 0x7fa68c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.865+0000 7fa692200640 1 -- 172.21.15.26:0/1555375832 >> 172.21.15.26:0/1555375832 conn(0x7fa68c06a800 msgr2=0x7fa68c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:02.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.867+0000 7fa692200640 1 -- 172.21.15.26:0/1555375832 shutdown_connections 2024-01-15T18:06:02.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.867+0000 7fa692200640 1 -- 172.21.15.26:0/1555375832 wait complete. 2024-01-15T18:06:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.871+0000 7fa692200640 1 Processor -- start 2024-01-15T18:06:02.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.872+0000 7fa692200640 1 -- start start 2024-01-15T18:06:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.874+0000 7fa692200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c0cf7d0 0x7fa68c140690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.874+0000 7fa692200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68c143ae0 0x7fa68c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa692200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 0x7fa68c1f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa692200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c142360 con 0x7fa68c143ae0 2024-01-15T18:06:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa692200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0cd5c0 con 0x7fa68c15e3b0 2024-01-15T18:06:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa692200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0cd8a0 con 0x7fa68c0cf7d0 2024-01-15T18:06:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa683e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68c143ae0 0x7fa68c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa68be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 0x7fa68c1f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa68b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c0cf7d0 0x7fa68c140690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa68be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 0x7fa68c1f9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:37146/0 (socket says 172.21.15.26:37146) 2024-01-15T18:06:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa68b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c0cf7d0 0x7fa68c140690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:37722/0 (socket says 172.21.15.26:37722) 2024-01-15T18:06:02.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.875+0000 7fa68be00640 1 -- 172.21.15.26:0/3972541287 learned_addr learned my addr 172.21.15.26:0/3972541287 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:02.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.876+0000 7fa68be00640 1 -- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c0cf7d0 msgr2=0x7fa68c140690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.876+0000 7fa68be00640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c0cf7d0 0x7fa68c140690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.876+0000 7fa68be00640 1 -- 172.21.15.26:0/3972541287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68c143ae0 msgr2=0x7fa68c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:02.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.876+0000 7fa68be00640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68c143ae0 0x7fa68c0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:02.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.876+0000 7fa68be00640 1 -- 172.21.15.26:0/3972541287 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa684009a70 con 0x7fa68c15e3b0 2024-01-15T18:06:02.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.877+0000 7fa68be00640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 0x7fa68c1f9c20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa684009d90 tx=0x7fa684004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.878+0000 7fa683400640 1 -- 172.21.15.26:0/3972541287 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684040650 con 0x7fa68c15e3b0 2024-01-15T18:06:02.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.878+0000 7fa683400640 1 -- 172.21.15.26:0/3972541287 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6840407b0 con 0x7fa68c15e3b0 2024-01-15T18:06:02.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.878+0000 7fa683400640 1 -- 172.21.15.26:0/3972541287 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa684040b60 con 0x7fa68c15e3b0 2024-01-15T18:06:02.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.878+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68c0cdb80 con 0x7fa68c15e3b0 2024-01-15T18:06:02.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.879+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68c0ce140 con 0x7fa68c15e3b0 2024-01-15T18:06:02.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.880+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68c0675d0 con 0x7fa68c15e3b0 2024-01-15T18:06:02.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.882+0000 7fa683400640 1 -- 172.21.15.26:0/3972541287 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 25) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fa684040cc0 con 0x7fa68c15e3b0 2024-01-15T18:06:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.883+0000 7fa683400640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66406be90 0x7fa66406e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:02.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.884+0000 7fa683400640 1 -- 172.21.15.26:0/3972541287 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fa6840ab260 con 0x7fa68c15e3b0 2024-01-15T18:06:02.888 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.884+0000 7fa68b400640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66406be90 0x7fa66406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:02.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.885+0000 7fa68b400640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66406be90 0x7fa66406e350 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009b40 tx=0x7fa67c009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:02.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:02.893+0000 7fa683400640 1 -- 172.21.15.26:0/3972541287 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa684077c30 con 0x7fa68c15e3b0 2024-01-15T18:06:03.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.192+0000 7fa683400640 1 -- 172.21.15.26:0/3972541287 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fa684078000 con 0x7fa68c15e3b0 2024-01-15T18:06:03.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.250+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone4188", "target": ["mon-mgr", ""]}) v1 -- 0x7fa68c0b2410 con 0x7fa66406be90 2024-01-15T18:06:03.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.429+0000 7fa683400640 1 -- 172.21.15.26:0/3972541287 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa67c011a60 con 0x7fa66406be90 2024-01-15T18:06:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.432+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66406be90 msgr2=0x7fa66406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.432+0000 7fa692200640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66406be90 0x7fa66406e350 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009b40 tx=0x7fa67c009190 comp rx=0 tx=0).stop 2024-01-15T18:06:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.432+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 msgr2=0x7fa68c1f9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.432+0000 7fa692200640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 0x7fa68c1f9c20 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa684009d90 tx=0x7fa684004a40 comp rx=0 tx=0).stop 2024-01-15T18:06:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.432+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 shutdown_connections 2024-01-15T18:06:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.432+0000 7fa692200640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa66406be90 0x7fa66406e350 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.433+0000 7fa692200640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa68c15e3b0 0x7fa68c1f9c20 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.433+0000 7fa692200640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa68c143ae0 0x7fa68c0ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.433+0000 7fa692200640 1 --2- 172.21.15.26:0/3972541287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa68c0cf7d0 0x7fa68c140690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.433+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 >> 172.21.15.26:0/3972541287 conn(0x7fa68c06a800 msgr2=0x7fa68c15c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.433+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 shutdown_connections 2024-01-15T18:06:03.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.433+0000 7fa692200640 1 -- 172.21.15.26:0/3972541287 wait complete. 2024-01-15T18:06:03.458 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone4189 2024-01-15T18:06:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.671+0000 7fb901800640 1 Processor -- start 2024-01-15T18:06:03.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.671+0000 7fb901800640 1 -- start start 2024-01-15T18:06:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.673+0000 7fb901800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb8fc15d310 0x7fb8fc071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.673+0000 7fb901800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb8fc0aedc0 0x7fb8fc06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.673+0000 7fb901800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8fc06d620 con 0x7fb8fc06a3e0 2024-01-15T18:06:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.673+0000 7fb901800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc06d900 con 0x7fb8fc0aedc0 2024-01-15T18:06:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.673+0000 7fb901800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8fc06dbe0 con 0x7fb8fc15d310 2024-01-15T18:06:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.673+0000 7fb900e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb8fc06a3e0 0x7fb8fc0ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:35682/0 (socket says 172.21.15.26:35682) 2024-01-15T18:06:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.674+0000 7fb900e00640 1 -- 172.21.15.26:0/2131212506 learned_addr learned my addr 172.21.15.26:0/2131212506 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:06:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.674+0000 7fb8f3e00640 1 --2- 172.21.15.26:0/2131212506 >> v2:172.21.15.26:3300/0 conn(0x7fb8fc0aedc0 0x7fb8fc06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.674+0000 7fb8fbe00640 1 --2- 172.21.15.26:0/2131212506 >> v2:172.21.15.124:3300/0 conn(0x7fb8fc15d310 0x7fb8fc071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.674+0000 7fb8fb400640 1 -- 172.21.15.26:0/2131212506 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043717766 0 0) 0x7fb8fc06d620 con 0x7fb8fc06a3e0 2024-01-15T18:06:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.674+0000 7fb8fb400640 1 -- 172.21.15.26:0/2131212506 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8dc003520 con 0x7fb8fc06a3e0 2024-01-15T18:06:03.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.674+0000 7fb8f3e00640 1 -- 172.21.15.26:0/2131212506 >> v2:172.21.15.124:3300/0 conn(0x7fb8fc15d310 msgr2=0x7fb8fc071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.674+0000 7fb8f3e00640 1 --2- 172.21.15.26:0/2131212506 >> v2:172.21.15.124:3300/0 conn(0x7fb8fc15d310 0x7fb8fc071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.674+0000 7fb8f3e00640 1 -- 172.21.15.26:0/2131212506 >> v1:172.21.15.26:6789/0 conn(0x7fb8fc06a3e0 legacy=0x7fb8fc0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.674+0000 7fb8f3e00640 1 -- 172.21.15.26:0/2131212506 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8fc06dec0 con 0x7fb8fc0aedc0 2024-01-15T18:06:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.676+0000 7fb8f3e00640 1 --2- 172.21.15.26:0/2131212506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc0aedc0 0x7fb8fc06e3e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc071390 tx=0x7fb8e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e72f18edee72d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:03.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.676+0000 7fb8fb400640 1 -- 172.21.15.26:0/2131212506 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4031830 con 0x7fb8fc0aedc0 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.676+0000 7fb8fb400640 1 -- 172.21.15.26:0/2131212506 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e4031990 con 0x7fb8fc0aedc0 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.676+0000 7fb8fb400640 1 -- 172.21.15.26:0/2131212506 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4031d20 con 0x7fb8fc0aedc0 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.677+0000 7fb901800640 1 -- 172.21.15.26:0/2131212506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc0aedc0 msgr2=0x7fb8fc06e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.677+0000 7fb901800640 1 --2- 172.21.15.26:0/2131212506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc0aedc0 0x7fb8fc06e3e0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc071390 tx=0x7fb8e402b8d0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.677+0000 7fb901800640 1 -- 172.21.15.26:0/2131212506 shutdown_connections 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.677+0000 7fb901800640 1 --2- 172.21.15.26:0/2131212506 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc0aedc0 0x7fb8fc06e3e0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.677+0000 7fb901800640 1 --2- 172.21.15.26:0/2131212506 >> v2:172.21.15.124:3300/0 conn(0x7fb8fc15d310 0x7fb8fc071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.677+0000 7fb901800640 1 -- 172.21.15.26:0/2131212506 >> 172.21.15.26:0/2131212506 conn(0x7fb8fc06b4f0 msgr2=0x7fb8fc06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.677+0000 7fb901800640 1 -- 172.21.15.26:0/2131212506 shutdown_connections 2024-01-15T18:06:03.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.677+0000 7fb901800640 1 -- 172.21.15.26:0/2131212506 wait complete. 2024-01-15T18:06:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.679+0000 7fb901800640 1 Processor -- start 2024-01-15T18:06:03.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.679+0000 7fb901800640 1 -- start start 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.680+0000 7fb901800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8fc06a3e0 0x7fb8fc14ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.680+0000 7fb901800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8fc0aedc0 0x7fb8fc13fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.680+0000 7fb901800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc15d310 0x7fb8fc13c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.680+0000 7fb901800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc14fd90 con 0x7fb8fc15d310 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.680+0000 7fb901800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc13c620 con 0x7fb8fc0aedc0 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.680+0000 7fb901800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8fc13ca10 con 0x7fb8fc06a3e0 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.680+0000 7fb900e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc15d310 0x7fb8fc13c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.680+0000 7fb900e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc15d310 0x7fb8fc13c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38670/0 (socket says 172.21.15.26:38670) 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.680+0000 7fb900e00640 1 -- 172.21.15.26:0/4285085059 learned_addr learned my addr 172.21.15.26:0/4285085059 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:03.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb8fbe00640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8fc06a3e0 0x7fb8fc14ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb900e00640 1 -- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8fc06a3e0 msgr2=0x7fb8fc14ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb900e00640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8fc06a3e0 0x7fb8fc14ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb900e00640 1 -- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8fc0aedc0 msgr2=0x7fb8fc13fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb900e00640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8fc0aedc0 0x7fb8fc13fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb900e00640 1 -- 172.21.15.26:0/4285085059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e4009400 con 0x7fb8fc15d310 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb900e00640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc15d310 0x7fb8fc13c0e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400cb40 tx=0x7fb8f400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb8f8c00640 1 -- 172.21.15.26:0/4285085059 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400aca0 con 0x7fb8fc15d310 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb8f8c00640 1 -- 172.21.15.26:0/4285085059 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f400ae00 con 0x7fb8fc15d310 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb8f8c00640 1 -- 172.21.15.26:0/4285085059 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4012620 con 0x7fb8fc15d310 2024-01-15T18:06:03.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.681+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8fc13ccf0 con 0x7fb8fc15d310 2024-01-15T18:06:03.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.682+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8fc13d2b0 con 0x7fb8fc15d310 2024-01-15T18:06:03.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.683+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8fc067da0 con 0x7fb8fc15d310 2024-01-15T18:06:03.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.684+0000 7fb8f8c00640 1 -- 172.21.15.26:0/4285085059 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fb8f4007070 con 0x7fb8fc15d310 2024-01-15T18:06:03.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.685+0000 7fb8f8c00640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb8d406bf20 0x7fb8d406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:03.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.685+0000 7fb8f8c00640 1 -- 172.21.15.26:0/4285085059 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fb8f4088960 con 0x7fb8fc15d310 2024-01-15T18:06:03.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.685+0000 7fb8fbe00640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb8d406bf20 0x7fb8d406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:03.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.687+0000 7fb8fbe00640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb8d406bf20 0x7fb8d406e3e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec006fd0 tx=0x7fb8ec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.690+0000 7fb8f8c00640 1 -- 172.21.15.26:0/4285085059 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8f4054f60 con 0x7fb8fc15d310 2024-01-15T18:06:03.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:03.922+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone4189", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8fc0725a0 con 0x7fb8d406bf20 2024-01-15T18:06:16.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.264+0000 7fb8f8c00640 1 -- 172.21.15.26:0/4285085059 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb8ec009480 con 0x7fb8d406bf20 2024-01-15T18:06:16.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb8d406bf20 msgr2=0x7fb8d406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb8d406bf20 0x7fb8d406e3e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec006fd0 tx=0x7fb8ec008040 comp rx=0 tx=0).stop 2024-01-15T18:06:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc15d310 msgr2=0x7fb8fc13c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc15d310 0x7fb8fc13c0e0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400cb40 tx=0x7fb8f400a060 comp rx=0 tx=0).stop 2024-01-15T18:06:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 shutdown_connections 2024-01-15T18:06:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb8d406bf20 0x7fb8d406e3e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8fc15d310 0x7fb8fc13c0e0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8fc0aedc0 0x7fb8fc13fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 --2- 172.21.15.26:0/4285085059 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8fc06a3e0 0x7fb8fc14ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.267+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 >> 172.21.15.26:0/4285085059 conn(0x7fb8fc06b4f0 msgr2=0x7fb8fc06b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:16.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.268+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 shutdown_connections 2024-01-15T18:06:16.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.268+0000 7fb901800640 1 -- 172.21.15.26:0/4285085059 wait complete. 2024-01-15T18:06:16.292 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41810 2024-01-15T18:06:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.521+0000 7f8b8ce00640 1 Processor -- start 2024-01-15T18:06:16.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.521+0000 7f8b8ce00640 1 -- start start 2024-01-15T18:06:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.522+0000 7f8b8ce00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b880ae5a0 0x7f8b8815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.523+0000 7f8b8ce00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b88070c40 con 0x7f8b880b3ae0 2024-01-15T18:06:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.523+0000 7f8b8ce00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8806c470 con 0x7f8b880cf290 2024-01-15T18:06:16.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.523+0000 7f8b8ce00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b88071450 con 0x7f8b880ae5a0 2024-01-15T18:06:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.523+0000 7f8b7fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8b880b3ae0 0x7f8b8806cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:44062/0 (socket says 172.21.15.26:44062) 2024-01-15T18:06:16.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.523+0000 7f8b7fe00640 1 -- 172.21.15.26:0/1411714205 learned_addr learned my addr 172.21.15.26:0/1411714205 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:06:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.523+0000 7f8b85e00640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b880ae5a0 0x7f8b8815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.523+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844000168 0 0) 0x7f8b88070c40 con 0x7f8b880b3ae0 2024-01-15T18:06:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.523+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b68003520 con 0x7f8b880b3ae0 2024-01-15T18:06:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203713175 0 0) 0x7f8b68003520 con 0x7f8b880b3ae0 2024-01-15T18:06:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b880ae5a0 msgr2=0x7f8b8815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b880ae5a0 0x7f8b8815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 >> v1:172.21.15.124:6789/0 conn(0x7f8b880cf290 legacy=0x7f8b880b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:16.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8806c750 con 0x7f8b880b3ae0 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8b70001670 con 0x7f8b880b3ae0 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 >> v1:172.21.15.26:6789/0 conn(0x7f8b880b3ae0 legacy=0x7f8b8806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680039a0 0x7f8b68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b680047c0 0x7f8b68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b880ae5a0 0x7f8b68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b88071450 con 0x7f8b680047c0 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800bac0 con 0x7f8b680039a0 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6800bc80 con 0x7f8b880ae5a0 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.524+0000 7f8b7fe00640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b680047c0 0x7f8b68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.525+0000 7f8b85e00640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680039a0 0x7f8b68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.525+0000 7f8b86800640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b880ae5a0 0x7f8b68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.525+0000 7f8b7fe00640 1 -- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b880ae5a0 msgr2=0x7f8b68005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:16.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.525+0000 7f8b7fe00640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b880ae5a0 0x7f8b68005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.525+0000 7f8b7fe00640 1 -- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680039a0 msgr2=0x7f8b68003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.525+0000 7f8b7fe00640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680039a0 0x7f8b68003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.525+0000 7f8b7fe00640 1 -- 172.21.15.26:0/1411714205 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68003520 con 0x7f8b680047c0 2024-01-15T18:06:16.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.525+0000 7f8b86800640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b880ae5a0 0x7f8b68005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.526+0000 7f8b7fe00640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b680047c0 0x7f8b68004c40 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8b70001eb0 tx=0x7f8b680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f60dd5123431d16 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.526+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70005ad0 con 0x7f8b680047c0 2024-01-15T18:06:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.526+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b70005c30 con 0x7f8b680047c0 2024-01-15T18:06:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.526+0000 7f8b85400640 1 -- 172.21.15.26:0/1411714205 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7002c850 con 0x7f8b680047c0 2024-01-15T18:06:16.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.527+0000 7f8b8ce00640 1 -- 172.21.15.26:0/1411714205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b680047c0 msgr2=0x7f8b68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:16.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.527+0000 7f8b8ce00640 1 --2- 172.21.15.26:0/1411714205 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b680047c0 0x7f8b68004c40 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8b70001eb0 tx=0x7f8b680057f0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.527+0000 7f8b86800640 1 -- 172.21.15.26:0/1411714205 reap_dead start 2024-01-15T18:06:16.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.527+0000 7f8b8ce00640 1 -- 172.21.15.26:0/1411714205 shutdown_connections 2024-01-15T18:06:16.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.527+0000 7f8b8ce00640 1 -- 172.21.15.26:0/1411714205 >> 172.21.15.26:0/1411714205 conn(0x7f8b8806a800 msgr2=0x7f8b8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:16.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.527+0000 7f8b8ce00640 1 -- 172.21.15.26:0/1411714205 shutdown_connections 2024-01-15T18:06:16.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.527+0000 7f8b8ce00640 1 -- 172.21.15.26:0/1411714205 wait complete. 2024-01-15T18:06:16.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.529+0000 7f8b8ce00640 1 Processor -- start 2024-01-15T18:06:16.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.529+0000 7f8b8ce00640 1 -- start start 2024-01-15T18:06:16.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.530+0000 7f8b8ce00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680047c0 0x7f8b881f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:16.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.530+0000 7f8b8ce00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b8814d520 0x7f8b880ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.530+0000 7f8b8ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b880ccb30 0x7f8b880ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.530+0000 7f8b8ce00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880cc710 con 0x7f8b880ccb30 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.530+0000 7f8b8ce00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b880cbb20 con 0x7f8b680047c0 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.530+0000 7f8b8ce00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8814d2e0 con 0x7f8b8814d520 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b86800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b880ccb30 0x7f8b880ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b86800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b880ccb30 0x7f8b880ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60856/0 (socket says 172.21.15.26:60856) 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b86800640 1 -- 172.21.15.26:0/2063140939 learned_addr learned my addr 172.21.15.26:0/2063140939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b85e00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680047c0 0x7f8b881f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b7fe00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b8814d520 0x7f8b880ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b86800640 1 -- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b8814d520 msgr2=0x7f8b880ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b86800640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b8814d520 0x7f8b880ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b86800640 1 -- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680047c0 msgr2=0x7f8b881f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b86800640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680047c0 0x7f8b881f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b86800640 1 -- 172.21.15.26:0/2063140939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b880cbd70 con 0x7f8b880ccb30 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b7fe00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b8814d520 0x7f8b880ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b85e00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680047c0 0x7f8b881f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.531+0000 7f8b86800640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b880ccb30 0x7f8b880ccfb0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8b880aeb00 tx=0x7f8b8000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.532+0000 7f8b7e000640 1 -- 172.21.15.26:0/2063140939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80006d50 con 0x7f8b880ccb30 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.532+0000 7f8b7e000640 1 -- 172.21.15.26:0/2063140939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b80003970 con 0x7f8b880ccb30 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.532+0000 7f8b7e000640 1 -- 172.21.15.26:0/2063140939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80003d20 con 0x7f8b880ccb30 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.532+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b880cc050 con 0x7f8b880ccb30 2024-01-15T18:06:16.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.532+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b880c4160 con 0x7f8b880ccb30 2024-01-15T18:06:16.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.533+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b880cc850 con 0x7f8b880ccb30 2024-01-15T18:06:16.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.534+0000 7f8b7e000640 1 -- 172.21.15.26:0/2063140939 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f8b80005070 con 0x7f8b880ccb30 2024-01-15T18:06:16.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.535+0000 7f8b7e000640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b6006bf50 0x7f8b6006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.535+0000 7f8b7e000640 1 -- 172.21.15.26:0/2063140939 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f8b80088700 con 0x7f8b880ccb30 2024-01-15T18:06:16.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.536+0000 7f8b85e00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b6006bf50 0x7f8b6006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:16.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.537+0000 7f8b85e00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b6006bf50 0x7f8b6006e410 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009a00 tx=0x7f8b78008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:16.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.540+0000 7f8b7e000640 1 -- 172.21.15.26:0/2063140939 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b880cc850 con 0x7f8b880ccb30 2024-01-15T18:06:16.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:16.769+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41810", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b88070550 con 0x7f8b6006bf50 2024-01-15T18:06:22.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.709+0000 7f8b7e000640 1 -- 172.21.15.26:0/2063140939 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8b78009670 con 0x7f8b6006bf50 2024-01-15T18:06:22.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b6006bf50 msgr2=0x7f8b6006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:22.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b6006bf50 0x7f8b6006e410 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8b78009a00 tx=0x7f8b78008040 comp rx=0 tx=0).stop 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b880ccb30 msgr2=0x7f8b880ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b880ccb30 0x7f8b880ccfb0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8b880aeb00 tx=0x7f8b8000dc00 comp rx=0 tx=0).stop 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 shutdown_connections 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b8814d520 0x7f8b880ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b6006bf50 0x7f8b6006e410 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b880ccb30 0x7f8b880ccfb0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 --2- 172.21.15.26:0/2063140939 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b680047c0 0x7f8b881f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 >> 172.21.15.26:0/2063140939 conn(0x7f8b8806a800 msgr2=0x7f8b8815da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 shutdown_connections 2024-01-15T18:06:22.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.713+0000 7f8b8ce00640 1 -- 172.21.15.26:0/2063140939 wait complete. 2024-01-15T18:06:22.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41811 2024-01-15T18:06:22.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.968+0000 7f7770a00640 1 Processor -- start 2024-01-15T18:06:22.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.968+0000 7f7770a00640 1 -- start start 2024-01-15T18:06:22.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7770a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 0x7f776c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7770a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f776c15bf10 0x7f776c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7770a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f776c0cc980 0x7f776c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7770a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c15f970 con 0x7f776c15bf10 2024-01-15T18:06:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7770a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776c15f4e0 con 0x7f776c0cc980 2024-01-15T18:06:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7770a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0cce10 con 0x7f776c0adf20 2024-01-15T18:06:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7761c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f776c15bf10 0x7f776c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7761c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f776c15bf10 0x7f776c0cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47678/0 (socket says 172.21.15.26:47678) 2024-01-15T18:06:22.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7761c00640 1 -- 172.21.15.26:0/1468901468 learned_addr learned my addr 172.21.15.26:0/1468901468 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f776a800640 1 --2- 172.21.15.26:0/1468901468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 0x7f776c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f776b200640 1 --2- 172.21.15.26:0/1468901468 >> v2:172.21.15.124:3300/0 conn(0x7f776c0cc980 0x7f776c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.970+0000 7f7761c00640 1 -- 172.21.15.26:0/1468901468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 msgr2=0x7f776c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.971+0000 7f7761c00640 1 --2- 172.21.15.26:0/1468901468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 0x7f776c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.971+0000 7f7761c00640 1 -- 172.21.15.26:0/1468901468 >> v2:172.21.15.124:3300/0 conn(0x7f776c0cc980 msgr2=0x7f776c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.971+0000 7f7761c00640 1 --2- 172.21.15.26:0/1468901468 >> v2:172.21.15.124:3300/0 conn(0x7f776c0cc980 0x7f776c143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.971+0000 7f7761c00640 1 -- 172.21.15.26:0/1468901468 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c071130 con 0x7f776c15bf10 2024-01-15T18:06:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.971+0000 7f776a800640 1 --2- 172.21.15.26:0/1468901468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 0x7f776c070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:22.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.971+0000 7f776b200640 1 --2- 172.21.15.26:0/1468901468 >> v2:172.21.15.124:3300/0 conn(0x7f776c0cc980 0x7f776c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.972+0000 7f7761c00640 1 --2- 172.21.15.26:0/1468901468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c15bf10 0x7f776c0cc440 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f776c06e490 tx=0x7f775402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f540320cbed0bd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.972+0000 7f7769e00640 1 -- 172.21.15.26:0/1468901468 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775402c9e0 con 0x7f776c15bf10 2024-01-15T18:06:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.972+0000 7f7769e00640 1 -- 172.21.15.26:0/1468901468 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775402cb40 con 0x7f776c15bf10 2024-01-15T18:06:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.972+0000 7f7769e00640 1 -- 172.21.15.26:0/1468901468 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77540343f0 con 0x7f776c15bf10 2024-01-15T18:06:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.973+0000 7f7770a00640 1 -- 172.21.15.26:0/1468901468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c15bf10 msgr2=0x7f776c0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:22.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.973+0000 7f7770a00640 1 --2- 172.21.15.26:0/1468901468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c15bf10 0x7f776c0cc440 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f776c06e490 tx=0x7f775402be40 comp rx=0 tx=0).stop 2024-01-15T18:06:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.973+0000 7f7770a00640 1 -- 172.21.15.26:0/1468901468 shutdown_connections 2024-01-15T18:06:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.973+0000 7f7770a00640 1 --2- 172.21.15.26:0/1468901468 >> v2:172.21.15.124:3300/0 conn(0x7f776c0cc980 0x7f776c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.973+0000 7f7770a00640 1 --2- 172.21.15.26:0/1468901468 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c15bf10 0x7f776c0cc440 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.973+0000 7f7770a00640 1 --2- 172.21.15.26:0/1468901468 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 0x7f776c070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.973+0000 7f7770a00640 1 -- 172.21.15.26:0/1468901468 >> 172.21.15.26:0/1468901468 conn(0x7f776c06a7d0 msgr2=0x7f776c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.973+0000 7f7770a00640 1 -- 172.21.15.26:0/1468901468 shutdown_connections 2024-01-15T18:06:22.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.973+0000 7f7770a00640 1 -- 172.21.15.26:0/1468901468 wait complete. 2024-01-15T18:06:22.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.975+0000 7f7770a00640 1 Processor -- start 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.975+0000 7f7770a00640 1 -- start start 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.976+0000 7f7770a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 0x7f776c0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.976+0000 7f7770a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c0cc980 0x7f776c142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.976+0000 7f7770a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f776c15bf10 0x7f776c15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.976+0000 7f7770a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c0b0100 con 0x7f776c0cc980 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.976+0000 7f7770a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776c1f3870 con 0x7f776c15bf10 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.976+0000 7f7770a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776c15e050 con 0x7f776c0adf20 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.976+0000 7f7761c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c0cc980 0x7f776c142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.976+0000 7f7761c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c0cc980 0x7f776c142790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47684/0 (socket says 172.21.15.26:47684) 2024-01-15T18:06:22.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.976+0000 7f7761c00640 1 -- 172.21.15.26:0/1576427638 learned_addr learned my addr 172.21.15.26:0/1576427638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f776b200640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f776c15bf10 0x7f776c15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7761c00640 1 -- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 msgr2=0x7f776c0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7761c00640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 0x7f776c0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7761c00640 1 -- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f776c15bf10 msgr2=0x7f776c15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7761c00640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f776c15bf10 0x7f776c15cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7761c00640 1 -- 172.21.15.26:0/1576427638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7754009b30 con 0x7f776c0cc980 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7761c00640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c0cc980 0x7f776c142790 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f7754009f40 tx=0x7f7754004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7763400640 1 -- 172.21.15.26:0/1576427638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775403e450 con 0x7f776c0cc980 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7763400640 1 -- 172.21.15.26:0/1576427638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775403e5b0 con 0x7f776c0cc980 2024-01-15T18:06:22.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7763400640 1 -- 172.21.15.26:0/1576427638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775403e960 con 0x7f776c0cc980 2024-01-15T18:06:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.977+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776c15e580 con 0x7f776c0cc980 2024-01-15T18:06:22.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.978+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776c1f3260 con 0x7f776c0cc980 2024-01-15T18:06:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.979+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f776c0c8e80 con 0x7f776c0cc980 2024-01-15T18:06:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.980+0000 7f7763400640 1 -- 172.21.15.26:0/1576427638 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f775403eac0 con 0x7f776c0cc980 2024-01-15T18:06:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.980+0000 7f7763400640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f774406bf20 0x7f774406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.981+0000 7f7763400640 1 -- 172.21.15.26:0/1576427638 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f77540ab700 con 0x7f776c0cc980 2024-01-15T18:06:22.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.981+0000 7f776a800640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f774406bf20 0x7f774406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:22.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.982+0000 7f776a800640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f774406bf20 0x7f774406e3e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f775c006b10 tx=0x7f775c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:22.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:22.985+0000 7f7763400640 1 -- 172.21.15.26:0/1576427638 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7754077d40 con 0x7f776c0cc980 2024-01-15T18:06:23.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:23.218+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41811", "target": ["mon-mgr", ""]}) v1 -- 0x7f776c15ee90 con 0x7f774406bf20 2024-01-15T18:06:26.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.445+0000 7f7763400640 1 -- 172.21.15.26:0/1576427638 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f775c00ad80 con 0x7f774406bf20 2024-01-15T18:06:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.451+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f774406bf20 msgr2=0x7f774406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.451+0000 7f7770a00640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f774406bf20 0x7f774406e3e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f775c006b10 tx=0x7f775c008040 comp rx=0 tx=0).stop 2024-01-15T18:06:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c0cc980 msgr2=0x7f776c142790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c0cc980 0x7f776c142790 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f7754009f40 tx=0x7f7754004a40 comp rx=0 tx=0).stop 2024-01-15T18:06:26.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 shutdown_connections 2024-01-15T18:06:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f774406bf20 0x7f774406e3e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f776c15bf10 0x7f776c15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f776c0cc980 0x7f776c142790 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 --2- 172.21.15.26:0/1576427638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f776c0adf20 0x7f776c0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 >> 172.21.15.26:0/1576427638 conn(0x7f776c06a7d0 msgr2=0x7f776c06d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 shutdown_connections 2024-01-15T18:06:26.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.452+0000 7f7770a00640 1 -- 172.21.15.26:0/1576427638 wait complete. 2024-01-15T18:06:26.487 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41812 2024-01-15T18:06:26.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.761+0000 7f338fa00640 1 Processor -- start 2024-01-15T18:06:26.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.761+0000 7f338fa00640 1 -- start start 2024-01-15T18:06:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.764+0000 7f338fa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3388153950 0x7f338806c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.764+0000 7f338fa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 0x7f338806ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.764+0000 7f338fa00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33880b2b70 con 0x7f33880b16d0 2024-01-15T18:06:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.764+0000 7f338fa00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3388153de0 con 0x7f3388153950 2024-01-15T18:06:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.764+0000 7f338fa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33880b2e50 con 0x7f338806c9b0 2024-01-15T18:06:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.764+0000 7f3387e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3388153950 0x7f338806c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:26.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.764+0000 7f3387e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3388153950 0x7f338806c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47698/0 (socket says 172.21.15.26:47698) 2024-01-15T18:06:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.764+0000 7f3387e00640 1 -- 172.21.15.26:0/3166295590 learned_addr learned my addr 172.21.15.26:0/3166295590 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.765+0000 7f338da00640 1 --2- 172.21.15.26:0/3166295590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 0x7f338806ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.765+0000 7f3387e00640 1 -- 172.21.15.26:0/3166295590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 msgr2=0x7f338806ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.765+0000 7f3387e00640 1 --2- 172.21.15.26:0/3166295590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 0x7f338806ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.765+0000 7f3387e00640 1 -- 172.21.15.26:0/3166295590 >> v1:172.21.15.124:6789/0 conn(0x7f33880b16d0 legacy=0x7f33880cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:26.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.765+0000 7f3387e00640 1 -- 172.21.15.26:0/3166295590 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33880b3130 con 0x7f3388153950 2024-01-15T18:06:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.767+0000 7f3387e00640 1 --2- 172.21.15.26:0/3166295590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 0x7f338806c470 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3370009870 tx=0x7f337002dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2998f2d1ca781306 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.767+0000 7f3387400640 1 -- 172.21.15.26:0/3166295590 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337002e950 con 0x7f3388153950 2024-01-15T18:06:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.767+0000 7f3387400640 1 -- 172.21.15.26:0/3166295590 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337002eab0 con 0x7f3388153950 2024-01-15T18:06:26.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.767+0000 7f3387400640 1 -- 172.21.15.26:0/3166295590 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337002ee40 con 0x7f3388153950 2024-01-15T18:06:26.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.768+0000 7f338fa00640 1 -- 172.21.15.26:0/3166295590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 msgr2=0x7f338806c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:26.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.768+0000 7f338fa00640 1 --2- 172.21.15.26:0/3166295590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 0x7f338806c470 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f3370009870 tx=0x7f337002dba0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.768+0000 7f338fa00640 1 -- 172.21.15.26:0/3166295590 shutdown_connections 2024-01-15T18:06:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.768+0000 7f338fa00640 1 --2- 172.21.15.26:0/3166295590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 0x7f338806ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.769+0000 7f338fa00640 1 --2- 172.21.15.26:0/3166295590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 0x7f338806c470 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.769+0000 7f338fa00640 1 -- 172.21.15.26:0/3166295590 >> 172.21.15.26:0/3166295590 conn(0x7f33880b0be0 msgr2=0x7f33880b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.769+0000 7f338fa00640 1 -- 172.21.15.26:0/3166295590 shutdown_connections 2024-01-15T18:06:26.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.769+0000 7f338fa00640 1 -- 172.21.15.26:0/3166295590 wait complete. 2024-01-15T18:06:26.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.771+0000 7f338fa00640 1 Processor -- start 2024-01-15T18:06:26.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.771+0000 7f338fa00640 1 -- start start 2024-01-15T18:06:26.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338fa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 0x7f33881487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:26.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338fa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33880b16d0 0x7f3388149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:26.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338fa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 0x7f33880c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:26.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338fa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33880c7b90 con 0x7f3388153950 2024-01-15T18:06:26.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338fa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33880c7e70 con 0x7f33880b16d0 2024-01-15T18:06:26.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338fa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33880c8150 con 0x7f338806c9b0 2024-01-15T18:06:26.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 0x7f33880c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:26.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f3387e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33880b16d0 0x7f3388149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 0x7f33880c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47702/0 (socket says 172.21.15.26:47702) 2024-01-15T18:06:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338da00640 1 -- 172.21.15.26:0/289919492 learned_addr learned my addr 172.21.15.26:0/289919492 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.773+0000 7f338d000640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 0x7f33881487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.774+0000 7f3387e00640 1 -- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 msgr2=0x7f33881487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.774+0000 7f3387e00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 0x7f33881487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.774+0000 7f3387e00640 1 -- 172.21.15.26:0/289919492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 msgr2=0x7f33880c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.774+0000 7f3387e00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 0x7f33880c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.774+0000 7f3387e00640 1 -- 172.21.15.26:0/289919492 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33700094d0 con 0x7f33880b16d0 2024-01-15T18:06:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.774+0000 7f338da00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 0x7f33880c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:26.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.774+0000 7f338d000640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 0x7f33881487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:26.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.774+0000 7f3387e00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33880b16d0 0x7f3388149360 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3370035000 tx=0x7f33700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:26.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.775+0000 7f3384c00640 1 -- 172.21.15.26:0/289919492 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370041670 con 0x7f33880b16d0 2024-01-15T18:06:26.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.775+0000 7f3384c00640 1 -- 172.21.15.26:0/289919492 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33700417d0 con 0x7f33880b16d0 2024-01-15T18:06:26.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.775+0000 7f3384c00640 1 -- 172.21.15.26:0/289919492 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370041b80 con 0x7f33880b16d0 2024-01-15T18:06:26.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.775+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33880c8430 con 0x7f33880b16d0 2024-01-15T18:06:26.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.775+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3388152100 con 0x7f33880b16d0 2024-01-15T18:06:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.776+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33880675d0 con 0x7f33880b16d0 2024-01-15T18:06:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.777+0000 7f3384c00640 1 -- 172.21.15.26:0/289919492 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f337003d070 con 0x7f33880b16d0 2024-01-15T18:06:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.778+0000 7f3384c00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f336006bf20 0x7f336006e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.779+0000 7f3384c00640 1 -- 172.21.15.26:0/289919492 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f33700ab090 con 0x7f33880b16d0 2024-01-15T18:06:26.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.779+0000 7f338d000640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f336006bf20 0x7f336006e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:26.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.780+0000 7f338d000640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f336006bf20 0x7f336006e3e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3378007fd0 tx=0x7f3378009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:26.786+0000 7f3384c00640 1 -- 172.21.15.26:0/289919492 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3370077a70 con 0x7f33880b16d0 2024-01-15T18:06:27.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.152+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41812", "target": ["mon-mgr", ""]}) v1 -- 0x7f33880b36f0 con 0x7f336006bf20 2024-01-15T18:06:27.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.349+0000 7f3384c00640 1 -- 172.21.15.26:0/289919492 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3378001cf0 con 0x7f336006bf20 2024-01-15T18:06:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f336006bf20 msgr2=0x7f336006e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f336006bf20 0x7f336006e3e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3378007fd0 tx=0x7f3378009040 comp rx=0 tx=0).stop 2024-01-15T18:06:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33880b16d0 msgr2=0x7f3388149360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33880b16d0 0x7f3388149360 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3370035000 tx=0x7f33700046c0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 shutdown_connections 2024-01-15T18:06:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f336006bf20 0x7f336006e3e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3388153950 0x7f33880c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33880b16d0 0x7f3388149360 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 --2- 172.21.15.26:0/289919492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f338806c9b0 0x7f33881487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 >> 172.21.15.26:0/289919492 conn(0x7f33880b0be0 msgr2=0x7f33880b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 shutdown_connections 2024-01-15T18:06:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.353+0000 7f338fa00640 1 -- 172.21.15.26:0/289919492 wait complete. 2024-01-15T18:06:27.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41813 2024-01-15T18:06:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.591+0000 7f7078e00640 1 Processor -- start 2024-01-15T18:06:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.591+0000 7f7078e00640 1 -- start start 2024-01-15T18:06:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.593+0000 7f7078e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7074143ae0 0x7f70740cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.593+0000 7f7078e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f70740cf7d0 0x7f707415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.593+0000 7f7078e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70740cfc30 con 0x7f707415e1e0 2024-01-15T18:06:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.593+0000 7f7078e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70740b1ee0 con 0x7f70740cf7d0 2024-01-15T18:06:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.593+0000 7f7078e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70740b2780 con 0x7f7074143ae0 2024-01-15T18:06:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.593+0000 7f7072800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f707415e1e0 0x7f707415a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45764/0 (socket says 172.21.15.26:45764) 2024-01-15T18:06:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.593+0000 7f7072800640 1 -- 172.21.15.26:0/2965384904 learned_addr learned my addr 172.21.15.26:0/2965384904 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:06:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7071e00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7074143ae0 0x7f70740cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7071400640 1 --2- 172.21.15.26:0/2965384904 >> v2:172.21.15.124:3300/0 conn(0x7f70740cf7d0 0x7f707415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971931103 0 0) 0x7f70740cfc30 con 0x7f707415e1e0 2024-01-15T18:06:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7054005730 con 0x7f707415e1e0 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 795762187 0 0) 0x7f7054005730 con 0x7f707415e1e0 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7074143ae0 msgr2=0x7f70740cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7074143ae0 0x7f70740cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 >> v2:172.21.15.124:3300/0 conn(0x7f70740cf7d0 msgr2=0x7f707415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 --2- 172.21.15.26:0/2965384904 >> v2:172.21.15.124:3300/0 conn(0x7f70740cf7d0 0x7f707415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707415e990 con 0x7f707415e1e0 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f706c0019f0 con 0x7f707415e1e0 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 >> v1:172.21.15.26:6789/0 conn(0x7f707415e1e0 legacy=0x7f707415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7054005b80 0x7f7054005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.594+0000 7f7070a00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540069d0 0x7f7054006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7070a00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7054007390 0x7f705400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70740b1ee0 con 0x7f7054005b80 2024-01-15T18:06:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70740b2780 con 0x7f7054007390 2024-01-15T18:06:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705400e0d0 con 0x7f70540069d0 2024-01-15T18:06:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7071e00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7074143ae0 0x7f70740cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:06:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7071e00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7054005b80 0x7f7054005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7072800640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7054007390 0x7f705400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7071400640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540069d0 0x7f7054006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:27.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7071e00640 1 -- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540069d0 msgr2=0x7f7054006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7071e00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540069d0 0x7f7054006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7071e00640 1 -- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7054007390 msgr2=0x7f705400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7071e00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7054007390 0x7f705400db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.595+0000 7f7071e00640 1 -- 172.21.15.26:0/2965384904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054005730 con 0x7f7054005b80 2024-01-15T18:06:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.596+0000 7f7071e00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7054005b80 0x7f7054005fa0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7064009b00 tx=0x7f7054007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba3df4738a340e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.597+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064004030 con 0x7f7054005b80 2024-01-15T18:06:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.597+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7064004190 con 0x7f7054005b80 2024-01-15T18:06:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.597+0000 7f7070a00640 1 -- 172.21.15.26:0/2965384904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064004520 con 0x7f7054005b80 2024-01-15T18:06:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.597+0000 7f7078e00640 1 -- 172.21.15.26:0/2965384904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7054005b80 msgr2=0x7f7054005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.597+0000 7f7078e00640 1 --2- 172.21.15.26:0/2965384904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7054005b80 0x7f7054005fa0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f7064009b00 tx=0x7f7054007a00 comp rx=0 tx=0).stop 2024-01-15T18:06:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.597+0000 7f7072800640 1 -- 172.21.15.26:0/2965384904 reap_dead start 2024-01-15T18:06:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.598+0000 7f7078e00640 1 -- 172.21.15.26:0/2965384904 shutdown_connections 2024-01-15T18:06:27.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.598+0000 7f7078e00640 1 -- 172.21.15.26:0/2965384904 >> 172.21.15.26:0/2965384904 conn(0x7f707406a800 msgr2=0x7f707406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.598+0000 7f7078e00640 1 -- 172.21.15.26:0/2965384904 shutdown_connections 2024-01-15T18:06:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.598+0000 7f7078e00640 1 -- 172.21.15.26:0/2965384904 wait complete. 2024-01-15T18:06:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.600+0000 7f7078e00640 1 Processor -- start 2024-01-15T18:06:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.600+0000 7f7078e00640 1 -- start start 2024-01-15T18:06:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7078e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7054005b80 0x7f707413bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7078e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70740adea0 0x7f707406c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7078e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70740ae300 0x7f707406c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7078e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707413f310 con 0x7f70740ae300 2024-01-15T18:06:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7078e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707413e690 con 0x7f70740adea0 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7078e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707413e970 con 0x7f7054005b80 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7072800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70740ae300 0x7f707406c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7072800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70740ae300 0x7f707406c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:47722/0 (socket says 172.21.15.26:47722) 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7072800640 1 -- 172.21.15.26:0/565119895 learned_addr learned my addr 172.21.15.26:0/565119895 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7071e00640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7054005b80 0x7f707413bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.601+0000 7f7071400640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70740adea0 0x7f707406c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f7072800640 1 -- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7054005b80 msgr2=0x7f707413bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f7072800640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7054005b80 0x7f707413bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f7072800640 1 -- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70740adea0 msgr2=0x7f707406c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f7072800640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70740adea0 0x7f707406c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f7072800640 1 -- 172.21.15.26:0/565119895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707413ec50 con 0x7f70740ae300 2024-01-15T18:06:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f7072800640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70740ae300 0x7f707406c9b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f70740b1bf0 tx=0x7f706c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f706a000640 1 -- 172.21.15.26:0/565119895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c00eb00 con 0x7f70740ae300 2024-01-15T18:06:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f706a000640 1 -- 172.21.15.26:0/565119895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c00ec60 con 0x7f70740ae300 2024-01-15T18:06:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f706a000640 1 -- 172.21.15.26:0/565119895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c013420 con 0x7f70740ae300 2024-01-15T18:06:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.602+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707413ef30 con 0x7f70740ae300 2024-01-15T18:06:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.603+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70740ac8d0 con 0x7f70740ae300 2024-01-15T18:06:27.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.604+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70740675d0 con 0x7f70740ae300 2024-01-15T18:06:27.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.605+0000 7f706a000640 1 -- 172.21.15.26:0/565119895 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f706c013580 con 0x7f70740ae300 2024-01-15T18:06:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.606+0000 7f706a000640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f704c06bf20 0x7f704c06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.606+0000 7f706a000640 1 -- 172.21.15.26:0/565119895 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f706c088940 con 0x7f70740ae300 2024-01-15T18:06:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.606+0000 7f7071e00640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f704c06bf20 0x7f704c06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.607+0000 7f7071e00640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f704c06bf20 0x7f704c06e3e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7064035000 tx=0x7f7064030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:27.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.611+0000 7f706a000640 1 -- 172.21.15.26:0/565119895 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f706c055200 con 0x7f70740ae300 2024-01-15T18:06:27.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:27.846+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41813", "target": ["mon-mgr", ""]}) v1 -- 0x7f707413eae0 con 0x7f704c06bf20 2024-01-15T18:06:29.468 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:06:29.476 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:06:37.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.762+0000 7f706a000640 1 -- 172.21.15.26:0/565119895 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f707413eae0 con 0x7f704c06bf20 2024-01-15T18:06:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f704c06bf20 msgr2=0x7f704c06e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f704c06bf20 0x7f704c06e3e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7064035000 tx=0x7f7064030000 comp rx=0 tx=0).stop 2024-01-15T18:06:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70740ae300 msgr2=0x7f707406c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70740ae300 0x7f707406c9b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f70740b1bf0 tx=0x7f706c007bc0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 shutdown_connections 2024-01-15T18:06:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f704c06bf20 0x7f704c06e3e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70740ae300 0x7f707406c9b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f70740adea0 0x7f707406c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 --2- 172.21.15.26:0/565119895 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7054005b80 0x7f707413bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 >> 172.21.15.26:0/565119895 conn(0x7f707406a800 msgr2=0x7f7074070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 shutdown_connections 2024-01-15T18:06:37.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:37.766+0000 7f7078e00640 1 -- 172.21.15.26:0/565119895 wait complete. 2024-01-15T18:06:37.792 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41814 2024-01-15T18:06:38.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.003+0000 7f0491200640 1 Processor -- start 2024-01-15T18:06:38.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.004+0000 7f0491200640 1 -- start start 2024-01-15T18:06:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.005+0000 7f0491200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f048c0cffa0 0x7f048c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.005+0000 7f0491200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0d1ea0 0x7f048c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.005+0000 7f0491200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048c0b2630 con 0x7f048c15d2c0 2024-01-15T18:06:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.005+0000 7f0491200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048c0ceca0 con 0x7f048c0cffa0 2024-01-15T18:06:38.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.005+0000 7f0491200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0cef80 con 0x7f048c0d1ea0 2024-01-15T18:06:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.006+0000 7f048be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0d1ea0 0x7f048c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.006+0000 7f048aa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f048c0cffa0 0x7f048c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:38.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.006+0000 7f048be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0d1ea0 0x7f048c0b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:55358/0 (socket says 172.21.15.26:55358) 2024-01-15T18:06:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.006+0000 7f048be00640 1 -- 172.21.15.26:0/874180637 learned_addr learned my addr 172.21.15.26:0/874180637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.007+0000 7f048be00640 1 -- 172.21.15.26:0/874180637 >> v2:172.21.15.124:3300/0 conn(0x7f048c0cffa0 msgr2=0x7f048c15d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.007+0000 7f048be00640 1 --2- 172.21.15.26:0/874180637 >> v2:172.21.15.124:3300/0 conn(0x7f048c0cffa0 0x7f048c15d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.007+0000 7f048be00640 1 -- 172.21.15.26:0/874180637 >> v1:172.21.15.124:6789/0 conn(0x7f048c15d2c0 legacy=0x7f048c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.007+0000 7f048be00640 1 -- 172.21.15.26:0/874180637 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c0b2880 con 0x7f048c0d1ea0 2024-01-15T18:06:38.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.008+0000 7f048be00640 1 --2- 172.21.15.26:0/874180637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0d1ea0 0x7f048c0b3080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0484009870 tx=0x7f048402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d8a82186a9e91b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f048a000640 1 -- 172.21.15.26:0/874180637 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484031640 con 0x7f048c0d1ea0 2024-01-15T18:06:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f048a000640 1 -- 172.21.15.26:0/874180637 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04840317a0 con 0x7f048c0d1ea0 2024-01-15T18:06:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f048a000640 1 -- 172.21.15.26:0/874180637 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484031b30 con 0x7f048c0d1ea0 2024-01-15T18:06:38.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f0491200640 1 -- 172.21.15.26:0/874180637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0d1ea0 msgr2=0x7f048c0b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f0491200640 1 --2- 172.21.15.26:0/874180637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0d1ea0 0x7f048c0b3080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0484009870 tx=0x7f048402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:06:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f0491200640 1 -- 172.21.15.26:0/874180637 shutdown_connections 2024-01-15T18:06:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f0491200640 1 --2- 172.21.15.26:0/874180637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0d1ea0 0x7f048c0b3080 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f0491200640 1 --2- 172.21.15.26:0/874180637 >> v2:172.21.15.124:3300/0 conn(0x7f048c0cffa0 0x7f048c15d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f0491200640 1 -- 172.21.15.26:0/874180637 >> 172.21.15.26:0/874180637 conn(0x7f048c0d30b0 msgr2=0x7f048c0d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f0491200640 1 -- 172.21.15.26:0/874180637 shutdown_connections 2024-01-15T18:06:38.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.009+0000 7f0491200640 1 -- 172.21.15.26:0/874180637 wait complete. 2024-01-15T18:06:38.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.011+0000 7f0491200640 1 Processor -- start 2024-01-15T18:06:38.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.011+0000 7f0491200640 1 -- start start 2024-01-15T18:06:38.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.012+0000 7f0491200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0cffa0 0x7f048c14b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:38.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.012+0000 7f0491200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f048c0d1ea0 0x7f048c0a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:38.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.012+0000 7f0491200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c15d2c0 0x7f048c09e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.012+0000 7f0491200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c09df70 con 0x7f048c15d2c0 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.012+0000 7f0491200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048c14ba80 con 0x7f048c0d1ea0 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.012+0000 7f0491200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048c0a0dd0 con 0x7f048c0cffa0 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c15d2c0 0x7f048c09e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c15d2c0 0x7f048c09e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56074/0 (socket says 172.21.15.26:56074) 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048be00640 1 -- 172.21.15.26:0/1262306200 learned_addr learned my addr 172.21.15.26:0/1262306200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048aa00640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f048c0d1ea0 0x7f048c0a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048b400640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0cffa0 0x7f048c14b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048be00640 1 -- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0cffa0 msgr2=0x7f048c14b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:38.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048be00640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0cffa0 0x7f048c14b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048be00640 1 -- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f048c0d1ea0 msgr2=0x7f048c0a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048be00640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f048c0d1ea0 0x7f048c0a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048be00640 1 -- 172.21.15.26:0/1262306200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04840094d0 con 0x7f048c15d2c0 2024-01-15T18:06:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048aa00640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f048c0d1ea0 0x7f048c0a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.013+0000 7f048b400640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0cffa0 0x7f048c14b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:06:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.014+0000 7f048be00640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c15d2c0 0x7f048c09e880 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0484009870 tx=0x7f04840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.014+0000 7f0483400640 1 -- 172.21.15.26:0/1262306200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048403a930 con 0x7f048c15d2c0 2024-01-15T18:06:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.014+0000 7f0483400640 1 -- 172.21.15.26:0/1262306200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048403aab0 con 0x7f048c15d2c0 2024-01-15T18:06:38.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.014+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f048c09d2f0 con 0x7f048c15d2c0 2024-01-15T18:06:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.014+0000 7f0483400640 1 -- 172.21.15.26:0/1262306200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048403ae60 con 0x7f048c15d2c0 2024-01-15T18:06:38.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.014+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f048c09d8b0 con 0x7f048c15d2c0 2024-01-15T18:06:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.015+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f048c0d2770 con 0x7f048c15d2c0 2024-01-15T18:06:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.017+0000 7f0483400640 1 -- 172.21.15.26:0/1262306200 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f048402f050 con 0x7f048c15d2c0 2024-01-15T18:06:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.017+0000 7f0483400640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f046406bf20 0x7f046406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.017+0000 7f0483400640 1 -- 172.21.15.26:0/1262306200 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f04840aa840 con 0x7f048c15d2c0 2024-01-15T18:06:38.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.018+0000 7f048b400640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f046406bf20 0x7f046406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:38.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.019+0000 7f048b400640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f046406bf20 0x7f046406e3e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f047c001660 tx=0x7f047c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:38.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.022+0000 7f0483400640 1 -- 172.21.15.26:0/1262306200 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0484076ee0 con 0x7f048c15d2c0 2024-01-15T18:06:38.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:38.259+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41814", "target": ["mon-mgr", ""]}) v1 -- 0x7f048c0d3bd0 con 0x7f046406bf20 2024-01-15T18:06:44.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.111+0000 7f0483400640 1 -- 172.21.15.26:0/1262306200 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f047c0012a0 con 0x7f046406bf20 2024-01-15T18:06:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f046406bf20 msgr2=0x7f046406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f046406bf20 0x7f046406e3e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f047c001660 tx=0x7f047c009040 comp rx=0 tx=0).stop 2024-01-15T18:06:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c15d2c0 msgr2=0x7f048c09e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c15d2c0 0x7f048c09e880 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0484009870 tx=0x7f04840046c0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 shutdown_connections 2024-01-15T18:06:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f046406bf20 0x7f046406e3e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f048c15d2c0 0x7f048c09e880 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f048c0d1ea0 0x7f048c0a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 --2- 172.21.15.26:0/1262306200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f048c0cffa0 0x7f048c14b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.114+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 >> 172.21.15.26:0/1262306200 conn(0x7f048c0d30b0 msgr2=0x7f048c15c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.115+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 shutdown_connections 2024-01-15T18:06:44.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.115+0000 7f0491200640 1 -- 172.21.15.26:0/1262306200 wait complete. 2024-01-15T18:06:44.141 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41815 2024-01-15T18:06:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.351+0000 7fe969a00640 1 Processor -- start 2024-01-15T18:06:44.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.351+0000 7fe969a00640 1 -- start start 2024-01-15T18:06:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.352+0000 7fe969a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 0x7fe964070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.353+0000 7fe969a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe96415bf10 0x7fe9640cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.353+0000 7fe969a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe96415f4e0 con 0x7fe9640cc980 2024-01-15T18:06:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.353+0000 7fe969a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe96415f970 con 0x7fe96415bf10 2024-01-15T18:06:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.353+0000 7fe969a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9640cce10 con 0x7fe9640adf20 2024-01-15T18:06:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.353+0000 7fe963400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe96415bf10 0x7fe9640cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.353+0000 7fe969000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe9640cc980 0x7fe964143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58480/0 (socket says 172.21.15.26:58480) 2024-01-15T18:06:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.353+0000 7fe969000640 1 -- 172.21.15.26:0/3960063658 learned_addr learned my addr 172.21.15.26:0/3960063658 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:06:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.353+0000 7fe963e00640 1 --2- 172.21.15.26:0/3960063658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 0x7fe964070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.354+0000 7fe963400640 1 -- 172.21.15.26:0/3960063658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 msgr2=0x7fe964070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.354+0000 7fe963400640 1 --2- 172.21.15.26:0/3960063658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 0x7fe964070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.354+0000 7fe963400640 1 -- 172.21.15.26:0/3960063658 >> v1:172.21.15.26:6789/0 conn(0x7fe9640cc980 legacy=0x7fe964143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.354+0000 7fe963400640 1 -- 172.21.15.26:0/3960063658 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe964071130 con 0x7fe96415bf10 2024-01-15T18:06:44.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.354+0000 7fe962a00640 1 -- 172.21.15.26:0/3960063658 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803227238 0 0) 0x7fe96415f4e0 con 0x7fe9640cc980 2024-01-15T18:06:44.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.355+0000 7fe963400640 1 --2- 172.21.15.26:0/3960063658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96415bf10 0x7fe9640cc440 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009d90 tx=0x7fe94c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a371dbf885eb46 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.355+0000 7fe962a00640 1 -- 172.21.15.26:0/3960063658 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c02cbc0 con 0x7fe96415bf10 2024-01-15T18:06:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.355+0000 7fe962a00640 1 -- 172.21.15.26:0/3960063658 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c02cd20 con 0x7fe96415bf10 2024-01-15T18:06:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.355+0000 7fe962a00640 1 -- 172.21.15.26:0/3960063658 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c034660 con 0x7fe96415bf10 2024-01-15T18:06:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.356+0000 7fe969a00640 1 -- 172.21.15.26:0/3960063658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96415bf10 msgr2=0x7fe9640cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.356+0000 7fe969a00640 1 --2- 172.21.15.26:0/3960063658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96415bf10 0x7fe9640cc440 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009d90 tx=0x7fe94c02be10 comp rx=0 tx=0).stop 2024-01-15T18:06:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.356+0000 7fe969a00640 1 -- 172.21.15.26:0/3960063658 shutdown_connections 2024-01-15T18:06:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.356+0000 7fe969a00640 1 --2- 172.21.15.26:0/3960063658 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe96415bf10 0x7fe9640cc440 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.356+0000 7fe969a00640 1 --2- 172.21.15.26:0/3960063658 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 0x7fe964070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.356+0000 7fe969a00640 1 -- 172.21.15.26:0/3960063658 >> 172.21.15.26:0/3960063658 conn(0x7fe96406a7d0 msgr2=0x7fe96406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.356+0000 7fe969a00640 1 -- 172.21.15.26:0/3960063658 shutdown_connections 2024-01-15T18:06:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.356+0000 7fe969a00640 1 -- 172.21.15.26:0/3960063658 wait complete. 2024-01-15T18:06:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.358+0000 7fe969a00640 1 Processor -- start 2024-01-15T18:06:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.358+0000 7fe969a00640 1 -- start start 2024-01-15T18:06:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.359+0000 7fe969a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 0x7fe96415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.359+0000 7fe969a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640cc980 0x7fe964142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.359+0000 7fe969a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe96415bf10 0x7fe96415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.359+0000 7fe969a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe964141060 con 0x7fe9640cc980 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.359+0000 7fe969a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe964141340 con 0x7fe96415bf10 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.359+0000 7fe969a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe964141620 con 0x7fe9640adf20 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.359+0000 7fe963400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640cc980 0x7fe964142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.360+0000 7fe963400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640cc980 0x7fe964142100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58904/0 (socket says 172.21.15.26:58904) 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.360+0000 7fe963400640 1 -- 172.21.15.26:0/2563044460 learned_addr learned my addr 172.21.15.26:0/2563044460 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.360+0000 7fe963e00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 0x7fe96415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.360+0000 7fe963400640 1 -- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 msgr2=0x7fe96415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.360+0000 7fe963400640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 0x7fe96415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.360+0000 7fe963400640 1 -- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe96415bf10 msgr2=0x7fe96415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.360+0000 7fe963400640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe96415bf10 0x7fe96415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.360+0000 7fe963400640 1 -- 172.21.15.26:0/2563044460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c009a70 con 0x7fe9640cc980 2024-01-15T18:06:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.360+0000 7fe963400640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640cc980 0x7fe964142100 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009d90 tx=0x7fe94c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.361+0000 7fe95be00640 1 -- 172.21.15.26:0/2563044460 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c03e610 con 0x7fe9640cc980 2024-01-15T18:06:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.361+0000 7fe95be00640 1 -- 172.21.15.26:0/2563044460 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c02ce90 con 0x7fe9640cc980 2024-01-15T18:06:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.361+0000 7fe95be00640 1 -- 172.21.15.26:0/2563044460 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c03e970 con 0x7fe9640cc980 2024-01-15T18:06:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.361+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe964141900 con 0x7fe9640cc980 2024-01-15T18:06:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.361+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe96415ac50 con 0x7fe9640cc980 2024-01-15T18:06:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.362+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe964141e20 con 0x7fe9640cc980 2024-01-15T18:06:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.363+0000 7fe95be00640 1 -- 172.21.15.26:0/2563044460 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fe94c03b050 con 0x7fe9640cc980 2024-01-15T18:06:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.364+0000 7fe95be00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe93c06bf20 0x7fe93c06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:44.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.364+0000 7fe95be00640 1 -- 172.21.15.26:0/2563044460 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fe94c0ab040 con 0x7fe9640cc980 2024-01-15T18:06:44.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.365+0000 7fe963e00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe93c06bf20 0x7fe93c06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:44.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.366+0000 7fe963e00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe93c06bf20 0x7fe93c06e3e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe954006fd0 tx=0x7fe954008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:44.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.369+0000 7fe95be00640 1 -- 172.21.15.26:0/2563044460 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe964141e20 con 0x7fe9640cc980 2024-01-15T18:06:44.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:44.597+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41815", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9641414b0 con 0x7fe93c06bf20 2024-01-15T18:06:47.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.927+0000 7fe95be00640 1 -- 172.21.15.26:0/2563044460 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe9641414b0 con 0x7fe93c06bf20 2024-01-15T18:06:47.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe93c06bf20 msgr2=0x7fe93c06e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:47.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe93c06bf20 0x7fe93c06e3e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe954006fd0 tx=0x7fe954008040 comp rx=0 tx=0).stop 2024-01-15T18:06:47.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640cc980 msgr2=0x7fe964142100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640cc980 0x7fe964142100 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009d90 tx=0x7fe94c004a40 comp rx=0 tx=0).stop 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 shutdown_connections 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe93c06bf20 0x7fe93c06e3e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe96415bf10 0x7fe96415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9640cc980 0x7fe964142100 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 --2- 172.21.15.26:0/2563044460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9640adf20 0x7fe96415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 >> 172.21.15.26:0/2563044460 conn(0x7fe96406a7d0 msgr2=0x7fe96406d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 shutdown_connections 2024-01-15T18:06:47.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:47.931+0000 7fe969a00640 1 -- 172.21.15.26:0/2563044460 wait complete. 2024-01-15T18:06:47.958 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41816 2024-01-15T18:06:48.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.186+0000 7fc059400640 1 Processor -- start 2024-01-15T18:06:48.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.186+0000 7fc059400640 1 -- start start 2024-01-15T18:06:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.187+0000 7fc059400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc05415be10 0x7fc054166060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:48.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.187+0000 7fc059400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0540cc730 con 0x7fc05415b320 2024-01-15T18:06:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.187+0000 7fc059400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0540cbe00 con 0x7fc0540ccbc0 2024-01-15T18:06:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.187+0000 7fc059400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0540cc0e0 con 0x7fc05415be10 2024-01-15T18:06:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.188+0000 7fc058a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc05415be10 0x7fc054166060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.188+0000 7fc058a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc05415be10 0x7fc054166060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58908/0 (socket says 172.21.15.26:58908) 2024-01-15T18:06:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.188+0000 7fc058a00640 1 -- 172.21.15.26:0/3627943971 learned_addr learned my addr 172.21.15.26:0/3627943971 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:48.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.188+0000 7fc058a00640 1 -- 172.21.15.26:0/3627943971 >> v1:172.21.15.124:6789/0 conn(0x7fc0540ccbc0 legacy=0x7fc05415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.188+0000 7fc058a00640 1 -- 172.21.15.26:0/3627943971 >> v1:172.21.15.26:6789/0 conn(0x7fc05415b320 legacy=0x7fc0540ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.188+0000 7fc058a00640 1 -- 172.21.15.26:0/3627943971 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0540cc3c0 con 0x7fc05415be10 2024-01-15T18:06:48.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.189+0000 7fc058a00640 1 --2- 172.21.15.26:0/3627943971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415be10 0x7fc054166060 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0098a0 tx=0x7fc04c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433b4c53c7341632 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.190+0000 7fc052a00640 1 -- 172.21.15.26:0/3627943971 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c030840 con 0x7fc05415be10 2024-01-15T18:06:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.190+0000 7fc052a00640 1 -- 172.21.15.26:0/3627943971 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04c0309a0 con 0x7fc05415be10 2024-01-15T18:06:48.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.190+0000 7fc052a00640 1 -- 172.21.15.26:0/3627943971 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c030d30 con 0x7fc05415be10 2024-01-15T18:06:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.190+0000 7fc059400640 1 -- 172.21.15.26:0/3627943971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415be10 msgr2=0x7fc054166060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.190+0000 7fc059400640 1 --2- 172.21.15.26:0/3627943971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415be10 0x7fc054166060 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0098a0 tx=0x7fc04c02b980 comp rx=0 tx=0).stop 2024-01-15T18:06:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.190+0000 7fc059400640 1 -- 172.21.15.26:0/3627943971 shutdown_connections 2024-01-15T18:06:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.190+0000 7fc059400640 1 --2- 172.21.15.26:0/3627943971 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415be10 0x7fc054166060 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.191+0000 7fc059400640 1 -- 172.21.15.26:0/3627943971 >> 172.21.15.26:0/3627943971 conn(0x7fc0540b2a60 msgr2=0x7fc0540b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.191+0000 7fc059400640 1 -- 172.21.15.26:0/3627943971 shutdown_connections 2024-01-15T18:06:48.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.191+0000 7fc059400640 1 -- 172.21.15.26:0/3627943971 wait complete. 2024-01-15T18:06:48.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.192+0000 7fc059400640 1 Processor -- start 2024-01-15T18:06:48.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.193+0000 7fc059400640 1 -- start start 2024-01-15T18:06:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc059400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0540ccbc0 0x7fc0540b6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc059400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415b320 0x7fc0540cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc059400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc05415be10 0x7fc0540cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc059400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc054142490 con 0x7fc05415b320 2024-01-15T18:06:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc059400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0541e9dd0 con 0x7fc0540ccbc0 2024-01-15T18:06:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc059400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0541ea0b0 con 0x7fc05415be10 2024-01-15T18:06:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc053400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415b320 0x7fc0540cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:48.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc053400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415b320 0x7fc0540cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58920/0 (socket says 172.21.15.26:58920) 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc053400640 1 -- 172.21.15.26:0/1768981125 learned_addr learned my addr 172.21.15.26:0/1768981125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc053e00640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0540ccbc0 0x7fc0540b6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc058a00640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc05415be10 0x7fc0540cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc053400640 1 -- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc05415be10 msgr2=0x7fc0540cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc053400640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc05415be10 0x7fc0540cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc053400640 1 -- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0540ccbc0 msgr2=0x7fc0540b6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc053400640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0540ccbc0 0x7fc0540b6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc053400640 1 -- 172.21.15.26:0/1768981125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0094d0 con 0x7fc05415b320 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.194+0000 7fc058a00640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc05415be10 0x7fc0540cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.195+0000 7fc053400640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415b320 0x7fc0540cda00 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c430 tx=0x7fc03c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.195+0000 7fc04be00640 1 -- 172.21.15.26:0/1768981125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c011790 con 0x7fc05415b320 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.195+0000 7fc04be00640 1 -- 172.21.15.26:0/1768981125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c0118f0 con 0x7fc05415b320 2024-01-15T18:06:48.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.195+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0541426a0 con 0x7fc05415b320 2024-01-15T18:06:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.195+0000 7fc04be00640 1 -- 172.21.15.26:0/1768981125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c011ce0 con 0x7fc05415b320 2024-01-15T18:06:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.195+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0540b69b0 con 0x7fc05415b320 2024-01-15T18:06:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.197+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc054142bc0 con 0x7fc05415b320 2024-01-15T18:06:48.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.198+0000 7fc04be00640 1 -- 172.21.15.26:0/1768981125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fc03c01c460 con 0x7fc05415b320 2024-01-15T18:06:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.198+0000 7fc04be00640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc02c06bf20 0x7fc02c06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.199+0000 7fc04be00640 1 -- 172.21.15.26:0/1768981125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fc03c088540 con 0x7fc05415b320 2024-01-15T18:06:48.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.199+0000 7fc053e00640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc02c06bf20 0x7fc02c06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:48.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.200+0000 7fc053e00640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc02c06bf20 0x7fc02c06e3e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc0440012e0 tx=0x7fc044009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:48.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.203+0000 7fc04be00640 1 -- 172.21.15.26:0/1768981125 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc03c054e60 con 0x7fc05415b320 2024-01-15T18:06:48.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:48.437+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41816", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0540b3730 con 0x7fc02c06bf20 2024-01-15T18:06:51.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.111+0000 7fc04be00640 1 -- 172.21.15.26:0/1768981125 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc044001a00 con 0x7fc02c06bf20 2024-01-15T18:06:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc02c06bf20 msgr2=0x7fc02c06e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:51.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc02c06bf20 0x7fc02c06e3e0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc0440012e0 tx=0x7fc044009040 comp rx=0 tx=0).stop 2024-01-15T18:06:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415b320 msgr2=0x7fc0540cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415b320 0x7fc0540cda00 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c430 tx=0x7fc03c00c8f0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 shutdown_connections 2024-01-15T18:06:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc05415be10 0x7fc0540cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc02c06bf20 0x7fc02c06e3e0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc05415b320 0x7fc0540cda00 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 --2- 172.21.15.26:0/1768981125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0540ccbc0 0x7fc0540b6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 >> 172.21.15.26:0/1768981125 conn(0x7fc0540b2a60 msgr2=0x7fc05406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:51.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 shutdown_connections 2024-01-15T18:06:51.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.114+0000 7fc059400640 1 -- 172.21.15.26:0/1768981125 wait complete. 2024-01-15T18:06:51.140 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41817 2024-01-15T18:06:51.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.335+0000 7ff9f0e00640 1 Processor -- start 2024-01-15T18:06:51.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.335+0000 7ff9f0e00640 1 -- start start 2024-01-15T18:06:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.337+0000 7ff9f0e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9ec143ae0 0x7ff9ec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.337+0000 7ff9f0e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 0x7ff9ec0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.337+0000 7ff9f0e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ec0cfc30 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.337+0000 7ff9f0e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ec0b1ee0 con 0x7ff9ec143ae0 2024-01-15T18:06:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.337+0000 7ff9f0e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0b2780 con 0x7ff9ec15e3b0 2024-01-15T18:06:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.337+0000 7ff9e9e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff9ec143ae0 0x7ff9ec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.337+0000 7ff9e1200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff9ec0cf7d0 0x7ff9ec15dca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58496/0 (socket says 172.21.15.26:58496) 2024-01-15T18:06:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.337+0000 7ff9e1200640 1 -- 172.21.15.26:0/2098726364 learned_addr learned my addr 172.21.15.26:0/2098726364 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:06:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.338+0000 7ff9ea800640 1 --2- 172.21.15.26:0/2098726364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 0x7ff9ec0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.338+0000 7ff9e9400640 1 -- 172.21.15.26:0/2098726364 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393021192 0 0) 0x7ff9ec0cfc30 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.338+0000 7ff9e9400640 1 -- 172.21.15.26:0/2098726364 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9cc003520 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.338+0000 7ff9e9e00640 1 -- 172.21.15.26:0/2098726364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 msgr2=0x7ff9ec0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:06:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.338+0000 7ff9e9e00640 1 --2- 172.21.15.26:0/2098726364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 0x7ff9ec0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.338+0000 7ff9e9e00640 1 -- 172.21.15.26:0/2098726364 >> v1:172.21.15.26:6789/0 conn(0x7ff9ec0cf7d0 legacy=0x7ff9ec15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.338+0000 7ff9e9e00640 1 -- 172.21.15.26:0/2098726364 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec15e990 con 0x7ff9ec143ae0 2024-01-15T18:06:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.339+0000 7ff9e9e00640 1 --2- 172.21.15.26:0/2098726364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec143ae0 0x7ff9ec0cf290 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009d90 tx=0x7ff9dc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3187900e2fe78426 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.339+0000 7ff9e9400640 1 -- 172.21.15.26:0/2098726364 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc02cc50 con 0x7ff9ec143ae0 2024-01-15T18:06:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.340+0000 7ff9e9400640 1 -- 172.21.15.26:0/2098726364 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc02cdb0 con 0x7ff9ec143ae0 2024-01-15T18:06:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.340+0000 7ff9e9400640 1 -- 172.21.15.26:0/2098726364 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc034640 con 0x7ff9ec143ae0 2024-01-15T18:06:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.340+0000 7ff9f0e00640 1 -- 172.21.15.26:0/2098726364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec143ae0 msgr2=0x7ff9ec0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.340+0000 7ff9f0e00640 1 --2- 172.21.15.26:0/2098726364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec143ae0 0x7ff9ec0cf290 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009d90 tx=0x7ff9dc02bea0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.340+0000 7ff9f0e00640 1 -- 172.21.15.26:0/2098726364 shutdown_connections 2024-01-15T18:06:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.340+0000 7ff9f0e00640 1 --2- 172.21.15.26:0/2098726364 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 0x7ff9ec0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.340+0000 7ff9f0e00640 1 --2- 172.21.15.26:0/2098726364 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec143ae0 0x7ff9ec0cf290 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.340+0000 7ff9f0e00640 1 -- 172.21.15.26:0/2098726364 >> 172.21.15.26:0/2098726364 conn(0x7ff9ec06a800 msgr2=0x7ff9ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:06:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.341+0000 7ff9f0e00640 1 -- 172.21.15.26:0/2098726364 shutdown_connections 2024-01-15T18:06:51.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.341+0000 7ff9f0e00640 1 -- 172.21.15.26:0/2098726364 wait complete. 2024-01-15T18:06:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.342+0000 7ff9f0e00640 1 Processor -- start 2024-01-15T18:06:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.342+0000 7ff9f0e00640 1 -- start start 2024-01-15T18:06:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.343+0000 7ff9f0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec0cf7d0 0x7ff9ec1324f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:51.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.343+0000 7ff9f0e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9ec143ae0 0x7ff9ec138990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9f0e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 0x7ff9ec0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9f0e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec135ae0 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9f0e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec134ed0 con 0x7ff9ec143ae0 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9f0e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec1351b0 con 0x7ff9ec15e3b0 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9e9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec0cf7d0 0x7ff9ec1324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9e9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec0cf7d0 0x7ff9ec1324f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58938/0 (socket says 172.21.15.26:58938) 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9e9e00640 1 -- 172.21.15.26:0/4157095949 learned_addr learned my addr 172.21.15.26:0/4157095949 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9e1200640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9ec143ae0 0x7ff9ec138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9ea800640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 0x7ff9ec0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9e9e00640 1 -- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 msgr2=0x7ff9ec0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:51.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9e9e00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 0x7ff9ec0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9e9e00640 1 -- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9ec143ae0 msgr2=0x7ff9ec138990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:06:51.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9e9e00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9ec143ae0 0x7ff9ec138990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:06:51.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9e9e00640 1 -- 172.21.15.26:0/4157095949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9dc009a70 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.344+0000 7ff9ea800640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 0x7ff9ec0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:06:51.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.345+0000 7ff9e9e00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec0cf7d0 0x7ff9ec1324f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009eb0 tx=0x7ff9dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:51.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.345+0000 7ff9e2a00640 1 -- 172.21.15.26:0/4157095949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc03e610 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.345+0000 7ff9e2a00640 1 -- 172.21.15.26:0/4157095949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9dc03e790 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.345+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ec135490 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.345+0000 7ff9e2a00640 1 -- 172.21.15.26:0/4157095949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc03eb40 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.345+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec132a30 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.346+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ec0675d0 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.347+0000 7ff9e2a00640 1 -- 172.21.15.26:0/4157095949 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 26) v1 ==== 90661+0+0 (secure 0 0 0) 0x7ff9dc03b050 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.348+0000 7ff9e2a00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff9c406bf20 0x7ff9c406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:06:51.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.348+0000 7ff9e2a00640 1 -- 172.21.15.26:0/4157095949 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7ff9dc0aaa60 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.349+0000 7ff9e1200640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff9c406bf20 0x7ff9c406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:06:51.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.350+0000 7ff9e1200640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff9c406bf20 0x7ff9c406e3e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec15b610 tx=0x7ff9d4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:06:51.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.353+0000 7ff9e2a00640 1 -- 172.21.15.26:0/4157095949 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9dc0770a0 con 0x7ff9ec0cf7d0 2024-01-15T18:06:51.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:51.586+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41817", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9ec0b2410 con 0x7ff9c406bf20 2024-01-15T18:06:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:06:53.144+0000 7ff9e2a00640 1 -- 172.21.15.26:0/4157095949 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7ff9dc077470 con 0x7ff9ec0cf7d0 2024-01-15T18:06:59.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:06:59.624 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:07:01.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.313+0000 7ff9e2a00640 1 -- 172.21.15.26:0/4157095949 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff9d40013f0 con 0x7ff9c406bf20 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.316+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff9c406bf20 msgr2=0x7ff9c406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.316+0000 7ff9f0e00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff9c406bf20 0x7ff9c406e3e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec15b610 tx=0x7ff9d4009040 comp rx=0 tx=0).stop 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.316+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec0cf7d0 msgr2=0x7ff9ec1324f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.316+0000 7ff9f0e00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec0cf7d0 0x7ff9ec1324f0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009eb0 tx=0x7ff9dc004a40 comp rx=0 tx=0).stop 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.317+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 shutdown_connections 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.317+0000 7ff9f0e00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9ec15e3b0 0x7ff9ec0702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.317+0000 7ff9f0e00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff9c406bf20 0x7ff9c406e3e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.317+0000 7ff9f0e00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff9ec143ae0 0x7ff9ec138990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.317+0000 7ff9f0e00640 1 --2- 172.21.15.26:0/4157095949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff9ec0cf7d0 0x7ff9ec1324f0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.317+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 >> 172.21.15.26:0/4157095949 conn(0x7ff9ec06a800 msgr2=0x7ff9ec15cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.317+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 shutdown_connections 2024-01-15T18:07:01.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.317+0000 7ff9f0e00640 1 -- 172.21.15.26:0/4157095949 wait complete. 2024-01-15T18:07:01.342 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41818 2024-01-15T18:07:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.556+0000 7fe1a1c00640 1 Processor -- start 2024-01-15T18:07:01.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.556+0000 7fe1a1c00640 1 -- start start 2024-01-15T18:07:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.558+0000 7fe1a1c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe19c071e40 0x7fe19c0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.558+0000 7fe1a1c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe19c15d770 0x7fe19c15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.558+0000 7fe1a1c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19c06c730 con 0x7fe19c143810 2024-01-15T18:07:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.558+0000 7fe1a1c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19c143bf0 con 0x7fe19c15d770 2024-01-15T18:07:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.558+0000 7fe1a1c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe19c0722f0 con 0x7fe19c071e40 2024-01-15T18:07:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.558+0000 7fe19be00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe19c143810 0x7fe19c15e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33992/0 (socket says 172.21.15.26:33992) 2024-01-15T18:07:01.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.558+0000 7fe19be00640 1 -- 172.21.15.26:0/3976259016 learned_addr learned my addr 172.21.15.26:0/3976259016 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.558+0000 7fe1a1200640 1 --2- 172.21.15.26:0/3976259016 >> v2:172.21.15.26:3300/0 conn(0x7fe19c15d770 0x7fe19c15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.559+0000 7fe19b400640 1 --2- 172.21.15.26:0/3976259016 >> v2:172.21.15.124:3300/0 conn(0x7fe19c071e40 0x7fe19c0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.559+0000 7fe19aa00640 1 -- 172.21.15.26:0/3976259016 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334251950 0 0) 0x7fe19c06c730 con 0x7fe19c143810 2024-01-15T18:07:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.559+0000 7fe19aa00640 1 -- 172.21.15.26:0/3976259016 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe17c003520 con 0x7fe19c143810 2024-01-15T18:07:01.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.559+0000 7fe1a1200640 1 -- 172.21.15.26:0/3976259016 >> v2:172.21.15.124:3300/0 conn(0x7fe19c071e40 msgr2=0x7fe19c0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.559+0000 7fe1a1200640 1 --2- 172.21.15.26:0/3976259016 >> v2:172.21.15.124:3300/0 conn(0x7fe19c071e40 0x7fe19c0ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.559+0000 7fe1a1200640 1 -- 172.21.15.26:0/3976259016 >> v1:172.21.15.26:6789/0 conn(0x7fe19c143810 legacy=0x7fe19c15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.559+0000 7fe1a1200640 1 -- 172.21.15.26:0/3976259016 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19c0724d0 con 0x7fe19c15d770 2024-01-15T18:07:01.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.561+0000 7fe1a1200640 1 --2- 172.21.15.26:0/3976259016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 0x7fe19c15dbc0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0b0a40 tx=0x7fe19402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d36527f9df62bd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.561+0000 7fe19aa00640 1 -- 172.21.15.26:0/3976259016 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19402cb60 con 0x7fe19c15d770 2024-01-15T18:07:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.561+0000 7fe19aa00640 1 -- 172.21.15.26:0/3976259016 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19402ccc0 con 0x7fe19c15d770 2024-01-15T18:07:01.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.561+0000 7fe19aa00640 1 -- 172.21.15.26:0/3976259016 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194034660 con 0x7fe19c15d770 2024-01-15T18:07:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.562+0000 7fe1a1c00640 1 -- 172.21.15.26:0/3976259016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 msgr2=0x7fe19c15dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.562+0000 7fe1a1c00640 1 --2- 172.21.15.26:0/3976259016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 0x7fe19c15dbc0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0b0a40 tx=0x7fe19402bdb0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.562+0000 7fe1a1c00640 1 -- 172.21.15.26:0/3976259016 shutdown_connections 2024-01-15T18:07:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.562+0000 7fe1a1c00640 1 --2- 172.21.15.26:0/3976259016 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 0x7fe19c15dbc0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.562+0000 7fe1a1c00640 1 --2- 172.21.15.26:0/3976259016 >> v2:172.21.15.124:3300/0 conn(0x7fe19c071e40 0x7fe19c0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.562+0000 7fe1a1c00640 1 -- 172.21.15.26:0/3976259016 >> 172.21.15.26:0/3976259016 conn(0x7fe19c06a7d0 msgr2=0x7fe19c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.562+0000 7fe1a1c00640 1 -- 172.21.15.26:0/3976259016 shutdown_connections 2024-01-15T18:07:01.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.562+0000 7fe1a1c00640 1 -- 172.21.15.26:0/3976259016 wait complete. 2024-01-15T18:07:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.564+0000 7fe1a1c00640 1 Processor -- start 2024-01-15T18:07:01.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.564+0000 7fe1a1c00640 1 -- start start 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.565+0000 7fe1a1c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe19c071e40 0x7fe19c0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.565+0000 7fe1a1c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe19c143810 0x7fe19c13d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.565+0000 7fe1a1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 0x7fe19c071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.565+0000 7fe1a1c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c143fc0 con 0x7fe19c15d770 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.565+0000 7fe1a1c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c0af3b0 con 0x7fe19c143810 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.565+0000 7fe1a1c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19c0aea80 con 0x7fe19c071e40 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe1a1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 0x7fe19c071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe1a1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 0x7fe19c071310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44764/0 (socket says 172.21.15.26:44764) 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe1a1200640 1 -- 172.21.15.26:0/2871424600 learned_addr learned my addr 172.21.15.26:0/2871424600 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:01.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe19b400640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe19c143810 0x7fe19c13d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe1a1200640 1 -- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe19c071e40 msgr2=0x7fe19c0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe1a1200640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe19c071e40 0x7fe19c0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe1a1200640 1 -- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe19c143810 msgr2=0x7fe19c13d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:01.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe1a1200640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe19c143810 0x7fe19c13d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:01.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe1a1200640 1 -- 172.21.15.26:0/2871424600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194009a70 con 0x7fe19c15d770 2024-01-15T18:07:01.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.566+0000 7fe1a1200640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 0x7fe19c071310 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fe194009e80 tx=0x7fe194004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:01.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.567+0000 7fe193e00640 1 -- 172.21.15.26:0/2871424600 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194040650 con 0x7fe19c15d770 2024-01-15T18:07:01.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.567+0000 7fe193e00640 1 -- 172.21.15.26:0/2871424600 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1940407b0 con 0x7fe19c15d770 2024-01-15T18:07:01.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.567+0000 7fe193e00640 1 -- 172.21.15.26:0/2871424600 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194040b60 con 0x7fe19c15d770 2024-01-15T18:07:01.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.567+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19c0aed60 con 0x7fe19c15d770 2024-01-15T18:07:01.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.567+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe19c0b1870 con 0x7fe19c15d770 2024-01-15T18:07:01.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.568+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe19c067450 con 0x7fe19c15d770 2024-01-15T18:07:01.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.569+0000 7fe193e00640 1 -- 172.21.15.26:0/2871424600 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fe194040cc0 con 0x7fe19c15d770 2024-01-15T18:07:01.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.570+0000 7fe193e00640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17406be90 0x7fe17406e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:01.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.570+0000 7fe193e00640 1 -- 172.21.15.26:0/2871424600 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fe1940ab320 con 0x7fe19c15d770 2024-01-15T18:07:01.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.571+0000 7fe19be00640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17406be90 0x7fe17406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:01.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.572+0000 7fe19be00640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17406be90 0x7fe17406e350 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0ce020 tx=0x7fe18c007930 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:01.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.575+0000 7fe193e00640 1 -- 172.21.15.26:0/2871424600 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe194077be0 con 0x7fe19c15d770 2024-01-15T18:07:01.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:01.807+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41818", "target": ["mon-mgr", ""]}) v1 -- 0x7fe19c0aebf0 con 0x7fe17406be90 2024-01-15T18:07:05.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.555+0000 7fe193e00640 1 -- 172.21.15.26:0/2871424600 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe19c0aebf0 con 0x7fe17406be90 2024-01-15T18:07:05.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.558+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17406be90 msgr2=0x7fe17406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.558+0000 7fe1a1c00640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17406be90 0x7fe17406e350 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe19c0ce020 tx=0x7fe18c007930 comp rx=0 tx=0).stop 2024-01-15T18:07:05.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 msgr2=0x7fe19c071310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 0x7fe19c071310 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fe194009e80 tx=0x7fe194004a40 comp rx=0 tx=0).stop 2024-01-15T18:07:05.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 shutdown_connections 2024-01-15T18:07:05.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17406be90 0x7fe17406e350 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19c15d770 0x7fe19c071310 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe19c143810 0x7fe19c13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 --2- 172.21.15.26:0/2871424600 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe19c071e40 0x7fe19c0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 >> 172.21.15.26:0/2871424600 conn(0x7fe19c06a7d0 msgr2=0x7fe19c06abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:05.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 shutdown_connections 2024-01-15T18:07:05.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.559+0000 7fe1a1c00640 1 -- 172.21.15.26:0/2871424600 wait complete. 2024-01-15T18:07:05.583 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41819 2024-01-15T18:07:05.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.795+0000 7f6078200640 1 Processor -- start 2024-01-15T18:07:05.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.796+0000 7f6078200640 1 -- start start 2024-01-15T18:07:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.797+0000 7f6078200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f607009a170 0x7f60700b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.797+0000 7f6078200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f607006c6b0 0x7f60700cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.797+0000 7f6078200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607006cea0 con 0x7f60700cd240 2024-01-15T18:07:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.797+0000 7f6078200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607006cbc0 con 0x7f607009a170 2024-01-15T18:07:05.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.797+0000 7f6078200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607006f350 con 0x7f607006c6b0 2024-01-15T18:07:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.798+0000 7f6074e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f607009a170 0x7f60700b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.798+0000 7f6074e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f607009a170 0x7f60700b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39128/0 (socket says 172.21.15.26:39128) 2024-01-15T18:07:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.798+0000 7f6074e00640 1 -- 172.21.15.26:0/1382939304 learned_addr learned my addr 172.21.15.26:0/1382939304 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.798+0000 7f6076200640 1 --2- 172.21.15.26:0/1382939304 >> v2:172.21.15.124:3300/0 conn(0x7f607006c6b0 0x7f60700cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.798+0000 7f6074e00640 1 -- 172.21.15.26:0/1382939304 >> v2:172.21.15.124:3300/0 conn(0x7f607006c6b0 msgr2=0x7f60700cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.798+0000 7f6074e00640 1 --2- 172.21.15.26:0/1382939304 >> v2:172.21.15.124:3300/0 conn(0x7f607006c6b0 0x7f60700cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.799+0000 7f6074e00640 1 -- 172.21.15.26:0/1382939304 >> v1:172.21.15.124:6789/0 conn(0x7f60700cd240 legacy=0x7f6070099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.799+0000 7f6074e00640 1 -- 172.21.15.26:0/1382939304 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607006f630 con 0x7f607009a170 2024-01-15T18:07:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.800+0000 7f6074e00640 1 --2- 172.21.15.26:0/1382939304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 0x7f60700b3820 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f60700cdc40 tx=0x7f605802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dbf12a51fe8b0a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.800+0000 7f606fe00640 1 -- 172.21.15.26:0/1382939304 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058030860 con 0x7f607009a170 2024-01-15T18:07:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.801+0000 7f606fe00640 1 -- 172.21.15.26:0/1382939304 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60580309c0 con 0x7f607009a170 2024-01-15T18:07:05.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.801+0000 7f606fe00640 1 -- 172.21.15.26:0/1382939304 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058030d50 con 0x7f607009a170 2024-01-15T18:07:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.801+0000 7f6078200640 1 -- 172.21.15.26:0/1382939304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 msgr2=0x7f60700b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.801+0000 7f6078200640 1 --2- 172.21.15.26:0/1382939304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 0x7f60700b3820 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f60700cdc40 tx=0x7f605802b9a0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.801+0000 7f6078200640 1 -- 172.21.15.26:0/1382939304 shutdown_connections 2024-01-15T18:07:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.801+0000 7f6078200640 1 --2- 172.21.15.26:0/1382939304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 0x7f60700b3820 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.801+0000 7f6078200640 1 --2- 172.21.15.26:0/1382939304 >> v2:172.21.15.124:3300/0 conn(0x7f607006c6b0 0x7f60700cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.801+0000 7f6078200640 1 -- 172.21.15.26:0/1382939304 >> 172.21.15.26:0/1382939304 conn(0x7f60700b2d00 msgr2=0x7f60700b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.802+0000 7f6078200640 1 -- 172.21.15.26:0/1382939304 shutdown_connections 2024-01-15T18:07:05.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.802+0000 7f6078200640 1 -- 172.21.15.26:0/1382939304 wait complete. 2024-01-15T18:07:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.803+0000 7f6078200640 1 Processor -- start 2024-01-15T18:07:05.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.804+0000 7f6078200640 1 -- start start 2024-01-15T18:07:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.804+0000 7f6078200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607006c6b0 0x7f607015ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6078200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 0x7f60700b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6078200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60700cd240 0x7f60700c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6078200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60700c6a70 con 0x7f607009a170 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6078200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60700c5ea0 con 0x7f60700cd240 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6078200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607015d3c0 con 0x7f607006c6b0 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6074e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 0x7f60700b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6074e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 0x7f60700b4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39136/0 (socket says 172.21.15.26:39136) 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6074e00640 1 -- 172.21.15.26:0/136228764 learned_addr learned my addr 172.21.15.26:0/136228764 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6075800640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607006c6b0 0x7f607015ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6076200640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60700cd240 0x7f60700c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6074e00640 1 -- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607006c6b0 msgr2=0x7f607015ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6074e00640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607006c6b0 0x7f607015ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6074e00640 1 -- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60700cd240 msgr2=0x7f60700c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6074e00640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60700cd240 0x7f60700c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6074e00640 1 -- 172.21.15.26:0/136228764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60580094d0 con 0x7f607009a170 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.805+0000 7f6076200640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60700cd240 0x7f60700c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.806+0000 7f6074e00640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 0x7f60700b4850 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f605802b9e0 tx=0x7f6058004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.806+0000 7f606d600640 1 -- 172.21.15.26:0/136228764 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058039d60 con 0x7f607009a170 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.806+0000 7f606d600640 1 -- 172.21.15.26:0/136228764 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6058043400 con 0x7f607009a170 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.806+0000 7f606d600640 1 -- 172.21.15.26:0/136228764 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60580437b0 con 0x7f607009a170 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.806+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60700b51d0 con 0x7f607009a170 2024-01-15T18:07:05.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.806+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60700c6420 con 0x7f607009a170 2024-01-15T18:07:05.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.807+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60700675d0 con 0x7f607009a170 2024-01-15T18:07:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.808+0000 7f606d600640 1 -- 172.21.15.26:0/136228764 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f6058043910 con 0x7f607009a170 2024-01-15T18:07:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.809+0000 7f606d600640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f604806be90 0x7f604806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.810+0000 7f606d600640 1 -- 172.21.15.26:0/136228764 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f605803e070 con 0x7f607009a170 2024-01-15T18:07:05.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.810+0000 7f6075800640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f604806be90 0x7f604806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:05.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.811+0000 7f6075800640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f604806be90 0x7f604806e350 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6060000f60 tx=0x7f6060009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:05.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:05.814+0000 7f606d600640 1 -- 172.21.15.26:0/136228764 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6058078200 con 0x7f607009a170 2024-01-15T18:07:06.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:06.052+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41819", "target": ["mon-mgr", ""]}) v1 -- 0x7f60700cd620 con 0x7f604806be90 2024-01-15T18:07:11.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.477+0000 7f606d600640 1 -- 172.21.15.26:0/136228764 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6060001360 con 0x7f604806be90 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.480+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f604806be90 msgr2=0x7f604806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.480+0000 7f6078200640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f604806be90 0x7f604806e350 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6060000f60 tx=0x7f6060009040 comp rx=0 tx=0).stop 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.480+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 msgr2=0x7f60700b4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.480+0000 7f6078200640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 0x7f60700b4850 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f605802b9e0 tx=0x7f6058004730 comp rx=0 tx=0).stop 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.481+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 shutdown_connections 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.481+0000 7f6078200640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f604806be90 0x7f604806e350 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.481+0000 7f6078200640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60700cd240 0x7f60700c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.481+0000 7f6078200640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f607009a170 0x7f60700b4850 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.481+0000 7f6078200640 1 --2- 172.21.15.26:0/136228764 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f607006c6b0 0x7f607015ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.481+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 >> 172.21.15.26:0/136228764 conn(0x7f60700b2d00 msgr2=0x7f607006b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.481+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 shutdown_connections 2024-01-15T18:07:11.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.481+0000 7f6078200640 1 -- 172.21.15.26:0/136228764 wait complete. 2024-01-15T18:07:11.505 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone41820 2024-01-15T18:07:11.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.730+0000 7f0c1b200640 1 Processor -- start 2024-01-15T18:07:11.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.730+0000 7f0c1b200640 1 -- start start 2024-01-15T18:07:11.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.732+0000 7f0c1b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c1c0cfde0 0x7f0c1c0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.732+0000 7f0c1b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0d1c60 con 0x7f0c1c0cf9a0 2024-01-15T18:07:11.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.732+0000 7f0c1b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0b2730 con 0x7f0c1c15d4b0 2024-01-15T18:07:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.732+0000 7f0c1b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0d1ec0 con 0x7f0c1c0cfde0 2024-01-15T18:07:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.732+0000 7f0c19400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0c1c0cf9a0 0x7f0c1c0b31d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:32986/0 (socket says 172.21.15.26:32986) 2024-01-15T18:07:11.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.732+0000 7f0c19400640 1 -- 172.21.15.26:0/2375372852 learned_addr learned my addr 172.21.15.26:0/2375372852 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.732+0000 7f0c1a800640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c1c0cfde0 0x7f0c1c0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800005743 0 0) 0x7f0c1c0d1c60 con 0x7f0c1c0cf9a0 2024-01-15T18:07:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bfc003520 con 0x7f0c1c0cf9a0 2024-01-15T18:07:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067632870 0 0) 0x7f0c1c0b2730 con 0x7f0c1c15d4b0 2024-01-15T18:07:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c1c0d1c60 con 0x7f0c1c15d4b0 2024-01-15T18:07:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4199731720 0 0) 0x7f0bfc003520 con 0x7f0c1c0cf9a0 2024-01-15T18:07:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c1c0cfde0 msgr2=0x7f0c1c0b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c1c0cfde0 0x7f0c1c0b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 >> v1:172.21.15.124:6789/0 conn(0x7f0c1c15d4b0 legacy=0x7f0c1c0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c0b29d0 con 0x7f0c1c0cf9a0 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f0c040015f0 con 0x7f0c1c0cf9a0 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 >> v1:172.21.15.26:6789/0 conn(0x7f0c1c0cf9a0 legacy=0x7f0c1c0b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc003b30 0x7f0bfc003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0bfc0047c0 0x7f0bfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c1c0cfde0 0x7f0bfc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0d1ec0 con 0x7f0bfc003b30 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc00ba30 con 0x7f0bfc0047c0 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.733+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bfc00bbf0 con 0x7f0c1c0cfde0 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.734+0000 7f0c19e00640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc003b30 0x7f0bfc003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.734+0000 7f0c19400640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0bfc0047c0 0x7f0bfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.734+0000 7f0c1a800640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c1c0cfde0 0x7f0bfc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.734+0000 7f0c19e00640 1 -- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c1c0cfde0 msgr2=0x7f0bfc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.734+0000 7f0c19e00640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c1c0cfde0 0x7f0bfc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.734+0000 7f0c19e00640 1 -- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0bfc0047c0 msgr2=0x7f0bfc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.734+0000 7f0c19e00640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0bfc0047c0 0x7f0bfc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.734+0000 7f0c19e00640 1 -- 172.21.15.26:0/2375372852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc003520 con 0x7f0bfc003b30 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.735+0000 7f0c19e00640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc003b30 0x7f0bfc003f30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c001fa0 tx=0x7f0bfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf19210bb93adba server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.736+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c003b70 con 0x7f0bfc003b30 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.736+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c003cd0 con 0x7f0bfc003b30 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.736+0000 7f0c18a00640 1 -- 172.21.15.26:0/2375372852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0052f0 con 0x7f0bfc003b30 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.736+0000 7f0c1b200640 1 -- 172.21.15.26:0/2375372852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc003b30 msgr2=0x7f0bfc003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.736+0000 7f0c1b200640 1 --2- 172.21.15.26:0/2375372852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0bfc003b30 0x7f0bfc003f30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c001fa0 tx=0x7f0bfc0057f0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.736+0000 7f0c1a800640 1 -- 172.21.15.26:0/2375372852 reap_dead start 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.736+0000 7f0c1b200640 1 -- 172.21.15.26:0/2375372852 shutdown_connections 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.736+0000 7f0c1b200640 1 -- 172.21.15.26:0/2375372852 >> 172.21.15.26:0/2375372852 conn(0x7f0c1c0d3000 msgr2=0x7f0c1c0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:11.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.737+0000 7f0c1b200640 1 -- 172.21.15.26:0/2375372852 shutdown_connections 2024-01-15T18:07:11.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.737+0000 7f0c1b200640 1 -- 172.21.15.26:0/2375372852 wait complete. 2024-01-15T18:07:11.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.738+0000 7f0c1b200640 1 Processor -- start 2024-01-15T18:07:11.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.739+0000 7f0c1b200640 1 -- start start 2024-01-15T18:07:11.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.739+0000 7f0c1b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bfc003b30 0x7f0c1c0b7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.739+0000 7f0c1b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c1c0b6a00 0x7f0c1c0ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.739+0000 7f0c1b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c1c14c000 0x7f0c1c14c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.739+0000 7f0c1b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c14c9c0 con 0x7f0c1c14c000 2024-01-15T18:07:11.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.739+0000 7f0c1b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0b7b30 con 0x7f0c1c0b6a00 2024-01-15T18:07:11.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.739+0000 7f0c1b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c0b7e10 con 0x7f0bfc003b30 2024-01-15T18:07:11.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c1a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c1c14c000 0x7f0c1c14c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c1a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c1c14c000 0x7f0c1c14c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:39140/0 (socket says 172.21.15.26:39140) 2024-01-15T18:07:11.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c1a800640 1 -- 172.21.15.26:0/1364002252 learned_addr learned my addr 172.21.15.26:0/1364002252 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c19400640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c1c0b6a00 0x7f0c1c0ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c19e00640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bfc003b30 0x7f0c1c0b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c1a800640 1 -- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bfc003b30 msgr2=0x7f0c1c0b7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c1a800640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bfc003b30 0x7f0c1c0b7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c1a800640 1 -- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c1c0b6a00 msgr2=0x7f0c1c0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c1a800640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c1c0b6a00 0x7f0c1c0ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c1a800640 1 -- 172.21.15.26:0/1364002252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c0b80f0 con 0x7f0c1c14c000 2024-01-15T18:07:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.740+0000 7f0c19400640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c1c0b6a00 0x7f0c1c0ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.741+0000 7f0c1a800640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c1c14c000 0x7f0c1c14c480 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c06aaa0 tx=0x7f0c14011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.741+0000 7f0c12000640 1 -- 172.21.15.26:0/1364002252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14004500 con 0x7f0c1c14c000 2024-01-15T18:07:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.741+0000 7f0c12000640 1 -- 172.21.15.26:0/1364002252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c14003610 con 0x7f0c1c14c000 2024-01-15T18:07:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.741+0000 7f0c12000640 1 -- 172.21.15.26:0/1364002252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c140039e0 con 0x7f0c1c14c000 2024-01-15T18:07:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.741+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1c0b83d0 con 0x7f0c1c14c000 2024-01-15T18:07:11.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.741+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1c0b7750 con 0x7f0c1c14c000 2024-01-15T18:07:11.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.742+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1c067450 con 0x7f0c1c14c000 2024-01-15T18:07:11.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.743+0000 7f0c12000640 1 -- 172.21.15.26:0/1364002252 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f0c14003b40 con 0x7f0c1c14c000 2024-01-15T18:07:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.744+0000 7f0c12000640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bf406bec0 0x7f0bf406e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.744+0000 7f0c12000640 1 -- 172.21.15.26:0/1364002252 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f0c140886d0 con 0x7f0c1c14c000 2024-01-15T18:07:11.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.745+0000 7f0c19e00640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bf406bec0 0x7f0bf406e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:11.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.747+0000 7f0c19e00640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bf406bec0 0x7f0bf406e380 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c001fa0 tx=0x7f0c0c009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:11.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.749+0000 7f0c12000640 1 -- 172.21.15.26:0/1364002252 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c14054ca0 con 0x7f0c1c14c000 2024-01-15T18:07:11.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:11.982+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone41820", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c1c0b7910 con 0x7f0bf406bec0 2024-01-15T18:07:16.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.114+0000 7f0c12000640 1 -- 172.21.15.26:0/1364002252 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0c0c006b50 con 0x7f0bf406bec0 2024-01-15T18:07:16.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bf406bec0 msgr2=0x7f0bf406e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bf406bec0 0x7f0bf406e380 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c001fa0 tx=0x7f0c0c009030 comp rx=0 tx=0).stop 2024-01-15T18:07:16.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c1c14c000 msgr2=0x7f0c1c14c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c1c14c000 0x7f0c1c14c480 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c06aaa0 tx=0x7f0c14011920 comp rx=0 tx=0).stop 2024-01-15T18:07:16.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 shutdown_connections 2024-01-15T18:07:16.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bf406bec0 0x7f0bf406e380 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c1c14c000 0x7f0c1c14c480 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c1c0b6a00 0x7f0c1c0ceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 --2- 172.21.15.26:0/1364002252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0bfc003b30 0x7f0c1c0b7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 >> 172.21.15.26:0/1364002252 conn(0x7f0c1c0d3000 msgr2=0x7f0c1c0cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:16.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 shutdown_connections 2024-01-15T18:07:16.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.118+0000 7f0c1b200640 1 -- 172.21.15.26:0/1364002252 wait complete. 2024-01-15T18:07:16.144 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:16.145 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4181 2024-01-15T18:07:16.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.366+0000 7f2755c00640 1 Processor -- start 2024-01-15T18:07:16.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.366+0000 7f2755c00640 1 -- start start 2024-01-15T18:07:16.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.368+0000 7f2755c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2750071e40 0x7f27500ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.368+0000 7f2755c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275006c730 con 0x7f275015d770 2024-01-15T18:07:16.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.368+0000 7f2755c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27500724d0 con 0x7f2750143810 2024-01-15T18:07:16.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.368+0000 7f2755c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2750143bf0 con 0x7f2750071e40 2024-01-15T18:07:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.368+0000 7f2747e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2750071e40 0x7f27500ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.368+0000 7f2747e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2750071e40 0x7f27500ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46112/0 (socket says 172.21.15.26:46112) 2024-01-15T18:07:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.368+0000 7f2747e00640 1 -- 172.21.15.26:0/2952342181 learned_addr learned my addr 172.21.15.26:0/2952342181 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.369+0000 7f2747e00640 1 -- 172.21.15.26:0/2952342181 >> v1:172.21.15.124:6789/0 conn(0x7f2750143810 legacy=0x7f275015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.369+0000 7f2747e00640 1 -- 172.21.15.26:0/2952342181 >> v1:172.21.15.26:6789/0 conn(0x7f275015d770 legacy=0x7f275015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.369+0000 7f2747e00640 1 -- 172.21.15.26:0/2952342181 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275006c9f0 con 0x7f2750071e40 2024-01-15T18:07:16.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.370+0000 7f2747e00640 1 --2- 172.21.15.26:0/2952342181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 0x7f27500ae170 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2738009870 tx=0x7f273802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb534c70466de8a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:16.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.370+0000 7f274f400640 1 -- 172.21.15.26:0/2952342181 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738031870 con 0x7f2750071e40 2024-01-15T18:07:16.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.370+0000 7f274f400640 1 -- 172.21.15.26:0/2952342181 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27380319d0 con 0x7f2750071e40 2024-01-15T18:07:16.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.370+0000 7f274f400640 1 -- 172.21.15.26:0/2952342181 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738031d60 con 0x7f2750071e40 2024-01-15T18:07:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.371+0000 7f2755c00640 1 -- 172.21.15.26:0/2952342181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 msgr2=0x7f27500ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.371+0000 7f2755c00640 1 --2- 172.21.15.26:0/2952342181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 0x7f27500ae170 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2738009870 tx=0x7f273802b910 comp rx=0 tx=0).stop 2024-01-15T18:07:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.371+0000 7f2755c00640 1 -- 172.21.15.26:0/2952342181 shutdown_connections 2024-01-15T18:07:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.371+0000 7f2755c00640 1 --2- 172.21.15.26:0/2952342181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 0x7f27500ae170 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.371+0000 7f2755c00640 1 -- 172.21.15.26:0/2952342181 >> 172.21.15.26:0/2952342181 conn(0x7f275006a7d0 msgr2=0x7f275006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.371+0000 7f2755c00640 1 -- 172.21.15.26:0/2952342181 shutdown_connections 2024-01-15T18:07:16.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.371+0000 7f2755c00640 1 -- 172.21.15.26:0/2952342181 wait complete. 2024-01-15T18:07:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.373+0000 7f2755c00640 1 Processor -- start 2024-01-15T18:07:16.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.374+0000 7f2755c00640 1 -- start start 2024-01-15T18:07:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f2755c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 0x7f275015cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f2755c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2750143810 0x7f27501449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f2755c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f275015d770 0x7f27500b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f2755c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27500722f0 con 0x7f2750071e40 2024-01-15T18:07:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f2755c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27500aeaa0 con 0x7f2750143810 2024-01-15T18:07:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f2755c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27500aed80 con 0x7f275015d770 2024-01-15T18:07:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f274fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 0x7f275015cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f274fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 0x7f275015cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46118/0 (socket says 172.21.15.26:46118) 2024-01-15T18:07:16.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f274fe00640 1 -- 172.21.15.26:0/2013089429 learned_addr learned my addr 172.21.15.26:0/2013089429 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f2747e00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2750143810 0x7f27501449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.375+0000 7f2755200640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f275015d770 0x7f27500b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f274fe00640 1 -- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f275015d770 msgr2=0x7f27500b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f274fe00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f275015d770 0x7f27500b0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f274fe00640 1 -- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2750143810 msgr2=0x7f27501449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f274fe00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2750143810 0x7f27501449f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f274fe00640 1 -- 172.21.15.26:0/2013089429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27380094d0 con 0x7f2750071e40 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f274fe00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 0x7f275015cdb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f274000c450 tx=0x7f274000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f274cc00640 1 -- 172.21.15.26:0/2013089429 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27400052d0 con 0x7f2750071e40 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f274cc00640 1 -- 172.21.15.26:0/2013089429 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2740005430 con 0x7f2750071e40 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f274cc00640 1 -- 172.21.15.26:0/2013089429 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740012870 con 0x7f2750071e40 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.376+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27500af060 con 0x7f2750071e40 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.377+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275015d2f0 con 0x7f2750071e40 2024-01-15T18:07:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.378+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27500af580 con 0x7f2750071e40 2024-01-15T18:07:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.379+0000 7f274cc00640 1 -- 172.21.15.26:0/2013089429 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f27400129d0 con 0x7f2750071e40 2024-01-15T18:07:16.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.380+0000 7f274cc00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f272806be90 0x7f272806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.380+0000 7f274cc00640 1 -- 172.21.15.26:0/2013089429 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f2740088350 con 0x7f2750071e40 2024-01-15T18:07:16.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.380+0000 7f2747e00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f272806be90 0x7f272806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.382+0000 7f2747e00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f272806be90 0x7f272806e350 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2738036000 tx=0x7f273803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:16.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.385+0000 7f274cc00640 1 -- 172.21.15.26:0/2013089429 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27500af580 con 0x7f2750071e40 2024-01-15T18:07:16.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.620+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4181", "target": ["mon-mgr", ""]}) v1 -- 0x7f275006cfb0 con 0x7f272806be90 2024-01-15T18:07:16.636 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:16.632+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:16.636 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:16.632+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:16.636 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:16.632+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:16.636 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:16.632+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:16.636 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:16.632+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:16.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.670+0000 7f274cc00640 1 -- 172.21.15.26:0/2013089429 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f273803a420 con 0x7f272806be90 2024-01-15T18:07:16.674 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:16.674 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:16.675 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:07:16.675 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:16.675 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f272806be90 msgr2=0x7f272806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f272806be90 0x7f272806e350 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2738036000 tx=0x7f273803c040 comp rx=0 tx=0).stop 2024-01-15T18:07:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 msgr2=0x7f275015cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 0x7f275015cdb0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f274000c450 tx=0x7f274000c910 comp rx=0 tx=0).stop 2024-01-15T18:07:16.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 shutdown_connections 2024-01-15T18:07:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f275015d770 0x7f27500b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f272806be90 0x7f272806e350 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2750143810 0x7f27501449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 --2- 172.21.15.26:0/2013089429 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2750071e40 0x7f275015cdb0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 >> 172.21.15.26:0/2013089429 conn(0x7f275006a7d0 msgr2=0x7f275013f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 shutdown_connections 2024-01-15T18:07:16.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.674+0000 7f2755c00640 1 -- 172.21.15.26:0/2013089429 wait complete. 2024-01-15T18:07:16.699 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:16.699 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4182 2024-01-15T18:07:16.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.922+0000 7f4536400640 1 Processor -- start 2024-01-15T18:07:16.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.922+0000 7f4536400640 1 -- start start 2024-01-15T18:07:16.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.923+0000 7f4536400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 0x7f45300d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.923+0000 7f4536400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f45300cf4c0 0x7f45301614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.923+0000 7f4536400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f453015cc90 0x7f453015d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.923+0000 7f4536400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45300b2920 con 0x7f453015cc90 2024-01-15T18:07:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.923+0000 7f4536400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4530161a00 con 0x7f45300cf4c0 2024-01-15T18:07:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.923+0000 7f4536400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45300cb670 con 0x7f45301610e0 2024-01-15T18:07:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.924+0000 7f452fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f453015cc90 0x7f453015d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.924+0000 7f452fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f453015cc90 0x7f453015d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46126/0 (socket says 172.21.15.26:46126) 2024-01-15T18:07:16.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.924+0000 7f452fe00640 1 -- 172.21.15.26:0/2472088556 learned_addr learned my addr 172.21.15.26:0/2472088556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:16.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.924+0000 7f452ea00640 1 --2- 172.21.15.26:0/2472088556 >> v2:172.21.15.124:3300/0 conn(0x7f45300cf4c0 0x7f45301614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.924+0000 7f452f400640 1 --2- 172.21.15.26:0/2472088556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 0x7f45300d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.925+0000 7f452fe00640 1 -- 172.21.15.26:0/2472088556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 msgr2=0x7f45300d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:16.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.925+0000 7f452fe00640 1 --2- 172.21.15.26:0/2472088556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 0x7f45300d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.925+0000 7f452fe00640 1 -- 172.21.15.26:0/2472088556 >> v2:172.21.15.124:3300/0 conn(0x7f45300cf4c0 msgr2=0x7f45301614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:16.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.925+0000 7f452fe00640 1 --2- 172.21.15.26:0/2472088556 >> v2:172.21.15.124:3300/0 conn(0x7f45300cf4c0 0x7f45301614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.925+0000 7f452fe00640 1 -- 172.21.15.26:0/2472088556 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45300cb950 con 0x7f453015cc90 2024-01-15T18:07:16.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.925+0000 7f452ea00640 1 --2- 172.21.15.26:0/2472088556 >> v2:172.21.15.124:3300/0 conn(0x7f45300cf4c0 0x7f45301614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:16.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.926+0000 7f452fe00640 1 --2- 172.21.15.26:0/2472088556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f453015cc90 0x7f453015d120 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4530073e10 tx=0x7f452802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c13045ed8f8825 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.926+0000 7f452e000640 1 -- 172.21.15.26:0/2472088556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4528031870 con 0x7f453015cc90 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.926+0000 7f452e000640 1 -- 172.21.15.26:0/2472088556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45280319d0 con 0x7f453015cc90 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.926+0000 7f452e000640 1 -- 172.21.15.26:0/2472088556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4528031d60 con 0x7f453015cc90 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.927+0000 7f4536400640 1 -- 172.21.15.26:0/2472088556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f453015cc90 msgr2=0x7f453015d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.927+0000 7f4536400640 1 --2- 172.21.15.26:0/2472088556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f453015cc90 0x7f453015d120 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f4530073e10 tx=0x7f452802b910 comp rx=0 tx=0).stop 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.927+0000 7f4536400640 1 -- 172.21.15.26:0/2472088556 shutdown_connections 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.927+0000 7f4536400640 1 --2- 172.21.15.26:0/2472088556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f453015cc90 0x7f453015d120 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.927+0000 7f4536400640 1 --2- 172.21.15.26:0/2472088556 >> v2:172.21.15.124:3300/0 conn(0x7f45300cf4c0 0x7f45301614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.927+0000 7f4536400640 1 --2- 172.21.15.26:0/2472088556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 0x7f45300d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.927+0000 7f4536400640 1 -- 172.21.15.26:0/2472088556 >> 172.21.15.26:0/2472088556 conn(0x7f45300d1020 msgr2=0x7f45300d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.927+0000 7f4536400640 1 -- 172.21.15.26:0/2472088556 shutdown_connections 2024-01-15T18:07:16.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.927+0000 7f4536400640 1 -- 172.21.15.26:0/2472088556 wait complete. 2024-01-15T18:07:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.929+0000 7f4536400640 1 Processor -- start 2024-01-15T18:07:16.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.929+0000 7f4536400640 1 -- start start 2024-01-15T18:07:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.930+0000 7f4536400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45300cf4c0 0x7f45300b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.930+0000 7f4536400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f453015cc90 0x7f4530154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.930+0000 7f4536400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 0x7f4530151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.930+0000 7f4536400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45300c9100 con 0x7f45300cf4c0 2024-01-15T18:07:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.930+0000 7f4536400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45300c8580 con 0x7f453015cc90 2024-01-15T18:07:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.930+0000 7f4536400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45300c8860 con 0x7f45301610e0 2024-01-15T18:07:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.930+0000 7f452f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45300cf4c0 0x7f45300b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.930+0000 7f452f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45300cf4c0 0x7f45300b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46140/0 (socket says 172.21.15.26:46140) 2024-01-15T18:07:16.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.930+0000 7f452f400640 1 -- 172.21.15.26:0/3305784573 learned_addr learned my addr 172.21.15.26:0/3305784573 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452ea00640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f453015cc90 0x7f4530154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452fe00640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 0x7f4530151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452f400640 1 -- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 msgr2=0x7f4530151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452f400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 0x7f4530151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452f400640 1 -- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f453015cc90 msgr2=0x7f4530154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:16.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452f400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f453015cc90 0x7f4530154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452f400640 1 -- 172.21.15.26:0/3305784573 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45280094d0 con 0x7f45300cf4c0 2024-01-15T18:07:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452fe00640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 0x7f4530151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452ea00640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f453015cc90 0x7f4530154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f452f400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45300cf4c0 0x7f45300b5fb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4520007ef0 tx=0x7f452000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f4527400640 1 -- 172.21.15.26:0/3305784573 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520014620 con 0x7f45300cf4c0 2024-01-15T18:07:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f4527400640 1 -- 172.21.15.26:0/3305784573 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4520014780 con 0x7f45300cf4c0 2024-01-15T18:07:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f4527400640 1 -- 172.21.15.26:0/3305784573 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4520014b30 con 0x7f45300cf4c0 2024-01-15T18:07:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45300c8b40 con 0x7f45300cf4c0 2024-01-15T18:07:16.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.931+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45300b64f0 con 0x7f45300cf4c0 2024-01-15T18:07:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.933+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45300b66e0 con 0x7f45300cf4c0 2024-01-15T18:07:16.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.934+0000 7f4527400640 1 -- 172.21.15.26:0/3305784573 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f4520014c90 con 0x7f45300cf4c0 2024-01-15T18:07:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.934+0000 7f4527400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f450806be90 0x7f450806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.935+0000 7f4527400640 1 -- 172.21.15.26:0/3305784573 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f4520088f50 con 0x7f45300cf4c0 2024-01-15T18:07:16.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.935+0000 7f452ea00640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f450806be90 0x7f450806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:16.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.936+0000 7f452ea00640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f450806be90 0x7f450806e350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4518006fd0 tx=0x7f4518008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:16.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:16.939+0000 7f4527400640 1 -- 172.21.15.26:0/3305784573 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4520055810 con 0x7f45300cf4c0 2024-01-15T18:07:17.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.173+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4182", "target": ["mon-mgr", ""]}) v1 -- 0x7f45300cc1f0 con 0x7f450806be90 2024-01-15T18:07:17.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.199+0000 7f4527400640 1 -- 172.21.15.26:0/3305784573 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f451800c040 con 0x7f450806be90 2024-01-15T18:07:17.203 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:17.203 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:17.204 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:07:17.204 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:17.204 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:17.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f450806be90 msgr2=0x7f450806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f450806be90 0x7f450806e350 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4518006fd0 tx=0x7f4518008040 comp rx=0 tx=0).stop 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45300cf4c0 msgr2=0x7f45300b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45300cf4c0 0x7f45300b5fb0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4520007ef0 tx=0x7f452000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 shutdown_connections 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f45301610e0 0x7f4530151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f450806be90 0x7f450806e350 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f453015cc90 0x7f4530154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 --2- 172.21.15.26:0/3305784573 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f45300cf4c0 0x7f45300b5fb0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 >> 172.21.15.26:0/3305784573 conn(0x7f45300d1020 msgr2=0x7f45300cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 shutdown_connections 2024-01-15T18:07:17.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.202+0000 7f4536400640 1 -- 172.21.15.26:0/3305784573 wait complete. 2024-01-15T18:07:17.224 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:17.225 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4183 2024-01-15T18:07:17.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.450+0000 7ff43fe00640 1 Processor -- start 2024-01-15T18:07:17.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.450+0000 7ff43fe00640 1 -- start start 2024-01-15T18:07:17.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.452+0000 7ff43fe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff44015a690 0x7ff44006c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.452+0000 7ff43fe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4400b1e30 con 0x7ff4400b3b20 2024-01-15T18:07:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.452+0000 7ff43fe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44015ab20 con 0x7ff44006cc70 2024-01-15T18:07:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.452+0000 7ff43fe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4400b20a0 con 0x7ff44015a690 2024-01-15T18:07:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.452+0000 7ff43ea00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff4400b3b20 0x7ff4401505b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56966/0 (socket says 172.21.15.26:56966) 2024-01-15T18:07:17.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.452+0000 7ff43ea00640 1 -- 172.21.15.26:0/591370004 learned_addr learned my addr 172.21.15.26:0/591370004 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:17.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.452+0000 7ff435e00640 1 --2- 172.21.15.26:0/591370004 >> v2:172.21.15.124:3300/0 conn(0x7ff44015a690 0x7ff44006c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543417110 0 0) 0x7ff4400b1e30 con 0x7ff4400b3b20 2024-01-15T18:07:17.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff420005730 con 0x7ff4400b3b20 2024-01-15T18:07:17.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2496989868 0 0) 0x7ff420005730 con 0x7ff4400b3b20 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 >> v2:172.21.15.124:3300/0 conn(0x7ff44015a690 msgr2=0x7ff44006c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 --2- 172.21.15.26:0/591370004 >> v2:172.21.15.124:3300/0 conn(0x7ff44015a690 0x7ff44006c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 >> v1:172.21.15.124:6789/0 conn(0x7ff44006cc70 legacy=0x7ff44006d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4400b2380 con 0x7ff4400b3b20 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7ff430001670 con 0x7ff4400b3b20 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 >> v1:172.21.15.26:6789/0 conn(0x7ff4400b3b20 legacy=0x7ff4401505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff420005bd0 0x7ff420005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4200069d0 0x7ff420006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff420007390 0x7ff42000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4400b20a0 con 0x7ff4200069d0 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42000e290 con 0x7ff420005bd0 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.453+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42000e450 con 0x7ff420007390 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.454+0000 7ff435e00640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4200069d0 0x7ff420006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.454+0000 7ff43ea00640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff420005bd0 0x7ff420005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.454+0000 7ff43f400640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff420007390 0x7ff42000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.454+0000 7ff435e00640 1 -- 172.21.15.26:0/591370004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff420007390 msgr2=0x7ff42000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.454+0000 7ff435e00640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff420007390 0x7ff42000db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.454+0000 7ff435e00640 1 -- 172.21.15.26:0/591370004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff420005bd0 msgr2=0x7ff420005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.454+0000 7ff435e00640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff420005bd0 0x7ff420005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.454+0000 7ff435e00640 1 -- 172.21.15.26:0/591370004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff420005730 con 0x7ff4200069d0 2024-01-15T18:07:17.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.454+0000 7ff43f400640 1 -- 172.21.15.26:0/591370004 reap_dead start 2024-01-15T18:07:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.455+0000 7ff435e00640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4200069d0 0x7ff420006e50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff428009ac0 tx=0x7ff420007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e84dd492fa6afd5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42802c6d0 con 0x7ff4200069d0 2024-01-15T18:07:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42802c830 con 0x7ff4200069d0 2024-01-15T18:07:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43e000640 1 -- 172.21.15.26:0/591370004 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42802cbc0 con 0x7ff4200069d0 2024-01-15T18:07:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43fe00640 1 -- 172.21.15.26:0/591370004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4200069d0 msgr2=0x7ff420006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43fe00640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4200069d0 0x7ff420006e50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff428009ac0 tx=0x7ff420007a00 comp rx=0 tx=0).stop 2024-01-15T18:07:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43fe00640 1 -- 172.21.15.26:0/591370004 shutdown_connections 2024-01-15T18:07:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43fe00640 1 --2- 172.21.15.26:0/591370004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff4200069d0 0x7ff420006e50 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43fe00640 1 -- 172.21.15.26:0/591370004 >> 172.21.15.26:0/591370004 conn(0x7ff44006a7d0 msgr2=0x7ff44006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43fe00640 1 -- 172.21.15.26:0/591370004 shutdown_connections 2024-01-15T18:07:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.456+0000 7ff43fe00640 1 -- 172.21.15.26:0/591370004 wait complete. 2024-01-15T18:07:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.458+0000 7ff43fe00640 1 Processor -- start 2024-01-15T18:07:17.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.459+0000 7ff43fe00640 1 -- start start 2024-01-15T18:07:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.459+0000 7ff43fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200069d0 0x7ff44015c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43fe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4401e5270 0x7ff4401e56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440070160 0x7ff4400705e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4401e6dd0 con 0x7ff440070160 2024-01-15T18:07:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43fe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4401e5d30 con 0x7ff4401e5270 2024-01-15T18:07:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4401e6010 con 0x7ff4200069d0 2024-01-15T18:07:17.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440070160 0x7ff4400705e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440070160 0x7ff4400705e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46154/0 (socket says 172.21.15.26:46154) 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43f400640 1 -- 172.21.15.26:0/3978954058 learned_addr learned my addr 172.21.15.26:0/3978954058 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43ea00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200069d0 0x7ff44015c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff435e00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4401e5270 0x7ff4401e56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43f400640 1 -- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200069d0 msgr2=0x7ff44015c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43f400640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200069d0 0x7ff44015c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43f400640 1 -- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4401e5270 msgr2=0x7ff4401e56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43f400640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4401e5270 0x7ff4401e56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff43f400640 1 -- 172.21.15.26:0/3978954058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4401e62f0 con 0x7ff440070160 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.460+0000 7ff435e00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4401e5270 0x7ff4401e56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:17.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.461+0000 7ff43f400640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440070160 0x7ff4400705e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff44006e2e0 tx=0x7ff43800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.461+0000 7ff437400640 1 -- 172.21.15.26:0/3978954058 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438003b70 con 0x7ff440070160 2024-01-15T18:07:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.461+0000 7ff437400640 1 -- 172.21.15.26:0/3978954058 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438003cd0 con 0x7ff440070160 2024-01-15T18:07:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.461+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4401e65d0 con 0x7ff440070160 2024-01-15T18:07:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.461+0000 7ff437400640 1 -- 172.21.15.26:0/3978954058 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380052b0 con 0x7ff440070160 2024-01-15T18:07:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.461+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44015dd80 con 0x7ff440070160 2024-01-15T18:07:17.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.462+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4401e6af0 con 0x7ff440070160 2024-01-15T18:07:17.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.463+0000 7ff437400640 1 -- 172.21.15.26:0/3978954058 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7ff438005410 con 0x7ff440070160 2024-01-15T18:07:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.464+0000 7ff437400640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff41806bee0 0x7ff41806e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.464+0000 7ff437400640 1 -- 172.21.15.26:0/3978954058 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7ff438087ef0 con 0x7ff440070160 2024-01-15T18:07:17.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.465+0000 7ff43ea00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff41806bee0 0x7ff41806e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.466+0000 7ff43ea00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff41806bee0 0x7ff41806e3a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff430000c00 tx=0x7ff430009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.469+0000 7ff437400640 1 -- 172.21.15.26:0/3978954058 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4401e6af0 con 0x7ff440070160 2024-01-15T18:07:17.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.703+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4183", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4400b41e0 con 0x7ff41806bee0 2024-01-15T18:07:17.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.723+0000 7ff437400640 1 -- 172.21.15.26:0/3978954058 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7ff430001690 con 0x7ff41806bee0 2024-01-15T18:07:17.727 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:17.727 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:17.727 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:07:17.727 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:17.727 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:17.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff41806bee0 msgr2=0x7ff41806e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff41806bee0 0x7ff41806e3a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff430000c00 tx=0x7ff430009040 comp rx=0 tx=0).stop 2024-01-15T18:07:17.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440070160 msgr2=0x7ff4400705e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440070160 0x7ff4400705e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff44006e2e0 tx=0x7ff43800dc00 comp rx=0 tx=0).stop 2024-01-15T18:07:17.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 shutdown_connections 2024-01-15T18:07:17.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff41806bee0 0x7ff41806e3a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff440070160 0x7ff4400705e0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff4401e5270 0x7ff4401e56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 --2- 172.21.15.26:0/3978954058 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff4200069d0 0x7ff44015c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 >> 172.21.15.26:0/3978954058 conn(0x7ff44006a7d0 msgr2=0x7ff44015ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:17.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 shutdown_connections 2024-01-15T18:07:17.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.726+0000 7ff43fe00640 1 -- 172.21.15.26:0/3978954058 wait complete. 2024-01-15T18:07:17.750 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:17.750 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4184 2024-01-15T18:07:17.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.960+0000 7f12ef000640 1 Processor -- start 2024-01-15T18:07:17.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.960+0000 7f12ef000640 1 -- start start 2024-01-15T18:07:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.961+0000 7f12ef000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f12e80cf140 0x7f12e80cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.961+0000 7f12ef000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12e80cfa60 0x7f12e80cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.962+0000 7f12ef000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e80d0430 con 0x7f12e80b07b0 2024-01-15T18:07:17.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.962+0000 7f12ef000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e80b1b70 con 0x7f12e80cfa60 2024-01-15T18:07:17.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.962+0000 7f12ef000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e80b1e50 con 0x7f12e80cf140 2024-01-15T18:07:17.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.962+0000 7f12ed200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12e80cfa60 0x7f12e80cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.962+0000 7f12ed200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12e80cfa60 0x7f12e80cfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46156/0 (socket says 172.21.15.26:46156) 2024-01-15T18:07:17.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.962+0000 7f12ed200640 1 -- 172.21.15.26:0/1246847586 learned_addr learned my addr 172.21.15.26:0/1246847586 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:17.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.962+0000 7f12edc00640 1 --2- 172.21.15.26:0/1246847586 >> v2:172.21.15.124:3300/0 conn(0x7f12e80cf140 0x7f12e80cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.962+0000 7f12ed200640 1 -- 172.21.15.26:0/1246847586 >> v2:172.21.15.124:3300/0 conn(0x7f12e80cf140 msgr2=0x7f12e80cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:17.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.962+0000 7f12ed200640 1 --2- 172.21.15.26:0/1246847586 >> v2:172.21.15.124:3300/0 conn(0x7f12e80cf140 0x7f12e80cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.963+0000 7f12ed200640 1 -- 172.21.15.26:0/1246847586 >> v1:172.21.15.124:6789/0 conn(0x7f12e80b07b0 legacy=0x7f12e80b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:17.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.963+0000 7f12ed200640 1 -- 172.21.15.26:0/1246847586 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e80b2130 con 0x7f12e80cfa60 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.964+0000 7f12ed200640 1 --2- 172.21.15.26:0/1246847586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cfa60 0x7f12e80cfef0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f12e806e080 tx=0x7f12d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bab8382c414fce server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.964+0000 7f12e7e00640 1 -- 172.21.15.26:0/1246847586 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d002ca80 con 0x7f12e80cfa60 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.964+0000 7f12e7e00640 1 -- 172.21.15.26:0/1246847586 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d002cbe0 con 0x7f12e80cfa60 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.964+0000 7f12e7e00640 1 -- 172.21.15.26:0/1246847586 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d00343f0 con 0x7f12e80cfa60 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.965+0000 7f12ef000640 1 -- 172.21.15.26:0/1246847586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cfa60 msgr2=0x7f12e80cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.965+0000 7f12ef000640 1 --2- 172.21.15.26:0/1246847586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cfa60 0x7f12e80cfef0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f12e806e080 tx=0x7f12d002bee0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.965+0000 7f12ef000640 1 -- 172.21.15.26:0/1246847586 shutdown_connections 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.965+0000 7f12ef000640 1 --2- 172.21.15.26:0/1246847586 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cfa60 0x7f12e80cfef0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.965+0000 7f12ef000640 1 --2- 172.21.15.26:0/1246847586 >> v2:172.21.15.124:3300/0 conn(0x7f12e80cf140 0x7f12e80cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.965+0000 7f12ef000640 1 -- 172.21.15.26:0/1246847586 >> 172.21.15.26:0/1246847586 conn(0x7f12e80d0eb0 msgr2=0x7f12e80d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:17.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.965+0000 7f12ef000640 1 -- 172.21.15.26:0/1246847586 shutdown_connections 2024-01-15T18:07:17.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.965+0000 7f12ef000640 1 -- 172.21.15.26:0/1246847586 wait complete. 2024-01-15T18:07:17.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.967+0000 7f12ef000640 1 Processor -- start 2024-01-15T18:07:17.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.967+0000 7f12ef000640 1 -- start start 2024-01-15T18:07:17.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.968+0000 7f12ef000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12e80b07b0 0x7f12e815f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.968+0000 7f12ef000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cf140 0x7f12e81f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.968+0000 7f12ef000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12e80cfa60 0x7f12e81f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.968+0000 7f12ef000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e815f060 con 0x7f12e80cf140 2024-01-15T18:07:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.968+0000 7f12ef000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e81f7000 con 0x7f12e80b07b0 2024-01-15T18:07:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.968+0000 7f12ef000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e815e9a0 con 0x7f12e80cfa60 2024-01-15T18:07:17.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12ed200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cf140 0x7f12e81f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12ed200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cf140 0x7f12e81f6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46168/0 (socket says 172.21.15.26:46168) 2024-01-15T18:07:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12ed200640 1 -- 172.21.15.26:0/144395203 learned_addr learned my addr 172.21.15.26:0/144395203 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12ed200640 1 -- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12e80cfa60 msgr2=0x7f12e81f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12ed200640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12e80cfa60 0x7f12e81f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12ed200640 1 -- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12e80b07b0 msgr2=0x7f12e815f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12edc00640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12e80b07b0 0x7f12e815f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12ed200640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12e80b07b0 0x7f12e815f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:17.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12ed200640 1 -- 172.21.15.26:0/144395203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0009b30 con 0x7f12e80cf140 2024-01-15T18:07:17.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12edc00640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12e80b07b0 0x7f12e815f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:17.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.969+0000 7f12ed200640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cf140 0x7f12e81f6490 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f12d0000f80 tx=0x7f12d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.970+0000 7f12e5600640 1 -- 172.21.15.26:0/144395203 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0009510 con 0x7f12e80cf140 2024-01-15T18:07:17.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.970+0000 7f12e5600640 1 -- 172.21.15.26:0/144395203 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d003e450 con 0x7f12e80cf140 2024-01-15T18:07:17.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.970+0000 7f12e5600640 1 -- 172.21.15.26:0/144395203 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d003e800 con 0x7f12e80cf140 2024-01-15T18:07:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.970+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e815ec80 con 0x7f12e80cf140 2024-01-15T18:07:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.970+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e81475d0 con 0x7f12e80cf140 2024-01-15T18:07:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.971+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e815f1a0 con 0x7f12e80cf140 2024-01-15T18:07:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.972+0000 7f12e5600640 1 -- 172.21.15.26:0/144395203 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f12d002cd50 con 0x7f12e80cf140 2024-01-15T18:07:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.973+0000 7f12e5600640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12c006be90 0x7f12c006e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.973+0000 7f12e5600640 1 -- 172.21.15.26:0/144395203 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f12d00ab120 con 0x7f12e80cf140 2024-01-15T18:07:17.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.974+0000 7f12edc00640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12c006be90 0x7f12c006e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.975+0000 7f12edc00640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12c006be90 0x7f12c006e350 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f12d8006fd0 tx=0x7f12d8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:17.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:17.978+0000 7f12e5600640 1 -- 172.21.15.26:0/144395203 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12e815f1a0 con 0x7f12e80cf140 2024-01-15T18:07:18.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.208+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4184", "target": ["mon-mgr", ""]}) v1 -- 0x7f12e815eb10 con 0x7f12c006be90 2024-01-15T18:07:18.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.231+0000 7f12e5600640 1 -- 172.21.15.26:0/144395203 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f12e815eb10 con 0x7f12c006be90 2024-01-15T18:07:18.234 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:18.234 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:18.235 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:07:18.235 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:18.235 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12c006be90 msgr2=0x7f12c006e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12c006be90 0x7f12c006e350 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f12d8006fd0 tx=0x7f12d8008040 comp rx=0 tx=0).stop 2024-01-15T18:07:18.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cf140 msgr2=0x7f12e81f6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cf140 0x7f12e81f6490 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f12d0000f80 tx=0x7f12d0004a40 comp rx=0 tx=0).stop 2024-01-15T18:07:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 shutdown_connections 2024-01-15T18:07:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12e80cfa60 0x7f12e81f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f12c006be90 0x7f12c006e350 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12e80cf140 0x7f12e81f6490 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 --2- 172.21.15.26:0/144395203 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12e80b07b0 0x7f12e815f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.233+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 >> 172.21.15.26:0/144395203 conn(0x7f12e80d0eb0 msgr2=0x7f12e80afec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.234+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 shutdown_connections 2024-01-15T18:07:18.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.234+0000 7f12ef000640 1 -- 172.21.15.26:0/144395203 wait complete. 2024-01-15T18:07:18.256 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:18.257 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4185 2024-01-15T18:07:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.468+0000 7f4275000640 1 Processor -- start 2024-01-15T18:07:18.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.469+0000 7f4275000640 1 -- start start 2024-01-15T18:07:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.470+0000 7f4275000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42700b35a0 0x7f42700cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.470+0000 7f4275000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42700ce440 con 0x7f427013cf10 2024-01-15T18:07:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.470+0000 7f4275000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42700ce720 con 0x7f427013fcb0 2024-01-15T18:07:18.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.470+0000 7f4275000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42700cea00 con 0x7f42700b35a0 2024-01-15T18:07:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.471+0000 7f426f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f427013cf10 0x7f427013f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56974/0 (socket says 172.21.15.26:56974) 2024-01-15T18:07:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.471+0000 7f426f400640 1 -- 172.21.15.26:0/2688078948 learned_addr learned my addr 172.21.15.26:0/2688078948 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.471+0000 7f426fe00640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42700b35a0 0x7f42700cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.471+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677772942 0 0) 0x7f42700ce440 con 0x7f427013cf10 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.471+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4250003520 con 0x7f427013cf10 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2082689664 0 0) 0x7f4250003520 con 0x7f427013cf10 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42700b35a0 msgr2=0x7f42700cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42700b35a0 0x7f42700cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 >> v1:172.21.15.124:6789/0 conn(0x7f427013fcb0 legacy=0x7f42700cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42700b1d90 con 0x7f427013cf10 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f4260001670 con 0x7f427013cf10 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 >> v1:172.21.15.26:6789/0 conn(0x7f427013cf10 legacy=0x7f427013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42500039a0 0x7f4250003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500047c0 0x7f4250004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42700b35a0 0x7f4250005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42700cea00 con 0x7f42500039a0 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425000ba30 con 0x7f42500047c0 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425000de00 con 0x7f42700b35a0 2024-01-15T18:07:18.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426f400640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42500039a0 0x7f4250003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.472+0000 7f426ea00640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500047c0 0x7f4250004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.473+0000 7f426f400640 1 -- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42700b35a0 msgr2=0x7f4250005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.473+0000 7f426f400640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f42700b35a0 0x7f4250005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.473+0000 7f426f400640 1 -- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500047c0 msgr2=0x7f4250004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.473+0000 7f426f400640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500047c0 0x7f4250004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.473+0000 7f426f400640 1 -- 172.21.15.26:0/2688078948 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4250003520 con 0x7f42500039a0 2024-01-15T18:07:18.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.474+0000 7f426f400640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42500039a0 0x7f4250003dc0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f426000c210 tx=0x7f42500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc31a77f6f140b4c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.474+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260005ad0 con 0x7f42500039a0 2024-01-15T18:07:18.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.474+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4260005c30 con 0x7f42500039a0 2024-01-15T18:07:18.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.474+0000 7f426e000640 1 -- 172.21.15.26:0/2688078948 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426002cab0 con 0x7f42500039a0 2024-01-15T18:07:18.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.475+0000 7f4275000640 1 -- 172.21.15.26:0/2688078948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42500039a0 msgr2=0x7f4250003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.475+0000 7f4275000640 1 --2- 172.21.15.26:0/2688078948 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f42500039a0 0x7f4250003dc0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f426000c210 tx=0x7f42500057f0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.475+0000 7f426fe00640 1 -- 172.21.15.26:0/2688078948 reap_dead start 2024-01-15T18:07:18.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.475+0000 7f4275000640 1 -- 172.21.15.26:0/2688078948 shutdown_connections 2024-01-15T18:07:18.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.475+0000 7f4275000640 1 -- 172.21.15.26:0/2688078948 >> 172.21.15.26:0/2688078948 conn(0x7f427006a7d0 msgr2=0x7f427006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.475+0000 7f4275000640 1 -- 172.21.15.26:0/2688078948 shutdown_connections 2024-01-15T18:07:18.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.475+0000 7f4275000640 1 -- 172.21.15.26:0/2688078948 wait complete. 2024-01-15T18:07:18.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.477+0000 7f4275000640 1 Processor -- start 2024-01-15T18:07:18.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.477+0000 7f4275000640 1 -- start start 2024-01-15T18:07:18.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f4275000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500039a0 0x7f427015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f4275000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4270137ec0 0x7f427013e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f4275000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f427013ed80 0x7f427013a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f4275000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427013f200 con 0x7f4270137ec0 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f4275000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427015be00 con 0x7f42500039a0 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f4275000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f427015c0e0 con 0x7f427013ed80 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f426ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4270137ec0 0x7f427013e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f426ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4270137ec0 0x7f427013e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46192/0 (socket says 172.21.15.26:46192) 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f426ea00640 1 -- 172.21.15.26:0/735297154 learned_addr learned my addr 172.21.15.26:0/735297154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f426f400640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500039a0 0x7f427015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f426fe00640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f427013ed80 0x7f427013a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f426ea00640 1 -- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f427013ed80 msgr2=0x7f427013a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.478+0000 7f426ea00640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f427013ed80 0x7f427013a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.479+0000 7f426ea00640 1 -- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500039a0 msgr2=0x7f427015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.479+0000 7f426ea00640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500039a0 0x7f427015cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.479+0000 7f426ea00640 1 -- 172.21.15.26:0/735297154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427015c3c0 con 0x7f4270137ec0 2024-01-15T18:07:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.479+0000 7f426f400640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500039a0 0x7f427015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.479+0000 7f426ea00640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4270137ec0 0x7f427013e840 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f425800e410 tx=0x7f425800e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.479+0000 7f4267400640 1 -- 172.21.15.26:0/735297154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42580052e0 con 0x7f4270137ec0 2024-01-15T18:07:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.479+0000 7f4267400640 1 -- 172.21.15.26:0/735297154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4258005440 con 0x7f4270137ec0 2024-01-15T18:07:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.479+0000 7f4267400640 1 -- 172.21.15.26:0/735297154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258003810 con 0x7f4270137ec0 2024-01-15T18:07:18.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.479+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427015c6a0 con 0x7f4270137ec0 2024-01-15T18:07:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.480+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427015d3e0 con 0x7f4270137ec0 2024-01-15T18:07:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.481+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42700afda0 con 0x7f4270137ec0 2024-01-15T18:07:18.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.482+0000 7f4267400640 1 -- 172.21.15.26:0/735297154 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7f4258003970 con 0x7f4270137ec0 2024-01-15T18:07:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.482+0000 7f4267400640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f424806be90 0x7f424806e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.482+0000 7f4267400640 1 -- 172.21.15.26:0/735297154 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f4258087f00 con 0x7f4270137ec0 2024-01-15T18:07:18.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.483+0000 7f426f400640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f424806be90 0x7f424806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.484+0000 7f426f400640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f424806be90 0x7f424806e350 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4260034000 tx=0x7f4260009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.486+0000 7f4267400640 1 -- 172.21.15.26:0/735297154 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4258054670 con 0x7f4270137ec0 2024-01-15T18:07:18.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.711+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4185", "target": ["mon-mgr", ""]}) v1 -- 0x7f4270067450 con 0x7f424806be90 2024-01-15T18:07:18.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.732+0000 7f4267400640 1 -- 172.21.15.26:0/735297154 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f4270067450 con 0x7f424806be90 2024-01-15T18:07:18.736 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:18.736 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:18.736 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:18.736 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:18.736 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:18.736 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:18.736 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:18.736 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:18.737 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:18.737 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:18.737 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:18.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:18.737 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:18.737 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:18.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.734+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f424806be90 msgr2=0x7f424806e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.734+0000 7f4275000640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f424806be90 0x7f424806e350 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4260034000 tx=0x7f4260009030 comp rx=0 tx=0).stop 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.734+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4270137ec0 msgr2=0x7f427013e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.734+0000 7f4275000640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4270137ec0 0x7f427013e840 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f425800e410 tx=0x7f425800e8d0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.735+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 shutdown_connections 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.735+0000 7f4275000640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f427013ed80 0x7f427013a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.735+0000 7f4275000640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f424806be90 0x7f424806e350 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.735+0000 7f4275000640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4270137ec0 0x7f427013e840 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.735+0000 7f4275000640 1 --2- 172.21.15.26:0/735297154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f42500039a0 0x7f427015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.735+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 >> 172.21.15.26:0/735297154 conn(0x7f427006a7d0 msgr2=0x7f42700b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.735+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 shutdown_connections 2024-01-15T18:07:18.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.735+0000 7f4275000640 1 -- 172.21.15.26:0/735297154 wait complete. 2024-01-15T18:07:18.756 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:18.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4186 2024-01-15T18:07:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.972+0000 7fbd40e00640 1 Processor -- start 2024-01-15T18:07:18.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.972+0000 7fbd40e00640 1 -- start start 2024-01-15T18:07:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.973+0000 7fbd40e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c15e3b0 0x7fbd3c164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.973+0000 7fbd40e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3c0b1ee0 con 0x7fbd3c143ae0 2024-01-15T18:07:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.973+0000 7fbd40e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3c0b2780 con 0x7fbd3c0cf9a0 2024-01-15T18:07:18.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd40e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c15e990 con 0x7fbd3c15e3b0 2024-01-15T18:07:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd39e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbd3c143ae0 0x7fbd3c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56984/0 (socket says 172.21.15.26:56984) 2024-01-15T18:07:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd39e00640 1 -- 172.21.15.26:0/3822269383 learned_addr learned my addr 172.21.15.26:0/3822269383 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467814666 0 0) 0x7fbd3c0b1ee0 con 0x7fbd3c143ae0 2024-01-15T18:07:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd3a800640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c15e3b0 0x7fbd3c164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd1c005730 con 0x7fbd3c143ae0 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757193534 0 0) 0x7fbd1c005730 con 0x7fbd3c143ae0 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c15e3b0 msgr2=0x7fbd3c164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd38a00640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c15e3b0 0x7fbd3c164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.974+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 >> v1:172.21.15.124:6789/0 conn(0x7fbd3c0cf9a0 legacy=0x7fbd3c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c0b2130 con 0x7fbd3c143ae0 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fbd2c001670 con 0x7fbd3c143ae0 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 >> v1:172.21.15.26:6789/0 conn(0x7fbd3c143ae0 legacy=0x7fbd3c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd38a00640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd1c005bb0 0x7fbd1c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd38a00640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd1c0069d0 0x7fbd1c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd38a00640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c007390 0x7fbd1c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c15e990 con 0x7fbd1c005bb0 2024-01-15T18:07:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c00e290 con 0x7fbd1c007390 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c00e450 con 0x7fbd1c0069d0 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd3a800640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c15e3b0 0x7fbd3c164de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd39e00640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd1c005bb0 0x7fbd1c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd39400640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd1c0069d0 0x7fbd1c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd39e00640 1 -- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd1c0069d0 msgr2=0x7fbd1c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd39e00640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd1c0069d0 0x7fbd1c006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd39e00640 1 -- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c007390 msgr2=0x7fbd1c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd3a800640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c007390 0x7fbd1c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.975+0000 7fbd39e00640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c007390 0x7fbd1c00db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.976+0000 7fbd39e00640 1 -- 172.21.15.26:0/3822269383 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c005730 con 0x7fbd1c005bb0 2024-01-15T18:07:18.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.977+0000 7fbd39e00640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd1c005bb0 0x7fbd1c005fd0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00c0e0 tx=0x7fbd1c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bed4a6c5faf63f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.977+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c005ad0 con 0x7fbd1c005bb0 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.977+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2c005c30 con 0x7fbd1c005bb0 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.977+0000 7fbd38a00640 1 -- 172.21.15.26:0/3822269383 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c02c7d0 con 0x7fbd1c005bb0 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.977+0000 7fbd40e00640 1 -- 172.21.15.26:0/3822269383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd1c005bb0 msgr2=0x7fbd1c005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.977+0000 7fbd40e00640 1 --2- 172.21.15.26:0/3822269383 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd1c005bb0 0x7fbd1c005fd0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c00c0e0 tx=0x7fbd1c007a00 comp rx=0 tx=0).stop 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.978+0000 7fbd3a800640 1 -- 172.21.15.26:0/3822269383 reap_dead start 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.978+0000 7fbd40e00640 1 -- 172.21.15.26:0/3822269383 shutdown_connections 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.978+0000 7fbd40e00640 1 -- 172.21.15.26:0/3822269383 >> 172.21.15.26:0/3822269383 conn(0x7fbd3c06a800 msgr2=0x7fbd3c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.978+0000 7fbd40e00640 1 -- 172.21.15.26:0/3822269383 shutdown_connections 2024-01-15T18:07:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.978+0000 7fbd40e00640 1 -- 172.21.15.26:0/3822269383 wait complete. 2024-01-15T18:07:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.979+0000 7fbd40e00640 1 Processor -- start 2024-01-15T18:07:18.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.980+0000 7fbd40e00640 1 -- start start 2024-01-15T18:07:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd40e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c005bb0 0x7fbd3c13da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd40e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c0cc610 0x7fbd3c0af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd40e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c15c6d0 0x7fbd3c15cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd40e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c15d090 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd40e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0ccfa0 con 0x7fbd1c005bb0 2024-01-15T18:07:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd40e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c0cd7b0 con 0x7fbd3c0cc610 2024-01-15T18:07:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd3a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c15c6d0 0x7fbd3c15cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd3a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c15c6d0 0x7fbd3c15cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46210/0 (socket says 172.21.15.26:46210) 2024-01-15T18:07:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd3a800640 1 -- 172.21.15.26:0/1220287633 learned_addr learned my addr 172.21.15.26:0/1220287633 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd39400640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c0cc610 0x7fbd3c0af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd39e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c005bb0 0x7fbd3c13da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd3a800640 1 -- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c0cc610 msgr2=0x7fbd3c0af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd3a800640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c0cc610 0x7fbd3c0af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd3a800640 1 -- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c005bb0 msgr2=0x7fbd3c13da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd3a800640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c005bb0 0x7fbd3c13da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd3a800640 1 -- 172.21.15.26:0/1220287633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c0cd1f0 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd39400640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c0cc610 0x7fbd3c0af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.981+0000 7fbd39e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c005bb0 0x7fbd3c13da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.982+0000 7fbd3a800640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c15c6d0 0x7fbd3c15cb50 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0cc3c0 tx=0x7fbd34004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.982+0000 7fbd32000640 1 -- 172.21.15.26:0/1220287633 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34007cb0 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.982+0000 7fbd32000640 1 -- 172.21.15.26:0/1220287633 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd34007e50 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.982+0000 7fbd32000640 1 -- 172.21.15.26:0/1220287633 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd34014640 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.982+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3c0cd4d0 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.982+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3c0afaa0 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.983+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c0ad2e0 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.984+0000 7fbd32000640 1 -- 172.21.15.26:0/1220287633 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 27) v1 ==== 90610+0+0 (secure 0 0 0) 0x7fbd340147a0 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.985+0000 7fbd32000640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd1406be90 0x7fbd1406e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.985+0000 7fbd32000640 1 -- 172.21.15.26:0/1220287633 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fbd34088210 con 0x7fbd3c15c6d0 2024-01-15T18:07:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.986+0000 7fbd39e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd1406be90 0x7fbd1406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.987+0000 7fbd39e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd1406be90 0x7fbd1406e350 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c034000 tx=0x7fbd2c037040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:18.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:18.989+0000 7fbd32000640 1 -- 172.21.15.26:0/1220287633 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd3c0ad2e0 con 0x7fbd3c15c6d0 2024-01-15T18:07:19.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.221+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4186", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd3c0ad2e0 con 0x7fbd1406be90 2024-01-15T18:07:19.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.245+0000 7fbd32000640 1 -- 172.21.15.26:0/1220287633 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7fbd3c0ad2e0 con 0x7fbd1406be90 2024-01-15T18:07:19.248 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:19.248 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:19.249 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:19.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd1406be90 msgr2=0x7fbd1406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd1406be90 0x7fbd1406e350 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c034000 tx=0x7fbd2c037040 comp rx=0 tx=0).stop 2024-01-15T18:07:19.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c15c6d0 msgr2=0x7fbd3c15cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c15c6d0 0x7fbd3c15cb50 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0cc3c0 tx=0x7fbd34004840 comp rx=0 tx=0).stop 2024-01-15T18:07:19.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 shutdown_connections 2024-01-15T18:07:19.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd3c0cc610 0x7fbd3c0af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd1406be90 0x7fbd1406e350 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd3c15c6d0 0x7fbd3c15cb50 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 --2- 172.21.15.26:0/1220287633 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd1c005bb0 0x7fbd3c13da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 >> 172.21.15.26:0/1220287633 conn(0x7fbd3c06a800 msgr2=0x7fbd3c15b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:19.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.247+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 shutdown_connections 2024-01-15T18:07:19.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.248+0000 7fbd40e00640 1 -- 172.21.15.26:0/1220287633 wait complete. 2024-01-15T18:07:19.270 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:19.270 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4187 2024-01-15T18:07:19.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.498+0000 7f4634a00640 1 Processor -- start 2024-01-15T18:07:19.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.498+0000 7f4634a00640 1 -- start start 2024-01-15T18:07:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.499+0000 7f4634a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13df80 0x7f462c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.499+0000 7f4634a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f462c0cf9a0 0x7f462c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.499+0000 7f4634a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462c06c470 con 0x7f462c13e360 2024-01-15T18:07:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.500+0000 7f4634a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f462c06c750 con 0x7f462c0cf9a0 2024-01-15T18:07:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.500+0000 7f4634a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c06ca30 con 0x7f462c13df80 2024-01-15T18:07:19.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.500+0000 7f4632a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f462c0cf9a0 0x7f462c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.500+0000 7f4632a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f462c0cf9a0 0x7f462c0cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46224/0 (socket says 172.21.15.26:46224) 2024-01-15T18:07:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.500+0000 7f4632a00640 1 -- 172.21.15.26:0/3010331713 learned_addr learned my addr 172.21.15.26:0/3010331713 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.500+0000 7f4632000640 1 --2- 172.21.15.26:0/3010331713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13df80 0x7f462c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.501+0000 7f4632a00640 1 -- 172.21.15.26:0/3010331713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13df80 msgr2=0x7f462c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.501+0000 7f4632a00640 1 --2- 172.21.15.26:0/3010331713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13df80 0x7f462c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.501+0000 7f4632a00640 1 -- 172.21.15.26:0/3010331713 >> v1:172.21.15.124:6789/0 conn(0x7f462c13e360 legacy=0x7f462c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:19.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.501+0000 7f4632a00640 1 -- 172.21.15.26:0/3010331713 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c06cd10 con 0x7f462c0cf9a0 2024-01-15T18:07:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.502+0000 7f4632a00640 1 --2- 172.21.15.26:0/3010331713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c0cf9a0 0x7f462c0cd840 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f4624009870 tx=0x7f462402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b70a25271d4860 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.502+0000 7f4630c00640 1 -- 172.21.15.26:0/3010331713 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624031640 con 0x7f462c0cf9a0 2024-01-15T18:07:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.502+0000 7f4630c00640 1 -- 172.21.15.26:0/3010331713 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46240317a0 con 0x7f462c0cf9a0 2024-01-15T18:07:19.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.502+0000 7f4630c00640 1 -- 172.21.15.26:0/3010331713 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624031b30 con 0x7f462c0cf9a0 2024-01-15T18:07:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.503+0000 7f4634a00640 1 -- 172.21.15.26:0/3010331713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c0cf9a0 msgr2=0x7f462c0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.503+0000 7f4634a00640 1 --2- 172.21.15.26:0/3010331713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c0cf9a0 0x7f462c0cd840 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f4624009870 tx=0x7f462402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.503+0000 7f4634a00640 1 -- 172.21.15.26:0/3010331713 shutdown_connections 2024-01-15T18:07:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.503+0000 7f4634a00640 1 --2- 172.21.15.26:0/3010331713 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c0cf9a0 0x7f462c0cd840 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.503+0000 7f4634a00640 1 --2- 172.21.15.26:0/3010331713 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13df80 0x7f462c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.503+0000 7f4634a00640 1 -- 172.21.15.26:0/3010331713 >> 172.21.15.26:0/3010331713 conn(0x7f462c06a800 msgr2=0x7f462c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.503+0000 7f4634a00640 1 -- 172.21.15.26:0/3010331713 shutdown_connections 2024-01-15T18:07:19.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.503+0000 7f4634a00640 1 -- 172.21.15.26:0/3010331713 wait complete. 2024-01-15T18:07:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.505+0000 7f4634a00640 1 Processor -- start 2024-01-15T18:07:19.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.505+0000 7f4634a00640 1 -- start start 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4634a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f462c0cf9a0 0x7f462c13cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4634a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c13df80 0x7f462c157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4634a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13e360 0x7f462c1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4634a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c13d450 con 0x7f462c13df80 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4634a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c071890 con 0x7f462c0cf9a0 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4634a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c0715b0 con 0x7f462c13e360 2024-01-15T18:07:19.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4631600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c13df80 0x7f462c157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4631600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c13df80 0x7f462c157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46238/0 (socket says 172.21.15.26:46238) 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4631600640 1 -- 172.21.15.26:0/1869274400 learned_addr learned my addr 172.21.15.26:0/1869274400 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.506+0000 7f4632a00640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13e360 0x7f462c1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f4632000640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f462c0cf9a0 0x7f462c13cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f4631600640 1 -- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13e360 msgr2=0x7f462c1580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f4631600640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13e360 0x7f462c1580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f4631600640 1 -- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f462c0cf9a0 msgr2=0x7f462c13cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f4631600640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f462c0cf9a0 0x7f462c13cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f4631600640 1 -- 172.21.15.26:0/1869274400 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240094d0 con 0x7f462c13df80 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f4632000640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f462c0cf9a0 0x7f462c13cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f4631600640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c13df80 0x7f462c157b80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f461400c7c0 tx=0x7f461400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f462a000640 1 -- 172.21.15.26:0/1869274400 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461400aa50 con 0x7f462c13df80 2024-01-15T18:07:19.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f462a000640 1 -- 172.21.15.26:0/1869274400 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461400abb0 con 0x7f462c13df80 2024-01-15T18:07:19.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c0aca00 con 0x7f462c13df80 2024-01-15T18:07:19.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.507+0000 7f462a000640 1 -- 172.21.15.26:0/1869274400 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614012420 con 0x7f462c13df80 2024-01-15T18:07:19.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.508+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c13d860 con 0x7f462c13df80 2024-01-15T18:07:19.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.509+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c0acf20 con 0x7f462c13df80 2024-01-15T18:07:19.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.510+0000 7f462a000640 1 -- 172.21.15.26:0/1869274400 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f4614012580 con 0x7f462c13df80 2024-01-15T18:07:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.511+0000 7f462a000640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f460406bf20 0x7f460406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:19.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.511+0000 7f462a000640 1 -- 172.21.15.26:0/1869274400 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f4614088550 con 0x7f462c13df80 2024-01-15T18:07:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.511+0000 7f4632000640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f460406bf20 0x7f460406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:19.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.512+0000 7f4632000640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f460406bf20 0x7f460406e3e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f461c006fd0 tx=0x7f461c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.515+0000 7f462a000640 1 -- 172.21.15.26:0/1869274400 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f462c0acf20 con 0x7f462c13df80 2024-01-15T18:07:19.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.747+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4187", "target": ["mon-mgr", ""]}) v1 -- 0x7f462c15f9c0 con 0x7f460406bf20 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.769+0000 7f462a000640 1 -- 172.21.15.26:0/1869274400 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f461c0095d0 con 0x7f460406bf20 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:19.773 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:19.774 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:19.774 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:19.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f460406bf20 msgr2=0x7f460406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f460406bf20 0x7f460406e3e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f461c006fd0 tx=0x7f461c008040 comp rx=0 tx=0).stop 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c13df80 msgr2=0x7f462c157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c13df80 0x7f462c157b80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f461400c7c0 tx=0x7f461400cc80 comp rx=0 tx=0).stop 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 shutdown_connections 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f462c13e360 0x7f462c1580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f460406bf20 0x7f460406e3e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f462c13df80 0x7f462c157b80 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 --2- 172.21.15.26:0/1869274400 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f462c0cf9a0 0x7f462c13cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 >> 172.21.15.26:0/1869274400 conn(0x7f462c06a800 msgr2=0x7f462c0ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 shutdown_connections 2024-01-15T18:07:19.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:19.772+0000 7f4634a00640 1 -- 172.21.15.26:0/1869274400 wait complete. 2024-01-15T18:07:19.794 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:19.794 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4188 2024-01-15T18:07:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.012+0000 7f9ff8e00640 1 Processor -- start 2024-01-15T18:07:20.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.012+0000 7f9ff8e00640 1 -- start start 2024-01-15T18:07:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.014+0000 7f9ff8e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 0x7f9ff40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.014+0000 7f9ff8e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ff415f9c0 0x7f9ff40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.014+0000 7f9ff8e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff415c5f0 con 0x7f9ff413df80 2024-01-15T18:07:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.014+0000 7f9ff8e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff406c470 con 0x7f9ff415f9c0 2024-01-15T18:07:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.014+0000 7f9ff8e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff406c750 con 0x7f9ff413e360 2024-01-15T18:07:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.014+0000 7f9ff2800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ff415f9c0 0x7f9ff40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.014+0000 7f9ff2800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ff415f9c0 0x7f9ff40cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46248/0 (socket says 172.21.15.26:46248) 2024-01-15T18:07:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.014+0000 7f9ff2800640 1 -- 172.21.15.26:0/4082504460 learned_addr learned my addr 172.21.15.26:0/4082504460 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:20.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.014+0000 7f9ff1400640 1 --2- 172.21.15.26:0/4082504460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 0x7f9ff40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.015+0000 7f9ff2800640 1 -- 172.21.15.26:0/4082504460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 msgr2=0x7f9ff40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.015+0000 7f9ff2800640 1 --2- 172.21.15.26:0/4082504460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 0x7f9ff40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.015+0000 7f9ff2800640 1 -- 172.21.15.26:0/4082504460 >> v1:172.21.15.26:6789/0 conn(0x7f9ff413df80 legacy=0x7f9ff415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.015+0000 7f9ff2800640 1 -- 172.21.15.26:0/4082504460 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff406ca30 con 0x7f9ff415f9c0 2024-01-15T18:07:20.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.015+0000 7f9ff1400640 1 --2- 172.21.15.26:0/4082504460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 0x7f9ff40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.015+0000 7f9ff0a00640 1 -- 172.21.15.26:0/4082504460 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662050012 0 0) 0x7f9ff415c5f0 con 0x7f9ff413df80 2024-01-15T18:07:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.016+0000 7f9ff2800640 1 --2- 172.21.15.26:0/4082504460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff415f9c0 0x7f9ff40cf7d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009870 tx=0x7f9fec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c44a5624c87f0f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.017+0000 7f9ff0a00640 1 -- 172.21.15.26:0/4082504460 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec030840 con 0x7f9ff415f9c0 2024-01-15T18:07:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.017+0000 7f9ff0a00640 1 -- 172.21.15.26:0/4082504460 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec0309a0 con 0x7f9ff415f9c0 2024-01-15T18:07:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.017+0000 7f9ff0a00640 1 -- 172.21.15.26:0/4082504460 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec030d30 con 0x7f9ff415f9c0 2024-01-15T18:07:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.017+0000 7f9ff8e00640 1 -- 172.21.15.26:0/4082504460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff415f9c0 msgr2=0x7f9ff40cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.017+0000 7f9ff8e00640 1 --2- 172.21.15.26:0/4082504460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff415f9c0 0x7f9ff40cf7d0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009870 tx=0x7f9fec02b980 comp rx=0 tx=0).stop 2024-01-15T18:07:20.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.018+0000 7f9ff8e00640 1 -- 172.21.15.26:0/4082504460 shutdown_connections 2024-01-15T18:07:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.018+0000 7f9ff8e00640 1 --2- 172.21.15.26:0/4082504460 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff415f9c0 0x7f9ff40cf7d0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.018+0000 7f9ff8e00640 1 --2- 172.21.15.26:0/4082504460 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 0x7f9ff40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.018+0000 7f9ff8e00640 1 -- 172.21.15.26:0/4082504460 >> 172.21.15.26:0/4082504460 conn(0x7f9ff406a800 msgr2=0x7f9ff406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.018+0000 7f9ff8e00640 1 -- 172.21.15.26:0/4082504460 shutdown_connections 2024-01-15T18:07:20.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.018+0000 7f9ff8e00640 1 -- 172.21.15.26:0/4082504460 wait complete. 2024-01-15T18:07:20.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.019+0000 7f9ff8e00640 1 Processor -- start 2024-01-15T18:07:20.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.020+0000 7f9ff8e00640 1 -- start start 2024-01-15T18:07:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff413df80 0x7f9ff40ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff8e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 0x7f9ff40b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff8e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ff415f9c0 0x7f9ff413d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff8e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40b42f0 con 0x7f9ff413df80 2024-01-15T18:07:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff8e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff413dd80 con 0x7f9ff415f9c0 2024-01-15T18:07:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff8e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff40ab9c0 con 0x7f9ff413e360 2024-01-15T18:07:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff413df80 0x7f9ff40ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff413df80 0x7f9ff40ab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46262/0 (socket says 172.21.15.26:46262) 2024-01-15T18:07:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff1e00640 1 -- 172.21.15.26:0/1227765556 learned_addr learned my addr 172.21.15.26:0/1227765556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff1e00640 1 -- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 msgr2=0x7f9ff40b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff1e00640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 0x7f9ff40b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff2800640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ff415f9c0 0x7f9ff413d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.021+0000 7f9ff1e00640 1 -- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ff415f9c0 msgr2=0x7f9ff413d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.022+0000 7f9ff1e00640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ff415f9c0 0x7f9ff413d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.022+0000 7f9ff1e00640 1 -- 172.21.15.26:0/1227765556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec0094d0 con 0x7f9ff413df80 2024-01-15T18:07:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.022+0000 7f9ff1e00640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff413df80 0x7f9ff40ab390 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c7f0 tx=0x7f9fe400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.022+0000 7f9fea000640 1 -- 172.21.15.26:0/1227765556 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4003da0 con 0x7f9ff413df80 2024-01-15T18:07:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.022+0000 7f9fea000640 1 -- 172.21.15.26:0/1227765556 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe4005070 con 0x7f9ff413df80 2024-01-15T18:07:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.022+0000 7f9fea000640 1 -- 172.21.15.26:0/1227765556 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4005420 con 0x7f9ff413df80 2024-01-15T18:07:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.022+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff40abca0 con 0x7f9ff413df80 2024-01-15T18:07:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.022+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff40a9720 con 0x7f9ff413df80 2024-01-15T18:07:20.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.024+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff40ac1c0 con 0x7f9ff413df80 2024-01-15T18:07:20.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.025+0000 7f9fea000640 1 -- 172.21.15.26:0/1227765556 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f9fe4005580 con 0x7f9ff413df80 2024-01-15T18:07:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.026+0000 7f9fea000640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9fcc06bf70 0x7f9fcc06e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.026+0000 7f9fea000640 1 -- 172.21.15.26:0/1227765556 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f9fe40881d0 con 0x7f9ff413df80 2024-01-15T18:07:20.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.027+0000 7f9ff1400640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9fcc06bf70 0x7f9fcc06e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.028+0000 7f9ff1400640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9fcc06bf70 0x7f9fcc06e430 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006e80 tx=0x7f9fdc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:20.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.031+0000 7f9fea000640 1 -- 172.21.15.26:0/1227765556 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fe4054810 con 0x7f9ff413df80 2024-01-15T18:07:20.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.265+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4188", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ff406cff0 con 0x7f9fcc06bf70 2024-01-15T18:07:20.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.288+0000 7f9fea000640 1 -- 172.21.15.26:0/1227765556 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f9fdc009460 con 0x7f9fcc06bf70 2024-01-15T18:07:20.291 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:20.291 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:20.292 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9fcc06bf70 msgr2=0x7f9fcc06e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9fcc06bf70 0x7f9fcc06e430 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006e80 tx=0x7f9fdc008040 comp rx=0 tx=0).stop 2024-01-15T18:07:20.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff413df80 msgr2=0x7f9ff40ab390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff413df80 0x7f9ff40ab390 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c7f0 tx=0x7f9fe400ccb0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 shutdown_connections 2024-01-15T18:07:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ff413e360 0x7f9ff40b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9fcc06bf70 0x7f9fcc06e430 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ff415f9c0 0x7f9ff413d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 --2- 172.21.15.26:0/1227765556 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ff413df80 0x7f9ff40ab390 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.290+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 >> 172.21.15.26:0/1227765556 conn(0x7f9ff406a800 msgr2=0x7f9ff406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.291+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 shutdown_connections 2024-01-15T18:07:20.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.291+0000 7f9ff8e00640 1 -- 172.21.15.26:0/1227765556 wait complete. 2024-01-15T18:07:20.313 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:20.314 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone4189 2024-01-15T18:07:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.533+0000 7fecf2a00640 1 Processor -- start 2024-01-15T18:07:20.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.534+0000 7fecf2a00640 1 -- start start 2024-01-15T18:07:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.535+0000 7fecf2a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fecec0cf9a0 0x7fecec0cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.535+0000 7fecf2a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec0d0320 con 0x7fecec15d4b0 2024-01-15T18:07:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.535+0000 7fecf2a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecec0d1c60 con 0x7fecec0b31d0 2024-01-15T18:07:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.535+0000 7fecf2a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecec0b2730 con 0x7fecec0cf9a0 2024-01-15T18:07:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.536+0000 7fecf1600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fecec15d4b0 0x7fecec0d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57008/0 (socket says 172.21.15.26:57008) 2024-01-15T18:07:20.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.536+0000 7fecf1600640 1 -- 172.21.15.26:0/3192298995 learned_addr learned my addr 172.21.15.26:0/3192298995 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:20.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.536+0000 7fecf0c00640 1 --2- 172.21.15.26:0/3192298995 >> v2:172.21.15.124:3300/0 conn(0x7fecec0cf9a0 0x7fecec0cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.536+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509399195 0 0) 0x7fecec0d0320 con 0x7fecec15d4b0 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.536+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feccc003520 con 0x7fecec15d4b0 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.536+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 834615410 0 0) 0x7feccc003520 con 0x7fecec15d4b0 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.536+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 >> v2:172.21.15.124:3300/0 conn(0x7fecec0cf9a0 msgr2=0x7fecec0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.536+0000 7fecebe00640 1 --2- 172.21.15.26:0/3192298995 >> v2:172.21.15.124:3300/0 conn(0x7fecec0cf9a0 0x7fecec0cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.536+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 >> v1:172.21.15.124:6789/0 conn(0x7fecec0b31d0 legacy=0x7fecec0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec0d1ec0 con 0x7fecec15d4b0 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fecdc0019f0 con 0x7fecec15d4b0 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 >> v1:172.21.15.26:6789/0 conn(0x7fecec15d4b0 legacy=0x7fecec0d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecebe00640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc003970 0x7feccc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecebe00640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feccc0047c0 0x7feccc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecebe00640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc005180 0x7feccc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b2730 con 0x7feccc003970 2024-01-15T18:07:20.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc00c020 con 0x7feccc005180 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc00c1e0 con 0x7feccc0047c0 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecf1600640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc003970 0x7feccc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.537+0000 7fecf0c00640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feccc0047c0 0x7feccc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.538+0000 7fecf2000640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc005180 0x7feccc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.538+0000 7fecf1600640 1 -- 172.21.15.26:0/3192298995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feccc0047c0 msgr2=0x7feccc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.538+0000 7fecf1600640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feccc0047c0 0x7feccc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.538+0000 7fecf1600640 1 -- 172.21.15.26:0/3192298995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc005180 msgr2=0x7feccc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.538+0000 7fecf1600640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc005180 0x7feccc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.538+0000 7fecf1600640 1 -- 172.21.15.26:0/3192298995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc003520 con 0x7feccc003970 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.538+0000 7fecf2000640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc005180 0x7feccc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:20.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.538+0000 7fecf2000640 1 -- 172.21.15.26:0/3192298995 reap_dead start 2024-01-15T18:07:20.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.539+0000 7fecf1600640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc003970 0x7feccc003d90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00a830 tx=0x7feccc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e433d1267c9c1f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:20.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.539+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0072d0 con 0x7feccc003970 2024-01-15T18:07:20.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.539+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecdc02f070 con 0x7feccc003970 2024-01-15T18:07:20.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.539+0000 7fecebe00640 1 -- 172.21.15.26:0/3192298995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc007690 con 0x7feccc003970 2024-01-15T18:07:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.540+0000 7fecf2a00640 1 -- 172.21.15.26:0/3192298995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc003970 msgr2=0x7feccc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.540+0000 7fecf2a00640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc003970 0x7feccc003d90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00a830 tx=0x7feccc0057f0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.540+0000 7fecf2a00640 1 -- 172.21.15.26:0/3192298995 shutdown_connections 2024-01-15T18:07:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.540+0000 7fecf2a00640 1 --2- 172.21.15.26:0/3192298995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feccc003970 0x7feccc003d90 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.540+0000 7fecf2a00640 1 -- 172.21.15.26:0/3192298995 >> 172.21.15.26:0/3192298995 conn(0x7fecec0d3000 msgr2=0x7fecec0d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.540+0000 7fecf2a00640 1 -- 172.21.15.26:0/3192298995 shutdown_connections 2024-01-15T18:07:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.540+0000 7fecf2a00640 1 -- 172.21.15.26:0/3192298995 wait complete. 2024-01-15T18:07:20.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.542+0000 7fecf2a00640 1 Processor -- start 2024-01-15T18:07:20.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.542+0000 7fecf2a00640 1 -- start start 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.543+0000 7fecf2a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc003970 0x7fecec0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.543+0000 7fecf2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecec0b65d0 0x7fecec1f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.543+0000 7fecf2a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fecec15a710 0x7fecec06b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.543+0000 7fecf2a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec06cc40 con 0x7fecec0b65d0 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.543+0000 7fecf2a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec0b5bc0 con 0x7feccc003970 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.543+0000 7fecf2a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecec1f6450 con 0x7fecec15a710 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.543+0000 7fecf0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecec0b65d0 0x7fecec1f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.543+0000 7fecf0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecec0b65d0 0x7fecec1f6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46284/0 (socket says 172.21.15.26:46284) 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.543+0000 7fecf0c00640 1 -- 172.21.15.26:0/401727013 learned_addr learned my addr 172.21.15.26:0/401727013 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:20.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf1600640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc003970 0x7fecec0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf2000640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fecec15a710 0x7fecec06b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf0c00640 1 -- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fecec15a710 msgr2=0x7fecec06b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf0c00640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fecec15a710 0x7fecec06b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf0c00640 1 -- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc003970 msgr2=0x7fecec0b8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf0c00640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc003970 0x7fecec0b8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf0c00640 1 -- 172.21.15.26:0/401727013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec15ab90 con 0x7fecec0b65d0 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf1600640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc003970 0x7fecec0b8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf0c00640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecec0b65d0 0x7fecec1f6ea0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fecd4011730 tx=0x7fecd4011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fece9600640 1 -- 172.21.15.26:0/401727013 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400f9c0 con 0x7fecec0b65d0 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fece9600640 1 -- 172.21.15.26:0/401727013 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd400fb20 con 0x7fecec0b65d0 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fece9600640 1 -- 172.21.15.26:0/401727013 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400fe90 con 0x7fecec0b65d0 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.544+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec0b5e10 con 0x7fecec0b65d0 2024-01-15T18:07:20.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.545+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec1f66a0 con 0x7fecec0b65d0 2024-01-15T18:07:20.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.546+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec067450 con 0x7fecec0b65d0 2024-01-15T18:07:20.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.547+0000 7fece9600640 1 -- 172.21.15.26:0/401727013 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fecd4015460 con 0x7fecec0b65d0 2024-01-15T18:07:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.547+0000 7fece9600640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fecc406bf50 0x7fecc406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.548+0000 7fece9600640 1 -- 172.21.15.26:0/401727013 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fecd4088a40 con 0x7fecec0b65d0 2024-01-15T18:07:20.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.548+0000 7fecf1600640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fecc406bf50 0x7fecc406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:20.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.549+0000 7fecf1600640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fecc406bf50 0x7fecc406e410 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fecdc034000 tx=0x7fecdc007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:20.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.552+0000 7fece9600640 1 -- 172.21.15.26:0/401727013 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fecd4055300 con 0x7fecec0b65d0 2024-01-15T18:07:20.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.781+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone4189", "target": ["mon-mgr", ""]}) v1 -- 0x7fecec0d21a0 con 0x7fecc406bf50 2024-01-15T18:07:20.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.805+0000 7fece9600640 1 -- 172.21.15.26:0/401727013 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7fecdc040460 con 0x7fecc406bf50 2024-01-15T18:07:20.809 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:20.809 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:20.809 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:20.809 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:20.809 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:20.810 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:20.810 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:20.810 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:20.810 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:20.810 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:20.810 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:20.810 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:20.810 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:20.810 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fecc406bf50 msgr2=0x7fecc406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fecc406bf50 0x7fecc406e410 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fecdc034000 tx=0x7fecdc007000 comp rx=0 tx=0).stop 2024-01-15T18:07:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecec0b65d0 msgr2=0x7fecec1f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecec0b65d0 0x7fecec1f6ea0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fecd4011730 tx=0x7fecd4011bf0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 shutdown_connections 2024-01-15T18:07:20.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fecec15a710 0x7fecec06b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fecc406bf50 0x7fecc406e410 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fecec0b65d0 0x7fecec1f6ea0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 --2- 172.21.15.26:0/401727013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feccc003970 0x7fecec0b8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 >> 172.21.15.26:0/401727013 conn(0x7fecec0d3000 msgr2=0x7fecec0d34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 shutdown_connections 2024-01-15T18:07:20.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:20.808+0000 7fecf2a00640 1 -- 172.21.15.26:0/401727013 wait complete. 2024-01-15T18:07:20.830 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:20.830 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41810 2024-01-15T18:07:21.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.049+0000 7fa951c00640 1 Processor -- start 2024-01-15T18:07:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.049+0000 7fa951c00640 1 -- start start 2024-01-15T18:07:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.051+0000 7fa951c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa94c0ccb50 0x7fa94c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.051+0000 7fa951c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fa94c15bf10 0x7fa94c143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.051+0000 7fa951c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c15f4e0 con 0x7fa94c0adf20 2024-01-15T18:07:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.051+0000 7fa951c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c15f970 con 0x7fa94c0ccb50 2024-01-15T18:07:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.051+0000 7fa951c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c071130 con 0x7fa94c15bf10 2024-01-15T18:07:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.051+0000 7fa94be00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa94c0adf20 0x7fa94c0cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57018/0 (socket says 172.21.15.26:57018) 2024-01-15T18:07:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.051+0000 7fa94b400640 1 --2- 172.21.15.26:0/2731895570 >> v2:172.21.15.26:3300/0 conn(0x7fa94c0ccb50 0x7fa94c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.051+0000 7fa94be00640 1 -- 172.21.15.26:0/2731895570 learned_addr learned my addr 172.21.15.26:0/2731895570 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.052+0000 7fa94b400640 1 -- 172.21.15.26:0/2731895570 >> v2:172.21.15.124:3300/0 conn(0x7fa94c15bf10 msgr2=0x7fa94c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.052+0000 7fa94b400640 1 --2- 172.21.15.26:0/2731895570 >> v2:172.21.15.124:3300/0 conn(0x7fa94c15bf10 0x7fa94c143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.052+0000 7fa94b400640 1 -- 172.21.15.26:0/2731895570 >> v1:172.21.15.26:6789/0 conn(0x7fa94c0adf20 legacy=0x7fa94c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.052+0000 7fa94b400640 1 -- 172.21.15.26:0/2731895570 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c15ebb0 con 0x7fa94c0ccb50 2024-01-15T18:07:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.053+0000 7fa94b400640 1 --2- 172.21.15.26:0/2731895570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0ccb50 0x7fa94c070a30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa94c06ff20 tx=0x7fa93402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d51bcc5ad62756 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.053+0000 7fa94aa00640 1 -- 172.21.15.26:0/2731895570 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93402e070 con 0x7fa94c0ccb50 2024-01-15T18:07:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.053+0000 7fa94aa00640 1 -- 172.21.15.26:0/2731895570 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9340315a0 con 0x7fa94c0ccb50 2024-01-15T18:07:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.053+0000 7fa94aa00640 1 -- 172.21.15.26:0/2731895570 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa934031930 con 0x7fa94c0ccb50 2024-01-15T18:07:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.054+0000 7fa951c00640 1 -- 172.21.15.26:0/2731895570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0ccb50 msgr2=0x7fa94c070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.054+0000 7fa951c00640 1 --2- 172.21.15.26:0/2731895570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0ccb50 0x7fa94c070a30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa94c06ff20 tx=0x7fa93402b840 comp rx=0 tx=0).stop 2024-01-15T18:07:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.054+0000 7fa951c00640 1 -- 172.21.15.26:0/2731895570 shutdown_connections 2024-01-15T18:07:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.054+0000 7fa951c00640 1 --2- 172.21.15.26:0/2731895570 >> v2:172.21.15.124:3300/0 conn(0x7fa94c15bf10 0x7fa94c143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.054+0000 7fa951c00640 1 --2- 172.21.15.26:0/2731895570 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0ccb50 0x7fa94c070a30 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.054+0000 7fa951c00640 1 -- 172.21.15.26:0/2731895570 >> 172.21.15.26:0/2731895570 conn(0x7fa94c06a7d0 msgr2=0x7fa94c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.054+0000 7fa951c00640 1 -- 172.21.15.26:0/2731895570 shutdown_connections 2024-01-15T18:07:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.054+0000 7fa951c00640 1 -- 172.21.15.26:0/2731895570 wait complete. 2024-01-15T18:07:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.056+0000 7fa951c00640 1 Processor -- start 2024-01-15T18:07:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.056+0000 7fa951c00640 1 -- start start 2024-01-15T18:07:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.057+0000 7fa951c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0adf20 0x7fa94c0c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.057+0000 7fa951c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa94c0ccb50 0x7fa94c15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.057+0000 7fa951c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa94c15bf10 0x7fa94c071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.057+0000 7fa951c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0c4840 con 0x7fa94c0adf20 2024-01-15T18:07:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.057+0000 7fa951c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0c51a0 con 0x7fa94c15bf10 2024-01-15T18:07:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.057+0000 7fa951c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0c3ea0 con 0x7fa94c0ccb50 2024-01-15T18:07:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.057+0000 7fa94be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0adf20 0x7fa94c0c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.057+0000 7fa94be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0adf20 0x7fa94c0c4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46298/0 (socket says 172.21.15.26:46298) 2024-01-15T18:07:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.057+0000 7fa94be00640 1 -- 172.21.15.26:0/228101395 learned_addr learned my addr 172.21.15.26:0/228101395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa94be00640 1 -- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa94c0ccb50 msgr2=0x7fa94c15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa94be00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa94c0ccb50 0x7fa94c15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa94be00640 1 -- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa94c15bf10 msgr2=0x7fa94c071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa94be00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa94c15bf10 0x7fa94c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa94be00640 1 -- 172.21.15.26:0/228101395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa934009400 con 0x7fa94c0adf20 2024-01-15T18:07:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa94be00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0adf20 0x7fa94c0c4c60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c4a0 tx=0x7fa93c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa943e00640 1 -- 172.21.15.26:0/228101395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c005ce0 con 0x7fa94c0adf20 2024-01-15T18:07:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa943e00640 1 -- 172.21.15.26:0/228101395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c005e40 con 0x7fa94c0adf20 2024-01-15T18:07:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa943e00640 1 -- 172.21.15.26:0/228101395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c007270 con 0x7fa94c0adf20 2024-01-15T18:07:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c0c4180 con 0x7fa94c0adf20 2024-01-15T18:07:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.058+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c15ac20 con 0x7fa94c0adf20 2024-01-15T18:07:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.059+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c0c4980 con 0x7fa94c0adf20 2024-01-15T18:07:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.060+0000 7fa943e00640 1 -- 172.21.15.26:0/228101395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fa93c007490 con 0x7fa94c0adf20 2024-01-15T18:07:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.061+0000 7fa943e00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa92406bf20 0x7fa92406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.061+0000 7fa943e00640 1 -- 172.21.15.26:0/228101395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fa93c087ca0 con 0x7fa94c0adf20 2024-01-15T18:07:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.062+0000 7fa94b400640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa92406bf20 0x7fa92406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.063+0000 7fa94b400640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa92406bf20 0x7fa92406e3e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa934036000 tx=0x7fa934030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:21.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.066+0000 7fa943e00640 1 -- 172.21.15.26:0/228101395 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa93c054390 con 0x7fa94c0adf20 2024-01-15T18:07:21.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.297+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41810", "target": ["mon-mgr", ""]}) v1 -- 0x7fa94c15ae10 con 0x7fa92406bf20 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.324+0000 7fa943e00640 1 -- 172.21.15.26:0/228101395 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7fa934031af0 con 0x7fa92406bf20 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:21.328 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:21.329 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:21.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:21.329 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:21.329 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa92406bf20 msgr2=0x7fa92406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa92406bf20 0x7fa92406e3e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa934036000 tx=0x7fa934030000 comp rx=0 tx=0).stop 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0adf20 msgr2=0x7fa94c0c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0adf20 0x7fa94c0c4c60 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c4a0 tx=0x7fa93c00c960 comp rx=0 tx=0).stop 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 shutdown_connections 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa94c0ccb50 0x7fa94c15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa92406bf20 0x7fa92406e3e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa94c15bf10 0x7fa94c071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 --2- 172.21.15.26:0/228101395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa94c0adf20 0x7fa94c0c4c60 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 >> 172.21.15.26:0/228101395 conn(0x7fa94c06a7d0 msgr2=0x7fa94c06cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 shutdown_connections 2024-01-15T18:07:21.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.327+0000 7fa951c00640 1 -- 172.21.15.26:0/228101395 wait complete. 2024-01-15T18:07:21.348 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:21.348 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41811 2024-01-15T18:07:21.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.564+0000 7fccd3c00640 1 Processor -- start 2024-01-15T18:07:21.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.564+0000 7fccd3c00640 1 -- start start 2024-01-15T18:07:21.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.566+0000 7fccd3c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcccc06a3e0 0x7fcccc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.566+0000 7fccd3c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcccc0d0b60 0x7fcccc0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.566+0000 7fccd3c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcccc071de0 con 0x7fcccc06a7c0 2024-01-15T18:07:21.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.566+0000 7fccd3c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcccc0af320 con 0x7fcccc06a3e0 2024-01-15T18:07:21.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.566+0000 7fccd3c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc072030 con 0x7fcccc0d0b60 2024-01-15T18:07:21.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.566+0000 7fccd1200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcccc06a3e0 0x7fcccc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:21.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.566+0000 7fccd1c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcccc06a7c0 0x7fcccc06f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57028/0 (socket says 172.21.15.26:57028) 2024-01-15T18:07:21.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.566+0000 7fccd1c00640 1 -- 172.21.15.26:0/2580090310 learned_addr learned my addr 172.21.15.26:0/2580090310 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:21.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.567+0000 7fccd1200640 1 -- 172.21.15.26:0/2580090310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcccc0d0b60 msgr2=0x7fcccc0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:21.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.567+0000 7fccd1200640 1 --2- 172.21.15.26:0/2580090310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcccc0d0b60 0x7fcccc0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.567+0000 7fccd1200640 1 -- 172.21.15.26:0/2580090310 >> v1:172.21.15.26:6789/0 conn(0x7fcccc06a7c0 legacy=0x7fcccc06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.567+0000 7fccd1200640 1 -- 172.21.15.26:0/2580090310 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc072310 con 0x7fcccc06a3e0 2024-01-15T18:07:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.568+0000 7fccd1200640 1 --2- 172.21.15.26:0/2580090310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a3e0 0x7fcccc0ac8d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0097a0 tx=0x7fccbc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f152da2ba79b1cbf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.569+0000 7fcccb400640 1 -- 172.21.15.26:0/2580090310 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc02d040 con 0x7fcccc06a3e0 2024-01-15T18:07:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.569+0000 7fcccb400640 1 -- 172.21.15.26:0/2580090310 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc031810 con 0x7fcccc06a3e0 2024-01-15T18:07:21.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.569+0000 7fcccb400640 1 -- 172.21.15.26:0/2580090310 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc031ba0 con 0x7fcccc06a3e0 2024-01-15T18:07:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.569+0000 7fccd3c00640 1 -- 172.21.15.26:0/2580090310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a3e0 msgr2=0x7fcccc0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.569+0000 7fccd3c00640 1 --2- 172.21.15.26:0/2580090310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a3e0 0x7fcccc0ac8d0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fccbc0097a0 tx=0x7fccbc02b8b0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.569+0000 7fccd3c00640 1 -- 172.21.15.26:0/2580090310 shutdown_connections 2024-01-15T18:07:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.570+0000 7fccd3c00640 1 --2- 172.21.15.26:0/2580090310 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcccc0d0b60 0x7fcccc0aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.570+0000 7fccd3c00640 1 --2- 172.21.15.26:0/2580090310 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a3e0 0x7fcccc0ac8d0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.570+0000 7fccd3c00640 1 -- 172.21.15.26:0/2580090310 >> 172.21.15.26:0/2580090310 conn(0x7fcccc0cfef0 msgr2=0x7fcccc0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.570+0000 7fccd3c00640 1 -- 172.21.15.26:0/2580090310 shutdown_connections 2024-01-15T18:07:21.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.570+0000 7fccd3c00640 1 -- 172.21.15.26:0/2580090310 wait complete. 2024-01-15T18:07:21.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.571+0000 7fccd3c00640 1 Processor -- start 2024-01-15T18:07:21.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.572+0000 7fccd3c00640 1 -- start start 2024-01-15T18:07:21.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.572+0000 7fccd3c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcccc06a3e0 0x7fcccc152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.572+0000 7fccd3c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a7c0 0x7fcccc15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fccd3c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcccc0d0b60 0x7fcccc15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fccd3c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc15d5f0 con 0x7fcccc06a7c0 2024-01-15T18:07:21.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fccd3c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc14f6c0 con 0x7fcccc0d0b60 2024-01-15T18:07:21.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fccd3c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc14f9a0 con 0x7fcccc06a3e0 2024-01-15T18:07:21.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fcccbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a7c0 0x7fcccc15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:21.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fcccbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a7c0 0x7fcccc15b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:46312/0 (socket says 172.21.15.26:46312) 2024-01-15T18:07:21.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fcccbe00640 1 -- 172.21.15.26:0/1872965816 learned_addr learned my addr 172.21.15.26:0/1872965816 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:21.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fccd1200640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcccc06a3e0 0x7fcccc152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:21.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fccd1c00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcccc0d0b60 0x7fcccc15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:21.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fcccbe00640 1 -- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcccc06a3e0 msgr2=0x7fcccc152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fcccbe00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcccc06a3e0 0x7fcccc152320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fcccbe00640 1 -- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcccc0d0b60 msgr2=0x7fcccc15ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fcccbe00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcccc0d0b60 0x7fcccc15ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.573+0000 7fcccbe00640 1 -- 172.21.15.26:0/1872965816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc009400 con 0x7fcccc06a7c0 2024-01-15T18:07:21.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.574+0000 7fcccbe00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a7c0 0x7fcccc15b500 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fccb400b410 tx=0x7fccb400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:21.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.574+0000 7fccc8c00640 1 -- 172.21.15.26:0/1872965816 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4011830 con 0x7fcccc06a7c0 2024-01-15T18:07:21.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.574+0000 7fccc8c00640 1 -- 172.21.15.26:0/1872965816 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb40119b0 con 0x7fcccc06a7c0 2024-01-15T18:07:21.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.574+0000 7fccc8c00640 1 -- 172.21.15.26:0/1872965816 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4011d60 con 0x7fcccc06a7c0 2024-01-15T18:07:21.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.574+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc14fc80 con 0x7fcccc06a7c0 2024-01-15T18:07:21.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.574+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc150240 con 0x7fcccc06a7c0 2024-01-15T18:07:21.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.575+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc0675d0 con 0x7fcccc06a7c0 2024-01-15T18:07:21.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.576+0000 7fccc8c00640 1 -- 172.21.15.26:0/1872965816 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fccb401a400 con 0x7fcccc06a7c0 2024-01-15T18:07:21.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.577+0000 7fccc8c00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcca406bf20 0x7fcca406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:21.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.577+0000 7fccc8c00640 1 -- 172.21.15.26:0/1872965816 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fccb4088a30 con 0x7fcccc06a7c0 2024-01-15T18:07:21.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.578+0000 7fccd1200640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcca406bf20 0x7fcca406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:21.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.579+0000 7fccd1200640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcca406bf20 0x7fcca406e3e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fccbc036000 tx=0x7fccbc03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:21.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.582+0000 7fccc8c00640 1 -- 172.21.15.26:0/1872965816 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fccb4054fd0 con 0x7fcccc06a7c0 2024-01-15T18:07:21.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.813+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41811", "target": ["mon-mgr", ""]}) v1 -- 0x7fcccc160a60 con 0x7fcca406bf20 2024-01-15T18:07:21.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.842+0000 7fccc8c00640 1 -- 172.21.15.26:0/1872965816 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7fcccc160a60 con 0x7fcca406bf20 2024-01-15T18:07:21.845 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:21.846 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.844+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcca406bf20 msgr2=0x7fcca406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.844+0000 7fccd3c00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcca406bf20 0x7fcca406e3e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fccbc036000 tx=0x7fccbc03c040 comp rx=0 tx=0).stop 2024-01-15T18:07:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.844+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a7c0 msgr2=0x7fcccc15b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:21.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.844+0000 7fccd3c00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a7c0 0x7fcccc15b500 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fccb400b410 tx=0x7fccb400b8d0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.845+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 shutdown_connections 2024-01-15T18:07:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.845+0000 7fccd3c00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcca406bf20 0x7fcca406e3e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.845+0000 7fccd3c00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcccc0d0b60 0x7fcccc15ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.845+0000 7fccd3c00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcccc06a7c0 0x7fcccc15b500 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.845+0000 7fccd3c00640 1 --2- 172.21.15.26:0/1872965816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcccc06a3e0 0x7fcccc152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:21.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.845+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 >> 172.21.15.26:0/1872965816 conn(0x7fcccc0cfef0 msgr2=0x7fcccc0ae080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.845+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 shutdown_connections 2024-01-15T18:07:21.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:21.845+0000 7fccd3c00640 1 -- 172.21.15.26:0/1872965816 wait complete. 2024-01-15T18:07:21.868 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:21.868 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41812 2024-01-15T18:07:22.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.082+0000 7ff543600640 1 Processor -- start 2024-01-15T18:07:22.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.082+0000 7ff543600640 1 -- start start 2024-01-15T18:07:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.083+0000 7ff543600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff53c150760 0x7ff53c072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.083+0000 7ff543600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53c0b3710 con 0x7ff53c0b32e0 2024-01-15T18:07:22.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.083+0000 7ff543600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53c071190 con 0x7ff53c15d300 2024-01-15T18:07:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.083+0000 7ff543600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53c071470 con 0x7ff53c150760 2024-01-15T18:07:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.084+0000 7ff540c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff53c150760 0x7ff53c072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.084+0000 7ff541600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff53c0b32e0 0x7ff53c0ce040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53236/0 (socket says 172.21.15.26:53236) 2024-01-15T18:07:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.084+0000 7ff541600640 1 -- 172.21.15.26:0/731249562 learned_addr learned my addr 172.21.15.26:0/731249562 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:22.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.084+0000 7ff53b400640 1 -- 172.21.15.26:0/731249562 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284331897 0 0) 0x7ff53c0b3710 con 0x7ff53c0b32e0 2024-01-15T18:07:22.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.084+0000 7ff53b400640 1 -- 172.21.15.26:0/731249562 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff51c003520 con 0x7ff53c0b32e0 2024-01-15T18:07:22.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.085+0000 7ff540c00640 1 -- 172.21.15.26:0/731249562 >> v1:172.21.15.124:6789/0 conn(0x7ff53c15d300 legacy=0x7ff53c0b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:22.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.085+0000 7ff540c00640 1 -- 172.21.15.26:0/731249562 >> v1:172.21.15.26:6789/0 conn(0x7ff53c0b32e0 legacy=0x7ff53c0ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.085+0000 7ff540c00640 1 -- 172.21.15.26:0/731249562 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c071750 con 0x7ff53c150760 2024-01-15T18:07:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.086+0000 7ff540c00640 1 --2- 172.21.15.26:0/731249562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 0x7ff53c072230 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00a940 tx=0x7ff52c02cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b2656a91781b4c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.087+0000 7ff53b400640 1 -- 172.21.15.26:0/731249562 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c02db40 con 0x7ff53c150760 2024-01-15T18:07:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.087+0000 7ff53b400640 1 -- 172.21.15.26:0/731249562 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c02dca0 con 0x7ff53c150760 2024-01-15T18:07:22.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.087+0000 7ff53b400640 1 -- 172.21.15.26:0/731249562 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c035660 con 0x7ff53c150760 2024-01-15T18:07:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.087+0000 7ff543600640 1 -- 172.21.15.26:0/731249562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 msgr2=0x7ff53c072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.087+0000 7ff543600640 1 --2- 172.21.15.26:0/731249562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 0x7ff53c072230 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00a940 tx=0x7ff52c02cd90 comp rx=0 tx=0).stop 2024-01-15T18:07:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.088+0000 7ff543600640 1 -- 172.21.15.26:0/731249562 shutdown_connections 2024-01-15T18:07:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.088+0000 7ff543600640 1 --2- 172.21.15.26:0/731249562 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 0x7ff53c072230 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.088+0000 7ff543600640 1 -- 172.21.15.26:0/731249562 >> 172.21.15.26:0/731249562 conn(0x7ff53c06a800 msgr2=0x7ff53c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.088+0000 7ff543600640 1 -- 172.21.15.26:0/731249562 shutdown_connections 2024-01-15T18:07:22.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.088+0000 7ff543600640 1 -- 172.21.15.26:0/731249562 wait complete. 2024-01-15T18:07:22.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.089+0000 7ff543600640 1 Processor -- start 2024-01-15T18:07:22.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.090+0000 7ff543600640 1 -- start start 2024-01-15T18:07:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.090+0000 7ff543600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff53c0b32e0 0x7ff53c13f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.090+0000 7ff543600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 0x7ff53c146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.090+0000 7ff543600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff53c15d300 0x7ff53c147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.090+0000 7ff543600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0cc140 con 0x7ff53c150760 2024-01-15T18:07:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.090+0000 7ff543600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c1478b0 con 0x7ff53c15d300 2024-01-15T18:07:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.090+0000 7ff543600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c147b90 con 0x7ff53c0b32e0 2024-01-15T18:07:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff53be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 0x7ff53c146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff53be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 0x7ff53c146d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38606/0 (socket says 172.21.15.26:38606) 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff53be00640 1 -- 172.21.15.26:0/2837926303 learned_addr learned my addr 172.21.15.26:0/2837926303 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff541600640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff53c15d300 0x7ff53c147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff540c00640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff53c0b32e0 0x7ff53c13f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff53be00640 1 -- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff53c0b32e0 msgr2=0x7ff53c13f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff53be00640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff53c0b32e0 0x7ff53c13f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff53be00640 1 -- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff53c15d300 msgr2=0x7ff53c147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff53be00640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff53c15d300 0x7ff53c147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff53be00640 1 -- 172.21.15.26:0/2837926303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c00a5a0 con 0x7ff53c150760 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.091+0000 7ff540c00640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff53c0b32e0 0x7ff53c13f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:22.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.092+0000 7ff53be00640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 0x7ff53c146d40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff52400b3f0 tx=0x7ff52400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:22.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.092+0000 7ff538c00640 1 -- 172.21.15.26:0/2837926303 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524011830 con 0x7ff53c150760 2024-01-15T18:07:22.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.092+0000 7ff538c00640 1 -- 172.21.15.26:0/2837926303 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff524011990 con 0x7ff53c150760 2024-01-15T18:07:22.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.092+0000 7ff538c00640 1 -- 172.21.15.26:0/2837926303 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524011d40 con 0x7ff53c150760 2024-01-15T18:07:22.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.092+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c147e70 con 0x7ff53c150760 2024-01-15T18:07:22.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.092+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c13fd50 con 0x7ff53c150760 2024-01-15T18:07:22.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.093+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53c0cd3a0 con 0x7ff53c150760 2024-01-15T18:07:22.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.094+0000 7ff538c00640 1 -- 172.21.15.26:0/2837926303 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7ff52401a400 con 0x7ff53c150760 2024-01-15T18:07:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.096+0000 7ff538c00640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff51406bf20 0x7ff51406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.096+0000 7ff538c00640 1 -- 172.21.15.26:0/2837926303 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7ff5240886a0 con 0x7ff53c150760 2024-01-15T18:07:22.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.096+0000 7ff540c00640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff51406bf20 0x7ff51406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.097+0000 7ff540c00640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff51406bf20 0x7ff51406e3e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff52c037000 tx=0x7ff52c031000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:22.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.100+0000 7ff538c00640 1 -- 172.21.15.26:0/2837926303 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff53c0cd3a0 con 0x7ff53c150760 2024-01-15T18:07:22.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.326+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41812", "target": ["mon-mgr", ""]}) v1 -- 0x7ff53c0ccfe0 con 0x7ff51406bf20 2024-01-15T18:07:22.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.350+0000 7ff538c00640 1 -- 172.21.15.26:0/2837926303 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7ff52c035540 con 0x7ff51406bf20 2024-01-15T18:07:22.354 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:22.354 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:22.354 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:22.354 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:22.354 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:22.354 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:22.355 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:22.355 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:22.355 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:22.355 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:22.355 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:22.355 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:22.355 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:22.355 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff51406bf20 msgr2=0x7ff51406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff51406bf20 0x7ff51406e3e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff52c037000 tx=0x7ff52c031000 comp rx=0 tx=0).stop 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 msgr2=0x7ff53c146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 0x7ff53c146d40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff52400b3f0 tx=0x7ff52400b8b0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 shutdown_connections 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff51406bf20 0x7ff51406e3e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff53c15d300 0x7ff53c147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff53c150760 0x7ff53c146d40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 --2- 172.21.15.26:0/2837926303 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff53c0b32e0 0x7ff53c13f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 >> 172.21.15.26:0/2837926303 conn(0x7ff53c06a800 msgr2=0x7ff53c149ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 shutdown_connections 2024-01-15T18:07:22.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.353+0000 7ff543600640 1 -- 172.21.15.26:0/2837926303 wait complete. 2024-01-15T18:07:22.375 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:22.375 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41813 2024-01-15T18:07:22.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.587+0000 7fd9fbe00640 1 Processor -- start 2024-01-15T18:07:22.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.587+0000 7fd9fbe00640 1 -- start start 2024-01-15T18:07:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.588+0000 7fd9fbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9fc0ccb50 0x7fd9fc070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.588+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9fc15bf10 0x7fd9fc143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.588+0000 7fd9fbe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc15f4e0 con 0x7fd9fc0adf20 2024-01-15T18:07:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.588+0000 7fd9fbe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc15f970 con 0x7fd9fc0ccb50 2024-01-15T18:07:22.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.588+0000 7fd9fbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15c3c0 con 0x7fd9fc15bf10 2024-01-15T18:07:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.589+0000 7fd9fa000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9fc0ccb50 0x7fd9fc070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.589+0000 7fd9fa000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd9fc0ccb50 0x7fd9fc070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38618/0 (socket says 172.21.15.26:38618) 2024-01-15T18:07:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.589+0000 7fd9fa000640 1 -- 172.21.15.26:0/3098491559 learned_addr learned my addr 172.21.15.26:0/3098491559 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.589+0000 7fd9fb400640 1 --2- 172.21.15.26:0/3098491559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9fc15bf10 0x7fd9fc143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.589+0000 7fd9fa000640 1 -- 172.21.15.26:0/3098491559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9fc15bf10 msgr2=0x7fd9fc143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.589+0000 7fd9fa000640 1 --2- 172.21.15.26:0/3098491559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9fc15bf10 0x7fd9fc143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.589+0000 7fd9fa000640 1 -- 172.21.15.26:0/3098491559 >> v1:172.21.15.124:6789/0 conn(0x7fd9fc0adf20 legacy=0x7fd9fc0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.589+0000 7fd9fa000640 1 -- 172.21.15.26:0/3098491559 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc071130 con 0x7fd9fc0ccb50 2024-01-15T18:07:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.590+0000 7fd9fa000640 1 --2- 172.21.15.26:0/3098491559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0ccb50 0x7fd9fc070a30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009870 tx=0x7fd9e402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60520c9b542ca0d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9f9600640 1 -- 172.21.15.26:0/3098491559 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e40317c0 con 0x7fd9fc0ccb50 2024-01-15T18:07:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9f9600640 1 -- 172.21.15.26:0/3098491559 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e4031920 con 0x7fd9fc0ccb50 2024-01-15T18:07:22.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9f9600640 1 -- 172.21.15.26:0/3098491559 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4031cb0 con 0x7fd9fc0ccb50 2024-01-15T18:07:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9fbe00640 1 -- 172.21.15.26:0/3098491559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0ccb50 msgr2=0x7fd9fc070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9fbe00640 1 --2- 172.21.15.26:0/3098491559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0ccb50 0x7fd9fc070a30 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009870 tx=0x7fd9e402b860 comp rx=0 tx=0).stop 2024-01-15T18:07:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9fbe00640 1 -- 172.21.15.26:0/3098491559 shutdown_connections 2024-01-15T18:07:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9fbe00640 1 --2- 172.21.15.26:0/3098491559 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9fc15bf10 0x7fd9fc143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9fbe00640 1 --2- 172.21.15.26:0/3098491559 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0ccb50 0x7fd9fc070a30 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9fbe00640 1 -- 172.21.15.26:0/3098491559 >> 172.21.15.26:0/3098491559 conn(0x7fd9fc06a7d0 msgr2=0x7fd9fc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9fbe00640 1 -- 172.21.15.26:0/3098491559 shutdown_connections 2024-01-15T18:07:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.591+0000 7fd9fbe00640 1 -- 172.21.15.26:0/3098491559 wait complete. 2024-01-15T18:07:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.593+0000 7fd9fbe00640 1 Processor -- start 2024-01-15T18:07:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.593+0000 7fd9fbe00640 1 -- start start 2024-01-15T18:07:22.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.594+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0adf20 0x7fd9fc13ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.594+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9fc0ccb50 0x7fd9fc14c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.594+0000 7fd9fbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9fc15bf10 0x7fd9fc15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.594+0000 7fd9fbe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc14d4f0 con 0x7fd9fc0adf20 2024-01-15T18:07:22.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.594+0000 7fd9fbe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc14a980 con 0x7fd9fc0ccb50 2024-01-15T18:07:22.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.594+0000 7fd9fbe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc14cf30 con 0x7fd9fc15bf10 2024-01-15T18:07:22.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.594+0000 7fd9faa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0adf20 0x7fd9fc13ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.594+0000 7fd9faa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0adf20 0x7fd9fc13ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38622/0 (socket says 172.21.15.26:38622) 2024-01-15T18:07:22.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.594+0000 7fd9faa00640 1 -- 172.21.15.26:0/2674852294 learned_addr learned my addr 172.21.15.26:0/2674852294 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:22.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9faa00640 1 -- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9fc15bf10 msgr2=0x7fd9fc15b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9faa00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9fc15bf10 0x7fd9fc15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9faa00640 1 -- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9fc0ccb50 msgr2=0x7fd9fc14c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9faa00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9fc0ccb50 0x7fd9fc14c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9faa00640 1 -- 172.21.15.26:0/2674852294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40094d0 con 0x7fd9fc0adf20 2024-01-15T18:07:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9faa00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0adf20 0x7fd9fc13ec20 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c7c0 tx=0x7fd9ec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9f2a00640 1 -- 172.21.15.26:0/2674852294 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00aa50 con 0x7fd9fc0adf20 2024-01-15T18:07:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9f2a00640 1 -- 172.21.15.26:0/2674852294 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec00abb0 con 0x7fd9fc0adf20 2024-01-15T18:07:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc14d210 con 0x7fd9fc0adf20 2024-01-15T18:07:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.595+0000 7fd9f2a00640 1 -- 172.21.15.26:0/2674852294 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec012420 con 0x7fd9fc0adf20 2024-01-15T18:07:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.596+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc13f160 con 0x7fd9fc0adf20 2024-01-15T18:07:22.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.597+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc14a470 con 0x7fd9fc0adf20 2024-01-15T18:07:22.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.598+0000 7fd9f2a00640 1 -- 172.21.15.26:0/2674852294 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fd9ec012580 con 0x7fd9fc0adf20 2024-01-15T18:07:22.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.599+0000 7fd9f2a00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9d406bf20 0x7fd9d406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:22.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.599+0000 7fd9f2a00640 1 -- 172.21.15.26:0/2674852294 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fd9ec088550 con 0x7fd9fc0adf20 2024-01-15T18:07:22.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.600+0000 7fd9fa000640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9d406bf20 0x7fd9d406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:22.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.601+0000 7fd9fa000640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9d406bf20 0x7fd9d406e3e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4036000 tx=0x7fd9e4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:22.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.604+0000 7fd9f2a00640 1 -- 172.21.15.26:0/2674852294 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd9fc14a470 con 0x7fd9fc0adf20 2024-01-15T18:07:22.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.834+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41813", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9fc15f170 con 0x7fd9d406bf20 2024-01-15T18:07:22.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.854+0000 7fd9f2a00640 1 -- 172.21.15.26:0/2674852294 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7fd9e4031a90 con 0x7fd9d406bf20 2024-01-15T18:07:22.858 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:22.858 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:22.859 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9d406bf20 msgr2=0x7fd9d406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9d406bf20 0x7fd9d406e3e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4036000 tx=0x7fd9e4030000 comp rx=0 tx=0).stop 2024-01-15T18:07:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0adf20 msgr2=0x7fd9fc13ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0adf20 0x7fd9fc13ec20 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00c7c0 tx=0x7fd9ec00cc80 comp rx=0 tx=0).stop 2024-01-15T18:07:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 shutdown_connections 2024-01-15T18:07:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9fc15bf10 0x7fd9fc15b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9d406bf20 0x7fd9d406e3e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9fc0ccb50 0x7fd9fc14c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 --2- 172.21.15.26:0/2674852294 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9fc0adf20 0x7fd9fc13ec20 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:22.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 >> 172.21.15.26:0/2674852294 conn(0x7fd9fc06a7d0 msgr2=0x7fd9fc06da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:22.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 shutdown_connections 2024-01-15T18:07:22.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:22.857+0000 7fd9fbe00640 1 -- 172.21.15.26:0/2674852294 wait complete. 2024-01-15T18:07:22.879 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:22.879 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41814 2024-01-15T18:07:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.095+0000 7f99ab800640 1 Processor -- start 2024-01-15T18:07:23.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.095+0000 7f99ab800640 1 -- start start 2024-01-15T18:07:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.097+0000 7f99ab800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99a40b3ae0 0x7f99a415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.097+0000 7f99ab800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a4070c40 con 0x7f99a40cf290 2024-01-15T18:07:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.097+0000 7f99ab800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a406c470 con 0x7f99a40ae5a0 2024-01-15T18:07:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.097+0000 7f99ab800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a4071450 con 0x7f99a40b3ae0 2024-01-15T18:07:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.097+0000 7f99a3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99a40b3ae0 0x7f99a415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.097+0000 7f99a9800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f99a40cf290 0x7f99a40b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53250/0 (socket says 172.21.15.26:53250) 2024-01-15T18:07:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.097+0000 7f99a3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f99a40b3ae0 0x7f99a415e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38628/0 (socket says 172.21.15.26:38628) 2024-01-15T18:07:23.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.097+0000 7f99a9800640 1 -- 172.21.15.26:0/3205085673 learned_addr learned my addr 172.21.15.26:0/3205085673 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:23.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.098+0000 7f99a3e00640 1 -- 172.21.15.26:0/3205085673 >> v1:172.21.15.124:6789/0 conn(0x7f99a40ae5a0 legacy=0x7f99a406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:23.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.098+0000 7f99a3e00640 1 -- 172.21.15.26:0/3205085673 >> v1:172.21.15.26:6789/0 conn(0x7f99a40cf290 legacy=0x7f99a40b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.098+0000 7f99a3e00640 1 -- 172.21.15.26:0/3205085673 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a406c750 con 0x7f99a40b3ae0 2024-01-15T18:07:23.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.098+0000 7f99a3400640 1 -- 172.21.15.26:0/3205085673 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910222161 0 0) 0x7f99a4070c40 con 0x7f99a40cf290 2024-01-15T18:07:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.099+0000 7f99a3e00640 1 --2- 172.21.15.26:0/3205085673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 0x7f99a415e430 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f998c0097a0 tx=0x7f998c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc986ba1233d799 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.099+0000 7f99a3400640 1 -- 172.21.15.26:0/3205085673 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c030610 con 0x7f99a40b3ae0 2024-01-15T18:07:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.099+0000 7f99a3400640 1 -- 172.21.15.26:0/3205085673 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c030770 con 0x7f99a40b3ae0 2024-01-15T18:07:23.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.099+0000 7f99a3400640 1 -- 172.21.15.26:0/3205085673 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c030b00 con 0x7f99a40b3ae0 2024-01-15T18:07:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.100+0000 7f99ab800640 1 -- 172.21.15.26:0/3205085673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 msgr2=0x7f99a415e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.100+0000 7f99ab800640 1 --2- 172.21.15.26:0/3205085673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 0x7f99a415e430 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f998c0097a0 tx=0x7f998c02b960 comp rx=0 tx=0).stop 2024-01-15T18:07:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.100+0000 7f99ab800640 1 -- 172.21.15.26:0/3205085673 shutdown_connections 2024-01-15T18:07:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.100+0000 7f99ab800640 1 --2- 172.21.15.26:0/3205085673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 0x7f99a415e430 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.100+0000 7f99ab800640 1 -- 172.21.15.26:0/3205085673 >> 172.21.15.26:0/3205085673 conn(0x7f99a406a800 msgr2=0x7f99a406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.100+0000 7f99ab800640 1 -- 172.21.15.26:0/3205085673 shutdown_connections 2024-01-15T18:07:23.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.100+0000 7f99ab800640 1 -- 172.21.15.26:0/3205085673 wait complete. 2024-01-15T18:07:23.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.102+0000 7f99ab800640 1 Processor -- start 2024-01-15T18:07:23.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.102+0000 7f99ab800640 1 -- start start 2024-01-15T18:07:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.103+0000 7f99ab800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99a40ae5a0 0x7f99a4157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.103+0000 7f99ab800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 0x7f99a40cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.103+0000 7f99ab800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99a40cf290 0x7f99a4160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.103+0000 7f99ab800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a415cd10 con 0x7f99a40b3ae0 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.103+0000 7f99ab800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40ce8d0 con 0x7f99a40cf290 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.103+0000 7f99ab800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a40cce20 con 0x7f99a40ae5a0 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.103+0000 7f99a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 0x7f99a40cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.103+0000 7f99a3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 0x7f99a40cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38634/0 (socket says 172.21.15.26:38634) 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.103+0000 7f99a3e00640 1 -- 172.21.15.26:0/2749479833 learned_addr learned my addr 172.21.15.26:0/2749479833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a9800640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99a40cf290 0x7f99a4160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a8e00640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99a40ae5a0 0x7f99a4157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a3e00640 1 -- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99a40ae5a0 msgr2=0x7f99a4157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a3e00640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99a40ae5a0 0x7f99a4157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a3e00640 1 -- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99a40cf290 msgr2=0x7f99a4160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a3e00640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99a40cf290 0x7f99a4160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a3e00640 1 -- 172.21.15.26:0/2749479833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c009400 con 0x7f99a40b3ae0 2024-01-15T18:07:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a9800640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99a40cf290 0x7f99a4160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a3e00640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 0x7f99a40cc8e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f998c0098c0 tx=0x7f998c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a0c00640 1 -- 172.21.15.26:0/2749479833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c03e040 con 0x7f99a40b3ae0 2024-01-15T18:07:23.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.104+0000 7f99a0c00640 1 -- 172.21.15.26:0/2749479833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c039cb0 con 0x7f99a40b3ae0 2024-01-15T18:07:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.105+0000 7f99a0c00640 1 -- 172.21.15.26:0/2749479833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c044630 con 0x7f99a40b3ae0 2024-01-15T18:07:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.105+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a415c690 con 0x7f99a40b3ae0 2024-01-15T18:07:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.105+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a415c930 con 0x7f99a40b3ae0 2024-01-15T18:07:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.106+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a415ce50 con 0x7f99a40b3ae0 2024-01-15T18:07:23.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.107+0000 7f99a0c00640 1 -- 172.21.15.26:0/2749479833 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f998c04d3b0 con 0x7f99a40b3ae0 2024-01-15T18:07:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.107+0000 7f99a0c00640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f997c06bf20 0x7f997c06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.108+0000 7f99a0c00640 1 -- 172.21.15.26:0/2749479833 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f998c0aad20 con 0x7f99a40b3ae0 2024-01-15T18:07:23.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.108+0000 7f99a8e00640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f997c06bf20 0x7f997c06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.109+0000 7f99a8e00640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f997c06bf20 0x7f997c06e3e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9994003d90 tx=0x7f9994009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.112+0000 7f99a0c00640 1 -- 172.21.15.26:0/2749479833 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f99a415ce50 con 0x7f99a40b3ae0 2024-01-15T18:07:23.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.334+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41814", "target": ["mon-mgr", ""]}) v1 -- 0x7f99a4070550 con 0x7f997c06bf20 2024-01-15T18:07:23.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.361+0000 7f99a0c00640 1 -- 172.21.15.26:0/2749479833 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f9994003980 con 0x7f997c06bf20 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:23.365 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.363+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f997c06bf20 msgr2=0x7f997c06e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.363+0000 7f99ab800640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f997c06bf20 0x7f997c06e3e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9994003d90 tx=0x7f9994009040 comp rx=0 tx=0).stop 2024-01-15T18:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.363+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 msgr2=0x7f99a40cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.363+0000 7f99ab800640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 0x7f99a40cc8e0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f998c0098c0 tx=0x7f998c004730 comp rx=0 tx=0).stop 2024-01-15T18:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.364+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 shutdown_connections 2024-01-15T18:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.364+0000 7f99ab800640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f997c06bf20 0x7f997c06e3e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.364+0000 7f99ab800640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99a40cf290 0x7f99a4160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.364+0000 7f99ab800640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99a40b3ae0 0x7f99a40cc8e0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.364+0000 7f99ab800640 1 --2- 172.21.15.26:0/2749479833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f99a40ae5a0 0x7f99a4157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.364+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 >> 172.21.15.26:0/2749479833 conn(0x7f99a406a800 msgr2=0x7f99a406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.364+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 shutdown_connections 2024-01-15T18:07:23.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.364+0000 7f99ab800640 1 -- 172.21.15.26:0/2749479833 wait complete. 2024-01-15T18:07:23.385 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:23.386 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41815 2024-01-15T18:07:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.606+0000 7f7fae800640 1 Processor -- start 2024-01-15T18:07:23.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.606+0000 7f7fae800640 1 -- start start 2024-01-15T18:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.607+0000 7f7fae800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7fa80cfde0 0x7f7fa80b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.607+0000 7f7fae800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa80d1c60 con 0x7f7fa815d4b0 2024-01-15T18:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.607+0000 7f7fae800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa80b2730 con 0x7f7fa80cf9a0 2024-01-15T18:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.607+0000 7f7fae800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80d1ec0 con 0x7f7fa80cfde0 2024-01-15T18:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.608+0000 7f7fad400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7fa815d4b0 0x7f7fa80d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53264/0 (socket says 172.21.15.26:53264) 2024-01-15T18:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.608+0000 7f7fad400640 1 -- 172.21.15.26:0/1431452745 learned_addr learned my addr 172.21.15.26:0/1431452745 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.608+0000 7f7fade00640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7fa80cfde0 0x7f7fa80b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.608+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039132842 0 0) 0x7f7fa80d1c60 con 0x7f7fa815d4b0 2024-01-15T18:07:23.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.608+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f88003520 con 0x7f7fa815d4b0 2024-01-15T18:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4283297949 0 0) 0x7f7f88003520 con 0x7f7fa815d4b0 2024-01-15T18:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7fa80cfde0 msgr2=0x7f7fa80b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7fa80cfde0 0x7f7fa80b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 >> v1:172.21.15.124:6789/0 conn(0x7f7fa80cf9a0 legacy=0x7f7fa80b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa80b29d0 con 0x7f7fa815d4b0 2024-01-15T18:07:23.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f7f980019f0 con 0x7f7fa815d4b0 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 >> v1:172.21.15.26:6789/0 conn(0x7f7fa815d4b0 legacy=0x7f7fa80d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f880039a0 0x7f7f88003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880047c0 0x7f7f88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7fa80cfde0 0x7f7f88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80d1ec0 con 0x7f7f880039a0 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8800ba90 con 0x7f7f880047c0 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8800bc50 con 0x7f7fa80cfde0 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fad400640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f880039a0 0x7f7f88003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7fade00640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7fa80cfde0 0x7f7f88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.609+0000 7f7faca00640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880047c0 0x7f7f88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.610+0000 7f7fad400640 1 -- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7fa80cfde0 msgr2=0x7f7f88005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.610+0000 7f7fad400640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7fa80cfde0 0x7f7f88005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.610+0000 7f7fad400640 1 -- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880047c0 msgr2=0x7f7f88004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:23.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.610+0000 7f7fad400640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f880047c0 0x7f7f88004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.610+0000 7f7fad400640 1 -- 172.21.15.26:0/1431452745 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f88003520 con 0x7f7f880039a0 2024-01-15T18:07:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.611+0000 7f7fad400640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f880039a0 0x7f7f88003dc0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f7f98002230 tx=0x7f7f880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf463d46dbbfedf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.611+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98034630 con 0x7f7f880039a0 2024-01-15T18:07:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.611+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9802f070 con 0x7f7f880039a0 2024-01-15T18:07:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.611+0000 7f7fa7e00640 1 -- 172.21.15.26:0/1431452745 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f980349d0 con 0x7f7f880039a0 2024-01-15T18:07:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.612+0000 7f7fae800640 1 -- 172.21.15.26:0/1431452745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f880039a0 msgr2=0x7f7f88003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.612+0000 7f7fae800640 1 --2- 172.21.15.26:0/1431452745 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f880039a0 0x7f7f88003dc0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f7f98002230 tx=0x7f7f880057f0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.612+0000 7f7fade00640 1 -- 172.21.15.26:0/1431452745 reap_dead start 2024-01-15T18:07:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.612+0000 7f7fae800640 1 -- 172.21.15.26:0/1431452745 shutdown_connections 2024-01-15T18:07:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.612+0000 7f7fae800640 1 -- 172.21.15.26:0/1431452745 >> 172.21.15.26:0/1431452745 conn(0x7f7fa80d3000 msgr2=0x7f7fa80d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.612+0000 7f7fae800640 1 -- 172.21.15.26:0/1431452745 shutdown_connections 2024-01-15T18:07:23.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.612+0000 7f7fae800640 1 -- 172.21.15.26:0/1431452745 wait complete. 2024-01-15T18:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.614+0000 7f7fae800640 1 Processor -- start 2024-01-15T18:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.614+0000 7f7fae800640 1 -- start start 2024-01-15T18:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fae800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880039a0 0x7f7fa814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fae800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7fa815a540 0x7f7fa809f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fae800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa815b4f0 0x7f7fa815b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fae800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa815beb0 con 0x7f7fa815b4f0 2024-01-15T18:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fae800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80a0030 con 0x7f7fa815a540 2024-01-15T18:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fae800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa80a0310 con 0x7f7f880039a0 2024-01-15T18:07:23.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fade00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa815b4f0 0x7f7fa815b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fade00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa815b4f0 0x7f7fa815b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38654/0 (socket says 172.21.15.26:38654) 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fade00640 1 -- 172.21.15.26:0/2750993552 learned_addr learned my addr 172.21.15.26:0/2750993552 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7faca00640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7fa815a540 0x7f7fa809f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.615+0000 7f7fad400640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880039a0 0x7f7fa814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fade00640 1 -- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880039a0 msgr2=0x7f7fa814d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fade00640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880039a0 0x7f7fa814d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fade00640 1 -- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7fa815a540 msgr2=0x7f7fa809f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fade00640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7fa815a540 0x7f7fa809f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fade00640 1 -- 172.21.15.26:0/2750993552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa80a05f0 con 0x7f7fa815b4f0 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fad400640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880039a0 0x7f7fa814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fade00640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa815b4f0 0x7f7fa815b970 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f7fa806aaa0 tx=0x7f7fa0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fa5600640 1 -- 172.21.15.26:0/2750993552 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0004500 con 0x7f7fa815b4f0 2024-01-15T18:07:23.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fa5600640 1 -- 172.21.15.26:0/2750993552 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa0003610 con 0x7f7fa815b4f0 2024-01-15T18:07:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fa5600640 1 -- 172.21.15.26:0/2750993552 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00039e0 con 0x7f7fa815b4f0 2024-01-15T18:07:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa80a08d0 con 0x7f7fa815b4f0 2024-01-15T18:07:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.616+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa814d850 con 0x7f7fa815b4f0 2024-01-15T18:07:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.618+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa80d3e30 con 0x7f7fa815b4f0 2024-01-15T18:07:23.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.619+0000 7f7fa5600640 1 -- 172.21.15.26:0/2750993552 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f7fa0003b40 con 0x7f7fa815b4f0 2024-01-15T18:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.619+0000 7f7fa5600640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f8006bf50 0x7f7f8006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.620+0000 7f7fa5600640 1 -- 172.21.15.26:0/2750993552 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f7fa0088590 con 0x7f7fa815b4f0 2024-01-15T18:07:23.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.620+0000 7f7fad400640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f8006bf50 0x7f7f8006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:23.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.621+0000 7f7fad400640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f8006bf50 0x7f7f8006e410 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7f98034000 tx=0x7f7f9802f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:23.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.624+0000 7f7fa5600640 1 -- 172.21.15.26:0/2750993552 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7fa0054bd0 con 0x7f7fa815b4f0 2024-01-15T18:07:23.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.854+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41815", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fa80d21a0 con 0x7f7f8006bf50 2024-01-15T18:07:23.880 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.877+0000 7f7fa5600640 1 -- 172.21.15.26:0/2750993552 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f7f98034430 con 0x7f7f8006bf50 2024-01-15T18:07:23.880 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:23.880 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:23.881 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.879+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f8006bf50 msgr2=0x7f7f8006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.879+0000 7f7fae800640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f8006bf50 0x7f7f8006e410 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7f98034000 tx=0x7f7f9802f000 comp rx=0 tx=0).stop 2024-01-15T18:07:23.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.879+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa815b4f0 msgr2=0x7f7fa815b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.879+0000 7f7fae800640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa815b4f0 0x7f7fa815b970 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f7fa806aaa0 tx=0x7f7fa0011920 comp rx=0 tx=0).stop 2024-01-15T18:07:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.880+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 shutdown_connections 2024-01-15T18:07:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.880+0000 7f7fae800640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f8006bf50 0x7f7f8006e410 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.880+0000 7f7fae800640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7fa815b4f0 0x7f7fa815b970 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.880+0000 7f7fae800640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7fa815a540 0x7f7fa809f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.880+0000 7f7fae800640 1 --2- 172.21.15.26:0/2750993552 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f880039a0 0x7f7fa814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.880+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 >> 172.21.15.26:0/2750993552 conn(0x7f7fa80d3000 msgr2=0x7f7fa806c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.880+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 shutdown_connections 2024-01-15T18:07:23.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:23.880+0000 7f7fae800640 1 -- 172.21.15.26:0/2750993552 wait complete. 2024-01-15T18:07:23.904 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:23.904 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41816 2024-01-15T18:07:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.127+0000 7fe72a800640 1 Processor -- start 2024-01-15T18:07:24.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.127+0000 7fe72a800640 1 -- start start 2024-01-15T18:07:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.128+0000 7fe72a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe724160f80 0x7fe724161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.128+0000 7fe72a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe7240ce8f0 0x7fe7240ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe72a800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe7240cfbe0 0x7fe7240d0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe72a800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7241618a0 con 0x7fe7240ce8f0 2024-01-15T18:07:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe72a800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7240d0de0 con 0x7fe7240cfbe0 2024-01-15T18:07:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe72a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240d10c0 con 0x7fe724160f80 2024-01-15T18:07:24.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe722a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe7240ce8f0 0x7fe7240ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe722a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe7240ce8f0 0x7fe7240ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38658/0 (socket says 172.21.15.26:38658) 2024-01-15T18:07:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe722a00640 1 -- 172.21.15.26:0/701941833 learned_addr learned my addr 172.21.15.26:0/701941833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe723400640 1 --2- 172.21.15.26:0/701941833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe724160f80 0x7fe724161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe723e00640 1 --2- 172.21.15.26:0/701941833 >> v2:172.21.15.124:3300/0 conn(0x7fe7240cfbe0 0x7fe7240d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe722a00640 1 -- 172.21.15.26:0/701941833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe724160f80 msgr2=0x7fe724161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe722a00640 1 --2- 172.21.15.26:0/701941833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe724160f80 0x7fe724161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe722a00640 1 -- 172.21.15.26:0/701941833 >> v2:172.21.15.124:3300/0 conn(0x7fe7240cfbe0 msgr2=0x7fe7240d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe722a00640 1 --2- 172.21.15.26:0/701941833 >> v2:172.21.15.124:3300/0 conn(0x7fe7240cfbe0 0x7fe7240d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.129+0000 7fe722a00640 1 -- 172.21.15.26:0/701941833 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7240d13a0 con 0x7fe7240ce8f0 2024-01-15T18:07:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.131+0000 7fe722a00640 1 --2- 172.21.15.26:0/701941833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240ce8f0 0x7fe7240ced50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe7240c6a40 tx=0x7fe70c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361198dc224f7e5a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.131+0000 7fe722000640 1 -- 172.21.15.26:0/701941833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c031870 con 0x7fe7240ce8f0 2024-01-15T18:07:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.131+0000 7fe722000640 1 -- 172.21.15.26:0/701941833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c0319d0 con 0x7fe7240ce8f0 2024-01-15T18:07:24.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.131+0000 7fe722000640 1 -- 172.21.15.26:0/701941833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c031d60 con 0x7fe7240ce8f0 2024-01-15T18:07:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.132+0000 7fe72a800640 1 -- 172.21.15.26:0/701941833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240ce8f0 msgr2=0x7fe7240ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.132+0000 7fe72a800640 1 --2- 172.21.15.26:0/701941833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240ce8f0 0x7fe7240ced50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe7240c6a40 tx=0x7fe70c02b910 comp rx=0 tx=0).stop 2024-01-15T18:07:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.132+0000 7fe72a800640 1 -- 172.21.15.26:0/701941833 shutdown_connections 2024-01-15T18:07:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.132+0000 7fe72a800640 1 --2- 172.21.15.26:0/701941833 >> v2:172.21.15.124:3300/0 conn(0x7fe7240cfbe0 0x7fe7240d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.132+0000 7fe72a800640 1 --2- 172.21.15.26:0/701941833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240ce8f0 0x7fe7240ced50 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.132+0000 7fe72a800640 1 --2- 172.21.15.26:0/701941833 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe724160f80 0x7fe724161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.132+0000 7fe72a800640 1 -- 172.21.15.26:0/701941833 >> 172.21.15.26:0/701941833 conn(0x7fe72409c550 msgr2=0x7fe7240d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.133+0000 7fe72a800640 1 -- 172.21.15.26:0/701941833 shutdown_connections 2024-01-15T18:07:24.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.133+0000 7fe72a800640 1 -- 172.21.15.26:0/701941833 wait complete. 2024-01-15T18:07:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.135+0000 7fe72a800640 1 Processor -- start 2024-01-15T18:07:24.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.135+0000 7fe72a800640 1 -- start start 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.136+0000 7fe72a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240ce8f0 0x7fe724158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.136+0000 7fe72a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240cfbe0 0x7fe7240cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.136+0000 7fe72a800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe724160f80 0x7fe72415f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.137+0000 7fe72a800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240caca0 con 0x7fe7240cfbe0 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.137+0000 7fe72a800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe724158ad0 con 0x7fe724160f80 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.137+0000 7fe72a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7240cae90 con 0x7fe7240ce8f0 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.137+0000 7fe722a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240cfbe0 0x7fe7240cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.137+0000 7fe722a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240cfbe0 0x7fe7240cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38660/0 (socket says 172.21.15.26:38660) 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.137+0000 7fe722a00640 1 -- 172.21.15.26:0/1273763888 learned_addr learned my addr 172.21.15.26:0/1273763888 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.137+0000 7fe723400640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240ce8f0 0x7fe724158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.137+0000 7fe723e00640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe724160f80 0x7fe72415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.138+0000 7fe722a00640 1 -- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240ce8f0 msgr2=0x7fe724158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.138+0000 7fe722a00640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240ce8f0 0x7fe724158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.138+0000 7fe722a00640 1 -- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe724160f80 msgr2=0x7fe72415f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.138+0000 7fe722a00640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe724160f80 0x7fe72415f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.138+0000 7fe722a00640 1 -- 172.21.15.26:0/1273763888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c0094d0 con 0x7fe7240cfbe0 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.138+0000 7fe723e00640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe724160f80 0x7fe72415f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.138+0000 7fe722a00640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240cfbe0 0x7fe7240cbf30 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe70c02b950 tx=0x7fe70c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.138+0000 7fe71b400640 1 -- 172.21.15.26:0/1273763888 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c03ad30 con 0x7fe7240cfbe0 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.139+0000 7fe71b400640 1 -- 172.21.15.26:0/1273763888 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c0443f0 con 0x7fe7240cfbe0 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.139+0000 7fe71b400640 1 -- 172.21.15.26:0/1273763888 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0447a0 con 0x7fe7240cfbe0 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.139+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7240cb170 con 0x7fe7240cfbe0 2024-01-15T18:07:24.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.139+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7240cb730 con 0x7fe7240cfbe0 2024-01-15T18:07:24.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.140+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7240675d0 con 0x7fe7240cfbe0 2024-01-15T18:07:24.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.141+0000 7fe71b400640 1 -- 172.21.15.26:0/1273763888 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fe70c04d450 con 0x7fe7240cfbe0 2024-01-15T18:07:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.142+0000 7fe71b400640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6fc06bf20 0x7fe6fc06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.143+0000 7fe71b400640 1 -- 172.21.15.26:0/1273763888 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fe70c02d040 con 0x7fe7240cfbe0 2024-01-15T18:07:24.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.143+0000 7fe723400640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6fc06bf20 0x7fe6fc06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.144+0000 7fe723400640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6fc06bf20 0x7fe6fc06e3e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe714006fd0 tx=0x7fe714008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.147+0000 7fe71b400640 1 -- 172.21.15.26:0/1273763888 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe70c0774b0 con 0x7fe7240cfbe0 2024-01-15T18:07:24.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.389+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41816", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7240d1c40 con 0x7fe6fc06bf20 2024-01-15T18:07:24.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.409+0000 7fe71b400640 1 -- 172.21.15.26:0/1273763888 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7fe71400f860 con 0x7fe6fc06bf20 2024-01-15T18:07:24.413 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:24.413 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:24.414 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:24.415 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:24.415 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:24.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6fc06bf20 msgr2=0x7fe6fc06e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6fc06bf20 0x7fe6fc06e3e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe714006fd0 tx=0x7fe714008040 comp rx=0 tx=0).stop 2024-01-15T18:07:24.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240cfbe0 msgr2=0x7fe7240cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240cfbe0 0x7fe7240cbf30 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe70c02b950 tx=0x7fe70c0046c0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 shutdown_connections 2024-01-15T18:07:24.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6fc06bf20 0x7fe6fc06e3e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe724160f80 0x7fe72415f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe7240cfbe0 0x7fe7240cbf30 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 --2- 172.21.15.26:0/1273763888 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe7240ce8f0 0x7fe724158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 >> 172.21.15.26:0/1273763888 conn(0x7fe72409c550 msgr2=0x7fe7240b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 shutdown_connections 2024-01-15T18:07:24.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.412+0000 7fe72a800640 1 -- 172.21.15.26:0/1273763888 wait complete. 2024-01-15T18:07:24.436 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:24.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41817 2024-01-15T18:07:24.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.659+0000 7fd48d800640 1 Processor -- start 2024-01-15T18:07:24.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.660+0000 7fd48d800640 1 -- start start 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.661+0000 7fd48d800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd4880d0010 0x7fd4880b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.661+0000 7fd48d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4880b2280 0x7fd4880b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.661+0000 7fd48d800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4880d3c30 con 0x7fd48815d5e0 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.661+0000 7fd48d800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4880ce840 con 0x7fd4880b2280 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.661+0000 7fd48d800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4880d3e80 con 0x7fd4880d0010 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.662+0000 7fd48ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4880b2280 0x7fd4880b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.662+0000 7fd48ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd4880b2280 0x7fd4880b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38662/0 (socket says 172.21.15.26:38662) 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.662+0000 7fd48ce00640 1 -- 172.21.15.26:0/1141764462 learned_addr learned my addr 172.21.15.26:0/1141764462 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.662+0000 7fd487400640 1 --2- 172.21.15.26:0/1141764462 >> v2:172.21.15.124:3300/0 conn(0x7fd4880d0010 0x7fd4880b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.662+0000 7fd48ce00640 1 -- 172.21.15.26:0/1141764462 >> v2:172.21.15.124:3300/0 conn(0x7fd4880d0010 msgr2=0x7fd4880b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.662+0000 7fd48ce00640 1 --2- 172.21.15.26:0/1141764462 >> v2:172.21.15.124:3300/0 conn(0x7fd4880d0010 0x7fd4880b4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.662+0000 7fd48ce00640 1 -- 172.21.15.26:0/1141764462 >> v1:172.21.15.124:6789/0 conn(0x7fd48815d5e0 legacy=0x7fd4880cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:24.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.662+0000 7fd48ce00640 1 -- 172.21.15.26:0/1141764462 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4880ceac0 con 0x7fd4880b2280 2024-01-15T18:07:24.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.664+0000 7fd48ce00640 1 --2- 172.21.15.26:0/1141764462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 0x7fd4880b2a40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd48808c4e0 tx=0x7fd48002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56b84b275ff80f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.664+0000 7fd486a00640 1 -- 172.21.15.26:0/1141764462 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48002c960 con 0x7fd4880b2280 2024-01-15T18:07:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.664+0000 7fd486a00640 1 -- 172.21.15.26:0/1141764462 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48002cac0 con 0x7fd4880b2280 2024-01-15T18:07:24.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.664+0000 7fd486a00640 1 -- 172.21.15.26:0/1141764462 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48002ce50 con 0x7fd4880b2280 2024-01-15T18:07:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.665+0000 7fd48d800640 1 -- 172.21.15.26:0/1141764462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 msgr2=0x7fd4880b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.665+0000 7fd48d800640 1 --2- 172.21.15.26:0/1141764462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 0x7fd4880b2a40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fd48808c4e0 tx=0x7fd48002bdc0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.665+0000 7fd48d800640 1 -- 172.21.15.26:0/1141764462 shutdown_connections 2024-01-15T18:07:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.665+0000 7fd48d800640 1 --2- 172.21.15.26:0/1141764462 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 0x7fd4880b2a40 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.665+0000 7fd48d800640 1 --2- 172.21.15.26:0/1141764462 >> v2:172.21.15.124:3300/0 conn(0x7fd4880d0010 0x7fd4880b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.665+0000 7fd48d800640 1 -- 172.21.15.26:0/1141764462 >> 172.21.15.26:0/1141764462 conn(0x7fd4880d3110 msgr2=0x7fd4880d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.666+0000 7fd48d800640 1 -- 172.21.15.26:0/1141764462 shutdown_connections 2024-01-15T18:07:24.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.666+0000 7fd48d800640 1 -- 172.21.15.26:0/1141764462 wait complete. 2024-01-15T18:07:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.667+0000 7fd48d800640 1 Processor -- start 2024-01-15T18:07:24.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.668+0000 7fd48d800640 1 -- start start 2024-01-15T18:07:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.668+0000 7fd48d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 0x7fd48814f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd48d800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4880d0010 0x7fd4881e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd48d800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd48815d5e0 0x7fd4881e81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd48d800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48815e530 con 0x7fd4880b2280 2024-01-15T18:07:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd48d800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4881e86e0 con 0x7fd4880d0010 2024-01-15T18:07:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd48d800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4881e8c70 con 0x7fd48815d5e0 2024-01-15T18:07:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 0x7fd48814f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 0x7fd48814f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38676/0 (socket says 172.21.15.26:38676) 2024-01-15T18:07:24.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487e00640 1 -- 172.21.15.26:0/3488148145 learned_addr learned my addr 172.21.15.26:0/3488148145 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:24.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487400640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4880d0010 0x7fd4881e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd48ce00640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd48815d5e0 0x7fd4881e81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487e00640 1 -- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd48815d5e0 msgr2=0x7fd4881e81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487e00640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd48815d5e0 0x7fd4881e81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487e00640 1 -- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4880d0010 msgr2=0x7fd4881e7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487e00640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4880d0010 0x7fd4881e7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487e00640 1 -- 172.21.15.26:0/3488148145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd480009b30 con 0x7fd4880b2280 2024-01-15T18:07:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.669+0000 7fd487400640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4880d0010 0x7fd4881e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.670+0000 7fd487e00640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 0x7fd48814f3b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c410 tx=0x7fd47800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.670+0000 7fd47fe00640 1 -- 172.21.15.26:0/3488148145 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780035f0 con 0x7fd4880b2280 2024-01-15T18:07:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.670+0000 7fd47fe00640 1 -- 172.21.15.26:0/3488148145 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd478003750 con 0x7fd4880b2280 2024-01-15T18:07:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.670+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4881e8990 con 0x7fd4880b2280 2024-01-15T18:07:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.670+0000 7fd47fe00640 1 -- 172.21.15.26:0/3488148145 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd478003b40 con 0x7fd4880b2280 2024-01-15T18:07:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.670+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48815e1c0 con 0x7fd4880b2280 2024-01-15T18:07:24.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.671+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd488067450 con 0x7fd4880b2280 2024-01-15T18:07:24.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.673+0000 7fd47fe00640 1 -- 172.21.15.26:0/3488148145 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7fd478003ca0 con 0x7fd4880b2280 2024-01-15T18:07:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.674+0000 7fd47fe00640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd46006bf20 0x7fd46006e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:24.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.674+0000 7fd47fe00640 1 -- 172.21.15.26:0/3488148145 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fd478087f50 con 0x7fd4880b2280 2024-01-15T18:07:24.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.674+0000 7fd487400640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd46006bf20 0x7fd46006e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:24.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.675+0000 7fd487400640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd46006bf20 0x7fd46006e3e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd4880b8e10 tx=0x7fd470008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:24.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.678+0000 7fd47fe00640 1 -- 172.21.15.26:0/3488148145 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd478054500 con 0x7fd4880b2280 2024-01-15T18:07:24.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.910+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41817", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4880cf080 con 0x7fd46006bf20 2024-01-15T18:07:24.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.934+0000 7fd47fe00640 1 -- 172.21.15.26:0/3488148145 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7fd4700095d0 con 0x7fd46006bf20 2024-01-15T18:07:24.938 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:24.939 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:24.939 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:24.939 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:24.939 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:24.939 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:24.939 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:24.940 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:24.941 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:24.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd46006bf20 msgr2=0x7fd46006e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd46006bf20 0x7fd46006e3e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd4880b8e10 tx=0x7fd470008040 comp rx=0 tx=0).stop 2024-01-15T18:07:24.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 msgr2=0x7fd48814f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:24.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 0x7fd48814f3b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c410 tx=0x7fd47800c8d0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 shutdown_connections 2024-01-15T18:07:24.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd48815d5e0 0x7fd4881e81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd46006bf20 0x7fd46006e3e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd4880d0010 0x7fd4881e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 --2- 172.21.15.26:0/3488148145 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4880b2280 0x7fd48814f3b0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:24.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.937+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 >> 172.21.15.26:0/3488148145 conn(0x7fd4880d3110 msgr2=0x7fd48806c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:24.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.938+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 shutdown_connections 2024-01-15T18:07:24.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:24.938+0000 7fd48d800640 1 -- 172.21.15.26:0/3488148145 wait complete. 2024-01-15T18:07:24.962 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:24.962 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41818 2024-01-15T18:07:25.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.185+0000 7f894ee00640 1 Processor -- start 2024-01-15T18:07:25.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.186+0000 7f894ee00640 1 -- start start 2024-01-15T18:07:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.187+0000 7f894ee00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89480cf9a0 0x7f89480cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.187+0000 7f894ee00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89480b31d0 0x7f89480b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.187+0000 7f894ee00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89480d0320 con 0x7f894815d4b0 2024-01-15T18:07:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.187+0000 7f894ee00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89480d1c60 con 0x7f89480b31d0 2024-01-15T18:07:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.187+0000 7f894ee00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89480b2730 con 0x7f89480cf9a0 2024-01-15T18:07:25.191 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.188+0000 7f894da00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f894815d4b0 0x7f89480d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53268/0 (socket says 172.21.15.26:53268) 2024-01-15T18:07:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.188+0000 7f894da00640 1 -- 172.21.15.26:0/448566391 learned_addr learned my addr 172.21.15.26:0/448566391 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.188+0000 7f894e400640 1 --2- 172.21.15.26:0/448566391 >> v2:172.21.15.26:3300/0 conn(0x7f89480b31d0 0x7f89480b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.188+0000 7f894e400640 1 -- 172.21.15.26:0/448566391 >> v2:172.21.15.124:3300/0 conn(0x7f89480cf9a0 msgr2=0x7f89480cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.188+0000 7f894e400640 1 --2- 172.21.15.26:0/448566391 >> v2:172.21.15.124:3300/0 conn(0x7f89480cf9a0 0x7f89480cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.188+0000 7f8947e00640 1 -- 172.21.15.26:0/448566391 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985996317 0 0) 0x7f89480d0320 con 0x7f894815d4b0 2024-01-15T18:07:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.188+0000 7f894e400640 1 -- 172.21.15.26:0/448566391 >> v1:172.21.15.26:6789/0 conn(0x7f894815d4b0 legacy=0x7f89480d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.188+0000 7f894e400640 1 -- 172.21.15.26:0/448566391 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89480d1ec0 con 0x7f89480b31d0 2024-01-15T18:07:25.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.189+0000 7f894e400640 1 --2- 172.21.15.26:0/448566391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 0x7f89480b3660 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f894808b580 tx=0x7f894002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6216ab87f48aef7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:25.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.189+0000 7f8947e00640 1 -- 172.21.15.26:0/448566391 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894002cc70 con 0x7f89480b31d0 2024-01-15T18:07:25.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f8947e00640 1 -- 172.21.15.26:0/448566391 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894002cdd0 con 0x7f89480b31d0 2024-01-15T18:07:25.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f8947e00640 1 -- 172.21.15.26:0/448566391 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8940034660 con 0x7f89480b31d0 2024-01-15T18:07:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f894ee00640 1 -- 172.21.15.26:0/448566391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 msgr2=0x7f89480b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f894ee00640 1 --2- 172.21.15.26:0/448566391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 0x7f89480b3660 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f894808b580 tx=0x7f894002bec0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f894ee00640 1 -- 172.21.15.26:0/448566391 shutdown_connections 2024-01-15T18:07:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f894ee00640 1 --2- 172.21.15.26:0/448566391 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 0x7f89480b3660 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f894ee00640 1 --2- 172.21.15.26:0/448566391 >> v2:172.21.15.124:3300/0 conn(0x7f89480cf9a0 0x7f89480cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f894ee00640 1 -- 172.21.15.26:0/448566391 >> 172.21.15.26:0/448566391 conn(0x7f89480d3000 msgr2=0x7f89480d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f894ee00640 1 -- 172.21.15.26:0/448566391 shutdown_connections 2024-01-15T18:07:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.190+0000 7f894ee00640 1 -- 172.21.15.26:0/448566391 wait complete. 2024-01-15T18:07:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.192+0000 7f894ee00640 1 Processor -- start 2024-01-15T18:07:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.192+0000 7f894ee00640 1 -- start start 2024-01-15T18:07:25.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.193+0000 7f894ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 0x7f894815ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.193+0000 7f894ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89480cf9a0 0x7f89480ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.193+0000 7f894ee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f894815d4b0 0x7f89480cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.193+0000 7f894ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89480cba70 con 0x7f89480b31d0 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.193+0000 7f894ee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89480ca9d0 con 0x7f894815d4b0 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.193+0000 7f894ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89480cacb0 con 0x7f89480cf9a0 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.193+0000 7f894da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 0x7f894815ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 0x7f894815ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38696/0 (socket says 172.21.15.26:38696) 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894da00640 1 -- 172.21.15.26:0/3906151949 learned_addr learned my addr 172.21.15.26:0/3906151949 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894da00640 1 -- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89480cf9a0 msgr2=0x7f89480ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894e400640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f894815d4b0 0x7f89480cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894d000640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89480cf9a0 0x7f89480ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894da00640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89480cf9a0 0x7f89480ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894da00640 1 -- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f894815d4b0 msgr2=0x7f89480cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894da00640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f894815d4b0 0x7f89480cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894da00640 1 -- 172.21.15.26:0/3906151949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8940009a70 con 0x7f89480b31d0 2024-01-15T18:07:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894e400640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f894815d4b0 0x7f89480cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.194+0000 7f894da00640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 0x7f894815ba50 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f893800c430 tx=0x7f893800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.195+0000 7f8945600640 1 -- 172.21.15.26:0/3906151949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89380074e0 con 0x7f89480b31d0 2024-01-15T18:07:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.195+0000 7f8945600640 1 -- 172.21.15.26:0/3906151949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8938007660 con 0x7f89480b31d0 2024-01-15T18:07:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.195+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89480caf90 con 0x7f89480b31d0 2024-01-15T18:07:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.195+0000 7f8945600640 1 -- 172.21.15.26:0/3906151949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938004a20 con 0x7f89480b31d0 2024-01-15T18:07:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.195+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89480cb550 con 0x7f89480b31d0 2024-01-15T18:07:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.196+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8948067450 con 0x7f89480b31d0 2024-01-15T18:07:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.197+0000 7f8945600640 1 -- 172.21.15.26:0/3906151949 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f8938004b80 con 0x7f89480b31d0 2024-01-15T18:07:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.198+0000 7f8945600640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006bf20 0x7f892006e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.198+0000 7f8945600640 1 -- 172.21.15.26:0/3906151949 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f8938088670 con 0x7f89480b31d0 2024-01-15T18:07:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.199+0000 7f894d000640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006bf20 0x7f892006e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.200+0000 7f894d000640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006bf20 0x7f892006e3e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f89480cc9c0 tx=0x7f8930008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:25.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.203+0000 7f8945600640 1 -- 172.21.15.26:0/3906151949 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8938054cb0 con 0x7f89480b31d0 2024-01-15T18:07:25.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.440+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41818", "target": ["mon-mgr", ""]}) v1 -- 0x7f89480d02a0 con 0x7f892006bf20 2024-01-15T18:07:25.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.463+0000 7f8945600640 1 -- 172.21.15.26:0/3906151949 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f89480d02a0 con 0x7f892006bf20 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:25.467 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:25.468 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:25.468 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006bf20 msgr2=0x7f892006e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006bf20 0x7f892006e3e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f89480cc9c0 tx=0x7f8930008040 comp rx=0 tx=0).stop 2024-01-15T18:07:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 msgr2=0x7f894815ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 0x7f894815ba50 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f893800c430 tx=0x7f893800c8f0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 shutdown_connections 2024-01-15T18:07:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89480cf9a0 0x7f89480ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006bf20 0x7f892006e3e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f894815d4b0 0x7f89480cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 --2- 172.21.15.26:0/3906151949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89480b31d0 0x7f894815ba50 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 >> 172.21.15.26:0/3906151949 conn(0x7f89480d3000 msgr2=0x7f894806c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.465+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 shutdown_connections 2024-01-15T18:07:25.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.466+0000 7f894ee00640 1 -- 172.21.15.26:0/3906151949 wait complete. 2024-01-15T18:07:25.489 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:25.489 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41819 2024-01-15T18:07:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.709+0000 7f2208400640 1 Processor -- start 2024-01-15T18:07:25.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.709+0000 7f2208400640 1 -- start start 2024-01-15T18:07:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.711+0000 7f2208400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f220013df80 0x7f220015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.711+0000 7f2208400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f220013e360 0x7f22000cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.711+0000 7f2208400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220006c470 con 0x7f220015f9c0 2024-01-15T18:07:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.711+0000 7f2208400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220006c750 con 0x7f220013df80 2024-01-15T18:07:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.711+0000 7f2208400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220006ca30 con 0x7f220013e360 2024-01-15T18:07:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.711+0000 7f2205a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f220013df80 0x7f220015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.711+0000 7f2206400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f220015f9c0 0x7f22001408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53274/0 (socket says 172.21.15.26:53274) 2024-01-15T18:07:25.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.711+0000 7f2206400640 1 -- 172.21.15.26:0/1513380666 learned_addr learned my addr 172.21.15.26:0/1513380666 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.712+0000 7f2205000640 1 --2- 172.21.15.26:0/1513380666 >> v2:172.21.15.124:3300/0 conn(0x7f220013e360 0x7f22000cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.712+0000 7f2205a00640 1 -- 172.21.15.26:0/1513380666 >> v2:172.21.15.124:3300/0 conn(0x7f220013e360 msgr2=0x7f22000cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.712+0000 7f2205a00640 1 --2- 172.21.15.26:0/1513380666 >> v2:172.21.15.124:3300/0 conn(0x7f220013e360 0x7f22000cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.712+0000 7f2205a00640 1 -- 172.21.15.26:0/1513380666 >> v1:172.21.15.26:6789/0 conn(0x7f220015f9c0 legacy=0x7f22001408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.712+0000 7f2205a00640 1 -- 172.21.15.26:0/1513380666 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220006cd10 con 0x7f220013df80 2024-01-15T18:07:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.713+0000 7f2205a00640 1 --2- 172.21.15.26:0/1513380666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013df80 0x7f220015bee0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f220006ded0 tx=0x7f21f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261ddfe7ce36d9d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.713+0000 7f21ffe00640 1 -- 172.21.15.26:0/1513380666 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0031830 con 0x7f220013df80 2024-01-15T18:07:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.714+0000 7f21ffe00640 1 -- 172.21.15.26:0/1513380666 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f0031990 con 0x7f220013df80 2024-01-15T18:07:25.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.714+0000 7f21ffe00640 1 -- 172.21.15.26:0/1513380666 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0031d20 con 0x7f220013df80 2024-01-15T18:07:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.714+0000 7f2208400640 1 -- 172.21.15.26:0/1513380666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013df80 msgr2=0x7f220015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.714+0000 7f2208400640 1 --2- 172.21.15.26:0/1513380666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013df80 0x7f220015bee0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f220006ded0 tx=0x7f21f002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.714+0000 7f2208400640 1 -- 172.21.15.26:0/1513380666 shutdown_connections 2024-01-15T18:07:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.714+0000 7f2208400640 1 --2- 172.21.15.26:0/1513380666 >> v2:172.21.15.124:3300/0 conn(0x7f220013e360 0x7f22000cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.714+0000 7f2208400640 1 --2- 172.21.15.26:0/1513380666 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013df80 0x7f220015bee0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.714+0000 7f2208400640 1 -- 172.21.15.26:0/1513380666 >> 172.21.15.26:0/1513380666 conn(0x7f220006a800 msgr2=0x7f220006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.714+0000 7f2208400640 1 -- 172.21.15.26:0/1513380666 shutdown_connections 2024-01-15T18:07:25.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.715+0000 7f2208400640 1 -- 172.21.15.26:0/1513380666 wait complete. 2024-01-15T18:07:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.717+0000 7f2208400640 1 Processor -- start 2024-01-15T18:07:25.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.717+0000 7f2208400640 1 -- start start 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.718+0000 7f2208400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220013df80 0x7f2200070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2208400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013e360 0x7f220014aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2208400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f220015f9c0 0x7f22000a9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2208400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2200071f70 con 0x7f220013e360 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2208400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2200071360 con 0x7f220015f9c0 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2208400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2200071640 con 0x7f220013df80 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2205000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013e360 0x7f220014aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2205000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013e360 0x7f220014aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38722/0 (socket says 172.21.15.26:38722) 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2205000640 1 -- 172.21.15.26:0/339709284 learned_addr learned my addr 172.21.15.26:0/339709284 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2206400640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f220015f9c0 0x7f22000a9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2205000640 1 -- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220013df80 msgr2=0x7f2200070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2205000640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220013df80 0x7f2200070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2205000640 1 -- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f220015f9c0 msgr2=0x7f22000a9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2205000640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f220015f9c0 0x7f22000a9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.719+0000 7f2205000640 1 -- 172.21.15.26:0/339709284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f0009400 con 0x7f220013e360 2024-01-15T18:07:25.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.720+0000 7f2205000640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013e360 0x7f220014aed0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f21e800d7d0 tx=0x7f21e800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:25.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.720+0000 7f21fd600640 1 -- 172.21.15.26:0/339709284 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8018070 con 0x7f220013e360 2024-01-15T18:07:25.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.720+0000 7f21fd600640 1 -- 172.21.15.26:0/339709284 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e8004020 con 0x7f220013e360 2024-01-15T18:07:25.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.720+0000 7f21fd600640 1 -- 172.21.15.26:0/339709284 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e80041b0 con 0x7f220013e360 2024-01-15T18:07:25.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.720+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2200071920 con 0x7f220013e360 2024-01-15T18:07:25.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.720+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220014b410 con 0x7f220013e360 2024-01-15T18:07:25.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.721+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22000675d0 con 0x7f220013e360 2024-01-15T18:07:25.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.722+0000 7f21fd600640 1 -- 172.21.15.26:0/339709284 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f21e8004310 con 0x7f220013e360 2024-01-15T18:07:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.724+0000 7f21fd600640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21d806bf70 0x7f21d806e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.724+0000 7f21fd600640 1 -- 172.21.15.26:0/339709284 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f21e80884e0 con 0x7f220013e360 2024-01-15T18:07:25.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.725+0000 7f2205a00640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21d806bf70 0x7f21d806e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:25.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.726+0000 7f2205a00640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21d806bf70 0x7f21d806e430 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f21f0036000 tx=0x7f21f0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:25.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.730+0000 7f21fd600640 1 -- 172.21.15.26:0/339709284 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21e80bde90 con 0x7f220013e360 2024-01-15T18:07:25.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.966+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41819", "target": ["mon-mgr", ""]}) v1 -- 0x7f22000ce850 con 0x7f21d806bf70 2024-01-15T18:07:25.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.986+0000 7f21fd600640 1 -- 172.21.15.26:0/339709284 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f21f0031b00 con 0x7f21d806bf70 2024-01-15T18:07:25.990 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:25.990 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:25.990 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:25.991 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.989+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21d806bf70 msgr2=0x7f21d806e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.989+0000 7f2208400640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21d806bf70 0x7f21d806e430 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f21f0036000 tx=0x7f21f0030000 comp rx=0 tx=0).stop 2024-01-15T18:07:25.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.989+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013e360 msgr2=0x7f220014aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.989+0000 7f2208400640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013e360 0x7f220014aed0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f21e800d7d0 tx=0x7f21e800dc90 comp rx=0 tx=0).stop 2024-01-15T18:07:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.990+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 shutdown_connections 2024-01-15T18:07:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.990+0000 7f2208400640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f21d806bf70 0x7f21d806e430 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.990+0000 7f2208400640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f220015f9c0 0x7f22000a9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.990+0000 7f2208400640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f220013e360 0x7f220014aed0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.990+0000 7f2208400640 1 --2- 172.21.15.26:0/339709284 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f220013df80 0x7f2200070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.990+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 >> 172.21.15.26:0/339709284 conn(0x7f220006a800 msgr2=0x7f2200155620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.990+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 shutdown_connections 2024-01-15T18:07:25.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:25.990+0000 7f2208400640 1 -- 172.21.15.26:0/339709284 wait complete. 2024-01-15T18:07:26.014 INFO:tasks.cephfs.test_volumes:expected clone state = ('canceled', 'complete') 2024-01-15T18:07:26.014 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone41820 2024-01-15T18:07:26.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.241+0000 7f96ef400640 1 Processor -- start 2024-01-15T18:07:26.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.242+0000 7f96ef400640 1 -- start start 2024-01-15T18:07:26.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.243+0000 7f96ef400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f96e815d4b0 0x7f96e80d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.244+0000 7f96ef400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f96e80b41e0 0x7f96e80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.244+0000 7f96ef400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80d0320 con 0x7f96e80cf9a0 2024-01-15T18:07:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.244+0000 7f96ef400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80b2730 con 0x7f96e80b41e0 2024-01-15T18:07:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.244+0000 7f96ef400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80d1ec0 con 0x7f96e815d4b0 2024-01-15T18:07:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.244+0000 7f96eea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f96e80b41e0 0x7f96e80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.244+0000 7f96eea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f96e80b41e0 0x7f96e80cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38728/0 (socket says 172.21.15.26:38728) 2024-01-15T18:07:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.244+0000 7f96eea00640 1 -- 172.21.15.26:0/3332394833 learned_addr learned my addr 172.21.15.26:0/3332394833 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:26.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.244+0000 7f96ed600640 1 --1- 172.21.15.26:0/3332394833 >> v1:172.21.15.26:6789/0 conn(0x7f96e80cf9a0 0x7f96e80cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53288/0 (socket says 172.21.15.26:53288) 2024-01-15T18:07:26.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.244+0000 7f96ee000640 1 --2- 172.21.15.26:0/3332394833 >> v2:172.21.15.124:3300/0 conn(0x7f96e815d4b0 0x7f96e80d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.245+0000 7f96eea00640 1 -- 172.21.15.26:0/3332394833 >> v2:172.21.15.124:3300/0 conn(0x7f96e815d4b0 msgr2=0x7f96e80d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:26.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.245+0000 7f96eea00640 1 --2- 172.21.15.26:0/3332394833 >> v2:172.21.15.124:3300/0 conn(0x7f96e815d4b0 0x7f96e80d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.245+0000 7f96eea00640 1 -- 172.21.15.26:0/3332394833 >> v1:172.21.15.26:6789/0 conn(0x7f96e80cf9a0 legacy=0x7f96e80cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.245+0000 7f96eea00640 1 -- 172.21.15.26:0/3332394833 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80b29d0 con 0x7f96e80b41e0 2024-01-15T18:07:26.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.245+0000 7f96ecc00640 1 -- 172.21.15.26:0/3332394833 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346233561 0 0) 0x7f96e80d0320 con 0x7f96e80cf9a0 2024-01-15T18:07:26.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.246+0000 7f96eea00640 1 --2- 172.21.15.26:0/3332394833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e80b41e0 0x7f96e80cfde0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f96e808b5b0 tx=0x7f96e002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d539dcf23ecd60ce server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:26.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.246+0000 7f96ecc00640 1 -- 172.21.15.26:0/3332394833 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e002cb10 con 0x7f96e80b41e0 2024-01-15T18:07:26.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.246+0000 7f96ecc00640 1 -- 172.21.15.26:0/3332394833 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e002cc70 con 0x7f96e80b41e0 2024-01-15T18:07:26.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.246+0000 7f96ecc00640 1 -- 172.21.15.26:0/3332394833 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e00343f0 con 0x7f96e80b41e0 2024-01-15T18:07:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.247+0000 7f96ef400640 1 -- 172.21.15.26:0/3332394833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e80b41e0 msgr2=0x7f96e80cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.247+0000 7f96ef400640 1 --2- 172.21.15.26:0/3332394833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e80b41e0 0x7f96e80cfde0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f96e808b5b0 tx=0x7f96e002bf70 comp rx=0 tx=0).stop 2024-01-15T18:07:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.247+0000 7f96ef400640 1 -- 172.21.15.26:0/3332394833 shutdown_connections 2024-01-15T18:07:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.247+0000 7f96ef400640 1 --2- 172.21.15.26:0/3332394833 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e80b41e0 0x7f96e80cfde0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.247+0000 7f96ef400640 1 --2- 172.21.15.26:0/3332394833 >> v2:172.21.15.124:3300/0 conn(0x7f96e815d4b0 0x7f96e80d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.247+0000 7f96ef400640 1 -- 172.21.15.26:0/3332394833 >> 172.21.15.26:0/3332394833 conn(0x7f96e80d3000 msgr2=0x7f96e80d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.247+0000 7f96ef400640 1 -- 172.21.15.26:0/3332394833 shutdown_connections 2024-01-15T18:07:26.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.247+0000 7f96ef400640 1 -- 172.21.15.26:0/3332394833 wait complete. 2024-01-15T18:07:26.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.250+0000 7f96ef400640 1 Processor -- start 2024-01-15T18:07:26.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.250+0000 7f96ef400640 1 -- start start 2024-01-15T18:07:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.251+0000 7f96ef400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96e80b41e0 0x7f96e8155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.251+0000 7f96ef400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96e80cf9a0 0x7f96e8155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96ef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e815d4b0 0x7f96e8155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96ef400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8159ca0 con 0x7f96e815d4b0 2024-01-15T18:07:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96ef400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8157590 con 0x7f96e80b41e0 2024-01-15T18:07:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96ef400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e8157870 con 0x7f96e80cf9a0 2024-01-15T18:07:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96eea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e815d4b0 0x7f96e8155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96eea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e815d4b0 0x7f96e8155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38744/0 (socket says 172.21.15.26:38744) 2024-01-15T18:07:26.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96eea00640 1 -- 172.21.15.26:0/2059046635 learned_addr learned my addr 172.21.15.26:0/2059046635 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96ee000640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96e80b41e0 0x7f96e8155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96ed600640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96e80cf9a0 0x7f96e8155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96eea00640 1 -- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96e80cf9a0 msgr2=0x7f96e8155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96eea00640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96e80cf9a0 0x7f96e8155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96eea00640 1 -- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96e80b41e0 msgr2=0x7f96e8155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96eea00640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96e80b41e0 0x7f96e8155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.252+0000 7f96eea00640 1 -- 172.21.15.26:0/2059046635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0009b30 con 0x7f96e815d4b0 2024-01-15T18:07:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.253+0000 7f96eea00640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e815d4b0 0x7f96e8155cb0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f96e0036040 tx=0x7f96e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.253+0000 7f96e6000640 1 -- 172.21.15.26:0/2059046635 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0009510 con 0x7f96e815d4b0 2024-01-15T18:07:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.253+0000 7f96e6000640 1 -- 172.21.15.26:0/2059046635 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e003e450 con 0x7f96e815d4b0 2024-01-15T18:07:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.253+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e815da10 con 0x7f96e815d4b0 2024-01-15T18:07:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.253+0000 7f96e6000640 1 -- 172.21.15.26:0/2059046635 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e003e800 con 0x7f96e815d4b0 2024-01-15T18:07:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.253+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e8157b50 con 0x7f96e815d4b0 2024-01-15T18:07:26.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.255+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e815a120 con 0x7f96e815d4b0 2024-01-15T18:07:26.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.256+0000 7f96e6000640 1 -- 172.21.15.26:0/2059046635 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f96e003e960 con 0x7f96e815d4b0 2024-01-15T18:07:26.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.257+0000 7f96e6000640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96c006bf20 0x7f96c006e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.257+0000 7f96e6000640 1 -- 172.21.15.26:0/2059046635 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f96e00ab5a0 con 0x7f96e815d4b0 2024-01-15T18:07:26.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.258+0000 7f96ee000640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96c006bf20 0x7f96c006e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.259+0000 7f96ee000640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96c006bf20 0x7f96c006e3e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f96d8006fd0 tx=0x7f96d8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:26.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.261+0000 7f96e6000640 1 -- 172.21.15.26:0/2059046635 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96e0077d60 con 0x7f96e815d4b0 2024-01-15T18:07:26.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.505+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone41820", "target": ["mon-mgr", ""]}) v1 -- 0x7f96e8159b20 con 0x7f96c006bf20 2024-01-15T18:07:26.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.528+0000 7f96e6000640 1 -- 172.21.15.26:0/2059046635 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f96d800b5e0 con 0x7f96c006bf20 2024-01-15T18:07:26.532 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:07:26.532 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:07:26.532 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:26.533 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96c006bf20 msgr2=0x7f96c006e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96c006bf20 0x7f96c006e3e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f96d8006fd0 tx=0x7f96d8008040 comp rx=0 tx=0).stop 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e815d4b0 msgr2=0x7f96e8155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e815d4b0 0x7f96e8155cb0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f96e0036040 tx=0x7f96e0004a40 comp rx=0 tx=0).stop 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 shutdown_connections 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f96e80cf9a0 0x7f96e8155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96c006bf20 0x7f96c006e3e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f96e815d4b0 0x7f96e8155cb0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 --2- 172.21.15.26:0/2059046635 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f96e80b41e0 0x7f96e8155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 >> 172.21.15.26:0/2059046635 conn(0x7f96e80d3000 msgr2=0x7f96e80d3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.531+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 shutdown_connections 2024-01-15T18:07:26.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.532+0000 7f96ef400640 1 -- 172.21.15.26:0/2059046635 wait complete. 2024-01-15T18:07:26.556 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:07:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.776+0000 7f9afe800640 1 Processor -- start 2024-01-15T18:07:26.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.776+0000 7f9afe800640 1 -- start start 2024-01-15T18:07:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.778+0000 7f9afe800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9af815e3b0 0x7f9af8164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.778+0000 7f9afe800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80b1ee0 con 0x7f9af8143ae0 2024-01-15T18:07:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.778+0000 7f9afe800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80b2780 con 0x7f9af80cf9a0 2024-01-15T18:07:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.778+0000 7f9afe800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af815e990 con 0x7f9af815e3b0 2024-01-15T18:07:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.778+0000 7f9af7400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9af8143ae0 0x7f9af80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53300/0 (socket says 172.21.15.26:53300) 2024-01-15T18:07:26.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.778+0000 7f9af7400640 1 -- 172.21.15.26:0/2896418962 learned_addr learned my addr 172.21.15.26:0/2896418962 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af7e00640 1 --2- 172.21.15.26:0/2896418962 >> v2:172.21.15.124:3300/0 conn(0x7f9af815e3b0 0x7f9af8164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949303138 0 0) 0x7f9af80b1ee0 con 0x7f9af8143ae0 2024-01-15T18:07:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ad8003520 con 0x7f9af8143ae0 2024-01-15T18:07:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1724582309 0 0) 0x7f9ad8003520 con 0x7f9af8143ae0 2024-01-15T18:07:26.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 >> v2:172.21.15.124:3300/0 conn(0x7f9af815e3b0 msgr2=0x7f9af8164de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 --2- 172.21.15.26:0/2896418962 >> v2:172.21.15.124:3300/0 conn(0x7f9af815e3b0 0x7f9af8164de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 >> v1:172.21.15.124:6789/0 conn(0x7f9af80cf9a0 legacy=0x7f9af815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80b2130 con 0x7f9af8143ae0 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9ae8001670 con 0x7f9af8143ae0 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 >> v1:172.21.15.26:6789/0 conn(0x7f9af8143ae0 legacy=0x7f9af80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad80039c0 0x7f9ad8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9ad8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad8005180 0x7f9ad800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815e990 con 0x7f9ad80047c0 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad800c080 con 0x7f9ad8005180 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.779+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad800c240 con 0x7f9ad80039c0 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9aefe00640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9ad8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9af7400640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad80039c0 0x7f9ad8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9af7e00640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad8005180 0x7f9ad800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9aefe00640 1 -- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad80039c0 msgr2=0x7f9ad8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:26.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9aefe00640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad80039c0 0x7f9ad8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9aefe00640 1 -- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad8005180 msgr2=0x7f9ad800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:26.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9aefe00640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad8005180 0x7f9ad800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9aefe00640 1 -- 172.21.15.26:0/2896418962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad8003520 con 0x7f9ad80047c0 2024-01-15T18:07:26.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9af7e00640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ad8005180 0x7f9ad800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:26.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9af7e00640 1 -- 172.21.15.26:0/2896418962 reap_dead start 2024-01-15T18:07:26.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.780+0000 7f9af7400640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ad80039c0 0x7f9ad8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:26.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.781+0000 7f9aefe00640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9ad8004c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0001750 tx=0x7f9ad80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3273d835dd5de10 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:26.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae002f040 con 0x7f9ad80047c0 2024-01-15T18:07:26.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae00052d0 con 0x7f9ad80047c0 2024-01-15T18:07:26.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9af6a00640 1 -- 172.21.15.26:0/2896418962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0003830 con 0x7f9ad80047c0 2024-01-15T18:07:26.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9afe800640 1 -- 172.21.15.26:0/2896418962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 msgr2=0x7f9ad8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9afe800640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9ad8004c40 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0001750 tx=0x7f9ad80057f0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9afe800640 1 -- 172.21.15.26:0/2896418962 shutdown_connections 2024-01-15T18:07:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9afe800640 1 --2- 172.21.15.26:0/2896418962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9ad8004c40 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9afe800640 1 -- 172.21.15.26:0/2896418962 >> 172.21.15.26:0/2896418962 conn(0x7f9af806a800 msgr2=0x7f9af806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9afe800640 1 -- 172.21.15.26:0/2896418962 shutdown_connections 2024-01-15T18:07:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.782+0000 7f9afe800640 1 -- 172.21.15.26:0/2896418962 wait complete. 2024-01-15T18:07:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.784+0000 7f9afe800640 1 Processor -- start 2024-01-15T18:07:26.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.785+0000 7f9afe800640 1 -- start start 2024-01-15T18:07:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9afe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9af8159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9afe800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af815b530 0x7f9af80c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9afe800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af815b9b0 0x7f9af80c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9afe800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815cc00 con 0x7f9ad80047c0 2024-01-15T18:07:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9afe800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8159eb0 con 0x7f9af815b9b0 2024-01-15T18:07:26.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9afe800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815bf80 con 0x7f9af815b530 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9af7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9af8159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9af7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9af8159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38764/0 (socket says 172.21.15.26:38764) 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9af7400640 1 -- 172.21.15.26:0/1561215749 learned_addr learned my addr 172.21.15.26:0/1561215749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9af7400640 1 -- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af815b530 msgr2=0x7f9af80c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9af7400640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af815b530 0x7f9af80c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9af7e00640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af815b9b0 0x7f9af80c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9af7400640 1 -- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af815b9b0 msgr2=0x7f9af80c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9af7400640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af815b9b0 0x7f9af80c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.786+0000 7f9af7400640 1 -- 172.21.15.26:0/1561215749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af815c260 con 0x7f9ad80047c0 2024-01-15T18:07:26.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.787+0000 7f9af7400640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9af8159970 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800d7a0 tx=0x7f9ae800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.787+0000 7f9aef400640 1 -- 172.21.15.26:0/1561215749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80072d0 con 0x7f9ad80047c0 2024-01-15T18:07:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.787+0000 7f9aef400640 1 -- 172.21.15.26:0/1561215749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8007430 con 0x7f9ad80047c0 2024-01-15T18:07:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.787+0000 7f9aef400640 1 -- 172.21.15.26:0/1561215749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80047e0 con 0x7f9ad80047c0 2024-01-15T18:07:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.787+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af815c540 con 0x7f9ad80047c0 2024-01-15T18:07:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.787+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af81422d0 con 0x7f9ad80047c0 2024-01-15T18:07:26.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.788+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af80675d0 con 0x7f9ad80047c0 2024-01-15T18:07:26.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.789+0000 7f9aef400640 1 -- 172.21.15.26:0/1561215749 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f9ae8004940 con 0x7f9ad80047c0 2024-01-15T18:07:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.790+0000 7f9aef400640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ad006bf20 0x7f9ad006e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.790+0000 7f9aef400640 1 -- 172.21.15.26:0/1561215749 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f9ae8087ff0 con 0x7f9ad80047c0 2024-01-15T18:07:26.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.791+0000 7f9aefe00640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ad006bf20 0x7f9ad006e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:26.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.792+0000 7f9aefe00640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ad006bf20 0x7f9ad006e3e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0036000 tx=0x7f9ae0032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:26.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:26.795+0000 7f9aef400640 1 -- 172.21.15.26:0/1561215749 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ae8054690 con 0x7f9ad80047c0 2024-01-15T18:07:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.013+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9af815dca0 con 0x7f9ad006bf20 2024-01-15T18:07:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.016+0000 7f9aef400640 1 -- 172.21.15.26:0/1561215749 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f9ae0005440 con 0x7f9ad006bf20 2024-01-15T18:07:27.020 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:07:27.020 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.020 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:07:27.020 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:27.021 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:07:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ad006bf20 msgr2=0x7f9ad006e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ad006bf20 0x7f9ad006e3e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0036000 tx=0x7f9ae0032000 comp rx=0 tx=0).stop 2024-01-15T18:07:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 msgr2=0x7f9af8159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9af8159970 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800d7a0 tx=0x7f9ae800dc60 comp rx=0 tx=0).stop 2024-01-15T18:07:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 shutdown_connections 2024-01-15T18:07:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9af815b530 0x7f9af80c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9ad006bf20 0x7f9ad006e3e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9af815b9b0 0x7f9af80c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 --2- 172.21.15.26:0/1561215749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ad80047c0 0x7f9af8159970 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 >> 172.21.15.26:0/1561215749 conn(0x7f9af806a800 msgr2=0x7f9af80ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 shutdown_connections 2024-01-15T18:07:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.019+0000 7f9afe800640 1 -- 172.21.15.26:0/1561215749 wait complete. 2024-01-15T18:07:27.043 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:07:27.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.266+0000 7f4866400640 1 Processor -- start 2024-01-15T18:07:27.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.266+0000 7f4866400640 1 -- start start 2024-01-15T18:07:27.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.268+0000 7f4866400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 0x7f486013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.268+0000 7f4866400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f486013fae0 0x7f48600cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.268+0000 7f4866400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48600ce440 con 0x7f48600cdc20 2024-01-15T18:07:27.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.268+0000 7f4866400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486013ff60 con 0x7f486013fae0 2024-01-15T18:07:27.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.268+0000 7f4866400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48600ce720 con 0x7f486013cf10 2024-01-15T18:07:27.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.268+0000 7f4857c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f486013fae0 0x7f48600cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.268+0000 7f4857c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f486013fae0 0x7f48600cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38766/0 (socket says 172.21.15.26:38766) 2024-01-15T18:07:27.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.268+0000 7f4857c00640 1 -- 172.21.15.26:0/4237333755 learned_addr learned my addr 172.21.15.26:0/4237333755 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.268+0000 7f4865000640 1 --2- 172.21.15.26:0/4237333755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 0x7f486013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.269+0000 7f4857c00640 1 -- 172.21.15.26:0/4237333755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 msgr2=0x7f486013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.269+0000 7f4857c00640 1 --2- 172.21.15.26:0/4237333755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 0x7f486013f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.269+0000 7f4857c00640 1 -- 172.21.15.26:0/4237333755 >> v1:172.21.15.124:6789/0 conn(0x7f48600cdc20 legacy=0x7f486015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.269+0000 7f4857c00640 1 -- 172.21.15.26:0/4237333755 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48600cea00 con 0x7f486013fae0 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.269+0000 7f4865000640 1 --2- 172.21.15.26:0/4237333755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 0x7f486013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.270+0000 7f4857c00640 1 --2- 172.21.15.26:0/4237333755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 0x7f48600cd6e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4848009870 tx=0x7f484802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3b6f0c3302f4bb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.270+0000 7f485fe00640 1 -- 172.21.15.26:0/4237333755 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848031640 con 0x7f486013fae0 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.270+0000 7f485fe00640 1 -- 172.21.15.26:0/4237333755 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48480317a0 con 0x7f486013fae0 2024-01-15T18:07:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.270+0000 7f485fe00640 1 -- 172.21.15.26:0/4237333755 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848031b30 con 0x7f486013fae0 2024-01-15T18:07:27.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.271+0000 7f4866400640 1 -- 172.21.15.26:0/4237333755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 msgr2=0x7f48600cd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.271+0000 7f4866400640 1 --2- 172.21.15.26:0/4237333755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 0x7f48600cd6e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4848009870 tx=0x7f484802b8f0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.271+0000 7f4866400640 1 -- 172.21.15.26:0/4237333755 shutdown_connections 2024-01-15T18:07:27.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.271+0000 7f4866400640 1 --2- 172.21.15.26:0/4237333755 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 0x7f48600cd6e0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.271+0000 7f4866400640 1 --2- 172.21.15.26:0/4237333755 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 0x7f486013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.271+0000 7f4866400640 1 -- 172.21.15.26:0/4237333755 >> 172.21.15.26:0/4237333755 conn(0x7f486006a7d0 msgr2=0x7f486006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:27.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.271+0000 7f4866400640 1 -- 172.21.15.26:0/4237333755 shutdown_connections 2024-01-15T18:07:27.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.271+0000 7f4866400640 1 -- 172.21.15.26:0/4237333755 wait complete. 2024-01-15T18:07:27.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.273+0000 7f4866400640 1 Processor -- start 2024-01-15T18:07:27.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.273+0000 7f4866400640 1 -- start start 2024-01-15T18:07:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4866400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48600cdc20 0x7f486013eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4866400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 0x7f4860135ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4866400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 0x7f486013dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4866400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4860136400 con 0x7f486013fae0 2024-01-15T18:07:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4866400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486013e100 con 0x7f48600cdc20 2024-01-15T18:07:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4866400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486013e2f0 con 0x7f486013cf10 2024-01-15T18:07:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4865a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 0x7f486013dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4865a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 0x7f486013dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38778/0 (socket says 172.21.15.26:38778) 2024-01-15T18:07:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4865a00640 1 -- 172.21.15.26:0/540041972 learned_addr learned my addr 172.21.15.26:0/540041972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4865000640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48600cdc20 0x7f486013eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4857c00640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 0x7f4860135ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4865a00640 1 -- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 msgr2=0x7f4860135ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.274+0000 7f4865a00640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 0x7f4860135ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.275+0000 7f4865a00640 1 -- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48600cdc20 msgr2=0x7f486013eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.275+0000 7f4865a00640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48600cdc20 0x7f486013eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.275+0000 7f4865a00640 1 -- 172.21.15.26:0/540041972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48480094d0 con 0x7f486013fae0 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.275+0000 7f4865a00640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 0x7f486013dbc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f485800c410 tx=0x7f485800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.275+0000 7f485d600640 1 -- 172.21.15.26:0/540041972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858011790 con 0x7f486013fae0 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.275+0000 7f485d600640 1 -- 172.21.15.26:0/540041972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48580118f0 con 0x7f486013fae0 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.275+0000 7f485d600640 1 -- 172.21.15.26:0/540041972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858011ce0 con 0x7f486013fae0 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.275+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4860070c10 con 0x7f486013fae0 2024-01-15T18:07:27.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.275+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48600711d0 con 0x7f486013fae0 2024-01-15T18:07:27.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.277+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48600cc3d0 con 0x7f486013fae0 2024-01-15T18:07:27.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.277+0000 7f485d600640 1 -- 172.21.15.26:0/540041972 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f485801c460 con 0x7f486013fae0 2024-01-15T18:07:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.278+0000 7f485d600640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f483806bf50 0x7f483806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.279+0000 7f485d600640 1 -- 172.21.15.26:0/540041972 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f48580884e0 con 0x7f486013fae0 2024-01-15T18:07:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.279+0000 7f4865000640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f483806bf50 0x7f483806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.280+0000 7f4865000640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f483806bf50 0x7f483806e410 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f48500075b0 tx=0x7f4850007540 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.283+0000 7f485d600640 1 -- 172.21.15.26:0/540041972 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4858054b20 con 0x7f486013fae0 2024-01-15T18:07:27.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.508+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48600ce0b0 con 0x7f483806bf50 2024-01-15T18:07:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.553+0000 7f485d600640 1 -- 172.21.15.26:0/540041972 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1009 (secure 0 0 0) 0x7f4850007cf0 con 0x7f483806bf50 2024-01-15T18:07:27.557 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:07:27.557 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41817" 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41810" 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41811" 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.558 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41813" 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone4182" 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone4189" 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sv1" 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.559 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone4187" 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41818" 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41819" 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone4185" 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41815" 2024-01-15T18:07:27.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41814" 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41816" 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone4183" 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone4181" 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.561 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone4186" 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41812" 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone4188" 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone41820" 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone4184" 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:27.562 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:07:27.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f483806bf50 msgr2=0x7f483806e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f483806bf50 0x7f483806e410 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f48500075b0 tx=0x7f4850007540 comp rx=0 tx=0).stop 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 msgr2=0x7f486013dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 0x7f486013dbc0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f485800c410 tx=0x7f485800c8d0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 shutdown_connections 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f486013cf10 0x7f4860135ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f483806bf50 0x7f483806e410 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f486013fae0 0x7f486013dbc0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 --2- 172.21.15.26:0/540041972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f48600cdc20 0x7f486013eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.556+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 >> 172.21.15.26:0/540041972 conn(0x7f486006a7d0 msgr2=0x7f486015a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.557+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 shutdown_connections 2024-01-15T18:07:27.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.557+0000 7f4866400640 1 -- 172.21.15.26:0/540041972 wait complete. 2024-01-15T18:07:27.582 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41817 2024-01-15T18:07:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.807+0000 7f6787400640 1 Processor -- start 2024-01-15T18:07:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.807+0000 7f6787400640 1 -- start start 2024-01-15T18:07:27.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.809+0000 7f6787400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f678015afb0 0x7f678013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.809+0000 7f6787400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f678013ff30 con 0x7f678013fae0 2024-01-15T18:07:27.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.809+0000 7f6787400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67800cdc40 con 0x7f678013cfa0 2024-01-15T18:07:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.809+0000 7f6787400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67800cdec0 con 0x7f678015afb0 2024-01-15T18:07:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.810+0000 7f6785600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f678015afb0 0x7f678013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.810+0000 7f6786a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f678013fae0 0x7f67800b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53314/0 (socket says 172.21.15.26:53314) 2024-01-15T18:07:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.810+0000 7f6786a00640 1 -- 172.21.15.26:0/3003261841 learned_addr learned my addr 172.21.15.26:0/3003261841 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.810+0000 7f6784c00640 1 -- 172.21.15.26:0/3003261841 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364341024 0 0) 0x7f676c001110 con 0x7f678013fae0 2024-01-15T18:07:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.810+0000 7f6784c00640 1 -- 172.21.15.26:0/3003261841 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f675c003520 con 0x7f678013fae0 2024-01-15T18:07:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.810+0000 7f6785600640 1 -- 172.21.15.26:0/3003261841 >> v1:172.21.15.124:6789/0 conn(0x7f678013cfa0 legacy=0x7f67800cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.810+0000 7f6785600640 1 -- 172.21.15.26:0/3003261841 >> v1:172.21.15.26:6789/0 conn(0x7f678013fae0 legacy=0x7f67800b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.810+0000 7f6785600640 1 -- 172.21.15.26:0/3003261841 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67800ce1a0 con 0x7f678015afb0 2024-01-15T18:07:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.812+0000 7f6785600640 1 --2- 172.21.15.26:0/3003261841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678015afb0 0x7f678013f5a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f67640097a0 tx=0x7f676402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3619f8c3edceef73 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:27.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.812+0000 7f6784c00640 1 -- 172.21.15.26:0/3003261841 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764031830 con 0x7f678015afb0 2024-01-15T18:07:27.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.812+0000 7f6784c00640 1 -- 172.21.15.26:0/3003261841 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6764031990 con 0x7f678015afb0 2024-01-15T18:07:27.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.812+0000 7f6784c00640 1 -- 172.21.15.26:0/3003261841 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764031d20 con 0x7f678015afb0 2024-01-15T18:07:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.813+0000 7f6787400640 1 -- 172.21.15.26:0/3003261841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678015afb0 msgr2=0x7f678013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.813+0000 7f6787400640 1 --2- 172.21.15.26:0/3003261841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678015afb0 0x7f678013f5a0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f67640097a0 tx=0x7f676402b8d0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.813+0000 7f6787400640 1 -- 172.21.15.26:0/3003261841 shutdown_connections 2024-01-15T18:07:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.813+0000 7f6787400640 1 --2- 172.21.15.26:0/3003261841 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678015afb0 0x7f678013f5a0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.813+0000 7f6787400640 1 -- 172.21.15.26:0/3003261841 >> 172.21.15.26:0/3003261841 conn(0x7f678006a800 msgr2=0x7f678006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.813+0000 7f6787400640 1 -- 172.21.15.26:0/3003261841 shutdown_connections 2024-01-15T18:07:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.813+0000 7f6787400640 1 -- 172.21.15.26:0/3003261841 wait complete. 2024-01-15T18:07:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.815+0000 7f6787400640 1 Processor -- start 2024-01-15T18:07:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.815+0000 7f6787400640 1 -- start start 2024-01-15T18:07:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.816+0000 7f6787400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678013cfa0 0x7f67801f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.816+0000 7f6787400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f678013fae0 0x7f67801f4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.816+0000 7f6787400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f678015afb0 0x7f678013e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.816+0000 7f6787400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67801f3830 con 0x7f678013cfa0 2024-01-15T18:07:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.816+0000 7f6787400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678015b860 con 0x7f678015afb0 2024-01-15T18:07:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.816+0000 7f6787400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678015bb40 con 0x7f678013fae0 2024-01-15T18:07:27.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.816+0000 7f6786000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678013cfa0 0x7f67801f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.816+0000 7f6786000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678013cfa0 0x7f67801f32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38802/0 (socket says 172.21.15.26:38802) 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.816+0000 7f6786000640 1 -- 172.21.15.26:0/2994699875 learned_addr learned my addr 172.21.15.26:0/2994699875 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f6786000640 1 -- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f678013fae0 msgr2=0x7f67801f4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f6786000640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f678013fae0 0x7f67801f4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f6786000640 1 -- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f678015afb0 msgr2=0x7f678013e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f6786a00640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f678015afb0 0x7f678013e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f6786000640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f678015afb0 0x7f678013e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f6786000640 1 -- 172.21.15.26:0/2994699875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764009400 con 0x7f678013cfa0 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f6786000640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678013cfa0 0x7f67801f32f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f677400c870 tx=0x7f677400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f677e000640 1 -- 172.21.15.26:0/2994699875 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6774003b70 con 0x7f678013cfa0 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f677e000640 1 -- 172.21.15.26:0/2994699875 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6774003cf0 con 0x7f678013cfa0 2024-01-15T18:07:27.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678015be20 con 0x7f678013cfa0 2024-01-15T18:07:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f677e000640 1 -- 172.21.15.26:0/2994699875 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677400aa90 con 0x7f678013cfa0 2024-01-15T18:07:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.817+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678015c3e0 con 0x7f678013cfa0 2024-01-15T18:07:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.819+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6780073c00 con 0x7f678013cfa0 2024-01-15T18:07:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.820+0000 7f677e000640 1 -- 172.21.15.26:0/2994699875 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f677400abf0 con 0x7f678013cfa0 2024-01-15T18:07:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.821+0000 7f677e000640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f675406bf20 0x7f675406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.821+0000 7f677e000640 1 -- 172.21.15.26:0/2994699875 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f67740884b0 con 0x7f678013cfa0 2024-01-15T18:07:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.822+0000 7f6785600640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f675406bf20 0x7f675406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.823+0000 7f6785600640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f675406bf20 0x7f675406e3e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6764036000 tx=0x7f6764030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:27.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:27.826+0000 7f677e000640 1 -- 172.21.15.26:0/2994699875 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6774054d70 con 0x7f678013cfa0 2024-01-15T18:07:28.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.046+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41817", "target": ["mon-mgr", ""]}) v1 -- 0x7f678015bcb0 con 0x7f675406bf20 2024-01-15T18:07:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.097+0000 7f677e000640 1 -- 172.21.15.26:0/2994699875 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f678015bcb0 con 0x7f675406bf20 2024-01-15T18:07:28.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.099+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f675406bf20 msgr2=0x7f675406e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.099+0000 7f6787400640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f675406bf20 0x7f675406e3e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6764036000 tx=0x7f6764030000 comp rx=0 tx=0).stop 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678013cfa0 msgr2=0x7f67801f32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678013cfa0 0x7f67801f32f0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f677400c870 tx=0x7f677400cd30 comp rx=0 tx=0).stop 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 shutdown_connections 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f678013fae0 0x7f67801f4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f675406bf20 0x7f675406e3e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f678015afb0 0x7f678013e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 --2- 172.21.15.26:0/2994699875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f678013cfa0 0x7f67801f32f0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 >> 172.21.15.26:0/2994699875 conn(0x7f678006a800 msgr2=0x7f6780153c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 shutdown_connections 2024-01-15T18:07:28.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.100+0000 7f6787400640 1 -- 172.21.15.26:0/2994699875 wait complete. 2024-01-15T18:07:28.110 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.106+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.110 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.106+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.110 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.106+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.111 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.106+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.111 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.107+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41810 2024-01-15T18:07:28.140 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.136+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.140 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.136+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.140 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.136+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.140 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.136+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.140 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:07:28.136+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:07:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.338+0000 7f345a200640 1 Processor -- start 2024-01-15T18:07:28.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.338+0000 7f345a200640 1 -- start start 2024-01-15T18:07:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.339+0000 7f345a200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f345413df80 0x7f345415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.339+0000 7f345a200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345413e360 0x7f34540cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.339+0000 7f345a200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345406c470 con 0x7f345415f9c0 2024-01-15T18:07:28.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.339+0000 7f345a200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345406c750 con 0x7f345413df80 2024-01-15T18:07:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.339+0000 7f345a200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345406ca30 con 0x7f345413e360 2024-01-15T18:07:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.340+0000 7f3453e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f345415f9c0 0x7f34541408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:53324/0 (socket says 172.21.15.26:53324) 2024-01-15T18:07:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.340+0000 7f3453e00640 1 -- 172.21.15.26:0/2637986247 learned_addr learned my addr 172.21.15.26:0/2637986247 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.340+0000 7f3452a00640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345413e360 0x7f34540cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.340+0000 7f3453400640 1 --2- 172.21.15.26:0/2637986247 >> v2:172.21.15.124:3300/0 conn(0x7f345413df80 0x7f345415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.340+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1927965271 0 0) 0x7f345406c470 con 0x7f345415f9c0 2024-01-15T18:07:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.340+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3434003520 con 0x7f345415f9c0 2024-01-15T18:07:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2095892353 0 0) 0x7f3434003520 con 0x7f345415f9c0 2024-01-15T18:07:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345413e360 msgr2=0x7f34540cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345413e360 0x7f34540cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 >> v2:172.21.15.124:3300/0 conn(0x7f345413df80 msgr2=0x7f345415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 --2- 172.21.15.26:0/2637986247 >> v2:172.21.15.124:3300/0 conn(0x7f345413df80 0x7f345415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345406cd10 con 0x7f345415f9c0 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f344c0019f0 con 0x7f345415f9c0 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 >> v1:172.21.15.26:6789/0 conn(0x7f345415f9c0 legacy=0x7f34541408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434003970 0x7f3434003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34340047c0 0x7f3434004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345413e360 0x7f3434005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345406c750 con 0x7f3434003970 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345406ca30 con 0x7f34340047c0 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343400b980 con 0x7f345413e360 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.341+0000 7f3453400640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434003970 0x7f3434003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.342+0000 7f3452a00640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34340047c0 0x7f3434004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.342+0000 7f3453e00640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345413e360 0x7f3434005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.342+0000 7f3453400640 1 -- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345413e360 msgr2=0x7f3434005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.342+0000 7f3453400640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345413e360 0x7f3434005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.342+0000 7f3453400640 1 -- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34340047c0 msgr2=0x7f3434004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.342+0000 7f3453400640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34340047c0 0x7f3434004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.342+0000 7f3453400640 1 -- 172.21.15.26:0/2637986247 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434003520 con 0x7f3434003970 2024-01-15T18:07:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.342+0000 7f3453e00640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f345413e360 0x7f3434005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.343+0000 7f3453400640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434003970 0x7f3434003d90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ac0 tx=0x7f34340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e9156e795b2c6d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.343+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344402c710 con 0x7f3434003970 2024-01-15T18:07:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.343+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344402c870 con 0x7f3434003970 2024-01-15T18:07:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.344+0000 7f3452000640 1 -- 172.21.15.26:0/2637986247 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344402cc00 con 0x7f3434003970 2024-01-15T18:07:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.344+0000 7f345a200640 1 -- 172.21.15.26:0/2637986247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434003970 msgr2=0x7f3434003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.344+0000 7f345a200640 1 --2- 172.21.15.26:0/2637986247 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3434003970 0x7f3434003d90 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f3444009ac0 tx=0x7f34340057f0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.344+0000 7f3453e00640 1 -- 172.21.15.26:0/2637986247 reap_dead start 2024-01-15T18:07:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.344+0000 7f345a200640 1 -- 172.21.15.26:0/2637986247 shutdown_connections 2024-01-15T18:07:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.344+0000 7f345a200640 1 -- 172.21.15.26:0/2637986247 >> 172.21.15.26:0/2637986247 conn(0x7f345406a800 msgr2=0x7f345406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.344+0000 7f345a200640 1 -- 172.21.15.26:0/2637986247 shutdown_connections 2024-01-15T18:07:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.345+0000 7f345a200640 1 -- 172.21.15.26:0/2637986247 wait complete. 2024-01-15T18:07:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.346+0000 7f345a200640 1 Processor -- start 2024-01-15T18:07:28.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.347+0000 7f345a200640 1 -- start start 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f345a200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3434003970 0x7f3454135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f345a200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34540706b0 0x7f3454071c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f345a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345413d640 0x7f34541529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f345a200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3454071810 con 0x7f345413d640 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f345a200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3454070b90 con 0x7f3434003970 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f345a200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3454070e70 con 0x7f34540706b0 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3453e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345413d640 0x7f34541529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3453e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345413d640 0x7f34541529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38822/0 (socket says 172.21.15.26:38822) 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3453e00640 1 -- 172.21.15.26:0/3814836499 learned_addr learned my addr 172.21.15.26:0/3814836499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3453400640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3434003970 0x7f3454135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3452a00640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34540706b0 0x7f3454071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3453e00640 1 -- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34540706b0 msgr2=0x7f3454071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3453e00640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34540706b0 0x7f3454071c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3453e00640 1 -- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3434003970 msgr2=0x7f3454135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3453e00640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3434003970 0x7f3454135640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.348+0000 7f3453e00640 1 -- 172.21.15.26:0/3814836499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3454071150 con 0x7f345413d640 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.349+0000 7f3453e00640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345413d640 0x7f34541529c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f345415fda0 tx=0x7f344c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.349+0000 7f344b400640 1 -- 172.21.15.26:0/3814836499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c004d60 con 0x7f345413d640 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.349+0000 7f344b400640 1 -- 172.21.15.26:0/3814836499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c00bb80 con 0x7f345413d640 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.349+0000 7f344b400640 1 -- 172.21.15.26:0/3814836499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c007110 con 0x7f345413d640 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.349+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3454071430 con 0x7f345413d640 2024-01-15T18:07:28.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.350+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3454072170 con 0x7f345413d640 2024-01-15T18:07:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.351+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3454071950 con 0x7f345413d640 2024-01-15T18:07:28.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.352+0000 7f344b400640 1 -- 172.21.15.26:0/3814836499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 28) v1 ==== 90661+0+0 (secure 0 0 0) 0x7f344c007480 con 0x7f345413d640 2024-01-15T18:07:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.353+0000 7f344b400640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f342c06bf50 0x7f342c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:28.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.353+0000 7f344b400640 1 -- 172.21.15.26:0/3814836499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f344c088f10 con 0x7f345413d640 2024-01-15T18:07:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.353+0000 7f3453400640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f342c06bf50 0x7f342c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.355+0000 7f3453400640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f342c06bf50 0x7f342c06e410 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3444036000 tx=0x7f3444030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:28.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.357+0000 7f344b400640 1 -- 172.21.15.26:0/3814836499 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3454071950 con 0x7f345413d640 2024-01-15T18:07:28.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.409+0000 7f344b400640 1 -- 172.21.15.26:0/3814836499 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f344c055920 con 0x7f345413d640 2024-01-15T18:07:28.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:28.583+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41810", "target": ["mon-mgr", ""]}) v1 -- 0x7f34540ce850 con 0x7f342c06bf50 2024-01-15T18:07:29.756 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:07:29.764 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:07:35.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.949+0000 7f344b400640 1 -- 172.21.15.26:0/3814836499 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f344402c9e0 con 0x7f342c06bf50 2024-01-15T18:07:35.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f342c06bf50 msgr2=0x7f342c06e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:35.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f342c06bf50 0x7f342c06e410 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3444036000 tx=0x7f3444030000 comp rx=0 tx=0).stop 2024-01-15T18:07:35.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345413d640 msgr2=0x7f34541529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:35.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345413d640 0x7f34541529c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f345415fda0 tx=0x7f344c00b4a0 comp rx=0 tx=0).stop 2024-01-15T18:07:35.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 shutdown_connections 2024-01-15T18:07:35.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34540706b0 0x7f3454071c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:35.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f342c06bf50 0x7f342c06e410 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:35.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345413d640 0x7f34541529c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:35.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 --2- 172.21.15.26:0/3814836499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3434003970 0x7f3454135640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:35.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.954+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 >> 172.21.15.26:0/3814836499 conn(0x7f345406a800 msgr2=0x7f3454154660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:35.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.955+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 shutdown_connections 2024-01-15T18:07:35.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:35.955+0000 7f345a200640 1 -- 172.21.15.26:0/3814836499 wait complete. 2024-01-15T18:07:35.985 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41811 2024-01-15T18:07:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.210+0000 7feb9f600640 1 Processor -- start 2024-01-15T18:07:36.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.210+0000 7feb9f600640 1 -- start start 2024-01-15T18:07:36.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.212+0000 7feb9f600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feb9813df80 0x7feb9815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.212+0000 7feb9f600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb980cf9a0 0x7feb980cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.212+0000 7feb9f600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9806c470 con 0x7feb9813e360 2024-01-15T18:07:36.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.212+0000 7feb9f600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb9806c750 con 0x7feb980cf9a0 2024-01-15T18:07:36.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.212+0000 7feb9f600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb9806ca30 con 0x7feb9813df80 2024-01-15T18:07:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.212+0000 7feb9d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb980cf9a0 0x7feb980cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.213+0000 7feb9d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feb980cf9a0 0x7feb980cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50522/0 (socket says 172.21.15.26:50522) 2024-01-15T18:07:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.213+0000 7feb9d600640 1 -- 172.21.15.26:0/2496257944 learned_addr learned my addr 172.21.15.26:0/2496257944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.213+0000 7feb9cc00640 1 --2- 172.21.15.26:0/2496257944 >> v2:172.21.15.124:3300/0 conn(0x7feb9813df80 0x7feb9815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.213+0000 7feb9d600640 1 -- 172.21.15.26:0/2496257944 >> v2:172.21.15.124:3300/0 conn(0x7feb9813df80 msgr2=0x7feb9815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.213+0000 7feb9d600640 1 --2- 172.21.15.26:0/2496257944 >> v2:172.21.15.124:3300/0 conn(0x7feb9813df80 0x7feb9815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.213+0000 7feb9d600640 1 -- 172.21.15.26:0/2496257944 >> v1:172.21.15.124:6789/0 conn(0x7feb9813e360 legacy=0x7feb980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.213+0000 7feb9d600640 1 -- 172.21.15.26:0/2496257944 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9806cd10 con 0x7feb980cf9a0 2024-01-15T18:07:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.214+0000 7feb9d600640 1 --2- 172.21.15.26:0/2496257944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 0x7feb980cd840 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7feb98072770 tx=0x7feb9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6582af69b78b615d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.215+0000 7feb97400640 1 -- 172.21.15.26:0/2496257944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9002ca80 con 0x7feb980cf9a0 2024-01-15T18:07:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.215+0000 7feb97400640 1 -- 172.21.15.26:0/2496257944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9002cbe0 con 0x7feb980cf9a0 2024-01-15T18:07:36.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.215+0000 7feb97400640 1 -- 172.21.15.26:0/2496257944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb900343f0 con 0x7feb980cf9a0 2024-01-15T18:07:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.215+0000 7feb9f600640 1 -- 172.21.15.26:0/2496257944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 msgr2=0x7feb980cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.215+0000 7feb9f600640 1 --2- 172.21.15.26:0/2496257944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 0x7feb980cd840 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7feb98072770 tx=0x7feb9002bee0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.215+0000 7feb9f600640 1 -- 172.21.15.26:0/2496257944 shutdown_connections 2024-01-15T18:07:36.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.215+0000 7feb9f600640 1 --2- 172.21.15.26:0/2496257944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 0x7feb980cd840 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.215+0000 7feb9f600640 1 --2- 172.21.15.26:0/2496257944 >> v2:172.21.15.124:3300/0 conn(0x7feb9813df80 0x7feb9815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.215+0000 7feb9f600640 1 -- 172.21.15.26:0/2496257944 >> 172.21.15.26:0/2496257944 conn(0x7feb9806a800 msgr2=0x7feb9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:36.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.216+0000 7feb9f600640 1 -- 172.21.15.26:0/2496257944 shutdown_connections 2024-01-15T18:07:36.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.216+0000 7feb9f600640 1 -- 172.21.15.26:0/2496257944 wait complete. 2024-01-15T18:07:36.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.217+0000 7feb9f600640 1 Processor -- start 2024-01-15T18:07:36.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.217+0000 7feb9f600640 1 -- start start 2024-01-15T18:07:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.218+0000 7feb9f600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 0x7feb98137900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.218+0000 7feb9f600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb9813df80 0x7feb9813c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.218+0000 7feb9f600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb9813e360 0x7feb9813cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.218+0000 7feb9f600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb98139660 con 0x7feb980cf9a0 2024-01-15T18:07:36.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.218+0000 7feb9f600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb981f1400 con 0x7feb9813e360 2024-01-15T18:07:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.218+0000 7feb9f600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb981f16e0 con 0x7feb9813df80 2024-01-15T18:07:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.219+0000 7feb9cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 0x7feb98137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.219+0000 7feb9cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 0x7feb98137900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50534/0 (socket says 172.21.15.26:50534) 2024-01-15T18:07:36.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.219+0000 7feb9cc00640 1 -- 172.21.15.26:0/3702157408 learned_addr learned my addr 172.21.15.26:0/3702157408 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.219+0000 7feb9cc00640 1 -- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb9813df80 msgr2=0x7feb9813c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.219+0000 7feb9cc00640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb9813df80 0x7feb9813c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.219+0000 7feb9cc00640 1 -- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb9813e360 msgr2=0x7feb9813cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.219+0000 7feb9cc00640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb9813e360 0x7feb9813cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.219+0000 7feb9cc00640 1 -- 172.21.15.26:0/3702157408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90009b30 con 0x7feb980cf9a0 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.219+0000 7feb9cc00640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 0x7feb98137900 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7feb8800d800 tx=0x7feb8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.220+0000 7feb94c00640 1 -- 172.21.15.26:0/3702157408 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800b6c0 con 0x7feb980cf9a0 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.220+0000 7feb94c00640 1 -- 172.21.15.26:0/3702157408 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8800b820 con 0x7feb980cf9a0 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.220+0000 7feb94c00640 1 -- 172.21.15.26:0/3702157408 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800bbb0 con 0x7feb980cf9a0 2024-01-15T18:07:36.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.220+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb981f19c0 con 0x7feb980cf9a0 2024-01-15T18:07:36.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.220+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb98137e40 con 0x7feb980cf9a0 2024-01-15T18:07:36.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.221+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb980675d0 con 0x7feb980cf9a0 2024-01-15T18:07:36.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.222+0000 7feb94c00640 1 -- 172.21.15.26:0/3702157408 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7feb8800bd10 con 0x7feb980cf9a0 2024-01-15T18:07:36.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.223+0000 7feb94c00640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb7006c090 0x7feb7006e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.223+0000 7feb94c00640 1 -- 172.21.15.26:0/3702157408 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7feb88088e80 con 0x7feb980cf9a0 2024-01-15T18:07:36.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.223+0000 7feb97e00640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb7006c090 0x7feb7006e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.225+0000 7feb97e00640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb7006c090 0x7feb7006e550 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feb980b2df0 tx=0x7feb800079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:36.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.228+0000 7feb94c00640 1 -- 172.21.15.26:0/3702157408 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb88055490 con 0x7feb980cf9a0 2024-01-15T18:07:36.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.451+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41811", "target": ["mon-mgr", ""]}) v1 -- 0x7feb9815f9c0 con 0x7feb7006c090 2024-01-15T18:07:36.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.496+0000 7feb94c00640 1 -- 172.21.15.26:0/3702157408 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7feb80014400 con 0x7feb7006c090 2024-01-15T18:07:36.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.499+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb7006c090 msgr2=0x7feb7006e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.499+0000 7feb9f600640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb7006c090 0x7feb7006e550 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feb980b2df0 tx=0x7feb800079b0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.499+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 msgr2=0x7feb98137900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.499+0000 7feb9f600640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 0x7feb98137900 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7feb8800d800 tx=0x7feb8800dcc0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.500+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 shutdown_connections 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.500+0000 7feb9f600640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feb9813df80 0x7feb9813c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.500+0000 7feb9f600640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feb7006c090 0x7feb7006e550 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.500+0000 7feb9f600640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feb9813e360 0x7feb9813cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.500+0000 7feb9f600640 1 --2- 172.21.15.26:0/3702157408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feb980cf9a0 0x7feb98137900 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.500+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 >> 172.21.15.26:0/3702157408 conn(0x7feb9806a800 msgr2=0x7feb980b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.500+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 shutdown_connections 2024-01-15T18:07:36.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.500+0000 7feb9f600640 1 -- 172.21.15.26:0/3702157408 wait complete. 2024-01-15T18:07:36.527 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41813 2024-01-15T18:07:36.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.732+0000 7f38a3200640 1 Processor -- start 2024-01-15T18:07:36.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.732+0000 7f38a3200640 1 -- start start 2024-01-15T18:07:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.734+0000 7f38a3200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f38a4159cb0 0x7f38a406a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.734+0000 7f38a3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a406ac90 0x7f38a40b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.734+0000 7f38a3200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a4071340 con 0x7f38a40b3660 2024-01-15T18:07:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.734+0000 7f38a3200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a415d9e0 con 0x7f38a4159cb0 2024-01-15T18:07:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.734+0000 7f38a3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a415e840 con 0x7f38a406ac90 2024-01-15T18:07:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.734+0000 7f38a2800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38a40b3660 0x7f38a406d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42506/0 (socket says 172.21.15.26:42506) 2024-01-15T18:07:36.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.734+0000 7f38a2800640 1 -- 172.21.15.26:0/3916393901 learned_addr learned my addr 172.21.15.26:0/3916393901 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.735+0000 7f38a1e00640 1 --2- 172.21.15.26:0/3916393901 >> v2:172.21.15.124:3300/0 conn(0x7f38a4159cb0 0x7f38a406a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.735+0000 7f389be00640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a406ac90 0x7f38a40b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.735+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410794272 0 0) 0x7f38a4071340 con 0x7f38a40b3660 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.735+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3884005730 con 0x7f38a40b3660 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.735+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448235514 0 0) 0x7f3884005730 con 0x7f38a40b3660 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.735+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a406ac90 msgr2=0x7f38a40b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.735+0000 7f38a1400640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a406ac90 0x7f38a40b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.735+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 >> v2:172.21.15.124:3300/0 conn(0x7f38a4159cb0 msgr2=0x7f38a406a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.735+0000 7f38a1400640 1 --2- 172.21.15.26:0/3916393901 >> v2:172.21.15.124:3300/0 conn(0x7f38a4159cb0 0x7f38a406a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a406b0f0 con 0x7f38a40b3660 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f389c0019f0 con 0x7f38a40b3660 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 >> v1:172.21.15.26:6789/0 conn(0x7f38a40b3660 legacy=0x7f38a406d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1400640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884005b80 0x7f3884005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1400640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a406ac90 0x7f38840069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1400640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3884006530 0x7f3884006f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a415d9e0 con 0x7f3884006530 2024-01-15T18:07:36.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a415e840 con 0x7f3884005b80 2024-01-15T18:07:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3884007790 con 0x7f38a406ac90 2024-01-15T18:07:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a2800640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3884006530 0x7f3884006f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f38a1e00640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884005b80 0x7f3884005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.736+0000 7f389be00640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a406ac90 0x7f38840069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.737+0000 7f38a2800640 1 -- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a406ac90 msgr2=0x7f38840069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.737+0000 7f38a2800640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a406ac90 0x7f38840069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.737+0000 7f38a2800640 1 -- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884005b80 msgr2=0x7f3884005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.737+0000 7f38a2800640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884005b80 0x7f3884005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.737+0000 7f38a2800640 1 -- 172.21.15.26:0/3916393901 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3884005730 con 0x7f3884006530 2024-01-15T18:07:36.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.738+0000 7f38a2800640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3884006530 0x7f3884006f30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f389c00a770 tx=0x7f3884007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8359199291da8f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:36.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.738+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c034630 con 0x7f3884006530 2024-01-15T18:07:36.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.738+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c02f070 con 0x7f3884006530 2024-01-15T18:07:36.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.738+0000 7f38a1400640 1 -- 172.21.15.26:0/3916393901 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0349f0 con 0x7f3884006530 2024-01-15T18:07:36.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.738+0000 7f38a3200640 1 -- 172.21.15.26:0/3916393901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3884006530 msgr2=0x7f3884006f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.738+0000 7f38a3200640 1 --2- 172.21.15.26:0/3916393901 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3884006530 0x7f3884006f30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f389c00a770 tx=0x7f3884007a00 comp rx=0 tx=0).stop 2024-01-15T18:07:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.738+0000 7f38a2800640 1 -- 172.21.15.26:0/3916393901 reap_dead start 2024-01-15T18:07:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.739+0000 7f38a3200640 1 -- 172.21.15.26:0/3916393901 shutdown_connections 2024-01-15T18:07:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.739+0000 7f38a3200640 1 -- 172.21.15.26:0/3916393901 >> 172.21.15.26:0/3916393901 conn(0x7f38a40b1060 msgr2=0x7f38a40b1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.739+0000 7f38a3200640 1 -- 172.21.15.26:0/3916393901 shutdown_connections 2024-01-15T18:07:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.739+0000 7f38a3200640 1 -- 172.21.15.26:0/3916393901 wait complete. 2024-01-15T18:07:36.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.740+0000 7f38a3200640 1 Processor -- start 2024-01-15T18:07:36.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.741+0000 7f38a3200640 1 -- start start 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.741+0000 7f38a3200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884006530 0x7f38a409c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f38a3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a414c9a0 0x7f38a406d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f38a3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415a090 0x7f38a4141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f38a3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a409aff0 con 0x7f38a414c9a0 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f38a3200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a409b2b0 con 0x7f3884006530 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f38a3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a409b590 con 0x7f38a415a090 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f389be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a414c9a0 0x7f38a406d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f389be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a414c9a0 0x7f38a406d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50550/0 (socket says 172.21.15.26:50550) 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f389be00640 1 -- 172.21.15.26:0/2830561499 learned_addr learned my addr 172.21.15.26:0/2830561499 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f38a1e00640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884006530 0x7f38a409c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f389be00640 1 -- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415a090 msgr2=0x7f38a4141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f389be00640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415a090 0x7f38a4141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f389be00640 1 -- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884006530 msgr2=0x7f38a409c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f389be00640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884006530 0x7f38a409c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f389be00640 1 -- 172.21.15.26:0/2830561499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a409b980 con 0x7f38a414c9a0 2024-01-15T18:07:36.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.742+0000 7f38a1e00640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884006530 0x7f38a409c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.743+0000 7f389be00640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a414c9a0 0x7f38a406d800 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f388c011600 tx=0x7f388c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.743+0000 7f389a000640 1 -- 172.21.15.26:0/2830561499 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00f920 con 0x7f38a414c9a0 2024-01-15T18:07:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.743+0000 7f389a000640 1 -- 172.21.15.26:0/2830561499 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c00fa80 con 0x7f38a414c9a0 2024-01-15T18:07:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.743+0000 7f389a000640 1 -- 172.21.15.26:0/2830561499 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c00fdf0 con 0x7f38a414c9a0 2024-01-15T18:07:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.743+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a409bc60 con 0x7f38a414c9a0 2024-01-15T18:07:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.743+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a409c6c0 con 0x7f38a414c9a0 2024-01-15T18:07:36.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.744+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a409b7d0 con 0x7f38a414c9a0 2024-01-15T18:07:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.746+0000 7f389a000640 1 -- 172.21.15.26:0/2830561499 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f388c015460 con 0x7f38a414c9a0 2024-01-15T18:07:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.747+0000 7f389a000640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f387c06c070 0x7f387c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.747+0000 7f389a000640 1 -- 172.21.15.26:0/2830561499 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f388c088d10 con 0x7f38a414c9a0 2024-01-15T18:07:36.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.748+0000 7f38a1e00640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f387c06c070 0x7f387c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:36.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.750+0000 7f38a1e00640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f387c06c070 0x7f387c06e530 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3894009a00 tx=0x7f3894008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:36.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.752+0000 7f389a000640 1 -- 172.21.15.26:0/2830561499 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f388c055320 con 0x7f38a414c9a0 2024-01-15T18:07:36.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:36.974+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41813", "target": ["mon-mgr", ""]}) v1 -- 0x7f38a409b420 con 0x7f387c06c070 2024-01-15T18:07:37.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.018+0000 7f389a000640 1 -- 172.21.15.26:0/2830561499 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f38a409b420 con 0x7f387c06c070 2024-01-15T18:07:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f387c06c070 msgr2=0x7f387c06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f387c06c070 0x7f387c06e530 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3894009a00 tx=0x7f3894008040 comp rx=0 tx=0).stop 2024-01-15T18:07:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a414c9a0 msgr2=0x7f38a406d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a414c9a0 0x7f38a406d800 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f388c011600 tx=0x7f388c011ac0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 shutdown_connections 2024-01-15T18:07:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38a415a090 0x7f38a4141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f387c06c070 0x7f387c06e530 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38a414c9a0 0x7f38a406d800 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 --2- 172.21.15.26:0/2830561499 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3884006530 0x7f38a409c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 >> 172.21.15.26:0/2830561499 conn(0x7f38a40b1060 msgr2=0x7f38a40cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 shutdown_connections 2024-01-15T18:07:37.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.021+0000 7f38a3200640 1 -- 172.21.15.26:0/2830561499 wait complete. 2024-01-15T18:07:37.048 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone4182 2024-01-15T18:07:37.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.258+0000 7f7e5e200640 1 Processor -- start 2024-01-15T18:07:37.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.258+0000 7f7e5e200640 1 -- start start 2024-01-15T18:07:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.260+0000 7f7e5e200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7e58143c40 0x7f7e58071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.260+0000 7f7e5e200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7e580ccc10 0x7f7e5806d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.260+0000 7f7e5e200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e58070e50 con 0x7f7e580717d0 2024-01-15T18:07:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.260+0000 7f7e5e200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e580723c0 con 0x7f7e580ccc10 2024-01-15T18:07:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.260+0000 7f7e5e200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e58144020 con 0x7f7e58143c40 2024-01-15T18:07:37.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.260+0000 7f7e5d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7e580ccc10 0x7f7e5806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.260+0000 7f7e57e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7e580717d0 0x7f7e580cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42508/0 (socket says 172.21.15.26:42508) 2024-01-15T18:07:37.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.260+0000 7f7e57e00640 1 -- 172.21.15.26:0/2968642033 learned_addr learned my addr 172.21.15.26:0/2968642033 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:37.265 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.261+0000 7f7e5ce00640 1 --2- 172.21.15.26:0/2968642033 >> v2:172.21.15.124:3300/0 conn(0x7f7e58143c40 0x7f7e58071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.261+0000 7f7e5d800640 1 -- 172.21.15.26:0/2968642033 >> v2:172.21.15.124:3300/0 conn(0x7f7e58143c40 msgr2=0x7f7e58071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.261+0000 7f7e5d800640 1 --2- 172.21.15.26:0/2968642033 >> v2:172.21.15.124:3300/0 conn(0x7f7e58143c40 0x7f7e58071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.261+0000 7f7e5d800640 1 -- 172.21.15.26:0/2968642033 >> v1:172.21.15.26:6789/0 conn(0x7f7e580717d0 legacy=0x7f7e580cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.261+0000 7f7e57400640 1 -- 172.21.15.26:0/2968642033 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411200583 0 0) 0x7f7e58070e50 con 0x7f7e580717d0 2024-01-15T18:07:37.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.261+0000 7f7e5d800640 1 -- 172.21.15.26:0/2968642033 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e58071e00 con 0x7f7e580ccc10 2024-01-15T18:07:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.263+0000 7f7e5d800640 1 --2- 172.21.15.26:0/2968642033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 0x7f7e5806d160 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7e5806e410 tx=0x7f7e5002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6b730c68f00b7c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.263+0000 7f7e57400640 1 -- 172.21.15.26:0/2968642033 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5002cbe0 con 0x7f7e580ccc10 2024-01-15T18:07:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.263+0000 7f7e57400640 1 -- 172.21.15.26:0/2968642033 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5002cd40 con 0x7f7e580ccc10 2024-01-15T18:07:37.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.263+0000 7f7e57400640 1 -- 172.21.15.26:0/2968642033 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50034660 con 0x7f7e580ccc10 2024-01-15T18:07:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.264+0000 7f7e5e200640 1 -- 172.21.15.26:0/2968642033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 msgr2=0x7f7e5806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.264+0000 7f7e5e200640 1 --2- 172.21.15.26:0/2968642033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 0x7f7e5806d160 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7e5806e410 tx=0x7f7e5002be30 comp rx=0 tx=0).stop 2024-01-15T18:07:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.264+0000 7f7e5e200640 1 -- 172.21.15.26:0/2968642033 shutdown_connections 2024-01-15T18:07:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.264+0000 7f7e5e200640 1 --2- 172.21.15.26:0/2968642033 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 0x7f7e5806d160 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.264+0000 7f7e5e200640 1 --2- 172.21.15.26:0/2968642033 >> v2:172.21.15.124:3300/0 conn(0x7f7e58143c40 0x7f7e58071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.264+0000 7f7e5e200640 1 -- 172.21.15.26:0/2968642033 >> 172.21.15.26:0/2968642033 conn(0x7f7e5806a800 msgr2=0x7f7e5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.264+0000 7f7e5e200640 1 -- 172.21.15.26:0/2968642033 shutdown_connections 2024-01-15T18:07:37.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.264+0000 7f7e5e200640 1 -- 172.21.15.26:0/2968642033 wait complete. 2024-01-15T18:07:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.266+0000 7f7e5e200640 1 Processor -- start 2024-01-15T18:07:37.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.266+0000 7f7e5e200640 1 -- start start 2024-01-15T18:07:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e5e200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e580717d0 0x7f7e580ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e5e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 0x7f7e580c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e5e200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e58143c40 0x7f7e580c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e5e200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580ae2b0 con 0x7f7e580ccc10 2024-01-15T18:07:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e5e200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e580cae00 con 0x7f7e580717d0 2024-01-15T18:07:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e5e200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5812b950 con 0x7f7e58143c40 2024-01-15T18:07:37.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e57e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 0x7f7e580c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e57e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 0x7f7e580c6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50568/0 (socket says 172.21.15.26:50568) 2024-01-15T18:07:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e57e00640 1 -- 172.21.15.26:0/2728657673 learned_addr learned my addr 172.21.15.26:0/2728657673 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.267+0000 7f7e5ce00640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e580717d0 0x7f7e580ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.268+0000 7f7e5d800640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e58143c40 0x7f7e580c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.268+0000 7f7e57e00640 1 -- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e58143c40 msgr2=0x7f7e580c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.268+0000 7f7e57e00640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e58143c40 0x7f7e580c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.268+0000 7f7e57e00640 1 -- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e580717d0 msgr2=0x7f7e580ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.268+0000 7f7e57e00640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e580717d0 0x7f7e580ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.268+0000 7f7e57e00640 1 -- 172.21.15.26:0/2728657673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e50009a70 con 0x7f7e580ccc10 2024-01-15T18:07:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.269+0000 7f7e57e00640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 0x7f7e580c6d70 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c460 tx=0x7f7e4000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.269+0000 7f7e54c00640 1 -- 172.21.15.26:0/2728657673 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e400074e0 con 0x7f7e580ccc10 2024-01-15T18:07:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.269+0000 7f7e54c00640 1 -- 172.21.15.26:0/2728657673 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e40007660 con 0x7f7e580ccc10 2024-01-15T18:07:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.269+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e580c8de0 con 0x7f7e580ccc10 2024-01-15T18:07:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.269+0000 7f7e54c00640 1 -- 172.21.15.26:0/2728657673 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e40004a20 con 0x7f7e580ccc10 2024-01-15T18:07:37.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.269+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5812b300 con 0x7f7e580ccc10 2024-01-15T18:07:37.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.270+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e58067450 con 0x7f7e580ccc10 2024-01-15T18:07:37.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.271+0000 7f7e54c00640 1 -- 172.21.15.26:0/2728657673 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f7e40004b80 con 0x7f7e580ccc10 2024-01-15T18:07:37.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.272+0000 7f7e54c00640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7e3006c040 0x7f7e3006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.272+0000 7f7e54c00640 1 -- 172.21.15.26:0/2728657673 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f7e40087aa0 con 0x7f7e580ccc10 2024-01-15T18:07:37.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.273+0000 7f7e5ce00640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7e3006c040 0x7f7e3006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.274+0000 7f7e5ce00640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7e3006c040 0x7f7e3006e500 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009b40 tx=0x7f7e48009190 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:37.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.277+0000 7f7e54c00640 1 -- 172.21.15.26:0/2728657673 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e40054160 con 0x7f7e580ccc10 2024-01-15T18:07:37.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.507+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone4182", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e5812b180 con 0x7f7e3006c040 2024-01-15T18:07:37.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.549+0000 7f7e54c00640 1 -- 172.21.15.26:0/2728657673 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7e48011a60 con 0x7f7e3006c040 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7e3006c040 msgr2=0x7f7e3006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7e3006c040 0x7f7e3006e500 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009b40 tx=0x7f7e48009190 comp rx=0 tx=0).stop 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 msgr2=0x7f7e580c6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 0x7f7e580c6d70 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000c460 tx=0x7f7e4000c920 comp rx=0 tx=0).stop 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 shutdown_connections 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7e58143c40 0x7f7e580c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7e3006c040 0x7f7e3006e500 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7e580ccc10 0x7f7e580c6d70 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 --2- 172.21.15.26:0/2728657673 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7e580717d0 0x7f7e580ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 >> 172.21.15.26:0/2728657673 conn(0x7f7e5806a800 msgr2=0x7f7e58144190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 shutdown_connections 2024-01-15T18:07:37.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.553+0000 7f7e5e200640 1 -- 172.21.15.26:0/2728657673 wait complete. 2024-01-15T18:07:37.579 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone4189 2024-01-15T18:07:37.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.796+0000 7f520c200640 1 Processor -- start 2024-01-15T18:07:37.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.796+0000 7f520c200640 1 -- start start 2024-01-15T18:07:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.797+0000 7f520c200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52040ae5a0 0x7f520415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.797+0000 7f520c200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f52040b3ae0 0x7f520406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.798+0000 7f520c200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f520406d120 0x7f52040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.798+0000 7f520c200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5204071450 con 0x7f520406d120 2024-01-15T18:07:37.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.798+0000 7f520c200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5204070c40 con 0x7f52040b3ae0 2024-01-15T18:07:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.798+0000 7f520c200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520406c470 con 0x7f52040ae5a0 2024-01-15T18:07:37.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.798+0000 7f520a200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f520406d120 0x7f52040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.798+0000 7f520a200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f520406d120 0x7f52040cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50584/0 (socket says 172.21.15.26:50584) 2024-01-15T18:07:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.798+0000 7f520a200640 1 -- 172.21.15.26:0/4205086060 learned_addr learned my addr 172.21.15.26:0/4205086060 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.798+0000 7f5208e00640 1 --2- 172.21.15.26:0/4205086060 >> v2:172.21.15.124:3300/0 conn(0x7f52040b3ae0 0x7f520406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.798+0000 7f5209800640 1 --2- 172.21.15.26:0/4205086060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52040ae5a0 0x7f520415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.799+0000 7f520a200640 1 -- 172.21.15.26:0/4205086060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52040ae5a0 msgr2=0x7f520415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.799+0000 7f520a200640 1 --2- 172.21.15.26:0/4205086060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52040ae5a0 0x7f520415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.799+0000 7f520a200640 1 -- 172.21.15.26:0/4205086060 >> v2:172.21.15.124:3300/0 conn(0x7f52040b3ae0 msgr2=0x7f520406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.799+0000 7f520a200640 1 --2- 172.21.15.26:0/4205086060 >> v2:172.21.15.124:3300/0 conn(0x7f52040b3ae0 0x7f520406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.799+0000 7f520a200640 1 -- 172.21.15.26:0/4205086060 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520406c750 con 0x7f520406d120 2024-01-15T18:07:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.800+0000 7f520a200640 1 --2- 172.21.15.26:0/4205086060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f520406d120 0x7f52040cf290 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f52040cfed0 tx=0x7f51fc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9073ac72f2ee5bf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.800+0000 7f5203e00640 1 -- 172.21.15.26:0/4205086060 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc031860 con 0x7f520406d120 2024-01-15T18:07:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.800+0000 7f5203e00640 1 -- 172.21.15.26:0/4205086060 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc0319c0 con 0x7f520406d120 2024-01-15T18:07:37.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.800+0000 7f5203e00640 1 -- 172.21.15.26:0/4205086060 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc031d50 con 0x7f520406d120 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.801+0000 7f520c200640 1 -- 172.21.15.26:0/4205086060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f520406d120 msgr2=0x7f52040cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.801+0000 7f520c200640 1 --2- 172.21.15.26:0/4205086060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f520406d120 0x7f52040cf290 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f52040cfed0 tx=0x7f51fc02b900 comp rx=0 tx=0).stop 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.801+0000 7f520c200640 1 -- 172.21.15.26:0/4205086060 shutdown_connections 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.801+0000 7f520c200640 1 --2- 172.21.15.26:0/4205086060 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f520406d120 0x7f52040cf290 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.801+0000 7f520c200640 1 --2- 172.21.15.26:0/4205086060 >> v2:172.21.15.124:3300/0 conn(0x7f52040b3ae0 0x7f520406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.801+0000 7f520c200640 1 --2- 172.21.15.26:0/4205086060 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52040ae5a0 0x7f520415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.801+0000 7f520c200640 1 -- 172.21.15.26:0/4205086060 >> 172.21.15.26:0/4205086060 conn(0x7f520406a800 msgr2=0x7f520406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.801+0000 7f520c200640 1 -- 172.21.15.26:0/4205086060 shutdown_connections 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.801+0000 7f520c200640 1 -- 172.21.15.26:0/4205086060 wait complete. 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.803+0000 7f520c200640 1 Processor -- start 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.803+0000 7f520c200640 1 -- start start 2024-01-15T18:07:37.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.804+0000 7f520c200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f520406d120 0x7f52040ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.804+0000 7f520c200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52040ae5a0 0x7f52040cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.804+0000 7f520c200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52040b3ae0 0x7f52041597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.804+0000 7f520c200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040cc7f0 con 0x7f52040ae5a0 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.804+0000 7f520c200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040ccad0 con 0x7f52040b3ae0 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.804+0000 7f520c200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52040ccdb0 con 0x7f520406d120 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.804+0000 7f5208e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52040ae5a0 0x7f52040cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5208e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52040ae5a0 0x7f52040cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50594/0 (socket says 172.21.15.26:50594) 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5208e00640 1 -- 172.21.15.26:0/1594179056 learned_addr learned my addr 172.21.15.26:0/1594179056 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f520a200640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52040b3ae0 0x7f52041597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5209800640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f520406d120 0x7f52040ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5208e00640 1 -- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f520406d120 msgr2=0x7f52040ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5208e00640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f520406d120 0x7f52040ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5208e00640 1 -- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52040b3ae0 msgr2=0x7f52041597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5208e00640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52040b3ae0 0x7f52041597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:37.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5208e00640 1 -- 172.21.15.26:0/1594179056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0094d0 con 0x7f52040ae5a0 2024-01-15T18:07:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5209800640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f520406d120 0x7f52040ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:37.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.805+0000 7f5208e00640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52040ae5a0 0x7f52040cafa0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00d5f0 tx=0x7f51ec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.806+0000 7f5201600640 1 -- 172.21.15.26:0/1594179056 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec0125f0 con 0x7f52040ae5a0 2024-01-15T18:07:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.806+0000 7f5201600640 1 -- 172.21.15.26:0/1594179056 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec012750 con 0x7f52040ae5a0 2024-01-15T18:07:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.806+0000 7f5201600640 1 -- 172.21.15.26:0/1594179056 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec012b00 con 0x7f52040ae5a0 2024-01-15T18:07:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.806+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52040cd090 con 0x7f52040ae5a0 2024-01-15T18:07:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.806+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52040ce590 con 0x7f52040ae5a0 2024-01-15T18:07:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.807+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52040675d0 con 0x7f52040ae5a0 2024-01-15T18:07:37.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.808+0000 7f5201600640 1 -- 172.21.15.26:0/1594179056 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f51ec012c60 con 0x7f52040ae5a0 2024-01-15T18:07:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.809+0000 7f5201600640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f51dc06c040 0x7f51dc06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.809+0000 7f5201600640 1 -- 172.21.15.26:0/1594179056 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f51ec088de0 con 0x7f52040ae5a0 2024-01-15T18:07:37.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.810+0000 7f5209800640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f51dc06c040 0x7f51dc06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:37.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.811+0000 7f5209800640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f51dc06c040 0x7f51dc06e500 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009f70 tx=0x7f51f400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:37.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:37.813+0000 7f5201600640 1 -- 172.21.15.26:0/1594179056 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51ec0553f0 con 0x7f52040ae5a0 2024-01-15T18:07:38.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:38.036+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone4189", "target": ["mon-mgr", ""]}) v1 -- 0x7f520406af60 con 0x7f51dc06c040 2024-01-15T18:07:40.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.948+0000 7f5201600640 1 -- 172.21.15.26:0/1594179056 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f520406af60 con 0x7f51dc06c040 2024-01-15T18:07:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.952+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f51dc06c040 msgr2=0x7f51dc06e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.952+0000 7f520c200640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f51dc06c040 0x7f51dc06e500 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009f70 tx=0x7f51f400b040 comp rx=0 tx=0).stop 2024-01-15T18:07:40.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.952+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52040ae5a0 msgr2=0x7f52040cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.952+0000 7f520c200640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52040ae5a0 0x7f52040cafa0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00d5f0 tx=0x7f51ec00dab0 comp rx=0 tx=0).stop 2024-01-15T18:07:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.953+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 shutdown_connections 2024-01-15T18:07:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.953+0000 7f520c200640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f51dc06c040 0x7f51dc06e500 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.953+0000 7f520c200640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52040b3ae0 0x7f52041597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.953+0000 7f520c200640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52040ae5a0 0x7f52040cafa0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.953+0000 7f520c200640 1 --2- 172.21.15.26:0/1594179056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f520406d120 0x7f52040ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.953+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 >> 172.21.15.26:0/1594179056 conn(0x7f520406a800 msgr2=0x7f52040b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.953+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 shutdown_connections 2024-01-15T18:07:40.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:40.953+0000 7f520c200640 1 -- 172.21.15.26:0/1594179056 wait complete. 2024-01-15T18:07:40.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:07:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.194+0000 7f7c0fe00640 1 Processor -- start 2024-01-15T18:07:41.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.194+0000 7f7c0fe00640 1 -- start start 2024-01-15T18:07:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.196+0000 7f7c0fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c10143c40 0x7f7c10071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.196+0000 7f7c0fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c100717d0 0x7f7c100cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.196+0000 7f7c0fe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7c100cca40 0x7f7c1015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.196+0000 7f7c0fe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c100723c0 con 0x7f7c100717d0 2024-01-15T18:07:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.196+0000 7f7c0fe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c10070e50 con 0x7f7c100cca40 2024-01-15T18:07:41.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.196+0000 7f7c0fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c100cce70 con 0x7f7c10143c40 2024-01-15T18:07:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.196+0000 7f7c0e000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c100717d0 0x7f7c100cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.196+0000 7f7c0e000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7c100717d0 0x7f7c100cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50608/0 (socket says 172.21.15.26:50608) 2024-01-15T18:07:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.196+0000 7f7c0e000640 1 -- 172.21.15.26:0/1352539043 learned_addr learned my addr 172.21.15.26:0/1352539043 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.197+0000 7f7c0ea00640 1 --2- 172.21.15.26:0/1352539043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c10143c40 0x7f7c10071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.197+0000 7f7c0f400640 1 --2- 172.21.15.26:0/1352539043 >> v2:172.21.15.124:3300/0 conn(0x7f7c100cca40 0x7f7c1015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.197+0000 7f7c0e000640 1 -- 172.21.15.26:0/1352539043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c10143c40 msgr2=0x7f7c10071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.197+0000 7f7c0e000640 1 --2- 172.21.15.26:0/1352539043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c10143c40 0x7f7c10071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.197+0000 7f7c0e000640 1 -- 172.21.15.26:0/1352539043 >> v2:172.21.15.124:3300/0 conn(0x7f7c100cca40 msgr2=0x7f7c1015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.197+0000 7f7c0e000640 1 --2- 172.21.15.26:0/1352539043 >> v2:172.21.15.124:3300/0 conn(0x7f7c100cca40 0x7f7c1015eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.197+0000 7f7c0e000640 1 -- 172.21.15.26:0/1352539043 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10144020 con 0x7f7c100717d0 2024-01-15T18:07:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.197+0000 7f7c0f400640 1 --2- 172.21.15.26:0/1352539043 >> v2:172.21.15.124:3300/0 conn(0x7f7c100cca40 0x7f7c1015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.198+0000 7f7c0e000640 1 --2- 172.21.15.26:0/1352539043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100717d0 0x7f7c100cc500 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7c100cc3f0 tx=0x7f7bf802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae86bcc5e58ea372 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0d600640 1 -- 172.21.15.26:0/1352539043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf80317e0 con 0x7f7c100717d0 2024-01-15T18:07:41.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0d600640 1 -- 172.21.15.26:0/1352539043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf8031940 con 0x7f7c100717d0 2024-01-15T18:07:41.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0d600640 1 -- 172.21.15.26:0/1352539043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8031cd0 con 0x7f7c100717d0 2024-01-15T18:07:41.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0fe00640 1 -- 172.21.15.26:0/1352539043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100717d0 msgr2=0x7f7c100cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:41.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/1352539043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100717d0 0x7f7c100cc500 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f7c100cc3f0 tx=0x7f7bf802b880 comp rx=0 tx=0).stop 2024-01-15T18:07:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0fe00640 1 -- 172.21.15.26:0/1352539043 shutdown_connections 2024-01-15T18:07:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/1352539043 >> v2:172.21.15.124:3300/0 conn(0x7f7c100cca40 0x7f7c1015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/1352539043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c100717d0 0x7f7c100cc500 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/1352539043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c10143c40 0x7f7c10071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.199+0000 7f7c0fe00640 1 -- 172.21.15.26:0/1352539043 >> 172.21.15.26:0/1352539043 conn(0x7f7c1006a800 msgr2=0x7f7c1006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.200+0000 7f7c0fe00640 1 -- 172.21.15.26:0/1352539043 shutdown_connections 2024-01-15T18:07:41.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.200+0000 7f7c0fe00640 1 -- 172.21.15.26:0/1352539043 wait complete. 2024-01-15T18:07:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.201+0000 7f7c0fe00640 1 Processor -- start 2024-01-15T18:07:41.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.201+0000 7f7c0fe00640 1 -- start start 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.202+0000 7f7c0fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c100717d0 0x7f7c1013c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.202+0000 7f7c0fe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c100cca40 0x7f7c10152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.202+0000 7f7c0fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c10143c40 0x7f7c101528c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.202+0000 7f7c0fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c10157220 con 0x7f7c10143c40 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.202+0000 7f7c0fe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c10156610 con 0x7f7c100cca40 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.202+0000 7f7c0fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c101568f0 con 0x7f7c100717d0 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c10143c40 0x7f7c101528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c10143c40 0x7f7c101528c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50616/0 (socket says 172.21.15.26:50616) 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0f400640 1 -- 172.21.15.26:0/2267137917 learned_addr learned my addr 172.21.15.26:0/2267137917 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0ea00640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c100717d0 0x7f7c1013c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0f400640 1 -- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c100717d0 msgr2=0x7f7c1013c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0f400640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c100717d0 0x7f7c1013c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0f400640 1 -- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c100cca40 msgr2=0x7f7c10152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0f400640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c100cca40 0x7f7c10152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0f400640 1 -- 172.21.15.26:0/2267137917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf80094d0 con 0x7f7c10143c40 2024-01-15T18:07:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.203+0000 7f7c0f400640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c10143c40 0x7f7c101528c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800cef0 tx=0x7f7c0800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.204+0000 7f7c07400640 1 -- 172.21.15.26:0/2267137917 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800d840 con 0x7f7c10143c40 2024-01-15T18:07:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.204+0000 7f7c07400640 1 -- 172.21.15.26:0/2267137917 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0800d9c0 con 0x7f7c10143c40 2024-01-15T18:07:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.204+0000 7f7c07400640 1 -- 172.21.15.26:0/2267137917 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0800dd70 con 0x7f7c10143c40 2024-01-15T18:07:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.204+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c10156bd0 con 0x7f7c10143c40 2024-01-15T18:07:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.204+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1013c700 con 0x7f7c10143c40 2024-01-15T18:07:41.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.205+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c10067450 con 0x7f7c10143c40 2024-01-15T18:07:41.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.206+0000 7f7c07400640 1 -- 172.21.15.26:0/2267137917 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f7c0801e3b0 con 0x7f7c10143c40 2024-01-15T18:07:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.207+0000 7f7c07400640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7be806c040 0x7f7be806e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.207+0000 7f7c07400640 1 -- 172.21.15.26:0/2267137917 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f7c080893d0 con 0x7f7c10143c40 2024-01-15T18:07:41.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.208+0000 7f7c0ea00640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7be806c040 0x7f7be806e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.209+0000 7f7c0ea00640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7be806c040 0x7f7be806e500 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b7e0 tx=0x7f7c00008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:41.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.212+0000 7f7c07400640 1 -- 172.21.15.26:0/2267137917 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c08055aa0 con 0x7f7c10143c40 2024-01-15T18:07:41.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.438+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c100b1d90 con 0x7f7be806c040 2024-01-15T18:07:41.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.477+0000 7f7c07400640 1 -- 172.21.15.26:0/2267137917 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f7c100b1d90 con 0x7f7be806c040 2024-01-15T18:07:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:07:41.481 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:07:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:07:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1" 2024-01-15T18:07:41.481 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:07:41.481 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:07:41.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7be806c040 msgr2=0x7f7be806e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:41.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7be806c040 0x7f7be806e500 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b7e0 tx=0x7f7c00008040 comp rx=0 tx=0).stop 2024-01-15T18:07:41.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c10143c40 msgr2=0x7f7c101528c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:41.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c10143c40 0x7f7c101528c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800cef0 tx=0x7f7c0800a710 comp rx=0 tx=0).stop 2024-01-15T18:07:41.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 shutdown_connections 2024-01-15T18:07:41.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7be806c040 0x7f7be806e500 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7c10143c40 0x7f7c101528c0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7c100cca40 0x7f7c10152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 --2- 172.21.15.26:0/2267137917 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7c100717d0 0x7f7c1013c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 >> 172.21.15.26:0/2267137917 conn(0x7f7c1006a800 msgr2=0x7f7c1006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:41.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 shutdown_connections 2024-01-15T18:07:41.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.480+0000 7f7c0fe00640 1 -- 172.21.15.26:0/2267137917 wait complete. 2024-01-15T18:07:41.503 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:07:41.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.729+0000 7f0a9ca00640 1 Processor -- start 2024-01-15T18:07:41.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.729+0000 7f0a9ca00640 1 -- start start 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.730+0000 7f0a9ca00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0a98160920 0x7f0a980b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.730+0000 7f0a9ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a980cf9f0 0x7f0a980cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.730+0000 7f0a9ca00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a980d0a10 con 0x7f0a98161010 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.730+0000 7f0a9ca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980d03e0 con 0x7f0a980cf9f0 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.730+0000 7f0a9ca00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a980d1f50 con 0x7f0a98160920 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.731+0000 7f0a95e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a980cf9f0 0x7f0a980cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.731+0000 7f0a95e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a980cf9f0 0x7f0a980cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50620/0 (socket says 172.21.15.26:50620) 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.731+0000 7f0a95e00640 1 -- 172.21.15.26:0/4083175374 learned_addr learned my addr 172.21.15.26:0/4083175374 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.731+0000 7f0a96800640 1 --2- 172.21.15.26:0/4083175374 >> v2:172.21.15.124:3300/0 conn(0x7f0a98160920 0x7f0a980b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.731+0000 7f0a95e00640 1 -- 172.21.15.26:0/4083175374 >> v2:172.21.15.124:3300/0 conn(0x7f0a98160920 msgr2=0x7f0a980b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.731+0000 7f0a95e00640 1 --2- 172.21.15.26:0/4083175374 >> v2:172.21.15.124:3300/0 conn(0x7f0a98160920 0x7f0a980b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.731+0000 7f0a95e00640 1 -- 172.21.15.26:0/4083175374 >> v1:172.21.15.124:6789/0 conn(0x7f0a98161010 legacy=0x7f0a98161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.731+0000 7f0a95e00640 1 -- 172.21.15.26:0/4083175374 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a980cf080 con 0x7f0a980cf9f0 2024-01-15T18:07:41.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.733+0000 7f0a95e00640 1 --2- 172.21.15.26:0/4083175374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a980cf9f0 0x7f0a980cfea0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0a9808c060 tx=0x7f0a8002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea4f5ce057a89f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.733+0000 7f0a95400640 1 -- 172.21.15.26:0/4083175374 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8002ca80 con 0x7f0a980cf9f0 2024-01-15T18:07:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.733+0000 7f0a95400640 1 -- 172.21.15.26:0/4083175374 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8002cbe0 con 0x7f0a980cf9f0 2024-01-15T18:07:41.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.733+0000 7f0a95400640 1 -- 172.21.15.26:0/4083175374 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800343f0 con 0x7f0a980cf9f0 2024-01-15T18:07:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.734+0000 7f0a9ca00640 1 -- 172.21.15.26:0/4083175374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a980cf9f0 msgr2=0x7f0a980cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.734+0000 7f0a9ca00640 1 --2- 172.21.15.26:0/4083175374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a980cf9f0 0x7f0a980cfea0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f0a9808c060 tx=0x7f0a8002bee0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.734+0000 7f0a9ca00640 1 -- 172.21.15.26:0/4083175374 shutdown_connections 2024-01-15T18:07:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.734+0000 7f0a9ca00640 1 --2- 172.21.15.26:0/4083175374 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a980cf9f0 0x7f0a980cfea0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.734+0000 7f0a9ca00640 1 --2- 172.21.15.26:0/4083175374 >> v2:172.21.15.124:3300/0 conn(0x7f0a98160920 0x7f0a980b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.734+0000 7f0a9ca00640 1 -- 172.21.15.26:0/4083175374 >> 172.21.15.26:0/4083175374 conn(0x7f0a980d3090 msgr2=0x7f0a980d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.734+0000 7f0a9ca00640 1 -- 172.21.15.26:0/4083175374 shutdown_connections 2024-01-15T18:07:41.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.734+0000 7f0a9ca00640 1 -- 172.21.15.26:0/4083175374 wait complete. 2024-01-15T18:07:41.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.736+0000 7f0a9ca00640 1 Processor -- start 2024-01-15T18:07:41.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.736+0000 7f0a9ca00640 1 -- start start 2024-01-15T18:07:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a9ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a980cf9f0 0x7f0a98150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a9ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98160920 0x7f0a980b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a9ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98161010 0x7f0a980cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a9ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980cb650 con 0x7f0a98161010 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a9ca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980cb930 con 0x7f0a980cf9f0 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a9ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980cbc10 con 0x7f0a98160920 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a97200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98161010 0x7f0a980cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a97200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98161010 0x7f0a980cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50630/0 (socket says 172.21.15.26:50630) 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a97200640 1 -- 172.21.15.26:0/1537631505 learned_addr learned my addr 172.21.15.26:0/1537631505 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.738+0000 7f0a95e00640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98160920 0x7f0a980b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a96800640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a980cf9f0 0x7f0a98150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a97200640 1 -- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98160920 msgr2=0x7f0a980b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a97200640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98160920 0x7f0a980b68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a97200640 1 -- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a980cf9f0 msgr2=0x7f0a98150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:41.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a97200640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a980cf9f0 0x7f0a98150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a97200640 1 -- 172.21.15.26:0/1537631505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a80009b30 con 0x7f0a98161010 2024-01-15T18:07:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a95e00640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98160920 0x7f0a980b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a97200640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98161010 0x7f0a980cc6f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800c390 tx=0x7f0a8800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:41.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a8ea00640 1 -- 172.21.15.26:0/1537631505 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a880035f0 con 0x7f0a98161010 2024-01-15T18:07:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a8ea00640 1 -- 172.21.15.26:0/1537631505 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a88003750 con 0x7f0a98161010 2024-01-15T18:07:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a8ea00640 1 -- 172.21.15.26:0/1537631505 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a88003b00 con 0x7f0a98161010 2024-01-15T18:07:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.739+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a980cbef0 con 0x7f0a98161010 2024-01-15T18:07:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.740+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a981508d0 con 0x7f0a98161010 2024-01-15T18:07:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.741+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a980cc410 con 0x7f0a98161010 2024-01-15T18:07:41.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.742+0000 7f0a8ea00640 1 -- 172.21.15.26:0/1537631505 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f0a88003c60 con 0x7f0a98161010 2024-01-15T18:07:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.742+0000 7f0a8ea00640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c040 0x7f0a7006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:41.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.743+0000 7f0a8ea00640 1 -- 172.21.15.26:0/1537631505 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f0a88087b40 con 0x7f0a98161010 2024-01-15T18:07:41.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.743+0000 7f0a96800640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c040 0x7f0a7006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:41.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.744+0000 7f0a96800640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c040 0x7f0a7006e500 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0a90006fd0 tx=0x7f0a90008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:41.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.748+0000 7f0a8ea00640 1 -- 172.21.15.26:0/1537631505 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a88054150 con 0x7f0a98161010 2024-01-15T18:07:41.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:41.979+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a980cffd0 con 0x7f0a7006c040 2024-01-15T18:07:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.052+0000 7f0a8ea00640 1 -- 172.21.15.26:0/1537631505 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0a980cffd0 con 0x7f0a7006c040 2024-01-15T18:07:42.056 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:07:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c040 msgr2=0x7f0a7006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c040 0x7f0a7006e500 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0a90006fd0 tx=0x7f0a90008040 comp rx=0 tx=0).stop 2024-01-15T18:07:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98161010 msgr2=0x7f0a980cc6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98161010 0x7f0a980cc6f0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f0a8800c390 tx=0x7f0a8800c850 comp rx=0 tx=0).stop 2024-01-15T18:07:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 shutdown_connections 2024-01-15T18:07:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98160920 0x7f0a980b68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c040 0x7f0a7006e500 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98161010 0x7f0a980cc6f0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 --2- 172.21.15.26:0/1537631505 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a980cf9f0 0x7f0a98150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.055+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 >> 172.21.15.26:0/1537631505 conn(0x7f0a980d3090 msgr2=0x7f0a9806b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.056+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 shutdown_connections 2024-01-15T18:07:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.056+0000 7f0a9ca00640 1 -- 172.21.15.26:0/1537631505 wait complete. 2024-01-15T18:07:42.081 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:07:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.294+0000 7ff397e00640 1 Processor -- start 2024-01-15T18:07:42.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.295+0000 7ff397e00640 1 -- start start 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.296+0000 7ff397e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3980ccb50 0x7ff398070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.296+0000 7ff397e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39815f4e0 con 0x7ff3980adf20 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.296+0000 7ff397e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff398071130 con 0x7ff39815bf10 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.296+0000 7ff397e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff39815f970 con 0x7ff3980ccb50 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.297+0000 7ff396000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3980ccb50 0x7ff398070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.297+0000 7ff396000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff3980ccb50 0x7ff398070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40904/0 (socket says 172.21.15.26:40904) 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.297+0000 7ff396000640 1 -- 172.21.15.26:0/353862248 learned_addr learned my addr 172.21.15.26:0/353862248 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.297+0000 7ff396a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff3980adf20 0x7ff3980cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45608/0 (socket says 172.21.15.26:45608) 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.297+0000 7ff396000640 1 -- 172.21.15.26:0/353862248 >> v1:172.21.15.124:6789/0 conn(0x7ff39815bf10 legacy=0x7ff398143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.297+0000 7ff396000640 1 -- 172.21.15.26:0/353862248 >> v1:172.21.15.26:6789/0 conn(0x7ff3980adf20 legacy=0x7ff3980cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.297+0000 7ff396000640 1 -- 172.21.15.26:0/353862248 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39815ebb0 con 0x7ff3980ccb50 2024-01-15T18:07:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.299+0000 7ff396000640 1 --2- 172.21.15.26:0/353862248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3980ccb50 0x7ff398070a30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff380009870 tx=0x7ff38002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94cee82ce4c7412 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.299+0000 7ff395600640 1 -- 172.21.15.26:0/353862248 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380031870 con 0x7ff3980ccb50 2024-01-15T18:07:42.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.299+0000 7ff395600640 1 -- 172.21.15.26:0/353862248 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3800319d0 con 0x7ff3980ccb50 2024-01-15T18:07:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.299+0000 7ff395600640 1 -- 172.21.15.26:0/353862248 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff380031d60 con 0x7ff3980ccb50 2024-01-15T18:07:42.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.299+0000 7ff397e00640 1 -- 172.21.15.26:0/353862248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3980ccb50 msgr2=0x7ff398070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.299+0000 7ff397e00640 1 --2- 172.21.15.26:0/353862248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3980ccb50 0x7ff398070a30 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff380009870 tx=0x7ff38002b910 comp rx=0 tx=0).stop 2024-01-15T18:07:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.300+0000 7ff397e00640 1 -- 172.21.15.26:0/353862248 shutdown_connections 2024-01-15T18:07:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.300+0000 7ff397e00640 1 --2- 172.21.15.26:0/353862248 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3980ccb50 0x7ff398070a30 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.300+0000 7ff397e00640 1 -- 172.21.15.26:0/353862248 >> 172.21.15.26:0/353862248 conn(0x7ff39806a7d0 msgr2=0x7ff39806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.300+0000 7ff397e00640 1 -- 172.21.15.26:0/353862248 shutdown_connections 2024-01-15T18:07:42.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.300+0000 7ff397e00640 1 -- 172.21.15.26:0/353862248 wait complete. 2024-01-15T18:07:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.302+0000 7ff397e00640 1 Processor -- start 2024-01-15T18:07:42.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.302+0000 7ff397e00640 1 -- start start 2024-01-15T18:07:42.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.304+0000 7ff397e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3980adf20 0x7ff3980c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.304+0000 7ff397e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3980ccb50 0x7ff3981302c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.304+0000 7ff397e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39815bf10 0x7ff39813f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.304+0000 7ff397e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3981313a0 con 0x7ff39815bf10 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.304+0000 7ff397e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3980c4370 con 0x7ff3980adf20 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.304+0000 7ff397e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff398130a00 con 0x7ff3980ccb50 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.304+0000 7ff397400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39815bf10 0x7ff39813f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.304+0000 7ff397400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39815bf10 0x7ff39813f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40918/0 (socket says 172.21.15.26:40918) 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.304+0000 7ff397400640 1 -- 172.21.15.26:0/2632568922 learned_addr learned my addr 172.21.15.26:0/2632568922 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.305+0000 7ff396a00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3980adf20 0x7ff3980c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.305+0000 7ff396000640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3980ccb50 0x7ff3981302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.305+0000 7ff397400640 1 -- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3980ccb50 msgr2=0x7ff3981302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.305+0000 7ff397400640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3980ccb50 0x7ff3981302c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.305+0000 7ff397400640 1 -- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3980adf20 msgr2=0x7ff3980c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.305+0000 7ff397400640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3980adf20 0x7ff3980c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.305+0000 7ff397400640 1 -- 172.21.15.26:0/2632568922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3800094d0 con 0x7ff39815bf10 2024-01-15T18:07:42.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.305+0000 7ff397400640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39815bf10 0x7ff39813f890 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff39000c780 tx=0x7ff39000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:42.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.306+0000 7ff38ea00640 1 -- 172.21.15.26:0/2632568922 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff390004b60 con 0x7ff39815bf10 2024-01-15T18:07:42.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.306+0000 7ff38ea00640 1 -- 172.21.15.26:0/2632568922 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39000ce90 con 0x7ff39815bf10 2024-01-15T18:07:42.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.306+0000 7ff38ea00640 1 -- 172.21.15.26:0/2632568922 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39000a850 con 0x7ff39815bf10 2024-01-15T18:07:42.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.306+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff398130ce0 con 0x7ff39815bf10 2024-01-15T18:07:42.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.306+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39813ff10 con 0x7ff39815bf10 2024-01-15T18:07:42.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.307+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3981314e0 con 0x7ff39815bf10 2024-01-15T18:07:42.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.308+0000 7ff38ea00640 1 -- 172.21.15.26:0/2632568922 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7ff39000a9b0 con 0x7ff39815bf10 2024-01-15T18:07:42.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.309+0000 7ff38ea00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff37006c040 0x7ff37006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.309+0000 7ff38ea00640 1 -- 172.21.15.26:0/2632568922 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7ff390088a70 con 0x7ff39815bf10 2024-01-15T18:07:42.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.310+0000 7ff396a00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff37006c040 0x7ff37006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.311+0000 7ff396a00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff37006c040 0x7ff37006e500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff388000f60 tx=0x7ff388007c80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:42.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.314+0000 7ff38ea00640 1 -- 172.21.15.26:0/2632568922 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff3981314e0 con 0x7ff39815bf10 2024-01-15T18:07:42.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.534+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3980b0ca0 con 0x7ff37006c040 2024-01-15T18:07:42.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.577+0000 7ff38ea00640 1 -- 172.21.15.26:0/2632568922 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff3980b0ca0 con 0x7ff37006c040 2024-01-15T18:07:42.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff37006c040 msgr2=0x7ff37006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff37006c040 0x7ff37006e500 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff388000f60 tx=0x7ff388007c80 comp rx=0 tx=0).stop 2024-01-15T18:07:42.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39815bf10 msgr2=0x7ff39813f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39815bf10 0x7ff39813f890 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff39000c780 tx=0x7ff39000cc40 comp rx=0 tx=0).stop 2024-01-15T18:07:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 shutdown_connections 2024-01-15T18:07:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff3980ccb50 0x7ff3981302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff37006c040 0x7ff37006e500 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff39815bf10 0x7ff39813f890 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 --2- 172.21.15.26:0/2632568922 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff3980adf20 0x7ff3980c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 >> 172.21.15.26:0/2632568922 conn(0x7ff39806a7d0 msgr2=0x7ff3980713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 shutdown_connections 2024-01-15T18:07:42.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.580+0000 7ff397e00640 1 -- 172.21.15.26:0/2632568922 wait complete. 2024-01-15T18:07:42.604 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone4187 2024-01-15T18:07:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.806+0000 7fad3c800640 1 Processor -- start 2024-01-15T18:07:42.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.806+0000 7fad3c800640 1 -- start start 2024-01-15T18:07:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.808+0000 7fad3c800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fad34143ae0 0x7fad340cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.808+0000 7fad3c800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad3415e3b0 0x7fad340702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.808+0000 7fad3c800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad340cfc30 con 0x7fad340cf7d0 2024-01-15T18:07:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.808+0000 7fad3c800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340b1ee0 con 0x7fad3415e3b0 2024-01-15T18:07:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.808+0000 7fad3c800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340b2780 con 0x7fad34143ae0 2024-01-15T18:07:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.808+0000 7fad3a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad3415e3b0 0x7fad340702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.808+0000 7fad3a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad3415e3b0 0x7fad340702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40924/0 (socket says 172.21.15.26:40924) 2024-01-15T18:07:42.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.808+0000 7fad3a800640 1 -- 172.21.15.26:0/1507307701 learned_addr learned my addr 172.21.15.26:0/1507307701 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:42.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.809+0000 7fad39e00640 1 --2- 172.21.15.26:0/1507307701 >> v2:172.21.15.124:3300/0 conn(0x7fad34143ae0 0x7fad340cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.809+0000 7fad3a800640 1 -- 172.21.15.26:0/1507307701 >> v2:172.21.15.124:3300/0 conn(0x7fad34143ae0 msgr2=0x7fad340cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:42.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.809+0000 7fad3a800640 1 --2- 172.21.15.26:0/1507307701 >> v2:172.21.15.124:3300/0 conn(0x7fad34143ae0 0x7fad340cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.809+0000 7fad3a800640 1 -- 172.21.15.26:0/1507307701 >> v1:172.21.15.124:6789/0 conn(0x7fad340cf7d0 legacy=0x7fad3415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:42.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.809+0000 7fad3a800640 1 -- 172.21.15.26:0/1507307701 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3415e990 con 0x7fad3415e3b0 2024-01-15T18:07:42.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.811+0000 7fad3a800640 1 --2- 172.21.15.26:0/1507307701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 0x7fad340702c0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fad340cfea0 tx=0x7fad2c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5169e0f836bb49cb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:42.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.811+0000 7fad38a00640 1 -- 172.21.15.26:0/1507307701 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c02ca80 con 0x7fad3415e3b0 2024-01-15T18:07:42.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.811+0000 7fad38a00640 1 -- 172.21.15.26:0/1507307701 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c02cbe0 con 0x7fad3415e3b0 2024-01-15T18:07:42.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.811+0000 7fad38a00640 1 -- 172.21.15.26:0/1507307701 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0343f0 con 0x7fad3415e3b0 2024-01-15T18:07:42.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.812+0000 7fad3c800640 1 -- 172.21.15.26:0/1507307701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 msgr2=0x7fad340702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.812+0000 7fad3c800640 1 --2- 172.21.15.26:0/1507307701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 0x7fad340702c0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fad340cfea0 tx=0x7fad2c02bee0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.812+0000 7fad3c800640 1 -- 172.21.15.26:0/1507307701 shutdown_connections 2024-01-15T18:07:42.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.812+0000 7fad3c800640 1 --2- 172.21.15.26:0/1507307701 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 0x7fad340702c0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.812+0000 7fad3c800640 1 --2- 172.21.15.26:0/1507307701 >> v2:172.21.15.124:3300/0 conn(0x7fad34143ae0 0x7fad340cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.812+0000 7fad3c800640 1 -- 172.21.15.26:0/1507307701 >> 172.21.15.26:0/1507307701 conn(0x7fad3406a800 msgr2=0x7fad3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:42.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.812+0000 7fad3c800640 1 -- 172.21.15.26:0/1507307701 shutdown_connections 2024-01-15T18:07:42.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.812+0000 7fad3c800640 1 -- 172.21.15.26:0/1507307701 wait complete. 2024-01-15T18:07:42.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.814+0000 7fad3c800640 1 Processor -- start 2024-01-15T18:07:42.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.814+0000 7fad3c800640 1 -- start start 2024-01-15T18:07:42.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.814+0000 7fad3c800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf7d0 0x7fad34138a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.814+0000 7fad3c800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad34143ae0 0x7fad340ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.815+0000 7fad3c800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 0x7fad340ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.815+0000 7fad3c800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3413eb30 con 0x7fad3415e3b0 2024-01-15T18:07:42.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.815+0000 7fad3c800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad34139250 con 0x7fad340cf7d0 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.815+0000 7fad3c800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3413a0b0 con 0x7fad34143ae0 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.815+0000 7fad3a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 0x7fad340ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.815+0000 7fad3a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 0x7fad340ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40940/0 (socket says 172.21.15.26:40940) 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.815+0000 7fad3a800640 1 -- 172.21.15.26:0/415658916 learned_addr learned my addr 172.21.15.26:0/415658916 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.815+0000 7fad39400640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad34143ae0 0x7fad340ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.815+0000 7fad39e00640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf7d0 0x7fad34138a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad3a800640 1 -- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad34143ae0 msgr2=0x7fad340ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad3a800640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad34143ae0 0x7fad340ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad3a800640 1 -- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf7d0 msgr2=0x7fad34138a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad3a800640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf7d0 0x7fad34138a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:42.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad3a800640 1 -- 172.21.15.26:0/415658916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c009b30 con 0x7fad3415e3b0 2024-01-15T18:07:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad3a800640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 0x7fad340ace10 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fad2c036040 tx=0x7fad2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad32000640 1 -- 172.21.15.26:0/415658916 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c009510 con 0x7fad3415e3b0 2024-01-15T18:07:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad32000640 1 -- 172.21.15.26:0/415658916 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c03e450 con 0x7fad3415e3b0 2024-01-15T18:07:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad32000640 1 -- 172.21.15.26:0/415658916 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c03e800 con 0x7fad3415e3b0 2024-01-15T18:07:42.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.816+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad341394a0 con 0x7fad3415e3b0 2024-01-15T18:07:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.817+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad34139a60 con 0x7fad3415e3b0 2024-01-15T18:07:42.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.818+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3413a260 con 0x7fad3415e3b0 2024-01-15T18:07:42.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.820+0000 7fad32000640 1 -- 172.21.15.26:0/415658916 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fad2c02cd50 con 0x7fad3415e3b0 2024-01-15T18:07:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.820+0000 7fad32000640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c040 0x7fad0c06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.821+0000 7fad32000640 1 -- 172.21.15.26:0/415658916 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fad2c0ab810 con 0x7fad3415e3b0 2024-01-15T18:07:42.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.821+0000 7fad39e00640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c040 0x7fad0c06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:42.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.822+0000 7fad39e00640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c040 0x7fad0c06e500 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad24006fd0 tx=0x7fad24008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:42.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:42.825+0000 7fad32000640 1 -- 172.21.15.26:0/415658916 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad3413a260 con 0x7fad3415e3b0 2024-01-15T18:07:43.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.046+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone4187", "target": ["mon-mgr", ""]}) v1 -- 0x7fad340b2410 con 0x7fad0c06c040 2024-01-15T18:07:43.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.088+0000 7fad32000640 1 -- 172.21.15.26:0/415658916 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fad2400b200 con 0x7fad0c06c040 2024-01-15T18:07:43.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c040 msgr2=0x7fad0c06e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c040 0x7fad0c06e500 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fad24006fd0 tx=0x7fad24008040 comp rx=0 tx=0).stop 2024-01-15T18:07:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 msgr2=0x7fad340ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 0x7fad340ace10 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fad2c036040 tx=0x7fad2c004a40 comp rx=0 tx=0).stop 2024-01-15T18:07:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 shutdown_connections 2024-01-15T18:07:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad34143ae0 0x7fad340ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c040 0x7fad0c06e500 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad3415e3b0 0x7fad340ace10 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 --2- 172.21.15.26:0/415658916 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf7d0 0x7fad34138a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 >> 172.21.15.26:0/415658916 conn(0x7fad3406a800 msgr2=0x7fad340b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:43.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 shutdown_connections 2024-01-15T18:07:43.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.091+0000 7fad3c800640 1 -- 172.21.15.26:0/415658916 wait complete. 2024-01-15T18:07:43.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41818 2024-01-15T18:07:43.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.315+0000 7fbdd3e00640 1 Processor -- start 2024-01-15T18:07:43.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.315+0000 7fbdd3e00640 1 -- start start 2024-01-15T18:07:43.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.316+0000 7fbdd3e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd415d3c0 0x7fbdd40d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.317+0000 7fbdd3e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbdd40b4110 0x7fbdd40cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.317+0000 7fbdd3e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd40b2c70 con 0x7fbdd40cfa40 2024-01-15T18:07:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.317+0000 7fbdd3e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd40d03c0 con 0x7fbdd40b4110 2024-01-15T18:07:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.317+0000 7fbdd3e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40cf6d0 con 0x7fbdd415d3c0 2024-01-15T18:07:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.317+0000 7fbdcbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbdd40cfa40 0x7fbdd40cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45616/0 (socket says 172.21.15.26:45616) 2024-01-15T18:07:43.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.317+0000 7fbdcbe00640 1 -- 172.21.15.26:0/1548195763 learned_addr learned my addr 172.21.15.26:0/1548195763 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:43.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.317+0000 7fbdd3400640 1 --2- 172.21.15.26:0/1548195763 >> v2:172.21.15.124:3300/0 conn(0x7fbdd40b4110 0x7fbdd40cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.317+0000 7fbdd2a00640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd415d3c0 0x7fbdd40d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.317+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052740644 0 0) 0x7fbdd40b2c70 con 0x7fbdd40cfa40 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb4003520 con 0x7fbdd40cfa40 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 471653871 0 0) 0x7fbdb4003520 con 0x7fbdd40cfa40 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd415d3c0 msgr2=0x7fbdd40d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd415d3c0 0x7fbdd40d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 >> v2:172.21.15.124:3300/0 conn(0x7fbdd40b4110 msgr2=0x7fbdd40cfe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 --2- 172.21.15.26:0/1548195763 >> v2:172.21.15.124:3300/0 conn(0x7fbdd40b4110 0x7fbdd40cfe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd40cf3f0 con 0x7fbdd40cfa40 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fbdbc001670 con 0x7fbdd40cfa40 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 >> v1:172.21.15.26:6789/0 conn(0x7fbdd40cfa40 legacy=0x7fbdd40cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd415d3c0 0x7fbdb4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdb4004d00 0x7fbdb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40d03c0 con 0x7fbdb4003ef0 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40cf6d0 con 0x7fbdb4004d00 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb400b980 con 0x7fbdd415d3c0 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.318+0000 7fbdcbe00640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.319+0000 7fbdcbe00640 1 -- 172.21.15.26:0/1548195763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd415d3c0 msgr2=0x7fbdb4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.319+0000 7fbdcbe00640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd415d3c0 0x7fbdb4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.319+0000 7fbdcbe00640 1 -- 172.21.15.26:0/1548195763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdb4004d00 msgr2=0x7fbdb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.319+0000 7fbdcbe00640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdb4004d00 0x7fbdb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.319+0000 7fbdcbe00640 1 -- 172.21.15.26:0/1548195763 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb4003520 con 0x7fbdb4003ef0 2024-01-15T18:07:43.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.320+0000 7fbdcbe00640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdb40047c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00c210 tx=0x7fbdb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad18d5a9be9e179 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.320+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc005ad0 con 0x7fbdb4003ef0 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.320+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc005c30 con 0x7fbdb4003ef0 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.320+0000 7fbdd2000640 1 -- 172.21.15.26:0/1548195763 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc02caf0 con 0x7fbdb4003ef0 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.321+0000 7fbdd3e00640 1 -- 172.21.15.26:0/1548195763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 msgr2=0x7fbdb40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.321+0000 7fbdd3e00640 1 --2- 172.21.15.26:0/1548195763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdb40047c0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00c210 tx=0x7fbdb40057f0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.321+0000 7fbdd3400640 1 -- 172.21.15.26:0/1548195763 reap_dead start 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.321+0000 7fbdd3e00640 1 -- 172.21.15.26:0/1548195763 shutdown_connections 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.321+0000 7fbdd3e00640 1 -- 172.21.15.26:0/1548195763 >> 172.21.15.26:0/1548195763 conn(0x7fbdd40d3050 msgr2=0x7fbdd40d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.321+0000 7fbdd3e00640 1 -- 172.21.15.26:0/1548195763 shutdown_connections 2024-01-15T18:07:43.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.321+0000 7fbdd3e00640 1 -- 172.21.15.26:0/1548195763 wait complete. 2024-01-15T18:07:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.323+0000 7fbdd3e00640 1 Processor -- start 2024-01-15T18:07:43.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.323+0000 7fbdd3e00640 1 -- start start 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.324+0000 7fbdd3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdd4155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.324+0000 7fbdd3e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd40b0e30 0x7fbdd40cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.324+0000 7fbdd3e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd41563a0 0x7fbdd40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.324+0000 7fbdd3e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd4156d60 con 0x7fbdb4003ef0 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.324+0000 7fbdd3e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40cb160 con 0x7fbdd41563a0 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.324+0000 7fbdd3e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40ccff0 con 0x7fbdd40b0e30 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.324+0000 7fbdd2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdd4155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.324+0000 7fbdd2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdd4155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40958/0 (socket says 172.21.15.26:40958) 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.324+0000 7fbdd2a00640 1 -- 172.21.15.26:0/3793630974 learned_addr learned my addr 172.21.15.26:0/3793630974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:43.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdd2a00640 1 -- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd40b0e30 msgr2=0x7fbdd40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdd2a00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd40b0e30 0x7fbdd40cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdd2a00640 1 -- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd41563a0 msgr2=0x7fbdd40ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdd2a00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd41563a0 0x7fbdd40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdd2a00640 1 -- 172.21.15.26:0/3793630974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd40cae80 con 0x7fbdb4003ef0 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdd2a00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdd4155d10 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400e7f0 tx=0x7fbdc400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdcaa00640 1 -- 172.21.15.26:0/3793630974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc400c860 con 0x7fbdb4003ef0 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdcaa00640 1 -- 172.21.15.26:0/3793630974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc400c9c0 con 0x7fbdb4003ef0 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdcaa00640 1 -- 172.21.15.26:0/3793630974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc400cd50 con 0x7fbdb4003ef0 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd40b6e90 con 0x7fbdb4003ef0 2024-01-15T18:07:43.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.325+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd40b7450 con 0x7fbdb4003ef0 2024-01-15T18:07:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.327+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd408ba80 con 0x7fbdb4003ef0 2024-01-15T18:07:43.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.327+0000 7fbdcaa00640 1 -- 172.21.15.26:0/3793630974 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fbdc4013470 con 0x7fbdb4003ef0 2024-01-15T18:07:43.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.328+0000 7fbdcaa00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbdac06c070 0x7fbdac06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.329+0000 7fbdcaa00640 1 -- 172.21.15.26:0/3793630974 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fbdc40887d0 con 0x7fbdb4003ef0 2024-01-15T18:07:43.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.329+0000 7fbdcbe00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbdac06c070 0x7fbdac06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.331+0000 7fbdcbe00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbdac06c070 0x7fbdac06e530 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc034000 tx=0x7fbdbc02e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:43.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.334+0000 7fbdcaa00640 1 -- 172.21.15.26:0/3793630974 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbdc4054e40 con 0x7fbdb4003ef0 2024-01-15T18:07:43.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.560+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41818", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdd40b2900 con 0x7fbdac06c070 2024-01-15T18:07:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.597+0000 7fbdcaa00640 1 -- 172.21.15.26:0/3793630974 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbdbc001430 con 0x7fbdac06c070 2024-01-15T18:07:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbdac06c070 msgr2=0x7fbdac06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbdac06c070 0x7fbdac06e530 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc034000 tx=0x7fbdbc02e000 comp rx=0 tx=0).stop 2024-01-15T18:07:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 msgr2=0x7fbdd4155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdd4155d10 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400e7f0 tx=0x7fbdc400ecb0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 shutdown_connections 2024-01-15T18:07:43.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbdd40b0e30 0x7fbdd40cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbdac06c070 0x7fbdac06e530 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbdd41563a0 0x7fbdd40ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 --2- 172.21.15.26:0/3793630974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbdb4003ef0 0x7fbdd4155d10 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 >> 172.21.15.26:0/3793630974 conn(0x7fbdd40d3050 msgr2=0x7fbdd406c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 shutdown_connections 2024-01-15T18:07:43.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.600+0000 7fbdd3e00640 1 -- 172.21.15.26:0/3793630974 wait complete. 2024-01-15T18:07:43.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41819 2024-01-15T18:07:43.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.827+0000 7f3b86e00640 1 Processor -- start 2024-01-15T18:07:43.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.827+0000 7f3b86e00640 1 -- start start 2024-01-15T18:07:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.828+0000 7f3b86e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3b80160920 0x7f3b800b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.828+0000 7f3b86e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b800cf9f0 0x7f3b800cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.828+0000 7f3b86e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b800d0a10 con 0x7f3b80161010 2024-01-15T18:07:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.828+0000 7f3b86e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b800d03b0 con 0x7f3b80160920 2024-01-15T18:07:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.828+0000 7f3b86e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800d1f50 con 0x7f3b800cf9f0 2024-01-15T18:07:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.829+0000 7f3b85a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3b80160920 0x7f3b800b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.829+0000 7f3b85a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3b80160920 0x7f3b800b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40960/0 (socket says 172.21.15.26:40960) 2024-01-15T18:07:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.829+0000 7f3b85a00640 1 -- 172.21.15.26:0/539034532 learned_addr learned my addr 172.21.15.26:0/539034532 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:43.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.829+0000 7f3b85000640 1 --2- 172.21.15.26:0/539034532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b800cf9f0 0x7f3b800cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.829+0000 7f3b85a00640 1 -- 172.21.15.26:0/539034532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b800cf9f0 msgr2=0x7f3b800cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.829+0000 7f3b85a00640 1 --2- 172.21.15.26:0/539034532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b800cf9f0 0x7f3b800cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.829+0000 7f3b85a00640 1 -- 172.21.15.26:0/539034532 >> v1:172.21.15.124:6789/0 conn(0x7f3b80161010 legacy=0x7f3b801614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:43.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.829+0000 7f3b85a00640 1 -- 172.21.15.26:0/539034532 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b800cf080 con 0x7f3b80160920 2024-01-15T18:07:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.830+0000 7f3b85a00640 1 --2- 172.21.15.26:0/539034532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80160920 0x7f3b800b3f30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009e50 tx=0x7f3b7002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab50c5e6daa126b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b7fe00640 1 -- 172.21.15.26:0/539034532 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7002ca70 con 0x7f3b80160920 2024-01-15T18:07:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b7fe00640 1 -- 172.21.15.26:0/539034532 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7002cbd0 con 0x7f3b80160920 2024-01-15T18:07:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b7fe00640 1 -- 172.21.15.26:0/539034532 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b700343f0 con 0x7f3b80160920 2024-01-15T18:07:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b86e00640 1 -- 172.21.15.26:0/539034532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80160920 msgr2=0x7f3b800b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b86e00640 1 --2- 172.21.15.26:0/539034532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80160920 0x7f3b800b3f30 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f3b70009e50 tx=0x7f3b7002bed0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b86e00640 1 -- 172.21.15.26:0/539034532 shutdown_connections 2024-01-15T18:07:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b86e00640 1 --2- 172.21.15.26:0/539034532 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b800cf9f0 0x7f3b800cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b86e00640 1 --2- 172.21.15.26:0/539034532 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80160920 0x7f3b800b3f30 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b86e00640 1 -- 172.21.15.26:0/539034532 >> 172.21.15.26:0/539034532 conn(0x7f3b800d3090 msgr2=0x7f3b800d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b86e00640 1 -- 172.21.15.26:0/539034532 shutdown_connections 2024-01-15T18:07:43.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.831+0000 7f3b86e00640 1 -- 172.21.15.26:0/539034532 wait complete. 2024-01-15T18:07:43.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.834+0000 7f3b86e00640 1 Processor -- start 2024-01-15T18:07:43.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.834+0000 7f3b86e00640 1 -- start start 2024-01-15T18:07:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.835+0000 7f3b86e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b800cf9f0 0x7f3b8015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.835+0000 7f3b86e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b80160920 0x7f3b8015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.835+0000 7f3b86e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80161010 0x7f3b8006c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.835+0000 7f3b86e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b80160200 con 0x7f3b80161010 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.835+0000 7f3b86e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b800b7220 con 0x7f3b800cf9f0 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.835+0000 7f3b86e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8015f960 con 0x7f3b80160920 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.835+0000 7f3b86400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80161010 0x7f3b8006c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.835+0000 7f3b86400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80161010 0x7f3b8006c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40962/0 (socket says 172.21.15.26:40962) 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.835+0000 7f3b86400640 1 -- 172.21.15.26:0/1390784366 learned_addr learned my addr 172.21.15.26:0/1390784366 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.836+0000 7f3b85a00640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b800cf9f0 0x7f3b8015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.836+0000 7f3b85000640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b80160920 0x7f3b8015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.836+0000 7f3b86400640 1 -- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b80160920 msgr2=0x7f3b8015f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.836+0000 7f3b86400640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b80160920 0x7f3b8015f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.836+0000 7f3b86400640 1 -- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b800cf9f0 msgr2=0x7f3b8015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:43.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.836+0000 7f3b86400640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b800cf9f0 0x7f3b8015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.836+0000 7f3b86400640 1 -- 172.21.15.26:0/1390784366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b70009b30 con 0x7f3b80161010 2024-01-15T18:07:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.836+0000 7f3b86400640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80161010 0x7f3b8006c240 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800b410 tx=0x7f3b7800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.836+0000 7f3b7d600640 1 -- 172.21.15.26:0/1390784366 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78004020 con 0x7f3b80161010 2024-01-15T18:07:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.837+0000 7f3b7d600640 1 -- 172.21.15.26:0/1390784366 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b78004180 con 0x7f3b80161010 2024-01-15T18:07:43.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.837+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8015fc40 con 0x7f3b80161010 2024-01-15T18:07:43.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.837+0000 7f3b7d600640 1 -- 172.21.15.26:0/1390784366 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b78011630 con 0x7f3b80161010 2024-01-15T18:07:43.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.837+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8015a5d0 con 0x7f3b80161010 2024-01-15T18:07:43.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.838+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b800b73d0 con 0x7f3b80161010 2024-01-15T18:07:43.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.839+0000 7f3b7d600640 1 -- 172.21.15.26:0/1390784366 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f3b78011790 con 0x7f3b80161010 2024-01-15T18:07:43.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.840+0000 7f3b7d600640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3b5806c070 0x7f3b5806e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:43.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.840+0000 7f3b7d600640 1 -- 172.21.15.26:0/1390784366 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f3b78088600 con 0x7f3b80161010 2024-01-15T18:07:43.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.840+0000 7f3b85a00640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3b5806c070 0x7f3b5806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:43.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.842+0000 7f3b85a00640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3b5806c070 0x7f3b5806e530 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3b70035000 tx=0x7f3b70030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:43.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:43.844+0000 7f3b7d600640 1 -- 172.21.15.26:0/1390784366 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b800b73d0 con 0x7f3b80161010 2024-01-15T18:07:44.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.068+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41819", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b800ceac0 con 0x7f3b5806c070 2024-01-15T18:07:44.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.104+0000 7f3b7d600640 1 -- 172.21.15.26:0/1390784366 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3b7003e430 con 0x7f3b5806c070 2024-01-15T18:07:44.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3b5806c070 msgr2=0x7f3b5806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3b5806c070 0x7f3b5806e530 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3b70035000 tx=0x7f3b70030000 comp rx=0 tx=0).stop 2024-01-15T18:07:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80161010 msgr2=0x7f3b8006c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80161010 0x7f3b8006c240 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800b410 tx=0x7f3b7800b8d0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 shutdown_connections 2024-01-15T18:07:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3b80160920 0x7f3b8015f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3b5806c070 0x7f3b5806e530 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3b80161010 0x7f3b8006c240 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 --2- 172.21.15.26:0/1390784366 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3b800cf9f0 0x7f3b8015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 >> 172.21.15.26:0/1390784366 conn(0x7f3b800d3090 msgr2=0x7f3b800b3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:44.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 shutdown_connections 2024-01-15T18:07:44.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.108+0000 7f3b86e00640 1 -- 172.21.15.26:0/1390784366 wait complete. 2024-01-15T18:07:44.135 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone4185 2024-01-15T18:07:44.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.351+0000 7f640ca00640 1 Processor -- start 2024-01-15T18:07:44.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.351+0000 7f640ca00640 1 -- start start 2024-01-15T18:07:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.352+0000 7f640ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64081607a0 0x7f64080b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f640ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6408160b80 0x7f64081617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f640ca00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64080b2b90 con 0x7f64080b3eb0 2024-01-15T18:07:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f640ca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64080d21d0 con 0x7f64081607a0 2024-01-15T18:07:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f640ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080b2e30 con 0x7f6408160b80 2024-01-15T18:07:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f6406800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64081607a0 0x7f64080b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f6406800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f64081607a0 0x7f64080b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40976/0 (socket says 172.21.15.26:40976) 2024-01-15T18:07:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f6406800640 1 -- 172.21.15.26:0/2807976511 learned_addr learned my addr 172.21.15.26:0/2807976511 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f6407200640 1 --2- 172.21.15.26:0/2807976511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6408160b80 0x7f64081617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f6406800640 1 -- 172.21.15.26:0/2807976511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6408160b80 msgr2=0x7f64081617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f6406800640 1 --2- 172.21.15.26:0/2807976511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6408160b80 0x7f64081617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f6406800640 1 -- 172.21.15.26:0/2807976511 >> v1:172.21.15.124:6789/0 conn(0x7f64080b3eb0 legacy=0x7f64081610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f6406800640 1 -- 172.21.15.26:0/2807976511 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080b3110 con 0x7f64081607a0 2024-01-15T18:07:44.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.353+0000 7f6407200640 1 --2- 172.21.15.26:0/2807976511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6408160b80 0x7f64081617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.354+0000 7f6406800640 1 --2- 172.21.15.26:0/2807976511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64081607a0 0x7f64080b4330 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009870 tx=0x7f63f802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c245a8e9cd95a7c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.355+0000 7f6405400640 1 -- 172.21.15.26:0/2807976511 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8031640 con 0x7f64081607a0 2024-01-15T18:07:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.355+0000 7f6405400640 1 -- 172.21.15.26:0/2807976511 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f80317a0 con 0x7f64081607a0 2024-01-15T18:07:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.355+0000 7f6405400640 1 -- 172.21.15.26:0/2807976511 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8031b30 con 0x7f64081607a0 2024-01-15T18:07:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.355+0000 7f640ca00640 1 -- 172.21.15.26:0/2807976511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64081607a0 msgr2=0x7f64080b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.355+0000 7f640ca00640 1 --2- 172.21.15.26:0/2807976511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64081607a0 0x7f64080b4330 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009870 tx=0x7f63f802b8f0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.356+0000 7f640ca00640 1 -- 172.21.15.26:0/2807976511 shutdown_connections 2024-01-15T18:07:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.356+0000 7f640ca00640 1 --2- 172.21.15.26:0/2807976511 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6408160b80 0x7f64081617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.356+0000 7f640ca00640 1 --2- 172.21.15.26:0/2807976511 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f64081607a0 0x7f64080b4330 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.356+0000 7f640ca00640 1 -- 172.21.15.26:0/2807976511 >> 172.21.15.26:0/2807976511 conn(0x7f640809c590 msgr2=0x7f64080d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.356+0000 7f640ca00640 1 -- 172.21.15.26:0/2807976511 shutdown_connections 2024-01-15T18:07:44.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.356+0000 7f640ca00640 1 -- 172.21.15.26:0/2807976511 wait complete. 2024-01-15T18:07:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.357+0000 7f640ca00640 1 Processor -- start 2024-01-15T18:07:44.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.358+0000 7f640ca00640 1 -- start start 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f640ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64080b3eb0 0x7f640815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f640ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64081607a0 0x7f64080b8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f640ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6408160b80 0x7f640806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f640ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081f3f40 con 0x7f6408160b80 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f640ca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640815de60 con 0x7f64080b3eb0 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f640ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080b8dd0 con 0x7f64081607a0 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6407200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6408160b80 0x7f640806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6407200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6408160b80 0x7f640806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40984/0 (socket says 172.21.15.26:40984) 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6407200640 1 -- 172.21.15.26:0/3718938637 learned_addr learned my addr 172.21.15.26:0/3718938637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6406800640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64080b3eb0 0x7f640815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6405e00640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64081607a0 0x7f64080b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6407200640 1 -- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64081607a0 msgr2=0x7f64080b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6407200640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64081607a0 0x7f64080b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6407200640 1 -- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64080b3eb0 msgr2=0x7f640815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6407200640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64080b3eb0 0x7f640815d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.359+0000 7f6407200640 1 -- 172.21.15.26:0/3718938637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80094d0 con 0x7f6408160b80 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.360+0000 7f6407200640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6408160b80 0x7f640806b9a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f640000cef0 tx=0x7f640000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.360+0000 7f63fea00640 1 -- 172.21.15.26:0/3718938637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640000d840 con 0x7f6408160b80 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.360+0000 7f63fea00640 1 -- 172.21.15.26:0/3718938637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640000d9c0 con 0x7f6408160b80 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.360+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64081f3000 con 0x7f6408160b80 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.360+0000 7f63fea00640 1 -- 172.21.15.26:0/3718938637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640000dd70 con 0x7f6408160b80 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.360+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64081f35a0 con 0x7f6408160b80 2024-01-15T18:07:44.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.361+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6408067450 con 0x7f6408160b80 2024-01-15T18:07:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.362+0000 7f63fea00640 1 -- 172.21.15.26:0/3718938637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f640001e3b0 con 0x7f6408160b80 2024-01-15T18:07:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.363+0000 7f63fea00640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f63e006c070 0x7f63e006e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.363+0000 7f63fea00640 1 -- 172.21.15.26:0/3718938637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7f6400089390 con 0x7f6408160b80 2024-01-15T18:07:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.364+0000 7f6406800640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f63e006c070 0x7f63e006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.365+0000 7f6406800640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f63e006c070 0x7f63e006e530 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f63f8036000 tx=0x7f63f803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:44.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.368+0000 7f63fea00640 1 -- 172.21.15.26:0/3718938637 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64000bee90 con 0x7f6408160b80 2024-01-15T18:07:44.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.586+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone4185", "target": ["mon-mgr", ""]}) v1 -- 0x7f64080cbc10 con 0x7f63e006c070 2024-01-15T18:07:44.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.629+0000 7f63fea00640 1 -- 172.21.15.26:0/3718938637 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f64080cbc10 con 0x7f63e006c070 2024-01-15T18:07:44.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.631+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f63e006c070 msgr2=0x7f63e006e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.631+0000 7f640ca00640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f63e006c070 0x7f63e006e530 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f63f8036000 tx=0x7f63f803c040 comp rx=0 tx=0).stop 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.631+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6408160b80 msgr2=0x7f640806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.631+0000 7f640ca00640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6408160b80 0x7f640806b9a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f640000cef0 tx=0x7f640000a710 comp rx=0 tx=0).stop 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.632+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 shutdown_connections 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.632+0000 7f640ca00640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f64081607a0 0x7f64080b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.632+0000 7f640ca00640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f63e006c070 0x7f63e006e530 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.632+0000 7f640ca00640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6408160b80 0x7f640806b9a0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.632+0000 7f640ca00640 1 --2- 172.21.15.26:0/3718938637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f64080b3eb0 0x7f640815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.632+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 >> 172.21.15.26:0/3718938637 conn(0x7f640809c590 msgr2=0x7f64080cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.632+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 shutdown_connections 2024-01-15T18:07:44.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.632+0000 7f640ca00640 1 -- 172.21.15.26:0/3718938637 wait complete. 2024-01-15T18:07:44.656 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41815 2024-01-15T18:07:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.921+0000 7fd9f1e00640 1 Processor -- start 2024-01-15T18:07:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.921+0000 7fd9f1e00640 1 -- start start 2024-01-15T18:07:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.922+0000 7fd9f1e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 0x7fd9ec0b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.922+0000 7fd9f1e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec1464c0 con 0x7fd9ec1460e0 2024-01-15T18:07:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.922+0000 7fd9f1e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec0ada80 con 0x7fd9ec0ae520 2024-01-15T18:07:44.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.922+0000 7fd9f1e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0ae100 con 0x7fd9ec06de00 2024-01-15T18:07:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.923+0000 7fd9f0a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd9ec1460e0 0x7fd9ec0703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45620/0 (socket says 172.21.15.26:45620) 2024-01-15T18:07:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.923+0000 7fd9f0a00640 1 -- 172.21.15.26:0/1700608195 learned_addr learned my addr 172.21.15.26:0/1700608195 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.923+0000 7fd9f1400640 1 --2- 172.21.15.26:0/1700608195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 0x7fd9ec0b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.923+0000 7fd9f1400640 1 -- 172.21.15.26:0/1700608195 >> v1:172.21.15.124:6789/0 conn(0x7fd9ec0ae520 legacy=0x7fd9ec06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.923+0000 7fd9f1400640 1 -- 172.21.15.26:0/1700608195 >> v1:172.21.15.26:6789/0 conn(0x7fd9ec1460e0 legacy=0x7fd9ec0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.923+0000 7fd9f1400640 1 -- 172.21.15.26:0/1700608195 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec0711f0 con 0x7fd9ec06de00 2024-01-15T18:07:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.923+0000 7fd9eb400640 1 -- 172.21.15.26:0/1700608195 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381619226 0 0) 0x7fd9ec0ada80 con 0x7fd9ec0ae520 2024-01-15T18:07:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.924+0000 7fd9f1400640 1 --2- 172.21.15.26:0/1700608195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 0x7fd9ec0b1f50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0713e0 tx=0x7fd9e402bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82cec7fa201d0dfc server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.925+0000 7fd9eb400640 1 -- 172.21.15.26:0/1700608195 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e402c9f0 con 0x7fd9ec06de00 2024-01-15T18:07:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.925+0000 7fd9eb400640 1 -- 172.21.15.26:0/1700608195 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e402cb50 con 0x7fd9ec06de00 2024-01-15T18:07:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.925+0000 7fd9eb400640 1 -- 172.21.15.26:0/1700608195 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4034440 con 0x7fd9ec06de00 2024-01-15T18:07:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.925+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1700608195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 msgr2=0x7fd9ec0b1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.925+0000 7fd9f1e00640 1 --2- 172.21.15.26:0/1700608195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 0x7fd9ec0b1f50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0713e0 tx=0x7fd9e402bcc0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.925+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1700608195 shutdown_connections 2024-01-15T18:07:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.925+0000 7fd9f1e00640 1 --2- 172.21.15.26:0/1700608195 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 0x7fd9ec0b1f50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.925+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1700608195 >> 172.21.15.26:0/1700608195 conn(0x7fd9ec0cf000 msgr2=0x7fd9ec0cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.925+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1700608195 shutdown_connections 2024-01-15T18:07:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.926+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1700608195 wait complete. 2024-01-15T18:07:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.927+0000 7fd9f1e00640 1 Processor -- start 2024-01-15T18:07:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.927+0000 7fd9f1e00640 1 -- start start 2024-01-15T18:07:44.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.928+0000 7fd9f1e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 0x7fd9ec1f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.928+0000 7fd9f1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ec0ae520 0x7fd9ec0c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.928+0000 7fd9f1e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1460e0 0x7fd9ec0c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.928+0000 7fd9f1e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec15e950 con 0x7fd9ec0ae520 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.928+0000 7fd9f1e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec15ec30 con 0x7fd9ec1460e0 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.928+0000 7fd9f1e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec15ef10 con 0x7fd9ec06de00 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.928+0000 7fd9ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ec0ae520 0x7fd9ec0c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9ebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ec0ae520 0x7fd9ec0c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41000/0 (socket says 172.21.15.26:41000) 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9ebe00640 1 -- 172.21.15.26:0/1238790906 learned_addr learned my addr 172.21.15.26:0/1238790906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9f0a00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 0x7fd9ec1f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9f1400640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1460e0 0x7fd9ec0c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9ebe00640 1 -- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 msgr2=0x7fd9ec1f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9ebe00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 0x7fd9ec1f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9ebe00640 1 -- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1460e0 msgr2=0x7fd9ec0c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9ebe00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1460e0 0x7fd9ec0c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9ebe00640 1 -- 172.21.15.26:0/1238790906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e4009a70 con 0x7fd9ec0ae520 2024-01-15T18:07:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.929+0000 7fd9ebe00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ec0ae520 0x7fd9ec0c6850 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c520 tx=0x7fd9d400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.930+0000 7fd9e8c00640 1 -- 172.21.15.26:0/1238790906 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d40035f0 con 0x7fd9ec0ae520 2024-01-15T18:07:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.930+0000 7fd9e8c00640 1 -- 172.21.15.26:0/1238790906 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d4003750 con 0x7fd9ec0ae520 2024-01-15T18:07:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.930+0000 7fd9e8c00640 1 -- 172.21.15.26:0/1238790906 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4003b20 con 0x7fd9ec0ae520 2024-01-15T18:07:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.930+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec15f1f0 con 0x7fd9ec0ae520 2024-01-15T18:07:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.930+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec1f5dd0 con 0x7fd9ec0ae520 2024-01-15T18:07:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.931+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec067450 con 0x7fd9ec0ae520 2024-01-15T18:07:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.932+0000 7fd9e8c00640 1 -- 172.21.15.26:0/1238790906 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fd9d4003c80 con 0x7fd9ec0ae520 2024-01-15T18:07:44.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.933+0000 7fd9e8c00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9c406c070 0x7fd9c406e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:44.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.933+0000 7fd9e8c00640 1 -- 172.21.15.26:0/1238790906 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5852+0+0 (secure 0 0 0) 0x7fd9d4087e70 con 0x7fd9ec0ae520 2024-01-15T18:07:44.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.934+0000 7fd9f0a00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9c406c070 0x7fd9c406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.934+0000 7fd9f0a00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9c406c070 0x7fd9c406e530 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000f60 tx=0x7fd9dc007e20 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:44.940 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:44.937+0000 7fd9e8c00640 1 -- 172.21.15.26:0/1238790906 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd9d4054510 con 0x7fd9ec0ae520 2024-01-15T18:07:45.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.171+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41815", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9ec15eda0 con 0x7fd9c406c070 2024-01-15T18:07:45.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.953+0000 7fd9e8c00640 1 -- 172.21.15.26:0/1238790906 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd9ec15eda0 con 0x7fd9c406c070 2024-01-15T18:07:45.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.957+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9c406c070 msgr2=0x7fd9c406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:45.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.957+0000 7fd9f1e00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9c406c070 0x7fd9c406e530 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc000f60 tx=0x7fd9dc007e20 comp rx=0 tx=0).stop 2024-01-15T18:07:45.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.957+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ec0ae520 msgr2=0x7fd9ec0c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:45.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.957+0000 7fd9f1e00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ec0ae520 0x7fd9ec0c6850 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c520 tx=0x7fd9d400c9e0 comp rx=0 tx=0).stop 2024-01-15T18:07:45.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.959+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 shutdown_connections 2024-01-15T18:07:45.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.959+0000 7fd9f1e00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd9c406c070 0x7fd9c406e530 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:45.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.959+0000 7fd9f1e00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9ec1460e0 0x7fd9ec0c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:45.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.959+0000 7fd9f1e00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9ec0ae520 0x7fd9ec0c6850 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.959+0000 7fd9f1e00640 1 --2- 172.21.15.26:0/1238790906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd9ec06de00 0x7fd9ec1f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.959+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 >> 172.21.15.26:0/1238790906 conn(0x7fd9ec0cf000 msgr2=0x7fd9ec0713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.960+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 shutdown_connections 2024-01-15T18:07:45.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:45.960+0000 7fd9f1e00640 1 -- 172.21.15.26:0/1238790906 wait complete. 2024-01-15T18:07:45.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41814 2024-01-15T18:07:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.222+0000 7f1576e00640 1 Processor -- start 2024-01-15T18:07:46.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.223+0000 7f1576e00640 1 -- start start 2024-01-15T18:07:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.224+0000 7f1576e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1570160920 0x7f15700b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.224+0000 7f1576e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cfea0 0x7f1570161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.224+0000 7f1576e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15700d0a10 con 0x7f15700cf9f0 2024-01-15T18:07:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.224+0000 7f1576e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15700d0330 con 0x7f1570160920 2024-01-15T18:07:46.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.224+0000 7f1576e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15700d1f50 con 0x7f15700cfea0 2024-01-15T18:07:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.225+0000 7f1575a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1570160920 0x7f15700b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.225+0000 7f1576400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cfea0 0x7f1570161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.225+0000 7f1575a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1570160920 0x7f15700b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:60530/0 (socket says 172.21.15.26:60530) 2024-01-15T18:07:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.225+0000 7f1575a00640 1 -- 172.21.15.26:0/2210185906 learned_addr learned my addr 172.21.15.26:0/2210185906 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.225+0000 7f1575a00640 1 -- 172.21.15.26:0/2210185906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cfea0 msgr2=0x7f1570161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.225+0000 7f1575a00640 1 --2- 172.21.15.26:0/2210185906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cfea0 0x7f1570161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.225+0000 7f1575a00640 1 -- 172.21.15.26:0/2210185906 >> v1:172.21.15.124:6789/0 conn(0x7f15700cf9f0 legacy=0x7f1570161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.225+0000 7f1575a00640 1 -- 172.21.15.26:0/2210185906 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15700cf080 con 0x7f1570160920 2024-01-15T18:07:46.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.227+0000 7f1575a00640 1 --2- 172.21.15.26:0/2210185906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1570160920 0x7f15700b3f30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1568009e50 tx=0x7f156802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c9925e722d1001 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:46.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.227+0000 7f156fe00640 1 -- 172.21.15.26:0/2210185906 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156802cb00 con 0x7f1570160920 2024-01-15T18:07:46.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.227+0000 7f156fe00640 1 -- 172.21.15.26:0/2210185906 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156802cc60 con 0x7f1570160920 2024-01-15T18:07:46.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.227+0000 7f156fe00640 1 -- 172.21.15.26:0/2210185906 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15680343f0 con 0x7f1570160920 2024-01-15T18:07:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.228+0000 7f1576e00640 1 -- 172.21.15.26:0/2210185906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1570160920 msgr2=0x7f15700b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.228+0000 7f1576e00640 1 --2- 172.21.15.26:0/2210185906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1570160920 0x7f15700b3f30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1568009e50 tx=0x7f156802bf60 comp rx=0 tx=0).stop 2024-01-15T18:07:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.228+0000 7f1576e00640 1 -- 172.21.15.26:0/2210185906 shutdown_connections 2024-01-15T18:07:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.228+0000 7f1576e00640 1 --2- 172.21.15.26:0/2210185906 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cfea0 0x7f1570161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.228+0000 7f1576e00640 1 --2- 172.21.15.26:0/2210185906 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1570160920 0x7f15700b3f30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.228+0000 7f1576e00640 1 -- 172.21.15.26:0/2210185906 >> 172.21.15.26:0/2210185906 conn(0x7f15700d3090 msgr2=0x7f15700d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.228+0000 7f1576e00640 1 -- 172.21.15.26:0/2210185906 shutdown_connections 2024-01-15T18:07:46.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.228+0000 7f1576e00640 1 -- 172.21.15.26:0/2210185906 wait complete. 2024-01-15T18:07:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.230+0000 7f1576e00640 1 Processor -- start 2024-01-15T18:07:46.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.230+0000 7f1576e00640 1 -- start start 2024-01-15T18:07:46.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.231+0000 7f1576e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cf9f0 0x7f15700ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.231+0000 7f1576e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15700cfea0 0x7f15700b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.231+0000 7f1576e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1570160920 0x7f1570151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.231+0000 7f1576e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15700b8b90 con 0x7f1570160920 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.231+0000 7f1576e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15700ca7f0 con 0x7f15700cfea0 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.231+0000 7f1576e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15700caad0 con 0x7f15700cf9f0 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.231+0000 7f1576400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1570160920 0x7f1570151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.231+0000 7f1576400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1570160920 0x7f1570151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41008/0 (socket says 172.21.15.26:41008) 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.231+0000 7f1576400640 1 -- 172.21.15.26:0/2223884485 learned_addr learned my addr 172.21.15.26:0/2223884485 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f1575000640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15700cfea0 0x7f15700b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f1575a00640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cf9f0 0x7f15700ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f1576400640 1 -- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cf9f0 msgr2=0x7f15700ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f1576400640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cf9f0 0x7f15700ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f1576400640 1 -- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15700cfea0 msgr2=0x7f15700b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f1576400640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15700cfea0 0x7f15700b8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f1576400640 1 -- 172.21.15.26:0/2223884485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1568009b30 con 0x7f1570160920 2024-01-15T18:07:46.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f1576400640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1570160920 0x7f1570151480 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f155800e930 tx=0x7f155800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:46.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f156d600640 1 -- 172.21.15.26:0/2223884485 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558009690 con 0x7f1570160920 2024-01-15T18:07:46.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f156d600640 1 -- 172.21.15.26:0/2223884485 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155800c980 con 0x7f1570160920 2024-01-15T18:07:46.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.232+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15700cadb0 con 0x7f1570160920 2024-01-15T18:07:46.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.233+0000 7f156d600640 1 -- 172.21.15.26:0/2223884485 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155800cd30 con 0x7f1570160920 2024-01-15T18:07:46.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.233+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15700cb370 con 0x7f1570160920 2024-01-15T18:07:46.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.234+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15700cb590 con 0x7f1570160920 2024-01-15T18:07:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.235+0000 7f156d600640 1 -- 172.21.15.26:0/2223884485 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f1558013470 con 0x7f1570160920 2024-01-15T18:07:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.236+0000 7f156d600640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f154806c040 0x7f154806e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.236+0000 7f156d600640 1 -- 172.21.15.26:0/2223884485 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5908+0+0 (secure 0 0 0) 0x7f1558088c50 con 0x7f1570160920 2024-01-15T18:07:46.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.236+0000 7f1575a00640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f154806c040 0x7f154806e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:46.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.237+0000 7f1575a00640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f154806c040 0x7f154806e500 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1568035000 tx=0x7f1568030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:46.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.240+0000 7f156d600640 1 -- 172.21.15.26:0/2223884485 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1558055290 con 0x7f1570160920 2024-01-15T18:07:46.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.463+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41814", "target": ["mon-mgr", ""]}) v1 -- 0x7f15700ceac0 con 0x7f154806c040 2024-01-15T18:07:46.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.503+0000 7f156d600640 1 -- 172.21.15.26:0/2223884485 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f15680440d0 con 0x7f154806c040 2024-01-15T18:07:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f154806c040 msgr2=0x7f154806e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f154806c040 0x7f154806e500 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1568035000 tx=0x7f1568030000 comp rx=0 tx=0).stop 2024-01-15T18:07:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1570160920 msgr2=0x7f1570151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1570160920 0x7f1570151480 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f155800e930 tx=0x7f155800edf0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 shutdown_connections 2024-01-15T18:07:46.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f154806c040 0x7f154806e500 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1570160920 0x7f1570151480 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f15700cfea0 0x7f15700b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 --2- 172.21.15.26:0/2223884485 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f15700cf9f0 0x7f15700ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.505+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 >> 172.21.15.26:0/2223884485 conn(0x7f15700d3090 msgr2=0x7f157015afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:46.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.506+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 shutdown_connections 2024-01-15T18:07:46.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.506+0000 7f1576e00640 1 -- 172.21.15.26:0/2223884485 wait complete. 2024-01-15T18:07:46.528 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41816 2024-01-15T18:07:46.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.745+0000 7fbf4b200640 1 Processor -- start 2024-01-15T18:07:46.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.745+0000 7fbf4b200640 1 -- start start 2024-01-15T18:07:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.747+0000 7fbf4b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c146770 0x7fbf4c06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.747+0000 7fbf4b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c0b2e10 con 0x7fbf4c071f00 2024-01-15T18:07:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.747+0000 7fbf4b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c146bc0 con 0x7fbf4c071410 2024-01-15T18:07:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.747+0000 7fbf4b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c06cec0 con 0x7fbf4c146770 2024-01-15T18:07:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.747+0000 7fbf41200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbf4c071f00 0x7fbf4c164ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45624/0 (socket says 172.21.15.26:45624) 2024-01-15T18:07:46.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.747+0000 7fbf41200640 1 -- 172.21.15.26:0/654531030 learned_addr learned my addr 172.21.15.26:0/654531030 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.747+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679253158 0 0) 0x7fbf4c0b2e10 con 0x7fbf4c071f00 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.747+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf2c003520 con 0x7fbf4c071f00 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 939316688 0 0) 0x7fbf2c003520 con 0x7fbf4c071f00 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c146770 msgr2=0x7fbf4c06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 --2- 172.21.15.26:0/654531030 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c146770 0x7fbf4c06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 >> v1:172.21.15.124:6789/0 conn(0x7fbf4c071410 legacy=0x7fbf4c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c06d1a0 con 0x7fbf4c071f00 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fbf34001670 con 0x7fbf4c071f00 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 >> v1:172.21.15.26:6789/0 conn(0x7fbf4c071f00 legacy=0x7fbf4c164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 --2- 172.21.15.26:0/654531030 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c146770 0x7fbf2c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 --2- 172.21.15.26:0/654531030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2c004090 0x7fbf2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 --2- 172.21.15.26:0/654531030 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf2c004d70 0x7fbf2c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c06cec0 con 0x7fbf2c004090 2024-01-15T18:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c00bc80 con 0x7fbf2c004d70 2024-01-15T18:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2c00be40 con 0x7fbf4c146770 2024-01-15T18:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.748+0000 7fbf41200640 1 --2- 172.21.15.26:0/654531030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2c004090 0x7fbf2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.749+0000 7fbf41200640 1 -- 172.21.15.26:0/654531030 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c146770 msgr2=0x7fbf2c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.749+0000 7fbf41200640 1 --2- 172.21.15.26:0/654531030 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c146770 0x7fbf2c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.749+0000 7fbf41200640 1 -- 172.21.15.26:0/654531030 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf2c004d70 msgr2=0x7fbf2c0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.749+0000 7fbf41200640 1 --2- 172.21.15.26:0/654531030 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf2c004d70 0x7fbf2c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.749+0000 7fbf41200640 1 -- 172.21.15.26:0/654531030 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c003520 con 0x7fbf2c004090 2024-01-15T18:07:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.750+0000 7fbf41200640 1 --2- 172.21.15.26:0/654531030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2c004090 0x7fbf2c0047c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400c210 tx=0x7fbf2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2776cf0434478eef server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:46.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.750+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34005ad0 con 0x7fbf2c004090 2024-01-15T18:07:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.750+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf34005c50 con 0x7fbf2c004090 2024-01-15T18:07:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.750+0000 7fbf49400640 1 -- 172.21.15.26:0/654531030 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3402c8d0 con 0x7fbf2c004090 2024-01-15T18:07:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.750+0000 7fbf4b200640 1 -- 172.21.15.26:0/654531030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2c004090 msgr2=0x7fbf2c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.750+0000 7fbf4b200640 1 --2- 172.21.15.26:0/654531030 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf2c004090 0x7fbf2c0047c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400c210 tx=0x7fbf2c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.750+0000 7fbf4a800640 1 -- 172.21.15.26:0/654531030 reap_dead start 2024-01-15T18:07:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.751+0000 7fbf4b200640 1 -- 172.21.15.26:0/654531030 shutdown_connections 2024-01-15T18:07:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.751+0000 7fbf4b200640 1 -- 172.21.15.26:0/654531030 >> 172.21.15.26:0/654531030 conn(0x7fbf4c06a800 msgr2=0x7fbf4c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.751+0000 7fbf4b200640 1 -- 172.21.15.26:0/654531030 shutdown_connections 2024-01-15T18:07:46.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.751+0000 7fbf4b200640 1 -- 172.21.15.26:0/654531030 wait complete. 2024-01-15T18:07:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.753+0000 7fbf4b200640 1 Processor -- start 2024-01-15T18:07:46.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.753+0000 7fbf4b200640 1 -- start start 2024-01-15T18:07:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf4b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf2c004090 0x7fbf4c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf4b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf4c13ffa0 0x7fbf4c0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf4b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c0ae660 0x7fbf4c0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf4b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c14d6f0 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf4b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0cd570 con 0x7fbf2c004090 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf4b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c1506d0 con 0x7fbf4c0ae660 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf41200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf4c13ffa0 0x7fbf4c0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf41200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf4c13ffa0 0x7fbf4c0b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41026/0 (socket says 172.21.15.26:41026) 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf41200640 1 -- 172.21.15.26:0/641209238 learned_addr learned my addr 172.21.15.26:0/641209238 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf41200640 1 -- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c0ae660 msgr2=0x7fbf4c0aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.754+0000 7fbf41200640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c0ae660 0x7fbf4c0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.755+0000 7fbf41200640 1 -- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf2c004090 msgr2=0x7fbf4c150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.755+0000 7fbf41200640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf2c004090 0x7fbf4c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.755+0000 7fbf41200640 1 -- 172.21.15.26:0/641209238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0cb540 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.755+0000 7fbf41200640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf4c13ffa0 0x7fbf4c0b1e60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fbf3402c470 tx=0x7fbf34001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.755+0000 7fbf42a00640 1 -- 172.21.15.26:0/641209238 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf34039070 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.755+0000 7fbf42a00640 1 -- 172.21.15.26:0/641209238 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3402cce0 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.755+0000 7fbf42a00640 1 -- 172.21.15.26:0/641209238 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3403c040 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.755+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c0cb820 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.756+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c0cce40 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.757+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c0cb9e0 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.758+0000 7fbf42a00640 1 -- 172.21.15.26:0/641209238 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fbf34032070 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.759+0000 7fbf42a00640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbf2406c090 0x7fbf2406e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:46.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.759+0000 7fbf42a00640 1 -- 172.21.15.26:0/641209238 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5908+0+0 (secure 0 0 0) 0x7fbf340a9cf0 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.760+0000 7fbf49e00640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbf2406c090 0x7fbf2406e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:46.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.763+0000 7fbf49e00640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbf2406c090 0x7fbf2406e550 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c001c50 tx=0x7fbf3c007d10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:46.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.764+0000 7fbf42a00640 1 -- 172.21.15.26:0/641209238 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf34076340 con 0x7fbf4c13ffa0 2024-01-15T18:07:46.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:46.976+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41816", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf4c0cd090 con 0x7fbf2406c090 2024-01-15T18:07:50.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.962+0000 7fbf42a00640 1 -- 172.21.15.26:0/641209238 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbf4c0cd090 con 0x7fbf2406c090 2024-01-15T18:07:50.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.965+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbf2406c090 msgr2=0x7fbf2406e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:50.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.965+0000 7fbf4b200640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbf2406c090 0x7fbf2406e550 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c001c50 tx=0x7fbf3c007d10 comp rx=0 tx=0).stop 2024-01-15T18:07:50.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf4c13ffa0 msgr2=0x7fbf4c0b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf4c13ffa0 0x7fbf4c0b1e60 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fbf3402c470 tx=0x7fbf34001a00 comp rx=0 tx=0).stop 2024-01-15T18:07:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 shutdown_connections 2024-01-15T18:07:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf4c0ae660 0x7fbf4c0aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbf2406c090 0x7fbf2406e550 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf4c13ffa0 0x7fbf4c0b1e60 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 --2- 172.21.15.26:0/641209238 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf2c004090 0x7fbf4c150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 >> 172.21.15.26:0/641209238 conn(0x7fbf4c06a800 msgr2=0x7fbf4c0717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 shutdown_connections 2024-01-15T18:07:50.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:50.966+0000 7fbf4b200640 1 -- 172.21.15.26:0/641209238 wait complete. 2024-01-15T18:07:50.994 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone4183 2024-01-15T18:07:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.206+0000 7f44d5c00640 1 Processor -- start 2024-01-15T18:07:51.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.207+0000 7f44d5c00640 1 -- start start 2024-01-15T18:07:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.208+0000 7f44d5c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f44d01607a0 0x7f44d00b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.208+0000 7f44d5c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f44d0160b80 0x7f44d01617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.208+0000 7f44d5c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d00b2b90 con 0x7f44d00b3eb0 2024-01-15T18:07:51.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.208+0000 7f44d5c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d00d21d0 con 0x7f44d01607a0 2024-01-15T18:07:51.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.208+0000 7f44d5c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d00b2e30 con 0x7f44d0160b80 2024-01-15T18:07:51.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44c7200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f44d00b3eb0 0x7f44d01610c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45630/0 (socket says 172.21.15.26:45630) 2024-01-15T18:07:51.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44c7200640 1 -- 172.21.15.26:0/4139728575 learned_addr learned my addr 172.21.15.26:0/4139728575 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:51.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44cfe00640 1 --2- 172.21.15.26:0/4139728575 >> v2:172.21.15.26:3300/0 conn(0x7f44d01607a0 0x7f44d00b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44d5200640 1 --2- 172.21.15.26:0/4139728575 >> v2:172.21.15.124:3300/0 conn(0x7f44d0160b80 0x7f44d01617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44cf400640 1 -- 172.21.15.26:0/4139728575 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088669085 0 0) 0x7f44d00b2b90 con 0x7f44d00b3eb0 2024-01-15T18:07:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44cf400640 1 -- 172.21.15.26:0/4139728575 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b0003520 con 0x7f44d00b3eb0 2024-01-15T18:07:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44cfe00640 1 -- 172.21.15.26:0/4139728575 >> v2:172.21.15.124:3300/0 conn(0x7f44d0160b80 msgr2=0x7f44d01617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44cfe00640 1 --2- 172.21.15.26:0/4139728575 >> v2:172.21.15.124:3300/0 conn(0x7f44d0160b80 0x7f44d01617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44cfe00640 1 -- 172.21.15.26:0/4139728575 >> v1:172.21.15.26:6789/0 conn(0x7f44d00b3eb0 legacy=0x7f44d01610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.209+0000 7f44cfe00640 1 -- 172.21.15.26:0/4139728575 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d00b3110 con 0x7f44d01607a0 2024-01-15T18:07:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.211+0000 7f44cfe00640 1 --2- 172.21.15.26:0/4139728575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d01607a0 0x7f44d00b4330 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f44d00b27b0 tx=0x7f44c002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fddd4f21a19c311 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.211+0000 7f44cf400640 1 -- 172.21.15.26:0/4139728575 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c002cc80 con 0x7f44d01607a0 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.211+0000 7f44cf400640 1 -- 172.21.15.26:0/4139728575 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c002cde0 con 0x7f44d01607a0 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.211+0000 7f44cf400640 1 -- 172.21.15.26:0/4139728575 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0034660 con 0x7f44d01607a0 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.212+0000 7f44d5c00640 1 -- 172.21.15.26:0/4139728575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d01607a0 msgr2=0x7f44d00b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.212+0000 7f44d5c00640 1 --2- 172.21.15.26:0/4139728575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d01607a0 0x7f44d00b4330 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f44d00b27b0 tx=0x7f44c002bed0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.212+0000 7f44d5c00640 1 -- 172.21.15.26:0/4139728575 shutdown_connections 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.212+0000 7f44d5c00640 1 --2- 172.21.15.26:0/4139728575 >> v2:172.21.15.124:3300/0 conn(0x7f44d0160b80 0x7f44d01617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.212+0000 7f44d5c00640 1 --2- 172.21.15.26:0/4139728575 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d01607a0 0x7f44d00b4330 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.212+0000 7f44d5c00640 1 -- 172.21.15.26:0/4139728575 >> 172.21.15.26:0/4139728575 conn(0x7f44d009c590 msgr2=0x7f44d00d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.212+0000 7f44d5c00640 1 -- 172.21.15.26:0/4139728575 shutdown_connections 2024-01-15T18:07:51.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.212+0000 7f44d5c00640 1 -- 172.21.15.26:0/4139728575 wait complete. 2024-01-15T18:07:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.214+0000 7f44d5c00640 1 Processor -- start 2024-01-15T18:07:51.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.214+0000 7f44d5c00640 1 -- start start 2024-01-15T18:07:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44d00b3eb0 0x7f44d014d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44d01607a0 0x7f44d01503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d0160b80 0x7f44d014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d015dd90 con 0x7f44d0160b80 2024-01-15T18:07:51.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d01508f0 con 0x7f44d00b3eb0 2024-01-15T18:07:51.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b78c0 con 0x7f44d01607a0 2024-01-15T18:07:51.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d0160b80 0x7f44d014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d0160b80 0x7f44d014d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41052/0 (socket says 172.21.15.26:41052) 2024-01-15T18:07:51.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5200640 1 -- 172.21.15.26:0/1738592709 learned_addr learned my addr 172.21.15.26:0/1738592709 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44c7200640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44d01607a0 0x7f44d01503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44cfe00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44d00b3eb0 0x7f44d014d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5200640 1 -- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44d01607a0 msgr2=0x7f44d01503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5200640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44d01607a0 0x7f44d01503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5200640 1 -- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44d00b3eb0 msgr2=0x7f44d014d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5200640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44d00b3eb0 0x7f44d014d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44d5200640 1 -- 172.21.15.26:0/1738592709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c0009a70 con 0x7f44d0160b80 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.215+0000 7f44cfe00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44d00b3eb0 0x7f44d014d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.216+0000 7f44d5200640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d0160b80 0x7f44d014d910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f44c800e7f0 tx=0x7f44c800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.216+0000 7f44ccc00640 1 -- 172.21.15.26:0/1738592709 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c8009510 con 0x7f44d0160b80 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.216+0000 7f44ccc00640 1 -- 172.21.15.26:0/1738592709 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c800ee40 con 0x7f44d0160b80 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.216+0000 7f44ccc00640 1 -- 172.21.15.26:0/1738592709 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800cb80 con 0x7f44d0160b80 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.216+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d00b7b60 con 0x7f44d0160b80 2024-01-15T18:07:51.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.216+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d00b8120 con 0x7f44d0160b80 2024-01-15T18:07:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.218+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d00b8340 con 0x7f44d0160b80 2024-01-15T18:07:51.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.219+0000 7f44ccc00640 1 -- 172.21.15.26:0/1738592709 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f44c800cce0 con 0x7f44d0160b80 2024-01-15T18:07:51.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.220+0000 7f44ccc00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f44a806c070 0x7f44a806e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.220+0000 7f44ccc00640 1 -- 172.21.15.26:0/1738592709 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5880+0+0 (secure 0 0 0) 0x7f44c8088940 con 0x7f44d0160b80 2024-01-15T18:07:51.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.221+0000 7f44cfe00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f44a806c070 0x7f44a806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.222+0000 7f44cfe00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f44a806c070 0x7f44a806e530 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f44c0036000 tx=0x7f44c0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.225+0000 7f44ccc00640 1 -- 172.21.15.26:0/1738592709 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44c8054fb0 con 0x7f44d0160b80 2024-01-15T18:07:51.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.455+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone4183", "target": ["mon-mgr", ""]}) v1 -- 0x7f44d00b39b0 con 0x7f44a806c070 2024-01-15T18:07:51.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.497+0000 7f44ccc00640 1 -- 172.21.15.26:0/1738592709 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f44c00345c0 con 0x7f44a806c070 2024-01-15T18:07:51.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f44a806c070 msgr2=0x7f44a806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f44a806c070 0x7f44a806e530 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f44c0036000 tx=0x7f44c0030000 comp rx=0 tx=0).stop 2024-01-15T18:07:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d0160b80 msgr2=0x7f44d014d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d0160b80 0x7f44d014d910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f44c800e7f0 tx=0x7f44c800ebf0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 shutdown_connections 2024-01-15T18:07:51.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44d01607a0 0x7f44d01503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f44a806c070 0x7f44a806e530 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f44d0160b80 0x7f44d014d910 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 --2- 172.21.15.26:0/1738592709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f44d00b3eb0 0x7f44d014d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.500+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 >> 172.21.15.26:0/1738592709 conn(0x7f44d009c590 msgr2=0x7f44d006b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:51.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.501+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 shutdown_connections 2024-01-15T18:07:51.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.501+0000 7f44d5c00640 1 -- 172.21.15.26:0/1738592709 wait complete. 2024-01-15T18:07:51.527 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone4181 2024-01-15T18:07:51.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.734+0000 7fe19ca00640 1 Processor -- start 2024-01-15T18:07:51.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.734+0000 7fe19ca00640 1 -- start start 2024-01-15T18:07:51.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.735+0000 7fe19ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 0x7fe198146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.736+0000 7fe19ca00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1980b2e10 con 0x7fe198071f00 2024-01-15T18:07:51.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.736+0000 7fe19ca00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe19806cc20 con 0x7fe19806c7c0 2024-01-15T18:07:51.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.736+0000 7fe19ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19806cec0 con 0x7fe198071410 2024-01-15T18:07:51.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.736+0000 7fe197200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe198071f00 0x7fe198166cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45632/0 (socket says 172.21.15.26:45632) 2024-01-15T18:07:51.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.736+0000 7fe197200640 1 -- 172.21.15.26:0/2670237050 learned_addr learned my addr 172.21.15.26:0/2670237050 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:51.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.736+0000 7fe196800640 1 --2- 172.21.15.26:0/2670237050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 0x7fe198146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.737+0000 7fe195e00640 1 -- 172.21.15.26:0/2670237050 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842118356 0 0) 0x7fe1980b2e10 con 0x7fe198071f00 2024-01-15T18:07:51.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.737+0000 7fe195e00640 1 -- 172.21.15.26:0/2670237050 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe178003520 con 0x7fe198071f00 2024-01-15T18:07:51.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.737+0000 7fe195e00640 1 -- 172.21.15.26:0/2670237050 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 434020977 0 0) 0x7fe19806cc20 con 0x7fe19806c7c0 2024-01-15T18:07:51.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.737+0000 7fe195e00640 1 -- 172.21.15.26:0/2670237050 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1980b2e10 con 0x7fe19806c7c0 2024-01-15T18:07:51.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.737+0000 7fe196800640 1 -- 172.21.15.26:0/2670237050 >> v1:172.21.15.124:6789/0 conn(0x7fe19806c7c0 legacy=0x7fe1980717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.737+0000 7fe196800640 1 -- 172.21.15.26:0/2670237050 >> v1:172.21.15.26:6789/0 conn(0x7fe198071f00 legacy=0x7fe198166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.737+0000 7fe195e00640 1 -- 172.21.15.26:0/2670237050 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208588773 0 0) 0x7fe178003520 con 0x7fe198071f00 2024-01-15T18:07:51.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.737+0000 7fe196800640 1 -- 172.21.15.26:0/2670237050 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19806d1a0 con 0x7fe198071410 2024-01-15T18:07:51.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.739+0000 7fe196800640 1 --2- 172.21.15.26:0/2670237050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 0x7fe198146770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe1880097d0 tx=0x7fe18802b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c98626c1e930d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.740+0000 7fe195e00640 1 -- 172.21.15.26:0/2670237050 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1900019f0 con 0x7fe198071410 2024-01-15T18:07:51.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.740+0000 7fe195e00640 1 -- 172.21.15.26:0/2670237050 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18802e050 con 0x7fe198071410 2024-01-15T18:07:51.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.740+0000 7fe195e00640 1 -- 172.21.15.26:0/2670237050 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188030730 con 0x7fe198071410 2024-01-15T18:07:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.740+0000 7fe19ca00640 1 -- 172.21.15.26:0/2670237050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 msgr2=0x7fe198146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.740+0000 7fe19ca00640 1 --2- 172.21.15.26:0/2670237050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 0x7fe198146770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe1880097d0 tx=0x7fe18802b800 comp rx=0 tx=0).stop 2024-01-15T18:07:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.740+0000 7fe19ca00640 1 -- 172.21.15.26:0/2670237050 shutdown_connections 2024-01-15T18:07:51.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.740+0000 7fe19ca00640 1 --2- 172.21.15.26:0/2670237050 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 0x7fe198146770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.740+0000 7fe19ca00640 1 -- 172.21.15.26:0/2670237050 >> 172.21.15.26:0/2670237050 conn(0x7fe19806a800 msgr2=0x7fe19806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:51.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.741+0000 7fe19ca00640 1 -- 172.21.15.26:0/2670237050 shutdown_connections 2024-01-15T18:07:51.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.741+0000 7fe19ca00640 1 -- 172.21.15.26:0/2670237050 wait complete. 2024-01-15T18:07:51.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.743+0000 7fe19ca00640 1 Processor -- start 2024-01-15T18:07:51.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.743+0000 7fe19ca00640 1 -- start start 2024-01-15T18:07:51.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.744+0000 7fe19ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19806c7c0 0x7fe198132e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.744+0000 7fe19ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 0x7fe1981f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.744+0000 7fe19ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe198071f00 0x7fe1980cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.744+0000 7fe19ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1981309f0 con 0x7fe19806c7c0 2024-01-15T18:07:51.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.744+0000 7fe19ca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1981303e0 con 0x7fe198071f00 2024-01-15T18:07:51.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.744+0000 7fe19ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe198144710 con 0x7fe198071410 2024-01-15T18:07:51.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe196800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19806c7c0 0x7fe198132e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe196800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19806c7c0 0x7fe198132e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41066/0 (socket says 172.21.15.26:41066) 2024-01-15T18:07:51.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe196800640 1 -- 172.21.15.26:0/505837809 learned_addr learned my addr 172.21.15.26:0/505837809 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe18dc00640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 0x7fe1981f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe197200640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe198071f00 0x7fe1980cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe196800640 1 -- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 msgr2=0x7fe1981f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe196800640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 0x7fe1981f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe196800640 1 -- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe198071f00 msgr2=0x7fe1980cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe196800640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe198071f00 0x7fe1980cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:51.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.745+0000 7fe196800640 1 -- 172.21.15.26:0/505837809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188009400 con 0x7fe19806c7c0 2024-01-15T18:07:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.746+0000 7fe196800640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19806c7c0 0x7fe198132e90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe1880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.746+0000 7fe18f400640 1 -- 172.21.15.26:0/505837809 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe188004230 con 0x7fe19806c7c0 2024-01-15T18:07:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.746+0000 7fe18f400640 1 -- 172.21.15.26:0/505837809 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe188004390 con 0x7fe19806c7c0 2024-01-15T18:07:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.746+0000 7fe18f400640 1 -- 172.21.15.26:0/505837809 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18803c860 con 0x7fe19806c7c0 2024-01-15T18:07:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.746+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe198130680 con 0x7fe19806c7c0 2024-01-15T18:07:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.747+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1981f68f0 con 0x7fe19806c7c0 2024-01-15T18:07:51.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.748+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1981f6ae0 con 0x7fe19806c7c0 2024-01-15T18:07:51.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.749+0000 7fe18f400640 1 -- 172.21.15.26:0/505837809 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fe188039050 con 0x7fe19806c7c0 2024-01-15T18:07:51.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.750+0000 7fe18f400640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17006c040 0x7fe17006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:51.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.750+0000 7fe18f400640 1 -- 172.21.15.26:0/505837809 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5880+0+0 (secure 0 0 0) 0x7fe1880aa020 con 0x7fe19806c7c0 2024-01-15T18:07:51.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.750+0000 7fe18dc00640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17006c040 0x7fe17006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:51.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.752+0000 7fe18dc00640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17006c040 0x7fe17006e500 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe1981f06b0 tx=0x7fe180009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:51.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.754+0000 7fe18f400640 1 -- 172.21.15.26:0/505837809 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe188076800 con 0x7fe19806c7c0 2024-01-15T18:07:51.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:51.976+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone4181", "target": ["mon-mgr", ""]}) v1 -- 0x7fe198073c60 con 0x7fe17006c040 2024-01-15T18:07:52.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.026+0000 7fe18f400640 1 -- 172.21.15.26:0/505837809 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe198073c60 con 0x7fe17006c040 2024-01-15T18:07:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17006c040 msgr2=0x7fe17006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17006c040 0x7fe17006e500 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe1981f06b0 tx=0x7fe180009040 comp rx=0 tx=0).stop 2024-01-15T18:07:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19806c7c0 msgr2=0x7fe198132e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19806c7c0 0x7fe198132e90 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fe188000c00 tx=0x7fe1880046c0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 shutdown_connections 2024-01-15T18:07:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe198071410 0x7fe1981f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe17006c040 0x7fe17006e500 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe198071f00 0x7fe1980cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 --2- 172.21.15.26:0/505837809 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe19806c7c0 0x7fe198132e90 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.029+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 >> 172.21.15.26:0/505837809 conn(0x7fe19806a800 msgr2=0x7fe19815ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.030+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 shutdown_connections 2024-01-15T18:07:52.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.030+0000 7fe19ca00640 1 -- 172.21.15.26:0/505837809 wait complete. 2024-01-15T18:07:52.054 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone4186 2024-01-15T18:07:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.262+0000 7f0cdb200640 1 Processor -- start 2024-01-15T18:07:52.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.262+0000 7f0cdb200640 1 -- start start 2024-01-15T18:07:52.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.264+0000 7f0cdb200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 0x7f0cd40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.264+0000 7f0cdb200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0cd40cf9f0 0x7f0cd40cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.264+0000 7f0cdb200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40d0a10 con 0x7f0cd4161010 2024-01-15T18:07:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.264+0000 7f0cdb200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd40d03b0 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.264+0000 7f0cdb200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40d1f50 con 0x7f0cd4160920 2024-01-15T18:07:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.264+0000 7f0cd9400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0cd40cf9f0 0x7f0cd40cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.264+0000 7f0cda800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0cd4161010 0x7f0cd41614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50678/0 (socket says 172.21.15.26:50678) 2024-01-15T18:07:52.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.264+0000 7f0cda800640 1 -- 172.21.15.26:0/300589850 learned_addr learned my addr 172.21.15.26:0/300589850 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.264+0000 7f0cd9e00640 1 --2- 172.21.15.26:0/300589850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 0x7f0cd40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.265+0000 7f0cd9400640 1 -- 172.21.15.26:0/300589850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 msgr2=0x7f0cd40b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.265+0000 7f0cd9400640 1 --2- 172.21.15.26:0/300589850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 0x7f0cd40b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.265+0000 7f0cd9400640 1 -- 172.21.15.26:0/300589850 >> v1:172.21.15.26:6789/0 conn(0x7f0cd4161010 legacy=0x7f0cd41614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.265+0000 7f0cd9400640 1 -- 172.21.15.26:0/300589850 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40cf080 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.266+0000 7f0cd9400640 1 --2- 172.21.15.26:0/300589850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 0x7f0cd40cfe70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0097a0 tx=0x7f0cbc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c316552bd20d39ac server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.266+0000 7f0cd8a00640 1 -- 172.21.15.26:0/300589850 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc02d040 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.267+0000 7f0cd8a00640 1 -- 172.21.15.26:0/300589850 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc031810 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.267+0000 7f0cd8a00640 1 -- 172.21.15.26:0/300589850 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc031ba0 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.267+0000 7f0cdb200640 1 -- 172.21.15.26:0/300589850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 msgr2=0x7f0cd40cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.267+0000 7f0cdb200640 1 --2- 172.21.15.26:0/300589850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 0x7f0cd40cfe70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0097a0 tx=0x7f0cbc02b8b0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.267+0000 7f0cdb200640 1 -- 172.21.15.26:0/300589850 shutdown_connections 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.267+0000 7f0cdb200640 1 --2- 172.21.15.26:0/300589850 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 0x7f0cd40cfe70 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.267+0000 7f0cdb200640 1 --2- 172.21.15.26:0/300589850 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 0x7f0cd40b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.267+0000 7f0cdb200640 1 -- 172.21.15.26:0/300589850 >> 172.21.15.26:0/300589850 conn(0x7f0cd40d3090 msgr2=0x7f0cd40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.268+0000 7f0cdb200640 1 -- 172.21.15.26:0/300589850 shutdown_connections 2024-01-15T18:07:52.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.268+0000 7f0cdb200640 1 -- 172.21.15.26:0/300589850 wait complete. 2024-01-15T18:07:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.269+0000 7f0cdb200640 1 Processor -- start 2024-01-15T18:07:52.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.270+0000 7f0cdb200640 1 -- start start 2024-01-15T18:07:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.270+0000 7f0cdb200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 0x7f0cd415fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.270+0000 7f0cdb200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 0x7f0cd40cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.270+0000 7f0cdb200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cd4161010 0x7f0cd414c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.270+0000 7f0cdb200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40b6500 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.270+0000 7f0cdb200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40cc410 con 0x7f0cd4161010 2024-01-15T18:07:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.270+0000 7f0cdb200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40cc6f0 con 0x7f0cd4160920 2024-01-15T18:07:52.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 0x7f0cd415fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 0x7f0cd415fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52544/0 (socket says 172.21.15.26:52544) 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9e00640 1 -- 172.21.15.26:0/1984650224 learned_addr learned my addr 172.21.15.26:0/1984650224 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cda800640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cd4161010 0x7f0cd414c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9400640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 0x7f0cd40cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9e00640 1 -- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 msgr2=0x7f0cd40cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9e00640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 0x7f0cd40cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9e00640 1 -- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cd4161010 msgr2=0x7f0cd414c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9e00640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cd4161010 0x7f0cd414c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9e00640 1 -- 172.21.15.26:0/1984650224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc009400 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cda800640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cd4161010 0x7f0cd414c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.271+0000 7f0cd9400640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 0x7f0cd40cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:52.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.272+0000 7f0cd9e00640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 0x7f0cd415fd20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400d800 tx=0x7f0cc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.272+0000 7f0cd2000640 1 -- 172.21.15.26:0/1984650224 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400b6c0 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.272+0000 7f0cd2000640 1 -- 172.21.15.26:0/1984650224 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc400b820 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.272+0000 7f0cd2000640 1 -- 172.21.15.26:0/1984650224 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc400bbb0 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.272+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd40cc9d0 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.272+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd40cc100 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.273+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd4067450 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.274+0000 7f0cd2000640 1 -- 172.21.15.26:0/1984650224 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f0cc400bd10 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.275+0000 7f0cd2000640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0cac06c070 0x7f0cac06e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.275+0000 7f0cd2000640 1 -- 172.21.15.26:0/1984650224 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5880+0+0 (secure 0 0 0) 0x7f0cc4088e20 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.276+0000 7f0cd9400640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0cac06c070 0x7f0cac06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.277+0000 7f0cd9400640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0cac06c070 0x7f0cac06e530 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc036000 tx=0x7f0cbc03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.288+0000 7f0cd2000640 1 -- 172.21.15.26:0/1984650224 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0cc4055430 con 0x7f0cd40cf9f0 2024-01-15T18:07:52.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.510+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone4186", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cd40ceac0 con 0x7f0cac06c070 2024-01-15T18:07:52.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.555+0000 7f0cd2000640 1 -- 172.21.15.26:0/1984650224 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0cbc03a420 con 0x7f0cac06c070 2024-01-15T18:07:52.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.557+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0cac06c070 msgr2=0x7f0cac06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.557+0000 7f0cdb200640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0cac06c070 0x7f0cac06e530 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc036000 tx=0x7f0cbc03c040 comp rx=0 tx=0).stop 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.557+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 msgr2=0x7f0cd415fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.557+0000 7f0cdb200640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 0x7f0cd415fd20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f0cc400d800 tx=0x7f0cc400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.558+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 shutdown_connections 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.558+0000 7f0cdb200640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0cd4160920 0x7f0cd40cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.558+0000 7f0cdb200640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0cac06c070 0x7f0cac06e530 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.558+0000 7f0cdb200640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0cd4161010 0x7f0cd414c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.558+0000 7f0cdb200640 1 --2- 172.21.15.26:0/1984650224 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0cd40cf9f0 0x7f0cd415fd20 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.558+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 >> 172.21.15.26:0/1984650224 conn(0x7f0cd40d3090 msgr2=0x7f0cd40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.558+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 shutdown_connections 2024-01-15T18:07:52.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.558+0000 7f0cdb200640 1 -- 172.21.15.26:0/1984650224 wait complete. 2024-01-15T18:07:52.585 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41812 2024-01-15T18:07:52.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.786+0000 7f7909a00640 1 Processor -- start 2024-01-15T18:07:52.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.787+0000 7f7909a00640 1 -- start start 2024-01-15T18:07:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.788+0000 7f7909a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f79040cc990 0x7f790406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.788+0000 7f7909a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7904070e50 con 0x7f79040cc500 2024-01-15T18:07:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.788+0000 7f7909a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7904144020 con 0x7f7904143c40 2024-01-15T18:07:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.788+0000 7f7909a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79040723c0 con 0x7f79040cc990 2024-01-15T18:07:52.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.789+0000 7f78fbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f79040cc500 0x7f790406d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50694/0 (socket says 172.21.15.26:50694) 2024-01-15T18:07:52.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.789+0000 7f78fbe00640 1 -- 172.21.15.26:0/1448823399 learned_addr learned my addr 172.21.15.26:0/1448823399 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.789+0000 7f7909000640 1 --2- 172.21.15.26:0/1448823399 >> v2:172.21.15.124:3300/0 conn(0x7f79040cc990 0x7f790406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.789+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004316208 0 0) 0x7f7904070e50 con 0x7f79040cc500 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.789+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e4003520 con 0x7f79040cc500 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2316068313 0 0) 0x7f78e4003520 con 0x7f79040cc500 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 >> v2:172.21.15.124:3300/0 conn(0x7f79040cc990 msgr2=0x7f790406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 --2- 172.21.15.26:0/1448823399 >> v2:172.21.15.124:3300/0 conn(0x7f79040cc990 0x7f790406d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 >> v1:172.21.15.124:6789/0 conn(0x7f7904143c40 legacy=0x7f7904071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7904071e00 con 0x7f79040cc500 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f78ec001670 con 0x7f79040cc500 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 >> v1:172.21.15.26:6789/0 conn(0x7f79040cc500 legacy=0x7f790406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78e40039c0 0x7f78e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78e40047c0 0x7f78e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78e40051d0 0x7f78e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79040723c0 con 0x7f78e40051d0 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e400c120 con 0x7f78e40047c0 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e400c2e0 con 0x7f78e40039c0 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7909000640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78e40051d0 0x7f78e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f78fbe00640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78e40047c0 0x7f78e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.790+0000 7f7903e00640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78e40039c0 0x7f78e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.791+0000 7f7909000640 1 -- 172.21.15.26:0/1448823399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78e40039c0 msgr2=0x7f78e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.791+0000 7f7909000640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78e40039c0 0x7f78e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.791+0000 7f7909000640 1 -- 172.21.15.26:0/1448823399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78e40047c0 msgr2=0x7f78e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.791+0000 7f7909000640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78e40047c0 0x7f78e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.791+0000 7f7909000640 1 -- 172.21.15.26:0/1448823399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e4003520 con 0x7f78e40051d0 2024-01-15T18:07:52.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.792+0000 7f7909000640 1 -- 172.21.15.26:0/1448823399 reap_dead start 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.792+0000 7f7909000640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78e40051d0 0x7f78e400b980 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009f90 tx=0x7f78e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e99bcc0240c205 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.792+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02cc20 con 0x7f78e40051d0 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.792+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78fc00f040 con 0x7f78e40051d0 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.792+0000 7f7903400640 1 -- 172.21.15.26:0/1448823399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc003990 con 0x7f78e40051d0 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.793+0000 7f7909a00640 1 -- 172.21.15.26:0/1448823399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78e40051d0 msgr2=0x7f78e400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.793+0000 7f7909a00640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78e40051d0 0x7f78e400b980 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009f90 tx=0x7f78e40057f0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.793+0000 7f7909a00640 1 -- 172.21.15.26:0/1448823399 shutdown_connections 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.793+0000 7f7909a00640 1 --2- 172.21.15.26:0/1448823399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78e40051d0 0x7f78e400b980 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.793+0000 7f7909a00640 1 -- 172.21.15.26:0/1448823399 >> 172.21.15.26:0/1448823399 conn(0x7f790406a800 msgr2=0x7f790406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.793+0000 7f7909a00640 1 -- 172.21.15.26:0/1448823399 shutdown_connections 2024-01-15T18:07:52.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.793+0000 7f7909a00640 1 -- 172.21.15.26:0/1448823399 wait complete. 2024-01-15T18:07:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.795+0000 7f7909a00640 1 Processor -- start 2024-01-15T18:07:52.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.795+0000 7f7909a00640 1 -- start start 2024-01-15T18:07:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.796+0000 7f7909a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78e40051d0 0x7f79040adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.796+0000 7f7909a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790415e050 0x7f79041f0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.796+0000 7f7909a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f790413b260 0x7f790413b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.796+0000 7f7909a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7904141060 con 0x7f790415e050 2024-01-15T18:07:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.796+0000 7f7909a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7904141340 con 0x7f790413b260 2024-01-15T18:07:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.796+0000 7f7909a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7904141620 con 0x7f78e40051d0 2024-01-15T18:07:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.796+0000 7f78fbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790415e050 0x7f79041f0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.796+0000 7f78fbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790415e050 0x7f79041f0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52550/0 (socket says 172.21.15.26:52550) 2024-01-15T18:07:52.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.796+0000 7f78fbe00640 1 -- 172.21.15.26:0/2014945723 learned_addr learned my addr 172.21.15.26:0/2014945723 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f7909000640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f790413b260 0x7f790413b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f7903e00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78e40051d0 0x7f79040adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f78fbe00640 1 -- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78e40051d0 msgr2=0x7f79040adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f78fbe00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78e40051d0 0x7f79040adbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f78fbe00640 1 -- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f790413b260 msgr2=0x7f790413b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f78fbe00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f790413b260 0x7f790413b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f78fbe00640 1 -- 172.21.15.26:0/2014945723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7904141900 con 0x7f790415e050 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f78fbe00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790415e050 0x7f79041f0f10 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f78ec007290 tx=0x7f78ec007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f7900c00640 1 -- 172.21.15.26:0/2014945723 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec001650 con 0x7f790415e050 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f7900c00640 1 -- 172.21.15.26:0/2014945723 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec00f9c0 con 0x7f790415e050 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7904141be0 con 0x7f790415e050 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f7900c00640 1 -- 172.21.15.26:0/2014945723 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00fd70 con 0x7f790415e050 2024-01-15T18:07:52.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.797+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79040ae0f0 con 0x7f790415e050 2024-01-15T18:07:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.798+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7904067450 con 0x7f790415e050 2024-01-15T18:07:52.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.800+0000 7f7900c00640 1 -- 172.21.15.26:0/2014945723 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f78ec01d460 con 0x7f790415e050 2024-01-15T18:07:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.801+0000 7f7900c00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78dc06c040 0x7f78dc06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:52.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.801+0000 7f7900c00640 1 -- 172.21.15.26:0/2014945723 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5880+0+0 (secure 0 0 0) 0x7f78ec088ad0 con 0x7f790415e050 2024-01-15T18:07:52.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.801+0000 7f7903e00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78dc06c040 0x7f78dc06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:52.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.802+0000 7f7903e00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78dc06c040 0x7f78dc06e500 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f78f4007fd0 tx=0x7f78f4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:52.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:52.805+0000 7f7900c00640 1 -- 172.21.15.26:0/2014945723 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78ec0550e0 con 0x7f790415e050 2024-01-15T18:07:53.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.025+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41812", "target": ["mon-mgr", ""]}) v1 -- 0x7f79041414b0 con 0x7f78dc06c040 2024-01-15T18:07:53.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.068+0000 7f7900c00640 1 -- 172.21.15.26:0/2014945723 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f79041414b0 con 0x7f78dc06c040 2024-01-15T18:07:53.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.070+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78dc06c040 msgr2=0x7f78dc06e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.070+0000 7f7909a00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78dc06c040 0x7f78dc06e500 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f78f4007fd0 tx=0x7f78f4009040 comp rx=0 tx=0).stop 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.070+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790415e050 msgr2=0x7f79041f0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.070+0000 7f7909a00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790415e050 0x7f79041f0f10 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f78ec007290 tx=0x7f78ec007690 comp rx=0 tx=0).stop 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.071+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 shutdown_connections 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.071+0000 7f7909a00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78dc06c040 0x7f78dc06e500 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.071+0000 7f7909a00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f790413b260 0x7f790413b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.071+0000 7f7909a00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f790415e050 0x7f79041f0f10 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.071+0000 7f7909a00640 1 --2- 172.21.15.26:0/2014945723 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78e40051d0 0x7f79040adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.071+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 >> 172.21.15.26:0/2014945723 conn(0x7f790406a800 msgr2=0x7f790406d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.071+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 shutdown_connections 2024-01-15T18:07:53.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.071+0000 7f7909a00640 1 -- 172.21.15.26:0/2014945723 wait complete. 2024-01-15T18:07:53.097 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone4188 2024-01-15T18:07:53.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.304+0000 7f925be00640 1 Processor -- start 2024-01-15T18:07:53.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.304+0000 7f925be00640 1 -- start start 2024-01-15T18:07:53.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.305+0000 7f925be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f925c0cd820 0x7f925c0ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.305+0000 7f925be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0d0640 0x7f925c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f925c0ce140 0x7f925c0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925c0d0c40 con 0x7f925c0ce140 2024-01-15T18:07:53.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925c0d0f20 con 0x7f925c0cd820 2024-01-15T18:07:53.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0d1200 con 0x7f925c0d0640 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f925c0ce140 0x7f925c0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f925c0ce140 0x7f925c0b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52564/0 (socket says 172.21.15.26:52564) 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925b400640 1 -- 172.21.15.26:0/2091824598 learned_addr learned my addr 172.21.15.26:0/2091824598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925aa00640 1 --2- 172.21.15.26:0/2091824598 >> v2:172.21.15.124:3300/0 conn(0x7f925c0cd820 0x7f925c0ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925b400640 1 -- 172.21.15.26:0/2091824598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0d0640 msgr2=0x7f925c0cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f9253e00640 1 --2- 172.21.15.26:0/2091824598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0d0640 0x7f925c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925b400640 1 --2- 172.21.15.26:0/2091824598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0d0640 0x7f925c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925b400640 1 -- 172.21.15.26:0/2091824598 >> v2:172.21.15.124:3300/0 conn(0x7f925c0cd820 msgr2=0x7f925c0ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925b400640 1 --2- 172.21.15.26:0/2091824598 >> v2:172.21.15.124:3300/0 conn(0x7f925c0cd820 0x7f925c0ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925b400640 1 -- 172.21.15.26:0/2091824598 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c0d14e0 con 0x7f925c0ce140 2024-01-15T18:07:53.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.306+0000 7f925aa00640 1 --2- 172.21.15.26:0/2091824598 >> v2:172.21.15.124:3300/0 conn(0x7f925c0cd820 0x7f925c0ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:53.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.308+0000 7f925b400640 1 --2- 172.21.15.26:0/2091824598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 0x7f925c0b0420 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f925c071710 tx=0x7f925402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3830e4dd45ae9f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:53.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.308+0000 7f925a000640 1 -- 172.21.15.26:0/2091824598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925402ca80 con 0x7f925c0ce140 2024-01-15T18:07:53.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.308+0000 7f925a000640 1 -- 172.21.15.26:0/2091824598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925402cbe0 con 0x7f925c0ce140 2024-01-15T18:07:53.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.308+0000 7f925a000640 1 -- 172.21.15.26:0/2091824598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540343f0 con 0x7f925c0ce140 2024-01-15T18:07:53.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.309+0000 7f925be00640 1 -- 172.21.15.26:0/2091824598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 msgr2=0x7f925c0b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.309+0000 7f925be00640 1 --2- 172.21.15.26:0/2091824598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 0x7f925c0b0420 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f925c071710 tx=0x7f925402bee0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.309+0000 7f925be00640 1 -- 172.21.15.26:0/2091824598 shutdown_connections 2024-01-15T18:07:53.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.309+0000 7f925be00640 1 --2- 172.21.15.26:0/2091824598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 0x7f925c0b0420 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.309+0000 7f925be00640 1 --2- 172.21.15.26:0/2091824598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0d0640 0x7f925c0cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.309+0000 7f925be00640 1 --2- 172.21.15.26:0/2091824598 >> v2:172.21.15.124:3300/0 conn(0x7f925c0cd820 0x7f925c0ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.309+0000 7f925be00640 1 -- 172.21.15.26:0/2091824598 >> 172.21.15.26:0/2091824598 conn(0x7f925c099a60 msgr2=0x7f925c099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:53.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.309+0000 7f925be00640 1 -- 172.21.15.26:0/2091824598 shutdown_connections 2024-01-15T18:07:53.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.309+0000 7f925be00640 1 -- 172.21.15.26:0/2091824598 wait complete. 2024-01-15T18:07:53.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.311+0000 7f925be00640 1 Processor -- start 2024-01-15T18:07:53.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.311+0000 7f925be00640 1 -- start start 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.312+0000 7f925be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0cd820 0x7f925c1f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.312+0000 7f925be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 0x7f925c160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.312+0000 7f925be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f925c0d0640 0x7f925c0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.312+0000 7f925be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0cd0d0 con 0x7f925c0ce140 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.312+0000 7f925be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0cadc0 con 0x7f925c0d0640 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.312+0000 7f925be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0cb250 con 0x7f925c0cd820 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.312+0000 7f9253e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 0x7f925c160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.312+0000 7f9253e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 0x7f925c160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52578/0 (socket says 172.21.15.26:52578) 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.312+0000 7f9253e00640 1 -- 172.21.15.26:0/3942941500 learned_addr learned my addr 172.21.15.26:0/3942941500 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.313+0000 7f925b400640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f925c0d0640 0x7f925c0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.313+0000 7f925aa00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0cd820 0x7f925c1f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.313+0000 7f9253e00640 1 -- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0cd820 msgr2=0x7f925c1f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.313+0000 7f9253e00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0cd820 0x7f925c1f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.313+0000 7f9253e00640 1 -- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f925c0d0640 msgr2=0x7f925c0cbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.313+0000 7f9253e00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f925c0d0640 0x7f925c0cbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.313+0000 7f9253e00640 1 -- 172.21.15.26:0/3942941500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9254009b30 con 0x7f925c0ce140 2024-01-15T18:07:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.313+0000 7f925b400640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f925c0d0640 0x7f925c0cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:53.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.313+0000 7f9253e00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 0x7f925c160000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f924400c7f0 tx=0x7f924400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.314+0000 7f9252a00640 1 -- 172.21.15.26:0/3942941500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244004220 con 0x7f925c0ce140 2024-01-15T18:07:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.314+0000 7f9252a00640 1 -- 172.21.15.26:0/3942941500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92440043a0 con 0x7f925c0ce140 2024-01-15T18:07:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.314+0000 7f9252a00640 1 -- 172.21.15.26:0/3942941500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244007860 con 0x7f925c0ce140 2024-01-15T18:07:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.314+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c0ca1b0 con 0x7f925c0ce140 2024-01-15T18:07:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.314+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c0ca770 con 0x7f925c0ce140 2024-01-15T18:07:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.315+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c0caf50 con 0x7f925c0ce140 2024-01-15T18:07:53.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.316+0000 7f9252a00640 1 -- 172.21.15.26:0/3942941500 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f9244007bd0 con 0x7f925c0ce140 2024-01-15T18:07:53.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.317+0000 7f9252a00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f923406c040 0x7f923406e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.318+0000 7f9252a00640 1 -- 172.21.15.26:0/3942941500 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5880+0+0 (secure 0 0 0) 0x7f9244088870 con 0x7f925c0ce140 2024-01-15T18:07:53.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.318+0000 7f925aa00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f923406c040 0x7f923406e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.320+0000 7f925aa00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f923406c040 0x7f923406e500 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b4c90 tx=0x7f924c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:53.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.322+0000 7f9252a00640 1 -- 172.21.15.26:0/3942941500 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9244054e80 con 0x7f925c0ce140 2024-01-15T18:07:53.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.546+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone4188", "target": ["mon-mgr", ""]}) v1 -- 0x7f925c0ce570 con 0x7f923406c040 2024-01-15T18:07:53.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.583+0000 7f9252a00640 1 -- 172.21.15.26:0/3942941500 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f924c00c040 con 0x7f923406c040 2024-01-15T18:07:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f923406c040 msgr2=0x7f923406e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f923406c040 0x7f923406e500 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f925c0b4c90 tx=0x7f924c008040 comp rx=0 tx=0).stop 2024-01-15T18:07:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 msgr2=0x7f925c160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 0x7f925c160000 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f924400c7f0 tx=0x7f924400ccb0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 shutdown_connections 2024-01-15T18:07:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f923406c040 0x7f923406e500 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f925c0d0640 0x7f925c0cbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f925c0ce140 0x7f925c160000 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 --2- 172.21.15.26:0/3942941500 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f925c0cd820 0x7f925c1f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 >> 172.21.15.26:0/3942941500 conn(0x7f925c099a60 msgr2=0x7f925c0b4df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 shutdown_connections 2024-01-15T18:07:53.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.587+0000 7f925be00640 1 -- 172.21.15.26:0/3942941500 wait complete. 2024-01-15T18:07:53.614 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone41820 2024-01-15T18:07:53.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.853+0000 7f28a4a00640 1 Processor -- start 2024-01-15T18:07:53.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.854+0000 7f28a4a00640 1 -- start start 2024-01-15T18:07:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.856+0000 7f28a4a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f289c15d320 0x7f289c06cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.856+0000 7f28a4a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289c153d30 con 0x7f289c15ce90 2024-01-15T18:07:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.856+0000 7f28a4a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289c0b2b70 con 0x7f289c153950 2024-01-15T18:07:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.856+0000 7f28a4a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289c0b2e50 con 0x7f289c15d320 2024-01-15T18:07:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.857+0000 7f28a2000640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f289c153950 0x7f289c0cb0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:34124/0 (socket says 172.21.15.26:34124) 2024-01-15T18:07:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.857+0000 7f28a2000640 1 -- 172.21.15.26:0/3844403709 learned_addr learned my addr 172.21.15.26:0/3844403709 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.857+0000 7f28a2a00640 1 --2- 172.21.15.26:0/3844403709 >> v2:172.21.15.124:3300/0 conn(0x7f289c15d320 0x7f289c06cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.857+0000 7f28a0c00640 1 -- 172.21.15.26:0/3844403709 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267267469 0 0) 0x7f289c0b2b70 con 0x7f289c153950 2024-01-15T18:07:53.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.857+0000 7f28a0c00640 1 -- 172.21.15.26:0/3844403709 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f287c003520 con 0x7f289c153950 2024-01-15T18:07:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.858+0000 7f28a2a00640 1 -- 172.21.15.26:0/3844403709 >> v1:172.21.15.124:6789/0 conn(0x7f289c153950 legacy=0x7f289c0cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.858+0000 7f28a2a00640 1 -- 172.21.15.26:0/3844403709 >> v1:172.21.15.26:6789/0 conn(0x7f289c15ce90 legacy=0x7f289c06c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.858+0000 7f28a2a00640 1 -- 172.21.15.26:0/3844403709 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c0b3130 con 0x7f289c15d320 2024-01-15T18:07:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.860+0000 7f28a2a00640 1 --2- 172.21.15.26:0/3844403709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289c15d320 0x7f289c06cb80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f28840090e0 tx=0x7f288402cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98bfaecdbdde59c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.861+0000 7f28a0c00640 1 -- 172.21.15.26:0/3844403709 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288402da40 con 0x7f289c15d320 2024-01-15T18:07:53.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.861+0000 7f28a0c00640 1 -- 172.21.15.26:0/3844403709 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288402dba0 con 0x7f289c15d320 2024-01-15T18:07:53.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.861+0000 7f28a0c00640 1 -- 172.21.15.26:0/3844403709 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884035440 con 0x7f289c15d320 2024-01-15T18:07:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.862+0000 7f28a4a00640 1 -- 172.21.15.26:0/3844403709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289c15d320 msgr2=0x7f289c06cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.862+0000 7f28a4a00640 1 --2- 172.21.15.26:0/3844403709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289c15d320 0x7f289c06cb80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f28840090e0 tx=0x7f288402cc90 comp rx=0 tx=0).stop 2024-01-15T18:07:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.863+0000 7f28a4a00640 1 -- 172.21.15.26:0/3844403709 shutdown_connections 2024-01-15T18:07:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.863+0000 7f28a4a00640 1 --2- 172.21.15.26:0/3844403709 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289c15d320 0x7f289c06cb80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.863+0000 7f28a4a00640 1 -- 172.21.15.26:0/3844403709 >> 172.21.15.26:0/3844403709 conn(0x7f289c0b0be0 msgr2=0x7f289c0b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.863+0000 7f28a4a00640 1 -- 172.21.15.26:0/3844403709 shutdown_connections 2024-01-15T18:07:53.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.864+0000 7f28a4a00640 1 -- 172.21.15.26:0/3844403709 wait complete. 2024-01-15T18:07:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.867+0000 7f28a4a00640 1 Processor -- start 2024-01-15T18:07:53.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.867+0000 7f28a4a00640 1 -- start start 2024-01-15T18:07:53.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.869+0000 7f28a4a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f289c153950 0x7f289c1573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.869+0000 7f28a4a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f289c15ce90 0x7f289c154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.869+0000 7f28a4a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289c15d320 0x7f289c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.869+0000 7f28a4a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c155640 con 0x7f289c15ce90 2024-01-15T18:07:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.869+0000 7f28a4a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c154a30 con 0x7f289c15d320 2024-01-15T18:07:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.869+0000 7f28a4a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289c154d10 con 0x7f289c153950 2024-01-15T18:07:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.869+0000 7f28a2000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f289c153950 0x7f289c1573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.869+0000 7f28a2000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f289c153950 0x7f289c1573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:41810/0 (socket says 172.21.15.26:41810) 2024-01-15T18:07:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.870+0000 7f28a2000640 1 -- 172.21.15.26:0/2814788741 learned_addr learned my addr 172.21.15.26:0/2814788741 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:53.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.870+0000 7f28a2a00640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289c15d320 0x7f289c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.871+0000 7f28a2000640 1 -- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289c15d320 msgr2=0x7f289c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.871+0000 7f28a2000640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289c15d320 0x7f289c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.871+0000 7f28a2000640 1 -- 172.21.15.26:0/2814788741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f289c15ce90 msgr2=0x7f289c154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.872+0000 7f28a1600640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f289c15ce90 0x7f289c154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.872+0000 7f28a2000640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f289c15ce90 0x7f289c154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.872+0000 7f28a2000640 1 -- 172.21.15.26:0/2814788741 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884009c90 con 0x7f289c153950 2024-01-15T18:07:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.872+0000 7f28a1600640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f289c15ce90 0x7f289c154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.873+0000 7f28a2000640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f289c153950 0x7f289c1573c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f289400cbc0 tx=0x7f289400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:53.877 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.873+0000 7f289a000640 1 -- 172.21.15.26:0/2814788741 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289400ad40 con 0x7f289c153950 2024-01-15T18:07:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.873+0000 7f289a000640 1 -- 172.21.15.26:0/2814788741 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2894012400 con 0x7f289c153950 2024-01-15T18:07:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.873+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c154ff0 con 0x7f289c153950 2024-01-15T18:07:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.873+0000 7f289a000640 1 -- 172.21.15.26:0/2814788741 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289400aea0 con 0x7f289c153950 2024-01-15T18:07:53.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.874+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c154740 con 0x7f289c153950 2024-01-15T18:07:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.877+0000 7f289a000640 1 -- 172.21.15.26:0/2814788741 <== mon.2 v2:172.21.15.124:3301/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f2894007070 con 0x7f289c153950 2024-01-15T18:07:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.877+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c0675d0 con 0x7f289c153950 2024-01-15T18:07:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.879+0000 7f289a000640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f287406c110 0x7f287406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:53.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.879+0000 7f289a000640 1 -- 172.21.15.26:0/2814788741 <== mon.2 v2:172.21.15.124:3301/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5880+0+0 (secure 0 0 0) 0x7f28940889f0 con 0x7f289c153950 2024-01-15T18:07:53.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.880+0000 7f28a1600640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f287406c110 0x7f287406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:53.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.882+0000 7f28a1600640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f287406c110 0x7f287406e5d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f289c15cd50 tx=0x7f288c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:53.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:53.888+0000 7f289a000640 1 -- 172.21.15.26:0/2814788741 <== mon.2 v2:172.21.15.124:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2894054f70 con 0x7f289c153950 2024-01-15T18:07:54.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.280+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone41820", "target": ["mon-mgr", ""]}) v1 -- 0x7f289c15d740 con 0x7f287406c110 2024-01-15T18:07:54.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.306+0000 7f289a000640 1 -- 172.21.15.26:0/2814788741 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f288c0013e0 con 0x7f287406c110 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f287406c110 msgr2=0x7f287406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f287406c110 0x7f287406e5d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f289c15cd50 tx=0x7f288c009040 comp rx=0 tx=0).stop 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f289c153950 msgr2=0x7f289c1573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f289c153950 0x7f289c1573c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f289400cbc0 tx=0x7f289400a430 comp rx=0 tx=0).stop 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 shutdown_connections 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f287406c110 0x7f287406e5d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f289c15d320 0x7f289c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f289c15ce90 0x7f289c154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 --2- 172.21.15.26:0/2814788741 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f289c153950 0x7f289c1573c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 >> 172.21.15.26:0/2814788741 conn(0x7f289c0b0be0 msgr2=0x7f289c0b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:54.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 shutdown_connections 2024-01-15T18:07:54.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.309+0000 7f28a4a00640 1 -- 172.21.15.26:0/2814788741 wait complete. 2024-01-15T18:07:54.331 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone4184 2024-01-15T18:07:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.525+0000 7fba9f200640 1 Processor -- start 2024-01-15T18:07:54.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.525+0000 7fba9f200640 1 -- start start 2024-01-15T18:07:54.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.527+0000 7fba9f200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba98071410 0x7fba98146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:54.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.527+0000 7fba9f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 0x7fba980717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:54.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.527+0000 7fba9f200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9806cc20 con 0x7fba98071d30 2024-01-15T18:07:54.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.527+0000 7fba9f200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9806cec0 con 0x7fba98071410 2024-01-15T18:07:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.527+0000 7fba9f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9806d1a0 con 0x7fba9806c7c0 2024-01-15T18:07:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba98071410 0x7fba98146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9e800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fba98071d30 0x7fba980b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50706/0 (socket says 172.21.15.26:50706) 2024-01-15T18:07:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9e800640 1 -- 172.21.15.26:0/4244574162 learned_addr learned my addr 172.21.15.26:0/4244574162 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:54.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9d400640 1 --2- 172.21.15.26:0/4244574162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 0x7fba980717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9ca00640 1 -- 172.21.15.26:0/4244574162 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329451398 0 0) 0x7fba9806cc20 con 0x7fba98071d30 2024-01-15T18:07:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9ca00640 1 -- 172.21.15.26:0/4244574162 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba78003520 con 0x7fba98071d30 2024-01-15T18:07:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9de00640 1 -- 172.21.15.26:0/4244574162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 msgr2=0x7fba980717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9de00640 1 --2- 172.21.15.26:0/4244574162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 0x7fba980717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9de00640 1 -- 172.21.15.26:0/4244574162 >> v1:172.21.15.26:6789/0 conn(0x7fba98071d30 legacy=0x7fba980b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9de00640 1 -- 172.21.15.26:0/4244574162 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9806d480 con 0x7fba98071410 2024-01-15T18:07:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.528+0000 7fba9d400640 1 --2- 172.21.15.26:0/4244574162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 0x7fba980717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:07:54.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.530+0000 7fba9de00640 1 --2- 172.21.15.26:0/4244574162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071410 0x7fba98146770 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fba880097a0 tx=0x7fba8802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ef1ef2a7f8672b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.530+0000 7fba9ca00640 1 -- 172.21.15.26:0/4244574162 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8802d040 con 0x7fba98071410 2024-01-15T18:07:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.530+0000 7fba9ca00640 1 -- 172.21.15.26:0/4244574162 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba88031810 con 0x7fba98071410 2024-01-15T18:07:54.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.530+0000 7fba9ca00640 1 -- 172.21.15.26:0/4244574162 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88031ba0 con 0x7fba98071410 2024-01-15T18:07:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.531+0000 7fba9f200640 1 -- 172.21.15.26:0/4244574162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071410 msgr2=0x7fba98146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.531+0000 7fba9f200640 1 --2- 172.21.15.26:0/4244574162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071410 0x7fba98146770 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fba880097a0 tx=0x7fba8802b8b0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.531+0000 7fba9f200640 1 -- 172.21.15.26:0/4244574162 shutdown_connections 2024-01-15T18:07:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.531+0000 7fba9f200640 1 --2- 172.21.15.26:0/4244574162 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 0x7fba980717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.531+0000 7fba9f200640 1 --2- 172.21.15.26:0/4244574162 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071410 0x7fba98146770 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.531+0000 7fba9f200640 1 -- 172.21.15.26:0/4244574162 >> 172.21.15.26:0/4244574162 conn(0x7fba9806a800 msgr2=0x7fba9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.531+0000 7fba9f200640 1 -- 172.21.15.26:0/4244574162 shutdown_connections 2024-01-15T18:07:54.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.531+0000 7fba9f200640 1 -- 172.21.15.26:0/4244574162 wait complete. 2024-01-15T18:07:54.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.533+0000 7fba9f200640 1 Processor -- start 2024-01-15T18:07:54.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.533+0000 7fba9f200640 1 -- start start 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.534+0000 7fba9f200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 0x7fba981f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.534+0000 7fba9f200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba98071410 0x7fba980c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.534+0000 7fba9f200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071d30 0x7fba980c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.534+0000 7fba9f200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba981f3820 con 0x7fba98071d30 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.534+0000 7fba9f200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba981f3260 con 0x7fba98071410 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.534+0000 7fba9f200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba981f3540 con 0x7fba9806c7c0 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.534+0000 7fba9e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071d30 0x7fba980c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071d30 0x7fba980c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52616/0 (socket says 172.21.15.26:52616) 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9e800640 1 -- 172.21.15.26:0/507454412 learned_addr learned my addr 172.21.15.26:0/507454412 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:54.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9de00640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 0x7fba981f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9d400640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba98071410 0x7fba980c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9e800640 1 -- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 msgr2=0x7fba981f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9e800640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 0x7fba981f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9e800640 1 -- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba98071410 msgr2=0x7fba980c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9e800640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba98071410 0x7fba980c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9e800640 1 -- 172.21.15.26:0/507454412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba88009400 con 0x7fba98071d30 2024-01-15T18:07:54.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.535+0000 7fba9e800640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071d30 0x7fba980c7630 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fba9000c390 tx=0x7fba9000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:54.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.536+0000 7fba96000640 1 -- 172.21.15.26:0/507454412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90005ce0 con 0x7fba98071d30 2024-01-15T18:07:54.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.536+0000 7fba96000640 1 -- 172.21.15.26:0/507454412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba90005e40 con 0x7fba98071d30 2024-01-15T18:07:54.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.536+0000 7fba96000640 1 -- 172.21.15.26:0/507454412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90007310 con 0x7fba98071d30 2024-01-15T18:07:54.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.536+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba98144180 con 0x7fba98071d30 2024-01-15T18:07:54.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.536+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba981314a0 con 0x7fba98071d30 2024-01-15T18:07:54.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.537+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba981446a0 con 0x7fba98071d30 2024-01-15T18:07:54.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.539+0000 7fba96000640 1 -- 172.21.15.26:0/507454412 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fba90007470 con 0x7fba98071d30 2024-01-15T18:07:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.539+0000 7fba96000640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba7006c040 0x7fba7006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:54.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.540+0000 7fba96000640 1 -- 172.21.15.26:0/507454412 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5880+0+0 (secure 0 0 0) 0x7fba90087f10 con 0x7fba98071d30 2024-01-15T18:07:54.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.540+0000 7fba9de00640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba7006c040 0x7fba7006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:54.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.541+0000 7fba9de00640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba7006c040 0x7fba7006e500 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fba88036000 tx=0x7fba8803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:54.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.544+0000 7fba96000640 1 -- 172.21.15.26:0/507454412 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba981446a0 con 0x7fba98071d30 2024-01-15T18:07:54.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.771+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone4184", "target": ["mon-mgr", ""]}) v1 -- 0x7fba98072150 con 0x7fba7006c040 2024-01-15T18:07:54.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.821+0000 7fba96000640 1 -- 172.21.15.26:0/507454412 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fba88031bc0 con 0x7fba7006c040 2024-01-15T18:07:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba7006c040 msgr2=0x7fba7006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba7006c040 0x7fba7006e500 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fba88036000 tx=0x7fba8803c040 comp rx=0 tx=0).stop 2024-01-15T18:07:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071d30 msgr2=0x7fba980c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071d30 0x7fba980c7630 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fba9000c390 tx=0x7fba9000c850 comp rx=0 tx=0).stop 2024-01-15T18:07:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 shutdown_connections 2024-01-15T18:07:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fba7006c040 0x7fba7006e500 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba98071d30 0x7fba980c7630 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba98071410 0x7fba980c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 --2- 172.21.15.26:0/507454412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba9806c7c0 0x7fba981f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.825+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 >> 172.21.15.26:0/507454412 conn(0x7fba9806a800 msgr2=0x7fba980aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.826+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 shutdown_connections 2024-01-15T18:07:54.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:54.827+0000 7fba9f200640 1 -- 172.21.15.26:0/507454412 wait complete. 2024-01-15T18:07:54.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:07:54.861 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:07:54.942 INFO:teuthology.orchestra.run.smithi026.stdout:2 2024-01-15T18:07:54.944 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:07:55.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.222+0000 7fcfc7c00640 1 Processor -- start 2024-01-15T18:07:55.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.222+0000 7fcfc7c00640 1 -- start start 2024-01-15T18:07:55.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.224+0000 7fcfc7c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcfc00ae5a0 0x7fcfc015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.224+0000 7fcfc7c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 0x7fcfc006cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.224+0000 7fcfc7c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc0070c40 con 0x7fcfc006d120 2024-01-15T18:07:55.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.224+0000 7fcfc7c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc0071450 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.224+0000 7fcfc7c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc006c470 con 0x7fcfc00b3ae0 2024-01-15T18:07:55.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.225+0000 7fcfc5200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcfc00ae5a0 0x7fcfc015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.225+0000 7fcfc5200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcfc00ae5a0 0x7fcfc015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:57594/0 (socket says 172.21.15.26:57594) 2024-01-15T18:07:55.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.225+0000 7fcfc5200640 1 -- 172.21.15.26:0/2880335974 learned_addr learned my addr 172.21.15.26:0/2880335974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:55.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.225+0000 7fcfbfe00640 1 --2- 172.21.15.26:0/2880335974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 0x7fcfc006cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.226+0000 7fcfbfe00640 1 -- 172.21.15.26:0/2880335974 >> v2:172.21.15.124:3300/0 conn(0x7fcfc00ae5a0 msgr2=0x7fcfc015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:55.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.226+0000 7fcfbfe00640 1 --2- 172.21.15.26:0/2880335974 >> v2:172.21.15.124:3300/0 conn(0x7fcfc00ae5a0 0x7fcfc015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.226+0000 7fcfbfe00640 1 -- 172.21.15.26:0/2880335974 >> v1:172.21.15.26:6789/0 conn(0x7fcfc006d120 legacy=0x7fcfc00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.226+0000 7fcfbfe00640 1 -- 172.21.15.26:0/2880335974 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc006c750 con 0x7fcfc00b3ae0 2024-01-15T18:07:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.227+0000 7fcfbfe00640 1 --2- 172.21.15.26:0/2880335974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 0x7fcfc006cbe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80097a0 tx=0x7fcfa802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=157cca0a91761e3a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.227+0000 7fcfbf400640 1 -- 172.21.15.26:0/2880335974 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa802d040 con 0x7fcfc00b3ae0 2024-01-15T18:07:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfbf400640 1 -- 172.21.15.26:0/2880335974 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8031810 con 0x7fcfc00b3ae0 2024-01-15T18:07:55.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfbf400640 1 -- 172.21.15.26:0/2880335974 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8031ba0 con 0x7fcfc00b3ae0 2024-01-15T18:07:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfc7c00640 1 -- 172.21.15.26:0/2880335974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 msgr2=0x7fcfc006cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfc7c00640 1 --2- 172.21.15.26:0/2880335974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 0x7fcfc006cbe0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80097a0 tx=0x7fcfa802b8b0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfc7c00640 1 -- 172.21.15.26:0/2880335974 shutdown_connections 2024-01-15T18:07:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfc7c00640 1 --2- 172.21.15.26:0/2880335974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 0x7fcfc006cbe0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfc7c00640 1 --2- 172.21.15.26:0/2880335974 >> v2:172.21.15.124:3300/0 conn(0x7fcfc00ae5a0 0x7fcfc015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfc7c00640 1 -- 172.21.15.26:0/2880335974 >> 172.21.15.26:0/2880335974 conn(0x7fcfc006a800 msgr2=0x7fcfc006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:55.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfc7c00640 1 -- 172.21.15.26:0/2880335974 shutdown_connections 2024-01-15T18:07:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.228+0000 7fcfc7c00640 1 -- 172.21.15.26:0/2880335974 wait complete. 2024-01-15T18:07:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.230+0000 7fcfc7c00640 1 Processor -- start 2024-01-15T18:07:55.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.230+0000 7fcfc7c00640 1 -- start start 2024-01-15T18:07:55.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc7c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc006d120 0x7fcfc01559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc7c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc00ae5a0 0x7fcfc0156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc7c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 0x7fcfc01567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc7c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00cbef0 con 0x7fcfc006d120 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc7c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00cbc10 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc7c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00cde00 con 0x7fcfc00b3ae0 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc5200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc006d120 0x7fcfc01559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc5200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc006d120 0x7fcfc01559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52618/0 (socket says 172.21.15.26:52618) 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc5200640 1 -- 172.21.15.26:0/232275330 learned_addr learned my addr 172.21.15.26:0/232275330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfbfe00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc00ae5a0 0x7fcfc0156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.231+0000 7fcfc5c00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 0x7fcfc01567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.232+0000 7fcfbfe00640 1 -- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 msgr2=0x7fcfc01567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.232+0000 7fcfbfe00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 0x7fcfc01567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.232+0000 7fcfbfe00640 1 -- 172.21.15.26:0/232275330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc006d120 msgr2=0x7fcfc01559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.232+0000 7fcfbfe00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc006d120 0x7fcfc01559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.232+0000 7fcfbfe00640 1 -- 172.21.15.26:0/232275330 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8009400 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.232+0000 7fcfc5200640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc006d120 0x7fcfc01559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:07:55.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.232+0000 7fcfbfe00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc00ae5a0 0x7fcfc0156290 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8033000 tx=0x7fcfa80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.233+0000 7fcfbcc00640 1 -- 172.21.15.26:0/232275330 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8006e10 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.233+0000 7fcfbcc00640 1 -- 172.21.15.26:0/232275330 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa803ad60 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.233+0000 7fcfbcc00640 1 -- 172.21.15.26:0/232275330 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8044630 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.233+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc00ce0e0 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.233+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc00cc540 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.234+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc00cdc60 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.235+0000 7fcfbcc00640 1 -- 172.21.15.26:0/232275330 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 29) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fcfa8044790 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.236+0000 7fcfbcc00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9806c040 0x7fcf9806e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:55.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.236+0000 7fcfbcc00640 1 -- 172.21.15.26:0/232275330 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5880+0+0 (secure 0 0 0) 0x7fcfa80aaba0 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.236+0000 7fcfc5200640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9806c040 0x7fcf9806e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:55.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.237+0000 7fcfc5200640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9806c040 0x7fcf9806e500 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00b1770 tx=0x7fcfb0009240 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:55.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.239+0000 7fcfbcc00640 1 -- 172.21.15.26:0/232275330 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfa8077260 con 0x7fcfc00ae5a0 2024-01-15T18:07:55.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:55.449+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfc0070550 con 0x7fcf9806c040 2024-01-15T18:07:55.701 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-01-15T18:07:55.696+0000 7f9819c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:07:55.789 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T18:07:57.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:57.215+0000 7fcfbcc00640 1 -- 172.21.15.26:0/232275330 <== mon.1 v2:172.21.15.124:3300/0 7 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fcfa80aa740 con 0x7fcfc00ae5a0 2024-01-15T18:07:58.734 INFO:teuthology.orchestra.run.smithi026.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:07:58.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.728+0000 7fcfbcc00640 1 -- 172.21.15.26:0/232275330 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fcfb0014440 con 0x7fcf9806c040 2024-01-15T18:07:58.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.732+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9806c040 msgr2=0x7fcf9806e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:58.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.732+0000 7fcfc7c00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9806c040 0x7fcf9806e500 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00b1770 tx=0x7fcfb0009240 comp rx=0 tx=0).stop 2024-01-15T18:07:58.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.732+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc00ae5a0 msgr2=0x7fcfc0156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:58.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.732+0000 7fcfc7c00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc00ae5a0 0x7fcfc0156290 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8033000 tx=0x7fcfa80046c0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.733+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 shutdown_connections 2024-01-15T18:07:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.733+0000 7fcfc7c00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcfc00b3ae0 0x7fcfc01567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.733+0000 7fcfc7c00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcf9806c040 0x7fcf9806e500 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.733+0000 7fcfc7c00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcfc00ae5a0 0x7fcfc0156290 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.733+0000 7fcfc7c00640 1 --2- 172.21.15.26:0/232275330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcfc006d120 0x7fcfc01559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.733+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 >> 172.21.15.26:0/232275330 conn(0x7fcfc006a800 msgr2=0x7fcfc00cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.733+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 shutdown_connections 2024-01-15T18:07:58.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:58.734+0000 7fcfc7c00640 1 -- 172.21.15.26:0/232275330 wait complete. 2024-01-15T18:07:58.769 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:07:58.769 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:07:58.906 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:07:59.039 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:07:59.039 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:07:59.039 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:07:59.078 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:07:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.307+0000 7f50e2200640 1 Processor -- start 2024-01-15T18:07:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.307+0000 7f50e2200640 1 -- start start 2024-01-15T18:07:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.309+0000 7f50e2200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50dc13cfa0 0x7f50dc15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.309+0000 7f50e2200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50dc0cd530 0x7f50dc13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.309+0000 7f50e2200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50dc0cdec0 con 0x7f50dc0cd9b0 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.309+0000 7f50e2200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50dc0ce1a0 con 0x7f50dc13cfa0 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.309+0000 7f50e2200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0ce480 con 0x7f50dc0cd530 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.309+0000 7f50e0e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50dc13cfa0 0x7f50dc15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.309+0000 7f50e1800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f50dc0cd9b0 0x7f50dc15d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50724/0 (socket says 172.21.15.26:50724) 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.309+0000 7f50e0e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50dc13cfa0 0x7f50dc15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52636/0 (socket says 172.21.15.26:52636) 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.309+0000 7f50e1800640 1 -- 172.21.15.26:0/784584731 learned_addr learned my addr 172.21.15.26:0/784584731 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.310+0000 7f50dbe00640 1 --2- 172.21.15.26:0/784584731 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50dc0cd530 0x7f50dc13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.310+0000 7f50e0e00640 1 -- 172.21.15.26:0/784584731 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50dc0cd530 msgr2=0x7f50dc13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.310+0000 7f50e0e00640 1 --2- 172.21.15.26:0/784584731 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50dc0cd530 0x7f50dc13f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.310+0000 7f50e0e00640 1 -- 172.21.15.26:0/784584731 >> v1:172.21.15.26:6789/0 conn(0x7f50dc0cd9b0 legacy=0x7f50dc15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.310+0000 7f50e0e00640 1 -- 172.21.15.26:0/784584731 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc0ce760 con 0x7f50dc13cfa0 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.311+0000 7f50e0e00640 1 --2- 172.21.15.26:0/784584731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc13cfa0 0x7f50dc15afb0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0097a0 tx=0x7f50cc02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c791ff50e5dd9d9d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.311+0000 7f50db400640 1 -- 172.21.15.26:0/784584731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc031780 con 0x7f50dc13cfa0 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.311+0000 7f50db400640 1 -- 172.21.15.26:0/784584731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc0318e0 con 0x7f50dc13cfa0 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.311+0000 7f50db400640 1 -- 172.21.15.26:0/784584731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc031c70 con 0x7f50dc13cfa0 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.312+0000 7f50e2200640 1 -- 172.21.15.26:0/784584731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc13cfa0 msgr2=0x7f50dc15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.312+0000 7f50e2200640 1 --2- 172.21.15.26:0/784584731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc13cfa0 0x7f50dc15afb0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f50cc0097a0 tx=0x7f50cc02b820 comp rx=0 tx=0).stop 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.312+0000 7f50e2200640 1 -- 172.21.15.26:0/784584731 shutdown_connections 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.312+0000 7f50e2200640 1 --2- 172.21.15.26:0/784584731 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50dc0cd530 0x7f50dc13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.312+0000 7f50e2200640 1 --2- 172.21.15.26:0/784584731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc13cfa0 0x7f50dc15afb0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.312+0000 7f50e2200640 1 -- 172.21.15.26:0/784584731 >> 172.21.15.26:0/784584731 conn(0x7f50dc06a800 msgr2=0x7f50dc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.312+0000 7f50e2200640 1 -- 172.21.15.26:0/784584731 shutdown_connections 2024-01-15T18:07:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.312+0000 7f50e2200640 1 -- 172.21.15.26:0/784584731 wait complete. 2024-01-15T18:07:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.314+0000 7f50e2200640 1 Processor -- start 2024-01-15T18:07:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.314+0000 7f50e2200640 1 -- start start 2024-01-15T18:07:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.315+0000 7f50e2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc0cd530 0x7f50dc13c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.315+0000 7f50e2200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50dc0cd9b0 0x7f50dc0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.315+0000 7f50e2200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50dc13cfa0 0x7f50dc0c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.315+0000 7f50e2200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc13c700 con 0x7f50dc0cd530 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.315+0000 7f50e2200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0c9d10 con 0x7f50dc13cfa0 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.315+0000 7f50e2200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc0c9ff0 con 0x7f50dc0cd9b0 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.315+0000 7f50e0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc0cd530 0x7f50dc13c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.315+0000 7f50e0e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc0cd530 0x7f50dc13c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52644/0 (socket says 172.21.15.26:52644) 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.315+0000 7f50e0e00640 1 -- 172.21.15.26:0/1457136086 learned_addr learned my addr 172.21.15.26:0/1457136086 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50e0e00640 1 -- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50dc0cd9b0 msgr2=0x7f50dc0c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50e0e00640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50dc0cd9b0 0x7f50dc0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50e0e00640 1 -- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50dc13cfa0 msgr2=0x7f50dc0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50e0e00640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50dc13cfa0 0x7f50dc0c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50e0e00640 1 -- 172.21.15.26:0/1457136086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc009400 con 0x7f50dc0cd530 2024-01-15T18:07:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50e0e00640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc0cd530 0x7f50dc13c1c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f50cc034040 tx=0x7f50cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50d8c00640 1 -- 172.21.15.26:0/1457136086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc0328a0 con 0x7f50dc0cd530 2024-01-15T18:07:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50d8c00640 1 -- 172.21.15.26:0/1457136086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc032a20 con 0x7f50dc0cd530 2024-01-15T18:07:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50d8c00640 1 -- 172.21.15.26:0/1457136086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc032dd0 con 0x7f50dc0cd530 2024-01-15T18:07:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.316+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc0ca2d0 con 0x7f50dc0cd530 2024-01-15T18:07:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.317+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc0ca920 con 0x7f50dc0cd530 2024-01-15T18:07:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.318+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50dc067450 con 0x7f50dc0cd530 2024-01-15T18:07:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.319+0000 7f50d8c00640 1 -- 172.21.15.26:0/1457136086 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f50cc045400 con 0x7f50dc0cd530 2024-01-15T18:07:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.320+0000 7f50d8c00640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50b406bfe0 0x7f50b406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.320+0000 7f50d8c00640 1 -- 172.21.15.26:0/1457136086 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f50cc0aab50 con 0x7f50dc0cd530 2024-01-15T18:07:59.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.320+0000 7f50dbe00640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50b406bfe0 0x7f50b406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.321+0000 7f50dbe00640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50b406bfe0 0x7f50b406e4a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0cca30 tx=0x7f50c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:59.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.324+0000 7f50d8c00640 1 -- 172.21.15.26:0/1457136086 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50cc077530 con 0x7f50dc0cd530 2024-01-15T18:07:59.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.517+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50dc0cbff0 con 0x7f50dc0cd530 2024-01-15T18:07:59.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.518+0000 7f50d8c00640 1 -- 172.21.15.26:0/1457136086 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13050 (secure 0 0 0) 0x7f50cc07ab50 con 0x7f50dc0cd530 2024-01-15T18:07:59.523 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:07:59.523 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:07:58.712271+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1813593645":"2024-01-16T18:07:55.696353+0000","172.21.15.26:6833/1813593645":"2024-01-16T18:07:55.696353+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:07:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.520+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50b406bfe0 msgr2=0x7f50b406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.520+0000 7f50e2200640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50b406bfe0 0x7f50b406e4a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f50dc0cca30 tx=0x7f50c4008040 comp rx=0 tx=0).stop 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc0cd530 msgr2=0x7f50dc13c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc0cd530 0x7f50dc13c1c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f50cc034040 tx=0x7f50cc0046c0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 shutdown_connections 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50dc0cd9b0 0x7f50dc0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f50b406bfe0 0x7f50b406e4a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f50dc13cfa0 0x7f50dc0c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 --2- 172.21.15.26:0/1457136086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50dc0cd530 0x7f50dc13c1c0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 >> 172.21.15.26:0/1457136086 conn(0x7f50dc06a800 msgr2=0x7f50dc153120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 shutdown_connections 2024-01-15T18:07:59.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.521+0000 7f50e2200640 1 -- 172.21.15.26:0/1457136086 wait complete. 2024-01-15T18:07:59.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:07:59.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:07:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.758+0000 7fea48a00640 1 Processor -- start 2024-01-15T18:07:59.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.759+0000 7fea48a00640 1 -- start start 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.760+0000 7fea48a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea4415d4b0 0x7fea440d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.760+0000 7fea48a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.760+0000 7fea48a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fea440cddf0 0x7fea440ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.760+0000 7fea48a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea440d0320 con 0x7fea4415d4b0 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.760+0000 7fea48a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea440b2730 con 0x7fea440cddf0 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.760+0000 7fea48a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea440d1ec0 con 0x7fea440cf9a0 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea42800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea4415d4b0 0x7fea440d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea42800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea4415d4b0 0x7fea440d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52656/0 (socket says 172.21.15.26:52656) 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea42800640 1 -- 172.21.15.26:0/1775127540 learned_addr learned my addr 172.21.15.26:0/1775127540 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:59.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea3be00640 1 --2- 172.21.15.26:0/1775127540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea42800640 1 -- 172.21.15.26:0/1775127540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 msgr2=0x7fea440cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea42800640 1 --2- 172.21.15.26:0/1775127540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea42800640 1 -- 172.21.15.26:0/1775127540 >> v2:172.21.15.124:3300/0 conn(0x7fea440cddf0 msgr2=0x7fea440ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:07:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea42800640 1 --2- 172.21.15.26:0/1775127540 >> v2:172.21.15.124:3300/0 conn(0x7fea440cddf0 0x7fea440ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea42800640 1 -- 172.21.15.26:0/1775127540 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea440b29d0 con 0x7fea4415d4b0 2024-01-15T18:07:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.761+0000 7fea3be00640 1 --2- 172.21.15.26:0/1775127540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.762+0000 7fea42800640 1 --2- 172.21.15.26:0/1775127540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4415d4b0 0x7fea440d1550 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fea34009870 tx=0x7fea3402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584f1707ffa7217e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.763+0000 7fea41e00640 1 -- 172.21.15.26:0/1775127540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34031640 con 0x7fea4415d4b0 2024-01-15T18:07:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.763+0000 7fea41e00640 1 -- 172.21.15.26:0/1775127540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea340317a0 con 0x7fea4415d4b0 2024-01-15T18:07:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.763+0000 7fea41e00640 1 -- 172.21.15.26:0/1775127540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34031b30 con 0x7fea4415d4b0 2024-01-15T18:07:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.763+0000 7fea48a00640 1 -- 172.21.15.26:0/1775127540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4415d4b0 msgr2=0x7fea440d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.763+0000 7fea48a00640 1 --2- 172.21.15.26:0/1775127540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4415d4b0 0x7fea440d1550 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fea34009870 tx=0x7fea3402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.764+0000 7fea48a00640 1 -- 172.21.15.26:0/1775127540 shutdown_connections 2024-01-15T18:07:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.764+0000 7fea48a00640 1 --2- 172.21.15.26:0/1775127540 >> v2:172.21.15.124:3300/0 conn(0x7fea440cddf0 0x7fea440ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.764+0000 7fea48a00640 1 --2- 172.21.15.26:0/1775127540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.764+0000 7fea48a00640 1 --2- 172.21.15.26:0/1775127540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4415d4b0 0x7fea440d1550 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.764+0000 7fea48a00640 1 -- 172.21.15.26:0/1775127540 >> 172.21.15.26:0/1775127540 conn(0x7fea440d3000 msgr2=0x7fea440d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.764+0000 7fea48a00640 1 -- 172.21.15.26:0/1775127540 shutdown_connections 2024-01-15T18:07:59.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.764+0000 7fea48a00640 1 -- 172.21.15.26:0/1775127540 wait complete. 2024-01-15T18:07:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.766+0000 7fea48a00640 1 Processor -- start 2024-01-15T18:07:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.767+0000 7fea48a00640 1 -- start start 2024-01-15T18:07:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.768+0000 7fea48a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea440cddf0 0x7fea440b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.768+0000 7fea48a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.768+0000 7fea48a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4415d4b0 0x7fea441e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.768+0000 7fea48a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea44146a00 con 0x7fea440cddf0 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.768+0000 7fea48a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea440b13d0 con 0x7fea4415d4b0 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.768+0000 7fea48a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea440b10f0 con 0x7fea440cf9a0 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.768+0000 7fea42800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea440cddf0 0x7fea440b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.768+0000 7fea42800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea440cddf0 0x7fea440b8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52664/0 (socket says 172.21.15.26:52664) 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.768+0000 7fea42800640 1 -- 172.21.15.26:0/3773076454 learned_addr learned my addr 172.21.15.26:0/3773076454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea3be00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea42800640 1 -- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 msgr2=0x7fea440b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea42800640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea43200640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4415d4b0 0x7fea441e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea42800640 1 -- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4415d4b0 msgr2=0x7fea441e9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea42800640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4415d4b0 0x7fea441e9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea42800640 1 -- 172.21.15.26:0/3773076454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea340094d0 con 0x7fea440cddf0 2024-01-15T18:07:59.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea3be00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea43200640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4415d4b0 0x7fea441e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:07:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea42800640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea440cddf0 0x7fea440b8670 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fea34000c00 tx=0x7fea340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.769+0000 7fea3aa00640 1 -- 172.21.15.26:0/3773076454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3403ac60 con 0x7fea440cddf0 2024-01-15T18:07:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.770+0000 7fea3aa00640 1 -- 172.21.15.26:0/3773076454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3403adc0 con 0x7fea440cddf0 2024-01-15T18:07:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.770+0000 7fea3aa00640 1 -- 172.21.15.26:0/3773076454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34044630 con 0x7fea440cddf0 2024-01-15T18:07:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.770+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4415da10 con 0x7fea440cddf0 2024-01-15T18:07:59.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.770+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea44146690 con 0x7fea440cddf0 2024-01-15T18:07:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.771+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea44067450 con 0x7fea440cddf0 2024-01-15T18:07:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.772+0000 7fea3aa00640 1 -- 172.21.15.26:0/3773076454 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fea3402f050 con 0x7fea440cddf0 2024-01-15T18:07:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.773+0000 7fea3aa00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1c06bfb0 0x7fea1c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:07:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.773+0000 7fea3aa00640 1 -- 172.21.15.26:0/3773076454 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fea340aa880 con 0x7fea440cddf0 2024-01-15T18:07:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.773+0000 7fea3be00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1c06bfb0 0x7fea1c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:07:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.775+0000 7fea3be00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1c06bfb0 0x7fea1c06e470 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006fd0 tx=0x7fea2c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:07:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.777+0000 7fea3aa00640 1 -- 172.21.15.26:0/3773076454 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea34076eb0 con 0x7fea440cddf0 2024-01-15T18:07:59.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.973+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea440cd290 con 0x7fea440cddf0 2024-01-15T18:07:59.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.974+0000 7fea3aa00640 1 -- 172.21.15.26:0/3773076454 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea340350a0 con 0x7fea440cddf0 2024-01-15T18:07:59.979 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1c06bfb0 msgr2=0x7fea1c06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1c06bfb0 0x7fea1c06e470 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fea2c006fd0 tx=0x7fea2c008040 comp rx=0 tx=0).stop 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea440cddf0 msgr2=0x7fea440b8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea440cddf0 0x7fea440b8670 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fea34000c00 tx=0x7fea340046c0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 shutdown_connections 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea440cf9a0 0x7fea440b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea1c06bfb0 0x7fea1c06e470 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4415d4b0 0x7fea441e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 --2- 172.21.15.26:0/3773076454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea440cddf0 0x7fea440b8670 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:07:59.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 >> 172.21.15.26:0/3773076454 conn(0x7fea440d3000 msgr2=0x7fea440b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:07:59.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 shutdown_connections 2024-01-15T18:07:59.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:07:59.977+0000 7fea48a00640 1 -- 172.21.15.26:0/3773076454 wait complete. 2024-01-15T18:08:00.002 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:08:00.161 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:08:00.169 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:08:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.207+0000 7f6cf2600640 1 Processor -- start 2024-01-15T18:08:00.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.208+0000 7f6cf2600640 1 -- start start 2024-01-15T18:08:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.209+0000 7f6cf2600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6cec14b2c0 0x7f6cec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.209+0000 7f6cf2600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec0cfbf0 con 0x7f6cec0cf7d0 2024-01-15T18:08:00.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.209+0000 7f6cf2600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec071690 con 0x7f6cec071070 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.209+0000 7f6cf2600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec14b6a0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.209+0000 7f6ceb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6cec14b2c0 0x7f6cec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.209+0000 7f6ceb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6cec14b2c0 0x7f6cec0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52668/0 (socket says 172.21.15.26:52668) 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.209+0000 7f6ce2800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6cec0cf7d0 0x7f6cec070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50740/0 (socket says 172.21.15.26:50740) 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.209+0000 7f6ceb400640 1 -- 172.21.15.26:0/10012408 learned_addr learned my addr 172.21.15.26:0/10012408 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.210+0000 7f6ceb400640 1 -- 172.21.15.26:0/10012408 >> v1:172.21.15.124:6789/0 conn(0x7f6cec071070 legacy=0x7f6cec0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.210+0000 7f6ceb400640 1 -- 172.21.15.26:0/10012408 >> v1:172.21.15.26:6789/0 conn(0x7f6cec0cf7d0 legacy=0x7f6cec070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.210+0000 7f6ceb400640 1 -- 172.21.15.26:0/10012408 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec071970 con 0x7f6cec14b2c0 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.210+0000 7f6ceaa00640 1 -- 172.21.15.26:0/10012408 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304927624 0 0) 0x7f6cec0cfbf0 con 0x7f6cec0cf7d0 2024-01-15T18:08:00.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.211+0000 7f6ceb400640 1 --2- 172.21.15.26:0/10012408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 0x7f6cec0cf290 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009870 tx=0x7f6cdc02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520387ec8394aa96 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.211+0000 7f6ceaa00640 1 -- 172.21.15.26:0/10012408 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc02f850 con 0x7f6cec14b2c0 2024-01-15T18:08:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.211+0000 7f6ceaa00640 1 -- 172.21.15.26:0/10012408 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc02f9b0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.212+0000 7f6ceaa00640 1 -- 172.21.15.26:0/10012408 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc02fd40 con 0x7f6cec14b2c0 2024-01-15T18:08:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.212+0000 7f6cf2600640 1 -- 172.21.15.26:0/10012408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 msgr2=0x7f6cec0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.212+0000 7f6cf2600640 1 --2- 172.21.15.26:0/10012408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 0x7f6cec0cf290 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009870 tx=0x7f6cdc02ba30 comp rx=0 tx=0).stop 2024-01-15T18:08:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.212+0000 7f6cf2600640 1 -- 172.21.15.26:0/10012408 shutdown_connections 2024-01-15T18:08:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.212+0000 7f6cf2600640 1 --2- 172.21.15.26:0/10012408 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 0x7f6cec0cf290 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.212+0000 7f6cf2600640 1 -- 172.21.15.26:0/10012408 >> 172.21.15.26:0/10012408 conn(0x7f6cec06a800 msgr2=0x7f6cec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.212+0000 7f6cf2600640 1 -- 172.21.15.26:0/10012408 shutdown_connections 2024-01-15T18:08:00.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.212+0000 7f6cf2600640 1 -- 172.21.15.26:0/10012408 wait complete. 2024-01-15T18:08:00.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.214+0000 7f6cf2600640 1 Processor -- start 2024-01-15T18:08:00.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.214+0000 7f6cf2600640 1 -- start start 2024-01-15T18:08:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.215+0000 7f6cf2600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cec071070 0x7f6cec1422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.215+0000 7f6cf2600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cec0cf7d0 0x7f6cec1473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.215+0000 7f6cf2600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 0x7f6cec154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.215+0000 7f6cf2600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec142800 con 0x7f6cec14b2c0 2024-01-15T18:08:00.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.215+0000 7f6cf2600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec146330 con 0x7f6cec071070 2024-01-15T18:08:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.215+0000 7f6cf2600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec146610 con 0x7f6cec0cf7d0 2024-01-15T18:08:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6cebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 0x7f6cec154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6ceb400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cec071070 0x7f6cec1422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6ce2800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cec0cf7d0 0x7f6cec1473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6ceb400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cec071070 0x7f6cec1422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:57656/0 (socket says 172.21.15.26:57656) 2024-01-15T18:08:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6ceb400640 1 -- 172.21.15.26:0/333071206 learned_addr learned my addr 172.21.15.26:0/333071206 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:00.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6cebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 0x7f6cec154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52682/0 (socket says 172.21.15.26:52682) 2024-01-15T18:08:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6cebe00640 1 -- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cec0cf7d0 msgr2=0x7f6cec1473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6cebe00640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cec0cf7d0 0x7f6cec1473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6cebe00640 1 -- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cec071070 msgr2=0x7f6cec1422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6cebe00640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cec071070 0x7f6cec1422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6cebe00640 1 -- 172.21.15.26:0/333071206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0094d0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.216+0000 7f6cebe00640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 0x7f6cec154da0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c690 tx=0x7f6ce400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.217+0000 7f6ce3e00640 1 -- 172.21.15.26:0/333071206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4003b70 con 0x7f6cec14b2c0 2024-01-15T18:08:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.217+0000 7f6ce3e00640 1 -- 172.21.15.26:0/333071206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce4003cd0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.217+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec1468f0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.217+0000 7f6ce3e00640 1 -- 172.21.15.26:0/333071206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4005310 con 0x7f6cec14b2c0 2024-01-15T18:08:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.217+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec146eb0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.218+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec0675d0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.219+0000 7f6ce3e00640 1 -- 172.21.15.26:0/333071206 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f6ce4005470 con 0x7f6cec14b2c0 2024-01-15T18:08:00.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.220+0000 7f6ce3e00640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6cc406bfb0 0x7f6cc406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:00.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.220+0000 7f6ce3e00640 1 -- 172.21.15.26:0/333071206 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f6ce4088200 con 0x7f6cec14b2c0 2024-01-15T18:08:00.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.221+0000 7f6ceb400640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6cc406bfb0 0x7f6cc406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:00.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.221+0000 7f6ceb400640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6cc406bfb0 0x7f6cc406e470 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc02ba70 tx=0x7f6cdc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.225+0000 7f6ce3e00640 1 -- 172.21.15.26:0/333071206 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ce4054bd0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.434+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6cec0b32b0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.435+0000 7f6ce3e00640 1 -- 172.21.15.26:0/333071206 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v79) v1 ==== 80+0+122 (secure 0 0 0) 0x7f6ce40581f0 con 0x7f6cec14b2c0 2024-01-15T18:08:00.440 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6835/1813593645 2024-01-16T18:07:55.696353+0000 2024-01-15T18:08:00.440 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6833/1813593645 2024-01-16T18:07:55.696353+0000 2024-01-15T18:08:00.440 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:08:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6cc406bfb0 msgr2=0x7f6cc406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6cc406bfb0 0x7f6cc406e470 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc02ba70 tx=0x7f6cdc008030 comp rx=0 tx=0).stop 2024-01-15T18:08:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 msgr2=0x7f6cec154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 0x7f6cec154da0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400c690 tx=0x7f6ce400cb50 comp rx=0 tx=0).stop 2024-01-15T18:08:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 shutdown_connections 2024-01-15T18:08:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6cec0cf7d0 0x7f6cec1473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6cc406bfb0 0x7f6cc406e470 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6cec14b2c0 0x7f6cec154da0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:00.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 --2- 172.21.15.26:0/333071206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6cec071070 0x7f6cec1422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 >> 172.21.15.26:0/333071206 conn(0x7f6cec06a800 msgr2=0x7f6cec1492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 shutdown_connections 2024-01-15T18:08:00.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:00.438+0000 7f6cf2600640 1 -- 172.21.15.26:0/333071206 wait complete. 2024-01-15T18:08:00.461 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-01-15T18:08:00.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:00.461 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:08:00.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[62099]: fuse finished with error 0 and tester_r 0 2024-01-15T18:08:00.556 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:08:00.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:00.556 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:08:00.775 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:08:00.775 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:08:00.832 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:08:01.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.058+0000 7f0c1a400640 1 Processor -- start 2024-01-15T18:08:01.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.058+0000 7f0c1a400640 1 -- start start 2024-01-15T18:08:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.060+0000 7f0c1a400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0c14162430 0x7f0c140b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.060+0000 7f0c1a400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0c140b3650 0x7f0c1406c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.060+0000 7f0c1a400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c140702e0 con 0x7f0c14162810 2024-01-15T18:08:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.060+0000 7f0c1a400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140705c0 con 0x7f0c14162430 2024-01-15T18:08:01.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.060+0000 7f0c1a400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140708a0 con 0x7f0c140b3650 2024-01-15T18:08:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.060+0000 7f0c19000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0c14162430 0x7f0c140b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.061+0000 7f0c19000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0c14162430 0x7f0c140b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52698/0 (socket says 172.21.15.26:52698) 2024-01-15T18:08:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.061+0000 7f0c19000640 1 -- 172.21.15.26:0/215943195 learned_addr learned my addr 172.21.15.26:0/215943195 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.061+0000 7f0c19a00640 1 --2- 172.21.15.26:0/215943195 >> v2:172.21.15.124:3300/0 conn(0x7f0c140b3650 0x7f0c1406c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.061+0000 7f0c19000640 1 -- 172.21.15.26:0/215943195 >> v2:172.21.15.124:3300/0 conn(0x7f0c140b3650 msgr2=0x7f0c1406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.061+0000 7f0c19000640 1 --2- 172.21.15.26:0/215943195 >> v2:172.21.15.124:3300/0 conn(0x7f0c140b3650 0x7f0c1406c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.061+0000 7f0c19000640 1 -- 172.21.15.26:0/215943195 >> v1:172.21.15.124:6789/0 conn(0x7f0c14162810 legacy=0x7f0c14071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:01.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.061+0000 7f0c19000640 1 -- 172.21.15.26:0/215943195 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14070b80 con 0x7f0c14162430 2024-01-15T18:08:01.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.063+0000 7f0c19000640 1 --2- 172.21.15.26:0/215943195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14162430 0x7f0c140b3110 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0c14161ad0 tx=0x7f0c0402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a861dd5ac0a335 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:01.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.063+0000 7f0c13400640 1 -- 172.21.15.26:0/215943195 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04031870 con 0x7f0c14162430 2024-01-15T18:08:01.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.063+0000 7f0c13400640 1 -- 172.21.15.26:0/215943195 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c040319d0 con 0x7f0c14162430 2024-01-15T18:08:01.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.063+0000 7f0c13400640 1 -- 172.21.15.26:0/215943195 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04031d60 con 0x7f0c14162430 2024-01-15T18:08:01.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.064+0000 7f0c1a400640 1 -- 172.21.15.26:0/215943195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14162430 msgr2=0x7f0c140b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.064+0000 7f0c1a400640 1 --2- 172.21.15.26:0/215943195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14162430 0x7f0c140b3110 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f0c14161ad0 tx=0x7f0c0402b910 comp rx=0 tx=0).stop 2024-01-15T18:08:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.064+0000 7f0c1a400640 1 -- 172.21.15.26:0/215943195 shutdown_connections 2024-01-15T18:08:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.064+0000 7f0c1a400640 1 --2- 172.21.15.26:0/215943195 >> v2:172.21.15.124:3300/0 conn(0x7f0c140b3650 0x7f0c1406c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.064+0000 7f0c1a400640 1 --2- 172.21.15.26:0/215943195 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14162430 0x7f0c140b3110 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.064+0000 7f0c1a400640 1 -- 172.21.15.26:0/215943195 >> 172.21.15.26:0/215943195 conn(0x7f0c1406a800 msgr2=0x7f0c1406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.064+0000 7f0c1a400640 1 -- 172.21.15.26:0/215943195 shutdown_connections 2024-01-15T18:08:01.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.064+0000 7f0c1a400640 1 -- 172.21.15.26:0/215943195 wait complete. 2024-01-15T18:08:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.066+0000 7f0c1a400640 1 Processor -- start 2024-01-15T18:08:01.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.066+0000 7f0c1a400640 1 -- start start 2024-01-15T18:08:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.067+0000 7f0c1a400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c140b3650 0x7f0c1415de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.067+0000 7f0c1a400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c14162430 0x7f0c14159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.067+0000 7f0c1a400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14159970 0x7f0c14163c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.067+0000 7f0c1a400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1415aae0 con 0x7f0c14159970 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.067+0000 7f0c1a400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14159ed0 con 0x7f0c14162430 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.067+0000 7f0c1a400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1415a1b0 con 0x7f0c140b3650 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.067+0000 7f0c19a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14159970 0x7f0c14163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.067+0000 7f0c19a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14159970 0x7f0c14163c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52704/0 (socket says 172.21.15.26:52704) 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.067+0000 7f0c19a00640 1 -- 172.21.15.26:0/4080579621 learned_addr learned my addr 172.21.15.26:0/4080579621 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.068+0000 7f0c19000640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c140b3650 0x7f0c1415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.068+0000 7f0c13e00640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c14162430 0x7f0c14159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.068+0000 7f0c19a00640 1 -- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c140b3650 msgr2=0x7f0c1415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.068+0000 7f0c19a00640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c140b3650 0x7f0c1415de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.068+0000 7f0c19a00640 1 -- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c14162430 msgr2=0x7f0c14159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.068+0000 7f0c19a00640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c14162430 0x7f0c14159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.068+0000 7f0c19a00640 1 -- 172.21.15.26:0/4080579621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c040094d0 con 0x7f0c14159970 2024-01-15T18:08:01.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.068+0000 7f0c19a00640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14159970 0x7f0c14163c50 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c007e30 tx=0x7f0c0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.069+0000 7f0c10c00640 1 -- 172.21.15.26:0/4080579621 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c014620 con 0x7f0c14159970 2024-01-15T18:08:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.069+0000 7f0c10c00640 1 -- 172.21.15.26:0/4080579621 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c00ee90 con 0x7f0c14159970 2024-01-15T18:08:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.069+0000 7f0c10c00640 1 -- 172.21.15.26:0/4080579621 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0149c0 con 0x7f0c14159970 2024-01-15T18:08:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.069+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1415a490 con 0x7f0c14159970 2024-01-15T18:08:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.069+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1415c300 con 0x7f0c14159970 2024-01-15T18:08:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.070+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c14071240 con 0x7f0c14159970 2024-01-15T18:08:01.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.071+0000 7f0c10c00640 1 -- 172.21.15.26:0/4080579621 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f0c0c014b20 con 0x7f0c14159970 2024-01-15T18:08:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.072+0000 7f0c10c00640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bec06bfb0 0x7f0bec06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.072+0000 7f0c10c00640 1 -- 172.21.15.26:0/4080579621 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f0c0c088f70 con 0x7f0c14159970 2024-01-15T18:08:01.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.072+0000 7f0c19000640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bec06bfb0 0x7f0bec06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.074+0000 7f0c19000640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bec06bfb0 0x7f0bec06e470 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0c04036000 tx=0x7f0c0403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:01.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.076+0000 7f0c10c00640 1 -- 172.21.15.26:0/4080579621 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c0c055830 con 0x7f0c14159970 2024-01-15T18:08:01.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.278+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c1415c490 con 0x7f0c14159970 2024-01-15T18:08:01.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.280+0000 7f0c10c00640 1 -- 172.21.15.26:0/4080579621 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13050 (secure 0 0 0) 0x7f0c0c058e50 con 0x7f0c14159970 2024-01-15T18:08:01.285 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:01.285 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:07:58.712271+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1813593645":"2024-01-16T18:07:55.696353+0000","172.21.15.26:6833/1813593645":"2024-01-16T18:07:55.696353+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:08:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bec06bfb0 msgr2=0x7f0bec06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bec06bfb0 0x7f0bec06e470 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0c04036000 tx=0x7f0c0403c040 comp rx=0 tx=0).stop 2024-01-15T18:08:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14159970 msgr2=0x7f0c14163c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14159970 0x7f0c14163c50 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c007e30 tx=0x7f0c0c00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 shutdown_connections 2024-01-15T18:08:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0bec06bfb0 0x7f0bec06e470 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0c14159970 0x7f0c14163c50 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0c14162430 0x7f0c14159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 --2- 172.21.15.26:0/4080579621 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0c140b3650 0x7f0c1415de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 >> 172.21.15.26:0/4080579621 conn(0x7f0c1406a800 msgr2=0x7f0c1406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 shutdown_connections 2024-01-15T18:08:01.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.283+0000 7f0c1a400640 1 -- 172.21.15.26:0/4080579621 wait complete. 2024-01-15T18:08:01.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:08:01.306 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:08:01.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.528+0000 7f0f56400640 1 Processor -- start 2024-01-15T18:08:01.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.528+0000 7f0f56400640 1 -- start start 2024-01-15T18:08:01.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.530+0000 7f0f56400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f500cf9a0 0x7f0f50166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.530+0000 7f0f56400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5015c5f0 con 0x7f0f5013df80 2024-01-15T18:08:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.530+0000 7f0f56400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5006c470 con 0x7f0f5013e360 2024-01-15T18:08:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.530+0000 7f0f56400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5006c750 con 0x7f0f500cf9a0 2024-01-15T18:08:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.530+0000 7f0f4f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0f5013df80 0x7f0f5015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:50742/0 (socket says 172.21.15.26:50742) 2024-01-15T18:08:01.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.530+0000 7f0f4f400640 1 -- 172.21.15.26:0/3106695381 learned_addr learned my addr 172.21.15.26:0/3106695381 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.530+0000 7f0f4fe00640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f500cf9a0 0x7f0f50166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178350847 0 0) 0x7f0f5015c5f0 con 0x7f0f5013df80 2024-01-15T18:08:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f30003520 con 0x7f0f5013df80 2024-01-15T18:08:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061071359 0 0) 0x7f0f30003520 con 0x7f0f5013df80 2024-01-15T18:08:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f500cf9a0 msgr2=0x7f0f50166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:01.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f500cf9a0 0x7f0f50166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 >> v1:172.21.15.124:6789/0 conn(0x7f0f5013e360 legacy=0x7f0f500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5006ca30 con 0x7f0f5013df80 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f0f400019f0 con 0x7f0f5013df80 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 >> v1:172.21.15.26:6789/0 conn(0x7f0f5013df80 legacy=0x7f0f5015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f300039a0 0x7f0f30003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f500cf9a0 0x7f0f300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5006c750 con 0x7f0f30004d00 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3000ba90 con 0x7f0f300039a0 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.531+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3000bc50 con 0x7f0f500cf9a0 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.532+0000 7f0f4fe00640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.532+0000 7f0f4f400640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f300039a0 0x7f0f30003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.532+0000 7f0f46800640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f500cf9a0 0x7f0f300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.532+0000 7f0f4fe00640 1 -- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f500cf9a0 msgr2=0x7f0f300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.532+0000 7f0f4fe00640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f500cf9a0 0x7f0f300047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.532+0000 7f0f4fe00640 1 -- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f300039a0 msgr2=0x7f0f30003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.532+0000 7f0f4fe00640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f300039a0 0x7f0f30003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.532+0000 7f0f4fe00640 1 -- 172.21.15.26:0/3106695381 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30003520 con 0x7f0f30004d00 2024-01-15T18:08:01.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.532+0000 7f0f46800640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f500cf9a0 0x7f0f300047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.533+0000 7f0f4fe00640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f30005180 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f0f50155d40 tx=0x7f0f300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed64344ae5f466c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:01.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4802c660 con 0x7f0f30004d00 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4802c7c0 con 0x7f0f30004d00 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f4ea00640 1 -- 172.21.15.26:0/3106695381 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4802cb50 con 0x7f0f30004d00 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f56400640 1 -- 172.21.15.26:0/3106695381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 msgr2=0x7f0f30005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f56400640 1 --2- 172.21.15.26:0/3106695381 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f30005180 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f0f50155d40 tx=0x7f0f300057f0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f4fe00640 1 -- 172.21.15.26:0/3106695381 reap_dead start 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f56400640 1 -- 172.21.15.26:0/3106695381 shutdown_connections 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f56400640 1 -- 172.21.15.26:0/3106695381 >> 172.21.15.26:0/3106695381 conn(0x7f0f5006a800 msgr2=0x7f0f5006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f56400640 1 -- 172.21.15.26:0/3106695381 shutdown_connections 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.534+0000 7f0f56400640 1 -- 172.21.15.26:0/3106695381 wait complete. 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.536+0000 7f0f56400640 1 Processor -- start 2024-01-15T18:08:01.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.536+0000 7f0f56400640 1 -- start start 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.537+0000 7f0f56400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f5013d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.537+0000 7f0f56400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f501e5e30 0x7f0f5015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.537+0000 7f0f56400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f501e62b0 0x7f0f5015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.537+0000 7f0f56400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5013ddf0 con 0x7f0f30004d00 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.537+0000 7f0f56400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501e7920 con 0x7f0f501e5e30 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.537+0000 7f0f56400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501e6880 con 0x7f0f501e62b0 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f5013d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f5013d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52734/0 (socket says 172.21.15.26:52734) 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4f400640 1 -- 172.21.15.26:0/2122243585 learned_addr learned my addr 172.21.15.26:0/2122243585 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4fe00640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f501e62b0 0x7f0f5015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4f400640 1 -- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f501e62b0 msgr2=0x7f0f5015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f46800640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f501e5e30 0x7f0f5015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4f400640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f501e62b0 0x7f0f5015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4f400640 1 -- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f501e5e30 msgr2=0x7f0f5015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4f400640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f501e5e30 0x7f0f5015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4f400640 1 -- 172.21.15.26:0/2122243585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f501e6b60 con 0x7f0f30004d00 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4fe00640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f501e62b0 0x7f0f5015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f46800640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f501e5e30 0x7f0f5015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.538+0000 7f0f4f400640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f5013d8b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0f40005fd0 tx=0x7f0f4000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.539+0000 7f0f47e00640 1 -- 172.21.15.26:0/2122243585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4000edf0 con 0x7f0f30004d00 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.539+0000 7f0f47e00640 1 -- 172.21.15.26:0/2122243585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40013420 con 0x7f0f30004d00 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.539+0000 7f0f47e00640 1 -- 172.21.15.26:0/2122243585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400137d0 con 0x7f0f30004d00 2024-01-15T18:08:01.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.539+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f501e6e40 con 0x7f0f30004d00 2024-01-15T18:08:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.539+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f501e7400 con 0x7f0f30004d00 2024-01-15T18:08:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.540+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f500675d0 con 0x7f0f30004d00 2024-01-15T18:08:01.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.541+0000 7f0f47e00640 1 -- 172.21.15.26:0/2122243585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f0f40013930 con 0x7f0f30004d00 2024-01-15T18:08:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.542+0000 7f0f47e00640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0f2806bfe0 0x7f0f2806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.542+0000 7f0f47e00640 1 -- 172.21.15.26:0/2122243585 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f0f400890c0 con 0x7f0f30004d00 2024-01-15T18:08:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.543+0000 7f0f46800640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0f2806bfe0 0x7f0f2806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.544+0000 7f0f46800640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0f2806bfe0 0x7f0f2806e4a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0f50150830 tx=0x7f0f38009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:01.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.546+0000 7f0f47e00640 1 -- 172.21.15.26:0/2122243585 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f40059180 con 0x7f0f30004d00 2024-01-15T18:08:01.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.742+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0f5006df00 con 0x7f0f30004d00 2024-01-15T18:08:01.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.743+0000 7f0f47e00640 1 -- 172.21.15.26:0/2122243585 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0f400018e0 con 0x7f0f30004d00 2024-01-15T18:08:01.748 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:08:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0f2806bfe0 msgr2=0x7f0f2806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0f2806bfe0 0x7f0f2806e4a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0f50150830 tx=0x7f0f38009040 comp rx=0 tx=0).stop 2024-01-15T18:08:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 msgr2=0x7f0f5013d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f5013d8b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0f40005fd0 tx=0x7f0f4000e530 comp rx=0 tx=0).stop 2024-01-15T18:08:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 shutdown_connections 2024-01-15T18:08:01.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0f501e62b0 0x7f0f5015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0f2806bfe0 0x7f0f2806e4a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0f501e5e30 0x7f0f5015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 --2- 172.21.15.26:0/2122243585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0f30004d00 0x7f0f5013d8b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 >> 172.21.15.26:0/2122243585 conn(0x7f0f5006a800 msgr2=0x7f0f500b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.746+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 shutdown_connections 2024-01-15T18:08:01.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.747+0000 7f0f56400640 1 -- 172.21.15.26:0/2122243585 wait complete. 2024-01-15T18:08:01.772 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:08:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.977+0000 7f9f68a00640 1 Processor -- start 2024-01-15T18:08:01.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.978+0000 7f9f68a00640 1 -- start start 2024-01-15T18:08:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.979+0000 7f9f68a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f6413cfa0 0x7f9f6415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.979+0000 7f9f68a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9f640cd530 0x7f9f6413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.979+0000 7f9f68a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f640cd9e0 0x7f9f6413fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.979+0000 7f9f68a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f640cdec0 con 0x7f9f6413cfa0 2024-01-15T18:08:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.979+0000 7f9f68a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f640ce1a0 con 0x7f9f640cd530 2024-01-15T18:08:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.979+0000 7f9f68a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f640ce480 con 0x7f9f640cd9e0 2024-01-15T18:08:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f62800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f6413cfa0 0x7f9f6415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f62800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9f6413cfa0 0x7f9f6415afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49902/0 (socket says 172.21.15.26:49902) 2024-01-15T18:08:01.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f62800640 1 -- 172.21.15.26:0/4255121281 learned_addr learned my addr 172.21.15.26:0/4255121281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f61e00640 1 --2- 172.21.15.26:0/4255121281 >> v2:172.21.15.124:3300/0 conn(0x7f9f640cd530 0x7f9f6413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f63200640 1 --2- 172.21.15.26:0/4255121281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f640cd9e0 0x7f9f6413fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f62800640 1 -- 172.21.15.26:0/4255121281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f640cd9e0 msgr2=0x7f9f6413fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f62800640 1 --2- 172.21.15.26:0/4255121281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f640cd9e0 0x7f9f6413fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f62800640 1 -- 172.21.15.26:0/4255121281 >> v2:172.21.15.124:3300/0 conn(0x7f9f640cd530 msgr2=0x7f9f6413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f62800640 1 --2- 172.21.15.26:0/4255121281 >> v2:172.21.15.124:3300/0 conn(0x7f9f640cd530 0x7f9f6413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f62800640 1 -- 172.21.15.26:0/4255121281 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640ce760 con 0x7f9f6413cfa0 2024-01-15T18:08:01.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.980+0000 7f9f61e00640 1 --2- 172.21.15.26:0/4255121281 >> v2:172.21.15.124:3300/0 conn(0x7f9f640cd530 0x7f9f6413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.982+0000 7f9f62800640 1 --2- 172.21.15.26:0/4255121281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f6413cfa0 0x7f9f6415afb0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9f6415b4f0 tx=0x7f9f5402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9784aff232665a90 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.982+0000 7f9f61400640 1 -- 172.21.15.26:0/4255121281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f540317d0 con 0x7f9f6413cfa0 2024-01-15T18:08:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.982+0000 7f9f61400640 1 -- 172.21.15.26:0/4255121281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f54031930 con 0x7f9f6413cfa0 2024-01-15T18:08:01.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.982+0000 7f9f61400640 1 -- 172.21.15.26:0/4255121281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f54031cc0 con 0x7f9f6413cfa0 2024-01-15T18:08:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.983+0000 7f9f68a00640 1 -- 172.21.15.26:0/4255121281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f6413cfa0 msgr2=0x7f9f6415afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.983+0000 7f9f68a00640 1 --2- 172.21.15.26:0/4255121281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f6413cfa0 0x7f9f6415afb0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f9f6415b4f0 tx=0x7f9f5402b870 comp rx=0 tx=0).stop 2024-01-15T18:08:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.983+0000 7f9f68a00640 1 -- 172.21.15.26:0/4255121281 shutdown_connections 2024-01-15T18:08:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.983+0000 7f9f68a00640 1 --2- 172.21.15.26:0/4255121281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f640cd9e0 0x7f9f6413fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.983+0000 7f9f68a00640 1 --2- 172.21.15.26:0/4255121281 >> v2:172.21.15.124:3300/0 conn(0x7f9f640cd530 0x7f9f6413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.983+0000 7f9f68a00640 1 --2- 172.21.15.26:0/4255121281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f6413cfa0 0x7f9f6415afb0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.983+0000 7f9f68a00640 1 -- 172.21.15.26:0/4255121281 >> 172.21.15.26:0/4255121281 conn(0x7f9f6406a800 msgr2=0x7f9f6406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:01.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.983+0000 7f9f68a00640 1 -- 172.21.15.26:0/4255121281 shutdown_connections 2024-01-15T18:08:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.983+0000 7f9f68a00640 1 -- 172.21.15.26:0/4255121281 wait complete. 2024-01-15T18:08:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.985+0000 7f9f68a00640 1 Processor -- start 2024-01-15T18:08:01.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.985+0000 7f9f68a00640 1 -- start start 2024-01-15T18:08:01.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.986+0000 7f9f68a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f640cd530 0x7f9f6414e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.986+0000 7f9f68a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f640cd9e0 0x7f9f641518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.986+0000 7f9f68a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6413cfa0 0x7f9f6412e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.986+0000 7f9f68a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f641f12f0 con 0x7f9f640cd9e0 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.986+0000 7f9f68a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64150830 con 0x7f9f640cd530 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.986+0000 7f9f68a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f64150b10 con 0x7f9f6413cfa0 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.986+0000 7f9f61e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f640cd9e0 0x7f9f641518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.986+0000 7f9f61e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f640cd9e0 0x7f9f641518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49904/0 (socket says 172.21.15.26:49904) 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.986+0000 7f9f61e00640 1 -- 172.21.15.26:0/471824200 learned_addr learned my addr 172.21.15.26:0/471824200 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f62800640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f640cd530 0x7f9f6414e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f63200640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6413cfa0 0x7f9f6412e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f61e00640 1 -- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6413cfa0 msgr2=0x7f9f6412e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f61e00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6413cfa0 0x7f9f6412e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f61e00640 1 -- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f640cd530 msgr2=0x7f9f6414e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f61e00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f640cd530 0x7f9f6414e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f61e00640 1 -- 172.21.15.26:0/471824200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f540094d0 con 0x7f9f640cd9e0 2024-01-15T18:08:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f62800640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f640cd530 0x7f9f6414e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:01.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f61e00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f640cd9e0 0x7f9f641518d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00e7e0 tx=0x7f9f4c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f5aa00640 1 -- 172.21.15.26:0/471824200 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004030 con 0x7f9f640cd9e0 2024-01-15T18:08:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.987+0000 7f9f5aa00640 1 -- 172.21.15.26:0/471824200 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4c004190 con 0x7f9f640cd9e0 2024-01-15T18:08:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.988+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f64150df0 con 0x7f9f640cd9e0 2024-01-15T18:08:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.988+0000 7f9f5aa00640 1 -- 172.21.15.26:0/471824200 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004520 con 0x7f9f640cd9e0 2024-01-15T18:08:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.988+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f641513b0 con 0x7f9f640cd9e0 2024-01-15T18:08:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.989+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f64067450 con 0x7f9f640cd9e0 2024-01-15T18:08:01.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.990+0000 7f9f5aa00640 1 -- 172.21.15.26:0/471824200 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f9f4c00c6d0 con 0x7f9f640cd9e0 2024-01-15T18:08:01.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.991+0000 7f9f5aa00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f3c06bfb0 0x7f9f3c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.991+0000 7f9f5aa00640 1 -- 172.21.15.26:0/471824200 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f9f4c088370 con 0x7f9f640cd9e0 2024-01-15T18:08:01.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.992+0000 7f9f62800640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f3c06bfb0 0x7f9f3c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:01.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.993+0000 7f9f62800640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f3c06bfb0 0x7f9f3c06e470 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f54036000 tx=0x7f9f5403d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:01.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:01.995+0000 7f9f5aa00640 1 -- 172.21.15.26:0/471824200 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f4c054c30 con 0x7f9f640cd9e0 2024-01-15T18:08:02.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.201+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9f640cbff0 con 0x7f9f640cd9e0 2024-01-15T18:08:02.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.202+0000 7f9f5aa00640 1 -- 172.21.15.26:0/471824200 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v79) v1 ==== 80+0+122 (secure 0 0 0) 0x7f9f4c020020 con 0x7f9f640cd9e0 2024-01-15T18:08:02.207 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6835/1813593645 2024-01-16T18:07:55.696353+0000 2024-01-15T18:08:02.207 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.26:6833/1813593645 2024-01-16T18:07:55.696353+0000 2024-01-15T18:08:02.207 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:08:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f3c06bfb0 msgr2=0x7f9f3c06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f3c06bfb0 0x7f9f3c06e470 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f54036000 tx=0x7f9f5403d040 comp rx=0 tx=0).stop 2024-01-15T18:08:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f640cd9e0 msgr2=0x7f9f641518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f640cd9e0 0x7f9f641518d0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c00e7e0 tx=0x7f9f4c00eca0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 shutdown_connections 2024-01-15T18:08:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9f6413cfa0 0x7f9f6412e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9f3c06bfb0 0x7f9f3c06e470 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9f640cd9e0 0x7f9f641518d0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 --2- 172.21.15.26:0/471824200 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9f640cd530 0x7f9f6414e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 >> 172.21.15.26:0/471824200 conn(0x7f9f6406a800 msgr2=0x7f9f64152110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 shutdown_connections 2024-01-15T18:08:02.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.205+0000 7f9f68a00640 1 -- 172.21.15.26:0/471824200 wait complete. 2024-01-15T18:08:02.229 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi124.front.sepia.ceph.com... 2024-01-15T18:08:02.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:02.229 DEBUG:teuthology.orchestra.run.smithi124:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:08:02.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[45691]: fuse finished with error 0 and tester_r 0 2024-01-15T18:08:02.329 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:08:02.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:02.329 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:08:02.518 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:08:02.518 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:08:02.579 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:08:02.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:08:02.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:02.580 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:08:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:08:02.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:08:02.635 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:08:02.635 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:08:02.691 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:08:02.691 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:08:02.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:02.691 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:08:02.746 INFO:teuthology.orchestra.run.smithi124.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:08:02.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:08:02.747 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:08:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.975+0000 7fd2e6800640 1 Processor -- start 2024-01-15T18:08:02.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.976+0000 7fd2e6800640 1 -- start start 2024-01-15T18:08:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.977+0000 7fd2e6800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2e0157c30 0x7fd2e00cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.977+0000 7fd2e6800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0071200 0x7fd2e006dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.977+0000 7fd2e6800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e015deb0 con 0x7fd2e01618f0 2024-01-15T18:08:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.977+0000 7fd2e6800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e015ea30 con 0x7fd2e0157c30 2024-01-15T18:08:02.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.977+0000 7fd2e6800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e006d860 con 0x7fd2e0071200 2024-01-15T18:08:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.978+0000 7fd2df400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2e0157c30 0x7fd2e00cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.978+0000 7fd2df400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd2e0157c30 0x7fd2e00cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49914/0 (socket says 172.21.15.26:49914) 2024-01-15T18:08:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.978+0000 7fd2df400640 1 -- 172.21.15.26:0/900115457 learned_addr learned my addr 172.21.15.26:0/900115457 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.978+0000 7fd2dfe00640 1 --2- 172.21.15.26:0/900115457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0071200 0x7fd2e006dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.978+0000 7fd2df400640 1 -- 172.21.15.26:0/900115457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0071200 msgr2=0x7fd2e006dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.978+0000 7fd2df400640 1 --2- 172.21.15.26:0/900115457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0071200 0x7fd2e006dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.978+0000 7fd2df400640 1 -- 172.21.15.26:0/900115457 >> v1:172.21.15.124:6789/0 conn(0x7fd2e01618f0 legacy=0x7fd2e0070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.978+0000 7fd2df400640 1 -- 172.21.15.26:0/900115457 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e015e190 con 0x7fd2e0157c30 2024-01-15T18:08:02.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.978+0000 7fd2dfe00640 1 --2- 172.21.15.26:0/900115457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0071200 0x7fd2e006dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.980+0000 7fd2df400640 1 --2- 172.21.15.26:0/900115457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e0157c30 0x7fd2e00cd600 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009870 tx=0x7fd2d002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9503132ba5b5b4d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.981+0000 7fd2de000640 1 -- 172.21.15.26:0/900115457 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0030840 con 0x7fd2e0157c30 2024-01-15T18:08:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.981+0000 7fd2de000640 1 -- 172.21.15.26:0/900115457 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d00309a0 con 0x7fd2e0157c30 2024-01-15T18:08:02.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.981+0000 7fd2de000640 1 -- 172.21.15.26:0/900115457 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0030d30 con 0x7fd2e0157c30 2024-01-15T18:08:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.981+0000 7fd2e6800640 1 -- 172.21.15.26:0/900115457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e0157c30 msgr2=0x7fd2e00cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.981+0000 7fd2e6800640 1 --2- 172.21.15.26:0/900115457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e0157c30 0x7fd2e00cd600 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0009870 tx=0x7fd2d002b980 comp rx=0 tx=0).stop 2024-01-15T18:08:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.981+0000 7fd2e6800640 1 -- 172.21.15.26:0/900115457 shutdown_connections 2024-01-15T18:08:02.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.981+0000 7fd2e6800640 1 --2- 172.21.15.26:0/900115457 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0071200 0x7fd2e006dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.981+0000 7fd2e6800640 1 --2- 172.21.15.26:0/900115457 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e0157c30 0x7fd2e00cd600 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.981+0000 7fd2e6800640 1 -- 172.21.15.26:0/900115457 >> 172.21.15.26:0/900115457 conn(0x7fd2e00cf700 msgr2=0x7fd2e00cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.982+0000 7fd2e6800640 1 -- 172.21.15.26:0/900115457 shutdown_connections 2024-01-15T18:08:02.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.982+0000 7fd2e6800640 1 -- 172.21.15.26:0/900115457 wait complete. 2024-01-15T18:08:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.983+0000 7fd2e6800640 1 Processor -- start 2024-01-15T18:08:02.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.984+0000 7fd2e6800640 1 -- start start 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.984+0000 7fd2e6800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2e0071200 0x7fd2e01490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.984+0000 7fd2e6800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0157c30 0x7fd2e015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.984+0000 7fd2e6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e01618f0 0x7fd2e0145d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2e6800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e01f6b10 con 0x7fd2e01618f0 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2e6800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e0146c50 con 0x7fd2e0071200 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2e6800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e015b1e0 con 0x7fd2e0157c30 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e01618f0 0x7fd2e0145d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e01618f0 0x7fd2e0145d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49926/0 (socket says 172.21.15.26:49926) 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2dfe00640 1 -- 172.21.15.26:0/2932566410 learned_addr learned my addr 172.21.15.26:0/2932566410 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:02.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2df400640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2e0071200 0x7fd2e01490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2dea00640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0157c30 0x7fd2e015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2dfe00640 1 -- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0157c30 msgr2=0x7fd2e015aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2dfe00640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0157c30 0x7fd2e015aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2dfe00640 1 -- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2e0071200 msgr2=0x7fd2e01490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2dfe00640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2e0071200 0x7fd2e01490e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:02.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.985+0000 7fd2dfe00640 1 -- 172.21.15.26:0/2932566410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d00094d0 con 0x7fd2e01618f0 2024-01-15T18:08:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.986+0000 7fd2dfe00640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e01618f0 0x7fd2e0145d40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8007e20 tx=0x7fd2d800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.986+0000 7fd2d7400640 1 -- 172.21.15.26:0/2932566410 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8003b90 con 0x7fd2e01618f0 2024-01-15T18:08:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.986+0000 7fd2d7400640 1 -- 172.21.15.26:0/2932566410 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d800f040 con 0x7fd2e01618f0 2024-01-15T18:08:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.986+0000 7fd2d7400640 1 -- 172.21.15.26:0/2932566410 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d800de80 con 0x7fd2e01618f0 2024-01-15T18:08:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.986+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e0146280 con 0x7fd2e01618f0 2024-01-15T18:08:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.986+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e0146600 con 0x7fd2e01618f0 2024-01-15T18:08:02.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.988+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e00675d0 con 0x7fd2e01618f0 2024-01-15T18:08:02.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.989+0000 7fd2d7400640 1 -- 172.21.15.26:0/2932566410 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fd2d8004030 con 0x7fd2e01618f0 2024-01-15T18:08:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.989+0000 7fd2d7400640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd2b806bfb0 0x7fd2b806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.990+0000 7fd2d7400640 1 -- 172.21.15.26:0/2932566410 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fd2d8088860 con 0x7fd2e01618f0 2024-01-15T18:08:02.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.990+0000 7fd2df400640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd2b806bfb0 0x7fd2b806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:02.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.991+0000 7fd2df400640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd2b806bfb0 0x7fd2b806e470 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0036000 tx=0x7fd2d0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:02.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:02.994+0000 7fd2d7400640 1 -- 172.21.15.26:0/2932566410 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2d8055180 con 0x7fd2e01618f0 2024-01-15T18:08:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.226+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2e0146d90 con 0x7fd2e01618f0 2024-01-15T18:08:03.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.227+0000 7fd2d7400640 1 -- 172.21.15.26:0/2932566410 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fd2d80587a0 con 0x7fd2e01618f0 2024-01-15T18:08:03.232 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:03.232 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1986791067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1986791067},{"type":"v1","addr":"172.21.15.26:6834","nonce":1986791067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6082,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3285856051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3285856051},{"type":"v1","addr":"172.21.15.124:6836","nonce":3285856051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6085,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/198642905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":198642905},{"type":"v1","addr":"172.21.15.124:6837","nonce":198642905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":8100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4154591970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4154591970},{"type":"v1","addr":"172.21.15.26:6835","nonce":4154591970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[]} 2024-01-15T18:08:03.233 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 47 2024-01-15T18:08:03.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd2b806bfb0 msgr2=0x7fd2b806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd2b806bfb0 0x7fd2b806e470 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0036000 tx=0x7fd2d0008030 comp rx=0 tx=0).stop 2024-01-15T18:08:03.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e01618f0 msgr2=0x7fd2e0145d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e01618f0 0x7fd2e0145d40 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8007e20 tx=0x7fd2d800d4a0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 shutdown_connections 2024-01-15T18:08:03.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd2e0157c30 0x7fd2e015aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd2b806bfb0 0x7fd2b806e470 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd2e01618f0 0x7fd2e0145d40 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 --2- 172.21.15.26:0/2932566410 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd2e0071200 0x7fd2e01490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 >> 172.21.15.26:0/2932566410 conn(0x7fd2e00cf700 msgr2=0x7fd2e01571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:03.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 shutdown_connections 2024-01-15T18:08:03.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.230+0000 7fd2e6800640 1 -- 172.21.15.26:0/2932566410 wait complete. 2024-01-15T18:08:03.254 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:08:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.477+0000 7f89c6600640 1 Processor -- start 2024-01-15T18:08:03.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.477+0000 7f89c6600640 1 -- start start 2024-01-15T18:08:03.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.478+0000 7f89c6600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89c0143ae0 0x7f89c00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89c6600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f89c00cf7d0 0x7f89c015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89c6600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c015e1e0 0x7f89c015a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89c6600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c00cfc30 con 0x7f89c0143ae0 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89c6600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c00b1ee0 con 0x7f89c00cf7d0 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89c6600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00b2780 con 0x7f89c015e1e0 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89bf400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89c0143ae0 0x7f89c00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89bf400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f89c0143ae0 0x7f89c00cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49936/0 (socket says 172.21.15.26:49936) 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89bf400640 1 -- 172.21.15.26:0/2650059634 learned_addr learned my addr 172.21.15.26:0/2650059634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89bea00640 1 --2- 172.21.15.26:0/2650059634 >> v2:172.21.15.124:3300/0 conn(0x7f89c00cf7d0 0x7f89c015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89bfe00640 1 --2- 172.21.15.26:0/2650059634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c015e1e0 0x7f89c015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.479+0000 7f89bf400640 1 -- 172.21.15.26:0/2650059634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c015e1e0 msgr2=0x7f89c015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.480+0000 7f89bf400640 1 --2- 172.21.15.26:0/2650059634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c015e1e0 0x7f89c015a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.480+0000 7f89bf400640 1 -- 172.21.15.26:0/2650059634 >> v2:172.21.15.124:3300/0 conn(0x7f89c00cf7d0 msgr2=0x7f89c015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.480+0000 7f89bf400640 1 --2- 172.21.15.26:0/2650059634 >> v2:172.21.15.124:3300/0 conn(0x7f89c00cf7d0 0x7f89c015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.480+0000 7f89bf400640 1 -- 172.21.15.26:0/2650059634 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c015e990 con 0x7f89c0143ae0 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.481+0000 7f89bf400640 1 --2- 172.21.15.26:0/2650059634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c0143ae0 0x7f89c00cf290 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f89c00cb830 tx=0x7f89b002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a157ec4ca17396b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.481+0000 7f89be000640 1 -- 172.21.15.26:0/2650059634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b002f8e0 con 0x7f89c0143ae0 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.481+0000 7f89be000640 1 -- 172.21.15.26:0/2650059634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b002fa40 con 0x7f89c0143ae0 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.481+0000 7f89be000640 1 -- 172.21.15.26:0/2650059634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b002fdd0 con 0x7f89c0143ae0 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.482+0000 7f89c6600640 1 -- 172.21.15.26:0/2650059634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c0143ae0 msgr2=0x7f89c00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.482+0000 7f89c6600640 1 --2- 172.21.15.26:0/2650059634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c0143ae0 0x7f89c00cf290 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f89c00cb830 tx=0x7f89b002db20 comp rx=0 tx=0).stop 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.482+0000 7f89c6600640 1 -- 172.21.15.26:0/2650059634 shutdown_connections 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.482+0000 7f89c6600640 1 --2- 172.21.15.26:0/2650059634 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c015e1e0 0x7f89c015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.482+0000 7f89c6600640 1 --2- 172.21.15.26:0/2650059634 >> v2:172.21.15.124:3300/0 conn(0x7f89c00cf7d0 0x7f89c015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.482+0000 7f89c6600640 1 --2- 172.21.15.26:0/2650059634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c0143ae0 0x7f89c00cf290 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.482+0000 7f89c6600640 1 -- 172.21.15.26:0/2650059634 >> 172.21.15.26:0/2650059634 conn(0x7f89c006a800 msgr2=0x7f89c006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:03.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.482+0000 7f89c6600640 1 -- 172.21.15.26:0/2650059634 shutdown_connections 2024-01-15T18:08:03.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.482+0000 7f89c6600640 1 -- 172.21.15.26:0/2650059634 wait complete. 2024-01-15T18:08:03.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.484+0000 7f89c6600640 1 Processor -- start 2024-01-15T18:08:03.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.484+0000 7f89c6600640 1 -- start start 2024-01-15T18:08:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.485+0000 7f89c6600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c00cf7d0 0x7f89c0159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.485+0000 7f89c6600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c0143ae0 0x7f89c00c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.485+0000 7f89c6600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89c015e1e0 0x7f89c00c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.485+0000 7f89c6600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00ae020 con 0x7f89c00cf7d0 2024-01-15T18:08:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.485+0000 7f89c6600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00ad410 con 0x7f89c015e1e0 2024-01-15T18:08:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.485+0000 7f89c6600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c00ad6f0 con 0x7f89c0143ae0 2024-01-15T18:08:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.485+0000 7f89bf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c00cf7d0 0x7f89c0159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.485+0000 7f89bf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c00cf7d0 0x7f89c0159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49938/0 (socket says 172.21.15.26:49938) 2024-01-15T18:08:03.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.485+0000 7f89bf400640 1 -- 172.21.15.26:0/1159167125 learned_addr learned my addr 172.21.15.26:0/1159167125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89bea00640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c0143ae0 0x7f89c00c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89bfe00640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89c015e1e0 0x7f89c00c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89bf400640 1 -- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c0143ae0 msgr2=0x7f89c00c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89bf400640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c0143ae0 0x7f89c00c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89bf400640 1 -- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89c015e1e0 msgr2=0x7f89c00c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89bf400640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89c015e1e0 0x7f89c00c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89bf400640 1 -- 172.21.15.26:0/1159167125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b00094d0 con 0x7f89c00cf7d0 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89bf400640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c00cf7d0 0x7f89c0159890 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f89b00097c0 tx=0x7f89b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89b7400640 1 -- 172.21.15.26:0/1159167125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b0037c60 con 0x7f89c00cf7d0 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89b7400640 1 -- 172.21.15.26:0/1159167125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b0037dc0 con 0x7f89c00cf7d0 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89b7400640 1 -- 172.21.15.26:0/1159167125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b00426b0 con 0x7f89c00cf7d0 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c00ad9d0 con 0x7f89c00cf7d0 2024-01-15T18:08:03.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.486+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c00c6c40 con 0x7f89c00cf7d0 2024-01-15T18:08:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.488+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c00675d0 con 0x7f89c00cf7d0 2024-01-15T18:08:03.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.489+0000 7f89b7400640 1 -- 172.21.15.26:0/1159167125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f89b0042810 con 0x7f89c00cf7d0 2024-01-15T18:08:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.490+0000 7f89b7400640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899806bfb0 0x7f899806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.490+0000 7f89b7400640 1 -- 172.21.15.26:0/1159167125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f89b0033070 con 0x7f89c00cf7d0 2024-01-15T18:08:03.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.490+0000 7f89bea00640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899806bfb0 0x7f899806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.491+0000 7f89bea00640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899806bfb0 0x7f899806e470 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f89c0142dd0 tx=0x7f89a8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:03.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.494+0000 7f89b7400640 1 -- 172.21.15.26:0/1159167125 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89b0078030 con 0x7f89c00cf7d0 2024-01-15T18:08:03.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.688+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f89c015c120 con 0x7f89c00cf7d0 2024-01-15T18:08:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.695+0000 7f89b7400640 1 -- 172.21.15.26:0/1159167125 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f89b007b650 con 0x7f89c00cf7d0 2024-01-15T18:08:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899806bfb0 msgr2=0x7f899806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899806bfb0 0x7f899806e470 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f89c0142dd0 tx=0x7f89a8008040 comp rx=0 tx=0).stop 2024-01-15T18:08:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c00cf7d0 msgr2=0x7f89c0159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c00cf7d0 0x7f89c0159890 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f89b00097c0 tx=0x7f89b00046c0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 shutdown_connections 2024-01-15T18:08:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89c0143ae0 0x7f89c00c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f899806bfb0 0x7f899806e470 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f89c015e1e0 0x7f89c00c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 --2- 172.21.15.26:0/1159167125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f89c00cf7d0 0x7f89c0159890 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 >> 172.21.15.26:0/1159167125 conn(0x7f89c006a800 msgr2=0x7f89c00b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 shutdown_connections 2024-01-15T18:08:03.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.698+0000 7f89c6600640 1 -- 172.21.15.26:0/1159167125 wait complete. 2024-01-15T18:08:03.722 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:08:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.942+0000 7f8b38600640 1 Processor -- start 2024-01-15T18:08:03.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.942+0000 7f8b38600640 1 -- start start 2024-01-15T18:08:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.944+0000 7f8b38600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b3015e3b0 0x7f8b30164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.944+0000 7f8b38600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b300b1ee0 con 0x7f8b30143ae0 2024-01-15T18:08:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.944+0000 7f8b38600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b300b2780 con 0x7f8b300cf9a0 2024-01-15T18:08:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.944+0000 7f8b38600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3015e990 con 0x7f8b3015e3b0 2024-01-15T18:08:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.944+0000 7f8b36600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8b3015e3b0 0x7f8b30164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.944+0000 7f8b35c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8b30143ae0 0x7f8b300cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36390/0 (socket says 172.21.15.26:36390) 2024-01-15T18:08:03.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.944+0000 7f8b35c00640 1 -- 172.21.15.26:0/1064731111 learned_addr learned my addr 172.21.15.26:0/1064731111 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.945+0000 7f8b2fe00640 1 -- 172.21.15.26:0/1064731111 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938529343 0 0) 0x7f8b300b1ee0 con 0x7f8b30143ae0 2024-01-15T18:08:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.945+0000 7f8b2fe00640 1 -- 172.21.15.26:0/1064731111 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b10005730 con 0x7f8b30143ae0 2024-01-15T18:08:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.945+0000 7f8b36600640 1 -- 172.21.15.26:0/1064731111 >> v1:172.21.15.124:6789/0 conn(0x7f8b300cf9a0 legacy=0x7f8b3015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.945+0000 7f8b36600640 1 -- 172.21.15.26:0/1064731111 >> v1:172.21.15.26:6789/0 conn(0x7f8b30143ae0 legacy=0x7f8b300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.945+0000 7f8b36600640 1 -- 172.21.15.26:0/1064731111 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b300b2130 con 0x7f8b3015e3b0 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.946+0000 7f8b36600640 1 --2- 172.21.15.26:0/1064731111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b3015e3b0 0x7f8b30164de0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009d90 tx=0x7f8b2802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f2111b0abb8615 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.946+0000 7f8b2fe00640 1 -- 172.21.15.26:0/1064731111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2802ccf0 con 0x7f8b3015e3b0 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.946+0000 7f8b2fe00640 1 -- 172.21.15.26:0/1064731111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2802ce50 con 0x7f8b3015e3b0 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.946+0000 7f8b2fe00640 1 -- 172.21.15.26:0/1064731111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28034640 con 0x7f8b3015e3b0 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.947+0000 7f8b38600640 1 -- 172.21.15.26:0/1064731111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b3015e3b0 msgr2=0x7f8b30164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.947+0000 7f8b38600640 1 --2- 172.21.15.26:0/1064731111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b3015e3b0 0x7f8b30164de0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009d90 tx=0x7f8b2802bf40 comp rx=0 tx=0).stop 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.947+0000 7f8b38600640 1 -- 172.21.15.26:0/1064731111 shutdown_connections 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.947+0000 7f8b38600640 1 --2- 172.21.15.26:0/1064731111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b3015e3b0 0x7f8b30164de0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.947+0000 7f8b38600640 1 -- 172.21.15.26:0/1064731111 >> 172.21.15.26:0/1064731111 conn(0x7f8b3006a800 msgr2=0x7f8b3006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.947+0000 7f8b38600640 1 -- 172.21.15.26:0/1064731111 shutdown_connections 2024-01-15T18:08:03.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.947+0000 7f8b38600640 1 -- 172.21.15.26:0/1064731111 wait complete. 2024-01-15T18:08:03.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.949+0000 7f8b38600640 1 Processor -- start 2024-01-15T18:08:03.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.949+0000 7f8b38600640 1 -- start start 2024-01-15T18:08:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.950+0000 7f8b38600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b300cf9a0 0x7f8b30131600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.950+0000 7f8b38600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b30143ae0 0x7f8b30159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.950+0000 7f8b38600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b3015e3b0 0x7f8b3006c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.950+0000 7f8b38600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b30131f00 con 0x7f8b30143ae0 2024-01-15T18:08:03.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.950+0000 7f8b38600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b301321e0 con 0x7f8b300cf9a0 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.950+0000 7f8b38600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b301324c0 con 0x7f8b3015e3b0 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.950+0000 7f8b35200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b30143ae0 0x7f8b30159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.950+0000 7f8b35200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b30143ae0 0x7f8b30159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49958/0 (socket says 172.21.15.26:49958) 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.950+0000 7f8b35200640 1 -- 172.21.15.26:0/4004818727 learned_addr learned my addr 172.21.15.26:0/4004818727 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b35c00640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b300cf9a0 0x7f8b30131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b36600640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b3015e3b0 0x7f8b3006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b35200640 1 -- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b3015e3b0 msgr2=0x7f8b3006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b35200640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b3015e3b0 0x7f8b3006c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b35200640 1 -- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b300cf9a0 msgr2=0x7f8b30131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b35200640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b300cf9a0 0x7f8b30131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b35200640 1 -- 172.21.15.26:0/4004818727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b28009a70 con 0x7f8b30143ae0 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b35c00640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b300cf9a0 0x7f8b30131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b35200640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b30143ae0 0x7f8b30159890 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800c400 tx=0x7f8b1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:03.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.951+0000 7f8b2d600640 1 -- 172.21.15.26:0/4004818727 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b180052d0 con 0x7f8b30143ae0 2024-01-15T18:08:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.952+0000 7f8b2d600640 1 -- 172.21.15.26:0/4004818727 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b18005450 con 0x7f8b30143ae0 2024-01-15T18:08:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.952+0000 7f8b2d600640 1 -- 172.21.15.26:0/4004818727 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b18007260 con 0x7f8b30143ae0 2024-01-15T18:08:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.952+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3012f610 con 0x7f8b30143ae0 2024-01-15T18:08:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.952+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b30131b40 con 0x7f8b30143ae0 2024-01-15T18:08:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.953+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3012fb30 con 0x7f8b30143ae0 2024-01-15T18:08:03.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.954+0000 7f8b2d600640 1 -- 172.21.15.26:0/4004818727 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f8b180073c0 con 0x7f8b30143ae0 2024-01-15T18:08:03.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.955+0000 7f8b2d600640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0806bfb0 0x7f8b0806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:03.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.955+0000 7f8b2d600640 1 -- 172.21.15.26:0/4004818727 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f8b18087a10 con 0x7f8b30143ae0 2024-01-15T18:08:03.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.956+0000 7f8b35c00640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0806bfb0 0x7f8b0806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:03.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.957+0000 7f8b35c00640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0806bfb0 0x7f8b0806e470 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8b20007fd0 tx=0x7f8b20009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:03.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:03.959+0000 7f8b2d600640 1 -- 172.21.15.26:0/4004818727 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b3012fb30 con 0x7f8b30143ae0 2024-01-15T18:08:04.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.155+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f8b3012fb30 con 0x7f8b30143ae0 2024-01-15T18:08:04.165 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.160+0000 7f8b2d600640 1 -- 172.21.15.26:0/4004818727 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v29) v1 ==== 120+0+0 (secure 0 0 0) 0x7f8b3012fb30 con 0x7f8b30143ae0 2024-01-15T18:08:04.168 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0806bfb0 msgr2=0x7f8b0806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0806bfb0 0x7f8b0806e470 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f8b20007fd0 tx=0x7f8b20009040 comp rx=0 tx=0).stop 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b30143ae0 msgr2=0x7f8b30159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b30143ae0 0x7f8b30159890 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800c400 tx=0x7f8b1800c8c0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 shutdown_connections 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8b3015e3b0 0x7f8b3006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8b0806bfb0 0x7f8b0806e470 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8b30143ae0 0x7f8b30159890 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 --2- 172.21.15.26:0/4004818727 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8b300cf9a0 0x7f8b30131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 >> 172.21.15.26:0/4004818727 conn(0x7f8b3006a800 msgr2=0x7f8b300ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 shutdown_connections 2024-01-15T18:08:04.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.164+0000 7f8b38600640 1 -- 172.21.15.26:0/4004818727 wait complete. 2024-01-15T18:08:04.189 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added' 2024-01-15T18:08:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.409+0000 7fe2bfc00640 1 Processor -- start 2024-01-15T18:08:04.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.409+0000 7fe2bfc00640 1 -- start start 2024-01-15T18:08:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.411+0000 7fe2bfc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe2b813df80 0x7fe2b815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.411+0000 7fe2bfc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b806c470 con 0x7fe2b813e360 2024-01-15T18:08:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.411+0000 7fe2bfc00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b806c750 con 0x7fe2b80cf9a0 2024-01-15T18:08:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.411+0000 7fe2bfc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b806ca30 con 0x7fe2b813df80 2024-01-15T18:08:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.411+0000 7fe2b7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe2b813e360 0x7fe2b80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36404/0 (socket says 172.21.15.26:36404) 2024-01-15T18:08:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.411+0000 7fe2b7e00640 1 -- 172.21.15.26:0/3150555731 learned_addr learned my addr 172.21.15.26:0/3150555731 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:04.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.411+0000 7fe2bd200640 1 --2- 172.21.15.26:0/3150555731 >> v2:172.21.15.26:3300/0 conn(0x7fe2b813df80 0x7fe2b815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.412+0000 7fe2b7400640 1 -- 172.21.15.26:0/3150555731 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654116176 0 0) 0x7fe2b806c470 con 0x7fe2b813e360 2024-01-15T18:08:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.412+0000 7fe2b7400640 1 -- 172.21.15.26:0/3150555731 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe298003520 con 0x7fe2b813e360 2024-01-15T18:08:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.412+0000 7fe2bd200640 1 -- 172.21.15.26:0/3150555731 >> v1:172.21.15.124:6789/0 conn(0x7fe2b80cf9a0 legacy=0x7fe2b80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.412+0000 7fe2bd200640 1 -- 172.21.15.26:0/3150555731 >> v1:172.21.15.26:6789/0 conn(0x7fe2b813e360 legacy=0x7fe2b80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.412+0000 7fe2bd200640 1 -- 172.21.15.26:0/3150555731 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b806cd10 con 0x7fe2b813df80 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.413+0000 7fe2bd200640 1 --2- 172.21.15.26:0/3150555731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813df80 0x7fe2b815bee0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80097a0 tx=0x7fe2a802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191f71855a2e7c83 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2b7400640 1 -- 172.21.15.26:0/3150555731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8030610 con 0x7fe2b813df80 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2b7400640 1 -- 172.21.15.26:0/3150555731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8030770 con 0x7fe2b813df80 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2b7400640 1 -- 172.21.15.26:0/3150555731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8030b00 con 0x7fe2b813df80 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3150555731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813df80 msgr2=0x7fe2b815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2bfc00640 1 --2- 172.21.15.26:0/3150555731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813df80 0x7fe2b815bee0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80097a0 tx=0x7fe2a802b960 comp rx=0 tx=0).stop 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3150555731 shutdown_connections 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2bfc00640 1 --2- 172.21.15.26:0/3150555731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813df80 0x7fe2b815bee0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3150555731 >> 172.21.15.26:0/3150555731 conn(0x7fe2b806a800 msgr2=0x7fe2b806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3150555731 shutdown_connections 2024-01-15T18:08:04.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.414+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3150555731 wait complete. 2024-01-15T18:08:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.416+0000 7fe2bfc00640 1 Processor -- start 2024-01-15T18:08:04.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.416+0000 7fe2bfc00640 1 -- start start 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.417+0000 7fe2bfc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b80cf9a0 0x7fe2b8152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.417+0000 7fe2bfc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2b813df80 0x7fe2b81e5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.417+0000 7fe2bfc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813e360 0x7fe2b813d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.417+0000 7fe2bfc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b81e5a80 con 0x7fe2b813e360 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.417+0000 7fe2bfc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8152f60 con 0x7fe2b80cf9a0 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.417+0000 7fe2bfc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8153240 con 0x7fe2b813df80 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.417+0000 7fe2bdc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813e360 0x7fe2b813d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2bdc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813e360 0x7fe2b813d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49974/0 (socket says 172.21.15.26:49974) 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2bdc00640 1 -- 172.21.15.26:0/3263414161 learned_addr learned my addr 172.21.15.26:0/3263414161 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:04.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2bd200640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b80cf9a0 0x7fe2b8152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2b7e00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2b813df80 0x7fe2b81e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2bdc00640 1 -- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2b813df80 msgr2=0x7fe2b81e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2bdc00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2b813df80 0x7fe2b81e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2bdc00640 1 -- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b80cf9a0 msgr2=0x7fe2b8152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2bdc00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b80cf9a0 0x7fe2b8152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2bdc00640 1 -- 172.21.15.26:0/3263414161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8009400 con 0x7fe2b813e360 2024-01-15T18:08:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2b7e00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2b813df80 0x7fe2b81e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:04.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.418+0000 7fe2bdc00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813e360 0x7fe2b813d150 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c680 tx=0x7fe2b000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.419+0000 7fe2b4c00640 1 -- 172.21.15.26:0/3263414161 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0003b70 con 0x7fe2b813e360 2024-01-15T18:08:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.419+0000 7fe2b4c00640 1 -- 172.21.15.26:0/3263414161 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b0003cd0 con 0x7fe2b813e360 2024-01-15T18:08:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.419+0000 7fe2b4c00640 1 -- 172.21.15.26:0/3263414161 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0005310 con 0x7fe2b813e360 2024-01-15T18:08:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.419+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b8153520 con 0x7fe2b813e360 2024-01-15T18:08:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.419+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b8153ae0 con 0x7fe2b813e360 2024-01-15T18:08:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.420+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b80675d0 con 0x7fe2b813e360 2024-01-15T18:08:04.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.421+0000 7fe2b4c00640 1 -- 172.21.15.26:0/3263414161 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fe2b0005470 con 0x7fe2b813e360 2024-01-15T18:08:04.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.423+0000 7fe2b4c00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe29006bfe0 0x7fe29006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:04.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.423+0000 7fe2b4c00640 1 -- 172.21.15.26:0/3263414161 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fe2b0088350 con 0x7fe2b813e360 2024-01-15T18:08:04.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.423+0000 7fe2bd200640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe29006bfe0 0x7fe29006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:04.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.424+0000 7fe2bd200640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe29006bfe0 0x7fe29006e4a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8036000 tx=0x7fe2a8008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:04.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.427+0000 7fe2b4c00640 1 -- 172.21.15.26:0/3263414161 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2b0091080 con 0x7fe2b813e360 2024-01-15T18:08:04.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.693+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added"]} v 0) v1 -- 0x7fe2b8155ae0 con 0x7fe2b813e360 2024-01-15T18:08:04.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.738+0000 7fe2b4c00640 1 -- 172.21.15.26:0/3263414161 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_progress_drops_when_new_jobs_are_added"]}]=0 v588) v1 ==== 166+0+0 (secure 0 0 0) 0x7fe2b0054c10 con 0x7fe2b813e360 2024-01-15T18:08:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe29006bfe0 msgr2=0x7fe29006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe29006bfe0 0x7fe29006e4a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8036000 tx=0x7fe2a8008030 comp rx=0 tx=0).stop 2024-01-15T18:08:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813e360 msgr2=0x7fe2b813d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813e360 0x7fe2b813d150 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c680 tx=0x7fe2b000cb40 comp rx=0 tx=0).stop 2024-01-15T18:08:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 shutdown_connections 2024-01-15T18:08:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe2b813df80 0x7fe2b81e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe29006bfe0 0x7fe29006e4a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe2b813e360 0x7fe2b813d150 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 --2- 172.21.15.26:0/3263414161 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe2b80cf9a0 0x7fe2b8152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 >> 172.21.15.26:0/3263414161 conn(0x7fe2b806a800 msgr2=0x7fe2b80afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 shutdown_connections 2024-01-15T18:08:04.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.741+0000 7fe2bfc00640 1 -- 172.21.15.26:0/3263414161 wait complete. 2024-01-15T18:08:04.767 INFO:tasks.cephfs_test_runner:X = number of clones issued. ... ok 2024-01-15T18:08:04.767 INFO:tasks.cephfs_test_runner:Starting test: test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgress) Test that the command "ceph fs clone status" prints progress stats 2024-01-15T18:08:04.767 INFO:tasks.cephfs_test_runner:test_stats_are_printed (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:08:04.768 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:08:04.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.985+0000 7f1b51800640 1 Processor -- start 2024-01-15T18:08:04.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.985+0000 7f1b51800640 1 -- start start 2024-01-15T18:08:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.986+0000 7f1b51800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0b3ae0 0x7f1b4c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.986+0000 7f1b51800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b4c0cf290 0x7f1b4c0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.986+0000 7f1b51800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4c070c40 con 0x7f1b4c0ae5a0 2024-01-15T18:08:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.986+0000 7f1b51800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c071450 con 0x7f1b4c0cf290 2024-01-15T18:08:04.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.986+0000 7f1b51800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0cfc60 con 0x7f1b4c0b3ae0 2024-01-15T18:08:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.987+0000 7f1b4b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1b4c0cf290 0x7f1b4c0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.987+0000 7f1b4aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1b4c0ae5a0 0x7f1b4c06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36416/0 (socket says 172.21.15.26:36416) 2024-01-15T18:08:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.987+0000 7f1b4aa00640 1 -- 172.21.15.26:0/3688070304 learned_addr learned my addr 172.21.15.26:0/3688070304 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.987+0000 7f1b4a000640 1 --2- 172.21.15.26:0/3688070304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0b3ae0 0x7f1b4c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.987+0000 7f1b4b400640 1 -- 172.21.15.26:0/3688070304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0b3ae0 msgr2=0x7f1b4c15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.987+0000 7f1b4b400640 1 --2- 172.21.15.26:0/3688070304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0b3ae0 0x7f1b4c15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.987+0000 7f1b4b400640 1 -- 172.21.15.26:0/3688070304 >> v1:172.21.15.26:6789/0 conn(0x7f1b4c0ae5a0 legacy=0x7f1b4c06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.987+0000 7f1b4b400640 1 -- 172.21.15.26:0/3688070304 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c06c470 con 0x7f1b4c0cf290 2024-01-15T18:08:04.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.989+0000 7f1b4b400640 1 --2- 172.21.15.26:0/3688070304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0cf290 0x7f1b4c0cf720 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009d90 tx=0x7f1b4402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0958f3ef38c1c11 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:04.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.989+0000 7f1b49600640 1 -- 172.21.15.26:0/3688070304 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4402cce0 con 0x7f1b4c0cf290 2024-01-15T18:08:04.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.989+0000 7f1b49600640 1 -- 172.21.15.26:0/3688070304 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4402ce40 con 0x7f1b4c0cf290 2024-01-15T18:08:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.989+0000 7f1b49600640 1 -- 172.21.15.26:0/3688070304 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44034660 con 0x7f1b4c0cf290 2024-01-15T18:08:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.990+0000 7f1b51800640 1 -- 172.21.15.26:0/3688070304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0cf290 msgr2=0x7f1b4c0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.990+0000 7f1b51800640 1 --2- 172.21.15.26:0/3688070304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0cf290 0x7f1b4c0cf720 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009d90 tx=0x7f1b4402bf30 comp rx=0 tx=0).stop 2024-01-15T18:08:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.990+0000 7f1b51800640 1 -- 172.21.15.26:0/3688070304 shutdown_connections 2024-01-15T18:08:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.990+0000 7f1b51800640 1 --2- 172.21.15.26:0/3688070304 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0cf290 0x7f1b4c0cf720 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.990+0000 7f1b51800640 1 --2- 172.21.15.26:0/3688070304 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0b3ae0 0x7f1b4c15e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.990+0000 7f1b51800640 1 -- 172.21.15.26:0/3688070304 >> 172.21.15.26:0/3688070304 conn(0x7f1b4c06a800 msgr2=0x7f1b4c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.990+0000 7f1b51800640 1 -- 172.21.15.26:0/3688070304 shutdown_connections 2024-01-15T18:08:04.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.990+0000 7f1b51800640 1 -- 172.21.15.26:0/3688070304 wait complete. 2024-01-15T18:08:04.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.992+0000 7f1b51800640 1 Processor -- start 2024-01-15T18:08:04.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.992+0000 7f1b51800640 1 -- start start 2024-01-15T18:08:05.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b51800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0ae5a0 0x7f1b4c149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b51800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b4c0b3ae0 0x7f1b4c15c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b51800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0cf290 0x7f1b4c15cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b51800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c151d80 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b51800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c152060 con 0x7f1b4c0b3ae0 2024-01-15T18:08:05.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b51800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c152340 con 0x7f1b4c0cf290 2024-01-15T18:08:05.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b4aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0ae5a0 0x7f1b4c149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b4aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0ae5a0 0x7f1b4c149560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49992/0 (socket says 172.21.15.26:49992) 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b4aa00640 1 -- 172.21.15.26:0/520053944 learned_addr learned my addr 172.21.15.26:0/520053944 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.994+0000 7f1b4a000640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b4c0b3ae0 0x7f1b4c15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b4b400640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0cf290 0x7f1b4c15cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b4aa00640 1 -- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0cf290 msgr2=0x7f1b4c15cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b4aa00640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0cf290 0x7f1b4c15cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b4aa00640 1 -- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b4c0b3ae0 msgr2=0x7f1b4c15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b4aa00640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b4c0b3ae0 0x7f1b4c15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b4aa00640 1 -- 172.21.15.26:0/520053944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009a70 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b4aa00640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0ae5a0 0x7f1b4c149560 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00c4a0 tx=0x7f1b3c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b42a00640 1 -- 172.21.15.26:0/520053944 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c005ce0 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b42a00640 1 -- 172.21.15.26:0/520053944 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c005e40 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b42a00640 1 -- 172.21.15.26:0/520053944 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c007270 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.995+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c152620 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.996+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c1545d0 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.997+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c0675d0 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.998+0000 7f1b42a00640 1 -- 172.21.15.26:0/520053944 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f1b3c007490 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.999+0000 7f1b42a00640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1b2406bfb0 0x7f1b2406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.999+0000 7f1b42a00640 1 -- 172.21.15.26:0/520053944 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f1b3c087ad0 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:04.999+0000 7f1b4a000640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1b2406bfb0 0x7f1b2406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.001+0000 7f1b4a000640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1b2406bfb0 0x7f1b2406e470 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c15a4b0 tx=0x7f1b34008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:05.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.004+0000 7f1b42a00640 1 -- 172.21.15.26:0/520053944 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b3c054440 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.207+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b4c0cb620 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.209+0000 7f1b42a00640 1 -- 172.21.15.26:0/520053944 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+13050 (secure 0 0 0) 0x7f1b3c017090 con 0x7f1b4c0ae5a0 2024-01-15T18:08:05.214 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:05.214 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":79,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:07:58.712271+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.26:6835/1813593645":"2024-01-16T18:07:55.696353+0000","172.21.15.26:6833/1813593645":"2024-01-16T18:07:55.696353+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1b2406bfb0 msgr2=0x7f1b2406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1b2406bfb0 0x7f1b2406e470 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c15a4b0 tx=0x7f1b34008040 comp rx=0 tx=0).stop 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0ae5a0 msgr2=0x7f1b4c149560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0ae5a0 0x7f1b4c149560 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00c4a0 tx=0x7f1b3c00c960 comp rx=0 tx=0).stop 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 shutdown_connections 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1b4c0cf290 0x7f1b4c15cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1b2406bfb0 0x7f1b2406e470 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1b4c0b3ae0 0x7f1b4c15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 --2- 172.21.15.26:0/520053944 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1b4c0ae5a0 0x7f1b4c149560 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 >> 172.21.15.26:0/520053944 conn(0x7f1b4c06a800 msgr2=0x7f1b4c160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 shutdown_connections 2024-01-15T18:08:05.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.212+0000 7f1b51800640 1 -- 172.21.15.26:0/520053944 wait complete. 2024-01-15T18:08:05.237 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:08:05.238 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:08:05.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.449+0000 7fe906e00640 1 Processor -- start 2024-01-15T18:08:05.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.449+0000 7fe906e00640 1 -- start start 2024-01-15T18:08:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.450+0000 7fe906e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe9000ce8f0 0x7fe9000ced30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.450+0000 7fe906e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9000b2a00 0x7fe9000b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.450+0000 7fe906e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9000d0de0 con 0x7fe900160f80 2024-01-15T18:08:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.450+0000 7fe906e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000d10c0 con 0x7fe9000ce8f0 2024-01-15T18:08:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.450+0000 7fe906e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000d13a0 con 0x7fe9000b2a00 2024-01-15T18:08:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.451+0000 7fe8ff400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe9000ce8f0 0x7fe9000ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.451+0000 7fe8ff400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe9000ce8f0 0x7fe9000ced30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50002/0 (socket says 172.21.15.26:50002) 2024-01-15T18:08:05.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.451+0000 7fe8ff400640 1 -- 172.21.15.26:0/4292244754 learned_addr learned my addr 172.21.15.26:0/4292244754 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.451+0000 7fe904e00640 1 --2- 172.21.15.26:0/4292244754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9000b2a00 0x7fe9000b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.451+0000 7fe8ff400640 1 -- 172.21.15.26:0/4292244754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9000b2a00 msgr2=0x7fe9000b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.452+0000 7fe8ff400640 1 --2- 172.21.15.26:0/4292244754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9000b2a00 0x7fe9000b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.452+0000 7fe8ff400640 1 -- 172.21.15.26:0/4292244754 >> v1:172.21.15.124:6789/0 conn(0x7fe900160f80 legacy=0x7fe900161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.452+0000 7fe8ff400640 1 -- 172.21.15.26:0/4292244754 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000d1680 con 0x7fe9000ce8f0 2024-01-15T18:08:05.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.452+0000 7fe904e00640 1 --2- 172.21.15.26:0/4292244754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9000b2a00 0x7fe9000b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe8ff400640 1 --2- 172.21.15.26:0/4292244754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000ce8f0 0x7fe9000ced30 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009870 tx=0x7fe8e802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aae7b79bfa0e55a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe8fea00640 1 -- 172.21.15.26:0/4292244754 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8030840 con 0x7fe9000ce8f0 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe8fea00640 1 -- 172.21.15.26:0/4292244754 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e80309a0 con 0x7fe9000ce8f0 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe8fea00640 1 -- 172.21.15.26:0/4292244754 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8030d30 con 0x7fe9000ce8f0 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe906e00640 1 -- 172.21.15.26:0/4292244754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000ce8f0 msgr2=0x7fe9000ced30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe906e00640 1 --2- 172.21.15.26:0/4292244754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000ce8f0 0x7fe9000ced30 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009870 tx=0x7fe8e802b980 comp rx=0 tx=0).stop 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe906e00640 1 -- 172.21.15.26:0/4292244754 shutdown_connections 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe906e00640 1 --2- 172.21.15.26:0/4292244754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe9000b2a00 0x7fe9000b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe906e00640 1 --2- 172.21.15.26:0/4292244754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000ce8f0 0x7fe9000ced30 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe906e00640 1 -- 172.21.15.26:0/4292244754 >> 172.21.15.26:0/4292244754 conn(0x7fe90009c550 msgr2=0x7fe9000d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe906e00640 1 -- 172.21.15.26:0/4292244754 shutdown_connections 2024-01-15T18:08:05.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.454+0000 7fe906e00640 1 -- 172.21.15.26:0/4292244754 wait complete. 2024-01-15T18:08:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.456+0000 7fe906e00640 1 Processor -- start 2024-01-15T18:08:05.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.456+0000 7fe906e00640 1 -- start start 2024-01-15T18:08:05.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.457+0000 7fe906e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000b2a00 0x7fe9000b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe906e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9000ce8f0 0x7fe90015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe906e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe900160f80 0x7fe90015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe906e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9000b91a0 con 0x7fe9000b2a00 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe906e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90015ec60 con 0x7fe9000ce8f0 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe906e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90015ef40 con 0x7fe900160f80 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ffe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000b2a00 0x7fe9000b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ffe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000b2a00 0x7fe9000b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50016/0 (socket says 172.21.15.26:50016) 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ffe00640 1 -- 172.21.15.26:0/2188919710 learned_addr learned my addr 172.21.15.26:0/2188919710 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ff400640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9000ce8f0 0x7fe90015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ffe00640 1 -- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe900160f80 msgr2=0x7fe90015fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ffe00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe900160f80 0x7fe90015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ffe00640 1 -- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9000ce8f0 msgr2=0x7fe90015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ffe00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9000ce8f0 0x7fe90015b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ffe00640 1 -- 172.21.15.26:0/2188919710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e80094d0 con 0x7fe9000b2a00 2024-01-15T18:08:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ff400640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9000ce8f0 0x7fe90015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8ffe00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000b2a00 0x7fe9000b8c60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000c400 tx=0x7fe8f000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8f7e00640 1 -- 172.21.15.26:0/2188919710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00052d0 con 0x7fe9000b2a00 2024-01-15T18:08:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8f7e00640 1 -- 172.21.15.26:0/2188919710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f0005430 con 0x7fe9000b2a00 2024-01-15T18:08:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe8f7e00640 1 -- 172.21.15.26:0/2188919710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0003830 con 0x7fe9000b2a00 2024-01-15T18:08:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90015f220 con 0x7fe9000b2a00 2024-01-15T18:08:05.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.459+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90015f7e0 con 0x7fe9000b2a00 2024-01-15T18:08:05.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.460+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9000675d0 con 0x7fe9000b2a00 2024-01-15T18:08:05.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.462+0000 7fe8f7e00640 1 -- 172.21.15.26:0/2188919710 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fe8f0003990 con 0x7fe9000b2a00 2024-01-15T18:08:05.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.463+0000 7fe8f7e00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe8d806bfb0 0x7fe8d806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.463+0000 7fe8f7e00640 1 -- 172.21.15.26:0/2188919710 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fe8f0087b50 con 0x7fe9000b2a00 2024-01-15T18:08:05.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.464+0000 7fe8ff400640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe8d806bfb0 0x7fe8d806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.466+0000 7fe8ff400640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe8d806bfb0 0x7fe8d806e470 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8036000 tx=0x7fe8e803b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:05.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.468+0000 7fe8f7e00640 1 -- 172.21.15.26:0/2188919710 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8f0054450 con 0x7fe9000b2a00 2024-01-15T18:08:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.662+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9000d3920 con 0x7fe9000b2a00 2024-01-15T18:08:05.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.663+0000 7fe8f7e00640 1 -- 172.21.15.26:0/2188919710 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8f0017020 con 0x7fe9000b2a00 2024-01-15T18:08:05.668 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:08:05.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.665+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe8d806bfb0 msgr2=0x7fe8d806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.665+0000 7fe906e00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe8d806bfb0 0x7fe8d806e470 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8036000 tx=0x7fe8e803b040 comp rx=0 tx=0).stop 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.665+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000b2a00 msgr2=0x7fe9000b8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.665+0000 7fe906e00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000b2a00 0x7fe9000b8c60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000c400 tx=0x7fe8f000c8c0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.666+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 shutdown_connections 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.666+0000 7fe906e00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe900160f80 0x7fe90015fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.666+0000 7fe906e00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe8d806bfb0 0x7fe8d806e470 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.666+0000 7fe906e00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe9000ce8f0 0x7fe90015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.666+0000 7fe906e00640 1 --2- 172.21.15.26:0/2188919710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe9000b2a00 0x7fe9000b8c60 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.666+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 >> 172.21.15.26:0/2188919710 conn(0x7fe90009c550 msgr2=0x7fe900161360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.666+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 shutdown_connections 2024-01-15T18:08:05.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.666+0000 7fe906e00640 1 -- 172.21.15.26:0/2188919710 wait complete. 2024-01-15T18:08:05.690 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed' 2024-01-15T18:08:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.909+0000 7fdaf5c00640 1 Processor -- start 2024-01-15T18:08:05.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.910+0000 7fdaf5c00640 1 -- start start 2024-01-15T18:08:05.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.911+0000 7fdaf5c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdaf0162430 0x7fdaf00b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.911+0000 7fdaf5c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdaf0162810 0x7fdaf0071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.911+0000 7fdaf5c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf00702e0 con 0x7fdaf00b3650 2024-01-15T18:08:05.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.911+0000 7fdaf5c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf00705c0 con 0x7fdaf0162430 2024-01-15T18:08:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.911+0000 7fdaf5c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf00708a0 con 0x7fdaf0162810 2024-01-15T18:08:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.912+0000 7fdaefe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdaf0162430 0x7fdaf00b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.912+0000 7fdaefe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdaf0162430 0x7fdaf00b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50024/0 (socket says 172.21.15.26:50024) 2024-01-15T18:08:05.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.912+0000 7fdaefe00640 1 -- 172.21.15.26:0/1227662651 learned_addr learned my addr 172.21.15.26:0/1227662651 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:05.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.912+0000 7fdae7e00640 1 --2- 172.21.15.26:0/1227662651 >> v2:172.21.15.124:3300/0 conn(0x7fdaf0162810 0x7fdaf0071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.913+0000 7fdaefe00640 1 -- 172.21.15.26:0/1227662651 >> v2:172.21.15.124:3300/0 conn(0x7fdaf0162810 msgr2=0x7fdaf0071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:05.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.913+0000 7fdaefe00640 1 --2- 172.21.15.26:0/1227662651 >> v2:172.21.15.124:3300/0 conn(0x7fdaf0162810 0x7fdaf0071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.913+0000 7fdaefe00640 1 -- 172.21.15.26:0/1227662651 >> v1:172.21.15.124:6789/0 conn(0x7fdaf00b3650 legacy=0x7fdaf01650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.913+0000 7fdaefe00640 1 -- 172.21.15.26:0/1227662651 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf0070b80 con 0x7fdaf0162430 2024-01-15T18:08:05.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.915+0000 7fdaefe00640 1 --2- 172.21.15.26:0/1227662651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 0x7fdaf00b3110 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fdaf015c090 tx=0x7fdae002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324fbf9dda1bf494 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:05.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.916+0000 7fdaef400640 1 -- 172.21.15.26:0/1227662651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0030860 con 0x7fdaf0162430 2024-01-15T18:08:05.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.916+0000 7fdaef400640 1 -- 172.21.15.26:0/1227662651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae00309c0 con 0x7fdaf0162430 2024-01-15T18:08:05.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.916+0000 7fdaef400640 1 -- 172.21.15.26:0/1227662651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0030d50 con 0x7fdaf0162430 2024-01-15T18:08:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.917+0000 7fdaf5c00640 1 -- 172.21.15.26:0/1227662651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 msgr2=0x7fdaf00b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.917+0000 7fdaf5c00640 1 --2- 172.21.15.26:0/1227662651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 0x7fdaf00b3110 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fdaf015c090 tx=0x7fdae002b9a0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.917+0000 7fdaf5c00640 1 -- 172.21.15.26:0/1227662651 shutdown_connections 2024-01-15T18:08:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.917+0000 7fdaf5c00640 1 --2- 172.21.15.26:0/1227662651 >> v2:172.21.15.124:3300/0 conn(0x7fdaf0162810 0x7fdaf0071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.917+0000 7fdaf5c00640 1 --2- 172.21.15.26:0/1227662651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 0x7fdaf00b3110 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.917+0000 7fdaf5c00640 1 -- 172.21.15.26:0/1227662651 >> 172.21.15.26:0/1227662651 conn(0x7fdaf006a800 msgr2=0x7fdaf006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.917+0000 7fdaf5c00640 1 -- 172.21.15.26:0/1227662651 shutdown_connections 2024-01-15T18:08:05.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.917+0000 7fdaf5c00640 1 -- 172.21.15.26:0/1227662651 wait complete. 2024-01-15T18:08:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.919+0000 7fdaf5c00640 1 Processor -- start 2024-01-15T18:08:05.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.919+0000 7fdaf5c00640 1 -- start start 2024-01-15T18:08:05.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.920+0000 7fdaf5c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdaf00b3650 0x7fdaf0146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.920+0000 7fdaf5c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 0x7fdaf0164630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.920+0000 7fdaf5c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf0153e80 0x7fdaf0158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.920+0000 7fdaf5c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0165590 con 0x7fdaf0162430 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.920+0000 7fdaf5c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf01ebcc0 con 0x7fdaf00b3650 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.920+0000 7fdaf5c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf0164fd0 con 0x7fdaf0153e80 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.920+0000 7fdae7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 0x7fdaf0164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.920+0000 7fdae7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 0x7fdaf0164630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50026/0 (socket says 172.21.15.26:50026) 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.920+0000 7fdae7e00640 1 -- 172.21.15.26:0/3005640475 learned_addr learned my addr 172.21.15.26:0/3005640475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdaefe00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdaf00b3650 0x7fdaf0146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdaf5200640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf0153e80 0x7fdaf0158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdae7e00640 1 -- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf0153e80 msgr2=0x7fdaf0158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdae7e00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf0153e80 0x7fdaf0158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdae7e00640 1 -- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdaf00b3650 msgr2=0x7fdaf0146dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:05.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdae7e00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdaf00b3650 0x7fdaf0146dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdae7e00640 1 -- 172.21.15.26:0/3005640475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae00094d0 con 0x7fdaf0162430 2024-01-15T18:08:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdae7e00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 0x7fdaf0164630 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fdad800e870 tx=0x7fdad800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdaecc00640 1 -- 172.21.15.26:0/3005640475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad80094f0 con 0x7fdaf0162430 2024-01-15T18:08:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdaecc00640 1 -- 172.21.15.26:0/3005640475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad800c8e0 con 0x7fdaf0162430 2024-01-15T18:08:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf01652b0 con 0x7fdaf0162430 2024-01-15T18:08:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.921+0000 7fdaecc00640 1 -- 172.21.15.26:0/3005640475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad800cc70 con 0x7fdaf0162430 2024-01-15T18:08:05.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.922+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf0147310 con 0x7fdaf0162430 2024-01-15T18:08:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.923+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf01ebe30 con 0x7fdaf0162430 2024-01-15T18:08:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.925+0000 7fdaecc00640 1 -- 172.21.15.26:0/3005640475 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fdad8013470 con 0x7fdaf0162430 2024-01-15T18:08:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.925+0000 7fdaecc00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdac806bfe0 0x7fdac806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:05.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.925+0000 7fdaecc00640 1 -- 172.21.15.26:0/3005640475 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7fdad8088940 con 0x7fdaf0162430 2024-01-15T18:08:05.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.926+0000 7fdaefe00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdac806bfe0 0x7fdac806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:05.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.927+0000 7fdaefe00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdac806bfe0 0x7fdac806e4a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdae0036000 tx=0x7fdae0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:05.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:05.930+0000 7fdaecc00640 1 -- 172.21.15.26:0/3005640475 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdad8055260 con 0x7fdaf0162430 2024-01-15T18:08:06.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.194+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed"]} v 0) v1 -- 0x7fdaf0067450 con 0x7fdaf0162430 2024-01-15T18:08:06.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.752+0000 7fdaecc00640 1 -- 172.21.15.26:0/3005640475 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed"]}]=0 v590) v1 ==== 148+0+0 (secure 0 0 0) 0x7fdad8058880 con 0x7fdaf0162430 2024-01-15T18:08:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.755+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdac806bfe0 msgr2=0x7fdac806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.755+0000 7fdaf5c00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdac806bfe0 0x7fdac806e4a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fdae0036000 tx=0x7fdae0008030 comp rx=0 tx=0).stop 2024-01-15T18:08:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.755+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 msgr2=0x7fdaf0164630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.755+0000 7fdaf5c00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 0x7fdaf0164630 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fdad800e870 tx=0x7fdad800ed30 comp rx=0 tx=0).stop 2024-01-15T18:08:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.756+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 shutdown_connections 2024-01-15T18:08:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.756+0000 7fdaf5c00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdaf0153e80 0x7fdaf0158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.756+0000 7fdaf5c00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdac806bfe0 0x7fdac806e4a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.756+0000 7fdaf5c00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdaf0162430 0x7fdaf0164630 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:06.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.756+0000 7fdaf5c00640 1 --2- 172.21.15.26:0/3005640475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdaf00b3650 0x7fdaf0146dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:06.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.756+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 >> 172.21.15.26:0/3005640475 conn(0x7fdaf006a800 msgr2=0x7fdaf00cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:06.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.756+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 shutdown_connections 2024-01-15T18:08:06.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:06.756+0000 7fdaf5c00640 1 -- 172.21.15.26:0/3005640475 wait complete. 2024-01-15T18:08:06.785 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:08:07.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.006+0000 7f2b62e00640 1 Processor -- start 2024-01-15T18:08:07.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.006+0000 7f2b62e00640 1 -- start start 2024-01-15T18:08:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.008+0000 7f2b62e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c13df80 0x7f2b5c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.008+0000 7f2b62e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5c06c470 con 0x7f2b5c0cf9a0 2024-01-15T18:08:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.008+0000 7f2b62e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5c06c750 con 0x7f2b5c13e360 2024-01-15T18:08:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.008+0000 7f2b62e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c06ca30 con 0x7f2b5c13df80 2024-01-15T18:08:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.009+0000 7f2b60e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2b5c0cf9a0 0x7f2b5c0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36432/0 (socket says 172.21.15.26:36432) 2024-01-15T18:08:07.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.009+0000 7f2b60e00640 1 -- 172.21.15.26:0/2124315118 learned_addr learned my addr 172.21.15.26:0/2124315118 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:07.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.009+0000 7f2b5be00640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c13df80 0x7f2b5c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.009+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657129705 0 0) 0x7f2b5c06c470 con 0x7f2b5c0cf9a0 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.009+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b3c003520 con 0x7f2b5c0cf9a0 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2771969683 0 0) 0x7f2b3c003520 con 0x7f2b5c0cf9a0 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c13df80 msgr2=0x7f2b5c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c13df80 0x7f2b5c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 >> v1:172.21.15.124:6789/0 conn(0x7f2b5c13e360 legacy=0x7f2b5c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c06cd10 con 0x7f2b5c0cf9a0 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f2b540019f0 con 0x7f2b5c0cf9a0 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 >> v1:172.21.15.26:6789/0 conn(0x7f2b5c0cf9a0 legacy=0x7f2b5c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c13df80 0x7f2b3c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c003f70 0x7f2b3c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c004d00 0x7f2b3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c06ca30 con 0x7f2b3c003f70 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c00bac0 con 0x7f2b3c004d00 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c00bc80 con 0x7f2b5c13df80 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.010+0000 7f2b53e00640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c003f70 0x7f2b3c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.011+0000 7f2b60e00640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c004d00 0x7f2b3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.011+0000 7f2b5be00640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c13df80 0x7f2b3c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.011+0000 7f2b53e00640 1 -- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c13df80 msgr2=0x7f2b3c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.011+0000 7f2b53e00640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c13df80 0x7f2b3c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.011+0000 7f2b53e00640 1 -- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c004d00 msgr2=0x7f2b3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.011+0000 7f2b53e00640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c004d00 0x7f2b3c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.011+0000 7f2b53e00640 1 -- 172.21.15.26:0/2124315118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c003520 con 0x7f2b3c003f70 2024-01-15T18:08:07.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.011+0000 7f2b5be00640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c13df80 0x7f2b3c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.012+0000 7f2b53e00640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c003f70 0x7f2b3c0047c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f2b44001750 tx=0x7f2b3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8d587b81232316 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.012+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b440052d0 con 0x7f2b3c003f70 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.012+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4402e050 con 0x7f2b3c003f70 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.012+0000 7f2b5b400640 1 -- 172.21.15.26:0/2124315118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44003830 con 0x7f2b3c003f70 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.013+0000 7f2b62e00640 1 -- 172.21.15.26:0/2124315118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c003f70 msgr2=0x7f2b3c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.013+0000 7f2b62e00640 1 --2- 172.21.15.26:0/2124315118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b3c003f70 0x7f2b3c0047c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f2b44001750 tx=0x7f2b3c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.013+0000 7f2b60e00640 1 -- 172.21.15.26:0/2124315118 reap_dead start 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.013+0000 7f2b62e00640 1 -- 172.21.15.26:0/2124315118 shutdown_connections 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.013+0000 7f2b62e00640 1 -- 172.21.15.26:0/2124315118 >> 172.21.15.26:0/2124315118 conn(0x7f2b5c06a800 msgr2=0x7f2b5c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.013+0000 7f2b62e00640 1 -- 172.21.15.26:0/2124315118 shutdown_connections 2024-01-15T18:08:07.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.013+0000 7f2b62e00640 1 -- 172.21.15.26:0/2124315118 wait complete. 2024-01-15T18:08:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.015+0000 7f2b62e00640 1 Processor -- start 2024-01-15T18:08:07.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.015+0000 7f2b62e00640 1 -- start start 2024-01-15T18:08:07.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.016+0000 7f2b62e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c003f70 0x7f2b5c13d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.016+0000 7f2b62e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c15d8a0 0x7f2b5c15dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.016+0000 7f2b62e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c14bb40 0x7f2b5c14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.016+0000 7f2b62e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c15e260 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.016+0000 7f2b62e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c14c870 con 0x7f2b3c003f70 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.016+0000 7f2b62e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c14cb50 con 0x7f2b5c14bb40 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.016+0000 7f2b53e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c15d8a0 0x7f2b5c15dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b53e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c15d8a0 0x7f2b5c15dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50036/0 (socket says 172.21.15.26:50036) 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b53e00640 1 -- 172.21.15.26:0/1436579349 learned_addr learned my addr 172.21.15.26:0/1436579349 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b5be00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c003f70 0x7f2b5c13d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b60e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c14bb40 0x7f2b5c14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b53e00640 1 -- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c14bb40 msgr2=0x7f2b5c14bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b53e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c14bb40 0x7f2b5c14bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b53e00640 1 -- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c003f70 msgr2=0x7f2b5c13d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b53e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c003f70 0x7f2b5c13d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b53e00640 1 -- 172.21.15.26:0/1436579349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c14ce30 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.017+0000 7f2b53e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c15d8a0 0x7f2b5c15dd20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2b44008fd0 tx=0x7f2b44001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:07.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.018+0000 7f2b58c00640 1 -- 172.21.15.26:0/1436579349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44007a30 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.018+0000 7f2b58c00640 1 -- 172.21.15.26:0/1436579349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b440069e0 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.018+0000 7f2b58c00640 1 -- 172.21.15.26:0/1436579349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44006d90 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.018+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c14d110 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.018+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c14d910 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.019+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c0675d0 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.020+0000 7f2b58c00640 1 -- 172.21.15.26:0/1436579349 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f2b440035f0 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.021+0000 7f2b58c00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bfb0 0x7f2b3406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.021+0000 7f2b58c00640 1 -- 172.21.15.26:0/1436579349 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f2b440aa590 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.022+0000 7f2b5be00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bfb0 0x7f2b3406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.023+0000 7f2b5be00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bfb0 0x7f2b3406e470 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c159ba0 tx=0x7f2b4c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:07.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.026+0000 7f2b58c00640 1 -- 172.21.15.26:0/1436579349 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b44076f70 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.223+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2b5c06df00 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.229+0000 7f2b58c00640 1 -- 172.21.15.26:0/1436579349 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b4404b020 con 0x7f2b5c15d8a0 2024-01-15T18:08:07.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bfb0 msgr2=0x7f2b3406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bfb0 0x7f2b3406e470 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c159ba0 tx=0x7f2b4c008040 comp rx=0 tx=0).stop 2024-01-15T18:08:07.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c15d8a0 msgr2=0x7f2b5c15dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c15d8a0 0x7f2b5c15dd20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f2b44008fd0 tx=0x7f2b44001120 comp rx=0 tx=0).stop 2024-01-15T18:08:07.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 shutdown_connections 2024-01-15T18:08:07.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2b5c14bb40 0x7f2b5c14bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b3406bfb0 0x7f2b3406e470 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2b5c15d8a0 0x7f2b5c15dd20 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 --2- 172.21.15.26:0/1436579349 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2b3c003f70 0x7f2b5c13d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 >> 172.21.15.26:0/1436579349 conn(0x7f2b5c06a800 msgr2=0x7f2b5c0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:07.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 shutdown_connections 2024-01-15T18:08:07.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.232+0000 7f2b62e00640 1 -- 172.21.15.26:0/1436579349 wait complete. 2024-01-15T18:08:07.256 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:08:07.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:08:07.377 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:08:07.493 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:08:07.494 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:08:07.494 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:08:07.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:08:07.556 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:08:07.556 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:08:07.612 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:08:07.613 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:08:07.613 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:08:07.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:08:07.673 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:08:07.673 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:08:07.729 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:08:07.729 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:08:07.729 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:08:07.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.959+0000 7f95f7800640 1 Processor -- start 2024-01-15T18:08:07.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.959+0000 7f95f7800640 1 -- start start 2024-01-15T18:08:07.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.960+0000 7f95f7800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f95f013e360 0x7f95f00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.960+0000 7f95f7800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 0x7f95f00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.960+0000 7f95f7800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f015c5f0 con 0x7f95f013df80 2024-01-15T18:08:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.960+0000 7f95f7800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f006c470 con 0x7f95f013e360 2024-01-15T18:08:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.960+0000 7f95f7800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f006c750 con 0x7f95f015f9c0 2024-01-15T18:08:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.961+0000 7f95f4e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f95f013df80 0x7f95f015bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36448/0 (socket says 172.21.15.26:36448) 2024-01-15T18:08:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.961+0000 7f95f4e00640 1 -- 172.21.15.26:0/1723215208 learned_addr learned my addr 172.21.15.26:0/1723215208 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.961+0000 7f95efe00640 1 --2- 172.21.15.26:0/1723215208 >> v2:172.21.15.26:3300/0 conn(0x7f95f013e360 0x7f95f00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.961+0000 7f95f5800640 1 --2- 172.21.15.26:0/1723215208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 0x7f95f00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.961+0000 7f95efe00640 1 -- 172.21.15.26:0/1723215208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 msgr2=0x7f95f00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.961+0000 7f95efe00640 1 --2- 172.21.15.26:0/1723215208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 0x7f95f00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.961+0000 7f95efe00640 1 -- 172.21.15.26:0/1723215208 >> v1:172.21.15.26:6789/0 conn(0x7f95f013df80 legacy=0x7f95f015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.961+0000 7f95efe00640 1 -- 172.21.15.26:0/1723215208 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f006ca30 con 0x7f95f013e360 2024-01-15T18:08:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.963+0000 7f95efe00640 1 --2- 172.21.15.26:0/1723215208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 0x7f95f00cf290 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097a0 tx=0x7f95d802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b62c861571a2b34 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.963+0000 7f95ef400640 1 -- 172.21.15.26:0/1723215208 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d802d040 con 0x7f95f013e360 2024-01-15T18:08:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.963+0000 7f95ef400640 1 -- 172.21.15.26:0/1723215208 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d8031810 con 0x7f95f013e360 2024-01-15T18:08:07.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.963+0000 7f95ef400640 1 -- 172.21.15.26:0/1723215208 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8031ba0 con 0x7f95f013e360 2024-01-15T18:08:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.963+0000 7f95f7800640 1 -- 172.21.15.26:0/1723215208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 msgr2=0x7f95f00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.963+0000 7f95f7800640 1 --2- 172.21.15.26:0/1723215208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 0x7f95f00cf290 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097a0 tx=0x7f95d802b8b0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.964+0000 7f95f7800640 1 -- 172.21.15.26:0/1723215208 shutdown_connections 2024-01-15T18:08:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.964+0000 7f95f7800640 1 --2- 172.21.15.26:0/1723215208 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 0x7f95f00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.964+0000 7f95f7800640 1 --2- 172.21.15.26:0/1723215208 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 0x7f95f00cf290 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.964+0000 7f95f7800640 1 -- 172.21.15.26:0/1723215208 >> 172.21.15.26:0/1723215208 conn(0x7f95f006a800 msgr2=0x7f95f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.964+0000 7f95f7800640 1 -- 172.21.15.26:0/1723215208 shutdown_connections 2024-01-15T18:08:07.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.964+0000 7f95f7800640 1 -- 172.21.15.26:0/1723215208 wait complete. 2024-01-15T18:08:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.966+0000 7f95f7800640 1 Processor -- start 2024-01-15T18:08:07.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.966+0000 7f95f7800640 1 -- start start 2024-01-15T18:08:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.967+0000 7f95f7800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95f013df80 0x7f95f014d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.967+0000 7f95f7800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 0x7f95f0150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.967+0000 7f95f7800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 0x7f95f01f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.967+0000 7f95f7800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f0153350 con 0x7f95f013e360 2024-01-15T18:08:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.967+0000 7f95f7800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f01526d0 con 0x7f95f013df80 2024-01-15T18:08:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.967+0000 7f95f7800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f01529b0 con 0x7f95f015f9c0 2024-01-15T18:08:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.967+0000 7f95efe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 0x7f95f0150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.967+0000 7f95efe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 0x7f95f0150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50060/0 (socket says 172.21.15.26:50060) 2024-01-15T18:08:07.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.967+0000 7f95efe00640 1 -- 172.21.15.26:0/4120139082 learned_addr learned my addr 172.21.15.26:0/4120139082 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:07.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95f5800640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 0x7f95f01f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95f4e00640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95f013df80 0x7f95f014d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95efe00640 1 -- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 msgr2=0x7f95f01f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95efe00640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 0x7f95f01f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95efe00640 1 -- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95f013df80 msgr2=0x7f95f014d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95efe00640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95f013df80 0x7f95f014d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95efe00640 1 -- 172.21.15.26:0/4120139082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d8009400 con 0x7f95f013e360 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95efe00640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 0x7f95f0150590 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f95d80098c0 tx=0x7f95d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95ecc00640 1 -- 172.21.15.26:0/4120139082 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8044400 con 0x7f95f013e360 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95ecc00640 1 -- 172.21.15.26:0/4120139082 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d8044580 con 0x7f95f013e360 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95ecc00640 1 -- 172.21.15.26:0/4120139082 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8044930 con 0x7f95f013e360 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.968+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f0152c90 con 0x7f95f013e360 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.969+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f014d650 con 0x7f95f013e360 2024-01-15T18:08:07.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.970+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f00675d0 con 0x7f95f013e360 2024-01-15T18:08:07.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.971+0000 7f95ecc00640 1 -- 172.21.15.26:0/4120139082 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f95d804d440 con 0x7f95f013e360 2024-01-15T18:08:07.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.972+0000 7f95ecc00640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f95c806bfb0 0x7f95c806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:07.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.972+0000 7f95ecc00640 1 -- 172.21.15.26:0/4120139082 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f95d80aaa30 con 0x7f95f013e360 2024-01-15T18:08:07.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.972+0000 7f95f4e00640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f95c806bfb0 0x7f95c806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:07.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.974+0000 7f95f4e00640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f95c806bfb0 0x7f95c806e470 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f95e0000f60 tx=0x7f95e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:07.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:07.976+0000 7f95ecc00640 1 -- 172.21.15.26:0/4120139082 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95d8077410 con 0x7f95f013e360 2024-01-15T18:08:08.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.206+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95f006df00 con 0x7f95f013e360 2024-01-15T18:08:08.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.208+0000 7f95ecc00640 1 -- 172.21.15.26:0/4120139082 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f95d807aa30 con 0x7f95f013e360 2024-01-15T18:08:08.212 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:08.212 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6076,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6834/1986791067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":1986791067},{"type":"v1","addr":"172.21.15.26:6834","nonce":1986791067}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6082,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/3285856051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":3285856051},{"type":"v1","addr":"172.21.15.124:6836","nonce":3285856051}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6085,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/198642905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":198642905},{"type":"v1","addr":"172.21.15.124:6837","nonce":198642905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":8100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/4154591970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6833","nonce":4154591970},{"type":"v1","addr":"172.21.15.26:6835","nonce":4154591970}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[]} 2024-01-15T18:08:08.212 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 47 2024-01-15T18:08:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f95c806bfb0 msgr2=0x7f95c806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f95c806bfb0 0x7f95c806e470 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f95e0000f60 tx=0x7f95e0009040 comp rx=0 tx=0).stop 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 msgr2=0x7f95f0150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 0x7f95f0150590 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f95d80098c0 tx=0x7f95d80046c0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 shutdown_connections 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f95f015f9c0 0x7f95f01f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f95c806bfb0 0x7f95c806e470 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f95f013e360 0x7f95f0150590 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 --2- 172.21.15.26:0/4120139082 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f95f013df80 0x7f95f014d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.210+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 >> 172.21.15.26:0/4120139082 conn(0x7f95f006a800 msgr2=0x7f95f00affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.211+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 shutdown_connections 2024-01-15T18:08:08.215 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.211+0000 7f95f7800640 1 -- 172.21.15.26:0/4120139082 wait complete. 2024-01-15T18:08:08.234 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:08:08.234 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:08:08.234 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:08:08.234 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:08:08.234 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:08:08.234 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:08:08.235 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:08:08.235 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:08:08.235 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:08:08.235 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:08:08.235 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:08:08.235 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:08:08.235 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:08:08.235 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:08:08.236 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:08:08.236 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:08:08.252 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:08:08.252 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:08:08.253 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:08:08.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:08:08.256 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:08:08.257 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:08:08.257 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:08:08.258 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:08:08.299 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:08:08.299 DEBUG:teuthology.parallel:result is None 2024-01-15T18:08:08.304 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:08:08.304 DEBUG:teuthology.parallel:result is None 2024-01-15T18:08:08.305 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:08:08.305 DEBUG:teuthology.parallel:result is None 2024-01-15T18:08:08.310 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:08:08.310 DEBUG:teuthology.parallel:result is None 2024-01-15T18:08:08.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:08:08.459 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T18:08:08.461 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-15T18:08:08.463 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T18:08:08.464 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T18:08:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.504+0000 7f0933e00640 1 Processor -- start 2024-01-15T18:08:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.504+0000 7f0933e00640 1 -- start start 2024-01-15T18:08:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.505+0000 7f0933e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f09340d0100 0x7f09340b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0933e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09340d0f40 0x7f09340d1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0933e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340d0a10 con 0x7f0934160920 2024-01-15T18:08:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0933e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340cf080 con 0x7f09340d0f40 2024-01-15T18:08:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0933e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340c6be0 con 0x7f09340d0100 2024-01-15T18:08:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0933400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f09340d0f40 0x7f09340d1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0932a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0934160920 0x7f09340cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36466/0 (socket says 172.21.15.26:36466) 2024-01-15T18:08:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0932a00640 1 -- 172.21.15.26:0/3546694500 learned_addr learned my addr 172.21.15.26:0/3546694500 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:08.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f092be00640 1 --2- 172.21.15.26:0/3546694500 >> v2:172.21.15.124:3300/0 conn(0x7f09340d0100 0x7f09340b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0933400640 1 -- 172.21.15.26:0/3546694500 >> v2:172.21.15.124:3300/0 conn(0x7f09340d0100 msgr2=0x7f09340b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0933400640 1 --2- 172.21.15.26:0/3546694500 >> v2:172.21.15.124:3300/0 conn(0x7f09340d0100 0x7f09340b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0933400640 1 -- 172.21.15.26:0/3546694500 >> v1:172.21.15.26:6789/0 conn(0x7f0934160920 legacy=0x7f09340cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.506+0000 7f0933400640 1 -- 172.21.15.26:0/3546694500 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340ce7e0 con 0x7f09340d0f40 2024-01-15T18:08:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.507+0000 7f0933400640 1 --2- 172.21.15.26:0/3546694500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 0x7f09340d1390 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f093408c1f0 tx=0x7f092c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3311569c225594c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:08.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.508+0000 7f0932000640 1 -- 172.21.15.26:0/3546694500 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c02cc70 con 0x7f09340d0f40 2024-01-15T18:08:08.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.508+0000 7f0932000640 1 -- 172.21.15.26:0/3546694500 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c02cdd0 con 0x7f09340d0f40 2024-01-15T18:08:08.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.508+0000 7f0932000640 1 -- 172.21.15.26:0/3546694500 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c034660 con 0x7f09340d0f40 2024-01-15T18:08:08.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.508+0000 7f0933e00640 1 -- 172.21.15.26:0/3546694500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 msgr2=0x7f09340d1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.508+0000 7f0933e00640 1 --2- 172.21.15.26:0/3546694500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 0x7f09340d1390 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f093408c1f0 tx=0x7f092c02bec0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.508+0000 7f0933e00640 1 -- 172.21.15.26:0/3546694500 shutdown_connections 2024-01-15T18:08:08.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.508+0000 7f0933e00640 1 --2- 172.21.15.26:0/3546694500 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 0x7f09340d1390 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.508+0000 7f0933e00640 1 --2- 172.21.15.26:0/3546694500 >> v2:172.21.15.124:3300/0 conn(0x7f09340d0100 0x7f09340b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.508+0000 7f0933e00640 1 -- 172.21.15.26:0/3546694500 >> 172.21.15.26:0/3546694500 conn(0x7f09340d3090 msgr2=0x7f09340d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:08.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.509+0000 7f0933e00640 1 -- 172.21.15.26:0/3546694500 shutdown_connections 2024-01-15T18:08:08.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.509+0000 7f0933e00640 1 -- 172.21.15.26:0/3546694500 wait complete. 2024-01-15T18:08:08.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.510+0000 7f0933e00640 1 Processor -- start 2024-01-15T18:08:08.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.510+0000 7f0933e00640 1 -- start start 2024-01-15T18:08:08.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.511+0000 7f0933e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09340d0100 0x7f0934159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.511+0000 7f0933e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 0x7f0934159050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.511+0000 7f0933e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0934160920 0x7f09340b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.511+0000 7f0933e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340b6630 con 0x7f09340d0f40 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.511+0000 7f0933e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340b68b0 con 0x7f09340d0100 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.511+0000 7f0933e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340b6b90 con 0x7f0934160920 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f092be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 0x7f0934159050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f092be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 0x7f0934159050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50108/0 (socket says 172.21.15.26:50108) 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f092be00640 1 -- 172.21.15.26:0/762377523 learned_addr learned my addr 172.21.15.26:0/762377523 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f0932a00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09340d0100 0x7f0934159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f0933400640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0934160920 0x7f09340b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f092be00640 1 -- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0934160920 msgr2=0x7f09340b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f092be00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0934160920 0x7f09340b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f092be00640 1 -- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09340d0100 msgr2=0x7f0934159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f092be00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09340d0100 0x7f0934159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f092be00640 1 -- 172.21.15.26:0/762377523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c009a70 con 0x7f09340d0f40 2024-01-15T18:08:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.512+0000 7f092be00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 0x7f0934159050 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f091c00e9b0 tx=0x7f091c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.513+0000 7f092aa00640 1 -- 172.21.15.26:0/762377523 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c009670 con 0x7f09340d0f40 2024-01-15T18:08:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.513+0000 7f092aa00640 1 -- 172.21.15.26:0/762377523 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c00c9a0 con 0x7f09340d0f40 2024-01-15T18:08:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.513+0000 7f092aa00640 1 -- 172.21.15.26:0/762377523 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c00cd50 con 0x7f09340d0f40 2024-01-15T18:08:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.513+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09340b6e70 con 0x7f09340d0f40 2024-01-15T18:08:08.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.513+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0934159590 con 0x7f09340d0f40 2024-01-15T18:08:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.514+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0934066730 con 0x7f09340d0f40 2024-01-15T18:08:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.515+0000 7f092aa00640 1 -- 172.21.15.26:0/762377523 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f091c013470 con 0x7f09340d0f40 2024-01-15T18:08:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.516+0000 7f092aa00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f090c06bfb0 0x7f090c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:08.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.516+0000 7f092aa00640 1 -- 172.21.15.26:0/762377523 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f091c0888d0 con 0x7f09340d0f40 2024-01-15T18:08:08.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.516+0000 7f0932a00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f090c06bfb0 0x7f090c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:08.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.517+0000 7f0932a00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f090c06bfb0 0x7f090c06e470 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0924005880 tx=0x7f0924009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:08.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.520+0000 7f092aa00640 1 -- 172.21.15.26:0/762377523 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f091c055180 con 0x7f09340d0f40 2024-01-15T18:08:08.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.732+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f093415fd80 con 0x7f09340d0f40 2024-01-15T18:08:08.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.779+0000 7f092aa00640 1 -- 172.21.15.26:0/762377523 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) v1 ==== 97+0+0 (secure 0 0 0) 0x7f091c0587a0 con 0x7f09340d0f40 2024-01-15T18:08:08.783 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-01-15T18:08:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f090c06bfb0 msgr2=0x7f090c06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f090c06bfb0 0x7f090c06e470 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0924005880 tx=0x7f0924009040 comp rx=0 tx=0).stop 2024-01-15T18:08:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 msgr2=0x7f0934159050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 0x7f0934159050 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f091c00e9b0 tx=0x7f091c00ee70 comp rx=0 tx=0).stop 2024-01-15T18:08:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 shutdown_connections 2024-01-15T18:08:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0934160920 0x7f09340b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f090c06bfb0 0x7f090c06e470 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f09340d0f40 0x7f0934159050 secure :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f091c00e9b0 tx=0x7f091c00ee70 comp rx=0 tx=0).stop 2024-01-15T18:08:08.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 --2- 172.21.15.26:0/762377523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f09340d0100 0x7f0934159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:08.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 >> 172.21.15.26:0/762377523 conn(0x7f09340d3090 msgr2=0x7f093415c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:08.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.783+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 shutdown_connections 2024-01-15T18:08:08.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:08.784+0000 7f0933e00640 1 -- 172.21.15.26:0/762377523 wait complete. 2024-01-15T18:08:08.808 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:08:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.016+0000 7f9d3d800640 1 Processor -- start 2024-01-15T18:08:09.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.016+0000 7f9d3d800640 1 -- start start 2024-01-15T18:08:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.018+0000 7f9d3d800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9d380cf9a0 0x7f9d3815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.018+0000 7f9d3d800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d380b1ee0 con 0x7f9d38143ae0 2024-01-15T18:08:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.018+0000 7f9d3d800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d380b2780 con 0x7f9d3815e1e0 2024-01-15T18:08:09.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.018+0000 7f9d3d800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3815e990 con 0x7f9d380cf9a0 2024-01-15T18:08:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.018+0000 7f9d36a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9d38143ae0 0x7f9d380cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36482/0 (socket says 172.21.15.26:36482) 2024-01-15T18:08:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.018+0000 7f9d36a00640 1 -- 172.21.15.26:0/2764162831 learned_addr learned my addr 172.21.15.26:0/2764162831 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.019+0000 7f9d2de00640 1 --2- 172.21.15.26:0/2764162831 >> v2:172.21.15.124:3300/0 conn(0x7f9d380cf9a0 0x7f9d3815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.019+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220679113 0 0) 0x7f9d380b1ee0 con 0x7f9d38143ae0 2024-01-15T18:08:09.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.019+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d18003520 con 0x7f9d38143ae0 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.019+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3734382387 0 0) 0x7f9d18003520 con 0x7f9d38143ae0 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 >> v2:172.21.15.124:3300/0 conn(0x7f9d380cf9a0 msgr2=0x7f9d3815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 --2- 172.21.15.26:0/2764162831 >> v2:172.21.15.124:3300/0 conn(0x7f9d380cf9a0 0x7f9d3815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 >> v1:172.21.15.124:6789/0 conn(0x7f9d3815e1e0 legacy=0x7f9d380702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380b2130 con 0x7f9d38143ae0 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9d28001670 con 0x7f9d38143ae0 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 >> v1:172.21.15.26:6789/0 conn(0x7f9d38143ae0 legacy=0x7f9d380cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 --2- 172.21.15.26:0/2764162831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d180039c0 0x7f9d18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 --2- 172.21.15.26:0/2764162831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 --2- 172.21.15.26:0/2764162831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d18005180 0x7f9d1800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3815e990 con 0x7f9d180047c0 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1800e290 con 0x7f9d18005180 2024-01-15T18:08:09.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1800e450 con 0x7f9d180039c0 2024-01-15T18:08:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d2de00640 1 --2- 172.21.15.26:0/2764162831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d2de00640 1 -- 172.21.15.26:0/2764162831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d180039c0 msgr2=0x7f9d18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d2de00640 1 --2- 172.21.15.26:0/2764162831 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d180039c0 0x7f9d18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d2de00640 1 -- 172.21.15.26:0/2764162831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d18005180 msgr2=0x7f9d1800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d2de00640 1 --2- 172.21.15.26:0/2764162831 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d18005180 0x7f9d1800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d2de00640 1 -- 172.21.15.26:0/2764162831 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18003520 con 0x7f9d180047c0 2024-01-15T18:08:09.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.020+0000 7f9d37400640 1 -- 172.21.15.26:0/2764162831 reap_dead start 2024-01-15T18:08:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.022+0000 7f9d2de00640 1 --2- 172.21.15.26:0/2764162831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d18004c40 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009fc0 tx=0x7f9d180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b73935a73d13ba7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.022+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2002cc30 con 0x7f9d180047c0 2024-01-15T18:08:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.022+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2000f040 con 0x7f9d180047c0 2024-01-15T18:08:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.022+0000 7f9d36000640 1 -- 172.21.15.26:0/2764162831 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20034440 con 0x7f9d180047c0 2024-01-15T18:08:09.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.022+0000 7f9d3d800640 1 -- 172.21.15.26:0/2764162831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 msgr2=0x7f9d18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.022+0000 7f9d3d800640 1 --2- 172.21.15.26:0/2764162831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d18004c40 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009fc0 tx=0x7f9d180057f0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.022+0000 7f9d3d800640 1 -- 172.21.15.26:0/2764162831 shutdown_connections 2024-01-15T18:08:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.022+0000 7f9d3d800640 1 --2- 172.21.15.26:0/2764162831 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d18004c40 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.022+0000 7f9d3d800640 1 -- 172.21.15.26:0/2764162831 >> 172.21.15.26:0/2764162831 conn(0x7f9d3806a800 msgr2=0x7f9d3806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.023+0000 7f9d3d800640 1 -- 172.21.15.26:0/2764162831 shutdown_connections 2024-01-15T18:08:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.023+0000 7f9d3d800640 1 -- 172.21.15.26:0/2764162831 wait complete. 2024-01-15T18:08:09.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.024+0000 7f9d3d800640 1 Processor -- start 2024-01-15T18:08:09.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.024+0000 7f9d3d800640 1 -- start start 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.025+0000 7f9d3d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d381f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.025+0000 7f9d3d800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380ae1b0 0x7f9d3815b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d3d800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d3815b640 0x7f9d380aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d3d800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d381f73a0 con 0x7f9d180047c0 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d3d800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3813fef0 con 0x7f9d3815b640 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d3d800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3815c0a0 con 0x7f9d380ae1b0 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d36a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d381f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d36a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d381f6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50132/0 (socket says 172.21.15.26:50132) 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d36a00640 1 -- 172.21.15.26:0/537889281 learned_addr learned my addr 172.21.15.26:0/537889281 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d37400640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d3815b640 0x7f9d380aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d36a00640 1 -- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380ae1b0 msgr2=0x7f9d3815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d36a00640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380ae1b0 0x7f9d3815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d36a00640 1 -- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d3815b640 msgr2=0x7f9d380aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d36a00640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d3815b640 0x7f9d380aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d36a00640 1 -- 172.21.15.26:0/537889281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d380a85f0 con 0x7f9d180047c0 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.026+0000 7f9d37400640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d3815b640 0x7f9d380aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.027+0000 7f9d36a00640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d381f6e60 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800c730 tx=0x7f9d2800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.027+0000 7f9d2f400640 1 -- 172.21.15.26:0/537889281 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280072d0 con 0x7f9d180047c0 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.027+0000 7f9d2f400640 1 -- 172.21.15.26:0/537889281 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d28007430 con 0x7f9d180047c0 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.027+0000 7f9d2f400640 1 -- 172.21.15.26:0/537889281 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280047e0 con 0x7f9d180047c0 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.027+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3815bae0 con 0x7f9d180047c0 2024-01-15T18:08:09.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.027+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d380a8890 con 0x7f9d180047c0 2024-01-15T18:08:09.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.028+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d38067f50 con 0x7f9d180047c0 2024-01-15T18:08:09.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.029+0000 7f9d2f400640 1 -- 172.21.15.26:0/537889281 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f9d28004940 con 0x7f9d180047c0 2024-01-15T18:08:09.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.030+0000 7f9d2f400640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d1006bfe0 0x7f9d1006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.030+0000 7f9d2f400640 1 -- 172.21.15.26:0/537889281 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f9d28087a90 con 0x7f9d180047c0 2024-01-15T18:08:09.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.031+0000 7f9d2de00640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d1006bfe0 0x7f9d1006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.032+0000 7f9d2de00640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d1006bfe0 0x7f9d1006e4a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9d20036000 tx=0x7f9d2002e070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:09.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.034+0000 7f9d2f400640 1 -- 172.21.15.26:0/537889281 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d28054350 con 0x7f9d180047c0 2024-01-15T18:08:09.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.304+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9d380a8a80 con 0x7f9d180047c0 2024-01-15T18:08:09.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.306+0000 7f9d2f400640 1 -- 172.21.15.26:0/537889281 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9d28018090 con 0x7f9d180047c0 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVl3erNJhAANxPK+39yVM5neq9iKC1HkA==", 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.310 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlLaUlDRAAeKaen5hLlwDDWY6OAMRsPQ==", 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlaIAVOBAAA8LEFM1HtzRtNooQCCjS/w==", 2024-01-15T18:08:09.311 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlmpTBHRAAUwbZeVpysMsn56QRLRJCoA==", 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:08:09.312 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA7caVlCTmgDhAAPWb0qEh134xxyF8lM9DoNg==", 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.313 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA+caVlaxF5JxAA23Z9j21ATy3GVlhgndDIxw==", 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBCcaVlcRtjBRAAOFN9Xf5kN5MXL6yQfRI27g==", 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:08:09.314 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBFcaVlvAzEHhAACDoNh7BZY9gbKTdGwIrhlg==", 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.315 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBMcaVl8yiOMBAAdI4NTeSB1d/nSHRaIDMcFw==", 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBQcaVltf8sARAAfPaJAfoTgQoEivw90jRz/Q==", 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.316 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBTcaVltuKiGhAAT0cIJiRkQ38QXT/h6Gyggg==", 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.317 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBWcaVlbJqVMhAAKjpl6pAHbytjZWMYeRbRjA==", 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA==", 2024-01-15T18:08:09.318 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVlrVpqBxAAxOL0iuAlOvfR+KgSsin3vQ==", 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:08:09.319 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlO8vdNxAAlUE/TFCDQoXuxvwLPwUvtg==", 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.320 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVl5qgCEhAALgt7d3OjEmq5bc4nAD2rTw==", 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA2caVlyg4eHBAAwTCXVk8hO9+zcZ1tUA3gfA==", 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:08:09.321 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlvuBcGhAAGqdCMSffJwnMKlFh3CtyCw==", 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.322 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVle8Z+DhAA9cWxzc4puvrHAedqupPG3Q==", 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-01-15T18:08:09.323 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:08:09.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d1006bfe0 msgr2=0x7f9d1006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d1006bfe0 0x7f9d1006e4a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9d20036000 tx=0x7f9d2002e070 comp rx=0 tx=0).stop 2024-01-15T18:08:09.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 msgr2=0x7f9d381f6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d381f6e60 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800c730 tx=0x7f9d2800cbf0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 shutdown_connections 2024-01-15T18:08:09.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d380ae1b0 0x7f9d3815b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d1006bfe0 0x7f9d1006e4a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d3815b640 0x7f9d380aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 --2- 172.21.15.26:0/537889281 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d180047c0 0x7f9d381f6e60 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 >> 172.21.15.26:0/537889281 conn(0x7f9d3806a800 msgr2=0x7f9d38142740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:09.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 shutdown_connections 2024-01-15T18:08:09.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.309+0000 7f9d3d800640 1 -- 172.21.15.26:0/537889281 wait complete. 2024-01-15T18:08:09.332 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:08:09.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.549+0000 7f9229a00640 1 Processor -- start 2024-01-15T18:08:09.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.550+0000 7f9229a00640 1 -- start start 2024-01-15T18:08:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.551+0000 7f9229a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f922414b2c0 0x7f92240cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.551+0000 7f9229a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f92240cf7d0 0x7f9224070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.551+0000 7f9229a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9224070ea0 0x7f92240b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.551+0000 7f9229a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922414b6a0 con 0x7f92240cf7d0 2024-01-15T18:08:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.551+0000 7f9229a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92240cfbf0 con 0x7f9224070ea0 2024-01-15T18:08:09.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.551+0000 7f9229a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9224071690 con 0x7f922414b2c0 2024-01-15T18:08:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9219e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f92240cf7d0 0x7f9224070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9219e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f92240cf7d0 0x7f9224070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50134/0 (socket says 172.21.15.26:50134) 2024-01-15T18:08:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9219e00640 1 -- 172.21.15.26:0/3295181637 learned_addr learned my addr 172.21.15.26:0/3295181637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9222a00640 1 --2- 172.21.15.26:0/3295181637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f922414b2c0 0x7f92240cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9223400640 1 --2- 172.21.15.26:0/3295181637 >> v2:172.21.15.124:3300/0 conn(0x7f9224070ea0 0x7f92240b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9219e00640 1 -- 172.21.15.26:0/3295181637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f922414b2c0 msgr2=0x7f92240cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9219e00640 1 --2- 172.21.15.26:0/3295181637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f922414b2c0 0x7f92240cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9219e00640 1 -- 172.21.15.26:0/3295181637 >> v2:172.21.15.124:3300/0 conn(0x7f9224070ea0 msgr2=0x7f92240b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9219e00640 1 --2- 172.21.15.26:0/3295181637 >> v2:172.21.15.124:3300/0 conn(0x7f9224070ea0 0x7f92240b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.552+0000 7f9219e00640 1 -- 172.21.15.26:0/3295181637 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9224071970 con 0x7f92240cf7d0 2024-01-15T18:08:09.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.554+0000 7f9219e00640 1 --2- 172.21.15.26:0/3295181637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92240cf7d0 0x7f9224070960 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f922406e2a0 tx=0x7f920c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce6b3baa8aa6d6b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.554+0000 7f9222000640 1 -- 172.21.15.26:0/3295181637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c031870 con 0x7f92240cf7d0 2024-01-15T18:08:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.554+0000 7f9222000640 1 -- 172.21.15.26:0/3295181637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c0319d0 con 0x7f92240cf7d0 2024-01-15T18:08:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.554+0000 7f9222000640 1 -- 172.21.15.26:0/3295181637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c031d60 con 0x7f92240cf7d0 2024-01-15T18:08:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.555+0000 7f9229a00640 1 -- 172.21.15.26:0/3295181637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92240cf7d0 msgr2=0x7f9224070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.555+0000 7f9229a00640 1 --2- 172.21.15.26:0/3295181637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92240cf7d0 0x7f9224070960 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f922406e2a0 tx=0x7f920c02b910 comp rx=0 tx=0).stop 2024-01-15T18:08:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.555+0000 7f9229a00640 1 -- 172.21.15.26:0/3295181637 shutdown_connections 2024-01-15T18:08:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.555+0000 7f9229a00640 1 --2- 172.21.15.26:0/3295181637 >> v2:172.21.15.124:3300/0 conn(0x7f9224070ea0 0x7f92240b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.555+0000 7f9229a00640 1 --2- 172.21.15.26:0/3295181637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f92240cf7d0 0x7f9224070960 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.555+0000 7f9229a00640 1 --2- 172.21.15.26:0/3295181637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f922414b2c0 0x7f92240cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.555+0000 7f9229a00640 1 -- 172.21.15.26:0/3295181637 >> 172.21.15.26:0/3295181637 conn(0x7f922406a800 msgr2=0x7f922406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.555+0000 7f9229a00640 1 -- 172.21.15.26:0/3295181637 shutdown_connections 2024-01-15T18:08:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.555+0000 7f9229a00640 1 -- 172.21.15.26:0/3295181637 wait complete. 2024-01-15T18:08:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.557+0000 7f9229a00640 1 Processor -- start 2024-01-15T18:08:09.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.557+0000 7f9229a00640 1 -- start start 2024-01-15T18:08:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.558+0000 7f9229a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9224070ea0 0x7f92241f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.558+0000 7f9229a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92240cf7d0 0x7f92240ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.558+0000 7f9229a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f922414b2c0 0x7f92240adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.558+0000 7f9229a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92241eaef0 con 0x7f922414b2c0 2024-01-15T18:08:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.558+0000 7f9229a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92241f7030 con 0x7f92240cf7d0 2024-01-15T18:08:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.558+0000 7f9229a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9224133890 con 0x7f9224070ea0 2024-01-15T18:08:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9223400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f922414b2c0 0x7f92240adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9223400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f922414b2c0 0x7f92240adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50146/0 (socket says 172.21.15.26:50146) 2024-01-15T18:08:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9223400640 1 -- 172.21.15.26:0/439983925 learned_addr learned my addr 172.21.15.26:0/439983925 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9219e00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92240cf7d0 0x7f92240ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9222a00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9224070ea0 0x7f92241f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9223400640 1 -- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9224070ea0 msgr2=0x7f92241f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9223400640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9224070ea0 0x7f92241f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9223400640 1 -- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92240cf7d0 msgr2=0x7f92240ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9223400640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92240cf7d0 0x7f92240ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.559+0000 7f9223400640 1 -- 172.21.15.26:0/439983925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c0094d0 con 0x7f922414b2c0 2024-01-15T18:08:09.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.560+0000 7f9223400640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f922414b2c0 0x7f92240adf60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f921c007d20 tx=0x7f921c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.560+0000 7f921b400640 1 -- 172.21.15.26:0/439983925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c014420 con 0x7f922414b2c0 2024-01-15T18:08:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.560+0000 7f921b400640 1 -- 172.21.15.26:0/439983925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c0145a0 con 0x7f922414b2c0 2024-01-15T18:08:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.560+0000 7f921b400640 1 -- 172.21.15.26:0/439983925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c014950 con 0x7f922414b2c0 2024-01-15T18:08:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.560+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9224133b70 con 0x7f922414b2c0 2024-01-15T18:08:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.560+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9224134130 con 0x7f922414b2c0 2024-01-15T18:08:09.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.561+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92240675d0 con 0x7f922414b2c0 2024-01-15T18:08:09.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.562+0000 7f921b400640 1 -- 172.21.15.26:0/439983925 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f921c014ab0 con 0x7f922414b2c0 2024-01-15T18:08:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.564+0000 7f921b400640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f91fc06bfb0 0x7f91fc06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:09.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.564+0000 7f921b400640 1 -- 172.21.15.26:0/439983925 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f921c088ef0 con 0x7f922414b2c0 2024-01-15T18:08:09.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.564+0000 7f9222a00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f91fc06bfb0 0x7f91fc06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:09.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.566+0000 7f9222a00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f91fc06bfb0 0x7f91fc06e470 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9214006fd0 tx=0x7f9214008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:09.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.568+0000 7f921b400640 1 -- 172.21.15.26:0/439983925 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f921c0557b0 con 0x7f922414b2c0 2024-01-15T18:08:09.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.764+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f922406af60 con 0x7f922414b2c0 2024-01-15T18:08:09.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.766+0000 7f921b400640 1 -- 172.21.15.26:0/439983925 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+12921 (secure 0 0 0) 0x7f921c058dd0 con 0x7f922414b2c0 2024-01-15T18:08:09.770 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:09.771 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":83,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:08:08.791853+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:08:09.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f91fc06bfb0 msgr2=0x7f91fc06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f91fc06bfb0 0x7f91fc06e470 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9214006fd0 tx=0x7f9214008040 comp rx=0 tx=0).stop 2024-01-15T18:08:09.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f922414b2c0 msgr2=0x7f92240adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:09.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f922414b2c0 0x7f92240adf60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f921c007d20 tx=0x7f921c00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 shutdown_connections 2024-01-15T18:08:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f91fc06bfb0 0x7f91fc06e470 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f922414b2c0 0x7f92240adf60 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f92240cf7d0 0x7f92240ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 --2- 172.21.15.26:0/439983925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9224070ea0 0x7f92241f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.769+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 >> 172.21.15.26:0/439983925 conn(0x7f922406a800 msgr2=0x7f92240ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.770+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 shutdown_connections 2024-01-15T18:08:09.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:09.770+0000 7f9229a00640 1 -- 172.21.15.26:0/439983925 wait complete. 2024-01-15T18:08:09.793 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:08:09.793 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:08:10.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.014+0000 7fee3fe00640 1 Processor -- start 2024-01-15T18:08:10.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.014+0000 7fee3fe00640 1 -- start start 2024-01-15T18:08:10.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.016+0000 7fee3fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee40071e40 0x7fee400ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.016+0000 7fee3fe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fee4015d770 0x7fee4015dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.016+0000 7fee3fe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4006c730 con 0x7fee40143810 2024-01-15T18:08:10.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.016+0000 7fee3fe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee40143bf0 con 0x7fee40071e40 2024-01-15T18:08:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.016+0000 7fee3fe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee400722f0 con 0x7fee4015d770 2024-01-15T18:08:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.017+0000 7fee37e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee40071e40 0x7fee400ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.017+0000 7fee37e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fee40071e40 0x7fee400ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50154/0 (socket says 172.21.15.26:50154) 2024-01-15T18:08:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.017+0000 7fee37e00640 1 -- 172.21.15.26:0/2467921750 learned_addr learned my addr 172.21.15.26:0/2467921750 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.017+0000 7fee3f400640 1 --2- 172.21.15.26:0/2467921750 >> v2:172.21.15.124:3300/0 conn(0x7fee4015d770 0x7fee4015dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.017+0000 7fee37e00640 1 -- 172.21.15.26:0/2467921750 >> v2:172.21.15.124:3300/0 conn(0x7fee4015d770 msgr2=0x7fee4015dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.017+0000 7fee37e00640 1 --2- 172.21.15.26:0/2467921750 >> v2:172.21.15.124:3300/0 conn(0x7fee4015d770 0x7fee4015dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.017+0000 7fee37e00640 1 -- 172.21.15.26:0/2467921750 >> v1:172.21.15.124:6789/0 conn(0x7fee40143810 legacy=0x7fee4015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:10.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.017+0000 7fee37e00640 1 -- 172.21.15.26:0/2467921750 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee400724d0 con 0x7fee40071e40 2024-01-15T18:08:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.018+0000 7fee37e00640 1 --2- 172.21.15.26:0/2467921750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee40071e40 0x7fee400ae170 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fee400b4440 tx=0x7fee2802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3cb635d8d0747 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.019+0000 7fee3e000640 1 -- 172.21.15.26:0/2467921750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28031750 con 0x7fee40071e40 2024-01-15T18:08:10.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.019+0000 7fee3e000640 1 -- 172.21.15.26:0/2467921750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee280318b0 con 0x7fee40071e40 2024-01-15T18:08:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.019+0000 7fee3e000640 1 -- 172.21.15.26:0/2467921750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee28031c40 con 0x7fee40071e40 2024-01-15T18:08:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.019+0000 7fee3fe00640 1 -- 172.21.15.26:0/2467921750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee40071e40 msgr2=0x7fee400ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.019+0000 7fee3fe00640 1 --2- 172.21.15.26:0/2467921750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee40071e40 0x7fee400ae170 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fee400b4440 tx=0x7fee2802b7f0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.020+0000 7fee3fe00640 1 -- 172.21.15.26:0/2467921750 shutdown_connections 2024-01-15T18:08:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.020+0000 7fee3fe00640 1 --2- 172.21.15.26:0/2467921750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee40071e40 0x7fee400ae170 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.020+0000 7fee3fe00640 1 --2- 172.21.15.26:0/2467921750 >> v2:172.21.15.124:3300/0 conn(0x7fee4015d770 0x7fee4015dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.020+0000 7fee3fe00640 1 -- 172.21.15.26:0/2467921750 >> 172.21.15.26:0/2467921750 conn(0x7fee4006a7d0 msgr2=0x7fee4006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.020+0000 7fee3fe00640 1 -- 172.21.15.26:0/2467921750 shutdown_connections 2024-01-15T18:08:10.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.020+0000 7fee3fe00640 1 -- 172.21.15.26:0/2467921750 wait complete. 2024-01-15T18:08:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.021+0000 7fee3fe00640 1 Processor -- start 2024-01-15T18:08:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.022+0000 7fee3fe00640 1 -- start start 2024-01-15T18:08:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.022+0000 7fee3fe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee40071e40 0x7fee401f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.022+0000 7fee3fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee40143810 0x7fee400cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015d770 0x7fee4015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee400ccac0 con 0x7fee4015d770 2024-01-15T18:08:10.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3fe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee400aff00 con 0x7fee40071e40 2024-01-15T18:08:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee401f3c40 con 0x7fee40143810 2024-01-15T18:08:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015d770 0x7fee4015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015d770 0x7fee4015ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50162/0 (socket says 172.21.15.26:50162) 2024-01-15T18:08:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3f400640 1 -- 172.21.15.26:0/2507263345 learned_addr learned my addr 172.21.15.26:0/2507263345 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee37e00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee40143810 0x7fee400cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3ea00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee40071e40 0x7fee401f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3f400640 1 -- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee40143810 msgr2=0x7fee400cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3f400640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee40143810 0x7fee400cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3f400640 1 -- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee40071e40 msgr2=0x7fee401f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3f400640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee40071e40 0x7fee401f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.023+0000 7fee3f400640 1 -- 172.21.15.26:0/2507263345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee280094d0 con 0x7fee4015d770 2024-01-15T18:08:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.024+0000 7fee3f400640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015d770 0x7fee4015ca90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fee38007e60 tx=0x7fee3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.024+0000 7fee36a00640 1 -- 172.21.15.26:0/2507263345 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38014420 con 0x7fee4015d770 2024-01-15T18:08:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.024+0000 7fee36a00640 1 -- 172.21.15.26:0/2507263345 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee380145a0 con 0x7fee4015d770 2024-01-15T18:08:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.024+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee400cbfc0 con 0x7fee4015d770 2024-01-15T18:08:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.024+0000 7fee36a00640 1 -- 172.21.15.26:0/2507263345 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee38014950 con 0x7fee4015d770 2024-01-15T18:08:10.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.024+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee400cc500 con 0x7fee4015d770 2024-01-15T18:08:10.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.025+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee400afc60 con 0x7fee4015d770 2024-01-15T18:08:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.026+0000 7fee36a00640 1 -- 172.21.15.26:0/2507263345 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fee38014ab0 con 0x7fee4015d770 2024-01-15T18:08:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.027+0000 7fee36a00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fee1806bfb0 0x7fee1806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.027+0000 7fee36a00640 1 -- 172.21.15.26:0/2507263345 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fee38088f70 con 0x7fee4015d770 2024-01-15T18:08:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.028+0000 7fee3ea00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fee1806bfb0 0x7fee1806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.029+0000 7fee3ea00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fee1806bfb0 0x7fee1806e470 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fee30004ea0 tx=0x7fee30009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:10.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.032+0000 7fee36a00640 1 -- 172.21.15.26:0/2507263345 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fee400afc60 con 0x7fee4015d770 2024-01-15T18:08:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.226+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee400b0040 con 0x7fee4015d770 2024-01-15T18:08:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.227+0000 7fee36a00640 1 -- 172.21.15.26:0/2507263345 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee400b0040 con 0x7fee4015d770 2024-01-15T18:08:10.231 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:08:10.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fee1806bfb0 msgr2=0x7fee1806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fee1806bfb0 0x7fee1806e470 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fee30004ea0 tx=0x7fee30009040 comp rx=0 tx=0).stop 2024-01-15T18:08:10.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015d770 msgr2=0x7fee4015ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015d770 0x7fee4015ca90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fee38007e60 tx=0x7fee3800e4b0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 shutdown_connections 2024-01-15T18:08:10.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fee40143810 0x7fee400cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fee1806bfb0 0x7fee1806e470 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fee4015d770 0x7fee4015ca90 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 --2- 172.21.15.26:0/2507263345 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fee40071e40 0x7fee401f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 >> 172.21.15.26:0/2507263345 conn(0x7fee4006a7d0 msgr2=0x7fee400ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 shutdown_connections 2024-01-15T18:08:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.230+0000 7fee3fe00640 1 -- 172.21.15.26:0/2507263345 wait complete. 2024-01-15T18:08:10.253 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:08:10.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.473+0000 7fc0dfa00640 1 Processor -- start 2024-01-15T18:08:10.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.473+0000 7fc0dfa00640 1 -- start start 2024-01-15T18:08:10.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.475+0000 7fc0dfa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 0x7fc0d815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.475+0000 7fc0dfa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0d80b3ae0 0x7fc0d806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.475+0000 7fc0dfa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0d806d120 0x7fc0d80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.475+0000 7fc0dfa00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d8071450 con 0x7fc0d80b3ae0 2024-01-15T18:08:10.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.475+0000 7fc0dfa00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0d8070c40 con 0x7fc0d806d120 2024-01-15T18:08:10.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.475+0000 7fc0dfa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d806c470 con 0x7fc0d80ae5a0 2024-01-15T18:08:10.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.475+0000 7fc0d7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0d80b3ae0 0x7fc0d806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.476+0000 7fc0d7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc0d80b3ae0 0x7fc0d806cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50176/0 (socket says 172.21.15.26:50176) 2024-01-15T18:08:10.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.476+0000 7fc0d7e00640 1 -- 172.21.15.26:0/4174768749 learned_addr learned my addr 172.21.15.26:0/4174768749 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:10.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.476+0000 7fc0dda00640 1 --2- 172.21.15.26:0/4174768749 >> v2:172.21.15.124:3300/0 conn(0x7fc0d806d120 0x7fc0d80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.476+0000 7fc0dd000640 1 --2- 172.21.15.26:0/4174768749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 0x7fc0d815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.476+0000 7fc0d7e00640 1 -- 172.21.15.26:0/4174768749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 msgr2=0x7fc0d815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:10.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.476+0000 7fc0d7e00640 1 --2- 172.21.15.26:0/4174768749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 0x7fc0d815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.476+0000 7fc0d7e00640 1 -- 172.21.15.26:0/4174768749 >> v2:172.21.15.124:3300/0 conn(0x7fc0d806d120 msgr2=0x7fc0d80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:10.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.476+0000 7fc0d7e00640 1 --2- 172.21.15.26:0/4174768749 >> v2:172.21.15.124:3300/0 conn(0x7fc0d806d120 0x7fc0d80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.476+0000 7fc0d7e00640 1 -- 172.21.15.26:0/4174768749 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d806c750 con 0x7fc0d80b3ae0 2024-01-15T18:08:10.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.478+0000 7fc0d7e00640 1 --2- 172.21.15.26:0/4174768749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80b3ae0 0x7fc0d806cbe0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc0d806e230 tx=0x7fc0c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0f7a12ddf1b1f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:10.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.478+0000 7fc0d7400640 1 -- 172.21.15.26:0/4174768749 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0031870 con 0x7fc0d80b3ae0 2024-01-15T18:08:10.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.478+0000 7fc0d7400640 1 -- 172.21.15.26:0/4174768749 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c00319d0 con 0x7fc0d80b3ae0 2024-01-15T18:08:10.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0d7400640 1 -- 172.21.15.26:0/4174768749 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0031d60 con 0x7fc0d80b3ae0 2024-01-15T18:08:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0dfa00640 1 -- 172.21.15.26:0/4174768749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80b3ae0 msgr2=0x7fc0d806cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/4174768749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80b3ae0 0x7fc0d806cbe0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fc0d806e230 tx=0x7fc0c002b910 comp rx=0 tx=0).stop 2024-01-15T18:08:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0dfa00640 1 -- 172.21.15.26:0/4174768749 shutdown_connections 2024-01-15T18:08:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/4174768749 >> v2:172.21.15.124:3300/0 conn(0x7fc0d806d120 0x7fc0d80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/4174768749 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d80b3ae0 0x7fc0d806cbe0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/4174768749 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 0x7fc0d815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0dfa00640 1 -- 172.21.15.26:0/4174768749 >> 172.21.15.26:0/4174768749 conn(0x7fc0d806a800 msgr2=0x7fc0d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0dfa00640 1 -- 172.21.15.26:0/4174768749 shutdown_connections 2024-01-15T18:08:10.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.479+0000 7fc0dfa00640 1 -- 172.21.15.26:0/4174768749 wait complete. 2024-01-15T18:08:10.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.482+0000 7fc0dfa00640 1 Processor -- start 2024-01-15T18:08:10.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.482+0000 7fc0dfa00640 1 -- start start 2024-01-15T18:08:10.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.483+0000 7fc0dfa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d806d120 0x7fc0d80c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.483+0000 7fc0dfa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 0x7fc0d814b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.483+0000 7fc0dfa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d80b3ae0 0x7fc0d815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.483+0000 7fc0dfa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8156910 con 0x7fc0d806d120 2024-01-15T18:08:10.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.483+0000 7fc0dfa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8156bf0 con 0x7fc0d80b3ae0 2024-01-15T18:08:10.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.483+0000 7fc0dfa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d8156ed0 con 0x7fc0d80ae5a0 2024-01-15T18:08:10.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.483+0000 7fc0dd000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d806d120 0x7fc0d80c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.483+0000 7fc0dd000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d806d120 0x7fc0d80c3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50178/0 (socket says 172.21.15.26:50178) 2024-01-15T18:08:10.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.483+0000 7fc0dd000640 1 -- 172.21.15.26:0/3458816487 learned_addr learned my addr 172.21.15.26:0/3458816487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.484+0000 7fc0dd000640 1 -- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 msgr2=0x7fc0d814b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.484+0000 7fc0d7e00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 0x7fc0d814b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.484+0000 7fc0dd000640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 0x7fc0d814b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.484+0000 7fc0dda00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d80b3ae0 0x7fc0d815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.484+0000 7fc0dd000640 1 -- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d80b3ae0 msgr2=0x7fc0d815ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.484+0000 7fc0dd000640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d80b3ae0 0x7fc0d815ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.484+0000 7fc0dd000640 1 -- 172.21.15.26:0/3458816487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c00094d0 con 0x7fc0d806d120 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.484+0000 7fc0dd000640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d806d120 0x7fc0d80c3e50 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800efc0 tx=0x7fc0c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.484+0000 7fc0d4c00640 1 -- 172.21.15.26:0/3458816487 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8014480 con 0x7fc0d806d120 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.485+0000 7fc0d4c00640 1 -- 172.21.15.26:0/3458816487 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c800f040 con 0x7fc0d806d120 2024-01-15T18:08:10.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.485+0000 7fc0d4c00640 1 -- 172.21.15.26:0/3458816487 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8014800 con 0x7fc0d806d120 2024-01-15T18:08:10.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.485+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0d81571b0 con 0x7fc0d806d120 2024-01-15T18:08:10.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.485+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0d814be10 con 0x7fc0d806d120 2024-01-15T18:08:10.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.486+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0d80675d0 con 0x7fc0d806d120 2024-01-15T18:08:10.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.487+0000 7fc0d4c00640 1 -- 172.21.15.26:0/3458816487 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fc0c8014960 con 0x7fc0d806d120 2024-01-15T18:08:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.488+0000 7fc0d4c00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006bfb0 0x7fc0b006e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.488+0000 7fc0d4c00640 1 -- 172.21.15.26:0/3458816487 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fc0c80894d0 con 0x7fc0d806d120 2024-01-15T18:08:10.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.489+0000 7fc0d7e00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006bfb0 0x7fc0b006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.490+0000 7fc0d7e00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006bfb0 0x7fc0b006e470 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0036000 tx=0x7fc0c003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:10.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.492+0000 7fc0d4c00640 1 -- 172.21.15.26:0/3458816487 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0c8055b00 con 0x7fc0d806d120 2024-01-15T18:08:10.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.691+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc0d815a940 con 0x7fc0d806d120 2024-01-15T18:08:10.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.692+0000 7fc0d4c00640 1 -- 172.21.15.26:0/3458816487 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc0c8059120 con 0x7fc0d806d120 2024-01-15T18:08:10.696 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:10.696 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:08:10.696 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:10.696 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-15T18:08:10.696 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-15T18:08:10.696 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:10.696 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:08:10.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.694+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006bfb0 msgr2=0x7fc0b006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.694+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006bfb0 0x7fc0b006e470 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0036000 tx=0x7fc0c003c040 comp rx=0 tx=0).stop 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.694+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d806d120 msgr2=0x7fc0d80c3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.694+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d806d120 0x7fc0d80c3e50 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800efc0 tx=0x7fc0c800c490 comp rx=0 tx=0).stop 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.695+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 shutdown_connections 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.695+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0d80ae5a0 0x7fc0d814b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.695+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc0b006bfb0 0x7fc0b006e470 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.695+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0d80b3ae0 0x7fc0d815ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.695+0000 7fc0dfa00640 1 --2- 172.21.15.26:0/3458816487 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0d806d120 0x7fc0d80c3e50 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.695+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 >> 172.21.15.26:0/3458816487 conn(0x7fc0d806a800 msgr2=0x7fc0d80b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.695+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 shutdown_connections 2024-01-15T18:08:10.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.695+0000 7fc0dfa00640 1 -- 172.21.15.26:0/3458816487 wait complete. 2024-01-15T18:08:10.719 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:08:10.719 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:08:10.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.947+0000 7f3d4b200640 1 Processor -- start 2024-01-15T18:08:10.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.948+0000 7f3d4b200640 1 -- start start 2024-01-15T18:08:10.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.950+0000 7f3d4b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d4c071410 0x7f3d4c146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.950+0000 7f3d4b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c071f00 0x7f3d4c166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.950+0000 7f3d4b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d4c0b2e10 con 0x7f3d4c06c7c0 2024-01-15T18:08:10.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.950+0000 7f3d4b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d4c06cc50 con 0x7f3d4c071410 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.950+0000 7f3d4b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c06cec0 con 0x7f3d4c071f00 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.950+0000 7f3d49e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d4c071410 0x7f3d4c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.950+0000 7f3d43e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3d4c06c7c0 0x7f3d4c0717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36492/0 (socket says 172.21.15.26:36492) 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.950+0000 7f3d49e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d4c071410 0x7f3d4c146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50188/0 (socket says 172.21.15.26:50188) 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.950+0000 7f3d43e00640 1 -- 172.21.15.26:0/3833512721 learned_addr learned my addr 172.21.15.26:0/3833512721 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.951+0000 7f3d4a800640 1 --2- 172.21.15.26:0/3833512721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c071f00 0x7f3d4c166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.951+0000 7f3d49e00640 1 -- 172.21.15.26:0/3833512721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c071f00 msgr2=0x7f3d4c166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.951+0000 7f3d49e00640 1 --2- 172.21.15.26:0/3833512721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c071f00 0x7f3d4c166cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.951+0000 7f3d49e00640 1 -- 172.21.15.26:0/3833512721 >> v1:172.21.15.26:6789/0 conn(0x7f3d4c06c7c0 legacy=0x7f3d4c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.951+0000 7f3d49e00640 1 -- 172.21.15.26:0/3833512721 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c06d1a0 con 0x7f3d4c071410 2024-01-15T18:08:10.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.951+0000 7f3d49400640 1 -- 172.21.15.26:0/3833512721 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137746969 0 0) 0x7f3d4c0b2e10 con 0x7f3d4c06c7c0 2024-01-15T18:08:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.952+0000 7f3d49e00640 1 --2- 172.21.15.26:0/3833512721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071410 0x7f3d4c146770 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0097a0 tx=0x7f3d3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36041893b5c51b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d49400640 1 -- 172.21.15.26:0/3833512721 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c031830 con 0x7f3d4c071410 2024-01-15T18:08:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d49400640 1 -- 172.21.15.26:0/3833512721 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d3c031990 con 0x7f3d4c071410 2024-01-15T18:08:10.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d49400640 1 -- 172.21.15.26:0/3833512721 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d3c031d20 con 0x7f3d4c071410 2024-01-15T18:08:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d4b200640 1 -- 172.21.15.26:0/3833512721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071410 msgr2=0x7f3d4c146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d4b200640 1 --2- 172.21.15.26:0/3833512721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071410 0x7f3d4c146770 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c0097a0 tx=0x7f3d3c02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d4b200640 1 -- 172.21.15.26:0/3833512721 shutdown_connections 2024-01-15T18:08:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d4b200640 1 --2- 172.21.15.26:0/3833512721 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c071f00 0x7f3d4c166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d4b200640 1 --2- 172.21.15.26:0/3833512721 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071410 0x7f3d4c146770 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d4b200640 1 -- 172.21.15.26:0/3833512721 >> 172.21.15.26:0/3833512721 conn(0x7f3d4c06a800 msgr2=0x7f3d4c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d4b200640 1 -- 172.21.15.26:0/3833512721 shutdown_connections 2024-01-15T18:08:10.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.953+0000 7f3d4b200640 1 -- 172.21.15.26:0/3833512721 wait complete. 2024-01-15T18:08:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.955+0000 7f3d4b200640 1 Processor -- start 2024-01-15T18:08:10.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.956+0000 7f3d4b200640 1 -- start start 2024-01-15T18:08:10.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d4b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c06c7c0 0x7f3d4c13a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d4b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d4c071410 0x7f3d4c13d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d4b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071f00 0x7f3d4c13dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d4b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c139320 con 0x7f3d4c071f00 2024-01-15T18:08:10.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d4b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c14d560 con 0x7f3d4c071410 2024-01-15T18:08:10.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d4b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c14d7b0 con 0x7f3d4c06c7c0 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d4a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071f00 0x7f3d4c13dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d4a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071f00 0x7f3d4c13dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50192/0 (socket says 172.21.15.26:50192) 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d4a800640 1 -- 172.21.15.26:0/812403753 learned_addr learned my addr 172.21.15.26:0/812403753 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d43e00640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d4c071410 0x7f3d4c13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.957+0000 7f3d49e00640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c06c7c0 0x7f3d4c13a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d4a800640 1 -- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c06c7c0 msgr2=0x7f3d4c13a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d4a800640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c06c7c0 0x7f3d4c13a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d4a800640 1 -- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d4c071410 msgr2=0x7f3d4c13d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d4a800640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d4c071410 0x7f3d4c13d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d4a800640 1 -- 172.21.15.26:0/812403753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3c009400 con 0x7f3d4c071f00 2024-01-15T18:08:10.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d49e00640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c06c7c0 0x7f3d4c13a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:10.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d4a800640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071f00 0x7f3d4c13dc80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400cc60 tx=0x7f3d4400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:10.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d42000640 1 -- 172.21.15.26:0/812403753 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4400ce70 con 0x7f3d4c071f00 2024-01-15T18:08:10.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d42000640 1 -- 172.21.15.26:0/812403753 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d44003990 con 0x7f3d4c071f00 2024-01-15T18:08:10.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d42000640 1 -- 172.21.15.26:0/812403753 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d44003d20 con 0x7f3d4c071f00 2024-01-15T18:08:10.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.958+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d4c14da90 con 0x7f3d4c071f00 2024-01-15T18:08:10.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.959+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d4c14e050 con 0x7f3d4c071f00 2024-01-15T18:08:10.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.960+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d4c14e270 con 0x7f3d4c071f00 2024-01-15T18:08:10.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.961+0000 7f3d42000640 1 -- 172.21.15.26:0/812403753 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f3d44003e80 con 0x7f3d4c071f00 2024-01-15T18:08:10.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.962+0000 7f3d42000640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d2406bfb0 0x7f3d2406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:10.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.962+0000 7f3d42000640 1 -- 172.21.15.26:0/812403753 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f3d44088940 con 0x7f3d4c071f00 2024-01-15T18:08:10.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.963+0000 7f3d49e00640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d2406bfb0 0x7f3d2406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:10.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.964+0000 7f3d49e00640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d2406bfb0 0x7f3d2406e470 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c036000 tx=0x7f3d3c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:10.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:10.967+0000 7f3d42000640 1 -- 172.21.15.26:0/812403753 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d44055260 con 0x7f3d4c071f00 2024-01-15T18:08:11.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:11.169+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3d4c067450 con 0x7f3d4c071f00 2024-01-15T18:08:12.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.155+0000 7f3d42000640 1 -- 172.21.15.26:0/812403753 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3d4401d020 con 0x7f3d4c071f00 2024-01-15T18:08:12.160 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-15T18:08:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d2406bfb0 msgr2=0x7f3d2406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d2406bfb0 0x7f3d2406e470 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f3d3c036000 tx=0x7f3d3c008030 comp rx=0 tx=0).stop 2024-01-15T18:08:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071f00 msgr2=0x7f3d4c13dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071f00 0x7f3d4c13dc80 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400cc60 tx=0x7f3d4400a5c0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 shutdown_connections 2024-01-15T18:08:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d2406bfb0 0x7f3d2406e470 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d4c071f00 0x7f3d4c13dc80 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d4c071410 0x7f3d4c13d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 --2- 172.21.15.26:0/812403753 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d4c06c7c0 0x7f3d4c13a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.159+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 >> 172.21.15.26:0/812403753 conn(0x7f3d4c06a800 msgr2=0x7f3d4c0aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:12.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.160+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 shutdown_connections 2024-01-15T18:08:12.164 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.160+0000 7f3d4b200640 1 -- 172.21.15.26:0/812403753 wait complete. 2024-01-15T18:08:12.192 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:08:12.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.395+0000 7f8027800640 1 Processor -- start 2024-01-15T18:08:12.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.395+0000 7f8027800640 1 -- start start 2024-01-15T18:08:12.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.396+0000 7f8027800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802015ed20 0x7f8020141360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:12.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.396+0000 7f8027800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f802006c850 0x7f802006cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:12.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.396+0000 7f8027800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802013e440 con 0x7f802015f100 2024-01-15T18:08:12.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.396+0000 7f8027800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80200ce1d0 con 0x7f802006c850 2024-01-15T18:08:12.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.396+0000 7f8027800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80200ce4b0 con 0x7f802015ed20 2024-01-15T18:08:12.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.397+0000 7f8025800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f802006c850 0x7f802006cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:12.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.397+0000 7f8025800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f802006c850 0x7f802006cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:42644/0 (socket says 172.21.15.26:42644) 2024-01-15T18:08:12.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.397+0000 7f8025800640 1 -- 172.21.15.26:0/200563660 learned_addr learned my addr 172.21.15.26:0/200563660 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:12.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.397+0000 7f8024e00640 1 --2- 172.21.15.26:0/200563660 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802015ed20 0x7f8020141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:12.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.398+0000 7f8024e00640 1 -- 172.21.15.26:0/200563660 >> v2:172.21.15.124:3300/0 conn(0x7f802006c850 msgr2=0x7f802006cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:12.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.398+0000 7f8024e00640 1 --2- 172.21.15.26:0/200563660 >> v2:172.21.15.124:3300/0 conn(0x7f802006c850 0x7f802006cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.398+0000 7f8024e00640 1 -- 172.21.15.26:0/200563660 >> v1:172.21.15.124:6789/0 conn(0x7f802015f100 legacy=0x7f80200cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:12.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.398+0000 7f8024e00640 1 -- 172.21.15.26:0/200563660 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80200ce790 con 0x7f802015ed20 2024-01-15T18:08:12.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.400+0000 7f8024e00640 1 --2- 172.21.15.26:0/200563660 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802015ed20 0x7f8020141360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80100097a0 tx=0x7f801002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894f7c785dadfcb9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:12.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.400+0000 7f801fe00640 1 -- 172.21.15.26:0/200563660 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010031780 con 0x7f802015ed20 2024-01-15T18:08:12.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.400+0000 7f801fe00640 1 -- 172.21.15.26:0/200563660 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80100318e0 con 0x7f802015ed20 2024-01-15T18:08:12.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.400+0000 7f801fe00640 1 -- 172.21.15.26:0/200563660 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8010031c70 con 0x7f802015ed20 2024-01-15T18:08:12.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.401+0000 7f8027800640 1 -- 172.21.15.26:0/200563660 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802015ed20 msgr2=0x7f8020141360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:12.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.401+0000 7f8027800640 1 --2- 172.21.15.26:0/200563660 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802015ed20 0x7f8020141360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f80100097a0 tx=0x7f801002b820 comp rx=0 tx=0).stop 2024-01-15T18:08:12.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.401+0000 7f8027800640 1 -- 172.21.15.26:0/200563660 shutdown_connections 2024-01-15T18:08:12.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.401+0000 7f8027800640 1 --2- 172.21.15.26:0/200563660 >> v2:172.21.15.124:3300/0 conn(0x7f802006c850 0x7f802006cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.401+0000 7f8027800640 1 --2- 172.21.15.26:0/200563660 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802015ed20 0x7f8020141360 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.401+0000 7f8027800640 1 -- 172.21.15.26:0/200563660 >> 172.21.15.26:0/200563660 conn(0x7f80200b0be0 msgr2=0x7f80200b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:12.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.401+0000 7f8027800640 1 -- 172.21.15.26:0/200563660 shutdown_connections 2024-01-15T18:08:12.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.401+0000 7f8027800640 1 -- 172.21.15.26:0/200563660 wait complete. 2024-01-15T18:08:12.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.403+0000 7f8027800640 1 Processor -- start 2024-01-15T18:08:12.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.403+0000 7f8027800640 1 -- start start 2024-01-15T18:08:12.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.404+0000 7f8027800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802006c850 0x7f80201f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.404+0000 7f8027800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f802015ed20 0x7f80200cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.404+0000 7f8027800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802015f100 0x7f80200cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.404+0000 7f8027800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8020140870 con 0x7f802015ed20 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.404+0000 7f8027800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802013fcf0 con 0x7f802015f100 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.404+0000 7f8027800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802013ffd0 con 0x7f802006c850 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8017e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f802015ed20 0x7f80200cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8017e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f802015ed20 0x7f80200cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60658/0 (socket says 172.21.15.26:60658) 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8017e00640 1 -- 172.21.15.26:0/240616772 learned_addr learned my addr 172.21.15.26:0/240616772 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8024e00640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802006c850 0x7f80201f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:12.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8025800640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802015f100 0x7f80200cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8017e00640 1 -- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802006c850 msgr2=0x7f80201f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8017e00640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802006c850 0x7f80201f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8017e00640 1 -- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802015f100 msgr2=0x7f80200cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8017e00640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802015f100 0x7f80200cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.405+0000 7f8017e00640 1 -- 172.21.15.26:0/240616772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8010009400 con 0x7f802015ed20 2024-01-15T18:08:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.406+0000 7f8017e00640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f802015ed20 0x7f80200cede0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f800800c450 tx=0x7f800800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.406+0000 7f801d600640 1 -- 172.21.15.26:0/240616772 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80080052d0 con 0x7f802015ed20 2024-01-15T18:08:12.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.406+0000 7f801d600640 1 -- 172.21.15.26:0/240616772 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8008005430 con 0x7f802015ed20 2024-01-15T18:08:12.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.406+0000 7f801d600640 1 -- 172.21.15.26:0/240616772 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8008012870 con 0x7f802015ed20 2024-01-15T18:08:12.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.406+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802015f6c0 con 0x7f802015ed20 2024-01-15T18:08:12.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.406+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8020140590 con 0x7f802015ed20 2024-01-15T18:08:12.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.407+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80200c3e50 con 0x7f802015ed20 2024-01-15T18:08:12.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.409+0000 7f801d600640 1 -- 172.21.15.26:0/240616772 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f80080129d0 con 0x7f802015ed20 2024-01-15T18:08:12.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.409+0000 7f801d600640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ff806bfb0 0x7f7ff806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:12.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.410+0000 7f801d600640 1 -- 172.21.15.26:0/240616772 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f8008088000 con 0x7f802015ed20 2024-01-15T18:08:12.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.410+0000 7f8024e00640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ff806bfb0 0x7f7ff806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:12.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.411+0000 7f8024e00640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ff806bfb0 0x7f7ff806e470 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8010036000 tx=0x7f801003d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:12.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.415+0000 7f801d600640 1 -- 172.21.15.26:0/240616772 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80200c3e50 con 0x7f802015ed20 2024-01-15T18:08:12.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:12.609+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f80200c3e50 con 0x7f802015ed20 2024-01-15T18:08:13.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.172+0000 7f801d600640 1 -- 172.21.15.26:0/240616772 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7f80200c3e50 con 0x7f802015ed20 2024-01-15T18:08:13.177 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-15T18:08:13.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.177+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ff806bfb0 msgr2=0x7f7ff806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.177+0000 7f8027800640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ff806bfb0 0x7f7ff806e470 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8010036000 tx=0x7f801003d040 comp rx=0 tx=0).stop 2024-01-15T18:08:13.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.177+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f802015ed20 msgr2=0x7f80200cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.177+0000 7f8027800640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f802015ed20 0x7f80200cede0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f800800c450 tx=0x7f800800c910 comp rx=0 tx=0).stop 2024-01-15T18:08:13.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.178+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 shutdown_connections 2024-01-15T18:08:13.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.178+0000 7f8027800640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7ff806bfb0 0x7f7ff806e470 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.178+0000 7f8027800640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f802015f100 0x7f80200cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.178+0000 7f8027800640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f802015ed20 0x7f80200cede0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.178+0000 7f8027800640 1 --2- 172.21.15.26:0/240616772 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f802006c850 0x7f80201f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.178+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 >> 172.21.15.26:0/240616772 conn(0x7f80200b0be0 msgr2=0x7f80200af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:13.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.178+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 shutdown_connections 2024-01-15T18:08:13.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.178+0000 7f8027800640 1 -- 172.21.15.26:0/240616772 wait complete. 2024-01-15T18:08:13.217 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:08:13.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.398+0000 7f75f1600640 1 Processor -- start 2024-01-15T18:08:13.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.398+0000 7f75f1600640 1 -- start start 2024-01-15T18:08:13.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.400+0000 7f75f1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ec13e360 0x7f75ec0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.400+0000 7f75f1600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec15c5f0 con 0x7f75ec13df80 2024-01-15T18:08:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.400+0000 7f75f1600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec06c470 con 0x7f75ec15f9c0 2024-01-15T18:08:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.400+0000 7f75f1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec06c750 con 0x7f75ec13e360 2024-01-15T18:08:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.400+0000 7f75eaa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f75ec13df80 0x7f75ec15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33524/0 (socket says 172.21.15.26:33524) 2024-01-15T18:08:13.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.400+0000 7f75eaa00640 1 -- 172.21.15.26:0/3898345645 learned_addr learned my addr 172.21.15.26:0/3898345645 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75e1e00640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ec13e360 0x7f75ec0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912911653 0 0) 0x7f75ec15c5f0 con 0x7f75ec13df80 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75cc003520 con 0x7f75ec13df80 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2983168029 0 0) 0x7f75cc003520 con 0x7f75ec13df80 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ec13e360 msgr2=0x7f75ec0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ec13e360 0x7f75ec0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 >> v1:172.21.15.124:6789/0 conn(0x7f75ec15f9c0 legacy=0x7f75ec0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec06ca30 con 0x7f75ec13df80 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f75dc0019f0 con 0x7f75ec13df80 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 >> v1:172.21.15.26:6789/0 conn(0x7f75ec13df80 legacy=0x7f75ec15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75cc0039a0 0x7f75cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75cc0047c0 0x7f75cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ec13e360 0x7f75cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec06c750 con 0x7f75cc0039a0 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc00ba90 con 0x7f75cc0047c0 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.401+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc00bc50 con 0x7f75ec13e360 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.402+0000 7f75eaa00640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75cc0039a0 0x7f75cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.402+0000 7f75eaa00640 1 -- 172.21.15.26:0/3898345645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ec13e360 msgr2=0x7f75cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.402+0000 7f75eaa00640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75ec13e360 0x7f75cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.402+0000 7f75eaa00640 1 -- 172.21.15.26:0/3898345645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75cc0047c0 msgr2=0x7f75cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.402+0000 7f75eaa00640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75cc0047c0 0x7f75cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.402+0000 7f75eaa00640 1 -- 172.21.15.26:0/3898345645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc003520 con 0x7f75cc0039a0 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.403+0000 7f75eaa00640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75cc0039a0 0x7f75cc003dc0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f75dc002230 tx=0x7f75cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0882bf6b5de9b05 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.403+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc034630 con 0x7f75cc0039a0 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.403+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc02f070 con 0x7f75cc0039a0 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.403+0000 7f75ea000640 1 -- 172.21.15.26:0/3898345645 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc0349d0 con 0x7f75cc0039a0 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.404+0000 7f75f1600640 1 -- 172.21.15.26:0/3898345645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75cc0039a0 msgr2=0x7f75cc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.404+0000 7f75f1600640 1 --2- 172.21.15.26:0/3898345645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75cc0039a0 0x7f75cc003dc0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f75dc002230 tx=0x7f75cc0057f0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.404+0000 7f75eb400640 1 -- 172.21.15.26:0/3898345645 reap_dead start 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.404+0000 7f75f1600640 1 -- 172.21.15.26:0/3898345645 shutdown_connections 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.404+0000 7f75f1600640 1 -- 172.21.15.26:0/3898345645 >> 172.21.15.26:0/3898345645 conn(0x7f75ec06a800 msgr2=0x7f75ec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.404+0000 7f75f1600640 1 -- 172.21.15.26:0/3898345645 shutdown_connections 2024-01-15T18:08:13.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.404+0000 7f75f1600640 1 -- 172.21.15.26:0/3898345645 wait complete. 2024-01-15T18:08:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.406+0000 7f75f1600640 1 Processor -- start 2024-01-15T18:08:13.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.406+0000 7f75f1600640 1 -- start start 2024-01-15T18:08:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.407+0000 7f75f1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75cc0039a0 0x7f75ec0b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.407+0000 7f75f1600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ec156aa0 0x7f75ec1f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.407+0000 7f75f1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec151d10 0x7f75ec1f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.407+0000 7f75f1600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec156f30 con 0x7f75ec151d10 2024-01-15T18:08:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.407+0000 7f75f1600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec157210 con 0x7f75ec156aa0 2024-01-15T18:08:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.407+0000 7f75f1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1574f0 con 0x7f75cc0039a0 2024-01-15T18:08:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.407+0000 7f75eb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec151d10 0x7f75ec1f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.407+0000 7f75eb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec151d10 0x7f75ec1f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60680/0 (socket says 172.21.15.26:60680) 2024-01-15T18:08:13.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.407+0000 7f75eb400640 1 -- 172.21.15.26:0/1941263679 learned_addr learned my addr 172.21.15.26:0/1941263679 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75eaa00640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75cc0039a0 0x7f75ec0b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75e1e00640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ec156aa0 0x7f75ec1f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75eb400640 1 -- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75cc0039a0 msgr2=0x7f75ec0b2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75eb400640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75cc0039a0 0x7f75ec0b2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75eb400640 1 -- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ec156aa0 msgr2=0x7f75ec1f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75eb400640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ec156aa0 0x7f75ec1f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75eb400640 1 -- 172.21.15.26:0/1941263679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec1577d0 con 0x7f75ec151d10 2024-01-15T18:08:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75e1e00640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ec156aa0 0x7f75ec1f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:13.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75eb400640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec151d10 0x7f75ec1f2ba0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f75ec15b6b0 tx=0x7f75e400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:13.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.408+0000 7f75e3400640 1 -- 172.21.15.26:0/1941263679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e40074e0 con 0x7f75ec151d10 2024-01-15T18:08:13.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.409+0000 7f75e3400640 1 -- 172.21.15.26:0/1941263679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e4007640 con 0x7f75ec151d10 2024-01-15T18:08:13.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.409+0000 7f75e3400640 1 -- 172.21.15.26:0/1941263679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4004260 con 0x7f75ec151d10 2024-01-15T18:08:13.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.409+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec157ab0 con 0x7f75ec151d10 2024-01-15T18:08:13.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.409+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec0b36b0 con 0x7f75ec151d10 2024-01-15T18:08:13.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.410+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec157ca0 con 0x7f75ec151d10 2024-01-15T18:08:13.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.411+0000 7f75e3400640 1 -- 172.21.15.26:0/1941263679 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f75e40057c0 con 0x7f75ec151d10 2024-01-15T18:08:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.412+0000 7f75e3400640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75c406bfb0 0x7f75c406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.412+0000 7f75e3400640 1 -- 172.21.15.26:0/1941263679 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5665+0+0 (secure 0 0 0) 0x7f75e4087660 con 0x7f75ec151d10 2024-01-15T18:08:13.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.413+0000 7f75eaa00640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75c406bfb0 0x7f75c406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:13.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.414+0000 7f75eaa00640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75c406bfb0 0x7f75c406e470 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f75dc034000 tx=0x7f75dc02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:13.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.418+0000 7f75e3400640 1 -- 172.21.15.26:0/1941263679 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75e4053f20 con 0x7f75ec151d10 2024-01-15T18:08:13.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:13.652+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f75ec0675d0 con 0x7f75ec151d10 2024-01-15T18:08:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.177+0000 7f75e3400640 1 -- 172.21.15.26:0/1941263679 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v50) v1 ==== 174+0+0 (secure 0 0 0) 0x7f75e4057540 con 0x7f75ec151d10 2024-01-15T18:08:14.181 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 12 and data pool 13 2024-01-15T18:08:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75c406bfb0 msgr2=0x7f75c406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75c406bfb0 0x7f75c406e470 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f75dc034000 tx=0x7f75dc02f000 comp rx=0 tx=0).stop 2024-01-15T18:08:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec151d10 msgr2=0x7f75ec1f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec151d10 0x7f75ec1f2ba0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f75ec15b6b0 tx=0x7f75e400c820 comp rx=0 tx=0).stop 2024-01-15T18:08:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 shutdown_connections 2024-01-15T18:08:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75c406bfb0 0x7f75c406e470 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f75ec151d10 0x7f75ec1f2ba0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f75ec156aa0 0x7f75ec1f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 --2- 172.21.15.26:0/1941263679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f75cc0039a0 0x7f75ec0b2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.181+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 >> 172.21.15.26:0/1941263679 conn(0x7f75ec06a800 msgr2=0x7f75ec15ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.182+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 shutdown_connections 2024-01-15T18:08:14.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.182+0000 7f75f1600640 1 -- 172.21.15.26:0/1941263679 wait complete. 2024-01-15T18:08:14.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:08:14.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.404+0000 7f5d9be00640 1 Processor -- start 2024-01-15T18:08:14.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.404+0000 7f5d9be00640 1 -- start start 2024-01-15T18:08:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d9c0cc500 0x7f5d9c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5d9c06d160 0x7f5d9c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c070e50 con 0x7f5d9c143c40 2024-01-15T18:08:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c0723c0 con 0x7f5d9c0cc500 2024-01-15T18:08:14.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c0cced0 con 0x7f5d9c06d160 2024-01-15T18:08:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d9c0cc500 0x7f5d9c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d9c0cc500 0x7f5d9c0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60684/0 (socket says 172.21.15.26:60684) 2024-01-15T18:08:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9a000640 1 -- 172.21.15.26:0/115505338 learned_addr learned my addr 172.21.15.26:0/115505338 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9b400640 1 --2- 172.21.15.26:0/115505338 >> v2:172.21.15.124:3300/0 conn(0x7f5d9c06d160 0x7f5d9c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9a000640 1 -- 172.21.15.26:0/115505338 >> v2:172.21.15.124:3300/0 conn(0x7f5d9c06d160 msgr2=0x7f5d9c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9a000640 1 --2- 172.21.15.26:0/115505338 >> v2:172.21.15.124:3300/0 conn(0x7f5d9c06d160 0x7f5d9c06d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9a000640 1 -- 172.21.15.26:0/115505338 >> v1:172.21.15.26:6789/0 conn(0x7f5d9c143c40 legacy=0x7f5d9c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9a000640 1 -- 172.21.15.26:0/115505338 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c144020 con 0x7f5d9c0cc500 2024-01-15T18:08:14.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.406+0000 7f5d9b400640 1 --2- 172.21.15.26:0/115505338 >> v2:172.21.15.124:3300/0 conn(0x7f5d9c06d160 0x7f5d9c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:14.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.408+0000 7f5d9a000640 1 --2- 172.21.15.26:0/115505338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 0x7f5d9c0cc990 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c070690 tx=0x7f5d8402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4538fee8acebf64f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.408+0000 7f5d99600640 1 -- 172.21.15.26:0/115505338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8402ca80 con 0x7f5d9c0cc500 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.408+0000 7f5d99600640 1 -- 172.21.15.26:0/115505338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8402cbe0 con 0x7f5d9c0cc500 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.408+0000 7f5d99600640 1 -- 172.21.15.26:0/115505338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d840343f0 con 0x7f5d9c0cc500 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.408+0000 7f5d9be00640 1 -- 172.21.15.26:0/115505338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 msgr2=0x7f5d9c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.408+0000 7f5d9be00640 1 --2- 172.21.15.26:0/115505338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 0x7f5d9c0cc990 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c070690 tx=0x7f5d8402bee0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.409+0000 7f5d9be00640 1 -- 172.21.15.26:0/115505338 shutdown_connections 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.409+0000 7f5d9be00640 1 --2- 172.21.15.26:0/115505338 >> v2:172.21.15.124:3300/0 conn(0x7f5d9c06d160 0x7f5d9c06d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.409+0000 7f5d9be00640 1 --2- 172.21.15.26:0/115505338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 0x7f5d9c0cc990 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.409+0000 7f5d9be00640 1 -- 172.21.15.26:0/115505338 >> 172.21.15.26:0/115505338 conn(0x7f5d9c06a800 msgr2=0x7f5d9c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.409+0000 7f5d9be00640 1 -- 172.21.15.26:0/115505338 shutdown_connections 2024-01-15T18:08:14.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.409+0000 7f5d9be00640 1 -- 172.21.15.26:0/115505338 wait complete. 2024-01-15T18:08:14.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.410+0000 7f5d9be00640 1 Processor -- start 2024-01-15T18:08:14.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.411+0000 7f5d9be00640 1 -- start start 2024-01-15T18:08:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.411+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d9c06d160 0x7f5d9c0af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.411+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 0x7f5d9c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.411+0000 7f5d9be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c143c40 0x7f5d9c0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.411+0000 7f5d9be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c13edf0 con 0x7f5d9c0cc500 2024-01-15T18:08:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0c8180 con 0x7f5d9c143c40 2024-01-15T18:08:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0b2440 con 0x7f5d9c06d160 2024-01-15T18:08:14.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 0x7f5d9c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 0x7f5d9c0b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60698/0 (socket says 172.21.15.26:60698) 2024-01-15T18:08:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9a000640 1 -- 172.21.15.26:0/1084364311 learned_addr learned my addr 172.21.15.26:0/1084364311 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9b400640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c143c40 0x7f5d9c0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9a000640 1 -- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d9c06d160 msgr2=0x7f5d9c0af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9a000640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d9c06d160 0x7f5d9c0af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9a000640 1 -- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c143c40 msgr2=0x7f5d9c0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9a000640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c143c40 0x7f5d9c0c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.412+0000 7f5d9a000640 1 -- 172.21.15.26:0/1084364311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84009b30 con 0x7f5d9c0cc500 2024-01-15T18:08:14.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.413+0000 7f5d9a000640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 0x7f5d9c0b1f00 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000f80 tx=0x7f5d84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:14.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.413+0000 7f5d92a00640 1 -- 172.21.15.26:0/1084364311 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8403e450 con 0x7f5d9c0cc500 2024-01-15T18:08:14.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.413+0000 7f5d92a00640 1 -- 172.21.15.26:0/1084364311 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8403e5d0 con 0x7f5d9c0cc500 2024-01-15T18:08:14.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.413+0000 7f5d92a00640 1 -- 172.21.15.26:0/1084364311 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8403e980 con 0x7f5d9c0cc500 2024-01-15T18:08:14.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.413+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c0c83d0 con 0x7f5d9c0cc500 2024-01-15T18:08:14.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.413+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c0c8990 con 0x7f5d9c0cc500 2024-01-15T18:08:14.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.414+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c067450 con 0x7f5d9c0cc500 2024-01-15T18:08:14.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.415+0000 7f5d92a00640 1 -- 172.21.15.26:0/1084364311 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f5d8402cd50 con 0x7f5d9c0cc500 2024-01-15T18:08:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.416+0000 7f5d92a00640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d7406c000 0x7f5d7406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.416+0000 7f5d92a00640 1 -- 172.21.15.26:0/1084364311 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5d840ab8b0 con 0x7f5d9c0cc500 2024-01-15T18:08:14.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.417+0000 7f5d9aa00640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d7406c000 0x7f5d7406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.418+0000 7f5d9aa00640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d7406c000 0x7f5d7406e4c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000f60 tx=0x7f5d8c0079b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:14.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.421+0000 7f5d92a00640 1 -- 172.21.15.26:0/1084364311 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d84077fc0 con 0x7f5d9c0cc500 2024-01-15T18:08:14.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.615+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d9c0ceb30 con 0x7f5d9c0cc500 2024-01-15T18:08:14.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.617+0000 7f5d92a00640 1 -- 172.21.15.26:0/1084364311 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16636 (secure 0 0 0) 0x7f5d84033e80 con 0x7f5d9c0cc500 2024-01-15T18:08:14.621 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:14.621 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":86,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:08:13.653462+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:08:11.226253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-15T18:08:12.655650+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.619+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d7406c000 msgr2=0x7f5d7406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.619+0000 7f5d9be00640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d7406c000 0x7f5d7406e4c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c000f60 tx=0x7f5d8c0079b0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.619+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 msgr2=0x7f5d9c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.619+0000 7f5d9be00640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 0x7f5d9c0b1f00 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000f80 tx=0x7f5d84004a40 comp rx=0 tx=0).stop 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.620+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 shutdown_connections 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.620+0000 7f5d9be00640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d7406c000 0x7f5d7406e4c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.620+0000 7f5d9be00640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d9c143c40 0x7f5d9c0c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.620+0000 7f5d9be00640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d9c0cc500 0x7f5d9c0b1f00 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.620+0000 7f5d9be00640 1 --2- 172.21.15.26:0/1084364311 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d9c06d160 0x7f5d9c0af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.620+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 >> 172.21.15.26:0/1084364311 conn(0x7f5d9c06a800 msgr2=0x7f5d9c06d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.620+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 shutdown_connections 2024-01-15T18:08:14.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.620+0000 7f5d9be00640 1 -- 172.21.15.26:0/1084364311 wait complete. 2024-01-15T18:08:14.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:08:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.851+0000 7f23e2e00640 1 Processor -- start 2024-01-15T18:08:14.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.852+0000 7f23e2e00640 1 -- start start 2024-01-15T18:08:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.853+0000 7f23e2e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f23dc0d0b60 0x7f23dc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.853+0000 7f23e2e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc071de0 con 0x7f23dc06a3e0 2024-01-15T18:08:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.853+0000 7f23e2e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23dc0af270 con 0x7f23dc0aede0 2024-01-15T18:08:14.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.853+0000 7f23e2e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23dc072030 con 0x7f23dc0d0b60 2024-01-15T18:08:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.854+0000 7f23dbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f23dc06a3e0 0x7f23dc06a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33542/0 (socket says 172.21.15.26:33542) 2024-01-15T18:08:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.854+0000 7f23dbe00640 1 -- 172.21.15.26:0/2116236982 learned_addr learned my addr 172.21.15.26:0/2116236982 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.854+0000 7f23db400640 1 --2- 172.21.15.26:0/2116236982 >> v2:172.21.15.26:3300/0 conn(0x7f23dc0d0b60 0x7f23dc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.854+0000 7f23db400640 1 -- 172.21.15.26:0/2116236982 >> v1:172.21.15.124:6789/0 conn(0x7f23dc0aede0 legacy=0x7f23dc06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.854+0000 7f23db400640 1 -- 172.21.15.26:0/2116236982 >> v1:172.21.15.26:6789/0 conn(0x7f23dc06a3e0 legacy=0x7f23dc06a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.854+0000 7f23db400640 1 -- 172.21.15.26:0/2116236982 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc072310 con 0x7f23dc0d0b60 2024-01-15T18:08:14.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.854+0000 7f23daa00640 1 -- 172.21.15.26:0/2116236982 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932855379 0 0) 0x7f23dc071de0 con 0x7f23dc06a3e0 2024-01-15T18:08:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.856+0000 7f23db400640 1 --2- 172.21.15.26:0/2116236982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc0d0b60 0x7f23dc0ac8d0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009d90 tx=0x7f23c402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f0d0725249a06a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.856+0000 7f23daa00640 1 -- 172.21.15.26:0/2116236982 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c402cc60 con 0x7f23dc0d0b60 2024-01-15T18:08:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.856+0000 7f23daa00640 1 -- 172.21.15.26:0/2116236982 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c402cdc0 con 0x7f23dc0d0b60 2024-01-15T18:08:14.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.856+0000 7f23daa00640 1 -- 172.21.15.26:0/2116236982 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4034660 con 0x7f23dc0d0b60 2024-01-15T18:08:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.857+0000 7f23e2e00640 1 -- 172.21.15.26:0/2116236982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc0d0b60 msgr2=0x7f23dc0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.857+0000 7f23e2e00640 1 --2- 172.21.15.26:0/2116236982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc0d0b60 0x7f23dc0ac8d0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009d90 tx=0x7f23c402beb0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.857+0000 7f23e2e00640 1 -- 172.21.15.26:0/2116236982 shutdown_connections 2024-01-15T18:08:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.857+0000 7f23e2e00640 1 --2- 172.21.15.26:0/2116236982 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc0d0b60 0x7f23dc0ac8d0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.857+0000 7f23e2e00640 1 -- 172.21.15.26:0/2116236982 >> 172.21.15.26:0/2116236982 conn(0x7f23dc0cfef0 msgr2=0x7f23dc0d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.857+0000 7f23e2e00640 1 -- 172.21.15.26:0/2116236982 shutdown_connections 2024-01-15T18:08:14.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.857+0000 7f23e2e00640 1 -- 172.21.15.26:0/2116236982 wait complete. 2024-01-15T18:08:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.859+0000 7f23e2e00640 1 Processor -- start 2024-01-15T18:08:14.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.859+0000 7f23e2e00640 1 -- start start 2024-01-15T18:08:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.860+0000 7f23e2e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc06a3e0 0x7f23dc1f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.860+0000 7f23e2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23dc0aede0 0x7f23dc156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.860+0000 7f23e2e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23dc0d0b60 0x7f23dc156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.860+0000 7f23e2e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc157de0 con 0x7f23dc06a3e0 2024-01-15T18:08:14.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.860+0000 7f23e2e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc157080 con 0x7f23dc0d0b60 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.860+0000 7f23e2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23dc157740 con 0x7f23dc0aede0 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.860+0000 7f23dbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc06a3e0 0x7f23dc1f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.860+0000 7f23dbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc06a3e0 0x7f23dc1f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60728/0 (socket says 172.21.15.26:60728) 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.860+0000 7f23dbe00640 1 -- 172.21.15.26:0/148394368 learned_addr learned my addr 172.21.15.26:0/148394368 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23db400640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23dc0aede0 0x7f23dc156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23e0e00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23dc0d0b60 0x7f23dc156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23dbe00640 1 -- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23dc0aede0 msgr2=0x7f23dc156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23dbe00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23dc0aede0 0x7f23dc156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23dbe00640 1 -- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23dc0d0b60 msgr2=0x7f23dc156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23dbe00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23dc0d0b60 0x7f23dc156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23dbe00640 1 -- 172.21.15.26:0/148394368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4009a70 con 0x7f23dc06a3e0 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23dbe00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc06a3e0 0x7f23dc1f6a60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00c430 tx=0x7f23cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:14.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23d3e00640 1 -- 172.21.15.26:0/148394368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0074e0 con 0x7f23dc06a3e0 2024-01-15T18:08:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23d3e00640 1 -- 172.21.15.26:0/148394368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc007660 con 0x7f23dc06a3e0 2024-01-15T18:08:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23d3e00640 1 -- 172.21.15.26:0/148394368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc004a20 con 0x7f23dc06a3e0 2024-01-15T18:08:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23dc0cf600 con 0x7f23dc06a3e0 2024-01-15T18:08:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.861+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc157a00 con 0x7f23dc06a3e0 2024-01-15T18:08:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.862+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23dc157f20 con 0x7f23dc06a3e0 2024-01-15T18:08:14.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.863+0000 7f23d3e00640 1 -- 172.21.15.26:0/148394368 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f23cc004b80 con 0x7f23dc06a3e0 2024-01-15T18:08:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.864+0000 7f23d3e00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f23b406bfb0 0x7f23b406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.865+0000 7f23d3e00640 1 -- 172.21.15.26:0/148394368 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f23cc088440 con 0x7f23dc06a3e0 2024-01-15T18:08:14.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.865+0000 7f23db400640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f23b406bfb0 0x7f23b406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:14.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.866+0000 7f23db400640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f23b406bfb0 0x7f23b406e470 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f23c4036000 tx=0x7f23c4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:14.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:14.869+0000 7f23d3e00640 1 -- 172.21.15.26:0/148394368 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23dc157f20 con 0x7f23dc06a3e0 2024-01-15T18:08:15.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.068+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23dc157f20 con 0x7f23dc06a3e0 2024-01-15T18:08:15.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.070+0000 7f23d3e00640 1 -- 172.21.15.26:0/148394368 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16636 (secure 0 0 0) 0x7f23dc157f20 con 0x7f23dc06a3e0 2024-01-15T18:08:15.074 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:15.074 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":86,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:08:13.653462+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:08:11.226253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-15T18:08:12.655650+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.072+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f23b406bfb0 msgr2=0x7f23b406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.072+0000 7f23e2e00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f23b406bfb0 0x7f23b406e470 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f23c4036000 tx=0x7f23c4030000 comp rx=0 tx=0).stop 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.072+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc06a3e0 msgr2=0x7f23dc1f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.072+0000 7f23e2e00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc06a3e0 0x7f23dc1f6a60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00c430 tx=0x7f23cc00c8f0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.072+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 shutdown_connections 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.072+0000 7f23e2e00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f23dc0aede0 0x7f23dc156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.072+0000 7f23e2e00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f23b406bfb0 0x7f23b406e470 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.072+0000 7f23e2e00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f23dc0d0b60 0x7f23dc156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.073+0000 7f23e2e00640 1 --2- 172.21.15.26:0/148394368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f23dc06a3e0 0x7f23dc1f6a60 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.073+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 >> 172.21.15.26:0/148394368 conn(0x7f23dc0cfef0 msgr2=0x7f23dc15c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.073+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 shutdown_connections 2024-01-15T18:08:15.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.073+0000 7f23e2e00640 1 -- 172.21.15.26:0/148394368 wait complete. 2024-01-15T18:08:15.096 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:08:15.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.322+0000 7ff2aa800640 1 Processor -- start 2024-01-15T18:08:15.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.322+0000 7ff2aa800640 1 -- start start 2024-01-15T18:08:15.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2aa800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2a415d4b0 0x7ff2a40d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2aa800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff2a40cf9a0 0x7ff2a40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2aa800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2a40cddf0 0x7ff2a40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2aa800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40d0320 con 0x7ff2a415d4b0 2024-01-15T18:08:15.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2aa800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40b2730 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2aa800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40d1ec0 con 0x7ff2a40cddf0 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2a9400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2a415d4b0 0x7ff2a40d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2a9400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff2a415d4b0 0x7ff2a40d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60742/0 (socket says 172.21.15.26:60742) 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2a9400640 1 -- 172.21.15.26:0/4165824813 learned_addr learned my addr 172.21.15.26:0/4165824813 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2a9400640 1 -- 172.21.15.26:0/4165824813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2a40cddf0 msgr2=0x7ff2a40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2a8a00640 1 --2- 172.21.15.26:0/4165824813 >> v2:172.21.15.124:3300/0 conn(0x7ff2a40cf9a0 0x7ff2a40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2a9400640 1 --2- 172.21.15.26:0/4165824813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2a40cddf0 0x7ff2a40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2a9400640 1 -- 172.21.15.26:0/4165824813 >> v2:172.21.15.124:3300/0 conn(0x7ff2a40cf9a0 msgr2=0x7ff2a40cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2a9400640 1 --2- 172.21.15.26:0/4165824813 >> v2:172.21.15.124:3300/0 conn(0x7ff2a40cf9a0 0x7ff2a40cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.324+0000 7ff2a9400640 1 -- 172.21.15.26:0/4165824813 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40b29d0 con 0x7ff2a415d4b0 2024-01-15T18:08:15.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.325+0000 7ff2a8a00640 1 --2- 172.21.15.26:0/4165824813 >> v2:172.21.15.124:3300/0 conn(0x7ff2a40cf9a0 0x7ff2a40cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.326+0000 7ff2a9400640 1 --2- 172.21.15.26:0/4165824813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a415d4b0 0x7ff2a40d1550 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff2a408b580 tx=0x7ff28c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cf8f777c6c5236 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.326+0000 7ff2a3e00640 1 -- 172.21.15.26:0/4165824813 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c031870 con 0x7ff2a415d4b0 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.326+0000 7ff2a3e00640 1 -- 172.21.15.26:0/4165824813 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c0319d0 con 0x7ff2a415d4b0 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.326+0000 7ff2a3e00640 1 -- 172.21.15.26:0/4165824813 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c031d60 con 0x7ff2a415d4b0 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.327+0000 7ff2aa800640 1 -- 172.21.15.26:0/4165824813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a415d4b0 msgr2=0x7ff2a40d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.327+0000 7ff2aa800640 1 --2- 172.21.15.26:0/4165824813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a415d4b0 0x7ff2a40d1550 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff2a408b580 tx=0x7ff28c02b910 comp rx=0 tx=0).stop 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.327+0000 7ff2aa800640 1 -- 172.21.15.26:0/4165824813 shutdown_connections 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.327+0000 7ff2aa800640 1 --2- 172.21.15.26:0/4165824813 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2a40cddf0 0x7ff2a40ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.327+0000 7ff2aa800640 1 --2- 172.21.15.26:0/4165824813 >> v2:172.21.15.124:3300/0 conn(0x7ff2a40cf9a0 0x7ff2a40cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.327+0000 7ff2aa800640 1 --2- 172.21.15.26:0/4165824813 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a415d4b0 0x7ff2a40d1550 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.327+0000 7ff2aa800640 1 -- 172.21.15.26:0/4165824813 >> 172.21.15.26:0/4165824813 conn(0x7ff2a40d3000 msgr2=0x7ff2a40d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.327+0000 7ff2aa800640 1 -- 172.21.15.26:0/4165824813 shutdown_connections 2024-01-15T18:08:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.327+0000 7ff2aa800640 1 -- 172.21.15.26:0/4165824813 wait complete. 2024-01-15T18:08:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.329+0000 7ff2aa800640 1 Processor -- start 2024-01-15T18:08:15.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.329+0000 7ff2aa800640 1 -- start start 2024-01-15T18:08:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2aa800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2a40cddf0 0x7ff2a40b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2aa800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a40cf9a0 0x7ff2a409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2aa800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2a415d4b0 0x7ff2a4153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2aa800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40b6df0 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2aa800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40b70d0 con 0x7ff2a40cddf0 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2aa800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40b73b0 con 0x7ff2a415d4b0 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2a8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a40cf9a0 0x7ff2a409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2a8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a40cf9a0 0x7ff2a409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60752/0 (socket says 172.21.15.26:60752) 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2a8a00640 1 -- 172.21.15.26:0/3748844574 learned_addr learned my addr 172.21.15.26:0/3748844574 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2a9400640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2a40cddf0 0x7ff2a40b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.330+0000 7ff2a9e00640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2a415d4b0 0x7ff2a4153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a8a00640 1 -- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2a415d4b0 msgr2=0x7ff2a4153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a8a00640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2a415d4b0 0x7ff2a4153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a8a00640 1 -- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2a40cddf0 msgr2=0x7ff2a40b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a8a00640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2a40cddf0 0x7ff2a40b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a8a00640 1 -- 172.21.15.26:0/3748844574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c0094d0 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a9400640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2a40cddf0 0x7ff2a40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:15.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a8a00640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a40cf9a0 0x7ff2a409ec20 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff29400cf60 tx=0x7ff29400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:15.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a1600640 1 -- 172.21.15.26:0/3748844574 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400d840 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a1600640 1 -- 172.21.15.26:0/3748844574 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29400d9a0 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2a1600640 1 -- 172.21.15.26:0/3748844574 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29400dd30 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a415da10 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.331+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40b7970 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.333+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a4067450 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.334+0000 7ff2a1600640 1 -- 172.21.15.26:0/3748844574 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7ff294015440 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.335+0000 7ff2a1600640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff27c06bfb0 0x7ff27c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:15.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.335+0000 7ff2a1600640 1 -- 172.21.15.26:0/3748844574 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff294088fd0 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.335+0000 7ff2a9400640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff27c06bfb0 0x7ff27c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:15.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.336+0000 7ff2a9400640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff27c06bfb0 0x7ff27c06e470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff28c036000 tx=0x7ff28c03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:15.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.339+0000 7ff2a1600640 1 -- 172.21.15.26:0/3748844574 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff294092080 con 0x7ff2a40cf9a0 2024-01-15T18:08:15.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:15.572+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff2a40b3b60 con 0x7ff2a40cf9a0 2024-01-15T18:08:16.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.216+0000 7ff2a1600640 1 -- 172.21.15.26:0/3748844574 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff29401a030 con 0x7ff2a40cf9a0 2024-01-15T18:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff27c06bfb0 msgr2=0x7ff27c06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:16.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff27c06bfb0 0x7ff27c06e470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff28c036000 tx=0x7ff28c03c040 comp rx=0 tx=0).stop 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a40cf9a0 msgr2=0x7ff2a409ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a40cf9a0 0x7ff2a409ec20 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff29400cf60 tx=0x7ff29400cf90 comp rx=0 tx=0).stop 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 shutdown_connections 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff2a415d4b0 0x7ff2a4153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff27c06bfb0 0x7ff27c06e470 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff2a40cf9a0 0x7ff2a409ec20 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 --2- 172.21.15.26:0/3748844574 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff2a40cddf0 0x7ff2a40b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.219+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 >> 172.21.15.26:0/3748844574 conn(0x7ff2a40d3000 msgr2=0x7ff2a40b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.220+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 shutdown_connections 2024-01-15T18:08:16.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.220+0000 7ff2aa800640 1 -- 172.21.15.26:0/3748844574 wait complete. 2024-01-15T18:08:16.244 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:08:16.245 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:08:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.464+0000 7f026fe00640 1 Processor -- start 2024-01-15T18:08:16.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.464+0000 7f026fe00640 1 -- start start 2024-01-15T18:08:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.465+0000 7f026fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700adf20 0x7f0270070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.465+0000 7f026fe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f027015bf10 0x7f02700cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.466+0000 7f026fe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f027015f4e0 con 0x7f02700cc980 2024-01-15T18:08:16.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.466+0000 7f026fe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027015f970 con 0x7f027015bf10 2024-01-15T18:08:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.466+0000 7f026fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02700cce10 con 0x7f02700adf20 2024-01-15T18:08:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.466+0000 7f026ea00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700adf20 0x7f0270070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.466+0000 7f026ea00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700adf20 0x7f0270070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:35322/0 (socket says 172.21.15.26:35322) 2024-01-15T18:08:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.466+0000 7f026ea00640 1 -- 172.21.15.26:0/12930556 learned_addr learned my addr 172.21.15.26:0/12930556 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:16.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.466+0000 7f026e000640 1 --2- 172.21.15.26:0/12930556 >> v2:172.21.15.124:3300/0 conn(0x7f027015bf10 0x7f02700cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.467+0000 7f026ea00640 1 -- 172.21.15.26:0/12930556 >> v2:172.21.15.124:3300/0 conn(0x7f027015bf10 msgr2=0x7f02700cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.467+0000 7f026ea00640 1 --2- 172.21.15.26:0/12930556 >> v2:172.21.15.124:3300/0 conn(0x7f027015bf10 0x7f02700cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.467+0000 7f026ea00640 1 -- 172.21.15.26:0/12930556 >> v1:172.21.15.124:6789/0 conn(0x7f02700cc980 legacy=0x7f0270143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:16.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.467+0000 7f026ea00640 1 -- 172.21.15.26:0/12930556 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0270071130 con 0x7f02700adf20 2024-01-15T18:08:16.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.469+0000 7f026ea00640 1 --2- 172.21.15.26:0/12930556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700adf20 0x7f0270070a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0260009870 tx=0x7f026002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e29bf2b94e73480 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.469+0000 7f026d600640 1 -- 172.21.15.26:0/12930556 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260031640 con 0x7f02700adf20 2024-01-15T18:08:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.469+0000 7f026d600640 1 -- 172.21.15.26:0/12930556 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02600317a0 con 0x7f02700adf20 2024-01-15T18:08:16.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.469+0000 7f026d600640 1 -- 172.21.15.26:0/12930556 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260031b30 con 0x7f02700adf20 2024-01-15T18:08:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.470+0000 7f026fe00640 1 -- 172.21.15.26:0/12930556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700adf20 msgr2=0x7f0270070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.470+0000 7f026fe00640 1 --2- 172.21.15.26:0/12930556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700adf20 0x7f0270070a30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0260009870 tx=0x7f026002b8f0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.470+0000 7f026fe00640 1 -- 172.21.15.26:0/12930556 shutdown_connections 2024-01-15T18:08:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.470+0000 7f026fe00640 1 --2- 172.21.15.26:0/12930556 >> v2:172.21.15.124:3300/0 conn(0x7f027015bf10 0x7f02700cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.470+0000 7f026fe00640 1 --2- 172.21.15.26:0/12930556 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700adf20 0x7f0270070a30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.470+0000 7f026fe00640 1 -- 172.21.15.26:0/12930556 >> 172.21.15.26:0/12930556 conn(0x7f027006a7d0 msgr2=0x7f027006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:16.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.470+0000 7f026fe00640 1 -- 172.21.15.26:0/12930556 shutdown_connections 2024-01-15T18:08:16.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.470+0000 7f026fe00640 1 -- 172.21.15.26:0/12930556 wait complete. 2024-01-15T18:08:16.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.472+0000 7f026fe00640 1 Processor -- start 2024-01-15T18:08:16.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.472+0000 7f026fe00640 1 -- start start 2024-01-15T18:08:16.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.473+0000 7f026fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02700adf20 0x7f027013cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:16.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.473+0000 7f026fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700cc980 0x7f02701f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:16.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.473+0000 7f026fe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f027015bf10 0x7f0270140470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.473+0000 7f026fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027013c290 con 0x7f02700adf20 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.473+0000 7f026fe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02701f3610 con 0x7f027015bf10 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.473+0000 7f026fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02701409b0 con 0x7f02700cc980 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02700adf20 0x7f027013cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02700adf20 0x7f027013cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60764/0 (socket says 172.21.15.26:60764) 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026ea00640 1 -- 172.21.15.26:0/2955984828 learned_addr learned my addr 172.21.15.26:0/2955984828 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026e000640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700cc980 0x7f02701f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026ea00640 1 -- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700cc980 msgr2=0x7f02701f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026ea00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700cc980 0x7f02701f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026ea00640 1 -- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f027015bf10 msgr2=0x7f0270140470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026ea00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f027015bf10 0x7f0270140470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026ea00640 1 -- 172.21.15.26:0/2955984828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02600094d0 con 0x7f02700adf20 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026e000640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700cc980 0x7f02701f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:16.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.474+0000 7f026ea00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02700adf20 0x7f027013cef0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f02600090b0 tx=0x7f02600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.475+0000 7f0266a00640 1 -- 172.21.15.26:0/2955984828 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026003a9f0 con 0x7f02700adf20 2024-01-15T18:08:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.475+0000 7f0266a00640 1 -- 172.21.15.26:0/2955984828 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026003ab70 con 0x7f02700adf20 2024-01-15T18:08:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.475+0000 7f0266a00640 1 -- 172.21.15.26:0/2955984828 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02600443f0 con 0x7f02700adf20 2024-01-15T18:08:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.475+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027013e190 con 0x7f02700adf20 2024-01-15T18:08:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.475+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027013d860 con 0x7f02700adf20 2024-01-15T18:08:16.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.476+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0270067450 con 0x7f02700adf20 2024-01-15T18:08:16.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.477+0000 7f0266a00640 1 -- 172.21.15.26:0/2955984828 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f026002f050 con 0x7f02700adf20 2024-01-15T18:08:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.479+0000 7f0266a00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f024806bfb0 0x7f024806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:16.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.479+0000 7f0266a00640 1 -- 172.21.15.26:0/2955984828 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f02600aa600 con 0x7f02700adf20 2024-01-15T18:08:16.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.479+0000 7f026e000640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f024806bfb0 0x7f024806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:16.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.480+0000 7f026e000640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f024806bfb0 0x7f024806e470 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f027015e6e0 tx=0x7f0258008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:16.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.484+0000 7f0266a00640 1 -- 172.21.15.26:0/2955984828 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0260076c30 con 0x7f02700adf20 2024-01-15T18:08:16.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:16.723+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f02700b1c10 con 0x7f02700adf20 2024-01-15T18:08:17.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.236+0000 7f0266a00640 1 -- 172.21.15.26:0/2955984828 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f026007a250 con 0x7f02700adf20 2024-01-15T18:08:17.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f024806bfb0 msgr2=0x7f024806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f024806bfb0 0x7f024806e470 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f027015e6e0 tx=0x7f0258008040 comp rx=0 tx=0).stop 2024-01-15T18:08:17.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02700adf20 msgr2=0x7f027013cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02700adf20 0x7f027013cef0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f02600090b0 tx=0x7f02600046c0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 shutdown_connections 2024-01-15T18:08:17.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f02700cc980 0x7f02701f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f024806bfb0 0x7f024806e470 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f027015bf10 0x7f0270140470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 --2- 172.21.15.26:0/2955984828 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f02700adf20 0x7f027013cef0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.239+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 >> 172.21.15.26:0/2955984828 conn(0x7f027006a7d0 msgr2=0x7f02700cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:17.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.240+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 shutdown_connections 2024-01-15T18:08:17.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.240+0000 7f026fe00640 1 -- 172.21.15.26:0/2955984828 wait complete. 2024-01-15T18:08:17.268 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:08:17.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:08:17.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.486+0000 7fdc56200640 1 Processor -- start 2024-01-15T18:08:17.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.487+0000 7fdc56200640 1 -- start start 2024-01-15T18:08:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.488+0000 7fdc56200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc5013df80 0x7fdc5015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.488+0000 7fdc56200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdc5013e360 0x7fdc500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.488+0000 7fdc56200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc5015f9c0 0x7fdc500cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.488+0000 7fdc56200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc5006c470 con 0x7fdc5013df80 2024-01-15T18:08:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.488+0000 7fdc56200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc5006c750 con 0x7fdc5013e360 2024-01-15T18:08:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.488+0000 7fdc56200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5006ca30 con 0x7fdc5015f9c0 2024-01-15T18:08:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc4f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc5013df80 0x7fdc5015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc4f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdc5013df80 0x7fdc5015bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60766/0 (socket says 172.21.15.26:60766) 2024-01-15T18:08:17.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc4f400640 1 -- 172.21.15.26:0/2760979601 learned_addr learned my addr 172.21.15.26:0/2760979601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc46800640 1 --2- 172.21.15.26:0/2760979601 >> v2:172.21.15.124:3300/0 conn(0x7fdc5013e360 0x7fdc500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc4f400640 1 -- 172.21.15.26:0/2760979601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc5015f9c0 msgr2=0x7fdc500cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc4f400640 1 --2- 172.21.15.26:0/2760979601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc5015f9c0 0x7fdc500cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc4f400640 1 -- 172.21.15.26:0/2760979601 >> v2:172.21.15.124:3300/0 conn(0x7fdc5013e360 msgr2=0x7fdc500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc4f400640 1 --2- 172.21.15.26:0/2760979601 >> v2:172.21.15.124:3300/0 conn(0x7fdc5013e360 0x7fdc500cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc4f400640 1 -- 172.21.15.26:0/2760979601 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5006cd10 con 0x7fdc5013df80 2024-01-15T18:08:17.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.489+0000 7fdc46800640 1 --2- 172.21.15.26:0/2760979601 >> v2:172.21.15.124:3300/0 conn(0x7fdc5013e360 0x7fdc500cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.490+0000 7fdc4f400640 1 --2- 172.21.15.26:0/2760979601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5013df80 0x7fdc5015bee0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdc5006ded0 tx=0x7fdc4002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db20af931163f54 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.491+0000 7fdc4ea00640 1 -- 172.21.15.26:0/2760979601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40031860 con 0x7fdc5013df80 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.491+0000 7fdc4ea00640 1 -- 172.21.15.26:0/2760979601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc400319c0 con 0x7fdc5013df80 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.491+0000 7fdc4ea00640 1 -- 172.21.15.26:0/2760979601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40031d50 con 0x7fdc5013df80 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.492+0000 7fdc56200640 1 -- 172.21.15.26:0/2760979601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5013df80 msgr2=0x7fdc5015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.492+0000 7fdc56200640 1 --2- 172.21.15.26:0/2760979601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5013df80 0x7fdc5015bee0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdc5006ded0 tx=0x7fdc4002b900 comp rx=0 tx=0).stop 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.492+0000 7fdc56200640 1 -- 172.21.15.26:0/2760979601 shutdown_connections 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.492+0000 7fdc56200640 1 --2- 172.21.15.26:0/2760979601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc5015f9c0 0x7fdc500cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.492+0000 7fdc56200640 1 --2- 172.21.15.26:0/2760979601 >> v2:172.21.15.124:3300/0 conn(0x7fdc5013e360 0x7fdc500cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.492+0000 7fdc56200640 1 --2- 172.21.15.26:0/2760979601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5013df80 0x7fdc5015bee0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.492+0000 7fdc56200640 1 -- 172.21.15.26:0/2760979601 >> 172.21.15.26:0/2760979601 conn(0x7fdc5006a800 msgr2=0x7fdc5006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.492+0000 7fdc56200640 1 -- 172.21.15.26:0/2760979601 shutdown_connections 2024-01-15T18:08:17.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.492+0000 7fdc56200640 1 -- 172.21.15.26:0/2760979601 wait complete. 2024-01-15T18:08:17.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.494+0000 7fdc56200640 1 Processor -- start 2024-01-15T18:08:17.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.495+0000 7fdc56200640 1 -- start start 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.496+0000 7fdc56200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc5013df80 0x7fdc50139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.496+0000 7fdc56200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc5013e360 0x7fdc500b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.496+0000 7fdc56200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5015f9c0 0x7fdc5015da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.496+0000 7fdc56200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50139e60 con 0x7fdc5015f9c0 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.496+0000 7fdc56200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc500b2fc0 con 0x7fdc5013e360 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.496+0000 7fdc56200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5015c980 con 0x7fdc5013df80 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.496+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5015f9c0 0x7fdc5015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.496+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5015f9c0 0x7fdc5015da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60780/0 (socket says 172.21.15.26:60780) 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.496+0000 7fdc4fe00640 1 -- 172.21.15.26:0/1976233470 learned_addr learned my addr 172.21.15.26:0/1976233470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.497+0000 7fdc4f400640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc5013df80 0x7fdc50139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.497+0000 7fdc46800640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc5013e360 0x7fdc500b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.497+0000 7fdc4fe00640 1 -- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc5013df80 msgr2=0x7fdc50139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.497+0000 7fdc4fe00640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc5013df80 0x7fdc50139920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.497+0000 7fdc4fe00640 1 -- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc5013e360 msgr2=0x7fdc500b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.497+0000 7fdc4fe00640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc5013e360 0x7fdc500b2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.497+0000 7fdc4fe00640 1 -- 172.21.15.26:0/1976233470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc400094d0 con 0x7fdc5015f9c0 2024-01-15T18:08:17.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.497+0000 7fdc4fe00640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5015f9c0 0x7fdc5015da20 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800b9e0 tx=0x7fdc4800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.498+0000 7fdc47e00640 1 -- 172.21.15.26:0/1976233470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800cb20 con 0x7fdc5015f9c0 2024-01-15T18:08:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.498+0000 7fdc47e00640 1 -- 172.21.15.26:0/1976233470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4800cc80 con 0x7fdc5015f9c0 2024-01-15T18:08:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.498+0000 7fdc47e00640 1 -- 172.21.15.26:0/1976233470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48014680 con 0x7fdc5015f9c0 2024-01-15T18:08:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.498+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5015cc60 con 0x7fdc5015f9c0 2024-01-15T18:08:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.498+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5015d220 con 0x7fdc5015f9c0 2024-01-15T18:08:17.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.499+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc500675d0 con 0x7fdc5015f9c0 2024-01-15T18:08:17.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.500+0000 7fdc47e00640 1 -- 172.21.15.26:0/1976233470 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fdc4801d3b0 con 0x7fdc5015f9c0 2024-01-15T18:08:17.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.501+0000 7fdc47e00640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc2806bfb0 0x7fdc2806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.501+0000 7fdc47e00640 1 -- 172.21.15.26:0/1976233470 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fdc48088530 con 0x7fdc5015f9c0 2024-01-15T18:08:17.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.502+0000 7fdc4f400640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc2806bfb0 0x7fdc2806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.503+0000 7fdc4f400640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc2806bfb0 0x7fdc2806e470 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdc40036000 tx=0x7fdc4003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:17.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.505+0000 7fdc47e00640 1 -- 172.21.15.26:0/1976233470 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc48054df0 con 0x7fdc5015f9c0 2024-01-15T18:08:17.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.737+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc500b31f0 con 0x7fdc5015f9c0 2024-01-15T18:08:17.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.738+0000 7fdc47e00640 1 -- 172.21.15.26:0/1976233470 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fdc48019020 con 0x7fdc5015f9c0 2024-01-15T18:08:17.742 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:17.742 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/923394527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":923394527},{"type":"v1","addr":"172.21.15.26:6833","nonce":923394527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/350014829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":350014829},{"type":"v1","addr":"172.21.15.26:6835","nonce":350014829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":36005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/1826406197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1826406197},{"type":"v1","addr":"172.21.15.124:6836","nonce":1826406197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:08:13.654102+0000","modified":"2024-01-15T18:08:16.725399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36008":{"gid":36008,"name":"d","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2740807412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2740807412},{"type":"v1","addr":"172.21.15.124:6837","nonce":2740807412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-01-15T18:08:17.742 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 53 2024-01-15T18:08:17.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc2806bfb0 msgr2=0x7fdc2806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc2806bfb0 0x7fdc2806e470 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdc40036000 tx=0x7fdc4003c040 comp rx=0 tx=0).stop 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5015f9c0 msgr2=0x7fdc5015da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5015f9c0 0x7fdc5015da20 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800b9e0 tx=0x7fdc4800bea0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 shutdown_connections 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdc2806bfb0 0x7fdc2806e470 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdc5015f9c0 0x7fdc5015da20 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdc5013e360 0x7fdc500b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 --2- 172.21.15.26:0/1976233470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdc5013df80 0x7fdc50139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.740+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 >> 172.21.15.26:0/1976233470 conn(0x7fdc5006a800 msgr2=0x7fdc500ce530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.741+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 shutdown_connections 2024-01-15T18:08:17.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.741+0000 7fdc56200640 1 -- 172.21.15.26:0/1976233470 wait complete. 2024-01-15T18:08:17.766 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:08:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.983+0000 7f8533200640 1 Processor -- start 2024-01-15T18:08:17.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.983+0000 7f8533200640 1 -- start start 2024-01-15T18:08:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.985+0000 7f8533200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f853413cfa0 0x7f853415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.985+0000 7f8533200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85340cd530 0x7f853413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.985+0000 7f8533200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85340cdec0 con 0x7f85340cd9b0 2024-01-15T18:08:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.985+0000 7f8533200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340ce1a0 con 0x7f853413cfa0 2024-01-15T18:08:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.985+0000 7f8533200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85340ce480 con 0x7f85340cd530 2024-01-15T18:08:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.985+0000 7f8531e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f853413cfa0 0x7f853415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.985+0000 7f8531e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f853413cfa0 0x7f853415afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60790/0 (socket says 172.21.15.26:60790) 2024-01-15T18:08:17.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.985+0000 7f8531e00640 1 -- 172.21.15.26:0/1845676643 learned_addr learned my addr 172.21.15.26:0/1845676643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.986+0000 7f8531400640 1 --2- 172.21.15.26:0/1845676643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85340cd530 0x7f853413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.986+0000 7f8531e00640 1 -- 172.21.15.26:0/1845676643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85340cd530 msgr2=0x7f853413f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.986+0000 7f8531e00640 1 --2- 172.21.15.26:0/1845676643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85340cd530 0x7f853413f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.986+0000 7f8531e00640 1 -- 172.21.15.26:0/1845676643 >> v1:172.21.15.26:6789/0 conn(0x7f85340cd9b0 legacy=0x7f853415d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.986+0000 7f8531e00640 1 -- 172.21.15.26:0/1845676643 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340ce760 con 0x7f853413cfa0 2024-01-15T18:08:17.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.986+0000 7f8531400640 1 --2- 172.21.15.26:0/1845676643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85340cd530 0x7f853413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.987+0000 7f8531e00640 1 --2- 172.21.15.26:0/1845676643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853413cfa0 0x7f853415afb0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f851c009870 tx=0x7f851c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50fe6f1f9c95ede server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.987+0000 7f8530a00640 1 -- 172.21.15.26:0/1845676643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c031640 con 0x7f853413cfa0 2024-01-15T18:08:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.987+0000 7f8530a00640 1 -- 172.21.15.26:0/1845676643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c0317a0 con 0x7f853413cfa0 2024-01-15T18:08:17.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.987+0000 7f8530a00640 1 -- 172.21.15.26:0/1845676643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c031b30 con 0x7f853413cfa0 2024-01-15T18:08:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.988+0000 7f8533200640 1 -- 172.21.15.26:0/1845676643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853413cfa0 msgr2=0x7f853415afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.988+0000 7f8533200640 1 --2- 172.21.15.26:0/1845676643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853413cfa0 0x7f853415afb0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f851c009870 tx=0x7f851c02b8f0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.988+0000 7f8533200640 1 -- 172.21.15.26:0/1845676643 shutdown_connections 2024-01-15T18:08:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.988+0000 7f8533200640 1 --2- 172.21.15.26:0/1845676643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85340cd530 0x7f853413f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.988+0000 7f8533200640 1 --2- 172.21.15.26:0/1845676643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f853413cfa0 0x7f853415afb0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.988+0000 7f8533200640 1 -- 172.21.15.26:0/1845676643 >> 172.21.15.26:0/1845676643 conn(0x7f853406a800 msgr2=0x7f853406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.988+0000 7f8533200640 1 -- 172.21.15.26:0/1845676643 shutdown_connections 2024-01-15T18:08:17.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.988+0000 7f8533200640 1 -- 172.21.15.26:0/1845676643 wait complete. 2024-01-15T18:08:17.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.990+0000 7f8533200640 1 Processor -- start 2024-01-15T18:08:17.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.990+0000 7f8533200640 1 -- start start 2024-01-15T18:08:17.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8533200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340cd530 0x7f8534158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8533200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85340cd9b0 0x7f853413b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8533200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853413cfa0 0x7f8534154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8533200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853413bf60 con 0x7f85340cd530 2024-01-15T18:08:17.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8533200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853413b9a0 con 0x7f85340cd9b0 2024-01-15T18:08:17.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8533200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853413bc80 con 0x7f853413cfa0 2024-01-15T18:08:17.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340cd530 0x7f8534158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340cd530 0x7f8534158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60794/0 (socket says 172.21.15.26:60794) 2024-01-15T18:08:17.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531e00640 1 -- 172.21.15.26:0/408459507 learned_addr learned my addr 172.21.15.26:0/408459507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:17.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531400640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85340cd9b0 0x7f853413b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8532800640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853413cfa0 0x7f8534154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:17.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531e00640 1 -- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853413cfa0 msgr2=0x7f8534154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531e00640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853413cfa0 0x7f8534154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531e00640 1 -- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85340cd9b0 msgr2=0x7f853413b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531e00640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85340cd9b0 0x7f853413b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531e00640 1 -- 172.21.15.26:0/408459507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f851c0094d0 con 0x7f85340cd530 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.991+0000 7f8531400640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85340cd9b0 0x7f853413b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.992+0000 7f8531e00640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340cd530 0x7f8534158550 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f851c000c00 tx=0x7f851c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.992+0000 7f852a000640 1 -- 172.21.15.26:0/408459507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c03ac60 con 0x7f85340cd530 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.992+0000 7f852a000640 1 -- 172.21.15.26:0/408459507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f851c03ade0 con 0x7f85340cd530 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.992+0000 7f852a000640 1 -- 172.21.15.26:0/408459507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851c044630 con 0x7f85340cd530 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.992+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853414a730 con 0x7f85340cd530 2024-01-15T18:08:17.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.992+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8534158a90 con 0x7f85340cd530 2024-01-15T18:08:17.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.993+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534067450 con 0x7f85340cd530 2024-01-15T18:08:17.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.994+0000 7f852a000640 1 -- 172.21.15.26:0/408459507 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f851c02f050 con 0x7f85340cd530 2024-01-15T18:08:17.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.996+0000 7f852a000640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f850c06bfb0 0x7f850c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:17.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.996+0000 7f852a000640 1 -- 172.21.15.26:0/408459507 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f851c0aa980 con 0x7f85340cd530 2024-01-15T18:08:18.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.996+0000 7f8531400640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f850c06bfb0 0x7f850c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:18.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:17.997+0000 7f8531400640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f850c06bfb0 0x7f850c06e470 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8534139ed0 tx=0x7f8524007550 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:18.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.001+0000 7f852a000640 1 -- 172.21.15.26:0/408459507 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f851c077070 con 0x7f85340cd530 2024-01-15T18:08:18.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.192+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85340cbff0 con 0x7f85340cd530 2024-01-15T18:08:18.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.195+0000 7f852a000640 1 -- 172.21.15.26:0/408459507 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16623 (secure 0 0 0) 0x7f851c07a690 con 0x7f85340cd530 2024-01-15T18:08:18.199 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:18.199 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":87,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:08:15.197704+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:08:11.226253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-01-15T18:08:12.655650+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:08:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f850c06bfb0 msgr2=0x7f850c06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f850c06bfb0 0x7f850c06e470 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8534139ed0 tx=0x7f8524007550 comp rx=0 tx=0).stop 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340cd530 msgr2=0x7f8534158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340cd530 0x7f8534158550 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f851c000c00 tx=0x7f851c0046c0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 shutdown_connections 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f853413cfa0 0x7f8534154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f850c06bfb0 0x7f850c06e470 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85340cd9b0 0x7f853413b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 --2- 172.21.15.26:0/408459507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85340cd530 0x7f8534158550 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.197+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 >> 172.21.15.26:0/408459507 conn(0x7f853406a800 msgr2=0x7f8534150860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.198+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 shutdown_connections 2024-01-15T18:08:18.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.198+0000 7f8533200640 1 -- 172.21.15.26:0/408459507 wait complete. 2024-01-15T18:08:18.221 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:08:18.221 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:08:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.436+0000 7fa79ee00640 1 Processor -- start 2024-01-15T18:08:18.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.437+0000 7fa79ee00640 1 -- start start 2024-01-15T18:08:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.438+0000 7fa79ee00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa79813e360 0x7fa7980cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.438+0000 7fa79ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa79815f9c0 0x7fa7980cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.438+0000 7fa79ee00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79815c5f0 con 0x7fa79813df80 2024-01-15T18:08:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.438+0000 7fa79ee00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa79806c470 con 0x7fa79813e360 2024-01-15T18:08:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.438+0000 7fa79ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79806c750 con 0x7fa79815f9c0 2024-01-15T18:08:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.438+0000 7fa797400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fa79813e360 0x7fa7980cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:18.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.438+0000 7fa797e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa79813df80 0x7fa79815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33560/0 (socket says 172.21.15.26:33560) 2024-01-15T18:08:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.438+0000 7fa797e00640 1 -- 172.21.15.26:0/3974071085 learned_addr learned my addr 172.21.15.26:0/3974071085 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.438+0000 7fa797400640 1 --2- 172.21.15.26:0/3974071085 >> v2:172.21.15.26:3300/0 conn(0x7fa79813e360 0x7fa7980cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60796/0 (socket says 172.21.15.26:60796) 2024-01-15T18:08:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.439+0000 7fa797400640 1 -- 172.21.15.26:0/3974071085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa79815f9c0 msgr2=0x7fa7980cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.439+0000 7fa797400640 1 --2- 172.21.15.26:0/3974071085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa79815f9c0 0x7fa7980cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.439+0000 7fa797400640 1 -- 172.21.15.26:0/3974071085 >> v1:172.21.15.26:6789/0 conn(0x7fa79813df80 legacy=0x7fa79815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.439+0000 7fa797400640 1 -- 172.21.15.26:0/3974071085 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79806ca30 con 0x7fa79813e360 2024-01-15T18:08:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.440+0000 7fa797400640 1 --2- 172.21.15.26:0/3974071085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 0x7fa7980cf290 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fa7800097a0 tx=0x7fa78002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906b1030883ddf33 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.440+0000 7fa796a00640 1 -- 172.21.15.26:0/3974071085 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78002d040 con 0x7fa79813e360 2024-01-15T18:08:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.440+0000 7fa796a00640 1 -- 172.21.15.26:0/3974071085 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa780031810 con 0x7fa79813e360 2024-01-15T18:08:18.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.440+0000 7fa796a00640 1 -- 172.21.15.26:0/3974071085 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780031ba0 con 0x7fa79813e360 2024-01-15T18:08:18.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.441+0000 7fa79ee00640 1 -- 172.21.15.26:0/3974071085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 msgr2=0x7fa7980cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.441+0000 7fa79ee00640 1 --2- 172.21.15.26:0/3974071085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 0x7fa7980cf290 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fa7800097a0 tx=0x7fa78002b8b0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.441+0000 7fa79ee00640 1 -- 172.21.15.26:0/3974071085 shutdown_connections 2024-01-15T18:08:18.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.441+0000 7fa79ee00640 1 --2- 172.21.15.26:0/3974071085 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa79815f9c0 0x7fa7980cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.441+0000 7fa79ee00640 1 --2- 172.21.15.26:0/3974071085 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 0x7fa7980cf290 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.441+0000 7fa79ee00640 1 -- 172.21.15.26:0/3974071085 >> 172.21.15.26:0/3974071085 conn(0x7fa79806a800 msgr2=0x7fa79806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.441+0000 7fa79ee00640 1 -- 172.21.15.26:0/3974071085 shutdown_connections 2024-01-15T18:08:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.441+0000 7fa79ee00640 1 -- 172.21.15.26:0/3974071085 wait complete. 2024-01-15T18:08:18.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.443+0000 7fa79ee00640 1 Processor -- start 2024-01-15T18:08:18.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.443+0000 7fa79ee00640 1 -- start start 2024-01-15T18:08:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa79ee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa79813df80 0x7fa7981f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa79ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 0x7fa79813cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa79ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa79815f9c0 0x7fa7981e6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa79ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981f3760 con 0x7fa79813e360 2024-01-15T18:08:18.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa79ee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79813d870 con 0x7fa79813df80 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa79ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981e6bc0 con 0x7fa79815f9c0 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 0x7fa79813cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 0x7fa79813cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60800/0 (socket says 172.21.15.26:60800) 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797400640 1 -- 172.21.15.26:0/2830712065 learned_addr learned my addr 172.21.15.26:0/2830712065 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797400640 1 -- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa79815f9c0 msgr2=0x7fa7981e6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797400640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa79815f9c0 0x7fa7981e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797400640 1 -- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa79813df80 msgr2=0x7fa7981f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797e00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa79813df80 0x7fa7981f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797400640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa79813df80 0x7fa7981f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797400640 1 -- 172.21.15.26:0/2830712065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa780009400 con 0x7fa79813e360 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.445+0000 7fa797e00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa79813df80 0x7fa7981f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:18.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.446+0000 7fa797400640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 0x7fa79813cd10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa78002b8f0 tx=0x7fa7800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.446+0000 7fa78fe00640 1 -- 172.21.15.26:0/2830712065 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78003ad60 con 0x7fa79813e360 2024-01-15T18:08:18.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.446+0000 7fa78fe00640 1 -- 172.21.15.26:0/2830712065 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7800443f0 con 0x7fa79813e360 2024-01-15T18:08:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.446+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7981e6ea0 con 0x7fa79813e360 2024-01-15T18:08:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.446+0000 7fa78fe00640 1 -- 172.21.15.26:0/2830712065 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7800447a0 con 0x7fa79813e360 2024-01-15T18:08:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.446+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7981e7460 con 0x7fa79813e360 2024-01-15T18:08:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.448+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7980675d0 con 0x7fa79813e360 2024-01-15T18:08:18.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.448+0000 7fa78fe00640 1 -- 172.21.15.26:0/2830712065 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fa78004d450 con 0x7fa79813e360 2024-01-15T18:08:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.449+0000 7fa78fe00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa77006bfb0 0x7fa77006e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.450+0000 7fa78fe00640 1 -- 172.21.15.26:0/2830712065 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa7800aace0 con 0x7fa79813e360 2024-01-15T18:08:18.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.450+0000 7fa797e00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa77006bfb0 0x7fa77006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:18.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.451+0000 7fa797e00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa77006bfb0 0x7fa77006e470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa788000f60 tx=0x7fa788009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:18.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.454+0000 7fa78fe00640 1 -- 172.21.15.26:0/2830712065 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa780077310 con 0x7fa79813e360 2024-01-15T18:08:18.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.648+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa79806cff0 con 0x7fa77006bfb0 2024-01-15T18:08:18.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.675+0000 7fa78fe00640 1 -- 172.21.15.26:0/2830712065 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286531 (secure 0 0 0) 0x7fa788005890 con 0x7fa77006bfb0 2024-01-15T18:08:18.684 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:18.686 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa77006bfb0 msgr2=0x7fa77006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa77006bfb0 0x7fa77006e470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa788000f60 tx=0x7fa788009040 comp rx=0 tx=0).stop 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 msgr2=0x7fa79813cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 0x7fa79813cd10 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa78002b8f0 tx=0x7fa7800046c0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 shutdown_connections 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa79815f9c0 0x7fa7981e6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa77006bfb0 0x7fa77006e470 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa79813e360 0x7fa79813cd10 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 --2- 172.21.15.26:0/2830712065 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa79813df80 0x7fa7981f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 >> 172.21.15.26:0/2830712065 conn(0x7fa79806a800 msgr2=0x7fa7980aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 shutdown_connections 2024-01-15T18:08:18.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.682+0000 7fa79ee00640 1 -- 172.21.15.26:0/2830712065 wait complete. 2024-01-15T18:08:18.707 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-01-15T18:08:18.222939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514100,"kb_used_data":9544,"kb_used_omap":12,"kb_used_meta":504435,"kb_avail":754460620,"statfs":{"total":773094113280,"available":772567674880,"internally_reserved":0,"allocated":9773056,"data_stored":4856300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516542000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001852"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212934+0000","last_change":"2024-01-15T18:08:14.205339+0000","last_active":"2024-01-15T18:08:15.212934+0000","last_peered":"2024-01-15T18:08:15.212934+0000","last_clean":"2024-01-15T18:08:15.212934+0000","last_became_active":"2024-01-15T18:08:14.205009+0000","last_became_peered":"2024-01-15T18:08:14.205009+0000","last_unstale":"2024-01-15T18:08:15.212934+0000","last_undegraded":"2024-01-15T18:08:15.212934+0000","last_fullsized":"2024-01-15T18:08:15.212934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:35.060937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259421+0000","last_change":"2024-01-15T18:08:14.211597+0000","last_active":"2024-01-15T18:08:16.259421+0000","last_peered":"2024-01-15T18:08:16.259421+0000","last_clean":"2024-01-15T18:08:16.259421+0000","last_became_active":"2024-01-15T18:08:14.211327+0000","last_became_peered":"2024-01-15T18:08:14.211327+0000","last_unstale":"2024-01-15T18:08:16.259421+0000","last_undegraded":"2024-01-15T18:08:16.259421+0000","last_fullsized":"2024-01-15T18:08:16.259421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:40.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257946+0000","last_change":"2024-01-15T18:08:14.212643+0000","last_active":"2024-01-15T18:08:16.257946+0000","last_peered":"2024-01-15T18:08:16.257946+0000","last_clean":"2024-01-15T18:08:16.257946+0000","last_became_active":"2024-01-15T18:08:14.212269+0000","last_became_peered":"2024-01-15T18:08:14.212269+0000","last_unstale":"2024-01-15T18:08:16.257946+0000","last_undegraded":"2024-01-15T18:08:16.257946+0000","last_fullsized":"2024-01-15T18:08:16.257946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:45:02.732821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257846+0000","last_change":"2024-01-15T18:08:14.202421+0000","last_active":"2024-01-15T18:08:16.257846+0000","last_peered":"2024-01-15T18:08:16.257846+0000","last_clean":"2024-01-15T18:08:16.257846+0000","last_became_active":"2024-01-15T18:08:14.202076+0000","last_became_peered":"2024-01-15T18:08:14.202076+0000","last_unstale":"2024-01-15T18:08:16.257846+0000","last_undegraded":"2024-01-15T18:08:16.257846+0000","last_fullsized":"2024-01-15T18:08:16.257846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:52.451530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213065+0000","last_change":"2024-01-15T18:08:14.201420+0000","last_active":"2024-01-15T18:08:15.213065+0000","last_peered":"2024-01-15T18:08:15.213065+0000","last_clean":"2024-01-15T18:08:15.213065+0000","last_became_active":"2024-01-15T18:08:14.199479+0000","last_became_peered":"2024-01-15T18:08:14.199479+0000","last_unstale":"2024-01-15T18:08:15.213065+0000","last_undegraded":"2024-01-15T18:08:15.213065+0000","last_fullsized":"2024-01-15T18:08:15.213065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:02.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258511+0000","last_change":"2024-01-15T18:08:14.221652+0000","last_active":"2024-01-15T18:08:16.258511+0000","last_peered":"2024-01-15T18:08:16.258511+0000","last_clean":"2024-01-15T18:08:16.258511+0000","last_became_active":"2024-01-15T18:08:14.220744+0000","last_became_peered":"2024-01-15T18:08:14.220744+0000","last_unstale":"2024-01-15T18:08:16.258511+0000","last_undegraded":"2024-01-15T18:08:16.258511+0000","last_fullsized":"2024-01-15T18:08:16.258511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:29:34.345687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.256741+0000","last_change":"2024-01-15T18:08:14.211362+0000","last_active":"2024-01-15T18:08:16.256741+0000","last_peered":"2024-01-15T18:08:16.256741+0000","last_clean":"2024-01-15T18:08:16.256741+0000","last_became_active":"2024-01-15T18:08:14.211099+0000","last_became_peered":"2024-01-15T18:08:14.211099+0000","last_unstale":"2024-01-15T18:08:16.256741+0000","last_undegraded":"2024-01-15T18:08:16.256741+0000","last_fullsized":"2024-01-15T18:08:16.256741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:22:47.996600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258644+0000","last_change":"2024-01-15T18:08:14.220426+0000","last_active":"2024-01-15T18:08:16.258644+0000","last_peered":"2024-01-15T18:08:16.258644+0000","last_clean":"2024-01-15T18:08:16.258644+0000","last_became_active":"2024-01-15T18:08:14.219622+0000","last_became_peered":"2024-01-15T18:08:14.219622+0000","last_unstale":"2024-01-15T18:08:16.258644+0000","last_undegraded":"2024-01-15T18:08:16.258644+0000","last_fullsized":"2024-01-15T18:08:16.258644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:35.060937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259135+0000","last_change":"2024-01-15T18:08:14.213300+0000","last_active":"2024-01-15T18:08:16.259135+0000","last_peered":"2024-01-15T18:08:16.259135+0000","last_clean":"2024-01-15T18:08:16.259135+0000","last_became_active":"2024-01-15T18:08:14.213029+0000","last_became_peered":"2024-01-15T18:08:14.213029+0000","last_unstale":"2024-01-15T18:08:16.259135+0000","last_undegraded":"2024-01-15T18:08:16.259135+0000","last_fullsized":"2024-01-15T18:08:16.259135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:00:03.016857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212888+0000","last_change":"2024-01-15T18:08:14.217683+0000","last_active":"2024-01-15T18:08:15.212888+0000","last_peered":"2024-01-15T18:08:15.212888+0000","last_clean":"2024-01-15T18:08:15.212888+0000","last_became_active":"2024-01-15T18:08:14.217330+0000","last_became_peered":"2024-01-15T18:08:14.217330+0000","last_unstale":"2024-01-15T18:08:15.212888+0000","last_undegraded":"2024-01-15T18:08:15.212888+0000","last_fullsized":"2024-01-15T18:08:15.212888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:29:34.345687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259099+0000","last_change":"2024-01-15T18:08:14.212851+0000","last_active":"2024-01-15T18:08:16.259099+0000","last_peered":"2024-01-15T18:08:16.259099+0000","last_clean":"2024-01-15T18:08:16.259099+0000","last_became_active":"2024-01-15T18:08:14.212510+0000","last_became_peered":"2024-01-15T18:08:14.212510+0000","last_unstale":"2024-01-15T18:08:16.259099+0000","last_undegraded":"2024-01-15T18:08:16.259099+0000","last_fullsized":"2024-01-15T18:08:16.259099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:31.868467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210164+0000","last_change":"2024-01-15T18:08:14.216968+0000","last_active":"2024-01-15T18:08:15.210164+0000","last_peered":"2024-01-15T18:08:15.210164+0000","last_clean":"2024-01-15T18:08:15.210164+0000","last_became_active":"2024-01-15T18:08:14.215796+0000","last_became_peered":"2024-01-15T18:08:14.215796+0000","last_unstale":"2024-01-15T18:08:15.210164+0000","last_undegraded":"2024-01-15T18:08:15.210164+0000","last_fullsized":"2024-01-15T18:08:15.210164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:01:37.514417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257762+0000","last_change":"2024-01-15T18:08:14.211568+0000","last_active":"2024-01-15T18:08:16.257762+0000","last_peered":"2024-01-15T18:08:16.257762+0000","last_clean":"2024-01-15T18:08:16.257762+0000","last_became_active":"2024-01-15T18:08:14.211256+0000","last_became_peered":"2024-01-15T18:08:14.211256+0000","last_unstale":"2024-01-15T18:08:16.257762+0000","last_undegraded":"2024-01-15T18:08:16.257762+0000","last_fullsized":"2024-01-15T18:08:16.257762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:16.437633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257635+0000","last_change":"2024-01-15T18:08:14.201090+0000","last_active":"2024-01-15T18:08:16.257635+0000","last_peered":"2024-01-15T18:08:16.257635+0000","last_clean":"2024-01-15T18:08:16.257635+0000","last_became_active":"2024-01-15T18:08:14.200793+0000","last_became_peered":"2024-01-15T18:08:14.200793+0000","last_unstale":"2024-01-15T18:08:16.257635+0000","last_undegraded":"2024-01-15T18:08:16.257635+0000","last_fullsized":"2024-01-15T18:08:16.257635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:19:35.807826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.203802+0000","last_change":"2024-01-15T18:08:14.207360+0000","last_active":"2024-01-15T18:08:15.203802+0000","last_peered":"2024-01-15T18:08:15.203802+0000","last_clean":"2024-01-15T18:08:15.203802+0000","last_became_active":"2024-01-15T18:08:14.207079+0000","last_became_peered":"2024-01-15T18:08:14.207079+0000","last_unstale":"2024-01-15T18:08:15.203802+0000","last_undegraded":"2024-01-15T18:08:15.203802+0000","last_fullsized":"2024-01-15T18:08:15.203802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:51:08.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.211040+0000","last_change":"2024-01-15T18:08:14.208857+0000","last_active":"2024-01-15T18:08:15.211040+0000","last_peered":"2024-01-15T18:08:15.211040+0000","last_clean":"2024-01-15T18:08:15.211040+0000","last_became_active":"2024-01-15T18:08:14.207171+0000","last_became_peered":"2024-01-15T18:08:14.207171+0000","last_unstale":"2024-01-15T18:08:15.211040+0000","last_undegraded":"2024-01-15T18:08:15.211040+0000","last_fullsized":"2024-01-15T18:08:15.211040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:20:41.894318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212518+0000","last_change":"2024-01-15T18:08:14.201493+0000","last_active":"2024-01-15T18:08:15.212518+0000","last_peered":"2024-01-15T18:08:15.212518+0000","last_clean":"2024-01-15T18:08:15.212518+0000","last_became_active":"2024-01-15T18:08:14.200900+0000","last_became_peered":"2024-01-15T18:08:14.200900+0000","last_unstale":"2024-01-15T18:08:15.212518+0000","last_undegraded":"2024-01-15T18:08:15.212518+0000","last_fullsized":"2024-01-15T18:08:15.212518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:29:34.345687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258367+0000","last_change":"2024-01-15T18:08:14.207516+0000","last_active":"2024-01-15T18:08:16.258367+0000","last_peered":"2024-01-15T18:08:16.258367+0000","last_clean":"2024-01-15T18:08:16.258367+0000","last_became_active":"2024-01-15T18:08:14.207247+0000","last_became_peered":"2024-01-15T18:08:14.207247+0000","last_unstale":"2024-01-15T18:08:16.258367+0000","last_undegraded":"2024-01-15T18:08:16.258367+0000","last_fullsized":"2024-01-15T18:08:16.258367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:27:06.733990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.088810+0000","last_change":"2024-01-15T18:08:14.222306+0000","last_active":"2024-01-15T18:08:16.088810+0000","last_peered":"2024-01-15T18:08:16.088810+0000","last_clean":"2024-01-15T18:08:16.088810+0000","last_became_active":"2024-01-15T18:08:14.222037+0000","last_became_peered":"2024-01-15T18:08:14.222037+0000","last_unstale":"2024-01-15T18:08:16.088810+0000","last_undegraded":"2024-01-15T18:08:16.088810+0000","last_fullsized":"2024-01-15T18:08:16.088810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:38.896192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258040+0000","last_change":"2024-01-15T18:08:14.214903+0000","last_active":"2024-01-15T18:08:16.258040+0000","last_peered":"2024-01-15T18:08:16.258040+0000","last_clean":"2024-01-15T18:08:16.258040+0000","last_became_active":"2024-01-15T18:08:14.214223+0000","last_became_peered":"2024-01-15T18:08:14.214223+0000","last_unstale":"2024-01-15T18:08:16.258040+0000","last_undegraded":"2024-01-15T18:08:16.258040+0000","last_fullsized":"2024-01-15T18:08:16.258040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:06:36.312091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258073+0000","last_change":"2024-01-15T18:08:14.210307+0000","last_active":"2024-01-15T18:08:16.258073+0000","last_peered":"2024-01-15T18:08:16.258073+0000","last_clean":"2024-01-15T18:08:16.258073+0000","last_became_active":"2024-01-15T18:08:14.205910+0000","last_became_peered":"2024-01-15T18:08:14.205910+0000","last_unstale":"2024-01-15T18:08:16.258073+0000","last_undegraded":"2024-01-15T18:08:16.258073+0000","last_fullsized":"2024-01-15T18:08:16.258073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:37.149934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257695+0000","last_change":"2024-01-15T18:08:14.214724+0000","last_active":"2024-01-15T18:08:16.257695+0000","last_peered":"2024-01-15T18:08:16.257695+0000","last_clean":"2024-01-15T18:08:16.257695+0000","last_became_active":"2024-01-15T18:08:14.214219+0000","last_became_peered":"2024-01-15T18:08:14.214219+0000","last_unstale":"2024-01-15T18:08:16.257695+0000","last_undegraded":"2024-01-15T18:08:16.257695+0000","last_fullsized":"2024-01-15T18:08:16.257695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:45:29.726519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210213+0000","last_change":"2024-01-15T18:08:14.206870+0000","last_active":"2024-01-15T18:08:15.210213+0000","last_peered":"2024-01-15T18:08:15.210213+0000","last_clean":"2024-01-15T18:08:15.210213+0000","last_became_active":"2024-01-15T18:08:14.206593+0000","last_became_peered":"2024-01-15T18:08:14.206593+0000","last_unstale":"2024-01-15T18:08:15.210213+0000","last_undegraded":"2024-01-15T18:08:15.210213+0000","last_fullsized":"2024-01-15T18:08:15.210213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:20:41.894318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.256452+0000","last_change":"2024-01-15T18:08:14.208074+0000","last_active":"2024-01-15T18:08:16.256452+0000","last_peered":"2024-01-15T18:08:16.256452+0000","last_clean":"2024-01-15T18:08:16.256452+0000","last_became_active":"2024-01-15T18:08:14.203256+0000","last_became_peered":"2024-01-15T18:08:14.203256+0000","last_unstale":"2024-01-15T18:08:16.256452+0000","last_undegraded":"2024-01-15T18:08:16.256452+0000","last_fullsized":"2024-01-15T18:08:16.256452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:28.203925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259513+0000","last_change":"2024-01-15T18:08:14.201039+0000","last_active":"2024-01-15T18:08:16.259513+0000","last_peered":"2024-01-15T18:08:16.259513+0000","last_clean":"2024-01-15T18:08:16.259513+0000","last_became_active":"2024-01-15T18:08:14.200739+0000","last_became_peered":"2024-01-15T18:08:14.200739+0000","last_unstale":"2024-01-15T18:08:16.259513+0000","last_undegraded":"2024-01-15T18:08:16.259513+0000","last_fullsized":"2024-01-15T18:08:16.259513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:32:48.450924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258169+0000","last_change":"2024-01-15T18:08:14.218802+0000","last_active":"2024-01-15T18:08:16.258169+0000","last_peered":"2024-01-15T18:08:16.258169+0000","last_clean":"2024-01-15T18:08:16.258169+0000","last_became_active":"2024-01-15T18:08:14.218527+0000","last_became_peered":"2024-01-15T18:08:14.218527+0000","last_unstale":"2024-01-15T18:08:16.258169+0000","last_undegraded":"2024-01-15T18:08:16.258169+0000","last_fullsized":"2024-01-15T18:08:16.258169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:58.531844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212596+0000","last_change":"2024-01-15T18:08:14.201989+0000","last_active":"2024-01-15T18:08:15.212596+0000","last_peered":"2024-01-15T18:08:15.212596+0000","last_clean":"2024-01-15T18:08:15.212596+0000","last_became_active":"2024-01-15T18:08:14.201682+0000","last_became_peered":"2024-01-15T18:08:14.201682+0000","last_unstale":"2024-01-15T18:08:15.212596+0000","last_undegraded":"2024-01-15T18:08:15.212596+0000","last_fullsized":"2024-01-15T18:08:15.212596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:35.060937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210325+0000","last_change":"2024-01-15T18:08:14.204423+0000","last_active":"2024-01-15T18:08:15.210325+0000","last_peered":"2024-01-15T18:08:15.210325+0000","last_clean":"2024-01-15T18:08:15.210325+0000","last_became_active":"2024-01-15T18:08:14.204103+0000","last_became_peered":"2024-01-15T18:08:14.204103+0000","last_unstale":"2024-01-15T18:08:15.210325+0000","last_undegraded":"2024-01-15T18:08:15.210325+0000","last_fullsized":"2024-01-15T18:08:15.210325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:39:31.130502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257995+0000","last_change":"2024-01-15T18:08:14.204294+0000","last_active":"2024-01-15T18:08:16.257995+0000","last_peered":"2024-01-15T18:08:16.257995+0000","last_clean":"2024-01-15T18:08:16.257995+0000","last_became_active":"2024-01-15T18:08:14.204033+0000","last_became_peered":"2024-01-15T18:08:14.204033+0000","last_unstale":"2024-01-15T18:08:16.257995+0000","last_undegraded":"2024-01-15T18:08:16.257995+0000","last_fullsized":"2024-01-15T18:08:16.257995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:53.420076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257851+0000","last_change":"2024-01-15T18:08:14.205679+0000","last_active":"2024-01-15T18:08:16.257851+0000","last_peered":"2024-01-15T18:08:16.257851+0000","last_clean":"2024-01-15T18:08:16.257851+0000","last_became_active":"2024-01-15T18:08:14.203591+0000","last_became_peered":"2024-01-15T18:08:14.203591+0000","last_unstale":"2024-01-15T18:08:16.257851+0000","last_undegraded":"2024-01-15T18:08:16.257851+0000","last_fullsized":"2024-01-15T18:08:16.257851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:09:04.507538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.211290+0000","last_change":"2024-01-15T18:08:14.205902+0000","last_active":"2024-01-15T18:08:15.211290+0000","last_peered":"2024-01-15T18:08:15.211290+0000","last_clean":"2024-01-15T18:08:15.211290+0000","last_became_active":"2024-01-15T18:08:14.203710+0000","last_became_peered":"2024-01-15T18:08:14.203710+0000","last_unstale":"2024-01-15T18:08:15.211290+0000","last_undegraded":"2024-01-15T18:08:15.211290+0000","last_fullsized":"2024-01-15T18:08:15.211290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:39:31.130502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.209766+0000","last_change":"2024-01-15T18:08:14.200416+0000","last_active":"2024-01-15T18:08:15.209766+0000","last_peered":"2024-01-15T18:08:15.209766+0000","last_clean":"2024-01-15T18:08:15.209766+0000","last_became_active":"2024-01-15T18:08:14.200095+0000","last_became_peered":"2024-01-15T18:08:14.200095+0000","last_unstale":"2024-01-15T18:08:15.209766+0000","last_undegraded":"2024-01-15T18:08:15.209766+0000","last_fullsized":"2024-01-15T18:08:15.209766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:20:37.554185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.211958+0000","last_change":"2024-01-15T18:08:14.207686+0000","last_active":"2024-01-15T18:08:15.211958+0000","last_peered":"2024-01-15T18:08:15.211958+0000","last_clean":"2024-01-15T18:08:15.211958+0000","last_became_active":"2024-01-15T18:08:14.205835+0000","last_became_peered":"2024-01-15T18:08:14.205835+0000","last_unstale":"2024-01-15T18:08:15.211958+0000","last_undegraded":"2024-01-15T18:08:15.211958+0000","last_fullsized":"2024-01-15T18:08:15.211958+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:38:54.083263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.217198+0000","last_change":"2024-01-15T18:08:13.201391+0000","last_active":"2024-01-15T18:08:14.217198+0000","last_peered":"2024-01-15T18:08:14.217198+0000","last_clean":"2024-01-15T18:08:14.217198+0000","last_became_active":"2024-01-15T18:08:13.201072+0000","last_became_peered":"2024-01-15T18:08:13.201072+0000","last_unstale":"2024-01-15T18:08:14.217198+0000","last_undegraded":"2024-01-15T18:08:14.217198+0000","last_fullsized":"2024-01-15T18:08:14.217198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:31.926730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213104+0000","last_change":"2024-01-15T18:08:14.213109+0000","last_active":"2024-01-15T18:08:15.213104+0000","last_peered":"2024-01-15T18:08:15.213104+0000","last_clean":"2024-01-15T18:08:15.213104+0000","last_became_active":"2024-01-15T18:08:14.207157+0000","last_became_peered":"2024-01-15T18:08:14.207157+0000","last_unstale":"2024-01-15T18:08:15.213104+0000","last_undegraded":"2024-01-15T18:08:15.213104+0000","last_fullsized":"2024-01-15T18:08:15.213104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:33.977943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.220065+0000","last_change":"2024-01-15T18:08:13.198543+0000","last_active":"2024-01-15T18:08:14.220065+0000","last_peered":"2024-01-15T18:08:14.220065+0000","last_clean":"2024-01-15T18:08:14.220065+0000","last_became_active":"2024-01-15T18:08:13.197094+0000","last_became_peered":"2024-01-15T18:08:13.197094+0000","last_unstale":"2024-01-15T18:08:14.220065+0000","last_undegraded":"2024-01-15T18:08:14.220065+0000","last_fullsized":"2024-01-15T18:08:14.220065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:52:57.254176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210526+0000","last_change":"2024-01-15T18:08:14.217169+0000","last_active":"2024-01-15T18:08:15.210526+0000","last_peered":"2024-01-15T18:08:15.210526+0000","last_clean":"2024-01-15T18:08:15.210526+0000","last_became_active":"2024-01-15T18:08:14.216179+0000","last_became_peered":"2024-01-15T18:08:14.216179+0000","last_unstale":"2024-01-15T18:08:15.210526+0000","last_undegraded":"2024-01-15T18:08:15.210526+0000","last_fullsized":"2024-01-15T18:08:15.210526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:38:54.083263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.224457+0000","last_change":"2024-01-15T18:08:13.206368+0000","last_active":"2024-01-15T18:08:14.224457+0000","last_peered":"2024-01-15T18:08:14.224457+0000","last_clean":"2024-01-15T18:08:14.224457+0000","last_became_active":"2024-01-15T18:08:13.205180+0000","last_became_peered":"2024-01-15T18:08:13.205180+0000","last_unstale":"2024-01-15T18:08:14.224457+0000","last_undegraded":"2024-01-15T18:08:14.224457+0000","last_fullsized":"2024-01-15T18:08:14.224457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:01.034813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257928+0000","last_change":"2024-01-15T18:08:14.222292+0000","last_active":"2024-01-15T18:08:16.257928+0000","last_peered":"2024-01-15T18:08:16.257928+0000","last_clean":"2024-01-15T18:08:16.257928+0000","last_became_active":"2024-01-15T18:08:14.221624+0000","last_became_peered":"2024-01-15T18:08:14.221624+0000","last_unstale":"2024-01-15T18:08:16.257928+0000","last_undegraded":"2024-01-15T18:08:16.257928+0000","last_fullsized":"2024-01-15T18:08:16.257928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:13:07.165947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.225102+0000","last_change":"2024-01-15T18:08:13.206418+0000","last_active":"2024-01-15T18:08:14.225102+0000","last_peered":"2024-01-15T18:08:14.225102+0000","last_clean":"2024-01-15T18:08:14.225102+0000","last_became_active":"2024-01-15T18:08:13.205259+0000","last_became_peered":"2024-01-15T18:08:13.205259+0000","last_unstale":"2024-01-15T18:08:14.225102+0000","last_undegraded":"2024-01-15T18:08:14.225102+0000","last_fullsized":"2024-01-15T18:08:14.225102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:29:33.339575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257985+0000","last_change":"2024-01-15T18:08:14.205937+0000","last_active":"2024-01-15T18:08:16.257985+0000","last_peered":"2024-01-15T18:08:16.257985+0000","last_clean":"2024-01-15T18:08:16.257985+0000","last_became_active":"2024-01-15T18:08:14.205244+0000","last_became_peered":"2024-01-15T18:08:14.205244+0000","last_unstale":"2024-01-15T18:08:16.257985+0000","last_undegraded":"2024-01-15T18:08:16.257985+0000","last_fullsized":"2024-01-15T18:08:16.257985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:00:57.294299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.217464+0000","last_change":"2024-01-15T18:08:13.200457+0000","last_active":"2024-01-15T18:08:14.217464+0000","last_peered":"2024-01-15T18:08:14.217464+0000","last_clean":"2024-01-15T18:08:14.217464+0000","last_became_active":"2024-01-15T18:08:13.200168+0000","last_became_peered":"2024-01-15T18:08:13.200168+0000","last_unstale":"2024-01-15T18:08:14.217464+0000","last_undegraded":"2024-01-15T18:08:14.217464+0000","last_fullsized":"2024-01-15T18:08:14.217464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:31:16.895925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213385+0000","last_change":"2024-01-15T18:08:14.214541+0000","last_active":"2024-01-15T18:08:15.213385+0000","last_peered":"2024-01-15T18:08:15.213385+0000","last_clean":"2024-01-15T18:08:15.213385+0000","last_became_active":"2024-01-15T18:08:14.214256+0000","last_became_peered":"2024-01-15T18:08:14.214256+0000","last_unstale":"2024-01-15T18:08:15.213385+0000","last_undegraded":"2024-01-15T18:08:15.213385+0000","last_fullsized":"2024-01-15T18:08:15.213385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:00:03.016857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.220273+0000","last_change":"2024-01-15T18:08:13.191881+0000","last_active":"2024-01-15T18:08:14.220273+0000","last_peered":"2024-01-15T18:08:14.220273+0000","last_clean":"2024-01-15T18:08:14.220273+0000","last_became_active":"2024-01-15T18:08:13.191220+0000","last_became_peered":"2024-01-15T18:08:13.191220+0000","last_unstale":"2024-01-15T18:08:14.220273+0000","last_undegraded":"2024-01-15T18:08:14.220273+0000","last_fullsized":"2024-01-15T18:08:14.220273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:20:36.548073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257169+0000","last_change":"2024-01-15T18:08:14.221203+0000","last_active":"2024-01-15T18:08:16.257169+0000","last_peered":"2024-01-15T18:08:16.257169+0000","last_clean":"2024-01-15T18:08:16.257169+0000","last_became_active":"2024-01-15T18:08:14.220378+0000","last_became_peered":"2024-01-15T18:08:14.220378+0000","last_unstale":"2024-01-15T18:08:16.257169+0000","last_undegraded":"2024-01-15T18:08:16.257169+0000","last_fullsized":"2024-01-15T18:08:16.257169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:20:18.545134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.199249+0000","last_change":"2024-01-15T18:08:13.202372+0000","last_active":"2024-01-15T18:08:14.199249+0000","last_peered":"2024-01-15T18:08:14.199249+0000","last_clean":"2024-01-15T18:08:14.199249+0000","last_became_active":"2024-01-15T18:08:13.202106+0000","last_became_peered":"2024-01-15T18:08:13.202106+0000","last_unstale":"2024-01-15T18:08:14.199249+0000","last_undegraded":"2024-01-15T18:08:14.199249+0000","last_fullsized":"2024-01-15T18:08:14.199249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:34.054825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.203903+0000","last_change":"2024-01-15T18:08:14.203670+0000","last_active":"2024-01-15T18:08:15.203903+0000","last_peered":"2024-01-15T18:08:15.203903+0000","last_clean":"2024-01-15T18:08:15.203903+0000","last_became_active":"2024-01-15T18:08:14.203214+0000","last_became_peered":"2024-01-15T18:08:14.203214+0000","last_unstale":"2024-01-15T18:08:15.203903+0000","last_undegraded":"2024-01-15T18:08:15.203903+0000","last_fullsized":"2024-01-15T18:08:15.203903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:09.872734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.187069+0000","last_change":"2024-01-15T18:08:13.204304+0000","last_active":"2024-01-15T18:08:14.187069+0000","last_peered":"2024-01-15T18:08:14.187069+0000","last_clean":"2024-01-15T18:08:14.187069+0000","last_became_active":"2024-01-15T18:08:13.204000+0000","last_became_peered":"2024-01-15T18:08:13.204000+0000","last_unstale":"2024-01-15T18:08:14.187069+0000","last_undegraded":"2024-01-15T18:08:14.187069+0000","last_fullsized":"2024-01-15T18:08:14.187069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:20:32.338335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.204375+0000","last_change":"2024-01-15T18:08:14.214098+0000","last_active":"2024-01-15T18:08:15.204375+0000","last_peered":"2024-01-15T18:08:15.204375+0000","last_clean":"2024-01-15T18:08:15.204375+0000","last_became_active":"2024-01-15T18:08:14.213768+0000","last_became_peered":"2024-01-15T18:08:14.213768+0000","last_unstale":"2024-01-15T18:08:15.204375+0000","last_undegraded":"2024-01-15T18:08:15.204375+0000","last_fullsized":"2024-01-15T18:08:15.204375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:02:11.079420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.196443+0000","last_change":"2024-01-15T18:08:13.197699+0000","last_active":"2024-01-15T18:08:14.196443+0000","last_peered":"2024-01-15T18:08:14.196443+0000","last_clean":"2024-01-15T18:08:14.196443+0000","last_became_active":"2024-01-15T18:08:13.197224+0000","last_became_peered":"2024-01-15T18:08:13.197224+0000","last_unstale":"2024-01-15T18:08:14.196443+0000","last_undegraded":"2024-01-15T18:08:14.196443+0000","last_fullsized":"2024-01-15T18:08:14.196443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:38:22.415229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259470+0000","last_change":"2024-01-15T18:08:14.220949+0000","last_active":"2024-01-15T18:08:16.259470+0000","last_peered":"2024-01-15T18:08:16.259470+0000","last_clean":"2024-01-15T18:08:16.259470+0000","last_became_active":"2024-01-15T18:08:14.220670+0000","last_became_peered":"2024-01-15T18:08:14.220670+0000","last_unstale":"2024-01-15T18:08:16.259470+0000","last_undegraded":"2024-01-15T18:08:16.259470+0000","last_fullsized":"2024-01-15T18:08:16.259470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:45:17.395330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.198092+0000","last_change":"2024-01-15T18:08:13.204031+0000","last_active":"2024-01-15T18:08:14.198092+0000","last_peered":"2024-01-15T18:08:14.198092+0000","last_clean":"2024-01-15T18:08:14.198092+0000","last_became_active":"2024-01-15T18:08:13.203462+0000","last_became_peered":"2024-01-15T18:08:13.203462+0000","last_unstale":"2024-01-15T18:08:14.198092+0000","last_undegraded":"2024-01-15T18:08:14.198092+0000","last_fullsized":"2024-01-15T18:08:14.198092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:52:57.254176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.088636+0000","last_change":"2024-01-15T18:08:14.225446+0000","last_active":"2024-01-15T18:08:16.088636+0000","last_peered":"2024-01-15T18:08:16.088636+0000","last_clean":"2024-01-15T18:08:16.088636+0000","last_became_active":"2024-01-15T18:08:14.224731+0000","last_became_peered":"2024-01-15T18:08:14.224731+0000","last_unstale":"2024-01-15T18:08:16.088636+0000","last_undegraded":"2024-01-15T18:08:16.088636+0000","last_fullsized":"2024-01-15T18:08:16.088636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:00:47.935443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.218816+0000","last_change":"2024-01-15T18:08:13.198087+0000","last_active":"2024-01-15T18:08:14.218816+0000","last_peered":"2024-01-15T18:08:14.218816+0000","last_clean":"2024-01-15T18:08:14.218816+0000","last_became_active":"2024-01-15T18:08:13.197428+0000","last_became_peered":"2024-01-15T18:08:13.197428+0000","last_unstale":"2024-01-15T18:08:14.218816+0000","last_undegraded":"2024-01-15T18:08:14.218816+0000","last_fullsized":"2024-01-15T18:08:14.218816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:18:16.880458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212508+0000","last_change":"2024-01-15T18:08:14.207758+0000","last_active":"2024-01-15T18:08:15.212508+0000","last_peered":"2024-01-15T18:08:15.212508+0000","last_clean":"2024-01-15T18:08:15.212508+0000","last_became_active":"2024-01-15T18:08:14.207382+0000","last_became_peered":"2024-01-15T18:08:14.207382+0000","last_unstale":"2024-01-15T18:08:15.212508+0000","last_undegraded":"2024-01-15T18:08:15.212508+0000","last_fullsized":"2024-01-15T18:08:15.212508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:32:15.267128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.226769+0000","last_change":"2024-01-15T18:08:13.207430+0000","last_active":"2024-01-15T18:08:14.226769+0000","last_peered":"2024-01-15T18:08:14.226769+0000","last_clean":"2024-01-15T18:08:14.226769+0000","last_became_active":"2024-01-15T18:08:13.207137+0000","last_became_peered":"2024-01-15T18:08:13.207137+0000","last_unstale":"2024-01-15T18:08:14.226769+0000","last_undegraded":"2024-01-15T18:08:14.226769+0000","last_fullsized":"2024-01-15T18:08:14.226769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:21:32.238092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.088363+0000","last_change":"2024-01-15T18:08:14.200200+0000","last_active":"2024-01-15T18:08:16.088363+0000","last_peered":"2024-01-15T18:08:16.088363+0000","last_clean":"2024-01-15T18:08:16.088363+0000","last_became_active":"2024-01-15T18:08:14.199890+0000","last_became_peered":"2024-01-15T18:08:14.199890+0000","last_unstale":"2024-01-15T18:08:16.088363+0000","last_undegraded":"2024-01-15T18:08:16.088363+0000","last_fullsized":"2024-01-15T18:08:16.088363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:45:17.395330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.226141+0000","last_change":"2024-01-15T18:08:13.202751+0000","last_active":"2024-01-15T18:08:14.226141+0000","last_peered":"2024-01-15T18:08:14.226141+0000","last_clean":"2024-01-15T18:08:14.226141+0000","last_became_active":"2024-01-15T18:08:13.202104+0000","last_became_peered":"2024-01-15T18:08:13.202104+0000","last_unstale":"2024-01-15T18:08:14.226141+0000","last_undegraded":"2024-01-15T18:08:14.226141+0000","last_fullsized":"2024-01-15T18:08:14.226141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:22:46.990488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257731+0000","last_change":"2024-01-15T18:08:14.213649+0000","last_active":"2024-01-15T18:08:16.257731+0000","last_peered":"2024-01-15T18:08:16.257731+0000","last_clean":"2024-01-15T18:08:16.257731+0000","last_became_active":"2024-01-15T18:08:14.213255+0000","last_became_peered":"2024-01-15T18:08:14.213255+0000","last_unstale":"2024-01-15T18:08:16.257731+0000","last_undegraded":"2024-01-15T18:08:16.257731+0000","last_fullsized":"2024-01-15T18:08:16.257731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:42.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.201096+0000","last_change":"2024-01-15T18:08:13.194370+0000","last_active":"2024-01-15T18:08:14.201096+0000","last_peered":"2024-01-15T18:08:14.201096+0000","last_clean":"2024-01-15T18:08:14.201096+0000","last_became_active":"2024-01-15T18:08:13.193625+0000","last_became_peered":"2024-01-15T18:08:13.193625+0000","last_unstale":"2024-01-15T18:08:14.201096+0000","last_undegraded":"2024-01-15T18:08:14.201096+0000","last_fullsized":"2024-01-15T18:08:14.201096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:39.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257160+0000","last_change":"2024-01-15T18:08:14.217681+0000","last_active":"2024-01-15T18:08:16.257160+0000","last_peered":"2024-01-15T18:08:16.257160+0000","last_clean":"2024-01-15T18:08:16.257160+0000","last_became_active":"2024-01-15T18:08:14.216412+0000","last_became_peered":"2024-01-15T18:08:14.216412+0000","last_unstale":"2024-01-15T18:08:16.257160+0000","last_undegraded":"2024-01-15T18:08:16.257160+0000","last_fullsized":"2024-01-15T18:08:16.257160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:37.149934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.211209+0000","last_change":"2024-01-15T18:08:13.203484+0000","last_active":"2024-01-15T18:08:14.211209+0000","last_peered":"2024-01-15T18:08:14.211209+0000","last_clean":"2024-01-15T18:08:14.211209+0000","last_became_active":"2024-01-15T18:08:13.203206+0000","last_became_peered":"2024-01-15T18:08:13.203206+0000","last_unstale":"2024-01-15T18:08:14.211209+0000","last_undegraded":"2024-01-15T18:08:14.211209+0000","last_fullsized":"2024-01-15T18:08:14.211209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:32:14.261016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257535+0000","last_change":"2024-01-15T18:08:14.211524+0000","last_active":"2024-01-15T18:08:16.257535+0000","last_peered":"2024-01-15T18:08:16.257535+0000","last_clean":"2024-01-15T18:08:16.257535+0000","last_became_active":"2024-01-15T18:08:14.211249+0000","last_became_peered":"2024-01-15T18:08:14.211249+0000","last_unstale":"2024-01-15T18:08:16.257535+0000","last_undegraded":"2024-01-15T18:08:16.257535+0000","last_fullsized":"2024-01-15T18:08:16.257535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:27:06.733990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.225460+0000","last_change":"2024-01-15T18:08:13.208116+0000","last_active":"2024-01-15T18:08:14.225460+0000","last_peered":"2024-01-15T18:08:14.225460+0000","last_clean":"2024-01-15T18:08:14.225460+0000","last_became_active":"2024-01-15T18:08:13.207828+0000","last_became_peered":"2024-01-15T18:08:13.207828+0000","last_unstale":"2024-01-15T18:08:14.225460+0000","last_undegraded":"2024-01-15T18:08:14.225460+0000","last_fullsized":"2024-01-15T18:08:14.225460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:21:32.238092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258472+0000","last_change":"2024-01-15T18:08:14.215780+0000","last_active":"2024-01-15T18:08:16.258472+0000","last_peered":"2024-01-15T18:08:16.258472+0000","last_clean":"2024-01-15T18:08:16.258472+0000","last_became_active":"2024-01-15T18:08:14.215318+0000","last_became_peered":"2024-01-15T18:08:14.215318+0000","last_unstale":"2024-01-15T18:08:16.258472+0000","last_undegraded":"2024-01-15T18:08:16.258472+0000","last_fullsized":"2024-01-15T18:08:16.258472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:57:52.268562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.207418+0000","last_change":"2024-01-15T18:08:13.206233+0000","last_active":"2024-01-15T18:08:14.207418+0000","last_peered":"2024-01-15T18:08:14.207418+0000","last_clean":"2024-01-15T18:08:14.207418+0000","last_became_active":"2024-01-15T18:08:13.205764+0000","last_became_peered":"2024-01-15T18:08:13.205764+0000","last_unstale":"2024-01-15T18:08:14.207418+0000","last_undegraded":"2024-01-15T18:08:14.207418+0000","last_fullsized":"2024-01-15T18:08:14.207418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:54.245751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213264+0000","last_change":"2024-01-15T18:08:14.219217+0000","last_active":"2024-01-15T18:08:15.213264+0000","last_peered":"2024-01-15T18:08:15.213264+0000","last_clean":"2024-01-15T18:08:15.213264+0000","last_became_active":"2024-01-15T18:08:14.218075+0000","last_became_peered":"2024-01-15T18:08:14.218075+0000","last_unstale":"2024-01-15T18:08:15.213264+0000","last_undegraded":"2024-01-15T18:08:15.213264+0000","last_fullsized":"2024-01-15T18:08:15.213264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:02.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.220358+0000","last_change":"2024-01-15T18:08:13.209597+0000","last_active":"2024-01-15T18:08:14.220358+0000","last_peered":"2024-01-15T18:08:14.220358+0000","last_clean":"2024-01-15T18:08:14.220358+0000","last_became_active":"2024-01-15T18:08:13.209194+0000","last_became_peered":"2024-01-15T18:08:13.209194+0000","last_unstale":"2024-01-15T18:08:14.220358+0000","last_undegraded":"2024-01-15T18:08:14.220358+0000","last_fullsized":"2024-01-15T18:08:14.220358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:32:14.261016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212883+0000","last_change":"2024-01-15T18:08:14.201377+0000","last_active":"2024-01-15T18:08:15.212883+0000","last_peered":"2024-01-15T18:08:15.212883+0000","last_clean":"2024-01-15T18:08:15.212883+0000","last_became_active":"2024-01-15T18:08:14.199198+0000","last_became_peered":"2024-01-15T18:08:14.199198+0000","last_unstale":"2024-01-15T18:08:15.212883+0000","last_undegraded":"2024-01-15T18:08:15.212883+0000","last_fullsized":"2024-01-15T18:08:15.212883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:33.977943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.199325+0000","last_change":"2024-01-15T18:08:13.200813+0000","last_active":"2024-01-15T18:08:14.199325+0000","last_peered":"2024-01-15T18:08:14.199325+0000","last_clean":"2024-01-15T18:08:14.199325+0000","last_became_active":"2024-01-15T18:08:13.200515+0000","last_became_peered":"2024-01-15T18:08:13.200515+0000","last_unstale":"2024-01-15T18:08:14.199325+0000","last_undegraded":"2024-01-15T18:08:14.199325+0000","last_fullsized":"2024-01-15T18:08:14.199325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:32.971831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257494+0000","last_change":"2024-01-15T18:08:14.197310+0000","last_active":"2024-01-15T18:08:16.257494+0000","last_peered":"2024-01-15T18:08:16.257494+0000","last_clean":"2024-01-15T18:08:16.257494+0000","last_became_active":"2024-01-15T18:08:14.196982+0000","last_became_peered":"2024-01-15T18:08:14.196982+0000","last_unstale":"2024-01-15T18:08:16.257494+0000","last_undegraded":"2024-01-15T18:08:16.257494+0000","last_fullsized":"2024-01-15T18:08:16.257494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:44:54.584810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.218852+0000","last_change":"2024-01-15T18:08:13.201215+0000","last_active":"2024-01-15T18:08:14.218852+0000","last_peered":"2024-01-15T18:08:14.218852+0000","last_clean":"2024-01-15T18:08:14.218852+0000","last_became_active":"2024-01-15T18:08:13.200067+0000","last_became_peered":"2024-01-15T18:08:13.200067+0000","last_unstale":"2024-01-15T18:08:14.218852+0000","last_undegraded":"2024-01-15T18:08:14.218852+0000","last_fullsized":"2024-01-15T18:08:14.218852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:01:31.833008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258869+0000","last_change":"2024-01-15T18:08:14.221303+0000","last_active":"2024-01-15T18:08:16.258869+0000","last_peered":"2024-01-15T18:08:16.258869+0000","last_clean":"2024-01-15T18:08:16.258869+0000","last_became_active":"2024-01-15T18:08:14.220981+0000","last_became_peered":"2024-01-15T18:08:14.220981+0000","last_unstale":"2024-01-15T18:08:16.258869+0000","last_undegraded":"2024-01-15T18:08:16.258869+0000","last_fullsized":"2024-01-15T18:08:16.258869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:33.977943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.184518+0000","last_change":"2024-01-15T18:08:13.206092+0000","last_active":"2024-01-15T18:08:14.184518+0000","last_peered":"2024-01-15T18:08:14.184518+0000","last_clean":"2024-01-15T18:08:14.184518+0000","last_became_active":"2024-01-15T18:08:13.205598+0000","last_became_peered":"2024-01-15T18:08:13.205598+0000","last_unstale":"2024-01-15T18:08:14.184518+0000","last_undegraded":"2024-01-15T18:08:14.184518+0000","last_fullsized":"2024-01-15T18:08:14.184518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:30.508678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.256455+0000","last_change":"2024-01-15T18:08:14.217833+0000","last_active":"2024-01-15T18:08:16.256455+0000","last_peered":"2024-01-15T18:08:16.256455+0000","last_clean":"2024-01-15T18:08:16.256455+0000","last_became_active":"2024-01-15T18:08:14.217432+0000","last_became_peered":"2024-01-15T18:08:14.217432+0000","last_unstale":"2024-01-15T18:08:16.256455+0000","last_undegraded":"2024-01-15T18:08:16.256455+0000","last_fullsized":"2024-01-15T18:08:16.256455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:57.359384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.185369+0000","last_change":"2024-01-15T18:08:13.201583+0000","last_active":"2024-01-15T18:08:14.185369+0000","last_peered":"2024-01-15T18:08:14.185369+0000","last_clean":"2024-01-15T18:08:14.185369+0000","last_became_active":"2024-01-15T18:08:13.201291+0000","last_became_peered":"2024-01-15T18:08:13.201291+0000","last_unstale":"2024-01-15T18:08:14.185369+0000","last_undegraded":"2024-01-15T18:08:14.185369+0000","last_fullsized":"2024-01-15T18:08:14.185369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:20:32.338335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258891+0000","last_change":"2024-01-15T18:08:14.215539+0000","last_active":"2024-01-15T18:08:16.258891+0000","last_peered":"2024-01-15T18:08:16.258891+0000","last_clean":"2024-01-15T18:08:16.258891+0000","last_became_active":"2024-01-15T18:08:14.215269+0000","last_became_peered":"2024-01-15T18:08:14.215269+0000","last_unstale":"2024-01-15T18:08:16.258891+0000","last_undegraded":"2024-01-15T18:08:16.258891+0000","last_fullsized":"2024-01-15T18:08:16.258891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:02.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.200566+0000","last_change":"2024-01-15T18:08:13.200034+0000","last_active":"2024-01-15T18:08:14.200566+0000","last_peered":"2024-01-15T18:08:14.200566+0000","last_clean":"2024-01-15T18:08:14.200566+0000","last_became_active":"2024-01-15T18:08:13.199719+0000","last_became_peered":"2024-01-15T18:08:13.199719+0000","last_unstale":"2024-01-15T18:08:14.200566+0000","last_undegraded":"2024-01-15T18:08:14.200566+0000","last_fullsized":"2024-01-15T18:08:14.200566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:40:01.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212496+0000","last_change":"2024-01-15T18:08:14.213547+0000","last_active":"2024-01-15T18:08:15.212496+0000","last_peered":"2024-01-15T18:08:15.212496+0000","last_clean":"2024-01-15T18:08:15.212496+0000","last_became_active":"2024-01-15T18:08:14.213296+0000","last_became_peered":"2024-01-15T18:08:14.213296+0000","last_unstale":"2024-01-15T18:08:15.212496+0000","last_undegraded":"2024-01-15T18:08:15.212496+0000","last_fullsized":"2024-01-15T18:08:15.212496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:21:33.244204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.218630+0000","last_change":"2024-01-15T18:08:13.208311+0000","last_active":"2024-01-15T18:08:14.218630+0000","last_peered":"2024-01-15T18:08:14.218630+0000","last_clean":"2024-01-15T18:08:14.218630+0000","last_became_active":"2024-01-15T18:08:13.207539+0000","last_became_peered":"2024-01-15T18:08:13.207539+0000","last_unstale":"2024-01-15T18:08:14.218630+0000","last_undegraded":"2024-01-15T18:08:14.218630+0000","last_fullsized":"2024-01-15T18:08:14.218630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:19:34.801714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258818+0000","last_change":"2024-01-15T18:08:14.220846+0000","last_active":"2024-01-15T18:08:16.258818+0000","last_peered":"2024-01-15T18:08:16.258818+0000","last_clean":"2024-01-15T18:08:16.258818+0000","last_became_active":"2024-01-15T18:08:14.220469+0000","last_became_peered":"2024-01-15T18:08:14.220469+0000","last_unstale":"2024-01-15T18:08:16.258818+0000","last_undegraded":"2024-01-15T18:08:16.258818+0000","last_fullsized":"2024-01-15T18:08:16.258818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:40:03.935147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.193694+0000","last_change":"2024-01-15T18:08:13.204715+0000","last_active":"2024-01-15T18:08:14.193694+0000","last_peered":"2024-01-15T18:08:14.193694+0000","last_clean":"2024-01-15T18:08:14.193694+0000","last_became_active":"2024-01-15T18:08:13.204391+0000","last_became_peered":"2024-01-15T18:08:13.204391+0000","last_unstale":"2024-01-15T18:08:14.193694+0000","last_undegraded":"2024-01-15T18:08:14.193694+0000","last_fullsized":"2024-01-15T18:08:14.193694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:06:35.305979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210683+0000","last_change":"2024-01-15T18:08:14.217631+0000","last_active":"2024-01-15T18:08:15.210683+0000","last_peered":"2024-01-15T18:08:15.210683+0000","last_clean":"2024-01-15T18:08:15.210683+0000","last_became_active":"2024-01-15T18:08:14.216578+0000","last_became_peered":"2024-01-15T18:08:14.216578+0000","last_unstale":"2024-01-15T18:08:15.210683+0000","last_undegraded":"2024-01-15T18:08:15.210683+0000","last_fullsized":"2024-01-15T18:08:15.210683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:21:33.244204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.196019+0000","last_change":"2024-01-15T18:08:13.201746+0000","last_active":"2024-01-15T18:08:14.196019+0000","last_peered":"2024-01-15T18:08:14.196019+0000","last_clean":"2024-01-15T18:08:14.196019+0000","last_became_active":"2024-01-15T18:08:13.201382+0000","last_became_peered":"2024-01-15T18:08:13.201382+0000","last_unstale":"2024-01-15T18:08:14.196019+0000","last_undegraded":"2024-01-15T18:08:14.196019+0000","last_fullsized":"2024-01-15T18:08:14.196019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:42.053642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213062+0000","last_change":"2024-01-15T18:08:14.206746+0000","last_active":"2024-01-15T18:08:15.213062+0000","last_peered":"2024-01-15T18:08:15.213062+0000","last_clean":"2024-01-15T18:08:15.213062+0000","last_became_active":"2024-01-15T18:08:14.204676+0000","last_became_peered":"2024-01-15T18:08:14.204676+0000","last_unstale":"2024-01-15T18:08:15.213062+0000","last_undegraded":"2024-01-15T18:08:15.213062+0000","last_fullsized":"2024-01-15T18:08:15.213062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:10.354228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.196218+0000","last_change":"2024-01-15T18:08:13.189673+0000","last_active":"2024-01-15T18:08:14.196218+0000","last_peered":"2024-01-15T18:08:14.196218+0000","last_clean":"2024-01-15T18:08:14.196218+0000","last_became_active":"2024-01-15T18:08:13.189374+0000","last_became_peered":"2024-01-15T18:08:13.189374+0000","last_unstale":"2024-01-15T18:08:14.196218+0000","last_undegraded":"2024-01-15T18:08:14.196218+0000","last_fullsized":"2024-01-15T18:08:14.196218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:50:10.044020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258990+0000","last_change":"2024-01-15T18:08:14.223338+0000","last_active":"2024-01-15T18:08:16.258990+0000","last_peered":"2024-01-15T18:08:16.258990+0000","last_clean":"2024-01-15T18:08:16.258990+0000","last_became_active":"2024-01-15T18:08:14.222080+0000","last_became_peered":"2024-01-15T18:08:14.222080+0000","last_unstale":"2024-01-15T18:08:16.258990+0000","last_undegraded":"2024-01-15T18:08:16.258990+0000","last_fullsized":"2024-01-15T18:08:16.258990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:10.354228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.200546+0000","last_change":"2024-01-15T18:08:13.209814+0000","last_active":"2024-01-15T18:08:14.200546+0000","last_peered":"2024-01-15T18:08:14.200546+0000","last_clean":"2024-01-15T18:08:14.200546+0000","last_became_active":"2024-01-15T18:08:13.208708+0000","last_became_peered":"2024-01-15T18:08:13.208708+0000","last_unstale":"2024-01-15T18:08:14.200546+0000","last_undegraded":"2024-01-15T18:08:14.200546+0000","last_fullsized":"2024-01-15T18:08:14.200546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:09.348116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.088655+0000","last_change":"2024-01-15T18:08:14.223030+0000","last_active":"2024-01-15T18:08:16.088655+0000","last_peered":"2024-01-15T18:08:16.088655+0000","last_clean":"2024-01-15T18:08:16.088655+0000","last_became_active":"2024-01-15T18:08:14.221307+0000","last_became_peered":"2024-01-15T18:08:14.221307+0000","last_unstale":"2024-01-15T18:08:16.088655+0000","last_undegraded":"2024-01-15T18:08:16.088655+0000","last_fullsized":"2024-01-15T18:08:16.088655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:49:35.594008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.188354+0000","last_change":"2024-01-15T18:08:13.190400+0000","last_active":"2024-01-15T18:08:14.188354+0000","last_peered":"2024-01-15T18:08:14.188354+0000","last_clean":"2024-01-15T18:08:14.188354+0000","last_became_active":"2024-01-15T18:08:13.190006+0000","last_became_peered":"2024-01-15T18:08:13.190006+0000","last_unstale":"2024-01-15T18:08:14.188354+0000","last_undegraded":"2024-01-15T18:08:14.188354+0000","last_fullsized":"2024-01-15T18:08:14.188354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:01:36.508305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257184+0000","last_change":"2024-01-15T18:08:14.219320+0000","last_active":"2024-01-15T18:08:16.257184+0000","last_peered":"2024-01-15T18:08:16.257184+0000","last_clean":"2024-01-15T18:08:16.257184+0000","last_became_active":"2024-01-15T18:08:14.218353+0000","last_became_peered":"2024-01-15T18:08:14.218353+0000","last_unstale":"2024-01-15T18:08:16.257184+0000","last_undegraded":"2024-01-15T18:08:16.257184+0000","last_fullsized":"2024-01-15T18:08:16.257184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:56.902259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.195914+0000","last_change":"2024-01-15T18:08:13.199997+0000","last_active":"2024-01-15T18:08:14.195914+0000","last_peered":"2024-01-15T18:08:14.195914+0000","last_clean":"2024-01-15T18:08:14.195914+0000","last_became_active":"2024-01-15T18:08:13.198486+0000","last_became_peered":"2024-01-15T18:08:13.198486+0000","last_unstale":"2024-01-15T18:08:14.195914+0000","last_undegraded":"2024-01-15T18:08:14.195914+0000","last_fullsized":"2024-01-15T18:08:14.195914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:37:49.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.211490+0000","last_change":"2024-01-15T18:08:14.218836+0000","last_active":"2024-01-15T18:08:15.211490+0000","last_peered":"2024-01-15T18:08:15.211490+0000","last_clean":"2024-01-15T18:08:15.211490+0000","last_became_active":"2024-01-15T18:08:14.218487+0000","last_became_peered":"2024-01-15T18:08:14.218487+0000","last_unstale":"2024-01-15T18:08:15.211490+0000","last_undegraded":"2024-01-15T18:08:15.211490+0000","last_fullsized":"2024-01-15T18:08:15.211490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:32:15.267128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.186113+0000","last_change":"2024-01-15T18:08:13.203003+0000","last_active":"2024-01-15T18:08:14.186113+0000","last_peered":"2024-01-15T18:08:14.186113+0000","last_clean":"2024-01-15T18:08:14.186113+0000","last_became_active":"2024-01-15T18:08:13.202706+0000","last_became_peered":"2024-01-15T18:08:13.202706+0000","last_unstale":"2024-01-15T18:08:14.186113+0000","last_undegraded":"2024-01-15T18:08:14.186113+0000","last_fullsized":"2024-01-15T18:08:14.186113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:37:49.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259024+0000","last_change":"2024-01-15T18:08:14.218124+0000","last_active":"2024-01-15T18:08:16.259024+0000","last_peered":"2024-01-15T18:08:16.259024+0000","last_clean":"2024-01-15T18:08:16.259024+0000","last_became_active":"2024-01-15T18:08:14.217043+0000","last_became_peered":"2024-01-15T18:08:14.217043+0000","last_unstale":"2024-01-15T18:08:16.259024+0000","last_undegraded":"2024-01-15T18:08:16.259024+0000","last_fullsized":"2024-01-15T18:08:16.259024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:14.974406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.221504+0000","last_change":"2024-01-15T18:08:13.208928+0000","last_active":"2024-01-15T18:08:14.221504+0000","last_peered":"2024-01-15T18:08:14.221504+0000","last_clean":"2024-01-15T18:08:14.221504+0000","last_became_active":"2024-01-15T18:08:13.208647+0000","last_became_peered":"2024-01-15T18:08:13.208647+0000","last_unstale":"2024-01-15T18:08:14.221504+0000","last_undegraded":"2024-01-15T18:08:14.221504+0000","last_fullsized":"2024-01-15T18:08:14.221504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:37.222623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.208031+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:08:14.208031+0000","last_peered":"2024-01-15T18:08:14.208031+0000","last_clean":"2024-01-15T18:08:14.208031+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:08:14.208031+0000","last_undegraded":"2024-01-15T18:08:14.208031+0000","last_fullsized":"2024-01-15T18:08:14.208031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542307,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":1662976,"data_stored":1049281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77330890},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.667,"15min":1.363},"min":{"1min":0.84399999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.823,"5min":19.843,"15min":19.843},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.8979999999999999,"15min":1.593},"min":{"1min":0.91800000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5509999999999999,"5min":30.498000000000001,"15min":38.137},"last":1.774}]},{"osd":1,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.7090000000000001,"15min":1.413},"min":{"1min":0.86799999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5700000000000001,"5min":19.917000000000002,"15min":19.917000000000002},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.6140000000000001,"15min":1.597},"min":{"1min":0.98799999999999999,"5min":0.71199999999999997,"15min":0.42999999999999999},"max":{"1min":1.667,"5min":10.361000000000001,"15min":51.843000000000004},"last":1.266}]},{"osd":2,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.573,"15min":1.4059999999999999},"min":{"1min":0.73199999999999998,"5min":0.50600000000000001,"15min":0.437},"max":{"1min":1.607,"5min":12.061,"15min":12.061},"last":1.833},{"interface":"front","average":{"1min":1.367,"5min":1.8049999999999999,"15min":1.5429999999999999},"min":{"1min":1.036,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":1.639,"5min":13.478999999999999,"15min":13.478999999999999},"last":1.4610000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.778,"15min":1.5369999999999999},"min":{"1min":0.91500000000000004,"5min":0.56000000000000005,"15min":0.47899999999999998},"max":{"1min":1.544,"5min":16.632000000000001,"15min":16.632000000000001},"last":1.893},{"interface":"front","average":{"1min":1.355,"5min":1.853,"15min":1.6240000000000001},"min":{"1min":0.94199999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.768,"5min":13.439,"15min":13.439},"last":1.575}]},{"osd":4,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.7929999999999999,"15min":1.7190000000000001},"min":{"1min":0.878,"5min":0.54200000000000004,"15min":0.50700000000000001},"max":{"1min":1.5820000000000001,"5min":23.042000000000002,"15min":23.042000000000002},"last":1.514},{"interface":"front","average":{"1min":1.331,"5min":2.1469999999999998,"15min":1.7330000000000001},"min":{"1min":0.98999999999999999,"5min":0.52700000000000002,"15min":0.51800000000000002},"max":{"1min":1.7330000000000001,"5min":38.682000000000002,"15min":38.682000000000002},"last":1.71}]},{"osd":5,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.8169999999999999,"15min":1.8200000000000001},"min":{"1min":0.98999999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.669,"5min":13.379,"15min":25.006},"last":1.665},{"interface":"front","average":{"1min":1.3839999999999999,"5min":2.0110000000000001,"15min":1.855},"min":{"1min":1.0629999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.6819999999999999,"5min":22.933,"15min":22.933},"last":1.855}]},{"osd":6,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":2.165,"15min":2.0539999999999998},"min":{"1min":1.2470000000000001,"5min":0.66400000000000003,"15min":0.59299999999999997},"max":{"1min":1.7849999999999999,"5min":23.004000000000001,"15min":23.834},"last":1.639},{"interface":"front","average":{"1min":1.452,"5min":1.8400000000000001,"15min":1.998},"min":{"1min":1.05,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.825,"5min":9.2599999999999998,"15min":63.932000000000002},"last":1.95}]}]},{"osd":6,"up_from":14,"seq":60129542306,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58600,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313240,"statfs":{"total":96636764160,"available":96576757760,"internally_reserved":0,"allocated":1077248,"data_stored":459537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.454,"15min":1.369},"min":{"1min":0.98299999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.702,"5min":8.593,"15min":12.544},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.579,"15min":1.389},"min":{"1min":0.86599999999999999,"5min":0.73199999999999998,"15min":0.68400000000000005},"max":{"1min":1.7390000000000001,"5min":14.013,"15min":14.013},"last":0.93799999999999994}]},{"osd":1,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.361,"15min":1.3029999999999999},"min":{"1min":0.79000000000000004,"5min":0.63700000000000001,"15min":0.55100000000000005},"max":{"1min":1.9359999999999999,"5min":8.7729999999999997,"15min":12.811999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.8680000000000001,"15min":1.534},"min":{"1min":0.93700000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.369,"5min":23.495999999999999,"15min":23.495999999999999},"last":1.0620000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.9970000000000001,"15min":1.5720000000000001},"min":{"1min":0.86499999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.6140000000000001,"5min":23.087,"15min":23.087},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.4119999999999999,"15min":1.4339999999999999},"min":{"1min":0.90500000000000003,"5min":0.51400000000000001,"15min":0.504},"max":{"1min":1.5529999999999999,"5min":8.8379999999999992,"15min":12.897},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.758,"15min":1.657},"min":{"1min":0.95799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8660000000000001,"5min":13.597,"15min":13.597},"last":1.294},{"interface":"front","average":{"1min":1.429,"5min":1.7989999999999999,"15min":1.597},"min":{"1min":1.103,"5min":0.78400000000000003,"15min":0.70699999999999996},"max":{"1min":1.821,"5min":13.795,"15min":14.066000000000001},"last":1.254}]},{"osd":4,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":2.1499999999999999,"15min":1.6839999999999999},"min":{"1min":0.85499999999999998,"5min":0.59599999999999997,"15min":0.51800000000000002},"max":{"1min":1.647,"5min":23.327000000000002,"15min":23.327000000000002},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.6379999999999999,"15min":1.659},"min":{"1min":1.0109999999999999,"5min":0.52000000000000002,"15min":0.45800000000000002},"max":{"1min":1.752,"5min":8.7040000000000006,"15min":17.289000000000001},"last":1.0129999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.7769999999999999,"15min":1.5700000000000001},"min":{"1min":0.94099999999999995,"5min":0.627,"15min":0.51600000000000001},"max":{"1min":1.7729999999999999,"5min":13.593,"15min":14.176},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.5489999999999999,"5min":2.2440000000000002,"15min":1.8959999999999999},"min":{"1min":1.331,"5min":0.621,"15min":0.54700000000000004},"max":{"1min":1.875,"5min":23.417999999999999,"15min":23.417999999999999},"last":1.425}]},{"osd":7,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":2.1379999999999999,"15min":1.77},"min":{"1min":1.101,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.905,"5min":21.265000000000001,"15min":21.265000000000001},"last":1.129},{"interface":"front","average":{"1min":1.48,"5min":2.5299999999999998,"15min":2.0099999999999998},"min":{"1min":1.175,"5min":0.65300000000000002,"15min":0.59699999999999998},"max":{"1min":1.8779999999999999,"5min":23.738,"15min":23.738},"last":1.476}]}]},{"osd":5,"up_from":14,"seq":60129542306,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54444,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317396,"statfs":{"total":96636764160,"available":96581013504,"internally_reserved":0,"allocated":1081344,"data_stored":460477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.375,"15min":1.2849999999999999},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.6419999999999999,"5min":19.266999999999999,"15min":19.266999999999999},"last":1.171},{"interface":"front","average":{"1min":1.151,"5min":1.1990000000000001,"15min":1.3759999999999999},"min":{"1min":0.82199999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5489999999999999,"5min":3.1760000000000002,"15min":17.184999999999999},"last":1.3140000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.3919999999999999,"15min":1.343},"min":{"1min":0.78300000000000003,"5min":0.60699999999999998,"15min":0.499},"max":{"1min":1.6839999999999999,"5min":9.1259999999999994,"15min":16.297000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.4890000000000001,"15min":1.3180000000000001},"min":{"1min":0.85899999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5860000000000001,"5min":9.2379999999999995,"15min":9.2379999999999995},"last":0.90600000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.296,"15min":1.266},"min":{"1min":1.0029999999999999,"5min":0.621,"15min":0.621},"max":{"1min":1.6859999999999999,"5min":4.0369999999999999,"15min":8.1820000000000004},"last":1.117},{"interface":"front","average":{"1min":1.373,"5min":1.454,"15min":1.4199999999999999},"min":{"1min":0.90000000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.673,"5min":9.2159999999999993,"15min":16.158999999999999},"last":1.472}]},{"osd":3,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.506,"15min":1.47},"min":{"1min":1,"5min":0.63700000000000001,"15min":0.56100000000000005},"max":{"1min":1.6850000000000001,"5min":9.1929999999999996,"15min":14.914},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.552,"15min":1.458},"min":{"1min":0.93200000000000005,"5min":0.64600000000000002,"15min":0.63300000000000001},"max":{"1min":1.804,"5min":7.0110000000000001,"15min":7.0110000000000001},"last":1.456}]},{"osd":4,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.6000000000000001,"15min":1.5089999999999999},"min":{"1min":0.84299999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6919999999999999,"5min":12.24,"15min":16.414999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.5449999999999999,"15min":1.6319999999999999},"min":{"1min":1.0029999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7010000000000001,"5min":9.1519999999999992,"15min":17.991},"last":1.429}]},{"osd":6,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.5349999999999999,"15min":1.5329999999999999},"min":{"1min":1.0680000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6819999999999999,"5min":3.7559999999999998,"15min":6.4630000000000001},"last":1.395},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.6040000000000001,"15min":1.605},"min":{"1min":0.995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.76,"5min":9.1649999999999991,"15min":16.364000000000001},"last":1.371}]},{"osd":7,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.7170000000000001,"15min":1.627},"min":{"1min":0.90900000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.742,"5min":9.0600000000000005,"15min":9.0600000000000005},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.885,"15min":1.7370000000000001},"min":{"1min":1.2549999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.865,"5min":19.356999999999999,"15min":19.356999999999999},"last":1.5069999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542306,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58592,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313248,"statfs":{"total":96636764160,"available":96576765952,"internally_reserved":0,"allocated":1064960,"data_stored":458843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3640000000000001,"15min":1.3360000000000001},"min":{"1min":0.51800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6080000000000001,"5min":7.8869999999999996,"15min":11.413},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4590000000000001,"15min":1.405},"min":{"1min":0.91600000000000004,"5min":0.63,"15min":0.53700000000000003},"max":{"1min":2.004,"5min":7.9160000000000004,"15min":9.7439999999999998},"last":1.0640000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3620000000000001,"15min":1.377},"min":{"1min":0.78500000000000003,"5min":0.61399999999999999,"15min":0.45000000000000001},"max":{"1min":1.706,"5min":11.128,"15min":11.987},"last":1.389},{"interface":"front","average":{"1min":1.151,"5min":1.282,"15min":1.331},"min":{"1min":0.69499999999999995,"5min":0.60799999999999998,"15min":0.55300000000000005},"max":{"1min":1.72,"5min":5.6109999999999998,"15min":16.058},"last":0.97899999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3500000000000001,"15min":1.5269999999999999},"min":{"1min":0.71899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6919999999999999,"5min":5.0960000000000001,"15min":13.865},"last":0.877},{"interface":"front","average":{"1min":1.337,"5min":1.5029999999999999,"15min":1.526},"min":{"1min":1.1319999999999999,"5min":0.83299999999999996,"15min":0.69499999999999995},"max":{"1min":1.5389999999999999,"5min":5.7050000000000001,"15min":13.903},"last":1.3080000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5389999999999999,"15min":1.637},"min":{"1min":1.0069999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6619999999999999,"5min":11.773,"15min":26.055},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.403,"15min":1.526},"min":{"1min":0.97899999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9099999999999999,"5min":4.0419999999999998,"15min":13.25},"last":1.1739999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.6499999999999999,"15min":1.6160000000000001},"min":{"1min":0.85699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7989999999999999,"5min":29.422000000000001,"15min":29.422000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.5720000000000001,"15min":1.573},"min":{"1min":1.04,"5min":0.53700000000000003,"15min":0.42199999999999999},"max":{"1min":1.7250000000000001,"5min":12.332000000000001,"15min":12.332000000000001},"last":1.264}]},{"osd":6,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.55,"15min":1.5920000000000001},"min":{"1min":0.93000000000000005,"5min":0.59699999999999998,"15min":0.45200000000000001},"max":{"1min":1.837,"5min":11.09,"15min":18.260000000000002},"last":1.135},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.6100000000000001,"15min":1.7230000000000001},"min":{"1min":0.99399999999999999,"5min":0.59999999999999998,"15min":0.48099999999999998},"max":{"1min":1.9239999999999999,"5min":9.8740000000000006,"15min":11.657999999999999},"last":1.573}]},{"osd":7,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.744,"15min":1.829},"min":{"1min":1,"5min":0.60499999999999998,"15min":0.503},"max":{"1min":2.0110000000000001,"5min":12.247,"15min":12.247},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.591,"5min":1.788,"15min":1.8979999999999999},"min":{"1min":0.97799999999999998,"5min":0.61399999999999999,"15min":0.53600000000000003},"max":{"1min":2.0169999999999999,"5min":12.311999999999999,"15min":12.311999999999999},"last":1.349}]}]},{"osd":3,"up_from":14,"seq":60129542308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68272,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303568,"statfs":{"total":96636764160,"available":96566853632,"internally_reserved":0,"allocated":1077248,"data_stored":460455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.373,"15min":1.2030000000000001},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.339,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.5029999999999999,"15min":1.3799999999999999},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5229999999999999,"5min":18.795999999999999,"15min":21.047000000000001},"last":0.96499999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.2969999999999999,"15min":1.298},"min":{"1min":0.66400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3400000000000001,"5min":12.372,"15min":17.777999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2250000000000001},"min":{"1min":0.47099999999999997,"5min":0.46700000000000003,"15min":0.39500000000000002},"max":{"1min":1.8200000000000001,"5min":11.002000000000001,"15min":11.002000000000001},"last":0.65200000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.339,"15min":1.272},"min":{"1min":0.60499999999999998,"5min":0.375,"15min":0.375},"max":{"1min":2.7519999999999998,"5min":10.964,"15min":10.964},"last":0.752},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.669,"15min":1.528},"min":{"1min":0.82299999999999995,"5min":0.57099999999999995,"15min":0.54000000000000004},"max":{"1min":2.3210000000000002,"5min":20.495000000000001,"15min":20.495000000000001},"last":1.1479999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.823,"15min":1.5740000000000001},"min":{"1min":1.042,"5min":0.71599999999999997,"15min":0.63500000000000001},"max":{"1min":2.5,"5min":13.536,"15min":13.536},"last":1.002},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.905,"15min":1.6739999999999999},"min":{"1min":0.83299999999999996,"5min":0.65100000000000002,"15min":0.54600000000000004},"max":{"1min":1.738,"5min":16.398,"15min":16.398},"last":0.71099999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.845,"15min":1.6779999999999999},"min":{"1min":0.875,"5min":0.67200000000000004,"15min":0.57299999999999995},"max":{"1min":2.423,"5min":20.606000000000002,"15min":20.606000000000002},"last":1.244},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.736,"15min":1.6899999999999999},"min":{"1min":0.94699999999999995,"5min":0.70199999999999996,"15min":0.68000000000000005},"max":{"1min":2.8820000000000001,"5min":11.603,"15min":11.603},"last":1.2729999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":2.0379999999999998,"15min":1.8580000000000001},"min":{"1min":1.0629999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.873,"5min":13.648999999999999,"15min":13.648999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.994,"15min":1.8280000000000001},"min":{"1min":0.99099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.1459999999999999,"5min":20.556000000000001,"15min":20.556000000000001},"last":1.0680000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":2.0910000000000002,"15min":1.8879999999999999},"min":{"1min":0.88600000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.6299999999999999,"5min":28.843,"15min":28.843},"last":1.133},{"interface":"front","average":{"1min":1.478,"5min":2.0339999999999998,"15min":1.909},"min":{"1min":0.98999999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1179999999999999,"5min":15.143000000000001,"15min":15.143000000000001},"last":1.048}]}]},{"osd":2,"up_from":14,"seq":60129542308,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64096,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307744,"statfs":{"total":96636764160,"available":96571129856,"internally_reserved":0,"allocated":1069056,"data_stored":459425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.1000000000000001,"15min":1.0960000000000001},"min":{"1min":0.71199999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2849999999999999,"5min":12.278,"15min":14.427},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.7370000000000001,"15min":1.319},"min":{"1min":0.50900000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.653,"5min":15.314,"15min":15.314},"last":0.629}]},{"osd":1,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.5900000000000001,"15min":1.454},"min":{"1min":0.83899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.579,"5min":34.463999999999999,"15min":34.463999999999999},"last":1.488},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.923,"15min":1.4490000000000001},"min":{"1min":0.63700000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.9159999999999999,"5min":25.859999999999999,"15min":25.859999999999999},"last":0.96799999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.4510000000000001,"15min":1.3799999999999999},"min":{"1min":0.77200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.9059999999999999,"5min":12.93,"15min":19.535},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.873,"15min":1.6439999999999999},"min":{"1min":0.97999999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.044,"5min":22.783999999999999,"15min":22.783999999999999},"last":1.393}]},{"osd":4,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":2.012,"15min":1.804},"min":{"1min":1.1839999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.79,"5min":29.986999999999998,"15min":29.986999999999998},"last":1.77},{"interface":"front","average":{"1min":1.4830000000000001,"5min":2.1339999999999999,"15min":1.7410000000000001},"min":{"1min":1.2390000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.768,"5min":21.199999999999999,"15min":21.199999999999999},"last":1.3100000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.7210000000000001,"15min":1.655},"min":{"1min":1.048,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9710000000000001,"5min":12.227,"15min":19.681999999999999},"last":1.155},{"interface":"front","average":{"1min":1.542,"5min":1.8600000000000001,"15min":1.768},"min":{"1min":1.248,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.2869999999999999,"5min":12.634,"15min":14.702999999999999},"last":1.962}]},{"osd":6,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":2.1840000000000002,"15min":1.778},"min":{"1min":0.871,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.9870000000000001,"5min":15.683999999999999,"15min":15.683999999999999},"last":1.575},{"interface":"front","average":{"1min":1.5700000000000001,"5min":2.0419999999999998,"15min":1.792},"min":{"1min":0.95999999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.3279999999999998,"5min":12.385,"15min":19.783000000000001},"last":1.4299999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":2.8980000000000001,"15min":2.0649999999999999},"min":{"1min":1.2170000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.411,"5min":57.006999999999998,"15min":57.006999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.619,"5min":2.4630000000000001,"15min":2.1299999999999999},"min":{"1min":1.2649999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2280000000000002,"5min":22.853999999999999,"15min":22.853999999999999},"last":1.635}]}]},{"osd":1,"up_from":14,"seq":60129542307,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68264,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303576,"statfs":{"total":96636764160,"available":96566861824,"internally_reserved":0,"allocated":1077248,"data_stored":459037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.575,"15min":1.1870000000000001},"min":{"1min":0.67700000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.6139999999999999,"5min":30.460999999999999,"15min":30.460999999999999},"last":1.778},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.323,"15min":1.107},"min":{"1min":0.38700000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.6970000000000001,"5min":18.658000000000001,"15min":18.658000000000001},"last":0.81000000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.1990000000000001,"15min":1.149},"min":{"1min":0.63600000000000001,"5min":0.47299999999999998,"15min":0.376},"max":{"1min":1.851,"5min":7.117,"15min":21.126000000000001},"last":0.88},{"interface":"front","average":{"1min":1.458,"5min":1.8600000000000001,"15min":1.53},"min":{"1min":0.73099999999999998,"5min":0.52700000000000002,"15min":0.48699999999999999},"max":{"1min":3.0409999999999999,"5min":30.593,"15min":30.593},"last":1.8839999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.7849999999999999,"15min":1.74},"min":{"1min":0.82499999999999996,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.1400000000000001,"5min":30.547000000000001,"15min":30.547000000000001},"last":2.0179999999999998},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4339999999999999,"15min":1.339},"min":{"1min":0.67500000000000004,"5min":0.437,"15min":0.437},"max":{"1min":5.1740000000000004,"5min":7.9610000000000003,"15min":15.510999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.702,"5min":1.7330000000000001,"15min":1.7370000000000001},"min":{"1min":1.006,"5min":0.72699999999999998,"15min":0.72499999999999998},"max":{"1min":5.2229999999999999,"5min":9.3650000000000002,"15min":14.827},"last":1.401},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6259999999999999,"15min":1.7230000000000001},"min":{"1min":0.90000000000000002,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.419,"5min":9.3010000000000002,"15min":14.851000000000001},"last":1.1819999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6599999999999999,"15min":1.6850000000000001},"min":{"1min":0.80800000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.282,"5min":6.0700000000000003,"15min":13.228999999999999},"last":1.302},{"interface":"front","average":{"1min":1.548,"5min":2.3340000000000001,"15min":1.9510000000000001},"min":{"1min":1.024,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.8119999999999998,"5min":34.042999999999999,"15min":34.042999999999999},"last":2.153}]},{"osd":6,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.9390000000000001,"15min":1.8759999999999999},"min":{"1min":0.86299999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":3.5539999999999998,"5min":34.158000000000001,"15min":34.158000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.714,"5min":1.7010000000000001,"15min":1.6160000000000001},"min":{"1min":0.872,"5min":0.62,"15min":0.62},"max":{"1min":5.2539999999999996,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.5629999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":2.351,"15min":1.859},"min":{"1min":1.121,"5min":0.80300000000000005,"15min":0.64600000000000002},"max":{"1min":2.9319999999999999,"5min":33.659999999999997,"15min":33.659999999999997},"last":2.2269999999999999},{"interface":"front","average":{"1min":1.673,"5min":1.913,"15min":1.7949999999999999},"min":{"1min":0.85699999999999998,"5min":0.70299999999999996,"15min":0.60499999999999998},"max":{"1min":3.6629999999999998,"5min":11.882999999999999,"15min":24.289999999999999},"last":1.3660000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64676,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307164,"statfs":{"total":96636764160,"available":96570535936,"internally_reserved":0,"allocated":1662976,"data_stored":1049245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.0580000000000001,"15min":0.97399999999999998},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":1.4730000000000001,"5min":6.1890000000000001,"15min":6.1890000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3540000000000001,"15min":1.2350000000000001},"min":{"1min":0.54200000000000004,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":1.671,"5min":11.470000000000001,"15min":11.470000000000001},"last":1.1020000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2769999999999999,"15min":1.167},"min":{"1min":0.49099999999999999,"5min":0.45500000000000002,"15min":0.27600000000000002},"max":{"1min":1.7490000000000001,"5min":11.59,"15min":11.59},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.3600000000000001,"15min":1.0860000000000001},"min":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.35899999999999999},"max":{"1min":1.331,"5min":14.512,"15min":14.512},"last":0.84999999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.514,"15min":1.3899999999999999},"min":{"1min":0.56599999999999995,"5min":0.50800000000000001,"15min":0.41699999999999998},"max":{"1min":1.7230000000000001,"5min":11.544,"15min":11.544},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.47299999999999998,"5min":0.35199999999999998,"15min":0.29599999999999999},"max":{"1min":1.7669999999999999,"5min":6.1539999999999999,"15min":14.74},"last":1.925}]},{"osd":4,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.948,"15min":1.55},"min":{"1min":0.84299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.444,"5min":32.115000000000002,"15min":32.115000000000002},"last":1.429},{"interface":"front","average":{"1min":1.498,"5min":1.768,"15min":1.681},"min":{"1min":1.139,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.8320000000000001,"5min":11.992000000000001,"15min":15.975},"last":1.679}]},{"osd":5,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.99,"15min":1.6439999999999999},"min":{"1min":0.97399999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.625,"5min":32.174999999999997,"15min":32.174999999999997},"last":1.53},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.7869999999999999,"15min":1.6319999999999999},"min":{"1min":1.016,"5min":0.442,"15min":0.442},"max":{"1min":1.724,"5min":17.225000000000001,"15min":17.225000000000001},"last":1.1850000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.9810000000000001,"15min":1.71},"min":{"1min":1.1120000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.794,"5min":16.228000000000002,"15min":16.228000000000002},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":2.2309999999999999,"15min":1.742},"min":{"1min":0.92800000000000005,"5min":0.67800000000000005,"15min":0.625},"max":{"1min":1.853,"5min":32.216999999999999,"15min":32.216999999999999},"last":1.637}]},{"osd":7,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":2.7080000000000002,"15min":2.0510000000000002},"min":{"1min":1.3069999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.335,"5min":34.131,"15min":34.131},"last":1.474},{"interface":"front","average":{"1min":1.573,"5min":2.2610000000000001,"15min":1.8660000000000001},"min":{"1min":1.073,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.9099999999999999,"5min":21.884,"15min":21.884},"last":1.7829999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:08:18.713 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:08:18.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.972+0000 7f6e7b200640 1 Processor -- start 2024-01-15T18:08:18.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.972+0000 7f6e7b200640 1 -- start start 2024-01-15T18:08:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.974+0000 7f6e7b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e7c0cc990 0x7f6e7c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.974+0000 7f6e7b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c070e50 con 0x7f6e7c143c40 2024-01-15T18:08:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.974+0000 7f6e7b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c144020 con 0x7f6e7c0cc500 2024-01-15T18:08:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.974+0000 7f6e7b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0723c0 con 0x7f6e7c0cc990 2024-01-15T18:08:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.974+0000 7f6e7a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e7c0cc990 0x7f6e7c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.974+0000 7f6e79e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6e7c143c40 0x7f6e7c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33576/0 (socket says 172.21.15.26:33576) 2024-01-15T18:08:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.974+0000 7f6e7a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6e7c0cc990 0x7f6e7c06d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60808/0 (socket says 172.21.15.26:60808) 2024-01-15T18:08:18.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.974+0000 7f6e79e00640 1 -- 172.21.15.26:0/3269327018 learned_addr learned my addr 172.21.15.26:0/3269327018 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.975+0000 7f6e7a800640 1 -- 172.21.15.26:0/3269327018 >> v1:172.21.15.124:6789/0 conn(0x7f6e7c0cc500 legacy=0x7f6e7c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.975+0000 7f6e7a800640 1 -- 172.21.15.26:0/3269327018 >> v1:172.21.15.26:6789/0 conn(0x7f6e7c143c40 legacy=0x7f6e7c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.975+0000 7f6e7a800640 1 -- 172.21.15.26:0/3269327018 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c071e00 con 0x7f6e7c0cc990 2024-01-15T18:08:18.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.976+0000 7f6e7a800640 1 --2- 172.21.15.26:0/3269327018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc990 0x7f6e7c06d870 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009d90 tx=0x7f6e7402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c989ab88b495ce2c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.976+0000 7f6e78a00640 1 -- 172.21.15.26:0/3269327018 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7402cc60 con 0x7f6e7c0cc990 2024-01-15T18:08:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.976+0000 7f6e78a00640 1 -- 172.21.15.26:0/3269327018 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7402cdc0 con 0x7f6e7c0cc990 2024-01-15T18:08:18.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.976+0000 7f6e78a00640 1 -- 172.21.15.26:0/3269327018 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74034660 con 0x7f6e7c0cc990 2024-01-15T18:08:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.977+0000 7f6e7b200640 1 -- 172.21.15.26:0/3269327018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc990 msgr2=0x7f6e7c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.977+0000 7f6e7b200640 1 --2- 172.21.15.26:0/3269327018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc990 0x7f6e7c06d870 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009d90 tx=0x7f6e7402beb0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.977+0000 7f6e7b200640 1 -- 172.21.15.26:0/3269327018 shutdown_connections 2024-01-15T18:08:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.977+0000 7f6e7b200640 1 --2- 172.21.15.26:0/3269327018 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc990 0x7f6e7c06d870 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.977+0000 7f6e7b200640 1 -- 172.21.15.26:0/3269327018 >> 172.21.15.26:0/3269327018 conn(0x7f6e7c06a800 msgr2=0x7f6e7c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.977+0000 7f6e7b200640 1 -- 172.21.15.26:0/3269327018 shutdown_connections 2024-01-15T18:08:18.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.977+0000 7f6e7b200640 1 -- 172.21.15.26:0/3269327018 wait complete. 2024-01-15T18:08:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.980+0000 7f6e7b200640 1 Processor -- start 2024-01-15T18:08:18.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.980+0000 7f6e7b200640 1 -- start start 2024-01-15T18:08:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.981+0000 7f6e7b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc500 0x7f6e7c0adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.981+0000 7f6e7b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e7c0cc990 0x7f6e7c0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.981+0000 7f6e7b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e7c143c40 0x7f6e7c1e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.981+0000 7f6e7b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0adfe0 con 0x7f6e7c0cc500 2024-01-15T18:08:18.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.981+0000 7f6e7b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0b2800 con 0x7f6e7c143c40 2024-01-15T18:08:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.981+0000 7f6e7b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0b2ae0 con 0x7f6e7c0cc990 2024-01-15T18:08:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc500 0x7f6e7c0adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc500 0x7f6e7c0adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60824/0 (socket says 172.21.15.26:60824) 2024-01-15T18:08:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79e00640 1 -- 172.21.15.26:0/1041870271 learned_addr learned my addr 172.21.15.26:0/1041870271 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79400640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e7c0cc990 0x7f6e7c0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79e00640 1 -- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e7c0cc990 msgr2=0x7f6e7c0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79e00640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e7c0cc990 0x7f6e7c0b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79e00640 1 -- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e7c143c40 msgr2=0x7f6e7c1e47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79e00640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e7c143c40 0x7f6e7c1e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79e00640 1 -- 172.21.15.26:0/1041870271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e74009a70 con 0x7f6e7c0cc500 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79400640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e7c0cc990 0x7f6e7c0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.982+0000 7f6e79e00640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc500 0x7f6e7c0adaa0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00c480 tx=0x7f6e6c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.983+0000 7f6e72000640 1 -- 172.21.15.26:0/1041870271 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c005ce0 con 0x7f6e7c0cc500 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.983+0000 7f6e72000640 1 -- 172.21.15.26:0/1041870271 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c005e40 con 0x7f6e7c0cc500 2024-01-15T18:08:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.983+0000 7f6e72000640 1 -- 172.21.15.26:0/1041870271 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0072f0 con 0x7f6e7c0cc500 2024-01-15T18:08:18.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.983+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c0b2dc0 con 0x7f6e7c0cc500 2024-01-15T18:08:18.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.983+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c0b3380 con 0x7f6e7c0cc500 2024-01-15T18:08:18.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.984+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c0b35a0 con 0x7f6e7c0cc500 2024-01-15T18:08:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.985+0000 7f6e72000640 1 -- 172.21.15.26:0/1041870271 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f6e6c007510 con 0x7f6e7c0cc500 2024-01-15T18:08:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.986+0000 7f6e72000640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e5406bfb0 0x7f6e5406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.986+0000 7f6e72000640 1 -- 172.21.15.26:0/1041870271 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6e6c087a90 con 0x7f6e7c0cc500 2024-01-15T18:08:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.987+0000 7f6e79400640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e5406bfb0 0x7f6e5406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:18.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.988+0000 7f6e79400640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e5406bfb0 0x7f6e5406e470 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c1e53b0 tx=0x7f6e64009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:18.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:18.990+0000 7f6e72000640 1 -- 172.21.15.26:0/1041870271 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e6c054400 con 0x7f6e7c0cc500 2024-01-15T18:08:19.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.182+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e7c142950 con 0x7f6e5406bfb0 2024-01-15T18:08:19.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.205+0000 7f6e72000640 1 -- 172.21.15.26:0/1041870271 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286531 (secure 0 0 0) 0x7f6e7c142950 con 0x7f6e5406bfb0 2024-01-15T18:08:19.213 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:19.214 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:08:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e5406bfb0 msgr2=0x7f6e5406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e5406bfb0 0x7f6e5406e470 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c1e53b0 tx=0x7f6e64009040 comp rx=0 tx=0).stop 2024-01-15T18:08:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc500 msgr2=0x7f6e7c0adaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc500 0x7f6e7c0adaa0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00c480 tx=0x7f6e6c00c940 comp rx=0 tx=0).stop 2024-01-15T18:08:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 shutdown_connections 2024-01-15T18:08:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e7c0cc990 0x7f6e7c0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e5406bfb0 0x7f6e5406e470 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e7c143c40 0x7f6e7c1e47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 --2- 172.21.15.26:0/1041870271 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e7c0cc500 0x7f6e7c0adaa0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.211+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 >> 172.21.15.26:0/1041870271 conn(0x7f6e7c06a800 msgr2=0x7f6e7c140090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.212+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 shutdown_connections 2024-01-15T18:08:19.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.212+0000 7f6e7b200640 1 -- 172.21.15.26:0/1041870271 wait complete. 2024-01-15T18:08:19.235 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-01-15T18:08:18.222939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514100,"kb_used_data":9544,"kb_used_omap":12,"kb_used_meta":504435,"kb_avail":754460620,"statfs":{"total":773094113280,"available":772567674880,"internally_reserved":0,"allocated":9773056,"data_stored":4856300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":516542000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001852"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212934+0000","last_change":"2024-01-15T18:08:14.205339+0000","last_active":"2024-01-15T18:08:15.212934+0000","last_peered":"2024-01-15T18:08:15.212934+0000","last_clean":"2024-01-15T18:08:15.212934+0000","last_became_active":"2024-01-15T18:08:14.205009+0000","last_became_peered":"2024-01-15T18:08:14.205009+0000","last_unstale":"2024-01-15T18:08:15.212934+0000","last_undegraded":"2024-01-15T18:08:15.212934+0000","last_fullsized":"2024-01-15T18:08:15.212934+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:35.060937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259421+0000","last_change":"2024-01-15T18:08:14.211597+0000","last_active":"2024-01-15T18:08:16.259421+0000","last_peered":"2024-01-15T18:08:16.259421+0000","last_clean":"2024-01-15T18:08:16.259421+0000","last_became_active":"2024-01-15T18:08:14.211327+0000","last_became_peered":"2024-01-15T18:08:14.211327+0000","last_unstale":"2024-01-15T18:08:16.259421+0000","last_undegraded":"2024-01-15T18:08:16.259421+0000","last_fullsized":"2024-01-15T18:08:16.259421+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:40.852891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257946+0000","last_change":"2024-01-15T18:08:14.212643+0000","last_active":"2024-01-15T18:08:16.257946+0000","last_peered":"2024-01-15T18:08:16.257946+0000","last_clean":"2024-01-15T18:08:16.257946+0000","last_became_active":"2024-01-15T18:08:14.212269+0000","last_became_peered":"2024-01-15T18:08:14.212269+0000","last_unstale":"2024-01-15T18:08:16.257946+0000","last_undegraded":"2024-01-15T18:08:16.257946+0000","last_fullsized":"2024-01-15T18:08:16.257946+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:45:02.732821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257846+0000","last_change":"2024-01-15T18:08:14.202421+0000","last_active":"2024-01-15T18:08:16.257846+0000","last_peered":"2024-01-15T18:08:16.257846+0000","last_clean":"2024-01-15T18:08:16.257846+0000","last_became_active":"2024-01-15T18:08:14.202076+0000","last_became_peered":"2024-01-15T18:08:14.202076+0000","last_unstale":"2024-01-15T18:08:16.257846+0000","last_undegraded":"2024-01-15T18:08:16.257846+0000","last_fullsized":"2024-01-15T18:08:16.257846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:52.451530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213065+0000","last_change":"2024-01-15T18:08:14.201420+0000","last_active":"2024-01-15T18:08:15.213065+0000","last_peered":"2024-01-15T18:08:15.213065+0000","last_clean":"2024-01-15T18:08:15.213065+0000","last_became_active":"2024-01-15T18:08:14.199479+0000","last_became_peered":"2024-01-15T18:08:14.199479+0000","last_unstale":"2024-01-15T18:08:15.213065+0000","last_undegraded":"2024-01-15T18:08:15.213065+0000","last_fullsized":"2024-01-15T18:08:15.213065+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:02.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258511+0000","last_change":"2024-01-15T18:08:14.221652+0000","last_active":"2024-01-15T18:08:16.258511+0000","last_peered":"2024-01-15T18:08:16.258511+0000","last_clean":"2024-01-15T18:08:16.258511+0000","last_became_active":"2024-01-15T18:08:14.220744+0000","last_became_peered":"2024-01-15T18:08:14.220744+0000","last_unstale":"2024-01-15T18:08:16.258511+0000","last_undegraded":"2024-01-15T18:08:16.258511+0000","last_fullsized":"2024-01-15T18:08:16.258511+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:29:34.345687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.256741+0000","last_change":"2024-01-15T18:08:14.211362+0000","last_active":"2024-01-15T18:08:16.256741+0000","last_peered":"2024-01-15T18:08:16.256741+0000","last_clean":"2024-01-15T18:08:16.256741+0000","last_became_active":"2024-01-15T18:08:14.211099+0000","last_became_peered":"2024-01-15T18:08:14.211099+0000","last_unstale":"2024-01-15T18:08:16.256741+0000","last_undegraded":"2024-01-15T18:08:16.256741+0000","last_fullsized":"2024-01-15T18:08:16.256741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:22:47.996600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258644+0000","last_change":"2024-01-15T18:08:14.220426+0000","last_active":"2024-01-15T18:08:16.258644+0000","last_peered":"2024-01-15T18:08:16.258644+0000","last_clean":"2024-01-15T18:08:16.258644+0000","last_became_active":"2024-01-15T18:08:14.219622+0000","last_became_peered":"2024-01-15T18:08:14.219622+0000","last_unstale":"2024-01-15T18:08:16.258644+0000","last_undegraded":"2024-01-15T18:08:16.258644+0000","last_fullsized":"2024-01-15T18:08:16.258644+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:35.060937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259135+0000","last_change":"2024-01-15T18:08:14.213300+0000","last_active":"2024-01-15T18:08:16.259135+0000","last_peered":"2024-01-15T18:08:16.259135+0000","last_clean":"2024-01-15T18:08:16.259135+0000","last_became_active":"2024-01-15T18:08:14.213029+0000","last_became_peered":"2024-01-15T18:08:14.213029+0000","last_unstale":"2024-01-15T18:08:16.259135+0000","last_undegraded":"2024-01-15T18:08:16.259135+0000","last_fullsized":"2024-01-15T18:08:16.259135+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:00:03.016857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212888+0000","last_change":"2024-01-15T18:08:14.217683+0000","last_active":"2024-01-15T18:08:15.212888+0000","last_peered":"2024-01-15T18:08:15.212888+0000","last_clean":"2024-01-15T18:08:15.212888+0000","last_became_active":"2024-01-15T18:08:14.217330+0000","last_became_peered":"2024-01-15T18:08:14.217330+0000","last_unstale":"2024-01-15T18:08:15.212888+0000","last_undegraded":"2024-01-15T18:08:15.212888+0000","last_fullsized":"2024-01-15T18:08:15.212888+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:29:34.345687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259099+0000","last_change":"2024-01-15T18:08:14.212851+0000","last_active":"2024-01-15T18:08:16.259099+0000","last_peered":"2024-01-15T18:08:16.259099+0000","last_clean":"2024-01-15T18:08:16.259099+0000","last_became_active":"2024-01-15T18:08:14.212510+0000","last_became_peered":"2024-01-15T18:08:14.212510+0000","last_unstale":"2024-01-15T18:08:16.259099+0000","last_undegraded":"2024-01-15T18:08:16.259099+0000","last_fullsized":"2024-01-15T18:08:16.259099+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:11:31.868467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210164+0000","last_change":"2024-01-15T18:08:14.216968+0000","last_active":"2024-01-15T18:08:15.210164+0000","last_peered":"2024-01-15T18:08:15.210164+0000","last_clean":"2024-01-15T18:08:15.210164+0000","last_became_active":"2024-01-15T18:08:14.215796+0000","last_became_peered":"2024-01-15T18:08:14.215796+0000","last_unstale":"2024-01-15T18:08:15.210164+0000","last_undegraded":"2024-01-15T18:08:15.210164+0000","last_fullsized":"2024-01-15T18:08:15.210164+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:01:37.514417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257762+0000","last_change":"2024-01-15T18:08:14.211568+0000","last_active":"2024-01-15T18:08:16.257762+0000","last_peered":"2024-01-15T18:08:16.257762+0000","last_clean":"2024-01-15T18:08:16.257762+0000","last_became_active":"2024-01-15T18:08:14.211256+0000","last_became_peered":"2024-01-15T18:08:14.211256+0000","last_unstale":"2024-01-15T18:08:16.257762+0000","last_undegraded":"2024-01-15T18:08:16.257762+0000","last_fullsized":"2024-01-15T18:08:16.257762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:16.437633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257635+0000","last_change":"2024-01-15T18:08:14.201090+0000","last_active":"2024-01-15T18:08:16.257635+0000","last_peered":"2024-01-15T18:08:16.257635+0000","last_clean":"2024-01-15T18:08:16.257635+0000","last_became_active":"2024-01-15T18:08:14.200793+0000","last_became_peered":"2024-01-15T18:08:14.200793+0000","last_unstale":"2024-01-15T18:08:16.257635+0000","last_undegraded":"2024-01-15T18:08:16.257635+0000","last_fullsized":"2024-01-15T18:08:16.257635+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:19:35.807826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.203802+0000","last_change":"2024-01-15T18:08:14.207360+0000","last_active":"2024-01-15T18:08:15.203802+0000","last_peered":"2024-01-15T18:08:15.203802+0000","last_clean":"2024-01-15T18:08:15.203802+0000","last_became_active":"2024-01-15T18:08:14.207079+0000","last_became_peered":"2024-01-15T18:08:14.207079+0000","last_unstale":"2024-01-15T18:08:15.203802+0000","last_undegraded":"2024-01-15T18:08:15.203802+0000","last_fullsized":"2024-01-15T18:08:15.203802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:51:08.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.211040+0000","last_change":"2024-01-15T18:08:14.208857+0000","last_active":"2024-01-15T18:08:15.211040+0000","last_peered":"2024-01-15T18:08:15.211040+0000","last_clean":"2024-01-15T18:08:15.211040+0000","last_became_active":"2024-01-15T18:08:14.207171+0000","last_became_peered":"2024-01-15T18:08:14.207171+0000","last_unstale":"2024-01-15T18:08:15.211040+0000","last_undegraded":"2024-01-15T18:08:15.211040+0000","last_fullsized":"2024-01-15T18:08:15.211040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:20:41.894318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212518+0000","last_change":"2024-01-15T18:08:14.201493+0000","last_active":"2024-01-15T18:08:15.212518+0000","last_peered":"2024-01-15T18:08:15.212518+0000","last_clean":"2024-01-15T18:08:15.212518+0000","last_became_active":"2024-01-15T18:08:14.200900+0000","last_became_peered":"2024-01-15T18:08:14.200900+0000","last_unstale":"2024-01-15T18:08:15.212518+0000","last_undegraded":"2024-01-15T18:08:15.212518+0000","last_fullsized":"2024-01-15T18:08:15.212518+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:29:34.345687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258367+0000","last_change":"2024-01-15T18:08:14.207516+0000","last_active":"2024-01-15T18:08:16.258367+0000","last_peered":"2024-01-15T18:08:16.258367+0000","last_clean":"2024-01-15T18:08:16.258367+0000","last_became_active":"2024-01-15T18:08:14.207247+0000","last_became_peered":"2024-01-15T18:08:14.207247+0000","last_unstale":"2024-01-15T18:08:16.258367+0000","last_undegraded":"2024-01-15T18:08:16.258367+0000","last_fullsized":"2024-01-15T18:08:16.258367+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:27:06.733990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.088810+0000","last_change":"2024-01-15T18:08:14.222306+0000","last_active":"2024-01-15T18:08:16.088810+0000","last_peered":"2024-01-15T18:08:16.088810+0000","last_clean":"2024-01-15T18:08:16.088810+0000","last_became_active":"2024-01-15T18:08:14.222037+0000","last_became_peered":"2024-01-15T18:08:14.222037+0000","last_unstale":"2024-01-15T18:08:16.088810+0000","last_undegraded":"2024-01-15T18:08:16.088810+0000","last_fullsized":"2024-01-15T18:08:16.088810+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:29:38.896192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258040+0000","last_change":"2024-01-15T18:08:14.214903+0000","last_active":"2024-01-15T18:08:16.258040+0000","last_peered":"2024-01-15T18:08:16.258040+0000","last_clean":"2024-01-15T18:08:16.258040+0000","last_became_active":"2024-01-15T18:08:14.214223+0000","last_became_peered":"2024-01-15T18:08:14.214223+0000","last_unstale":"2024-01-15T18:08:16.258040+0000","last_undegraded":"2024-01-15T18:08:16.258040+0000","last_fullsized":"2024-01-15T18:08:16.258040+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:06:36.312091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258073+0000","last_change":"2024-01-15T18:08:14.210307+0000","last_active":"2024-01-15T18:08:16.258073+0000","last_peered":"2024-01-15T18:08:16.258073+0000","last_clean":"2024-01-15T18:08:16.258073+0000","last_became_active":"2024-01-15T18:08:14.205910+0000","last_became_peered":"2024-01-15T18:08:14.205910+0000","last_unstale":"2024-01-15T18:08:16.258073+0000","last_undegraded":"2024-01-15T18:08:16.258073+0000","last_fullsized":"2024-01-15T18:08:16.258073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:37.149934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257695+0000","last_change":"2024-01-15T18:08:14.214724+0000","last_active":"2024-01-15T18:08:16.257695+0000","last_peered":"2024-01-15T18:08:16.257695+0000","last_clean":"2024-01-15T18:08:16.257695+0000","last_became_active":"2024-01-15T18:08:14.214219+0000","last_became_peered":"2024-01-15T18:08:14.214219+0000","last_unstale":"2024-01-15T18:08:16.257695+0000","last_undegraded":"2024-01-15T18:08:16.257695+0000","last_fullsized":"2024-01-15T18:08:16.257695+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:45:29.726519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210213+0000","last_change":"2024-01-15T18:08:14.206870+0000","last_active":"2024-01-15T18:08:15.210213+0000","last_peered":"2024-01-15T18:08:15.210213+0000","last_clean":"2024-01-15T18:08:15.210213+0000","last_became_active":"2024-01-15T18:08:14.206593+0000","last_became_peered":"2024-01-15T18:08:14.206593+0000","last_unstale":"2024-01-15T18:08:15.210213+0000","last_undegraded":"2024-01-15T18:08:15.210213+0000","last_fullsized":"2024-01-15T18:08:15.210213+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:20:41.894318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.256452+0000","last_change":"2024-01-15T18:08:14.208074+0000","last_active":"2024-01-15T18:08:16.256452+0000","last_peered":"2024-01-15T18:08:16.256452+0000","last_clean":"2024-01-15T18:08:16.256452+0000","last_became_active":"2024-01-15T18:08:14.203256+0000","last_became_peered":"2024-01-15T18:08:14.203256+0000","last_unstale":"2024-01-15T18:08:16.256452+0000","last_undegraded":"2024-01-15T18:08:16.256452+0000","last_fullsized":"2024-01-15T18:08:16.256452+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:28.203925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259513+0000","last_change":"2024-01-15T18:08:14.201039+0000","last_active":"2024-01-15T18:08:16.259513+0000","last_peered":"2024-01-15T18:08:16.259513+0000","last_clean":"2024-01-15T18:08:16.259513+0000","last_became_active":"2024-01-15T18:08:14.200739+0000","last_became_peered":"2024-01-15T18:08:14.200739+0000","last_unstale":"2024-01-15T18:08:16.259513+0000","last_undegraded":"2024-01-15T18:08:16.259513+0000","last_fullsized":"2024-01-15T18:08:16.259513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:32:48.450924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258169+0000","last_change":"2024-01-15T18:08:14.218802+0000","last_active":"2024-01-15T18:08:16.258169+0000","last_peered":"2024-01-15T18:08:16.258169+0000","last_clean":"2024-01-15T18:08:16.258169+0000","last_became_active":"2024-01-15T18:08:14.218527+0000","last_became_peered":"2024-01-15T18:08:14.218527+0000","last_unstale":"2024-01-15T18:08:16.258169+0000","last_undegraded":"2024-01-15T18:08:16.258169+0000","last_fullsized":"2024-01-15T18:08:16.258169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:13:58.531844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212596+0000","last_change":"2024-01-15T18:08:14.201989+0000","last_active":"2024-01-15T18:08:15.212596+0000","last_peered":"2024-01-15T18:08:15.212596+0000","last_clean":"2024-01-15T18:08:15.212596+0000","last_became_active":"2024-01-15T18:08:14.201682+0000","last_became_peered":"2024-01-15T18:08:14.201682+0000","last_unstale":"2024-01-15T18:08:15.212596+0000","last_undegraded":"2024-01-15T18:08:15.212596+0000","last_fullsized":"2024-01-15T18:08:15.212596+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:35.060937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210325+0000","last_change":"2024-01-15T18:08:14.204423+0000","last_active":"2024-01-15T18:08:15.210325+0000","last_peered":"2024-01-15T18:08:15.210325+0000","last_clean":"2024-01-15T18:08:15.210325+0000","last_became_active":"2024-01-15T18:08:14.204103+0000","last_became_peered":"2024-01-15T18:08:14.204103+0000","last_unstale":"2024-01-15T18:08:15.210325+0000","last_undegraded":"2024-01-15T18:08:15.210325+0000","last_fullsized":"2024-01-15T18:08:15.210325+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:39:31.130502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257995+0000","last_change":"2024-01-15T18:08:14.204294+0000","last_active":"2024-01-15T18:08:16.257995+0000","last_peered":"2024-01-15T18:08:16.257995+0000","last_clean":"2024-01-15T18:08:16.257995+0000","last_became_active":"2024-01-15T18:08:14.204033+0000","last_became_peered":"2024-01-15T18:08:14.204033+0000","last_unstale":"2024-01-15T18:08:16.257995+0000","last_undegraded":"2024-01-15T18:08:16.257995+0000","last_fullsized":"2024-01-15T18:08:16.257995+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:58:53.420076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257851+0000","last_change":"2024-01-15T18:08:14.205679+0000","last_active":"2024-01-15T18:08:16.257851+0000","last_peered":"2024-01-15T18:08:16.257851+0000","last_clean":"2024-01-15T18:08:16.257851+0000","last_became_active":"2024-01-15T18:08:14.203591+0000","last_became_peered":"2024-01-15T18:08:14.203591+0000","last_unstale":"2024-01-15T18:08:16.257851+0000","last_undegraded":"2024-01-15T18:08:16.257851+0000","last_fullsized":"2024-01-15T18:08:16.257851+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:09:04.507538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.211290+0000","last_change":"2024-01-15T18:08:14.205902+0000","last_active":"2024-01-15T18:08:15.211290+0000","last_peered":"2024-01-15T18:08:15.211290+0000","last_clean":"2024-01-15T18:08:15.211290+0000","last_became_active":"2024-01-15T18:08:14.203710+0000","last_became_peered":"2024-01-15T18:08:14.203710+0000","last_unstale":"2024-01-15T18:08:15.211290+0000","last_undegraded":"2024-01-15T18:08:15.211290+0000","last_fullsized":"2024-01-15T18:08:15.211290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:39:31.130502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.209766+0000","last_change":"2024-01-15T18:08:14.200416+0000","last_active":"2024-01-15T18:08:15.209766+0000","last_peered":"2024-01-15T18:08:15.209766+0000","last_clean":"2024-01-15T18:08:15.209766+0000","last_became_active":"2024-01-15T18:08:14.200095+0000","last_became_peered":"2024-01-15T18:08:14.200095+0000","last_unstale":"2024-01-15T18:08:15.209766+0000","last_undegraded":"2024-01-15T18:08:15.209766+0000","last_fullsized":"2024-01-15T18:08:15.209766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:20:37.554185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.211958+0000","last_change":"2024-01-15T18:08:14.207686+0000","last_active":"2024-01-15T18:08:15.211958+0000","last_peered":"2024-01-15T18:08:15.211958+0000","last_clean":"2024-01-15T18:08:15.211958+0000","last_became_active":"2024-01-15T18:08:14.205835+0000","last_became_peered":"2024-01-15T18:08:14.205835+0000","last_unstale":"2024-01-15T18:08:15.211958+0000","last_undegraded":"2024-01-15T18:08:15.211958+0000","last_fullsized":"2024-01-15T18:08:15.211958+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:38:54.083263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.217198+0000","last_change":"2024-01-15T18:08:13.201391+0000","last_active":"2024-01-15T18:08:14.217198+0000","last_peered":"2024-01-15T18:08:14.217198+0000","last_clean":"2024-01-15T18:08:14.217198+0000","last_became_active":"2024-01-15T18:08:13.201072+0000","last_became_peered":"2024-01-15T18:08:13.201072+0000","last_unstale":"2024-01-15T18:08:14.217198+0000","last_undegraded":"2024-01-15T18:08:14.217198+0000","last_fullsized":"2024-01-15T18:08:14.217198+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:31.926730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213104+0000","last_change":"2024-01-15T18:08:14.213109+0000","last_active":"2024-01-15T18:08:15.213104+0000","last_peered":"2024-01-15T18:08:15.213104+0000","last_clean":"2024-01-15T18:08:15.213104+0000","last_became_active":"2024-01-15T18:08:14.207157+0000","last_became_peered":"2024-01-15T18:08:14.207157+0000","last_unstale":"2024-01-15T18:08:15.213104+0000","last_undegraded":"2024-01-15T18:08:15.213104+0000","last_fullsized":"2024-01-15T18:08:15.213104+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:33.977943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.220065+0000","last_change":"2024-01-15T18:08:13.198543+0000","last_active":"2024-01-15T18:08:14.220065+0000","last_peered":"2024-01-15T18:08:14.220065+0000","last_clean":"2024-01-15T18:08:14.220065+0000","last_became_active":"2024-01-15T18:08:13.197094+0000","last_became_peered":"2024-01-15T18:08:13.197094+0000","last_unstale":"2024-01-15T18:08:14.220065+0000","last_undegraded":"2024-01-15T18:08:14.220065+0000","last_fullsized":"2024-01-15T18:08:14.220065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:52:57.254176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210526+0000","last_change":"2024-01-15T18:08:14.217169+0000","last_active":"2024-01-15T18:08:15.210526+0000","last_peered":"2024-01-15T18:08:15.210526+0000","last_clean":"2024-01-15T18:08:15.210526+0000","last_became_active":"2024-01-15T18:08:14.216179+0000","last_became_peered":"2024-01-15T18:08:14.216179+0000","last_unstale":"2024-01-15T18:08:15.210526+0000","last_undegraded":"2024-01-15T18:08:15.210526+0000","last_fullsized":"2024-01-15T18:08:15.210526+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:38:54.083263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.224457+0000","last_change":"2024-01-15T18:08:13.206368+0000","last_active":"2024-01-15T18:08:14.224457+0000","last_peered":"2024-01-15T18:08:14.224457+0000","last_clean":"2024-01-15T18:08:14.224457+0000","last_became_active":"2024-01-15T18:08:13.205180+0000","last_became_peered":"2024-01-15T18:08:13.205180+0000","last_unstale":"2024-01-15T18:08:14.224457+0000","last_undegraded":"2024-01-15T18:08:14.224457+0000","last_fullsized":"2024-01-15T18:08:14.224457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:01.034813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257928+0000","last_change":"2024-01-15T18:08:14.222292+0000","last_active":"2024-01-15T18:08:16.257928+0000","last_peered":"2024-01-15T18:08:16.257928+0000","last_clean":"2024-01-15T18:08:16.257928+0000","last_became_active":"2024-01-15T18:08:14.221624+0000","last_became_peered":"2024-01-15T18:08:14.221624+0000","last_unstale":"2024-01-15T18:08:16.257928+0000","last_undegraded":"2024-01-15T18:08:16.257928+0000","last_fullsized":"2024-01-15T18:08:16.257928+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:13:07.165947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.225102+0000","last_change":"2024-01-15T18:08:13.206418+0000","last_active":"2024-01-15T18:08:14.225102+0000","last_peered":"2024-01-15T18:08:14.225102+0000","last_clean":"2024-01-15T18:08:14.225102+0000","last_became_active":"2024-01-15T18:08:13.205259+0000","last_became_peered":"2024-01-15T18:08:13.205259+0000","last_unstale":"2024-01-15T18:08:14.225102+0000","last_undegraded":"2024-01-15T18:08:14.225102+0000","last_fullsized":"2024-01-15T18:08:14.225102+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:29:33.339575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257985+0000","last_change":"2024-01-15T18:08:14.205937+0000","last_active":"2024-01-15T18:08:16.257985+0000","last_peered":"2024-01-15T18:08:16.257985+0000","last_clean":"2024-01-15T18:08:16.257985+0000","last_became_active":"2024-01-15T18:08:14.205244+0000","last_became_peered":"2024-01-15T18:08:14.205244+0000","last_unstale":"2024-01-15T18:08:16.257985+0000","last_undegraded":"2024-01-15T18:08:16.257985+0000","last_fullsized":"2024-01-15T18:08:16.257985+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:00:57.294299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.217464+0000","last_change":"2024-01-15T18:08:13.200457+0000","last_active":"2024-01-15T18:08:14.217464+0000","last_peered":"2024-01-15T18:08:14.217464+0000","last_clean":"2024-01-15T18:08:14.217464+0000","last_became_active":"2024-01-15T18:08:13.200168+0000","last_became_peered":"2024-01-15T18:08:13.200168+0000","last_unstale":"2024-01-15T18:08:14.217464+0000","last_undegraded":"2024-01-15T18:08:14.217464+0000","last_fullsized":"2024-01-15T18:08:14.217464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:31:16.895925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213385+0000","last_change":"2024-01-15T18:08:14.214541+0000","last_active":"2024-01-15T18:08:15.213385+0000","last_peered":"2024-01-15T18:08:15.213385+0000","last_clean":"2024-01-15T18:08:15.213385+0000","last_became_active":"2024-01-15T18:08:14.214256+0000","last_became_peered":"2024-01-15T18:08:14.214256+0000","last_unstale":"2024-01-15T18:08:15.213385+0000","last_undegraded":"2024-01-15T18:08:15.213385+0000","last_fullsized":"2024-01-15T18:08:15.213385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:00:03.016857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.220273+0000","last_change":"2024-01-15T18:08:13.191881+0000","last_active":"2024-01-15T18:08:14.220273+0000","last_peered":"2024-01-15T18:08:14.220273+0000","last_clean":"2024-01-15T18:08:14.220273+0000","last_became_active":"2024-01-15T18:08:13.191220+0000","last_became_peered":"2024-01-15T18:08:13.191220+0000","last_unstale":"2024-01-15T18:08:14.220273+0000","last_undegraded":"2024-01-15T18:08:14.220273+0000","last_fullsized":"2024-01-15T18:08:14.220273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:20:36.548073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257169+0000","last_change":"2024-01-15T18:08:14.221203+0000","last_active":"2024-01-15T18:08:16.257169+0000","last_peered":"2024-01-15T18:08:16.257169+0000","last_clean":"2024-01-15T18:08:16.257169+0000","last_became_active":"2024-01-15T18:08:14.220378+0000","last_became_peered":"2024-01-15T18:08:14.220378+0000","last_unstale":"2024-01-15T18:08:16.257169+0000","last_undegraded":"2024-01-15T18:08:16.257169+0000","last_fullsized":"2024-01-15T18:08:16.257169+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:20:18.545134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.199249+0000","last_change":"2024-01-15T18:08:13.202372+0000","last_active":"2024-01-15T18:08:14.199249+0000","last_peered":"2024-01-15T18:08:14.199249+0000","last_clean":"2024-01-15T18:08:14.199249+0000","last_became_active":"2024-01-15T18:08:13.202106+0000","last_became_peered":"2024-01-15T18:08:13.202106+0000","last_unstale":"2024-01-15T18:08:14.199249+0000","last_undegraded":"2024-01-15T18:08:14.199249+0000","last_fullsized":"2024-01-15T18:08:14.199249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:23:34.054825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.203903+0000","last_change":"2024-01-15T18:08:14.203670+0000","last_active":"2024-01-15T18:08:15.203903+0000","last_peered":"2024-01-15T18:08:15.203903+0000","last_clean":"2024-01-15T18:08:15.203903+0000","last_became_active":"2024-01-15T18:08:14.203214+0000","last_became_peered":"2024-01-15T18:08:14.203214+0000","last_unstale":"2024-01-15T18:08:15.203903+0000","last_undegraded":"2024-01-15T18:08:15.203903+0000","last_fullsized":"2024-01-15T18:08:15.203903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:09.872734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.187069+0000","last_change":"2024-01-15T18:08:13.204304+0000","last_active":"2024-01-15T18:08:14.187069+0000","last_peered":"2024-01-15T18:08:14.187069+0000","last_clean":"2024-01-15T18:08:14.187069+0000","last_became_active":"2024-01-15T18:08:13.204000+0000","last_became_peered":"2024-01-15T18:08:13.204000+0000","last_unstale":"2024-01-15T18:08:14.187069+0000","last_undegraded":"2024-01-15T18:08:14.187069+0000","last_fullsized":"2024-01-15T18:08:14.187069+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:20:32.338335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.204375+0000","last_change":"2024-01-15T18:08:14.214098+0000","last_active":"2024-01-15T18:08:15.204375+0000","last_peered":"2024-01-15T18:08:15.204375+0000","last_clean":"2024-01-15T18:08:15.204375+0000","last_became_active":"2024-01-15T18:08:14.213768+0000","last_became_peered":"2024-01-15T18:08:14.213768+0000","last_unstale":"2024-01-15T18:08:15.204375+0000","last_undegraded":"2024-01-15T18:08:15.204375+0000","last_fullsized":"2024-01-15T18:08:15.204375+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:02:11.079420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.196443+0000","last_change":"2024-01-15T18:08:13.197699+0000","last_active":"2024-01-15T18:08:14.196443+0000","last_peered":"2024-01-15T18:08:14.196443+0000","last_clean":"2024-01-15T18:08:14.196443+0000","last_became_active":"2024-01-15T18:08:13.197224+0000","last_became_peered":"2024-01-15T18:08:13.197224+0000","last_unstale":"2024-01-15T18:08:14.196443+0000","last_undegraded":"2024-01-15T18:08:14.196443+0000","last_fullsized":"2024-01-15T18:08:14.196443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:38:22.415229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259470+0000","last_change":"2024-01-15T18:08:14.220949+0000","last_active":"2024-01-15T18:08:16.259470+0000","last_peered":"2024-01-15T18:08:16.259470+0000","last_clean":"2024-01-15T18:08:16.259470+0000","last_became_active":"2024-01-15T18:08:14.220670+0000","last_became_peered":"2024-01-15T18:08:14.220670+0000","last_unstale":"2024-01-15T18:08:16.259470+0000","last_undegraded":"2024-01-15T18:08:16.259470+0000","last_fullsized":"2024-01-15T18:08:16.259470+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:45:17.395330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.198092+0000","last_change":"2024-01-15T18:08:13.204031+0000","last_active":"2024-01-15T18:08:14.198092+0000","last_peered":"2024-01-15T18:08:14.198092+0000","last_clean":"2024-01-15T18:08:14.198092+0000","last_became_active":"2024-01-15T18:08:13.203462+0000","last_became_peered":"2024-01-15T18:08:13.203462+0000","last_unstale":"2024-01-15T18:08:14.198092+0000","last_undegraded":"2024-01-15T18:08:14.198092+0000","last_fullsized":"2024-01-15T18:08:14.198092+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:52:57.254176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.088636+0000","last_change":"2024-01-15T18:08:14.225446+0000","last_active":"2024-01-15T18:08:16.088636+0000","last_peered":"2024-01-15T18:08:16.088636+0000","last_clean":"2024-01-15T18:08:16.088636+0000","last_became_active":"2024-01-15T18:08:14.224731+0000","last_became_peered":"2024-01-15T18:08:14.224731+0000","last_unstale":"2024-01-15T18:08:16.088636+0000","last_undegraded":"2024-01-15T18:08:16.088636+0000","last_fullsized":"2024-01-15T18:08:16.088636+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:00:47.935443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.218816+0000","last_change":"2024-01-15T18:08:13.198087+0000","last_active":"2024-01-15T18:08:14.218816+0000","last_peered":"2024-01-15T18:08:14.218816+0000","last_clean":"2024-01-15T18:08:14.218816+0000","last_became_active":"2024-01-15T18:08:13.197428+0000","last_became_peered":"2024-01-15T18:08:13.197428+0000","last_unstale":"2024-01-15T18:08:14.218816+0000","last_undegraded":"2024-01-15T18:08:14.218816+0000","last_fullsized":"2024-01-15T18:08:14.218816+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:18:16.880458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212508+0000","last_change":"2024-01-15T18:08:14.207758+0000","last_active":"2024-01-15T18:08:15.212508+0000","last_peered":"2024-01-15T18:08:15.212508+0000","last_clean":"2024-01-15T18:08:15.212508+0000","last_became_active":"2024-01-15T18:08:14.207382+0000","last_became_peered":"2024-01-15T18:08:14.207382+0000","last_unstale":"2024-01-15T18:08:15.212508+0000","last_undegraded":"2024-01-15T18:08:15.212508+0000","last_fullsized":"2024-01-15T18:08:15.212508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:32:15.267128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.226769+0000","last_change":"2024-01-15T18:08:13.207430+0000","last_active":"2024-01-15T18:08:14.226769+0000","last_peered":"2024-01-15T18:08:14.226769+0000","last_clean":"2024-01-15T18:08:14.226769+0000","last_became_active":"2024-01-15T18:08:13.207137+0000","last_became_peered":"2024-01-15T18:08:13.207137+0000","last_unstale":"2024-01-15T18:08:14.226769+0000","last_undegraded":"2024-01-15T18:08:14.226769+0000","last_fullsized":"2024-01-15T18:08:14.226769+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:21:32.238092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.088363+0000","last_change":"2024-01-15T18:08:14.200200+0000","last_active":"2024-01-15T18:08:16.088363+0000","last_peered":"2024-01-15T18:08:16.088363+0000","last_clean":"2024-01-15T18:08:16.088363+0000","last_became_active":"2024-01-15T18:08:14.199890+0000","last_became_peered":"2024-01-15T18:08:14.199890+0000","last_unstale":"2024-01-15T18:08:16.088363+0000","last_undegraded":"2024-01-15T18:08:16.088363+0000","last_fullsized":"2024-01-15T18:08:16.088363+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:45:17.395330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.226141+0000","last_change":"2024-01-15T18:08:13.202751+0000","last_active":"2024-01-15T18:08:14.226141+0000","last_peered":"2024-01-15T18:08:14.226141+0000","last_clean":"2024-01-15T18:08:14.226141+0000","last_became_active":"2024-01-15T18:08:13.202104+0000","last_became_peered":"2024-01-15T18:08:13.202104+0000","last_unstale":"2024-01-15T18:08:14.226141+0000","last_undegraded":"2024-01-15T18:08:14.226141+0000","last_fullsized":"2024-01-15T18:08:14.226141+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:22:46.990488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257731+0000","last_change":"2024-01-15T18:08:14.213649+0000","last_active":"2024-01-15T18:08:16.257731+0000","last_peered":"2024-01-15T18:08:16.257731+0000","last_clean":"2024-01-15T18:08:16.257731+0000","last_became_active":"2024-01-15T18:08:14.213255+0000","last_became_peered":"2024-01-15T18:08:14.213255+0000","last_unstale":"2024-01-15T18:08:16.257731+0000","last_undegraded":"2024-01-15T18:08:16.257731+0000","last_fullsized":"2024-01-15T18:08:16.257731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:26:42.957652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.201096+0000","last_change":"2024-01-15T18:08:13.194370+0000","last_active":"2024-01-15T18:08:14.201096+0000","last_peered":"2024-01-15T18:08:14.201096+0000","last_clean":"2024-01-15T18:08:14.201096+0000","last_became_active":"2024-01-15T18:08:13.193625+0000","last_became_peered":"2024-01-15T18:08:13.193625+0000","last_unstale":"2024-01-15T18:08:14.201096+0000","last_undegraded":"2024-01-15T18:08:14.201096+0000","last_fullsized":"2024-01-15T18:08:14.201096+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:44:39.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257160+0000","last_change":"2024-01-15T18:08:14.217681+0000","last_active":"2024-01-15T18:08:16.257160+0000","last_peered":"2024-01-15T18:08:16.257160+0000","last_clean":"2024-01-15T18:08:16.257160+0000","last_became_active":"2024-01-15T18:08:14.216412+0000","last_became_peered":"2024-01-15T18:08:14.216412+0000","last_unstale":"2024-01-15T18:08:16.257160+0000","last_undegraded":"2024-01-15T18:08:16.257160+0000","last_fullsized":"2024-01-15T18:08:16.257160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:37.149934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.211209+0000","last_change":"2024-01-15T18:08:13.203484+0000","last_active":"2024-01-15T18:08:14.211209+0000","last_peered":"2024-01-15T18:08:14.211209+0000","last_clean":"2024-01-15T18:08:14.211209+0000","last_became_active":"2024-01-15T18:08:13.203206+0000","last_became_peered":"2024-01-15T18:08:13.203206+0000","last_unstale":"2024-01-15T18:08:14.211209+0000","last_undegraded":"2024-01-15T18:08:14.211209+0000","last_fullsized":"2024-01-15T18:08:14.211209+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:32:14.261016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257535+0000","last_change":"2024-01-15T18:08:14.211524+0000","last_active":"2024-01-15T18:08:16.257535+0000","last_peered":"2024-01-15T18:08:16.257535+0000","last_clean":"2024-01-15T18:08:16.257535+0000","last_became_active":"2024-01-15T18:08:14.211249+0000","last_became_peered":"2024-01-15T18:08:14.211249+0000","last_unstale":"2024-01-15T18:08:16.257535+0000","last_undegraded":"2024-01-15T18:08:16.257535+0000","last_fullsized":"2024-01-15T18:08:16.257535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:27:06.733990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.225460+0000","last_change":"2024-01-15T18:08:13.208116+0000","last_active":"2024-01-15T18:08:14.225460+0000","last_peered":"2024-01-15T18:08:14.225460+0000","last_clean":"2024-01-15T18:08:14.225460+0000","last_became_active":"2024-01-15T18:08:13.207828+0000","last_became_peered":"2024-01-15T18:08:13.207828+0000","last_unstale":"2024-01-15T18:08:14.225460+0000","last_undegraded":"2024-01-15T18:08:14.225460+0000","last_fullsized":"2024-01-15T18:08:14.225460+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:21:32.238092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258472+0000","last_change":"2024-01-15T18:08:14.215780+0000","last_active":"2024-01-15T18:08:16.258472+0000","last_peered":"2024-01-15T18:08:16.258472+0000","last_clean":"2024-01-15T18:08:16.258472+0000","last_became_active":"2024-01-15T18:08:14.215318+0000","last_became_peered":"2024-01-15T18:08:14.215318+0000","last_unstale":"2024-01-15T18:08:16.258472+0000","last_undegraded":"2024-01-15T18:08:16.258472+0000","last_fullsized":"2024-01-15T18:08:16.258472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:57:52.268562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.207418+0000","last_change":"2024-01-15T18:08:13.206233+0000","last_active":"2024-01-15T18:08:14.207418+0000","last_peered":"2024-01-15T18:08:14.207418+0000","last_clean":"2024-01-15T18:08:14.207418+0000","last_became_active":"2024-01-15T18:08:13.205764+0000","last_became_peered":"2024-01-15T18:08:13.205764+0000","last_unstale":"2024-01-15T18:08:14.207418+0000","last_undegraded":"2024-01-15T18:08:14.207418+0000","last_fullsized":"2024-01-15T18:08:14.207418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:19:54.245751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213264+0000","last_change":"2024-01-15T18:08:14.219217+0000","last_active":"2024-01-15T18:08:15.213264+0000","last_peered":"2024-01-15T18:08:15.213264+0000","last_clean":"2024-01-15T18:08:15.213264+0000","last_became_active":"2024-01-15T18:08:14.218075+0000","last_became_peered":"2024-01-15T18:08:14.218075+0000","last_unstale":"2024-01-15T18:08:15.213264+0000","last_undegraded":"2024-01-15T18:08:15.213264+0000","last_fullsized":"2024-01-15T18:08:15.213264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:02.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.220358+0000","last_change":"2024-01-15T18:08:13.209597+0000","last_active":"2024-01-15T18:08:14.220358+0000","last_peered":"2024-01-15T18:08:14.220358+0000","last_clean":"2024-01-15T18:08:14.220358+0000","last_became_active":"2024-01-15T18:08:13.209194+0000","last_became_peered":"2024-01-15T18:08:13.209194+0000","last_unstale":"2024-01-15T18:08:14.220358+0000","last_undegraded":"2024-01-15T18:08:14.220358+0000","last_fullsized":"2024-01-15T18:08:14.220358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:32:14.261016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212883+0000","last_change":"2024-01-15T18:08:14.201377+0000","last_active":"2024-01-15T18:08:15.212883+0000","last_peered":"2024-01-15T18:08:15.212883+0000","last_clean":"2024-01-15T18:08:15.212883+0000","last_became_active":"2024-01-15T18:08:14.199198+0000","last_became_peered":"2024-01-15T18:08:14.199198+0000","last_unstale":"2024-01-15T18:08:15.212883+0000","last_undegraded":"2024-01-15T18:08:15.212883+0000","last_fullsized":"2024-01-15T18:08:15.212883+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:33.977943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.199325+0000","last_change":"2024-01-15T18:08:13.200813+0000","last_active":"2024-01-15T18:08:14.199325+0000","last_peered":"2024-01-15T18:08:14.199325+0000","last_clean":"2024-01-15T18:08:14.199325+0000","last_became_active":"2024-01-15T18:08:13.200515+0000","last_became_peered":"2024-01-15T18:08:13.200515+0000","last_unstale":"2024-01-15T18:08:14.199325+0000","last_undegraded":"2024-01-15T18:08:14.199325+0000","last_fullsized":"2024-01-15T18:08:14.199325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:32.971831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257494+0000","last_change":"2024-01-15T18:08:14.197310+0000","last_active":"2024-01-15T18:08:16.257494+0000","last_peered":"2024-01-15T18:08:16.257494+0000","last_clean":"2024-01-15T18:08:16.257494+0000","last_became_active":"2024-01-15T18:08:14.196982+0000","last_became_peered":"2024-01-15T18:08:14.196982+0000","last_unstale":"2024-01-15T18:08:16.257494+0000","last_undegraded":"2024-01-15T18:08:16.257494+0000","last_fullsized":"2024-01-15T18:08:16.257494+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:44:54.584810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.218852+0000","last_change":"2024-01-15T18:08:13.201215+0000","last_active":"2024-01-15T18:08:14.218852+0000","last_peered":"2024-01-15T18:08:14.218852+0000","last_clean":"2024-01-15T18:08:14.218852+0000","last_became_active":"2024-01-15T18:08:13.200067+0000","last_became_peered":"2024-01-15T18:08:13.200067+0000","last_unstale":"2024-01-15T18:08:14.218852+0000","last_undegraded":"2024-01-15T18:08:14.218852+0000","last_fullsized":"2024-01-15T18:08:14.218852+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:01:31.833008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258869+0000","last_change":"2024-01-15T18:08:14.221303+0000","last_active":"2024-01-15T18:08:16.258869+0000","last_peered":"2024-01-15T18:08:16.258869+0000","last_clean":"2024-01-15T18:08:16.258869+0000","last_became_active":"2024-01-15T18:08:14.220981+0000","last_became_peered":"2024-01-15T18:08:14.220981+0000","last_unstale":"2024-01-15T18:08:16.258869+0000","last_undegraded":"2024-01-15T18:08:16.258869+0000","last_fullsized":"2024-01-15T18:08:16.258869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:35:33.977943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.184518+0000","last_change":"2024-01-15T18:08:13.206092+0000","last_active":"2024-01-15T18:08:14.184518+0000","last_peered":"2024-01-15T18:08:14.184518+0000","last_clean":"2024-01-15T18:08:14.184518+0000","last_became_active":"2024-01-15T18:08:13.205598+0000","last_became_peered":"2024-01-15T18:08:13.205598+0000","last_unstale":"2024-01-15T18:08:14.184518+0000","last_undegraded":"2024-01-15T18:08:14.184518+0000","last_fullsized":"2024-01-15T18:08:14.184518+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:30.508678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.256455+0000","last_change":"2024-01-15T18:08:14.217833+0000","last_active":"2024-01-15T18:08:16.256455+0000","last_peered":"2024-01-15T18:08:16.256455+0000","last_clean":"2024-01-15T18:08:16.256455+0000","last_became_active":"2024-01-15T18:08:14.217432+0000","last_became_peered":"2024-01-15T18:08:14.217432+0000","last_unstale":"2024-01-15T18:08:16.256455+0000","last_undegraded":"2024-01-15T18:08:16.256455+0000","last_fullsized":"2024-01-15T18:08:16.256455+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:37:57.359384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.185369+0000","last_change":"2024-01-15T18:08:13.201583+0000","last_active":"2024-01-15T18:08:14.185369+0000","last_peered":"2024-01-15T18:08:14.185369+0000","last_clean":"2024-01-15T18:08:14.185369+0000","last_became_active":"2024-01-15T18:08:13.201291+0000","last_became_peered":"2024-01-15T18:08:13.201291+0000","last_unstale":"2024-01-15T18:08:14.185369+0000","last_undegraded":"2024-01-15T18:08:14.185369+0000","last_fullsized":"2024-01-15T18:08:14.185369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:20:32.338335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258891+0000","last_change":"2024-01-15T18:08:14.215539+0000","last_active":"2024-01-15T18:08:16.258891+0000","last_peered":"2024-01-15T18:08:16.258891+0000","last_clean":"2024-01-15T18:08:16.258891+0000","last_became_active":"2024-01-15T18:08:14.215269+0000","last_became_peered":"2024-01-15T18:08:14.215269+0000","last_unstale":"2024-01-15T18:08:16.258891+0000","last_undegraded":"2024-01-15T18:08:16.258891+0000","last_fullsized":"2024-01-15T18:08:16.258891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:44:02.040925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.200566+0000","last_change":"2024-01-15T18:08:13.200034+0000","last_active":"2024-01-15T18:08:14.200566+0000","last_peered":"2024-01-15T18:08:14.200566+0000","last_clean":"2024-01-15T18:08:14.200566+0000","last_became_active":"2024-01-15T18:08:13.199719+0000","last_became_peered":"2024-01-15T18:08:13.199719+0000","last_unstale":"2024-01-15T18:08:14.200566+0000","last_undegraded":"2024-01-15T18:08:14.200566+0000","last_fullsized":"2024-01-15T18:08:14.200566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:40:01.041511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.212496+0000","last_change":"2024-01-15T18:08:14.213547+0000","last_active":"2024-01-15T18:08:15.212496+0000","last_peered":"2024-01-15T18:08:15.212496+0000","last_clean":"2024-01-15T18:08:15.212496+0000","last_became_active":"2024-01-15T18:08:14.213296+0000","last_became_peered":"2024-01-15T18:08:14.213296+0000","last_unstale":"2024-01-15T18:08:15.212496+0000","last_undegraded":"2024-01-15T18:08:15.212496+0000","last_fullsized":"2024-01-15T18:08:15.212496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:21:33.244204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.218630+0000","last_change":"2024-01-15T18:08:13.208311+0000","last_active":"2024-01-15T18:08:14.218630+0000","last_peered":"2024-01-15T18:08:14.218630+0000","last_clean":"2024-01-15T18:08:14.218630+0000","last_became_active":"2024-01-15T18:08:13.207539+0000","last_became_peered":"2024-01-15T18:08:13.207539+0000","last_unstale":"2024-01-15T18:08:14.218630+0000","last_undegraded":"2024-01-15T18:08:14.218630+0000","last_fullsized":"2024-01-15T18:08:14.218630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:19:34.801714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258818+0000","last_change":"2024-01-15T18:08:14.220846+0000","last_active":"2024-01-15T18:08:16.258818+0000","last_peered":"2024-01-15T18:08:16.258818+0000","last_clean":"2024-01-15T18:08:16.258818+0000","last_became_active":"2024-01-15T18:08:14.220469+0000","last_became_peered":"2024-01-15T18:08:14.220469+0000","last_unstale":"2024-01-15T18:08:16.258818+0000","last_undegraded":"2024-01-15T18:08:16.258818+0000","last_fullsized":"2024-01-15T18:08:16.258818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:40:03.935147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.193694+0000","last_change":"2024-01-15T18:08:13.204715+0000","last_active":"2024-01-15T18:08:14.193694+0000","last_peered":"2024-01-15T18:08:14.193694+0000","last_clean":"2024-01-15T18:08:14.193694+0000","last_became_active":"2024-01-15T18:08:13.204391+0000","last_became_peered":"2024-01-15T18:08:13.204391+0000","last_unstale":"2024-01-15T18:08:14.193694+0000","last_undegraded":"2024-01-15T18:08:14.193694+0000","last_fullsized":"2024-01-15T18:08:14.193694+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:06:35.305979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.210683+0000","last_change":"2024-01-15T18:08:14.217631+0000","last_active":"2024-01-15T18:08:15.210683+0000","last_peered":"2024-01-15T18:08:15.210683+0000","last_clean":"2024-01-15T18:08:15.210683+0000","last_became_active":"2024-01-15T18:08:14.216578+0000","last_became_peered":"2024-01-15T18:08:14.216578+0000","last_unstale":"2024-01-15T18:08:15.210683+0000","last_undegraded":"2024-01-15T18:08:15.210683+0000","last_fullsized":"2024-01-15T18:08:15.210683+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:21:33.244204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.196019+0000","last_change":"2024-01-15T18:08:13.201746+0000","last_active":"2024-01-15T18:08:14.196019+0000","last_peered":"2024-01-15T18:08:14.196019+0000","last_clean":"2024-01-15T18:08:14.196019+0000","last_became_active":"2024-01-15T18:08:13.201382+0000","last_became_peered":"2024-01-15T18:08:13.201382+0000","last_unstale":"2024-01-15T18:08:14.196019+0000","last_undegraded":"2024-01-15T18:08:14.196019+0000","last_fullsized":"2024-01-15T18:08:14.196019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:42.053642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.213062+0000","last_change":"2024-01-15T18:08:14.206746+0000","last_active":"2024-01-15T18:08:15.213062+0000","last_peered":"2024-01-15T18:08:15.213062+0000","last_clean":"2024-01-15T18:08:15.213062+0000","last_became_active":"2024-01-15T18:08:14.204676+0000","last_became_peered":"2024-01-15T18:08:14.204676+0000","last_unstale":"2024-01-15T18:08:15.213062+0000","last_undegraded":"2024-01-15T18:08:15.213062+0000","last_fullsized":"2024-01-15T18:08:15.213062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:10.354228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.196218+0000","last_change":"2024-01-15T18:08:13.189673+0000","last_active":"2024-01-15T18:08:14.196218+0000","last_peered":"2024-01-15T18:08:14.196218+0000","last_clean":"2024-01-15T18:08:14.196218+0000","last_became_active":"2024-01-15T18:08:13.189374+0000","last_became_peered":"2024-01-15T18:08:13.189374+0000","last_unstale":"2024-01-15T18:08:14.196218+0000","last_undegraded":"2024-01-15T18:08:14.196218+0000","last_fullsized":"2024-01-15T18:08:14.196218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:50:10.044020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.258990+0000","last_change":"2024-01-15T18:08:14.223338+0000","last_active":"2024-01-15T18:08:16.258990+0000","last_peered":"2024-01-15T18:08:16.258990+0000","last_clean":"2024-01-15T18:08:16.258990+0000","last_became_active":"2024-01-15T18:08:14.222080+0000","last_became_peered":"2024-01-15T18:08:14.222080+0000","last_unstale":"2024-01-15T18:08:16.258990+0000","last_undegraded":"2024-01-15T18:08:16.258990+0000","last_fullsized":"2024-01-15T18:08:16.258990+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:10.354228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.200546+0000","last_change":"2024-01-15T18:08:13.209814+0000","last_active":"2024-01-15T18:08:14.200546+0000","last_peered":"2024-01-15T18:08:14.200546+0000","last_clean":"2024-01-15T18:08:14.200546+0000","last_became_active":"2024-01-15T18:08:13.208708+0000","last_became_peered":"2024-01-15T18:08:13.208708+0000","last_unstale":"2024-01-15T18:08:14.200546+0000","last_undegraded":"2024-01-15T18:08:14.200546+0000","last_fullsized":"2024-01-15T18:08:14.200546+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:48:09.348116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.088655+0000","last_change":"2024-01-15T18:08:14.223030+0000","last_active":"2024-01-15T18:08:16.088655+0000","last_peered":"2024-01-15T18:08:16.088655+0000","last_clean":"2024-01-15T18:08:16.088655+0000","last_became_active":"2024-01-15T18:08:14.221307+0000","last_became_peered":"2024-01-15T18:08:14.221307+0000","last_unstale":"2024-01-15T18:08:16.088655+0000","last_undegraded":"2024-01-15T18:08:16.088655+0000","last_fullsized":"2024-01-15T18:08:16.088655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:49:35.594008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.188354+0000","last_change":"2024-01-15T18:08:13.190400+0000","last_active":"2024-01-15T18:08:14.188354+0000","last_peered":"2024-01-15T18:08:14.188354+0000","last_clean":"2024-01-15T18:08:14.188354+0000","last_became_active":"2024-01-15T18:08:13.190006+0000","last_became_peered":"2024-01-15T18:08:13.190006+0000","last_unstale":"2024-01-15T18:08:14.188354+0000","last_undegraded":"2024-01-15T18:08:14.188354+0000","last_fullsized":"2024-01-15T18:08:14.188354+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:01:36.508305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.257184+0000","last_change":"2024-01-15T18:08:14.219320+0000","last_active":"2024-01-15T18:08:16.257184+0000","last_peered":"2024-01-15T18:08:16.257184+0000","last_clean":"2024-01-15T18:08:16.257184+0000","last_became_active":"2024-01-15T18:08:14.218353+0000","last_became_peered":"2024-01-15T18:08:14.218353+0000","last_unstale":"2024-01-15T18:08:16.257184+0000","last_undegraded":"2024-01-15T18:08:16.257184+0000","last_fullsized":"2024-01-15T18:08:16.257184+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:42:56.902259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.195914+0000","last_change":"2024-01-15T18:08:13.199997+0000","last_active":"2024-01-15T18:08:14.195914+0000","last_peered":"2024-01-15T18:08:14.195914+0000","last_clean":"2024-01-15T18:08:14.195914+0000","last_became_active":"2024-01-15T18:08:13.198486+0000","last_became_peered":"2024-01-15T18:08:13.198486+0000","last_unstale":"2024-01-15T18:08:14.195914+0000","last_undegraded":"2024-01-15T18:08:14.195914+0000","last_fullsized":"2024-01-15T18:08:14.195914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:37:49.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:15.211490+0000","last_change":"2024-01-15T18:08:14.218836+0000","last_active":"2024-01-15T18:08:15.211490+0000","last_peered":"2024-01-15T18:08:15.211490+0000","last_clean":"2024-01-15T18:08:15.211490+0000","last_became_active":"2024-01-15T18:08:14.218487+0000","last_became_peered":"2024-01-15T18:08:14.218487+0000","last_unstale":"2024-01-15T18:08:15.211490+0000","last_undegraded":"2024-01-15T18:08:15.211490+0000","last_fullsized":"2024-01-15T18:08:15.211490+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:32:15.267128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.186113+0000","last_change":"2024-01-15T18:08:13.203003+0000","last_active":"2024-01-15T18:08:14.186113+0000","last_peered":"2024-01-15T18:08:14.186113+0000","last_clean":"2024-01-15T18:08:14.186113+0000","last_became_active":"2024-01-15T18:08:13.202706+0000","last_became_peered":"2024-01-15T18:08:13.202706+0000","last_unstale":"2024-01-15T18:08:14.186113+0000","last_undegraded":"2024-01-15T18:08:14.186113+0000","last_fullsized":"2024-01-15T18:08:14.186113+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:37:49.857246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-01-15T18:08:16.259024+0000","last_change":"2024-01-15T18:08:14.218124+0000","last_active":"2024-01-15T18:08:16.259024+0000","last_peered":"2024-01-15T18:08:16.259024+0000","last_clean":"2024-01-15T18:08:16.259024+0000","last_became_active":"2024-01-15T18:08:14.217043+0000","last_became_peered":"2024-01-15T18:08:14.217043+0000","last_unstale":"2024-01-15T18:08:16.259024+0000","last_undegraded":"2024-01-15T18:08:16.259024+0000","last_fullsized":"2024-01-15T18:08:16.259024+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:13.157653+0000","last_clean_scrub_stamp":"2024-01-15T18:08:13.157653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:14.974406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.221504+0000","last_change":"2024-01-15T18:08:13.208928+0000","last_active":"2024-01-15T18:08:14.221504+0000","last_peered":"2024-01-15T18:08:14.221504+0000","last_clean":"2024-01-15T18:08:14.221504+0000","last_became_active":"2024-01-15T18:08:13.208647+0000","last_became_peered":"2024-01-15T18:08:13.208647+0000","last_unstale":"2024-01-15T18:08:14.221504+0000","last_undegraded":"2024-01-15T18:08:14.221504+0000","last_fullsized":"2024-01-15T18:08:14.221504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:08:12.151541+0000","last_clean_scrub_stamp":"2024-01-15T18:08:12.151541+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:16:37.222623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-15T18:08:14.208031+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:08:14.208031+0000","last_peered":"2024-01-15T18:08:14.208031+0000","last_clean":"2024-01-15T18:08:14.208031+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:08:14.208031+0000","last_undegraded":"2024-01-15T18:08:14.208031+0000","last_fullsized":"2024-01-15T18:08:14.208031+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542307,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77156,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94294684,"statfs":{"total":96636764160,"available":96557756416,"internally_reserved":0,"allocated":1662976,"data_stored":1049281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77330890},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.667,"15min":1.363},"min":{"1min":0.84399999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.823,"5min":19.843,"15min":19.843},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.8979999999999999,"15min":1.593},"min":{"1min":0.91800000000000004,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.5509999999999999,"5min":30.498000000000001,"15min":38.137},"last":1.774}]},{"osd":1,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.7090000000000001,"15min":1.413},"min":{"1min":0.86799999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5700000000000001,"5min":19.917000000000002,"15min":19.917000000000002},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.6140000000000001,"15min":1.597},"min":{"1min":0.98799999999999999,"5min":0.71199999999999997,"15min":0.42999999999999999},"max":{"1min":1.667,"5min":10.361000000000001,"15min":51.843000000000004},"last":1.266}]},{"osd":2,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.573,"15min":1.4059999999999999},"min":{"1min":0.73199999999999998,"5min":0.50600000000000001,"15min":0.437},"max":{"1min":1.607,"5min":12.061,"15min":12.061},"last":1.833},{"interface":"front","average":{"1min":1.367,"5min":1.8049999999999999,"15min":1.5429999999999999},"min":{"1min":1.036,"5min":0.48999999999999999,"15min":0.47399999999999998},"max":{"1min":1.639,"5min":13.478999999999999,"15min":13.478999999999999},"last":1.4610000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.778,"15min":1.5369999999999999},"min":{"1min":0.91500000000000004,"5min":0.56000000000000005,"15min":0.47899999999999998},"max":{"1min":1.544,"5min":16.632000000000001,"15min":16.632000000000001},"last":1.893},{"interface":"front","average":{"1min":1.355,"5min":1.853,"15min":1.6240000000000001},"min":{"1min":0.94199999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.768,"5min":13.439,"15min":13.439},"last":1.575}]},{"osd":4,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.7929999999999999,"15min":1.7190000000000001},"min":{"1min":0.878,"5min":0.54200000000000004,"15min":0.50700000000000001},"max":{"1min":1.5820000000000001,"5min":23.042000000000002,"15min":23.042000000000002},"last":1.514},{"interface":"front","average":{"1min":1.331,"5min":2.1469999999999998,"15min":1.7330000000000001},"min":{"1min":0.98999999999999999,"5min":0.52700000000000002,"15min":0.51800000000000002},"max":{"1min":1.7330000000000001,"5min":38.682000000000002,"15min":38.682000000000002},"last":1.71}]},{"osd":5,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.8169999999999999,"15min":1.8200000000000001},"min":{"1min":0.98999999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.669,"5min":13.379,"15min":25.006},"last":1.665},{"interface":"front","average":{"1min":1.3839999999999999,"5min":2.0110000000000001,"15min":1.855},"min":{"1min":1.0629999999999999,"5min":0.624,"15min":0.624},"max":{"1min":1.6819999999999999,"5min":22.933,"15min":22.933},"last":1.855}]},{"osd":6,"last update":"Mon Jan 15 18:07:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":2.165,"15min":2.0539999999999998},"min":{"1min":1.2470000000000001,"5min":0.66400000000000003,"15min":0.59299999999999997},"max":{"1min":1.7849999999999999,"5min":23.004000000000001,"15min":23.834},"last":1.639},{"interface":"front","average":{"1min":1.452,"5min":1.8400000000000001,"15min":1.998},"min":{"1min":1.05,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.825,"5min":9.2599999999999998,"15min":63.932000000000002},"last":1.95}]}]},{"osd":6,"up_from":14,"seq":60129542306,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58600,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313240,"statfs":{"total":96636764160,"available":96576757760,"internally_reserved":0,"allocated":1077248,"data_stored":459537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.454,"15min":1.369},"min":{"1min":0.98299999999999998,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.702,"5min":8.593,"15min":12.544},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.579,"15min":1.389},"min":{"1min":0.86599999999999999,"5min":0.73199999999999998,"15min":0.68400000000000005},"max":{"1min":1.7390000000000001,"5min":14.013,"15min":14.013},"last":0.93799999999999994}]},{"osd":1,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.361,"15min":1.3029999999999999},"min":{"1min":0.79000000000000004,"5min":0.63700000000000001,"15min":0.55100000000000005},"max":{"1min":1.9359999999999999,"5min":8.7729999999999997,"15min":12.811999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.8680000000000001,"15min":1.534},"min":{"1min":0.93700000000000006,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.369,"5min":23.495999999999999,"15min":23.495999999999999},"last":1.0620000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.9970000000000001,"15min":1.5720000000000001},"min":{"1min":0.86499999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.6140000000000001,"5min":23.087,"15min":23.087},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.212,"5min":1.4119999999999999,"15min":1.4339999999999999},"min":{"1min":0.90500000000000003,"5min":0.51400000000000001,"15min":0.504},"max":{"1min":1.5529999999999999,"5min":8.8379999999999992,"15min":12.897},"last":0.73799999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.758,"15min":1.657},"min":{"1min":0.95799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.8660000000000001,"5min":13.597,"15min":13.597},"last":1.294},{"interface":"front","average":{"1min":1.429,"5min":1.7989999999999999,"15min":1.597},"min":{"1min":1.103,"5min":0.78400000000000003,"15min":0.70699999999999996},"max":{"1min":1.821,"5min":13.795,"15min":14.066000000000001},"last":1.254}]},{"osd":4,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":2.1499999999999999,"15min":1.6839999999999999},"min":{"1min":0.85499999999999998,"5min":0.59599999999999997,"15min":0.51800000000000002},"max":{"1min":1.647,"5min":23.327000000000002,"15min":23.327000000000002},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.6379999999999999,"15min":1.659},"min":{"1min":1.0109999999999999,"5min":0.52000000000000002,"15min":0.45800000000000002},"max":{"1min":1.752,"5min":8.7040000000000006,"15min":17.289000000000001},"last":1.0129999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.7769999999999999,"15min":1.5700000000000001},"min":{"1min":0.94099999999999995,"5min":0.627,"15min":0.51600000000000001},"max":{"1min":1.7729999999999999,"5min":13.593,"15min":14.176},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.5489999999999999,"5min":2.2440000000000002,"15min":1.8959999999999999},"min":{"1min":1.331,"5min":0.621,"15min":0.54700000000000004},"max":{"1min":1.875,"5min":23.417999999999999,"15min":23.417999999999999},"last":1.425}]},{"osd":7,"last update":"Mon Jan 15 18:07:16 2024","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":2.1379999999999999,"15min":1.77},"min":{"1min":1.101,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.905,"5min":21.265000000000001,"15min":21.265000000000001},"last":1.129},{"interface":"front","average":{"1min":1.48,"5min":2.5299999999999998,"15min":2.0099999999999998},"min":{"1min":1.175,"5min":0.65300000000000002,"15min":0.59699999999999998},"max":{"1min":1.8779999999999999,"5min":23.738,"15min":23.738},"last":1.476}]}]},{"osd":5,"up_from":14,"seq":60129542306,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54444,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317396,"statfs":{"total":96636764160,"available":96581013504,"internally_reserved":0,"allocated":1081344,"data_stored":460477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.375,"15min":1.2849999999999999},"min":{"1min":0.69599999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.6419999999999999,"5min":19.266999999999999,"15min":19.266999999999999},"last":1.171},{"interface":"front","average":{"1min":1.151,"5min":1.1990000000000001,"15min":1.3759999999999999},"min":{"1min":0.82199999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5489999999999999,"5min":3.1760000000000002,"15min":17.184999999999999},"last":1.3140000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.3919999999999999,"15min":1.343},"min":{"1min":0.78300000000000003,"5min":0.60699999999999998,"15min":0.499},"max":{"1min":1.6839999999999999,"5min":9.1259999999999994,"15min":16.297000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.4890000000000001,"15min":1.3180000000000001},"min":{"1min":0.85899999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.5860000000000001,"5min":9.2379999999999995,"15min":9.2379999999999995},"last":0.90600000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.296,"15min":1.266},"min":{"1min":1.0029999999999999,"5min":0.621,"15min":0.621},"max":{"1min":1.6859999999999999,"5min":4.0369999999999999,"15min":8.1820000000000004},"last":1.117},{"interface":"front","average":{"1min":1.373,"5min":1.454,"15min":1.4199999999999999},"min":{"1min":0.90000000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.673,"5min":9.2159999999999993,"15min":16.158999999999999},"last":1.472}]},{"osd":3,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.506,"15min":1.47},"min":{"1min":1,"5min":0.63700000000000001,"15min":0.56100000000000005},"max":{"1min":1.6850000000000001,"5min":9.1929999999999996,"15min":14.914},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.552,"15min":1.458},"min":{"1min":0.93200000000000005,"5min":0.64600000000000002,"15min":0.63300000000000001},"max":{"1min":1.804,"5min":7.0110000000000001,"15min":7.0110000000000001},"last":1.456}]},{"osd":4,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.6000000000000001,"15min":1.5089999999999999},"min":{"1min":0.84299999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.6919999999999999,"5min":12.24,"15min":16.414999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.5449999999999999,"15min":1.6319999999999999},"min":{"1min":1.0029999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7010000000000001,"5min":9.1519999999999992,"15min":17.991},"last":1.429}]},{"osd":6,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.5349999999999999,"15min":1.5329999999999999},"min":{"1min":1.0680000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.6819999999999999,"5min":3.7559999999999998,"15min":6.4630000000000001},"last":1.395},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.6040000000000001,"15min":1.605},"min":{"1min":0.995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.76,"5min":9.1649999999999991,"15min":16.364000000000001},"last":1.371}]},{"osd":7,"last update":"Mon Jan 15 18:07:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.7170000000000001,"15min":1.627},"min":{"1min":0.90900000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.742,"5min":9.0600000000000005,"15min":9.0600000000000005},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.885,"15min":1.7370000000000001},"min":{"1min":1.2549999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.865,"5min":19.356999999999999,"15min":19.356999999999999},"last":1.5069999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542306,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58592,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313248,"statfs":{"total":96636764160,"available":96576765952,"internally_reserved":0,"allocated":1064960,"data_stored":458843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.3640000000000001,"15min":1.3360000000000001},"min":{"1min":0.51800000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.6080000000000001,"5min":7.8869999999999996,"15min":11.413},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.4590000000000001,"15min":1.405},"min":{"1min":0.91600000000000004,"5min":0.63,"15min":0.53700000000000003},"max":{"1min":2.004,"5min":7.9160000000000004,"15min":9.7439999999999998},"last":1.0640000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3620000000000001,"15min":1.377},"min":{"1min":0.78500000000000003,"5min":0.61399999999999999,"15min":0.45000000000000001},"max":{"1min":1.706,"5min":11.128,"15min":11.987},"last":1.389},{"interface":"front","average":{"1min":1.151,"5min":1.282,"15min":1.331},"min":{"1min":0.69499999999999995,"5min":0.60799999999999998,"15min":0.55300000000000005},"max":{"1min":1.72,"5min":5.6109999999999998,"15min":16.058},"last":0.97899999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3500000000000001,"15min":1.5269999999999999},"min":{"1min":0.71899999999999997,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.6919999999999999,"5min":5.0960000000000001,"15min":13.865},"last":0.877},{"interface":"front","average":{"1min":1.337,"5min":1.5029999999999999,"15min":1.526},"min":{"1min":1.1319999999999999,"5min":0.83299999999999996,"15min":0.69499999999999995},"max":{"1min":1.5389999999999999,"5min":5.7050000000000001,"15min":13.903},"last":1.3080000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.5389999999999999,"15min":1.637},"min":{"1min":1.0069999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.6619999999999999,"5min":11.773,"15min":26.055},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.403,"15min":1.526},"min":{"1min":0.97899999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9099999999999999,"5min":4.0419999999999998,"15min":13.25},"last":1.1739999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.6499999999999999,"15min":1.6160000000000001},"min":{"1min":0.85699999999999998,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.7989999999999999,"5min":29.422000000000001,"15min":29.422000000000001},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.5720000000000001,"15min":1.573},"min":{"1min":1.04,"5min":0.53700000000000003,"15min":0.42199999999999999},"max":{"1min":1.7250000000000001,"5min":12.332000000000001,"15min":12.332000000000001},"last":1.264}]},{"osd":6,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.55,"15min":1.5920000000000001},"min":{"1min":0.93000000000000005,"5min":0.59699999999999998,"15min":0.45200000000000001},"max":{"1min":1.837,"5min":11.09,"15min":18.260000000000002},"last":1.135},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.6100000000000001,"15min":1.7230000000000001},"min":{"1min":0.99399999999999999,"5min":0.59999999999999998,"15min":0.48099999999999998},"max":{"1min":1.9239999999999999,"5min":9.8740000000000006,"15min":11.657999999999999},"last":1.573}]},{"osd":7,"last update":"Mon Jan 15 18:07:14 2024","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.744,"15min":1.829},"min":{"1min":1,"5min":0.60499999999999998,"15min":0.503},"max":{"1min":2.0110000000000001,"5min":12.247,"15min":12.247},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.591,"5min":1.788,"15min":1.8979999999999999},"min":{"1min":0.97799999999999998,"5min":0.61399999999999999,"15min":0.53600000000000003},"max":{"1min":2.0169999999999999,"5min":12.311999999999999,"15min":12.311999999999999},"last":1.349}]}]},{"osd":3,"up_from":14,"seq":60129542308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68272,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303568,"statfs":{"total":96636764160,"available":96566853632,"internally_reserved":0,"allocated":1077248,"data_stored":460455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68811207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":1.373,"15min":1.2030000000000001},"min":{"1min":0.40500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.339,"5min":18.399999999999999,"15min":18.399999999999999},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.5029999999999999,"15min":1.3799999999999999},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.5229999999999999,"5min":18.795999999999999,"15min":21.047000000000001},"last":0.96499999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":1.2969999999999999,"15min":1.298},"min":{"1min":0.66400000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3400000000000001,"5min":12.372,"15min":17.777999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.3160000000000001,"15min":1.2250000000000001},"min":{"1min":0.47099999999999997,"5min":0.46700000000000003,"15min":0.39500000000000002},"max":{"1min":1.8200000000000001,"5min":11.002000000000001,"15min":11.002000000000001},"last":0.65200000000000002}]},{"osd":2,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.339,"15min":1.272},"min":{"1min":0.60499999999999998,"5min":0.375,"15min":0.375},"max":{"1min":2.7519999999999998,"5min":10.964,"15min":10.964},"last":0.752},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.669,"15min":1.528},"min":{"1min":0.82299999999999995,"5min":0.57099999999999995,"15min":0.54000000000000004},"max":{"1min":2.3210000000000002,"5min":20.495000000000001,"15min":20.495000000000001},"last":1.1479999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.823,"15min":1.5740000000000001},"min":{"1min":1.042,"5min":0.71599999999999997,"15min":0.63500000000000001},"max":{"1min":2.5,"5min":13.536,"15min":13.536},"last":1.002},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.905,"15min":1.6739999999999999},"min":{"1min":0.83299999999999996,"5min":0.65100000000000002,"15min":0.54600000000000004},"max":{"1min":1.738,"5min":16.398,"15min":16.398},"last":0.71099999999999997}]},{"osd":5,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.845,"15min":1.6779999999999999},"min":{"1min":0.875,"5min":0.67200000000000004,"15min":0.57299999999999995},"max":{"1min":2.423,"5min":20.606000000000002,"15min":20.606000000000002},"last":1.244},{"interface":"front","average":{"1min":1.6040000000000001,"5min":1.736,"15min":1.6899999999999999},"min":{"1min":0.94699999999999995,"5min":0.70199999999999996,"15min":0.68000000000000005},"max":{"1min":2.8820000000000001,"5min":11.603,"15min":11.603},"last":1.2729999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":2.0379999999999998,"15min":1.8580000000000001},"min":{"1min":1.0629999999999999,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.873,"5min":13.648999999999999,"15min":13.648999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.994,"15min":1.8280000000000001},"min":{"1min":0.99099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.1459999999999999,"5min":20.556000000000001,"15min":20.556000000000001},"last":1.0680000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":2.0910000000000002,"15min":1.8879999999999999},"min":{"1min":0.88600000000000001,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.6299999999999999,"5min":28.843,"15min":28.843},"last":1.133},{"interface":"front","average":{"1min":1.478,"5min":2.0339999999999998,"15min":1.909},"min":{"1min":0.98999999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1179999999999999,"5min":15.143000000000001,"15min":15.143000000000001},"last":1.048}]}]},{"osd":2,"up_from":14,"seq":60129542308,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64096,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307744,"statfs":{"total":96636764160,"available":96571129856,"internally_reserved":0,"allocated":1069056,"data_stored":459425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":1.1000000000000001,"15min":1.0960000000000001},"min":{"1min":0.71199999999999997,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2849999999999999,"5min":12.278,"15min":14.427},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.7370000000000001,"15min":1.319},"min":{"1min":0.50900000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.653,"5min":15.314,"15min":15.314},"last":0.629}]},{"osd":1,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.5900000000000001,"15min":1.454},"min":{"1min":0.83899999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.579,"5min":34.463999999999999,"15min":34.463999999999999},"last":1.488},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.923,"15min":1.4490000000000001},"min":{"1min":0.63700000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.9159999999999999,"5min":25.859999999999999,"15min":25.859999999999999},"last":0.96799999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.4510000000000001,"15min":1.3799999999999999},"min":{"1min":0.77200000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.9059999999999999,"5min":12.93,"15min":19.535},"last":0.75600000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.873,"15min":1.6439999999999999},"min":{"1min":0.97999999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.044,"5min":22.783999999999999,"15min":22.783999999999999},"last":1.393}]},{"osd":4,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":2.012,"15min":1.804},"min":{"1min":1.1839999999999999,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.79,"5min":29.986999999999998,"15min":29.986999999999998},"last":1.77},{"interface":"front","average":{"1min":1.4830000000000001,"5min":2.1339999999999999,"15min":1.7410000000000001},"min":{"1min":1.2390000000000001,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.768,"5min":21.199999999999999,"15min":21.199999999999999},"last":1.3100000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.7210000000000001,"15min":1.655},"min":{"1min":1.048,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.9710000000000001,"5min":12.227,"15min":19.681999999999999},"last":1.155},{"interface":"front","average":{"1min":1.542,"5min":1.8600000000000001,"15min":1.768},"min":{"1min":1.248,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.2869999999999999,"5min":12.634,"15min":14.702999999999999},"last":1.962}]},{"osd":6,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.5489999999999999,"5min":2.1840000000000002,"15min":1.778},"min":{"1min":0.871,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.9870000000000001,"5min":15.683999999999999,"15min":15.683999999999999},"last":1.575},{"interface":"front","average":{"1min":1.5700000000000001,"5min":2.0419999999999998,"15min":1.792},"min":{"1min":0.95999999999999996,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.3279999999999998,"5min":12.385,"15min":19.783000000000001},"last":1.4299999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6819999999999999,"5min":2.8980000000000001,"15min":2.0649999999999999},"min":{"1min":1.2170000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.411,"5min":57.006999999999998,"15min":57.006999999999998},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.619,"5min":2.4630000000000001,"15min":2.1299999999999999},"min":{"1min":1.2649999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.2280000000000002,"5min":22.853999999999999,"15min":22.853999999999999},"last":1.635}]}]},{"osd":1,"up_from":14,"seq":60129542307,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68264,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303576,"statfs":{"total":96636764160,"available":96566861824,"internally_reserved":0,"allocated":1077248,"data_stored":459037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.575,"15min":1.1870000000000001},"min":{"1min":0.67700000000000005,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.6139999999999999,"5min":30.460999999999999,"15min":30.460999999999999},"last":1.778},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.323,"15min":1.107},"min":{"1min":0.38700000000000001,"5min":0.314,"15min":0.314},"max":{"1min":1.6970000000000001,"5min":18.658000000000001,"15min":18.658000000000001},"last":0.81000000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":1.1990000000000001,"15min":1.149},"min":{"1min":0.63600000000000001,"5min":0.47299999999999998,"15min":0.376},"max":{"1min":1.851,"5min":7.117,"15min":21.126000000000001},"last":0.88},{"interface":"front","average":{"1min":1.458,"5min":1.8600000000000001,"15min":1.53},"min":{"1min":0.73099999999999998,"5min":0.52700000000000002,"15min":0.48699999999999999},"max":{"1min":3.0409999999999999,"5min":30.593,"15min":30.593},"last":1.8839999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.7849999999999999,"15min":1.74},"min":{"1min":0.82499999999999996,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.1400000000000001,"5min":30.547000000000001,"15min":30.547000000000001},"last":2.0179999999999998},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4339999999999999,"15min":1.339},"min":{"1min":0.67500000000000004,"5min":0.437,"15min":0.437},"max":{"1min":5.1740000000000004,"5min":7.9610000000000003,"15min":15.510999999999999},"last":0.93700000000000006}]},{"osd":4,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.702,"5min":1.7330000000000001,"15min":1.7370000000000001},"min":{"1min":1.006,"5min":0.72699999999999998,"15min":0.72499999999999998},"max":{"1min":5.2229999999999999,"5min":9.3650000000000002,"15min":14.827},"last":1.401},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.6259999999999999,"15min":1.7230000000000001},"min":{"1min":0.90000000000000002,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":3.419,"5min":9.3010000000000002,"15min":14.851000000000001},"last":1.1819999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.6599999999999999,"15min":1.6850000000000001},"min":{"1min":0.80800000000000005,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":3.282,"5min":6.0700000000000003,"15min":13.228999999999999},"last":1.302},{"interface":"front","average":{"1min":1.548,"5min":2.3340000000000001,"15min":1.9510000000000001},"min":{"1min":1.024,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.8119999999999998,"5min":34.042999999999999,"15min":34.042999999999999},"last":2.153}]},{"osd":6,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.9390000000000001,"15min":1.8759999999999999},"min":{"1min":0.86299999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":3.5539999999999998,"5min":34.158000000000001,"15min":34.158000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.714,"5min":1.7010000000000001,"15min":1.6160000000000001},"min":{"1min":0.872,"5min":0.62,"15min":0.62},"max":{"1min":5.2539999999999996,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.5629999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:07:21 2024","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":2.351,"15min":1.859},"min":{"1min":1.121,"5min":0.80300000000000005,"15min":0.64600000000000002},"max":{"1min":2.9319999999999999,"5min":33.659999999999997,"15min":33.659999999999997},"last":2.2269999999999999},{"interface":"front","average":{"1min":1.673,"5min":1.913,"15min":1.7949999999999999},"min":{"1min":0.85699999999999998,"5min":0.70299999999999996,"15min":0.60499999999999998},"max":{"1min":3.6629999999999998,"5min":11.882999999999999,"15min":24.289999999999999},"last":1.3660000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542308,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64676,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307164,"statfs":{"total":96636764160,"available":96570535936,"internally_reserved":0,"allocated":1662976,"data_stored":1049245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.0580000000000001,"15min":0.97399999999999998},"min":{"1min":0.42799999999999999,"5min":0.32400000000000001,"15min":0.312},"max":{"1min":1.4730000000000001,"5min":6.1890000000000001,"15min":6.1890000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3540000000000001,"15min":1.2350000000000001},"min":{"1min":0.54200000000000004,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":1.671,"5min":11.470000000000001,"15min":11.470000000000001},"last":1.1020000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2769999999999999,"15min":1.167},"min":{"1min":0.49099999999999999,"5min":0.45500000000000002,"15min":0.27600000000000002},"max":{"1min":1.7490000000000001,"5min":11.59,"15min":11.59},"last":1.3480000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":1.3600000000000001,"15min":1.0860000000000001},"min":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.35899999999999999},"max":{"1min":1.331,"5min":14.512,"15min":14.512},"last":0.84999999999999998}]},{"osd":3,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.514,"15min":1.3899999999999999},"min":{"1min":0.56599999999999995,"5min":0.50800000000000001,"15min":0.41699999999999998},"max":{"1min":1.7230000000000001,"5min":11.544,"15min":11.544},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.2450000000000001,"15min":1.2330000000000001},"min":{"1min":0.47299999999999998,"5min":0.35199999999999998,"15min":0.29599999999999999},"max":{"1min":1.7669999999999999,"5min":6.1539999999999999,"15min":14.74},"last":1.925}]},{"osd":4,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.948,"15min":1.55},"min":{"1min":0.84299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.444,"5min":32.115000000000002,"15min":32.115000000000002},"last":1.429},{"interface":"front","average":{"1min":1.498,"5min":1.768,"15min":1.681},"min":{"1min":1.139,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.8320000000000001,"5min":11.992000000000001,"15min":15.975},"last":1.679}]},{"osd":5,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.99,"15min":1.6439999999999999},"min":{"1min":0.97399999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.625,"5min":32.174999999999997,"15min":32.174999999999997},"last":1.53},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.7869999999999999,"15min":1.6319999999999999},"min":{"1min":1.016,"5min":0.442,"15min":0.442},"max":{"1min":1.724,"5min":17.225000000000001,"15min":17.225000000000001},"last":1.1850000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.9810000000000001,"15min":1.71},"min":{"1min":1.1120000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.794,"5min":16.228000000000002,"15min":16.228000000000002},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":2.2309999999999999,"15min":1.742},"min":{"1min":0.92800000000000005,"5min":0.67800000000000005,"15min":0.625},"max":{"1min":1.853,"5min":32.216999999999999,"15min":32.216999999999999},"last":1.637}]},{"osd":7,"last update":"Mon Jan 15 18:07:17 2024","interfaces":[{"interface":"back","average":{"1min":1.6739999999999999,"5min":2.7080000000000002,"15min":2.0510000000000002},"min":{"1min":1.3069999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.335,"5min":34.131,"15min":34.131},"last":1.474},{"interface":"front","average":{"1min":1.573,"5min":2.2610000000000001,"15min":1.8660000000000001},"min":{"1min":1.073,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.9099999999999999,"5min":21.884,"15min":21.884},"last":1.7829999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:08:19.239 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:08:19.239 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:08:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.511+0000 7f3481e00640 1 Processor -- start 2024-01-15T18:08:19.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.511+0000 7f3481e00640 1 -- start start 2024-01-15T18:08:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.513+0000 7f3481e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b47d0 0x7f347c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.513+0000 7f3481e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347c0ceb80 con 0x7f347c0b43f0 2024-01-15T18:08:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.513+0000 7f3481e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347c15bac0 con 0x7f347c0adfc0 2024-01-15T18:08:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.513+0000 7f3481e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c15bda0 con 0x7f347c0b47d0 2024-01-15T18:08:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.513+0000 7f3480a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f347c0b43f0 0x7f347c0ad8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33586/0 (socket says 172.21.15.26:33586) 2024-01-15T18:08:19.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.513+0000 7f3480a00640 1 -- 172.21.15.26:0/3489250176 learned_addr learned my addr 172.21.15.26:0/3489250176 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.514+0000 7f3481400640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b47d0 0x7f347c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.514+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228357222 0 0) 0x7f347c0ceb80 con 0x7f347c0b43f0 2024-01-15T18:08:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.514+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f345c003520 con 0x7f347c0b43f0 2024-01-15T18:08:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.514+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1893661702 0 0) 0x7f345c003520 con 0x7f347c0b43f0 2024-01-15T18:08:19.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.514+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b47d0 msgr2=0x7f347c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.514+0000 7f347b400640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b47d0 0x7f347c0ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.514+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 >> v1:172.21.15.124:6789/0 conn(0x7f347c0adfc0 legacy=0x7f347c0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.514+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c15c080 con 0x7f347c0b43f0 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.514+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f346c001a10 con 0x7f347c0b43f0 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 >> v1:172.21.15.26:6789/0 conn(0x7f347c0b43f0 legacy=0x7f347c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347b400640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345c0039a0 0x7f345c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347b400640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f345c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347b400640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b47d0 0x7f345c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c15bda0 con 0x7f345c0047c0 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c00ba90 con 0x7f345c0039a0 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c00bc50 con 0x7f347c0b47d0 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347be00640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f345c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f3480a00640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345c0039a0 0x7f345c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f3481400640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b47d0 0x7f345c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347be00640 1 -- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b47d0 msgr2=0x7f345c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347be00640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b47d0 0x7f345c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347be00640 1 -- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345c0039a0 msgr2=0x7f345c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347be00640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f345c0039a0 0x7f345c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.515+0000 7f347be00640 1 -- 172.21.15.26:0/3489250176 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c003520 con 0x7f345c0047c0 2024-01-15T18:08:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.517+0000 7f347be00640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f345c004c40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3464001720 tx=0x7f345c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677e1d5af37bbfb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.517+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640052d0 con 0x7f345c0047c0 2024-01-15T18:08:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.517+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346402e050 con 0x7f345c0047c0 2024-01-15T18:08:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.517+0000 7f347b400640 1 -- 172.21.15.26:0/3489250176 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640035f0 con 0x7f345c0047c0 2024-01-15T18:08:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.518+0000 7f3481e00640 1 -- 172.21.15.26:0/3489250176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 msgr2=0x7f345c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.518+0000 7f3481e00640 1 --2- 172.21.15.26:0/3489250176 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f345c004c40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f3464001720 tx=0x7f345c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.518+0000 7f3481400640 1 -- 172.21.15.26:0/3489250176 reap_dead start 2024-01-15T18:08:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.518+0000 7f3481e00640 1 -- 172.21.15.26:0/3489250176 shutdown_connections 2024-01-15T18:08:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.518+0000 7f3481e00640 1 -- 172.21.15.26:0/3489250176 >> 172.21.15.26:0/3489250176 conn(0x7f347c0b0b70 msgr2=0x7f347c0b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.518+0000 7f3481e00640 1 -- 172.21.15.26:0/3489250176 shutdown_connections 2024-01-15T18:08:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.518+0000 7f3481e00640 1 -- 172.21.15.26:0/3489250176 wait complete. 2024-01-15T18:08:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.520+0000 7f3481e00640 1 Processor -- start 2024-01-15T18:08:19.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.520+0000 7f3481e00640 1 -- start start 2024-01-15T18:08:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.521+0000 7f3481e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f347c14f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.521+0000 7f3481e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b2e80 0x7f347c0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.521+0000 7f3481e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f347c14c2a0 0x7f347c0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.521+0000 7f3481e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c0b3300 con 0x7f345c0047c0 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.521+0000 7f3481e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347c0b2cc0 con 0x7f347c14c2a0 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.521+0000 7f3481e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347c14c9a0 con 0x7f347c0b2e80 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.521+0000 7f3480a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f347c14f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.521+0000 7f3480a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f347c14f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60834/0 (socket says 172.21.15.26:60834) 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.521+0000 7f3480a00640 1 -- 172.21.15.26:0/833851868 learned_addr learned my addr 172.21.15.26:0/833851868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3481400640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f347c14c2a0 0x7f347c0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f347be00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b2e80 0x7f347c0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3480a00640 1 -- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b2e80 msgr2=0x7f347c0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3480a00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b2e80 0x7f347c0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3480a00640 1 -- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f347c14c2a0 msgr2=0x7f347c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3480a00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f347c14c2a0 0x7f347c0ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3480a00640 1 -- 172.21.15.26:0/833851868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0b3510 con 0x7f345c0047c0 2024-01-15T18:08:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f347be00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b2e80 0x7f347c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3480a00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f347c14f110 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f346c0079e0 tx=0x7f346c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:19.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3478c00640 1 -- 172.21.15.26:0/833851868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c00edf0 con 0x7f345c0047c0 2024-01-15T18:08:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3478c00640 1 -- 172.21.15.26:0/833851868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346c013420 con 0x7f345c0047c0 2024-01-15T18:08:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347c0b37f0 con 0x7f345c0047c0 2024-01-15T18:08:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.522+0000 7f3478c00640 1 -- 172.21.15.26:0/833851868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c0137b0 con 0x7f345c0047c0 2024-01-15T18:08:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.523+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347c0b3db0 con 0x7f345c0047c0 2024-01-15T18:08:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.524+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347c0b3fd0 con 0x7f345c0047c0 2024-01-15T18:08:19.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.525+0000 7f3478c00640 1 -- 172.21.15.26:0/833851868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f346c013910 con 0x7f345c0047c0 2024-01-15T18:08:19.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.526+0000 7f3478c00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f345406bfb0 0x7f345406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:19.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.526+0000 7f3478c00640 1 -- 172.21.15.26:0/833851868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f346c088fe0 con 0x7f345c0047c0 2024-01-15T18:08:19.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.526+0000 7f347be00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f345406bfb0 0x7f345406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:19.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.527+0000 7f347be00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f345406bfb0 0x7f345406e470 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3464033000 tx=0x7f3464009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:19.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.530+0000 7f3478c00640 1 -- 172.21.15.26:0/833851868 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f346c059150 con 0x7f345c0047c0 2024-01-15T18:08:19.799 INFO:tasks.ceph.mds.d.smithi124.stderr:2024-01-15T18:08:19.793+0000 7f36a2400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:08:19.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.798+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f347c067450 con 0x7f345c0047c0 2024-01-15T18:08:19.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.804+0000 7f3478c00640 1 -- 172.21.15.26:0/833851868 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f346c001870 con 0x7f345c0047c0 2024-01-15T18:08:19.808 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-15T18:08:19.808 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA== 2024-01-15T18:08:19.808 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-01-15T18:08:19.808 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-01-15T18:08:19.808 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:08:19.808 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-01-15T18:08:19.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f345406bfb0 msgr2=0x7f345406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f345406bfb0 0x7f345406e470 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3464033000 tx=0x7f3464009030 comp rx=0 tx=0).stop 2024-01-15T18:08:19.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 msgr2=0x7f347c14f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:19.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f347c14f110 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f346c0079e0 tx=0x7f346c00e530 comp rx=0 tx=0).stop 2024-01-15T18:08:19.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 shutdown_connections 2024-01-15T18:08:19.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f347c0b2e80 0x7f347c0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f345406bfb0 0x7f345406e470 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f347c14c2a0 0x7f347c0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 --2- 172.21.15.26:0/833851868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f345c0047c0 0x7f347c14f110 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:19.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.807+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 >> 172.21.15.26:0/833851868 conn(0x7f347c0b0b70 msgr2=0x7f347c15e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:19.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.808+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 shutdown_connections 2024-01-15T18:08:19.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:19.808+0000 7f3481e00640 1 -- 172.21.15.26:0/833851868 wait complete. 2024-01-15T18:08:19.832 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:08:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.044+0000 7fab0a600640 1 Processor -- start 2024-01-15T18:08:20.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.045+0000 7fab0a600640 1 -- start start 2024-01-15T18:08:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.046+0000 7fab0a600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fab040d0b60 0x7fab040ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.046+0000 7fab0a600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab04071de0 con 0x7fab0406a3e0 2024-01-15T18:08:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.046+0000 7fab0a600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab040af270 con 0x7fab040aede0 2024-01-15T18:08:20.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.046+0000 7fab0a600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab04072030 con 0x7fab040d0b60 2024-01-15T18:08:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.047+0000 7fab03400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fab0406a3e0 0x7fab0406a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33598/0 (socket says 172.21.15.26:33598) 2024-01-15T18:08:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.047+0000 7fab03400640 1 -- 172.21.15.26:0/2775417444 learned_addr learned my addr 172.21.15.26:0/2775417444 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.047+0000 7fab02a00640 1 --2- 172.21.15.26:0/2775417444 >> v2:172.21.15.26:3300/0 conn(0x7fab040d0b60 0x7fab040ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.047+0000 7fab02a00640 1 -- 172.21.15.26:0/2775417444 >> v1:172.21.15.124:6789/0 conn(0x7fab040aede0 legacy=0x7fab0406f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.047+0000 7fab02a00640 1 -- 172.21.15.26:0/2775417444 >> v1:172.21.15.26:6789/0 conn(0x7fab0406a3e0 legacy=0x7fab0406a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.047+0000 7fab02000640 1 -- 172.21.15.26:0/2775417444 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462184688 0 0) 0x7fab04071de0 con 0x7fab0406a3e0 2024-01-15T18:08:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.047+0000 7fab02a00640 1 -- 172.21.15.26:0/2775417444 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04072310 con 0x7fab040d0b60 2024-01-15T18:08:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.049+0000 7fab02a00640 1 --2- 172.21.15.26:0/2775417444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040d0b60 0x7fab040ac8d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7faaec0097a0 tx=0x7faaec02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c193e21e21780db9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:20.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.049+0000 7fab02000640 1 -- 172.21.15.26:0/2775417444 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec030610 con 0x7fab040d0b60 2024-01-15T18:08:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.049+0000 7fab02000640 1 -- 172.21.15.26:0/2775417444 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec030770 con 0x7fab040d0b60 2024-01-15T18:08:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.049+0000 7fab02000640 1 -- 172.21.15.26:0/2775417444 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec030b00 con 0x7fab040d0b60 2024-01-15T18:08:20.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.050+0000 7fab0a600640 1 -- 172.21.15.26:0/2775417444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040d0b60 msgr2=0x7fab040ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.050+0000 7fab0a600640 1 --2- 172.21.15.26:0/2775417444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040d0b60 0x7fab040ac8d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7faaec0097a0 tx=0x7faaec02b960 comp rx=0 tx=0).stop 2024-01-15T18:08:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.050+0000 7fab0a600640 1 -- 172.21.15.26:0/2775417444 shutdown_connections 2024-01-15T18:08:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.050+0000 7fab0a600640 1 --2- 172.21.15.26:0/2775417444 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040d0b60 0x7fab040ac8d0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.050+0000 7fab0a600640 1 -- 172.21.15.26:0/2775417444 >> 172.21.15.26:0/2775417444 conn(0x7fab040cfef0 msgr2=0x7fab040d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.050+0000 7fab0a600640 1 -- 172.21.15.26:0/2775417444 shutdown_connections 2024-01-15T18:08:20.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.050+0000 7fab0a600640 1 -- 172.21.15.26:0/2775417444 wait complete. 2024-01-15T18:08:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.052+0000 7fab0a600640 1 Processor -- start 2024-01-15T18:08:20.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.052+0000 7fab0a600640 1 -- start start 2024-01-15T18:08:20.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.053+0000 7fab0a600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab0406a3e0 0x7fab041518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.053+0000 7fab0a600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040aede0 0x7fab04157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.053+0000 7fab0a600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040d0b60 0x7fab040cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.053+0000 7fab0a600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab04152170 con 0x7fab040aede0 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.053+0000 7fab0a600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab04157c60 con 0x7fab040d0b60 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.053+0000 7fab0a600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab04157e90 con 0x7fab0406a3e0 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.053+0000 7fab02a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040aede0 0x7fab04157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.053+0000 7fab02a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040aede0 0x7fab04157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60846/0 (socket says 172.21.15.26:60846) 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.053+0000 7fab02a00640 1 -- 172.21.15.26:0/3099425292 learned_addr learned my addr 172.21.15.26:0/3099425292 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.054+0000 7fab03400640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab0406a3e0 0x7fab041518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.054+0000 7fab03e00640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040d0b60 0x7fab040cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.054+0000 7fab02a00640 1 -- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab0406a3e0 msgr2=0x7fab041518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.054+0000 7fab02a00640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab0406a3e0 0x7fab041518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.054+0000 7fab02a00640 1 -- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040d0b60 msgr2=0x7fab040cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.054+0000 7fab02a00640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040d0b60 0x7fab040cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.054+0000 7fab02a00640 1 -- 172.21.15.26:0/3099425292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec009400 con 0x7fab040aede0 2024-01-15T18:08:20.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.054+0000 7fab02a00640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040aede0 0x7fab04157720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7faaec0098c0 tx=0x7faaec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.054+0000 7faafb400640 1 -- 172.21.15.26:0/3099425292 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec03e040 con 0x7fab040aede0 2024-01-15T18:08:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.055+0000 7faafb400640 1 -- 172.21.15.26:0/3099425292 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec039cb0 con 0x7fab040aede0 2024-01-15T18:08:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.055+0000 7faafb400640 1 -- 172.21.15.26:0/3099425292 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec044630 con 0x7fab040aede0 2024-01-15T18:08:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.055+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab04158170 con 0x7fab040aede0 2024-01-15T18:08:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.055+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab04158730 con 0x7fab040aede0 2024-01-15T18:08:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.056+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab040675d0 con 0x7fab040aede0 2024-01-15T18:08:20.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.057+0000 7faafb400640 1 -- 172.21.15.26:0/3099425292 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7faaec04d3b0 con 0x7fab040aede0 2024-01-15T18:08:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.058+0000 7faafb400640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faadc06bfe0 0x7faadc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.058+0000 7faafb400640 1 -- 172.21.15.26:0/3099425292 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7faaec0aad30 con 0x7fab040aede0 2024-01-15T18:08:20.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.058+0000 7fab03400640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faadc06bfe0 0x7faadc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.059+0000 7fab03400640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faadc06bfe0 0x7faadc06e4a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faaf40012e0 tx=0x7faaf4005c10 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:20.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.062+0000 7faafb400640 1 -- 172.21.15.26:0/3099425292 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faaec0775f0 con 0x7fab040aede0 2024-01-15T18:08:20.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.300+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab040ae570 con 0x7fab040aede0 2024-01-15T18:08:20.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.301+0000 7faafb400640 1 -- 172.21.15.26:0/3099425292 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4919 (secure 0 0 0) 0x7faaec049090 con 0x7fab040aede0 2024-01-15T18:08:20.305 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:08:20.305 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/923394527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":923394527},{"type":"v1","addr":"172.21.15.26:6833","nonce":923394527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/350014829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":350014829},{"type":"v1","addr":"172.21.15.26:6835","nonce":350014829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":36005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/1826406197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1826406197},{"type":"v1","addr":"172.21.15.124:6836","nonce":1826406197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:08:13.654102+0000","modified":"2024-01-15T18:08:16.725399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36008},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36008":{"gid":36008,"name":"d","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.124:6837/2740807412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":2740807412},{"type":"v1","addr":"172.21.15.124:6837","nonce":2740807412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-01-15T18:08:20.305 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 53 2024-01-15T18:08:20.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faadc06bfe0 msgr2=0x7faadc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faadc06bfe0 0x7faadc06e4a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7faaf40012e0 tx=0x7faaf4005c10 comp rx=0 tx=0).stop 2024-01-15T18:08:20.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040aede0 msgr2=0x7fab04157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040aede0 0x7fab04157720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7faaec0098c0 tx=0x7faaec004730 comp rx=0 tx=0).stop 2024-01-15T18:08:20.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 shutdown_connections 2024-01-15T18:08:20.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faadc06bfe0 0x7faadc06e4a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040d0b60 0x7fab040cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040aede0 0x7fab04157720 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 --2- 172.21.15.26:0/3099425292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab0406a3e0 0x7fab041518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 >> 172.21.15.26:0/3099425292 conn(0x7fab040cfef0 msgr2=0x7fab0415bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:20.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 shutdown_connections 2024-01-15T18:08:20.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.304+0000 7fab0a600640 1 -- 172.21.15.26:0/3099425292 wait complete. 2024-01-15T18:08:20.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:08:13.654102+0000', 'modified': '2024-01-15T18:08:16.725399+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 36008}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_36008': {'gid': 36008, 'name': 'd', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.124:6837/2740807412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6835', 'nonce': 2740807412}, {'type': 'v1', 'addr': '172.21.15.124:6837', 'nonce': 2740807412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:08:20.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:08:20.730 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-15T18:08:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.959+0000 7f43a2600640 1 Processor -- start 2024-01-15T18:08:20.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.960+0000 7f43a2600640 1 -- start start 2024-01-15T18:08:20.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.961+0000 7f43a2600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f439c143c40 0x7f439c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.961+0000 7f43a2600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f439c0ccc10 0x7f439c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.961+0000 7f43a2600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439c070e50 con 0x7f439c0717d0 2024-01-15T18:08:20.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.961+0000 7f43a2600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c0723c0 con 0x7f439c0ccc10 2024-01-15T18:08:20.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.961+0000 7f43a2600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c144020 con 0x7f439c143c40 2024-01-15T18:08:20.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.962+0000 7f4393e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f439c0717d0 0x7f439c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33604/0 (socket says 172.21.15.26:33604) 2024-01-15T18:08:20.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.962+0000 7f4393e00640 1 -- 172.21.15.26:0/1422752864 learned_addr learned my addr 172.21.15.26:0/1422752864 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:20.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.962+0000 7f43a1c00640 1 --2- 172.21.15.26:0/1422752864 >> v2:172.21.15.26:3300/0 conn(0x7f439c0ccc10 0x7f439c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.962+0000 7f43a1200640 1 --2- 172.21.15.26:0/1422752864 >> v2:172.21.15.124:3300/0 conn(0x7f439c143c40 0x7f439c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.962+0000 7f439be00640 1 -- 172.21.15.26:0/1422752864 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834647541 0 0) 0x7f439c070e50 con 0x7f439c0717d0 2024-01-15T18:08:20.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.962+0000 7f439be00640 1 -- 172.21.15.26:0/1422752864 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f437c003520 con 0x7f439c0717d0 2024-01-15T18:08:20.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.962+0000 7f43a1c00640 1 -- 172.21.15.26:0/1422752864 >> v2:172.21.15.124:3300/0 conn(0x7f439c143c40 msgr2=0x7f439c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:20.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.962+0000 7f43a1c00640 1 --2- 172.21.15.26:0/1422752864 >> v2:172.21.15.124:3300/0 conn(0x7f439c143c40 0x7f439c071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.962+0000 7f43a1c00640 1 -- 172.21.15.26:0/1422752864 >> v1:172.21.15.26:6789/0 conn(0x7f439c0717d0 legacy=0x7f439c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.963+0000 7f43a1c00640 1 -- 172.21.15.26:0/1422752864 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c071e00 con 0x7f439c0ccc10 2024-01-15T18:08:20.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.964+0000 7f43a1c00640 1 --2- 172.21.15.26:0/1422752864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c0ccc10 0x7f439c06d160 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f439c06e410 tx=0x7f439402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab1f806c95fb13f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.964+0000 7f439be00640 1 -- 172.21.15.26:0/1422752864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439402cbe0 con 0x7f439c0ccc10 2024-01-15T18:08:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.964+0000 7f439be00640 1 -- 172.21.15.26:0/1422752864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439402cd40 con 0x7f439c0ccc10 2024-01-15T18:08:20.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.964+0000 7f439be00640 1 -- 172.21.15.26:0/1422752864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394034660 con 0x7f439c0ccc10 2024-01-15T18:08:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.965+0000 7f43a2600640 1 -- 172.21.15.26:0/1422752864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c0ccc10 msgr2=0x7f439c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.965+0000 7f43a2600640 1 --2- 172.21.15.26:0/1422752864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c0ccc10 0x7f439c06d160 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f439c06e410 tx=0x7f439402be30 comp rx=0 tx=0).stop 2024-01-15T18:08:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.965+0000 7f43a2600640 1 -- 172.21.15.26:0/1422752864 shutdown_connections 2024-01-15T18:08:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.965+0000 7f43a2600640 1 --2- 172.21.15.26:0/1422752864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c0ccc10 0x7f439c06d160 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.965+0000 7f43a2600640 1 --2- 172.21.15.26:0/1422752864 >> v2:172.21.15.124:3300/0 conn(0x7f439c143c40 0x7f439c071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.965+0000 7f43a2600640 1 -- 172.21.15.26:0/1422752864 >> 172.21.15.26:0/1422752864 conn(0x7f439c06a800 msgr2=0x7f439c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.965+0000 7f43a2600640 1 -- 172.21.15.26:0/1422752864 shutdown_connections 2024-01-15T18:08:20.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.965+0000 7f43a2600640 1 -- 172.21.15.26:0/1422752864 wait complete. 2024-01-15T18:08:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.967+0000 7f43a2600640 1 Processor -- start 2024-01-15T18:08:20.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.967+0000 7f43a2600640 1 -- start start 2024-01-15T18:08:20.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.968+0000 7f43a2600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f439c0717d0 0x7f439c133450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.968+0000 7f43a2600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f439c0ccc10 0x7f439c1426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.968+0000 7f43a2600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c143c40 0x7f439c142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.968+0000 7f43a2600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c133990 con 0x7f439c143c40 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.968+0000 7f43a2600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c151ac0 con 0x7f439c0717d0 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.968+0000 7f43a2600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439c151da0 con 0x7f439c0ccc10 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.968+0000 7f43a1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c143c40 0x7f439c142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c143c40 0x7f439c142c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60862/0 (socket says 172.21.15.26:60862) 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1c00640 1 -- 172.21.15.26:0/1397095036 learned_addr learned my addr 172.21.15.26:0/1397095036 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1200640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f439c0717d0 0x7f439c133450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1c00640 1 -- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f439c0ccc10 msgr2=0x7f439c1426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1c00640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f439c0ccc10 0x7f439c1426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1c00640 1 -- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f439c0717d0 msgr2=0x7f439c133450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1c00640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f439c0717d0 0x7f439c133450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1c00640 1 -- 172.21.15.26:0/1397095036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4394009a70 con 0x7f439c143c40 2024-01-15T18:08:20.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1200640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f439c0717d0 0x7f439c133450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:20.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.969+0000 7f43a1c00640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c143c40 0x7f439c142c20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f4394037040 tx=0x7f4394004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:20.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.970+0000 7f4399600640 1 -- 172.21.15.26:0/1397095036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394040650 con 0x7f439c143c40 2024-01-15T18:08:20.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.970+0000 7f4399600640 1 -- 172.21.15.26:0/1397095036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43940407b0 con 0x7f439c143c40 2024-01-15T18:08:20.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.970+0000 7f4399600640 1 -- 172.21.15.26:0/1397095036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394040b60 con 0x7f439c143c40 2024-01-15T18:08:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.970+0000 7f43a2600640 1 -- 172.21.15.26:0/1397095036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439c152080 con 0x7f439c143c40 2024-01-15T18:08:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.970+0000 7f43a2600640 1 -- 172.21.15.26:0/1397095036 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c1542d0 con 0x7f439c143c40 2024-01-15T18:08:20.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.972+0000 7f4399600640 1 -- 172.21.15.26:0/1397095036 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f4394040cc0 con 0x7f439c143c40 2024-01-15T18:08:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.973+0000 7f4399600640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437406bfe0 0x7f437406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.973+0000 7f4399600640 1 -- 172.21.15.26:0/1397095036 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f43940ab5a0 con 0x7f439c143c40 2024-01-15T18:08:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.974+0000 7f43a1200640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437406bfe0 0x7f437406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.974+0000 7f43a6977740 1 Processor -- start 2024-01-15T18:08:20.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.974+0000 7f43a6977740 1 -- start start 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a6977740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ec1070 0x562302ec1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a6977740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ebbd70 0x562302ec0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a6977740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec07b0 0x562302f04730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a6977740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562302e1f880 con 0x562302ec07b0 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a6977740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562302e2eae0 con 0x562302ebbd70 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a6977740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562302e2c1d0 con 0x562302ec1070 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a1200640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437406bfe0 0x7f437406e4a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f438c009ac0 tx=0x7f438c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec07b0 0x562302f04730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec07b0 0x562302f04730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60868/0 (socket says 172.21.15.26:60868) 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a1c00640 1 -- 172.21.15.26:0/2140508368 learned_addr learned my addr 172.21.15.26:0/2140508368 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f43a1200640 1 --2- 172.21.15.26:0/2140508368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ec1070 0x562302ec1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.975+0000 7f4393e00640 1 --2- 172.21.15.26:0/2140508368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ebbd70 0x562302ec0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.976+0000 7f43a1c00640 1 -- 172.21.15.26:0/2140508368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ec1070 msgr2=0x562302ec1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.976+0000 7f43a1c00640 1 --2- 172.21.15.26:0/2140508368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ec1070 0x562302ec1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.976+0000 7f43a1c00640 1 -- 172.21.15.26:0/2140508368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ebbd70 msgr2=0x562302ec0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.976+0000 7f43a1c00640 1 --2- 172.21.15.26:0/2140508368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ebbd70 0x562302ec0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.976+0000 7f43a1c00640 1 -- 172.21.15.26:0/2140508368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562302def860 con 0x562302ec07b0 2024-01-15T18:08:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.976+0000 7f43a1c00640 1 --2- 172.21.15.26:0/2140508368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec07b0 0x562302f04730 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f43940ab450 tx=0x7f439407b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2ad1e023c79e73 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.976+0000 7f4391600640 1 -- 172.21.15.26:0/2140508368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394093180 con 0x562302ec07b0 2024-01-15T18:08:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.976+0000 7f4391600640 1 -- 172.21.15.26:0/2140508368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439407b890 con 0x562302ec07b0 2024-01-15T18:08:20.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.976+0000 7f4391600640 1 -- 172.21.15.26:0/2140508368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439407f9e0 con 0x562302ec07b0 2024-01-15T18:08:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.977+0000 7f43a6977740 1 -- 172.21.15.26:0/2140508368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec07b0 msgr2=0x562302f04730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.977+0000 7f43a6977740 1 --2- 172.21.15.26:0/2140508368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec07b0 0x562302f04730 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f43940ab450 tx=0x7f439407b490 comp rx=0 tx=0).stop 2024-01-15T18:08:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.977+0000 7f43a6977740 1 -- 172.21.15.26:0/2140508368 shutdown_connections 2024-01-15T18:08:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.977+0000 7f43a6977740 1 --2- 172.21.15.26:0/2140508368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec07b0 0x562302f04730 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.977+0000 7f43a6977740 1 --2- 172.21.15.26:0/2140508368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ebbd70 0x562302ec0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.977+0000 7f43a6977740 1 --2- 172.21.15.26:0/2140508368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ec1070 0x562302ec1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.977+0000 7f43a6977740 1 -- 172.21.15.26:0/2140508368 >> 172.21.15.26:0/2140508368 conn(0x562302e82c70 msgr2=0x562302f06b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.977+0000 7f43a6977740 1 -- 172.21.15.26:0/2140508368 shutdown_connections 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.977+0000 7f43a6977740 1 -- 172.21.15.26:0/2140508368 wait complete. 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.978+0000 7f43a6977740 1 -- start start 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.978+0000 7f43a6977740 1 Processor -- start 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a6977740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 0x562302ee84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a6977740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f0ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a6977740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302f0cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a6977740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562302e27ac0 con 0x562302ec1070 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a6977740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562302dc42f0 con 0x562302ec07b0 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a6977740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562302de1c70 con 0x562302ebbd70 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a6977740 10 client.0 _ll_register_callbacks cb 0x562302de63b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:08:20.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302f0cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302f0cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60878/0 (socket says 172.21.15.26:60878) 2024-01-15T18:08:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.979+0000 7f43a1c00640 1 -- 172.21.15.26:0/3859004032 learned_addr learned my addr 172.21.15.26:0/3859004032 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f43a1c00640 1 -- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 msgr2=0x562302ee84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f43a1c00640 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 0x562302ee84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f43a1c00640 1 -- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 msgr2=0x562302f0ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f43a1c00640 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f0ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f43a1c00640 1 -- 172.21.15.26:0/3859004032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c1542d0 con 0x562302ec1070 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f43a1c00640 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302f0cf80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4394079be0 tx=0x7f439402c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3efd0f09b402c5e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f438a000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f438a000640 1 -- 172.21.15.26:0/3859004032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439409a200 con 0x562302ec1070 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f438a000640 1 -- 172.21.15.26:0/3859004032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4394083b90 con 0x562302ec1070 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f43a6977740 1 -- 172.21.15.26:0/3859004032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562302e20aa0 con 0x562302ec1070 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f438a000640 1 -- 172.21.15.26:0/3859004032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f439407f9e0 con 0x562302ec1070 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f43a6977740 10 client.8373 fetch_fsmap learned FSMap version 53 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.980+0000 7f43a6977740 1 -- 172.21.15.26:0/3859004032 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x562302e26140 con 0x562302ec1070 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.981+0000 7f438a000640 1 -- 172.21.15.26:0/3859004032 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562302e20aa0 con 0x562302ec1070 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.981+0000 7f438a000640 1 -- 172.21.15.26:0/3859004032 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4394050020 con 0x562302ec1070 2024-01-15T18:08:20.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.981+0000 7f43a6977740 10 client.8373 fetch_fsmap finished waiting for FSMap version 53 2024-01-15T18:08:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.981+0000 7f43a6977740 10 client.8373 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:08:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.981+0000 7f43a6977740 20 client.8373 populate_metadata read hostname 'smithi026' 2024-01-15T18:08:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.981+0000 7f43a6977740 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] conn(0x562302f0e980 0x562302f0ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.981+0000 7f43a6977740 4 client.8373 mds_command: new command op to 36008 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:08:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.981+0000 7f43a6977740 1 -- 172.21.15.26:0/3859004032 --> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562302e207c0 con 0x562302f0e980 2024-01-15T18:08:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.982+0000 7f43a1200640 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] conn(0x562302f0e980 0x562302f0ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.983+0000 7f43a1200640 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] conn(0x562302f0e980 0x562302f0ed60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54f7eddaee63300f server_cookie=a16524b2e906a5d6 in_seq=0 out_seq=0 2024-01-15T18:08:20.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.983+0000 7f438a000640 10 client.8373 ms_handle_connect on v2:172.21.15.124:6835/2740807412 2024-01-15T18:08:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f438a000640 1 -- 172.21.15.26:0/3859004032 <== mds.0 v2:172.21.15.124:6835/2740807412 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562302e207c0 con 0x562302f0e980 2024-01-15T18:08:20.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f438a000640 10 client.8373 handle_command_reply: tid=0 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 client.8373 shutdown 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 -- 172.21.15.26:0/3859004032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 msgr2=0x562302f0cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302f0cf80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f4394079be0 tx=0x7f439402c3d0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 -- 172.21.15.26:0/3859004032 shutdown_connections 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] conn(0x562302f0e980 0x562302f0ed60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302f0cf80 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f0ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 --2- 172.21.15.26:0/3859004032 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 0x562302ee84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 -- 172.21.15.26:0/3859004032 >> 172.21.15.26:0/3859004032 conn(0x562302e82c70 msgr2=0x562302ebadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f438a000640 0 client.8373 ms_handle_reset on v2:172.21.15.124:6835/2740807412 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 -- 172.21.15.26:0/3859004032 shutdown_connections 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 1 -- 172.21.15.26:0/3859004032 wait complete. 2024-01-15T18:08:20.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:20.985+0000 7f43a6977740 20 client.8373 trim_cache size 0 max 16384 2024-01-15T18:08:21.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.003+0000 7f43a6977740 1 Processor -- start 2024-01-15T18:08:21.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.003+0000 7f43a6977740 1 -- start start 2024-01-15T18:08:21.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f43a6977740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ebbd70 0x562302ee9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f43a6977740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f15c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f43a6977740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ec1070 0x562302f161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f43a6977740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562302b8af20 con 0x562302ebbd70 2024-01-15T18:08:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f43a6977740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562302ebdd60 con 0x562302ec07b0 2024-01-15T18:08:21.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f43a6977740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562302eb80b0 con 0x562302ec1070 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f43a1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ebbd70 0x562302ee9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f43a1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ebbd70 0x562302ee9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60886/0 (socket says 172.21.15.26:60886) 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f43a1200640 1 -- 172.21.15.26:0/4142218995 learned_addr learned my addr 172.21.15.26:0/4142218995 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.004+0000 7f4393e00640 1 --2- 172.21.15.26:0/4142218995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f15c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.005+0000 7f43a1200640 1 -- 172.21.15.26:0/4142218995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ec1070 msgr2=0x562302f161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.005+0000 7f43a1200640 1 --2- 172.21.15.26:0/4142218995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ec1070 0x562302f161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.005+0000 7f43a1200640 1 -- 172.21.15.26:0/4142218995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 msgr2=0x562302f15c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.005+0000 7f43a1200640 1 --2- 172.21.15.26:0/4142218995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f15c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.005+0000 7f43a1200640 1 -- 172.21.15.26:0/4142218995 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562302dc3b80 con 0x562302ebbd70 2024-01-15T18:08:21.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.005+0000 7f43a1200640 1 --2- 172.21.15.26:0/4142218995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ebbd70 0x562302ee9660 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f438c010b90 tx=0x7f438c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1408cf89ccbff218 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.005+0000 7f4392000640 1 -- 172.21.15.26:0/4142218995 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c015d70 con 0x562302ebbd70 2024-01-15T18:08:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.005+0000 7f4392000640 1 -- 172.21.15.26:0/4142218995 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c027150 con 0x562302ebbd70 2024-01-15T18:08:21.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.005+0000 7f4392000640 1 -- 172.21.15.26:0/4142218995 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c0274e0 con 0x562302ebbd70 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.006+0000 7f43a6977740 1 -- 172.21.15.26:0/4142218995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ebbd70 msgr2=0x562302ee9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.006+0000 7f43a6977740 1 --2- 172.21.15.26:0/4142218995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ebbd70 0x562302ee9660 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f438c010b90 tx=0x7f438c013000 comp rx=0 tx=0).stop 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.006+0000 7f43a6977740 1 -- 172.21.15.26:0/4142218995 shutdown_connections 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.006+0000 7f43a6977740 1 --2- 172.21.15.26:0/4142218995 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ec1070 0x562302f161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.006+0000 7f43a6977740 1 --2- 172.21.15.26:0/4142218995 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f15c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.006+0000 7f43a6977740 1 --2- 172.21.15.26:0/4142218995 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ebbd70 0x562302ee9660 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.006+0000 7f43a6977740 1 -- 172.21.15.26:0/4142218995 >> 172.21.15.26:0/4142218995 conn(0x562302e82c70 msgr2=0x562302ee82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.006+0000 7f43a6977740 1 -- 172.21.15.26:0/4142218995 shutdown_connections 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.006+0000 7f43a6977740 1 -- 172.21.15.26:0/4142218995 wait complete. 2024-01-15T18:08:21.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.007+0000 7f43a6977740 1 -- start start 2024-01-15T18:08:21.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.007+0000 7f43a6977740 1 Processor -- start 2024-01-15T18:08:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a6977740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 0x562302f59450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a6977740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f16080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a6977740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302ee9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a6977740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x562302eedd30 con 0x562302ec1070 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a6977740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x562302f0d6d0 con 0x562302ec07b0 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a6977740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x562302f19880 con 0x562302ebbd70 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a6977740 10 client.0 _ll_register_callbacks cb 0x7f439407b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302ee9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a1c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302ee9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60888/0 (socket says 172.21.15.26:60888) 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a1c00640 1 -- 172.21.15.26:0/2164026116 learned_addr learned my addr 172.21.15.26:0/2164026116 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.008+0000 7f43a1200640 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 0x562302f59450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f4393e00640 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f16080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f43a1c00640 1 -- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 msgr2=0x562302f59450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f43a1c00640 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 0x562302f59450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f43a1c00640 1 -- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 msgr2=0x562302f16080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f43a1c00640 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f16080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f43a1c00640 1 -- 172.21.15.26:0/2164026116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c00e050 con 0x562302ec1070 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f43a1200640 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 0x562302f59450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f43a1c00640 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302ee9660 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f4394077d90 tx=0x7f4394009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34ae34d65f9c620 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f438a000640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f438a000640 1 -- 172.21.15.26:0/2164026116 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439409a200 con 0x562302ec1070 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f438a000640 1 -- 172.21.15.26:0/2164026116 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4394083b90 con 0x562302ec1070 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.009+0000 7f43a6977740 1 -- 172.21.15.26:0/2164026116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f439407b890 con 0x562302ec1070 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f438a000640 1 -- 172.21.15.26:0/2164026116 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f439407f9e0 con 0x562302ec1070 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f43a6977740 10 client.8379 fetch_fsmap learned FSMap version 53 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f43a6977740 1 -- 172.21.15.26:0/2164026116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x562302de9170 con 0x562302ec1070 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f438a000640 1 -- 172.21.15.26:0/2164026116 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439407b890 con 0x562302ec1070 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f438a000640 1 -- 172.21.15.26:0/2164026116 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4394050040 con 0x562302ec1070 2024-01-15T18:08:21.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f43a6977740 10 client.8379 fetch_fsmap finished waiting for FSMap version 53 2024-01-15T18:08:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f43a6977740 10 client.8379 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:08:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f43a6977740 20 client.8379 populate_metadata read hostname 'smithi026' 2024-01-15T18:08:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f43a6977740 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] conn(0x562302ee9d90 0x562302eed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f43a6977740 4 client.8379 mds_command: new command op to 36008 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:08:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.010+0000 7f43a6977740 1 -- 172.21.15.26:0/2164026116 --> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562302b286c0 con 0x562302ee9d90 2024-01-15T18:08:21.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.011+0000 7f43a1200640 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] conn(0x562302ee9d90 0x562302eed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.012+0000 7f43a1200640 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] conn(0x562302ee9d90 0x562302eed140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20dd5b4ac5b676f server_cookie=2bcfb0c6f17f9ec3 in_seq=0 out_seq=0 2024-01-15T18:08:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.012+0000 7f438a000640 10 client.8379 ms_handle_connect on v2:172.21.15.124:6835/2740807412 2024-01-15T18:08:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.012+0000 7f438a000640 1 -- 172.21.15.26:0/2164026116 <== mds.0 v2:172.21.15.124:6835/2740807412 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x562302b286c0 con 0x562302ee9d90 2024-01-15T18:08:21.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.012+0000 7f438a000640 10 client.8379 handle_command_reply: tid=0 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.012+0000 7f43a6977740 1 client.8379 shutdown 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.012+0000 7f43a6977740 1 -- 172.21.15.26:0/2164026116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 msgr2=0x562302ee9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.012+0000 7f43a6977740 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302ee9660 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f4394077d90 tx=0x7f4394009750 comp rx=0 tx=0).stop 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f43a6977740 1 -- 172.21.15.26:0/2164026116 shutdown_connections 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f43a6977740 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:6835/2740807412,v1:172.21.15.124:6837/2740807412] conn(0x562302ee9d90 0x562302eed140 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f43a6977740 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x562302ec1070 0x562302ee9660 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f43a6977740 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x562302ec07b0 0x562302f16080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f43a6977740 1 --2- 172.21.15.26:0/2164026116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x562302ebbd70 0x562302f59450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f438a000640 0 client.8379 ms_handle_reset on v2:172.21.15.124:6835/2740807412 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f43a6977740 1 -- 172.21.15.26:0/2164026116 >> 172.21.15.26:0/2164026116 conn(0x562302e82c70 msgr2=0x562302eef490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f43a6977740 1 -- 172.21.15.26:0/2164026116 shutdown_connections 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f43a6977740 1 -- 172.21.15.26:0/2164026116 wait complete. 2024-01-15T18:08:21.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.013+0000 7f43a6977740 20 client.8379 trim_cache size 0 max 16384 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 36008, 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 6.8317539260000002, 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 53, 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 87, 2024-01-15T18:08:21.020 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 87, 2024-01-15T18:08:21.021 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 12.547211186 2024-01-15T18:08:21.021 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:08:21.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 -- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437406bfe0 msgr2=0x7f437406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437406bfe0 0x7f437406e4a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f438c009ac0 tx=0x7f438c008040 comp rx=0 tx=0).stop 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 -- 172.21.15.26:0/1397095036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c143c40 msgr2=0x7f439c142c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c143c40 0x7f439c142c20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f4394037040 tx=0x7f4394004a40 comp rx=0 tx=0).stop 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 -- 172.21.15.26:0/1397095036 shutdown_connections 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f439c0ccc10 0x7f439c1426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437406bfe0 0x7f437406e4a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f439c143c40 0x7f439c142c20 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 --2- 172.21.15.26:0/1397095036 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f439c0717d0 0x7f439c133450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 -- 172.21.15.26:0/1397095036 >> 172.21.15.26:0/1397095036 conn(0x7f439c06a800 msgr2=0x7f439c06d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 -- 172.21.15.26:0/1397095036 shutdown_connections 2024-01-15T18:08:21.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.014+0000 7f43a2600640 1 -- 172.21.15.26:0/1397095036 wait complete. 2024-01-15T18:08:21.034 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:08:21.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.243+0000 7fd013c00640 1 Processor -- start 2024-01-15T18:08:21.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.244+0000 7fd013c00640 1 -- start start 2024-01-15T18:08:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.245+0000 7fd013c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd00c09a170 0x7fd00c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.245+0000 7fd013c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd00c06c6b0 0x7fd00c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.245+0000 7fd013c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c06cea0 con 0x7fd00c0cd240 2024-01-15T18:08:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.245+0000 7fd013c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c06cbc0 con 0x7fd00c06c6b0 2024-01-15T18:08:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.245+0000 7fd013c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c06f350 con 0x7fd00c09a170 2024-01-15T18:08:21.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd011200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd00c0cd240 0x7fd00c099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33620/0 (socket says 172.21.15.26:33620) 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd011200640 1 -- 172.21.15.26:0/300681032 learned_addr learned my addr 172.21.15.26:0/300681032 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd011c00640 1 --2- 172.21.15.26:0/300681032 >> v2:172.21.15.26:3300/0 conn(0x7fd00c06c6b0 0x7fd00c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd003e00640 1 --2- 172.21.15.26:0/300681032 >> v2:172.21.15.124:3300/0 conn(0x7fd00c09a170 0x7fd00c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd00be00640 1 -- 172.21.15.26:0/300681032 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281028298 0 0) 0x7fd00c06cea0 con 0x7fd00c0cd240 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd00be00640 1 -- 172.21.15.26:0/300681032 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfec003520 con 0x7fd00c0cd240 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd011c00640 1 -- 172.21.15.26:0/300681032 >> v2:172.21.15.124:3300/0 conn(0x7fd00c09a170 msgr2=0x7fd00c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd011c00640 1 --2- 172.21.15.26:0/300681032 >> v2:172.21.15.124:3300/0 conn(0x7fd00c09a170 0x7fd00c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd011c00640 1 -- 172.21.15.26:0/300681032 >> v1:172.21.15.26:6789/0 conn(0x7fd00c0cd240 legacy=0x7fd00c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd011c00640 1 -- 172.21.15.26:0/300681032 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c06f630 con 0x7fd00c06c6b0 2024-01-15T18:08:21.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.246+0000 7fd003e00640 1 --2- 172.21.15.26:0/300681032 >> v2:172.21.15.124:3300/0 conn(0x7fd00c09a170 0x7fd00c0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:21.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.248+0000 7fd011c00640 1 --2- 172.21.15.26:0/300681032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c06c6b0 0x7fd00c0cf640 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd00c071800 tx=0x7fd00402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdce5fd1f8b2f896 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.248+0000 7fd00be00640 1 -- 172.21.15.26:0/300681032 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00402cc70 con 0x7fd00c06c6b0 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.248+0000 7fd00be00640 1 -- 172.21.15.26:0/300681032 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00402cdd0 con 0x7fd00c06c6b0 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.248+0000 7fd00be00640 1 -- 172.21.15.26:0/300681032 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004034640 con 0x7fd00c06c6b0 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.249+0000 7fd013c00640 1 -- 172.21.15.26:0/300681032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c06c6b0 msgr2=0x7fd00c0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.249+0000 7fd013c00640 1 --2- 172.21.15.26:0/300681032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c06c6b0 0x7fd00c0cf640 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd00c071800 tx=0x7fd00402bec0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.249+0000 7fd013c00640 1 -- 172.21.15.26:0/300681032 shutdown_connections 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.249+0000 7fd013c00640 1 --2- 172.21.15.26:0/300681032 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c06c6b0 0x7fd00c0cf640 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.249+0000 7fd013c00640 1 --2- 172.21.15.26:0/300681032 >> v2:172.21.15.124:3300/0 conn(0x7fd00c09a170 0x7fd00c0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.249+0000 7fd013c00640 1 -- 172.21.15.26:0/300681032 >> 172.21.15.26:0/300681032 conn(0x7fd00c0b2d00 msgr2=0x7fd00c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.249+0000 7fd013c00640 1 -- 172.21.15.26:0/300681032 shutdown_connections 2024-01-15T18:08:21.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.249+0000 7fd013c00640 1 -- 172.21.15.26:0/300681032 wait complete. 2024-01-15T18:08:21.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.251+0000 7fd013c00640 1 Processor -- start 2024-01-15T18:08:21.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.251+0000 7fd013c00640 1 -- start start 2024-01-15T18:08:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.252+0000 7fd013c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd00c06c6b0 0x7fd00c1f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.252+0000 7fd013c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c09a170 0x7fd00c14f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.252+0000 7fd013c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd00c0cd240 0x7fd00c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.252+0000 7fd013c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c1f2ce0 con 0x7fd00c09a170 2024-01-15T18:08:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.252+0000 7fd013c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0c9a50 con 0x7fd00c06c6b0 2024-01-15T18:08:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.252+0000 7fd013c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c0c9d30 con 0x7fd00c0cd240 2024-01-15T18:08:21.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.252+0000 7fd003e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c09a170 0x7fd00c14f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.252+0000 7fd003e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c09a170 0x7fd00c14f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60908/0 (socket says 172.21.15.26:60908) 2024-01-15T18:08:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.252+0000 7fd003e00640 1 -- 172.21.15.26:0/3234629800 learned_addr learned my addr 172.21.15.26:0/3234629800 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:21.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd011200640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd00c06c6b0 0x7fd00c1f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd011c00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd00c0cd240 0x7fd00c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd003e00640 1 -- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd00c0cd240 msgr2=0x7fd00c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd003e00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd00c0cd240 0x7fd00c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd003e00640 1 -- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd00c06c6b0 msgr2=0x7fd00c1f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd003e00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd00c06c6b0 0x7fd00c1f27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd003e00640 1 -- 172.21.15.26:0/3234629800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004009a70 con 0x7fd00c09a170 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd003e00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c09a170 0x7fd00c14f410 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fcff400e730 tx=0x7fcff400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd009600640 1 -- 172.21.15.26:0/3234629800 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4004230 con 0x7fd00c09a170 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd009600640 1 -- 172.21.15.26:0/3234629800 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff4004390 con 0x7fd00c09a170 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd009600640 1 -- 172.21.15.26:0/3234629800 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff400c8f0 con 0x7fd00c09a170 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.253+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c0ca010 con 0x7fd00c09a170 2024-01-15T18:08:21.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.254+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c0ca5d0 con 0x7fd00c09a170 2024-01-15T18:08:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.255+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c06fd30 con 0x7fd00c09a170 2024-01-15T18:08:21.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.256+0000 7fd009600640 1 -- 172.21.15.26:0/3234629800 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7fcff400ca50 con 0x7fd00c09a170 2024-01-15T18:08:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.257+0000 7fd009600640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcfe406bfb0 0x7fcfe406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:21.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.257+0000 7fd009600640 1 -- 172.21.15.26:0/3234629800 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcff400ee40 con 0x7fd00c09a170 2024-01-15T18:08:21.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.257+0000 7fd011200640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcfe406bfb0 0x7fcfe406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:21.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.258+0000 7fd011200640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcfe406bfb0 0x7fcfe406e470 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005880 tx=0x7fcffc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:21.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.262+0000 7fd009600640 1 -- 172.21.15.26:0/3234629800 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcff4054b40 con 0x7fd00c09a170 2024-01-15T18:08:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.499+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd00c0675d0 con 0x7fd00c09a170 2024-01-15T18:08:21.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.500+0000 7fd009600640 1 -- 172.21.15.26:0/3234629800 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcff4020020 con 0x7fd00c09a170 2024-01-15T18:08:21.503 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:08:21.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcfe406bfb0 msgr2=0x7fcfe406e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcfe406bfb0 0x7fcfe406e470 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fcffc005880 tx=0x7fcffc009040 comp rx=0 tx=0).stop 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c09a170 msgr2=0x7fd00c14f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c09a170 0x7fd00c14f410 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fcff400e730 tx=0x7fcff400ebf0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 shutdown_connections 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd00c0cd240 0x7fd00c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcfe406bfb0 0x7fcfe406e470 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd00c09a170 0x7fd00c14f410 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 --2- 172.21.15.26:0/3234629800 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd00c06c6b0 0x7fd00c1f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 >> 172.21.15.26:0/3234629800 conn(0x7fd00c0b2d00 msgr2=0x7fd00c06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 shutdown_connections 2024-01-15T18:08:21.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:21.503+0000 7fd013c00640 1 -- 172.21.15.26:0/3234629800 wait complete. 2024-01-15T18:08:21.526 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:08:21.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:08:21.526 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-15T18:08:21.526 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:08:21.526 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:08:21.526 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:08:21.526 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:08:21.526 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:08:21.526 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:08:21.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:21.527 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-15T18:08:21.580 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:08:21.580 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:08:21.580 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:08:21.580 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:21.580 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T18:08:21.580 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:21.580 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:08:21.580 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41816sec preferred_lft 41816sec 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:08:21.581 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::a4e5:3ff:fe91:fcad/64 scope link 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::f464:8fff:fecd:58d2/64 scope link 2024-01-15T18:08:21.582 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:21.583 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:08:21.583 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether fe:82:c8:bc:3f:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:08:21.583 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::fc82:c8ff:febc:3fd2/64 scope link 2024-01-15T18:08:21.583 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:21.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:21.584 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:08:21.677 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-15T18:08:21.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:21.682 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-15T18:08:21.742 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-15T18:08:21.742 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:08:21.742 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-15T18:08:21.743 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-15T18:08:21.743 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:08:21.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:21.743 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T18:08:21.743 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T18:08:21.743 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-15T18:08:21.743 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-15T18:08:21.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-15T18:08:21.744 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T18:08:21.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:21.941 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T18:08:21.989 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:08:21.990 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:08:21.991 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:08:21.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:21.991 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-15T18:08:22.084 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:08:22.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:08:22.085 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:08:22.142 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:08:22.143 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:08:22.143 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:08:22.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:08:22.299 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:08:22.356 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:08:22.356 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:08:22.356 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:08:22.356 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:08:22.356 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:08:22.356 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:08:22.356 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:08:22.357 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:08:22.358 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:08:22.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:22.359 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:08:22.408 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:08:22.408 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:08:22.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:08:22.492 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:08:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:08:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:08:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:08:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:08:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:08:22.542 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:08:22.543 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:08:22.544 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:08:22.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:22.546 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:08:22.608 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:2024-01-15T18:08:22.604+0000 7fb623c0b480 -1 init, newargv = 0x55fa6bdad9d0 newargc=15 2024-01-15T18:08:22.608 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[69208]: starting ceph client 2024-01-15T18:08:22.618 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[69208]: starting fuse 2024-01-15T18:08:23.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:08:23.803 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:08:23.862 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:08:23.862 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:08:23.862 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:08:23.862 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:08:23.863 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:08:23.864 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:08:23.865 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:08:23.865 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:08:23.865 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:08:23.865 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:08:23.865 INFO:teuthology.orchestra.run.smithi026.stdout:/home/ubuntu/cephtest/mnt.0 2024-01-15T18:08:23.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:23.866 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:08:23.921 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T18:08:23.922 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:08:23.922 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:08:23.922 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-01-15T18:08:23.922 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-01-15T18:08:23.922 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-01-15T18:08:23.922 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-01-15T18:08:23.922 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:08:23.922 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-01-15T18:08:23.923 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-01-15T18:08:23.924 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:08:23.924 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:08:23.924 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.0")) 2024-01-15T18:08:23.924 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-01-15T18:08:24.113 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.0.69208.asok 2024-01-15T18:08:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:08:24.131 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.69208.asok 2024-01-15T18:08:24.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:24.132 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.69208.asok status 2024-01-15T18:08:24.374 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:08:24.374 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "0", 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "69208", 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8394, 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-01-15T18:08:24.375 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 8394 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3879420060 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3879420060 2024-01-15T18:08:24.376 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:08:24.377 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.8394 192.168.0.1:0/3879420060", 2024-01-15T18:08:24.377 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/3879420060", 2024-01-15T18:08:24.377 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-01-15T18:08:24.377 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 53, 2024-01-15T18:08:24.377 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 87, 2024-01-15T18:08:24.377 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:08:24.377 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-01-15T18:08:24.377 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-01-15T18:08:24.377 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:08:24.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:24.390 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-01-15T18:08:24.390 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:08:24.447 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-01-15T18:08:24.448 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:08:24.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:24.448 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:08:24.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:08:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.768+0000 7f3e57c00640 1 Processor -- start 2024-01-15T18:08:24.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.768+0000 7f3e57c00640 1 -- start start 2024-01-15T18:08:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.770+0000 7f3e57c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3e500ae5a0 0x7f3e5015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.770+0000 7f3e57c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e50070c40 con 0x7f3e500cf290 2024-01-15T18:08:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.770+0000 7f3e57c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5006c470 con 0x7f3e500b3ae0 2024-01-15T18:08:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.770+0000 7f3e57c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e50071450 con 0x7f3e500ae5a0 2024-01-15T18:08:24.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.770+0000 7f3e55c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3e500cf290 0x7f3e500b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59968/0 (socket says 172.21.15.26:59968) 2024-01-15T18:08:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.771+0000 7f3e55c00640 1 -- 172.21.15.26:0/3089472735 learned_addr learned my addr 172.21.15.26:0/3089472735 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.771+0000 7f3e55200640 1 --2- 172.21.15.26:0/3089472735 >> v2:172.21.15.124:3300/0 conn(0x7f3e500ae5a0 0x7f3e5015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.771+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243288827 0 0) 0x7f3e50070c40 con 0x7f3e500cf290 2024-01-15T18:08:24.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.771+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e30003520 con 0x7f3e500cf290 2024-01-15T18:08:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.771+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 620192173 0 0) 0x7f3e30003520 con 0x7f3e500cf290 2024-01-15T18:08:24.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.771+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 >> v2:172.21.15.124:3300/0 conn(0x7f3e500ae5a0 msgr2=0x7f3e5015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.771+0000 7f3e4f400640 1 --2- 172.21.15.26:0/3089472735 >> v2:172.21.15.124:3300/0 conn(0x7f3e500ae5a0 0x7f3e5015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.771+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 >> v1:172.21.15.124:6789/0 conn(0x7f3e500b3ae0 legacy=0x7f3e5006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.771+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5006c750 con 0x7f3e500cf290 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f3e480019f0 con 0x7f3e500cf290 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 >> v1:172.21.15.26:6789/0 conn(0x7f3e500cf290 legacy=0x7f3e500b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e4f400640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e30003970 0x7f3e30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e4f400640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e300047c0 0x7f3e30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e4f400640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e30005180 0x7f3e3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50071450 con 0x7f3e300047c0 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3000c030 con 0x7f3e30005180 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3000c1f0 con 0x7f3e30003970 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e4fe00640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e300047c0 0x7f3e30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.772+0000 7f3e55200640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e30003970 0x7f3e30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:24.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.773+0000 7f3e4fe00640 1 -- 172.21.15.26:0/3089472735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e30003970 msgr2=0x7f3e30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.773+0000 7f3e4fe00640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e30003970 0x7f3e30003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.773+0000 7f3e4fe00640 1 -- 172.21.15.26:0/3089472735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e30005180 msgr2=0x7f3e3000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.773+0000 7f3e4fe00640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e30005180 0x7f3e3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.773+0000 7f3e4fe00640 1 -- 172.21.15.26:0/3089472735 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e30003520 con 0x7f3e300047c0 2024-01-15T18:08:24.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.773+0000 7f3e55c00640 1 -- 172.21.15.26:0/3089472735 reap_dead start 2024-01-15T18:08:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.774+0000 7f3e4fe00640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e300047c0 0x7f3e30004c40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f3e38001750 tx=0x7f3e300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b273475712d6e862 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.774+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e380052d0 con 0x7f3e300047c0 2024-01-15T18:08:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.774+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3802e050 con 0x7f3e300047c0 2024-01-15T18:08:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.774+0000 7f3e4f400640 1 -- 172.21.15.26:0/3089472735 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e38003830 con 0x7f3e300047c0 2024-01-15T18:08:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.775+0000 7f3e57c00640 1 -- 172.21.15.26:0/3089472735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e300047c0 msgr2=0x7f3e30004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.775+0000 7f3e57c00640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e300047c0 0x7f3e30004c40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f3e38001750 tx=0x7f3e300057f0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.775+0000 7f3e57c00640 1 -- 172.21.15.26:0/3089472735 shutdown_connections 2024-01-15T18:08:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.775+0000 7f3e57c00640 1 --2- 172.21.15.26:0/3089472735 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e300047c0 0x7f3e30004c40 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.775+0000 7f3e57c00640 1 -- 172.21.15.26:0/3089472735 >> 172.21.15.26:0/3089472735 conn(0x7f3e5006a800 msgr2=0x7f3e5006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.775+0000 7f3e57c00640 1 -- 172.21.15.26:0/3089472735 shutdown_connections 2024-01-15T18:08:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.775+0000 7f3e57c00640 1 -- 172.21.15.26:0/3089472735 wait complete. 2024-01-15T18:08:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.777+0000 7f3e57c00640 1 Processor -- start 2024-01-15T18:08:24.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.777+0000 7f3e57c00640 1 -- start start 2024-01-15T18:08:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e57c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e300047c0 0x7f3e50154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e57c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e501f2bc0 0x7f3e50154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e57c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5015cf70 0x7f3e501553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e57c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e501558f0 con 0x7f3e5015cf70 2024-01-15T18:08:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e57c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5014f9f0 con 0x7f3e501f2bc0 2024-01-15T18:08:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e57c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50160380 con 0x7f3e300047c0 2024-01-15T18:08:24.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e55c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5015cf70 0x7f3e501553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e55c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5015cf70 0x7f3e501553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38496/0 (socket says 172.21.15.26:38496) 2024-01-15T18:08:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e55c00640 1 -- 172.21.15.26:0/3635608043 learned_addr learned my addr 172.21.15.26:0/3635608043 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e55200640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e300047c0 0x7f3e50154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.778+0000 7f3e4fe00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e501f2bc0 0x7f3e50154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e55c00640 1 -- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e300047c0 msgr2=0x7f3e50154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e55c00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e300047c0 0x7f3e50154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e55c00640 1 -- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e501f2bc0 msgr2=0x7f3e50154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e55c00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e501f2bc0 0x7f3e50154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e55c00640 1 -- 172.21.15.26:0/3635608043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50153b70 con 0x7f3e5015cf70 2024-01-15T18:08:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e55200640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e300047c0 0x7f3e50154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e4fe00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e501f2bc0 0x7f3e50154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e55c00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5015cf70 0x7f3e501553b0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cf670 tx=0x7f3e4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e4cc00640 1 -- 172.21.15.26:0/3635608043 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800a610 con 0x7f3e5015cf70 2024-01-15T18:08:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e4cc00640 1 -- 172.21.15.26:0/3635608043 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4800a770 con 0x7f3e5015cf70 2024-01-15T18:08:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e4cc00640 1 -- 172.21.15.26:0/3635608043 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800ab00 con 0x7f3e5015cf70 2024-01-15T18:08:24.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e50153e50 con 0x7f3e5015cf70 2024-01-15T18:08:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.779+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e50154410 con 0x7f3e5015cf70 2024-01-15T18:08:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.781+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e50154600 con 0x7f3e5015cf70 2024-01-15T18:08:24.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.782+0000 7f3e4cc00640 1 -- 172.21.15.26:0/3635608043 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f3e4800ac60 con 0x7f3e5015cf70 2024-01-15T18:08:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.783+0000 7f3e4cc00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e2806bfb0 0x7f3e2806e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:24.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.783+0000 7f3e4cc00640 1 -- 172.21.15.26:0/3635608043 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3e48087df0 con 0x7f3e5015cf70 2024-01-15T18:08:24.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.784+0000 7f3e55200640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e2806bfb0 0x7f3e2806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:24.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.785+0000 7f3e55200640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e2806bfb0 0x7f3e2806e470 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3e5015a620 tx=0x7f3e40008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:24.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:24.788+0000 7f3e4cc00640 1 -- 172.21.15.26:0/3635608043 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e480546b0 con 0x7f3e5015cf70 2024-01-15T18:08:25.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.023+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3e500675d0 con 0x7f3e5015cf70 2024-01-15T18:08:25.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.023+0000 7f3e4cc00640 1 -- 172.21.15.26:0/3635608043 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3e4801a030 con 0x7f3e5015cf70 2024-01-15T18:08:25.027 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:08:25.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e2806bfb0 msgr2=0x7f3e2806e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:25.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e2806bfb0 0x7f3e2806e470 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3e5015a620 tx=0x7f3e40008040 comp rx=0 tx=0).stop 2024-01-15T18:08:25.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5015cf70 msgr2=0x7f3e501553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:25.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5015cf70 0x7f3e501553b0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cf670 tx=0x7f3e4800ccb0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 shutdown_connections 2024-01-15T18:08:25.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3e2806bfb0 0x7f3e2806e470 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3e5015cf70 0x7f3e501553b0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3e501f2bc0 0x7f3e50154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 --2- 172.21.15.26:0/3635608043 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3e300047c0 0x7f3e50154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:25.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 >> 172.21.15.26:0/3635608043 conn(0x7f3e5006a800 msgr2=0x7f3e500acd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:25.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 shutdown_connections 2024-01-15T18:08:25.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:25.026+0000 7f3e57c00640 1 -- 172.21.15.26:0/3635608043 wait complete. 2024-01-15T18:08:25.049 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:08:25.049 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:08:25.049 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2024-01-15T18:08:25.050 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:08:25.050 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:08:25.050 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:08:25.050 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:08:25.050 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:08:25.050 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:08:25.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:25.050 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2024-01-15T18:08:25.108 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:08:25.108 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:08:25.109 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 41810sec preferred_lft 41810sec 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4c2d:94ff:fe47:f54c/64 scope link 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:25.110 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:08:25.111 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 26:4f:4f:5b:0b:31 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:08:25.111 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::244f:4fff:fe5b:b31/64 scope link 2024-01-15T18:08:25.111 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:25.111 INFO:teuthology.orchestra.run.smithi124.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:08:25.111 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:08:25.111 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::2070:16ff:fe92:4a4a/64 scope link 2024-01-15T18:08:25.112 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:08:25.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:25.112 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:08:25.200 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T18:08:25.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:25.205 DEBUG:teuthology.orchestra.run.smithi124:> route 2024-01-15T18:08:25.254 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2024-01-15T18:08:25.254 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:08:25.254 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:08:25.254 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:08:25.254 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:08:25.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:25.255 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T18:08:25.255 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T18:08:25.255 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:08:25.255 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:08:25.255 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:08:25.255 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T18:08:25.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:25.425 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T18:08:25.472 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:08:25.472 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:08:25.472 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:08:25.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:25.473 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2024-01-15T18:08:25.566 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:08:25.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:08:25.566 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:08:25.620 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:08:25.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:08:25.620 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:08:25.670 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:08:25.768 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:08:25.822 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:08:25.822 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:08:25.822 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:08:25.822 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:08:25.822 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:08:25.822 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:08:25.822 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:08:25.823 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:08:25.824 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:08:25.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:25.826 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:08:25.879 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:08:25.880 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:08:25.885 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:08:25.964 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:08:26.008 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:08:26.008 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:08:26.008 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:08:26.008 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:08:26.008 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:08:26.008 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:08:26.008 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:08:26.008 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:08:26.008 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:08:26.009 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:08:26.010 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:08:26.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:26.011 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:08:26.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:2024-01-15T18:08:26.067+0000 7f4ca0449480 -1 init, newargv = 0x563a7d25b550 newargc=15 2024-01-15T18:08:26.071 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[46583]: starting ceph client 2024-01-15T18:08:26.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[46583]: starting fuse 2024-01-15T18:08:26.097 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T18:08:26.097 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:08:26.097 DEBUG:teuthology.orchestra.run.smithi124:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:08:26.097 DEBUG:teuthology.orchestra.run.smithi124:> import glob 2024-01-15T18:08:26.097 DEBUG:teuthology.orchestra.run.smithi124:> import re 2024-01-15T18:08:26.097 DEBUG:teuthology.orchestra.run.smithi124:> import os 2024-01-15T18:08:26.097 DEBUG:teuthology.orchestra.run.smithi124:> import subprocess 2024-01-15T18:08:26.097 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:08:26.097 DEBUG:teuthology.orchestra.run.smithi124:> def _find_admin_socket(client_name): 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> files = glob.glob(asok_path) 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> # Given a non-glob path, it better be there 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> if "*" not in asok_path: 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> assert(len(files) == 1) 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> return files[0] 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> for f in files: 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> contents = proc_f.read() 2024-01-15T18:08:26.098 DEBUG:teuthology.orchestra.run.smithi124:> if mountpoint in contents: 2024-01-15T18:08:26.099 DEBUG:teuthology.orchestra.run.smithi124:> return f 2024-01-15T18:08:26.099 DEBUG:teuthology.orchestra.run.smithi124:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:08:26.099 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:08:26.099 DEBUG:teuthology.orchestra.run.smithi124:> print(_find_admin_socket("client.1")) 2024-01-15T18:08:26.099 DEBUG:teuthology.orchestra.run.smithi124:> ' 2024-01-15T18:08:26.297 INFO:teuthology.orchestra.run.smithi124.stdout:/var/run/ceph/ceph-client.1.46583.asok 2024-01-15T18:08:26.301 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T18:08:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:08:26.313 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.46583.asok 2024-01-15T18:08:26.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:26.314 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.46583.asok status 2024-01-15T18:08:26.548 INFO:teuthology.orchestra.run.smithi124.stdout:{ 2024-01-15T18:08:26.548 INFO:teuthology.orchestra.run.smithi124.stdout: "metadata": { 2024-01-15T18:08:26.548 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:08:26.548 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:08:26.548 INFO:teuthology.orchestra.run.smithi124.stdout: "entity_id": "1", 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "hostname": "smithi124", 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "pid": "46583", 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "root": "/" 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_count": 0, 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_pinned_count": 0, 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "id": 6796, 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "inst": { 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "name": { 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "client", 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "num": 6796 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:08:26.549 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 1105774647 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: } 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 1105774647 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "inst_str": "client.6796 192.168.0.1:0/1105774647", 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "addr_str": "192.168.0.1:0/1105774647", 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "inode_count": 1, 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "mds_epoch": 53, 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch": 87, 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:08:26.550 INFO:teuthology.orchestra.run.smithi124.stdout: "blocklisted": false, 2024-01-15T18:08:26.551 INFO:teuthology.orchestra.run.smithi124.stdout: "fs_name": "cephfs" 2024-01-15T18:08:26.551 INFO:teuthology.orchestra.run.smithi124.stdout:} 2024-01-15T18:08:26.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:26.563 DEBUG:teuthology.orchestra.run.smithi124:> stat --file-system '--printf=%T 2024-01-15T18:08:26.563 DEBUG:teuthology.orchestra.run.smithi124:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:08:26.617 INFO:teuthology.orchestra.run.smithi124.stdout:fuseblk 2024-01-15T18:08:26.617 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:08:26.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:26.617 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:08:26.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:08:26.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.941+0000 7f3c98e00640 1 Processor -- start 2024-01-15T18:08:26.945 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.941+0000 7f3c98e00640 1 -- start start 2024-01-15T18:08:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.942+0000 7f3c98e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c94143c40 0x7f3c94071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:26.946 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.943+0000 7f3c98e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c94070e50 con 0x7f3c940717d0 2024-01-15T18:08:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.943+0000 7f3c98e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c94144020 con 0x7f3c940ccc10 2024-01-15T18:08:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.943+0000 7f3c98e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940723c0 con 0x7f3c94143c40 2024-01-15T18:08:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.943+0000 7f3c92a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3c940717d0 0x7f3c940cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59970/0 (socket says 172.21.15.26:59970) 2024-01-15T18:08:26.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.943+0000 7f3c92a00640 1 -- 172.21.15.26:0/1763940392 learned_addr learned my addr 172.21.15.26:0/1763940392 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.943+0000 7f3c93400640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c94143c40 0x7f3c94071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115873297 0 0) 0x7f3c94070e50 con 0x7f3c940717d0 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c74003520 con 0x7f3c940717d0 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3245628939 0 0) 0x7f3c74003520 con 0x7f3c940717d0 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c94143c40 msgr2=0x7f3c94071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c94143c40 0x7f3c94071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 >> v1:172.21.15.124:6789/0 conn(0x7f3c940ccc10 legacy=0x7f3c9406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c94071e00 con 0x7f3c940717d0 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f3c7c001670 con 0x7f3c940717d0 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 >> v1:172.21.15.26:6789/0 conn(0x7f3c940717d0 legacy=0x7f3c940cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c94143c40 0x7f3c740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:26.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c74003ee0 0x7f3c740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c74004d00 0x7f3c74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c940723c0 con 0x7f3c74003ee0 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7400bac0 con 0x7f3c74004d00 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.944+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7400bc80 con 0x7f3c94143c40 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.945+0000 7f3c92a00640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c74003ee0 0x7f3c740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.945+0000 7f3c93400640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c94143c40 0x7f3c740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.945+0000 7f3c93e00640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c74004d00 0x7f3c74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.945+0000 7f3c92a00640 1 -- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c94143c40 msgr2=0x7f3c740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.945+0000 7f3c92a00640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c94143c40 0x7f3c740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.945+0000 7f3c92a00640 1 -- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c74004d00 msgr2=0x7f3c74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:26.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.945+0000 7f3c92a00640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c74004d00 0x7f3c74005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.945+0000 7f3c92a00640 1 -- 172.21.15.26:0/1763940392 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74003520 con 0x7f3c74003ee0 2024-01-15T18:08:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.945+0000 7f3c93400640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c94143c40 0x7f3c740039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:08:26.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.946+0000 7f3c92a00640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c74003ee0 0x7f3c740047c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00c190 tx=0x7f3c740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e552460163aa8a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.946+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c005ad0 con 0x7f3c74003ee0 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.946+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c005c30 con 0x7f3c74003ee0 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.946+0000 7f3c92000640 1 -- 172.21.15.26:0/1763940392 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c02c810 con 0x7f3c74003ee0 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.947+0000 7f3c98e00640 1 -- 172.21.15.26:0/1763940392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c74003ee0 msgr2=0x7f3c740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.947+0000 7f3c98e00640 1 --2- 172.21.15.26:0/1763940392 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c74003ee0 0x7f3c740047c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00c190 tx=0x7f3c740057f0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.947+0000 7f3c93e00640 1 -- 172.21.15.26:0/1763940392 reap_dead start 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.948+0000 7f3c98e00640 1 -- 172.21.15.26:0/1763940392 shutdown_connections 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.948+0000 7f3c98e00640 1 -- 172.21.15.26:0/1763940392 >> 172.21.15.26:0/1763940392 conn(0x7f3c9406a800 msgr2=0x7f3c9406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.948+0000 7f3c98e00640 1 -- 172.21.15.26:0/1763940392 shutdown_connections 2024-01-15T18:08:26.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.948+0000 7f3c98e00640 1 -- 172.21.15.26:0/1763940392 wait complete. 2024-01-15T18:08:26.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.949+0000 7f3c98e00640 1 Processor -- start 2024-01-15T18:08:26.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.950+0000 7f3c98e00640 1 -- start start 2024-01-15T18:08:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c98e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c74003ee0 0x7f3c940b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c98e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c94140df0 0x7f3c94159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c98e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c9415a420 0x7f3c9415a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:26.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c98e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9415b970 con 0x7f3c9415a420 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c98e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c94142800 con 0x7f3c94140df0 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c98e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c94141340 con 0x7f3c74003ee0 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c93e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c9415a420 0x7f3c9415a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c93e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c9415a420 0x7f3c9415a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38512/0 (socket says 172.21.15.26:38512) 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c93e00640 1 -- 172.21.15.26:0/4119592330 learned_addr learned my addr 172.21.15.26:0/4119592330 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c93400640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c74003ee0 0x7f3c940b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c92a00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c94140df0 0x7f3c94159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c93e00640 1 -- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c74003ee0 msgr2=0x7f3c940b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c93e00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c74003ee0 0x7f3c940b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c93e00640 1 -- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c94140df0 msgr2=0x7f3c94159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c93e00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c94140df0 0x7f3c94159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c93e00640 1 -- 172.21.15.26:0/4119592330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c94141620 con 0x7f3c9415a420 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.951+0000 7f3c92a00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c94140df0 0x7f3c94159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:26.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.952+0000 7f3c93e00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c9415a420 0x7f3c9415a8a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3c940cbcd0 tx=0x7f3c8c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.952+0000 7f3c8b400640 1 -- 172.21.15.26:0/4119592330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c003970 con 0x7f3c9415a420 2024-01-15T18:08:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.952+0000 7f3c8b400640 1 -- 172.21.15.26:0/4119592330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c003ad0 con 0x7f3c9415a420 2024-01-15T18:08:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.952+0000 7f3c8b400640 1 -- 172.21.15.26:0/4119592330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c003e80 con 0x7f3c9415a420 2024-01-15T18:08:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.952+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c94141900 con 0x7f3c9415a420 2024-01-15T18:08:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.952+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c940b3700 con 0x7f3c9415a420 2024-01-15T18:08:26.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.953+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c940b38c0 con 0x7f3c9415a420 2024-01-15T18:08:26.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.954+0000 7f3c8b400640 1 -- 172.21.15.26:0/4119592330 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f3c8c0050d0 con 0x7f3c9415a420 2024-01-15T18:08:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.955+0000 7f3c8b400640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6c06bfb0 0x7f3c6c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.955+0000 7f3c8b400640 1 -- 172.21.15.26:0/4119592330 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3c8c088980 con 0x7f3c9415a420 2024-01-15T18:08:26.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.956+0000 7f3c93400640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6c06bfb0 0x7f3c6c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:26.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.957+0000 7f3c93400640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6c06bfb0 0x7f3c6c06e470 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3c941f0700 tx=0x7f3c84008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:26.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:26.960+0000 7f3c8b400640 1 -- 172.21.15.26:0/4119592330 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c8c054de0 con 0x7f3c9415a420 2024-01-15T18:08:27.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.194+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3c94067450 con 0x7f3c9415a420 2024-01-15T18:08:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.278+0000 7f3c8b400640 1 -- 172.21.15.26:0/4119592330 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3c8c058400 con 0x7f3c9415a420 2024-01-15T18:08:27.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.281+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6c06bfb0 msgr2=0x7f3c6c06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.281+0000 7f3c98e00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6c06bfb0 0x7f3c6c06e470 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3c941f0700 tx=0x7f3c84008040 comp rx=0 tx=0).stop 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.281+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c9415a420 msgr2=0x7f3c9415a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.281+0000 7f3c98e00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c9415a420 0x7f3c9415a8a0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3c940cbcd0 tx=0x7f3c8c00eb90 comp rx=0 tx=0).stop 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.282+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 shutdown_connections 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.282+0000 7f3c98e00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6c06bfb0 0x7f3c6c06e470 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.282+0000 7f3c98e00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c9415a420 0x7f3c9415a8a0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.282+0000 7f3c98e00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c94140df0 0x7f3c94159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.282+0000 7f3c98e00640 1 --2- 172.21.15.26:0/4119592330 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c74003ee0 0x7f3c940b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.282+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 >> 172.21.15.26:0/4119592330 conn(0x7f3c9406a800 msgr2=0x7f3c94140010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.282+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 shutdown_connections 2024-01-15T18:08:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.282+0000 7f3c98e00640 1 -- 172.21.15.26:0/4119592330 wait complete. 2024-01-15T18:08:27.310 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:08:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.527+0000 7f6a77e00640 1 Processor -- start 2024-01-15T18:08:27.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.527+0000 7f6a77e00640 1 -- start start 2024-01-15T18:08:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.529+0000 7f6a77e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6a78162430 0x7f6a780b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.529+0000 7f6a77e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a78162810 0x7f6a78071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.529+0000 7f6a77e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a780702e0 con 0x7f6a780b3650 2024-01-15T18:08:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.529+0000 7f6a77e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a780705c0 con 0x7f6a78162810 2024-01-15T18:08:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.529+0000 7f6a77e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a780708a0 con 0x7f6a78162430 2024-01-15T18:08:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.529+0000 7f6a76000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a78162810 0x7f6a78071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.530+0000 7f6a76000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a78162810 0x7f6a78071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38514/0 (socket says 172.21.15.26:38514) 2024-01-15T18:08:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.530+0000 7f6a76000640 1 -- 172.21.15.26:0/2501227634 learned_addr learned my addr 172.21.15.26:0/2501227634 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.530+0000 7f6a76a00640 1 --2- 172.21.15.26:0/2501227634 >> v2:172.21.15.124:3300/0 conn(0x7f6a78162430 0x7f6a780b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.530+0000 7f6a76000640 1 -- 172.21.15.26:0/2501227634 >> v2:172.21.15.124:3300/0 conn(0x7f6a78162430 msgr2=0x7f6a780b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.530+0000 7f6a76000640 1 --2- 172.21.15.26:0/2501227634 >> v2:172.21.15.124:3300/0 conn(0x7f6a78162430 0x7f6a780b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.530+0000 7f6a76000640 1 -- 172.21.15.26:0/2501227634 >> v1:172.21.15.124:6789/0 conn(0x7f6a780b3650 legacy=0x7f6a781650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.530+0000 7f6a76000640 1 -- 172.21.15.26:0/2501227634 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a78070b80 con 0x7f6a78162810 2024-01-15T18:08:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.531+0000 7f6a76000640 1 --2- 172.21.15.26:0/2501227634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162810 0x7f6a78071240 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6a780b2c20 tx=0x7f6a6802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bb7cff4d293a5a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.532+0000 7f6a75600640 1 -- 172.21.15.26:0/2501227634 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68030860 con 0x7f6a78162810 2024-01-15T18:08:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.532+0000 7f6a75600640 1 -- 172.21.15.26:0/2501227634 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a680309c0 con 0x7f6a78162810 2024-01-15T18:08:27.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.532+0000 7f6a75600640 1 -- 172.21.15.26:0/2501227634 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68030d50 con 0x7f6a78162810 2024-01-15T18:08:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.532+0000 7f6a77e00640 1 -- 172.21.15.26:0/2501227634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162810 msgr2=0x7f6a78071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.532+0000 7f6a77e00640 1 --2- 172.21.15.26:0/2501227634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162810 0x7f6a78071240 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f6a780b2c20 tx=0x7f6a6802b9a0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.533+0000 7f6a77e00640 1 -- 172.21.15.26:0/2501227634 shutdown_connections 2024-01-15T18:08:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.533+0000 7f6a77e00640 1 --2- 172.21.15.26:0/2501227634 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162810 0x7f6a78071240 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.533+0000 7f6a77e00640 1 --2- 172.21.15.26:0/2501227634 >> v2:172.21.15.124:3300/0 conn(0x7f6a78162430 0x7f6a780b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.533+0000 7f6a77e00640 1 -- 172.21.15.26:0/2501227634 >> 172.21.15.26:0/2501227634 conn(0x7f6a7806a800 msgr2=0x7f6a7806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.533+0000 7f6a77e00640 1 -- 172.21.15.26:0/2501227634 shutdown_connections 2024-01-15T18:08:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.533+0000 7f6a77e00640 1 -- 172.21.15.26:0/2501227634 wait complete. 2024-01-15T18:08:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.534+0000 7f6a77e00640 1 Processor -- start 2024-01-15T18:08:27.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.535+0000 7f6a77e00640 1 -- start start 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a77e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a780b3650 0x7f6a78158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a77e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162430 0x7f6a78148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a77e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a781560d0 0x7f6a7815a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a77e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78158f50 con 0x7f6a78162430 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a77e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78149530 con 0x7f6a780b3650 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a77e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a781591d0 con 0x7f6a781560d0 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a76000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162430 0x7f6a78148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a76000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162430 0x7f6a78148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38524/0 (socket says 172.21.15.26:38524) 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a76000640 1 -- 172.21.15.26:0/4257286239 learned_addr learned my addr 172.21.15.26:0/4257286239 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a77400640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a781560d0 0x7f6a7815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a76000640 1 -- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a781560d0 msgr2=0x7f6a7815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a76a00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a780b3650 0x7f6a78158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a76000640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a781560d0 0x7f6a7815a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a76000640 1 -- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a780b3650 msgr2=0x7f6a78158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a76000640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a780b3650 0x7f6a78158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a76000640 1 -- 172.21.15.26:0/4257286239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680094d0 con 0x7f6a78162430 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.536+0000 7f6a77400640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a781560d0 0x7f6a7815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.537+0000 7f6a76000640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162430 0x7f6a78148ff0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6a68000c00 tx=0x7f6a68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.537+0000 7f6a6ea00640 1 -- 172.21.15.26:0/4257286239 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68039b50 con 0x7f6a78162430 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.537+0000 7f6a6ea00640 1 -- 172.21.15.26:0/4257286239 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a68039cb0 con 0x7f6a78162430 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.537+0000 7f6a6ea00640 1 -- 172.21.15.26:0/4257286239 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a68043650 con 0x7f6a78162430 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.537+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a781594b0 con 0x7f6a78162430 2024-01-15T18:08:27.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.537+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a78159a70 con 0x7f6a78162430 2024-01-15T18:08:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.538+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a78067450 con 0x7f6a78162430 2024-01-15T18:08:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.539+0000 7f6a6ea00640 1 -- 172.21.15.26:0/4257286239 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f6a6802f3b0 con 0x7f6a78162430 2024-01-15T18:08:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.540+0000 7f6a6ea00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a5006bfb0 0x7f6a5006e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.540+0000 7f6a6ea00640 1 -- 172.21.15.26:0/4257286239 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6a680aa940 con 0x7f6a78162430 2024-01-15T18:08:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.541+0000 7f6a76a00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a5006bfb0 0x7f6a5006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:27.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.542+0000 7f6a76a00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a5006bfb0 0x7f6a5006e470 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6a60006fd0 tx=0x7f6a60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.544+0000 7f6a6ea00640 1 -- 172.21.15.26:0/4257286239 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a68077170 con 0x7f6a78162430 2024-01-15T18:08:27.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.766+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a780b2ef0 con 0x7f6a5006bfb0 2024-01-15T18:08:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.769+0000 7f6a6ea00640 1 -- 172.21.15.26:0/4257286239 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6a780b2ef0 con 0x7f6a5006bfb0 2024-01-15T18:08:27.773 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:08:27.773 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:08:27.773 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:08:27.773 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:08:27.774 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:08:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a5006bfb0 msgr2=0x7f6a5006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a5006bfb0 0x7f6a5006e470 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6a60006fd0 tx=0x7f6a60008040 comp rx=0 tx=0).stop 2024-01-15T18:08:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162430 msgr2=0x7f6a78148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162430 0x7f6a78148ff0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f6a68000c00 tx=0x7f6a68004730 comp rx=0 tx=0).stop 2024-01-15T18:08:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 shutdown_connections 2024-01-15T18:08:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a781560d0 0x7f6a7815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a5006bfb0 0x7f6a5006e470 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a78162430 0x7f6a78148ff0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 --2- 172.21.15.26:0/4257286239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a780b3650 0x7f6a78158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:27.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 >> 172.21.15.26:0/4257286239 conn(0x7f6a7806a800 msgr2=0x7f6a78071990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:27.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 shutdown_connections 2024-01-15T18:08:27.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:27.772+0000 7f6a77e00640 1 -- 172.21.15.26:0/4257286239 wait complete. 2024-01-15T18:08:27.796 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:08:28.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.021+0000 7f25d7e00640 1 Processor -- start 2024-01-15T18:08:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.021+0000 7f25d7e00640 1 -- start start 2024-01-15T18:08:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.022+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d8160890 0x7f25d80d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.022+0000 7f25d7e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f25d8160c70 0x7f25d80d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.023+0000 7f25d7e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d80cf6a0 con 0x7f25d80c6be0 2024-01-15T18:08:28.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.023+0000 7f25d7e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d80d1430 con 0x7f25d8160c70 2024-01-15T18:08:28.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.023+0000 7f25d7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d80b4370 con 0x7f25d8160890 2024-01-15T18:08:28.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.023+0000 7f25d6a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d8160890 0x7f25d80d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.023+0000 7f25d6a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d8160890 0x7f25d80d3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:44234/0 (socket says 172.21.15.26:44234) 2024-01-15T18:08:28.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.023+0000 7f25d6a00640 1 -- 172.21.15.26:0/954116761 learned_addr learned my addr 172.21.15.26:0/954116761 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:28.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.023+0000 7f25d6000640 1 --2- 172.21.15.26:0/954116761 >> v2:172.21.15.124:3300/0 conn(0x7f25d8160c70 0x7f25d80d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.024+0000 7f25d5600640 1 -- 172.21.15.26:0/954116761 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528103386 0 0) 0x7f25d80cf6a0 con 0x7f25d80c6be0 2024-01-15T18:08:28.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.024+0000 7f25d5600640 1 -- 172.21.15.26:0/954116761 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b8003520 con 0x7f25d80c6be0 2024-01-15T18:08:28.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.024+0000 7f25d6a00640 1 -- 172.21.15.26:0/954116761 >> v2:172.21.15.124:3300/0 conn(0x7f25d8160c70 msgr2=0x7f25d80d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:28.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.024+0000 7f25d6a00640 1 --2- 172.21.15.26:0/954116761 >> v2:172.21.15.124:3300/0 conn(0x7f25d8160c70 0x7f25d80d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.024+0000 7f25d6a00640 1 -- 172.21.15.26:0/954116761 >> v1:172.21.15.124:6789/0 conn(0x7f25d80c6be0 legacy=0x7f25d8161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.024+0000 7f25d6a00640 1 -- 172.21.15.26:0/954116761 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d80ce7c0 con 0x7f25d8160890 2024-01-15T18:08:28.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.026+0000 7f25d6a00640 1 --2- 172.21.15.26:0/954116761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d8160890 0x7f25d80d3bb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009870 tx=0x7f25c802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84ad7e60a1d43d08 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:28.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.026+0000 7f25d5600640 1 -- 172.21.15.26:0/954116761 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c80317c0 con 0x7f25d8160890 2024-01-15T18:08:28.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.026+0000 7f25d5600640 1 -- 172.21.15.26:0/954116761 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c8031920 con 0x7f25d8160890 2024-01-15T18:08:28.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.026+0000 7f25d5600640 1 -- 172.21.15.26:0/954116761 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c8031cb0 con 0x7f25d8160890 2024-01-15T18:08:28.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.027+0000 7f25d7e00640 1 -- 172.21.15.26:0/954116761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d8160890 msgr2=0x7f25d80d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.027+0000 7f25d7e00640 1 --2- 172.21.15.26:0/954116761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d8160890 0x7f25d80d3bb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009870 tx=0x7f25c802b860 comp rx=0 tx=0).stop 2024-01-15T18:08:28.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.027+0000 7f25d7e00640 1 -- 172.21.15.26:0/954116761 shutdown_connections 2024-01-15T18:08:28.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.027+0000 7f25d7e00640 1 --2- 172.21.15.26:0/954116761 >> v2:172.21.15.124:3300/0 conn(0x7f25d8160c70 0x7f25d80d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.027+0000 7f25d7e00640 1 --2- 172.21.15.26:0/954116761 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d8160890 0x7f25d80d3bb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.027+0000 7f25d7e00640 1 -- 172.21.15.26:0/954116761 >> 172.21.15.26:0/954116761 conn(0x7f25d80d3090 msgr2=0x7f25d80d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:28.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.027+0000 7f25d7e00640 1 -- 172.21.15.26:0/954116761 shutdown_connections 2024-01-15T18:08:28.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.027+0000 7f25d7e00640 1 -- 172.21.15.26:0/954116761 wait complete. 2024-01-15T18:08:28.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.029+0000 7f25d7e00640 1 Processor -- start 2024-01-15T18:08:28.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.029+0000 7f25d7e00640 1 -- start start 2024-01-15T18:08:28.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.030+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d80c6be0 0x7f25d81538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.030+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25d8160890 0x7f25d8152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.030+0000 7f25d7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d8160c70 0x7f25d80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.030+0000 7f25d7e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d81f65f0 con 0x7f25d8160c70 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.030+0000 7f25d7e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d8159770 con 0x7f25d8160890 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.030+0000 7f25d7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d8153f50 con 0x7f25d80c6be0 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.030+0000 7f25d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d8160c70 0x7f25d80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d8160c70 0x7f25d80cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38534/0 (socket says 172.21.15.26:38534) 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d7400640 1 -- 172.21.15.26:0/2878209096 learned_addr learned my addr 172.21.15.26:0/2878209096 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d6a00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d80c6be0 0x7f25d81538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d6000640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25d8160890 0x7f25d8152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d7400640 1 -- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d80c6be0 msgr2=0x7f25d81538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d7400640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d80c6be0 0x7f25d81538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d7400640 1 -- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25d8160890 msgr2=0x7f25d8152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d7400640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25d8160890 0x7f25d8152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d7400640 1 -- 172.21.15.26:0/2878209096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c80094d0 con 0x7f25d8160c70 2024-01-15T18:08:28.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.031+0000 7f25d7400640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d8160c70 0x7f25d80cb630 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f25d0001eb0 tx=0x7f25d000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:28.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.032+0000 7f25cea00640 1 -- 172.21.15.26:0/2878209096 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d00052d0 con 0x7f25d8160c70 2024-01-15T18:08:28.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.032+0000 7f25cea00640 1 -- 172.21.15.26:0/2878209096 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d0005430 con 0x7f25d8160c70 2024-01-15T18:08:28.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.032+0000 7f25cea00640 1 -- 172.21.15.26:0/2878209096 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d00037f0 con 0x7f25d8160c70 2024-01-15T18:08:28.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.032+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d81f6840 con 0x7f25d8160c70 2024-01-15T18:08:28.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.032+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d81f6e00 con 0x7f25d8160c70 2024-01-15T18:08:28.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.033+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d8067450 con 0x7f25d8160c70 2024-01-15T18:08:28.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.034+0000 7f25cea00640 1 -- 172.21.15.26:0/2878209096 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f25d0003950 con 0x7f25d8160c70 2024-01-15T18:08:28.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.035+0000 7f25cea00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f25b006bfb0 0x7f25b006e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.035+0000 7f25cea00640 1 -- 172.21.15.26:0/2878209096 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f25d0087a00 con 0x7f25d8160c70 2024-01-15T18:08:28.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.036+0000 7f25d6a00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f25b006bfb0 0x7f25b006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.037+0000 7f25d6a00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f25b006bfb0 0x7f25b006e470 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f25c8036000 tx=0x7f25c803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:28.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.039+0000 7f25cea00640 1 -- 172.21.15.26:0/2878209096 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25d0054370 con 0x7f25d8160c70 2024-01-15T18:08:28.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.236+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f25d808c7e0 con 0x7f25d8160c70 2024-01-15T18:08:28.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.236+0000 7f25cea00640 1 -- 172.21.15.26:0/2878209096 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f25d0057990 con 0x7f25d8160c70 2024-01-15T18:08:28.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.239+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f25b006bfb0 msgr2=0x7f25b006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.239+0000 7f25d7e00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f25b006bfb0 0x7f25b006e470 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f25c8036000 tx=0x7f25c803d040 comp rx=0 tx=0).stop 2024-01-15T18:08:28.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.239+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d8160c70 msgr2=0x7f25d80cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.239+0000 7f25d7e00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d8160c70 0x7f25d80cb630 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f25d0001eb0 tx=0x7f25d000c820 comp rx=0 tx=0).stop 2024-01-15T18:08:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.240+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 shutdown_connections 2024-01-15T18:08:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.240+0000 7f25d7e00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f25b006bfb0 0x7f25b006e470 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.240+0000 7f25d7e00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25d8160c70 0x7f25d80cb630 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.240+0000 7f25d7e00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f25d8160890 0x7f25d8152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.240+0000 7f25d7e00640 1 --2- 172.21.15.26:0/2878209096 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25d80c6be0 0x7f25d81538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.240+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 >> 172.21.15.26:0/2878209096 conn(0x7f25d80d3090 msgr2=0x7f25d80b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.240+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 shutdown_connections 2024-01-15T18:08:28.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.240+0000 7f25d7e00640 1 -- 172.21.15.26:0/2878209096 wait complete. 2024-01-15T18:08:28.263 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:08:28.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.478+0000 7f69f3e00640 1 Processor -- start 2024-01-15T18:08:28.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.478+0000 7f69f3e00640 1 -- start start 2024-01-15T18:08:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.479+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40d0100 0x7f69f40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.479+0000 7f69f3e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f69f40d0f40 0x7f69f40d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.479+0000 7f69f3e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f40d0a10 con 0x7f69f4160920 2024-01-15T18:08:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.479+0000 7f69f3e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f40cf080 con 0x7f69f40d0f40 2024-01-15T18:08:28.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.479+0000 7f69f3e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40c6be0 con 0x7f69f40d0100 2024-01-15T18:08:28.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.480+0000 7f69f2a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f69f4160920 0x7f69f40cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59980/0 (socket says 172.21.15.26:59980) 2024-01-15T18:08:28.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.480+0000 7f69f2a00640 1 -- 172.21.15.26:0/4175185496 learned_addr learned my addr 172.21.15.26:0/4175185496 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:08:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.480+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280226168 0 0) 0x7f69f40d0a10 con 0x7f69f4160920 2024-01-15T18:08:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.480+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d4003520 con 0x7f69f4160920 2024-01-15T18:08:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.480+0000 7f69f2000640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40d0100 0x7f69f40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.480+0000 7f69f3400640 1 --2- 172.21.15.26:0/4175185496 >> v2:172.21.15.124:3300/0 conn(0x7f69f40d0f40 0x7f69f40d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2069118657 0 0) 0x7f69d4003520 con 0x7f69f4160920 2024-01-15T18:08:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40d0100 msgr2=0x7f69f40b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40d0100 0x7f69f40b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 >> v2:172.21.15.124:3300/0 conn(0x7f69f40d0f40 msgr2=0x7f69f40d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:28.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 --2- 172.21.15.26:0/4175185496 >> v2:172.21.15.124:3300/0 conn(0x7f69f40d0f40 0x7f69f40d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f40ce7e0 con 0x7f69f4160920 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f69e4001670 con 0x7f69f4160920 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 >> v1:172.21.15.26:6789/0 conn(0x7f69f4160920 legacy=0x7f69f40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69d4003970 0x7f69d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40d0100 0x7f69d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40cf080 con 0x7f69d4004d00 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40c6be0 con 0x7f69d4003970 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d400ba30 con 0x7f69f40d0100 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f3400640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f2a00640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69d4003970 0x7f69d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.481+0000 7f69f2000640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40d0100 0x7f69d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.482+0000 7f69f3400640 1 -- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40d0100 msgr2=0x7f69d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:28.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.482+0000 7f69f3400640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40d0100 0x7f69d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.482+0000 7f69f3400640 1 -- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69d4003970 msgr2=0x7f69d4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.482+0000 7f69f3400640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69d4003970 0x7f69d4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.482+0000 7f69f3400640 1 -- 172.21.15.26:0/4175185496 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4003520 con 0x7f69d4004d00 2024-01-15T18:08:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.483+0000 7f69f3400640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69d4005180 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f69ec007390 tx=0x7f69d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ccb269d077c420a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.483+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec02ccf0 con 0x7f69d4004d00 2024-01-15T18:08:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.483+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ec02ce50 con 0x7f69d4004d00 2024-01-15T18:08:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.483+0000 7f69f1600640 1 -- 172.21.15.26:0/4175185496 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec034660 con 0x7f69d4004d00 2024-01-15T18:08:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.484+0000 7f69f3e00640 1 -- 172.21.15.26:0/4175185496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 msgr2=0x7f69d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.484+0000 7f69f3e00640 1 --2- 172.21.15.26:0/4175185496 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69d4005180 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f69ec007390 tx=0x7f69d40057f0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.484+0000 7f69f3400640 1 -- 172.21.15.26:0/4175185496 reap_dead start 2024-01-15T18:08:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.484+0000 7f69f3e00640 1 -- 172.21.15.26:0/4175185496 shutdown_connections 2024-01-15T18:08:28.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.484+0000 7f69f3e00640 1 -- 172.21.15.26:0/4175185496 >> 172.21.15.26:0/4175185496 conn(0x7f69f40d3090 msgr2=0x7f69f40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:28.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.484+0000 7f69f3e00640 1 -- 172.21.15.26:0/4175185496 shutdown_connections 2024-01-15T18:08:28.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.484+0000 7f69f3e00640 1 -- 172.21.15.26:0/4175185496 wait complete. 2024-01-15T18:08:28.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.486+0000 7f69f3e00640 1 Processor -- start 2024-01-15T18:08:28.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.486+0000 7f69f3e00640 1 -- start start 2024-01-15T18:08:28.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.487+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69f41f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.487+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69f41f2e20 0x7f69f41f0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.487+0000 7f69f3e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40cb1b0 0x7f69f40cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.487+0000 7f69f3e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f41f73f0 con 0x7f69d4004d00 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.487+0000 7f69f3e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f40b7810 con 0x7f69f41f2e20 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.487+0000 7f69f3e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f41f3e40 con 0x7f69f40cb1b0 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.487+0000 7f69f2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69f41f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.487+0000 7f69f2a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69f41f6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38554/0 (socket says 172.21.15.26:38554) 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.487+0000 7f69f2a00640 1 -- 172.21.15.26:0/1325966075 learned_addr learned my addr 172.21.15.26:0/1325966075 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.488+0000 7f69f2000640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69f41f2e20 0x7f69f41f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.488+0000 7f69f3400640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40cb1b0 0x7f69f40cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.488+0000 7f69f2a00640 1 -- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40cb1b0 msgr2=0x7f69f40cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.488+0000 7f69f2a00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40cb1b0 0x7f69f40cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.488+0000 7f69f2a00640 1 -- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69f41f2e20 msgr2=0x7f69f41f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.488+0000 7f69f2a00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69f41f2e20 0x7f69f41f0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.488+0000 7f69f2a00640 1 -- 172.21.15.26:0/1325966075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f40ccfa0 con 0x7f69d4004d00 2024-01-15T18:08:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.488+0000 7f69f3400640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40cb1b0 0x7f69f40cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:08:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.488+0000 7f69f2a00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69f41f6eb0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f69e4001eb0 tx=0x7f69e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.489+0000 7f69eaa00640 1 -- 172.21.15.26:0/1325966075 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400c8d0 con 0x7f69d4004d00 2024-01-15T18:08:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.489+0000 7f69eaa00640 1 -- 172.21.15.26:0/1325966075 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e400ca50 con 0x7f69d4004d00 2024-01-15T18:08:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.489+0000 7f69eaa00640 1 -- 172.21.15.26:0/1325966075 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400ce00 con 0x7f69d4004d00 2024-01-15T18:08:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.489+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f41f0880 con 0x7f69d4004d00 2024-01-15T18:08:28.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.489+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f41f3880 con 0x7f69d4004d00 2024-01-15T18:08:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.490+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f4067450 con 0x7f69d4004d00 2024-01-15T18:08:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.491+0000 7f69eaa00640 1 -- 172.21.15.26:0/1325966075 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f69e401d460 con 0x7f69d4004d00 2024-01-15T18:08:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.492+0000 7f69eaa00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f69cc06bfb0 0x7f69cc06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.492+0000 7f69eaa00640 1 -- 172.21.15.26:0/1325966075 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f69e4088900 con 0x7f69d4004d00 2024-01-15T18:08:28.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.493+0000 7f69f2000640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f69cc06bfb0 0x7f69cc06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:28.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.494+0000 7f69f2000640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f69cc06bfb0 0x7f69cc06e470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f69f415a360 tx=0x7f69dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:28.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.496+0000 7f69eaa00640 1 -- 172.21.15.26:0/1325966075 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69e4054f30 con 0x7f69d4004d00 2024-01-15T18:08:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.721+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f69f40ceda0 con 0x7f69cc06bfb0 2024-01-15T18:08:28.737 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:08:28.733+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:28.737 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:08:28.733+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:28.737 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:08:28.733+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:28.737 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:08:28.733+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:28.737 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:08:28.733+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:08:28.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.803+0000 7f69eaa00640 1 -- 172.21.15.26:0/1325966075 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f69dc00b220 con 0x7f69cc06bfb0 2024-01-15T18:08:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f69cc06bfb0 msgr2=0x7f69cc06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f69cc06bfb0 0x7f69cc06e470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f69f415a360 tx=0x7f69dc008040 comp rx=0 tx=0).stop 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 msgr2=0x7f69f41f6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69f41f6eb0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f69e4001eb0 tx=0x7f69e400ec80 comp rx=0 tx=0).stop 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 shutdown_connections 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f69f40cb1b0 0x7f69f40cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f69cc06bfb0 0x7f69cc06e470 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69f41f2e20 0x7f69f41f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 --2- 172.21.15.26:0/1325966075 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69d4004d00 0x7f69f41f6eb0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.806+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 >> 172.21.15.26:0/1325966075 conn(0x7f69f40d3090 msgr2=0x7f69f40cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.807+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 shutdown_connections 2024-01-15T18:08:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:28.807+0000 7f69f3e00640 1 -- 172.21.15.26:0/1325966075 wait complete. 2024-01-15T18:08:28.831 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:08:29.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.058+0000 7f6347e00640 1 Processor -- start 2024-01-15T18:08:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.059+0000 7f6347e00640 1 -- start start 2024-01-15T18:08:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.060+0000 7f6347e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f634815d1d0 0x7f63480d1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.060+0000 7f6347e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63480b1f70 0x7f63480d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.060+0000 7f6347e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f634815d5b0 0x7f63480cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.060+0000 7f6347e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63480d0090 con 0x7f63480b1f70 2024-01-15T18:08:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.060+0000 7f6347e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63480d02e0 con 0x7f634815d1d0 2024-01-15T18:08:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.060+0000 7f6347e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63480d05c0 con 0x7f634815d5b0 2024-01-15T18:08:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.060+0000 7f633de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63480b1f70 0x7f63480d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.060+0000 7f633de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f63480b1f70 0x7f63480d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38560/0 (socket says 172.21.15.26:38560) 2024-01-15T18:08:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.060+0000 7f633de00640 1 -- 172.21.15.26:0/66482912 learned_addr learned my addr 172.21.15.26:0/66482912 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.061+0000 7f6346a00640 1 --2- 172.21.15.26:0/66482912 >> v2:172.21.15.124:3300/0 conn(0x7f634815d1d0 0x7f63480d1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.061+0000 7f6347400640 1 --2- 172.21.15.26:0/66482912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f634815d5b0 0x7f63480cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.061+0000 7f633de00640 1 -- 172.21.15.26:0/66482912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f634815d5b0 msgr2=0x7f63480cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.061+0000 7f633de00640 1 --2- 172.21.15.26:0/66482912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f634815d5b0 0x7f63480cdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.061+0000 7f633de00640 1 -- 172.21.15.26:0/66482912 >> v2:172.21.15.124:3300/0 conn(0x7f634815d1d0 msgr2=0x7f63480d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:08:29.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.061+0000 7f633de00640 1 --2- 172.21.15.26:0/66482912 >> v2:172.21.15.124:3300/0 conn(0x7f634815d1d0 0x7f63480d1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.061+0000 7f633de00640 1 -- 172.21.15.26:0/66482912 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63480d08a0 con 0x7f63480b1f70 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.062+0000 7f633de00640 1 --2- 172.21.15.26:0/66482912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63480b1f70 0x7f63480d1b70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f634808b460 tx=0x7f633002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3793122254c389 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.062+0000 7f6346000640 1 -- 172.21.15.26:0/66482912 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330031870 con 0x7f63480b1f70 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.062+0000 7f6346000640 1 -- 172.21.15.26:0/66482912 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63300319d0 con 0x7f63480b1f70 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.062+0000 7f6346000640 1 -- 172.21.15.26:0/66482912 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330031d60 con 0x7f63480b1f70 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.063+0000 7f6347e00640 1 -- 172.21.15.26:0/66482912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63480b1f70 msgr2=0x7f63480d1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.063+0000 7f6347e00640 1 --2- 172.21.15.26:0/66482912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63480b1f70 0x7f63480d1b70 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f634808b460 tx=0x7f633002b910 comp rx=0 tx=0).stop 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.063+0000 7f6347e00640 1 -- 172.21.15.26:0/66482912 shutdown_connections 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.063+0000 7f6347e00640 1 --2- 172.21.15.26:0/66482912 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f634815d5b0 0x7f63480cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.063+0000 7f6347e00640 1 --2- 172.21.15.26:0/66482912 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63480b1f70 0x7f63480d1b70 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.063+0000 7f6347e00640 1 --2- 172.21.15.26:0/66482912 >> v2:172.21.15.124:3300/0 conn(0x7f634815d1d0 0x7f63480d1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.063+0000 7f6347e00640 1 -- 172.21.15.26:0/66482912 >> 172.21.15.26:0/66482912 conn(0x7f63480d3050 msgr2=0x7f63480d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.063+0000 7f6347e00640 1 -- 172.21.15.26:0/66482912 shutdown_connections 2024-01-15T18:08:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.063+0000 7f6347e00640 1 -- 172.21.15.26:0/66482912 wait complete. 2024-01-15T18:08:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.065+0000 7f6347e00640 1 Processor -- start 2024-01-15T18:08:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.065+0000 7f6347e00640 1 -- start start 2024-01-15T18:08:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.066+0000 7f6347e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63480b1f70 0x7f634815ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.066+0000 7f6347e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f634815d1d0 0x7f63480cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.066+0000 7f6347e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f634815d5b0 0x7f634815fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.066+0000 7f6347e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63480b6680 con 0x7f634815d1d0 2024-01-15T18:08:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.066+0000 7f6347e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63480b7970 con 0x7f634815d5b0 2024-01-15T18:08:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.066+0000 7f6347e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63480b68d0 con 0x7f63480b1f70 2024-01-15T18:08:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.066+0000 7f633de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f634815d1d0 0x7f63480cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.066+0000 7f633de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f634815d1d0 0x7f63480cbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38568/0 (socket says 172.21.15.26:38568) 2024-01-15T18:08:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.066+0000 7f633de00640 1 -- 172.21.15.26:0/3454425257 learned_addr learned my addr 172.21.15.26:0/3454425257 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:08:29.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.067+0000 7f6347400640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f634815d5b0 0x7f634815fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.067+0000 7f6346a00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63480b1f70 0x7f634815ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.067+0000 7f633de00640 1 -- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63480b1f70 msgr2=0x7f634815ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.067+0000 7f633de00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63480b1f70 0x7f634815ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.067+0000 7f633de00640 1 -- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f634815d5b0 msgr2=0x7f634815fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.067+0000 7f633de00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f634815d5b0 0x7f634815fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.067+0000 7f633de00640 1 -- 172.21.15.26:0/3454425257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63300094d0 con 0x7f634815d1d0 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.067+0000 7f633de00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f634815d1d0 0x7f63480cbb00 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f633002b950 tx=0x7f63300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.068+0000 7f633f400640 1 -- 172.21.15.26:0/3454425257 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330031860 con 0x7f634815d1d0 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.068+0000 7f633f400640 1 -- 172.21.15.26:0/3454425257 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63300443f0 con 0x7f634815d1d0 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.068+0000 7f633f400640 1 -- 172.21.15.26:0/3454425257 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300447a0 con 0x7f634815d1d0 2024-01-15T18:08:29.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.068+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63480b6bb0 con 0x7f634815d1d0 2024-01-15T18:08:29.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.068+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63480b7170 con 0x7f634815d1d0 2024-01-15T18:08:29.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.069+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6348067450 con 0x7f634815d1d0 2024-01-15T18:08:29.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.070+0000 7f633f400640 1 -- 172.21.15.26:0/3454425257 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 30) v1 ==== 90712+0+0 (secure 0 0 0) 0x7f633004d450 con 0x7f634815d1d0 2024-01-15T18:08:29.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.071+0000 7f633f400640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f632006bfb0 0x7f632006e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:08:29.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.072+0000 7f633f400640 1 -- 172.21.15.26:0/3454425257 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f633002d040 con 0x7f634815d1d0 2024-01-15T18:08:29.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.072+0000 7f6346a00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f632006bfb0 0x7f632006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:08:29.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.073+0000 7f6346a00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f632006bfb0 0x7f632006e470 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f633800a990 tx=0x7f6338008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:08:29.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.076+0000 7f633f400640 1 -- 172.21.15.26:0/3454425257 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6330077630 con 0x7f634815d1d0 2024-01-15T18:08:29.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.307+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f63480d1030 con 0x7f632006bfb0 2024-01-15T18:08:29.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.318+0000 7f633f400640 1 -- 172.21.15.26:0/3454425257 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f633800c050 con 0x7f632006bfb0 2024-01-15T18:08:29.322 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df 2024-01-15T18:08:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f632006bfb0 msgr2=0x7f632006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f632006bfb0 0x7f632006e470 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f633800a990 tx=0x7f6338008040 comp rx=0 tx=0).stop 2024-01-15T18:08:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f634815d1d0 msgr2=0x7f63480cbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:08:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f634815d1d0 0x7f63480cbb00 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f633002b950 tx=0x7f63300046c0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 shutdown_connections 2024-01-15T18:08:29.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f632006bfb0 0x7f632006e470 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f634815d5b0 0x7f634815fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f634815d1d0 0x7f63480cbb00 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 --2- 172.21.15.26:0/3454425257 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63480b1f70 0x7f634815ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:08:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 >> 172.21.15.26:0/3454425257 conn(0x7f63480d3050 msgr2=0x7f634806af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:08:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 shutdown_connections 2024-01-15T18:08:29.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:08:29.321+0000 7f6347e00640 1 -- 172.21.15.26:0/3454425257 wait complete. 2024-01-15T18:08:29.344 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df 2024-01-15T18:08:29.344 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:08:29.344 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:08:29.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:29.396 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:08:30.299 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:08:30.308 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:08:47.025 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:08:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:08:47.026 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 17.495 s, 61.4 MB/s 2024-01-15T18:08:47.027 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:08:47.028 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:08:47.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:08:47.085 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:09:00.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:09:00.551 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:09:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:09:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:09:01.750 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.5985 s, 73.6 MB/s 2024-01-15T18:09:01.752 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:09:01.752 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:09:01.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:01.809 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:09:16.381 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:09:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:09:16.382 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.5053 s, 74.0 MB/s 2024-01-15T18:09:16.383 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 2 2024-01-15T18:09:16.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.605+0000 7fcc0b600640 1 Processor -- start 2024-01-15T18:09:16.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.605+0000 7fcc0b600640 1 -- start start 2024-01-15T18:09:16.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.607+0000 7fcc0b600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcc040cf9a0 0x7fcc0415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.607+0000 7fcc0b600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc0415e1e0 0x7fcc040702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.607+0000 7fcc0b600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc040b1ee0 con 0x7fcc04143ae0 2024-01-15T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.607+0000 7fcc0b600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc040b2780 con 0x7fcc040cf9a0 2024-01-15T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.607+0000 7fcc0b600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0415e990 con 0x7fcc0415e1e0 2024-01-15T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.607+0000 7fcc08c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcc04143ae0 0x7fcc040cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52684/0 (socket says 172.21.15.26:52684) 2024-01-15T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.607+0000 7fcc08c00640 1 -- 172.21.15.26:0/2565797932 learned_addr learned my addr 172.21.15.26:0/2565797932 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:16.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.607+0000 7fcc03e00640 1 --2- 172.21.15.26:0/2565797932 >> v2:172.21.15.26:3300/0 conn(0x7fcc040cf9a0 0x7fcc0415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.608+0000 7fcc09600640 1 --2- 172.21.15.26:0/2565797932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc0415e1e0 0x7fcc040702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.608+0000 7fcc03400640 1 -- 172.21.15.26:0/2565797932 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575757166 0 0) 0x7fcc040b1ee0 con 0x7fcc04143ae0 2024-01-15T18:09:16.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.608+0000 7fcc03400640 1 -- 172.21.15.26:0/2565797932 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe4003520 con 0x7fcc04143ae0 2024-01-15T18:09:16.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.608+0000 7fcc03e00640 1 -- 172.21.15.26:0/2565797932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc0415e1e0 msgr2=0x7fcc040702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:16.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.608+0000 7fcc03e00640 1 --2- 172.21.15.26:0/2565797932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc0415e1e0 0x7fcc040702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.608+0000 7fcc03e00640 1 -- 172.21.15.26:0/2565797932 >> v1:172.21.15.26:6789/0 conn(0x7fcc04143ae0 legacy=0x7fcc040cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.608+0000 7fcc03e00640 1 -- 172.21.15.26:0/2565797932 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040b2130 con 0x7fcc040cf9a0 2024-01-15T18:09:16.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.609+0000 7fcc03e00640 1 --2- 172.21.15.26:0/2565797932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc040cf9a0 0x7fcc0415dca0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0097a0 tx=0x7fcbec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861377e4b7fcea9f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:16.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.609+0000 7fcc03400640 1 -- 172.21.15.26:0/2565797932 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec031830 con 0x7fcc040cf9a0 2024-01-15T18:09:16.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.609+0000 7fcc03400640 1 -- 172.21.15.26:0/2565797932 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec031990 con 0x7fcc040cf9a0 2024-01-15T18:09:16.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.609+0000 7fcc03400640 1 -- 172.21.15.26:0/2565797932 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec031d20 con 0x7fcc040cf9a0 2024-01-15T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.610+0000 7fcc0b600640 1 -- 172.21.15.26:0/2565797932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc040cf9a0 msgr2=0x7fcc0415dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.610+0000 7fcc0b600640 1 --2- 172.21.15.26:0/2565797932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc040cf9a0 0x7fcc0415dca0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0097a0 tx=0x7fcbec02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.610+0000 7fcc0b600640 1 -- 172.21.15.26:0/2565797932 shutdown_connections 2024-01-15T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.610+0000 7fcc0b600640 1 --2- 172.21.15.26:0/2565797932 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc0415e1e0 0x7fcc040702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.610+0000 7fcc0b600640 1 --2- 172.21.15.26:0/2565797932 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc040cf9a0 0x7fcc0415dca0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.610+0000 7fcc0b600640 1 -- 172.21.15.26:0/2565797932 >> 172.21.15.26:0/2565797932 conn(0x7fcc0406a800 msgr2=0x7fcc0406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:16.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.610+0000 7fcc0b600640 1 -- 172.21.15.26:0/2565797932 shutdown_connections 2024-01-15T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.610+0000 7fcc0b600640 1 -- 172.21.15.26:0/2565797932 wait complete. 2024-01-15T18:09:16.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.612+0000 7fcc0b600640 1 Processor -- start 2024-01-15T18:09:16.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.612+0000 7fcc0b600640 1 -- start start 2024-01-15T18:09:16.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.613+0000 7fcc0b600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc040cf9a0 0x7fcc04159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.613+0000 7fcc0b600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04143ae0 0x7fcc04141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.613+0000 7fcc0b600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc0415e1e0 0x7fcc040adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.613+0000 7fcc0b600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc04141190 con 0x7fcc04143ae0 2024-01-15T18:09:16.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.613+0000 7fcc0b600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc041402c0 con 0x7fcc040cf9a0 2024-01-15T18:09:16.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.613+0000 7fcc0b600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc04140580 con 0x7fcc0415e1e0 2024-01-15T18:09:16.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.613+0000 7fcc03e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04143ae0 0x7fcc04141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.614+0000 7fcc03e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04143ae0 0x7fcc04141620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50254/0 (socket says 172.21.15.26:50254) 2024-01-15T18:09:16.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.614+0000 7fcc03e00640 1 -- 172.21.15.26:0/2424631891 learned_addr learned my addr 172.21.15.26:0/2424631891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.614+0000 7fcc03e00640 1 -- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc0415e1e0 msgr2=0x7fcc040adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.614+0000 7fcc03e00640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc0415e1e0 0x7fcc040adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.614+0000 7fcc03e00640 1 -- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc040cf9a0 msgr2=0x7fcc04159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.614+0000 7fcc03e00640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc040cf9a0 0x7fcc04159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.614+0000 7fcc03e00640 1 -- 172.21.15.26:0/2424631891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec009400 con 0x7fcc04143ae0 2024-01-15T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.614+0000 7fcc03e00640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04143ae0 0x7fcc04141620 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcbec02bee0 tx=0x7fcbec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.615+0000 7fcc00c00640 1 -- 172.21.15.26:0/2424631891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec03a6f0 con 0x7fcc04143ae0 2024-01-15T18:09:16.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.615+0000 7fcc00c00640 1 -- 172.21.15.26:0/2424631891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec03a870 con 0x7fcc04143ae0 2024-01-15T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.615+0000 7fcc00c00640 1 -- 172.21.15.26:0/2424631891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec03ac20 con 0x7fcc04143ae0 2024-01-15T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.615+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc04140860 con 0x7fcc04143ae0 2024-01-15T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.615+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc04140e20 con 0x7fcc04143ae0 2024-01-15T18:09:16.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.616+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc04140fe0 con 0x7fcc04143ae0 2024-01-15T18:09:16.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.617+0000 7fcc00c00640 1 -- 172.21.15.26:0/2424631891 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 31) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fcbec030380 con 0x7fcc04143ae0 2024-01-15T18:09:16.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.618+0000 7fcc00c00640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbdc06c040 0x7fcbdc06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:16.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.618+0000 7fcc00c00640 1 -- 172.21.15.26:0/2424631891 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fcbec0aa220 con 0x7fcc04143ae0 2024-01-15T18:09:16.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.619+0000 7fcc08c00640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbdc06c040 0x7fcbdc06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:16.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.620+0000 7fcc08c00640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbdc06c040 0x7fcbdc06e500 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4001610 tx=0x7fcbf4007930 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:16.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.623+0000 7fcc00c00640 1 -- 172.21.15.26:0/2424631891 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcbec0dfe90 con 0x7fcc04143ae0 2024-01-15T18:09:16.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.818+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7fcc041412d0 con 0x7fcc04143ae0 2024-01-15T18:09:16.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.824+0000 7fcc00c00640 1 -- 172.21.15.26:0/2424631891 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v31)=0 v31) v1 ==== 135+0+0 (secure 0 0 0) 0x7fcc041412d0 con 0x7fcc04143ae0 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbdc06c040 msgr2=0x7fcbdc06e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbdc06c040 0x7fcbdc06e500 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4001610 tx=0x7fcbf4007930 comp rx=0 tx=0).stop 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04143ae0 msgr2=0x7fcc04141620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04143ae0 0x7fcc04141620 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcbec02bee0 tx=0x7fcbec0046c0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 shutdown_connections 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc0415e1e0 0x7fcc040adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbdc06c040 0x7fcbdc06e500 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc04143ae0 0x7fcc04141620 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 --2- 172.21.15.26:0/2424631891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc040cf9a0 0x7fcc04159950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:16.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 >> 172.21.15.26:0/2424631891 conn(0x7fcc0406a800 msgr2=0x7fcc04142d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:16.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 shutdown_connections 2024-01-15T18:09:16.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:16.827+0000 7fcc0b600640 1 -- 172.21.15.26:0/2424631891 wait complete. 2024-01-15T18:09:16.849 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:09:17.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.075+0000 7fb6e3200640 1 Processor -- start 2024-01-15T18:09:17.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.076+0000 7fb6e3200640 1 -- start start 2024-01-15T18:09:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.077+0000 7fb6e3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e41616d0 0x7fb6e40aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.077+0000 7fb6e3200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb6e40af140 0x7fb6e406d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.077+0000 7fb6e3200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e40af5a0 con 0x7fb6e415d2c0 2024-01-15T18:09:17.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.077+0000 7fb6e3200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e40b2cc0 con 0x7fb6e40af140 2024-01-15T18:09:17.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.077+0000 7fb6e3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4160c80 con 0x7fb6e41616d0 2024-01-15T18:09:17.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.078+0000 7fb6e2800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb6e40af140 0x7fb6e406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.078+0000 7fb6e2800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb6e40af140 0x7fb6e406d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50264/0 (socket says 172.21.15.26:50264) 2024-01-15T18:09:17.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.078+0000 7fb6e2800640 1 -- 172.21.15.26:0/267511111 learned_addr learned my addr 172.21.15.26:0/267511111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.078+0000 7fb6e1400640 1 --2- 172.21.15.26:0/267511111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e41616d0 0x7fb6e40aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.078+0000 7fb6e2800640 1 -- 172.21.15.26:0/267511111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e41616d0 msgr2=0x7fb6e40aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.078+0000 7fb6e2800640 1 --2- 172.21.15.26:0/267511111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e41616d0 0x7fb6e40aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.078+0000 7fb6e2800640 1 -- 172.21.15.26:0/267511111 >> v1:172.21.15.124:6789/0 conn(0x7fb6e415d2c0 legacy=0x7fb6e40b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.079+0000 7fb6e2800640 1 -- 172.21.15.26:0/267511111 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e4161240 con 0x7fb6e40af140 2024-01-15T18:09:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.080+0000 7fb6e2800640 1 --2- 172.21.15.26:0/267511111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 0x7fb6e406d000 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009870 tx=0x7fb6dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9824d1d4d485d9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:17.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.080+0000 7fb6e0a00640 1 -- 172.21.15.26:0/267511111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc031640 con 0x7fb6e40af140 2024-01-15T18:09:17.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.081+0000 7fb6e0a00640 1 -- 172.21.15.26:0/267511111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc0317a0 con 0x7fb6e40af140 2024-01-15T18:09:17.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.081+0000 7fb6e0a00640 1 -- 172.21.15.26:0/267511111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc031b30 con 0x7fb6e40af140 2024-01-15T18:09:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.081+0000 7fb6e3200640 1 -- 172.21.15.26:0/267511111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 msgr2=0x7fb6e406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.081+0000 7fb6e3200640 1 --2- 172.21.15.26:0/267511111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 0x7fb6e406d000 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009870 tx=0x7fb6dc02b8f0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.081+0000 7fb6e3200640 1 -- 172.21.15.26:0/267511111 shutdown_connections 2024-01-15T18:09:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.081+0000 7fb6e3200640 1 --2- 172.21.15.26:0/267511111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 0x7fb6e406d000 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.081+0000 7fb6e3200640 1 --2- 172.21.15.26:0/267511111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e41616d0 0x7fb6e40aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.081+0000 7fb6e3200640 1 -- 172.21.15.26:0/267511111 >> 172.21.15.26:0/267511111 conn(0x7fb6e40b19c0 msgr2=0x7fb6e40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.082+0000 7fb6e3200640 1 -- 172.21.15.26:0/267511111 shutdown_connections 2024-01-15T18:09:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.082+0000 7fb6e3200640 1 -- 172.21.15.26:0/267511111 wait complete. 2024-01-15T18:09:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.083+0000 7fb6e3200640 1 Processor -- start 2024-01-15T18:09:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.084+0000 7fb6e3200640 1 -- start start 2024-01-15T18:09:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.084+0000 7fb6e3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 0x7fb6e4145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e415d2c0 0x7fb6e40c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e3200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6e41616d0 0x7fb6e415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4146aa0 con 0x7fb6e40af140 2024-01-15T18:09:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e3200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e41456c0 con 0x7fb6e41616d0 2024-01-15T18:09:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e4145e90 con 0x7fb6e415d2c0 2024-01-15T18:09:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 0x7fb6e4145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 0x7fb6e4145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50276/0 (socket says 172.21.15.26:50276) 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1e00640 1 -- 172.21.15.26:0/1414238389 learned_addr learned my addr 172.21.15.26:0/1414238389 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e2800640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6e41616d0 0x7fb6e415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1400640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e415d2c0 0x7fb6e40c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1e00640 1 -- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e415d2c0 msgr2=0x7fb6e40c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1e00640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e415d2c0 0x7fb6e40c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1e00640 1 -- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6e41616d0 msgr2=0x7fb6e415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1e00640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6e41616d0 0x7fb6e415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1e00640 1 -- 172.21.15.26:0/1414238389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc0094d0 con 0x7fb6e40af140 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e1400640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e415d2c0 0x7fb6e40c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.085+0000 7fb6e2800640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6e41616d0 0x7fb6e415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.086+0000 7fb6e1e00640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 0x7fb6e4145180 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400b3f0 tx=0x7fb6d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.086+0000 7fb6da000640 1 -- 172.21.15.26:0/1414238389 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4011830 con 0x7fb6e40af140 2024-01-15T18:09:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.086+0000 7fb6da000640 1 -- 172.21.15.26:0/1414238389 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d4011990 con 0x7fb6e40af140 2024-01-15T18:09:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.086+0000 7fb6da000640 1 -- 172.21.15.26:0/1414238389 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4011d40 con 0x7fb6e40af140 2024-01-15T18:09:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.086+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e4146170 con 0x7fb6e40af140 2024-01-15T18:09:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.086+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e4146730 con 0x7fb6e40af140 2024-01-15T18:09:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.088+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e4067b60 con 0x7fb6e40af140 2024-01-15T18:09:17.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.088+0000 7fb6da000640 1 -- 172.21.15.26:0/1414238389 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 31) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fb6d401a400 con 0x7fb6e40af140 2024-01-15T18:09:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.089+0000 7fb6da000640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6bc06c040 0x7fb6bc06e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.089+0000 7fb6da000640 1 -- 172.21.15.26:0/1414238389 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fb6d40887b0 con 0x7fb6e40af140 2024-01-15T18:09:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.090+0000 7fb6e1400640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6bc06c040 0x7fb6bc06e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.091+0000 7fb6e1400640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6bc06c040 0x7fb6bc06e500 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb6e41f6880 tx=0x7fb6cc00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:17.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.094+0000 7fb6da000640 1 -- 172.21.15.26:0/1414238389 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6d4054d00 con 0x7fb6e40af140 2024-01-15T18:09:17.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.322+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6e406e480 con 0x7fb6bc06c040 2024-01-15T18:09:17.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.358+0000 7fb6da000640 1 -- 172.21.15.26:0/1414238389 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb6e406e480 con 0x7fb6bc06c040 2024-01-15T18:09:17.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6bc06c040 msgr2=0x7fb6bc06e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6bc06c040 0x7fb6bc06e500 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb6e41f6880 tx=0x7fb6cc00b040 comp rx=0 tx=0).stop 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 msgr2=0x7fb6e4145180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 0x7fb6e4145180 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400b3f0 tx=0x7fb6d400b8b0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 shutdown_connections 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb6e415d2c0 0x7fb6e40c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb6bc06c040 0x7fb6bc06e500 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb6e41616d0 0x7fb6e415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 --2- 172.21.15.26:0/1414238389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb6e40af140 0x7fb6e4145180 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 >> 172.21.15.26:0/1414238389 conn(0x7fb6e40b19c0 msgr2=0x7fb6e40adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 shutdown_connections 2024-01-15T18:09:17.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.361+0000 7fb6e3200640 1 -- 172.21.15.26:0/1414238389 wait complete. 2024-01-15T18:09:17.382 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:09:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.602+0000 7fd7bd600640 1 Processor -- start 2024-01-15T18:09:17.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.603+0000 7fd7bd600640 1 -- start start 2024-01-15T18:09:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.604+0000 7fd7bd600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 0x7fd7b80d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.604+0000 7fd7bd600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd7b80b41e0 0x7fd7b80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.604+0000 7fd7bd600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b80d0320 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.604+0000 7fd7bd600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b80b2730 con 0x7fd7b80b41e0 2024-01-15T18:09:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.604+0000 7fd7bd600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80d1ec0 con 0x7fd7b815d4b0 2024-01-15T18:09:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.605+0000 7fd7b7400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd7b80cf9a0 0x7fd7b80cddf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52700/0 (socket says 172.21.15.26:52700) 2024-01-15T18:09:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.605+0000 7fd7b7400640 1 -- 172.21.15.26:0/2590776597 learned_addr learned my addr 172.21.15.26:0/2590776597 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:17.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.605+0000 7fd7bcc00640 1 --2- 172.21.15.26:0/2590776597 >> v2:172.21.15.26:3300/0 conn(0x7fd7b80b41e0 0x7fd7b80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.605+0000 7fd7b7e00640 1 --2- 172.21.15.26:0/2590776597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 0x7fd7b80d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.605+0000 7fd7bcc00640 1 -- 172.21.15.26:0/2590776597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 msgr2=0x7fd7b80d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:17.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.605+0000 7fd7bcc00640 1 --2- 172.21.15.26:0/2590776597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 0x7fd7b80d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.605+0000 7fd7bcc00640 1 -- 172.21.15.26:0/2590776597 >> v1:172.21.15.26:6789/0 conn(0x7fd7b80cf9a0 legacy=0x7fd7b80cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.605+0000 7fd7bcc00640 1 -- 172.21.15.26:0/2590776597 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80b29d0 con 0x7fd7b80b41e0 2024-01-15T18:09:17.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.606+0000 7fd7bcc00640 1 --2- 172.21.15.26:0/2590776597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80b41e0 0x7fd7b80cfde0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00097d0 tx=0x7fd7b002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7e9459f6845451 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:17.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7b6a00640 1 -- 172.21.15.26:0/2590776597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0031800 con 0x7fd7b80b41e0 2024-01-15T18:09:17.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7b6a00640 1 -- 172.21.15.26:0/2590776597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0031960 con 0x7fd7b80b41e0 2024-01-15T18:09:17.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7b6a00640 1 -- 172.21.15.26:0/2590776597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0031cf0 con 0x7fd7b80b41e0 2024-01-15T18:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7bd600640 1 -- 172.21.15.26:0/2590776597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80b41e0 msgr2=0x7fd7b80cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7bd600640 1 --2- 172.21.15.26:0/2590776597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80b41e0 0x7fd7b80cfde0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00097d0 tx=0x7fd7b002b8a0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7bd600640 1 -- 172.21.15.26:0/2590776597 shutdown_connections 2024-01-15T18:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7bd600640 1 --2- 172.21.15.26:0/2590776597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80b41e0 0x7fd7b80cfde0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7bd600640 1 --2- 172.21.15.26:0/2590776597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 0x7fd7b80d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7bd600640 1 -- 172.21.15.26:0/2590776597 >> 172.21.15.26:0/2590776597 conn(0x7fd7b80d3000 msgr2=0x7fd7b80d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:17.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7bd600640 1 -- 172.21.15.26:0/2590776597 shutdown_connections 2024-01-15T18:09:17.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.607+0000 7fd7bd600640 1 -- 172.21.15.26:0/2590776597 wait complete. 2024-01-15T18:09:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.609+0000 7fd7bd600640 1 Processor -- start 2024-01-15T18:09:17.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.609+0000 7fd7bd600640 1 -- start start 2024-01-15T18:09:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.610+0000 7fd7bd600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b80b41e0 0x7fd7b81f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.610+0000 7fd7bd600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80cf9a0 0x7fd7b81f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.610+0000 7fd7bd600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 0x7fd7b815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.610+0000 7fd7bd600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b815dba0 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.610+0000 7fd7bd600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b815ea00 con 0x7fd7b80b41e0 2024-01-15T18:09:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.610+0000 7fd7bd600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81f3c30 con 0x7fd7b815d4b0 2024-01-15T18:09:17.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7b7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80cf9a0 0x7fd7b81f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7b7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80cf9a0 0x7fd7b81f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50282/0 (socket says 172.21.15.26:50282) 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7b7400640 1 -- 172.21.15.26:0/2099072769 learned_addr learned my addr 172.21.15.26:0/2099072769 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7bcc00640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 0x7fd7b815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7b7e00640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b80b41e0 0x7fd7b81f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7b7400640 1 -- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 msgr2=0x7fd7b815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7b7400640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 0x7fd7b815f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7b7400640 1 -- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b80b41e0 msgr2=0x7fd7b81f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7b7400640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b80b41e0 0x7fd7b81f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7b7400640 1 -- 172.21.15.26:0/2099072769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b0009400 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.611+0000 7fd7bcc00640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 0x7fd7b815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.612+0000 7fd7b7400640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80cf9a0 0x7fd7b81f36f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000cba0 tx=0x7fd7a000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:17.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.612+0000 7fd7afe00640 1 -- 172.21.15.26:0/2099072769 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000ad40 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.612+0000 7fd7afe00640 1 -- 172.21.15.26:0/2099072769 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a0012400 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.612+0000 7fd7afe00640 1 -- 172.21.15.26:0/2099072769 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000aea0 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.612+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b815da10 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.612+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b815e0d0 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.617 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.613+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b8067450 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.614+0000 7fd7afe00640 1 -- 172.21.15.26:0/2099072769 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 31) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fd7a0007070 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.615+0000 7fd7afe00640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd79006c040 0x7fd79006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.615+0000 7fd7afe00640 1 -- 172.21.15.26:0/2099072769 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fd7a00887d0 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.616+0000 7fd7b7e00640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd79006c040 0x7fd79006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.617+0000 7fd7b7e00640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd79006c040 0x7fd79006e500 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8006fd0 tx=0x7fd7a8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:17.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.620+0000 7fd7afe00640 1 -- 172.21.15.26:0/2099072769 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd7a0054e90 con 0x7fd7b80cf9a0 2024-01-15T18:09:17.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.847+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7b815d890 con 0x7fd79006c040 2024-01-15T18:09:17.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.862+0000 7fd7afe00640 1 -- 172.21.15.26:0/2099072769 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fd7a800b5e0 con 0x7fd79006c040 2024-01-15T18:09:17.866 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df 2024-01-15T18:09:17.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd79006c040 msgr2=0x7fd79006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd79006c040 0x7fd79006e500 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8006fd0 tx=0x7fd7a8008040 comp rx=0 tx=0).stop 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80cf9a0 msgr2=0x7fd7b81f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80cf9a0 0x7fd7b81f36f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000cba0 tx=0x7fd7a000a430 comp rx=0 tx=0).stop 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 shutdown_connections 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd7b815d4b0 0x7fd7b815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd79006c040 0x7fd79006e500 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd7b80cf9a0 0x7fd7b81f36f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 --2- 172.21.15.26:0/2099072769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd7b80b41e0 0x7fd7b81f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 >> 172.21.15.26:0/2099072769 conn(0x7fd7b80d3000 msgr2=0x7fd7b806c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 shutdown_connections 2024-01-15T18:09:17.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:17.865+0000 7fd7bd600640 1 -- 172.21.15.26:0/2099072769 wait complete. 2024-01-15T18:09:17.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:17.889 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df) 2024-01-15T18:09:17.953 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df 2024-01-15T18:09:17.953 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:09:17.953 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:09:17.955 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:09:19.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:19.956 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df) 2024-01-15T18:09:20.029 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/aad2a165-f581-4d32-9038-7e5d9b5873df 2024-01-15T18:09:20.029 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:09:20.029 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:09:20.030 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:09:20.030 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-01-15T18:09:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.266+0000 7f6a8e400640 1 Processor -- start 2024-01-15T18:09:20.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.267+0000 7f6a8e400640 1 -- start start 2024-01-15T18:09:20.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.268+0000 7f6a8e400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6a88160c70 0x7f6a880d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.268+0000 7f6a8e400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a880cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.268+0000 7f6a8e400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a880cf6a0 con 0x7f6a88160890 2024-01-15T18:09:20.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.268+0000 7f6a8e400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a880d1430 con 0x7f6a88160c70 2024-01-15T18:09:20.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.268+0000 7f6a8e400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a880b4370 con 0x7f6a880c6be0 2024-01-15T18:09:20.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.269+0000 7f6a87e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6a88160c70 0x7f6a880d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.269+0000 7f6a8da00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a880cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.269+0000 7f6a8da00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a880cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:54714/0 (socket says 172.21.15.26:54714) 2024-01-15T18:09:20.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.269+0000 7f6a8da00640 1 -- 172.21.15.26:0/2811271338 learned_addr learned my addr 172.21.15.26:0/2811271338 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:20.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.270+0000 7f6a8da00640 1 -- 172.21.15.26:0/2811271338 >> v2:172.21.15.124:3300/0 conn(0x7f6a88160c70 msgr2=0x7f6a880d3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:20.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.270+0000 7f6a8da00640 1 --2- 172.21.15.26:0/2811271338 >> v2:172.21.15.124:3300/0 conn(0x7f6a88160c70 0x7f6a880d3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.270+0000 7f6a8da00640 1 -- 172.21.15.26:0/2811271338 >> v1:172.21.15.124:6789/0 conn(0x7f6a88160890 legacy=0x7f6a880d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.270+0000 7f6a8da00640 1 -- 172.21.15.26:0/2811271338 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a880ce7c0 con 0x7f6a880c6be0 2024-01-15T18:09:20.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.271+0000 7f6a8da00640 1 --2- 172.21.15.26:0/2811271338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a880cfb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009e50 tx=0x7f6a8002be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=955dc77a2a317c05 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.271+0000 7f6a87400640 1 -- 172.21.15.26:0/2811271338 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8002c9f0 con 0x7f6a880c6be0 2024-01-15T18:09:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.271+0000 7f6a87400640 1 -- 172.21.15.26:0/2811271338 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8002cb50 con 0x7f6a880c6be0 2024-01-15T18:09:20.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.271+0000 7f6a87400640 1 -- 172.21.15.26:0/2811271338 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800343f0 con 0x7f6a880c6be0 2024-01-15T18:09:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.272+0000 7f6a8e400640 1 -- 172.21.15.26:0/2811271338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 msgr2=0x7f6a880cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.272+0000 7f6a8e400640 1 --2- 172.21.15.26:0/2811271338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a880cfb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009e50 tx=0x7f6a8002be50 comp rx=0 tx=0).stop 2024-01-15T18:09:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.272+0000 7f6a8e400640 1 -- 172.21.15.26:0/2811271338 shutdown_connections 2024-01-15T18:09:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.272+0000 7f6a8e400640 1 --2- 172.21.15.26:0/2811271338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a880cfb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.272+0000 7f6a8e400640 1 --2- 172.21.15.26:0/2811271338 >> v2:172.21.15.124:3300/0 conn(0x7f6a88160c70 0x7f6a880d3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.272+0000 7f6a8e400640 1 -- 172.21.15.26:0/2811271338 >> 172.21.15.26:0/2811271338 conn(0x7f6a880d3090 msgr2=0x7f6a880d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.272+0000 7f6a8e400640 1 -- 172.21.15.26:0/2811271338 shutdown_connections 2024-01-15T18:09:20.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.272+0000 7f6a8e400640 1 -- 172.21.15.26:0/2811271338 wait complete. 2024-01-15T18:09:20.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.274+0000 7f6a8e400640 1 Processor -- start 2024-01-15T18:09:20.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.274+0000 7f6a8e400640 1 -- start start 2024-01-15T18:09:20.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.275+0000 7f6a8e400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a88153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.275+0000 7f6a8e400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a88160890 0x7f6a88152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.275+0000 7f6a8e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a88160c70 0x7f6a881f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.275+0000 7f6a8e400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a88153fc0 con 0x7f6a88160c70 2024-01-15T18:09:20.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.275+0000 7f6a8e400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a88152b60 con 0x7f6a88160890 2024-01-15T18:09:20.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.275+0000 7f6a8e400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a88155650 con 0x7f6a880c6be0 2024-01-15T18:09:20.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.275+0000 7f6a8da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a88160c70 0x7f6a881f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a8da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a88160c70 0x7f6a881f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50286/0 (socket says 172.21.15.26:50286) 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a8da00640 1 -- 172.21.15.26:0/3506630264 learned_addr learned my addr 172.21.15.26:0/3506630264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a87e00640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a88160890 0x7f6a88152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a8d000640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a88153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a8da00640 1 -- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 msgr2=0x7f6a88153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a8da00640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a88153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a8da00640 1 -- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a88160890 msgr2=0x7f6a88152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a8da00640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a88160890 0x7f6a88152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a8da00640 1 -- 172.21.15.26:0/3506630264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a80009b30 con 0x7f6a88160c70 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.276+0000 7f6a87e00640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a88160890 0x7f6a88152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.277+0000 7f6a8da00640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a88160c70 0x7f6a881f6500 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009e50 tx=0x7f6a80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.277+0000 7f6a84c00640 1 -- 172.21.15.26:0/3506630264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80009510 con 0x7f6a88160c70 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.277+0000 7f6a84c00640 1 -- 172.21.15.26:0/3506630264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8003e450 con 0x7f6a88160c70 2024-01-15T18:09:20.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.277+0000 7f6a84c00640 1 -- 172.21.15.26:0/3506630264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8003e800 con 0x7f6a88160c70 2024-01-15T18:09:20.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.277+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a881541d0 con 0x7f6a88160c70 2024-01-15T18:09:20.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.277+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a88155bd0 con 0x7f6a88160c70 2024-01-15T18:09:20.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.278+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a88155f10 con 0x7f6a88160c70 2024-01-15T18:09:20.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.279+0000 7f6a84c00640 1 -- 172.21.15.26:0/3506630264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 31) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f6a8003e960 con 0x7f6a88160c70 2024-01-15T18:09:20.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.280+0000 7f6a84c00640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a6006c040 0x7f6a6006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.280+0000 7f6a84c00640 1 -- 172.21.15.26:0/3506630264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f6a800ab5e0 con 0x7f6a88160c70 2024-01-15T18:09:20.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.281+0000 7f6a8d000640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a6006c040 0x7f6a6006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.282+0000 7f6a8d000640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a6006c040 0x7f6a6006e500 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6a880c6bb0 tx=0x7f6a78009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:20.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.285+0000 7f6a84c00640 1 -- 172.21.15.26:0/3506630264 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a80077bf0 con 0x7f6a88160c70 2024-01-15T18:09:20.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.521+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a880ced80 con 0x7f6a6006c040 2024-01-15T18:09:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.666+0000 7f6a84c00640 1 -- 172.21.15.26:0/3506630264 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6a780012c0 con 0x7f6a6006c040 2024-01-15T18:09:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.669+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a6006c040 msgr2=0x7f6a6006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a6006c040 0x7f6a6006e500 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6a880c6bb0 tx=0x7f6a78009040 comp rx=0 tx=0).stop 2024-01-15T18:09:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a88160c70 msgr2=0x7f6a881f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a88160c70 0x7f6a881f6500 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009e50 tx=0x7f6a80004a40 comp rx=0 tx=0).stop 2024-01-15T18:09:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 shutdown_connections 2024-01-15T18:09:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a6006c040 0x7f6a6006e500 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a88160c70 0x7f6a881f6500 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a88160890 0x7f6a88152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 --2- 172.21.15.26:0/3506630264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a880c6be0 0x7f6a88153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 >> 172.21.15.26:0/3506630264 conn(0x7f6a880d3090 msgr2=0x7f6a8806ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 shutdown_connections 2024-01-15T18:09:20.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.670+0000 7f6a8e400640 1 -- 172.21.15.26:0/3506630264 wait complete. 2024-01-15T18:09:20.676 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.673+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.676 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.673+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.676 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.673+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.676 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.673+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.677 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.673+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.693 INFO:tasks.cephfs.test_volumes:expected clone state = ('in-progress',) 2024-01-15T18:09:20.694 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:20.696 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.693+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.697 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.693+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.697 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.693+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.697 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.693+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.697 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:20.693+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:20.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.914+0000 7fbb15600640 1 Processor -- start 2024-01-15T18:09:20.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.914+0000 7fbb15600640 1 -- start start 2024-01-15T18:09:20.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.916+0000 7fbb15600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb1013cf10 0x7fbb1013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.916+0000 7fbb15600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbb100cddf0 0x7fbb100b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.916+0000 7fbb15600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb100ce440 con 0x7fbb1013fae0 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.916+0000 7fbb15600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1013ff90 con 0x7fbb1013cf10 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.916+0000 7fbb15600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb100ce220 con 0x7fbb100cddf0 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.916+0000 7fbb0fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb1013cf10 0x7fbb1013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.917+0000 7fbb07e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbb1013fae0 0x7fbb100cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52710/0 (socket says 172.21.15.26:52710) 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.917+0000 7fbb0fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbb1013cf10 0x7fbb1013f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50294/0 (socket says 172.21.15.26:50294) 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.917+0000 7fbb07e00640 1 -- 172.21.15.26:0/3789494784 learned_addr learned my addr 172.21.15.26:0/3789494784 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.917+0000 7fbb14c00640 1 --2- 172.21.15.26:0/3789494784 >> v2:172.21.15.124:3300/0 conn(0x7fbb100cddf0 0x7fbb100b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.917+0000 7fbb0fe00640 1 -- 172.21.15.26:0/3789494784 >> v2:172.21.15.124:3300/0 conn(0x7fbb100cddf0 msgr2=0x7fbb100b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:20.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.917+0000 7fbb0fe00640 1 --2- 172.21.15.26:0/3789494784 >> v2:172.21.15.124:3300/0 conn(0x7fbb100cddf0 0x7fbb100b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.917+0000 7fbb0fe00640 1 -- 172.21.15.26:0/3789494784 >> v1:172.21.15.26:6789/0 conn(0x7fbb1013fae0 legacy=0x7fbb100cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.917+0000 7fbb0fe00640 1 -- 172.21.15.26:0/3789494784 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100ce720 con 0x7fbb1013cf10 2024-01-15T18:09:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.918+0000 7fbb0fe00640 1 --2- 172.21.15.26:0/3789494784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013cf10 0x7fbb1013f5a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fbb10157f90 tx=0x7fbb0002dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d17376baa14e73 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.918+0000 7fbb0f400640 1 -- 172.21.15.26:0/3789494784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0002f8a0 con 0x7fbb1013cf10 2024-01-15T18:09:20.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.919+0000 7fbb0f400640 1 -- 172.21.15.26:0/3789494784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0002fa00 con 0x7fbb1013cf10 2024-01-15T18:09:20.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.919+0000 7fbb0f400640 1 -- 172.21.15.26:0/3789494784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0002fd90 con 0x7fbb1013cf10 2024-01-15T18:09:20.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.919+0000 7fbb15600640 1 -- 172.21.15.26:0/3789494784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013cf10 msgr2=0x7fbb1013f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.919+0000 7fbb15600640 1 --2- 172.21.15.26:0/3789494784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013cf10 0x7fbb1013f5a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fbb10157f90 tx=0x7fbb0002dae0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.920+0000 7fbb15600640 1 -- 172.21.15.26:0/3789494784 shutdown_connections 2024-01-15T18:09:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.920+0000 7fbb15600640 1 --2- 172.21.15.26:0/3789494784 >> v2:172.21.15.124:3300/0 conn(0x7fbb100cddf0 0x7fbb100b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.920+0000 7fbb15600640 1 --2- 172.21.15.26:0/3789494784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013cf10 0x7fbb1013f5a0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.920+0000 7fbb15600640 1 -- 172.21.15.26:0/3789494784 >> 172.21.15.26:0/3789494784 conn(0x7fbb1006a7d0 msgr2=0x7fbb1006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.920+0000 7fbb15600640 1 -- 172.21.15.26:0/3789494784 shutdown_connections 2024-01-15T18:09:20.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.920+0000 7fbb15600640 1 -- 172.21.15.26:0/3789494784 wait complete. 2024-01-15T18:09:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.921+0000 7fbb15600640 1 Processor -- start 2024-01-15T18:09:20.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.922+0000 7fbb15600640 1 -- start start 2024-01-15T18:09:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb15600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb100cddf0 0x7fbb10151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb15600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1013cf10 0x7fbb10139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb15600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013fae0 0x7fbb1012ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb15600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1013e980 con 0x7fbb1013fae0 2024-01-15T18:09:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb15600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1013a520 con 0x7fbb100cddf0 2024-01-15T18:09:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb15600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1013a800 con 0x7fbb1013cf10 2024-01-15T18:09:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb14c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013fae0 0x7fbb1012ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb14c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013fae0 0x7fbb1012ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50298/0 (socket says 172.21.15.26:50298) 2024-01-15T18:09:20.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb14c00640 1 -- 172.21.15.26:0/2477023568 learned_addr learned my addr 172.21.15.26:0/2477023568 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb07e00640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1013cf10 0x7fbb10139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb14c00640 1 -- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1013cf10 msgr2=0x7fbb10139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb14c00640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1013cf10 0x7fbb10139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb14c00640 1 -- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb100cddf0 msgr2=0x7fbb10151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb14c00640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb100cddf0 0x7fbb10151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:20.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.923+0000 7fbb14c00640 1 -- 172.21.15.26:0/2477023568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00009400 con 0x7fbb1013fae0 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.924+0000 7fbb14c00640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013fae0 0x7fbb1012ff20 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800cd70 tx=0x7fbb0800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.924+0000 7fbb0cc00640 1 -- 172.21.15.26:0/2477023568 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08003b90 con 0x7fbb1013fae0 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.924+0000 7fbb0cc00640 1 -- 172.21.15.26:0/2477023568 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08003cf0 con 0x7fbb1013fae0 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.924+0000 7fbb0cc00640 1 -- 172.21.15.26:0/2477023568 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08004240 con 0x7fbb1013fae0 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.924+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1013e5a0 con 0x7fbb1013fae0 2024-01-15T18:09:20.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.924+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb10130460 con 0x7fbb1013fae0 2024-01-15T18:09:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.926+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1013eac0 con 0x7fbb1013fae0 2024-01-15T18:09:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.927+0000 7fbb0cc00640 1 -- 172.21.15.26:0/2477023568 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 31) v1 ==== 90763+0+0 (secure 0 0 0) 0x7fbb080043a0 con 0x7fbb1013fae0 2024-01-15T18:09:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.928+0000 7fbb0cc00640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbae806c040 0x7fbae806e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:20.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.928+0000 7fbb0cc00640 1 -- 172.21.15.26:0/2477023568 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fbb08012070 con 0x7fbb1013fae0 2024-01-15T18:09:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.928+0000 7fbb0fe00640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbae806c040 0x7fbae806e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:20.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.929+0000 7fbb0fe00640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbae806c040 0x7fbae806e500 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbb00038000 tx=0x7fbb00032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:20.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:20.932+0000 7fbb0cc00640 1 -- 172.21.15.26:0/2477023568 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb1013eac0 con 0x7fbb1013fae0 2024-01-15T18:09:21.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.165+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb100b2070 con 0x7fbae806c040 2024-01-15T18:09:21.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.183+0000 7fbb0cc00640 1 -- 172.21.15.26:0/2477023568 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fbb0002fb70 con 0x7fbae806c040 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:21.187 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.185+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbae806c040 msgr2=0x7fbae806e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.185+0000 7fbb15600640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbae806c040 0x7fbae806e500 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbb00038000 tx=0x7fbb00032000 comp rx=0 tx=0).stop 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013fae0 msgr2=0x7fbb1012ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013fae0 0x7fbb1012ff20 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800cd70 tx=0x7fbb0800a660 comp rx=0 tx=0).stop 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 shutdown_connections 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbb1013cf10 0x7fbb10139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbae806c040 0x7fbae806e500 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbb1013fae0 0x7fbb1012ff20 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 --2- 172.21.15.26:0/2477023568 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbb100cddf0 0x7fbb10151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 >> 172.21.15.26:0/2477023568 conn(0x7fbb1006a7d0 msgr2=0x7fbb10071760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 shutdown_connections 2024-01-15T18:09:21.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:21.186+0000 7fbb15600640 1 -- 172.21.15.26:0/2477023568 wait complete. 2024-01-15T18:09:21.208 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:09:22.210 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.439+0000 7f52bd000640 1 Processor -- start 2024-01-15T18:09:22.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.439+0000 7f52bd000640 1 -- start start 2024-01-15T18:09:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.440+0000 7f52bd000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f52b815d4b0 0x7f52b80d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.440+0000 7f52bd000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b80b41e0 0x7f52b80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.440+0000 7f52bd000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52b80d0320 con 0x7f52b80cf9a0 2024-01-15T18:09:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.440+0000 7f52bd000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52b80b2730 con 0x7f52b815d4b0 2024-01-15T18:09:22.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.440+0000 7f52bd000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b80d1ec0 con 0x7f52b80b41e0 2024-01-15T18:09:22.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.441+0000 7f52b7400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f52b815d4b0 0x7f52b80d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.441+0000 7f52b7400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f52b815d4b0 0x7f52b80d1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:54092/0 (socket says 172.21.15.26:54092) 2024-01-15T18:09:22.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.441+0000 7f52b7400640 1 -- 172.21.15.26:0/1414728213 learned_addr learned my addr 172.21.15.26:0/1414728213 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:22.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.441+0000 7f52b7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b80b41e0 0x7f52b80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.442+0000 7f52b7400640 1 -- 172.21.15.26:0/1414728213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b80b41e0 msgr2=0x7f52b80cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.442+0000 7f52b7400640 1 --2- 172.21.15.26:0/1414728213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b80b41e0 0x7f52b80cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.442+0000 7f52b7400640 1 -- 172.21.15.26:0/1414728213 >> v1:172.21.15.124:6789/0 conn(0x7f52b80cf9a0 legacy=0x7f52b80cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.442+0000 7f52b7400640 1 -- 172.21.15.26:0/1414728213 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b80b29d0 con 0x7f52b815d4b0 2024-01-15T18:09:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.443+0000 7f52b7400640 1 --2- 172.21.15.26:0/1414728213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b815d4b0 0x7f52b80d1550 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009e50 tx=0x7f52a802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f9bb0eace8bd431 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.444+0000 7f52b6000640 1 -- 172.21.15.26:0/1414728213 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a802cb00 con 0x7f52b815d4b0 2024-01-15T18:09:22.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.444+0000 7f52b6000640 1 -- 172.21.15.26:0/1414728213 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a802cc60 con 0x7f52b815d4b0 2024-01-15T18:09:22.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.444+0000 7f52b6000640 1 -- 172.21.15.26:0/1414728213 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a80343f0 con 0x7f52b815d4b0 2024-01-15T18:09:22.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.445+0000 7f52bd000640 1 -- 172.21.15.26:0/1414728213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b815d4b0 msgr2=0x7f52b80d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.445+0000 7f52bd000640 1 --2- 172.21.15.26:0/1414728213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b815d4b0 0x7f52b80d1550 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009e50 tx=0x7f52a802bf60 comp rx=0 tx=0).stop 2024-01-15T18:09:22.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.445+0000 7f52bd000640 1 -- 172.21.15.26:0/1414728213 shutdown_connections 2024-01-15T18:09:22.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.445+0000 7f52bd000640 1 --2- 172.21.15.26:0/1414728213 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b80b41e0 0x7f52b80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.445+0000 7f52bd000640 1 --2- 172.21.15.26:0/1414728213 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b815d4b0 0x7f52b80d1550 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.445+0000 7f52bd000640 1 -- 172.21.15.26:0/1414728213 >> 172.21.15.26:0/1414728213 conn(0x7f52b80d3000 msgr2=0x7f52b80d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:22.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.445+0000 7f52bd000640 1 -- 172.21.15.26:0/1414728213 shutdown_connections 2024-01-15T18:09:22.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.445+0000 7f52bd000640 1 -- 172.21.15.26:0/1414728213 wait complete. 2024-01-15T18:09:22.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.447+0000 7f52bd000640 1 Processor -- start 2024-01-15T18:09:22.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.447+0000 7f52bd000640 1 -- start start 2024-01-15T18:09:22.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.448+0000 7f52bd000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b80b41e0 0x7f52b815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52bd000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52b80cf9a0 0x7f52b80b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52bd000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b815d4b0 0x7f52b80b8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52bd000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b81f3ec0 con 0x7f52b80cf9a0 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52bd000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b815c500 con 0x7f52b80b41e0 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52bd000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52b809ce60 con 0x7f52b815d4b0 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52b6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52b80cf9a0 0x7f52b80b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52b6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52b80cf9a0 0x7f52b80b8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34908/0 (socket says 172.21.15.26:34908) 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52b6a00640 1 -- 172.21.15.26:0/3631519302 learned_addr learned my addr 172.21.15.26:0/3631519302 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52b7400640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b80b41e0 0x7f52b815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.449+0000 7f52b7e00640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b815d4b0 0x7f52b80b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52b6a00640 1 -- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b815d4b0 msgr2=0x7f52b80b8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52b6a00640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b815d4b0 0x7f52b80b8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52b6a00640 1 -- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b80b41e0 msgr2=0x7f52b815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52b6a00640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b80b41e0 0x7f52b815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52b6a00640 1 -- 172.21.15.26:0/3631519302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a8009b30 con 0x7f52b80cf9a0 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52b7400640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b80b41e0 0x7f52b815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52b6a00640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52b80cf9a0 0x7f52b80b8230 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f52a000c450 tx=0x7f52a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52af400640 1 -- 172.21.15.26:0/3631519302 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a00052d0 con 0x7f52b80cf9a0 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52af400640 1 -- 172.21.15.26:0/3631519302 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a0005430 con 0x7f52b80cf9a0 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52af400640 1 -- 172.21.15.26:0/3631519302 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0012870 con 0x7f52b80cf9a0 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.450+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b815da10 con 0x7f52b80cf9a0 2024-01-15T18:09:22.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.451+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b809d400 con 0x7f52b80cf9a0 2024-01-15T18:09:22.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.452+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b80b3fd0 con 0x7f52b80cf9a0 2024-01-15T18:09:22.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.453+0000 7f52af400640 1 -- 172.21.15.26:0/3631519302 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 31) v1 ==== 90763+0+0 (secure 0 0 0) 0x7f52a00129d0 con 0x7f52b80cf9a0 2024-01-15T18:09:22.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.454+0000 7f52af400640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f529006c040 0x7f529006e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:22.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.454+0000 7f52af400640 1 -- 172.21.15.26:0/3631519302 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f52a0088430 con 0x7f52b80cf9a0 2024-01-15T18:09:22.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.454+0000 7f52b7400640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f529006c040 0x7f529006e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:22.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.456+0000 7f52b7400640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f529006c040 0x7f529006e500 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f52a8035000 tx=0x7f52a8030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:22.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.461+0000 7f52af400640 1 -- 172.21.15.26:0/3631519302 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52a0054a40 con 0x7f52b80cf9a0 2024-01-15T18:09:22.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.693+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f52b815d890 con 0x7f529006c040 2024-01-15T18:09:22.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.708+0000 7f52af400640 1 -- 172.21.15.26:0/3631519302 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f52a80440c0 con 0x7f529006c040 2024-01-15T18:09:22.711 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:22.712 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:22.712 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:09:22.712 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:22.712 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:22.712 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:22.712 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:22.712 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:22.712 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:22.712 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:22.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f529006c040 msgr2=0x7f529006e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f529006c040 0x7f529006e500 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f52a8035000 tx=0x7f52a8030000 comp rx=0 tx=0).stop 2024-01-15T18:09:22.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52b80cf9a0 msgr2=0x7f52b80b8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:22.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52b80cf9a0 0x7f52b80b8230 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f52a000c450 tx=0x7f52a000c910 comp rx=0 tx=0).stop 2024-01-15T18:09:22.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 shutdown_connections 2024-01-15T18:09:22.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f52b815d4b0 0x7f52b80b8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f529006c040 0x7f529006e500 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52b80cf9a0 0x7f52b80b8230 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 --2- 172.21.15.26:0/3631519302 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f52b80b41e0 0x7f52b815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:22.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 >> 172.21.15.26:0/3631519302 conn(0x7f52b80d3000 msgr2=0x7f52b806c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:22.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.710+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 shutdown_connections 2024-01-15T18:09:22.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:22.711+0000 7f52bd000640 1 -- 172.21.15.26:0/3631519302 wait complete. 2024-01-15T18:09:22.733 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:09:23.734 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.968+0000 7fcbea400640 1 Processor -- start 2024-01-15T18:09:23.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.968+0000 7fcbea400640 1 -- start start 2024-01-15T18:09:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.969+0000 7fcbea400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcbe413cf10 0x7fcbe413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbea400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcbe413fae0 0x7fcbe40cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbea400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe40ce440 con 0x7fcbe40cdc20 2024-01-15T18:09:23.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbea400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe413ff90 con 0x7fcbe413cf10 2024-01-15T18:09:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbea400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe40ce720 con 0x7fcbe413fae0 2024-01-15T18:09:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbe9000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcbe413cf10 0x7fcbe413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbe9000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcbe413cf10 0x7fcbe413f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34914/0 (socket says 172.21.15.26:34914) 2024-01-15T18:09:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbe9000640 1 -- 172.21.15.26:0/2043500775 learned_addr learned my addr 172.21.15.26:0/2043500775 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbdbc00640 1 --2- 172.21.15.26:0/2043500775 >> v2:172.21.15.124:3300/0 conn(0x7fcbe413fae0 0x7fcbe40cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbe9000640 1 -- 172.21.15.26:0/2043500775 >> v2:172.21.15.124:3300/0 conn(0x7fcbe413fae0 msgr2=0x7fcbe40cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbe9000640 1 --2- 172.21.15.26:0/2043500775 >> v2:172.21.15.124:3300/0 conn(0x7fcbe413fae0 0x7fcbe40cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbe9000640 1 -- 172.21.15.26:0/2043500775 >> v1:172.21.15.124:6789/0 conn(0x7fcbe40cdc20 legacy=0x7fcbe415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbe9000640 1 -- 172.21.15.26:0/2043500775 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe40cea00 con 0x7fcbe413cf10 2024-01-15T18:09:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.970+0000 7fcbdbc00640 1 --2- 172.21.15.26:0/2043500775 >> v2:172.21.15.124:3300/0 conn(0x7fcbe413fae0 0x7fcbe40cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:23.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.972+0000 7fcbe9000640 1 --2- 172.21.15.26:0/2043500775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 0x7fcbe413f5a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcbe406ffb0 tx=0x7fcbd402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc3a31a57077b2f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.972+0000 7fcbe3e00640 1 -- 172.21.15.26:0/2043500775 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4031870 con 0x7fcbe413cf10 2024-01-15T18:09:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.972+0000 7fcbe3e00640 1 -- 172.21.15.26:0/2043500775 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd40319d0 con 0x7fcbe413cf10 2024-01-15T18:09:23.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.972+0000 7fcbe3e00640 1 -- 172.21.15.26:0/2043500775 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4031d60 con 0x7fcbe413cf10 2024-01-15T18:09:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.973+0000 7fcbea400640 1 -- 172.21.15.26:0/2043500775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 msgr2=0x7fcbe413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.973+0000 7fcbea400640 1 --2- 172.21.15.26:0/2043500775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 0x7fcbe413f5a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fcbe406ffb0 tx=0x7fcbd402b910 comp rx=0 tx=0).stop 2024-01-15T18:09:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.973+0000 7fcbea400640 1 -- 172.21.15.26:0/2043500775 shutdown_connections 2024-01-15T18:09:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.973+0000 7fcbea400640 1 --2- 172.21.15.26:0/2043500775 >> v2:172.21.15.124:3300/0 conn(0x7fcbe413fae0 0x7fcbe40cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.973+0000 7fcbea400640 1 --2- 172.21.15.26:0/2043500775 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 0x7fcbe413f5a0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.973+0000 7fcbea400640 1 -- 172.21.15.26:0/2043500775 >> 172.21.15.26:0/2043500775 conn(0x7fcbe406a7d0 msgr2=0x7fcbe406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.973+0000 7fcbea400640 1 -- 172.21.15.26:0/2043500775 shutdown_connections 2024-01-15T18:09:23.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.973+0000 7fcbea400640 1 -- 172.21.15.26:0/2043500775 wait complete. 2024-01-15T18:09:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.975+0000 7fcbea400640 1 Processor -- start 2024-01-15T18:09:23.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.975+0000 7fcbea400640 1 -- start start 2024-01-15T18:09:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.976+0000 7fcbea400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbe40cdc20 0x7fcbe413d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.976+0000 7fcbea400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 0x7fcbe413b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.976+0000 7fcbea400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcbe413fae0 0x7fcbe413b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.976+0000 7fcbea400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe4156800 con 0x7fcbe413cf10 2024-01-15T18:09:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.976+0000 7fcbea400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe415ba00 con 0x7fcbe413fae0 2024-01-15T18:09:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.976+0000 7fcbea400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe4156240 con 0x7fcbe40cdc20 2024-01-15T18:09:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.976+0000 7fcbdbc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 0x7fcbe413b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.976+0000 7fcbdbc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 0x7fcbe413b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34930/0 (socket says 172.21.15.26:34930) 2024-01-15T18:09:23.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.976+0000 7fcbdbc00640 1 -- 172.21.15.26:0/2335679395 learned_addr learned my addr 172.21.15.26:0/2335679395 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbe9a00640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcbe413fae0 0x7fcbe413b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbdbc00640 1 -- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbe40cdc20 msgr2=0x7fcbe413d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbdbc00640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbe40cdc20 0x7fcbe413d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbdbc00640 1 -- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcbe413fae0 msgr2=0x7fcbe413b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbdbc00640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcbe413fae0 0x7fcbe413b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbdbc00640 1 -- 172.21.15.26:0/2335679395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd40094d0 con 0x7fcbe413cf10 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbdbc00640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 0x7fcbe413b420 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00c820 tx=0x7fcbcc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbe1600640 1 -- 172.21.15.26:0/2335679395 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc007a30 con 0x7fcbe413cf10 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbe1600640 1 -- 172.21.15.26:0/2335679395 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbcc007b90 con 0x7fcbe413cf10 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbe1600640 1 -- 172.21.15.26:0/2335679395 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00ce10 con 0x7fcbe413cf10 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe4156520 con 0x7fcbe413cf10 2024-01-15T18:09:23.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.977+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe413bea0 con 0x7fcbe413cf10 2024-01-15T18:09:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.978+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe4067450 con 0x7fcbe413cf10 2024-01-15T18:09:23.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.980+0000 7fcbe1600640 1 -- 172.21.15.26:0/2335679395 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7fcbcc0113f0 con 0x7fcbe413cf10 2024-01-15T18:09:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.980+0000 7fcbe1600640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbbc06c160 0x7fcbbc06e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.981+0000 7fcbe1600640 1 -- 172.21.15.26:0/2335679395 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fcbcc088ab0 con 0x7fcbe413cf10 2024-01-15T18:09:23.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.981+0000 7fcbe9000640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbbc06c160 0x7fcbbc06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:23.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.982+0000 7fcbe9000640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbbc06c160 0x7fcbbc06e620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4036000 tx=0x7fcbd4030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:23.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:23.985+0000 7fcbe1600640 1 -- 172.21.15.26:0/2335679395 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcbcc055090 con 0x7fcbe413cf10 2024-01-15T18:09:24.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.215+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbe40b2070 con 0x7fcbbc06c160 2024-01-15T18:09:24.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.230+0000 7fcbe1600640 1 -- 172.21.15.26:0/2335679395 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fcbd4031bc0 con 0x7fcbbc06c160 2024-01-15T18:09:24.234 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:24.234 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:24.234 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:09:24.234 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:24.235 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:24.235 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:24.235 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:24.235 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:24.235 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:24.235 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbbc06c160 msgr2=0x7fcbbc06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbbc06c160 0x7fcbbc06e620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4036000 tx=0x7fcbd4030000 comp rx=0 tx=0).stop 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 msgr2=0x7fcbe413b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 0x7fcbe413b420 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00c820 tx=0x7fcbcc00cce0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 shutdown_connections 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcbbc06c160 0x7fcbbc06e620 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcbe413fae0 0x7fcbe413b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcbe413cf10 0x7fcbe413b420 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 --2- 172.21.15.26:0/2335679395 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcbe40cdc20 0x7fcbe413d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 >> 172.21.15.26:0/2335679395 conn(0x7fcbe406a7d0 msgr2=0x7fcbe40ae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:24.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 shutdown_connections 2024-01-15T18:09:24.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:24.233+0000 7fcbea400640 1 -- 172.21.15.26:0/2335679395 wait complete. 2024-01-15T18:09:24.253 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:09:25.254 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:25.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.486+0000 7fec55200640 1 Processor -- start 2024-01-15T18:09:25.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.487+0000 7fec55200640 1 -- start start 2024-01-15T18:09:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.488+0000 7fec55200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec50143ae0 0x7fec500cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.488+0000 7fec55200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fec500cf7d0 0x7fec5015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:25.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.488+0000 7fec55200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec5015e1e0 0x7fec5015a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.488+0000 7fec55200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec500cfc30 con 0x7fec5015e1e0 2024-01-15T18:09:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.488+0000 7fec55200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec500b1ee0 con 0x7fec500cf7d0 2024-01-15T18:09:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.488+0000 7fec55200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec500b2780 con 0x7fec50143ae0 2024-01-15T18:09:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec4e800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec5015e1e0 0x7fec5015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec4e800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec5015e1e0 0x7fec5015a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34938/0 (socket says 172.21.15.26:34938) 2024-01-15T18:09:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec4e800640 1 -- 172.21.15.26:0/1201056641 learned_addr learned my addr 172.21.15.26:0/1201056641 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec45200640 1 --2- 172.21.15.26:0/1201056641 >> v2:172.21.15.124:3300/0 conn(0x7fec500cf7d0 0x7fec5015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec4de00640 1 --2- 172.21.15.26:0/1201056641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec50143ae0 0x7fec500cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec4e800640 1 -- 172.21.15.26:0/1201056641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec50143ae0 msgr2=0x7fec500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec4e800640 1 --2- 172.21.15.26:0/1201056641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec50143ae0 0x7fec500cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec4e800640 1 -- 172.21.15.26:0/1201056641 >> v2:172.21.15.124:3300/0 conn(0x7fec500cf7d0 msgr2=0x7fec5015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec4e800640 1 --2- 172.21.15.26:0/1201056641 >> v2:172.21.15.124:3300/0 conn(0x7fec500cf7d0 0x7fec5015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec4e800640 1 -- 172.21.15.26:0/1201056641 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5015e990 con 0x7fec5015e1e0 2024-01-15T18:09:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.489+0000 7fec45200640 1 --2- 172.21.15.26:0/1201056641 >> v2:172.21.15.124:3300/0 conn(0x7fec500cf7d0 0x7fec5015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.491+0000 7fec4e800640 1 --2- 172.21.15.26:0/1201056641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5015e1e0 0x7fec5015a670 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fec5006e350 tx=0x7fec4802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d931eed755c31cb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.491+0000 7fec4d400640 1 -- 172.21.15.26:0/1201056641 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4802ca80 con 0x7fec5015e1e0 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.491+0000 7fec4d400640 1 -- 172.21.15.26:0/1201056641 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4802cbe0 con 0x7fec5015e1e0 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.491+0000 7fec4d400640 1 -- 172.21.15.26:0/1201056641 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec480343f0 con 0x7fec5015e1e0 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.491+0000 7fec55200640 1 -- 172.21.15.26:0/1201056641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5015e1e0 msgr2=0x7fec5015a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.491+0000 7fec55200640 1 --2- 172.21.15.26:0/1201056641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5015e1e0 0x7fec5015a670 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fec5006e350 tx=0x7fec4802bee0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.492+0000 7fec55200640 1 -- 172.21.15.26:0/1201056641 shutdown_connections 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.492+0000 7fec55200640 1 --2- 172.21.15.26:0/1201056641 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec5015e1e0 0x7fec5015a670 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.492+0000 7fec55200640 1 --2- 172.21.15.26:0/1201056641 >> v2:172.21.15.124:3300/0 conn(0x7fec500cf7d0 0x7fec5015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.492+0000 7fec55200640 1 --2- 172.21.15.26:0/1201056641 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec50143ae0 0x7fec500cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.492+0000 7fec55200640 1 -- 172.21.15.26:0/1201056641 >> 172.21.15.26:0/1201056641 conn(0x7fec5006a800 msgr2=0x7fec5006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.492+0000 7fec55200640 1 -- 172.21.15.26:0/1201056641 shutdown_connections 2024-01-15T18:09:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.492+0000 7fec55200640 1 -- 172.21.15.26:0/1201056641 wait complete. 2024-01-15T18:09:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.494+0000 7fec55200640 1 Processor -- start 2024-01-15T18:09:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.494+0000 7fec55200640 1 -- start start 2024-01-15T18:09:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec55200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec500cf7d0 0x7fec50142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec55200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec50143ae0 0x7fec5006c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec55200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec5015e1e0 0x7fec5006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec55200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec501622a0 con 0x7fec500cf7d0 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec55200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec50161690 con 0x7fec50143ae0 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec55200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec50161970 con 0x7fec5015e1e0 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec4de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec500cf7d0 0x7fec50142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec4de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec500cf7d0 0x7fec50142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34942/0 (socket says 172.21.15.26:34942) 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec4de00640 1 -- 172.21.15.26:0/1377044112 learned_addr learned my addr 172.21.15.26:0/1377044112 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec4de00640 1 -- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec5015e1e0 msgr2=0x7fec5006c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec45200640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec50143ae0 0x7fec5006c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec4de00640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec5015e1e0 0x7fec5006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec4de00640 1 -- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec50143ae0 msgr2=0x7fec5006c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec4de00640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec50143ae0 0x7fec5006c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.495+0000 7fec4de00640 1 -- 172.21.15.26:0/1377044112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec48009b30 con 0x7fec500cf7d0 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.496+0000 7fec4de00640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec500cf7d0 0x7fec50142cb0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fec4000d800 tx=0x7fec4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.496+0000 7fec46a00640 1 -- 172.21.15.26:0/1377044112 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4000b6c0 con 0x7fec500cf7d0 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.496+0000 7fec46a00640 1 -- 172.21.15.26:0/1377044112 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec4000b820 con 0x7fec500cf7d0 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.496+0000 7fec46a00640 1 -- 172.21.15.26:0/1377044112 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4000bbb0 con 0x7fec500cf7d0 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.496+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec50161c50 con 0x7fec500cf7d0 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.496+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec501431f0 con 0x7fec500cf7d0 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.498+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec500675d0 con 0x7fec500cf7d0 2024-01-15T18:09:25.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.499+0000 7fec46a00640 1 -- 172.21.15.26:0/1377044112 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7fec4000bd10 con 0x7fec500cf7d0 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.499+0000 7fec46a00640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec2806c1b0 0x7fec2806e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.500+0000 7fec46a00640 1 -- 172.21.15.26:0/1377044112 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fec40088ef0 con 0x7fec500cf7d0 2024-01-15T18:09:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.500+0000 7fec45200640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec2806c1b0 0x7fec2806e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.501+0000 7fec45200640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec2806c1b0 0x7fec2806e670 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fec38006fd0 tx=0x7fec38008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:25.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.504+0000 7fec46a00640 1 -- 172.21.15.26:0/1377044112 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec400554d0 con 0x7fec500cf7d0 2024-01-15T18:09:25.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.735+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fec5015e640 con 0x7fec2806c1b0 2024-01-15T18:09:25.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.751+0000 7fec46a00640 1 -- 172.21.15.26:0/1377044112 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fec3800c040 con 0x7fec2806c1b0 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:25.755 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:25.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec2806c1b0 msgr2=0x7fec2806e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:25.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec2806c1b0 0x7fec2806e670 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fec38006fd0 tx=0x7fec38008040 comp rx=0 tx=0).stop 2024-01-15T18:09:25.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec500cf7d0 msgr2=0x7fec50142cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:25.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec500cf7d0 0x7fec50142cb0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fec4000d800 tx=0x7fec4000dcc0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 shutdown_connections 2024-01-15T18:09:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec5015e1e0 0x7fec5006c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec2806c1b0 0x7fec2806e670 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec50143ae0 0x7fec5006c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 --2- 172.21.15.26:0/1377044112 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec500cf7d0 0x7fec50142cb0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.753+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 >> 172.21.15.26:0/1377044112 conn(0x7fec5006a800 msgr2=0x7fec5015c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.754+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 shutdown_connections 2024-01-15T18:09:25.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:25.754+0000 7fec55200640 1 -- 172.21.15.26:0/1377044112 wait complete. 2024-01-15T18:09:25.775 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:09:26.777 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.006+0000 7f3da0800640 1 Processor -- start 2024-01-15T18:09:27.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.006+0000 7f3da0800640 1 -- start start 2024-01-15T18:09:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.008+0000 7f3da0800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d9813df80 0x7f3d9815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.008+0000 7f3da0800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9813e360 0x7f3d980cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.008+0000 7f3da0800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9806c470 con 0x7f3d9815f9c0 2024-01-15T18:09:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.008+0000 7f3da0800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9806c750 con 0x7f3d9813df80 2024-01-15T18:09:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.008+0000 7f3da0800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9806ca30 con 0x7f3d9813e360 2024-01-15T18:09:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.008+0000 7f3d9de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d9813df80 0x7f3d9815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.009+0000 7f3d9de00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3d9813df80 0x7f3d9815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34948/0 (socket says 172.21.15.26:34948) 2024-01-15T18:09:27.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.009+0000 7f3d9de00640 1 -- 172.21.15.26:0/3961206152 learned_addr learned my addr 172.21.15.26:0/3961206152 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.009+0000 7f3d9de00640 1 -- 172.21.15.26:0/3961206152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9813e360 msgr2=0x7f3d980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.009+0000 7f3d9d400640 1 --2- 172.21.15.26:0/3961206152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9813e360 0x7f3d980cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.009+0000 7f3d9de00640 1 --2- 172.21.15.26:0/3961206152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9813e360 0x7f3d980cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.009+0000 7f3d9de00640 1 -- 172.21.15.26:0/3961206152 >> v1:172.21.15.124:6789/0 conn(0x7f3d9815f9c0 legacy=0x7f3d981408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:27.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.009+0000 7f3d9de00640 1 -- 172.21.15.26:0/3961206152 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9806cd10 con 0x7f3d9813df80 2024-01-15T18:09:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.010+0000 7f3d9de00640 1 --2- 172.21.15.26:0/3961206152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813df80 0x7f3d9815bee0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009870 tx=0x7f3d8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c042dfbbcbea3d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3d9ca00640 1 -- 172.21.15.26:0/3961206152 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88031640 con 0x7f3d9813df80 2024-01-15T18:09:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3d9ca00640 1 -- 172.21.15.26:0/3961206152 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d880317a0 con 0x7f3d9813df80 2024-01-15T18:09:27.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3d9ca00640 1 -- 172.21.15.26:0/3961206152 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88031b30 con 0x7f3d9813df80 2024-01-15T18:09:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3da0800640 1 -- 172.21.15.26:0/3961206152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813df80 msgr2=0x7f3d9815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3da0800640 1 --2- 172.21.15.26:0/3961206152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813df80 0x7f3d9815bee0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009870 tx=0x7f3d8802b8f0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3da0800640 1 -- 172.21.15.26:0/3961206152 shutdown_connections 2024-01-15T18:09:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3da0800640 1 --2- 172.21.15.26:0/3961206152 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9813e360 0x7f3d980cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3da0800640 1 --2- 172.21.15.26:0/3961206152 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813df80 0x7f3d9815bee0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3da0800640 1 -- 172.21.15.26:0/3961206152 >> 172.21.15.26:0/3961206152 conn(0x7f3d9806a800 msgr2=0x7f3d9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.011+0000 7f3da0800640 1 -- 172.21.15.26:0/3961206152 shutdown_connections 2024-01-15T18:09:27.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.012+0000 7f3da0800640 1 -- 172.21.15.26:0/3961206152 wait complete. 2024-01-15T18:09:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.013+0000 7f3da0800640 1 Processor -- start 2024-01-15T18:09:27.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.013+0000 7f3da0800640 1 -- start start 2024-01-15T18:09:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.014+0000 7f3da0800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d9813df80 0x7f3d980b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.014+0000 7f3da0800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813e360 0x7f3d981501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:27.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.014+0000 7f3da0800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9815f9c0 0x7f3d98150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.014+0000 7f3da0800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d98153e10 con 0x7f3d9813e360 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.014+0000 7f3da0800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d98150c50 con 0x7f3d9813df80 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.014+0000 7f3da0800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d98153190 con 0x7f3d9815f9c0 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813e360 0x7f3d981501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813e360 0x7f3d981501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34952/0 (socket says 172.21.15.26:34952) 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9d400640 1 -- 172.21.15.26:0/540515461 learned_addr learned my addr 172.21.15.26:0/540515461 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9de00640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d9813df80 0x7f3d980b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9e800640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9815f9c0 0x7f3d98150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9d400640 1 -- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9815f9c0 msgr2=0x7f3d98150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9d400640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9815f9c0 0x7f3d98150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9d400640 1 -- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d9813df80 msgr2=0x7f3d980b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9d400640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d9813df80 0x7f3d980b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9d400640 1 -- 172.21.15.26:0/540515461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880094d0 con 0x7f3d9813e360 2024-01-15T18:09:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.015+0000 7f3d9de00640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d9813df80 0x7f3d980b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.016+0000 7f3d9d400640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813e360 0x7f3d981501d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c7f0 tx=0x7f3d8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.016+0000 7f3d96000640 1 -- 172.21.15.26:0/540515461 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80004220 con 0x7f3d9813e360 2024-01-15T18:09:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.016+0000 7f3d96000640 1 -- 172.21.15.26:0/540515461 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d800043a0 con 0x7f3d9813e360 2024-01-15T18:09:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.016+0000 7f3d96000640 1 -- 172.21.15.26:0/540515461 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80007860 con 0x7f3d9813e360 2024-01-15T18:09:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.016+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d98153470 con 0x7f3d9813e360 2024-01-15T18:09:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.016+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d98153a30 con 0x7f3d9813e360 2024-01-15T18:09:27.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.017+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d98153f50 con 0x7f3d9813e360 2024-01-15T18:09:27.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.018+0000 7f3d96000640 1 -- 172.21.15.26:0/540515461 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f3d80007bd0 con 0x7f3d9813e360 2024-01-15T18:09:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.019+0000 7f3d96000640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d7006c160 0x7f3d7006e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.019+0000 7f3d96000640 1 -- 172.21.15.26:0/540515461 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f3d80088910 con 0x7f3d9813e360 2024-01-15T18:09:27.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.020+0000 7f3d9de00640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d7006c160 0x7f3d7006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:27.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.021+0000 7f3d9de00640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d7006c160 0x7f3d7006e620 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3d88036000 tx=0x7f3d8803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:27.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.026+0000 7f3d96000640 1 -- 172.21.15.26:0/540515461 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d98153f50 con 0x7f3d9813e360 2024-01-15T18:09:27.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.266+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d980ce850 con 0x7f3d7006c160 2024-01-15T18:09:27.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.280+0000 7f3d96000640 1 -- 172.21.15.26:0/540515461 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f3d88031a90 con 0x7f3d7006c160 2024-01-15T18:09:27.284 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:27.284 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:27.284 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:09:27.284 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:27.284 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:27.284 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:27.284 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:27.284 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:27.285 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:27.285 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d7006c160 msgr2=0x7f3d7006e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d7006c160 0x7f3d7006e620 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3d88036000 tx=0x7f3d8803c040 comp rx=0 tx=0).stop 2024-01-15T18:09:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813e360 msgr2=0x7f3d981501d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813e360 0x7f3d981501d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c7f0 tx=0x7f3d8000ccb0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 shutdown_connections 2024-01-15T18:09:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3d9815f9c0 0x7f3d98150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3d7006c160 0x7f3d7006e620 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3d9813e360 0x7f3d981501d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 --2- 172.21.15.26:0/540515461 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3d9813df80 0x7f3d980b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:27.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 >> 172.21.15.26:0/540515461 conn(0x7f3d9806a800 msgr2=0x7f3d98070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:27.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.283+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 shutdown_connections 2024-01-15T18:09:27.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:27.284+0000 7f3da0800640 1 -- 172.21.15.26:0/540515461 wait complete. 2024-01-15T18:09:27.306 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:09:28.307 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.535+0000 7f1f30200640 1 Processor -- start 2024-01-15T18:09:28.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.535+0000 7f1f30200640 1 -- start start 2024-01-15T18:09:28.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.537+0000 7f1f30200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cd240 0x7f1f280b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.537+0000 7f1f30200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f2806c6b0 0x7f1f28099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.537+0000 7f1f30200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2806cea0 con 0x7f1f28099fa0 2024-01-15T18:09:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.537+0000 7f1f30200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2809a430 con 0x7f1f2806c6b0 2024-01-15T18:09:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.537+0000 7f1f30200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2806cbc0 con 0x7f1f280cd240 2024-01-15T18:09:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.537+0000 7f1f2ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f2806c6b0 0x7f1f28099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.538+0000 7f1f2ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f2806c6b0 0x7f1f28099a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34968/0 (socket says 172.21.15.26:34968) 2024-01-15T18:09:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.538+0000 7f1f2ce00640 1 -- 172.21.15.26:0/675563053 learned_addr learned my addr 172.21.15.26:0/675563053 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.538+0000 7f1f2d800640 1 --2- 172.21.15.26:0/675563053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cd240 0x7f1f280b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.538+0000 7f1f2ce00640 1 -- 172.21.15.26:0/675563053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cd240 msgr2=0x7f1f280b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.538+0000 7f1f2ce00640 1 --2- 172.21.15.26:0/675563053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cd240 0x7f1f280b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.538+0000 7f1f2ce00640 1 -- 172.21.15.26:0/675563053 >> v1:172.21.15.26:6789/0 conn(0x7f1f28099fa0 legacy=0x7f1f2806b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.538+0000 7f1f2ce00640 1 -- 172.21.15.26:0/675563053 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2806f350 con 0x7f1f2806c6b0 2024-01-15T18:09:28.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.538+0000 7f1f2d800640 1 --2- 172.21.15.26:0/675563053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cd240 0x7f1f280b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.539+0000 7f1f2ce00640 1 --2- 172.21.15.26:0/675563053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f2806c6b0 0x7f1f28099a60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009870 tx=0x7f1f1002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a340ea9568d6670 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.540+0000 7f1f27e00640 1 -- 172.21.15.26:0/675563053 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10030840 con 0x7f1f2806c6b0 2024-01-15T18:09:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.540+0000 7f1f27e00640 1 -- 172.21.15.26:0/675563053 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f100309a0 con 0x7f1f2806c6b0 2024-01-15T18:09:28.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.540+0000 7f1f27e00640 1 -- 172.21.15.26:0/675563053 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10030d30 con 0x7f1f2806c6b0 2024-01-15T18:09:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.540+0000 7f1f30200640 1 -- 172.21.15.26:0/675563053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f2806c6b0 msgr2=0x7f1f28099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.540+0000 7f1f30200640 1 --2- 172.21.15.26:0/675563053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f2806c6b0 0x7f1f28099a60 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009870 tx=0x7f1f1002b980 comp rx=0 tx=0).stop 2024-01-15T18:09:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.541+0000 7f1f30200640 1 -- 172.21.15.26:0/675563053 shutdown_connections 2024-01-15T18:09:28.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.541+0000 7f1f30200640 1 --2- 172.21.15.26:0/675563053 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f2806c6b0 0x7f1f28099a60 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.541+0000 7f1f30200640 1 --2- 172.21.15.26:0/675563053 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f280cd240 0x7f1f280b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.541+0000 7f1f30200640 1 -- 172.21.15.26:0/675563053 >> 172.21.15.26:0/675563053 conn(0x7f1f280b2d00 msgr2=0x7f1f280b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.541+0000 7f1f30200640 1 -- 172.21.15.26:0/675563053 shutdown_connections 2024-01-15T18:09:28.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.541+0000 7f1f30200640 1 -- 172.21.15.26:0/675563053 wait complete. 2024-01-15T18:09:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.542+0000 7f1f30200640 1 Processor -- start 2024-01-15T18:09:28.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.543+0000 7f1f30200640 1 -- start start 2024-01-15T18:09:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f30200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2806c6b0 0x7f1f280cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f30200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f28099fa0 0x7f1f280b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f30200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f280cd240 0x7f1f280c5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f30200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280b45a0 con 0x7f1f280cd240 2024-01-15T18:09:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f30200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f281f2b30 con 0x7f1f2806c6b0 2024-01-15T18:09:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f30200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f280b47d0 con 0x7f1f28099fa0 2024-01-15T18:09:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f280cd240 0x7f1f280c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f280cd240 0x7f1f280c5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34972/0 (socket says 172.21.15.26:34972) 2024-01-15T18:09:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2e200640 1 -- 172.21.15.26:0/813910025 learned_addr learned my addr 172.21.15.26:0/813910025 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2ce00640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f28099fa0 0x7f1f280b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2d800640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2806c6b0 0x7f1f280cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2e200640 1 -- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f28099fa0 msgr2=0x7f1f280b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2e200640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f28099fa0 0x7f1f280b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2e200640 1 -- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2806c6b0 msgr2=0x7f1f280cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2e200640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2806c6b0 0x7f1f280cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.544+0000 7f1f2e200640 1 -- 172.21.15.26:0/813910025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100094d0 con 0x7f1f280cd240 2024-01-15T18:09:28.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.545+0000 7f1f2e200640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f280cd240 0x7f1f280c5200 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c390 tx=0x7f1f2000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.545+0000 7f1f25600640 1 -- 172.21.15.26:0/813910025 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200052d0 con 0x7f1f280cd240 2024-01-15T18:09:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.545+0000 7f1f25600640 1 -- 172.21.15.26:0/813910025 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f20005430 con 0x7f1f280cd240 2024-01-15T18:09:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.545+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f280b4ab0 con 0x7f1f280cd240 2024-01-15T18:09:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.545+0000 7f1f25600640 1 -- 172.21.15.26:0/813910025 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20003810 con 0x7f1f280cd240 2024-01-15T18:09:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.545+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f280b5070 con 0x7f1f280cd240 2024-01-15T18:09:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.546+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f280675d0 con 0x7f1f280cd240 2024-01-15T18:09:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.548+0000 7f1f25600640 1 -- 172.21.15.26:0/813910025 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f1f20003970 con 0x7f1f280cd240 2024-01-15T18:09:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.548+0000 7f1f25600640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f0006c160 0x7f1f0006e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.549+0000 7f1f25600640 1 -- 172.21.15.26:0/813910025 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f1f20087eb0 con 0x7f1f280cd240 2024-01-15T18:09:28.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.549+0000 7f1f2d800640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f0006c160 0x7f1f0006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:28.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.550+0000 7f1f2d800640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f0006c160 0x7f1f0006e620 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1f18003fd0 tx=0x7f1f18003ce0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:28.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.553+0000 7f1f25600640 1 -- 172.21.15.26:0/813910025 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f20054490 con 0x7f1f280cd240 2024-01-15T18:09:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.783+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f280cd620 con 0x7f1f0006c160 2024-01-15T18:09:28.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.799+0000 7f1f25600640 1 -- 172.21.15.26:0/813910025 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7f1f18003a90 con 0x7f1f0006c160 2024-01-15T18:09:28.802 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:09:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:28.803 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:28.803 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f0006c160 msgr2=0x7f1f0006e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f0006c160 0x7f1f0006e620 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1f18003fd0 tx=0x7f1f18003ce0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f280cd240 msgr2=0x7f1f280c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:28.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f280cd240 0x7f1f280c5200 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f1f2000c390 tx=0x7f1f2000c850 comp rx=0 tx=0).stop 2024-01-15T18:09:28.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 shutdown_connections 2024-01-15T18:09:28.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f28099fa0 0x7f1f280b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f0006c160 0x7f1f0006e620 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f280cd240 0x7f1f280c5200 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 --2- 172.21.15.26:0/813910025 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f2806c6b0 0x7f1f280cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:28.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 >> 172.21.15.26:0/813910025 conn(0x7f1f280b2d00 msgr2=0x7f1f2806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:28.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.801+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 shutdown_connections 2024-01-15T18:09:28.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:28.802+0000 7f1f30200640 1 -- 172.21.15.26:0/813910025 wait complete. 2024-01-15T18:09:28.823 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:09:29.824 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:30.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.044+0000 7fc16b400640 1 Processor -- start 2024-01-15T18:09:30.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.044+0000 7fc16b400640 1 -- start start 2024-01-15T18:09:30.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.045+0000 7fc16b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc16415ec60 0x7fc1640cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.046+0000 7fc16b400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc16415f040 0x7fc1640cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.046+0000 7fc16b400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1640cc1a0 con 0x7fc16415ab00 2024-01-15T18:09:30.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.046+0000 7fc16b400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1640cf2f0 con 0x7fc16415f040 2024-01-15T18:09:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.046+0000 7fc16b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640cc480 con 0x7fc16415ec60 2024-01-15T18:09:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.046+0000 7fc16aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc16415ab00 0x7fc16415af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:40952/0 (socket says 172.21.15.26:40952) 2024-01-15T18:09:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.046+0000 7fc16aa00640 1 -- 172.21.15.26:0/2825186908 learned_addr learned my addr 172.21.15.26:0/2825186908 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.046+0000 7fc16a000640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc16415ec60 0x7fc1640cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.046+0000 7fc169600640 1 --2- 172.21.15.26:0/2825186908 >> v2:172.21.15.124:3300/0 conn(0x7fc16415f040 0x7fc1640cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848403573 0 0) 0x7fc1640cc1a0 con 0x7fc16415ab00 2024-01-15T18:09:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc144003520 con 0x7fc16415ab00 2024-01-15T18:09:30.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1082260069 0 0) 0x7fc144003520 con 0x7fc16415ab00 2024-01-15T18:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc16415ec60 msgr2=0x7fc1640cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc16415ec60 0x7fc1640cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 >> v2:172.21.15.124:3300/0 conn(0x7fc16415f040 msgr2=0x7fc1640cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 --2- 172.21.15.26:0/2825186908 >> v2:172.21.15.124:3300/0 conn(0x7fc16415f040 0x7fc1640cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1640cc760 con 0x7fc16415ab00 2024-01-15T18:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fc15c001670 con 0x7fc16415ab00 2024-01-15T18:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 >> v1:172.21.15.26:6789/0 conn(0x7fc16415ab00 legacy=0x7fc16415af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc144003970 0x7fc144003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc16415ec60 0x7fc1440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc144004d00 0x7fc144005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640cf2f0 con 0x7fc144003970 2024-01-15T18:09:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640cc480 con 0x7fc144004d00 2024-01-15T18:09:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14400b980 con 0x7fc16415ec60 2024-01-15T18:09:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.047+0000 7fc16a000640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc144003970 0x7fc144003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.048+0000 7fc16a000640 1 -- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc16415ec60 msgr2=0x7fc1440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:30.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.048+0000 7fc16a000640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc16415ec60 0x7fc1440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.048+0000 7fc16aa00640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc144004d00 0x7fc144005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.048+0000 7fc16a000640 1 -- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc144004d00 msgr2=0x7fc144005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.048+0000 7fc16a000640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc144004d00 0x7fc144005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.048+0000 7fc16a000640 1 -- 172.21.15.26:0/2825186908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc144003520 con 0x7fc144003970 2024-01-15T18:09:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.049+0000 7fc16a000640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc144003970 0x7fc144003d90 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009ac0 tx=0x7fc1440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddacead9902ce4e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.049+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c02c710 con 0x7fc144003970 2024-01-15T18:09:30.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.049+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c02c870 con 0x7fc144003970 2024-01-15T18:09:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.049+0000 7fc168c00640 1 -- 172.21.15.26:0/2825186908 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c02cc00 con 0x7fc144003970 2024-01-15T18:09:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.050+0000 7fc16b400640 1 -- 172.21.15.26:0/2825186908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc144003970 msgr2=0x7fc144003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.050+0000 7fc16b400640 1 --2- 172.21.15.26:0/2825186908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc144003970 0x7fc144003d90 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009ac0 tx=0x7fc1440057f0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.050+0000 7fc16aa00640 1 -- 172.21.15.26:0/2825186908 reap_dead start 2024-01-15T18:09:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.050+0000 7fc16b400640 1 -- 172.21.15.26:0/2825186908 shutdown_connections 2024-01-15T18:09:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.050+0000 7fc16b400640 1 -- 172.21.15.26:0/2825186908 >> 172.21.15.26:0/2825186908 conn(0x7fc1640d0eb0 msgr2=0x7fc1640d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.050+0000 7fc16b400640 1 -- 172.21.15.26:0/2825186908 shutdown_connections 2024-01-15T18:09:30.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.050+0000 7fc16b400640 1 -- 172.21.15.26:0/2825186908 wait complete. 2024-01-15T18:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.052+0000 7fc16b400640 1 Processor -- start 2024-01-15T18:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.052+0000 7fc16b400640 1 -- start start 2024-01-15T18:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.053+0000 7fc16b400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc144003970 0x7fc164159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.053+0000 7fc16b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc164159ce0 0x7fc16415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1641f2ef0 0x7fc1641f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641e4470 con 0x7fc1641f2ef0 2024-01-15T18:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16b400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641e46c0 con 0x7fc144003970 2024-01-15T18:09:30.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1641e49a0 con 0x7fc164159ce0 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1641f2ef0 0x7fc1641f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1641f2ef0 0x7fc1641f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34988/0 (socket says 172.21.15.26:34988) 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16aa00640 1 -- 172.21.15.26:0/793106629 learned_addr learned my addr 172.21.15.26:0/793106629 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16a000640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc144003970 0x7fc164159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc169600640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc164159ce0 0x7fc16415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16aa00640 1 -- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc164159ce0 msgr2=0x7fc16415a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16aa00640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc164159ce0 0x7fc16415a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16aa00640 1 -- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc144003970 msgr2=0x7fc164159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16aa00640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc144003970 0x7fc164159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.054+0000 7fc16aa00640 1 -- 172.21.15.26:0/793106629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1641e4c80 con 0x7fc1641f2ef0 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.055+0000 7fc16aa00640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1641f2ef0 0x7fc1641f3380 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc164071690 tx=0x7fc15c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.055+0000 7fc162000640 1 -- 172.21.15.26:0/793106629 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c0074e0 con 0x7fc1641f2ef0 2024-01-15T18:09:30.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.055+0000 7fc162000640 1 -- 172.21.15.26:0/793106629 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c007660 con 0x7fc1641f2ef0 2024-01-15T18:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.055+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1641e4f60 con 0x7fc1641f2ef0 2024-01-15T18:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.055+0000 7fc162000640 1 -- 172.21.15.26:0/793106629 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c004a20 con 0x7fc1641f2ef0 2024-01-15T18:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.055+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc164159980 con 0x7fc1641f2ef0 2024-01-15T18:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.056+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1641e5480 con 0x7fc1641f2ef0 2024-01-15T18:09:30.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.057+0000 7fc162000640 1 -- 172.21.15.26:0/793106629 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7fc15c004b80 con 0x7fc1641f2ef0 2024-01-15T18:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.058+0000 7fc162000640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc13c06c160 0x7fc13c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.058+0000 7fc162000640 1 -- 172.21.15.26:0/793106629 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fc15c0886b0 con 0x7fc1641f2ef0 2024-01-15T18:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.059+0000 7fc16a000640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc13c06c160 0x7fc13c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.060+0000 7fc16a000640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc13c06c160 0x7fc13c06e620 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc14c036000 tx=0x7fc14c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:30.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.063+0000 7fc162000640 1 -- 172.21.15.26:0/793106629 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc1641e5480 con 0x7fc1641f2ef0 2024-01-15T18:09:30.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.296+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc164159b40 con 0x7fc13c06c160 2024-01-15T18:09:30.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.311+0000 7fc162000640 1 -- 172.21.15.26:0/793106629 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fc14c02ca60 con 0x7fc13c06c160 2024-01-15T18:09:30.315 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:30.315 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:09:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:30.316 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:30.316 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:30.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc13c06c160 msgr2=0x7fc13c06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc13c06c160 0x7fc13c06e620 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc14c036000 tx=0x7fc14c030000 comp rx=0 tx=0).stop 2024-01-15T18:09:30.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1641f2ef0 msgr2=0x7fc1641f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:30.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1641f2ef0 0x7fc1641f3380 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fc164071690 tx=0x7fc15c00d9d0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 shutdown_connections 2024-01-15T18:09:30.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc164159ce0 0x7fc16415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc13c06c160 0x7fc13c06e620 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1641f2ef0 0x7fc1641f3380 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 --2- 172.21.15.26:0/793106629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc144003970 0x7fc164159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 >> 172.21.15.26:0/793106629 conn(0x7fc1640d0eb0 msgr2=0x7fc1640b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 shutdown_connections 2024-01-15T18:09:30.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:30.314+0000 7fc16b400640 1 -- 172.21.15.26:0/793106629 wait complete. 2024-01-15T18:09:30.337 INFO:tasks.cephfs.test_volumes:current clone state = pending 2024-01-15T18:09:30.681 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:09:30.689 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:09:31.339 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.568+0000 7f6e49000640 1 Processor -- start 2024-01-15T18:09:31.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.569+0000 7f6e49000640 1 -- start start 2024-01-15T18:09:31.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.571+0000 7f6e49000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6e44143810 0x7f6e44071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:31.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.571+0000 7f6e49000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4415d770 0x7f6e4415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.571+0000 7f6e49000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440ae620 con 0x7f6e440ae170 2024-01-15T18:09:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.571+0000 7f6e49000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e44143bf0 con 0x7f6e44143810 2024-01-15T18:09:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.571+0000 7f6e49000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415e140 con 0x7f6e4415d770 2024-01-15T18:09:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.571+0000 7f6e43400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6e44143810 0x7f6e44071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.571+0000 7f6e43e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4415d770 0x7f6e4415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:31.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.572+0000 7f6e43e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4415d770 0x7f6e4415dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:45768/0 (socket says 172.21.15.26:45768) 2024-01-15T18:09:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.572+0000 7f6e43e00640 1 -- 172.21.15.26:0/815068941 learned_addr learned my addr 172.21.15.26:0/815068941 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.572+0000 7f6e43400640 1 -- 172.21.15.26:0/815068941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4415d770 msgr2=0x7f6e4415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.572+0000 7f6e43400640 1 --2- 172.21.15.26:0/815068941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4415d770 0x7f6e4415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.572+0000 7f6e43400640 1 -- 172.21.15.26:0/815068941 >> v1:172.21.15.124:6789/0 conn(0x7f6e440ae170 legacy=0x7f6e4415e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:31.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.572+0000 7f6e43400640 1 -- 172.21.15.26:0/815068941 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4406c730 con 0x7f6e44143810 2024-01-15T18:09:31.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.574+0000 7f6e43400640 1 --2- 172.21.15.26:0/815068941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e44143810 0x7f6e44071e40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097a0 tx=0x7f6e3402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6bcbce6c73b4659 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.574+0000 7f6e42000640 1 -- 172.21.15.26:0/815068941 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340316f0 con 0x7f6e44143810 2024-01-15T18:09:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.575+0000 7f6e42000640 1 -- 172.21.15.26:0/815068941 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e34031850 con 0x7f6e44143810 2024-01-15T18:09:31.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.575+0000 7f6e42000640 1 -- 172.21.15.26:0/815068941 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34031be0 con 0x7f6e44143810 2024-01-15T18:09:31.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.575+0000 7f6e49000640 1 -- 172.21.15.26:0/815068941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e44143810 msgr2=0x7f6e44071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:31.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.575+0000 7f6e49000640 1 --2- 172.21.15.26:0/815068941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e44143810 0x7f6e44071e40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6e340097a0 tx=0x7f6e3402b790 comp rx=0 tx=0).stop 2024-01-15T18:09:31.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.575+0000 7f6e49000640 1 -- 172.21.15.26:0/815068941 shutdown_connections 2024-01-15T18:09:31.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.575+0000 7f6e49000640 1 --2- 172.21.15.26:0/815068941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e4415d770 0x7f6e4415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:31.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.575+0000 7f6e49000640 1 --2- 172.21.15.26:0/815068941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e44143810 0x7f6e44071e40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:31.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.575+0000 7f6e49000640 1 -- 172.21.15.26:0/815068941 >> 172.21.15.26:0/815068941 conn(0x7f6e4406a7d0 msgr2=0x7f6e4406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:31.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.576+0000 7f6e49000640 1 -- 172.21.15.26:0/815068941 shutdown_connections 2024-01-15T18:09:31.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.576+0000 7f6e49000640 1 -- 172.21.15.26:0/815068941 wait complete. 2024-01-15T18:09:31.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.578+0000 7f6e49000640 1 Processor -- start 2024-01-15T18:09:31.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.578+0000 7f6e49000640 1 -- start start 2024-01-15T18:09:31.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e49000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e440ae170 0x7f6e4413d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:31.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e49000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e44143810 0x7f6e441452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:31.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e49000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4415d770 0x7f6e44145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e49000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440cb680 con 0x7f6e440ae170 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e49000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440cb960 con 0x7f6e4415d770 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e49000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440cc000 con 0x7f6e44143810 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e43400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e440ae170 0x7f6e4413d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e43400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e440ae170 0x7f6e4413d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:35004/0 (socket says 172.21.15.26:35004) 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e43400640 1 -- 172.21.15.26:0/423043579 learned_addr learned my addr 172.21.15.26:0/423043579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e43e00640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4415d770 0x7f6e44145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.580+0000 7f6e42a00640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e44143810 0x7f6e441452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e43400640 1 -- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e44143810 msgr2=0x7f6e441452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e43400640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e44143810 0x7f6e441452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e43400640 1 -- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4415d770 msgr2=0x7f6e44145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e43400640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4415d770 0x7f6e44145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:31.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e43400640 1 -- 172.21.15.26:0/423043579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e34009400 con 0x7f6e440ae170 2024-01-15T18:09:31.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e42a00640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e44143810 0x7f6e441452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:31.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e43400640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e440ae170 0x7f6e4413d1f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098c0 tx=0x7f6e340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:31.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e3b400640 1 -- 172.21.15.26:0/423043579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34034480 con 0x7f6e440ae170 2024-01-15T18:09:31.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e3b400640 1 -- 172.21.15.26:0/423043579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e34034600 con 0x7f6e440ae170 2024-01-15T18:09:31.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e3b400640 1 -- 172.21.15.26:0/423043579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340349b0 con 0x7f6e440ae170 2024-01-15T18:09:31.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.581+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e440cc2e0 con 0x7f6e440ae170 2024-01-15T18:09:31.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.582+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4413e120 con 0x7f6e440ae170 2024-01-15T18:09:31.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.583+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e440cbb80 con 0x7f6e440ae170 2024-01-15T18:09:31.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.584+0000 7f6e3b400640 1 -- 172.21.15.26:0/423043579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f6e3403b040 con 0x7f6e440ae170 2024-01-15T18:09:31.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.585+0000 7f6e3b400640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e1c06c160 0x7f6e1c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:31.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.585+0000 7f6e3b400640 1 -- 172.21.15.26:0/423043579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f6e340aa190 con 0x7f6e440ae170 2024-01-15T18:09:31.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.586+0000 7f6e42a00640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e1c06c160 0x7f6e1c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:31.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.587+0000 7f6e42a00640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e1c06c160 0x7f6e1c06e620 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6e441f6670 tx=0x7f6e2c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:31.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.591+0000 7f6e3b400640 1 -- 172.21.15.26:0/423043579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e340768f0 con 0x7f6e440ae170 2024-01-15T18:09:31.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:31.826+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e4413e310 con 0x7f6e1c06c160 2024-01-15T18:09:33.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.499+0000 7f6e3b400640 1 -- 172.21.15.26:0/423043579 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f6e2c001340 con 0x7f6e1c06c160 2024-01-15T18:09:33.503 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:09:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:33.503 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:33.504 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:33.504 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.502+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e1c06c160 msgr2=0x7f6e1c06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.502+0000 7f6e49000640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e1c06c160 0x7f6e1c06e620 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6e441f6670 tx=0x7f6e2c009040 comp rx=0 tx=0).stop 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e440ae170 msgr2=0x7f6e4413d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e440ae170 0x7f6e4413d1f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f6e340098c0 tx=0x7f6e340046c0 comp rx=0 tx=0).stop 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 shutdown_connections 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6e44143810 0x7f6e441452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6e1c06c160 0x7f6e1c06e620 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6e4415d770 0x7f6e44145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 --2- 172.21.15.26:0/423043579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6e440ae170 0x7f6e4413d1f0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 >> 172.21.15.26:0/423043579 conn(0x7f6e4406a7d0 msgr2=0x7f6e44142ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 shutdown_connections 2024-01-15T18:09:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:33.503+0000 7f6e49000640 1 -- 172.21.15.26:0/423043579 wait complete. 2024-01-15T18:09:35.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:35.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.792+0000 7fd091a00640 1 Processor -- start 2024-01-15T18:09:35.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.792+0000 7fd091a00640 1 -- start start 2024-01-15T18:09:35.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.794+0000 7fd091a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0cf9a0 0x7fd08c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:35.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.794+0000 7fd091a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd08c15e1e0 0x7fd08c0702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:35.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.795+0000 7fd091a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08c0b1ee0 con 0x7fd08c143ae0 2024-01-15T18:09:35.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.795+0000 7fd091a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08c0b2780 con 0x7fd08c15e1e0 2024-01-15T18:09:35.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.795+0000 7fd091a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c15e990 con 0x7fd08c0cf9a0 2024-01-15T18:09:35.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.795+0000 7fd08aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd08c143ae0 0x7fd08c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49324/0 (socket says 172.21.15.26:49324) 2024-01-15T18:09:35.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.795+0000 7fd08aa00640 1 -- 172.21.15.26:0/1338735047 learned_addr learned my addr 172.21.15.26:0/1338735047 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:35.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.795+0000 7fd08a000640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0cf9a0 0x7fd08c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536605053 0 0) 0x7fd08c0b1ee0 con 0x7fd08c143ae0 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd06c003520 con 0x7fd08c143ae0 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd08b400640 1 --2- 172.21.15.26:0/1338735047 >> v2:172.21.15.124:3300/0 conn(0x7fd08c15e1e0 0x7fd08c0702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2706103949 0 0) 0x7fd06c003520 con 0x7fd08c143ae0 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0cf9a0 msgr2=0x7fd08c15dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0cf9a0 0x7fd08c15dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 >> v2:172.21.15.124:3300/0 conn(0x7fd08c15e1e0 msgr2=0x7fd08c0702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 --2- 172.21.15.26:0/1338735047 >> v2:172.21.15.124:3300/0 conn(0x7fd08c15e1e0 0x7fd08c0702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c0b2130 con 0x7fd08c143ae0 2024-01-15T18:09:35.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd07c0019f0 con 0x7fd08c143ae0 2024-01-15T18:09:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 >> v1:172.21.15.26:6789/0 conn(0x7fd08c143ae0 legacy=0x7fd08c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd06c003970 0x7fd06c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c0047c0 0x7fd06c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.796+0000 7fd089600640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0cf9a0 0x7fd06c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.797+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0b2780 con 0x7fd06c0047c0 2024-01-15T18:09:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.797+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c15e990 con 0x7fd06c003970 2024-01-15T18:09:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.797+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c00b9d0 con 0x7fd08c0cf9a0 2024-01-15T18:09:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.797+0000 7fd08aa00640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd06c003970 0x7fd06c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:35.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.797+0000 7fd08a000640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c0047c0 0x7fd06c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.798+0000 7fd08aa00640 1 -- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0cf9a0 msgr2=0x7fd06c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.798+0000 7fd08aa00640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0cf9a0 0x7fd06c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.798+0000 7fd08aa00640 1 -- 172.21.15.26:0/1338735047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c0047c0 msgr2=0x7fd06c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.798+0000 7fd08aa00640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c0047c0 0x7fd06c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.798+0000 7fd08aa00640 1 -- 172.21.15.26:0/1338735047 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c003520 con 0x7fd06c003970 2024-01-15T18:09:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.798+0000 7fd08a000640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c0047c0 0x7fd06c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:35.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.799+0000 7fd08aa00640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd06c003970 0x7fd06c003d90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00a760 tx=0x7fd06c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde04211e4a5c348 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:35.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.803+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c034630 con 0x7fd06c003970 2024-01-15T18:09:35.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.803+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c02f070 con 0x7fd06c003970 2024-01-15T18:09:35.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.803+0000 7fd089600640 1 -- 172.21.15.26:0/1338735047 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0349d0 con 0x7fd06c003970 2024-01-15T18:09:35.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.804+0000 7fd091a00640 1 -- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd06c003970 msgr2=0x7fd06c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:35.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.804+0000 7fd091a00640 1 --2- 172.21.15.26:0/1338735047 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd06c003970 0x7fd06c003d90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00a760 tx=0x7fd06c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:09:35.807 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.804+0000 7fd08b400640 1 -- 172.21.15.26:0/1338735047 reap_dead start 2024-01-15T18:09:35.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.804+0000 7fd091a00640 1 -- 172.21.15.26:0/1338735047 shutdown_connections 2024-01-15T18:09:35.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.804+0000 7fd091a00640 1 -- 172.21.15.26:0/1338735047 >> 172.21.15.26:0/1338735047 conn(0x7fd08c06a800 msgr2=0x7fd08c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:35.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.804+0000 7fd091a00640 1 -- 172.21.15.26:0/1338735047 shutdown_connections 2024-01-15T18:09:35.808 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.804+0000 7fd091a00640 1 -- 172.21.15.26:0/1338735047 wait complete. 2024-01-15T18:09:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.809+0000 7fd091a00640 1 Processor -- start 2024-01-15T18:09:35.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.810+0000 7fd091a00640 1 -- start start 2024-01-15T18:09:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.812+0000 7fd091a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0c49d0 0x7fd08c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.812+0000 7fd091a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd08c0c4db0 0x7fd08c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.812+0000 7fd091a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c155ca0 0x7fd08c156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.812+0000 7fd091a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c156690 con 0x7fd08c155ca0 2024-01-15T18:09:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.812+0000 7fd091a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0ce5f0 con 0x7fd08c0c4db0 2024-01-15T18:09:35.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.812+0000 7fd091a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c0c55c0 con 0x7fd08c0c49d0 2024-01-15T18:09:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.813+0000 7fd08b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c155ca0 0x7fd08c156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.813+0000 7fd08a000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd08c0c4db0 0x7fd08c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.813+0000 7fd08a000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd08c0c4db0 0x7fd08c0cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:48990/0 (socket says 172.21.15.26:48990) 2024-01-15T18:09:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.813+0000 7fd08a000640 1 -- 172.21.15.26:0/23638020 learned_addr learned my addr 172.21.15.26:0/23638020 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:35.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.813+0000 7fd08aa00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0c49d0 0x7fd08c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.814+0000 7fd08b400640 1 -- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0c49d0 msgr2=0x7fd08c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.814+0000 7fd08b400640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0c49d0 0x7fd08c0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.814+0000 7fd08b400640 1 -- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd08c0c4db0 msgr2=0x7fd08c0cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.814+0000 7fd08b400640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd08c0c4db0 0x7fd08c0cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.814+0000 7fd08b400640 1 -- 172.21.15.26:0/23638020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c0cd9e0 con 0x7fd08c155ca0 2024-01-15T18:09:35.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.814+0000 7fd08a000640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd08c0c4db0 0x7fd08c0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.815+0000 7fd08b400640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c155ca0 0x7fd08c156150 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0b1bf0 tx=0x7fd08400d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.815+0000 7fd082a00640 1 -- 172.21.15.26:0/23638020 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840125f0 con 0x7fd08c155ca0 2024-01-15T18:09:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.815+0000 7fd082a00640 1 -- 172.21.15.26:0/23638020 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084012770 con 0x7fd08c155ca0 2024-01-15T18:09:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.815+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c0cdcc0 con 0x7fd08c155ca0 2024-01-15T18:09:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.815+0000 7fd082a00640 1 -- 172.21.15.26:0/23638020 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084012b20 con 0x7fd08c155ca0 2024-01-15T18:09:35.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.816+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c0ce280 con 0x7fd08c155ca0 2024-01-15T18:09:35.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.817+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08c0ce780 con 0x7fd08c155ca0 2024-01-15T18:09:35.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.819+0000 7fd082a00640 1 -- 172.21.15.26:0/23638020 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7fd084012c80 con 0x7fd08c155ca0 2024-01-15T18:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.820+0000 7fd082a00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd06406c160 0x7fd06406e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.820+0000 7fd082a00640 1 -- 172.21.15.26:0/23638020 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fd084088550 con 0x7fd08c155ca0 2024-01-15T18:09:35.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.820+0000 7fd08aa00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd06406c160 0x7fd06406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:35.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.821+0000 7fd08aa00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd06406c160 0x7fd06406e620 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd07c034000 tx=0x7fd07c009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:35.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:35.827+0000 7fd082a00640 1 -- 172.21.15.26:0/23638020 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd084054b30 con 0x7fd08c155ca0 2024-01-15T18:09:36.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.177+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd08c15e640 con 0x7fd06406c160 2024-01-15T18:09:36.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.197+0000 7fd082a00640 1 -- 172.21.15.26:0/23638020 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+344 (secure 0 0 0) 0x7fd07c034430 con 0x7fd06406c160 2024-01-15T18:09:36.201 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:36.201 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:36.201 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:09:36.201 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:36.201 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:36.201 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:36.201 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 0.0, 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "0 B/3 GB", 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "0/3", 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:36.202 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:36.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.200+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd06406c160 msgr2=0x7fd06406e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:36.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.200+0000 7fd091a00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd06406c160 0x7fd06406e620 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fd07c034000 tx=0x7fd07c009030 comp rx=0 tx=0).stop 2024-01-15T18:09:36.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.200+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c155ca0 msgr2=0x7fd08c156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.200+0000 7fd091a00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c155ca0 0x7fd08c156150 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd08c0b1bf0 tx=0x7fd08400d820 comp rx=0 tx=0).stop 2024-01-15T18:09:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.201+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 shutdown_connections 2024-01-15T18:09:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.201+0000 7fd091a00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd06406c160 0x7fd06406e620 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.201+0000 7fd091a00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd08c155ca0 0x7fd08c156150 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.201+0000 7fd091a00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd08c0c4db0 0x7fd08c0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.201+0000 7fd091a00640 1 --2- 172.21.15.26:0/23638020 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd08c0c49d0 0x7fd08c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.201+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 >> 172.21.15.26:0/23638020 conn(0x7fd08c06a800 msgr2=0x7fd08c0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.201+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 shutdown_connections 2024-01-15T18:09:36.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:36.201+0000 7fd091a00640 1 -- 172.21.15.26:0/23638020 wait complete. 2024-01-15T18:09:36.232 INFO:tasks.cephfs.test_volumes:percetange completed = 0.0 2024-01-15T18:09:38.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:38.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.463+0000 7f9abec00640 1 Processor -- start 2024-01-15T18:09:38.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.463+0000 7f9abec00640 1 -- start start 2024-01-15T18:09:38.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9abec00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ab8143ae0 0x7f9ab80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:38.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9abec00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9ab80cf7d0 0x7f9ab815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:38.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9abec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab815e1e0 0x7f9ab815a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:38.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9abec00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab80cfc30 con 0x7f9ab8143ae0 2024-01-15T18:09:38.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9abec00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab80b1ee0 con 0x7f9ab80cf7d0 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9abec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80b2780 con 0x7f9ab815e1e0 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9ab7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ab8143ae0 0x7f9ab80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9ab7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ab8143ae0 0x7f9ab80cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58260/0 (socket says 172.21.15.26:58260) 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9ab7e00640 1 -- 172.21.15.26:0/3374495272 learned_addr learned my addr 172.21.15.26:0/3374495272 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9abcc00640 1 --2- 172.21.15.26:0/3374495272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab815e1e0 0x7f9ab815a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.465+0000 7f9aaf200640 1 --2- 172.21.15.26:0/3374495272 >> v2:172.21.15.124:3300/0 conn(0x7f9ab80cf7d0 0x7f9ab815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.466+0000 7f9ab7e00640 1 -- 172.21.15.26:0/3374495272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab815e1e0 msgr2=0x7f9ab815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.466+0000 7f9ab7e00640 1 --2- 172.21.15.26:0/3374495272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab815e1e0 0x7f9ab815a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.466+0000 7f9ab7e00640 1 -- 172.21.15.26:0/3374495272 >> v2:172.21.15.124:3300/0 conn(0x7f9ab80cf7d0 msgr2=0x7f9ab815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.466+0000 7f9ab7e00640 1 --2- 172.21.15.26:0/3374495272 >> v2:172.21.15.124:3300/0 conn(0x7f9ab80cf7d0 0x7f9ab815dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.466+0000 7f9ab7e00640 1 -- 172.21.15.26:0/3374495272 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab815e990 con 0x7f9ab8143ae0 2024-01-15T18:09:38.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.466+0000 7f9aaf200640 1 --2- 172.21.15.26:0/3374495272 >> v2:172.21.15.124:3300/0 conn(0x7f9ab80cf7d0 0x7f9ab815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.467+0000 7f9ab7e00640 1 --2- 172.21.15.26:0/3374495272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab8143ae0 0x7f9ab80cf290 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80cb830 tx=0x7f9aa802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12fbd774dc0d7bf2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.467+0000 7f9ab7400640 1 -- 172.21.15.26:0/3374495272 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8031860 con 0x7f9ab8143ae0 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.467+0000 7f9ab7400640 1 -- 172.21.15.26:0/3374495272 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa80319c0 con 0x7f9ab8143ae0 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.467+0000 7f9ab7400640 1 -- 172.21.15.26:0/3374495272 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8031d50 con 0x7f9ab8143ae0 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.468+0000 7f9abec00640 1 -- 172.21.15.26:0/3374495272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab8143ae0 msgr2=0x7f9ab80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.468+0000 7f9abec00640 1 --2- 172.21.15.26:0/3374495272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab8143ae0 0x7f9ab80cf290 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80cb830 tx=0x7f9aa802b900 comp rx=0 tx=0).stop 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.468+0000 7f9abec00640 1 -- 172.21.15.26:0/3374495272 shutdown_connections 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.468+0000 7f9abec00640 1 --2- 172.21.15.26:0/3374495272 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab815e1e0 0x7f9ab815a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.468+0000 7f9abec00640 1 --2- 172.21.15.26:0/3374495272 >> v2:172.21.15.124:3300/0 conn(0x7f9ab80cf7d0 0x7f9ab815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.468+0000 7f9abec00640 1 --2- 172.21.15.26:0/3374495272 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab8143ae0 0x7f9ab80cf290 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.468+0000 7f9abec00640 1 -- 172.21.15.26:0/3374495272 >> 172.21.15.26:0/3374495272 conn(0x7f9ab806a800 msgr2=0x7f9ab806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:38.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.468+0000 7f9abec00640 1 -- 172.21.15.26:0/3374495272 shutdown_connections 2024-01-15T18:09:38.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.468+0000 7f9abec00640 1 -- 172.21.15.26:0/3374495272 wait complete. 2024-01-15T18:09:38.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.470+0000 7f9abec00640 1 Processor -- start 2024-01-15T18:09:38.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.470+0000 7f9abec00640 1 -- start start 2024-01-15T18:09:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.471+0000 7f9abec00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ab80cf7d0 0x7f9ab80c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.471+0000 7f9abec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab8143ae0 0x7f9ab8141910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.471+0000 7f9abec00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab815e1e0 0x7f9ab80ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.471+0000 7f9abec00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80c6ce0 con 0x7f9ab815e1e0 2024-01-15T18:09:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.471+0000 7f9abec00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80c6160 con 0x7f9ab80cf7d0 2024-01-15T18:09:38.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.471+0000 7f9abec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80c6440 con 0x7f9ab8143ae0 2024-01-15T18:09:38.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9abcc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab815e1e0 0x7f9ab80ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9abcc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab815e1e0 0x7f9ab80ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58268/0 (socket says 172.21.15.26:58268) 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9abcc00640 1 -- 172.21.15.26:0/333914669 learned_addr learned my addr 172.21.15.26:0/333914669 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9aaf200640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab8143ae0 0x7f9ab8141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9ab7e00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ab80cf7d0 0x7f9ab80c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9abcc00640 1 -- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab8143ae0 msgr2=0x7f9ab8141910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9abcc00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab8143ae0 0x7f9ab8141910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9abcc00640 1 -- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ab80cf7d0 msgr2=0x7f9ab80c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9abcc00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ab80cf7d0 0x7f9ab80c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.472+0000 7f9abcc00640 1 -- 172.21.15.26:0/333914669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa80094d0 con 0x7f9ab815e1e0 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.473+0000 7f9abcc00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab815e1e0 0x7f9ab80ada50 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000d5d0 tx=0x7f9ab000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.473+0000 7f9ab4c00640 1 -- 172.21.15.26:0/333914669 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab00125f0 con 0x7f9ab815e1e0 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.473+0000 7f9ab4c00640 1 -- 172.21.15.26:0/333914669 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab0012770 con 0x7f9ab815e1e0 2024-01-15T18:09:38.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.473+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab80c6720 con 0x7f9ab815e1e0 2024-01-15T18:09:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.473+0000 7f9ab4c00640 1 -- 172.21.15.26:0/333914669 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0012b20 con 0x7f9ab815e1e0 2024-01-15T18:09:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.473+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab80c80f0 con 0x7f9ab815e1e0 2024-01-15T18:09:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.475+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab80b2bf0 con 0x7f9ab815e1e0 2024-01-15T18:09:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.476+0000 7f9ab4c00640 1 -- 172.21.15.26:0/333914669 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f9ab0012c80 con 0x7f9ab815e1e0 2024-01-15T18:09:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.477+0000 7f9ab4c00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9006c160 0x7f9a9006e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:38.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.477+0000 7f9ab4c00640 1 -- 172.21.15.26:0/333914669 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f9ab0088f00 con 0x7f9ab815e1e0 2024-01-15T18:09:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.477+0000 7f9ab7e00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9006c160 0x7f9a9006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:38.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.479+0000 7f9ab7e00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9006c160 0x7f9a9006e620 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8036000 tx=0x7f9aa803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:38.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.481+0000 7f9ab4c00640 1 -- 172.21.15.26:0/333914669 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ab80b2bf0 con 0x7f9ab815e1e0 2024-01-15T18:09:38.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.710+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ab815e670 con 0x7f9a9006c160 2024-01-15T18:09:38.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.725+0000 7f9ab4c00640 1 -- 172.21.15.26:0/333914669 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f9aa8031bb0 con 0x7f9a9006c160 2024-01-15T18:09:38.728 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:38.728 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:38.728 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:09:38.728 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:38.729 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:38.730 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9006c160 msgr2=0x7f9a9006e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9006c160 0x7f9a9006e620 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8036000 tx=0x7f9aa803c040 comp rx=0 tx=0).stop 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab815e1e0 msgr2=0x7f9ab80ada50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab815e1e0 0x7f9ab80ada50 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000d5d0 tx=0x7f9ab000da90 comp rx=0 tx=0).stop 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 shutdown_connections 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ab8143ae0 0x7f9ab8141910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9a9006c160 0x7f9a9006e620 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ab815e1e0 0x7f9ab80ada50 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 --2- 172.21.15.26:0/333914669 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ab80cf7d0 0x7f9ab80c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:38.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.727+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 >> 172.21.15.26:0/333914669 conn(0x7f9ab806a800 msgr2=0x7f9ab815c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.728+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 shutdown_connections 2024-01-15T18:09:38.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:38.728+0000 7f9abec00640 1 -- 172.21.15.26:0/333914669 wait complete. 2024-01-15T18:09:38.751 INFO:tasks.cephfs.test_volumes:percetange completed = 33.33333333333333 2024-01-15T18:09:40.752 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.020+0000 7f802d600640 1 Processor -- start 2024-01-15T18:09:41.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.021+0000 7f802d600640 1 -- start start 2024-01-15T18:09:41.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.022+0000 7f802d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80280d0ae0 0x7f80280b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:41.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.023+0000 7f802d600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f802815b2e0 0x7f80280cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:41.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.023+0000 7f802d600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80280b1db0 con 0x7f80280d0ec0 2024-01-15T18:09:41.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.023+0000 7f802d600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80280cfe00 con 0x7f802815b2e0 2024-01-15T18:09:41.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.023+0000 7f802d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80280b11a0 con 0x7f80280d0ae0 2024-01-15T18:09:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.023+0000 7f802cc00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f80280d0ec0 0x7f80280cd820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49336/0 (socket says 172.21.15.26:49336) 2024-01-15T18:09:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.023+0000 7f802cc00640 1 -- 172.21.15.26:0/322713409 learned_addr learned my addr 172.21.15.26:0/322713409 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:41.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.024+0000 7f8027e00640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80280d0ae0 0x7f80280b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.024+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95901790 0 0) 0x7f80280b1db0 con 0x7f80280d0ec0 2024-01-15T18:09:41.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.024+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8008003520 con 0x7f80280d0ec0 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8027400640 1 --2- 172.21.15.26:0/322713409 >> v2:172.21.15.124:3300/0 conn(0x7f802815b2e0 0x7f80280cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3754960832 0 0) 0x7f8008003520 con 0x7f80280d0ec0 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80280d0ae0 msgr2=0x7f80280b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f80280d0ae0 0x7f80280b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 >> v2:172.21.15.124:3300/0 conn(0x7f802815b2e0 msgr2=0x7f80280cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 --2- 172.21.15.26:0/322713409 >> v2:172.21.15.124:3300/0 conn(0x7f802815b2e0 0x7f80280cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80280b1480 con 0x7f80280d0ec0 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f80200019f0 con 0x7f80280d0ec0 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 >> v1:172.21.15.26:6789/0 conn(0x7f80280d0ec0 legacy=0x7f80280cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:41.029 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8008003970 0x7f8008003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:41.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008004360 0x7f8008004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:41.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8008004f60 0x7f800800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:41.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80280cfe00 con 0x7f8008004360 2024-01-15T18:09:41.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80280b11a0 con 0x7f8008004f60 2024-01-15T18:09:41.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.025+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800800bef0 con 0x7f8008003970 2024-01-15T18:09:41.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.026+0000 7f8027400640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008004360 0x7f8008004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:41.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.026+0000 7f802cc00640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8008004f60 0x7f800800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:41.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.027+0000 7f802cc00640 1 -- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8008003970 msgr2=0x7f8008003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:41.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.028+0000 7f802cc00640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8008003970 0x7f8008003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.028+0000 7f802cc00640 1 -- 172.21.15.26:0/322713409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008004360 msgr2=0x7f8008004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:41.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.028+0000 7f802cc00640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008004360 0x7f8008004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.031 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.028+0000 7f802cc00640 1 -- 172.21.15.26:0/322713409 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8008003520 con 0x7f8008004f60 2024-01-15T18:09:41.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.028+0000 7f8027400640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8008004360 0x7f8008004840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.030+0000 7f802cc00640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8008004f60 0x7f800800b980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f80200109f0 tx=0x7f80080057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6970edebc465fcf9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.031+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020010110 con 0x7f8008004f60 2024-01-15T18:09:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.031+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020010270 con 0x7f8008004f60 2024-01-15T18:09:41.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.031+0000 7f8026a00640 1 -- 172.21.15.26:0/322713409 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802000f720 con 0x7f8008004f60 2024-01-15T18:09:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.033+0000 7f802d600640 1 -- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8008004f60 msgr2=0x7f800800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.033+0000 7f802d600640 1 --2- 172.21.15.26:0/322713409 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8008004f60 0x7f800800b980 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f80200109f0 tx=0x7f80080057f0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.033+0000 7f802cc00640 1 -- 172.21.15.26:0/322713409 reap_dead start 2024-01-15T18:09:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.033+0000 7f802d600640 1 -- 172.21.15.26:0/322713409 shutdown_connections 2024-01-15T18:09:41.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.033+0000 7f802d600640 1 -- 172.21.15.26:0/322713409 >> 172.21.15.26:0/322713409 conn(0x7f8028099a60 msgr2=0x7f8028099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.033+0000 7f802d600640 1 -- 172.21.15.26:0/322713409 shutdown_connections 2024-01-15T18:09:41.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.033+0000 7f802d600640 1 -- 172.21.15.26:0/322713409 wait complete. 2024-01-15T18:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.036+0000 7f802d600640 1 Processor -- start 2024-01-15T18:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.037+0000 7f802d600640 1 -- start start 2024-01-15T18:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.039+0000 7f802d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8008004f60 0x7f8028156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.039+0000 7f802d600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80280ca510 0x7f8028156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.039+0000 7f802d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80280c4670 0x7f802806b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.039+0000 7f802d600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80280c8ea0 con 0x7f80280c4670 2024-01-15T18:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.039+0000 7f802d600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80280c8890 con 0x7f80280ca510 2024-01-15T18:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.039+0000 7f802d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80280ca9f0 con 0x7f8008004f60 2024-01-15T18:09:41.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.040+0000 7f802cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80280c4670 0x7f802806b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.040+0000 7f8027e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8008004f60 0x7f8028156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.040+0000 7f802cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80280c4670 0x7f802806b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58290/0 (socket says 172.21.15.26:58290) 2024-01-15T18:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.040+0000 7f802cc00640 1 -- 172.21.15.26:0/1778222363 learned_addr learned my addr 172.21.15.26:0/1778222363 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:41.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.040+0000 7f8027400640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80280ca510 0x7f8028156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.041+0000 7f802cc00640 1 -- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8008004f60 msgr2=0x7f8028156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.041+0000 7f802cc00640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8008004f60 0x7f8028156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.041+0000 7f802cc00640 1 -- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80280ca510 msgr2=0x7f8028156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.041+0000 7f802cc00640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80280ca510 0x7f8028156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.041+0000 7f802cc00640 1 -- 172.21.15.26:0/1778222363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80280c8b30 con 0x7f80280c4670 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.042+0000 7f802cc00640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80280c4670 0x7f802806b4f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f802003a040 tx=0x7f8020001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.042+0000 7f801fe00640 1 -- 172.21.15.26:0/1778222363 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020010110 con 0x7f80280c4670 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.042+0000 7f801fe00640 1 -- 172.21.15.26:0/1778222363 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802000fba0 con 0x7f80280c4670 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.042+0000 7f801fe00640 1 -- 172.21.15.26:0/1778222363 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020001fd0 con 0x7f80280c4670 2024-01-15T18:09:41.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.042+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80280cacd0 con 0x7f80280c4670 2024-01-15T18:09:41.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.043+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80281575a0 con 0x7f80280c4670 2024-01-15T18:09:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.044+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8028067b60 con 0x7f80280c4670 2024-01-15T18:09:41.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.045+0000 7f801fe00640 1 -- 172.21.15.26:0/1778222363 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f80200070d0 con 0x7f80280c4670 2024-01-15T18:09:41.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.046+0000 7f801fe00640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f800006c160 0x7f800006e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.046+0000 7f801fe00640 1 -- 172.21.15.26:0/1778222363 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f80200ae710 con 0x7f80280c4670 2024-01-15T18:09:41.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.047+0000 7f8027e00640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f800006c160 0x7f800006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:41.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.048+0000 7f8027e00640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f800006c160 0x7f800006e620 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8018004770 tx=0x7f80180094a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:41.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.052+0000 7f801fe00640 1 -- 172.21.15.26:0/1778222363 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f802007ad50 con 0x7f80280c4670 2024-01-15T18:09:41.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.434+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f80280b1a40 con 0x7f800006c160 2024-01-15T18:09:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.448+0000 7f801fe00640 1 -- 172.21.15.26:0/1778222363 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f8018009810 con 0x7f800006c160 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:09:41.452 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 33.33333333333333, 2024-01-15T18:09:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "1 GB/3 GB", 2024-01-15T18:09:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "1/3", 2024-01-15T18:09:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:09:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:09:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:41.453 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:41.453 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f800006c160 msgr2=0x7f800006e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f800006c160 0x7f800006e620 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8018004770 tx=0x7f80180094a0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80280c4670 msgr2=0x7f802806b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80280c4670 0x7f802806b4f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f802003a040 tx=0x7f8020001d80 comp rx=0 tx=0).stop 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 shutdown_connections 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f800006c160 0x7f800006e620 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f80280c4670 0x7f802806b4f0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f80280ca510 0x7f8028156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 --2- 172.21.15.26:0/1778222363 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8008004f60 0x7f8028156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 >> 172.21.15.26:0/1778222363 conn(0x7f8028099a60 msgr2=0x7f80281586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 shutdown_connections 2024-01-15T18:09:41.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:41.451+0000 7f802d600640 1 -- 172.21.15.26:0/1778222363 wait complete. 2024-01-15T18:09:41.480 INFO:tasks.cephfs.test_volumes:percetange completed = 33.33333333333333 2024-01-15T18:09:43.482 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:43.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.749+0000 7f3910e00640 1 Processor -- start 2024-01-15T18:09:43.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.750+0000 7f3910e00640 1 -- start start 2024-01-15T18:09:43.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.751+0000 7f3910e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f390c0cf9a0 0x7f390c070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:43.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.751+0000 7f3910e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f390c070ea0 0x7f390c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:43.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.751+0000 7f3910e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f390c071690 con 0x7f390c14b2c0 2024-01-15T18:09:43.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.751+0000 7f3910e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c14b6a0 con 0x7f390c0cf9a0 2024-01-15T18:09:43.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.751+0000 7f3910e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390c071970 con 0x7f390c070ea0 2024-01-15T18:09:43.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.752+0000 7f3903e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f390c0cf9a0 0x7f390c070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:43.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.752+0000 7f3903e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f390c0cf9a0 0x7f390c070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41714/0 (socket says 172.21.15.26:41714) 2024-01-15T18:09:43.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.752+0000 7f3903e00640 1 -- 172.21.15.26:0/1135002223 learned_addr learned my addr 172.21.15.26:0/1135002223 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:43.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.752+0000 7f390a800640 1 --2- 172.21.15.26:0/1135002223 >> v2:172.21.15.124:3300/0 conn(0x7f390c070ea0 0x7f390c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:43.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.752+0000 7f3903e00640 1 -- 172.21.15.26:0/1135002223 >> v2:172.21.15.124:3300/0 conn(0x7f390c070ea0 msgr2=0x7f390c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:43.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.752+0000 7f3903e00640 1 --2- 172.21.15.26:0/1135002223 >> v2:172.21.15.124:3300/0 conn(0x7f390c070ea0 0x7f390c0cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:43.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.753+0000 7f3903e00640 1 -- 172.21.15.26:0/1135002223 >> v1:172.21.15.124:6789/0 conn(0x7f390c14b2c0 legacy=0x7f390c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:43.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.753+0000 7f3903e00640 1 -- 172.21.15.26:0/1135002223 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390c071c50 con 0x7f390c0cf9a0 2024-01-15T18:09:43.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.754+0000 7f3903e00640 1 --2- 172.21.15.26:0/1135002223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390c0cf9a0 0x7f390c070960 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f390c165330 tx=0x7f38f402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498eff5870016b22 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:43.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.755+0000 7f3909400640 1 -- 172.21.15.26:0/1135002223 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4030860 con 0x7f390c0cf9a0 2024-01-15T18:09:43.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.755+0000 7f3909400640 1 -- 172.21.15.26:0/1135002223 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f40309c0 con 0x7f390c0cf9a0 2024-01-15T18:09:43.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.755+0000 7f3909400640 1 -- 172.21.15.26:0/1135002223 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4030d50 con 0x7f390c0cf9a0 2024-01-15T18:09:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.756+0000 7f3910e00640 1 -- 172.21.15.26:0/1135002223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390c0cf9a0 msgr2=0x7f390c070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.756+0000 7f3910e00640 1 --2- 172.21.15.26:0/1135002223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390c0cf9a0 0x7f390c070960 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f390c165330 tx=0x7f38f402b9a0 comp rx=0 tx=0).stop 2024-01-15T18:09:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.756+0000 7f3910e00640 1 -- 172.21.15.26:0/1135002223 shutdown_connections 2024-01-15T18:09:43.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.756+0000 7f3910e00640 1 --2- 172.21.15.26:0/1135002223 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390c0cf9a0 0x7f390c070960 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:43.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.756+0000 7f3910e00640 1 --2- 172.21.15.26:0/1135002223 >> v2:172.21.15.124:3300/0 conn(0x7f390c070ea0 0x7f390c0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:43.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.756+0000 7f3910e00640 1 -- 172.21.15.26:0/1135002223 >> 172.21.15.26:0/1135002223 conn(0x7f390c06a800 msgr2=0x7f390c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:43.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.756+0000 7f3910e00640 1 -- 172.21.15.26:0/1135002223 shutdown_connections 2024-01-15T18:09:43.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.757+0000 7f3910e00640 1 -- 172.21.15.26:0/1135002223 wait complete. 2024-01-15T18:09:43.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.759+0000 7f3910e00640 1 Processor -- start 2024-01-15T18:09:43.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.759+0000 7f3910e00640 1 -- start start 2024-01-15T18:09:43.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.761+0000 7f3910e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f390c070ea0 0x7f390c15c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:43.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.761+0000 7f3910e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f390c0cf9a0 0x7f390c144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:43.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.761+0000 7f3910e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390c14b2c0 0x7f390c164710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:43.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.761+0000 7f3910e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c143cb0 con 0x7f390c14b2c0 2024-01-15T18:09:43.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.761+0000 7f3910e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c142660 con 0x7f390c070ea0 2024-01-15T18:09:43.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.761+0000 7f3910e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f390c142dc0 con 0x7f390c0cf9a0 2024-01-15T18:09:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.761+0000 7f390a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390c14b2c0 0x7f390c164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.761+0000 7f3909e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f390c070ea0 0x7f390c15c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.762+0000 7f3903e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f390c0cf9a0 0x7f390c144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.762+0000 7f3909e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f390c070ea0 0x7f390c15c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:51600/0 (socket says 172.21.15.26:51600) 2024-01-15T18:09:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.762+0000 7f3909e00640 1 -- 172.21.15.26:0/4012276891 learned_addr learned my addr 172.21.15.26:0/4012276891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.762+0000 7f3909e00640 1 -- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f390c0cf9a0 msgr2=0x7f390c144140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.762+0000 7f3909e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f390c0cf9a0 0x7f390c144140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:43.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.762+0000 7f3909e00640 1 -- 172.21.15.26:0/4012276891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390c14b2c0 msgr2=0x7f390c164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.762+0000 7f3909e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390c14b2c0 0x7f390c164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.762+0000 7f3909e00640 1 -- 172.21.15.26:0/4012276891 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f40094d0 con 0x7f390c070ea0 2024-01-15T18:09:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.762+0000 7f3903e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f390c0cf9a0 0x7f390c144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:09:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.763+0000 7f3909e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f390c070ea0 0x7f390c15c140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c400 tx=0x7f38fc00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:43.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.763+0000 7f3902000640 1 -- 172.21.15.26:0/4012276891 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0052d0 con 0x7f390c070ea0 2024-01-15T18:09:43.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.763+0000 7f3902000640 1 -- 172.21.15.26:0/4012276891 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc005430 con 0x7f390c070ea0 2024-01-15T18:09:43.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.763+0000 7f3902000640 1 -- 172.21.15.26:0/4012276891 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc003810 con 0x7f390c070ea0 2024-01-15T18:09:43.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.763+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c1430a0 con 0x7f390c070ea0 2024-01-15T18:09:43.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.764+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c143660 con 0x7f390c070ea0 2024-01-15T18:09:43.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.766+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c0675d0 con 0x7f390c070ea0 2024-01-15T18:09:43.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.767+0000 7f3902000640 1 -- 172.21.15.26:0/4012276891 <== mon.1 v2:172.21.15.124:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f38fc003970 con 0x7f390c070ea0 2024-01-15T18:09:43.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.769+0000 7f3902000640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38e406c1b0 0x7f38e406e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.769+0000 7f3902000640 1 -- 172.21.15.26:0/4012276891 <== mon.1 v2:172.21.15.124:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f38fc087ec0 con 0x7f390c070ea0 2024-01-15T18:09:43.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.770+0000 7f3903e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38e406c1b0 0x7f38e406e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:43.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.771+0000 7f3903e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38e406c1b0 0x7f38e406e670 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f38f4036000 tx=0x7f38f4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:43.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:43.785+0000 7f3902000640 1 -- 172.21.15.26:0/4012276891 <== mon.1 v2:172.21.15.124:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38fc054500 con 0x7f390c070ea0 2024-01-15T18:09:44.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.133+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f390c0cbc40 con 0x7f38e406c1b0 2024-01-15T18:09:44.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.147+0000 7f3902000640 1 -- 172.21.15.26:0/4012276891 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+359 (secure 0 0 0) 0x7f38f4039450 con 0x7f38e406c1b0 2024-01-15T18:09:44.151 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:44.151 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:44.151 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "in-progress", 2024-01-15T18:09:44.151 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:09:44.151 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:09:44.151 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:09:44.151 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_report": { 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: "percentage completed": 66.66666666666666, 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: "amount copied": "2 GB/3 GB", 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: "regfiles copied": "2/3", 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: "subdirs copied": "0/1", 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: "symlinks copied": "0/0" 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:44.152 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38e406c1b0 msgr2=0x7f38e406e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38e406c1b0 0x7f38e406e670 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f38f4036000 tx=0x7f38f4008030 comp rx=0 tx=0).stop 2024-01-15T18:09:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f390c070ea0 msgr2=0x7f390c15c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f390c070ea0 0x7f390c15c140 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c400 tx=0x7f38fc00c8c0 comp rx=0 tx=0).stop 2024-01-15T18:09:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 shutdown_connections 2024-01-15T18:09:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f390c0cf9a0 0x7f390c144140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38e406c1b0 0x7f38e406e670 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:44.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f390c14b2c0 0x7f390c164710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 --2- 172.21.15.26:0/4012276891 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f390c070ea0 0x7f390c15c140 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 >> 172.21.15.26:0/4012276891 conn(0x7f390c06a800 msgr2=0x7f390c0b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 shutdown_connections 2024-01-15T18:09:44.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:44.150+0000 7f3910e00640 1 -- 172.21.15.26:0/4012276891 wait complete. 2024-01-15T18:09:44.183 INFO:tasks.cephfs.test_volumes:percetange completed = 66.66666666666666 2024-01-15T18:09:46.184 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.415+0000 7ffa83200640 1 Processor -- start 2024-01-15T18:09:46.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.416+0000 7ffa83200640 1 -- start start 2024-01-15T18:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.417+0000 7ffa83200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa8413cf10 0x7ffa8413f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.417+0000 7ffa83200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ffa840cddf0 0x7ffa840b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.417+0000 7ffa83200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa840ce440 con 0x7ffa8413fae0 2024-01-15T18:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.417+0000 7ffa83200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8413ff90 con 0x7ffa8413cf10 2024-01-15T18:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.417+0000 7ffa83200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa840ce220 con 0x7ffa840cddf0 2024-01-15T18:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.417+0000 7ffa81e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffa8413cf10 0x7ffa8413f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.417+0000 7ffa79200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ffa8413fae0 0x7ffa840cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41590/0 (socket says 172.21.15.26:41590) 2024-01-15T18:09:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.417+0000 7ffa79200640 1 -- 172.21.15.26:0/3005071639 learned_addr learned my addr 172.21.15.26:0/3005071639 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.418+0000 7ffa82800640 1 --2- 172.21.15.26:0/3005071639 >> v2:172.21.15.124:3300/0 conn(0x7ffa840cddf0 0x7ffa840b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.418+0000 7ffa81e00640 1 -- 172.21.15.26:0/3005071639 >> v2:172.21.15.124:3300/0 conn(0x7ffa840cddf0 msgr2=0x7ffa840b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.418+0000 7ffa81e00640 1 --2- 172.21.15.26:0/3005071639 >> v2:172.21.15.124:3300/0 conn(0x7ffa840cddf0 0x7ffa840b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.418+0000 7ffa81e00640 1 -- 172.21.15.26:0/3005071639 >> v1:172.21.15.26:6789/0 conn(0x7ffa8413fae0 legacy=0x7ffa840cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.418+0000 7ffa81e00640 1 -- 172.21.15.26:0/3005071639 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa840ce720 con 0x7ffa8413cf10 2024-01-15T18:09:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.418+0000 7ffa81400640 1 -- 172.21.15.26:0/3005071639 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823935245 0 0) 0x7ffa840ce440 con 0x7ffa8413fae0 2024-01-15T18:09:46.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.418+0000 7ffa82800640 1 --2- 172.21.15.26:0/3005071639 >> v2:172.21.15.124:3300/0 conn(0x7ffa840cddf0 0x7ffa840b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:46.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.419+0000 7ffa81e00640 1 --2- 172.21.15.26:0/3005071639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413cf10 0x7ffa8413f5a0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ffa84157f90 tx=0x7ffa7402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c859c21f584b57f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.419+0000 7ffa81400640 1 -- 172.21.15.26:0/3005071639 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7402cc70 con 0x7ffa8413cf10 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.419+0000 7ffa81400640 1 -- 172.21.15.26:0/3005071639 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7402cdd0 con 0x7ffa8413cf10 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.419+0000 7ffa81400640 1 -- 172.21.15.26:0/3005071639 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa74034660 con 0x7ffa8413cf10 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.420+0000 7ffa83200640 1 -- 172.21.15.26:0/3005071639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413cf10 msgr2=0x7ffa8413f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.420+0000 7ffa83200640 1 --2- 172.21.15.26:0/3005071639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413cf10 0x7ffa8413f5a0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ffa84157f90 tx=0x7ffa7402bec0 comp rx=0 tx=0).stop 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.420+0000 7ffa83200640 1 -- 172.21.15.26:0/3005071639 shutdown_connections 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.420+0000 7ffa83200640 1 --2- 172.21.15.26:0/3005071639 >> v2:172.21.15.124:3300/0 conn(0x7ffa840cddf0 0x7ffa840b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.420+0000 7ffa83200640 1 --2- 172.21.15.26:0/3005071639 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413cf10 0x7ffa8413f5a0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.420+0000 7ffa83200640 1 -- 172.21.15.26:0/3005071639 >> 172.21.15.26:0/3005071639 conn(0x7ffa8406a7d0 msgr2=0x7ffa8406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.420+0000 7ffa83200640 1 -- 172.21.15.26:0/3005071639 shutdown_connections 2024-01-15T18:09:46.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.420+0000 7ffa83200640 1 -- 172.21.15.26:0/3005071639 wait complete. 2024-01-15T18:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.422+0000 7ffa83200640 1 Processor -- start 2024-01-15T18:09:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.422+0000 7ffa83200640 1 -- start start 2024-01-15T18:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.423+0000 7ffa83200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa840cddf0 0x7ffa841530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.423+0000 7ffa83200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa8413cf10 0x7ffa84071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.423+0000 7ffa83200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413fae0 0x7ffa8415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.423+0000 7ffa83200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa84070160 con 0x7ffa8413fae0 2024-01-15T18:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.423+0000 7ffa83200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa84070400 con 0x7ffa8413cf10 2024-01-15T18:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.423+0000 7ffa83200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa840706e0 con 0x7ffa840cddf0 2024-01-15T18:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.423+0000 7ffa82800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413fae0 0x7ffa8415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.423+0000 7ffa82800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413fae0 0x7ffa8415cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41744/0 (socket says 172.21.15.26:41744) 2024-01-15T18:09:46.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.423+0000 7ffa82800640 1 -- 172.21.15.26:0/3387515594 learned_addr learned my addr 172.21.15.26:0/3387515594 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa79200640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa8413cf10 0x7ffa84071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa82800640 1 -- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa840cddf0 msgr2=0x7ffa841530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa81e00640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa840cddf0 0x7ffa841530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:46.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa82800640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa840cddf0 0x7ffa841530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa82800640 1 -- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa8413cf10 msgr2=0x7ffa84071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa82800640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa8413cf10 0x7ffa84071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa82800640 1 -- 172.21.15.26:0/3387515594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa74009a70 con 0x7ffa8413fae0 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa82800640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413fae0 0x7ffa8415cb50 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c7f0 tx=0x7ffa7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa7aa00640 1 -- 172.21.15.26:0/3387515594 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c007820 con 0x7ffa8413fae0 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.424+0000 7ffa7aa00640 1 -- 172.21.15.26:0/3387515594 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c007980 con 0x7ffa8413fae0 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.425+0000 7ffa7aa00640 1 -- 172.21.15.26:0/3387515594 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c007d70 con 0x7ffa8413fae0 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.425+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa840709c0 con 0x7ffa8413fae0 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.425+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa84070f80 con 0x7ffa8413fae0 2024-01-15T18:09:46.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.426+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa84071170 con 0x7ffa8413fae0 2024-01-15T18:09:46.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.427+0000 7ffa7aa00640 1 -- 172.21.15.26:0/3387515594 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7ffa7c0113f0 con 0x7ffa8413fae0 2024-01-15T18:09:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.428+0000 7ffa7aa00640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa5c06c160 0x7ffa5c06e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:46.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.428+0000 7ffa7aa00640 1 -- 172.21.15.26:0/3387515594 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7ffa7c088cc0 con 0x7ffa8413fae0 2024-01-15T18:09:46.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.429+0000 7ffa81e00640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa5c06c160 0x7ffa5c06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:46.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.430+0000 7ffa81e00640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa5c06c160 0x7ffa5c06e620 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffa74036000 tx=0x7ffa74030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:46.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.432+0000 7ffa7aa00640 1 -- 172.21.15.26:0/3387515594 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa84071170 con 0x7ffa8413fae0 2024-01-15T18:09:46.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:46.665+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa840b2070 con 0x7ffa5c06c160 2024-01-15T18:09:48.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.491+0000 7ffa7aa00640 1 -- 172.21.15.26:0/3387515594 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7ffa74003b60 con 0x7ffa5c06c160 2024-01-15T18:09:48.495 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:48.495 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:48.495 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:09:48.495 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:48.495 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.494+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa5c06c160 msgr2=0x7ffa5c06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.494+0000 7ffa83200640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa5c06c160 0x7ffa5c06e620 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffa74036000 tx=0x7ffa74030000 comp rx=0 tx=0).stop 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.494+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413fae0 msgr2=0x7ffa8415cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.494+0000 7ffa83200640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413fae0 0x7ffa8415cb50 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c7f0 tx=0x7ffa7c00ccb0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.495+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 shutdown_connections 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.495+0000 7ffa83200640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffa5c06c160 0x7ffa5c06e620 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.495+0000 7ffa83200640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffa8413fae0 0x7ffa8415cb50 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.495+0000 7ffa83200640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffa8413cf10 0x7ffa84071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.495+0000 7ffa83200640 1 --2- 172.21.15.26:0/3387515594 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffa840cddf0 0x7ffa841530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.495+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 >> 172.21.15.26:0/3387515594 conn(0x7ffa8406a7d0 msgr2=0x7ffa84153bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.495+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 shutdown_connections 2024-01-15T18:09:48.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.495+0000 7ffa83200640 1 -- 172.21.15.26:0/3387515594 wait complete. 2024-01-15T18:09:48.522 INFO:tasks.cephfs.test_volumes:expected clone state = ('complete',) 2024-01-15T18:09:48.522 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:09:48.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.746+0000 7f9acd000640 1 Processor -- start 2024-01-15T18:09:48.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.746+0000 7f9acd000640 1 -- start start 2024-01-15T18:09:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.748+0000 7f9acd000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ac815d4b0 0x7f9ac80d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.748+0000 7f9acd000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac80cf9a0 0x7f9ac80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:48.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.748+0000 7f9acd000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac80d0320 con 0x7f9ac80cddf0 2024-01-15T18:09:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.748+0000 7f9acd000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac80b2730 con 0x7f9ac815d4b0 2024-01-15T18:09:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.748+0000 7f9acd000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80d1ec0 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.748+0000 7f9ac7400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9ac815d4b0 0x7f9ac80d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.748+0000 7f9ac7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9ac80cddf0 0x7f9ac80ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41594/0 (socket says 172.21.15.26:41594) 2024-01-15T18:09:48.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.748+0000 7f9ac7e00640 1 -- 172.21.15.26:0/4062693696 learned_addr learned my addr 172.21.15.26:0/4062693696 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.748+0000 7f9abe800640 1 --2- 172.21.15.26:0/4062693696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac80cf9a0 0x7f9ac80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.749+0000 7f9ac7400640 1 -- 172.21.15.26:0/4062693696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac80cf9a0 msgr2=0x7f9ac80cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.749+0000 7f9ac7400640 1 --2- 172.21.15.26:0/4062693696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac80cf9a0 0x7f9ac80cfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.749+0000 7f9ac7400640 1 -- 172.21.15.26:0/4062693696 >> v1:172.21.15.26:6789/0 conn(0x7f9ac80cddf0 legacy=0x7f9ac80ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.749+0000 7f9ac7400640 1 -- 172.21.15.26:0/4062693696 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80b29d0 con 0x7f9ac815d4b0 2024-01-15T18:09:48.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.750+0000 7f9ac7400640 1 --2- 172.21.15.26:0/4062693696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac815d4b0 0x7f9ac80d1550 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80097d0 tx=0x7f9ab802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5072bcedf948f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:48.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.750+0000 7f9ac6a00640 1 -- 172.21.15.26:0/4062693696 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8031800 con 0x7f9ac815d4b0 2024-01-15T18:09:48.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.750+0000 7f9ac6a00640 1 -- 172.21.15.26:0/4062693696 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab8031960 con 0x7f9ac815d4b0 2024-01-15T18:09:48.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.750+0000 7f9ac6a00640 1 -- 172.21.15.26:0/4062693696 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8031cf0 con 0x7f9ac815d4b0 2024-01-15T18:09:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.751+0000 7f9acd000640 1 -- 172.21.15.26:0/4062693696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac815d4b0 msgr2=0x7f9ac80d1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.751+0000 7f9acd000640 1 --2- 172.21.15.26:0/4062693696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac815d4b0 0x7f9ac80d1550 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80097d0 tx=0x7f9ab802b8a0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.751+0000 7f9acd000640 1 -- 172.21.15.26:0/4062693696 shutdown_connections 2024-01-15T18:09:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.751+0000 7f9acd000640 1 --2- 172.21.15.26:0/4062693696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac80cf9a0 0x7f9ac80cfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.751+0000 7f9acd000640 1 --2- 172.21.15.26:0/4062693696 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac815d4b0 0x7f9ac80d1550 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.751+0000 7f9acd000640 1 -- 172.21.15.26:0/4062693696 >> 172.21.15.26:0/4062693696 conn(0x7f9ac80d3000 msgr2=0x7f9ac80d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.751+0000 7f9acd000640 1 -- 172.21.15.26:0/4062693696 shutdown_connections 2024-01-15T18:09:48.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.751+0000 7f9acd000640 1 -- 172.21.15.26:0/4062693696 wait complete. 2024-01-15T18:09:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.753+0000 7f9acd000640 1 Processor -- start 2024-01-15T18:09:48.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.753+0000 7f9acd000640 1 -- start start 2024-01-15T18:09:48.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.754+0000 7f9acd000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ac80cddf0 0x7f9ac8150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.754+0000 7f9acd000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac80cf9a0 0x7f9ac81605d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.754+0000 7f9acd000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac815d4b0 0x7f9ac815b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.754+0000 7f9acd000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80cae80 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.754+0000 7f9acd000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac81512e0 con 0x7f9ac80cddf0 2024-01-15T18:09:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.754+0000 7f9acd000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80ca270 con 0x7f9ac815d4b0 2024-01-15T18:09:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.754+0000 7f9abe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac80cf9a0 0x7f9ac81605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.754+0000 7f9abe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac80cf9a0 0x7f9ac81605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41756/0 (socket says 172.21.15.26:41756) 2024-01-15T18:09:48.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.754+0000 7f9abe800640 1 -- 172.21.15.26:0/2268011653 learned_addr learned my addr 172.21.15.26:0/2268011653 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:48.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9abe800640 1 -- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac815d4b0 msgr2=0x7f9ac815b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:48.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9abe800640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac815d4b0 0x7f9ac815b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9abe800640 1 -- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ac80cddf0 msgr2=0x7f9ac8150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:48.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9abe800640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ac80cddf0 0x7f9ac8150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:48.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9abe800640 1 -- 172.21.15.26:0/2268011653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8009400 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9abe800640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac80cf9a0 0x7f9ac81605d0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0007ef0 tx=0x7f9ab000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9abfe00640 1 -- 172.21.15.26:0/2268011653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0014620 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9abfe00640 1 -- 172.21.15.26:0/2268011653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab0014780 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9abfe00640 1 -- 172.21.15.26:0/2268011653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0014b30 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac815da10 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.755+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac80ca830 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.757+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac8067450 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.757+0000 7f9abfe00640 1 -- 172.21.15.26:0/2268011653 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f9ab0014c90 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.759+0000 7f9abfe00640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9aa006c160 0x7f9aa006e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.759+0000 7f9abfe00640 1 -- 172.21.15.26:0/2268011653 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f9ab0089360 con 0x7f9ac80cf9a0 2024-01-15T18:09:48.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.759+0000 7f9ac7400640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9aa006c160 0x7f9aa006e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:48.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.760+0000 7f9ac7400640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9aa006c160 0x7f9aa006e620 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8036000 tx=0x7f9ab8030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.764+0000 7f9abfe00640 1 -- 172.21.15.26:0/2268011653 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ab0055940 con 0x7f9ac80cf9a0 2024-01-15T18:09:49.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:48.997+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac80d2d80 con 0x7f9aa006c160 2024-01-15T18:09:49.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.014+0000 7f9abfe00640 1 -- 172.21.15.26:0/2268011653 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+46 (secure 0 0 0) 0x7f9ac80d2d80 con 0x7f9aa006c160 2024-01-15T18:09:49.017 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:09:49.018 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:09:49.018 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "complete" 2024-01-15T18:09:49.018 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:49.018 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:09:49.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.016+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9aa006c160 msgr2=0x7f9aa006e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.016+0000 7f9acd000640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9aa006c160 0x7f9aa006e620 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8036000 tx=0x7f9ab8030000 comp rx=0 tx=0).stop 2024-01-15T18:09:49.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.016+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac80cf9a0 msgr2=0x7f9ac81605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.016+0000 7f9acd000640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac80cf9a0 0x7f9ac81605d0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f9ab0007ef0 tx=0x7f9ab000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.016+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 shutdown_connections 2024-01-15T18:09:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.016+0000 7f9acd000640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ac815d4b0 0x7f9ac815b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.016+0000 7f9acd000640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9aa006c160 0x7f9aa006e620 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.016+0000 7f9acd000640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9ac80cf9a0 0x7f9ac81605d0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.016+0000 7f9acd000640 1 --2- 172.21.15.26:0/2268011653 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ac80cddf0 0x7f9ac8150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.017+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 >> 172.21.15.26:0/2268011653 conn(0x7f9ac80d3000 msgr2=0x7f9ac80ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.017+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 shutdown_connections 2024-01-15T18:09:49.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.017+0000 7f9acd000640 1 -- 172.21.15.26:0/2268011653 wait complete. 2024-01-15T18:09:49.041 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:09:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.265+0000 7f4436600640 1 Processor -- start 2024-01-15T18:09:49.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.265+0000 7f4436600640 1 -- start start 2024-01-15T18:09:49.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.267+0000 7f4436600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443013e360 0x7f44300cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.267+0000 7f4436600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443015c5f0 con 0x7f443015f9c0 2024-01-15T18:09:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.267+0000 7f4436600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443006c470 con 0x7f443013df80 2024-01-15T18:09:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.267+0000 7f4436600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443006c750 con 0x7f443013e360 2024-01-15T18:09:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.267+0000 7f442fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f443015f9c0 0x7f44300cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41610/0 (socket says 172.21.15.26:41610) 2024-01-15T18:09:49.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.267+0000 7f442fe00640 1 -- 172.21.15.26:0/771717367 learned_addr learned my addr 172.21.15.26:0/771717367 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:49.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442ea00640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443013e360 0x7f44300cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1532315566 0 0) 0x7f443015c5f0 con 0x7f443015f9c0 2024-01-15T18:09:49.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4410003520 con 0x7f443015f9c0 2024-01-15T18:09:49.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854195061 0 0) 0x7f443006c470 con 0x7f443013df80 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f443015c5f0 con 0x7f443013df80 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3767336150 0 0) 0x7f4410003520 con 0x7f443015f9c0 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443013e360 msgr2=0x7f44300cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443013e360 0x7f44300cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 >> v1:172.21.15.124:6789/0 conn(0x7f443013df80 legacy=0x7f443015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443006ca30 con 0x7f443015f9c0 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f44280019f0 con 0x7f443015f9c0 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 >> v1:172.21.15.26:6789/0 conn(0x7f443015f9c0 legacy=0x7f44300cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.268+0000 7f442e000640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4410003b30 0x7f4410003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442e000640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443013e360 0x7f44100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442e000640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f4410005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443006c750 con 0x7f4410004d00 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441000ba60 con 0x7f4410003b30 2024-01-15T18:09:49.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441000bc20 con 0x7f443013e360 2024-01-15T18:09:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442fe00640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f4410005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442ea00640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443013e360 0x7f44100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442f400640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4410003b30 0x7f4410003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442fe00640 1 -- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443013e360 msgr2=0x7f44100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442fe00640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443013e360 0x7f44100047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442fe00640 1 -- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4410003b30 msgr2=0x7f4410003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442fe00640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4410003b30 0x7f4410003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.269+0000 7f442fe00640 1 -- 172.21.15.26:0/771717367 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4410003520 con 0x7f4410004d00 2024-01-15T18:09:49.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.271+0000 7f442fe00640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f4410005180 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f442800a700 tx=0x7f44100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473f48a7fa7a483e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:49.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.271+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428034630 con 0x7f4410004d00 2024-01-15T18:09:49.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.271+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442802f070 con 0x7f4410004d00 2024-01-15T18:09:49.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.271+0000 7f442e000640 1 -- 172.21.15.26:0/771717367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44280349f0 con 0x7f4410004d00 2024-01-15T18:09:49.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.271+0000 7f4436600640 1 -- 172.21.15.26:0/771717367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 msgr2=0x7f4410005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.271+0000 7f4436600640 1 --2- 172.21.15.26:0/771717367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f4410005180 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f442800a700 tx=0x7f44100057f0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.271+0000 7f442fe00640 1 -- 172.21.15.26:0/771717367 reap_dead start 2024-01-15T18:09:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.272+0000 7f4436600640 1 -- 172.21.15.26:0/771717367 shutdown_connections 2024-01-15T18:09:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.272+0000 7f4436600640 1 -- 172.21.15.26:0/771717367 >> 172.21.15.26:0/771717367 conn(0x7f443006a800 msgr2=0x7f443006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.272+0000 7f4436600640 1 -- 172.21.15.26:0/771717367 shutdown_connections 2024-01-15T18:09:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.272+0000 7f4436600640 1 -- 172.21.15.26:0/771717367 wait complete. 2024-01-15T18:09:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.273+0000 7f4436600640 1 Processor -- start 2024-01-15T18:09:49.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.274+0000 7f4436600640 1 -- start start 2024-01-15T18:09:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.274+0000 7f4436600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f44301324d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.274+0000 7f4436600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f443015db10 0x7f44301f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f4436600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443015df70 0x7f44301f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f4436600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443013dc50 con 0x7f4410004d00 2024-01-15T18:09:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f4436600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44301f7710 con 0x7f443015db10 2024-01-15T18:09:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f4436600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44301f7430 con 0x7f443015df70 2024-01-15T18:09:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f44301324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f44301324d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41764/0 (socket says 172.21.15.26:41764) 2024-01-15T18:09:49.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442f400640 1 -- 172.21.15.26:0/2532923644 learned_addr learned my addr 172.21.15.26:0/2532923644 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442ea00640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f443015db10 0x7f44301f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442fe00640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443015df70 0x7f44301f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442f400640 1 -- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443015df70 msgr2=0x7f44301f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442f400640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443015df70 0x7f44301f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442f400640 1 -- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f443015db10 msgr2=0x7f44301f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442f400640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f443015db10 0x7f44301f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.275+0000 7f442f400640 1 -- 172.21.15.26:0/2532923644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44301319f0 con 0x7f4410004d00 2024-01-15T18:09:49.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.276+0000 7f442f400640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f44301324d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f442000e780 tx=0x7f442000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:49.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.276+0000 7f4427400640 1 -- 172.21.15.26:0/2532923644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44200050d0 con 0x7f4410004d00 2024-01-15T18:09:49.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.276+0000 7f4427400640 1 -- 172.21.15.26:0/2532923644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4420005230 con 0x7f4410004d00 2024-01-15T18:09:49.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.276+0000 7f4427400640 1 -- 172.21.15.26:0/2532923644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442000c6c0 con 0x7f4410004d00 2024-01-15T18:09:49.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.276+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4430131cd0 con 0x7f4410004d00 2024-01-15T18:09:49.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.276+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44300ac010 con 0x7f4410004d00 2024-01-15T18:09:49.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.277+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44300675d0 con 0x7f4410004d00 2024-01-15T18:09:49.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.278+0000 7f4427400640 1 -- 172.21.15.26:0/2532923644 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f442000c820 con 0x7f4410004d00 2024-01-15T18:09:49.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.279+0000 7f4427400640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f440806c160 0x7f440806e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.279+0000 7f4427400640 1 -- 172.21.15.26:0/2532923644 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f4420088890 con 0x7f4410004d00 2024-01-15T18:09:49.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.280+0000 7f442ea00640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f440806c160 0x7f440806e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.281+0000 7f442ea00640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f440806c160 0x7f440806e620 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f44300b0520 tx=0x7f4418008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:49.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.283+0000 7f4427400640 1 -- 172.21.15.26:0/2532923644 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4420054ed0 con 0x7f4410004d00 2024-01-15T18:09:49.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.510+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f443006cff0 con 0x7f440806c160 2024-01-15T18:09:49.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.514+0000 7f4427400640 1 -- 172.21.15.26:0/2532923644 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4418009680 con 0x7f440806c160 2024-01-15T18:09:49.518 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:09:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:09:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:09:49.518 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:49.518 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:09:49.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f440806c160 msgr2=0x7f440806e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f440806c160 0x7f440806e620 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f44300b0520 tx=0x7f4418008040 comp rx=0 tx=0).stop 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 msgr2=0x7f44301324d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f44301324d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f442000e780 tx=0x7f442000ec40 comp rx=0 tx=0).stop 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 shutdown_connections 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f443015df70 0x7f44301f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f440806c160 0x7f440806e620 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f443015db10 0x7f44301f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 --2- 172.21.15.26:0/2532923644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4410004d00 0x7f44301324d0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 >> 172.21.15.26:0/2532923644 conn(0x7f443006a800 msgr2=0x7f44300af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 shutdown_connections 2024-01-15T18:09:49.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.517+0000 7f4436600640 1 -- 172.21.15.26:0/2532923644 wait complete. 2024-01-15T18:09:49.541 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:09:49.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.766+0000 7f77c4600640 1 Processor -- start 2024-01-15T18:09:49.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.766+0000 7f77c4600640 1 -- start start 2024-01-15T18:09:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.768+0000 7f77c4600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77bc0cd170 0x7f77bc071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.768+0000 7f77c4600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc071690 con 0x7f77bc0cf9a0 2024-01-15T18:09:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.768+0000 7f77c4600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc071970 con 0x7f77bc14b2c0 2024-01-15T18:09:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.768+0000 7f77c4600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77bc14b6a0 con 0x7f77bc0cd170 2024-01-15T18:09:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.768+0000 7f77c2600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f77bc0cd170 0x7f77bc071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.768+0000 7f77c1200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f77bc0cf9a0 0x7f77bc070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41616/0 (socket says 172.21.15.26:41616) 2024-01-15T18:09:49.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.768+0000 7f77c1200640 1 -- 172.21.15.26:0/2683702449 learned_addr learned my addr 172.21.15.26:0/2683702449 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.769+0000 7f77c2600640 1 -- 172.21.15.26:0/2683702449 >> v1:172.21.15.124:6789/0 conn(0x7f77bc14b2c0 legacy=0x7f77bc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.769+0000 7f77c2600640 1 -- 172.21.15.26:0/2683702449 >> v1:172.21.15.26:6789/0 conn(0x7f77bc0cf9a0 legacy=0x7f77bc070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.769+0000 7f77c2600640 1 -- 172.21.15.26:0/2683702449 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc071c50 con 0x7f77bc0cd170 2024-01-15T18:09:49.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.769+0000 7f77bbe00640 1 -- 172.21.15.26:0/2683702449 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970261740 0 0) 0x7f77bc071690 con 0x7f77bc0cf9a0 2024-01-15T18:09:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.770+0000 7f77c2600640 1 --2- 172.21.15.26:0/2683702449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc0cd170 0x7f77bc071070 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009d90 tx=0x7f77b402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97967777d0a357b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.770+0000 7f77bbe00640 1 -- 172.21.15.26:0/2683702449 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b402cc60 con 0x7f77bc0cd170 2024-01-15T18:09:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.770+0000 7f77bbe00640 1 -- 172.21.15.26:0/2683702449 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b402cdc0 con 0x7f77bc0cd170 2024-01-15T18:09:49.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.770+0000 7f77bbe00640 1 -- 172.21.15.26:0/2683702449 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b4034660 con 0x7f77bc0cd170 2024-01-15T18:09:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.771+0000 7f77c4600640 1 -- 172.21.15.26:0/2683702449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc0cd170 msgr2=0x7f77bc071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.771+0000 7f77c4600640 1 --2- 172.21.15.26:0/2683702449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc0cd170 0x7f77bc071070 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009d90 tx=0x7f77b402beb0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.771+0000 7f77c4600640 1 -- 172.21.15.26:0/2683702449 shutdown_connections 2024-01-15T18:09:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.771+0000 7f77c4600640 1 --2- 172.21.15.26:0/2683702449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc0cd170 0x7f77bc071070 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.771+0000 7f77c4600640 1 -- 172.21.15.26:0/2683702449 >> 172.21.15.26:0/2683702449 conn(0x7f77bc06a800 msgr2=0x7f77bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.771+0000 7f77c4600640 1 -- 172.21.15.26:0/2683702449 shutdown_connections 2024-01-15T18:09:49.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.771+0000 7f77c4600640 1 -- 172.21.15.26:0/2683702449 wait complete. 2024-01-15T18:09:49.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.773+0000 7f77c4600640 1 Processor -- start 2024-01-15T18:09:49.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.773+0000 7f77c4600640 1 -- start start 2024-01-15T18:09:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.774+0000 7f77c4600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77bc0cd170 0x7f77bc13f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.774+0000 7f77c4600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77bc0cf9a0 0x7f77bc143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.774+0000 7f77c4600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc14b2c0 0x7f77bc144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.774+0000 7f77c4600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc145f00 con 0x7f77bc14b2c0 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.774+0000 7f77c4600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc13f6d0 con 0x7f77bc0cd170 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.774+0000 7f77c4600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc1435b0 con 0x7f77bc0cf9a0 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c2600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc14b2c0 0x7f77bc144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c2600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc14b2c0 0x7f77bc144a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41776/0 (socket says 172.21.15.26:41776) 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c2600640 1 -- 172.21.15.26:0/2927036260 learned_addr learned my addr 172.21.15.26:0/2927036260 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c1c00640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77bc0cd170 0x7f77bc13f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c2600640 1 -- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77bc0cf9a0 msgr2=0x7f77bc143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c1200640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77bc0cf9a0 0x7f77bc143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c2600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77bc0cf9a0 0x7f77bc143070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c2600640 1 -- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77bc0cd170 msgr2=0x7f77bc13f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c2600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77bc0cd170 0x7f77bc13f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:49.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.775+0000 7f77c2600640 1 -- 172.21.15.26:0/2927036260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b4009a70 con 0x7f77bc14b2c0 2024-01-15T18:09:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.776+0000 7f77c2600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc14b2c0 0x7f77bc144a80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009eb0 tx=0x7f77b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:49.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.776+0000 7f77b9600640 1 -- 172.21.15.26:0/2927036260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b403e610 con 0x7f77bc14b2c0 2024-01-15T18:09:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.776+0000 7f77b9600640 1 -- 172.21.15.26:0/2927036260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b403e790 con 0x7f77bc14b2c0 2024-01-15T18:09:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.776+0000 7f77b9600640 1 -- 172.21.15.26:0/2927036260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b403eb40 con 0x7f77bc14b2c0 2024-01-15T18:09:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.776+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc144fc0 con 0x7f77bc14b2c0 2024-01-15T18:09:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.776+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc145560 con 0x7f77bc14b2c0 2024-01-15T18:09:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.777+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc0675d0 con 0x7f77bc14b2c0 2024-01-15T18:09:49.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.778+0000 7f77b9600640 1 -- 172.21.15.26:0/2927036260 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f77b403b050 con 0x7f77bc14b2c0 2024-01-15T18:09:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.779+0000 7f77b9600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f779406c160 0x7f779406e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:49.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.779+0000 7f77b9600640 1 -- 172.21.15.26:0/2927036260 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f77b40ab400 con 0x7f77bc14b2c0 2024-01-15T18:09:49.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.780+0000 7f77c1c00640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f779406c160 0x7f779406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:49.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.781+0000 7f77c1c00640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f779406c160 0x7f779406e620 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f77ac006fd0 tx=0x7f77ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:49.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:49.783+0000 7f77b9600640 1 -- 172.21.15.26:0/2927036260 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77b40779e0 con 0x7f77bc14b2c0 2024-01-15T18:09:50.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.003+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77bc149730 con 0x7f779406c160 2024-01-15T18:09:50.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.014+0000 7f77b9600640 1 -- 172.21.15.26:0/2927036260 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+79 (secure 0 0 0) 0x7f77bc149730 con 0x7f779406c160 2024-01-15T18:09:50.017 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:09:50.018 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:09:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:09:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sv1" 2024-01-15T18:09:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:09:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:09:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1clone1" 2024-01-15T18:09:50.018 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:50.018 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:09:50.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f779406c160 msgr2=0x7f779406e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f779406c160 0x7f779406e620 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f77ac006fd0 tx=0x7f77ac008040 comp rx=0 tx=0).stop 2024-01-15T18:09:50.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc14b2c0 msgr2=0x7f77bc144a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc14b2c0 0x7f77bc144a80 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009eb0 tx=0x7f77b4004a40 comp rx=0 tx=0).stop 2024-01-15T18:09:50.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 shutdown_connections 2024-01-15T18:09:50.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f77bc0cf9a0 0x7f77bc143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f779406c160 0x7f779406e620 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f77bc14b2c0 0x7f77bc144a80 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 --2- 172.21.15.26:0/2927036260 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f77bc0cd170 0x7f77bc13f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 >> 172.21.15.26:0/2927036260 conn(0x7f77bc06a800 msgr2=0x7f77bc06cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:50.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 shutdown_connections 2024-01-15T18:09:50.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.016+0000 7f77c4600640 1 -- 172.21.15.26:0/2927036260 wait complete. 2024-01-15T18:09:50.039 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:09:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.253+0000 7f8eac400640 1 Processor -- start 2024-01-15T18:09:50.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.253+0000 7f8eac400640 1 -- start start 2024-01-15T18:09:50.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.254+0000 7f8eac400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ea413df80 0x7f8ea415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.254+0000 7f8eac400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea40cf9a0 0x7f8ea40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.254+0000 7f8eac400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea406c470 con 0x7f8ea413e360 2024-01-15T18:09:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8eac400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea406c750 con 0x7f8ea413df80 2024-01-15T18:09:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8eac400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea406ca30 con 0x7f8ea40cf9a0 2024-01-15T18:09:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8ea9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ea413df80 0x7f8ea415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8ea9a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8ea413df80 0x7f8ea415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41778/0 (socket says 172.21.15.26:41778) 2024-01-15T18:09:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8ea9a00640 1 -- 172.21.15.26:0/1538173815 learned_addr learned my addr 172.21.15.26:0/1538173815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:50.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8ea9a00640 1 -- 172.21.15.26:0/1538173815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea40cf9a0 msgr2=0x7f8ea40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:50.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8eaa400640 1 --2- 172.21.15.26:0/1538173815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea40cf9a0 0x7f8ea40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8ea9a00640 1 --2- 172.21.15.26:0/1538173815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea40cf9a0 0x7f8ea40cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8ea9a00640 1 -- 172.21.15.26:0/1538173815 >> v1:172.21.15.124:6789/0 conn(0x7f8ea413e360 legacy=0x7f8ea40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:50.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8ea9a00640 1 -- 172.21.15.26:0/1538173815 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea406cd10 con 0x7f8ea413df80 2024-01-15T18:09:50.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.255+0000 7f8eaa400640 1 --2- 172.21.15.26:0/1538173815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea40cf9a0 0x7f8ea40cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:50.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.257+0000 7f8ea9a00640 1 --2- 172.21.15.26:0/1538173815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413df80 0x7f8ea415bee0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009870 tx=0x7f8e9402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b728ec0ee2a2b267 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:50.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.257+0000 7f8ea3e00640 1 -- 172.21.15.26:0/1538173815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94031640 con 0x7f8ea413df80 2024-01-15T18:09:50.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.257+0000 7f8ea3e00640 1 -- 172.21.15.26:0/1538173815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e940317a0 con 0x7f8ea413df80 2024-01-15T18:09:50.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.257+0000 7f8ea3e00640 1 -- 172.21.15.26:0/1538173815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e94031b30 con 0x7f8ea413df80 2024-01-15T18:09:50.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.258+0000 7f8eac400640 1 -- 172.21.15.26:0/1538173815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413df80 msgr2=0x7f8ea415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.258+0000 7f8eac400640 1 --2- 172.21.15.26:0/1538173815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413df80 0x7f8ea415bee0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009870 tx=0x7f8e9402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.258+0000 7f8eac400640 1 -- 172.21.15.26:0/1538173815 shutdown_connections 2024-01-15T18:09:50.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.258+0000 7f8eac400640 1 --2- 172.21.15.26:0/1538173815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea40cf9a0 0x7f8ea40cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.258+0000 7f8eac400640 1 --2- 172.21.15.26:0/1538173815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413df80 0x7f8ea415bee0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.258+0000 7f8eac400640 1 -- 172.21.15.26:0/1538173815 >> 172.21.15.26:0/1538173815 conn(0x7f8ea406a800 msgr2=0x7f8ea406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:50.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.258+0000 7f8eac400640 1 -- 172.21.15.26:0/1538173815 shutdown_connections 2024-01-15T18:09:50.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.258+0000 7f8eac400640 1 -- 172.21.15.26:0/1538173815 wait complete. 2024-01-15T18:09:50.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.260+0000 7f8eac400640 1 Processor -- start 2024-01-15T18:09:50.263 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.260+0000 7f8eac400640 1 -- start start 2024-01-15T18:09:50.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.261+0000 7f8eac400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40cf9a0 0x7f8ea413d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.261+0000 7f8eac400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea413df80 0x7f8ea413b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.266 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.261+0000 7f8eac400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413e360 0x7f8ea40cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.261+0000 7f8eac400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41e5ff0 con 0x7f8ea413e360 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.261+0000 7f8eac400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41e62d0 con 0x7f8ea40cf9a0 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.261+0000 7f8eac400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41e65b0 con 0x7f8ea413df80 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.261+0000 7f8eaa400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413e360 0x7f8ea40cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.261+0000 7f8eaa400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413e360 0x7f8ea40cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41782/0 (socket says 172.21.15.26:41782) 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.261+0000 7f8eaa400640 1 -- 172.21.15.26:0/3738680136 learned_addr learned my addr 172.21.15.26:0/3738680136 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8ea9a00640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40cf9a0 0x7f8ea413d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8ea9000640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea413df80 0x7f8ea413b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8eaa400640 1 -- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea413df80 msgr2=0x7f8ea413b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8eaa400640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea413df80 0x7f8ea413b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8eaa400640 1 -- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40cf9a0 msgr2=0x7f8ea413d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8eaa400640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40cf9a0 0x7f8ea413d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8eaa400640 1 -- 172.21.15.26:0/3738680136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e940094d0 con 0x7f8ea413e360 2024-01-15T18:09:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8eaa400640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413e360 0x7f8ea40cd840 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00cd80 tx=0x7f8e9c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8ea1600640 1 -- 172.21.15.26:0/3738680136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c00f680 con 0x7f8ea413e360 2024-01-15T18:09:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8ea1600640 1 -- 172.21.15.26:0/3738680136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c00f800 con 0x7f8ea413e360 2024-01-15T18:09:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8ea1600640 1 -- 172.21.15.26:0/3738680136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c00fbb0 con 0x7f8ea413e360 2024-01-15T18:09:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.262+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea41e6890 con 0x7f8ea413e360 2024-01-15T18:09:50.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.263+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea41e6e50 con 0x7f8ea413e360 2024-01-15T18:09:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.264+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea41e7070 con 0x7f8ea413e360 2024-01-15T18:09:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.265+0000 7f8ea1600640 1 -- 172.21.15.26:0/3738680136 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f8e9c00b040 con 0x7f8ea413e360 2024-01-15T18:09:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.265+0000 7f8ea1600640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e7c06c190 0x7f8e7c06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.265+0000 7f8ea1600640 1 -- 172.21.15.26:0/3738680136 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f8e9c089100 con 0x7f8ea413e360 2024-01-15T18:09:50.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.266+0000 7f8ea9a00640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e7c06c190 0x7f8e7c06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.270 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.267+0000 7f8ea9a00640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e7c06c190 0x7f8e7c06e650 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8e94036000 tx=0x7f8e9403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:50.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.270+0000 7f8ea1600640 1 -- 172.21.15.26:0/3738680136 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e9c055990 con 0x7f8ea413e360 2024-01-15T18:09:50.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.491+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ea4155220 con 0x7f8e7c06c190 2024-01-15T18:09:50.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.526+0000 7f8ea1600640 1 -- 172.21.15.26:0/3738680136 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8ea4155220 con 0x7f8e7c06c190 2024-01-15T18:09:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:09:50.530 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:09:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:09:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1" 2024-01-15T18:09:50.530 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:09:50.530 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:09:50.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e7c06c190 msgr2=0x7f8e7c06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e7c06c190 0x7f8e7c06e650 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8e94036000 tx=0x7f8e9403c040 comp rx=0 tx=0).stop 2024-01-15T18:09:50.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413e360 msgr2=0x7f8ea40cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413e360 0x7f8ea40cd840 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00cd80 tx=0x7f8e9c00a650 comp rx=0 tx=0).stop 2024-01-15T18:09:50.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 shutdown_connections 2024-01-15T18:09:50.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8ea413df80 0x7f8ea413b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e7c06c190 0x7f8e7c06e650 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8ea413e360 0x7f8ea40cd840 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 --2- 172.21.15.26:0/3738680136 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8ea40cf9a0 0x7f8ea413d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.528+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 >> 172.21.15.26:0/3738680136 conn(0x7f8ea406a800 msgr2=0x7f8ea4153f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:50.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.529+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 shutdown_connections 2024-01-15T18:09:50.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.529+0000 7f8eac400640 1 -- 172.21.15.26:0/3738680136 wait complete. 2024-01-15T18:09:50.549 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:09:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.774+0000 7f8480a00640 1 Processor -- start 2024-01-15T18:09:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.775+0000 7f8480a00640 1 -- start start 2024-01-15T18:09:50.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.776+0000 7f8480a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f847c0b3f30 0x7f847c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.776+0000 7f8480a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c0d0a10 con 0x7f847c160920 2024-01-15T18:09:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.776+0000 7f8480a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c0cf080 con 0x7f847c0d0100 2024-01-15T18:09:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.776+0000 7f8480a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847c0c6be0 con 0x7f847c0b3f30 2024-01-15T18:09:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.776+0000 7f847b400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f847c160920 0x7f847c0cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41626/0 (socket says 172.21.15.26:41626) 2024-01-15T18:09:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.776+0000 7f847b400640 1 -- 172.21.15.26:0/3872686486 learned_addr learned my addr 172.21.15.26:0/3872686486 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847be00640 1 --2- 172.21.15.26:0/3872686486 >> v2:172.21.15.124:3300/0 conn(0x7f847c0b3f30 0x7f847c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4238835092 0 0) 0x7f847c0d0a10 con 0x7f847c160920 2024-01-15T18:09:50.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f845c003520 con 0x7f847c160920 2024-01-15T18:09:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 406082253 0 0) 0x7f845c003520 con 0x7f847c160920 2024-01-15T18:09:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 >> v2:172.21.15.124:3300/0 conn(0x7f847c0b3f30 msgr2=0x7f847c0d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 --2- 172.21.15.26:0/3872686486 >> v2:172.21.15.124:3300/0 conn(0x7f847c0b3f30 0x7f847c0d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 >> v1:172.21.15.124:6789/0 conn(0x7f847c0d0100 legacy=0x7f847c0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c0ce7e0 con 0x7f847c160920 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f846c0019f0 con 0x7f847c160920 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 >> v1:172.21.15.26:6789/0 conn(0x7f847c160920 legacy=0x7f847c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f845c003970 0x7f845c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f845c0047c0 0x7f845c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f845c005180 0x7f845c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0c6be0 con 0x7f845c005180 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c00c020 con 0x7f845c003970 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.777+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c00c1e0 con 0x7f845c0047c0 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.778+0000 7f847be00640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f845c005180 0x7f845c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.778+0000 7f847b400640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f845c003970 0x7f845c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.778+0000 7f847be00640 1 -- 172.21.15.26:0/3872686486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f845c0047c0 msgr2=0x7f845c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.778+0000 7f847be00640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f845c0047c0 0x7f845c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.778+0000 7f847be00640 1 -- 172.21.15.26:0/3872686486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f845c003970 msgr2=0x7f845c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.778+0000 7f847be00640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f845c003970 0x7f845c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.778+0000 7f847be00640 1 -- 172.21.15.26:0/3872686486 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c003520 con 0x7f845c005180 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.779+0000 7f847be00640 1 -- 172.21.15.26:0/3872686486 reap_dead start 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.779+0000 7f847be00640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f845c005180 0x7f845c00b980 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8474009ac0 tx=0x7f845c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18346c5752987246 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.779+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474004020 con 0x7f845c005180 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.779+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8474004180 con 0x7f845c005180 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.779+0000 7f847a000640 1 -- 172.21.15.26:0/3872686486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847402c710 con 0x7f845c005180 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.780+0000 7f8480a00640 1 -- 172.21.15.26:0/3872686486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f845c005180 msgr2=0x7f845c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.780+0000 7f8480a00640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f845c005180 0x7f845c00b980 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f8474009ac0 tx=0x7f845c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.780+0000 7f8480a00640 1 -- 172.21.15.26:0/3872686486 shutdown_connections 2024-01-15T18:09:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.780+0000 7f8480a00640 1 --2- 172.21.15.26:0/3872686486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f845c005180 0x7f845c00b980 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.780+0000 7f8480a00640 1 -- 172.21.15.26:0/3872686486 >> 172.21.15.26:0/3872686486 conn(0x7f847c0d3090 msgr2=0x7f847c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.780+0000 7f8480a00640 1 -- 172.21.15.26:0/3872686486 shutdown_connections 2024-01-15T18:09:50.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.780+0000 7f8480a00640 1 -- 172.21.15.26:0/3872686486 wait complete. 2024-01-15T18:09:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.782+0000 7f8480a00640 1 Processor -- start 2024-01-15T18:09:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.782+0000 7f8480a00640 1 -- start start 2024-01-15T18:09:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.783+0000 7f8480a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f845c005180 0x7f847c0b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.783+0000 7f8480a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f847c15a4b0 0x7f847c0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.783+0000 7f8480a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f847c0cbca0 0x7f847c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.783+0000 7f8480a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0cc120 con 0x7f847c15a4b0 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.783+0000 7f8480a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0b6660 con 0x7f847c0cbca0 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.783+0000 7f8480a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0b71e0 con 0x7f845c005180 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.783+0000 7f847aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f847c15a4b0 0x7f847c0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.783+0000 7f847aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f847c15a4b0 0x7f847c0cb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41808/0 (socket says 172.21.15.26:41808) 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.783+0000 7f847aa00640 1 -- 172.21.15.26:0/2082067986 learned_addr learned my addr 172.21.15.26:0/2082067986 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f847be00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f847c0cbca0 0x7f847c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f847aa00640 1 -- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f845c005180 msgr2=0x7f847c0b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f847aa00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f845c005180 0x7f847c0b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f847aa00640 1 -- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f847c0cbca0 msgr2=0x7f847c0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f847aa00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f847c0cbca0 0x7f847c0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:50.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f847aa00640 1 -- 172.21.15.26:0/2082067986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c15ad50 con 0x7f847c15a4b0 2024-01-15T18:09:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f847be00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f847c0cbca0 0x7f847c0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f847aa00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f847c15a4b0 0x7f847c0cb760 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f846400c460 tx=0x7f846400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f8473400640 1 -- 172.21.15.26:0/2082067986 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640052d0 con 0x7f847c15a4b0 2024-01-15T18:09:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f8473400640 1 -- 172.21.15.26:0/2082067986 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8464005450 con 0x7f847c15a4b0 2024-01-15T18:09:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f8473400640 1 -- 172.21.15.26:0/2082067986 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464012870 con 0x7f847c15a4b0 2024-01-15T18:09:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.784+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c0b68b0 con 0x7f847c15a4b0 2024-01-15T18:09:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.785+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c0b6e70 con 0x7f847c15a4b0 2024-01-15T18:09:50.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.786+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c067450 con 0x7f847c15a4b0 2024-01-15T18:09:50.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.787+0000 7f8473400640 1 -- 172.21.15.26:0/2082067986 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f84640129d0 con 0x7f847c15a4b0 2024-01-15T18:09:50.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.788+0000 7f8473400640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f845406c160 0x7f845406e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:50.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.788+0000 7f8473400640 1 -- 172.21.15.26:0/2082067986 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f84640881c0 con 0x7f847c15a4b0 2024-01-15T18:09:50.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.789+0000 7f847b400640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f845406c160 0x7f845406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:50.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.790+0000 7f847b400640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f845406c160 0x7f845406e620 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f847c0c9020 tx=0x7f846c001f90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:50.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:50.793+0000 7f8473400640 1 -- 172.21.15.26:0/2082067986 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8464054a80 con 0x7f847c15a4b0 2024-01-15T18:09:51.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.013+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f847c160d00 con 0x7f845406c160 2024-01-15T18:09:51.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.088+0000 7f8473400640 1 -- 172.21.15.26:0/2082067986 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f846c001950 con 0x7f845406c160 2024-01-15T18:09:51.092 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:09:51.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.091+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f845406c160 msgr2=0x7f845406e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f845406c160 0x7f845406e620 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f847c0c9020 tx=0x7f846c001f90 comp rx=0 tx=0).stop 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f847c15a4b0 msgr2=0x7f847c0cb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f847c15a4b0 0x7f847c0cb760 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f846400c460 tx=0x7f846400c920 comp rx=0 tx=0).stop 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 shutdown_connections 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f845406c160 0x7f845406e620 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f847c0cbca0 0x7f847c0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f847c15a4b0 0x7f847c0cb760 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 --2- 172.21.15.26:0/2082067986 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f845c005180 0x7f847c0b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 >> 172.21.15.26:0/2082067986 conn(0x7f847c0d3090 msgr2=0x7f847c06c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 shutdown_connections 2024-01-15T18:09:51.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.092+0000 7f8480a00640 1 -- 172.21.15.26:0/2082067986 wait complete. 2024-01-15T18:09:51.113 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:09:51.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.329+0000 7f2cf9600640 1 Processor -- start 2024-01-15T18:09:51.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.329+0000 7f2cf9600640 1 -- start start 2024-01-15T18:09:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.330+0000 7f2cf9600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40cf9a0 0x7f2cf40cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.330+0000 7f2cf9600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2cf40b31d0 0x7f2cf40b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.331+0000 7f2cf9600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf40d0320 con 0x7f2cf415d4b0 2024-01-15T18:09:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.331+0000 7f2cf9600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf40d1c60 con 0x7f2cf40b31d0 2024-01-15T18:09:51.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.331+0000 7f2cf9600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40b2730 con 0x7f2cf40cf9a0 2024-01-15T18:09:51.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.331+0000 7f2cf3400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40cf9a0 0x7f2cf40cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.331+0000 7f2cf8c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2cf40b31d0 0x7f2cf40b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.331+0000 7f2cf8c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2cf40b31d0 0x7f2cf40b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:51720/0 (socket says 172.21.15.26:51720) 2024-01-15T18:09:51.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.331+0000 7f2cf8c00640 1 -- 172.21.15.26:0/453378343 learned_addr learned my addr 172.21.15.26:0/453378343 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:51.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.331+0000 7f2cf3400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40cf9a0 0x7f2cf40cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:49854/0 (socket says 172.21.15.26:49854) 2024-01-15T18:09:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.332+0000 7f2cf3400640 1 -- 172.21.15.26:0/453378343 >> v2:172.21.15.124:3300/0 conn(0x7f2cf40b31d0 msgr2=0x7f2cf40b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.332+0000 7f2cf3400640 1 --2- 172.21.15.26:0/453378343 >> v2:172.21.15.124:3300/0 conn(0x7f2cf40b31d0 0x7f2cf40b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.332+0000 7f2cf3400640 1 -- 172.21.15.26:0/453378343 >> v1:172.21.15.124:6789/0 conn(0x7f2cf415d4b0 legacy=0x7f2cf40d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.332+0000 7f2cf3400640 1 -- 172.21.15.26:0/453378343 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf40d1ec0 con 0x7f2cf40cf9a0 2024-01-15T18:09:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.333+0000 7f2cf3400640 1 --2- 172.21.15.26:0/453378343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40cf9a0 0x7f2cf40cfde0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0097d0 tx=0x7f2cdc02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c690793753e1086 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:51.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.334+0000 7f2cf2a00640 1 -- 172.21.15.26:0/453378343 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc031800 con 0x7f2cf40cf9a0 2024-01-15T18:09:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.334+0000 7f2cf2a00640 1 -- 172.21.15.26:0/453378343 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc031960 con 0x7f2cf40cf9a0 2024-01-15T18:09:51.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.334+0000 7f2cf2a00640 1 -- 172.21.15.26:0/453378343 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc031cf0 con 0x7f2cf40cf9a0 2024-01-15T18:09:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.335+0000 7f2cf9600640 1 -- 172.21.15.26:0/453378343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40cf9a0 msgr2=0x7f2cf40cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.335+0000 7f2cf9600640 1 --2- 172.21.15.26:0/453378343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40cf9a0 0x7f2cf40cfde0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0097d0 tx=0x7f2cdc02b8a0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.335+0000 7f2cf9600640 1 -- 172.21.15.26:0/453378343 shutdown_connections 2024-01-15T18:09:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.335+0000 7f2cf9600640 1 --2- 172.21.15.26:0/453378343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40cf9a0 0x7f2cf40cfde0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.335+0000 7f2cf9600640 1 --2- 172.21.15.26:0/453378343 >> v2:172.21.15.124:3300/0 conn(0x7f2cf40b31d0 0x7f2cf40b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.335+0000 7f2cf9600640 1 -- 172.21.15.26:0/453378343 >> 172.21.15.26:0/453378343 conn(0x7f2cf40d3000 msgr2=0x7f2cf40d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.335+0000 7f2cf9600640 1 -- 172.21.15.26:0/453378343 shutdown_connections 2024-01-15T18:09:51.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.335+0000 7f2cf9600640 1 -- 172.21.15.26:0/453378343 wait complete. 2024-01-15T18:09:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.337+0000 7f2cf9600640 1 Processor -- start 2024-01-15T18:09:51.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.338+0000 7f2cf9600640 1 -- start start 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf9600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b31d0 0x7f2cf40ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf9600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf40cf9a0 0x7f2cf41f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf9600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf415d4b0 0x7f2cf415fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf9600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf40a0880 con 0x7f2cf415d4b0 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf9600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf415dba0 con 0x7f2cf40cf9a0 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf9600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf415e440 con 0x7f2cf40b31d0 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf415d4b0 0x7f2cf415fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf415d4b0 0x7f2cf415fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41812/0 (socket says 172.21.15.26:41812) 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf8c00640 1 -- 172.21.15.26:0/2444517985 learned_addr learned my addr 172.21.15.26:0/2444517985 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:51.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.339+0000 7f2cf3e00640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b31d0 0x7f2cf40ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.344 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.340+0000 7f2cf3400640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf40cf9a0 0x7f2cf41f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.340+0000 7f2cf8c00640 1 -- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b31d0 msgr2=0x7f2cf40ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.340+0000 7f2cf8c00640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b31d0 0x7f2cf40ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.340+0000 7f2cf8c00640 1 -- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf40cf9a0 msgr2=0x7f2cf41f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.340+0000 7f2cf8c00640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf40cf9a0 0x7f2cf41f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.340+0000 7f2cf8c00640 1 -- 172.21.15.26:0/2444517985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc009400 con 0x7f2cf415d4b0 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.340+0000 7f2cf3400640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf40cf9a0 0x7f2cf41f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.340+0000 7f2cf8c00640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf415d4b0 0x7f2cf415fe70 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009cf0 tx=0x7f2cec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.341+0000 7f2cebe00640 1 -- 172.21.15.26:0/2444517985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec009b60 con 0x7f2cf415d4b0 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.341+0000 7f2cebe00640 1 -- 172.21.15.26:0/2444517985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec00f040 con 0x7f2cf415d4b0 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.341+0000 7f2cebe00640 1 -- 172.21.15.26:0/2444517985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec003990 con 0x7f2cf415d4b0 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.341+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf41f7580 con 0x7f2cf415d4b0 2024-01-15T18:09:51.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.341+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf415ddf0 con 0x7f2cf415d4b0 2024-01-15T18:09:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.342+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf415e5d0 con 0x7f2cf415d4b0 2024-01-15T18:09:51.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.343+0000 7f2cebe00640 1 -- 172.21.15.26:0/2444517985 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7f2cec003af0 con 0x7f2cf415d4b0 2024-01-15T18:09:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.344+0000 7f2cebe00640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ccc06c160 0x7f2ccc06e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.345+0000 7f2cebe00640 1 -- 172.21.15.26:0/2444517985 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7f2cec088c10 con 0x7f2cf415d4b0 2024-01-15T18:09:51.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.345+0000 7f2cf3e00640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ccc06c160 0x7f2ccc06e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.346+0000 7f2cf3e00640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ccc06c160 0x7f2ccc06e620 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4000f60 tx=0x7f2ce4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:51.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.349+0000 7f2cebe00640 1 -- 172.21.15.26:0/2444517985 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2cec0554d0 con 0x7f2cf415d4b0 2024-01-15T18:09:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.570+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cf40d08e0 con 0x7f2ccc06c160 2024-01-15T18:09:51.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.619+0000 7f2cebe00640 1 -- 172.21.15.26:0/2444517985 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2cf40d08e0 con 0x7f2ccc06c160 2024-01-15T18:09:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ccc06c160 msgr2=0x7f2ccc06e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ccc06c160 0x7f2ccc06e620 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4000f60 tx=0x7f2ce4009040 comp rx=0 tx=0).stop 2024-01-15T18:09:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf415d4b0 msgr2=0x7f2cf415fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf415d4b0 0x7f2cf415fe70 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009cf0 tx=0x7f2cec00c490 comp rx=0 tx=0).stop 2024-01-15T18:09:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 shutdown_connections 2024-01-15T18:09:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ccc06c160 0x7f2ccc06e620 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cf415d4b0 0x7f2cf415fe70 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cf40cf9a0 0x7f2cf41f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 --2- 172.21.15.26:0/2444517985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cf40b31d0 0x7f2cf40ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 >> 172.21.15.26:0/2444517985 conn(0x7f2cf40d3000 msgr2=0x7f2cf40cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:51.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.622+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 shutdown_connections 2024-01-15T18:09:51.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.623+0000 7f2cf9600640 1 -- 172.21.15.26:0/2444517985 wait complete. 2024-01-15T18:09:51.632 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.627+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.632 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.628+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.632 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.628+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.632 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.628+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.632 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.628+0000 7f8f58200640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.646 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-01-15T18:09:51.655 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.650+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.655 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.650+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.656 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.650+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.656 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.650+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.656 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:09:51.650+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:09:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.857+0000 7fda31800640 1 Processor -- start 2024-01-15T18:09:51.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.857+0000 7fda31800640 1 -- start start 2024-01-15T18:09:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.858+0000 7fda31800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c144150 0x7fda2c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.859+0000 7fda31800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c06ad80 con 0x7fda2c06f160 2024-01-15T18:09:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.859+0000 7fda31800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c0cc8b0 con 0x7fda2c06a920 2024-01-15T18:09:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.859+0000 7fda31800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c06f580 con 0x7fda2c144150 2024-01-15T18:09:51.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.859+0000 7fda30e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fda2c06f160 0x7fda2c164da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41630/0 (socket says 172.21.15.26:41630) 2024-01-15T18:09:51.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.859+0000 7fda30e00640 1 -- 172.21.15.26:0/59342464 learned_addr learned my addr 172.21.15.26:0/59342464 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.859+0000 7fda2be00640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c144150 0x7fda2c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.859+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744414139 0 0) 0x7fda2c06ad80 con 0x7fda2c06f160 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda0c003520 con 0x7fda2c06f160 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 886122035 0 0) 0x7fda0c003520 con 0x7fda2c06f160 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c144150 msgr2=0x7fda2c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c144150 0x7fda2c06a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 >> v1:172.21.15.124:6789/0 conn(0x7fda2c06a920 legacy=0x7fda2c06ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c0ccb90 con 0x7fda2c06f160 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fda24001670 con 0x7fda2c06f160 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 >> v1:172.21.15.26:6789/0 conn(0x7fda2c06f160 legacy=0x7fda2c164da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c144150 0x7fda0c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda0c003f20 0x7fda0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c004d00 0x7fda0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c00ba30 con 0x7fda0c003f20 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c00bba0 con 0x7fda0c004d00 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c00bd60 con 0x7fda2c144150 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.860+0000 7fda2b400640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda0c003f20 0x7fda0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.861+0000 7fda2be00640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c144150 0x7fda0c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.861+0000 7fda30e00640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c004d00 0x7fda0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.861+0000 7fda2b400640 1 -- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c144150 msgr2=0x7fda0c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.861+0000 7fda2b400640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c144150 0x7fda0c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.861+0000 7fda2b400640 1 -- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c004d00 msgr2=0x7fda0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.861+0000 7fda2b400640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c004d00 0x7fda0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.861+0000 7fda2b400640 1 -- 172.21.15.26:0/59342464 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c003520 con 0x7fda0c003f20 2024-01-15T18:09:51.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.861+0000 7fda30e00640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c004d00 0x7fda0c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:51.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.861+0000 7fda2be00640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c144150 0x7fda0c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.862+0000 7fda2b400640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda0c003f20 0x7fda0c0047c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fda14001750 tx=0x7fda0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134cdd8869953868 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.862+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140052d0 con 0x7fda0c003f20 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.862+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1402e050 con 0x7fda0c003f20 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.862+0000 7fda2aa00640 1 -- 172.21.15.26:0/59342464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14003830 con 0x7fda0c003f20 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.863+0000 7fda31800640 1 -- 172.21.15.26:0/59342464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda0c003f20 msgr2=0x7fda0c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.863+0000 7fda31800640 1 --2- 172.21.15.26:0/59342464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda0c003f20 0x7fda0c0047c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fda14001750 tx=0x7fda0c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.863+0000 7fda30e00640 1 -- 172.21.15.26:0/59342464 reap_dead start 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.863+0000 7fda31800640 1 -- 172.21.15.26:0/59342464 shutdown_connections 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.863+0000 7fda31800640 1 -- 172.21.15.26:0/59342464 >> 172.21.15.26:0/59342464 conn(0x7fda2c0b1d40 msgr2=0x7fda2c0b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.863+0000 7fda31800640 1 -- 172.21.15.26:0/59342464 shutdown_connections 2024-01-15T18:09:51.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.863+0000 7fda31800640 1 -- 172.21.15.26:0/59342464 wait complete. 2024-01-15T18:09:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.865+0000 7fda31800640 1 Processor -- start 2024-01-15T18:09:51.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.865+0000 7fda31800640 1 -- start start 2024-01-15T18:09:51.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.866+0000 7fda31800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c003f20 0x7fda2c15b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.866+0000 7fda31800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1eee60 0x7fda2c13c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.866+0000 7fda31800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c0b3350 0x7fda2c0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.866+0000 7fda31800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c0b3d10 con 0x7fda2c1eee60 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.866+0000 7fda31800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c1ef2e0 con 0x7fda0c003f20 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.866+0000 7fda31800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c15bf30 con 0x7fda2c0b3350 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda2b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1eee60 0x7fda2c13c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda2b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1eee60 0x7fda2c13c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41838/0 (socket says 172.21.15.26:41838) 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda2b400640 1 -- 172.21.15.26:0/256316179 learned_addr learned my addr 172.21.15.26:0/256316179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda2be00640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c003f20 0x7fda2c15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda30e00640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c0b3350 0x7fda2c0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda2b400640 1 -- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c0b3350 msgr2=0x7fda2c0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda2b400640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c0b3350 0x7fda2c0b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda2b400640 1 -- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c003f20 msgr2=0x7fda2c15b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda2b400640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c003f20 0x7fda2c15b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda2b400640 1 -- 172.21.15.26:0/256316179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c15c210 con 0x7fda2c1eee60 2024-01-15T18:09:51.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.867+0000 7fda30e00640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c0b3350 0x7fda2c0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.868+0000 7fda2b400640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1eee60 0x7fda2c13c740 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fda14008fd0 tx=0x7fda14001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.868+0000 7fda23e00640 1 -- 172.21.15.26:0/256316179 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140076c0 con 0x7fda2c1eee60 2024-01-15T18:09:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.868+0000 7fda23e00640 1 -- 172.21.15.26:0/256316179 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda14007a30 con 0x7fda2c1eee60 2024-01-15T18:09:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.868+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c15c4f0 con 0x7fda2c1eee60 2024-01-15T18:09:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.868+0000 7fda23e00640 1 -- 172.21.15.26:0/256316179 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14006be0 con 0x7fda2c1eee60 2024-01-15T18:09:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.868+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c15cab0 con 0x7fda2c1eee60 2024-01-15T18:09:51.873 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.870+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c067b60 con 0x7fda2c1eee60 2024-01-15T18:09:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.870+0000 7fda23e00640 1 -- 172.21.15.26:0/256316179 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 32) v1 ==== 90865+0+0 (secure 0 0 0) 0x7fda14006d40 con 0x7fda2c1eee60 2024-01-15T18:09:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.871+0000 7fda23e00640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0406c160 0x7fda0406e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:51.874 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.871+0000 7fda23e00640 1 -- 172.21.15.26:0/256316179 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5794+0+0 (secure 0 0 0) 0x7fda140aa790 con 0x7fda2c1eee60 2024-01-15T18:09:51.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.872+0000 7fda2be00640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0406c160 0x7fda0406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:51.876 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.873+0000 7fda2be00640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0406c160 0x7fda0406e620 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fda1c003be0 tx=0x7fda1c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:51.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:51.876+0000 7fda23e00640 1 -- 172.21.15.26:0/256316179 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda14077050 con 0x7fda2c1eee60 2024-01-15T18:09:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.122+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fda2c0b3d10 con 0x7fda0406c160 2024-01-15T18:09:52.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.171+0000 7fda23e00640 1 -- 172.21.15.26:0/256316179 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fda2c0b3d10 con 0x7fda0406c160 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0406c160 msgr2=0x7fda0406e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0406c160 0x7fda0406e620 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fda1c003be0 tx=0x7fda1c00b040 comp rx=0 tx=0).stop 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1eee60 msgr2=0x7fda2c13c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1eee60 0x7fda2c13c740 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fda14008fd0 tx=0x7fda14001120 comp rx=0 tx=0).stop 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 shutdown_connections 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fda2c0b3350 0x7fda2c0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fda0406c160 0x7fda0406e620 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fda2c1eee60 0x7fda2c13c740 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 --2- 172.21.15.26:0/256316179 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fda0c003f20 0x7fda2c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:52.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 >> 172.21.15.26:0/256316179 conn(0x7fda2c0b1d40 msgr2=0x7fda2c06f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 shutdown_connections 2024-01-15T18:09:52.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:52.174+0000 7fda31800640 1 -- 172.21.15.26:0/256316179 wait complete. 2024-01-15T18:09:52.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:52.198 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:09:52.257 INFO:teuthology.orchestra.run.smithi026.stdout:3 2024-01-15T18:09:57.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:09:57.259 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:09:57.323 INFO:teuthology.orchestra.run.smithi026.stdout:2 2024-01-15T18:09:57.329 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:09:57.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.544+0000 7f716e400640 1 Processor -- start 2024-01-15T18:09:57.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.544+0000 7f716e400640 1 -- start start 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.545+0000 7f716e400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f71680b3b20 0x7f716815a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.546+0000 7f716e400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f716815d8a0 0x7f7168164520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.546+0000 7f716e400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71680b1e30 con 0x7f71681505b0 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.546+0000 7f716e400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716815abd0 con 0x7f716815d8a0 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.546+0000 7f716e400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71680b20a0 con 0x7f71680b3b20 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.546+0000 7f716da00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f716815d8a0 0x7f7168164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.546+0000 7f716da00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f716815d8a0 0x7f7168164520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59540/0 (socket says 172.21.15.26:59540) 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.546+0000 7f716da00640 1 -- 172.21.15.26:0/1523009690 learned_addr learned my addr 172.21.15.26:0/1523009690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.546+0000 7f716d000640 1 --2- 172.21.15.26:0/1523009690 >> v2:172.21.15.124:3300/0 conn(0x7f71680b3b20 0x7f716815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.547+0000 7f716da00640 1 -- 172.21.15.26:0/1523009690 >> v2:172.21.15.124:3300/0 conn(0x7f71680b3b20 msgr2=0x7f716815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:57.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.547+0000 7f716da00640 1 --2- 172.21.15.26:0/1523009690 >> v2:172.21.15.124:3300/0 conn(0x7f71680b3b20 0x7f716815a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.547+0000 7f716da00640 1 -- 172.21.15.26:0/1523009690 >> v1:172.21.15.124:6789/0 conn(0x7f71681505b0 legacy=0x7f716815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:09:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.547+0000 7f716da00640 1 -- 172.21.15.26:0/1523009690 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71680b2380 con 0x7f716815d8a0 2024-01-15T18:09:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.548+0000 7f716da00640 1 --2- 172.21.15.26:0/1523009690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f716815d8a0 0x7f7168164520 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f716806e370 tx=0x7f716002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89dd9ed8428414d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:57.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.548+0000 7f7167e00640 1 -- 172.21.15.26:0/1523009690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716002ea90 con 0x7f716815d8a0 2024-01-15T18:09:57.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.548+0000 7f7167e00640 1 -- 172.21.15.26:0/1523009690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716002ebf0 con 0x7f716815d8a0 2024-01-15T18:09:57.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.548+0000 7f7167e00640 1 -- 172.21.15.26:0/1523009690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160009600 con 0x7f716815d8a0 2024-01-15T18:09:57.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.549+0000 7f716e400640 1 -- 172.21.15.26:0/1523009690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f716815d8a0 msgr2=0x7f7168164520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.549+0000 7f716e400640 1 --2- 172.21.15.26:0/1523009690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f716815d8a0 0x7f7168164520 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f716806e370 tx=0x7f716002e0f0 comp rx=0 tx=0).stop 2024-01-15T18:09:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.549+0000 7f716e400640 1 -- 172.21.15.26:0/1523009690 shutdown_connections 2024-01-15T18:09:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.549+0000 7f716e400640 1 --2- 172.21.15.26:0/1523009690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f716815d8a0 0x7f7168164520 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.549+0000 7f716e400640 1 --2- 172.21.15.26:0/1523009690 >> v2:172.21.15.124:3300/0 conn(0x7f71680b3b20 0x7f716815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.549+0000 7f716e400640 1 -- 172.21.15.26:0/1523009690 >> 172.21.15.26:0/1523009690 conn(0x7f716806a7d0 msgr2=0x7f716806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:09:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.549+0000 7f716e400640 1 -- 172.21.15.26:0/1523009690 shutdown_connections 2024-01-15T18:09:57.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.549+0000 7f716e400640 1 -- 172.21.15.26:0/1523009690 wait complete. 2024-01-15T18:09:57.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.551+0000 7f716e400640 1 Processor -- start 2024-01-15T18:09:57.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.551+0000 7f716e400640 1 -- start start 2024-01-15T18:09:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.552+0000 7f716e400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71680b3b20 0x7f71681f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.552+0000 7f716e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71681505b0 0x7f71681f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.552+0000 7f716e400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f716815d8a0 0x7f7168153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.552+0000 7f716e400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71681f42d0 con 0x7f71681505b0 2024-01-15T18:09:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.552+0000 7f716e400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71681f7590 con 0x7f716815d8a0 2024-01-15T18:09:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.552+0000 7f716e400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71681f36c0 con 0x7f71680b3b20 2024-01-15T18:09:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.552+0000 7f715fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71681505b0 0x7f71681f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.552+0000 7f715fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71681505b0 0x7f71681f7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59550/0 (socket says 172.21.15.26:59550) 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.552+0000 7f715fe00640 1 -- 172.21.15.26:0/4214553601 learned_addr learned my addr 172.21.15.26:0/4214553601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f716da00640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f716815d8a0 0x7f7168153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f715fe00640 1 -- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71680b3b20 msgr2=0x7f71681f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f716d000640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71680b3b20 0x7f71681f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f715fe00640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71680b3b20 0x7f71681f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f715fe00640 1 -- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f716815d8a0 msgr2=0x7f7168153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f715fe00640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f716815d8a0 0x7f7168153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f715fe00640 1 -- 172.21.15.26:0/4214553601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716000bd40 con 0x7f71681505b0 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f716da00640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f716815d8a0 0x7f7168153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f715fe00640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71681505b0 0x7f71681f7050 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f715000c410 tx=0x7f715000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.553+0000 7f7165600640 1 -- 172.21.15.26:0/4214553601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71500035f0 con 0x7f71681505b0 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.554+0000 7f7165600640 1 -- 172.21.15.26:0/4214553601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7150003750 con 0x7f71681505b0 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.554+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71681f39a0 con 0x7f71681505b0 2024-01-15T18:09:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.554+0000 7f7165600640 1 -- 172.21.15.26:0/4214553601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7150003b40 con 0x7f71681505b0 2024-01-15T18:09:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.554+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71681f3f60 con 0x7f71681505b0 2024-01-15T18:09:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.555+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71681f4440 con 0x7f71681505b0 2024-01-15T18:09:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.556+0000 7f7165600640 1 -- 172.21.15.26:0/4214553601 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 33) v1 ==== 90967+0+0 (secure 0 0 0) 0x7f7150003ca0 con 0x7f71681505b0 2024-01-15T18:09:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.557+0000 7f7165600640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f714006c280 0x7f714006e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:09:57.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.557+0000 7f7165600640 1 -- 172.21.15.26:0/4214553601 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5850+0+0 (secure 0 0 0) 0x7f7150088140 con 0x7f71681505b0 2024-01-15T18:09:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.558+0000 7f716d000640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f714006c280 0x7f714006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:09:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.559+0000 7f716d000640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f714006c280 0x7f714006e740 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:09:57.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.561+0000 7f7165600640 1 -- 172.21.15.26:0/4214553601 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71500546f0 con 0x7f71681505b0 2024-01-15T18:09:57.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:57.780+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7f71680ce3f0 con 0x7f714006c280 2024-01-15T18:09:57.846 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-01-15T18:09:57.842+0000 7f9819c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:09:57.937 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T18:09:59.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:09:59.440+0000 7f7165600640 1 -- 172.21.15.26:0/4214553601 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f7150054ac0 con 0x7f71681505b0 2024-01-15T18:10:00.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.221+0000 7f7165600640 1 -- 172.21.15.26:0/4214553601 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f71680ce3f0 con 0x7f714006c280 2024-01-15T18:10:00.225 INFO:teuthology.orchestra.run.smithi026.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:10:00.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.224+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f714006c280 msgr2=0x7f714006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.224+0000 7f716e400640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f714006c280 0x7f714006e740 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f7158006fd0 tx=0x7f7158008040 comp rx=0 tx=0).stop 2024-01-15T18:10:00.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.224+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71681505b0 msgr2=0x7f71681f7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.224+0000 7f716e400640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71681505b0 0x7f71681f7050 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f715000c410 tx=0x7f715000c8d0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.225+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 shutdown_connections 2024-01-15T18:10:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.225+0000 7f716e400640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f714006c280 0x7f714006e740 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.225+0000 7f716e400640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f716815d8a0 0x7f7168153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.225+0000 7f716e400640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f71681505b0 0x7f71681f7050 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.225+0000 7f716e400640 1 --2- 172.21.15.26:0/4214553601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f71680b3b20 0x7f71681f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.225+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 >> 172.21.15.26:0/4214553601 conn(0x7f716806a7d0 msgr2=0x7f71680ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.225+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 shutdown_connections 2024-01-15T18:10:00.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.225+0000 7f716e400640 1 -- 172.21.15.26:0/4214553601 wait complete. 2024-01-15T18:10:00.249 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:10:00.249 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:10:00.363 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:10:00.482 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:10:00.482 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:10:00.482 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:00.544 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:10:00.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.771+0000 7f28f6400640 1 Processor -- start 2024-01-15T18:10:00.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.772+0000 7f28f6400640 1 -- start start 2024-01-15T18:10:00.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.773+0000 7f28f6400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f28f00cf9a0 0x7f28f015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.773+0000 7f28f6400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28f015e1e0 0x7f28f00702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.773+0000 7f28f6400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f00b1ee0 con 0x7f28f0143ae0 2024-01-15T18:10:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.773+0000 7f28f6400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f00b2780 con 0x7f28f015e1e0 2024-01-15T18:10:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.773+0000 7f28f6400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f015e990 con 0x7f28f00cf9a0 2024-01-15T18:10:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.773+0000 7f28efe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28f015e1e0 0x7f28f00702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.773+0000 7f28efe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28f015e1e0 0x7f28f00702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59554/0 (socket says 172.21.15.26:59554) 2024-01-15T18:10:00.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.773+0000 7f28efe00640 1 -- 172.21.15.26:0/231640235 learned_addr learned my addr 172.21.15.26:0/231640235 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.774+0000 7f28eea00640 1 --2- 172.21.15.26:0/231640235 >> v2:172.21.15.124:3300/0 conn(0x7f28f00cf9a0 0x7f28f015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.774+0000 7f28efe00640 1 -- 172.21.15.26:0/231640235 >> v2:172.21.15.124:3300/0 conn(0x7f28f00cf9a0 msgr2=0x7f28f015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.774+0000 7f28efe00640 1 --2- 172.21.15.26:0/231640235 >> v2:172.21.15.124:3300/0 conn(0x7f28f00cf9a0 0x7f28f015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.774+0000 7f28efe00640 1 -- 172.21.15.26:0/231640235 >> v1:172.21.15.124:6789/0 conn(0x7f28f0143ae0 legacy=0x7f28f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.774+0000 7f28efe00640 1 -- 172.21.15.26:0/231640235 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f00b2130 con 0x7f28f015e1e0 2024-01-15T18:10:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.774+0000 7f28eea00640 1 --2- 172.21.15.26:0/231640235 >> v2:172.21.15.124:3300/0 conn(0x7f28f00cf9a0 0x7f28f015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:00.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.775+0000 7f28efe00640 1 --2- 172.21.15.26:0/231640235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 0x7f28f00702c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f28f00cb830 tx=0x7f28e802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde8dcda0e577d3f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.775+0000 7f28ee000640 1 -- 172.21.15.26:0/231640235 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8031870 con 0x7f28f015e1e0 2024-01-15T18:10:00.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.775+0000 7f28ee000640 1 -- 172.21.15.26:0/231640235 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e80319d0 con 0x7f28f015e1e0 2024-01-15T18:10:00.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.775+0000 7f28ee000640 1 -- 172.21.15.26:0/231640235 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8031d60 con 0x7f28f015e1e0 2024-01-15T18:10:00.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.776+0000 7f28f6400640 1 -- 172.21.15.26:0/231640235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 msgr2=0x7f28f00702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.776+0000 7f28f6400640 1 --2- 172.21.15.26:0/231640235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 0x7f28f00702c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f28f00cb830 tx=0x7f28e802b910 comp rx=0 tx=0).stop 2024-01-15T18:10:00.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.776+0000 7f28f6400640 1 -- 172.21.15.26:0/231640235 shutdown_connections 2024-01-15T18:10:00.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.776+0000 7f28f6400640 1 --2- 172.21.15.26:0/231640235 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 0x7f28f00702c0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.776+0000 7f28f6400640 1 --2- 172.21.15.26:0/231640235 >> v2:172.21.15.124:3300/0 conn(0x7f28f00cf9a0 0x7f28f015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.776+0000 7f28f6400640 1 -- 172.21.15.26:0/231640235 >> 172.21.15.26:0/231640235 conn(0x7f28f006a800 msgr2=0x7f28f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:00.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.776+0000 7f28f6400640 1 -- 172.21.15.26:0/231640235 shutdown_connections 2024-01-15T18:10:00.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.776+0000 7f28f6400640 1 -- 172.21.15.26:0/231640235 wait complete. 2024-01-15T18:10:00.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.778+0000 7f28f6400640 1 Processor -- start 2024-01-15T18:10:00.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.778+0000 7f28f6400640 1 -- start start 2024-01-15T18:10:00.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28f6400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f00cf9a0 0x7f28f0158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28f6400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f0143ae0 0x7f28f015c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28f6400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 0x7f28f015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28f6400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0145850 con 0x7f28f015e1e0 2024-01-15T18:10:00.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28f6400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0144f40 con 0x7f28f0143ae0 2024-01-15T18:10:00.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28f6400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0136d10 con 0x7f28f00cf9a0 2024-01-15T18:10:00.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28efe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 0x7f28f015cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28efe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 0x7f28f015cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59564/0 (socket says 172.21.15.26:59564) 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28efe00640 1 -- 172.21.15.26:0/318861782 learned_addr learned my addr 172.21.15.26:0/318861782 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28eea00640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f0143ae0 0x7f28f015c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.779+0000 7f28ef400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f00cf9a0 0x7f28f0158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28efe00640 1 -- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f00cf9a0 msgr2=0x7f28f0158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28efe00640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f00cf9a0 0x7f28f0158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28efe00640 1 -- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f0143ae0 msgr2=0x7f28f015c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28efe00640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f0143ae0 0x7f28f015c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28efe00640 1 -- 172.21.15.26:0/318861782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80094d0 con 0x7f28f015e1e0 2024-01-15T18:10:00.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28efe00640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 0x7f28f015cd90 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f28e802b950 tx=0x7f28e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28e7400640 1 -- 172.21.15.26:0/318861782 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e803adf0 con 0x7f28f015e1e0 2024-01-15T18:10:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28e7400640 1 -- 172.21.15.26:0/318861782 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e80443f0 con 0x7f28f015e1e0 2024-01-15T18:10:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28e7400640 1 -- 172.21.15.26:0/318861782 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80447a0 con 0x7f28f015e1e0 2024-01-15T18:10:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.780+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f0145200 con 0x7f28f015e1e0 2024-01-15T18:10:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.781+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f01592d0 con 0x7f28f015e1e0 2024-01-15T18:10:00.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.782+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f01459c0 con 0x7f28f015e1e0 2024-01-15T18:10:00.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.782+0000 7f28e7400640 1 -- 172.21.15.26:0/318861782 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f28e8044900 con 0x7f28f015e1e0 2024-01-15T18:10:00.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.784+0000 7f28e7400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:00.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.784+0000 7f28e7400640 1 -- 172.21.15.26:0/318861782 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f28e802d040 con 0x7f28f015e1e0 2024-01-15T18:10:00.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.784+0000 7f28ef400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:00.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.786+0000 7f28ef400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005520 tx=0x7f28e0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:00.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.789+0000 7f28e7400640 1 -- 172.21.15.26:0/318861782 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28e8077800 con 0x7f28f015e1e0 2024-01-15T18:10:00.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:10:00.827 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:10:00.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.985+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28f00675d0 con 0x7f28f015e1e0 2024-01-15T18:10:00.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.987+0000 7f28e7400640 1 -- 172.21.15.26:0/318861782 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+13096 (secure 0 0 0) 0x7f28e807ae20 con 0x7f28f015e1e0 2024-01-15T18:10:00.991 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:00.991 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":93,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:10:00.190534+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/2740807412":"2024-01-16T18:09:57.843253+0000","172.21.15.124:6835/2740807412":"2024-01-16T18:09:57.843253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 msgr2=0x7f28c806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f28e0005520 tx=0x7f28e0009040 comp rx=0 tx=0).stop 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 msgr2=0x7f28f015cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 0x7f28f015cd90 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f28e802b950 tx=0x7f28e80046c0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 shutdown_connections 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015e1e0 0x7f28f015cd90 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f0143ae0 0x7f28f015c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 --2- 172.21.15.26:0/318861782 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f00cf9a0 0x7f28f0158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 >> 172.21.15.26:0/318861782 conn(0x7f28f006a800 msgr2=0x7f28f006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 shutdown_connections 2024-01-15T18:10:00.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:00.990+0000 7f28f6400640 1 -- 172.21.15.26:0/318861782 wait complete. 2024-01-15T18:10:01.014 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:10:01.015 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:10:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.239+0000 7fb041000640 1 Processor -- start 2024-01-15T18:10:01.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.239+0000 7fb041000640 1 -- start start 2024-01-15T18:10:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.240+0000 7fb041000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb03c13e360 0x7fb03c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.240+0000 7fb041000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c15c5f0 con 0x7fb03c13df80 2024-01-15T18:10:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.240+0000 7fb041000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c06c470 con 0x7fb03c15f9c0 2024-01-15T18:10:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.240+0000 7fb041000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c06c750 con 0x7fb03c13e360 2024-01-15T18:10:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.241+0000 7fb039e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb03c13df80 0x7fb03c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55624/0 (socket says 172.21.15.26:55624) 2024-01-15T18:10:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.241+0000 7fb039e00640 1 -- 172.21.15.26:0/2394795895 learned_addr learned my addr 172.21.15.26:0/2394795895 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:01.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.241+0000 7fb039400640 1 --2- 172.21.15.26:0/2394795895 >> v2:172.21.15.26:3300/0 conn(0x7fb03c13e360 0x7fb03c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.241+0000 7fb038a00640 1 -- 172.21.15.26:0/2394795895 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882931188 0 0) 0x7fb03c15c5f0 con 0x7fb03c13df80 2024-01-15T18:10:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.241+0000 7fb039400640 1 -- 172.21.15.26:0/2394795895 >> v1:172.21.15.124:6789/0 conn(0x7fb03c15f9c0 legacy=0x7fb03c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.241+0000 7fb039400640 1 -- 172.21.15.26:0/2394795895 >> v1:172.21.15.26:6789/0 conn(0x7fb03c13df80 legacy=0x7fb03c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.241+0000 7fb039400640 1 -- 172.21.15.26:0/2394795895 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c06ca30 con 0x7fb03c13e360 2024-01-15T18:10:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.242+0000 7fb039400640 1 --2- 172.21.15.26:0/2394795895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c13e360 0x7fb03c0cf290 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb0240097a0 tx=0x7fb02402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6629f7ab81e00b1d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.243+0000 7fb038a00640 1 -- 172.21.15.26:0/2394795895 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024031830 con 0x7fb03c13e360 2024-01-15T18:10:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.243+0000 7fb038a00640 1 -- 172.21.15.26:0/2394795895 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb024031990 con 0x7fb03c13e360 2024-01-15T18:10:01.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.243+0000 7fb038a00640 1 -- 172.21.15.26:0/2394795895 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024031d20 con 0x7fb03c13e360 2024-01-15T18:10:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.243+0000 7fb041000640 1 -- 172.21.15.26:0/2394795895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c13e360 msgr2=0x7fb03c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.243+0000 7fb041000640 1 --2- 172.21.15.26:0/2394795895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c13e360 0x7fb03c0cf290 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fb0240097a0 tx=0x7fb02402b8d0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.243+0000 7fb041000640 1 -- 172.21.15.26:0/2394795895 shutdown_connections 2024-01-15T18:10:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.243+0000 7fb041000640 1 --2- 172.21.15.26:0/2394795895 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c13e360 0x7fb03c0cf290 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.243+0000 7fb041000640 1 -- 172.21.15.26:0/2394795895 >> 172.21.15.26:0/2394795895 conn(0x7fb03c06a800 msgr2=0x7fb03c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.244+0000 7fb041000640 1 -- 172.21.15.26:0/2394795895 shutdown_connections 2024-01-15T18:10:01.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.244+0000 7fb041000640 1 -- 172.21.15.26:0/2394795895 wait complete. 2024-01-15T18:10:01.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.245+0000 7fb041000640 1 Processor -- start 2024-01-15T18:10:01.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.245+0000 7fb041000640 1 -- start start 2024-01-15T18:10:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.246+0000 7fb041000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb03c13df80 0x7fb03c1f17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.246+0000 7fb041000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb03c13e360 0x7fb03c0b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.246+0000 7fb041000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c15f9c0 0x7fb03c0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.246+0000 7fb041000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c14cc90 con 0x7fb03c15f9c0 2024-01-15T18:10:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.246+0000 7fb041000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c14b950 con 0x7fb03c13e360 2024-01-15T18:10:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.246+0000 7fb041000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c14bbf0 con 0x7fb03c13df80 2024-01-15T18:10:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb03a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c15f9c0 0x7fb03c0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb03a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c15f9c0 0x7fb03c0ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59568/0 (socket says 172.21.15.26:59568) 2024-01-15T18:10:01.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb03a800640 1 -- 172.21.15.26:0/1167330687 learned_addr learned my addr 172.21.15.26:0/1167330687 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb039e00640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb03c13df80 0x7fb03c1f17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb03a800640 1 -- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb03c13df80 msgr2=0x7fb03c1f17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb039400640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb03c13e360 0x7fb03c0b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb03a800640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb03c13df80 0x7fb03c1f17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb03a800640 1 -- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb03c13e360 msgr2=0x7fb03c0b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb03a800640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb03c13e360 0x7fb03c0b33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb03a800640 1 -- 172.21.15.26:0/1167330687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb024009400 con 0x7fb03c15f9c0 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb039e00640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb03c13df80 0x7fb03c1f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.247+0000 7fb03a800640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c15f9c0 0x7fb03c0ce000 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c700 tx=0x7fb03400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.248+0000 7fb032000640 1 -- 172.21.15.26:0/1167330687 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03400aa50 con 0x7fb03c15f9c0 2024-01-15T18:10:01.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.248+0000 7fb032000640 1 -- 172.21.15.26:0/1167330687 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03400abb0 con 0x7fb03c15f9c0 2024-01-15T18:10:01.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.248+0000 7fb032000640 1 -- 172.21.15.26:0/1167330687 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03400ce10 con 0x7fb03c15f9c0 2024-01-15T18:10:01.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.248+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c14bed0 con 0x7fb03c15f9c0 2024-01-15T18:10:01.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.248+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c14c490 con 0x7fb03c15f9c0 2024-01-15T18:10:01.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.249+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c0675d0 con 0x7fb03c15f9c0 2024-01-15T18:10:01.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.250+0000 7fb032000640 1 -- 172.21.15.26:0/1167330687 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fb034012420 con 0x7fb03c15f9c0 2024-01-15T18:10:01.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.251+0000 7fb032000640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb01406c220 0x7fb01406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.251+0000 7fb032000640 1 -- 172.21.15.26:0/1167330687 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fb034088b70 con 0x7fb03c15f9c0 2024-01-15T18:10:01.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.252+0000 7fb039e00640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb01406c220 0x7fb01406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.253+0000 7fb039e00640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb01406c220 0x7fb01406e6e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb02c000f60 tx=0x7fb02c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:01.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.256+0000 7fb032000640 1 -- 172.21.15.26:0/1167330687 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb034055430 con 0x7fb03c15f9c0 2024-01-15T18:10:01.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.450+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb03c06df00 con 0x7fb03c15f9c0 2024-01-15T18:10:01.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.452+0000 7fb032000640 1 -- 172.21.15.26:0/1167330687 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0340160a0 con 0x7fb03c15f9c0 2024-01-15T18:10:01.456 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:10:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb01406c220 msgr2=0x7fb01406e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb01406c220 0x7fb01406e6e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb02c000f60 tx=0x7fb02c009040 comp rx=0 tx=0).stop 2024-01-15T18:10:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c15f9c0 msgr2=0x7fb03c0ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c15f9c0 0x7fb03c0ce000 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c700 tx=0x7fb03400cbc0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 shutdown_connections 2024-01-15T18:10:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb01406c220 0x7fb01406e6e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb03c15f9c0 0x7fb03c0ce000 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb03c13e360 0x7fb03c0b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 --2- 172.21.15.26:0/1167330687 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb03c13df80 0x7fb03c1f17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 >> 172.21.15.26:0/1167330687 conn(0x7fb03c06a800 msgr2=0x7fb03c15dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.455+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 shutdown_connections 2024-01-15T18:10:01.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.456+0000 7fb041000640 1 -- 172.21.15.26:0/1167330687 wait complete. 2024-01-15T18:10:01.480 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:10:01.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.693+0000 7fb42e800640 1 Processor -- start 2024-01-15T18:10:01.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.693+0000 7fb42e800640 1 -- start start 2024-01-15T18:10:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.695+0000 7fb42e800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb42813cfa0 0x7fb42815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.695+0000 7fb42e800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 0x7fb42813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.695+0000 7fb42e800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4280cdec0 con 0x7fb4280cd9b0 2024-01-15T18:10:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.695+0000 7fb42e800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4280ce1a0 con 0x7fb42813cfa0 2024-01-15T18:10:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.695+0000 7fb42e800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280ce480 con 0x7fb4280cd530 2024-01-15T18:10:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.695+0000 7fb42de00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb4280cd9b0 0x7fb42815d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55638/0 (socket says 172.21.15.26:55638) 2024-01-15T18:10:01.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.696+0000 7fb42de00640 1 -- 172.21.15.26:0/2272411953 learned_addr learned my addr 172.21.15.26:0/2272411953 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.696+0000 7fb42d400640 1 --2- 172.21.15.26:0/2272411953 >> v2:172.21.15.124:3300/0 conn(0x7fb42813cfa0 0x7fb42815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.696+0000 7fb42ca00640 1 --2- 172.21.15.26:0/2272411953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 0x7fb42813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.696+0000 7fb427e00640 1 -- 172.21.15.26:0/2272411953 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179913389 0 0) 0x7fb4280cdec0 con 0x7fb4280cd9b0 2024-01-15T18:10:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.696+0000 7fb427e00640 1 -- 172.21.15.26:0/2272411953 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb408003520 con 0x7fb4280cd9b0 2024-01-15T18:10:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.696+0000 7fb42d400640 1 -- 172.21.15.26:0/2272411953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 msgr2=0x7fb42813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.696+0000 7fb42d400640 1 --2- 172.21.15.26:0/2272411953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 0x7fb42813f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.696+0000 7fb42d400640 1 -- 172.21.15.26:0/2272411953 >> v1:172.21.15.26:6789/0 conn(0x7fb4280cd9b0 legacy=0x7fb42815d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.696+0000 7fb42d400640 1 -- 172.21.15.26:0/2272411953 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280ce760 con 0x7fb42813cfa0 2024-01-15T18:10:01.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.698+0000 7fb42d400640 1 --2- 172.21.15.26:0/2272411953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb42813cfa0 0x7fb42815afb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb4180097a0 tx=0x7fb41802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aac4513dd2590cb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:01.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.698+0000 7fb427e00640 1 -- 172.21.15.26:0/2272411953 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418031780 con 0x7fb42813cfa0 2024-01-15T18:10:01.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.698+0000 7fb427e00640 1 -- 172.21.15.26:0/2272411953 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4180318e0 con 0x7fb42813cfa0 2024-01-15T18:10:01.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.698+0000 7fb427e00640 1 -- 172.21.15.26:0/2272411953 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418031c70 con 0x7fb42813cfa0 2024-01-15T18:10:01.702 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.699+0000 7fb42e800640 1 -- 172.21.15.26:0/2272411953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb42813cfa0 msgr2=0x7fb42815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.699+0000 7fb42e800640 1 --2- 172.21.15.26:0/2272411953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb42813cfa0 0x7fb42815afb0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb4180097a0 tx=0x7fb41802b820 comp rx=0 tx=0).stop 2024-01-15T18:10:01.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.699+0000 7fb42e800640 1 -- 172.21.15.26:0/2272411953 shutdown_connections 2024-01-15T18:10:01.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.699+0000 7fb42e800640 1 --2- 172.21.15.26:0/2272411953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 0x7fb42813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.699+0000 7fb42e800640 1 --2- 172.21.15.26:0/2272411953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb42813cfa0 0x7fb42815afb0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.699+0000 7fb42e800640 1 -- 172.21.15.26:0/2272411953 >> 172.21.15.26:0/2272411953 conn(0x7fb42806a800 msgr2=0x7fb42806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:01.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.699+0000 7fb42e800640 1 -- 172.21.15.26:0/2272411953 shutdown_connections 2024-01-15T18:10:01.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.699+0000 7fb42e800640 1 -- 172.21.15.26:0/2272411953 wait complete. 2024-01-15T18:10:01.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.701+0000 7fb42e800640 1 Processor -- start 2024-01-15T18:10:01.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.702+0000 7fb42e800640 1 -- start start 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.703+0000 7fb42e800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 0x7fb428159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.703+0000 7fb42e800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4280cd9b0 0x7fb428150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.703+0000 7fb42e800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42813cfa0 0x7fb428156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.703+0000 7fb42e800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42815a050 con 0x7fb42813cfa0 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.703+0000 7fb42e800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb428151300 con 0x7fb4280cd9b0 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.703+0000 7fb42e800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4281575d0 con 0x7fb4280cd530 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.703+0000 7fb42de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42813cfa0 0x7fb428156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.703+0000 7fb42de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42813cfa0 0x7fb428156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:59584/0 (socket says 172.21.15.26:59584) 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.703+0000 7fb42de00640 1 -- 172.21.15.26:0/1277218398 learned_addr learned my addr 172.21.15.26:0/1277218398 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.704+0000 7fb42ca00640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4280cd9b0 0x7fb428150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.704+0000 7fb42d400640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 0x7fb428159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.704+0000 7fb42de00640 1 -- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 msgr2=0x7fb428159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.704+0000 7fb42de00640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 0x7fb428159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.704+0000 7fb42de00640 1 -- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4280cd9b0 msgr2=0x7fb428150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.704+0000 7fb42de00640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4280cd9b0 0x7fb428150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.704+0000 7fb42de00640 1 -- 172.21.15.26:0/1277218398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb418009400 con 0x7fb42813cfa0 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.704+0000 7fb42de00640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42813cfa0 0x7fb428156d80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fb42000cb40 tx=0x7fb42000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.705+0000 7fb425600640 1 -- 172.21.15.26:0/1277218398 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42000aca0 con 0x7fb42813cfa0 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.705+0000 7fb425600640 1 -- 172.21.15.26:0/1277218398 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42000ae00 con 0x7fb42813cfa0 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.705+0000 7fb425600640 1 -- 172.21.15.26:0/1277218398 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420012620 con 0x7fb42813cfa0 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.705+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4281578b0 con 0x7fb42813cfa0 2024-01-15T18:10:01.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.705+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4281595f0 con 0x7fb42813cfa0 2024-01-15T18:10:01.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.706+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb428159810 con 0x7fb42813cfa0 2024-01-15T18:10:01.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.707+0000 7fb425600640 1 -- 172.21.15.26:0/1277218398 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fb420007070 con 0x7fb42813cfa0 2024-01-15T18:10:01.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.708+0000 7fb425600640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40006c1f0 0x7fb40006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:01.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.708+0000 7fb425600640 1 -- 172.21.15.26:0/1277218398 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fb420088670 con 0x7fb42813cfa0 2024-01-15T18:10:01.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.709+0000 7fb42d400640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40006c1f0 0x7fb40006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.710+0000 7fb42d400640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40006c1f0 0x7fb40006e6b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb418036000 tx=0x7fb41803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.713+0000 7fb425600640 1 -- 172.21.15.26:0/1277218398 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb420054fe0 con 0x7fb42813cfa0 2024-01-15T18:10:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.924+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb428067450 con 0x7fb42813cfa0 2024-01-15T18:10:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.925+0000 7fb425600640 1 -- 172.21.15.26:0/1277218398 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) v1 ==== 80+0+124 (secure 0 0 0) 0x7fb420017080 con 0x7fb42813cfa0 2024-01-15T18:10:01.929 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6837/2740807412 2024-01-16T18:09:57.843253+0000 2024-01-15T18:10:01.929 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6835/2740807412 2024-01-16T18:09:57.843253+0000 2024-01-15T18:10:01.929 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:10:01.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40006c1f0 msgr2=0x7fb40006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40006c1f0 0x7fb40006e6b0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb418036000 tx=0x7fb41803d040 comp rx=0 tx=0).stop 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42813cfa0 msgr2=0x7fb428156d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42813cfa0 0x7fb428156d80 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fb42000cb40 tx=0x7fb42000a060 comp rx=0 tx=0).stop 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 shutdown_connections 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb40006c1f0 0x7fb40006e6b0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb42813cfa0 0x7fb428156d80 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb4280cd9b0 0x7fb428150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 --2- 172.21.15.26:0/1277218398 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb4280cd530 0x7fb428159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 >> 172.21.15.26:0/1277218398 conn(0x7fb42806a800 msgr2=0x7fb4280b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 shutdown_connections 2024-01-15T18:10:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:01.928+0000 7fb42e800640 1 -- 172.21.15.26:0/1277218398 wait complete. 2024-01-15T18:10:01.952 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-01-15T18:10:01.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:01.952 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:10:02.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[69208]: fuse finished with error 0 and tester_r 0 2024-01-15T18:10:02.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:10:02.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:02.046 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:10:02.258 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:10:02.259 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:02.316 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:10:02.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.543+0000 7f7412e00640 1 Processor -- start 2024-01-15T18:10:02.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.543+0000 7f7412e00640 1 -- start start 2024-01-15T18:10:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.545+0000 7f7412e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f740c0cc500 0x7f740c0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.545+0000 7f7412e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c070e50 con 0x7f740c143c40 2024-01-15T18:10:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.545+0000 7f7412e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740c0ccea0 con 0x7f740c06d160 2024-01-15T18:10:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.545+0000 7f7412e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0723c0 con 0x7f740c0cc500 2024-01-15T18:10:02.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.545+0000 7f7411a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f740c143c40 0x7f740c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59488/0 (socket says 172.21.15.26:59488) 2024-01-15T18:10:02.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.545+0000 7f7411a00640 1 -- 172.21.15.26:0/90833159 learned_addr learned my addr 172.21.15.26:0/90833159 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.545+0000 7f7411000640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f740c0cc500 0x7f740c0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.545+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454708448 0 0) 0x7f740c070e50 con 0x7f740c143c40 2024-01-15T18:10:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.545+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73ec003520 con 0x7f740c143c40 2024-01-15T18:10:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2245269385 0 0) 0x7f73ec003520 con 0x7f740c143c40 2024-01-15T18:10:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f740c0cc500 msgr2=0x7f740c0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f740c0cc500 0x7f740c0cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 >> v1:172.21.15.124:6789/0 conn(0x7f740c06d160 legacy=0x7f740c06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c144020 con 0x7f740c143c40 2024-01-15T18:10:02.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f73fc0019f0 con 0x7f740c143c40 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 >> v1:172.21.15.26:6789/0 conn(0x7f740c143c40 legacy=0x7f740c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f740c0cc500 0x7f73ec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ec003f70 0x7f73ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73ec004d00 0x7f73ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0723c0 con 0x7f73ec003f70 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec00bb70 con 0x7f73ec004d00 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73ec00bd30 con 0x7f740c0cc500 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.546+0000 7f7411000640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ec003f70 0x7f73ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.547+0000 7f7411a00640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f740c0cc500 0x7f73ec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.547+0000 7f7412400640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73ec004d00 0x7f73ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.547+0000 7f7411000640 1 -- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f740c0cc500 msgr2=0x7f73ec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.547+0000 7f7411000640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f740c0cc500 0x7f73ec0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.547+0000 7f7411000640 1 -- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73ec004d00 msgr2=0x7f73ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.547+0000 7f7411000640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73ec004d00 0x7f73ec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.547+0000 7f7411000640 1 -- 172.21.15.26:0/90833159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec003520 con 0x7f73ec003f70 2024-01-15T18:10:02.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.547+0000 7f7412400640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f73ec004d00 0x7f73ec005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.548+0000 7f7411000640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ec003f70 0x7f73ec0047c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009ac0 tx=0x7f73ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ef92e2a36d166c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.548+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f402c6b0 con 0x7f73ec003f70 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.548+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f402c810 con 0x7f73ec003f70 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.548+0000 7f740be00640 1 -- 172.21.15.26:0/90833159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f402cba0 con 0x7f73ec003f70 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.549+0000 7f7412e00640 1 -- 172.21.15.26:0/90833159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ec003f70 msgr2=0x7f73ec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.549+0000 7f7412e00640 1 --2- 172.21.15.26:0/90833159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f73ec003f70 0x7f73ec0047c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009ac0 tx=0x7f73ec0057f0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.549+0000 7f7412400640 1 -- 172.21.15.26:0/90833159 reap_dead start 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.549+0000 7f7412e00640 1 -- 172.21.15.26:0/90833159 shutdown_connections 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.549+0000 7f7412e00640 1 -- 172.21.15.26:0/90833159 >> 172.21.15.26:0/90833159 conn(0x7f740c06a800 msgr2=0x7f740c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.549+0000 7f7412e00640 1 -- 172.21.15.26:0/90833159 shutdown_connections 2024-01-15T18:10:02.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.549+0000 7f7412e00640 1 -- 172.21.15.26:0/90833159 wait complete. 2024-01-15T18:10:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.551+0000 7f7412e00640 1 Processor -- start 2024-01-15T18:10:02.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.551+0000 7f7412e00640 1 -- start start 2024-01-15T18:10:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.552+0000 7f7412e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73ec003f70 0x7f740c1f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.552+0000 7f7412e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f740c1f3990 0x7f740c0ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.552+0000 7f7412e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f740c0abe90 0x7f740c0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.552+0000 7f7412e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0ad250 con 0x7f740c0abe90 2024-01-15T18:10:02.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.552+0000 7f7412e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740c15a4e0 con 0x7f740c1f3990 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.552+0000 7f7412e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740c0ac310 con 0x7f73ec003f70 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.552+0000 7f7412400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f740c0abe90 0x7f740c0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7412400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f740c0abe90 0x7f740c0ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40274/0 (socket says 172.21.15.26:40274) 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7412400640 1 -- 172.21.15.26:0/1275951186 learned_addr learned my addr 172.21.15.26:0/1275951186 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7411000640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f740c1f3990 0x7f740c0ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7411a00640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73ec003f70 0x7f740c1f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7412400640 1 -- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73ec003f70 msgr2=0x7f740c1f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7412400640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73ec003f70 0x7f740c1f0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7412400640 1 -- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f740c1f3990 msgr2=0x7f740c0ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7412400640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f740c1f3990 0x7f740c0ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7412400640 1 -- 172.21.15.26:0/1275951186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c0ac5d0 con 0x7f740c0abe90 2024-01-15T18:10:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.553+0000 7f7412400640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f740c0abe90 0x7f740c0ad670 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f740c06e3c0 tx=0x7f740400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.554+0000 7f7409600640 1 -- 172.21.15.26:0/1275951186 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740400b070 con 0x7f740c0abe90 2024-01-15T18:10:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.554+0000 7f7409600640 1 -- 172.21.15.26:0/1275951186 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7404007030 con 0x7f740c0abe90 2024-01-15T18:10:02.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.554+0000 7f7409600640 1 -- 172.21.15.26:0/1275951186 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74040073e0 con 0x7f740c0abe90 2024-01-15T18:10:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.554+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740c0ac8b0 con 0x7f740c0abe90 2024-01-15T18:10:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.554+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740c0ace70 con 0x7f740c0abe90 2024-01-15T18:10:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.555+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740c0ad390 con 0x7f740c0abe90 2024-01-15T18:10:02.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.556+0000 7f7409600640 1 -- 172.21.15.26:0/1275951186 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f7404007540 con 0x7f740c0abe90 2024-01-15T18:10:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.557+0000 7f7409600640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f73e406c1f0 0x7f73e406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.557+0000 7f7409600640 1 -- 172.21.15.26:0/1275951186 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f74040881c0 con 0x7f740c0abe90 2024-01-15T18:10:02.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.557+0000 7f7411a00640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f73e406c1f0 0x7f73e406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:02.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.558+0000 7f7411a00640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f73e406c1f0 0x7f73e406e6b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f73fc002230 tx=0x7f73fc001290 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:02.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.561+0000 7f7409600640 1 -- 172.21.15.26:0/1275951186 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f740c0ad390 con 0x7f740c0abe90 2024-01-15T18:10:02.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.758+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f740c0ad390 con 0x7f740c0abe90 2024-01-15T18:10:02.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.759+0000 7f7409600640 1 -- 172.21.15.26:0/1275951186 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+13096 (secure 0 0 0) 0x7f740c0ad390 con 0x7f740c0abe90 2024-01-15T18:10:02.763 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:02.763 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":93,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:10:00.190534+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/2740807412":"2024-01-16T18:09:57.843253+0000","172.21.15.124:6835/2740807412":"2024-01-16T18:09:57.843253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:10:02.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f73e406c1f0 msgr2=0x7f73e406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f73e406c1f0 0x7f73e406e6b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f73fc002230 tx=0x7f73fc001290 comp rx=0 tx=0).stop 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f740c0abe90 msgr2=0x7f740c0ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f740c0abe90 0x7f740c0ad670 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f740c06e3c0 tx=0x7f740400d890 comp rx=0 tx=0).stop 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 shutdown_connections 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f73e406c1f0 0x7f73e406e6b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f740c0abe90 0x7f740c0ad670 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f740c1f3990 0x7f740c0ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 --2- 172.21.15.26:0/1275951186 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f73ec003f70 0x7f740c1f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 >> 172.21.15.26:0/1275951186 conn(0x7f740c06a800 msgr2=0x7f740c140250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 shutdown_connections 2024-01-15T18:10:02.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:02.762+0000 7f7412e00640 1 -- 172.21.15.26:0/1275951186 wait complete. 2024-01-15T18:10:02.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:10:02.785 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:10:03.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.007+0000 7f2181800640 1 Processor -- start 2024-01-15T18:10:03.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.008+0000 7f2181800640 1 -- start start 2024-01-15T18:10:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.009+0000 7f2181800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c09a170 0x7f217c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.009+0000 7f2181800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f217c06c6b0 0x7f217c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.009+0000 7f2181800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c06cea0 con 0x7f217c0cd240 2024-01-15T18:10:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.009+0000 7f2181800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c06cbc0 con 0x7f217c06c6b0 2024-01-15T18:10:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.009+0000 7f2181800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c06f350 con 0x7f217c09a170 2024-01-15T18:10:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.010+0000 7f217aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f217c0cd240 0x7f217c099a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59494/0 (socket says 172.21.15.26:59494) 2024-01-15T18:10:03.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.010+0000 7f217aa00640 1 -- 172.21.15.26:0/689497263 learned_addr learned my addr 172.21.15.26:0/689497263 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.010+0000 7f217a000640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c09a170 0x7f217c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.010+0000 7f217b400640 1 --2- 172.21.15.26:0/689497263 >> v2:172.21.15.124:3300/0 conn(0x7f217c06c6b0 0x7f217c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.010+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091270802 0 0) 0x7f217c06cea0 con 0x7f217c0cd240 2024-01-15T18:10:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.010+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f215c003520 con 0x7f217c0cd240 2024-01-15T18:10:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2139369672 0 0) 0x7f215c003520 con 0x7f217c0cd240 2024-01-15T18:10:03.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c09a170 msgr2=0x7f217c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c09a170 0x7f217c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 >> v2:172.21.15.124:3300/0 conn(0x7f217c06c6b0 msgr2=0x7f217c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 --2- 172.21.15.26:0/689497263 >> v2:172.21.15.124:3300/0 conn(0x7f217c06c6b0 0x7f217c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c06f630 con 0x7f217c0cd240 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f216c0019f0 con 0x7f217c0cd240 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 >> v1:172.21.15.26:6789/0 conn(0x7f217c0cd240 legacy=0x7f217c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f215c003970 0x7f215c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c09a170 0x7f215c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c004d00 0x7f215c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c06cbc0 con 0x7f215c003970 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c06f350 con 0x7f215c004d00 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215c00b980 con 0x7f217c09a170 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f217aa00640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f215c003970 0x7f215c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.011+0000 7f217a000640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c09a170 0x7f215c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.012+0000 7f217aa00640 1 -- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c09a170 msgr2=0x7f215c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.012+0000 7f217b400640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c004d00 0x7f215c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.012+0000 7f217aa00640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c09a170 0x7f215c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.012+0000 7f217aa00640 1 -- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c004d00 msgr2=0x7f215c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.012+0000 7f217aa00640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c004d00 0x7f215c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.012+0000 7f217aa00640 1 -- 172.21.15.26:0/689497263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c003520 con 0x7f215c003970 2024-01-15T18:10:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.013+0000 7f217aa00640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f215c003970 0x7f215c003d90 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f216c00a790 tx=0x7f215c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fd9d1a0ab57ce4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.013+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c034630 con 0x7f215c003970 2024-01-15T18:10:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.013+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c02f070 con 0x7f215c003970 2024-01-15T18:10:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.013+0000 7f2179600640 1 -- 172.21.15.26:0/689497263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216c0349f0 con 0x7f215c003970 2024-01-15T18:10:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.014+0000 7f2181800640 1 -- 172.21.15.26:0/689497263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f215c003970 msgr2=0x7f215c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.014+0000 7f2181800640 1 --2- 172.21.15.26:0/689497263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f215c003970 0x7f215c003d90 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f216c00a790 tx=0x7f215c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.014+0000 7f217b400640 1 -- 172.21.15.26:0/689497263 reap_dead start 2024-01-15T18:10:03.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.014+0000 7f2181800640 1 -- 172.21.15.26:0/689497263 shutdown_connections 2024-01-15T18:10:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.014+0000 7f2181800640 1 -- 172.21.15.26:0/689497263 >> 172.21.15.26:0/689497263 conn(0x7f217c0b2d00 msgr2=0x7f217c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.014+0000 7f2181800640 1 -- 172.21.15.26:0/689497263 shutdown_connections 2024-01-15T18:10:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.014+0000 7f2181800640 1 -- 172.21.15.26:0/689497263 wait complete. 2024-01-15T18:10:03.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.016+0000 7f2181800640 1 Processor -- start 2024-01-15T18:10:03.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.016+0000 7f2181800640 1 -- start start 2024-01-15T18:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.017+0000 7f2181800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c003970 0x7f217c0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.017+0000 7f2181800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f217c0b50b0 0x7f217c0c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.017+0000 7f2181800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c0ca0f0 0x7f217c0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.017+0000 7f2181800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b6460 con 0x7f217c0b50b0 2024-01-15T18:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.017+0000 7f2181800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b5530 con 0x7f215c003970 2024-01-15T18:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.017+0000 7f2181800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0caab0 con 0x7f217c0ca0f0 2024-01-15T18:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.017+0000 7f217a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f217c0b50b0 0x7f217c0c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f217c0b50b0 0x7f217c0c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40304/0 (socket says 172.21.15.26:40304) 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217a000640 1 -- 172.21.15.26:0/108443466 learned_addr learned my addr 172.21.15.26:0/108443466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217aa00640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c003970 0x7f217c0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217b400640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c0ca0f0 0x7f217c0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217a000640 1 -- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c0ca0f0 msgr2=0x7f217c0ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217a000640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c0ca0f0 0x7f217c0ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217a000640 1 -- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c003970 msgr2=0x7f217c0c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217a000640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c003970 0x7f217c0c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217a000640 1 -- 172.21.15.26:0/108443466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0b57e0 con 0x7f217c0b50b0 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217aa00640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c003970 0x7f217c0c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.018+0000 7f217a000640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f217c0b50b0 0x7f217c0c9bb0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f2164011600 tx=0x7f2164011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.019+0000 7f2172a00640 1 -- 172.21.15.26:0/108443466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400f920 con 0x7f217c0b50b0 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.019+0000 7f2172a00640 1 -- 172.21.15.26:0/108443466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216400fa80 con 0x7f217c0b50b0 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.019+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c0b5ac0 con 0x7f217c0b50b0 2024-01-15T18:10:03.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.019+0000 7f2172a00640 1 -- 172.21.15.26:0/108443466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400fdf0 con 0x7f217c0b50b0 2024-01-15T18:10:03.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.019+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c0b6080 con 0x7f217c0b50b0 2024-01-15T18:10:03.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.020+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c0675d0 con 0x7f217c0b50b0 2024-01-15T18:10:03.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.021+0000 7f2172a00640 1 -- 172.21.15.26:0/108443466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f2164015460 con 0x7f217c0b50b0 2024-01-15T18:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.022+0000 7f2172a00640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215406c1f0 0x7f215406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.022+0000 7f2172a00640 1 -- 172.21.15.26:0/108443466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f2164089260 con 0x7f217c0b50b0 2024-01-15T18:10:03.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.023+0000 7f217aa00640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215406c1f0 0x7f215406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.024+0000 7f217aa00640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215406c1f0 0x7f215406e6b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f216c034000 tx=0x7f216c009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:03.030 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.027+0000 7f2172a00640 1 -- 172.21.15.26:0/108443466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2164055830 con 0x7f217c0b50b0 2024-01-15T18:10:03.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.223+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f217c0b2a70 con 0x7f217c0b50b0 2024-01-15T18:10:03.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.224+0000 7f2172a00640 1 -- 172.21.15.26:0/108443466 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2164058e50 con 0x7f217c0b50b0 2024-01-15T18:10:03.227 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:10:03.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.226+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215406c1f0 msgr2=0x7f215406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.226+0000 7f2181800640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215406c1f0 0x7f215406e6b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f216c034000 tx=0x7f216c009030 comp rx=0 tx=0).stop 2024-01-15T18:10:03.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.226+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f217c0b50b0 msgr2=0x7f217c0c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.226+0000 7f2181800640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f217c0b50b0 0x7f217c0c9bb0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f2164011600 tx=0x7f2164011ac0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.227+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 shutdown_connections 2024-01-15T18:10:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.227+0000 7f2181800640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f217c0ca0f0 0x7f217c0ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.227+0000 7f2181800640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f215406c1f0 0x7f215406e6b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.227+0000 7f2181800640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f217c0b50b0 0x7f217c0c9bb0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.227+0000 7f2181800640 1 --2- 172.21.15.26:0/108443466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f215c003970 0x7f217c0c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.227+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 >> 172.21.15.26:0/108443466 conn(0x7f217c0b2d00 msgr2=0x7f217c0733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.227+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 shutdown_connections 2024-01-15T18:10:03.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.227+0000 7f2181800640 1 -- 172.21.15.26:0/108443466 wait complete. 2024-01-15T18:10:03.250 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.466+0000 7fe50d400640 1 Processor -- start 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.466+0000 7fe50d400640 1 -- start start 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.467+0000 7fe50d400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe50813cfa0 0x7fe50815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe50d400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe5080cd530 0x7fe50813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe50d400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5080cd9b0 0x7fe50813fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe50d400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5080cdec0 con 0x7fe5080cd9b0 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe50d400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5080ce1a0 con 0x7fe5080cd530 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe50d400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080ce480 con 0x7fe50813cfa0 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe50ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5080cd9b0 0x7fe50813fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe50ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe5080cd9b0 0x7fe50813fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40318/0 (socket says 172.21.15.26:40318) 2024-01-15T18:10:03.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe50ca00640 1 -- 172.21.15.26:0/2161862531 learned_addr learned my addr 172.21.15.26:0/2161862531 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:03.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe507e00640 1 --2- 172.21.15.26:0/2161862531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe50813cfa0 0x7fe50815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.468+0000 7fe507400640 1 --2- 172.21.15.26:0/2161862531 >> v2:172.21.15.124:3300/0 conn(0x7fe5080cd530 0x7fe50813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.469+0000 7fe50ca00640 1 -- 172.21.15.26:0/2161862531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe50813cfa0 msgr2=0x7fe50815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:03.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.469+0000 7fe50ca00640 1 --2- 172.21.15.26:0/2161862531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe50813cfa0 0x7fe50815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.469+0000 7fe50ca00640 1 -- 172.21.15.26:0/2161862531 >> v2:172.21.15.124:3300/0 conn(0x7fe5080cd530 msgr2=0x7fe50813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:03.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.469+0000 7fe50ca00640 1 --2- 172.21.15.26:0/2161862531 >> v2:172.21.15.124:3300/0 conn(0x7fe5080cd530 0x7fe50813f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.469+0000 7fe50ca00640 1 -- 172.21.15.26:0/2161862531 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5080ce760 con 0x7fe5080cd9b0 2024-01-15T18:10:03.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.469+0000 7fe507400640 1 --2- 172.21.15.26:0/2161862531 >> v2:172.21.15.124:3300/0 conn(0x7fe5080cd530 0x7fe50813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:03.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.470+0000 7fe50ca00640 1 --2- 172.21.15.26:0/2161862531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 0x7fe50813fae0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fe50806e2a0 tx=0x7fe50002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0295f85f7ed9f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:03.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.471+0000 7fe506a00640 1 -- 172.21.15.26:0/2161862531 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50002c9e0 con 0x7fe5080cd9b0 2024-01-15T18:10:03.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.471+0000 7fe506a00640 1 -- 172.21.15.26:0/2161862531 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50002cb40 con 0x7fe5080cd9b0 2024-01-15T18:10:03.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.471+0000 7fe506a00640 1 -- 172.21.15.26:0/2161862531 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5000343f0 con 0x7fe5080cd9b0 2024-01-15T18:10:03.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.471+0000 7fe50d400640 1 -- 172.21.15.26:0/2161862531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 msgr2=0x7fe50813fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.471+0000 7fe50d400640 1 --2- 172.21.15.26:0/2161862531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 0x7fe50813fae0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fe50806e2a0 tx=0x7fe50002be40 comp rx=0 tx=0).stop 2024-01-15T18:10:03.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.472+0000 7fe50d400640 1 -- 172.21.15.26:0/2161862531 shutdown_connections 2024-01-15T18:10:03.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.472+0000 7fe50d400640 1 --2- 172.21.15.26:0/2161862531 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 0x7fe50813fae0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.472+0000 7fe50d400640 1 --2- 172.21.15.26:0/2161862531 >> v2:172.21.15.124:3300/0 conn(0x7fe5080cd530 0x7fe50813f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.472+0000 7fe50d400640 1 --2- 172.21.15.26:0/2161862531 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe50813cfa0 0x7fe50815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.472+0000 7fe50d400640 1 -- 172.21.15.26:0/2161862531 >> 172.21.15.26:0/2161862531 conn(0x7fe50806a800 msgr2=0x7fe50806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:03.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.472+0000 7fe50d400640 1 -- 172.21.15.26:0/2161862531 shutdown_connections 2024-01-15T18:10:03.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.472+0000 7fe50d400640 1 -- 172.21.15.26:0/2161862531 wait complete. 2024-01-15T18:10:03.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.474+0000 7fe50d400640 1 Processor -- start 2024-01-15T18:10:03.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.474+0000 7fe50d400640 1 -- start start 2024-01-15T18:10:03.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe50d400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5080cd530 0x7fe50813c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe50d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 0x7fe508154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe50d400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe50813cfa0 0x7fe50815c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe50d400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe508155390 con 0x7fe5080cd9b0 2024-01-15T18:10:03.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe50d400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe508155670 con 0x7fe50813cfa0 2024-01-15T18:10:03.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe50d400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe508155950 con 0x7fe5080cd530 2024-01-15T18:10:03.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe507400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 0x7fe508154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe507400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 0x7fe508154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40328/0 (socket says 172.21.15.26:40328) 2024-01-15T18:10:03.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe507400640 1 -- 172.21.15.26:0/2822312089 learned_addr learned my addr 172.21.15.26:0/2822312089 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.475+0000 7fe50ca00640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe50813cfa0 0x7fe50815c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe507e00640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5080cd530 0x7fe50813c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe507400640 1 -- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5080cd530 msgr2=0x7fe50813c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe507400640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5080cd530 0x7fe50813c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe507400640 1 -- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe50813cfa0 msgr2=0x7fe50815c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe507400640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe50813cfa0 0x7fe50815c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe507400640 1 -- 172.21.15.26:0/2822312089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe500009b30 con 0x7fe5080cd9b0 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe507e00640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5080cd530 0x7fe50813c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe507400640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 0x7fe508154a90 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000c7f0 tx=0x7fe4f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe4ffe00640 1 -- 172.21.15.26:0/2822312089 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0007a30 con 0x7fe5080cd9b0 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe4ffe00640 1 -- 172.21.15.26:0/2822312089 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f0007b90 con 0x7fe5080cd9b0 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe4ffe00640 1 -- 172.21.15.26:0/2822312089 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000ce40 con 0x7fe5080cd9b0 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.476+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe508155c30 con 0x7fe5080cd9b0 2024-01-15T18:10:03.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.477+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe508154fd0 con 0x7fe5080cd9b0 2024-01-15T18:10:03.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.478+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe508067450 con 0x7fe5080cd9b0 2024-01-15T18:10:03.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.479+0000 7fe4ffe00640 1 -- 172.21.15.26:0/2822312089 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fe4f00113f0 con 0x7fe5080cd9b0 2024-01-15T18:10:03.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.480+0000 7fe4ffe00640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4e006c220 0x7fe4e006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:03.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.480+0000 7fe4ffe00640 1 -- 172.21.15.26:0/2822312089 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fe4f00887d0 con 0x7fe5080cd9b0 2024-01-15T18:10:03.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.481+0000 7fe507e00640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4e006c220 0x7fe4e006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:03.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.482+0000 7fe507e00640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4e006c220 0x7fe4e006e6e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8006fd0 tx=0x7fe4f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:03.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.485+0000 7fe4ffe00640 1 -- 172.21.15.26:0/2822312089 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4f00550f0 con 0x7fe5080cd9b0 2024-01-15T18:10:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.692+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe5080cbff0 con 0x7fe5080cd9b0 2024-01-15T18:10:03.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.693+0000 7fe4ffe00640 1 -- 172.21.15.26:0/2822312089 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) v1 ==== 80+0+124 (secure 0 0 0) 0x7fe4f0058710 con 0x7fe5080cd9b0 2024-01-15T18:10:03.697 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6837/2740807412 2024-01-16T18:09:57.843253+0000 2024-01-15T18:10:03.697 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6835/2740807412 2024-01-16T18:09:57.843253+0000 2024-01-15T18:10:03.697 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:10:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.695+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4e006c220 msgr2=0x7fe4e006e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.695+0000 7fe50d400640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4e006c220 0x7fe4e006e6e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8006fd0 tx=0x7fe4f8008040 comp rx=0 tx=0).stop 2024-01-15T18:10:03.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.695+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 msgr2=0x7fe508154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.695+0000 7fe50d400640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 0x7fe508154a90 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000c7f0 tx=0x7fe4f000cbf0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.696+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 shutdown_connections 2024-01-15T18:10:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.696+0000 7fe50d400640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe4e006c220 0x7fe4e006e6e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.696+0000 7fe50d400640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe50813cfa0 0x7fe50815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.696+0000 7fe50d400640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe5080cd9b0 0x7fe508154a90 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.696+0000 7fe50d400640 1 --2- 172.21.15.26:0/2822312089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe5080cd530 0x7fe50813c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.696+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 >> 172.21.15.26:0/2822312089 conn(0x7fe50806a800 msgr2=0x7fe5080aeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.696+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 shutdown_connections 2024-01-15T18:10:03.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:03.696+0000 7fe50d400640 1 -- 172.21.15.26:0/2822312089 wait complete. 2024-01-15T18:10:03.717 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi124.front.sepia.ceph.com... 2024-01-15T18:10:03.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:03.718 DEBUG:teuthology.orchestra.run.smithi124:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:10:03.811 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[46583]: fuse finished with error 0 and tester_r 0 2024-01-15T18:10:03.815 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:10:03.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:03.816 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:10:04.000 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:10:04.000 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:04.061 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:10:04.061 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:10:04.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:04.061 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:10:04.115 INFO:teuthology.orchestra.run.smithi026.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:10:04.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:10:04.117 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:10:04.117 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:04.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:10:04.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:10:04.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:04.174 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:10:04.227 INFO:teuthology.orchestra.run.smithi124.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:10:04.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:10:04.228 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:10:04.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.454+0000 7feda9000640 1 Processor -- start 2024-01-15T18:10:04.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.454+0000 7feda9000640 1 -- start start 2024-01-15T18:10:04.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.455+0000 7feda9000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7feda40ce6a0 0x7feda40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.455+0000 7feda9000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda40cca90 con 0x7feda415ea50 2024-01-15T18:10:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.455+0000 7feda9000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda40ccd70 con 0x7feda415e670 2024-01-15T18:10:04.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.455+0000 7feda9000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda40cf170 con 0x7feda40ce6a0 2024-01-15T18:10:04.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda3e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7feda415ea50 0x7feda40cd620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59500/0 (socket says 172.21.15.26:59500) 2024-01-15T18:10:04.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda3e00640 1 -- 172.21.15.26:0/4160490781 learned_addr learned my addr 172.21.15.26:0/4160490781 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:04.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2a00640 1 --2- 172.21.15.26:0/4160490781 >> v2:172.21.15.124:3300/0 conn(0x7feda40ce6a0 0x7feda40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271899714 0 0) 0x7feda40cca90 con 0x7feda415ea50 2024-01-15T18:10:04.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed84003520 con 0x7feda415ea50 2024-01-15T18:10:04.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1811215544 0 0) 0x7fed84003520 con 0x7feda415ea50 2024-01-15T18:10:04.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 >> v2:172.21.15.124:3300/0 conn(0x7feda40ce6a0 msgr2=0x7feda40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:04.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 --2- 172.21.15.26:0/4160490781 >> v2:172.21.15.124:3300/0 conn(0x7feda40ce6a0 0x7feda40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 >> v1:172.21.15.124:6789/0 conn(0x7feda415e670 legacy=0x7feda40b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40ceab0 con 0x7feda415ea50 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fed9c0019f0 con 0x7feda415ea50 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 >> v1:172.21.15.26:6789/0 conn(0x7feda415ea50 legacy=0x7feda40cd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 --2- 172.21.15.26:0/4160490781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed84003970 0x7fed84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.456+0000 7feda2000640 1 --2- 172.21.15.26:0/4160490781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7fed84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2000640 1 --2- 172.21.15.26:0/4160490781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed84005180 0x7fed8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40cf170 con 0x7fed840047c0 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8400c020 con 0x7fed84003970 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8400c1e0 con 0x7fed84005180 2024-01-15T18:10:04.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2a00640 1 --2- 172.21.15.26:0/4160490781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7fed84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2a00640 1 -- 172.21.15.26:0/4160490781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed84005180 msgr2=0x7fed8400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2a00640 1 --2- 172.21.15.26:0/4160490781 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fed84005180 0x7fed8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2a00640 1 -- 172.21.15.26:0/4160490781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed84003970 msgr2=0x7fed84003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2a00640 1 --2- 172.21.15.26:0/4160490781 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fed84003970 0x7fed84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda2a00640 1 -- 172.21.15.26:0/4160490781 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed84003520 con 0x7fed840047c0 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.457+0000 7feda3e00640 1 -- 172.21.15.26:0/4160490781 reap_dead start 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.458+0000 7feda2a00640 1 --2- 172.21.15.26:0/4160490781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7fed84004c40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009ac0 tx=0x7fed840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb7a164aeadd3ee server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c02c6b0 con 0x7fed840047c0 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c02c810 con 0x7fed840047c0 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda2000640 1 -- 172.21.15.26:0/4160490781 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c02cba0 con 0x7fed840047c0 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda9000640 1 -- 172.21.15.26:0/4160490781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 msgr2=0x7fed84004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda9000640 1 --2- 172.21.15.26:0/4160490781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7fed84004c40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009ac0 tx=0x7fed840057f0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda9000640 1 -- 172.21.15.26:0/4160490781 shutdown_connections 2024-01-15T18:10:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda9000640 1 --2- 172.21.15.26:0/4160490781 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7fed84004c40 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda9000640 1 -- 172.21.15.26:0/4160490781 >> 172.21.15.26:0/4160490781 conn(0x7feda40b2a30 msgr2=0x7feda40b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda9000640 1 -- 172.21.15.26:0/4160490781 shutdown_connections 2024-01-15T18:10:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.459+0000 7feda9000640 1 -- 172.21.15.26:0/4160490781 wait complete. 2024-01-15T18:10:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.461+0000 7feda9000640 1 Processor -- start 2024-01-15T18:10:04.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.461+0000 7feda9000640 1 -- start start 2024-01-15T18:10:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.462+0000 7feda9000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7feda40add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.462+0000 7feda9000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feda40c6c00 0x7feda4149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.462+0000 7feda9000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feda4149f60 0x7feda414a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.462+0000 7feda9000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda40c6800 con 0x7fed840047c0 2024-01-15T18:10:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.462+0000 7feda9000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda4150860 con 0x7feda40c6c00 2024-01-15T18:10:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.462+0000 7feda9000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda40c6170 con 0x7feda4149f60 2024-01-15T18:10:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7feda40add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7feda40add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40356/0 (socket says 172.21.15.26:40356) 2024-01-15T18:10:04.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3400640 1 -- 172.21.15.26:0/1156552816 learned_addr learned my addr 172.21.15.26:0/1156552816 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda2a00640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feda40c6c00 0x7feda4149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3400640 1 -- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feda4149f60 msgr2=0x7feda414a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3e00640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feda4149f60 0x7feda414a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3400640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feda4149f60 0x7feda414a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3400640 1 -- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feda40c6c00 msgr2=0x7feda4149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3400640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feda40c6c00 0x7feda4149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3400640 1 -- 172.21.15.26:0/1156552816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda40cab40 con 0x7fed840047c0 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda2a00640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feda40c6c00 0x7feda4149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3e00640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feda4149f60 0x7feda414a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.463+0000 7feda3400640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7feda40add10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c390 tx=0x7fed9400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:04.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.464+0000 7fed9b400640 1 -- 172.21.15.26:0/1156552816 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed940052d0 con 0x7fed840047c0 2024-01-15T18:10:04.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.464+0000 7fed9b400640 1 -- 172.21.15.26:0/1156552816 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed94005450 con 0x7fed840047c0 2024-01-15T18:10:04.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.464+0000 7fed9b400640 1 -- 172.21.15.26:0/1156552816 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94007260 con 0x7fed840047c0 2024-01-15T18:10:04.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.464+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda40cae20 con 0x7fed840047c0 2024-01-15T18:10:04.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.464+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda40c7080 con 0x7fed840047c0 2024-01-15T18:10:04.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.465+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda4067b60 con 0x7fed840047c0 2024-01-15T18:10:04.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.466+0000 7fed9b400640 1 -- 172.21.15.26:0/1156552816 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fed940073c0 con 0x7fed840047c0 2024-01-15T18:10:04.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.467+0000 7fed9b400640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed7c06c1f0 0x7fed7c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.467+0000 7fed9b400640 1 -- 172.21.15.26:0/1156552816 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fed94087dd0 con 0x7fed840047c0 2024-01-15T18:10:04.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.468+0000 7feda2a00640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed7c06c1f0 0x7fed7c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.469+0000 7feda2a00640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed7c06c1f0 0x7fed7c06e6b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fed8c036000 tx=0x7fed8c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:04.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.471+0000 7fed9b400640 1 -- 172.21.15.26:0/1156552816 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed940543a0 con 0x7fed840047c0 2024-01-15T18:10:04.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.701+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feda40b6e60 con 0x7fed840047c0 2024-01-15T18:10:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.702+0000 7fed9b400640 1 -- 172.21.15.26:0/1156552816 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3698 (secure 0 0 0) 0x7fed940579c0 con 0x7fed840047c0 2024-01-15T18:10:04.706 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:04.706 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/923394527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":923394527},{"type":"v1","addr":"172.21.15.26:6833","nonce":923394527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/350014829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":350014829},{"type":"v1","addr":"172.21.15.26:6835","nonce":350014829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":36005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/1826406197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1826406197},{"type":"v1","addr":"172.21.15.124:6836","nonce":1826406197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":36191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/926524128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":926524128},{"type":"v1","addr":"172.21.15.124:6837","nonce":926524128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2024-01-15T18:10:04.706 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 56 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed7c06c1f0 msgr2=0x7fed7c06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed7c06c1f0 0x7fed7c06e6b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fed8c036000 tx=0x7fed8c030000 comp rx=0 tx=0).stop 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 msgr2=0x7feda40add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7feda40add10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fed9400c390 tx=0x7fed9400c850 comp rx=0 tx=0).stop 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 shutdown_connections 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feda4149f60 0x7feda414a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fed7c06c1f0 0x7fed7c06e6b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feda40c6c00 0x7feda4149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 --2- 172.21.15.26:0/1156552816 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fed840047c0 0x7feda40add10 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 >> 172.21.15.26:0/1156552816 conn(0x7feda40b2a30 msgr2=0x7feda40d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:04.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 shutdown_connections 2024-01-15T18:10:04.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.705+0000 7feda9000640 1 -- 172.21.15.26:0/1156552816 wait complete. 2024-01-15T18:10:04.731 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:10:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.954+0000 7fc0bee00640 1 Processor -- start 2024-01-15T18:10:04.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.954+0000 7fc0bee00640 1 -- start start 2024-01-15T18:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.956+0000 7fc0bee00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0b80ccb50 0x7fc0b8070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.956+0000 7fc0bee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b815bf10 0x7fc0b8143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.956+0000 7fc0bee00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b815f4e0 con 0x7fc0b80adf20 2024-01-15T18:10:04.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.956+0000 7fc0bee00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b815f970 con 0x7fc0b80ccb50 2024-01-15T18:10:04.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.956+0000 7fc0bee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b815c3c0 con 0x7fc0b815bf10 2024-01-15T18:10:04.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.956+0000 7fc0bd000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0b80ccb50 0x7fc0b8070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.957+0000 7fc0bd000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc0b80ccb50 0x7fc0b8070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:50170/0 (socket says 172.21.15.26:50170) 2024-01-15T18:10:04.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.957+0000 7fc0be400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b815bf10 0x7fc0b8143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.957+0000 7fc0bd000640 1 -- 172.21.15.26:0/3605367107 learned_addr learned my addr 172.21.15.26:0/3605367107 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.957+0000 7fc0bd000640 1 -- 172.21.15.26:0/3605367107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b815bf10 msgr2=0x7fc0b8143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.957+0000 7fc0bd000640 1 --2- 172.21.15.26:0/3605367107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b815bf10 0x7fc0b8143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.957+0000 7fc0bd000640 1 -- 172.21.15.26:0/3605367107 >> v1:172.21.15.124:6789/0 conn(0x7fc0b80adf20 legacy=0x7fc0b80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.957+0000 7fc0bd000640 1 -- 172.21.15.26:0/3605367107 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b8071130 con 0x7fc0b80ccb50 2024-01-15T18:10:04.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.959+0000 7fc0bd000640 1 --2- 172.21.15.26:0/3605367107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0b80ccb50 0x7fc0b8070a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009870 tx=0x7fc0a002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ced851867c56880 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:04.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.959+0000 7fc0b7e00640 1 -- 172.21.15.26:0/3605367107 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0031640 con 0x7fc0b80ccb50 2024-01-15T18:10:04.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.959+0000 7fc0b7e00640 1 -- 172.21.15.26:0/3605367107 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a00317a0 con 0x7fc0b80ccb50 2024-01-15T18:10:04.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.959+0000 7fc0b7e00640 1 -- 172.21.15.26:0/3605367107 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0031b30 con 0x7fc0b80ccb50 2024-01-15T18:10:04.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.960+0000 7fc0bee00640 1 -- 172.21.15.26:0/3605367107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0b80ccb50 msgr2=0x7fc0b8070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.960+0000 7fc0bee00640 1 --2- 172.21.15.26:0/3605367107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0b80ccb50 0x7fc0b8070a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009870 tx=0x7fc0a002b8f0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.960+0000 7fc0bee00640 1 -- 172.21.15.26:0/3605367107 shutdown_connections 2024-01-15T18:10:04.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.960+0000 7fc0bee00640 1 --2- 172.21.15.26:0/3605367107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b815bf10 0x7fc0b8143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.960+0000 7fc0bee00640 1 --2- 172.21.15.26:0/3605367107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0b80ccb50 0x7fc0b8070a30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.960+0000 7fc0bee00640 1 -- 172.21.15.26:0/3605367107 >> 172.21.15.26:0/3605367107 conn(0x7fc0b806a7d0 msgr2=0x7fc0b806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:04.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.960+0000 7fc0bee00640 1 -- 172.21.15.26:0/3605367107 shutdown_connections 2024-01-15T18:10:04.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.960+0000 7fc0bee00640 1 -- 172.21.15.26:0/3605367107 wait complete. 2024-01-15T18:10:04.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.962+0000 7fc0bee00640 1 Processor -- start 2024-01-15T18:10:04.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.962+0000 7fc0bee00640 1 -- start start 2024-01-15T18:10:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.963+0000 7fc0bee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0b80adf20 0x7fc0b813a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.963+0000 7fc0bee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b80ccb50 0x7fc0b8140510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.963+0000 7fc0bee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0b815bf10 0x7fc0b8140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.963+0000 7fc0bee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b813abb0 con 0x7fc0b815bf10 2024-01-15T18:10:04.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.963+0000 7fc0bee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b81e48a0 con 0x7fc0b80adf20 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.963+0000 7fc0bee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b81e4b80 con 0x7fc0b80ccb50 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.963+0000 7fc0be400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0b815bf10 0x7fc0b8140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.963+0000 7fc0be400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0b815bf10 0x7fc0b8140a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40366/0 (socket says 172.21.15.26:40366) 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.963+0000 7fc0be400640 1 -- 172.21.15.26:0/487205170 learned_addr learned my addr 172.21.15.26:0/487205170 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.964+0000 7fc0bda00640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0b80adf20 0x7fc0b813a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.964+0000 7fc0bd000640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b80ccb50 0x7fc0b8140510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.964+0000 7fc0be400640 1 -- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b80ccb50 msgr2=0x7fc0b8140510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.964+0000 7fc0be400640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b80ccb50 0x7fc0b8140510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.964+0000 7fc0be400640 1 -- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0b80adf20 msgr2=0x7fc0b813a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.964+0000 7fc0be400640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0b80adf20 0x7fc0b813a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.964+0000 7fc0be400640 1 -- 172.21.15.26:0/487205170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a00094d0 con 0x7fc0b815bf10 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.964+0000 7fc0be400640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0b815bf10 0x7fc0b8140a50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000e7f0 tx=0x7fc0b000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.965+0000 7fc0b5600640 1 -- 172.21.15.26:0/487205170 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0009510 con 0x7fc0b815bf10 2024-01-15T18:10:04.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.965+0000 7fc0b5600640 1 -- 172.21.15.26:0/487205170 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b000ee40 con 0x7fc0b815bf10 2024-01-15T18:10:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.965+0000 7fc0b5600640 1 -- 172.21.15.26:0/487205170 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000cb80 con 0x7fc0b815bf10 2024-01-15T18:10:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.965+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b81e4e60 con 0x7fc0b815bf10 2024-01-15T18:10:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.965+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b81e60a0 con 0x7fc0b815bf10 2024-01-15T18:10:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.966+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b81e5380 con 0x7fc0b815bf10 2024-01-15T18:10:04.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.967+0000 7fc0b5600640 1 -- 172.21.15.26:0/487205170 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fc0b000cce0 con 0x7fc0b815bf10 2024-01-15T18:10:04.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.969+0000 7fc0b5600640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc09006c1f0 0x7fc09006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:04.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.969+0000 7fc0b5600640 1 -- 172.21.15.26:0/487205170 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fc0b0088bb0 con 0x7fc0b815bf10 2024-01-15T18:10:04.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.969+0000 7fc0bda00640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc09006c1f0 0x7fc09006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:04.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.970+0000 7fc0bda00640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc09006c1f0 0x7fc09006e6b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80012e0 tx=0x7fc0a8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:04.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:04.974+0000 7fc0b5600640 1 -- 172.21.15.26:0/487205170 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0b81e5380 con 0x7fc0b815bf10 2024-01-15T18:10:05.170 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.166+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc0b81e5380 con 0x7fc0b815bf10 2024-01-15T18:10:05.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.171+0000 7fc0b5600640 1 -- 172.21.15.26:0/487205170 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc0b81e5380 con 0x7fc0b815bf10 2024-01-15T18:10:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc09006c1f0 msgr2=0x7fc09006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc09006c1f0 0x7fc09006e6b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80012e0 tx=0x7fc0a8009040 comp rx=0 tx=0).stop 2024-01-15T18:10:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0b815bf10 msgr2=0x7fc0b8140a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0b815bf10 0x7fc0b8140a50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000e7f0 tx=0x7fc0b000ebf0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 shutdown_connections 2024-01-15T18:10:05.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc0b80ccb50 0x7fc0b8140510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc09006c1f0 0x7fc09006e6b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc0b815bf10 0x7fc0b8140a50 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 --2- 172.21.15.26:0/487205170 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc0b80adf20 0x7fc0b813a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.174+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 >> 172.21.15.26:0/487205170 conn(0x7fc0b806a7d0 msgr2=0x7fc0b806d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.175+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 shutdown_connections 2024-01-15T18:10:05.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.175+0000 7fc0bee00640 1 -- 172.21.15.26:0/487205170 wait complete. 2024-01-15T18:10:05.196 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:10:05.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.401+0000 7f19b2800640 1 Processor -- start 2024-01-15T18:10:05.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.401+0000 7f19b2800640 1 -- start start 2024-01-15T18:10:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.403+0000 7f19b2800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19ac0ae5a0 0x7f19ac15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.403+0000 7f19b2800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ac070c40 con 0x7f19ac0b3ae0 2024-01-15T18:10:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.403+0000 7f19b2800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ac06c470 con 0x7f19ac0cf290 2024-01-15T18:10:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.403+0000 7f19b2800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ac071450 con 0x7f19ac0ae5a0 2024-01-15T18:10:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.403+0000 7f19ab400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f19ac0ae5a0 0x7f19ac15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.403+0000 7f19a3e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f19ac0b3ae0 0x7f19ac06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59512/0 (socket says 172.21.15.26:59512) 2024-01-15T18:10:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.403+0000 7f19a3e00640 1 -- 172.21.15.26:0/2792355464 learned_addr learned my addr 172.21.15.26:0/2792355464 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:05.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.404+0000 7f19ab400640 1 -- 172.21.15.26:0/2792355464 >> v1:172.21.15.124:6789/0 conn(0x7f19ac0cf290 legacy=0x7f19ac0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.404+0000 7f19ab400640 1 -- 172.21.15.26:0/2792355464 >> v1:172.21.15.26:6789/0 conn(0x7f19ac0b3ae0 legacy=0x7f19ac06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.404+0000 7f19ab400640 1 -- 172.21.15.26:0/2792355464 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac06c750 con 0x7f19ac0ae5a0 2024-01-15T18:10:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.405+0000 7f19ab400640 1 --2- 172.21.15.26:0/2792355464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0ae5a0 0x7f19ac15e430 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f199c0097a0 tx=0x7f199c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d57cf3e4a6ff7da server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:05.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.405+0000 7f19aaa00640 1 -- 172.21.15.26:0/2792355464 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c030710 con 0x7f19ac0ae5a0 2024-01-15T18:10:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.405+0000 7f19aaa00640 1 -- 172.21.15.26:0/2792355464 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c02e050 con 0x7f19ac0ae5a0 2024-01-15T18:10:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.405+0000 7f19aaa00640 1 -- 172.21.15.26:0/2792355464 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c030a90 con 0x7f19ac0ae5a0 2024-01-15T18:10:05.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.406+0000 7f19b2800640 1 -- 172.21.15.26:0/2792355464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0ae5a0 msgr2=0x7f19ac15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.406+0000 7f19b2800640 1 --2- 172.21.15.26:0/2792355464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0ae5a0 0x7f19ac15e430 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f199c0097a0 tx=0x7f199c02b790 comp rx=0 tx=0).stop 2024-01-15T18:10:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.406+0000 7f19b2800640 1 -- 172.21.15.26:0/2792355464 shutdown_connections 2024-01-15T18:10:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.406+0000 7f19b2800640 1 --2- 172.21.15.26:0/2792355464 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0ae5a0 0x7f19ac15e430 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.406+0000 7f19b2800640 1 -- 172.21.15.26:0/2792355464 >> 172.21.15.26:0/2792355464 conn(0x7f19ac06a800 msgr2=0x7f19ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.406+0000 7f19b2800640 1 -- 172.21.15.26:0/2792355464 shutdown_connections 2024-01-15T18:10:05.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.406+0000 7f19b2800640 1 -- 172.21.15.26:0/2792355464 wait complete. 2024-01-15T18:10:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.408+0000 7f19b2800640 1 Processor -- start 2024-01-15T18:10:05.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.409+0000 7f19b2800640 1 -- start start 2024-01-15T18:10:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.410+0000 7f19b2800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19ac0ae5a0 0x7f19ac1f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.410+0000 7f19b2800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19ac0b3ae0 0x7f19ac0cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.410+0000 7f19b2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0cf290 0x7f19ac1f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.410+0000 7f19b2800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac157d00 con 0x7f19ac0cf290 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.410+0000 7f19b2800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac1f3f30 con 0x7f19ac0ae5a0 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.410+0000 7f19b2800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0cbde0 con 0x7f19ac0b3ae0 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.410+0000 7f19abe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0cf290 0x7f19ac1f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.410+0000 7f19abe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0cf290 0x7f19ac1f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40386/0 (socket says 172.21.15.26:40386) 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.410+0000 7f19abe00640 1 -- 172.21.15.26:0/2925992579 learned_addr learned my addr 172.21.15.26:0/2925992579 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.411+0000 7f19ab400640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19ac0ae5a0 0x7f19ac1f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.411+0000 7f19a3e00640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19ac0b3ae0 0x7f19ac0cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.411+0000 7f19abe00640 1 -- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19ac0b3ae0 msgr2=0x7f19ac0cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.411+0000 7f19abe00640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19ac0b3ae0 0x7f19ac0cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.411+0000 7f19abe00640 1 -- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19ac0ae5a0 msgr2=0x7f19ac1f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.411+0000 7f19abe00640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19ac0ae5a0 0x7f19ac1f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.411+0000 7f19abe00640 1 -- 172.21.15.26:0/2925992579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c009400 con 0x7f19ac0cf290 2024-01-15T18:10:05.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.411+0000 7f19abe00640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0cf290 0x7f19ac1f02f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b970 tx=0x7f19a400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:05.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.411+0000 7f19a3400640 1 -- 172.21.15.26:0/2925992579 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a400cb40 con 0x7f19ac0cf290 2024-01-15T18:10:05.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.412+0000 7f19a3400640 1 -- 172.21.15.26:0/2925992579 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a400cca0 con 0x7f19ac0cf290 2024-01-15T18:10:05.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.412+0000 7f19a3400640 1 -- 172.21.15.26:0/2925992579 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4014680 con 0x7f19ac0cf290 2024-01-15T18:10:05.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.412+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac1f69f0 con 0x7f19ac0cf290 2024-01-15T18:10:05.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.412+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac1f6fb0 con 0x7f19ac0cf290 2024-01-15T18:10:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.413+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac0675d0 con 0x7f19ac0cf290 2024-01-15T18:10:05.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.414+0000 7f19a3400640 1 -- 172.21.15.26:0/2925992579 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f19a401d3b0 con 0x7f19ac0cf290 2024-01-15T18:10:05.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.415+0000 7f19a3400640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f198406c1f0 0x7f198406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.415+0000 7f19a3400640 1 -- 172.21.15.26:0/2925992579 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f19a4088610 con 0x7f19ac0cf290 2024-01-15T18:10:05.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.416+0000 7f19ab400640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f198406c1f0 0x7f198406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.417+0000 7f19ab400640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f198406c1f0 0x7f198406e6b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f199c034000 tx=0x7f199c02f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:05.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.421+0000 7f19a3400640 1 -- 172.21.15.26:0/2925992579 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19a4054dd0 con 0x7f19ac0cf290 2024-01-15T18:10:05.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.608+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f19ac1591a0 con 0x7f19ac0cf290 2024-01-15T18:10:05.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.613+0000 7f19a3400640 1 -- 172.21.15.26:0/2925992579 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7f19a40583f0 con 0x7f19ac0cf290 2024-01-15T18:10:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f198406c1f0 msgr2=0x7f198406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f198406c1f0 0x7f198406e6b0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f199c034000 tx=0x7f199c02f000 comp rx=0 tx=0).stop 2024-01-15T18:10:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0cf290 msgr2=0x7f19ac1f02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0cf290 0x7f19ac1f02f0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f19a400b970 tx=0x7f19a400be30 comp rx=0 tx=0).stop 2024-01-15T18:10:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 shutdown_connections 2024-01-15T18:10:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19ac0b3ae0 0x7f19ac0cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f198406c1f0 0x7f198406e6b0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f19ac0cf290 0x7f19ac1f02f0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 --2- 172.21.15.26:0/2925992579 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f19ac0ae5a0 0x7f19ac1f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.616+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 >> 172.21.15.26:0/2925992579 conn(0x7f19ac06a800 msgr2=0x7f19ac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.617+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 shutdown_connections 2024-01-15T18:10:05.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.617+0000 7f19b2800640 1 -- 172.21.15.26:0/2925992579 wait complete. 2024-01-15T18:10:05.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed' 2024-01-15T18:10:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.848+0000 7fc937200640 1 Processor -- start 2024-01-15T18:10:05.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.848+0000 7fc937200640 1 -- start start 2024-01-15T18:10:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.849+0000 7fc937200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc930143c40 0x7fc930071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc937200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc9300717d0 0x7fc9300cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc937200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc930070e50 con 0x7fc9300cca40 2024-01-15T18:10:05.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc937200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9300723c0 con 0x7fc930143c40 2024-01-15T18:10:05.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc937200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9300cce40 con 0x7fc9300717d0 2024-01-15T18:10:05.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc935e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc930143c40 0x7fc930071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc935e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc930143c40 0x7fc930071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40400/0 (socket says 172.21.15.26:40400) 2024-01-15T18:10:05.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc935e00640 1 -- 172.21.15.26:0/3542689217 learned_addr learned my addr 172.21.15.26:0/3542689217 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:05.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc935e00640 1 -- 172.21.15.26:0/3542689217 >> v2:172.21.15.124:3300/0 conn(0x7fc9300717d0 msgr2=0x7fc9300cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc935e00640 1 --2- 172.21.15.26:0/3542689217 >> v2:172.21.15.124:3300/0 conn(0x7fc9300717d0 0x7fc9300cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc935e00640 1 -- 172.21.15.26:0/3542689217 >> v1:172.21.15.124:6789/0 conn(0x7fc9300cca40 legacy=0x7fc93015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.850+0000 7fc935e00640 1 -- 172.21.15.26:0/3542689217 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93006e170 con 0x7fc930143c40 2024-01-15T18:10:05.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.851+0000 7fc935e00640 1 --2- 172.21.15.26:0/3542689217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 0x7fc930071290 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fc9300b44a0 tx=0x7fc92002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c125f8de4359715 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.852+0000 7fc934a00640 1 -- 172.21.15.26:0/3542689217 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92002c9f0 con 0x7fc930143c40 2024-01-15T18:10:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.852+0000 7fc934a00640 1 -- 172.21.15.26:0/3542689217 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92002cb50 con 0x7fc930143c40 2024-01-15T18:10:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.852+0000 7fc934a00640 1 -- 172.21.15.26:0/3542689217 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9200343f0 con 0x7fc930143c40 2024-01-15T18:10:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.852+0000 7fc937200640 1 -- 172.21.15.26:0/3542689217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 msgr2=0x7fc930071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.852+0000 7fc937200640 1 --2- 172.21.15.26:0/3542689217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 0x7fc930071290 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fc9300b44a0 tx=0x7fc92002be50 comp rx=0 tx=0).stop 2024-01-15T18:10:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.853+0000 7fc937200640 1 -- 172.21.15.26:0/3542689217 shutdown_connections 2024-01-15T18:10:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.853+0000 7fc937200640 1 --2- 172.21.15.26:0/3542689217 >> v2:172.21.15.124:3300/0 conn(0x7fc9300717d0 0x7fc9300cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.853+0000 7fc937200640 1 --2- 172.21.15.26:0/3542689217 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 0x7fc930071290 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.853+0000 7fc937200640 1 -- 172.21.15.26:0/3542689217 >> 172.21.15.26:0/3542689217 conn(0x7fc93006a800 msgr2=0x7fc93006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:05.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.853+0000 7fc937200640 1 -- 172.21.15.26:0/3542689217 shutdown_connections 2024-01-15T18:10:05.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.853+0000 7fc937200640 1 -- 172.21.15.26:0/3542689217 wait complete. 2024-01-15T18:10:05.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.854+0000 7fc937200640 1 Processor -- start 2024-01-15T18:10:05.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.855+0000 7fc937200640 1 -- start start 2024-01-15T18:10:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.855+0000 7fc937200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9300717d0 0x7fc9301e0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.855+0000 7fc937200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9300cca40 0x7fc9301e0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc937200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 0x7fc9301e0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc937200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9301f6380 con 0x7fc930143c40 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc937200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93013f1f0 con 0x7fc9300717d0 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc937200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc930140b30 con 0x7fc9300cca40 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc936800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 0x7fc9301e0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc936800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 0x7fc9301e0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40404/0 (socket says 172.21.15.26:40404) 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc936800640 1 -- 172.21.15.26:0/4027163447 learned_addr learned my addr 172.21.15.26:0/4027163447 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc935e00640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9300717d0 0x7fc9301e0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc935400640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9300cca40 0x7fc9301e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc936800640 1 -- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9300cca40 msgr2=0x7fc9301e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc936800640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9300cca40 0x7fc9301e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc936800640 1 -- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9300717d0 msgr2=0x7fc9301e0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:05.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc936800640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9300717d0 0x7fc9301e0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.856+0000 7fc936800640 1 -- 172.21.15.26:0/4027163447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc920009b30 con 0x7fc930143c40 2024-01-15T18:10:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.857+0000 7fc936800640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 0x7fc9301e0ac0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c3c0 tx=0x7fc92800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.857+0000 7fc92e000640 1 -- 172.21.15.26:0/4027163447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280035f0 con 0x7fc930143c40 2024-01-15T18:10:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.857+0000 7fc92e000640 1 -- 172.21.15.26:0/4027163447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc928003750 con 0x7fc930143c40 2024-01-15T18:10:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.857+0000 7fc92e000640 1 -- 172.21.15.26:0/4027163447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928003b00 con 0x7fc930143c40 2024-01-15T18:10:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.857+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc930140e10 con 0x7fc930143c40 2024-01-15T18:10:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.857+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9301413d0 con 0x7fc930143c40 2024-01-15T18:10:05.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.858+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9300706c0 con 0x7fc930143c40 2024-01-15T18:10:05.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.860+0000 7fc92e000640 1 -- 172.21.15.26:0/4027163447 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fc928003c60 con 0x7fc930143c40 2024-01-15T18:10:05.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.861+0000 7fc92e000640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc90806c1f0 0x7fc90806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:05.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.861+0000 7fc92e000640 1 -- 172.21.15.26:0/4027163447 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fc928087bc0 con 0x7fc930143c40 2024-01-15T18:10:05.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.862+0000 7fc935e00640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc90806c1f0 0x7fc90806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:05.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.863+0000 7fc935e00640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc90806c1f0 0x7fc90806e6b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc920035000 tx=0x7fc920030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:05.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:05.865+0000 7fc92e000640 1 -- 172.21.15.26:0/4027163447 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc928054190 con 0x7fc930143c40 2024-01-15T18:10:06.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.121+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed"]} v 0) v1 -- 0x7fc930067450 con 0x7fc930143c40 2024-01-15T18:10:06.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.485+0000 7fc92e000640 1 -- 172.21.15.26:0/4027163447 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_stats_are_printed"]}]=0 v677) v1 ==== 145+0+0 (secure 0 0 0) 0x7fc9280170d0 con 0x7fc930143c40 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc90806c1f0 msgr2=0x7fc90806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc90806c1f0 0x7fc90806e6b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc920035000 tx=0x7fc920030000 comp rx=0 tx=0).stop 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 msgr2=0x7fc9301e0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 0x7fc9301e0ac0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c3c0 tx=0x7fc92800c880 comp rx=0 tx=0).stop 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 shutdown_connections 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc9300cca40 0x7fc9301e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc90806c1f0 0x7fc90806e6b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc930143c40 0x7fc9301e0ac0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 --2- 172.21.15.26:0/4027163447 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc9300717d0 0x7fc9301e0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 >> 172.21.15.26:0/4027163447 conn(0x7fc93006a800 msgr2=0x7fc9300af660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:06.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.488+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 shutdown_connections 2024-01-15T18:10:06.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.489+0000 7fc937200640 1 -- 172.21.15.26:0/4027163447 wait complete. 2024-01-15T18:10:06.515 INFO:tasks.cephfs_test_runner:Test that the command "ceph fs clone status" prints progress stats ... ok 2024-01-15T18:10:06.515 INFO:tasks.cephfs_test_runner:Starting test: test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:10:06.515 INFO:tasks.cephfs_test_runner:test_when_1_clone_is_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:10:06.515 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:10:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.727+0000 7f1722000640 1 Processor -- start 2024-01-15T18:10:06.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.727+0000 7f1722000640 1 -- start start 2024-01-15T18:10:06.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.729+0000 7f1722000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f171c13df80 0x7f171c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:06.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.729+0000 7f1722000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c0cf9a0 0x7f171c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:06.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.729+0000 7f1722000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c06c470 con 0x7f171c13e360 2024-01-15T18:10:06.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.729+0000 7f1722000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c06c750 con 0x7f171c13df80 2024-01-15T18:10:06.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.729+0000 7f1722000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c06ca30 con 0x7f171c0cf9a0 2024-01-15T18:10:06.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.729+0000 7f1713e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f171c13e360 0x7f171c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59514/0 (socket says 172.21.15.26:59514) 2024-01-15T18:10:06.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.729+0000 7f1713e00640 1 -- 172.21.15.26:0/2097728804 learned_addr learned my addr 172.21.15.26:0/2097728804 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.729+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483490605 0 0) 0x7f171c06c470 con 0x7f171c13e360 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.729+0000 7f171be00640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c0cf9a0 0x7f171c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171b400640 1 --2- 172.21.15.26:0/2097728804 >> v2:172.21.15.124:3300/0 conn(0x7f171c13df80 0x7f171c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16fc003520 con 0x7f171c13e360 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 234712724 0 0) 0x7f16fc003520 con 0x7f171c13e360 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c0cf9a0 msgr2=0x7f171c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c0cf9a0 0x7f171c0cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 >> v2:172.21.15.124:3300/0 conn(0x7f171c13df80 msgr2=0x7f171c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 --2- 172.21.15.26:0/2097728804 >> v2:172.21.15.124:3300/0 conn(0x7f171c13df80 0x7f171c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c06cd10 con 0x7f171c13e360 2024-01-15T18:10:06.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f1704001670 con 0x7f171c13e360 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 >> v1:172.21.15.26:6789/0 conn(0x7f171c13e360 legacy=0x7f171c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f16fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c0cf9a0 0x7f16fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16fc004d00 0x7f16fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c06c750 con 0x7f16fc003970 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c06ca30 con 0x7f16fc004d00 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.730+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc00b980 con 0x7f171c0cf9a0 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f171b400640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f16fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f1713e00640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c0cf9a0 0x7f16fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f171be00640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16fc004d00 0x7f16fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f171b400640 1 -- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c0cf9a0 msgr2=0x7f16fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f171b400640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c0cf9a0 0x7f16fc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f171b400640 1 -- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16fc004d00 msgr2=0x7f16fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f171b400640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16fc004d00 0x7f16fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f171b400640 1 -- 172.21.15.26:0/2097728804 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc003520 con 0x7f16fc003970 2024-01-15T18:10:06.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f1713e00640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c0cf9a0 0x7f16fc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:06.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.731+0000 7f171be00640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f16fc004d00 0x7f16fc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.732+0000 7f171b400640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f16fc003d90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f170c009af0 tx=0x7f16fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de52d416a67f9c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.732+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c02c710 con 0x7f16fc003970 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.732+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c02c870 con 0x7f16fc003970 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.732+0000 7f171aa00640 1 -- 172.21.15.26:0/2097728804 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c02cc00 con 0x7f16fc003970 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.733+0000 7f1722000640 1 -- 172.21.15.26:0/2097728804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 msgr2=0x7f16fc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.733+0000 7f1722000640 1 --2- 172.21.15.26:0/2097728804 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f16fc003d90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f170c009af0 tx=0x7f16fc0057f0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.733+0000 7f171be00640 1 -- 172.21.15.26:0/2097728804 reap_dead start 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.733+0000 7f1722000640 1 -- 172.21.15.26:0/2097728804 shutdown_connections 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.733+0000 7f1722000640 1 -- 172.21.15.26:0/2097728804 >> 172.21.15.26:0/2097728804 conn(0x7f171c06a800 msgr2=0x7f171c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.733+0000 7f1722000640 1 -- 172.21.15.26:0/2097728804 shutdown_connections 2024-01-15T18:10:06.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.733+0000 7f1722000640 1 -- 172.21.15.26:0/2097728804 wait complete. 2024-01-15T18:10:06.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.735+0000 7f1722000640 1 Processor -- start 2024-01-15T18:10:06.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.735+0000 7f1722000640 1 -- start start 2024-01-15T18:10:06.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.736+0000 7f1722000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f171c13bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.736+0000 7f1722000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c13c320 0x7f171c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f1722000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c0a84c0 0x7f171c0b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f1722000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c13d4a0 con 0x7f16fc003970 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f1722000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c13c890 con 0x7f171c0a84c0 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f1722000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c13cb70 con 0x7f171c13c320 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f171b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f171c13bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f171b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f171c13bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40422/0 (socket says 172.21.15.26:40422) 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f171b400640 1 -- 172.21.15.26:0/527738091 learned_addr learned my addr 172.21.15.26:0/527738091 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f1713e00640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c13c320 0x7f171c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f171b400640 1 -- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c13c320 msgr2=0x7f171c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f171b400640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c13c320 0x7f171c0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f171be00640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c0a84c0 0x7f171c0b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f171b400640 1 -- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c0a84c0 msgr2=0x7f171c0b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f171b400640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c0a84c0 0x7f171c0b00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.737+0000 7f171b400640 1 -- 172.21.15.26:0/527738091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c13ce50 con 0x7f16fc003970 2024-01-15T18:10:06.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.738+0000 7f171b400640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f171c13bd20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f170c037040 tx=0x7f170c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:06.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.738+0000 7f1713400640 1 -- 172.21.15.26:0/527738091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c003810 con 0x7f16fc003970 2024-01-15T18:10:06.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.738+0000 7f1713400640 1 -- 172.21.15.26:0/527738091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c02de90 con 0x7f16fc003970 2024-01-15T18:10:06.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.738+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c13d130 con 0x7f16fc003970 2024-01-15T18:10:06.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.738+0000 7f1713400640 1 -- 172.21.15.26:0/527738091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c003b70 con 0x7f16fc003970 2024-01-15T18:10:06.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.738+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c15a180 con 0x7f16fc003970 2024-01-15T18:10:06.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.739+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c06af90 con 0x7f16fc003970 2024-01-15T18:10:06.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.740+0000 7f1713400640 1 -- 172.21.15.26:0/527738091 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f170c003cd0 con 0x7f16fc003970 2024-01-15T18:10:06.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.741+0000 7f1713400640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406c1f0 0x7f16f406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:06.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.741+0000 7f1713400640 1 -- 172.21.15.26:0/527738091 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f170c0aa6b0 con 0x7f16fc003970 2024-01-15T18:10:06.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.742+0000 7f1713e00640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406c1f0 0x7f16f406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:06.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.743+0000 7f1713e00640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406c1f0 0x7f16f406e6b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f17040018e0 tx=0x7f1704009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:06.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.746+0000 7f1713400640 1 -- 172.21.15.26:0/527738091 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f170c076e70 con 0x7f16fc003970 2024-01-15T18:10:06.947 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.943+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f171c0675d0 con 0x7f16fc003970 2024-01-15T18:10:06.950 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.945+0000 7f1713400640 1 -- 172.21.15.26:0/527738091 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+13096 (secure 0 0 0) 0x7f170c0aae60 con 0x7f16fc003970 2024-01-15T18:10:06.950 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:06.950 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":93,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:10:00.190534+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/2740807412":"2024-01-16T18:09:57.843253+0000","172.21.15.124:6835/2740807412":"2024-01-16T18:09:57.843253+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":13,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:10:06.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406c1f0 msgr2=0x7f16f406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:06.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406c1f0 0x7f16f406e6b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f17040018e0 tx=0x7f1704009040 comp rx=0 tx=0).stop 2024-01-15T18:10:06.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 msgr2=0x7f171c13bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:06.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f171c13bd20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f170c037040 tx=0x7f170c004730 comp rx=0 tx=0).stop 2024-01-15T18:10:06.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 shutdown_connections 2024-01-15T18:10:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f171c13c320 0x7f171c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f16f406c1f0 0x7f16f406e6b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f171c0a84c0 0x7f171c0b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 --2- 172.21.15.26:0/527738091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f16fc003970 0x7f171c13bd20 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 >> 172.21.15.26:0/527738091 conn(0x7f171c06a800 msgr2=0x7f171c15d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 shutdown_connections 2024-01-15T18:10:06.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:06.948+0000 7f1722000640 1 -- 172.21.15.26:0/527738091 wait complete. 2024-01-15T18:10:06.971 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:10:06.971 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:10:07.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.178+0000 7fb85ba00640 1 Processor -- start 2024-01-15T18:10:07.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.179+0000 7fb85ba00640 1 -- start start 2024-01-15T18:10:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.180+0000 7fb85ba00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb85413df80 0x7fb85415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.180+0000 7fb85ba00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb85413e360 0x7fb8540cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.180+0000 7fb85ba00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb85406c470 con 0x7fb85415f9c0 2024-01-15T18:10:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.180+0000 7fb85ba00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85406c750 con 0x7fb85413df80 2024-01-15T18:10:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.180+0000 7fb85ba00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85406ca30 con 0x7fb85413e360 2024-01-15T18:10:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.181+0000 7fb859000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb85413df80 0x7fb85415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.181+0000 7fb859000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb85413df80 0x7fb85415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40424/0 (socket says 172.21.15.26:40424) 2024-01-15T18:10:07.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.181+0000 7fb859000640 1 -- 172.21.15.26:0/3640584507 learned_addr learned my addr 172.21.15.26:0/3640584507 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.181+0000 7fb859000640 1 -- 172.21.15.26:0/3640584507 >> v2:172.21.15.124:3300/0 conn(0x7fb85413e360 msgr2=0x7fb8540cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.181+0000 7fb859000640 1 --2- 172.21.15.26:0/3640584507 >> v2:172.21.15.124:3300/0 conn(0x7fb85413e360 0x7fb8540cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.181+0000 7fb859000640 1 -- 172.21.15.26:0/3640584507 >> v1:172.21.15.124:6789/0 conn(0x7fb85415f9c0 legacy=0x7fb8541408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:07.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.181+0000 7fb859000640 1 -- 172.21.15.26:0/3640584507 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85406cd10 con 0x7fb85413df80 2024-01-15T18:10:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.183+0000 7fb859000640 1 --2- 172.21.15.26:0/3640584507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85413df80 0x7fb85415bee0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb85406ded0 tx=0x7fb84402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca620aeaabf0c91 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.183+0000 7fb853400640 1 -- 172.21.15.26:0/3640584507 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844031870 con 0x7fb85413df80 2024-01-15T18:10:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.183+0000 7fb853400640 1 -- 172.21.15.26:0/3640584507 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8440319d0 con 0x7fb85413df80 2024-01-15T18:10:07.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.183+0000 7fb853400640 1 -- 172.21.15.26:0/3640584507 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844031d60 con 0x7fb85413df80 2024-01-15T18:10:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.184+0000 7fb85ba00640 1 -- 172.21.15.26:0/3640584507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85413df80 msgr2=0x7fb85415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.188 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.184+0000 7fb85ba00640 1 --2- 172.21.15.26:0/3640584507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85413df80 0x7fb85415bee0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fb85406ded0 tx=0x7fb84402b910 comp rx=0 tx=0).stop 2024-01-15T18:10:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.184+0000 7fb85ba00640 1 -- 172.21.15.26:0/3640584507 shutdown_connections 2024-01-15T18:10:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.184+0000 7fb85ba00640 1 --2- 172.21.15.26:0/3640584507 >> v2:172.21.15.124:3300/0 conn(0x7fb85413e360 0x7fb8540cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.184+0000 7fb85ba00640 1 --2- 172.21.15.26:0/3640584507 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85413df80 0x7fb85415bee0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.184+0000 7fb85ba00640 1 -- 172.21.15.26:0/3640584507 >> 172.21.15.26:0/3640584507 conn(0x7fb85406a800 msgr2=0x7fb85406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.184+0000 7fb85ba00640 1 -- 172.21.15.26:0/3640584507 shutdown_connections 2024-01-15T18:10:07.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.184+0000 7fb85ba00640 1 -- 172.21.15.26:0/3640584507 wait complete. 2024-01-15T18:10:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.186+0000 7fb85ba00640 1 Processor -- start 2024-01-15T18:10:07.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.186+0000 7fb85ba00640 1 -- start start 2024-01-15T18:10:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.187+0000 7fb85ba00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb85413df80 0x7fb854150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.192 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.187+0000 7fb85ba00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb85413e360 0x7fb85413ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.187+0000 7fb85ba00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85415f9c0 0x7fb854151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.187+0000 7fb85ba00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8541533e0 con 0x7fb85415f9c0 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.187+0000 7fb85ba00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb854150950 con 0x7fb85413df80 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.187+0000 7fb85ba00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8541501d0 con 0x7fb85413e360 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.187+0000 7fb859a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85415f9c0 0x7fb854151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.187+0000 7fb859a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85415f9c0 0x7fb854151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40436/0 (socket says 172.21.15.26:40436) 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.187+0000 7fb859a00640 1 -- 172.21.15.26:0/789328899 learned_addr learned my addr 172.21.15.26:0/789328899 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.188+0000 7fb859000640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb85413df80 0x7fb854150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.188+0000 7fb853e00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb85413e360 0x7fb85413ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.188+0000 7fb859a00640 1 -- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb85413e360 msgr2=0x7fb85413ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.188+0000 7fb859a00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb85413e360 0x7fb85413ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.188+0000 7fb859a00640 1 -- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb85413df80 msgr2=0x7fb854150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.188+0000 7fb859a00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb85413df80 0x7fb854150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.188+0000 7fb859a00640 1 -- 172.21.15.26:0/789328899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8440094d0 con 0x7fb85415f9c0 2024-01-15T18:10:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.188+0000 7fb859a00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85415f9c0 0x7fb854151de0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00c780 tx=0x7fb84c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.189+0000 7fb850c00640 1 -- 172.21.15.26:0/789328899 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c003d80 con 0x7fb85415f9c0 2024-01-15T18:10:07.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.189+0000 7fb850c00640 1 -- 172.21.15.26:0/789328899 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c00ce30 con 0x7fb85415f9c0 2024-01-15T18:10:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.189+0000 7fb850c00640 1 -- 172.21.15.26:0/789328899 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c005290 con 0x7fb85415f9c0 2024-01-15T18:10:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.189+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85413d8f0 con 0x7fb85415f9c0 2024-01-15T18:10:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.189+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb854152d90 con 0x7fb85415f9c0 2024-01-15T18:10:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.190+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb854153550 con 0x7fb85415f9c0 2024-01-15T18:10:07.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.191+0000 7fb850c00640 1 -- 172.21.15.26:0/789328899 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fb84c0053f0 con 0x7fb85415f9c0 2024-01-15T18:10:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.192+0000 7fb850c00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb82c06c220 0x7fb82c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.192+0000 7fb850c00640 1 -- 172.21.15.26:0/789328899 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fb84c087e50 con 0x7fb85415f9c0 2024-01-15T18:10:07.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.193+0000 7fb859000640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb82c06c220 0x7fb82c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.194+0000 7fb859000640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb82c06c220 0x7fb82c06e6e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb844036000 tx=0x7fb84403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:07.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.197+0000 7fb850c00640 1 -- 172.21.15.26:0/789328899 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb84c054420 con 0x7fb85415f9c0 2024-01-15T18:10:07.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.394+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8540675d0 con 0x7fb85415f9c0 2024-01-15T18:10:07.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.395+0000 7fb850c00640 1 -- 172.21.15.26:0/789328899 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb84c0170a0 con 0x7fb85415f9c0 2024-01-15T18:10:07.400 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:10:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb82c06c220 msgr2=0x7fb82c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb82c06c220 0x7fb82c06e6e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fb844036000 tx=0x7fb84403c040 comp rx=0 tx=0).stop 2024-01-15T18:10:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85415f9c0 msgr2=0x7fb854151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85415f9c0 0x7fb854151de0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00c780 tx=0x7fb84c00cc40 comp rx=0 tx=0).stop 2024-01-15T18:10:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 shutdown_connections 2024-01-15T18:10:07.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb85413e360 0x7fb85413ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb82c06c220 0x7fb82c06e6e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb85415f9c0 0x7fb854151de0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 --2- 172.21.15.26:0/789328899 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb85413df80 0x7fb854150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.398+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 >> 172.21.15.26:0/789328899 conn(0x7fb85406a800 msgr2=0x7fb8540b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:07.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.399+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 shutdown_connections 2024-01-15T18:10:07.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.399+0000 7fb85ba00640 1 -- 172.21.15.26:0/789328899 wait complete. 2024-01-15T18:10:07.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled' 2024-01-15T18:10:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.631+0000 7fc1ea400640 1 Processor -- start 2024-01-15T18:10:07.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.631+0000 7fc1ea400640 1 -- start start 2024-01-15T18:10:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.633+0000 7fc1ea400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc1e415e3b0 0x7fc1e4164de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.633+0000 7fc1ea400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e40b1ee0 con 0x7fc1e4143ae0 2024-01-15T18:10:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.633+0000 7fc1ea400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e40b2780 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.633+0000 7fc1ea400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e415e990 con 0x7fc1e415e3b0 2024-01-15T18:10:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.633+0000 7fc1e3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc1e415e3b0 0x7fc1e4164de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.633+0000 7fc1e3400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc1e4143ae0 0x7fc1e40cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59524/0 (socket says 172.21.15.26:59524) 2024-01-15T18:10:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.633+0000 7fc1e3400640 1 -- 172.21.15.26:0/2902227878 learned_addr learned my addr 172.21.15.26:0/2902227878 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.634+0000 7fc1e2000640 1 -- 172.21.15.26:0/2902227878 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771276759 0 0) 0x7fc1e40b1ee0 con 0x7fc1e4143ae0 2024-01-15T18:10:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.634+0000 7fc1e3e00640 1 -- 172.21.15.26:0/2902227878 >> v1:172.21.15.124:6789/0 conn(0x7fc1e40cf9a0 legacy=0x7fc1e415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.634+0000 7fc1e3e00640 1 -- 172.21.15.26:0/2902227878 >> v1:172.21.15.26:6789/0 conn(0x7fc1e4143ae0 legacy=0x7fc1e40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.634+0000 7fc1e3e00640 1 -- 172.21.15.26:0/2902227878 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e40b2130 con 0x7fc1e415e3b0 2024-01-15T18:10:07.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.635+0000 7fc1e3e00640 1 --2- 172.21.15.26:0/2902227878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e415e3b0 0x7fc1e4164de0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009d90 tx=0x7fc1dc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874afc7dfae3ccbe server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.635+0000 7fc1e2000640 1 -- 172.21.15.26:0/2902227878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc02cd00 con 0x7fc1e415e3b0 2024-01-15T18:10:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.635+0000 7fc1e2000640 1 -- 172.21.15.26:0/2902227878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1dc02ce60 con 0x7fc1e415e3b0 2024-01-15T18:10:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.635+0000 7fc1e2000640 1 -- 172.21.15.26:0/2902227878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1dc034660 con 0x7fc1e415e3b0 2024-01-15T18:10:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.636+0000 7fc1ea400640 1 -- 172.21.15.26:0/2902227878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e415e3b0 msgr2=0x7fc1e4164de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.636+0000 7fc1ea400640 1 --2- 172.21.15.26:0/2902227878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e415e3b0 0x7fc1e4164de0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc009d90 tx=0x7fc1dc02bf50 comp rx=0 tx=0).stop 2024-01-15T18:10:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.636+0000 7fc1ea400640 1 -- 172.21.15.26:0/2902227878 shutdown_connections 2024-01-15T18:10:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.636+0000 7fc1ea400640 1 --2- 172.21.15.26:0/2902227878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e415e3b0 0x7fc1e4164de0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.636+0000 7fc1ea400640 1 -- 172.21.15.26:0/2902227878 >> 172.21.15.26:0/2902227878 conn(0x7fc1e406a800 msgr2=0x7fc1e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.636+0000 7fc1ea400640 1 -- 172.21.15.26:0/2902227878 shutdown_connections 2024-01-15T18:10:07.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.636+0000 7fc1ea400640 1 -- 172.21.15.26:0/2902227878 wait complete. 2024-01-15T18:10:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.638+0000 7fc1ea400640 1 Processor -- start 2024-01-15T18:10:07.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.638+0000 7fc1ea400640 1 -- start start 2024-01-15T18:10:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.639+0000 7fc1ea400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40cf9a0 0x7fc1e413a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.639+0000 7fc1ea400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1e4143ae0 0x7fc1e4142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.639+0000 7fc1ea400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1e415e3b0 0x7fc1e40af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.639+0000 7fc1ea400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41593b0 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.639+0000 7fc1ea400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e4159690 con 0x7fc1e415e3b0 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.639+0000 7fc1ea400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e4159970 con 0x7fc1e4143ae0 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.639+0000 7fc1e3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40cf9a0 0x7fc1e413a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.639+0000 7fc1e3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40cf9a0 0x7fc1e413a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40454/0 (socket says 172.21.15.26:40454) 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.639+0000 7fc1e3400640 1 -- 172.21.15.26:0/1085591117 learned_addr learned my addr 172.21.15.26:0/1085591117 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1e3e00640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1e415e3b0 0x7fc1e40af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1e2a00640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1e4143ae0 0x7fc1e4142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1e3400640 1 -- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1e4143ae0 msgr2=0x7fc1e4142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1e3400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1e4143ae0 0x7fc1e4142c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1e3400640 1 -- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1e415e3b0 msgr2=0x7fc1e40af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1e3400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1e415e3b0 0x7fc1e40af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1e3400640 1 -- 172.21.15.26:0/1085591117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc009a70 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1e3e00640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1e415e3b0 0x7fc1e40af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1e3400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40cf9a0 0x7fc1e413a590 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400c4a0 tx=0x7fc1d400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1db400640 1 -- 172.21.15.26:0/1085591117 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d40074e0 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1db400640 1 -- 172.21.15.26:0/1085591117 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d4007640 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1db400640 1 -- 172.21.15.26:0/1085591117 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d4004a00 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.640+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e4168110 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.641+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e4167dc0 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.642+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e4159b90 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.643+0000 7fc1db400640 1 -- 172.21.15.26:0/1085591117 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fc1d4004b60 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.644+0000 7fc1db400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1bc06c220 0x7fc1bc06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:07.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.644+0000 7fc1db400640 1 -- 172.21.15.26:0/1085591117 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fc1d4087ac0 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.644+0000 7fc1e2a00640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1bc06c220 0x7fc1bc06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:07.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.645+0000 7fc1e2a00640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1bc06c220 0x7fc1bc06e6e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40b2040 tx=0x7fc1cc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:07.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.648+0000 7fc1db400640 1 -- 172.21.15.26:0/1085591117 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc1e4159b90 con 0x7fc1e40cf9a0 2024-01-15T18:10:07.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:07.916+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled"]} v 0) v1 -- 0x7fc1e406c3f0 con 0x7fc1e40cf9a0 2024-01-15T18:10:08.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.499+0000 7fc1db400640 1 -- 172.21.15.26:0/1085591117 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled"]}]=0 v679) v1 ==== 156+0+0 (secure 0 0 0) 0x7fc1d40180e0 con 0x7fc1e40cf9a0 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1bc06c220 msgr2=0x7fc1bc06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1bc06c220 0x7fc1bc06e6e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40b2040 tx=0x7fc1cc009040 comp rx=0 tx=0).stop 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40cf9a0 msgr2=0x7fc1e413a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40cf9a0 0x7fc1e413a590 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400c4a0 tx=0x7fc1d400c960 comp rx=0 tx=0).stop 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 shutdown_connections 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc1e4143ae0 0x7fc1e4142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc1bc06c220 0x7fc1bc06e6e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc1e415e3b0 0x7fc1e40af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 --2- 172.21.15.26:0/1085591117 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc1e40cf9a0 0x7fc1e413a590 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 >> 172.21.15.26:0/1085591117 conn(0x7fc1e406a800 msgr2=0x7fc1e415b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 shutdown_connections 2024-01-15T18:10:08.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.503+0000 7fc1ea400640 1 -- 172.21.15.26:0/1085591117 wait complete. 2024-01-15T18:10:08.529 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:10:08.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.742+0000 7fc2c2c00640 1 Processor -- start 2024-01-15T18:10:08.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.742+0000 7fc2c2c00640 1 -- start start 2024-01-15T18:10:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.744+0000 7fc2c2c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2bc1447f0 0x7fc2bc15c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.744+0000 7fc2c2c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc2bc0cccf0 0x7fc2bc15c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.744+0000 7fc2c2c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2bc15c5c0 con 0x7fc2bc06c470 2024-01-15T18:10:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.744+0000 7fc2c2c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2bc06d690 con 0x7fc2bc1447f0 2024-01-15T18:10:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.744+0000 7fc2c2c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2bc0cebb0 con 0x7fc2bc0cccf0 2024-01-15T18:10:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.744+0000 7fc2c1800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2bc1447f0 0x7fc2bc15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.744+0000 7fc2c1800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc2bc1447f0 0x7fc2bc15c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40466/0 (socket says 172.21.15.26:40466) 2024-01-15T18:10:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.744+0000 7fc2c1800640 1 -- 172.21.15.26:0/2841009760 learned_addr learned my addr 172.21.15.26:0/2841009760 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.745+0000 7fc2c1800640 1 -- 172.21.15.26:0/2841009760 >> v2:172.21.15.124:3300/0 conn(0x7fc2bc0cccf0 msgr2=0x7fc2bc15c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.745+0000 7fc2c2200640 1 --2- 172.21.15.26:0/2841009760 >> v2:172.21.15.124:3300/0 conn(0x7fc2bc0cccf0 0x7fc2bc15c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.745+0000 7fc2c1800640 1 --2- 172.21.15.26:0/2841009760 >> v2:172.21.15.124:3300/0 conn(0x7fc2bc0cccf0 0x7fc2bc15c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.745+0000 7fc2c1800640 1 -- 172.21.15.26:0/2841009760 >> v1:172.21.15.124:6789/0 conn(0x7fc2bc06c470 legacy=0x7fc2bc0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:08.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.745+0000 7fc2c1800640 1 -- 172.21.15.26:0/2841009760 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc06ca10 con 0x7fc2bc1447f0 2024-01-15T18:10:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.746+0000 7fc2c1800640 1 --2- 172.21.15.26:0/2841009760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 0x7fc2bc15c080 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0aea60 tx=0x7fc2ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33fe7da2420ebc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.746+0000 7fc2bbe00640 1 -- 172.21.15.26:0/2841009760 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0317e0 con 0x7fc2bc1447f0 2024-01-15T18:10:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.746+0000 7fc2bbe00640 1 -- 172.21.15.26:0/2841009760 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac031940 con 0x7fc2bc1447f0 2024-01-15T18:10:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.746+0000 7fc2bbe00640 1 -- 172.21.15.26:0/2841009760 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac031cd0 con 0x7fc2bc1447f0 2024-01-15T18:10:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.747+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2841009760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 msgr2=0x7fc2bc15c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.747+0000 7fc2c2c00640 1 --2- 172.21.15.26:0/2841009760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 0x7fc2bc15c080 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0aea60 tx=0x7fc2ac02b880 comp rx=0 tx=0).stop 2024-01-15T18:10:08.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.747+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2841009760 shutdown_connections 2024-01-15T18:10:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.747+0000 7fc2c2c00640 1 --2- 172.21.15.26:0/2841009760 >> v2:172.21.15.124:3300/0 conn(0x7fc2bc0cccf0 0x7fc2bc15c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.747+0000 7fc2c2c00640 1 --2- 172.21.15.26:0/2841009760 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 0x7fc2bc15c080 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.747+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2841009760 >> 172.21.15.26:0/2841009760 conn(0x7fc2bc06a3e0 msgr2=0x7fc2bc06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.747+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2841009760 shutdown_connections 2024-01-15T18:10:08.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.747+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2841009760 wait complete. 2024-01-15T18:10:08.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.749+0000 7fc2c2c00640 1 Processor -- start 2024-01-15T18:10:08.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.749+0000 7fc2c2c00640 1 -- start start 2024-01-15T18:10:08.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c2c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc06c470 0x7fc2bc139d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c2c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc0cccf0 0x7fc2bc13e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c2c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 0x7fc2bc1f0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c2c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc13a2b0 con 0x7fc2bc1447f0 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c2c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc13f490 con 0x7fc2bc0cccf0 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c2c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc13f770 con 0x7fc2bc06c470 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 0x7fc2bc1f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 0x7fc2bc1f0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40468/0 (socket says 172.21.15.26:40468) 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c2200640 1 -- 172.21.15.26:0/2140547611 learned_addr learned my addr 172.21.15.26:0/2140547611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c0e00640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc0cccf0 0x7fc2bc13e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.750+0000 7fc2c1800640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc06c470 0x7fc2bc139d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2c2200640 1 -- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc06c470 msgr2=0x7fc2bc139d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2c2200640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc06c470 0x7fc2bc139d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.756 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2c2200640 1 -- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc0cccf0 msgr2=0x7fc2bc13e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2c2200640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc0cccf0 0x7fc2bc13e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2c2200640 1 -- 172.21.15.26:0/2140547611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac0094d0 con 0x7fc2bc1447f0 2024-01-15T18:10:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2c0e00640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc0cccf0 0x7fc2bc13e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2c2200640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 0x7fc2bc1f0890 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400cd40 tx=0x7fc2b400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2b9600640 1 -- 172.21.15.26:0/2140547611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4003b90 con 0x7fc2bc1447f0 2024-01-15T18:10:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2b9600640 1 -- 172.21.15.26:0/2140547611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b4003cf0 con 0x7fc2bc1447f0 2024-01-15T18:10:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2b9600640 1 -- 172.21.15.26:0/2140547611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4004240 con 0x7fc2bc1447f0 2024-01-15T18:10:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2bc13fa50 con 0x7fc2bc1447f0 2024-01-15T18:10:08.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.751+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2bc140010 con 0x7fc2bc1447f0 2024-01-15T18:10:08.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.752+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2bc140230 con 0x7fc2bc1447f0 2024-01-15T18:10:08.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.753+0000 7fc2b9600640 1 -- 172.21.15.26:0/2140547611 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fc2b40043a0 con 0x7fc2bc1447f0 2024-01-15T18:10:08.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.754+0000 7fc2b9600640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc29406c1f0 0x7fc29406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:08.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.754+0000 7fc2b9600640 1 -- 172.21.15.26:0/2140547611 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fc2b4012070 con 0x7fc2bc1447f0 2024-01-15T18:10:08.759 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.755+0000 7fc2c1800640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc29406c1f0 0x7fc29406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:08.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.756+0000 7fc2c1800640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc29406c1f0 0x7fc29406e6b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac036000 tx=0x7fc2ac03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:08.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.759+0000 7fc2b9600640 1 -- 172.21.15.26:0/2140547611 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2b40559f0 con 0x7fc2bc1447f0 2024-01-15T18:10:08.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.952+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc2bc067450 con 0x7fc2bc1447f0 2024-01-15T18:10:08.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.959+0000 7fc2b9600640 1 -- 172.21.15.26:0/2140547611 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2b4059010 con 0x7fc2bc1447f0 2024-01-15T18:10:08.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc29406c1f0 msgr2=0x7fc29406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc29406c1f0 0x7fc29406e6b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac036000 tx=0x7fc2ac03d040 comp rx=0 tx=0).stop 2024-01-15T18:10:08.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 msgr2=0x7fc2bc1f0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:08.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 0x7fc2bc1f0890 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400cd40 tx=0x7fc2b400a660 comp rx=0 tx=0).stop 2024-01-15T18:10:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 shutdown_connections 2024-01-15T18:10:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc29406c1f0 0x7fc29406e6b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc2bc1447f0 0x7fc2bc1f0890 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc2bc0cccf0 0x7fc2bc13e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 --2- 172.21.15.26:0/2140547611 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc2bc06c470 0x7fc2bc139d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.961+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 >> 172.21.15.26:0/2140547611 conn(0x7fc2bc06a3e0 msgr2=0x7fc2bc15c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.962+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 shutdown_connections 2024-01-15T18:10:08.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:08.962+0000 7fc2c2c00640 1 -- 172.21.15.26:0/2140547611 wait complete. 2024-01-15T18:10:08.985 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:10:08.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:10:09.103 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:10:09.220 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:10:09.221 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:10:09.221 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:09.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:10:09.281 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:10:09.281 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:09.336 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:10:09.336 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:10:09.336 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:09.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:10:09.395 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:10:09.395 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:09.451 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:10:09.451 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:10:09.451 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:10:09.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.675+0000 7f9b8ba00640 1 Processor -- start 2024-01-15T18:10:09.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.675+0000 7f9b8ba00640 1 -- start start 2024-01-15T18:10:09.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.677+0000 7f9b8ba00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b8413df80 0x7f9b8415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.677+0000 7f9b8ba00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9b840cf9a0 0x7f9b840cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.677+0000 7f9b8ba00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8406c470 con 0x7f9b8413e360 2024-01-15T18:10:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.677+0000 7f9b8ba00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8406c750 con 0x7f9b8413df80 2024-01-15T18:10:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.677+0000 7f9b8ba00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8406ca30 con 0x7f9b840cf9a0 2024-01-15T18:10:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.677+0000 7f9b89000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b8413df80 0x7f9b8415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.677+0000 7f9b89000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b8413df80 0x7f9b8415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40470/0 (socket says 172.21.15.26:40470) 2024-01-15T18:10:09.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.677+0000 7f9b89000640 1 -- 172.21.15.26:0/1890377215 learned_addr learned my addr 172.21.15.26:0/1890377215 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.678+0000 7f9b89a00640 1 --2- 172.21.15.26:0/1890377215 >> v2:172.21.15.124:3300/0 conn(0x7f9b840cf9a0 0x7f9b840cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.678+0000 7f9b89000640 1 -- 172.21.15.26:0/1890377215 >> v2:172.21.15.124:3300/0 conn(0x7f9b840cf9a0 msgr2=0x7f9b840cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.678+0000 7f9b89000640 1 --2- 172.21.15.26:0/1890377215 >> v2:172.21.15.124:3300/0 conn(0x7f9b840cf9a0 0x7f9b840cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.678+0000 7f9b89000640 1 -- 172.21.15.26:0/1890377215 >> v1:172.21.15.124:6789/0 conn(0x7f9b8413e360 legacy=0x7f9b840cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.678+0000 7f9b89000640 1 -- 172.21.15.26:0/1890377215 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8406cd10 con 0x7f9b8413df80 2024-01-15T18:10:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.678+0000 7f9b89a00640 1 --2- 172.21.15.26:0/1890377215 >> v2:172.21.15.124:3300/0 conn(0x7f9b840cf9a0 0x7f9b840cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:09.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.679+0000 7f9b89000640 1 --2- 172.21.15.26:0/1890377215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 0x7f9b8415bee0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f9b840ccb10 tx=0x7f9b7402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d1029118bca092 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:09.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b83e00640 1 -- 172.21.15.26:0/1890377215 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7402ca80 con 0x7f9b8413df80 2024-01-15T18:10:09.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b83e00640 1 -- 172.21.15.26:0/1890377215 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7402cbe0 con 0x7f9b8413df80 2024-01-15T18:10:09.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b83e00640 1 -- 172.21.15.26:0/1890377215 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b740343f0 con 0x7f9b8413df80 2024-01-15T18:10:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b8ba00640 1 -- 172.21.15.26:0/1890377215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 msgr2=0x7f9b8415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b8ba00640 1 --2- 172.21.15.26:0/1890377215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 0x7f9b8415bee0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f9b840ccb10 tx=0x7f9b7402bee0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b8ba00640 1 -- 172.21.15.26:0/1890377215 shutdown_connections 2024-01-15T18:10:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b8ba00640 1 --2- 172.21.15.26:0/1890377215 >> v2:172.21.15.124:3300/0 conn(0x7f9b840cf9a0 0x7f9b840cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b8ba00640 1 --2- 172.21.15.26:0/1890377215 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 0x7f9b8415bee0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b8ba00640 1 -- 172.21.15.26:0/1890377215 >> 172.21.15.26:0/1890377215 conn(0x7f9b8406a800 msgr2=0x7f9b8406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b8ba00640 1 -- 172.21.15.26:0/1890377215 shutdown_connections 2024-01-15T18:10:09.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.680+0000 7f9b8ba00640 1 -- 172.21.15.26:0/1890377215 wait complete. 2024-01-15T18:10:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.682+0000 7f9b8ba00640 1 Processor -- start 2024-01-15T18:10:09.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.682+0000 7f9b8ba00640 1 -- start start 2024-01-15T18:10:09.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.683+0000 7f9b8ba00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b840cf9a0 0x7f9b841528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.683+0000 7f9b8ba00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 0x7f9b840a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.683+0000 7f9b8ba00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b8413e360 0x7f9b8413d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.683+0000 7f9b8ba00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b840a9720 con 0x7f9b8413df80 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.683+0000 7f9b8ba00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84152fa0 con 0x7f9b8413e360 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.683+0000 7f9b8ba00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84153280 con 0x7f9b840cf9a0 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b7bc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 0x7f9b840a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b7bc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 0x7f9b840a91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40480/0 (socket says 172.21.15.26:40480) 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b7bc00640 1 -- 172.21.15.26:0/3807680928 learned_addr learned my addr 172.21.15.26:0/3807680928 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b89a00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b8413e360 0x7f9b8413d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b89000640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b840cf9a0 0x7f9b841528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b7bc00640 1 -- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b840cf9a0 msgr2=0x7f9b841528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:09.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b7bc00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b840cf9a0 0x7f9b841528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b7bc00640 1 -- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b8413e360 msgr2=0x7f9b8413d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b7bc00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b8413e360 0x7f9b8413d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.684+0000 7f9b7bc00640 1 -- 172.21.15.26:0/3807680928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b74009b30 con 0x7f9b8413df80 2024-01-15T18:10:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.685+0000 7f9b7bc00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 0x7f9b840a91e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c400 tx=0x7f9b6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:09.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.685+0000 7f9b81600640 1 -- 172.21.15.26:0/3807680928 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c0035f0 con 0x7f9b8413df80 2024-01-15T18:10:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.685+0000 7f9b81600640 1 -- 172.21.15.26:0/3807680928 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6c003750 con 0x7f9b8413df80 2024-01-15T18:10:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.685+0000 7f9b81600640 1 -- 172.21.15.26:0/3807680928 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c003b40 con 0x7f9b8413df80 2024-01-15T18:10:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.685+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b84153560 con 0x7f9b8413df80 2024-01-15T18:10:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.685+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b84153b20 con 0x7f9b8413df80 2024-01-15T18:10:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.686+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b840675d0 con 0x7f9b8413df80 2024-01-15T18:10:09.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.687+0000 7f9b81600640 1 -- 172.21.15.26:0/3807680928 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f9b6c003ca0 con 0x7f9b8413df80 2024-01-15T18:10:09.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.688+0000 7f9b81600640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5c06c220 0x7f9b5c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:09.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.688+0000 7f9b81600640 1 -- 172.21.15.26:0/3807680928 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f9b6c087b40 con 0x7f9b8413df80 2024-01-15T18:10:09.693 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.689+0000 7f9b89000640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5c06c220 0x7f9b5c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:09.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.690+0000 7f9b89000640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5c06c220 0x7f9b5c06e6e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9b74035000 tx=0x7f9b74030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:09.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.692+0000 7f9b81600640 1 -- 172.21.15.26:0/3807680928 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b6c0541c0 con 0x7f9b8413df80 2024-01-15T18:10:09.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.923+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b8406df00 con 0x7f9b8413df80 2024-01-15T18:10:09.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.924+0000 7f9b81600640 1 -- 172.21.15.26:0/3807680928 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3698 (secure 0 0 0) 0x7f9b6c017e90 con 0x7f9b8413df80 2024-01-15T18:10:09.929 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:09.929 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8223,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/923394527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":923394527},{"type":"v1","addr":"172.21.15.26:6833","nonce":923394527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":8226,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/350014829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":350014829},{"type":"v1","addr":"172.21.15.26:6835","nonce":350014829}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":36005,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6836/1826406197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1826406197},{"type":"v1","addr":"172.21.15.124:6836","nonce":1826406197}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":36191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/926524128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6835","nonce":926524128},{"type":"v1","addr":"172.21.15.124:6837","nonce":926524128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2024-01-15T18:10:09.930 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 56 2024-01-15T18:10:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.927+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5c06c220 msgr2=0x7f9b5c06e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.927+0000 7f9b8ba00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5c06c220 0x7f9b5c06e6e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f9b74035000 tx=0x7f9b74030000 comp rx=0 tx=0).stop 2024-01-15T18:10:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.927+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 msgr2=0x7f9b840a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.927+0000 7f9b8ba00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 0x7f9b840a91e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c00c400 tx=0x7f9b6c00c8c0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.927+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 shutdown_connections 2024-01-15T18:10:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.927+0000 7f9b8ba00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b5c06c220 0x7f9b5c06e6e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.927+0000 7f9b8ba00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b8413e360 0x7f9b8413d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.927+0000 7f9b8ba00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b8413df80 0x7f9b840a91e0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.927+0000 7f9b8ba00640 1 --2- 172.21.15.26:0/3807680928 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b840cf9a0 0x7f9b841528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.928+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 >> 172.21.15.26:0/3807680928 conn(0x7f9b8406a800 msgr2=0x7f9b840cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.928+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 shutdown_connections 2024-01-15T18:10:09.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:09.928+0000 7f9b8ba00640 1 -- 172.21.15.26:0/3807680928 wait complete. 2024-01-15T18:10:09.953 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:10:09.953 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:10:09.953 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:10:09.953 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:10:09.954 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:10:09.954 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:10:09.954 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:10:09.954 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:10:09.954 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:10:09.954 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:10:09.954 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:10:09.954 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:10:09.955 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:10:09.955 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:10:09.955 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:10:09.955 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:10:09.974 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:10:09.975 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:10:09.979 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:10:09.979 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:10:09.991 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:10:09.992 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:10:10.026 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:10:10.027 DEBUG:teuthology.parallel:result is None 2024-01-15T18:10:10.032 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:10:10.033 DEBUG:teuthology.parallel:result is None 2024-01-15T18:10:10.040 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:10:10.040 DEBUG:teuthology.parallel:result is None 2024-01-15T18:10:10.187 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T18:10:10.189 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T18:10:10.193 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-01-15T18:10:10.230 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T18:10:15.989 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:10:15.990 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:10:15.998 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:10:15.998 DEBUG:teuthology.parallel:result is None 2024-01-15T18:10:15.999 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:10:16.164 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-15T18:10:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.198+0000 7f40f6000640 1 Processor -- start 2024-01-15T18:10:16.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.198+0000 7f40f6000640 1 -- start start 2024-01-15T18:10:16.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.199+0000 7f40f6000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f40f015d3c0 0x7f40f00d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.199+0000 7f40f6000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f40f00cfa40 0x7f40f00cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.199+0000 7f40f6000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f00b2c70 con 0x7f40f00cde90 2024-01-15T18:10:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.199+0000 7f40f6000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f00d03f0 con 0x7f40f00cfa40 2024-01-15T18:10:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.199+0000 7f40f6000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f00cf6d0 con 0x7f40f015d3c0 2024-01-15T18:10:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.199+0000 7f40e7c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f40f00cfa40 0x7f40f00cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.199+0000 7f40e7c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f40f00cfa40 0x7f40f00cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57056/0 (socket says 172.21.15.26:57056) 2024-01-15T18:10:16.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.199+0000 7f40e7c00640 1 -- 172.21.15.26:0/3012459942 learned_addr learned my addr 172.21.15.26:0/3012459942 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.200+0000 7f40e7c00640 1 -- 172.21.15.26:0/3012459942 >> v2:172.21.15.124:3300/0 conn(0x7f40f015d3c0 msgr2=0x7f40f00d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.200+0000 7f40f4c00640 1 --2- 172.21.15.26:0/3012459942 >> v2:172.21.15.124:3300/0 conn(0x7f40f015d3c0 0x7f40f00d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.200+0000 7f40e7c00640 1 --2- 172.21.15.26:0/3012459942 >> v2:172.21.15.124:3300/0 conn(0x7f40f015d3c0 0x7f40f00d1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.200+0000 7f40e7c00640 1 -- 172.21.15.26:0/3012459942 >> v1:172.21.15.124:6789/0 conn(0x7f40f00cde90 legacy=0x7f40f00ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.200+0000 7f40e7c00640 1 -- 172.21.15.26:0/3012459942 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f00cf3f0 con 0x7f40f00cfa40 2024-01-15T18:10:16.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.200+0000 7f40f4c00640 1 --2- 172.21.15.26:0/3012459942 >> v2:172.21.15.124:3300/0 conn(0x7f40f015d3c0 0x7f40f00d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:16.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.201+0000 7f40e7c00640 1 --2- 172.21.15.26:0/3012459942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cfa40 0x7f40f00cfeb0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f40f008b6c0 tx=0x7f40d802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ecfa449a94d139 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:16.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.201+0000 7f40efe00640 1 -- 172.21.15.26:0/3012459942 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d802ca80 con 0x7f40f00cfa40 2024-01-15T18:10:16.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.201+0000 7f40efe00640 1 -- 172.21.15.26:0/3012459942 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d802cbe0 con 0x7f40f00cfa40 2024-01-15T18:10:16.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.201+0000 7f40efe00640 1 -- 172.21.15.26:0/3012459942 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d80343f0 con 0x7f40f00cfa40 2024-01-15T18:10:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.202+0000 7f40f6000640 1 -- 172.21.15.26:0/3012459942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cfa40 msgr2=0x7f40f00cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.202+0000 7f40f6000640 1 --2- 172.21.15.26:0/3012459942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cfa40 0x7f40f00cfeb0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f40f008b6c0 tx=0x7f40d802bee0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.202+0000 7f40f6000640 1 -- 172.21.15.26:0/3012459942 shutdown_connections 2024-01-15T18:10:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.202+0000 7f40f6000640 1 --2- 172.21.15.26:0/3012459942 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cfa40 0x7f40f00cfeb0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.202+0000 7f40f6000640 1 --2- 172.21.15.26:0/3012459942 >> v2:172.21.15.124:3300/0 conn(0x7f40f015d3c0 0x7f40f00d1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.202+0000 7f40f6000640 1 -- 172.21.15.26:0/3012459942 >> 172.21.15.26:0/3012459942 conn(0x7f40f00d3050 msgr2=0x7f40f00d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.202+0000 7f40f6000640 1 -- 172.21.15.26:0/3012459942 shutdown_connections 2024-01-15T18:10:16.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.202+0000 7f40f6000640 1 -- 172.21.15.26:0/3012459942 wait complete. 2024-01-15T18:10:16.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.204+0000 7f40f6000640 1 Processor -- start 2024-01-15T18:10:16.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.204+0000 7f40f6000640 1 -- start start 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.205+0000 7f40f6000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cde90 0x7f40f00cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.205+0000 7f40f6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40f00cfa40 0x7f40f00b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.205+0000 7f40f6000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40f015d3c0 0x7f40f00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.205+0000 7f40f6000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f015e570 con 0x7f40f00cde90 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.205+0000 7f40f6000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f00b8800 con 0x7f40f015d3c0 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.205+0000 7f40f6000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f01609e0 con 0x7f40f00cfa40 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.205+0000 7f40f4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cde90 0x7f40f00cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.205+0000 7f40f4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cde90 0x7f40f00cc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57070/0 (socket says 172.21.15.26:57070) 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.205+0000 7f40f4c00640 1 -- 172.21.15.26:0/1209742966 learned_addr learned my addr 172.21.15.26:0/1209742966 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40f5600640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40f015d3c0 0x7f40f00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40f4c00640 1 -- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40f00cfa40 msgr2=0x7f40f00b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:16.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40f4c00640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40f00cfa40 0x7f40f00b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40f4c00640 1 -- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40f015d3c0 msgr2=0x7f40f00b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40f4c00640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40f015d3c0 0x7f40f00b77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40f4c00640 1 -- 172.21.15.26:0/1209742966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d8009b30 con 0x7f40f00cde90 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40f5600640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40f015d3c0 0x7f40f00b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40f4c00640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cde90 0x7f40f00cc8f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c820 tx=0x7f40e000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40ed600640 1 -- 172.21.15.26:0/1209742966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007a30 con 0x7f40f00cde90 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40ed600640 1 -- 172.21.15.26:0/1209742966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e0007b90 con 0x7f40f00cde90 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40ed600640 1 -- 172.21.15.26:0/1209742966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000ce10 con 0x7f40f00cde90 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.206+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f015dbd0 con 0x7f40f00cde90 2024-01-15T18:10:16.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.207+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f015e190 con 0x7f40f00cde90 2024-01-15T18:10:16.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.208+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f015e7e0 con 0x7f40f00cde90 2024-01-15T18:10:16.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.209+0000 7f40ed600640 1 -- 172.21.15.26:0/1209742966 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f40e00113f0 con 0x7f40f00cde90 2024-01-15T18:10:16.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.210+0000 7f40ed600640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f40c806c1f0 0x7f40c806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:16.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.210+0000 7f40ed600640 1 -- 172.21.15.26:0/1209742966 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 4999+0+0 (secure 0 0 0) 0x7f40e0088770 con 0x7f40f00cde90 2024-01-15T18:10:16.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.210+0000 7f40e7c00640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f40c806c1f0 0x7f40c806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:16.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.211+0000 7f40e7c00640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f40c806c1f0 0x7f40c806e6b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f40d8035000 tx=0x7f40d802e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:16.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.214+0000 7f40ed600640 1 -- 172.21.15.26:0/1209742966 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40e0055090 con 0x7f40f00cde90 2024-01-15T18:10:16.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:16.426+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f40f008ba80 con 0x7f40f00cde90 2024-01-15T18:10:16.907 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-01-15T18:10:17.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.537+0000 7f40ed600640 1 -- 172.21.15.26:0/1209742966 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) v1 ==== 97+0+0 (secure 0 0 0) 0x7f40e00586b0 con 0x7f40f00cde90 2024-01-15T18:10:17.542 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-01-15T18:10:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.541+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f40c806c1f0 msgr2=0x7f40c806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.541+0000 7f40f6000640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f40c806c1f0 0x7f40c806e6b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f40d8035000 tx=0x7f40d802e000 comp rx=0 tx=0).stop 2024-01-15T18:10:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cde90 msgr2=0x7f40f00cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:17.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cde90 0x7f40f00cc8f0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c820 tx=0x7f40e000cce0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 shutdown_connections 2024-01-15T18:10:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f40f00cfa40 0x7f40f00b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f40c806c1f0 0x7f40c806e6b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f40f015d3c0 0x7f40f00b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 --2- 172.21.15.26:0/1209742966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f40f00cde90 0x7f40f00cc8f0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 >> 172.21.15.26:0/1209742966 conn(0x7f40f00d3050 msgr2=0x7f40f00b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 shutdown_connections 2024-01-15T18:10:17.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.542+0000 7f40f6000640 1 -- 172.21.15.26:0/1209742966 wait complete. 2024-01-15T18:10:17.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:10:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.776+0000 7fafbe800640 1 Processor -- start 2024-01-15T18:10:17.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.776+0000 7fafbe800640 1 -- start start 2024-01-15T18:10:17.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.778+0000 7fafbe800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b3ec0 0x7fafb81535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:17.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.778+0000 7fafbe800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fafb80b25f0 0x7fafb806c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:17.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.778+0000 7fafbe800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb80b3190 con 0x7fafb806e1e0 2024-01-15T18:10:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.779+0000 7fafbe800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb80b2c10 con 0x7fafb80b25f0 2024-01-15T18:10:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.779+0000 7fafbe800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80b33e0 con 0x7fafb80b3ec0 2024-01-15T18:10:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.779+0000 7fafbca00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fafb806e1e0 0x7fafb80b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60968/0 (socket says 172.21.15.26:60968) 2024-01-15T18:10:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.779+0000 7fafbca00640 1 -- 172.21.15.26:0/3196851726 learned_addr learned my addr 172.21.15.26:0/3196851726 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.779+0000 7fafbde00640 1 --2- 172.21.15.26:0/3196851726 >> v2:172.21.15.26:3300/0 conn(0x7fafb80b25f0 0x7fafb806c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:17.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.779+0000 7fafbd400640 1 --2- 172.21.15.26:0/3196851726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b3ec0 0x7fafb81535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.779+0000 7fafbde00640 1 -- 172.21.15.26:0/3196851726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b3ec0 msgr2=0x7fafb81535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.780+0000 7fafbde00640 1 --2- 172.21.15.26:0/3196851726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b3ec0 0x7fafb81535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.780+0000 7fafbde00640 1 -- 172.21.15.26:0/3196851726 >> v1:172.21.15.26:6789/0 conn(0x7fafb806e1e0 legacy=0x7fafb80b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.780+0000 7fafbde00640 1 -- 172.21.15.26:0/3196851726 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb80b36c0 con 0x7fafb80b25f0 2024-01-15T18:10:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.780+0000 7fafb7e00640 1 -- 172.21.15.26:0/3196851726 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849619802 0 0) 0x7fafb80b3190 con 0x7fafb806e1e0 2024-01-15T18:10:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.780+0000 7fafbd400640 1 --2- 172.21.15.26:0/3196851726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b3ec0 0x7fafb81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:17.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.781+0000 7fafbde00640 1 --2- 172.21.15.26:0/3196851726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b25f0 0x7fafb806c470 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097a0 tx=0x7fafb002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732758266b80842 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:17.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.781+0000 7fafb7e00640 1 -- 172.21.15.26:0/3196851726 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb002e070 con 0x7fafb80b25f0 2024-01-15T18:10:17.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.781+0000 7fafb7e00640 1 -- 172.21.15.26:0/3196851726 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb00315a0 con 0x7fafb80b25f0 2024-01-15T18:10:17.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.781+0000 7fafb7e00640 1 -- 172.21.15.26:0/3196851726 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0031930 con 0x7fafb80b25f0 2024-01-15T18:10:17.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.782+0000 7fafbe800640 1 -- 172.21.15.26:0/3196851726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b25f0 msgr2=0x7fafb806c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:17.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.782+0000 7fafbe800640 1 --2- 172.21.15.26:0/3196851726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b25f0 0x7fafb806c470 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097a0 tx=0x7fafb002b840 comp rx=0 tx=0).stop 2024-01-15T18:10:17.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.782+0000 7fafbe800640 1 -- 172.21.15.26:0/3196851726 shutdown_connections 2024-01-15T18:10:17.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.782+0000 7fafbe800640 1 --2- 172.21.15.26:0/3196851726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b25f0 0x7fafb806c470 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.782+0000 7fafbe800640 1 --2- 172.21.15.26:0/3196851726 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b3ec0 0x7fafb81535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.782+0000 7fafbe800640 1 -- 172.21.15.26:0/3196851726 >> 172.21.15.26:0/3196851726 conn(0x7fafb806a3e0 msgr2=0x7fafb806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:17.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.782+0000 7fafbe800640 1 -- 172.21.15.26:0/3196851726 shutdown_connections 2024-01-15T18:10:17.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.782+0000 7fafbe800640 1 -- 172.21.15.26:0/3196851726 wait complete. 2024-01-15T18:10:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.783+0000 7fafbe800640 1 Processor -- start 2024-01-15T18:10:17.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.784+0000 7fafbe800640 1 -- start start 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.784+0000 7fafbe800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fafb806e1e0 0x7fafb8158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbe800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b25f0 0x7fafb8144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbe800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b3ec0 0x7fafb81471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbe800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb8157e50 con 0x7fafb80b3ec0 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbe800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb8158130 con 0x7fafb806e1e0 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbe800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb8158410 con 0x7fafb80b25f0 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b3ec0 0x7fafb81471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b3ec0 0x7fafb81471a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57076/0 (socket says 172.21.15.26:57076) 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbde00640 1 -- 172.21.15.26:0/4032812937 learned_addr learned my addr 172.21.15.26:0/4032812937 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbd400640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fafb806e1e0 0x7fafb8158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbde00640 1 -- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b25f0 msgr2=0x7fafb8144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbde00640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b25f0 0x7fafb8144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbde00640 1 -- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fafb806e1e0 msgr2=0x7fafb8158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbde00640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fafb806e1e0 0x7fafb8158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.785+0000 7fafbde00640 1 -- 172.21.15.26:0/4032812937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb0009400 con 0x7fafb80b3ec0 2024-01-15T18:10:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.786+0000 7fafbde00640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b3ec0 0x7fafb81471a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097a0 tx=0x7fafb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.786+0000 7fafb5600640 1 -- 172.21.15.26:0/4032812937 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0032840 con 0x7fafb80b3ec0 2024-01-15T18:10:17.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.786+0000 7fafb5600640 1 -- 172.21.15.26:0/4032812937 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb00329c0 con 0x7fafb80b3ec0 2024-01-15T18:10:17.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.786+0000 7fafb5600640 1 -- 172.21.15.26:0/4032812937 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0032d70 con 0x7fafb80b3ec0 2024-01-15T18:10:17.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.786+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb81586f0 con 0x7fafb80b3ec0 2024-01-15T18:10:17.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.786+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb8145650 con 0x7fafb80b3ec0 2024-01-15T18:10:17.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.788+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb8158c10 con 0x7fafb80b3ec0 2024-01-15T18:10:17.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.788+0000 7fafb5600640 1 -- 172.21.15.26:0/4032812937 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fafb003b3f0 con 0x7fafb80b3ec0 2024-01-15T18:10:17.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.789+0000 7fafb5600640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf9006c1f0 0x7faf9006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:17.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.790+0000 7fafb5600640 1 -- 172.21.15.26:0/4032812937 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fafb00aa970 con 0x7fafb80b3ec0 2024-01-15T18:10:17.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.790+0000 7fafbd400640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf9006c1f0 0x7faf9006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:17.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.791+0000 7fafbd400640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf9006c1f0 0x7faf9006e6b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fafa8003fd0 tx=0x7fafa8003ce0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:17.798 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:17.794+0000 7fafb5600640 1 -- 172.21.15.26:0/4032812937 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fafb8158c10 con 0x7fafb80b3ec0 2024-01-15T18:10:18.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.064+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fafb8158c10 con 0x7fafb80b3ec0 2024-01-15T18:10:18.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.065+0000 7fafb5600640 1 -- 172.21.15.26:0/4032812937 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fafb8158c10 con 0x7fafb80b3ec0 2024-01-15T18:10:18.069 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVl3erNJhAANxPK+39yVM5neq9iKC1HkA==", 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.070 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlLaUlDRAAeKaen5hLlwDDWY6OAMRsPQ==", 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlaIAVOBAAA8LEFM1HtzRtNooQCCjS/w==", 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.071 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlmpTBHRAAUwbZeVpysMsn56QRLRJCoA==", 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:10:18.072 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA7caVlCTmgDhAAPWb0qEh134xxyF8lM9DoNg==", 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-01-15T18:10:18.073 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA+caVlaxF5JxAA23Z9j21ATy3GVlhgndDIxw==", 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBCcaVlcRtjBRAAOFN9Xf5kN5MXL6yQfRI27g==", 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.074 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBFcaVlvAzEHhAACDoNh7BZY9gbKTdGwIrhlg==", 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBMcaVl8yiOMBAAdI4NTeSB1d/nSHRaIDMcFw==", 2024-01-15T18:10:18.075 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBQcaVltf8sARAAfPaJAfoTgQoEivw90jRz/Q==", 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.076 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBTcaVltuKiGhAAT0cIJiRkQ38QXT/h6Gyggg==", 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBWcaVlbJqVMhAAKjpl6pAHbytjZWMYeRbRjA==", 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.077 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA==", 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.078 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVlrVpqBxAAxOL0iuAlOvfR+KgSsin3vQ==", 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlO8vdNxAAlUE/TFCDQoXuxvwLPwUvtg==", 2024-01-15T18:10:18.079 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVl5qgCEhAALgt7d3OjEmq5bc4nAD2rTw==", 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:10:18.080 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA2caVlyg4eHBAAwTCXVk8hO9+zcZ1tUA3gfA==", 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-01-15T18:10:18.081 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlvuBcGhAAGqdCMSffJwnMKlFh3CtyCw==", 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVle8Z+DhAA9cWxzc4puvrHAedqupPG3Q==", 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:10:18.082 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:10:18.083 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.083 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:18.083 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-01-15T18:10:18.083 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:10:18.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.067+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf9006c1f0 msgr2=0x7faf9006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.067+0000 7fafbe800640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf9006c1f0 0x7faf9006e6b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fafa8003fd0 tx=0x7fafa8003ce0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.067+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b3ec0 msgr2=0x7fafb81471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.067+0000 7fafbe800640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b3ec0 0x7fafb81471a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fafb00097a0 tx=0x7fafb00046c0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.068+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 shutdown_connections 2024-01-15T18:10:18.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.068+0000 7fafbe800640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fafb80b25f0 0x7fafb8144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.068+0000 7fafbe800640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faf9006c1f0 0x7faf9006e6b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.068+0000 7fafbe800640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fafb80b3ec0 0x7fafb81471a0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.068+0000 7fafbe800640 1 --2- 172.21.15.26:0/4032812937 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fafb806e1e0 0x7fafb8158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.068+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 >> 172.21.15.26:0/4032812937 conn(0x7fafb806a3e0 msgr2=0x7fafb80b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:18.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.068+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 shutdown_connections 2024-01-15T18:10:18.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.068+0000 7fafbe800640 1 -- 172.21.15.26:0/4032812937 wait complete. 2024-01-15T18:10:18.090 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:10:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.303+0000 7f1a8e000640 1 Processor -- start 2024-01-15T18:10:18.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.303+0000 7f1a8e000640 1 -- start start 2024-01-15T18:10:18.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.305+0000 7f1a8e000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1a880cc500 0x7f1a880cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.305+0000 7f1a8e000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a8806d160 0x7f1a8806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.305+0000 7f1a8e000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a88070e50 con 0x7f1a88143c40 2024-01-15T18:10:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.305+0000 7f1a8e000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a880723c0 con 0x7f1a8806d160 2024-01-15T18:10:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.305+0000 7f1a8e000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a880cced0 con 0x7f1a880cc500 2024-01-15T18:10:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.305+0000 7f1a8d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a8806d160 0x7f1a8806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.305+0000 7f1a8d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1a8806d160 0x7f1a8806d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57080/0 (socket says 172.21.15.26:57080) 2024-01-15T18:10:18.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.305+0000 7f1a8d600640 1 -- 172.21.15.26:0/187021153 learned_addr learned my addr 172.21.15.26:0/187021153 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.306+0000 7f1a8d600640 1 -- 172.21.15.26:0/187021153 >> v2:172.21.15.124:3300/0 conn(0x7f1a880cc500 msgr2=0x7f1a880cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.306+0000 7f1a87e00640 1 --2- 172.21.15.26:0/187021153 >> v2:172.21.15.124:3300/0 conn(0x7f1a880cc500 0x7f1a880cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.306+0000 7f1a8d600640 1 --2- 172.21.15.26:0/187021153 >> v2:172.21.15.124:3300/0 conn(0x7f1a880cc500 0x7f1a880cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.306+0000 7f1a8d600640 1 -- 172.21.15.26:0/187021153 >> v1:172.21.15.124:6789/0 conn(0x7f1a88143c40 legacy=0x7f1a88071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.306+0000 7f1a8d600640 1 -- 172.21.15.26:0/187021153 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a88144020 con 0x7f1a8806d160 2024-01-15T18:10:18.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.307+0000 7f1a8d600640 1 --2- 172.21.15.26:0/187021153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a8806d160 0x7f1a8806d560 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f1a880b44a0 tx=0x7f1a8002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18e03409eba492c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.308+0000 7f1a87400640 1 -- 172.21.15.26:0/187021153 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a800317e0 con 0x7f1a8806d160 2024-01-15T18:10:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.308+0000 7f1a87400640 1 -- 172.21.15.26:0/187021153 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a80031940 con 0x7f1a8806d160 2024-01-15T18:10:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.308+0000 7f1a87400640 1 -- 172.21.15.26:0/187021153 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80031cd0 con 0x7f1a8806d160 2024-01-15T18:10:18.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.308+0000 7f1a8e000640 1 -- 172.21.15.26:0/187021153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a8806d160 msgr2=0x7f1a8806d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.308+0000 7f1a8e000640 1 --2- 172.21.15.26:0/187021153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a8806d160 0x7f1a8806d560 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f1a880b44a0 tx=0x7f1a8002b880 comp rx=0 tx=0).stop 2024-01-15T18:10:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.309+0000 7f1a8e000640 1 -- 172.21.15.26:0/187021153 shutdown_connections 2024-01-15T18:10:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.309+0000 7f1a8e000640 1 --2- 172.21.15.26:0/187021153 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a8806d160 0x7f1a8806d560 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.309+0000 7f1a8e000640 1 --2- 172.21.15.26:0/187021153 >> v2:172.21.15.124:3300/0 conn(0x7f1a880cc500 0x7f1a880cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.309+0000 7f1a8e000640 1 -- 172.21.15.26:0/187021153 >> 172.21.15.26:0/187021153 conn(0x7f1a8806a800 msgr2=0x7f1a8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.309+0000 7f1a8e000640 1 -- 172.21.15.26:0/187021153 shutdown_connections 2024-01-15T18:10:18.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.309+0000 7f1a8e000640 1 -- 172.21.15.26:0/187021153 wait complete. 2024-01-15T18:10:18.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.311+0000 7f1a8e000640 1 Processor -- start 2024-01-15T18:10:18.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.311+0000 7f1a8e000640 1 -- start start 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8e000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8806d160 0x7f1a8813ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8e000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a880cc500 0x7f1a8813df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a88143c40 0x7f1a881e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8e000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8813e480 con 0x7f1a88143c40 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8e000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a881e7500 con 0x7f1a880cc500 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8e000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a881e68f0 con 0x7f1a8806d160 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a88143c40 0x7f1a881e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a88143c40 0x7f1a881e03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57082/0 (socket says 172.21.15.26:57082) 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8d600640 1 -- 172.21.15.26:0/1700431228 learned_addr learned my addr 172.21.15.26:0/1700431228 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8cc00640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8806d160 0x7f1a8813ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a87e00640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a880cc500 0x7f1a8813df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8d600640 1 -- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8806d160 msgr2=0x7f1a8813ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8d600640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8806d160 0x7f1a8813ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8d600640 1 -- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a880cc500 msgr2=0x7f1a8813df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8d600640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a880cc500 0x7f1a8813df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a8d600640 1 -- 172.21.15.26:0/1700431228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a800094d0 con 0x7f1a88143c40 2024-01-15T18:10:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.312+0000 7f1a87e00640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a880cc500 0x7f1a8813df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:18.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.313+0000 7f1a8d600640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a88143c40 0x7f1a881e03e0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f1a80034040 tx=0x7f1a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.313+0000 7f1a84c00640 1 -- 172.21.15.26:0/1700431228 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80032940 con 0x7f1a88143c40 2024-01-15T18:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.313+0000 7f1a84c00640 1 -- 172.21.15.26:0/1700431228 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a80032aa0 con 0x7f1a88143c40 2024-01-15T18:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.313+0000 7f1a84c00640 1 -- 172.21.15.26:0/1700431228 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80032e10 con 0x7f1a88143c40 2024-01-15T18:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.313+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a881e6bd0 con 0x7f1a88143c40 2024-01-15T18:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.313+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a881e7190 con 0x7f1a88143c40 2024-01-15T18:10:18.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.315+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a881e7690 con 0x7f1a88143c40 2024-01-15T18:10:18.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.316+0000 7f1a84c00640 1 -- 172.21.15.26:0/1700431228 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f1a8004d3f0 con 0x7f1a88143c40 2024-01-15T18:10:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.317+0000 7f1a84c00640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a6006c240 0x7f1a6006e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.317+0000 7f1a84c00640 1 -- 172.21.15.26:0/1700431228 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f1a800aa5c0 con 0x7f1a88143c40 2024-01-15T18:10:18.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.317+0000 7f1a8cc00640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a6006c240 0x7f1a6006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.318+0000 7f1a8cc00640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a6006c240 0x7f1a6006e700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009f20 tx=0x7f1a780094d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:18.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.321+0000 7f1a84c00640 1 -- 172.21.15.26:0/1700431228 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a80076fa0 con 0x7f1a88143c40 2024-01-15T18:10:18.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.518+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a88067450 con 0x7f1a88143c40 2024-01-15T18:10:18.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.520+0000 7f1a84c00640 1 -- 172.21.15.26:0/1700431228 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+12921 (secure 0 0 0) 0x7f1a8007a5c0 con 0x7f1a88143c40 2024-01-15T18:10:18.524 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:18.525 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":98,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:10:17.535578+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a6006c240 msgr2=0x7f1a6006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a6006c240 0x7f1a6006e700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009f20 tx=0x7f1a780094d0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a88143c40 msgr2=0x7f1a881e03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a88143c40 0x7f1a881e03e0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f1a80034040 tx=0x7f1a800046c0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 shutdown_connections 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1a6006c240 0x7f1a6006e700 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1a88143c40 0x7f1a881e03e0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1a880cc500 0x7f1a8813df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 --2- 172.21.15.26:0/1700431228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1a8806d160 0x7f1a8813ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.523+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 >> 172.21.15.26:0/1700431228 conn(0x7f1a8806a800 msgr2=0x7f1a8806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.524+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 shutdown_connections 2024-01-15T18:10:18.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.524+0000 7f1a8e000640 1 -- 172.21.15.26:0/1700431228 wait complete. 2024-01-15T18:10:18.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:10:18.548 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:10:18.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.769+0000 7fabe2800640 1 Processor -- start 2024-01-15T18:10:18.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.770+0000 7fabe2800640 1 -- start start 2024-01-15T18:10:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.771+0000 7fabe2800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fabdc0adf20 0x7fabdc070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.771+0000 7fabe2800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0ccb50 0x7fabdc143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.772+0000 7fabe2800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc15f4e0 con 0x7fabdc15bf10 2024-01-15T18:10:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.772+0000 7fabe2800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc15f970 con 0x7fabdc0adf20 2024-01-15T18:10:18.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.772+0000 7fabe2800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc071130 con 0x7fabdc0ccb50 2024-01-15T18:10:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.772+0000 7fabe1e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0ccb50 0x7fabdc143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.772+0000 7fabe1e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0ccb50 0x7fabdc143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:56836/0 (socket says 172.21.15.26:56836) 2024-01-15T18:10:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.772+0000 7fabe1e00640 1 -- 172.21.15.26:0/2738625696 learned_addr learned my addr 172.21.15.26:0/2738625696 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.772+0000 7fabe1400640 1 --2- 172.21.15.26:0/2738625696 >> v2:172.21.15.124:3300/0 conn(0x7fabdc0adf20 0x7fabdc070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.773+0000 7fabe1e00640 1 -- 172.21.15.26:0/2738625696 >> v2:172.21.15.124:3300/0 conn(0x7fabdc0adf20 msgr2=0x7fabdc070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:18.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.773+0000 7fabe1e00640 1 --2- 172.21.15.26:0/2738625696 >> v2:172.21.15.124:3300/0 conn(0x7fabdc0adf20 0x7fabdc070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.773+0000 7fabe1e00640 1 -- 172.21.15.26:0/2738625696 >> v1:172.21.15.124:6789/0 conn(0x7fabdc15bf10 legacy=0x7fabdc0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.773+0000 7fabdbe00640 1 -- 172.21.15.26:0/2738625696 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304743246 0 0) 0x7fabdc15f4e0 con 0x7fabdc15bf10 2024-01-15T18:10:18.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.773+0000 7fabe1e00640 1 -- 172.21.15.26:0/2738625696 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc15c390 con 0x7fabdc0ccb50 2024-01-15T18:10:18.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.774+0000 7fabe1e00640 1 --2- 172.21.15.26:0/2738625696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0ccb50 0x7fabdc143d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009870 tx=0x7fabd402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe79c7fa608cd513 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:18.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.775+0000 7fabdbe00640 1 -- 172.21.15.26:0/2738625696 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4031640 con 0x7fabdc0ccb50 2024-01-15T18:10:18.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.775+0000 7fabdbe00640 1 -- 172.21.15.26:0/2738625696 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40317a0 con 0x7fabdc0ccb50 2024-01-15T18:10:18.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.775+0000 7fabdbe00640 1 -- 172.21.15.26:0/2738625696 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4031b30 con 0x7fabdc0ccb50 2024-01-15T18:10:18.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.776+0000 7fabe2800640 1 -- 172.21.15.26:0/2738625696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0ccb50 msgr2=0x7fabdc143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.776+0000 7fabe2800640 1 --2- 172.21.15.26:0/2738625696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0ccb50 0x7fabdc143d30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009870 tx=0x7fabd402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.776+0000 7fabe2800640 1 -- 172.21.15.26:0/2738625696 shutdown_connections 2024-01-15T18:10:18.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.776+0000 7fabe2800640 1 --2- 172.21.15.26:0/2738625696 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0ccb50 0x7fabdc143d30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.776+0000 7fabe2800640 1 --2- 172.21.15.26:0/2738625696 >> v2:172.21.15.124:3300/0 conn(0x7fabdc0adf20 0x7fabdc070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.776+0000 7fabe2800640 1 -- 172.21.15.26:0/2738625696 >> 172.21.15.26:0/2738625696 conn(0x7fabdc06a7d0 msgr2=0x7fabdc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:18.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.776+0000 7fabe2800640 1 -- 172.21.15.26:0/2738625696 shutdown_connections 2024-01-15T18:10:18.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.776+0000 7fabe2800640 1 -- 172.21.15.26:0/2738625696 wait complete. 2024-01-15T18:10:18.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.778+0000 7fabe2800640 1 Processor -- start 2024-01-15T18:10:18.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.778+0000 7fabe2800640 1 -- start start 2024-01-15T18:10:18.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe2800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0adf20 0x7fabdc0a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabdc0ccb50 0x7fabdc0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe2800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabdc15bf10 0x7fabdc0ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe2800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0caf00 con 0x7fabdc0ccb50 2024-01-15T18:10:18.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe2800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc1f0680 con 0x7fabdc15bf10 2024-01-15T18:10:18.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe2800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0a94b0 con 0x7fabdc0adf20 2024-01-15T18:10:18.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabdc0ccb50 0x7fabdc0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe0a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabdc0ccb50 0x7fabdc0ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57090/0 (socket says 172.21.15.26:57090) 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe0a00640 1 -- 172.21.15.26:0/1807373564 learned_addr learned my addr 172.21.15.26:0/1807373564 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe1400640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0adf20 0x7fabdc0a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.780+0000 7fabe1e00640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabdc15bf10 0x7fabdc0ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabe0a00640 1 -- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0adf20 msgr2=0x7fabdc0a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabe0a00640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0adf20 0x7fabdc0a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabe0a00640 1 -- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabdc15bf10 msgr2=0x7fabdc0ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabe0a00640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabdc15bf10 0x7fabdc0ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabe0a00640 1 -- 172.21.15.26:0/1807373564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd40094d0 con 0x7fabdc0ccb50 2024-01-15T18:10:18.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabe0a00640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabdc0ccb50 0x7fabdc0ca9c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c820 tx=0x7fabcc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabd9600640 1 -- 172.21.15.26:0/1807373564 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc003da0 con 0x7fabdc0ccb50 2024-01-15T18:10:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabd9600640 1 -- 172.21.15.26:0/1807373564 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc005070 con 0x7fabdc0ccb50 2024-01-15T18:10:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabd9600640 1 -- 172.21.15.26:0/1807373564 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc005400 con 0x7fabdc0ccb50 2024-01-15T18:10:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc0a9790 con 0x7fabdc0ccb50 2024-01-15T18:10:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.781+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc0ac7e0 con 0x7fabdc0ccb50 2024-01-15T18:10:18.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.783+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc1f0850 con 0x7fabdc0ccb50 2024-01-15T18:10:18.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.784+0000 7fabd9600640 1 -- 172.21.15.26:0/1807373564 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fabcc005560 con 0x7fabdc0ccb50 2024-01-15T18:10:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.786+0000 7fabd9600640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fabb406c1f0 0x7fabb406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.786+0000 7fabd9600640 1 -- 172.21.15.26:0/1807373564 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fabcc087f60 con 0x7fabdc0ccb50 2024-01-15T18:10:18.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.786+0000 7fabe1400640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fabb406c1f0 0x7fabb406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:18.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.787+0000 7fabe1400640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fabb406c1f0 0x7fabb406e6b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fabdc142340 tx=0x7fabc4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:18.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.790+0000 7fabd9600640 1 -- 172.21.15.26:0/1807373564 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fabdc1f0850 con 0x7fabdc0ccb50 2024-01-15T18:10:18.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.981+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabdc1f0850 con 0x7fabdc0ccb50 2024-01-15T18:10:18.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.982+0000 7fabd9600640 1 -- 172.21.15.26:0/1807373564 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabdc1f0850 con 0x7fabdc0ccb50 2024-01-15T18:10:18.987 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:10:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fabb406c1f0 msgr2=0x7fabb406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fabb406c1f0 0x7fabb406e6b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fabdc142340 tx=0x7fabc4008040 comp rx=0 tx=0).stop 2024-01-15T18:10:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabdc0ccb50 msgr2=0x7fabdc0ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:18.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabdc0ccb50 0x7fabdc0ca9c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c820 tx=0x7fabcc00cce0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 shutdown_connections 2024-01-15T18:10:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fabb406c1f0 0x7fabb406e6b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fabdc15bf10 0x7fabdc0ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fabdc0ccb50 0x7fabdc0ca9c0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 --2- 172.21.15.26:0/1807373564 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fabdc0adf20 0x7fabdc0a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:18.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 >> 172.21.15.26:0/1807373564 conn(0x7fabdc06a7d0 msgr2=0x7fabdc15b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.985+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 shutdown_connections 2024-01-15T18:10:18.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:18.986+0000 7fabe2800640 1 -- 172.21.15.26:0/1807373564 wait complete. 2024-01-15T18:10:19.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:10:19.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.228+0000 7f581ea00640 1 Processor -- start 2024-01-15T18:10:19.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.228+0000 7f581ea00640 1 -- start start 2024-01-15T18:10:19.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.229+0000 7f581ea00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5818160f80 0x7f5818161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f581ea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58180ce8f0 0x7f58180ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f581ea00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58181618a0 con 0x7f58180cfbe0 2024-01-15T18:10:19.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f581ea00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58180d0de0 con 0x7f58180ce8f0 2024-01-15T18:10:19.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f581ea00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58180d10c0 con 0x7f5818160f80 2024-01-15T18:10:19.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f5817400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58180ce8f0 0x7f58180ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f5817400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f58180ce8f0 0x7f58180ced50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57098/0 (socket says 172.21.15.26:57098) 2024-01-15T18:10:19.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f5817400640 1 -- 172.21.15.26:0/2623475138 learned_addr learned my addr 172.21.15.26:0/2623475138 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:19.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f5817e00640 1 --2- 172.21.15.26:0/2623475138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5818160f80 0x7f5818161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f5817400640 1 -- 172.21.15.26:0/2623475138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5818160f80 msgr2=0x7f5818161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:19.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f5817400640 1 --2- 172.21.15.26:0/2623475138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5818160f80 0x7f5818161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.230+0000 7f5817400640 1 -- 172.21.15.26:0/2623475138 >> v1:172.21.15.124:6789/0 conn(0x7f58180cfbe0 legacy=0x7f58180d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.231+0000 7f5817400640 1 -- 172.21.15.26:0/2623475138 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58180d13a0 con 0x7f58180ce8f0 2024-01-15T18:10:19.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.232+0000 7f5817400640 1 --2- 172.21.15.26:0/2623475138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180ce8f0 0x7f58180ced50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f5800009870 tx=0x7f580002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6fe4f718839a6a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:19.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.232+0000 7f5816a00640 1 -- 172.21.15.26:0/2623475138 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800030840 con 0x7f58180ce8f0 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.232+0000 7f5816a00640 1 -- 172.21.15.26:0/2623475138 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58000309a0 con 0x7f58180ce8f0 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.232+0000 7f5816a00640 1 -- 172.21.15.26:0/2623475138 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800030d30 con 0x7f58180ce8f0 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.233+0000 7f581ea00640 1 -- 172.21.15.26:0/2623475138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180ce8f0 msgr2=0x7f58180ced50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.233+0000 7f581ea00640 1 --2- 172.21.15.26:0/2623475138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180ce8f0 0x7f58180ced50 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f5800009870 tx=0x7f580002b980 comp rx=0 tx=0).stop 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.233+0000 7f581ea00640 1 -- 172.21.15.26:0/2623475138 shutdown_connections 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.233+0000 7f581ea00640 1 --2- 172.21.15.26:0/2623475138 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180ce8f0 0x7f58180ced50 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.233+0000 7f581ea00640 1 --2- 172.21.15.26:0/2623475138 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5818160f80 0x7f5818161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.233+0000 7f581ea00640 1 -- 172.21.15.26:0/2623475138 >> 172.21.15.26:0/2623475138 conn(0x7f581809c550 msgr2=0x7f58180d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.233+0000 7f581ea00640 1 -- 172.21.15.26:0/2623475138 shutdown_connections 2024-01-15T18:10:19.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.233+0000 7f581ea00640 1 -- 172.21.15.26:0/2623475138 wait complete. 2024-01-15T18:10:19.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.235+0000 7f581ea00640 1 Processor -- start 2024-01-15T18:10:19.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.235+0000 7f581ea00640 1 -- start start 2024-01-15T18:10:19.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f581ea00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58180ce8f0 0x7f5818158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f581ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180cfbe0 0x7f581815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f581ea00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5818160f80 0x7f58180b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f581ea00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5818158cb0 con 0x7f58180cfbe0 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f581ea00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5818160850 con 0x7f5818160f80 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f581ea00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581815f7b0 con 0x7f58180ce8f0 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f5817400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180cfbe0 0x7f581815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f5817400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180cfbe0 0x7f581815ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57100/0 (socket says 172.21.15.26:57100) 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f5817400640 1 -- 172.21.15.26:0/3651282382 learned_addr learned my addr 172.21.15.26:0/3651282382 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.236+0000 7f5817e00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58180ce8f0 0x7f5818158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f581ca00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5818160f80 0x7f58180b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f5817400640 1 -- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58180ce8f0 msgr2=0x7f5818158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f5817400640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58180ce8f0 0x7f5818158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f5817400640 1 -- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5818160f80 msgr2=0x7f58180b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f5817400640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5818160f80 0x7f58180b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f5817400640 1 -- 172.21.15.26:0/3651282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58000094d0 con 0x7f58180cfbe0 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f5817400640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180cfbe0 0x7f581815ee40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5800000c00 tx=0x7f5800004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f580fe00640 1 -- 172.21.15.26:0/3651282382 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800039a90 con 0x7f58180cfbe0 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f580fe00640 1 -- 172.21.15.26:0/3651282382 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5800039bf0 con 0x7f58180cfbe0 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f580fe00640 1 -- 172.21.15.26:0/3651282382 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800043410 con 0x7f58180cfbe0 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.237+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581815fa90 con 0x7f58180cfbe0 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.238+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5818160050 con 0x7f58180cfbe0 2024-01-15T18:10:19.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.239+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581815c1b0 con 0x7f58180cfbe0 2024-01-15T18:10:19.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.240+0000 7f580fe00640 1 -- 172.21.15.26:0/3651282382 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f580002f390 con 0x7f58180cfbe0 2024-01-15T18:10:19.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.241+0000 7f580fe00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57f006c1f0 0x7f57f006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.241+0000 7f580fe00640 1 -- 172.21.15.26:0/3651282382 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f58000aa8e0 con 0x7f58180cfbe0 2024-01-15T18:10:19.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.241+0000 7f5817e00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57f006c1f0 0x7f57f006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.242+0000 7f5817e00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57f006c1f0 0x7f57f006e6b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5808006fd0 tx=0x7f5808008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:19.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.245+0000 7f580fe00640 1 -- 172.21.15.26:0/3651282382 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58000771a0 con 0x7f58180cfbe0 2024-01-15T18:10:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.445+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f58180675d0 con 0x7f58180cfbe0 2024-01-15T18:10:19.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.446+0000 7f580fe00640 1 -- 172.21.15.26:0/3651282382 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5800034060 con 0x7f58180cfbe0 2024-01-15T18:10:19.451 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:19.451 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:10:19.451 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:19.451 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-15T18:10:19.451 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-15T18:10:19.451 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:19.451 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:10:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57f006c1f0 msgr2=0x7f57f006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57f006c1f0 0x7f57f006e6b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5808006fd0 tx=0x7f5808008040 comp rx=0 tx=0).stop 2024-01-15T18:10:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180cfbe0 msgr2=0x7f581815ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180cfbe0 0x7f581815ee40 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f5800000c00 tx=0x7f5800004730 comp rx=0 tx=0).stop 2024-01-15T18:10:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 shutdown_connections 2024-01-15T18:10:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f57f006c1f0 0x7f57f006e6b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5818160f80 0x7f58180b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f58180cfbe0 0x7f581815ee40 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 --2- 172.21.15.26:0/3651282382 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f58180ce8f0 0x7f5818158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 >> 172.21.15.26:0/3651282382 conn(0x7f581809c550 msgr2=0x7f58180d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:19.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 shutdown_connections 2024-01-15T18:10:19.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.449+0000 7f581ea00640 1 -- 172.21.15.26:0/3651282382 wait complete. 2024-01-15T18:10:19.475 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:10:19.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:10:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.696+0000 7fe144a00640 1 Processor -- start 2024-01-15T18:10:19.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.696+0000 7fe144a00640 1 -- start start 2024-01-15T18:10:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe144a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe140071410 0x7fe140146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe144a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe14006c7c0 0x7fe1400717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe144a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14006cc20 con 0x7fe140071d30 2024-01-15T18:10:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe144a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14006cec0 con 0x7fe14006c7c0 2024-01-15T18:10:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe144a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14006d1a0 con 0x7fe140071410 2024-01-15T18:10:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe13ea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe14006c7c0 0x7fe1400717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe13ea00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe14006c7c0 0x7fe1400717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57108/0 (socket says 172.21.15.26:57108) 2024-01-15T18:10:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe13ea00640 1 -- 172.21.15.26:0/3184319638 learned_addr learned my addr 172.21.15.26:0/3184319638 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:19.703 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe13f400640 1 --2- 172.21.15.26:0/3184319638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe140071410 0x7fe140146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe13ea00640 1 -- 172.21.15.26:0/3184319638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe140071410 msgr2=0x7fe140146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe13ea00640 1 --2- 172.21.15.26:0/3184319638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe140071410 0x7fe140146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.698+0000 7fe13ea00640 1 -- 172.21.15.26:0/3184319638 >> v1:172.21.15.124:6789/0 conn(0x7fe140071d30 legacy=0x7fe1400b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.699+0000 7fe13ea00640 1 -- 172.21.15.26:0/3184319638 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14006d480 con 0x7fe14006c7c0 2024-01-15T18:10:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.699+0000 7fe13f400640 1 --2- 172.21.15.26:0/3184319638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe140071410 0x7fe140146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:19.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.700+0000 7fe13ea00640 1 --2- 172.21.15.26:0/3184319638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14006c7c0 0x7fe1400717f0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe128009870 tx=0x7fe12802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d85b55954880bd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.700+0000 7fe13e000640 1 -- 172.21.15.26:0/3184319638 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128031640 con 0x7fe14006c7c0 2024-01-15T18:10:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.700+0000 7fe13e000640 1 -- 172.21.15.26:0/3184319638 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1280317a0 con 0x7fe14006c7c0 2024-01-15T18:10:19.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.700+0000 7fe13e000640 1 -- 172.21.15.26:0/3184319638 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe128031b30 con 0x7fe14006c7c0 2024-01-15T18:10:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.701+0000 7fe144a00640 1 -- 172.21.15.26:0/3184319638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14006c7c0 msgr2=0x7fe1400717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.701+0000 7fe144a00640 1 --2- 172.21.15.26:0/3184319638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14006c7c0 0x7fe1400717f0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe128009870 tx=0x7fe12802b8f0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.701+0000 7fe144a00640 1 -- 172.21.15.26:0/3184319638 shutdown_connections 2024-01-15T18:10:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.701+0000 7fe144a00640 1 --2- 172.21.15.26:0/3184319638 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe14006c7c0 0x7fe1400717f0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.701+0000 7fe144a00640 1 --2- 172.21.15.26:0/3184319638 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe140071410 0x7fe140146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.701+0000 7fe144a00640 1 -- 172.21.15.26:0/3184319638 >> 172.21.15.26:0/3184319638 conn(0x7fe14006a800 msgr2=0x7fe14006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.701+0000 7fe144a00640 1 -- 172.21.15.26:0/3184319638 shutdown_connections 2024-01-15T18:10:19.706 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.701+0000 7fe144a00640 1 -- 172.21.15.26:0/3184319638 wait complete. 2024-01-15T18:10:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.704+0000 7fe144a00640 1 Processor -- start 2024-01-15T18:10:19.708 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.704+0000 7fe144a00640 1 -- start start 2024-01-15T18:10:19.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.705+0000 7fe144a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe14006c7c0 0x7fe14015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.705+0000 7fe144a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe140071410 0x7fe14015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.705+0000 7fe144a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe140071d30 0x7fe1400b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.705+0000 7fe144a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14015eb80 con 0x7fe140071d30 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.705+0000 7fe144a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14015ee20 con 0x7fe140071410 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.705+0000 7fe144a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14015f100 con 0x7fe14006c7c0 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe140071d30 0x7fe1400b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe140071d30 0x7fe1400b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57116/0 (socket says 172.21.15.26:57116) 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13fe00640 1 -- 172.21.15.26:0/1093777083 learned_addr learned my addr 172.21.15.26:0/1093777083 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13f400640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe14006c7c0 0x7fe14015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13ea00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe140071410 0x7fe14015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13fe00640 1 -- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe14006c7c0 msgr2=0x7fe14015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13fe00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe14006c7c0 0x7fe14015c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13fe00640 1 -- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe140071410 msgr2=0x7fe14015fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:19.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13fe00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe140071410 0x7fe14015fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:19.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13fe00640 1 -- 172.21.15.26:0/1093777083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280094d0 con 0x7fe140071d30 2024-01-15T18:10:19.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.706+0000 7fe13fe00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe140071d30 0x7fe1400b3780 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b970 tx=0x7fe13800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:19.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.707+0000 7fe137400640 1 -- 172.21.15.26:0/1093777083 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13800cb40 con 0x7fe140071d30 2024-01-15T18:10:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.707+0000 7fe137400640 1 -- 172.21.15.26:0/1093777083 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13800cca0 con 0x7fe140071d30 2024-01-15T18:10:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.707+0000 7fe137400640 1 -- 172.21.15.26:0/1093777083 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138014680 con 0x7fe140071d30 2024-01-15T18:10:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.707+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14015f3e0 con 0x7fe140071d30 2024-01-15T18:10:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.707+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14015f9a0 con 0x7fe140071d30 2024-01-15T18:10:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.708+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14015fb90 con 0x7fe140071d30 2024-01-15T18:10:19.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.709+0000 7fe137400640 1 -- 172.21.15.26:0/1093777083 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fe13801d3b0 con 0x7fe140071d30 2024-01-15T18:10:19.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.710+0000 7fe137400640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe11806c1f0 0x7fe11806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:19.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.710+0000 7fe137400640 1 -- 172.21.15.26:0/1093777083 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fe138088610 con 0x7fe140071d30 2024-01-15T18:10:19.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.711+0000 7fe13f400640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe11806c1f0 0x7fe11806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:19.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.712+0000 7fe13f400640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe11806c1f0 0x7fe11806e6b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe13000a990 tx=0x7fe130008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:19.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.714+0000 7fe137400640 1 -- 172.21.15.26:0/1093777083 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe138054dd0 con 0x7fe140071d30 2024-01-15T18:10:19.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:19.910+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe140067450 con 0x7fe140071d30 2024-01-15T18:10:20.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.570+0000 7fe137400640 1 -- 172.21.15.26:0/1093777083 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe1380583f0 con 0x7fe140071d30 2024-01-15T18:10:20.575 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-15T18:10:20.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe11806c1f0 msgr2=0x7fe11806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe11806c1f0 0x7fe11806e6b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe13000a990 tx=0x7fe130008040 comp rx=0 tx=0).stop 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe140071d30 msgr2=0x7fe1400b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe140071d30 0x7fe1400b3780 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b970 tx=0x7fe13800be30 comp rx=0 tx=0).stop 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 shutdown_connections 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe11806c1f0 0x7fe11806e6b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe140071d30 0x7fe1400b3780 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe140071410 0x7fe14015fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 --2- 172.21.15.26:0/1093777083 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe14006c7c0 0x7fe14015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 >> 172.21.15.26:0/1093777083 conn(0x7fe14006a800 msgr2=0x7fe140144230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 shutdown_connections 2024-01-15T18:10:20.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.574+0000 7fe144a00640 1 -- 172.21.15.26:0/1093777083 wait complete. 2024-01-15T18:10:20.605 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:10:20.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.814+0000 7f7b37c00640 1 Processor -- start 2024-01-15T18:10:20.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.814+0000 7f7b37c00640 1 -- start start 2024-01-15T18:10:20.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b37c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b3015ed20 0x7f7b30141360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:20.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b37c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 0x7f7b300cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:20.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b37c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3013e440 con 0x7f7b300cdcc0 2024-01-15T18:10:20.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b37c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b300ce1d0 con 0x7f7b3015ed20 2024-01-15T18:10:20.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b37c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300ce4b0 con 0x7f7b3015f100 2024-01-15T18:10:20.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b35200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b3015ed20 0x7f7b30141360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:20.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b35c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7b300cdcc0 0x7f7b3015b730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60970/0 (socket says 172.21.15.26:60970) 2024-01-15T18:10:20.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b35200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b3015ed20 0x7f7b30141360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57122/0 (socket says 172.21.15.26:57122) 2024-01-15T18:10:20.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b35c00640 1 -- 172.21.15.26:0/4043649661 learned_addr learned my addr 172.21.15.26:0/4043649661 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:20.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b2fe00640 1 --2- 172.21.15.26:0/4043649661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 0x7f7b300cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:20.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b2f400640 1 -- 172.21.15.26:0/4043649661 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633898768 0 0) 0x7f7b3013e440 con 0x7f7b300cdcc0 2024-01-15T18:10:20.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b35200640 1 -- 172.21.15.26:0/4043649661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 msgr2=0x7f7b300cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:20.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b35200640 1 --2- 172.21.15.26:0/4043649661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 0x7f7b300cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:20.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b35200640 1 -- 172.21.15.26:0/4043649661 >> v1:172.21.15.26:6789/0 conn(0x7f7b300cdcc0 legacy=0x7f7b3015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:20.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.816+0000 7f7b35200640 1 -- 172.21.15.26:0/4043649661 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300ce790 con 0x7f7b3015ed20 2024-01-15T18:10:20.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.817+0000 7f7b35200640 1 --2- 172.21.15.26:0/4043649661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b3015ed20 0x7f7b30141360 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7b200097a0 tx=0x7f7b2002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f0aa0b5297d3b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:20.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.818+0000 7f7b2f400640 1 -- 172.21.15.26:0/4043649661 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2002e070 con 0x7f7b3015ed20 2024-01-15T18:10:20.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.818+0000 7f7b2f400640 1 -- 172.21.15.26:0/4043649661 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b200315a0 con 0x7f7b3015ed20 2024-01-15T18:10:20.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.818+0000 7f7b2f400640 1 -- 172.21.15.26:0/4043649661 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20031930 con 0x7f7b3015ed20 2024-01-15T18:10:20.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.818+0000 7f7b37c00640 1 -- 172.21.15.26:0/4043649661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b3015ed20 msgr2=0x7f7b30141360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:20.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.820+0000 7f7b37c00640 1 --2- 172.21.15.26:0/4043649661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b3015ed20 0x7f7b30141360 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f7b200097a0 tx=0x7f7b2002b840 comp rx=0 tx=0).stop 2024-01-15T18:10:20.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.820+0000 7f7b37c00640 1 -- 172.21.15.26:0/4043649661 shutdown_connections 2024-01-15T18:10:20.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.820+0000 7f7b37c00640 1 --2- 172.21.15.26:0/4043649661 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 0x7f7b300cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:20.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.820+0000 7f7b37c00640 1 --2- 172.21.15.26:0/4043649661 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b3015ed20 0x7f7b30141360 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:20.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.820+0000 7f7b37c00640 1 -- 172.21.15.26:0/4043649661 >> 172.21.15.26:0/4043649661 conn(0x7f7b300b0be0 msgr2=0x7f7b300b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:20.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.820+0000 7f7b37c00640 1 -- 172.21.15.26:0/4043649661 shutdown_connections 2024-01-15T18:10:20.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.820+0000 7f7b37c00640 1 -- 172.21.15.26:0/4043649661 wait complete. 2024-01-15T18:10:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.821+0000 7f7b37c00640 1 Processor -- start 2024-01-15T18:10:20.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.821+0000 7f7b37c00640 1 -- start start 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b37c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300cdcc0 0x7f7b3015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b37c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b3015ed20 0x7f7b300acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b37c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 0x7f7b300ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b37c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3015b220 con 0x7f7b300cdcc0 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b37c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300a9f70 con 0x7f7b3015ed20 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b37c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300a9c90 con 0x7f7b3015f100 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300cdcc0 0x7f7b3015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300cdcc0 0x7f7b3015ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57124/0 (socket says 172.21.15.26:57124) 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35200640 1 -- 172.21.15.26:0/4029705871 learned_addr learned my addr 172.21.15.26:0/4029705871 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35c00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 0x7f7b300ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:20.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b2fe00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b3015ed20 0x7f7b300acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35200640 1 -- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 msgr2=0x7f7b300ad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35200640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 0x7f7b300ad300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35200640 1 -- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b3015ed20 msgr2=0x7f7b300acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35200640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b3015ed20 0x7f7b300acdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35200640 1 -- 172.21.15.26:0/4029705871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20009400 con 0x7f7b300cdcc0 2024-01-15T18:10:20.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b2fe00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b3015ed20 0x7f7b300acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:20.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b35200640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300cdcc0 0x7f7b3015ace0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7b20034040 tx=0x7f7b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:20.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b2cc00640 1 -- 172.21.15.26:0/4029705871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20032840 con 0x7f7b300cdcc0 2024-01-15T18:10:20.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b2cc00640 1 -- 172.21.15.26:0/4029705871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b200329a0 con 0x7f7b300cdcc0 2024-01-15T18:10:20.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b2cc00640 1 -- 172.21.15.26:0/4029705871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20032d50 con 0x7f7b300cdcc0 2024-01-15T18:10:20.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3015f6c0 con 0x7f7b300cdcc0 2024-01-15T18:10:20.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.823+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3012d540 con 0x7f7b300cdcc0 2024-01-15T18:10:20.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.825+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b30067670 con 0x7f7b300cdcc0 2024-01-15T18:10:20.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.826+0000 7f7b2cc00640 1 -- 172.21.15.26:0/4029705871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f7b2004e430 con 0x7f7b300cdcc0 2024-01-15T18:10:20.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.827+0000 7f7b2cc00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b0806c1f0 0x7f7b0806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:20.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.827+0000 7f7b2cc00640 1 -- 172.21.15.26:0/4029705871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f7b200aabe0 con 0x7f7b300cdcc0 2024-01-15T18:10:20.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.828+0000 7f7b2fe00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b0806c1f0 0x7f7b0806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:20.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.830+0000 7f7b2fe00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b0806c1f0 0x7f7b0806e6b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7b18006fd0 tx=0x7f7b18008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:20.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:20.832+0000 7f7b2cc00640 1 -- 172.21.15.26:0/4029705871 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b200772d0 con 0x7f7b300cdcc0 2024-01-15T18:10:21.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.027+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7b300cdaa0 con 0x7f7b300cdcc0 2024-01-15T18:10:21.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.585+0000 7f7b2cc00640 1 -- 172.21.15.26:0/4029705871 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7b2007a8f0 con 0x7f7b300cdcc0 2024-01-15T18:10:21.589 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-15T18:10:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b0806c1f0 msgr2=0x7f7b0806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b0806c1f0 0x7f7b0806e6b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7b18006fd0 tx=0x7f7b18008040 comp rx=0 tx=0).stop 2024-01-15T18:10:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300cdcc0 msgr2=0x7f7b3015ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300cdcc0 0x7f7b3015ace0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f7b20034040 tx=0x7f7b200046c0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 shutdown_connections 2024-01-15T18:10:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b3015f100 0x7f7b300ad300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b0806c1f0 0x7f7b0806e6b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b3015ed20 0x7f7b300acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 --2- 172.21.15.26:0/4029705871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b300cdcc0 0x7f7b3015ace0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 >> 172.21.15.26:0/4029705871 conn(0x7f7b300b0be0 msgr2=0x7f7b30140670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 shutdown_connections 2024-01-15T18:10:21.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.589+0000 7f7b37c00640 1 -- 172.21.15.26:0/4029705871 wait complete. 2024-01-15T18:10:21.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:10:21.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.812+0000 7f28f6000640 1 Processor -- start 2024-01-15T18:10:21.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.813+0000 7f28f6000640 1 -- start start 2024-01-15T18:10:21.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.814+0000 7f28f6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013df80 0x7f28f015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.814+0000 7f28f6000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f28f013e360 0x7f28f00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.814+0000 7f28f6000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28f015f9c0 0x7f28f00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.814+0000 7f28f6000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f006c470 con 0x7f28f015f9c0 2024-01-15T18:10:21.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.814+0000 7f28f6000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f006c750 con 0x7f28f013e360 2024-01-15T18:10:21.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.814+0000 7f28f6000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f006ca30 con 0x7f28f013df80 2024-01-15T18:10:21.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28efe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28f015f9c0 0x7f28f00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28efe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f28f015f9c0 0x7f28f00cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57138/0 (socket says 172.21.15.26:57138) 2024-01-15T18:10:21.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28efe00640 1 -- 172.21.15.26:0/146674489 learned_addr learned my addr 172.21.15.26:0/146674489 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:21.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28eea00640 1 --2- 172.21.15.26:0/146674489 >> v2:172.21.15.124:3300/0 conn(0x7f28f013e360 0x7f28f00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28ef400640 1 --2- 172.21.15.26:0/146674489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013df80 0x7f28f015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28efe00640 1 -- 172.21.15.26:0/146674489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013df80 msgr2=0x7f28f015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:21.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28efe00640 1 --2- 172.21.15.26:0/146674489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013df80 0x7f28f015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28efe00640 1 -- 172.21.15.26:0/146674489 >> v2:172.21.15.124:3300/0 conn(0x7f28f013e360 msgr2=0x7f28f00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:21.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28efe00640 1 --2- 172.21.15.26:0/146674489 >> v2:172.21.15.124:3300/0 conn(0x7f28f013e360 0x7f28f00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.815+0000 7f28efe00640 1 -- 172.21.15.26:0/146674489 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f006cd10 con 0x7f28f015f9c0 2024-01-15T18:10:21.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.816+0000 7f28efe00640 1 --2- 172.21.15.26:0/146674489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015f9c0 0x7f28f00cf7d0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f28f00cfed0 tx=0x7f28e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ba6932f7b8cdd0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:21.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28ee000640 1 -- 172.21.15.26:0/146674489 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e802ca70 con 0x7f28f015f9c0 2024-01-15T18:10:21.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28ee000640 1 -- 172.21.15.26:0/146674489 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e802cbd0 con 0x7f28f015f9c0 2024-01-15T18:10:21.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28ee000640 1 -- 172.21.15.26:0/146674489 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80343f0 con 0x7f28f015f9c0 2024-01-15T18:10:21.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28f6000640 1 -- 172.21.15.26:0/146674489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015f9c0 msgr2=0x7f28f00cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28f6000640 1 --2- 172.21.15.26:0/146674489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015f9c0 0x7f28f00cf7d0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f28f00cfed0 tx=0x7f28e802bed0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28f6000640 1 -- 172.21.15.26:0/146674489 shutdown_connections 2024-01-15T18:10:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28f6000640 1 --2- 172.21.15.26:0/146674489 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f015f9c0 0x7f28f00cf7d0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28f6000640 1 --2- 172.21.15.26:0/146674489 >> v2:172.21.15.124:3300/0 conn(0x7f28f013e360 0x7f28f00cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28f6000640 1 --2- 172.21.15.26:0/146674489 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013df80 0x7f28f015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28f6000640 1 -- 172.21.15.26:0/146674489 >> 172.21.15.26:0/146674489 conn(0x7f28f006a800 msgr2=0x7f28f006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:21.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.817+0000 7f28f6000640 1 -- 172.21.15.26:0/146674489 shutdown_connections 2024-01-15T18:10:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.818+0000 7f28f6000640 1 -- 172.21.15.26:0/146674489 wait complete. 2024-01-15T18:10:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.819+0000 7f28f6000640 1 Processor -- start 2024-01-15T18:10:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.819+0000 7f28f6000640 1 -- start start 2024-01-15T18:10:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.820+0000 7f28f6000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f013df80 0x7f28f0158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.820+0000 7f28f6000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013e360 0x7f28f0154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.820+0000 7f28f6000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f015f9c0 0x7f28f01567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.820+0000 7f28f6000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f0157640 con 0x7f28f013df80 2024-01-15T18:10:21.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.820+0000 7f28f6000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00b3200 con 0x7f28f015f9c0 2024-01-15T18:10:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.820+0000 7f28f6000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f013bac0 con 0x7f28f013e360 2024-01-15T18:10:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28ef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f013df80 0x7f28f0158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28ef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f013df80 0x7f28f0158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57154/0 (socket says 172.21.15.26:57154) 2024-01-15T18:10:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28ef400640 1 -- 172.21.15.26:0/15947672 learned_addr learned my addr 172.21.15.26:0/15947672 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28ef400640 1 -- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013e360 msgr2=0x7f28f0154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28eea00640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013e360 0x7f28f0154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28ef400640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013e360 0x7f28f0154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28ef400640 1 -- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f015f9c0 msgr2=0x7f28f01567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28efe00640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f015f9c0 0x7f28f01567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28ef400640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f015f9c0 0x7f28f01567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28ef400640 1 -- 172.21.15.26:0/15947672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e8009b30 con 0x7f28f013df80 2024-01-15T18:10:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28efe00640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f015f9c0 0x7f28f01567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.821+0000 7f28ef400640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f013df80 0x7f28f0158a10 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f28e000d620 tx=0x7f28e000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.822+0000 7f28e7400640 1 -- 172.21.15.26:0/15947672 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e000b620 con 0x7f28f013df80 2024-01-15T18:10:21.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.822+0000 7f28e7400640 1 -- 172.21.15.26:0/15947672 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e000b780 con 0x7f28f013df80 2024-01-15T18:10:21.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.822+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f0158f50 con 0x7f28f013df80 2024-01-15T18:10:21.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.822+0000 7f28e7400640 1 -- 172.21.15.26:0/15947672 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e000bb30 con 0x7f28f013df80 2024-01-15T18:10:21.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.822+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f00b28d0 con 0x7f28f013df80 2024-01-15T18:10:21.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.823+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f00675d0 con 0x7f28f013df80 2024-01-15T18:10:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.824+0000 7f28e7400640 1 -- 172.21.15.26:0/15947672 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f28e000bc90 con 0x7f28f013df80 2024-01-15T18:10:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.825+0000 7f28e7400640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.825+0000 7f28e7400640 1 -- 172.21.15.26:0/15947672 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5665+0+0 (secure 0 0 0) 0x7f28e0088ea0 con 0x7f28f013df80 2024-01-15T18:10:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.826+0000 7f28eea00640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.827+0000 7f28eea00640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f28f00cc260 tx=0x7f28d8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:21.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:21.830+0000 7f28e7400640 1 -- 172.21.15.26:0/15947672 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28e0090070 con 0x7f28f013df80 2024-01-15T18:10:22.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.069+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f28f006df00 con 0x7f28f013df80 2024-01-15T18:10:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.589+0000 7f28e7400640 1 -- 172.21.15.26:0/15947672 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v59) v1 ==== 174+0+0 (secure 0 0 0) 0x7f28e0055470 con 0x7f28f013df80 2024-01-15T18:10:22.593 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 14 and data pool 15 2024-01-15T18:10:22.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.593+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 msgr2=0x7f28c806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.593+0000 7f28f6000640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f28f00cc260 tx=0x7f28d8008040 comp rx=0 tx=0).stop 2024-01-15T18:10:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.593+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f013df80 msgr2=0x7f28f0158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:22.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.593+0000 7f28f6000640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f013df80 0x7f28f0158a10 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f28e000d620 tx=0x7f28e000dae0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.595+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 shutdown_connections 2024-01-15T18:10:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.595+0000 7f28f6000640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f28f013e360 0x7f28f0154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.595+0000 7f28f6000640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28c806c1f0 0x7f28c806e6b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.595+0000 7f28f6000640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f28f015f9c0 0x7f28f01567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.595+0000 7f28f6000640 1 --2- 172.21.15.26:0/15947672 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f28f013df80 0x7f28f0158a10 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.595+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 >> 172.21.15.26:0/15947672 conn(0x7f28f006a800 msgr2=0x7f28f00b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:22.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.600+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 shutdown_connections 2024-01-15T18:10:22.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.600+0000 7f28f6000640 1 -- 172.21.15.26:0/15947672 wait complete. 2024-01-15T18:10:22.645 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:10:22.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.823+0000 7fe405000640 1 Processor -- start 2024-01-15T18:10:22.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.824+0000 7fe405000640 1 -- start start 2024-01-15T18:10:22.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.825+0000 7fe405000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe40009a170 0x7fe4000b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:22.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.825+0000 7fe405000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe40006c6b0 0x7fe4000cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:22.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.825+0000 7fe405000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40006cea0 con 0x7fe4000cd240 2024-01-15T18:10:22.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.825+0000 7fe405000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40006cbc0 con 0x7fe40009a170 2024-01-15T18:10:22.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.825+0000 7fe405000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40006f350 con 0x7fe40006c6b0 2024-01-15T18:10:22.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.825+0000 7fe3fd400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe40009a170 0x7fe4000b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:22.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.825+0000 7fe3fd400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe40009a170 0x7fe4000b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48492/0 (socket says 172.21.15.26:48492) 2024-01-15T18:10:22.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.825+0000 7fe3fd400640 1 -- 172.21.15.26:0/2858133379 learned_addr learned my addr 172.21.15.26:0/2858133379 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:22.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.826+0000 7fe3fe800640 1 --2- 172.21.15.26:0/2858133379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe40006c6b0 0x7fe4000cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:22.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.826+0000 7fe3fd400640 1 -- 172.21.15.26:0/2858133379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe40006c6b0 msgr2=0x7fe4000cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:22.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.826+0000 7fe3fd400640 1 --2- 172.21.15.26:0/2858133379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe40006c6b0 0x7fe4000cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.826+0000 7fe3fd400640 1 -- 172.21.15.26:0/2858133379 >> v1:172.21.15.124:6789/0 conn(0x7fe4000cd240 legacy=0x7fe400099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:22.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.826+0000 7fe3fd400640 1 -- 172.21.15.26:0/2858133379 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40006f630 con 0x7fe40009a170 2024-01-15T18:10:22.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.827+0000 7fe3fd400640 1 --2- 172.21.15.26:0/2858133379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 0x7fe4000b3820 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009e50 tx=0x7fe3f002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731c1a8da7b7d3b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:22.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.827+0000 7fe3fca00640 1 -- 172.21.15.26:0/2858133379 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f002cb00 con 0x7fe40009a170 2024-01-15T18:10:22.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.827+0000 7fe3fca00640 1 -- 172.21.15.26:0/2858133379 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f002cc60 con 0x7fe40009a170 2024-01-15T18:10:22.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.827+0000 7fe3fca00640 1 -- 172.21.15.26:0/2858133379 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00343f0 con 0x7fe40009a170 2024-01-15T18:10:22.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.828+0000 7fe405000640 1 -- 172.21.15.26:0/2858133379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 msgr2=0x7fe4000b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:22.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.828+0000 7fe405000640 1 --2- 172.21.15.26:0/2858133379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 0x7fe4000b3820 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009e50 tx=0x7fe3f002bf60 comp rx=0 tx=0).stop 2024-01-15T18:10:22.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.828+0000 7fe405000640 1 -- 172.21.15.26:0/2858133379 shutdown_connections 2024-01-15T18:10:22.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.828+0000 7fe405000640 1 --2- 172.21.15.26:0/2858133379 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe40006c6b0 0x7fe4000cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.828+0000 7fe405000640 1 --2- 172.21.15.26:0/2858133379 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 0x7fe4000b3820 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.828+0000 7fe405000640 1 -- 172.21.15.26:0/2858133379 >> 172.21.15.26:0/2858133379 conn(0x7fe4000b2d00 msgr2=0x7fe4000b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:22.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.828+0000 7fe405000640 1 -- 172.21.15.26:0/2858133379 shutdown_connections 2024-01-15T18:10:22.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.828+0000 7fe405000640 1 -- 172.21.15.26:0/2858133379 wait complete. 2024-01-15T18:10:22.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.830+0000 7fe405000640 1 Processor -- start 2024-01-15T18:10:22.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.830+0000 7fe405000640 1 -- start start 2024-01-15T18:10:22.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.831+0000 7fe405000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe40006c6b0 0x7fe4000c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:22.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.831+0000 7fe405000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 0x7fe4000b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:22.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.831+0000 7fe405000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4000cd240 0x7fe4001f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:22.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.831+0000 7fe405000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b4fb0 con 0x7fe40009a170 2024-01-15T18:10:22.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.831+0000 7fe405000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001f2b80 con 0x7fe40006c6b0 2024-01-15T18:10:22.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.831+0000 7fe405000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000b51e0 con 0x7fe4000cd240 2024-01-15T18:10:22.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.831+0000 7fe3fd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 0x7fe4000b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.831+0000 7fe3fd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 0x7fe4000b4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48498/0 (socket says 172.21.15.26:48498) 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.831+0000 7fe3fd400640 1 -- 172.21.15.26:0/3588083191 learned_addr learned my addr 172.21.15.26:0/3588083191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3fd400640 1 -- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4000cd240 msgr2=0x7fe4001f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3fe800640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4000cd240 0x7fe4001f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3fde00640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe40006c6b0 0x7fe4000c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3fd400640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4000cd240 0x7fe4001f2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3fd400640 1 -- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe40006c6b0 msgr2=0x7fe4000c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3fd400640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe40006c6b0 0x7fe4000c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3fd400640 1 -- 172.21.15.26:0/3588083191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0009b30 con 0x7fe40009a170 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3fde00640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe40006c6b0 0x7fe4000c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:22.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3fd400640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 0x7fe4000b4a70 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000f80 tx=0x7fe3f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:22.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3f6000640 1 -- 172.21.15.26:0/3588083191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0009510 con 0x7fe40009a170 2024-01-15T18:10:22.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3f6000640 1 -- 172.21.15.26:0/3588083191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f003e450 con 0x7fe40009a170 2024-01-15T18:10:22.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe3f6000640 1 -- 172.21.15.26:0/3588083191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f003e800 con 0x7fe40009a170 2024-01-15T18:10:22.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.832+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4000b54c0 con 0x7fe40009a170 2024-01-15T18:10:22.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.833+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4000b5a80 con 0x7fe40009a170 2024-01-15T18:10:22.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.834+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4000675d0 con 0x7fe40009a170 2024-01-15T18:10:22.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.835+0000 7fe3f6000640 1 -- 172.21.15.26:0/3588083191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fe3f003e960 con 0x7fe40009a170 2024-01-15T18:10:22.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.835+0000 7fe3f6000640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe3d806c1f0 0x7fe3d806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:22.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.836+0000 7fe3f6000640 1 -- 172.21.15.26:0/3588083191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe3f00ab620 con 0x7fe40009a170 2024-01-15T18:10:22.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.836+0000 7fe3fde00640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe3d806c1f0 0x7fe3d806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:22.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.837+0000 7fe3fde00640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe3d806c1f0 0x7fe3d806e6b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8000f60 tx=0x7fe3e8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:22.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:22.840+0000 7fe3f6000640 1 -- 172.21.15.26:0/3588083191 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3f0077d50 con 0x7fe40009a170 2024-01-15T18:10:23.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.036+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4000ce860 con 0x7fe40009a170 2024-01-15T18:10:23.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.038+0000 7fe3f6000640 1 -- 172.21.15.26:0/3588083191 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16595 (secure 0 0 0) 0x7fe3f007b370 con 0x7fe40009a170 2024-01-15T18:10:23.042 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:23.042 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":101,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:10:22.071155+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:10:19.966552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-15T18:10:21.081716+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe3d806c1f0 msgr2=0x7fe3d806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe3d806c1f0 0x7fe3d806e6b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8000f60 tx=0x7fe3e8009040 comp rx=0 tx=0).stop 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 msgr2=0x7fe4000b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 0x7fe4000b4a70 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0000f80 tx=0x7fe3f0004a40 comp rx=0 tx=0).stop 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 shutdown_connections 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe4000cd240 0x7fe4001f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe3d806c1f0 0x7fe3d806e6b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe40009a170 0x7fe4000b4a70 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 --2- 172.21.15.26:0/3588083191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe40006c6b0 0x7fe4000c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 >> 172.21.15.26:0/3588083191 conn(0x7fe4000b2d00 msgr2=0x7fe400099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:23.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 shutdown_connections 2024-01-15T18:10:23.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.041+0000 7fe405000640 1 -- 172.21.15.26:0/3588083191 wait complete. 2024-01-15T18:10:23.064 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:10:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.284+0000 7f565b200640 1 Processor -- start 2024-01-15T18:10:23.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.285+0000 7f565b200640 1 -- start start 2024-01-15T18:10:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.286+0000 7f565b200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f565c160920 0x7f565c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.286+0000 7f565b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f565c0cfea0 0x7f565c161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.286+0000 7f565b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565c0d0a10 con 0x7f565c0cf9f0 2024-01-15T18:10:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.286+0000 7f565b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565c0d1f50 con 0x7f565c0cfea0 2024-01-15T18:10:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.286+0000 7f565b200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f565c0cf080 con 0x7f565c160920 2024-01-15T18:10:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.287+0000 7f565a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f565c0cfea0 0x7f565c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.287+0000 7f565a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f565c0cfea0 0x7f565c161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48500/0 (socket says 172.21.15.26:48500) 2024-01-15T18:10:23.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.287+0000 7f565a800640 1 -- 172.21.15.26:0/1194244873 learned_addr learned my addr 172.21.15.26:0/1194244873 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.287+0000 7f5659e00640 1 --2- 172.21.15.26:0/1194244873 >> v2:172.21.15.124:3300/0 conn(0x7f565c160920 0x7f565c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.287+0000 7f565a800640 1 -- 172.21.15.26:0/1194244873 >> v2:172.21.15.124:3300/0 conn(0x7f565c160920 msgr2=0x7f565c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.287+0000 7f565a800640 1 --2- 172.21.15.26:0/1194244873 >> v2:172.21.15.124:3300/0 conn(0x7f565c160920 0x7f565c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.287+0000 7f565a800640 1 -- 172.21.15.26:0/1194244873 >> v1:172.21.15.124:6789/0 conn(0x7f565c0cf9f0 legacy=0x7f565c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:23.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.287+0000 7f565a800640 1 -- 172.21.15.26:0/1194244873 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565c0d0310 con 0x7f565c0cfea0 2024-01-15T18:10:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.288+0000 7f565a800640 1 --2- 172.21.15.26:0/1194244873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c0cfea0 0x7f565c161720 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f565c08c060 tx=0x7f565402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18c40f897ccb4a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.289+0000 7f5658a00640 1 -- 172.21.15.26:0/1194244873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654031870 con 0x7f565c0cfea0 2024-01-15T18:10:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.289+0000 7f5658a00640 1 -- 172.21.15.26:0/1194244873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56540319d0 con 0x7f565c0cfea0 2024-01-15T18:10:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.289+0000 7f5658a00640 1 -- 172.21.15.26:0/1194244873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654031d60 con 0x7f565c0cfea0 2024-01-15T18:10:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.289+0000 7f565b200640 1 -- 172.21.15.26:0/1194244873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c0cfea0 msgr2=0x7f565c161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.289+0000 7f565b200640 1 --2- 172.21.15.26:0/1194244873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c0cfea0 0x7f565c161720 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f565c08c060 tx=0x7f565402b910 comp rx=0 tx=0).stop 2024-01-15T18:10:23.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.289+0000 7f565b200640 1 -- 172.21.15.26:0/1194244873 shutdown_connections 2024-01-15T18:10:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.289+0000 7f565b200640 1 --2- 172.21.15.26:0/1194244873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c0cfea0 0x7f565c161720 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.289+0000 7f565b200640 1 --2- 172.21.15.26:0/1194244873 >> v2:172.21.15.124:3300/0 conn(0x7f565c160920 0x7f565c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.289+0000 7f565b200640 1 -- 172.21.15.26:0/1194244873 >> 172.21.15.26:0/1194244873 conn(0x7f565c0d3090 msgr2=0x7f565c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.290+0000 7f565b200640 1 -- 172.21.15.26:0/1194244873 shutdown_connections 2024-01-15T18:10:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.290+0000 7f565b200640 1 -- 172.21.15.26:0/1194244873 wait complete. 2024-01-15T18:10:23.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.291+0000 7f565b200640 1 Processor -- start 2024-01-15T18:10:23.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.292+0000 7f565b200640 1 -- start start 2024-01-15T18:10:23.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.292+0000 7f565b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565c0cf9f0 0x7f565c0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.292+0000 7f565b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f565c0cfea0 0x7f565c1f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.292+0000 7f565b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c160920 0x7f565c0ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.292+0000 7f565b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0cd530 con 0x7f565c160920 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.292+0000 7f565b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0b4040 con 0x7f565c0cf9f0 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.292+0000 7f565b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565c0b4320 con 0x7f565c0cfea0 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f565a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c160920 0x7f565c0ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f565a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c160920 0x7f565c0ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48512/0 (socket says 172.21.15.26:48512) 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f565a800640 1 -- 172.21.15.26:0/3814179148 learned_addr learned my addr 172.21.15.26:0/3814179148 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f5659400640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f565c0cfea0 0x7f565c1f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f5659e00640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565c0cf9f0 0x7f565c0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f565a800640 1 -- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f565c0cfea0 msgr2=0x7f565c1f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f565a800640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f565c0cfea0 0x7f565c1f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f565a800640 1 -- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565c0cf9f0 msgr2=0x7f565c0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.297 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f565a800640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565c0cf9f0 0x7f565c0b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.293+0000 7f565a800640 1 -- 172.21.15.26:0/3814179148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56540094d0 con 0x7f565c160920 2024-01-15T18:10:23.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.294+0000 7f565a800640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c160920 0x7f565c0ca7e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f56540090b0 tx=0x7f56540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:23.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.294+0000 7f5652000640 1 -- 172.21.15.26:0/3814179148 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565403ac60 con 0x7f565c160920 2024-01-15T18:10:23.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.294+0000 7f5652000640 1 -- 172.21.15.26:0/3814179148 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565403adc0 con 0x7f565c160920 2024-01-15T18:10:23.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.294+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f565c0b4600 con 0x7f565c160920 2024-01-15T18:10:23.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.294+0000 7f5652000640 1 -- 172.21.15.26:0/3814179148 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654044630 con 0x7f565c160920 2024-01-15T18:10:23.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.294+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f565c0b3810 con 0x7f565c160920 2024-01-15T18:10:23.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.295+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f565c0b4b20 con 0x7f565c160920 2024-01-15T18:10:23.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.296+0000 7f5652000640 1 -- 172.21.15.26:0/3814179148 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f565404d450 con 0x7f565c160920 2024-01-15T18:10:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.297+0000 7f5652000640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f563406c1f0 0x7f563406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.297+0000 7f5652000640 1 -- 172.21.15.26:0/3814179148 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f565402d040 con 0x7f565c160920 2024-01-15T18:10:23.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.298+0000 7f5659e00640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f563406c1f0 0x7f563406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.299+0000 7f5659e00640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f563406c1f0 0x7f563406e6b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f564c006fd0 tx=0x7f564c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:23.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.302+0000 7f5652000640 1 -- 172.21.15.26:0/3814179148 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f565c0b4b20 con 0x7f565c160920 2024-01-15T18:10:23.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.494+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f565c0b4b20 con 0x7f565c160920 2024-01-15T18:10:23.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.497+0000 7f5652000640 1 -- 172.21.15.26:0/3814179148 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16595 (secure 0 0 0) 0x7f565c0b4b20 con 0x7f565c160920 2024-01-15T18:10:23.500 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:23.501 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":101,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:10:22.071155+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:10:19.966552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-15T18:10:21.081716+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:10:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.499+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f563406c1f0 msgr2=0x7f563406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.499+0000 7f565b200640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f563406c1f0 0x7f563406e6b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f564c006fd0 tx=0x7f564c008040 comp rx=0 tx=0).stop 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.499+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c160920 msgr2=0x7f565c0ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.499+0000 7f565b200640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c160920 0x7f565c0ca7e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f56540090b0 tx=0x7f56540046c0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.500+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 shutdown_connections 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.500+0000 7f565b200640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f565c0cfea0 0x7f565c1f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.500+0000 7f565b200640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f563406c1f0 0x7f563406e6b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.500+0000 7f565b200640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f565c160920 0x7f565c0ca7e0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.500+0000 7f565b200640 1 --2- 172.21.15.26:0/3814179148 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f565c0cf9f0 0x7f565c0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.500+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 >> 172.21.15.26:0/3814179148 conn(0x7f565c0d3090 msgr2=0x7f565c06b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.500+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 shutdown_connections 2024-01-15T18:10:23.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.500+0000 7f565b200640 1 -- 172.21.15.26:0/3814179148 wait complete. 2024-01-15T18:10:23.525 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:10:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.755+0000 7f2bc5600640 1 Processor -- start 2024-01-15T18:10:23.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.755+0000 7f2bc5600640 1 -- start start 2024-01-15T18:10:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.756+0000 7f2bc5600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2bc00cdf00 0x7f2bc00d1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.756+0000 7f2bc5600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bc00b01e0 0x7f2bc00b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.756+0000 7f2bc5600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc015b5f0 con 0x7f2bc015b210 2024-01-15T18:10:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.756+0000 7f2bc5600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc00cce60 con 0x7f2bc00cdf00 2024-01-15T18:10:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.756+0000 7f2bc5600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00cf510 con 0x7f2bc00b01e0 2024-01-15T18:10:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.757+0000 7f2bbfe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2bc015b210 0x7f2bc00cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51480/0 (socket says 172.21.15.26:51480) 2024-01-15T18:10:23.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.757+0000 7f2bbfe00640 1 -- 172.21.15.26:0/741836779 learned_addr learned my addr 172.21.15.26:0/741836779 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:23.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.757+0000 7f2bc4c00640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bc00b01e0 0x7f2bc00b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.757+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457964007 0 0) 0x7f2bc015b5f0 con 0x7f2bc015b210 2024-01-15T18:10:23.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.757+0000 7f2bbf400640 1 --2- 172.21.15.26:0/741836779 >> v2:172.21.15.124:3300/0 conn(0x7f2bc00cdf00 0x7f2bc00d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.757+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba0003520 con 0x7f2bc015b210 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1603492622 0 0) 0x7f2ba0003520 con 0x7f2bc015b210 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bc00b01e0 msgr2=0x7f2bc00b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bc00b01e0 0x7f2bc00b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 >> v2:172.21.15.124:3300/0 conn(0x7f2bc00cdf00 msgr2=0x7f2bc00d1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 --2- 172.21.15.26:0/741836779 >> v2:172.21.15.124:3300/0 conn(0x7f2bc00cdf00 0x7f2bc00d1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00cf7f0 con 0x7f2bc015b210 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f2ba8001670 con 0x7f2bc015b210 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 >> v1:172.21.15.26:6789/0 conn(0x7f2bc015b210 legacy=0x7f2bc00cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bc00b01e0 0x7f2ba0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba0003f40 0x7f2ba00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ba0004d00 0x7f2ba0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00cce60 con 0x7f2ba0003f40 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00cf510 con 0x7f2ba0004d00 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba000bae0 con 0x7f2bc00b01e0 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbf400640 1 --2- 172.21.15.26:0/741836779 >> v2:172.21.15.124:3300/0 conn(0x7f2bc00cdf00 0x7f2bc00d1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbf400640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba0003f40 0x7f2ba00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bbfe00640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bc00b01e0 0x7f2ba0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.758+0000 7f2bc4c00640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ba0004d00 0x7f2ba0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.759+0000 7f2bbf400640 1 -- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bc00b01e0 msgr2=0x7f2ba0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.759+0000 7f2bbf400640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2bc00b01e0 0x7f2ba0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.759+0000 7f2bbf400640 1 -- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ba0004d00 msgr2=0x7f2ba0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.759+0000 7f2bbf400640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ba0004d00 0x7f2ba0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.759+0000 7f2bbf400640 1 -- 172.21.15.26:0/741836779 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba0003520 con 0x7f2ba0003f40 2024-01-15T18:10:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.760+0000 7f2bbf400640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba0003f40 0x7f2ba00047c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00098a0 tx=0x7f2ba00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359da6a22bd80cc5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.760+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0037600 con 0x7f2ba0003f40 2024-01-15T18:10:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.760+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb0037760 con 0x7f2ba0003f40 2024-01-15T18:10:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.760+0000 7f2bbea00640 1 -- 172.21.15.26:0/741836779 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0037ad0 con 0x7f2ba0003f40 2024-01-15T18:10:23.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.761+0000 7f2bc5600640 1 -- 172.21.15.26:0/741836779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba0003f40 msgr2=0x7f2ba00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.761+0000 7f2bc5600640 1 --2- 172.21.15.26:0/741836779 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ba0003f40 0x7f2ba00047c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f2bb00098a0 tx=0x7f2ba00057f0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.761+0000 7f2bc4c00640 1 -- 172.21.15.26:0/741836779 reap_dead start 2024-01-15T18:10:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.761+0000 7f2bc5600640 1 -- 172.21.15.26:0/741836779 shutdown_connections 2024-01-15T18:10:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.761+0000 7f2bc5600640 1 -- 172.21.15.26:0/741836779 >> 172.21.15.26:0/741836779 conn(0x7f2bc00d0e00 msgr2=0x7f2bc00d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.761+0000 7f2bc5600640 1 -- 172.21.15.26:0/741836779 shutdown_connections 2024-01-15T18:10:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.761+0000 7f2bc5600640 1 -- 172.21.15.26:0/741836779 wait complete. 2024-01-15T18:10:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.763+0000 7f2bc5600640 1 Processor -- start 2024-01-15T18:10:23.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.763+0000 7f2bc5600640 1 -- start start 2024-01-15T18:10:23.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc5600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ba0003f40 0x7f2bc00caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc5600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bc015d410 0x7f2bc015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc5600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc01e49d0 0x7f2bc01e4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc5600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00c7ed0 con 0x7f2bc01e49d0 2024-01-15T18:10:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc5600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc01e5390 con 0x7f2bc015d410 2024-01-15T18:10:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc5600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc015dc40 con 0x7f2ba0003f40 2024-01-15T18:10:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc01e49d0 0x7f2bc01e4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc4c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc01e49d0 0x7f2bc01e4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48536/0 (socket says 172.21.15.26:48536) 2024-01-15T18:10:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc4c00640 1 -- 172.21.15.26:0/1069730968 learned_addr learned my addr 172.21.15.26:0/1069730968 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:23.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bbf400640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bc015d410 0x7f2bc015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bbfe00640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ba0003f40 0x7f2bc00caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc4c00640 1 -- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ba0003f40 msgr2=0x7f2bc00caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc4c00640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ba0003f40 0x7f2bc00caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc4c00640 1 -- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bc015d410 msgr2=0x7f2bc015c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.764+0000 7f2bc4c00640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bc015d410 0x7f2bc015c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.765+0000 7f2bc4c00640 1 -- 172.21.15.26:0/1069730968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc015d960 con 0x7f2bc01e49d0 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.765+0000 7f2bbf400640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bc015d410 0x7f2bc015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.765+0000 7f2bc4c00640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc01e49d0 0x7f2bc01e4e50 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f2bc015ec40 tx=0x7f2bb800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.765+0000 7f2bb7e00640 1 -- 172.21.15.26:0/1069730968 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8017400 con 0x7f2bc01e49d0 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.765+0000 7f2bb7e00640 1 -- 172.21.15.26:0/1069730968 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8011040 con 0x7f2bc01e49d0 2024-01-15T18:10:23.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.765+0000 7f2bb7e00640 1 -- 172.21.15.26:0/1069730968 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80177c0 con 0x7f2bc01e49d0 2024-01-15T18:10:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.765+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc00c47f0 con 0x7f2bc01e49d0 2024-01-15T18:10:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.765+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc00c4db0 con 0x7f2bc01e49d0 2024-01-15T18:10:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.766+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc00c4650 con 0x7f2bc01e49d0 2024-01-15T18:10:23.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.767+0000 7f2bb7e00640 1 -- 172.21.15.26:0/1069730968 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f2bb8017920 con 0x7f2bc01e49d0 2024-01-15T18:10:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.769+0000 7f2bb7e00640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b9806c1f0 0x7f2b9806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.769+0000 7f2bb7e00640 1 -- 172.21.15.26:0/1069730968 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2bb8088c00 con 0x7f2bc01e49d0 2024-01-15T18:10:23.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.769+0000 7f2bbfe00640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b9806c1f0 0x7f2b9806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:23.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.770+0000 7f2bbfe00640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b9806c1f0 0x7f2b9806e6b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8001050 tx=0x7f2ba8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:23.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:23.774+0000 7f2bb7e00640 1 -- 172.21.15.26:0/1069730968 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2bc00c4650 con 0x7f2bc01e49d0 2024-01-15T18:10:24.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.009+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2bc00c4650 con 0x7f2bc01e49d0 2024-01-15T18:10:24.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.636+0000 7f2bb7e00640 1 -- 172.21.15.26:0/1069730968 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2bc00c4650 con 0x7f2bc01e49d0 2024-01-15T18:10:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b9806c1f0 msgr2=0x7f2b9806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b9806c1f0 0x7f2b9806e6b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8001050 tx=0x7f2ba8009040 comp rx=0 tx=0).stop 2024-01-15T18:10:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc01e49d0 msgr2=0x7f2bc01e4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc01e49d0 0x7f2bc01e4e50 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f2bc015ec40 tx=0x7f2bb800f4a0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 shutdown_connections 2024-01-15T18:10:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2b9806c1f0 0x7f2b9806e6b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2bc01e49d0 0x7f2bc01e4e50 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2bc015d410 0x7f2bc015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 --2- 172.21.15.26:0/1069730968 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ba0003f40 0x7f2bc00caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 >> 172.21.15.26:0/1069730968 conn(0x7f2bc00d0e00 msgr2=0x7f2bc00adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 shutdown_connections 2024-01-15T18:10:24.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.640+0000 7f2bc5600640 1 -- 172.21.15.26:0/1069730968 wait complete. 2024-01-15T18:10:24.667 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:10:24.667 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:10:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.887+0000 7fe0abe00640 1 Processor -- start 2024-01-15T18:10:24.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.888+0000 7fe0abe00640 1 -- start start 2024-01-15T18:10:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.889+0000 7fe0abe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe0ac0adf20 0x7fe0ac070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.889+0000 7fe0abe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac15f4e0 con 0x7fe0ac0ccb50 2024-01-15T18:10:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.889+0000 7fe0abe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ac071130 con 0x7fe0ac15bf10 2024-01-15T18:10:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.889+0000 7fe0abe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ac15f970 con 0x7fe0ac0adf20 2024-01-15T18:10:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.890+0000 7fe0ab400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe0ac0ccb50 0x7fe0ac143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51490/0 (socket says 172.21.15.26:51490) 2024-01-15T18:10:24.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.890+0000 7fe0ab400640 1 -- 172.21.15.26:0/2281045830 learned_addr learned my addr 172.21.15.26:0/2281045830 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:24.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.890+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124982296 0 0) 0x7fe0ac15f4e0 con 0x7fe0ac0ccb50 2024-01-15T18:10:24.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.890+0000 7fe0aaa00640 1 --2- 172.21.15.26:0/2281045830 >> v2:172.21.15.124:3300/0 conn(0x7fe0ac0adf20 0x7fe0ac070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.890+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe08c003520 con 0x7fe0ac0ccb50 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2037891094 0 0) 0x7fe08c003520 con 0x7fe0ac0ccb50 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 >> v2:172.21.15.124:3300/0 conn(0x7fe0ac0adf20 msgr2=0x7fe0ac070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 --2- 172.21.15.26:0/2281045830 >> v2:172.21.15.124:3300/0 conn(0x7fe0ac0adf20 0x7fe0ac070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 >> v1:172.21.15.124:6789/0 conn(0x7fe0ac15bf10 legacy=0x7fe0ac0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac15c390 con 0x7fe0ac0ccb50 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fe0a40019f0 con 0x7fe0ac0ccb50 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 >> v1:172.21.15.26:6789/0 conn(0x7fe0ac0ccb50 legacy=0x7fe0ac143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe08c0039c0 0x7fe08c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0047c0 0x7fe08c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe08c0052a0 0x7fe08c00fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac15f970 con 0x7fe08c0052a0 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0105d0 con 0x7fe08c0039c0 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c010790 con 0x7fe08c0047c0 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.891+0000 7fe0ab400640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe08c0052a0 0x7fe08c00fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.892+0000 7fe0aaa00640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe08c0039c0 0x7fe08c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.892+0000 7fe0a3e00640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0047c0 0x7fe08c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.892+0000 7fe0ab400640 1 -- 172.21.15.26:0/2281045830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0047c0 msgr2=0x7fe08c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.892+0000 7fe0ab400640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0047c0 0x7fe08c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.892+0000 7fe0ab400640 1 -- 172.21.15.26:0/2281045830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe08c0039c0 msgr2=0x7fe08c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.892+0000 7fe0ab400640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe08c0039c0 0x7fe08c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.892+0000 7fe0ab400640 1 -- 172.21.15.26:0/2281045830 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c003520 con 0x7fe08c0052a0 2024-01-15T18:10:24.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.892+0000 7fe0a3e00640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0047c0 0x7fe08c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.893+0000 7fe0ab400640 1 -- 172.21.15.26:0/2281045830 reap_dead start 2024-01-15T18:10:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.893+0000 7fe0ab400640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe08c0052a0 0x7fe08c00fd90 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400a750 tx=0x7fe08c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2907ccbe5a07164e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:24.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.893+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4034630 con 0x7fe08c0052a0 2024-01-15T18:10:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.893+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a402f070 con 0x7fe08c0052a0 2024-01-15T18:10:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.893+0000 7fe0aa000640 1 -- 172.21.15.26:0/2281045830 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a40349f0 con 0x7fe08c0052a0 2024-01-15T18:10:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.894+0000 7fe0abe00640 1 -- 172.21.15.26:0/2281045830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe08c0052a0 msgr2=0x7fe08c00fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.894+0000 7fe0abe00640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe08c0052a0 0x7fe08c00fd90 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400a750 tx=0x7fe08c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.894+0000 7fe0abe00640 1 -- 172.21.15.26:0/2281045830 shutdown_connections 2024-01-15T18:10:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.894+0000 7fe0abe00640 1 --2- 172.21.15.26:0/2281045830 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe08c0052a0 0x7fe08c00fd90 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.894+0000 7fe0abe00640 1 -- 172.21.15.26:0/2281045830 >> 172.21.15.26:0/2281045830 conn(0x7fe0ac06a7d0 msgr2=0x7fe0ac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.894+0000 7fe0abe00640 1 -- 172.21.15.26:0/2281045830 shutdown_connections 2024-01-15T18:10:24.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.894+0000 7fe0abe00640 1 -- 172.21.15.26:0/2281045830 wait complete. 2024-01-15T18:10:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.896+0000 7fe0abe00640 1 Processor -- start 2024-01-15T18:10:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.896+0000 7fe0abe00640 1 -- start start 2024-01-15T18:10:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.897+0000 7fe0abe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0052a0 0x7fe0ac0cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.897+0000 7fe0abe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac15a140 0x7fe0ac138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.897+0000 7fe0abe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ac15a690 0x7fe0ac138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.897+0000 7fe0abe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac13e490 con 0x7fe0ac15a690 2024-01-15T18:10:24.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.897+0000 7fe0abe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac13fa60 con 0x7fe0ac15a140 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.897+0000 7fe0abe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac13fd40 con 0x7fe08c0052a0 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.897+0000 7fe0ab400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ac15a690 0x7fe0ac138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.897+0000 7fe0ab400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ac15a690 0x7fe0ac138b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48552/0 (socket says 172.21.15.26:48552) 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.897+0000 7fe0ab400640 1 -- 172.21.15.26:0/575826344 learned_addr learned my addr 172.21.15.26:0/575826344 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0aaa00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0052a0 0x7fe0ac0cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0a3e00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac15a140 0x7fe0ac138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0ab400640 1 -- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0052a0 msgr2=0x7fe0ac0cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0ab400640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0052a0 0x7fe0ac0cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0ab400640 1 -- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac15a140 msgr2=0x7fe0ac138630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0ab400640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac15a140 0x7fe0ac138630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0ab400640 1 -- 172.21.15.26:0/575826344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac140020 con 0x7fe0ac15a690 2024-01-15T18:10:24.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0ab400640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ac15a690 0x7fe0ac138b70 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400a870 tx=0x7fe0ac15b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0a2a00640 1 -- 172.21.15.26:0/575826344 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4004d60 con 0x7fe0ac15a690 2024-01-15T18:10:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0a2a00640 1 -- 172.21.15.26:0/575826344 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a4001250 con 0x7fe0ac15a690 2024-01-15T18:10:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0a2a00640 1 -- 172.21.15.26:0/575826344 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4034670 con 0x7fe0ac15a690 2024-01-15T18:10:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.898+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac13e140 con 0x7fe0ac15a690 2024-01-15T18:10:24.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.899+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac13e800 con 0x7fe0ac15a690 2024-01-15T18:10:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.900+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac067450 con 0x7fe0ac15a690 2024-01-15T18:10:24.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.901+0000 7fe0a2a00640 1 -- 172.21.15.26:0/575826344 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fe0a4031030 con 0x7fe0ac15a690 2024-01-15T18:10:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.902+0000 7fe0a2a00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe08406c240 0x7fe08406e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.902+0000 7fe0a2a00640 1 -- 172.21.15.26:0/575826344 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe0a40a9960 con 0x7fe0ac15a690 2024-01-15T18:10:24.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.902+0000 7fe0aaa00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe08406c240 0x7fe08406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:24.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.904+0000 7fe0aaa00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe08406c240 0x7fe08406e700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe09c006fd0 tx=0x7fe09c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:24.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:24.906+0000 7fe0a2a00640 1 -- 172.21.15.26:0/575826344 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0a40762d0 con 0x7fe0ac15a690 2024-01-15T18:10:25.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.139+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe0ac1401b0 con 0x7fe0ac15a690 2024-01-15T18:10:25.525 INFO:tasks.ceph.mds.d.smithi124.stderr:2024-01-15T18:10:25.522+0000 7fe605c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:10:25.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.646+0000 7fe0a2a00640 1 -- 172.21.15.26:0/575826344 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe0a40430c0 con 0x7fe0ac15a690 2024-01-15T18:10:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.650+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe08406c240 msgr2=0x7fe08406e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.650+0000 7fe0abe00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe08406c240 0x7fe08406e700 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe09c006fd0 tx=0x7fe09c008040 comp rx=0 tx=0).stop 2024-01-15T18:10:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.650+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ac15a690 msgr2=0x7fe0ac138b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.650+0000 7fe0abe00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ac15a690 0x7fe0ac138b70 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400a870 tx=0x7fe0ac15b490 comp rx=0 tx=0).stop 2024-01-15T18:10:25.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.651+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 shutdown_connections 2024-01-15T18:10:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.651+0000 7fe0abe00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe08406c240 0x7fe08406e700 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.651+0000 7fe0abe00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe0ac15a690 0x7fe0ac138b70 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.651+0000 7fe0abe00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe0ac15a140 0x7fe0ac138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.651+0000 7fe0abe00640 1 --2- 172.21.15.26:0/575826344 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe08c0052a0 0x7fe0ac0cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.651+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 >> 172.21.15.26:0/575826344 conn(0x7fe0ac06a7d0 msgr2=0x7fe0ac0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.651+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 shutdown_connections 2024-01-15T18:10:25.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.651+0000 7fe0abe00640 1 -- 172.21.15.26:0/575826344 wait complete. 2024-01-15T18:10:25.675 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:10:25.676 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:10:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.892+0000 7fad3b400640 1 Processor -- start 2024-01-15T18:10:25.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.892+0000 7fad3b400640 1 -- start start 2024-01-15T18:10:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.894+0000 7fad3b400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fad340ae5a0 0x7fad3415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.894+0000 7fad3b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad340cf290 0x7fad340cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.894+0000 7fad3b400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad34070c40 con 0x7fad340b3ae0 2024-01-15T18:10:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.894+0000 7fad3b400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad34071450 con 0x7fad340cf290 2024-01-15T18:10:25.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.894+0000 7fad3b400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340cfc60 con 0x7fad340ae5a0 2024-01-15T18:10:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.895+0000 7fad39400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad340cf290 0x7fad340cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.895+0000 7fad39400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fad340cf290 0x7fad340cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48554/0 (socket says 172.21.15.26:48554) 2024-01-15T18:10:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.895+0000 7fad39400640 1 -- 172.21.15.26:0/3985145029 learned_addr learned my addr 172.21.15.26:0/3985145029 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.895+0000 7fad38a00640 1 --2- 172.21.15.26:0/3985145029 >> v2:172.21.15.124:3300/0 conn(0x7fad340ae5a0 0x7fad3415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.895+0000 7fad39400640 1 -- 172.21.15.26:0/3985145029 >> v2:172.21.15.124:3300/0 conn(0x7fad340ae5a0 msgr2=0x7fad3415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.895+0000 7fad39400640 1 --2- 172.21.15.26:0/3985145029 >> v2:172.21.15.124:3300/0 conn(0x7fad340ae5a0 0x7fad3415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.895+0000 7fad39400640 1 -- 172.21.15.26:0/3985145029 >> v1:172.21.15.124:6789/0 conn(0x7fad340b3ae0 legacy=0x7fad3406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:25.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.895+0000 7fad39400640 1 -- 172.21.15.26:0/3985145029 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3406c470 con 0x7fad340cf290 2024-01-15T18:10:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.897+0000 7fad39400640 1 --2- 172.21.15.26:0/3985145029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340cf290 0x7fad340cf720 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fad340cfe20 tx=0x7fad2c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52cfc934443f8b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.897+0000 7fad33e00640 1 -- 172.21.15.26:0/3985145029 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c02cbb0 con 0x7fad340cf290 2024-01-15T18:10:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.897+0000 7fad33e00640 1 -- 172.21.15.26:0/3985145029 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c02cd10 con 0x7fad340cf290 2024-01-15T18:10:25.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.897+0000 7fad33e00640 1 -- 172.21.15.26:0/3985145029 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c034610 con 0x7fad340cf290 2024-01-15T18:10:25.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.898+0000 7fad3b400640 1 -- 172.21.15.26:0/3985145029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340cf290 msgr2=0x7fad340cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:25.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.898+0000 7fad3b400640 1 --2- 172.21.15.26:0/3985145029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340cf290 0x7fad340cf720 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fad340cfe20 tx=0x7fad2c02c010 comp rx=0 tx=0).stop 2024-01-15T18:10:25.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.898+0000 7fad3b400640 1 -- 172.21.15.26:0/3985145029 shutdown_connections 2024-01-15T18:10:25.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.898+0000 7fad3b400640 1 --2- 172.21.15.26:0/3985145029 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340cf290 0x7fad340cf720 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.898+0000 7fad3b400640 1 --2- 172.21.15.26:0/3985145029 >> v2:172.21.15.124:3300/0 conn(0x7fad340ae5a0 0x7fad3415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.898+0000 7fad3b400640 1 -- 172.21.15.26:0/3985145029 >> 172.21.15.26:0/3985145029 conn(0x7fad3406a800 msgr2=0x7fad3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.898+0000 7fad3b400640 1 -- 172.21.15.26:0/3985145029 shutdown_connections 2024-01-15T18:10:25.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.898+0000 7fad3b400640 1 -- 172.21.15.26:0/3985145029 wait complete. 2024-01-15T18:10:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.900+0000 7fad3b400640 1 Processor -- start 2024-01-15T18:10:25.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.900+0000 7fad3b400640 1 -- start start 2024-01-15T18:10:25.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.901+0000 7fad3b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340ae5a0 0x7fad34160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.901+0000 7fad3b400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad340b3ae0 0x7fad34161470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.901+0000 7fad3b400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf290 0x7fad34158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad3b400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad34156f70 con 0x7fad340ae5a0 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad3b400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad34159270 con 0x7fad340cf290 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad3b400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad34159550 con 0x7fad340b3ae0 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad38a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340ae5a0 0x7fad34160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad38a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340ae5a0 0x7fad34160f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48566/0 (socket says 172.21.15.26:48566) 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad38a00640 1 -- 172.21.15.26:0/734960780 learned_addr learned my addr 172.21.15.26:0/734960780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad2be00640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad340b3ae0 0x7fad34161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad39400640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf290 0x7fad34158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad38a00640 1 -- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad340b3ae0 msgr2=0x7fad34161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:25.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad38a00640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad340b3ae0 0x7fad34161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad38a00640 1 -- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf290 msgr2=0x7fad34158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad38a00640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf290 0x7fad34158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.902+0000 7fad38a00640 1 -- 172.21.15.26:0/734960780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c009b30 con 0x7fad340ae5a0 2024-01-15T18:10:25.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.903+0000 7fad38a00640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340ae5a0 0x7fad34160f30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fad2400d800 tx=0x7fad2400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:25.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.903+0000 7fad31600640 1 -- 172.21.15.26:0/734960780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400b6c0 con 0x7fad340ae5a0 2024-01-15T18:10:25.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.903+0000 7fad31600640 1 -- 172.21.15.26:0/734960780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2400b820 con 0x7fad340ae5a0 2024-01-15T18:10:25.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.903+0000 7fad31600640 1 -- 172.21.15.26:0/734960780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400bbb0 con 0x7fad340ae5a0 2024-01-15T18:10:25.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.903+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad34159830 con 0x7fad340ae5a0 2024-01-15T18:10:25.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.903+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad34159df0 con 0x7fad340ae5a0 2024-01-15T18:10:25.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.905+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad34159fb0 con 0x7fad340ae5a0 2024-01-15T18:10:25.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.906+0000 7fad31600640 1 -- 172.21.15.26:0/734960780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fad2400bd10 con 0x7fad340ae5a0 2024-01-15T18:10:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.906+0000 7fad31600640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c1f0 0x7fad0c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.907+0000 7fad31600640 1 -- 172.21.15.26:0/734960780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fad24088f70 con 0x7fad340ae5a0 2024-01-15T18:10:25.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.907+0000 7fad2be00640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c1f0 0x7fad0c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:25.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.908+0000 7fad2be00640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c1f0 0x7fad0c06e6b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fad340ce8a0 tx=0x7fad1c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:25.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:25.911+0000 7fad31600640 1 -- 172.21.15.26:0/734960780 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad24055540 con 0x7fad340ae5a0 2024-01-15T18:10:26.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.144+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad340675d0 con 0x7fad340ae5a0 2024-01-15T18:10:26.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.145+0000 7fad31600640 1 -- 172.21.15.26:0/734960780 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fad24058b60 con 0x7fad340ae5a0 2024-01-15T18:10:26.149 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:26.149 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7015,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1631584304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1631584304},{"type":"v1","addr":"172.21.15.124:6835","nonce":1631584304}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2396377458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2396377458},{"type":"v1","addr":"172.21.15.26:6833","nonce":2396377458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/81533892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":81533892},{"type":"v1","addr":"172.21.15.26:6835","nonce":81533892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:10:22.071509+0000","modified":"2024-01-15T18:10:25.141235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36242":{"gid":36242,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":5,"addr":"172.21.15.124:6837/421656183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":421656183},{"type":"v1","addr":"172.21.15.124:6837","nonce":421656183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-01-15T18:10:26.150 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 63 2024-01-15T18:10:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c1f0 msgr2=0x7fad0c06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c1f0 0x7fad0c06e6b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fad340ce8a0 tx=0x7fad1c009040 comp rx=0 tx=0).stop 2024-01-15T18:10:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340ae5a0 msgr2=0x7fad34160f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340ae5a0 0x7fad34160f30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fad2400d800 tx=0x7fad2400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 shutdown_connections 2024-01-15T18:10:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fad340b3ae0 0x7fad34161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fad0c06c1f0 0x7fad0c06e6b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fad340cf290 0x7fad34158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 --2- 172.21.15.26:0/734960780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fad340ae5a0 0x7fad34160f30 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.148+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 >> 172.21.15.26:0/734960780 conn(0x7fad3406a800 msgr2=0x7fad3414fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.149+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 shutdown_connections 2024-01-15T18:10:26.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.149+0000 7fad3b400640 1 -- 172.21.15.26:0/734960780 wait complete. 2024-01-15T18:10:26.173 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:10:26.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.392+0000 7f997be00640 1 Processor -- start 2024-01-15T18:10:26.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.392+0000 7f997be00640 1 -- start start 2024-01-15T18:10:26.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.393+0000 7f997be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 0x7f9974161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.393+0000 7f997be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f99740ced50 0x7f99740d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.393+0000 7f997be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740cf1b0 con 0x7f99740ce8f0 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.393+0000 7f997be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99741618a0 con 0x7f99740ced50 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.393+0000 7f997be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740d0de0 con 0x7f9974160f80 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.394+0000 7f9978a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f99740ce8f0 0x7f99740cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51500/0 (socket says 172.21.15.26:51500) 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.394+0000 7f9978a00640 1 -- 172.21.15.26:0/3684587866 learned_addr learned my addr 172.21.15.26:0/3684587866 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.394+0000 7f9979400640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 0x7f9974161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.394+0000 7f9979e00640 1 --2- 172.21.15.26:0/3684587866 >> v2:172.21.15.124:3300/0 conn(0x7f99740ced50 0x7f99740d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.394+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1207908947 0 0) 0x7f99740cf1b0 con 0x7f99740ce8f0 2024-01-15T18:10:26.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.394+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9954003520 con 0x7f99740ce8f0 2024-01-15T18:10:26.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 52567749 0 0) 0x7f9954003520 con 0x7f99740ce8f0 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 msgr2=0x7f9974161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 0x7f9974161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 >> v2:172.21.15.124:3300/0 conn(0x7f99740ced50 msgr2=0x7f99740d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 --2- 172.21.15.26:0/3684587866 >> v2:172.21.15.124:3300/0 conn(0x7f99740ced50 0x7f99740d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740d10c0 con 0x7f99740ce8f0 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f995c0019f0 con 0x7f99740ce8f0 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9979400640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 0x7f9974161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 >> v1:172.21.15.26:6789/0 conn(0x7f99740ce8f0 legacy=0x7f99740cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 0x7f9954003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9954003eb0 0x7f99540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9954004d00 0x7f9954005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99741618a0 con 0x7f9954003eb0 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740d0de0 con 0x7f9954004d00 2024-01-15T18:10:26.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995400b980 con 0x7f9974160f80 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.395+0000 7f9978a00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9954003eb0 0x7f99540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.396+0000 7f9979400640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 0x7f9954003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.396+0000 7f9979e00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9954004d00 0x7f9954005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.396+0000 7f9978a00640 1 -- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 msgr2=0x7f9954003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.396+0000 7f9978a00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 0x7f9954003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.396+0000 7f9978a00640 1 -- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9954004d00 msgr2=0x7f9954005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.396+0000 7f9978a00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9954004d00 0x7f9954005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.396+0000 7f9978a00640 1 -- 172.21.15.26:0/3684587866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9964009400 con 0x7f9954003eb0 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.396+0000 7f9979400640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9974160f80 0x7f9954003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.396+0000 7f9979e00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9954004d00 0x7f9954005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.397+0000 7f9978a00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9954003eb0 0x7f99540047c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f995c00a860 tx=0x7f99540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ede04e3e8b659f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.397+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c034630 con 0x7f9954003eb0 2024-01-15T18:10:26.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.397+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c02f070 con 0x7f9954003eb0 2024-01-15T18:10:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.397+0000 7f9973e00640 1 -- 172.21.15.26:0/3684587866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c034a30 con 0x7f9954003eb0 2024-01-15T18:10:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.398+0000 7f997be00640 1 -- 172.21.15.26:0/3684587866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9954003eb0 msgr2=0x7f99540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.398+0000 7f997be00640 1 --2- 172.21.15.26:0/3684587866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9954003eb0 0x7f99540047c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f995c00a860 tx=0x7f99540057f0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.398+0000 7f9979e00640 1 -- 172.21.15.26:0/3684587866 reap_dead start 2024-01-15T18:10:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.398+0000 7f997be00640 1 -- 172.21.15.26:0/3684587866 shutdown_connections 2024-01-15T18:10:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.398+0000 7f997be00640 1 -- 172.21.15.26:0/3684587866 >> 172.21.15.26:0/3684587866 conn(0x7f997409c550 msgr2=0x7f99740d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.398+0000 7f997be00640 1 -- 172.21.15.26:0/3684587866 shutdown_connections 2024-01-15T18:10:26.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.398+0000 7f997be00640 1 -- 172.21.15.26:0/3684587866 wait complete. 2024-01-15T18:10:26.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.400+0000 7f997be00640 1 Processor -- start 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.400+0000 7f997be00640 1 -- start start 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.401+0000 7f997be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9954003eb0 0x7f99741514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.401+0000 7f997be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99740cc3c0 0x7f99740cc840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.401+0000 7f997be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99741524b0 0x7f9974152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.401+0000 7f997be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974152e50 con 0x7f99741524b0 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.401+0000 7f997be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740cbf30 con 0x7f99740cc3c0 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.401+0000 7f997be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740cb390 con 0x7f9954003eb0 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.401+0000 7f9979e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99741524b0 0x7f9974152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.401+0000 7f9979e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99741524b0 0x7f9974152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48576/0 (socket says 172.21.15.26:48576) 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.401+0000 7f9979e00640 1 -- 172.21.15.26:0/3435968126 learned_addr learned my addr 172.21.15.26:0/3435968126 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.402+0000 7f9979400640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9954003eb0 0x7f99741514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.402+0000 7f9978a00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99740cc3c0 0x7f99740cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.402+0000 7f9979e00640 1 -- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9954003eb0 msgr2=0x7f99741514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.402+0000 7f9979e00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9954003eb0 0x7f99741514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.402+0000 7f9979e00640 1 -- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99740cc3c0 msgr2=0x7f99740cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.402+0000 7f9979e00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99740cc3c0 0x7f99740cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.402+0000 7f9979e00640 1 -- 172.21.15.26:0/3435968126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974151a30 con 0x7f99741524b0 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.402+0000 7f9978a00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99740cc3c0 0x7f99740cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.402+0000 7f9979e00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99741524b0 0x7f9974152910 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f996c010790 tx=0x7f996c010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.403+0000 7f9971600640 1 -- 172.21.15.26:0/3435968126 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00e8d0 con 0x7f99741524b0 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.403+0000 7f9971600640 1 -- 172.21.15.26:0/3435968126 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00ea30 con 0x7f99741524b0 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.403+0000 7f9971600640 1 -- 172.21.15.26:0/3435968126 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00edc0 con 0x7f99741524b0 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.403+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99740cb600 con 0x7f99741524b0 2024-01-15T18:10:26.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.403+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99740cbbc0 con 0x7f99741524b0 2024-01-15T18:10:26.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.404+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99740675d0 con 0x7f99741524b0 2024-01-15T18:10:26.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.405+0000 7f9971600640 1 -- 172.21.15.26:0/3435968126 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f996c01c400 con 0x7f99741524b0 2024-01-15T18:10:26.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.406+0000 7f9971600640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f994c06c1f0 0x7f994c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.406+0000 7f9971600640 1 -- 172.21.15.26:0/3435968126 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f996c088830 con 0x7f99741524b0 2024-01-15T18:10:26.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.406+0000 7f9979400640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f994c06c1f0 0x7f994c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.408+0000 7f9979400640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f994c06c1f0 0x7f994c06e6b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9974158de0 tx=0x7f9964008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:26.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.410+0000 7f9971600640 1 -- 172.21.15.26:0/3435968126 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f996c054eb0 con 0x7f99741524b0 2024-01-15T18:10:26.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.609+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99740cc070 con 0x7f99741524b0 2024-01-15T18:10:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.611+0000 7f9971600640 1 -- 172.21.15.26:0/3435968126 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f996c000e60 con 0x7f99741524b0 2024-01-15T18:10:26.615 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:26.615 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":102,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:10:23.589882+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:10:19.966552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-01-15T18:10:21.081716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f994c06c1f0 msgr2=0x7f994c06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f994c06c1f0 0x7f994c06e6b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9974158de0 tx=0x7f9964008040 comp rx=0 tx=0).stop 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99741524b0 msgr2=0x7f9974152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99741524b0 0x7f9974152910 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f996c010790 tx=0x7f996c010c50 comp rx=0 tx=0).stop 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 shutdown_connections 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f994c06c1f0 0x7f994c06e6b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f99741524b0 0x7f9974152910 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f99740cc3c0 0x7f99740cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 --2- 172.21.15.26:0/3435968126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9954003eb0 0x7f99741514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 >> 172.21.15.26:0/3435968126 conn(0x7f997409c550 msgr2=0x7f99740d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 shutdown_connections 2024-01-15T18:10:26.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.614+0000 7f997be00640 1 -- 172.21.15.26:0/3435968126 wait complete. 2024-01-15T18:10:26.638 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:10:26.638 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:10:26.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.852+0000 7f2565a00640 1 Processor -- start 2024-01-15T18:10:26.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.852+0000 7f2565a00640 1 -- start start 2024-01-15T18:10:26.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.854+0000 7f2565a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f256013df80 0x7f256015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.854+0000 7f2565a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25600cf9a0 0x7f25600cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.854+0000 7f2565a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256006c470 con 0x7f256013e360 2024-01-15T18:10:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.854+0000 7f2565a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256006c750 con 0x7f256013df80 2024-01-15T18:10:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.854+0000 7f2565a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256006ca30 con 0x7f25600cf9a0 2024-01-15T18:10:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.855+0000 7f255f400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25600cf9a0 0x7f25600cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.855+0000 7f255f400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25600cf9a0 0x7f25600cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:55744/0 (socket says 172.21.15.26:55744) 2024-01-15T18:10:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.855+0000 7f255f400640 1 -- 172.21.15.26:0/2813711110 learned_addr learned my addr 172.21.15.26:0/2813711110 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:26.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.855+0000 7f255ea00640 1 --2- 172.21.15.26:0/2813711110 >> v2:172.21.15.124:3300/0 conn(0x7f256013df80 0x7f256015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.856+0000 7f255f400640 1 -- 172.21.15.26:0/2813711110 >> v2:172.21.15.124:3300/0 conn(0x7f256013df80 msgr2=0x7f256015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.856+0000 7f255f400640 1 --2- 172.21.15.26:0/2813711110 >> v2:172.21.15.124:3300/0 conn(0x7f256013df80 0x7f256015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.856+0000 7f255f400640 1 -- 172.21.15.26:0/2813711110 >> v1:172.21.15.124:6789/0 conn(0x7f256013e360 legacy=0x7f25600cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.856+0000 7f255e000640 1 -- 172.21.15.26:0/2813711110 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98564554 0 0) 0x7f256006c470 con 0x7f256013e360 2024-01-15T18:10:26.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.856+0000 7f255f400640 1 -- 172.21.15.26:0/2813711110 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256006cd10 con 0x7f25600cf9a0 2024-01-15T18:10:26.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.857+0000 7f255f400640 1 --2- 172.21.15.26:0/2813711110 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25600cf9a0 0x7f25600cd840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e50 tx=0x7f255802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd42de0e8d0cbdb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:26.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.858+0000 7f255e000640 1 -- 172.21.15.26:0/2813711110 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255802cb00 con 0x7f25600cf9a0 2024-01-15T18:10:26.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.858+0000 7f255e000640 1 -- 172.21.15.26:0/2813711110 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255802cc60 con 0x7f25600cf9a0 2024-01-15T18:10:26.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.858+0000 7f255e000640 1 -- 172.21.15.26:0/2813711110 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25580343f0 con 0x7f25600cf9a0 2024-01-15T18:10:26.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.858+0000 7f2565a00640 1 -- 172.21.15.26:0/2813711110 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25600cf9a0 msgr2=0x7f25600cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.858+0000 7f2565a00640 1 --2- 172.21.15.26:0/2813711110 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25600cf9a0 0x7f25600cd840 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2558009e50 tx=0x7f255802bf60 comp rx=0 tx=0).stop 2024-01-15T18:10:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.859+0000 7f2565a00640 1 -- 172.21.15.26:0/2813711110 shutdown_connections 2024-01-15T18:10:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.859+0000 7f2565a00640 1 --2- 172.21.15.26:0/2813711110 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f25600cf9a0 0x7f25600cd840 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.859+0000 7f2565a00640 1 --2- 172.21.15.26:0/2813711110 >> v2:172.21.15.124:3300/0 conn(0x7f256013df80 0x7f256015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.859+0000 7f2565a00640 1 -- 172.21.15.26:0/2813711110 >> 172.21.15.26:0/2813711110 conn(0x7f256006a800 msgr2=0x7f256006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.859+0000 7f2565a00640 1 -- 172.21.15.26:0/2813711110 shutdown_connections 2024-01-15T18:10:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.859+0000 7f2565a00640 1 -- 172.21.15.26:0/2813711110 wait complete. 2024-01-15T18:10:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.861+0000 7f2565a00640 1 Processor -- start 2024-01-15T18:10:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.861+0000 7f2565a00640 1 -- start start 2024-01-15T18:10:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f2565a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25600cf9a0 0x7f25601531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f2565a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f256013df80 0x7f25600b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f2565a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f256013e360 0x7f25600b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f2565a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b1ca0 con 0x7f25600cf9a0 2024-01-15T18:10:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f2565a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b1f80 con 0x7f256013df80 2024-01-15T18:10:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f2565a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b2260 con 0x7f256013e360 2024-01-15T18:10:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f255ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25600cf9a0 0x7f25601531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f255ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25600cf9a0 0x7f25601531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48590/0 (socket says 172.21.15.26:48590) 2024-01-15T18:10:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f255ea00640 1 -- 172.21.15.26:0/3271983470 learned_addr learned my addr 172.21.15.26:0/3271983470 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.863+0000 7f2557e00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f256013df80 0x7f25600b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f255f400640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f256013e360 0x7f25600b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f255ea00640 1 -- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f256013e360 msgr2=0x7f25600b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f255ea00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f256013e360 0x7f25600b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f255ea00640 1 -- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f256013df80 msgr2=0x7f25600b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f255ea00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f256013df80 0x7f25600b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f255ea00640 1 -- 172.21.15.26:0/3271983470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2558009b30 con 0x7f25600cf9a0 2024-01-15T18:10:26.868 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f255ea00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25600cf9a0 0x7f25601531c0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f255000e9b0 tx=0x7f255000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f2556a00640 1 -- 172.21.15.26:0/3271983470 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550009670 con 0x7f25600cf9a0 2024-01-15T18:10:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f2556a00640 1 -- 172.21.15.26:0/3271983470 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255000c9a0 con 0x7f25600cf9a0 2024-01-15T18:10:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f2556a00640 1 -- 172.21.15.26:0/3271983470 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255000cd50 con 0x7f25600cf9a0 2024-01-15T18:10:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.864+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25600b2540 con 0x7f25600cf9a0 2024-01-15T18:10:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.865+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25600b2b00 con 0x7f25600cf9a0 2024-01-15T18:10:26.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.866+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25600675d0 con 0x7f25600cf9a0 2024-01-15T18:10:26.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.867+0000 7f2556a00640 1 -- 172.21.15.26:0/3271983470 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f2550013470 con 0x7f25600cf9a0 2024-01-15T18:10:26.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.868+0000 7f2556a00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f253806c1f0 0x7f253806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:26.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.868+0000 7f2556a00640 1 -- 172.21.15.26:0/3271983470 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f25500889c0 con 0x7f25600cf9a0 2024-01-15T18:10:26.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.868+0000 7f2557e00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f253806c1f0 0x7f253806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:26.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.869+0000 7f2557e00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f253806c1f0 0x7f253806e6b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f256015f300 tx=0x7f2548009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:26.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:26.872+0000 7f2556a00640 1 -- 172.21.15.26:0/3271983470 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25500551f0 con 0x7f25600cf9a0 2024-01-15T18:10:27.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.063+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25600b23d0 con 0x7f253806c1f0 2024-01-15T18:10:27.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.090+0000 7f2556a00640 1 -- 172.21.15.26:0/3271983470 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281699 (secure 0 0 0) 0x7f25600b23d0 con 0x7f253806c1f0 2024-01-15T18:10:27.098 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:27.099 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:10:27.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f253806c1f0 msgr2=0x7f253806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f253806c1f0 0x7f253806e6b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f256015f300 tx=0x7f2548009040 comp rx=0 tx=0).stop 2024-01-15T18:10:27.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25600cf9a0 msgr2=0x7f25601531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25600cf9a0 0x7f25601531c0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f255000e9b0 tx=0x7f255000ee70 comp rx=0 tx=0).stop 2024-01-15T18:10:27.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 shutdown_connections 2024-01-15T18:10:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f256013e360 0x7f25600b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f253806c1f0 0x7f253806e6b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f256013df80 0x7f25600b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 --2- 172.21.15.26:0/3271983470 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f25600cf9a0 0x7f25601531c0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.095+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 >> 172.21.15.26:0/3271983470 conn(0x7f256006a800 msgr2=0x7f25600afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.096+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 shutdown_connections 2024-01-15T18:10:27.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.096+0000 7f2565a00640 1 -- 172.21.15.26:0/3271983470 wait complete. 2024-01-15T18:10:27.119 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2024-01-15T18:10:26.272048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":580416,"kb_used_data":10768,"kb_used_omap":12,"kb_used_meta":569523,"kb_avail":754394304,"statfs":{"total":773094113280,"available":772499767296,"internally_reserved":0,"allocated":11026432,"data_stored":5436266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":583192112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697703"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611673+0000","last_change":"2024-01-15T18:10:21.611673+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611673+0000","last_undegraded":"2024-01-15T18:10:21.611673+0000","last_fullsized":"2024-01-15T18:10:21.611673+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611314+0000","last_change":"2024-01-15T18:10:21.611314+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611314+0000","last_undegraded":"2024-01-15T18:10:21.611314+0000","last_fullsized":"2024-01-15T18:10:21.611314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599418+0000","last_change":"2024-01-15T18:10:21.599418+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599418+0000","last_undegraded":"2024-01-15T18:10:21.599418+0000","last_fullsized":"2024-01-15T18:10:21.599418+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601091+0000","last_change":"2024-01-15T18:10:21.601091+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601091+0000","last_undegraded":"2024-01-15T18:10:21.601091+0000","last_fullsized":"2024-01-15T18:10:21.601091+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596994+0000","last_change":"2024-01-15T18:10:21.596994+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596994+0000","last_undegraded":"2024-01-15T18:10:21.596994+0000","last_fullsized":"2024-01-15T18:10:21.596994+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.607608+0000","last_change":"2024-01-15T18:10:21.607608+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.607608+0000","last_undegraded":"2024-01-15T18:10:21.607608+0000","last_fullsized":"2024-01-15T18:10:21.607608+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611612+0000","last_change":"2024-01-15T18:10:21.611612+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611612+0000","last_undegraded":"2024-01-15T18:10:21.611612+0000","last_fullsized":"2024-01-15T18:10:21.611612+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.610269+0000","last_change":"2024-01-15T18:10:21.610269+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.610269+0000","last_undegraded":"2024-01-15T18:10:21.610269+0000","last_fullsized":"2024-01-15T18:10:21.610269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.616486+0000","last_change":"2024-01-15T18:10:21.616486+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.616486+0000","last_undegraded":"2024-01-15T18:10:21.616486+0000","last_fullsized":"2024-01-15T18:10:21.616486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597244+0000","last_change":"2024-01-15T18:10:21.597244+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597244+0000","last_undegraded":"2024-01-15T18:10:21.597244+0000","last_fullsized":"2024-01-15T18:10:21.597244+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.616047+0000","last_change":"2024-01-15T18:10:21.616047+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.616047+0000","last_undegraded":"2024-01-15T18:10:21.616047+0000","last_fullsized":"2024-01-15T18:10:21.616047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.605943+0000","last_change":"2024-01-15T18:10:21.605943+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.605943+0000","last_undegraded":"2024-01-15T18:10:21.605943+0000","last_fullsized":"2024-01-15T18:10:21.605943+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.602017+0000","last_change":"2024-01-15T18:10:21.602017+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.602017+0000","last_undegraded":"2024-01-15T18:10:21.602017+0000","last_fullsized":"2024-01-15T18:10:21.602017+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600750+0000","last_change":"2024-01-15T18:10:21.600750+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600750+0000","last_undegraded":"2024-01-15T18:10:21.600750+0000","last_fullsized":"2024-01-15T18:10:21.600750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600926+0000","last_change":"2024-01-15T18:10:21.600926+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600926+0000","last_undegraded":"2024-01-15T18:10:21.600926+0000","last_fullsized":"2024-01-15T18:10:21.600926+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608161+0000","last_change":"2024-01-15T18:10:21.608161+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608161+0000","last_undegraded":"2024-01-15T18:10:21.608161+0000","last_fullsized":"2024-01-15T18:10:21.608161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608276+0000","last_change":"2024-01-15T18:10:21.608276+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608276+0000","last_undegraded":"2024-01-15T18:10:21.608276+0000","last_fullsized":"2024-01-15T18:10:21.608276+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601345+0000","last_change":"2024-01-15T18:10:21.601345+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601345+0000","last_undegraded":"2024-01-15T18:10:21.601345+0000","last_fullsized":"2024-01-15T18:10:21.601345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596934+0000","last_change":"2024-01-15T18:10:21.596934+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596934+0000","last_undegraded":"2024-01-15T18:10:21.596934+0000","last_fullsized":"2024-01-15T18:10:21.596934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.604940+0000","last_change":"2024-01-15T18:10:21.604940+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.604940+0000","last_undegraded":"2024-01-15T18:10:21.604940+0000","last_fullsized":"2024-01-15T18:10:21.604940+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597709+0000","last_change":"2024-01-15T18:10:21.597709+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597709+0000","last_undegraded":"2024-01-15T18:10:21.597709+0000","last_fullsized":"2024-01-15T18:10:21.597709+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608794+0000","last_change":"2024-01-15T18:10:21.608794+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608794+0000","last_undegraded":"2024-01-15T18:10:21.608794+0000","last_fullsized":"2024-01-15T18:10:21.608794+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608380+0000","last_change":"2024-01-15T18:10:21.608380+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608380+0000","last_undegraded":"2024-01-15T18:10:21.608380+0000","last_fullsized":"2024-01-15T18:10:21.608380+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.606909+0000","last_change":"2024-01-15T18:10:21.606909+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.606909+0000","last_undegraded":"2024-01-15T18:10:21.606909+0000","last_fullsized":"2024-01-15T18:10:21.606909+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596824+0000","last_change":"2024-01-15T18:10:21.596824+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596824+0000","last_undegraded":"2024-01-15T18:10:21.596824+0000","last_fullsized":"2024-01-15T18:10:21.596824+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608926+0000","last_change":"2024-01-15T18:10:21.608926+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608926+0000","last_undegraded":"2024-01-15T18:10:21.608926+0000","last_fullsized":"2024-01-15T18:10:21.608926+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.615205+0000","last_change":"2024-01-15T18:10:21.615205+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.615205+0000","last_undegraded":"2024-01-15T18:10:21.615205+0000","last_fullsized":"2024-01-15T18:10:21.615205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601293+0000","last_change":"2024-01-15T18:10:21.601293+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601293+0000","last_undegraded":"2024-01-15T18:10:21.601293+0000","last_fullsized":"2024-01-15T18:10:21.601293+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.606505+0000","last_change":"2024-01-15T18:10:21.606505+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.606505+0000","last_undegraded":"2024-01-15T18:10:21.606505+0000","last_fullsized":"2024-01-15T18:10:21.606505+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600333+0000","last_change":"2024-01-15T18:10:21.600333+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600333+0000","last_undegraded":"2024-01-15T18:10:21.600333+0000","last_fullsized":"2024-01-15T18:10:21.600333+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596109+0000","last_change":"2024-01-15T18:10:21.596109+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596109+0000","last_undegraded":"2024-01-15T18:10:21.596109+0000","last_fullsized":"2024-01-15T18:10:21.596109+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.604227+0000","last_change":"2024-01-15T18:10:21.604227+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.604227+0000","last_undegraded":"2024-01-15T18:10:21.604227+0000","last_fullsized":"2024-01-15T18:10:21.604227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599903+0000","last_change":"2024-01-15T18:10:21.599903+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599903+0000","last_undegraded":"2024-01-15T18:10:21.599903+0000","last_fullsized":"2024-01-15T18:10:21.599903+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.618790+0000","last_change":"2024-01-15T18:10:21.616691+0000","last_active":"2024-01-15T18:10:21.618790+0000","last_peered":"2024-01-15T18:10:21.618790+0000","last_clean":"2024-01-15T18:10:21.618790+0000","last_became_active":"2024-01-15T18:10:21.616428+0000","last_became_peered":"2024-01-15T18:10:21.616428+0000","last_unstale":"2024-01-15T18:10:21.618790+0000","last_undegraded":"2024-01-15T18:10:21.618790+0000","last_fullsized":"2024-01-15T18:10:21.618790+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.598656+0000","last_change":"2024-01-15T18:10:21.598656+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.598656+0000","last_undegraded":"2024-01-15T18:10:21.598656+0000","last_fullsized":"2024-01-15T18:10:21.598656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.624911+0000","last_change":"2024-01-15T18:10:21.624181+0000","last_active":"2024-01-15T18:10:21.624911+0000","last_peered":"2024-01-15T18:10:21.624911+0000","last_clean":"2024-01-15T18:10:21.624911+0000","last_became_active":"2024-01-15T18:10:21.623845+0000","last_became_peered":"2024-01-15T18:10:21.623845+0000","last_unstale":"2024-01-15T18:10:21.624911+0000","last_undegraded":"2024-01-15T18:10:21.624911+0000","last_fullsized":"2024-01-15T18:10:21.624911+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:31:30.348979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.607177+0000","last_change":"2024-01-15T18:10:21.607177+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.607177+0000","last_undegraded":"2024-01-15T18:10:21.607177+0000","last_fullsized":"2024-01-15T18:10:21.607177+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623538+0000","last_change":"2024-01-15T18:10:21.621640+0000","last_active":"2024-01-15T18:10:21.623538+0000","last_peered":"2024-01-15T18:10:21.623538+0000","last_clean":"2024-01-15T18:10:21.623538+0000","last_became_active":"2024-01-15T18:10:21.619850+0000","last_became_peered":"2024-01-15T18:10:21.619850+0000","last_unstale":"2024-01-15T18:10:21.623538+0000","last_undegraded":"2024-01-15T18:10:21.623538+0000","last_fullsized":"2024-01-15T18:10:21.623538+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:26.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608242+0000","last_change":"2024-01-15T18:10:21.608242+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608242+0000","last_undegraded":"2024-01-15T18:10:21.608242+0000","last_fullsized":"2024-01-15T18:10:21.608242+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622223+0000","last_change":"2024-01-15T18:10:21.619185+0000","last_active":"2024-01-15T18:10:21.622223+0000","last_peered":"2024-01-15T18:10:21.622223+0000","last_clean":"2024-01-15T18:10:21.622223+0000","last_became_active":"2024-01-15T18:10:21.618436+0000","last_became_peered":"2024-01-15T18:10:21.618436+0000","last_unstale":"2024-01-15T18:10:21.622223+0000","last_undegraded":"2024-01-15T18:10:21.622223+0000","last_fullsized":"2024-01-15T18:10:21.622223+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:15.951926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599261+0000","last_change":"2024-01-15T18:10:21.599261+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599261+0000","last_undegraded":"2024-01-15T18:10:21.599261+0000","last_fullsized":"2024-01-15T18:10:21.599261+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622304+0000","last_change":"2024-01-15T18:10:21.618207+0000","last_active":"2024-01-15T18:10:21.622304+0000","last_peered":"2024-01-15T18:10:21.622304+0000","last_clean":"2024-01-15T18:10:21.622304+0000","last_became_active":"2024-01-15T18:10:21.617907+0000","last_became_peered":"2024-01-15T18:10:21.617907+0000","last_unstale":"2024-01-15T18:10:21.622304+0000","last_undegraded":"2024-01-15T18:10:21.622304+0000","last_fullsized":"2024-01-15T18:10:21.622304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:10.136007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611462+0000","last_change":"2024-01-15T18:10:21.611462+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611462+0000","last_undegraded":"2024-01-15T18:10:21.611462+0000","last_fullsized":"2024-01-15T18:10:21.611462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.618613+0000","last_change":"2024-01-15T18:10:21.616023+0000","last_active":"2024-01-15T18:10:21.618613+0000","last_peered":"2024-01-15T18:10:21.618613+0000","last_clean":"2024-01-15T18:10:21.618613+0000","last_became_active":"2024-01-15T18:10:21.615732+0000","last_became_peered":"2024-01-15T18:10:21.615732+0000","last_unstale":"2024-01-15T18:10:21.618613+0000","last_undegraded":"2024-01-15T18:10:21.618613+0000","last_fullsized":"2024-01-15T18:10:21.618613+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:30.881887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.595870+0000","last_change":"2024-01-15T18:10:21.595870+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.595870+0000","last_undegraded":"2024-01-15T18:10:21.595870+0000","last_fullsized":"2024-01-15T18:10:21.595870+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.616352+0000","last_change":"2024-01-15T18:10:21.613918+0000","last_active":"2024-01-15T18:10:21.616352+0000","last_peered":"2024-01-15T18:10:21.616352+0000","last_clean":"2024-01-15T18:10:21.616352+0000","last_became_active":"2024-01-15T18:10:21.613184+0000","last_became_peered":"2024-01-15T18:10:21.613184+0000","last_unstale":"2024-01-15T18:10:21.616352+0000","last_undegraded":"2024-01-15T18:10:21.616352+0000","last_fullsized":"2024-01-15T18:10:21.616352+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:26.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.609479+0000","last_change":"2024-01-15T18:10:21.609479+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.609479+0000","last_undegraded":"2024-01-15T18:10:21.609479+0000","last_fullsized":"2024-01-15T18:10:21.609479+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617154+0000","last_change":"2024-01-15T18:10:21.608770+0000","last_active":"2024-01-15T18:10:21.617154+0000","last_peered":"2024-01-15T18:10:21.617154+0000","last_clean":"2024-01-15T18:10:21.617154+0000","last_became_active":"2024-01-15T18:10:21.607618+0000","last_became_peered":"2024-01-15T18:10:21.607618+0000","last_unstale":"2024-01-15T18:10:21.617154+0000","last_undegraded":"2024-01-15T18:10:21.617154+0000","last_fullsized":"2024-01-15T18:10:21.617154+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:59.671748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.605852+0000","last_change":"2024-01-15T18:10:21.605852+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.605852+0000","last_undegraded":"2024-01-15T18:10:21.605852+0000","last_fullsized":"2024-01-15T18:10:21.605852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.616985+0000","last_change":"2024-01-15T18:10:21.613722+0000","last_active":"2024-01-15T18:10:21.616985+0000","last_peered":"2024-01-15T18:10:21.616985+0000","last_clean":"2024-01-15T18:10:21.616985+0000","last_became_active":"2024-01-15T18:10:21.613117+0000","last_became_peered":"2024-01-15T18:10:21.613117+0000","last_unstale":"2024-01-15T18:10:21.616985+0000","last_undegraded":"2024-01-15T18:10:21.616985+0000","last_fullsized":"2024-01-15T18:10:21.616985+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:11.338333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611500+0000","last_change":"2024-01-15T18:10:21.611500+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611500+0000","last_undegraded":"2024-01-15T18:10:21.611500+0000","last_fullsized":"2024-01-15T18:10:21.611500+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.615908+0000","last_change":"2024-01-15T18:10:21.612961+0000","last_active":"2024-01-15T18:10:21.615908+0000","last_peered":"2024-01-15T18:10:21.615908+0000","last_clean":"2024-01-15T18:10:21.615908+0000","last_became_active":"2024-01-15T18:10:21.612624+0000","last_became_peered":"2024-01-15T18:10:21.612624+0000","last_unstale":"2024-01-15T18:10:21.615908+0000","last_undegraded":"2024-01-15T18:10:21.615908+0000","last_fullsized":"2024-01-15T18:10:21.615908+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:28.761075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.598336+0000","last_change":"2024-01-15T18:10:21.598336+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.598336+0000","last_undegraded":"2024-01-15T18:10:21.598336+0000","last_fullsized":"2024-01-15T18:10:21.598336+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623853+0000","last_change":"2024-01-15T18:10:21.623180+0000","last_active":"2024-01-15T18:10:21.623853+0000","last_peered":"2024-01-15T18:10:21.623853+0000","last_clean":"2024-01-15T18:10:21.623853+0000","last_became_active":"2024-01-15T18:10:21.622427+0000","last_became_peered":"2024-01-15T18:10:21.622427+0000","last_unstale":"2024-01-15T18:10:21.623853+0000","last_undegraded":"2024-01-15T18:10:21.623853+0000","last_fullsized":"2024-01-15T18:10:21.623853+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.176712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.614635+0000","last_change":"2024-01-15T18:10:21.614635+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.614635+0000","last_undegraded":"2024-01-15T18:10:21.614635+0000","last_fullsized":"2024-01-15T18:10:21.614635+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.620216+0000","last_change":"2024-01-15T18:10:21.617457+0000","last_active":"2024-01-15T18:10:21.620216+0000","last_peered":"2024-01-15T18:10:21.620216+0000","last_clean":"2024-01-15T18:10:21.620216+0000","last_became_active":"2024-01-15T18:10:21.617193+0000","last_became_peered":"2024-01-15T18:10:21.617193+0000","last_unstale":"2024-01-15T18:10:21.620216+0000","last_undegraded":"2024-01-15T18:10:21.620216+0000","last_fullsized":"2024-01-15T18:10:21.620216+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.598031+0000","last_change":"2024-01-15T18:10:21.598031+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.598031+0000","last_undegraded":"2024-01-15T18:10:21.598031+0000","last_fullsized":"2024-01-15T18:10:21.598031+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622228+0000","last_change":"2024-01-15T18:10:21.620555+0000","last_active":"2024-01-15T18:10:21.622228+0000","last_peered":"2024-01-15T18:10:21.622228+0000","last_clean":"2024-01-15T18:10:21.622228+0000","last_became_active":"2024-01-15T18:10:21.620232+0000","last_became_peered":"2024-01-15T18:10:21.620232+0000","last_unstale":"2024-01-15T18:10:21.622228+0000","last_undegraded":"2024-01-15T18:10:21.622228+0000","last_fullsized":"2024-01-15T18:10:21.622228+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:45.631921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597549+0000","last_change":"2024-01-15T18:10:21.597549+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597549+0000","last_undegraded":"2024-01-15T18:10:21.597549+0000","last_fullsized":"2024-01-15T18:10:21.597549+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.621611+0000","last_change":"2024-01-15T18:10:21.618350+0000","last_active":"2024-01-15T18:10:21.621611+0000","last_peered":"2024-01-15T18:10:21.621611+0000","last_clean":"2024-01-15T18:10:21.621611+0000","last_became_active":"2024-01-15T18:10:21.618029+0000","last_became_peered":"2024-01-15T18:10:21.618029+0000","last_unstale":"2024-01-15T18:10:21.621611+0000","last_undegraded":"2024-01-15T18:10:21.621611+0000","last_fullsized":"2024-01-15T18:10:21.621611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:04.305445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608054+0000","last_change":"2024-01-15T18:10:21.608054+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608054+0000","last_undegraded":"2024-01-15T18:10:21.608054+0000","last_fullsized":"2024-01-15T18:10:21.608054+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623404+0000","last_change":"2024-01-15T18:10:21.622418+0000","last_active":"2024-01-15T18:10:21.623404+0000","last_peered":"2024-01-15T18:10:21.623404+0000","last_clean":"2024-01-15T18:10:21.623404+0000","last_became_active":"2024-01-15T18:10:21.621566+0000","last_became_peered":"2024-01-15T18:10:21.621566+0000","last_unstale":"2024-01-15T18:10:21.623404+0000","last_undegraded":"2024-01-15T18:10:21.623404+0000","last_fullsized":"2024-01-15T18:10:21.623404+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:15.951926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601789+0000","last_change":"2024-01-15T18:10:21.601789+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601789+0000","last_undegraded":"2024-01-15T18:10:21.601789+0000","last_fullsized":"2024-01-15T18:10:21.601789+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617178+0000","last_change":"2024-01-15T18:10:21.613977+0000","last_active":"2024-01-15T18:10:21.617178+0000","last_peered":"2024-01-15T18:10:21.617178+0000","last_clean":"2024-01-15T18:10:21.617178+0000","last_became_active":"2024-01-15T18:10:21.613679+0000","last_became_peered":"2024-01-15T18:10:21.613679+0000","last_unstale":"2024-01-15T18:10:21.617178+0000","last_undegraded":"2024-01-15T18:10:21.617178+0000","last_fullsized":"2024-01-15T18:10:21.617178+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:16:05.935030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.595658+0000","last_change":"2024-01-15T18:10:21.595658+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.595658+0000","last_undegraded":"2024-01-15T18:10:21.595658+0000","last_fullsized":"2024-01-15T18:10:21.595658+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.621153+0000","last_change":"2024-01-15T18:10:21.618794+0000","last_active":"2024-01-15T18:10:21.621153+0000","last_peered":"2024-01-15T18:10:21.621153+0000","last_clean":"2024-01-15T18:10:21.621153+0000","last_became_active":"2024-01-15T18:10:21.618479+0000","last_became_peered":"2024-01-15T18:10:21.618479+0000","last_unstale":"2024-01-15T18:10:21.621153+0000","last_undegraded":"2024-01-15T18:10:21.621153+0000","last_fullsized":"2024-01-15T18:10:21.621153+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601227+0000","last_change":"2024-01-15T18:10:21.601227+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601227+0000","last_undegraded":"2024-01-15T18:10:21.601227+0000","last_fullsized":"2024-01-15T18:10:21.601227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.615155+0000","last_change":"2024-01-15T18:10:21.608435+0000","last_active":"2024-01-15T18:10:21.615155+0000","last_peered":"2024-01-15T18:10:21.615155+0000","last_clean":"2024-01-15T18:10:21.615155+0000","last_became_active":"2024-01-15T18:10:21.608053+0000","last_became_peered":"2024-01-15T18:10:21.608053+0000","last_unstale":"2024-01-15T18:10:21.615155+0000","last_undegraded":"2024-01-15T18:10:21.615155+0000","last_fullsized":"2024-01-15T18:10:21.615155+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.612014+0000","last_change":"2024-01-15T18:10:21.612014+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.612014+0000","last_undegraded":"2024-01-15T18:10:21.612014+0000","last_fullsized":"2024-01-15T18:10:21.612014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623513+0000","last_change":"2024-01-15T18:10:21.620384+0000","last_active":"2024-01-15T18:10:21.623513+0000","last_peered":"2024-01-15T18:10:21.623513+0000","last_clean":"2024-01-15T18:10:21.623513+0000","last_became_active":"2024-01-15T18:10:21.617541+0000","last_became_peered":"2024-01-15T18:10:21.617541+0000","last_unstale":"2024-01-15T18:10:21.623513+0000","last_undegraded":"2024-01-15T18:10:21.623513+0000","last_fullsized":"2024-01-15T18:10:21.623513+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599884+0000","last_change":"2024-01-15T18:10:21.599884+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599884+0000","last_undegraded":"2024-01-15T18:10:21.599884+0000","last_fullsized":"2024-01-15T18:10:21.599884+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.624202+0000","last_change":"2024-01-15T18:10:21.620621+0000","last_active":"2024-01-15T18:10:21.624202+0000","last_peered":"2024-01-15T18:10:21.624202+0000","last_clean":"2024-01-15T18:10:21.624202+0000","last_became_active":"2024-01-15T18:10:21.620314+0000","last_became_peered":"2024-01-15T18:10:21.620314+0000","last_unstale":"2024-01-15T18:10:21.624202+0000","last_undegraded":"2024-01-15T18:10:21.624202+0000","last_fullsized":"2024-01-15T18:10:21.624202+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.362159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597254+0000","last_change":"2024-01-15T18:10:21.597254+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597254+0000","last_undegraded":"2024-01-15T18:10:21.597254+0000","last_fullsized":"2024-01-15T18:10:21.597254+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617327+0000","last_change":"2024-01-15T18:10:21.614065+0000","last_active":"2024-01-15T18:10:21.617327+0000","last_peered":"2024-01-15T18:10:21.617327+0000","last_clean":"2024-01-15T18:10:21.617327+0000","last_became_active":"2024-01-15T18:10:21.613749+0000","last_became_peered":"2024-01-15T18:10:21.613749+0000","last_unstale":"2024-01-15T18:10:21.617327+0000","last_undegraded":"2024-01-15T18:10:21.617327+0000","last_fullsized":"2024-01-15T18:10:21.617327+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:30.543941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600363+0000","last_change":"2024-01-15T18:10:21.600363+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600363+0000","last_undegraded":"2024-01-15T18:10:21.600363+0000","last_fullsized":"2024-01-15T18:10:21.600363+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.612587+0000","last_change":"2024-01-15T18:10:21.608384+0000","last_active":"2024-01-15T18:10:21.612587+0000","last_peered":"2024-01-15T18:10:21.612587+0000","last_clean":"2024-01-15T18:10:21.612587+0000","last_became_active":"2024-01-15T18:10:21.607947+0000","last_became_peered":"2024-01-15T18:10:21.607947+0000","last_unstale":"2024-01-15T18:10:21.612587+0000","last_undegraded":"2024-01-15T18:10:21.612587+0000","last_fullsized":"2024-01-15T18:10:21.612587+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:12:47.963640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.595935+0000","last_change":"2024-01-15T18:10:21.595935+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.595935+0000","last_undegraded":"2024-01-15T18:10:21.595935+0000","last_fullsized":"2024-01-15T18:10:21.595935+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622275+0000","last_change":"2024-01-15T18:10:21.620127+0000","last_active":"2024-01-15T18:10:21.622275+0000","last_peered":"2024-01-15T18:10:21.622275+0000","last_clean":"2024-01-15T18:10:21.622275+0000","last_became_active":"2024-01-15T18:10:21.619856+0000","last_became_peered":"2024-01-15T18:10:21.619856+0000","last_unstale":"2024-01-15T18:10:21.622275+0000","last_undegraded":"2024-01-15T18:10:21.622275+0000","last_fullsized":"2024-01-15T18:10:21.622275+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:31:29.108419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.605332+0000","last_change":"2024-01-15T18:10:21.605332+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.605332+0000","last_undegraded":"2024-01-15T18:10:21.605332+0000","last_fullsized":"2024-01-15T18:10:21.605332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622081+0000","last_change":"2024-01-15T18:10:21.619938+0000","last_active":"2024-01-15T18:10:21.622081+0000","last_peered":"2024-01-15T18:10:21.622081+0000","last_clean":"2024-01-15T18:10:21.622081+0000","last_became_active":"2024-01-15T18:10:21.618751+0000","last_became_peered":"2024-01-15T18:10:21.618751+0000","last_unstale":"2024-01-15T18:10:21.622081+0000","last_undegraded":"2024-01-15T18:10:21.622081+0000","last_fullsized":"2024-01-15T18:10:21.622081+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.362159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.610825+0000","last_change":"2024-01-15T18:10:21.610825+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.610825+0000","last_undegraded":"2024-01-15T18:10:21.610825+0000","last_fullsized":"2024-01-15T18:10:21.610825+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622264+0000","last_change":"2024-01-15T18:10:21.618976+0000","last_active":"2024-01-15T18:10:21.622264+0000","last_peered":"2024-01-15T18:10:21.622264+0000","last_clean":"2024-01-15T18:10:21.622264+0000","last_became_active":"2024-01-15T18:10:21.618682+0000","last_became_peered":"2024-01-15T18:10:21.618682+0000","last_unstale":"2024-01-15T18:10:21.622264+0000","last_undegraded":"2024-01-15T18:10:21.622264+0000","last_fullsized":"2024-01-15T18:10:21.622264+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:12:47.963640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.603686+0000","last_change":"2024-01-15T18:10:21.603686+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.603686+0000","last_undegraded":"2024-01-15T18:10:21.603686+0000","last_fullsized":"2024-01-15T18:10:21.603686+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.624207+0000","last_change":"2024-01-15T18:10:21.623001+0000","last_active":"2024-01-15T18:10:21.624207+0000","last_peered":"2024-01-15T18:10:21.624207+0000","last_clean":"2024-01-15T18:10:21.624207+0000","last_became_active":"2024-01-15T18:10:21.622691+0000","last_became_peered":"2024-01-15T18:10:21.622691+0000","last_unstale":"2024-01-15T18:10:21.624207+0000","last_undegraded":"2024-01-15T18:10:21.624207+0000","last_fullsized":"2024-01-15T18:10:21.624207+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:30.543941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601674+0000","last_change":"2024-01-15T18:10:21.601674+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601674+0000","last_undegraded":"2024-01-15T18:10:21.601674+0000","last_fullsized":"2024-01-15T18:10:21.601674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.620715+0000","last_change":"2024-01-15T18:10:21.618463+0000","last_active":"2024-01-15T18:10:21.620715+0000","last_peered":"2024-01-15T18:10:21.620715+0000","last_clean":"2024-01-15T18:10:21.620715+0000","last_became_active":"2024-01-15T18:10:21.618150+0000","last_became_peered":"2024-01-15T18:10:21.618150+0000","last_unstale":"2024-01-15T18:10:21.620715+0000","last_undegraded":"2024-01-15T18:10:21.620715+0000","last_fullsized":"2024-01-15T18:10:21.620715+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:15.951926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599443+0000","last_change":"2024-01-15T18:10:21.599443+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599443+0000","last_undegraded":"2024-01-15T18:10:21.599443+0000","last_fullsized":"2024-01-15T18:10:21.599443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617800+0000","last_change":"2024-01-15T18:10:21.613099+0000","last_active":"2024-01-15T18:10:21.617800+0000","last_peered":"2024-01-15T18:10:21.617800+0000","last_clean":"2024-01-15T18:10:21.617800+0000","last_became_active":"2024-01-15T18:10:21.612818+0000","last_became_peered":"2024-01-15T18:10:21.612818+0000","last_unstale":"2024-01-15T18:10:21.617800+0000","last_undegraded":"2024-01-15T18:10:21.617800+0000","last_fullsized":"2024-01-15T18:10:21.617800+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:34.484071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611680+0000","last_change":"2024-01-15T18:10:21.611680+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611680+0000","last_undegraded":"2024-01-15T18:10:21.611680+0000","last_fullsized":"2024-01-15T18:10:21.611680+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.614848+0000","last_change":"2024-01-15T18:10:21.610659+0000","last_active":"2024-01-15T18:10:21.614848+0000","last_peered":"2024-01-15T18:10:21.614848+0000","last_clean":"2024-01-15T18:10:21.614848+0000","last_became_active":"2024-01-15T18:10:21.610364+0000","last_became_peered":"2024-01-15T18:10:21.610364+0000","last_unstale":"2024-01-15T18:10:21.614848+0000","last_undegraded":"2024-01-15T18:10:21.614848+0000","last_fullsized":"2024-01-15T18:10:21.614848+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599272+0000","last_change":"2024-01-15T18:10:21.599272+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599272+0000","last_undegraded":"2024-01-15T18:10:21.599272+0000","last_fullsized":"2024-01-15T18:10:21.599272+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.616668+0000","last_change":"2024-01-15T18:10:21.613230+0000","last_active":"2024-01-15T18:10:21.616668+0000","last_peered":"2024-01-15T18:10:21.616668+0000","last_clean":"2024-01-15T18:10:21.616668+0000","last_became_active":"2024-01-15T18:10:21.612943+0000","last_became_peered":"2024-01-15T18:10:21.612943+0000","last_unstale":"2024-01-15T18:10:21.616668+0000","last_undegraded":"2024-01-15T18:10:21.616668+0000","last_fullsized":"2024-01-15T18:10:21.616668+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:52:25.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.594607+0000","last_change":"2024-01-15T18:10:21.594607+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.594607+0000","last_undegraded":"2024-01-15T18:10:21.594607+0000","last_fullsized":"2024-01-15T18:10:21.594607+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623244+0000","last_change":"2024-01-15T18:10:21.621660+0000","last_active":"2024-01-15T18:10:21.623244+0000","last_peered":"2024-01-15T18:10:21.623244+0000","last_clean":"2024-01-15T18:10:21.623244+0000","last_became_active":"2024-01-15T18:10:21.620662+0000","last_became_peered":"2024-01-15T18:10:21.620662+0000","last_unstale":"2024-01-15T18:10:21.623244+0000","last_undegraded":"2024-01-15T18:10:21.623244+0000","last_fullsized":"2024-01-15T18:10:21.623244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:25.948319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597443+0000","last_change":"2024-01-15T18:10:21.597443+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597443+0000","last_undegraded":"2024-01-15T18:10:21.597443+0000","last_fullsized":"2024-01-15T18:10:21.597443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.625143+0000","last_change":"2024-01-15T18:10:21.624625+0000","last_active":"2024-01-15T18:10:21.625143+0000","last_peered":"2024-01-15T18:10:21.625143+0000","last_clean":"2024-01-15T18:10:21.625143+0000","last_became_active":"2024-01-15T18:10:21.624271+0000","last_became_peered":"2024-01-15T18:10:21.624271+0000","last_unstale":"2024-01-15T18:10:21.625143+0000","last_undegraded":"2024-01-15T18:10:21.625143+0000","last_fullsized":"2024-01-15T18:10:21.625143+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:55.724175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":233,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:10:20.590380+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:10:20.590380+0000","last_peered":"2024-01-15T18:10:20.590380+0000","last_clean":"2024-01-15T18:10:20.590380+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:10:20.590380+0000","last_undegraded":"2024-01-15T18:10:20.590380+0000","last_fullsized":"2024-01-15T18:10:20.590380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542332,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81464,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290376,"statfs":{"total":96636764160,"available":96553345024,"internally_reserved":0,"allocated":1814528,"data_stored":1117561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":81590730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3280000000000001,"15min":1.3740000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.40400000000000003},"max":{"1min":1.742,"5min":11.449999999999999,"15min":19.843},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.3759999999999999,"15min":1.629},"min":{"1min":0.89200000000000002,"5min":0.61499999999999999,"15min":0.53400000000000003},"max":{"1min":7.8250000000000002,"5min":7.8250000000000002,"15min":38.137},"last":1.26}]},{"osd":1,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.319,"15min":1.411},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.45600000000000002},"max":{"1min":1.577,"5min":11.51,"15min":19.917000000000002},"last":1.046},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.4079999999999999,"15min":1.605},"min":{"1min":0.85699999999999998,"5min":0.81100000000000005,"15min":0.42999999999999999},"max":{"1min":1.841,"5min":6.7480000000000002,"15min":51.843000000000004},"last":0.97499999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.371,"15min":1.425},"min":{"1min":0.97199999999999998,"5min":0.67200000000000004,"15min":0.437},"max":{"1min":3.3900000000000001,"5min":12.061,"15min":12.061},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.4430000000000001,"15min":1.532},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.47399999999999998},"max":{"1min":1.806,"5min":11.66,"15min":13.478999999999999},"last":1.5700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.544},"min":{"1min":0.85299999999999998,"5min":0.83199999999999996,"15min":0.47899999999999998},"max":{"1min":1.9079999999999999,"5min":7.4669999999999996,"15min":16.632000000000001},"last":1.363},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.5069999999999999,"15min":1.6160000000000001},"min":{"1min":0.97299999999999998,"5min":0.94199999999999995,"15min":0.56699999999999995},"max":{"1min":1.8520000000000001,"5min":11.611000000000001,"15min":13.439},"last":1.1679999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":2.319,"5min":1.728,"15min":1.7589999999999999},"min":{"1min":1.0169999999999999,"5min":0.81000000000000005,"15min":0.50700000000000001},"max":{"1min":17.067,"5min":23.042000000000002,"15min":23.042000000000002},"last":1.337},{"interface":"front","average":{"1min":1.425,"5min":1.5269999999999999,"15min":1.7270000000000001},"min":{"1min":0.94599999999999995,"5min":0.78800000000000003,"15min":0.51800000000000002},"max":{"1min":2.0779999999999998,"5min":16.867999999999999,"15min":38.682000000000002},"last":1.534}]},{"osd":5,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5640000000000001,"15min":1.8089999999999999},"min":{"1min":1.0329999999999999,"5min":0.89800000000000002,"15min":0.52700000000000002},"max":{"1min":1.843,"5min":11.696999999999999,"15min":25.006},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.7110000000000001,"15min":1.8400000000000001},"min":{"1min":1.0880000000000001,"5min":0.97499999999999998,"15min":0.624},"max":{"1min":2.04,"5min":22.933,"15min":22.933},"last":1.591}]},{"osd":6,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.9850000000000001,"15min":2.0419999999999998},"min":{"1min":1.1870000000000001,"5min":1.123,"15min":0.59299999999999997},"max":{"1min":2.2229999999999999,"5min":23.004000000000001,"15min":23.834},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.607,"15min":1.998},"min":{"1min":1.0680000000000001,"5min":0.84099999999999997,"15min":0.58299999999999996},"max":{"1min":2.1539999999999999,"5min":7.5220000000000002,"15min":63.932000000000002},"last":1.7490000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542331,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80884,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290956,"statfs":{"total":96636764160,"available":96553938944,"internally_reserved":0,"allocated":1220608,"data_stored":527193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81590726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.825,"5min":1.512,"15min":1.4139999999999999},"min":{"1min":0.79100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"max":{"1min":14.794,"5min":14.794,"15min":14.794},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.345,"15min":1.407},"min":{"1min":0.80000000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.5430000000000001,"5min":4.5430000000000001,"15min":14.013},"last":1.8160000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.3660000000000001,"15min":1.353},"min":{"1min":0.67900000000000005,"5min":0.63700000000000001,"15min":0.55100000000000005},"max":{"1min":15.064,"5min":15.064,"15min":15.064},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.508,"15min":1.53},"min":{"1min":0.85299999999999998,"5min":0.74199999999999999,"15min":0.66200000000000003},"max":{"1min":1.6899999999999999,"5min":23.495999999999999,"15min":23.495999999999999},"last":1.4339999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.667,"15min":1.587},"min":{"1min":0.81000000000000005,"5min":0.73799999999999999,"15min":0.59399999999999997},"max":{"1min":1.7490000000000001,"5min":23.087,"15min":23.087},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.871,"5min":1.4910000000000001,"15min":1.4590000000000001},"min":{"1min":0.86499999999999999,"5min":0.51400000000000001,"15min":0.504},"max":{"1min":14.94,"5min":14.94,"15min":14.94},"last":0.99199999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9750000000000001,"5min":1.7629999999999999,"15min":1.6950000000000001},"min":{"1min":0.95299999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":15.151999999999999,"5min":15.151999999999999,"15min":15.151999999999999},"last":1.244},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5580000000000001,"15min":1.601},"min":{"1min":0.89600000000000002,"5min":0.76100000000000001,"15min":0.70699999999999996},"max":{"1min":4.6849999999999996,"5min":13.708,"15min":14.066000000000001},"last":1.9059999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.577,"15min":1.661},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.51800000000000002},"max":{"1min":1.8720000000000001,"5min":23.327000000000002,"15min":23.327000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":2.4329999999999998,"5min":1.71,"15min":1.716},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.45800000000000002},"max":{"1min":15.239000000000001,"5min":15.239000000000001,"15min":17.289000000000001},"last":1.375}]},{"osd":5,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.4039999999999999,"15min":1.5669999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51600000000000001},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":14.176},"last":1.702},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.8580000000000001,"15min":1.8700000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.54700000000000004},"max":{"1min":2.044,"5min":23.417999999999999,"15min":23.417999999999999},"last":1.5289999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.716,"15min":1.7689999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":4.8239999999999998,"5min":16.454000000000001,"15min":21.265000000000001},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.7370000000000001,"5min":2.0089999999999999,"15min":1.9950000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.59699999999999998},"max":{"1min":3.8879999999999999,"5min":23.738,"15min":23.738},"last":1.6160000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542331,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58752,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313088,"statfs":{"total":96636764160,"available":96576602112,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.234,"15min":1.2829999999999999},"min":{"1min":0.73399999999999999,"5min":0.54200000000000004,"15min":0.47799999999999998},"max":{"1min":2.2839999999999998,"5min":7.6580000000000004,"15min":19.266999999999999},"last":1.55},{"interface":"front","average":{"1min":1.254,"5min":1.196,"15min":1.353},"min":{"1min":0.82299999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.738,"5min":2.1869999999999998,"15min":17.184999999999999},"last":1.028}]},{"osd":1,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2450000000000001,"15min":1.339},"min":{"1min":0.747,"5min":0.60699999999999998,"15min":0.499},"max":{"1min":2.355,"5min":5.3879999999999999,"15min":16.297000000000001},"last":1.472},{"interface":"front","average":{"1min":1.23,"5min":1.359,"15min":1.3280000000000001},"min":{"1min":0.78900000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8580000000000001,"5min":7.7649999999999997,"15min":9.2379999999999995},"last":1.3799999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.228,"15min":1.26},"min":{"1min":0.80400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.5429999999999999,"5min":2.1110000000000002,"15min":8.1820000000000004},"last":0.876},{"interface":"front","average":{"1min":1.431,"5min":1.349,"15min":1.4330000000000001},"min":{"1min":0.97299999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":16.158999999999999},"last":1.6359999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.427,"15min":1.486},"min":{"1min":0.89400000000000002,"5min":0.63700000000000001,"15min":0.56100000000000005},"max":{"1min":2.5179999999999998,"5min":7.8609999999999998,"15min":14.914},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.446,"15min":1.4550000000000001},"min":{"1min":1.0189999999999999,"5min":0.64600000000000002,"15min":0.63300000000000001},"max":{"1min":1.8109999999999999,"5min":2.0139999999999998,"15min":7.0110000000000001},"last":1.139}]},{"osd":4,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.329,"15min":1.5},"min":{"1min":0.80800000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":16.414999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.51,"15min":1.6319999999999999},"min":{"1min":0.56200000000000006,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.665,"5min":7.5359999999999996,"15min":17.991},"last":1.8500000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.45,"15min":1.5009999999999999},"min":{"1min":0.72699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.881,"5min":3.7559999999999998,"15min":6.4630000000000001},"last":1.244},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4830000000000001,"15min":1.599},"min":{"1min":1.0009999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":16.364000000000001},"last":1.7949999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.669,"15min":1.6220000000000001},"min":{"1min":0.78900000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.399,"5min":7.96,"15min":9.0600000000000005},"last":1.766},{"interface":"front","average":{"1min":1.587,"5min":1.6000000000000001,"15min":1.712},"min":{"1min":0.98599999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":19.356999999999999},"last":1.347}]}]},{"osd":4,"up_from":14,"seq":60129542331,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76736,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295104,"statfs":{"total":96636764160,"available":96558186496,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.286,"15min":1.341},"min":{"1min":0.70999999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.8170000000000002,"5min":3.8170000000000002,"15min":11.413},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3420000000000001,"15min":1.3919999999999999},"min":{"1min":0.72599999999999998,"5min":0.53800000000000003,"15min":0.53700000000000003},"max":{"1min":2.7919999999999998,"5min":3.726,"15min":9.7439999999999998},"last":1.5740000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.341,"15min":1.3660000000000001},"min":{"1min":0.73399999999999999,"5min":0.58899999999999997,"15min":0.45000000000000001},"max":{"1min":2.8700000000000001,"5min":11.128,"15min":11.987},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2330000000000001,"15min":1.3089999999999999},"min":{"1min":0.81100000000000005,"5min":0.69499999999999995,"15min":0.55300000000000005},"max":{"1min":1.909,"5min":3.3610000000000002,"15min":16.058},"last":1.397}]},{"osd":2,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3029999999999999,"15min":1.496},"min":{"1min":0.81299999999999994,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.6070000000000002,"5min":5.0960000000000001,"15min":13.865},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.401,"15min":1.5229999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.69499999999999995},"max":{"1min":1.899,"5min":5.1340000000000003,"15min":13.903},"last":1.325}]},{"osd":3,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.4430000000000001,"15min":1.6080000000000001},"min":{"1min":0.85999999999999999,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":3.8999999999999999,"5min":10.765000000000001,"15min":26.055},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.542,"5min":1.3500000000000001,"15min":1.4910000000000001},"min":{"1min":0.76700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":13.25},"last":1.466}]},{"osd":5,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.331,"15min":1.5700000000000001},"min":{"1min":0.86799999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":29.422000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.5329999999999999,"15min":1.538},"min":{"1min":0.85499999999999998,"5min":0.62,"15min":0.42199999999999999},"max":{"1min":3.0139999999999998,"5min":12.332000000000001,"15min":12.332000000000001},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.498,"15min":1.546},"min":{"1min":0.97399999999999998,"5min":0.59699999999999998,"15min":0.45200000000000001},"max":{"1min":1.9590000000000001,"5min":11.09,"15min":18.260000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.744,"5min":1.6299999999999999,"15min":1.696},"min":{"1min":1.1519999999999999,"5min":0.78700000000000003,"15min":0.48099999999999998},"max":{"1min":3.968,"5min":9.8740000000000006,"15min":11.657999999999999},"last":1.365}]},{"osd":7,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.6739999999999999,"15min":1.788},"min":{"1min":1.133,"5min":0.68200000000000005,"15min":0.503},"max":{"1min":3.0680000000000001,"5min":12.247,"15min":12.247},"last":1.125},{"interface":"front","average":{"1min":1.857,"5min":1.756,"15min":1.8660000000000001},"min":{"1min":1.3839999999999999,"5min":0.81899999999999995,"15min":0.53600000000000003},"max":{"1min":4.0209999999999999,"5min":12.311999999999999,"15min":12.311999999999999},"last":1.5189999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542333,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72580,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299260,"statfs":{"total":96636764160,"available":96562442240,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.113,"15min":1.2210000000000001},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.33300000000000002},"max":{"1min":1.284,"5min":9.2899999999999991,"15min":18.399999999999999},"last":1.304},{"interface":"front","average":{"1min":1.048,"5min":1.127,"15min":1.387},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":1.8140000000000001,"5min":2.23,"15min":21.047000000000001},"last":1.137}]},{"osd":1,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.92400000000000004,"15min":1.272},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.415,"5min":3.0409999999999999,"15min":17.777999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.3680000000000001,"15min":1.288},"min":{"1min":0.57399999999999995,"5min":0.46300000000000002,"15min":0.39500000000000002},"max":{"1min":1.6890000000000001,"5min":11.228999999999999,"15min":11.228999999999999},"last":0.77900000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.224,"15min":1.2789999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.518,"5min":2.7519999999999998,"15min":10.964},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.5489999999999999,"15min":1.554},"min":{"1min":0.60799999999999998,"5min":0.57099999999999995,"15min":0.54000000000000004},"max":{"1min":1.857,"5min":9.4920000000000009,"15min":20.495000000000001},"last":0.72399999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.401,"15min":1.5589999999999999},"min":{"1min":0.871,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.6419999999999999,"5min":6.319,"15min":13.536},"last":1.395},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.456,"15min":1.6399999999999999},"min":{"1min":0.748,"5min":0.63700000000000001,"15min":0.54600000000000004},"max":{"1min":1.736,"5min":16.398,"15min":16.398},"last":1.3420000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.681,"15min":1.6950000000000001},"min":{"1min":1.046,"5min":0.875,"15min":0.57299999999999995},"max":{"1min":1.6819999999999999,"5min":9.5690000000000008,"15min":20.606000000000002},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5589999999999999,"15min":1.665},"min":{"1min":0.97299999999999998,"5min":0.70199999999999996,"15min":0.68000000000000005},"max":{"1min":1.758,"5min":5.6980000000000004,"15min":11.603},"last":1.637}]},{"osd":6,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.615,"15min":1.7949999999999999},"min":{"1min":0.81000000000000005,"5min":0.61399999999999999,"15min":0.53500000000000003},"max":{"1min":2.0870000000000002,"5min":13.648999999999999,"15min":13.648999999999999},"last":1.25},{"interface":"front","average":{"1min":1.502,"5min":1.8109999999999999,"15min":1.8220000000000001},"min":{"1min":0.98699999999999999,"5min":0.78300000000000003,"15min":0.65800000000000003},"max":{"1min":2.0249999999999999,"5min":13.679,"15min":20.556000000000001},"last":1.093}]},{"osd":7,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.552,"15min":1.8480000000000001},"min":{"1min":0.78800000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8260000000000001,"5min":3.5590000000000002,"15min":28.843},"last":1.528},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.5760000000000001,"15min":1.857},"min":{"1min":0.97999999999999998,"5min":0.72799999999999998,"15min":0.67200000000000004},"max":{"1min":1.968,"5min":9.4100000000000001,"15min":15.143000000000001},"last":1.4570000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68420,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303420,"statfs":{"total":96636764160,"available":96566702080,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.129,"15min":1.095},"min":{"1min":0.68300000000000005,"5min":0.32100000000000001,"15min":0.26600000000000001},"max":{"1min":2.9359999999999999,"5min":12.278,"15min":14.427},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.159,"5min":1.4970000000000001,"15min":1.337},"min":{"1min":0.68799999999999994,"5min":0.50900000000000001,"15min":0.39300000000000002},"max":{"1min":2.25,"5min":15.314,"15min":15.314},"last":1.399}]},{"osd":1,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.5980000000000001,"15min":1.4470000000000001},"min":{"1min":0.80700000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.7029999999999998,"5min":34.463999999999999,"15min":34.463999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.351,"5min":1.7989999999999999,"15min":1.454},"min":{"1min":0.76800000000000002,"5min":0.54200000000000004,"15min":0.373},"max":{"1min":1.881,"5min":25.859999999999999,"15min":25.859999999999999},"last":1.2729999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.9430000000000001,"5min":1.26,"15min":1.4179999999999999},"min":{"1min":0.76200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":19.535},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.804,"15min":1.6240000000000001},"min":{"1min":0.95999999999999996,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":2.8050000000000002,"5min":22.783999999999999,"15min":22.783999999999999},"last":1.03}]},{"osd":4,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.603,"15min":1.784},"min":{"1min":1.113,"5min":0.73199999999999998,"15min":0.55700000000000005},"max":{"1min":2.8759999999999999,"5min":12.728,"15min":29.986999999999998},"last":1.155},{"interface":"front","average":{"1min":1.607,"5min":1.7609999999999999,"15min":1.742},"min":{"1min":1.0449999999999999,"5min":0.71299999999999997,"15min":0.56799999999999995},"max":{"1min":2.286,"5min":15.553000000000001,"15min":21.199999999999999},"last":1.679}]},{"osd":5,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":2.9350000000000001,"5min":1.855,"15min":1.746},"min":{"1min":0.84299999999999997,"5min":0.63700000000000001,"15min":0.51500000000000001},"max":{"1min":14.066000000000001,"5min":14.066000000000001,"15min":19.681999999999999},"last":1.089},{"interface":"front","average":{"1min":2.5379999999999998,"5min":2.0179999999999998,"15min":1.823},"min":{"1min":1.2010000000000001,"5min":0.84799999999999998,"15min":0.60899999999999999},"max":{"1min":14.025,"5min":14.025,"15min":14.702999999999999},"last":1.3600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":2.032,"15min":1.7869999999999999},"min":{"1min":1.1599999999999999,"5min":0.76800000000000002,"15min":0.68000000000000005},"max":{"1min":2.2919999999999998,"5min":15.683999999999999,"15min":15.683999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":2.2999999999999998,"5min":2.0819999999999999,"15min":1.853},"min":{"1min":0.96999999999999997,"5min":0.71999999999999997,"15min":0.58899999999999997},"max":{"1min":12.629,"5min":12.629,"15min":19.783000000000001},"last":1.2410000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":2.0129999999999999,"15min":2.0710000000000002},"min":{"1min":1.4510000000000001,"5min":0.77800000000000002,"15min":0.63100000000000001},"max":{"1min":2.3279999999999998,"5min":17.099,"15min":57.006999999999998},"last":1.597},{"interface":"front","average":{"1min":2.3919999999999999,"5min":2.1600000000000001,"15min":2.1930000000000001},"min":{"1min":1.081,"5min":1.0089999999999999,"15min":0.56100000000000005},"max":{"1min":12.754,"5min":22.853999999999999,"15min":22.853999999999999},"last":1.621}]}]},{"osd":1,"up_from":14,"seq":60129542332,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72580,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299260,"statfs":{"total":96636764160,"available":96562442240,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.363,"15min":1.2070000000000001},"min":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.34399999999999997},"max":{"1min":2.3839999999999999,"5min":30.460999999999999,"15min":30.460999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1759999999999999,"15min":1.117},"min":{"1min":0.61499999999999999,"5min":0.38700000000000001,"15min":0.314},"max":{"1min":2.0960000000000001,"5min":4.4059999999999997,"15min":18.658000000000001},"last":1.2310000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.198,"15min":1.1970000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":2.0619999999999998,"5min":7.117,"15min":21.126000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.752,"15min":1.5620000000000001},"min":{"1min":0.75700000000000001,"5min":0.70899999999999996,"15min":0.48699999999999999},"max":{"1min":2.1499999999999999,"5min":30.593,"15min":30.593},"last":1.042}]},{"osd":3,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.6719999999999999,"15min":1.617},"min":{"1min":0.76100000000000001,"5min":0.58099999999999996,"15min":0.48599999999999999},"max":{"1min":2.0579999999999998,"5min":30.547000000000001,"15min":30.547000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3480000000000001,"15min":1.3420000000000001},"min":{"1min":0.84999999999999998,"5min":0.52700000000000002,"15min":0.437},"max":{"1min":2.21,"5min":5.4729999999999999,"15min":15.510999999999999},"last":1.4730000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.663,"15min":1.6990000000000001},"min":{"1min":1.111,"5min":0.75900000000000001,"15min":0.72499999999999998},"max":{"1min":2.7599999999999998,"5min":8.8170000000000002,"15min":14.827},"last":1.585},{"interface":"front","average":{"1min":1.53,"5min":1.5429999999999999,"15min":1.657},"min":{"1min":0.95699999999999996,"5min":0.72999999999999998,"15min":0.69299999999999995},"max":{"1min":2.359,"5min":3.7440000000000002,"15min":14.851000000000001},"last":1.619}]},{"osd":5,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5349999999999999,"15min":1.6160000000000001},"min":{"1min":1.036,"5min":0.77600000000000002,"15min":0.59599999999999997},"max":{"1min":2.266,"5min":3.798,"15min":13.228999999999999},"last":1.329},{"interface":"front","average":{"1min":1.593,"5min":1.986,"15min":1.9450000000000001},"min":{"1min":1.0429999999999999,"5min":0.78700000000000003,"15min":0.68300000000000005},"max":{"1min":2.2949999999999999,"5min":34.042999999999999,"15min":34.042999999999999},"last":1.375}]},{"osd":6,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.579,"15min":1.7829999999999999},"min":{"1min":0.90100000000000002,"5min":0.83899999999999997,"15min":0.64100000000000001},"max":{"1min":2.5550000000000002,"5min":3.5539999999999998,"15min":34.158000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.736,"15min":1.6299999999999999},"min":{"1min":1.115,"5min":0.67900000000000005,"15min":0.62},"max":{"1min":4.1699999999999999,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.6779999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":2.0979999999999999,"15min":1.8660000000000001},"min":{"1min":0.95999999999999996,"5min":0.77300000000000002,"15min":0.64600000000000002},"max":{"1min":2.5750000000000002,"5min":33.659999999999997,"15min":33.659999999999997},"last":1.73},{"interface":"front","average":{"1min":1.694,"5min":1.8540000000000001,"15min":1.8009999999999999},"min":{"1min":1.002,"5min":0.81000000000000005,"15min":0.60499999999999998},"max":{"1min":2.613,"5min":11.882999999999999,"15min":24.289999999999999},"last":1.7949999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542333,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69000,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302840,"statfs":{"total":96636764160,"available":96566108160,"internally_reserved":0,"allocated":1826816,"data_stored":1123892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2210000000000001,"15min":1.083},"min":{"1min":0.437,"5min":0.42799999999999999,"15min":0.312},"max":{"1min":2.8929999999999998,"5min":12.050000000000001,"15min":12.050000000000001},"last":1.212},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.3340000000000001},"min":{"1min":0.50900000000000001,"5min":0.49099999999999999,"15min":0.36199999999999999},"max":{"1min":2.9780000000000002,"5min":11.962,"15min":11.962},"last":1.169}]},{"osd":2,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.179,"15min":1.2090000000000001},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.27600000000000002},"max":{"1min":1.595,"5min":7.4560000000000004,"15min":11.59},"last":0.877},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.1459999999999999,"15min":1.167},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35899999999999999},"max":{"1min":1.5649999999999999,"5min":7.3239999999999998,"15min":14.512},"last":1.5649999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.425,"15min":1.4319999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.41699999999999998},"max":{"1min":1.5669999999999999,"5min":11.904,"15min":11.904},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1379999999999999,"15min":1.2490000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.29599999999999999},"max":{"1min":1.6850000000000001,"5min":3.202,"15min":14.74},"last":0.81299999999999994}]},{"osd":4,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.375,"15min":1.5900000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6140000000000001,"5min":7.4000000000000004,"15min":32.115000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.571,"15min":1.702},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.67700000000000005},"max":{"1min":1.8260000000000001,"5min":9.0790000000000006,"15min":15.975},"last":1.288}]},{"osd":5,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.556,"15min":1.722},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":1.6200000000000001,"5min":12.006,"15min":32.174999999999997},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.514,"15min":1.6359999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.442},"max":{"1min":1.8939999999999999,"5min":4.7119999999999997,"15min":17.225000000000001},"last":1.605}]},{"osd":6,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.7849999999999999,"15min":1.7989999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.70299999999999996},"max":{"1min":3.4399999999999999,"5min":24.039999999999999,"15min":24.039999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.7549999999999999,"15min":1.794},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.625},"max":{"1min":1.7450000000000001,"5min":18.143999999999998,"15min":32.216999999999999},"last":1.4490000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.853,"15min":2.1230000000000002},"min":{"1min":0.754,"5min":0.754,"15min":0.69099999999999995},"max":{"1min":1.7350000000000001,"5min":26.427,"15min":34.131},"last":1.504},{"interface":"front","average":{"1min":1.474,"5min":2.089,"15min":2.0379999999999998},"min":{"1min":0.753,"5min":0.753,"15min":0.61799999999999999},"max":{"1min":3.9790000000000001,"5min":44.694000000000003,"15min":44.694000000000003},"last":1.4850000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:10:27.125 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:10:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.384+0000 7f33fa600640 1 Processor -- start 2024-01-15T18:10:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.385+0000 7f33fa600640 1 -- start start 2024-01-15T18:10:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.386+0000 7f33fa600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33f40ae5a0 0x7f33f415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.386+0000 7f33fa600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f33f40b3ae0 0x7f33f406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.386+0000 7f33fa600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f406d120 0x7f33f40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.386+0000 7f33fa600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f4071450 con 0x7f33f40ae5a0 2024-01-15T18:10:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.386+0000 7f33fa600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f4070c40 con 0x7f33f40b3ae0 2024-01-15T18:10:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.386+0000 7f33fa600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f406c470 con 0x7f33f406d120 2024-01-15T18:10:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33f40ae5a0 0x7f33f415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33f40ae5a0 0x7f33f415e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48604/0 (socket says 172.21.15.26:48604) 2024-01-15T18:10:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3400640 1 -- 172.21.15.26:0/4203211572 learned_addr learned my addr 172.21.15.26:0/4203211572 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33ea800640 1 --2- 172.21.15.26:0/4203211572 >> v2:172.21.15.124:3300/0 conn(0x7f33f40b3ae0 0x7f33f406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3400640 1 -- 172.21.15.26:0/4203211572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f406d120 msgr2=0x7f33f40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3e00640 1 --2- 172.21.15.26:0/4203211572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f406d120 0x7f33f40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3400640 1 --2- 172.21.15.26:0/4203211572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f406d120 0x7f33f40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3400640 1 -- 172.21.15.26:0/4203211572 >> v2:172.21.15.124:3300/0 conn(0x7f33f40b3ae0 msgr2=0x7f33f406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3400640 1 --2- 172.21.15.26:0/4203211572 >> v2:172.21.15.124:3300/0 conn(0x7f33f40b3ae0 0x7f33f406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3400640 1 -- 172.21.15.26:0/4203211572 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f406c750 con 0x7f33f40ae5a0 2024-01-15T18:10:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33ea800640 1 --2- 172.21.15.26:0/4203211572 >> v2:172.21.15.124:3300/0 conn(0x7f33f40b3ae0 0x7f33f406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.387+0000 7f33f3e00640 1 --2- 172.21.15.26:0/4203211572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f406d120 0x7f33f40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:27.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.388+0000 7f33f3400640 1 --2- 172.21.15.26:0/4203211572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f40ae5a0 0x7f33f415e430 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f33f40cfea0 tx=0x7f33e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9172496d89048e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:27.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.389+0000 7f33f2a00640 1 -- 172.21.15.26:0/4203211572 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4031870 con 0x7f33f40ae5a0 2024-01-15T18:10:27.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.389+0000 7f33f2a00640 1 -- 172.21.15.26:0/4203211572 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e40319d0 con 0x7f33f40ae5a0 2024-01-15T18:10:27.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.389+0000 7f33f2a00640 1 -- 172.21.15.26:0/4203211572 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4031d60 con 0x7f33f40ae5a0 2024-01-15T18:10:27.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.390+0000 7f33fa600640 1 -- 172.21.15.26:0/4203211572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f40ae5a0 msgr2=0x7f33f415e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.390+0000 7f33fa600640 1 --2- 172.21.15.26:0/4203211572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f40ae5a0 0x7f33f415e430 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f33f40cfea0 tx=0x7f33e402b910 comp rx=0 tx=0).stop 2024-01-15T18:10:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.390+0000 7f33fa600640 1 -- 172.21.15.26:0/4203211572 shutdown_connections 2024-01-15T18:10:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.390+0000 7f33fa600640 1 --2- 172.21.15.26:0/4203211572 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f406d120 0x7f33f40cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.390+0000 7f33fa600640 1 --2- 172.21.15.26:0/4203211572 >> v2:172.21.15.124:3300/0 conn(0x7f33f40b3ae0 0x7f33f406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.390+0000 7f33fa600640 1 --2- 172.21.15.26:0/4203211572 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f40ae5a0 0x7f33f415e430 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.390+0000 7f33fa600640 1 -- 172.21.15.26:0/4203211572 >> 172.21.15.26:0/4203211572 conn(0x7f33f406a800 msgr2=0x7f33f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.390+0000 7f33fa600640 1 -- 172.21.15.26:0/4203211572 shutdown_connections 2024-01-15T18:10:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.390+0000 7f33fa600640 1 -- 172.21.15.26:0/4203211572 wait complete. 2024-01-15T18:10:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.392+0000 7f33fa600640 1 Processor -- start 2024-01-15T18:10:27.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.392+0000 7f33fa600640 1 -- start start 2024-01-15T18:10:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.393+0000 7f33fa600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f406d120 0x7f33f40cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.393+0000 7f33fa600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f40ae5a0 0x7f33f40b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.393+0000 7f33fa600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33f40b3ae0 0x7f33f40b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.393+0000 7f33fa600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f414a450 con 0x7f33f406d120 2024-01-15T18:10:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.393+0000 7f33fa600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4149b20 con 0x7f33f40b3ae0 2024-01-15T18:10:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.393+0000 7f33fa600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4149e00 con 0x7f33f40ae5a0 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.393+0000 7f33f3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f406d120 0x7f33f40cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.393+0000 7f33f3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f406d120 0x7f33f40cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48620/0 (socket says 172.21.15.26:48620) 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.393+0000 7f33f3400640 1 -- 172.21.15.26:0/3872543432 learned_addr learned my addr 172.21.15.26:0/3872543432 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33ea800640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f40ae5a0 0x7f33f40b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33f3400640 1 -- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f40ae5a0 msgr2=0x7f33f40b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33f3400640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f40ae5a0 0x7f33f40b2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33f3400640 1 -- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33f40b3ae0 msgr2=0x7f33f40b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33f3400640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33f40b3ae0 0x7f33f40b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33f3400640 1 -- 172.21.15.26:0/3872543432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40094d0 con 0x7f33f406d120 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33ea800640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f40ae5a0 0x7f33f40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33f3400640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f406d120 0x7f33f40cde80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f33e4000c00 tx=0x7f33e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:27.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33ebe00640 1 -- 172.21.15.26:0/3872543432 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e403ad10 con 0x7f33f406d120 2024-01-15T18:10:27.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33ebe00640 1 -- 172.21.15.26:0/3872543432 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e403ae70 con 0x7f33f406d120 2024-01-15T18:10:27.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.394+0000 7f33ebe00640 1 -- 172.21.15.26:0/3872543432 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4044610 con 0x7f33f406d120 2024-01-15T18:10:27.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.395+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f414a0e0 con 0x7f33f406d120 2024-01-15T18:10:27.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.395+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f41f6c30 con 0x7f33f406d120 2024-01-15T18:10:27.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.396+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f41f6e50 con 0x7f33f406d120 2024-01-15T18:10:27.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.397+0000 7f33ebe00640 1 -- 172.21.15.26:0/3872543432 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f33e404d450 con 0x7f33f406d120 2024-01-15T18:10:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.398+0000 7f33ebe00640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f33cc06c240 0x7f33cc06e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.398+0000 7f33ebe00640 1 -- 172.21.15.26:0/3872543432 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f33e402d040 con 0x7f33f406d120 2024-01-15T18:10:27.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.399+0000 7f33ea800640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f33cc06c240 0x7f33cc06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.400+0000 7f33ea800640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f33cc06c240 0x7f33cc06e700 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f33dc006fd0 tx=0x7f33dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:27.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.402+0000 7f33ebe00640 1 -- 172.21.15.26:0/3872543432 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33e40776f0 con 0x7f33f406d120 2024-01-15T18:10:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.592+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33f4070550 con 0x7f33cc06c240 2024-01-15T18:10:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.615+0000 7f33ebe00640 1 -- 172.21.15.26:0/3872543432 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281699 (secure 0 0 0) 0x7f33dc00c040 con 0x7f33cc06c240 2024-01-15T18:10:27.622 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:27.624 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:10:27.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f33cc06c240 msgr2=0x7f33cc06e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f33cc06c240 0x7f33cc06e700 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f33dc006fd0 tx=0x7f33dc008040 comp rx=0 tx=0).stop 2024-01-15T18:10:27.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f406d120 msgr2=0x7f33f40cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f406d120 0x7f33f40cde80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f33e4000c00 tx=0x7f33e40046c0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 shutdown_connections 2024-01-15T18:10:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33f40ae5a0 0x7f33f40b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f33cc06c240 0x7f33cc06e700 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f33f40b3ae0 0x7f33f40b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 --2- 172.21.15.26:0/3872543432 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33f406d120 0x7f33f40cde80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 >> 172.21.15.26:0/3872543432 conn(0x7f33f406a800 msgr2=0x7f33f406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 shutdown_connections 2024-01-15T18:10:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.622+0000 7f33fa600640 1 -- 172.21.15.26:0/3872543432 wait complete. 2024-01-15T18:10:27.646 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2024-01-15T18:10:26.272048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":580416,"kb_used_data":10768,"kb_used_omap":12,"kb_used_meta":569523,"kb_avail":754394304,"statfs":{"total":773094113280,"available":772499767296,"internally_reserved":0,"allocated":11026432,"data_stored":5436266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":583192112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697703"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611673+0000","last_change":"2024-01-15T18:10:21.611673+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611673+0000","last_undegraded":"2024-01-15T18:10:21.611673+0000","last_fullsized":"2024-01-15T18:10:21.611673+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611314+0000","last_change":"2024-01-15T18:10:21.611314+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611314+0000","last_undegraded":"2024-01-15T18:10:21.611314+0000","last_fullsized":"2024-01-15T18:10:21.611314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599418+0000","last_change":"2024-01-15T18:10:21.599418+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599418+0000","last_undegraded":"2024-01-15T18:10:21.599418+0000","last_fullsized":"2024-01-15T18:10:21.599418+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601091+0000","last_change":"2024-01-15T18:10:21.601091+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601091+0000","last_undegraded":"2024-01-15T18:10:21.601091+0000","last_fullsized":"2024-01-15T18:10:21.601091+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596994+0000","last_change":"2024-01-15T18:10:21.596994+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596994+0000","last_undegraded":"2024-01-15T18:10:21.596994+0000","last_fullsized":"2024-01-15T18:10:21.596994+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.607608+0000","last_change":"2024-01-15T18:10:21.607608+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.607608+0000","last_undegraded":"2024-01-15T18:10:21.607608+0000","last_fullsized":"2024-01-15T18:10:21.607608+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611612+0000","last_change":"2024-01-15T18:10:21.611612+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611612+0000","last_undegraded":"2024-01-15T18:10:21.611612+0000","last_fullsized":"2024-01-15T18:10:21.611612+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.610269+0000","last_change":"2024-01-15T18:10:21.610269+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.610269+0000","last_undegraded":"2024-01-15T18:10:21.610269+0000","last_fullsized":"2024-01-15T18:10:21.610269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.616486+0000","last_change":"2024-01-15T18:10:21.616486+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.616486+0000","last_undegraded":"2024-01-15T18:10:21.616486+0000","last_fullsized":"2024-01-15T18:10:21.616486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597244+0000","last_change":"2024-01-15T18:10:21.597244+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597244+0000","last_undegraded":"2024-01-15T18:10:21.597244+0000","last_fullsized":"2024-01-15T18:10:21.597244+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.616047+0000","last_change":"2024-01-15T18:10:21.616047+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.616047+0000","last_undegraded":"2024-01-15T18:10:21.616047+0000","last_fullsized":"2024-01-15T18:10:21.616047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.605943+0000","last_change":"2024-01-15T18:10:21.605943+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.605943+0000","last_undegraded":"2024-01-15T18:10:21.605943+0000","last_fullsized":"2024-01-15T18:10:21.605943+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.602017+0000","last_change":"2024-01-15T18:10:21.602017+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.602017+0000","last_undegraded":"2024-01-15T18:10:21.602017+0000","last_fullsized":"2024-01-15T18:10:21.602017+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600750+0000","last_change":"2024-01-15T18:10:21.600750+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600750+0000","last_undegraded":"2024-01-15T18:10:21.600750+0000","last_fullsized":"2024-01-15T18:10:21.600750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600926+0000","last_change":"2024-01-15T18:10:21.600926+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600926+0000","last_undegraded":"2024-01-15T18:10:21.600926+0000","last_fullsized":"2024-01-15T18:10:21.600926+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608161+0000","last_change":"2024-01-15T18:10:21.608161+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608161+0000","last_undegraded":"2024-01-15T18:10:21.608161+0000","last_fullsized":"2024-01-15T18:10:21.608161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608276+0000","last_change":"2024-01-15T18:10:21.608276+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608276+0000","last_undegraded":"2024-01-15T18:10:21.608276+0000","last_fullsized":"2024-01-15T18:10:21.608276+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601345+0000","last_change":"2024-01-15T18:10:21.601345+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601345+0000","last_undegraded":"2024-01-15T18:10:21.601345+0000","last_fullsized":"2024-01-15T18:10:21.601345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596934+0000","last_change":"2024-01-15T18:10:21.596934+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596934+0000","last_undegraded":"2024-01-15T18:10:21.596934+0000","last_fullsized":"2024-01-15T18:10:21.596934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.604940+0000","last_change":"2024-01-15T18:10:21.604940+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.604940+0000","last_undegraded":"2024-01-15T18:10:21.604940+0000","last_fullsized":"2024-01-15T18:10:21.604940+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597709+0000","last_change":"2024-01-15T18:10:21.597709+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597709+0000","last_undegraded":"2024-01-15T18:10:21.597709+0000","last_fullsized":"2024-01-15T18:10:21.597709+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608794+0000","last_change":"2024-01-15T18:10:21.608794+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608794+0000","last_undegraded":"2024-01-15T18:10:21.608794+0000","last_fullsized":"2024-01-15T18:10:21.608794+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608380+0000","last_change":"2024-01-15T18:10:21.608380+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608380+0000","last_undegraded":"2024-01-15T18:10:21.608380+0000","last_fullsized":"2024-01-15T18:10:21.608380+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.606909+0000","last_change":"2024-01-15T18:10:21.606909+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.606909+0000","last_undegraded":"2024-01-15T18:10:21.606909+0000","last_fullsized":"2024-01-15T18:10:21.606909+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596824+0000","last_change":"2024-01-15T18:10:21.596824+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596824+0000","last_undegraded":"2024-01-15T18:10:21.596824+0000","last_fullsized":"2024-01-15T18:10:21.596824+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608926+0000","last_change":"2024-01-15T18:10:21.608926+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608926+0000","last_undegraded":"2024-01-15T18:10:21.608926+0000","last_fullsized":"2024-01-15T18:10:21.608926+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.615205+0000","last_change":"2024-01-15T18:10:21.615205+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.615205+0000","last_undegraded":"2024-01-15T18:10:21.615205+0000","last_fullsized":"2024-01-15T18:10:21.615205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601293+0000","last_change":"2024-01-15T18:10:21.601293+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601293+0000","last_undegraded":"2024-01-15T18:10:21.601293+0000","last_fullsized":"2024-01-15T18:10:21.601293+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.606505+0000","last_change":"2024-01-15T18:10:21.606505+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.606505+0000","last_undegraded":"2024-01-15T18:10:21.606505+0000","last_fullsized":"2024-01-15T18:10:21.606505+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600333+0000","last_change":"2024-01-15T18:10:21.600333+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600333+0000","last_undegraded":"2024-01-15T18:10:21.600333+0000","last_fullsized":"2024-01-15T18:10:21.600333+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596109+0000","last_change":"2024-01-15T18:10:21.596109+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596109+0000","last_undegraded":"2024-01-15T18:10:21.596109+0000","last_fullsized":"2024-01-15T18:10:21.596109+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.604227+0000","last_change":"2024-01-15T18:10:21.604227+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.604227+0000","last_undegraded":"2024-01-15T18:10:21.604227+0000","last_fullsized":"2024-01-15T18:10:21.604227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599903+0000","last_change":"2024-01-15T18:10:21.599903+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599903+0000","last_undegraded":"2024-01-15T18:10:21.599903+0000","last_fullsized":"2024-01-15T18:10:21.599903+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.618790+0000","last_change":"2024-01-15T18:10:21.616691+0000","last_active":"2024-01-15T18:10:21.618790+0000","last_peered":"2024-01-15T18:10:21.618790+0000","last_clean":"2024-01-15T18:10:21.618790+0000","last_became_active":"2024-01-15T18:10:21.616428+0000","last_became_peered":"2024-01-15T18:10:21.616428+0000","last_unstale":"2024-01-15T18:10:21.618790+0000","last_undegraded":"2024-01-15T18:10:21.618790+0000","last_fullsized":"2024-01-15T18:10:21.618790+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.598656+0000","last_change":"2024-01-15T18:10:21.598656+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.598656+0000","last_undegraded":"2024-01-15T18:10:21.598656+0000","last_fullsized":"2024-01-15T18:10:21.598656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.624911+0000","last_change":"2024-01-15T18:10:21.624181+0000","last_active":"2024-01-15T18:10:21.624911+0000","last_peered":"2024-01-15T18:10:21.624911+0000","last_clean":"2024-01-15T18:10:21.624911+0000","last_became_active":"2024-01-15T18:10:21.623845+0000","last_became_peered":"2024-01-15T18:10:21.623845+0000","last_unstale":"2024-01-15T18:10:21.624911+0000","last_undegraded":"2024-01-15T18:10:21.624911+0000","last_fullsized":"2024-01-15T18:10:21.624911+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:31:30.348979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.607177+0000","last_change":"2024-01-15T18:10:21.607177+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.607177+0000","last_undegraded":"2024-01-15T18:10:21.607177+0000","last_fullsized":"2024-01-15T18:10:21.607177+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623538+0000","last_change":"2024-01-15T18:10:21.621640+0000","last_active":"2024-01-15T18:10:21.623538+0000","last_peered":"2024-01-15T18:10:21.623538+0000","last_clean":"2024-01-15T18:10:21.623538+0000","last_became_active":"2024-01-15T18:10:21.619850+0000","last_became_peered":"2024-01-15T18:10:21.619850+0000","last_unstale":"2024-01-15T18:10:21.623538+0000","last_undegraded":"2024-01-15T18:10:21.623538+0000","last_fullsized":"2024-01-15T18:10:21.623538+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:26.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608242+0000","last_change":"2024-01-15T18:10:21.608242+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608242+0000","last_undegraded":"2024-01-15T18:10:21.608242+0000","last_fullsized":"2024-01-15T18:10:21.608242+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622223+0000","last_change":"2024-01-15T18:10:21.619185+0000","last_active":"2024-01-15T18:10:21.622223+0000","last_peered":"2024-01-15T18:10:21.622223+0000","last_clean":"2024-01-15T18:10:21.622223+0000","last_became_active":"2024-01-15T18:10:21.618436+0000","last_became_peered":"2024-01-15T18:10:21.618436+0000","last_unstale":"2024-01-15T18:10:21.622223+0000","last_undegraded":"2024-01-15T18:10:21.622223+0000","last_fullsized":"2024-01-15T18:10:21.622223+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:15.951926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599261+0000","last_change":"2024-01-15T18:10:21.599261+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599261+0000","last_undegraded":"2024-01-15T18:10:21.599261+0000","last_fullsized":"2024-01-15T18:10:21.599261+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622304+0000","last_change":"2024-01-15T18:10:21.618207+0000","last_active":"2024-01-15T18:10:21.622304+0000","last_peered":"2024-01-15T18:10:21.622304+0000","last_clean":"2024-01-15T18:10:21.622304+0000","last_became_active":"2024-01-15T18:10:21.617907+0000","last_became_peered":"2024-01-15T18:10:21.617907+0000","last_unstale":"2024-01-15T18:10:21.622304+0000","last_undegraded":"2024-01-15T18:10:21.622304+0000","last_fullsized":"2024-01-15T18:10:21.622304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:10.136007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611462+0000","last_change":"2024-01-15T18:10:21.611462+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611462+0000","last_undegraded":"2024-01-15T18:10:21.611462+0000","last_fullsized":"2024-01-15T18:10:21.611462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.618613+0000","last_change":"2024-01-15T18:10:21.616023+0000","last_active":"2024-01-15T18:10:21.618613+0000","last_peered":"2024-01-15T18:10:21.618613+0000","last_clean":"2024-01-15T18:10:21.618613+0000","last_became_active":"2024-01-15T18:10:21.615732+0000","last_became_peered":"2024-01-15T18:10:21.615732+0000","last_unstale":"2024-01-15T18:10:21.618613+0000","last_undegraded":"2024-01-15T18:10:21.618613+0000","last_fullsized":"2024-01-15T18:10:21.618613+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:30.881887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.595870+0000","last_change":"2024-01-15T18:10:21.595870+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.595870+0000","last_undegraded":"2024-01-15T18:10:21.595870+0000","last_fullsized":"2024-01-15T18:10:21.595870+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.616352+0000","last_change":"2024-01-15T18:10:21.613918+0000","last_active":"2024-01-15T18:10:21.616352+0000","last_peered":"2024-01-15T18:10:21.616352+0000","last_clean":"2024-01-15T18:10:21.616352+0000","last_became_active":"2024-01-15T18:10:21.613184+0000","last_became_peered":"2024-01-15T18:10:21.613184+0000","last_unstale":"2024-01-15T18:10:21.616352+0000","last_undegraded":"2024-01-15T18:10:21.616352+0000","last_fullsized":"2024-01-15T18:10:21.616352+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:26.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.609479+0000","last_change":"2024-01-15T18:10:21.609479+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.609479+0000","last_undegraded":"2024-01-15T18:10:21.609479+0000","last_fullsized":"2024-01-15T18:10:21.609479+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617154+0000","last_change":"2024-01-15T18:10:21.608770+0000","last_active":"2024-01-15T18:10:21.617154+0000","last_peered":"2024-01-15T18:10:21.617154+0000","last_clean":"2024-01-15T18:10:21.617154+0000","last_became_active":"2024-01-15T18:10:21.607618+0000","last_became_peered":"2024-01-15T18:10:21.607618+0000","last_unstale":"2024-01-15T18:10:21.617154+0000","last_undegraded":"2024-01-15T18:10:21.617154+0000","last_fullsized":"2024-01-15T18:10:21.617154+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:59.671748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.605852+0000","last_change":"2024-01-15T18:10:21.605852+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.605852+0000","last_undegraded":"2024-01-15T18:10:21.605852+0000","last_fullsized":"2024-01-15T18:10:21.605852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.616985+0000","last_change":"2024-01-15T18:10:21.613722+0000","last_active":"2024-01-15T18:10:21.616985+0000","last_peered":"2024-01-15T18:10:21.616985+0000","last_clean":"2024-01-15T18:10:21.616985+0000","last_became_active":"2024-01-15T18:10:21.613117+0000","last_became_peered":"2024-01-15T18:10:21.613117+0000","last_unstale":"2024-01-15T18:10:21.616985+0000","last_undegraded":"2024-01-15T18:10:21.616985+0000","last_fullsized":"2024-01-15T18:10:21.616985+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:11.338333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611500+0000","last_change":"2024-01-15T18:10:21.611500+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611500+0000","last_undegraded":"2024-01-15T18:10:21.611500+0000","last_fullsized":"2024-01-15T18:10:21.611500+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.615908+0000","last_change":"2024-01-15T18:10:21.612961+0000","last_active":"2024-01-15T18:10:21.615908+0000","last_peered":"2024-01-15T18:10:21.615908+0000","last_clean":"2024-01-15T18:10:21.615908+0000","last_became_active":"2024-01-15T18:10:21.612624+0000","last_became_peered":"2024-01-15T18:10:21.612624+0000","last_unstale":"2024-01-15T18:10:21.615908+0000","last_undegraded":"2024-01-15T18:10:21.615908+0000","last_fullsized":"2024-01-15T18:10:21.615908+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:28.761075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.598336+0000","last_change":"2024-01-15T18:10:21.598336+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.598336+0000","last_undegraded":"2024-01-15T18:10:21.598336+0000","last_fullsized":"2024-01-15T18:10:21.598336+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623853+0000","last_change":"2024-01-15T18:10:21.623180+0000","last_active":"2024-01-15T18:10:21.623853+0000","last_peered":"2024-01-15T18:10:21.623853+0000","last_clean":"2024-01-15T18:10:21.623853+0000","last_became_active":"2024-01-15T18:10:21.622427+0000","last_became_peered":"2024-01-15T18:10:21.622427+0000","last_unstale":"2024-01-15T18:10:21.623853+0000","last_undegraded":"2024-01-15T18:10:21.623853+0000","last_fullsized":"2024-01-15T18:10:21.623853+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.176712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.614635+0000","last_change":"2024-01-15T18:10:21.614635+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.614635+0000","last_undegraded":"2024-01-15T18:10:21.614635+0000","last_fullsized":"2024-01-15T18:10:21.614635+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.620216+0000","last_change":"2024-01-15T18:10:21.617457+0000","last_active":"2024-01-15T18:10:21.620216+0000","last_peered":"2024-01-15T18:10:21.620216+0000","last_clean":"2024-01-15T18:10:21.620216+0000","last_became_active":"2024-01-15T18:10:21.617193+0000","last_became_peered":"2024-01-15T18:10:21.617193+0000","last_unstale":"2024-01-15T18:10:21.620216+0000","last_undegraded":"2024-01-15T18:10:21.620216+0000","last_fullsized":"2024-01-15T18:10:21.620216+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.598031+0000","last_change":"2024-01-15T18:10:21.598031+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.598031+0000","last_undegraded":"2024-01-15T18:10:21.598031+0000","last_fullsized":"2024-01-15T18:10:21.598031+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622228+0000","last_change":"2024-01-15T18:10:21.620555+0000","last_active":"2024-01-15T18:10:21.622228+0000","last_peered":"2024-01-15T18:10:21.622228+0000","last_clean":"2024-01-15T18:10:21.622228+0000","last_became_active":"2024-01-15T18:10:21.620232+0000","last_became_peered":"2024-01-15T18:10:21.620232+0000","last_unstale":"2024-01-15T18:10:21.622228+0000","last_undegraded":"2024-01-15T18:10:21.622228+0000","last_fullsized":"2024-01-15T18:10:21.622228+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:45.631921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597549+0000","last_change":"2024-01-15T18:10:21.597549+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597549+0000","last_undegraded":"2024-01-15T18:10:21.597549+0000","last_fullsized":"2024-01-15T18:10:21.597549+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.621611+0000","last_change":"2024-01-15T18:10:21.618350+0000","last_active":"2024-01-15T18:10:21.621611+0000","last_peered":"2024-01-15T18:10:21.621611+0000","last_clean":"2024-01-15T18:10:21.621611+0000","last_became_active":"2024-01-15T18:10:21.618029+0000","last_became_peered":"2024-01-15T18:10:21.618029+0000","last_unstale":"2024-01-15T18:10:21.621611+0000","last_undegraded":"2024-01-15T18:10:21.621611+0000","last_fullsized":"2024-01-15T18:10:21.621611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:04.305445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608054+0000","last_change":"2024-01-15T18:10:21.608054+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608054+0000","last_undegraded":"2024-01-15T18:10:21.608054+0000","last_fullsized":"2024-01-15T18:10:21.608054+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623404+0000","last_change":"2024-01-15T18:10:21.622418+0000","last_active":"2024-01-15T18:10:21.623404+0000","last_peered":"2024-01-15T18:10:21.623404+0000","last_clean":"2024-01-15T18:10:21.623404+0000","last_became_active":"2024-01-15T18:10:21.621566+0000","last_became_peered":"2024-01-15T18:10:21.621566+0000","last_unstale":"2024-01-15T18:10:21.623404+0000","last_undegraded":"2024-01-15T18:10:21.623404+0000","last_fullsized":"2024-01-15T18:10:21.623404+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:15.951926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601789+0000","last_change":"2024-01-15T18:10:21.601789+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601789+0000","last_undegraded":"2024-01-15T18:10:21.601789+0000","last_fullsized":"2024-01-15T18:10:21.601789+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617178+0000","last_change":"2024-01-15T18:10:21.613977+0000","last_active":"2024-01-15T18:10:21.617178+0000","last_peered":"2024-01-15T18:10:21.617178+0000","last_clean":"2024-01-15T18:10:21.617178+0000","last_became_active":"2024-01-15T18:10:21.613679+0000","last_became_peered":"2024-01-15T18:10:21.613679+0000","last_unstale":"2024-01-15T18:10:21.617178+0000","last_undegraded":"2024-01-15T18:10:21.617178+0000","last_fullsized":"2024-01-15T18:10:21.617178+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:16:05.935030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.595658+0000","last_change":"2024-01-15T18:10:21.595658+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.595658+0000","last_undegraded":"2024-01-15T18:10:21.595658+0000","last_fullsized":"2024-01-15T18:10:21.595658+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.621153+0000","last_change":"2024-01-15T18:10:21.618794+0000","last_active":"2024-01-15T18:10:21.621153+0000","last_peered":"2024-01-15T18:10:21.621153+0000","last_clean":"2024-01-15T18:10:21.621153+0000","last_became_active":"2024-01-15T18:10:21.618479+0000","last_became_peered":"2024-01-15T18:10:21.618479+0000","last_unstale":"2024-01-15T18:10:21.621153+0000","last_undegraded":"2024-01-15T18:10:21.621153+0000","last_fullsized":"2024-01-15T18:10:21.621153+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601227+0000","last_change":"2024-01-15T18:10:21.601227+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601227+0000","last_undegraded":"2024-01-15T18:10:21.601227+0000","last_fullsized":"2024-01-15T18:10:21.601227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.615155+0000","last_change":"2024-01-15T18:10:21.608435+0000","last_active":"2024-01-15T18:10:21.615155+0000","last_peered":"2024-01-15T18:10:21.615155+0000","last_clean":"2024-01-15T18:10:21.615155+0000","last_became_active":"2024-01-15T18:10:21.608053+0000","last_became_peered":"2024-01-15T18:10:21.608053+0000","last_unstale":"2024-01-15T18:10:21.615155+0000","last_undegraded":"2024-01-15T18:10:21.615155+0000","last_fullsized":"2024-01-15T18:10:21.615155+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.612014+0000","last_change":"2024-01-15T18:10:21.612014+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.612014+0000","last_undegraded":"2024-01-15T18:10:21.612014+0000","last_fullsized":"2024-01-15T18:10:21.612014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623513+0000","last_change":"2024-01-15T18:10:21.620384+0000","last_active":"2024-01-15T18:10:21.623513+0000","last_peered":"2024-01-15T18:10:21.623513+0000","last_clean":"2024-01-15T18:10:21.623513+0000","last_became_active":"2024-01-15T18:10:21.617541+0000","last_became_peered":"2024-01-15T18:10:21.617541+0000","last_unstale":"2024-01-15T18:10:21.623513+0000","last_undegraded":"2024-01-15T18:10:21.623513+0000","last_fullsized":"2024-01-15T18:10:21.623513+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599884+0000","last_change":"2024-01-15T18:10:21.599884+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599884+0000","last_undegraded":"2024-01-15T18:10:21.599884+0000","last_fullsized":"2024-01-15T18:10:21.599884+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.624202+0000","last_change":"2024-01-15T18:10:21.620621+0000","last_active":"2024-01-15T18:10:21.624202+0000","last_peered":"2024-01-15T18:10:21.624202+0000","last_clean":"2024-01-15T18:10:21.624202+0000","last_became_active":"2024-01-15T18:10:21.620314+0000","last_became_peered":"2024-01-15T18:10:21.620314+0000","last_unstale":"2024-01-15T18:10:21.624202+0000","last_undegraded":"2024-01-15T18:10:21.624202+0000","last_fullsized":"2024-01-15T18:10:21.624202+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.362159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597254+0000","last_change":"2024-01-15T18:10:21.597254+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597254+0000","last_undegraded":"2024-01-15T18:10:21.597254+0000","last_fullsized":"2024-01-15T18:10:21.597254+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617327+0000","last_change":"2024-01-15T18:10:21.614065+0000","last_active":"2024-01-15T18:10:21.617327+0000","last_peered":"2024-01-15T18:10:21.617327+0000","last_clean":"2024-01-15T18:10:21.617327+0000","last_became_active":"2024-01-15T18:10:21.613749+0000","last_became_peered":"2024-01-15T18:10:21.613749+0000","last_unstale":"2024-01-15T18:10:21.617327+0000","last_undegraded":"2024-01-15T18:10:21.617327+0000","last_fullsized":"2024-01-15T18:10:21.617327+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:30.543941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600363+0000","last_change":"2024-01-15T18:10:21.600363+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600363+0000","last_undegraded":"2024-01-15T18:10:21.600363+0000","last_fullsized":"2024-01-15T18:10:21.600363+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.612587+0000","last_change":"2024-01-15T18:10:21.608384+0000","last_active":"2024-01-15T18:10:21.612587+0000","last_peered":"2024-01-15T18:10:21.612587+0000","last_clean":"2024-01-15T18:10:21.612587+0000","last_became_active":"2024-01-15T18:10:21.607947+0000","last_became_peered":"2024-01-15T18:10:21.607947+0000","last_unstale":"2024-01-15T18:10:21.612587+0000","last_undegraded":"2024-01-15T18:10:21.612587+0000","last_fullsized":"2024-01-15T18:10:21.612587+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:12:47.963640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.595935+0000","last_change":"2024-01-15T18:10:21.595935+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.595935+0000","last_undegraded":"2024-01-15T18:10:21.595935+0000","last_fullsized":"2024-01-15T18:10:21.595935+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622275+0000","last_change":"2024-01-15T18:10:21.620127+0000","last_active":"2024-01-15T18:10:21.622275+0000","last_peered":"2024-01-15T18:10:21.622275+0000","last_clean":"2024-01-15T18:10:21.622275+0000","last_became_active":"2024-01-15T18:10:21.619856+0000","last_became_peered":"2024-01-15T18:10:21.619856+0000","last_unstale":"2024-01-15T18:10:21.622275+0000","last_undegraded":"2024-01-15T18:10:21.622275+0000","last_fullsized":"2024-01-15T18:10:21.622275+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:31:29.108419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.605332+0000","last_change":"2024-01-15T18:10:21.605332+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.605332+0000","last_undegraded":"2024-01-15T18:10:21.605332+0000","last_fullsized":"2024-01-15T18:10:21.605332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622081+0000","last_change":"2024-01-15T18:10:21.619938+0000","last_active":"2024-01-15T18:10:21.622081+0000","last_peered":"2024-01-15T18:10:21.622081+0000","last_clean":"2024-01-15T18:10:21.622081+0000","last_became_active":"2024-01-15T18:10:21.618751+0000","last_became_peered":"2024-01-15T18:10:21.618751+0000","last_unstale":"2024-01-15T18:10:21.622081+0000","last_undegraded":"2024-01-15T18:10:21.622081+0000","last_fullsized":"2024-01-15T18:10:21.622081+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.362159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.610825+0000","last_change":"2024-01-15T18:10:21.610825+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.610825+0000","last_undegraded":"2024-01-15T18:10:21.610825+0000","last_fullsized":"2024-01-15T18:10:21.610825+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622264+0000","last_change":"2024-01-15T18:10:21.618976+0000","last_active":"2024-01-15T18:10:21.622264+0000","last_peered":"2024-01-15T18:10:21.622264+0000","last_clean":"2024-01-15T18:10:21.622264+0000","last_became_active":"2024-01-15T18:10:21.618682+0000","last_became_peered":"2024-01-15T18:10:21.618682+0000","last_unstale":"2024-01-15T18:10:21.622264+0000","last_undegraded":"2024-01-15T18:10:21.622264+0000","last_fullsized":"2024-01-15T18:10:21.622264+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:12:47.963640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.603686+0000","last_change":"2024-01-15T18:10:21.603686+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.603686+0000","last_undegraded":"2024-01-15T18:10:21.603686+0000","last_fullsized":"2024-01-15T18:10:21.603686+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.624207+0000","last_change":"2024-01-15T18:10:21.623001+0000","last_active":"2024-01-15T18:10:21.624207+0000","last_peered":"2024-01-15T18:10:21.624207+0000","last_clean":"2024-01-15T18:10:21.624207+0000","last_became_active":"2024-01-15T18:10:21.622691+0000","last_became_peered":"2024-01-15T18:10:21.622691+0000","last_unstale":"2024-01-15T18:10:21.624207+0000","last_undegraded":"2024-01-15T18:10:21.624207+0000","last_fullsized":"2024-01-15T18:10:21.624207+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:30.543941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601674+0000","last_change":"2024-01-15T18:10:21.601674+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601674+0000","last_undegraded":"2024-01-15T18:10:21.601674+0000","last_fullsized":"2024-01-15T18:10:21.601674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.620715+0000","last_change":"2024-01-15T18:10:21.618463+0000","last_active":"2024-01-15T18:10:21.620715+0000","last_peered":"2024-01-15T18:10:21.620715+0000","last_clean":"2024-01-15T18:10:21.620715+0000","last_became_active":"2024-01-15T18:10:21.618150+0000","last_became_peered":"2024-01-15T18:10:21.618150+0000","last_unstale":"2024-01-15T18:10:21.620715+0000","last_undegraded":"2024-01-15T18:10:21.620715+0000","last_fullsized":"2024-01-15T18:10:21.620715+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:15.951926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599443+0000","last_change":"2024-01-15T18:10:21.599443+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599443+0000","last_undegraded":"2024-01-15T18:10:21.599443+0000","last_fullsized":"2024-01-15T18:10:21.599443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617800+0000","last_change":"2024-01-15T18:10:21.613099+0000","last_active":"2024-01-15T18:10:21.617800+0000","last_peered":"2024-01-15T18:10:21.617800+0000","last_clean":"2024-01-15T18:10:21.617800+0000","last_became_active":"2024-01-15T18:10:21.612818+0000","last_became_peered":"2024-01-15T18:10:21.612818+0000","last_unstale":"2024-01-15T18:10:21.617800+0000","last_undegraded":"2024-01-15T18:10:21.617800+0000","last_fullsized":"2024-01-15T18:10:21.617800+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:34.484071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611680+0000","last_change":"2024-01-15T18:10:21.611680+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611680+0000","last_undegraded":"2024-01-15T18:10:21.611680+0000","last_fullsized":"2024-01-15T18:10:21.611680+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.614848+0000","last_change":"2024-01-15T18:10:21.610659+0000","last_active":"2024-01-15T18:10:21.614848+0000","last_peered":"2024-01-15T18:10:21.614848+0000","last_clean":"2024-01-15T18:10:21.614848+0000","last_became_active":"2024-01-15T18:10:21.610364+0000","last_became_peered":"2024-01-15T18:10:21.610364+0000","last_unstale":"2024-01-15T18:10:21.614848+0000","last_undegraded":"2024-01-15T18:10:21.614848+0000","last_fullsized":"2024-01-15T18:10:21.614848+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599272+0000","last_change":"2024-01-15T18:10:21.599272+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599272+0000","last_undegraded":"2024-01-15T18:10:21.599272+0000","last_fullsized":"2024-01-15T18:10:21.599272+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.616668+0000","last_change":"2024-01-15T18:10:21.613230+0000","last_active":"2024-01-15T18:10:21.616668+0000","last_peered":"2024-01-15T18:10:21.616668+0000","last_clean":"2024-01-15T18:10:21.616668+0000","last_became_active":"2024-01-15T18:10:21.612943+0000","last_became_peered":"2024-01-15T18:10:21.612943+0000","last_unstale":"2024-01-15T18:10:21.616668+0000","last_undegraded":"2024-01-15T18:10:21.616668+0000","last_fullsized":"2024-01-15T18:10:21.616668+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:52:25.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.594607+0000","last_change":"2024-01-15T18:10:21.594607+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.594607+0000","last_undegraded":"2024-01-15T18:10:21.594607+0000","last_fullsized":"2024-01-15T18:10:21.594607+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623244+0000","last_change":"2024-01-15T18:10:21.621660+0000","last_active":"2024-01-15T18:10:21.623244+0000","last_peered":"2024-01-15T18:10:21.623244+0000","last_clean":"2024-01-15T18:10:21.623244+0000","last_became_active":"2024-01-15T18:10:21.620662+0000","last_became_peered":"2024-01-15T18:10:21.620662+0000","last_unstale":"2024-01-15T18:10:21.623244+0000","last_undegraded":"2024-01-15T18:10:21.623244+0000","last_fullsized":"2024-01-15T18:10:21.623244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:25.948319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597443+0000","last_change":"2024-01-15T18:10:21.597443+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597443+0000","last_undegraded":"2024-01-15T18:10:21.597443+0000","last_fullsized":"2024-01-15T18:10:21.597443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.625143+0000","last_change":"2024-01-15T18:10:21.624625+0000","last_active":"2024-01-15T18:10:21.625143+0000","last_peered":"2024-01-15T18:10:21.625143+0000","last_clean":"2024-01-15T18:10:21.625143+0000","last_became_active":"2024-01-15T18:10:21.624271+0000","last_became_peered":"2024-01-15T18:10:21.624271+0000","last_unstale":"2024-01-15T18:10:21.625143+0000","last_undegraded":"2024-01-15T18:10:21.625143+0000","last_fullsized":"2024-01-15T18:10:21.625143+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:55.724175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":233,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:10:20.590380+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:10:20.590380+0000","last_peered":"2024-01-15T18:10:20.590380+0000","last_clean":"2024-01-15T18:10:20.590380+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:10:20.590380+0000","last_undegraded":"2024-01-15T18:10:20.590380+0000","last_fullsized":"2024-01-15T18:10:20.590380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542332,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81464,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290376,"statfs":{"total":96636764160,"available":96553345024,"internally_reserved":0,"allocated":1814528,"data_stored":1117561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":81590730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3280000000000001,"15min":1.3740000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.40400000000000003},"max":{"1min":1.742,"5min":11.449999999999999,"15min":19.843},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.3759999999999999,"15min":1.629},"min":{"1min":0.89200000000000002,"5min":0.61499999999999999,"15min":0.53400000000000003},"max":{"1min":7.8250000000000002,"5min":7.8250000000000002,"15min":38.137},"last":1.26}]},{"osd":1,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.319,"15min":1.411},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.45600000000000002},"max":{"1min":1.577,"5min":11.51,"15min":19.917000000000002},"last":1.046},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.4079999999999999,"15min":1.605},"min":{"1min":0.85699999999999998,"5min":0.81100000000000005,"15min":0.42999999999999999},"max":{"1min":1.841,"5min":6.7480000000000002,"15min":51.843000000000004},"last":0.97499999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.371,"15min":1.425},"min":{"1min":0.97199999999999998,"5min":0.67200000000000004,"15min":0.437},"max":{"1min":3.3900000000000001,"5min":12.061,"15min":12.061},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.4430000000000001,"15min":1.532},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.47399999999999998},"max":{"1min":1.806,"5min":11.66,"15min":13.478999999999999},"last":1.5700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.544},"min":{"1min":0.85299999999999998,"5min":0.83199999999999996,"15min":0.47899999999999998},"max":{"1min":1.9079999999999999,"5min":7.4669999999999996,"15min":16.632000000000001},"last":1.363},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.5069999999999999,"15min":1.6160000000000001},"min":{"1min":0.97299999999999998,"5min":0.94199999999999995,"15min":0.56699999999999995},"max":{"1min":1.8520000000000001,"5min":11.611000000000001,"15min":13.439},"last":1.1679999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":2.319,"5min":1.728,"15min":1.7589999999999999},"min":{"1min":1.0169999999999999,"5min":0.81000000000000005,"15min":0.50700000000000001},"max":{"1min":17.067,"5min":23.042000000000002,"15min":23.042000000000002},"last":1.337},{"interface":"front","average":{"1min":1.425,"5min":1.5269999999999999,"15min":1.7270000000000001},"min":{"1min":0.94599999999999995,"5min":0.78800000000000003,"15min":0.51800000000000002},"max":{"1min":2.0779999999999998,"5min":16.867999999999999,"15min":38.682000000000002},"last":1.534}]},{"osd":5,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5640000000000001,"15min":1.8089999999999999},"min":{"1min":1.0329999999999999,"5min":0.89800000000000002,"15min":0.52700000000000002},"max":{"1min":1.843,"5min":11.696999999999999,"15min":25.006},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.7110000000000001,"15min":1.8400000000000001},"min":{"1min":1.0880000000000001,"5min":0.97499999999999998,"15min":0.624},"max":{"1min":2.04,"5min":22.933,"15min":22.933},"last":1.591}]},{"osd":6,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.9850000000000001,"15min":2.0419999999999998},"min":{"1min":1.1870000000000001,"5min":1.123,"15min":0.59299999999999997},"max":{"1min":2.2229999999999999,"5min":23.004000000000001,"15min":23.834},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.607,"15min":1.998},"min":{"1min":1.0680000000000001,"5min":0.84099999999999997,"15min":0.58299999999999996},"max":{"1min":2.1539999999999999,"5min":7.5220000000000002,"15min":63.932000000000002},"last":1.7490000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542331,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80884,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290956,"statfs":{"total":96636764160,"available":96553938944,"internally_reserved":0,"allocated":1220608,"data_stored":527193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81590726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.825,"5min":1.512,"15min":1.4139999999999999},"min":{"1min":0.79100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"max":{"1min":14.794,"5min":14.794,"15min":14.794},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.345,"15min":1.407},"min":{"1min":0.80000000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.5430000000000001,"5min":4.5430000000000001,"15min":14.013},"last":1.8160000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.3660000000000001,"15min":1.353},"min":{"1min":0.67900000000000005,"5min":0.63700000000000001,"15min":0.55100000000000005},"max":{"1min":15.064,"5min":15.064,"15min":15.064},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.508,"15min":1.53},"min":{"1min":0.85299999999999998,"5min":0.74199999999999999,"15min":0.66200000000000003},"max":{"1min":1.6899999999999999,"5min":23.495999999999999,"15min":23.495999999999999},"last":1.4339999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.667,"15min":1.587},"min":{"1min":0.81000000000000005,"5min":0.73799999999999999,"15min":0.59399999999999997},"max":{"1min":1.7490000000000001,"5min":23.087,"15min":23.087},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.871,"5min":1.4910000000000001,"15min":1.4590000000000001},"min":{"1min":0.86499999999999999,"5min":0.51400000000000001,"15min":0.504},"max":{"1min":14.94,"5min":14.94,"15min":14.94},"last":0.99199999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9750000000000001,"5min":1.7629999999999999,"15min":1.6950000000000001},"min":{"1min":0.95299999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":15.151999999999999,"5min":15.151999999999999,"15min":15.151999999999999},"last":1.244},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5580000000000001,"15min":1.601},"min":{"1min":0.89600000000000002,"5min":0.76100000000000001,"15min":0.70699999999999996},"max":{"1min":4.6849999999999996,"5min":13.708,"15min":14.066000000000001},"last":1.9059999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.577,"15min":1.661},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.51800000000000002},"max":{"1min":1.8720000000000001,"5min":23.327000000000002,"15min":23.327000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":2.4329999999999998,"5min":1.71,"15min":1.716},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.45800000000000002},"max":{"1min":15.239000000000001,"5min":15.239000000000001,"15min":17.289000000000001},"last":1.375}]},{"osd":5,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.4039999999999999,"15min":1.5669999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51600000000000001},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":14.176},"last":1.702},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.8580000000000001,"15min":1.8700000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.54700000000000004},"max":{"1min":2.044,"5min":23.417999999999999,"15min":23.417999999999999},"last":1.5289999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.716,"15min":1.7689999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":4.8239999999999998,"5min":16.454000000000001,"15min":21.265000000000001},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.7370000000000001,"5min":2.0089999999999999,"15min":1.9950000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.59699999999999998},"max":{"1min":3.8879999999999999,"5min":23.738,"15min":23.738},"last":1.6160000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542331,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58752,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313088,"statfs":{"total":96636764160,"available":96576602112,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.234,"15min":1.2829999999999999},"min":{"1min":0.73399999999999999,"5min":0.54200000000000004,"15min":0.47799999999999998},"max":{"1min":2.2839999999999998,"5min":7.6580000000000004,"15min":19.266999999999999},"last":1.55},{"interface":"front","average":{"1min":1.254,"5min":1.196,"15min":1.353},"min":{"1min":0.82299999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.738,"5min":2.1869999999999998,"15min":17.184999999999999},"last":1.028}]},{"osd":1,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2450000000000001,"15min":1.339},"min":{"1min":0.747,"5min":0.60699999999999998,"15min":0.499},"max":{"1min":2.355,"5min":5.3879999999999999,"15min":16.297000000000001},"last":1.472},{"interface":"front","average":{"1min":1.23,"5min":1.359,"15min":1.3280000000000001},"min":{"1min":0.78900000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8580000000000001,"5min":7.7649999999999997,"15min":9.2379999999999995},"last":1.3799999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.228,"15min":1.26},"min":{"1min":0.80400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.5429999999999999,"5min":2.1110000000000002,"15min":8.1820000000000004},"last":0.876},{"interface":"front","average":{"1min":1.431,"5min":1.349,"15min":1.4330000000000001},"min":{"1min":0.97299999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":16.158999999999999},"last":1.6359999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.427,"15min":1.486},"min":{"1min":0.89400000000000002,"5min":0.63700000000000001,"15min":0.56100000000000005},"max":{"1min":2.5179999999999998,"5min":7.8609999999999998,"15min":14.914},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.446,"15min":1.4550000000000001},"min":{"1min":1.0189999999999999,"5min":0.64600000000000002,"15min":0.63300000000000001},"max":{"1min":1.8109999999999999,"5min":2.0139999999999998,"15min":7.0110000000000001},"last":1.139}]},{"osd":4,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.329,"15min":1.5},"min":{"1min":0.80800000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":16.414999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.51,"15min":1.6319999999999999},"min":{"1min":0.56200000000000006,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.665,"5min":7.5359999999999996,"15min":17.991},"last":1.8500000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.45,"15min":1.5009999999999999},"min":{"1min":0.72699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.881,"5min":3.7559999999999998,"15min":6.4630000000000001},"last":1.244},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4830000000000001,"15min":1.599},"min":{"1min":1.0009999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":16.364000000000001},"last":1.7949999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.669,"15min":1.6220000000000001},"min":{"1min":0.78900000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.399,"5min":7.96,"15min":9.0600000000000005},"last":1.766},{"interface":"front","average":{"1min":1.587,"5min":1.6000000000000001,"15min":1.712},"min":{"1min":0.98599999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":19.356999999999999},"last":1.347}]}]},{"osd":4,"up_from":14,"seq":60129542331,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76736,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295104,"statfs":{"total":96636764160,"available":96558186496,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.286,"15min":1.341},"min":{"1min":0.70999999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.8170000000000002,"5min":3.8170000000000002,"15min":11.413},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3420000000000001,"15min":1.3919999999999999},"min":{"1min":0.72599999999999998,"5min":0.53800000000000003,"15min":0.53700000000000003},"max":{"1min":2.7919999999999998,"5min":3.726,"15min":9.7439999999999998},"last":1.5740000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.341,"15min":1.3660000000000001},"min":{"1min":0.73399999999999999,"5min":0.58899999999999997,"15min":0.45000000000000001},"max":{"1min":2.8700000000000001,"5min":11.128,"15min":11.987},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2330000000000001,"15min":1.3089999999999999},"min":{"1min":0.81100000000000005,"5min":0.69499999999999995,"15min":0.55300000000000005},"max":{"1min":1.909,"5min":3.3610000000000002,"15min":16.058},"last":1.397}]},{"osd":2,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3029999999999999,"15min":1.496},"min":{"1min":0.81299999999999994,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.6070000000000002,"5min":5.0960000000000001,"15min":13.865},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.401,"15min":1.5229999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.69499999999999995},"max":{"1min":1.899,"5min":5.1340000000000003,"15min":13.903},"last":1.325}]},{"osd":3,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.4430000000000001,"15min":1.6080000000000001},"min":{"1min":0.85999999999999999,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":3.8999999999999999,"5min":10.765000000000001,"15min":26.055},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.542,"5min":1.3500000000000001,"15min":1.4910000000000001},"min":{"1min":0.76700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":13.25},"last":1.466}]},{"osd":5,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.331,"15min":1.5700000000000001},"min":{"1min":0.86799999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":29.422000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.5329999999999999,"15min":1.538},"min":{"1min":0.85499999999999998,"5min":0.62,"15min":0.42199999999999999},"max":{"1min":3.0139999999999998,"5min":12.332000000000001,"15min":12.332000000000001},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.498,"15min":1.546},"min":{"1min":0.97399999999999998,"5min":0.59699999999999998,"15min":0.45200000000000001},"max":{"1min":1.9590000000000001,"5min":11.09,"15min":18.260000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.744,"5min":1.6299999999999999,"15min":1.696},"min":{"1min":1.1519999999999999,"5min":0.78700000000000003,"15min":0.48099999999999998},"max":{"1min":3.968,"5min":9.8740000000000006,"15min":11.657999999999999},"last":1.365}]},{"osd":7,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.6739999999999999,"15min":1.788},"min":{"1min":1.133,"5min":0.68200000000000005,"15min":0.503},"max":{"1min":3.0680000000000001,"5min":12.247,"15min":12.247},"last":1.125},{"interface":"front","average":{"1min":1.857,"5min":1.756,"15min":1.8660000000000001},"min":{"1min":1.3839999999999999,"5min":0.81899999999999995,"15min":0.53600000000000003},"max":{"1min":4.0209999999999999,"5min":12.311999999999999,"15min":12.311999999999999},"last":1.5189999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542333,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72580,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299260,"statfs":{"total":96636764160,"available":96562442240,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.113,"15min":1.2210000000000001},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.33300000000000002},"max":{"1min":1.284,"5min":9.2899999999999991,"15min":18.399999999999999},"last":1.304},{"interface":"front","average":{"1min":1.048,"5min":1.127,"15min":1.387},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":1.8140000000000001,"5min":2.23,"15min":21.047000000000001},"last":1.137}]},{"osd":1,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.92400000000000004,"15min":1.272},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.415,"5min":3.0409999999999999,"15min":17.777999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.3680000000000001,"15min":1.288},"min":{"1min":0.57399999999999995,"5min":0.46300000000000002,"15min":0.39500000000000002},"max":{"1min":1.6890000000000001,"5min":11.228999999999999,"15min":11.228999999999999},"last":0.77900000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.224,"15min":1.2789999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.518,"5min":2.7519999999999998,"15min":10.964},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.5489999999999999,"15min":1.554},"min":{"1min":0.60799999999999998,"5min":0.57099999999999995,"15min":0.54000000000000004},"max":{"1min":1.857,"5min":9.4920000000000009,"15min":20.495000000000001},"last":0.72399999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.401,"15min":1.5589999999999999},"min":{"1min":0.871,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.6419999999999999,"5min":6.319,"15min":13.536},"last":1.395},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.456,"15min":1.6399999999999999},"min":{"1min":0.748,"5min":0.63700000000000001,"15min":0.54600000000000004},"max":{"1min":1.736,"5min":16.398,"15min":16.398},"last":1.3420000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.681,"15min":1.6950000000000001},"min":{"1min":1.046,"5min":0.875,"15min":0.57299999999999995},"max":{"1min":1.6819999999999999,"5min":9.5690000000000008,"15min":20.606000000000002},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5589999999999999,"15min":1.665},"min":{"1min":0.97299999999999998,"5min":0.70199999999999996,"15min":0.68000000000000005},"max":{"1min":1.758,"5min":5.6980000000000004,"15min":11.603},"last":1.637}]},{"osd":6,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.615,"15min":1.7949999999999999},"min":{"1min":0.81000000000000005,"5min":0.61399999999999999,"15min":0.53500000000000003},"max":{"1min":2.0870000000000002,"5min":13.648999999999999,"15min":13.648999999999999},"last":1.25},{"interface":"front","average":{"1min":1.502,"5min":1.8109999999999999,"15min":1.8220000000000001},"min":{"1min":0.98699999999999999,"5min":0.78300000000000003,"15min":0.65800000000000003},"max":{"1min":2.0249999999999999,"5min":13.679,"15min":20.556000000000001},"last":1.093}]},{"osd":7,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.552,"15min":1.8480000000000001},"min":{"1min":0.78800000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8260000000000001,"5min":3.5590000000000002,"15min":28.843},"last":1.528},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.5760000000000001,"15min":1.857},"min":{"1min":0.97999999999999998,"5min":0.72799999999999998,"15min":0.67200000000000004},"max":{"1min":1.968,"5min":9.4100000000000001,"15min":15.143000000000001},"last":1.4570000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68420,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303420,"statfs":{"total":96636764160,"available":96566702080,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.129,"15min":1.095},"min":{"1min":0.68300000000000005,"5min":0.32100000000000001,"15min":0.26600000000000001},"max":{"1min":2.9359999999999999,"5min":12.278,"15min":14.427},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.159,"5min":1.4970000000000001,"15min":1.337},"min":{"1min":0.68799999999999994,"5min":0.50900000000000001,"15min":0.39300000000000002},"max":{"1min":2.25,"5min":15.314,"15min":15.314},"last":1.399}]},{"osd":1,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.5980000000000001,"15min":1.4470000000000001},"min":{"1min":0.80700000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.7029999999999998,"5min":34.463999999999999,"15min":34.463999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.351,"5min":1.7989999999999999,"15min":1.454},"min":{"1min":0.76800000000000002,"5min":0.54200000000000004,"15min":0.373},"max":{"1min":1.881,"5min":25.859999999999999,"15min":25.859999999999999},"last":1.2729999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.9430000000000001,"5min":1.26,"15min":1.4179999999999999},"min":{"1min":0.76200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":19.535},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.804,"15min":1.6240000000000001},"min":{"1min":0.95999999999999996,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":2.8050000000000002,"5min":22.783999999999999,"15min":22.783999999999999},"last":1.03}]},{"osd":4,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.603,"15min":1.784},"min":{"1min":1.113,"5min":0.73199999999999998,"15min":0.55700000000000005},"max":{"1min":2.8759999999999999,"5min":12.728,"15min":29.986999999999998},"last":1.155},{"interface":"front","average":{"1min":1.607,"5min":1.7609999999999999,"15min":1.742},"min":{"1min":1.0449999999999999,"5min":0.71299999999999997,"15min":0.56799999999999995},"max":{"1min":2.286,"5min":15.553000000000001,"15min":21.199999999999999},"last":1.679}]},{"osd":5,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":2.9350000000000001,"5min":1.855,"15min":1.746},"min":{"1min":0.84299999999999997,"5min":0.63700000000000001,"15min":0.51500000000000001},"max":{"1min":14.066000000000001,"5min":14.066000000000001,"15min":19.681999999999999},"last":1.089},{"interface":"front","average":{"1min":2.5379999999999998,"5min":2.0179999999999998,"15min":1.823},"min":{"1min":1.2010000000000001,"5min":0.84799999999999998,"15min":0.60899999999999999},"max":{"1min":14.025,"5min":14.025,"15min":14.702999999999999},"last":1.3600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":2.032,"15min":1.7869999999999999},"min":{"1min":1.1599999999999999,"5min":0.76800000000000002,"15min":0.68000000000000005},"max":{"1min":2.2919999999999998,"5min":15.683999999999999,"15min":15.683999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":2.2999999999999998,"5min":2.0819999999999999,"15min":1.853},"min":{"1min":0.96999999999999997,"5min":0.71999999999999997,"15min":0.58899999999999997},"max":{"1min":12.629,"5min":12.629,"15min":19.783000000000001},"last":1.2410000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":2.0129999999999999,"15min":2.0710000000000002},"min":{"1min":1.4510000000000001,"5min":0.77800000000000002,"15min":0.63100000000000001},"max":{"1min":2.3279999999999998,"5min":17.099,"15min":57.006999999999998},"last":1.597},{"interface":"front","average":{"1min":2.3919999999999999,"5min":2.1600000000000001,"15min":2.1930000000000001},"min":{"1min":1.081,"5min":1.0089999999999999,"15min":0.56100000000000005},"max":{"1min":12.754,"5min":22.853999999999999,"15min":22.853999999999999},"last":1.621}]}]},{"osd":1,"up_from":14,"seq":60129542332,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72580,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299260,"statfs":{"total":96636764160,"available":96562442240,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.363,"15min":1.2070000000000001},"min":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.34399999999999997},"max":{"1min":2.3839999999999999,"5min":30.460999999999999,"15min":30.460999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1759999999999999,"15min":1.117},"min":{"1min":0.61499999999999999,"5min":0.38700000000000001,"15min":0.314},"max":{"1min":2.0960000000000001,"5min":4.4059999999999997,"15min":18.658000000000001},"last":1.2310000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.198,"15min":1.1970000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":2.0619999999999998,"5min":7.117,"15min":21.126000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.752,"15min":1.5620000000000001},"min":{"1min":0.75700000000000001,"5min":0.70899999999999996,"15min":0.48699999999999999},"max":{"1min":2.1499999999999999,"5min":30.593,"15min":30.593},"last":1.042}]},{"osd":3,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.6719999999999999,"15min":1.617},"min":{"1min":0.76100000000000001,"5min":0.58099999999999996,"15min":0.48599999999999999},"max":{"1min":2.0579999999999998,"5min":30.547000000000001,"15min":30.547000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3480000000000001,"15min":1.3420000000000001},"min":{"1min":0.84999999999999998,"5min":0.52700000000000002,"15min":0.437},"max":{"1min":2.21,"5min":5.4729999999999999,"15min":15.510999999999999},"last":1.4730000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.663,"15min":1.6990000000000001},"min":{"1min":1.111,"5min":0.75900000000000001,"15min":0.72499999999999998},"max":{"1min":2.7599999999999998,"5min":8.8170000000000002,"15min":14.827},"last":1.585},{"interface":"front","average":{"1min":1.53,"5min":1.5429999999999999,"15min":1.657},"min":{"1min":0.95699999999999996,"5min":0.72999999999999998,"15min":0.69299999999999995},"max":{"1min":2.359,"5min":3.7440000000000002,"15min":14.851000000000001},"last":1.619}]},{"osd":5,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5349999999999999,"15min":1.6160000000000001},"min":{"1min":1.036,"5min":0.77600000000000002,"15min":0.59599999999999997},"max":{"1min":2.266,"5min":3.798,"15min":13.228999999999999},"last":1.329},{"interface":"front","average":{"1min":1.593,"5min":1.986,"15min":1.9450000000000001},"min":{"1min":1.0429999999999999,"5min":0.78700000000000003,"15min":0.68300000000000005},"max":{"1min":2.2949999999999999,"5min":34.042999999999999,"15min":34.042999999999999},"last":1.375}]},{"osd":6,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.579,"15min":1.7829999999999999},"min":{"1min":0.90100000000000002,"5min":0.83899999999999997,"15min":0.64100000000000001},"max":{"1min":2.5550000000000002,"5min":3.5539999999999998,"15min":34.158000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.736,"15min":1.6299999999999999},"min":{"1min":1.115,"5min":0.67900000000000005,"15min":0.62},"max":{"1min":4.1699999999999999,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.6779999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":2.0979999999999999,"15min":1.8660000000000001},"min":{"1min":0.95999999999999996,"5min":0.77300000000000002,"15min":0.64600000000000002},"max":{"1min":2.5750000000000002,"5min":33.659999999999997,"15min":33.659999999999997},"last":1.73},{"interface":"front","average":{"1min":1.694,"5min":1.8540000000000001,"15min":1.8009999999999999},"min":{"1min":1.002,"5min":0.81000000000000005,"15min":0.60499999999999998},"max":{"1min":2.613,"5min":11.882999999999999,"15min":24.289999999999999},"last":1.7949999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542333,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69000,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302840,"statfs":{"total":96636764160,"available":96566108160,"internally_reserved":0,"allocated":1826816,"data_stored":1123892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2210000000000001,"15min":1.083},"min":{"1min":0.437,"5min":0.42799999999999999,"15min":0.312},"max":{"1min":2.8929999999999998,"5min":12.050000000000001,"15min":12.050000000000001},"last":1.212},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.3340000000000001},"min":{"1min":0.50900000000000001,"5min":0.49099999999999999,"15min":0.36199999999999999},"max":{"1min":2.9780000000000002,"5min":11.962,"15min":11.962},"last":1.169}]},{"osd":2,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.179,"15min":1.2090000000000001},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.27600000000000002},"max":{"1min":1.595,"5min":7.4560000000000004,"15min":11.59},"last":0.877},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.1459999999999999,"15min":1.167},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35899999999999999},"max":{"1min":1.5649999999999999,"5min":7.3239999999999998,"15min":14.512},"last":1.5649999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.425,"15min":1.4319999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.41699999999999998},"max":{"1min":1.5669999999999999,"5min":11.904,"15min":11.904},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1379999999999999,"15min":1.2490000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.29599999999999999},"max":{"1min":1.6850000000000001,"5min":3.202,"15min":14.74},"last":0.81299999999999994}]},{"osd":4,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.375,"15min":1.5900000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6140000000000001,"5min":7.4000000000000004,"15min":32.115000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.571,"15min":1.702},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.67700000000000005},"max":{"1min":1.8260000000000001,"5min":9.0790000000000006,"15min":15.975},"last":1.288}]},{"osd":5,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.556,"15min":1.722},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":1.6200000000000001,"5min":12.006,"15min":32.174999999999997},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.514,"15min":1.6359999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.442},"max":{"1min":1.8939999999999999,"5min":4.7119999999999997,"15min":17.225000000000001},"last":1.605}]},{"osd":6,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.7849999999999999,"15min":1.7989999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.70299999999999996},"max":{"1min":3.4399999999999999,"5min":24.039999999999999,"15min":24.039999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.7549999999999999,"15min":1.794},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.625},"max":{"1min":1.7450000000000001,"5min":18.143999999999998,"15min":32.216999999999999},"last":1.4490000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.853,"15min":2.1230000000000002},"min":{"1min":0.754,"5min":0.754,"15min":0.69099999999999995},"max":{"1min":1.7350000000000001,"5min":26.427,"15min":34.131},"last":1.504},{"interface":"front","average":{"1min":1.474,"5min":2.089,"15min":2.0379999999999998},"min":{"1min":0.753,"5min":0.753,"15min":0.61799999999999999},"max":{"1min":3.9790000000000001,"5min":44.694000000000003,"15min":44.694000000000003},"last":1.4850000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:10:27.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:10:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.910+0000 7f6af1600640 1 Processor -- start 2024-01-15T18:10:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.911+0000 7f6af1600640 1 -- start start 2024-01-15T18:10:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.912+0000 7f6af1600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6aec143c40 0x7f6aec071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.912+0000 7f6af1600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6aec0717d0 0x7f6aec0cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.912+0000 7f6af1600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aec070e50 con 0x7f6aec0cca40 2024-01-15T18:10:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.912+0000 7f6af1600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec0723c0 con 0x7f6aec143c40 2024-01-15T18:10:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.912+0000 7f6af1600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aec0cce40 con 0x7f6aec0717d0 2024-01-15T18:10:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6aebe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6aec143c40 0x7f6aec071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6aebe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6aec143c40 0x7f6aec071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48630/0 (socket says 172.21.15.26:48630) 2024-01-15T18:10:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6af0c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6aec0cca40 0x7f6aec15eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51514/0 (socket says 172.21.15.26:51514) 2024-01-15T18:10:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6aebe00640 1 -- 172.21.15.26:0/2074087461 learned_addr learned my addr 172.21.15.26:0/2074087461 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6ae3e00640 1 --2- 172.21.15.26:0/2074087461 >> v2:172.21.15.124:3300/0 conn(0x7f6aec0717d0 0x7f6aec0cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6aebe00640 1 -- 172.21.15.26:0/2074087461 >> v2:172.21.15.124:3300/0 conn(0x7f6aec0717d0 msgr2=0x7f6aec0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6aebe00640 1 --2- 172.21.15.26:0/2074087461 >> v2:172.21.15.124:3300/0 conn(0x7f6aec0717d0 0x7f6aec0cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6aebe00640 1 -- 172.21.15.26:0/2074087461 >> v1:172.21.15.26:6789/0 conn(0x7f6aec0cca40 legacy=0x7f6aec15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6aebe00640 1 -- 172.21.15.26:0/2074087461 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec06e170 con 0x7f6aec143c40 2024-01-15T18:10:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.913+0000 7f6aeb400640 1 -- 172.21.15.26:0/2074087461 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676195954 0 0) 0x7f6aec070e50 con 0x7f6aec0cca40 2024-01-15T18:10:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.914+0000 7f6aebe00640 1 --2- 172.21.15.26:0/2074087461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec143c40 0x7f6aec071290 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0b44a0 tx=0x7f6adc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69dacbc58fada70 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.915+0000 7f6aeb400640 1 -- 172.21.15.26:0/2074087461 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc031870 con 0x7f6aec143c40 2024-01-15T18:10:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.915+0000 7f6aeb400640 1 -- 172.21.15.26:0/2074087461 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc0319d0 con 0x7f6aec143c40 2024-01-15T18:10:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.915+0000 7f6aeb400640 1 -- 172.21.15.26:0/2074087461 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc031d60 con 0x7f6aec143c40 2024-01-15T18:10:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.915+0000 7f6af1600640 1 -- 172.21.15.26:0/2074087461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec143c40 msgr2=0x7f6aec071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.915+0000 7f6af1600640 1 --2- 172.21.15.26:0/2074087461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec143c40 0x7f6aec071290 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0b44a0 tx=0x7f6adc02b910 comp rx=0 tx=0).stop 2024-01-15T18:10:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.915+0000 7f6af1600640 1 -- 172.21.15.26:0/2074087461 shutdown_connections 2024-01-15T18:10:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.915+0000 7f6af1600640 1 --2- 172.21.15.26:0/2074087461 >> v2:172.21.15.124:3300/0 conn(0x7f6aec0717d0 0x7f6aec0cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.915+0000 7f6af1600640 1 --2- 172.21.15.26:0/2074087461 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec143c40 0x7f6aec071290 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.916+0000 7f6af1600640 1 -- 172.21.15.26:0/2074087461 >> 172.21.15.26:0/2074087461 conn(0x7f6aec06a800 msgr2=0x7f6aec06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.916+0000 7f6af1600640 1 -- 172.21.15.26:0/2074087461 shutdown_connections 2024-01-15T18:10:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.916+0000 7f6af1600640 1 -- 172.21.15.26:0/2074087461 wait complete. 2024-01-15T18:10:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.917+0000 7f6af1600640 1 Processor -- start 2024-01-15T18:10:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.917+0000 7f6af1600640 1 -- start start 2024-01-15T18:10:27.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.918+0000 7f6af1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec0717d0 0x7f6aec14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.918+0000 7f6af1600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6aec0cca40 0x7f6aec139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.918+0000 7f6af1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aec143c40 0x7f6aec13ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.918+0000 7f6af1600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec13b960 con 0x7f6aec0717d0 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6af1600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec13e6d0 con 0x7f6aec0cca40 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6af1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aec139cb0 con 0x7f6aec143c40 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec0717d0 0x7f6aec14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec0717d0 0x7f6aec14f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48632/0 (socket says 172.21.15.26:48632) 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6aebe00640 1 -- 172.21.15.26:0/1398245722 learned_addr learned my addr 172.21.15.26:0/1398245722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6ae3e00640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6aec0cca40 0x7f6aec139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6aebe00640 1 -- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aec143c40 msgr2=0x7f6aec13ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6aebe00640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aec143c40 0x7f6aec13ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6aebe00640 1 -- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6aec0cca40 msgr2=0x7f6aec139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6aebe00640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6aec0cca40 0x7f6aec139770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.919+0000 7f6aebe00640 1 -- 172.21.15.26:0/1398245722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc0094d0 con 0x7f6aec0717d0 2024-01-15T18:10:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.920+0000 7f6aebe00640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec0717d0 0x7f6aec14f060 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f6adc02b950 tx=0x7f6adc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:27.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.920+0000 7f6ae8c00640 1 -- 172.21.15.26:0/1398245722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc03ad30 con 0x7f6aec0717d0 2024-01-15T18:10:27.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.920+0000 7f6ae8c00640 1 -- 172.21.15.26:0/1398245722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6adc0443f0 con 0x7f6aec0717d0 2024-01-15T18:10:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.920+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aec13b680 con 0x7f6aec0717d0 2024-01-15T18:10:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.920+0000 7f6ae8c00640 1 -- 172.21.15.26:0/1398245722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6adc0447a0 con 0x7f6aec0717d0 2024-01-15T18:10:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.920+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aec13e2f0 con 0x7f6aec0717d0 2024-01-15T18:10:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.921+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aec13de70 con 0x7f6aec0717d0 2024-01-15T18:10:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.922+0000 7f6ae8c00640 1 -- 172.21.15.26:0/1398245722 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f6adc04d450 con 0x7f6aec0717d0 2024-01-15T18:10:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.923+0000 7f6ae8c00640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ac406c1f0 0x7f6ac406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.923+0000 7f6ae8c00640 1 -- 172.21.15.26:0/1398245722 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6adc02d040 con 0x7f6aec0717d0 2024-01-15T18:10:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.924+0000 7f6ae3e00640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ac406c1f0 0x7f6ac406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.925+0000 7f6ae3e00640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ac406c1f0 0x7f6ac406e6b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6aec143380 tx=0x7f6ad4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:27.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:27.928+0000 7f6ae8c00640 1 -- 172.21.15.26:0/1398245722 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6aec13de70 con 0x7f6aec0717d0 2024-01-15T18:10:28.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.118+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6aec0720e0 con 0x7f6ac406c1f0 2024-01-15T18:10:28.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.141+0000 7f6ae8c00640 1 -- 172.21.15.26:0/1398245722 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281699 (secure 0 0 0) 0x7f6ad4009d80 con 0x7f6ac406c1f0 2024-01-15T18:10:28.148 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:28.150 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:10:28.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ac406c1f0 msgr2=0x7f6ac406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ac406c1f0 0x7f6ac406e6b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6aec143380 tx=0x7f6ad4008040 comp rx=0 tx=0).stop 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec0717d0 msgr2=0x7f6aec14f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec0717d0 0x7f6aec14f060 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f6adc02b950 tx=0x7f6adc0046c0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 shutdown_connections 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6aec143c40 0x7f6aec13ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ac406c1f0 0x7f6ac406e6b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6aec0cca40 0x7f6aec139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 --2- 172.21.15.26:0/1398245722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6aec0717d0 0x7f6aec14f060 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 >> 172.21.15.26:0/1398245722 conn(0x7f6aec06a800 msgr2=0x7f6aec140570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 shutdown_connections 2024-01-15T18:10:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.147+0000 7f6af1600640 1 -- 172.21.15.26:0/1398245722 wait complete. 2024-01-15T18:10:28.170 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2024-01-15T18:10:26.272048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":580416,"kb_used_data":10768,"kb_used_omap":12,"kb_used_meta":569523,"kb_avail":754394304,"statfs":{"total":773094113280,"available":772499767296,"internally_reserved":0,"allocated":11026432,"data_stored":5436266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":583192112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.697703"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611673+0000","last_change":"2024-01-15T18:10:21.611673+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611673+0000","last_undegraded":"2024-01-15T18:10:21.611673+0000","last_fullsized":"2024-01-15T18:10:21.611673+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611314+0000","last_change":"2024-01-15T18:10:21.611314+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611314+0000","last_undegraded":"2024-01-15T18:10:21.611314+0000","last_fullsized":"2024-01-15T18:10:21.611314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599418+0000","last_change":"2024-01-15T18:10:21.599418+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599418+0000","last_undegraded":"2024-01-15T18:10:21.599418+0000","last_fullsized":"2024-01-15T18:10:21.599418+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601091+0000","last_change":"2024-01-15T18:10:21.601091+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601091+0000","last_undegraded":"2024-01-15T18:10:21.601091+0000","last_fullsized":"2024-01-15T18:10:21.601091+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596994+0000","last_change":"2024-01-15T18:10:21.596994+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596994+0000","last_undegraded":"2024-01-15T18:10:21.596994+0000","last_fullsized":"2024-01-15T18:10:21.596994+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.607608+0000","last_change":"2024-01-15T18:10:21.607608+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.607608+0000","last_undegraded":"2024-01-15T18:10:21.607608+0000","last_fullsized":"2024-01-15T18:10:21.607608+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611612+0000","last_change":"2024-01-15T18:10:21.611612+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611612+0000","last_undegraded":"2024-01-15T18:10:21.611612+0000","last_fullsized":"2024-01-15T18:10:21.611612+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.610269+0000","last_change":"2024-01-15T18:10:21.610269+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.610269+0000","last_undegraded":"2024-01-15T18:10:21.610269+0000","last_fullsized":"2024-01-15T18:10:21.610269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.616486+0000","last_change":"2024-01-15T18:10:21.616486+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.616486+0000","last_undegraded":"2024-01-15T18:10:21.616486+0000","last_fullsized":"2024-01-15T18:10:21.616486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597244+0000","last_change":"2024-01-15T18:10:21.597244+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597244+0000","last_undegraded":"2024-01-15T18:10:21.597244+0000","last_fullsized":"2024-01-15T18:10:21.597244+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.616047+0000","last_change":"2024-01-15T18:10:21.616047+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.616047+0000","last_undegraded":"2024-01-15T18:10:21.616047+0000","last_fullsized":"2024-01-15T18:10:21.616047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.605943+0000","last_change":"2024-01-15T18:10:21.605943+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.605943+0000","last_undegraded":"2024-01-15T18:10:21.605943+0000","last_fullsized":"2024-01-15T18:10:21.605943+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.602017+0000","last_change":"2024-01-15T18:10:21.602017+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.602017+0000","last_undegraded":"2024-01-15T18:10:21.602017+0000","last_fullsized":"2024-01-15T18:10:21.602017+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600750+0000","last_change":"2024-01-15T18:10:21.600750+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600750+0000","last_undegraded":"2024-01-15T18:10:21.600750+0000","last_fullsized":"2024-01-15T18:10:21.600750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600926+0000","last_change":"2024-01-15T18:10:21.600926+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600926+0000","last_undegraded":"2024-01-15T18:10:21.600926+0000","last_fullsized":"2024-01-15T18:10:21.600926+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608161+0000","last_change":"2024-01-15T18:10:21.608161+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608161+0000","last_undegraded":"2024-01-15T18:10:21.608161+0000","last_fullsized":"2024-01-15T18:10:21.608161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608276+0000","last_change":"2024-01-15T18:10:21.608276+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608276+0000","last_undegraded":"2024-01-15T18:10:21.608276+0000","last_fullsized":"2024-01-15T18:10:21.608276+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601345+0000","last_change":"2024-01-15T18:10:21.601345+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601345+0000","last_undegraded":"2024-01-15T18:10:21.601345+0000","last_fullsized":"2024-01-15T18:10:21.601345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596934+0000","last_change":"2024-01-15T18:10:21.596934+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596934+0000","last_undegraded":"2024-01-15T18:10:21.596934+0000","last_fullsized":"2024-01-15T18:10:21.596934+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.604940+0000","last_change":"2024-01-15T18:10:21.604940+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.604940+0000","last_undegraded":"2024-01-15T18:10:21.604940+0000","last_fullsized":"2024-01-15T18:10:21.604940+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597709+0000","last_change":"2024-01-15T18:10:21.597709+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597709+0000","last_undegraded":"2024-01-15T18:10:21.597709+0000","last_fullsized":"2024-01-15T18:10:21.597709+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608794+0000","last_change":"2024-01-15T18:10:21.608794+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608794+0000","last_undegraded":"2024-01-15T18:10:21.608794+0000","last_fullsized":"2024-01-15T18:10:21.608794+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608380+0000","last_change":"2024-01-15T18:10:21.608380+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608380+0000","last_undegraded":"2024-01-15T18:10:21.608380+0000","last_fullsized":"2024-01-15T18:10:21.608380+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.606909+0000","last_change":"2024-01-15T18:10:21.606909+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.606909+0000","last_undegraded":"2024-01-15T18:10:21.606909+0000","last_fullsized":"2024-01-15T18:10:21.606909+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596824+0000","last_change":"2024-01-15T18:10:21.596824+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596824+0000","last_undegraded":"2024-01-15T18:10:21.596824+0000","last_fullsized":"2024-01-15T18:10:21.596824+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608926+0000","last_change":"2024-01-15T18:10:21.608926+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608926+0000","last_undegraded":"2024-01-15T18:10:21.608926+0000","last_fullsized":"2024-01-15T18:10:21.608926+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.615205+0000","last_change":"2024-01-15T18:10:21.615205+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.615205+0000","last_undegraded":"2024-01-15T18:10:21.615205+0000","last_fullsized":"2024-01-15T18:10:21.615205+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601293+0000","last_change":"2024-01-15T18:10:21.601293+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601293+0000","last_undegraded":"2024-01-15T18:10:21.601293+0000","last_fullsized":"2024-01-15T18:10:21.601293+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.606505+0000","last_change":"2024-01-15T18:10:21.606505+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.606505+0000","last_undegraded":"2024-01-15T18:10:21.606505+0000","last_fullsized":"2024-01-15T18:10:21.606505+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600333+0000","last_change":"2024-01-15T18:10:21.600333+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600333+0000","last_undegraded":"2024-01-15T18:10:21.600333+0000","last_fullsized":"2024-01-15T18:10:21.600333+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.596109+0000","last_change":"2024-01-15T18:10:21.596109+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.596109+0000","last_undegraded":"2024-01-15T18:10:21.596109+0000","last_fullsized":"2024-01-15T18:10:21.596109+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.604227+0000","last_change":"2024-01-15T18:10:21.604227+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.604227+0000","last_undegraded":"2024-01-15T18:10:21.604227+0000","last_fullsized":"2024-01-15T18:10:21.604227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599903+0000","last_change":"2024-01-15T18:10:21.599903+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599903+0000","last_undegraded":"2024-01-15T18:10:21.599903+0000","last_fullsized":"2024-01-15T18:10:21.599903+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.618790+0000","last_change":"2024-01-15T18:10:21.616691+0000","last_active":"2024-01-15T18:10:21.618790+0000","last_peered":"2024-01-15T18:10:21.618790+0000","last_clean":"2024-01-15T18:10:21.618790+0000","last_became_active":"2024-01-15T18:10:21.616428+0000","last_became_peered":"2024-01-15T18:10:21.616428+0000","last_unstale":"2024-01-15T18:10:21.618790+0000","last_undegraded":"2024-01-15T18:10:21.618790+0000","last_fullsized":"2024-01-15T18:10:21.618790+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.598656+0000","last_change":"2024-01-15T18:10:21.598656+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.598656+0000","last_undegraded":"2024-01-15T18:10:21.598656+0000","last_fullsized":"2024-01-15T18:10:21.598656+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.624911+0000","last_change":"2024-01-15T18:10:21.624181+0000","last_active":"2024-01-15T18:10:21.624911+0000","last_peered":"2024-01-15T18:10:21.624911+0000","last_clean":"2024-01-15T18:10:21.624911+0000","last_became_active":"2024-01-15T18:10:21.623845+0000","last_became_peered":"2024-01-15T18:10:21.623845+0000","last_unstale":"2024-01-15T18:10:21.624911+0000","last_undegraded":"2024-01-15T18:10:21.624911+0000","last_fullsized":"2024-01-15T18:10:21.624911+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:31:30.348979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.607177+0000","last_change":"2024-01-15T18:10:21.607177+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.607177+0000","last_undegraded":"2024-01-15T18:10:21.607177+0000","last_fullsized":"2024-01-15T18:10:21.607177+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623538+0000","last_change":"2024-01-15T18:10:21.621640+0000","last_active":"2024-01-15T18:10:21.623538+0000","last_peered":"2024-01-15T18:10:21.623538+0000","last_clean":"2024-01-15T18:10:21.623538+0000","last_became_active":"2024-01-15T18:10:21.619850+0000","last_became_peered":"2024-01-15T18:10:21.619850+0000","last_unstale":"2024-01-15T18:10:21.623538+0000","last_undegraded":"2024-01-15T18:10:21.623538+0000","last_fullsized":"2024-01-15T18:10:21.623538+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:26.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608242+0000","last_change":"2024-01-15T18:10:21.608242+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608242+0000","last_undegraded":"2024-01-15T18:10:21.608242+0000","last_fullsized":"2024-01-15T18:10:21.608242+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622223+0000","last_change":"2024-01-15T18:10:21.619185+0000","last_active":"2024-01-15T18:10:21.622223+0000","last_peered":"2024-01-15T18:10:21.622223+0000","last_clean":"2024-01-15T18:10:21.622223+0000","last_became_active":"2024-01-15T18:10:21.618436+0000","last_became_peered":"2024-01-15T18:10:21.618436+0000","last_unstale":"2024-01-15T18:10:21.622223+0000","last_undegraded":"2024-01-15T18:10:21.622223+0000","last_fullsized":"2024-01-15T18:10:21.622223+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:15.951926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599261+0000","last_change":"2024-01-15T18:10:21.599261+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599261+0000","last_undegraded":"2024-01-15T18:10:21.599261+0000","last_fullsized":"2024-01-15T18:10:21.599261+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622304+0000","last_change":"2024-01-15T18:10:21.618207+0000","last_active":"2024-01-15T18:10:21.622304+0000","last_peered":"2024-01-15T18:10:21.622304+0000","last_clean":"2024-01-15T18:10:21.622304+0000","last_became_active":"2024-01-15T18:10:21.617907+0000","last_became_peered":"2024-01-15T18:10:21.617907+0000","last_unstale":"2024-01-15T18:10:21.622304+0000","last_undegraded":"2024-01-15T18:10:21.622304+0000","last_fullsized":"2024-01-15T18:10:21.622304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:10.136007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611462+0000","last_change":"2024-01-15T18:10:21.611462+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611462+0000","last_undegraded":"2024-01-15T18:10:21.611462+0000","last_fullsized":"2024-01-15T18:10:21.611462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.618613+0000","last_change":"2024-01-15T18:10:21.616023+0000","last_active":"2024-01-15T18:10:21.618613+0000","last_peered":"2024-01-15T18:10:21.618613+0000","last_clean":"2024-01-15T18:10:21.618613+0000","last_became_active":"2024-01-15T18:10:21.615732+0000","last_became_peered":"2024-01-15T18:10:21.615732+0000","last_unstale":"2024-01-15T18:10:21.618613+0000","last_undegraded":"2024-01-15T18:10:21.618613+0000","last_fullsized":"2024-01-15T18:10:21.618613+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:30.881887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.595870+0000","last_change":"2024-01-15T18:10:21.595870+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.595870+0000","last_undegraded":"2024-01-15T18:10:21.595870+0000","last_fullsized":"2024-01-15T18:10:21.595870+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.616352+0000","last_change":"2024-01-15T18:10:21.613918+0000","last_active":"2024-01-15T18:10:21.616352+0000","last_peered":"2024-01-15T18:10:21.616352+0000","last_clean":"2024-01-15T18:10:21.616352+0000","last_became_active":"2024-01-15T18:10:21.613184+0000","last_became_peered":"2024-01-15T18:10:21.613184+0000","last_unstale":"2024-01-15T18:10:21.616352+0000","last_undegraded":"2024-01-15T18:10:21.616352+0000","last_fullsized":"2024-01-15T18:10:21.616352+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:26.905412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.609479+0000","last_change":"2024-01-15T18:10:21.609479+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.609479+0000","last_undegraded":"2024-01-15T18:10:21.609479+0000","last_fullsized":"2024-01-15T18:10:21.609479+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617154+0000","last_change":"2024-01-15T18:10:21.608770+0000","last_active":"2024-01-15T18:10:21.617154+0000","last_peered":"2024-01-15T18:10:21.617154+0000","last_clean":"2024-01-15T18:10:21.617154+0000","last_became_active":"2024-01-15T18:10:21.607618+0000","last_became_peered":"2024-01-15T18:10:21.607618+0000","last_unstale":"2024-01-15T18:10:21.617154+0000","last_undegraded":"2024-01-15T18:10:21.617154+0000","last_fullsized":"2024-01-15T18:10:21.617154+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:59:59.671748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.605852+0000","last_change":"2024-01-15T18:10:21.605852+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.605852+0000","last_undegraded":"2024-01-15T18:10:21.605852+0000","last_fullsized":"2024-01-15T18:10:21.605852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.616985+0000","last_change":"2024-01-15T18:10:21.613722+0000","last_active":"2024-01-15T18:10:21.616985+0000","last_peered":"2024-01-15T18:10:21.616985+0000","last_clean":"2024-01-15T18:10:21.616985+0000","last_became_active":"2024-01-15T18:10:21.613117+0000","last_became_peered":"2024-01-15T18:10:21.613117+0000","last_unstale":"2024-01-15T18:10:21.616985+0000","last_undegraded":"2024-01-15T18:10:21.616985+0000","last_fullsized":"2024-01-15T18:10:21.616985+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:11.338333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611500+0000","last_change":"2024-01-15T18:10:21.611500+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611500+0000","last_undegraded":"2024-01-15T18:10:21.611500+0000","last_fullsized":"2024-01-15T18:10:21.611500+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.615908+0000","last_change":"2024-01-15T18:10:21.612961+0000","last_active":"2024-01-15T18:10:21.615908+0000","last_peered":"2024-01-15T18:10:21.615908+0000","last_clean":"2024-01-15T18:10:21.615908+0000","last_became_active":"2024-01-15T18:10:21.612624+0000","last_became_peered":"2024-01-15T18:10:21.612624+0000","last_unstale":"2024-01-15T18:10:21.615908+0000","last_undegraded":"2024-01-15T18:10:21.615908+0000","last_fullsized":"2024-01-15T18:10:21.615908+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:37:28.761075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.598336+0000","last_change":"2024-01-15T18:10:21.598336+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.598336+0000","last_undegraded":"2024-01-15T18:10:21.598336+0000","last_fullsized":"2024-01-15T18:10:21.598336+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623853+0000","last_change":"2024-01-15T18:10:21.623180+0000","last_active":"2024-01-15T18:10:21.623853+0000","last_peered":"2024-01-15T18:10:21.623853+0000","last_clean":"2024-01-15T18:10:21.623853+0000","last_became_active":"2024-01-15T18:10:21.622427+0000","last_became_peered":"2024-01-15T18:10:21.622427+0000","last_unstale":"2024-01-15T18:10:21.623853+0000","last_undegraded":"2024-01-15T18:10:21.623853+0000","last_fullsized":"2024-01-15T18:10:21.623853+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.176712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.614635+0000","last_change":"2024-01-15T18:10:21.614635+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.614635+0000","last_undegraded":"2024-01-15T18:10:21.614635+0000","last_fullsized":"2024-01-15T18:10:21.614635+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.620216+0000","last_change":"2024-01-15T18:10:21.617457+0000","last_active":"2024-01-15T18:10:21.620216+0000","last_peered":"2024-01-15T18:10:21.620216+0000","last_clean":"2024-01-15T18:10:21.620216+0000","last_became_active":"2024-01-15T18:10:21.617193+0000","last_became_peered":"2024-01-15T18:10:21.617193+0000","last_unstale":"2024-01-15T18:10:21.620216+0000","last_undegraded":"2024-01-15T18:10:21.620216+0000","last_fullsized":"2024-01-15T18:10:21.620216+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.598031+0000","last_change":"2024-01-15T18:10:21.598031+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.598031+0000","last_undegraded":"2024-01-15T18:10:21.598031+0000","last_fullsized":"2024-01-15T18:10:21.598031+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622228+0000","last_change":"2024-01-15T18:10:21.620555+0000","last_active":"2024-01-15T18:10:21.622228+0000","last_peered":"2024-01-15T18:10:21.622228+0000","last_clean":"2024-01-15T18:10:21.622228+0000","last_became_active":"2024-01-15T18:10:21.620232+0000","last_became_peered":"2024-01-15T18:10:21.620232+0000","last_unstale":"2024-01-15T18:10:21.622228+0000","last_undegraded":"2024-01-15T18:10:21.622228+0000","last_fullsized":"2024-01-15T18:10:21.622228+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:45.631921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597549+0000","last_change":"2024-01-15T18:10:21.597549+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597549+0000","last_undegraded":"2024-01-15T18:10:21.597549+0000","last_fullsized":"2024-01-15T18:10:21.597549+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.621611+0000","last_change":"2024-01-15T18:10:21.618350+0000","last_active":"2024-01-15T18:10:21.621611+0000","last_peered":"2024-01-15T18:10:21.621611+0000","last_clean":"2024-01-15T18:10:21.621611+0000","last_became_active":"2024-01-15T18:10:21.618029+0000","last_became_peered":"2024-01-15T18:10:21.618029+0000","last_unstale":"2024-01-15T18:10:21.621611+0000","last_undegraded":"2024-01-15T18:10:21.621611+0000","last_fullsized":"2024-01-15T18:10:21.621611+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:04.305445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.608054+0000","last_change":"2024-01-15T18:10:21.608054+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.608054+0000","last_undegraded":"2024-01-15T18:10:21.608054+0000","last_fullsized":"2024-01-15T18:10:21.608054+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623404+0000","last_change":"2024-01-15T18:10:21.622418+0000","last_active":"2024-01-15T18:10:21.623404+0000","last_peered":"2024-01-15T18:10:21.623404+0000","last_clean":"2024-01-15T18:10:21.623404+0000","last_became_active":"2024-01-15T18:10:21.621566+0000","last_became_peered":"2024-01-15T18:10:21.621566+0000","last_unstale":"2024-01-15T18:10:21.623404+0000","last_undegraded":"2024-01-15T18:10:21.623404+0000","last_fullsized":"2024-01-15T18:10:21.623404+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:15.951926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601789+0000","last_change":"2024-01-15T18:10:21.601789+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601789+0000","last_undegraded":"2024-01-15T18:10:21.601789+0000","last_fullsized":"2024-01-15T18:10:21.601789+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617178+0000","last_change":"2024-01-15T18:10:21.613977+0000","last_active":"2024-01-15T18:10:21.617178+0000","last_peered":"2024-01-15T18:10:21.617178+0000","last_clean":"2024-01-15T18:10:21.617178+0000","last_became_active":"2024-01-15T18:10:21.613679+0000","last_became_peered":"2024-01-15T18:10:21.613679+0000","last_unstale":"2024-01-15T18:10:21.617178+0000","last_undegraded":"2024-01-15T18:10:21.617178+0000","last_fullsized":"2024-01-15T18:10:21.617178+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:16:05.935030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.595658+0000","last_change":"2024-01-15T18:10:21.595658+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.595658+0000","last_undegraded":"2024-01-15T18:10:21.595658+0000","last_fullsized":"2024-01-15T18:10:21.595658+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.621153+0000","last_change":"2024-01-15T18:10:21.618794+0000","last_active":"2024-01-15T18:10:21.621153+0000","last_peered":"2024-01-15T18:10:21.621153+0000","last_clean":"2024-01-15T18:10:21.621153+0000","last_became_active":"2024-01-15T18:10:21.618479+0000","last_became_peered":"2024-01-15T18:10:21.618479+0000","last_unstale":"2024-01-15T18:10:21.621153+0000","last_undegraded":"2024-01-15T18:10:21.621153+0000","last_fullsized":"2024-01-15T18:10:21.621153+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601227+0000","last_change":"2024-01-15T18:10:21.601227+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601227+0000","last_undegraded":"2024-01-15T18:10:21.601227+0000","last_fullsized":"2024-01-15T18:10:21.601227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.615155+0000","last_change":"2024-01-15T18:10:21.608435+0000","last_active":"2024-01-15T18:10:21.615155+0000","last_peered":"2024-01-15T18:10:21.615155+0000","last_clean":"2024-01-15T18:10:21.615155+0000","last_became_active":"2024-01-15T18:10:21.608053+0000","last_became_peered":"2024-01-15T18:10:21.608053+0000","last_unstale":"2024-01-15T18:10:21.615155+0000","last_undegraded":"2024-01-15T18:10:21.615155+0000","last_fullsized":"2024-01-15T18:10:21.615155+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.612014+0000","last_change":"2024-01-15T18:10:21.612014+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.612014+0000","last_undegraded":"2024-01-15T18:10:21.612014+0000","last_fullsized":"2024-01-15T18:10:21.612014+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623513+0000","last_change":"2024-01-15T18:10:21.620384+0000","last_active":"2024-01-15T18:10:21.623513+0000","last_peered":"2024-01-15T18:10:21.623513+0000","last_clean":"2024-01-15T18:10:21.623513+0000","last_became_active":"2024-01-15T18:10:21.617541+0000","last_became_peered":"2024-01-15T18:10:21.617541+0000","last_unstale":"2024-01-15T18:10:21.623513+0000","last_undegraded":"2024-01-15T18:10:21.623513+0000","last_fullsized":"2024-01-15T18:10:21.623513+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599884+0000","last_change":"2024-01-15T18:10:21.599884+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599884+0000","last_undegraded":"2024-01-15T18:10:21.599884+0000","last_fullsized":"2024-01-15T18:10:21.599884+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.624202+0000","last_change":"2024-01-15T18:10:21.620621+0000","last_active":"2024-01-15T18:10:21.624202+0000","last_peered":"2024-01-15T18:10:21.624202+0000","last_clean":"2024-01-15T18:10:21.624202+0000","last_became_active":"2024-01-15T18:10:21.620314+0000","last_became_peered":"2024-01-15T18:10:21.620314+0000","last_unstale":"2024-01-15T18:10:21.624202+0000","last_undegraded":"2024-01-15T18:10:21.624202+0000","last_fullsized":"2024-01-15T18:10:21.624202+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.362159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597254+0000","last_change":"2024-01-15T18:10:21.597254+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597254+0000","last_undegraded":"2024-01-15T18:10:21.597254+0000","last_fullsized":"2024-01-15T18:10:21.597254+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617327+0000","last_change":"2024-01-15T18:10:21.614065+0000","last_active":"2024-01-15T18:10:21.617327+0000","last_peered":"2024-01-15T18:10:21.617327+0000","last_clean":"2024-01-15T18:10:21.617327+0000","last_became_active":"2024-01-15T18:10:21.613749+0000","last_became_peered":"2024-01-15T18:10:21.613749+0000","last_unstale":"2024-01-15T18:10:21.617327+0000","last_undegraded":"2024-01-15T18:10:21.617327+0000","last_fullsized":"2024-01-15T18:10:21.617327+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:30.543941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.600363+0000","last_change":"2024-01-15T18:10:21.600363+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.600363+0000","last_undegraded":"2024-01-15T18:10:21.600363+0000","last_fullsized":"2024-01-15T18:10:21.600363+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.612587+0000","last_change":"2024-01-15T18:10:21.608384+0000","last_active":"2024-01-15T18:10:21.612587+0000","last_peered":"2024-01-15T18:10:21.612587+0000","last_clean":"2024-01-15T18:10:21.612587+0000","last_became_active":"2024-01-15T18:10:21.607947+0000","last_became_peered":"2024-01-15T18:10:21.607947+0000","last_unstale":"2024-01-15T18:10:21.612587+0000","last_undegraded":"2024-01-15T18:10:21.612587+0000","last_fullsized":"2024-01-15T18:10:21.612587+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:12:47.963640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.595935+0000","last_change":"2024-01-15T18:10:21.595935+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.595935+0000","last_undegraded":"2024-01-15T18:10:21.595935+0000","last_fullsized":"2024-01-15T18:10:21.595935+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622275+0000","last_change":"2024-01-15T18:10:21.620127+0000","last_active":"2024-01-15T18:10:21.622275+0000","last_peered":"2024-01-15T18:10:21.622275+0000","last_clean":"2024-01-15T18:10:21.622275+0000","last_became_active":"2024-01-15T18:10:21.619856+0000","last_became_peered":"2024-01-15T18:10:21.619856+0000","last_unstale":"2024-01-15T18:10:21.622275+0000","last_undegraded":"2024-01-15T18:10:21.622275+0000","last_fullsized":"2024-01-15T18:10:21.622275+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:31:29.108419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.605332+0000","last_change":"2024-01-15T18:10:21.605332+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.605332+0000","last_undegraded":"2024-01-15T18:10:21.605332+0000","last_fullsized":"2024-01-15T18:10:21.605332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622081+0000","last_change":"2024-01-15T18:10:21.619938+0000","last_active":"2024-01-15T18:10:21.622081+0000","last_peered":"2024-01-15T18:10:21.622081+0000","last_clean":"2024-01-15T18:10:21.622081+0000","last_became_active":"2024-01-15T18:10:21.618751+0000","last_became_peered":"2024-01-15T18:10:21.618751+0000","last_unstale":"2024-01-15T18:10:21.622081+0000","last_undegraded":"2024-01-15T18:10:21.622081+0000","last_fullsized":"2024-01-15T18:10:21.622081+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:52.362159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.610825+0000","last_change":"2024-01-15T18:10:21.610825+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.610825+0000","last_undegraded":"2024-01-15T18:10:21.610825+0000","last_fullsized":"2024-01-15T18:10:21.610825+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.622264+0000","last_change":"2024-01-15T18:10:21.618976+0000","last_active":"2024-01-15T18:10:21.622264+0000","last_peered":"2024-01-15T18:10:21.622264+0000","last_clean":"2024-01-15T18:10:21.622264+0000","last_became_active":"2024-01-15T18:10:21.618682+0000","last_became_peered":"2024-01-15T18:10:21.618682+0000","last_unstale":"2024-01-15T18:10:21.622264+0000","last_undegraded":"2024-01-15T18:10:21.622264+0000","last_fullsized":"2024-01-15T18:10:21.622264+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:12:47.963640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.603686+0000","last_change":"2024-01-15T18:10:21.603686+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.603686+0000","last_undegraded":"2024-01-15T18:10:21.603686+0000","last_fullsized":"2024-01-15T18:10:21.603686+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.624207+0000","last_change":"2024-01-15T18:10:21.623001+0000","last_active":"2024-01-15T18:10:21.624207+0000","last_peered":"2024-01-15T18:10:21.624207+0000","last_clean":"2024-01-15T18:10:21.624207+0000","last_became_active":"2024-01-15T18:10:21.622691+0000","last_became_peered":"2024-01-15T18:10:21.622691+0000","last_unstale":"2024-01-15T18:10:21.624207+0000","last_undegraded":"2024-01-15T18:10:21.624207+0000","last_fullsized":"2024-01-15T18:10:21.624207+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:15:30.543941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.601674+0000","last_change":"2024-01-15T18:10:21.601674+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.601674+0000","last_undegraded":"2024-01-15T18:10:21.601674+0000","last_fullsized":"2024-01-15T18:10:21.601674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.620715+0000","last_change":"2024-01-15T18:10:21.618463+0000","last_active":"2024-01-15T18:10:21.620715+0000","last_peered":"2024-01-15T18:10:21.620715+0000","last_clean":"2024-01-15T18:10:21.620715+0000","last_became_active":"2024-01-15T18:10:21.618150+0000","last_became_peered":"2024-01-15T18:10:21.618150+0000","last_unstale":"2024-01-15T18:10:21.620715+0000","last_undegraded":"2024-01-15T18:10:21.620715+0000","last_fullsized":"2024-01-15T18:10:21.620715+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:56:15.951926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599443+0000","last_change":"2024-01-15T18:10:21.599443+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599443+0000","last_undegraded":"2024-01-15T18:10:21.599443+0000","last_fullsized":"2024-01-15T18:10:21.599443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.617800+0000","last_change":"2024-01-15T18:10:21.613099+0000","last_active":"2024-01-15T18:10:21.617800+0000","last_peered":"2024-01-15T18:10:21.617800+0000","last_clean":"2024-01-15T18:10:21.617800+0000","last_became_active":"2024-01-15T18:10:21.612818+0000","last_became_peered":"2024-01-15T18:10:21.612818+0000","last_unstale":"2024-01-15T18:10:21.617800+0000","last_undegraded":"2024-01-15T18:10:21.617800+0000","last_fullsized":"2024-01-15T18:10:21.617800+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:19:34.484071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.611680+0000","last_change":"2024-01-15T18:10:21.611680+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.611680+0000","last_undegraded":"2024-01-15T18:10:21.611680+0000","last_fullsized":"2024-01-15T18:10:21.611680+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.614848+0000","last_change":"2024-01-15T18:10:21.610659+0000","last_active":"2024-01-15T18:10:21.614848+0000","last_peered":"2024-01-15T18:10:21.614848+0000","last_clean":"2024-01-15T18:10:21.614848+0000","last_became_active":"2024-01-15T18:10:21.610364+0000","last_became_peered":"2024-01-15T18:10:21.610364+0000","last_unstale":"2024-01-15T18:10:21.614848+0000","last_undegraded":"2024-01-15T18:10:21.614848+0000","last_fullsized":"2024-01-15T18:10:21.614848+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.599272+0000","last_change":"2024-01-15T18:10:21.599272+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.599272+0000","last_undegraded":"2024-01-15T18:10:21.599272+0000","last_fullsized":"2024-01-15T18:10:21.599272+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.616668+0000","last_change":"2024-01-15T18:10:21.613230+0000","last_active":"2024-01-15T18:10:21.616668+0000","last_peered":"2024-01-15T18:10:21.616668+0000","last_clean":"2024-01-15T18:10:21.616668+0000","last_became_active":"2024-01-15T18:10:21.612943+0000","last_became_peered":"2024-01-15T18:10:21.612943+0000","last_unstale":"2024-01-15T18:10:21.616668+0000","last_undegraded":"2024-01-15T18:10:21.616668+0000","last_fullsized":"2024-01-15T18:10:21.616668+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:52:25.395916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.594607+0000","last_change":"2024-01-15T18:10:21.594607+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.594607+0000","last_undegraded":"2024-01-15T18:10:21.594607+0000","last_fullsized":"2024-01-15T18:10:21.594607+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.623244+0000","last_change":"2024-01-15T18:10:21.621660+0000","last_active":"2024-01-15T18:10:21.623244+0000","last_peered":"2024-01-15T18:10:21.623244+0000","last_clean":"2024-01-15T18:10:21.623244+0000","last_became_active":"2024-01-15T18:10:21.620662+0000","last_became_peered":"2024-01-15T18:10:21.620662+0000","last_unstale":"2024-01-15T18:10:21.623244+0000","last_undegraded":"2024-01-15T18:10:21.623244+0000","last_fullsized":"2024-01-15T18:10:21.623244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:22:25.948319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-01-15T18:10:21.597443+0000","last_change":"2024-01-15T18:10:21.597443+0000","last_active":"2024-01-15T18:10:21.572319+0000","last_peered":"2024-01-15T18:10:21.572319+0000","last_clean":"2024-01-15T18:10:21.572319+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-01-15T18:10:21.597443+0000","last_undegraded":"2024-01-15T18:10:21.597443+0000","last_fullsized":"2024-01-15T18:10:21.597443+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-01-15T18:10:21.625143+0000","last_change":"2024-01-15T18:10:21.624625+0000","last_active":"2024-01-15T18:10:21.625143+0000","last_peered":"2024-01-15T18:10:21.625143+0000","last_clean":"2024-01-15T18:10:21.625143+0000","last_became_active":"2024-01-15T18:10:21.624271+0000","last_became_peered":"2024-01-15T18:10:21.624271+0000","last_unstale":"2024-01-15T18:10:21.625143+0000","last_undegraded":"2024-01-15T18:10:21.625143+0000","last_fullsized":"2024-01-15T18:10:21.625143+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:55.724175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":233,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-01-15T18:10:20.590380+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:10:20.590380+0000","last_peered":"2024-01-15T18:10:20.590380+0000","last_clean":"2024-01-15T18:10:20.590380+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:10:20.590380+0000","last_undegraded":"2024-01-15T18:10:20.590380+0000","last_fullsized":"2024-01-15T18:10:20.590380+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542332,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81464,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290376,"statfs":{"total":96636764160,"available":96553345024,"internally_reserved":0,"allocated":1814528,"data_stored":1117561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":81590730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3280000000000001,"15min":1.3740000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.40400000000000003},"max":{"1min":1.742,"5min":11.449999999999999,"15min":19.843},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.3759999999999999,"15min":1.629},"min":{"1min":0.89200000000000002,"5min":0.61499999999999999,"15min":0.53400000000000003},"max":{"1min":7.8250000000000002,"5min":7.8250000000000002,"15min":38.137},"last":1.26}]},{"osd":1,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.319,"15min":1.411},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.45600000000000002},"max":{"1min":1.577,"5min":11.51,"15min":19.917000000000002},"last":1.046},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.4079999999999999,"15min":1.605},"min":{"1min":0.85699999999999998,"5min":0.81100000000000005,"15min":0.42999999999999999},"max":{"1min":1.841,"5min":6.7480000000000002,"15min":51.843000000000004},"last":0.97499999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.371,"15min":1.425},"min":{"1min":0.97199999999999998,"5min":0.67200000000000004,"15min":0.437},"max":{"1min":3.3900000000000001,"5min":12.061,"15min":12.061},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.399,"5min":1.4430000000000001,"15min":1.532},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.47399999999999998},"max":{"1min":1.806,"5min":11.66,"15min":13.478999999999999},"last":1.5700000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.544},"min":{"1min":0.85299999999999998,"5min":0.83199999999999996,"15min":0.47899999999999998},"max":{"1min":1.9079999999999999,"5min":7.4669999999999996,"15min":16.632000000000001},"last":1.363},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.5069999999999999,"15min":1.6160000000000001},"min":{"1min":0.97299999999999998,"5min":0.94199999999999995,"15min":0.56699999999999995},"max":{"1min":1.8520000000000001,"5min":11.611000000000001,"15min":13.439},"last":1.1679999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":2.319,"5min":1.728,"15min":1.7589999999999999},"min":{"1min":1.0169999999999999,"5min":0.81000000000000005,"15min":0.50700000000000001},"max":{"1min":17.067,"5min":23.042000000000002,"15min":23.042000000000002},"last":1.337},{"interface":"front","average":{"1min":1.425,"5min":1.5269999999999999,"15min":1.7270000000000001},"min":{"1min":0.94599999999999995,"5min":0.78800000000000003,"15min":0.51800000000000002},"max":{"1min":2.0779999999999998,"5min":16.867999999999999,"15min":38.682000000000002},"last":1.534}]},{"osd":5,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5640000000000001,"15min":1.8089999999999999},"min":{"1min":1.0329999999999999,"5min":0.89800000000000002,"15min":0.52700000000000002},"max":{"1min":1.843,"5min":11.696999999999999,"15min":25.006},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.7110000000000001,"15min":1.8400000000000001},"min":{"1min":1.0880000000000001,"5min":0.97499999999999998,"15min":0.624},"max":{"1min":2.04,"5min":22.933,"15min":22.933},"last":1.591}]},{"osd":6,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.9850000000000001,"15min":2.0419999999999998},"min":{"1min":1.1870000000000001,"5min":1.123,"15min":0.59299999999999997},"max":{"1min":2.2229999999999999,"5min":23.004000000000001,"15min":23.834},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.607,"15min":1.998},"min":{"1min":1.0680000000000001,"5min":0.84099999999999997,"15min":0.58299999999999996},"max":{"1min":2.1539999999999999,"5min":7.5220000000000002,"15min":63.932000000000002},"last":1.7490000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542331,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80884,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290956,"statfs":{"total":96636764160,"available":96553938944,"internally_reserved":0,"allocated":1220608,"data_stored":527193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81590726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.825,"5min":1.512,"15min":1.4139999999999999},"min":{"1min":0.79100000000000004,"5min":0.55900000000000005,"15min":0.53100000000000003},"max":{"1min":14.794,"5min":14.794,"15min":14.794},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.345,"15min":1.407},"min":{"1min":0.80000000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":4.5430000000000001,"5min":4.5430000000000001,"15min":14.013},"last":1.8160000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.7110000000000001,"5min":1.3660000000000001,"15min":1.353},"min":{"1min":0.67900000000000005,"5min":0.63700000000000001,"15min":0.55100000000000005},"max":{"1min":15.064,"5min":15.064,"15min":15.064},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.508,"15min":1.53},"min":{"1min":0.85299999999999998,"5min":0.74199999999999999,"15min":0.66200000000000003},"max":{"1min":1.6899999999999999,"5min":23.495999999999999,"15min":23.495999999999999},"last":1.4339999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.667,"15min":1.587},"min":{"1min":0.81000000000000005,"5min":0.73799999999999999,"15min":0.59399999999999997},"max":{"1min":1.7490000000000001,"5min":23.087,"15min":23.087},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.871,"5min":1.4910000000000001,"15min":1.4590000000000001},"min":{"1min":0.86499999999999999,"5min":0.51400000000000001,"15min":0.504},"max":{"1min":14.94,"5min":14.94,"15min":14.94},"last":0.99199999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.9750000000000001,"5min":1.7629999999999999,"15min":1.6950000000000001},"min":{"1min":0.95299999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":15.151999999999999,"5min":15.151999999999999,"15min":15.151999999999999},"last":1.244},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5580000000000001,"15min":1.601},"min":{"1min":0.89600000000000002,"5min":0.76100000000000001,"15min":0.70699999999999996},"max":{"1min":4.6849999999999996,"5min":13.708,"15min":14.066000000000001},"last":1.9059999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.577,"15min":1.661},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.51800000000000002},"max":{"1min":1.8720000000000001,"5min":23.327000000000002,"15min":23.327000000000002},"last":1.1180000000000001},{"interface":"front","average":{"1min":2.4329999999999998,"5min":1.71,"15min":1.716},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.45800000000000002},"max":{"1min":15.239000000000001,"5min":15.239000000000001,"15min":17.289000000000001},"last":1.375}]},{"osd":5,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.4039999999999999,"15min":1.5669999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51600000000000001},"max":{"1min":4.9260000000000002,"5min":4.9260000000000002,"15min":14.176},"last":1.702},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.8580000000000001,"15min":1.8700000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.54700000000000004},"max":{"1min":2.044,"5min":23.417999999999999,"15min":23.417999999999999},"last":1.5289999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.716,"15min":1.7689999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":4.8239999999999998,"5min":16.454000000000001,"15min":21.265000000000001},"last":1.9890000000000001},{"interface":"front","average":{"1min":1.7370000000000001,"5min":2.0089999999999999,"15min":1.9950000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.59699999999999998},"max":{"1min":3.8879999999999999,"5min":23.738,"15min":23.738},"last":1.6160000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542331,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58752,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313088,"statfs":{"total":96636764160,"available":96576602112,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.234,"15min":1.2829999999999999},"min":{"1min":0.73399999999999999,"5min":0.54200000000000004,"15min":0.47799999999999998},"max":{"1min":2.2839999999999998,"5min":7.6580000000000004,"15min":19.266999999999999},"last":1.55},{"interface":"front","average":{"1min":1.254,"5min":1.196,"15min":1.353},"min":{"1min":0.82299999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.738,"5min":2.1869999999999998,"15min":17.184999999999999},"last":1.028}]},{"osd":1,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2450000000000001,"15min":1.339},"min":{"1min":0.747,"5min":0.60699999999999998,"15min":0.499},"max":{"1min":2.355,"5min":5.3879999999999999,"15min":16.297000000000001},"last":1.472},{"interface":"front","average":{"1min":1.23,"5min":1.359,"15min":1.3280000000000001},"min":{"1min":0.78900000000000003,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8580000000000001,"5min":7.7649999999999997,"15min":9.2379999999999995},"last":1.3799999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.228,"15min":1.26},"min":{"1min":0.80400000000000005,"5min":0.621,"15min":0.621},"max":{"1min":1.5429999999999999,"5min":2.1110000000000002,"15min":8.1820000000000004},"last":0.876},{"interface":"front","average":{"1min":1.431,"5min":1.349,"15min":1.4330000000000001},"min":{"1min":0.97299999999999998,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":16.158999999999999},"last":1.6359999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.427,"15min":1.486},"min":{"1min":0.89400000000000002,"5min":0.63700000000000001,"15min":0.56100000000000005},"max":{"1min":2.5179999999999998,"5min":7.8609999999999998,"15min":14.914},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.446,"15min":1.4550000000000001},"min":{"1min":1.0189999999999999,"5min":0.64600000000000002,"15min":0.63300000000000001},"max":{"1min":1.8109999999999999,"5min":2.0139999999999998,"15min":7.0110000000000001},"last":1.139}]},{"osd":4,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.329,"15min":1.5},"min":{"1min":0.80800000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.4689999999999999,"5min":2.4689999999999999,"15min":16.414999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.51,"15min":1.6319999999999999},"min":{"1min":0.56200000000000006,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":2.665,"5min":7.5359999999999996,"15min":17.991},"last":1.8500000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.45,"15min":1.5009999999999999},"min":{"1min":0.72699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.881,"5min":3.7559999999999998,"15min":6.4630000000000001},"last":1.244},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4830000000000001,"15min":1.599},"min":{"1min":1.0009999999999999,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":2.7010000000000001,"5min":2.7010000000000001,"15min":16.364000000000001},"last":1.7949999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.669,"15min":1.6220000000000001},"min":{"1min":0.78900000000000003,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.399,"5min":7.96,"15min":9.0600000000000005},"last":1.766},{"interface":"front","average":{"1min":1.587,"5min":1.6000000000000001,"15min":1.712},"min":{"1min":0.98599999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.6059999999999999,"5min":2.6059999999999999,"15min":19.356999999999999},"last":1.347}]}]},{"osd":4,"up_from":14,"seq":60129542331,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76736,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295104,"statfs":{"total":96636764160,"available":96558186496,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.286,"15min":1.341},"min":{"1min":0.70999999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":3.8170000000000002,"5min":3.8170000000000002,"15min":11.413},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3420000000000001,"15min":1.3919999999999999},"min":{"1min":0.72599999999999998,"5min":0.53800000000000003,"15min":0.53700000000000003},"max":{"1min":2.7919999999999998,"5min":3.726,"15min":9.7439999999999998},"last":1.5740000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.341,"15min":1.3660000000000001},"min":{"1min":0.73399999999999999,"5min":0.58899999999999997,"15min":0.45000000000000001},"max":{"1min":2.8700000000000001,"5min":11.128,"15min":11.987},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.2330000000000001,"15min":1.3089999999999999},"min":{"1min":0.81100000000000005,"5min":0.69499999999999995,"15min":0.55300000000000005},"max":{"1min":1.909,"5min":3.3610000000000002,"15min":16.058},"last":1.397}]},{"osd":2,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3029999999999999,"15min":1.496},"min":{"1min":0.81299999999999994,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":3.6070000000000002,"5min":5.0960000000000001,"15min":13.865},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.401,"15min":1.5229999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.69499999999999995},"max":{"1min":1.899,"5min":5.1340000000000003,"15min":13.903},"last":1.325}]},{"osd":3,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.4430000000000001,"15min":1.6080000000000001},"min":{"1min":0.85999999999999999,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":3.8999999999999999,"5min":10.765000000000001,"15min":26.055},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.542,"5min":1.3500000000000001,"15min":1.4910000000000001},"min":{"1min":0.76700000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":13.25},"last":1.466}]},{"osd":5,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.331,"15min":1.5700000000000001},"min":{"1min":0.86799999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":3.7189999999999999,"5min":3.7189999999999999,"15min":29.422000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.5329999999999999,"15min":1.538},"min":{"1min":0.85499999999999998,"5min":0.62,"15min":0.42199999999999999},"max":{"1min":3.0139999999999998,"5min":12.332000000000001,"15min":12.332000000000001},"last":1.2669999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.498,"15min":1.546},"min":{"1min":0.97399999999999998,"5min":0.59699999999999998,"15min":0.45200000000000001},"max":{"1min":1.9590000000000001,"5min":11.09,"15min":18.260000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.744,"5min":1.6299999999999999,"15min":1.696},"min":{"1min":1.1519999999999999,"5min":0.78700000000000003,"15min":0.48099999999999998},"max":{"1min":3.968,"5min":9.8740000000000006,"15min":11.657999999999999},"last":1.365}]},{"osd":7,"last update":"Mon Jan 15 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":1.724,"5min":1.6739999999999999,"15min":1.788},"min":{"1min":1.133,"5min":0.68200000000000005,"15min":0.503},"max":{"1min":3.0680000000000001,"5min":12.247,"15min":12.247},"last":1.125},{"interface":"front","average":{"1min":1.857,"5min":1.756,"15min":1.8660000000000001},"min":{"1min":1.3839999999999999,"5min":0.81899999999999995,"15min":0.53600000000000003},"max":{"1min":4.0209999999999999,"5min":12.311999999999999,"15min":12.311999999999999},"last":1.5189999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542333,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72580,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299260,"statfs":{"total":96636764160,"available":96562442240,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.113,"15min":1.2210000000000001},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.33300000000000002},"max":{"1min":1.284,"5min":9.2899999999999991,"15min":18.399999999999999},"last":1.304},{"interface":"front","average":{"1min":1.048,"5min":1.127,"15min":1.387},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":1.8140000000000001,"5min":2.23,"15min":21.047000000000001},"last":1.137}]},{"osd":1,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.92400000000000004,"15min":1.272},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.415,"5min":3.0409999999999999,"15min":17.777999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.3680000000000001,"15min":1.288},"min":{"1min":0.57399999999999995,"5min":0.46300000000000002,"15min":0.39500000000000002},"max":{"1min":1.6890000000000001,"5min":11.228999999999999,"15min":11.228999999999999},"last":0.77900000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.224,"15min":1.2789999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.518,"5min":2.7519999999999998,"15min":10.964},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.5489999999999999,"15min":1.554},"min":{"1min":0.60799999999999998,"5min":0.57099999999999995,"15min":0.54000000000000004},"max":{"1min":1.857,"5min":9.4920000000000009,"15min":20.495000000000001},"last":0.72399999999999998}]},{"osd":4,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.401,"15min":1.5589999999999999},"min":{"1min":0.871,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.6419999999999999,"5min":6.319,"15min":13.536},"last":1.395},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.456,"15min":1.6399999999999999},"min":{"1min":0.748,"5min":0.63700000000000001,"15min":0.54600000000000004},"max":{"1min":1.736,"5min":16.398,"15min":16.398},"last":1.3420000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.681,"15min":1.6950000000000001},"min":{"1min":1.046,"5min":0.875,"15min":0.57299999999999995},"max":{"1min":1.6819999999999999,"5min":9.5690000000000008,"15min":20.606000000000002},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5589999999999999,"15min":1.665},"min":{"1min":0.97299999999999998,"5min":0.70199999999999996,"15min":0.68000000000000005},"max":{"1min":1.758,"5min":5.6980000000000004,"15min":11.603},"last":1.637}]},{"osd":6,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.615,"15min":1.7949999999999999},"min":{"1min":0.81000000000000005,"5min":0.61399999999999999,"15min":0.53500000000000003},"max":{"1min":2.0870000000000002,"5min":13.648999999999999,"15min":13.648999999999999},"last":1.25},{"interface":"front","average":{"1min":1.502,"5min":1.8109999999999999,"15min":1.8220000000000001},"min":{"1min":0.98699999999999999,"5min":0.78300000000000003,"15min":0.65800000000000003},"max":{"1min":2.0249999999999999,"5min":13.679,"15min":20.556000000000001},"last":1.093}]},{"osd":7,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.552,"15min":1.8480000000000001},"min":{"1min":0.78800000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8260000000000001,"5min":3.5590000000000002,"15min":28.843},"last":1.528},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.5760000000000001,"15min":1.857},"min":{"1min":0.97999999999999998,"5min":0.72799999999999998,"15min":0.67200000000000004},"max":{"1min":1.968,"5min":9.4100000000000001,"15min":15.143000000000001},"last":1.4570000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542333,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68420,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303420,"statfs":{"total":96636764160,"available":96566702080,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.129,"15min":1.095},"min":{"1min":0.68300000000000005,"5min":0.32100000000000001,"15min":0.26600000000000001},"max":{"1min":2.9359999999999999,"5min":12.278,"15min":14.427},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.159,"5min":1.4970000000000001,"15min":1.337},"min":{"1min":0.68799999999999994,"5min":0.50900000000000001,"15min":0.39300000000000002},"max":{"1min":2.25,"5min":15.314,"15min":15.314},"last":1.399}]},{"osd":1,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.5980000000000001,"15min":1.4470000000000001},"min":{"1min":0.80700000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.7029999999999998,"5min":34.463999999999999,"15min":34.463999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":1.351,"5min":1.7989999999999999,"15min":1.454},"min":{"1min":0.76800000000000002,"5min":0.54200000000000004,"15min":0.373},"max":{"1min":1.881,"5min":25.859999999999999,"15min":25.859999999999999},"last":1.2729999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.9430000000000001,"5min":1.26,"15min":1.4179999999999999},"min":{"1min":0.76200000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":12.449999999999999,"5min":12.449999999999999,"15min":19.535},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.804,"15min":1.6240000000000001},"min":{"1min":0.95999999999999996,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":2.8050000000000002,"5min":22.783999999999999,"15min":22.783999999999999},"last":1.03}]},{"osd":4,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.603,"15min":1.784},"min":{"1min":1.113,"5min":0.73199999999999998,"15min":0.55700000000000005},"max":{"1min":2.8759999999999999,"5min":12.728,"15min":29.986999999999998},"last":1.155},{"interface":"front","average":{"1min":1.607,"5min":1.7609999999999999,"15min":1.742},"min":{"1min":1.0449999999999999,"5min":0.71299999999999997,"15min":0.56799999999999995},"max":{"1min":2.286,"5min":15.553000000000001,"15min":21.199999999999999},"last":1.679}]},{"osd":5,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":2.9350000000000001,"5min":1.855,"15min":1.746},"min":{"1min":0.84299999999999997,"5min":0.63700000000000001,"15min":0.51500000000000001},"max":{"1min":14.066000000000001,"5min":14.066000000000001,"15min":19.681999999999999},"last":1.089},{"interface":"front","average":{"1min":2.5379999999999998,"5min":2.0179999999999998,"15min":1.823},"min":{"1min":1.2010000000000001,"5min":0.84799999999999998,"15min":0.60899999999999999},"max":{"1min":14.025,"5min":14.025,"15min":14.702999999999999},"last":1.3600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.6279999999999999,"5min":2.032,"15min":1.7869999999999999},"min":{"1min":1.1599999999999999,"5min":0.76800000000000002,"15min":0.68000000000000005},"max":{"1min":2.2919999999999998,"5min":15.683999999999999,"15min":15.683999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":2.2999999999999998,"5min":2.0819999999999999,"15min":1.853},"min":{"1min":0.96999999999999997,"5min":0.71999999999999997,"15min":0.58899999999999997},"max":{"1min":12.629,"5min":12.629,"15min":19.783000000000001},"last":1.2410000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:09:23 2024","interfaces":[{"interface":"back","average":{"1min":1.7969999999999999,"5min":2.0129999999999999,"15min":2.0710000000000002},"min":{"1min":1.4510000000000001,"5min":0.77800000000000002,"15min":0.63100000000000001},"max":{"1min":2.3279999999999998,"5min":17.099,"15min":57.006999999999998},"last":1.597},{"interface":"front","average":{"1min":2.3919999999999999,"5min":2.1600000000000001,"15min":2.1930000000000001},"min":{"1min":1.081,"5min":1.0089999999999999,"15min":0.56100000000000005},"max":{"1min":12.754,"5min":22.853999999999999,"15min":22.853999999999999},"last":1.621}]}]},{"osd":1,"up_from":14,"seq":60129542332,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72580,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299260,"statfs":{"total":96636764160,"available":96562442240,"internally_reserved":0,"allocated":1232896,"data_stored":533524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.363,"15min":1.2070000000000001},"min":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.34399999999999997},"max":{"1min":2.3839999999999999,"5min":30.460999999999999,"15min":30.460999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.1759999999999999,"15min":1.117},"min":{"1min":0.61499999999999999,"5min":0.38700000000000001,"15min":0.314},"max":{"1min":2.0960000000000001,"5min":4.4059999999999997,"15min":18.658000000000001},"last":1.2310000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.198,"15min":1.1970000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":2.0619999999999998,"5min":7.117,"15min":21.126000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.752,"15min":1.5620000000000001},"min":{"1min":0.75700000000000001,"5min":0.70899999999999996,"15min":0.48699999999999999},"max":{"1min":2.1499999999999999,"5min":30.593,"15min":30.593},"last":1.042}]},{"osd":3,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.6719999999999999,"15min":1.617},"min":{"1min":0.76100000000000001,"5min":0.58099999999999996,"15min":0.48599999999999999},"max":{"1min":2.0579999999999998,"5min":30.547000000000001,"15min":30.547000000000001},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3480000000000001,"15min":1.3420000000000001},"min":{"1min":0.84999999999999998,"5min":0.52700000000000002,"15min":0.437},"max":{"1min":2.21,"5min":5.4729999999999999,"15min":15.510999999999999},"last":1.4730000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.663,"15min":1.6990000000000001},"min":{"1min":1.111,"5min":0.75900000000000001,"15min":0.72499999999999998},"max":{"1min":2.7599999999999998,"5min":8.8170000000000002,"15min":14.827},"last":1.585},{"interface":"front","average":{"1min":1.53,"5min":1.5429999999999999,"15min":1.657},"min":{"1min":0.95699999999999996,"5min":0.72999999999999998,"15min":0.69299999999999995},"max":{"1min":2.359,"5min":3.7440000000000002,"15min":14.851000000000001},"last":1.619}]},{"osd":5,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5349999999999999,"15min":1.6160000000000001},"min":{"1min":1.036,"5min":0.77600000000000002,"15min":0.59599999999999997},"max":{"1min":2.266,"5min":3.798,"15min":13.228999999999999},"last":1.329},{"interface":"front","average":{"1min":1.593,"5min":1.986,"15min":1.9450000000000001},"min":{"1min":1.0429999999999999,"5min":0.78700000000000003,"15min":0.68300000000000005},"max":{"1min":2.2949999999999999,"5min":34.042999999999999,"15min":34.042999999999999},"last":1.375}]},{"osd":6,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.579,"15min":1.7829999999999999},"min":{"1min":0.90100000000000002,"5min":0.83899999999999997,"15min":0.64100000000000001},"max":{"1min":2.5550000000000002,"5min":3.5539999999999998,"15min":34.158000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.736,"15min":1.6299999999999999},"min":{"1min":1.115,"5min":0.67900000000000005,"15min":0.62},"max":{"1min":4.1699999999999999,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.6779999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":2.0979999999999999,"15min":1.8660000000000001},"min":{"1min":0.95999999999999996,"5min":0.77300000000000002,"15min":0.64600000000000002},"max":{"1min":2.5750000000000002,"5min":33.659999999999997,"15min":33.659999999999997},"last":1.73},{"interface":"front","average":{"1min":1.694,"5min":1.8540000000000001,"15min":1.8009999999999999},"min":{"1min":1.002,"5min":0.81000000000000005,"15min":0.60499999999999998},"max":{"1min":2.613,"5min":11.882999999999999,"15min":24.289999999999999},"last":1.7949999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542333,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69000,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302840,"statfs":{"total":96636764160,"available":96566108160,"internally_reserved":0,"allocated":1826816,"data_stored":1123892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2210000000000001,"15min":1.083},"min":{"1min":0.437,"5min":0.42799999999999999,"15min":0.312},"max":{"1min":2.8929999999999998,"5min":12.050000000000001,"15min":12.050000000000001},"last":1.212},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.3340000000000001},"min":{"1min":0.50900000000000001,"5min":0.49099999999999999,"15min":0.36199999999999999},"max":{"1min":2.9780000000000002,"5min":11.962,"15min":11.962},"last":1.169}]},{"osd":2,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.179,"15min":1.2090000000000001},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.27600000000000002},"max":{"1min":1.595,"5min":7.4560000000000004,"15min":11.59},"last":0.877},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.1459999999999999,"15min":1.167},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35899999999999999},"max":{"1min":1.5649999999999999,"5min":7.3239999999999998,"15min":14.512},"last":1.5649999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.425,"15min":1.4319999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.41699999999999998},"max":{"1min":1.5669999999999999,"5min":11.904,"15min":11.904},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1379999999999999,"15min":1.2490000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.29599999999999999},"max":{"1min":1.6850000000000001,"5min":3.202,"15min":14.74},"last":0.81299999999999994}]},{"osd":4,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.375,"15min":1.5900000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6140000000000001,"5min":7.4000000000000004,"15min":32.115000000000002},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.571,"15min":1.702},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.67700000000000005},"max":{"1min":1.8260000000000001,"5min":9.0790000000000006,"15min":15.975},"last":1.288}]},{"osd":5,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.556,"15min":1.722},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":1.6200000000000001,"5min":12.006,"15min":32.174999999999997},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.514,"15min":1.6359999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.442},"max":{"1min":1.8939999999999999,"5min":4.7119999999999997,"15min":17.225000000000001},"last":1.605}]},{"osd":6,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.7849999999999999,"15min":1.7989999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.70299999999999996},"max":{"1min":3.4399999999999999,"5min":24.039999999999999,"15min":24.039999999999999},"last":1.3460000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.7549999999999999,"15min":1.794},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.625},"max":{"1min":1.7450000000000001,"5min":18.143999999999998,"15min":32.216999999999999},"last":1.4490000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.853,"15min":2.1230000000000002},"min":{"1min":0.754,"5min":0.754,"15min":0.69099999999999995},"max":{"1min":1.7350000000000001,"5min":26.427,"15min":34.131},"last":1.504},{"interface":"front","average":{"1min":1.474,"5min":2.089,"15min":2.0379999999999998},"min":{"1min":0.753,"5min":0.753,"15min":0.61799999999999999},"max":{"1min":3.9790000000000001,"5min":44.694000000000003,"15min":44.694000000000003},"last":1.4850000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:10:28.174 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.611673+0000', 'last_change': '2024-01-15T18:10:21.611673+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.611673+0000', 'last_undegraded': '2024-01-15T18:10:21.611673+0000', 'last_fullsized': '2024-01-15T18:10:21.611673+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.611314+0000', 'last_change': '2024-01-15T18:10:21.611314+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.611314+0000', 'last_undegraded': '2024-01-15T18:10:21.611314+0000', 'last_fullsized': '2024-01-15T18:10:21.611314+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.599418+0000', 'last_change': '2024-01-15T18:10:21.599418+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.599418+0000', 'last_undegraded': '2024-01-15T18:10:21.599418+0000', 'last_fullsized': '2024-01-15T18:10:21.599418+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.601091+0000', 'last_change': '2024-01-15T18:10:21.601091+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.601091+0000', 'last_undegraded': '2024-01-15T18:10:21.601091+0000', 'last_fullsized': '2024-01-15T18:10:21.601091+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.596994+0000', 'last_change': '2024-01-15T18:10:21.596994+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.596994+0000', 'last_undegraded': '2024-01-15T18:10:21.596994+0000', 'last_fullsized': '2024-01-15T18:10:21.596994+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2024-01-15T18:10:28.175 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.607608+0000', 'last_change': '2024-01-15T18:10:21.607608+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.607608+0000', 'last_undegraded': '2024-01-15T18:10:21.607608+0000', 'last_fullsized': '2024-01-15T18:10:21.607608+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.611612+0000', 'last_change': '2024-01-15T18:10:21.611612+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.611612+0000', 'last_undegraded': '2024-01-15T18:10:21.611612+0000', 'last_fullsized': '2024-01-15T18:10:21.611612+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.610269+0000', 'last_change': '2024-01-15T18:10:21.610269+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.610269+0000', 'last_undegraded': '2024-01-15T18:10:21.610269+0000', 'last_fullsized': '2024-01-15T18:10:21.610269+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.616486+0000', 'last_change': '2024-01-15T18:10:21.616486+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.616486+0000', 'last_undegraded': '2024-01-15T18:10:21.616486+0000', 'last_fullsized': '2024-01-15T18:10:21.616486+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.597244+0000', 'last_change': '2024-01-15T18:10:21.597244+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.597244+0000', 'last_undegraded': '2024-01-15T18:10:21.597244+0000', 'last_fullsized': '2024-01-15T18:10:21.597244+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2024-01-15T18:10:28.176 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.616047+0000', 'last_change': '2024-01-15T18:10:21.616047+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.616047+0000', 'last_undegraded': '2024-01-15T18:10:21.616047+0000', 'last_fullsized': '2024-01-15T18:10:21.616047+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.605943+0000', 'last_change': '2024-01-15T18:10:21.605943+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.605943+0000', 'last_undegraded': '2024-01-15T18:10:21.605943+0000', 'last_fullsized': '2024-01-15T18:10:21.605943+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.602017+0000', 'last_change': '2024-01-15T18:10:21.602017+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.602017+0000', 'last_undegraded': '2024-01-15T18:10:21.602017+0000', 'last_fullsized': '2024-01-15T18:10:21.602017+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.600750+0000', 'last_change': '2024-01-15T18:10:21.600750+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.600750+0000', 'last_undegraded': '2024-01-15T18:10:21.600750+0000', 'last_fullsized': '2024-01-15T18:10:21.600750+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.600926+0000', 'last_change': '2024-01-15T18:10:21.600926+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.600926+0000', 'last_undegraded': '2024-01-15T18:10:21.600926+0000', 'last_fullsized': '2024-01-15T18:10:21.600926+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2024-01-15T18:10:28.177 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.608161+0000', 'last_change': '2024-01-15T18:10:21.608161+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.608161+0000', 'last_undegraded': '2024-01-15T18:10:21.608161+0000', 'last_fullsized': '2024-01-15T18:10:21.608161+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.608276+0000', 'last_change': '2024-01-15T18:10:21.608276+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.608276+0000', 'last_undegraded': '2024-01-15T18:10:21.608276+0000', 'last_fullsized': '2024-01-15T18:10:21.608276+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.601345+0000', 'last_change': '2024-01-15T18:10:21.601345+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.601345+0000', 'last_undegraded': '2024-01-15T18:10:21.601345+0000', 'last_fullsized': '2024-01-15T18:10:21.601345+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.596934+0000', 'last_change': '2024-01-15T18:10:21.596934+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.596934+0000', 'last_undegraded': '2024-01-15T18:10:21.596934+0000', 'last_fullsized': '2024-01-15T18:10:21.596934+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.604940+0000', 'last_change': '2024-01-15T18:10:21.604940+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.604940+0000', 'last_undegraded': '2024-01-15T18:10:21.604940+0000', 'last_fullsized': '2024-01-15T18:10:21.604940+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.597709+0000', 'last_change': '2024-01-15T18:10:21.597709+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.597709+0000', 'last_undegraded': '2024-01-15T18:10:21.597709+0000', 'last_fullsized': '2024-01-15T18:10:21.597709+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2024-01-15T18:10:28.178 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.608794+0000', 'last_change': '2024-01-15T18:10:21.608794+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.608794+0000', 'last_undegraded': '2024-01-15T18:10:21.608794+0000', 'last_fullsized': '2024-01-15T18:10:21.608794+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.608380+0000', 'last_change': '2024-01-15T18:10:21.608380+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.608380+0000', 'last_undegraded': '2024-01-15T18:10:21.608380+0000', 'last_fullsized': '2024-01-15T18:10:21.608380+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.606909+0000', 'last_change': '2024-01-15T18:10:21.606909+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.606909+0000', 'last_undegraded': '2024-01-15T18:10:21.606909+0000', 'last_fullsized': '2024-01-15T18:10:21.606909+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.596824+0000', 'last_change': '2024-01-15T18:10:21.596824+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.596824+0000', 'last_undegraded': '2024-01-15T18:10:21.596824+0000', 'last_fullsized': '2024-01-15T18:10:21.596824+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.608926+0000', 'last_change': '2024-01-15T18:10:21.608926+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.608926+0000', 'last_undegraded': '2024-01-15T18:10:21.608926+0000', 'last_fullsized': '2024-01-15T18:10:21.608926+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.615205+0000', 'last_change': '2024-01-15T18:10:21.615205+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.615205+0000', 'last_undegraded': '2024-01-15T18:10:21.615205+0000', 'last_fullsized': '2024-01-15T18:10:21.615205+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:10:28.179 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.601293+0000', 'last_change': '2024-01-15T18:10:21.601293+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.601293+0000', 'last_undegraded': '2024-01-15T18:10:21.601293+0000', 'last_fullsized': '2024-01-15T18:10:21.601293+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.606505+0000', 'last_change': '2024-01-15T18:10:21.606505+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.606505+0000', 'last_undegraded': '2024-01-15T18:10:21.606505+0000', 'last_fullsized': '2024-01-15T18:10:21.606505+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.600333+0000', 'last_change': '2024-01-15T18:10:21.600333+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.600333+0000', 'last_undegraded': '2024-01-15T18:10:21.600333+0000', 'last_fullsized': '2024-01-15T18:10:21.600333+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.596109+0000', 'last_change': '2024-01-15T18:10:21.596109+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.596109+0000', 'last_undegraded': '2024-01-15T18:10:21.596109+0000', 'last_fullsized': '2024-01-15T18:10:21.596109+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.604227+0000', 'last_change': '2024-01-15T18:10:21.604227+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.604227+0000', 'last_undegraded': '2024-01-15T18:10:21.604227+0000', 'last_fullsized': '2024-01-15T18:10:21.604227+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:10:28.180 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.599903+0000', 'last_change': '2024-01-15T18:10:21.599903+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.599903+0000', 'last_undegraded': '2024-01-15T18:10:21.599903+0000', 'last_fullsized': '2024-01-15T18:10:21.599903+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.598656+0000', 'last_change': '2024-01-15T18:10:21.598656+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.598656+0000', 'last_undegraded': '2024-01-15T18:10:21.598656+0000', 'last_fullsized': '2024-01-15T18:10:21.598656+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.607177+0000', 'last_change': '2024-01-15T18:10:21.607177+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.607177+0000', 'last_undegraded': '2024-01-15T18:10:21.607177+0000', 'last_fullsized': '2024-01-15T18:10:21.607177+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.608242+0000', 'last_change': '2024-01-15T18:10:21.608242+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.608242+0000', 'last_undegraded': '2024-01-15T18:10:21.608242+0000', 'last_fullsized': '2024-01-15T18:10:21.608242+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.599261+0000', 'last_change': '2024-01-15T18:10:21.599261+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.599261+0000', 'last_undegraded': '2024-01-15T18:10:21.599261+0000', 'last_fullsized': '2024-01-15T18:10:21.599261+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.611462+0000', 'last_change': '2024-01-15T18:10:21.611462+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.611462+0000', 'last_undegraded': '2024-01-15T18:10:21.611462+0000', 'last_fullsized': '2024-01-15T18:10:21.611462+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:10:28.181 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.595870+0000', 'last_change': '2024-01-15T18:10:21.595870+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.595870+0000', 'last_undegraded': '2024-01-15T18:10:21.595870+0000', 'last_fullsized': '2024-01-15T18:10:21.595870+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.609479+0000', 'last_change': '2024-01-15T18:10:21.609479+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.609479+0000', 'last_undegraded': '2024-01-15T18:10:21.609479+0000', 'last_fullsized': '2024-01-15T18:10:21.609479+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.605852+0000', 'last_change': '2024-01-15T18:10:21.605852+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.605852+0000', 'last_undegraded': '2024-01-15T18:10:21.605852+0000', 'last_fullsized': '2024-01-15T18:10:21.605852+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.611500+0000', 'last_change': '2024-01-15T18:10:21.611500+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.611500+0000', 'last_undegraded': '2024-01-15T18:10:21.611500+0000', 'last_fullsized': '2024-01-15T18:10:21.611500+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.598336+0000', 'last_change': '2024-01-15T18:10:21.598336+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.598336+0000', 'last_undegraded': '2024-01-15T18:10:21.598336+0000', 'last_fullsized': '2024-01-15T18:10:21.598336+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-01-15T18:10:28.182 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.614635+0000', 'last_change': '2024-01-15T18:10:21.614635+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.614635+0000', 'last_undegraded': '2024-01-15T18:10:21.614635+0000', 'last_fullsized': '2024-01-15T18:10:21.614635+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.598031+0000', 'last_change': '2024-01-15T18:10:21.598031+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.598031+0000', 'last_undegraded': '2024-01-15T18:10:21.598031+0000', 'last_fullsized': '2024-01-15T18:10:21.598031+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.597549+0000', 'last_change': '2024-01-15T18:10:21.597549+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.597549+0000', 'last_undegraded': '2024-01-15T18:10:21.597549+0000', 'last_fullsized': '2024-01-15T18:10:21.597549+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.608054+0000', 'last_change': '2024-01-15T18:10:21.608054+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.608054+0000', 'last_undegraded': '2024-01-15T18:10:21.608054+0000', 'last_fullsized': '2024-01-15T18:10:21.608054+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.601789+0000', 'last_change': '2024-01-15T18:10:21.601789+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.601789+0000', 'last_undegraded': '2024-01-15T18:10:21.601789+0000', 'last_fullsized': '2024-01-15T18:10:21.601789+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.595658+0000', 'last_change': '2024-01-15T18:10:21.595658+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.595658+0000', 'last_undegraded': '2024-01-15T18:10:21.595658+0000', 'last_fullsized': '2024-01-15T18:10:21.595658+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:10:28.183 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.601227+0000', 'last_change': '2024-01-15T18:10:21.601227+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.601227+0000', 'last_undegraded': '2024-01-15T18:10:21.601227+0000', 'last_fullsized': '2024-01-15T18:10:21.601227+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.612014+0000', 'last_change': '2024-01-15T18:10:21.612014+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.612014+0000', 'last_undegraded': '2024-01-15T18:10:21.612014+0000', 'last_fullsized': '2024-01-15T18:10:21.612014+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.599884+0000', 'last_change': '2024-01-15T18:10:21.599884+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.599884+0000', 'last_undegraded': '2024-01-15T18:10:21.599884+0000', 'last_fullsized': '2024-01-15T18:10:21.599884+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.597254+0000', 'last_change': '2024-01-15T18:10:21.597254+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.597254+0000', 'last_undegraded': '2024-01-15T18:10:21.597254+0000', 'last_fullsized': '2024-01-15T18:10:21.597254+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.600363+0000', 'last_change': '2024-01-15T18:10:21.600363+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.600363+0000', 'last_undegraded': '2024-01-15T18:10:21.600363+0000', 'last_fullsized': '2024-01-15T18:10:21.600363+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-01-15T18:10:28.184 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.595935+0000', 'last_change': '2024-01-15T18:10:21.595935+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.595935+0000', 'last_undegraded': '2024-01-15T18:10:21.595935+0000', 'last_fullsized': '2024-01-15T18:10:21.595935+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.605332+0000', 'last_change': '2024-01-15T18:10:21.605332+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.605332+0000', 'last_undegraded': '2024-01-15T18:10:21.605332+0000', 'last_fullsized': '2024-01-15T18:10:21.605332+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.610825+0000', 'last_change': '2024-01-15T18:10:21.610825+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.610825+0000', 'last_undegraded': '2024-01-15T18:10:21.610825+0000', 'last_fullsized': '2024-01-15T18:10:21.610825+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.603686+0000', 'last_change': '2024-01-15T18:10:21.603686+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.603686+0000', 'last_undegraded': '2024-01-15T18:10:21.603686+0000', 'last_fullsized': '2024-01-15T18:10:21.603686+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.601674+0000', 'last_change': '2024-01-15T18:10:21.601674+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.601674+0000', 'last_undegraded': '2024-01-15T18:10:21.601674+0000', 'last_fullsized': '2024-01-15T18:10:21.601674+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.599443+0000', 'last_change': '2024-01-15T18:10:21.599443+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.599443+0000', 'last_undegraded': '2024-01-15T18:10:21.599443+0000', 'last_fullsized': '2024-01-15T18:10:21.599443+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-01-15T18:10:28.185 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.611680+0000', 'last_change': '2024-01-15T18:10:21.611680+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.611680+0000', 'last_undegraded': '2024-01-15T18:10:21.611680+0000', 'last_fullsized': '2024-01-15T18:10:21.611680+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-01-15T18:10:28.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-01-15T18:10:28.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.599272+0000', 'last_change': '2024-01-15T18:10:21.599272+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.599272+0000', 'last_undegraded': '2024-01-15T18:10:21.599272+0000', 'last_fullsized': '2024-01-15T18:10:21.599272+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-01-15T18:10:28.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.594607+0000', 'last_change': '2024-01-15T18:10:21.594607+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.594607+0000', 'last_undegraded': '2024-01-15T18:10:21.594607+0000', 'last_fullsized': '2024-01-15T18:10:21.594607+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-01-15T18:10:28.186 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-01-15T18:10:28.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-01-15T18:10:21.597443+0000', 'last_change': '2024-01-15T18:10:21.597443+0000', 'last_active': '2024-01-15T18:10:21.572319+0000', 'last_peered': '2024-01-15T18:10:21.572319+0000', 'last_clean': '2024-01-15T18:10:21.572319+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-01-15T18:10:21.597443+0000', 'last_undegraded': '2024-01-15T18:10:21.597443+0000', 'last_fullsized': '2024-01-15T18:10:21.597443+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'last_clean_scrub_stamp': '2024-01-15T18:10:21.572319+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-01-15T18:10:28.186 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-01-15T18:10:28.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.432+0000 7f1c3d000640 1 Processor -- start 2024-01-15T18:10:28.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.433+0000 7f1c3d000640 1 -- start start 2024-01-15T18:10:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.434+0000 7f1c3d000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1c38157c30 0x7f1c380cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.434+0000 7f1c3d000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1c38071200 0x7f1c3806dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.434+0000 7f1c3d000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c3815deb0 con 0x7f1c381618f0 2024-01-15T18:10:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.434+0000 7f1c3d000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3815ea30 con 0x7f1c38071200 2024-01-15T18:10:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.434+0000 7f1c3d000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c3806d860 con 0x7f1c38157c30 2024-01-15T18:10:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.435+0000 7f1c36800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1c38071200 0x7f1c3806dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.435+0000 7f1c35400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1c381618f0 0x7f1c38070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51524/0 (socket says 172.21.15.26:51524) 2024-01-15T18:10:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.435+0000 7f1c35400640 1 -- 172.21.15.26:0/3256234940 learned_addr learned my addr 172.21.15.26:0/3256234940 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.435+0000 7f1c35e00640 1 --2- 172.21.15.26:0/3256234940 >> v2:172.21.15.124:3300/0 conn(0x7f1c38157c30 0x7f1c380cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.435+0000 7f1c36800640 1 -- 172.21.15.26:0/3256234940 >> v2:172.21.15.124:3300/0 conn(0x7f1c38157c30 msgr2=0x7f1c380cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.435+0000 7f1c36800640 1 --2- 172.21.15.26:0/3256234940 >> v2:172.21.15.124:3300/0 conn(0x7f1c38157c30 0x7f1c380cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.435+0000 7f1c36800640 1 -- 172.21.15.26:0/3256234940 >> v1:172.21.15.26:6789/0 conn(0x7f1c381618f0 legacy=0x7f1c38070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.435+0000 7f1c36800640 1 -- 172.21.15.26:0/3256234940 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3815e190 con 0x7f1c38071200 2024-01-15T18:10:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.437+0000 7f1c36800640 1 --2- 172.21.15.26:0/3256234940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c38071200 0x7f1c3806dcb0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f1c380723f0 tx=0x7f1c3002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef250bc215bd7384 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.437+0000 7f1c34a00640 1 -- 172.21.15.26:0/3256234940 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3002cc70 con 0x7f1c38071200 2024-01-15T18:10:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.437+0000 7f1c34a00640 1 -- 172.21.15.26:0/3256234940 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3002cdd0 con 0x7f1c38071200 2024-01-15T18:10:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.437+0000 7f1c34a00640 1 -- 172.21.15.26:0/3256234940 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30034660 con 0x7f1c38071200 2024-01-15T18:10:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.437+0000 7f1c3d000640 1 -- 172.21.15.26:0/3256234940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c38071200 msgr2=0x7f1c3806dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.437+0000 7f1c3d000640 1 --2- 172.21.15.26:0/3256234940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c38071200 0x7f1c3806dcb0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f1c380723f0 tx=0x7f1c3002bec0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.438+0000 7f1c3d000640 1 -- 172.21.15.26:0/3256234940 shutdown_connections 2024-01-15T18:10:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.438+0000 7f1c3d000640 1 --2- 172.21.15.26:0/3256234940 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c38071200 0x7f1c3806dcb0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.438+0000 7f1c3d000640 1 --2- 172.21.15.26:0/3256234940 >> v2:172.21.15.124:3300/0 conn(0x7f1c38157c30 0x7f1c380cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.438+0000 7f1c3d000640 1 -- 172.21.15.26:0/3256234940 >> 172.21.15.26:0/3256234940 conn(0x7f1c380cf700 msgr2=0x7f1c380cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.438+0000 7f1c3d000640 1 -- 172.21.15.26:0/3256234940 shutdown_connections 2024-01-15T18:10:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.438+0000 7f1c3d000640 1 -- 172.21.15.26:0/3256234940 wait complete. 2024-01-15T18:10:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.440+0000 7f1c3d000640 1 Processor -- start 2024-01-15T18:10:28.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.440+0000 7f1c3d000640 1 -- start start 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c3d000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c38071200 0x7f1c38143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c3d000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c38157c30 0x7f1c38158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c3d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c381618f0 0x7f1c381585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c3d000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c381445d0 con 0x7f1c381618f0 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c3d000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38146b10 con 0x7f1c38157c30 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c3d000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c38143c30 con 0x7f1c38071200 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c36800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c381618f0 0x7f1c381585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c36800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c381618f0 0x7f1c381585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48664/0 (socket says 172.21.15.26:48664) 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c36800640 1 -- 172.21.15.26:0/2059444129 learned_addr learned my addr 172.21.15.26:0/2059444129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.441+0000 7f1c35e00640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c38071200 0x7f1c38143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c35400640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c38157c30 0x7f1c38158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c36800640 1 -- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c38071200 msgr2=0x7f1c38143280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c36800640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c38071200 0x7f1c38143280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c36800640 1 -- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c38157c30 msgr2=0x7f1c38158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c36800640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c38157c30 0x7f1c38158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c36800640 1 -- 172.21.15.26:0/2059444129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c30009a70 con 0x7f1c381618f0 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c35400640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c38157c30 0x7f1c38158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c36800640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c381618f0 0x7f1c381585b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009e80 tx=0x7f1c30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c2e000640 1 -- 172.21.15.26:0/2059444129 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30040650 con 0x7f1c381618f0 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c2e000640 1 -- 172.21.15.26:0/2059444129 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c300407d0 con 0x7f1c381618f0 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c2e000640 1 -- 172.21.15.26:0/2059444129 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30040b80 con 0x7f1c381618f0 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.442+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c38143f10 con 0x7f1c381618f0 2024-01-15T18:10:28.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.443+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c381437c0 con 0x7f1c381618f0 2024-01-15T18:10:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.444+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c38144710 con 0x7f1c381618f0 2024-01-15T18:10:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.445+0000 7f1c2e000640 1 -- 172.21.15.26:0/2059444129 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f1c30040ce0 con 0x7f1c381618f0 2024-01-15T18:10:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.445+0000 7f1c2e000640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1006c1f0 0x7f1c1006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.446+0000 7f1c2e000640 1 -- 172.21.15.26:0/2059444129 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1c300ab400 con 0x7f1c381618f0 2024-01-15T18:10:28.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.446+0000 7f1c35e00640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1006c1f0 0x7f1c1006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.447+0000 7f1c35e00640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1006c1f0 0x7f1c1006e6b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1c28006fd0 tx=0x7f1c28008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.450+0000 7f1c2e000640 1 -- 172.21.15.26:0/2059444129 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c38144710 con 0x7f1c381618f0 2024-01-15T18:10:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.718+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1c38144710 con 0x7f1c381618f0 2024-01-15T18:10:28.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.719+0000 7f1c2e000640 1 -- 172.21.15.26:0/2059444129 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1015 (secure 0 0 0) 0x7f1c38144710 con 0x7f1c381618f0 2024-01-15T18:10:28.722 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:28.722 INFO:teuthology.orchestra.run.smithi026.stdout:{"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":942,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1705341296,"num_in_osds":8,"osd_in_since":1705341288,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":598962176,"bytes_avail":772495151104,"bytes_total":773094113280,"write_bytes_sec":2146,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":63,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":36242}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-01-15T18:09:20.249078+0000","services":{}},"progress_events":{}} 2024-01-15T18:10:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1006c1f0 msgr2=0x7f1c1006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1006c1f0 0x7f1c1006e6b0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1c28006fd0 tx=0x7f1c28008040 comp rx=0 tx=0).stop 2024-01-15T18:10:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c381618f0 msgr2=0x7f1c381585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c381618f0 0x7f1c381585b0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009e80 tx=0x7f1c30004a40 comp rx=0 tx=0).stop 2024-01-15T18:10:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 shutdown_connections 2024-01-15T18:10:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1c1006c1f0 0x7f1c1006e6b0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1c381618f0 0x7f1c381585b0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1c38157c30 0x7f1c38158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 --2- 172.21.15.26:0/2059444129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1c38071200 0x7f1c38143280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.721+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 >> 172.21.15.26:0/2059444129 conn(0x7f1c380cf700 msgr2=0x7f1c380b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:28.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.722+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 shutdown_connections 2024-01-15T18:10:28.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.722+0000 7f1c3d000640 1 -- 172.21.15.26:0/2059444129 wait complete. 2024-01-15T18:10:28.745 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-01-15T18:10:28.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:10:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.963+0000 7f527a000640 1 Processor -- start 2024-01-15T18:10:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.963+0000 7f527a000640 1 -- start start 2024-01-15T18:10:28.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.964+0000 7f527a000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274160920 0x7f52740b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.964+0000 7f527a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52740cf9f0 0x7f52740cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.964+0000 7f527a000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52740d0a10 con 0x7f5274161010 2024-01-15T18:10:28.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.964+0000 7f527a000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52740d03b0 con 0x7f52740cf9f0 2024-01-15T18:10:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.964+0000 7f527a000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52740d1f50 con 0x7f5274160920 2024-01-15T18:10:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.965+0000 7f5273e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f52740cf9f0 0x7f52740cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.965+0000 7f5279600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5274161010 0x7f52741614a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:51534/0 (socket says 172.21.15.26:51534) 2024-01-15T18:10:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.965+0000 7f5279600640 1 -- 172.21.15.26:0/4194035365 learned_addr learned my addr 172.21.15.26:0/4194035365 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:28.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.965+0000 7f5278c00640 1 --2- 172.21.15.26:0/4194035365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274160920 0x7f52740b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.965+0000 7f5273e00640 1 -- 172.21.15.26:0/4194035365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274160920 msgr2=0x7f52740b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.965+0000 7f5273e00640 1 --2- 172.21.15.26:0/4194035365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274160920 0x7f52740b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.965+0000 7f5273e00640 1 -- 172.21.15.26:0/4194035365 >> v1:172.21.15.26:6789/0 conn(0x7f5274161010 legacy=0x7f52741614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.966+0000 7f5273e00640 1 -- 172.21.15.26:0/4194035365 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740cf080 con 0x7f52740cf9f0 2024-01-15T18:10:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.965+0000 7f5273400640 1 -- 172.21.15.26:0/4194035365 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111058610 0 0) 0x7f52740d0a10 con 0x7f5274161010 2024-01-15T18:10:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.967+0000 7f5273e00640 1 --2- 172.21.15.26:0/4194035365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 0x7f52740cfe70 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f525c0097a0 tx=0x7f525c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faabb98455b0ec8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.967+0000 7f5273400640 1 -- 172.21.15.26:0/4194035365 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c031830 con 0x7f52740cf9f0 2024-01-15T18:10:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.967+0000 7f5273400640 1 -- 172.21.15.26:0/4194035365 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c031990 con 0x7f52740cf9f0 2024-01-15T18:10:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.967+0000 7f5273400640 1 -- 172.21.15.26:0/4194035365 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c031d20 con 0x7f52740cf9f0 2024-01-15T18:10:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.967+0000 7f527a000640 1 -- 172.21.15.26:0/4194035365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 msgr2=0x7f52740cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.967+0000 7f527a000640 1 --2- 172.21.15.26:0/4194035365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 0x7f52740cfe70 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f525c0097a0 tx=0x7f525c02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.968+0000 7f527a000640 1 -- 172.21.15.26:0/4194035365 shutdown_connections 2024-01-15T18:10:28.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.968+0000 7f527a000640 1 --2- 172.21.15.26:0/4194035365 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 0x7f52740cfe70 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.968+0000 7f527a000640 1 --2- 172.21.15.26:0/4194035365 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274160920 0x7f52740b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.968+0000 7f527a000640 1 -- 172.21.15.26:0/4194035365 >> 172.21.15.26:0/4194035365 conn(0x7f52740d3090 msgr2=0x7f52740d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:28.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.968+0000 7f527a000640 1 -- 172.21.15.26:0/4194035365 shutdown_connections 2024-01-15T18:10:28.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.968+0000 7f527a000640 1 -- 172.21.15.26:0/4194035365 wait complete. 2024-01-15T18:10:28.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.969+0000 7f527a000640 1 Processor -- start 2024-01-15T18:10:28.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.970+0000 7f527a000640 1 -- start start 2024-01-15T18:10:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f527a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 0x7f527414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f527a000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5274160920 0x7f52740b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f527a000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274161010 0x7f52741f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f527a000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52741f6280 con 0x7f52740cf9f0 2024-01-15T18:10:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f527a000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527414c060 con 0x7f5274160920 2024-01-15T18:10:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f527a000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52741f6490 con 0x7f5274161010 2024-01-15T18:10:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5278c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 0x7f527414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5278c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 0x7f527414bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48682/0 (socket says 172.21.15.26:48682) 2024-01-15T18:10:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5278c00640 1 -- 172.21.15.26:0/2530817269 learned_addr learned my addr 172.21.15.26:0/2530817269 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5279600640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274161010 0x7f52741f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5273e00640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5274160920 0x7f52740b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5278c00640 1 -- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274161010 msgr2=0x7f52741f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5278c00640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274161010 0x7f52741f20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5278c00640 1 -- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5274160920 msgr2=0x7f52740b74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5278c00640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5274160920 0x7f52740b74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.971+0000 7f5278c00640 1 -- 172.21.15.26:0/2530817269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c009400 con 0x7f52740cf9f0 2024-01-15T18:10:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.972+0000 7f5273e00640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5274160920 0x7f52740b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:28.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.972+0000 7f5278c00640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 0x7f527414bb20 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f526400d800 tx=0x7f526400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.972+0000 7f5270c00640 1 -- 172.21.15.26:0/2530817269 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400b6c0 con 0x7f52740cf9f0 2024-01-15T18:10:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.972+0000 7f5270c00640 1 -- 172.21.15.26:0/2530817269 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526400b820 con 0x7f52740cf9f0 2024-01-15T18:10:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.972+0000 7f5270c00640 1 -- 172.21.15.26:0/2530817269 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526400bbb0 con 0x7f52740cf9f0 2024-01-15T18:10:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.972+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52741f6770 con 0x7f52740cf9f0 2024-01-15T18:10:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.972+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52741f6d30 con 0x7f52740cf9f0 2024-01-15T18:10:28.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.974+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52740c95c0 con 0x7f52740cf9f0 2024-01-15T18:10:28.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.974+0000 7f5270c00640 1 -- 172.21.15.26:0/2530817269 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f526400bd10 con 0x7f52740cf9f0 2024-01-15T18:10:28.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.975+0000 7f5270c00640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f524c06c1f0 0x7f524c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:28.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.975+0000 7f5270c00640 1 -- 172.21.15.26:0/2530817269 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5264088f70 con 0x7f52740cf9f0 2024-01-15T18:10:28.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.976+0000 7f5273e00640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f524c06c1f0 0x7f524c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:28.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.977+0000 7f5273e00640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f524c06c1f0 0x7f524c06e6b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f525c036000 tx=0x7f525c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:28.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:28.980+0000 7f5270c00640 1 -- 172.21.15.26:0/2530817269 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5264055540 con 0x7f52740cf9f0 2024-01-15T18:10:29.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.175+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52740ceac0 con 0x7f524c06c1f0 2024-01-15T18:10:29.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.199+0000 7f5270c00640 1 -- 172.21.15.26:0/2530817269 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286997 (secure 0 0 0) 0x7f525c045400 con 0x7f524c06c1f0 2024-01-15T18:10:29.209 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:29.210 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:10:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f524c06c1f0 msgr2=0x7f524c06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f524c06c1f0 0x7f524c06e6b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f525c036000 tx=0x7f525c008030 comp rx=0 tx=0).stop 2024-01-15T18:10:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 msgr2=0x7f527414bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 0x7f527414bb20 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f526400d800 tx=0x7f526400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:10:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 shutdown_connections 2024-01-15T18:10:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5274161010 0x7f52741f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f524c06c1f0 0x7f524c06e6b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5274160920 0x7f52740b74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:29.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 --2- 172.21.15.26:0/2530817269 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f52740cf9f0 0x7f527414bb20 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 >> 172.21.15.26:0/2530817269 conn(0x7f52740d3090 msgr2=0x7f5274161420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.205+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 shutdown_connections 2024-01-15T18:10:29.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:29.206+0000 7f527a000640 1 -- 172.21.15.26:0/2530817269 wait complete. 2024-01-15T18:10:29.232 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":561,"stamp":"2024-01-15T18:10:28.272647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":584924,"kb_used_data":11048,"kb_used_omap":12,"kb_used_meta":573747,"kb_avail":754389796,"statfs":{"total":773094113280,"available":772495151104,"internally_reserved":0,"allocated":11313152,"data_stored":5559244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":587517488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.679157"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544220+0000","last_change":"2024-01-15T18:10:22.628324+0000","last_active":"2024-01-15T18:10:24.544220+0000","last_peered":"2024-01-15T18:10:24.544220+0000","last_clean":"2024-01-15T18:10:24.544220+0000","last_became_active":"2024-01-15T18:10:22.628033+0000","last_became_peered":"2024-01-15T18:10:22.628033+0000","last_unstale":"2024-01-15T18:10:24.544220+0000","last_undegraded":"2024-01-15T18:10:24.544220+0000","last_fullsized":"2024-01-15T18:10:24.544220+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:54.391721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543739+0000","last_change":"2024-01-15T18:10:22.630135+0000","last_active":"2024-01-15T18:10:24.543739+0000","last_peered":"2024-01-15T18:10:24.543739+0000","last_clean":"2024-01-15T18:10:24.543739+0000","last_became_active":"2024-01-15T18:10:22.629836+0000","last_became_peered":"2024-01-15T18:10:22.629836+0000","last_unstale":"2024-01-15T18:10:24.543739+0000","last_undegraded":"2024-01-15T18:10:24.543739+0000","last_fullsized":"2024-01-15T18:10:24.543739+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:53.944791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544345+0000","last_change":"2024-01-15T18:10:22.622997+0000","last_active":"2024-01-15T18:10:24.544345+0000","last_peered":"2024-01-15T18:10:24.544345+0000","last_clean":"2024-01-15T18:10:24.544345+0000","last_became_active":"2024-01-15T18:10:22.622681+0000","last_became_peered":"2024-01-15T18:10:22.622681+0000","last_unstale":"2024-01-15T18:10:24.544345+0000","last_undegraded":"2024-01-15T18:10:24.544345+0000","last_fullsized":"2024-01-15T18:10:24.544345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:08:18.277916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599751+0000","last_change":"2024-01-15T18:10:22.604572+0000","last_active":"2024-01-15T18:10:23.599751+0000","last_peered":"2024-01-15T18:10:23.599751+0000","last_clean":"2024-01-15T18:10:23.599751+0000","last_became_active":"2024-01-15T18:10:22.604033+0000","last_became_peered":"2024-01-15T18:10:22.604033+0000","last_unstale":"2024-01-15T18:10:23.599751+0000","last_undegraded":"2024-01-15T18:10:23.599751+0000","last_fullsized":"2024-01-15T18:10:23.599751+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:42:15.871289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541683+0000","last_change":"2024-01-15T18:10:22.626871+0000","last_active":"2024-01-15T18:10:24.541683+0000","last_peered":"2024-01-15T18:10:24.541683+0000","last_clean":"2024-01-15T18:10:24.541683+0000","last_became_active":"2024-01-15T18:10:22.626415+0000","last_became_peered":"2024-01-15T18:10:22.626415+0000","last_unstale":"2024-01-15T18:10:24.541683+0000","last_undegraded":"2024-01-15T18:10:24.541683+0000","last_fullsized":"2024-01-15T18:10:24.541683+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:59.565363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.605436+0000","last_change":"2024-01-15T18:10:22.614413+0000","last_active":"2024-01-15T18:10:23.605436+0000","last_peered":"2024-01-15T18:10:23.605436+0000","last_clean":"2024-01-15T18:10:23.605436+0000","last_became_active":"2024-01-15T18:10:22.613917+0000","last_became_peered":"2024-01-15T18:10:22.613917+0000","last_unstale":"2024-01-15T18:10:23.605436+0000","last_undegraded":"2024-01-15T18:10:23.605436+0000","last_fullsized":"2024-01-15T18:10:23.605436+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:17.084662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604969+0000","last_change":"2024-01-15T18:10:22.613303+0000","last_active":"2024-01-15T18:10:23.604969+0000","last_peered":"2024-01-15T18:10:23.604969+0000","last_clean":"2024-01-15T18:10:23.604969+0000","last_became_active":"2024-01-15T18:10:22.612865+0000","last_became_peered":"2024-01-15T18:10:22.612865+0000","last_unstale":"2024-01-15T18:10:23.604969+0000","last_undegraded":"2024-01-15T18:10:23.604969+0000","last_fullsized":"2024-01-15T18:10:23.604969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:00.289344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542095+0000","last_change":"2024-01-15T18:10:22.634156+0000","last_active":"2024-01-15T18:10:24.542095+0000","last_peered":"2024-01-15T18:10:24.542095+0000","last_clean":"2024-01-15T18:10:24.542095+0000","last_became_active":"2024-01-15T18:10:22.633797+0000","last_became_peered":"2024-01-15T18:10:22.633797+0000","last_unstale":"2024-01-15T18:10:24.542095+0000","last_undegraded":"2024-01-15T18:10:24.542095+0000","last_fullsized":"2024-01-15T18:10:24.542095+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:08:26.541990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604602+0000","last_change":"2024-01-15T18:10:22.613983+0000","last_active":"2024-01-15T18:10:23.604602+0000","last_peered":"2024-01-15T18:10:23.604602+0000","last_clean":"2024-01-15T18:10:23.604602+0000","last_became_active":"2024-01-15T18:10:22.613680+0000","last_became_peered":"2024-01-15T18:10:22.613680+0000","last_unstale":"2024-01-15T18:10:23.604602+0000","last_undegraded":"2024-01-15T18:10:23.604602+0000","last_fullsized":"2024-01-15T18:10:23.604602+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:17.994389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542036+0000","last_change":"2024-01-15T18:10:22.628655+0000","last_active":"2024-01-15T18:10:24.542036+0000","last_peered":"2024-01-15T18:10:24.542036+0000","last_clean":"2024-01-15T18:10:24.542036+0000","last_became_active":"2024-01-15T18:10:22.628353+0000","last_became_peered":"2024-01-15T18:10:22.628353+0000","last_unstale":"2024-01-15T18:10:24.542036+0000","last_undegraded":"2024-01-15T18:10:24.542036+0000","last_fullsized":"2024-01-15T18:10:24.542036+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.571225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.605677+0000","last_change":"2024-01-15T18:10:22.617671+0000","last_active":"2024-01-15T18:10:23.605677+0000","last_peered":"2024-01-15T18:10:23.605677+0000","last_clean":"2024-01-15T18:10:23.605677+0000","last_became_active":"2024-01-15T18:10:22.617378+0000","last_became_peered":"2024-01-15T18:10:22.617378+0000","last_unstale":"2024-01-15T18:10:23.605677+0000","last_undegraded":"2024-01-15T18:10:23.605677+0000","last_fullsized":"2024-01-15T18:10:23.605677+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:28.198670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.603327+0000","last_change":"2024-01-15T18:10:22.631118+0000","last_active":"2024-01-15T18:10:23.603327+0000","last_peered":"2024-01-15T18:10:23.603327+0000","last_clean":"2024-01-15T18:10:23.603327+0000","last_became_active":"2024-01-15T18:10:22.629350+0000","last_became_peered":"2024-01-15T18:10:22.629350+0000","last_unstale":"2024-01-15T18:10:23.603327+0000","last_undegraded":"2024-01-15T18:10:23.603327+0000","last_fullsized":"2024-01-15T18:10:23.603327+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:30.260896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599447+0000","last_change":"2024-01-15T18:10:22.605368+0000","last_active":"2024-01-15T18:10:23.599447+0000","last_peered":"2024-01-15T18:10:23.599447+0000","last_clean":"2024-01-15T18:10:23.599447+0000","last_became_active":"2024-01-15T18:10:22.605059+0000","last_became_peered":"2024-01-15T18:10:22.605059+0000","last_unstale":"2024-01-15T18:10:23.599447+0000","last_undegraded":"2024-01-15T18:10:23.599447+0000","last_fullsized":"2024-01-15T18:10:23.599447+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:00.427701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352852+0000","last_change":"2024-01-15T18:10:22.634829+0000","last_active":"2024-01-15T18:10:24.352852+0000","last_peered":"2024-01-15T18:10:24.352852+0000","last_clean":"2024-01-15T18:10:24.352852+0000","last_became_active":"2024-01-15T18:10:22.634489+0000","last_became_peered":"2024-01-15T18:10:22.634489+0000","last_unstale":"2024-01-15T18:10:24.352852+0000","last_undegraded":"2024-01-15T18:10:24.352852+0000","last_fullsized":"2024-01-15T18:10:24.352852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:03:27.703474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.598920+0000","last_change":"2024-01-15T18:10:22.609316+0000","last_active":"2024-01-15T18:10:23.598920+0000","last_peered":"2024-01-15T18:10:23.598920+0000","last_clean":"2024-01-15T18:10:23.598920+0000","last_became_active":"2024-01-15T18:10:22.609027+0000","last_became_peered":"2024-01-15T18:10:22.609027+0000","last_unstale":"2024-01-15T18:10:23.598920+0000","last_undegraded":"2024-01-15T18:10:23.598920+0000","last_fullsized":"2024-01-15T18:10:23.598920+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:07.756348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.603818+0000","last_change":"2024-01-15T18:10:22.624718+0000","last_active":"2024-01-15T18:10:23.603818+0000","last_peered":"2024-01-15T18:10:23.603818+0000","last_clean":"2024-01-15T18:10:23.603818+0000","last_became_active":"2024-01-15T18:10:22.623285+0000","last_became_peered":"2024-01-15T18:10:22.623285+0000","last_unstale":"2024-01-15T18:10:23.603818+0000","last_undegraded":"2024-01-15T18:10:23.603818+0000","last_fullsized":"2024-01-15T18:10:23.603818+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:53.944791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543809+0000","last_change":"2024-01-15T18:10:22.626677+0000","last_active":"2024-01-15T18:10:24.543809+0000","last_peered":"2024-01-15T18:10:24.543809+0000","last_clean":"2024-01-15T18:10:24.543809+0000","last_became_active":"2024-01-15T18:10:22.626382+0000","last_became_peered":"2024-01-15T18:10:22.626382+0000","last_unstale":"2024-01-15T18:10:24.543809+0000","last_undegraded":"2024-01-15T18:10:24.543809+0000","last_fullsized":"2024-01-15T18:10:24.543809+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:30:21.693594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.600204+0000","last_change":"2024-01-15T18:10:22.604594+0000","last_active":"2024-01-15T18:10:23.600204+0000","last_peered":"2024-01-15T18:10:23.600204+0000","last_clean":"2024-01-15T18:10:23.600204+0000","last_became_active":"2024-01-15T18:10:22.604178+0000","last_became_peered":"2024-01-15T18:10:22.604178+0000","last_unstale":"2024-01-15T18:10:23.600204+0000","last_undegraded":"2024-01-15T18:10:23.600204+0000","last_fullsized":"2024-01-15T18:10:23.600204+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:54:56.471353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.354085+0000","last_change":"2024-01-15T18:10:22.612335+0000","last_active":"2024-01-15T18:10:24.354085+0000","last_peered":"2024-01-15T18:10:24.354085+0000","last_clean":"2024-01-15T18:10:24.354085+0000","last_became_active":"2024-01-15T18:10:22.612045+0000","last_became_peered":"2024-01-15T18:10:22.612045+0000","last_unstale":"2024-01-15T18:10:24.354085+0000","last_undegraded":"2024-01-15T18:10:24.354085+0000","last_fullsized":"2024-01-15T18:10:24.354085+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:45.137589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.603431+0000","last_change":"2024-01-15T18:10:22.607947+0000","last_active":"2024-01-15T18:10:23.603431+0000","last_peered":"2024-01-15T18:10:23.603431+0000","last_clean":"2024-01-15T18:10:23.603431+0000","last_became_active":"2024-01-15T18:10:22.607640+0000","last_became_peered":"2024-01-15T18:10:22.607640+0000","last_unstale":"2024-01-15T18:10:23.603431+0000","last_undegraded":"2024-01-15T18:10:23.603431+0000","last_fullsized":"2024-01-15T18:10:23.603431+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:59.565363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.353359+0000","last_change":"2024-01-15T18:10:22.618490+0000","last_active":"2024-01-15T18:10:24.353359+0000","last_peered":"2024-01-15T18:10:24.353359+0000","last_clean":"2024-01-15T18:10:24.353359+0000","last_became_active":"2024-01-15T18:10:22.618198+0000","last_became_peered":"2024-01-15T18:10:22.618198+0000","last_unstale":"2024-01-15T18:10:24.353359+0000","last_undegraded":"2024-01-15T18:10:24.353359+0000","last_fullsized":"2024-01-15T18:10:24.353359+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:54.391721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541948+0000","last_change":"2024-01-15T18:10:22.630121+0000","last_active":"2024-01-15T18:10:24.541948+0000","last_peered":"2024-01-15T18:10:24.541948+0000","last_clean":"2024-01-15T18:10:24.541948+0000","last_became_active":"2024-01-15T18:10:22.629804+0000","last_became_peered":"2024-01-15T18:10:22.629804+0000","last_unstale":"2024-01-15T18:10:24.541948+0000","last_undegraded":"2024-01-15T18:10:24.541948+0000","last_fullsized":"2024-01-15T18:10:24.541948+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:37.061370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543047+0000","last_change":"2024-01-15T18:10:22.630806+0000","last_active":"2024-01-15T18:10:24.543047+0000","last_peered":"2024-01-15T18:10:24.543047+0000","last_clean":"2024-01-15T18:10:24.543047+0000","last_became_active":"2024-01-15T18:10:22.630493+0000","last_became_peered":"2024-01-15T18:10:22.630493+0000","last_unstale":"2024-01-15T18:10:24.543047+0000","last_undegraded":"2024-01-15T18:10:24.543047+0000","last_fullsized":"2024-01-15T18:10:24.543047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:50.984602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604235+0000","last_change":"2024-01-15T18:10:22.624955+0000","last_active":"2024-01-15T18:10:23.604235+0000","last_peered":"2024-01-15T18:10:23.604235+0000","last_clean":"2024-01-15T18:10:23.604235+0000","last_became_active":"2024-01-15T18:10:22.623819+0000","last_became_peered":"2024-01-15T18:10:22.623819+0000","last_unstale":"2024-01-15T18:10:23.604235+0000","last_undegraded":"2024-01-15T18:10:23.604235+0000","last_fullsized":"2024-01-15T18:10:23.604235+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:30:21.693594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542534+0000","last_change":"2024-01-15T18:10:22.639462+0000","last_active":"2024-01-15T18:10:24.542534+0000","last_peered":"2024-01-15T18:10:24.542534+0000","last_clean":"2024-01-15T18:10:24.542534+0000","last_became_active":"2024-01-15T18:10:22.638113+0000","last_became_peered":"2024-01-15T18:10:22.638113+0000","last_unstale":"2024-01-15T18:10:24.542534+0000","last_undegraded":"2024-01-15T18:10:24.542534+0000","last_fullsized":"2024-01-15T18:10:24.542534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:41:49.892254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544137+0000","last_change":"2024-01-15T18:10:22.626150+0000","last_active":"2024-01-15T18:10:24.544137+0000","last_peered":"2024-01-15T18:10:24.544137+0000","last_clean":"2024-01-15T18:10:24.544137+0000","last_became_active":"2024-01-15T18:10:22.625274+0000","last_became_peered":"2024-01-15T18:10:22.625274+0000","last_unstale":"2024-01-15T18:10:24.544137+0000","last_undegraded":"2024-01-15T18:10:24.544137+0000","last_fullsized":"2024-01-15T18:10:24.544137+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:01:00.967705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.605931+0000","last_change":"2024-01-15T18:10:22.622195+0000","last_active":"2024-01-15T18:10:23.605931+0000","last_peered":"2024-01-15T18:10:23.605931+0000","last_clean":"2024-01-15T18:10:23.605931+0000","last_became_active":"2024-01-15T18:10:22.621911+0000","last_became_peered":"2024-01-15T18:10:22.621911+0000","last_unstale":"2024-01-15T18:10:23.605931+0000","last_undegraded":"2024-01-15T18:10:23.605931+0000","last_fullsized":"2024-01-15T18:10:23.605931+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:26:56.885786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541875+0000","last_change":"2024-01-15T18:10:22.631688+0000","last_active":"2024-01-15T18:10:24.541875+0000","last_peered":"2024-01-15T18:10:24.541875+0000","last_clean":"2024-01-15T18:10:24.541875+0000","last_became_active":"2024-01-15T18:10:22.631361+0000","last_became_peered":"2024-01-15T18:10:22.631361+0000","last_unstale":"2024-01-15T18:10:24.541875+0000","last_undegraded":"2024-01-15T18:10:24.541875+0000","last_fullsized":"2024-01-15T18:10:24.541875+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:13:32.757599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542277+0000","last_change":"2024-01-15T18:10:22.626068+0000","last_active":"2024-01-15T18:10:24.542277+0000","last_peered":"2024-01-15T18:10:24.542277+0000","last_clean":"2024-01-15T18:10:24.542277+0000","last_became_active":"2024-01-15T18:10:22.625748+0000","last_became_peered":"2024-01-15T18:10:22.625748+0000","last_unstale":"2024-01-15T18:10:24.542277+0000","last_undegraded":"2024-01-15T18:10:24.542277+0000","last_fullsized":"2024-01-15T18:10:24.542277+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:41:49.892254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352997+0000","last_change":"2024-01-15T18:10:22.615020+0000","last_active":"2024-01-15T18:10:24.352997+0000","last_peered":"2024-01-15T18:10:24.352997+0000","last_clean":"2024-01-15T18:10:24.352997+0000","last_became_active":"2024-01-15T18:10:22.614651+0000","last_became_peered":"2024-01-15T18:10:22.614651+0000","last_unstale":"2024-01-15T18:10:24.352997+0000","last_undegraded":"2024-01-15T18:10:24.352997+0000","last_fullsized":"2024-01-15T18:10:24.352997+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:55.841561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599161+0000","last_change":"2024-01-15T18:10:22.631452+0000","last_active":"2024-01-15T18:10:23.599161+0000","last_peered":"2024-01-15T18:10:23.599161+0000","last_clean":"2024-01-15T18:10:23.599161+0000","last_became_active":"2024-01-15T18:10:22.631187+0000","last_became_peered":"2024-01-15T18:10:22.631187+0000","last_unstale":"2024-01-15T18:10:23.599161+0000","last_undegraded":"2024-01-15T18:10:23.599161+0000","last_fullsized":"2024-01-15T18:10:23.599161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:03.152957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541942+0000","last_change":"2024-01-15T18:10:22.619198+0000","last_active":"2024-01-15T18:10:24.541942+0000","last_peered":"2024-01-15T18:10:24.541942+0000","last_clean":"2024-01-15T18:10:24.541942+0000","last_became_active":"2024-01-15T18:10:22.618925+0000","last_became_peered":"2024-01-15T18:10:22.618925+0000","last_unstale":"2024-01-15T18:10:24.541942+0000","last_undegraded":"2024-01-15T18:10:24.541942+0000","last_fullsized":"2024-01-15T18:10:24.541942+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:03:27.703474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.354248+0000","last_change":"2024-01-15T18:10:22.613941+0000","last_active":"2024-01-15T18:10:24.354248+0000","last_peered":"2024-01-15T18:10:24.354248+0000","last_clean":"2024-01-15T18:10:24.354248+0000","last_became_active":"2024-01-15T18:10:22.613478+0000","last_became_peered":"2024-01-15T18:10:22.613478+0000","last_unstale":"2024-01-15T18:10:24.354248+0000","last_undegraded":"2024-01-15T18:10:24.354248+0000","last_fullsized":"2024-01-15T18:10:24.354248+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:45.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.640892+0000","last_change":"2024-01-15T18:10:21.616691+0000","last_active":"2024-01-15T18:10:22.640892+0000","last_peered":"2024-01-15T18:10:22.640892+0000","last_clean":"2024-01-15T18:10:22.640892+0000","last_became_active":"2024-01-15T18:10:21.616428+0000","last_became_peered":"2024-01-15T18:10:21.616428+0000","last_unstale":"2024-01-15T18:10:22.640892+0000","last_undegraded":"2024-01-15T18:10:22.640892+0000","last_fullsized":"2024-01-15T18:10:22.640892+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:38.559745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.600420+0000","last_change":"2024-01-15T18:10:22.628019+0000","last_active":"2024-01-15T18:10:23.600420+0000","last_peered":"2024-01-15T18:10:23.600420+0000","last_clean":"2024-01-15T18:10:23.600420+0000","last_became_active":"2024-01-15T18:10:22.627022+0000","last_became_peered":"2024-01-15T18:10:22.627022+0000","last_unstale":"2024-01-15T18:10:23.600420+0000","last_undegraded":"2024-01-15T18:10:23.600420+0000","last_fullsized":"2024-01-15T18:10:23.600420+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:53:17.828426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.629274+0000","last_change":"2024-01-15T18:10:21.624181+0000","last_active":"2024-01-15T18:10:22.629274+0000","last_peered":"2024-01-15T18:10:22.629274+0000","last_clean":"2024-01-15T18:10:22.629274+0000","last_became_active":"2024-01-15T18:10:21.623845+0000","last_became_peered":"2024-01-15T18:10:21.623845+0000","last_unstale":"2024-01-15T18:10:22.629274+0000","last_undegraded":"2024-01-15T18:10:22.629274+0000","last_fullsized":"2024-01-15T18:10:22.629274+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:38.559745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543793+0000","last_change":"2024-01-15T18:10:22.639046+0000","last_active":"2024-01-15T18:10:24.543793+0000","last_peered":"2024-01-15T18:10:24.543793+0000","last_clean":"2024-01-15T18:10:24.543793+0000","last_became_active":"2024-01-15T18:10:22.638587+0000","last_became_peered":"2024-01-15T18:10:22.638587+0000","last_unstale":"2024-01-15T18:10:24.543793+0000","last_undegraded":"2024-01-15T18:10:24.543793+0000","last_fullsized":"2024-01-15T18:10:24.543793+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:55.841561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.637291+0000","last_change":"2024-01-15T18:10:21.621640+0000","last_active":"2024-01-15T18:10:22.637291+0000","last_peered":"2024-01-15T18:10:22.637291+0000","last_clean":"2024-01-15T18:10:22.637291+0000","last_became_active":"2024-01-15T18:10:21.619850+0000","last_became_peered":"2024-01-15T18:10:21.619850+0000","last_unstale":"2024-01-15T18:10:22.637291+0000","last_undegraded":"2024-01-15T18:10:22.637291+0000","last_fullsized":"2024-01-15T18:10:22.637291+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:58.553883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604306+0000","last_change":"2024-01-15T18:10:22.627539+0000","last_active":"2024-01-15T18:10:23.604306+0000","last_peered":"2024-01-15T18:10:23.604306+0000","last_clean":"2024-01-15T18:10:23.604306+0000","last_became_active":"2024-01-15T18:10:22.627192+0000","last_became_peered":"2024-01-15T18:10:22.627192+0000","last_unstale":"2024-01-15T18:10:23.604306+0000","last_undegraded":"2024-01-15T18:10:23.604306+0000","last_fullsized":"2024-01-15T18:10:23.604306+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:13:32.757599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.640292+0000","last_change":"2024-01-15T18:10:21.619185+0000","last_active":"2024-01-15T18:10:22.640292+0000","last_peered":"2024-01-15T18:10:22.640292+0000","last_clean":"2024-01-15T18:10:22.640292+0000","last_became_active":"2024-01-15T18:10:21.618436+0000","last_became_peered":"2024-01-15T18:10:21.618436+0000","last_unstale":"2024-01-15T18:10:22.640292+0000","last_undegraded":"2024-01-15T18:10:22.640292+0000","last_fullsized":"2024-01-15T18:10:22.640292+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:31:30.348979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.412317+0000","last_change":"2024-01-15T18:10:22.625949+0000","last_active":"2024-01-15T18:10:24.412317+0000","last_peered":"2024-01-15T18:10:24.412317+0000","last_clean":"2024-01-15T18:10:24.412317+0000","last_became_active":"2024-01-15T18:10:22.625639+0000","last_became_peered":"2024-01-15T18:10:22.625639+0000","last_unstale":"2024-01-15T18:10:24.412317+0000","last_undegraded":"2024-01-15T18:10:24.412317+0000","last_fullsized":"2024-01-15T18:10:24.412317+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:35.642706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.639492+0000","last_change":"2024-01-15T18:10:21.618207+0000","last_active":"2024-01-15T18:10:22.639492+0000","last_peered":"2024-01-15T18:10:22.639492+0000","last_clean":"2024-01-15T18:10:22.639492+0000","last_became_active":"2024-01-15T18:10:21.617907+0000","last_became_peered":"2024-01-15T18:10:21.617907+0000","last_unstale":"2024-01-15T18:10:22.639492+0000","last_undegraded":"2024-01-15T18:10:22.639492+0000","last_fullsized":"2024-01-15T18:10:22.639492+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:43:42.041130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604237+0000","last_change":"2024-01-15T18:10:22.629889+0000","last_active":"2024-01-15T18:10:23.604237+0000","last_peered":"2024-01-15T18:10:23.604237+0000","last_clean":"2024-01-15T18:10:23.604237+0000","last_became_active":"2024-01-15T18:10:22.629541+0000","last_became_peered":"2024-01-15T18:10:22.629541+0000","last_unstale":"2024-01-15T18:10:23.604237+0000","last_undegraded":"2024-01-15T18:10:23.604237+0000","last_fullsized":"2024-01-15T18:10:23.604237+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:01:00.967705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.635571+0000","last_change":"2024-01-15T18:10:21.616023+0000","last_active":"2024-01-15T18:10:22.635571+0000","last_peered":"2024-01-15T18:10:22.635571+0000","last_clean":"2024-01-15T18:10:22.635571+0000","last_became_active":"2024-01-15T18:10:21.615732+0000","last_became_peered":"2024-01-15T18:10:21.615732+0000","last_unstale":"2024-01-15T18:10:22.635571+0000","last_undegraded":"2024-01-15T18:10:22.635571+0000","last_fullsized":"2024-01-15T18:10:22.635571+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:25.499049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599063+0000","last_change":"2024-01-15T18:10:22.608070+0000","last_active":"2024-01-15T18:10:23.599063+0000","last_peered":"2024-01-15T18:10:23.599063+0000","last_clean":"2024-01-15T18:10:23.599063+0000","last_became_active":"2024-01-15T18:10:22.607671+0000","last_became_peered":"2024-01-15T18:10:22.607671+0000","last_unstale":"2024-01-15T18:10:23.599063+0000","last_undegraded":"2024-01-15T18:10:23.599063+0000","last_fullsized":"2024-01-15T18:10:23.599063+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:10:18.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.618549+0000","last_change":"2024-01-15T18:10:21.613918+0000","last_active":"2024-01-15T18:10:22.618549+0000","last_peered":"2024-01-15T18:10:22.618549+0000","last_clean":"2024-01-15T18:10:22.618549+0000","last_became_active":"2024-01-15T18:10:21.613184+0000","last_became_peered":"2024-01-15T18:10:21.613184+0000","last_unstale":"2024-01-15T18:10:22.618549+0000","last_undegraded":"2024-01-15T18:10:22.618549+0000","last_fullsized":"2024-01-15T18:10:22.618549+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:47:40.399123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.602321+0000","last_change":"2024-01-15T18:10:22.616092+0000","last_active":"2024-01-15T18:10:23.602321+0000","last_peered":"2024-01-15T18:10:23.602321+0000","last_clean":"2024-01-15T18:10:23.602321+0000","last_became_active":"2024-01-15T18:10:22.615769+0000","last_became_peered":"2024-01-15T18:10:22.615769+0000","last_unstale":"2024-01-15T18:10:23.602321+0000","last_undegraded":"2024-01-15T18:10:23.602321+0000","last_fullsized":"2024-01-15T18:10:23.602321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:26.510529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.605502+0000","last_change":"2024-01-15T18:10:21.608770+0000","last_active":"2024-01-15T18:10:22.605502+0000","last_peered":"2024-01-15T18:10:22.605502+0000","last_clean":"2024-01-15T18:10:22.605502+0000","last_became_active":"2024-01-15T18:10:21.607618+0000","last_became_peered":"2024-01-15T18:10:21.607618+0000","last_unstale":"2024-01-15T18:10:22.605502+0000","last_undegraded":"2024-01-15T18:10:22.605502+0000","last_fullsized":"2024-01-15T18:10:22.605502+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:12:24.068260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544399+0000","last_change":"2024-01-15T18:10:22.636462+0000","last_active":"2024-01-15T18:10:24.544399+0000","last_peered":"2024-01-15T18:10:24.544399+0000","last_clean":"2024-01-15T18:10:24.544399+0000","last_became_active":"2024-01-15T18:10:22.636149+0000","last_became_peered":"2024-01-15T18:10:22.636149+0000","last_unstale":"2024-01-15T18:10:24.544399+0000","last_undegraded":"2024-01-15T18:10:24.544399+0000","last_fullsized":"2024-01-15T18:10:24.544399+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:17.084662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.616212+0000","last_change":"2024-01-15T18:10:21.613722+0000","last_active":"2024-01-15T18:10:22.616212+0000","last_peered":"2024-01-15T18:10:22.616212+0000","last_clean":"2024-01-15T18:10:22.616212+0000","last_became_active":"2024-01-15T18:10:21.613117+0000","last_became_peered":"2024-01-15T18:10:21.613117+0000","last_unstale":"2024-01-15T18:10:22.616212+0000","last_undegraded":"2024-01-15T18:10:22.616212+0000","last_fullsized":"2024-01-15T18:10:22.616212+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:04.970896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604568+0000","last_change":"2024-01-15T18:10:22.630879+0000","last_active":"2024-01-15T18:10:23.604568+0000","last_peered":"2024-01-15T18:10:23.604568+0000","last_clean":"2024-01-15T18:10:23.604568+0000","last_became_active":"2024-01-15T18:10:22.628981+0000","last_became_peered":"2024-01-15T18:10:22.628981+0000","last_unstale":"2024-01-15T18:10:23.604568+0000","last_undegraded":"2024-01-15T18:10:23.604568+0000","last_fullsized":"2024-01-15T18:10:23.604568+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:02:38.272375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.596370+0000","last_change":"2024-01-15T18:10:21.612961+0000","last_active":"2024-01-15T18:10:22.596370+0000","last_peered":"2024-01-15T18:10:22.596370+0000","last_clean":"2024-01-15T18:10:22.596370+0000","last_became_active":"2024-01-15T18:10:21.612624+0000","last_became_peered":"2024-01-15T18:10:21.612624+0000","last_unstale":"2024-01-15T18:10:22.596370+0000","last_undegraded":"2024-01-15T18:10:22.596370+0000","last_fullsized":"2024-01-15T18:10:22.596370+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:38.055323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542008+0000","last_change":"2024-01-15T18:10:22.637523+0000","last_active":"2024-01-15T18:10:24.542008+0000","last_peered":"2024-01-15T18:10:24.542008+0000","last_clean":"2024-01-15T18:10:24.542008+0000","last_became_active":"2024-01-15T18:10:22.637208+0000","last_became_peered":"2024-01-15T18:10:22.637208+0000","last_unstale":"2024-01-15T18:10:24.542008+0000","last_undegraded":"2024-01-15T18:10:24.542008+0000","last_fullsized":"2024-01-15T18:10:24.542008+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:17.994389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.631979+0000","last_change":"2024-01-15T18:10:21.623180+0000","last_active":"2024-01-15T18:10:22.631979+0000","last_peered":"2024-01-15T18:10:22.631979+0000","last_clean":"2024-01-15T18:10:22.631979+0000","last_became_active":"2024-01-15T18:10:21.622427+0000","last_became_peered":"2024-01-15T18:10:21.622427+0000","last_unstale":"2024-01-15T18:10:22.631979+0000","last_undegraded":"2024-01-15T18:10:22.631979+0000","last_fullsized":"2024-01-15T18:10:22.631979+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:36.049889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.605951+0000","last_change":"2024-01-15T18:10:22.636790+0000","last_active":"2024-01-15T18:10:23.605951+0000","last_peered":"2024-01-15T18:10:23.605951+0000","last_clean":"2024-01-15T18:10:23.605951+0000","last_became_active":"2024-01-15T18:10:22.636403+0000","last_became_peered":"2024-01-15T18:10:22.636403+0000","last_unstale":"2024-01-15T18:10:23.605951+0000","last_undegraded":"2024-01-15T18:10:23.605951+0000","last_fullsized":"2024-01-15T18:10:23.605951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:07.478954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.641464+0000","last_change":"2024-01-15T18:10:21.617457+0000","last_active":"2024-01-15T18:10:22.641464+0000","last_peered":"2024-01-15T18:10:22.641464+0000","last_clean":"2024-01-15T18:10:22.641464+0000","last_became_active":"2024-01-15T18:10:21.617193+0000","last_became_peered":"2024-01-15T18:10:21.617193+0000","last_unstale":"2024-01-15T18:10:22.641464+0000","last_undegraded":"2024-01-15T18:10:22.641464+0000","last_fullsized":"2024-01-15T18:10:22.641464+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.176712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599146+0000","last_change":"2024-01-15T18:10:22.611497+0000","last_active":"2024-01-15T18:10:23.599146+0000","last_peered":"2024-01-15T18:10:23.599146+0000","last_clean":"2024-01-15T18:10:23.599146+0000","last_became_active":"2024-01-15T18:10:22.610906+0000","last_became_peered":"2024-01-15T18:10:22.610906+0000","last_unstale":"2024-01-15T18:10:23.599146+0000","last_undegraded":"2024-01-15T18:10:23.599146+0000","last_fullsized":"2024-01-15T18:10:23.599146+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:53:07.920195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.638166+0000","last_change":"2024-01-15T18:10:21.620555+0000","last_active":"2024-01-15T18:10:22.638166+0000","last_peered":"2024-01-15T18:10:22.638166+0000","last_clean":"2024-01-15T18:10:22.638166+0000","last_became_active":"2024-01-15T18:10:21.620232+0000","last_became_peered":"2024-01-15T18:10:21.620232+0000","last_unstale":"2024-01-15T18:10:22.638166+0000","last_undegraded":"2024-01-15T18:10:22.638166+0000","last_fullsized":"2024-01-15T18:10:22.638166+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:44.150606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543678+0000","last_change":"2024-01-15T18:10:22.617936+0000","last_active":"2024-01-15T18:10:24.543678+0000","last_peered":"2024-01-15T18:10:24.543678+0000","last_clean":"2024-01-15T18:10:24.543678+0000","last_became_active":"2024-01-15T18:10:22.613686+0000","last_became_peered":"2024-01-15T18:10:22.613686+0000","last_unstale":"2024-01-15T18:10:24.543678+0000","last_undegraded":"2024-01-15T18:10:24.543678+0000","last_fullsized":"2024-01-15T18:10:24.543678+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:19.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.599297+0000","last_change":"2024-01-15T18:10:21.618350+0000","last_active":"2024-01-15T18:10:22.599297+0000","last_peered":"2024-01-15T18:10:22.599297+0000","last_clean":"2024-01-15T18:10:22.599297+0000","last_became_active":"2024-01-15T18:10:21.618029+0000","last_became_peered":"2024-01-15T18:10:21.618029+0000","last_unstale":"2024-01-15T18:10:22.599297+0000","last_undegraded":"2024-01-15T18:10:22.599297+0000","last_fullsized":"2024-01-15T18:10:22.599297+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:43:42.041130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.601881+0000","last_change":"2024-01-15T18:10:22.627528+0000","last_active":"2024-01-15T18:10:23.601881+0000","last_peered":"2024-01-15T18:10:23.601881+0000","last_clean":"2024-01-15T18:10:23.601881+0000","last_became_active":"2024-01-15T18:10:22.627225+0000","last_became_peered":"2024-01-15T18:10:22.627225+0000","last_unstale":"2024-01-15T18:10:23.601881+0000","last_undegraded":"2024-01-15T18:10:23.601881+0000","last_fullsized":"2024-01-15T18:10:23.601881+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:15:11.824545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.599198+0000","last_change":"2024-01-15T18:10:21.622418+0000","last_active":"2024-01-15T18:10:22.599198+0000","last_peered":"2024-01-15T18:10:22.599198+0000","last_clean":"2024-01-15T18:10:22.599198+0000","last_became_active":"2024-01-15T18:10:21.621566+0000","last_became_peered":"2024-01-15T18:10:21.621566+0000","last_unstale":"2024-01-15T18:10:22.599198+0000","last_undegraded":"2024-01-15T18:10:22.599198+0000","last_fullsized":"2024-01-15T18:10:22.599198+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:25.792080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352843+0000","last_change":"2024-01-15T18:10:22.611042+0000","last_active":"2024-01-15T18:10:24.352843+0000","last_peered":"2024-01-15T18:10:24.352843+0000","last_clean":"2024-01-15T18:10:24.352843+0000","last_became_active":"2024-01-15T18:10:22.610178+0000","last_became_peered":"2024-01-15T18:10:22.610178+0000","last_unstale":"2024-01-15T18:10:24.352843+0000","last_undegraded":"2024-01-15T18:10:24.352843+0000","last_fullsized":"2024-01-15T18:10:24.352843+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:19.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.639911+0000","last_change":"2024-01-15T18:10:21.613977+0000","last_active":"2024-01-15T18:10:22.639911+0000","last_peered":"2024-01-15T18:10:22.639911+0000","last_clean":"2024-01-15T18:10:22.639911+0000","last_became_active":"2024-01-15T18:10:21.613679+0000","last_became_peered":"2024-01-15T18:10:21.613679+0000","last_unstale":"2024-01-15T18:10:22.639911+0000","last_undegraded":"2024-01-15T18:10:22.639911+0000","last_fullsized":"2024-01-15T18:10:22.639911+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:35:35.695097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542506+0000","last_change":"2024-01-15T18:10:22.633721+0000","last_active":"2024-01-15T18:10:24.542506+0000","last_peered":"2024-01-15T18:10:24.542506+0000","last_clean":"2024-01-15T18:10:24.542506+0000","last_became_active":"2024-01-15T18:10:22.633243+0000","last_became_peered":"2024-01-15T18:10:22.633243+0000","last_unstale":"2024-01-15T18:10:24.542506+0000","last_undegraded":"2024-01-15T18:10:24.542506+0000","last_fullsized":"2024-01-15T18:10:24.542506+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:08:26.541990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.629411+0000","last_change":"2024-01-15T18:10:21.618794+0000","last_active":"2024-01-15T18:10:22.629411+0000","last_peered":"2024-01-15T18:10:22.629411+0000","last_clean":"2024-01-15T18:10:22.629411+0000","last_became_active":"2024-01-15T18:10:21.618479+0000","last_became_peered":"2024-01-15T18:10:21.618479+0000","last_unstale":"2024-01-15T18:10:22.629411+0000","last_undegraded":"2024-01-15T18:10:22.629411+0000","last_fullsized":"2024-01-15T18:10:22.629411+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:31:30.348979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.412253+0000","last_change":"2024-01-15T18:10:22.636257+0000","last_active":"2024-01-15T18:10:24.412253+0000","last_peered":"2024-01-15T18:10:24.412253+0000","last_clean":"2024-01-15T18:10:24.412253+0000","last_became_active":"2024-01-15T18:10:22.635916+0000","last_became_peered":"2024-01-15T18:10:22.635916+0000","last_unstale":"2024-01-15T18:10:24.412253+0000","last_undegraded":"2024-01-15T18:10:24.412253+0000","last_fullsized":"2024-01-15T18:10:24.412253+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:26.803560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.639108+0000","last_change":"2024-01-15T18:10:21.608435+0000","last_active":"2024-01-15T18:10:22.639108+0000","last_peered":"2024-01-15T18:10:22.639108+0000","last_clean":"2024-01-15T18:10:22.639108+0000","last_became_active":"2024-01-15T18:10:21.608053+0000","last_became_peered":"2024-01-15T18:10:21.608053+0000","last_unstale":"2024-01-15T18:10:22.639108+0000","last_undegraded":"2024-01-15T18:10:22.639108+0000","last_fullsized":"2024-01-15T18:10:22.639108+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:04.043402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.606183+0000","last_change":"2024-01-15T18:10:22.634622+0000","last_active":"2024-01-15T18:10:23.606183+0000","last_peered":"2024-01-15T18:10:23.606183+0000","last_clean":"2024-01-15T18:10:23.606183+0000","last_became_active":"2024-01-15T18:10:22.634336+0000","last_became_peered":"2024-01-15T18:10:22.634336+0000","last_unstale":"2024-01-15T18:10:23.606183+0000","last_undegraded":"2024-01-15T18:10:23.606183+0000","last_fullsized":"2024-01-15T18:10:23.606183+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:39:01.560791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.617549+0000","last_change":"2024-01-15T18:10:21.620384+0000","last_active":"2024-01-15T18:10:22.617549+0000","last_peered":"2024-01-15T18:10:22.617549+0000","last_clean":"2024-01-15T18:10:22.617549+0000","last_became_active":"2024-01-15T18:10:21.617541+0000","last_became_peered":"2024-01-15T18:10:21.617541+0000","last_unstale":"2024-01-15T18:10:22.617549+0000","last_undegraded":"2024-01-15T18:10:22.617549+0000","last_fullsized":"2024-01-15T18:10:22.617549+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:04.043402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.353124+0000","last_change":"2024-01-15T18:10:22.610157+0000","last_active":"2024-01-15T18:10:24.353124+0000","last_peered":"2024-01-15T18:10:24.353124+0000","last_clean":"2024-01-15T18:10:24.353124+0000","last_became_active":"2024-01-15T18:10:22.609810+0000","last_became_peered":"2024-01-15T18:10:22.609810+0000","last_unstale":"2024-01-15T18:10:24.353124+0000","last_undegraded":"2024-01-15T18:10:24.353124+0000","last_fullsized":"2024-01-15T18:10:24.353124+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:31.941151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.634881+0000","last_change":"2024-01-15T18:10:21.620621+0000","last_active":"2024-01-15T18:10:22.634881+0000","last_peered":"2024-01-15T18:10:22.634881+0000","last_clean":"2024-01-15T18:10:22.634881+0000","last_became_active":"2024-01-15T18:10:21.620314+0000","last_became_peered":"2024-01-15T18:10:21.620314+0000","last_unstale":"2024-01-15T18:10:22.634881+0000","last_undegraded":"2024-01-15T18:10:22.634881+0000","last_fullsized":"2024-01-15T18:10:22.634881+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:29.249416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.353673+0000","last_change":"2024-01-15T18:10:22.619733+0000","last_active":"2024-01-15T18:10:24.353673+0000","last_peered":"2024-01-15T18:10:24.353673+0000","last_clean":"2024-01-15T18:10:24.353673+0000","last_became_active":"2024-01-15T18:10:22.619428+0000","last_became_peered":"2024-01-15T18:10:22.619428+0000","last_unstale":"2024-01-15T18:10:24.353673+0000","last_undegraded":"2024-01-15T18:10:24.353673+0000","last_fullsized":"2024-01-15T18:10:24.353673+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:01:57.679690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.621610+0000","last_change":"2024-01-15T18:10:21.614065+0000","last_active":"2024-01-15T18:10:22.621610+0000","last_peered":"2024-01-15T18:10:22.621610+0000","last_clean":"2024-01-15T18:10:22.621610+0000","last_became_active":"2024-01-15T18:10:21.613749+0000","last_became_peered":"2024-01-15T18:10:21.613749+0000","last_unstale":"2024-01-15T18:10:22.621610+0000","last_undegraded":"2024-01-15T18:10:22.621610+0000","last_fullsized":"2024-01-15T18:10:22.621610+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352680+0000","last_change":"2024-01-15T18:10:22.611155+0000","last_active":"2024-01-15T18:10:24.352680+0000","last_peered":"2024-01-15T18:10:24.352680+0000","last_clean":"2024-01-15T18:10:24.352680+0000","last_became_active":"2024-01-15T18:10:22.610491+0000","last_became_peered":"2024-01-15T18:10:22.610491+0000","last_unstale":"2024-01-15T18:10:24.352680+0000","last_undegraded":"2024-01-15T18:10:24.352680+0000","last_fullsized":"2024-01-15T18:10:24.352680+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:05.982376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.623192+0000","last_change":"2024-01-15T18:10:21.608384+0000","last_active":"2024-01-15T18:10:22.623192+0000","last_peered":"2024-01-15T18:10:22.623192+0000","last_clean":"2024-01-15T18:10:22.623192+0000","last_became_active":"2024-01-15T18:10:21.607947+0000","last_became_peered":"2024-01-15T18:10:21.607947+0000","last_unstale":"2024-01-15T18:10:22.623192+0000","last_undegraded":"2024-01-15T18:10:22.623192+0000","last_fullsized":"2024-01-15T18:10:22.623192+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:21:00.361963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541652+0000","last_change":"2024-01-15T18:10:22.625644+0000","last_active":"2024-01-15T18:10:24.541652+0000","last_peered":"2024-01-15T18:10:24.541652+0000","last_clean":"2024-01-15T18:10:24.541652+0000","last_became_active":"2024-01-15T18:10:22.625365+0000","last_became_peered":"2024-01-15T18:10:22.625365+0000","last_unstale":"2024-01-15T18:10:24.541652+0000","last_undegraded":"2024-01-15T18:10:24.541652+0000","last_fullsized":"2024-01-15T18:10:24.541652+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:30.260896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.609030+0000","last_change":"2024-01-15T18:10:21.620127+0000","last_active":"2024-01-15T18:10:22.609030+0000","last_peered":"2024-01-15T18:10:22.609030+0000","last_clean":"2024-01-15T18:10:22.609030+0000","last_became_active":"2024-01-15T18:10:21.619856+0000","last_became_peered":"2024-01-15T18:10:21.619856+0000","last_unstale":"2024-01-15T18:10:22.609030+0000","last_undegraded":"2024-01-15T18:10:22.609030+0000","last_fullsized":"2024-01-15T18:10:22.609030+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:56:04.291772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544396+0000","last_change":"2024-01-15T18:10:22.622337+0000","last_active":"2024-01-15T18:10:24.544396+0000","last_peered":"2024-01-15T18:10:24.544396+0000","last_clean":"2024-01-15T18:10:24.544396+0000","last_became_active":"2024-01-15T18:10:22.622029+0000","last_became_peered":"2024-01-15T18:10:22.622029+0000","last_unstale":"2024-01-15T18:10:24.544396+0000","last_undegraded":"2024-01-15T18:10:24.544396+0000","last_fullsized":"2024-01-15T18:10:24.544396+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:00.289344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.638565+0000","last_change":"2024-01-15T18:10:21.619938+0000","last_active":"2024-01-15T18:10:22.638565+0000","last_peered":"2024-01-15T18:10:22.638565+0000","last_clean":"2024-01-15T18:10:22.638565+0000","last_became_active":"2024-01-15T18:10:21.618751+0000","last_became_peered":"2024-01-15T18:10:21.618751+0000","last_unstale":"2024-01-15T18:10:22.638565+0000","last_undegraded":"2024-01-15T18:10:22.638565+0000","last_fullsized":"2024-01-15T18:10:22.638565+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544586+0000","last_change":"2024-01-15T18:10:22.624519+0000","last_active":"2024-01-15T18:10:24.544586+0000","last_peered":"2024-01-15T18:10:24.544586+0000","last_clean":"2024-01-15T18:10:24.544586+0000","last_became_active":"2024-01-15T18:10:22.624227+0000","last_became_peered":"2024-01-15T18:10:22.624227+0000","last_unstale":"2024-01-15T18:10:24.544586+0000","last_undegraded":"2024-01-15T18:10:24.544586+0000","last_fullsized":"2024-01-15T18:10:24.544586+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:01:57.679690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544659+0000","last_change":"2024-01-15T18:10:21.618976+0000","last_active":"2024-01-15T18:10:24.544659+0000","last_peered":"2024-01-15T18:10:24.544659+0000","last_clean":"2024-01-15T18:10:24.544659+0000","last_became_active":"2024-01-15T18:10:21.618682+0000","last_became_peered":"2024-01-15T18:10:21.618682+0000","last_unstale":"2024-01-15T18:10:24.544659+0000","last_undegraded":"2024-01-15T18:10:24.544659+0000","last_fullsized":"2024-01-15T18:10:24.544659+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:16.982908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544175+0000","last_change":"2024-01-15T18:10:22.633839+0000","last_active":"2024-01-15T18:10:24.544175+0000","last_peered":"2024-01-15T18:10:24.544175+0000","last_clean":"2024-01-15T18:10:24.544175+0000","last_became_active":"2024-01-15T18:10:22.633500+0000","last_became_peered":"2024-01-15T18:10:22.633500+0000","last_unstale":"2024-01-15T18:10:24.544175+0000","last_undegraded":"2024-01-15T18:10:24.544175+0000","last_fullsized":"2024-01-15T18:10:24.544175+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:31.941151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.618013+0000","last_change":"2024-01-15T18:10:21.623001+0000","last_active":"2024-01-15T18:10:22.618013+0000","last_peered":"2024-01-15T18:10:22.618013+0000","last_clean":"2024-01-15T18:10:22.618013+0000","last_became_active":"2024-01-15T18:10:21.622691+0000","last_became_peered":"2024-01-15T18:10:21.622691+0000","last_unstale":"2024-01-15T18:10:22.618013+0000","last_undegraded":"2024-01-15T18:10:22.618013+0000","last_fullsized":"2024-01-15T18:10:22.618013+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:13:31.746119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541936+0000","last_change":"2024-01-15T18:10:22.625848+0000","last_active":"2024-01-15T18:10:24.541936+0000","last_peered":"2024-01-15T18:10:24.541936+0000","last_clean":"2024-01-15T18:10:24.541936+0000","last_became_active":"2024-01-15T18:10:22.625079+0000","last_became_peered":"2024-01-15T18:10:22.625079+0000","last_unstale":"2024-01-15T18:10:24.541936+0000","last_undegraded":"2024-01-15T18:10:24.541936+0000","last_fullsized":"2024-01-15T18:10:24.541936+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:02:38.272375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.621043+0000","last_change":"2024-01-15T18:10:21.618463+0000","last_active":"2024-01-15T18:10:22.621043+0000","last_peered":"2024-01-15T18:10:22.621043+0000","last_clean":"2024-01-15T18:10:22.621043+0000","last_became_active":"2024-01-15T18:10:21.618150+0000","last_became_peered":"2024-01-15T18:10:21.618150+0000","last_unstale":"2024-01-15T18:10:22.621043+0000","last_undegraded":"2024-01-15T18:10:22.621043+0000","last_fullsized":"2024-01-15T18:10:22.621043+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:25.499049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.353411+0000","last_change":"2024-01-15T18:10:22.610959+0000","last_active":"2024-01-15T18:10:24.353411+0000","last_peered":"2024-01-15T18:10:24.353411+0000","last_clean":"2024-01-15T18:10:24.353411+0000","last_became_active":"2024-01-15T18:10:22.609991+0000","last_became_peered":"2024-01-15T18:10:22.609991+0000","last_unstale":"2024-01-15T18:10:24.353411+0000","last_undegraded":"2024-01-15T18:10:24.353411+0000","last_fullsized":"2024-01-15T18:10:24.353411+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:08:18.277916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.618356+0000","last_change":"2024-01-15T18:10:21.613099+0000","last_active":"2024-01-15T18:10:22.618356+0000","last_peered":"2024-01-15T18:10:22.618356+0000","last_clean":"2024-01-15T18:10:22.618356+0000","last_became_active":"2024-01-15T18:10:21.612818+0000","last_became_peered":"2024-01-15T18:10:21.612818+0000","last_unstale":"2024-01-15T18:10:22.618356+0000","last_undegraded":"2024-01-15T18:10:22.618356+0000","last_fullsized":"2024-01-15T18:10:22.618356+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:02:37.260895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.602721+0000","last_change":"2024-01-15T18:10:22.633062+0000","last_active":"2024-01-15T18:10:23.602721+0000","last_peered":"2024-01-15T18:10:23.602721+0000","last_clean":"2024-01-15T18:10:23.602721+0000","last_became_active":"2024-01-15T18:10:22.632748+0000","last_became_peered":"2024-01-15T18:10:22.632748+0000","last_unstale":"2024-01-15T18:10:23.602721+0000","last_undegraded":"2024-01-15T18:10:23.602721+0000","last_fullsized":"2024-01-15T18:10:23.602721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:05.054882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.618599+0000","last_change":"2024-01-15T18:10:21.610659+0000","last_active":"2024-01-15T18:10:22.618599+0000","last_peered":"2024-01-15T18:10:22.618599+0000","last_clean":"2024-01-15T18:10:22.618599+0000","last_became_active":"2024-01-15T18:10:21.610364+0000","last_became_peered":"2024-01-15T18:10:21.610364+0000","last_unstale":"2024-01-15T18:10:22.618599+0000","last_undegraded":"2024-01-15T18:10:22.618599+0000","last_fullsized":"2024-01-15T18:10:22.618599+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.176712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352674+0000","last_change":"2024-01-15T18:10:22.617540+0000","last_active":"2024-01-15T18:10:24.352674+0000","last_peered":"2024-01-15T18:10:24.352674+0000","last_clean":"2024-01-15T18:10:24.352674+0000","last_became_active":"2024-01-15T18:10:22.617134+0000","last_became_peered":"2024-01-15T18:10:22.617134+0000","last_unstale":"2024-01-15T18:10:24.352674+0000","last_undegraded":"2024-01-15T18:10:24.352674+0000","last_fullsized":"2024-01-15T18:10:24.352674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:20:22.417375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.597058+0000","last_change":"2024-01-15T18:10:21.613230+0000","last_active":"2024-01-15T18:10:22.597058+0000","last_peered":"2024-01-15T18:10:22.597058+0000","last_clean":"2024-01-15T18:10:22.597058+0000","last_became_active":"2024-01-15T18:10:21.612943+0000","last_became_peered":"2024-01-15T18:10:21.612943+0000","last_unstale":"2024-01-15T18:10:22.597058+0000","last_undegraded":"2024-01-15T18:10:22.597058+0000","last_fullsized":"2024-01-15T18:10:22.597058+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:32.108693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541750+0000","last_change":"2024-01-15T18:10:22.628468+0000","last_active":"2024-01-15T18:10:24.541750+0000","last_peered":"2024-01-15T18:10:24.541750+0000","last_clean":"2024-01-15T18:10:24.541750+0000","last_became_active":"2024-01-15T18:10:22.627865+0000","last_became_peered":"2024-01-15T18:10:22.627865+0000","last_unstale":"2024-01-15T18:10:24.541750+0000","last_undegraded":"2024-01-15T18:10:24.541750+0000","last_fullsized":"2024-01-15T18:10:24.541750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:37.061370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.597495+0000","last_change":"2024-01-15T18:10:21.621660+0000","last_active":"2024-01-15T18:10:22.597495+0000","last_peered":"2024-01-15T18:10:22.597495+0000","last_clean":"2024-01-15T18:10:22.597495+0000","last_became_active":"2024-01-15T18:10:21.620662+0000","last_became_peered":"2024-01-15T18:10:21.620662+0000","last_unstale":"2024-01-15T18:10:22.597495+0000","last_undegraded":"2024-01-15T18:10:22.597495+0000","last_fullsized":"2024-01-15T18:10:22.597495+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:10.136007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.354289+0000","last_change":"2024-01-15T18:10:22.613744+0000","last_active":"2024-01-15T18:10:24.354289+0000","last_peered":"2024-01-15T18:10:24.354289+0000","last_clean":"2024-01-15T18:10:24.354289+0000","last_became_active":"2024-01-15T18:10:22.609665+0000","last_became_peered":"2024-01-15T18:10:22.609665+0000","last_unstale":"2024-01-15T18:10:24.354289+0000","last_undegraded":"2024-01-15T18:10:24.354289+0000","last_fullsized":"2024-01-15T18:10:24.354289+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:11.522155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.628995+0000","last_change":"2024-01-15T18:10:21.624625+0000","last_active":"2024-01-15T18:10:22.628995+0000","last_peered":"2024-01-15T18:10:22.628995+0000","last_clean":"2024-01-15T18:10:22.628995+0000","last_became_active":"2024-01-15T18:10:21.624271+0000","last_became_peered":"2024-01-15T18:10:21.624271+0000","last_unstale":"2024-01-15T18:10:22.628995+0000","last_undegraded":"2024-01-15T18:10:22.628995+0000","last_fullsized":"2024-01-15T18:10:22.628995+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:41:48.880774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":235,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.606602+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:10:22.606602+0000","last_peered":"2024-01-15T18:10:22.606602+0000","last_clean":"2024-01-15T18:10:22.606602+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:10:22.606602+0000","last_undegraded":"2024-01-15T18:10:22.606602+0000","last_fullsized":"2024-01-15T18:10:22.606602+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542333,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85736,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":83902,"kb_avail":94286104,"statfs":{"total":96636764160,"available":96548970496,"internally_reserved":0,"allocated":1859584,"data_stored":1137191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":85916106},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3280000000000001,"15min":1.3740000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.40400000000000003},"max":{"1min":1.742,"5min":11.449999999999999,"15min":19.843},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.3759999999999999,"15min":1.629},"min":{"1min":0.89200000000000002,"5min":0.61499999999999999,"15min":0.53400000000000003},"max":{"1min":7.8250000000000002,"5min":7.8250000000000002,"15min":38.137},"last":1.0329999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.319,"15min":1.411},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.45600000000000002},"max":{"1min":1.577,"5min":11.51,"15min":19.917000000000002},"last":1.613},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.4079999999999999,"15min":1.605},"min":{"1min":0.85699999999999998,"5min":0.81100000000000005,"15min":0.42999999999999999},"max":{"1min":1.841,"5min":6.7480000000000002,"15min":51.843000000000004},"last":1.3779999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.371,"15min":1.425},"min":{"1min":0.97199999999999998,"5min":0.67200000000000004,"15min":0.437},"max":{"1min":3.3900000000000001,"5min":12.061,"15min":12.061},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4430000000000001,"15min":1.532},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.47399999999999998},"max":{"1min":1.806,"5min":11.66,"15min":13.478999999999999},"last":1.8180000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.544},"min":{"1min":0.85299999999999998,"5min":0.83199999999999996,"15min":0.47899999999999998},"max":{"1min":1.9079999999999999,"5min":7.4669999999999996,"15min":16.632000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.5069999999999999,"15min":1.6160000000000001},"min":{"1min":0.97299999999999998,"5min":0.94199999999999995,"15min":0.56699999999999995},"max":{"1min":1.8520000000000001,"5min":11.611000000000001,"15min":13.439},"last":1.919}]},{"osd":4,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":2.319,"5min":1.728,"15min":1.7589999999999999},"min":{"1min":1.0169999999999999,"5min":0.81000000000000005,"15min":0.50700000000000001},"max":{"1min":17.067,"5min":23.042000000000002,"15min":23.042000000000002},"last":1.456},{"interface":"front","average":{"1min":1.425,"5min":1.5269999999999999,"15min":1.7270000000000001},"min":{"1min":0.94599999999999995,"5min":0.78800000000000003,"15min":0.51800000000000002},"max":{"1min":2.0779999999999998,"5min":16.867999999999999,"15min":38.682000000000002},"last":1.4119999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5640000000000001,"15min":1.8089999999999999},"min":{"1min":1.0329999999999999,"5min":0.89800000000000002,"15min":0.52700000000000002},"max":{"1min":1.843,"5min":11.696999999999999,"15min":25.006},"last":1.97},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.7110000000000001,"15min":1.8400000000000001},"min":{"1min":1.0880000000000001,"5min":0.97499999999999998,"15min":0.624},"max":{"1min":2.04,"5min":22.933,"15min":22.933},"last":1.774}]},{"osd":6,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.9850000000000001,"15min":2.0419999999999998},"min":{"1min":1.1870000000000001,"5min":1.123,"15min":0.59299999999999997},"max":{"1min":2.2229999999999999,"5min":23.004000000000001,"15min":23.834},"last":1.861},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.607,"15min":1.998},"min":{"1min":1.0680000000000001,"5min":0.84099999999999997,"15min":0.58299999999999996},"max":{"1min":2.1539999999999999,"5min":7.5220000000000002,"15min":63.932000000000002},"last":1.7230000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542332,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80936,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290904,"statfs":{"total":96636764160,"available":96553885696,"internally_reserved":0,"allocated":1265664,"data_stored":547363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81590726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":2.597,"5min":1.746,"15min":1.502},"min":{"1min":0.68700000000000006,"5min":0.55900000000000005,"15min":0.53100000000000003},"max":{"1min":31.805,"5min":31.805,"15min":31.805},"last":1.125},{"interface":"front","average":{"1min":2.7010000000000001,"5min":1.591,"15min":1.51},"min":{"1min":0.73199999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":31.702999999999999,"5min":31.702999999999999,"15min":31.702999999999999},"last":1.1950000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.357,"15min":1.3480000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.55100000000000005},"max":{"1min":1.6719999999999999,"5min":15.064,"15min":15.064},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.5229999999999999,"15min":1.54},"min":{"1min":0.86499999999999999,"5min":0.74199999999999999,"15min":0.66200000000000003},"max":{"1min":4.0579999999999998,"5min":23.495999999999999,"15min":23.495999999999999},"last":1.492}]},{"osd":2,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.629,"15min":1.5980000000000001},"min":{"1min":0.89400000000000002,"5min":0.73799999999999999,"15min":0.59399999999999997},"max":{"1min":3.9849999999999999,"5min":23.087,"15min":23.087},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.4950000000000001,"15min":1.4430000000000001},"min":{"1min":0.67900000000000005,"5min":0.51400000000000001,"15min":0.504},"max":{"1min":2.048,"5min":14.94,"15min":14.94},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.611,"15min":1.6890000000000001},"min":{"1min":0.71799999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.1269999999999998,"5min":15.151999999999999,"15min":15.151999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4319999999999999,"15min":1.6040000000000001},"min":{"1min":0.874,"5min":0.76100000000000001,"15min":0.70699999999999996},"max":{"1min":2.431,"5min":4.6849999999999996,"15min":14.066000000000001},"last":1.2989999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.6060000000000001,"15min":1.663},"min":{"1min":0.92000000000000004,"5min":0.56100000000000005,"15min":0.51800000000000002},"max":{"1min":3.9169999999999998,"5min":23.327000000000002,"15min":23.327000000000002},"last":1.345},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.74,"15min":1.704},"min":{"1min":0.76000000000000001,"5min":0.61899999999999999,"15min":0.45800000000000002},"max":{"1min":2.9929999999999999,"5min":15.239000000000001,"15min":17.289000000000001},"last":1.2609999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.413,"15min":1.5629999999999999},"min":{"1min":0.81499999999999995,"5min":0.52500000000000002,"15min":0.51600000000000001},"max":{"1min":2.6949999999999998,"5min":4.9260000000000002,"15min":14.176},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.8879999999999999,"15min":1.8660000000000001},"min":{"1min":1.335,"5min":0.64400000000000002,"15min":0.54700000000000004},"max":{"1min":4.1790000000000003,"5min":23.417999999999999,"15min":23.417999999999999},"last":1.6160000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5629999999999999,"15min":1.7669999999999999},"min":{"1min":0.84999999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.5880000000000001,"5min":4.8239999999999998,"15min":21.265000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.6890000000000001,"5min":2.0289999999999999,"15min":1.99},"min":{"1min":1.163,"5min":0.77100000000000002,"15min":0.59699999999999998},"max":{"1min":4.0220000000000002,"5min":23.738,"15min":23.738},"last":1.5640000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58780,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313060,"statfs":{"total":96636764160,"available":96576573440,"internally_reserved":0,"allocated":1261568,"data_stored":546781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1870000000000001,"15min":1.282},"min":{"1min":0.72199999999999998,"5min":0.69599999999999995,"15min":0.47799999999999998},"max":{"1min":1.649,"5min":2.2839999999999998,"15min":19.266999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1870000000000001,"15min":1.337},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.51400000000000001},"max":{"1min":1.6930000000000001,"5min":2.1869999999999998,"15min":17.184999999999999},"last":1.3839999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.234,"15min":1.3380000000000001},"min":{"1min":0.91300000000000003,"5min":0.69699999999999995,"15min":0.499},"max":{"1min":1.472,"5min":5.3879999999999999,"15min":16.297000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.3,"5min":1.29,"15min":1.335},"min":{"1min":0.84399999999999997,"5min":0.78900000000000003,"15min":0.59099999999999997},"max":{"1min":1.833,"5min":2.1200000000000001,"15min":9.2379999999999995},"last":1.4650000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2350000000000001,"15min":1.264},"min":{"1min":0.876,"5min":0.78600000000000003,"15min":0.621},"max":{"1min":1.915,"5min":1.927,"15min":8.1820000000000004},"last":1.585},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.345,"15min":1.4379999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.56000000000000005},"max":{"1min":1.748,"5min":2.5539999999999998,"15min":16.158999999999999},"last":1.524}]},{"osd":3,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3660000000000001,"15min":1.49},"min":{"1min":0.88,"5min":0.84799999999999998,"15min":0.56100000000000005},"max":{"1min":1.6819999999999999,"5min":2.5179999999999998,"15min":14.914},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4259999999999999,"15min":1.454},"min":{"1min":1.044,"5min":0.93200000000000005,"15min":0.63300000000000001},"max":{"1min":1.782,"5min":1.98,"15min":7.0110000000000001},"last":1.7030000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.3420000000000001,"15min":1.4990000000000001},"min":{"1min":0.88700000000000001,"5min":0.51900000000000002,"15min":0.41399999999999998},"max":{"1min":1.855,"5min":2.4689999999999999,"15min":16.414999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4299999999999999,"15min":1.621},"min":{"1min":0.93000000000000005,"5min":0.56200000000000006,"15min":0.32800000000000001},"max":{"1min":1.8500000000000001,"5min":2.665,"15min":17.991},"last":1.2829999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4390000000000001,"15min":1.4830000000000001},"min":{"1min":1.0249999999999999,"5min":0.72699999999999998,"15min":0.39600000000000002},"max":{"1min":1.849,"5min":3.7559999999999998,"15min":6.4630000000000001},"last":1.131},{"interface":"front","average":{"1min":1.474,"5min":1.4750000000000001,"15min":1.593},"min":{"1min":1.0109999999999999,"5min":0.56899999999999995,"15min":0.53000000000000003},"max":{"1min":1.9790000000000001,"5min":2.7010000000000001,"15min":16.364000000000001},"last":1.6850000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.591,"15min":1.6140000000000001},"min":{"1min":1.0620000000000001,"5min":0.78900000000000003,"15min":0.45800000000000002},"max":{"1min":1.887,"5min":5.0659999999999998,"15min":9.0600000000000005},"last":1.837},{"interface":"front","average":{"1min":1.575,"5min":1.585,"15min":1.698},"min":{"1min":1.1779999999999999,"5min":0.98599999999999999,"15min":0.57299999999999995},"max":{"1min":2.2029999999999998,"5min":2.6059999999999999,"15min":19.356999999999999},"last":1.8979999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542332,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76780,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295060,"statfs":{"total":96636764160,"available":96558141440,"internally_reserved":0,"allocated":1277952,"data_stored":548447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.984,"5min":1.4350000000000001,"15min":1.3919999999999999},"min":{"1min":0.76700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":17.346,"5min":17.346,"15min":17.346},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.345,"15min":1.3899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.7170000000000001,"5min":3.726,"15min":9.7439999999999998},"last":1.5469999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.208,"15min":1.347},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4870000000000001,"5min":2.8700000000000001,"15min":11.987},"last":1.006},{"interface":"front","average":{"1min":1.274,"5min":1.2230000000000001,"15min":1.304},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.55300000000000005},"max":{"1min":3.7370000000000001,"5min":3.7370000000000001,"15min":16.058},"last":1.3779999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.9470000000000001,"5min":1.389,"15min":1.526},"min":{"1min":0.64500000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":18.442,"5min":18.442,"15min":18.442},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3700000000000001,"15min":1.526},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.69499999999999995},"max":{"1min":3.8250000000000002,"5min":3.8250000000000002,"15min":13.903},"last":1.742}]},{"osd":3,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.948,"5min":1.4650000000000001,"15min":1.6319999999999999},"min":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":17.364999999999998,"5min":17.364999999999998,"15min":26.055},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.333,"15min":1.46},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5449999999999999,"5min":2.9279999999999999,"15min":13.25},"last":1.3340000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.8839999999999999,"5min":1.458,"15min":1.5820000000000001},"min":{"1min":0.50700000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":17.292999999999999,"5min":17.292999999999999,"15min":29.422000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.5469999999999999,"15min":1.5189999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.42199999999999999},"max":{"1min":4.9400000000000004,"5min":12.332000000000001,"15min":12.332000000000001},"last":1.2010000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.5269999999999999,"15min":1.524},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45200000000000001},"max":{"1min":5.3150000000000004,"5min":11.09,"15min":18.260000000000002},"last":1.591},{"interface":"front","average":{"1min":2.0910000000000002,"5min":1.768,"15min":1.712},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.48099999999999998},"max":{"1min":17.224,"5min":17.224,"15min":17.224},"last":1.6870000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.6519999999999999,"15min":1.752},"min":{"1min":0.625,"5min":0.625,"15min":0.503},"max":{"1min":1.8560000000000001,"5min":12.247,"15min":12.247},"last":1.6579999999999999},{"interface":"front","average":{"1min":2.2029999999999998,"5min":1.8919999999999999,"15min":1.8839999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.53600000000000003},"max":{"1min":17.324999999999999,"5min":17.324999999999999,"15min":17.324999999999999},"last":1.79}]}]},{"osd":3,"up_from":14,"seq":60129542334,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72620,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299220,"statfs":{"total":96636764160,"available":96562401280,"internally_reserved":0,"allocated":1273856,"data_stored":548345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.113,"15min":1.2210000000000001},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.33300000000000002},"max":{"1min":1.284,"5min":9.2899999999999991,"15min":18.399999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.048,"5min":1.127,"15min":1.387},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":1.8140000000000001,"5min":2.23,"15min":21.047000000000001},"last":1.3540000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.92400000000000004,"15min":1.272},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.415,"5min":3.0409999999999999,"15min":17.777999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.3680000000000001,"15min":1.288},"min":{"1min":0.57399999999999995,"5min":0.46300000000000002,"15min":0.39500000000000002},"max":{"1min":1.6890000000000001,"5min":11.228999999999999,"15min":11.228999999999999},"last":1.073}]},{"osd":2,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.224,"15min":1.2789999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.518,"5min":2.7519999999999998,"15min":10.964},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.5489999999999999,"15min":1.554},"min":{"1min":0.60799999999999998,"5min":0.57099999999999995,"15min":0.54000000000000004},"max":{"1min":1.857,"5min":9.4920000000000009,"15min":20.495000000000001},"last":1.536}]},{"osd":4,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.401,"15min":1.5589999999999999},"min":{"1min":0.871,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.6419999999999999,"5min":6.319,"15min":13.536},"last":1.135},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.456,"15min":1.6399999999999999},"min":{"1min":0.748,"5min":0.63700000000000001,"15min":0.54600000000000004},"max":{"1min":1.736,"5min":16.398,"15min":16.398},"last":1.2430000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.681,"15min":1.6950000000000001},"min":{"1min":1.046,"5min":0.875,"15min":0.57299999999999995},"max":{"1min":1.6819999999999999,"5min":9.5690000000000008,"15min":20.606000000000002},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5589999999999999,"15min":1.665},"min":{"1min":0.97299999999999998,"5min":0.70199999999999996,"15min":0.68000000000000005},"max":{"1min":1.758,"5min":5.6980000000000004,"15min":11.603},"last":1.577}]},{"osd":6,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.615,"15min":1.7949999999999999},"min":{"1min":0.81000000000000005,"5min":0.61399999999999999,"15min":0.53500000000000003},"max":{"1min":2.0870000000000002,"5min":13.648999999999999,"15min":13.648999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.502,"5min":1.8109999999999999,"15min":1.8220000000000001},"min":{"1min":0.98699999999999999,"5min":0.78300000000000003,"15min":0.65800000000000003},"max":{"1min":2.0249999999999999,"5min":13.679,"15min":20.556000000000001},"last":1.288}]},{"osd":7,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.552,"15min":1.8480000000000001},"min":{"1min":0.78800000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8260000000000001,"5min":3.5590000000000002,"15min":28.843},"last":1.498},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.5760000000000001,"15min":1.857},"min":{"1min":0.97999999999999998,"5min":0.72799999999999998,"15min":0.67200000000000004},"max":{"1min":1.968,"5min":9.4100000000000001,"15min":15.143000000000001},"last":1.4530000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542334,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68444,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303396,"statfs":{"total":96636764160,"available":96566677504,"internally_reserved":0,"allocated":1261568,"data_stored":546745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.091},"min":{"1min":0.48699999999999999,"5min":0.32100000000000001,"15min":0.26600000000000001},"max":{"1min":1.728,"5min":5.1859999999999999,"15min":14.427},"last":1.169},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.3779999999999999,"15min":1.3460000000000001},"min":{"1min":0.61799999999999999,"5min":0.50900000000000001,"15min":0.39300000000000002},"max":{"1min":1.7829999999999999,"5min":15.314,"15min":15.314},"last":1.006}]},{"osd":1,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2649999999999999,"15min":1.4370000000000001},"min":{"1min":0.67600000000000005,"5min":0.51500000000000001,"15min":0.39200000000000002},"max":{"1min":1.641,"5min":9.0969999999999995,"15min":34.463999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.524,"15min":1.4530000000000001},"min":{"1min":0.88200000000000001,"5min":0.54200000000000004,"15min":0.373},"max":{"1min":1.6120000000000001,"5min":15.407,"15min":25.859999999999999},"last":1.351}]},{"osd":3,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.26,"15min":1.4179999999999999},"min":{"1min":0.78500000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.7529999999999999,"5min":12.449999999999999,"15min":19.535},"last":1.282},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.518,"15min":1.6160000000000001},"min":{"1min":0.875,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":1.7829999999999999,"5min":22.783999999999999,"15min":22.783999999999999},"last":1.5780000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4710000000000001,"15min":1.774},"min":{"1min":1.018,"5min":0.73199999999999998,"15min":0.55700000000000005},"max":{"1min":1.9139999999999999,"5min":4.9429999999999996,"15min":29.986999999999998},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.73,"15min":1.736},"min":{"1min":1.0329999999999999,"5min":0.81000000000000005,"15min":0.56799999999999995},"max":{"1min":1.843,"5min":15.553000000000001,"15min":21.199999999999999},"last":1.6379999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.766,"15min":1.738},"min":{"1min":1.0569999999999999,"5min":0.84299999999999997,"15min":0.51500000000000001},"max":{"1min":1.881,"5min":14.066000000000001,"15min":19.681999999999999},"last":1.468},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.879,"15min":1.8220000000000001},"min":{"1min":1.2929999999999999,"5min":0.84799999999999998,"15min":0.60899999999999999},"max":{"1min":2.2189999999999999,"5min":14.025,"15min":14.702999999999999},"last":1.8520000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.889,"15min":1.788},"min":{"1min":1.024,"5min":0.76800000000000002,"15min":0.68000000000000005},"max":{"1min":1.976,"5min":15.683999999999999,"15min":15.683999999999999},"last":1.516},{"interface":"front","average":{"1min":1.655,"5min":2.0299999999999998,"15min":1.867},"min":{"1min":1.069,"5min":0.93500000000000005,"15min":0.58899999999999997},"max":{"1min":2.2879999999999998,"5min":12.629,"15min":19.783000000000001},"last":2.0569999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.9890000000000001,"15min":2.0699999999999998},"min":{"1min":1.4359999999999999,"5min":0.86899999999999999,"15min":0.63100000000000001},"max":{"1min":1.968,"5min":17.099,"15min":57.006999999999998},"last":1.786},{"interface":"front","average":{"1min":1.7250000000000001,"5min":2.0859999999999999,"15min":2.2080000000000002},"min":{"1min":1.131,"5min":1.081,"15min":0.56100000000000005},"max":{"1min":2.3359999999999999,"5min":22.853999999999999,"15min":22.853999999999999},"last":2.2130000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542333,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72600,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299240,"statfs":{"total":96636764160,"available":96562421760,"internally_reserved":0,"allocated":1257472,"data_stored":546711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.363,"15min":1.2070000000000001},"min":{"1min":0.49399999999999999,"5min":0.48899999999999999,"15min":0.34399999999999997},"max":{"1min":2.3839999999999999,"5min":30.460999999999999,"15min":30.460999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.1759999999999999,"15min":1.117},"min":{"1min":0.61499999999999999,"5min":0.38700000000000001,"15min":0.314},"max":{"1min":2.0960000000000001,"5min":4.4059999999999997,"15min":18.658000000000001},"last":0.80500000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":1.198,"15min":1.1970000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":2.0619999999999998,"5min":7.117,"15min":21.126000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.429,"5min":1.752,"15min":1.5620000000000001},"min":{"1min":0.75700000000000001,"5min":0.70899999999999996,"15min":0.48699999999999999},"max":{"1min":2.1499999999999999,"5min":30.593,"15min":30.593},"last":1.214}]},{"osd":3,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.6719999999999999,"15min":1.617},"min":{"1min":0.76100000000000001,"5min":0.58099999999999996,"15min":0.48599999999999999},"max":{"1min":2.0579999999999998,"5min":30.547000000000001,"15min":30.547000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3480000000000001,"15min":1.3420000000000001},"min":{"1min":0.84999999999999998,"5min":0.52700000000000002,"15min":0.437},"max":{"1min":2.21,"5min":5.4729999999999999,"15min":15.510999999999999},"last":1.056}]},{"osd":4,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.5900000000000001,"5min":1.663,"15min":1.6990000000000001},"min":{"1min":1.111,"5min":0.75900000000000001,"15min":0.72499999999999998},"max":{"1min":2.7599999999999998,"5min":8.8170000000000002,"15min":14.827},"last":1.675},{"interface":"front","average":{"1min":1.53,"5min":1.5429999999999999,"15min":1.657},"min":{"1min":0.95699999999999996,"5min":0.72999999999999998,"15min":0.69299999999999995},"max":{"1min":2.359,"5min":3.7440000000000002,"15min":14.851000000000001},"last":0.98899999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5349999999999999,"15min":1.6160000000000001},"min":{"1min":1.036,"5min":0.77600000000000002,"15min":0.59599999999999997},"max":{"1min":2.266,"5min":3.798,"15min":13.228999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.593,"5min":1.986,"15min":1.9450000000000001},"min":{"1min":1.0429999999999999,"5min":0.78700000000000003,"15min":0.68300000000000005},"max":{"1min":2.2949999999999999,"5min":34.042999999999999,"15min":34.042999999999999},"last":1.3220000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.579,"15min":1.7829999999999999},"min":{"1min":0.90100000000000002,"5min":0.83899999999999997,"15min":0.64100000000000001},"max":{"1min":2.5550000000000002,"5min":3.5539999999999998,"15min":34.158000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.736,"15min":1.6299999999999999},"min":{"1min":1.115,"5min":0.67900000000000005,"15min":0.62},"max":{"1min":4.1699999999999999,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.5529999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.6950000000000001,"5min":2.0979999999999999,"15min":1.8660000000000001},"min":{"1min":0.95999999999999996,"5min":0.77300000000000002,"15min":0.64600000000000002},"max":{"1min":2.5750000000000002,"5min":33.659999999999997,"15min":33.659999999999997},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.694,"5min":1.8540000000000001,"15min":1.8009999999999999},"min":{"1min":1.002,"5min":0.81000000000000005,"15min":0.60499999999999998},"max":{"1min":2.613,"5min":11.882999999999999,"15min":24.289999999999999},"last":1.427}]}]},{"osd":0,"up_from":14,"seq":60129542334,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69028,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302812,"statfs":{"total":96636764160,"available":96566079488,"internally_reserved":0,"allocated":1855488,"data_stored":1137661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2210000000000001,"15min":1.083},"min":{"1min":0.437,"5min":0.42799999999999999,"15min":0.312},"max":{"1min":2.8929999999999998,"5min":12.050000000000001,"15min":12.050000000000001},"last":1.389},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.3340000000000001},"min":{"1min":0.50900000000000001,"5min":0.49099999999999999,"15min":0.36199999999999999},"max":{"1min":2.9780000000000002,"5min":11.962,"15min":11.962},"last":1.359}]},{"osd":2,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.179,"15min":1.2090000000000001},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.27600000000000002},"max":{"1min":1.595,"5min":7.4560000000000004,"15min":11.59},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.1459999999999999,"15min":1.167},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35899999999999999},"max":{"1min":1.5649999999999999,"5min":7.3239999999999998,"15min":14.512},"last":0.876}]},{"osd":3,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.425,"15min":1.4319999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.41699999999999998},"max":{"1min":1.5669999999999999,"5min":11.904,"15min":11.904},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1379999999999999,"15min":1.2490000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.29599999999999999},"max":{"1min":1.6850000000000001,"5min":3.202,"15min":14.74},"last":1.744}]},{"osd":4,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.375,"15min":1.5900000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6140000000000001,"5min":7.4000000000000004,"15min":32.115000000000002},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.284,"5min":1.571,"15min":1.702},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.67700000000000005},"max":{"1min":1.8260000000000001,"5min":9.0790000000000006,"15min":15.975},"last":1.498}]},{"osd":5,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.556,"15min":1.722},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":1.6200000000000001,"5min":12.006,"15min":32.174999999999997},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.514,"15min":1.6359999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.442},"max":{"1min":1.8939999999999999,"5min":4.7119999999999997,"15min":17.225000000000001},"last":1.5569999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.7849999999999999,"15min":1.7989999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.70299999999999996},"max":{"1min":3.4399999999999999,"5min":24.039999999999999,"15min":24.039999999999999},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.7549999999999999,"15min":1.794},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.625},"max":{"1min":1.7450000000000001,"5min":18.143999999999998,"15min":32.216999999999999},"last":1.0369999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.853,"15min":2.1230000000000002},"min":{"1min":0.754,"5min":0.754,"15min":0.69099999999999995},"max":{"1min":1.7350000000000001,"5min":26.427,"15min":34.131},"last":1.611},{"interface":"front","average":{"1min":1.474,"5min":2.089,"15min":2.0379999999999998},"min":{"1min":0.753,"5min":0.753,"15min":0.61799999999999999},"max":{"1min":3.9790000000000001,"5min":44.694000000000003,"15min":44.694000000000003},"last":1.806}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:10:30.975 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:10:30.983 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:10:32.237 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:10:32.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.450+0000 7f0a9ee00640 1 Processor -- start 2024-01-15T18:10:32.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.450+0000 7f0a9ee00640 1 -- start start 2024-01-15T18:10:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.452+0000 7f0a9ee00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0a98157c30 0x7f0a980cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.452+0000 7f0a9ee00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a981618f0 0x7f0a98070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.452+0000 7f0a9ee00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a98071450 con 0x7f0a98071030 2024-01-15T18:10:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.452+0000 7f0a9ee00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9815ea30 con 0x7f0a981618f0 2024-01-15T18:10:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.452+0000 7f0a9ee00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9806d860 con 0x7f0a98157c30 2024-01-15T18:10:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.452+0000 7f0a8fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a981618f0 0x7f0a98070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.452+0000 7f0a8fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0a981618f0 0x7f0a98070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54004/0 (socket says 172.21.15.26:54004) 2024-01-15T18:10:32.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.452+0000 7f0a8fe00640 1 -- 172.21.15.26:0/1673650722 learned_addr learned my addr 172.21.15.26:0/1673650722 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:32.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.453+0000 7f0a8fe00640 1 -- 172.21.15.26:0/1673650722 >> v2:172.21.15.124:3300/0 conn(0x7f0a98157c30 msgr2=0x7f0a980cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:32.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.453+0000 7f0a8fe00640 1 --2- 172.21.15.26:0/1673650722 >> v2:172.21.15.124:3300/0 conn(0x7f0a98157c30 0x7f0a980cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.453+0000 7f0a8fe00640 1 -- 172.21.15.26:0/1673650722 >> v1:172.21.15.124:6789/0 conn(0x7f0a98071030 legacy=0x7f0a9806dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:32.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.453+0000 7f0a8fe00640 1 -- 172.21.15.26:0/1673650722 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9815deb0 con 0x7f0a981618f0 2024-01-15T18:10:32.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.454+0000 7f0a8fe00640 1 --2- 172.21.15.26:0/1673650722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a981618f0 0x7f0a98070af0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0a980723f0 tx=0x7f0a8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd26ed7618ec046 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:32.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.454+0000 7f0a97400640 1 -- 172.21.15.26:0/1673650722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80031870 con 0x7f0a981618f0 2024-01-15T18:10:32.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.454+0000 7f0a97400640 1 -- 172.21.15.26:0/1673650722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a800319d0 con 0x7f0a981618f0 2024-01-15T18:10:32.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.454+0000 7f0a97400640 1 -- 172.21.15.26:0/1673650722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80031d60 con 0x7f0a981618f0 2024-01-15T18:10:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.455+0000 7f0a9ee00640 1 -- 172.21.15.26:0/1673650722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a981618f0 msgr2=0x7f0a98070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.455+0000 7f0a9ee00640 1 --2- 172.21.15.26:0/1673650722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a981618f0 0x7f0a98070af0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f0a980723f0 tx=0x7f0a8002b910 comp rx=0 tx=0).stop 2024-01-15T18:10:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.455+0000 7f0a9ee00640 1 -- 172.21.15.26:0/1673650722 shutdown_connections 2024-01-15T18:10:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.455+0000 7f0a9ee00640 1 --2- 172.21.15.26:0/1673650722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a981618f0 0x7f0a98070af0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.455+0000 7f0a9ee00640 1 --2- 172.21.15.26:0/1673650722 >> v2:172.21.15.124:3300/0 conn(0x7f0a98157c30 0x7f0a980cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.455+0000 7f0a9ee00640 1 -- 172.21.15.26:0/1673650722 >> 172.21.15.26:0/1673650722 conn(0x7f0a980cf700 msgr2=0x7f0a980cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.455+0000 7f0a9ee00640 1 -- 172.21.15.26:0/1673650722 shutdown_connections 2024-01-15T18:10:32.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.455+0000 7f0a9ee00640 1 -- 172.21.15.26:0/1673650722 wait complete. 2024-01-15T18:10:32.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.457+0000 7f0a9ee00640 1 Processor -- start 2024-01-15T18:10:32.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.457+0000 7f0a9ee00640 1 -- start start 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.458+0000 7f0a9ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98071030 0x7f0a98158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.458+0000 7f0a9ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98157c30 0x7f0a980cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.458+0000 7f0a9ee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a981618f0 0x7f0a98147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.458+0000 7f0a9ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a980b0b10 con 0x7f0a98157c30 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.458+0000 7f0a9ee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a981581c0 con 0x7f0a981618f0 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.458+0000 7f0a9ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a98158460 con 0x7f0a98071030 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.458+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98157c30 0x7f0a980cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.458+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98157c30 0x7f0a980cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54008/0 (socket says 172.21.15.26:54008) 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.458+0000 7f0a8fe00640 1 -- 172.21.15.26:0/2755553667 learned_addr learned my addr 172.21.15.26:0/2755553667 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:32.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.459+0000 7f0a97e00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98071030 0x7f0a98158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.459+0000 7f0a9ce00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a981618f0 0x7f0a98147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.459+0000 7f0a8fe00640 1 -- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98071030 msgr2=0x7f0a98158f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.459+0000 7f0a8fe00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98071030 0x7f0a98158f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.459+0000 7f0a8fe00640 1 -- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a981618f0 msgr2=0x7f0a98147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.459+0000 7f0a8fe00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a981618f0 0x7f0a98147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.459+0000 7f0a8fe00640 1 -- 172.21.15.26:0/2755553667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800094d0 con 0x7f0a98157c30 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.459+0000 7f0a9ce00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a981618f0 0x7f0a98147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.459+0000 7f0a8fe00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98157c30 0x7f0a980cd870 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c00 tx=0x7f0a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.460+0000 7f0a94c00640 1 -- 172.21.15.26:0/2755553667 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8003adf0 con 0x7f0a98157c30 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.460+0000 7f0a94c00640 1 -- 172.21.15.26:0/2755553667 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a800443f0 con 0x7f0a98157c30 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.460+0000 7f0a94c00640 1 -- 172.21.15.26:0/2755553667 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800447a0 con 0x7f0a98157c30 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.460+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a98158740 con 0x7f0a98157c30 2024-01-15T18:10:32.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.460+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a980cddb0 con 0x7f0a98157c30 2024-01-15T18:10:32.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.461+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a98158c60 con 0x7f0a98157c30 2024-01-15T18:10:32.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.462+0000 7f0a94c00640 1 -- 172.21.15.26:0/2755553667 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f0a8004d450 con 0x7f0a98157c30 2024-01-15T18:10:32.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.463+0000 7f0a94c00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c1f0 0x7f0a7006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.463+0000 7f0a94c00640 1 -- 172.21.15.26:0/2755553667 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0a8002d040 con 0x7f0a98157c30 2024-01-15T18:10:32.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.464+0000 7f0a97e00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c1f0 0x7f0a7006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.465+0000 7f0a97e00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c1f0 0x7f0a7006e6b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009890 tx=0x7f0a88008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:32.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.468+0000 7f0a94c00640 1 -- 172.21.15.26:0/2755553667 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a98158c60 con 0x7f0a98157c30 2024-01-15T18:10:32.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.659+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a9815e470 con 0x7f0a7006c1f0 2024-01-15T18:10:32.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.688+0000 7f0a94c00640 1 -- 172.21.15.26:0/2755553667 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286996 (secure 0 0 0) 0x7f0a88009480 con 0x7f0a7006c1f0 2024-01-15T18:10:32.695 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:32.697 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:10:32.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c1f0 msgr2=0x7f0a7006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:32.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c1f0 0x7f0a7006e6b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0a88009890 tx=0x7f0a88008040 comp rx=0 tx=0).stop 2024-01-15T18:10:32.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98157c30 msgr2=0x7f0a980cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:32.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98157c30 0x7f0a980cd870 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0a80000c00 tx=0x7f0a800046c0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 shutdown_connections 2024-01-15T18:10:32.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0a7006c1f0 0x7f0a7006e6b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0a981618f0 0x7f0a98147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0a98157c30 0x7f0a980cd870 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 --2- 172.21.15.26:0/2755553667 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0a98071030 0x7f0a98158f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 >> 172.21.15.26:0/2755553667 conn(0x7f0a980cf700 msgr2=0x7f0a9815fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:32.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 shutdown_connections 2024-01-15T18:10:32.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.694+0000 7f0a9ee00640 1 -- 172.21.15.26:0/2755553667 wait complete. 2024-01-15T18:10:32.715 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":563,"stamp":"2024-01-15T18:10:32.274355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":584924,"kb_used_data":11048,"kb_used_omap":12,"kb_used_meta":573747,"kb_avail":754389796,"statfs":{"total":773094113280,"available":772495151104,"internally_reserved":0,"allocated":11313152,"data_stored":5559244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":587517488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.681013"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544220+0000","last_change":"2024-01-15T18:10:22.628324+0000","last_active":"2024-01-15T18:10:24.544220+0000","last_peered":"2024-01-15T18:10:24.544220+0000","last_clean":"2024-01-15T18:10:24.544220+0000","last_became_active":"2024-01-15T18:10:22.628033+0000","last_became_peered":"2024-01-15T18:10:22.628033+0000","last_unstale":"2024-01-15T18:10:24.544220+0000","last_undegraded":"2024-01-15T18:10:24.544220+0000","last_fullsized":"2024-01-15T18:10:24.544220+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:54.391721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543739+0000","last_change":"2024-01-15T18:10:22.630135+0000","last_active":"2024-01-15T18:10:24.543739+0000","last_peered":"2024-01-15T18:10:24.543739+0000","last_clean":"2024-01-15T18:10:24.543739+0000","last_became_active":"2024-01-15T18:10:22.629836+0000","last_became_peered":"2024-01-15T18:10:22.629836+0000","last_unstale":"2024-01-15T18:10:24.543739+0000","last_undegraded":"2024-01-15T18:10:24.543739+0000","last_fullsized":"2024-01-15T18:10:24.543739+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:53.944791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544345+0000","last_change":"2024-01-15T18:10:22.622997+0000","last_active":"2024-01-15T18:10:24.544345+0000","last_peered":"2024-01-15T18:10:24.544345+0000","last_clean":"2024-01-15T18:10:24.544345+0000","last_became_active":"2024-01-15T18:10:22.622681+0000","last_became_peered":"2024-01-15T18:10:22.622681+0000","last_unstale":"2024-01-15T18:10:24.544345+0000","last_undegraded":"2024-01-15T18:10:24.544345+0000","last_fullsized":"2024-01-15T18:10:24.544345+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:08:18.277916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599751+0000","last_change":"2024-01-15T18:10:22.604572+0000","last_active":"2024-01-15T18:10:23.599751+0000","last_peered":"2024-01-15T18:10:23.599751+0000","last_clean":"2024-01-15T18:10:23.599751+0000","last_became_active":"2024-01-15T18:10:22.604033+0000","last_became_peered":"2024-01-15T18:10:22.604033+0000","last_unstale":"2024-01-15T18:10:23.599751+0000","last_undegraded":"2024-01-15T18:10:23.599751+0000","last_fullsized":"2024-01-15T18:10:23.599751+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:42:15.871289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541683+0000","last_change":"2024-01-15T18:10:22.626871+0000","last_active":"2024-01-15T18:10:24.541683+0000","last_peered":"2024-01-15T18:10:24.541683+0000","last_clean":"2024-01-15T18:10:24.541683+0000","last_became_active":"2024-01-15T18:10:22.626415+0000","last_became_peered":"2024-01-15T18:10:22.626415+0000","last_unstale":"2024-01-15T18:10:24.541683+0000","last_undegraded":"2024-01-15T18:10:24.541683+0000","last_fullsized":"2024-01-15T18:10:24.541683+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:59.565363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.605436+0000","last_change":"2024-01-15T18:10:22.614413+0000","last_active":"2024-01-15T18:10:23.605436+0000","last_peered":"2024-01-15T18:10:23.605436+0000","last_clean":"2024-01-15T18:10:23.605436+0000","last_became_active":"2024-01-15T18:10:22.613917+0000","last_became_peered":"2024-01-15T18:10:22.613917+0000","last_unstale":"2024-01-15T18:10:23.605436+0000","last_undegraded":"2024-01-15T18:10:23.605436+0000","last_fullsized":"2024-01-15T18:10:23.605436+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:17.084662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604969+0000","last_change":"2024-01-15T18:10:22.613303+0000","last_active":"2024-01-15T18:10:23.604969+0000","last_peered":"2024-01-15T18:10:23.604969+0000","last_clean":"2024-01-15T18:10:23.604969+0000","last_became_active":"2024-01-15T18:10:22.612865+0000","last_became_peered":"2024-01-15T18:10:22.612865+0000","last_unstale":"2024-01-15T18:10:23.604969+0000","last_undegraded":"2024-01-15T18:10:23.604969+0000","last_fullsized":"2024-01-15T18:10:23.604969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:00.289344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542095+0000","last_change":"2024-01-15T18:10:22.634156+0000","last_active":"2024-01-15T18:10:24.542095+0000","last_peered":"2024-01-15T18:10:24.542095+0000","last_clean":"2024-01-15T18:10:24.542095+0000","last_became_active":"2024-01-15T18:10:22.633797+0000","last_became_peered":"2024-01-15T18:10:22.633797+0000","last_unstale":"2024-01-15T18:10:24.542095+0000","last_undegraded":"2024-01-15T18:10:24.542095+0000","last_fullsized":"2024-01-15T18:10:24.542095+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:08:26.541990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604602+0000","last_change":"2024-01-15T18:10:22.613983+0000","last_active":"2024-01-15T18:10:23.604602+0000","last_peered":"2024-01-15T18:10:23.604602+0000","last_clean":"2024-01-15T18:10:23.604602+0000","last_became_active":"2024-01-15T18:10:22.613680+0000","last_became_peered":"2024-01-15T18:10:22.613680+0000","last_unstale":"2024-01-15T18:10:23.604602+0000","last_undegraded":"2024-01-15T18:10:23.604602+0000","last_fullsized":"2024-01-15T18:10:23.604602+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:17.994389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542036+0000","last_change":"2024-01-15T18:10:22.628655+0000","last_active":"2024-01-15T18:10:24.542036+0000","last_peered":"2024-01-15T18:10:24.542036+0000","last_clean":"2024-01-15T18:10:24.542036+0000","last_became_active":"2024-01-15T18:10:22.628353+0000","last_became_peered":"2024-01-15T18:10:22.628353+0000","last_unstale":"2024-01-15T18:10:24.542036+0000","last_undegraded":"2024-01-15T18:10:24.542036+0000","last_fullsized":"2024-01-15T18:10:24.542036+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.571225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.605677+0000","last_change":"2024-01-15T18:10:22.617671+0000","last_active":"2024-01-15T18:10:23.605677+0000","last_peered":"2024-01-15T18:10:23.605677+0000","last_clean":"2024-01-15T18:10:23.605677+0000","last_became_active":"2024-01-15T18:10:22.617378+0000","last_became_peered":"2024-01-15T18:10:22.617378+0000","last_unstale":"2024-01-15T18:10:23.605677+0000","last_undegraded":"2024-01-15T18:10:23.605677+0000","last_fullsized":"2024-01-15T18:10:23.605677+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:28.198670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.603327+0000","last_change":"2024-01-15T18:10:22.631118+0000","last_active":"2024-01-15T18:10:23.603327+0000","last_peered":"2024-01-15T18:10:23.603327+0000","last_clean":"2024-01-15T18:10:23.603327+0000","last_became_active":"2024-01-15T18:10:22.629350+0000","last_became_peered":"2024-01-15T18:10:22.629350+0000","last_unstale":"2024-01-15T18:10:23.603327+0000","last_undegraded":"2024-01-15T18:10:23.603327+0000","last_fullsized":"2024-01-15T18:10:23.603327+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:30.260896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599447+0000","last_change":"2024-01-15T18:10:22.605368+0000","last_active":"2024-01-15T18:10:23.599447+0000","last_peered":"2024-01-15T18:10:23.599447+0000","last_clean":"2024-01-15T18:10:23.599447+0000","last_became_active":"2024-01-15T18:10:22.605059+0000","last_became_peered":"2024-01-15T18:10:22.605059+0000","last_unstale":"2024-01-15T18:10:23.599447+0000","last_undegraded":"2024-01-15T18:10:23.599447+0000","last_fullsized":"2024-01-15T18:10:23.599447+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:51:00.427701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352852+0000","last_change":"2024-01-15T18:10:22.634829+0000","last_active":"2024-01-15T18:10:24.352852+0000","last_peered":"2024-01-15T18:10:24.352852+0000","last_clean":"2024-01-15T18:10:24.352852+0000","last_became_active":"2024-01-15T18:10:22.634489+0000","last_became_peered":"2024-01-15T18:10:22.634489+0000","last_unstale":"2024-01-15T18:10:24.352852+0000","last_undegraded":"2024-01-15T18:10:24.352852+0000","last_fullsized":"2024-01-15T18:10:24.352852+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:03:27.703474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.598920+0000","last_change":"2024-01-15T18:10:22.609316+0000","last_active":"2024-01-15T18:10:23.598920+0000","last_peered":"2024-01-15T18:10:23.598920+0000","last_clean":"2024-01-15T18:10:23.598920+0000","last_became_active":"2024-01-15T18:10:22.609027+0000","last_became_peered":"2024-01-15T18:10:22.609027+0000","last_unstale":"2024-01-15T18:10:23.598920+0000","last_undegraded":"2024-01-15T18:10:23.598920+0000","last_fullsized":"2024-01-15T18:10:23.598920+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:57:07.756348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.603818+0000","last_change":"2024-01-15T18:10:22.624718+0000","last_active":"2024-01-15T18:10:23.603818+0000","last_peered":"2024-01-15T18:10:23.603818+0000","last_clean":"2024-01-15T18:10:23.603818+0000","last_became_active":"2024-01-15T18:10:22.623285+0000","last_became_peered":"2024-01-15T18:10:22.623285+0000","last_unstale":"2024-01-15T18:10:23.603818+0000","last_undegraded":"2024-01-15T18:10:23.603818+0000","last_fullsized":"2024-01-15T18:10:23.603818+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:10:53.944791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543809+0000","last_change":"2024-01-15T18:10:22.626677+0000","last_active":"2024-01-15T18:10:24.543809+0000","last_peered":"2024-01-15T18:10:24.543809+0000","last_clean":"2024-01-15T18:10:24.543809+0000","last_became_active":"2024-01-15T18:10:22.626382+0000","last_became_peered":"2024-01-15T18:10:22.626382+0000","last_unstale":"2024-01-15T18:10:24.543809+0000","last_undegraded":"2024-01-15T18:10:24.543809+0000","last_fullsized":"2024-01-15T18:10:24.543809+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:30:21.693594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.600204+0000","last_change":"2024-01-15T18:10:22.604594+0000","last_active":"2024-01-15T18:10:23.600204+0000","last_peered":"2024-01-15T18:10:23.600204+0000","last_clean":"2024-01-15T18:10:23.600204+0000","last_became_active":"2024-01-15T18:10:22.604178+0000","last_became_peered":"2024-01-15T18:10:22.604178+0000","last_unstale":"2024-01-15T18:10:23.600204+0000","last_undegraded":"2024-01-15T18:10:23.600204+0000","last_fullsized":"2024-01-15T18:10:23.600204+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:54:56.471353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.354085+0000","last_change":"2024-01-15T18:10:22.612335+0000","last_active":"2024-01-15T18:10:24.354085+0000","last_peered":"2024-01-15T18:10:24.354085+0000","last_clean":"2024-01-15T18:10:24.354085+0000","last_became_active":"2024-01-15T18:10:22.612045+0000","last_became_peered":"2024-01-15T18:10:22.612045+0000","last_unstale":"2024-01-15T18:10:24.354085+0000","last_undegraded":"2024-01-15T18:10:24.354085+0000","last_fullsized":"2024-01-15T18:10:24.354085+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:18:45.137589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.603431+0000","last_change":"2024-01-15T18:10:22.607947+0000","last_active":"2024-01-15T18:10:23.603431+0000","last_peered":"2024-01-15T18:10:23.603431+0000","last_clean":"2024-01-15T18:10:23.603431+0000","last_became_active":"2024-01-15T18:10:22.607640+0000","last_became_peered":"2024-01-15T18:10:22.607640+0000","last_unstale":"2024-01-15T18:10:23.603431+0000","last_undegraded":"2024-01-15T18:10:23.603431+0000","last_fullsized":"2024-01-15T18:10:23.603431+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:59.565363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.353359+0000","last_change":"2024-01-15T18:10:22.618490+0000","last_active":"2024-01-15T18:10:24.353359+0000","last_peered":"2024-01-15T18:10:24.353359+0000","last_clean":"2024-01-15T18:10:24.353359+0000","last_became_active":"2024-01-15T18:10:22.618198+0000","last_became_peered":"2024-01-15T18:10:22.618198+0000","last_unstale":"2024-01-15T18:10:24.353359+0000","last_undegraded":"2024-01-15T18:10:24.353359+0000","last_fullsized":"2024-01-15T18:10:24.353359+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:52:54.391721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541948+0000","last_change":"2024-01-15T18:10:22.630121+0000","last_active":"2024-01-15T18:10:24.541948+0000","last_peered":"2024-01-15T18:10:24.541948+0000","last_clean":"2024-01-15T18:10:24.541948+0000","last_became_active":"2024-01-15T18:10:22.629804+0000","last_became_peered":"2024-01-15T18:10:22.629804+0000","last_unstale":"2024-01-15T18:10:24.541948+0000","last_undegraded":"2024-01-15T18:10:24.541948+0000","last_fullsized":"2024-01-15T18:10:24.541948+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:37.061370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543047+0000","last_change":"2024-01-15T18:10:22.630806+0000","last_active":"2024-01-15T18:10:24.543047+0000","last_peered":"2024-01-15T18:10:24.543047+0000","last_clean":"2024-01-15T18:10:24.543047+0000","last_became_active":"2024-01-15T18:10:22.630493+0000","last_became_peered":"2024-01-15T18:10:22.630493+0000","last_unstale":"2024-01-15T18:10:24.543047+0000","last_undegraded":"2024-01-15T18:10:24.543047+0000","last_fullsized":"2024-01-15T18:10:24.543047+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:50.984602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604235+0000","last_change":"2024-01-15T18:10:22.624955+0000","last_active":"2024-01-15T18:10:23.604235+0000","last_peered":"2024-01-15T18:10:23.604235+0000","last_clean":"2024-01-15T18:10:23.604235+0000","last_became_active":"2024-01-15T18:10:22.623819+0000","last_became_peered":"2024-01-15T18:10:22.623819+0000","last_unstale":"2024-01-15T18:10:23.604235+0000","last_undegraded":"2024-01-15T18:10:23.604235+0000","last_fullsized":"2024-01-15T18:10:23.604235+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:30:21.693594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542534+0000","last_change":"2024-01-15T18:10:22.639462+0000","last_active":"2024-01-15T18:10:24.542534+0000","last_peered":"2024-01-15T18:10:24.542534+0000","last_clean":"2024-01-15T18:10:24.542534+0000","last_became_active":"2024-01-15T18:10:22.638113+0000","last_became_peered":"2024-01-15T18:10:22.638113+0000","last_unstale":"2024-01-15T18:10:24.542534+0000","last_undegraded":"2024-01-15T18:10:24.542534+0000","last_fullsized":"2024-01-15T18:10:24.542534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:41:49.892254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544137+0000","last_change":"2024-01-15T18:10:22.626150+0000","last_active":"2024-01-15T18:10:24.544137+0000","last_peered":"2024-01-15T18:10:24.544137+0000","last_clean":"2024-01-15T18:10:24.544137+0000","last_became_active":"2024-01-15T18:10:22.625274+0000","last_became_peered":"2024-01-15T18:10:22.625274+0000","last_unstale":"2024-01-15T18:10:24.544137+0000","last_undegraded":"2024-01-15T18:10:24.544137+0000","last_fullsized":"2024-01-15T18:10:24.544137+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:01:00.967705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.605931+0000","last_change":"2024-01-15T18:10:22.622195+0000","last_active":"2024-01-15T18:10:23.605931+0000","last_peered":"2024-01-15T18:10:23.605931+0000","last_clean":"2024-01-15T18:10:23.605931+0000","last_became_active":"2024-01-15T18:10:22.621911+0000","last_became_peered":"2024-01-15T18:10:22.621911+0000","last_unstale":"2024-01-15T18:10:23.605931+0000","last_undegraded":"2024-01-15T18:10:23.605931+0000","last_fullsized":"2024-01-15T18:10:23.605931+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:26:56.885786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541875+0000","last_change":"2024-01-15T18:10:22.631688+0000","last_active":"2024-01-15T18:10:24.541875+0000","last_peered":"2024-01-15T18:10:24.541875+0000","last_clean":"2024-01-15T18:10:24.541875+0000","last_became_active":"2024-01-15T18:10:22.631361+0000","last_became_peered":"2024-01-15T18:10:22.631361+0000","last_unstale":"2024-01-15T18:10:24.541875+0000","last_undegraded":"2024-01-15T18:10:24.541875+0000","last_fullsized":"2024-01-15T18:10:24.541875+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:13:32.757599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542277+0000","last_change":"2024-01-15T18:10:22.626068+0000","last_active":"2024-01-15T18:10:24.542277+0000","last_peered":"2024-01-15T18:10:24.542277+0000","last_clean":"2024-01-15T18:10:24.542277+0000","last_became_active":"2024-01-15T18:10:22.625748+0000","last_became_peered":"2024-01-15T18:10:22.625748+0000","last_unstale":"2024-01-15T18:10:24.542277+0000","last_undegraded":"2024-01-15T18:10:24.542277+0000","last_fullsized":"2024-01-15T18:10:24.542277+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:41:49.892254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352997+0000","last_change":"2024-01-15T18:10:22.615020+0000","last_active":"2024-01-15T18:10:24.352997+0000","last_peered":"2024-01-15T18:10:24.352997+0000","last_clean":"2024-01-15T18:10:24.352997+0000","last_became_active":"2024-01-15T18:10:22.614651+0000","last_became_peered":"2024-01-15T18:10:22.614651+0000","last_unstale":"2024-01-15T18:10:24.352997+0000","last_undegraded":"2024-01-15T18:10:24.352997+0000","last_fullsized":"2024-01-15T18:10:24.352997+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:55.841561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599161+0000","last_change":"2024-01-15T18:10:22.631452+0000","last_active":"2024-01-15T18:10:23.599161+0000","last_peered":"2024-01-15T18:10:23.599161+0000","last_clean":"2024-01-15T18:10:23.599161+0000","last_became_active":"2024-01-15T18:10:22.631187+0000","last_became_peered":"2024-01-15T18:10:22.631187+0000","last_unstale":"2024-01-15T18:10:23.599161+0000","last_undegraded":"2024-01-15T18:10:23.599161+0000","last_fullsized":"2024-01-15T18:10:23.599161+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:03.152957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541942+0000","last_change":"2024-01-15T18:10:22.619198+0000","last_active":"2024-01-15T18:10:24.541942+0000","last_peered":"2024-01-15T18:10:24.541942+0000","last_clean":"2024-01-15T18:10:24.541942+0000","last_became_active":"2024-01-15T18:10:22.618925+0000","last_became_peered":"2024-01-15T18:10:22.618925+0000","last_unstale":"2024-01-15T18:10:24.541942+0000","last_undegraded":"2024-01-15T18:10:24.541942+0000","last_fullsized":"2024-01-15T18:10:24.541942+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:03:27.703474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.354248+0000","last_change":"2024-01-15T18:10:22.613941+0000","last_active":"2024-01-15T18:10:24.354248+0000","last_peered":"2024-01-15T18:10:24.354248+0000","last_clean":"2024-01-15T18:10:24.354248+0000","last_became_active":"2024-01-15T18:10:22.613478+0000","last_became_peered":"2024-01-15T18:10:22.613478+0000","last_unstale":"2024-01-15T18:10:24.354248+0000","last_undegraded":"2024-01-15T18:10:24.354248+0000","last_fullsized":"2024-01-15T18:10:24.354248+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:45.959711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.640892+0000","last_change":"2024-01-15T18:10:21.616691+0000","last_active":"2024-01-15T18:10:22.640892+0000","last_peered":"2024-01-15T18:10:22.640892+0000","last_clean":"2024-01-15T18:10:22.640892+0000","last_became_active":"2024-01-15T18:10:21.616428+0000","last_became_peered":"2024-01-15T18:10:21.616428+0000","last_unstale":"2024-01-15T18:10:22.640892+0000","last_undegraded":"2024-01-15T18:10:22.640892+0000","last_fullsized":"2024-01-15T18:10:22.640892+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:38.559745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.600420+0000","last_change":"2024-01-15T18:10:22.628019+0000","last_active":"2024-01-15T18:10:23.600420+0000","last_peered":"2024-01-15T18:10:23.600420+0000","last_clean":"2024-01-15T18:10:23.600420+0000","last_became_active":"2024-01-15T18:10:22.627022+0000","last_became_peered":"2024-01-15T18:10:22.627022+0000","last_unstale":"2024-01-15T18:10:23.600420+0000","last_undegraded":"2024-01-15T18:10:23.600420+0000","last_fullsized":"2024-01-15T18:10:23.600420+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:53:17.828426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.629274+0000","last_change":"2024-01-15T18:10:21.624181+0000","last_active":"2024-01-15T18:10:22.629274+0000","last_peered":"2024-01-15T18:10:22.629274+0000","last_clean":"2024-01-15T18:10:22.629274+0000","last_became_active":"2024-01-15T18:10:21.623845+0000","last_became_peered":"2024-01-15T18:10:21.623845+0000","last_unstale":"2024-01-15T18:10:22.629274+0000","last_undegraded":"2024-01-15T18:10:22.629274+0000","last_fullsized":"2024-01-15T18:10:22.629274+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:38.559745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543793+0000","last_change":"2024-01-15T18:10:22.639046+0000","last_active":"2024-01-15T18:10:24.543793+0000","last_peered":"2024-01-15T18:10:24.543793+0000","last_clean":"2024-01-15T18:10:24.543793+0000","last_became_active":"2024-01-15T18:10:22.638587+0000","last_became_peered":"2024-01-15T18:10:22.638587+0000","last_unstale":"2024-01-15T18:10:24.543793+0000","last_undegraded":"2024-01-15T18:10:24.543793+0000","last_fullsized":"2024-01-15T18:10:24.543793+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:55.841561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.637291+0000","last_change":"2024-01-15T18:10:21.621640+0000","last_active":"2024-01-15T18:10:22.637291+0000","last_peered":"2024-01-15T18:10:22.637291+0000","last_clean":"2024-01-15T18:10:22.637291+0000","last_became_active":"2024-01-15T18:10:21.619850+0000","last_became_peered":"2024-01-15T18:10:21.619850+0000","last_unstale":"2024-01-15T18:10:22.637291+0000","last_undegraded":"2024-01-15T18:10:22.637291+0000","last_fullsized":"2024-01-15T18:10:22.637291+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:58.553883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604306+0000","last_change":"2024-01-15T18:10:22.627539+0000","last_active":"2024-01-15T18:10:23.604306+0000","last_peered":"2024-01-15T18:10:23.604306+0000","last_clean":"2024-01-15T18:10:23.604306+0000","last_became_active":"2024-01-15T18:10:22.627192+0000","last_became_peered":"2024-01-15T18:10:22.627192+0000","last_unstale":"2024-01-15T18:10:23.604306+0000","last_undegraded":"2024-01-15T18:10:23.604306+0000","last_fullsized":"2024-01-15T18:10:23.604306+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:13:32.757599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.640292+0000","last_change":"2024-01-15T18:10:21.619185+0000","last_active":"2024-01-15T18:10:22.640292+0000","last_peered":"2024-01-15T18:10:22.640292+0000","last_clean":"2024-01-15T18:10:22.640292+0000","last_became_active":"2024-01-15T18:10:21.618436+0000","last_became_peered":"2024-01-15T18:10:21.618436+0000","last_unstale":"2024-01-15T18:10:22.640292+0000","last_undegraded":"2024-01-15T18:10:22.640292+0000","last_fullsized":"2024-01-15T18:10:22.640292+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:31:30.348979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.412317+0000","last_change":"2024-01-15T18:10:22.625949+0000","last_active":"2024-01-15T18:10:24.412317+0000","last_peered":"2024-01-15T18:10:24.412317+0000","last_clean":"2024-01-15T18:10:24.412317+0000","last_became_active":"2024-01-15T18:10:22.625639+0000","last_became_peered":"2024-01-15T18:10:22.625639+0000","last_unstale":"2024-01-15T18:10:24.412317+0000","last_undegraded":"2024-01-15T18:10:24.412317+0000","last_fullsized":"2024-01-15T18:10:24.412317+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:37:35.642706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.639492+0000","last_change":"2024-01-15T18:10:21.618207+0000","last_active":"2024-01-15T18:10:22.639492+0000","last_peered":"2024-01-15T18:10:22.639492+0000","last_clean":"2024-01-15T18:10:22.639492+0000","last_became_active":"2024-01-15T18:10:21.617907+0000","last_became_peered":"2024-01-15T18:10:21.617907+0000","last_unstale":"2024-01-15T18:10:22.639492+0000","last_undegraded":"2024-01-15T18:10:22.639492+0000","last_fullsized":"2024-01-15T18:10:22.639492+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:43:42.041130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604237+0000","last_change":"2024-01-15T18:10:22.629889+0000","last_active":"2024-01-15T18:10:23.604237+0000","last_peered":"2024-01-15T18:10:23.604237+0000","last_clean":"2024-01-15T18:10:23.604237+0000","last_became_active":"2024-01-15T18:10:22.629541+0000","last_became_peered":"2024-01-15T18:10:22.629541+0000","last_unstale":"2024-01-15T18:10:23.604237+0000","last_undegraded":"2024-01-15T18:10:23.604237+0000","last_fullsized":"2024-01-15T18:10:23.604237+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:01:00.967705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.635571+0000","last_change":"2024-01-15T18:10:21.616023+0000","last_active":"2024-01-15T18:10:22.635571+0000","last_peered":"2024-01-15T18:10:22.635571+0000","last_clean":"2024-01-15T18:10:22.635571+0000","last_became_active":"2024-01-15T18:10:21.615732+0000","last_became_peered":"2024-01-15T18:10:21.615732+0000","last_unstale":"2024-01-15T18:10:22.635571+0000","last_undegraded":"2024-01-15T18:10:22.635571+0000","last_fullsized":"2024-01-15T18:10:22.635571+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:25.499049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599063+0000","last_change":"2024-01-15T18:10:22.608070+0000","last_active":"2024-01-15T18:10:23.599063+0000","last_peered":"2024-01-15T18:10:23.599063+0000","last_clean":"2024-01-15T18:10:23.599063+0000","last_became_active":"2024-01-15T18:10:22.607671+0000","last_became_peered":"2024-01-15T18:10:22.607671+0000","last_unstale":"2024-01-15T18:10:23.599063+0000","last_undegraded":"2024-01-15T18:10:23.599063+0000","last_fullsized":"2024-01-15T18:10:23.599063+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:10:18.167742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.618549+0000","last_change":"2024-01-15T18:10:21.613918+0000","last_active":"2024-01-15T18:10:22.618549+0000","last_peered":"2024-01-15T18:10:22.618549+0000","last_clean":"2024-01-15T18:10:22.618549+0000","last_became_active":"2024-01-15T18:10:21.613184+0000","last_became_peered":"2024-01-15T18:10:21.613184+0000","last_unstale":"2024-01-15T18:10:22.618549+0000","last_undegraded":"2024-01-15T18:10:22.618549+0000","last_fullsized":"2024-01-15T18:10:22.618549+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:47:40.399123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.602321+0000","last_change":"2024-01-15T18:10:22.616092+0000","last_active":"2024-01-15T18:10:23.602321+0000","last_peered":"2024-01-15T18:10:23.602321+0000","last_clean":"2024-01-15T18:10:23.602321+0000","last_became_active":"2024-01-15T18:10:22.615769+0000","last_became_peered":"2024-01-15T18:10:22.615769+0000","last_unstale":"2024-01-15T18:10:23.602321+0000","last_undegraded":"2024-01-15T18:10:23.602321+0000","last_fullsized":"2024-01-15T18:10:23.602321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:26.510529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.605502+0000","last_change":"2024-01-15T18:10:21.608770+0000","last_active":"2024-01-15T18:10:22.605502+0000","last_peered":"2024-01-15T18:10:22.605502+0000","last_clean":"2024-01-15T18:10:22.605502+0000","last_became_active":"2024-01-15T18:10:21.607618+0000","last_became_peered":"2024-01-15T18:10:21.607618+0000","last_unstale":"2024-01-15T18:10:22.605502+0000","last_undegraded":"2024-01-15T18:10:22.605502+0000","last_fullsized":"2024-01-15T18:10:22.605502+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:12:24.068260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544399+0000","last_change":"2024-01-15T18:10:22.636462+0000","last_active":"2024-01-15T18:10:24.544399+0000","last_peered":"2024-01-15T18:10:24.544399+0000","last_clean":"2024-01-15T18:10:24.544399+0000","last_became_active":"2024-01-15T18:10:22.636149+0000","last_became_peered":"2024-01-15T18:10:22.636149+0000","last_unstale":"2024-01-15T18:10:24.544399+0000","last_undegraded":"2024-01-15T18:10:24.544399+0000","last_fullsized":"2024-01-15T18:10:24.544399+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:16:17.084662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.616212+0000","last_change":"2024-01-15T18:10:21.613722+0000","last_active":"2024-01-15T18:10:22.616212+0000","last_peered":"2024-01-15T18:10:22.616212+0000","last_clean":"2024-01-15T18:10:22.616212+0000","last_became_active":"2024-01-15T18:10:21.613117+0000","last_became_peered":"2024-01-15T18:10:21.613117+0000","last_unstale":"2024-01-15T18:10:22.616212+0000","last_undegraded":"2024-01-15T18:10:22.616212+0000","last_fullsized":"2024-01-15T18:10:22.616212+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:04.970896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.604568+0000","last_change":"2024-01-15T18:10:22.630879+0000","last_active":"2024-01-15T18:10:23.604568+0000","last_peered":"2024-01-15T18:10:23.604568+0000","last_clean":"2024-01-15T18:10:23.604568+0000","last_became_active":"2024-01-15T18:10:22.628981+0000","last_became_peered":"2024-01-15T18:10:22.628981+0000","last_unstale":"2024-01-15T18:10:23.604568+0000","last_undegraded":"2024-01-15T18:10:23.604568+0000","last_fullsized":"2024-01-15T18:10:23.604568+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:02:38.272375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.596370+0000","last_change":"2024-01-15T18:10:21.612961+0000","last_active":"2024-01-15T18:10:22.596370+0000","last_peered":"2024-01-15T18:10:22.596370+0000","last_clean":"2024-01-15T18:10:22.596370+0000","last_became_active":"2024-01-15T18:10:21.612624+0000","last_became_peered":"2024-01-15T18:10:21.612624+0000","last_unstale":"2024-01-15T18:10:22.596370+0000","last_undegraded":"2024-01-15T18:10:22.596370+0000","last_fullsized":"2024-01-15T18:10:22.596370+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:22:38.055323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542008+0000","last_change":"2024-01-15T18:10:22.637523+0000","last_active":"2024-01-15T18:10:24.542008+0000","last_peered":"2024-01-15T18:10:24.542008+0000","last_clean":"2024-01-15T18:10:24.542008+0000","last_became_active":"2024-01-15T18:10:22.637208+0000","last_became_peered":"2024-01-15T18:10:22.637208+0000","last_unstale":"2024-01-15T18:10:24.542008+0000","last_undegraded":"2024-01-15T18:10:24.542008+0000","last_fullsized":"2024-01-15T18:10:24.542008+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:17.994389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.631979+0000","last_change":"2024-01-15T18:10:21.623180+0000","last_active":"2024-01-15T18:10:22.631979+0000","last_peered":"2024-01-15T18:10:22.631979+0000","last_clean":"2024-01-15T18:10:22.631979+0000","last_became_active":"2024-01-15T18:10:21.622427+0000","last_became_peered":"2024-01-15T18:10:21.622427+0000","last_unstale":"2024-01-15T18:10:22.631979+0000","last_undegraded":"2024-01-15T18:10:22.631979+0000","last_fullsized":"2024-01-15T18:10:22.631979+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:36.049889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.605951+0000","last_change":"2024-01-15T18:10:22.636790+0000","last_active":"2024-01-15T18:10:23.605951+0000","last_peered":"2024-01-15T18:10:23.605951+0000","last_clean":"2024-01-15T18:10:23.605951+0000","last_became_active":"2024-01-15T18:10:22.636403+0000","last_became_peered":"2024-01-15T18:10:22.636403+0000","last_unstale":"2024-01-15T18:10:23.605951+0000","last_undegraded":"2024-01-15T18:10:23.605951+0000","last_fullsized":"2024-01-15T18:10:23.605951+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:15:07.478954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.641464+0000","last_change":"2024-01-15T18:10:21.617457+0000","last_active":"2024-01-15T18:10:22.641464+0000","last_peered":"2024-01-15T18:10:22.641464+0000","last_clean":"2024-01-15T18:10:22.641464+0000","last_became_active":"2024-01-15T18:10:21.617193+0000","last_became_peered":"2024-01-15T18:10:21.617193+0000","last_unstale":"2024-01-15T18:10:22.641464+0000","last_undegraded":"2024-01-15T18:10:22.641464+0000","last_fullsized":"2024-01-15T18:10:22.641464+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.176712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.599146+0000","last_change":"2024-01-15T18:10:22.611497+0000","last_active":"2024-01-15T18:10:23.599146+0000","last_peered":"2024-01-15T18:10:23.599146+0000","last_clean":"2024-01-15T18:10:23.599146+0000","last_became_active":"2024-01-15T18:10:22.610906+0000","last_became_peered":"2024-01-15T18:10:22.610906+0000","last_unstale":"2024-01-15T18:10:23.599146+0000","last_undegraded":"2024-01-15T18:10:23.599146+0000","last_fullsized":"2024-01-15T18:10:23.599146+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:53:07.920195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.638166+0000","last_change":"2024-01-15T18:10:21.620555+0000","last_active":"2024-01-15T18:10:22.638166+0000","last_peered":"2024-01-15T18:10:22.638166+0000","last_clean":"2024-01-15T18:10:22.638166+0000","last_became_active":"2024-01-15T18:10:21.620232+0000","last_became_peered":"2024-01-15T18:10:21.620232+0000","last_unstale":"2024-01-15T18:10:22.638166+0000","last_undegraded":"2024-01-15T18:10:22.638166+0000","last_fullsized":"2024-01-15T18:10:22.638166+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:44.150606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.543678+0000","last_change":"2024-01-15T18:10:22.617936+0000","last_active":"2024-01-15T18:10:24.543678+0000","last_peered":"2024-01-15T18:10:24.543678+0000","last_clean":"2024-01-15T18:10:24.543678+0000","last_became_active":"2024-01-15T18:10:22.613686+0000","last_became_peered":"2024-01-15T18:10:22.613686+0000","last_unstale":"2024-01-15T18:10:24.543678+0000","last_undegraded":"2024-01-15T18:10:24.543678+0000","last_fullsized":"2024-01-15T18:10:24.543678+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:19.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.599297+0000","last_change":"2024-01-15T18:10:21.618350+0000","last_active":"2024-01-15T18:10:22.599297+0000","last_peered":"2024-01-15T18:10:22.599297+0000","last_clean":"2024-01-15T18:10:22.599297+0000","last_became_active":"2024-01-15T18:10:21.618029+0000","last_became_peered":"2024-01-15T18:10:21.618029+0000","last_unstale":"2024-01-15T18:10:22.599297+0000","last_undegraded":"2024-01-15T18:10:22.599297+0000","last_fullsized":"2024-01-15T18:10:22.599297+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:43:42.041130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.601881+0000","last_change":"2024-01-15T18:10:22.627528+0000","last_active":"2024-01-15T18:10:23.601881+0000","last_peered":"2024-01-15T18:10:23.601881+0000","last_clean":"2024-01-15T18:10:23.601881+0000","last_became_active":"2024-01-15T18:10:22.627225+0000","last_became_peered":"2024-01-15T18:10:22.627225+0000","last_unstale":"2024-01-15T18:10:23.601881+0000","last_undegraded":"2024-01-15T18:10:23.601881+0000","last_fullsized":"2024-01-15T18:10:23.601881+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:15:11.824545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.599198+0000","last_change":"2024-01-15T18:10:21.622418+0000","last_active":"2024-01-15T18:10:22.599198+0000","last_peered":"2024-01-15T18:10:22.599198+0000","last_clean":"2024-01-15T18:10:22.599198+0000","last_became_active":"2024-01-15T18:10:21.621566+0000","last_became_peered":"2024-01-15T18:10:21.621566+0000","last_unstale":"2024-01-15T18:10:22.599198+0000","last_undegraded":"2024-01-15T18:10:22.599198+0000","last_fullsized":"2024-01-15T18:10:22.599198+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:25.792080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352843+0000","last_change":"2024-01-15T18:10:22.611042+0000","last_active":"2024-01-15T18:10:24.352843+0000","last_peered":"2024-01-15T18:10:24.352843+0000","last_clean":"2024-01-15T18:10:24.352843+0000","last_became_active":"2024-01-15T18:10:22.610178+0000","last_became_peered":"2024-01-15T18:10:22.610178+0000","last_unstale":"2024-01-15T18:10:24.352843+0000","last_undegraded":"2024-01-15T18:10:24.352843+0000","last_fullsized":"2024-01-15T18:10:24.352843+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:19.613931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":20,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.639911+0000","last_change":"2024-01-15T18:10:21.613977+0000","last_active":"2024-01-15T18:10:22.639911+0000","last_peered":"2024-01-15T18:10:22.639911+0000","last_clean":"2024-01-15T18:10:22.639911+0000","last_became_active":"2024-01-15T18:10:21.613679+0000","last_became_peered":"2024-01-15T18:10:21.613679+0000","last_unstale":"2024-01-15T18:10:22.639911+0000","last_undegraded":"2024-01-15T18:10:22.639911+0000","last_fullsized":"2024-01-15T18:10:22.639911+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:35:35.695097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.542506+0000","last_change":"2024-01-15T18:10:22.633721+0000","last_active":"2024-01-15T18:10:24.542506+0000","last_peered":"2024-01-15T18:10:24.542506+0000","last_clean":"2024-01-15T18:10:24.542506+0000","last_became_active":"2024-01-15T18:10:22.633243+0000","last_became_peered":"2024-01-15T18:10:22.633243+0000","last_unstale":"2024-01-15T18:10:24.542506+0000","last_undegraded":"2024-01-15T18:10:24.542506+0000","last_fullsized":"2024-01-15T18:10:24.542506+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:08:26.541990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.629411+0000","last_change":"2024-01-15T18:10:21.618794+0000","last_active":"2024-01-15T18:10:22.629411+0000","last_peered":"2024-01-15T18:10:22.629411+0000","last_clean":"2024-01-15T18:10:22.629411+0000","last_became_active":"2024-01-15T18:10:21.618479+0000","last_became_peered":"2024-01-15T18:10:21.618479+0000","last_unstale":"2024-01-15T18:10:22.629411+0000","last_undegraded":"2024-01-15T18:10:22.629411+0000","last_fullsized":"2024-01-15T18:10:22.629411+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:31:30.348979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.412253+0000","last_change":"2024-01-15T18:10:22.636257+0000","last_active":"2024-01-15T18:10:24.412253+0000","last_peered":"2024-01-15T18:10:24.412253+0000","last_clean":"2024-01-15T18:10:24.412253+0000","last_became_active":"2024-01-15T18:10:22.635916+0000","last_became_peered":"2024-01-15T18:10:22.635916+0000","last_unstale":"2024-01-15T18:10:24.412253+0000","last_undegraded":"2024-01-15T18:10:24.412253+0000","last_fullsized":"2024-01-15T18:10:24.412253+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:07:26.803560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.639108+0000","last_change":"2024-01-15T18:10:21.608435+0000","last_active":"2024-01-15T18:10:22.639108+0000","last_peered":"2024-01-15T18:10:22.639108+0000","last_clean":"2024-01-15T18:10:22.639108+0000","last_became_active":"2024-01-15T18:10:21.608053+0000","last_became_peered":"2024-01-15T18:10:21.608053+0000","last_unstale":"2024-01-15T18:10:22.639108+0000","last_undegraded":"2024-01-15T18:10:22.639108+0000","last_fullsized":"2024-01-15T18:10:22.639108+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:04.043402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.606183+0000","last_change":"2024-01-15T18:10:22.634622+0000","last_active":"2024-01-15T18:10:23.606183+0000","last_peered":"2024-01-15T18:10:23.606183+0000","last_clean":"2024-01-15T18:10:23.606183+0000","last_became_active":"2024-01-15T18:10:22.634336+0000","last_became_peered":"2024-01-15T18:10:22.634336+0000","last_unstale":"2024-01-15T18:10:23.606183+0000","last_undegraded":"2024-01-15T18:10:23.606183+0000","last_fullsized":"2024-01-15T18:10:23.606183+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:39:01.560791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.617549+0000","last_change":"2024-01-15T18:10:21.620384+0000","last_active":"2024-01-15T18:10:22.617549+0000","last_peered":"2024-01-15T18:10:22.617549+0000","last_clean":"2024-01-15T18:10:22.617549+0000","last_became_active":"2024-01-15T18:10:21.617541+0000","last_became_peered":"2024-01-15T18:10:21.617541+0000","last_unstale":"2024-01-15T18:10:22.617549+0000","last_undegraded":"2024-01-15T18:10:22.617549+0000","last_fullsized":"2024-01-15T18:10:22.617549+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:04.043402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.353124+0000","last_change":"2024-01-15T18:10:22.610157+0000","last_active":"2024-01-15T18:10:24.353124+0000","last_peered":"2024-01-15T18:10:24.353124+0000","last_clean":"2024-01-15T18:10:24.353124+0000","last_became_active":"2024-01-15T18:10:22.609810+0000","last_became_peered":"2024-01-15T18:10:22.609810+0000","last_unstale":"2024-01-15T18:10:24.353124+0000","last_undegraded":"2024-01-15T18:10:24.353124+0000","last_fullsized":"2024-01-15T18:10:24.353124+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:31.941151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.634881+0000","last_change":"2024-01-15T18:10:21.620621+0000","last_active":"2024-01-15T18:10:22.634881+0000","last_peered":"2024-01-15T18:10:22.634881+0000","last_clean":"2024-01-15T18:10:22.634881+0000","last_became_active":"2024-01-15T18:10:21.620314+0000","last_became_peered":"2024-01-15T18:10:21.620314+0000","last_unstale":"2024-01-15T18:10:22.634881+0000","last_undegraded":"2024-01-15T18:10:22.634881+0000","last_fullsized":"2024-01-15T18:10:22.634881+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:29.249416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.353673+0000","last_change":"2024-01-15T18:10:22.619733+0000","last_active":"2024-01-15T18:10:24.353673+0000","last_peered":"2024-01-15T18:10:24.353673+0000","last_clean":"2024-01-15T18:10:24.353673+0000","last_became_active":"2024-01-15T18:10:22.619428+0000","last_became_peered":"2024-01-15T18:10:22.619428+0000","last_unstale":"2024-01-15T18:10:24.353673+0000","last_undegraded":"2024-01-15T18:10:24.353673+0000","last_fullsized":"2024-01-15T18:10:24.353673+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:01:57.679690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.621610+0000","last_change":"2024-01-15T18:10:21.614065+0000","last_active":"2024-01-15T18:10:22.621610+0000","last_peered":"2024-01-15T18:10:22.621610+0000","last_clean":"2024-01-15T18:10:22.621610+0000","last_became_active":"2024-01-15T18:10:21.613749+0000","last_became_peered":"2024-01-15T18:10:21.613749+0000","last_unstale":"2024-01-15T18:10:22.621610+0000","last_undegraded":"2024-01-15T18:10:22.621610+0000","last_fullsized":"2024-01-15T18:10:22.621610+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:32:47.791824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352680+0000","last_change":"2024-01-15T18:10:22.611155+0000","last_active":"2024-01-15T18:10:24.352680+0000","last_peered":"2024-01-15T18:10:24.352680+0000","last_clean":"2024-01-15T18:10:24.352680+0000","last_became_active":"2024-01-15T18:10:22.610491+0000","last_became_peered":"2024-01-15T18:10:22.610491+0000","last_unstale":"2024-01-15T18:10:24.352680+0000","last_undegraded":"2024-01-15T18:10:24.352680+0000","last_fullsized":"2024-01-15T18:10:24.352680+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:05.982376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.623192+0000","last_change":"2024-01-15T18:10:21.608384+0000","last_active":"2024-01-15T18:10:22.623192+0000","last_peered":"2024-01-15T18:10:22.623192+0000","last_clean":"2024-01-15T18:10:22.623192+0000","last_became_active":"2024-01-15T18:10:21.607947+0000","last_became_peered":"2024-01-15T18:10:21.607947+0000","last_unstale":"2024-01-15T18:10:22.623192+0000","last_undegraded":"2024-01-15T18:10:22.623192+0000","last_fullsized":"2024-01-15T18:10:22.623192+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:21:00.361963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541652+0000","last_change":"2024-01-15T18:10:22.625644+0000","last_active":"2024-01-15T18:10:24.541652+0000","last_peered":"2024-01-15T18:10:24.541652+0000","last_clean":"2024-01-15T18:10:24.541652+0000","last_became_active":"2024-01-15T18:10:22.625365+0000","last_became_peered":"2024-01-15T18:10:22.625365+0000","last_unstale":"2024-01-15T18:10:24.541652+0000","last_undegraded":"2024-01-15T18:10:24.541652+0000","last_fullsized":"2024-01-15T18:10:24.541652+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:50:30.260896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.609030+0000","last_change":"2024-01-15T18:10:21.620127+0000","last_active":"2024-01-15T18:10:22.609030+0000","last_peered":"2024-01-15T18:10:22.609030+0000","last_clean":"2024-01-15T18:10:22.609030+0000","last_became_active":"2024-01-15T18:10:21.619856+0000","last_became_peered":"2024-01-15T18:10:21.619856+0000","last_unstale":"2024-01-15T18:10:22.609030+0000","last_undegraded":"2024-01-15T18:10:22.609030+0000","last_fullsized":"2024-01-15T18:10:22.609030+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:56:04.291772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544396+0000","last_change":"2024-01-15T18:10:22.622337+0000","last_active":"2024-01-15T18:10:24.544396+0000","last_peered":"2024-01-15T18:10:24.544396+0000","last_clean":"2024-01-15T18:10:24.544396+0000","last_became_active":"2024-01-15T18:10:22.622029+0000","last_became_peered":"2024-01-15T18:10:22.622029+0000","last_unstale":"2024-01-15T18:10:24.544396+0000","last_undegraded":"2024-01-15T18:10:24.544396+0000","last_fullsized":"2024-01-15T18:10:24.544396+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:08:00.289344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":14,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.638565+0000","last_change":"2024-01-15T18:10:21.619938+0000","last_active":"2024-01-15T18:10:22.638565+0000","last_peered":"2024-01-15T18:10:22.638565+0000","last_clean":"2024-01-15T18:10:22.638565+0000","last_became_active":"2024-01-15T18:10:21.618751+0000","last_became_peered":"2024-01-15T18:10:21.618751+0000","last_unstale":"2024-01-15T18:10:22.638565+0000","last_undegraded":"2024-01-15T18:10:22.638565+0000","last_fullsized":"2024-01-15T18:10:22.638565+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:17:27.750448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544586+0000","last_change":"2024-01-15T18:10:22.624519+0000","last_active":"2024-01-15T18:10:24.544586+0000","last_peered":"2024-01-15T18:10:24.544586+0000","last_clean":"2024-01-15T18:10:24.544586+0000","last_became_active":"2024-01-15T18:10:22.624227+0000","last_became_peered":"2024-01-15T18:10:22.624227+0000","last_unstale":"2024-01-15T18:10:24.544586+0000","last_undegraded":"2024-01-15T18:10:24.544586+0000","last_fullsized":"2024-01-15T18:10:24.544586+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:01:57.679690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"102'1","reported_seq":13,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544659+0000","last_change":"2024-01-15T18:10:21.618976+0000","last_active":"2024-01-15T18:10:24.544659+0000","last_peered":"2024-01-15T18:10:24.544659+0000","last_clean":"2024-01-15T18:10:24.544659+0000","last_became_active":"2024-01-15T18:10:21.618682+0000","last_became_peered":"2024-01-15T18:10:21.618682+0000","last_unstale":"2024-01-15T18:10:24.544659+0000","last_undegraded":"2024-01-15T18:10:24.544659+0000","last_fullsized":"2024-01-15T18:10:24.544659+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:47:16.982908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.544175+0000","last_change":"2024-01-15T18:10:22.633839+0000","last_active":"2024-01-15T18:10:24.544175+0000","last_peered":"2024-01-15T18:10:24.544175+0000","last_clean":"2024-01-15T18:10:24.544175+0000","last_became_active":"2024-01-15T18:10:22.633500+0000","last_became_peered":"2024-01-15T18:10:22.633500+0000","last_unstale":"2024-01-15T18:10:24.544175+0000","last_undegraded":"2024-01-15T18:10:24.544175+0000","last_fullsized":"2024-01-15T18:10:24.544175+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:31.941151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.618013+0000","last_change":"2024-01-15T18:10:21.623001+0000","last_active":"2024-01-15T18:10:22.618013+0000","last_peered":"2024-01-15T18:10:22.618013+0000","last_clean":"2024-01-15T18:10:22.618013+0000","last_became_active":"2024-01-15T18:10:21.622691+0000","last_became_peered":"2024-01-15T18:10:21.622691+0000","last_unstale":"2024-01-15T18:10:22.618013+0000","last_undegraded":"2024-01-15T18:10:22.618013+0000","last_fullsized":"2024-01-15T18:10:22.618013+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:13:31.746119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541936+0000","last_change":"2024-01-15T18:10:22.625848+0000","last_active":"2024-01-15T18:10:24.541936+0000","last_peered":"2024-01-15T18:10:24.541936+0000","last_clean":"2024-01-15T18:10:24.541936+0000","last_became_active":"2024-01-15T18:10:22.625079+0000","last_became_peered":"2024-01-15T18:10:22.625079+0000","last_unstale":"2024-01-15T18:10:24.541936+0000","last_undegraded":"2024-01-15T18:10:24.541936+0000","last_fullsized":"2024-01-15T18:10:24.541936+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:02:38.272375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.621043+0000","last_change":"2024-01-15T18:10:21.618463+0000","last_active":"2024-01-15T18:10:22.621043+0000","last_peered":"2024-01-15T18:10:22.621043+0000","last_clean":"2024-01-15T18:10:22.621043+0000","last_became_active":"2024-01-15T18:10:21.618150+0000","last_became_peered":"2024-01-15T18:10:21.618150+0000","last_unstale":"2024-01-15T18:10:22.621043+0000","last_undegraded":"2024-01-15T18:10:22.621043+0000","last_fullsized":"2024-01-15T18:10:22.621043+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:25.499049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.353411+0000","last_change":"2024-01-15T18:10:22.610959+0000","last_active":"2024-01-15T18:10:24.353411+0000","last_peered":"2024-01-15T18:10:24.353411+0000","last_clean":"2024-01-15T18:10:24.353411+0000","last_became_active":"2024-01-15T18:10:22.609991+0000","last_became_peered":"2024-01-15T18:10:22.609991+0000","last_unstale":"2024-01-15T18:10:24.353411+0000","last_undegraded":"2024-01-15T18:10:24.353411+0000","last_fullsized":"2024-01-15T18:10:24.353411+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:08:18.277916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.618356+0000","last_change":"2024-01-15T18:10:21.613099+0000","last_active":"2024-01-15T18:10:22.618356+0000","last_peered":"2024-01-15T18:10:22.618356+0000","last_clean":"2024-01-15T18:10:22.618356+0000","last_became_active":"2024-01-15T18:10:21.612818+0000","last_became_peered":"2024-01-15T18:10:21.612818+0000","last_unstale":"2024-01-15T18:10:22.618356+0000","last_undegraded":"2024-01-15T18:10:22.618356+0000","last_fullsized":"2024-01-15T18:10:22.618356+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:02:37.260895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:23.602721+0000","last_change":"2024-01-15T18:10:22.633062+0000","last_active":"2024-01-15T18:10:23.602721+0000","last_peered":"2024-01-15T18:10:23.602721+0000","last_clean":"2024-01-15T18:10:23.602721+0000","last_became_active":"2024-01-15T18:10:22.632748+0000","last_became_peered":"2024-01-15T18:10:22.632748+0000","last_unstale":"2024-01-15T18:10:23.602721+0000","last_undegraded":"2024-01-15T18:10:23.602721+0000","last_fullsized":"2024-01-15T18:10:23.602721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:50:05.054882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.618599+0000","last_change":"2024-01-15T18:10:21.610659+0000","last_active":"2024-01-15T18:10:22.618599+0000","last_peered":"2024-01-15T18:10:22.618599+0000","last_clean":"2024-01-15T18:10:22.618599+0000","last_became_active":"2024-01-15T18:10:21.610364+0000","last_became_peered":"2024-01-15T18:10:21.610364+0000","last_unstale":"2024-01-15T18:10:22.618599+0000","last_undegraded":"2024-01-15T18:10:22.618599+0000","last_fullsized":"2024-01-15T18:10:22.618599+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:58:22.176712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.352674+0000","last_change":"2024-01-15T18:10:22.617540+0000","last_active":"2024-01-15T18:10:24.352674+0000","last_peered":"2024-01-15T18:10:24.352674+0000","last_clean":"2024-01-15T18:10:24.352674+0000","last_became_active":"2024-01-15T18:10:22.617134+0000","last_became_peered":"2024-01-15T18:10:22.617134+0000","last_unstale":"2024-01-15T18:10:24.352674+0000","last_undegraded":"2024-01-15T18:10:24.352674+0000","last_fullsized":"2024-01-15T18:10:24.352674+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:20:22.417375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.597058+0000","last_change":"2024-01-15T18:10:21.613230+0000","last_active":"2024-01-15T18:10:22.597058+0000","last_peered":"2024-01-15T18:10:22.597058+0000","last_clean":"2024-01-15T18:10:22.597058+0000","last_became_active":"2024-01-15T18:10:21.612943+0000","last_became_peered":"2024-01-15T18:10:21.612943+0000","last_unstale":"2024-01-15T18:10:22.597058+0000","last_undegraded":"2024-01-15T18:10:22.597058+0000","last_fullsized":"2024-01-15T18:10:22.597058+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:32.108693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.541750+0000","last_change":"2024-01-15T18:10:22.628468+0000","last_active":"2024-01-15T18:10:24.541750+0000","last_peered":"2024-01-15T18:10:24.541750+0000","last_clean":"2024-01-15T18:10:24.541750+0000","last_became_active":"2024-01-15T18:10:22.627865+0000","last_became_peered":"2024-01-15T18:10:22.627865+0000","last_unstale":"2024-01-15T18:10:24.541750+0000","last_undegraded":"2024-01-15T18:10:24.541750+0000","last_fullsized":"2024-01-15T18:10:24.541750+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:46:37.061370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.597495+0000","last_change":"2024-01-15T18:10:21.621660+0000","last_active":"2024-01-15T18:10:22.597495+0000","last_peered":"2024-01-15T18:10:22.597495+0000","last_clean":"2024-01-15T18:10:22.597495+0000","last_became_active":"2024-01-15T18:10:21.620662+0000","last_became_peered":"2024-01-15T18:10:21.620662+0000","last_unstale":"2024-01-15T18:10:22.597495+0000","last_undegraded":"2024-01-15T18:10:22.597495+0000","last_fullsized":"2024-01-15T18:10:22.597495+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:10.136007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-01-15T18:10:24.354289+0000","last_change":"2024-01-15T18:10:22.613744+0000","last_active":"2024-01-15T18:10:24.354289+0000","last_peered":"2024-01-15T18:10:24.354289+0000","last_clean":"2024-01-15T18:10:24.354289+0000","last_became_active":"2024-01-15T18:10:22.609665+0000","last_became_peered":"2024-01-15T18:10:22.609665+0000","last_unstale":"2024-01-15T18:10:24.354289+0000","last_undegraded":"2024-01-15T18:10:24.354289+0000","last_fullsized":"2024-01-15T18:10:24.354289+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:21.572319+0000","last_clean_scrub_stamp":"2024-01-15T18:10:21.572319+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:54:11.522155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.628995+0000","last_change":"2024-01-15T18:10:21.624625+0000","last_active":"2024-01-15T18:10:22.628995+0000","last_peered":"2024-01-15T18:10:22.628995+0000","last_clean":"2024-01-15T18:10:22.628995+0000","last_became_active":"2024-01-15T18:10:21.624271+0000","last_became_peered":"2024-01-15T18:10:21.624271+0000","last_unstale":"2024-01-15T18:10:22.628995+0000","last_undegraded":"2024-01-15T18:10:22.628995+0000","last_fullsized":"2024-01-15T18:10:22.628995+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:10:20.560838+0000","last_clean_scrub_stamp":"2024-01-15T18:10:20.560838+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:41:48.880774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":235,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-01-15T18:10:22.606602+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:10:22.606602+0000","last_peered":"2024-01-15T18:10:22.606602+0000","last_clean":"2024-01-15T18:10:22.606602+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:10:22.606602+0000","last_undegraded":"2024-01-15T18:10:22.606602+0000","last_fullsized":"2024-01-15T18:10:22.606602+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542333,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85736,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":83902,"kb_avail":94286104,"statfs":{"total":96636764160,"available":96548970496,"internally_reserved":0,"allocated":1859584,"data_stored":1137191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":85916106},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3280000000000001,"15min":1.3740000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.40400000000000003},"max":{"1min":1.742,"5min":11.449999999999999,"15min":19.843},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.3759999999999999,"15min":1.629},"min":{"1min":0.89200000000000002,"5min":0.61499999999999999,"15min":0.53400000000000003},"max":{"1min":7.8250000000000002,"5min":7.8250000000000002,"15min":38.137},"last":1.0329999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.319,"15min":1.411},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.45600000000000002},"max":{"1min":1.577,"5min":11.51,"15min":19.917000000000002},"last":1.613},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.4079999999999999,"15min":1.605},"min":{"1min":0.85699999999999998,"5min":0.81100000000000005,"15min":0.42999999999999999},"max":{"1min":1.841,"5min":6.7480000000000002,"15min":51.843000000000004},"last":1.3779999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.371,"15min":1.425},"min":{"1min":0.97199999999999998,"5min":0.67200000000000004,"15min":0.437},"max":{"1min":3.3900000000000001,"5min":12.061,"15min":12.061},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4430000000000001,"15min":1.532},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.47399999999999998},"max":{"1min":1.806,"5min":11.66,"15min":13.478999999999999},"last":1.8180000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.393,"15min":1.544},"min":{"1min":0.85299999999999998,"5min":0.83199999999999996,"15min":0.47899999999999998},"max":{"1min":1.9079999999999999,"5min":7.4669999999999996,"15min":16.632000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.4370000000000001,"5min":1.5069999999999999,"15min":1.6160000000000001},"min":{"1min":0.97299999999999998,"5min":0.94199999999999995,"15min":0.56699999999999995},"max":{"1min":1.8520000000000001,"5min":11.611000000000001,"15min":13.439},"last":1.919}]},{"osd":4,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":2.319,"5min":1.728,"15min":1.7589999999999999},"min":{"1min":1.0169999999999999,"5min":0.81000000000000005,"15min":0.50700000000000001},"max":{"1min":17.067,"5min":23.042000000000002,"15min":23.042000000000002},"last":1.456},{"interface":"front","average":{"1min":1.425,"5min":1.5269999999999999,"15min":1.7270000000000001},"min":{"1min":0.94599999999999995,"5min":0.78800000000000003,"15min":0.51800000000000002},"max":{"1min":2.0779999999999998,"5min":16.867999999999999,"15min":38.682000000000002},"last":1.4119999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.5640000000000001,"15min":1.8089999999999999},"min":{"1min":1.0329999999999999,"5min":0.89800000000000002,"15min":0.52700000000000002},"max":{"1min":1.843,"5min":11.696999999999999,"15min":25.006},"last":1.97},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.7110000000000001,"15min":1.8400000000000001},"min":{"1min":1.0880000000000001,"5min":0.97499999999999998,"15min":0.624},"max":{"1min":2.04,"5min":22.933,"15min":22.933},"last":1.774}]},{"osd":6,"last update":"Mon Jan 15 18:09:27 2024","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.9850000000000001,"15min":2.0419999999999998},"min":{"1min":1.1870000000000001,"5min":1.123,"15min":0.59299999999999997},"max":{"1min":2.2229999999999999,"5min":23.004000000000001,"15min":23.834},"last":1.861},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.607,"15min":1.998},"min":{"1min":1.0680000000000001,"5min":0.84099999999999997,"15min":0.58299999999999996},"max":{"1min":2.1539999999999999,"5min":7.5220000000000002,"15min":63.932000000000002},"last":1.7230000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542332,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80936,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":79678,"kb_avail":94290904,"statfs":{"total":96636764160,"available":96553885696,"internally_reserved":0,"allocated":1265664,"data_stored":547363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81590726},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":2.597,"5min":1.746,"15min":1.502},"min":{"1min":0.68700000000000006,"5min":0.55900000000000005,"15min":0.53100000000000003},"max":{"1min":31.805,"5min":31.805,"15min":31.805},"last":1.125},{"interface":"front","average":{"1min":2.7010000000000001,"5min":1.591,"15min":1.51},"min":{"1min":0.73199999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":31.702999999999999,"5min":31.702999999999999,"15min":31.702999999999999},"last":1.1950000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.357,"15min":1.3480000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.55100000000000005},"max":{"1min":1.6719999999999999,"5min":15.064,"15min":15.064},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.5229999999999999,"15min":1.54},"min":{"1min":0.86499999999999999,"5min":0.74199999999999999,"15min":0.66200000000000003},"max":{"1min":4.0579999999999998,"5min":23.495999999999999,"15min":23.495999999999999},"last":1.492}]},{"osd":2,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.629,"15min":1.5980000000000001},"min":{"1min":0.89400000000000002,"5min":0.73799999999999999,"15min":0.59399999999999997},"max":{"1min":3.9849999999999999,"5min":23.087,"15min":23.087},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.4950000000000001,"15min":1.4430000000000001},"min":{"1min":0.67900000000000005,"5min":0.51400000000000001,"15min":0.504},"max":{"1min":2.048,"5min":14.94,"15min":14.94},"last":0.82899999999999996}]},{"osd":3,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.611,"15min":1.6890000000000001},"min":{"1min":0.71799999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.1269999999999998,"5min":15.151999999999999,"15min":15.151999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4319999999999999,"15min":1.6040000000000001},"min":{"1min":0.874,"5min":0.76100000000000001,"15min":0.70699999999999996},"max":{"1min":2.431,"5min":4.6849999999999996,"15min":14.066000000000001},"last":1.2989999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.6060000000000001,"15min":1.663},"min":{"1min":0.92000000000000004,"5min":0.56100000000000005,"15min":0.51800000000000002},"max":{"1min":3.9169999999999998,"5min":23.327000000000002,"15min":23.327000000000002},"last":1.345},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.74,"15min":1.704},"min":{"1min":0.76000000000000001,"5min":0.61899999999999999,"15min":0.45800000000000002},"max":{"1min":2.9929999999999999,"5min":15.239000000000001,"15min":17.289000000000001},"last":1.2609999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.413,"15min":1.5629999999999999},"min":{"1min":0.81499999999999995,"5min":0.52500000000000002,"15min":0.51600000000000001},"max":{"1min":2.6949999999999998,"5min":4.9260000000000002,"15min":14.176},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.7010000000000001,"5min":1.8879999999999999,"15min":1.8660000000000001},"min":{"1min":1.335,"5min":0.64400000000000002,"15min":0.54700000000000004},"max":{"1min":4.1790000000000003,"5min":23.417999999999999,"15min":23.417999999999999},"last":1.6160000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:26 2024","interfaces":[{"interface":"back","average":{"1min":1.5529999999999999,"5min":1.5629999999999999,"15min":1.7669999999999999},"min":{"1min":0.84999999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.5880000000000001,"5min":4.8239999999999998,"15min":21.265000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.6890000000000001,"5min":2.0289999999999999,"15min":1.99},"min":{"1min":1.163,"5min":0.77100000000000002,"15min":0.59699999999999998},"max":{"1min":4.0220000000000002,"5min":23.738,"15min":23.738},"last":1.5640000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58780,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313060,"statfs":{"total":96636764160,"available":96576573440,"internally_reserved":0,"allocated":1261568,"data_stored":546781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.1870000000000001,"15min":1.282},"min":{"1min":0.72199999999999998,"5min":0.69599999999999995,"15min":0.47799999999999998},"max":{"1min":1.649,"5min":2.2839999999999998,"15min":19.266999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1870000000000001,"15min":1.337},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.51400000000000001},"max":{"1min":1.6930000000000001,"5min":2.1869999999999998,"15min":17.184999999999999},"last":1.3839999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.234,"15min":1.3380000000000001},"min":{"1min":0.91300000000000003,"5min":0.69699999999999995,"15min":0.499},"max":{"1min":1.472,"5min":5.3879999999999999,"15min":16.297000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.3,"5min":1.29,"15min":1.335},"min":{"1min":0.84399999999999997,"5min":0.78900000000000003,"15min":0.59099999999999997},"max":{"1min":1.833,"5min":2.1200000000000001,"15min":9.2379999999999995},"last":1.4650000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2350000000000001,"15min":1.264},"min":{"1min":0.876,"5min":0.78600000000000003,"15min":0.621},"max":{"1min":1.915,"5min":1.927,"15min":8.1820000000000004},"last":1.585},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.345,"15min":1.4379999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.56000000000000005},"max":{"1min":1.748,"5min":2.5539999999999998,"15min":16.158999999999999},"last":1.524}]},{"osd":3,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3660000000000001,"15min":1.49},"min":{"1min":0.88,"5min":0.84799999999999998,"15min":0.56100000000000005},"max":{"1min":1.6819999999999999,"5min":2.5179999999999998,"15min":14.914},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.4259999999999999,"15min":1.454},"min":{"1min":1.044,"5min":0.93200000000000005,"15min":0.63300000000000001},"max":{"1min":1.782,"5min":1.98,"15min":7.0110000000000001},"last":1.7030000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.3420000000000001,"15min":1.4990000000000001},"min":{"1min":0.88700000000000001,"5min":0.51900000000000002,"15min":0.41399999999999998},"max":{"1min":1.855,"5min":2.4689999999999999,"15min":16.414999999999999},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.4299999999999999,"15min":1.621},"min":{"1min":0.93000000000000005,"5min":0.56200000000000006,"15min":0.32800000000000001},"max":{"1min":1.8500000000000001,"5min":2.665,"15min":17.991},"last":1.2829999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4390000000000001,"15min":1.4830000000000001},"min":{"1min":1.0249999999999999,"5min":0.72699999999999998,"15min":0.39600000000000002},"max":{"1min":1.849,"5min":3.7559999999999998,"15min":6.4630000000000001},"last":1.131},{"interface":"front","average":{"1min":1.474,"5min":1.4750000000000001,"15min":1.593},"min":{"1min":1.0109999999999999,"5min":0.56899999999999995,"15min":0.53000000000000003},"max":{"1min":1.9790000000000001,"5min":2.7010000000000001,"15min":16.364000000000001},"last":1.6850000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.591,"15min":1.6140000000000001},"min":{"1min":1.0620000000000001,"5min":0.78900000000000003,"15min":0.45800000000000002},"max":{"1min":1.887,"5min":5.0659999999999998,"15min":9.0600000000000005},"last":1.837},{"interface":"front","average":{"1min":1.575,"5min":1.585,"15min":1.698},"min":{"1min":1.1779999999999999,"5min":0.98599999999999999,"15min":0.57299999999999995},"max":{"1min":2.2029999999999998,"5min":2.6059999999999999,"15min":19.356999999999999},"last":1.8979999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542332,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76780,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94295060,"statfs":{"total":96636764160,"available":96558141440,"internally_reserved":0,"allocated":1277952,"data_stored":548447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.984,"5min":1.4350000000000001,"15min":1.3919999999999999},"min":{"1min":0.76700000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":17.346,"5min":17.346,"15min":17.346},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.321,"5min":1.345,"15min":1.3899999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.7170000000000001,"5min":3.726,"15min":9.7439999999999998},"last":1.5469999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.208,"15min":1.347},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4870000000000001,"5min":2.8700000000000001,"15min":11.987},"last":1.006},{"interface":"front","average":{"1min":1.274,"5min":1.2230000000000001,"15min":1.304},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.55300000000000005},"max":{"1min":3.7370000000000001,"5min":3.7370000000000001,"15min":16.058},"last":1.3779999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.9470000000000001,"5min":1.389,"15min":1.526},"min":{"1min":0.64500000000000002,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":18.442,"5min":18.442,"15min":18.442},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3700000000000001,"15min":1.526},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.69499999999999995},"max":{"1min":3.8250000000000002,"5min":3.8250000000000002,"15min":13.903},"last":1.742}]},{"osd":3,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.948,"5min":1.4650000000000001,"15min":1.6319999999999999},"min":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.56699999999999995},"max":{"1min":17.364999999999998,"5min":17.364999999999998,"15min":26.055},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.333,"15min":1.46},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.5449999999999999,"5min":2.9279999999999999,"15min":13.25},"last":1.3340000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.8839999999999999,"5min":1.458,"15min":1.5820000000000001},"min":{"1min":0.50700000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":17.292999999999999,"5min":17.292999999999999,"15min":29.422000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.5469999999999999,"15min":1.5189999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.42199999999999999},"max":{"1min":4.9400000000000004,"5min":12.332000000000001,"15min":12.332000000000001},"last":1.2010000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.5269999999999999,"15min":1.524},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45200000000000001},"max":{"1min":5.3150000000000004,"5min":11.09,"15min":18.260000000000002},"last":1.591},{"interface":"front","average":{"1min":2.0910000000000002,"5min":1.768,"15min":1.712},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.48099999999999998},"max":{"1min":17.224,"5min":17.224,"15min":17.224},"last":1.6870000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.6519999999999999,"15min":1.752},"min":{"1min":0.625,"5min":0.625,"15min":0.503},"max":{"1min":1.8560000000000001,"5min":12.247,"15min":12.247},"last":1.6579999999999999},{"interface":"front","average":{"1min":2.2029999999999998,"5min":1.8919999999999999,"15min":1.8839999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.53600000000000003},"max":{"1min":17.324999999999999,"5min":17.324999999999999,"15min":17.324999999999999},"last":1.79}]}]},{"osd":3,"up_from":14,"seq":60129542335,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72620,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299220,"statfs":{"total":96636764160,"available":96562401280,"internally_reserved":0,"allocated":1273856,"data_stored":548345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.113,"15min":1.2210000000000001},"min":{"1min":0.52400000000000002,"5min":0.39900000000000002,"15min":0.33300000000000002},"max":{"1min":1.284,"5min":9.2899999999999991,"15min":18.399999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.048,"5min":1.127,"15min":1.387},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":1.8140000000000001,"5min":2.23,"15min":21.047000000000001},"last":1.208}]},{"osd":1,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.92400000000000004,"15min":1.272},"min":{"1min":0.38100000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.415,"5min":3.0409999999999999,"15min":17.777999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.3680000000000001,"15min":1.288},"min":{"1min":0.57399999999999995,"5min":0.46300000000000002,"15min":0.39500000000000002},"max":{"1min":1.6890000000000001,"5min":11.228999999999999,"15min":11.228999999999999},"last":1.9670000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.224,"15min":1.2789999999999999},"min":{"1min":0.50800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":1.518,"5min":2.7519999999999998,"15min":10.964},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.5489999999999999,"15min":1.554},"min":{"1min":0.60799999999999998,"5min":0.57099999999999995,"15min":0.54000000000000004},"max":{"1min":1.857,"5min":9.4920000000000009,"15min":20.495000000000001},"last":1.7549999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.401,"15min":1.5589999999999999},"min":{"1min":0.871,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":1.6419999999999999,"5min":6.319,"15min":13.536},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.456,"15min":1.6399999999999999},"min":{"1min":0.748,"5min":0.63700000000000001,"15min":0.54600000000000004},"max":{"1min":1.736,"5min":16.398,"15min":16.398},"last":1.4339999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.681,"15min":1.6950000000000001},"min":{"1min":1.046,"5min":0.875,"15min":0.57299999999999995},"max":{"1min":1.6819999999999999,"5min":9.5690000000000008,"15min":20.606000000000002},"last":1.607},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5589999999999999,"15min":1.665},"min":{"1min":0.97299999999999998,"5min":0.70199999999999996,"15min":0.68000000000000005},"max":{"1min":1.758,"5min":5.6980000000000004,"15min":11.603},"last":1.1120000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.615,"15min":1.7949999999999999},"min":{"1min":0.81000000000000005,"5min":0.61399999999999999,"15min":0.53500000000000003},"max":{"1min":2.0870000000000002,"5min":13.648999999999999,"15min":13.648999999999999},"last":1.673},{"interface":"front","average":{"1min":1.502,"5min":1.8109999999999999,"15min":1.8220000000000001},"min":{"1min":0.98699999999999999,"5min":0.78300000000000003,"15min":0.65800000000000003},"max":{"1min":2.0249999999999999,"5min":13.679,"15min":20.556000000000001},"last":1.8340000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:19 2024","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.552,"15min":1.8480000000000001},"min":{"1min":0.78800000000000003,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.8260000000000001,"5min":3.5590000000000002,"15min":28.843},"last":1.794},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.5760000000000001,"15min":1.857},"min":{"1min":0.97999999999999998,"5min":0.72799999999999998,"15min":0.67200000000000004},"max":{"1min":1.968,"5min":9.4100000000000001,"15min":15.143000000000001},"last":1.873}]}]},{"osd":2,"up_from":14,"seq":60129542335,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68444,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303396,"statfs":{"total":96636764160,"available":96566677504,"internally_reserved":0,"allocated":1261568,"data_stored":546745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":1.002,"15min":1.091},"min":{"1min":0.48699999999999999,"5min":0.32100000000000001,"15min":0.26600000000000001},"max":{"1min":1.728,"5min":5.1859999999999999,"15min":14.427},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.3779999999999999,"15min":1.3460000000000001},"min":{"1min":0.61799999999999999,"5min":0.50900000000000001,"15min":0.39300000000000002},"max":{"1min":1.7829999999999999,"5min":15.314,"15min":15.314},"last":1.5900000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2649999999999999,"15min":1.4370000000000001},"min":{"1min":0.67600000000000005,"5min":0.51500000000000001,"15min":0.39200000000000002},"max":{"1min":1.641,"5min":9.0969999999999995,"15min":34.463999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.524,"15min":1.4530000000000001},"min":{"1min":0.88200000000000001,"5min":0.54200000000000004,"15min":0.373},"max":{"1min":1.6120000000000001,"5min":15.407,"15min":25.859999999999999},"last":1.018}]},{"osd":3,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.26,"15min":1.4179999999999999},"min":{"1min":0.78500000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.7529999999999999,"5min":12.449999999999999,"15min":19.535},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.518,"15min":1.6160000000000001},"min":{"1min":0.875,"5min":0.46999999999999997,"15min":0.46400000000000002},"max":{"1min":1.7829999999999999,"5min":22.783999999999999,"15min":22.783999999999999},"last":1.0960000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4710000000000001,"15min":1.774},"min":{"1min":1.018,"5min":0.73199999999999998,"15min":0.55700000000000005},"max":{"1min":1.9139999999999999,"5min":4.9429999999999996,"15min":29.986999999999998},"last":1.206},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.73,"15min":1.736},"min":{"1min":1.0329999999999999,"5min":0.81000000000000005,"15min":0.56799999999999995},"max":{"1min":1.843,"5min":15.553000000000001,"15min":21.199999999999999},"last":1.647}]},{"osd":5,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.766,"15min":1.738},"min":{"1min":1.0569999999999999,"5min":0.84299999999999997,"15min":0.51500000000000001},"max":{"1min":1.881,"5min":14.066000000000001,"15min":19.681999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.6100000000000001,"5min":1.879,"15min":1.8220000000000001},"min":{"1min":1.2929999999999999,"5min":0.84799999999999998,"15min":0.60899999999999999},"max":{"1min":2.2189999999999999,"5min":14.025,"15min":14.702999999999999},"last":1.4350000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.889,"15min":1.788},"min":{"1min":1.024,"5min":0.76800000000000002,"15min":0.68000000000000005},"max":{"1min":1.976,"5min":15.683999999999999,"15min":15.683999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.655,"5min":2.0299999999999998,"15min":1.867},"min":{"1min":1.069,"5min":0.93500000000000005,"15min":0.58899999999999997},"max":{"1min":2.2879999999999998,"5min":12.629,"15min":19.783000000000001},"last":1.3089999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:10:24 2024","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.9890000000000001,"15min":2.0699999999999998},"min":{"1min":1.4359999999999999,"5min":0.86899999999999999,"15min":0.63100000000000001},"max":{"1min":1.968,"5min":17.099,"15min":57.006999999999998},"last":1.52},{"interface":"front","average":{"1min":1.7250000000000001,"5min":2.0859999999999999,"15min":2.2080000000000002},"min":{"1min":1.131,"5min":1.081,"15min":0.56100000000000005},"max":{"1min":2.3359999999999999,"5min":22.853999999999999,"15min":22.853999999999999},"last":1.569}]}]},{"osd":1,"up_from":14,"seq":60129542334,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72600,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299240,"statfs":{"total":96636764160,"available":96562421760,"internally_reserved":0,"allocated":1257472,"data_stored":546711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:10:29 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":1.3140000000000001,"15min":1.212},"min":{"1min":0.59199999999999997,"5min":0.48899999999999999,"15min":0.34399999999999997},"max":{"1min":1.506,"5min":30.460999999999999,"15min":30.460999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.516,"5min":1.258,"15min":1.1519999999999999},"min":{"1min":0.628,"5min":0.38700000000000001,"15min":0.314},"max":{"1min":9.4589999999999996,"5min":9.4589999999999996,"15min":18.658000000000001},"last":1.133}]},{"osd":2,"last update":"Mon Jan 15 18:10:29 2024","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.24,"15min":1.2270000000000001},"min":{"1min":0.66100000000000003,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":4.7119999999999997,"5min":7.117,"15min":21.126000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.302,"5min":1.7030000000000001,"15min":1.575},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.48699999999999999},"max":{"1min":1.8360000000000001,"5min":30.593,"15min":30.593},"last":1.5429999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:10:29 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.6180000000000001,"15min":1.552},"min":{"1min":0.81399999999999995,"5min":0.58099999999999996,"15min":0.48599999999999999},"max":{"1min":1.722,"5min":30.547000000000001,"15min":30.547000000000001},"last":1.2310000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.3240000000000001,"15min":1.3460000000000001},"min":{"1min":0.85899999999999999,"5min":0.52700000000000002,"15min":0.437},"max":{"1min":1.5580000000000001,"5min":5.4729999999999999,"15min":15.510999999999999},"last":1.3420000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:10:29 2024","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.6619999999999999,"15min":1.679},"min":{"1min":1.2090000000000001,"5min":0.75900000000000001,"15min":0.72499999999999998},"max":{"1min":1.796,"5min":8.8170000000000002,"15min":14.827},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.569,"15min":1.6299999999999999},"min":{"1min":0.91400000000000003,"5min":0.72999999999999998,"15min":0.69299999999999995},"max":{"1min":4.6369999999999996,"5min":4.6369999999999996,"15min":14.851000000000001},"last":1.6850000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:10:29 2024","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.5489999999999999,"15min":1.5900000000000001},"min":{"1min":0.98399999999999999,"5min":0.77600000000000002,"15min":0.59599999999999997},"max":{"1min":4.7759999999999998,"5min":4.7759999999999998,"15min":13.228999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.47,"5min":1.9390000000000001,"15min":1.9370000000000001},"min":{"1min":1.0820000000000001,"5min":0.78700000000000003,"15min":0.68300000000000005},"max":{"1min":1.859,"5min":34.042999999999999,"15min":34.042999999999999},"last":1.748}]},{"osd":6,"last update":"Mon Jan 15 18:10:29 2024","interfaces":[{"interface":"back","average":{"1min":1.99,"5min":1.6599999999999999,"15min":1.7709999999999999},"min":{"1min":1.054,"5min":0.83899999999999997,"15min":0.64100000000000001},"max":{"1min":10.250999999999999,"5min":10.250999999999999,"15min":34.158000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.7270000000000001,"15min":1.6299999999999999},"min":{"1min":1.1399999999999999,"5min":0.67900000000000005,"15min":0.62},"max":{"1min":1.9350000000000001,"5min":8.9909999999999997,"15min":8.9909999999999997},"last":1.4670000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:10:29 2024","interfaces":[{"interface":"back","average":{"1min":1.698,"5min":2.0720000000000001,"15min":1.877},"min":{"1min":1.012,"5min":0.77300000000000002,"15min":0.64600000000000002},"max":{"1min":3.7709999999999999,"5min":33.659999999999997,"15min":33.659999999999997},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.7170000000000001,"5min":1.768,"15min":1.8120000000000001},"min":{"1min":1.123,"5min":0.81000000000000005,"15min":0.60499999999999998},"max":{"1min":4.8380000000000001,"5min":7.8410000000000002,"15min":24.289999999999999},"last":1.282}]}]},{"osd":0,"up_from":14,"seq":60129542334,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69028,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302812,"statfs":{"total":96636764160,"available":96566079488,"internally_reserved":0,"allocated":1855488,"data_stored":1137661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.2210000000000001,"15min":1.083},"min":{"1min":0.437,"5min":0.42799999999999999,"15min":0.312},"max":{"1min":2.8929999999999998,"5min":12.050000000000001,"15min":12.050000000000001},"last":1.389},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.4430000000000001,"15min":1.3340000000000001},"min":{"1min":0.50900000000000001,"5min":0.49099999999999999,"15min":0.36199999999999999},"max":{"1min":2.9780000000000002,"5min":11.962,"15min":11.962},"last":1.359}]},{"osd":2,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.179,"15min":1.2090000000000001},"min":{"1min":0.56000000000000005,"5min":0.47399999999999998,"15min":0.27600000000000002},"max":{"1min":1.595,"5min":7.4560000000000004,"15min":11.59},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":1.1459999999999999,"15min":1.167},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.35899999999999999},"max":{"1min":1.5649999999999999,"5min":7.3239999999999998,"15min":14.512},"last":0.876}]},{"osd":3,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.425,"15min":1.4319999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.41699999999999998},"max":{"1min":1.5669999999999999,"5min":11.904,"15min":11.904},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.1379999999999999,"15min":1.2490000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.29599999999999999},"max":{"1min":1.6850000000000001,"5min":3.202,"15min":14.74},"last":1.744}]},{"osd":4,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.375,"15min":1.5900000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6140000000000001,"5min":7.4000000000000004,"15min":32.115000000000002},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.284,"5min":1.571,"15min":1.702},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.67700000000000005},"max":{"1min":1.8260000000000001,"5min":9.0790000000000006,"15min":15.975},"last":1.498}]},{"osd":5,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.556,"15min":1.722},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":1.6200000000000001,"5min":12.006,"15min":32.174999999999997},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.514,"15min":1.6359999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.442},"max":{"1min":1.8939999999999999,"5min":4.7119999999999997,"15min":17.225000000000001},"last":1.5569999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.7849999999999999,"15min":1.7989999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.70299999999999996},"max":{"1min":3.4399999999999999,"5min":24.039999999999999,"15min":24.039999999999999},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.7549999999999999,"15min":1.794},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.625},"max":{"1min":1.7450000000000001,"5min":18.143999999999998,"15min":32.216999999999999},"last":1.0369999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.853,"15min":2.1230000000000002},"min":{"1min":0.754,"5min":0.754,"15min":0.69099999999999995},"max":{"1min":1.7350000000000001,"5min":26.427,"15min":34.131},"last":1.611},{"interface":"front","average":{"1min":1.474,"5min":2.089,"15min":2.0379999999999998},"min":{"1min":0.753,"5min":0.753,"15min":0.61799999999999999},"max":{"1min":3.9790000000000001,"5min":44.694000000000003,"15min":44.694000000000003},"last":1.806}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:10:32.719 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:10:32.720 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:10:32.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.971+0000 7f7d4dc00640 1 Processor -- start 2024-01-15T18:10:32.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.971+0000 7f7d4dc00640 1 -- start start 2024-01-15T18:10:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.973+0000 7f7d4dc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7d480d1660 0x7f7d480d3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.973+0000 7f7d4dc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1a40 0x7f7d480ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.973+0000 7f7d4dc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d480b3310 con 0x7f7d480cf6b0 2024-01-15T18:10:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.973+0000 7f7d4dc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d480b2a50 con 0x7f7d480d1660 2024-01-15T18:10:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.973+0000 7f7d4dc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480cebe0 con 0x7f7d480d1a40 2024-01-15T18:10:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.973+0000 7f7d4d200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7d480cf6b0 0x7f7d480cfb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52752/0 (socket says 172.21.15.26:52752) 2024-01-15T18:10:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.973+0000 7f7d4d200640 1 -- 172.21.15.26:0/546633297 learned_addr learned my addr 172.21.15.26:0/546633297 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.973+0000 7f7d47e00640 1 --2- 172.21.15.26:0/546633297 >> v2:172.21.15.26:3300/0 conn(0x7f7d480d1660 0x7f7d480d3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.973+0000 7f7d47400640 1 --2- 172.21.15.26:0/546633297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1a40 0x7f7d480ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.974+0000 7f7d47e00640 1 -- 172.21.15.26:0/546633297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1a40 msgr2=0x7f7d480ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:32.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.974+0000 7f7d47e00640 1 --2- 172.21.15.26:0/546633297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1a40 0x7f7d480ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.974+0000 7f7d47e00640 1 -- 172.21.15.26:0/546633297 >> v1:172.21.15.26:6789/0 conn(0x7f7d480cf6b0 legacy=0x7f7d480cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:32.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.974+0000 7f7d46a00640 1 -- 172.21.15.26:0/546633297 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523995950 0 0) 0x7f7d480b3310 con 0x7f7d480cf6b0 2024-01-15T18:10:32.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.974+0000 7f7d47e00640 1 -- 172.21.15.26:0/546633297 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d480b2cc0 con 0x7f7d480d1660 2024-01-15T18:10:32.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.975+0000 7f7d47e00640 1 --2- 172.21.15.26:0/546633297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480d1660 0x7f7d480d3a00 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009d90 tx=0x7f7d3802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e84c63b23d2d531 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:32.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.975+0000 7f7d46a00640 1 -- 172.21.15.26:0/546633297 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3802cc50 con 0x7f7d480d1660 2024-01-15T18:10:32.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.975+0000 7f7d46a00640 1 -- 172.21.15.26:0/546633297 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3802cdb0 con 0x7f7d480d1660 2024-01-15T18:10:32.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.975+0000 7f7d46a00640 1 -- 172.21.15.26:0/546633297 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38034660 con 0x7f7d480d1660 2024-01-15T18:10:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.976+0000 7f7d4dc00640 1 -- 172.21.15.26:0/546633297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480d1660 msgr2=0x7f7d480d3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.976+0000 7f7d4dc00640 1 --2- 172.21.15.26:0/546633297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480d1660 0x7f7d480d3a00 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009d90 tx=0x7f7d3802bea0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.976+0000 7f7d4dc00640 1 -- 172.21.15.26:0/546633297 shutdown_connections 2024-01-15T18:10:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.976+0000 7f7d4dc00640 1 --2- 172.21.15.26:0/546633297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1a40 0x7f7d480ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.976+0000 7f7d4dc00640 1 --2- 172.21.15.26:0/546633297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480d1660 0x7f7d480d3a00 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.976+0000 7f7d4dc00640 1 -- 172.21.15.26:0/546633297 >> 172.21.15.26:0/546633297 conn(0x7f7d4809c350 msgr2=0x7f7d480d3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.976+0000 7f7d4dc00640 1 -- 172.21.15.26:0/546633297 shutdown_connections 2024-01-15T18:10:32.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.976+0000 7f7d4dc00640 1 -- 172.21.15.26:0/546633297 wait complete. 2024-01-15T18:10:32.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.978+0000 7f7d4dc00640 1 Processor -- start 2024-01-15T18:10:32.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.978+0000 7f7d4dc00640 1 -- start start 2024-01-15T18:10:32.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.979+0000 7f7d4dc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480cf6b0 0x7f7d480b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.979+0000 7f7d4dc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1660 0x7f7d480d2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.979+0000 7f7d4dc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d480d1a40 0x7f7d480d3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.979+0000 7f7d4dc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4814b8a0 con 0x7f7d480cf6b0 2024-01-15T18:10:32.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.979+0000 7f7d4dc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480b8990 con 0x7f7d480d1a40 2024-01-15T18:10:32.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.979+0000 7f7d4dc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4814af70 con 0x7f7d480d1660 2024-01-15T18:10:32.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480cf6b0 0x7f7d480b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480cf6b0 0x7f7d480b62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54026/0 (socket says 172.21.15.26:54026) 2024-01-15T18:10:32.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47e00640 1 -- 172.21.15.26:0/2560924585 learned_addr learned my addr 172.21.15.26:0/2560924585 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47400640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1660 0x7f7d480d2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d4d200640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d480d1a40 0x7f7d480d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47e00640 1 -- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1660 msgr2=0x7f7d480d2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47e00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1660 0x7f7d480d2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47e00640 1 -- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d480d1a40 msgr2=0x7f7d480d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47e00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d480d1a40 0x7f7d480d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47e00640 1 -- 172.21.15.26:0/2560924585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38009a70 con 0x7f7d480cf6b0 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d47400640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1660 0x7f7d480d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.980+0000 7f7d4d200640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d480d1a40 0x7f7d480d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.981+0000 7f7d47e00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480cf6b0 0x7f7d480b62c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009d90 tx=0x7f7d38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.981+0000 7f7d3fe00640 1 -- 172.21.15.26:0/2560924585 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38040650 con 0x7f7d480cf6b0 2024-01-15T18:10:32.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.981+0000 7f7d3fe00640 1 -- 172.21.15.26:0/2560924585 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d380407b0 con 0x7f7d480cf6b0 2024-01-15T18:10:32.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.981+0000 7f7d3fe00640 1 -- 172.21.15.26:0/2560924585 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38040b60 con 0x7f7d480cf6b0 2024-01-15T18:10:32.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.981+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4814b250 con 0x7f7d480cf6b0 2024-01-15T18:10:32.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.981+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d480b7b80 con 0x7f7d480cf6b0 2024-01-15T18:10:32.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.982+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d48067b60 con 0x7f7d480cf6b0 2024-01-15T18:10:32.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.984+0000 7f7d3fe00640 1 -- 172.21.15.26:0/2560924585 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f7d38040cc0 con 0x7f7d480cf6b0 2024-01-15T18:10:32.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.985+0000 7f7d3fe00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d2006c1f0 0x7f7d2006e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:32.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.985+0000 7f7d3fe00640 1 -- 172.21.15.26:0/2560924585 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7d380ab5b0 con 0x7f7d480cf6b0 2024-01-15T18:10:32.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.986+0000 7f7d47400640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d2006c1f0 0x7f7d2006e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:32.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.987+0000 7f7d47400640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d2006c1f0 0x7f7d2006e6b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7d30006fd0 tx=0x7f7d30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:32.990+0000 7f7d3fe00640 1 -- 172.21.15.26:0/2560924585 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d38077db0 con 0x7f7d480cf6b0 2024-01-15T18:10:33.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.244+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7d480c6380 con 0x7f7d480cf6b0 2024-01-15T18:10:33.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.249+0000 7f7d3fe00640 1 -- 172.21.15.26:0/2560924585 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7d380440b0 con 0x7f7d480cf6b0 2024-01-15T18:10:33.252 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-15T18:10:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA== 2024-01-15T18:10:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-01-15T18:10:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-01-15T18:10:33.253 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:10:33.253 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-01-15T18:10:33.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d2006c1f0 msgr2=0x7f7d2006e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:33.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d2006c1f0 0x7f7d2006e6b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7d30006fd0 tx=0x7f7d30008040 comp rx=0 tx=0).stop 2024-01-15T18:10:33.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480cf6b0 msgr2=0x7f7d480b62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480cf6b0 0x7f7d480b62c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009d90 tx=0x7f7d38004a40 comp rx=0 tx=0).stop 2024-01-15T18:10:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 shutdown_connections 2024-01-15T18:10:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7d480d1660 0x7f7d480d2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7d2006c1f0 0x7f7d2006e6b0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7d480d1a40 0x7f7d480d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 --2- 172.21.15.26:0/2560924585 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7d480cf6b0 0x7f7d480b62c0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 >> 172.21.15.26:0/2560924585 conn(0x7f7d4809c350 msgr2=0x7f7d480b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 shutdown_connections 2024-01-15T18:10:33.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.252+0000 7f7d4dc00640 1 -- 172.21.15.26:0/2560924585 wait complete. 2024-01-15T18:10:33.274 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:10:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.489+0000 7f9463e00640 1 Processor -- start 2024-01-15T18:10:33.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.489+0000 7f9463e00640 1 -- start start 2024-01-15T18:10:33.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.491+0000 7f9463e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9464143c40 0x7f9464071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:33.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.491+0000 7f9463e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f94640ccc10 0x7f946406d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:33.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.491+0000 7f9463e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9464070e50 con 0x7f94640717d0 2024-01-15T18:10:33.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.491+0000 7f9463e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94640723c0 con 0x7f94640ccc10 2024-01-15T18:10:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.491+0000 7f9463e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9464144020 con 0x7f9464143c40 2024-01-15T18:10:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.491+0000 7f9462000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f94640717d0 0x7f94640cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52758/0 (socket says 172.21.15.26:52758) 2024-01-15T18:10:33.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.491+0000 7f9462000640 1 -- 172.21.15.26:0/2160518144 learned_addr learned my addr 172.21.15.26:0/2160518144 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.491+0000 7f9462a00640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9464143c40 0x7f9464071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9463400640 1 --2- 172.21.15.26:0/2160518144 >> v2:172.21.15.124:3300/0 conn(0x7f94640ccc10 0x7f946406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2213507871 0 0) 0x7f9464070e50 con 0x7f94640717d0 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9444003520 con 0x7f94640717d0 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 488389807 0 0) 0x7f9444003520 con 0x7f94640717d0 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9464143c40 msgr2=0x7f9464071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9464143c40 0x7f9464071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 >> v2:172.21.15.124:3300/0 conn(0x7f94640ccc10 msgr2=0x7f946406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 --2- 172.21.15.26:0/2160518144 >> v2:172.21.15.124:3300/0 conn(0x7f94640ccc10 0x7f946406d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464071e00 con 0x7f94640717d0 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f944c001670 con 0x7f94640717d0 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 >> v1:172.21.15.26:6789/0 conn(0x7f94640717d0 legacy=0x7f94640cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:33.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9444003970 0x7f9444003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94440047c0 0x7f9444004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f944400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94640723c0 con 0x7f9444005180 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9464144020 con 0x7f9444003970 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.492+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944400bf40 con 0x7f94440047c0 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.493+0000 7f9463400640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f944400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.493+0000 7f9462a00640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9444003970 0x7f9444003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.493+0000 7f9462000640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94440047c0 0x7f9444004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.493+0000 7f9463400640 1 -- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94440047c0 msgr2=0x7f9444004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.493+0000 7f9463400640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f94440047c0 0x7f9444004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.493+0000 7f9463400640 1 -- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9444003970 msgr2=0x7f9444003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.493+0000 7f9463400640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9444003970 0x7f9444003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.493+0000 7f9463400640 1 -- 172.21.15.26:0/2160518144 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9444003520 con 0x7f9444005180 2024-01-15T18:10:33.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.493+0000 7f9462a00640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9444003970 0x7f9444003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.495+0000 7f9463400640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f944400b980 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f945c009ac0 tx=0x7f94440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62122ac1a5230d46 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.495+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c02c710 con 0x7f9444005180 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.495+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c02c870 con 0x7f9444005180 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.495+0000 7f9461600640 1 -- 172.21.15.26:0/2160518144 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c02cc00 con 0x7f9444005180 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.496+0000 7f9463e00640 1 -- 172.21.15.26:0/2160518144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 msgr2=0x7f944400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.496+0000 7f9463e00640 1 --2- 172.21.15.26:0/2160518144 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f944400b980 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f945c009ac0 tx=0x7f94440057f0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.496+0000 7f9463400640 1 -- 172.21.15.26:0/2160518144 reap_dead start 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.496+0000 7f9463e00640 1 -- 172.21.15.26:0/2160518144 shutdown_connections 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.496+0000 7f9463e00640 1 -- 172.21.15.26:0/2160518144 >> 172.21.15.26:0/2160518144 conn(0x7f946406a800 msgr2=0x7f946406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.496+0000 7f9463e00640 1 -- 172.21.15.26:0/2160518144 shutdown_connections 2024-01-15T18:10:33.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.496+0000 7f9463e00640 1 -- 172.21.15.26:0/2160518144 wait complete. 2024-01-15T18:10:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.498+0000 7f9463e00640 1 Processor -- start 2024-01-15T18:10:33.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.498+0000 7f9463e00640 1 -- start start 2024-01-15T18:10:33.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.499+0000 7f9463e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f946413b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.499+0000 7f9463e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9464142080 0x7f94641541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.499+0000 7f9463e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9464142500 0x7f94641546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.499+0000 7f9463e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94641413d0 con 0x7f9444005180 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.499+0000 7f9463e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946413b560 con 0x7f9464142500 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.499+0000 7f9463e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9464140330 con 0x7f9464142080 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.499+0000 7f9462a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f946413b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.499+0000 7f9462a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f946413b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54052/0 (socket says 172.21.15.26:54052) 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.499+0000 7f9462a00640 1 -- 172.21.15.26:0/34570069 learned_addr learned my addr 172.21.15.26:0/34570069 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f9462000640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9464142080 0x7f94641541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f9463400640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9464142500 0x7f94641546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f9462a00640 1 -- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9464142080 msgr2=0x7f94641541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f9462a00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9464142080 0x7f94641541b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f9462a00640 1 -- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9464142500 msgr2=0x7f94641546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:33.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f9462a00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9464142500 0x7f94641546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f9462a00640 1 -- 172.21.15.26:0/34570069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464140610 con 0x7f9444005180 2024-01-15T18:10:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f9462a00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f946413b020 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f9454007f00 tx=0x7f9454010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f945aa00640 1 -- 172.21.15.26:0/34570069 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945400e050 con 0x7f9444005180 2024-01-15T18:10:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f945aa00640 1 -- 172.21.15.26:0/34570069 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945400f040 con 0x7f9444005180 2024-01-15T18:10:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f945aa00640 1 -- 172.21.15.26:0/34570069 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454003b90 con 0x7f9444005180 2024-01-15T18:10:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.500+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94641408f0 con 0x7f9444005180 2024-01-15T18:10:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.501+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9464140eb0 con 0x7f9444005180 2024-01-15T18:10:33.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.502+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94640706e0 con 0x7f9444005180 2024-01-15T18:10:33.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.503+0000 7f945aa00640 1 -- 172.21.15.26:0/34570069 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f9454003cf0 con 0x7f9444005180 2024-01-15T18:10:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.504+0000 7f945aa00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f943c06c210 0x7f943c06e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.504+0000 7f945aa00640 1 -- 172.21.15.26:0/34570069 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9454089510 con 0x7f9444005180 2024-01-15T18:10:33.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.504+0000 7f9462000640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f943c06c210 0x7f943c06e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:33.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.505+0000 7f9462000640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f943c06c210 0x7f943c06e6d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f944c001050 tx=0x7f944c001170 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:33.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.508+0000 7f945aa00640 1 -- 172.21.15.26:0/34570069 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9454055d70 con 0x7f9444005180 2024-01-15T18:10:33.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.739+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9464067450 con 0x7f9444005180 2024-01-15T18:10:33.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.740+0000 7f945aa00640 1 -- 172.21.15.26:0/34570069 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f9454059390 con 0x7f9444005180 2024-01-15T18:10:33.744 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:10:33.744 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7015,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1631584304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1631584304},{"type":"v1","addr":"172.21.15.124:6835","nonce":1631584304}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2396377458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2396377458},{"type":"v1","addr":"172.21.15.26:6833","nonce":2396377458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/81533892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":81533892},{"type":"v1","addr":"172.21.15.26:6835","nonce":81533892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:10:22.071509+0000","modified":"2024-01-15T18:10:25.141235+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":36242},"failed":[],"damaged":[],"stopped":[],"info":{"gid_36242":{"gid":36242,"name":"d","rank":0,"incarnation":59,"state":"up:active","state_seq":5,"addr":"172.21.15.124:6837/421656183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":421656183},{"type":"v1","addr":"172.21.15.124:6837","nonce":421656183}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-01-15T18:10:33.744 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 63 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f943c06c210 msgr2=0x7f943c06e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f943c06c210 0x7f943c06e6d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f944c001050 tx=0x7f944c001170 comp rx=0 tx=0).stop 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 msgr2=0x7f946413b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f946413b020 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f9454007f00 tx=0x7f9454010590 comp rx=0 tx=0).stop 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 shutdown_connections 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9464142080 0x7f94641541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f943c06c210 0x7f943c06e6d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9464142500 0x7f94641546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 --2- 172.21.15.26:0/34570069 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9444005180 0x7f946413b020 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.743+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 >> 172.21.15.26:0/34570069 conn(0x7f946406a800 msgr2=0x7f94640b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.744+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 shutdown_connections 2024-01-15T18:10:33.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:33.744+0000 7f9463e00640 1 -- 172.21.15.26:0/34570069 wait complete. 2024-01-15T18:10:34.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:10:22.071509+0000', 'modified': '2024-01-15T18:10:25.141235+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 36242}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_36242': {'gid': 36242, 'name': 'd', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.124:6837/421656183', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.124:6836', 'nonce': 421656183}, {'type': 'v1', 'addr': '172.21.15.124:6837', 'nonce': 421656183}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:10:34.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:10:34.171 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-15T18:10:34.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.399+0000 7f6a45c00640 1 Processor -- start 2024-01-15T18:10:34.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.399+0000 7f6a45c00640 1 -- start start 2024-01-15T18:10:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.401+0000 7f6a45c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6a4013df80 0x7f6a4015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.401+0000 7f6a45c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a400cf9a0 0x7f6a400cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.401+0000 7f6a45c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4006c470 con 0x7f6a4013e360 2024-01-15T18:10:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.401+0000 7f6a45c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4006c750 con 0x7f6a400cf9a0 2024-01-15T18:10:34.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.401+0000 7f6a45c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4006ca30 con 0x7f6a4013df80 2024-01-15T18:10:34.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.401+0000 7f6a3f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a400cf9a0 0x7f6a400cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.402+0000 7f6a3f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6a400cf9a0 0x7f6a400cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54054/0 (socket says 172.21.15.26:54054) 2024-01-15T18:10:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.402+0000 7f6a3f400640 1 -- 172.21.15.26:0/712603763 learned_addr learned my addr 172.21.15.26:0/712603763 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.402+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/712603763 >> v2:172.21.15.124:3300/0 conn(0x7f6a4013df80 0x7f6a4015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.402+0000 7f6a3f400640 1 -- 172.21.15.26:0/712603763 >> v2:172.21.15.124:3300/0 conn(0x7f6a4013df80 msgr2=0x7f6a4015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.402+0000 7f6a3f400640 1 --2- 172.21.15.26:0/712603763 >> v2:172.21.15.124:3300/0 conn(0x7f6a4013df80 0x7f6a4015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.402+0000 7f6a3f400640 1 -- 172.21.15.26:0/712603763 >> v1:172.21.15.124:6789/0 conn(0x7f6a4013e360 legacy=0x7f6a400cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.402+0000 7f6a3f400640 1 -- 172.21.15.26:0/712603763 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4006cd10 con 0x7f6a400cf9a0 2024-01-15T18:10:34.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.403+0000 7f6a3f400640 1 --2- 172.21.15.26:0/712603763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a400cf9a0 0x7f6a400cd840 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f6a40072770 tx=0x7f6a3802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159df69914d9eb0d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a3d600640 1 -- 172.21.15.26:0/712603763 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3802cb20 con 0x7f6a400cf9a0 2024-01-15T18:10:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a3d600640 1 -- 172.21.15.26:0/712603763 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3802cc80 con 0x7f6a400cf9a0 2024-01-15T18:10:34.407 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a3d600640 1 -- 172.21.15.26:0/712603763 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380343f0 con 0x7f6a400cf9a0 2024-01-15T18:10:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a45c00640 1 -- 172.21.15.26:0/712603763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a400cf9a0 msgr2=0x7f6a400cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a45c00640 1 --2- 172.21.15.26:0/712603763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a400cf9a0 0x7f6a400cd840 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f6a40072770 tx=0x7f6a3802bf80 comp rx=0 tx=0).stop 2024-01-15T18:10:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a45c00640 1 -- 172.21.15.26:0/712603763 shutdown_connections 2024-01-15T18:10:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a45c00640 1 --2- 172.21.15.26:0/712603763 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a400cf9a0 0x7f6a400cd840 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a45c00640 1 --2- 172.21.15.26:0/712603763 >> v2:172.21.15.124:3300/0 conn(0x7f6a4013df80 0x7f6a4015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a45c00640 1 -- 172.21.15.26:0/712603763 >> 172.21.15.26:0/712603763 conn(0x7f6a4006a800 msgr2=0x7f6a4006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.404+0000 7f6a45c00640 1 -- 172.21.15.26:0/712603763 shutdown_connections 2024-01-15T18:10:34.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.405+0000 7f6a45c00640 1 -- 172.21.15.26:0/712603763 wait complete. 2024-01-15T18:10:34.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.406+0000 7f6a45c00640 1 Processor -- start 2024-01-15T18:10:34.409 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.406+0000 7f6a45c00640 1 -- start start 2024-01-15T18:10:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.407+0000 7f6a45c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a400cf9a0 0x7f6a400b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.407+0000 7f6a45c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a4013df80 0x7f6a40153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.407+0000 7f6a45c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a4013e360 0x7f6a400a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.407+0000 7f6a45c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401f3090 con 0x7f6a4013df80 2024-01-15T18:10:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.407+0000 7f6a45c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40153870 con 0x7f6a400cf9a0 2024-01-15T18:10:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.407+0000 7f6a45c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401f31f0 con 0x7f6a4013e360 2024-01-15T18:10:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a4013df80 0x7f6a40153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a4013df80 0x7f6a40153330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54068/0 (socket says 172.21.15.26:54068) 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3e000640 1 -- 172.21.15.26:0/774239984 learned_addr learned my addr 172.21.15.26:0/774239984 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a400cf9a0 0x7f6a400b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3e000640 1 -- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a4013e360 msgr2=0x7f6a400a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3f400640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a4013e360 0x7f6a400a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3e000640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a4013e360 0x7f6a400a8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3e000640 1 -- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a400cf9a0 msgr2=0x7f6a400b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3e000640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a400cf9a0 0x7f6a400b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3e000640 1 -- 172.21.15.26:0/774239984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38009b30 con 0x7f6a4013df80 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.408+0000 7f6a3e000640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a4013df80 0x7f6a40153330 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800c400 tx=0x7f6a2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.409+0000 7f6a36a00640 1 -- 172.21.15.26:0/774239984 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a280052d0 con 0x7f6a4013df80 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.409+0000 7f6a36a00640 1 -- 172.21.15.26:0/774239984 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a28005430 con 0x7f6a4013df80 2024-01-15T18:10:34.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.409+0000 7f6a36a00640 1 -- 172.21.15.26:0/774239984 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28003830 con 0x7f6a4013df80 2024-01-15T18:10:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.409+0000 7f6a45c00640 1 -- 172.21.15.26:0/774239984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a400b3640 con 0x7f6a4013df80 2024-01-15T18:10:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.409+0000 7f6a45c00640 1 -- 172.21.15.26:0/774239984 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a400b3c00 con 0x7f6a4013df80 2024-01-15T18:10:34.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.411+0000 7f6a36a00640 1 -- 172.21.15.26:0/774239984 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f6a28003990 con 0x7f6a4013df80 2024-01-15T18:10:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.412+0000 7f6a36a00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a1806c1f0 0x7f6a1806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.412+0000 7f6a36a00640 1 -- 172.21.15.26:0/774239984 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6a28087d70 con 0x7f6a4013df80 2024-01-15T18:10:34.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.413+0000 7f6a487b2740 1 Processor -- start 2024-01-15T18:10:34.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.413+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a1806c1f0 0x7f6a1806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.413+0000 7f6a487b2740 1 -- start start 2024-01-15T18:10:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 0x557ef4463280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a487b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef43c1880 con 0x557ef44625e0 2024-01-15T18:10:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a487b2740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef43d0ae0 con 0x557ef4462ea0 2024-01-15T18:10:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a487b2740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ef43ce1d0 con 0x557ef445dba0 2024-01-15T18:10:34.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a1806c1f0 0x7f6a1806e6b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6a30006fd0 tx=0x7f6a30008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54078/0 (socket says 172.21.15.26:54078) 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a3f400640 1 -- 172.21.15.26:0/3309357598 learned_addr learned my addr 172.21.15.26:0/3309357598 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 0x557ef4463280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.414+0000 7f6a3e000640 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a3f400640 1 -- 172.21.15.26:0/3309357598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 msgr2=0x557ef44620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a3f400640 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a3f400640 1 -- 172.21.15.26:0/3309357598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 msgr2=0x557ef4463280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a3f400640 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 0x557ef4463280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a3f400640 1 -- 172.21.15.26:0/3309357598 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef4391860 con 0x557ef44625e0 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a3e000640 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:34.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a3f400640 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a6560 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009f10 tx=0x7f6a3802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbc15301739c7ed server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a2f400640 1 -- 172.21.15.26:0/3309357598 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3802cb20 con 0x557ef44625e0 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a2f400640 1 -- 172.21.15.26:0/3309357598 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38040470 con 0x557ef44625e0 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.415+0000 7f6a2f400640 1 -- 172.21.15.26:0/3309357598 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a380407e0 con 0x557ef44625e0 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.416+0000 7f6a487b2740 1 -- 172.21.15.26:0/3309357598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 msgr2=0x557ef44a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.416+0000 7f6a487b2740 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a6560 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f6a38009f10 tx=0x7f6a3802e000 comp rx=0 tx=0).stop 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.416+0000 7f6a487b2740 1 -- 172.21.15.26:0/3309357598 shutdown_connections 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.416+0000 7f6a487b2740 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.416+0000 7f6a487b2740 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a6560 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.416+0000 7f6a487b2740 1 --2- 172.21.15.26:0/3309357598 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 0x557ef4463280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.416+0000 7f6a487b2740 1 -- 172.21.15.26:0/3309357598 >> 172.21.15.26:0/3309357598 conn(0x557ef4424c70 msgr2=0x557ef44a8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.417+0000 7f6a487b2740 1 -- 172.21.15.26:0/3309357598 shutdown_connections 2024-01-15T18:10:34.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.417+0000 7f6a487b2740 1 -- 172.21.15.26:0/3309357598 wait complete. 2024-01-15T18:10:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.417+0000 7f6a487b2740 1 -- start start 2024-01-15T18:10:34.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.418+0000 7f6a487b2740 1 Processor -- start 2024-01-15T18:10:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 0x557ef44a0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a487b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef43c9ac0 con 0x557ef44625e0 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a487b2740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef43662f0 con 0x557ef4462ea0 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a487b2740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ef4383c70 con 0x557ef445dba0 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a487b2740 10 client.0 _ll_register_callbacks cb 0x557ef43883b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54092/0 (socket says 172.21.15.26:54092) 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3e000640 1 -- 172.21.15.26:0/1634860871 learned_addr learned my addr 172.21.15.26:0/1634860871 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3f400640 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 0x557ef44a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3e000640 1 -- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 msgr2=0x557ef44af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3e000640 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3e000640 1 -- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 msgr2=0x557ef44a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3e000640 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 0x557ef44a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.419+0000 7f6a3e000640 1 -- 172.21.15.26:0/1634860871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4006cd10 con 0x557ef44625e0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.420+0000 7f6a3e000640 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a20e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800a040 tx=0x7f6a28058ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0760b9d4a173cf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.420+0000 7f6a27e00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.420+0000 7f6a27e00640 1 -- 172.21.15.26:0/1634860871 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a280605f0 con 0x557ef44625e0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.420+0000 7f6a27e00640 1 -- 172.21.15.26:0/1634860871 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6a28060160 con 0x557ef44625e0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.420+0000 7f6a487b2740 1 -- 172.21.15.26:0/1634860871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ef43c2aa0 con 0x557ef44625e0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.420+0000 7f6a27e00640 1 -- 172.21.15.26:0/1634860871 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a28060320 con 0x557ef44625e0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.420+0000 7f6a487b2740 10 client.8904 fetch_fsmap learned FSMap version 63 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.420+0000 7f6a487b2740 1 -- 172.21.15.26:0/1634860871 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x557ef43c8140 con 0x557ef44625e0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.420+0000 7f6a27e00640 1 -- 172.21.15.26:0/1634860871 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ef43c2aa0 con 0x557ef44625e0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.421+0000 7f6a27e00640 1 -- 172.21.15.26:0/1634860871 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a2805e1f0 con 0x557ef44625e0 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.421+0000 7f6a487b2740 10 client.8904 fetch_fsmap finished waiting for FSMap version 63 2024-01-15T18:10:34.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.421+0000 7f6a487b2740 10 client.8904 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:10:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.421+0000 7f6a487b2740 20 client.8904 populate_metadata read hostname 'smithi026' 2024-01-15T18:10:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.421+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] conn(0x557ef44ae590 0x557ef44c6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.421+0000 7f6a487b2740 4 client.8904 mds_command: new command op to 36242 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:10:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.421+0000 7f6a487b2740 1 -- 172.21.15.26:0/1634860871 --> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ef43c27c0 con 0x557ef44ae590 2024-01-15T18:10:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.421+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] conn(0x557ef44ae590 0x557ef44c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.422+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] conn(0x557ef44ae590 0x557ef44c6130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3fd5b340d18d8 server_cookie=39ff2d29d7e04aee in_seq=0 out_seq=0 2024-01-15T18:10:34.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.422+0000 7f6a27e00640 10 client.8904 ms_handle_connect on v2:172.21.15.124:6836/421656183 2024-01-15T18:10:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.424+0000 7f6a27e00640 1 -- 172.21.15.26:0/1634860871 <== mds.0 v2:172.21.15.124:6836/421656183 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ef43c27c0 con 0x557ef44ae590 2024-01-15T18:10:34.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.424+0000 7f6a27e00640 10 client.8904 handle_command_reply: tid=0 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.424+0000 7f6a487b2740 1 client.8904 shutdown 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 -- 172.21.15.26:0/1634860871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 msgr2=0x557ef44a20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a20e0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800a040 tx=0x7f6a28058ab0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 -- 172.21.15.26:0/1634860871 shutdown_connections 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] conn(0x557ef44ae590 0x557ef44c6130 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef4462ea0 0x557ef44a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef44625e0 0x557ef44a20e0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1634860871 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a27e00640 0 client.8904 ms_handle_reset on v2:172.21.15.124:6836/421656183 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 -- 172.21.15.26:0/1634860871 >> 172.21.15.26:0/1634860871 conn(0x557ef4424c70 msgr2=0x557ef445cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 -- 172.21.15.26:0/1634860871 shutdown_connections 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 1 -- 172.21.15.26:0/1634860871 wait complete. 2024-01-15T18:10:34.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.425+0000 7f6a487b2740 20 client.8904 trim_cache size 0 max 16384 2024-01-15T18:10:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.442+0000 7f6a487b2740 1 Processor -- start 2024-01-15T18:10:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.442+0000 7f6a487b2740 1 -- start start 2024-01-15T18:10:34.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.443+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.443+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef44625e0 0x557ef44ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.443+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef44aef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.443+0000 7f6a487b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef412cf20 con 0x557ef4462ea0 2024-01-15T18:10:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.443+0000 7f6a487b2740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef44a3400 con 0x557ef44625e0 2024-01-15T18:10:34.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.443+0000 7f6a487b2740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ef4491370 con 0x557ef445dba0 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef44aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef44aef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54104/0 (socket says 172.21.15.26:54104) 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3f400640 1 -- 172.21.15.26:0/1287028475 learned_addr learned my addr 172.21.15.26:0/1287028475 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/1287028475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3e000640 1 --2- 172.21.15.26:0/1287028475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef44625e0 0x557ef44ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3f400640 1 -- 172.21.15.26:0/1287028475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 msgr2=0x557ef44ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3f400640 1 --2- 172.21.15.26:0/1287028475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3f400640 1 -- 172.21.15.26:0/1287028475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef44625e0 msgr2=0x557ef44ae9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3f400640 1 --2- 172.21.15.26:0/1287028475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef44625e0 0x557ef44ae9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.444+0000 7f6a3f400640 1 -- 172.21.15.26:0/1287028475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef4365b80 con 0x557ef4462ea0 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a3f400640 1 --2- 172.21.15.26:0/1287028475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef44aef00 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f6a380072a0 tx=0x7f6a38009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ea561d29d073c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a2fe00640 1 -- 172.21.15.26:0/1287028475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3802cb20 con 0x557ef4462ea0 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a2fe00640 1 -- 172.21.15.26:0/1287028475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38003990 con 0x557ef4462ea0 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a2fe00640 1 -- 172.21.15.26:0/1287028475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38003d20 con 0x557ef4462ea0 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a487b2740 1 -- 172.21.15.26:0/1287028475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 msgr2=0x557ef44aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1287028475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef44aef00 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f6a380072a0 tx=0x7f6a38009000 comp rx=0 tx=0).stop 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a487b2740 1 -- 172.21.15.26:0/1287028475 shutdown_connections 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1287028475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef44aef00 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1287028475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef44625e0 0x557ef44ae9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.445+0000 7f6a487b2740 1 --2- 172.21.15.26:0/1287028475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.446+0000 7f6a487b2740 1 -- 172.21.15.26:0/1287028475 >> 172.21.15.26:0/1287028475 conn(0x557ef4424c70 msgr2=0x557ef44a5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.446+0000 7f6a487b2740 1 -- 172.21.15.26:0/1287028475 shutdown_connections 2024-01-15T18:10:34.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.446+0000 7f6a487b2740 1 -- 172.21.15.26:0/1287028475 wait complete. 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.446+0000 7f6a487b2740 1 -- start start 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.446+0000 7f6a487b2740 1 Processor -- start 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.447+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.447+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef44625e0 0x557ef4512ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.447+0000 7f6a487b2740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef445cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.447+0000 7f6a487b2740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef445da10 con 0x557ef4462ea0 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.447+0000 7f6a487b2740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef44bf820 con 0x557ef44625e0 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.447+0000 7f6a487b2740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ef44bf9c0 con 0x557ef445dba0 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a487b2740 10 client.0 _ll_register_callbacks cb 0x7f6a28088000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef445cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a3f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef445cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54106/0 (socket says 172.21.15.26:54106) 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a3f400640 1 -- 172.21.15.26:0/2865125521 learned_addr learned my addr 172.21.15.26:0/2865125521 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a3f400640 1 -- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 msgr2=0x557ef44c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a3f400640 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44c1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a3f400640 1 -- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef44625e0 msgr2=0x557ef4512ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a3f400640 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef44625e0 0x557ef4512ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.448+0000 7f6a3f400640 1 -- 172.21.15.26:0/2865125521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ef4391860 con 0x557ef4462ea0 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a3f400640 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef445cb80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f6a38040040 tx=0x7f6a38004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bd96e1b5060f0b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a27e00640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a27e00640 1 -- 172.21.15.26:0/2865125521 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38004230 con 0x557ef4462ea0 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a27e00640 1 -- 172.21.15.26:0/2865125521 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6a38004390 con 0x557ef4462ea0 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a487b2740 1 -- 172.21.15.26:0/2865125521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a38040470 con 0x557ef4462ea0 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a27e00640 1 -- 172.21.15.26:0/2865125521 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a38003e90 con 0x557ef4462ea0 2024-01-15T18:10:34.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a487b2740 10 client.8910 fetch_fsmap learned FSMap version 63 2024-01-15T18:10:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a487b2740 1 -- 172.21.15.26:0/2865125521 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x557ef438b170 con 0x557ef4462ea0 2024-01-15T18:10:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a27e00640 1 -- 172.21.15.26:0/2865125521 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38040470 con 0x557ef4462ea0 2024-01-15T18:10:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.449+0000 7f6a27e00640 1 -- 172.21.15.26:0/2865125521 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a38050070 con 0x557ef4462ea0 2024-01-15T18:10:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.450+0000 7f6a487b2740 10 client.8910 fetch_fsmap finished waiting for FSMap version 63 2024-01-15T18:10:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.450+0000 7f6a487b2740 10 client.8910 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-15T18:10:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.450+0000 7f6a487b2740 20 client.8910 populate_metadata read hostname 'smithi026' 2024-01-15T18:10:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.450+0000 7f6a487b2740 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] conn(0x557ef44b9480 0x557ef445d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.450+0000 7f6a487b2740 4 client.8910 mds_command: new command op to 36242 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:10:34.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.450+0000 7f6a487b2740 1 -- 172.21.15.26:0/2865125521 --> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557ef40ca6c0 con 0x557ef44b9480 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.450+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] conn(0x557ef44b9480 0x557ef445d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.451+0000 7f6a3ea00640 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] conn(0x557ef44b9480 0x557ef445d0c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1096e238018c38c server_cookie=b135c871bf2109eb in_seq=0 out_seq=0 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.451+0000 7f6a27e00640 10 client.8910 ms_handle_connect on v2:172.21.15.124:6836/421656183 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.451+0000 7f6a27e00640 1 -- 172.21.15.26:0/2865125521 <== mds.0 v2:172.21.15.124:6836/421656183 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x557ef40ca6c0 con 0x557ef44b9480 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.451+0000 7f6a27e00640 10 client.8910 handle_command_reply: tid=0 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 client.8910 shutdown 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 -- 172.21.15.26:0/2865125521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 msgr2=0x557ef445cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef445cb80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f6a38040040 tx=0x7f6a38004ab0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 -- 172.21.15.26:0/2865125521 shutdown_connections 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:6836/421656183,v1:172.21.15.124:6837/421656183] conn(0x557ef44b9480 0x557ef445d0c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x557ef4462ea0 0x557ef445cb80 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x557ef44625e0 0x557ef4512ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 --2- 172.21.15.26:0/2865125521 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x557ef445dba0 0x557ef44c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a27e00640 0 client.8910 ms_handle_reset on v2:172.21.15.124:6836/421656183 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 -- 172.21.15.26:0/2865125521 >> 172.21.15.26:0/2865125521 conn(0x557ef4424c70 msgr2=0x557ef44ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 -- 172.21.15.26:0/2865125521 shutdown_connections 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 1 -- 172.21.15.26:0/2865125521 wait complete. 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.452+0000 7f6a487b2740 20 client.8910 trim_cache size 0 max 16384 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 36242, 2024-01-15T18:10:34.458 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-15T18:10:34.459 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-15T18:10:34.459 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-15T18:10:34.459 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 11.858782787000001, 2024-01-15T18:10:34.459 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 63, 2024-01-15T18:10:34.459 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 102, 2024-01-15T18:10:34.459 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 102, 2024-01-15T18:10:34.459 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 24.261094756999999 2024-01-15T18:10:34.459 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 -- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a1806c1f0 msgr2=0x7f6a1806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a1806c1f0 0x7f6a1806e6b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f6a30006fd0 tx=0x7f6a30008040 comp rx=0 tx=0).stop 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 -- 172.21.15.26:0/774239984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a4013df80 msgr2=0x7f6a40153330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a4013df80 0x7f6a40153330 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6a2800c400 tx=0x7f6a2800c8c0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 -- 172.21.15.26:0/774239984 shutdown_connections 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a4013e360 0x7f6a400a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6a1806c1f0 0x7f6a1806e6b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6a4013df80 0x7f6a40153330 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 --2- 172.21.15.26:0/774239984 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a400cf9a0 0x7f6a400b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 -- 172.21.15.26:0/774239984 >> 172.21.15.26:0/774239984 conn(0x7f6a4006a800 msgr2=0x7f6a400b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 -- 172.21.15.26:0/774239984 shutdown_connections 2024-01-15T18:10:34.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.453+0000 7f6a45c00640 1 -- 172.21.15.26:0/774239984 wait complete. 2024-01-15T18:10:34.475 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:10:34.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.681+0000 7f3a98a00640 1 Processor -- start 2024-01-15T18:10:34.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.682+0000 7f3a98a00640 1 -- start start 2024-01-15T18:10:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.683+0000 7f3a98a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3a940cc500 0x7f3a940cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.683+0000 7f3a98a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a9406d160 0x7f3a9406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.683+0000 7f3a98a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a94070e50 con 0x7f3a94143c40 2024-01-15T18:10:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.683+0000 7f3a98a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a940723c0 con 0x7f3a940cc500 2024-01-15T18:10:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.683+0000 7f3a98a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940cced0 con 0x7f3a9406d160 2024-01-15T18:10:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.684+0000 7f3a93400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3a94143c40 0x7f3a94071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52772/0 (socket says 172.21.15.26:52772) 2024-01-15T18:10:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.684+0000 7f3a93400640 1 -- 172.21.15.26:0/1189887750 learned_addr learned my addr 172.21.15.26:0/1189887750 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:34.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.684+0000 7f3a92a00640 1 --2- 172.21.15.26:0/1189887750 >> v2:172.21.15.26:3300/0 conn(0x7f3a940cc500 0x7f3a940cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.684+0000 7f3a93e00640 1 --2- 172.21.15.26:0/1189887750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a9406d160 0x7f3a9406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.684+0000 7f3a92a00640 1 -- 172.21.15.26:0/1189887750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a9406d160 msgr2=0x7f3a9406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:34.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.684+0000 7f3a92a00640 1 --2- 172.21.15.26:0/1189887750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a9406d160 0x7f3a9406d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.684+0000 7f3a92a00640 1 -- 172.21.15.26:0/1189887750 >> v1:172.21.15.26:6789/0 conn(0x7f3a94143c40 legacy=0x7f3a94071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.684+0000 7f3a92a00640 1 -- 172.21.15.26:0/1189887750 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94144020 con 0x7f3a940cc500 2024-01-15T18:10:34.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.686+0000 7f3a92a00640 1 --2- 172.21.15.26:0/1189887750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a940cc500 0x7f3a940cc990 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0097a0 tx=0x7f3a7c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0b8f676fbd2fb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.686+0000 7f3a92000640 1 -- 172.21.15.26:0/1189887750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c031780 con 0x7f3a940cc500 2024-01-15T18:10:34.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.686+0000 7f3a92000640 1 -- 172.21.15.26:0/1189887750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c0318e0 con 0x7f3a940cc500 2024-01-15T18:10:34.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.686+0000 7f3a92000640 1 -- 172.21.15.26:0/1189887750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c031c70 con 0x7f3a940cc500 2024-01-15T18:10:34.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.687+0000 7f3a98a00640 1 -- 172.21.15.26:0/1189887750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a940cc500 msgr2=0x7f3a940cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.687+0000 7f3a98a00640 1 --2- 172.21.15.26:0/1189887750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a940cc500 0x7f3a940cc990 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0097a0 tx=0x7f3a7c02b820 comp rx=0 tx=0).stop 2024-01-15T18:10:34.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.687+0000 7f3a98a00640 1 -- 172.21.15.26:0/1189887750 shutdown_connections 2024-01-15T18:10:34.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.687+0000 7f3a98a00640 1 --2- 172.21.15.26:0/1189887750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a9406d160 0x7f3a9406d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.687+0000 7f3a98a00640 1 --2- 172.21.15.26:0/1189887750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a940cc500 0x7f3a940cc990 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.687+0000 7f3a98a00640 1 -- 172.21.15.26:0/1189887750 >> 172.21.15.26:0/1189887750 conn(0x7f3a9406a800 msgr2=0x7f3a9406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.687+0000 7f3a98a00640 1 -- 172.21.15.26:0/1189887750 shutdown_connections 2024-01-15T18:10:34.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.687+0000 7f3a98a00640 1 -- 172.21.15.26:0/1189887750 wait complete. 2024-01-15T18:10:34.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.689+0000 7f3a98a00640 1 Processor -- start 2024-01-15T18:10:34.692 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.689+0000 7f3a98a00640 1 -- start start 2024-01-15T18:10:34.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.690+0000 7f3a98a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a9406d160 0x7f3a941e7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.690+0000 7f3a98a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a940cc500 0x7f3a941e0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.690+0000 7f3a98a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a94143c40 0x7f3a941f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.690+0000 7f3a98a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940aaf10 con 0x7f3a9406d160 2024-01-15T18:10:34.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.690+0000 7f3a98a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940ac3b0 con 0x7f3a94143c40 2024-01-15T18:10:34.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.690+0000 7f3a98a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940ac600 con 0x7f3a940cc500 2024-01-15T18:10:34.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.691+0000 7f3a93400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a9406d160 0x7f3a941e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.691+0000 7f3a93400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a9406d160 0x7f3a941e7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54122/0 (socket says 172.21.15.26:54122) 2024-01-15T18:10:34.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.691+0000 7f3a93400640 1 -- 172.21.15.26:0/2439968007 learned_addr learned my addr 172.21.15.26:0/2439968007 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:34.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.691+0000 7f3a93400640 1 -- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a940cc500 msgr2=0x7f3a941e0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.691+0000 7f3a93400640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a940cc500 0x7f3a941e0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.691+0000 7f3a93400640 1 -- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a94143c40 msgr2=0x7f3a941f1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.691+0000 7f3a93400640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a94143c40 0x7f3a941f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.691+0000 7f3a93400640 1 -- 172.21.15.26:0/2439968007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c009400 con 0x7f3a9406d160 2024-01-15T18:10:34.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.691+0000 7f3a93400640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a9406d160 0x7f3a941e7200 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c400 tx=0x7f3a8400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.693+0000 7f3a8b400640 1 -- 172.21.15.26:0/2439968007 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84005ce0 con 0x7f3a9406d160 2024-01-15T18:10:34.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.693+0000 7f3a8b400640 1 -- 172.21.15.26:0/2439968007 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84005e40 con 0x7f3a9406d160 2024-01-15T18:10:34.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.693+0000 7f3a8b400640 1 -- 172.21.15.26:0/2439968007 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84007310 con 0x7f3a9406d160 2024-01-15T18:10:34.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.693+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a940ac8e0 con 0x7f3a9406d160 2024-01-15T18:10:34.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.693+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a940ad180 con 0x7f3a9406d160 2024-01-15T18:10:34.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.694+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a940acad0 con 0x7f3a9406d160 2024-01-15T18:10:34.698 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.695+0000 7f3a8b400640 1 -- 172.21.15.26:0/2439968007 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f3a84007470 con 0x7f3a9406d160 2024-01-15T18:10:34.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.696+0000 7f3a8b400640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a6c06c1f0 0x7f3a6c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:34.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.696+0000 7f3a8b400640 1 -- 172.21.15.26:0/2439968007 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3a84087c80 con 0x7f3a9406d160 2024-01-15T18:10:34.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.697+0000 7f3a92a00640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a6c06c1f0 0x7f3a6c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:34.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.698+0000 7f3a92a00640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a6c06c1f0 0x7f3a6c06e6b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c036000 tx=0x7f3a7c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:34.704 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.701+0000 7f3a8b400640 1 -- 172.21.15.26:0/2439968007 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a84054540 con 0x7f3a9406d160 2024-01-15T18:10:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.935+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3a940accd0 con 0x7f3a9406d160 2024-01-15T18:10:34.939 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.936+0000 7f3a8b400640 1 -- 172.21.15.26:0/2439968007 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3a84017070 con 0x7f3a9406d160 2024-01-15T18:10:34.940 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:10:34.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a6c06c1f0 msgr2=0x7f3a6c06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a6c06c1f0 0x7f3a6c06e6b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c036000 tx=0x7f3a7c030000 comp rx=0 tx=0).stop 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a9406d160 msgr2=0x7f3a941e7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a9406d160 0x7f3a941e7200 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c400 tx=0x7f3a8400c8c0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 shutdown_connections 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3a940cc500 0x7f3a941e0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3a6c06c1f0 0x7f3a6c06e6b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3a94143c40 0x7f3a941f1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 --2- 172.21.15.26:0/2439968007 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3a9406d160 0x7f3a941e7200 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 >> 172.21.15.26:0/2439968007 conn(0x7f3a9406a800 msgr2=0x7f3a940af3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 shutdown_connections 2024-01-15T18:10:34.943 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:34.939+0000 7f3a98a00640 1 -- 172.21.15.26:0/2439968007 wait complete. 2024-01-15T18:10:34.961 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:10:34.962 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:10:34.962 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-15T18:10:34.962 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:10:34.962 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:10:34.962 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:10:34.962 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:10:34.962 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:10:34.962 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:10:34.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:34.962 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-15T18:10:35.017 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:10:35.017 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:10:35.017 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:10:35.017 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:35.017 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T18:10:35.017 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:35.017 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:10:35.017 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:35.017 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41683sec preferred_lft 41683sec 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:35.018 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::a4e5:3ff:fe91:fcad/64 scope link 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::f464:8fff:fecd:58d2/64 scope link 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether fe:82:c8:bc:3f:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::fc82:c8ff:febc:3fd2/64 scope link 2024-01-15T18:10:35.019 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:35.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:35.021 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:10:35.114 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-15T18:10:35.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:35.118 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-15T18:10:35.173 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-15T18:10:35.174 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:10:35.174 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-15T18:10:35.174 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-15T18:10:35.174 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:10:35.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:35.176 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T18:10:35.176 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T18:10:35.176 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-15T18:10:35.176 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-15T18:10:35.177 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-15T18:10:35.177 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T18:10:35.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:35.371 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T18:10:35.423 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:10:35.423 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:10:35.424 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:10:35.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:35.425 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-15T18:10:35.525 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:10:35.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:10:35.525 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:10:35.580 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:10:35.581 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:10:35.581 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:10:35.635 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:10:35.735 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:10:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:10:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:10:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:10:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:10:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:10:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:10:35.792 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:10:35.793 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:10:35.794 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:10:35.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:35.795 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:10:35.847 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:10:35.847 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:10:35.855 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:10:35.937 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:10:35.981 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:10:35.981 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:10:35.982 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:10:35.983 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:10:35.984 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:10:35.984 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:10:35.984 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:10:35.984 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:10:35.984 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:10:35.984 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:10:35.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:35.985 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:10:36.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:2024-01-15T18:10:36.051+0000 7f7e3aa0b480 -1 init, newargv = 0x5633bec3a890 newargc=15 2024-01-15T18:10:36.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[73102]: starting ceph client 2024-01-15T18:10:36.064 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[73102]: starting fuse 2024-01-15T18:10:36.072 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T18:10:36.072 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-01-15T18:10:36.073 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.0")) 2024-01-15T18:10:36.074 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-01-15T18:10:36.289 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.0.73102.asok 2024-01-15T18:10:36.294 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:10:36.308 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.73102.asok 2024-01-15T18:10:36.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:36.308 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.73102.asok status 2024-01-15T18:10:36.559 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:10:36.559 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-01-15T18:10:36.559 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:10:36.559 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:10:36.559 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "0", 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "73102", 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 8922, 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 8922 2024-01-15T18:10:36.560 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 309705251 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 309705251 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.8922 192.168.0.1:0/309705251", 2024-01-15T18:10:36.561 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/309705251", 2024-01-15T18:10:36.562 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-01-15T18:10:36.562 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 63, 2024-01-15T18:10:36.562 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 102, 2024-01-15T18:10:36.562 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:10:36.562 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-01-15T18:10:36.562 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-01-15T18:10:36.562 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:10:36.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:36.573 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-01-15T18:10:36.573 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:10:36.629 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-01-15T18:10:36.629 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:10:36.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:36.629 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:10:36.730 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:10:36.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.947+0000 7f572be00640 1 Processor -- start 2024-01-15T18:10:36.951 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.947+0000 7f572be00640 1 -- start start 2024-01-15T18:10:36.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.949+0000 7f572be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f572c0cc500 0x7f572c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:36.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.949+0000 7f572be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f572c06d160 0x7f572c06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:36.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.949+0000 7f572be00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572c070e50 con 0x7f572c143c40 2024-01-15T18:10:36.952 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.949+0000 7f572be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572c0723c0 con 0x7f572c06d160 2024-01-15T18:10:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.949+0000 7f572be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572c0cced0 con 0x7f572c0cc500 2024-01-15T18:10:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.949+0000 7f572b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f572c06d160 0x7f572c06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.949+0000 7f572b400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f572c06d160 0x7f572c06d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54126/0 (socket says 172.21.15.26:54126) 2024-01-15T18:10:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.949+0000 7f572b400640 1 -- 172.21.15.26:0/1425422179 learned_addr learned my addr 172.21.15.26:0/1425422179 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.950+0000 7f5721e00640 1 --2- 172.21.15.26:0/1425422179 >> v2:172.21.15.124:3300/0 conn(0x7f572c0cc500 0x7f572c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:36.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.950+0000 7f572b400640 1 -- 172.21.15.26:0/1425422179 >> v2:172.21.15.124:3300/0 conn(0x7f572c0cc500 msgr2=0x7f572c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.950+0000 7f572b400640 1 --2- 172.21.15.26:0/1425422179 >> v2:172.21.15.124:3300/0 conn(0x7f572c0cc500 0x7f572c0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.950+0000 7f572b400640 1 -- 172.21.15.26:0/1425422179 >> v1:172.21.15.124:6789/0 conn(0x7f572c143c40 legacy=0x7f572c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.950+0000 7f572b400640 1 -- 172.21.15.26:0/1425422179 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c144020 con 0x7f572c06d160 2024-01-15T18:10:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.951+0000 7f572b400640 1 --2- 172.21.15.26:0/1425422179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c06d160 0x7f572c06d560 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f572c0b44a0 tx=0x7f572402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d6fcd9f58b072e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.951+0000 7f572a000640 1 -- 172.21.15.26:0/1425422179 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572402c9f0 con 0x7f572c06d160 2024-01-15T18:10:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.951+0000 7f572a000640 1 -- 172.21.15.26:0/1425422179 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572402cb50 con 0x7f572c06d160 2024-01-15T18:10:36.954 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.951+0000 7f572a000640 1 -- 172.21.15.26:0/1425422179 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57240343f0 con 0x7f572c06d160 2024-01-15T18:10:36.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.952+0000 7f572be00640 1 -- 172.21.15.26:0/1425422179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c06d160 msgr2=0x7f572c06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:36.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.952+0000 7f572be00640 1 --2- 172.21.15.26:0/1425422179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c06d160 0x7f572c06d560 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f572c0b44a0 tx=0x7f572402be50 comp rx=0 tx=0).stop 2024-01-15T18:10:36.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.952+0000 7f572be00640 1 -- 172.21.15.26:0/1425422179 shutdown_connections 2024-01-15T18:10:36.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.952+0000 7f572be00640 1 --2- 172.21.15.26:0/1425422179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c06d160 0x7f572c06d560 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.952+0000 7f572be00640 1 --2- 172.21.15.26:0/1425422179 >> v2:172.21.15.124:3300/0 conn(0x7f572c0cc500 0x7f572c0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.952+0000 7f572be00640 1 -- 172.21.15.26:0/1425422179 >> 172.21.15.26:0/1425422179 conn(0x7f572c06a800 msgr2=0x7f572c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.952+0000 7f572be00640 1 -- 172.21.15.26:0/1425422179 shutdown_connections 2024-01-15T18:10:36.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.952+0000 7f572be00640 1 -- 172.21.15.26:0/1425422179 wait complete. 2024-01-15T18:10:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.954+0000 7f572be00640 1 Processor -- start 2024-01-15T18:10:36.957 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.954+0000 7f572be00640 1 -- start start 2024-01-15T18:10:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.955+0000 7f572be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f572c06d160 0x7f572c0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.955+0000 7f572be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c0cc500 0x7f572c0c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.955+0000 7f572be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c143c40 0x7f572c0c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.955+0000 7f572be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c0c93d0 con 0x7f572c143c40 2024-01-15T18:10:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.955+0000 7f572be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572c0c96b0 con 0x7f572c0cc500 2024-01-15T18:10:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.955+0000 7f572be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572c0c9990 con 0x7f572c06d160 2024-01-15T18:10:36.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.955+0000 7f572b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c143c40 0x7f572c0c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.955+0000 7f572b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c143c40 0x7f572c0c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54142/0 (socket says 172.21.15.26:54142) 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.955+0000 7f572b400640 1 -- 172.21.15.26:0/2977121218 learned_addr learned my addr 172.21.15.26:0/2977121218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f5721e00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c0cc500 0x7f572c0c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f572aa00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f572c06d160 0x7f572c0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f572b400640 1 -- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f572c06d160 msgr2=0x7f572c0ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f572b400640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f572c06d160 0x7f572c0ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f572b400640 1 -- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c0cc500 msgr2=0x7f572c0c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f572b400640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c0cc500 0x7f572c0c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f572b400640 1 -- 172.21.15.26:0/2977121218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5724009b30 con 0x7f572c143c40 2024-01-15T18:10:36.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f572b400640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c143c40 0x7f572c0c8b90 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5724036040 tx=0x7f5724004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f5723400640 1 -- 172.21.15.26:0/2977121218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572403e450 con 0x7f572c143c40 2024-01-15T18:10:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f5723400640 1 -- 172.21.15.26:0/2977121218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572403e5d0 con 0x7f572c143c40 2024-01-15T18:10:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f5723400640 1 -- 172.21.15.26:0/2977121218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572403e980 con 0x7f572c143c40 2024-01-15T18:10:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.956+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c0c9c70 con 0x7f572c143c40 2024-01-15T18:10:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.957+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572c0caab0 con 0x7f572c143c40 2024-01-15T18:10:36.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.958+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572c067450 con 0x7f572c143c40 2024-01-15T18:10:36.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.959+0000 7f5723400640 1 -- 172.21.15.26:0/2977121218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f572403eae0 con 0x7f572c143c40 2024-01-15T18:10:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.960+0000 7f5723400640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f570406c1f0 0x7f570406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.960+0000 7f5723400640 1 -- 172.21.15.26:0/2977121218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f57240ab6c0 con 0x7f572c143c40 2024-01-15T18:10:36.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.960+0000 7f572aa00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f570406c1f0 0x7f570406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:36.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.961+0000 7f572aa00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f570406c1f0 0x7f570406e6b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f571c00a750 tx=0x7f571c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:36.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:36.964+0000 7f5723400640 1 -- 172.21.15.26:0/2977121218 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5724077f30 con 0x7f572c143c40 2024-01-15T18:10:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.200+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f572c0ae390 con 0x7f572c143c40 2024-01-15T18:10:37.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.201+0000 7f5723400640 1 -- 172.21.15.26:0/2977121218 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5724043060 con 0x7f572c143c40 2024-01-15T18:10:37.205 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:10:37.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f570406c1f0 msgr2=0x7f570406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f570406c1f0 0x7f570406e6b0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f571c00a750 tx=0x7f571c008040 comp rx=0 tx=0).stop 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c143c40 msgr2=0x7f572c0c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c143c40 0x7f572c0c8b90 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5724036040 tx=0x7f5724004a40 comp rx=0 tx=0).stop 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 shutdown_connections 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f570406c1f0 0x7f570406e6b0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f572c143c40 0x7f572c0c8b90 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f572c0cc500 0x7f572c0c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 --2- 172.21.15.26:0/2977121218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f572c06d160 0x7f572c0ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 >> 172.21.15.26:0/2977121218 conn(0x7f572c06a800 msgr2=0x7f572c071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 shutdown_connections 2024-01-15T18:10:37.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:37.204+0000 7f572be00640 1 -- 172.21.15.26:0/2977121218 wait complete. 2024-01-15T18:10:37.228 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:10:37.228 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:10:37.228 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2024-01-15T18:10:37.228 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:10:37.228 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:10:37.228 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:10:37.228 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:10:37.228 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:10:37.228 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:10:37.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:37.229 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2024-01-15T18:10:37.285 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:10:37.285 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:10:37.285 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:37.286 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 41678sec preferred_lft 41678sec 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4c2d:94ff:fe47:f54c/64 scope link 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 26:4f:4f:5b:0b:31 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:10:37.287 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::244f:4fff:fe5b:b31/64 scope link 2024-01-15T18:10:37.288 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:37.288 INFO:teuthology.orchestra.run.smithi124.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:10:37.288 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:10:37.288 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::2070:16ff:fe92:4a4a/64 scope link 2024-01-15T18:10:37.288 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:10:37.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:37.289 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:10:37.378 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T18:10:37.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:37.383 DEBUG:teuthology.orchestra.run.smithi124:> route 2024-01-15T18:10:37.435 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2024-01-15T18:10:37.435 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:10:37.435 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:10:37.435 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:10:37.435 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:10:37.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:37.437 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T18:10:37.437 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T18:10:37.437 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:10:37.437 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:10:37.437 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:10:37.437 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T18:10:37.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:37.613 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T18:10:37.663 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:10:37.663 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:10:37.664 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:10:37.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:37.664 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2024-01-15T18:10:37.750 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:10:37.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:10:37.750 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:10:37.803 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:10:37.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:10:37.803 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:10:37.852 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:10:37.946 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:10:38.001 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:10:38.001 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:10:38.002 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:10:38.003 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:10:38.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:38.005 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:10:38.056 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:10:38.056 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:10:38.063 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:10:38.141 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:10:38.180 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:10:38.180 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:10:38.180 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:10:38.180 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:10:38.180 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:10:38.180 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:10:38.180 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:10:38.181 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:10:38.182 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:10:38.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:38.183 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:10:38.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:2024-01-15T18:10:38.247+0000 7ffa4f66a480 -1 init, newargv = 0x563691457980 newargc=15 2024-01-15T18:10:38.251 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[47374]: starting ceph client 2024-01-15T18:10:38.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[47374]: starting fuse 2024-01-15T18:10:38.269 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T18:10:38.270 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> import glob 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> import re 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> import os 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> import subprocess 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> def _find_admin_socket(client_name): 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> files = glob.glob(asok_path) 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> # Given a non-glob path, it better be there 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> if "*" not in asok_path: 2024-01-15T18:10:38.270 DEBUG:teuthology.orchestra.run.smithi124:> assert(len(files) == 1) 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> return files[0] 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> for f in files: 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> contents = proc_f.read() 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> if mountpoint in contents: 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> return f 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> print(_find_admin_socket("client.1")) 2024-01-15T18:10:38.271 DEBUG:teuthology.orchestra.run.smithi124:> ' 2024-01-15T18:10:38.466 INFO:teuthology.orchestra.run.smithi124.stdout:/var/run/ceph/ceph-client.1.47374.asok 2024-01-15T18:10:38.470 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T18:10:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:10:38.483 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.47374.asok 2024-01-15T18:10:38.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:38.483 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.47374.asok status 2024-01-15T18:10:38.719 INFO:teuthology.orchestra.run.smithi124.stdout:{ 2024-01-15T18:10:38.719 INFO:teuthology.orchestra.run.smithi124.stdout: "metadata": { 2024-01-15T18:10:38.719 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:10:38.719 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:10:38.719 INFO:teuthology.orchestra.run.smithi124.stdout: "entity_id": "1", 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "hostname": "smithi124", 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "pid": "47374", 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "root": "/" 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_count": 0, 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_pinned_count": 0, 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "id": 7114, 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "inst": { 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "name": { 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "client", 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "num": 7114 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:10:38.720 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 3336021130 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: } 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 3336021130 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "inst_str": "client.7114 192.168.0.1:0/3336021130", 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "addr_str": "192.168.0.1:0/3336021130", 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "inode_count": 1, 2024-01-15T18:10:38.721 INFO:teuthology.orchestra.run.smithi124.stdout: "mds_epoch": 63, 2024-01-15T18:10:38.722 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch": 102, 2024-01-15T18:10:38.722 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:10:38.722 INFO:teuthology.orchestra.run.smithi124.stdout: "blocklisted": false, 2024-01-15T18:10:38.722 INFO:teuthology.orchestra.run.smithi124.stdout: "fs_name": "cephfs" 2024-01-15T18:10:38.722 INFO:teuthology.orchestra.run.smithi124.stdout:} 2024-01-15T18:10:38.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:38.734 DEBUG:teuthology.orchestra.run.smithi124:> stat --file-system '--printf=%T 2024-01-15T18:10:38.734 DEBUG:teuthology.orchestra.run.smithi124:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:10:38.787 INFO:teuthology.orchestra.run.smithi124.stdout:fuseblk 2024-01-15T18:10:38.787 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:10:38.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:38.787 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:10:38.877 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:10:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.108+0000 7f2ed1600640 1 Processor -- start 2024-01-15T18:10:39.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.109+0000 7f2ed1600640 1 -- start start 2024-01-15T18:10:39.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.110+0000 7f2ed1600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2ecc071410 0x7f2ecc146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.110+0000 7f2ed1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc06c7c0 0x7f2ecc0717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.110+0000 7f2ed1600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ecc06cc20 con 0x7f2ecc071d30 2024-01-15T18:10:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.110+0000 7f2ed1600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ecc06cec0 con 0x7f2ecc071410 2024-01-15T18:10:39.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.110+0000 7f2ed1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc06d1a0 con 0x7f2ecc06c7c0 2024-01-15T18:10:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.111+0000 7f2ecbe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2ecc071410 0x7f2ecc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.111+0000 7f2ecb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc06c7c0 0x7f2ecc0717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.111+0000 7f2ecbe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2ecc071410 0x7f2ecc146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:60450/0 (socket says 172.21.15.26:60450) 2024-01-15T18:10:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.111+0000 7f2ecbe00640 1 -- 172.21.15.26:0/437292947 learned_addr learned my addr 172.21.15.26:0/437292947 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.111+0000 7f2ecb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc06c7c0 0x7f2ecc0717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:41546/0 (socket says 172.21.15.26:41546) 2024-01-15T18:10:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.111+0000 7f2ed0c00640 1 --1- 172.21.15.26:0/437292947 >> v1:172.21.15.124:6789/0 conn(0x7f2ecc071d30 0x7f2ecc0b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:37684/0 (socket says 172.21.15.26:37684) 2024-01-15T18:10:39.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.112+0000 7f2ecbe00640 1 -- 172.21.15.26:0/437292947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc06c7c0 msgr2=0x7f2ecc0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.112+0000 7f2ecbe00640 1 --2- 172.21.15.26:0/437292947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc06c7c0 0x7f2ecc0717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.112+0000 7f2ecbe00640 1 -- 172.21.15.26:0/437292947 >> v1:172.21.15.124:6789/0 conn(0x7f2ecc071d30 legacy=0x7f2ecc0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:39.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.112+0000 7f2ecbe00640 1 -- 172.21.15.26:0/437292947 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ecc06d480 con 0x7f2ecc071410 2024-01-15T18:10:39.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.114+0000 7f2ecbe00640 1 --2- 172.21.15.26:0/437292947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ecc071410 0x7f2ecc146770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009870 tx=0x7f2ebc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cf251f8fdffd4f9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.115+0000 7f2ecaa00640 1 -- 172.21.15.26:0/437292947 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc030840 con 0x7f2ecc071410 2024-01-15T18:10:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.115+0000 7f2ecaa00640 1 -- 172.21.15.26:0/437292947 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc0309a0 con 0x7f2ecc071410 2024-01-15T18:10:39.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.115+0000 7f2ecaa00640 1 -- 172.21.15.26:0/437292947 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc030d30 con 0x7f2ecc071410 2024-01-15T18:10:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.115+0000 7f2ed1600640 1 -- 172.21.15.26:0/437292947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ecc071410 msgr2=0x7f2ecc146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.115+0000 7f2ed1600640 1 --2- 172.21.15.26:0/437292947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ecc071410 0x7f2ecc146770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009870 tx=0x7f2ebc02b980 comp rx=0 tx=0).stop 2024-01-15T18:10:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.116+0000 7f2ed1600640 1 -- 172.21.15.26:0/437292947 shutdown_connections 2024-01-15T18:10:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.116+0000 7f2ed1600640 1 --2- 172.21.15.26:0/437292947 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc06c7c0 0x7f2ecc0717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.116+0000 7f2ed1600640 1 --2- 172.21.15.26:0/437292947 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ecc071410 0x7f2ecc146770 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.116+0000 7f2ed1600640 1 -- 172.21.15.26:0/437292947 >> 172.21.15.26:0/437292947 conn(0x7f2ecc06a800 msgr2=0x7f2ecc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.116+0000 7f2ed1600640 1 -- 172.21.15.26:0/437292947 shutdown_connections 2024-01-15T18:10:39.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.116+0000 7f2ed1600640 1 -- 172.21.15.26:0/437292947 wait complete. 2024-01-15T18:10:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.118+0000 7f2ed1600640 1 Processor -- start 2024-01-15T18:10:39.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.118+0000 7f2ed1600640 1 -- start start 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.119+0000 7f2ed1600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ecc06c7c0 0x7f2ecc15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ed1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc071410 0x7f2ecc144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ed1600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc071d30 0x7f2ecc1446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ed1600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc132050 con 0x7f2ecc071410 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ed1600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc140f90 con 0x7f2ecc06c7c0 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ed1600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ecc0c4f80 con 0x7f2ecc071d30 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ecb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc071410 0x7f2ecc144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ecb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc071410 0x7f2ecc144170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54158/0 (socket says 172.21.15.26:54158) 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ecb400640 1 -- 172.21.15.26:0/3388554512 learned_addr learned my addr 172.21.15.26:0/3388554512 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ecbe00640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ecc06c7c0 0x7f2ecc15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ed0c00640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc071d30 0x7f2ecc1446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ecb400640 1 -- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc071d30 msgr2=0x7f2ecc1446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ecb400640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc071d30 0x7f2ecc1446b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ecb400640 1 -- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ecc06c7c0 msgr2=0x7f2ecc15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ecb400640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ecc06c7c0 0x7f2ecc15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.120+0000 7f2ecb400640 1 -- 172.21.15.26:0/3388554512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0094d0 con 0x7f2ecc071410 2024-01-15T18:10:39.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.121+0000 7f2ecb400640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc071410 0x7f2ecc144170 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400e9b0 tx=0x7f2eb400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.121+0000 7f2ec3e00640 1 -- 172.21.15.26:0/3388554512 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4009670 con 0x7f2ecc071410 2024-01-15T18:10:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.121+0000 7f2ec3e00640 1 -- 172.21.15.26:0/3388554512 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb400c9a0 con 0x7f2ecc071410 2024-01-15T18:10:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.121+0000 7f2ec3e00640 1 -- 172.21.15.26:0/3388554512 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400cd50 con 0x7f2ecc071410 2024-01-15T18:10:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.121+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ecc131cb0 con 0x7f2ecc071410 2024-01-15T18:10:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.121+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ecc0c4ca0 con 0x7f2ecc071410 2024-01-15T18:10:39.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.122+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ecc132190 con 0x7f2ecc071410 2024-01-15T18:10:39.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.124+0000 7f2ec3e00640 1 -- 172.21.15.26:0/3388554512 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f2eb4013470 con 0x7f2ecc071410 2024-01-15T18:10:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.125+0000 7f2ec3e00640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ea406c1f0 0x7f2ea406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.125+0000 7f2ec3e00640 1 -- 172.21.15.26:0/3388554512 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2eb40888f0 con 0x7f2ecc071410 2024-01-15T18:10:39.128 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.125+0000 7f2ecbe00640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ea406c1f0 0x7f2ea406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.126+0000 7f2ecbe00640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ea406c1f0 0x7f2ea406e6b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc036000 tx=0x7f2ebc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:39.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.129+0000 7f2ec3e00640 1 -- 172.21.15.26:0/3388554512 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2eb40551b0 con 0x7f2ecc071410 2024-01-15T18:10:39.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.363+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2ecc0717f0 con 0x7f2ecc071410 2024-01-15T18:10:39.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.713+0000 7f2ec3e00640 1 -- 172.21.15.26:0/3388554512 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 136+0+0 (secure 0 0 0) 0x7f2eb40587d0 con 0x7f2ecc071410 2024-01-15T18:10:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.716+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ea406c1f0 msgr2=0x7f2ea406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.716+0000 7f2ed1600640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ea406c1f0 0x7f2ea406e6b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc036000 tx=0x7f2ebc008030 comp rx=0 tx=0).stop 2024-01-15T18:10:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.716+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc071410 msgr2=0x7f2ecc144170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:39.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.716+0000 7f2ed1600640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc071410 0x7f2ecc144170 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400e9b0 tx=0x7f2eb400ee70 comp rx=0 tx=0).stop 2024-01-15T18:10:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.717+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 shutdown_connections 2024-01-15T18:10:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.717+0000 7f2ed1600640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ecc071d30 0x7f2ecc1446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.717+0000 7f2ed1600640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ea406c1f0 0x7f2ea406e6b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.717+0000 7f2ed1600640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ecc071410 0x7f2ecc144170 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.717+0000 7f2ed1600640 1 --2- 172.21.15.26:0/3388554512 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ecc06c7c0 0x7f2ecc15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.717+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 >> 172.21.15.26:0/3388554512 conn(0x7f2ecc06a800 msgr2=0x7f2ecc0b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.717+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 shutdown_connections 2024-01-15T18:10:39.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.717+0000 7f2ed1600640 1 -- 172.21.15.26:0/3388554512 wait complete. 2024-01-15T18:10:39.744 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:10:39.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.959+0000 7ffafb200640 1 Processor -- start 2024-01-15T18:10:39.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.959+0000 7ffafb200640 1 -- start start 2024-01-15T18:10:39.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.961+0000 7ffafb200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ffafc143c40 0x7ffafc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.961+0000 7ffafb200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffafc0ccc10 0x7ffafc06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.961+0000 7ffafb200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc070e50 con 0x7ffafc0717d0 2024-01-15T18:10:39.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.961+0000 7ffafb200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc0723c0 con 0x7ffafc0ccc10 2024-01-15T18:10:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.961+0000 7ffafb200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffafc144020 con 0x7ffafc143c40 2024-01-15T18:10:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.961+0000 7ffafa800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffafc0ccc10 0x7ffafc06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.961+0000 7ffafa800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ffafc0ccc10 0x7ffafc06d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54168/0 (socket says 172.21.15.26:54168) 2024-01-15T18:10:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.961+0000 7ffafa800640 1 -- 172.21.15.26:0/3236794295 learned_addr learned my addr 172.21.15.26:0/3236794295 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.961+0000 7ffaf9e00640 1 --2- 172.21.15.26:0/3236794295 >> v2:172.21.15.124:3300/0 conn(0x7ffafc143c40 0x7ffafc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.962+0000 7ffafa800640 1 -- 172.21.15.26:0/3236794295 >> v2:172.21.15.124:3300/0 conn(0x7ffafc143c40 msgr2=0x7ffafc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:39.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.962+0000 7ffafa800640 1 --2- 172.21.15.26:0/3236794295 >> v2:172.21.15.124:3300/0 conn(0x7ffafc143c40 0x7ffafc071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.962+0000 7ffafa800640 1 -- 172.21.15.26:0/3236794295 >> v1:172.21.15.124:6789/0 conn(0x7ffafc0717d0 legacy=0x7ffafc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:39.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.962+0000 7ffafa800640 1 -- 172.21.15.26:0/3236794295 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc071e00 con 0x7ffafc0ccc10 2024-01-15T18:10:39.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.963+0000 7ffafa800640 1 --2- 172.21.15.26:0/3236794295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 0x7ffafc06d160 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ffafc06e410 tx=0x7ffaec02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9456332900cba0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:39.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.963+0000 7ffaf8a00640 1 -- 172.21.15.26:0/3236794295 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec02c9f0 con 0x7ffafc0ccc10 2024-01-15T18:10:39.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.963+0000 7ffaf8a00640 1 -- 172.21.15.26:0/3236794295 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaec02cb50 con 0x7ffafc0ccc10 2024-01-15T18:10:39.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.963+0000 7ffaf8a00640 1 -- 172.21.15.26:0/3236794295 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0343f0 con 0x7ffafc0ccc10 2024-01-15T18:10:39.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.964+0000 7ffafb200640 1 -- 172.21.15.26:0/3236794295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 msgr2=0x7ffafc06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:39.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.964+0000 7ffafb200640 1 --2- 172.21.15.26:0/3236794295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 0x7ffafc06d160 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ffafc06e410 tx=0x7ffaec02be50 comp rx=0 tx=0).stop 2024-01-15T18:10:39.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.964+0000 7ffafb200640 1 -- 172.21.15.26:0/3236794295 shutdown_connections 2024-01-15T18:10:39.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.964+0000 7ffafb200640 1 --2- 172.21.15.26:0/3236794295 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 0x7ffafc06d160 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.964+0000 7ffafb200640 1 --2- 172.21.15.26:0/3236794295 >> v2:172.21.15.124:3300/0 conn(0x7ffafc143c40 0x7ffafc071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.964+0000 7ffafb200640 1 -- 172.21.15.26:0/3236794295 >> 172.21.15.26:0/3236794295 conn(0x7ffafc06a800 msgr2=0x7ffafc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:39.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.964+0000 7ffafb200640 1 -- 172.21.15.26:0/3236794295 shutdown_connections 2024-01-15T18:10:39.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.964+0000 7ffafb200640 1 -- 172.21.15.26:0/3236794295 wait complete. 2024-01-15T18:10:39.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.966+0000 7ffafb200640 1 Processor -- start 2024-01-15T18:10:39.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.966+0000 7ffafb200640 1 -- start start 2024-01-15T18:10:39.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.967+0000 7ffafb200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffafc0717d0 0x7ffafc0ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.967+0000 7ffafb200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 0x7ffafc0a9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.967+0000 7ffafb200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc143c40 0x7ffafc0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.967+0000 7ffafb200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0aa500 con 0x7ffafc0ccc10 2024-01-15T18:10:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.967+0000 7ffafb200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0a9670 con 0x7ffafc143c40 2024-01-15T18:10:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.967+0000 7ffafb200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc0a9880 con 0x7ffafc0717d0 2024-01-15T18:10:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.967+0000 7ffaf9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 0x7ffafc0a9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.967+0000 7ffaf9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 0x7ffafc0a9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54182/0 (socket says 172.21.15.26:54182) 2024-01-15T18:10:39.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.967+0000 7ffaf9400640 1 -- 172.21.15.26:0/2426869430 learned_addr learned my addr 172.21.15.26:0/2426869430 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.968+0000 7ffaf9400640 1 -- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffafc0717d0 msgr2=0x7ffafc0ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.968+0000 7ffaf9e00640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffafc0717d0 0x7ffafc0ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.968+0000 7ffafa800640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc143c40 0x7ffafc0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.968+0000 7ffaf9400640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffafc0717d0 0x7ffafc0ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.968+0000 7ffaf9400640 1 -- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc143c40 msgr2=0x7ffafc0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.968+0000 7ffaf9400640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc143c40 0x7ffafc0af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.968+0000 7ffaf9400640 1 -- 172.21.15.26:0/2426869430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec009b30 con 0x7ffafc0ccc10 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.968+0000 7ffaf9400640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 0x7ffafc0a9130 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7ffae400b3e0 tx=0x7ffae400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.968+0000 7ffaf2000640 1 -- 172.21.15.26:0/2426869430 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4011830 con 0x7ffafc0ccc10 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.969+0000 7ffaf2000640 1 -- 172.21.15.26:0/2426869430 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae40119b0 con 0x7ffafc0ccc10 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.969+0000 7ffaf2000640 1 -- 172.21.15.26:0/2426869430 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4011d60 con 0x7ffafc0ccc10 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.969+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc0a9b60 con 0x7ffafc0ccc10 2024-01-15T18:10:39.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.969+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc0aa120 con 0x7ffafc0ccc10 2024-01-15T18:10:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.970+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc0aa640 con 0x7ffafc0ccc10 2024-01-15T18:10:39.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.971+0000 7ffaf2000640 1 -- 172.21.15.26:0/2426869430 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7ffae401a400 con 0x7ffafc0ccc10 2024-01-15T18:10:39.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.972+0000 7ffaf2000640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffad406c1f0 0x7ffad406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:39.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.972+0000 7ffaf2000640 1 -- 172.21.15.26:0/2426869430 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ffae4088860 con 0x7ffafc0ccc10 2024-01-15T18:10:39.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.973+0000 7ffaf9e00640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffad406c1f0 0x7ffad406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:39.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.974+0000 7ffaf9e00640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffad406c1f0 0x7ffad406e6b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4006fd0 tx=0x7ffaf4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:39.979 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:39.976+0000 7ffaf2000640 1 -- 172.21.15.26:0/2426869430 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffafc0aa640 con 0x7ffafc0ccc10 2024-01-15T18:10:40.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.195+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffafc06cc40 con 0x7ffad406c1f0 2024-01-15T18:10:40.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.198+0000 7ffaf2000640 1 -- 172.21.15.26:0/2426869430 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ffaf4009d60 con 0x7ffad406c1f0 2024-01-15T18:10:40.202 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:10:40.202 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:10:40.202 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:10:40.202 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:10:40.202 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:10:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffad406c1f0 msgr2=0x7ffad406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffad406c1f0 0x7ffad406e6b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4006fd0 tx=0x7ffaf4008040 comp rx=0 tx=0).stop 2024-01-15T18:10:40.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 msgr2=0x7ffafc0a9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 0x7ffafc0a9130 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7ffae400b3e0 tx=0x7ffae400b8a0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 shutdown_connections 2024-01-15T18:10:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ffad406c1f0 0x7ffad406e6b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ffafc143c40 0x7ffafc0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ffafc0ccc10 0x7ffafc0a9130 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 --2- 172.21.15.26:0/2426869430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ffafc0717d0 0x7ffafc0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 >> 172.21.15.26:0/2426869430 conn(0x7ffafc06a800 msgr2=0x7ffafc0cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 shutdown_connections 2024-01-15T18:10:40.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.201+0000 7ffafb200640 1 -- 172.21.15.26:0/2426869430 wait complete. 2024-01-15T18:10:40.225 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:10:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.447+0000 7fd073200640 1 Processor -- start 2024-01-15T18:10:40.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.448+0000 7fd073200640 1 -- start start 2024-01-15T18:10:40.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.449+0000 7fd073200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd06c160920 0x7fd06c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.449+0000 7fd073200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c0cf9f0 0x7fd06c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.449+0000 7fd073200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd06c161010 0x7fd06c1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.449+0000 7fd073200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c0d03b0 con 0x7fd06c161010 2024-01-15T18:10:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.449+0000 7fd073200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c0d0a10 con 0x7fd06c160920 2024-01-15T18:10:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.449+0000 7fd073200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0cf080 con 0x7fd06c0cf9f0 2024-01-15T18:10:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.449+0000 7fd072800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd06c161010 0x7fd06c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd072800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd06c161010 0x7fd06c1614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54192/0 (socket says 172.21.15.26:54192) 2024-01-15T18:10:40.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd072800640 1 -- 172.21.15.26:0/3942122832 learned_addr learned my addr 172.21.15.26:0/3942122832 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd071e00640 1 --2- 172.21.15.26:0/3942122832 >> v2:172.21.15.124:3300/0 conn(0x7fd06c160920 0x7fd06c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd072800640 1 -- 172.21.15.26:0/3942122832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c0cf9f0 msgr2=0x7fd06c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd071400640 1 --2- 172.21.15.26:0/3942122832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c0cf9f0 0x7fd06c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd072800640 1 --2- 172.21.15.26:0/3942122832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c0cf9f0 0x7fd06c0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd072800640 1 -- 172.21.15.26:0/3942122832 >> v2:172.21.15.124:3300/0 conn(0x7fd06c160920 msgr2=0x7fd06c0b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd072800640 1 --2- 172.21.15.26:0/3942122832 >> v2:172.21.15.124:3300/0 conn(0x7fd06c160920 0x7fd06c0b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd072800640 1 -- 172.21.15.26:0/3942122832 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0c6be0 con 0x7fd06c161010 2024-01-15T18:10:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd071400640 1 --2- 172.21.15.26:0/3942122832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c0cf9f0 0x7fd06c0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:40.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.450+0000 7fd071e00640 1 --2- 172.21.15.26:0/3942122832 >> v2:172.21.15.124:3300/0 conn(0x7fd06c160920 0x7fd06c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:40.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.451+0000 7fd072800640 1 --2- 172.21.15.26:0/3942122832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c161010 0x7fd06c1614a0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd06c08c060 tx=0x7fd06402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af88ddf5c73b49b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.451+0000 7fd070a00640 1 -- 172.21.15.26:0/3942122832 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064031870 con 0x7fd06c161010 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.451+0000 7fd070a00640 1 -- 172.21.15.26:0/3942122832 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0640319d0 con 0x7fd06c161010 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.451+0000 7fd070a00640 1 -- 172.21.15.26:0/3942122832 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064031d60 con 0x7fd06c161010 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.452+0000 7fd073200640 1 -- 172.21.15.26:0/3942122832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c161010 msgr2=0x7fd06c1614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.452+0000 7fd073200640 1 --2- 172.21.15.26:0/3942122832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c161010 0x7fd06c1614a0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fd06c08c060 tx=0x7fd06402b910 comp rx=0 tx=0).stop 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.452+0000 7fd073200640 1 -- 172.21.15.26:0/3942122832 shutdown_connections 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.452+0000 7fd073200640 1 --2- 172.21.15.26:0/3942122832 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c161010 0x7fd06c1614a0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.452+0000 7fd073200640 1 --2- 172.21.15.26:0/3942122832 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c0cf9f0 0x7fd06c0cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.452+0000 7fd073200640 1 --2- 172.21.15.26:0/3942122832 >> v2:172.21.15.124:3300/0 conn(0x7fd06c160920 0x7fd06c0b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.452+0000 7fd073200640 1 -- 172.21.15.26:0/3942122832 >> 172.21.15.26:0/3942122832 conn(0x7fd06c0d3090 msgr2=0x7fd06c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.452+0000 7fd073200640 1 -- 172.21.15.26:0/3942122832 shutdown_connections 2024-01-15T18:10:40.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.452+0000 7fd073200640 1 -- 172.21.15.26:0/3942122832 wait complete. 2024-01-15T18:10:40.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.454+0000 7fd073200640 1 Processor -- start 2024-01-15T18:10:40.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.454+0000 7fd073200640 1 -- start start 2024-01-15T18:10:40.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd073200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd06c0cf9f0 0x7fd06c0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd073200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c160920 0x7fd06c1f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd073200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c161010 0x7fd06c158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd073200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0b7150 con 0x7fd06c160920 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd073200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c157bd0 con 0x7fd06c0cf9f0 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd073200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0b7c90 con 0x7fd06c161010 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd071400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c160920 0x7fd06c1f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd071400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c160920 0x7fd06c1f3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54198/0 (socket says 172.21.15.26:54198) 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd071400640 1 -- 172.21.15.26:0/106737716 learned_addr learned my addr 172.21.15.26:0/106737716 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd071400640 1 -- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c161010 msgr2=0x7fd06c158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd071400640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c161010 0x7fd06c158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd071400640 1 -- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd06c0cf9f0 msgr2=0x7fd06c0cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.455+0000 7fd071400640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd06c0cf9f0 0x7fd06c0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.456+0000 7fd071400640 1 -- 172.21.15.26:0/106737716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0640094d0 con 0x7fd06c160920 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.456+0000 7fd071400640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c160920 0x7fd06c1f3210 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd05400c7f0 tx=0x7fd05400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.456+0000 7fd06a000640 1 -- 172.21.15.26:0/106737716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05400cea0 con 0x7fd06c160920 2024-01-15T18:10:40.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.456+0000 7fd06a000640 1 -- 172.21.15.26:0/106737716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd054007a30 con 0x7fd06c160920 2024-01-15T18:10:40.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.456+0000 7fd06a000640 1 -- 172.21.15.26:0/106737716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054007dc0 con 0x7fd06c160920 2024-01-15T18:10:40.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.456+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c0b7360 con 0x7fd06c160920 2024-01-15T18:10:40.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.456+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c0b7920 con 0x7fd06c160920 2024-01-15T18:10:40.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.457+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c067450 con 0x7fd06c160920 2024-01-15T18:10:40.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.458+0000 7fd06a000640 1 -- 172.21.15.26:0/106737716 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fd0540113f0 con 0x7fd06c160920 2024-01-15T18:10:40.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.460+0000 7fd06a000640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd04406c1f0 0x7fd04406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.460+0000 7fd06a000640 1 -- 172.21.15.26:0/106737716 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd054088870 con 0x7fd06c160920 2024-01-15T18:10:40.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.461+0000 7fd071e00640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd04406c1f0 0x7fd04406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.462+0000 7fd071e00640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd04406c1f0 0x7fd04406e6b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd05c008ea0 tx=0x7fd05c0083d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:40.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.466+0000 7fd06a000640 1 -- 172.21.15.26:0/106737716 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd054055130 con 0x7fd06c160920 2024-01-15T18:10:40.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.664+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd06c0b7e00 con 0x7fd06c160920 2024-01-15T18:10:40.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.665+0000 7fd06a000640 1 -- 172.21.15.26:0/106737716 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd054058750 con 0x7fd06c160920 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd04406c1f0 msgr2=0x7fd04406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd04406c1f0 0x7fd04406e6b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd05c008ea0 tx=0x7fd05c0083d0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c160920 msgr2=0x7fd06c1f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c160920 0x7fd06c1f3210 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd05400c7f0 tx=0x7fd05400ccb0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 shutdown_connections 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd06c161010 0x7fd06c158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd04406c1f0 0x7fd04406e6b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd06c160920 0x7fd06c1f3210 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 --2- 172.21.15.26:0/106737716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd06c0cf9f0 0x7fd06c0cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 >> 172.21.15.26:0/106737716 conn(0x7fd06c0d3090 msgr2=0x7fd06c06c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 shutdown_connections 2024-01-15T18:10:40.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.668+0000 7fd073200640 1 -- 172.21.15.26:0/106737716 wait complete. 2024-01-15T18:10:40.689 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_delay 10 2024-01-15T18:10:40.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.902+0000 7f7616200640 1 Processor -- start 2024-01-15T18:10:40.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.903+0000 7f7616200640 1 -- start start 2024-01-15T18:10:40.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.904+0000 7f7616200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f7610070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.904+0000 7f7616200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f76100ccb50 0x7f7610143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.904+0000 7f7616200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f761015f4e0 con 0x7f761015bf10 2024-01-15T18:10:40.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.904+0000 7f7616200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f761015f970 con 0x7f76100ccb50 2024-01-15T18:10:40.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.904+0000 7f7616200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7610071130 con 0x7f76100adf20 2024-01-15T18:10:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.905+0000 7f7615800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f76100ccb50 0x7f7610143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.905+0000 7f7614e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f7610070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.905+0000 7f7615800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f76100ccb50 0x7f7610143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:60508/0 (socket says 172.21.15.26:60508) 2024-01-15T18:10:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.905+0000 7f7614e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f7610070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:41596/0 (socket says 172.21.15.26:41596) 2024-01-15T18:10:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.905+0000 7f7607c00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f761015bf10 0x7f76100cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:37690/0 (socket says 172.21.15.26:37690) 2024-01-15T18:10:40.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.905+0000 7f7614e00640 1 -- 172.21.15.26:0/1467090169 learned_addr learned my addr 172.21.15.26:0/1467090169 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:40.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.906+0000 7f7614e00640 1 -- 172.21.15.26:0/1467090169 >> v2:172.21.15.124:3300/0 conn(0x7f76100ccb50 msgr2=0x7f7610143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:40.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.906+0000 7f7614e00640 1 --2- 172.21.15.26:0/1467090169 >> v2:172.21.15.124:3300/0 conn(0x7f76100ccb50 0x7f7610143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.906+0000 7f7614e00640 1 -- 172.21.15.26:0/1467090169 >> v1:172.21.15.124:6789/0 conn(0x7f761015bf10 legacy=0x7f76100cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.906+0000 7f7614e00640 1 -- 172.21.15.26:0/1467090169 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f761015c390 con 0x7f76100adf20 2024-01-15T18:10:40.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.908+0000 7f7614e00640 1 --2- 172.21.15.26:0/1467090169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f7610070a30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7600009870 tx=0x7f760002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b214eae62e4676c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:40.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.908+0000 7f760fe00640 1 -- 172.21.15.26:0/1467090169 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600031640 con 0x7f76100adf20 2024-01-15T18:10:40.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.908+0000 7f760fe00640 1 -- 172.21.15.26:0/1467090169 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76000317a0 con 0x7f76100adf20 2024-01-15T18:10:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.908+0000 7f760fe00640 1 -- 172.21.15.26:0/1467090169 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600031b30 con 0x7f76100adf20 2024-01-15T18:10:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.909+0000 7f7616200640 1 -- 172.21.15.26:0/1467090169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 msgr2=0x7f7610070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.909+0000 7f7616200640 1 --2- 172.21.15.26:0/1467090169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f7610070a30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7600009870 tx=0x7f760002b8f0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.909+0000 7f7616200640 1 -- 172.21.15.26:0/1467090169 shutdown_connections 2024-01-15T18:10:40.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.909+0000 7f7616200640 1 --2- 172.21.15.26:0/1467090169 >> v2:172.21.15.124:3300/0 conn(0x7f76100ccb50 0x7f7610143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.909+0000 7f7616200640 1 --2- 172.21.15.26:0/1467090169 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f7610070a30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.909+0000 7f7616200640 1 -- 172.21.15.26:0/1467090169 >> 172.21.15.26:0/1467090169 conn(0x7f761006a7d0 msgr2=0x7f761006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.909+0000 7f7616200640 1 -- 172.21.15.26:0/1467090169 shutdown_connections 2024-01-15T18:10:40.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.909+0000 7f7616200640 1 -- 172.21.15.26:0/1467090169 wait complete. 2024-01-15T18:10:40.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.911+0000 7f7616200640 1 Processor -- start 2024-01-15T18:10:40.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.911+0000 7f7616200640 1 -- start start 2024-01-15T18:10:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.912+0000 7f7616200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f76100ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.912+0000 7f7616200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76100ccb50 0x7f76100c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.912+0000 7f7616200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761015bf10 0x7f7610133f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.912+0000 7f7616200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76100c4240 con 0x7f76100ccb50 2024-01-15T18:10:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.912+0000 7f7616200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76100c4620 con 0x7f761015bf10 2024-01-15T18:10:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.912+0000 7f7616200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76100c48c0 con 0x7f76100adf20 2024-01-15T18:10:40.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7607c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76100ccb50 0x7f76100c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7607c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76100ccb50 0x7f76100c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54210/0 (socket says 172.21.15.26:54210) 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7607c00640 1 -- 172.21.15.26:0/602063206 learned_addr learned my addr 172.21.15.26:0/602063206 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7614e00640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f76100ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7615800640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761015bf10 0x7f7610133f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7607c00640 1 -- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 msgr2=0x7f76100ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7607c00640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f76100ab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7607c00640 1 -- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761015bf10 msgr2=0x7f7610133f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7607c00640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761015bf10 0x7f7610133f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7607c00640 1 -- 172.21.15.26:0/602063206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000094d0 con 0x7f76100ccb50 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7614e00640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f76100ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7615800640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761015bf10 0x7f7610133f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.913+0000 7f7607c00640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76100ccb50 0x7f76100c3d00 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f75f800c3c0 tx=0x7f75f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.914+0000 7f760d600640 1 -- 172.21.15.26:0/602063206 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f80074e0 con 0x7f76100ccb50 2024-01-15T18:10:40.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.914+0000 7f760d600640 1 -- 172.21.15.26:0/602063206 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f8007640 con 0x7f76100ccb50 2024-01-15T18:10:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.914+0000 7f760d600640 1 -- 172.21.15.26:0/602063206 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8004a00 con 0x7f76100ccb50 2024-01-15T18:10:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.914+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76100c4ba0 con 0x7f76100ccb50 2024-01-15T18:10:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.914+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76100ab830 con 0x7f76100ccb50 2024-01-15T18:10:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.915+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76100a9110 con 0x7f76100ccb50 2024-01-15T18:10:40.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.916+0000 7f760d600640 1 -- 172.21.15.26:0/602063206 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f75f8004b60 con 0x7f76100ccb50 2024-01-15T18:10:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.917+0000 7f760d600640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75e806c1f0 0x7f75e806e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:40.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.917+0000 7f760d600640 1 -- 172.21.15.26:0/602063206 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f75f8088080 con 0x7f76100ccb50 2024-01-15T18:10:40.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.918+0000 7f7614e00640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75e806c1f0 0x7f75e806e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:40.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.919+0000 7f7614e00640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75e806c1f0 0x7f75e806e6b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7600036000 tx=0x7f760003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:40.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:40.922+0000 7f760d600640 1 -- 172.21.15.26:0/602063206 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76100a9110 con 0x7f76100ccb50 2024-01-15T18:10:41.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.117+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}] v 0) v1 -- 0x7f76100a9110 con 0x7f76100ccb50 2024-01-15T18:10:41.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.122+0000 7f760d600640 1 -- 172.21.15.26:0/602063206 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}]=0 v35)=0 v35) v1 ==== 131+0+0 (secure 0 0 0) 0x7f76100a9110 con 0x7f76100ccb50 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75e806c1f0 msgr2=0x7f75e806e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75e806c1f0 0x7f75e806e6b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7600036000 tx=0x7f760003c040 comp rx=0 tx=0).stop 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76100ccb50 msgr2=0x7f76100c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76100ccb50 0x7f76100c3d00 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f75f800c3c0 tx=0x7f75f800c880 comp rx=0 tx=0).stop 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 shutdown_connections 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f75e806c1f0 0x7f75e806e6b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f761015bf10 0x7f7610133f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f76100ccb50 0x7f76100c3d00 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 --2- 172.21.15.26:0/602063206 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f76100adf20 0x7f76100ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 >> 172.21.15.26:0/602063206 conn(0x7f761006a7d0 msgr2=0x7f761006abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.125+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 shutdown_connections 2024-01-15T18:10:41.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.126+0000 7f7616200640 1 -- 172.21.15.26:0/602063206 wait complete. 2024-01-15T18:10:41.149 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 120 2024-01-15T18:10:41.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.369+0000 7f1f78a00640 1 Processor -- start 2024-01-15T18:10:41.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.369+0000 7f1f78a00640 1 -- start start 2024-01-15T18:10:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.370+0000 7f1f78a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f1f740d1d40 0x7f1f740b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.370+0000 7f1f78a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f740b1f70 0x7f1f740b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.370+0000 7f1f78a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740d0090 con 0x7f1f7415d1d0 2024-01-15T18:10:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.371+0000 7f1f78a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740d02e0 con 0x7f1f740b1f70 2024-01-15T18:10:41.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.371+0000 7f1f78a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740d05c0 con 0x7f1f740d1d40 2024-01-15T18:10:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.371+0000 7f1f73e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f740b1f70 0x7f1f740b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.371+0000 7f1f73e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f1f740b1f70 0x7f1f740b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54212/0 (socket says 172.21.15.26:54212) 2024-01-15T18:10:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.371+0000 7f1f73e00640 1 -- 172.21.15.26:0/1489522478 learned_addr learned my addr 172.21.15.26:0/1489522478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.371+0000 7f1f72a00640 1 --2- 172.21.15.26:0/1489522478 >> v2:172.21.15.124:3300/0 conn(0x7f1f740d1d40 0x7f1f740b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.372+0000 7f1f73e00640 1 -- 172.21.15.26:0/1489522478 >> v2:172.21.15.124:3300/0 conn(0x7f1f740d1d40 msgr2=0x7f1f740b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.372+0000 7f1f73e00640 1 --2- 172.21.15.26:0/1489522478 >> v2:172.21.15.124:3300/0 conn(0x7f1f740d1d40 0x7f1f740b01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.372+0000 7f1f73e00640 1 -- 172.21.15.26:0/1489522478 >> v1:172.21.15.124:6789/0 conn(0x7f1f7415d1d0 legacy=0x7f1f740d1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.372+0000 7f1f73e00640 1 -- 172.21.15.26:0/1489522478 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740d08a0 con 0x7f1f740b1f70 2024-01-15T18:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.373+0000 7f1f73e00640 1 --2- 172.21.15.26:0/1489522478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f740b1f70 0x7f1f740b0700 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009870 tx=0x7f1f6c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59deec9d0d861f2a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.373+0000 7f1f72000640 1 -- 172.21.15.26:0/1489522478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c030860 con 0x7f1f740b1f70 2024-01-15T18:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.373+0000 7f1f72000640 1 -- 172.21.15.26:0/1489522478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c0309c0 con 0x7f1f740b1f70 2024-01-15T18:10:41.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.373+0000 7f1f72000640 1 -- 172.21.15.26:0/1489522478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c030d50 con 0x7f1f740b1f70 2024-01-15T18:10:41.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.374+0000 7f1f78a00640 1 -- 172.21.15.26:0/1489522478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f740b1f70 msgr2=0x7f1f740b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.374+0000 7f1f78a00640 1 --2- 172.21.15.26:0/1489522478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f740b1f70 0x7f1f740b0700 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009870 tx=0x7f1f6c02b9a0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.374+0000 7f1f78a00640 1 -- 172.21.15.26:0/1489522478 shutdown_connections 2024-01-15T18:10:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.374+0000 7f1f78a00640 1 --2- 172.21.15.26:0/1489522478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f740b1f70 0x7f1f740b0700 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.374+0000 7f1f78a00640 1 --2- 172.21.15.26:0/1489522478 >> v2:172.21.15.124:3300/0 conn(0x7f1f740d1d40 0x7f1f740b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.374+0000 7f1f78a00640 1 -- 172.21.15.26:0/1489522478 >> 172.21.15.26:0/1489522478 conn(0x7f1f740d3050 msgr2=0x7f1f740d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.374+0000 7f1f78a00640 1 -- 172.21.15.26:0/1489522478 shutdown_connections 2024-01-15T18:10:41.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.374+0000 7f1f78a00640 1 -- 172.21.15.26:0/1489522478 wait complete. 2024-01-15T18:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.376+0000 7f1f78a00640 1 Processor -- start 2024-01-15T18:10:41.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.376+0000 7f1f78a00640 1 -- start start 2024-01-15T18:10:41.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.377+0000 7f1f78a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740b1f70 0x7f1f740b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.377+0000 7f1f78a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740d1d40 0x7f1f740b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.377+0000 7f1f78a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7415d1d0 0x7f1f7415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.377+0000 7f1f78a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740cb170 con 0x7f1f7415d1d0 2024-01-15T18:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.377+0000 7f1f78a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740cb3c0 con 0x7f1f740b1f70 2024-01-15T18:10:41.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.377+0000 7f1f78a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740cb6a0 con 0x7f1f740d1d40 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7415d1d0 0x7f1f7415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7415d1d0 0x7f1f7415faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54226/0 (socket says 172.21.15.26:54226) 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73e00640 1 -- 172.21.15.26:0/322829005 learned_addr learned my addr 172.21.15.26:0/322829005 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73400640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740b1f70 0x7f1f740b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f72a00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740d1d40 0x7f1f740b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73e00640 1 -- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740d1d40 msgr2=0x7f1f740b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73e00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740d1d40 0x7f1f740b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73e00640 1 -- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740b1f70 msgr2=0x7f1f740b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73e00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740b1f70 0x7f1f740b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73e00640 1 -- 172.21.15.26:0/322829005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0094d0 con 0x7f1f7415d1d0 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.378+0000 7f1f73400640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740b1f70 0x7f1f740b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.379+0000 7f1f73e00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7415d1d0 0x7f1f7415faf0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009870 tx=0x7f1f6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:41.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.379+0000 7f1f6b400640 1 -- 172.21.15.26:0/322829005 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c039c90 con 0x7f1f7415d1d0 2024-01-15T18:10:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.379+0000 7f1f6b400640 1 -- 172.21.15.26:0/322829005 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c039e10 con 0x7f1f7415d1d0 2024-01-15T18:10:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.379+0000 7f1f6b400640 1 -- 172.21.15.26:0/322829005 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c043640 con 0x7f1f7415d1d0 2024-01-15T18:10:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.379+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740cb980 con 0x7f1f7415d1d0 2024-01-15T18:10:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.379+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f740cbf40 con 0x7f1f7415d1d0 2024-01-15T18:10:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.380+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f74066730 con 0x7f1f7415d1d0 2024-01-15T18:10:41.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.381+0000 7f1f6b400640 1 -- 172.21.15.26:0/322829005 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f1f6c0437a0 con 0x7f1f7415d1d0 2024-01-15T18:10:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.382+0000 7f1f6b400640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f4c06c1f0 0x7f1f4c06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.382+0000 7f1f6b400640 1 -- 172.21.15.26:0/322829005 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1f6c03e070 con 0x7f1f7415d1d0 2024-01-15T18:10:41.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.383+0000 7f1f73400640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f4c06c1f0 0x7f1f4c06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.384+0000 7f1f73400640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f4c06c1f0 0x7f1f4c06e6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c007fd0 tx=0x7f1f5c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:41.390 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.387+0000 7f1f6b400640 1 -- 172.21.15.26:0/322829005 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f6c078090 con 0x7f1f7415d1d0 2024-01-15T18:10:41.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.582+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f1f74067450 con 0x7f1f7415d1d0 2024-01-15T18:10:41.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.588+0000 7f1f6b400640 1 -- 172.21.15.26:0/322829005 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v36)=0 v36) v1 ==== 137+0+0 (secure 0 0 0) 0x7f1f6c07b6b0 con 0x7f1f7415d1d0 2024-01-15T18:10:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.591+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f4c06c1f0 msgr2=0x7f1f4c06e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.591+0000 7f1f78a00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f4c06c1f0 0x7f1f4c06e6b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c007fd0 tx=0x7f1f5c009040 comp rx=0 tx=0).stop 2024-01-15T18:10:41.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.591+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7415d1d0 msgr2=0x7f1f7415faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.591+0000 7f1f78a00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7415d1d0 0x7f1f7415faf0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009870 tx=0x7f1f6c004730 comp rx=0 tx=0).stop 2024-01-15T18:10:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.592+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 shutdown_connections 2024-01-15T18:10:41.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.592+0000 7f1f78a00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f740d1d40 0x7f1f740b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.592+0000 7f1f78a00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f4c06c1f0 0x7f1f4c06e6b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.592+0000 7f1f78a00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f7415d1d0 0x7f1f7415faf0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.592+0000 7f1f78a00640 1 --2- 172.21.15.26:0/322829005 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f740b1f70 0x7f1f740b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.592+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 >> 172.21.15.26:0/322829005 conn(0x7f1f740d3050 msgr2=0x7f1f740cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.592+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 shutdown_connections 2024-01-15T18:10:41.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.592+0000 7f1f78a00640 1 -- 172.21.15.26:0/322829005 wait complete. 2024-01-15T18:10:41.613 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:10:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.826+0000 7f9693400640 1 Processor -- start 2024-01-15T18:10:41.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.827+0000 7f9693400640 1 -- start start 2024-01-15T18:10:41.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.828+0000 7f9693400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c0cf9a0 0x7f968c070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.828+0000 7f9693400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f968c070ea0 0x7f968c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9693400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968c071690 con 0x7f968c14b2c0 2024-01-15T18:10:41.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9693400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968c14b6a0 con 0x7f968c070ea0 2024-01-15T18:10:41.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9693400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c071970 con 0x7f968c0cf9a0 2024-01-15T18:10:41.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9691400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f968c070ea0 0x7f968c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9691400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f968c070ea0 0x7f968c0cc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54230/0 (socket says 172.21.15.26:54230) 2024-01-15T18:10:41.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9691400640 1 -- 172.21.15.26:0/381077036 learned_addr learned my addr 172.21.15.26:0/381077036 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:41.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f968be00640 1 --2- 172.21.15.26:0/381077036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c0cf9a0 0x7f968c070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9691400640 1 -- 172.21.15.26:0/381077036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c0cf9a0 msgr2=0x7f968c070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:41.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9691400640 1 --2- 172.21.15.26:0/381077036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c0cf9a0 0x7f968c070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9691400640 1 -- 172.21.15.26:0/381077036 >> v1:172.21.15.124:6789/0 conn(0x7f968c14b2c0 legacy=0x7f968c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:41.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.829+0000 7f9691400640 1 -- 172.21.15.26:0/381077036 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968c071c50 con 0x7f968c070ea0 2024-01-15T18:10:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.831+0000 7f9691400640 1 --2- 172.21.15.26:0/381077036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c070ea0 0x7f968c0cc160 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f9684009e50 tx=0x7f968402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d3280e068b13e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.831+0000 7f968b400640 1 -- 172.21.15.26:0/381077036 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968402ca70 con 0x7f968c070ea0 2024-01-15T18:10:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.831+0000 7f968b400640 1 -- 172.21.15.26:0/381077036 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968402cbd0 con 0x7f968c070ea0 2024-01-15T18:10:41.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.831+0000 7f968b400640 1 -- 172.21.15.26:0/381077036 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96840343f0 con 0x7f968c070ea0 2024-01-15T18:10:41.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.832+0000 7f9693400640 1 -- 172.21.15.26:0/381077036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c070ea0 msgr2=0x7f968c0cc160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.832+0000 7f9693400640 1 --2- 172.21.15.26:0/381077036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c070ea0 0x7f968c0cc160 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f9684009e50 tx=0x7f968402bed0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.832+0000 7f9693400640 1 -- 172.21.15.26:0/381077036 shutdown_connections 2024-01-15T18:10:41.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.832+0000 7f9693400640 1 --2- 172.21.15.26:0/381077036 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c070ea0 0x7f968c0cc160 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.832+0000 7f9693400640 1 --2- 172.21.15.26:0/381077036 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c0cf9a0 0x7f968c070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.832+0000 7f9693400640 1 -- 172.21.15.26:0/381077036 >> 172.21.15.26:0/381077036 conn(0x7f968c06a800 msgr2=0x7f968c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.832+0000 7f9693400640 1 -- 172.21.15.26:0/381077036 shutdown_connections 2024-01-15T18:10:41.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.832+0000 7f9693400640 1 -- 172.21.15.26:0/381077036 wait complete. 2024-01-15T18:10:41.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.834+0000 7f9693400640 1 Processor -- start 2024-01-15T18:10:41.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.834+0000 7f9693400640 1 -- start start 2024-01-15T18:10:41.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.835+0000 7f9693400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c070ea0 0x7f968c15b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.835+0000 7f9693400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f968c0cf9a0 0x7f968c164600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.835+0000 7f9693400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c14b2c0 0x7f968c13f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.835+0000 7f9693400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c1490f0 con 0x7f968c14b2c0 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.835+0000 7f9693400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968c15b790 con 0x7f968c0cf9a0 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.835+0000 7f9693400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968c148e10 con 0x7f968c070ea0 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.835+0000 7f9691400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c14b2c0 0x7f968c13f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.835+0000 7f9691400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c14b2c0 0x7f968c13f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:54236/0 (socket says 172.21.15.26:54236) 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.835+0000 7f9691400640 1 -- 172.21.15.26:0/1744407320 learned_addr learned my addr 172.21.15.26:0/1744407320 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9690a00640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c070ea0 0x7f968c15b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f968be00640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f968c0cf9a0 0x7f968c164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9691400640 1 -- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c070ea0 msgr2=0x7f968c15b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9691400640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c070ea0 0x7f968c15b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9691400640 1 -- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f968c0cf9a0 msgr2=0x7f968c164600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:41.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9691400640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f968c0cf9a0 0x7f968c164600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:41.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9691400640 1 -- 172.21.15.26:0/1744407320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9684009b30 con 0x7f968c14b2c0 2024-01-15T18:10:41.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9691400640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c14b2c0 0x7f968c13f840 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f9684009e50 tx=0x7f9684004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:41.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9688c00640 1 -- 172.21.15.26:0/1744407320 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968403e450 con 0x7f968c14b2c0 2024-01-15T18:10:41.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9688c00640 1 -- 172.21.15.26:0/1744407320 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968403e5b0 con 0x7f968c14b2c0 2024-01-15T18:10:41.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.836+0000 7f9688c00640 1 -- 172.21.15.26:0/1744407320 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968403e960 con 0x7f968c14b2c0 2024-01-15T18:10:41.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.837+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968c161e00 con 0x7f968c14b2c0 2024-01-15T18:10:41.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.837+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968c1623c0 con 0x7f968c14b2c0 2024-01-15T18:10:41.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.838+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968c0675d0 con 0x7f968c14b2c0 2024-01-15T18:10:41.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.839+0000 7f9688c00640 1 -- 172.21.15.26:0/1744407320 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7f968403eac0 con 0x7f968c14b2c0 2024-01-15T18:10:41.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.840+0000 7f9688c00640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f966406c1f0 0x7f966406e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:41.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.840+0000 7f9688c00640 1 -- 172.21.15.26:0/1744407320 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f96840ab490 con 0x7f968c14b2c0 2024-01-15T18:10:41.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.840+0000 7f9690a00640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f966406c1f0 0x7f966406e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:41.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.841+0000 7f9690a00640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f966406c1f0 0x7f966406e6b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f967c006fd0 tx=0x7f967c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:41.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:41.844+0000 7f9688c00640 1 -- 172.21.15.26:0/1744407320 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9684077d50 con 0x7f968c14b2c0 2024-01-15T18:10:42.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.066+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f968c073c60 con 0x7f966406c1f0 2024-01-15T18:10:42.082 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:10:42.079+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:42.082 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:10:42.079+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:42.082 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:10:42.079+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:42.082 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:10:42.079+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:42.083 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:10:42.079+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:10:42.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.142+0000 7f9688c00640 1 -- 172.21.15.26:0/1744407320 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f968c073c60 con 0x7f966406c1f0 2024-01-15T18:10:42.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f966406c1f0 msgr2=0x7f966406e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f966406c1f0 0x7f966406e6b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f967c006fd0 tx=0x7f967c008040 comp rx=0 tx=0).stop 2024-01-15T18:10:42.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c14b2c0 msgr2=0x7f968c13f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c14b2c0 0x7f968c13f840 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f9684009e50 tx=0x7f9684004a40 comp rx=0 tx=0).stop 2024-01-15T18:10:42.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 shutdown_connections 2024-01-15T18:10:42.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f966406c1f0 0x7f966406e6b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f968c14b2c0 0x7f968c13f840 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f968c0cf9a0 0x7f968c164600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 --2- 172.21.15.26:0/1744407320 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f968c070ea0 0x7f968c15b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.145+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 >> 172.21.15.26:0/1744407320 conn(0x7f968c06a800 msgr2=0x7f968c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:42.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.146+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 shutdown_connections 2024-01-15T18:10:42.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.146+0000 7f9693400640 1 -- 172.21.15.26:0/1744407320 wait complete. 2024-01-15T18:10:42.170 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:10:42.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.395+0000 7fc5f8600640 1 Processor -- start 2024-01-15T18:10:42.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.395+0000 7fc5f8600640 1 -- start start 2024-01-15T18:10:42.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.397+0000 7fc5f8600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f00ced30 0x7fc5f00b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.397+0000 7fc5f8600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00cf1c0 con 0x7fc5f00ce8f0 2024-01-15T18:10:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.397+0000 7fc5f8600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00d0de0 con 0x7fc5f0160f80 2024-01-15T18:10:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.397+0000 7fc5f8600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00d10c0 con 0x7fc5f00ced30 2024-01-15T18:10:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.397+0000 7fc5f5200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc5f00ce8f0 0x7fc5f00b2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:52212/0 (socket says 172.21.15.26:52212) 2024-01-15T18:10:42.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.397+0000 7fc5f5200640 1 -- 172.21.15.26:0/2574694414 learned_addr learned my addr 172.21.15.26:0/2574694414 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5f6600640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f00ced30 0x7fc5f00b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237385020 0 0) 0x7fc5e0001110 con 0x7fc5f00ce8f0 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5d0003520 con 0x7fc5f00ce8f0 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784116013 0 0) 0x7fc5d0003520 con 0x7fc5f00ce8f0 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f00ced30 msgr2=0x7fc5f00b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5efe00640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f00ced30 0x7fc5f00b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 >> v1:172.21.15.124:6789/0 conn(0x7fc5f0160f80 legacy=0x7fc5f0161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00d13a0 con 0x7fc5f00ce8f0 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fc5e0001800 con 0x7fc5f00ce8f0 2024-01-15T18:10:42.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.398+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 >> v1:172.21.15.26:6789/0 conn(0x7fc5f00ce8f0 legacy=0x7fc5f00b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5efe00640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5d00039a0 0x7fc5d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5efe00640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f00ced30 0x7fc5d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5efe00640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5d0004d00 0x7fc5d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00d10c0 con 0x7fc5d00039a0 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d000ba60 con 0x7fc5d0004d00 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d000bc20 con 0x7fc5f00ced30 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5f5c00640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5d00039a0 0x7fc5d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5f6600640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5d0004d00 0x7fc5d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5f5200640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f00ced30 0x7fc5d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5f5c00640 1 -- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f00ced30 msgr2=0x7fc5d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5f5c00640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5f00ced30 0x7fc5d00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5f5c00640 1 -- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5d0004d00 msgr2=0x7fc5d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5f5c00640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5d0004d00 0x7fc5d0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.399+0000 7fc5f5c00640 1 -- 172.21.15.26:0/2574694414 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d0003520 con 0x7fc5d00039a0 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.401+0000 7fc5f5c00640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5d00039a0 0x7fc5d0003dc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8001720 tx=0x7fc5d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7033cbe2f04fd3f4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.401+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80052d0 con 0x7fc5d00039a0 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.401+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d802e050 con 0x7fc5d00039a0 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.401+0000 7fc5efe00640 1 -- 172.21.15.26:0/2574694414 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d80035f0 con 0x7fc5d00039a0 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.402+0000 7fc5f8600640 1 -- 172.21.15.26:0/2574694414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5d00039a0 msgr2=0x7fc5d0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.402+0000 7fc5f8600640 1 --2- 172.21.15.26:0/2574694414 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5d00039a0 0x7fc5d0003dc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8001720 tx=0x7fc5d00057f0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.402+0000 7fc5f6600640 1 -- 172.21.15.26:0/2574694414 reap_dead start 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.402+0000 7fc5f8600640 1 -- 172.21.15.26:0/2574694414 shutdown_connections 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.402+0000 7fc5f8600640 1 -- 172.21.15.26:0/2574694414 >> 172.21.15.26:0/2574694414 conn(0x7fc5f009c550 msgr2=0x7fc5f00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.402+0000 7fc5f8600640 1 -- 172.21.15.26:0/2574694414 shutdown_connections 2024-01-15T18:10:42.406 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.402+0000 7fc5f8600640 1 -- 172.21.15.26:0/2574694414 wait complete. 2024-01-15T18:10:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.405+0000 7fc5f8600640 1 Processor -- start 2024-01-15T18:10:42.408 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.405+0000 7fc5f8600640 1 -- start start 2024-01-15T18:10:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.406+0000 7fc5f8600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5d00039a0 0x7fc5f015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.406+0000 7fc5f8600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f00caaa0 0x7fc5f01e7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.406+0000 7fc5f8600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f01e7890 0x7fc5f00c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.406+0000 7fc5f8600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01e7d10 con 0x7fc5f01e7890 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f8600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00cc190 con 0x7fc5f00caaa0 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f8600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00cc3e0 con 0x7fc5d00039a0 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f6600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f01e7890 0x7fc5f00c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f6600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f01e7890 0x7fc5f00c6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55736/0 (socket says 172.21.15.26:55736) 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f6600640 1 -- 172.21.15.26:0/1429682428 learned_addr learned my addr 172.21.15.26:0/1429682428 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f5c00640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5d00039a0 0x7fc5f015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f5200640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f00caaa0 0x7fc5f01e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f6600640 1 -- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5d00039a0 msgr2=0x7fc5f015aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f6600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5d00039a0 0x7fc5f015aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f6600640 1 -- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f00caaa0 msgr2=0x7fc5f01e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f6600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f00caaa0 0x7fc5f01e7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.411 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f6600640 1 -- 172.21.15.26:0/1429682428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f00cc6c0 con 0x7fc5f01e7890 2024-01-15T18:10:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.407+0000 7fc5f5200640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f00caaa0 0x7fc5f01e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:10:42.412 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.408+0000 7fc5f6600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f01e7890 0x7fc5f00c6d40 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00d2bd0 tx=0x7fc5e80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.408+0000 7fc5ed600640 1 -- 172.21.15.26:0/1429682428 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800e070 con 0x7fc5f01e7890 2024-01-15T18:10:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.408+0000 7fc5ed600640 1 -- 172.21.15.26:0/1429682428 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8011e10 con 0x7fc5f01e7890 2024-01-15T18:10:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.408+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f00cc9a0 con 0x7fc5f01e7890 2024-01-15T18:10:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.408+0000 7fc5ed600640 1 -- 172.21.15.26:0/1429682428 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8003990 con 0x7fc5f01e7890 2024-01-15T18:10:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.408+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f00ccf60 con 0x7fc5f01e7890 2024-01-15T18:10:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.409+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f00675d0 con 0x7fc5f01e7890 2024-01-15T18:10:42.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.411+0000 7fc5ed600640 1 -- 172.21.15.26:0/1429682428 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 34) v1 ==== 90916+0+0 (secure 0 0 0) 0x7fc5e8003af0 con 0x7fc5f01e7890 2024-01-15T18:10:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.411+0000 7fc5ed600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5c806c220 0x7fc5c806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.412+0000 7fc5ed600640 1 -- 172.21.15.26:0/1429682428 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc5e8088c50 con 0x7fc5f01e7890 2024-01-15T18:10:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.412+0000 7fc5f5c00640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5c806c220 0x7fc5c806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.413+0000 7fc5f5c00640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5c806c220 0x7fc5c806e6e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8033000 tx=0x7fc5d8009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:42.420 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.417+0000 7fc5ed600640 1 -- 172.21.15.26:0/1429682428 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5e8055510 con 0x7fc5f01e7890 2024-01-15T18:10:42.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.642+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5f015df60 con 0x7fc5c806c220 2024-01-15T18:10:42.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.657+0000 7fc5ed600640 1 -- 172.21.15.26:0/1429682428 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fc5f015df60 con 0x7fc5c806c220 2024-01-15T18:10:42.660 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9 2024-01-15T18:10:42.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.659+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5c806c220 msgr2=0x7fc5c806e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.659+0000 7fc5f8600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5c806c220 0x7fc5c806e6e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8033000 tx=0x7fc5d8009030 comp rx=0 tx=0).stop 2024-01-15T18:10:42.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f01e7890 msgr2=0x7fc5f00c6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f01e7890 0x7fc5f00c6d40 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00d2bd0 tx=0x7fc5e80115a0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 shutdown_connections 2024-01-15T18:10:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5c806c220 0x7fc5c806e6e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc5f01e7890 0x7fc5f00c6d40 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc5f00caaa0 0x7fc5f01e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 --2- 172.21.15.26:0/1429682428 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc5d00039a0 0x7fc5f015aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 >> 172.21.15.26:0/1429682428 conn(0x7fc5f009c550 msgr2=0x7fc5f00b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 shutdown_connections 2024-01-15T18:10:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.660+0000 7fc5f8600640 1 -- 172.21.15.26:0/1429682428 wait complete. 2024-01-15T18:10:42.684 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:10:42.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.907+0000 7f6b71000640 1 Processor -- start 2024-01-15T18:10:42.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.907+0000 7f6b71000640 1 -- start start 2024-01-15T18:10:42.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.909+0000 7f6b71000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6b6c0adf20 0x7f6b6c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.909+0000 7f6b71000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c15bf10 0x7f6b6c0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.909+0000 7f6b71000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c15f4e0 con 0x7f6b6c0cc980 2024-01-15T18:10:42.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.909+0000 7f6b71000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c15f970 con 0x7f6b6c0adf20 2024-01-15T18:10:42.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.909+0000 7f6b71000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0cce10 con 0x7f6b6c15bf10 2024-01-15T18:10:42.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.910+0000 7f6b63e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c15bf10 0x7f6b6c0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.910+0000 7f6b63e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c15bf10 0x7f6b6c0cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:43132/0 (socket says 172.21.15.26:43132) 2024-01-15T18:10:42.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.910+0000 7f6b63e00640 1 -- 172.21.15.26:0/1977356865 learned_addr learned my addr 172.21.15.26:0/1977356865 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:42.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.910+0000 7f6b6b400640 1 --2- 172.21.15.26:0/1977356865 >> v2:172.21.15.124:3300/0 conn(0x7f6b6c0adf20 0x7f6b6c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.911+0000 7f6b63e00640 1 -- 172.21.15.26:0/1977356865 >> v2:172.21.15.124:3300/0 conn(0x7f6b6c0adf20 msgr2=0x7f6b6c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:10:42.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.911+0000 7f6b63e00640 1 --2- 172.21.15.26:0/1977356865 >> v2:172.21.15.124:3300/0 conn(0x7f6b6c0adf20 0x7f6b6c070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.911+0000 7f6b63e00640 1 -- 172.21.15.26:0/1977356865 >> v1:172.21.15.124:6789/0 conn(0x7f6b6c0cc980 legacy=0x7f6b6c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.911+0000 7f6b63e00640 1 -- 172.21.15.26:0/1977356865 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c071130 con 0x7f6b6c15bf10 2024-01-15T18:10:42.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.912+0000 7f6b63e00640 1 --2- 172.21.15.26:0/1977356865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c15bf10 0x7f6b6c0cc440 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009870 tx=0x7f6b5402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc470402039d455 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:42.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b6aa00640 1 -- 172.21.15.26:0/1977356865 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54031640 con 0x7f6b6c15bf10 2024-01-15T18:10:42.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b6aa00640 1 -- 172.21.15.26:0/1977356865 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b540317a0 con 0x7f6b6c15bf10 2024-01-15T18:10:42.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b6aa00640 1 -- 172.21.15.26:0/1977356865 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54031b30 con 0x7f6b6c15bf10 2024-01-15T18:10:42.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b71000640 1 -- 172.21.15.26:0/1977356865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c15bf10 msgr2=0x7f6b6c0cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b71000640 1 --2- 172.21.15.26:0/1977356865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c15bf10 0x7f6b6c0cc440 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b54009870 tx=0x7f6b5402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b71000640 1 -- 172.21.15.26:0/1977356865 shutdown_connections 2024-01-15T18:10:42.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b71000640 1 --2- 172.21.15.26:0/1977356865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c15bf10 0x7f6b6c0cc440 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b71000640 1 --2- 172.21.15.26:0/1977356865 >> v2:172.21.15.124:3300/0 conn(0x7f6b6c0adf20 0x7f6b6c070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b71000640 1 -- 172.21.15.26:0/1977356865 >> 172.21.15.26:0/1977356865 conn(0x7f6b6c06a7d0 msgr2=0x7f6b6c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:42.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.913+0000 7f6b71000640 1 -- 172.21.15.26:0/1977356865 shutdown_connections 2024-01-15T18:10:42.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.914+0000 7f6b71000640 1 -- 172.21.15.26:0/1977356865 wait complete. 2024-01-15T18:10:42.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.915+0000 7f6b71000640 1 Processor -- start 2024-01-15T18:10:42.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.915+0000 7f6b71000640 1 -- start start 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.916+0000 7f6b71000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6c0adf20 0x7f6b6c0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.916+0000 7f6b71000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c0cc980 0x7f6b6c1f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.916+0000 7f6b71000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b6c15bf10 0x7f6b6c15c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.916+0000 7f6b71000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0af790 con 0x7f6b6c0adf20 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.916+0000 7f6b71000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0b3ec0 con 0x7f6b6c15bf10 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.916+0000 7f6b71000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c1f3950 con 0x7f6b6c0cc980 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b6b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6c0adf20 0x7f6b6c0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b6b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6c0adf20 0x7f6b6c0b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55752/0 (socket says 172.21.15.26:55752) 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b6b400640 1 -- 172.21.15.26:0/3186002523 learned_addr learned my addr 172.21.15.26:0/3186002523 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b63e00640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c0cc980 0x7f6b6c1f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b6be00640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b6c15bf10 0x7f6b6c15c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b6b400640 1 -- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c0cc980 msgr2=0x7f6b6c1f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b6b400640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c0cc980 0x7f6b6c1f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b6b400640 1 -- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b6c15bf10 msgr2=0x7f6b6c15c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b6b400640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b6c15bf10 0x7f6b6c15c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.917+0000 7f6b6b400640 1 -- 172.21.15.26:0/3186002523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b540094d0 con 0x7f6b6c0adf20 2024-01-15T18:10:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.918+0000 7f6b6b400640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6c0adf20 0x7f6b6c0b3980 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00e870 tx=0x7f6b5c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:42.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.918+0000 7f6b63400640 1 -- 172.21.15.26:0/3186002523 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c009510 con 0x7f6b6c0adf20 2024-01-15T18:10:42.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.918+0000 7f6b63400640 1 -- 172.21.15.26:0/3186002523 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c00eec0 con 0x7f6b6c0adf20 2024-01-15T18:10:42.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.918+0000 7f6b63400640 1 -- 172.21.15.26:0/3186002523 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c00cb80 con 0x7f6b6c0adf20 2024-01-15T18:10:42.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.918+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c0aeb80 con 0x7f6b6c0adf20 2024-01-15T18:10:42.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.918+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c0af140 con 0x7f6b6c0adf20 2024-01-15T18:10:42.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.920+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c067450 con 0x7f6b6c0adf20 2024-01-15T18:10:42.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.921+0000 7f6b63400640 1 -- 172.21.15.26:0/3186002523 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 35) v1 ==== 90967+0+0 (secure 0 0 0) 0x7f6b5c00cd20 con 0x7f6b6c0adf20 2024-01-15T18:10:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.921+0000 7f6b63400640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b4406c2d0 0x7f6b4406e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:10:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.922+0000 7f6b63400640 1 -- 172.21.15.26:0/3186002523 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6b5c088b90 con 0x7f6b6c0adf20 2024-01-15T18:10:42.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.922+0000 7f6b63e00640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b4406c2d0 0x7f6b4406e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:10:42.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.923+0000 7f6b63e00640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b4406c2d0 0x7f6b4406e790 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6b54036000 tx=0x7f6b5403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:10:42.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:42.926+0000 7f6b63400640 1 -- 172.21.15.26:0/3186002523 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b5c055180 con 0x7f6b6c0adf20 2024-01-15T18:10:43.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.151+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b6c15ee90 con 0x7f6b4406c2d0 2024-01-15T18:10:43.169 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.166+0000 7f6b63400640 1 -- 172.21.15.26:0/3186002523 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f6b54031b20 con 0x7f6b4406c2d0 2024-01-15T18:10:43.169 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9 2024-01-15T18:10:43.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b4406c2d0 msgr2=0x7f6b4406e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:43.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b4406c2d0 0x7f6b4406e790 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6b54036000 tx=0x7f6b5403c040 comp rx=0 tx=0).stop 2024-01-15T18:10:43.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6c0adf20 msgr2=0x7f6b6c0b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:10:43.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6c0adf20 0x7f6b6c0b3980 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00e870 tx=0x7f6b5c00ec70 comp rx=0 tx=0).stop 2024-01-15T18:10:43.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 shutdown_connections 2024-01-15T18:10:43.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b6c0cc980 0x7f6b6c1f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:43.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b4406c2d0 0x7f6b4406e790 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:43.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b6c15bf10 0x7f6b6c15c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 --2- 172.21.15.26:0/3186002523 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b6c0adf20 0x7f6b6c0b3980 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 >> 172.21.15.26:0/3186002523 conn(0x7f6b6c06a7d0 msgr2=0x7f6b6c1430c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.168+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 shutdown_connections 2024-01-15T18:10:43.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:10:43.169+0000 7f6b71000640 1 -- 172.21.15.26:0/3186002523 wait complete. 2024-01-15T18:10:43.192 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9 2024-01-15T18:10:43.193 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:10:43.193 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:43.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:43.250 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:10:59.512 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:10:59.512 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:10:59.512 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.1739 s, 66.4 MB/s 2024-01-15T18:10:59.514 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:10:59.514 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:10:59.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:10:59.572 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:11:01.123 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:11:01.131 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:11:14.824 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:11:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:11:14.825 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.1877 s, 70.7 MB/s 2024-01-15T18:11:14.826 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:11:14.827 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:14.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:14.884 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:11:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:11:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:11:29.113 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.1677 s, 75.8 MB/s 2024-01-15T18:11:29.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:11:29.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.353+0000 7f5d51c00640 1 Processor -- start 2024-01-15T18:11:29.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.353+0000 7f5d51c00640 1 -- start start 2024-01-15T18:11:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.354+0000 7f5d51c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5d4c13fcb0 0x7f5d4c164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.354+0000 7f5d51c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4c0cdc40 con 0x7f5d4c15afb0 2024-01-15T18:11:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.355+0000 7f5d51c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4c0cdec0 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.355+0000 7f5d51c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c0ce1a0 con 0x7f5d4c13fcb0 2024-01-15T18:11:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.355+0000 7f5d43e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5d4c15afb0 0x7f5d4c13f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58998/0 (socket says 172.21.15.26:58998) 2024-01-15T18:11:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.355+0000 7f5d43e00640 1 -- 172.21.15.26:0/319749710 learned_addr learned my addr 172.21.15.26:0/319749710 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:29.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.355+0000 7f5d51200640 1 --2- 172.21.15.26:0/319749710 >> v2:172.21.15.26:3300/0 conn(0x7f5d4c13fcb0 0x7f5d4c164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.355+0000 7f5d4b400640 1 -- 172.21.15.26:0/319749710 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268662206 0 0) 0x7f5d4c0cdc40 con 0x7f5d4c15afb0 2024-01-15T18:11:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.356+0000 7f5d4b400640 1 -- 172.21.15.26:0/319749710 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d2c003520 con 0x7f5d4c15afb0 2024-01-15T18:11:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.356+0000 7f5d51200640 1 -- 172.21.15.26:0/319749710 >> v1:172.21.15.124:6789/0 conn(0x7f5d4c13cfa0 legacy=0x7f5d4c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.356+0000 7f5d51200640 1 -- 172.21.15.26:0/319749710 >> v1:172.21.15.26:6789/0 conn(0x7f5d4c15afb0 legacy=0x7f5d4c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.356+0000 7f5d51200640 1 -- 172.21.15.26:0/319749710 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0ce480 con 0x7f5d4c13fcb0 2024-01-15T18:11:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.357+0000 7f5d51200640 1 --2- 172.21.15.26:0/319749710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13fcb0 0x7f5d4c164d80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009d90 tx=0x7f5d4402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e00fdd70e30c25a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.357+0000 7f5d4b400640 1 -- 172.21.15.26:0/319749710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4402cc60 con 0x7f5d4c13fcb0 2024-01-15T18:11:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.357+0000 7f5d4b400640 1 -- 172.21.15.26:0/319749710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4402cdc0 con 0x7f5d4c13fcb0 2024-01-15T18:11:29.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.357+0000 7f5d4b400640 1 -- 172.21.15.26:0/319749710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44034660 con 0x7f5d4c13fcb0 2024-01-15T18:11:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.358+0000 7f5d51c00640 1 -- 172.21.15.26:0/319749710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13fcb0 msgr2=0x7f5d4c164d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.358+0000 7f5d51c00640 1 --2- 172.21.15.26:0/319749710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13fcb0 0x7f5d4c164d80 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009d90 tx=0x7f5d4402beb0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.358+0000 7f5d51c00640 1 -- 172.21.15.26:0/319749710 shutdown_connections 2024-01-15T18:11:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.358+0000 7f5d51c00640 1 --2- 172.21.15.26:0/319749710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13fcb0 0x7f5d4c164d80 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.358+0000 7f5d51c00640 1 -- 172.21.15.26:0/319749710 >> 172.21.15.26:0/319749710 conn(0x7f5d4c06a800 msgr2=0x7f5d4c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.358+0000 7f5d51c00640 1 -- 172.21.15.26:0/319749710 shutdown_connections 2024-01-15T18:11:29.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.358+0000 7f5d51c00640 1 -- 172.21.15.26:0/319749710 wait complete. 2024-01-15T18:11:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.360+0000 7f5d51c00640 1 Processor -- start 2024-01-15T18:11:29.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.360+0000 7f5d51c00640 1 -- start start 2024-01-15T18:11:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d51c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13cfa0 0x7f5d4c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d51c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c13fcb0 0x7f5d4c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d51c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15afb0 0x7f5d4c071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d51c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c13a3a0 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d51c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c1506d0 con 0x7f5d4c13fcb0 2024-01-15T18:11:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d51c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c139d70 con 0x7f5d4c15afb0 2024-01-15T18:11:29.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13cfa0 0x7f5d4c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d4be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13cfa0 0x7f5d4c150190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37276/0 (socket says 172.21.15.26:37276) 2024-01-15T18:11:29.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d4be00640 1 -- 172.21.15.26:0/4012126154 learned_addr learned my addr 172.21.15.26:0/4012126154 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.361+0000 7f5d43e00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c13fcb0 0x7f5d4c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d51200640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15afb0 0x7f5d4c071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d4be00640 1 -- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15afb0 msgr2=0x7f5d4c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d4be00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15afb0 0x7f5d4c071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d4be00640 1 -- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c13fcb0 msgr2=0x7f5d4c1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d4be00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c13fcb0 0x7f5d4c1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d4be00640 1 -- 172.21.15.26:0/4012126154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d44009a70 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d43e00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c13fcb0 0x7f5d4c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d4be00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13cfa0 0x7f5d4c150190 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00c450 tx=0x7f5d3c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d48c00640 1 -- 172.21.15.26:0/4012126154 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0052d0 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d48c00640 1 -- 172.21.15.26:0/4012126154 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c005430 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c156f00 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d48c00640 1 -- 172.21.15.26:0/4012126154 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c012870 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.362+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c13a030 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.364+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c067450 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.365+0000 7f5d48c00640 1 -- 172.21.15.26:0/4012126154 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 35) v1 ==== 90967+0+0 (secure 0 0 0) 0x7f5d3c0129d0 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.366+0000 7f5d48c00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d2406c2b0 0x7f5d2406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.366+0000 7f5d48c00640 1 -- 172.21.15.26:0/4012126154 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5d3c088270 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.367+0000 7f5d43e00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d2406c2b0 0x7f5d2406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.368+0000 7f5d43e00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d2406c2b0 0x7f5d2406e770 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0cbfc0 tx=0x7f5d34009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:29.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.371+0000 7f5d48c00640 1 -- 172.21.15.26:0/4012126154 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d3c054aa0 con 0x7f5d4c13cfa0 2024-01-15T18:11:29.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.601+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d4c0cea40 con 0x7f5d2406c2b0 2024-01-15T18:11:29.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.636+0000 7f5d48c00640 1 -- 172.21.15.26:0/4012126154 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5d34001330 con 0x7f5d2406c2b0 2024-01-15T18:11:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d2406c2b0 msgr2=0x7f5d2406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d2406c2b0 0x7f5d2406e770 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0cbfc0 tx=0x7f5d34009040 comp rx=0 tx=0).stop 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13cfa0 msgr2=0x7f5d4c150190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13cfa0 0x7f5d4c150190 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00c450 tx=0x7f5d3c00c910 comp rx=0 tx=0).stop 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 shutdown_connections 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5d4c15afb0 0x7f5d4c071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5d2406c2b0 0x7f5d2406e770 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5d4c13fcb0 0x7f5d4c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 --2- 172.21.15.26:0/4012126154 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5d4c13cfa0 0x7f5d4c150190 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 >> 172.21.15.26:0/4012126154 conn(0x7f5d4c06a800 msgr2=0x7f5d4c06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 shutdown_connections 2024-01-15T18:11:29.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.639+0000 7f5d51c00640 1 -- 172.21.15.26:0/4012126154 wait complete. 2024-01-15T18:11:29.660 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:11:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.877+0000 7f7058a00640 1 Processor -- start 2024-01-15T18:11:29.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.878+0000 7f7058a00640 1 -- start start 2024-01-15T18:11:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.879+0000 7f7058a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 0x7f7054070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.879+0000 7f7058a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f705415bf10 0x7f70540cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.879+0000 7f7058a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705415f4e0 con 0x7f70540cc980 2024-01-15T18:11:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.879+0000 7f7058a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705415f970 con 0x7f705415bf10 2024-01-15T18:11:29.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.879+0000 7f7058a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70540cce10 con 0x7f70540adf20 2024-01-15T18:11:29.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.879+0000 7f7052a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f705415bf10 0x7f70540cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.880+0000 7f7053e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f70540cc980 0x7f7054143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59002/0 (socket says 172.21.15.26:59002) 2024-01-15T18:11:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.880+0000 7f7053e00640 1 -- 172.21.15.26:0/693595801 learned_addr learned my addr 172.21.15.26:0/693595801 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.880+0000 7f7053400640 1 --2- 172.21.15.26:0/693595801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 0x7f7054070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.880+0000 7f7052a00640 1 -- 172.21.15.26:0/693595801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 msgr2=0x7f7054070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.880+0000 7f7052a00640 1 --2- 172.21.15.26:0/693595801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 0x7f7054070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.880+0000 7f7052a00640 1 -- 172.21.15.26:0/693595801 >> v1:172.21.15.26:6789/0 conn(0x7f70540cc980 legacy=0x7f7054143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.880+0000 7f7052a00640 1 -- 172.21.15.26:0/693595801 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054071130 con 0x7f705415bf10 2024-01-15T18:11:29.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.880+0000 7f7052000640 1 -- 172.21.15.26:0/693595801 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111318617 0 0) 0x7f705415f4e0 con 0x7f70540cc980 2024-01-15T18:11:29.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.881+0000 7f7052a00640 1 --2- 172.21.15.26:0/693595801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705415bf10 0x7f70540cc440 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f703c0097a0 tx=0x7f703c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d66a4d406271fb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.882+0000 7f7052000640 1 -- 172.21.15.26:0/693595801 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c031780 con 0x7f705415bf10 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.882+0000 7f7052000640 1 -- 172.21.15.26:0/693595801 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c0318e0 con 0x7f705415bf10 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.882+0000 7f7052000640 1 -- 172.21.15.26:0/693595801 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c031c70 con 0x7f705415bf10 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.882+0000 7f7058a00640 1 -- 172.21.15.26:0/693595801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705415bf10 msgr2=0x7f70540cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.882+0000 7f7058a00640 1 --2- 172.21.15.26:0/693595801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705415bf10 0x7f70540cc440 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f703c0097a0 tx=0x7f703c02b820 comp rx=0 tx=0).stop 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.882+0000 7f7058a00640 1 -- 172.21.15.26:0/693595801 shutdown_connections 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.882+0000 7f7058a00640 1 --2- 172.21.15.26:0/693595801 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f705415bf10 0x7f70540cc440 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.882+0000 7f7058a00640 1 --2- 172.21.15.26:0/693595801 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 0x7f7054070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.882+0000 7f7058a00640 1 -- 172.21.15.26:0/693595801 >> 172.21.15.26:0/693595801 conn(0x7f705406a7d0 msgr2=0x7f705406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.883+0000 7f7058a00640 1 -- 172.21.15.26:0/693595801 shutdown_connections 2024-01-15T18:11:29.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.883+0000 7f7058a00640 1 -- 172.21.15.26:0/693595801 wait complete. 2024-01-15T18:11:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.884+0000 7f7058a00640 1 Processor -- start 2024-01-15T18:11:29.887 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.884+0000 7f7058a00640 1 -- start start 2024-01-15T18:11:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.885+0000 7f7058a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 0x7f70540cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.885+0000 7f7058a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70540cc980 0x7f70540a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.885+0000 7f7058a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f705415bf10 0x7f705406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.885+0000 7f7058a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705412f760 con 0x7f70540cc980 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.885+0000 7f7058a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70540a8f80 con 0x7f705415bf10 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.885+0000 7f7058a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70540a9260 con 0x7f70540adf20 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7052a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70540cc980 0x7f70540a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7052a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70540cc980 0x7f70540a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37298/0 (socket says 172.21.15.26:37298) 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7052a00640 1 -- 172.21.15.26:0/2955084842 learned_addr learned my addr 172.21.15.26:0/2955084842 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7052a00640 1 -- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 msgr2=0x7f70540cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7053400640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 0x7f70540cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7053e00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f705415bf10 0x7f705406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7052a00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 0x7f70540cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7052a00640 1 -- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f705415bf10 msgr2=0x7f705406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7052a00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f705415bf10 0x7f705406d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:29.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7052a00640 1 -- 172.21.15.26:0/2955084842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c009400 con 0x7f70540cc980 2024-01-15T18:11:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.886+0000 7f7053e00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f705415bf10 0x7f705406d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:29.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.887+0000 7f7052a00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70540cc980 0x7f70540a84c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f703c000c00 tx=0x7f703c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.887+0000 7f704b400640 1 -- 172.21.15.26:0/2955084842 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c031f10 con 0x7f70540cc980 2024-01-15T18:11:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.887+0000 7f704b400640 1 -- 172.21.15.26:0/2955084842 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c004250 con 0x7f70540cc980 2024-01-15T18:11:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.887+0000 7f704b400640 1 -- 172.21.15.26:0/2955084842 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c032620 con 0x7f70540cc980 2024-01-15T18:11:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.887+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70540a9540 con 0x7f70540cc980 2024-01-15T18:11:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.887+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705413f6c0 con 0x7f70540cc980 2024-01-15T18:11:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.888+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70540cd300 con 0x7f70540cc980 2024-01-15T18:11:29.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.889+0000 7f704b400640 1 -- 172.21.15.26:0/2955084842 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 35) v1 ==== 90967+0+0 (secure 0 0 0) 0x7f703c032780 con 0x7f70540cc980 2024-01-15T18:11:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.890+0000 7f704b400640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f702c06c2d0 0x7f702c06e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:29.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.890+0000 7f704b400640 1 -- 172.21.15.26:0/2955084842 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f703c0aadf0 con 0x7f70540cc980 2024-01-15T18:11:29.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.891+0000 7f7053400640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f702c06c2d0 0x7f702c06e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:29.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.892+0000 7f7053400640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f702c06c2d0 0x7f702c06e790 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7044006fd0 tx=0x7f7044008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:29.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:29.895+0000 7f704b400640 1 -- 172.21.15.26:0/2955084842 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f703c077430 con 0x7f70540cc980 2024-01-15T18:11:30.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.120+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f70540a90f0 con 0x7f702c06c2d0 2024-01-15T18:11:30.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.135+0000 7f704b400640 1 -- 172.21.15.26:0/2955084842 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f70540a90f0 con 0x7f702c06c2d0 2024-01-15T18:11:30.139 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9 2024-01-15T18:11:30.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f702c06c2d0 msgr2=0x7f702c06e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f702c06c2d0 0x7f702c06e790 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7044006fd0 tx=0x7f7044008040 comp rx=0 tx=0).stop 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70540cc980 msgr2=0x7f70540a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70540cc980 0x7f70540a84c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f703c000c00 tx=0x7f703c0046c0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 shutdown_connections 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f702c06c2d0 0x7f702c06e790 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f705415bf10 0x7f705406d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f70540cc980 0x7f70540a84c0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 --2- 172.21.15.26:0/2955084842 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f70540adf20 0x7f70540cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 >> 172.21.15.26:0/2955084842 conn(0x7f705406a7d0 msgr2=0x7f70540af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 shutdown_connections 2024-01-15T18:11:30.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.138+0000 7f7058a00640 1 -- 172.21.15.26:0/2955084842 wait complete. 2024-01-15T18:11:30.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:30.162 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9) 2024-01-15T18:11:30.226 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9 2024-01-15T18:11:30.226 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:11:30.226 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:11:30.227 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:11:30.227 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 ss1clone1 2024-01-15T18:11:30.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.448+0000 7f2a50200640 1 Processor -- start 2024-01-15T18:11:30.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.448+0000 7f2a50200640 1 -- start start 2024-01-15T18:11:30.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.449+0000 7f2a50200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a480cf9a0 0x7f2a48070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.449+0000 7f2a50200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2a48070ea0 0x7f2a480cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.449+0000 7f2a50200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a48071690 con 0x7f2a4814b2c0 2024-01-15T18:11:30.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.450+0000 7f2a50200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4814b6a0 con 0x7f2a480cf9a0 2024-01-15T18:11:30.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.450+0000 7f2a50200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a48071970 con 0x7f2a48070ea0 2024-01-15T18:11:30.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.450+0000 7f2a4d800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2a4814b2c0 0x7f2a480cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59004/0 (socket says 172.21.15.26:59004) 2024-01-15T18:11:30.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.450+0000 7f2a4ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2a480cf9a0 0x7f2a48070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.450+0000 7f2a4d800640 1 -- 172.21.15.26:0/3489234710 learned_addr learned my addr 172.21.15.26:0/3489234710 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:30.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.450+0000 7f2a4e200640 1 --2- 172.21.15.26:0/3489234710 >> v2:172.21.15.124:3300/0 conn(0x7f2a48070ea0 0x7f2a480cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.450+0000 7f2a4ce00640 1 -- 172.21.15.26:0/3489234710 >> v2:172.21.15.124:3300/0 conn(0x7f2a48070ea0 msgr2=0x7f2a480cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:30.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.450+0000 7f2a47e00640 1 -- 172.21.15.26:0/3489234710 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028651137 0 0) 0x7f2a48071690 con 0x7f2a4814b2c0 2024-01-15T18:11:30.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.451+0000 7f2a4ce00640 1 --2- 172.21.15.26:0/3489234710 >> v2:172.21.15.124:3300/0 conn(0x7f2a48070ea0 0x7f2a480cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.451+0000 7f2a4ce00640 1 -- 172.21.15.26:0/3489234710 >> v1:172.21.15.26:6789/0 conn(0x7f2a4814b2c0 legacy=0x7f2a480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.451+0000 7f2a4ce00640 1 -- 172.21.15.26:0/3489234710 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48071c50 con 0x7f2a480cf9a0 2024-01-15T18:11:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.452+0000 7f2a4ce00640 1 --2- 172.21.15.26:0/3489234710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a480cf9a0 0x7f2a48070960 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f2a48165330 tx=0x7f2a3002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b89f88b5014087 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.452+0000 7f2a47e00640 1 -- 172.21.15.26:0/3489234710 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30031830 con 0x7f2a480cf9a0 2024-01-15T18:11:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.452+0000 7f2a47e00640 1 -- 172.21.15.26:0/3489234710 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a30031990 con 0x7f2a480cf9a0 2024-01-15T18:11:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.452+0000 7f2a47e00640 1 -- 172.21.15.26:0/3489234710 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30031d20 con 0x7f2a480cf9a0 2024-01-15T18:11:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.453+0000 7f2a50200640 1 -- 172.21.15.26:0/3489234710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a480cf9a0 msgr2=0x7f2a48070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.453+0000 7f2a50200640 1 --2- 172.21.15.26:0/3489234710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a480cf9a0 0x7f2a48070960 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f2a48165330 tx=0x7f2a3002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.453+0000 7f2a50200640 1 -- 172.21.15.26:0/3489234710 shutdown_connections 2024-01-15T18:11:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.453+0000 7f2a50200640 1 --2- 172.21.15.26:0/3489234710 >> v2:172.21.15.124:3300/0 conn(0x7f2a48070ea0 0x7f2a480cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.453+0000 7f2a50200640 1 --2- 172.21.15.26:0/3489234710 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a480cf9a0 0x7f2a48070960 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.453+0000 7f2a50200640 1 -- 172.21.15.26:0/3489234710 >> 172.21.15.26:0/3489234710 conn(0x7f2a4806a800 msgr2=0x7f2a4806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:30.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.453+0000 7f2a50200640 1 -- 172.21.15.26:0/3489234710 shutdown_connections 2024-01-15T18:11:30.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.453+0000 7f2a50200640 1 -- 172.21.15.26:0/3489234710 wait complete. 2024-01-15T18:11:30.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.455+0000 7f2a50200640 1 Processor -- start 2024-01-15T18:11:30.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.455+0000 7f2a50200640 1 -- start start 2024-01-15T18:11:30.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a50200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a48070ea0 0x7f2a48144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a50200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a480cf9a0 0x7f2a48145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a50200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a4814b2c0 0x7f2a4813fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a50200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48144940 con 0x7f2a4814b2c0 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a50200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48143cc0 con 0x7f2a48070ea0 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a50200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48143fa0 con 0x7f2a480cf9a0 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a4e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a4814b2c0 0x7f2a4813fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a4e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a4814b2c0 0x7f2a4813fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37322/0 (socket says 172.21.15.26:37322) 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a4e200640 1 -- 172.21.15.26:0/2521055980 learned_addr learned my addr 172.21.15.26:0/2521055980 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a4d800640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a48070ea0 0x7f2a48144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.456+0000 7f2a4ce00640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a480cf9a0 0x7f2a48145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a4e200640 1 -- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a480cf9a0 msgr2=0x7f2a48145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a4e200640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a480cf9a0 0x7f2a48145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a4e200640 1 -- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a48070ea0 msgr2=0x7f2a48144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a4e200640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a48070ea0 0x7f2a48144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a4e200640 1 -- 172.21.15.26:0/2521055980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a30009400 con 0x7f2a4814b2c0 2024-01-15T18:11:30.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a4e200640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a4814b2c0 0x7f2a4813fd10 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f2a40007e40 tx=0x7f2a4000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a45600640 1 -- 172.21.15.26:0/2521055980 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a40014620 con 0x7f2a4814b2c0 2024-01-15T18:11:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a45600640 1 -- 172.21.15.26:0/2521055980 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4000ee90 con 0x7f2a4814b2c0 2024-01-15T18:11:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a48144280 con 0x7f2a4814b2c0 2024-01-15T18:11:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.457+0000 7f2a45600640 1 -- 172.21.15.26:0/2521055980 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a400149e0 con 0x7f2a4814b2c0 2024-01-15T18:11:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.458+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a48143950 con 0x7f2a4814b2c0 2024-01-15T18:11:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.459+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a48144a80 con 0x7f2a4814b2c0 2024-01-15T18:11:30.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.460+0000 7f2a45600640 1 -- 172.21.15.26:0/2521055980 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 35) v1 ==== 90967+0+0 (secure 0 0 0) 0x7f2a40014b40 con 0x7f2a4814b2c0 2024-01-15T18:11:30.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.461+0000 7f2a45600640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2006c280 0x7f2a2006e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:30.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.461+0000 7f2a45600640 1 -- 172.21.15.26:0/2521055980 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2a40089130 con 0x7f2a4814b2c0 2024-01-15T18:11:30.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.462+0000 7f2a4d800640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2006c280 0x7f2a2006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:30.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.463+0000 7f2a4d800640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2006c280 0x7f2a2006e740 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2a38000f60 tx=0x7f2a38007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:30.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.466+0000 7f2a45600640 1 -- 172.21.15.26:0/2521055980 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a48144a80 con 0x7f2a4814b2c0 2024-01-15T18:11:30.699 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.696+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a480cbc40 con 0x7f2a2006c280 2024-01-15T18:11:30.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.854+0000 7f2a45600640 1 -- 172.21.15.26:0/2521055980 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2a38007270 con 0x7f2a2006c280 2024-01-15T18:11:30.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2006c280 msgr2=0x7f2a2006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2006c280 0x7f2a2006e740 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2a38000f60 tx=0x7f2a38007c00 comp rx=0 tx=0).stop 2024-01-15T18:11:30.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a4814b2c0 msgr2=0x7f2a4813fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:30.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a4814b2c0 0x7f2a4813fd10 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f2a40007e40 tx=0x7f2a4000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 shutdown_connections 2024-01-15T18:11:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2a480cf9a0 0x7f2a48145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2a2006c280 0x7f2a2006e740 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2a4814b2c0 0x7f2a4813fd10 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 --2- 172.21.15.26:0/2521055980 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2a48070ea0 0x7f2a48144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 >> 172.21.15.26:0/2521055980 conn(0x7f2a4806a800 msgr2=0x7f2a480b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 shutdown_connections 2024-01-15T18:11:30.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:30.857+0000 7f2a50200640 1 -- 172.21.15.26:0/2521055980 wait complete. 2024-01-15T18:11:30.866 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.863+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:30.866 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.863+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:30.866 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.863+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:30.866 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.863+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:30.867 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.863+0000 7f8f5b400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:30.881 INFO:tasks.cephfs.test_volumes:expected clone state = ('pending', 'in-progress') 2024-01-15T18:11:30.881 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:11:30.897 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.894+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:30.897 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.894+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:30.898 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.894+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:30.898 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.894+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:30.898 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:30.894+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.105+0000 7fe292400640 1 Processor -- start 2024-01-15T18:11:31.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.105+0000 7fe292400640 1 -- start start 2024-01-15T18:11:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.107+0000 7fe292400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c070a30 0x7fe28c15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.107+0000 7fe292400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28c15f4e0 con 0x7fe28c0adf20 2024-01-15T18:11:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.107+0000 7fe292400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe28c070ee0 con 0x7fe28c0ccb50 2024-01-15T18:11:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.107+0000 7fe292400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c15c450 con 0x7fe28c070a30 2024-01-15T18:11:31.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.107+0000 7fe291000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe28c0adf20 0x7fe28c0cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59006/0 (socket says 172.21.15.26:59006) 2024-01-15T18:11:31.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.107+0000 7fe291000640 1 -- 172.21.15.26:0/3932990548 learned_addr learned my addr 172.21.15.26:0/3932990548 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:31.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe291a00640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c070a30 0x7fe28c15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366348458 0 0) 0x7fe28c15f4e0 con 0x7fe28c0adf20 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe26c003520 con 0x7fe28c0adf20 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 78152412 0 0) 0x7fe26c003520 con 0x7fe28c0adf20 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c070a30 msgr2=0x7fe28c15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c070a30 0x7fe28c15bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 >> v1:172.21.15.124:6789/0 conn(0x7fe28c0ccb50 legacy=0x7fe28c143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c071130 con 0x7fe28c0adf20 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fe27c001670 con 0x7fe28c0adf20 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 >> v1:172.21.15.26:6789/0 conn(0x7fe28c0adf20 legacy=0x7fe28c0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c070a30 0x7fe26c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe26c003f70 0x7fe26c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe26c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c15c450 con 0x7fe26c004d00 2024-01-15T18:11:31.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c00bb70 con 0x7fe26c003f70 2024-01-15T18:11:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.108+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c00bd30 con 0x7fe28c070a30 2024-01-15T18:11:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.109+0000 7fe291a00640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe26c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.109+0000 7fe28be00640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe26c003f70 0x7fe26c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.109+0000 7fe291000640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c070a30 0x7fe26c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.109+0000 7fe291a00640 1 -- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c070a30 msgr2=0x7fe26c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.109+0000 7fe291a00640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c070a30 0x7fe26c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.109+0000 7fe291a00640 1 -- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe26c003f70 msgr2=0x7fe26c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.109+0000 7fe291a00640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe26c003f70 0x7fe26c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.109+0000 7fe291a00640 1 -- 172.21.15.26:0/3932990548 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c003520 con 0x7fe26c004d00 2024-01-15T18:11:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.110+0000 7fe291a00640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe26c005180 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe284009fc0 tx=0x7fe26c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799b7c16377326fd server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28402cc20 con 0x7fe26c004d00 2024-01-15T18:11:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28400f040 con 0x7fe26c004d00 2024-01-15T18:11:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe28b400640 1 -- 172.21.15.26:0/3932990548 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe284034440 con 0x7fe26c004d00 2024-01-15T18:11:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe292400640 1 -- 172.21.15.26:0/3932990548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 msgr2=0x7fe26c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe292400640 1 --2- 172.21.15.26:0/3932990548 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe26c005180 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe284009fc0 tx=0x7fe26c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe291a00640 1 -- 172.21.15.26:0/3932990548 reap_dead start 2024-01-15T18:11:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe292400640 1 -- 172.21.15.26:0/3932990548 shutdown_connections 2024-01-15T18:11:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe292400640 1 -- 172.21.15.26:0/3932990548 >> 172.21.15.26:0/3932990548 conn(0x7fe28c06a7d0 msgr2=0x7fe28c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe292400640 1 -- 172.21.15.26:0/3932990548 shutdown_connections 2024-01-15T18:11:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.111+0000 7fe292400640 1 -- 172.21.15.26:0/3932990548 wait complete. 2024-01-15T18:11:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.113+0000 7fe292400640 1 Processor -- start 2024-01-15T18:11:31.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.113+0000 7fe292400640 1 -- start start 2024-01-15T18:11:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.114+0000 7fe292400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe28c0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.114+0000 7fe292400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe28c1f3330 0x7fe28c1f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.114+0000 7fe292400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c0ac900 0x7fe28c0acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.114+0000 7fe292400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0ad2d0 con 0x7fe26c004d00 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.114+0000 7fe292400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0caee0 con 0x7fe28c1f3330 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.114+0000 7fe292400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28c0b22e0 con 0x7fe28c0ac900 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.115+0000 7fe291000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe28c0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.115+0000 7fe291000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe28c0ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37350/0 (socket says 172.21.15.26:37350) 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.115+0000 7fe291000640 1 -- 172.21.15.26:0/4176306651 learned_addr learned my addr 172.21.15.26:0/4176306651 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.115+0000 7fe291000640 1 -- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c0ac900 msgr2=0x7fe28c0acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.115+0000 7fe291000640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c0ac900 0x7fe28c0acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.115+0000 7fe291000640 1 -- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe28c1f3330 msgr2=0x7fe28c1f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.115+0000 7fe291000640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe28c1f3330 0x7fe28c1f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.115+0000 7fe291000640 1 -- 172.21.15.26:0/4176306651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28c0b25c0 con 0x7fe26c004d00 2024-01-15T18:11:31.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.115+0000 7fe291000640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe28c0ca9a0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e7f0 tx=0x7fe27c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.116+0000 7fe288c00640 1 -- 172.21.15.26:0/4176306651 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00c7f0 con 0x7fe26c004d00 2024-01-15T18:11:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.116+0000 7fe288c00640 1 -- 172.21.15.26:0/4176306651 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c00c950 con 0x7fe26c004d00 2024-01-15T18:11:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.116+0000 7fe288c00640 1 -- 172.21.15.26:0/4176306651 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00ccc0 con 0x7fe26c004d00 2024-01-15T18:11:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.116+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe28c0b28a0 con 0x7fe26c004d00 2024-01-15T18:11:31.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.116+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28c0b2e60 con 0x7fe26c004d00 2024-01-15T18:11:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.117+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28c0b3080 con 0x7fe26c004d00 2024-01-15T18:11:31.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.118+0000 7fe288c00640 1 -- 172.21.15.26:0/4176306651 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 35) v1 ==== 90967+0+0 (secure 0 0 0) 0x7fe27c01d460 con 0x7fe26c004d00 2024-01-15T18:11:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.119+0000 7fe288c00640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe26406c280 0x7fe26406e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.119+0000 7fe288c00640 1 -- 172.21.15.26:0/4176306651 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe27c0886c0 con 0x7fe26c004d00 2024-01-15T18:11:31.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.120+0000 7fe28be00640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe26406c280 0x7fe26406e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.121+0000 7fe28be00640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe26406c280 0x7fe26406e740 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe274007fd0 tx=0x7fe274009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.124+0000 7fe288c00640 1 -- 172.21.15.26:0/4176306651 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe27c054f80 con 0x7fe26c004d00 2024-01-15T18:11:31.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:11:31.274 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:11:31.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.359+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe28c15ee90 con 0x7fe26406c280 2024-01-15T18:11:31.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.374+0000 7fe288c00640 1 -- 172.21.15.26:0/4176306651 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fe2740050d0 con 0x7fe26406c280 2024-01-15T18:11:31.377 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:11:31.377 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:11:31.377 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:11:31.378 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:11:31.378 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:11:31.378 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:11:31.378 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:11:31.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:11:31.378 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:11:31.378 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:11:31.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.377+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe26406c280 msgr2=0x7fe26406e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.377+0000 7fe292400640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe26406c280 0x7fe26406e740 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe274007fd0 tx=0x7fe274009040 comp rx=0 tx=0).stop 2024-01-15T18:11:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.377+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 msgr2=0x7fe28c0ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.377+0000 7fe292400640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe28c0ca9a0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe27c00e7f0 tx=0x7fe27c00ecb0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.377+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 shutdown_connections 2024-01-15T18:11:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.377+0000 7fe292400640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe28c0ac900 0x7fe28c0acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.377+0000 7fe292400640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe26406c280 0x7fe26406e740 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.378+0000 7fe292400640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe28c1f3330 0x7fe28c1f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.378+0000 7fe292400640 1 --2- 172.21.15.26:0/4176306651 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe26c004d00 0x7fe28c0ca9a0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.378+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 >> 172.21.15.26:0/4176306651 conn(0x7fe28c06a7d0 msgr2=0x7fe28c15bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:31.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.378+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 shutdown_connections 2024-01-15T18:11:31.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.378+0000 7fe292400640 1 -- 172.21.15.26:0/4176306651 wait complete. 2024-01-15T18:11:31.403 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs ss1clone1 2024-01-15T18:11:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.619+0000 7f126ee00640 1 Processor -- start 2024-01-15T18:11:31.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.619+0000 7f126ee00640 1 -- start start 2024-01-15T18:11:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.621+0000 7f126ee00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f12680ae5a0 0x7f126815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.621+0000 7f126ee00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1268070c40 con 0x7f12680cf290 2024-01-15T18:11:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.621+0000 7f126ee00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126806c470 con 0x7f12680b3ae0 2024-01-15T18:11:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.621+0000 7f126ee00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1268071450 con 0x7f12680ae5a0 2024-01-15T18:11:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.621+0000 7f126ce00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f12680cf290 0x7f12680b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59016/0 (socket says 172.21.15.26:59016) 2024-01-15T18:11:31.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.621+0000 7f126ce00640 1 -- 172.21.15.26:0/712337368 learned_addr learned my addr 172.21.15.26:0/712337368 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1267e00640 1 --2- 172.21.15.26:0/712337368 >> v2:172.21.15.124:3300/0 conn(0x7f12680ae5a0 0x7f126815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450690823 0 0) 0x7f1268070c40 con 0x7f12680cf290 2024-01-15T18:11:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1248003520 con 0x7f12680cf290 2024-01-15T18:11:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3863313238 0 0) 0x7f1248003520 con 0x7f12680cf290 2024-01-15T18:11:31.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 >> v2:172.21.15.124:3300/0 conn(0x7f12680ae5a0 msgr2=0x7f126815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 --2- 172.21.15.26:0/712337368 >> v2:172.21.15.124:3300/0 conn(0x7f12680ae5a0 0x7f126815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 >> v1:172.21.15.124:6789/0 conn(0x7f12680b3ae0 legacy=0x7f126806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126806c750 con 0x7f12680cf290 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f1260001670 con 0x7f12680cf290 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 >> v1:172.21.15.26:6789/0 conn(0x7f12680cf290 legacy=0x7f12680b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 --2- 172.21.15.26:0/712337368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12480039c0 0x7f1248003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 --2- 172.21.15.26:0/712337368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480047c0 0x7f1248004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 --2- 172.21.15.26:0/712337368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12480051d0 0x7f124800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1268071450 con 0x7f12480047c0 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124800c110 con 0x7f12480039c0 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124800c2d0 con 0x7f12480051d0 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.622+0000 7f1267400640 1 --2- 172.21.15.26:0/712337368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480047c0 0x7f1248004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.623+0000 7f1267400640 1 -- 172.21.15.26:0/712337368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12480051d0 msgr2=0x7f124800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:31.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.623+0000 7f1267400640 1 --2- 172.21.15.26:0/712337368 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f12480051d0 0x7f124800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.623+0000 7f1267400640 1 -- 172.21.15.26:0/712337368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12480039c0 msgr2=0x7f1248003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.623+0000 7f1267400640 1 --2- 172.21.15.26:0/712337368 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12480039c0 0x7f1248003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.623+0000 7f1267400640 1 -- 172.21.15.26:0/712337368 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1248003520 con 0x7f12480047c0 2024-01-15T18:11:31.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.623+0000 7f126ce00640 1 -- 172.21.15.26:0/712337368 reap_dead start 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.624+0000 7f1267400640 1 --2- 172.21.15.26:0/712337368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480047c0 0x7f1248004c40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f1258001e50 tx=0x7f12480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd0659bb83dfb5e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.624+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258003b70 con 0x7f12480047c0 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.624+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1258003cd0 con 0x7f12480047c0 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.624+0000 7f1266a00640 1 -- 172.21.15.26:0/712337368 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258005310 con 0x7f12480047c0 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.625+0000 7f126ee00640 1 -- 172.21.15.26:0/712337368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480047c0 msgr2=0x7f1248004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.625+0000 7f126ee00640 1 --2- 172.21.15.26:0/712337368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480047c0 0x7f1248004c40 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f1258001e50 tx=0x7f12480057f0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.625+0000 7f126ee00640 1 -- 172.21.15.26:0/712337368 shutdown_connections 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.625+0000 7f126ee00640 1 --2- 172.21.15.26:0/712337368 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12480047c0 0x7f1248004c40 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.625+0000 7f126ee00640 1 -- 172.21.15.26:0/712337368 >> 172.21.15.26:0/712337368 conn(0x7f126806a800 msgr2=0x7f126806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:31.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.625+0000 7f126ee00640 1 -- 172.21.15.26:0/712337368 shutdown_connections 2024-01-15T18:11:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.625+0000 7f126ee00640 1 -- 172.21.15.26:0/712337368 wait complete. 2024-01-15T18:11:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.627+0000 7f126ee00640 1 Processor -- start 2024-01-15T18:11:31.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.627+0000 7f126ee00640 1 -- start start 2024-01-15T18:11:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.628+0000 7f126ee00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12480047c0 0x7f12681503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.628+0000 7f126ee00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1268144a40 0x7f126815c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.628+0000 7f126ee00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126815ca60 0x7f126815f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.628+0000 7f126ee00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126815eb20 con 0x7f126815ca60 2024-01-15T18:11:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.628+0000 7f126ee00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126815cef0 con 0x7f12480047c0 2024-01-15T18:11:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.628+0000 7f126ee00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126815d160 con 0x7f1268144a40 2024-01-15T18:11:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.628+0000 7f126ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126815ca60 0x7f126815f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f126ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126815ca60 0x7f126815f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37356/0 (socket says 172.21.15.26:37356) 2024-01-15T18:11:31.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f126ce00640 1 -- 172.21.15.26:0/1366514210 learned_addr learned my addr 172.21.15.26:0/1366514210 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f1267e00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12480047c0 0x7f12681503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f1267400640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1268144a40 0x7f126815c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f126ce00640 1 -- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1268144a40 msgr2=0x7f126815c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f126ce00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1268144a40 0x7f126815c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f126ce00640 1 -- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12480047c0 msgr2=0x7f12681503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f126ce00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12480047c0 0x7f12681503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f126ce00640 1 -- 172.21.15.26:0/1366514210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126815ed30 con 0x7f126815ca60 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f1267e00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12480047c0 0x7f12681503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f126ce00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126815ca60 0x7f126815f810 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f12680cf670 tx=0x7f126000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.629+0000 7f125fe00640 1 -- 172.21.15.26:0/1366514210 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260007440 con 0x7f126815ca60 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.630+0000 7f125fe00640 1 -- 172.21.15.26:0/1366514210 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12600075c0 con 0x7f126815ca60 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.630+0000 7f125fe00640 1 -- 172.21.15.26:0/1366514210 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260004a20 con 0x7f126815ca60 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.630+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126815f010 con 0x7f126815ca60 2024-01-15T18:11:31.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.630+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1268144ec0 con 0x7f126815ca60 2024-01-15T18:11:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.631+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126815f530 con 0x7f126815ca60 2024-01-15T18:11:31.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.632+0000 7f125fe00640 1 -- 172.21.15.26:0/1366514210 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 35) v1 ==== 90967+0+0 (secure 0 0 0) 0x7f1260004b80 con 0x7f126815ca60 2024-01-15T18:11:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.633+0000 7f125fe00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f124006c280 0x7f124006e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:31.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.633+0000 7f125fe00640 1 -- 172.21.15.26:0/1366514210 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1260087c40 con 0x7f126815ca60 2024-01-15T18:11:31.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.634+0000 7f1267e00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f124006c280 0x7f124006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:31.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.635+0000 7f1267e00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f124006c280 0x7f124006e740 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1250006fd0 tx=0x7f1250008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:31.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.637+0000 7f125fe00640 1 -- 172.21.15.26:0/1366514210 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f126815f530 con 0x7f126815ca60 2024-01-15T18:11:31.872 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:31.869+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f1268070550 con 0x7f124006c280 2024-01-15T18:11:33.410 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:33.407+0000 7f125fe00640 1 -- 172.21.15.26:0/1366514210 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7f126000ed60 con 0x7f126815ca60 2024-01-15T18:11:45.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.391+0000 7f125fe00640 1 -- 172.21.15.26:0/1366514210 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1250009530 con 0x7f124006c280 2024-01-15T18:11:45.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f124006c280 msgr2=0x7f124006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f124006c280 0x7f124006e740 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f1250006fd0 tx=0x7f1250008040 comp rx=0 tx=0).stop 2024-01-15T18:11:45.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126815ca60 msgr2=0x7f126815f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126815ca60 0x7f126815f810 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f12680cf670 tx=0x7f126000c8e0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 shutdown_connections 2024-01-15T18:11:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1268144a40 0x7f126815c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f124006c280 0x7f124006e740 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f126815ca60 0x7f126815f810 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 --2- 172.21.15.26:0/1366514210 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12480047c0 0x7f12681503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.395+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 >> 172.21.15.26:0/1366514210 conn(0x7f126806a800 msgr2=0x7f12681592f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:45.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.396+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 shutdown_connections 2024-01-15T18:11:45.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.396+0000 7f126ee00640 1 -- 172.21.15.26:0/1366514210 wait complete. 2024-01-15T18:11:45.423 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs ss1clone1 2024-01-15T18:11:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.634+0000 7f56caa00640 1 Processor -- start 2024-01-15T18:11:45.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.635+0000 7f56caa00640 1 -- start start 2024-01-15T18:11:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.636+0000 7f56caa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f56c4143ae0 0x7f56c40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.636+0000 7f56caa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40cf7d0 0x7f56c415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.636+0000 7f56caa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c40cfc30 con 0x7f56c415e1e0 2024-01-15T18:11:45.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.636+0000 7f56caa00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c40b1ee0 con 0x7f56c4143ae0 2024-01-15T18:11:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.636+0000 7f56caa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c40b2780 con 0x7f56c40cf7d0 2024-01-15T18:11:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.637+0000 7f56c8a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f56c415e1e0 0x7f56c415a670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47866/0 (socket says 172.21.15.26:47866) 2024-01-15T18:11:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.637+0000 7f56c8a00640 1 -- 172.21.15.26:0/505726066 learned_addr learned my addr 172.21.15.26:0/505726066 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.637+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91187221 0 0) 0x7f56c40cfc30 con 0x7f56c415e1e0 2024-01-15T18:11:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.637+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a4003520 con 0x7f56c415e1e0 2024-01-15T18:11:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.637+0000 7f56bb200640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40cf7d0 0x7f56c415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.644 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.637+0000 7f56c3e00640 1 --2- 172.21.15.26:0/505726066 >> v2:172.21.15.124:3300/0 conn(0x7f56c4143ae0 0x7f56c40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2234423207 0 0) 0x7f56a4003520 con 0x7f56c415e1e0 2024-01-15T18:11:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40cf7d0 msgr2=0x7f56c415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40cf7d0 0x7f56c415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 >> v2:172.21.15.124:3300/0 conn(0x7f56c4143ae0 msgr2=0x7f56c40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 --2- 172.21.15.26:0/505726066 >> v2:172.21.15.124:3300/0 conn(0x7f56c4143ae0 0x7f56c40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c415e990 con 0x7f56c415e1e0 2024-01-15T18:11:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f56bc0019f0 con 0x7f56c415e1e0 2024-01-15T18:11:45.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 >> v1:172.21.15.26:6789/0 conn(0x7f56c415e1e0 legacy=0x7f56c415a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003970 0x7f56a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40cf7d0 0x7f56a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4004d00 0x7f56a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c40b1ee0 con 0x7f56a4003970 2024-01-15T18:11:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c40b2780 con 0x7f56a4004d00 2024-01-15T18:11:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a400bab0 con 0x7f56c40cf7d0 2024-01-15T18:11:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3e00640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003970 0x7f56a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.646 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3e00640 1 -- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40cf7d0 msgr2=0x7f56a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56bb200640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40cf7d0 0x7f56a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c8a00640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4004d00 0x7f56a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3e00640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40cf7d0 0x7f56a40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.638+0000 7f56c3e00640 1 -- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4004d00 msgr2=0x7f56a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.639+0000 7f56c3e00640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4004d00 0x7f56a4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.639+0000 7f56c3e00640 1 -- 172.21.15.26:0/505726066 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a4003520 con 0x7f56a4003970 2024-01-15T18:11:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.640+0000 7f56c3e00640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003970 0x7f56a4003d90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f56b400ba30 tx=0x7f56a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf24e9a94440ac77 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.647 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.640+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4032680 con 0x7f56a4003970 2024-01-15T18:11:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.640+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b40327e0 con 0x7f56a4003970 2024-01-15T18:11:45.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.640+0000 7f56c3400640 1 -- 172.21.15.26:0/505726066 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4032bb0 con 0x7f56a4003970 2024-01-15T18:11:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.640+0000 7f56caa00640 1 -- 172.21.15.26:0/505726066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003970 msgr2=0x7f56a4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.640+0000 7f56caa00640 1 --2- 172.21.15.26:0/505726066 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003970 0x7f56a4003d90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f56b400ba30 tx=0x7f56a40057f0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.640+0000 7f56c8a00640 1 -- 172.21.15.26:0/505726066 reap_dead start 2024-01-15T18:11:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.641+0000 7f56caa00640 1 -- 172.21.15.26:0/505726066 shutdown_connections 2024-01-15T18:11:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.641+0000 7f56caa00640 1 -- 172.21.15.26:0/505726066 >> 172.21.15.26:0/505726066 conn(0x7f56c406a800 msgr2=0x7f56c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.641+0000 7f56caa00640 1 -- 172.21.15.26:0/505726066 shutdown_connections 2024-01-15T18:11:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.641+0000 7f56caa00640 1 -- 172.21.15.26:0/505726066 wait complete. 2024-01-15T18:11:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.642+0000 7f56caa00640 1 Processor -- start 2024-01-15T18:11:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.643+0000 7f56caa00640 1 -- start start 2024-01-15T18:11:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.643+0000 7f56caa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4003970 0x7f56c415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.643+0000 7f56caa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c41427a0 0x7f56c415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56caa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c40cc610 0x7f56c40cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56caa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c41432d0 con 0x7f56c40cc610 2024-01-15T18:11:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56caa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4142d10 con 0x7f56a4003970 2024-01-15T18:11:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56caa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4142ff0 con 0x7f56c41427a0 2024-01-15T18:11:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c40cc610 0x7f56c40cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c40cc610 0x7f56c40cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40818/0 (socket says 172.21.15.26:40818) 2024-01-15T18:11:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c8a00640 1 -- 172.21.15.26:0/519495071 learned_addr learned my addr 172.21.15.26:0/519495071 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:45.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56bb200640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c41427a0 0x7f56c415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c3e00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4003970 0x7f56c415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c8a00640 1 -- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c41427a0 msgr2=0x7f56c415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c8a00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c41427a0 0x7f56c415cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c8a00640 1 -- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4003970 msgr2=0x7f56c415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c8a00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4003970 0x7f56c415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c8a00640 1 -- 172.21.15.26:0/519495071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c4157a30 con 0x7f56c40cc610 2024-01-15T18:11:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.644+0000 7f56c3e00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4003970 0x7f56c415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:45.656 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.645+0000 7f56c8a00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c40cc610 0x7f56c40cca90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f56c40cfe50 tx=0x7f56bc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.645+0000 7f56c0c00640 1 -- 172.21.15.26:0/519495071 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc004d60 con 0x7f56c40cc610 2024-01-15T18:11:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.645+0000 7f56c0c00640 1 -- 172.21.15.26:0/519495071 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc00bb80 con 0x7f56c40cc610 2024-01-15T18:11:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.645+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c4157d10 con 0x7f56c40cc610 2024-01-15T18:11:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.645+0000 7f56c0c00640 1 -- 172.21.15.26:0/519495071 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc007070 con 0x7f56c40cc610 2024-01-15T18:11:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.645+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c415d170 con 0x7f56c40cc610 2024-01-15T18:11:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.647+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c4071d00 con 0x7f56c40cc610 2024-01-15T18:11:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.647+0000 7f56c0c00640 1 -- 172.21.15.26:0/519495071 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7f56bc0071d0 con 0x7f56c40cc610 2024-01-15T18:11:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.649+0000 7f56c0c00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c3d0 0x7f569c06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:45.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.650+0000 7f56c0c00640 1 -- 172.21.15.26:0/519495071 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f56bc088b90 con 0x7f56c40cc610 2024-01-15T18:11:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.650+0000 7f56c3e00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c3d0 0x7f569c06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:45.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.651+0000 7f56c3e00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c3d0 0x7f569c06e890 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f56b400ba00 tx=0x7f56b403d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:45.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.657+0000 7f56c0c00640 1 -- 172.21.15.26:0/519495071 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56bc0553c0 con 0x7f56c40cc610 2024-01-15T18:11:45.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:45.902+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f56c4139160 con 0x7f569c06c3d0 2024-01-15T18:11:46.421 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.418+0000 7f56c0c00640 1 -- 172.21.15.26:0/519495071 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+243 (secure 0 0 0) 0x7f56c4139160 con 0x7f569c06c3d0 2024-01-15T18:11:46.422 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:11:46.422 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:11:46.422 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "canceled", 2024-01-15T18:11:46.422 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:11:46.422 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:11:46.422 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:11:46.422 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:11:46.423 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:11:46.423 INFO:teuthology.orchestra.run.smithi026.stdout: "failure": { 2024-01-15T18:11:46.423 INFO:teuthology.orchestra.run.smithi026.stdout: "errno": "4", 2024-01-15T18:11:46.423 INFO:teuthology.orchestra.run.smithi026.stdout: "error_msg": "user interrupted clone operation" 2024-01-15T18:11:46.423 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:11:46.423 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:11:46.423 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:11:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.421+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c3d0 msgr2=0x7f569c06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.421+0000 7f56caa00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c3d0 0x7f569c06e890 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f56b400ba00 tx=0x7f56b403d040 comp rx=0 tx=0).stop 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.421+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c40cc610 msgr2=0x7f56c40cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.421+0000 7f56caa00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c40cc610 0x7f56c40cca90 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f56c40cfe50 tx=0x7f56bc00b4a0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.422+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 shutdown_connections 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.422+0000 7f56caa00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c41427a0 0x7f56c415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.422+0000 7f56caa00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c3d0 0x7f569c06e890 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.422+0000 7f56caa00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c40cc610 0x7f56c40cca90 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.422+0000 7f56caa00640 1 --2- 172.21.15.26:0/519495071 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4003970 0x7f56c415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.422+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 >> 172.21.15.26:0/519495071 conn(0x7f56c406a800 msgr2=0x7f56c406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.422+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 shutdown_connections 2024-01-15T18:11:46.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.422+0000 7f56caa00640 1 -- 172.21.15.26:0/519495071 wait complete. 2024-01-15T18:11:46.447 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format json-pretty 2024-01-15T18:11:46.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.664+0000 7f07fb200640 1 Processor -- start 2024-01-15T18:11:46.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.664+0000 7f07fb200640 1 -- start start 2024-01-15T18:11:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.666+0000 7f07fb200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f4162430 0x7f07f40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.666+0000 7f07fb200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f40702e0 con 0x7f07f40b3650 2024-01-15T18:11:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.666+0000 7f07fb200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f40705c0 con 0x7f07f4162810 2024-01-15T18:11:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.666+0000 7f07fb200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f40708a0 con 0x7f07f4162430 2024-01-15T18:11:46.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.666+0000 7f07fa800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f07f40b3650 0x7f07f406c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47874/0 (socket says 172.21.15.26:47874) 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.666+0000 7f07fa800640 1 -- 172.21.15.26:0/279072084 learned_addr learned my addr 172.21.15.26:0/279072084 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.666+0000 7f07f9e00640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f4162430 0x7f07f40b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273943766 0 0) 0x7f07f40702e0 con 0x7f07f40b3650 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d4003520 con 0x7f07f40b3650 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4156327526 0 0) 0x7f07d4003520 con 0x7f07f40b3650 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f4162430 msgr2=0x7f07f40b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f4162430 0x7f07f40b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 >> v1:172.21.15.124:6789/0 conn(0x7f07f4162810 legacy=0x7f07f4071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f4070b80 con 0x7f07f40b3650 2024-01-15T18:11:46.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f07ec0019f0 con 0x7f07f40b3650 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 >> v1:172.21.15.26:6789/0 conn(0x7f07f40b3650 legacy=0x7f07f406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f4162430 0x7f07d40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4003f20 0x7f07d40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07d4004d00 0x7f07d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f40708a0 con 0x7f07d4004d00 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d400ba60 con 0x7f07d4003f20 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.667+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d400bc20 con 0x7f07f4162430 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.668+0000 7f07fa800640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07d4004d00 0x7f07d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.668+0000 7f07f9e00640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f4162430 0x7f07d40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:46.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.668+0000 7f07f9400640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4003f20 0x7f07d40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.668+0000 7f07fa800640 1 -- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f4162430 msgr2=0x7f07d40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.668+0000 7f07fa800640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f4162430 0x7f07d40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.668+0000 7f07fa800640 1 -- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4003f20 msgr2=0x7f07d40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:46.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.668+0000 7f07fa800640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4003f20 0x7f07d40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.668+0000 7f07fa800640 1 -- 172.21.15.26:0/279072084 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d4003520 con 0x7f07d4004d00 2024-01-15T18:11:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.668+0000 7f07f9400640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4003f20 0x7f07d40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.670+0000 7f07fa800640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07d4004d00 0x7f07d4005180 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f07ec00a830 tx=0x7f07d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb66c9c464f9416e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.670+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec0072d0 con 0x7f07d4004d00 2024-01-15T18:11:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.670+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec02f070 con 0x7f07d4004d00 2024-01-15T18:11:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.670+0000 7f07f8a00640 1 -- 172.21.15.26:0/279072084 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec0076f0 con 0x7f07d4004d00 2024-01-15T18:11:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.671+0000 7f07fb200640 1 -- 172.21.15.26:0/279072084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07d4004d00 msgr2=0x7f07d4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.671+0000 7f07fb200640 1 --2- 172.21.15.26:0/279072084 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07d4004d00 0x7f07d4005180 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f07ec00a830 tx=0x7f07d40057f0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.671+0000 7f07fa800640 1 -- 172.21.15.26:0/279072084 reap_dead start 2024-01-15T18:11:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.671+0000 7f07fb200640 1 -- 172.21.15.26:0/279072084 shutdown_connections 2024-01-15T18:11:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.671+0000 7f07fb200640 1 -- 172.21.15.26:0/279072084 >> 172.21.15.26:0/279072084 conn(0x7f07f406a800 msgr2=0x7f07f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.671+0000 7f07fb200640 1 -- 172.21.15.26:0/279072084 shutdown_connections 2024-01-15T18:11:46.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.671+0000 7f07fb200640 1 -- 172.21.15.26:0/279072084 wait complete. 2024-01-15T18:11:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.673+0000 7f07fb200640 1 Processor -- start 2024-01-15T18:11:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.673+0000 7f07fb200640 1 -- start start 2024-01-15T18:11:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.674+0000 7f07fb200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4004d00 0x7f07f415f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.674+0000 7f07fb200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415f990 0x7f07f41fa720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.674+0000 7f07fb200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f41fac60 0x7f07f41fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:46.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.674+0000 7f07fb200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f415fe10 con 0x7f07f415f990 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.674+0000 7f07fb200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f415d150 con 0x7f07d4004d00 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.674+0000 7f07fb200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f415b770 con 0x7f07f41fac60 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.674+0000 7f07f9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415f990 0x7f07f41fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.674+0000 7f07f9400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415f990 0x7f07f41fa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40828/0 (socket says 172.21.15.26:40828) 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.674+0000 7f07f9400640 1 -- 172.21.15.26:0/886540198 learned_addr learned my addr 172.21.15.26:0/886540198 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07fa800640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f41fac60 0x7f07f41fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f9400640 1 -- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f41fac60 msgr2=0x7f07f41fb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f9e00640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4004d00 0x7f07f415f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f9400640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f41fac60 0x7f07f41fb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f9400640 1 -- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4004d00 msgr2=0x7f07f415f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f9400640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4004d00 0x7f07f415f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f9400640 1 -- 172.21.15.26:0/886540198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f415ba50 con 0x7f07f415f990 2024-01-15T18:11:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f9400640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415f990 0x7f07f41fa720 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00e550 tx=0x7f07dc00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f2000640 1 -- 172.21.15.26:0/886540198 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc0052e0 con 0x7f07f415f990 2024-01-15T18:11:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f2000640 1 -- 172.21.15.26:0/886540198 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc005440 con 0x7f07f415f990 2024-01-15T18:11:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07f2000640 1 -- 172.21.15.26:0/886540198 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc003830 con 0x7f07f415f990 2024-01-15T18:11:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.675+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f415cb00 con 0x7f07f415f990 2024-01-15T18:11:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.676+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f415a310 con 0x7f07f415f990 2024-01-15T18:11:46.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.677+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f4067450 con 0x7f07f415f990 2024-01-15T18:11:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.678+0000 7f07f2000640 1 -- 172.21.15.26:0/886540198 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7f07dc003990 con 0x7f07f415f990 2024-01-15T18:11:46.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.679+0000 7f07f2000640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07cc06c3d0 0x7f07cc06e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:46.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.679+0000 7f07f2000640 1 -- 172.21.15.26:0/886540198 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f07dc088460 con 0x7f07f415f990 2024-01-15T18:11:46.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.679+0000 7f07f9e00640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07cc06c3d0 0x7f07cc06e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:46.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.680+0000 7f07f9e00640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07cc06c3d0 0x7f07cc06e890 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f07e40099e0 tx=0x7f07e4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:46.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.683+0000 7f07f2000640 1 -- 172.21.15.26:0/886540198 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07dc054d20 con 0x7f07f415f990 2024-01-15T18:11:46.948 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.945+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f07f4067c80 con 0x7f07f415f990 2024-01-15T18:11:46.949 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.946+0000 7f07f2000640 1 -- 172.21.15.26:0/886540198 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1855 (secure 0 0 0) 0x7f07dc058340 con 0x7f07f415f990 2024-01-15T18:11:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:11:46.950 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:11:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: "fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:11:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: "health": { 2024-01-15T18:11:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "HEALTH_OK", 2024-01-15T18:11:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: "checks": {}, 2024-01-15T18:11:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: "mutes": [] 2024-01-15T18:11:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:11:46.950 INFO:teuthology.orchestra.run.smithi026.stdout: "election_epoch": 8, 2024-01-15T18:11:46.951 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum": [ 2024-01-15T18:11:46.951 INFO:teuthology.orchestra.run.smithi026.stdout: 0, 2024-01-15T18:11:46.951 INFO:teuthology.orchestra.run.smithi026.stdout: 1, 2024-01-15T18:11:46.951 INFO:teuthology.orchestra.run.smithi026.stdout: 2 2024-01-15T18:11:46.951 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:11:46.951 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_names": [ 2024-01-15T18:11:46.951 INFO:teuthology.orchestra.run.smithi026.stdout: "a", 2024-01-15T18:11:46.951 INFO:teuthology.orchestra.run.smithi026.stdout: "b", 2024-01-15T18:11:46.951 INFO:teuthology.orchestra.run.smithi026.stdout: "c" 2024-01-15T18:11:46.952 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:11:46.952 INFO:teuthology.orchestra.run.smithi026.stdout: "quorum_age": 1021, 2024-01-15T18:11:46.952 INFO:teuthology.orchestra.run.smithi026.stdout: "monmap": { 2024-01-15T18:11:46.952 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 1, 2024-01-15T18:11:46.952 INFO:teuthology.orchestra.run.smithi026.stdout: "min_mon_release_name": "reef", 2024-01-15T18:11:46.952 INFO:teuthology.orchestra.run.smithi026.stdout: "num_mons": 3 2024-01-15T18:11:46.952 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:11:46.952 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap": { 2024-01-15T18:11:46.952 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 102, 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: "num_osds": 8, 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: "num_up_osds": 8, 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_up_since": 1705341296, 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: "num_in_osds": 8, 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_in_since": 1705341288, 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: "num_remapped_pgs": 0 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: "pgmap": { 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: "pgs_by_state": [ 2024-01-15T18:11:46.953 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: "state_name": "active+clean", 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: "count": 97 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pgs": 97, 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: "num_pools": 3, 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: "num_objects": 797, 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: "data_bytes": 3222115851, 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_used": 6990798848, 2024-01-15T18:11:46.954 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_avail": 766103314432, 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: "bytes_total": 773094113280, 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: "read_bytes_sec": 170, 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: "write_bytes_sec": 2814, 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: "read_op_per_sec": 0, 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: "write_op_per_sec": 1 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: "fsmap": { 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 64, 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 7, 2024-01-15T18:11:46.955 INFO:teuthology.orchestra.run.smithi026.stdout: "up": 1, 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: "in": 1, 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: "max": 1, 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: "by_rank": [ 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: "filesystem_id": 7, 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: "rank": 0, 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "d", 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: "status": "up:active", 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: "gid": 36242 2024-01-15T18:11:46.956 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: "up:standby": 3 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: "mgrmap": { 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: "available": true, 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: "num_standbys": 1, 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: "modules": [ 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: "iostat", 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: "nfs", 2024-01-15T18:11:46.957 INFO:teuthology.orchestra.run.smithi026.stdout: "restful" 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout: ], 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout: "servicemap": { 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout: "epoch": 22, 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout: "modified": "2024-01-15T18:11:26.297000+0000", 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout: "services": {} 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout: "progress_events": {} 2024-01-15T18:11:46.958 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:11:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07cc06c3d0 msgr2=0x7f07cc06e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07cc06c3d0 0x7f07cc06e890 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f07e40099e0 tx=0x7f07e4008040 comp rx=0 tx=0).stop 2024-01-15T18:11:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415f990 msgr2=0x7f07f41fa720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415f990 0x7f07f41fa720 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00e550 tx=0x7f07dc00ea10 comp rx=0 tx=0).stop 2024-01-15T18:11:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 shutdown_connections 2024-01-15T18:11:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f07f41fac60 0x7f07f41fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07cc06c3d0 0x7f07cc06e890 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f07f415f990 0x7f07f41fa720 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 --2- 172.21.15.26:0/886540198 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f07d4004d00 0x7f07f415f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:46.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 >> 172.21.15.26:0/886540198 conn(0x7f07f406a800 msgr2=0x7f07f40afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:46.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 shutdown_connections 2024-01-15T18:11:46.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:46.949+0000 7f07fb200640 1 -- 172.21.15.26:0/886540198 wait complete. 2024-01-15T18:11:46.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:46.971 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -lh volumes/_nogroup/ss1clone1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9) 2024-01-15T18:11:47.029 INFO:teuthology.orchestra.run.smithi026.stderr:ls: cannot access 'volumes/_nogroup/ss1clone1/7d50dc34-647b-4d1e-b01c-34f713ad4bb9': No such file or directory 2024-01-15T18:11:47.031 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-01-15T18:11:47.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:11:47.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.244+0000 7fb0cae00640 1 Processor -- start 2024-01-15T18:11:47.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.244+0000 7fb0cae00640 1 -- start start 2024-01-15T18:11:47.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.245+0000 7fb0cae00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb0c4143c40 0x7fb0c4071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.245+0000 7fb0cae00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0c40717d0 0x7fb0c40cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.245+0000 7fb0cae00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c4070e50 con 0x7fb0c40cca40 2024-01-15T18:11:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.245+0000 7fb0cae00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c40723c0 con 0x7fb0c40717d0 2024-01-15T18:11:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.245+0000 7fb0cae00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c40cce40 con 0x7fb0c4143c40 2024-01-15T18:11:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.246+0000 7fb0c9000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb0c40717d0 0x7fb0c40cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.246+0000 7fb0ca400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb0c40cca40 0x7fb0c415eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47890/0 (socket says 172.21.15.26:47890) 2024-01-15T18:11:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.246+0000 7fb0ca400640 1 -- 172.21.15.26:0/627035839 learned_addr learned my addr 172.21.15.26:0/627035839 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.246+0000 7fb0c9a00640 1 --2- 172.21.15.26:0/627035839 >> v2:172.21.15.124:3300/0 conn(0x7fb0c4143c40 0x7fb0c4071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.246+0000 7fb0c9000640 1 -- 172.21.15.26:0/627035839 >> v2:172.21.15.124:3300/0 conn(0x7fb0c4143c40 msgr2=0x7fb0c4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:47.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.246+0000 7fb0c9000640 1 --2- 172.21.15.26:0/627035839 >> v2:172.21.15.124:3300/0 conn(0x7fb0c4143c40 0x7fb0c4071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.246+0000 7fb0c9000640 1 -- 172.21.15.26:0/627035839 >> v1:172.21.15.26:6789/0 conn(0x7fb0c40cca40 legacy=0x7fb0c415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.247+0000 7fb0c9000640 1 -- 172.21.15.26:0/627035839 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c406e170 con 0x7fb0c40717d0 2024-01-15T18:11:47.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.248+0000 7fb0c9000640 1 --2- 172.21.15.26:0/627035839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 0x7fb0c40cc500 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb0c406e410 tx=0x7fb0ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbf41a82be2a6d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.248+0000 7fb0c3e00640 1 -- 172.21.15.26:0/627035839 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac02e070 con 0x7fb0c40717d0 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.248+0000 7fb0c3e00640 1 -- 172.21.15.26:0/627035839 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac0315a0 con 0x7fb0c40717d0 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.248+0000 7fb0c3e00640 1 -- 172.21.15.26:0/627035839 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac031930 con 0x7fb0c40717d0 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.248+0000 7fb0cae00640 1 -- 172.21.15.26:0/627035839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 msgr2=0x7fb0c40cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.248+0000 7fb0cae00640 1 --2- 172.21.15.26:0/627035839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 0x7fb0c40cc500 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fb0c406e410 tx=0x7fb0ac02b840 comp rx=0 tx=0).stop 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.249+0000 7fb0cae00640 1 -- 172.21.15.26:0/627035839 shutdown_connections 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.249+0000 7fb0cae00640 1 --2- 172.21.15.26:0/627035839 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 0x7fb0c40cc500 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.249+0000 7fb0cae00640 1 --2- 172.21.15.26:0/627035839 >> v2:172.21.15.124:3300/0 conn(0x7fb0c4143c40 0x7fb0c4071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.249+0000 7fb0cae00640 1 -- 172.21.15.26:0/627035839 >> 172.21.15.26:0/627035839 conn(0x7fb0c406a800 msgr2=0x7fb0c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:47.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.249+0000 7fb0cae00640 1 -- 172.21.15.26:0/627035839 shutdown_connections 2024-01-15T18:11:47.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.249+0000 7fb0cae00640 1 -- 172.21.15.26:0/627035839 wait complete. 2024-01-15T18:11:47.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.250+0000 7fb0cae00640 1 Processor -- start 2024-01-15T18:11:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.251+0000 7fb0cae00640 1 -- start start 2024-01-15T18:11:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.251+0000 7fb0cae00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 0x7fb0c406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.251+0000 7fb0cae00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0c40cca40 0x7fb0c413bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.251+0000 7fb0cae00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0c4143c40 0x7fb0c413c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.251+0000 7fb0cae00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4156ba0 con 0x7fb0c40717d0 2024-01-15T18:11:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.251+0000 7fb0cae00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40b3f40 con 0x7fb0c40cca40 2024-01-15T18:11:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.251+0000 7fb0cae00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40b1ca0 con 0x7fb0c4143c40 2024-01-15T18:11:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 0x7fb0c406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 0x7fb0c406d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40840/0 (socket says 172.21.15.26:40840) 2024-01-15T18:11:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9a00640 1 -- 172.21.15.26:0/858612629 learned_addr learned my addr 172.21.15.26:0/858612629 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0ca400640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0c4143c40 0x7fb0c413c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9000640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0c40cca40 0x7fb0c413bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9a00640 1 -- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0c4143c40 msgr2=0x7fb0c413c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9a00640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0c4143c40 0x7fb0c413c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9a00640 1 -- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0c40cca40 msgr2=0x7fb0c413bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9a00640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0c40cca40 0x7fb0c413bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9a00640 1 -- 172.21.15.26:0/858612629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac009400 con 0x7fb0c40717d0 2024-01-15T18:11:47.258 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0ca400640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0c4143c40 0x7fb0c413c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:47.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.252+0000 7fb0c9000640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0c40cca40 0x7fb0c413bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:47.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.253+0000 7fb0c9a00640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 0x7fb0c406d2d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400d5f0 tx=0x7fb0b400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:47.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.253+0000 7fb0c1600640 1 -- 172.21.15.26:0/858612629 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40125f0 con 0x7fb0c40717d0 2024-01-15T18:11:47.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.253+0000 7fb0c1600640 1 -- 172.21.15.26:0/858612629 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b4012750 con 0x7fb0c40717d0 2024-01-15T18:11:47.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.253+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c40b1f80 con 0x7fb0c40717d0 2024-01-15T18:11:47.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.253+0000 7fb0c1600640 1 -- 172.21.15.26:0/858612629 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4012b00 con 0x7fb0c40717d0 2024-01-15T18:11:47.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.253+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c40b2540 con 0x7fb0c40717d0 2024-01-15T18:11:47.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.254+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c40b26d0 con 0x7fb0c40717d0 2024-01-15T18:11:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.256+0000 7fb0c1600640 1 -- 172.21.15.26:0/858612629 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7fb0b4012c60 con 0x7fb0c40717d0 2024-01-15T18:11:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.257+0000 7fb0c1600640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb09c06c3a0 0x7fb09c06e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.257+0000 7fb0c1600640 1 -- 172.21.15.26:0/858612629 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb0b4088c40 con 0x7fb0c40717d0 2024-01-15T18:11:47.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.257+0000 7fb0c9000640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb09c06c3a0 0x7fb09c06e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.262 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.259+0000 7fb0c9000640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb09c06c3a0 0x7fb09c06e860 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac036000 tx=0x7fb0ac03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:47.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.261+0000 7fb0c1600640 1 -- 172.21.15.26:0/858612629 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb0b4055500 con 0x7fb0c40717d0 2024-01-15T18:11:47.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.481+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0c40720e0 con 0x7fb09c06c3a0 2024-01-15T18:11:47.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.484+0000 7fb0c1600640 1 -- 172.21.15.26:0/858612629 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fb0ac031ae0 con 0x7fb09c06c3a0 2024-01-15T18:11:47.488 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:11:47.488 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:11:47.489 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:11:47.489 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:11:47.489 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:11:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb09c06c3a0 msgr2=0x7fb09c06e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb09c06c3a0 0x7fb09c06e860 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac036000 tx=0x7fb0ac03d040 comp rx=0 tx=0).stop 2024-01-15T18:11:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 msgr2=0x7fb0c406d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 0x7fb0c406d2d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400d5f0 tx=0x7fb0b400dab0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 shutdown_connections 2024-01-15T18:11:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0c4143c40 0x7fb0c413c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb09c06c3a0 0x7fb09c06e860 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb0c40cca40 0x7fb0c413bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 --2- 172.21.15.26:0/858612629 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb0c40717d0 0x7fb0c406d2d0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.487+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 >> 172.21.15.26:0/858612629 conn(0x7fb0c406a800 msgr2=0x7fb0c40aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.488+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 shutdown_connections 2024-01-15T18:11:47.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.488+0000 7fb0cae00640 1 -- 172.21.15.26:0/858612629 wait complete. 2024-01-15T18:11:47.510 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:11:47.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.732+0000 7f0046c00640 1 Processor -- start 2024-01-15T18:11:47.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.732+0000 7f0046c00640 1 -- start start 2024-01-15T18:11:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.734+0000 7f0046c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f00400cf9a0 0x7f0040070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.734+0000 7f0046c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0040070ea0 0x7f00400cc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.734+0000 7f0046c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0040071690 con 0x7f004014b2c0 2024-01-15T18:11:47.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.734+0000 7f0046c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004014b6a0 con 0x7f00400cf9a0 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.734+0000 7f0046c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0040071970 con 0x7f0040070ea0 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.734+0000 7f003fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f004014b2c0 0x7f00400cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47906/0 (socket says 172.21.15.26:47906) 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.734+0000 7f003fe00640 1 -- 172.21.15.26:0/82590836 learned_addr learned my addr 172.21.15.26:0/82590836 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.734+0000 7f0037e00640 1 --2- 172.21.15.26:0/82590836 >> v2:172.21.15.26:3300/0 conn(0x7f00400cf9a0 0x7f0040070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.735+0000 7f0044c00640 1 --2- 172.21.15.26:0/82590836 >> v2:172.21.15.124:3300/0 conn(0x7f0040070ea0 0x7f00400cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.735+0000 7f003f400640 1 -- 172.21.15.26:0/82590836 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290101775 0 0) 0x7f0040071690 con 0x7f004014b2c0 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.735+0000 7f003f400640 1 -- 172.21.15.26:0/82590836 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0020003520 con 0x7f004014b2c0 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.735+0000 7f0037e00640 1 -- 172.21.15.26:0/82590836 >> v2:172.21.15.124:3300/0 conn(0x7f0040070ea0 msgr2=0x7f00400cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.735+0000 7f0037e00640 1 --2- 172.21.15.26:0/82590836 >> v2:172.21.15.124:3300/0 conn(0x7f0040070ea0 0x7f00400cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.735+0000 7f0037e00640 1 -- 172.21.15.26:0/82590836 >> v1:172.21.15.26:6789/0 conn(0x7f004014b2c0 legacy=0x7f00400cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.735+0000 7f0037e00640 1 -- 172.21.15.26:0/82590836 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0040071c50 con 0x7f00400cf9a0 2024-01-15T18:11:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.736+0000 7f0037e00640 1 --2- 172.21.15.26:0/82590836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00400cf9a0 0x7f0040070960 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f0040165330 tx=0x7f002802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7720bbf12fa4aac0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.736+0000 7f003f400640 1 -- 172.21.15.26:0/82590836 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028031830 con 0x7f00400cf9a0 2024-01-15T18:11:47.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.736+0000 7f003f400640 1 -- 172.21.15.26:0/82590836 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0028031990 con 0x7f00400cf9a0 2024-01-15T18:11:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.736+0000 7f003f400640 1 -- 172.21.15.26:0/82590836 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0028031d00 con 0x7f00400cf9a0 2024-01-15T18:11:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.737+0000 7f0046c00640 1 -- 172.21.15.26:0/82590836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00400cf9a0 msgr2=0x7f0040070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.737+0000 7f0046c00640 1 --2- 172.21.15.26:0/82590836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00400cf9a0 0x7f0040070960 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f0040165330 tx=0x7f002802b8d0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.737+0000 7f0046c00640 1 -- 172.21.15.26:0/82590836 shutdown_connections 2024-01-15T18:11:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.737+0000 7f0046c00640 1 --2- 172.21.15.26:0/82590836 >> v2:172.21.15.124:3300/0 conn(0x7f0040070ea0 0x7f00400cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.737+0000 7f0046c00640 1 --2- 172.21.15.26:0/82590836 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f00400cf9a0 0x7f0040070960 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.737+0000 7f0046c00640 1 -- 172.21.15.26:0/82590836 >> 172.21.15.26:0/82590836 conn(0x7f004006a800 msgr2=0x7f004006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.737+0000 7f0046c00640 1 -- 172.21.15.26:0/82590836 shutdown_connections 2024-01-15T18:11:47.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.737+0000 7f0046c00640 1 -- 172.21.15.26:0/82590836 wait complete. 2024-01-15T18:11:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.739+0000 7f0046c00640 1 Processor -- start 2024-01-15T18:11:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.739+0000 7f0046c00640 1 -- start start 2024-01-15T18:11:47.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.740+0000 7f0046c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0040070ea0 0x7f004015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.740+0000 7f0046c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00400cf9a0 0x7f00401642d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.740+0000 7f0046c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004014b2c0 0x7f0040164810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.740+0000 7f0046c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040163730 con 0x7f004014b2c0 2024-01-15T18:11:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.740+0000 7f0046c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0040162b20 con 0x7f00400cf9a0 2024-01-15T18:11:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.740+0000 7f0046c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0040162e00 con 0x7f0040070ea0 2024-01-15T18:11:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.740+0000 7f0044c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004014b2c0 0x7f0040164810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.740+0000 7f0044c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004014b2c0 0x7f0040164810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40870/0 (socket says 172.21.15.26:40870) 2024-01-15T18:11:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.740+0000 7f0044c00640 1 -- 172.21.15.26:0/3200892738 learned_addr learned my addr 172.21.15.26:0/3200892738 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:47.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f0037e00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00400cf9a0 0x7f00401642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f003fe00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0040070ea0 0x7f004015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f0044c00640 1 -- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0040070ea0 msgr2=0x7f004015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f0044c00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0040070ea0 0x7f004015c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f0044c00640 1 -- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00400cf9a0 msgr2=0x7f00401642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f0044c00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00400cf9a0 0x7f00401642d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f0044c00640 1 -- 172.21.15.26:0/3200892738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0028009400 con 0x7f004014b2c0 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f0037e00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00400cf9a0 0x7f00401642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f0044c00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004014b2c0 0x7f0040164810 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f0038007e40 tx=0x7f003800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f003cc00640 1 -- 172.21.15.26:0/3200892738 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038014620 con 0x7f004014b2c0 2024-01-15T18:11:47.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f003cc00640 1 -- 172.21.15.26:0/3200892738 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003800ee90 con 0x7f004014b2c0 2024-01-15T18:11:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f003cc00640 1 -- 172.21.15.26:0/3200892738 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380149e0 con 0x7f004014b2c0 2024-01-15T18:11:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.741+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00401630e0 con 0x7f004014b2c0 2024-01-15T18:11:47.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.742+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0040162830 con 0x7f004014b2c0 2024-01-15T18:11:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.743+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00401638a0 con 0x7f004014b2c0 2024-01-15T18:11:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.744+0000 7f003cc00640 1 -- 172.21.15.26:0/3200892738 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7f0038014b40 con 0x7f004014b2c0 2024-01-15T18:11:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.745+0000 7f003cc00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f001806c3a0 0x7f001806e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.745+0000 7f003cc00640 1 -- 172.21.15.26:0/3200892738 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0038089150 con 0x7f004014b2c0 2024-01-15T18:11:47.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.746+0000 7f003fe00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f001806c3a0 0x7f001806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:47.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.747+0000 7f003fe00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f001806c3a0 0x7f001806e860 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0030000f60 tx=0x7f0030009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:47.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.750+0000 7f003cc00640 1 -- 172.21.15.26:0/3200892738 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0038055a50 con 0x7f004014b2c0 2024-01-15T18:11:47.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.974+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f004013ffe0 con 0x7f001806c3a0 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.985+0000 7f003cc00640 1 -- 172.21.15.26:0/3200892738 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+79 (secure 0 0 0) 0x7f004013ffe0 con 0x7f001806c3a0 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sv1" 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1clone1" 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:11:47.988 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f001806c3a0 msgr2=0x7f001806e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f001806c3a0 0x7f001806e860 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f0030000f60 tx=0x7f0030009040 comp rx=0 tx=0).stop 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004014b2c0 msgr2=0x7f0040164810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004014b2c0 0x7f0040164810 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f0038007e40 tx=0x7f003800e4b0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 shutdown_connections 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f001806c3a0 0x7f001806e860 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f004014b2c0 0x7f0040164810 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f00400cf9a0 0x7f00401642d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 --2- 172.21.15.26:0/3200892738 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0040070ea0 0x7f004015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.987+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 >> 172.21.15.26:0/3200892738 conn(0x7f004006a800 msgr2=0x7f0040149340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.988+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 shutdown_connections 2024-01-15T18:11:47.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:47.988+0000 7f0046c00640 1 -- 172.21.15.26:0/3200892738 wait complete. 2024-01-15T18:11:48.011 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs sv1 --format json 2024-01-15T18:11:48.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.239+0000 7fc606800640 1 Processor -- start 2024-01-15T18:11:48.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.239+0000 7fc606800640 1 -- start start 2024-01-15T18:11:48.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.241+0000 7fc606800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc6000ce310 0x7fc6000ced50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.241+0000 7fc606800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6000d0640 0x7fc600166f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.241+0000 7fc606800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6000b1430 con 0x7fc6000cd820 2024-01-15T18:11:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.241+0000 7fc606800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6000d0a50 con 0x7fc6000d0640 2024-01-15T18:11:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.241+0000 7fc606800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6000d0c40 con 0x7fc6000ce310 2024-01-15T18:11:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.241+0000 7fc605e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6000d0640 0x7fc600166f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.241+0000 7fc605e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc6000d0640 0x7fc600166f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40878/0 (socket says 172.21.15.26:40878) 2024-01-15T18:11:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.241+0000 7fc605e00640 1 -- 172.21.15.26:0/2992332180 learned_addr learned my addr 172.21.15.26:0/2992332180 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.242+0000 7fc604a00640 1 --2- 172.21.15.26:0/2992332180 >> v2:172.21.15.124:3300/0 conn(0x7fc6000ce310 0x7fc6000ced50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.242+0000 7fc605e00640 1 -- 172.21.15.26:0/2992332180 >> v2:172.21.15.124:3300/0 conn(0x7fc6000ce310 msgr2=0x7fc6000ced50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.242+0000 7fc605e00640 1 --2- 172.21.15.26:0/2992332180 >> v2:172.21.15.124:3300/0 conn(0x7fc6000ce310 0x7fc6000ced50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.242+0000 7fc605e00640 1 -- 172.21.15.26:0/2992332180 >> v1:172.21.15.124:6789/0 conn(0x7fc6000cd820 legacy=0x7fc6000cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.242+0000 7fc605e00640 1 -- 172.21.15.26:0/2992332180 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000d0f20 con 0x7fc6000d0640 2024-01-15T18:11:48.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.243+0000 7fc605e00640 1 --2- 172.21.15.26:0/2992332180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000d0640 0x7fc600166f20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc6000cebf0 tx=0x7fc5f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc63142527f9274 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.244+0000 7fc5ffe00640 1 -- 172.21.15.26:0/2992332180 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f80317e0 con 0x7fc6000d0640 2024-01-15T18:11:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.244+0000 7fc5ffe00640 1 -- 172.21.15.26:0/2992332180 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f8031940 con 0x7fc6000d0640 2024-01-15T18:11:48.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.244+0000 7fc5ffe00640 1 -- 172.21.15.26:0/2992332180 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8031cd0 con 0x7fc6000d0640 2024-01-15T18:11:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.244+0000 7fc606800640 1 -- 172.21.15.26:0/2992332180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000d0640 msgr2=0x7fc600166f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.244+0000 7fc606800640 1 --2- 172.21.15.26:0/2992332180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000d0640 0x7fc600166f20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fc6000cebf0 tx=0x7fc5f802b880 comp rx=0 tx=0).stop 2024-01-15T18:11:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.245+0000 7fc606800640 1 -- 172.21.15.26:0/2992332180 shutdown_connections 2024-01-15T18:11:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.245+0000 7fc606800640 1 --2- 172.21.15.26:0/2992332180 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000d0640 0x7fc600166f20 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.245+0000 7fc606800640 1 --2- 172.21.15.26:0/2992332180 >> v2:172.21.15.124:3300/0 conn(0x7fc6000ce310 0x7fc6000ced50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.245+0000 7fc606800640 1 -- 172.21.15.26:0/2992332180 >> 172.21.15.26:0/2992332180 conn(0x7fc600099a60 msgr2=0x7fc600099e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.245+0000 7fc606800640 1 -- 172.21.15.26:0/2992332180 shutdown_connections 2024-01-15T18:11:48.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.245+0000 7fc606800640 1 -- 172.21.15.26:0/2992332180 wait complete. 2024-01-15T18:11:48.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.247+0000 7fc606800640 1 Processor -- start 2024-01-15T18:11:48.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.247+0000 7fc606800640 1 -- start start 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.248+0000 7fc606800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6000cd820 0x7fc6001f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.248+0000 7fc606800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000ce310 0x7fc6000af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.248+0000 7fc606800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6000d0640 0x7fc6001f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.248+0000 7fc606800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001f7480 con 0x7fc6000ce310 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.248+0000 7fc606800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000afbe0 con 0x7fc6000d0640 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.248+0000 7fc606800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001f6be0 con 0x7fc6000cd820 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.248+0000 7fc604a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000ce310 0x7fc6000af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.248+0000 7fc604a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000ce310 0x7fc6000af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40886/0 (socket says 172.21.15.26:40886) 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.248+0000 7fc604a00640 1 -- 172.21.15.26:0/1490845226 learned_addr learned my addr 172.21.15.26:0/1490845226 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc605e00640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6000d0640 0x7fc6001f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc605400640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6000cd820 0x7fc6001f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc604a00640 1 -- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6000cd820 msgr2=0x7fc6001f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:48.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc604a00640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6000cd820 0x7fc6001f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc604a00640 1 -- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6000d0640 msgr2=0x7fc6001f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc604a00640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6000d0640 0x7fc6001f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc604a00640 1 -- 172.21.15.26:0/1490845226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f80094d0 con 0x7fc6000ce310 2024-01-15T18:11:48.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc604a00640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000ce310 0x7fc6000af6a0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800d5f0 tx=0x7fc5e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:48.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc5fd600640 1 -- 172.21.15.26:0/1490845226 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80125f0 con 0x7fc6000ce310 2024-01-15T18:11:48.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc5fd600640 1 -- 172.21.15.26:0/1490845226 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8012750 con 0x7fc6000ce310 2024-01-15T18:11:48.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc5fd600640 1 -- 172.21.15.26:0/1490845226 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8012b00 con 0x7fc6000ce310 2024-01-15T18:11:48.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.249+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6001f6ec0 con 0x7fc6000ce310 2024-01-15T18:11:48.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.250+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60015dae0 con 0x7fc6000ce310 2024-01-15T18:11:48.254 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.251+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60015d940 con 0x7fc6000ce310 2024-01-15T18:11:48.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.252+0000 7fc5fd600640 1 -- 172.21.15.26:0/1490845226 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7fc5e8012c60 con 0x7fc6000ce310 2024-01-15T18:11:48.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.253+0000 7fc5fd600640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5d806c3a0 0x7fc5d806e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.253+0000 7fc5fd600640 1 -- 172.21.15.26:0/1490845226 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc5e8088c10 con 0x7fc6000ce310 2024-01-15T18:11:48.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.253+0000 7fc605400640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5d806c3a0 0x7fc5d806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.257 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.255+0000 7fc605400640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5d806c3a0 0x7fc5d806e860 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc60015ed70 tx=0x7fc5f00094d0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:48.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.257+0000 7fc5fd600640 1 -- 172.21.15.26:0/1490845226 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc60015d940 con 0x7fc6000ce310 2024-01-15T18:11:48.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.487+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6000d17c0 con 0x7fc5d806c3a0 2024-01-15T18:11:48.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.520+0000 7fc5fd600640 1 -- 172.21.15.26:0/1490845226 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fc5f000ce90 con 0x7fc5d806c3a0 2024-01-15T18:11:48.523 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:11:48.523 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:11:48.523 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:11:48.523 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "ss1" 2024-01-15T18:11:48.523 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:11:48.523 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:11:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5d806c3a0 msgr2=0x7fc5d806e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:48.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5d806c3a0 0x7fc5d806e860 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc60015ed70 tx=0x7fc5f00094d0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000ce310 msgr2=0x7fc6000af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000ce310 0x7fc6000af6a0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800d5f0 tx=0x7fc5e800dab0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 shutdown_connections 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc5d806c3a0 0x7fc5d806e860 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6000d0640 0x7fc6001f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6000ce310 0x7fc6000af6a0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 --2- 172.21.15.26:0/1490845226 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6000cd820 0x7fc6001f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 >> 172.21.15.26:0/1490845226 conn(0x7fc600099a60 msgr2=0x7fc6000733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.522+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 shutdown_connections 2024-01-15T18:11:48.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.523+0000 7fc606800640 1 -- 172.21.15.26:0/1490845226 wait complete. 2024-01-15T18:11:48.545 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm --force --format json cephfs sv1 ss1 2024-01-15T18:11:48.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.763+0000 7f8e46800640 1 Processor -- start 2024-01-15T18:11:48.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.764+0000 7f8e46800640 1 -- start start 2024-01-15T18:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.765+0000 7f8e46800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8e4009a170 0x7f8e400b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.765+0000 7f8e46800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e4006c6b0 0x7f8e400cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.765+0000 7f8e46800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4006cea0 con 0x7f8e400cd240 2024-01-15T18:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.765+0000 7f8e46800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4006cbc0 con 0x7f8e4006c6b0 2024-01-15T18:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.765+0000 7f8e46800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4006f350 con 0x7f8e4009a170 2024-01-15T18:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.766+0000 7f8e3fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e4006c6b0 0x7f8e400cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.766+0000 7f8e3fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8e4006c6b0 0x7f8e400cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40898/0 (socket says 172.21.15.26:40898) 2024-01-15T18:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.766+0000 7f8e3fe00640 1 -- 172.21.15.26:0/3513137367 learned_addr learned my addr 172.21.15.26:0/3513137367 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:48.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.766+0000 7f8e3fe00640 1 -- 172.21.15.26:0/3513137367 >> v2:172.21.15.124:3300/0 conn(0x7f8e4009a170 msgr2=0x7f8e400b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:48.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.766+0000 7f8e3fe00640 1 --2- 172.21.15.26:0/3513137367 >> v2:172.21.15.124:3300/0 conn(0x7f8e4009a170 0x7f8e400b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.766+0000 7f8e3fe00640 1 -- 172.21.15.26:0/3513137367 >> v1:172.21.15.124:6789/0 conn(0x7f8e400cd240 legacy=0x7f8e40099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:48.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.766+0000 7f8e3fe00640 1 -- 172.21.15.26:0/3513137367 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4006f630 con 0x7f8e4006c6b0 2024-01-15T18:11:48.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.767+0000 7f8e3fe00640 1 --2- 172.21.15.26:0/3513137367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4006c6b0 0x7f8e400cf640 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f8e40071800 tx=0x7f8e3802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91a7f8b30f05df3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:48.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e3e000640 1 -- 172.21.15.26:0/3513137367 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3802ca80 con 0x7f8e4006c6b0 2024-01-15T18:11:48.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e3e000640 1 -- 172.21.15.26:0/3513137367 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3802cbe0 con 0x7f8e4006c6b0 2024-01-15T18:11:48.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e3e000640 1 -- 172.21.15.26:0/3513137367 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380343f0 con 0x7f8e4006c6b0 2024-01-15T18:11:48.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e46800640 1 -- 172.21.15.26:0/3513137367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4006c6b0 msgr2=0x7f8e400cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:48.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e46800640 1 --2- 172.21.15.26:0/3513137367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4006c6b0 0x7f8e400cf640 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f8e40071800 tx=0x7f8e3802bee0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e46800640 1 -- 172.21.15.26:0/3513137367 shutdown_connections 2024-01-15T18:11:48.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e46800640 1 --2- 172.21.15.26:0/3513137367 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4006c6b0 0x7f8e400cf640 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e46800640 1 --2- 172.21.15.26:0/3513137367 >> v2:172.21.15.124:3300/0 conn(0x7f8e4009a170 0x7f8e400b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e46800640 1 -- 172.21.15.26:0/3513137367 >> 172.21.15.26:0/3513137367 conn(0x7f8e400b2d00 msgr2=0x7f8e400b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:48.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.768+0000 7f8e46800640 1 -- 172.21.15.26:0/3513137367 shutdown_connections 2024-01-15T18:11:48.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.769+0000 7f8e46800640 1 -- 172.21.15.26:0/3513137367 wait complete. 2024-01-15T18:11:48.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.770+0000 7f8e46800640 1 Processor -- start 2024-01-15T18:11:48.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.770+0000 7f8e46800640 1 -- start start 2024-01-15T18:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.771+0000 7f8e46800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4006c6b0 0x7f8e40155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.771+0000 7f8e46800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4009a170 0x7f8e4014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.771+0000 7f8e46800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e400cd240 0x7f8e400b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.771+0000 7f8e46800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400cb100 con 0x7f8e4009a170 2024-01-15T18:11:48.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.771+0000 7f8e46800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40151bd0 con 0x7f8e400cd240 2024-01-15T18:11:48.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.771+0000 7f8e46800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401f7300 con 0x7f8e4006c6b0 2024-01-15T18:11:48.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.771+0000 7f8e3ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4009a170 0x7f8e4014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4009a170 0x7f8e4014efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40910/0 (socket says 172.21.15.26:40910) 2024-01-15T18:11:48.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3ea00640 1 -- 172.21.15.26:0/726664868 learned_addr learned my addr 172.21.15.26:0/726664868 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3fe00640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e400cd240 0x7f8e400b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3ea00640 1 -- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4006c6b0 msgr2=0x7f8e40155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3f400640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4006c6b0 0x7f8e40155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3ea00640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4006c6b0 0x7f8e40155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3ea00640 1 -- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e400cd240 msgr2=0x7f8e400b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3ea00640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e400cd240 0x7f8e400b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3ea00640 1 -- 172.21.15.26:0/726664868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e38009b30 con 0x7f8e4009a170 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3fe00640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e400cd240 0x7f8e400b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e3ea00640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4009a170 0x7f8e4014efd0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800b410 tx=0x7f8e2800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.772+0000 7f8e37400640 1 -- 172.21.15.26:0/726664868 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28011830 con 0x7f8e4009a170 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.773+0000 7f8e37400640 1 -- 172.21.15.26:0/726664868 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e280119b0 con 0x7f8e4009a170 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.773+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e400cab40 con 0x7f8e4009a170 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.773+0000 7f8e37400640 1 -- 172.21.15.26:0/726664868 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e28011d60 con 0x7f8e4009a170 2024-01-15T18:11:48.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.773+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e401557f0 con 0x7f8e4009a170 2024-01-15T18:11:48.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.774+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e400675d0 con 0x7f8e4009a170 2024-01-15T18:11:48.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.775+0000 7f8e37400640 1 -- 172.21.15.26:0/726664868 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7f8e2801a400 con 0x7f8e4009a170 2024-01-15T18:11:48.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.776+0000 7f8e37400640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e1806c3a0 0x7f8e1806e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:48.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.776+0000 7f8e37400640 1 -- 172.21.15.26:0/726664868 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8e280888f0 con 0x7f8e4009a170 2024-01-15T18:11:48.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.776+0000 7f8e3f400640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e1806c3a0 0x7f8e1806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:48.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.777+0000 7f8e3f400640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e1806c3a0 0x7f8e1806e860 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8e30003d90 tx=0x7f8e30009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:48.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.780+0000 7f8e37400640 1 -- 172.21.15.26:0/726664868 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e280551b0 con 0x7f8e4009a170 2024-01-15T18:11:49.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:48.997+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "force": true, "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e400ce7f0 con 0x7f8e1806c3a0 2024-01-15T18:11:49.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.061+0000 7f8e37400640 1 -- 172.21.15.26:0/726664868 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8e400ce7f0 con 0x7f8e1806c3a0 2024-01-15T18:11:49.065 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:11:49.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e1806c3a0 msgr2=0x7f8e1806e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e1806c3a0 0x7f8e1806e860 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8e30003d90 tx=0x7f8e30009040 comp rx=0 tx=0).stop 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4009a170 msgr2=0x7f8e4014efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4009a170 0x7f8e4014efd0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8e2800b410 tx=0x7f8e2800b8d0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 shutdown_connections 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8e1806c3a0 0x7f8e1806e860 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8e400cd240 0x7f8e400b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8e4009a170 0x7f8e4014efd0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 --2- 172.21.15.26:0/726664868 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8e4006c6b0 0x7f8e40155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 >> 172.21.15.26:0/726664868 conn(0x7f8e400b2d00 msgr2=0x7f8e400cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 shutdown_connections 2024-01-15T18:11:49.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.065+0000 7f8e46800640 1 -- 172.21.15.26:0/726664868 wait complete. 2024-01-15T18:11:49.090 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs sv1 2024-01-15T18:11:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.312+0000 7f3756c00640 1 Processor -- start 2024-01-15T18:11:49.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.312+0000 7f3756c00640 1 -- start start 2024-01-15T18:11:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.313+0000 7f3756c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f37500cd170 0x7f3750071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.314+0000 7f3756c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3750071690 con 0x7f37500cf9a0 2024-01-15T18:11:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.314+0000 7f3756c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3750071970 con 0x7f375014b2c0 2024-01-15T18:11:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.314+0000 7f3756c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375014b6a0 con 0x7f37500cd170 2024-01-15T18:11:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.314+0000 7f3747e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f37500cf9a0 0x7f3750070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47912/0 (socket says 172.21.15.26:47912) 2024-01-15T18:11:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.314+0000 7f3747e00640 1 -- 172.21.15.26:0/1761716642 learned_addr learned my addr 172.21.15.26:0/1761716642 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:49.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.314+0000 7f3754c00640 1 --2- 172.21.15.26:0/1761716642 >> v2:172.21.15.26:3300/0 conn(0x7f37500cd170 0x7f3750071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.314+0000 7f374f400640 1 -- 172.21.15.26:0/1761716642 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586512033 0 0) 0x7f3750071690 con 0x7f37500cf9a0 2024-01-15T18:11:49.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.314+0000 7f374f400640 1 -- 172.21.15.26:0/1761716642 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3730003520 con 0x7f37500cf9a0 2024-01-15T18:11:49.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.315+0000 7f3754c00640 1 -- 172.21.15.26:0/1761716642 >> v1:172.21.15.124:6789/0 conn(0x7f375014b2c0 legacy=0x7f37500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.315+0000 7f3754c00640 1 -- 172.21.15.26:0/1761716642 >> v1:172.21.15.26:6789/0 conn(0x7f37500cf9a0 legacy=0x7f3750070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.315+0000 7f3754c00640 1 -- 172.21.15.26:0/1761716642 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3750071c50 con 0x7f37500cd170 2024-01-15T18:11:49.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.316+0000 7f3754c00640 1 --2- 172.21.15.26:0/1761716642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 0x7f3750071070 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3748009d90 tx=0x7f374802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95a9422ee008d07 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:49.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.316+0000 7f374f400640 1 -- 172.21.15.26:0/1761716642 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374802cd00 con 0x7f37500cd170 2024-01-15T18:11:49.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.316+0000 7f374f400640 1 -- 172.21.15.26:0/1761716642 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374802ce60 con 0x7f37500cd170 2024-01-15T18:11:49.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.316+0000 7f374f400640 1 -- 172.21.15.26:0/1761716642 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748034660 con 0x7f37500cd170 2024-01-15T18:11:49.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.317+0000 7f3756c00640 1 -- 172.21.15.26:0/1761716642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 msgr2=0x7f3750071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.317+0000 7f3756c00640 1 --2- 172.21.15.26:0/1761716642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 0x7f3750071070 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f3748009d90 tx=0x7f374802bf50 comp rx=0 tx=0).stop 2024-01-15T18:11:49.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.317+0000 7f3756c00640 1 -- 172.21.15.26:0/1761716642 shutdown_connections 2024-01-15T18:11:49.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.317+0000 7f3756c00640 1 --2- 172.21.15.26:0/1761716642 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 0x7f3750071070 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.317+0000 7f3756c00640 1 -- 172.21.15.26:0/1761716642 >> 172.21.15.26:0/1761716642 conn(0x7f375006a800 msgr2=0x7f375006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:49.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.317+0000 7f3756c00640 1 -- 172.21.15.26:0/1761716642 shutdown_connections 2024-01-15T18:11:49.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.317+0000 7f3756c00640 1 -- 172.21.15.26:0/1761716642 wait complete. 2024-01-15T18:11:49.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.319+0000 7f3756c00640 1 Processor -- start 2024-01-15T18:11:49.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.319+0000 7f3756c00640 1 -- start start 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f3756c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 0x7f3750160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f3756c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37500cf9a0 0x7f375015e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f3756c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f375014b2c0 0x7f375015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f3756c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375015f9d0 con 0x7f37500cd170 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f3756c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375015fcb0 con 0x7f375014b2c0 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f3756c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375015ff90 con 0x7f37500cf9a0 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f374fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 0x7f3750160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f374fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 0x7f3750160a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40936/0 (socket says 172.21.15.26:40936) 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f374fe00640 1 -- 172.21.15.26:0/2826705875 learned_addr learned my addr 172.21.15.26:0/2826705875 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f3754c00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f375014b2c0 0x7f375015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f3747e00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37500cf9a0 0x7f375015e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f374fe00640 1 -- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37500cf9a0 msgr2=0x7f375015e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f374fe00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37500cf9a0 0x7f375015e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f374fe00640 1 -- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f375014b2c0 msgr2=0x7f375015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f374fe00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f375014b2c0 0x7f375015bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f374fe00640 1 -- 172.21.15.26:0/2826705875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3748009a70 con 0x7f37500cd170 2024-01-15T18:11:49.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.320+0000 7f3754c00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f375014b2c0 0x7f375015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:49.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.321+0000 7f374fe00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 0x7f3750160a70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f374000c400 tx=0x7f374000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.321+0000 7f374cc00640 1 -- 172.21.15.26:0/2826705875 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37400052d0 con 0x7f37500cd170 2024-01-15T18:11:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.321+0000 7f374cc00640 1 -- 172.21.15.26:0/2826705875 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3740005430 con 0x7f37500cd170 2024-01-15T18:11:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.321+0000 7f374cc00640 1 -- 172.21.15.26:0/2826705875 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3740003830 con 0x7f37500cd170 2024-01-15T18:11:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.321+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3750160270 con 0x7f37500cd170 2024-01-15T18:11:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.321+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375015ee30 con 0x7f37500cd170 2024-01-15T18:11:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.322+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37500675d0 con 0x7f37500cd170 2024-01-15T18:11:49.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.324+0000 7f374cc00640 1 -- 172.21.15.26:0/2826705875 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7f3740003990 con 0x7f37500cd170 2024-01-15T18:11:49.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.325+0000 7f374cc00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f372806c3d0 0x7f372806e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.325+0000 7f374cc00640 1 -- 172.21.15.26:0/2826705875 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3740087fd0 con 0x7f37500cd170 2024-01-15T18:11:49.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.326+0000 7f3747e00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f372806c3d0 0x7f372806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.327+0000 7f3747e00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f372806c3d0 0x7f372806e890 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f375015c640 tx=0x7f3738009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:49.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.330+0000 7f374cc00640 1 -- 172.21.15.26:0/2826705875 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3740054550 con 0x7f37500cd170 2024-01-15T18:11:49.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.551+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f375015fe20 con 0x7f372806c3d0 2024-01-15T18:11:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.594+0000 7f374cc00640 1 -- 172.21.15.26:0/2826705875 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f375015fe20 con 0x7f372806c3d0 2024-01-15T18:11:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.596+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f372806c3d0 msgr2=0x7f372806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.596+0000 7f3756c00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f372806c3d0 0x7f372806e890 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f375015c640 tx=0x7f3738009040 comp rx=0 tx=0).stop 2024-01-15T18:11:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.596+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 msgr2=0x7f3750160a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.596+0000 7f3756c00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 0x7f3750160a70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f374000c400 tx=0x7f374000c8c0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.597+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 shutdown_connections 2024-01-15T18:11:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.597+0000 7f3756c00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f37500cf9a0 0x7f375015e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.597+0000 7f3756c00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f372806c3d0 0x7f372806e890 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.597+0000 7f3756c00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f375014b2c0 0x7f375015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.597+0000 7f3756c00640 1 --2- 172.21.15.26:0/2826705875 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f37500cd170 0x7f3750160a70 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.597+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 >> 172.21.15.26:0/2826705875 conn(0x7f375006a800 msgr2=0x7f37500cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.597+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 shutdown_connections 2024-01-15T18:11:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.597+0000 7f3756c00640 1 -- 172.21.15.26:0/2826705875 wait complete. 2024-01-15T18:11:49.604 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.600+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.604 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.600+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.604 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.600+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.604 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.600+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.604 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.600+0000 7f8f56400640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.624 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs ss1clone1 2024-01-15T18:11:49.627 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.623+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.627 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.623+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.627 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.623+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.627 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.623+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.627 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:11:49.623+0000 7f8f57800640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:11:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.839+0000 7f86d5c00640 1 Processor -- start 2024-01-15T18:11:49.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.839+0000 7f86d5c00640 1 -- start start 2024-01-15T18:11:49.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.841+0000 7f86d5c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f86d00ac8d0 0x7f86d00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.841+0000 7f86d5c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d0071de0 con 0x7f86d006a3e0 2024-01-15T18:11:49.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.841+0000 7f86d5c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d0072030 con 0x7f86d00d0b60 2024-01-15T18:11:49.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.841+0000 7f86d5c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d00acd60 con 0x7f86d00ac8d0 2024-01-15T18:11:49.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.841+0000 7f86cea00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86d006a3e0 0x7f86d006a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:47920/0 (socket says 172.21.15.26:47920) 2024-01-15T18:11:49.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.841+0000 7f86cea00640 1 -- 172.21.15.26:0/2260193140 learned_addr learned my addr 172.21.15.26:0/2260193140 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:49.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.842+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508007670 0 0) 0x7f86d0071de0 con 0x7f86d006a3e0 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.842+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b0003520 con 0x7f86d006a3e0 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.842+0000 7f86cf400640 1 --2- 172.21.15.26:0/2260193140 >> v2:172.21.15.124:3300/0 conn(0x7f86d00ac8d0 0x7f86d00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.842+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440454824 0 0) 0x7f86b0003520 con 0x7f86d006a3e0 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.842+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 >> v2:172.21.15.124:3300/0 conn(0x7f86d00ac8d0 msgr2=0x7f86d00aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.842+0000 7f86ce000640 1 --2- 172.21.15.26:0/2260193140 >> v2:172.21.15.124:3300/0 conn(0x7f86d00ac8d0 0x7f86d00aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.842+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 >> v1:172.21.15.124:6789/0 conn(0x7f86d00d0b60 legacy=0x7f86d006f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.842+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d0072310 con 0x7f86d006a3e0 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.842+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f86c0001670 con 0x7f86d006a3e0 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 >> v1:172.21.15.26:6789/0 conn(0x7f86d006a3e0 legacy=0x7f86d006a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86ce000640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b0003970 0x7f86b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86ce000640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86b00047c0 0x7f86b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86ce000640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86b0005180 0x7f86b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d00acd60 con 0x7f86b0005180 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b000c0d0 con 0x7f86b00047c0 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b000c290 con 0x7f86b0003970 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86cf400640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86b0005180 0x7f86b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86c7e00640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86b00047c0 0x7f86b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86cea00640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b0003970 0x7f86b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86cf400640 1 -- 172.21.15.26:0/2260193140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b0003970 msgr2=0x7f86b0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.843+0000 7f86cf400640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b0003970 0x7f86b0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.844+0000 7f86cf400640 1 -- 172.21.15.26:0/2260193140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86b00047c0 msgr2=0x7f86b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.844+0000 7f86cf400640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86b00047c0 0x7f86b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.844+0000 7f86cf400640 1 -- 172.21.15.26:0/2260193140 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b0003520 con 0x7f86b0005180 2024-01-15T18:11:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.844+0000 7f86cf400640 1 -- 172.21.15.26:0/2260193140 reap_dead start 2024-01-15T18:11:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.845+0000 7f86cf400640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86b0005180 0x7f86b000b980 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f86c8007360 tx=0x7f86b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dec81848cb1a1d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.845+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c802ccf0 con 0x7f86b0005180 2024-01-15T18:11:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.845+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c802ce90 con 0x7f86b0005180 2024-01-15T18:11:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.845+0000 7f86ce000640 1 -- 172.21.15.26:0/2260193140 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c8003bd0 con 0x7f86b0005180 2024-01-15T18:11:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.846+0000 7f86d5c00640 1 -- 172.21.15.26:0/2260193140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86b0005180 msgr2=0x7f86b000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.846+0000 7f86d5c00640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86b0005180 0x7f86b000b980 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f86c8007360 tx=0x7f86b00057f0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.846+0000 7f86d5c00640 1 -- 172.21.15.26:0/2260193140 shutdown_connections 2024-01-15T18:11:49.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.846+0000 7f86d5c00640 1 --2- 172.21.15.26:0/2260193140 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86b0005180 0x7f86b000b980 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.846+0000 7f86d5c00640 1 -- 172.21.15.26:0/2260193140 >> 172.21.15.26:0/2260193140 conn(0x7f86d00cfef0 msgr2=0x7f86d00d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.846+0000 7f86d5c00640 1 -- 172.21.15.26:0/2260193140 shutdown_connections 2024-01-15T18:11:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.846+0000 7f86d5c00640 1 -- 172.21.15.26:0/2260193140 wait complete. 2024-01-15T18:11:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.848+0000 7f86d5c00640 1 Processor -- start 2024-01-15T18:11:49.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.848+0000 7f86d5c00640 1 -- start start 2024-01-15T18:11:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.849+0000 7f86d5c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b0005180 0x7f86d015fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.849+0000 7f86d5c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d00cd350 0x7f86d014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.849+0000 7f86d5c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86d00ce960 0x7f86d00acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.849+0000 7f86d5c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d015ff40 con 0x7f86d00cd350 2024-01-15T18:11:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.849+0000 7f86d5c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d014ade0 con 0x7f86d00ce960 2024-01-15T18:11:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.849+0000 7f86d5c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d00cd8c0 con 0x7f86b0005180 2024-01-15T18:11:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.849+0000 7f86c7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d00cd350 0x7f86d014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.849+0000 7f86c7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d00cd350 0x7f86d014a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40944/0 (socket says 172.21.15.26:40944) 2024-01-15T18:11:49.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.849+0000 7f86c7e00640 1 -- 172.21.15.26:0/1038236591 learned_addr learned my addr 172.21.15.26:0/1038236591 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86cf400640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86d00ce960 0x7f86d00acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86c7e00640 1 -- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b0005180 msgr2=0x7f86d015fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86cea00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b0005180 0x7f86d015fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86c7e00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b0005180 0x7f86d015fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86c7e00640 1 -- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86d00ce960 msgr2=0x7f86d00acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86c7e00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86d00ce960 0x7f86d00acea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86c7e00640 1 -- 172.21.15.26:0/1038236591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d00cdba0 con 0x7f86d00cd350 2024-01-15T18:11:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86cf400640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86d00ce960 0x7f86d00acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:49.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86c7e00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d00cd350 0x7f86d014a8a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f86b800e440 tx=0x7f86b800e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86c6a00640 1 -- 172.21.15.26:0/1038236591 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b80052e0 con 0x7f86d00cd350 2024-01-15T18:11:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86c6a00640 1 -- 172.21.15.26:0/1038236591 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b8005440 con 0x7f86d00cd350 2024-01-15T18:11:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86c6a00640 1 -- 172.21.15.26:0/1038236591 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b8003830 con 0x7f86d00cd350 2024-01-15T18:11:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.850+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d00cde80 con 0x7f86d00cd350 2024-01-15T18:11:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.851+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d00ce440 con 0x7f86d00cd350 2024-01-15T18:11:49.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.852+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d00ce600 con 0x7f86d00cd350 2024-01-15T18:11:49.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.854+0000 7f86c6a00640 1 -- 172.21.15.26:0/1038236591 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7f86b8003990 con 0x7f86d00cd350 2024-01-15T18:11:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.855+0000 7f86c6a00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f86a806c3a0 0x7f86a806e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.855+0000 7f86c6a00640 1 -- 172.21.15.26:0/1038236591 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f86b8088420 con 0x7f86d00cd350 2024-01-15T18:11:49.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.855+0000 7f86cea00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f86a806c3a0 0x7f86a806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.857+0000 7f86cea00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f86a806c3a0 0x7f86a806e860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f86c0000f60 tx=0x7f86c0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:49.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:49.859+0000 7f86c6a00640 1 -- 172.21.15.26:0/1038236591 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86b8054c90 con 0x7f86d00cd350 2024-01-15T18:11:50.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.091+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "ss1clone1", "target": ["mon-mgr", ""]}) v1 -- 0x7f86d006eb60 con 0x7f86a806c3a0 2024-01-15T18:11:50.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.144+0000 7f86c6a00640 1 -- 172.21.15.26:0/1038236591 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f86c000d040 con 0x7f86a806c3a0 2024-01-15T18:11:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f86a806c3a0 msgr2=0x7f86a806e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f86a806c3a0 0x7f86a806e860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f86c0000f60 tx=0x7f86c0009040 comp rx=0 tx=0).stop 2024-01-15T18:11:50.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d00cd350 msgr2=0x7f86d014a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d00cd350 0x7f86d014a8a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f86b800e440 tx=0x7f86b800e900 comp rx=0 tx=0).stop 2024-01-15T18:11:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 shutdown_connections 2024-01-15T18:11:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f86a806c3a0 0x7f86a806e860 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86d00ce960 0x7f86d00acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86d00cd350 0x7f86d014a8a0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 --2- 172.21.15.26:0/1038236591 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b0005180 0x7f86d015fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 >> 172.21.15.26:0/1038236591 conn(0x7f86d00cfef0 msgr2=0x7f86d00713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 shutdown_connections 2024-01-15T18:11:50.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.148+0000 7f86d5c00640 1 -- 172.21.15.26:0/1038236591 wait complete. 2024-01-15T18:11:50.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:50.169 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-01-15T18:11:50.222 INFO:teuthology.orchestra.run.smithi026.stdout:2 2024-01-15T18:11:50.223 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm cephfs --yes-i-really-mean-it 2024-01-15T18:11:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.422+0000 7fc566600640 1 Processor -- start 2024-01-15T18:11:50.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.423+0000 7fc566600640 1 -- start start 2024-01-15T18:11:50.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.425+0000 7fc566600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc56006c730 0x7fc5600b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.425+0000 7fc566600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc560070160 0x7fc560071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.425+0000 7fc566600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc56006cb10 0x7fc56006cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.425+0000 7fc566600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5600708d0 con 0x7fc56006cb10 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.425+0000 7fc566600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc560070bb0 con 0x7fc56006c730 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.425+0000 7fc566600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc560070e90 con 0x7fc560070160 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.425+0000 7fc565c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc56006cb10 0x7fc56006cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.426+0000 7fc565c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc56006cb10 0x7fc56006cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40948/0 (socket says 172.21.15.26:40948) 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.426+0000 7fc55fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc560070160 0x7fc560071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.426+0000 7fc565c00640 1 -- 172.21.15.26:0/2494629136 learned_addr learned my addr 172.21.15.26:0/2494629136 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:50.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.426+0000 7fc565200640 1 --2- 172.21.15.26:0/2494629136 >> v2:172.21.15.124:3300/0 conn(0x7fc56006c730 0x7fc5600b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:50.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.426+0000 7fc565c00640 1 -- 172.21.15.26:0/2494629136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc560070160 msgr2=0x7fc560071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:50.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.426+0000 7fc565c00640 1 --2- 172.21.15.26:0/2494629136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc560070160 0x7fc560071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.426+0000 7fc565c00640 1 -- 172.21.15.26:0/2494629136 >> v2:172.21.15.124:3300/0 conn(0x7fc56006c730 msgr2=0x7fc5600b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:50.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.426+0000 7fc565c00640 1 --2- 172.21.15.26:0/2494629136 >> v2:172.21.15.124:3300/0 conn(0x7fc56006c730 0x7fc5600b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.426+0000 7fc565c00640 1 -- 172.21.15.26:0/2494629136 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc560071170 con 0x7fc56006cb10 2024-01-15T18:11:50.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.427+0000 7fc565c00640 1 --2- 172.21.15.26:0/2494629136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56006cb10 0x7fc56006cfa0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fc5600727c0 tx=0x7fc55802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72a1251198c1b32 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:50.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.427+0000 7fc55f400640 1 -- 172.21.15.26:0/2494629136 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55802ca70 con 0x7fc56006cb10 2024-01-15T18:11:50.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.427+0000 7fc55f400640 1 -- 172.21.15.26:0/2494629136 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55802cbd0 con 0x7fc56006cb10 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.427+0000 7fc55f400640 1 -- 172.21.15.26:0/2494629136 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5580343f0 con 0x7fc56006cb10 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.428+0000 7fc566600640 1 -- 172.21.15.26:0/2494629136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56006cb10 msgr2=0x7fc56006cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.428+0000 7fc566600640 1 --2- 172.21.15.26:0/2494629136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56006cb10 0x7fc56006cfa0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fc5600727c0 tx=0x7fc55802bed0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.428+0000 7fc566600640 1 -- 172.21.15.26:0/2494629136 shutdown_connections 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.428+0000 7fc566600640 1 --2- 172.21.15.26:0/2494629136 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc56006cb10 0x7fc56006cfa0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.428+0000 7fc566600640 1 --2- 172.21.15.26:0/2494629136 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc560070160 0x7fc560071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.428+0000 7fc566600640 1 --2- 172.21.15.26:0/2494629136 >> v2:172.21.15.124:3300/0 conn(0x7fc56006c730 0x7fc5600b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.428+0000 7fc566600640 1 -- 172.21.15.26:0/2494629136 >> 172.21.15.26:0/2494629136 conn(0x7fc56006a7d0 msgr2=0x7fc56006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.428+0000 7fc566600640 1 -- 172.21.15.26:0/2494629136 shutdown_connections 2024-01-15T18:11:50.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.428+0000 7fc566600640 1 -- 172.21.15.26:0/2494629136 wait complete. 2024-01-15T18:11:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.430+0000 7fc566600640 1 Processor -- start 2024-01-15T18:11:50.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.430+0000 7fc566600640 1 -- start start 2024-01-15T18:11:50.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.431+0000 7fc566600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc56006c730 0x7fc5601fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.431+0000 7fc566600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc56006cb10 0x7fc560145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.431+0000 7fc566600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc560070160 0x7fc5600cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.431+0000 7fc566600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56015aca0 con 0x7fc560070160 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.431+0000 7fc566600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56015a120 con 0x7fc56006c730 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.431+0000 7fc566600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56015a400 con 0x7fc56006cb10 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc560070160 0x7fc5600cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc560070160 0x7fc5600cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:40952/0 (socket says 172.21.15.26:40952) 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565c00640 1 -- 172.21.15.26:0/3287293740 learned_addr learned my addr 172.21.15.26:0/3287293740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc55fe00640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc56006cb10 0x7fc560145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565200640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc56006c730 0x7fc5601fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565c00640 1 -- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc56006cb10 msgr2=0x7fc560145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565c00640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc56006cb10 0x7fc560145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565c00640 1 -- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc56006c730 msgr2=0x7fc5601fa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565c00640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc56006c730 0x7fc5601fa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565c00640 1 -- 172.21.15.26:0/3287293740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc558009b30 con 0x7fc560070160 2024-01-15T18:11:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.432+0000 7fc565200640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc56006c730 0x7fc5601fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.433+0000 7fc565c00640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc560070160 0x7fc5600cd350 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fc558036040 tx=0x7fc558004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.433+0000 7fc55cc00640 1 -- 172.21.15.26:0/3287293740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558009510 con 0x7fc560070160 2024-01-15T18:11:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.433+0000 7fc55cc00640 1 -- 172.21.15.26:0/3287293740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55803e450 con 0x7fc560070160 2024-01-15T18:11:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.433+0000 7fc55cc00640 1 -- 172.21.15.26:0/3287293740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55803e800 con 0x7fc560070160 2024-01-15T18:11:50.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.433+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56015a6e0 con 0x7fc560070160 2024-01-15T18:11:50.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.433+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc560139510 con 0x7fc560070160 2024-01-15T18:11:50.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.434+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc560146f90 con 0x7fc560070160 2024-01-15T18:11:50.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.435+0000 7fc55cc00640 1 -- 172.21.15.26:0/3287293740 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 36) v1 ==== 91069+0+0 (secure 0 0 0) 0x7fc55803e960 con 0x7fc560070160 2024-01-15T18:11:50.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.436+0000 7fc55cc00640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53806c3a0 0x7fc53806e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.437+0000 7fc55cc00640 1 -- 172.21.15.26:0/3287293740 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5821+0+0 (secure 0 0 0) 0x7fc5580ab3c0 con 0x7fc560070160 2024-01-15T18:11:50.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.437+0000 7fc565200640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53806c3a0 0x7fc53806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:50.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.438+0000 7fc565200640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53806c3a0 0x7fc53806e860 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc55000ba50 tx=0x7fc550008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:50.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.441+0000 7fc55cc00640 1 -- 172.21.15.26:0/3287293740 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc560146f90 con 0x7fc560070160 2024-01-15T18:11:50.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:50.668+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "cephfs", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) v1 -- 0x7fc56014a200 con 0x7fc53806c3a0 2024-01-15T18:11:51.157 INFO:tasks.ceph.mon.a.smithi026.stderr:2024-01-15T18:11:51.154+0000 7f9819c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-15T18:11:51.246 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T18:11:51.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:51.440+0000 7fc55cc00640 1 -- 172.21.15.26:0/3287293740 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 37) v1 ==== 91171+0+0 (secure 0 0 0) 0x7fc558078050 con 0x7fc560070160 2024-01-15T18:11:53.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:53.459+0000 7fc55cc00640 1 -- 172.21.15.26:0/3287293740 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fc558079760 con 0x7fc560070160 2024-01-15T18:11:54.198 INFO:teuthology.orchestra.run.smithi026.stdout:metadata pool: cephfs_metadata data pool: ['cephfs_data'] removed 2024-01-15T18:11:54.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.193+0000 7fc55cc00640 1 -- 172.21.15.26:0/3287293740 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fc56014a200 con 0x7fc53806c3a0 2024-01-15T18:11:54.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.198+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53806c3a0 msgr2=0x7fc53806e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.198+0000 7fc566600640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53806c3a0 0x7fc53806e860 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc55000ba50 tx=0x7fc550008040 comp rx=0 tx=0).stop 2024-01-15T18:11:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.198+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc560070160 msgr2=0x7fc5600cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:54.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.198+0000 7fc566600640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc560070160 0x7fc5600cd350 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fc558036040 tx=0x7fc558004a40 comp rx=0 tx=0).stop 2024-01-15T18:11:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.199+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 shutdown_connections 2024-01-15T18:11:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.199+0000 7fc566600640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc56006cb10 0x7fc560145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.199+0000 7fc566600640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc53806c3a0 0x7fc53806e860 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.199+0000 7fc566600640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc560070160 0x7fc5600cd350 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.199+0000 7fc566600640 1 --2- 172.21.15.26:0/3287293740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc56006c730 0x7fc5601fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.199+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 >> 172.21.15.26:0/3287293740 conn(0x7fc56006a7d0 msgr2=0x7fc5600b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.199+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 shutdown_connections 2024-01-15T18:11:54.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.199+0000 7fc566600640 1 -- 172.21.15.26:0/3287293740 wait complete. 2024-01-15T18:11:54.240 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:11:54.240 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:11:54.363 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:11:54.460 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:11:54.460 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:11:54.460 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:54.535 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:11:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.736+0000 7f56cbc00640 1 Processor -- start 2024-01-15T18:11:54.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.737+0000 7f56cbc00640 1 -- start start 2024-01-15T18:11:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.738+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40ae5a0 0x7f56c415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.738+0000 7f56cbc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c4070c40 con 0x7f56c40cf290 2024-01-15T18:11:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.738+0000 7f56cbc00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c406c470 con 0x7f56c40b3ae0 2024-01-15T18:11:54.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.738+0000 7f56cbc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4071450 con 0x7f56c40ae5a0 2024-01-15T18:11:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.738+0000 7f56c9c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f56c40cf290 0x7f56c40b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56602/0 (socket says 172.21.15.26:56602) 2024-01-15T18:11:54.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.738+0000 7f56c9c00640 1 -- 172.21.15.26:0/559761936 learned_addr learned my addr 172.21.15.26:0/559761936 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c9200640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40ae5a0 0x7f56c415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059040995 0 0) 0x7f56c4070c40 con 0x7f56c40cf290 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56a4003520 con 0x7f56c40cf290 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495904126 0 0) 0x7f56a4003520 con 0x7f56c40cf290 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40ae5a0 msgr2=0x7f56c415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c3e00640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40ae5a0 0x7f56c415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 >> v1:172.21.15.124:6789/0 conn(0x7f56c40b3ae0 legacy=0x7f56c406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c406c750 con 0x7f56c40cf290 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f56bc0019f0 con 0x7f56c40cf290 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.739+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 >> v1:172.21.15.26:6789/0 conn(0x7f56c40cf290 legacy=0x7f56c40b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:54.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56c3e00640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40ae5a0 0x7f56a40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56c3e00640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003f70 0x7f56a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56c3e00640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4004d00 0x7f56a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4071450 con 0x7f56a4003f70 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a400bb50 con 0x7f56a4004d00 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a400bd10 con 0x7f56c40ae5a0 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56bbc00640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003f70 0x7f56a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56c9200640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40ae5a0 0x7f56a40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56bbc00640 1 -- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40ae5a0 msgr2=0x7f56a40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56bbc00640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56c40ae5a0 0x7f56a40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56bbc00640 1 -- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4004d00 msgr2=0x7f56a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56bbc00640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56a4004d00 0x7f56a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.740+0000 7f56bbc00640 1 -- 172.21.15.26:0/559761936 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a4003520 con 0x7f56a4003f70 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.741+0000 7f56bbc00640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003f70 0x7f56a40047c0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f56ac001750 tx=0x7f56a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5591a72dc4a83bab server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0052d0 con 0x7f56a4003f70 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac02e050 con 0x7f56a4003f70 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56c3e00640 1 -- 172.21.15.26:0/559761936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac003830 con 0x7f56a4003f70 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56cbc00640 1 -- 172.21.15.26:0/559761936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003f70 msgr2=0x7f56a40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56cbc00640 1 --2- 172.21.15.26:0/559761936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56a4003f70 0x7f56a40047c0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f56ac001750 tx=0x7f56a40057f0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56c9c00640 1 -- 172.21.15.26:0/559761936 reap_dead start 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56cbc00640 1 -- 172.21.15.26:0/559761936 shutdown_connections 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56cbc00640 1 -- 172.21.15.26:0/559761936 >> 172.21.15.26:0/559761936 conn(0x7f56c406a800 msgr2=0x7f56c406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56cbc00640 1 -- 172.21.15.26:0/559761936 shutdown_connections 2024-01-15T18:11:54.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.742+0000 7f56cbc00640 1 -- 172.21.15.26:0/559761936 wait complete. 2024-01-15T18:11:54.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.744+0000 7f56cbc00640 1 Processor -- start 2024-01-15T18:11:54.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.744+0000 7f56cbc00640 1 -- start start 2024-01-15T18:11:54.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.745+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a4003f70 0x7f56c414abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:54.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.745+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c4147890 0x7f56c40cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:54.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.745+0000 7f56cbc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56c40cc1b0 0x7f56c4149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:54.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.745+0000 7f56cbc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4149180 con 0x7f56c4147890 2024-01-15T18:11:54.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.745+0000 7f56cbc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c414b100 con 0x7f56c40cc1b0 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.745+0000 7f56cbc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c40c4110 con 0x7f56a4003f70 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.745+0000 7f56bbc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c4147890 0x7f56c40cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56bbc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c4147890 0x7f56c40cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37304/0 (socket says 172.21.15.26:37304) 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56bbc00640 1 -- 172.21.15.26:0/2264744962 learned_addr learned my addr 172.21.15.26:0/2264744962 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56c9200640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a4003f70 0x7f56c414abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56c9c00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56c40cc1b0 0x7f56c4149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56bbc00640 1 -- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a4003f70 msgr2=0x7f56c414abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56bbc00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a4003f70 0x7f56c414abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56bbc00640 1 -- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56c40cc1b0 msgr2=0x7f56c4149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56bbc00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56c40cc1b0 0x7f56c4149610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56bbc00640 1 -- 172.21.15.26:0/2264744962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c4148570 con 0x7f56c4147890 2024-01-15T18:11:54.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.746+0000 7f56bbc00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c4147890 0x7f56c40cbc70 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005040 tx=0x7f56ac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:54.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.747+0000 7f56c1600640 1 -- 172.21.15.26:0/2264744962 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac007a30 con 0x7f56c4147890 2024-01-15T18:11:54.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.747+0000 7f56c1600640 1 -- 172.21.15.26:0/2264744962 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac0069e0 con 0x7f56c4147890 2024-01-15T18:11:54.751 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.747+0000 7f56c1600640 1 -- 172.21.15.26:0/2264744962 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac006d90 con 0x7f56c4147890 2024-01-15T18:11:54.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.747+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c4148850 con 0x7f56c4147890 2024-01-15T18:11:54.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.747+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c4148e10 con 0x7f56c4147890 2024-01-15T18:11:54.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.748+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c4148fa0 con 0x7f56c4147890 2024-01-15T18:11:54.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.749+0000 7f56c1600640 1 -- 172.21.15.26:0/2264744962 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f56ac0035f0 con 0x7f56c4147890 2024-01-15T18:11:54.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.750+0000 7f56c1600640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c460 0x7f569c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:54.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.751+0000 7f56c1600640 1 -- 172.21.15.26:0/2264744962 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f56ac0aa9a0 con 0x7f56c4147890 2024-01-15T18:11:54.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.751+0000 7f56c9200640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c460 0x7f569c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:54.755 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.752+0000 7f56c9200640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c460 0x7f569c06e920 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009a10 tx=0x7f56b4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:54.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.755+0000 7f56c1600640 1 -- 172.21.15.26:0/2264744962 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56ac076fa0 con 0x7f56c4147890 2024-01-15T18:11:54.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.950+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56c40675d0 con 0x7f56c4147890 2024-01-15T18:11:54.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.952+0000 7f56c1600640 1 -- 172.21.15.26:0/2264744962 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+13104 (secure 0 0 0) 0x7f56ac04b070 con 0x7f56c4147890 2024-01-15T18:11:54.956 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:11:54.956 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":106,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:11:54.175615+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/421656183":"2024-01-16T18:11:51.154279+0000","172.21.15.124:6836/421656183":"2024-01-16T18:11:51.154279+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c460 msgr2=0x7f569c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c460 0x7f569c06e920 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009a10 tx=0x7f56b4008040 comp rx=0 tx=0).stop 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c4147890 msgr2=0x7f56c40cbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c4147890 0x7f56c40cbc70 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005040 tx=0x7f56ac001120 comp rx=0 tx=0).stop 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 shutdown_connections 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f569c06c460 0x7f569c06e920 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f56c40cc1b0 0x7f56c4149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f56c4147890 0x7f56c40cbc70 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 --2- 172.21.15.26:0/2264744962 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f56a4003f70 0x7f56c414abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 >> 172.21.15.26:0/2264744962 conn(0x7f56c406a800 msgr2=0x7f56c415da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 shutdown_connections 2024-01-15T18:11:54.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:54.955+0000 7f56cbc00640 1 -- 172.21.15.26:0/2264744962 wait complete. 2024-01-15T18:11:54.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:11:54.977 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:11:55.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.194+0000 7fce23200640 1 Processor -- start 2024-01-15T18:11:55.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.195+0000 7fce23200640 1 -- start start 2024-01-15T18:11:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.196+0000 7fce23200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce240adf20 0x7fce24070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.196+0000 7fce23200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2415f4e0 con 0x7fce2415bf10 2024-01-15T18:11:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.196+0000 7fce23200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce24071130 con 0x7fce240ccb50 2024-01-15T18:11:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.196+0000 7fce23200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2415f970 con 0x7fce240adf20 2024-01-15T18:11:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.197+0000 7fce1be00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fce2415bf10 0x7fce240cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56604/0 (socket says 172.21.15.26:56604) 2024-01-15T18:11:55.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.197+0000 7fce1be00640 1 -- 172.21.15.26:0/3032183925 learned_addr learned my addr 172.21.15.26:0/3032183925 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:55.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.197+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268663952 0 0) 0x7fce2415f4e0 con 0x7fce2415bf10 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.197+0000 7fce21e00640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce240adf20 0x7fce24070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.197+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce04003520 con 0x7fce2415bf10 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.197+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 11890673 0 0) 0x7fce04003520 con 0x7fce2415bf10 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce240adf20 msgr2=0x7fce24070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce240adf20 0x7fce24070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 >> v1:172.21.15.124:6789/0 conn(0x7fce240ccb50 legacy=0x7fce24143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2415c390 con 0x7fce2415bf10 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fce0c001670 con 0x7fce2415bf10 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 >> v1:172.21.15.26:6789/0 conn(0x7fce2415bf10 legacy=0x7fce240cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce240adf20 0x7fce040039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce04004d00 0x7fce04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2415f970 con 0x7fce04003f70 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0400bb70 con 0x7fce04004d00 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0400bd30 con 0x7fce240adf20 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce1be00640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce21e00640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce240adf20 0x7fce040039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce1be00640 1 -- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce240adf20 msgr2=0x7fce040039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce1be00640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce240adf20 0x7fce040039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce1be00640 1 -- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce04004d00 msgr2=0x7fce04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.198+0000 7fce22800640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce04004d00 0x7fce04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.199+0000 7fce1be00640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce04004d00 0x7fce04005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.199+0000 7fce1be00640 1 -- 172.21.15.26:0/3032183925 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04003520 con 0x7fce04003f70 2024-01-15T18:11:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.199+0000 7fce22800640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce04004d00 0x7fce04005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:55.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.200+0000 7fce1be00640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce040047c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c0e0 tx=0x7fce040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a82bed93175eb39 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.200+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c005ad0 con 0x7fce04003f70 2024-01-15T18:11:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.200+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0c005c30 con 0x7fce04003f70 2024-01-15T18:11:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.200+0000 7fce21400640 1 -- 172.21.15.26:0/3032183925 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c02c7d0 con 0x7fce04003f70 2024-01-15T18:11:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.201+0000 7fce23200640 1 -- 172.21.15.26:0/3032183925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 msgr2=0x7fce040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.201+0000 7fce23200640 1 --2- 172.21.15.26:0/3032183925 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce040047c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00c0e0 tx=0x7fce040057f0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.201+0000 7fce22800640 1 -- 172.21.15.26:0/3032183925 reap_dead start 2024-01-15T18:11:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.201+0000 7fce23200640 1 -- 172.21.15.26:0/3032183925 shutdown_connections 2024-01-15T18:11:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.201+0000 7fce23200640 1 -- 172.21.15.26:0/3032183925 >> 172.21.15.26:0/3032183925 conn(0x7fce2406a7d0 msgr2=0x7fce2406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.201+0000 7fce23200640 1 -- 172.21.15.26:0/3032183925 shutdown_connections 2024-01-15T18:11:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.201+0000 7fce23200640 1 -- 172.21.15.26:0/3032183925 wait complete. 2024-01-15T18:11:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.203+0000 7fce23200640 1 Processor -- start 2024-01-15T18:11:55.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.203+0000 7fce23200640 1 -- start start 2024-01-15T18:11:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.204+0000 7fce23200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce240c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.204+0000 7fce23200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce241337e0 0x7fce240b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.204+0000 7fce23200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce2415a4d0 0x7fce2415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.204+0000 7fce23200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2415ae70 con 0x7fce04003f70 2024-01-15T18:11:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.204+0000 7fce23200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce240c43c0 con 0x7fce241337e0 2024-01-15T18:11:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.204+0000 7fce23200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce240c46a0 con 0x7fce2415a4d0 2024-01-15T18:11:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.204+0000 7fce21e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce240c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.204+0000 7fce21e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce240c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37330/0 (socket says 172.21.15.26:37330) 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.204+0000 7fce21e00640 1 -- 172.21.15.26:0/683903646 learned_addr learned my addr 172.21.15.26:0/683903646 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce1be00640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce241337e0 0x7fce240b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce22800640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce2415a4d0 0x7fce2415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce21e00640 1 -- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce2415a4d0 msgr2=0x7fce2415a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce21e00640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce2415a4d0 0x7fce2415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce21e00640 1 -- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce241337e0 msgr2=0x7fce240b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce21e00640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce241337e0 0x7fce240b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce21e00640 1 -- 172.21.15.26:0/683903646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce240c4980 con 0x7fce04003f70 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce1be00640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce241337e0 0x7fce240b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce21e00640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce240c3d00 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fce14011600 tx=0x7fce14011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce1a000640 1 -- 172.21.15.26:0/683903646 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1400f920 con 0x7fce04003f70 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce1a000640 1 -- 172.21.15.26:0/683903646 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1400fa80 con 0x7fce04003f70 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce1a000640 1 -- 172.21.15.26:0/683903646 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1400fdf0 con 0x7fce04003f70 2024-01-15T18:11:55.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.205+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce240c4c60 con 0x7fce04003f70 2024-01-15T18:11:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.206+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce240b12b0 con 0x7fce04003f70 2024-01-15T18:11:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.207+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce240c47e0 con 0x7fce04003f70 2024-01-15T18:11:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.207+0000 7fce1a000640 1 -- 172.21.15.26:0/683903646 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fce14015460 con 0x7fce04003f70 2024-01-15T18:11:55.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.208+0000 7fce1a000640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdfc06c430 0x7fcdfc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.209+0000 7fce1a000640 1 -- 172.21.15.26:0/683903646 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fce14089000 con 0x7fce04003f70 2024-01-15T18:11:55.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.209+0000 7fce1be00640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdfc06c430 0x7fcdfc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.210+0000 7fce1be00640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdfc06c430 0x7fcdfc06e8f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fce0c034000 tx=0x7fce0c037040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:55.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.213+0000 7fce1a000640 1 -- 172.21.15.26:0/683903646 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce240c47e0 con 0x7fce04003f70 2024-01-15T18:11:55.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.410+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce240c47e0 con 0x7fce04003f70 2024-01-15T18:11:55.414 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.411+0000 7fce1a000640 1 -- 172.21.15.26:0/683903646 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce240c47e0 con 0x7fce04003f70 2024-01-15T18:11:55.414 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:11:55.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdfc06c430 msgr2=0x7fcdfc06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdfc06c430 0x7fcdfc06e8f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fce0c034000 tx=0x7fce0c037040 comp rx=0 tx=0).stop 2024-01-15T18:11:55.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 msgr2=0x7fce240c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce240c3d00 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fce14011600 tx=0x7fce14011ac0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 shutdown_connections 2024-01-15T18:11:55.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fce2415a4d0 0x7fce2415a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcdfc06c430 0x7fcdfc06e8f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fce241337e0 0x7fce240b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 --2- 172.21.15.26:0/683903646 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fce04003f70 0x7fce240c3d00 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 >> 172.21.15.26:0/683903646 conn(0x7fce2406a7d0 msgr2=0x7fce2415b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:55.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 shutdown_connections 2024-01-15T18:11:55.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.414+0000 7fce23200640 1 -- 172.21.15.26:0/683903646 wait complete. 2024-01-15T18:11:55.435 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:11:55.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.645+0000 7f5000800640 1 Processor -- start 2024-01-15T18:11:55.648 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.645+0000 7f5000800640 1 -- start start 2024-01-15T18:11:55.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.646+0000 7f5000800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ff813e360 0x7f4ff80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.646+0000 7f5000800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4ff815f9c0 0x7f4ff80cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.646+0000 7f5000800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff815c5f0 con 0x7f4ff813df80 2024-01-15T18:11:55.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.646+0000 7f5000800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff806c470 con 0x7f4ff813e360 2024-01-15T18:11:55.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.646+0000 7f5000800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff806c750 con 0x7f4ff815f9c0 2024-01-15T18:11:55.650 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.647+0000 7f4ffd400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ff813e360 0x7f4ff80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.647+0000 7f4ffd400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4ff813e360 0x7f4ff80cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37332/0 (socket says 172.21.15.26:37332) 2024-01-15T18:11:55.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.647+0000 7f4ffd400640 1 -- 172.21.15.26:0/2540298541 learned_addr learned my addr 172.21.15.26:0/2540298541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:55.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.647+0000 7f4ffe800640 1 --2- 172.21.15.26:0/2540298541 >> v2:172.21.15.124:3300/0 conn(0x7f4ff815f9c0 0x7f4ff80cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.647+0000 7f4ffd400640 1 -- 172.21.15.26:0/2540298541 >> v2:172.21.15.124:3300/0 conn(0x7f4ff815f9c0 msgr2=0x7f4ff80cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:55.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.648+0000 7f4ffd400640 1 --2- 172.21.15.26:0/2540298541 >> v2:172.21.15.124:3300/0 conn(0x7f4ff815f9c0 0x7f4ff80cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.648+0000 7f4ffd400640 1 -- 172.21.15.26:0/2540298541 >> v1:172.21.15.124:6789/0 conn(0x7f4ff813df80 legacy=0x7f4ff815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:55.651 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.648+0000 7f4ffd400640 1 -- 172.21.15.26:0/2540298541 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff806ca30 con 0x7f4ff813e360 2024-01-15T18:11:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.649+0000 7f4ffd400640 1 --2- 172.21.15.26:0/2540298541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 0x7f4ff80cf290 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f4ff806ded0 tx=0x7f4fe002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88e2d01da247733 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.649+0000 7f4ffca00640 1 -- 172.21.15.26:0/2540298541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0031640 con 0x7f4ff813e360 2024-01-15T18:11:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.649+0000 7f4ffca00640 1 -- 172.21.15.26:0/2540298541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe00317a0 con 0x7f4ff813e360 2024-01-15T18:11:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.649+0000 7f4ffca00640 1 -- 172.21.15.26:0/2540298541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0031b30 con 0x7f4ff813e360 2024-01-15T18:11:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.650+0000 7f5000800640 1 -- 172.21.15.26:0/2540298541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 msgr2=0x7f4ff80cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.650+0000 7f5000800640 1 --2- 172.21.15.26:0/2540298541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 0x7f4ff80cf290 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f4ff806ded0 tx=0x7f4fe002b8f0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.650+0000 7f5000800640 1 -- 172.21.15.26:0/2540298541 shutdown_connections 2024-01-15T18:11:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.650+0000 7f5000800640 1 --2- 172.21.15.26:0/2540298541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 0x7f4ff80cf290 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.653 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.650+0000 7f5000800640 1 --2- 172.21.15.26:0/2540298541 >> v2:172.21.15.124:3300/0 conn(0x7f4ff815f9c0 0x7f4ff80cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.650+0000 7f5000800640 1 -- 172.21.15.26:0/2540298541 >> 172.21.15.26:0/2540298541 conn(0x7f4ff806a800 msgr2=0x7f4ff806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:55.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.650+0000 7f5000800640 1 -- 172.21.15.26:0/2540298541 shutdown_connections 2024-01-15T18:11:55.654 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.650+0000 7f5000800640 1 -- 172.21.15.26:0/2540298541 wait complete. 2024-01-15T18:11:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.652+0000 7f5000800640 1 Processor -- start 2024-01-15T18:11:55.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.652+0000 7f5000800640 1 -- start start 2024-01-15T18:11:55.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f5000800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff813df80 0x7f4ff81530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f5000800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 0x7f4ff813ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f5000800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff815f9c0 0x7f4ff80cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f5000800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8152c90 con 0x7f4ff813e360 2024-01-15T18:11:55.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f5000800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff81e5930 con 0x7f4ff815f9c0 2024-01-15T18:11:55.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f5000800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff81535f0 con 0x7f4ff813df80 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 0x7f4ff813ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffd400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 0x7f4ff813ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37340/0 (socket says 172.21.15.26:37340) 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffd400640 1 -- 172.21.15.26:0/65056903 learned_addr learned my addr 172.21.15.26:0/65056903 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffde00640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff813df80 0x7f4ff81530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffe800640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff815f9c0 0x7f4ff80cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffd400640 1 -- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff813df80 msgr2=0x7f4ff81530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffd400640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff813df80 0x7f4ff81530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffd400640 1 -- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff815f9c0 msgr2=0x7f4ff80cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffd400640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff815f9c0 0x7f4ff80cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffd400640 1 -- 172.21.15.26:0/65056903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe00094d0 con 0x7f4ff813e360 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffde00640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff813df80 0x7f4ff81530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.653+0000 7f4ffe800640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff815f9c0 0x7f4ff80cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.654+0000 7f4ffd400640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 0x7f4ff813ca50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0000c00 tx=0x7f4fe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:55.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.654+0000 7f4ff6000640 1 -- 172.21.15.26:0/65056903 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe003ac60 con 0x7f4ff813e360 2024-01-15T18:11:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.654+0000 7f4ff6000640 1 -- 172.21.15.26:0/65056903 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe003adc0 con 0x7f4ff813e360 2024-01-15T18:11:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.654+0000 7f4ff6000640 1 -- 172.21.15.26:0/65056903 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0044610 con 0x7f4ff813e360 2024-01-15T18:11:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.654+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff8152010 con 0x7f4ff813e360 2024-01-15T18:11:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.654+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff81525d0 con 0x7f4ff813e360 2024-01-15T18:11:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.656+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff8152dd0 con 0x7f4ff813e360 2024-01-15T18:11:55.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.656+0000 7f4ff6000640 1 -- 172.21.15.26:0/65056903 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f4fe002f050 con 0x7f4ff813e360 2024-01-15T18:11:55.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.657+0000 7f4ff6000640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fd006c430 0x7f4fd006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:55.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.657+0000 7f4ff6000640 1 -- 172.21.15.26:0/65056903 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f4fe00aa9b0 con 0x7f4ff813e360 2024-01-15T18:11:55.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.658+0000 7f4ffde00640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fd006c430 0x7f4fd006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:55.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.659+0000 7f4ffde00640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fd006c430 0x7f4fd006e8f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8072170 tx=0x7f4fe8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:55.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.661+0000 7f4ff6000640 1 -- 172.21.15.26:0/65056903 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ff8152dd0 con 0x7f4ff813e360 2024-01-15T18:11:55.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.871+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ff8152dd0 con 0x7f4ff813e360 2024-01-15T18:11:55.875 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.872+0000 7f4ff6000640 1 -- 172.21.15.26:0/65056903 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v106) v1 ==== 80+0+122 (secure 0 0 0) 0x7f4ff8152dd0 con 0x7f4ff813e360 2024-01-15T18:11:55.875 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6837/421656183 2024-01-16T18:11:51.154279+0000 2024-01-15T18:11:55.875 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6836/421656183 2024-01-16T18:11:51.154279+0000 2024-01-15T18:11:55.876 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:11:55.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fd006c430 msgr2=0x7f4fd006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fd006c430 0x7f4fd006e8f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8072170 tx=0x7f4fe8008040 comp rx=0 tx=0).stop 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 msgr2=0x7f4ff813ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 0x7f4ff813ca50 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0000c00 tx=0x7f4fe00046c0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 shutdown_connections 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fd006c430 0x7f4fd006e8f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff815f9c0 0x7f4ff80cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4ff813e360 0x7f4ff813ca50 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 --2- 172.21.15.26:0/65056903 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff813df80 0x7f4ff81530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 >> 172.21.15.26:0/65056903 conn(0x7f4ff806a800 msgr2=0x7f4ff815bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.875+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 shutdown_connections 2024-01-15T18:11:55.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:55.876+0000 7f5000800640 1 -- 172.21.15.26:0/65056903 wait complete. 2024-01-15T18:11:55.897 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-01-15T18:11:55.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:55.897 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:11:55.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[73102]: fuse finished with error 0 and tester_r 0 2024-01-15T18:11:55.996 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:11:55.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:55.996 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:11:56.250 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:11:56.250 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:56.308 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:11:56.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.531+0000 7fd4b3200640 1 Processor -- start 2024-01-15T18:11:56.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.531+0000 7fd4b3200640 1 -- start start 2024-01-15T18:11:56.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.533+0000 7fd4b3200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd4ac13cfa0 0x7fd4ac15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:56.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.533+0000 7fd4b3200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac0cdec0 con 0x7fd4ac0cd530 2024-01-15T18:11:56.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.533+0000 7fd4b3200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac0ce1a0 con 0x7fd4ac13fcb0 2024-01-15T18:11:56.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.533+0000 7fd4b3200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac0ce480 con 0x7fd4ac13cfa0 2024-01-15T18:11:56.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.533+0000 7fd4b1400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd4ac0cd530 0x7fd4ac13f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56608/0 (socket says 172.21.15.26:56608) 2024-01-15T18:11:56.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.533+0000 7fd4b1400640 1 -- 172.21.15.26:0/3698782780 learned_addr learned my addr 172.21.15.26:0/3698782780 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239222023 0 0) 0x7fd4ac0cdec0 con 0x7fd4ac0cd530 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd48c003520 con 0x7fd4ac0cd530 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b1e00640 1 --2- 172.21.15.26:0/3698782780 >> v2:172.21.15.124:3300/0 conn(0x7fd4ac13cfa0 0x7fd4ac15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187470426 0 0) 0x7fd48c003520 con 0x7fd4ac0cd530 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 >> v2:172.21.15.124:3300/0 conn(0x7fd4ac13cfa0 msgr2=0x7fd4ac15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 --2- 172.21.15.26:0/3698782780 >> v2:172.21.15.124:3300/0 conn(0x7fd4ac13cfa0 0x7fd4ac15afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 >> v1:172.21.15.124:6789/0 conn(0x7fd4ac13fcb0 legacy=0x7fd4ac0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac0ce760 con 0x7fd4ac0cd530 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd494001670 con 0x7fd4ac0cd530 2024-01-15T18:11:56.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 >> v1:172.21.15.26:6789/0 conn(0x7fd4ac0cd530 legacy=0x7fd4ac13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd48c0039c0 0x7fd48c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0047c0 0x7fd48c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd48c005250 0x7fd48c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0ce480 con 0x7fd48c0039c0 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c00c1a0 con 0x7fd48c0047c0 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c00c360 con 0x7fd48c005250 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.534+0000 7fd4b1e00640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd48c0039c0 0x7fd48c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.535+0000 7fd4b1e00640 1 -- 172.21.15.26:0/3698782780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd48c005250 msgr2=0x7fd48c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.535+0000 7fd4b1400640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0047c0 0x7fd48c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.535+0000 7fd4b1e00640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd48c005250 0x7fd48c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.535+0000 7fd4b1e00640 1 -- 172.21.15.26:0/3698782780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0047c0 msgr2=0x7fd48c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.535+0000 7fd4b1e00640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0047c0 0x7fd48c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.535+0000 7fd4b1e00640 1 -- 172.21.15.26:0/3698782780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c003520 con 0x7fd48c0039c0 2024-01-15T18:11:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.535+0000 7fd4b2800640 1 -- 172.21.15.26:0/3698782780 reap_dead start 2024-01-15T18:11:56.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.535+0000 7fd4b1400640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0047c0 0x7fd48c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.536+0000 7fd4b1e00640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd48c0039c0 0x7fd48c003dc0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00ba30 tx=0x7fd48c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a04b984f04e1261 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.536+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c032660 con 0x7fd48c0039c0 2024-01-15T18:11:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.537+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c0327c0 con 0x7fd48c0039c0 2024-01-15T18:11:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.537+0000 7fd4b0a00640 1 -- 172.21.15.26:0/3698782780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c032b50 con 0x7fd48c0039c0 2024-01-15T18:11:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.537+0000 7fd4b3200640 1 -- 172.21.15.26:0/3698782780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd48c0039c0 msgr2=0x7fd48c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:56.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.537+0000 7fd4b3200640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd48c0039c0 0x7fd48c003dc0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00ba30 tx=0x7fd48c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.537+0000 7fd4b3200640 1 -- 172.21.15.26:0/3698782780 shutdown_connections 2024-01-15T18:11:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.537+0000 7fd4b3200640 1 --2- 172.21.15.26:0/3698782780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd48c0039c0 0x7fd48c003dc0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.537+0000 7fd4b3200640 1 -- 172.21.15.26:0/3698782780 >> 172.21.15.26:0/3698782780 conn(0x7fd4ac06a800 msgr2=0x7fd4ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.537+0000 7fd4b3200640 1 -- 172.21.15.26:0/3698782780 shutdown_connections 2024-01-15T18:11:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.537+0000 7fd4b3200640 1 -- 172.21.15.26:0/3698782780 wait complete. 2024-01-15T18:11:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.539+0000 7fd4b3200640 1 Processor -- start 2024-01-15T18:11:56.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.539+0000 7fd4b3200640 1 -- start start 2024-01-15T18:11:56.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.540+0000 7fd4b3200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0039c0 0x7fd4ac0ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:56.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.540+0000 7fd4b3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4ac157490 0x7fd4ac1e4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:56.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.540+0000 7fd4b3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac13c1e0 0x7fd4ac13c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:56.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.540+0000 7fd4b3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac13cba0 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.540+0000 7fd4b3200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0ad960 con 0x7fd48c0039c0 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.540+0000 7fd4b3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0adc40 con 0x7fd4ac157490 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac13c1e0 0x7fd4ac13c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac13c1e0 0x7fd4ac13c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37360/0 (socket says 172.21.15.26:37360) 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b2800640 1 -- 172.21.15.26:0/2553185008 learned_addr learned my addr 172.21.15.26:0/2553185008 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b1e00640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0039c0 0x7fd4ac0ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b1400640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4ac157490 0x7fd4ac1e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b2800640 1 -- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4ac157490 msgr2=0x7fd4ac1e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b2800640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4ac157490 0x7fd4ac1e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b2800640 1 -- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0039c0 msgr2=0x7fd4ac0ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b2800640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0039c0 0x7fd4ac0ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b2800640 1 -- 172.21.15.26:0/2553185008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac0adf20 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.541+0000 7fd4b1e00640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0039c0 0x7fd4ac0ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.542+0000 7fd4b2800640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac13c1e0 0x7fd4ac13c660 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac06ce70 tx=0x7fd4a400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.542+0000 7fd4aa000640 1 -- 172.21.15.26:0/2553185008 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4007070 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.542+0000 7fd4aa000640 1 -- 172.21.15.26:0/2553185008 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a40071f0 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.542+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac0ae200 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.542+0000 7fd4aa000640 1 -- 172.21.15.26:0/2553185008 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a40075a0 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.542+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac157910 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.543+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac13d650 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.544+0000 7fd4aa000640 1 -- 172.21.15.26:0/2553185008 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fd4a4007700 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.545+0000 7fd4aa000640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd48406c460 0x7fd48406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:56.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.545+0000 7fd4aa000640 1 -- 172.21.15.26:0/2553185008 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fd4a4087f30 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.546+0000 7fd4b1e00640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd48406c460 0x7fd48406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:56.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.547+0000 7fd4b1e00640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd48406c460 0x7fd48406e920 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00ba00 tx=0x7fd49c03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:56.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.550+0000 7fd4aa000640 1 -- 172.21.15.26:0/2553185008 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4ac13d650 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.750+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4ac13d650 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.751+0000 7fd4aa000640 1 -- 172.21.15.26:0/2553185008 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+13104 (secure 0 0 0) 0x7fd4ac13d650 con 0x7fd4ac13c1e0 2024-01-15T18:11:56.755 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:11:56.755 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":106,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:11:54.175615+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/421656183":"2024-01-16T18:11:51.154279+0000","172.21.15.124:6836/421656183":"2024-01-16T18:11:51.154279+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:11:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd48406c460 msgr2=0x7fd48406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd48406c460 0x7fd48406e920 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00ba00 tx=0x7fd49c03d040 comp rx=0 tx=0).stop 2024-01-15T18:11:56.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac13c1e0 msgr2=0x7fd4ac13c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac13c1e0 0x7fd4ac13c660 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fd4ac06ce70 tx=0x7fd4a400c980 comp rx=0 tx=0).stop 2024-01-15T18:11:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 shutdown_connections 2024-01-15T18:11:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd4ac157490 0x7fd4ac1e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd48406c460 0x7fd48406e920 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4ac13c1e0 0x7fd4ac13c660 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 --2- 172.21.15.26:0/2553185008 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd48c0039c0 0x7fd4ac0ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 >> 172.21.15.26:0/2553185008 conn(0x7fd4ac06a800 msgr2=0x7fd4ac0afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 shutdown_connections 2024-01-15T18:11:56.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.754+0000 7fd4b3200640 1 -- 172.21.15.26:0/2553185008 wait complete. 2024-01-15T18:11:56.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:11:56.778 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:11:56.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.994+0000 7f0390e00640 1 Processor -- start 2024-01-15T18:11:56.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.995+0000 7f0390e00640 1 -- start start 2024-01-15T18:11:57.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.996+0000 7f0390e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f038c13df80 0x7f038c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.996+0000 7f0390e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f038c13e360 0x7f038c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.996+0000 7f0390e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f038c15f9c0 0x7f038c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.996+0000 7f0390e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038c06c470 con 0x7f038c13df80 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.996+0000 7f0390e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f038c06c750 con 0x7f038c13e360 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.996+0000 7f0390e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c06ca30 con 0x7f038c15f9c0 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.997+0000 7f0389e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f038c13df80 0x7f038c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.997+0000 7f0389e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f038c13df80 0x7f038c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37376/0 (socket says 172.21.15.26:37376) 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.997+0000 7f0389e00640 1 -- 172.21.15.26:0/2624005941 learned_addr learned my addr 172.21.15.26:0/2624005941 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.997+0000 7f0389e00640 1 -- 172.21.15.26:0/2624005941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f038c15f9c0 msgr2=0x7f038c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.997+0000 7f0389e00640 1 --2- 172.21.15.26:0/2624005941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f038c15f9c0 0x7f038c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.997+0000 7f0389e00640 1 -- 172.21.15.26:0/2624005941 >> v2:172.21.15.124:3300/0 conn(0x7f038c13e360 msgr2=0x7f038c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.997+0000 7f0389e00640 1 --2- 172.21.15.26:0/2624005941 >> v2:172.21.15.124:3300/0 conn(0x7f038c13e360 0x7f038c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.997+0000 7f0389e00640 1 -- 172.21.15.26:0/2624005941 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c06cd10 con 0x7f038c13df80 2024-01-15T18:11:57.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.998+0000 7f0389e00640 1 --2- 172.21.15.26:0/2624005941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c13df80 0x7f038c15bee0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f038c06ded0 tx=0x7f037c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1371f0d4ed2cf22b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:57.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.999+0000 7f0388a00640 1 -- 172.21.15.26:0/2624005941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c031870 con 0x7f038c13df80 2024-01-15T18:11:57.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.999+0000 7f0388a00640 1 -- 172.21.15.26:0/2624005941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c0319d0 con 0x7f038c13df80 2024-01-15T18:11:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.999+0000 7f0388a00640 1 -- 172.21.15.26:0/2624005941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c031d60 con 0x7f038c13df80 2024-01-15T18:11:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.999+0000 7f0390e00640 1 -- 172.21.15.26:0/2624005941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c13df80 msgr2=0x7f038c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:56.999+0000 7f0390e00640 1 --2- 172.21.15.26:0/2624005941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c13df80 0x7f038c15bee0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f038c06ded0 tx=0x7f037c02b910 comp rx=0 tx=0).stop 2024-01-15T18:11:57.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.000+0000 7f0390e00640 1 -- 172.21.15.26:0/2624005941 shutdown_connections 2024-01-15T18:11:57.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.000+0000 7f0390e00640 1 --2- 172.21.15.26:0/2624005941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f038c15f9c0 0x7f038c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.000+0000 7f0390e00640 1 --2- 172.21.15.26:0/2624005941 >> v2:172.21.15.124:3300/0 conn(0x7f038c13e360 0x7f038c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.000+0000 7f0390e00640 1 --2- 172.21.15.26:0/2624005941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c13df80 0x7f038c15bee0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.000+0000 7f0390e00640 1 -- 172.21.15.26:0/2624005941 >> 172.21.15.26:0/2624005941 conn(0x7f038c06a800 msgr2=0x7f038c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:57.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.000+0000 7f0390e00640 1 -- 172.21.15.26:0/2624005941 shutdown_connections 2024-01-15T18:11:57.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.000+0000 7f0390e00640 1 -- 172.21.15.26:0/2624005941 wait complete. 2024-01-15T18:11:57.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.002+0000 7f0390e00640 1 Processor -- start 2024-01-15T18:11:57.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.002+0000 7f0390e00640 1 -- start start 2024-01-15T18:11:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f0390e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f038c13df80 0x7f038c14ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f0390e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f038c13e360 0x7f038c070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f0390e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c15f9c0 0x7f038c071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f0390e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c153d70 con 0x7f038c15f9c0 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f0390e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038c154050 con 0x7f038c13df80 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f0390e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038c154330 con 0x7f038c13e360 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f038a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c15f9c0 0x7f038c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f038a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c15f9c0 0x7f038c071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37384/0 (socket says 172.21.15.26:37384) 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f038a800640 1 -- 172.21.15.26:0/2580736537 learned_addr learned my addr 172.21.15.26:0/2580736537 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f0389e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f038c13df80 0x7f038c14ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.003+0000 7f0389400640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f038c13e360 0x7f038c070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f038a800640 1 -- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f038c13e360 msgr2=0x7f038c070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f038a800640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f038c13e360 0x7f038c070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f038a800640 1 -- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f038c13df80 msgr2=0x7f038c14ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f038a800640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f038c13df80 0x7f038c14ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f038a800640 1 -- 172.21.15.26:0/2580736537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c0094d0 con 0x7f038c15f9c0 2024-01-15T18:11:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f0389e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f038c13df80 0x7f038c14ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f038a800640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c15f9c0 0x7f038c071530 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f038400b9e0 tx=0x7f038400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f0382000640 1 -- 172.21.15.26:0/2580736537 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038400cb20 con 0x7f038c15f9c0 2024-01-15T18:11:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f0382000640 1 -- 172.21.15.26:0/2580736537 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038400cc80 con 0x7f038c15f9c0 2024-01-15T18:11:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f0382000640 1 -- 172.21.15.26:0/2580736537 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384014680 con 0x7f038c15f9c0 2024-01-15T18:11:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038c154610 con 0x7f038c15f9c0 2024-01-15T18:11:57.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.004+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038c1504d0 con 0x7f038c15f9c0 2024-01-15T18:11:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.006+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c154b30 con 0x7f038c15f9c0 2024-01-15T18:11:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.007+0000 7f0382000640 1 -- 172.21.15.26:0/2580736537 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f038401d3b0 con 0x7f038c15f9c0 2024-01-15T18:11:57.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.007+0000 7f0382000640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f036406c430 0x7f036406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.008+0000 7f0382000640 1 -- 172.21.15.26:0/2580736537 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f0384088a70 con 0x7f038c15f9c0 2024-01-15T18:11:57.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.008+0000 7f0389e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f036406c430 0x7f036406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.009+0000 7f0389e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f036406c430 0x7f036406e8f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f037c036000 tx=0x7f037c03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:57.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.012+0000 7f0382000640 1 -- 172.21.15.26:0/2580736537 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f038c154b30 con 0x7f038c15f9c0 2024-01-15T18:11:57.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.209+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f038c154b30 con 0x7f038c15f9c0 2024-01-15T18:11:57.213 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.210+0000 7f0382000640 1 -- 172.21.15.26:0/2580736537 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f038c154b30 con 0x7f038c15f9c0 2024-01-15T18:11:57.214 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:11:57.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f036406c430 msgr2=0x7f036406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f036406c430 0x7f036406e8f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f037c036000 tx=0x7f037c03c040 comp rx=0 tx=0).stop 2024-01-15T18:11:57.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c15f9c0 msgr2=0x7f038c071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c15f9c0 0x7f038c071530 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f038400b9e0 tx=0x7f038400bea0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 shutdown_connections 2024-01-15T18:11:57.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f038c13e360 0x7f038c070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f036406c430 0x7f036406e8f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f038c15f9c0 0x7f038c071530 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 --2- 172.21.15.26:0/2580736537 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f038c13df80 0x7f038c14ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.213+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 >> 172.21.15.26:0/2580736537 conn(0x7f038c06a800 msgr2=0x7f038c13f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:57.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.214+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 shutdown_connections 2024-01-15T18:11:57.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.214+0000 7f0390e00640 1 -- 172.21.15.26:0/2580736537 wait complete. 2024-01-15T18:11:57.236 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:11:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.462+0000 7f2773200640 1 Processor -- start 2024-01-15T18:11:57.465 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.462+0000 7f2773200640 1 -- start start 2024-01-15T18:11:57.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.464+0000 7f2773200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2774160920 0x7f27740b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.467 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.464+0000 7f2773200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27740d0a10 con 0x7f27740cfe70 2024-01-15T18:11:57.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.464+0000 7f2773200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2774161e30 con 0x7f27740cf9f0 2024-01-15T18:11:57.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.464+0000 7f2773200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27740d1f50 con 0x7f2774160920 2024-01-15T18:11:57.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.464+0000 7f2772800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f27740cfe70 0x7f2774161720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56618/0 (socket says 172.21.15.26:56618) 2024-01-15T18:11:57.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.464+0000 7f2772800640 1 -- 172.21.15.26:0/3711815478 learned_addr learned my addr 172.21.15.26:0/3711815478 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:57.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.464+0000 7f2771e00640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2774160920 0x7f27740b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.465+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483498892 0 0) 0x7f27740d0a10 con 0x7f27740cfe70 2024-01-15T18:11:57.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.465+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2754003520 con 0x7f27740cfe70 2024-01-15T18:11:57.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.465+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958036770 0 0) 0x7f2774161e30 con 0x7f27740cf9f0 2024-01-15T18:11:57.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.465+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27740d0a10 con 0x7f27740cf9f0 2024-01-15T18:11:57.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3669875290 0 0) 0x7f2754003520 con 0x7f27740cfe70 2024-01-15T18:11:57.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2774160920 msgr2=0x7f27740b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:57.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2774160920 0x7f27740b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 >> v1:172.21.15.124:6789/0 conn(0x7f27740cf9f0 legacy=0x7f2774161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27740cf080 con 0x7f27740cfe70 2024-01-15T18:11:57.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f276c001670 con 0x7f27740cfe70 2024-01-15T18:11:57.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 >> v1:172.21.15.26:6789/0 conn(0x7f27740cfe70 legacy=0x7f2774161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2754004b90 0x7f2754004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754003760 0x7f2754003be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2754004120 0x7f275400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27740d1f50 con 0x7f2754003760 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275400bfa0 con 0x7f2754004120 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275400c160 con 0x7f2754004b90 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2771e00640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2774160920 0x7f27740b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.466+0000 7f2771400640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754003760 0x7f2754003be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.467+0000 7f2772800640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2754004120 0x7f275400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.467+0000 7f2771400640 1 -- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2754004b90 msgr2=0x7f2754004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:57.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.467+0000 7f2771400640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2754004b90 0x7f2754004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.467+0000 7f2771400640 1 -- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2754004120 msgr2=0x7f275400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:57.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.467+0000 7f2771400640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2754004120 0x7f275400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.467+0000 7f2771400640 1 -- 172.21.15.26:0/3711815478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2754003520 con 0x7f2754003760 2024-01-15T18:11:57.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.468+0000 7f2771400640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754003760 0x7f2754003be0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f275c001f40 tx=0x7f27540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492c39644828148f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:57.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.470+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c003b70 con 0x7f2754003760 2024-01-15T18:11:57.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.470+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c003cf0 con 0x7f2754003760 2024-01-15T18:11:57.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.470+0000 7f2770a00640 1 -- 172.21.15.26:0/3711815478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c005310 con 0x7f2754003760 2024-01-15T18:11:57.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.470+0000 7f2773200640 1 -- 172.21.15.26:0/3711815478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754003760 msgr2=0x7f2754003be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.470+0000 7f2773200640 1 --2- 172.21.15.26:0/3711815478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2754003760 0x7f2754003be0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f275c001f40 tx=0x7f27540057f0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.470+0000 7f2772800640 1 -- 172.21.15.26:0/3711815478 reap_dead start 2024-01-15T18:11:57.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.470+0000 7f2773200640 1 -- 172.21.15.26:0/3711815478 shutdown_connections 2024-01-15T18:11:57.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.470+0000 7f2773200640 1 -- 172.21.15.26:0/3711815478 >> 172.21.15.26:0/3711815478 conn(0x7f27740d3090 msgr2=0x7f27740d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:57.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.470+0000 7f2773200640 1 -- 172.21.15.26:0/3711815478 shutdown_connections 2024-01-15T18:11:57.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.471+0000 7f2773200640 1 -- 172.21.15.26:0/3711815478 wait complete. 2024-01-15T18:11:57.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.472+0000 7f2773200640 1 Processor -- start 2024-01-15T18:11:57.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.473+0000 7f2773200640 1 -- start start 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.473+0000 7f2773200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2754003760 0x7f27741515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2773200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27740c94a0 0x7f277415c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2773200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27740cc790 0x7f2774152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2773200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2774151ae0 con 0x7f27740c94a0 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2773200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27740c99a0 con 0x7f27740cc790 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2773200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27740cac10 con 0x7f2754003760 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2771400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27740c94a0 0x7f277415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2771400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27740c94a0 0x7f277415c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37410/0 (socket says 172.21.15.26:37410) 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2771400640 1 -- 172.21.15.26:0/4066183010 learned_addr learned my addr 172.21.15.26:0/4066183010 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:57.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2771e00640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2754003760 0x7f27741515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2772800640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27740cc790 0x7f2774152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2771400640 1 -- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2754003760 msgr2=0x7f27741515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2771400640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2754003760 0x7f27741515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2771400640 1 -- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27740cc790 msgr2=0x7f2774152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2771400640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27740cc790 0x7f2774152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.474+0000 7f2771400640 1 -- 172.21.15.26:0/4066183010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27740caef0 con 0x7f27740c94a0 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.475+0000 7f2771400640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27740c94a0 0x7f277415c650 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f275c02c080 tx=0x7f275c0015d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.475+0000 7f276a000640 1 -- 172.21.15.26:0/4066183010 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c03c070 con 0x7f27740c94a0 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.475+0000 7f276a000640 1 -- 172.21.15.26:0/4066183010 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275c0052e0 con 0x7f27740c94a0 2024-01-15T18:11:57.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.475+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27740cb1d0 con 0x7f27740c94a0 2024-01-15T18:11:57.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.475+0000 7f276a000640 1 -- 172.21.15.26:0/4066183010 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c0056b0 con 0x7f27740c94a0 2024-01-15T18:11:57.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.475+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277415cb90 con 0x7f27740c94a0 2024-01-15T18:11:57.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.477+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2774067450 con 0x7f27740c94a0 2024-01-15T18:11:57.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.478+0000 7f276a000640 1 -- 172.21.15.26:0/4066183010 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f275c02d4f0 con 0x7f27740c94a0 2024-01-15T18:11:57.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.478+0000 7f276a000640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f274c06c460 0x7f274c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:57.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.479+0000 7f276a000640 1 -- 172.21.15.26:0/4066183010 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f275c031070 con 0x7f27740c94a0 2024-01-15T18:11:57.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.479+0000 7f2771e00640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f274c06c460 0x7f274c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:57.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.480+0000 7f2771e00640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f274c06c460 0x7f274c06e920 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f277408c060 tx=0x7f2764008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:57.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.483+0000 7f276a000640 1 -- 172.21.15.26:0/4066183010 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f275c075eb0 con 0x7f27740c94a0 2024-01-15T18:11:57.696 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.693+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f277408c7e0 con 0x7f27740c94a0 2024-01-15T18:11:57.697 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.693+0000 7f276a000640 1 -- 172.21.15.26:0/4066183010 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v106) v1 ==== 80+0+122 (secure 0 0 0) 0x7f275c0794d0 con 0x7f27740c94a0 2024-01-15T18:11:57.697 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6837/421656183 2024-01-16T18:11:51.154279+0000 2024-01-15T18:11:57.697 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.15.124:6836/421656183 2024-01-16T18:11:51.154279+0000 2024-01-15T18:11:57.698 INFO:teuthology.orchestra.run.smithi026.stderr:listed 2 entries 2024-01-15T18:11:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f274c06c460 msgr2=0x7f274c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f274c06c460 0x7f274c06e920 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f277408c060 tx=0x7f2764008040 comp rx=0 tx=0).stop 2024-01-15T18:11:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27740c94a0 msgr2=0x7f277415c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27740c94a0 0x7f277415c650 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f275c02c080 tx=0x7f275c0015d0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.700 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 shutdown_connections 2024-01-15T18:11:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f274c06c460 0x7f274c06e920 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f27740cc790 0x7f2774152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f27740c94a0 0x7f277415c650 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 --2- 172.21.15.26:0/4066183010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2754003760 0x7f27741515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 >> 172.21.15.26:0/4066183010 conn(0x7f27740d3090 msgr2=0x7f277415b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.696+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 shutdown_connections 2024-01-15T18:11:57.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:57.697+0000 7f2773200640 1 -- 172.21.15.26:0/4066183010 wait complete. 2024-01-15T18:11:57.721 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi124.front.sepia.ceph.com... 2024-01-15T18:11:57.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:57.722 DEBUG:teuthology.orchestra.run.smithi124:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:11:57.820 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[47374]: fuse finished with error 0 and tester_r 0 2024-01-15T18:11:57.827 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:11:57.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:57.828 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:11:58.007 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:11:58.007 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:58.069 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:11:58.069 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:11:58.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:58.069 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-01-15T18:11:58.124 INFO:teuthology.orchestra.run.smithi026.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-01-15T18:11:58.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:11:58.125 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:11:58.126 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:11:58.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:11:58.184 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:11:58.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:11:58.185 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-01-15T18:11:58.238 INFO:teuthology.orchestra.run.smithi124.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-01-15T18:11:58.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:11:58.240 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:11:58.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.463+0000 7f78dfe00640 1 Processor -- start 2024-01-15T18:11:58.466 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.463+0000 7f78dfe00640 1 -- start start 2024-01-15T18:11:58.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.465+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.465+0000 7f78dfe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f78d813e360 0x7f78d80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.465+0000 7f78dfe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d806c470 con 0x7f78d815f9c0 2024-01-15T18:11:58.468 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.465+0000 7f78dfe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d806c750 con 0x7f78d813e360 2024-01-15T18:11:58.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.465+0000 7f78dfe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d806ca30 con 0x7f78d813df80 2024-01-15T18:11:58.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.466+0000 7f78dd400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.466+0000 7f78dca00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f78d813e360 0x7f78d80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.466+0000 7f78dd400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:60902/0 (socket says 172.21.15.26:60902) 2024-01-15T18:11:58.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.466+0000 7f78dd400640 1 -- 172.21.15.26:0/2180557430 learned_addr learned my addr 172.21.15.26:0/2180557430 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:58.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.467+0000 7f78dd400640 1 -- 172.21.15.26:0/2180557430 >> v2:172.21.15.124:3300/0 conn(0x7f78d813e360 msgr2=0x7f78d80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:58.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.467+0000 7f78dd400640 1 --2- 172.21.15.26:0/2180557430 >> v2:172.21.15.124:3300/0 conn(0x7f78d813e360 0x7f78d80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.467+0000 7f78dd400640 1 -- 172.21.15.26:0/2180557430 >> v1:172.21.15.124:6789/0 conn(0x7f78d815f9c0 legacy=0x7f78d81408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:58.470 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.467+0000 7f78dd400640 1 -- 172.21.15.26:0/2180557430 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d806cd10 con 0x7f78d813df80 2024-01-15T18:11:58.471 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.468+0000 7f78dd400640 1 --2- 172.21.15.26:0/2180557430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d815bee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009870 tx=0x7f78c802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57c867d4ed116336 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:58.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.469+0000 7f78d7e00640 1 -- 172.21.15.26:0/2180557430 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8030840 con 0x7f78d813df80 2024-01-15T18:11:58.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.469+0000 7f78d7e00640 1 -- 172.21.15.26:0/2180557430 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c80309a0 con 0x7f78d813df80 2024-01-15T18:11:58.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.469+0000 7f78d7e00640 1 -- 172.21.15.26:0/2180557430 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8030d30 con 0x7f78d813df80 2024-01-15T18:11:58.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.470+0000 7f78dfe00640 1 -- 172.21.15.26:0/2180557430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 msgr2=0x7f78d815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:58.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.470+0000 7f78dfe00640 1 --2- 172.21.15.26:0/2180557430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d815bee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009870 tx=0x7f78c802b980 comp rx=0 tx=0).stop 2024-01-15T18:11:58.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.470+0000 7f78dfe00640 1 -- 172.21.15.26:0/2180557430 shutdown_connections 2024-01-15T18:11:58.473 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.470+0000 7f78dfe00640 1 --2- 172.21.15.26:0/2180557430 >> v2:172.21.15.124:3300/0 conn(0x7f78d813e360 0x7f78d80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.470+0000 7f78dfe00640 1 --2- 172.21.15.26:0/2180557430 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d815bee0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.470+0000 7f78dfe00640 1 -- 172.21.15.26:0/2180557430 >> 172.21.15.26:0/2180557430 conn(0x7f78d806a800 msgr2=0x7f78d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:58.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.470+0000 7f78dfe00640 1 -- 172.21.15.26:0/2180557430 shutdown_connections 2024-01-15T18:11:58.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.470+0000 7f78dfe00640 1 -- 172.21.15.26:0/2180557430 wait complete. 2024-01-15T18:11:58.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.472+0000 7f78dfe00640 1 Processor -- start 2024-01-15T18:11:58.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.472+0000 7f78dfe00640 1 -- start start 2024-01-15T18:11:58.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d814f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d813e360 0x7f78d814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d815f9c0 0x7f78d80712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dfe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d81500c0 con 0x7f78d815f9c0 2024-01-15T18:11:58.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dfe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d81503a0 con 0x7f78d813e360 2024-01-15T18:11:58.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dfe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d814e240 con 0x7f78d813df80 2024-01-15T18:11:58.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d815f9c0 0x7f78d80712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d815f9c0 0x7f78d80712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37412/0 (socket says 172.21.15.26:37412) 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dde00640 1 -- 172.21.15.26:0/1989196249 learned_addr learned my addr 172.21.15.26:0/1989196249 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dca00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d813e360 0x7f78d814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.474+0000 7f78dd400640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d814f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78dde00640 1 -- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 msgr2=0x7f78d814f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78dde00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d814f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78dde00640 1 -- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d813e360 msgr2=0x7f78d814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78dde00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d813e360 0x7f78d814ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78dde00640 1 -- 172.21.15.26:0/1989196249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c80094d0 con 0x7f78d815f9c0 2024-01-15T18:11:58.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78dde00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d815f9c0 0x7f78d80712e0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f78d000c7a0 tx=0x7f78d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:58.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78d5600640 1 -- 172.21.15.26:0/1989196249 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0003b70 con 0x7f78d815f9c0 2024-01-15T18:11:58.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78d5600640 1 -- 172.21.15.26:0/1989196249 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d0003cf0 con 0x7f78d815f9c0 2024-01-15T18:11:58.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d814e520 con 0x7f78d815f9c0 2024-01-15T18:11:58.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.475+0000 7f78d5600640 1 -- 172.21.15.26:0/1989196249 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0003ea0 con 0x7f78d815f9c0 2024-01-15T18:11:58.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.476+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d814fd30 con 0x7f78d815f9c0 2024-01-15T18:11:58.480 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.477+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d814e0a0 con 0x7f78d815f9c0 2024-01-15T18:11:58.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.478+0000 7f78d5600640 1 -- 172.21.15.26:0/1989196249 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f78d000aa90 con 0x7f78d815f9c0 2024-01-15T18:11:58.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.478+0000 7f78d5600640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b006c460 0x7f78b006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.479+0000 7f78d5600640 1 -- 172.21.15.26:0/1989196249 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f78d0059200 con 0x7f78d815f9c0 2024-01-15T18:11:58.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.479+0000 7f78dd400640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b006c460 0x7f78b006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.480+0000 7f78dd400640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b006c460 0x7f78b006e920 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f78c8036000 tx=0x7f78c803b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:58.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.484+0000 7f78d5600640 1 -- 172.21.15.26:0/1989196249 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78d814e0a0 con 0x7f78d815f9c0 2024-01-15T18:11:58.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.715+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78d814e0a0 con 0x7f78d815f9c0 2024-01-15T18:11:58.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.716+0000 7f78d5600640 1 -- 172.21.15.26:0/1989196249 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3700 (secure 0 0 0) 0x7f78d814e0a0 con 0x7f78d815f9c0 2024-01-15T18:11:58.719 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:11:58.719 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7015,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1631584304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1631584304},{"type":"v1","addr":"172.21.15.124:6835","nonce":1631584304}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2396377458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2396377458},{"type":"v1","addr":"172.21.15.26:6833","nonce":2396377458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/81533892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":81533892},{"type":"v1","addr":"172.21.15.26:6835","nonce":81533892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":36419,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3961747278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3961747278},{"type":"v1","addr":"172.21.15.124:6837","nonce":3961747278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[]} 2024-01-15T18:11:58.720 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 66 2024-01-15T18:11:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.718+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b006c460 msgr2=0x7f78b006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:58.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.718+0000 7f78dfe00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b006c460 0x7f78b006e920 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f78c8036000 tx=0x7f78c803b040 comp rx=0 tx=0).stop 2024-01-15T18:11:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.718+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d815f9c0 msgr2=0x7f78d80712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.718+0000 7f78dfe00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d815f9c0 0x7f78d80712e0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f78d000c7a0 tx=0x7f78d000cc60 comp rx=0 tx=0).stop 2024-01-15T18:11:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.718+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 shutdown_connections 2024-01-15T18:11:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.718+0000 7f78dfe00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78b006c460 0x7f78b006e920 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.718+0000 7f78dfe00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d815f9c0 0x7f78d80712e0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.719+0000 7f78dfe00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d813e360 0x7f78d814ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.719+0000 7f78dfe00640 1 --2- 172.21.15.26:0/1989196249 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d813df80 0x7f78d814f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.719+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 >> 172.21.15.26:0/1989196249 conn(0x7f78d806a800 msgr2=0x7f78d815d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:58.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.719+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 shutdown_connections 2024-01-15T18:11:58.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.719+0000 7f78dfe00640 1 -- 172.21.15.26:0/1989196249 wait complete. 2024-01-15T18:11:58.743 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-01-15T18:11:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.967+0000 7f3381e00640 1 Processor -- start 2024-01-15T18:11:58.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.968+0000 7f3381e00640 1 -- start start 2024-01-15T18:11:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.969+0000 7f3381e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f337c13e360 0x7f337c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.969+0000 7f3381e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f337c15f9c0 0x7f337c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.969+0000 7f3381e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f337c15c5f0 con 0x7f337c13df80 2024-01-15T18:11:58.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.969+0000 7f3381e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337c06c470 con 0x7f337c13e360 2024-01-15T18:11:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.969+0000 7f3381e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337c06c750 con 0x7f337c15f9c0 2024-01-15T18:11:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.970+0000 7f337a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f337c13e360 0x7f337c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.970+0000 7f337a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f337c13e360 0x7f337c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37416/0 (socket says 172.21.15.26:37416) 2024-01-15T18:11:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.970+0000 7f337a000640 1 -- 172.21.15.26:0/1499646611 learned_addr learned my addr 172.21.15.26:0/1499646611 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.970+0000 7f337a000640 1 -- 172.21.15.26:0/1499646611 >> v2:172.21.15.124:3300/0 conn(0x7f337c15f9c0 msgr2=0x7f337c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.970+0000 7f337a000640 1 --2- 172.21.15.26:0/1499646611 >> v2:172.21.15.124:3300/0 conn(0x7f337c15f9c0 0x7f337c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.970+0000 7f337a000640 1 -- 172.21.15.26:0/1499646611 >> v1:172.21.15.124:6789/0 conn(0x7f337c13df80 legacy=0x7f337c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:58.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.970+0000 7f337a000640 1 -- 172.21.15.26:0/1499646611 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c06ca30 con 0x7f337c13e360 2024-01-15T18:11:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.971+0000 7f337a000640 1 --2- 172.21.15.26:0/1499646611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 0x7f337c0cf290 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f337c06ded0 tx=0x7f336402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62a5635525e1a9b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.971+0000 7f3379600640 1 -- 172.21.15.26:0/1499646611 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336402ca80 con 0x7f337c13e360 2024-01-15T18:11:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3379600640 1 -- 172.21.15.26:0/1499646611 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336402cbe0 con 0x7f337c13e360 2024-01-15T18:11:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3379600640 1 -- 172.21.15.26:0/1499646611 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33640343f0 con 0x7f337c13e360 2024-01-15T18:11:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3381e00640 1 -- 172.21.15.26:0/1499646611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 msgr2=0x7f337c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:58.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3381e00640 1 --2- 172.21.15.26:0/1499646611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 0x7f337c0cf290 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f337c06ded0 tx=0x7f336402bee0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3381e00640 1 -- 172.21.15.26:0/1499646611 shutdown_connections 2024-01-15T18:11:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3381e00640 1 --2- 172.21.15.26:0/1499646611 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 0x7f337c0cf290 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3381e00640 1 --2- 172.21.15.26:0/1499646611 >> v2:172.21.15.124:3300/0 conn(0x7f337c15f9c0 0x7f337c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3381e00640 1 -- 172.21.15.26:0/1499646611 >> 172.21.15.26:0/1499646611 conn(0x7f337c06a800 msgr2=0x7f337c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3381e00640 1 -- 172.21.15.26:0/1499646611 shutdown_connections 2024-01-15T18:11:58.977 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.972+0000 7f3381e00640 1 -- 172.21.15.26:0/1499646611 wait complete. 2024-01-15T18:11:58.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.974+0000 7f3381e00640 1 Processor -- start 2024-01-15T18:11:58.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.974+0000 7f3381e00640 1 -- start start 2024-01-15T18:11:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.975+0000 7f3381e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f337c13df80 0x7f337c070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.975+0000 7f3381e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 0x7f337c15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f3381e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f337c15f9c0 0x7f337c15a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f3381e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c071f90 con 0x7f337c13e360 2024-01-15T18:11:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f3381e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337c071310 con 0x7f337c15f9c0 2024-01-15T18:11:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f3381e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337c0715f0 con 0x7f337c13df80 2024-01-15T18:11:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 0x7f337c15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 0x7f337c15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37432/0 (socket says 172.21.15.26:37432) 2024-01-15T18:11:58.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337a000640 1 -- 172.21.15.26:0/2468989291 learned_addr learned my addr 172.21.15.26:0/2468989291 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337aa00640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f337c13df80 0x7f337c070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337a000640 1 -- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f337c13df80 msgr2=0x7f337c070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337a000640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f337c13df80 0x7f337c070c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337a000640 1 -- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f337c15f9c0 msgr2=0x7f337c15a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337b400640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f337c15f9c0 0x7f337c15a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337a000640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f337c15f9c0 0x7f337c15a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.976+0000 7f337a000640 1 -- 172.21.15.26:0/2468989291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3364009b30 con 0x7f337c13e360 2024-01-15T18:11:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.977+0000 7f337a000640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 0x7f337c15a180 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f33640093e0 tx=0x7f3364004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:58.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.977+0000 7f3373400640 1 -- 172.21.15.26:0/2468989291 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336403e450 con 0x7f337c13e360 2024-01-15T18:11:58.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.977+0000 7f3373400640 1 -- 172.21.15.26:0/2468989291 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336403e5d0 con 0x7f337c13e360 2024-01-15T18:11:58.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.977+0000 7f3373400640 1 -- 172.21.15.26:0/2468989291 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336403e980 con 0x7f337c13e360 2024-01-15T18:11:58.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.977+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337c0718d0 con 0x7f337c13e360 2024-01-15T18:11:58.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.977+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337c0b3210 con 0x7f337c13e360 2024-01-15T18:11:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.978+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337c0675d0 con 0x7f337c13e360 2024-01-15T18:11:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.979+0000 7f3373400640 1 -- 172.21.15.26:0/2468989291 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f336402cd50 con 0x7f337c13e360 2024-01-15T18:11:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.980+0000 7f3373400640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f335406c430 0x7f335406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:58.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.980+0000 7f3373400640 1 -- 172.21.15.26:0/2468989291 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f33640ab8a0 con 0x7f337c13e360 2024-01-15T18:11:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.981+0000 7f337aa00640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f335406c430 0x7f335406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:58.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.982+0000 7f337aa00640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f335406c430 0x7f335406e8f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f336c00a730 tx=0x7f336c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:58.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:58.985+0000 7f3373400640 1 -- 172.21.15.26:0/2468989291 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3364077f30 con 0x7f337c13e360 2024-01-15T18:11:59.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.183+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f337c0ce580 con 0x7f337c13e360 2024-01-15T18:11:59.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.189+0000 7f3373400640 1 -- 172.21.15.26:0/2468989291 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f33640430a0 con 0x7f337c13e360 2024-01-15T18:11:59.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f335406c430 msgr2=0x7f335406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f335406c430 0x7f335406e8f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f336c00a730 tx=0x7f336c008040 comp rx=0 tx=0).stop 2024-01-15T18:11:59.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 msgr2=0x7f337c15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 0x7f337c15a180 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f33640093e0 tx=0x7f3364004a40 comp rx=0 tx=0).stop 2024-01-15T18:11:59.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 shutdown_connections 2024-01-15T18:11:59.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f335406c430 0x7f335406e8f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f337c15f9c0 0x7f337c15a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f337c13e360 0x7f337c15a180 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 --2- 172.21.15.26:0/2468989291 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f337c13df80 0x7f337c070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 >> 172.21.15.26:0/2468989291 conn(0x7f337c06a800 msgr2=0x7f337c154390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:59.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.192+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 shutdown_connections 2024-01-15T18:11:59.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.193+0000 7f3381e00640 1 -- 172.21.15.26:0/2468989291 wait complete. 2024-01-15T18:11:59.216 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/snapshot_clone_delay 2024-01-15T18:11:59.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.441+0000 7f63b3600640 1 Processor -- start 2024-01-15T18:11:59.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.441+0000 7f63b3600640 1 -- start start 2024-01-15T18:11:59.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.443+0000 7f63b3600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.443+0000 7f63b3600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f63ac15f9c0 0x7f63ac0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.443+0000 7f63b3600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac15c5f0 con 0x7f63ac13df80 2024-01-15T18:11:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.443+0000 7f63b3600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac06c470 con 0x7f63ac15f9c0 2024-01-15T18:11:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.443+0000 7f63b3600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac06c750 con 0x7f63ac13e360 2024-01-15T18:11:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.444+0000 7f63abe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.444+0000 7f63abe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:60930/0 (socket says 172.21.15.26:60930) 2024-01-15T18:11:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.444+0000 7f63abe00640 1 -- 172.21.15.26:0/1729884217 learned_addr learned my addr 172.21.15.26:0/1729884217 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:59.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.444+0000 7f63b1600640 1 --2- 172.21.15.26:0/1729884217 >> v2:172.21.15.124:3300/0 conn(0x7f63ac15f9c0 0x7f63ac0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.444+0000 7f63abe00640 1 -- 172.21.15.26:0/1729884217 >> v2:172.21.15.124:3300/0 conn(0x7f63ac15f9c0 msgr2=0x7f63ac0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.444+0000 7f63abe00640 1 --2- 172.21.15.26:0/1729884217 >> v2:172.21.15.124:3300/0 conn(0x7f63ac15f9c0 0x7f63ac0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.444+0000 7f63abe00640 1 -- 172.21.15.26:0/1729884217 >> v1:172.21.15.124:6789/0 conn(0x7f63ac13df80 legacy=0x7f63ac15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.444+0000 7f63abe00640 1 -- 172.21.15.26:0/1729884217 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac06ca30 con 0x7f63ac13e360 2024-01-15T18:11:59.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.446+0000 7f63abe00640 1 --2- 172.21.15.26:0/1729884217 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac0cf290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6394009870 tx=0x7f639402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=572b066eb6bc0919 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:59.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.447+0000 7f63ab400640 1 -- 172.21.15.26:0/1729884217 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394031640 con 0x7f63ac13e360 2024-01-15T18:11:59.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.447+0000 7f63ab400640 1 -- 172.21.15.26:0/1729884217 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63940317a0 con 0x7f63ac13e360 2024-01-15T18:11:59.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.447+0000 7f63ab400640 1 -- 172.21.15.26:0/1729884217 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394031b30 con 0x7f63ac13e360 2024-01-15T18:11:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.448+0000 7f63b3600640 1 -- 172.21.15.26:0/1729884217 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 msgr2=0x7f63ac0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.448+0000 7f63b3600640 1 --2- 172.21.15.26:0/1729884217 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac0cf290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6394009870 tx=0x7f639402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.448+0000 7f63b3600640 1 -- 172.21.15.26:0/1729884217 shutdown_connections 2024-01-15T18:11:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.448+0000 7f63b3600640 1 --2- 172.21.15.26:0/1729884217 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac0cf290 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.448+0000 7f63b3600640 1 --2- 172.21.15.26:0/1729884217 >> v2:172.21.15.124:3300/0 conn(0x7f63ac15f9c0 0x7f63ac0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.448+0000 7f63b3600640 1 -- 172.21.15.26:0/1729884217 >> 172.21.15.26:0/1729884217 conn(0x7f63ac06a800 msgr2=0x7f63ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.448+0000 7f63b3600640 1 -- 172.21.15.26:0/1729884217 shutdown_connections 2024-01-15T18:11:59.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.448+0000 7f63b3600640 1 -- 172.21.15.26:0/1729884217 wait complete. 2024-01-15T18:11:59.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.450+0000 7f63b3600640 1 Processor -- start 2024-01-15T18:11:59.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.451+0000 7f63b3600640 1 -- start start 2024-01-15T18:11:59.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.452+0000 7f63b3600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63ac13df80 0x7f63ac070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.452+0000 7f63b3600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac137900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.452+0000 7f63b3600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63ac15f9c0 0x7f63ac13ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.452+0000 7f63b3600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac137e40 con 0x7f63ac15f9c0 2024-01-15T18:11:59.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.452+0000 7f63b3600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac071710 con 0x7f63ac13df80 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.452+0000 7f63b3600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac13b2b0 con 0x7f63ac13e360 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.452+0000 7f63b1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63ac15f9c0 0x7f63ac13ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63ac15f9c0 0x7f63ac13ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37448/0 (socket says 172.21.15.26:37448) 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b1600640 1 -- 172.21.15.26:0/1315883157 learned_addr learned my addr 172.21.15.26:0/1315883157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63abe00640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac137900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b0c00640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63ac13df80 0x7f63ac070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b1600640 1 -- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 msgr2=0x7f63ac137900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b1600640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac137900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b1600640 1 -- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63ac13df80 msgr2=0x7f63ac070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b1600640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63ac13df80 0x7f63ac070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b1600640 1 -- 172.21.15.26:0/1315883157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940094d0 con 0x7f63ac15f9c0 2024-01-15T18:11:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b0c00640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63ac13df80 0x7f63ac070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:11:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.453+0000 7f63b1600640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63ac15f9c0 0x7f63ac13ad70 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f63a4007e40 tx=0x7f63a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.454+0000 7f63a8c00640 1 -- 172.21.15.26:0/1315883157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a4014620 con 0x7f63ac15f9c0 2024-01-15T18:11:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.454+0000 7f63a8c00640 1 -- 172.21.15.26:0/1315883157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a400ee90 con 0x7f63ac15f9c0 2024-01-15T18:11:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.454+0000 7f63a8c00640 1 -- 172.21.15.26:0/1315883157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a40149e0 con 0x7f63ac15f9c0 2024-01-15T18:11:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.454+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac070b00 con 0x7f63ac15f9c0 2024-01-15T18:11:59.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.454+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac0710c0 con 0x7f63ac15f9c0 2024-01-15T18:11:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.455+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ac067000 con 0x7f63ac15f9c0 2024-01-15T18:11:59.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.456+0000 7f63a8c00640 1 -- 172.21.15.26:0/1315883157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f63a4014b40 con 0x7f63ac15f9c0 2024-01-15T18:11:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.457+0000 7f63a8c00640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f638406c460 0x7f638406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.457+0000 7f63a8c00640 1 -- 172.21.15.26:0/1315883157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f63a40893d0 con 0x7f63ac15f9c0 2024-01-15T18:11:59.461 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.458+0000 7f63b0c00640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f638406c460 0x7f638406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.459+0000 7f63b0c00640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f638406c460 0x7f638406e920 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f639c006fd0 tx=0x7f639c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:59.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.461+0000 7f63a8c00640 1 -- 172.21.15.26:0/1315883157 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63a4055940 con 0x7f63ac15f9c0 2024-01-15T18:11:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.655+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"} v 0) v1 -- 0x7f63ac071880 con 0x7f63ac15f9c0 2024-01-15T18:11:59.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.662+0000 7f63a8c00640 1 -- 172.21.15.26:0/1315883157 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/snapshot_clone_delay"}]=0 v38) v1 ==== 115+0+0 (secure 0 0 0) 0x7f63a4058f60 con 0x7f63ac15f9c0 2024-01-15T18:11:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f638406c460 msgr2=0x7f638406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f638406c460 0x7f638406e920 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f639c006fd0 tx=0x7f639c008040 comp rx=0 tx=0).stop 2024-01-15T18:11:59.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63ac15f9c0 msgr2=0x7f63ac13ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63ac15f9c0 0x7f63ac13ad70 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f63a4007e40 tx=0x7f63a400e4b0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 shutdown_connections 2024-01-15T18:11:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f63ac13e360 0x7f63ac137900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f638406c460 0x7f638406e920 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f63ac15f9c0 0x7f63ac13ad70 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 --2- 172.21.15.26:0/1315883157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f63ac13df80 0x7f63ac070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.665+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 >> 172.21.15.26:0/1315883157 conn(0x7f63ac06a800 msgr2=0x7f63ac15cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.666+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 shutdown_connections 2024-01-15T18:11:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.666+0000 7f63b3600640 1 -- 172.21.15.26:0/1315883157 wait complete. 2024-01-15T18:11:59.687 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/volumes/clone_delay_every_regfile 2024-01-15T18:11:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.904+0000 7f330ca00640 1 Processor -- start 2024-01-15T18:11:59.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.904+0000 7f330ca00640 1 -- start start 2024-01-15T18:11:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.906+0000 7f330ca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3308071e40 0x7f33080ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.906+0000 7f330ca00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f330815d770 0x7f330815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.906+0000 7f330ca00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330806c730 con 0x7f3308143810 2024-01-15T18:11:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.906+0000 7f330ca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3308143bf0 con 0x7f3308071e40 2024-01-15T18:11:59.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.906+0000 7f330ca00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33080722f0 con 0x7f330815d770 2024-01-15T18:11:59.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.906+0000 7f3307400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f3308143810 0x7f330815e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56620/0 (socket says 172.21.15.26:56620) 2024-01-15T18:11:59.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.906+0000 7f3307400640 1 -- 172.21.15.26:0/3269778911 learned_addr learned my addr 172.21.15.26:0/3269778911 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:11:59.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.906+0000 7f3306a00640 1 --2- 172.21.15.26:0/3269778911 >> v2:172.21.15.26:3300/0 conn(0x7f3308071e40 0x7f33080ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.907+0000 7f3306000640 1 -- 172.21.15.26:0/3269778911 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658504144 0 0) 0x7f330806c730 con 0x7f3308143810 2024-01-15T18:11:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.907+0000 7f3306000640 1 -- 172.21.15.26:0/3269778911 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32e8003520 con 0x7f3308143810 2024-01-15T18:11:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.907+0000 7f3307e00640 1 --2- 172.21.15.26:0/3269778911 >> v2:172.21.15.124:3300/0 conn(0x7f330815d770 0x7f330815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.907+0000 7f3306a00640 1 -- 172.21.15.26:0/3269778911 >> v2:172.21.15.124:3300/0 conn(0x7f330815d770 msgr2=0x7f330815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:11:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.907+0000 7f3306a00640 1 --2- 172.21.15.26:0/3269778911 >> v2:172.21.15.124:3300/0 conn(0x7f330815d770 0x7f330815dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.907+0000 7f3306a00640 1 -- 172.21.15.26:0/3269778911 >> v1:172.21.15.26:6789/0 conn(0x7f3308143810 legacy=0x7f330815e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.907+0000 7f3306a00640 1 -- 172.21.15.26:0/3269778911 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33080724d0 con 0x7f3308071e40 2024-01-15T18:11:59.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.907+0000 7f3307e00640 1 --2- 172.21.15.26:0/3269778911 >> v2:172.21.15.124:3300/0 conn(0x7f330815d770 0x7f330815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:11:59.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.908+0000 7f3306a00640 1 --2- 172.21.15.26:0/3269778911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308071e40 0x7f33080ae170 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f33080b4440 tx=0x7f32f002b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e33d445d09b5151c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.908+0000 7f3306000640 1 -- 172.21.15.26:0/3269778911 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0031710 con 0x7f3308071e40 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.908+0000 7f3306000640 1 -- 172.21.15.26:0/3269778911 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f002f050 con 0x7f3308071e40 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.908+0000 7f3306000640 1 -- 172.21.15.26:0/3269778911 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0031a90 con 0x7f3308071e40 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.909+0000 7f330ca00640 1 -- 172.21.15.26:0/3269778911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308071e40 msgr2=0x7f33080ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.909+0000 7f330ca00640 1 --2- 172.21.15.26:0/3269778911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308071e40 0x7f33080ae170 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f33080b4440 tx=0x7f32f002b7b0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.909+0000 7f330ca00640 1 -- 172.21.15.26:0/3269778911 shutdown_connections 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.909+0000 7f330ca00640 1 --2- 172.21.15.26:0/3269778911 >> v2:172.21.15.124:3300/0 conn(0x7f330815d770 0x7f330815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.909+0000 7f330ca00640 1 --2- 172.21.15.26:0/3269778911 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308071e40 0x7f33080ae170 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.909+0000 7f330ca00640 1 -- 172.21.15.26:0/3269778911 >> 172.21.15.26:0/3269778911 conn(0x7f330806a7d0 msgr2=0x7f330806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.909+0000 7f330ca00640 1 -- 172.21.15.26:0/3269778911 shutdown_connections 2024-01-15T18:11:59.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.909+0000 7f330ca00640 1 -- 172.21.15.26:0/3269778911 wait complete. 2024-01-15T18:11:59.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.911+0000 7f330ca00640 1 Processor -- start 2024-01-15T18:11:59.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.911+0000 7f330ca00640 1 -- start start 2024-01-15T18:11:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.912+0000 7f330ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3308071e40 0x7f33080cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.912+0000 7f330ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308143810 0x7f330815ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.912+0000 7f330ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f330815d770 0x7f3308071310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.912+0000 7f330ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33080ccaf0 con 0x7f3308143810 2024-01-15T18:11:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.912+0000 7f330ca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33080cd270 con 0x7f330815d770 2024-01-15T18:11:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.912+0000 7f330ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33080cd550 con 0x7f3308071e40 2024-01-15T18:11:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.912+0000 7f3306a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308143810 0x7f330815ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.912+0000 7f3306a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308143810 0x7f330815ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37472/0 (socket says 172.21.15.26:37472) 2024-01-15T18:11:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.912+0000 7f3306a00640 1 -- 172.21.15.26:0/1138721645 learned_addr learned my addr 172.21.15.26:0/1138721645 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:11:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.913+0000 7f3307400640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3308071e40 0x7f33080cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.913+0000 7f3307e00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f330815d770 0x7f3308071310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.913+0000 7f3306a00640 1 -- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3308071e40 msgr2=0x7f33080cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.913+0000 7f3306a00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3308071e40 0x7f33080cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.913+0000 7f3306a00640 1 -- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f330815d770 msgr2=0x7f3308071310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:11:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.913+0000 7f3306a00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f330815d770 0x7f3308071310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:11:59.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.913+0000 7f3306a00640 1 -- 172.21.15.26:0/1138721645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f0009400 con 0x7f3308143810 2024-01-15T18:11:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.913+0000 7f3306a00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308143810 0x7f330815ca90 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f32f0000c00 tx=0x7f32f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.914+0000 7f32ff400640 1 -- 172.21.15.26:0/1138721645 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0033890 con 0x7f3308143810 2024-01-15T18:11:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.914+0000 7f32ff400640 1 -- 172.21.15.26:0/1138721645 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f00339f0 con 0x7f3308143810 2024-01-15T18:11:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.914+0000 7f32ff400640 1 -- 172.21.15.26:0/1138721645 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0033da0 con 0x7f3308143810 2024-01-15T18:11:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.914+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33080cd830 con 0x7f3308143810 2024-01-15T18:11:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.914+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33080cddf0 con 0x7f3308143810 2024-01-15T18:11:59.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.915+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3308067450 con 0x7f3308143810 2024-01-15T18:11:59.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.916+0000 7f32ff400640 1 -- 172.21.15.26:0/1138721645 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f32f003a040 con 0x7f3308143810 2024-01-15T18:11:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.917+0000 7f32ff400640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32e006c430 0x7f32e006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:11:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.917+0000 7f32ff400640 1 -- 172.21.15.26:0/1138721645 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f32f00aa010 con 0x7f3308143810 2024-01-15T18:11:59.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.918+0000 7f3307400640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32e006c430 0x7f32e006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:11:59.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.919+0000 7f3307400640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32e006c430 0x7f32e006e8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32f80011e0 tx=0x7f32f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:11:59.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:11:59.921+0000 7f32ff400640 1 -- 172.21.15.26:0/1138721645 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32f00765c0 con 0x7f3308143810 2024-01-15T18:12:00.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.119+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"} v 0) v1 -- 0x7f33080cb350 con 0x7f3308143810 2024-01-15T18:12:00.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.126+0000 7f32ff400640 1 -- 172.21.15.26:0/1138721645 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/volumes/clone_delay_every_regfile"}]=0 v39) v1 ==== 120+0+0 (secure 0 0 0) 0x7f32f00380b0 con 0x7f3308143810 2024-01-15T18:12:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32e006c430 msgr2=0x7f32e006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32e006c430 0x7f32e006e8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32f80011e0 tx=0x7f32f8009040 comp rx=0 tx=0).stop 2024-01-15T18:12:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308143810 msgr2=0x7f330815ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308143810 0x7f330815ca90 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f32f0000c00 tx=0x7f32f00046c0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 shutdown_connections 2024-01-15T18:12:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32e006c430 0x7f32e006e8f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f330815d770 0x7f3308071310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3308143810 0x7f330815ca90 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 --2- 172.21.15.26:0/1138721645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3308071e40 0x7f33080cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 >> 172.21.15.26:0/1138721645 conn(0x7f330806a7d0 msgr2=0x7f33080b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 shutdown_connections 2024-01-15T18:12:00.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.129+0000 7f330ca00640 1 -- 172.21.15.26:0/1138721645 wait complete. 2024-01-15T18:12:00.153 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled' 2024-01-15T18:12:00.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.363+0000 7f86c2000640 1 Processor -- start 2024-01-15T18:12:00.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.364+0000 7f86c2000640 1 -- start start 2024-01-15T18:12:00.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.365+0000 7f86c2000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc0d1ad0 0x7f86bc0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:00.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.365+0000 7f86c2000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc0cec40 con 0x7f86bc0d1650 2024-01-15T18:12:00.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.365+0000 7f86c2000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc0d0420 con 0x7f86bc0b34e0 2024-01-15T18:12:00.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.365+0000 7f86c2000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0d0e70 con 0x7f86bc0d1ad0 2024-01-15T18:12:00.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86b3e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86bc0d1650 0x7f86bc0c6be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56632/0 (socket says 172.21.15.26:56632) 2024-01-15T18:12:00.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86b3e00640 1 -- 172.21.15.26:0/32353769 learned_addr learned my addr 172.21.15.26:0/32353769 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986458568 0 0) 0x7f86bc0cec40 con 0x7f86bc0d1650 2024-01-15T18:12:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86c1600640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc0d1ad0 0x7f86bc0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f869c003520 con 0x7f86bc0d1650 2024-01-15T18:12:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4172857866 0 0) 0x7f869c003520 con 0x7f86bc0d1650 2024-01-15T18:12:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc0d1ad0 msgr2=0x7f86bc0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86bbe00640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc0d1ad0 0x7f86bc0c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 >> v1:172.21.15.124:6789/0 conn(0x7f86bc0b34e0 legacy=0x7f86bc0b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:00.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc0d1f60 con 0x7f86bc0d1650 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.366+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f86a4001670 con 0x7f86bc0d1650 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 >> v1:172.21.15.26:6789/0 conn(0x7f86bc0d1650 legacy=0x7f86bc0c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86bbe00640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0039a0 0x7f869c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86bbe00640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f869c0047c0 0x7f869c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86bbe00640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0051d0 0x7f869c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0d0e70 con 0x7f869c0047c0 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c00c080 con 0x7f869c0039a0 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c00c240 con 0x7f869c0051d0 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86c1600640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc0d1ad0 0x7f86bc0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86b3e00640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f869c0047c0 0x7f869c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86c0c00640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0039a0 0x7f869c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:00.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86b3e00640 1 -- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0051d0 msgr2=0x7f869c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86b3e00640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f869c0051d0 0x7f869c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86b3e00640 1 -- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0039a0 msgr2=0x7f869c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86b3e00640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0039a0 0x7f869c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.373 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.367+0000 7f86b3e00640 1 -- 172.21.15.26:0/32353769 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c003520 con 0x7f869c0047c0 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.369+0000 7f86b3e00640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f869c0047c0 0x7f869c004c40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f86a400c0e0 tx=0x7f869c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fcd70c9123526f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.369+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4005ad0 con 0x7f869c0047c0 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.369+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a4005c30 con 0x7f869c0047c0 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.369+0000 7f86bbe00640 1 -- 172.21.15.26:0/32353769 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a402c810 con 0x7f869c0047c0 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.369+0000 7f86c2000640 1 -- 172.21.15.26:0/32353769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f869c0047c0 msgr2=0x7f869c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.370+0000 7f86c2000640 1 --2- 172.21.15.26:0/32353769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f869c0047c0 0x7f869c004c40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f86a400c0e0 tx=0x7f869c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.370+0000 7f86c1600640 1 -- 172.21.15.26:0/32353769 reap_dead start 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.370+0000 7f86c2000640 1 -- 172.21.15.26:0/32353769 shutdown_connections 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.370+0000 7f86c2000640 1 -- 172.21.15.26:0/32353769 >> 172.21.15.26:0/32353769 conn(0x7f86bc0d3090 msgr2=0x7f86bc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.370+0000 7f86c2000640 1 -- 172.21.15.26:0/32353769 shutdown_connections 2024-01-15T18:12:00.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.370+0000 7f86c2000640 1 -- 172.21.15.26:0/32353769 wait complete. 2024-01-15T18:12:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.372+0000 7f86c2000640 1 Processor -- start 2024-01-15T18:12:00.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.372+0000 7f86c2000640 1 -- start start 2024-01-15T18:12:00.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86c2000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0047c0 0x7f86bc0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86c2000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86bc0cc200 0x7f86bc0ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86c2000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc1f1fc0 0x7f86bc1f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86c2000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0c6830 con 0x7f86bc0cc200 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86c2000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc14cab0 con 0x7f869c0047c0 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86c2000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0cc750 con 0x7f86bc1f1fc0 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86bc0cc200 0x7f86bc0ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86bc0cc200 0x7f86bc0ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37496/0 (socket says 172.21.15.26:37496) 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86b3e00640 1 -- 172.21.15.26:0/1155900331 learned_addr learned my addr 172.21.15.26:0/1155900331 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86c0c00640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0047c0 0x7f86bc0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.373+0000 7f86c1600640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc1f1fc0 0x7f86bc1f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86b3e00640 1 -- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc1f1fc0 msgr2=0x7f86bc1f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:00.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86b3e00640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc1f1fc0 0x7f86bc1f2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86b3e00640 1 -- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0047c0 msgr2=0x7f86bc0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86b3e00640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0047c0 0x7f86bc0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86b3e00640 1 -- 172.21.15.26:0/1155900331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc0cca30 con 0x7f86bc0cc200 2024-01-15T18:12:00.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86c0c00640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0047c0 0x7f86bc0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86b3e00640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86bc0cc200 0x7f86bc0ca6a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f86a400c0e0 tx=0x7f86a4001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86b9600640 1 -- 172.21.15.26:0/1155900331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4039070 con 0x7f86bc0cc200 2024-01-15T18:12:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86b9600640 1 -- 172.21.15.26:0/1155900331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a402cc20 con 0x7f86bc0cc200 2024-01-15T18:12:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.374+0000 7f86b9600640 1 -- 172.21.15.26:0/1155900331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a403c040 con 0x7f86bc0cc200 2024-01-15T18:12:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.375+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc0ccd10 con 0x7f86bc0cc200 2024-01-15T18:12:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.375+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc0cd2d0 con 0x7f86bc0cc200 2024-01-15T18:12:00.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.376+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc0cd460 con 0x7f86bc0cc200 2024-01-15T18:12:00.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.377+0000 7f86b9600640 1 -- 172.21.15.26:0/1155900331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f86a4032070 con 0x7f86bc0cc200 2024-01-15T18:12:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.378+0000 7f86b9600640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f869406c460 0x7f869406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:00.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.378+0000 7f86b9600640 1 -- 172.21.15.26:0/1155900331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f86a40a9960 con 0x7f86bc0cc200 2024-01-15T18:12:00.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.379+0000 7f86c0c00640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f869406c460 0x7f869406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:00.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.380+0000 7f86c0c00640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f869406c460 0x7f869406e920 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0035f0 tx=0x7f86ac009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:00.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.383+0000 7f86b9600640 1 -- 172.21.15.26:0/1155900331 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86a4076220 con 0x7f86bc0cc200 2024-01-15T18:12:00.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:00.646+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled"]} v 0) v1 -- 0x7f86bc067450 con 0x7f86bc0cc200 2024-01-15T18:12:01.412 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:12:01.421 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:12:01.487 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.484+0000 7f86b9600640 1 -- 172.21.15.26:0/1155900331 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestCloneProgress.test_when_1_clone_is_cancelled"]}]=0 v760) v1 ==== 153+0+0 (secure 0 0 0) 0x7f86a4079840 con 0x7f86bc0cc200 2024-01-15T18:12:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f869406c460 msgr2=0x7f869406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f869406c460 0x7f869406e920 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0035f0 tx=0x7f86ac009040 comp rx=0 tx=0).stop 2024-01-15T18:12:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86bc0cc200 msgr2=0x7f86bc0ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86bc0cc200 0x7f86bc0ca6a0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f86a400c0e0 tx=0x7f86a4001930 comp rx=0 tx=0).stop 2024-01-15T18:12:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 shutdown_connections 2024-01-15T18:12:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86bc1f1fc0 0x7f86bc1f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f869406c460 0x7f869406e920 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f86bc0cc200 0x7f86bc0ca6a0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 --2- 172.21.15.26:0/1155900331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f869c0047c0 0x7f86bc0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.487+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 >> 172.21.15.26:0/1155900331 conn(0x7f86bc0d3090 msgr2=0x7f86bc0c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.488+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 shutdown_connections 2024-01-15T18:12:01.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.488+0000 7f86c2000640 1 -- 172.21.15.26:0/1155900331 wait complete. 2024-01-15T18:12:01.512 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the ... ok 2024-01-15T18:12:01.513 INFO:tasks.cephfs_test_runner:Starting test: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:12:01.513 INFO:tasks.cephfs_test_runner:test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:12:01.514 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:12:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.710+0000 7f6f10a00640 1 Processor -- start 2024-01-15T18:12:01.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.710+0000 7f6f10a00640 1 -- start start 2024-01-15T18:12:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.712+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f0c13cfa0 0x7f6f0c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.712+0000 7f6f10a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f0c0cd530 0x7f6f0c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.712+0000 7f6f10a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6f0c0cd9b0 0x7f6f0c13fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:01.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.712+0000 7f6f10a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0c0cdec0 con 0x7f6f0c0cd530 2024-01-15T18:12:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.712+0000 7f6f10a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0c0ce1a0 con 0x7f6f0c0cd9b0 2024-01-15T18:12:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.712+0000 7f6f10a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0ce480 con 0x7f6f0c13cfa0 2024-01-15T18:12:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.712+0000 7f6f02800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f0c0cd530 0x7f6f0c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.712+0000 7f6f02800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6f0c0cd530 0x7f6f0c13f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37500/0 (socket says 172.21.15.26:37500) 2024-01-15T18:12:01.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.712+0000 7f6f02800640 1 -- 172.21.15.26:0/3528964132 learned_addr learned my addr 172.21.15.26:0/3528964132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.713+0000 7f6f02800640 1 -- 172.21.15.26:0/3528964132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f0c13cfa0 msgr2=0x7f6f0c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.713+0000 7f6f02800640 1 --2- 172.21.15.26:0/3528964132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f0c13cfa0 0x7f6f0c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.713+0000 7f6f02800640 1 -- 172.21.15.26:0/3528964132 >> v2:172.21.15.124:3300/0 conn(0x7f6f0c0cd9b0 msgr2=0x7f6f0c13fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.713+0000 7f6f02800640 1 --2- 172.21.15.26:0/3528964132 >> v2:172.21.15.124:3300/0 conn(0x7f6f0c0cd9b0 0x7f6f0c13fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.713+0000 7f6f02800640 1 -- 172.21.15.26:0/3528964132 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c0ce760 con 0x7f6f0c0cd530 2024-01-15T18:12:01.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.714+0000 7f6f02800640 1 --2- 172.21.15.26:0/3528964132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 0x7f6f0c13f5a0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c06ce70 tx=0x7f6ef402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7b3aef1ec85b1f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.714+0000 7f6f0aa00640 1 -- 172.21.15.26:0/3528964132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40317e0 con 0x7f6f0c0cd530 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.714+0000 7f6f0aa00640 1 -- 172.21.15.26:0/3528964132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef4031940 con 0x7f6f0c0cd530 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.714+0000 7f6f0aa00640 1 -- 172.21.15.26:0/3528964132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4031cd0 con 0x7f6f0c0cd530 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.715+0000 7f6f10a00640 1 -- 172.21.15.26:0/3528964132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 msgr2=0x7f6f0c13f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.715+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3528964132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 0x7f6f0c13f5a0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c06ce70 tx=0x7f6ef402b880 comp rx=0 tx=0).stop 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.715+0000 7f6f10a00640 1 -- 172.21.15.26:0/3528964132 shutdown_connections 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.715+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3528964132 >> v2:172.21.15.124:3300/0 conn(0x7f6f0c0cd9b0 0x7f6f0c13fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.715+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3528964132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 0x7f6f0c13f5a0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.715+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3528964132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f0c13cfa0 0x7f6f0c15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.715+0000 7f6f10a00640 1 -- 172.21.15.26:0/3528964132 >> 172.21.15.26:0/3528964132 conn(0x7f6f0c06a800 msgr2=0x7f6f0c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:01.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.715+0000 7f6f10a00640 1 -- 172.21.15.26:0/3528964132 shutdown_connections 2024-01-15T18:12:01.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.715+0000 7f6f10a00640 1 -- 172.21.15.26:0/3528964132 wait complete. 2024-01-15T18:12:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.718+0000 7f6f10a00640 1 Processor -- start 2024-01-15T18:12:01.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.718+0000 7f6f10a00640 1 -- start start 2024-01-15T18:12:01.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.719+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 0x7f6f0c154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:01.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.719+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f0c0cd9b0 0x7f6f0c0addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:01.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.719+0000 7f6f10a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f0c13cfa0 0x7f6f0c156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:01.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.719+0000 7f6f10a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c155190 con 0x7f6f0c0cd530 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.719+0000 7f6f10a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c155470 con 0x7f6f0c0cd9b0 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.719+0000 7f6f10a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c155750 con 0x7f6f0c13cfa0 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.719+0000 7f6f0b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 0x7f6f0c154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.719+0000 7f6f0b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 0x7f6f0c154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:37506/0 (socket says 172.21.15.26:37506) 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.719+0000 7f6f0b400640 1 -- 172.21.15.26:0/3142552679 learned_addr learned my addr 172.21.15.26:0/3142552679 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.720+0000 7f6f02800640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f0c0cd9b0 0x7f6f0c0addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.720+0000 7f6f0be00640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f0c13cfa0 0x7f6f0c156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.720+0000 7f6f0b400640 1 -- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f0c13cfa0 msgr2=0x7f6f0c156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.720+0000 7f6f0b400640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f0c13cfa0 0x7f6f0c156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.720+0000 7f6f0b400640 1 -- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f0c0cd9b0 msgr2=0x7f6f0c0addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:01.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.720+0000 7f6f0b400640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f0c0cd9b0 0x7f6f0c0addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.720+0000 7f6f0b400640 1 -- 172.21.15.26:0/3142552679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40094d0 con 0x7f6f0c0cd530 2024-01-15T18:12:01.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.720+0000 7f6f0b400640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 0x7f6f0c154a90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00ba50 tx=0x7f6efc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:01.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.721+0000 7f6f03e00640 1 -- 172.21.15.26:0/3142552679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc00cbe0 con 0x7f6f0c0cd530 2024-01-15T18:12:01.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.721+0000 7f6f03e00640 1 -- 172.21.15.26:0/3142552679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6efc00cd60 con 0x7f6f0c0cd530 2024-01-15T18:12:01.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.721+0000 7f6f03e00640 1 -- 172.21.15.26:0/3142552679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc014680 con 0x7f6f0c0cd530 2024-01-15T18:12:01.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.721+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c155a30 con 0x7f6f0c0cd530 2024-01-15T18:12:01.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.721+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c156770 con 0x7f6f0c0cd530 2024-01-15T18:12:01.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.722+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c155f50 con 0x7f6f0c0cd530 2024-01-15T18:12:01.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.723+0000 7f6f03e00640 1 -- 172.21.15.26:0/3142552679 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f6efc01d3b0 con 0x7f6f0c0cd530 2024-01-15T18:12:01.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.724+0000 7f6f03e00640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ee406c430 0x7f6ee406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:01.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.724+0000 7f6f03e00640 1 -- 172.21.15.26:0/3142552679 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f6efc088ad0 con 0x7f6f0c0cd530 2024-01-15T18:12:01.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.724+0000 7f6f02800640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ee406c430 0x7f6ee406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:01.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.725+0000 7f6f02800640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ee406c430 0x7f6ee406e8f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4036000 tx=0x7f6ef403d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:01.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.728+0000 7f6f03e00640 1 -- 172.21.15.26:0/3142552679 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f0c155f50 con 0x7f6f0c0cd530 2024-01-15T18:12:01.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.923+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f0c155f50 con 0x7f6f0c0cd530 2024-01-15T18:12:01.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.925+0000 7f6f03e00640 1 -- 172.21.15.26:0/3142552679 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+13104 (secure 0 0 0) 0x7f6f0c155f50 con 0x7f6f0c0cd530 2024-01-15T18:12:01.929 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:01.929 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":106,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:11:54.175615+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.124:6837/421656183":"2024-01-16T18:11:51.154279+0000","172.21.15.124:6836/421656183":"2024-01-16T18:11:51.154279+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:12:01.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ee406c430 msgr2=0x7f6ee406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:01.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ee406c430 0x7f6ee406e8f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4036000 tx=0x7f6ef403d040 comp rx=0 tx=0).stop 2024-01-15T18:12:01.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 msgr2=0x7f6f0c154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:01.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 0x7f6f0c154a90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00ba50 tx=0x7f6efc00bf10 comp rx=0 tx=0).stop 2024-01-15T18:12:01.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 shutdown_connections 2024-01-15T18:12:01.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f0c13cfa0 0x7f6f0c156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6ee406c430 0x7f6ee406e8f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f0c0cd9b0 0x7f6f0c0addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 --2- 172.21.15.26:0/3142552679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f0c0cd530 0x7f6f0c154a90 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 >> 172.21.15.26:0/3142552679 conn(0x7f6f0c06a800 msgr2=0x7f6f0c0aeb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 shutdown_connections 2024-01-15T18:12:01.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:01.928+0000 7f6f10a00640 1 -- 172.21.15.26:0/3142552679 wait complete. 2024-01-15T18:12:01.950 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:12:01.950 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:12:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.169+0000 7f87cb000640 1 Processor -- start 2024-01-15T18:12:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.170+0000 7f87cb000640 1 -- start start 2024-01-15T18:12:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87cb000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f87c40ccb50 0x7f87c4070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87cb000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c415bf10 0x7f87c4143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87cb000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c415f4e0 con 0x7f87c40adf20 2024-01-15T18:12:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87cb000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c415f970 con 0x7f87c40ccb50 2024-01-15T18:12:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87cb000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c415c3c0 con 0x7f87c415bf10 2024-01-15T18:12:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87c9c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87c40adf20 0x7f87c40cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42724/0 (socket says 172.21.15.26:42724) 2024-01-15T18:12:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87c9c00640 1 -- 172.21.15.26:0/2114654867 learned_addr learned my addr 172.21.15.26:0/2114654867 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87c9200640 1 --2- 172.21.15.26:0/2114654867 >> v2:172.21.15.26:3300/0 conn(0x7f87c40ccb50 0x7f87c4070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87ca600640 1 --2- 172.21.15.26:0/2114654867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c415bf10 0x7f87c4143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.172+0000 7f87c3e00640 1 -- 172.21.15.26:0/2114654867 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 293171119 0 0) 0x7f87c415f4e0 con 0x7f87c40adf20 2024-01-15T18:12:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.173+0000 7f87c3e00640 1 -- 172.21.15.26:0/2114654867 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a4003520 con 0x7f87c40adf20 2024-01-15T18:12:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.173+0000 7f87c9200640 1 -- 172.21.15.26:0/2114654867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c415bf10 msgr2=0x7f87c4143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.173+0000 7f87c9200640 1 --2- 172.21.15.26:0/2114654867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c415bf10 0x7f87c4143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.173+0000 7f87c9200640 1 -- 172.21.15.26:0/2114654867 >> v1:172.21.15.26:6789/0 conn(0x7f87c40adf20 legacy=0x7f87c40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.173+0000 7f87c9200640 1 -- 172.21.15.26:0/2114654867 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c4071130 con 0x7f87c40ccb50 2024-01-15T18:12:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.174+0000 7f87c9200640 1 --2- 172.21.15.26:0/2114654867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40ccb50 0x7f87c4070a30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0097a0 tx=0x7f87ac02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a977ea2f598c384 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.174+0000 7f87c3e00640 1 -- 172.21.15.26:0/2114654867 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac031780 con 0x7f87c40ccb50 2024-01-15T18:12:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.174+0000 7f87c3e00640 1 -- 172.21.15.26:0/2114654867 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac0318e0 con 0x7f87c40ccb50 2024-01-15T18:12:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.174+0000 7f87c3e00640 1 -- 172.21.15.26:0/2114654867 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac031c70 con 0x7f87c40ccb50 2024-01-15T18:12:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.175+0000 7f87cb000640 1 -- 172.21.15.26:0/2114654867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40ccb50 msgr2=0x7f87c4070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.175+0000 7f87cb000640 1 --2- 172.21.15.26:0/2114654867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40ccb50 0x7f87c4070a30 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0097a0 tx=0x7f87ac02b820 comp rx=0 tx=0).stop 2024-01-15T18:12:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.175+0000 7f87cb000640 1 -- 172.21.15.26:0/2114654867 shutdown_connections 2024-01-15T18:12:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.175+0000 7f87cb000640 1 --2- 172.21.15.26:0/2114654867 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c415bf10 0x7f87c4143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.175+0000 7f87cb000640 1 --2- 172.21.15.26:0/2114654867 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40ccb50 0x7f87c4070a30 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.175+0000 7f87cb000640 1 -- 172.21.15.26:0/2114654867 >> 172.21.15.26:0/2114654867 conn(0x7f87c406a7d0 msgr2=0x7f87c406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.175+0000 7f87cb000640 1 -- 172.21.15.26:0/2114654867 shutdown_connections 2024-01-15T18:12:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.175+0000 7f87cb000640 1 -- 172.21.15.26:0/2114654867 wait complete. 2024-01-15T18:12:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.177+0000 7f87cb000640 1 Processor -- start 2024-01-15T18:12:02.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.177+0000 7f87cb000640 1 -- start start 2024-01-15T18:12:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.178+0000 7f87cb000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40adf20 0x7f87c413d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.178+0000 7f87cb000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c40ccb50 0x7f87c413fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.178+0000 7f87cb000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c415bf10 0x7f87c4140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.178+0000 7f87cb000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c41f2a90 con 0x7f87c40adf20 2024-01-15T18:12:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.178+0000 7f87cb000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40b21c0 con 0x7f87c415bf10 2024-01-15T18:12:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.178+0000 7f87cb000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40b24a0 con 0x7f87c40ccb50 2024-01-15T18:12:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.178+0000 7f87c9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40adf20 0x7f87c413d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.178+0000 7f87c9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40adf20 0x7f87c413d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38750/0 (socket says 172.21.15.26:38750) 2024-01-15T18:12:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.178+0000 7f87c9c00640 1 -- 172.21.15.26:0/2116314159 learned_addr learned my addr 172.21.15.26:0/2116314159 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87c9c00640 1 -- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c40ccb50 msgr2=0x7f87c413fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87c9c00640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c40ccb50 0x7f87c413fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87c9c00640 1 -- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c415bf10 msgr2=0x7f87c4140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87c9c00640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c415bf10 0x7f87c4140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87c9c00640 1 -- 172.21.15.26:0/2116314159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac009400 con 0x7f87c40adf20 2024-01-15T18:12:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87c9c00640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40adf20 0x7f87c413d740 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c500 tx=0x7f87b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87c1600640 1 -- 172.21.15.26:0/2116314159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4005ce0 con 0x7f87c40adf20 2024-01-15T18:12:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87c1600640 1 -- 172.21.15.26:0/2116314159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4005e40 con 0x7f87c40adf20 2024-01-15T18:12:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87c1600640 1 -- 172.21.15.26:0/2116314159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4007330 con 0x7f87c40adf20 2024-01-15T18:12:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.179+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c40b2780 con 0x7f87c40adf20 2024-01-15T18:12:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.180+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c40b1e90 con 0x7f87c40adf20 2024-01-15T18:12:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.181+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c4067450 con 0x7f87c40adf20 2024-01-15T18:12:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.182+0000 7f87c1600640 1 -- 172.21.15.26:0/2116314159 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f87b4007490 con 0x7f87c40adf20 2024-01-15T18:12:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.182+0000 7f87c1600640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f879c06c430 0x7f879c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.183+0000 7f87c1600640 1 -- 172.21.15.26:0/2116314159 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f87b4088130 con 0x7f87c40adf20 2024-01-15T18:12:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.183+0000 7f87c9200640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f879c06c430 0x7f879c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:02.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.184+0000 7f87c9200640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f879c06c430 0x7f879c06e8f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f87ac036000 tx=0x7f87ac030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:02.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.187+0000 7f87c1600640 1 -- 172.21.15.26:0/2116314159 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f87b4054760 con 0x7f87c40adf20 2024-01-15T18:12:02.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.382+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87c4072370 con 0x7f87c40adf20 2024-01-15T18:12:02.385 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.383+0000 7f87c1600640 1 -- 172.21.15.26:0/2116314159 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87b4017070 con 0x7f87c40adf20 2024-01-15T18:12:02.386 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f879c06c430 msgr2=0x7f879c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f879c06c430 0x7f879c06e8f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f87ac036000 tx=0x7f87ac030000 comp rx=0 tx=0).stop 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40adf20 msgr2=0x7f87c413d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40adf20 0x7f87c413d740 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f87b400c500 tx=0x7f87b400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 shutdown_connections 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87c40ccb50 0x7f87c413fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f879c06c430 0x7f879c06e8f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87c415bf10 0x7f87c4140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 --2- 172.21.15.26:0/2116314159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87c40adf20 0x7f87c413d740 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.385+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 >> 172.21.15.26:0/2116314159 conn(0x7f87c406a7d0 msgr2=0x7f87c4143850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.386+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 shutdown_connections 2024-01-15T18:12:02.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.386+0000 7f87cb000640 1 -- 172.21.15.26:0/2116314159 wait complete. 2024-01-15T18:12:02.406 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_3_clones_are_cancelled' 2024-01-15T18:12:02.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.616+0000 7f4817e00640 1 Processor -- start 2024-01-15T18:12:02.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.617+0000 7f4817e00640 1 -- start start 2024-01-15T18:12:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.618+0000 7f4817e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f48180b3f30 0x7f48180d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.618+0000 7f4817e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180d0a10 con 0x7f4818160920 2024-01-15T18:12:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.618+0000 7f4817e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48180cf080 con 0x7f48180d0100 2024-01-15T18:12:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.618+0000 7f4817e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48180c6be0 con 0x7f48180b3f30 2024-01-15T18:12:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.618+0000 7f4816a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4818160920 0x7f48180cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42732/0 (socket says 172.21.15.26:42732) 2024-01-15T18:12:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.618+0000 7f4816a00640 1 -- 172.21.15.26:0/1312741687 learned_addr learned my addr 172.21.15.26:0/1312741687 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:02.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.618+0000 7f4817400640 1 --2- 172.21.15.26:0/1312741687 >> v2:172.21.15.26:3300/0 conn(0x7f48180b3f30 0x7f48180d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:02.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.619+0000 7f4815600640 1 -- 172.21.15.26:0/1312741687 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364270735 0 0) 0x7f48180d0a10 con 0x7f4818160920 2024-01-15T18:12:02.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.619+0000 7f4815600640 1 -- 172.21.15.26:0/1312741687 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47f8003520 con 0x7f4818160920 2024-01-15T18:12:02.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.619+0000 7f4817400640 1 -- 172.21.15.26:0/1312741687 >> v1:172.21.15.124:6789/0 conn(0x7f48180d0100 legacy=0x7f48180d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:02.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.619+0000 7f4817400640 1 -- 172.21.15.26:0/1312741687 >> v1:172.21.15.26:6789/0 conn(0x7f4818160920 legacy=0x7f48180cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.619+0000 7f4817400640 1 -- 172.21.15.26:0/1312741687 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48180ce7e0 con 0x7f48180b3f30 2024-01-15T18:12:02.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.620+0000 7f4817400640 1 --2- 172.21.15.26:0/1312741687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 0x7f48180d1650 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f48100097a0 tx=0x7f481002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9a409ba7a690134 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:02.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.620+0000 7f4815600640 1 -- 172.21.15.26:0/1312741687 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810031830 con 0x7f48180b3f30 2024-01-15T18:12:02.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.621+0000 7f4815600640 1 -- 172.21.15.26:0/1312741687 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4810031990 con 0x7f48180b3f30 2024-01-15T18:12:02.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.621+0000 7f4815600640 1 -- 172.21.15.26:0/1312741687 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810031d20 con 0x7f48180b3f30 2024-01-15T18:12:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.621+0000 7f4817e00640 1 -- 172.21.15.26:0/1312741687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 msgr2=0x7f48180d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.621+0000 7f4817e00640 1 --2- 172.21.15.26:0/1312741687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 0x7f48180d1650 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f48100097a0 tx=0x7f481002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.621+0000 7f4817e00640 1 -- 172.21.15.26:0/1312741687 shutdown_connections 2024-01-15T18:12:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.621+0000 7f4817e00640 1 --2- 172.21.15.26:0/1312741687 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 0x7f48180d1650 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.621+0000 7f4817e00640 1 -- 172.21.15.26:0/1312741687 >> 172.21.15.26:0/1312741687 conn(0x7f48180d3090 msgr2=0x7f48180d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.621+0000 7f4817e00640 1 -- 172.21.15.26:0/1312741687 shutdown_connections 2024-01-15T18:12:02.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.621+0000 7f4817e00640 1 -- 172.21.15.26:0/1312741687 wait complete. 2024-01-15T18:12:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.623+0000 7f4817e00640 1 Processor -- start 2024-01-15T18:12:02.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.623+0000 7f4817e00640 1 -- start start 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.624+0000 7f4817e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 0x7f48180cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.624+0000 7f4817e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48180d0100 0x7f481814d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.624+0000 7f4817e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4818160920 0x7f48180b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.624+0000 7f4817e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481814db40 con 0x7f48180b3f30 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.624+0000 7f4817e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48180cab30 con 0x7f4818160920 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.624+0000 7f4817e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48180b7980 con 0x7f48180d0100 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.625+0000 7f4816a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 0x7f48180cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.625+0000 7f4816a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 0x7f48180cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38768/0 (socket says 172.21.15.26:38768) 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.625+0000 7f4816a00640 1 -- 172.21.15.26:0/113447091 learned_addr learned my addr 172.21.15.26:0/113447091 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.625+0000 7f4816a00640 1 -- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48180d0100 msgr2=0x7f481814d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.625+0000 7f4816a00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48180d0100 0x7f481814d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.625+0000 7f4816a00640 1 -- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4818160920 msgr2=0x7f48180b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.625+0000 7f4816a00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4818160920 0x7f48180b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.625+0000 7f4816a00640 1 -- 172.21.15.26:0/113447091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4810009400 con 0x7f48180b3f30 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.625+0000 7f4816a00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 0x7f48180cb880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f480800c410 tx=0x7f480800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.626+0000 7f480ea00640 1 -- 172.21.15.26:0/113447091 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4808005ce0 con 0x7f48180b3f30 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.626+0000 7f480ea00640 1 -- 172.21.15.26:0/113447091 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4808005e40 con 0x7f48180b3f30 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.626+0000 7f480ea00640 1 -- 172.21.15.26:0/113447091 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48080072f0 con 0x7f48180b3f30 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.626+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48180cadc0 con 0x7f48180b3f30 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.626+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48180cb380 con 0x7f48180b3f30 2024-01-15T18:12:02.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.627+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4818067450 con 0x7f48180b3f30 2024-01-15T18:12:02.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.628+0000 7f480ea00640 1 -- 172.21.15.26:0/113447091 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f4808007510 con 0x7f48180b3f30 2024-01-15T18:12:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.629+0000 7f480ea00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47f006c430 0x7f47f006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.629+0000 7f480ea00640 1 -- 172.21.15.26:0/113447091 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f4808088170 con 0x7f48180b3f30 2024-01-15T18:12:02.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.629+0000 7f4816000640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47f006c430 0x7f47f006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:02.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.631+0000 7f4816000640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47f006c430 0x7f47f006e8f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f481815d780 tx=0x7f4800007d30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:02.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.633+0000 7f480ea00640 1 -- 172.21.15.26:0/113447091 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48080547a0 con 0x7f48180b3f30 2024-01-15T18:12:02.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:02.898+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_3_clones_are_cancelled"]} v 0) v1 -- 0x7f481808c7e0 con 0x7f48180b3f30 2024-01-15T18:12:03.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.496+0000 7f480ea00640 1 -- 172.21.15.26:0/113447091 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestCloneProgress.test_when_3_clones_are_cancelled"]}]=0 v762) v1 ==== 158+0+0 (secure 0 0 0) 0x7f4808017020 con 0x7f48180b3f30 2024-01-15T18:12:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47f006c430 msgr2=0x7f47f006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47f006c430 0x7f47f006e8f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f481815d780 tx=0x7f4800007d30 comp rx=0 tx=0).stop 2024-01-15T18:12:03.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 msgr2=0x7f48180cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 0x7f48180cb880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f480800c410 tx=0x7f480800c8d0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 shutdown_connections 2024-01-15T18:12:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f48180d0100 0x7f481814d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f47f006c430 0x7f47f006e8f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4818160920 0x7f48180b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 --2- 172.21.15.26:0/113447091 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f48180b3f30 0x7f48180cb880 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 >> 172.21.15.26:0/113447091 conn(0x7f48180d3090 msgr2=0x7f481806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 shutdown_connections 2024-01-15T18:12:03.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.500+0000 7f4817e00640 1 -- 172.21.15.26:0/113447091 wait complete. 2024-01-15T18:12:03.525 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:12:03.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.746+0000 7f34cd600640 1 Processor -- start 2024-01-15T18:12:03.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.747+0000 7f34cd600640 1 -- start start 2024-01-15T18:12:03.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.748+0000 7f34cd600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f34c81607a0 0x7f34c80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:03.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.748+0000 7f34cd600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34c8160b80 0x7f34c81617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:03.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.748+0000 7f34cd600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34c80b2b90 con 0x7f34c80b3eb0 2024-01-15T18:12:03.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34cd600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c80d21d0 con 0x7f34c8160b80 2024-01-15T18:12:03.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34cd600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34c80b2e30 con 0x7f34c81607a0 2024-01-15T18:12:03.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34ccc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34c8160b80 0x7f34c81617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:03.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34ccc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f34c8160b80 0x7f34c81617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38782/0 (socket says 172.21.15.26:38782) 2024-01-15T18:12:03.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34ccc00640 1 -- 172.21.15.26:0/3272891582 learned_addr learned my addr 172.21.15.26:0/3272891582 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:03.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34c7e00640 1 --2- 172.21.15.26:0/3272891582 >> v2:172.21.15.124:3300/0 conn(0x7f34c81607a0 0x7f34c80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:03.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34ccc00640 1 -- 172.21.15.26:0/3272891582 >> v2:172.21.15.124:3300/0 conn(0x7f34c81607a0 msgr2=0x7f34c80b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:03.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34ccc00640 1 --2- 172.21.15.26:0/3272891582 >> v2:172.21.15.124:3300/0 conn(0x7f34c81607a0 0x7f34c80b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34ccc00640 1 -- 172.21.15.26:0/3272891582 >> v1:172.21.15.124:6789/0 conn(0x7f34c80b3eb0 legacy=0x7f34c81610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:03.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34ccc00640 1 -- 172.21.15.26:0/3272891582 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c80b3110 con 0x7f34c8160b80 2024-01-15T18:12:03.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.749+0000 7f34c7e00640 1 --2- 172.21.15.26:0/3272891582 >> v2:172.21.15.124:3300/0 conn(0x7f34c81607a0 0x7f34c80b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:03.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.750+0000 7f34ccc00640 1 --2- 172.21.15.26:0/3272891582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 0x7f34c81617d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f34c808d7c0 tx=0x7f34c002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d5ce76d6775e2f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:03.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.751+0000 7f34c7400640 1 -- 172.21.15.26:0/3272891582 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c002ca80 con 0x7f34c8160b80 2024-01-15T18:12:03.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.751+0000 7f34c7400640 1 -- 172.21.15.26:0/3272891582 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c002cbe0 con 0x7f34c8160b80 2024-01-15T18:12:03.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.751+0000 7f34c7400640 1 -- 172.21.15.26:0/3272891582 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c00343f0 con 0x7f34c8160b80 2024-01-15T18:12:03.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.751+0000 7f34cd600640 1 -- 172.21.15.26:0/3272891582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 msgr2=0x7f34c81617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:03.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.751+0000 7f34cd600640 1 --2- 172.21.15.26:0/3272891582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 0x7f34c81617d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f34c808d7c0 tx=0x7f34c002bee0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.752+0000 7f34cd600640 1 -- 172.21.15.26:0/3272891582 shutdown_connections 2024-01-15T18:12:03.757 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.752+0000 7f34cd600640 1 --2- 172.21.15.26:0/3272891582 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 0x7f34c81617d0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.752+0000 7f34cd600640 1 --2- 172.21.15.26:0/3272891582 >> v2:172.21.15.124:3300/0 conn(0x7f34c81607a0 0x7f34c80b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.752+0000 7f34cd600640 1 -- 172.21.15.26:0/3272891582 >> 172.21.15.26:0/3272891582 conn(0x7f34c809c590 msgr2=0x7f34c80d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:03.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.752+0000 7f34cd600640 1 -- 172.21.15.26:0/3272891582 shutdown_connections 2024-01-15T18:12:03.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.752+0000 7f34cd600640 1 -- 172.21.15.26:0/3272891582 wait complete. 2024-01-15T18:12:03.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.754+0000 7f34cd600640 1 Processor -- start 2024-01-15T18:12:03.758 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.754+0000 7f34cd600640 1 -- start start 2024-01-15T18:12:03.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.755+0000 7f34cd600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34c80b3eb0 0x7f34c80b8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:03.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.755+0000 7f34cd600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34c81607a0 0x7f34c815d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:03.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.755+0000 7f34cd600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 0x7f34c80c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:03.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.755+0000 7f34cd600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80b66e0 con 0x7f34c8160b80 2024-01-15T18:12:03.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.755+0000 7f34cd600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80cc280 con 0x7f34c80b3eb0 2024-01-15T18:12:03.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.755+0000 7f34cd600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34c80cc560 con 0x7f34c81607a0 2024-01-15T18:12:03.761 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.755+0000 7f34ccc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 0x7f34c80c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:03.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.755+0000 7f34ccc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 0x7f34c80c6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38796/0 (socket says 172.21.15.26:38796) 2024-01-15T18:12:03.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.755+0000 7f34ccc00640 1 -- 172.21.15.26:0/2227552057 learned_addr learned my addr 172.21.15.26:0/2227552057 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:03.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34ccc00640 1 -- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34c81607a0 msgr2=0x7f34c815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:03.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34ccc00640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34c81607a0 0x7f34c815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34ccc00640 1 -- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34c80b3eb0 msgr2=0x7f34c80b8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:03.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34ccc00640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34c80b3eb0 0x7f34c80b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34ccc00640 1 -- 172.21.15.26:0/2227552057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c0009b30 con 0x7f34c8160b80 2024-01-15T18:12:03.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34ccc00640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 0x7f34c80c6d80 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f34c0036040 tx=0x7f34c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:03.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34c4c00640 1 -- 172.21.15.26:0/2227552057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c003e450 con 0x7f34c8160b80 2024-01-15T18:12:03.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34c4c00640 1 -- 172.21.15.26:0/2227552057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c003e5d0 con 0x7f34c8160b80 2024-01-15T18:12:03.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34c4c00640 1 -- 172.21.15.26:0/2227552057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c003e980 con 0x7f34c8160b80 2024-01-15T18:12:03.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.756+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c80cc840 con 0x7f34c8160b80 2024-01-15T18:12:03.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.757+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c80b8bd0 con 0x7f34c8160b80 2024-01-15T18:12:03.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.758+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c80b6850 con 0x7f34c8160b80 2024-01-15T18:12:03.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.759+0000 7f34c4c00640 1 -- 172.21.15.26:0/2227552057 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f34c002cd50 con 0x7f34c8160b80 2024-01-15T18:12:03.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.759+0000 7f34c4c00640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34a006c430 0x7f34a006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:03.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.759+0000 7f34c4c00640 1 -- 172.21.15.26:0/2227552057 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f34c00ab6c0 con 0x7f34c8160b80 2024-01-15T18:12:03.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.760+0000 7f34c7e00640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34a006c430 0x7f34a006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:03.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.761+0000 7f34c7e00640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34a006c430 0x7f34a006e8f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006fd0 tx=0x7f34b8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:03.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.764+0000 7f34c4c00640 1 -- 172.21.15.26:0/2227552057 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34c80b6850 con 0x7f34c8160b80 2024-01-15T18:12:03.955 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.951+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f34c80ccd60 con 0x7f34c8160b80 2024-01-15T18:12:03.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.959+0000 7f34c4c00640 1 -- 172.21.15.26:0/2227552057 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f34c0043020 con 0x7f34c8160b80 2024-01-15T18:12:03.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.961+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34a006c430 msgr2=0x7f34a006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:03.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.961+0000 7f34cd600640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34a006c430 0x7f34a006e8f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f34b8006fd0 tx=0x7f34b8008040 comp rx=0 tx=0).stop 2024-01-15T18:12:03.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.961+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 msgr2=0x7f34c80c6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.961+0000 7f34cd600640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 0x7f34c80c6d80 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f34c0036040 tx=0x7f34c0004a40 comp rx=0 tx=0).stop 2024-01-15T18:12:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.962+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 shutdown_connections 2024-01-15T18:12:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.962+0000 7f34cd600640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f34c81607a0 0x7f34c815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.962+0000 7f34cd600640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f34a006c430 0x7f34a006e8f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.962+0000 7f34cd600640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f34c8160b80 0x7f34c80c6d80 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.962+0000 7f34cd600640 1 --2- 172.21.15.26:0/2227552057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f34c80b3eb0 0x7f34c80b8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.962+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 >> 172.21.15.26:0/2227552057 conn(0x7f34c809c590 msgr2=0x7f34c80d3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.962+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 shutdown_connections 2024-01-15T18:12:03.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:03.962+0000 7f34cd600640 1 -- 172.21.15.26:0/2227552057 wait complete. 2024-01-15T18:12:03.983 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-01-15T18:12:03.983 DEBUG:teuthology.orchestra.run.smithi026:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:12:04.089 DEBUG:teuthology.orchestra.run.smithi124:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-01-15T18:12:04.209 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-01-15T18:12:04.209 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:12:04.209 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:12:04.270 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:12:04.270 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:12:04.270 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:12:04.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:12:04.328 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:12:04.328 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:12:04.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:12:04.387 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:12:04.387 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:12:04.443 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:12:04.443 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-01-15T18:12:04.443 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:12:04.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.671+0000 7fc81d600640 1 Processor -- start 2024-01-15T18:12:04.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.672+0000 7fc81d600640 1 -- start start 2024-01-15T18:12:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.673+0000 7fc81d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81813e360 0x7fc8180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.673+0000 7fc81d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc81815f9c0 0x7fc8180cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:04.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.673+0000 7fc81d600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81815c5f0 con 0x7fc81813df80 2024-01-15T18:12:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.673+0000 7fc81d600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81806c470 con 0x7fc81815f9c0 2024-01-15T18:12:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.673+0000 7fc81d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806c750 con 0x7fc81813e360 2024-01-15T18:12:04.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.674+0000 7fc817400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc81815f9c0 0x7fc8180cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.674+0000 7fc817400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc81815f9c0 0x7fc8180cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38806/0 (socket says 172.21.15.26:38806) 2024-01-15T18:12:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.674+0000 7fc817400640 1 -- 172.21.15.26:0/628982343 learned_addr learned my addr 172.21.15.26:0/628982343 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.674+0000 7fc816a00640 1 --1- 172.21.15.26:0/628982343 >> v1:172.21.15.124:6789/0 conn(0x7fc81813df80 0x7fc81815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:39654/0 (socket says 172.21.15.26:39654) 2024-01-15T18:12:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.674+0000 7fc816000640 1 --2- 172.21.15.26:0/628982343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81813e360 0x7fc8180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.675+0000 7fc817400640 1 -- 172.21.15.26:0/628982343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81813e360 msgr2=0x7fc8180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.675+0000 7fc817400640 1 --2- 172.21.15.26:0/628982343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81813e360 0x7fc8180cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.675+0000 7fc817400640 1 -- 172.21.15.26:0/628982343 >> v1:172.21.15.124:6789/0 conn(0x7fc81813df80 legacy=0x7fc81815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:04.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.675+0000 7fc817400640 1 -- 172.21.15.26:0/628982343 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81806ca30 con 0x7fc81815f9c0 2024-01-15T18:12:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.676+0000 7fc817400640 1 --2- 172.21.15.26:0/628982343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81815f9c0 0x7fc8180cf7d0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fc810009870 tx=0x7fc81002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b8e9913afabb9c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.676+0000 7fc815600640 1 -- 172.21.15.26:0/628982343 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810030840 con 0x7fc81815f9c0 2024-01-15T18:12:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.676+0000 7fc815600640 1 -- 172.21.15.26:0/628982343 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8100309a0 con 0x7fc81815f9c0 2024-01-15T18:12:04.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.676+0000 7fc815600640 1 -- 172.21.15.26:0/628982343 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810030d30 con 0x7fc81815f9c0 2024-01-15T18:12:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.677+0000 7fc81d600640 1 -- 172.21.15.26:0/628982343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81815f9c0 msgr2=0x7fc8180cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.677+0000 7fc81d600640 1 --2- 172.21.15.26:0/628982343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81815f9c0 0x7fc8180cf7d0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fc810009870 tx=0x7fc81002b980 comp rx=0 tx=0).stop 2024-01-15T18:12:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.677+0000 7fc81d600640 1 -- 172.21.15.26:0/628982343 shutdown_connections 2024-01-15T18:12:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.677+0000 7fc81d600640 1 --2- 172.21.15.26:0/628982343 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81815f9c0 0x7fc8180cf7d0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.677+0000 7fc81d600640 1 --2- 172.21.15.26:0/628982343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81813e360 0x7fc8180cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.677+0000 7fc81d600640 1 -- 172.21.15.26:0/628982343 >> 172.21.15.26:0/628982343 conn(0x7fc81806a800 msgr2=0x7fc81806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.677+0000 7fc81d600640 1 -- 172.21.15.26:0/628982343 shutdown_connections 2024-01-15T18:12:04.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.677+0000 7fc81d600640 1 -- 172.21.15.26:0/628982343 wait complete. 2024-01-15T18:12:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.679+0000 7fc81d600640 1 Processor -- start 2024-01-15T18:12:04.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.679+0000 7fc81d600640 1 -- start start 2024-01-15T18:12:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.680+0000 7fc81d600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc81813df80 0x7fc818153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.680+0000 7fc81d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81813e360 0x7fc81815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.680+0000 7fc81d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81815f9c0 0x7fc8180705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:04.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.680+0000 7fc81d600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818153790 con 0x7fc81813e360 2024-01-15T18:12:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.680+0000 7fc81d600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8181574f0 con 0x7fc81813df80 2024-01-15T18:12:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.680+0000 7fc81d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81815aa40 con 0x7fc81815f9c0 2024-01-15T18:12:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.680+0000 7fc816000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81813e360 0x7fc81815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.680+0000 7fc816000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81813e360 0x7fc81815a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38822/0 (socket says 172.21.15.26:38822) 2024-01-15T18:12:04.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.680+0000 7fc816000640 1 -- 172.21.15.26:0/2726520174 learned_addr learned my addr 172.21.15.26:0/2726520174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc817400640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81815f9c0 0x7fc8180705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc816000640 1 -- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81815f9c0 msgr2=0x7fc8180705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc816000640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81815f9c0 0x7fc8180705d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc816000640 1 -- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc81813df80 msgr2=0x7fc818153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc816a00640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc81813df80 0x7fc818153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc816000640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc81813df80 0x7fc818153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc816000640 1 -- 172.21.15.26:0/2726520174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8100094d0 con 0x7fc81813e360 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc817400640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81815f9c0 0x7fc8180705d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc816000640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81813e360 0x7fc81815a1f0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fc800007ef0 tx=0x7fc80000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.681+0000 7fc80ea00640 1 -- 172.21.15.26:0/2726520174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800014620 con 0x7fc81813e360 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.682+0000 7fc80ea00640 1 -- 172.21.15.26:0/2726520174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc800014780 con 0x7fc81813e360 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.682+0000 7fc80ea00640 1 -- 172.21.15.26:0/2726520174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc800014b30 con 0x7fc81813e360 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.682+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc818156bc0 con 0x7fc81813e360 2024-01-15T18:12:04.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.682+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc818157180 con 0x7fc81813e360 2024-01-15T18:12:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.683+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8181576a0 con 0x7fc81813e360 2024-01-15T18:12:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.684+0000 7fc80ea00640 1 -- 172.21.15.26:0/2726520174 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fc800014c90 con 0x7fc81813e360 2024-01-15T18:12:04.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.685+0000 7fc80ea00640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc7f006c460 0x7fc7f006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.685+0000 7fc80ea00640 1 -- 172.21.15.26:0/2726520174 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7fc800089400 con 0x7fc81813e360 2024-01-15T18:12:04.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.685+0000 7fc816a00640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc7f006c460 0x7fc7f006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:04.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.686+0000 7fc816a00640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc7f006c460 0x7fc7f006e920 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc808000f60 tx=0x7fc808009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:04.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.692+0000 7fc80ea00640 1 -- 172.21.15.26:0/2726520174 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8181576a0 con 0x7fc81813e360 2024-01-15T18:12:04.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.928+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc818157660 con 0x7fc81813e360 2024-01-15T18:12:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.929+0000 7fc80ea00640 1 -- 172.21.15.26:0/2726520174 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3700 (secure 0 0 0) 0x7fc8000559d0 con 0x7fc81813e360 2024-01-15T18:12:04.932 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:04.932 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7015,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/1631584304","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":1631584304},{"type":"v1","addr":"172.21.15.124:6835","nonce":1631584304}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8742,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2396377458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2396377458},{"type":"v1","addr":"172.21.15.26:6833","nonce":2396377458}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8748,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6835/81533892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":81533892},{"type":"v1","addr":"172.21.15.26:6835","nonce":81533892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":36419,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/3961747278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":3961747278},{"type":"v1","addr":"172.21.15.124:6837","nonce":3961747278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66}],"filesystems":[]} 2024-01-15T18:12:04.932 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 66 2024-01-15T18:12:04.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc7f006c460 msgr2=0x7fc7f006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:04.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc7f006c460 0x7fc7f006e920 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc808000f60 tx=0x7fc808009040 comp rx=0 tx=0).stop 2024-01-15T18:12:04.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81813e360 msgr2=0x7fc81815a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:04.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81813e360 0x7fc81815a1f0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fc800007ef0 tx=0x7fc80000e4b0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 shutdown_connections 2024-01-15T18:12:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc81815f9c0 0x7fc8180705d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc7f006c460 0x7fc7f006e920 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc81813e360 0x7fc81815a1f0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 --2- 172.21.15.26:0/2726520174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc81813df80 0x7fc818153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 >> 172.21.15.26:0/2726520174 conn(0x7fc81806a800 msgr2=0x7fc81815cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 shutdown_connections 2024-01-15T18:12:04.936 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:04.932+0000 7fc81d600640 1 -- 172.21.15.26:0/2726520174 wait complete. 2024-01-15T18:12:04.954 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-15T18:12:04.954 INFO:tasks.ceph.mds.a:Stopping old one... 2024-01-15T18:12:04.954 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:12:04.954 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:12:04.955 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-15T18:12:04.955 INFO:tasks.ceph.mds.c:Stopping old one... 2024-01-15T18:12:04.955 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:12:04.955 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:12:04.955 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-15T18:12:04.955 INFO:tasks.ceph.mds.b:Stopping old one... 2024-01-15T18:12:04.955 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:12:04.956 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:12:04.956 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-15T18:12:04.956 INFO:tasks.ceph.mds.d:Stopping old one... 2024-01-15T18:12:04.956 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:12:04.956 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:12:04.974 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:12:04.974 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-01-15T18:12:04.976 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:12:04.976 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-01-15T18:12:04.983 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:12:04.984 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-01-15T18:12:04.986 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:12:04.986 DEBUG:teuthology.orchestra.run.smithi124:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-01-15T18:12:05.023 INFO:tasks.ceph.mds.a:Started 2024-01-15T18:12:05.024 DEBUG:teuthology.parallel:result is None 2024-01-15T18:12:05.030 INFO:tasks.ceph.mds.c:Started 2024-01-15T18:12:05.030 DEBUG:teuthology.parallel:result is None 2024-01-15T18:12:05.032 INFO:tasks.ceph.mds.d:Started 2024-01-15T18:12:05.033 DEBUG:teuthology.parallel:result is None 2024-01-15T18:12:05.039 INFO:tasks.ceph.mds.b:Started 2024-01-15T18:12:05.039 DEBUG:teuthology.parallel:result is None 2024-01-15T18:12:05.040 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-01-15T18:12:05.188 INFO:tasks.ceph.mds.c.smithi026.stdout:starting mds.c at 2024-01-15T18:12:05.190 INFO:tasks.ceph.mds.a.smithi026.stdout:starting mds.a at 2024-01-15T18:12:05.198 INFO:tasks.ceph.mds.d.smithi124.stdout:starting mds.d at 2024-01-15T18:12:05.209 INFO:tasks.ceph.mds.b.smithi124.stdout:starting mds.b at 2024-01-15T18:12:05.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.232+0000 7f0b31200640 1 Processor -- start 2024-01-15T18:12:05.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.232+0000 7f0b31200640 1 -- start start 2024-01-15T18:12:05.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.233+0000 7f0b31200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0b2c13e360 0x7f0b2c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:05.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.233+0000 7f0b31200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c15c5f0 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.233+0000 7f0b31200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c06c470 con 0x7f0b2c13df80 2024-01-15T18:12:05.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.233+0000 7f0b31200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2c06c750 con 0x7f0b2c13e360 2024-01-15T18:12:05.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.233+0000 7f0b2a800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f0b2c15f9c0 0x7f0b2c0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42764/0 (socket says 172.21.15.26:42764) 2024-01-15T18:12:05.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.233+0000 7f0b2a800640 1 -- 172.21.15.26:0/1801481860 learned_addr learned my addr 172.21.15.26:0/1801481860 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:05.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.233+0000 7f0b29e00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f0b2c13df80 0x7f0b2c15bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:39684/0 (socket says 172.21.15.26:39684) 2024-01-15T18:12:05.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.234+0000 7f0b21200640 1 --2- 172.21.15.26:0/1801481860 >> v2:172.21.15.124:3300/0 conn(0x7f0b2c13e360 0x7f0b2c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.234+0000 7f0b29400640 1 -- 172.21.15.26:0/1801481860 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696894488 0 0) 0x7f0b2c15c5f0 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.234+0000 7f0b29400640 1 -- 172.21.15.26:0/1801481860 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b0c003520 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.234+0000 7f0b21200640 1 -- 172.21.15.26:0/1801481860 >> v1:172.21.15.124:6789/0 conn(0x7f0b2c13df80 legacy=0x7f0b2c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.234+0000 7f0b21200640 1 -- 172.21.15.26:0/1801481860 >> v1:172.21.15.26:6789/0 conn(0x7f0b2c15f9c0 legacy=0x7f0b2c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:05.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.234+0000 7f0b21200640 1 -- 172.21.15.26:0/1801481860 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c06ca30 con 0x7f0b2c13e360 2024-01-15T18:12:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.235+0000 7f0b21200640 1 --2- 172.21.15.26:0/1801481860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b2c13e360 0x7f0b2c0cf290 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0b140097a0 tx=0x7f0b1402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5094a8fc5d7367be server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.236+0000 7f0b29400640 1 -- 172.21.15.26:0/1801481860 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14030610 con 0x7f0b2c13e360 2024-01-15T18:12:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.236+0000 7f0b29400640 1 -- 172.21.15.26:0/1801481860 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b14030770 con 0x7f0b2c13e360 2024-01-15T18:12:05.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.236+0000 7f0b29400640 1 -- 172.21.15.26:0/1801481860 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14030b00 con 0x7f0b2c13e360 2024-01-15T18:12:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.236+0000 7f0b31200640 1 -- 172.21.15.26:0/1801481860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b2c13e360 msgr2=0x7f0b2c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.236+0000 7f0b31200640 1 --2- 172.21.15.26:0/1801481860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b2c13e360 0x7f0b2c0cf290 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0b140097a0 tx=0x7f0b1402b960 comp rx=0 tx=0).stop 2024-01-15T18:12:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.237+0000 7f0b31200640 1 -- 172.21.15.26:0/1801481860 shutdown_connections 2024-01-15T18:12:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.237+0000 7f0b31200640 1 --2- 172.21.15.26:0/1801481860 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b2c13e360 0x7f0b2c0cf290 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.237+0000 7f0b31200640 1 -- 172.21.15.26:0/1801481860 >> 172.21.15.26:0/1801481860 conn(0x7f0b2c06a800 msgr2=0x7f0b2c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.237+0000 7f0b31200640 1 -- 172.21.15.26:0/1801481860 shutdown_connections 2024-01-15T18:12:05.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.237+0000 7f0b31200640 1 -- 172.21.15.26:0/1801481860 wait complete. 2024-01-15T18:12:05.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.238+0000 7f0b31200640 1 Processor -- start 2024-01-15T18:12:05.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.239+0000 7f0b31200640 1 -- start start 2024-01-15T18:12:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b31200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b2c13df80 0x7f0b2c14b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b31200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b2c13e360 0x7f0b2c0715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b31200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b2c15f9c0 0x7f0b2c1f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b31200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c14c970 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b31200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c1f64d0 con 0x7f0b2c13df80 2024-01-15T18:12:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b31200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c14bd60 con 0x7f0b2c13e360 2024-01-15T18:12:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b2a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b2c15f9c0 0x7f0b2c1f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b2a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b2c15f9c0 0x7f0b2c1f5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38868/0 (socket says 172.21.15.26:38868) 2024-01-15T18:12:05.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b2a800640 1 -- 172.21.15.26:0/1498416421 learned_addr learned my addr 172.21.15.26:0/1498416421 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b29e00640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b2c13df80 0x7f0b2c14b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:05.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b21200640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b2c13e360 0x7f0b2c0715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b2a800640 1 -- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b2c13e360 msgr2=0x7f0b2c0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b2a800640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b2c13e360 0x7f0b2c0715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b2a800640 1 -- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b2c13df80 msgr2=0x7f0b2c14b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b2a800640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b2c13df80 0x7f0b2c14b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.240+0000 7f0b2a800640 1 -- 172.21.15.26:0/1498416421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b14009400 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.241+0000 7f0b2a800640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b2c15f9c0 0x7f0b2c1f5f90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400cbc0 tx=0x7f0b2400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.241+0000 7f0b22a00640 1 -- 172.21.15.26:0/1498416421 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2400aca0 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.241+0000 7f0b22a00640 1 -- 172.21.15.26:0/1498416421 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2400ae00 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.241+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2c14c040 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.241+0000 7f0b22a00640 1 -- 172.21.15.26:0/1498416421 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24012620 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.241+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2c14c600 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.242+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2c14e000 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.243+0000 7f0b22a00640 1 -- 172.21.15.26:0/1498416421 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f0b24007070 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.244+0000 7f0b22a00640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0b0406c460 0x7f0b0406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:05.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.244+0000 7f0b22a00640 1 -- 172.21.15.26:0/1498416421 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5027+0+0 (secure 0 0 0) 0x7f0b240889b0 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.245+0000 7f0b29e00640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0b0406c460 0x7f0b0406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:05.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.246+0000 7f0b29e00640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0b0406c460 0x7f0b0406e920 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000f60 tx=0x7f0b1c000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:05.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.249+0000 7f0b22a00640 1 -- 172.21.15.26:0/1498416421 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b24055010 con 0x7f0b2c15f9c0 2024-01-15T18:12:05.460 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:05.457+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0b2c0675d0 con 0x7f0b2c15f9c0 2024-01-15T18:12:06.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.285+0000 7f0b22a00640 1 -- 172.21.15.26:0/1498416421 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0b24017070 con 0x7f0b2c15f9c0 2024-01-15T18:12:06.289 INFO:teuthology.orchestra.run.smithi026.stderr: removed all blocklist entries 2024-01-15T18:12:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0b0406c460 msgr2=0x7f0b0406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0b0406c460 0x7f0b0406e920 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c000f60 tx=0x7f0b1c000ed0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b2c15f9c0 msgr2=0x7f0b2c1f5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b2c15f9c0 0x7f0b2c1f5f90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400cbc0 tx=0x7f0b2400a060 comp rx=0 tx=0).stop 2024-01-15T18:12:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 shutdown_connections 2024-01-15T18:12:06.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0b2c13e360 0x7f0b2c0715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0b0406c460 0x7f0b0406e920 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0b2c15f9c0 0x7f0b2c1f5f90 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 --2- 172.21.15.26:0/1498416421 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0b2c13df80 0x7f0b2c14b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 >> 172.21.15.26:0/1498416421 conn(0x7f0b2c06a800 msgr2=0x7f0b2c15c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 shutdown_connections 2024-01-15T18:12:06.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.289+0000 7f0b31200640 1 -- 172.21.15.26:0/1498416421 wait complete. 2024-01-15T18:12:06.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-01-15T18:12:06.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.535+0000 7fd0cfe00640 1 Processor -- start 2024-01-15T18:12:06.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.536+0000 7fd0cfe00640 1 -- start start 2024-01-15T18:12:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.537+0000 7fd0cfe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d0153c10 0x7fd0d00b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.537+0000 7fd0cfe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd0d00b3e40 0x7fd0d006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.537+0000 7fd0cfe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d006d680 con 0x7fd0d006d240 2024-01-15T18:12:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.537+0000 7fd0cfe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d0159bf0 con 0x7fd0d00b3e40 2024-01-15T18:12:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.537+0000 7fd0cfe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d015aba0 con 0x7fd0d0153c10 2024-01-15T18:12:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.538+0000 7fd0cf400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd0d006d240 0x7fd0d00700f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42778/0 (socket says 172.21.15.26:42778) 2024-01-15T18:12:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.538+0000 7fd0cf400640 1 -- 172.21.15.26:0/2210918491 learned_addr learned my addr 172.21.15.26:0/2210918491 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:06.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.538+0000 7fd0cea00640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d0153c10 0x7fd0d00b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:06.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.538+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852805132 0 0) 0x7fd0d006d680 con 0x7fd0d006d240 2024-01-15T18:12:06.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.538+0000 7fd0ce000640 1 --2- 172.21.15.26:0/2210918491 >> v2:172.21.15.124:3300/0 conn(0x7fd0d00b3e40 0x7fd0d006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.538+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b0003520 con 0x7fd0d006d240 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.538+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3678500922 0 0) 0x7fd0b0003520 con 0x7fd0d006d240 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d0153c10 msgr2=0x7fd0d00b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d0153c10 0x7fd0d00b3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 >> v2:172.21.15.124:3300/0 conn(0x7fd0d00b3e40 msgr2=0x7fd0d006cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 --2- 172.21.15.26:0/2210918491 >> v2:172.21.15.124:3300/0 conn(0x7fd0d00b3e40 0x7fd0d006cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0159de0 con 0x7fd0d006d240 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd0c8001670 con 0x7fd0d006d240 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 >> v1:172.21.15.26:6789/0 conn(0x7fd0d006d240 legacy=0x7fd0d00700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d0153c10 0x7fd0b0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b0003f40 0x7fd0b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b0004d00 0x7fd0b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:06.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d015aba0 con 0x7fd0b0004d00 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b000b9b0 con 0x7fd0b0003f40 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b000bb70 con 0x7fd0d0153c10 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cf400640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b0004d00 0x7fd0b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0cea00640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d0153c10 0x7fd0b0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.539+0000 7fd0ce000640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b0003f40 0x7fd0b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.540+0000 7fd0cf400640 1 -- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d0153c10 msgr2=0x7fd0b0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.540+0000 7fd0cf400640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d0153c10 0x7fd0b0003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.540+0000 7fd0cf400640 1 -- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b0003f40 msgr2=0x7fd0b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.540+0000 7fd0cf400640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b0003f40 0x7fd0b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.540+0000 7fd0cf400640 1 -- 172.21.15.26:0/2210918491 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b0003520 con 0x7fd0b0004d00 2024-01-15T18:12:06.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.541+0000 7fd0cf400640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b0004d00 0x7fd0b0005180 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c110 tx=0x7fd0b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366d25b295d7f839 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:06.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.541+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8005ad0 con 0x7fd0b0004d00 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.541+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c8005c50 con 0x7fd0b0004d00 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.541+0000 7fd0cd600640 1 -- 172.21.15.26:0/2210918491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c802c830 con 0x7fd0b0004d00 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.541+0000 7fd0cfe00640 1 -- 172.21.15.26:0/2210918491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b0004d00 msgr2=0x7fd0b0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.541+0000 7fd0cfe00640 1 --2- 172.21.15.26:0/2210918491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0b0004d00 0x7fd0b0005180 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd0c800c110 tx=0x7fd0b00057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.541+0000 7fd0cf400640 1 -- 172.21.15.26:0/2210918491 reap_dead start 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.542+0000 7fd0cfe00640 1 -- 172.21.15.26:0/2210918491 shutdown_connections 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.542+0000 7fd0cfe00640 1 -- 172.21.15.26:0/2210918491 >> 172.21.15.26:0/2210918491 conn(0x7fd0d006a3e0 msgr2=0x7fd0d006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.542+0000 7fd0cfe00640 1 -- 172.21.15.26:0/2210918491 shutdown_connections 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.542+0000 7fd0cfe00640 1 -- 172.21.15.26:0/2210918491 wait complete. 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.544+0000 7fd0cfe00640 1 Processor -- start 2024-01-15T18:12:06.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.544+0000 7fd0cfe00640 1 -- start start 2024-01-15T18:12:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cfe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b0004d00 0x7fd0d013e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cfe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d013fb10 0x7fd0d0145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d0146440 0x7fd0d012ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cfe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d012de00 con 0x7fd0d0146440 2024-01-15T18:12:06.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cfe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d01468c0 con 0x7fd0b0004d00 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cfe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d013ece0 con 0x7fd0d013fb10 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d0146440 0x7fd0d012ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cf400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d0146440 0x7fd0d012ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38884/0 (socket says 172.21.15.26:38884) 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cf400640 1 -- 172.21.15.26:0/1855815118 learned_addr learned my addr 172.21.15.26:0/1855815118 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0ce000640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d013fb10 0x7fd0d0145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cea00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b0004d00 0x7fd0d013e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cf400640 1 -- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d013fb10 msgr2=0x7fd0d0145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cf400640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d013fb10 0x7fd0d0145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cf400640 1 -- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b0004d00 msgr2=0x7fd0d013e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cf400640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b0004d00 0x7fd0d013e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.545+0000 7fd0cf400640 1 -- 172.21.15.26:0/1855815118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d012db20 con 0x7fd0d0146440 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.546+0000 7fd0cf400640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d0146440 0x7fd0d012ce10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd0c802c4a0 tx=0x7fd0c80019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.546+0000 7fd0c6a00640 1 -- 172.21.15.26:0/1855815118 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c803c040 con 0x7fd0d0146440 2024-01-15T18:12:06.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.546+0000 7fd0c6a00640 1 -- 172.21.15.26:0/1855815118 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c802cc40 con 0x7fd0d0146440 2024-01-15T18:12:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.546+0000 7fd0c6a00640 1 -- 172.21.15.26:0/1855815118 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8007070 con 0x7fd0d0146440 2024-01-15T18:12:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.546+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d013f030 con 0x7fd0d0146440 2024-01-15T18:12:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.546+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d013f5f0 con 0x7fd0d0146440 2024-01-15T18:12:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.547+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d013ee90 con 0x7fd0d0146440 2024-01-15T18:12:06.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.548+0000 7fd0c6a00640 1 -- 172.21.15.26:0/1855815118 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fd0c8032070 con 0x7fd0d0146440 2024-01-15T18:12:06.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.549+0000 7fd0c6a00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd0a806c430 0x7fd0a806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.549+0000 7fd0c6a00640 1 -- 172.21.15.26:0/1855815118 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fd0c80a9260 con 0x7fd0d0146440 2024-01-15T18:12:06.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.550+0000 7fd0cea00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd0a806c430 0x7fd0a806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:06.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.551+0000 7fd0cea00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd0a806c430 0x7fd0a806e8f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0007380 tx=0x7fd0c000b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:06.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.556+0000 7fd0c6a00640 1 -- 172.21.15.26:0/1855815118 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0d013ee90 con 0x7fd0d0146440 2024-01-15T18:12:06.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.818+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd0d013ee90 con 0x7fd0d0146440 2024-01-15T18:12:06.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.819+0000 7fd0c6a00640 1 -- 172.21.15.26:0/1855815118 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd0d013ee90 con 0x7fd0d0146440 2024-01-15T18:12:06.822 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:06.822 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:12:06.822 INFO:teuthology.orchestra.run.smithi026.stdout: "auth_dump": [ 2024-01-15T18:12:06.822 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.a", 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVl3erNJhAANxPK+39yVM5neq9iKC1HkA==", 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.b", 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlLaUlDRAAeKaen5hLlwDDWY6OAMRsPQ==", 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.823 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.c", 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlaIAVOBAAA8LEFM1HtzRtNooQCCjS/w==", 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:12:06.824 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mds.d", 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlmpTBHRAAUwbZeVpysMsn56QRLRJCoA==", 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.825 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.0", 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA7caVlCTmgDhAAPWb0qEh134xxyF8lM9DoNg==", 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.1", 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA+caVlaxF5JxAA23Z9j21ATy3GVlhgndDIxw==", 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.826 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.2", 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBCcaVlcRtjBRAAOFN9Xf5kN5MXL6yQfRI27g==", 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.827 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.3", 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBFcaVlvAzEHhAACDoNh7BZY9gbKTdGwIrhlg==", 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.4", 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBMcaVl8yiOMBAAdI4NTeSB1d/nSHRaIDMcFw==", 2024-01-15T18:12:06.828 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.5", 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBQcaVltf8sARAAfPaJAfoTgQoEivw90jRz/Q==", 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.829 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.6", 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBTcaVltuKiGhAAT0cIJiRkQ38QXT/h6Gyggg==", 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "osd.7", 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQBWcaVlbJqVMhAAKjpl6pAHbytjZWMYeRbRjA==", 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.830 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow profile osd", 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile osd", 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.0", 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA==", 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow r", 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.831 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.1", 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVlrVpqBxAAxOL0iuAlOvfR+KgSsin3vQ==", 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.2", 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA4caVlO8vdNxAAlUE/TFCDQoXuxvwLPwUvtg==", 2024-01-15T18:12:06.832 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.3", 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA5caVl5qgCEhAALgt7d3OjEmq5bc4nAD2rTw==", 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow", 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow r", 2024-01-15T18:12:06.833 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow rw", 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow rwx" 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "client.admin", 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA2caVlyg4eHBAAwTCXVk8hO9+zcZ1tUA3gfA==", 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: "mgr": "allow *", 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow *", 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.834 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.x", 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVlvuBcGhAAGqdCMSffJwnMKlFh3CtyCw==", 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: "entity": "mgr.y", 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: "key": "AQA3caVle8Z+DhAA9cWxzc4puvrHAedqupPG3Q==", 2024-01-15T18:12:06.835 INFO:teuthology.orchestra.run.smithi026.stdout: "caps": { 2024-01-15T18:12:06.836 INFO:teuthology.orchestra.run.smithi026.stdout: "mds": "allow *", 2024-01-15T18:12:06.836 INFO:teuthology.orchestra.run.smithi026.stdout: "mon": "allow profile mgr", 2024-01-15T18:12:06.836 INFO:teuthology.orchestra.run.smithi026.stdout: "osd": "allow *" 2024-01-15T18:12:06.836 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.836 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:06.836 INFO:teuthology.orchestra.run.smithi026.stdout: ] 2024-01-15T18:12:06.836 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:12:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd0a806c430 msgr2=0x7fd0a806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd0a806c430 0x7fd0a806e8f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd0c0007380 tx=0x7fd0c000b040 comp rx=0 tx=0).stop 2024-01-15T18:12:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d0146440 msgr2=0x7fd0d012ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d0146440 0x7fd0d012ce10 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd0c802c4a0 tx=0x7fd0c80019a0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 shutdown_connections 2024-01-15T18:12:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0d013fb10 0x7fd0d0145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd0a806c430 0x7fd0a806e8f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0d0146440 0x7fd0d012ce10 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 --2- 172.21.15.26:0/1855815118 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd0b0004d00 0x7fd0d013e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:06.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 >> 172.21.15.26:0/1855815118 conn(0x7fd0d006a3e0 msgr2=0x7fd0d006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:06.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 shutdown_connections 2024-01-15T18:12:06.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:06.822+0000 7fd0cfe00640 1 -- 172.21.15.26:0/1855815118 wait complete. 2024-01-15T18:12:06.846 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:12:07.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.060+0000 7fb12c000640 1 Processor -- start 2024-01-15T18:12:07.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.061+0000 7fb12c000640 1 -- start start 2024-01-15T18:12:07.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.062+0000 7fb12c000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb12413df80 0x7fb12415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.062+0000 7fb12c000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1240cf9a0 0x7fb1240cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.062+0000 7fb12c000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12406c470 con 0x7fb12413e360 2024-01-15T18:12:07.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.062+0000 7fb12c000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12406c750 con 0x7fb12413df80 2024-01-15T18:12:07.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.062+0000 7fb12c000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12406ca30 con 0x7fb1240cf9a0 2024-01-15T18:12:07.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.062+0000 7fb128c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb12413e360 0x7fb1240cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42794/0 (socket says 172.21.15.26:42794) 2024-01-15T18:12:07.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.062+0000 7fb128c00640 1 -- 172.21.15.26:0/305611116 learned_addr learned my addr 172.21.15.26:0/305611116 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb12a000640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1240cf9a0 0x7fb1240cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856201892 0 0) 0x7fb12406c470 con 0x7fb12413e360 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb129600640 1 --2- 172.21.15.26:0/305611116 >> v2:172.21.15.124:3300/0 conn(0x7fb12413df80 0x7fb12415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb104003520 con 0x7fb12413e360 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2708577944 0 0) 0x7fb104003520 con 0x7fb12413e360 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1240cf9a0 msgr2=0x7fb1240cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1240cf9a0 0x7fb1240cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 >> v2:172.21.15.124:3300/0 conn(0x7fb12413df80 msgr2=0x7fb12415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 --2- 172.21.15.26:0/305611116 >> v2:172.21.15.124:3300/0 conn(0x7fb12413df80 0x7fb12415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12406cd10 con 0x7fb12413e360 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb10c001670 con 0x7fb12413e360 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 >> v1:172.21.15.26:6789/0 conn(0x7fb12413e360 legacy=0x7fb1240cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb104003970 0x7fb104003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1040047c0 0x7fb104004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1240cf9a0 0x7fb104005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12406c750 con 0x7fb1040047c0 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12406ca30 con 0x7fb104003970 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10400ba00 con 0x7fb1240cf9a0 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.063+0000 7fb129600640 1 --2- 172.21.15.26:0/305611116 >> v2:172.21.15.124:3300/0 conn(0x7fb12413df80 0x7fb12415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.064+0000 7fb128c00640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1040047c0 0x7fb104004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.064+0000 7fb12a000640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1240cf9a0 0x7fb104005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.064+0000 7fb129600640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb104003970 0x7fb104003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.064+0000 7fb128c00640 1 -- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1240cf9a0 msgr2=0x7fb104005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.064+0000 7fb128c00640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1240cf9a0 0x7fb104005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.064+0000 7fb128c00640 1 -- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb104003970 msgr2=0x7fb104003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.064+0000 7fb128c00640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb104003970 0x7fb104003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.064+0000 7fb128c00640 1 -- 172.21.15.26:0/305611116 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104003520 con 0x7fb1040047c0 2024-01-15T18:12:07.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.064+0000 7fb12a000640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb1240cf9a0 0x7fb104005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.065+0000 7fb128c00640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1040047c0 0x7fb104004c40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00c0e0 tx=0x7fb1040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716088c7abe75cb5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c005ad0 con 0x7fb1040047c0 2024-01-15T18:12:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c005c50 con 0x7fb1040047c0 2024-01-15T18:12:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb123e00640 1 -- 172.21.15.26:0/305611116 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c02c810 con 0x7fb1040047c0 2024-01-15T18:12:07.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb12c000640 1 -- 172.21.15.26:0/305611116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1040047c0 msgr2=0x7fb104004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb12c000640 1 --2- 172.21.15.26:0/305611116 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1040047c0 0x7fb104004c40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00c0e0 tx=0x7fb1040057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb12a000640 1 -- 172.21.15.26:0/305611116 reap_dead start 2024-01-15T18:12:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb12c000640 1 -- 172.21.15.26:0/305611116 shutdown_connections 2024-01-15T18:12:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb12c000640 1 -- 172.21.15.26:0/305611116 >> 172.21.15.26:0/305611116 conn(0x7fb12406a800 msgr2=0x7fb12406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb12c000640 1 -- 172.21.15.26:0/305611116 shutdown_connections 2024-01-15T18:12:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.066+0000 7fb12c000640 1 -- 172.21.15.26:0/305611116 wait complete. 2024-01-15T18:12:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.068+0000 7fb12c000640 1 Processor -- start 2024-01-15T18:12:07.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.068+0000 7fb12c000640 1 -- start start 2024-01-15T18:12:07.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.069+0000 7fb12c000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1040047c0 0x7fb12413d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.069+0000 7fb12c000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240702c0 0x7fb1240afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.069+0000 7fb12c000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb124070720 0x7fb1240b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.069+0000 7fb12c000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb124153ad0 con 0x7fb1240702c0 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.069+0000 7fb12c000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb124152e50 con 0x7fb1040047c0 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.069+0000 7fb12c000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb124153130 con 0x7fb124070720 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.069+0000 7fb128c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240702c0 0x7fb1240afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb128c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240702c0 0x7fb1240afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38896/0 (socket says 172.21.15.26:38896) 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb128c00640 1 -- 172.21.15.26:0/34270314 learned_addr learned my addr 172.21.15.26:0/34270314 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb12a000640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb124070720 0x7fb1240b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb129600640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1040047c0 0x7fb12413d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb128c00640 1 -- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb124070720 msgr2=0x7fb1240b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb128c00640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb124070720 0x7fb1240b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb128c00640 1 -- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1040047c0 msgr2=0x7fb12413d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb128c00640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1040047c0 0x7fb12413d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb128c00640 1 -- 172.21.15.26:0/34270314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124153410 con 0x7fb1240702c0 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb129600640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1040047c0 0x7fb12413d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.070+0000 7fb128c00640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240702c0 0x7fb1240afbe0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb10c02c3b0 tx=0x7fb10c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.071+0000 7fb121600640 1 -- 172.21.15.26:0/34270314 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c039070 con 0x7fb1240702c0 2024-01-15T18:12:07.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.071+0000 7fb121600640 1 -- 172.21.15.26:0/34270314 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c02cc20 con 0x7fb1240702c0 2024-01-15T18:12:07.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.071+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1241536f0 con 0x7fb1240702c0 2024-01-15T18:12:07.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.071+0000 7fb121600640 1 -- 172.21.15.26:0/34270314 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c03c040 con 0x7fb1240702c0 2024-01-15T18:12:07.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.071+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12414de40 con 0x7fb1240702c0 2024-01-15T18:12:07.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.072+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12414e030 con 0x7fb1240702c0 2024-01-15T18:12:07.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.073+0000 7fb121600640 1 -- 172.21.15.26:0/34270314 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fb10c032070 con 0x7fb1240702c0 2024-01-15T18:12:07.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.074+0000 7fb121600640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0fc06c460 0x7fb0fc06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.074+0000 7fb121600640 1 -- 172.21.15.26:0/34270314 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fb10c0a9aa0 con 0x7fb1240702c0 2024-01-15T18:12:07.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.075+0000 7fb129600640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0fc06c460 0x7fb0fc06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.076+0000 7fb129600640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0fc06c460 0x7fb0fc06e920 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb114006fd0 tx=0x7fb114008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.078+0000 7fb121600640 1 -- 172.21.15.26:0/34270314 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb12414e030 con 0x7fb1240702c0 2024-01-15T18:12:07.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.274+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb12414e030 con 0x7fb1240702c0 2024-01-15T18:12:07.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.276+0000 7fb121600640 1 -- 172.21.15.26:0/34270314 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+12975 (secure 0 0 0) 0x7fb12414e030 con 0x7fb1240702c0 2024-01-15T18:12:07.279 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:07.279 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":111,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:12:06.282358+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:12:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0fc06c460 msgr2=0x7fb0fc06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0fc06c460 0x7fb0fc06e920 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fb114006fd0 tx=0x7fb114008040 comp rx=0 tx=0).stop 2024-01-15T18:12:07.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240702c0 msgr2=0x7fb1240afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240702c0 0x7fb1240afbe0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fb10c02c3b0 tx=0x7fb10c001930 comp rx=0 tx=0).stop 2024-01-15T18:12:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 shutdown_connections 2024-01-15T18:12:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb124070720 0x7fb1240b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb0fc06c460 0x7fb0fc06e920 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb1240702c0 0x7fb1240afbe0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 --2- 172.21.15.26:0/34270314 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb1040047c0 0x7fb12413d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 >> 172.21.15.26:0/34270314 conn(0x7fb12406a800 msgr2=0x7fb12415c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 shutdown_connections 2024-01-15T18:12:07.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.280+0000 7fb12c000640 1 -- 172.21.15.26:0/34270314 wait complete. 2024-01-15T18:12:07.304 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:12:07.305 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:12:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.528+0000 7f2cdf200640 1 Processor -- start 2024-01-15T18:12:07.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.528+0000 7f2cdf200640 1 -- start start 2024-01-15T18:12:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.529+0000 7f2cdf200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2cd80cf9a0 0x7f2cd8166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.530+0000 7f2cdf200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd815c5f0 con 0x7f2cd813e360 2024-01-15T18:12:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.530+0000 7f2cdf200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd806c470 con 0x7f2cd813df80 2024-01-15T18:12:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.530+0000 7f2cdf200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd806c750 con 0x7f2cd80cf9a0 2024-01-15T18:12:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.530+0000 7f2cd7400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2cd813e360 0x7f2cd80cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42808/0 (socket says 172.21.15.26:42808) 2024-01-15T18:12:07.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.530+0000 7f2cd7400640 1 -- 172.21.15.26:0/4164970013 learned_addr learned my addr 172.21.15.26:0/4164970013 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.530+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792880275 0 0) 0x7f2cd815c5f0 con 0x7f2cd813e360 2024-01-15T18:12:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.530+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cb8003520 con 0x7f2cd813e360 2024-01-15T18:12:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.530+0000 7f2cdd200640 1 --2- 172.21.15.26:0/4164970013 >> v2:172.21.15.124:3300/0 conn(0x7f2cd80cf9a0 0x7f2cd8166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2956835762 0 0) 0x7f2cb8003520 con 0x7f2cd813e360 2024-01-15T18:12:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 >> v2:172.21.15.124:3300/0 conn(0x7f2cd80cf9a0 msgr2=0x7f2cd8166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 --2- 172.21.15.26:0/4164970013 >> v2:172.21.15.124:3300/0 conn(0x7f2cd80cf9a0 0x7f2cd8166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 >> v1:172.21.15.124:6789/0 conn(0x7f2cd813df80 legacy=0x7f2cd815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd806ca30 con 0x7f2cd813e360 2024-01-15T18:12:07.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f2cc0001670 con 0x7f2cd813e360 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 >> v1:172.21.15.26:6789/0 conn(0x7f2cd813e360 legacy=0x7f2cd80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cb80039c0 0x7f2cb8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80047c0 0x7f2cb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cb80051d0 0x7f2cb800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd806c750 con 0x7f2cb80051d0 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb800c120 con 0x7f2cb80039c0 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb800c2e0 con 0x7f2cb80047c0 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.531+0000 7f2cdd200640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cb80051d0 0x7f2cb800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.532+0000 7f2cd7e00640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cb80039c0 0x7f2cb8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.532+0000 7f2cd7400640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80047c0 0x7f2cb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.532+0000 7f2cdd200640 1 -- 172.21.15.26:0/4164970013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80047c0 msgr2=0x7f2cb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.532+0000 7f2cdd200640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80047c0 0x7f2cb8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.532+0000 7f2cdd200640 1 -- 172.21.15.26:0/4164970013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cb80039c0 msgr2=0x7f2cb8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.532+0000 7f2cdd200640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cb80039c0 0x7f2cb8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.532+0000 7f2cdd200640 1 -- 172.21.15.26:0/4164970013 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb8003520 con 0x7f2cb80051d0 2024-01-15T18:12:07.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.532+0000 7f2cd7400640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80047c0 0x7f2cb8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.533+0000 7f2cdd200640 1 -- 172.21.15.26:0/4164970013 reap_dead start 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.533+0000 7f2cdd200640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cb80051d0 0x7f2cb800b980 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8007330 tx=0x7f2cb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555e7dc807af11fa server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.533+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc802ccd0 con 0x7f2cb80051d0 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.534+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc802ce30 con 0x7f2cb80051d0 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.534+0000 7f2cd6a00640 1 -- 172.21.15.26:0/4164970013 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8003bd0 con 0x7f2cb80051d0 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.534+0000 7f2cdf200640 1 -- 172.21.15.26:0/4164970013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cb80051d0 msgr2=0x7f2cb800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.534+0000 7f2cdf200640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cb80051d0 0x7f2cb800b980 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8007330 tx=0x7f2cb80057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.534+0000 7f2cdf200640 1 -- 172.21.15.26:0/4164970013 shutdown_connections 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.534+0000 7f2cdf200640 1 --2- 172.21.15.26:0/4164970013 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cb80051d0 0x7f2cb800b980 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.534+0000 7f2cdf200640 1 -- 172.21.15.26:0/4164970013 >> 172.21.15.26:0/4164970013 conn(0x7f2cd806a800 msgr2=0x7f2cd806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.534+0000 7f2cdf200640 1 -- 172.21.15.26:0/4164970013 shutdown_connections 2024-01-15T18:12:07.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.534+0000 7f2cdf200640 1 -- 172.21.15.26:0/4164970013 wait complete. 2024-01-15T18:12:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.536+0000 7f2cdf200640 1 Processor -- start 2024-01-15T18:12:07.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.536+0000 7f2cdf200640 1 -- start start 2024-01-15T18:12:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.537+0000 7f2cdf200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80051d0 0x7f2cd813d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.537+0000 7f2cdf200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cd814aa00 0x7f2cd814f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.537+0000 7f2cdf200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cd814f570 0x7f2cd80b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.537+0000 7f2cdf200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd814af00 con 0x7f2cd814aa00 2024-01-15T18:12:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.537+0000 7f2cdf200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd814b1e0 con 0x7f2cd814f570 2024-01-15T18:12:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.537+0000 7f2cdf200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd814b4c0 con 0x7f2cb80051d0 2024-01-15T18:12:07.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cd814aa00 0x7f2cd814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cd814aa00 0x7f2cd814f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38912/0 (socket says 172.21.15.26:38912) 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7400640 1 -- 172.21.15.26:0/631055656 learned_addr learned my addr 172.21.15.26:0/631055656 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cdd200640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cd814f570 0x7f2cd80b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7e00640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80051d0 0x7f2cd813d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7400640 1 -- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80051d0 msgr2=0x7f2cd813d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7400640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80051d0 0x7f2cd813d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7400640 1 -- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cd814f570 msgr2=0x7f2cd80b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7400640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cd814f570 0x7f2cd80b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7400640 1 -- 172.21.15.26:0/631055656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd814b7a0 con 0x7f2cd814aa00 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cdd200640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cd814f570 0x7f2cd80b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:07.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.538+0000 7f2cd7400640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cd814aa00 0x7f2cd814f030 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000d810 tx=0x7f2cc000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.539+0000 7f2ccfe00640 1 -- 172.21.15.26:0/631055656 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00074e0 con 0x7f2cd814aa00 2024-01-15T18:12:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.539+0000 7f2ccfe00640 1 -- 172.21.15.26:0/631055656 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc0007640 con 0x7f2cd814aa00 2024-01-15T18:12:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.539+0000 7f2ccfe00640 1 -- 172.21.15.26:0/631055656 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0004a00 con 0x7f2cd814aa00 2024-01-15T18:12:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.539+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd814ba80 con 0x7f2cd814aa00 2024-01-15T18:12:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.539+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd80b3410 con 0x7f2cd814aa00 2024-01-15T18:12:07.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.540+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd80b3690 con 0x7f2cd814aa00 2024-01-15T18:12:07.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.541+0000 7f2ccfe00640 1 -- 172.21.15.26:0/631055656 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f2cc0004b60 con 0x7f2cd814aa00 2024-01-15T18:12:07.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.542+0000 7f2ccfe00640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2cb006c430 0x7f2cb006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.543+0000 7f2ccfe00640 1 -- 172.21.15.26:0/631055656 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4941+0+0 (secure 0 0 0) 0x7f2cc0088480 con 0x7f2cd814aa00 2024-01-15T18:12:07.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.543+0000 7f2cd7e00640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2cb006c430 0x7f2cb006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.544+0000 7f2cd7e00640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2cb006c430 0x7f2cb006e8f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0001510 tx=0x7f2cd0009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.547+0000 7f2ccfe00640 1 -- 172.21.15.26:0/631055656 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2cc00549f0 con 0x7f2cd814aa00 2024-01-15T18:12:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.740+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cd814c160 con 0x7f2cd814aa00 2024-01-15T18:12:07.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.740+0000 7f2ccfe00640 1 -- 172.21.15.26:0/631055656 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cc0019070 con 0x7f2cd814aa00 2024-01-15T18:12:07.744 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:12:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2cb006c430 msgr2=0x7f2cb006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2cb006c430 0x7f2cb006e8f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0001510 tx=0x7f2cd0009040 comp rx=0 tx=0).stop 2024-01-15T18:12:07.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cd814aa00 msgr2=0x7f2cd814f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cd814aa00 0x7f2cd814f030 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000d810 tx=0x7f2cc000dcd0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 shutdown_connections 2024-01-15T18:12:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2cb006c430 0x7f2cb006e8f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2cd814f570 0x7f2cd80b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2cd814aa00 0x7f2cd814f030 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 --2- 172.21.15.26:0/631055656 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2cb80051d0 0x7f2cd813d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 >> 172.21.15.26:0/631055656 conn(0x7f2cd806a800 msgr2=0x7f2cd815cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 shutdown_connections 2024-01-15T18:12:07.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.743+0000 7f2cdf200640 1 -- 172.21.15.26:0/631055656 wait complete. 2024-01-15T18:12:07.763 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-15T18:12:07.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.975+0000 7fe6fb200640 1 Processor -- start 2024-01-15T18:12:07.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.975+0000 7fe6fb200640 1 -- start start 2024-01-15T18:12:07.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.976+0000 7fe6fb200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe6fc143c40 0x7fe6fc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.977+0000 7fe6fb200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc070e50 con 0x7fe6fc0717d0 2024-01-15T18:12:07.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.977+0000 7fe6fb200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc144020 con 0x7fe6fc0ccc10 2024-01-15T18:12:07.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.977+0000 7fe6fb200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc0723c0 con 0x7fe6fc143c40 2024-01-15T18:12:07.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.977+0000 7fe6f9400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fe6fc0717d0 0x7fe6fc0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42824/0 (socket says 172.21.15.26:42824) 2024-01-15T18:12:07.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.977+0000 7fe6f9400640 1 -- 172.21.15.26:0/3505601966 learned_addr learned my addr 172.21.15.26:0/3505601966 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:07.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.977+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4244596632 0 0) 0x7fe6fc070e50 con 0x7fe6fc0717d0 2024-01-15T18:12:07.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.977+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6dc003520 con 0x7fe6fc0717d0 2024-01-15T18:12:07.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.977+0000 7fe6f9e00640 1 --2- 172.21.15.26:0/3505601966 >> v2:172.21.15.124:3300/0 conn(0x7fe6fc143c40 0x7fe6fc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 844569787 0 0) 0x7fe6dc003520 con 0x7fe6fc0717d0 2024-01-15T18:12:07.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 >> v2:172.21.15.124:3300/0 conn(0x7fe6fc143c40 msgr2=0x7fe6fc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 --2- 172.21.15.26:0/3505601966 >> v2:172.21.15.124:3300/0 conn(0x7fe6fc143c40 0x7fe6fc071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 >> v1:172.21.15.124:6789/0 conn(0x7fe6fc0ccc10 legacy=0x7fe6fc06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc071e00 con 0x7fe6fc0717d0 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fe6e4001670 con 0x7fe6fc0717d0 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 >> v1:172.21.15.26:6789/0 conn(0x7fe6fc0717d0 legacy=0x7fe6fc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc0039c0 0x7fe6dc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0047c0 0x7fe6dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6dc0052a0 0x7fe6dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc0723c0 con 0x7fe6dc0039c0 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc00c1c0 con 0x7fe6dc0047c0 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6dc00c380 con 0x7fe6dc0052a0 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f9e00640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc0039c0 0x7fe6dc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.978+0000 7fe6f9400640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0047c0 0x7fe6dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.979+0000 7fe6f9e00640 1 -- 172.21.15.26:0/3505601966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6dc0052a0 msgr2=0x7fe6dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.979+0000 7fe6f9e00640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6dc0052a0 0x7fe6dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.979+0000 7fe6f9e00640 1 -- 172.21.15.26:0/3505601966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0047c0 msgr2=0x7fe6dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:07.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.979+0000 7fe6f9e00640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0047c0 0x7fe6dc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.979+0000 7fe6f9e00640 1 -- 172.21.15.26:0/3505601966 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc003520 con 0x7fe6dc0039c0 2024-01-15T18:12:07.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.979+0000 7fe6f9400640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0047c0 0x7fe6dc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:07.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.979+0000 7fe6fa800640 1 -- 172.21.15.26:0/3505601966 reap_dead start 2024-01-15T18:12:07.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.980+0000 7fe6f9e00640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc0039c0 0x7fe6dc003de0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00ba30 tx=0x7fe6dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb70a322720d30a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.980+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec032660 con 0x7fe6dc0039c0 2024-01-15T18:12:07.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.980+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec0327c0 con 0x7fe6dc0039c0 2024-01-15T18:12:07.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.980+0000 7fe6f8a00640 1 -- 172.21.15.26:0/3505601966 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec032b50 con 0x7fe6dc0039c0 2024-01-15T18:12:07.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.980+0000 7fe6fb200640 1 -- 172.21.15.26:0/3505601966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc0039c0 msgr2=0x7fe6dc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.980+0000 7fe6fb200640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc0039c0 0x7fe6dc003de0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00ba30 tx=0x7fe6dc0057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.981+0000 7fe6fb200640 1 -- 172.21.15.26:0/3505601966 shutdown_connections 2024-01-15T18:12:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.981+0000 7fe6fb200640 1 --2- 172.21.15.26:0/3505601966 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6dc0039c0 0x7fe6dc003de0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.981+0000 7fe6fb200640 1 -- 172.21.15.26:0/3505601966 >> 172.21.15.26:0/3505601966 conn(0x7fe6fc06a800 msgr2=0x7fe6fc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.981+0000 7fe6fb200640 1 -- 172.21.15.26:0/3505601966 shutdown_connections 2024-01-15T18:12:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.981+0000 7fe6fb200640 1 -- 172.21.15.26:0/3505601966 wait complete. 2024-01-15T18:12:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.982+0000 7fe6fb200640 1 Processor -- start 2024-01-15T18:12:07.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.982+0000 7fe6fb200640 1 -- start start 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.983+0000 7fe6fb200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0039c0 0x7fe6fc1427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.983+0000 7fe6fb200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6fc135f70 0x7fe6fc13aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.983+0000 7fe6fb200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1363f0 0x7fe6fc13b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.983+0000 7fe6fb200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc1330e0 con 0x7fe6fc1363f0 2024-01-15T18:12:07.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.983+0000 7fe6fb200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc135280 con 0x7fe6dc0039c0 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.983+0000 7fe6fb200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc159a50 con 0x7fe6fc135f70 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6fa800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1363f0 0x7fe6fc13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6fa800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1363f0 0x7fe6fc13b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38922/0 (socket says 172.21.15.26:38922) 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6fa800640 1 -- 172.21.15.26:0/2114306388 learned_addr learned my addr 172.21.15.26:0/2114306388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6f9e00640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0039c0 0x7fe6fc1427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6fa800640 1 -- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6fc135f70 msgr2=0x7fe6fc13aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6fa800640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6fc135f70 0x7fe6fc13aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6fa800640 1 -- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0039c0 msgr2=0x7fe6fc1427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6fa800640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0039c0 0x7fe6fc1427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6fa800640 1 -- 172.21.15.26:0/2114306388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc135ae0 con 0x7fe6fc1363f0 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.984+0000 7fe6fa800640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1363f0 0x7fe6fc13b530 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0cc3f0 tx=0x7fe6f400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.985+0000 7fe6f2000640 1 -- 172.21.15.26:0/2114306388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4007070 con 0x7fe6fc1363f0 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.985+0000 7fe6f2000640 1 -- 172.21.15.26:0/2114306388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f40071d0 con 0x7fe6fc1363f0 2024-01-15T18:12:07.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.985+0000 7fe6f2000640 1 -- 172.21.15.26:0/2114306388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4007580 con 0x7fe6fc1363f0 2024-01-15T18:12:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.985+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc14a0a0 con 0x7fe6fc1363f0 2024-01-15T18:12:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.985+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc135490 con 0x7fe6fc1363f0 2024-01-15T18:12:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.986+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc135c70 con 0x7fe6fc1363f0 2024-01-15T18:12:07.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.987+0000 7fe6f2000640 1 -- 172.21.15.26:0/2114306388 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fe6f40076e0 con 0x7fe6fc1363f0 2024-01-15T18:12:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.988+0000 7fe6f2000640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6d406c460 0x7fe6d406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.988+0000 7fe6f2000640 1 -- 172.21.15.26:0/2114306388 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4941+0+0 (secure 0 0 0) 0x7fe6f4087b90 con 0x7fe6fc1363f0 2024-01-15T18:12:07.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.989+0000 7fe6f9e00640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6d406c460 0x7fe6d406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:07.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.990+0000 7fe6f9e00640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6d406c460 0x7fe6d406e920 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00ba00 tx=0x7fe6ec030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:07.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:07.992+0000 7fe6f2000640 1 -- 172.21.15.26:0/2114306388 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6f40541c0 con 0x7fe6fc1363f0 2024-01-15T18:12:08.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.190+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe6fc067450 con 0x7fe6fc1363f0 2024-01-15T18:12:08.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.191+0000 7fe6f2000640 1 -- 172.21.15.26:0/2114306388 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v111) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe6f4019090 con 0x7fe6fc1363f0 2024-01-15T18:12:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:08.194 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:12:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "poolnum": 1, 2024-01-15T18:12:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: "poolname": ".mgr" 2024-01-15T18:12:08.194 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:08.194 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.193+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6d406c460 msgr2=0x7fe6d406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.193+0000 7fe6fb200640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6d406c460 0x7fe6d406e920 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00ba00 tx=0x7fe6ec030000 comp rx=0 tx=0).stop 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.193+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1363f0 msgr2=0x7fe6fc13b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.193+0000 7fe6fb200640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1363f0 0x7fe6fc13b530 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0cc3f0 tx=0x7fe6f400c8b0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.194+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 shutdown_connections 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.194+0000 7fe6fb200640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe6fc135f70 0x7fe6fc13aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.194+0000 7fe6fb200640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe6d406c460 0x7fe6d406e920 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.194+0000 7fe6fb200640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe6fc1363f0 0x7fe6fc13b530 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.194+0000 7fe6fb200640 1 --2- 172.21.15.26:0/2114306388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe6dc0039c0 0x7fe6fc1427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.194+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 >> 172.21.15.26:0/2114306388 conn(0x7fe6fc06a800 msgr2=0x7fe6fc0b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:08.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.194+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 shutdown_connections 2024-01-15T18:12:08.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.194+0000 7fe6fb200640 1 -- 172.21.15.26:0/2114306388 wait complete. 2024-01-15T18:12:08.214 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-15T18:12:08.214 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-15T18:12:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.430+0000 7eff3a600640 1 Processor -- start 2024-01-15T18:12:08.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.430+0000 7eff3a600640 1 -- start start 2024-01-15T18:12:08.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.432+0000 7eff3a600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7eff3415da10 0x7eff340b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:08.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.432+0000 7eff3a600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff34150ba0 con 0x7eff34150760 2024-01-15T18:12:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.432+0000 7eff3a600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff34071190 con 0x7eff34072230 2024-01-15T18:12:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.432+0000 7eff3a600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff34072610 con 0x7eff3415da10 2024-01-15T18:12:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.432+0000 7eff32a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7eff34150760 0x7eff3415d300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:42834/0 (socket says 172.21.15.26:42834) 2024-01-15T18:12:08.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.433+0000 7eff32a00640 1 -- 172.21.15.26:0/1694032865 learned_addr learned my addr 172.21.15.26:0/1694032865 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:08.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.433+0000 7eff33e00640 1 --2- 172.21.15.26:0/1694032865 >> v2:172.21.15.124:3300/0 conn(0x7eff3415da10 0x7eff340b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:08.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.433+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454364810 0 0) 0x7eff34150ba0 con 0x7eff34150760 2024-01-15T18:12:08.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.433+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff14003520 con 0x7eff34150760 2024-01-15T18:12:08.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.433+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1117363098 0 0) 0x7eff14003520 con 0x7eff34150760 2024-01-15T18:12:08.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 >> v2:172.21.15.124:3300/0 conn(0x7eff3415da10 msgr2=0x7eff340b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:08.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 --2- 172.21.15.26:0/1694032865 >> v2:172.21.15.124:3300/0 conn(0x7eff3415da10 0x7eff340b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 >> v1:172.21.15.124:6789/0 conn(0x7eff34072230 legacy=0x7eff340cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34071470 con 0x7eff34150760 2024-01-15T18:12:08.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7eff1c0019f0 con 0x7eff34150760 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 >> v1:172.21.15.26:6789/0 conn(0x7eff34150760 legacy=0x7eff3415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14003970 0x7eff14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff140047c0 0x7eff14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff14005180 0x7eff1400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff34072610 con 0x7eff14005180 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1400e230 con 0x7eff140047c0 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1400e3f0 con 0x7eff14003970 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff33e00640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff14005180 0x7eff1400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff32a00640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff140047c0 0x7eff14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff33e00640 1 -- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14003970 msgr2=0x7eff14003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.434+0000 7eff33400640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14003970 0x7eff14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.435+0000 7eff33e00640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14003970 0x7eff14003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.435+0000 7eff33e00640 1 -- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff140047c0 msgr2=0x7eff14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:08.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.435+0000 7eff33e00640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff140047c0 0x7eff14004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.435+0000 7eff33e00640 1 -- 172.21.15.26:0/1694032865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff14003520 con 0x7eff14005180 2024-01-15T18:12:08.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.435+0000 7eff32a00640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff140047c0 0x7eff14004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:08.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.435+0000 7eff33400640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14003970 0x7eff14003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.435+0000 7eff33e00640 1 -- 172.21.15.26:0/1694032865 reap_dead start 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.436+0000 7eff33e00640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff14005180 0x7eff1400db90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009a90 tx=0x7eff140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f9d7a21b93486e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.436+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c02c6d0 con 0x7eff14005180 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.436+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c02c830 con 0x7eff14005180 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.436+0000 7eff32000640 1 -- 172.21.15.26:0/1694032865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c02cbe0 con 0x7eff14005180 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.436+0000 7eff3a600640 1 -- 172.21.15.26:0/1694032865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff14005180 msgr2=0x7eff1400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.436+0000 7eff3a600640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff14005180 0x7eff1400db90 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009a90 tx=0x7eff140057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.437+0000 7eff3a600640 1 -- 172.21.15.26:0/1694032865 shutdown_connections 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.437+0000 7eff3a600640 1 --2- 172.21.15.26:0/1694032865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff14005180 0x7eff1400db90 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.437+0000 7eff3a600640 1 -- 172.21.15.26:0/1694032865 >> 172.21.15.26:0/1694032865 conn(0x7eff3406a800 msgr2=0x7eff3406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:08.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.437+0000 7eff3a600640 1 -- 172.21.15.26:0/1694032865 shutdown_connections 2024-01-15T18:12:08.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.437+0000 7eff3a600640 1 -- 172.21.15.26:0/1694032865 wait complete. 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.438+0000 7eff3a600640 1 Processor -- start 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.439+0000 7eff3a600640 1 -- start start 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff3a600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14005180 0x7eff3414b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff3a600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff3414d590 0x7eff3414c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff3a600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff3414c850 0x7eff340ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff3a600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340cdf40 con 0x7eff3414c850 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff3a600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340cd370 con 0x7eff3414d590 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff3a600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3414ccd0 con 0x7eff14005180 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff3414c850 0x7eff340ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff3414c850 0x7eff340ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38930/0 (socket says 172.21.15.26:38930) 2024-01-15T18:12:08.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33e00640 1 -- 172.21.15.26:0/930333717 learned_addr learned my addr 172.21.15.26:0/930333717 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff32a00640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff3414d590 0x7eff3414c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33400640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14005180 0x7eff3414b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33e00640 1 -- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14005180 msgr2=0x7eff3414b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33e00640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14005180 0x7eff3414b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33e00640 1 -- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff3414d590 msgr2=0x7eff3414c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33e00640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff3414d590 0x7eff3414c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33e00640 1 -- 172.21.15.26:0/930333717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3414b750 con 0x7eff3414c850 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.440+0000 7eff33400640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14005180 0x7eff3414b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.441+0000 7eff33e00640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff3414c850 0x7eff340ac8d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009bb0 tx=0x7eff2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:08.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.441+0000 7eff2b400640 1 -- 172.21.15.26:0/930333717 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c034850 con 0x7eff3414c850 2024-01-15T18:12:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.441+0000 7eff2b400640 1 -- 172.21.15.26:0/930333717 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c0349b0 con 0x7eff3414c850 2024-01-15T18:12:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.441+0000 7eff2b400640 1 -- 172.21.15.26:0/930333717 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c034d40 con 0x7eff3414c850 2024-01-15T18:12:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.441+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff340cd5a0 con 0x7eff3414c850 2024-01-15T18:12:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.441+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff340cdb60 con 0x7eff3414c850 2024-01-15T18:12:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.442+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff340ce080 con 0x7eff3414c850 2024-01-15T18:12:08.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.444+0000 7eff2b400640 1 -- 172.21.15.26:0/930333717 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7eff2c03b040 con 0x7eff3414c850 2024-01-15T18:12:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.444+0000 7eff2b400640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7eff0c06c430 0x7eff0c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:08.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.445+0000 7eff2b400640 1 -- 172.21.15.26:0/930333717 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 4941+0+0 (secure 0 0 0) 0x7eff2c0a9800 con 0x7eff3414c850 2024-01-15T18:12:08.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.445+0000 7eff33400640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7eff0c06c430 0x7eff0c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:08.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.446+0000 7eff33400640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7eff0c06c430 0x7eff0c06e8f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7eff24001690 tx=0x7eff24009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:08.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.449+0000 7eff2b400640 1 -- 172.21.15.26:0/930333717 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff2c075e20 con 0x7eff3414c850 2024-01-15T18:12:08.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.639+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7eff340675d0 con 0x7eff3414c850 2024-01-15T18:12:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.982+0000 7eff2b400640 1 -- 172.21.15.26:0/930333717 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v112) v1 ==== 138+0+0 (secure 0 0 0) 0x7eff2c0390b0 con 0x7eff3414c850 2024-01-15T18:12:08.986 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_metadata' created 2024-01-15T18:12:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.986+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7eff0c06c430 msgr2=0x7eff0c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7eff0c06c430 0x7eff0c06e8f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7eff24001690 tx=0x7eff24009040 comp rx=0 tx=0).stop 2024-01-15T18:12:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff3414c850 msgr2=0x7eff340ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff3414c850 0x7eff340ac8d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009bb0 tx=0x7eff2c004730 comp rx=0 tx=0).stop 2024-01-15T18:12:08.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 shutdown_connections 2024-01-15T18:12:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7eff0c06c430 0x7eff0c06e8f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7eff3414c850 0x7eff340ac8d0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7eff3414d590 0x7eff3414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 --2- 172.21.15.26:0/930333717 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7eff14005180 0x7eff3414b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 >> 172.21.15.26:0/930333717 conn(0x7eff3406a800 msgr2=0x7eff3415c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 shutdown_connections 2024-01-15T18:12:08.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:08.987+0000 7eff3a600640 1 -- 172.21.15.26:0/930333717 wait complete. 2024-01-15T18:12:09.016 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-15T18:12:09.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.224+0000 7f2992c00640 1 Processor -- start 2024-01-15T18:12:09.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.225+0000 7f2992c00640 1 -- start start 2024-01-15T18:12:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.226+0000 7f2992c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f298c160920 0x7f298c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.226+0000 7f2992c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f298c0cfea0 0x7f298c161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.226+0000 7f2992c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298c0d0a10 con 0x7f298c0cf9f0 2024-01-15T18:12:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.226+0000 7f2992c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298c0d1f50 con 0x7f298c160920 2024-01-15T18:12:09.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.226+0000 7f2992c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298c0cf080 con 0x7f298c0cfea0 2024-01-15T18:12:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.226+0000 7f2991800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f298c160920 0x7f298c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.226+0000 7f2991800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f298c160920 0x7f298c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38942/0 (socket says 172.21.15.26:38942) 2024-01-15T18:12:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.226+0000 7f2991800640 1 -- 172.21.15.26:0/808555702 learned_addr learned my addr 172.21.15.26:0/808555702 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.227+0000 7f2991800640 1 -- 172.21.15.26:0/808555702 >> v2:172.21.15.124:3300/0 conn(0x7f298c0cfea0 msgr2=0x7f298c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.227+0000 7f2991800640 1 --2- 172.21.15.26:0/808555702 >> v2:172.21.15.124:3300/0 conn(0x7f298c0cfea0 0x7f298c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.227+0000 7f2991800640 1 -- 172.21.15.26:0/808555702 >> v1:172.21.15.124:6789/0 conn(0x7f298c0cf9f0 legacy=0x7f298c161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:09.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.227+0000 7f2991800640 1 -- 172.21.15.26:0/808555702 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c0d0310 con 0x7f298c160920 2024-01-15T18:12:09.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.228+0000 7f2991800640 1 --2- 172.21.15.26:0/808555702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c160920 0x7f298c0b3f30 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f298c08c2c0 tx=0x7f297c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d478a2832621fe6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:09.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.228+0000 7f298be00640 1 -- 172.21.15.26:0/808555702 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c031870 con 0x7f298c160920 2024-01-15T18:12:09.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.228+0000 7f298be00640 1 -- 172.21.15.26:0/808555702 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c0319d0 con 0x7f298c160920 2024-01-15T18:12:09.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.228+0000 7f298be00640 1 -- 172.21.15.26:0/808555702 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c031d60 con 0x7f298c160920 2024-01-15T18:12:09.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.229+0000 7f2992c00640 1 -- 172.21.15.26:0/808555702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c160920 msgr2=0x7f298c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:09.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.229+0000 7f2992c00640 1 --2- 172.21.15.26:0/808555702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c160920 0x7f298c0b3f30 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f298c08c2c0 tx=0x7f297c02b910 comp rx=0 tx=0).stop 2024-01-15T18:12:09.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.229+0000 7f2992c00640 1 -- 172.21.15.26:0/808555702 shutdown_connections 2024-01-15T18:12:09.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.229+0000 7f2992c00640 1 --2- 172.21.15.26:0/808555702 >> v2:172.21.15.124:3300/0 conn(0x7f298c0cfea0 0x7f298c161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:09.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.229+0000 7f2992c00640 1 --2- 172.21.15.26:0/808555702 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c160920 0x7f298c0b3f30 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:09.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.229+0000 7f2992c00640 1 -- 172.21.15.26:0/808555702 >> 172.21.15.26:0/808555702 conn(0x7f298c0d3090 msgr2=0x7f298c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:09.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.229+0000 7f2992c00640 1 -- 172.21.15.26:0/808555702 shutdown_connections 2024-01-15T18:12:09.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.229+0000 7f2992c00640 1 -- 172.21.15.26:0/808555702 wait complete. 2024-01-15T18:12:09.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.231+0000 7f2992c00640 1 Processor -- start 2024-01-15T18:12:09.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.231+0000 7f2992c00640 1 -- start start 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.232+0000 7f2992c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f298c0cf9f0 0x7f298c1f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.232+0000 7f2992c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c0cfea0 0x7f298c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.232+0000 7f2992c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f298c160920 0x7f298c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.232+0000 7f2992c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c1f6820 con 0x7f298c0cfea0 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.232+0000 7f2992c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0cb980 con 0x7f298c0cf9f0 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.232+0000 7f2992c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c15dab0 con 0x7f298c160920 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.232+0000 7f2990e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c0cfea0 0x7f298c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.232+0000 7f2990e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c0cfea0 0x7f298c0cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38946/0 (socket says 172.21.15.26:38946) 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.232+0000 7f2990e00640 1 -- 172.21.15.26:0/4175383943 learned_addr learned my addr 172.21.15.26:0/4175383943 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:09.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.233+0000 7f2991800640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f298c0cf9f0 0x7f298c1f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:09.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.233+0000 7f2992200640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f298c160920 0x7f298c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:09.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.233+0000 7f2990e00640 1 -- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f298c160920 msgr2=0x7f298c0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:09.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.233+0000 7f2990e00640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f298c160920 0x7f298c0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:09.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.233+0000 7f2990e00640 1 -- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f298c0cf9f0 msgr2=0x7f298c1f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:09.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.233+0000 7f2990e00640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f298c0cf9f0 0x7f298c1f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:09.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.233+0000 7f2990e00640 1 -- 172.21.15.26:0/4175383943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297c0094d0 con 0x7f298c0cfea0 2024-01-15T18:12:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.233+0000 7f2990e00640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c0cfea0 0x7f298c0cc770 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f297400ba50 tx=0x7f297400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.234+0000 7f2989600640 1 -- 172.21.15.26:0/4175383943 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297400cc10 con 0x7f298c0cfea0 2024-01-15T18:12:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.234+0000 7f2989600640 1 -- 172.21.15.26:0/4175383943 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297400cd70 con 0x7f298c0cfea0 2024-01-15T18:12:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.234+0000 7f2989600640 1 -- 172.21.15.26:0/4175383943 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2974014680 con 0x7f298c0cfea0 2024-01-15T18:12:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.234+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c0cad70 con 0x7f298c0cfea0 2024-01-15T18:12:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.234+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c0cb330 con 0x7f298c0cfea0 2024-01-15T18:12:09.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.235+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298c0cbb30 con 0x7f298c0cfea0 2024-01-15T18:12:09.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.237+0000 7f2989600640 1 -- 172.21.15.26:0/4175383943 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f297401e440 con 0x7f298c0cfea0 2024-01-15T18:12:09.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.238+0000 7f2989600640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f296406c430 0x7f296406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:09.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.238+0000 7f2989600640 1 -- 172.21.15.26:0/4175383943 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5311+0+0 (secure 0 0 0) 0x7f2974089110 con 0x7f298c0cfea0 2024-01-15T18:12:09.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.239+0000 7f2991800640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f296406c430 0x7f296406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:09.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.240+0000 7f2991800640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f296406c430 0x7f296406e8f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f297c036000 tx=0x7f297c03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:09.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.241+0000 7f2989600640 1 -- 172.21.15.26:0/4175383943 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2974055680 con 0x7f298c0cfea0 2024-01-15T18:12:09.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.441+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f298c0c91b0 con 0x7f298c0cfea0 2024-01-15T18:12:09.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.990+0000 7f2989600640 1 -- 172.21.15.26:0/4175383943 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v113) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2974019020 con 0x7f298c0cfea0 2024-01-15T18:12:09.994 INFO:teuthology.orchestra.run.smithi026.stderr:pool 'cephfs_data' created 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f296406c430 msgr2=0x7f296406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f296406c430 0x7f296406e8f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f297c036000 tx=0x7f297c03c040 comp rx=0 tx=0).stop 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c0cfea0 msgr2=0x7f298c0cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c0cfea0 0x7f298c0cc770 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f297400ba50 tx=0x7f297400bf10 comp rx=0 tx=0).stop 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 shutdown_connections 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f298c160920 0x7f298c0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f296406c430 0x7f296406e8f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f298c0cfea0 0x7f298c0cc770 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 --2- 172.21.15.26:0/4175383943 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f298c0cf9f0 0x7f298c1f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.994+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 >> 172.21.15.26:0/4175383943 conn(0x7f298c0d3090 msgr2=0x7f298c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.995+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 shutdown_connections 2024-01-15T18:12:09.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:09.995+0000 7f2992c00640 1 -- 172.21.15.26:0/4175383943 wait complete. 2024-01-15T18:12:10.031 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-15T18:12:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.215+0000 7f6fd2c00640 1 Processor -- start 2024-01-15T18:12:10.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.215+0000 7f6fd2c00640 1 -- start start 2024-01-15T18:12:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.216+0000 7f6fd2c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6fcc13df80 0x7f6fcc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.216+0000 7f6fd2c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6fcc0cf9a0 0x7f6fcc0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.216+0000 7f6fd2c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc06c470 con 0x7f6fcc13e360 2024-01-15T18:12:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.216+0000 7f6fd2c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc06c750 con 0x7f6fcc13df80 2024-01-15T18:12:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.216+0000 7f6fd2c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc06ca30 con 0x7f6fcc0cf9a0 2024-01-15T18:12:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.217+0000 7f6fcbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6fcc13df80 0x7f6fcc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.217+0000 7f6fcbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6fcc13df80 0x7f6fcc15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38962/0 (socket says 172.21.15.26:38962) 2024-01-15T18:12:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.217+0000 7f6fcbe00640 1 -- 172.21.15.26:0/3915648941 learned_addr learned my addr 172.21.15.26:0/3915648941 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.217+0000 7f6fcbe00640 1 -- 172.21.15.26:0/3915648941 >> v2:172.21.15.124:3300/0 conn(0x7f6fcc0cf9a0 msgr2=0x7f6fcc0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:10.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.217+0000 7f6fcbe00640 1 --2- 172.21.15.26:0/3915648941 >> v2:172.21.15.124:3300/0 conn(0x7f6fcc0cf9a0 0x7f6fcc0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.217+0000 7f6fcbe00640 1 -- 172.21.15.26:0/3915648941 >> v1:172.21.15.124:6789/0 conn(0x7f6fcc13e360 legacy=0x7f6fcc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.217+0000 7f6fcbe00640 1 -- 172.21.15.26:0/3915648941 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc06cd10 con 0x7f6fcc13df80 2024-01-15T18:12:10.221 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.218+0000 7f6fcbe00640 1 --2- 172.21.15.26:0/3915648941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13df80 0x7f6fcc15bee0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0ccb10 tx=0x7f6fbc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46001f3c4c332be server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fcaa00640 1 -- 172.21.15.26:0/3915648941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc031870 con 0x7f6fcc13df80 2024-01-15T18:12:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fcaa00640 1 -- 172.21.15.26:0/3915648941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc0319d0 con 0x7f6fcc13df80 2024-01-15T18:12:10.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fcaa00640 1 -- 172.21.15.26:0/3915648941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc031d60 con 0x7f6fcc13df80 2024-01-15T18:12:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3915648941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13df80 msgr2=0x7f6fcc15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fd2c00640 1 --2- 172.21.15.26:0/3915648941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13df80 0x7f6fcc15bee0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0ccb10 tx=0x7f6fbc02b910 comp rx=0 tx=0).stop 2024-01-15T18:12:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3915648941 shutdown_connections 2024-01-15T18:12:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fd2c00640 1 --2- 172.21.15.26:0/3915648941 >> v2:172.21.15.124:3300/0 conn(0x7f6fcc0cf9a0 0x7f6fcc0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fd2c00640 1 --2- 172.21.15.26:0/3915648941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13df80 0x7f6fcc15bee0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3915648941 >> 172.21.15.26:0/3915648941 conn(0x7f6fcc06a800 msgr2=0x7f6fcc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3915648941 shutdown_connections 2024-01-15T18:12:10.223 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.219+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3915648941 wait complete. 2024-01-15T18:12:10.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.221+0000 7f6fd2c00640 1 Processor -- start 2024-01-15T18:12:10.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.221+0000 7f6fd2c00640 1 -- start start 2024-01-15T18:12:10.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.222+0000 7f6fd2c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc0cf9a0 0x7f6fcc151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.222+0000 7f6fd2c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc13df80 0x7f6fcc1579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.222+0000 7f6fd2c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13e360 0x7f6fcc13ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.222+0000 7f6fd2c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0b2ef0 con 0x7f6fcc13e360 2024-01-15T18:12:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.222+0000 7f6fd2c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc155e10 con 0x7f6fcc0cf9a0 2024-01-15T18:12:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.222+0000 7f6fd2c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc1560f0 con 0x7f6fcc13df80 2024-01-15T18:12:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fd0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13e360 0x7f6fcc13ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fd0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13e360 0x7f6fcc13ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38966/0 (socket says 172.21.15.26:38966) 2024-01-15T18:12:10.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fd0c00640 1 -- 172.21.15.26:0/3387141135 learned_addr learned my addr 172.21.15.26:0/3387141135 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:10.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fcb400640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc13df80 0x7f6fcc1579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fcbe00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc0cf9a0 0x7f6fcc151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fd0c00640 1 -- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc13df80 msgr2=0x7f6fcc1579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fd0c00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc13df80 0x7f6fcc1579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fd0c00640 1 -- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc0cf9a0 msgr2=0x7f6fcc151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fd0c00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc0cf9a0 0x7f6fcc151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.223+0000 7f6fd0c00640 1 -- 172.21.15.26:0/3387141135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0094d0 con 0x7f6fcc13e360 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.224+0000 7f6fd0c00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13e360 0x7f6fcc13ab40 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400b9e0 tx=0x7f6fc400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.224+0000 7f6fc3e00640 1 -- 172.21.15.26:0/3387141135 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc400cb20 con 0x7f6fcc13e360 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.224+0000 7f6fc3e00640 1 -- 172.21.15.26:0/3387141135 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc400cca0 con 0x7f6fcc13e360 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.224+0000 7f6fc3e00640 1 -- 172.21.15.26:0/3387141135 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4014680 con 0x7f6fcc13e360 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.224+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc1563d0 con 0x7f6fcc13e360 2024-01-15T18:12:10.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.224+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc157f20 con 0x7f6fcc13e360 2024-01-15T18:12:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.225+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc0b3060 con 0x7f6fcc13e360 2024-01-15T18:12:10.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.227+0000 7f6fc3e00640 1 -- 172.21.15.26:0/3387141135 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f6fc401d3b0 con 0x7f6fcc13e360 2024-01-15T18:12:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.227+0000 7f6fc3e00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fa406c460 0x7f6fa406e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:10.230 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.228+0000 7f6fc3e00640 1 -- 172.21.15.26:0/3387141135 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5693+0+0 (secure 0 0 0) 0x7f6fc4088a00 con 0x7f6fcc13e360 2024-01-15T18:12:10.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.228+0000 7f6fcbe00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fa406c460 0x7f6fa406e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:10.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.229+0000 7f6fcbe00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fa406c460 0x7f6fa406e920 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc036000 tx=0x7f6fbc03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:10.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.233+0000 7f6fc3e00640 1 -- 172.21.15.26:0/3387141135 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fc4054f70 con 0x7f6fcc13e360 2024-01-15T18:12:10.469 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:10.466+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6fcc0675d0 con 0x7f6fcc13e360 2024-01-15T18:12:11.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.274+0000 7f6fc3e00640 1 -- 172.21.15.26:0/3387141135 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6fc4019070 con 0x7f6fcc13e360 2024-01-15T18:12:11.278 INFO:teuthology.orchestra.run.smithi026.stderr:new fs with metadata pool 16 and data pool 17 2024-01-15T18:12:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.279+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fa406c460 msgr2=0x7f6fa406e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.279+0000 7f6fd2c00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fa406c460 0x7f6fa406e920 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc036000 tx=0x7f6fbc03c040 comp rx=0 tx=0).stop 2024-01-15T18:12:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.279+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13e360 msgr2=0x7f6fcc13ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.279+0000 7f6fd2c00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13e360 0x7f6fcc13ab40 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400b9e0 tx=0x7f6fc400bea0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.280+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 shutdown_connections 2024-01-15T18:12:11.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.280+0000 7f6fd2c00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fcc13df80 0x7f6fcc1579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.280+0000 7f6fd2c00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fa406c460 0x7f6fa406e920 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.280+0000 7f6fd2c00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fcc13e360 0x7f6fcc13ab40 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.280+0000 7f6fd2c00640 1 --2- 172.21.15.26:0/3387141135 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fcc0cf9a0 0x7f6fcc151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.280+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 >> 172.21.15.26:0/3387141135 conn(0x7f6fcc06a800 msgr2=0x7f6fcc15e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:11.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.280+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 shutdown_connections 2024-01-15T18:12:11.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.280+0000 7f6fd2c00640 1 -- 172.21.15.26:0/3387141135 wait complete. 2024-01-15T18:12:11.309 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:12:11.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.495+0000 7f2e72000640 1 Processor -- start 2024-01-15T18:12:11.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.495+0000 7f2e72000640 1 -- start start 2024-01-15T18:12:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.497+0000 7f2e72000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2e6c0726f0 0x7f2e6c0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.497+0000 7f2e72000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.497+0000 7f2e72000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6c15dba0 con 0x7f2e6c146650 2024-01-15T18:12:11.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.497+0000 7f2e72000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6c0cd130 con 0x7f2e6c0726f0 2024-01-15T18:12:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.497+0000 7f2e72000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c06f800 con 0x7f2e6c15e030 2024-01-15T18:12:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.498+0000 7f2e71600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.498+0000 7f2e71600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c15e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:56144/0 (socket says 172.21.15.26:56144) 2024-01-15T18:12:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.498+0000 7f2e71600640 1 -- 172.21.15.26:0/4012217376 learned_addr learned my addr 172.21.15.26:0/4012217376 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:11.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.498+0000 7f2e6be00640 1 --2- 172.21.15.26:0/4012217376 >> v2:172.21.15.124:3300/0 conn(0x7f2e6c0726f0 0x7f2e6c0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.499+0000 7f2e71600640 1 -- 172.21.15.26:0/4012217376 >> v2:172.21.15.124:3300/0 conn(0x7f2e6c0726f0 msgr2=0x7f2e6c0b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:11.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.499+0000 7f2e71600640 1 --2- 172.21.15.26:0/4012217376 >> v2:172.21.15.124:3300/0 conn(0x7f2e6c0726f0 0x7f2e6c0b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.499+0000 7f2e71600640 1 -- 172.21.15.26:0/4012217376 >> v1:172.21.15.124:6789/0 conn(0x7f2e6c146650 legacy=0x7f2e6c0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.499+0000 7f2e71600640 1 -- 172.21.15.26:0/4012217376 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c06f240 con 0x7f2e6c15e030 2024-01-15T18:12:11.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.501+0000 7f2e71600640 1 --2- 172.21.15.26:0/4012217376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c15e4e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f631182a4a2226 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:11.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.501+0000 7f2e6b400640 1 -- 172.21.15.26:0/4012217376 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c030840 con 0x7f2e6c15e030 2024-01-15T18:12:11.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.501+0000 7f2e6b400640 1 -- 172.21.15.26:0/4012217376 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c0309a0 con 0x7f2e6c15e030 2024-01-15T18:12:11.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.501+0000 7f2e6b400640 1 -- 172.21.15.26:0/4012217376 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c030d30 con 0x7f2e6c15e030 2024-01-15T18:12:11.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.502+0000 7f2e72000640 1 -- 172.21.15.26:0/4012217376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 msgr2=0x7f2e6c15e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.502+0000 7f2e72000640 1 --2- 172.21.15.26:0/4012217376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c15e4e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009870 tx=0x7f2e5c02b980 comp rx=0 tx=0).stop 2024-01-15T18:12:11.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.502+0000 7f2e72000640 1 -- 172.21.15.26:0/4012217376 shutdown_connections 2024-01-15T18:12:11.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.502+0000 7f2e72000640 1 --2- 172.21.15.26:0/4012217376 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c15e4e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.502+0000 7f2e72000640 1 --2- 172.21.15.26:0/4012217376 >> v2:172.21.15.124:3300/0 conn(0x7f2e6c0726f0 0x7f2e6c0b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.502+0000 7f2e72000640 1 -- 172.21.15.26:0/4012217376 >> 172.21.15.26:0/4012217376 conn(0x7f2e6c0b1d40 msgr2=0x7f2e6c0b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.502+0000 7f2e72000640 1 -- 172.21.15.26:0/4012217376 shutdown_connections 2024-01-15T18:12:11.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.502+0000 7f2e72000640 1 -- 172.21.15.26:0/4012217376 wait complete. 2024-01-15T18:12:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.504+0000 7f2e72000640 1 Processor -- start 2024-01-15T18:12:11.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.504+0000 7f2e72000640 1 -- start start 2024-01-15T18:12:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.505+0000 7f2e72000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c0726f0 0x7f2e6c15ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.505+0000 7f2e72000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6c146650 0x7f2e6c0acd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.505+0000 7f2e72000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.505+0000 7f2e72000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0ae1e0 con 0x7f2e6c146650 2024-01-15T18:12:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.505+0000 7f2e72000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c15d3b0 con 0x7f2e6c0726f0 2024-01-15T18:12:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.505+0000 7f2e72000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c0ad560 con 0x7f2e6c15e030 2024-01-15T18:12:11.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e6be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6c146650 0x7f2e6c0acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e6be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6c146650 0x7f2e6c0acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38980/0 (socket says 172.21.15.26:38980) 2024-01-15T18:12:11.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e6be00640 1 -- 172.21.15.26:0/1760407288 learned_addr learned my addr 172.21.15.26:0/1760407288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:11.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e71600640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e70c00640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c0726f0 0x7f2e6c15ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e6be00640 1 -- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 msgr2=0x7f2e6c145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e6be00640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e6be00640 1 -- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c0726f0 msgr2=0x7f2e6c15ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e6be00640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c0726f0 0x7f2e6c15ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e6be00640 1 -- 172.21.15.26:0/1760407288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c0094d0 con 0x7f2e6c146650 2024-01-15T18:12:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e70c00640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c0726f0 0x7f2e6c15ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.506+0000 7f2e71600640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:11.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.507+0000 7f2e6be00640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6c146650 0x7f2e6c0acd10 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400e840 tx=0x7f2e5400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.507+0000 7f2e68c00640 1 -- 172.21.15.26:0/1760407288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e540094f0 con 0x7f2e6c146650 2024-01-15T18:12:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.507+0000 7f2e68c00640 1 -- 172.21.15.26:0/1760407288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5400c940 con 0x7f2e6c146650 2024-01-15T18:12:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.507+0000 7f2e68c00640 1 -- 172.21.15.26:0/1760407288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400ccd0 con 0x7f2e6c146650 2024-01-15T18:12:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.507+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c0ad840 con 0x7f2e6c146650 2024-01-15T18:12:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.507+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c0ade00 con 0x7f2e6c146650 2024-01-15T18:12:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.508+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6c0ae320 con 0x7f2e6c146650 2024-01-15T18:12:11.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.509+0000 7f2e68c00640 1 -- 172.21.15.26:0/1760407288 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f2e54013470 con 0x7f2e6c146650 2024-01-15T18:12:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.510+0000 7f2e68c00640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2e4406c430 0x7f2e4406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.510+0000 7f2e68c00640 1 -- 172.21.15.26:0/1760407288 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f2e54088b80 con 0x7f2e6c146650 2024-01-15T18:12:11.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.511+0000 7f2e70c00640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2e4406c430 0x7f2e4406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.512+0000 7f2e70c00640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2e4406c430 0x7f2e4406e8f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2e640012e0 tx=0x7f2e64009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:11.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.515+0000 7f2e68c00640 1 -- 172.21.15.26:0/1760407288 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e6c0ae320 con 0x7f2e6c146650 2024-01-15T18:12:11.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.713+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e6c0ae320 con 0x7f2e6c146650 2024-01-15T18:12:11.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.716+0000 7f2e68c00640 1 -- 172.21.15.26:0/1760407288 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+16648 (secure 0 0 0) 0x7f2e6c0ae320 con 0x7f2e6c146650 2024-01-15T18:12:11.720 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:11.720 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":114,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:12:10.467829+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:12:08.696115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:12:09.497434+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:12:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2e4406c430 msgr2=0x7f2e4406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2e4406c430 0x7f2e4406e8f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2e640012e0 tx=0x7f2e64009040 comp rx=0 tx=0).stop 2024-01-15T18:12:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6c146650 msgr2=0x7f2e6c0acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6c146650 0x7f2e6c0acd10 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f2e5400e840 tx=0x7f2e5400ed00 comp rx=0 tx=0).stop 2024-01-15T18:12:11.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 shutdown_connections 2024-01-15T18:12:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2e6c15e030 0x7f2e6c145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2e4406c430 0x7f2e4406e8f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2e6c146650 0x7f2e6c0acd10 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 --2- 172.21.15.26:0/1760407288 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2e6c0726f0 0x7f2e6c15ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 >> 172.21.15.26:0/1760407288 conn(0x7f2e6c0b1d40 msgr2=0x7f2e6c0b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 shutdown_connections 2024-01-15T18:12:11.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.719+0000 7f2e72000640 1 -- 172.21.15.26:0/1760407288 wait complete. 2024-01-15T18:12:11.740 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:12:11.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.957+0000 7efde1400640 1 Processor -- start 2024-01-15T18:12:11.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.957+0000 7efde1400640 1 -- start start 2024-01-15T18:12:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.959+0000 7efde1400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13cfa0 0x7efddc15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.959+0000 7efde1400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7efddc13fcb0 0x7efddc0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.959+0000 7efde1400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc0cdec0 con 0x7efddc0cd530 2024-01-15T18:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.959+0000 7efde1400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0ce1a0 con 0x7efddc13fcb0 2024-01-15T18:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.959+0000 7efde1400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0ce480 con 0x7efddc13cfa0 2024-01-15T18:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.959+0000 7efddbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13cfa0 0x7efddc15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.959+0000 7efddbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13cfa0 0x7efddc15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:49458/0 (socket says 172.21.15.26:49458) 2024-01-15T18:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.959+0000 7efddbe00640 1 -- 172.21.15.26:0/1884685434 learned_addr learned my addr 172.21.15.26:0/1884685434 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.960+0000 7efde0a00640 1 --2- 172.21.15.26:0/1884685434 >> v2:172.21.15.124:3300/0 conn(0x7efddc13fcb0 0x7efddc0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.960+0000 7efddbe00640 1 -- 172.21.15.26:0/1884685434 >> v2:172.21.15.124:3300/0 conn(0x7efddc13fcb0 msgr2=0x7efddc0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.960+0000 7efddbe00640 1 --2- 172.21.15.26:0/1884685434 >> v2:172.21.15.124:3300/0 conn(0x7efddc13fcb0 0x7efddc0b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.960+0000 7efddbe00640 1 -- 172.21.15.26:0/1884685434 >> v1:172.21.15.124:6789/0 conn(0x7efddc0cd530 legacy=0x7efddc13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.960+0000 7efddbe00640 1 -- 172.21.15.26:0/1884685434 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0ce760 con 0x7efddc13cfa0 2024-01-15T18:12:11.964 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.962+0000 7efddbe00640 1 --2- 172.21.15.26:0/1884685434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13cfa0 0x7efddc15afb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009e50 tx=0x7efdcc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d49b1b66d3df0b58 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.962+0000 7efddb400640 1 -- 172.21.15.26:0/1884685434 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc02ca70 con 0x7efddc13cfa0 2024-01-15T18:12:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.962+0000 7efddb400640 1 -- 172.21.15.26:0/1884685434 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc02cbd0 con 0x7efddc13cfa0 2024-01-15T18:12:11.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.962+0000 7efddb400640 1 -- 172.21.15.26:0/1884685434 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0343f0 con 0x7efddc13cfa0 2024-01-15T18:12:11.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.963+0000 7efde1400640 1 -- 172.21.15.26:0/1884685434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13cfa0 msgr2=0x7efddc15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.963+0000 7efde1400640 1 --2- 172.21.15.26:0/1884685434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13cfa0 0x7efddc15afb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009e50 tx=0x7efdcc02bed0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.963+0000 7efde1400640 1 -- 172.21.15.26:0/1884685434 shutdown_connections 2024-01-15T18:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.963+0000 7efde1400640 1 --2- 172.21.15.26:0/1884685434 >> v2:172.21.15.124:3300/0 conn(0x7efddc13fcb0 0x7efddc0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.963+0000 7efde1400640 1 --2- 172.21.15.26:0/1884685434 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13cfa0 0x7efddc15afb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.963+0000 7efde1400640 1 -- 172.21.15.26:0/1884685434 >> 172.21.15.26:0/1884685434 conn(0x7efddc06a800 msgr2=0x7efddc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.963+0000 7efde1400640 1 -- 172.21.15.26:0/1884685434 shutdown_connections 2024-01-15T18:12:11.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.963+0000 7efde1400640 1 -- 172.21.15.26:0/1884685434 wait complete. 2024-01-15T18:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.965+0000 7efde1400640 1 Processor -- start 2024-01-15T18:12:11.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.965+0000 7efde1400640 1 -- start start 2024-01-15T18:12:11.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.966+0000 7efde1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cd530 0x7efddc153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.966+0000 7efde1400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efddc13cfa0 0x7efddc13ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.966+0000 7efde1400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13fcb0 0x7efddc0ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.966+0000 7efde1400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc154180 con 0x7efddc0cd530 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.966+0000 7efde1400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc156db0 con 0x7efddc13cfa0 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.966+0000 7efde1400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc157090 con 0x7efddc13fcb0 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.966+0000 7efddbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cd530 0x7efddc153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.966+0000 7efddbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cd530 0x7efddc153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34628/0 (socket says 172.21.15.26:34628) 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.966+0000 7efddbe00640 1 -- 172.21.15.26:0/2849796848 learned_addr learned my addr 172.21.15.26:0/2849796848 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efde0a00640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13fcb0 0x7efddc0ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efdd3200640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efddc13cfa0 0x7efddc13ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efddbe00640 1 -- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13fcb0 msgr2=0x7efddc0ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efddbe00640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13fcb0 0x7efddc0ad8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efddbe00640 1 -- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efddc13cfa0 msgr2=0x7efddc13ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efddbe00640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efddc13cfa0 0x7efddc13ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efddbe00640 1 -- 172.21.15.26:0/2849796848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc009b30 con 0x7efddc0cd530 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efde0a00640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13fcb0 0x7efddc0ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efdd3200640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efddc13cfa0 0x7efddc13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efddbe00640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cd530 0x7efddc153c40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7efdcc000f80 tx=0x7efdcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.967+0000 7efdd8c00640 1 -- 172.21.15.26:0/2849796848 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc009510 con 0x7efddc0cd530 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.968+0000 7efdd8c00640 1 -- 172.21.15.26:0/2849796848 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc03e450 con 0x7efddc0cd530 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.968+0000 7efdd8c00640 1 -- 172.21.15.26:0/2849796848 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc03e800 con 0x7efddc0cd530 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.968+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc157370 con 0x7efddc0cd530 2024-01-15T18:12:11.973 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.968+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc157930 con 0x7efddc0cd530 2024-01-15T18:12:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.969+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc0cc8a0 con 0x7efddc0cd530 2024-01-15T18:12:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.970+0000 7efdd8c00640 1 -- 172.21.15.26:0/2849796848 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7efdcc03e960 con 0x7efddc0cd530 2024-01-15T18:12:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.971+0000 7efdd8c00640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdb406c430 0x7efdb406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.971+0000 7efdd8c00640 1 -- 172.21.15.26:0/2849796848 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5793+0+0 (secure 0 0 0) 0x7efdcc0ab890 con 0x7efddc0cd530 2024-01-15T18:12:11.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.972+0000 7efdd3200640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdb406c430 0x7efdb406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:11.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.973+0000 7efdd3200640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdb406c430 0x7efdb406e8f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efddc0b1800 tx=0x7efdc4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:11.978 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:11.975+0000 7efdd8c00640 1 -- 172.21.15.26:0/2849796848 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efdcc077ea0 con 0x7efddc0cd530 2024-01-15T18:12:12.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.174+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efddc067450 con 0x7efddc0cd530 2024-01-15T18:12:12.180 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.177+0000 7efdd8c00640 1 -- 172.21.15.26:0/2849796848 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16635 (secure 0 0 0) 0x7efdcc043090 con 0x7efddc0cd530 2024-01-15T18:12:12.181 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:12.181 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":115,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:12:12.014179+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:12:08.696115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:12:09.497434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:12:12.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdb406c430 msgr2=0x7efdb406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:12.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdb406c430 0x7efdb406e8f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efddc0b1800 tx=0x7efdc4009040 comp rx=0 tx=0).stop 2024-01-15T18:12:12.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cd530 msgr2=0x7efddc153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:12.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cd530 0x7efddc153c40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7efdcc000f80 tx=0x7efdcc004a40 comp rx=0 tx=0).stop 2024-01-15T18:12:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 shutdown_connections 2024-01-15T18:12:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efddc13fcb0 0x7efddc0ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdb406c430 0x7efdb406e8f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efddc13cfa0 0x7efddc13ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 --2- 172.21.15.26:0/2849796848 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efddc0cd530 0x7efddc153c40 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 >> 172.21.15.26:0/2849796848 conn(0x7efddc06a800 msgr2=0x7efddc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 shutdown_connections 2024-01-15T18:12:12.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.180+0000 7efde1400640 1 -- 172.21.15.26:0/2849796848 wait complete. 2024-01-15T18:12:12.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-15T18:12:12.422 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.419+0000 7f1fa0400640 1 Processor -- start 2024-01-15T18:12:12.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.419+0000 7f1fa0400640 1 -- start start 2024-01-15T18:12:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.421+0000 7f1fa0400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9813e360 0x7f1f980cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.421+0000 7f1fa0400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9815c5f0 con 0x7f1f9815f9c0 2024-01-15T18:12:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.421+0000 7f1fa0400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9806c470 con 0x7f1f9813df80 2024-01-15T18:12:12.424 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.421+0000 7f1fa0400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9806c750 con 0x7f1f9813e360 2024-01-15T18:12:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.422+0000 7f1f9e400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1f9815f9c0 0x7f1f980cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36466/0 (socket says 172.21.15.26:36466) 2024-01-15T18:12:12.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.422+0000 7f1f9e400640 1 -- 172.21.15.26:0/1805658399 learned_addr learned my addr 172.21.15.26:0/1805658399 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.422+0000 7f1f9d000640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9813e360 0x7f1f980cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.422+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892816995 0 0) 0x7f1f9815c5f0 con 0x7f1f9815f9c0 2024-01-15T18:12:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.422+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f78003520 con 0x7f1f9815f9c0 2024-01-15T18:12:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.422+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143101677 0 0) 0x7f1f9806c470 con 0x7f1f9813df80 2024-01-15T18:12:12.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f9815c5f0 con 0x7f1f9813df80 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3450439732 0 0) 0x7f1f78003520 con 0x7f1f9815f9c0 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9813e360 msgr2=0x7f1f980cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9813e360 0x7f1f980cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 >> v1:172.21.15.124:6789/0 conn(0x7f1f9813df80 legacy=0x7f1f9815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9806ca30 con 0x7f1f9815f9c0 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f1f900019f0 con 0x7f1f9815f9c0 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 >> v1:172.21.15.26:6789/0 conn(0x7f1f9815f9c0 legacy=0x7f1f980cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f78003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f780047c0 0x7f1f78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9813e360 0x7f1f78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9806c750 con 0x7f1f78003b30 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7800ba30 con 0x7f1f780047c0 2024-01-15T18:12:12.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7800bbf0 con 0x7f1f9813e360 2024-01-15T18:12:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f9da00640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f78003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f9e400640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9813e360 0x7f1f78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f9d000640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f780047c0 0x7f1f78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.423+0000 7f1f9da00640 1 -- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9813e360 msgr2=0x7f1f78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.424+0000 7f1f9da00640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9813e360 0x7f1f78005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.424+0000 7f1f9da00640 1 -- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f780047c0 msgr2=0x7f1f78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.424+0000 7f1f9da00640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f780047c0 0x7f1f78004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.424+0000 7f1f9da00640 1 -- 172.21.15.26:0/1805658399 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78003520 con 0x7f1f78003b30 2024-01-15T18:12:12.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.424+0000 7f1f9e400640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f9813e360 0x7f1f78005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.425+0000 7f1f9da00640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f78003f30 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1f880018a0 tx=0x7f1f780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f086a1dbebc22673 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.425+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880052d0 con 0x7f1f78003b30 2024-01-15T18:12:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.425+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f88005430 con 0x7f1f78003b30 2024-01-15T18:12:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.425+0000 7f1f97e00640 1 -- 172.21.15.26:0/1805658399 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88003810 con 0x7f1f78003b30 2024-01-15T18:12:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.425+0000 7f1fa0400640 1 -- 172.21.15.26:0/1805658399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 msgr2=0x7f1f78003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:12.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.425+0000 7f1fa0400640 1 --2- 172.21.15.26:0/1805658399 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f78003f30 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1f880018a0 tx=0x7f1f780057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.426+0000 7f1f9e400640 1 -- 172.21.15.26:0/1805658399 reap_dead start 2024-01-15T18:12:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.426+0000 7f1fa0400640 1 -- 172.21.15.26:0/1805658399 shutdown_connections 2024-01-15T18:12:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.426+0000 7f1fa0400640 1 -- 172.21.15.26:0/1805658399 >> 172.21.15.26:0/1805658399 conn(0x7f1f9806a800 msgr2=0x7f1f9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.426+0000 7f1fa0400640 1 -- 172.21.15.26:0/1805658399 shutdown_connections 2024-01-15T18:12:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.426+0000 7f1fa0400640 1 -- 172.21.15.26:0/1805658399 wait complete. 2024-01-15T18:12:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.428+0000 7f1fa0400640 1 Processor -- start 2024-01-15T18:12:12.431 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.428+0000 7f1fa0400640 1 -- start start 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.429+0000 7f1fa0400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f980a84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.429+0000 7f1fa0400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980adbe0 0x7f1f9813bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.429+0000 7f1fa0400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f981f7470 0x7f1f980cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.429+0000 7f1fa0400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f981f7050 con 0x7f1f78003b30 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.429+0000 7f1fa0400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9813d0d0 con 0x7f1f981f7470 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.429+0000 7f1fa0400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980ae3a0 con 0x7f1f980adbe0 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.429+0000 7f1f9da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f980a84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.429+0000 7f1f9da00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f980a84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34652/0 (socket says 172.21.15.26:34652) 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.429+0000 7f1f9da00640 1 -- 172.21.15.26:0/4133846540 learned_addr learned my addr 172.21.15.26:0/4133846540 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:12.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f9d000640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980adbe0 0x7f1f9813bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f9e400640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f981f7470 0x7f1f980cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f9da00640 1 -- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980adbe0 msgr2=0x7f1f9813bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f9da00640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980adbe0 0x7f1f9813bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f9da00640 1 -- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f981f7470 msgr2=0x7f1f980cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f9da00640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f981f7470 0x7f1f980cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f9da00640 1 -- 172.21.15.26:0/4133846540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f980a8a00 con 0x7f1f78003b30 2024-01-15T18:12:12.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f9da00640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f980a84c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1f88005040 tx=0x7f1f880012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f95600640 1 -- 172.21.15.26:0/4133846540 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88006ea0 con 0x7f1f78003b30 2024-01-15T18:12:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1f95600640 1 -- 172.21.15.26:0/4133846540 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f88007450 con 0x7f1f78003b30 2024-01-15T18:12:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.430+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f981f69d0 con 0x7f1f78003b30 2024-01-15T18:12:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.431+0000 7f1f95600640 1 -- 172.21.15.26:0/4133846540 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f880035f0 con 0x7f1f78003b30 2024-01-15T18:12:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.431+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f981f6c70 con 0x7f1f78003b30 2024-01-15T18:12:12.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.432+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f981f6830 con 0x7f1f78003b30 2024-01-15T18:12:12.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.433+0000 7f1f95600640 1 -- 172.21.15.26:0/4133846540 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f1f88003750 con 0x7f1f78003b30 2024-01-15T18:12:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.434+0000 7f1f95600640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006c430 0x7f1f7006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:12.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.434+0000 7f1f95600640 1 -- 172.21.15.26:0/4133846540 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f1f880aa590 con 0x7f1f78003b30 2024-01-15T18:12:12.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.435+0000 7f1f9d000640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006c430 0x7f1f7006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:12.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.436+0000 7f1f9d000640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006c430 0x7f1f7006e8f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1f98153e00 tx=0x7f1f80008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:12.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.439+0000 7f1f95600640 1 -- 172.21.15.26:0/4133846540 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f981f6830 con 0x7f1f78003b30 2024-01-15T18:12:12.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:12.673+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1f981f6830 con 0x7f1f78003b30 2024-01-15T18:12:13.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.297+0000 7f1f95600640 1 -- 172.21.15.26:0/4133846540 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v71) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1f981f6830 con 0x7f1f78003b30 2024-01-15T18:12:13.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006c430 msgr2=0x7f1f7006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:13.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006c430 0x7f1f7006e8f0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1f98153e00 tx=0x7f1f80008040 comp rx=0 tx=0).stop 2024-01-15T18:12:13.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 msgr2=0x7f1f980a84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:13.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f980a84c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1f88005040 tx=0x7f1f880012d0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 shutdown_connections 2024-01-15T18:12:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f1f980adbe0 0x7f1f9813bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f1f7006c430 0x7f1f7006e8f0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f1f981f7470 0x7f1f980cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 --2- 172.21.15.26:0/4133846540 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1f78003b30 0x7f1f980a84c0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 >> 172.21.15.26:0/4133846540 conn(0x7f1f9806a800 msgr2=0x7f1f9815c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 shutdown_connections 2024-01-15T18:12:13.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.301+0000 7f1fa0400640 1 -- 172.21.15.26:0/4133846540 wait complete. 2024-01-15T18:12:13.325 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-01-15T18:12:13.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-15T18:12:13.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.546+0000 7f5c8ec00640 1 Processor -- start 2024-01-15T18:12:13.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.546+0000 7f5c8ec00640 1 -- start start 2024-01-15T18:12:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.548+0000 7f5c8ec00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c880adf20 0x7f5c88070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.548+0000 7f5c8ec00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5c8815bf10 0x7f5c880cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.548+0000 7f5c8ec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c880cc980 0x7f5c88143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.548+0000 7f5c8ec00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8815f970 con 0x7f5c880adf20 2024-01-15T18:12:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.548+0000 7f5c8ec00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8815f4e0 con 0x7f5c8815bf10 2024-01-15T18:12:13.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.548+0000 7f5c8ec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c880cce10 con 0x7f5c880cc980 2024-01-15T18:12:13.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c880adf20 0x7f5c88070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:13.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5c880adf20 0x7f5c88070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34666/0 (socket says 172.21.15.26:34666) 2024-01-15T18:12:13.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8d800640 1 -- 172.21.15.26:0/2052835288 learned_addr learned my addr 172.21.15.26:0/2052835288 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8ce00640 1 --2- 172.21.15.26:0/2052835288 >> v2:172.21.15.124:3300/0 conn(0x7f5c8815bf10 0x7f5c880cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8e200640 1 --2- 172.21.15.26:0/2052835288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c880cc980 0x7f5c88143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8d800640 1 -- 172.21.15.26:0/2052835288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c880cc980 msgr2=0x7f5c88143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8d800640 1 --2- 172.21.15.26:0/2052835288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c880cc980 0x7f5c88143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8d800640 1 -- 172.21.15.26:0/2052835288 >> v2:172.21.15.124:3300/0 conn(0x7f5c8815bf10 msgr2=0x7f5c880cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8d800640 1 --2- 172.21.15.26:0/2052835288 >> v2:172.21.15.124:3300/0 conn(0x7f5c8815bf10 0x7f5c880cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.549+0000 7f5c8d800640 1 -- 172.21.15.26:0/2052835288 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c88071130 con 0x7f5c880adf20 2024-01-15T18:12:13.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.551+0000 7f5c8d800640 1 --2- 172.21.15.26:0/2052835288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880adf20 0x7f5c88070a30 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5c8815e9e0 tx=0x7f5c7802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345f0ddac791cb53 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:13.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.551+0000 7f5c87e00640 1 -- 172.21.15.26:0/2052835288 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7802c9e0 con 0x7f5c880adf20 2024-01-15T18:12:13.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.551+0000 7f5c87e00640 1 -- 172.21.15.26:0/2052835288 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7802cb40 con 0x7f5c880adf20 2024-01-15T18:12:13.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.551+0000 7f5c87e00640 1 -- 172.21.15.26:0/2052835288 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c780343f0 con 0x7f5c880adf20 2024-01-15T18:12:13.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.552+0000 7f5c8ec00640 1 -- 172.21.15.26:0/2052835288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880adf20 msgr2=0x7f5c88070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:13.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.552+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/2052835288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880adf20 0x7f5c88070a30 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f5c8815e9e0 tx=0x7f5c7802be40 comp rx=0 tx=0).stop 2024-01-15T18:12:13.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.552+0000 7f5c8ec00640 1 -- 172.21.15.26:0/2052835288 shutdown_connections 2024-01-15T18:12:13.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.552+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/2052835288 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c880cc980 0x7f5c88143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.552+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/2052835288 >> v2:172.21.15.124:3300/0 conn(0x7f5c8815bf10 0x7f5c880cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.552+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/2052835288 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880adf20 0x7f5c88070a30 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.552+0000 7f5c8ec00640 1 -- 172.21.15.26:0/2052835288 >> 172.21.15.26:0/2052835288 conn(0x7f5c8806a7d0 msgr2=0x7f5c8806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.552+0000 7f5c8ec00640 1 -- 172.21.15.26:0/2052835288 shutdown_connections 2024-01-15T18:12:13.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.552+0000 7f5c8ec00640 1 -- 172.21.15.26:0/2052835288 wait complete. 2024-01-15T18:12:13.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.555+0000 7f5c8ec00640 1 Processor -- start 2024-01-15T18:12:13.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.555+0000 7f5c8ec00640 1 -- start start 2024-01-15T18:12:13.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.556+0000 7f5c8ec00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c880adf20 0x7f5c8813ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:13.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.556+0000 7f5c8ec00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880cc980 0x7f5c8814c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:13.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.556+0000 7f5c8ec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c8815bf10 0x7f5c8815db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:13.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.556+0000 7f5c8ec00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881418c0 con 0x7f5c880cc980 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.556+0000 7f5c8ec00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815a8a0 con 0x7f5c880adf20 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.556+0000 7f5c8ec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8813f310 con 0x7f5c8815bf10 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.556+0000 7f5c8ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880cc980 0x7f5c8814c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.557+0000 7f5c8ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880cc980 0x7f5c8814c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34668/0 (socket says 172.21.15.26:34668) 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.557+0000 7f5c8ce00640 1 -- 172.21.15.26:0/1913226015 learned_addr learned my addr 172.21.15.26:0/1913226015 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.557+0000 7f5c8d800640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c880adf20 0x7f5c8813ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.557+0000 7f5c8e200640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c8815bf10 0x7f5c8815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.557+0000 7f5c8ce00640 1 -- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c8815bf10 msgr2=0x7f5c8815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.557+0000 7f5c8ce00640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c8815bf10 0x7f5c8815db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.557+0000 7f5c8ce00640 1 -- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c880adf20 msgr2=0x7f5c8813ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.557+0000 7f5c8ce00640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c880adf20 0x7f5c8813ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.557+0000 7f5c8ce00640 1 -- 172.21.15.26:0/1913226015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c78009b30 con 0x7f5c880cc980 2024-01-15T18:12:13.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.558+0000 7f5c8ce00640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880cc980 0x7f5c8814c550 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e9b0 tx=0x7f5c7000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:13.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.558+0000 7f5c85600640 1 -- 172.21.15.26:0/1913226015 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70009670 con 0x7f5c880cc980 2024-01-15T18:12:13.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.558+0000 7f5c85600640 1 -- 172.21.15.26:0/1913226015 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7000c9a0 con 0x7f5c880cc980 2024-01-15T18:12:13.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.558+0000 7f5c85600640 1 -- 172.21.15.26:0/1913226015 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000cd50 con 0x7f5c880cc980 2024-01-15T18:12:13.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.558+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c88141200 con 0x7f5c880cc980 2024-01-15T18:12:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.558+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c88140e20 con 0x7f5c880cc980 2024-01-15T18:12:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.559+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c88141040 con 0x7f5c880cc980 2024-01-15T18:12:13.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.560+0000 7f5c85600640 1 -- 172.21.15.26:0/1913226015 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f5c70013470 con 0x7f5c880cc980 2024-01-15T18:12:13.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.561+0000 7f5c85600640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c6006c430 0x7f5c6006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:13.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.561+0000 7f5c85600640 1 -- 172.21.15.26:0/1913226015 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f5c70088c70 con 0x7f5c880cc980 2024-01-15T18:12:13.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.562+0000 7f5c8d800640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c6006c430 0x7f5c6006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:13.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.563+0000 7f5c8d800640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c6006c430 0x7f5c6006e8f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5c78035000 tx=0x7f5c78030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:13.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.566+0000 7f5c85600640 1 -- 172.21.15.26:0/1913226015 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c70055290 con 0x7f5c880cc980 2024-01-15T18:12:13.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:13.800+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5c88067450 con 0x7f5c880cc980 2024-01-15T18:12:14.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.300+0000 7f5c85600640 1 -- 172.21.15.26:0/1913226015 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v72) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5c700170c0 con 0x7f5c880cc980 2024-01-15T18:12:14.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c6006c430 msgr2=0x7f5c6006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c6006c430 0x7f5c6006e8f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5c78035000 tx=0x7f5c78030000 comp rx=0 tx=0).stop 2024-01-15T18:12:14.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880cc980 msgr2=0x7f5c8814c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880cc980 0x7f5c8814c550 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f5c7000e9b0 tx=0x7f5c7000ee70 comp rx=0 tx=0).stop 2024-01-15T18:12:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 shutdown_connections 2024-01-15T18:12:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c8815bf10 0x7f5c8815db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c6006c430 0x7f5c6006e8f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c880cc980 0x7f5c8814c550 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 --2- 172.21.15.26:0/1913226015 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c880adf20 0x7f5c8813ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.303+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 >> 172.21.15.26:0/1913226015 conn(0x7f5c8806a7d0 msgr2=0x7f5c8806d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.304+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 shutdown_connections 2024-01-15T18:12:14.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.304+0000 7f5c8ec00640 1 -- 172.21.15.26:0/1913226015 wait complete. 2024-01-15T18:12:14.327 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-15T18:12:14.327 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:12:14.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.535+0000 7fec29a00640 1 Processor -- start 2024-01-15T18:12:14.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.536+0000 7fec29a00640 1 -- start start 2024-01-15T18:12:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.537+0000 7fec29a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec24160920 0x7fec240b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.537+0000 7fec29a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fec240cf9f0 0x7fec240cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.537+0000 7fec29a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec240d0a10 con 0x7fec24161010 2024-01-15T18:12:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.537+0000 7fec29a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240d03e0 con 0x7fec24160920 2024-01-15T18:12:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.537+0000 7fec29a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240d1f50 con 0x7fec240cf9f0 2024-01-15T18:12:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.538+0000 7fec23e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec24160920 0x7fec240b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.538+0000 7fec23e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec24160920 0x7fec240b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34672/0 (socket says 172.21.15.26:34672) 2024-01-15T18:12:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.538+0000 7fec23e00640 1 -- 172.21.15.26:0/1705061491 learned_addr learned my addr 172.21.15.26:0/1705061491 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.538+0000 7fec23e00640 1 -- 172.21.15.26:0/1705061491 >> v2:172.21.15.124:3300/0 conn(0x7fec240cf9f0 msgr2=0x7fec240cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:14.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.538+0000 7fec23e00640 1 --2- 172.21.15.26:0/1705061491 >> v2:172.21.15.124:3300/0 conn(0x7fec240cf9f0 0x7fec240cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.538+0000 7fec23e00640 1 -- 172.21.15.26:0/1705061491 >> v1:172.21.15.124:6789/0 conn(0x7fec24161010 legacy=0x7fec24161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:14.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.538+0000 7fec23e00640 1 -- 172.21.15.26:0/1705061491 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240cf080 con 0x7fec24160920 2024-01-15T18:12:14.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.539+0000 7fec23e00640 1 --2- 172.21.15.26:0/1705061491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24160920 0x7fec240b3f30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fec2408c1f0 tx=0x7fec1402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4aff404c4bee592 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:14.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.539+0000 7fec22a00640 1 -- 172.21.15.26:0/1705061491 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1402ca80 con 0x7fec24160920 2024-01-15T18:12:14.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.539+0000 7fec22a00640 1 -- 172.21.15.26:0/1705061491 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1402cbe0 con 0x7fec24160920 2024-01-15T18:12:14.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.539+0000 7fec22a00640 1 -- 172.21.15.26:0/1705061491 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec140343f0 con 0x7fec24160920 2024-01-15T18:12:14.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.540+0000 7fec29a00640 1 -- 172.21.15.26:0/1705061491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24160920 msgr2=0x7fec240b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.540+0000 7fec29a00640 1 --2- 172.21.15.26:0/1705061491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24160920 0x7fec240b3f30 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fec2408c1f0 tx=0x7fec1402bee0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.540+0000 7fec29a00640 1 -- 172.21.15.26:0/1705061491 shutdown_connections 2024-01-15T18:12:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.540+0000 7fec29a00640 1 --2- 172.21.15.26:0/1705061491 >> v2:172.21.15.124:3300/0 conn(0x7fec240cf9f0 0x7fec240cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.540+0000 7fec29a00640 1 --2- 172.21.15.26:0/1705061491 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24160920 0x7fec240b3f30 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.540+0000 7fec29a00640 1 -- 172.21.15.26:0/1705061491 >> 172.21.15.26:0/1705061491 conn(0x7fec240d3090 msgr2=0x7fec240d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.540+0000 7fec29a00640 1 -- 172.21.15.26:0/1705061491 shutdown_connections 2024-01-15T18:12:14.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.540+0000 7fec29a00640 1 -- 172.21.15.26:0/1705061491 wait complete. 2024-01-15T18:12:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.542+0000 7fec29a00640 1 Processor -- start 2024-01-15T18:12:14.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.542+0000 7fec29a00640 1 -- start start 2024-01-15T18:12:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.543+0000 7fec29a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec240cf9f0 0x7fec241f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.543+0000 7fec29a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec24160920 0x7fec241f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.543+0000 7fec29a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24161010 0x7fec240c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.543+0000 7fec29a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec241f3ff0 con 0x7fec24161010 2024-01-15T18:12:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.543+0000 7fec29a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec241f3520 con 0x7fec240cf9f0 2024-01-15T18:12:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.543+0000 7fec29a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec241f3750 con 0x7fec24160920 2024-01-15T18:12:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.543+0000 7fec29000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24161010 0x7fec240c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec29000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24161010 0x7fec240c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34674/0 (socket says 172.21.15.26:34674) 2024-01-15T18:12:14.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec29000640 1 -- 172.21.15.26:0/3631232157 learned_addr learned my addr 172.21.15.26:0/3631232157 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec23400640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec24160920 0x7fec241f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec23e00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec240cf9f0 0x7fec241f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec29000640 1 -- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec24160920 msgr2=0x7fec241f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec29000640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec24160920 0x7fec241f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec29000640 1 -- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec240cf9f0 msgr2=0x7fec241f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec29000640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec240cf9f0 0x7fec241f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec29000640 1 -- 172.21.15.26:0/3631232157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec14009b30 con 0x7fec24161010 2024-01-15T18:12:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec23e00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec240cf9f0 0x7fec241f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.544+0000 7fec29000640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24161010 0x7fec240c7c30 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00c390 tx=0x7fec1c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.545+0000 7fec1be00640 1 -- 172.21.15.26:0/3631232157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0035f0 con 0x7fec24161010 2024-01-15T18:12:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.545+0000 7fec1be00640 1 -- 172.21.15.26:0/3631232157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c003750 con 0x7fec24161010 2024-01-15T18:12:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.545+0000 7fec1be00640 1 -- 172.21.15.26:0/3631232157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c003b00 con 0x7fec24161010 2024-01-15T18:12:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.545+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec241f3a30 con 0x7fec24161010 2024-01-15T18:12:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.545+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec240c7710 con 0x7fec24161010 2024-01-15T18:12:14.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.546+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec240c7570 con 0x7fec24161010 2024-01-15T18:12:14.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.547+0000 7fec1be00640 1 -- 172.21.15.26:0/3631232157 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fec1c003c60 con 0x7fec24161010 2024-01-15T18:12:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.548+0000 7fec1be00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febfc06c430 0x7febfc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:14.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.548+0000 7fec1be00640 1 -- 172.21.15.26:0/3631232157 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fec1c087d60 con 0x7fec24161010 2024-01-15T18:12:14.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.549+0000 7fec23e00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febfc06c430 0x7febfc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:14.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.550+0000 7fec23e00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febfc06c430 0x7febfc06e8f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fec14035000 tx=0x7fec14030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:14.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.552+0000 7fec1be00640 1 -- 172.21.15.26:0/3631232157 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec240c7570 con 0x7fec24161010 2024-01-15T18:12:14.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.783+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec240c7570 con 0x7fec24161010 2024-01-15T18:12:14.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.784+0000 7fec1be00640 1 -- 172.21.15.26:0/3631232157 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fec240c7570 con 0x7fec24161010 2024-01-15T18:12:14.788 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:14.788 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/3480872940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3480872940},{"type":"v1","addr":"172.21.15.124:6835","nonce":3480872940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/662436785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":662436785},{"type":"v1","addr":"172.21.15.124:6837","nonce":662436785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":9222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2088544410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2088544410},{"type":"v1","addr":"172.21.15.26:6833","nonce":2088544410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:12:10.468680+0000","modified":"2024-01-15T18:12:13.801724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/483783473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":483783473},{"type":"v1","addr":"172.21.15.26:6835","nonce":483783473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-01-15T18:12:14.788 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 72 2024-01-15T18:12:14.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febfc06c430 msgr2=0x7febfc06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febfc06c430 0x7febfc06e8f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fec14035000 tx=0x7fec14030000 comp rx=0 tx=0).stop 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24161010 msgr2=0x7fec240c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24161010 0x7fec240c7c30 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00c390 tx=0x7fec1c00c850 comp rx=0 tx=0).stop 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 shutdown_connections 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec24160920 0x7fec241f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7febfc06c430 0x7febfc06e8f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec24161010 0x7fec240c7c30 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 --2- 172.21.15.26:0/3631232157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec240cf9f0 0x7fec241f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.787+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 >> 172.21.15.26:0/3631232157 conn(0x7fec240d3090 msgr2=0x7fec2406aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.788+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 shutdown_connections 2024-01-15T18:12:14.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:14.788+0000 7fec29a00640 1 -- 172.21.15.26:0/3631232157 wait complete. 2024-01-15T18:12:14.811 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:12:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.029+0000 7f3306600640 1 Processor -- start 2024-01-15T18:12:15.032 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.029+0000 7f3306600640 1 -- start start 2024-01-15T18:12:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.031+0000 7f3306600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3300143c40 0x7f3300071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.031+0000 7f3306600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33000717d0 0x7f33000cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.031+0000 7f3306600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000cca40 0x7f330015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.031+0000 7f3306600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33000723c0 con 0x7f33000717d0 2024-01-15T18:12:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.031+0000 7f3306600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3300070e50 con 0x7f3300143c40 2024-01-15T18:12:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.031+0000 7f3306600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33000cce40 con 0x7f33000cca40 2024-01-15T18:12:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.031+0000 7f32f7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33000717d0 0x7f33000cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.035 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.032+0000 7f32f7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f33000717d0 0x7f33000cc500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34688/0 (socket says 172.21.15.26:34688) 2024-01-15T18:12:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.032+0000 7f32f7e00640 1 -- 172.21.15.26:0/2991178384 learned_addr learned my addr 172.21.15.26:0/2991178384 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.032+0000 7f3305c00640 1 --2- 172.21.15.26:0/2991178384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000cca40 0x7f330015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.032+0000 7f3305200640 1 --2- 172.21.15.26:0/2991178384 >> v2:172.21.15.124:3300/0 conn(0x7f3300143c40 0x7f3300071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.032+0000 7f32f7e00640 1 -- 172.21.15.26:0/2991178384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000cca40 msgr2=0x7f330015eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.032+0000 7f32f7e00640 1 --2- 172.21.15.26:0/2991178384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000cca40 0x7f330015eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.032+0000 7f32f7e00640 1 -- 172.21.15.26:0/2991178384 >> v2:172.21.15.124:3300/0 conn(0x7f3300143c40 msgr2=0x7f3300071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.032+0000 7f32f7e00640 1 --2- 172.21.15.26:0/2991178384 >> v2:172.21.15.124:3300/0 conn(0x7f3300143c40 0x7f3300071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.032+0000 7f32f7e00640 1 -- 172.21.15.26:0/2991178384 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3300144020 con 0x7f33000717d0 2024-01-15T18:12:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.033+0000 7f32f7e00640 1 --2- 172.21.15.26:0/2991178384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000717d0 0x7f33000cc500 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f330006e410 tx=0x7f32e802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43aea31b6df342d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.034+0000 7f32ffe00640 1 -- 172.21.15.26:0/2991178384 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e80317e0 con 0x7f33000717d0 2024-01-15T18:12:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.034+0000 7f32ffe00640 1 -- 172.21.15.26:0/2991178384 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8031940 con 0x7f33000717d0 2024-01-15T18:12:15.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.034+0000 7f32ffe00640 1 -- 172.21.15.26:0/2991178384 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8031cd0 con 0x7f33000717d0 2024-01-15T18:12:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.034+0000 7f3306600640 1 -- 172.21.15.26:0/2991178384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000717d0 msgr2=0x7f33000cc500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.034+0000 7f3306600640 1 --2- 172.21.15.26:0/2991178384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000717d0 0x7f33000cc500 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f330006e410 tx=0x7f32e802b880 comp rx=0 tx=0).stop 2024-01-15T18:12:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.035+0000 7f3306600640 1 -- 172.21.15.26:0/2991178384 shutdown_connections 2024-01-15T18:12:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.035+0000 7f3306600640 1 --2- 172.21.15.26:0/2991178384 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000cca40 0x7f330015eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.035+0000 7f3306600640 1 --2- 172.21.15.26:0/2991178384 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000717d0 0x7f33000cc500 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.035+0000 7f3306600640 1 --2- 172.21.15.26:0/2991178384 >> v2:172.21.15.124:3300/0 conn(0x7f3300143c40 0x7f3300071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.035+0000 7f3306600640 1 -- 172.21.15.26:0/2991178384 >> 172.21.15.26:0/2991178384 conn(0x7f330006a800 msgr2=0x7f330006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.035+0000 7f3306600640 1 -- 172.21.15.26:0/2991178384 shutdown_connections 2024-01-15T18:12:15.038 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.035+0000 7f3306600640 1 -- 172.21.15.26:0/2991178384 wait complete. 2024-01-15T18:12:15.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.037+0000 7f3306600640 1 Processor -- start 2024-01-15T18:12:15.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.037+0000 7f3306600640 1 -- start start 2024-01-15T18:12:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f3306600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000717d0 0x7f33000b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f3306600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000cca40 0x7f330013df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f3306600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3300143c40 0x7f330013e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f3306600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33000b3360 con 0x7f33000cca40 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f3306600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33000b37f0 con 0x7f3300143c40 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f3306600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330013e9f0 con 0x7f33000717d0 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000cca40 0x7f330013df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f32f7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000cca40 0x7f330013df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34700/0 (socket says 172.21.15.26:34700) 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f32f7e00640 1 -- 172.21.15.26:0/3266979784 learned_addr learned my addr 172.21.15.26:0/3266979784 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f3305200640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000717d0 0x7f33000b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.038+0000 7f3305c00640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3300143c40 0x7f330013e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f32f7e00640 1 -- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000717d0 msgr2=0x7f33000b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f32f7e00640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000717d0 0x7f33000b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f32f7e00640 1 -- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3300143c40 msgr2=0x7f330013e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f32f7e00640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3300143c40 0x7f330013e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f32f7e00640 1 -- 172.21.15.26:0/3266979784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e80094d0 con 0x7f33000cca40 2024-01-15T18:12:15.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f3305200640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000717d0 0x7f33000b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f32f7e00640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000cca40 0x7f330013df70 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000c00 tx=0x7f32e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f32fd600640 1 -- 172.21.15.26:0/3266979784 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8032880 con 0x7f33000cca40 2024-01-15T18:12:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f32fd600640 1 -- 172.21.15.26:0/3266979784 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8032a20 con 0x7f33000cca40 2024-01-15T18:12:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33000b2750 con 0x7f33000cca40 2024-01-15T18:12:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.039+0000 7f32fd600640 1 -- 172.21.15.26:0/3266979784 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8032dd0 con 0x7f33000cca40 2024-01-15T18:12:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.040+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33000b2d10 con 0x7f33000cca40 2024-01-15T18:12:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.041+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33000b34d0 con 0x7f33000cca40 2024-01-15T18:12:15.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.041+0000 7f32fd600640 1 -- 172.21.15.26:0/3266979784 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f32e804e450 con 0x7f33000cca40 2024-01-15T18:12:15.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.043+0000 7f32fd600640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32d806c430 0x7f32d806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.043+0000 7f32fd600640 1 -- 172.21.15.26:0/3266979784 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f32e80aac00 con 0x7f33000cca40 2024-01-15T18:12:15.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.044+0000 7f3305200640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32d806c430 0x7f32d806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.045+0000 7f3305200640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32d806c430 0x7f32d806e8f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f32f0006fd0 tx=0x7f32f0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:15.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.047+0000 7f32fd600640 1 -- 172.21.15.26:0/3266979784 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32e80772e0 con 0x7f33000cca40 2024-01-15T18:12:15.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.241+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3300067450 con 0x7f33000cca40 2024-01-15T18:12:15.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.243+0000 7f32fd600640 1 -- 172.21.15.26:0/3266979784 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16635 (secure 0 0 0) 0x7f32e807a900 con 0x7f33000cca40 2024-01-15T18:12:15.247 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:15.247 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":115,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:12:12.014179+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:12:08.696115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:12:09.497434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:12:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32d806c430 msgr2=0x7f32d806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32d806c430 0x7f32d806e8f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f32f0006fd0 tx=0x7f32f0008040 comp rx=0 tx=0).stop 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000cca40 msgr2=0x7f330013df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000cca40 0x7f330013df70 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f32e8000c00 tx=0x7f32e80046c0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 shutdown_connections 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f32d806c430 0x7f32d806e8f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3300143c40 0x7f330013e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f33000cca40 0x7f330013df70 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 --2- 172.21.15.26:0/3266979784 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f33000717d0 0x7f33000b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 >> 172.21.15.26:0/3266979784 conn(0x7f330006a800 msgr2=0x7f33001407c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 shutdown_connections 2024-01-15T18:12:15.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.246+0000 7f3306600640 1 -- 172.21.15.26:0/3266979784 wait complete. 2024-01-15T18:12:15.267 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-15T18:12:15.267 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:12:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.481+0000 7fb5dd600640 1 Processor -- start 2024-01-15T18:12:15.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.481+0000 7fb5dd600640 1 -- start start 2024-01-15T18:12:15.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.483+0000 7fb5dd600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb5d80cc500 0x7fb5d80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.483+0000 7fb5dd600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d806d160 0x7fb5d806d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.483+0000 7fb5dd600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d8070e50 con 0x7fb5d8143c40 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.483+0000 7fb5dd600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80723c0 con 0x7fb5d80cc500 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.483+0000 7fb5dd600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80cced0 con 0x7fb5d806d160 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb5d8143c40 0x7fb5d8071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36474/0 (socket says 172.21.15.26:36474) 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7e00640 1 -- 172.21.15.26:0/2807627797 learned_addr learned my addr 172.21.15.26:0/2807627797 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5dcc00640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d806d160 0x7fb5d806d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991248406 0 0) 0x7fb5d8070e50 con 0x7fb5d8143c40 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b8003520 con 0x7fb5d8143c40 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5cf200640 1 --2- 172.21.15.26:0/2807627797 >> v2:172.21.15.124:3300/0 conn(0x7fb5d80cc500 0x7fb5d80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1834702792 0 0) 0x7fb5b8003520 con 0x7fb5d8143c40 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d806d160 msgr2=0x7fb5d806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7400640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d806d160 0x7fb5d806d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 >> v2:172.21.15.124:3300/0 conn(0x7fb5d80cc500 msgr2=0x7fb5d80cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7400640 1 --2- 172.21.15.26:0/2807627797 >> v2:172.21.15.124:3300/0 conn(0x7fb5d80cc500 0x7fb5d80cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8144020 con 0x7fb5d8143c40 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.484+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb5c8001690 con 0x7fb5d8143c40 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 >> v1:172.21.15.26:6789/0 conn(0x7fb5d8143c40 legacy=0x7fb5d8071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5d7400640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b8003970 0x7fb5b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5d7400640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5b80047c0 0x7fb5b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5d7400640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d806d160 0x7fb5b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80723c0 con 0x7fb5b80047c0 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80cced0 con 0x7fb5b8003970 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b800ba60 con 0x7fb5d806d160 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5cf200640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5b80047c0 0x7fb5b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5d7e00640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b8003970 0x7fb5b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5dcc00640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d806d160 0x7fb5b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5cf200640 1 -- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d806d160 msgr2=0x7fb5b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5cf200640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d806d160 0x7fb5b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5cf200640 1 -- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b8003970 msgr2=0x7fb5b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5cf200640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b8003970 0x7fb5b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.485+0000 7fb5cf200640 1 -- 172.21.15.26:0/2807627797 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b8003520 con 0x7fb5b80047c0 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.487+0000 7fb5cf200640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5b80047c0 0x7fb5b8004c40 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000ba00 tx=0x7fb5b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a61de40f52a27d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.487+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c002d040 con 0x7fb5b80047c0 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.487+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c0033610 con 0x7fb5b80047c0 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.487+0000 7fb5d7400640 1 -- 172.21.15.26:0/2807627797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00339a0 con 0x7fb5b80047c0 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.487+0000 7fb5dd600640 1 -- 172.21.15.26:0/2807627797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5b80047c0 msgr2=0x7fb5b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.487+0000 7fb5dd600640 1 --2- 172.21.15.26:0/2807627797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5b80047c0 0x7fb5b8004c40 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000ba00 tx=0x7fb5b80057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.487+0000 7fb5dcc00640 1 -- 172.21.15.26:0/2807627797 reap_dead start 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.487+0000 7fb5dd600640 1 -- 172.21.15.26:0/2807627797 shutdown_connections 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.487+0000 7fb5dd600640 1 -- 172.21.15.26:0/2807627797 >> 172.21.15.26:0/2807627797 conn(0x7fb5d806a800 msgr2=0x7fb5d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.488+0000 7fb5dd600640 1 -- 172.21.15.26:0/2807627797 shutdown_connections 2024-01-15T18:12:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.488+0000 7fb5dd600640 1 -- 172.21.15.26:0/2807627797 wait complete. 2024-01-15T18:12:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.489+0000 7fb5dd600640 1 Processor -- start 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.489+0000 7fb5dd600640 1 -- start start 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.490+0000 7fb5dd600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b80047c0 0x7fb5d81426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.490+0000 7fb5dd600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5d815a370 0x7fb5d815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.490+0000 7fb5dd600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d812b400 0x7fb5d812b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.490+0000 7fb5dd600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d812c150 con 0x7fb5d815a370 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.490+0000 7fb5dd600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815ad30 con 0x7fb5b80047c0 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.490+0000 7fb5dd600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d8129270 con 0x7fb5d812b400 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5cf200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5d815a370 0x7fb5d815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5cf200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5d815a370 0x7fb5d815a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34726/0 (socket says 172.21.15.26:34726) 2024-01-15T18:12:15.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5cf200640 1 -- 172.21.15.26:0/1240046904 learned_addr learned my addr 172.21.15.26:0/1240046904 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5d7e00640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b80047c0 0x7fb5d81426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5dcc00640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d812b400 0x7fb5d812b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5cf200640 1 -- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d812b400 msgr2=0x7fb5d812b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5cf200640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d812b400 0x7fb5d812b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5cf200640 1 -- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b80047c0 msgr2=0x7fb5d81426f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5cf200640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b80047c0 0x7fb5d81426f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5cf200640 1 -- 172.21.15.26:0/1240046904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8129550 con 0x7fb5d815a370 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5d7e00640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b80047c0 0x7fb5d81426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.491+0000 7fb5cf200640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5d815a370 0x7fb5d815a7f0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fb5c002bdf0 tx=0x7fb5c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.492+0000 7fb5d4c00640 1 -- 172.21.15.26:0/1240046904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0004220 con 0x7fb5d815a370 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.492+0000 7fb5d4c00640 1 -- 172.21.15.26:0/1240046904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c0040070 con 0x7fb5d815a370 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.492+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d8129830 con 0x7fb5d815a370 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.492+0000 7fb5d4c00640 1 -- 172.21.15.26:0/1240046904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0003610 con 0x7fb5d815a370 2024-01-15T18:12:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.492+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d812bdc0 con 0x7fb5d815a370 2024-01-15T18:12:15.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.493+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d8067450 con 0x7fb5d815a370 2024-01-15T18:12:15.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.494+0000 7fb5d4c00640 1 -- 172.21.15.26:0/1240046904 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fb5c0003770 con 0x7fb5d815a370 2024-01-15T18:12:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.495+0000 7fb5d4c00640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb5b006c430 0x7fb5b006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.495+0000 7fb5d4c00640 1 -- 172.21.15.26:0/1240046904 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fb5c00aaaa0 con 0x7fb5d815a370 2024-01-15T18:12:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.496+0000 7fb5d7e00640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb5b006c430 0x7fb5b006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.497+0000 7fb5d7e00640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb5b006c430 0x7fb5b006e8f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80cbd00 tx=0x7fb5c8000ed0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:15.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.500+0000 7fb5d4c00640 1 -- 172.21.15.26:0/1240046904 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb5c0077070 con 0x7fb5d815a370 2024-01-15T18:12:15.695 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.692+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5d806c960 con 0x7fb5b006c430 2024-01-15T18:12:15.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.717+0000 7fb5d4c00640 1 -- 172.21.15.26:0/1240046904 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287317 (secure 0 0 0) 0x7fb5c8005e00 con 0x7fb5b006c430 2024-01-15T18:12:15.725 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:15.727 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:12:15.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.724+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb5b006c430 msgr2=0x7fb5b006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.724+0000 7fb5dd600640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb5b006c430 0x7fb5b006e8f0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80cbd00 tx=0x7fb5c8000ed0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.724+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5d815a370 msgr2=0x7fb5d815a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:15.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.724+0000 7fb5dd600640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5d815a370 0x7fb5d815a7f0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fb5c002bdf0 tx=0x7fb5c0004730 comp rx=0 tx=0).stop 2024-01-15T18:12:15.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.725+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 shutdown_connections 2024-01-15T18:12:15.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.725+0000 7fb5dd600640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb5d812b400 0x7fb5d812b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.725+0000 7fb5dd600640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb5b006c430 0x7fb5b006e8f0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.725+0000 7fb5dd600640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb5d815a370 0x7fb5d815a7f0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.725+0000 7fb5dd600640 1 --2- 172.21.15.26:0/1240046904 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb5b80047c0 0x7fb5d81426f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:15.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.725+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 >> 172.21.15.26:0/1240046904 conn(0x7fb5d806a800 msgr2=0x7fb5d8071290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:15.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.725+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 shutdown_connections 2024-01-15T18:12:15.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:15.725+0000 7fb5dd600640 1 -- 172.21.15.26:0/1240046904 wait complete. 2024-01-15T18:12:15.747 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":627,"stamp":"2024-01-15T18:12:14.314052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":536184,"kb_used_data":12276,"kb_used_omap":12,"kb_used_meta":523763,"kb_avail":754438536,"statfs":{"total":773094113280,"available":772545060864,"internally_reserved":0,"allocated":12570624,"data_stored":6122260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":536333872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.326351"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058977+0000","last_change":"2024-01-15T18:12:11.049832+0000","last_active":"2024-01-15T18:12:11.058977+0000","last_peered":"2024-01-15T18:12:11.058977+0000","last_clean":"2024-01-15T18:12:11.058977+0000","last_became_active":"2024-01-15T18:12:11.048782+0000","last_became_peered":"2024-01-15T18:12:11.048782+0000","last_unstale":"2024-01-15T18:12:11.058977+0000","last_undegraded":"2024-01-15T18:12:11.058977+0000","last_fullsized":"2024-01-15T18:12:11.058977+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:19:47.603862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027038+0000","last_change":"2024-01-15T18:12:11.053712+0000","last_active":"2024-01-15T18:12:12.027038+0000","last_peered":"2024-01-15T18:12:12.027038+0000","last_clean":"2024-01-15T18:12:12.027038+0000","last_became_active":"2024-01-15T18:12:11.053279+0000","last_became_peered":"2024-01-15T18:12:11.053279+0000","last_unstale":"2024-01-15T18:12:12.027038+0000","last_undegraded":"2024-01-15T18:12:12.027038+0000","last_fullsized":"2024-01-15T18:12:12.027038+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177100+0000","last_change":"2024-01-15T18:12:11.058625+0000","last_active":"2024-01-15T18:12:12.177100+0000","last_peered":"2024-01-15T18:12:12.177100+0000","last_clean":"2024-01-15T18:12:12.177100+0000","last_became_active":"2024-01-15T18:12:11.057543+0000","last_became_peered":"2024-01-15T18:12:11.057543+0000","last_unstale":"2024-01-15T18:12:12.177100+0000","last_undegraded":"2024-01-15T18:12:12.177100+0000","last_fullsized":"2024-01-15T18:12:12.177100+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:17.569977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058584+0000","last_change":"2024-01-15T18:12:11.047313+0000","last_active":"2024-01-15T18:12:11.058584+0000","last_peered":"2024-01-15T18:12:11.058584+0000","last_clean":"2024-01-15T18:12:11.058584+0000","last_became_active":"2024-01-15T18:12:11.047008+0000","last_became_peered":"2024-01-15T18:12:11.047008+0000","last_unstale":"2024-01-15T18:12:11.058584+0000","last_undegraded":"2024-01-15T18:12:11.058584+0000","last_fullsized":"2024-01-15T18:12:11.058584+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:38:56.937377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026041+0000","last_change":"2024-01-15T18:12:11.035567+0000","last_active":"2024-01-15T18:12:12.026041+0000","last_peered":"2024-01-15T18:12:12.026041+0000","last_clean":"2024-01-15T18:12:12.026041+0000","last_became_active":"2024-01-15T18:12:11.034909+0000","last_became_peered":"2024-01-15T18:12:11.034909+0000","last_unstale":"2024-01-15T18:12:12.026041+0000","last_undegraded":"2024-01-15T18:12:12.026041+0000","last_fullsized":"2024-01-15T18:12:12.026041+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:46.811591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058229+0000","last_change":"2024-01-15T18:12:11.055133+0000","last_active":"2024-01-15T18:12:11.058229+0000","last_peered":"2024-01-15T18:12:11.058229+0000","last_clean":"2024-01-15T18:12:11.058229+0000","last_became_active":"2024-01-15T18:12:11.054803+0000","last_became_peered":"2024-01-15T18:12:11.054803+0000","last_unstale":"2024-01-15T18:12:11.058229+0000","last_undegraded":"2024-01-15T18:12:11.058229+0000","last_fullsized":"2024-01-15T18:12:11.058229+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:09:48.700182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.048700+0000","last_change":"2024-01-15T18:12:11.045438+0000","last_active":"2024-01-15T18:12:11.048700+0000","last_peered":"2024-01-15T18:12:11.048700+0000","last_clean":"2024-01-15T18:12:11.048700+0000","last_became_active":"2024-01-15T18:12:11.044607+0000","last_became_peered":"2024-01-15T18:12:11.044607+0000","last_unstale":"2024-01-15T18:12:11.048700+0000","last_undegraded":"2024-01-15T18:12:11.048700+0000","last_fullsized":"2024-01-15T18:12:11.048700+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:50.244523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177198+0000","last_change":"2024-01-15T18:12:11.051056+0000","last_active":"2024-01-15T18:12:12.177198+0000","last_peered":"2024-01-15T18:12:12.177198+0000","last_clean":"2024-01-15T18:12:12.177198+0000","last_became_active":"2024-01-15T18:12:11.050697+0000","last_became_peered":"2024-01-15T18:12:11.050697+0000","last_unstale":"2024-01-15T18:12:12.177198+0000","last_undegraded":"2024-01-15T18:12:12.177198+0000","last_fullsized":"2024-01-15T18:12:12.177198+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:56.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058955+0000","last_change":"2024-01-15T18:12:11.048227+0000","last_active":"2024-01-15T18:12:11.058955+0000","last_peered":"2024-01-15T18:12:11.058955+0000","last_clean":"2024-01-15T18:12:11.058955+0000","last_became_active":"2024-01-15T18:12:11.047693+0000","last_became_peered":"2024-01-15T18:12:11.047693+0000","last_unstale":"2024-01-15T18:12:11.058955+0000","last_undegraded":"2024-01-15T18:12:11.058955+0000","last_fullsized":"2024-01-15T18:12:11.058955+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:04.986605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.059982+0000","last_change":"2024-01-15T18:12:11.054486+0000","last_active":"2024-01-15T18:12:11.059982+0000","last_peered":"2024-01-15T18:12:11.059982+0000","last_clean":"2024-01-15T18:12:11.059982+0000","last_became_active":"2024-01-15T18:12:11.052429+0000","last_became_peered":"2024-01-15T18:12:11.052429+0000","last_unstale":"2024-01-15T18:12:11.059982+0000","last_undegraded":"2024-01-15T18:12:11.059982+0000","last_fullsized":"2024-01-15T18:12:11.059982+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027364+0000","last_change":"2024-01-15T18:12:11.043835+0000","last_active":"2024-01-15T18:12:12.027364+0000","last_peered":"2024-01-15T18:12:12.027364+0000","last_clean":"2024-01-15T18:12:12.027364+0000","last_became_active":"2024-01-15T18:12:11.042720+0000","last_became_peered":"2024-01-15T18:12:11.042720+0000","last_unstale":"2024-01-15T18:12:12.027364+0000","last_undegraded":"2024-01-15T18:12:12.027364+0000","last_fullsized":"2024-01-15T18:12:12.027364+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058563+0000","last_change":"2024-01-15T18:12:11.044468+0000","last_active":"2024-01-15T18:12:11.058563+0000","last_peered":"2024-01-15T18:12:11.058563+0000","last_clean":"2024-01-15T18:12:11.058563+0000","last_became_active":"2024-01-15T18:12:11.044155+0000","last_became_peered":"2024-01-15T18:12:11.044155+0000","last_unstale":"2024-01-15T18:12:11.058563+0000","last_undegraded":"2024-01-15T18:12:11.058563+0000","last_fullsized":"2024-01-15T18:12:11.058563+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177529+0000","last_change":"2024-01-15T18:12:11.052950+0000","last_active":"2024-01-15T18:12:12.177529+0000","last_peered":"2024-01-15T18:12:12.177529+0000","last_clean":"2024-01-15T18:12:12.177529+0000","last_became_active":"2024-01-15T18:12:11.051875+0000","last_became_peered":"2024-01-15T18:12:11.051875+0000","last_unstale":"2024-01-15T18:12:12.177529+0000","last_undegraded":"2024-01-15T18:12:12.177529+0000","last_fullsized":"2024-01-15T18:12:12.177529+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:22.849560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.176913+0000","last_change":"2024-01-15T18:12:11.056103+0000","last_active":"2024-01-15T18:12:12.176913+0000","last_peered":"2024-01-15T18:12:12.176913+0000","last_clean":"2024-01-15T18:12:12.176913+0000","last_became_active":"2024-01-15T18:12:11.055813+0000","last_became_peered":"2024-01-15T18:12:11.055813+0000","last_unstale":"2024-01-15T18:12:12.176913+0000","last_undegraded":"2024-01-15T18:12:12.176913+0000","last_fullsized":"2024-01-15T18:12:12.176913+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.025824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060652+0000","last_change":"2024-01-15T18:12:11.056863+0000","last_active":"2024-01-15T18:12:11.060652+0000","last_peered":"2024-01-15T18:12:11.060652+0000","last_clean":"2024-01-15T18:12:11.060652+0000","last_became_active":"2024-01-15T18:12:11.055217+0000","last_became_peered":"2024-01-15T18:12:11.055217+0000","last_unstale":"2024-01-15T18:12:11.060652+0000","last_undegraded":"2024-01-15T18:12:11.060652+0000","last_fullsized":"2024-01-15T18:12:11.060652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:18:05.495499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060095+0000","last_change":"2024-01-15T18:12:11.056227+0000","last_active":"2024-01-15T18:12:11.060095+0000","last_peered":"2024-01-15T18:12:11.060095+0000","last_clean":"2024-01-15T18:12:11.060095+0000","last_became_active":"2024-01-15T18:12:11.055919+0000","last_became_peered":"2024-01-15T18:12:11.055919+0000","last_unstale":"2024-01-15T18:12:11.060095+0000","last_undegraded":"2024-01-15T18:12:11.060095+0000","last_fullsized":"2024-01-15T18:12:11.060095+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:12.967981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026929+0000","last_change":"2024-01-15T18:12:11.043060+0000","last_active":"2024-01-15T18:12:12.026929+0000","last_peered":"2024-01-15T18:12:12.026929+0000","last_clean":"2024-01-15T18:12:12.026929+0000","last_became_active":"2024-01-15T18:12:11.042031+0000","last_became_peered":"2024-01-15T18:12:11.042031+0000","last_unstale":"2024-01-15T18:12:12.026929+0000","last_undegraded":"2024-01-15T18:12:12.026929+0000","last_fullsized":"2024-01-15T18:12:12.026929+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:44.791297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.046437+0000","last_change":"2024-01-15T18:12:11.045284+0000","last_active":"2024-01-15T18:12:11.046437+0000","last_peered":"2024-01-15T18:12:11.046437+0000","last_clean":"2024-01-15T18:12:11.046437+0000","last_became_active":"2024-01-15T18:12:11.042814+0000","last_became_peered":"2024-01-15T18:12:11.042814+0000","last_unstale":"2024-01-15T18:12:11.046437+0000","last_undegraded":"2024-01-15T18:12:11.046437+0000","last_fullsized":"2024-01-15T18:12:11.046437+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:31:04.806413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177655+0000","last_change":"2024-01-15T18:12:11.055119+0000","last_active":"2024-01-15T18:12:12.177655+0000","last_peered":"2024-01-15T18:12:12.177655+0000","last_clean":"2024-01-15T18:12:12.177655+0000","last_became_active":"2024-01-15T18:12:11.054790+0000","last_became_peered":"2024-01-15T18:12:11.054790+0000","last_unstale":"2024-01-15T18:12:12.177655+0000","last_undegraded":"2024-01-15T18:12:12.177655+0000","last_fullsized":"2024-01-15T18:12:12.177655+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:17.513018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027544+0000","last_change":"2024-01-15T18:12:11.052755+0000","last_active":"2024-01-15T18:12:12.027544+0000","last_peered":"2024-01-15T18:12:12.027544+0000","last_clean":"2024-01-15T18:12:12.027544+0000","last_became_active":"2024-01-15T18:12:11.052289+0000","last_became_peered":"2024-01-15T18:12:11.052289+0000","last_unstale":"2024-01-15T18:12:12.027544+0000","last_undegraded":"2024-01-15T18:12:12.027544+0000","last_fullsized":"2024-01-15T18:12:12.027544+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:38:56.937377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026877+0000","last_change":"2024-01-15T18:12:11.036748+0000","last_active":"2024-01-15T18:12:12.026877+0000","last_peered":"2024-01-15T18:12:12.026877+0000","last_clean":"2024-01-15T18:12:12.026877+0000","last_became_active":"2024-01-15T18:12:11.036003+0000","last_became_peered":"2024-01-15T18:12:11.036003+0000","last_unstale":"2024-01-15T18:12:12.026877+0000","last_undegraded":"2024-01-15T18:12:12.026877+0000","last_fullsized":"2024-01-15T18:12:12.026877+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:20.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022120+0000","last_change":"2024-01-15T18:12:11.050445+0000","last_active":"2024-01-15T18:12:12.022120+0000","last_peered":"2024-01-15T18:12:12.022120+0000","last_clean":"2024-01-15T18:12:12.022120+0000","last_became_active":"2024-01-15T18:12:11.044093+0000","last_became_peered":"2024-01-15T18:12:11.044093+0000","last_unstale":"2024-01-15T18:12:12.022120+0000","last_undegraded":"2024-01-15T18:12:12.022120+0000","last_fullsized":"2024-01-15T18:12:12.022120+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:38:59.097623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026316+0000","last_change":"2024-01-15T18:12:11.053397+0000","last_active":"2024-01-15T18:12:12.026316+0000","last_peered":"2024-01-15T18:12:12.026316+0000","last_clean":"2024-01-15T18:12:12.026316+0000","last_became_active":"2024-01-15T18:12:11.052428+0000","last_became_peered":"2024-01-15T18:12:11.052428+0000","last_unstale":"2024-01-15T18:12:12.026316+0000","last_undegraded":"2024-01-15T18:12:12.026316+0000","last_fullsized":"2024-01-15T18:12:12.026316+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:07.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.021417+0000","last_change":"2024-01-15T18:12:11.051820+0000","last_active":"2024-01-15T18:12:12.021417+0000","last_peered":"2024-01-15T18:12:12.021417+0000","last_clean":"2024-01-15T18:12:12.021417+0000","last_became_active":"2024-01-15T18:12:11.051488+0000","last_became_peered":"2024-01-15T18:12:11.051488+0000","last_unstale":"2024-01-15T18:12:12.021417+0000","last_undegraded":"2024-01-15T18:12:12.021417+0000","last_fullsized":"2024-01-15T18:12:12.021417+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:45.480493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027143+0000","last_change":"2024-01-15T18:12:11.041672+0000","last_active":"2024-01-15T18:12:12.027143+0000","last_peered":"2024-01-15T18:12:12.027143+0000","last_clean":"2024-01-15T18:12:12.027143+0000","last_became_active":"2024-01-15T18:12:11.041266+0000","last_became_peered":"2024-01-15T18:12:11.041266+0000","last_unstale":"2024-01-15T18:12:12.027143+0000","last_undegraded":"2024-01-15T18:12:12.027143+0000","last_fullsized":"2024-01-15T18:12:12.027143+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058937+0000","last_change":"2024-01-15T18:12:11.048567+0000","last_active":"2024-01-15T18:12:11.058937+0000","last_peered":"2024-01-15T18:12:11.058937+0000","last_clean":"2024-01-15T18:12:11.058937+0000","last_became_active":"2024-01-15T18:12:11.048234+0000","last_became_peered":"2024-01-15T18:12:11.048234+0000","last_unstale":"2024-01-15T18:12:11.058937+0000","last_undegraded":"2024-01-15T18:12:11.058937+0000","last_fullsized":"2024-01-15T18:12:11.058937+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.025824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.063349+0000","last_change":"2024-01-15T18:12:11.062039+0000","last_active":"2024-01-15T18:12:11.063349+0000","last_peered":"2024-01-15T18:12:11.063349+0000","last_clean":"2024-01-15T18:12:11.063349+0000","last_became_active":"2024-01-15T18:12:11.061724+0000","last_became_peered":"2024-01-15T18:12:11.061724+0000","last_unstale":"2024-01-15T18:12:11.063349+0000","last_undegraded":"2024-01-15T18:12:11.063349+0000","last_fullsized":"2024-01-15T18:12:11.063349+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:55:59.367011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022870+0000","last_change":"2024-01-15T18:12:11.044940+0000","last_active":"2024-01-15T18:12:12.022870+0000","last_peered":"2024-01-15T18:12:12.022870+0000","last_clean":"2024-01-15T18:12:12.022870+0000","last_became_active":"2024-01-15T18:12:11.043801+0000","last_became_peered":"2024-01-15T18:12:11.043801+0000","last_unstale":"2024-01-15T18:12:12.022870+0000","last_undegraded":"2024-01-15T18:12:12.022870+0000","last_fullsized":"2024-01-15T18:12:12.022870+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:43.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026950+0000","last_change":"2024-01-15T18:12:11.048119+0000","last_active":"2024-01-15T18:12:12.026950+0000","last_peered":"2024-01-15T18:12:12.026950+0000","last_clean":"2024-01-15T18:12:12.026950+0000","last_became_active":"2024-01-15T18:12:11.047739+0000","last_became_peered":"2024-01-15T18:12:11.047739+0000","last_unstale":"2024-01-15T18:12:12.026950+0000","last_undegraded":"2024-01-15T18:12:12.026950+0000","last_fullsized":"2024-01-15T18:12:12.026950+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:49:08.557895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.045331+0000","last_change":"2024-01-15T18:12:11.040703+0000","last_active":"2024-01-15T18:12:11.045331+0000","last_peered":"2024-01-15T18:12:11.045331+0000","last_clean":"2024-01-15T18:12:11.045331+0000","last_became_active":"2024-01-15T18:12:11.039393+0000","last_became_peered":"2024-01-15T18:12:11.039393+0000","last_unstale":"2024-01-15T18:12:11.045331+0000","last_undegraded":"2024-01-15T18:12:11.045331+0000","last_fullsized":"2024-01-15T18:12:11.045331+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.728315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.061046+0000","last_change":"2024-01-15T18:12:11.059603+0000","last_active":"2024-01-15T18:12:11.061046+0000","last_peered":"2024-01-15T18:12:11.061046+0000","last_clean":"2024-01-15T18:12:11.061046+0000","last_became_active":"2024-01-15T18:12:11.059276+0000","last_became_peered":"2024-01-15T18:12:11.059276+0000","last_unstale":"2024-01-15T18:12:11.061046+0000","last_undegraded":"2024-01-15T18:12:11.061046+0000","last_fullsized":"2024-01-15T18:12:11.061046+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:14.826977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058182+0000","last_change":"2024-01-15T18:12:11.053438+0000","last_active":"2024-01-15T18:12:11.058182+0000","last_peered":"2024-01-15T18:12:11.058182+0000","last_clean":"2024-01-15T18:12:11.058182+0000","last_became_active":"2024-01-15T18:12:11.052655+0000","last_became_peered":"2024-01-15T18:12:11.052655+0000","last_unstale":"2024-01-15T18:12:11.058182+0000","last_undegraded":"2024-01-15T18:12:11.058182+0000","last_fullsized":"2024-01-15T18:12:11.058182+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:49.378542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058324+0000","last_change":"2024-01-15T18:12:11.053689+0000","last_active":"2024-01-15T18:12:11.058324+0000","last_peered":"2024-01-15T18:12:11.058324+0000","last_clean":"2024-01-15T18:12:11.058324+0000","last_became_active":"2024-01-15T18:12:11.053389+0000","last_became_peered":"2024-01-15T18:12:11.053389+0000","last_unstale":"2024-01-15T18:12:11.058324+0000","last_undegraded":"2024-01-15T18:12:11.058324+0000","last_fullsized":"2024-01-15T18:12:11.058324+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:10.104432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"114'4","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.293911+0000","last_change":"2024-01-15T18:12:10.021097+0000","last_active":"2024-01-15T18:12:11.293911+0000","last_peered":"2024-01-15T18:12:11.293911+0000","last_clean":"2024-01-15T18:12:11.293911+0000","last_became_active":"2024-01-15T18:12:10.020810+0000","last_became_peered":"2024-01-15T18:12:10.020810+0000","last_unstale":"2024-01-15T18:12:11.293911+0000","last_undegraded":"2024-01-15T18:12:11.293911+0000","last_fullsized":"2024-01-15T18:12:11.293911+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:09:27.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.045710+0000","last_change":"2024-01-15T18:12:11.043347+0000","last_active":"2024-01-15T18:12:11.045710+0000","last_peered":"2024-01-15T18:12:11.045710+0000","last_clean":"2024-01-15T18:12:11.045710+0000","last_became_active":"2024-01-15T18:12:11.042427+0000","last_became_peered":"2024-01-15T18:12:11.042427+0000","last_unstale":"2024-01-15T18:12:11.045710+0000","last_undegraded":"2024-01-15T18:12:11.045710+0000","last_fullsized":"2024-01-15T18:12:11.045710+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:14.617011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177516+0000","last_change":"2024-01-15T18:12:10.023175+0000","last_active":"2024-01-15T18:12:12.177516+0000","last_peered":"2024-01-15T18:12:12.177516+0000","last_clean":"2024-01-15T18:12:12.177516+0000","last_became_active":"2024-01-15T18:12:10.022654+0000","last_became_peered":"2024-01-15T18:12:10.022654+0000","last_unstale":"2024-01-15T18:12:12.177516+0000","last_undegraded":"2024-01-15T18:12:12.177516+0000","last_fullsized":"2024-01-15T18:12:12.177516+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:12.136270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058266+0000","last_change":"2024-01-15T18:12:11.053138+0000","last_active":"2024-01-15T18:12:11.058266+0000","last_peered":"2024-01-15T18:12:11.058266+0000","last_clean":"2024-01-15T18:12:11.058266+0000","last_became_active":"2024-01-15T18:12:11.052812+0000","last_became_peered":"2024-01-15T18:12:11.052812+0000","last_unstale":"2024-01-15T18:12:11.058266+0000","last_undegraded":"2024-01-15T18:12:11.058266+0000","last_fullsized":"2024-01-15T18:12:11.058266+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:25:16.609507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"114'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026133+0000","last_change":"2024-01-15T18:12:10.018995+0000","last_active":"2024-01-15T18:12:12.026133+0000","last_peered":"2024-01-15T18:12:12.026133+0000","last_clean":"2024-01-15T18:12:12.026133+0000","last_became_active":"2024-01-15T18:12:10.018670+0000","last_became_peered":"2024-01-15T18:12:10.018670+0000","last_unstale":"2024-01-15T18:12:12.026133+0000","last_undegraded":"2024-01-15T18:12:12.026133+0000","last_fullsized":"2024-01-15T18:12:12.026133+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:39.773731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177729+0000","last_change":"2024-01-15T18:12:11.059355+0000","last_active":"2024-01-15T18:12:12.177729+0000","last_peered":"2024-01-15T18:12:12.177729+0000","last_clean":"2024-01-15T18:12:12.177729+0000","last_became_active":"2024-01-15T18:12:11.057592+0000","last_became_peered":"2024-01-15T18:12:11.057592+0000","last_unstale":"2024-01-15T18:12:12.177729+0000","last_undegraded":"2024-01-15T18:12:12.177729+0000","last_fullsized":"2024-01-15T18:12:12.177729+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:14.826977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"114'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025582+0000","last_change":"2024-01-15T18:12:10.020565+0000","last_active":"2024-01-15T18:12:12.025582+0000","last_peered":"2024-01-15T18:12:12.025582+0000","last_clean":"2024-01-15T18:12:12.025582+0000","last_became_active":"2024-01-15T18:12:10.020143+0000","last_became_peered":"2024-01-15T18:12:10.020143+0000","last_unstale":"2024-01-15T18:12:12.025582+0000","last_undegraded":"2024-01-15T18:12:12.025582+0000","last_fullsized":"2024-01-15T18:12:12.025582+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:09.129175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060615+0000","last_change":"2024-01-15T18:12:11.057876+0000","last_active":"2024-01-15T18:12:11.060615+0000","last_peered":"2024-01-15T18:12:11.060615+0000","last_clean":"2024-01-15T18:12:11.060615+0000","last_became_active":"2024-01-15T18:12:11.057471+0000","last_became_peered":"2024-01-15T18:12:11.057471+0000","last_unstale":"2024-01-15T18:12:11.060615+0000","last_undegraded":"2024-01-15T18:12:11.060615+0000","last_fullsized":"2024-01-15T18:12:11.060615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:13.150070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.292871+0000","last_change":"2024-01-15T18:12:10.020217+0000","last_active":"2024-01-15T18:12:11.292871+0000","last_peered":"2024-01-15T18:12:11.292871+0000","last_clean":"2024-01-15T18:12:11.292871+0000","last_became_active":"2024-01-15T18:12:10.019818+0000","last_became_peered":"2024-01-15T18:12:10.019818+0000","last_unstale":"2024-01-15T18:12:11.292871+0000","last_undegraded":"2024-01-15T18:12:11.292871+0000","last_fullsized":"2024-01-15T18:12:11.292871+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027232+0000","last_change":"2024-01-15T18:12:11.050383+0000","last_active":"2024-01-15T18:12:12.027232+0000","last_peered":"2024-01-15T18:12:12.027232+0000","last_clean":"2024-01-15T18:12:12.027232+0000","last_became_active":"2024-01-15T18:12:11.050002+0000","last_became_peered":"2024-01-15T18:12:11.050002+0000","last_unstale":"2024-01-15T18:12:12.027232+0000","last_undegraded":"2024-01-15T18:12:12.027232+0000","last_fullsized":"2024-01-15T18:12:12.027232+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022221+0000","last_change":"2024-01-15T18:12:10.013940+0000","last_active":"2024-01-15T18:12:12.022221+0000","last_peered":"2024-01-15T18:12:12.022221+0000","last_clean":"2024-01-15T18:12:12.022221+0000","last_became_active":"2024-01-15T18:12:10.009676+0000","last_became_peered":"2024-01-15T18:12:10.009676+0000","last_unstale":"2024-01-15T18:12:12.022221+0000","last_undegraded":"2024-01-15T18:12:12.022221+0000","last_fullsized":"2024-01-15T18:12:12.022221+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:00.059047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060665+0000","last_change":"2024-01-15T18:12:11.057984+0000","last_active":"2024-01-15T18:12:11.060665+0000","last_peered":"2024-01-15T18:12:11.060665+0000","last_clean":"2024-01-15T18:12:11.060665+0000","last_became_active":"2024-01-15T18:12:11.057640+0000","last_became_peered":"2024-01-15T18:12:11.057640+0000","last_unstale":"2024-01-15T18:12:11.060665+0000","last_undegraded":"2024-01-15T18:12:11.060665+0000","last_fullsized":"2024-01-15T18:12:11.060665+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026300+0000","last_change":"2024-01-15T18:12:10.002836+0000","last_active":"2024-01-15T18:12:12.026300+0000","last_peered":"2024-01-15T18:12:12.026300+0000","last_clean":"2024-01-15T18:12:12.026300+0000","last_became_active":"2024-01-15T18:12:10.002190+0000","last_became_peered":"2024-01-15T18:12:10.002190+0000","last_unstale":"2024-01-15T18:12:12.026300+0000","last_undegraded":"2024-01-15T18:12:12.026300+0000","last_fullsized":"2024-01-15T18:12:12.026300+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:39.115790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.050296+0000","last_change":"2024-01-15T18:12:11.048560+0000","last_active":"2024-01-15T18:12:11.050296+0000","last_peered":"2024-01-15T18:12:11.050296+0000","last_clean":"2024-01-15T18:12:11.050296+0000","last_became_active":"2024-01-15T18:12:11.046934+0000","last_became_peered":"2024-01-15T18:12:11.046934+0000","last_unstale":"2024-01-15T18:12:11.050296+0000","last_undegraded":"2024-01-15T18:12:11.050296+0000","last_fullsized":"2024-01-15T18:12:11.050296+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:18:56.428372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.042867+0000","last_change":"2024-01-15T18:12:10.020864+0000","last_active":"2024-01-15T18:12:11.042867+0000","last_peered":"2024-01-15T18:12:11.042867+0000","last_clean":"2024-01-15T18:12:11.042867+0000","last_became_active":"2024-01-15T18:12:10.020577+0000","last_became_peered":"2024-01-15T18:12:10.020577+0000","last_unstale":"2024-01-15T18:12:11.042867+0000","last_undegraded":"2024-01-15T18:12:11.042867+0000","last_fullsized":"2024-01-15T18:12:11.042867+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:55.380272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.050337+0000","last_change":"2024-01-15T18:12:11.048003+0000","last_active":"2024-01-15T18:12:11.050337+0000","last_peered":"2024-01-15T18:12:11.050337+0000","last_clean":"2024-01-15T18:12:11.050337+0000","last_became_active":"2024-01-15T18:12:11.047704+0000","last_became_peered":"2024-01-15T18:12:11.047704+0000","last_unstale":"2024-01-15T18:12:11.050337+0000","last_undegraded":"2024-01-15T18:12:11.050337+0000","last_fullsized":"2024-01-15T18:12:11.050337+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:57.996851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025881+0000","last_change":"2024-01-15T18:12:10.020498+0000","last_active":"2024-01-15T18:12:12.025881+0000","last_peered":"2024-01-15T18:12:12.025881+0000","last_clean":"2024-01-15T18:12:12.025881+0000","last_became_active":"2024-01-15T18:12:10.020028+0000","last_became_peered":"2024-01-15T18:12:10.020028+0000","last_unstale":"2024-01-15T18:12:12.025881+0000","last_undegraded":"2024-01-15T18:12:12.025881+0000","last_fullsized":"2024-01-15T18:12:12.025881+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060685+0000","last_change":"2024-01-15T18:12:11.060166+0000","last_active":"2024-01-15T18:12:11.060685+0000","last_peered":"2024-01-15T18:12:11.060685+0000","last_clean":"2024-01-15T18:12:11.060685+0000","last_became_active":"2024-01-15T18:12:11.059393+0000","last_became_peered":"2024-01-15T18:12:11.059393+0000","last_unstale":"2024-01-15T18:12:11.060685+0000","last_undegraded":"2024-01-15T18:12:11.060685+0000","last_fullsized":"2024-01-15T18:12:11.060685+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:07.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.023271+0000","last_change":"2024-01-15T18:12:10.007879+0000","last_active":"2024-01-15T18:12:12.023271+0000","last_peered":"2024-01-15T18:12:12.023271+0000","last_clean":"2024-01-15T18:12:12.023271+0000","last_became_active":"2024-01-15T18:12:10.007549+0000","last_became_peered":"2024-01-15T18:12:10.007549+0000","last_unstale":"2024-01-15T18:12:12.023271+0000","last_undegraded":"2024-01-15T18:12:12.023271+0000","last_fullsized":"2024-01-15T18:12:12.023271+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:33.832426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.023020+0000","last_change":"2024-01-15T18:12:11.031666+0000","last_active":"2024-01-15T18:12:12.023020+0000","last_peered":"2024-01-15T18:12:12.023020+0000","last_clean":"2024-01-15T18:12:12.023020+0000","last_became_active":"2024-01-15T18:12:11.030958+0000","last_became_peered":"2024-01-15T18:12:11.030958+0000","last_unstale":"2024-01-15T18:12:12.023020+0000","last_undegraded":"2024-01-15T18:12:12.023020+0000","last_fullsized":"2024-01-15T18:12:12.023020+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:58.112590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"114'3","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.295048+0000","last_change":"2024-01-15T18:12:10.024809+0000","last_active":"2024-01-15T18:12:11.295048+0000","last_peered":"2024-01-15T18:12:11.295048+0000","last_clean":"2024-01-15T18:12:11.295048+0000","last_became_active":"2024-01-15T18:12:10.024120+0000","last_became_peered":"2024-01-15T18:12:10.024120+0000","last_unstale":"2024-01-15T18:12:11.295048+0000","last_undegraded":"2024-01-15T18:12:11.295048+0000","last_fullsized":"2024-01-15T18:12:11.295048+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:17:59.130803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058186+0000","last_change":"2024-01-15T18:12:11.055694+0000","last_active":"2024-01-15T18:12:11.058186+0000","last_peered":"2024-01-15T18:12:11.058186+0000","last_clean":"2024-01-15T18:12:11.058186+0000","last_became_active":"2024-01-15T18:12:11.055290+0000","last_became_peered":"2024-01-15T18:12:11.055290+0000","last_unstale":"2024-01-15T18:12:11.058186+0000","last_undegraded":"2024-01-15T18:12:11.058186+0000","last_fullsized":"2024-01-15T18:12:11.058186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:11.917423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"114'6","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.298411+0000","last_change":"2024-01-15T18:12:10.022950+0000","last_active":"2024-01-15T18:12:11.298411+0000","last_peered":"2024-01-15T18:12:11.298411+0000","last_clean":"2024-01-15T18:12:11.298411+0000","last_became_active":"2024-01-15T18:12:10.021497+0000","last_became_peered":"2024-01-15T18:12:10.021497+0000","last_unstale":"2024-01-15T18:12:11.298411+0000","last_undegraded":"2024-01-15T18:12:11.298411+0000","last_fullsized":"2024-01-15T18:12:11.298411+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:16.556177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022413+0000","last_change":"2024-01-15T18:12:11.046101+0000","last_active":"2024-01-15T18:12:12.022413+0000","last_peered":"2024-01-15T18:12:12.022413+0000","last_clean":"2024-01-15T18:12:12.022413+0000","last_became_active":"2024-01-15T18:12:11.045774+0000","last_became_peered":"2024-01-15T18:12:11.045774+0000","last_unstale":"2024-01-15T18:12:12.022413+0000","last_undegraded":"2024-01-15T18:12:12.022413+0000","last_fullsized":"2024-01-15T18:12:12.022413+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:41:11.078868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"114'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.292297+0000","last_change":"2024-01-15T18:12:10.016912+0000","last_active":"2024-01-15T18:12:11.292297+0000","last_peered":"2024-01-15T18:12:11.292297+0000","last_clean":"2024-01-15T18:12:11.292297+0000","last_became_active":"2024-01-15T18:12:10.016618+0000","last_became_peered":"2024-01-15T18:12:10.016618+0000","last_unstale":"2024-01-15T18:12:11.292297+0000","last_undegraded":"2024-01-15T18:12:11.292297+0000","last_fullsized":"2024-01-15T18:12:11.292297+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:40:48.957828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022585+0000","last_change":"2024-01-15T18:12:11.051223+0000","last_active":"2024-01-15T18:12:12.022585+0000","last_peered":"2024-01-15T18:12:12.022585+0000","last_clean":"2024-01-15T18:12:12.022585+0000","last_became_active":"2024-01-15T18:12:11.050928+0000","last_became_peered":"2024-01-15T18:12:11.050928+0000","last_unstale":"2024-01-15T18:12:12.022585+0000","last_undegraded":"2024-01-15T18:12:12.022585+0000","last_fullsized":"2024-01-15T18:12:12.022585+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:11.318479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.033672+0000","last_change":"2024-01-15T18:12:10.015868+0000","last_active":"2024-01-15T18:12:11.033672+0000","last_peered":"2024-01-15T18:12:11.033672+0000","last_clean":"2024-01-15T18:12:11.033672+0000","last_became_active":"2024-01-15T18:12:10.015578+0000","last_became_peered":"2024-01-15T18:12:10.015578+0000","last_unstale":"2024-01-15T18:12:11.033672+0000","last_undegraded":"2024-01-15T18:12:11.033672+0000","last_fullsized":"2024-01-15T18:12:11.033672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.021548+0000","last_change":"2024-01-15T18:12:11.029654+0000","last_active":"2024-01-15T18:12:12.021548+0000","last_peered":"2024-01-15T18:12:12.021548+0000","last_clean":"2024-01-15T18:12:12.021548+0000","last_became_active":"2024-01-15T18:12:11.028865+0000","last_became_peered":"2024-01-15T18:12:11.028865+0000","last_unstale":"2024-01-15T18:12:12.021548+0000","last_undegraded":"2024-01-15T18:12:12.021548+0000","last_fullsized":"2024-01-15T18:12:12.021548+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:29.536846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"114'8","reported_seq":20,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025285+0000","last_change":"2024-01-15T18:12:10.022690+0000","last_active":"2024-01-15T18:12:12.025285+0000","last_peered":"2024-01-15T18:12:12.025285+0000","last_clean":"2024-01-15T18:12:12.025285+0000","last_became_active":"2024-01-15T18:12:10.022398+0000","last_became_peered":"2024-01-15T18:12:10.022398+0000","last_unstale":"2024-01-15T18:12:12.025285+0000","last_undegraded":"2024-01-15T18:12:12.025285+0000","last_fullsized":"2024-01-15T18:12:12.025285+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:52.527066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026275+0000","last_change":"2024-01-15T18:12:11.045910+0000","last_active":"2024-01-15T18:12:12.026275+0000","last_peered":"2024-01-15T18:12:12.026275+0000","last_clean":"2024-01-15T18:12:12.026275+0000","last_became_active":"2024-01-15T18:12:11.043735+0000","last_became_peered":"2024-01-15T18:12:11.043735+0000","last_unstale":"2024-01-15T18:12:12.026275+0000","last_undegraded":"2024-01-15T18:12:12.026275+0000","last_fullsized":"2024-01-15T18:12:12.026275+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.728315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"114'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.291522+0000","last_change":"2024-01-15T18:12:10.027277+0000","last_active":"2024-01-15T18:12:11.291522+0000","last_peered":"2024-01-15T18:12:11.291522+0000","last_clean":"2024-01-15T18:12:11.291522+0000","last_became_active":"2024-01-15T18:12:10.026730+0000","last_became_peered":"2024-01-15T18:12:10.026730+0000","last_unstale":"2024-01-15T18:12:11.291522+0000","last_undegraded":"2024-01-15T18:12:11.291522+0000","last_fullsized":"2024-01-15T18:12:11.291522+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:52.737012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.063299+0000","last_change":"2024-01-15T18:12:11.061270+0000","last_active":"2024-01-15T18:12:11.063299+0000","last_peered":"2024-01-15T18:12:11.063299+0000","last_clean":"2024-01-15T18:12:11.063299+0000","last_became_active":"2024-01-15T18:12:11.061002+0000","last_became_peered":"2024-01-15T18:12:11.061002+0000","last_unstale":"2024-01-15T18:12:11.063299+0000","last_undegraded":"2024-01-15T18:12:11.063299+0000","last_fullsized":"2024-01-15T18:12:11.063299+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:45.296624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"114'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026152+0000","last_change":"2024-01-15T18:12:10.020526+0000","last_active":"2024-01-15T18:12:12.026152+0000","last_peered":"2024-01-15T18:12:12.026152+0000","last_clean":"2024-01-15T18:12:12.026152+0000","last_became_active":"2024-01-15T18:12:10.019706+0000","last_became_peered":"2024-01-15T18:12:10.019706+0000","last_unstale":"2024-01-15T18:12:12.026152+0000","last_undegraded":"2024-01-15T18:12:12.026152+0000","last_fullsized":"2024-01-15T18:12:12.026152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:49.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177357+0000","last_change":"2024-01-15T18:12:11.060480+0000","last_active":"2024-01-15T18:12:12.177357+0000","last_peered":"2024-01-15T18:12:12.177357+0000","last_clean":"2024-01-15T18:12:12.177357+0000","last_became_active":"2024-01-15T18:12:11.060154+0000","last_became_peered":"2024-01-15T18:12:11.060154+0000","last_unstale":"2024-01-15T18:12:12.177357+0000","last_undegraded":"2024-01-15T18:12:12.177357+0000","last_fullsized":"2024-01-15T18:12:12.177357+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:52.364460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"114'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.293549+0000","last_change":"2024-01-15T18:12:10.003308+0000","last_active":"2024-01-15T18:12:11.293549+0000","last_peered":"2024-01-15T18:12:11.293549+0000","last_clean":"2024-01-15T18:12:11.293549+0000","last_became_active":"2024-01-15T18:12:10.002957+0000","last_became_peered":"2024-01-15T18:12:10.002957+0000","last_unstale":"2024-01-15T18:12:11.293549+0000","last_undegraded":"2024-01-15T18:12:11.293549+0000","last_fullsized":"2024-01-15T18:12:11.293549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:43.777497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.178001+0000","last_change":"2024-01-15T18:12:11.059234+0000","last_active":"2024-01-15T18:12:12.178001+0000","last_peered":"2024-01-15T18:12:12.178001+0000","last_clean":"2024-01-15T18:12:12.178001+0000","last_became_active":"2024-01-15T18:12:11.058945+0000","last_became_peered":"2024-01-15T18:12:11.058945+0000","last_unstale":"2024-01-15T18:12:12.178001+0000","last_undegraded":"2024-01-15T18:12:12.178001+0000","last_fullsized":"2024-01-15T18:12:12.178001+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:37.113206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.036566+0000","last_change":"2024-01-15T18:12:10.028589+0000","last_active":"2024-01-15T18:12:11.036566+0000","last_peered":"2024-01-15T18:12:11.036566+0000","last_clean":"2024-01-15T18:12:11.036566+0000","last_became_active":"2024-01-15T18:12:10.028284+0000","last_became_peered":"2024-01-15T18:12:10.028284+0000","last_unstale":"2024-01-15T18:12:11.036566+0000","last_undegraded":"2024-01-15T18:12:11.036566+0000","last_fullsized":"2024-01-15T18:12:11.036566+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:46:23.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.176920+0000","last_change":"2024-01-15T18:12:11.051787+0000","last_active":"2024-01-15T18:12:12.176920+0000","last_peered":"2024-01-15T18:12:12.176920+0000","last_clean":"2024-01-15T18:12:12.176920+0000","last_became_active":"2024-01-15T18:12:11.051403+0000","last_became_peered":"2024-01-15T18:12:11.051403+0000","last_unstale":"2024-01-15T18:12:12.176920+0000","last_undegraded":"2024-01-15T18:12:12.176920+0000","last_fullsized":"2024-01-15T18:12:12.176920+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:05:35.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"114'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.296005+0000","last_change":"2024-01-15T18:12:10.021913+0000","last_active":"2024-01-15T18:12:11.296005+0000","last_peered":"2024-01-15T18:12:11.296005+0000","last_clean":"2024-01-15T18:12:11.296005+0000","last_became_active":"2024-01-15T18:12:10.021344+0000","last_became_peered":"2024-01-15T18:12:10.021344+0000","last_unstale":"2024-01-15T18:12:11.296005+0000","last_undegraded":"2024-01-15T18:12:11.296005+0000","last_fullsized":"2024-01-15T18:12:11.296005+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:00.107213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.061435+0000","last_change":"2024-01-15T18:12:11.059848+0000","last_active":"2024-01-15T18:12:11.061435+0000","last_peered":"2024-01-15T18:12:11.061435+0000","last_clean":"2024-01-15T18:12:11.061435+0000","last_became_active":"2024-01-15T18:12:11.059311+0000","last_became_peered":"2024-01-15T18:12:11.059311+0000","last_unstale":"2024-01-15T18:12:11.061435+0000","last_undegraded":"2024-01-15T18:12:11.061435+0000","last_fullsized":"2024-01-15T18:12:11.061435+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:37.113206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025456+0000","last_change":"2024-01-15T18:12:10.018301+0000","last_active":"2024-01-15T18:12:12.025456+0000","last_peered":"2024-01-15T18:12:12.025456+0000","last_clean":"2024-01-15T18:12:12.025456+0000","last_became_active":"2024-01-15T18:12:10.017411+0000","last_became_peered":"2024-01-15T18:12:10.017411+0000","last_unstale":"2024-01-15T18:12:12.025456+0000","last_undegraded":"2024-01-15T18:12:12.025456+0000","last_fullsized":"2024-01-15T18:12:12.025456+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:32.688781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177614+0000","last_change":"2024-01-15T18:12:11.056765+0000","last_active":"2024-01-15T18:12:12.177614+0000","last_peered":"2024-01-15T18:12:12.177614+0000","last_clean":"2024-01-15T18:12:12.177614+0000","last_became_active":"2024-01-15T18:12:11.056311+0000","last_became_peered":"2024-01-15T18:12:11.056311+0000","last_unstale":"2024-01-15T18:12:12.177614+0000","last_undegraded":"2024-01-15T18:12:12.177614+0000","last_fullsized":"2024-01-15T18:12:12.177614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:53.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.042167+0000","last_change":"2024-01-15T18:12:10.023638+0000","last_active":"2024-01-15T18:12:11.042167+0000","last_peered":"2024-01-15T18:12:11.042167+0000","last_clean":"2024-01-15T18:12:11.042167+0000","last_became_active":"2024-01-15T18:12:10.022687+0000","last_became_peered":"2024-01-15T18:12:10.022687+0000","last_unstale":"2024-01-15T18:12:11.042167+0000","last_undegraded":"2024-01-15T18:12:11.042167+0000","last_fullsized":"2024-01-15T18:12:11.042167+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:54.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.021795+0000","last_change":"2024-01-15T18:12:11.062171+0000","last_active":"2024-01-15T18:12:12.021795+0000","last_peered":"2024-01-15T18:12:12.021795+0000","last_clean":"2024-01-15T18:12:12.021795+0000","last_became_active":"2024-01-15T18:12:11.061830+0000","last_became_peered":"2024-01-15T18:12:11.061830+0000","last_unstale":"2024-01-15T18:12:12.021795+0000","last_undegraded":"2024-01-15T18:12:12.021795+0000","last_fullsized":"2024-01-15T18:12:12.021795+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:50.541819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.043037+0000","last_change":"2024-01-15T18:12:10.024893+0000","last_active":"2024-01-15T18:12:11.043037+0000","last_peered":"2024-01-15T18:12:11.043037+0000","last_clean":"2024-01-15T18:12:11.043037+0000","last_became_active":"2024-01-15T18:12:10.024554+0000","last_became_peered":"2024-01-15T18:12:10.024554+0000","last_unstale":"2024-01-15T18:12:11.043037+0000","last_undegraded":"2024-01-15T18:12:11.043037+0000","last_fullsized":"2024-01-15T18:12:11.043037+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:51.350660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.055716+0000","last_change":"2024-01-15T18:12:11.051179+0000","last_active":"2024-01-15T18:12:11.055716+0000","last_peered":"2024-01-15T18:12:11.055716+0000","last_clean":"2024-01-15T18:12:11.055716+0000","last_became_active":"2024-01-15T18:12:11.050747+0000","last_became_peered":"2024-01-15T18:12:11.050747+0000","last_unstale":"2024-01-15T18:12:11.055716+0000","last_undegraded":"2024-01-15T18:12:11.055716+0000","last_fullsized":"2024-01-15T18:12:11.055716+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:40.129590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"114'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.023121+0000","last_change":"2024-01-15T18:12:10.025674+0000","last_active":"2024-01-15T18:12:12.023121+0000","last_peered":"2024-01-15T18:12:12.023121+0000","last_clean":"2024-01-15T18:12:12.023121+0000","last_became_active":"2024-01-15T18:12:10.025356+0000","last_became_peered":"2024-01-15T18:12:10.025356+0000","last_unstale":"2024-01-15T18:12:12.023121+0000","last_undegraded":"2024-01-15T18:12:12.023121+0000","last_fullsized":"2024-01-15T18:12:12.023121+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:06:50.285646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058998+0000","last_change":"2024-01-15T18:12:11.050465+0000","last_active":"2024-01-15T18:12:11.058998+0000","last_peered":"2024-01-15T18:12:11.058998+0000","last_clean":"2024-01-15T18:12:11.058998+0000","last_became_active":"2024-01-15T18:12:11.050116+0000","last_became_peered":"2024-01-15T18:12:11.050116+0000","last_unstale":"2024-01-15T18:12:11.058998+0000","last_undegraded":"2024-01-15T18:12:11.058998+0000","last_fullsized":"2024-01-15T18:12:11.058998+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:07:28.283888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.035076+0000","last_change":"2024-01-15T18:12:10.020848+0000","last_active":"2024-01-15T18:12:11.035076+0000","last_peered":"2024-01-15T18:12:11.035076+0000","last_clean":"2024-01-15T18:12:11.035076+0000","last_became_active":"2024-01-15T18:12:10.020497+0000","last_became_peered":"2024-01-15T18:12:10.020497+0000","last_unstale":"2024-01-15T18:12:11.035076+0000","last_undegraded":"2024-01-15T18:12:11.035076+0000","last_fullsized":"2024-01-15T18:12:11.035076+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:37.798936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.059490+0000","last_change":"2024-01-15T18:12:11.051721+0000","last_active":"2024-01-15T18:12:11.059490+0000","last_peered":"2024-01-15T18:12:11.059490+0000","last_clean":"2024-01-15T18:12:11.059490+0000","last_became_active":"2024-01-15T18:12:11.050810+0000","last_became_peered":"2024-01-15T18:12:11.050810+0000","last_unstale":"2024-01-15T18:12:11.059490+0000","last_undegraded":"2024-01-15T18:12:11.059490+0000","last_fullsized":"2024-01-15T18:12:11.059490+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.044948+0000","last_change":"2024-01-15T18:12:10.025006+0000","last_active":"2024-01-15T18:12:11.044948+0000","last_peered":"2024-01-15T18:12:11.044948+0000","last_clean":"2024-01-15T18:12:11.044948+0000","last_became_active":"2024-01-15T18:12:10.024524+0000","last_became_peered":"2024-01-15T18:12:10.024524+0000","last_unstale":"2024-01-15T18:12:11.044948+0000","last_undegraded":"2024-01-15T18:12:11.044948+0000","last_fullsized":"2024-01-15T18:12:11.044948+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:16.499218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.062797+0000","last_change":"2024-01-15T18:12:11.061751+0000","last_active":"2024-01-15T18:12:11.062797+0000","last_peered":"2024-01-15T18:12:11.062797+0000","last_clean":"2024-01-15T18:12:11.062797+0000","last_became_active":"2024-01-15T18:12:11.061454+0000","last_became_peered":"2024-01-15T18:12:11.061454+0000","last_unstale":"2024-01-15T18:12:11.062797+0000","last_undegraded":"2024-01-15T18:12:11.062797+0000","last_fullsized":"2024-01-15T18:12:11.062797+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:20.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.023109+0000","last_change":"2024-01-15T18:12:10.018428+0000","last_active":"2024-01-15T18:12:12.023109+0000","last_peered":"2024-01-15T18:12:12.023109+0000","last_clean":"2024-01-15T18:12:12.023109+0000","last_became_active":"2024-01-15T18:12:10.018057+0000","last_became_peered":"2024-01-15T18:12:10.018057+0000","last_unstale":"2024-01-15T18:12:12.023109+0000","last_undegraded":"2024-01-15T18:12:12.023109+0000","last_fullsized":"2024-01-15T18:12:12.023109+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:52:09.544387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026307+0000","last_change":"2024-01-15T18:12:11.039995+0000","last_active":"2024-01-15T18:12:12.026307+0000","last_peered":"2024-01-15T18:12:12.026307+0000","last_clean":"2024-01-15T18:12:12.026307+0000","last_became_active":"2024-01-15T18:12:11.039586+0000","last_became_peered":"2024-01-15T18:12:11.039586+0000","last_unstale":"2024-01-15T18:12:12.026307+0000","last_undegraded":"2024-01-15T18:12:12.026307+0000","last_fullsized":"2024-01-15T18:12:12.026307+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:38.812736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.031847+0000","last_change":"2024-01-15T18:12:10.019758+0000","last_active":"2024-01-15T18:12:11.031847+0000","last_peered":"2024-01-15T18:12:11.031847+0000","last_clean":"2024-01-15T18:12:11.031847+0000","last_became_active":"2024-01-15T18:12:10.019424+0000","last_became_peered":"2024-01-15T18:12:10.019424+0000","last_unstale":"2024-01-15T18:12:11.031847+0000","last_undegraded":"2024-01-15T18:12:11.031847+0000","last_fullsized":"2024-01-15T18:12:11.031847+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:45.797791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027287+0000","last_change":"2024-01-15T18:12:11.035857+0000","last_active":"2024-01-15T18:12:12.027287+0000","last_peered":"2024-01-15T18:12:12.027287+0000","last_clean":"2024-01-15T18:12:12.027287+0000","last_became_active":"2024-01-15T18:12:11.035238+0000","last_became_peered":"2024-01-15T18:12:11.035238+0000","last_unstale":"2024-01-15T18:12:12.027287+0000","last_undegraded":"2024-01-15T18:12:12.027287+0000","last_fullsized":"2024-01-15T18:12:12.027287+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026551+0000","last_change":"2024-01-15T18:12:10.017511+0000","last_active":"2024-01-15T18:12:12.026551+0000","last_peered":"2024-01-15T18:12:12.026551+0000","last_clean":"2024-01-15T18:12:12.026551+0000","last_became_active":"2024-01-15T18:12:10.017190+0000","last_became_peered":"2024-01-15T18:12:10.017190+0000","last_unstale":"2024-01-15T18:12:12.026551+0000","last_undegraded":"2024-01-15T18:12:12.026551+0000","last_fullsized":"2024-01-15T18:12:12.026551+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177633+0000","last_change":"2024-01-15T18:12:11.056667+0000","last_active":"2024-01-15T18:12:12.177633+0000","last_peered":"2024-01-15T18:12:12.177633+0000","last_clean":"2024-01-15T18:12:12.177633+0000","last_became_active":"2024-01-15T18:12:11.056111+0000","last_became_peered":"2024-01-15T18:12:11.056111+0000","last_unstale":"2024-01-15T18:12:12.177633+0000","last_undegraded":"2024-01-15T18:12:12.177633+0000","last_fullsized":"2024-01-15T18:12:12.177633+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:18:00.144603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025389+0000","last_change":"2024-01-15T18:12:10.021210+0000","last_active":"2024-01-15T18:12:12.025389+0000","last_peered":"2024-01-15T18:12:12.025389+0000","last_clean":"2024-01-15T18:12:12.025389+0000","last_became_active":"2024-01-15T18:12:10.020890+0000","last_became_peered":"2024-01-15T18:12:10.020890+0000","last_unstale":"2024-01-15T18:12:12.025389+0000","last_undegraded":"2024-01-15T18:12:12.025389+0000","last_fullsized":"2024-01-15T18:12:12.025389+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:44.292512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.061011+0000","last_change":"2024-01-15T18:12:11.059132+0000","last_active":"2024-01-15T18:12:11.061011+0000","last_peered":"2024-01-15T18:12:11.061011+0000","last_clean":"2024-01-15T18:12:11.061011+0000","last_became_active":"2024-01-15T18:12:11.057943+0000","last_became_peered":"2024-01-15T18:12:11.057943+0000","last_unstale":"2024-01-15T18:12:11.061011+0000","last_undegraded":"2024-01-15T18:12:11.061011+0000","last_fullsized":"2024-01-15T18:12:11.061011+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:05:35.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.043186+0000","last_change":"2024-01-15T18:12:10.010681+0000","last_active":"2024-01-15T18:12:11.043186+0000","last_peered":"2024-01-15T18:12:11.043186+0000","last_clean":"2024-01-15T18:12:11.043186+0000","last_became_active":"2024-01-15T18:12:10.009870+0000","last_became_peered":"2024-01-15T18:12:10.009870+0000","last_unstale":"2024-01-15T18:12:11.043186+0000","last_undegraded":"2024-01-15T18:12:11.043186+0000","last_fullsized":"2024-01-15T18:12:11.043186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:21.835760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.021670+0000","last_change":"2024-01-15T18:12:11.051496+0000","last_active":"2024-01-15T18:12:12.021670+0000","last_peered":"2024-01-15T18:12:12.021670+0000","last_clean":"2024-01-15T18:12:12.021670+0000","last_became_active":"2024-01-15T18:12:11.051092+0000","last_became_peered":"2024-01-15T18:12:11.051092+0000","last_unstale":"2024-01-15T18:12:12.021670+0000","last_undegraded":"2024-01-15T18:12:12.021670+0000","last_fullsized":"2024-01-15T18:12:12.021670+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:38:09.219058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.037109+0000","last_change":"2024-01-15T18:12:10.026923+0000","last_active":"2024-01-15T18:12:11.037109+0000","last_peered":"2024-01-15T18:12:11.037109+0000","last_clean":"2024-01-15T18:12:11.037109+0000","last_became_active":"2024-01-15T18:12:10.026651+0000","last_became_peered":"2024-01-15T18:12:10.026651+0000","last_unstale":"2024-01-15T18:12:11.037109+0000","last_undegraded":"2024-01-15T18:12:11.037109+0000","last_fullsized":"2024-01-15T18:12:11.037109+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:14.981958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":246,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.047580+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:12:11.047580+0000","last_peered":"2024-01-15T18:12:11.047580+0000","last_clean":"2024-01-15T18:12:11.047580+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:12:11.047580+0000","last_undegraded":"2024-01-15T18:12:11.047580+0000","last_fullsized":"2024-01-15T18:12:11.047580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542354,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65020,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306820,"statfs":{"total":96636764160,"available":96570183680,"internally_reserved":0,"allocated":2015232,"data_stored":1207274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2170000000000001,"15min":1.3779999999999999},"min":{"1min":0.84399999999999997,"5min":0.66300000000000003,"15min":0.40400000000000003},"max":{"1min":1.621,"5min":1.825,"15min":19.843},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3109999999999999,"15min":1.6379999999999999},"min":{"1min":0.99399999999999999,"5min":0.64900000000000002,"15min":0.53400000000000003},"max":{"1min":2.1960000000000002,"5min":7.8250000000000002,"15min":38.137},"last":0.997}]},{"osd":1,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.236,"15min":1.4119999999999999},"min":{"1min":0.79500000000000004,"5min":0.67000000000000004,"15min":0.45600000000000002},"max":{"1min":1.6020000000000001,"5min":1.9399999999999999,"15min":19.917000000000002},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.5649999999999999,"15min":1.694},"min":{"1min":0.91000000000000003,"5min":0.81100000000000005,"15min":0.42999999999999999},"max":{"1min":1.825,"5min":28.613,"15min":51.843000000000004},"last":1.5389999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.3300000000000001,"15min":1.4590000000000001},"min":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.437},"max":{"1min":4.9219999999999997,"5min":4.9219999999999997,"15min":12.061},"last":1.181},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4019999999999999,"15min":1.548},"min":{"1min":0.97399999999999998,"5min":0.78900000000000003,"15min":0.47399999999999998},"max":{"1min":1.837,"5min":8.2479999999999993,"15min":13.478999999999999},"last":1.47}]},{"osd":3,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":2.0510000000000002,"5min":1.49,"15min":1.599},"min":{"1min":0.94799999999999995,"5min":0.82499999999999996,"15min":0.47899999999999998},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":16.632000000000001},"last":1.409},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.421,"15min":1.609},"min":{"1min":0.95699999999999996,"5min":0.878,"15min":0.56699999999999995},"max":{"1min":1.702,"5min":2.2440000000000002,"15min":13.439},"last":1.8009999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.494,"15min":1.7330000000000001},"min":{"1min":0.64800000000000002,"5min":0.60699999999999998,"15min":0.50700000000000001},"max":{"1min":1.946,"5min":17.067,"15min":23.042000000000002},"last":1.714},{"interface":"front","average":{"1min":1.9370000000000001,"5min":1.4930000000000001,"15min":1.7569999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":7.6280000000000001,"5min":7.6280000000000001,"15min":38.682000000000002},"last":1.909}]},{"osd":5,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.478,"15min":1.796},"min":{"1min":0.86399999999999999,"5min":0.746,"15min":0.52700000000000002},"max":{"1min":1.786,"5min":2.1440000000000001,"15min":25.006},"last":1.381},{"interface":"front","average":{"1min":1.548,"5min":1.4750000000000001,"15min":1.8260000000000001},"min":{"1min":0.90300000000000002,"5min":0.627,"15min":0.624},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":22.933},"last":1.972}]},{"osd":6,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.591,"15min":2.0270000000000001},"min":{"1min":1.0529999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.982,"5min":2.2229999999999999,"15min":23.834},"last":2.081},{"interface":"front","average":{"1min":1.8,"5min":1.6299999999999999,"15min":2.0059999999999998},"min":{"1min":0.82799999999999996,"5min":0.64900000000000002,"15min":0.58299999999999996},"max":{"1min":4.6420000000000003,"5min":4.6420000000000003,"15min":63.932000000000002},"last":2.1640000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542353,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64436,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307404,"statfs":{"total":96636764160,"available":96570781696,"internally_reserved":0,"allocated":1417216,"data_stored":616360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.6830000000000001,"15min":1.502},"min":{"1min":0.71899999999999997,"5min":0.67600000000000005,"15min":0.53100000000000003},"max":{"1min":1.6859999999999999,"5min":31.805,"15min":31.805},"last":1.087},{"interface":"front","average":{"1min":1.284,"5min":1.583,"15min":1.518},"min":{"1min":1.0269999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5049999999999999,"5min":31.702999999999999,"15min":31.702999999999999},"last":1.01}]},{"osd":1,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3120000000000001,"15min":1.353},"min":{"1min":0.72999999999999998,"5min":0.60999999999999999,"15min":0.55100000000000005},"max":{"1min":1.7190000000000001,"5min":15.064,"15min":15.064},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.1930000000000001,"15min":1.5429999999999999},"min":{"1min":0.89400000000000002,"5min":0.74199999999999999,"15min":0.66200000000000003},"max":{"1min":1.6539999999999999,"5min":4.0579999999999998,"15min":23.495999999999999},"last":1.1579999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.329,"15min":1.6140000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.59399999999999997},"max":{"1min":1.915,"5min":3.9849999999999999,"15min":23.087},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.333,"5min":1.4379999999999999,"15min":1.431},"min":{"1min":0.84799999999999998,"5min":0.67900000000000005,"15min":0.504},"max":{"1min":2.4140000000000001,"5min":14.94,"15min":14.94},"last":1.3360000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.9430000000000001,"5min":1.649,"15min":1.7210000000000001},"min":{"1min":0.90600000000000003,"5min":0.71799999999999997,"15min":0.59899999999999998},"max":{"1min":9.8390000000000004,"5min":15.151999999999999,"15min":15.151999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.5229999999999999,"15min":1.6439999999999999},"min":{"1min":1.042,"5min":0.76100000000000001,"15min":0.70699999999999996},"max":{"1min":9.8900000000000006,"5min":9.8900000000000006,"15min":14.066000000000001},"last":1.284}]},{"osd":4,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.321,"15min":1.6599999999999999},"min":{"1min":0.96699999999999997,"5min":0.56100000000000005,"15min":0.51800000000000002},"max":{"1min":2.383,"5min":3.9169999999999998,"15min":23.327000000000002},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.6539999999999999,"15min":1.6950000000000001},"min":{"1min":0.99399999999999999,"5min":0.61899999999999999,"15min":0.45800000000000002},"max":{"1min":2.6600000000000001,"5min":15.239000000000001,"15min":17.289000000000001},"last":1.4890000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.427,"15min":1.5660000000000001},"min":{"1min":0.88600000000000001,"5min":0.52500000000000002,"15min":0.51600000000000001},"max":{"1min":2.4289999999999998,"5min":4.9260000000000002,"15min":14.176},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.619,"15min":1.863},"min":{"1min":1.431,"5min":0.64400000000000002,"15min":0.54700000000000004},"max":{"1min":2.6190000000000002,"5min":4.1790000000000003,"15min":23.417999999999999},"last":1.6220000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.573,"15min":1.7709999999999999},"min":{"1min":1.032,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.8380000000000001,"5min":4.8239999999999998,"15min":21.265000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.6200000000000001,"15min":1.9810000000000001},"min":{"1min":1.1439999999999999,"5min":0.77100000000000002,"15min":0.59699999999999998},"max":{"1min":2.1989999999999998,"5min":4.0220000000000002,"15min":23.738},"last":1.5449999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542353,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58940,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312900,"statfs":{"total":96636764160,"available":96576409600,"internally_reserved":0,"allocated":1421312,"data_stored":616894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.24,"15min":1.2949999999999999},"min":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.47799999999999998},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":19.266999999999999},"last":1.032},{"interface":"front","average":{"1min":1.155,"5min":1.177,"15min":1.321},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.51400000000000001},"max":{"1min":1.5820000000000001,"5min":2.1869999999999998,"15min":17.184999999999999},"last":0.96099999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2130000000000001,"15min":1.345},"min":{"1min":0.755,"5min":0.69699999999999995,"15min":0.499},"max":{"1min":1.746,"5min":2.355,"15min":16.297000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.254,"15min":1.3400000000000001},"min":{"1min":0.90800000000000003,"5min":0.78900000000000003,"15min":0.59099999999999997},"max":{"1min":2.097,"5min":2.097,"15min":9.2379999999999995},"last":1.5569999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":0.84099999999999997,"5min":0.78600000000000003,"15min":0.621},"max":{"1min":3.516,"5min":3.516,"15min":8.1820000000000004},"last":1.161},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3420000000000001,"15min":1.4430000000000001},"min":{"1min":0.80500000000000005,"5min":0.749,"15min":0.56000000000000005},"max":{"1min":1.7190000000000001,"5min":2.5539999999999998,"15min":16.158999999999999},"last":1.599}]},{"osd":3,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.379,"15min":1.5},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.63700000000000001},"max":{"1min":2.085,"5min":2.5179999999999998,"15min":14.914},"last":1.214},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.403,"15min":1.45},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.63300000000000001},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":7.0110000000000001},"last":1.266}]},{"osd":4,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3660000000000001,"15min":1.4970000000000001},"min":{"1min":0.878,"5min":0.80800000000000005,"15min":0.41399999999999998},"max":{"1min":2.3610000000000002,"5min":2.4689999999999999,"15min":16.414999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.4339999999999999,"15min":1.623},"min":{"1min":0.84599999999999997,"5min":0.56200000000000006,"15min":0.32800000000000001},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":17.991},"last":1.409}]},{"osd":6,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.409,"15min":1.4690000000000001},"min":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.39600000000000002},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":6.4630000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.48,"15min":1.5860000000000001},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.53000000000000003},"max":{"1min":1.7989999999999999,"5min":2.7010000000000001,"15min":16.364000000000001},"last":1.4750000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.5529999999999999,"15min":1.6120000000000001},"min":{"1min":0.99399999999999999,"5min":0.78900000000000003,"15min":0.45800000000000002},"max":{"1min":2.258,"5min":2.399,"15min":9.0600000000000005},"last":1.673},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5740000000000001,"15min":1.6830000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.57299999999999995},"max":{"1min":2.0750000000000002,"5min":2.6059999999999999,"15min":19.356999999999999},"last":1.5229999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542353,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64452,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307388,"statfs":{"total":96636764160,"available":96570765312,"internally_reserved":0,"allocated":1425408,"data_stored":622674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3999999999999999,"15min":1.395},"min":{"1min":0.82599999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.1219999999999999,"5min":17.346,"15min":17.346},"last":1.248},{"interface":"front","average":{"1min":1.304,"5min":1.3109999999999999,"15min":1.387},"min":{"1min":0.98399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.6479999999999999,"5min":2.7919999999999998,"15min":9.7439999999999998},"last":1.21}]},{"osd":1,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1919999999999999,"15min":1.335},"min":{"1min":0.76200000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.554,"5min":2.8700000000000001,"15min":11.987},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1870000000000001,"15min":1.29},"min":{"1min":0.82599999999999996,"5min":0.67500000000000004,"15min":0.55300000000000005},"max":{"1min":1.6259999999999999,"5min":3.7370000000000001,"15min":16.058},"last":0.78900000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.407,"15min":1.512},"min":{"1min":0.85799999999999998,"5min":0.64500000000000002,"15min":0.56899999999999995},"max":{"1min":1.6499999999999999,"5min":18.442,"15min":18.442},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3300000000000001,"15min":1.524},"min":{"1min":0.997,"5min":0.81899999999999995,"15min":0.69499999999999995},"max":{"1min":1.744,"5min":3.8250000000000002,"15min":13.903},"last":1.1459999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.488,"15min":1.6240000000000001},"min":{"1min":0.96299999999999997,"5min":0.69999999999999996,"15min":0.56699999999999995},"max":{"1min":2.0179999999999998,"5min":17.364999999999998,"15min":26.055},"last":1.361},{"interface":"front","average":{"1min":1.9179999999999999,"5min":1.45,"15min":1.4770000000000001},"min":{"1min":0.97699999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":9.3320000000000007,"5min":9.3320000000000007,"15min":13.25},"last":1.4350000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4339999999999999,"15min":1.5569999999999999},"min":{"1min":0.85799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.8080000000000001,"5min":17.292999999999999,"15min":29.422000000000001},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4219999999999999,"15min":1.4950000000000001},"min":{"1min":0.64000000000000001,"5min":0.53700000000000003,"15min":0.42199999999999999},"max":{"1min":2.1709999999999998,"5min":4.9400000000000004,"15min":12.332000000000001},"last":1.3129999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3540000000000001,"15min":1.502},"min":{"1min":0.79100000000000004,"5min":0.54200000000000004,"15min":0.45200000000000001},"max":{"1min":1.865,"5min":5.3150000000000004,"15min":18.260000000000002},"last":1.7350000000000001},{"interface":"front","average":{"1min":1.9239999999999999,"5min":1.7390000000000001,"15min":1.7170000000000001},"min":{"1min":0.80700000000000005,"5min":0.51100000000000001,"15min":0.48099999999999998},"max":{"1min":7.1920000000000002,"5min":17.224,"15min":17.224},"last":1.6060000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.538,"15min":1.7290000000000001},"min":{"1min":0.69799999999999995,"5min":0.625,"15min":0.503},"max":{"1min":2.5640000000000001,"5min":3.0680000000000001,"15min":12.247},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.647,"5min":1.7629999999999999,"15min":1.865},"min":{"1min":0.752,"5min":0.61799999999999999,"15min":0.53600000000000003},"max":{"1min":2.2229999999999999,"5min":17.324999999999999,"15min":17.324999999999999},"last":1.7010000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542355,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72768,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299072,"statfs":{"total":96636764160,"available":96562249728,"internally_reserved":0,"allocated":1429504,"data_stored":617972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.077,"15min":1.234},"min":{"1min":0.54900000000000004,"5min":0.39900000000000002,"15min":0.33300000000000002},"max":{"1min":3.1930000000000001,"5min":9.2899999999999991,"15min":18.399999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1279999999999999,"15min":1.3959999999999999},"min":{"1min":0.64500000000000002,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":1.917,"5min":2.1699999999999999,"15min":21.047000000000001},"last":0.91700000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.91800000000000004,"15min":1.274},"min":{"1min":0.57499999999999996,"5min":0.34899999999999998,"15min":0.32600000000000001},"max":{"1min":1.585,"5min":1.585,"15min":17.777999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3400000000000001,"15min":1.3029999999999999},"min":{"1min":0.73199999999999998,"5min":0.46300000000000002,"15min":0.39500000000000002},"max":{"1min":3.3119999999999998,"5min":11.228999999999999,"15min":11.228999999999999},"last":1.679}]},{"osd":2,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.238,"15min":1.288},"min":{"1min":0.59899999999999998,"5min":0.48499999999999999,"15min":0.375},"max":{"1min":2.109,"5min":2.7519999999999998,"15min":10.964},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.579,"5min":1.5149999999999999,"15min":1.5680000000000001},"min":{"1min":0.72399999999999998,"5min":0.57499999999999996,"15min":0.54000000000000004},"max":{"1min":3.8969999999999998,"5min":9.4920000000000009,"15min":20.495000000000001},"last":1.4490000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3460000000000001,"15min":1.5589999999999999},"min":{"1min":0.93100000000000005,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":2.0289999999999999,"5min":2.6179999999999999,"15min":13.536},"last":1.353},{"interface":"front","average":{"1min":1.458,"5min":1.2929999999999999,"15min":1.643},"min":{"1min":0.84199999999999997,"5min":0.63700000000000001,"15min":0.54600000000000004},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":16.398},"last":1.1259999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.855,"5min":1.627,"15min":1.716},"min":{"1min":1.288,"5min":0.875,"15min":0.57299999999999995},"max":{"1min":5.2069999999999999,"5min":9.5690000000000008,"15min":20.606000000000002},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.5740000000000001,"15min":1.6759999999999999},"min":{"1min":1.101,"5min":0.84399999999999997,"15min":0.68000000000000005},"max":{"1min":5.1470000000000002,"5min":5.1470000000000002,"15min":11.603},"last":1.5409999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.431,"15min":1.7949999999999999},"min":{"1min":1.0449999999999999,"5min":0.61399999999999999,"15min":0.53500000000000003},"max":{"1min":3.657,"5min":3.657,"15min":13.648999999999999},"last":1.401},{"interface":"front","average":{"1min":1.746,"5min":1.645,"15min":1.827},"min":{"1min":1.093,"5min":0.78300000000000003,"15min":0.65800000000000003},"max":{"1min":3.4740000000000002,"5min":9.6639999999999997,"15min":20.556000000000001},"last":1.7450000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.5569999999999999,"15min":1.8480000000000001},"min":{"1min":1.048,"5min":0.78800000000000003,"15min":0.60499999999999998},"max":{"1min":2.2490000000000001,"5min":2.9399999999999999,"15min":28.843},"last":1.643},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.5049999999999999,"15min":1.855},"min":{"1min":1.0880000000000001,"5min":0.72799999999999998,"15min":0.67200000000000004},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":15.143000000000001},"last":1.5009999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542355,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68604,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303236,"statfs":{"total":96636764160,"available":96566513664,"internally_reserved":0,"allocated":1413120,"data_stored":616290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98199999999999998,"15min":1.093},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.26600000000000001},"max":{"1min":2.5270000000000001,"5min":2.9359999999999999,"15min":14.427},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.292,"5min":1.161,"15min":1.367},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.39300000000000002},"max":{"1min":2.0489999999999999,"5min":2.25,"15min":15.314},"last":1.4650000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.204,"15min":1.4379999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.1429999999999998,"5min":2.7029999999999998,"15min":34.463999999999999},"last":1},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.2729999999999999,"15min":1.4650000000000001},"min":{"1min":0.71999999999999997,"5min":0.54200000000000004,"15min":0.373},"max":{"1min":2.3039999999999998,"5min":2.3929999999999998,"15min":25.859999999999999},"last":1.9630000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2969999999999999,"15min":1.4259999999999999},"min":{"1min":0.54300000000000004,"5min":0.48599999999999999,"15min":0.34399999999999997},"max":{"1min":2.2400000000000002,"5min":12.449999999999999,"15min":19.535},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.371,"15min":1.623},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.46400000000000002},"max":{"1min":2.7869999999999999,"5min":2.8050000000000002,"15min":22.783999999999999},"last":1.2509999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":1.518,"15min":1.7869999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.55700000000000005},"max":{"1min":5.4610000000000003,"5min":5.4610000000000003,"15min":29.986999999999998},"last":1.548},{"interface":"front","average":{"1min":1.7310000000000001,"5min":1.5149999999999999,"15min":1.7529999999999999},"min":{"1min":1.05,"5min":0.81000000000000005,"15min":0.56799999999999995},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":21.199999999999999},"last":1.327}]},{"osd":5,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.774,"15min":1.756},"min":{"1min":0.91100000000000003,"5min":0.84299999999999997,"15min":0.51500000000000001},"max":{"1min":4.2329999999999997,"5min":14.066000000000001,"15min":19.681999999999999},"last":1.77},{"interface":"front","average":{"1min":1.9199999999999999,"5min":1.8280000000000001,"15min":1.841},"min":{"1min":1.028,"5min":0.88700000000000001,"15min":0.60899999999999999},"max":{"1min":4.3079999999999998,"5min":14.025,"15min":14.702999999999999},"last":1.4370000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.841,"5min":1.603,"15min":1.8109999999999999},"min":{"1min":1.093,"5min":0.76800000000000002,"15min":0.68000000000000005},"max":{"1min":3.4590000000000001,"5min":3.4590000000000001,"15min":15.683999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.762,"15min":1.8919999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.58899999999999997},"max":{"1min":3.0089999999999999,"5min":12.629,"15min":19.783000000000001},"last":1.867}]},{"osd":7,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":2.5699999999999998,"5min":1.8440000000000001,"15min":2.133},"min":{"1min":1.204,"5min":0.86899999999999999,"15min":0.63100000000000001},"max":{"1min":15.738,"5min":15.738,"15min":57.006999999999998},"last":1.833},{"interface":"front","average":{"1min":1.806,"5min":1.8220000000000001,"15min":2.2269999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.56100000000000005},"max":{"1min":2.887,"5min":12.754,"15min":22.853999999999999},"last":2.016}]}]},{"osd":1,"up_from":14,"seq":60129542354,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72772,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299068,"statfs":{"total":96636764160,"available":96562245632,"internally_reserved":0,"allocated":1421312,"data_stored":616414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.016,"15min":1.23},"min":{"1min":0.70099999999999996,"5min":0.48899999999999999,"15min":0.34399999999999997},"max":{"1min":1.9399999999999999,"5min":2.6139999999999999,"15min":30.460999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.97,"5min":1.3360000000000001,"15min":1.218},"min":{"1min":0.93100000000000005,"5min":0.38700000000000001,"15min":0.314},"max":{"1min":17.157,"5min":17.157,"15min":18.658000000000001},"last":0.95999999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.202,"15min":1.2629999999999999},"min":{"1min":0.92100000000000004,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":2.5209999999999999,"5min":4.7119999999999997,"15min":21.126000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.3799999999999999,"15min":1.595},"min":{"1min":0.67000000000000004,"5min":0.54800000000000004,"15min":0.48699999999999999},"max":{"1min":2.2690000000000001,"5min":3.0409999999999999,"15min":30.593},"last":1.4570000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.3340000000000001,"15min":1.5009999999999999},"min":{"1min":0.76800000000000002,"5min":0.60799999999999998,"15min":0.48599999999999999},"max":{"1min":2.5019999999999998,"5min":3.1400000000000001,"15min":30.547000000000001},"last":1.153},{"interface":"front","average":{"1min":2.0870000000000002,"5min":1.405,"15min":1.4079999999999999},"min":{"1min":0.73799999999999999,"5min":0.52700000000000002,"15min":0.437},"max":{"1min":17.263000000000002,"5min":17.263000000000002,"15min":17.263000000000002},"last":1.0700000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":2.3730000000000002,"5min":1.716,"15min":1.718},"min":{"1min":1.1060000000000001,"5min":0.75900000000000001,"15min":0.72499999999999998},"max":{"1min":17.420000000000002,"5min":17.420000000000002,"15min":17.420000000000002},"last":1.706},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.569,"15min":1.611},"min":{"1min":1.2749999999999999,"5min":0.72999999999999998,"15min":0.69299999999999995},"max":{"1min":2.9569999999999999,"5min":4.6369999999999996,"15min":14.851000000000001},"last":1.603}]},{"osd":5,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.8660000000000001,"5min":1.593,"15min":1.583},"min":{"1min":1.1539999999999999,"5min":0.77600000000000002,"15min":0.59599999999999997},"max":{"1min":9.0440000000000005,"5min":9.0440000000000005,"15min":11.552},"last":1.7629999999999999},{"interface":"front","average":{"1min":1.73,"5min":1.573,"15min":1.9450000000000001},"min":{"1min":1.2829999999999999,"5min":0.78700000000000003,"15min":0.68300000000000005},"max":{"1min":2.4750000000000001,"5min":2.8119999999999998,"15min":34.042999999999999},"last":1.5600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":1.659,"15min":1.742},"min":{"1min":1.071,"5min":0.83899999999999997,"15min":0.64100000000000001},"max":{"1min":3.8580000000000001,"5min":10.250999999999999,"15min":34.158000000000001},"last":1.9139999999999999},{"interface":"front","average":{"1min":2.4820000000000002,"5min":1.768,"15min":1.6970000000000001},"min":{"1min":1.1870000000000001,"5min":0.67900000000000005,"15min":0.62},"max":{"1min":17.510000000000002,"5min":17.510000000000002,"15min":17.510000000000002},"last":1.506}]},{"osd":7,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.869,"5min":1.6950000000000001,"15min":1.899},"min":{"1min":1.2130000000000001,"5min":0.77300000000000002,"15min":0.64600000000000002},"max":{"1min":2.5899999999999999,"5min":3.7709999999999999,"15min":33.659999999999997},"last":1.657},{"interface":"front","average":{"1min":1.853,"5min":1.6919999999999999,"15min":1.8320000000000001},"min":{"1min":1.2110000000000001,"5min":0.81000000000000005,"15min":0.60499999999999998},"max":{"1min":2.8479999999999999,"5min":4.8380000000000001,"15min":24.289999999999999},"last":1.847}]}]},{"osd":0,"up_from":14,"seq":60129542355,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69192,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302648,"statfs":{"total":96636764160,"available":96565911552,"internally_reserved":0,"allocated":2027520,"data_stored":1208382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2609999999999999,"15min":1.103},"min":{"1min":0.47499999999999998,"5min":0.42799999999999999,"15min":0.312},"max":{"1min":2.5510000000000002,"5min":12.050000000000001,"15min":12.050000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.4450000000000001,"15min":1.3480000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.36199999999999999},"max":{"1min":3.1019999999999999,"5min":11.962,"15min":11.962},"last":1.1950000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.177,"15min":1.2190000000000001},"min":{"1min":0.495,"5min":0.47399999999999998,"15min":0.27600000000000002},"max":{"1min":2.1509999999999998,"5min":7.4560000000000004,"15min":11.59},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1419999999999999,"15min":1.1859999999999999},"min":{"1min":0.46100000000000002,"5min":0.38500000000000001,"15min":0.35899999999999999},"max":{"1min":2.0150000000000001,"5min":7.3239999999999998,"15min":14.512},"last":1.2729999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.4059999999999999,"15min":1.4319999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.41699999999999998},"max":{"1min":1.865,"5min":11.904,"15min":11.904},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.153,"15min":1.262},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.29599999999999999},"max":{"1min":2.657,"5min":3.202,"15min":14.74},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.387,"15min":1.6000000000000001},"min":{"1min":0.84399999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.1949999999999998,"5min":7.4000000000000004,"15min":32.115000000000002},"last":1.117},{"interface":"front","average":{"1min":1.46,"5min":1.548,"15min":1.7},"min":{"1min":0.82399999999999995,"5min":0.70499999999999996,"15min":0.67700000000000005},"max":{"1min":2.1269999999999998,"5min":9.0790000000000006,"15min":15.975},"last":1.53}]},{"osd":5,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5900000000000001,"15min":1.7350000000000001},"min":{"1min":0.86199999999999999,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":3.25,"5min":12.006,"15min":32.174999999999997},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4830000000000001,"15min":1.633},"min":{"1min":0.82699999999999996,"5min":0.57799999999999996,"15min":0.442},"max":{"1min":2.9769999999999999,"5min":4.7119999999999997,"15min":17.225000000000001},"last":0.78200000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.7529999999999999,"15min":1.8},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8809999999999998,"5min":24.039999999999999,"15min":24.039999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.6200000000000001,"15min":1.804},"min":{"1min":0.89700000000000002,"5min":0.73299999999999998,"15min":0.625},"max":{"1min":2.613,"5min":7.5359999999999996,"15min":32.216999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.857,"15min":2.121},"min":{"1min":0.86199999999999999,"5min":0.754,"15min":0.69099999999999995},"max":{"1min":2.5059999999999998,"5min":26.427,"15min":34.131},"last":1.431},{"interface":"front","average":{"1min":1.5880000000000001,"5min":2.0699999999999998,"15min":2.0419999999999998},"min":{"1min":0.84099999999999997,"5min":0.753,"15min":0.61799999999999999},"max":{"1min":2.7799999999999998,"5min":44.694000000000003,"15min":44.694000000000003},"last":1.042}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:12:15.753 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:12:15.978 INFO:tasks.ceph.mds.a.smithi026.stderr:2024-01-15T18:12:15.974+0000 7f4f12200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-15T18:12:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.008+0000 7f996ae00640 1 Processor -- start 2024-01-15T18:12:16.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.008+0000 7f996ae00640 1 -- start start 2024-01-15T18:12:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.010+0000 7f996ae00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f996413e360 0x7f99640cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.010+0000 7f996ae00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f996415f9c0 0x7f99640cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.010+0000 7f996ae00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996415c5f0 con 0x7f996413df80 2024-01-15T18:12:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.010+0000 7f996ae00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996406c470 con 0x7f996415f9c0 2024-01-15T18:12:16.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.010+0000 7f996ae00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996406c750 con 0x7f996413e360 2024-01-15T18:12:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.010+0000 7f9968e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f996415f9c0 0x7f99640cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.011+0000 7f9968e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f996415f9c0 0x7f99640cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34740/0 (socket says 172.21.15.26:34740) 2024-01-15T18:12:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.011+0000 7f9968e00640 1 -- 172.21.15.26:0/3108886936 learned_addr learned my addr 172.21.15.26:0/3108886936 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.011+0000 7f9963400640 1 --2- 172.21.15.26:0/3108886936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f996413e360 0x7f99640cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.011+0000 7f9968e00640 1 -- 172.21.15.26:0/3108886936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f996413e360 msgr2=0x7f99640cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.011+0000 7f9968e00640 1 --2- 172.21.15.26:0/3108886936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f996413e360 0x7f99640cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.011+0000 7f9968e00640 1 -- 172.21.15.26:0/3108886936 >> v1:172.21.15.124:6789/0 conn(0x7f996413df80 legacy=0x7f996415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:16.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.011+0000 7f9968e00640 1 -- 172.21.15.26:0/3108886936 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996406ca30 con 0x7f996415f9c0 2024-01-15T18:12:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.012+0000 7f9968e00640 1 --2- 172.21.15.26:0/3108886936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996415f9c0 0x7f99640cf7d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f995c009870 tx=0x7f995c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a8476fc734bb69 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.012+0000 7f9962a00640 1 -- 172.21.15.26:0/3108886936 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c031640 con 0x7f996415f9c0 2024-01-15T18:12:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f9962a00640 1 -- 172.21.15.26:0/3108886936 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c0317a0 con 0x7f996415f9c0 2024-01-15T18:12:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f9962a00640 1 -- 172.21.15.26:0/3108886936 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c031b30 con 0x7f996415f9c0 2024-01-15T18:12:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f996ae00640 1 -- 172.21.15.26:0/3108886936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996415f9c0 msgr2=0x7f99640cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f996ae00640 1 --2- 172.21.15.26:0/3108886936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996415f9c0 0x7f99640cf7d0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f995c009870 tx=0x7f995c02b8f0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f996ae00640 1 -- 172.21.15.26:0/3108886936 shutdown_connections 2024-01-15T18:12:16.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f996ae00640 1 --2- 172.21.15.26:0/3108886936 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996415f9c0 0x7f99640cf7d0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f996ae00640 1 --2- 172.21.15.26:0/3108886936 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f996413e360 0x7f99640cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f996ae00640 1 -- 172.21.15.26:0/3108886936 >> 172.21.15.26:0/3108886936 conn(0x7f996406a800 msgr2=0x7f996406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:16.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f996ae00640 1 -- 172.21.15.26:0/3108886936 shutdown_connections 2024-01-15T18:12:16.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.013+0000 7f996ae00640 1 -- 172.21.15.26:0/3108886936 wait complete. 2024-01-15T18:12:16.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.015+0000 7f996ae00640 1 Processor -- start 2024-01-15T18:12:16.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.015+0000 7f996ae00640 1 -- start start 2024-01-15T18:12:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.016+0000 7f996ae00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f996413df80 0x7f9964150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.016+0000 7f996ae00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996413e360 0x7f996413a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.016+0000 7f996ae00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f996415f9c0 0x7f996413a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.016+0000 7f996ae00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964152470 con 0x7f996413e360 2024-01-15T18:12:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.016+0000 7f996ae00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9964152750 con 0x7f996413df80 2024-01-15T18:12:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.016+0000 7f996ae00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9964152a30 con 0x7f996415f9c0 2024-01-15T18:12:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.016+0000 7f9963400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996413e360 0x7f996413a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.016+0000 7f9963400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996413e360 0x7f996413a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34750/0 (socket says 172.21.15.26:34750) 2024-01-15T18:12:16.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.016+0000 7f9963400640 1 -- 172.21.15.26:0/1931376070 learned_addr learned my addr 172.21.15.26:0/1931376070 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.017+0000 7f9963e00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f996413df80 0x7f9964150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.017+0000 7f9963400640 1 -- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f996415f9c0 msgr2=0x7f996413a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.017+0000 7f9963400640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f996415f9c0 0x7f996413a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.017+0000 7f9963400640 1 -- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f996413df80 msgr2=0x7f9964150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.017+0000 7f9963400640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f996413df80 0x7f9964150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.017+0000 7f9963400640 1 -- 172.21.15.26:0/1931376070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c0094d0 con 0x7f996413e360 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.017+0000 7f9963e00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f996413df80 0x7f9964150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.017+0000 7f9963400640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996413e360 0x7f996413a420 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f994c00d800 tx=0x7f994c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.017+0000 7f995be00640 1 -- 172.21.15.26:0/1931376070 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c00b6c0 con 0x7f996413e360 2024-01-15T18:12:16.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.018+0000 7f995be00640 1 -- 172.21.15.26:0/1931376070 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c00b820 con 0x7f996413e360 2024-01-15T18:12:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.018+0000 7f995be00640 1 -- 172.21.15.26:0/1931376070 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c00bbb0 con 0x7f996413e360 2024-01-15T18:12:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.018+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9964152d10 con 0x7f996413e360 2024-01-15T18:12:16.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.018+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9964150c90 con 0x7f996413e360 2024-01-15T18:12:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.019+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99640675d0 con 0x7f996413e360 2024-01-15T18:12:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.020+0000 7f995be00640 1 -- 172.21.15.26:0/1931376070 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f994c00bd10 con 0x7f996413e360 2024-01-15T18:12:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.021+0000 7f995be00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f993c06c460 0x7f993c06e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.021+0000 7f995be00640 1 -- 172.21.15.26:0/1931376070 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f994c089140 con 0x7f996413e360 2024-01-15T18:12:16.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.021+0000 7f9963e00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f993c06c460 0x7f993c06e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.022+0000 7f9963e00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f993c06c460 0x7f993c06e920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9954007e90 tx=0x7f99540073a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:16.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.025+0000 7f995be00640 1 -- 172.21.15.26:0/1931376070 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f994c055620 con 0x7f996413e360 2024-01-15T18:12:16.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.214+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99641528c0 con 0x7f993c06c460 2024-01-15T18:12:16.241 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.238+0000 7f995be00640 1 -- 172.21.15.26:0/1931376070 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287317 (secure 0 0 0) 0x7f99641528c0 con 0x7f993c06c460 2024-01-15T18:12:16.247 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:16.249 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:12:16.251 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f993c06c460 msgr2=0x7f993c06e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f993c06c460 0x7f993c06e920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9954007e90 tx=0x7f99540073a0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996413e360 msgr2=0x7f996413a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996413e360 0x7f996413a420 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f994c00d800 tx=0x7f994c00dcc0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 shutdown_connections 2024-01-15T18:12:16.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f996415f9c0 0x7f996413a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.252 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f993c06c460 0x7f993c06e920 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f996413e360 0x7f996413a420 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 --2- 172.21.15.26:0/1931376070 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f996413df80 0x7f9964150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 >> 172.21.15.26:0/1931376070 conn(0x7f996406a800 msgr2=0x7f996406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:16.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 shutdown_connections 2024-01-15T18:12:16.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.244+0000 7f996ae00640 1 -- 172.21.15.26:0/1931376070 wait complete. 2024-01-15T18:12:16.266 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":627,"stamp":"2024-01-15T18:12:14.314052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":536184,"kb_used_data":12276,"kb_used_omap":12,"kb_used_meta":523763,"kb_avail":754438536,"statfs":{"total":773094113280,"available":772545060864,"internally_reserved":0,"allocated":12570624,"data_stored":6122260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":536333872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.326351"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058977+0000","last_change":"2024-01-15T18:12:11.049832+0000","last_active":"2024-01-15T18:12:11.058977+0000","last_peered":"2024-01-15T18:12:11.058977+0000","last_clean":"2024-01-15T18:12:11.058977+0000","last_became_active":"2024-01-15T18:12:11.048782+0000","last_became_peered":"2024-01-15T18:12:11.048782+0000","last_unstale":"2024-01-15T18:12:11.058977+0000","last_undegraded":"2024-01-15T18:12:11.058977+0000","last_fullsized":"2024-01-15T18:12:11.058977+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:19:47.603862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027038+0000","last_change":"2024-01-15T18:12:11.053712+0000","last_active":"2024-01-15T18:12:12.027038+0000","last_peered":"2024-01-15T18:12:12.027038+0000","last_clean":"2024-01-15T18:12:12.027038+0000","last_became_active":"2024-01-15T18:12:11.053279+0000","last_became_peered":"2024-01-15T18:12:11.053279+0000","last_unstale":"2024-01-15T18:12:12.027038+0000","last_undegraded":"2024-01-15T18:12:12.027038+0000","last_fullsized":"2024-01-15T18:12:12.027038+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177100+0000","last_change":"2024-01-15T18:12:11.058625+0000","last_active":"2024-01-15T18:12:12.177100+0000","last_peered":"2024-01-15T18:12:12.177100+0000","last_clean":"2024-01-15T18:12:12.177100+0000","last_became_active":"2024-01-15T18:12:11.057543+0000","last_became_peered":"2024-01-15T18:12:11.057543+0000","last_unstale":"2024-01-15T18:12:12.177100+0000","last_undegraded":"2024-01-15T18:12:12.177100+0000","last_fullsized":"2024-01-15T18:12:12.177100+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:17.569977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058584+0000","last_change":"2024-01-15T18:12:11.047313+0000","last_active":"2024-01-15T18:12:11.058584+0000","last_peered":"2024-01-15T18:12:11.058584+0000","last_clean":"2024-01-15T18:12:11.058584+0000","last_became_active":"2024-01-15T18:12:11.047008+0000","last_became_peered":"2024-01-15T18:12:11.047008+0000","last_unstale":"2024-01-15T18:12:11.058584+0000","last_undegraded":"2024-01-15T18:12:11.058584+0000","last_fullsized":"2024-01-15T18:12:11.058584+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:38:56.937377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026041+0000","last_change":"2024-01-15T18:12:11.035567+0000","last_active":"2024-01-15T18:12:12.026041+0000","last_peered":"2024-01-15T18:12:12.026041+0000","last_clean":"2024-01-15T18:12:12.026041+0000","last_became_active":"2024-01-15T18:12:11.034909+0000","last_became_peered":"2024-01-15T18:12:11.034909+0000","last_unstale":"2024-01-15T18:12:12.026041+0000","last_undegraded":"2024-01-15T18:12:12.026041+0000","last_fullsized":"2024-01-15T18:12:12.026041+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:46.811591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058229+0000","last_change":"2024-01-15T18:12:11.055133+0000","last_active":"2024-01-15T18:12:11.058229+0000","last_peered":"2024-01-15T18:12:11.058229+0000","last_clean":"2024-01-15T18:12:11.058229+0000","last_became_active":"2024-01-15T18:12:11.054803+0000","last_became_peered":"2024-01-15T18:12:11.054803+0000","last_unstale":"2024-01-15T18:12:11.058229+0000","last_undegraded":"2024-01-15T18:12:11.058229+0000","last_fullsized":"2024-01-15T18:12:11.058229+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:09:48.700182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.048700+0000","last_change":"2024-01-15T18:12:11.045438+0000","last_active":"2024-01-15T18:12:11.048700+0000","last_peered":"2024-01-15T18:12:11.048700+0000","last_clean":"2024-01-15T18:12:11.048700+0000","last_became_active":"2024-01-15T18:12:11.044607+0000","last_became_peered":"2024-01-15T18:12:11.044607+0000","last_unstale":"2024-01-15T18:12:11.048700+0000","last_undegraded":"2024-01-15T18:12:11.048700+0000","last_fullsized":"2024-01-15T18:12:11.048700+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:50.244523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177198+0000","last_change":"2024-01-15T18:12:11.051056+0000","last_active":"2024-01-15T18:12:12.177198+0000","last_peered":"2024-01-15T18:12:12.177198+0000","last_clean":"2024-01-15T18:12:12.177198+0000","last_became_active":"2024-01-15T18:12:11.050697+0000","last_became_peered":"2024-01-15T18:12:11.050697+0000","last_unstale":"2024-01-15T18:12:12.177198+0000","last_undegraded":"2024-01-15T18:12:12.177198+0000","last_fullsized":"2024-01-15T18:12:12.177198+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:56.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058955+0000","last_change":"2024-01-15T18:12:11.048227+0000","last_active":"2024-01-15T18:12:11.058955+0000","last_peered":"2024-01-15T18:12:11.058955+0000","last_clean":"2024-01-15T18:12:11.058955+0000","last_became_active":"2024-01-15T18:12:11.047693+0000","last_became_peered":"2024-01-15T18:12:11.047693+0000","last_unstale":"2024-01-15T18:12:11.058955+0000","last_undegraded":"2024-01-15T18:12:11.058955+0000","last_fullsized":"2024-01-15T18:12:11.058955+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:35:04.986605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.059982+0000","last_change":"2024-01-15T18:12:11.054486+0000","last_active":"2024-01-15T18:12:11.059982+0000","last_peered":"2024-01-15T18:12:11.059982+0000","last_clean":"2024-01-15T18:12:11.059982+0000","last_became_active":"2024-01-15T18:12:11.052429+0000","last_became_peered":"2024-01-15T18:12:11.052429+0000","last_unstale":"2024-01-15T18:12:11.059982+0000","last_undegraded":"2024-01-15T18:12:11.059982+0000","last_fullsized":"2024-01-15T18:12:11.059982+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027364+0000","last_change":"2024-01-15T18:12:11.043835+0000","last_active":"2024-01-15T18:12:12.027364+0000","last_peered":"2024-01-15T18:12:12.027364+0000","last_clean":"2024-01-15T18:12:12.027364+0000","last_became_active":"2024-01-15T18:12:11.042720+0000","last_became_peered":"2024-01-15T18:12:11.042720+0000","last_unstale":"2024-01-15T18:12:12.027364+0000","last_undegraded":"2024-01-15T18:12:12.027364+0000","last_fullsized":"2024-01-15T18:12:12.027364+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058563+0000","last_change":"2024-01-15T18:12:11.044468+0000","last_active":"2024-01-15T18:12:11.058563+0000","last_peered":"2024-01-15T18:12:11.058563+0000","last_clean":"2024-01-15T18:12:11.058563+0000","last_became_active":"2024-01-15T18:12:11.044155+0000","last_became_peered":"2024-01-15T18:12:11.044155+0000","last_unstale":"2024-01-15T18:12:11.058563+0000","last_undegraded":"2024-01-15T18:12:11.058563+0000","last_fullsized":"2024-01-15T18:12:11.058563+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177529+0000","last_change":"2024-01-15T18:12:11.052950+0000","last_active":"2024-01-15T18:12:12.177529+0000","last_peered":"2024-01-15T18:12:12.177529+0000","last_clean":"2024-01-15T18:12:12.177529+0000","last_became_active":"2024-01-15T18:12:11.051875+0000","last_became_peered":"2024-01-15T18:12:11.051875+0000","last_unstale":"2024-01-15T18:12:12.177529+0000","last_undegraded":"2024-01-15T18:12:12.177529+0000","last_fullsized":"2024-01-15T18:12:12.177529+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:22.849560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.176913+0000","last_change":"2024-01-15T18:12:11.056103+0000","last_active":"2024-01-15T18:12:12.176913+0000","last_peered":"2024-01-15T18:12:12.176913+0000","last_clean":"2024-01-15T18:12:12.176913+0000","last_became_active":"2024-01-15T18:12:11.055813+0000","last_became_peered":"2024-01-15T18:12:11.055813+0000","last_unstale":"2024-01-15T18:12:12.176913+0000","last_undegraded":"2024-01-15T18:12:12.176913+0000","last_fullsized":"2024-01-15T18:12:12.176913+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.025824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060652+0000","last_change":"2024-01-15T18:12:11.056863+0000","last_active":"2024-01-15T18:12:11.060652+0000","last_peered":"2024-01-15T18:12:11.060652+0000","last_clean":"2024-01-15T18:12:11.060652+0000","last_became_active":"2024-01-15T18:12:11.055217+0000","last_became_peered":"2024-01-15T18:12:11.055217+0000","last_unstale":"2024-01-15T18:12:11.060652+0000","last_undegraded":"2024-01-15T18:12:11.060652+0000","last_fullsized":"2024-01-15T18:12:11.060652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:18:05.495499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060095+0000","last_change":"2024-01-15T18:12:11.056227+0000","last_active":"2024-01-15T18:12:11.060095+0000","last_peered":"2024-01-15T18:12:11.060095+0000","last_clean":"2024-01-15T18:12:11.060095+0000","last_became_active":"2024-01-15T18:12:11.055919+0000","last_became_peered":"2024-01-15T18:12:11.055919+0000","last_unstale":"2024-01-15T18:12:11.060095+0000","last_undegraded":"2024-01-15T18:12:11.060095+0000","last_fullsized":"2024-01-15T18:12:11.060095+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:02:12.967981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026929+0000","last_change":"2024-01-15T18:12:11.043060+0000","last_active":"2024-01-15T18:12:12.026929+0000","last_peered":"2024-01-15T18:12:12.026929+0000","last_clean":"2024-01-15T18:12:12.026929+0000","last_became_active":"2024-01-15T18:12:11.042031+0000","last_became_peered":"2024-01-15T18:12:11.042031+0000","last_unstale":"2024-01-15T18:12:12.026929+0000","last_undegraded":"2024-01-15T18:12:12.026929+0000","last_fullsized":"2024-01-15T18:12:12.026929+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:44.791297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.046437+0000","last_change":"2024-01-15T18:12:11.045284+0000","last_active":"2024-01-15T18:12:11.046437+0000","last_peered":"2024-01-15T18:12:11.046437+0000","last_clean":"2024-01-15T18:12:11.046437+0000","last_became_active":"2024-01-15T18:12:11.042814+0000","last_became_peered":"2024-01-15T18:12:11.042814+0000","last_unstale":"2024-01-15T18:12:11.046437+0000","last_undegraded":"2024-01-15T18:12:11.046437+0000","last_fullsized":"2024-01-15T18:12:11.046437+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:31:04.806413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177655+0000","last_change":"2024-01-15T18:12:11.055119+0000","last_active":"2024-01-15T18:12:12.177655+0000","last_peered":"2024-01-15T18:12:12.177655+0000","last_clean":"2024-01-15T18:12:12.177655+0000","last_became_active":"2024-01-15T18:12:11.054790+0000","last_became_peered":"2024-01-15T18:12:11.054790+0000","last_unstale":"2024-01-15T18:12:12.177655+0000","last_undegraded":"2024-01-15T18:12:12.177655+0000","last_fullsized":"2024-01-15T18:12:12.177655+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:17.513018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027544+0000","last_change":"2024-01-15T18:12:11.052755+0000","last_active":"2024-01-15T18:12:12.027544+0000","last_peered":"2024-01-15T18:12:12.027544+0000","last_clean":"2024-01-15T18:12:12.027544+0000","last_became_active":"2024-01-15T18:12:11.052289+0000","last_became_peered":"2024-01-15T18:12:11.052289+0000","last_unstale":"2024-01-15T18:12:12.027544+0000","last_undegraded":"2024-01-15T18:12:12.027544+0000","last_fullsized":"2024-01-15T18:12:12.027544+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:38:56.937377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026877+0000","last_change":"2024-01-15T18:12:11.036748+0000","last_active":"2024-01-15T18:12:12.026877+0000","last_peered":"2024-01-15T18:12:12.026877+0000","last_clean":"2024-01-15T18:12:12.026877+0000","last_became_active":"2024-01-15T18:12:11.036003+0000","last_became_peered":"2024-01-15T18:12:11.036003+0000","last_unstale":"2024-01-15T18:12:12.026877+0000","last_undegraded":"2024-01-15T18:12:12.026877+0000","last_fullsized":"2024-01-15T18:12:12.026877+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:20.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022120+0000","last_change":"2024-01-15T18:12:11.050445+0000","last_active":"2024-01-15T18:12:12.022120+0000","last_peered":"2024-01-15T18:12:12.022120+0000","last_clean":"2024-01-15T18:12:12.022120+0000","last_became_active":"2024-01-15T18:12:11.044093+0000","last_became_peered":"2024-01-15T18:12:11.044093+0000","last_unstale":"2024-01-15T18:12:12.022120+0000","last_undegraded":"2024-01-15T18:12:12.022120+0000","last_fullsized":"2024-01-15T18:12:12.022120+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:38:59.097623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026316+0000","last_change":"2024-01-15T18:12:11.053397+0000","last_active":"2024-01-15T18:12:12.026316+0000","last_peered":"2024-01-15T18:12:12.026316+0000","last_clean":"2024-01-15T18:12:12.026316+0000","last_became_active":"2024-01-15T18:12:11.052428+0000","last_became_peered":"2024-01-15T18:12:11.052428+0000","last_unstale":"2024-01-15T18:12:12.026316+0000","last_undegraded":"2024-01-15T18:12:12.026316+0000","last_fullsized":"2024-01-15T18:12:12.026316+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:07.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.021417+0000","last_change":"2024-01-15T18:12:11.051820+0000","last_active":"2024-01-15T18:12:12.021417+0000","last_peered":"2024-01-15T18:12:12.021417+0000","last_clean":"2024-01-15T18:12:12.021417+0000","last_became_active":"2024-01-15T18:12:11.051488+0000","last_became_peered":"2024-01-15T18:12:11.051488+0000","last_unstale":"2024-01-15T18:12:12.021417+0000","last_undegraded":"2024-01-15T18:12:12.021417+0000","last_fullsized":"2024-01-15T18:12:12.021417+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:45.480493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027143+0000","last_change":"2024-01-15T18:12:11.041672+0000","last_active":"2024-01-15T18:12:12.027143+0000","last_peered":"2024-01-15T18:12:12.027143+0000","last_clean":"2024-01-15T18:12:12.027143+0000","last_became_active":"2024-01-15T18:12:11.041266+0000","last_became_peered":"2024-01-15T18:12:11.041266+0000","last_unstale":"2024-01-15T18:12:12.027143+0000","last_undegraded":"2024-01-15T18:12:12.027143+0000","last_fullsized":"2024-01-15T18:12:12.027143+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058937+0000","last_change":"2024-01-15T18:12:11.048567+0000","last_active":"2024-01-15T18:12:11.058937+0000","last_peered":"2024-01-15T18:12:11.058937+0000","last_clean":"2024-01-15T18:12:11.058937+0000","last_became_active":"2024-01-15T18:12:11.048234+0000","last_became_peered":"2024-01-15T18:12:11.048234+0000","last_unstale":"2024-01-15T18:12:11.058937+0000","last_undegraded":"2024-01-15T18:12:11.058937+0000","last_fullsized":"2024-01-15T18:12:11.058937+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.025824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.063349+0000","last_change":"2024-01-15T18:12:11.062039+0000","last_active":"2024-01-15T18:12:11.063349+0000","last_peered":"2024-01-15T18:12:11.063349+0000","last_clean":"2024-01-15T18:12:11.063349+0000","last_became_active":"2024-01-15T18:12:11.061724+0000","last_became_peered":"2024-01-15T18:12:11.061724+0000","last_unstale":"2024-01-15T18:12:11.063349+0000","last_undegraded":"2024-01-15T18:12:11.063349+0000","last_fullsized":"2024-01-15T18:12:11.063349+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:55:59.367011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022870+0000","last_change":"2024-01-15T18:12:11.044940+0000","last_active":"2024-01-15T18:12:12.022870+0000","last_peered":"2024-01-15T18:12:12.022870+0000","last_clean":"2024-01-15T18:12:12.022870+0000","last_became_active":"2024-01-15T18:12:11.043801+0000","last_became_peered":"2024-01-15T18:12:11.043801+0000","last_unstale":"2024-01-15T18:12:12.022870+0000","last_undegraded":"2024-01-15T18:12:12.022870+0000","last_fullsized":"2024-01-15T18:12:12.022870+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:43.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026950+0000","last_change":"2024-01-15T18:12:11.048119+0000","last_active":"2024-01-15T18:12:12.026950+0000","last_peered":"2024-01-15T18:12:12.026950+0000","last_clean":"2024-01-15T18:12:12.026950+0000","last_became_active":"2024-01-15T18:12:11.047739+0000","last_became_peered":"2024-01-15T18:12:11.047739+0000","last_unstale":"2024-01-15T18:12:12.026950+0000","last_undegraded":"2024-01-15T18:12:12.026950+0000","last_fullsized":"2024-01-15T18:12:12.026950+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:49:08.557895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.045331+0000","last_change":"2024-01-15T18:12:11.040703+0000","last_active":"2024-01-15T18:12:11.045331+0000","last_peered":"2024-01-15T18:12:11.045331+0000","last_clean":"2024-01-15T18:12:11.045331+0000","last_became_active":"2024-01-15T18:12:11.039393+0000","last_became_peered":"2024-01-15T18:12:11.039393+0000","last_unstale":"2024-01-15T18:12:11.045331+0000","last_undegraded":"2024-01-15T18:12:11.045331+0000","last_fullsized":"2024-01-15T18:12:11.045331+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.728315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.061046+0000","last_change":"2024-01-15T18:12:11.059603+0000","last_active":"2024-01-15T18:12:11.061046+0000","last_peered":"2024-01-15T18:12:11.061046+0000","last_clean":"2024-01-15T18:12:11.061046+0000","last_became_active":"2024-01-15T18:12:11.059276+0000","last_became_peered":"2024-01-15T18:12:11.059276+0000","last_unstale":"2024-01-15T18:12:11.061046+0000","last_undegraded":"2024-01-15T18:12:11.061046+0000","last_fullsized":"2024-01-15T18:12:11.061046+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:14.826977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058182+0000","last_change":"2024-01-15T18:12:11.053438+0000","last_active":"2024-01-15T18:12:11.058182+0000","last_peered":"2024-01-15T18:12:11.058182+0000","last_clean":"2024-01-15T18:12:11.058182+0000","last_became_active":"2024-01-15T18:12:11.052655+0000","last_became_peered":"2024-01-15T18:12:11.052655+0000","last_unstale":"2024-01-15T18:12:11.058182+0000","last_undegraded":"2024-01-15T18:12:11.058182+0000","last_fullsized":"2024-01-15T18:12:11.058182+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:02:49.378542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058324+0000","last_change":"2024-01-15T18:12:11.053689+0000","last_active":"2024-01-15T18:12:11.058324+0000","last_peered":"2024-01-15T18:12:11.058324+0000","last_clean":"2024-01-15T18:12:11.058324+0000","last_became_active":"2024-01-15T18:12:11.053389+0000","last_became_peered":"2024-01-15T18:12:11.053389+0000","last_unstale":"2024-01-15T18:12:11.058324+0000","last_undegraded":"2024-01-15T18:12:11.058324+0000","last_fullsized":"2024-01-15T18:12:11.058324+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:10.104432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"114'4","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.293911+0000","last_change":"2024-01-15T18:12:10.021097+0000","last_active":"2024-01-15T18:12:11.293911+0000","last_peered":"2024-01-15T18:12:11.293911+0000","last_clean":"2024-01-15T18:12:11.293911+0000","last_became_active":"2024-01-15T18:12:10.020810+0000","last_became_peered":"2024-01-15T18:12:10.020810+0000","last_unstale":"2024-01-15T18:12:11.293911+0000","last_undegraded":"2024-01-15T18:12:11.293911+0000","last_fullsized":"2024-01-15T18:12:11.293911+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:09:27.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.045710+0000","last_change":"2024-01-15T18:12:11.043347+0000","last_active":"2024-01-15T18:12:11.045710+0000","last_peered":"2024-01-15T18:12:11.045710+0000","last_clean":"2024-01-15T18:12:11.045710+0000","last_became_active":"2024-01-15T18:12:11.042427+0000","last_became_peered":"2024-01-15T18:12:11.042427+0000","last_unstale":"2024-01-15T18:12:11.045710+0000","last_undegraded":"2024-01-15T18:12:11.045710+0000","last_fullsized":"2024-01-15T18:12:11.045710+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:19:14.617011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177516+0000","last_change":"2024-01-15T18:12:10.023175+0000","last_active":"2024-01-15T18:12:12.177516+0000","last_peered":"2024-01-15T18:12:12.177516+0000","last_clean":"2024-01-15T18:12:12.177516+0000","last_became_active":"2024-01-15T18:12:10.022654+0000","last_became_peered":"2024-01-15T18:12:10.022654+0000","last_unstale":"2024-01-15T18:12:12.177516+0000","last_undegraded":"2024-01-15T18:12:12.177516+0000","last_fullsized":"2024-01-15T18:12:12.177516+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:12.136270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058266+0000","last_change":"2024-01-15T18:12:11.053138+0000","last_active":"2024-01-15T18:12:11.058266+0000","last_peered":"2024-01-15T18:12:11.058266+0000","last_clean":"2024-01-15T18:12:11.058266+0000","last_became_active":"2024-01-15T18:12:11.052812+0000","last_became_peered":"2024-01-15T18:12:11.052812+0000","last_unstale":"2024-01-15T18:12:11.058266+0000","last_undegraded":"2024-01-15T18:12:11.058266+0000","last_fullsized":"2024-01-15T18:12:11.058266+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:25:16.609507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"114'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026133+0000","last_change":"2024-01-15T18:12:10.018995+0000","last_active":"2024-01-15T18:12:12.026133+0000","last_peered":"2024-01-15T18:12:12.026133+0000","last_clean":"2024-01-15T18:12:12.026133+0000","last_became_active":"2024-01-15T18:12:10.018670+0000","last_became_peered":"2024-01-15T18:12:10.018670+0000","last_unstale":"2024-01-15T18:12:12.026133+0000","last_undegraded":"2024-01-15T18:12:12.026133+0000","last_fullsized":"2024-01-15T18:12:12.026133+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:39.773731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177729+0000","last_change":"2024-01-15T18:12:11.059355+0000","last_active":"2024-01-15T18:12:12.177729+0000","last_peered":"2024-01-15T18:12:12.177729+0000","last_clean":"2024-01-15T18:12:12.177729+0000","last_became_active":"2024-01-15T18:12:11.057592+0000","last_became_peered":"2024-01-15T18:12:11.057592+0000","last_unstale":"2024-01-15T18:12:12.177729+0000","last_undegraded":"2024-01-15T18:12:12.177729+0000","last_fullsized":"2024-01-15T18:12:12.177729+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:14.826977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"114'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025582+0000","last_change":"2024-01-15T18:12:10.020565+0000","last_active":"2024-01-15T18:12:12.025582+0000","last_peered":"2024-01-15T18:12:12.025582+0000","last_clean":"2024-01-15T18:12:12.025582+0000","last_became_active":"2024-01-15T18:12:10.020143+0000","last_became_peered":"2024-01-15T18:12:10.020143+0000","last_unstale":"2024-01-15T18:12:12.025582+0000","last_undegraded":"2024-01-15T18:12:12.025582+0000","last_fullsized":"2024-01-15T18:12:12.025582+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:09.129175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060615+0000","last_change":"2024-01-15T18:12:11.057876+0000","last_active":"2024-01-15T18:12:11.060615+0000","last_peered":"2024-01-15T18:12:11.060615+0000","last_clean":"2024-01-15T18:12:11.060615+0000","last_became_active":"2024-01-15T18:12:11.057471+0000","last_became_peered":"2024-01-15T18:12:11.057471+0000","last_unstale":"2024-01-15T18:12:11.060615+0000","last_undegraded":"2024-01-15T18:12:11.060615+0000","last_fullsized":"2024-01-15T18:12:11.060615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:13.150070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.292871+0000","last_change":"2024-01-15T18:12:10.020217+0000","last_active":"2024-01-15T18:12:11.292871+0000","last_peered":"2024-01-15T18:12:11.292871+0000","last_clean":"2024-01-15T18:12:11.292871+0000","last_became_active":"2024-01-15T18:12:10.019818+0000","last_became_peered":"2024-01-15T18:12:10.019818+0000","last_unstale":"2024-01-15T18:12:11.292871+0000","last_undegraded":"2024-01-15T18:12:11.292871+0000","last_fullsized":"2024-01-15T18:12:11.292871+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027232+0000","last_change":"2024-01-15T18:12:11.050383+0000","last_active":"2024-01-15T18:12:12.027232+0000","last_peered":"2024-01-15T18:12:12.027232+0000","last_clean":"2024-01-15T18:12:12.027232+0000","last_became_active":"2024-01-15T18:12:11.050002+0000","last_became_peered":"2024-01-15T18:12:11.050002+0000","last_unstale":"2024-01-15T18:12:12.027232+0000","last_undegraded":"2024-01-15T18:12:12.027232+0000","last_fullsized":"2024-01-15T18:12:12.027232+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022221+0000","last_change":"2024-01-15T18:12:10.013940+0000","last_active":"2024-01-15T18:12:12.022221+0000","last_peered":"2024-01-15T18:12:12.022221+0000","last_clean":"2024-01-15T18:12:12.022221+0000","last_became_active":"2024-01-15T18:12:10.009676+0000","last_became_peered":"2024-01-15T18:12:10.009676+0000","last_unstale":"2024-01-15T18:12:12.022221+0000","last_undegraded":"2024-01-15T18:12:12.022221+0000","last_fullsized":"2024-01-15T18:12:12.022221+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:00.059047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060665+0000","last_change":"2024-01-15T18:12:11.057984+0000","last_active":"2024-01-15T18:12:11.060665+0000","last_peered":"2024-01-15T18:12:11.060665+0000","last_clean":"2024-01-15T18:12:11.060665+0000","last_became_active":"2024-01-15T18:12:11.057640+0000","last_became_peered":"2024-01-15T18:12:11.057640+0000","last_unstale":"2024-01-15T18:12:11.060665+0000","last_undegraded":"2024-01-15T18:12:11.060665+0000","last_fullsized":"2024-01-15T18:12:11.060665+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026300+0000","last_change":"2024-01-15T18:12:10.002836+0000","last_active":"2024-01-15T18:12:12.026300+0000","last_peered":"2024-01-15T18:12:12.026300+0000","last_clean":"2024-01-15T18:12:12.026300+0000","last_became_active":"2024-01-15T18:12:10.002190+0000","last_became_peered":"2024-01-15T18:12:10.002190+0000","last_unstale":"2024-01-15T18:12:12.026300+0000","last_undegraded":"2024-01-15T18:12:12.026300+0000","last_fullsized":"2024-01-15T18:12:12.026300+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:39.115790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.050296+0000","last_change":"2024-01-15T18:12:11.048560+0000","last_active":"2024-01-15T18:12:11.050296+0000","last_peered":"2024-01-15T18:12:11.050296+0000","last_clean":"2024-01-15T18:12:11.050296+0000","last_became_active":"2024-01-15T18:12:11.046934+0000","last_became_peered":"2024-01-15T18:12:11.046934+0000","last_unstale":"2024-01-15T18:12:11.050296+0000","last_undegraded":"2024-01-15T18:12:11.050296+0000","last_fullsized":"2024-01-15T18:12:11.050296+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:18:56.428372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.042867+0000","last_change":"2024-01-15T18:12:10.020864+0000","last_active":"2024-01-15T18:12:11.042867+0000","last_peered":"2024-01-15T18:12:11.042867+0000","last_clean":"2024-01-15T18:12:11.042867+0000","last_became_active":"2024-01-15T18:12:10.020577+0000","last_became_peered":"2024-01-15T18:12:10.020577+0000","last_unstale":"2024-01-15T18:12:11.042867+0000","last_undegraded":"2024-01-15T18:12:11.042867+0000","last_fullsized":"2024-01-15T18:12:11.042867+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:55.380272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.050337+0000","last_change":"2024-01-15T18:12:11.048003+0000","last_active":"2024-01-15T18:12:11.050337+0000","last_peered":"2024-01-15T18:12:11.050337+0000","last_clean":"2024-01-15T18:12:11.050337+0000","last_became_active":"2024-01-15T18:12:11.047704+0000","last_became_peered":"2024-01-15T18:12:11.047704+0000","last_unstale":"2024-01-15T18:12:11.050337+0000","last_undegraded":"2024-01-15T18:12:11.050337+0000","last_fullsized":"2024-01-15T18:12:11.050337+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:07:57.996851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025881+0000","last_change":"2024-01-15T18:12:10.020498+0000","last_active":"2024-01-15T18:12:12.025881+0000","last_peered":"2024-01-15T18:12:12.025881+0000","last_clean":"2024-01-15T18:12:12.025881+0000","last_became_active":"2024-01-15T18:12:10.020028+0000","last_became_peered":"2024-01-15T18:12:10.020028+0000","last_unstale":"2024-01-15T18:12:12.025881+0000","last_undegraded":"2024-01-15T18:12:12.025881+0000","last_fullsized":"2024-01-15T18:12:12.025881+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.060685+0000","last_change":"2024-01-15T18:12:11.060166+0000","last_active":"2024-01-15T18:12:11.060685+0000","last_peered":"2024-01-15T18:12:11.060685+0000","last_clean":"2024-01-15T18:12:11.060685+0000","last_became_active":"2024-01-15T18:12:11.059393+0000","last_became_peered":"2024-01-15T18:12:11.059393+0000","last_unstale":"2024-01-15T18:12:11.060685+0000","last_undegraded":"2024-01-15T18:12:11.060685+0000","last_fullsized":"2024-01-15T18:12:11.060685+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:07.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.023271+0000","last_change":"2024-01-15T18:12:10.007879+0000","last_active":"2024-01-15T18:12:12.023271+0000","last_peered":"2024-01-15T18:12:12.023271+0000","last_clean":"2024-01-15T18:12:12.023271+0000","last_became_active":"2024-01-15T18:12:10.007549+0000","last_became_peered":"2024-01-15T18:12:10.007549+0000","last_unstale":"2024-01-15T18:12:12.023271+0000","last_undegraded":"2024-01-15T18:12:12.023271+0000","last_fullsized":"2024-01-15T18:12:12.023271+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:33.832426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.023020+0000","last_change":"2024-01-15T18:12:11.031666+0000","last_active":"2024-01-15T18:12:12.023020+0000","last_peered":"2024-01-15T18:12:12.023020+0000","last_clean":"2024-01-15T18:12:12.023020+0000","last_became_active":"2024-01-15T18:12:11.030958+0000","last_became_peered":"2024-01-15T18:12:11.030958+0000","last_unstale":"2024-01-15T18:12:12.023020+0000","last_undegraded":"2024-01-15T18:12:12.023020+0000","last_fullsized":"2024-01-15T18:12:12.023020+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:58.112590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"114'3","reported_seq":14,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.295048+0000","last_change":"2024-01-15T18:12:10.024809+0000","last_active":"2024-01-15T18:12:11.295048+0000","last_peered":"2024-01-15T18:12:11.295048+0000","last_clean":"2024-01-15T18:12:11.295048+0000","last_became_active":"2024-01-15T18:12:10.024120+0000","last_became_peered":"2024-01-15T18:12:10.024120+0000","last_unstale":"2024-01-15T18:12:11.295048+0000","last_undegraded":"2024-01-15T18:12:11.295048+0000","last_fullsized":"2024-01-15T18:12:11.295048+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:17:59.130803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058186+0000","last_change":"2024-01-15T18:12:11.055694+0000","last_active":"2024-01-15T18:12:11.058186+0000","last_peered":"2024-01-15T18:12:11.058186+0000","last_clean":"2024-01-15T18:12:11.058186+0000","last_became_active":"2024-01-15T18:12:11.055290+0000","last_became_peered":"2024-01-15T18:12:11.055290+0000","last_unstale":"2024-01-15T18:12:11.058186+0000","last_undegraded":"2024-01-15T18:12:11.058186+0000","last_fullsized":"2024-01-15T18:12:11.058186+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:55:11.917423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"114'6","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.298411+0000","last_change":"2024-01-15T18:12:10.022950+0000","last_active":"2024-01-15T18:12:11.298411+0000","last_peered":"2024-01-15T18:12:11.298411+0000","last_clean":"2024-01-15T18:12:11.298411+0000","last_became_active":"2024-01-15T18:12:10.021497+0000","last_became_peered":"2024-01-15T18:12:10.021497+0000","last_unstale":"2024-01-15T18:12:11.298411+0000","last_undegraded":"2024-01-15T18:12:11.298411+0000","last_fullsized":"2024-01-15T18:12:11.298411+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:16.556177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022413+0000","last_change":"2024-01-15T18:12:11.046101+0000","last_active":"2024-01-15T18:12:12.022413+0000","last_peered":"2024-01-15T18:12:12.022413+0000","last_clean":"2024-01-15T18:12:12.022413+0000","last_became_active":"2024-01-15T18:12:11.045774+0000","last_became_peered":"2024-01-15T18:12:11.045774+0000","last_unstale":"2024-01-15T18:12:12.022413+0000","last_undegraded":"2024-01-15T18:12:12.022413+0000","last_fullsized":"2024-01-15T18:12:12.022413+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:41:11.078868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"114'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.292297+0000","last_change":"2024-01-15T18:12:10.016912+0000","last_active":"2024-01-15T18:12:11.292297+0000","last_peered":"2024-01-15T18:12:11.292297+0000","last_clean":"2024-01-15T18:12:11.292297+0000","last_became_active":"2024-01-15T18:12:10.016618+0000","last_became_peered":"2024-01-15T18:12:10.016618+0000","last_unstale":"2024-01-15T18:12:11.292297+0000","last_undegraded":"2024-01-15T18:12:11.292297+0000","last_fullsized":"2024-01-15T18:12:11.292297+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:40:48.957828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.022585+0000","last_change":"2024-01-15T18:12:11.051223+0000","last_active":"2024-01-15T18:12:12.022585+0000","last_peered":"2024-01-15T18:12:12.022585+0000","last_clean":"2024-01-15T18:12:12.022585+0000","last_became_active":"2024-01-15T18:12:11.050928+0000","last_became_peered":"2024-01-15T18:12:11.050928+0000","last_unstale":"2024-01-15T18:12:12.022585+0000","last_undegraded":"2024-01-15T18:12:12.022585+0000","last_fullsized":"2024-01-15T18:12:12.022585+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:11.318479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.033672+0000","last_change":"2024-01-15T18:12:10.015868+0000","last_active":"2024-01-15T18:12:11.033672+0000","last_peered":"2024-01-15T18:12:11.033672+0000","last_clean":"2024-01-15T18:12:11.033672+0000","last_became_active":"2024-01-15T18:12:10.015578+0000","last_became_peered":"2024-01-15T18:12:10.015578+0000","last_unstale":"2024-01-15T18:12:11.033672+0000","last_undegraded":"2024-01-15T18:12:11.033672+0000","last_fullsized":"2024-01-15T18:12:11.033672+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.021548+0000","last_change":"2024-01-15T18:12:11.029654+0000","last_active":"2024-01-15T18:12:12.021548+0000","last_peered":"2024-01-15T18:12:12.021548+0000","last_clean":"2024-01-15T18:12:12.021548+0000","last_became_active":"2024-01-15T18:12:11.028865+0000","last_became_peered":"2024-01-15T18:12:11.028865+0000","last_unstale":"2024-01-15T18:12:12.021548+0000","last_undegraded":"2024-01-15T18:12:12.021548+0000","last_fullsized":"2024-01-15T18:12:12.021548+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:29.536846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"114'8","reported_seq":20,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025285+0000","last_change":"2024-01-15T18:12:10.022690+0000","last_active":"2024-01-15T18:12:12.025285+0000","last_peered":"2024-01-15T18:12:12.025285+0000","last_clean":"2024-01-15T18:12:12.025285+0000","last_became_active":"2024-01-15T18:12:10.022398+0000","last_became_peered":"2024-01-15T18:12:10.022398+0000","last_unstale":"2024-01-15T18:12:12.025285+0000","last_undegraded":"2024-01-15T18:12:12.025285+0000","last_fullsized":"2024-01-15T18:12:12.025285+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:52.527066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026275+0000","last_change":"2024-01-15T18:12:11.045910+0000","last_active":"2024-01-15T18:12:12.026275+0000","last_peered":"2024-01-15T18:12:12.026275+0000","last_clean":"2024-01-15T18:12:12.026275+0000","last_became_active":"2024-01-15T18:12:11.043735+0000","last_became_peered":"2024-01-15T18:12:11.043735+0000","last_unstale":"2024-01-15T18:12:12.026275+0000","last_undegraded":"2024-01-15T18:12:12.026275+0000","last_fullsized":"2024-01-15T18:12:12.026275+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.728315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"114'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.291522+0000","last_change":"2024-01-15T18:12:10.027277+0000","last_active":"2024-01-15T18:12:11.291522+0000","last_peered":"2024-01-15T18:12:11.291522+0000","last_clean":"2024-01-15T18:12:11.291522+0000","last_became_active":"2024-01-15T18:12:10.026730+0000","last_became_peered":"2024-01-15T18:12:10.026730+0000","last_unstale":"2024-01-15T18:12:11.291522+0000","last_undegraded":"2024-01-15T18:12:11.291522+0000","last_fullsized":"2024-01-15T18:12:11.291522+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:52.737012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.063299+0000","last_change":"2024-01-15T18:12:11.061270+0000","last_active":"2024-01-15T18:12:11.063299+0000","last_peered":"2024-01-15T18:12:11.063299+0000","last_clean":"2024-01-15T18:12:11.063299+0000","last_became_active":"2024-01-15T18:12:11.061002+0000","last_became_peered":"2024-01-15T18:12:11.061002+0000","last_unstale":"2024-01-15T18:12:11.063299+0000","last_undegraded":"2024-01-15T18:12:11.063299+0000","last_fullsized":"2024-01-15T18:12:11.063299+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:28:45.296624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"114'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026152+0000","last_change":"2024-01-15T18:12:10.020526+0000","last_active":"2024-01-15T18:12:12.026152+0000","last_peered":"2024-01-15T18:12:12.026152+0000","last_clean":"2024-01-15T18:12:12.026152+0000","last_became_active":"2024-01-15T18:12:10.019706+0000","last_became_peered":"2024-01-15T18:12:10.019706+0000","last_unstale":"2024-01-15T18:12:12.026152+0000","last_undegraded":"2024-01-15T18:12:12.026152+0000","last_fullsized":"2024-01-15T18:12:12.026152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:49.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177357+0000","last_change":"2024-01-15T18:12:11.060480+0000","last_active":"2024-01-15T18:12:12.177357+0000","last_peered":"2024-01-15T18:12:12.177357+0000","last_clean":"2024-01-15T18:12:12.177357+0000","last_became_active":"2024-01-15T18:12:11.060154+0000","last_became_peered":"2024-01-15T18:12:11.060154+0000","last_unstale":"2024-01-15T18:12:12.177357+0000","last_undegraded":"2024-01-15T18:12:12.177357+0000","last_fullsized":"2024-01-15T18:12:12.177357+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:52.364460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"114'1","reported_seq":12,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.293549+0000","last_change":"2024-01-15T18:12:10.003308+0000","last_active":"2024-01-15T18:12:11.293549+0000","last_peered":"2024-01-15T18:12:11.293549+0000","last_clean":"2024-01-15T18:12:11.293549+0000","last_became_active":"2024-01-15T18:12:10.002957+0000","last_became_peered":"2024-01-15T18:12:10.002957+0000","last_unstale":"2024-01-15T18:12:11.293549+0000","last_undegraded":"2024-01-15T18:12:11.293549+0000","last_fullsized":"2024-01-15T18:12:11.293549+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:43.777497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.178001+0000","last_change":"2024-01-15T18:12:11.059234+0000","last_active":"2024-01-15T18:12:12.178001+0000","last_peered":"2024-01-15T18:12:12.178001+0000","last_clean":"2024-01-15T18:12:12.178001+0000","last_became_active":"2024-01-15T18:12:11.058945+0000","last_became_peered":"2024-01-15T18:12:11.058945+0000","last_unstale":"2024-01-15T18:12:12.178001+0000","last_undegraded":"2024-01-15T18:12:12.178001+0000","last_fullsized":"2024-01-15T18:12:12.178001+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:37.113206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.036566+0000","last_change":"2024-01-15T18:12:10.028589+0000","last_active":"2024-01-15T18:12:11.036566+0000","last_peered":"2024-01-15T18:12:11.036566+0000","last_clean":"2024-01-15T18:12:11.036566+0000","last_became_active":"2024-01-15T18:12:10.028284+0000","last_became_peered":"2024-01-15T18:12:10.028284+0000","last_unstale":"2024-01-15T18:12:11.036566+0000","last_undegraded":"2024-01-15T18:12:11.036566+0000","last_fullsized":"2024-01-15T18:12:11.036566+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:46:23.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.176920+0000","last_change":"2024-01-15T18:12:11.051787+0000","last_active":"2024-01-15T18:12:12.176920+0000","last_peered":"2024-01-15T18:12:12.176920+0000","last_clean":"2024-01-15T18:12:12.176920+0000","last_became_active":"2024-01-15T18:12:11.051403+0000","last_became_peered":"2024-01-15T18:12:11.051403+0000","last_unstale":"2024-01-15T18:12:12.176920+0000","last_undegraded":"2024-01-15T18:12:12.176920+0000","last_fullsized":"2024-01-15T18:12:12.176920+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:05:35.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"114'2","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.296005+0000","last_change":"2024-01-15T18:12:10.021913+0000","last_active":"2024-01-15T18:12:11.296005+0000","last_peered":"2024-01-15T18:12:11.296005+0000","last_clean":"2024-01-15T18:12:11.296005+0000","last_became_active":"2024-01-15T18:12:10.021344+0000","last_became_peered":"2024-01-15T18:12:10.021344+0000","last_unstale":"2024-01-15T18:12:11.296005+0000","last_undegraded":"2024-01-15T18:12:11.296005+0000","last_fullsized":"2024-01-15T18:12:11.296005+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:00.107213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.061435+0000","last_change":"2024-01-15T18:12:11.059848+0000","last_active":"2024-01-15T18:12:11.061435+0000","last_peered":"2024-01-15T18:12:11.061435+0000","last_clean":"2024-01-15T18:12:11.061435+0000","last_became_active":"2024-01-15T18:12:11.059311+0000","last_became_peered":"2024-01-15T18:12:11.059311+0000","last_unstale":"2024-01-15T18:12:11.061435+0000","last_undegraded":"2024-01-15T18:12:11.061435+0000","last_fullsized":"2024-01-15T18:12:11.061435+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:37.113206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025456+0000","last_change":"2024-01-15T18:12:10.018301+0000","last_active":"2024-01-15T18:12:12.025456+0000","last_peered":"2024-01-15T18:12:12.025456+0000","last_clean":"2024-01-15T18:12:12.025456+0000","last_became_active":"2024-01-15T18:12:10.017411+0000","last_became_peered":"2024-01-15T18:12:10.017411+0000","last_unstale":"2024-01-15T18:12:12.025456+0000","last_undegraded":"2024-01-15T18:12:12.025456+0000","last_fullsized":"2024-01-15T18:12:12.025456+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:32.688781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177614+0000","last_change":"2024-01-15T18:12:11.056765+0000","last_active":"2024-01-15T18:12:12.177614+0000","last_peered":"2024-01-15T18:12:12.177614+0000","last_clean":"2024-01-15T18:12:12.177614+0000","last_became_active":"2024-01-15T18:12:11.056311+0000","last_became_peered":"2024-01-15T18:12:11.056311+0000","last_unstale":"2024-01-15T18:12:12.177614+0000","last_undegraded":"2024-01-15T18:12:12.177614+0000","last_fullsized":"2024-01-15T18:12:12.177614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:53.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.042167+0000","last_change":"2024-01-15T18:12:10.023638+0000","last_active":"2024-01-15T18:12:11.042167+0000","last_peered":"2024-01-15T18:12:11.042167+0000","last_clean":"2024-01-15T18:12:11.042167+0000","last_became_active":"2024-01-15T18:12:10.022687+0000","last_became_peered":"2024-01-15T18:12:10.022687+0000","last_unstale":"2024-01-15T18:12:11.042167+0000","last_undegraded":"2024-01-15T18:12:11.042167+0000","last_fullsized":"2024-01-15T18:12:11.042167+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:54.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.021795+0000","last_change":"2024-01-15T18:12:11.062171+0000","last_active":"2024-01-15T18:12:12.021795+0000","last_peered":"2024-01-15T18:12:12.021795+0000","last_clean":"2024-01-15T18:12:12.021795+0000","last_became_active":"2024-01-15T18:12:11.061830+0000","last_became_peered":"2024-01-15T18:12:11.061830+0000","last_unstale":"2024-01-15T18:12:12.021795+0000","last_undegraded":"2024-01-15T18:12:12.021795+0000","last_fullsized":"2024-01-15T18:12:12.021795+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:50.541819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.043037+0000","last_change":"2024-01-15T18:12:10.024893+0000","last_active":"2024-01-15T18:12:11.043037+0000","last_peered":"2024-01-15T18:12:11.043037+0000","last_clean":"2024-01-15T18:12:11.043037+0000","last_became_active":"2024-01-15T18:12:10.024554+0000","last_became_peered":"2024-01-15T18:12:10.024554+0000","last_unstale":"2024-01-15T18:12:11.043037+0000","last_undegraded":"2024-01-15T18:12:11.043037+0000","last_fullsized":"2024-01-15T18:12:11.043037+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:51.350660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.055716+0000","last_change":"2024-01-15T18:12:11.051179+0000","last_active":"2024-01-15T18:12:11.055716+0000","last_peered":"2024-01-15T18:12:11.055716+0000","last_clean":"2024-01-15T18:12:11.055716+0000","last_became_active":"2024-01-15T18:12:11.050747+0000","last_became_peered":"2024-01-15T18:12:11.050747+0000","last_unstale":"2024-01-15T18:12:11.055716+0000","last_undegraded":"2024-01-15T18:12:11.055716+0000","last_fullsized":"2024-01-15T18:12:11.055716+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:40.129590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"114'2","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.023121+0000","last_change":"2024-01-15T18:12:10.025674+0000","last_active":"2024-01-15T18:12:12.023121+0000","last_peered":"2024-01-15T18:12:12.023121+0000","last_clean":"2024-01-15T18:12:12.023121+0000","last_became_active":"2024-01-15T18:12:10.025356+0000","last_became_peered":"2024-01-15T18:12:10.025356+0000","last_unstale":"2024-01-15T18:12:12.023121+0000","last_undegraded":"2024-01-15T18:12:12.023121+0000","last_fullsized":"2024-01-15T18:12:12.023121+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:06:50.285646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.058998+0000","last_change":"2024-01-15T18:12:11.050465+0000","last_active":"2024-01-15T18:12:11.058998+0000","last_peered":"2024-01-15T18:12:11.058998+0000","last_clean":"2024-01-15T18:12:11.058998+0000","last_became_active":"2024-01-15T18:12:11.050116+0000","last_became_peered":"2024-01-15T18:12:11.050116+0000","last_unstale":"2024-01-15T18:12:11.058998+0000","last_undegraded":"2024-01-15T18:12:11.058998+0000","last_fullsized":"2024-01-15T18:12:11.058998+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:07:28.283888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.035076+0000","last_change":"2024-01-15T18:12:10.020848+0000","last_active":"2024-01-15T18:12:11.035076+0000","last_peered":"2024-01-15T18:12:11.035076+0000","last_clean":"2024-01-15T18:12:11.035076+0000","last_became_active":"2024-01-15T18:12:10.020497+0000","last_became_peered":"2024-01-15T18:12:10.020497+0000","last_unstale":"2024-01-15T18:12:11.035076+0000","last_undegraded":"2024-01-15T18:12:11.035076+0000","last_fullsized":"2024-01-15T18:12:11.035076+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:37.798936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.059490+0000","last_change":"2024-01-15T18:12:11.051721+0000","last_active":"2024-01-15T18:12:11.059490+0000","last_peered":"2024-01-15T18:12:11.059490+0000","last_clean":"2024-01-15T18:12:11.059490+0000","last_became_active":"2024-01-15T18:12:11.050810+0000","last_became_peered":"2024-01-15T18:12:11.050810+0000","last_unstale":"2024-01-15T18:12:11.059490+0000","last_undegraded":"2024-01-15T18:12:11.059490+0000","last_fullsized":"2024-01-15T18:12:11.059490+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.044948+0000","last_change":"2024-01-15T18:12:10.025006+0000","last_active":"2024-01-15T18:12:11.044948+0000","last_peered":"2024-01-15T18:12:11.044948+0000","last_clean":"2024-01-15T18:12:11.044948+0000","last_became_active":"2024-01-15T18:12:10.024524+0000","last_became_peered":"2024-01-15T18:12:10.024524+0000","last_unstale":"2024-01-15T18:12:11.044948+0000","last_undegraded":"2024-01-15T18:12:11.044948+0000","last_fullsized":"2024-01-15T18:12:11.044948+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:16.499218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.062797+0000","last_change":"2024-01-15T18:12:11.061751+0000","last_active":"2024-01-15T18:12:11.062797+0000","last_peered":"2024-01-15T18:12:11.062797+0000","last_clean":"2024-01-15T18:12:11.062797+0000","last_became_active":"2024-01-15T18:12:11.061454+0000","last_became_peered":"2024-01-15T18:12:11.061454+0000","last_unstale":"2024-01-15T18:12:11.062797+0000","last_undegraded":"2024-01-15T18:12:11.062797+0000","last_fullsized":"2024-01-15T18:12:11.062797+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:20.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.023109+0000","last_change":"2024-01-15T18:12:10.018428+0000","last_active":"2024-01-15T18:12:12.023109+0000","last_peered":"2024-01-15T18:12:12.023109+0000","last_clean":"2024-01-15T18:12:12.023109+0000","last_became_active":"2024-01-15T18:12:10.018057+0000","last_became_peered":"2024-01-15T18:12:10.018057+0000","last_unstale":"2024-01-15T18:12:12.023109+0000","last_undegraded":"2024-01-15T18:12:12.023109+0000","last_fullsized":"2024-01-15T18:12:12.023109+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:52:09.544387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026307+0000","last_change":"2024-01-15T18:12:11.039995+0000","last_active":"2024-01-15T18:12:12.026307+0000","last_peered":"2024-01-15T18:12:12.026307+0000","last_clean":"2024-01-15T18:12:12.026307+0000","last_became_active":"2024-01-15T18:12:11.039586+0000","last_became_peered":"2024-01-15T18:12:11.039586+0000","last_unstale":"2024-01-15T18:12:12.026307+0000","last_undegraded":"2024-01-15T18:12:12.026307+0000","last_fullsized":"2024-01-15T18:12:12.026307+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:38.812736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.031847+0000","last_change":"2024-01-15T18:12:10.019758+0000","last_active":"2024-01-15T18:12:11.031847+0000","last_peered":"2024-01-15T18:12:11.031847+0000","last_clean":"2024-01-15T18:12:11.031847+0000","last_became_active":"2024-01-15T18:12:10.019424+0000","last_became_peered":"2024-01-15T18:12:10.019424+0000","last_unstale":"2024-01-15T18:12:11.031847+0000","last_undegraded":"2024-01-15T18:12:11.031847+0000","last_fullsized":"2024-01-15T18:12:11.031847+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:45.797791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.027287+0000","last_change":"2024-01-15T18:12:11.035857+0000","last_active":"2024-01-15T18:12:12.027287+0000","last_peered":"2024-01-15T18:12:12.027287+0000","last_clean":"2024-01-15T18:12:12.027287+0000","last_became_active":"2024-01-15T18:12:11.035238+0000","last_became_peered":"2024-01-15T18:12:11.035238+0000","last_unstale":"2024-01-15T18:12:12.027287+0000","last_undegraded":"2024-01-15T18:12:12.027287+0000","last_fullsized":"2024-01-15T18:12:12.027287+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.026551+0000","last_change":"2024-01-15T18:12:10.017511+0000","last_active":"2024-01-15T18:12:12.026551+0000","last_peered":"2024-01-15T18:12:12.026551+0000","last_clean":"2024-01-15T18:12:12.026551+0000","last_became_active":"2024-01-15T18:12:10.017190+0000","last_became_peered":"2024-01-15T18:12:10.017190+0000","last_unstale":"2024-01-15T18:12:12.026551+0000","last_undegraded":"2024-01-15T18:12:12.026551+0000","last_fullsized":"2024-01-15T18:12:12.026551+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.177633+0000","last_change":"2024-01-15T18:12:11.056667+0000","last_active":"2024-01-15T18:12:12.177633+0000","last_peered":"2024-01-15T18:12:12.177633+0000","last_clean":"2024-01-15T18:12:12.177633+0000","last_became_active":"2024-01-15T18:12:11.056111+0000","last_became_peered":"2024-01-15T18:12:11.056111+0000","last_unstale":"2024-01-15T18:12:12.177633+0000","last_undegraded":"2024-01-15T18:12:12.177633+0000","last_fullsized":"2024-01-15T18:12:12.177633+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:18:00.144603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.025389+0000","last_change":"2024-01-15T18:12:10.021210+0000","last_active":"2024-01-15T18:12:12.025389+0000","last_peered":"2024-01-15T18:12:12.025389+0000","last_clean":"2024-01-15T18:12:12.025389+0000","last_became_active":"2024-01-15T18:12:10.020890+0000","last_became_peered":"2024-01-15T18:12:10.020890+0000","last_unstale":"2024-01-15T18:12:12.025389+0000","last_undegraded":"2024-01-15T18:12:12.025389+0000","last_fullsized":"2024-01-15T18:12:12.025389+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:44.292512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.061011+0000","last_change":"2024-01-15T18:12:11.059132+0000","last_active":"2024-01-15T18:12:11.061011+0000","last_peered":"2024-01-15T18:12:11.061011+0000","last_clean":"2024-01-15T18:12:11.061011+0000","last_became_active":"2024-01-15T18:12:11.057943+0000","last_became_peered":"2024-01-15T18:12:11.057943+0000","last_unstale":"2024-01-15T18:12:11.061011+0000","last_undegraded":"2024-01-15T18:12:11.061011+0000","last_fullsized":"2024-01-15T18:12:11.061011+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:05:35.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.043186+0000","last_change":"2024-01-15T18:12:10.010681+0000","last_active":"2024-01-15T18:12:11.043186+0000","last_peered":"2024-01-15T18:12:11.043186+0000","last_clean":"2024-01-15T18:12:11.043186+0000","last_became_active":"2024-01-15T18:12:10.009870+0000","last_became_peered":"2024-01-15T18:12:10.009870+0000","last_unstale":"2024-01-15T18:12:11.043186+0000","last_undegraded":"2024-01-15T18:12:11.043186+0000","last_fullsized":"2024-01-15T18:12:11.043186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:21.835760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-01-15T18:12:12.021670+0000","last_change":"2024-01-15T18:12:11.051496+0000","last_active":"2024-01-15T18:12:12.021670+0000","last_peered":"2024-01-15T18:12:12.021670+0000","last_clean":"2024-01-15T18:12:12.021670+0000","last_became_active":"2024-01-15T18:12:11.051092+0000","last_became_peered":"2024-01-15T18:12:11.051092+0000","last_unstale":"2024-01-15T18:12:12.021670+0000","last_undegraded":"2024-01-15T18:12:12.021670+0000","last_fullsized":"2024-01-15T18:12:12.021670+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:38:09.219058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.037109+0000","last_change":"2024-01-15T18:12:10.026923+0000","last_active":"2024-01-15T18:12:11.037109+0000","last_peered":"2024-01-15T18:12:11.037109+0000","last_clean":"2024-01-15T18:12:11.037109+0000","last_became_active":"2024-01-15T18:12:10.026651+0000","last_became_peered":"2024-01-15T18:12:10.026651+0000","last_unstale":"2024-01-15T18:12:11.037109+0000","last_undegraded":"2024-01-15T18:12:11.037109+0000","last_fullsized":"2024-01-15T18:12:11.037109+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:14.981958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":246,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-01-15T18:12:11.047580+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:12:11.047580+0000","last_peered":"2024-01-15T18:12:11.047580+0000","last_clean":"2024-01-15T18:12:11.047580+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:12:11.047580+0000","last_undegraded":"2024-01-15T18:12:11.047580+0000","last_fullsized":"2024-01-15T18:12:11.047580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542354,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65020,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94306820,"statfs":{"total":96636764160,"available":96570183680,"internally_reserved":0,"allocated":2015232,"data_stored":1207274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64551370},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2170000000000001,"15min":1.3779999999999999},"min":{"1min":0.84399999999999997,"5min":0.66300000000000003,"15min":0.40400000000000003},"max":{"1min":1.621,"5min":1.825,"15min":19.843},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3109999999999999,"15min":1.6379999999999999},"min":{"1min":0.99399999999999999,"5min":0.64900000000000002,"15min":0.53400000000000003},"max":{"1min":2.1960000000000002,"5min":7.8250000000000002,"15min":38.137},"last":0.997}]},{"osd":1,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.236,"15min":1.4119999999999999},"min":{"1min":0.79500000000000004,"5min":0.67000000000000004,"15min":0.45600000000000002},"max":{"1min":1.6020000000000001,"5min":1.9399999999999999,"15min":19.917000000000002},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.5649999999999999,"15min":1.694},"min":{"1min":0.91000000000000003,"5min":0.81100000000000005,"15min":0.42999999999999999},"max":{"1min":1.825,"5min":28.613,"15min":51.843000000000004},"last":1.5389999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.3300000000000001,"15min":1.4590000000000001},"min":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.437},"max":{"1min":4.9219999999999997,"5min":4.9219999999999997,"15min":12.061},"last":1.181},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.4019999999999999,"15min":1.548},"min":{"1min":0.97399999999999998,"5min":0.78900000000000003,"15min":0.47399999999999998},"max":{"1min":1.837,"5min":8.2479999999999993,"15min":13.478999999999999},"last":1.47}]},{"osd":3,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":2.0510000000000002,"5min":1.49,"15min":1.599},"min":{"1min":0.94799999999999995,"5min":0.82499999999999996,"15min":0.47899999999999998},"max":{"1min":9.8800000000000008,"5min":9.8800000000000008,"15min":16.632000000000001},"last":1.409},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.421,"15min":1.609},"min":{"1min":0.95699999999999996,"5min":0.878,"15min":0.56699999999999995},"max":{"1min":1.702,"5min":2.2440000000000002,"15min":13.439},"last":1.8009999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.494,"15min":1.7330000000000001},"min":{"1min":0.64800000000000002,"5min":0.60699999999999998,"15min":0.50700000000000001},"max":{"1min":1.946,"5min":17.067,"15min":23.042000000000002},"last":1.714},{"interface":"front","average":{"1min":1.9370000000000001,"5min":1.4930000000000001,"15min":1.7569999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":7.6280000000000001,"5min":7.6280000000000001,"15min":38.682000000000002},"last":1.909}]},{"osd":5,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.478,"15min":1.796},"min":{"1min":0.86399999999999999,"5min":0.746,"15min":0.52700000000000002},"max":{"1min":1.786,"5min":2.1440000000000001,"15min":25.006},"last":1.381},{"interface":"front","average":{"1min":1.548,"5min":1.4750000000000001,"15min":1.8260000000000001},"min":{"1min":0.90300000000000002,"5min":0.627,"15min":0.624},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":22.933},"last":1.972}]},{"osd":6,"last update":"Mon Jan 15 18:11:29 2024","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.591,"15min":2.0270000000000001},"min":{"1min":1.0529999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.982,"5min":2.2229999999999999,"15min":23.834},"last":2.081},{"interface":"front","average":{"1min":1.8,"5min":1.6299999999999999,"15min":2.0059999999999998},"min":{"1min":0.82799999999999996,"5min":0.64900000000000002,"15min":0.58299999999999996},"max":{"1min":4.6420000000000003,"5min":4.6420000000000003,"15min":63.932000000000002},"last":2.1640000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542353,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64436,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307404,"statfs":{"total":96636764160,"available":96570781696,"internally_reserved":0,"allocated":1417216,"data_stored":616360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.6830000000000001,"15min":1.502},"min":{"1min":0.71899999999999997,"5min":0.67600000000000005,"15min":0.53100000000000003},"max":{"1min":1.6859999999999999,"5min":31.805,"15min":31.805},"last":1.087},{"interface":"front","average":{"1min":1.284,"5min":1.583,"15min":1.518},"min":{"1min":1.0269999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.5049999999999999,"5min":31.702999999999999,"15min":31.702999999999999},"last":1.01}]},{"osd":1,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3120000000000001,"15min":1.353},"min":{"1min":0.72999999999999998,"5min":0.60999999999999999,"15min":0.55100000000000005},"max":{"1min":1.7190000000000001,"5min":15.064,"15min":15.064},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.1930000000000001,"15min":1.5429999999999999},"min":{"1min":0.89400000000000002,"5min":0.74199999999999999,"15min":0.66200000000000003},"max":{"1min":1.6539999999999999,"5min":4.0579999999999998,"15min":23.495999999999999},"last":1.1579999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.329,"15min":1.6140000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.59399999999999997},"max":{"1min":1.915,"5min":3.9849999999999999,"15min":23.087},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.333,"5min":1.4379999999999999,"15min":1.431},"min":{"1min":0.84799999999999998,"5min":0.67900000000000005,"15min":0.504},"max":{"1min":2.4140000000000001,"5min":14.94,"15min":14.94},"last":1.3360000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.9430000000000001,"5min":1.649,"15min":1.7210000000000001},"min":{"1min":0.90600000000000003,"5min":0.71799999999999997,"15min":0.59899999999999998},"max":{"1min":9.8390000000000004,"5min":15.151999999999999,"15min":15.151999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.5229999999999999,"15min":1.6439999999999999},"min":{"1min":1.042,"5min":0.76100000000000001,"15min":0.70699999999999996},"max":{"1min":9.8900000000000006,"5min":9.8900000000000006,"15min":14.066000000000001},"last":1.284}]},{"osd":4,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.321,"15min":1.6599999999999999},"min":{"1min":0.96699999999999997,"5min":0.56100000000000005,"15min":0.51800000000000002},"max":{"1min":2.383,"5min":3.9169999999999998,"15min":23.327000000000002},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.6539999999999999,"15min":1.6950000000000001},"min":{"1min":0.99399999999999999,"5min":0.61899999999999999,"15min":0.45800000000000002},"max":{"1min":2.6600000000000001,"5min":15.239000000000001,"15min":17.289000000000001},"last":1.4890000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.427,"15min":1.5660000000000001},"min":{"1min":0.88600000000000001,"5min":0.52500000000000002,"15min":0.51600000000000001},"max":{"1min":2.4289999999999998,"5min":4.9260000000000002,"15min":14.176},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.619,"15min":1.863},"min":{"1min":1.431,"5min":0.64400000000000002,"15min":0.54700000000000004},"max":{"1min":2.6190000000000002,"5min":4.1790000000000003,"15min":23.417999999999999},"last":1.6220000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.637,"5min":1.573,"15min":1.7709999999999999},"min":{"1min":1.032,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.8380000000000001,"5min":4.8239999999999998,"15min":21.265000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.6200000000000001,"15min":1.9810000000000001},"min":{"1min":1.1439999999999999,"5min":0.77100000000000002,"15min":0.59699999999999998},"max":{"1min":2.1989999999999998,"5min":4.0220000000000002,"15min":23.738},"last":1.5449999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542353,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58940,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312900,"statfs":{"total":96636764160,"available":96576409600,"internally_reserved":0,"allocated":1421312,"data_stored":616894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.24,"15min":1.2949999999999999},"min":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.47799999999999998},"max":{"1min":4.9530000000000003,"5min":4.9530000000000003,"15min":19.266999999999999},"last":1.032},{"interface":"front","average":{"1min":1.155,"5min":1.177,"15min":1.321},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.51400000000000001},"max":{"1min":1.5820000000000001,"5min":2.1869999999999998,"15min":17.184999999999999},"last":0.96099999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2130000000000001,"15min":1.345},"min":{"1min":0.755,"5min":0.69699999999999995,"15min":0.499},"max":{"1min":1.746,"5min":2.355,"15min":16.297000000000001},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.254,"15min":1.3400000000000001},"min":{"1min":0.90800000000000003,"5min":0.78900000000000003,"15min":0.59099999999999997},"max":{"1min":2.097,"5min":2.097,"15min":9.2379999999999995},"last":1.5569999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.2649999999999999,"15min":1.2749999999999999},"min":{"1min":0.84099999999999997,"5min":0.78600000000000003,"15min":0.621},"max":{"1min":3.516,"5min":3.516,"15min":8.1820000000000004},"last":1.161},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3420000000000001,"15min":1.4430000000000001},"min":{"1min":0.80500000000000005,"5min":0.749,"15min":0.56000000000000005},"max":{"1min":1.7190000000000001,"5min":2.5539999999999998,"15min":16.158999999999999},"last":1.599}]},{"osd":3,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.379,"15min":1.5},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.63700000000000001},"max":{"1min":2.085,"5min":2.5179999999999998,"15min":14.914},"last":1.214},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.403,"15min":1.45},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.63300000000000001},"max":{"1min":1.8400000000000001,"5min":1.8620000000000001,"15min":7.0110000000000001},"last":1.266}]},{"osd":4,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3660000000000001,"15min":1.4970000000000001},"min":{"1min":0.878,"5min":0.80800000000000005,"15min":0.41399999999999998},"max":{"1min":2.3610000000000002,"5min":2.4689999999999999,"15min":16.414999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.516,"5min":1.4339999999999999,"15min":1.623},"min":{"1min":0.84599999999999997,"5min":0.56200000000000006,"15min":0.32800000000000001},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":17.991},"last":1.409}]},{"osd":6,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.409,"15min":1.4690000000000001},"min":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.39600000000000002},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":6.4630000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.48,"15min":1.5860000000000001},"min":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.53000000000000003},"max":{"1min":1.7989999999999999,"5min":2.7010000000000001,"15min":16.364000000000001},"last":1.4750000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:25 2024","interfaces":[{"interface":"back","average":{"1min":1.6000000000000001,"5min":1.5529999999999999,"15min":1.6120000000000001},"min":{"1min":0.99399999999999999,"5min":0.78900000000000003,"15min":0.45800000000000002},"max":{"1min":2.258,"5min":2.399,"15min":9.0600000000000005},"last":1.673},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5740000000000001,"15min":1.6830000000000001},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.57299999999999995},"max":{"1min":2.0750000000000002,"5min":2.6059999999999999,"15min":19.356999999999999},"last":1.5229999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542353,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64452,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94307388,"statfs":{"total":96636764160,"available":96570765312,"internally_reserved":0,"allocated":1425408,"data_stored":622674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.3999999999999999,"15min":1.395},"min":{"1min":0.82599999999999996,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.1219999999999999,"5min":17.346,"15min":17.346},"last":1.248},{"interface":"front","average":{"1min":1.304,"5min":1.3109999999999999,"15min":1.387},"min":{"1min":0.98399999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.6479999999999999,"5min":2.7919999999999998,"15min":9.7439999999999998},"last":1.21}]},{"osd":1,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1919999999999999,"15min":1.335},"min":{"1min":0.76200000000000001,"5min":0.439,"15min":0.439},"max":{"1min":1.554,"5min":2.8700000000000001,"15min":11.987},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1870000000000001,"15min":1.29},"min":{"1min":0.82599999999999996,"5min":0.67500000000000004,"15min":0.55300000000000005},"max":{"1min":1.6259999999999999,"5min":3.7370000000000001,"15min":16.058},"last":0.78900000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.407,"15min":1.512},"min":{"1min":0.85799999999999998,"5min":0.64500000000000002,"15min":0.56899999999999995},"max":{"1min":1.6499999999999999,"5min":18.442,"15min":18.442},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3300000000000001,"15min":1.524},"min":{"1min":0.997,"5min":0.81899999999999995,"15min":0.69499999999999995},"max":{"1min":1.744,"5min":3.8250000000000002,"15min":13.903},"last":1.1459999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.488,"15min":1.6240000000000001},"min":{"1min":0.96299999999999997,"5min":0.69999999999999996,"15min":0.56699999999999995},"max":{"1min":2.0179999999999998,"5min":17.364999999999998,"15min":26.055},"last":1.361},{"interface":"front","average":{"1min":1.9179999999999999,"5min":1.45,"15min":1.4770000000000001},"min":{"1min":0.97699999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":9.3320000000000007,"5min":9.3320000000000007,"15min":13.25},"last":1.4350000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4339999999999999,"15min":1.5569999999999999},"min":{"1min":0.85799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.8080000000000001,"5min":17.292999999999999,"15min":29.422000000000001},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.4219999999999999,"15min":1.4950000000000001},"min":{"1min":0.64000000000000001,"5min":0.53700000000000003,"15min":0.42199999999999999},"max":{"1min":2.1709999999999998,"5min":4.9400000000000004,"15min":12.332000000000001},"last":1.3129999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3540000000000001,"15min":1.502},"min":{"1min":0.79100000000000004,"5min":0.54200000000000004,"15min":0.45200000000000001},"max":{"1min":1.865,"5min":5.3150000000000004,"15min":18.260000000000002},"last":1.7350000000000001},{"interface":"front","average":{"1min":1.9239999999999999,"5min":1.7390000000000001,"15min":1.7170000000000001},"min":{"1min":0.80700000000000005,"5min":0.51100000000000001,"15min":0.48099999999999998},"max":{"1min":7.1920000000000002,"5min":17.224,"15min":17.224},"last":1.6060000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:24 2024","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.538,"15min":1.7290000000000001},"min":{"1min":0.69799999999999995,"5min":0.625,"15min":0.503},"max":{"1min":2.5640000000000001,"5min":3.0680000000000001,"15min":12.247},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.647,"5min":1.7629999999999999,"15min":1.865},"min":{"1min":0.752,"5min":0.61799999999999999,"15min":0.53600000000000003},"max":{"1min":2.2229999999999999,"5min":17.324999999999999,"15min":17.324999999999999},"last":1.7010000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542355,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72768,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299072,"statfs":{"total":96636764160,"available":96562249728,"internally_reserved":0,"allocated":1429504,"data_stored":617972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73071047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.077,"15min":1.234},"min":{"1min":0.54900000000000004,"5min":0.39900000000000002,"15min":0.33300000000000002},"max":{"1min":3.1930000000000001,"5min":9.2899999999999991,"15min":18.399999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1279999999999999,"15min":1.3959999999999999},"min":{"1min":0.64500000000000002,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":1.917,"5min":2.1699999999999999,"15min":21.047000000000001},"last":0.91700000000000004}]},{"osd":1,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.91800000000000004,"15min":1.274},"min":{"1min":0.57499999999999996,"5min":0.34899999999999998,"15min":0.32600000000000001},"max":{"1min":1.585,"5min":1.585,"15min":17.777999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3400000000000001,"15min":1.3029999999999999},"min":{"1min":0.73199999999999998,"5min":0.46300000000000002,"15min":0.39500000000000002},"max":{"1min":3.3119999999999998,"5min":11.228999999999999,"15min":11.228999999999999},"last":1.679}]},{"osd":2,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.238,"15min":1.288},"min":{"1min":0.59899999999999998,"5min":0.48499999999999999,"15min":0.375},"max":{"1min":2.109,"5min":2.7519999999999998,"15min":10.964},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.579,"5min":1.5149999999999999,"15min":1.5680000000000001},"min":{"1min":0.72399999999999998,"5min":0.57499999999999996,"15min":0.54000000000000004},"max":{"1min":3.8969999999999998,"5min":9.4920000000000009,"15min":20.495000000000001},"last":1.4490000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3460000000000001,"15min":1.5589999999999999},"min":{"1min":0.93100000000000005,"5min":0.67200000000000004,"15min":0.63500000000000001},"max":{"1min":2.0289999999999999,"5min":2.6179999999999999,"15min":13.536},"last":1.353},{"interface":"front","average":{"1min":1.458,"5min":1.2929999999999999,"15min":1.643},"min":{"1min":0.84199999999999997,"5min":0.63700000000000001,"15min":0.54600000000000004},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":16.398},"last":1.1259999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.855,"5min":1.627,"15min":1.716},"min":{"1min":1.288,"5min":0.875,"15min":0.57299999999999995},"max":{"1min":5.2069999999999999,"5min":9.5690000000000008,"15min":20.606000000000002},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.7949999999999999,"5min":1.5740000000000001,"15min":1.6759999999999999},"min":{"1min":1.101,"5min":0.84399999999999997,"15min":0.68000000000000005},"max":{"1min":5.1470000000000002,"5min":5.1470000000000002,"15min":11.603},"last":1.5409999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6579999999999999,"5min":1.431,"15min":1.7949999999999999},"min":{"1min":1.0449999999999999,"5min":0.61399999999999999,"15min":0.53500000000000003},"max":{"1min":3.657,"5min":3.657,"15min":13.648999999999999},"last":1.401},{"interface":"front","average":{"1min":1.746,"5min":1.645,"15min":1.827},"min":{"1min":1.093,"5min":0.78300000000000003,"15min":0.65800000000000003},"max":{"1min":3.4740000000000002,"5min":9.6639999999999997,"15min":20.556000000000001},"last":1.7450000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6879999999999999,"5min":1.5569999999999999,"15min":1.8480000000000001},"min":{"1min":1.048,"5min":0.78800000000000003,"15min":0.60499999999999998},"max":{"1min":2.2490000000000001,"5min":2.9399999999999999,"15min":28.843},"last":1.643},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.5049999999999999,"15min":1.855},"min":{"1min":1.0880000000000001,"5min":0.72799999999999998,"15min":0.67200000000000004},"max":{"1min":2.4409999999999998,"5min":2.4409999999999998,"15min":15.143000000000001},"last":1.5009999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542355,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68604,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303236,"statfs":{"total":96636764160,"available":96566513664,"internally_reserved":0,"allocated":1413120,"data_stored":616290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98199999999999998,"15min":1.093},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.26600000000000001},"max":{"1min":2.5270000000000001,"5min":2.9359999999999999,"15min":14.427},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.292,"5min":1.161,"15min":1.367},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.39300000000000002},"max":{"1min":2.0489999999999999,"5min":2.25,"15min":15.314},"last":1.4650000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.204,"15min":1.4379999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":2.1429999999999998,"5min":2.7029999999999998,"15min":34.463999999999999},"last":1},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.2729999999999999,"15min":1.4650000000000001},"min":{"1min":0.71999999999999997,"5min":0.54200000000000004,"15min":0.373},"max":{"1min":2.3039999999999998,"5min":2.3929999999999998,"15min":25.859999999999999},"last":1.9630000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2969999999999999,"15min":1.4259999999999999},"min":{"1min":0.54300000000000004,"5min":0.48599999999999999,"15min":0.34399999999999997},"max":{"1min":2.2400000000000002,"5min":12.449999999999999,"15min":19.535},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.5549999999999999,"5min":1.371,"15min":1.623},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.46400000000000002},"max":{"1min":2.7869999999999999,"5min":2.8050000000000002,"15min":22.783999999999999},"last":1.2509999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.8069999999999999,"5min":1.518,"15min":1.7869999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.55700000000000005},"max":{"1min":5.4610000000000003,"5min":5.4610000000000003,"15min":29.986999999999998},"last":1.548},{"interface":"front","average":{"1min":1.7310000000000001,"5min":1.5149999999999999,"15min":1.7529999999999999},"min":{"1min":1.05,"5min":0.81000000000000005,"15min":0.56799999999999995},"max":{"1min":2.7450000000000001,"5min":2.7450000000000001,"15min":21.199999999999999},"last":1.327}]},{"osd":5,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.744,"5min":1.774,"15min":1.756},"min":{"1min":0.91100000000000003,"5min":0.84299999999999997,"15min":0.51500000000000001},"max":{"1min":4.2329999999999997,"5min":14.066000000000001,"15min":19.681999999999999},"last":1.77},{"interface":"front","average":{"1min":1.9199999999999999,"5min":1.8280000000000001,"15min":1.841},"min":{"1min":1.028,"5min":0.88700000000000001,"15min":0.60899999999999999},"max":{"1min":4.3079999999999998,"5min":14.025,"15min":14.702999999999999},"last":1.4370000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.841,"5min":1.603,"15min":1.8109999999999999},"min":{"1min":1.093,"5min":0.76800000000000002,"15min":0.68000000000000005},"max":{"1min":3.4590000000000001,"5min":3.4590000000000001,"15min":15.683999999999999},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.8089999999999999,"5min":1.762,"15min":1.8919999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.58899999999999997},"max":{"1min":3.0089999999999999,"5min":12.629,"15min":19.783000000000001},"last":1.867}]},{"osd":7,"last update":"Mon Jan 15 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":2.5699999999999998,"5min":1.8440000000000001,"15min":2.133},"min":{"1min":1.204,"5min":0.86899999999999999,"15min":0.63100000000000001},"max":{"1min":15.738,"5min":15.738,"15min":57.006999999999998},"last":1.833},{"interface":"front","average":{"1min":1.806,"5min":1.8220000000000001,"15min":2.2269999999999999},"min":{"1min":0.998,"5min":0.998,"15min":0.56100000000000005},"max":{"1min":2.887,"5min":12.754,"15min":22.853999999999999},"last":2.016}]}]},{"osd":1,"up_from":14,"seq":60129542354,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72772,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94299068,"statfs":{"total":96636764160,"available":96562245632,"internally_reserved":0,"allocated":1421312,"data_stored":616414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.016,"15min":1.23},"min":{"1min":0.70099999999999996,"5min":0.48899999999999999,"15min":0.34399999999999997},"max":{"1min":1.9399999999999999,"5min":2.6139999999999999,"15min":30.460999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.97,"5min":1.3360000000000001,"15min":1.218},"min":{"1min":0.93100000000000005,"5min":0.38700000000000001,"15min":0.314},"max":{"1min":17.157,"5min":17.157,"15min":18.658000000000001},"last":0.95999999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.202,"15min":1.2629999999999999},"min":{"1min":0.92100000000000004,"5min":0.51800000000000002,"15min":0.376},"max":{"1min":2.5209999999999999,"5min":4.7119999999999997,"15min":21.126000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.419,"5min":1.3799999999999999,"15min":1.595},"min":{"1min":0.67000000000000004,"5min":0.54800000000000004,"15min":0.48699999999999999},"max":{"1min":2.2690000000000001,"5min":3.0409999999999999,"15min":30.593},"last":1.4570000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.3340000000000001,"15min":1.5009999999999999},"min":{"1min":0.76800000000000002,"5min":0.60799999999999998,"15min":0.48599999999999999},"max":{"1min":2.5019999999999998,"5min":3.1400000000000001,"15min":30.547000000000001},"last":1.153},{"interface":"front","average":{"1min":2.0870000000000002,"5min":1.405,"15min":1.4079999999999999},"min":{"1min":0.73799999999999999,"5min":0.52700000000000002,"15min":0.437},"max":{"1min":17.263000000000002,"5min":17.263000000000002,"15min":17.263000000000002},"last":1.0700000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":2.3730000000000002,"5min":1.716,"15min":1.718},"min":{"1min":1.1060000000000001,"5min":0.75900000000000001,"15min":0.72499999999999998},"max":{"1min":17.420000000000002,"5min":17.420000000000002,"15min":17.420000000000002},"last":1.706},{"interface":"front","average":{"1min":1.7230000000000001,"5min":1.569,"15min":1.611},"min":{"1min":1.2749999999999999,"5min":0.72999999999999998,"15min":0.69299999999999995},"max":{"1min":2.9569999999999999,"5min":4.6369999999999996,"15min":14.851000000000001},"last":1.603}]},{"osd":5,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.8660000000000001,"5min":1.593,"15min":1.583},"min":{"1min":1.1539999999999999,"5min":0.77600000000000002,"15min":0.59599999999999997},"max":{"1min":9.0440000000000005,"5min":9.0440000000000005,"15min":11.552},"last":1.7629999999999999},{"interface":"front","average":{"1min":1.73,"5min":1.573,"15min":1.9450000000000001},"min":{"1min":1.2829999999999999,"5min":0.78700000000000003,"15min":0.68300000000000005},"max":{"1min":2.4750000000000001,"5min":2.8119999999999998,"15min":34.042999999999999},"last":1.5600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.734,"5min":1.659,"15min":1.742},"min":{"1min":1.071,"5min":0.83899999999999997,"15min":0.64100000000000001},"max":{"1min":3.8580000000000001,"5min":10.250999999999999,"15min":34.158000000000001},"last":1.9139999999999999},{"interface":"front","average":{"1min":2.4820000000000002,"5min":1.768,"15min":1.6970000000000001},"min":{"1min":1.1870000000000001,"5min":0.67900000000000005,"15min":0.62},"max":{"1min":17.510000000000002,"5min":17.510000000000002,"15min":17.510000000000002},"last":1.506}]},{"osd":7,"last update":"Mon Jan 15 18:11:31 2024","interfaces":[{"interface":"back","average":{"1min":1.869,"5min":1.6950000000000001,"15min":1.899},"min":{"1min":1.2130000000000001,"5min":0.77300000000000002,"15min":0.64600000000000002},"max":{"1min":2.5899999999999999,"5min":3.7709999999999999,"15min":33.659999999999997},"last":1.657},{"interface":"front","average":{"1min":1.853,"5min":1.6919999999999999,"15min":1.8320000000000001},"min":{"1min":1.2110000000000001,"5min":0.81000000000000005,"15min":0.60499999999999998},"max":{"1min":2.8479999999999999,"5min":4.8380000000000001,"15min":24.289999999999999},"last":1.847}]}]},{"osd":0,"up_from":14,"seq":60129542355,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69192,"kb_used_data":1980,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302648,"statfs":{"total":96636764160,"available":96565911552,"internally_reserved":0,"allocated":2027520,"data_stored":1208382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.2609999999999999,"15min":1.103},"min":{"1min":0.47499999999999998,"5min":0.42799999999999999,"15min":0.312},"max":{"1min":2.5510000000000002,"5min":12.050000000000001,"15min":12.050000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.4450000000000001,"15min":1.3480000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.36199999999999999},"max":{"1min":3.1019999999999999,"5min":11.962,"15min":11.962},"last":1.1950000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.177,"15min":1.2190000000000001},"min":{"1min":0.495,"5min":0.47399999999999998,"15min":0.27600000000000002},"max":{"1min":2.1509999999999998,"5min":7.4560000000000004,"15min":11.59},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.1419999999999999,"15min":1.1859999999999999},"min":{"1min":0.46100000000000002,"5min":0.38500000000000001,"15min":0.35899999999999999},"max":{"1min":2.0150000000000001,"5min":7.3239999999999998,"15min":14.512},"last":1.2729999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.4059999999999999,"15min":1.4319999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.41699999999999998},"max":{"1min":1.865,"5min":11.904,"15min":11.904},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.153,"15min":1.262},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.29599999999999999},"max":{"1min":2.657,"5min":3.202,"15min":14.74},"last":0.70699999999999996}]},{"osd":4,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.387,"15min":1.6000000000000001},"min":{"1min":0.84399999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.1949999999999998,"5min":7.4000000000000004,"15min":32.115000000000002},"last":1.117},{"interface":"front","average":{"1min":1.46,"5min":1.548,"15min":1.7},"min":{"1min":0.82399999999999995,"5min":0.70499999999999996,"15min":0.67700000000000005},"max":{"1min":2.1269999999999998,"5min":9.0790000000000006,"15min":15.975},"last":1.53}]},{"osd":5,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5900000000000001,"15min":1.7350000000000001},"min":{"1min":0.86199999999999999,"5min":0.58699999999999997,"15min":0.47499999999999998},"max":{"1min":3.25,"5min":12.006,"15min":32.174999999999997},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.4830000000000001,"15min":1.633},"min":{"1min":0.82699999999999996,"5min":0.57799999999999996,"15min":0.442},"max":{"1min":2.9769999999999999,"5min":4.7119999999999997,"15min":17.225000000000001},"last":0.78200000000000003}]},{"osd":6,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.7529999999999999,"15min":1.8},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.8809999999999998,"5min":24.039999999999999,"15min":24.039999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.5800000000000001,"5min":1.6200000000000001,"15min":1.804},"min":{"1min":0.89700000000000002,"5min":0.73299999999999998,"15min":0.625},"max":{"1min":2.613,"5min":7.5359999999999996,"15min":32.216999999999999},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:11:22 2024","interfaces":[{"interface":"back","average":{"1min":1.6080000000000001,"5min":1.857,"15min":2.121},"min":{"1min":0.86199999999999999,"5min":0.754,"15min":0.69099999999999995},"max":{"1min":2.5059999999999998,"5min":26.427,"15min":34.131},"last":1.431},{"interface":"front","average":{"1min":1.5880000000000001,"5min":2.0699999999999998,"15min":2.0419999999999998},"min":{"1min":0.84099999999999997,"5min":0.753,"15min":0.61799999999999999},"max":{"1min":2.7799999999999998,"5min":44.694000000000003,"15min":44.694000000000003},"last":1.042}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:12:16.271 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-15T18:12:16.271 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-01-15T18:12:16.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.543+0000 7f813de00640 1 Processor -- start 2024-01-15T18:12:16.547 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.544+0000 7f813de00640 1 -- start start 2024-01-15T18:12:16.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.545+0000 7f813de00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f8138157dc0 0x7f8138070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.545+0000 7f813de00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813815ba40 con 0x7f8138071230 2024-01-15T18:12:16.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.546+0000 7f813de00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813815eb10 con 0x7f813815e450 2024-01-15T18:12:16.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.546+0000 7f813de00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81381581a0 con 0x7f8138157dc0 2024-01-15T18:12:16.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.546+0000 7f8136000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8138071230 0x7f813815dd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36482/0 (socket says 172.21.15.26:36482) 2024-01-15T18:12:16.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.546+0000 7f8136000640 1 -- 172.21.15.26:0/3723702544 learned_addr learned my addr 172.21.15.26:0/3723702544 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:16.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.546+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551049704 0 0) 0x7f813815ba40 con 0x7f8138071230 2024-01-15T18:12:16.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8118003520 con 0x7f8138071230 2024-01-15T18:12:16.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8136a00640 1 --2- 172.21.15.26:0/3723702544 >> v2:172.21.15.124:3300/0 conn(0x7f8138157dc0 0x7f8138070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 537807690 0 0) 0x7f8118003520 con 0x7f8138071230 2024-01-15T18:12:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 >> v2:172.21.15.124:3300/0 conn(0x7f8138157dc0 msgr2=0x7f8138070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 --2- 172.21.15.26:0/3723702544 >> v2:172.21.15.124:3300/0 conn(0x7f8138157dc0 0x7f8138070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 >> v1:172.21.15.124:6789/0 conn(0x7f813815e450 legacy=0x7f813815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8138071860 con 0x7f8138071230 2024-01-15T18:12:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f81200019f0 con 0x7f8138071230 2024-01-15T18:12:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 >> v1:172.21.15.26:6789/0 conn(0x7f8138071230 legacy=0x7f813815dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8118003970 0x7f8118003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8118004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8118005180 0x7f811800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81381581a0 con 0x7f81180047c0 2024-01-15T18:12:16.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811800c0d0 con 0x7f8118003970 2024-01-15T18:12:16.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811800c290 con 0x7f8118005180 2024-01-15T18:12:16.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.547+0000 7f8136a00640 1 --2- 172.21.15.26:0/3723702544 >> v2:172.21.15.124:3300/0 conn(0x7f8138157dc0 0x7f8138070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:16.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8136000640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8118004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8136000640 1 -- 172.21.15.26:0/3723702544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8118005180 msgr2=0x7f811800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:16.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8137400640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8118005180 0x7f811800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8136a00640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8118003970 0x7f8118003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8136000640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8118005180 0x7f811800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8136000640 1 -- 172.21.15.26:0/3723702544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8118003970 msgr2=0x7f8118003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:16.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8136000640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8118003970 0x7f8118003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8136000640 1 -- 172.21.15.26:0/3723702544 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8118003520 con 0x7f81180047c0 2024-01-15T18:12:16.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8137400640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8118005180 0x7f811800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:16.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.548+0000 7f8137400640 1 -- 172.21.15.26:0/3723702544 reap_dead start 2024-01-15T18:12:16.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.549+0000 7f8136000640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8118004c40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f812000a830 tx=0x7f81180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ad3929e656c4f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:16.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.550+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81200072d0 con 0x7f81180047c0 2024-01-15T18:12:16.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.550+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812002f070 con 0x7f81180047c0 2024-01-15T18:12:16.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.550+0000 7f8135600640 1 -- 172.21.15.26:0/3723702544 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120007690 con 0x7f81180047c0 2024-01-15T18:12:16.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.550+0000 7f813de00640 1 -- 172.21.15.26:0/3723702544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 msgr2=0x7f8118004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.550+0000 7f813de00640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8118004c40 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f812000a830 tx=0x7f81180057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.550+0000 7f813de00640 1 -- 172.21.15.26:0/3723702544 shutdown_connections 2024-01-15T18:12:16.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.550+0000 7f813de00640 1 --2- 172.21.15.26:0/3723702544 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8118004c40 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.550+0000 7f813de00640 1 -- 172.21.15.26:0/3723702544 >> 172.21.15.26:0/3723702544 conn(0x7f81380b0be0 msgr2=0x7f81380b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:16.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.550+0000 7f813de00640 1 -- 172.21.15.26:0/3723702544 shutdown_connections 2024-01-15T18:12:16.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.551+0000 7f813de00640 1 -- 172.21.15.26:0/3723702544 wait complete. 2024-01-15T18:12:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.552+0000 7f813de00640 1 Processor -- start 2024-01-15T18:12:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.552+0000 7f813de00640 1 -- start start 2024-01-15T18:12:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.553+0000 7f813de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8138148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.553+0000 7f813de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8138153720 0x7f8138158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.553+0000 7f813de00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8138153ba0 0x7f8138158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.553+0000 7f813de00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8138149070 con 0x7f81180047c0 2024-01-15T18:12:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.553+0000 7f813de00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8138151ef0 con 0x7f8138153ba0 2024-01-15T18:12:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.553+0000 7f813de00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8138154ff0 con 0x7f8138153720 2024-01-15T18:12:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8138148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8138148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34764/0 (socket says 172.21.15.26:34764) 2024-01-15T18:12:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136a00640 1 -- 172.21.15.26:0/2637856478 learned_addr learned my addr 172.21.15.26:0/2637856478 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136000640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8138153720 0x7f8138158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136a00640 1 -- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8138153720 msgr2=0x7f8138158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136a00640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8138153720 0x7f8138158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136a00640 1 -- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8138153ba0 msgr2=0x7f8138158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136a00640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8138153ba0 0x7f8138158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136a00640 1 -- 172.21.15.26:0/2637856478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81381543e0 con 0x7f81180047c0 2024-01-15T18:12:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.554+0000 7f8136a00640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8138148b30 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f812800f5a0 tx=0x7f812800fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.555+0000 7f812f400640 1 -- 172.21.15.26:0/2637856478 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128014890 con 0x7f81180047c0 2024-01-15T18:12:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.555+0000 7f812f400640 1 -- 172.21.15.26:0/2637856478 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f812800c030 con 0x7f81180047c0 2024-01-15T18:12:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.555+0000 7f812f400640 1 -- 172.21.15.26:0/2637856478 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128014c30 con 0x7f81180047c0 2024-01-15T18:12:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.555+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81381546c0 con 0x7f81180047c0 2024-01-15T18:12:16.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.555+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8138154c80 con 0x7f81180047c0 2024-01-15T18:12:16.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.556+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8138067d70 con 0x7f81180047c0 2024-01-15T18:12:16.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.557+0000 7f812f400640 1 -- 172.21.15.26:0/2637856478 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f81280133e0 con 0x7f81180047c0 2024-01-15T18:12:16.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.558+0000 7f812f400640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f811006c430 0x7f811006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:16.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.558+0000 7f812f400640 1 -- 172.21.15.26:0/2637856478 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f8128088f30 con 0x7f81180047c0 2024-01-15T18:12:16.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.559+0000 7f8136000640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f811006c430 0x7f811006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:16.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.560+0000 7f8136000640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f811006c430 0x7f811006e8f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8120034000 tx=0x7f8120007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:16.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.563+0000 7f812f400640 1 -- 172.21.15.26:0/2637856478 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f81280554a0 con 0x7f81180047c0 2024-01-15T18:12:16.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.833+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8138067670 con 0x7f81180047c0 2024-01-15T18:12:16.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.837+0000 7f812f400640 1 -- 172.21.15.26:0/2637856478 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f81280220e0 con 0x7f81180047c0 2024-01-15T18:12:16.840 INFO:teuthology.orchestra.run.smithi026.stdout:[client.0] 2024-01-15T18:12:16.840 INFO:teuthology.orchestra.run.smithi026.stdout: key = AQA4caVl7gLrKxAAp17tmb+u2/LGfyqj2B5/ZA== 2024-01-15T18:12:16.840 INFO:teuthology.orchestra.run.smithi026.stdout: caps mds = "allow" 2024-01-15T18:12:16.841 INFO:teuthology.orchestra.run.smithi026.stdout: caps mon = "allow r" 2024-01-15T18:12:16.841 INFO:teuthology.orchestra.run.smithi026.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-01-15T18:12:16.841 INFO:teuthology.orchestra.run.smithi026.stderr:updated caps for client.0 2024-01-15T18:12:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f811006c430 msgr2=0x7f811006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f811006c430 0x7f811006e8f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8120034000 tx=0x7f8120007000 comp rx=0 tx=0).stop 2024-01-15T18:12:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 msgr2=0x7f8138148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8138148b30 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f812800f5a0 tx=0x7f812800fa60 comp rx=0 tx=0).stop 2024-01-15T18:12:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 shutdown_connections 2024-01-15T18:12:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8138153720 0x7f8138158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f811006c430 0x7f811006e8f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8138153ba0 0x7f8138158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 --2- 172.21.15.26:0/2637856478 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f81180047c0 0x7f8138148b30 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:16.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.840+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 >> 172.21.15.26:0/2637856478 conn(0x7f81380b0be0 msgr2=0x7f813806cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:16.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.841+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 shutdown_connections 2024-01-15T18:12:16.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:16.841+0000 7f813de00640 1 -- 172.21.15.26:0/2637856478 wait complete. 2024-01-15T18:12:16.864 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:12:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.080+0000 7fde64600640 1 Processor -- start 2024-01-15T18:12:17.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.080+0000 7fde64600640 1 -- start start 2024-01-15T18:12:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.082+0000 7fde64600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde5c157c30 0x7fde5c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.082+0000 7fde64600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c15deb0 con 0x7fde5c1618f0 2024-01-15T18:12:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.082+0000 7fde64600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c15e190 con 0x7fde5c071200 2024-01-15T18:12:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.082+0000 7fde64600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5c15ea30 con 0x7fde5c157c30 2024-01-15T18:12:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.082+0000 7fde61c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde5c157c30 0x7fde5c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:17.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.082+0000 7fde61c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fde5c157c30 0x7fde5c0cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34766/0 (socket says 172.21.15.26:34766) 2024-01-15T18:12:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.082+0000 7fde61c00640 1 -- 172.21.15.26:0/1172361857 learned_addr learned my addr 172.21.15.26:0/1172361857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.083+0000 7fde61c00640 1 -- 172.21.15.26:0/1172361857 >> v1:172.21.15.124:6789/0 conn(0x7fde5c071200 legacy=0x7fde5c06dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.083+0000 7fde5be00640 1 -- 172.21.15.26:0/1172361857 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45176959 0 0) 0x7fde5c15deb0 con 0x7fde5c1618f0 2024-01-15T18:12:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.083+0000 7fde61c00640 1 -- 172.21.15.26:0/1172361857 >> v1:172.21.15.26:6789/0 conn(0x7fde5c1618f0 legacy=0x7fde5c070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.083+0000 7fde61c00640 1 -- 172.21.15.26:0/1172361857 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c15e470 con 0x7fde5c157c30 2024-01-15T18:12:17.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.085+0000 7fde61c00640 1 --2- 172.21.15.26:0/1172361857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c157c30 0x7fde5c0cd600 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009870 tx=0x7fde4c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bc9d14b6eaa43d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.085+0000 7fde5be00640 1 -- 172.21.15.26:0/1172361857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c02f840 con 0x7fde5c157c30 2024-01-15T18:12:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.085+0000 7fde5be00640 1 -- 172.21.15.26:0/1172361857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c02f9a0 con 0x7fde5c157c30 2024-01-15T18:12:17.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.085+0000 7fde5be00640 1 -- 172.21.15.26:0/1172361857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c02fd30 con 0x7fde5c157c30 2024-01-15T18:12:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.085+0000 7fde64600640 1 -- 172.21.15.26:0/1172361857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c157c30 msgr2=0x7fde5c0cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.085+0000 7fde64600640 1 --2- 172.21.15.26:0/1172361857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c157c30 0x7fde5c0cd600 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009870 tx=0x7fde4c02ba20 comp rx=0 tx=0).stop 2024-01-15T18:12:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.086+0000 7fde64600640 1 -- 172.21.15.26:0/1172361857 shutdown_connections 2024-01-15T18:12:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.086+0000 7fde64600640 1 --2- 172.21.15.26:0/1172361857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c157c30 0x7fde5c0cd600 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:17.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.086+0000 7fde64600640 1 -- 172.21.15.26:0/1172361857 >> 172.21.15.26:0/1172361857 conn(0x7fde5c0cf700 msgr2=0x7fde5c0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.086+0000 7fde64600640 1 -- 172.21.15.26:0/1172361857 shutdown_connections 2024-01-15T18:12:17.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.086+0000 7fde64600640 1 -- 172.21.15.26:0/1172361857 wait complete. 2024-01-15T18:12:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.088+0000 7fde64600640 1 Processor -- start 2024-01-15T18:12:17.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.088+0000 7fde64600640 1 -- start start 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.089+0000 7fde64600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde5c071200 0x7fde5c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.089+0000 7fde64600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde5c157c30 0x7fde5c149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.089+0000 7fde64600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c1618f0 0x7fde5c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.089+0000 7fde64600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c15b110 con 0x7fde5c1618f0 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.089+0000 7fde64600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c15b3f0 con 0x7fde5c071200 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.089+0000 7fde64600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c15b6d0 con 0x7fde5c157c30 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.089+0000 7fde62600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c1618f0 0x7fde5c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.089+0000 7fde62600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c1618f0 0x7fde5c158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34772/0 (socket says 172.21.15.26:34772) 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.089+0000 7fde62600640 1 -- 172.21.15.26:0/378905941 learned_addr learned my addr 172.21.15.26:0/378905941 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:17.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde62600640 1 -- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde5c157c30 msgr2=0x7fde5c149bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde62600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde5c157c30 0x7fde5c149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde61c00640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde5c071200 0x7fde5c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde62600640 1 -- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde5c071200 msgr2=0x7fde5c15aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde62600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde5c071200 0x7fde5c15aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde62600640 1 -- 172.21.15.26:0/378905941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0094d0 con 0x7fde5c1618f0 2024-01-15T18:12:17.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde62600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c1618f0 0x7fde5c158a30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fde5400c390 tx=0x7fde5400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:17.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde59600640 1 -- 172.21.15.26:0/378905941 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540052d0 con 0x7fde5c1618f0 2024-01-15T18:12:17.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde59600640 1 -- 172.21.15.26:0/378905941 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde54005430 con 0x7fde5c1618f0 2024-01-15T18:12:17.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde59600640 1 -- 172.21.15.26:0/378905941 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54003810 con 0x7fde5c1618f0 2024-01-15T18:12:17.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c15b9b0 con 0x7fde5c1618f0 2024-01-15T18:12:17.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.090+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c157740 con 0x7fde5c1618f0 2024-01-15T18:12:17.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.092+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5c15bed0 con 0x7fde5c1618f0 2024-01-15T18:12:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.092+0000 7fde59600640 1 -- 172.21.15.26:0/378905941 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fde54003970 con 0x7fde5c1618f0 2024-01-15T18:12:17.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.093+0000 7fde59600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde3406c430 0x7fde3406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:17.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.093+0000 7fde59600640 1 -- 172.21.15.26:0/378905941 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fde54088000 con 0x7fde5c1618f0 2024-01-15T18:12:17.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.094+0000 7fde61c00640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde3406c430 0x7fde3406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:17.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.095+0000 7fde61c00640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde3406c430 0x7fde3406e8f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fde4c036000 tx=0x7fde4c008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:17.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.098+0000 7fde59600640 1 -- 172.21.15.26:0/378905941 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde5c15bed0 con 0x7fde5c1618f0 2024-01-15T18:12:17.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.333+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde5c15bed0 con 0x7fde5c1618f0 2024-01-15T18:12:17.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.334+0000 7fde59600640 1 -- 172.21.15.26:0/378905941 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fde5c15bed0 con 0x7fde5c1618f0 2024-01-15T18:12:17.337 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:12:17.338 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":72,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/3480872940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3480872940},{"type":"v1","addr":"172.21.15.124:6835","nonce":3480872940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/662436785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":662436785},{"type":"v1","addr":"172.21.15.124:6837","nonce":662436785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":9222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2088544410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2088544410},{"type":"v1","addr":"172.21.15.26:6833","nonce":2088544410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:12:10.468680+0000","modified":"2024-01-15T18:12:13.801724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/483783473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":483783473},{"type":"v1","addr":"172.21.15.26:6835","nonce":483783473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-01-15T18:12:17.338 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 72 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde3406c430 msgr2=0x7fde3406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde3406c430 0x7fde3406e8f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fde4c036000 tx=0x7fde4c008030 comp rx=0 tx=0).stop 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c1618f0 msgr2=0x7fde5c158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c1618f0 0x7fde5c158a30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fde5400c390 tx=0x7fde5400c850 comp rx=0 tx=0).stop 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 shutdown_connections 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fde5c157c30 0x7fde5c149bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fde3406c430 0x7fde3406e8f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fde5c1618f0 0x7fde5c158a30 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 --2- 172.21.15.26:0/378905941 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fde5c071200 0x7fde5c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 >> 172.21.15.26:0/378905941 conn(0x7fde5c0cf700 msgr2=0x7fde5c0b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:17.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 shutdown_connections 2024-01-15T18:12:17.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.337+0000 7fde64600640 1 -- 172.21.15.26:0/378905941 wait complete. 2024-01-15T18:12:17.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-15T18:12:10.468680+0000', 'modified': '2024-01-15T18:12:13.801724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9225}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9225': {'gid': 9225, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.26:6835/483783473', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.26:6834', 'nonce': 483783473}, {'type': 'v1', 'addr': '172.21.15.26:6835', 'nonce': 483783473}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-15T18:12:17.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-15T18:12:17.764 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-01-15T18:12:17.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.995+0000 7f8601400640 1 Processor -- start 2024-01-15T18:12:17.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.995+0000 7f8601400640 1 -- start start 2024-01-15T18:12:18.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.997+0000 7f8601400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc15afb0 0x7f85fc13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.997+0000 7f8601400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f85fc13fae0 0x7f85fc0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.997+0000 7f8601400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc13ff00 con 0x7f85fc13cfa0 2024-01-15T18:12:18.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.997+0000 7f8601400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc0cdc40 con 0x7f85fc13fae0 2024-01-15T18:12:18.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.997+0000 7f8601400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0cdec0 con 0x7f85fc15afb0 2024-01-15T18:12:18.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.998+0000 7f85fb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc15afb0 0x7f85fc13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.998+0000 7f8600a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f85fc13fae0 0x7f85fc0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.998+0000 7f85fb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc15afb0 0x7f85fc13f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:49612/0 (socket says 172.21.15.26:49612) 2024-01-15T18:12:18.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.998+0000 7f85fbe00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f85fc13cfa0 0x7f85fc0cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:58238/0 (socket says 172.21.15.26:58238) 2024-01-15T18:12:18.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.998+0000 7f85fb400640 1 -- 172.21.15.26:0/1983680525 learned_addr learned my addr 172.21.15.26:0/1983680525 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:18.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.999+0000 7f8600a00640 1 -- 172.21.15.26:0/1983680525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc15afb0 msgr2=0x7f85fc13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.999+0000 7f8600a00640 1 --2- 172.21.15.26:0/1983680525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc15afb0 0x7f85fc13f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.999+0000 7f8600a00640 1 -- 172.21.15.26:0/1983680525 >> v1:172.21.15.124:6789/0 conn(0x7f85fc13cfa0 legacy=0x7f85fc0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.999+0000 7f8600a00640 1 -- 172.21.15.26:0/1983680525 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0ce1a0 con 0x7f85fc13fae0 2024-01-15T18:12:18.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:17.999+0000 7f85fb400640 1 --2- 172.21.15.26:0/1983680525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc15afb0 0x7f85fc13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:12:18.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.000+0000 7f8600a00640 1 --2- 172.21.15.26:0/1983680525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85fc13fae0 0x7f85fc0b2040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d90 tx=0x7f85f402e030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd4916011d82ea9 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.001+0000 7f85faa00640 1 -- 172.21.15.26:0/1983680525 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f402ede0 con 0x7f85fc13fae0 2024-01-15T18:12:18.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.001+0000 7f85faa00640 1 -- 172.21.15.26:0/1983680525 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f4036440 con 0x7f85fc13fae0 2024-01-15T18:12:18.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.001+0000 7f85faa00640 1 -- 172.21.15.26:0/1983680525 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40367d0 con 0x7f85fc13fae0 2024-01-15T18:12:18.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.001+0000 7f8601400640 1 -- 172.21.15.26:0/1983680525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85fc13fae0 msgr2=0x7f85fc0b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.001+0000 7f8601400640 1 --2- 172.21.15.26:0/1983680525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85fc13fae0 0x7f85fc0b2040 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009d90 tx=0x7f85f402e030 comp rx=0 tx=0).stop 2024-01-15T18:12:18.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.002+0000 7f8601400640 1 -- 172.21.15.26:0/1983680525 shutdown_connections 2024-01-15T18:12:18.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.002+0000 7f8601400640 1 --2- 172.21.15.26:0/1983680525 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc15afb0 0x7f85fc13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.002+0000 7f8601400640 1 --2- 172.21.15.26:0/1983680525 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85fc13fae0 0x7f85fc0b2040 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.002+0000 7f8601400640 1 -- 172.21.15.26:0/1983680525 >> 172.21.15.26:0/1983680525 conn(0x7f85fc06a800 msgr2=0x7f85fc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:18.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.002+0000 7f8601400640 1 -- 172.21.15.26:0/1983680525 shutdown_connections 2024-01-15T18:12:18.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.002+0000 7f8601400640 1 -- 172.21.15.26:0/1983680525 wait complete. 2024-01-15T18:12:18.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.004+0000 7f8601400640 1 Processor -- start 2024-01-15T18:12:18.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.004+0000 7f8601400640 1 -- start start 2024-01-15T18:12:18.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.005+0000 7f8601400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc13cfa0 0x7f85fc1571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.005+0000 7f8601400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85fc13fae0 0x7f85fc15b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.005+0000 7f8601400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85fc15afb0 0x7f85fc15bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.005+0000 7f8601400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc13a130 con 0x7f85fc13fae0 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.005+0000 7f8601400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc139240 con 0x7f85fc15afb0 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.005+0000 7f8601400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc139520 con 0x7f85fc13cfa0 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85fc13fae0 0x7f85fc15b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85fc13fae0 0x7f85fc15b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34778/0 (socket says 172.21.15.26:34778) 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fb400640 1 -- 172.21.15.26:0/1435774159 learned_addr learned my addr 172.21.15.26:0/1435774159 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fbe00640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc13cfa0 0x7f85fc1571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f8600a00640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85fc15afb0 0x7f85fc15bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fb400640 1 -- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc13cfa0 msgr2=0x7f85fc1571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fb400640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc13cfa0 0x7f85fc1571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fb400640 1 -- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85fc15afb0 msgr2=0x7f85fc15bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fb400640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85fc15afb0 0x7f85fc15bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fb400640 1 -- 172.21.15.26:0/1435774159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e40094d0 con 0x7f85fc13fae0 2024-01-15T18:12:18.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.006+0000 7f85fb400640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85fc13fae0 0x7f85fc15b5a0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f85e400e9b0 tx=0x7f85e400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.007+0000 7f85f3e00640 1 -- 172.21.15.26:0/1435774159 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400c9a0 con 0x7f85fc13fae0 2024-01-15T18:12:18.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.007+0000 7f85f3e00640 1 -- 172.21.15.26:0/1435774159 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e400cb00 con 0x7f85fc13fae0 2024-01-15T18:12:18.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.007+0000 7f85f3e00640 1 -- 172.21.15.26:0/1435774159 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400ce70 con 0x7f85fc13fae0 2024-01-15T18:12:18.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.007+0000 7f8601400640 1 -- 172.21.15.26:0/1435774159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f4009a70 con 0x7f85fc13fae0 2024-01-15T18:12:18.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.007+0000 7f8601400640 1 -- 172.21.15.26:0/1435774159 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc139ae0 con 0x7f85fc13fae0 2024-01-15T18:12:18.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.010+0000 7f85f3e00640 1 -- 172.21.15.26:0/1435774159 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f85e4013470 con 0x7f85fc13fae0 2024-01-15T18:12:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.011+0000 7f85f3e00640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f85d406c430 0x7f85d406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.011+0000 7f85f3e00640 1 -- 172.21.15.26:0/1435774159 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f85e40890c0 con 0x7f85fc13fae0 2024-01-15T18:12:18.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.012+0000 7f86055e3740 1 Processor -- start 2024-01-15T18:12:18.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.012+0000 7f86055e3740 1 -- start start 2024-01-15T18:12:18.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.012+0000 7f85fbe00640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f85d406c430 0x7f85d406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f86055e3740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 0x561fb1c57450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f86055e3740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c56270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f86055e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f86055e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1bb5880 con 0x561fb1c567b0 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f86055e3740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1bc4ae0 con 0x561fb1c57070 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f86055e3740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fb1bc21d0 con 0x561fb1c51d70 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f8600a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f8600a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c9a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34790/0 (socket says 172.21.15.26:34790) 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f8600a00640 1 -- 172.21.15.26:0/1779734111 learned_addr learned my addr 172.21.15.26:0/1779734111 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f85fbe00640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f85d406c430 0x7f85d406e8f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f85ec000f60 tx=0x7f85ec007cc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f8600a00640 1 -- 172.21.15.26:0/1779734111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 msgr2=0x561fb1c56270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f8600a00640 1 --2- 172.21.15.26:0/1779734111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c56270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f85fbe00640 1 --2- 172.21.15.26:0/1779734111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 0x561fb1c57450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f8600a00640 1 -- 172.21.15.26:0/1779734111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 msgr2=0x561fb1c57450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f8600a00640 1 --2- 172.21.15.26:0/1779734111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 0x561fb1c57450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.013+0000 7f8600a00640 1 -- 172.21.15.26:0/1779734111 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fb1b85860 con 0x561fb1c567b0 2024-01-15T18:12:18.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.014+0000 7f8600a00640 1 --2- 172.21.15.26:0/1779734111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c9a730 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009e80 tx=0x7f85f4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a420dc26441a42 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.014+0000 7f85f0c00640 1 -- 172.21.15.26:0/1779734111 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40365b0 con 0x561fb1c567b0 2024-01-15T18:12:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.014+0000 7f85f0c00640 1 -- 172.21.15.26:0/1779734111 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f4036730 con 0x561fb1c567b0 2024-01-15T18:12:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.014+0000 7f85f0c00640 1 -- 172.21.15.26:0/1779734111 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4036ae0 con 0x561fb1c567b0 2024-01-15T18:12:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.015+0000 7f86055e3740 1 -- 172.21.15.26:0/1779734111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 msgr2=0x561fb1c9a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.015+0000 7f86055e3740 1 --2- 172.21.15.26:0/1779734111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c9a730 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009e80 tx=0x7f85f4030000 comp rx=0 tx=0).stop 2024-01-15T18:12:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.015+0000 7f86055e3740 1 -- 172.21.15.26:0/1779734111 shutdown_connections 2024-01-15T18:12:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.015+0000 7f86055e3740 1 --2- 172.21.15.26:0/1779734111 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c56270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.015+0000 7f86055e3740 1 --2- 172.21.15.26:0/1779734111 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c9a730 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.015+0000 7f86055e3740 1 --2- 172.21.15.26:0/1779734111 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 0x561fb1c57450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.015+0000 7f86055e3740 1 -- 172.21.15.26:0/1779734111 >> 172.21.15.26:0/1779734111 conn(0x561fb1c18c70 msgr2=0x561fb1c9cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.015+0000 7f86055e3740 1 -- 172.21.15.26:0/1779734111 shutdown_connections 2024-01-15T18:12:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.015+0000 7f86055e3740 1 -- 172.21.15.26:0/1779734111 wait complete. 2024-01-15T18:12:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.016+0000 7f86055e3740 1 -- start start 2024-01-15T18:12:18.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.016+0000 7f86055e3740 1 Processor -- start 2024-01-15T18:12:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.017+0000 7f86055e3740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c6e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.017+0000 7f86055e3740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 0x561fb1c7f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.017+0000 7f86055e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c57070 0x561fb1c802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.017+0000 7f86055e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1bbdac0 con 0x561fb1c57070 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.017+0000 7f86055e3740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1b5a2f0 con 0x561fb1c567b0 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.017+0000 7f86055e3740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fb1b77c70 con 0x561fb1c51d70 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.017+0000 7f86055e3740 10 client.0 _ll_register_callbacks cb 0x561fb1b7c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f8600a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c57070 0x561fb1c802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f8600a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c57070 0x561fb1c802d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34800/0 (socket says 172.21.15.26:34800) 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f8600a00640 1 -- 172.21.15.26:0/2004798297 learned_addr learned my addr 172.21.15.26:0/2004798297 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f85fb400640 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 0x561fb1c7f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f85fbe00640 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c6e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f8600a00640 1 -- 172.21.15.26:0/2004798297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 msgr2=0x561fb1c6e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f8600a00640 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c6e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f8600a00640 1 -- 172.21.15.26:0/2004798297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 msgr2=0x561fb1c7f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f8600a00640 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 0x561fb1c7f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f8600a00640 1 -- 172.21.15.26:0/2004798297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc0ce1a0 con 0x561fb1c57070 2024-01-15T18:12:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.018+0000 7f85fbe00640 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c6e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.019+0000 7f8600a00640 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c57070 0x561fb1c802d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f85f402e070 tx=0x7f85f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9013feff6146d3b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.019+0000 7f85e9600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:12:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.019+0000 7f85e9600640 1 -- 172.21.15.26:0/2004798297 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4036cb0 con 0x561fb1c57070 2024-01-15T18:12:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.019+0000 7f85e9600640 1 -- 172.21.15.26:0/2004798297 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f85f4036e10 con 0x561fb1c57070 2024-01-15T18:12:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.019+0000 7f86055e3740 1 -- 172.21.15.26:0/2004798297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fb1bb6aa0 con 0x561fb1c57070 2024-01-15T18:12:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.019+0000 7f85e9600640 1 -- 172.21.15.26:0/2004798297 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f4004040 con 0x561fb1c57070 2024-01-15T18:12:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.019+0000 7f86055e3740 10 client.9348 fetch_fsmap learned FSMap version 72 2024-01-15T18:12:18.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.019+0000 7f86055e3740 1 -- 172.21.15.26:0/2004798297 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x561fb1bbc140 con 0x561fb1c57070 2024-01-15T18:12:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.020+0000 7f85e9600640 1 -- 172.21.15.26:0/2004798297 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561fb1bb6aa0 con 0x561fb1c57070 2024-01-15T18:12:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.020+0000 7f85e9600640 1 -- 172.21.15.26:0/2004798297 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85f4047070 con 0x561fb1c57070 2024-01-15T18:12:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.020+0000 7f86055e3740 10 client.9348 fetch_fsmap finished waiting for FSMap version 72 2024-01-15T18:12:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.020+0000 7f86055e3740 10 client.9348 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-15T18:12:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.020+0000 7f86055e3740 20 client.9348 populate_metadata read hostname 'smithi026' 2024-01-15T18:12:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.020+0000 7f86055e3740 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] conn(0x561fb1c7caa0 0x561fb1cb9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.020+0000 7f86055e3740 4 client.9348 mds_command: new command op to 9225 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.020+0000 7f86055e3740 1 -- 172.21.15.26:0/2004798297 --> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fb1bb67c0 con 0x561fb1c7caa0 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.021+0000 7f85fbe00640 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] conn(0x561fb1c7caa0 0x561fb1cb9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.021+0000 7f85fbe00640 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] conn(0x561fb1c7caa0 0x561fb1cb9ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c3f29dfdcd4b4c server_cookie=4d9ae27bb0ac0066 in_seq=0 out_seq=0 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.021+0000 7f85e9600640 10 client.9348 ms_handle_connect on v2:172.21.15.26:6834/483783473 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.022+0000 7f85e9600640 1 -- 172.21.15.26:0/2004798297 <== mds.0 v2:172.21.15.26:6834/483783473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561fb1bb67c0 con 0x561fb1c7caa0 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.022+0000 7f85e9600640 10 client.9348 handle_command_reply: tid=0 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.022+0000 7f86055e3740 1 client.9348 shutdown 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.022+0000 7f86055e3740 1 -- 172.21.15.26:0/2004798297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c57070 msgr2=0x561fb1c802d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.022+0000 7f86055e3740 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c57070 0x561fb1c802d0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f85f402e070 tx=0x7f85f4004ab0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f86055e3740 1 -- 172.21.15.26:0/2004798297 shutdown_connections 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f86055e3740 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] conn(0x561fb1c7caa0 0x561fb1cb9ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f86055e3740 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c57070 0x561fb1c802d0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f86055e3740 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 0x561fb1c7f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.027 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f86055e3740 1 --2- 172.21.15.26:0/2004798297 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c6e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f85e9600640 0 client.9348 ms_handle_reset on v2:172.21.15.26:6834/483783473 2024-01-15T18:12:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f86055e3740 1 -- 172.21.15.26:0/2004798297 >> 172.21.15.26:0/2004798297 conn(0x561fb1c18c70 msgr2=0x561fb1c99d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f86055e3740 1 -- 172.21.15.26:0/2004798297 shutdown_connections 2024-01-15T18:12:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f86055e3740 1 -- 172.21.15.26:0/2004798297 wait complete. 2024-01-15T18:12:18.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.023+0000 7f86055e3740 20 client.9348 trim_cache size 0 max 16384 2024-01-15T18:12:18.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.041+0000 7f86055e3740 1 Processor -- start 2024-01-15T18:12:18.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.041+0000 7f86055e3740 1 -- start start 2024-01-15T18:12:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.042+0000 7f86055e3740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c50f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.042+0000 7f86055e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c6d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.042+0000 7f86055e3740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 0x561fb1c6d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.042+0000 7f86055e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1920f20 con 0x561fb1c567b0 2024-01-15T18:12:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.042+0000 7f86055e3740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1c854a0 con 0x561fb1c57070 2024-01-15T18:12:18.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.042+0000 7f86055e3740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fb1bbfa40 con 0x561fb1c51d70 2024-01-15T18:12:18.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.042+0000 7f85fb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c6d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.042+0000 7f85fb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c6d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34810/0 (socket says 172.21.15.26:34810) 2024-01-15T18:12:18.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.042+0000 7f85fb400640 1 -- 172.21.15.26:0/3451235979 learned_addr learned my addr 172.21.15.26:0/3451235979 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85fbe00640 1 --2- 172.21.15.26:0/3451235979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c50f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f8600a00640 1 --2- 172.21.15.26:0/3451235979 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 0x561fb1c6d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85fb400640 1 -- 172.21.15.26:0/3451235979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 msgr2=0x561fb1c50f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85fb400640 1 --2- 172.21.15.26:0/3451235979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c50f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85fb400640 1 -- 172.21.15.26:0/3451235979 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 msgr2=0x561fb1c6d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85fb400640 1 --2- 172.21.15.26:0/3451235979 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 0x561fb1c6d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85fb400640 1 -- 172.21.15.26:0/3451235979 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fb1b59b80 con 0x561fb1c567b0 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85fb400640 1 --2- 172.21.15.26:0/3451235979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c6d170 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c520 tx=0x7f85e4058d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3166ac25d1c66e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85f1600640 1 -- 172.21.15.26:0/3451235979 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e405d440 con 0x561fb1c567b0 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85f1600640 1 -- 172.21.15.26:0/3451235979 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e4070070 con 0x561fb1c567b0 2024-01-15T18:12:18.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.043+0000 7f85f1600640 1 -- 172.21.15.26:0/3451235979 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e40615f0 con 0x561fb1c567b0 2024-01-15T18:12:18.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.044+0000 7f86055e3740 1 -- 172.21.15.26:0/3451235979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 msgr2=0x561fb1c6d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.044+0000 7f86055e3740 1 --2- 172.21.15.26:0/3451235979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c6d170 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f85e400c520 tx=0x7f85e4058d10 comp rx=0 tx=0).stop 2024-01-15T18:12:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.044+0000 7f86055e3740 1 -- 172.21.15.26:0/3451235979 shutdown_connections 2024-01-15T18:12:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.044+0000 7f86055e3740 1 --2- 172.21.15.26:0/3451235979 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c57070 0x561fb1c6d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.044+0000 7f86055e3740 1 --2- 172.21.15.26:0/3451235979 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c567b0 0x561fb1c6d170 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.044+0000 7f86055e3740 1 --2- 172.21.15.26:0/3451235979 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c51d70 0x561fb1c50f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.044+0000 7f86055e3740 1 -- 172.21.15.26:0/3451235979 >> 172.21.15.26:0/3451235979 conn(0x561fb1c18c70 msgr2=0x561fb1c80ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.045+0000 7f86055e3740 1 -- 172.21.15.26:0/3451235979 shutdown_connections 2024-01-15T18:12:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.045+0000 7f86055e3740 1 -- 172.21.15.26:0/3451235979 wait complete. 2024-01-15T18:12:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.045+0000 7f86055e3740 1 -- start start 2024-01-15T18:12:18.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.046+0000 7f86055e3740 1 Processor -- start 2024-01-15T18:12:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.046+0000 7f86055e3740 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c51d70 0x561fb1c80ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.046+0000 7f86055e3740 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 0x561fb1c51190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f86055e3740 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c57070 0x561fb1c516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f86055e3740 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1c9fd10 con 0x561fb1c51d70 2024-01-15T18:12:18.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f86055e3740 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fb1ca38d0 con 0x561fb1c567b0 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f86055e3740 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fb1c84f00 con 0x561fb1c57070 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f86055e3740 10 client.0 _ll_register_callbacks cb 0x7f85f4043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c51d70 0x561fb1c80ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fbe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c51d70 0x561fb1c80ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34824/0 (socket says 172.21.15.26:34824) 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fbe00640 1 -- 172.21.15.26:0/1179625872 learned_addr learned my addr 172.21.15.26:0/1179625872 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f8600a00640 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c57070 0x561fb1c516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fb400640 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 0x561fb1c51190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fbe00640 1 -- 172.21.15.26:0/1179625872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c57070 msgr2=0x561fb1c516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fbe00640 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c57070 0x561fb1c516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fbe00640 1 -- 172.21.15.26:0/1179625872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 msgr2=0x561fb1c51190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fbe00640 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 0x561fb1c51190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fbe00640 1 -- 172.21.15.26:0/1179625872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc139ae0 con 0x561fb1c51d70 2024-01-15T18:12:18.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.047+0000 7f85fb400640 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 0x561fb1c51190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.048+0000 7f85fbe00640 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c51d70 0x561fb1c80ed0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x561fb1c7ca20 tx=0x7f85ec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6badd7829589c9eb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.048+0000 7f85e9600640 10 client.0 ms_handle_connect on v2:172.21.15.26:3300/0 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.048+0000 7f85e9600640 1 -- 172.21.15.26:0/1179625872 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec01a040 con 0x561fb1c51d70 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.048+0000 7f85e9600640 1 -- 172.21.15.26:0/1179625872 <== mon.0 v2:172.21.15.26:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f85ec005050 con 0x561fb1c51d70 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.048+0000 7f86055e3740 1 -- 172.21.15.26:0/1179625872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85f4036730 con 0x561fb1c51d70 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.048+0000 7f85e9600640 1 -- 172.21.15.26:0/1179625872 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85ec005210 con 0x561fb1c51d70 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.048+0000 7f86055e3740 10 client.9354 fetch_fsmap learned FSMap version 72 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.048+0000 7f86055e3740 1 -- 172.21.15.26:0/1179625872 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) v3 -- 0x561fb1b7f170 con 0x561fb1c51d70 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.048+0000 7f85e9600640 1 -- 172.21.15.26:0/1179625872 <== mon.0 v2:172.21.15.26:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f4036730 con 0x561fb1c51d70 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f85e9600640 1 -- 172.21.15.26:0/1179625872 <== mon.0 v2:172.21.15.26:3300/0 5 ==== fsmap(e 72) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f85ec005370 con 0x561fb1c51d70 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f86055e3740 10 client.9354 fetch_fsmap finished waiting for FSMap version 72 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f86055e3740 10 client.9354 resolve_mds: resolved name 'a' to daemon mds.a 2024-01-15T18:12:18.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f86055e3740 20 client.9354 populate_metadata read hostname 'smithi026' 2024-01-15T18:12:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f86055e3740 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] conn(0x561fb1ca0330 0x561fb1ca0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f86055e3740 4 client.9354 mds_command: new command op to 9225 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-15T18:12:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f86055e3740 1 -- 172.21.15.26:0/1179625872 --> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561fb18be6c0 con 0x561fb1ca0330 2024-01-15T18:12:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f85fb400640 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] conn(0x561fb1ca0330 0x561fb1ca0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f85fb400640 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] conn(0x561fb1ca0330 0x561fb1ca0710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b119eb22f4d3a3f8 server_cookie=8ad054773b73a954 in_seq=0 out_seq=0 2024-01-15T18:12:18.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.049+0000 7f85e9600640 10 client.9354 ms_handle_connect on v2:172.21.15.26:6834/483783473 2024-01-15T18:12:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f85e9600640 1 -- 172.21.15.26:0/1179625872 <== mds.0 v2:172.21.15.26:6834/483783473 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561fb18be6c0 con 0x561fb1ca0330 2024-01-15T18:12:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f85e9600640 10 client.9354 handle_command_reply: tid=0 2024-01-15T18:12:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 client.9354 shutdown 2024-01-15T18:12:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 -- 172.21.15.26:0/1179625872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c51d70 msgr2=0x561fb1c80ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c51d70 0x561fb1c80ed0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x561fb1c7ca20 tx=0x7f85ec000bc0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 -- 172.21.15.26:0/1179625872 shutdown_connections 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.26:6834/483783473,v1:172.21.15.26:6835/483783473] conn(0x561fb1ca0330 0x561fb1ca0710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x561fb1c57070 0x561fb1c516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x561fb1c567b0 0x561fb1c51190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 --2- 172.21.15.26:0/1179625872 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x561fb1c51d70 0x561fb1c80ed0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f85e9600640 0 client.9354 ms_handle_reset on v2:172.21.15.26:6834/483783473 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 -- 172.21.15.26:0/1179625872 >> 172.21.15.26:0/1179625872 conn(0x561fb1c18c70 msgr2=0x561fb1c4d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 -- 172.21.15.26:0/1179625872 shutdown_connections 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 1 -- 172.21.15.26:0/1179625872 wait complete. 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.050+0000 7f86055e3740 20 client.9354 trim_cache size 0 max 16384 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:12:18.057 INFO:teuthology.orchestra.run.smithi026.stdout: "cluster_fsid": "64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1", 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "whoami": 0, 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9225, 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "want_state": "up:active", 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "up:active", 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs", 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "rank_uptime": 6.7743343449999998, 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "mdsmap_epoch": 72, 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch": 115, 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "osdmap_epoch_barrier": 115, 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout: "uptime": 12.857435417 2024-01-15T18:12:18.058 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:12:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 -- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f85d406c430 msgr2=0x7f85d406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f85d406c430 0x7f85d406e8f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f85ec000f60 tx=0x7f85ec007cc0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 -- 172.21.15.26:0/1435774159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85fc13fae0 msgr2=0x7f85fc15b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85fc13fae0 0x7f85fc15b5a0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f85e400e9b0 tx=0x7f85e400ee70 comp rx=0 tx=0).stop 2024-01-15T18:12:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 -- 172.21.15.26:0/1435774159 shutdown_connections 2024-01-15T18:12:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f85d406c430 0x7f85d406e8f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f85fc15afb0 0x7f85fc15bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f85fc13fae0 0x7f85fc15b5a0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 --2- 172.21.15.26:0/1435774159 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f85fc13cfa0 0x7f85fc1571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 -- 172.21.15.26:0/1435774159 >> 172.21.15.26:0/1435774159 conn(0x7f85fc06a800 msgr2=0x7f85fc0b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 -- 172.21.15.26:0/1435774159 shutdown_connections 2024-01-15T18:12:18.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.051+0000 7f8601400640 1 -- 172.21.15.26:0/1435774159 wait complete. 2024-01-15T18:12:18.071 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:12:18.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.275+0000 7f4ff9200640 1 Processor -- start 2024-01-15T18:12:18.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.275+0000 7f4ff9200640 1 -- start start 2024-01-15T18:12:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.277+0000 7f4ff9200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4ff40d1900 0x7f4ff406be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.277+0000 7f4ff9200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff40cb950 0x7f4ff40cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.277+0000 7f4ff9200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff40d1d10 con 0x7f4ff415d440 2024-01-15T18:12:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.277+0000 7f4ff9200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff40cfaf0 con 0x7f4ff40d1900 2024-01-15T18:12:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.277+0000 7f4ff9200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40cfd00 con 0x7f4ff40cb950 2024-01-15T18:12:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.277+0000 7f4ff3400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4ff415d440 0x7f4ff406b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36504/0 (socket says 172.21.15.26:36504) 2024-01-15T18:12:18.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.277+0000 7f4ff3400640 1 -- 172.21.15.26:0/560979473 learned_addr learned my addr 172.21.15.26:0/560979473 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:18.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.277+0000 7f4ff3e00640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff40cb950 0x7f4ff40cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2a00640 1 --2- 172.21.15.26:0/560979473 >> v2:172.21.15.124:3300/0 conn(0x7f4ff40d1900 0x7f4ff406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073034698 0 0) 0x7f4ff40d1d10 con 0x7f4ff415d440 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fd4003520 con 0x7f4ff415d440 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464016685 0 0) 0x7f4fd4003520 con 0x7f4ff415d440 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff40cb950 msgr2=0x7f4ff40cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff40cb950 0x7f4ff40cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 >> v2:172.21.15.124:3300/0 conn(0x7f4ff40d1900 msgr2=0x7f4ff406be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 --2- 172.21.15.26:0/560979473 >> v2:172.21.15.124:3300/0 conn(0x7f4ff40d1900 0x7f4ff406be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40cffe0 con 0x7f4ff415d440 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f4fe4001670 con 0x7f4ff415d440 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 >> v1:172.21.15.26:6789/0 conn(0x7f4ff415d440 legacy=0x7f4ff406b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4fd4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff40cb950 0x7f4fd40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fd4004d00 0x7f4fd4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40cfaf0 con 0x7f4fd4003970 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff40cfd00 con 0x7f4fd4004d00 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd400b980 con 0x7f4ff40cb950 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.278+0000 7f4ff3400640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4fd4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.279+0000 7f4ff3e00640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fd4004d00 0x7f4fd4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.279+0000 7f4ff2a00640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff40cb950 0x7f4fd40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.279+0000 7f4ff3400640 1 -- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff40cb950 msgr2=0x7f4fd40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.279+0000 7f4ff3400640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff40cb950 0x7f4fd40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.279+0000 7f4ff3400640 1 -- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fd4004d00 msgr2=0x7f4fd4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.279+0000 7f4ff3400640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fd4004d00 0x7f4fd4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.279+0000 7f4ff3400640 1 -- 172.21.15.26:0/560979473 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4003520 con 0x7f4fd4003970 2024-01-15T18:12:18.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.279+0000 7f4ff2a00640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff40cb950 0x7f4fd40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:18.285 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.279+0000 7f4ff3e00640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4fd4004d00 0x7f4fd4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.280+0000 7f4ff3400640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4fd4003d90 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c240 tx=0x7f4fd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae122d3e9e6acf4f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.280+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe402cb00 con 0x7f4fd4003970 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.281+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe402cc60 con 0x7f4fd4003970 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.281+0000 7f4ff2000640 1 -- 172.21.15.26:0/560979473 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4005af0 con 0x7f4fd4003970 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.281+0000 7f4ff9200640 1 -- 172.21.15.26:0/560979473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 msgr2=0x7f4fd4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.281+0000 7f4ff9200640 1 --2- 172.21.15.26:0/560979473 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4fd4003d90 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c240 tx=0x7f4fd40057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.281+0000 7f4ff3e00640 1 -- 172.21.15.26:0/560979473 reap_dead start 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.281+0000 7f4ff9200640 1 -- 172.21.15.26:0/560979473 shutdown_connections 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.281+0000 7f4ff9200640 1 -- 172.21.15.26:0/560979473 >> 172.21.15.26:0/560979473 conn(0x7f4ff40d0de0 msgr2=0x7f4ff40d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.281+0000 7f4ff9200640 1 -- 172.21.15.26:0/560979473 shutdown_connections 2024-01-15T18:12:18.286 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.281+0000 7f4ff9200640 1 -- 172.21.15.26:0/560979473 wait complete. 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.283+0000 7f4ff9200640 1 Processor -- start 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.283+0000 7f4ff9200640 1 -- start start 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.284+0000 7f4ff9200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4ff409d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.284+0000 7f4ff9200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff415b740 0x7f4ff409bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.284+0000 7f4ff9200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff415eaf0 0x7f4ff415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.284+0000 7f4ff9200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff409c480 con 0x7f4fd4003970 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.284+0000 7f4ff9200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff415f4e0 con 0x7f4ff415b740 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.284+0000 7f4ff9200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff415bbc0 con 0x7f4ff415eaf0 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4ff409d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4ff409d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34838/0 (socket says 172.21.15.26:34838) 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3400640 1 -- 172.21.15.26:0/1327350535 learned_addr learned my addr 172.21.15.26:0/1327350535 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3e00640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff415eaf0 0x7f4ff415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.289 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff2a00640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff415b740 0x7f4ff409bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3400640 1 -- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff415eaf0 msgr2=0x7f4ff415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3400640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff415eaf0 0x7f4ff415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3400640 1 -- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff415b740 msgr2=0x7f4ff409bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3400640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff415b740 0x7f4ff409bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3400640 1 -- 172.21.15.26:0/1327350535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff415c630 con 0x7f4fd4003970 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff2a00640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff415b740 0x7f4ff409bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.285+0000 7f4ff3400640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4ff409d870 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f4fe402c4a0 tx=0x7f4fe4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.286+0000 7f4feb400640 1 -- 172.21.15.26:0/1327350535 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4039030 con 0x7f4fd4003970 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.286+0000 7f4feb400640 1 -- 172.21.15.26:0/1327350535 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe4004bf0 con 0x7f4fd4003970 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.286+0000 7f4feb400640 1 -- 172.21.15.26:0/1327350535 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe403c040 con 0x7f4fd4003970 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.286+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff409df50 con 0x7f4fd4003970 2024-01-15T18:12:18.290 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.286+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff409e510 con 0x7f4fd4003970 2024-01-15T18:12:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.287+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff4067450 con 0x7f4fd4003970 2024-01-15T18:12:18.291 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.288+0000 7f4feb400640 1 -- 172.21.15.26:0/1327350535 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f4fe4032070 con 0x7f4fd4003970 2024-01-15T18:12:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.289+0000 7f4feb400640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fcc06c430 0x7f4fcc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.289+0000 7f4feb400640 1 -- 172.21.15.26:0/1327350535 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f4fe40a9aa0 con 0x7f4fd4003970 2024-01-15T18:12:18.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.289+0000 7f4ff2a00640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fcc06c430 0x7f4fcc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:18.293 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.291+0000 7f4ff2a00640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fcc06c430 0x7f4fcc06e8f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4ff415e4a0 tx=0x7f4fdc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:18.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.295+0000 7f4feb400640 1 -- 172.21.15.26:0/1327350535 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4fe4076360 con 0x7f4fd4003970 2024-01-15T18:12:18.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.528+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4ff40bb380 con 0x7f4fd4003970 2024-01-15T18:12:18.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.529+0000 7f4feb400640 1 -- 172.21.15.26:0/1327350535 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4fe4001c50 con 0x7f4fd4003970 2024-01-15T18:12:18.532 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:12:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.531+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fcc06c430 msgr2=0x7f4fcc06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.531+0000 7f4ff9200640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fcc06c430 0x7f4fcc06e8f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4ff415e4a0 tx=0x7f4fdc008040 comp rx=0 tx=0).stop 2024-01-15T18:12:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.531+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 msgr2=0x7f4ff409d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.531+0000 7f4ff9200640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4ff409d870 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f4fe402c4a0 tx=0x7f4fe4001a00 comp rx=0 tx=0).stop 2024-01-15T18:12:18.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.532+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 shutdown_connections 2024-01-15T18:12:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.532+0000 7f4ff9200640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4ff415eaf0 0x7f4ff415efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.532+0000 7f4ff9200640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4fcc06c430 0x7f4fcc06e8f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.532+0000 7f4ff9200640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4ff415b740 0x7f4ff409bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.532+0000 7f4ff9200640 1 --2- 172.21.15.26:0/1327350535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4fd4003970 0x7f4ff409d870 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.532+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 >> 172.21.15.26:0/1327350535 conn(0x7f4ff40d0de0 msgr2=0x7f4ff4160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.532+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 shutdown_connections 2024-01-15T18:12:18.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:18.532+0000 7f4ff9200640 1 -- 172.21.15.26:0/1327350535 wait complete. 2024-01-15T18:12:18.555 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:12:18.555 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:12:18.555 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi026.front.sepia.ceph.com 2024-01-15T18:12:18.555 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-15T18:12:18.555 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-15T18:12:18.555 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:12:18.555 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:12:18.555 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:12:18.555 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:12:18.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:18.556 DEBUG:teuthology.orchestra.run.smithi026:> ip addr 2024-01-15T18:12:18.607 INFO:teuthology.orchestra.run.smithi026.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:12:18.607 INFO:teuthology.orchestra.run.smithi026.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:12:18.607 INFO:teuthology.orchestra.run.smithi026.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:12:18.608 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:18.608 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 ::1/128 scope host 2024-01-15T18:12:18.608 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:18.608 INFO:teuthology.orchestra.run.smithi026.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:12:18.608 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:18.608 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f0 2024-01-15T18:12:18.608 INFO:teuthology.orchestra.run.smithi026.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:12:18.608 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether 0c:c4:7a:6b:ee:6b brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:18.608 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp7s0f1 2024-01-15T18:12:18.609 INFO:teuthology.orchestra.run.smithi026.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:12:18.609 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c0 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:18.609 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f0 2024-01-15T18:12:18.609 INFO:teuthology.orchestra.run.smithi026.stdout: inet 172.21.15.26/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-15T18:12:18.609 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft 41579sec preferred_lft 41579sec 2024-01-15T18:12:18.609 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::ae1f:6bff:fea5:cac0/64 scope link noprefixroute 2024-01-15T18:12:18.609 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:18.609 INFO:teuthology.orchestra.run.smithi026.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:12:18.609 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether ac:1f:6b:a5:ca:c1 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: altname enp1s0f1 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::a4e5:3ff:fe91:fcad/64 scope link 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether f6:64:8f:cd:58:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::f464:8fff:fecd:58d2/64 scope link 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: link/ether fe:82:c8:bc:3f:d2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: inet6 fe80::fc82:c8ff:febc:3fd2/64 scope link 2024-01-15T18:12:18.610 INFO:teuthology.orchestra.run.smithi026.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:18.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:18.612 DEBUG:teuthology.orchestra.run.smithi026:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:12:18.700 INFO:teuthology.orchestra.run.smithi026.stdout:1 2024-01-15T18:12:18.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:18.705 DEBUG:teuthology.orchestra.run.smithi026:> route 2024-01-15T18:12:18.758 INFO:teuthology.orchestra.run.smithi026.stdout:Kernel IP routing table 2024-01-15T18:12:18.758 INFO:teuthology.orchestra.run.smithi026.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:12:18.758 INFO:teuthology.orchestra.run.smithi026.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-15T18:12:18.758 INFO:teuthology.orchestra.run.smithi026.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-15T18:12:18.758 INFO:teuthology.orchestra.run.smithi026.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:12:18.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:18.759 DEBUG:teuthology.orchestra.run.smithi026:> (cd / && exec bash -c ' 2024-01-15T18:12:18.759 DEBUG:teuthology.orchestra.run.smithi026:> set -e 2024-01-15T18:12:18.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-15T18:12:18.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-15T18:12:18.759 DEBUG:teuthology.orchestra.run.smithi026:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-15T18:12:18.760 DEBUG:teuthology.orchestra.run.smithi026:> ') 2024-01-15T18:12:18.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:18.954 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T18:12:19.005 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:12:19.005 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:12:19.006 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:12:19.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:19.006 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link set brx.0 up 2024-01-15T18:12:19.106 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-01-15T18:12:19.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:12:19.106 DEBUG:teuthology.orchestra.run.smithi026:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-15T18:12:19.159 INFO:teuthology.orchestra.run.smithi026.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-15T18:12:19.159 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:12:19.160 DEBUG:teuthology.orchestra.run.smithi026:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:12:19.211 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:12:19.313 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:12:19.371 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:12:19.371 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:12:19.371 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:12:19.371 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:12:19.371 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:12:19.372 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:12:19.373 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:12:19.374 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:12:19.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:19.374 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:12:19.431 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:12:19.431 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-01-15T18:12:19.438 DEBUG:teuthology.orchestra.run.smithi026:> sudo modprobe fuse 2024-01-15T18:12:19.518 DEBUG:teuthology.orchestra.run.smithi026:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:12:19.562 INFO:teuthology.orchestra.run.smithi026.stdout:/proc 2024-01-15T18:12:19.562 INFO:teuthology.orchestra.run.smithi026.stdout:/sys 2024-01-15T18:12:19.562 INFO:teuthology.orchestra.run.smithi026.stdout:/dev 2024-01-15T18:12:19.562 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/security 2024-01-15T18:12:19.562 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/shm 2024-01-15T18:12:19.562 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/pts 2024-01-15T18:12:19.562 INFO:teuthology.orchestra.run.smithi026.stdout:/run 2024-01-15T18:12:19.562 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/cgroup 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/pstore 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/ 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/selinux 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/hugepages 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/dev/mqueue 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/debug 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/tracing 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/fs/fuse/connections 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/sys/kernel/config 2024-01-15T18:12:19.563 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/run/user/1000 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-0 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-1 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-2 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/var/lib/ceph/osd/ceph-3 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:12:19.564 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:12:19.565 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:12:19.565 INFO:teuthology.orchestra.run.smithi026.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:12:19.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:19.566 DEBUG:teuthology.orchestra.run.smithi026:> ls /sys/fs/fuse/connections 2024-01-15T18:12:19.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:2024-01-15T18:12:19.624+0000 7fd321c46480 -1 init, newargv = 0x558518788ca0 newargc=15 2024-01-15T18:12:19.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[76436]: starting ceph client 2024-01-15T18:12:19.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[76436]: starting fuse 2024-01-15T18:12:19.651 INFO:teuthology.orchestra.run.smithi026.stdout:41 2024-01-15T18:12:19.651 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:12:19.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:12:19.651 DEBUG:teuthology.orchestra.run.smithi026:> import glob 2024-01-15T18:12:19.651 DEBUG:teuthology.orchestra.run.smithi026:> import re 2024-01-15T18:12:19.651 DEBUG:teuthology.orchestra.run.smithi026:> import os 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> import subprocess 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> def _find_admin_socket(client_name): 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> files = glob.glob(asok_path) 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> # Given a non-glob path, it better be there 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> if "*" not in asok_path: 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> assert(len(files) == 1) 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> return files[0] 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:12:19.652 DEBUG:teuthology.orchestra.run.smithi026:> for f in files: 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> contents = proc_f.read() 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> if mountpoint in contents: 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> return f 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> print(_find_admin_socket("client.0")) 2024-01-15T18:12:19.653 DEBUG:teuthology.orchestra.run.smithi026:> ' 2024-01-15T18:12:19.853 INFO:teuthology.orchestra.run.smithi026.stdout:/var/run/ceph/ceph-client.0.76436.asok 2024-01-15T18:12:19.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:12:19.871 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76436.asok 2024-01-15T18:12:19.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:19.871 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76436.asok status 2024-01-15T18:12:20.117 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:12:20.117 INFO:teuthology.orchestra.run.smithi026.stdout: "metadata": { 2024-01-15T18:12:20.117 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:12:20.117 INFO:teuthology.orchestra.run.smithi026.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:12:20.117 INFO:teuthology.orchestra.run.smithi026.stdout: "entity_id": "0", 2024-01-15T18:12:20.117 INFO:teuthology.orchestra.run.smithi026.stdout: "hostname": "smithi026", 2024-01-15T18:12:20.117 INFO:teuthology.orchestra.run.smithi026.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-01-15T18:12:20.117 INFO:teuthology.orchestra.run.smithi026.stdout: "pid": "76436", 2024-01-15T18:12:20.117 INFO:teuthology.orchestra.run.smithi026.stdout: "root": "/" 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_count": 0, 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "dentry_pinned_count": 0, 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "id": 9366, 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "inst": { 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "name": { 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "client", 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "num": 9366 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:12:20.118 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3384207310 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": { 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "type": "v1", 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "nonce": 3384207310 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "inst_str": "client.9366 192.168.0.1:0/3384207310", 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "addr_str": "192.168.0.1:0/3384207310", 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "inode_count": 1, 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "mds_epoch": 72, 2024-01-15T18:12:20.119 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch": 115, 2024-01-15T18:12:20.120 INFO:teuthology.orchestra.run.smithi026.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:12:20.120 INFO:teuthology.orchestra.run.smithi026.stdout: "blocklisted": false, 2024-01-15T18:12:20.120 INFO:teuthology.orchestra.run.smithi026.stdout: "fs_name": "cephfs" 2024-01-15T18:12:20.120 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:12:20.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:20.133 DEBUG:teuthology.orchestra.run.smithi026:> stat --file-system '--printf=%T 2024-01-15T18:12:20.133 DEBUG:teuthology.orchestra.run.smithi026:> ' -- /home/ubuntu/cephtest/mnt.0 2024-01-15T18:12:20.187 INFO:teuthology.orchestra.run.smithi026.stdout:fuseblk 2024-01-15T18:12:20.187 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-01-15T18:12:20.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:20.188 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-15T18:12:20.290 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-15T18:12:20.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.512+0000 7f22b3200640 1 Processor -- start 2024-01-15T18:12:20.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.512+0000 7f22b3200640 1 -- start start 2024-01-15T18:12:20.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.514+0000 7f22b3200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f22ac0cc500 0x7f22ac0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:20.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.514+0000 7f22b3200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f22ac06d160 0x7f22ac06d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:20.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.514+0000 7f22b3200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac070e50 con 0x7f22ac143c40 2024-01-15T18:12:20.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.514+0000 7f22b3200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac0723c0 con 0x7f22ac06d160 2024-01-15T18:12:20.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.514+0000 7f22b3200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac0cced0 con 0x7f22ac0cc500 2024-01-15T18:12:20.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.514+0000 7f22b1e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f22ac143c40 0x7f22ac071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:36508/0 (socket says 172.21.15.26:36508) 2024-01-15T18:12:20.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.514+0000 7f22b1e00640 1 -- 172.21.15.26:0/1679767080 learned_addr learned my addr 172.21.15.26:0/1679767080 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:20.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.514+0000 7f22b2800640 1 --2- 172.21.15.26:0/1679767080 >> v2:172.21.15.26:3300/0 conn(0x7f22ac06d160 0x7f22ac06d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:20.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.515+0000 7f22b1400640 1 --2- 172.21.15.26:0/1679767080 >> v2:172.21.15.124:3300/0 conn(0x7f22ac0cc500 0x7f22ac0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:20.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.515+0000 7f22b0a00640 1 -- 172.21.15.26:0/1679767080 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352247228 0 0) 0x7f22ac070e50 con 0x7f22ac143c40 2024-01-15T18:12:20.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.515+0000 7f22b0a00640 1 -- 172.21.15.26:0/1679767080 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f228c003520 con 0x7f22ac143c40 2024-01-15T18:12:20.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.515+0000 7f22b2800640 1 -- 172.21.15.26:0/1679767080 >> v2:172.21.15.124:3300/0 conn(0x7f22ac0cc500 msgr2=0x7f22ac0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:20.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.515+0000 7f22b2800640 1 --2- 172.21.15.26:0/1679767080 >> v2:172.21.15.124:3300/0 conn(0x7f22ac0cc500 0x7f22ac0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.515+0000 7f22b2800640 1 -- 172.21.15.26:0/1679767080 >> v1:172.21.15.26:6789/0 conn(0x7f22ac143c40 legacy=0x7f22ac071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:20.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.515+0000 7f22b2800640 1 -- 172.21.15.26:0/1679767080 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac144020 con 0x7f22ac06d160 2024-01-15T18:12:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.516+0000 7f22b2800640 1 --2- 172.21.15.26:0/1679767080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac06d160 0x7f22ac06d560 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0b44a0 tx=0x7f22a402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94c4322f66a128f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.516+0000 7f22b0a00640 1 -- 172.21.15.26:0/1679767080 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a402cbf0 con 0x7f22ac06d160 2024-01-15T18:12:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.516+0000 7f22b0a00640 1 -- 172.21.15.26:0/1679767080 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a402cd50 con 0x7f22ac06d160 2024-01-15T18:12:20.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.516+0000 7f22b0a00640 1 -- 172.21.15.26:0/1679767080 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4034640 con 0x7f22ac06d160 2024-01-15T18:12:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.517+0000 7f22b3200640 1 -- 172.21.15.26:0/1679767080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac06d160 msgr2=0x7f22ac06d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.517+0000 7f22b3200640 1 --2- 172.21.15.26:0/1679767080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac06d160 0x7f22ac06d560 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0b44a0 tx=0x7f22a402be40 comp rx=0 tx=0).stop 2024-01-15T18:12:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.517+0000 7f22b3200640 1 -- 172.21.15.26:0/1679767080 shutdown_connections 2024-01-15T18:12:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.517+0000 7f22b3200640 1 --2- 172.21.15.26:0/1679767080 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac06d160 0x7f22ac06d560 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.517+0000 7f22b3200640 1 --2- 172.21.15.26:0/1679767080 >> v2:172.21.15.124:3300/0 conn(0x7f22ac0cc500 0x7f22ac0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.517+0000 7f22b3200640 1 -- 172.21.15.26:0/1679767080 >> 172.21.15.26:0/1679767080 conn(0x7f22ac06a800 msgr2=0x7f22ac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.517+0000 7f22b3200640 1 -- 172.21.15.26:0/1679767080 shutdown_connections 2024-01-15T18:12:20.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.517+0000 7f22b3200640 1 -- 172.21.15.26:0/1679767080 wait complete. 2024-01-15T18:12:20.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.519+0000 7f22b3200640 1 Processor -- start 2024-01-15T18:12:20.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.519+0000 7f22b3200640 1 -- start start 2024-01-15T18:12:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.520+0000 7f22b3200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22ac06d160 0x7f22ac15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.520+0000 7f22b3200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0cc500 0x7f22ac140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.520+0000 7f22b3200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22ac143c40 0x7f22ac140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.520+0000 7f22b3200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac12bfe0 con 0x7f22ac0cc500 2024-01-15T18:12:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.520+0000 7f22b3200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac12f4d0 con 0x7f22ac06d160 2024-01-15T18:12:20.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.520+0000 7f22b3200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac12ee10 con 0x7f22ac143c40 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0cc500 0x7f22ac140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b1400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0cc500 0x7f22ac140300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34846/0 (socket says 172.21.15.26:34846) 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b1400640 1 -- 172.21.15.26:0/2399889535 learned_addr learned my addr 172.21.15.26:0/2399889535 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b1e00640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22ac06d160 0x7f22ac15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b2800640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22ac143c40 0x7f22ac140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b1400640 1 -- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22ac143c40 msgr2=0x7f22ac140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b1400640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22ac143c40 0x7f22ac140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b1400640 1 -- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22ac06d160 msgr2=0x7f22ac15a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b1400640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22ac06d160 0x7f22ac15a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b1400640 1 -- 172.21.15.26:0/2399889535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4009a70 con 0x7f22ac0cc500 2024-01-15T18:12:20.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.521+0000 7f22b2800640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22ac143c40 0x7f22ac140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:20.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.522+0000 7f22b1400640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0cc500 0x7f22ac140300 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f229400d800 tx=0x7f229400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:20.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.522+0000 7f22aa000640 1 -- 172.21.15.26:0/2399889535 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229400b6c0 con 0x7f22ac0cc500 2024-01-15T18:12:20.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.522+0000 7f22aa000640 1 -- 172.21.15.26:0/2399889535 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229400b820 con 0x7f22ac0cc500 2024-01-15T18:12:20.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.522+0000 7f22aa000640 1 -- 172.21.15.26:0/2399889535 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229400bbb0 con 0x7f22ac0cc500 2024-01-15T18:12:20.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.522+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac12c1f0 con 0x7f22ac0cc500 2024-01-15T18:12:20.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.522+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac12c7b0 con 0x7f22ac0cc500 2024-01-15T18:12:20.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.523+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac12c970 con 0x7f22ac0cc500 2024-01-15T18:12:20.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.524+0000 7f22aa000640 1 -- 172.21.15.26:0/2399889535 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f229400bd10 con 0x7f22ac0cc500 2024-01-15T18:12:20.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.525+0000 7f22aa000640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f228406c430 0x7f228406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:20.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.525+0000 7f22aa000640 1 -- 172.21.15.26:0/2399889535 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f2294088fb0 con 0x7f22ac0cc500 2024-01-15T18:12:20.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.526+0000 7f22b1e00640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f228406c430 0x7f228406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:20.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.527+0000 7f22b1e00640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f228406c430 0x7f228406e8f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f229c005880 tx=0x7f229c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:20.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.531+0000 7f22aa000640 1 -- 172.21.15.26:0/2399889535 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2294055520 con 0x7f22ac0cc500 2024-01-15T18:12:20.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.762+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f22ac067450 con 0x7f22ac0cc500 2024-01-15T18:12:20.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.763+0000 7f22aa000640 1 -- 172.21.15.26:0/2399889535 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v72) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2294058b40 con 0x7f22ac0cc500 2024-01-15T18:12:20.767 INFO:teuthology.orchestra.run.smithi026.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-15T18:12:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.765+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f228406c430 msgr2=0x7f228406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.765+0000 7f22b3200640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f228406c430 0x7f228406e8f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f229c005880 tx=0x7f229c009040 comp rx=0 tx=0).stop 2024-01-15T18:12:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.765+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0cc500 msgr2=0x7f22ac140300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.765+0000 7f22b3200640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0cc500 0x7f22ac140300 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f229400d800 tx=0x7f229400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.766+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 shutdown_connections 2024-01-15T18:12:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.766+0000 7f22b3200640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f22ac143c40 0x7f22ac140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.766+0000 7f22b3200640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f228406c430 0x7f228406e8f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.766+0000 7f22b3200640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f22ac0cc500 0x7f22ac140300 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.766+0000 7f22b3200640 1 --2- 172.21.15.26:0/2399889535 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f22ac06d160 0x7f22ac15a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.766+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 >> 172.21.15.26:0/2399889535 conn(0x7f22ac06a800 msgr2=0x7f22ac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.766+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 shutdown_connections 2024-01-15T18:12:20.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:20.766+0000 7f22b3200640 1 -- 172.21.15.26:0/2399889535 wait complete. 2024-01-15T18:12:20.789 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-15T18:12:20.789 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-15T18:12:20.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi124.front.sepia.ceph.com 2024-01-15T18:12:20.789 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-01-15T18:12:20.789 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-01-15T18:12:20.789 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-15T18:12:20.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-15T18:12:20.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-15T18:12:20.790 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:12:20.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:20.790 DEBUG:teuthology.orchestra.run.smithi124:> ip addr 2024-01-15T18:12:20.848 INFO:teuthology.orchestra.run.smithi124.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-15T18:12:20.848 INFO:teuthology.orchestra.run.smithi124.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-15T18:12:20.848 INFO:teuthology.orchestra.run.smithi124.stdout: inet 127.0.0.1/8 scope host lo 2024-01-15T18:12:20.848 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:20.848 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 ::1/128 scope host 2024-01-15T18:12:20.848 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:20.848 INFO:teuthology.orchestra.run.smithi124.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a6 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f0 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:d8:a1:a7 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: altname enp7s0f1 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c4 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 0c:c4:7a:8f:d5:c5 brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: inet 172.21.15.124/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft 41575sec preferred_lft 41575sec 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::ec4:7aff:fe8f:d5c5/64 scope link 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-01-15T18:12:20.849 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::4c2d:94ff:fe47:f54c/64 scope link 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 26:4f:4f:5b:0b:31 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::244f:4fff:fe5b:b31/64 scope link 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: link/ether 22:70:16:92:4a:4a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: inet6 fe80::2070:16ff:fe92:4a4a/64 scope link 2024-01-15T18:12:20.850 INFO:teuthology.orchestra.run.smithi124.stdout: valid_lft forever preferred_lft forever 2024-01-15T18:12:20.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:20.851 DEBUG:teuthology.orchestra.run.smithi124:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-15T18:12:20.943 INFO:teuthology.orchestra.run.smithi124.stdout:1 2024-01-15T18:12:20.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:20.948 DEBUG:teuthology.orchestra.run.smithi124:> route 2024-01-15T18:12:20.999 INFO:teuthology.orchestra.run.smithi124.stdout:Kernel IP routing table 2024-01-15T18:12:20.999 INFO:teuthology.orchestra.run.smithi124.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-15T18:12:20.999 INFO:teuthology.orchestra.run.smithi124.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-15T18:12:20.999 INFO:teuthology.orchestra.run.smithi124.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-15T18:12:20.999 INFO:teuthology.orchestra.run.smithi124.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-15T18:12:21.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:21.000 DEBUG:teuthology.orchestra.run.smithi124:> (cd / && exec bash -c ' 2024-01-15T18:12:21.000 DEBUG:teuthology.orchestra.run.smithi124:> set -e 2024-01-15T18:12:21.000 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-15T18:12:21.000 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-15T18:12:21.001 DEBUG:teuthology.orchestra.run.smithi124:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-15T18:12:21.001 DEBUG:teuthology.orchestra.run.smithi124:> ') 2024-01-15T18:12:21.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:21.181 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T18:12:21.233 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:12:21.233 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:12:21.234 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:12:21.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:21.234 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link set brx.0 up 2024-01-15T18:12:21.326 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-01-15T18:12:21.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:12:21.327 DEBUG:teuthology.orchestra.run.smithi124:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-01-15T18:12:21.376 INFO:teuthology.orchestra.run.smithi124.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-01-15T18:12:21.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-15T18:12:21.377 DEBUG:teuthology.orchestra.run.smithi124:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:12:21.423 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:12:21.561 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:12:21.614 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:12:21.615 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:12:21.616 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:12:21.617 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:12:21.617 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:12:21.617 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:12:21.617 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:12:21.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:21.618 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:12:21.668 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-01-15T18:12:21.668 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-01-15T18:12:21.675 DEBUG:teuthology.orchestra.run.smithi124:> sudo modprobe fuse 2024-01-15T18:12:21.751 DEBUG:teuthology.orchestra.run.smithi124:> cat /proc/self/mounts | awk '{print $2}' 2024-01-15T18:12:21.793 INFO:teuthology.orchestra.run.smithi124.stdout:/proc 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/sys 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/dev 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/security 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/shm 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/pts 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/run 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/cgroup 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/pstore 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/ 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/selinux 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/proc/sys/fs/binfmt_misc 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/mqueue 2024-01-15T18:12:21.794 INFO:teuthology.orchestra.run.smithi124.stdout:/dev/hugepages 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/debug 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/tracing 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/fs/fuse/connections 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/sys/kernel/config 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-sysctl.service 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/nfs/rpc_pipefs 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/run/user/1000 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-4 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-5 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-6 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/var/lib/ceph/osd/ceph-7 2024-01-15T18:12:21.795 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns 2024-01-15T18:12:21.796 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:12:21.796 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:12:21.796 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:12:21.796 INFO:teuthology.orchestra.run.smithi124.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:12:21.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:21.797 DEBUG:teuthology.orchestra.run.smithi124:> ls /sys/fs/fuse/connections 2024-01-15T18:12:21.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:2024-01-15T18:12:21.856+0000 7f973763f480 -1 init, newargv = 0x5612a8896ad0 newargc=15 2024-01-15T18:12:21.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[48133]: starting ceph client 2024-01-15T18:12:21.871 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[48133]: starting fuse 2024-01-15T18:12:21.881 INFO:teuthology.orchestra.run.smithi124.stdout:41 2024-01-15T18:12:21.881 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-01-15T18:12:21.881 DEBUG:teuthology.orchestra.run.smithi124:> sudo stdin-killer -- python3 -c ' 2024-01-15T18:12:21.881 DEBUG:teuthology.orchestra.run.smithi124:> import glob 2024-01-15T18:12:21.881 DEBUG:teuthology.orchestra.run.smithi124:> import re 2024-01-15T18:12:21.881 DEBUG:teuthology.orchestra.run.smithi124:> import os 2024-01-15T18:12:21.881 DEBUG:teuthology.orchestra.run.smithi124:> import subprocess 2024-01-15T18:12:21.881 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:12:21.881 DEBUG:teuthology.orchestra.run.smithi124:> def _find_admin_socket(client_name): 2024-01-15T18:12:21.881 DEBUG:teuthology.orchestra.run.smithi124:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-01-15T18:12:21.881 DEBUG:teuthology.orchestra.run.smithi124:> files = glob.glob(asok_path) 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> # Given a non-glob path, it better be there 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> if "*" not in asok_path: 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> assert(len(files) == 1) 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> return files[0] 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> for f in files: 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> if os.path.exists("/proc/{0}".format(pid)): 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> contents = proc_f.read() 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> if mountpoint in contents: 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> return f 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> 2024-01-15T18:12:21.882 DEBUG:teuthology.orchestra.run.smithi124:> print(_find_admin_socket("client.1")) 2024-01-15T18:12:21.883 DEBUG:teuthology.orchestra.run.smithi124:> ' 2024-01-15T18:12:22.073 INFO:teuthology.orchestra.run.smithi124.stdout:/var/run/ceph/ceph-client.1.48133.asok 2024-01-15T18:12:22.078 INFO:teuthology.orchestra.run.smithi124.stderr:2024-01-15T18:12:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-01-15T18:12:22.090 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.48133.asok 2024-01-15T18:12:22.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:22.091 DEBUG:teuthology.orchestra.run.smithi124:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.48133.asok status 2024-01-15T18:12:22.330 INFO:teuthology.orchestra.run.smithi124.stdout:{ 2024-01-15T18:12:22.330 INFO:teuthology.orchestra.run.smithi124.stdout: "metadata": { 2024-01-15T18:12:22.330 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_sha1": "95c90f413c4bb65e3e56909855f2308fd9bda05b", 2024-01-15T18:12:22.330 INFO:teuthology.orchestra.run.smithi124.stdout: "ceph_version": "ceph version 18.0.0-7629-g95c90f41 (95c90f413c4bb65e3e56909855f2308fd9bda05b) reef (dev)", 2024-01-15T18:12:22.330 INFO:teuthology.orchestra.run.smithi124.stdout: "entity_id": "1", 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "hostname": "smithi124", 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "pid": "48133", 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "root": "/" 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_count": 0, 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "dentry_pinned_count": 0, 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "id": 7432, 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "inst": { 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "name": { 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "client", 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "num": 7432 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:12:22.331 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 968693173 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: } 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": { 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "type": "v1", 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "addr": "192.168.0.1:0", 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "nonce": 968693173 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: }, 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "inst_str": "client.7432 192.168.0.1:0/968693173", 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "addr_str": "192.168.0.1:0/968693173", 2024-01-15T18:12:22.332 INFO:teuthology.orchestra.run.smithi124.stdout: "inode_count": 1, 2024-01-15T18:12:22.333 INFO:teuthology.orchestra.run.smithi124.stdout: "mds_epoch": 72, 2024-01-15T18:12:22.333 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch": 115, 2024-01-15T18:12:22.333 INFO:teuthology.orchestra.run.smithi124.stdout: "osd_epoch_barrier": 0, 2024-01-15T18:12:22.333 INFO:teuthology.orchestra.run.smithi124.stdout: "blocklisted": false, 2024-01-15T18:12:22.333 INFO:teuthology.orchestra.run.smithi124.stdout: "fs_name": "cephfs" 2024-01-15T18:12:22.333 INFO:teuthology.orchestra.run.smithi124.stdout:} 2024-01-15T18:12:22.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:22.345 DEBUG:teuthology.orchestra.run.smithi124:> stat --file-system '--printf=%T 2024-01-15T18:12:22.345 DEBUG:teuthology.orchestra.run.smithi124:> ' -- /home/ubuntu/cephtest/mnt.1 2024-01-15T18:12:22.396 INFO:teuthology.orchestra.run.smithi124.stdout:fuseblk 2024-01-15T18:12:22.396 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-01-15T18:12:22.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:22.396 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-01-15T18:12:22.535 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-01-15T18:12:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.764+0000 7f894d200640 1 Processor -- start 2024-01-15T18:12:22.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.764+0000 7f894d200640 1 -- start start 2024-01-15T18:12:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.766+0000 7f894d200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f894813df80 0x7f894815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.766+0000 7f894d200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894806c470 con 0x7f894813e360 2024-01-15T18:12:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.766+0000 7f894d200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894806c750 con 0x7f89480cf9a0 2024-01-15T18:12:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.766+0000 7f894d200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894806ca30 con 0x7f894813df80 2024-01-15T18:12:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.766+0000 7f8945400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f894813e360 0x7f89480cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45204/0 (socket says 172.21.15.26:45204) 2024-01-15T18:12:22.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.766+0000 7f8945400640 1 -- 172.21.15.26:0/2821696981 learned_addr learned my addr 172.21.15.26:0/2821696981 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8945e00640 1 --2- 172.21.15.26:0/2821696981 >> v2:172.21.15.124:3300/0 conn(0x7f894813df80 0x7f894815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887950204 0 0) 0x7f894806c470 con 0x7f894813e360 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8928003520 con 0x7f894813e360 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1938334620 0 0) 0x7f8928003520 con 0x7f894813e360 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 >> v2:172.21.15.124:3300/0 conn(0x7f894813df80 msgr2=0x7f894815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 --2- 172.21.15.26:0/2821696981 >> v2:172.21.15.124:3300/0 conn(0x7f894813df80 0x7f894815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 >> v1:172.21.15.124:6789/0 conn(0x7f89480cf9a0 legacy=0x7f89480cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894806cd10 con 0x7f894813e360 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f8930001670 con 0x7f894813e360 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 >> v1:172.21.15.26:6789/0 conn(0x7f894813e360 legacy=0x7f89480cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:22.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.767+0000 7f8944a00640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f8928003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.768+0000 7f8944a00640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89280047c0 0x7f8928004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.768+0000 7f8944a00640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8928005180 0x7f892800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.768+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894806ca30 con 0x7f8928003970 2024-01-15T18:12:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.768+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892800c020 con 0x7f8928005180 2024-01-15T18:12:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.768+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892800c1e0 con 0x7f89280047c0 2024-01-15T18:12:22.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.768+0000 7f8945e00640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f8928003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.768+0000 7f8946800640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8928005180 0x7f892800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.768+0000 7f8945400640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89280047c0 0x7f8928004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.769+0000 7f8945e00640 1 -- 172.21.15.26:0/2821696981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89280047c0 msgr2=0x7f8928004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.769+0000 7f8945e00640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89280047c0 0x7f8928004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.769+0000 7f8945e00640 1 -- 172.21.15.26:0/2821696981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8928005180 msgr2=0x7f892800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.769+0000 7f8945e00640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8928005180 0x7f892800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.769+0000 7f8946800640 1 -- 172.21.15.26:0/2821696981 reap_dead start 2024-01-15T18:12:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.769+0000 7f8945e00640 1 -- 172.21.15.26:0/2821696981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8928003520 con 0x7f8928003970 2024-01-15T18:12:22.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.769+0000 7f8945400640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89280047c0 0x7f8928004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.770+0000 7f8945e00640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f8928003d90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f8938009ac0 tx=0x7f89280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf6528a557481c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:22.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.771+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893802c6b0 con 0x7f8928003970 2024-01-15T18:12:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.771+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893802c810 con 0x7f8928003970 2024-01-15T18:12:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.771+0000 7f8944a00640 1 -- 172.21.15.26:0/2821696981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893802cba0 con 0x7f8928003970 2024-01-15T18:12:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.771+0000 7f894d200640 1 -- 172.21.15.26:0/2821696981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 msgr2=0x7f8928003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.771+0000 7f894d200640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f8928003d90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f8938009ac0 tx=0x7f89280057f0 comp rx=0 tx=0).stop 2024-01-15T18:12:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.771+0000 7f894d200640 1 -- 172.21.15.26:0/2821696981 shutdown_connections 2024-01-15T18:12:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.771+0000 7f894d200640 1 --2- 172.21.15.26:0/2821696981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f8928003d90 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.771+0000 7f894d200640 1 -- 172.21.15.26:0/2821696981 >> 172.21.15.26:0/2821696981 conn(0x7f894806a800 msgr2=0x7f894806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.771+0000 7f894d200640 1 -- 172.21.15.26:0/2821696981 shutdown_connections 2024-01-15T18:12:22.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.772+0000 7f894d200640 1 -- 172.21.15.26:0/2821696981 wait complete. 2024-01-15T18:12:22.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.773+0000 7f894d200640 1 Processor -- start 2024-01-15T18:12:22.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.773+0000 7f894d200640 1 -- start start 2024-01-15T18:12:22.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.774+0000 7f894d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f894814b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.774+0000 7f894d200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89481f2da0 0x7f894814df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f894d200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8948071ad0 0x7f8948071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f894d200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894814e440 con 0x7f8928003970 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f894d200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8948070a30 con 0x7f8948071ad0 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f894d200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8948070d10 con 0x7f89481f2da0 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8945e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f894814b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8945e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f894814b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57902/0 (socket says 172.21.15.26:57902) 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8945e00640 1 -- 172.21.15.26:0/1952501433 learned_addr learned my addr 172.21.15.26:0/1952501433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8945400640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89481f2da0 0x7f894814df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8946800640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8948071ad0 0x7f8948071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8945e00640 1 -- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89481f2da0 msgr2=0x7f894814df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8945e00640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89481f2da0 0x7f894814df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8945e00640 1 -- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8948071ad0 msgr2=0x7f8948071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:22.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8945e00640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8948071ad0 0x7f8948071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:22.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.775+0000 7f8945e00640 1 -- 172.21.15.26:0/1952501433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8948070ff0 con 0x7f8928003970 2024-01-15T18:12:22.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.776+0000 7f8945e00640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f894814b110 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f893802c250 tx=0x7f8938004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:22.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.776+0000 7f893e000640 1 -- 172.21.15.26:0/1952501433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938003810 con 0x7f8928003970 2024-01-15T18:12:22.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.776+0000 7f893e000640 1 -- 172.21.15.26:0/1952501433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8938003970 con 0x7f8928003970 2024-01-15T18:12:22.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.776+0000 7f893e000640 1 -- 172.21.15.26:0/1952501433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8938003d00 con 0x7f8928003970 2024-01-15T18:12:22.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.776+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89480712d0 con 0x7f8928003970 2024-01-15T18:12:22.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.776+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894814e690 con 0x7f8928003970 2024-01-15T18:12:22.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.777+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894806dc60 con 0x7f8928003970 2024-01-15T18:12:22.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.778+0000 7f893e000640 1 -- 172.21.15.26:0/1952501433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f8938034440 con 0x7f8928003970 2024-01-15T18:12:22.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.780+0000 7f893e000640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006c460 0x7f892006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:22.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.780+0000 7f893e000640 1 -- 172.21.15.26:0/1952501433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f89380aac40 con 0x7f8928003970 2024-01-15T18:12:22.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.780+0000 7f8945400640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006c460 0x7f892006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:22.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.781+0000 7f8945400640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006c460 0x7f892006e920 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f89480b38e0 tx=0x7f8930009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:22.789 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:22.785+0000 7f893e000640 1 -- 172.21.15.26:0/1952501433 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89380772c0 con 0x7f8928003970 2024-01-15T18:12:23.026 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.022+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f89480675d0 con 0x7f8928003970 2024-01-15T18:12:23.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.382+0000 7f893e000640 1 -- 172.21.15.26:0/1952501433 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 136+0+0 (secure 0 0 0) 0x7f8938003e60 con 0x7f8928003970 2024-01-15T18:12:23.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.386+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006c460 msgr2=0x7f892006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.386+0000 7f894d200640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006c460 0x7f892006e920 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f89480b38e0 tx=0x7f8930009040 comp rx=0 tx=0).stop 2024-01-15T18:12:23.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.386+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 msgr2=0x7f894814b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.386+0000 7f894d200640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f894814b110 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f893802c250 tx=0x7f8938004730 comp rx=0 tx=0).stop 2024-01-15T18:12:23.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.386+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 shutdown_connections 2024-01-15T18:12:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.386+0000 7f894d200640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f89481f2da0 0x7f894814df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.387+0000 7f894d200640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f892006c460 0x7f892006e920 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.387+0000 7f894d200640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8948071ad0 0x7f8948071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.387+0000 7f894d200640 1 --2- 172.21.15.26:0/1952501433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8928003970 0x7f894814b110 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.387+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 >> 172.21.15.26:0/1952501433 conn(0x7f894806a800 msgr2=0x7f894815d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.387+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 shutdown_connections 2024-01-15T18:12:23.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.387+0000 7f894d200640 1 -- 172.21.15.26:0/1952501433 wait complete. 2024-01-15T18:12:23.413 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:12:23.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.623+0000 7f24b7000640 1 Processor -- start 2024-01-15T18:12:23.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.623+0000 7f24b7000640 1 -- start start 2024-01-15T18:12:23.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.625+0000 7f24b7000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f24b013df80 0x7f24b015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:23.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.625+0000 7f24b7000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24b013e360 0x7f24b00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:23.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.625+0000 7f24b7000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b006c470 con 0x7f24b015f9c0 2024-01-15T18:12:23.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.625+0000 7f24b7000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b006c750 con 0x7f24b013e360 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.625+0000 7f24b7000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b006ca30 con 0x7f24b013df80 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.625+0000 7f24b5000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f24b015f9c0 0x7f24b01408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45212/0 (socket says 172.21.15.26:45212) 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.625+0000 7f24a7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f24b013e360 0x7f24b00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.625+0000 7f24b5000640 1 -- 172.21.15.26:0/3822659835 learned_addr learned my addr 172.21.15.26:0/3822659835 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.626+0000 7f24afe00640 1 --2- 172.21.15.26:0/3822659835 >> v2:172.21.15.124:3300/0 conn(0x7f24b013df80 0x7f24b015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.626+0000 7f24af400640 1 -- 172.21.15.26:0/3822659835 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3332244993 0 0) 0x7f24b006c470 con 0x7f24b015f9c0 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.626+0000 7f24af400640 1 -- 172.21.15.26:0/3822659835 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2490003520 con 0x7f24b015f9c0 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.626+0000 7f24a7e00640 1 -- 172.21.15.26:0/3822659835 >> v2:172.21.15.124:3300/0 conn(0x7f24b013df80 msgr2=0x7f24b015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.626+0000 7f24a7e00640 1 --2- 172.21.15.26:0/3822659835 >> v2:172.21.15.124:3300/0 conn(0x7f24b013df80 0x7f24b015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.626+0000 7f24a7e00640 1 -- 172.21.15.26:0/3822659835 >> v1:172.21.15.26:6789/0 conn(0x7f24b015f9c0 legacy=0x7f24b01408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.626+0000 7f24a7e00640 1 -- 172.21.15.26:0/3822659835 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b006cd10 con 0x7f24b013e360 2024-01-15T18:12:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.627+0000 7f24a7e00640 1 --2- 172.21.15.26:0/3822659835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 0x7f24b00cf290 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f24b0072770 tx=0x7f249802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9458d5b05732820 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.627+0000 7f24af400640 1 -- 172.21.15.26:0/3822659835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498031830 con 0x7f24b013e360 2024-01-15T18:12:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.627+0000 7f24af400640 1 -- 172.21.15.26:0/3822659835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2498031990 con 0x7f24b013e360 2024-01-15T18:12:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.627+0000 7f24af400640 1 -- 172.21.15.26:0/3822659835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2498031d20 con 0x7f24b013e360 2024-01-15T18:12:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.628+0000 7f24b7000640 1 -- 172.21.15.26:0/3822659835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 msgr2=0x7f24b00cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.628+0000 7f24b7000640 1 --2- 172.21.15.26:0/3822659835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 0x7f24b00cf290 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f24b0072770 tx=0x7f249802b8d0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.628+0000 7f24b7000640 1 -- 172.21.15.26:0/3822659835 shutdown_connections 2024-01-15T18:12:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.628+0000 7f24b7000640 1 --2- 172.21.15.26:0/3822659835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 0x7f24b00cf290 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.628+0000 7f24b7000640 1 --2- 172.21.15.26:0/3822659835 >> v2:172.21.15.124:3300/0 conn(0x7f24b013df80 0x7f24b015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.628+0000 7f24b7000640 1 -- 172.21.15.26:0/3822659835 >> 172.21.15.26:0/3822659835 conn(0x7f24b006a800 msgr2=0x7f24b006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.628+0000 7f24b7000640 1 -- 172.21.15.26:0/3822659835 shutdown_connections 2024-01-15T18:12:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.628+0000 7f24b7000640 1 -- 172.21.15.26:0/3822659835 wait complete. 2024-01-15T18:12:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.630+0000 7f24b7000640 1 Processor -- start 2024-01-15T18:12:23.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.630+0000 7f24b7000640 1 -- start start 2024-01-15T18:12:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.631+0000 7f24b7000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b013df80 0x7f24b01f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.631+0000 7f24b7000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 0x7f24b01e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.631+0000 7f24b7000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b015f9c0 0x7f24b01e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.631+0000 7f24b7000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b01e6a30 con 0x7f24b013e360 2024-01-15T18:12:23.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.631+0000 7f24b7000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b01e6d10 con 0x7f24b015f9c0 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.631+0000 7f24b7000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b01e6ff0 con 0x7f24b013df80 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24a7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 0x7f24b01e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24a7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 0x7f24b01e60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57918/0 (socket says 172.21.15.26:57918) 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24a7e00640 1 -- 172.21.15.26:0/525631565 learned_addr learned my addr 172.21.15.26:0/525631565 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24b5000640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b015f9c0 0x7f24b01e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24a7e00640 1 -- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b013df80 msgr2=0x7f24b01f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24afe00640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b013df80 0x7f24b01f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24a7e00640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b013df80 0x7f24b01f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24a7e00640 1 -- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b015f9c0 msgr2=0x7f24b01e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24a7e00640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b015f9c0 0x7f24b01e8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24a7e00640 1 -- 172.21.15.26:0/525631565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498009400 con 0x7f24b013e360 2024-01-15T18:12:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24b5000640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b015f9c0 0x7f24b01e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.632+0000 7f24afe00640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b013df80 0x7f24b01f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.633+0000 7f24a7e00640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 0x7f24b01e60c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f2498009890 tx=0x7f24980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.633+0000 7f24acc00640 1 -- 172.21.15.26:0/525631565 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249803a6f0 con 0x7f24b013e360 2024-01-15T18:12:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.633+0000 7f24acc00640 1 -- 172.21.15.26:0/525631565 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249803a870 con 0x7f24b013e360 2024-01-15T18:12:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.633+0000 7f24acc00640 1 -- 172.21.15.26:0/525631565 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249803ac20 con 0x7f24b013e360 2024-01-15T18:12:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.633+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b01e72d0 con 0x7f24b013e360 2024-01-15T18:12:23.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.633+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b01e6600 con 0x7f24b013e360 2024-01-15T18:12:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.634+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b00675d0 con 0x7f24b013e360 2024-01-15T18:12:23.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.635+0000 7f24acc00640 1 -- 172.21.15.26:0/525631565 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f2498043410 con 0x7f24b013e360 2024-01-15T18:12:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.637+0000 7f24acc00640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f248806c430 0x7f248806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:23.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.637+0000 7f24acc00640 1 -- 172.21.15.26:0/525631565 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f24980aa740 con 0x7f24b013e360 2024-01-15T18:12:23.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.638+0000 7f24afe00640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f248806c430 0x7f248806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:23.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.639+0000 7f24afe00640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f248806c430 0x7f248806e8f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f24a0006fd0 tx=0x7f24a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:23.645 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.641+0000 7f24acc00640 1 -- 172.21.15.26:0/525631565 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2498076dd0 con 0x7f24b013e360 2024-01-15T18:12:23.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.859+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f24b01e6e80 con 0x7f248806c430 2024-01-15T18:12:23.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.863+0000 7f24acc00640 1 -- 172.21.15.26:0/525631565 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f24b01e6e80 con 0x7f248806c430 2024-01-15T18:12:23.867 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:12:23.867 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:12:23.867 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:12:23.867 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:12:23.867 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:12:23.869 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f248806c430 msgr2=0x7f248806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f248806c430 0x7f248806e8f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f24a0006fd0 tx=0x7f24a0008040 comp rx=0 tx=0).stop 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 msgr2=0x7f24b01e60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 0x7f24b01e60c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f2498009890 tx=0x7f24980046c0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 shutdown_connections 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f248806c430 0x7f248806e8f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f24b015f9c0 0x7f24b01e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f24b013e360 0x7f24b01e60c0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 --2- 172.21.15.26:0/525631565 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f24b013df80 0x7f24b01f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 >> 172.21.15.26:0/525631565 conn(0x7f24b006a800 msgr2=0x7f24b0159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:23.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 shutdown_connections 2024-01-15T18:12:23.871 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:23.865+0000 7f24b7000640 1 -- 172.21.15.26:0/525631565 wait complete. 2024-01-15T18:12:23.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-01-15T18:12:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.110+0000 7f2f39800640 1 Processor -- start 2024-01-15T18:12:24.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.110+0000 7f2f39800640 1 -- start start 2024-01-15T18:12:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.112+0000 7f2f39800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f3415e170 0x7f2f340cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.112+0000 7f2f39800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2f340cdef0 0x7f2f34070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.112+0000 7f2f39800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f340700f0 con 0x7f2f3413cda0 2024-01-15T18:12:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.112+0000 7f2f39800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f340ce720 con 0x7f2f3415e170 2024-01-15T18:12:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.112+0000 7f2f39800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f340cea00 con 0x7f2f340cdef0 2024-01-15T18:12:24.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.113+0000 7f2f2be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f3415e170 0x7f2f340cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.113+0000 7f2f2be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2f3415e170 0x7f2f340cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57926/0 (socket says 172.21.15.26:57926) 2024-01-15T18:12:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.113+0000 7f2f2be00640 1 -- 172.21.15.26:0/247476373 learned_addr learned my addr 172.21.15.26:0/247476373 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.113+0000 7f2f38e00640 1 --2- 172.21.15.26:0/247476373 >> v2:172.21.15.124:3300/0 conn(0x7f2f340cdef0 0x7f2f34070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.113+0000 7f2f2be00640 1 -- 172.21.15.26:0/247476373 >> v2:172.21.15.124:3300/0 conn(0x7f2f340cdef0 msgr2=0x7f2f34070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.113+0000 7f2f2be00640 1 --2- 172.21.15.26:0/247476373 >> v2:172.21.15.124:3300/0 conn(0x7f2f340cdef0 0x7f2f34070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.113+0000 7f2f2be00640 1 -- 172.21.15.26:0/247476373 >> v1:172.21.15.124:6789/0 conn(0x7f2f3413cda0 legacy=0x7f2f3415da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.113+0000 7f2f2be00640 1 -- 172.21.15.26:0/247476373 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f34070340 con 0x7f2f3415e170 2024-01-15T18:12:24.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.113+0000 7f2f38e00640 1 --2- 172.21.15.26:0/247476373 >> v2:172.21.15.124:3300/0 conn(0x7f2f340cdef0 0x7f2f34070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.114+0000 7f2f2be00640 1 --2- 172.21.15.26:0/247476373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 0x7f2f340cd9b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f2f340aea90 tx=0x7f2f1c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede593d8bf084c3d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.115+0000 7f2f33400640 1 -- 172.21.15.26:0/247476373 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c031800 con 0x7f2f3415e170 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.115+0000 7f2f33400640 1 -- 172.21.15.26:0/247476373 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c031960 con 0x7f2f3415e170 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.115+0000 7f2f33400640 1 -- 172.21.15.26:0/247476373 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c031cf0 con 0x7f2f3415e170 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.115+0000 7f2f39800640 1 -- 172.21.15.26:0/247476373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 msgr2=0x7f2f340cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.115+0000 7f2f39800640 1 --2- 172.21.15.26:0/247476373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 0x7f2f340cd9b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f2f340aea90 tx=0x7f2f1c02b8a0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.115+0000 7f2f39800640 1 -- 172.21.15.26:0/247476373 shutdown_connections 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.115+0000 7f2f39800640 1 --2- 172.21.15.26:0/247476373 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 0x7f2f340cd9b0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.115+0000 7f2f39800640 1 --2- 172.21.15.26:0/247476373 >> v2:172.21.15.124:3300/0 conn(0x7f2f340cdef0 0x7f2f34070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.115+0000 7f2f39800640 1 -- 172.21.15.26:0/247476373 >> 172.21.15.26:0/247476373 conn(0x7f2f3406a3e0 msgr2=0x7f2f3406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.116+0000 7f2f39800640 1 -- 172.21.15.26:0/247476373 shutdown_connections 2024-01-15T18:12:24.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.116+0000 7f2f39800640 1 -- 172.21.15.26:0/247476373 wait complete. 2024-01-15T18:12:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.117+0000 7f2f39800640 1 Processor -- start 2024-01-15T18:12:24.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.117+0000 7f2f39800640 1 -- start start 2024-01-15T18:12:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.118+0000 7f2f39800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f340cdef0 0x7f2f340b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.118+0000 7f2f39800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f3413cda0 0x7f2f3413b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.118+0000 7f2f39800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 0x7f2f3413b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.118+0000 7f2f39800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f341528b0 con 0x7f2f3415e170 2024-01-15T18:12:24.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.118+0000 7f2f39800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3413faa0 con 0x7f2f340cdef0 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.118+0000 7f2f39800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f341541d0 con 0x7f2f3413cda0 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f38e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 0x7f2f3413b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f38e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 0x7f2f3413b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57930/0 (socket says 172.21.15.26:57930) 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f38e00640 1 -- 172.21.15.26:0/2945037865 learned_addr learned my addr 172.21.15.26:0/2945037865 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f2be00640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f3413cda0 0x7f2f3413b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f38e00640 1 -- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f3413cda0 msgr2=0x7f2f3413b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f38e00640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f3413cda0 0x7f2f3413b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f38e00640 1 -- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f340cdef0 msgr2=0x7f2f340b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f38e00640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f340cdef0 0x7f2f340b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f38e00640 1 -- 172.21.15.26:0/2945037865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0094d0 con 0x7f2f3415e170 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.119+0000 7f2f38e00640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 0x7f2f3413b780 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c007df0 tx=0x7f2f2c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.120+0000 7f2f30c00640 1 -- 172.21.15.26:0/2945037865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c003990 con 0x7f2f3415e170 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.120+0000 7f2f30c00640 1 -- 172.21.15.26:0/2945037865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c00f040 con 0x7f2f3415e170 2024-01-15T18:12:24.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.120+0000 7f2f30c00640 1 -- 172.21.15.26:0/2945037865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c003d30 con 0x7f2f3415e170 2024-01-15T18:12:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.120+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3413f4e0 con 0x7f2f3415e170 2024-01-15T18:12:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.120+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f340b28d0 con 0x7f2f3415e170 2024-01-15T18:12:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.121+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f340b2730 con 0x7f2f3415e170 2024-01-15T18:12:24.126 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.122+0000 7f2f30c00640 1 -- 172.21.15.26:0/2945037865 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f2f2c004030 con 0x7f2f3415e170 2024-01-15T18:12:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.123+0000 7f2f30c00640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f0c06c430 0x7f2f0c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.123+0000 7f2f30c00640 1 -- 172.21.15.26:0/2945037865 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f2f2c088de0 con 0x7f2f3415e170 2024-01-15T18:12:24.127 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.124+0000 7f2f33e00640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f0c06c430 0x7f2f0c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.125+0000 7f2f33e00640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f0c06c430 0x7f2f0c06e8f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2f24006fd0 tx=0x7f2f24006bc0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.127+0000 7f2f30c00640 1 -- 172.21.15.26:0/2945037865 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f340b2730 con 0x7f2f3415e170 2024-01-15T18:12:24.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.324+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2f340b2730 con 0x7f2f3415e170 2024-01-15T18:12:24.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.325+0000 7f2f30c00640 1 -- 172.21.15.26:0/2945037865 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f340b2730 con 0x7f2f3415e170 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f0c06c430 msgr2=0x7f2f0c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f0c06c430 0x7f2f0c06e8f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2f24006fd0 tx=0x7f2f24006bc0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 msgr2=0x7f2f3413b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 0x7f2f3413b780 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c007df0 tx=0x7f2f2c00d4a0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 shutdown_connections 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2f3413cda0 0x7f2f3413b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f0c06c430 0x7f2f0c06e8f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2f3415e170 0x7f2f3413b780 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 --2- 172.21.15.26:0/2945037865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2f340cdef0 0x7f2f340b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 >> 172.21.15.26:0/2945037865 conn(0x7f2f3406a3e0 msgr2=0x7f2f3406cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:24.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 shutdown_connections 2024-01-15T18:12:24.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.328+0000 7f2f39800640 1 -- 172.21.15.26:0/2945037865 wait complete. 2024-01-15T18:12:24.353 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/snapshot_clone_delay 10 2024-01-15T18:12:24.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.578+0000 7f7012e00640 1 Processor -- start 2024-01-15T18:12:24.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.579+0000 7f7012e00640 1 -- start start 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.580+0000 7f7012e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 0x7f700c150760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.580+0000 7f7012e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f700c0cf290 0x7f700c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.580+0000 7f7012e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f700c0cf6d0 0x7f700c15d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.580+0000 7f7012e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c072610 con 0x7f700c0cf290 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.580+0000 7f7012e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c071190 con 0x7f700c0cf6d0 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.580+0000 7f7012e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c071470 con 0x7f700c072230 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f7003e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f700c0cf290 0x7f700c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f7003e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f700c0cf290 0x7f700c15d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57946/0 (socket says 172.21.15.26:57946) 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f7003e00640 1 -- 172.21.15.26:0/3532093563 learned_addr learned my addr 172.21.15.26:0/3532093563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:24.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f7010e00640 1 --2- 172.21.15.26:0/3532093563 >> v2:172.21.15.124:3300/0 conn(0x7f700c0cf6d0 0x7f700c15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f700be00640 1 --2- 172.21.15.26:0/3532093563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 0x7f700c150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f7003e00640 1 -- 172.21.15.26:0/3532093563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 msgr2=0x7f700c150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f7003e00640 1 --2- 172.21.15.26:0/3532093563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 0x7f700c150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f7003e00640 1 -- 172.21.15.26:0/3532093563 >> v2:172.21.15.124:3300/0 conn(0x7f700c0cf6d0 msgr2=0x7f700c15d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f7003e00640 1 --2- 172.21.15.26:0/3532093563 >> v2:172.21.15.124:3300/0 conn(0x7f700c0cf6d0 0x7f700c15d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.581+0000 7f7003e00640 1 -- 172.21.15.26:0/3532093563 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c071750 con 0x7f700c0cf290 2024-01-15T18:12:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.583+0000 7f7003e00640 1 --2- 172.21.15.26:0/3532093563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf290 0x7f700c15d300 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f700c0aeb00 tx=0x7f6ff402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dc55c687992c2e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.583+0000 7f700b400640 1 -- 172.21.15.26:0/3532093563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4031870 con 0x7f700c0cf290 2024-01-15T18:12:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.583+0000 7f700b400640 1 -- 172.21.15.26:0/3532093563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff40319d0 con 0x7f700c0cf290 2024-01-15T18:12:24.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.583+0000 7f700b400640 1 -- 172.21.15.26:0/3532093563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4031d60 con 0x7f700c0cf290 2024-01-15T18:12:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.584+0000 7f7012e00640 1 -- 172.21.15.26:0/3532093563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf290 msgr2=0x7f700c15d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.584+0000 7f7012e00640 1 --2- 172.21.15.26:0/3532093563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf290 0x7f700c15d300 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f700c0aeb00 tx=0x7f6ff402b910 comp rx=0 tx=0).stop 2024-01-15T18:12:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.584+0000 7f7012e00640 1 -- 172.21.15.26:0/3532093563 shutdown_connections 2024-01-15T18:12:24.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.584+0000 7f7012e00640 1 --2- 172.21.15.26:0/3532093563 >> v2:172.21.15.124:3300/0 conn(0x7f700c0cf6d0 0x7f700c15d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.584+0000 7f7012e00640 1 --2- 172.21.15.26:0/3532093563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf290 0x7f700c15d300 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.584+0000 7f7012e00640 1 --2- 172.21.15.26:0/3532093563 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 0x7f700c150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.584+0000 7f7012e00640 1 -- 172.21.15.26:0/3532093563 >> 172.21.15.26:0/3532093563 conn(0x7f700c06a800 msgr2=0x7f700c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:24.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.584+0000 7f7012e00640 1 -- 172.21.15.26:0/3532093563 shutdown_connections 2024-01-15T18:12:24.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.584+0000 7f7012e00640 1 -- 172.21.15.26:0/3532093563 wait complete. 2024-01-15T18:12:24.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.586+0000 7f7012e00640 1 Processor -- start 2024-01-15T18:12:24.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.586+0000 7f7012e00640 1 -- start start 2024-01-15T18:12:24.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7012e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 0x7f700c0ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7012e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f700c0cf290 0x7f700c06cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7012e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf6d0 0x7f700c06d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7012e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c1457c0 con 0x7f700c0cf6d0 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7012e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c152ef0 con 0x7f700c0cf290 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7012e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c1525c0 con 0x7f700c072230 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7010e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf6d0 0x7f700c06d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7010e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf6d0 0x7f700c06d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57962/0 (socket says 172.21.15.26:57962) 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7010e00640 1 -- 172.21.15.26:0/916550158 learned_addr learned my addr 172.21.15.26:0/916550158 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f700be00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 0x7f700c0ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.587+0000 7f7003e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f700c0cf290 0x7f700c06cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7010e00640 1 -- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 msgr2=0x7f700c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7010e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 0x7f700c0ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7010e00640 1 -- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f700c0cf290 msgr2=0x7f700c06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7010e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f700c0cf290 0x7f700c06cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7010e00640 1 -- 172.21.15.26:0/916550158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40094d0 con 0x7f700c0cf6d0 2024-01-15T18:12:24.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7010e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf6d0 0x7f700c06d480 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f7004007d80 tx=0x7f700400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7008c00640 1 -- 172.21.15.26:0/916550158 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004014420 con 0x7f700c0cf6d0 2024-01-15T18:12:24.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7008c00640 1 -- 172.21.15.26:0/916550158 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7004014580 con 0x7f700c0cf6d0 2024-01-15T18:12:24.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7008c00640 1 -- 172.21.15.26:0/916550158 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004014930 con 0x7f700c0cf6d0 2024-01-15T18:12:24.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c1528a0 con 0x7f700c0cf6d0 2024-01-15T18:12:24.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.588+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c1357b0 con 0x7f700c0cf6d0 2024-01-15T18:12:24.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.589+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c153080 con 0x7f700c0cf6d0 2024-01-15T18:12:24.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.591+0000 7f7008c00640 1 -- 172.21.15.26:0/916550158 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f7004014a90 con 0x7f700c0cf6d0 2024-01-15T18:12:24.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.591+0000 7f7008c00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fe406c430 0x7f6fe406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:24.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.592+0000 7f7008c00640 1 -- 172.21.15.26:0/916550158 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f70040893e0 con 0x7f700c0cf6d0 2024-01-15T18:12:24.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.592+0000 7f700be00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fe406c430 0x7f6fe406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:24.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.593+0000 7f700be00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fe406c430 0x7f6fe406e8f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc006fd0 tx=0x7f6ffc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:24.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.596+0000 7f7008c00640 1 -- 172.21.15.26:0/916550158 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7004055950 con 0x7f700c0cf6d0 2024-01-15T18:12:24.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.791+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}] v 0) v1 -- 0x7f700c0675d0 con 0x7f700c0cf6d0 2024-01-15T18:12:24.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.797+0000 7f7008c00640 1 -- 172.21.15.26:0/916550158 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/snapshot_clone_delay}]=0 v41)=0 v41) v1 ==== 131+0+0 (secure 0 0 0) 0x7f7004058f70 con 0x7f700c0cf6d0 2024-01-15T18:12:24.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fe406c430 msgr2=0x7f6fe406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fe406c430 0x7f6fe406e8f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc006fd0 tx=0x7f6ffc008040 comp rx=0 tx=0).stop 2024-01-15T18:12:24.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf6d0 msgr2=0x7f700c06d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:24.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf6d0 0x7f700c06d480 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f7004007d80 tx=0x7f700400e4b0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 shutdown_connections 2024-01-15T18:12:24.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fe406c430 0x7f6fe406e8f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f700c0cf6d0 0x7f700c06d480 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f700c0cf290 0x7f700c06cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 --2- 172.21.15.26:0/916550158 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f700c072230 0x7f700c0ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:24.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 >> 172.21.15.26:0/916550158 conn(0x7f700c06a800 msgr2=0x7f700c15b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:24.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 shutdown_connections 2024-01-15T18:12:24.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:24.800+0000 7f7012e00640 1 -- 172.21.15.26:0/916550158 wait complete. 2024-01-15T18:12:24.822 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/volumes/clone_delay_every_regfile 120 2024-01-15T18:12:25.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.042+0000 7f43aae00640 1 Processor -- start 2024-01-15T18:12:25.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.042+0000 7f43aae00640 1 -- start start 2024-01-15T18:12:25.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.043+0000 7f43aae00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a415b160 0x7f43a40ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.043+0000 7f43aae00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f43a40cdfa0 0x7f43a40cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.044+0000 7f43aae00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a40ce4b0 con 0x7f43a40ccaf0 2024-01-15T18:12:25.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.044+0000 7f43aae00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a415b540 con 0x7f43a40cdfa0 2024-01-15T18:12:25.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.044+0000 7f43aae00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40ce750 con 0x7f43a415b160 2024-01-15T18:12:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.044+0000 7f43a9a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a415b160 0x7f43a40ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.044+0000 7f43a9a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a415b160 0x7f43a40ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:48768/0 (socket says 172.21.15.26:48768) 2024-01-15T18:12:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.044+0000 7f43a9a00640 1 -- 172.21.15.26:0/2429196487 learned_addr learned my addr 172.21.15.26:0/2429196487 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.044+0000 7f43a9000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f43a40cdfa0 0x7f43a40cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.045+0000 7f43a9a00640 1 -- 172.21.15.26:0/2429196487 >> v2:172.21.15.124:3300/0 conn(0x7f43a40cdfa0 msgr2=0x7f43a40cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.045+0000 7f43a9a00640 1 --2- 172.21.15.26:0/2429196487 >> v2:172.21.15.124:3300/0 conn(0x7f43a40cdfa0 0x7f43a40cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.045+0000 7f43a9a00640 1 -- 172.21.15.26:0/2429196487 >> v1:172.21.15.124:6789/0 conn(0x7f43a40ccaf0 legacy=0x7f43a40b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:25.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.045+0000 7f43a9a00640 1 -- 172.21.15.26:0/2429196487 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a40cbdb0 con 0x7f43a415b160 2024-01-15T18:12:25.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.047+0000 7f43a9a00640 1 --2- 172.21.15.26:0/2429196487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a415b160 0x7f43a40ceb90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4394009870 tx=0x7f439402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adceea461fb449b5 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.047+0000 7f43a3e00640 1 -- 172.21.15.26:0/2429196487 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394031640 con 0x7f43a415b160 2024-01-15T18:12:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.048+0000 7f43a3e00640 1 -- 172.21.15.26:0/2429196487 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43940317a0 con 0x7f43a415b160 2024-01-15T18:12:25.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.048+0000 7f43a3e00640 1 -- 172.21.15.26:0/2429196487 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4394031b30 con 0x7f43a415b160 2024-01-15T18:12:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.048+0000 7f43aae00640 1 -- 172.21.15.26:0/2429196487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a415b160 msgr2=0x7f43a40ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.048+0000 7f43aae00640 1 --2- 172.21.15.26:0/2429196487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a415b160 0x7f43a40ceb90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4394009870 tx=0x7f439402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.048+0000 7f43aae00640 1 -- 172.21.15.26:0/2429196487 shutdown_connections 2024-01-15T18:12:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.048+0000 7f43aae00640 1 --2- 172.21.15.26:0/2429196487 >> v2:172.21.15.124:3300/0 conn(0x7f43a40cdfa0 0x7f43a40cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.048+0000 7f43aae00640 1 --2- 172.21.15.26:0/2429196487 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a415b160 0x7f43a40ceb90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.048+0000 7f43aae00640 1 -- 172.21.15.26:0/2429196487 >> 172.21.15.26:0/2429196487 conn(0x7f43a40b2a30 msgr2=0x7f43a40b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.049+0000 7f43aae00640 1 -- 172.21.15.26:0/2429196487 shutdown_connections 2024-01-15T18:12:25.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.049+0000 7f43aae00640 1 -- 172.21.15.26:0/2429196487 wait complete. 2024-01-15T18:12:25.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.050+0000 7f43aae00640 1 Processor -- start 2024-01-15T18:12:25.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.051+0000 7f43aae00640 1 -- start start 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.051+0000 7f43aae00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43a40ccaf0 0x7f43a40b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.051+0000 7f43aae00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a40cdfa0 0x7f43a4157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.051+0000 7f43aae00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a415b160 0x7f43a40c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.051+0000 7f43aae00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a406b970 con 0x7f43a415b160 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.051+0000 7f43aae00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40b62e0 con 0x7f43a40ccaf0 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.051+0000 7f43aae00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a406bbc0 con 0x7f43a40cdfa0 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43aa400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a415b160 0x7f43a40c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43aa400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a415b160 0x7f43a40c3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57972/0 (socket says 172.21.15.26:57972) 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43aa400640 1 -- 172.21.15.26:0/4225951209 learned_addr learned my addr 172.21.15.26:0/4225951209 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:25.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43a9000640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a40cdfa0 0x7f43a4157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43a9a00640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43a40ccaf0 0x7f43a40b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43aa400640 1 -- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a40cdfa0 msgr2=0x7f43a4157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43aa400640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a40cdfa0 0x7f43a4157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43aa400640 1 -- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43a40ccaf0 msgr2=0x7f43a40b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43aa400640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43a40ccaf0 0x7f43a40b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.052+0000 7f43aa400640 1 -- 172.21.15.26:0/4225951209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43940094d0 con 0x7f43a415b160 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.053+0000 7f43aa400640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a415b160 0x7f43a40c3720 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f439c00b970 tx=0x7f439c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.053+0000 7f43a1600640 1 -- 172.21.15.26:0/4225951209 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c00cb40 con 0x7f43a415b160 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.053+0000 7f43a1600640 1 -- 172.21.15.26:0/4225951209 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c00cca0 con 0x7f43a415b160 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.053+0000 7f43a1600640 1 -- 172.21.15.26:0/4225951209 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c014680 con 0x7f43a415b160 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.053+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a406bea0 con 0x7f43a415b160 2024-01-15T18:12:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.053+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a40b5f90 con 0x7f43a415b160 2024-01-15T18:12:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.055+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a406c3c0 con 0x7f43a415b160 2024-01-15T18:12:25.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.056+0000 7f43a1600640 1 -- 172.21.15.26:0/4225951209 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f439c01d3b0 con 0x7f43a415b160 2024-01-15T18:12:25.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.057+0000 7f43a1600640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437c06c430 0x7f437c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.057+0000 7f43a1600640 1 -- 172.21.15.26:0/4225951209 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f439c0888c0 con 0x7f43a415b160 2024-01-15T18:12:25.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.057+0000 7f43a9a00640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437c06c430 0x7f437c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.058+0000 7f43a9a00640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437c06c430 0x7f437c06e8f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4394036000 tx=0x7f439403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:25.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.061+0000 7f43a1600640 1 -- 172.21.15.26:0/4225951209 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43a406c3c0 con 0x7f43a415b160 2024-01-15T18:12:25.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.256+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}] v 0) v1 -- 0x7f43a406c3c0 con 0x7f43a415b160 2024-01-15T18:12:25.267 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.263+0000 7f43a1600640 1 -- 172.21.15.26:0/4225951209 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/volumes/clone_delay_every_regfile}]=0 v42)=0 v42) v1 ==== 137+0+0 (secure 0 0 0) 0x7f43a406c3c0 con 0x7f43a415b160 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437c06c430 msgr2=0x7f437c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437c06c430 0x7f437c06e8f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4394036000 tx=0x7f439403c040 comp rx=0 tx=0).stop 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a415b160 msgr2=0x7f43a40c3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a415b160 0x7f43a40c3720 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f439c00b970 tx=0x7f439c00be30 comp rx=0 tx=0).stop 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 shutdown_connections 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f43a40cdfa0 0x7f43a4157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f437c06c430 0x7f437c06e8f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f43a415b160 0x7f43a40c3720 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 --2- 172.21.15.26:0/4225951209 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f43a40ccaf0 0x7f43a40b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.266+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 >> 172.21.15.26:0/4225951209 conn(0x7f43a40b2a30 msgr2=0x7f43a406e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.267+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 shutdown_connections 2024-01-15T18:12:25.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.267+0000 7f43aae00640 1 -- 172.21.15.26:0/4225951209 wait complete. 2024-01-15T18:12:25.292 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv1 --mode=777 2024-01-15T18:12:25.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.517+0000 7f68d8c00640 1 Processor -- start 2024-01-15T18:12:25.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.518+0000 7f68d8c00640 1 -- start start 2024-01-15T18:12:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.519+0000 7f68d8c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f68d40d0100 0x7f68d40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.519+0000 7f68d8c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d40d0a10 con 0x7f68d40d0f40 2024-01-15T18:12:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.519+0000 7f68d8c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d40cf080 con 0x7f68d4160920 2024-01-15T18:12:25.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.519+0000 7f68d8c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68d40c6be0 con 0x7f68d40d0100 2024-01-15T18:12:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.520+0000 7f68cbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f68d40d0100 0x7f68d40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.520+0000 7f68d3e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f68d40d0f40 0x7f68d40d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45224/0 (socket says 172.21.15.26:45224) 2024-01-15T18:12:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.520+0000 7f68d3e00640 1 -- 172.21.15.26:0/320618878 learned_addr learned my addr 172.21.15.26:0/320618878 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:12:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.520+0000 7f68cbe00640 1 -- 172.21.15.26:0/320618878 >> v1:172.21.15.124:6789/0 conn(0x7f68d4160920 legacy=0x7f68d40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:25.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.520+0000 7f68cbe00640 1 -- 172.21.15.26:0/320618878 >> v1:172.21.15.26:6789/0 conn(0x7f68d40d0f40 legacy=0x7f68d40d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.520+0000 7f68cbe00640 1 -- 172.21.15.26:0/320618878 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d40ce7e0 con 0x7f68d40d0100 2024-01-15T18:12:25.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.521+0000 7f68cbe00640 1 --2- 172.21.15.26:0/320618878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d40d0100 0x7f68d40b3f30 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0097a0 tx=0x7f68bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7583e116744f54ca server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.522+0000 7f68d2a00640 1 -- 172.21.15.26:0/320618878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc031830 con 0x7f68d40d0100 2024-01-15T18:12:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.522+0000 7f68d2a00640 1 -- 172.21.15.26:0/320618878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc031990 con 0x7f68d40d0100 2024-01-15T18:12:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.522+0000 7f68d2a00640 1 -- 172.21.15.26:0/320618878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc031d20 con 0x7f68d40d0100 2024-01-15T18:12:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.522+0000 7f68d8c00640 1 -- 172.21.15.26:0/320618878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d40d0100 msgr2=0x7f68d40b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.522+0000 7f68d8c00640 1 --2- 172.21.15.26:0/320618878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d40d0100 0x7f68d40b3f30 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0097a0 tx=0x7f68bc02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.522+0000 7f68d8c00640 1 -- 172.21.15.26:0/320618878 shutdown_connections 2024-01-15T18:12:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.522+0000 7f68d8c00640 1 --2- 172.21.15.26:0/320618878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d40d0100 0x7f68d40b3f30 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.522+0000 7f68d8c00640 1 -- 172.21.15.26:0/320618878 >> 172.21.15.26:0/320618878 conn(0x7f68d40d3090 msgr2=0x7f68d40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.522+0000 7f68d8c00640 1 -- 172.21.15.26:0/320618878 shutdown_connections 2024-01-15T18:12:25.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.523+0000 7f68d8c00640 1 -- 172.21.15.26:0/320618878 wait complete. 2024-01-15T18:12:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.524+0000 7f68d8c00640 1 Processor -- start 2024-01-15T18:12:25.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.525+0000 7f68d8c00640 1 -- start start 2024-01-15T18:12:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.525+0000 7f68d8c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68d40d0100 0x7f68d4156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d8c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68d40d0f40 0x7f68d40ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4160920 0x7f68d415af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d8c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40cb740 con 0x7f68d4160920 2024-01-15T18:12:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d8c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40cbd00 con 0x7f68d40d0f40 2024-01-15T18:12:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d8c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d40cac20 con 0x7f68d40d0100 2024-01-15T18:12:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4160920 0x7f68d415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4160920 0x7f68d415af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:57994/0 (socket says 172.21.15.26:57994) 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3e00640 1 -- 172.21.15.26:0/776276864 learned_addr learned my addr 172.21.15.26:0/776276864 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3400640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68d40d0100 0x7f68d4156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3e00640 1 -- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68d40d0100 msgr2=0x7f68d4156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3e00640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68d40d0100 0x7f68d4156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3e00640 1 -- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68d40d0f40 msgr2=0x7f68d40ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3e00640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68d40d0f40 0x7f68d40ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3e00640 1 -- 172.21.15.26:0/776276864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc009400 con 0x7f68d4160920 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.526+0000 7f68d3400640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68d40d0100 0x7f68d4156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.527+0000 7f68d3e00640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4160920 0x7f68d415af60 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00cbf0 tx=0x7f68cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.527+0000 7f68cb400640 1 -- 172.21.15.26:0/776276864 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc00ad40 con 0x7f68d4160920 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.527+0000 7f68cb400640 1 -- 172.21.15.26:0/776276864 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68cc012400 con 0x7f68d4160920 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.527+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d40caf00 con 0x7f68d4160920 2024-01-15T18:12:25.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.527+0000 7f68cb400640 1 -- 172.21.15.26:0/776276864 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68cc00aea0 con 0x7f68d4160920 2024-01-15T18:12:25.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.527+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d40b8ac0 con 0x7f68d4160920 2024-01-15T18:12:25.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.528+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68d40cbe90 con 0x7f68d4160920 2024-01-15T18:12:25.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.529+0000 7f68cb400640 1 -- 172.21.15.26:0/776276864 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f68cc007070 con 0x7f68d4160920 2024-01-15T18:12:25.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.530+0000 7f68cb400640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ac06c430 0x7f68ac06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:25.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.530+0000 7f68cb400640 1 -- 172.21.15.26:0/776276864 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f68cc088930 con 0x7f68d4160920 2024-01-15T18:12:25.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.531+0000 7f68d3400640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ac06c430 0x7f68ac06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:25.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.532+0000 7f68d3400640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ac06c430 0x7f68ac06e8f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f68d415fb30 tx=0x7f68c4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:25.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.535+0000 7f68cb400640 1 -- 172.21.15.26:0/776276864 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68cc054f50 con 0x7f68d4160920 2024-01-15T18:12:25.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.756+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv1", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7f68d4160d00 con 0x7f68ac06c430 2024-01-15T18:12:25.771 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:12:25.768+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:25.771 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:12:25.768+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:25.772 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:12:25.768+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:25.772 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:12:25.768+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:25.772 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:12:25.768+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:12:25.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.852+0000 7f68cb400640 1 -- 172.21.15.26:0/776276864 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f68c400a420 con 0x7f68ac06c430 2024-01-15T18:12:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.855+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ac06c430 msgr2=0x7f68ac06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.855+0000 7f68d8c00640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ac06c430 0x7f68ac06e8f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f68d415fb30 tx=0x7f68c4008040 comp rx=0 tx=0).stop 2024-01-15T18:12:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.855+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4160920 msgr2=0x7f68d415af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.855+0000 7f68d8c00640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4160920 0x7f68d415af60 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00cbf0 tx=0x7f68cc00a430 comp rx=0 tx=0).stop 2024-01-15T18:12:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.856+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 shutdown_connections 2024-01-15T18:12:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.856+0000 7f68d8c00640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f68ac06c430 0x7f68ac06e8f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.856+0000 7f68d8c00640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f68d4160920 0x7f68d415af60 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.856+0000 7f68d8c00640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f68d40d0f40 0x7f68d40ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.856+0000 7f68d8c00640 1 --2- 172.21.15.26:0/776276864 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f68d40d0100 0x7f68d4156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.856+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 >> 172.21.15.26:0/776276864 conn(0x7f68d40d3090 msgr2=0x7f68d406c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.856+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 shutdown_connections 2024-01-15T18:12:25.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:25.856+0000 7f68d8c00640 1 -- 172.21.15.26:0/776276864 wait complete. 2024-01-15T18:12:25.881 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:12:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.096+0000 7fc697e00640 1 Processor -- start 2024-01-15T18:12:26.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.096+0000 7fc697e00640 1 -- start start 2024-01-15T18:12:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.098+0000 7fc697e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc69013df80 0x7fc69015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.098+0000 7fc697e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc69013e360 0x7fc6900cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.098+0000 7fc697e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69015f9c0 0x7fc6900cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.098+0000 7fc697e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69006c470 con 0x7fc69013e360 2024-01-15T18:12:26.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.098+0000 7fc697e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69006c750 con 0x7fc69013df80 2024-01-15T18:12:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.098+0000 7fc697e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69006ca30 con 0x7fc69015f9c0 2024-01-15T18:12:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.098+0000 7fc694a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc69013e360 0x7fc6900cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.098+0000 7fc694a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fc69013e360 0x7fc6900cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58000/0 (socket says 172.21.15.26:58000) 2024-01-15T18:12:26.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.098+0000 7fc694a00640 1 -- 172.21.15.26:0/880696731 learned_addr learned my addr 172.21.15.26:0/880696731 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.099+0000 7fc695400640 1 --2- 172.21.15.26:0/880696731 >> v2:172.21.15.124:3300/0 conn(0x7fc69013df80 0x7fc69015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.099+0000 7fc695e00640 1 --2- 172.21.15.26:0/880696731 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69015f9c0 0x7fc6900cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.099+0000 7fc694a00640 1 -- 172.21.15.26:0/880696731 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69015f9c0 msgr2=0x7fc6900cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.099+0000 7fc694a00640 1 --2- 172.21.15.26:0/880696731 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69015f9c0 0x7fc6900cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.099+0000 7fc694a00640 1 -- 172.21.15.26:0/880696731 >> v2:172.21.15.124:3300/0 conn(0x7fc69013df80 msgr2=0x7fc69015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.099+0000 7fc694a00640 1 --2- 172.21.15.26:0/880696731 >> v2:172.21.15.124:3300/0 conn(0x7fc69013df80 0x7fc69015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.099+0000 7fc694a00640 1 -- 172.21.15.26:0/880696731 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69006cd10 con 0x7fc69013e360 2024-01-15T18:12:26.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.101+0000 7fc694a00640 1 --2- 172.21.15.26:0/880696731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013e360 0x7fc6900cf290 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fc690072770 tx=0x7fc67802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44e889f6ea034cf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.102+0000 7fc68fe00640 1 -- 172.21.15.26:0/880696731 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678031870 con 0x7fc69013e360 2024-01-15T18:12:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.102+0000 7fc68fe00640 1 -- 172.21.15.26:0/880696731 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6780319d0 con 0x7fc69013e360 2024-01-15T18:12:26.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.102+0000 7fc68fe00640 1 -- 172.21.15.26:0/880696731 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678031d60 con 0x7fc69013e360 2024-01-15T18:12:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.102+0000 7fc697e00640 1 -- 172.21.15.26:0/880696731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013e360 msgr2=0x7fc6900cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.102+0000 7fc697e00640 1 --2- 172.21.15.26:0/880696731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013e360 0x7fc6900cf290 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fc690072770 tx=0x7fc67802b910 comp rx=0 tx=0).stop 2024-01-15T18:12:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.103+0000 7fc697e00640 1 -- 172.21.15.26:0/880696731 shutdown_connections 2024-01-15T18:12:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.103+0000 7fc697e00640 1 --2- 172.21.15.26:0/880696731 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69015f9c0 0x7fc6900cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.103+0000 7fc697e00640 1 --2- 172.21.15.26:0/880696731 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013e360 0x7fc6900cf290 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.103+0000 7fc697e00640 1 --2- 172.21.15.26:0/880696731 >> v2:172.21.15.124:3300/0 conn(0x7fc69013df80 0x7fc69015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.103+0000 7fc697e00640 1 -- 172.21.15.26:0/880696731 >> 172.21.15.26:0/880696731 conn(0x7fc69006a800 msgr2=0x7fc69006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.103+0000 7fc697e00640 1 -- 172.21.15.26:0/880696731 shutdown_connections 2024-01-15T18:12:26.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.103+0000 7fc697e00640 1 -- 172.21.15.26:0/880696731 wait complete. 2024-01-15T18:12:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.104+0000 7fc697e00640 1 Processor -- start 2024-01-15T18:12:26.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.105+0000 7fc697e00640 1 -- start start 2024-01-15T18:12:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc697e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013df80 0x7fc69013bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc697e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69013e360 0x7fc690153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc697e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69015f9c0 0x7fc6900b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc697e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69013acf0 con 0x7fc69013df80 2024-01-15T18:12:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc697e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69013afd0 con 0x7fc69015f9c0 2024-01-15T18:12:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc697e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69013b2b0 con 0x7fc69013e360 2024-01-15T18:12:26.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc695400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013df80 0x7fc69013bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc695400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013df80 0x7fc69013bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58002/0 (socket says 172.21.15.26:58002) 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc695400640 1 -- 172.21.15.26:0/2525859340 learned_addr learned my addr 172.21.15.26:0/2525859340 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc694a00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69013e360 0x7fc690153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc695e00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69015f9c0 0x7fc6900b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc695400640 1 -- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69013e360 msgr2=0x7fc690153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc695400640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69013e360 0x7fc690153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc695400640 1 -- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69015f9c0 msgr2=0x7fc6900b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc695400640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69015f9c0 0x7fc6900b31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.106+0000 7fc695400640 1 -- 172.21.15.26:0/2525859340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6780094d0 con 0x7fc69013df80 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.107+0000 7fc695400640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013df80 0x7fc69013bd90 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fc68000d5f0 tx=0x7fc68000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.107+0000 7fc68d600640 1 -- 172.21.15.26:0/2525859340 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6800125f0 con 0x7fc69013df80 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.107+0000 7fc68d600640 1 -- 172.21.15.26:0/2525859340 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc680012750 con 0x7fc69013df80 2024-01-15T18:12:26.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.107+0000 7fc68d600640 1 -- 172.21.15.26:0/2525859340 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680012b00 con 0x7fc69013df80 2024-01-15T18:12:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.107+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69013b590 con 0x7fc69013df80 2024-01-15T18:12:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.107+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc690154a70 con 0x7fc69013df80 2024-01-15T18:12:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.108+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6900675d0 con 0x7fc69013df80 2024-01-15T18:12:26.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.109+0000 7fc68d600640 1 -- 172.21.15.26:0/2525859340 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7fc680012c60 con 0x7fc69013df80 2024-01-15T18:12:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.111+0000 7fc68d600640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc66806c430 0x7fc66806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.111+0000 7fc68d600640 1 -- 172.21.15.26:0/2525859340 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7fc680088fd0 con 0x7fc69013df80 2024-01-15T18:12:26.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.112+0000 7fc694a00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc66806c430 0x7fc66806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.113+0000 7fc694a00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc66806c430 0x7fc66806e8f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc678036000 tx=0x7fc67803c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:26.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.116+0000 7fc68d600640 1 -- 172.21.15.26:0/2525859340 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc680055510 con 0x7fc69013df80 2024-01-15T18:12:26.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.337+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6901408d0 con 0x7fc66806c430 2024-01-15T18:12:26.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.349+0000 7fc68d600640 1 -- 172.21.15.26:0/2525859340 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fc678031bc0 con 0x7fc66806c430 2024-01-15T18:12:26.353 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc66806c430 msgr2=0x7fc66806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc66806c430 0x7fc66806e8f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc678036000 tx=0x7fc67803c040 comp rx=0 tx=0).stop 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013df80 msgr2=0x7fc69013bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013df80 0x7fc69013bd90 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fc68000d5f0 tx=0x7fc68000dab0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 shutdown_connections 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc69013e360 0x7fc690153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc66806c430 0x7fc66806e8f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc69015f9c0 0x7fc6900b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 --2- 172.21.15.26:0/2525859340 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc69013df80 0x7fc69013bd90 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.351+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 >> 172.21.15.26:0/2525859340 conn(0x7fc69006a800 msgr2=0x7fc6900af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.352+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 shutdown_connections 2024-01-15T18:12:26.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.352+0000 7fc697e00640 1 -- 172.21.15.26:0/2525859340 wait complete. 2024-01-15T18:12:26.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:12:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.594+0000 7f3afdc00640 1 Processor -- start 2024-01-15T18:12:26.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.594+0000 7f3afdc00640 1 -- start start 2024-01-15T18:12:26.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.596+0000 7f3afdc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3af80adf20 0x7f3af8070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.596+0000 7f3afdc00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3af815bf10 0x7f3af80cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.596+0000 7f3afdc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80cc980 0x7f3af8143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.596+0000 7f3afdc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af815f970 con 0x7f3af80adf20 2024-01-15T18:12:26.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.596+0000 7f3afdc00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af815f4e0 con 0x7f3af815bf10 2024-01-15T18:12:26.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.596+0000 7f3afdc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af80cce10 con 0x7f3af80cc980 2024-01-15T18:12:26.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.596+0000 7f3af7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3af80adf20 0x7f3af8070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.596+0000 7f3af7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f3af80adf20 0x7f3af8070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58004/0 (socket says 172.21.15.26:58004) 2024-01-15T18:12:26.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.596+0000 7f3af7e00640 1 -- 172.21.15.26:0/231555039 learned_addr learned my addr 172.21.15.26:0/231555039 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:26.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.597+0000 7f3af7400640 1 --2- 172.21.15.26:0/231555039 >> v2:172.21.15.124:3300/0 conn(0x7f3af815bf10 0x7f3af80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.597+0000 7f3afd200640 1 --2- 172.21.15.26:0/231555039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80cc980 0x7f3af8143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.597+0000 7f3af7e00640 1 -- 172.21.15.26:0/231555039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80cc980 msgr2=0x7f3af8143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:26.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.597+0000 7f3af7e00640 1 --2- 172.21.15.26:0/231555039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80cc980 0x7f3af8143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.597+0000 7f3af7e00640 1 -- 172.21.15.26:0/231555039 >> v2:172.21.15.124:3300/0 conn(0x7f3af815bf10 msgr2=0x7f3af80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:12:26.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.597+0000 7f3af7e00640 1 --2- 172.21.15.26:0/231555039 >> v2:172.21.15.124:3300/0 conn(0x7f3af815bf10 0x7f3af80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.597+0000 7f3af7e00640 1 -- 172.21.15.26:0/231555039 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af8071130 con 0x7f3af80adf20 2024-01-15T18:12:26.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.597+0000 7f3af7400640 1 --2- 172.21.15.26:0/231555039 >> v2:172.21.15.124:3300/0 conn(0x7f3af815bf10 0x7f3af80cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:12:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.598+0000 7f3af7e00640 1 --2- 172.21.15.26:0/231555039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af80adf20 0x7f3af8070a30 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f3af815e9e0 tx=0x7f3ae802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df9df00a7586a1a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3af6a00640 1 -- 172.21.15.26:0/231555039 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae80317e0 con 0x7f3af80adf20 2024-01-15T18:12:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3af6a00640 1 -- 172.21.15.26:0/231555039 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae8031940 con 0x7f3af80adf20 2024-01-15T18:12:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3af6a00640 1 -- 172.21.15.26:0/231555039 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8031cd0 con 0x7f3af80adf20 2024-01-15T18:12:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3afdc00640 1 -- 172.21.15.26:0/231555039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af80adf20 msgr2=0x7f3af8070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3afdc00640 1 --2- 172.21.15.26:0/231555039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af80adf20 0x7f3af8070a30 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f3af815e9e0 tx=0x7f3ae802b880 comp rx=0 tx=0).stop 2024-01-15T18:12:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3afdc00640 1 -- 172.21.15.26:0/231555039 shutdown_connections 2024-01-15T18:12:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3afdc00640 1 --2- 172.21.15.26:0/231555039 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80cc980 0x7f3af8143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3afdc00640 1 --2- 172.21.15.26:0/231555039 >> v2:172.21.15.124:3300/0 conn(0x7f3af815bf10 0x7f3af80cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3afdc00640 1 --2- 172.21.15.26:0/231555039 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af80adf20 0x7f3af8070a30 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.599+0000 7f3afdc00640 1 -- 172.21.15.26:0/231555039 >> 172.21.15.26:0/231555039 conn(0x7f3af806a7d0 msgr2=0x7f3af806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.600+0000 7f3afdc00640 1 -- 172.21.15.26:0/231555039 shutdown_connections 2024-01-15T18:12:26.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.600+0000 7f3afdc00640 1 -- 172.21.15.26:0/231555039 wait complete. 2024-01-15T18:12:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.601+0000 7f3afdc00640 1 Processor -- start 2024-01-15T18:12:26.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.602+0000 7f3afdc00640 1 -- start start 2024-01-15T18:12:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.602+0000 7f3afdc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80adf20 0x7f3af815a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.602+0000 7f3afdc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3af80cc980 0x7f3af813b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afdc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af815bf10 0x7f3af80afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afdc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8140030 con 0x7f3af815bf10 2024-01-15T18:12:26.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afdc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af815ad80 con 0x7f3af80cc980 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afdc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af813f340 con 0x7f3af80adf20 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afd200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af815bf10 0x7f3af80afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afd200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af815bf10 0x7f3af80afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58016/0 (socket says 172.21.15.26:58016) 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afd200640 1 -- 172.21.15.26:0/2204626690 learned_addr learned my addr 172.21.15.26:0/2204626690 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3af7400640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3af80cc980 0x7f3af813b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3af7e00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80adf20 0x7f3af815a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afd200640 1 -- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80adf20 msgr2=0x7f3af815a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afd200640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80adf20 0x7f3af815a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afd200640 1 -- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3af80cc980 msgr2=0x7f3af813b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afd200640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3af80cc980 0x7f3af813b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.603+0000 7f3afd200640 1 -- 172.21.15.26:0/2204626690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae80094d0 con 0x7f3af815bf10 2024-01-15T18:12:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.604+0000 7f3afd200640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af815bf10 0x7f3af80afe20 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f3af000eed0 tx=0x7f3af000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.604+0000 7f3aefe00640 1 -- 172.21.15.26:0/2204626690 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0010040 con 0x7f3af815bf10 2024-01-15T18:12:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.604+0000 7f3aefe00640 1 -- 172.21.15.26:0/2204626690 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af0014400 con 0x7f3af815bf10 2024-01-15T18:12:26.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.604+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af813ed80 con 0x7f3af815bf10 2024-01-15T18:12:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.604+0000 7f3aefe00640 1 -- 172.21.15.26:0/2204626690 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0009ea0 con 0x7f3af815bf10 2024-01-15T18:12:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.604+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af813d6d0 con 0x7f3af815bf10 2024-01-15T18:12:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.605+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af813d530 con 0x7f3af815bf10 2024-01-15T18:12:26.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.606+0000 7f3aefe00640 1 -- 172.21.15.26:0/2204626690 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 38) v1 ==== 91120+0+0 (secure 0 0 0) 0x7f3af00147b0 con 0x7f3af815bf10 2024-01-15T18:12:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.607+0000 7f3aefe00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ad006c460 0x7f3ad006e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:12:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.607+0000 7f3aefe00640 1 -- 172.21.15.26:0/2204626690 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5793+0+0 (secure 0 0 0) 0x7f3af00894e0 con 0x7f3af815bf10 2024-01-15T18:12:26.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.608+0000 7f3af7e00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ad006c460 0x7f3ad006e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:12:26.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.609+0000 7f3af7e00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ad006c460 0x7f3ad006e920 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8036000 tx=0x7f3ae803d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:12:26.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.611+0000 7f3aefe00640 1 -- 172.21.15.26:0/2204626690 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3af813d530 con 0x7f3af815bf10 2024-01-15T18:12:26.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.819+0000 7f3aefe00640 1 -- 172.21.15.26:0/2204626690 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 39) v1 ==== 91171+0+0 (secure 0 0 0) 0x7f3af0055e20 con 0x7f3af815bf10 2024-01-15T18:12:26.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.845+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3af815ee90 con 0x7f3ad006c460 2024-01-15T18:12:26.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.856+0000 7f3aefe00640 1 -- 172.21.15.26:0/2204626690 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f3ae8031b30 con 0x7f3ad006c460 2024-01-15T18:12:26.861 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc 2024-01-15T18:12:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ad006c460 msgr2=0x7f3ad006e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ad006c460 0x7f3ad006e920 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8036000 tx=0x7f3ae803d040 comp rx=0 tx=0).stop 2024-01-15T18:12:26.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af815bf10 msgr2=0x7f3af80afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:12:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af815bf10 0x7f3af80afe20 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f3af000eed0 tx=0x7f3af000b480 comp rx=0 tx=0).stop 2024-01-15T18:12:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 shutdown_connections 2024-01-15T18:12:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3ad006c460 0x7f3ad006e920 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3af815bf10 0x7f3af80afe20 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3af80cc980 0x7f3af813b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 --2- 172.21.15.26:0/2204626690 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3af80adf20 0x7f3af815a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:12:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 >> 172.21.15.26:0/2204626690 conn(0x7f3af806a7d0 msgr2=0x7f3af806db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:12:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 shutdown_connections 2024-01-15T18:12:26.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:12:26.859+0000 7f3afdc00640 1 -- 172.21.15.26:0/2204626690 wait complete. 2024-01-15T18:12:26.883 DEBUG:tasks.cephfs.test_volumes:filling subvolume sv1 with 3 files each 1024MB size under directory volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc 2024-01-15T18:12:26.883 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:12:26.883 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:12:26.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:26.936 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc/subvolume_file.0 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:12:31.568 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:12:31.577 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:12:43.459 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:12:43.460 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:12:43.460 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 16.4443 s, 65.3 MB/s 2024-01-15T18:12:43.463 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:12:43.463 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:12:43.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:43.521 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc/subvolume_file.1 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:12:57.816 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:12:57.817 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:12:57.817 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.2341 s, 75.4 MB/s 2024-01-15T18:12:57.818 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:12:57.819 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:12:57.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:12:57.873 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc/subvolume_file.2 bs=1M conv=fdatasync count=1024 seek=0) 2024-01-15T18:13:01.726 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:13:01.736 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:13:12.101 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records in 2024-01-15T18:13:12.101 INFO:teuthology.orchestra.run.smithi026.stderr:1024+0 records out 2024-01-15T18:13:12.101 INFO:teuthology.orchestra.run.smithi026.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 14.1693 s, 75.8 MB/s 2024-01-15T18:13:12.103 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs sv1 ss1 2024-01-15T18:13:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.321+0000 7fb39c600640 1 Processor -- start 2024-01-15T18:13:12.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.321+0000 7fb39c600640 1 -- start start 2024-01-15T18:13:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.323+0000 7fb39c600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb39413e360 0x7fb3940cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.323+0000 7fb39c600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39415f9c0 0x7fb3940cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.323+0000 7fb39c600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39415c5f0 con 0x7fb39413df80 2024-01-15T18:13:12.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.323+0000 7fb39c600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39406c470 con 0x7fb39413e360 2024-01-15T18:13:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.323+0000 7fb39c600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39406c750 con 0x7fb39415f9c0 2024-01-15T18:13:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.323+0000 7fb399c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb39413df80 0x7fb39415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60862/0 (socket says 172.21.15.26:60862) 2024-01-15T18:13:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.323+0000 7fb399200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb39413e360 0x7fb3940cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.323+0000 7fb399c00640 1 -- 172.21.15.26:0/3277102605 learned_addr learned my addr 172.21.15.26:0/3277102605 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:13:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.324+0000 7fb39a600640 1 --2- 172.21.15.26:0/3277102605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39415f9c0 0x7fb3940cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.324+0000 7fb393e00640 1 -- 172.21.15.26:0/3277102605 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934195489 0 0) 0x7fb39415c5f0 con 0x7fb39413df80 2024-01-15T18:13:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.324+0000 7fb399200640 1 -- 172.21.15.26:0/3277102605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39415f9c0 msgr2=0x7fb3940cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.324+0000 7fb399200640 1 --2- 172.21.15.26:0/3277102605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39415f9c0 0x7fb3940cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.324+0000 7fb399200640 1 -- 172.21.15.26:0/3277102605 >> v1:172.21.15.26:6789/0 conn(0x7fb39413df80 legacy=0x7fb39415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.324+0000 7fb399200640 1 -- 172.21.15.26:0/3277102605 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39406ca30 con 0x7fb39413e360 2024-01-15T18:13:12.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.325+0000 7fb399200640 1 --2- 172.21.15.26:0/3277102605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 0x7fb3940cf290 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0097a0 tx=0x7fb37c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f5322adb8f4a4a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:12.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.326+0000 7fb393e00640 1 -- 172.21.15.26:0/3277102605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c031830 con 0x7fb39413e360 2024-01-15T18:13:12.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.326+0000 7fb393e00640 1 -- 172.21.15.26:0/3277102605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c031990 con 0x7fb39413e360 2024-01-15T18:13:12.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.326+0000 7fb393e00640 1 -- 172.21.15.26:0/3277102605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c031d20 con 0x7fb39413e360 2024-01-15T18:13:12.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.326+0000 7fb39c600640 1 -- 172.21.15.26:0/3277102605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 msgr2=0x7fb3940cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.326+0000 7fb39c600640 1 --2- 172.21.15.26:0/3277102605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 0x7fb3940cf290 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fb37c0097a0 tx=0x7fb37c02b8d0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.326+0000 7fb39c600640 1 -- 172.21.15.26:0/3277102605 shutdown_connections 2024-01-15T18:13:12.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.326+0000 7fb39c600640 1 --2- 172.21.15.26:0/3277102605 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39415f9c0 0x7fb3940cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.326+0000 7fb39c600640 1 --2- 172.21.15.26:0/3277102605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 0x7fb3940cf290 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.326+0000 7fb39c600640 1 -- 172.21.15.26:0/3277102605 >> 172.21.15.26:0/3277102605 conn(0x7fb39406a800 msgr2=0x7fb39406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:12.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.327+0000 7fb39c600640 1 -- 172.21.15.26:0/3277102605 shutdown_connections 2024-01-15T18:13:12.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.327+0000 7fb39c600640 1 -- 172.21.15.26:0/3277102605 wait complete. 2024-01-15T18:13:12.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.328+0000 7fb39c600640 1 Processor -- start 2024-01-15T18:13:12.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.328+0000 7fb39c600640 1 -- start start 2024-01-15T18:13:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.329+0000 7fb39c600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39413df80 0x7fb39414d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.329+0000 7fb39c600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 0x7fb3941f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.329+0000 7fb39c600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb39415f9c0 0x7fb394150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.329+0000 7fb39c600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb394152aa0 con 0x7fb39413e360 2024-01-15T18:13:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.329+0000 7fb39c600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39414da50 con 0x7fb39415f9c0 2024-01-15T18:13:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.329+0000 7fb39c600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39414d870 con 0x7fb39413df80 2024-01-15T18:13:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 0x7fb3941f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 0x7fb3941f31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52014/0 (socket says 172.21.15.26:52014) 2024-01-15T18:13:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399200640 1 -- 172.21.15.26:0/3495078072 learned_addr learned my addr 172.21.15.26:0/3495078072 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399c00640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39413df80 0x7fb39414d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb39a600640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb39415f9c0 0x7fb394150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399200640 1 -- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39413df80 msgr2=0x7fb39414d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399200640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39413df80 0x7fb39414d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399200640 1 -- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb39415f9c0 msgr2=0x7fb394150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399200640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb39415f9c0 0x7fb394150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399200640 1 -- 172.21.15.26:0/3495078072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c009400 con 0x7fb39413e360 2024-01-15T18:13:12.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.330+0000 7fb399200640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 0x7fb3941f31b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fb37c02bee0 tx=0x7fb37c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:12.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.331+0000 7fb391600640 1 -- 172.21.15.26:0/3495078072 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c03a6f0 con 0x7fb39413e360 2024-01-15T18:13:12.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.331+0000 7fb391600640 1 -- 172.21.15.26:0/3495078072 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c03a870 con 0x7fb39413e360 2024-01-15T18:13:12.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.331+0000 7fb391600640 1 -- 172.21.15.26:0/3495078072 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c03ac20 con 0x7fb39413e360 2024-01-15T18:13:12.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.331+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39414e220 con 0x7fb39413e360 2024-01-15T18:13:12.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.331+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb394152700 con 0x7fb39413e360 2024-01-15T18:13:12.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.332+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb394152be0 con 0x7fb39413e360 2024-01-15T18:13:12.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.333+0000 7fb391600640 1 -- 172.21.15.26:0/3495078072 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 39) v1 ==== 91171+0+0 (secure 0 0 0) 0x7fb37c030380 con 0x7fb39413e360 2024-01-15T18:13:12.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.334+0000 7fb391600640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb36c06c4c0 0x7fb36c06e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.334+0000 7fb391600640 1 -- 172.21.15.26:0/3495078072 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fb37c0aa5b0 con 0x7fb39413e360 2024-01-15T18:13:12.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.335+0000 7fb399c00640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb36c06c4c0 0x7fb36c06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.339 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.336+0000 7fb399c00640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb36c06c4c0 0x7fb36c06e980 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb384000f60 tx=0x7fb384009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:12.341 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.339+0000 7fb391600640 1 -- 172.21.15.26:0/3495078072 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb37c0e0e90 con 0x7fb39413e360 2024-01-15T18:13:12.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.565+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb39406cff0 con 0x7fb36c06c4c0 2024-01-15T18:13:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.597+0000 7fb391600640 1 -- 172.21.15.26:0/3495078072 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb384001c80 con 0x7fb36c06c4c0 2024-01-15T18:13:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb36c06c4c0 msgr2=0x7fb36c06e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb36c06c4c0 0x7fb36c06e980 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb384000f60 tx=0x7fb384009040 comp rx=0 tx=0).stop 2024-01-15T18:13:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 msgr2=0x7fb3941f31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 0x7fb3941f31b0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fb37c02bee0 tx=0x7fb37c0046c0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 shutdown_connections 2024-01-15T18:13:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb36c06c4c0 0x7fb36c06e980 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb39415f9c0 0x7fb394150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb39413e360 0x7fb3941f31b0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 --2- 172.21.15.26:0/3495078072 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb39413df80 0x7fb39414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 >> 172.21.15.26:0/3495078072 conn(0x7fb39406a800 msgr2=0x7fb3940cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 shutdown_connections 2024-01-15T18:13:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.600+0000 7fb39c600640 1 -- 172.21.15.26:0/3495078072 wait complete. 2024-01-15T18:13:12.621 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv1 2024-01-15T18:13:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.843+0000 7efdef400640 1 Processor -- start 2024-01-15T18:13:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.844+0000 7efdef400640 1 -- start start 2024-01-15T18:13:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.845+0000 7efdef400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7efde80cf9a0 0x7efde80cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.845+0000 7efdef400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde80d0320 con 0x7efde815d4b0 2024-01-15T18:13:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.845+0000 7efdef400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde80d1c60 con 0x7efde80b31d0 2024-01-15T18:13:12.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.845+0000 7efdef400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde80b2730 con 0x7efde80cf9a0 2024-01-15T18:13:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.845+0000 7efdee000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7efde815d4b0 0x7efde80d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60864/0 (socket says 172.21.15.26:60864) 2024-01-15T18:13:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.845+0000 7efdee000640 1 -- 172.21.15.26:0/213203179 learned_addr learned my addr 172.21.15.26:0/213203179 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:13:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.846+0000 7efded600640 1 --2- 172.21.15.26:0/213203179 >> v2:172.21.15.26:3300/0 conn(0x7efde80cf9a0 0x7efde80cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.846+0000 7efdecc00640 1 -- 172.21.15.26:0/213203179 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895498011 0 0) 0x7efde80d0320 con 0x7efde815d4b0 2024-01-15T18:13:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.846+0000 7efded600640 1 -- 172.21.15.26:0/213203179 >> v1:172.21.15.124:6789/0 conn(0x7efde80b31d0 legacy=0x7efde80b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.846+0000 7efded600640 1 -- 172.21.15.26:0/213203179 >> v1:172.21.15.26:6789/0 conn(0x7efde815d4b0 legacy=0x7efde80d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.846+0000 7efded600640 1 -- 172.21.15.26:0/213203179 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde80d1ec0 con 0x7efde80cf9a0 2024-01-15T18:13:12.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.847+0000 7efded600640 1 --2- 172.21.15.26:0/213203179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 0x7efde80cfde0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7efdd00097a0 tx=0x7efdd002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89c617643174ed8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:12.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.848+0000 7efdecc00640 1 -- 172.21.15.26:0/213203179 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0031830 con 0x7efde80cf9a0 2024-01-15T18:13:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.848+0000 7efdecc00640 1 -- 172.21.15.26:0/213203179 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd0031990 con 0x7efde80cf9a0 2024-01-15T18:13:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.848+0000 7efdecc00640 1 -- 172.21.15.26:0/213203179 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0031d20 con 0x7efde80cf9a0 2024-01-15T18:13:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.848+0000 7efdef400640 1 -- 172.21.15.26:0/213203179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 msgr2=0x7efde80cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.848+0000 7efdef400640 1 --2- 172.21.15.26:0/213203179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 0x7efde80cfde0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7efdd00097a0 tx=0x7efdd002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.848+0000 7efdef400640 1 -- 172.21.15.26:0/213203179 shutdown_connections 2024-01-15T18:13:12.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.848+0000 7efdef400640 1 --2- 172.21.15.26:0/213203179 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 0x7efde80cfde0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.848+0000 7efdef400640 1 -- 172.21.15.26:0/213203179 >> 172.21.15.26:0/213203179 conn(0x7efde80d3000 msgr2=0x7efde80d3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:12.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.849+0000 7efdef400640 1 -- 172.21.15.26:0/213203179 shutdown_connections 2024-01-15T18:13:12.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.849+0000 7efdef400640 1 -- 172.21.15.26:0/213203179 wait complete. 2024-01-15T18:13:12.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.850+0000 7efdef400640 1 Processor -- start 2024-01-15T18:13:12.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.851+0000 7efdef400640 1 -- start start 2024-01-15T18:13:12.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.851+0000 7efdef400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde80b31d0 0x7efde80ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.851+0000 7efdef400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 0x7efde809f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.851+0000 7efdef400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde815d4b0 0x7efde815b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.851+0000 7efdef400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde80cacf0 con 0x7efde80cf9a0 2024-01-15T18:13:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.851+0000 7efdef400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde809e1b0 con 0x7efde80b31d0 2024-01-15T18:13:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.851+0000 7efdef400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde80caf90 con 0x7efde815d4b0 2024-01-15T18:13:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.852+0000 7efded600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 0x7efde809f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.852+0000 7efded600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 0x7efde809f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52040/0 (socket says 172.21.15.26:52040) 2024-01-15T18:13:12.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.852+0000 7efded600640 1 -- 172.21.15.26:0/537063550 learned_addr learned my addr 172.21.15.26:0/537063550 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.852+0000 7efded600640 1 -- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde815d4b0 msgr2=0x7efde815b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.852+0000 7efded600640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde815d4b0 0x7efde815b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.852+0000 7efded600640 1 -- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde80b31d0 msgr2=0x7efde80ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:12.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.852+0000 7efdee000640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde80b31d0 0x7efde80ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.852+0000 7efded600640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde80b31d0 0x7efde80ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.852+0000 7efded600640 1 -- 172.21.15.26:0/537063550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0009400 con 0x7efde80cf9a0 2024-01-15T18:13:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.853+0000 7efded600640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 0x7efde809f1b0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7efdd00097a0 tx=0x7efdd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.853+0000 7efde6000640 1 -- 172.21.15.26:0/537063550 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd003ca80 con 0x7efde80cf9a0 2024-01-15T18:13:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.853+0000 7efde6000640 1 -- 172.21.15.26:0/537063550 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd003cbe0 con 0x7efde80cf9a0 2024-01-15T18:13:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.853+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde80cb270 con 0x7efde80cf9a0 2024-01-15T18:13:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.853+0000 7efde6000640 1 -- 172.21.15.26:0/537063550 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0004030 con 0x7efde80cf9a0 2024-01-15T18:13:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.853+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde815c830 con 0x7efde80cf9a0 2024-01-15T18:13:12.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.854+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde80cb790 con 0x7efde80cf9a0 2024-01-15T18:13:12.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.855+0000 7efde6000640 1 -- 172.21.15.26:0/537063550 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 39) v1 ==== 91171+0+0 (secure 0 0 0) 0x7efdd0004190 con 0x7efde80cf9a0 2024-01-15T18:13:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.856+0000 7efde6000640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdc006c4c0 0x7efdc006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.856+0000 7efde6000640 1 -- 172.21.15.26:0/537063550 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7efdd00aafb0 con 0x7efde80cf9a0 2024-01-15T18:13:12.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.857+0000 7efdee000640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdc006c4c0 0x7efdc006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:12.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.858+0000 7efdee000640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdc006c4c0 0x7efdc006e980 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efdd8000f60 tx=0x7efdd8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:12.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:12.860+0000 7efde6000640 1 -- 172.21.15.26:0/537063550 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efde80cb790 con 0x7efde80cf9a0 2024-01-15T18:13:13.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.089+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv1", "target": ["mon-mgr", ""]}) v1 -- 0x7efde80d21a0 con 0x7efdc006c4c0 2024-01-15T18:13:13.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.101+0000 7efde6000640 1 -- 172.21.15.26:0/537063550 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7efdd80070d0 con 0x7efdc006c4c0 2024-01-15T18:13:13.104 INFO:teuthology.orchestra.run.smithi026.stdout:/volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc 2024-01-15T18:13:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdc006c4c0 msgr2=0x7efdc006e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdc006c4c0 0x7efdc006e980 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7efdd8000f60 tx=0x7efdd8009040 comp rx=0 tx=0).stop 2024-01-15T18:13:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 msgr2=0x7efde809f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 0x7efde809f1b0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7efdd00097a0 tx=0x7efdd00046c0 comp rx=0 tx=0).stop 2024-01-15T18:13:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 shutdown_connections 2024-01-15T18:13:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7efde815d4b0 0x7efde815b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7efdc006c4c0 0x7efdc006e980 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7efde80cf9a0 0x7efde809f1b0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:13.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 --2- 172.21.15.26:0/537063550 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7efde80b31d0 0x7efde80ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 >> 172.21.15.26:0/537063550 conn(0x7efde80d3000 msgr2=0x7efde806b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 shutdown_connections 2024-01-15T18:13:13.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:13.104+0000 7efdef400640 1 -- 172.21.15.26:0/537063550 wait complete. 2024-01-15T18:13:13.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:13:13.127 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc) 2024-01-15T18:13:13.189 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc 2024-01-15T18:13:13.189 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3138256896" 2024-01-15T18:13:13.189 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:13:13.190 INFO:tasks.cephfs.test_volumes:r_size = 3138256896 2024-01-15T18:13:15.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:13:15.192 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr -n ceph.dir.rbytes volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc) 2024-01-15T18:13:15.256 INFO:teuthology.orchestra.run.smithi026.stdout:# file: volumes/_nogroup/sv1/56335b58-5158-49ea-9920-55d354bd39bc 2024-01-15T18:13:15.256 INFO:teuthology.orchestra.run.smithi026.stdout:ceph.dir.rbytes="3221225472" 2024-01-15T18:13:15.256 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:13:15.257 INFO:tasks.cephfs.test_volumes:r_size = 3221225472 2024-01-15T18:13:15.258 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone1111 2024-01-15T18:13:15.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.483+0000 7f7002800640 1 Processor -- start 2024-01-15T18:13:15.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.483+0000 7f7002800640 1 -- start start 2024-01-15T18:13:15.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.485+0000 7f7002800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6ffc13df80 0x7f6ffc15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:15.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.485+0000 7f7002800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ffc0cf9a0 0x7f6ffc0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:15.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.485+0000 7f7002800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc06c470 con 0x7f6ffc13e360 2024-01-15T18:13:15.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.485+0000 7f7002800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ffc06c750 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.485+0000 7f7002800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ffc06ca30 con 0x7f6ffc13df80 2024-01-15T18:13:15.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.485+0000 7f6ffaa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6ffc13e360 0x7f6ffc0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60878/0 (socket says 172.21.15.26:60878) 2024-01-15T18:13:15.488 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.485+0000 7f6ffaa00640 1 -- 172.21.15.26:0/3420771447 learned_addr learned my addr 172.21.15.26:0/3420771447 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:13:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.485+0000 7f6ffbe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f6ffc0cf9a0 0x7f6ffc0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:15.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.486+0000 7f6ffb400640 1 --2- 172.21.15.26:0/3420771447 >> v2:172.21.15.124:3300/0 conn(0x7f6ffc13df80 0x7f6ffc15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.486+0000 7f6ffa000640 1 -- 172.21.15.26:0/3420771447 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576443295 0 0) 0x7f6ffc06c470 con 0x7f6ffc13e360 2024-01-15T18:13:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.486+0000 7f6ffa000640 1 -- 172.21.15.26:0/3420771447 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fdc003520 con 0x7f6ffc13e360 2024-01-15T18:13:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.486+0000 7f6ffbe00640 1 -- 172.21.15.26:0/3420771447 >> v2:172.21.15.124:3300/0 conn(0x7f6ffc13df80 msgr2=0x7f6ffc15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.486+0000 7f6ffbe00640 1 --2- 172.21.15.26:0/3420771447 >> v2:172.21.15.124:3300/0 conn(0x7f6ffc13df80 0x7f6ffc15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.486+0000 7f6ffbe00640 1 -- 172.21.15.26:0/3420771447 >> v1:172.21.15.26:6789/0 conn(0x7f6ffc13e360 legacy=0x7f6ffc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.486+0000 7f6ffbe00640 1 -- 172.21.15.26:0/3420771447 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc06cd10 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.490 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.486+0000 7f6ffb400640 1 --2- 172.21.15.26:0/3420771447 >> v2:172.21.15.124:3300/0 conn(0x7f6ffc13df80 0x7f6ffc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:13:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.487+0000 7f6ffbe00640 1 --2- 172.21.15.26:0/3420771447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 0x7f6ffc0cd840 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc072770 tx=0x7f6ff402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6781acd7485901 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.487+0000 7f6ffa000640 1 -- 172.21.15.26:0/3420771447 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4031830 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f6ffa000640 1 -- 172.21.15.26:0/3420771447 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4031990 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f6ffa000640 1 -- 172.21.15.26:0/3420771447 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4031d20 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f7002800640 1 -- 172.21.15.26:0/3420771447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 msgr2=0x7f6ffc0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f7002800640 1 --2- 172.21.15.26:0/3420771447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 0x7f6ffc0cd840 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc072770 tx=0x7f6ff402b8d0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f7002800640 1 -- 172.21.15.26:0/3420771447 shutdown_connections 2024-01-15T18:13:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f7002800640 1 --2- 172.21.15.26:0/3420771447 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 0x7f6ffc0cd840 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f7002800640 1 --2- 172.21.15.26:0/3420771447 >> v2:172.21.15.124:3300/0 conn(0x7f6ffc13df80 0x7f6ffc15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f7002800640 1 -- 172.21.15.26:0/3420771447 >> 172.21.15.26:0/3420771447 conn(0x7f6ffc06a800 msgr2=0x7f6ffc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f7002800640 1 -- 172.21.15.26:0/3420771447 shutdown_connections 2024-01-15T18:13:15.492 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.488+0000 7f7002800640 1 -- 172.21.15.26:0/3420771447 wait complete. 2024-01-15T18:13:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.490+0000 7f7002800640 1 Processor -- start 2024-01-15T18:13:15.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.490+0000 7f7002800640 1 -- start start 2024-01-15T18:13:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.491+0000 7f7002800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 0x7f6ffc13d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.491+0000 7f7002800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ffc13df80 0x7f6ffc153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:15.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.491+0000 7f7002800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ffc13e360 0x7f6ffc14e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.491+0000 7f7002800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc152b80 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.491+0000 7f7002800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc152dd0 con 0x7f6ffc13df80 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.491+0000 7f7002800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc1530b0 con 0x7f6ffc13e360 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 0x7f6ffc13d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 0x7f6ffc13d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52050/0 (socket says 172.21.15.26:52050) 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffb400640 1 -- 172.21.15.26:0/68558846 learned_addr learned my addr 172.21.15.26:0/68558846 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffaa00640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ffc13df80 0x7f6ffc153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffbe00640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ffc13e360 0x7f6ffc14e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffb400640 1 -- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ffc13e360 msgr2=0x7f6ffc14e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffb400640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ffc13e360 0x7f6ffc14e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffb400640 1 -- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ffc13df80 msgr2=0x7f6ffc153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:15.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffb400640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ffc13df80 0x7f6ffc153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.492+0000 7f6ffb400640 1 -- 172.21.15.26:0/68558846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff4009400 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.493+0000 7f6ffb400640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 0x7f6ffc13d7d0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009fc0 tx=0x7f6fec00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.493+0000 7f6ff3400640 1 -- 172.21.15.26:0/68558846 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec003b90 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.493+0000 7f6ff3400640 1 -- 172.21.15.26:0/68558846 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec00f040 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.493+0000 7f6ff3400640 1 -- 172.21.15.26:0/68558846 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec004030 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.493+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc153390 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.493+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc153950 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.494+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc0b0990 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.496+0000 7f6ff3400640 1 -- 172.21.15.26:0/68558846 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 39) v1 ==== 91171+0+0 (secure 0 0 0) 0x7f6fec004190 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.496+0000 7f6ff3400640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fd406c510 0x7f6fd406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:15.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.496+0000 7f6ff3400640 1 -- 172.21.15.26:0/68558846 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f6fec088f70 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.497+0000 7f6ffaa00640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fd406c510 0x7f6fd406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:15.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.498+0000 7f6ffaa00640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fd406c510 0x7f6fd406e9d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc14dbc0 tx=0x7f6fe4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:15.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.501+0000 7f6ff3400640 1 -- 172.21.15.26:0/68558846 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fec0554e0 con 0x7f6ffc0cf9a0 2024-01-15T18:13:15.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.733+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone1111", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ffc15f9c0 con 0x7f6fd406c510 2024-01-15T18:13:15.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.891+0000 7f6ff3400640 1 -- 172.21.15.26:0/68558846 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6fe4007cf0 con 0x7f6fd406c510 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fd406c510 msgr2=0x7f6fd406e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fd406c510 0x7f6fd406e9d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc14dbc0 tx=0x7f6fe4009040 comp rx=0 tx=0).stop 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 msgr2=0x7f6ffc13d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 0x7f6ffc13d7d0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009fc0 tx=0x7f6fec00d560 comp rx=0 tx=0).stop 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 shutdown_connections 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6ffc13e360 0x7f6ffc14e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fd406c510 0x7f6fd406e9d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6ffc13df80 0x7f6ffc153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 --2- 172.21.15.26:0/68558846 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6ffc0cf9a0 0x7f6ffc13d7d0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.894+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 >> 172.21.15.26:0/68558846 conn(0x7f6ffc06a800 msgr2=0x7f6ffc0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.895+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 shutdown_connections 2024-01-15T18:13:15.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:15.895+0000 7f7002800640 1 -- 172.21.15.26:0/68558846 wait complete. 2024-01-15T18:13:15.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.900+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:15.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.900+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:15.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.900+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:15.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.900+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:15.904 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.900+0000 7f8f5a000640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:15.919 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone1112 2024-01-15T18:13:15.931 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.928+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:15.931 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.928+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:15.931 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.928+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:15.931 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.928+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:15.931 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:15.928+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:16.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.140+0000 7f6f29c00640 1 Processor -- start 2024-01-15T18:13:16.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.140+0000 7f6f29c00640 1 -- start start 2024-01-15T18:13:16.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.142+0000 7f6f29c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 0x7f6f240b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.142+0000 7f6f29c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6f240d0f40 0x7f6f240d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.142+0000 7f6f29c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f240d0a10 con 0x7f6f24160920 2024-01-15T18:13:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.142+0000 7f6f29c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f240cf080 con 0x7f6f240d0f40 2024-01-15T18:13:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.142+0000 7f6f29c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240c6be0 con 0x7f6f240d0100 2024-01-15T18:13:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.142+0000 7f6f23e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6f24160920 0x7f6f240cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60890/0 (socket says 172.21.15.26:60890) 2024-01-15T18:13:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.142+0000 7f6f23e00640 1 -- 172.21.15.26:0/3698600437 learned_addr learned my addr 172.21.15.26:0/3698600437 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:13:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.142+0000 7f6f23400640 1 --2- 172.21.15.26:0/3698600437 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 0x7f6f240b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.143+0000 7f6f29200640 1 --2- 172.21.15.26:0/3698600437 >> v2:172.21.15.124:3300/0 conn(0x7f6f240d0f40 0x7f6f240d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.143+0000 7f6f22a00640 1 -- 172.21.15.26:0/3698600437 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2147121613 0 0) 0x7f6f240d0a10 con 0x7f6f24160920 2024-01-15T18:13:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.143+0000 7f6f22a00640 1 -- 172.21.15.26:0/3698600437 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f04003520 con 0x7f6f24160920 2024-01-15T18:13:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.143+0000 7f6f23400640 1 -- 172.21.15.26:0/3698600437 >> v2:172.21.15.124:3300/0 conn(0x7f6f240d0f40 msgr2=0x7f6f240d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.143+0000 7f6f23400640 1 --2- 172.21.15.26:0/3698600437 >> v2:172.21.15.124:3300/0 conn(0x7f6f240d0f40 0x7f6f240d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.143+0000 7f6f22a00640 1 -- 172.21.15.26:0/3698600437 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 547232174 0 0) 0x7f6f04003520 con 0x7f6f24160920 2024-01-15T18:13:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.143+0000 7f6f23400640 1 -- 172.21.15.26:0/3698600437 >> v1:172.21.15.26:6789/0 conn(0x7f6f24160920 legacy=0x7f6f240cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.143+0000 7f6f23400640 1 -- 172.21.15.26:0/3698600437 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f240ce7e0 con 0x7f6f240d0100 2024-01-15T18:13:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.145+0000 7f6f23400640 1 --2- 172.21.15.26:0/3698600437 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 0x7f6f240b3f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0097a0 tx=0x7f6f0c02b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eadf7fef43160b24 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.145+0000 7f6f22a00640 1 -- 172.21.15.26:0/3698600437 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14001670 con 0x7f6f240d0100 2024-01-15T18:13:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.145+0000 7f6f22a00640 1 -- 172.21.15.26:0/3698600437 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c02e050 con 0x7f6f240d0100 2024-01-15T18:13:16.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.145+0000 7f6f22a00640 1 -- 172.21.15.26:0/3698600437 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c030730 con 0x7f6f240d0100 2024-01-15T18:13:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.146+0000 7f6f29c00640 1 -- 172.21.15.26:0/3698600437 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 msgr2=0x7f6f240b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.146+0000 7f6f29c00640 1 --2- 172.21.15.26:0/3698600437 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 0x7f6f240b3f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0097a0 tx=0x7f6f0c02b790 comp rx=0 tx=0).stop 2024-01-15T18:13:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.146+0000 7f6f29c00640 1 -- 172.21.15.26:0/3698600437 shutdown_connections 2024-01-15T18:13:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.146+0000 7f6f29c00640 1 --2- 172.21.15.26:0/3698600437 >> v2:172.21.15.124:3300/0 conn(0x7f6f240d0f40 0x7f6f240d13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.146+0000 7f6f29c00640 1 --2- 172.21.15.26:0/3698600437 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 0x7f6f240b3f30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.146+0000 7f6f29c00640 1 -- 172.21.15.26:0/3698600437 >> 172.21.15.26:0/3698600437 conn(0x7f6f240d3090 msgr2=0x7f6f240d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.146+0000 7f6f29c00640 1 -- 172.21.15.26:0/3698600437 shutdown_connections 2024-01-15T18:13:16.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.146+0000 7f6f29c00640 1 -- 172.21.15.26:0/3698600437 wait complete. 2024-01-15T18:13:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.148+0000 7f6f29c00640 1 Processor -- start 2024-01-15T18:13:16.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.148+0000 7f6f29c00640 1 -- start start 2024-01-15T18:13:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.149+0000 7f6f29c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 0x7f6f24159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.149+0000 7f6f29c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f240d0f40 0x7f6f240cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.149+0000 7f6f29c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f24160920 0x7f6f240b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.149+0000 7f6f29c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240b7350 con 0x7f6f240d0f40 2024-01-15T18:13:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.149+0000 7f6f29c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240c6f80 con 0x7f6f24160920 2024-01-15T18:13:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.149+0000 7f6f29c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f240c7260 con 0x7f6f240d0100 2024-01-15T18:13:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.149+0000 7f6f23400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f240d0f40 0x7f6f240cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.149+0000 7f6f23400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f240d0f40 0x7f6f240cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52064/0 (socket says 172.21.15.26:52064) 2024-01-15T18:13:16.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.149+0000 7f6f23400640 1 -- 172.21.15.26:0/1103012289 learned_addr learned my addr 172.21.15.26:0/1103012289 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.150+0000 7f6f29200640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f24160920 0x7f6f240b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.150+0000 7f6f23e00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 0x7f6f24159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.150+0000 7f6f23400640 1 -- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 msgr2=0x7f6f24159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.150+0000 7f6f23400640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 0x7f6f24159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.150+0000 7f6f23400640 1 -- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f24160920 msgr2=0x7f6f240b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.150+0000 7f6f23400640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f24160920 0x7f6f240b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.150+0000 7f6f23400640 1 -- 172.21.15.26:0/1103012289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c009400 con 0x7f6f240d0f40 2024-01-15T18:13:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.150+0000 7f6f23400640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f240d0f40 0x7f6f240cb260 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0098c0 tx=0x7f6f0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.151+0000 7f6f1be00640 1 -- 172.21.15.26:0/1103012289 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c03d890 con 0x7f6f240d0f40 2024-01-15T18:13:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.151+0000 7f6f1be00640 1 -- 172.21.15.26:0/1103012289 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c03d9f0 con 0x7f6f240d0f40 2024-01-15T18:13:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.151+0000 7f6f1be00640 1 -- 172.21.15.26:0/1103012289 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c03dd80 con 0x7f6f240d0f40 2024-01-15T18:13:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.151+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f240c7540 con 0x7f6f240d0f40 2024-01-15T18:13:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.151+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f240cb7a0 con 0x7f6f240d0f40 2024-01-15T18:13:16.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.152+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f240c7a60 con 0x7f6f240d0f40 2024-01-15T18:13:16.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.153+0000 7f6f1be00640 1 -- 172.21.15.26:0/1103012289 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 39) v1 ==== 91171+0+0 (secure 0 0 0) 0x7f6f0c039040 con 0x7f6f240d0f40 2024-01-15T18:13:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.154+0000 7f6f1be00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6efc06c4c0 0x7f6efc06e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.154+0000 7f6f1be00640 1 -- 172.21.15.26:0/1103012289 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f6f0c0aabf0 con 0x7f6f240d0f40 2024-01-15T18:13:16.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.155+0000 7f6f23e00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6efc06c4c0 0x7f6efc06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.156+0000 7f6f23e00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6efc06c4c0 0x7f6efc06e980 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6f140011e0 tx=0x7f6f14009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:16.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.159+0000 7f6f1be00640 1 -- 172.21.15.26:0/1103012289 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f240c7a60 con 0x7f6f240d0f40 2024-01-15T18:13:16.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.393+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone1112", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f24160d00 con 0x7f6efc06c4c0 2024-01-15T18:13:16.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.555+0000 7f6f1be00640 1 -- 172.21.15.26:0/1103012289 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6f14001c10 con 0x7f6efc06c4c0 2024-01-15T18:13:16.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6efc06c4c0 msgr2=0x7f6efc06e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6efc06c4c0 0x7f6efc06e980 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f6f140011e0 tx=0x7f6f14009040 comp rx=0 tx=0).stop 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f240d0f40 msgr2=0x7f6f240cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f240d0f40 0x7f6f240cb260 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0098c0 tx=0x7f6f0c0046c0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 shutdown_connections 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6efc06c4c0 0x7f6efc06e980 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6f24160920 0x7f6f240b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6f240d0f40 0x7f6f240cb260 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 --2- 172.21.15.26:0/1103012289 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6f240d0100 0x7f6f24159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.558+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 >> 172.21.15.26:0/1103012289 conn(0x7f6f240d3090 msgr2=0x7f6f240b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.559+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 shutdown_connections 2024-01-15T18:13:16.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.559+0000 7f6f29c00640 1 -- 172.21.15.26:0/1103012289 wait complete. 2024-01-15T18:13:16.587 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs sv1 ss1 subvol_clone1113 2024-01-15T18:13:16.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.809+0000 7fc706a00640 1 Processor -- start 2024-01-15T18:13:16.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.809+0000 7fc706a00640 1 -- start start 2024-01-15T18:13:16.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.811+0000 7fc706a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc70013df80 0x7fc70015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.811+0000 7fc706a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70006c470 con 0x7fc70013e360 2024-01-15T18:13:16.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.811+0000 7fc706a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70006c750 con 0x7fc7000cf9a0 2024-01-15T18:13:16.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.811+0000 7fc706a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc70006ca30 con 0x7fc70013df80 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.811+0000 7fc6ff400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fc70013e360 0x7fc7000cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60904/0 (socket says 172.21.15.26:60904) 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.811+0000 7fc6ff400640 1 -- 172.21.15.26:0/3368568157 learned_addr learned my addr 172.21.15.26:0/3368568157 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.811+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985930061 0 0) 0x7fc70006c470 con 0x7fc70013e360 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6e0003520 con 0x7fc70013e360 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6ffe00640 1 --2- 172.21.15.26:0/3368568157 >> v2:172.21.15.124:3300/0 conn(0x7fc70013df80 0x7fc70015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1339525175 0 0) 0x7fc6e0003520 con 0x7fc70013e360 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 >> v2:172.21.15.124:3300/0 conn(0x7fc70013df80 msgr2=0x7fc70015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 --2- 172.21.15.26:0/3368568157 >> v2:172.21.15.124:3300/0 conn(0x7fc70013df80 0x7fc70015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 >> v1:172.21.15.124:6789/0 conn(0x7fc7000cf9a0 legacy=0x7fc7000cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:16.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70006cd10 con 0x7fc70013e360 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fc6e8001670 con 0x7fc70013e360 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 >> v1:172.21.15.26:6789/0 conn(0x7fc70013e360 legacy=0x7fc7000cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6e0003970 0x7fc6e0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6e00047c0 0x7fc6e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6e0005180 0x7fc6e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70006ca30 con 0x7fc6e0005180 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e000c0b0 con 0x7fc6e00047c0 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e000c270 con 0x7fc6e0003970 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.812+0000 7fc704a00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6e0005180 0x7fc6e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.813+0000 7fc6ffe00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6e0003970 0x7fc6e0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.813+0000 7fc6ff400640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6e00047c0 0x7fc6e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.818 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.813+0000 7fc704a00640 1 -- 172.21.15.26:0/3368568157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6e0003970 msgr2=0x7fc6e0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:16.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.813+0000 7fc704a00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6e0003970 0x7fc6e0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.813+0000 7fc704a00640 1 -- 172.21.15.26:0/3368568157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6e00047c0 msgr2=0x7fc6e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:16.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.813+0000 7fc704a00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6e00047c0 0x7fc6e0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.813+0000 7fc704a00640 1 -- 172.21.15.26:0/3368568157 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0003520 con 0x7fc6e0005180 2024-01-15T18:13:16.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.814+0000 7fc704a00640 1 -- 172.21.15.26:0/3368568157 reap_dead start 2024-01-15T18:13:16.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.814+0000 7fc704a00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6e0005180 0x7fc6e000b980 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc70015b6f0 tx=0x7fc6e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349640099634187c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:16.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.814+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8034070 con 0x7fc6e0005180 2024-01-15T18:13:16.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.814+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f8003970 con 0x7fc6e0005180 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.814+0000 7fc6fea00640 1 -- 172.21.15.26:0/3368568157 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f8003d00 con 0x7fc6e0005180 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.815+0000 7fc706a00640 1 -- 172.21.15.26:0/3368568157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6e0005180 msgr2=0x7fc6e000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.815+0000 7fc706a00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6e0005180 0x7fc6e000b980 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fc70015b6f0 tx=0x7fc6e00057f0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.815+0000 7fc706a00640 1 -- 172.21.15.26:0/3368568157 shutdown_connections 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.815+0000 7fc706a00640 1 --2- 172.21.15.26:0/3368568157 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc6e0005180 0x7fc6e000b980 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.815+0000 7fc706a00640 1 -- 172.21.15.26:0/3368568157 >> 172.21.15.26:0/3368568157 conn(0x7fc70006a800 msgr2=0x7fc70006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.815+0000 7fc706a00640 1 -- 172.21.15.26:0/3368568157 shutdown_connections 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.815+0000 7fc706a00640 1 -- 172.21.15.26:0/3368568157 wait complete. 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.817+0000 7fc706a00640 1 Processor -- start 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.817+0000 7fc706a00640 1 -- start start 2024-01-15T18:13:16.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.818+0000 7fc706a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6e0005180 0x7fc70015d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.818+0000 7fc706a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7000702c0 0x7fc70013d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.818+0000 7fc706a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc700070720 0x7fc70013d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.818+0000 7fc706a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc700070cd0 con 0x7fc7000702c0 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.818+0000 7fc706a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc700070fb0 con 0x7fc6e0005180 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.818+0000 7fc706a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc700071290 con 0x7fc700070720 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.818+0000 7fc6ff400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7000702c0 0x7fc70013d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc6ff400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7000702c0 0x7fc70013d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52070/0 (socket says 172.21.15.26:52070) 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc6ff400640 1 -- 172.21.15.26:0/2779393022 learned_addr learned my addr 172.21.15.26:0/2779393022 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc6ffe00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6e0005180 0x7fc70015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc704a00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc700070720 0x7fc70013d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc6ff400640 1 -- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc700070720 msgr2=0x7fc70013d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc6ff400640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc700070720 0x7fc70013d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc6ff400640 1 -- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6e0005180 msgr2=0x7fc70015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:16.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc6ff400640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6e0005180 0x7fc70015d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:16.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc6ff400640 1 -- 172.21.15.26:0/2779393022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7001f8360 con 0x7fc7000702c0 2024-01-15T18:13:16.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc704a00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc700070720 0x7fc70013d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:13:16.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.819+0000 7fc6ff400640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7000702c0 0x7fc70013d450 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800d7a0 tx=0x7fc6e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:16.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.820+0000 7fc6f7e00640 1 -- 172.21.15.26:0/2779393022 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80074e0 con 0x7fc7000702c0 2024-01-15T18:13:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.820+0000 7fc6f7e00640 1 -- 172.21.15.26:0/2779393022 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e8007640 con 0x7fc7000702c0 2024-01-15T18:13:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.820+0000 7fc6f7e00640 1 -- 172.21.15.26:0/2779393022 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8004a20 con 0x7fc7000702c0 2024-01-15T18:13:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.820+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7001f8060 con 0x7fc7000702c0 2024-01-15T18:13:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.820+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70015d760 con 0x7fc7000702c0 2024-01-15T18:13:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.821+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7000675d0 con 0x7fc7000702c0 2024-01-15T18:13:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.822+0000 7fc6f7e00640 1 -- 172.21.15.26:0/2779393022 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 39) v1 ==== 91171+0+0 (secure 0 0 0) 0x7fc6e8004b80 con 0x7fc7000702c0 2024-01-15T18:13:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.823+0000 7fc6f7e00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6d806c4c0 0x7fc6d806e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.823+0000 7fc6f7e00640 1 -- 172.21.15.26:0/2779393022 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fc6e8088400 con 0x7fc7000702c0 2024-01-15T18:13:16.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.824+0000 7fc6ffe00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6d806c4c0 0x7fc6d806e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:16.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.825+0000 7fc6ffe00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6d806c4c0 0x7fc6d806e980 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:16.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:16.828+0000 7fc6f7e00640 1 -- 172.21.15.26:0/2779393022 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6e80583e0 con 0x7fc7000702c0 2024-01-15T18:13:17.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:17.056+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "sv1", "snap_name": "ss1", "target_sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7000cf290 con 0x7fc6d806c4c0 2024-01-15T18:13:17.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:17.437+0000 7fc6f7e00640 1 -- 172.21.15.26:0/2779393022 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 40) v1 ==== 91273+0+0 (secure 0 0 0) 0x7fc6e8054d20 con 0x7fc7000702c0 2024-01-15T18:13:20.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.410+0000 7fc6f7e00640 1 -- 172.21.15.26:0/2779393022 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc6f000d1e0 con 0x7fc6d806c4c0 2024-01-15T18:13:20.416 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.413+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6d806c4c0 msgr2=0x7fc6d806e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.413+0000 7fc706a00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6d806c4c0 0x7fc6d806e980 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0006fd0 tx=0x7fc6f0008040 comp rx=0 tx=0).stop 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.413+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7000702c0 msgr2=0x7fc70013d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.413+0000 7fc706a00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7000702c0 0x7fc70013d450 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fc6e800d7a0 tx=0x7fc6e800dc60 comp rx=0 tx=0).stop 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.414+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 shutdown_connections 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.414+0000 7fc706a00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc700070720 0x7fc70013d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.414+0000 7fc706a00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc6d806c4c0 0x7fc6d806e980 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.414+0000 7fc706a00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc7000702c0 0x7fc70013d450 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.414+0000 7fc706a00640 1 --2- 172.21.15.26:0/2779393022 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6e0005180 0x7fc70015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.414+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 >> 172.21.15.26:0/2779393022 conn(0x7fc70006a800 msgr2=0x7fc70015ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.414+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 shutdown_connections 2024-01-15T18:13:20.417 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.414+0000 7fc706a00640 1 -- 172.21.15.26:0/2779393022 wait complete. 2024-01-15T18:13:20.436 INFO:tasks.cephfs.test_volumes:expected clone state = ('pending', 'in-progress') 2024-01-15T18:13:20.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone1113 2024-01-15T18:13:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.657+0000 7fb9ee600640 1 Processor -- start 2024-01-15T18:13:20.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.657+0000 7fb9ee600640 1 -- start start 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.659+0000 7fb9ee600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb9e8143ae0 0x7fb9e80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.659+0000 7fb9ee600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9e80cf7d0 0x7fb9e815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.659+0000 7fb9ee600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e80cfc30 con 0x7fb9e815e1e0 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.659+0000 7fb9ee600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e80b1ee0 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.659+0000 7fb9ee600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e80b2780 con 0x7fb9e8143ae0 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.659+0000 7fb9dfe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9e80cf7d0 0x7fb9e815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.659+0000 7fb9dfe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb9e80cf7d0 0x7fb9e815dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52082/0 (socket says 172.21.15.26:52082) 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.659+0000 7fb9dfe00640 1 -- 172.21.15.26:0/1749562631 learned_addr learned my addr 172.21.15.26:0/1749562631 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.660+0000 7fb9e7400640 1 --2- 172.21.15.26:0/1749562631 >> v2:172.21.15.124:3300/0 conn(0x7fb9e8143ae0 0x7fb9e80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.660+0000 7fb9dfe00640 1 -- 172.21.15.26:0/1749562631 >> v2:172.21.15.124:3300/0 conn(0x7fb9e8143ae0 msgr2=0x7fb9e80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.660+0000 7fb9dfe00640 1 --2- 172.21.15.26:0/1749562631 >> v2:172.21.15.124:3300/0 conn(0x7fb9e8143ae0 0x7fb9e80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.660+0000 7fb9dfe00640 1 -- 172.21.15.26:0/1749562631 >> v1:172.21.15.124:6789/0 conn(0x7fb9e815e1e0 legacy=0x7fb9e815a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:20.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.660+0000 7fb9dfe00640 1 -- 172.21.15.26:0/1749562631 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e815e990 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.661+0000 7fb9dfe00640 1 --2- 172.21.15.26:0/1749562631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 0x7fb9e815dca0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80cfea0 tx=0x7fb9d002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bdea1d43b5dca7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.661+0000 7fb9e6a00640 1 -- 172.21.15.26:0/1749562631 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d002ee90 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.661+0000 7fb9e6a00640 1 -- 172.21.15.26:0/1749562631 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0036440 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.661+0000 7fb9e6a00640 1 -- 172.21.15.26:0/1749562631 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d00367d0 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.662+0000 7fb9ee600640 1 -- 172.21.15.26:0/1749562631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 msgr2=0x7fb9e815dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:20.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.662+0000 7fb9ee600640 1 --2- 172.21.15.26:0/1749562631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 0x7fb9e815dca0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80cfea0 tx=0x7fb9d002e0e0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.662+0000 7fb9ee600640 1 -- 172.21.15.26:0/1749562631 shutdown_connections 2024-01-15T18:13:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.662+0000 7fb9ee600640 1 --2- 172.21.15.26:0/1749562631 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 0x7fb9e815dca0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.662+0000 7fb9ee600640 1 --2- 172.21.15.26:0/1749562631 >> v2:172.21.15.124:3300/0 conn(0x7fb9e8143ae0 0x7fb9e80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.662+0000 7fb9ee600640 1 -- 172.21.15.26:0/1749562631 >> 172.21.15.26:0/1749562631 conn(0x7fb9e806a800 msgr2=0x7fb9e806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.662+0000 7fb9ee600640 1 -- 172.21.15.26:0/1749562631 shutdown_connections 2024-01-15T18:13:20.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.662+0000 7fb9ee600640 1 -- 172.21.15.26:0/1749562631 wait complete. 2024-01-15T18:13:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.664+0000 7fb9ee600640 1 Processor -- start 2024-01-15T18:13:20.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.664+0000 7fb9ee600640 1 -- start start 2024-01-15T18:13:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.665+0000 7fb9ee600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 0x7fb9e81f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.665+0000 7fb9ee600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8143ae0 0x7fb9e815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.665+0000 7fb9ee600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e815e1e0 0x7fb9e81f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:20.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.665+0000 7fb9ee600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80c4350 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.665+0000 7fb9ee600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e81f3770 con 0x7fb9e8143ae0 2024-01-15T18:13:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.665+0000 7fb9ee600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e81f3a50 con 0x7fb9e815e1e0 2024-01-15T18:13:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.665+0000 7fb9e7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 0x7fb9e81f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 0x7fb9e81f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52086/0 (socket says 172.21.15.26:52086) 2024-01-15T18:13:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7400640 1 -- 172.21.15.26:0/2117277976 learned_addr learned my addr 172.21.15.26:0/2117277976 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:20.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9dfe00640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8143ae0 0x7fb9e815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7400640 1 -- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e815e1e0 msgr2=0x7fb9e81f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7e00640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e815e1e0 0x7fb9e81f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7400640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e815e1e0 0x7fb9e81f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7400640 1 -- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8143ae0 msgr2=0x7fb9e815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:20.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7400640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8143ae0 0x7fb9e815a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7400640 1 -- 172.21.15.26:0/2117277976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d000b770 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7e00640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e815e1e0 0x7fb9e81f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:13:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.666+0000 7fb9e7400640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 0x7fb9e81f2f20 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800d800 tx=0x7fb9d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.667+0000 7fb9df400640 1 -- 172.21.15.26:0/2117277976 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800b6c0 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.667+0000 7fb9df400640 1 -- 172.21.15.26:0/2117277976 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d800b820 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.667+0000 7fb9df400640 1 -- 172.21.15.26:0/2117277976 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800bbb0 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.667+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e81f3d30 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.667+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e81f3460 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.668+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e81f3f20 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.669+0000 7fb9df400640 1 -- 172.21.15.26:0/2117277976 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 40) v1 ==== 91273+0+0 (secure 0 0 0) 0x7fb9d800bd10 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.671+0000 7fb9df400640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9c006c5e0 0x7fb9c006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:20.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.671+0000 7fb9df400640 1 -- 172.21.15.26:0/2117277976 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fb9d8089180 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.671+0000 7fb9dfe00640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9c006c5e0 0x7fb9c006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:20.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.672+0000 7fb9dfe00640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9c006c5e0 0x7fb9c006eaa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0038000 tx=0x7fb9d0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:20.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.676+0000 7fb9df400640 1 -- 172.21.15.26:0/2117277976 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9e81f3f20 con 0x7fb9e80cf7d0 2024-01-15T18:13:20.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:20.911+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9e815e670 con 0x7fb9c006c5e0 2024-01-15T18:13:25.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.236+0000 7fb9df400640 1 -- 172.21.15.26:0/2117277976 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+144 (secure 0 0 0) 0x7fb9d0036930 con 0x7fb9c006c5e0 2024-01-15T18:13:25.239 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:13:25.239 INFO:teuthology.orchestra.run.smithi026.stdout: "status": { 2024-01-15T18:13:25.239 INFO:teuthology.orchestra.run.smithi026.stdout: "state": "pending", 2024-01-15T18:13:25.239 INFO:teuthology.orchestra.run.smithi026.stdout: "source": { 2024-01-15T18:13:25.239 INFO:teuthology.orchestra.run.smithi026.stdout: "volume": "cephfs", 2024-01-15T18:13:25.240 INFO:teuthology.orchestra.run.smithi026.stdout: "subvolume": "sv1", 2024-01-15T18:13:25.240 INFO:teuthology.orchestra.run.smithi026.stdout: "snapshot": "ss1" 2024-01-15T18:13:25.240 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:13:25.240 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:13:25.240 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.238+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9c006c5e0 msgr2=0x7fb9c006eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.238+0000 7fb9ee600640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9c006c5e0 0x7fb9c006eaa0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0038000 tx=0x7fb9d0008040 comp rx=0 tx=0).stop 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.238+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 msgr2=0x7fb9e81f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.238+0000 7fb9ee600640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 0x7fb9e81f2f20 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800d800 tx=0x7fb9d800dcc0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.239+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 shutdown_connections 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.239+0000 7fb9ee600640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e815e1e0 0x7fb9e81f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.239+0000 7fb9ee600640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9c006c5e0 0x7fb9c006eaa0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.239+0000 7fb9ee600640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e8143ae0 0x7fb9e815a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.239+0000 7fb9ee600640 1 --2- 172.21.15.26:0/2117277976 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e80cf7d0 0x7fb9e81f2f20 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.239+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 >> 172.21.15.26:0/2117277976 conn(0x7fb9e806a800 msgr2=0x7fb9e80afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:25.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.239+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 shutdown_connections 2024-01-15T18:13:25.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.239+0000 7fb9ee600640 1 -- 172.21.15.26:0/2117277976 wait complete. 2024-01-15T18:13:25.265 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone1111 2024-01-15T18:13:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.488+0000 7f3c91600640 1 Processor -- start 2024-01-15T18:13:25.491 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.488+0000 7f3c91600640 1 -- start start 2024-01-15T18:13:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.490+0000 7f3c91600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 0x7f3c8c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.490+0000 7f3c91600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3c8c13e360 0x7f3c8c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.490+0000 7f3c91600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8c06c470 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.490+0000 7f3c91600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c8c06c750 con 0x7f3c8c13e360 2024-01-15T18:13:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.490+0000 7f3c91600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c06ca30 con 0x7f3c8c13df80 2024-01-15T18:13:25.493 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.491+0000 7f3c8a000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3c8c13e360 0x7f3c8c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.491+0000 7f3c8aa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 0x7f3c8c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.491+0000 7f3c8a000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f3c8c13e360 0x7f3c8c0cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:33086/0 (socket says 172.21.15.26:33086) 2024-01-15T18:13:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.491+0000 7f3c8a000640 1 -- 172.21.15.26:0/1539671815 learned_addr learned my addr 172.21.15.26:0/1539671815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.491+0000 7f3c89600640 1 -- 172.21.15.26:0/1539671815 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173972884 0 0) 0x7f3c8c06c470 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.491+0000 7f3c89600640 1 -- 172.21.15.26:0/1539671815 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c6c003520 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.494 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.492+0000 7f3c8a000640 1 -- 172.21.15.26:0/1539671815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 msgr2=0x7f3c8c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.492+0000 7f3c8a000640 1 --2- 172.21.15.26:0/1539671815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 0x7f3c8c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.492+0000 7f3c8a000640 1 -- 172.21.15.26:0/1539671815 >> v1:172.21.15.124:6789/0 conn(0x7f3c8c15f9c0 legacy=0x7f3c8c1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:25.495 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.492+0000 7f3c8a000640 1 -- 172.21.15.26:0/1539671815 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8c06cd10 con 0x7f3c8c13e360 2024-01-15T18:13:25.496 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.493+0000 7f3c8a000640 1 --2- 172.21.15.26:0/1539671815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c8c13e360 0x7f3c8c0cf290 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009870 tx=0x7f3c7402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fe37f4c81e84a28 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.494+0000 7f3c89600640 1 -- 172.21.15.26:0/1539671815 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74030840 con 0x7f3c8c13e360 2024-01-15T18:13:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.494+0000 7f3c89600640 1 -- 172.21.15.26:0/1539671815 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c740309a0 con 0x7f3c8c13e360 2024-01-15T18:13:25.497 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.494+0000 7f3c89600640 1 -- 172.21.15.26:0/1539671815 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74030d30 con 0x7f3c8c13e360 2024-01-15T18:13:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.495+0000 7f3c91600640 1 -- 172.21.15.26:0/1539671815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c8c13e360 msgr2=0x7f3c8c0cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.495+0000 7f3c91600640 1 --2- 172.21.15.26:0/1539671815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c8c13e360 0x7f3c8c0cf290 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009870 tx=0x7f3c7402b980 comp rx=0 tx=0).stop 2024-01-15T18:13:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.495+0000 7f3c91600640 1 -- 172.21.15.26:0/1539671815 shutdown_connections 2024-01-15T18:13:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.495+0000 7f3c91600640 1 --2- 172.21.15.26:0/1539671815 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c8c13e360 0x7f3c8c0cf290 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.495+0000 7f3c91600640 1 --2- 172.21.15.26:0/1539671815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 0x7f3c8c15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.495+0000 7f3c91600640 1 -- 172.21.15.26:0/1539671815 >> 172.21.15.26:0/1539671815 conn(0x7f3c8c06a800 msgr2=0x7f3c8c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.495+0000 7f3c91600640 1 -- 172.21.15.26:0/1539671815 shutdown_connections 2024-01-15T18:13:25.498 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.495+0000 7f3c91600640 1 -- 172.21.15.26:0/1539671815 wait complete. 2024-01-15T18:13:25.499 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.497+0000 7f3c91600640 1 Processor -- start 2024-01-15T18:13:25.500 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.497+0000 7f3c91600640 1 -- start start 2024-01-15T18:13:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c91600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 0x7f3c8c0add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c91600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c8c13e360 0x7f3c8c153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c91600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c8c15f9c0 0x7f3c8c15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c91600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c0ae290 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c91600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c1322d0 con 0x7f3c8c13e360 2024-01-15T18:13:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c91600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8c131750 con 0x7f3c8c13df80 2024-01-15T18:13:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c8b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c8c15f9c0 0x7f3c8c15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c8b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c8c15f9c0 0x7f3c8c15d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42904/0 (socket says 172.21.15.26:42904) 2024-01-15T18:13:25.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c8b400640 1 -- 172.21.15.26:0/1102962132 learned_addr learned my addr 172.21.15.26:0/1102962132 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:25.502 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.498+0000 7f3c8aa00640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 0x7f3c8c0add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c8b400640 1 -- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 msgr2=0x7f3c8c0add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c8b400640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 0x7f3c8c0add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c8b400640 1 -- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c8c13e360 msgr2=0x7f3c8c153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c8b400640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c8c13e360 0x7f3c8c153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c8b400640 1 -- 172.21.15.26:0/1102962132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c740094d0 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c8b400640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c8c15f9c0 0x7f3c8c15d8a0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400c400 tx=0x7f3c8400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c82a00640 1 -- 172.21.15.26:0/1102962132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c840052d0 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c82a00640 1 -- 172.21.15.26:0/1102962132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c84005430 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8c131a30 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c82a00640 1 -- 172.21.15.26:0/1102962132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c840037f0 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.503 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.499+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8c131ff0 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.501+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8c0675d0 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.501+0000 7f3c82a00640 1 -- 172.21.15.26:0/1102962132 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 40) v1 ==== 91273+0+0 (secure 0 0 0) 0x7f3c84003950 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.502+0000 7f3c82a00640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6406c5e0 0x7f3c6406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:25.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.502+0000 7f3c82a00640 1 -- 172.21.15.26:0/1102962132 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f3c840880e0 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.503+0000 7f3c8aa00640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6406c5e0 0x7f3c6406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:25.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.504+0000 7f3c8aa00640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6406c5e0 0x7f3c6406eaa0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0073f0 tx=0x7f3c7c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:25.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.507+0000 7f3c82a00640 1 -- 172.21.15.26:0/1102962132 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c840546b0 con 0x7f3c8c15f9c0 2024-01-15T18:13:25.742 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:25.739+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone1111", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c8c0ce850 con 0x7f3c6406c5e0 2024-01-15T18:13:30.244 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:30.240+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:30.244 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:30.240+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:30.244 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:30.240+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:30.244 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:30.240+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:30.244 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:13:30.240+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:13:30.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.454+0000 7f3c82a00640 1 -- 172.21.15.26:0/1102962132 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3c7c00f050 con 0x7f3c6406c5e0 2024-01-15T18:13:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.458+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6406c5e0 msgr2=0x7f3c6406eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.458+0000 7f3c91600640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6406c5e0 0x7f3c6406eaa0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0073f0 tx=0x7f3c7c00b040 comp rx=0 tx=0).stop 2024-01-15T18:13:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.458+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c8c15f9c0 msgr2=0x7f3c8c15d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.458+0000 7f3c91600640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c8c15f9c0 0x7f3c8c15d8a0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400c400 tx=0x7f3c8400c8c0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.458+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 shutdown_connections 2024-01-15T18:13:30.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.458+0000 7f3c91600640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f3c6406c5e0 0x7f3c6406eaa0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.458+0000 7f3c91600640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f3c8c15f9c0 0x7f3c8c15d8a0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.458+0000 7f3c91600640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f3c8c13e360 0x7f3c8c153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.459+0000 7f3c91600640 1 --2- 172.21.15.26:0/1102962132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f3c8c13df80 0x7f3c8c0add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.459+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 >> 172.21.15.26:0/1102962132 conn(0x7f3c8c06a800 msgr2=0x7f3c8c15abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:30.463 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.459+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 shutdown_connections 2024-01-15T18:13:30.464 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.459+0000 7f3c91600640 1 -- 172.21.15.26:0/1102962132 wait complete. 2024-01-15T18:13:30.485 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone1112 2024-01-15T18:13:30.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.713+0000 7fba0fe00640 1 Processor -- start 2024-01-15T18:13:30.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.713+0000 7fba0fe00640 1 -- start start 2024-01-15T18:13:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.714+0000 7fba0fe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fba100adf20 0x7fba10070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.714+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 0x7fba100cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:30.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.714+0000 7fba0fe00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba100cc980 0x7fba10143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.714+0000 7fba0fe00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1015f970 con 0x7fba100cc980 2024-01-15T18:13:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.715+0000 7fba0fe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1015f4e0 con 0x7fba100adf20 2024-01-15T18:13:30.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.715+0000 7fba0fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba100cce10 con 0x7fba1015bf10 2024-01-15T18:13:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.715+0000 7fba0f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba100cc980 0x7fba10143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.715+0000 7fba0f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fba100cc980 0x7fba10143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42916/0 (socket says 172.21.15.26:42916) 2024-01-15T18:13:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.715+0000 7fba0f400640 1 -- 172.21.15.26:0/2918016270 learned_addr learned my addr 172.21.15.26:0/2918016270 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.715+0000 7fba07e00640 1 --2- 172.21.15.26:0/2918016270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 0x7fba100cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.715+0000 7fba0ea00640 1 --2- 172.21.15.26:0/2918016270 >> v2:172.21.15.124:3300/0 conn(0x7fba100adf20 0x7fba10070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.716+0000 7fba0f400640 1 -- 172.21.15.26:0/2918016270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 msgr2=0x7fba100cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.716+0000 7fba0f400640 1 --2- 172.21.15.26:0/2918016270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 0x7fba100cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.716+0000 7fba0f400640 1 -- 172.21.15.26:0/2918016270 >> v2:172.21.15.124:3300/0 conn(0x7fba100adf20 msgr2=0x7fba10070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:13:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.716+0000 7fba0f400640 1 --2- 172.21.15.26:0/2918016270 >> v2:172.21.15.124:3300/0 conn(0x7fba100adf20 0x7fba10070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.716+0000 7fba0f400640 1 -- 172.21.15.26:0/2918016270 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10071130 con 0x7fba100cc980 2024-01-15T18:13:30.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.716+0000 7fba07e00640 1 --2- 172.21.15.26:0/2918016270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 0x7fba100cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.717+0000 7fba0f400640 1 --2- 172.21.15.26:0/2918016270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba100cc980 0x7fba10143d30 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fba100b4150 tx=0x7fba0802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da050581975e9b8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.717+0000 7fba0e000640 1 -- 172.21.15.26:0/2918016270 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0802c9e0 con 0x7fba100cc980 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.717+0000 7fba0e000640 1 -- 172.21.15.26:0/2918016270 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0802cb40 con 0x7fba100cc980 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.717+0000 7fba0e000640 1 -- 172.21.15.26:0/2918016270 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba080343f0 con 0x7fba100cc980 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.718+0000 7fba0fe00640 1 -- 172.21.15.26:0/2918016270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba100cc980 msgr2=0x7fba10143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.718+0000 7fba0fe00640 1 --2- 172.21.15.26:0/2918016270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba100cc980 0x7fba10143d30 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fba100b4150 tx=0x7fba0802be40 comp rx=0 tx=0).stop 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.718+0000 7fba0fe00640 1 -- 172.21.15.26:0/2918016270 shutdown_connections 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.718+0000 7fba0fe00640 1 --2- 172.21.15.26:0/2918016270 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba100cc980 0x7fba10143d30 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.718+0000 7fba0fe00640 1 --2- 172.21.15.26:0/2918016270 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 0x7fba100cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.718+0000 7fba0fe00640 1 --2- 172.21.15.26:0/2918016270 >> v2:172.21.15.124:3300/0 conn(0x7fba100adf20 0x7fba10070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.718+0000 7fba0fe00640 1 -- 172.21.15.26:0/2918016270 >> 172.21.15.26:0/2918016270 conn(0x7fba1006a7d0 msgr2=0x7fba1006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:13:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.718+0000 7fba0fe00640 1 -- 172.21.15.26:0/2918016270 shutdown_connections 2024-01-15T18:13:30.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.718+0000 7fba0fe00640 1 -- 172.21.15.26:0/2918016270 wait complete. 2024-01-15T18:13:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.720+0000 7fba0fe00640 1 Processor -- start 2024-01-15T18:13:30.724 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.720+0000 7fba0fe00640 1 -- start start 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.721+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba100adf20 0x7fba10167260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.721+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba100cc980 0x7fba10153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.721+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 0x7fba10153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.721+0000 7fba0fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba100ab690 con 0x7fba100cc980 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.721+0000 7fba0fe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1013c8d0 con 0x7fba100adf20 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.721+0000 7fba0fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1013cbb0 con 0x7fba1015bf10 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.721+0000 7fba07e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba100cc980 0x7fba10153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.721+0000 7fba07e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba100cc980 0x7fba10153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:42928/0 (socket says 172.21.15.26:42928) 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.721+0000 7fba07e00640 1 -- 172.21.15.26:0/1233647355 learned_addr learned my addr 172.21.15.26:0/1233647355 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba0f400640 1 --2- 172.21.15.26:0/1233647355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 0x7fba10153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba07e00640 1 -- 172.21.15.26:0/1233647355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 msgr2=0x7fba10153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba07e00640 1 --2- 172.21.15.26:0/1233647355 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fba1015bf10 0x7fba10153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.726 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba07e00640 1 -- 172.21.15.26:0/1233647355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba100adf20 msgr2=0x7fba10167260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:13:30.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba07e00640 1 --2- 172.21.15.26:0/1233647355 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba100adf20 0x7fba10167260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:13:30.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba07e00640 1 -- 172.21.15.26:0/1233647355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba08009b30 con 0x7fba100cc980 2024-01-15T18:13:30.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba07e00640 1 --2- 172.21.15.26:0/1233647355 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba100cc980 0x7fba10153050 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800e7e0 tx=0x7fb9f800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:30.727 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba06a00640 1 -- 172.21.15.26:0/1233647355 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8004030 con 0x7fba100cc980 2024-01-15T18:13:30.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba06a00640 1 -- 172.21.15.26:0/1233647355 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8004190 con 0x7fba100cc980 2024-01-15T18:13:30.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba06a00640 1 -- 172.21.15.26:0/1233647355 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8004520 con 0x7fba100cc980 2024-01-15T18:13:30.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.722+0000 7fba0fe00640 1 -- 172.21.15.26:0/1233647355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1013ce90 con 0x7fba100cc980 2024-01-15T18:13:30.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.723+0000 7fba0fe00640 1 -- 172.21.15.26:0/1233647355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba101677a0 con 0x7fba100cc980 2024-01-15T18:13:30.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.724+0000 7fba0fe00640 1 -- 172.21.15.26:0/1233647355 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba100ab4f0 con 0x7fba100cc980 2024-01-15T18:13:30.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.725+0000 7fba06a00640 1 -- 172.21.15.26:0/1233647355 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 40) v1 ==== 91273+0+0 (secure 0 0 0) 0x7fb9f800c6d0 con 0x7fba100cc980 2024-01-15T18:13:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.726+0000 7fba06a00640 1 --2- 172.21.15.26:0/1233647355 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9e806c5e0 0x7fb9e806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:13:30.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.726+0000 7fba06a00640 1 -- 172.21.15.26:0/1233647355 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fb9f80885a0 con 0x7fba100cc980 2024-01-15T18:13:30.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.726+0000 7fba0ea00640 1 --2- 172.21.15.26:0/1233647355 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9e806c5e0 0x7fb9e806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:13:30.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.727+0000 7fba0ea00640 1 --2- 172.21.15.26:0/1233647355 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9e806c5e0 0x7fb9e806eaa0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b7f0 tx=0x7fba00008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:13:30.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.730+0000 7fba06a00640 1 -- 172.21.15.26:0/1233647355 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba100ab4f0 con 0x7fba100cc980 2024-01-15T18:13:30.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:30.966+0000 7fba0fe00640 1 -- 172.21.15.26:0/1233647355 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs clone cancel", "vol_name": "cephfs", "clone_name": "subvol_clone1112", "target": ["mon-mgr", ""]}) v1 -- 0x7fba1013cd20 con 0x7fb9e806c5e0 2024-01-15T18:13:31.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:13:31.323+0000 7fba06a00640 1 -- 172.21.15.26:0/1233647355 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mgrmap(e 41) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fb9f8055230 con 0x7fba100cc980 2024-01-15T18:13:31.867 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:13:31.875 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:14:02.029 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:14:02.038 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:14:32.200 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:14:32.209 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:14:53.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:14:53.459+0000 7fba06a00640 1 -- 172.21.15.26:0/1233647355 <== mon.0 v2:172.21.15.26:3300/0 8 ==== mgrmap(e 42) v1 ==== 91273+0+0 (secure 0 0 0) 0x7fb9f8003d40 con 0x7fba100cc980 2024-01-15T18:15:02.368 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:15:02.376 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:15:30.609 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-01-15T18:15:30.610 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-01-15T18:15:30.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.825+0000 7f54bca00640 1 Processor -- start 2024-01-15T18:15:30.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.826+0000 7f54bca00640 1 -- start start 2024-01-15T18:15:30.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.827+0000 7f54bca00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f54b8143c40 0x7f54b8071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.827+0000 7f54bca00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b8070e50 con 0x7f54b80ccc10 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.827+0000 7f54bca00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54b8144020 con 0x7f54b80717d0 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.827+0000 7f54bca00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54b80723c0 con 0x7f54b8143c40 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.828+0000 7f54b6800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f54b8143c40 0x7f54b8071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.828+0000 7f54b6800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f54b8143c40 0x7f54b8071290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50650/0 (socket says 172.21.15.26:50650) 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.828+0000 7f54b6800640 1 -- 172.21.15.26:0/1966163939 learned_addr learned my addr 172.21.15.26:0/1966163939 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.828+0000 7f54b6800640 1 -- 172.21.15.26:0/1966163939 >> v1:172.21.15.124:6789/0 conn(0x7f54b80717d0 legacy=0x7f54b80cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.828+0000 7f54b6800640 1 -- 172.21.15.26:0/1966163939 >> v1:172.21.15.26:6789/0 conn(0x7f54b80ccc10 legacy=0x7f54b806d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.828+0000 7f54b6800640 1 -- 172.21.15.26:0/1966163939 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b8071e00 con 0x7f54b8143c40 2024-01-15T18:15:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.829+0000 7f54b6800640 1 --2- 172.21.15.26:0/1966163939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b8143c40 0x7f54b8071290 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009870 tx=0x7f54a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a24cd3d63cc85b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.830+0000 7f54b5400640 1 -- 172.21.15.26:0/1966163939 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8031870 con 0x7f54b8143c40 2024-01-15T18:15:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.830+0000 7f54b5400640 1 -- 172.21.15.26:0/1966163939 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a80319d0 con 0x7f54b8143c40 2024-01-15T18:15:30.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.830+0000 7f54b5400640 1 -- 172.21.15.26:0/1966163939 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8031d60 con 0x7f54b8143c40 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.830+0000 7f54bca00640 1 -- 172.21.15.26:0/1966163939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b8143c40 msgr2=0x7f54b8071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.830+0000 7f54bca00640 1 --2- 172.21.15.26:0/1966163939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b8143c40 0x7f54b8071290 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009870 tx=0x7f54a802b910 comp rx=0 tx=0).stop 2024-01-15T18:15:30.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.831+0000 7f54bca00640 1 -- 172.21.15.26:0/1966163939 shutdown_connections 2024-01-15T18:15:30.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.831+0000 7f54bca00640 1 --2- 172.21.15.26:0/1966163939 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b8143c40 0x7f54b8071290 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.831+0000 7f54bca00640 1 -- 172.21.15.26:0/1966163939 >> 172.21.15.26:0/1966163939 conn(0x7f54b806a800 msgr2=0x7f54b806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:30.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.831+0000 7f54bca00640 1 -- 172.21.15.26:0/1966163939 shutdown_connections 2024-01-15T18:15:30.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.831+0000 7f54bca00640 1 -- 172.21.15.26:0/1966163939 wait complete. 2024-01-15T18:15:30.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.833+0000 7f54bca00640 1 Processor -- start 2024-01-15T18:15:30.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.833+0000 7f54bca00640 1 -- start start 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.834+0000 7f54bca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b80717d0 0x7f54b80c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.834+0000 7f54bca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54b80ccc10 0x7f54b80b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.834+0000 7f54bca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54b8143c40 0x7f54b8130e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.834+0000 7f54bca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b80c4780 con 0x7f54b80717d0 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.834+0000 7f54bca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54b80c4a60 con 0x7f54b80ccc10 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.834+0000 7f54bca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b80c4d40 con 0x7f54b8143c40 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.834+0000 7f54b6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b80717d0 0x7f54b80c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.834+0000 7f54b6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b80717d0 0x7f54b80c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50666/0 (socket says 172.21.15.26:50666) 2024-01-15T18:15:30.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.834+0000 7f54b6800640 1 -- 172.21.15.26:0/225786465 learned_addr learned my addr 172.21.15.26:0/225786465 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:30.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54b5e00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54b80ccc10 0x7f54b80b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54b6800640 1 -- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54b8143c40 msgr2=0x7f54b8130e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54b6800640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54b8143c40 0x7f54b8130e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54b6800640 1 -- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54b80ccc10 msgr2=0x7f54b80b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54b6800640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54b80ccc10 0x7f54b80b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54b6800640 1 -- 172.21.15.26:0/225786465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a80094d0 con 0x7f54b80717d0 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54b5e00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54b80ccc10 0x7f54b80b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54b6800640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b80717d0 0x7f54b80c5820 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009870 tx=0x7f54a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54aea00640 1 -- 172.21.15.26:0/225786465 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a8031820 con 0x7f54b80717d0 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54aea00640 1 -- 172.21.15.26:0/225786465 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54a80443f0 con 0x7f54b80717d0 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.835+0000 7f54aea00640 1 -- 172.21.15.26:0/225786465 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54a80447a0 con 0x7f54b80717d0 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.836+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b80c5020 con 0x7f54b80717d0 2024-01-15T18:15:30.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.836+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b80c5d60 con 0x7f54b80717d0 2024-01-15T18:15:30.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.837+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54b80c5540 con 0x7f54b80717d0 2024-01-15T18:15:30.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.838+0000 7f54aea00640 1 -- 172.21.15.26:0/225786465 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 42) v1 ==== 91273+0+0 (secure 0 0 0) 0x7f54a804d450 con 0x7f54b80717d0 2024-01-15T18:15:30.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.838+0000 7f54aea00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f549006c610 0x7f549006ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:30.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.839+0000 7f54aea00640 1 -- 172.21.15.26:0/225786465 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f54a802d040 con 0x7f54b80717d0 2024-01-15T18:15:30.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.839+0000 7f54b5e00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f549006c610 0x7f549006ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:30.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.840+0000 7f54b5e00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f549006c610 0x7f549006ead0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f54b81f6380 tx=0x7f54a0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:30.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:30.843+0000 7f54aea00640 1 -- 172.21.15.26:0/225786465 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54a80e1e90 con 0x7f54b80717d0 2024-01-15T18:15:31.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.073+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f54b806cc40 con 0x7f549006c610 2024-01-15T18:15:31.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.076+0000 7f54aea00640 1 -- 172.21.15.26:0/225786465 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f54a000ad80 con 0x7f549006c610 2024-01-15T18:15:31.079 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:15:31.079 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:15:31.079 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "cephfs" 2024-01-15T18:15:31.079 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:15:31.080 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f549006c610 msgr2=0x7f549006ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f549006c610 0x7f549006ead0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f54b81f6380 tx=0x7f54a0008040 comp rx=0 tx=0).stop 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b80717d0 msgr2=0x7f54b80c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b80717d0 0x7f54b80c5820 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009870 tx=0x7f54a80046c0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 shutdown_connections 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f54b8143c40 0x7f54b8130e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f549006c610 0x7f549006ead0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f54b80ccc10 0x7f54b80b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 --2- 172.21.15.26:0/225786465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f54b80717d0 0x7f54b80c5820 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 >> 172.21.15.26:0/225786465 conn(0x7f54b806a800 msgr2=0x7f54b8142c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 shutdown_connections 2024-01-15T18:15:31.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.079+0000 7f54bca00640 1 -- 172.21.15.26:0/225786465 wait complete. 2024-01-15T18:15:31.100 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs --format json 2024-01-15T18:15:31.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.314+0000 7f82ee800640 1 Processor -- start 2024-01-15T18:15:31.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.314+0000 7f82ee800640 1 -- start start 2024-01-15T18:15:31.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.316+0000 7f82ee800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82e80adf20 0x7f82e8070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.316+0000 7f82ee800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e815bf10 0x7f82e80cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.316+0000 7f82ee800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f82e80cc980 0x7f82e8143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.316+0000 7f82ee800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e815f970 con 0x7f82e80adf20 2024-01-15T18:15:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.316+0000 7f82ee800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e815f4e0 con 0x7f82e80cc980 2024-01-15T18:15:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.316+0000 7f82ee800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80cce10 con 0x7f82e815bf10 2024-01-15T18:15:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.316+0000 7f82ed400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82e80adf20 0x7f82e8070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.316+0000 7f82ed400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f82e80adf20 0x7f82e8070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50670/0 (socket says 172.21.15.26:50670) 2024-01-15T18:15:31.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.316+0000 7f82ed400640 1 -- 172.21.15.26:0/3358045086 learned_addr learned my addr 172.21.15.26:0/3358045086 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.317+0000 7f82eca00640 1 --2- 172.21.15.26:0/3358045086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e815bf10 0x7f82e80cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.317+0000 7f82ede00640 1 --2- 172.21.15.26:0/3358045086 >> v2:172.21.15.124:3300/0 conn(0x7f82e80cc980 0x7f82e8143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.317+0000 7f82ed400640 1 -- 172.21.15.26:0/3358045086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e815bf10 msgr2=0x7f82e80cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.317+0000 7f82ed400640 1 --2- 172.21.15.26:0/3358045086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e815bf10 0x7f82e80cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.317+0000 7f82ed400640 1 -- 172.21.15.26:0/3358045086 >> v2:172.21.15.124:3300/0 conn(0x7f82e80cc980 msgr2=0x7f82e8143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.317+0000 7f82ed400640 1 --2- 172.21.15.26:0/3358045086 >> v2:172.21.15.124:3300/0 conn(0x7f82e80cc980 0x7f82e8143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.317+0000 7f82ed400640 1 -- 172.21.15.26:0/3358045086 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8071130 con 0x7f82e80adf20 2024-01-15T18:15:31.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.317+0000 7f82eca00640 1 --2- 172.21.15.26:0/3358045086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e815bf10 0x7f82e80cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.318+0000 7f82ed400640 1 --2- 172.21.15.26:0/3358045086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 0x7f82e8070a30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f82e806e490 tx=0x7f82d802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1596ec3078a9079 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.318+0000 7f82e7e00640 1 -- 172.21.15.26:0/3358045086 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d80317d0 con 0x7f82e80adf20 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.318+0000 7f82e7e00640 1 -- 172.21.15.26:0/3358045086 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d8031930 con 0x7f82e80adf20 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.318+0000 7f82e7e00640 1 -- 172.21.15.26:0/3358045086 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8031cc0 con 0x7f82e80adf20 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.319+0000 7f82ee800640 1 -- 172.21.15.26:0/3358045086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 msgr2=0x7f82e8070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.319+0000 7f82ee800640 1 --2- 172.21.15.26:0/3358045086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 0x7f82e8070a30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f82e806e490 tx=0x7f82d802b870 comp rx=0 tx=0).stop 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.319+0000 7f82ee800640 1 -- 172.21.15.26:0/3358045086 shutdown_connections 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.319+0000 7f82ee800640 1 --2- 172.21.15.26:0/3358045086 >> v2:172.21.15.124:3300/0 conn(0x7f82e80cc980 0x7f82e8143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.319+0000 7f82ee800640 1 --2- 172.21.15.26:0/3358045086 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e815bf10 0x7f82e80cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.319+0000 7f82ee800640 1 --2- 172.21.15.26:0/3358045086 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 0x7f82e8070a30 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.319+0000 7f82ee800640 1 -- 172.21.15.26:0/3358045086 >> 172.21.15.26:0/3358045086 conn(0x7f82e806a7d0 msgr2=0x7f82e806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.319+0000 7f82ee800640 1 -- 172.21.15.26:0/3358045086 shutdown_connections 2024-01-15T18:15:31.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.319+0000 7f82ee800640 1 -- 172.21.15.26:0/3358045086 wait complete. 2024-01-15T18:15:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.321+0000 7f82ee800640 1 Processor -- start 2024-01-15T18:15:31.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.321+0000 7f82ee800640 1 -- start start 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.322+0000 7f82ee800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 0x7f82e80b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.322+0000 7f82ee800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e80cc980 0x7f82e813ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.322+0000 7f82ee800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82e815bf10 0x7f82e81f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.322+0000 7f82ee800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e813f4c0 con 0x7f82e80adf20 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.322+0000 7f82ee800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e8140d20 con 0x7f82e815bf10 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.322+0000 7f82ee800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b2ef0 con 0x7f82e80cc980 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82ed400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 0x7f82e80b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82ed400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 0x7f82e80b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50686/0 (socket says 172.21.15.26:50686) 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82ed400640 1 -- 172.21.15.26:0/3428296250 learned_addr learned my addr 172.21.15.26:0/3428296250 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82eca00640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e80cc980 0x7f82e813ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82ed400640 1 -- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e80cc980 msgr2=0x7f82e813ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82ed400640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e80cc980 0x7f82e813ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82ed400640 1 -- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82e815bf10 msgr2=0x7f82e81f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82ede00640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82e815bf10 0x7f82e81f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82ed400640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82e815bf10 0x7f82e81f2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.323+0000 7f82ed400640 1 -- 172.21.15.26:0/3428296250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80094d0 con 0x7f82e80adf20 2024-01-15T18:15:31.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.324+0000 7f82ed400640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 0x7f82e80b28a0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f82d8000c00 tx=0x7f82d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.324+0000 7f82e5600640 1 -- 172.21.15.26:0/3428296250 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8032870 con 0x7f82e80adf20 2024-01-15T18:15:31.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.324+0000 7f82e5600640 1 -- 172.21.15.26:0/3428296250 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d80329d0 con 0x7f82e80adf20 2024-01-15T18:15:31.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.324+0000 7f82e5600640 1 -- 172.21.15.26:0/3428296250 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8032d60 con 0x7f82e80adf20 2024-01-15T18:15:31.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.324+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e80b31d0 con 0x7f82e80adf20 2024-01-15T18:15:31.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.324+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e80b3790 con 0x7f82e80adf20 2024-01-15T18:15:31.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.325+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e80b3980 con 0x7f82e80adf20 2024-01-15T18:15:31.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.326+0000 7f82e5600640 1 -- 172.21.15.26:0/3428296250 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 42) v1 ==== 91273+0+0 (secure 0 0 0) 0x7f82d8030070 con 0x7f82e80adf20 2024-01-15T18:15:31.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.327+0000 7f82e5600640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f82c006c5e0 0x7f82c006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.327+0000 7f82e5600640 1 -- 172.21.15.26:0/3428296250 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f82d80aab10 con 0x7f82e80adf20 2024-01-15T18:15:31.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.328+0000 7f82eca00640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f82c006c5e0 0x7f82c006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.329+0000 7f82eca00640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f82c006c5e0 0x7f82c006eaa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f82e8140b10 tx=0x7f82d0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.332+0000 7f82e5600640 1 -- 172.21.15.26:0/3428296250 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82d80770e0 con 0x7f82e80adf20 2024-01-15T18:15:31.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.553+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82e815ee90 con 0x7f82c006c5e0 2024-01-15T18:15:31.566 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:31.563+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:31.566 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:31.563+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:31.566 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:31.563+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:31.566 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:31.563+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:31.566 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:31.563+0000 7f8f54600640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:15:31.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.589+0000 7f82e5600640 1 -- 172.21.15.26:0/3428296250 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+182 (secure 0 0 0) 0x7f82d000de60 con 0x7f82c006c5e0 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout:[ 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone1113" 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "sv1" 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone1111" 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: }, 2024-01-15T18:15:31.592 INFO:teuthology.orchestra.run.smithi026.stdout: { 2024-01-15T18:15:31.593 INFO:teuthology.orchestra.run.smithi026.stdout: "name": "subvol_clone1112" 2024-01-15T18:15:31.593 INFO:teuthology.orchestra.run.smithi026.stdout: } 2024-01-15T18:15:31.593 INFO:teuthology.orchestra.run.smithi026.stdout:] 2024-01-15T18:15:31.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f82c006c5e0 msgr2=0x7f82c006eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f82c006c5e0 0x7f82c006eaa0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f82e8140b10 tx=0x7f82d0008040 comp rx=0 tx=0).stop 2024-01-15T18:15:31.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 msgr2=0x7f82e80b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 0x7f82e80b28a0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f82d8000c00 tx=0x7f82d80046c0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 shutdown_connections 2024-01-15T18:15:31.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f82e80cc980 0x7f82e813ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f82c006c5e0 0x7f82c006eaa0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f82e815bf10 0x7f82e81f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 --2- 172.21.15.26:0/3428296250 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f82e80adf20 0x7f82e80b28a0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 >> 172.21.15.26:0/3428296250 conn(0x7f82e806a7d0 msgr2=0x7f82e8142c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 shutdown_connections 2024-01-15T18:15:31.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.592+0000 7f82ee800640 1 -- 172.21.15.26:0/3428296250 wait complete. 2024-01-15T18:15:31.615 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.842+0000 7fd778c00640 1 Processor -- start 2024-01-15T18:15:31.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.842+0000 7fd778c00640 1 -- start start 2024-01-15T18:15:31.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.844+0000 7fd778c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd774162430 0x7fd7740b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.844+0000 7fd778c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7740702e0 con 0x7fd774162810 2024-01-15T18:15:31.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.844+0000 7fd778c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7740705c0 con 0x7fd7740b3650 2024-01-15T18:15:31.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.844+0000 7fd778c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740708a0 con 0x7fd774162430 2024-01-15T18:15:31.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.845+0000 7fd76be00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd774162810 0x7fd774071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:59260/0 (socket says 172.21.15.26:59260) 2024-01-15T18:15:31.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.845+0000 7fd76be00640 1 -- 172.21.15.26:0/2131552372 learned_addr learned my addr 172.21.15.26:0/2131552372 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:15:31.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.845+0000 7fd773400640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd774162430 0x7fd7740b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.845+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189935382 0 0) 0x7fd7740702e0 con 0x7fd774162810 2024-01-15T18:15:31.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.845+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd754003520 con 0x7fd774162810 2024-01-15T18:15:31.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2397866915 0 0) 0x7fd754003520 con 0x7fd774162810 2024-01-15T18:15:31.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd774162430 msgr2=0x7fd7740b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd774162430 0x7fd7740b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 >> v1:172.21.15.124:6789/0 conn(0x7fd7740b3650 legacy=0x7fd77406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774070b80 con 0x7fd774162810 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd75c001670 con 0x7fd774162810 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 >> v1:172.21.15.26:6789/0 conn(0x7fd774162810 legacy=0x7fd774071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd774162430 0x7fd7540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd754003f20 0x7fd7540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd754004d00 0x7fd754005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740708a0 con 0x7fd754004d00 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75400ba60 con 0x7fd754003f20 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75400bc20 con 0x7fd774162430 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd773e00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd754004d00 0x7fd754005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd773400640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd774162430 0x7fd7540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.846+0000 7fd76be00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd754003f20 0x7fd7540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.847+0000 7fd773e00640 1 -- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd774162430 msgr2=0x7fd7540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.847+0000 7fd773e00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd774162430 0x7fd7540039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.847+0000 7fd773e00640 1 -- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd754003f20 msgr2=0x7fd7540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.847+0000 7fd773e00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd754003f20 0x7fd7540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.847+0000 7fd773e00640 1 -- 172.21.15.26:0/2131552372 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd754003520 con 0x7fd754004d00 2024-01-15T18:15:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.848+0000 7fd773e00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd754004d00 0x7fd754005180 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fd76c001e50 tx=0x7fd7540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9881adfef18830b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.848+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c003b70 con 0x7fd754004d00 2024-01-15T18:15:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.848+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c003cd0 con 0x7fd754004d00 2024-01-15T18:15:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.848+0000 7fd772a00640 1 -- 172.21.15.26:0/2131552372 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c005310 con 0x7fd754004d00 2024-01-15T18:15:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.848+0000 7fd778c00640 1 -- 172.21.15.26:0/2131552372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd754004d00 msgr2=0x7fd754005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.848+0000 7fd778c00640 1 --2- 172.21.15.26:0/2131552372 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd754004d00 0x7fd754005180 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fd76c001e50 tx=0x7fd7540057f0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.848+0000 7fd773e00640 1 -- 172.21.15.26:0/2131552372 reap_dead start 2024-01-15T18:15:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.849+0000 7fd778c00640 1 -- 172.21.15.26:0/2131552372 shutdown_connections 2024-01-15T18:15:31.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.849+0000 7fd778c00640 1 -- 172.21.15.26:0/2131552372 >> 172.21.15.26:0/2131552372 conn(0x7fd77406a800 msgr2=0x7fd77406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:31.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.849+0000 7fd778c00640 1 -- 172.21.15.26:0/2131552372 shutdown_connections 2024-01-15T18:15:31.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.849+0000 7fd778c00640 1 -- 172.21.15.26:0/2131552372 wait complete. 2024-01-15T18:15:31.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.851+0000 7fd778c00640 1 Processor -- start 2024-01-15T18:15:31.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.851+0000 7fd778c00640 1 -- start start 2024-01-15T18:15:31.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd778c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd754004d00 0x7fd7741e7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd778c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77415b570 0x7fd774142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd778c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd774143500 0x7fd7741539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd778c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd774157060 con 0x7fd77415b570 2024-01-15T18:15:31.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd778c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd774143980 con 0x7fd774143500 2024-01-15T18:15:31.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd778c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd774156450 con 0x7fd754004d00 2024-01-15T18:15:31.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd76be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77415b570 0x7fd774142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd76be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77415b570 0x7fd774142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50712/0 (socket says 172.21.15.26:50712) 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd76be00640 1 -- 172.21.15.26:0/3192379389 learned_addr learned my addr 172.21.15.26:0/3192379389 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd773400640 1 --2- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd754004d00 0x7fd7741e7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd773e00640 1 --2- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd774143500 0x7fd7741539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd76be00640 1 -- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd754004d00 msgr2=0x7fd7741e7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd76be00640 1 --2- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd754004d00 0x7fd7741e7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd76be00640 1 -- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd774143500 msgr2=0x7fd7741539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd76be00640 1 --2- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd774143500 0x7fd7741539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd76be00640 1 -- 172.21.15.26:0/3192379389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774156730 con 0x7fd77415b570 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.852+0000 7fd773e00640 1 --2- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd774143500 0x7fd7741539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:31.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.853+0000 7fd76be00640 1 --2- 172.21.15.26:0/3192379389 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd77415b570 0x7fd774142fc0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00e7f0 tx=0x7fd75c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.853+0000 7fd76b400640 1 -- 172.21.15.26:0/3192379389 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c00c8d0 con 0x7fd77415b570 2024-01-15T18:15:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.853+0000 7fd76b400640 1 -- 172.21.15.26:0/3192379389 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c00ca30 con 0x7fd77415b570 2024-01-15T18:15:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.853+0000 7fd76b400640 1 -- 172.21.15.26:0/3192379389 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c00cdc0 con 0x7fd77415b570 2024-01-15T18:15:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.853+0000 7fd778c00640 1 -- 172.21.15.26:0/3192379389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd774156a10 con 0x7fd77415b570 2024-01-15T18:15:31.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.853+0000 7fd778c00640 1 -- 172.21.15.26:0/3192379389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7741e7e50 con 0x7fd77415b570 2024-01-15T18:15:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.855+0000 7fd778c00640 1 -- 172.21.15.26:0/3192379389 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd774157210 con 0x7fd77415b570 2024-01-15T18:15:31.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.856+0000 7fd76b400640 1 -- 172.21.15.26:0/3192379389 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 42) v1 ==== 91273+0+0 (secure 0 0 0) 0x7fd75c01d460 con 0x7fd77415b570 2024-01-15T18:15:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.856+0000 7fd76b400640 1 --2- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd74c06c610 0x7fd74c06ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.857+0000 7fd76b400640 1 -- 172.21.15.26:0/3192379389 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fd75c088bb0 con 0x7fd77415b570 2024-01-15T18:15:31.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.857+0000 7fd773400640 1 --2- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd74c06c610 0x7fd74c06ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:31.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.858+0000 7fd773400640 1 --2- 172.21.15.26:0/3192379389 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd74c06c610 0x7fd74c06ead0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd764009950 tx=0x7fd764008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:31.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:31.861+0000 7fd76b400640 1 -- 172.21.15.26:0/3192379389 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd774157210 con 0x7fd77415b570 2024-01-15T18:15:32.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:32.080+0000 7fd778c00640 1 -- 172.21.15.26:0/3192379389 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7741e8010 con 0x7fd74c06c610 2024-01-15T18:15:32.113 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:32.110+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:32.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:32.111+0000 7fd76b400640 1 -- 172.21.15.26:0/3192379389 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fd7741e8010 con 0x7fd74c06c610 2024-01-15T18:15:32.114 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:32.132 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:32.657 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:15:32.665 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:15:34.134 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:34.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.369+0000 7f988f600640 1 Processor -- start 2024-01-15T18:15:34.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.369+0000 7f988f600640 1 -- start start 2024-01-15T18:15:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.371+0000 7f988f600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988813df80 0x7f988815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.371+0000 7f988f600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f988813e360 0x7f98880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.374 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.371+0000 7f988f600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988806c470 con 0x7f988815f9c0 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.371+0000 7f988f600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988806c750 con 0x7f988813e360 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.371+0000 7f988f600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988806ca30 con 0x7f988813df80 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.371+0000 7f9887e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f988813e360 0x7f98880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.371+0000 7f9887e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f988813e360 0x7f98880cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32958/0 (socket says 172.21.15.26:32958) 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.371+0000 7f9887e00640 1 -- 172.21.15.26:0/2425360449 learned_addr learned my addr 172.21.15.26:0/2425360449 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.372+0000 7f988cc00640 1 --2- 172.21.15.26:0/2425360449 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988813df80 0x7f988815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.372+0000 7f9887e00640 1 -- 172.21.15.26:0/2425360449 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988813df80 msgr2=0x7f988815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.372+0000 7f9887e00640 1 --2- 172.21.15.26:0/2425360449 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988813df80 0x7f988815bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.372+0000 7f9887e00640 1 -- 172.21.15.26:0/2425360449 >> v1:172.21.15.124:6789/0 conn(0x7f988815f9c0 legacy=0x7f98881408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:34.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.372+0000 7f9887e00640 1 -- 172.21.15.26:0/2425360449 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988806cd10 con 0x7f988813e360 2024-01-15T18:15:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.373+0000 7f9887e00640 1 --2- 172.21.15.26:0/2425360449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f988813e360 0x7f98880cf290 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f9870009870 tx=0x7f987002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47dc4e2881d974d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.374+0000 7f9887400640 1 -- 172.21.15.26:0/2425360449 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870031640 con 0x7f988813e360 2024-01-15T18:15:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.374+0000 7f9887400640 1 -- 172.21.15.26:0/2425360449 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98700317a0 con 0x7f988813e360 2024-01-15T18:15:34.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.374+0000 7f9887400640 1 -- 172.21.15.26:0/2425360449 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870031b30 con 0x7f988813e360 2024-01-15T18:15:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.374+0000 7f988f600640 1 -- 172.21.15.26:0/2425360449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f988813e360 msgr2=0x7f98880cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.374+0000 7f988f600640 1 --2- 172.21.15.26:0/2425360449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f988813e360 0x7f98880cf290 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f9870009870 tx=0x7f987002b8f0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.375+0000 7f988f600640 1 -- 172.21.15.26:0/2425360449 shutdown_connections 2024-01-15T18:15:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.375+0000 7f988f600640 1 --2- 172.21.15.26:0/2425360449 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f988813e360 0x7f98880cf290 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.375+0000 7f988f600640 1 --2- 172.21.15.26:0/2425360449 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988813df80 0x7f988815bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.375+0000 7f988f600640 1 -- 172.21.15.26:0/2425360449 >> 172.21.15.26:0/2425360449 conn(0x7f988806a800 msgr2=0x7f988806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.375+0000 7f988f600640 1 -- 172.21.15.26:0/2425360449 shutdown_connections 2024-01-15T18:15:34.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.375+0000 7f988f600640 1 -- 172.21.15.26:0/2425360449 wait complete. 2024-01-15T18:15:34.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.377+0000 7f988f600640 1 Processor -- start 2024-01-15T18:15:34.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.377+0000 7f988f600640 1 -- start start 2024-01-15T18:15:34.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.378+0000 7f988f600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f988813df80 0x7f98881502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.378+0000 7f988f600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f988813e360 0x7f9888150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.378+0000 7f988f600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988815f9c0 0x7f9888150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.378+0000 7f988f600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98881516a0 con 0x7f988813e360 2024-01-15T18:15:34.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.378+0000 7f988f600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988814f910 con 0x7f988813df80 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.378+0000 7f988f600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988814ed90 con 0x7f988815f9c0 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.378+0000 7f9887e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f988813e360 0x7f9888150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.378+0000 7f9887e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f988813e360 0x7f9888150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32970/0 (socket says 172.21.15.26:32970) 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.378+0000 7f9887e00640 1 -- 172.21.15.26:0/2039427077 learned_addr learned my addr 172.21.15.26:0/2039427077 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.379+0000 7f988cc00640 1 --2- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f988813df80 0x7f98881502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.379+0000 7f988d600640 1 --2- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988815f9c0 0x7f9888150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.379+0000 7f9887e00640 1 -- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988815f9c0 msgr2=0x7f9888150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.379+0000 7f9887e00640 1 --2- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988815f9c0 0x7f9888150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.379+0000 7f9887e00640 1 -- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f988813df80 msgr2=0x7f98881502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.379+0000 7f9887e00640 1 --2- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f988813df80 0x7f98881502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.379+0000 7f9887e00640 1 -- 172.21.15.26:0/2039427077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98700094d0 con 0x7f988813e360 2024-01-15T18:15:34.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.379+0000 7f988d600640 1 --2- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f988815f9c0 0x7f9888150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.379+0000 7f9887e00640 1 --2- 172.21.15.26:0/2039427077 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f988813e360 0x7f9888150840 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f9870009870 tx=0x7f98700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.380+0000 7f9884c00640 1 -- 172.21.15.26:0/2039427077 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987003ac60 con 0x7f988813e360 2024-01-15T18:15:34.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.380+0000 7f9884c00640 1 -- 172.21.15.26:0/2039427077 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987003adc0 con 0x7f988813e360 2024-01-15T18:15:34.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.380+0000 7f988f600640 1 -- 172.21.15.26:0/2039427077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988814f070 con 0x7f988813e360 2024-01-15T18:15:34.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.380+0000 7f9884c00640 1 -- 172.21.15.26:0/2039427077 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870044630 con 0x7f988813e360 2024-01-15T18:15:34.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.380+0000 7f988f600640 1 -- 172.21.15.26:0/2039427077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988814f630 con 0x7f988813e360 2024-01-15T18:15:34.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.381+0000 7f988f600640 1 -- 172.21.15.26:0/2039427077 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98880675d0 con 0x7f988813e360 2024-01-15T18:15:34.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.382+0000 7f9884c00640 1 -- 172.21.15.26:0/2039427077 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f987002f050 con 0x7f988813e360 2024-01-15T18:15:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.383+0000 7f9884c00640 1 --2- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f986006c670 0x7f986006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:34.386 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.384+0000 7f9884c00640 1 -- 172.21.15.26:0/2039427077 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f98700aaad0 con 0x7f988813e360 2024-01-15T18:15:34.387 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.384+0000 7f988cc00640 1 --2- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f986006c670 0x7f986006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:34.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.386+0000 7f988cc00640 1 --2- 172.21.15.26:0/2039427077 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f986006c670 0x7f986006eb30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f9878006fd0 tx=0x7f9878008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:34.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.388+0000 7f9884c00640 1 -- 172.21.15.26:0/2039427077 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9870076fe0 con 0x7f988813e360 2024-01-15T18:15:34.614 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.611+0000 7f988f600640 1 -- 172.21.15.26:0/2039427077 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f98880ce850 con 0x7f986006c670 2024-01-15T18:15:34.635 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:34.632+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:34.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:34.633+0000 7f9884c00640 1 -- 172.21.15.26:0/2039427077 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f987800f860 con 0x7f986006c670 2024-01-15T18:15:34.636 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:34.650 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:36.651 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.905+0000 7fac2c000640 1 Processor -- start 2024-01-15T18:15:36.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.905+0000 7fac2c000640 1 -- start start 2024-01-15T18:15:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.907+0000 7fac2c000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac240cd240 0x7fac240b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.907+0000 7fac2c000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac2409a170 0x7fac240cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.907+0000 7fac2c000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2406cea0 con 0x7fac2406c6b0 2024-01-15T18:15:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.907+0000 7fac2c000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2406cbc0 con 0x7fac2409a170 2024-01-15T18:15:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.907+0000 7fac2c000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2406f350 con 0x7fac240cd240 2024-01-15T18:15:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.907+0000 7fac2a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac2409a170 0x7fac240cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.907+0000 7fac2a000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fac2409a170 0x7fac240cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32986/0 (socket says 172.21.15.26:32986) 2024-01-15T18:15:36.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.907+0000 7fac2a000640 1 -- 172.21.15.26:0/3754019580 learned_addr learned my addr 172.21.15.26:0/3754019580 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.908+0000 7fac2a000640 1 -- 172.21.15.26:0/3754019580 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac240cd240 msgr2=0x7fac240b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.908+0000 7fac2a000640 1 --2- 172.21.15.26:0/3754019580 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac240cd240 0x7fac240b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.908+0000 7fac2a000640 1 -- 172.21.15.26:0/3754019580 >> v1:172.21.15.124:6789/0 conn(0x7fac2406c6b0 legacy=0x7fac24099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:36.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.908+0000 7fac2a000640 1 -- 172.21.15.26:0/3754019580 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2406f630 con 0x7fac2409a170 2024-01-15T18:15:36.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.910+0000 7fac2a000640 1 --2- 172.21.15.26:0/3754019580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2409a170 0x7fac240cf640 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009e50 tx=0x7fac1c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682fdee21ce5c671 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.910+0000 7fac23e00640 1 -- 172.21.15.26:0/3754019580 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c02ca70 con 0x7fac2409a170 2024-01-15T18:15:36.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.911+0000 7fac23e00640 1 -- 172.21.15.26:0/3754019580 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c02cbd0 con 0x7fac2409a170 2024-01-15T18:15:36.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.911+0000 7fac23e00640 1 -- 172.21.15.26:0/3754019580 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0343f0 con 0x7fac2409a170 2024-01-15T18:15:36.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.912+0000 7fac2c000640 1 -- 172.21.15.26:0/3754019580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2409a170 msgr2=0x7fac240cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.912+0000 7fac2c000640 1 --2- 172.21.15.26:0/3754019580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2409a170 0x7fac240cf640 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009e50 tx=0x7fac1c02bed0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.912+0000 7fac2c000640 1 -- 172.21.15.26:0/3754019580 shutdown_connections 2024-01-15T18:15:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.912+0000 7fac2c000640 1 --2- 172.21.15.26:0/3754019580 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac2409a170 0x7fac240cf640 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.912+0000 7fac2c000640 1 --2- 172.21.15.26:0/3754019580 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac240cd240 0x7fac240b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.912+0000 7fac2c000640 1 -- 172.21.15.26:0/3754019580 >> 172.21.15.26:0/3754019580 conn(0x7fac240b2d00 msgr2=0x7fac240b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.912+0000 7fac2c000640 1 -- 172.21.15.26:0/3754019580 shutdown_connections 2024-01-15T18:15:36.915 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.912+0000 7fac2c000640 1 -- 172.21.15.26:0/3754019580 wait complete. 2024-01-15T18:15:36.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.915+0000 7fac2c000640 1 Processor -- start 2024-01-15T18:15:36.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.915+0000 7fac2c000640 1 -- start start 2024-01-15T18:15:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.916+0000 7fac2c000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac2406c6b0 0x7fac240c7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.916+0000 7fac2c000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac2409a170 0x7fac240b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.916+0000 7fac2c000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac240cd240 0x7fac241514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.916+0000 7fac2c000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240c7b90 con 0x7fac240cd240 2024-01-15T18:15:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.916+0000 7fac2c000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240b5750 con 0x7fac2406c6b0 2024-01-15T18:15:36.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.916+0000 7fac2c000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac240c59f0 con 0x7fac2409a170 2024-01-15T18:15:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac2a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac240cd240 0x7fac241514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac2a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac240cd240 0x7fac241514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32988/0 (socket says 172.21.15.26:32988) 2024-01-15T18:15:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac2a000640 1 -- 172.21.15.26:0/211357756 learned_addr learned my addr 172.21.15.26:0/211357756 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac29600640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac2406c6b0 0x7fac240c7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac28c00640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac2409a170 0x7fac240b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac2a000640 1 -- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac2409a170 msgr2=0x7fac240b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac2a000640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac2409a170 0x7fac240b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac2a000640 1 -- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac2406c6b0 msgr2=0x7fac240c7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac2a000640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac2406c6b0 0x7fac240c7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac2a000640 1 -- 172.21.15.26:0/211357756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009b30 con 0x7fac240cd240 2024-01-15T18:15:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac28c00640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fac2409a170 0x7fac240b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.917+0000 7fac29600640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fac2406c6b0 0x7fac240c7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:36.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.918+0000 7fac2a000640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fac240cd240 0x7fac241514b0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fac1c036040 tx=0x7fac1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.918+0000 7fac21600640 1 -- 172.21.15.26:0/211357756 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c03e450 con 0x7fac240cd240 2024-01-15T18:15:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.918+0000 7fac21600640 1 -- 172.21.15.26:0/211357756 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c03e5b0 con 0x7fac240cd240 2024-01-15T18:15:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.918+0000 7fac21600640 1 -- 172.21.15.26:0/211357756 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c03e960 con 0x7fac240cd240 2024-01-15T18:15:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.918+0000 7fac2c000640 1 -- 172.21.15.26:0/211357756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac240c5cd0 con 0x7fac240cd240 2024-01-15T18:15:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.918+0000 7fac2c000640 1 -- 172.21.15.26:0/211357756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac240c56c0 con 0x7fac240cd240 2024-01-15T18:15:36.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.920+0000 7fac2c000640 1 -- 172.21.15.26:0/211357756 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2409ac80 con 0x7fac240cd240 2024-01-15T18:15:36.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.921+0000 7fac21600640 1 -- 172.21.15.26:0/211357756 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fac1c03eac0 con 0x7fac240cd240 2024-01-15T18:15:36.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.922+0000 7fac21600640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fabfc06c6a0 0x7fabfc06eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:36.924 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.922+0000 7fac21600640 1 -- 172.21.15.26:0/211357756 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fac1c0ab7f0 con 0x7fac240cd240 2024-01-15T18:15:36.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.922+0000 7fac29600640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fabfc06c6a0 0x7fabfc06eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:36.926 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.923+0000 7fac29600640 1 --2- 172.21.15.26:0/211357756 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fabfc06c6a0 0x7fabfc06eb60 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fac14009930 tx=0x7fac14008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:36.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:36.928+0000 7fac21600640 1 -- 172.21.15.26:0/211357756 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac1c077d90 con 0x7fac240cd240 2024-01-15T18:15:37.264 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:37.261+0000 7fac2c000640 1 -- 172.21.15.26:0/211357756 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fac240d0a20 con 0x7fabfc06c6a0 2024-01-15T18:15:37.294 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:37.290+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:37.292+0000 7fac21600640 1 -- 172.21.15.26:0/211357756 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fac240d0a20 con 0x7fabfc06c6a0 2024-01-15T18:15:37.295 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:37.311 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:39.312 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:39.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.545+0000 7fe35e000640 1 Processor -- start 2024-01-15T18:15:39.548 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.545+0000 7fe35e000640 1 -- start start 2024-01-15T18:15:39.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.547+0000 7fe35e000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe35815e880 0x7fe35806ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.547+0000 7fe35e000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe35809a170 0x7fe35806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.547+0000 7fe35e000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3580cfa70 con 0x7fe358070490 2024-01-15T18:15:39.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.547+0000 7fe35e000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3580cff00 con 0x7fe35815e880 2024-01-15T18:15:39.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.547+0000 7fe35e000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35806fc70 con 0x7fe35809a170 2024-01-15T18:15:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.547+0000 7fe357400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe35815e880 0x7fe35806ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.548+0000 7fe357400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe35815e880 0x7fe35806ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:32996/0 (socket says 172.21.15.26:32996) 2024-01-15T18:15:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.548+0000 7fe357400640 1 -- 172.21.15.26:0/51671769 learned_addr learned my addr 172.21.15.26:0/51671769 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.548+0000 7fe357e00640 1 --2- 172.21.15.26:0/51671769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe35809a170 0x7fe35806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.548+0000 7fe357400640 1 -- 172.21.15.26:0/51671769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe35809a170 msgr2=0x7fe35806e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.548+0000 7fe357400640 1 --2- 172.21.15.26:0/51671769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe35809a170 0x7fe35806e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.548+0000 7fe357400640 1 -- 172.21.15.26:0/51671769 >> v1:172.21.15.26:6789/0 conn(0x7fe358070490 legacy=0x7fe358099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.548+0000 7fe357400640 1 -- 172.21.15.26:0/51671769 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3580cf420 con 0x7fe35815e880 2024-01-15T18:15:39.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.548+0000 7fe357e00640 1 --2- 172.21.15.26:0/51671769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe35809a170 0x7fe35806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:39.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.549+0000 7fe357400640 1 --2- 172.21.15.26:0/51671769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35815e880 0x7fe35806ff50 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fe348009870 tx=0x7fe34802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bde4090895e633 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.550+0000 7fe356a00640 1 -- 172.21.15.26:0/51671769 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34802f830 con 0x7fe35815e880 2024-01-15T18:15:39.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.550+0000 7fe356a00640 1 -- 172.21.15.26:0/51671769 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34802f990 con 0x7fe35815e880 2024-01-15T18:15:39.552 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.550+0000 7fe356a00640 1 -- 172.21.15.26:0/51671769 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34802fd20 con 0x7fe35815e880 2024-01-15T18:15:39.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.550+0000 7fe35e000640 1 -- 172.21.15.26:0/51671769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35815e880 msgr2=0x7fe35806ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.550+0000 7fe35e000640 1 --2- 172.21.15.26:0/51671769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35815e880 0x7fe35806ff50 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fe348009870 tx=0x7fe34802ba10 comp rx=0 tx=0).stop 2024-01-15T18:15:39.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.551+0000 7fe35e000640 1 -- 172.21.15.26:0/51671769 shutdown_connections 2024-01-15T18:15:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.551+0000 7fe35e000640 1 --2- 172.21.15.26:0/51671769 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe35809a170 0x7fe35806e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.551+0000 7fe35e000640 1 --2- 172.21.15.26:0/51671769 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35815e880 0x7fe35806ff50 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.551+0000 7fe35e000640 1 -- 172.21.15.26:0/51671769 >> 172.21.15.26:0/51671769 conn(0x7fe3580b2d00 msgr2=0x7fe3580b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.551+0000 7fe35e000640 1 -- 172.21.15.26:0/51671769 shutdown_connections 2024-01-15T18:15:39.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.551+0000 7fe35e000640 1 -- 172.21.15.26:0/51671769 wait complete. 2024-01-15T18:15:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.553+0000 7fe35e000640 1 Processor -- start 2024-01-15T18:15:39.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.553+0000 7fe35e000640 1 -- start start 2024-01-15T18:15:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.554+0000 7fe35e000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe358070490 0x7fe3581f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.554+0000 7fe35e000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe35809a170 0x7fe3580b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.554+0000 7fe35e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35815e880 0x7fe3580c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.554+0000 7fe35e000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3581f3a70 con 0x7fe35815e880 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.554+0000 7fe35e000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580b47d0 con 0x7fe35809a170 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.554+0000 7fe35e000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580b4ab0 con 0x7fe358070490 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.554+0000 7fe357e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35815e880 0x7fe3580c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.554+0000 7fe357e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35815e880 0x7fe3580c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33008/0 (socket says 172.21.15.26:33008) 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.554+0000 7fe357e00640 1 -- 172.21.15.26:0/2058349296 learned_addr learned my addr 172.21.15.26:0/2058349296 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe357400640 1 --2- 172.21.15.26:0/2058349296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe358070490 0x7fe3581f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe357e00640 1 -- 172.21.15.26:0/2058349296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe358070490 msgr2=0x7fe3581f3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe357e00640 1 --2- 172.21.15.26:0/2058349296 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe358070490 0x7fe3581f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe357e00640 1 -- 172.21.15.26:0/2058349296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe35809a170 msgr2=0x7fe3580b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe357e00640 1 --2- 172.21.15.26:0/2058349296 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe35809a170 0x7fe3580b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe357e00640 1 -- 172.21.15.26:0/2058349296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3480094d0 con 0x7fe35815e880 2024-01-15T18:15:39.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe357e00640 1 --2- 172.21.15.26:0/2058349296 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe35815e880 0x7fe3580c3820 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fe35000cef0 tx=0x7fe35000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe34f400640 1 -- 172.21.15.26:0/2058349296 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35000d840 con 0x7fe35815e880 2024-01-15T18:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe34f400640 1 -- 172.21.15.26:0/2058349296 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35000d9c0 con 0x7fe35815e880 2024-01-15T18:15:39.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe34f400640 1 -- 172.21.15.26:0/2058349296 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35000dd70 con 0x7fe35815e880 2024-01-15T18:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.555+0000 7fe35e000640 1 -- 172.21.15.26:0/2058349296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3580b4d90 con 0x7fe35815e880 2024-01-15T18:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.556+0000 7fe35e000640 1 -- 172.21.15.26:0/2058349296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3580b5350 con 0x7fe35815e880 2024-01-15T18:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.557+0000 7fe35e000640 1 -- 172.21.15.26:0/2058349296 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3580b5570 con 0x7fe35815e880 2024-01-15T18:15:39.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.558+0000 7fe34f400640 1 -- 172.21.15.26:0/2058349296 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fe35001e3b0 con 0x7fe35815e880 2024-01-15T18:15:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.559+0000 7fe34f400640 1 --2- 172.21.15.26:0/2058349296 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe33006c670 0x7fe33006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:39.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.559+0000 7fe34f400640 1 -- 172.21.15.26:0/2058349296 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fe350089660 con 0x7fe35815e880 2024-01-15T18:15:39.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.559+0000 7fe357400640 1 --2- 172.21.15.26:0/2058349296 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe33006c670 0x7fe33006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:39.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.560+0000 7fe357400640 1 --2- 172.21.15.26:0/2058349296 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe33006c670 0x7fe33006eb30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe348036000 tx=0x7fe348039040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:39.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.563+0000 7fe34f400640 1 -- 172.21.15.26:0/2058349296 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe350055c90 con 0x7fe35815e880 2024-01-15T18:15:39.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.781+0000 7fe35e000640 1 -- 172.21.15.26:0/2058349296 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3580ccb30 con 0x7fe33006c670 2024-01-15T18:15:39.802 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:39.798+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:39.800+0000 7fe34f400640 1 -- 172.21.15.26:0/2058349296 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fe34802fc70 con 0x7fe33006c670 2024-01-15T18:15:39.802 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:39.813 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:41.815 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:42.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.047+0000 7f0477200640 1 Processor -- start 2024-01-15T18:15:42.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.047+0000 7f0477200640 1 -- start start 2024-01-15T18:15:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.049+0000 7f0477200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0470143ae0 0x7f04700cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.049+0000 7f0477200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f047015e3b0 0x7f04700702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.049+0000 7f0477200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04700cfc30 con 0x7f04700cf7d0 2024-01-15T18:15:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.049+0000 7f0477200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04700b1ee0 con 0x7f047015e3b0 2024-01-15T18:15:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.049+0000 7f0477200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04700b2780 con 0x7f0470143ae0 2024-01-15T18:15:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.049+0000 7f0475200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f047015e3b0 0x7f04700702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.049+0000 7f0475200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f047015e3b0 0x7f04700702c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44870/0 (socket says 172.21.15.26:44870) 2024-01-15T18:15:42.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.049+0000 7f0475200640 1 -- 172.21.15.26:0/2211938877 learned_addr learned my addr 172.21.15.26:0/2211938877 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.050+0000 7f046fe00640 1 --2- 172.21.15.26:0/2211938877 >> v2:172.21.15.124:3300/0 conn(0x7f0470143ae0 0x7f04700cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.050+0000 7f0475200640 1 -- 172.21.15.26:0/2211938877 >> v2:172.21.15.124:3300/0 conn(0x7f0470143ae0 msgr2=0x7f04700cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.050+0000 7f0475200640 1 --2- 172.21.15.26:0/2211938877 >> v2:172.21.15.124:3300/0 conn(0x7f0470143ae0 0x7f04700cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.050+0000 7f0475200640 1 -- 172.21.15.26:0/2211938877 >> v1:172.21.15.124:6789/0 conn(0x7f04700cf7d0 legacy=0x7f047015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.050+0000 7f0475200640 1 -- 172.21.15.26:0/2211938877 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047015e990 con 0x7f047015e3b0 2024-01-15T18:15:42.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.051+0000 7f0475200640 1 --2- 172.21.15.26:0/2211938877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f047015e3b0 0x7f04700702c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f04700cfea0 tx=0x7f046802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadbf03bcf859654 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:42.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.051+0000 7f046ea00640 1 -- 172.21.15.26:0/2211938877 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046802ca80 con 0x7f047015e3b0 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.051+0000 7f046ea00640 1 -- 172.21.15.26:0/2211938877 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046802cbe0 con 0x7f047015e3b0 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.051+0000 7f046ea00640 1 -- 172.21.15.26:0/2211938877 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04680343f0 con 0x7f047015e3b0 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.052+0000 7f0477200640 1 -- 172.21.15.26:0/2211938877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f047015e3b0 msgr2=0x7f04700702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.052+0000 7f0477200640 1 --2- 172.21.15.26:0/2211938877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f047015e3b0 0x7f04700702c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f04700cfea0 tx=0x7f046802bee0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.052+0000 7f0477200640 1 -- 172.21.15.26:0/2211938877 shutdown_connections 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.052+0000 7f0477200640 1 --2- 172.21.15.26:0/2211938877 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f047015e3b0 0x7f04700702c0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.052+0000 7f0477200640 1 --2- 172.21.15.26:0/2211938877 >> v2:172.21.15.124:3300/0 conn(0x7f0470143ae0 0x7f04700cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.052+0000 7f0477200640 1 -- 172.21.15.26:0/2211938877 >> 172.21.15.26:0/2211938877 conn(0x7f047006a800 msgr2=0x7f047006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.052+0000 7f0477200640 1 -- 172.21.15.26:0/2211938877 shutdown_connections 2024-01-15T18:15:42.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.052+0000 7f0477200640 1 -- 172.21.15.26:0/2211938877 wait complete. 2024-01-15T18:15:42.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.054+0000 7f0477200640 1 Processor -- start 2024-01-15T18:15:42.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.054+0000 7f0477200640 1 -- start start 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.055+0000 7f0477200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04700cf7d0 0x7f0470159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.055+0000 7f0477200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0470143ae0 0x7f04700cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.055+0000 7f0477200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047015e3b0 0x7f0470141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.055+0000 7f0477200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0470140230 con 0x7f04700cf7d0 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.055+0000 7f0477200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04700ccaf0 con 0x7f0470143ae0 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.055+0000 7f0477200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04700ccdd0 con 0x7f047015e3b0 2024-01-15T18:15:42.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.055+0000 7f046fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04700cf7d0 0x7f0470159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04700cf7d0 0x7f0470159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44876/0 (socket says 172.21.15.26:44876) 2024-01-15T18:15:42.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046fe00640 1 -- 172.21.15.26:0/687397339 learned_addr learned my addr 172.21.15.26:0/687397339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046f400640 1 --2- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0470143ae0 0x7f04700cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f0475200640 1 --2- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047015e3b0 0x7f0470141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046fe00640 1 -- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047015e3b0 msgr2=0x7f0470141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046fe00640 1 --2- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f047015e3b0 0x7f0470141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046fe00640 1 -- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0470143ae0 msgr2=0x7f04700cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046fe00640 1 --2- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0470143ae0 0x7f04700cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046fe00640 1 -- 172.21.15.26:0/687397339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0468009b30 con 0x7f04700cf7d0 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046f400640 1 --2- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0470143ae0 0x7f04700cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.056+0000 7f046fe00640 1 --2- 172.21.15.26:0/687397339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f04700cf7d0 0x7f0470159b70 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f046000c7f0 tx=0x7f046000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:42.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.057+0000 7f0467e00640 1 -- 172.21.15.26:0/687397339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460004220 con 0x7f04700cf7d0 2024-01-15T18:15:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.057+0000 7f0467e00640 1 -- 172.21.15.26:0/687397339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04600043a0 con 0x7f04700cf7d0 2024-01-15T18:15:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.057+0000 7f0467e00640 1 -- 172.21.15.26:0/687397339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0460007860 con 0x7f04700cf7d0 2024-01-15T18:15:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.057+0000 7f0477200640 1 -- 172.21.15.26:0/687397339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04700cd0b0 con 0x7f04700cf7d0 2024-01-15T18:15:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.057+0000 7f0477200640 1 -- 172.21.15.26:0/687397339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047015a0b0 con 0x7f04700cf7d0 2024-01-15T18:15:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.058+0000 7f0477200640 1 -- 172.21.15.26:0/687397339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04701403a0 con 0x7f04700cf7d0 2024-01-15T18:15:42.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.059+0000 7f0467e00640 1 -- 172.21.15.26:0/687397339 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f0460007bd0 con 0x7f04700cf7d0 2024-01-15T18:15:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.060+0000 7f0467e00640 1 --2- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f044806c670 0x7f044806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.060+0000 7f0467e00640 1 -- 172.21.15.26:0/687397339 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f0460088ba0 con 0x7f04700cf7d0 2024-01-15T18:15:42.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.061+0000 7f046f400640 1 --2- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f044806c670 0x7f044806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:42.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.062+0000 7f046f400640 1 --2- 172.21.15.26:0/687397339 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f044806c670 0x7f044806eb30 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f04700cea20 tx=0x7f0458009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:42.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.065+0000 7f0467e00640 1 -- 172.21.15.26:0/687397339 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04600550b0 con 0x7f04700cf7d0 2024-01-15T18:15:42.288 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.285+0000 7f0477200640 1 -- 172.21.15.26:0/687397339 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f04700b2410 con 0x7f044806c670 2024-01-15T18:15:42.308 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:42.304+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:42.305+0000 7f0467e00640 1 -- 172.21.15.26:0/687397339 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f0458005030 con 0x7f044806c670 2024-01-15T18:15:42.309 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:42.323 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:44.324 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.557+0000 7f60cde00640 1 Processor -- start 2024-01-15T18:15:44.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.558+0000 7f60cde00640 1 -- start start 2024-01-15T18:15:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.559+0000 7f60cde00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c813e360 0x7f60c80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.559+0000 7f60cde00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c815c5f0 con 0x7f60c813df80 2024-01-15T18:15:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.559+0000 7f60cde00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c806c470 con 0x7f60c815f9c0 2024-01-15T18:15:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.559+0000 7f60cde00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c806c750 con 0x7f60c813e360 2024-01-15T18:15:44.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.560+0000 7f60c6a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f60c813df80 0x7f60c815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33542/0 (socket says 172.21.15.26:33542) 2024-01-15T18:15:44.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.560+0000 7f60c6a00640 1 -- 172.21.15.26:0/2959854502 learned_addr learned my addr 172.21.15.26:0/2959854502 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:15:44.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.560+0000 7f60c6000640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c813e360 0x7f60c80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.560+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035731431 0 0) 0x7f60c815c5f0 con 0x7f60c813df80 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.560+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60a8003520 con 0x7f60c813df80 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3246810955 0 0) 0x7f60a8003520 con 0x7f60c813df80 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c813e360 msgr2=0x7f60c80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c813e360 0x7f60c80cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 >> v1:172.21.15.124:6789/0 conn(0x7f60c815f9c0 legacy=0x7f60c80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c806ca30 con 0x7f60c813df80 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f60b8001670 con 0x7f60c813df80 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 >> v1:172.21.15.26:6789/0 conn(0x7f60c813df80 legacy=0x7f60c815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60a80039a0 0x7f60a8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60a80047c0 0x7f60a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c813e360 0x7f60a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c806c750 con 0x7f60a80047c0 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a800ba90 con 0x7f60a80039a0 2024-01-15T18:15:44.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a800bc50 con 0x7f60c813e360 2024-01-15T18:15:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.561+0000 7f60c6000640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60a80047c0 0x7f60a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.562+0000 7f60c6a00640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60a80039a0 0x7f60a8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.562+0000 7f60c7400640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c813e360 0x7f60a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.562+0000 7f60c6000640 1 -- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c813e360 msgr2=0x7f60a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.562+0000 7f60c6000640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c813e360 0x7f60a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.562+0000 7f60c6000640 1 -- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60a80039a0 msgr2=0x7f60a8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.562+0000 7f60c6000640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60a80039a0 0x7f60a8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.562+0000 7f60c6000640 1 -- 172.21.15.26:0/2959854502 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8003520 con 0x7f60a80047c0 2024-01-15T18:15:44.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.562+0000 7f60c7400640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c813e360 0x7f60a8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:44.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.563+0000 7f60c6000640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60a80047c0 0x7f60a8004c40 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009f90 tx=0x7f60a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4461aa69c14917 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:44.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.563+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b002cba0 con 0x7f60a80047c0 2024-01-15T18:15:44.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.563+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b000f040 con 0x7f60a80047c0 2024-01-15T18:15:44.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.563+0000 7f60c5600640 1 -- 172.21.15.26:0/2959854502 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0034440 con 0x7f60a80047c0 2024-01-15T18:15:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.564+0000 7f60cde00640 1 -- 172.21.15.26:0/2959854502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60a80047c0 msgr2=0x7f60a8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.564+0000 7f60cde00640 1 --2- 172.21.15.26:0/2959854502 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60a80047c0 0x7f60a8004c40 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009f90 tx=0x7f60a80057f0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.564+0000 7f60c7400640 1 -- 172.21.15.26:0/2959854502 reap_dead start 2024-01-15T18:15:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.564+0000 7f60cde00640 1 -- 172.21.15.26:0/2959854502 shutdown_connections 2024-01-15T18:15:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.564+0000 7f60cde00640 1 -- 172.21.15.26:0/2959854502 >> 172.21.15.26:0/2959854502 conn(0x7f60c806a800 msgr2=0x7f60c806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.564+0000 7f60cde00640 1 -- 172.21.15.26:0/2959854502 shutdown_connections 2024-01-15T18:15:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.564+0000 7f60cde00640 1 -- 172.21.15.26:0/2959854502 wait complete. 2024-01-15T18:15:44.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.566+0000 7f60cde00640 1 Processor -- start 2024-01-15T18:15:44.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.566+0000 7f60cde00640 1 -- start start 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.567+0000 7f60cde00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60a80047c0 0x7f60c8152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.567+0000 7f60cde00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c80a9350 0x7f60c80b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.567+0000 7f60cde00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60c81351b0 0x7f60c8135630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.567+0000 7f60cde00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c8135b70 con 0x7f60a80047c0 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.567+0000 7f60cde00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c8153130 con 0x7f60c81351b0 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.567+0000 7f60cde00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c8153410 con 0x7f60c80a9350 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.568+0000 7f60c6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60a80047c0 0x7f60c8152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.568+0000 7f60c6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60a80047c0 0x7f60c8152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44896/0 (socket says 172.21.15.26:44896) 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.568+0000 7f60c6a00640 1 -- 172.21.15.26:0/263966498 learned_addr learned my addr 172.21.15.26:0/263966498 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.568+0000 7f60c6a00640 1 -- 172.21.15.26:0/263966498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c80a9350 msgr2=0x7f60c80b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.568+0000 7f60c6a00640 1 --2- 172.21.15.26:0/263966498 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f60c80a9350 0x7f60c80b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.568+0000 7f60c6a00640 1 -- 172.21.15.26:0/263966498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60c81351b0 msgr2=0x7f60c8135630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:44.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.568+0000 7f60c6a00640 1 --2- 172.21.15.26:0/263966498 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f60c81351b0 0x7f60c8135630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:44.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.568+0000 7f60c6a00640 1 -- 172.21.15.26:0/263966498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c81536f0 con 0x7f60a80047c0 2024-01-15T18:15:44.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.568+0000 7f60c6a00640 1 --2- 172.21.15.26:0/263966498 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f60a80047c0 0x7f60c8152aa0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f60b800e800 tx=0x7f60b800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:44.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.569+0000 7f60bf400640 1 -- 172.21.15.26:0/263966498 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800c870 con 0x7f60a80047c0 2024-01-15T18:15:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.569+0000 7f60bf400640 1 -- 172.21.15.26:0/263966498 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b800c9d0 con 0x7f60a80047c0 2024-01-15T18:15:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.569+0000 7f60bf400640 1 -- 172.21.15.26:0/263966498 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b800cd80 con 0x7f60a80047c0 2024-01-15T18:15:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.569+0000 7f60cde00640 1 -- 172.21.15.26:0/263966498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c81539d0 con 0x7f60a80047c0 2024-01-15T18:15:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.569+0000 7f60cde00640 1 -- 172.21.15.26:0/263966498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c80a97d0 con 0x7f60a80047c0 2024-01-15T18:15:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.570+0000 7f60cde00640 1 -- 172.21.15.26:0/263966498 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c80a9990 con 0x7f60a80047c0 2024-01-15T18:15:44.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.571+0000 7f60bf400640 1 -- 172.21.15.26:0/263966498 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f60b801d460 con 0x7f60a80047c0 2024-01-15T18:15:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.572+0000 7f60bf400640 1 --2- 172.21.15.26:0/263966498 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f60a006c670 0x7f60a006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:44.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.572+0000 7f60bf400640 1 -- 172.21.15.26:0/263966498 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f60b80888d0 con 0x7f60a80047c0 2024-01-15T18:15:44.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.573+0000 7f60c6000640 1 --2- 172.21.15.26:0/263966498 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f60a006c670 0x7f60a006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:44.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.574+0000 7f60c6000640 1 --2- 172.21.15.26:0/263966498 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f60a006c670 0x7f60a006eb30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f60b0036000 tx=0x7f60b002e070 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:44.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.576+0000 7f60bf400640 1 -- 172.21.15.26:0/263966498 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f60b8055190 con 0x7f60a80047c0 2024-01-15T18:15:44.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.797+0000 7f60cde00640 1 -- 172.21.15.26:0/263966498 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f60c806cff0 con 0x7f60a006c670 2024-01-15T18:15:44.820 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:44.816+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:44.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:44.817+0000 7f60bf400640 1 -- 172.21.15.26:0/263966498 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f60b002cc70 con 0x7f60a006c670 2024-01-15T18:15:44.820 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:44.835 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:46.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.073+0000 7f5f92c00640 1 Processor -- start 2024-01-15T18:15:47.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.074+0000 7f5f92c00640 1 -- start start 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.075+0000 7f5f92c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5f8c0d0100 0x7f5f8c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.075+0000 7f5f92c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8c0d0f40 0x7f5f8c0d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.075+0000 7f5f92c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8c0d0a10 con 0x7f5f8c160920 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.075+0000 7f5f92c00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f8c0cf080 con 0x7f5f8c0d0100 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.075+0000 7f5f92c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c0c6be0 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.076+0000 7f5f90e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5f8c0d0100 0x7f5f8c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.076+0000 7f5f91800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5f8c160920 0x7f5f8c0cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33548/0 (socket says 172.21.15.26:33548) 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.076+0000 7f5f91800640 1 -- 172.21.15.26:0/3257258412 learned_addr learned my addr 172.21.15.26:0/3257258412 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.076+0000 7f5f92200640 1 --2- 172.21.15.26:0/3257258412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8c0d0f40 0x7f5f8c0d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.076+0000 7f5f90e00640 1 -- 172.21.15.26:0/3257258412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8c0d0f40 msgr2=0x7f5f8c0d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:47.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.076+0000 7f5f90e00640 1 --2- 172.21.15.26:0/3257258412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8c0d0f40 0x7f5f8c0d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.076+0000 7f5f8be00640 1 -- 172.21.15.26:0/3257258412 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993251732 0 0) 0x7f5f8c0d0a10 con 0x7f5f8c160920 2024-01-15T18:15:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.076+0000 7f5f90e00640 1 -- 172.21.15.26:0/3257258412 >> v1:172.21.15.26:6789/0 conn(0x7f5f8c160920 legacy=0x7f5f8c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:47.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.076+0000 7f5f90e00640 1 -- 172.21.15.26:0/3257258412 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c0ce7e0 con 0x7f5f8c0d0100 2024-01-15T18:15:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.078+0000 7f5f90e00640 1 --2- 172.21.15.26:0/3257258412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8c0d0100 0x7f5f8c0b3f30 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5f740097a0 tx=0x7f5f7402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b0749f2eb325d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.078+0000 7f5f8be00640 1 -- 172.21.15.26:0/3257258412 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7402d040 con 0x7f5f8c0d0100 2024-01-15T18:15:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.078+0000 7f5f8be00640 1 -- 172.21.15.26:0/3257258412 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f74031810 con 0x7f5f8c0d0100 2024-01-15T18:15:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.078+0000 7f5f8be00640 1 -- 172.21.15.26:0/3257258412 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74031ba0 con 0x7f5f8c0d0100 2024-01-15T18:15:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.078+0000 7f5f92c00640 1 -- 172.21.15.26:0/3257258412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8c0d0100 msgr2=0x7f5f8c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.078+0000 7f5f92c00640 1 --2- 172.21.15.26:0/3257258412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8c0d0100 0x7f5f8c0b3f30 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5f740097a0 tx=0x7f5f7402b8b0 comp rx=0 tx=0).stop 2024-01-15T18:15:47.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.079+0000 7f5f92c00640 1 -- 172.21.15.26:0/3257258412 shutdown_connections 2024-01-15T18:15:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.079+0000 7f5f92c00640 1 --2- 172.21.15.26:0/3257258412 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8c0d0f40 0x7f5f8c0d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.079+0000 7f5f92c00640 1 --2- 172.21.15.26:0/3257258412 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8c0d0100 0x7f5f8c0b3f30 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.079+0000 7f5f92c00640 1 -- 172.21.15.26:0/3257258412 >> 172.21.15.26:0/3257258412 conn(0x7f5f8c0d3090 msgr2=0x7f5f8c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.079+0000 7f5f92c00640 1 -- 172.21.15.26:0/3257258412 shutdown_connections 2024-01-15T18:15:47.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.079+0000 7f5f92c00640 1 -- 172.21.15.26:0/3257258412 wait complete. 2024-01-15T18:15:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.080+0000 7f5f92c00640 1 Processor -- start 2024-01-15T18:15:47.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.081+0000 7f5f92c00640 1 -- start start 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.081+0000 7f5f92c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8c0d0100 0x7f5f8c0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.081+0000 7f5f92c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8c0d0f40 0x7f5f8c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.081+0000 7f5f92c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f8c160920 0x7f5f8c1f3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f92c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1f3900 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f92c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1f3be0 con 0x7f5f8c160920 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f92c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c1f3ec0 con 0x7f5f8c0d0100 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f90e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8c0d0f40 0x7f5f8c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f90e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8c0d0f40 0x7f5f8c15a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44910/0 (socket says 172.21.15.26:44910) 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f90e00640 1 -- 172.21.15.26:0/3302104983 learned_addr learned my addr 172.21.15.26:0/3302104983 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f91800640 1 --2- 172.21.15.26:0/3302104983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8c0d0100 0x7f5f8c0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:47.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f90e00640 1 -- 172.21.15.26:0/3302104983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8c0d0100 msgr2=0x7f5f8c0b8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:47.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f90e00640 1 --2- 172.21.15.26:0/3302104983 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5f8c0d0100 0x7f5f8c0b8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:47.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f90e00640 1 -- 172.21.15.26:0/3302104983 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f8c160920 msgr2=0x7f5f8c1f3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:47.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f90e00640 1 --2- 172.21.15.26:0/3302104983 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5f8c160920 0x7f5f8c1f3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:47.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.082+0000 7f5f90e00640 1 -- 172.21.15.26:0/3302104983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f74009400 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.083+0000 7f5f90e00640 1 --2- 172.21.15.26:0/3302104983 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5f8c0d0f40 0x7f5f8c15a730 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f5f74000c00 tx=0x7f5f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.083+0000 7f5f89600640 1 -- 172.21.15.26:0/3302104983 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7403ba60 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.083+0000 7f5f89600640 1 -- 172.21.15.26:0/3302104983 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f7403bbc0 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.083+0000 7f5f89600640 1 -- 172.21.15.26:0/3302104983 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74004030 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.083+0000 7f5f92c00640 1 -- 172.21.15.26:0/3302104983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c1f4210 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.083+0000 7f5f92c00640 1 -- 172.21.15.26:0/3302104983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c1f3580 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.084+0000 7f5f92c00640 1 -- 172.21.15.26:0/3302104983 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c1f4730 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.085+0000 7f5f89600640 1 -- 172.21.15.26:0/3302104983 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f5f74043070 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.086+0000 7f5f89600640 1 --2- 172.21.15.26:0/3302104983 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5f6406c6c0 0x7f5f6406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.086+0000 7f5f89600640 1 -- 172.21.15.26:0/3302104983 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f5f740aaac0 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.087+0000 7f5f91800640 1 --2- 172.21.15.26:0/3302104983 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5f6406c6c0 0x7f5f6406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:47.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.088+0000 7f5f91800640 1 --2- 172.21.15.26:0/3302104983 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5f6406c6c0 0x7f5f6406eb80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c000f60 tx=0x7f5f7c007c00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:47.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.091+0000 7f5f89600640 1 -- 172.21.15.26:0/3302104983 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f8c1f4730 con 0x7f5f8c0d0f40 2024-01-15T18:15:47.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.313+0000 7f5f92c00640 1 -- 172.21.15.26:0/3302104983 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f8c1f3d50 con 0x7f5f6406c6c0 2024-01-15T18:15:47.337 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:47.334+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:47.338 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:47.334+0000 7f5f89600640 1 -- 172.21.15.26:0/3302104983 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f5f8c1f3d50 con 0x7f5f6406c6c0 2024-01-15T18:15:47.338 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:47.353 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:49.355 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:49.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.590+0000 7fb00e000640 1 Processor -- start 2024-01-15T18:15:49.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.590+0000 7fb00e000640 1 -- start start 2024-01-15T18:15:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.591+0000 7fb00e000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb008160920 0x7fb0080b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.591+0000 7fb00e000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb0080cf9f0 0x7fb0080cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.591+0000 7fb00e000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0080d0a10 con 0x7fb008161010 2024-01-15T18:15:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.591+0000 7fb00e000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0080d03b0 con 0x7fb0080cf9f0 2024-01-15T18:15:49.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.591+0000 7fb00e000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080d1f50 con 0x7fb008160920 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.592+0000 7fb00cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb008160920 0x7fb0080b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.592+0000 7fb00cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb008160920 0x7fb0080b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:51576/0 (socket says 172.21.15.26:51576) 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.592+0000 7fb00cc00640 1 -- 172.21.15.26:0/2060172147 learned_addr learned my addr 172.21.15.26:0/2060172147 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.592+0000 7fafffc00640 1 --2- 172.21.15.26:0/2060172147 >> v2:172.21.15.124:3300/0 conn(0x7fb0080cf9f0 0x7fb0080cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.593+0000 7fb007e00640 1 -- 172.21.15.26:0/2060172147 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195043545 0 0) 0x7fb0080d0a10 con 0x7fb008161010 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.593+0000 7fb007e00640 1 -- 172.21.15.26:0/2060172147 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe8003520 con 0x7fb008161010 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.593+0000 7fb00cc00640 1 -- 172.21.15.26:0/2060172147 >> v2:172.21.15.124:3300/0 conn(0x7fb0080cf9f0 msgr2=0x7fb0080cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.593+0000 7fb00cc00640 1 --2- 172.21.15.26:0/2060172147 >> v2:172.21.15.124:3300/0 conn(0x7fb0080cf9f0 0x7fb0080cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.593+0000 7fb00cc00640 1 -- 172.21.15.26:0/2060172147 >> v1:172.21.15.124:6789/0 conn(0x7fb008161010 legacy=0x7fb0081614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.593+0000 7fb00cc00640 1 -- 172.21.15.26:0/2060172147 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080cf080 con 0x7fb008160920 2024-01-15T18:15:49.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.593+0000 7fafffc00640 1 --2- 172.21.15.26:0/2060172147 >> v2:172.21.15.124:3300/0 conn(0x7fb0080cf9f0 0x7fb0080cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:49.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.595+0000 7fb00cc00640 1 --2- 172.21.15.26:0/2060172147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb008160920 0x7fb0080b3f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2f2ba066f78899 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb007e00640 1 -- 172.21.15.26:0/2060172147 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8030840 con 0x7fb008160920 2024-01-15T18:15:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb007e00640 1 -- 172.21.15.26:0/2060172147 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff80309a0 con 0x7fb008160920 2024-01-15T18:15:49.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb007e00640 1 -- 172.21.15.26:0/2060172147 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8030d10 con 0x7fb008160920 2024-01-15T18:15:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb00e000640 1 -- 172.21.15.26:0/2060172147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb008160920 msgr2=0x7fb0080b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb00e000640 1 --2- 172.21.15.26:0/2060172147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb008160920 0x7fb0080b3f30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff802b980 comp rx=0 tx=0).stop 2024-01-15T18:15:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb00e000640 1 -- 172.21.15.26:0/2060172147 shutdown_connections 2024-01-15T18:15:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb00e000640 1 --2- 172.21.15.26:0/2060172147 >> v2:172.21.15.124:3300/0 conn(0x7fb0080cf9f0 0x7fb0080cfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb00e000640 1 --2- 172.21.15.26:0/2060172147 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb008160920 0x7fb0080b3f30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb00e000640 1 -- 172.21.15.26:0/2060172147 >> 172.21.15.26:0/2060172147 conn(0x7fb0080d3090 msgr2=0x7fb0080d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb00e000640 1 -- 172.21.15.26:0/2060172147 shutdown_connections 2024-01-15T18:15:49.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.596+0000 7fb00e000640 1 -- 172.21.15.26:0/2060172147 wait complete. 2024-01-15T18:15:49.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.598+0000 7fb00e000640 1 Processor -- start 2024-01-15T18:15:49.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.598+0000 7fb00e000640 1 -- start start 2024-01-15T18:15:49.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.599+0000 7fb00e000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0080cf9f0 0x7fb008148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.599+0000 7fb00e000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008160920 0x7fb00815eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.599+0000 7fb00e000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb008161010 0x7fb008149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.599+0000 7fb00e000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080b7080 con 0x7fb008160920 2024-01-15T18:15:49.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.599+0000 7fb00e000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080b7360 con 0x7fb008161010 2024-01-15T18:15:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.599+0000 7fb00e000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080b7640 con 0x7fb0080cf9f0 2024-01-15T18:15:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fafffc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008160920 0x7fb00815eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fafffc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008160920 0x7fb00815eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:44918/0 (socket says 172.21.15.26:44918) 2024-01-15T18:15:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fafffc00640 1 -- 172.21.15.26:0/1287437520 learned_addr learned my addr 172.21.15.26:0/1287437520 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:49.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fb00cc00640 1 --2- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0080cf9f0 0x7fb008148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fb00d600640 1 --2- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb008161010 0x7fb008149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fafffc00640 1 -- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0080cf9f0 msgr2=0x7fb008148ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fafffc00640 1 --2- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb0080cf9f0 0x7fb008148ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fafffc00640 1 -- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb008161010 msgr2=0x7fb008149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:49.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fafffc00640 1 --2- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb008161010 0x7fb008149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:49.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fafffc00640 1 -- 172.21.15.26:0/1287437520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff0009400 con 0x7fb008160920 2024-01-15T18:15:49.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.600+0000 7fb00d600640 1 --2- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb008161010 0x7fb008149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:49.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.601+0000 7fafffc00640 1 --2- 172.21.15.26:0/1287437520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb008160920 0x7fb00815eeb0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7faff000e960 tx=0x7faff000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:49.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.601+0000 7fb005600640 1 -- 172.21.15.26:0/1287437520 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff000ca00 con 0x7fb008160920 2024-01-15T18:15:49.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.601+0000 7fb005600640 1 -- 172.21.15.26:0/1287437520 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff000cb60 con 0x7fb008160920 2024-01-15T18:15:49.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.601+0000 7fb00e000640 1 -- 172.21.15.26:0/1287437520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff80094d0 con 0x7fb008160920 2024-01-15T18:15:49.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.601+0000 7fb005600640 1 -- 172.21.15.26:0/1287437520 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0013470 con 0x7fb008160920 2024-01-15T18:15:49.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.601+0000 7fb00e000640 1 -- 172.21.15.26:0/1287437520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0080b7c00 con 0x7fb008160920 2024-01-15T18:15:49.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.602+0000 7fb00e000640 1 -- 172.21.15.26:0/1287437520 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb008067450 con 0x7fb008160920 2024-01-15T18:15:49.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.603+0000 7fb005600640 1 -- 172.21.15.26:0/1287437520 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7faff00135d0 con 0x7fb008160920 2024-01-15T18:15:49.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.604+0000 7fb005600640 1 --2- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fafe006c670 0x7fafe006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:49.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.605+0000 7fb005600640 1 -- 172.21.15.26:0/1287437520 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7faff0088f30 con 0x7fb008160920 2024-01-15T18:15:49.608 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.605+0000 7fb00cc00640 1 --2- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fafe006c670 0x7fafe006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:49.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.606+0000 7fb00cc00640 1 --2- 172.21.15.26:0/1287437520 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fafe006c670 0x7fafe006eb30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7faff8036000 tx=0x7faff803b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:49.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.608+0000 7fb005600640 1 -- 172.21.15.26:0/1287437520 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faff00553c0 con 0x7fb008160920 2024-01-15T18:15:49.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.835+0000 7fb00e000640 1 -- 172.21.15.26:0/1287437520 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0080d22a0 con 0x7fafe006c670 2024-01-15T18:15:49.860 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:49.857+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:49.857+0000 7fb005600640 1 -- 172.21.15.26:0/1287437520 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fb0080d22a0 con 0x7fafe006c670 2024-01-15T18:15:49.860 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:49.874 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:51.875 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:52.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.106+0000 7f7f6ec00640 1 Processor -- start 2024-01-15T18:15:52.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.106+0000 7f7f6ec00640 1 -- start start 2024-01-15T18:15:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.108+0000 7f7f6ec00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f6815d3c0 0x7f7f680d1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.108+0000 7f7f6ec00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f680b2c70 con 0x7f7f680b4110 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.108+0000 7f7f6ec00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f680cf6d0 con 0x7f7f680cfa40 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.108+0000 7f7f6ec00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f680cf3f0 con 0x7f7f6815d3c0 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.108+0000 7f7f6d800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7f6815d3c0 0x7f7f680d1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.108+0000 7f7f6e200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f7f680b4110 0x7f7f680ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58684/0 (socket says 172.21.15.26:58684) 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.108+0000 7f7f6e200640 1 -- 172.21.15.26:0/3819117609 learned_addr learned my addr 172.21.15.26:0/3819117609 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:15:52.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.109+0000 7f7f6d800640 1 -- 172.21.15.26:0/3819117609 >> v1:172.21.15.124:6789/0 conn(0x7f7f680cfa40 legacy=0x7f7f680cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.109+0000 7f7f6d800640 1 -- 172.21.15.26:0/3819117609 >> v1:172.21.15.26:6789/0 conn(0x7f7f680b4110 legacy=0x7f7f680ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.109+0000 7f7f6d800640 1 -- 172.21.15.26:0/3819117609 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680b2620 con 0x7f7f6815d3c0 2024-01-15T18:15:52.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.109+0000 7f7f67e00640 1 -- 172.21.15.26:0/3819117609 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437105875 0 0) 0x7f7f680b2c70 con 0x7f7f680b4110 2024-01-15T18:15:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.110+0000 7f7f6d800640 1 --2- 172.21.15.26:0/3819117609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f6815d3c0 0x7f7f680d1f20 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f7f580097a0 tx=0x7f7f5802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81189ec56a5f03e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.111+0000 7f7f67e00640 1 -- 172.21.15.26:0/3819117609 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58031830 con 0x7f7f6815d3c0 2024-01-15T18:15:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.111+0000 7f7f67e00640 1 -- 172.21.15.26:0/3819117609 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f58031990 con 0x7f7f6815d3c0 2024-01-15T18:15:52.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.111+0000 7f7f67e00640 1 -- 172.21.15.26:0/3819117609 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58031d20 con 0x7f7f6815d3c0 2024-01-15T18:15:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.111+0000 7f7f6ec00640 1 -- 172.21.15.26:0/3819117609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f6815d3c0 msgr2=0x7f7f680d1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.111+0000 7f7f6ec00640 1 --2- 172.21.15.26:0/3819117609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f6815d3c0 0x7f7f680d1f20 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f7f580097a0 tx=0x7f7f5802b8d0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.111+0000 7f7f6ec00640 1 -- 172.21.15.26:0/3819117609 shutdown_connections 2024-01-15T18:15:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.111+0000 7f7f6ec00640 1 --2- 172.21.15.26:0/3819117609 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f6815d3c0 0x7f7f680d1f20 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.115 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.111+0000 7f7f6ec00640 1 -- 172.21.15.26:0/3819117609 >> 172.21.15.26:0/3819117609 conn(0x7f7f680d3050 msgr2=0x7f7f680d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.112+0000 7f7f6ec00640 1 -- 172.21.15.26:0/3819117609 shutdown_connections 2024-01-15T18:15:52.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.112+0000 7f7f6ec00640 1 -- 172.21.15.26:0/3819117609 wait complete. 2024-01-15T18:15:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.114+0000 7f7f6ec00640 1 Processor -- start 2024-01-15T18:15:52.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.114+0000 7f7f6ec00640 1 -- start start 2024-01-15T18:15:52.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.115+0000 7f7f6ec00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f680b4110 0x7f7f681570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.115+0000 7f7f6ec00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f680cfa40 0x7f7f680c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.115+0000 7f7f6ec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f6815d3c0 0x7f7f680cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.115+0000 7f7f6ec00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6815de40 con 0x7f7f680b4110 2024-01-15T18:15:52.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.115+0000 7f7f6ec00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6815e400 con 0x7f7f680cfa40 2024-01-15T18:15:52.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.115+0000 7f7f6ec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680ccb20 con 0x7f7f6815d3c0 2024-01-15T18:15:52.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.115+0000 7f7f6d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f680b4110 0x7f7f681570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6d800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f680b4110 0x7f7f681570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49118/0 (socket says 172.21.15.26:49118) 2024-01-15T18:15:52.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6d800640 1 -- 172.21.15.26:0/247154107 learned_addr learned my addr 172.21.15.26:0/247154107 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6e200640 1 --2- 172.21.15.26:0/247154107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f6815d3c0 0x7f7f680cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6ce00640 1 --2- 172.21.15.26:0/247154107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f680cfa40 0x7f7f680c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6d800640 1 -- 172.21.15.26:0/247154107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f6815d3c0 msgr2=0x7f7f680cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6d800640 1 --2- 172.21.15.26:0/247154107 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7f6815d3c0 0x7f7f680cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6d800640 1 -- 172.21.15.26:0/247154107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f680cfa40 msgr2=0x7f7f680c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6d800640 1 --2- 172.21.15.26:0/247154107 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7f680cfa40 0x7f7f680c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6d800640 1 -- 172.21.15.26:0/247154107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f58009400 con 0x7f7f680b4110 2024-01-15T18:15:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.116+0000 7f7f6d800640 1 --2- 172.21.15.26:0/247154107 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7f680b4110 0x7f7f681570e0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f7f580098c0 tx=0x7f7f580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:52.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.117+0000 7f7f65600640 1 -- 172.21.15.26:0/247154107 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5803aa60 con 0x7f7f680b4110 2024-01-15T18:15:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.117+0000 7f7f65600640 1 -- 172.21.15.26:0/247154107 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5803abc0 con 0x7f7f680b4110 2024-01-15T18:15:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.117+0000 7f7f65600640 1 -- 172.21.15.26:0/247154107 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f580443f0 con 0x7f7f680b4110 2024-01-15T18:15:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.117+0000 7f7f6ec00640 1 -- 172.21.15.26:0/247154107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f680cce00 con 0x7f7f680b4110 2024-01-15T18:15:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.117+0000 7f7f6ec00640 1 -- 172.21.15.26:0/247154107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f680cc780 con 0x7f7f680b4110 2024-01-15T18:15:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.118+0000 7f7f6ec00640 1 -- 172.21.15.26:0/247154107 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6808ba80 con 0x7f7f680b4110 2024-01-15T18:15:52.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.119+0000 7f7f65600640 1 -- 172.21.15.26:0/247154107 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f7f5804d450 con 0x7f7f680b4110 2024-01-15T18:15:52.123 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.120+0000 7f7f65600640 1 --2- 172.21.15.26:0/247154107 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f4006c6f0 0x7f7f4006ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.120+0000 7f7f65600640 1 -- 172.21.15.26:0/247154107 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f7f580aadf0 con 0x7f7f680b4110 2024-01-15T18:15:52.124 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.121+0000 7f7f6ce00640 1 --2- 172.21.15.26:0/247154107 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f4006c6f0 0x7f7f4006ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:52.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.122+0000 7f7f6ce00640 1 --2- 172.21.15.26:0/247154107 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7f4006c6f0 0x7f7f4006ebb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f7f681e9bc0 tx=0x7f7f50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:52.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.125+0000 7f7f65600640 1 -- 172.21.15.26:0/247154107 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f580b0e60 con 0x7f7f680b4110 2024-01-15T18:15:52.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.344+0000 7f7f6ec00640 1 -- 172.21.15.26:0/247154107 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f680d0f50 con 0x7f7f4006c6f0 2024-01-15T18:15:52.364 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:52.362+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:52.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:52.362+0000 7f7f65600640 1 -- 172.21.15.26:0/247154107 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f7f50004490 con 0x7f7f4006c6f0 2024-01-15T18:15:52.365 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:52.380 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:54.381 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:54.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.615+0000 7ff358a00640 1 Processor -- start 2024-01-15T18:15:54.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.615+0000 7ff358a00640 1 -- start start 2024-01-15T18:15:54.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.617+0000 7ff358a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff35013df80 0x7ff35015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.617+0000 7ff358a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff3500cf9a0 0x7ff3500cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.617+0000 7ff358a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35006c470 con 0x7ff35013e360 2024-01-15T18:15:54.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.617+0000 7ff358a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35006c750 con 0x7ff35013df80 2024-01-15T18:15:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.617+0000 7ff358a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35006ca30 con 0x7ff3500cf9a0 2024-01-15T18:15:54.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.617+0000 7ff356000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff35013df80 0x7ff35015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.617+0000 7ff355600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff35013e360 0x7ff3500cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58688/0 (socket says 172.21.15.26:58688) 2024-01-15T18:15:54.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.617+0000 7ff355600640 1 -- 172.21.15.26:0/1735937361 learned_addr learned my addr 172.21.15.26:0/1735937361 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:15:54.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.617+0000 7ff356a00640 1 --2- 172.21.15.26:0/1735937361 >> v2:172.21.15.124:3300/0 conn(0x7ff3500cf9a0 0x7ff3500cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.618+0000 7ff356000640 1 -- 172.21.15.26:0/1735937361 >> v2:172.21.15.124:3300/0 conn(0x7ff3500cf9a0 msgr2=0x7ff3500cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:54.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.618+0000 7ff356000640 1 --2- 172.21.15.26:0/1735937361 >> v2:172.21.15.124:3300/0 conn(0x7ff3500cf9a0 0x7ff3500cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.618+0000 7ff356000640 1 -- 172.21.15.26:0/1735937361 >> v1:172.21.15.26:6789/0 conn(0x7ff35013e360 legacy=0x7ff3500cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.618+0000 7ff356000640 1 -- 172.21.15.26:0/1735937361 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35006cd10 con 0x7ff35013df80 2024-01-15T18:15:54.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.618+0000 7ff354c00640 1 -- 172.21.15.26:0/1735937361 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592357309 0 0) 0x7ff35006c470 con 0x7ff35013e360 2024-01-15T18:15:54.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.619+0000 7ff356000640 1 --2- 172.21.15.26:0/1735937361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff35013df80 0x7ff35015bee0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ff3500ccb10 tx=0x7ff34002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9e2eda4bb4c250 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.619+0000 7ff354c00640 1 -- 172.21.15.26:0/1735937361 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34002cc80 con 0x7ff35013df80 2024-01-15T18:15:54.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff354c00640 1 -- 172.21.15.26:0/1735937361 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34002cde0 con 0x7ff35013df80 2024-01-15T18:15:54.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff354c00640 1 -- 172.21.15.26:0/1735937361 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340034660 con 0x7ff35013df80 2024-01-15T18:15:54.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff358a00640 1 -- 172.21.15.26:0/1735937361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff35013df80 msgr2=0x7ff35015bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff358a00640 1 --2- 172.21.15.26:0/1735937361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff35013df80 0x7ff35015bee0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7ff3500ccb10 tx=0x7ff34002bed0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff358a00640 1 -- 172.21.15.26:0/1735937361 shutdown_connections 2024-01-15T18:15:54.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff358a00640 1 --2- 172.21.15.26:0/1735937361 >> v2:172.21.15.124:3300/0 conn(0x7ff3500cf9a0 0x7ff3500cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff358a00640 1 --2- 172.21.15.26:0/1735937361 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff35013df80 0x7ff35015bee0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff358a00640 1 -- 172.21.15.26:0/1735937361 >> 172.21.15.26:0/1735937361 conn(0x7ff35006a800 msgr2=0x7ff35006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:54.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff358a00640 1 -- 172.21.15.26:0/1735937361 shutdown_connections 2024-01-15T18:15:54.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.620+0000 7ff358a00640 1 -- 172.21.15.26:0/1735937361 wait complete. 2024-01-15T18:15:54.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.622+0000 7ff358a00640 1 Processor -- start 2024-01-15T18:15:54.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.622+0000 7ff358a00640 1 -- start start 2024-01-15T18:15:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.623+0000 7ff358a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3500cf9a0 0x7ff350070d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.623+0000 7ff358a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff35013df80 0x7ff350159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.623+0000 7ff358a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff35013e360 0x7ff350159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.623+0000 7ff358a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff350072060 con 0x7ff3500cf9a0 2024-01-15T18:15:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.623+0000 7ff358a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff350071450 con 0x7ff35013e360 2024-01-15T18:15:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.623+0000 7ff358a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff350071730 con 0x7ff35013df80 2024-01-15T18:15:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.623+0000 7ff356000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3500cf9a0 0x7ff350070d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.623+0000 7ff356000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3500cf9a0 0x7ff350070d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49136/0 (socket says 172.21.15.26:49136) 2024-01-15T18:15:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff356000640 1 -- 172.21.15.26:0/3417334852 learned_addr learned my addr 172.21.15.26:0/3417334852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff355600640 1 --2- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff35013df80 0x7ff350159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff356a00640 1 --2- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff35013e360 0x7ff350159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff356000640 1 -- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff35013df80 msgr2=0x7ff350159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff356000640 1 --2- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff35013df80 0x7ff350159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff356000640 1 -- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff35013e360 msgr2=0x7ff350159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff356000640 1 --2- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff35013e360 0x7ff350159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff356000640 1 -- 172.21.15.26:0/3417334852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff340009a70 con 0x7ff3500cf9a0 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff356a00640 1 --2- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff35013e360 0x7ff350159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.624+0000 7ff356000640 1 --2- 172.21.15.26:0/3417334852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff3500cf9a0 0x7ff350070d90 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff340009e80 tx=0x7ff340004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.625+0000 7ff34e000640 1 -- 172.21.15.26:0/3417334852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340003b90 con 0x7ff3500cf9a0 2024-01-15T18:15:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.625+0000 7ff34e000640 1 -- 172.21.15.26:0/3417334852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff340003cf0 con 0x7ff3500cf9a0 2024-01-15T18:15:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.625+0000 7ff34e000640 1 -- 172.21.15.26:0/3417334852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340004220 con 0x7ff3500cf9a0 2024-01-15T18:15:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.625+0000 7ff358a00640 1 -- 172.21.15.26:0/3417334852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff350071a10 con 0x7ff3500cf9a0 2024-01-15T18:15:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.625+0000 7ff358a00640 1 -- 172.21.15.26:0/3417334852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff350131a30 con 0x7ff3500cf9a0 2024-01-15T18:15:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.626+0000 7ff358a00640 1 -- 172.21.15.26:0/3417334852 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3500675d0 con 0x7ff3500cf9a0 2024-01-15T18:15:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.627+0000 7ff34e000640 1 -- 172.21.15.26:0/3417334852 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7ff34003b050 con 0x7ff3500cf9a0 2024-01-15T18:15:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.628+0000 7ff34e000640 1 --2- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff32806c670 0x7ff32806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.628+0000 7ff34e000640 1 -- 172.21.15.26:0/3417334852 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7ff3400ab230 con 0x7ff3500cf9a0 2024-01-15T18:15:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.629+0000 7ff355600640 1 --2- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff32806c670 0x7ff32806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:54.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.630+0000 7ff355600640 1 --2- 172.21.15.26:0/3417334852 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff32806c670 0x7ff32806eb30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff338007fd0 tx=0x7ff338009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:54.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.633+0000 7ff34e000640 1 -- 172.21.15.26:0/3417334852 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff340077860 con 0x7ff3500cf9a0 2024-01-15T18:15:54.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.854+0000 7ff358a00640 1 -- 172.21.15.26:0/3417334852 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7ff35015f9c0 con 0x7ff32806c670 2024-01-15T18:15:54.878 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:54.875+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:54.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:54.875+0000 7ff34e000640 1 -- 172.21.15.26:0/3417334852 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7ff338001350 con 0x7ff32806c670 2024-01-15T18:15:54.878 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:54.898 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:56.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.133+0000 7f2945a00640 1 Processor -- start 2024-01-15T18:15:57.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.134+0000 7f2945a00640 1 -- start start 2024-01-15T18:15:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.135+0000 7f2945a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29400d0100 0x7f29400b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.135+0000 7f2945a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29400d0f40 0x7f29400d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.135+0000 7f2945a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29400d0a10 con 0x7f2940160920 2024-01-15T18:15:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.135+0000 7f2945a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29400cf080 con 0x7f29400d0f40 2024-01-15T18:15:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.135+0000 7f2945a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29400c6be0 con 0x7f29400d0100 2024-01-15T18:15:57.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.136+0000 7f293fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2940160920 0x7f29400cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:58704/0 (socket says 172.21.15.26:58704) 2024-01-15T18:15:57.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.136+0000 7f293fe00640 1 -- 172.21.15.26:0/2404175236 learned_addr learned my addr 172.21.15.26:0/2404175236 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.136+0000 7f2937e00640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29400d0100 0x7f29400b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.136+0000 7f2945000640 1 --2- 172.21.15.26:0/2404175236 >> v2:172.21.15.124:3300/0 conn(0x7f29400d0f40 0x7f29400d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.136+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553621272 0 0) 0x7f29400d0a10 con 0x7f2940160920 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.136+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2920003520 con 0x7f2940160920 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3001841233 0 0) 0x7f2920003520 con 0x7f2940160920 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29400d0100 msgr2=0x7f29400b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29400d0100 0x7f29400b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 >> v2:172.21.15.124:3300/0 conn(0x7f29400d0f40 msgr2=0x7f29400d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 --2- 172.21.15.26:0/2404175236 >> v2:172.21.15.124:3300/0 conn(0x7f29400d0f40 0x7f29400d13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29400ce7e0 con 0x7f2940160920 2024-01-15T18:15:57.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f2930001670 con 0x7f2940160920 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 >> v1:172.21.15.26:6789/0 conn(0x7f2940160920 legacy=0x7f29400cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29400d0100 0x7f2920003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2920003f40 0x7f29200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2920004d00 0x7f2920005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29400cf080 con 0x7f2920004d00 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29400c6be0 con 0x7f2920003f40 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292000b980 con 0x7f29400d0100 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.137+0000 7f2945000640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2920004d00 0x7f2920005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.138+0000 7f2937e00640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2920003f40 0x7f29200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.138+0000 7f293fe00640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29400d0100 0x7f2920003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.138+0000 7f2945000640 1 -- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29400d0100 msgr2=0x7f2920003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.138+0000 7f2945000640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29400d0100 0x7f2920003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.138+0000 7f2945000640 1 -- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2920003f40 msgr2=0x7f29200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:57.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.138+0000 7f2945000640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2920003f40 0x7f29200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.138+0000 7f2945000640 1 -- 172.21.15.26:0/2404175236 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920003520 con 0x7f2920004d00 2024-01-15T18:15:57.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.138+0000 7f293fe00640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29400d0100 0x7f2920003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:15:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.139+0000 7f2945000640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2920004d00 0x7f2920005180 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f293800e170 tx=0x7f29200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208f585d43a7c694 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.139+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293802cc20 con 0x7f2920004d00 2024-01-15T18:15:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.139+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293800f040 con 0x7f2920004d00 2024-01-15T18:15:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.139+0000 7f293f400640 1 -- 172.21.15.26:0/2404175236 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938034440 con 0x7f2920004d00 2024-01-15T18:15:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.140+0000 7f2945a00640 1 -- 172.21.15.26:0/2404175236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2920004d00 msgr2=0x7f2920005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.140+0000 7f2945a00640 1 --2- 172.21.15.26:0/2404175236 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2920004d00 0x7f2920005180 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f293800e170 tx=0x7f29200057f0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.140+0000 7f2945000640 1 -- 172.21.15.26:0/2404175236 reap_dead start 2024-01-15T18:15:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.140+0000 7f2945a00640 1 -- 172.21.15.26:0/2404175236 shutdown_connections 2024-01-15T18:15:57.145 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.140+0000 7f2945a00640 1 -- 172.21.15.26:0/2404175236 >> 172.21.15.26:0/2404175236 conn(0x7f29400d3090 msgr2=0x7f29400d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.140+0000 7f2945a00640 1 -- 172.21.15.26:0/2404175236 shutdown_connections 2024-01-15T18:15:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.140+0000 7f2945a00640 1 -- 172.21.15.26:0/2404175236 wait complete. 2024-01-15T18:15:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.142+0000 7f2945a00640 1 Processor -- start 2024-01-15T18:15:57.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.142+0000 7f2945a00640 1 -- start start 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.143+0000 7f2945a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2920004d00 0x7f294015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.143+0000 7f2945a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29400cd200 0x7f2940157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.143+0000 7f2945a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f294006c1c0 0x7f294006c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.143+0000 7f2945a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294006cb80 con 0x7f294006c1c0 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.143+0000 7f2945a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294015c4d0 con 0x7f29400cd200 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.143+0000 7f2945a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294015d3a0 con 0x7f2920004d00 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.143+0000 7f2945000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f294006c1c0 0x7f294006c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.144+0000 7f2945000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f294006c1c0 0x7f294006c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49152/0 (socket says 172.21.15.26:49152) 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.144+0000 7f2945000640 1 -- 172.21.15.26:0/3171876768 learned_addr learned my addr 172.21.15.26:0/3171876768 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.144+0000 7f2937e00640 1 --2- 172.21.15.26:0/3171876768 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29400cd200 0x7f2940157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.144+0000 7f293fe00640 1 --2- 172.21.15.26:0/3171876768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2920004d00 0x7f294015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.144+0000 7f2945000640 1 -- 172.21.15.26:0/3171876768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2920004d00 msgr2=0x7f294015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.144+0000 7f2945000640 1 --2- 172.21.15.26:0/3171876768 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2920004d00 0x7f294015ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.144+0000 7f2945000640 1 -- 172.21.15.26:0/3171876768 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29400cd200 msgr2=0x7f2940157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.144+0000 7f2945000640 1 --2- 172.21.15.26:0/3171876768 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29400cd200 0x7f2940157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.144+0000 7f2945000640 1 -- 172.21.15.26:0/3171876768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294015c720 con 0x7f294006c1c0 2024-01-15T18:15:57.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.145+0000 7f2945000640 1 --2- 172.21.15.26:0/3171876768 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f294006c1c0 0x7f294006c640 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f2938037040 tx=0x7f2938009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.145+0000 7f293cc00640 1 -- 172.21.15.26:0/3171876768 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938034d50 con 0x7f294006c1c0 2024-01-15T18:15:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.145+0000 7f293cc00640 1 -- 172.21.15.26:0/3171876768 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2938003990 con 0x7f294006c1c0 2024-01-15T18:15:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.145+0000 7f2945a00640 1 -- 172.21.15.26:0/3171876768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294015ca00 con 0x7f294006c1c0 2024-01-15T18:15:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.145+0000 7f293cc00640 1 -- 172.21.15.26:0/3171876768 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938003d20 con 0x7f294006c1c0 2024-01-15T18:15:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.145+0000 7f2945a00640 1 -- 172.21.15.26:0/3171876768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29400b69a0 con 0x7f294006c1c0 2024-01-15T18:15:57.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.146+0000 7f2945a00640 1 -- 172.21.15.26:0/3171876768 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2940067450 con 0x7f294006c1c0 2024-01-15T18:15:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.147+0000 7f293cc00640 1 -- 172.21.15.26:0/3171876768 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f293803b040 con 0x7f294006c1c0 2024-01-15T18:15:57.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.148+0000 7f293cc00640 1 --2- 172.21.15.26:0/3171876768 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f291806c670 0x7f291806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.148+0000 7f293cc00640 1 -- 172.21.15.26:0/3171876768 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f29380a9eb0 con 0x7f294006c1c0 2024-01-15T18:15:57.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.149+0000 7f293fe00640 1 --2- 172.21.15.26:0/3171876768 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f291806c670 0x7f291806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:57.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.150+0000 7f293fe00640 1 --2- 172.21.15.26:0/3171876768 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f291806c670 0x7f291806eb30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2930005c10 tx=0x7f2930009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:57.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.153+0000 7f293cc00640 1 -- 172.21.15.26:0/3171876768 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29380763c0 con 0x7f294006c1c0 2024-01-15T18:15:57.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.375+0000 7f2945a00640 1 -- 172.21.15.26:0/3171876768 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f2940160d00 con 0x7f291806c670 2024-01-15T18:15:57.398 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:57.395+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:57.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:57.396+0000 7f293cc00640 1 -- 172.21.15.26:0/3171876768 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f293000d050 con 0x7f291806c670 2024-01-15T18:15:57.399 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:57.412 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:15:59.414 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:15:59.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.652+0000 7f9c3de00640 1 Processor -- start 2024-01-15T18:15:59.655 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.652+0000 7f9c3de00640 1 -- start start 2024-01-15T18:15:59.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.654+0000 7f9c3de00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9c3813cfa0 0x7f9c3815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.654+0000 7f9c3de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c3813fcb0 0x7f9c380b2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.654+0000 7f9c3de00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c380cdec0 con 0x7f9c380cd530 2024-01-15T18:15:59.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.654+0000 7f9c3de00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c380ce1a0 con 0x7f9c3813cfa0 2024-01-15T18:15:59.657 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.654+0000 7f9c3de00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c380ce480 con 0x7f9c3813fcb0 2024-01-15T18:15:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.655+0000 7f9c3d400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c3813fcb0 0x7f9c380b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.655+0000 7f9c3ca00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9c3813cfa0 0x7f9c3815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.655+0000 7f9c37e00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f9c380cd530 0x7f9c3813f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:45012/0 (socket says 172.21.15.26:45012) 2024-01-15T18:15:59.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.655+0000 7f9c37e00640 1 -- 172.21.15.26:0/671204343 learned_addr learned my addr 172.21.15.26:0/671204343 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:15:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.656+0000 7f9c37400640 1 -- 172.21.15.26:0/671204343 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451814341 0 0) 0x7f9c380cdec0 con 0x7f9c380cd530 2024-01-15T18:15:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.656+0000 7f9c37400640 1 -- 172.21.15.26:0/671204343 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c18003520 con 0x7f9c380cd530 2024-01-15T18:15:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.656+0000 7f9c3d400640 1 -- 172.21.15.26:0/671204343 >> v2:172.21.15.124:3300/0 conn(0x7f9c3813cfa0 msgr2=0x7f9c3815afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:15:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.656+0000 7f9c3d400640 1 --2- 172.21.15.26:0/671204343 >> v2:172.21.15.124:3300/0 conn(0x7f9c3813cfa0 0x7f9c3815afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.656+0000 7f9c3d400640 1 -- 172.21.15.26:0/671204343 >> v1:172.21.15.124:6789/0 conn(0x7f9c380cd530 legacy=0x7f9c3813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.656+0000 7f9c3d400640 1 -- 172.21.15.26:0/671204343 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c380ce760 con 0x7f9c3813fcb0 2024-01-15T18:15:59.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.656+0000 7f9c3ca00640 1 --2- 172.21.15.26:0/671204343 >> v2:172.21.15.124:3300/0 conn(0x7f9c3813cfa0 0x7f9c3815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:15:59.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.658+0000 7f9c3d400640 1 --2- 172.21.15.26:0/671204343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c3813fcb0 0x7f9c380b2040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c300097a0 tx=0x7f9c3002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e5d2735f1ff5f00 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.658+0000 7f9c37400640 1 -- 172.21.15.26:0/671204343 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3002e070 con 0x7f9c3813fcb0 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.658+0000 7f9c37400640 1 -- 172.21.15.26:0/671204343 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c300315a0 con 0x7f9c3813fcb0 2024-01-15T18:15:59.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.658+0000 7f9c37400640 1 -- 172.21.15.26:0/671204343 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30031930 con 0x7f9c3813fcb0 2024-01-15T18:15:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.659+0000 7f9c3de00640 1 -- 172.21.15.26:0/671204343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c3813fcb0 msgr2=0x7f9c380b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.659+0000 7f9c3de00640 1 --2- 172.21.15.26:0/671204343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c3813fcb0 0x7f9c380b2040 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9c300097a0 tx=0x7f9c3002b840 comp rx=0 tx=0).stop 2024-01-15T18:15:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.659+0000 7f9c3de00640 1 -- 172.21.15.26:0/671204343 shutdown_connections 2024-01-15T18:15:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.659+0000 7f9c3de00640 1 --2- 172.21.15.26:0/671204343 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c3813fcb0 0x7f9c380b2040 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.659+0000 7f9c3de00640 1 --2- 172.21.15.26:0/671204343 >> v2:172.21.15.124:3300/0 conn(0x7f9c3813cfa0 0x7f9c3815afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.659+0000 7f9c3de00640 1 -- 172.21.15.26:0/671204343 >> 172.21.15.26:0/671204343 conn(0x7f9c3806a800 msgr2=0x7f9c3806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:15:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.659+0000 7f9c3de00640 1 -- 172.21.15.26:0/671204343 shutdown_connections 2024-01-15T18:15:59.662 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.659+0000 7f9c3de00640 1 -- 172.21.15.26:0/671204343 wait complete. 2024-01-15T18:15:59.663 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.661+0000 7f9c3de00640 1 Processor -- start 2024-01-15T18:15:59.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.661+0000 7f9c3de00640 1 -- start start 2024-01-15T18:15:59.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.662+0000 7f9c3de00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c380cd530 0x7f9c3813ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.662+0000 7f9c3de00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c3813cfa0 0x7f9c3813de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.662+0000 7f9c3de00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c3813fcb0 0x7f9c381f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.662+0000 7f9c3de00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3813e520 con 0x7f9c3813fcb0 2024-01-15T18:15:59.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.662+0000 7f9c3de00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c381388d0 con 0x7f9c3813cfa0 2024-01-15T18:15:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.662+0000 7f9c3de00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c38137fa0 con 0x7f9c380cd530 2024-01-15T18:15:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c3d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c3813fcb0 0x7f9c381f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c3d400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c3813fcb0 0x7f9c381f2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49162/0 (socket says 172.21.15.26:49162) 2024-01-15T18:15:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c3d400640 1 -- 172.21.15.26:0/3258034981 learned_addr learned my addr 172.21.15.26:0/3258034981 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:15:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c3ca00640 1 --2- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c380cd530 0x7f9c3813ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c37e00640 1 --2- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c3813cfa0 0x7f9c3813de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c3d400640 1 -- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c380cd530 msgr2=0x7f9c3813ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c3d400640 1 --2- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9c380cd530 0x7f9c3813ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c3d400640 1 -- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c3813cfa0 msgr2=0x7f9c3813de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:15:59.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c3d400640 1 --2- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c3813cfa0 0x7f9c3813de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:15:59.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c3d400640 1 -- 172.21.15.26:0/3258034981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2800bc80 con 0x7f9c3813fcb0 2024-01-15T18:15:59.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.663+0000 7f9c37e00640 1 --2- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c3813cfa0 0x7f9c3813de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:15:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.664+0000 7f9c3d400640 1 --2- 172.21.15.26:0/3258034981 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c3813fcb0 0x7f9c381f2600 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f9c300097a0 tx=0x7f9c300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.664+0000 7f9c34c00640 1 -- 172.21.15.26:0/3258034981 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30032840 con 0x7f9c3813fcb0 2024-01-15T18:15:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.664+0000 7f9c34c00640 1 -- 172.21.15.26:0/3258034981 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c300329c0 con 0x7f9c3813fcb0 2024-01-15T18:15:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.664+0000 7f9c3de00640 1 -- 172.21.15.26:0/3258034981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c30009400 con 0x7f9c3813fcb0 2024-01-15T18:15:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.664+0000 7f9c34c00640 1 -- 172.21.15.26:0/3258034981 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30032d70 con 0x7f9c3813fcb0 2024-01-15T18:15:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.664+0000 7f9c3de00640 1 -- 172.21.15.26:0/3258034981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c38138560 con 0x7f9c3813fcb0 2024-01-15T18:15:59.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.665+0000 7f9c3de00640 1 -- 172.21.15.26:0/3258034981 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c38138a60 con 0x7f9c3813fcb0 2024-01-15T18:15:59.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.667+0000 7f9c34c00640 1 -- 172.21.15.26:0/3258034981 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f9c3004e470 con 0x7f9c3813fcb0 2024-01-15T18:15:59.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.669+0000 7f9c34c00640 1 --2- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c1006c670 0x7f9c1006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:15:59.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.669+0000 7f9c34c00640 1 -- 172.21.15.26:0/3258034981 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f9c300aaa00 con 0x7f9c3813fcb0 2024-01-15T18:15:59.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.669+0000 7f9c3ca00640 1 --2- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c1006c670 0x7f9c1006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:15:59.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.670+0000 7f9c3ca00640 1 --2- 172.21.15.26:0/3258034981 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c1006c670 0x7f9c1006eb30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9c28003e20 tx=0x7f9c2800d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:15:59.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.672+0000 7f9c34c00640 1 -- 172.21.15.26:0/3258034981 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c30076f10 con 0x7f9c3813fcb0 2024-01-15T18:15:59.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.895+0000 7f9c3de00640 1 -- 172.21.15.26:0/3258034981 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c380b2550 con 0x7f9c1006c670 2024-01-15T18:15:59.920 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:15:59.917+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:15:59.918+0000 7f9c34c00640 1 -- 172.21.15.26:0/3258034981 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f9c380b2550 con 0x7f9c1006c670 2024-01-15T18:15:59.920 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:15:59.934 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:01.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.170+0000 7fe81aa00640 1 Processor -- start 2024-01-15T18:16:02.173 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.170+0000 7fe81aa00640 1 -- start start 2024-01-15T18:16:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.172+0000 7fe81aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe81413e360 0x7fe8140cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.172+0000 7fe81aa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fe81415f9c0 0x7fe8140cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.174 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.172+0000 7fe81aa00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81415c5f0 con 0x7fe81413df80 2024-01-15T18:16:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.172+0000 7fe81aa00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81406c470 con 0x7fe81413e360 2024-01-15T18:16:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.172+0000 7fe81aa00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81406c750 con 0x7fe81415f9c0 2024-01-15T18:16:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.172+0000 7fe80be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe81413e360 0x7fe8140cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.172+0000 7fe80be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fe81413e360 0x7fe8140cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58746/0 (socket says 172.21.15.26:58746) 2024-01-15T18:16:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.172+0000 7fe80be00640 1 -- 172.21.15.26:0/701414563 learned_addr learned my addr 172.21.15.26:0/701414563 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:02.175 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.172+0000 7fe818a00640 1 --2- 172.21.15.26:0/701414563 >> v2:172.21.15.124:3300/0 conn(0x7fe81415f9c0 0x7fe8140cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.173+0000 7fe80be00640 1 -- 172.21.15.26:0/701414563 >> v2:172.21.15.124:3300/0 conn(0x7fe81415f9c0 msgr2=0x7fe8140cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.173+0000 7fe80be00640 1 --2- 172.21.15.26:0/701414563 >> v2:172.21.15.124:3300/0 conn(0x7fe81415f9c0 0x7fe8140cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.173+0000 7fe80be00640 1 -- 172.21.15.26:0/701414563 >> v1:172.21.15.124:6789/0 conn(0x7fe81413df80 legacy=0x7fe81415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.173+0000 7fe80be00640 1 -- 172.21.15.26:0/701414563 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81406ca30 con 0x7fe81413e360 2024-01-15T18:16:02.176 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.174+0000 7fe80be00640 1 --2- 172.21.15.26:0/701414563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81413e360 0x7fe8140cf290 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fe81406ded0 tx=0x7fe7fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b79e900937cb17 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.174+0000 7fe813400640 1 -- 172.21.15.26:0/701414563 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc031640 con 0x7fe81413e360 2024-01-15T18:16:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.174+0000 7fe813400640 1 -- 172.21.15.26:0/701414563 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc0317a0 con 0x7fe81413e360 2024-01-15T18:16:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.174+0000 7fe813400640 1 -- 172.21.15.26:0/701414563 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc031b30 con 0x7fe81413e360 2024-01-15T18:16:02.177 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.175+0000 7fe81aa00640 1 -- 172.21.15.26:0/701414563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81413e360 msgr2=0x7fe8140cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.175+0000 7fe81aa00640 1 --2- 172.21.15.26:0/701414563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81413e360 0x7fe8140cf290 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fe81406ded0 tx=0x7fe7fc02b8f0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.175+0000 7fe81aa00640 1 -- 172.21.15.26:0/701414563 shutdown_connections 2024-01-15T18:16:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.175+0000 7fe81aa00640 1 --2- 172.21.15.26:0/701414563 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81413e360 0x7fe8140cf290 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.175+0000 7fe81aa00640 1 --2- 172.21.15.26:0/701414563 >> v2:172.21.15.124:3300/0 conn(0x7fe81415f9c0 0x7fe8140cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.175+0000 7fe81aa00640 1 -- 172.21.15.26:0/701414563 >> 172.21.15.26:0/701414563 conn(0x7fe81406a800 msgr2=0x7fe81406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.175+0000 7fe81aa00640 1 -- 172.21.15.26:0/701414563 shutdown_connections 2024-01-15T18:16:02.178 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.175+0000 7fe81aa00640 1 -- 172.21.15.26:0/701414563 wait complete. 2024-01-15T18:16:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.177+0000 7fe81aa00640 1 Processor -- start 2024-01-15T18:16:02.179 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.177+0000 7fe81aa00640 1 -- start start 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.178+0000 7fe81aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81413df80 0x7fe814150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.178+0000 7fe81aa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe81413e360 0x7fe81413d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.178+0000 7fe81aa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe81415f9c0 0x7fe8141f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.178+0000 7fe81aa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81414faa0 con 0x7fe81413df80 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.178+0000 7fe81aa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8141f1a10 con 0x7fe81415f9c0 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.178+0000 7fe81aa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8141514a0 con 0x7fe81413e360 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.178+0000 7fe813e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81413df80 0x7fe814150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.178+0000 7fe813e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81413df80 0x7fe814150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58756/0 (socket says 172.21.15.26:58756) 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.178+0000 7fe813e00640 1 -- 172.21.15.26:0/2469035643 learned_addr learned my addr 172.21.15.26:0/2469035643 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:02.181 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.179+0000 7fe818a00640 1 --2- 172.21.15.26:0/2469035643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe81415f9c0 0x7fe8141f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.182 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.179+0000 7fe80be00640 1 --2- 172.21.15.26:0/2469035643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe81413e360 0x7fe81413d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.179+0000 7fe813e00640 1 -- 172.21.15.26:0/2469035643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe81413e360 msgr2=0x7fe81413d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.179+0000 7fe813e00640 1 --2- 172.21.15.26:0/2469035643 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fe81413e360 0x7fe81413d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.179+0000 7fe813e00640 1 -- 172.21.15.26:0/2469035643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe81415f9c0 msgr2=0x7fe8141f0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.179+0000 7fe813e00640 1 --2- 172.21.15.26:0/2469035643 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fe81415f9c0 0x7fe8141f0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.179+0000 7fe813e00640 1 -- 172.21.15.26:0/2469035643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc0094d0 con 0x7fe81413df80 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.179+0000 7fe813e00640 1 --2- 172.21.15.26:0/2469035643 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fe81413df80 0x7fe814150f60 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe80400b420 tx=0x7fe80400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.180+0000 7fe810c00640 1 -- 172.21.15.26:0/2469035643 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804011830 con 0x7fe81413df80 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.180+0000 7fe810c00640 1 -- 172.21.15.26:0/2469035643 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe804011990 con 0x7fe81413df80 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.180+0000 7fe81aa00640 1 -- 172.21.15.26:0/2469035643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8141f1dd0 con 0x7fe81413df80 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.180+0000 7fe810c00640 1 -- 172.21.15.26:0/2469035643 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804011d40 con 0x7fe81413df80 2024-01-15T18:16:02.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.180+0000 7fe81aa00640 1 -- 172.21.15.26:0/2469035643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8141f2390 con 0x7fe81413df80 2024-01-15T18:16:02.184 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.181+0000 7fe81aa00640 1 -- 172.21.15.26:0/2469035643 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8140675d0 con 0x7fe81413df80 2024-01-15T18:16:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.182+0000 7fe810c00640 1 -- 172.21.15.26:0/2469035643 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fe80401b460 con 0x7fe81413df80 2024-01-15T18:16:02.185 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.183+0000 7fe810c00640 1 --2- 172.21.15.26:0/2469035643 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe7ec06c670 0x7fe7ec06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.183+0000 7fe810c00640 1 -- 172.21.15.26:0/2469035643 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fe804088400 con 0x7fe81413df80 2024-01-15T18:16:02.186 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.184+0000 7fe80be00640 1 --2- 172.21.15.26:0/2469035643 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe7ec06c670 0x7fe7ec06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:02.187 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.185+0000 7fe80be00640 1 --2- 172.21.15.26:0/2469035643 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fe7ec06c670 0x7fe7ec06eb30 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc036000 tx=0x7fe7fc03c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:02.190 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.188+0000 7fe810c00640 1 -- 172.21.15.26:0/2469035643 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe804054cc0 con 0x7fe81413df80 2024-01-15T18:16:02.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.416+0000 7fe81aa00640 1 -- 172.21.15.26:0/2469035643 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fe81406cff0 con 0x7fe7ec06c670 2024-01-15T18:16:02.440 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:02.437+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:02.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:02.438+0000 7fe810c00640 1 -- 172.21.15.26:0/2469035643 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fe7fc031b30 con 0x7fe7ec06c670 2024-01-15T18:16:02.440 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:02.462 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:02.795 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:16:02.803 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:16:04.464 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:04.701 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.698+0000 7f2ae6800640 1 Processor -- start 2024-01-15T18:16:04.714 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.698+0000 7f2ae6800640 1 -- start start 2024-01-15T18:16:04.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.700+0000 7f2ae6800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00ae5a0 0x7f2ae015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:04.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.700+0000 7f2ae6800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2ae00cf290 0x7f2ae00cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:04.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.700+0000 7f2ae6800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae0070c40 con 0x7f2ae00b3ae0 2024-01-15T18:16:04.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.700+0000 7f2ae6800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae0071450 con 0x7f2ae00cf290 2024-01-15T18:16:04.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.700+0000 7f2ae6800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae00cfc60 con 0x7f2ae00ae5a0 2024-01-15T18:16:04.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.701+0000 7f2adf400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00ae5a0 0x7f2ae015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.701+0000 7f2adf400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00ae5a0 0x7f2ae015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:38880/0 (socket says 172.21.15.26:38880) 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.701+0000 7f2adf400640 1 -- 172.21.15.26:0/1405353544 learned_addr learned my addr 172.21.15.26:0/1405353544 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.701+0000 7f2adfe00640 1 --2- 172.21.15.26:0/1405353544 >> v2:172.21.15.124:3300/0 conn(0x7f2ae00cf290 0x7f2ae00cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.702+0000 7f2adf400640 1 -- 172.21.15.26:0/1405353544 >> v2:172.21.15.124:3300/0 conn(0x7f2ae00cf290 msgr2=0x7f2ae00cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.702+0000 7f2adf400640 1 --2- 172.21.15.26:0/1405353544 >> v2:172.21.15.124:3300/0 conn(0x7f2ae00cf290 0x7f2ae00cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.702+0000 7f2adf400640 1 -- 172.21.15.26:0/1405353544 >> v1:172.21.15.124:6789/0 conn(0x7f2ae00b3ae0 legacy=0x7f2ae006cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.702+0000 7f2adf400640 1 -- 172.21.15.26:0/1405353544 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae006c470 con 0x7f2ae00ae5a0 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.702+0000 7f2adea00640 1 -- 172.21.15.26:0/1405353544 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481074153 0 0) 0x7f2ae0070c40 con 0x7f2ae00b3ae0 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.703+0000 7f2adf400640 1 --2- 172.21.15.26:0/1405353544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00ae5a0 0x7f2ae015e430 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009e50 tx=0x7f2ad002bfe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=338a6b98bb131d65 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.703+0000 7f2adea00640 1 -- 172.21.15.26:0/1405353544 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad002cb80 con 0x7f2ae00ae5a0 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.703+0000 7f2adea00640 1 -- 172.21.15.26:0/1405353544 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad002cce0 con 0x7f2ae00ae5a0 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.704+0000 7f2adea00640 1 -- 172.21.15.26:0/1405353544 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad0034610 con 0x7f2ae00ae5a0 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.704+0000 7f2ae6800640 1 -- 172.21.15.26:0/1405353544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00ae5a0 msgr2=0x7f2ae015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:04.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.704+0000 7f2ae6800640 1 --2- 172.21.15.26:0/1405353544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00ae5a0 0x7f2ae015e430 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009e50 tx=0x7f2ad002bfe0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.705+0000 7f2ae6800640 1 -- 172.21.15.26:0/1405353544 shutdown_connections 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.705+0000 7f2ae6800640 1 --2- 172.21.15.26:0/1405353544 >> v2:172.21.15.124:3300/0 conn(0x7f2ae00cf290 0x7f2ae00cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.705+0000 7f2ae6800640 1 --2- 172.21.15.26:0/1405353544 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00ae5a0 0x7f2ae015e430 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.705+0000 7f2ae6800640 1 -- 172.21.15.26:0/1405353544 >> 172.21.15.26:0/1405353544 conn(0x7f2ae006a800 msgr2=0x7f2ae006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.705+0000 7f2ae6800640 1 -- 172.21.15.26:0/1405353544 shutdown_connections 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.705+0000 7f2ae6800640 1 -- 172.21.15.26:0/1405353544 wait complete. 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.707+0000 7f2ae6800640 1 Processor -- start 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.707+0000 7f2ae6800640 1 -- start start 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2ae6800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae00ae5a0 0x7f2ae014f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2ae6800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00b3ae0 0x7f2ae01f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2ae6800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae00cf290 0x7f2ae01f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2ae6800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae01533a0 con 0x7f2ae00cf290 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2ae6800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae0160710 con 0x7f2ae00ae5a0 2024-01-15T18:16:04.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2ae6800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae015b600 con 0x7f2ae00b3ae0 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2adfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae00cf290 0x7f2ae01f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2adfe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae00cf290 0x7f2ae01f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58762/0 (socket says 172.21.15.26:58762) 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2adfe00640 1 -- 172.21.15.26:0/4033875972 learned_addr learned my addr 172.21.15.26:0/4033875972 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.709+0000 7f2adf400640 1 --2- 172.21.15.26:0/4033875972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae00ae5a0 0x7f2ae014f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.710+0000 7f2ad6800640 1 --2- 172.21.15.26:0/4033875972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00b3ae0 0x7f2ae01f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.710+0000 7f2adfe00640 1 -- 172.21.15.26:0/4033875972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00b3ae0 msgr2=0x7f2ae01f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.710+0000 7f2adfe00640 1 --2- 172.21.15.26:0/4033875972 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2ae00b3ae0 0x7f2ae01f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.710+0000 7f2adfe00640 1 -- 172.21.15.26:0/4033875972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae00ae5a0 msgr2=0x7f2ae014f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.710+0000 7f2adfe00640 1 --2- 172.21.15.26:0/4033875972 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2ae00ae5a0 0x7f2ae014f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.710+0000 7f2adfe00640 1 -- 172.21.15.26:0/4033875972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad0009b30 con 0x7f2ae00cf290 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.710+0000 7f2adfe00640 1 --2- 172.21.15.26:0/4033875972 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2ae00cf290 0x7f2ae01f6720 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f2ad800e7f0 tx=0x7f2ad800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.711+0000 7f2ad7e00640 1 -- 172.21.15.26:0/4033875972 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8009510 con 0x7f2ae00cf290 2024-01-15T18:16:04.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.711+0000 7f2ad7e00640 1 -- 172.21.15.26:0/4033875972 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad800ee40 con 0x7f2ae00cf290 2024-01-15T18:16:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.711+0000 7f2ad7e00640 1 -- 172.21.15.26:0/4033875972 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad800cb80 con 0x7f2ae00cf290 2024-01-15T18:16:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.711+0000 7f2ae6800640 1 -- 172.21.15.26:0/4033875972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae015b8e0 con 0x7f2ae00cf290 2024-01-15T18:16:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.711+0000 7f2ae6800640 1 -- 172.21.15.26:0/4033875972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae01609f0 con 0x7f2ae00cf290 2024-01-15T18:16:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.712+0000 7f2ae6800640 1 -- 172.21.15.26:0/4033875972 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae00675d0 con 0x7f2ae00cf290 2024-01-15T18:16:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.713+0000 7f2ad7e00640 1 -- 172.21.15.26:0/4033875972 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f2ad800cce0 con 0x7f2ae00cf290 2024-01-15T18:16:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.714+0000 7f2ad7e00640 1 --2- 172.21.15.26:0/4033875972 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ab806c670 0x7f2ab806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.714+0000 7f2ad7e00640 1 -- 172.21.15.26:0/4033875972 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f2ad8088c30 con 0x7f2ae00cf290 2024-01-15T18:16:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.715+0000 7f2adf400640 1 --2- 172.21.15.26:0/4033875972 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ab806c670 0x7f2ab806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:04.722 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.716+0000 7f2adf400640 1 --2- 172.21.15.26:0/4033875972 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2ab806c670 0x7f2ab806eb30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0035000 tx=0x7f2ad0030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:04.723 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.719+0000 7f2ad7e00640 1 -- 172.21.15.26:0/4033875972 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ad80551f0 con 0x7f2ae00cf290 2024-01-15T18:16:04.944 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.941+0000 7f2ae6800640 1 -- 172.21.15.26:0/4033875972 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ae0071170 con 0x7f2ab806c670 2024-01-15T18:16:04.962 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:04.959+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:04.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:04.960+0000 7f2ad7e00640 1 -- 172.21.15.26:0/4033875972 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f2ad003f440 con 0x7f2ab806c670 2024-01-15T18:16:04.963 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:04.976 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:06.977 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.211+0000 7f8fdaa00640 1 Processor -- start 2024-01-15T18:16:07.214 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.211+0000 7f8fdaa00640 1 -- start start 2024-01-15T18:16:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.213+0000 7f8fdaa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8fd40cc990 0x7f8fd406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.213+0000 7f8fdaa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd4070e50 con 0x7f8fd40cc500 2024-01-15T18:16:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.213+0000 7f8fdaa00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd4144020 con 0x7f8fd4143c40 2024-01-15T18:16:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.213+0000 7f8fdaa00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fd40723c0 con 0x7f8fd40cc990 2024-01-15T18:16:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.213+0000 7f8fd8c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f8fd40cc500 0x7f8fd406d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33408/0 (socket says 172.21.15.26:33408) 2024-01-15T18:16:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.213+0000 7f8fd8c00640 1 -- 172.21.15.26:0/3924311336 learned_addr learned my addr 172.21.15.26:0/3924311336 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:07.216 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.213+0000 7f8fda000640 1 --2- 172.21.15.26:0/3924311336 >> v2:172.21.15.26:3300/0 conn(0x7f8fd40cc990 0x7f8fd406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.214+0000 7f8fd3e00640 1 -- 172.21.15.26:0/3924311336 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257615236 0 0) 0x7f8fd4070e50 con 0x7f8fd40cc500 2024-01-15T18:16:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.214+0000 7f8fd3e00640 1 -- 172.21.15.26:0/3924311336 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb4003520 con 0x7f8fd40cc500 2024-01-15T18:16:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.214+0000 7f8fda000640 1 -- 172.21.15.26:0/3924311336 >> v1:172.21.15.124:6789/0 conn(0x7f8fd4143c40 legacy=0x7f8fd4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.214+0000 7f8fda000640 1 -- 172.21.15.26:0/3924311336 >> v1:172.21.15.26:6789/0 conn(0x7f8fd40cc500 legacy=0x7f8fd406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.217 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.214+0000 7f8fda000640 1 -- 172.21.15.26:0/3924311336 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd4071e00 con 0x7f8fd40cc990 2024-01-15T18:16:07.218 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.216+0000 7f8fda000640 1 --2- 172.21.15.26:0/3924311336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd40cc990 0x7f8fd406d870 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009d90 tx=0x7f8fcc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb1d2f0485c566 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.216+0000 7f8fd3e00640 1 -- 172.21.15.26:0/3924311336 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc02cc60 con 0x7f8fd40cc990 2024-01-15T18:16:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.216+0000 7f8fd3e00640 1 -- 172.21.15.26:0/3924311336 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc02cdc0 con 0x7f8fd40cc990 2024-01-15T18:16:07.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.216+0000 7f8fd3e00640 1 -- 172.21.15.26:0/3924311336 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc034660 con 0x7f8fd40cc990 2024-01-15T18:16:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.217+0000 7f8fdaa00640 1 -- 172.21.15.26:0/3924311336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd40cc990 msgr2=0x7f8fd406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.217+0000 7f8fdaa00640 1 --2- 172.21.15.26:0/3924311336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd40cc990 0x7f8fd406d870 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009d90 tx=0x7f8fcc02beb0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.217+0000 7f8fdaa00640 1 -- 172.21.15.26:0/3924311336 shutdown_connections 2024-01-15T18:16:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.217+0000 7f8fdaa00640 1 --2- 172.21.15.26:0/3924311336 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd40cc990 0x7f8fd406d870 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.217+0000 7f8fdaa00640 1 -- 172.21.15.26:0/3924311336 >> 172.21.15.26:0/3924311336 conn(0x7f8fd406a800 msgr2=0x7f8fd406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.217+0000 7f8fdaa00640 1 -- 172.21.15.26:0/3924311336 shutdown_connections 2024-01-15T18:16:07.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.218+0000 7f8fdaa00640 1 -- 172.21.15.26:0/3924311336 wait complete. 2024-01-15T18:16:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.219+0000 7f8fdaa00640 1 Processor -- start 2024-01-15T18:16:07.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.219+0000 7f8fdaa00640 1 -- start start 2024-01-15T18:16:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.220+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8fd40cc500 0x7f8fd40add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.220+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd40cc990 0x7f8fd40b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.220+0000 7f8fdaa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fd4143c40 0x7f8fd40b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.220+0000 7f8fdaa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd41f36a0 con 0x7f8fd40cc990 2024-01-15T18:16:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.220+0000 7f8fdaa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4141b40 con 0x7f8fd40cc500 2024-01-15T18:16:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.220+0000 7f8fdaa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd415a890 con 0x7f8fd4143c40 2024-01-15T18:16:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fd8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd40cc990 0x7f8fd40b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.224 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fd8c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd40cc990 0x7f8fd40b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58786/0 (socket says 172.21.15.26:58786) 2024-01-15T18:16:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fd8c00640 1 -- 172.21.15.26:0/2930536571 learned_addr learned my addr 172.21.15.26:0/2930536571 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fd9600640 1 --2- 172.21.15.26:0/2930536571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8fd40cc500 0x7f8fd40add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fda000640 1 --2- 172.21.15.26:0/2930536571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fd4143c40 0x7f8fd40b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fd8c00640 1 -- 172.21.15.26:0/2930536571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fd4143c40 msgr2=0x7f8fd40b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fd8c00640 1 --2- 172.21.15.26:0/2930536571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8fd4143c40 0x7f8fd40b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fd8c00640 1 -- 172.21.15.26:0/2930536571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8fd40cc500 msgr2=0x7f8fd40add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fd8c00640 1 --2- 172.21.15.26:0/2930536571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8fd40cc500 0x7f8fd40add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.221+0000 7f8fd8c00640 1 -- 172.21.15.26:0/2930536571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc009a70 con 0x7f8fd40cc990 2024-01-15T18:16:07.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.222+0000 7f8fd8c00640 1 --2- 172.21.15.26:0/2930536571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8fd40cc990 0x7f8fd40b3390 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c530 tx=0x7f8fbc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.222+0000 7f8fd1600640 1 -- 172.21.15.26:0/2930536571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc005ce0 con 0x7f8fd40cc990 2024-01-15T18:16:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.222+0000 7f8fd1600640 1 -- 172.21.15.26:0/2930536571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc005e40 con 0x7f8fd40cc990 2024-01-15T18:16:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.222+0000 7f8fd1600640 1 -- 172.21.15.26:0/2930536571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0072b0 con 0x7f8fd40cc990 2024-01-15T18:16:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.222+0000 7f8fdaa00640 1 -- 172.21.15.26:0/2930536571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd40ae2c0 con 0x7f8fd40cc990 2024-01-15T18:16:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.222+0000 7f8fdaa00640 1 -- 172.21.15.26:0/2930536571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd41f38f0 con 0x7f8fd40cc990 2024-01-15T18:16:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.223+0000 7f8fdaa00640 1 -- 172.21.15.26:0/2930536571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd4066e70 con 0x7f8fd40cc990 2024-01-15T18:16:07.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.224+0000 7f8fd1600640 1 -- 172.21.15.26:0/2930536571 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f8fbc007410 con 0x7f8fd40cc990 2024-01-15T18:16:07.227 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.225+0000 7f8fd1600640 1 --2- 172.21.15.26:0/2930536571 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8fac06c670 0x7f8fac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:07.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.225+0000 7f8fd1600640 1 -- 172.21.15.26:0/2930536571 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f8fbc088210 con 0x7f8fd40cc990 2024-01-15T18:16:07.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.226+0000 7f8fd9600640 1 --2- 172.21.15.26:0/2930536571 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8fac06c670 0x7f8fac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:07.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.227+0000 7f8fd9600640 1 --2- 172.21.15.26:0/2930536571 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8fac06c670 0x7f8fac06eb30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4000f60 tx=0x7f8fc4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:07.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.230+0000 7f8fd1600640 1 -- 172.21.15.26:0/2930536571 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8fbc08f030 con 0x7f8fd40cc990 2024-01-15T18:16:07.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.453+0000 7f8fdaa00640 1 -- 172.21.15.26:0/2930536571 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fd406cc40 con 0x7f8fac06c670 2024-01-15T18:16:07.479 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:07.476+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:07.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:07.477+0000 7f8fd1600640 1 -- 172.21.15.26:0/2930536571 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f8fc40050d0 con 0x7f8fac06c670 2024-01-15T18:16:07.479 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:07.491 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:09.492 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.725+0000 7fb527200640 1 Processor -- start 2024-01-15T18:16:09.728 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.725+0000 7fb527200640 1 -- start start 2024-01-15T18:16:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.726+0000 7fb527200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb528071410 0x7fb528146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.726+0000 7fb527200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fb52806c7c0 0x7fb5280717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb527200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52806cc50 con 0x7fb528071d30 2024-01-15T18:16:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb527200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52806cec0 con 0x7fb528071410 2024-01-15T18:16:09.729 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb527200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52806d1a0 con 0x7fb52806c7c0 2024-01-15T18:16:09.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb525e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb528071410 0x7fb528146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:09.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb526800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb528071d30 0x7fb5280b1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:33414/0 (socket says 172.21.15.26:33414) 2024-01-15T18:16:09.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb526800640 1 -- 172.21.15.26:0/3712171465 learned_addr learned my addr 172.21.15.26:0/3712171465 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:09.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb51fe00640 1 --2- 172.21.15.26:0/3712171465 >> v2:172.21.15.124:3300/0 conn(0x7fb52806c7c0 0x7fb5280717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:09.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb525e00640 1 -- 172.21.15.26:0/3712171465 >> v2:172.21.15.124:3300/0 conn(0x7fb52806c7c0 msgr2=0x7fb5280717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:09.730 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb525e00640 1 --2- 172.21.15.26:0/3712171465 >> v2:172.21.15.124:3300/0 conn(0x7fb52806c7c0 0x7fb5280717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:09.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb525e00640 1 -- 172.21.15.26:0/3712171465 >> v1:172.21.15.26:6789/0 conn(0x7fb528071d30 legacy=0x7fb5280b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:09.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.728+0000 7fb525e00640 1 -- 172.21.15.26:0/3712171465 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52806d480 con 0x7fb528071410 2024-01-15T18:16:09.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.727+0000 7fb525400640 1 -- 172.21.15.26:0/3712171465 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413980600 0 0) 0x7fb52806cc50 con 0x7fb528071d30 2024-01-15T18:16:09.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.729+0000 7fb525e00640 1 --2- 172.21.15.26:0/3712171465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb528071410 0x7fb528146770 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb528146ef0 tx=0x7fb51802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3241d6c0b7ed8da4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:09.731 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.729+0000 7fb525400640 1 -- 172.21.15.26:0/3712171465 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518031830 con 0x7fb528071410 2024-01-15T18:16:09.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.729+0000 7fb525400640 1 -- 172.21.15.26:0/3712171465 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb518031990 con 0x7fb528071410 2024-01-15T18:16:09.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.729+0000 7fb525400640 1 -- 172.21.15.26:0/3712171465 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518031d20 con 0x7fb528071410 2024-01-15T18:16:09.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.729+0000 7fb527200640 1 -- 172.21.15.26:0/3712171465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb528071410 msgr2=0x7fb528146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.729+0000 7fb527200640 1 --2- 172.21.15.26:0/3712171465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb528071410 0x7fb528146770 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb528146ef0 tx=0x7fb51802b8d0 comp rx=0 tx=0).stop 2024-01-15T18:16:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.730+0000 7fb527200640 1 -- 172.21.15.26:0/3712171465 shutdown_connections 2024-01-15T18:16:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.730+0000 7fb527200640 1 --2- 172.21.15.26:0/3712171465 >> v2:172.21.15.124:3300/0 conn(0x7fb52806c7c0 0x7fb5280717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.730+0000 7fb527200640 1 --2- 172.21.15.26:0/3712171465 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb528071410 0x7fb528146770 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.730+0000 7fb527200640 1 -- 172.21.15.26:0/3712171465 >> 172.21.15.26:0/3712171465 conn(0x7fb52806a800 msgr2=0x7fb52806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.730+0000 7fb527200640 1 -- 172.21.15.26:0/3712171465 shutdown_connections 2024-01-15T18:16:09.733 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.730+0000 7fb527200640 1 -- 172.21.15.26:0/3712171465 wait complete. 2024-01-15T18:16:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.732+0000 7fb527200640 1 Processor -- start 2024-01-15T18:16:09.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.732+0000 7fb527200640 1 -- start start 2024-01-15T18:16:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb527200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52806c7c0 0x7fb52815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb527200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb528071410 0x7fb52813ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:09.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb527200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb528071d30 0x7fb52813d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb527200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52813c040 con 0x7fb52806c7c0 2024-01-15T18:16:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb527200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52813cea0 con 0x7fb528071d30 2024-01-15T18:16:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb527200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52815cec0 con 0x7fb528071410 2024-01-15T18:16:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb525e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52806c7c0 0x7fb52815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb525e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52806c7c0 0x7fb52815c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:58794/0 (socket says 172.21.15.26:58794) 2024-01-15T18:16:09.736 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb525e00640 1 -- 172.21.15.26:0/3361448125 learned_addr learned my addr 172.21.15.26:0/3361448125 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb51fe00640 1 --2- 172.21.15.26:0/3361448125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb528071410 0x7fb52813ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb525e00640 1 -- 172.21.15.26:0/3361448125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb528071410 msgr2=0x7fb52813ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb525e00640 1 --2- 172.21.15.26:0/3361448125 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb528071410 0x7fb52813ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb526800640 1 --2- 172.21.15.26:0/3361448125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb528071d30 0x7fb52813d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb525e00640 1 -- 172.21.15.26:0/3361448125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb528071d30 msgr2=0x7fb52813d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:09.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb525e00640 1 --2- 172.21.15.26:0/3361448125 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb528071d30 0x7fb52813d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.733+0000 7fb525e00640 1 -- 172.21.15.26:0/3361448125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518009400 con 0x7fb52806c7c0 2024-01-15T18:16:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.734+0000 7fb525e00640 1 --2- 172.21.15.26:0/3361448125 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb52806c7c0 0x7fb52815c980 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fb518009890 tx=0x7fb5180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.734+0000 7fb51e000640 1 -- 172.21.15.26:0/3361448125 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518004230 con 0x7fb52806c7c0 2024-01-15T18:16:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.734+0000 7fb51e000640 1 -- 172.21.15.26:0/3361448125 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb518004390 con 0x7fb52806c7c0 2024-01-15T18:16:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.734+0000 7fb51e000640 1 -- 172.21.15.26:0/3361448125 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51803a920 con 0x7fb52806c7c0 2024-01-15T18:16:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.734+0000 7fb527200640 1 -- 172.21.15.26:0/3361448125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52813c290 con 0x7fb52806c7c0 2024-01-15T18:16:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.734+0000 7fb527200640 1 -- 172.21.15.26:0/3361448125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52813c850 con 0x7fb52806c7c0 2024-01-15T18:16:09.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.735+0000 7fb527200640 1 -- 172.21.15.26:0/3361448125 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb528067450 con 0x7fb52806c7c0 2024-01-15T18:16:09.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.736+0000 7fb51e000640 1 -- 172.21.15.26:0/3361448125 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fb51803aa80 con 0x7fb52806c7c0 2024-01-15T18:16:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.737+0000 7fb51e000640 1 --2- 172.21.15.26:0/3361448125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb50006c670 0x7fb50006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.737+0000 7fb51e000640 1 -- 172.21.15.26:0/3361448125 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fb5180ab2e0 con 0x7fb52806c7c0 2024-01-15T18:16:09.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.738+0000 7fb51fe00640 1 --2- 172.21.15.26:0/3361448125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb50006c670 0x7fb50006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:09.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.739+0000 7fb51fe00640 1 --2- 172.21.15.26:0/3361448125 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb50006c670 0x7fb50006eb30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb5280ce880 tx=0x7fb510008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:09.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.742+0000 7fb51e000640 1 -- 172.21.15.26:0/3361448125 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb518077850 con 0x7fb52806c7c0 2024-01-15T18:16:09.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.959+0000 7fb527200640 1 -- 172.21.15.26:0/3361448125 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fb528072180 con 0x7fb50006c670 2024-01-15T18:16:09.981 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:09.978+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:09.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:09.979+0000 7fb51e000640 1 -- 172.21.15.26:0/3361448125 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fb51000c040 con 0x7fb50006c670 2024-01-15T18:16:09.981 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:09.997 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:11.998 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:12.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.228+0000 7f907fe00640 1 Processor -- start 2024-01-15T18:16:12.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.228+0000 7f907fe00640 1 -- start start 2024-01-15T18:16:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.230+0000 7f907fe00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9080143c40 0x7f9080071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.230+0000 7f907fe00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9080070e50 con 0x7f90800ccc10 2024-01-15T18:16:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.230+0000 7f907fe00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9080144020 con 0x7f90800717d0 2024-01-15T18:16:12.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.230+0000 7f907fe00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90800723c0 con 0x7f9080143c40 2024-01-15T18:16:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.231+0000 7f907f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f90800ccc10 0x7f908006d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38896/0 (socket says 172.21.15.26:38896) 2024-01-15T18:16:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.231+0000 7f907f400640 1 -- 172.21.15.26:0/3801045051 learned_addr learned my addr 172.21.15.26:0/3801045051 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.231+0000 7f907ea00640 1 --2- 172.21.15.26:0/3801045051 >> v2:172.21.15.124:3300/0 conn(0x7f9080143c40 0x7f9080071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.231+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386165484 0 0) 0x7f9080070e50 con 0x7f90800ccc10 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.231+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9060003520 con 0x7f90800ccc10 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.231+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941496099 0 0) 0x7f9060003520 con 0x7f90800ccc10 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 >> v2:172.21.15.124:3300/0 conn(0x7f9080143c40 msgr2=0x7f9080071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 --2- 172.21.15.26:0/3801045051 >> v2:172.21.15.124:3300/0 conn(0x7f9080143c40 0x7f9080071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 >> v1:172.21.15.124:6789/0 conn(0x7f90800717d0 legacy=0x7f90800cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9080071e00 con 0x7f90800ccc10 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f90780019f0 con 0x7f90800ccc10 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 >> v1:172.21.15.26:6789/0 conn(0x7f90800ccc10 legacy=0x7f908006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90600039c0 0x7f9060003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90600047c0 0x7f9060004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90600051d0 0x7f906000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90800723c0 con 0x7f90600051d0 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906000c120 con 0x7f90600039c0 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906000c2e0 con 0x7f90600047c0 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907f400640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90600051d0 0x7f906000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f907ea00640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90600039c0 0x7f9060003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.232+0000 7f9077e00640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90600047c0 0x7f9060004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.233+0000 7f907f400640 1 -- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90600047c0 msgr2=0x7f9060004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.233+0000 7f907f400640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90600047c0 0x7f9060004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.233+0000 7f907f400640 1 -- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90600039c0 msgr2=0x7f9060003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.233+0000 7f907f400640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90600039c0 0x7f9060003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.233+0000 7f907f400640 1 -- 172.21.15.26:0/3801045051 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060003520 con 0x7f90600051d0 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.233+0000 7f907ea00640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90600039c0 0x7f9060003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:12.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.233+0000 7f9077e00640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f90600047c0 0x7f9060004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.234+0000 7f907f400640 1 -- 172.21.15.26:0/3801045051 reap_dead start 2024-01-15T18:16:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.234+0000 7f907f400640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90600051d0 0x7f906000b980 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f9078002230 tx=0x7f90600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50a848856df41ea server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.234+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9078034630 con 0x7f90600051d0 2024-01-15T18:16:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.234+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907802f070 con 0x7f90600051d0 2024-01-15T18:16:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.234+0000 7f907e000640 1 -- 172.21.15.26:0/3801045051 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90780349d0 con 0x7f90600051d0 2024-01-15T18:16:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.235+0000 7f907fe00640 1 -- 172.21.15.26:0/3801045051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90600051d0 msgr2=0x7f906000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.235+0000 7f907fe00640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90600051d0 0x7f906000b980 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f9078002230 tx=0x7f90600057f0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.235+0000 7f907fe00640 1 -- 172.21.15.26:0/3801045051 shutdown_connections 2024-01-15T18:16:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.235+0000 7f907fe00640 1 --2- 172.21.15.26:0/3801045051 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f90600051d0 0x7f906000b980 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.235+0000 7f907fe00640 1 -- 172.21.15.26:0/3801045051 >> 172.21.15.26:0/3801045051 conn(0x7f908006a800 msgr2=0x7f908006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.235+0000 7f907fe00640 1 -- 172.21.15.26:0/3801045051 shutdown_connections 2024-01-15T18:16:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.235+0000 7f907fe00640 1 -- 172.21.15.26:0/3801045051 wait complete. 2024-01-15T18:16:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.237+0000 7f907fe00640 1 Processor -- start 2024-01-15T18:16:12.240 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.237+0000 7f907fe00640 1 -- start start 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f907fe00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90600051d0 0x7f90801395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f907fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f908015a7d0 0x7f908013a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f907fe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f908015ac50 0x7f908013a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f907fe00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90801f16b0 con 0x7f908015a7d0 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f907fe00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90801f13d0 con 0x7f90600051d0 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f907fe00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9080138b00 con 0x7f908015ac50 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f9077e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f908015a7d0 0x7f908013a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f9077e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f908015a7d0 0x7f908013a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60820/0 (socket says 172.21.15.26:60820) 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f9077e00640 1 -- 172.21.15.26:0/562447263 learned_addr learned my addr 172.21.15.26:0/562447263 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f907ea00640 1 --2- 172.21.15.26:0/562447263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90600051d0 0x7f90801395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f907f400640 1 --2- 172.21.15.26:0/562447263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f908015ac50 0x7f908013a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f9077e00640 1 -- 172.21.15.26:0/562447263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f908015ac50 msgr2=0x7f908013a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f9077e00640 1 --2- 172.21.15.26:0/562447263 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f908015ac50 0x7f908013a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f9077e00640 1 -- 172.21.15.26:0/562447263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90600051d0 msgr2=0x7f90801395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f9077e00640 1 --2- 172.21.15.26:0/562447263 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f90600051d0 0x7f90801395e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.238+0000 7f9077e00640 1 -- 172.21.15.26:0/562447263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9080138de0 con 0x7f908015a7d0 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.239+0000 7f9077e00640 1 --2- 172.21.15.26:0/562447263 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f908015a7d0 0x7f908013a330 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f9068000f60 tx=0x7f906800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.239+0000 7f9076a00640 1 -- 172.21.15.26:0/562447263 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9068005480 con 0x7f908015a7d0 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.239+0000 7f9076a00640 1 -- 172.21.15.26:0/562447263 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9068003610 con 0x7f908015a7d0 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.239+0000 7f907fe00640 1 -- 172.21.15.26:0/562447263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90801390c0 con 0x7f908015a7d0 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.239+0000 7f9076a00640 1 -- 172.21.15.26:0/562447263 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90680039c0 con 0x7f908015a7d0 2024-01-15T18:16:12.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.239+0000 7f907fe00640 1 -- 172.21.15.26:0/562447263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90801f0830 con 0x7f908015a7d0 2024-01-15T18:16:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.240+0000 7f907fe00640 1 -- 172.21.15.26:0/562447263 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9080138960 con 0x7f908015a7d0 2024-01-15T18:16:12.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.241+0000 7f9076a00640 1 -- 172.21.15.26:0/562447263 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f9068003b20 con 0x7f908015a7d0 2024-01-15T18:16:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.242+0000 7f9076a00640 1 --2- 172.21.15.26:0/562447263 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f905806c670 0x7f905806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.242+0000 7f9076a00640 1 -- 172.21.15.26:0/562447263 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f9068088ad0 con 0x7f908015a7d0 2024-01-15T18:16:12.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.243+0000 7f907ea00640 1 --2- 172.21.15.26:0/562447263 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f905806c670 0x7f905806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:12.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.244+0000 7f907ea00640 1 --2- 172.21.15.26:0/562447263 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f905806c670 0x7f905806eb30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f90800707a0 tx=0x7f9070008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:12.250 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.248+0000 7f9076a00640 1 -- 172.21.15.26:0/562447263 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9080138960 con 0x7f908015a7d0 2024-01-15T18:16:12.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.475+0000 7f907fe00640 1 -- 172.21.15.26:0/562447263 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f908006cc40 con 0x7f905806c670 2024-01-15T18:16:12.500 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:12.497+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:12.501 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:12.498+0000 7f9076a00640 1 -- 172.21.15.26:0/562447263 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f90700118e0 con 0x7f905806c670 2024-01-15T18:16:12.501 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:12.520 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:14.521 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:14.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.757+0000 7f5c9aa00640 1 Processor -- start 2024-01-15T18:16:14.760 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.757+0000 7f5c9aa00640 1 -- start start 2024-01-15T18:16:14.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.759+0000 7f5c9aa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 0x7f5c94146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.759+0000 7f5c9aa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5c94071f00 0x7f5c94166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.759+0000 7f5c9aa00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c940b2e10 con 0x7f5c9406c7c0 2024-01-15T18:16:14.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.759+0000 7f5c9aa00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9406cc20 con 0x7f5c94071f00 2024-01-15T18:16:14.762 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.759+0000 7f5c9aa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9406cec0 con 0x7f5c94071410 2024-01-15T18:16:14.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.760+0000 7f5c99600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 0x7f5c94146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.760+0000 7f5c9a000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5c94071f00 0x7f5c94166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.760+0000 7f5c98c00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f5c9406c7c0 0x7f5c940717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:50286/0 (socket says 172.21.15.26:50286) 2024-01-15T18:16:14.763 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.760+0000 7f5c98c00640 1 -- 172.21.15.26:0/2679449242 learned_addr learned my addr 172.21.15.26:0/2679449242 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.761+0000 7f5c99600640 1 -- 172.21.15.26:0/2679449242 >> v2:172.21.15.124:3300/0 conn(0x7f5c94071f00 msgr2=0x7f5c94166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.761+0000 7f5c93e00640 1 -- 172.21.15.26:0/2679449242 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174496910 0 0) 0x7f5c940b2e10 con 0x7f5c9406c7c0 2024-01-15T18:16:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.761+0000 7f5c99600640 1 --2- 172.21.15.26:0/2679449242 >> v2:172.21.15.124:3300/0 conn(0x7f5c94071f00 0x7f5c94166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.761+0000 7f5c99600640 1 -- 172.21.15.26:0/2679449242 >> v1:172.21.15.124:6789/0 conn(0x7f5c9406c7c0 legacy=0x7f5c940717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:14.764 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.761+0000 7f5c99600640 1 -- 172.21.15.26:0/2679449242 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9406d1a0 con 0x7f5c94071410 2024-01-15T18:16:14.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.763+0000 7f5c99600640 1 --2- 172.21.15.26:0/2679449242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 0x7f5c94146770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009d90 tx=0x7f5c8402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3521a3699258f2cb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:14.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.763+0000 7f5c93e00640 1 -- 172.21.15.26:0/2679449242 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8402cc60 con 0x7f5c94071410 2024-01-15T18:16:14.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.764+0000 7f5c93e00640 1 -- 172.21.15.26:0/2679449242 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8402cdc0 con 0x7f5c94071410 2024-01-15T18:16:14.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.764+0000 7f5c93e00640 1 -- 172.21.15.26:0/2679449242 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84034660 con 0x7f5c94071410 2024-01-15T18:16:14.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.764+0000 7f5c9aa00640 1 -- 172.21.15.26:0/2679449242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 msgr2=0x7f5c94146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:14.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.764+0000 7f5c9aa00640 1 --2- 172.21.15.26:0/2679449242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 0x7f5c94146770 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009d90 tx=0x7f5c8402beb0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.764+0000 7f5c9aa00640 1 -- 172.21.15.26:0/2679449242 shutdown_connections 2024-01-15T18:16:14.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.764+0000 7f5c9aa00640 1 --2- 172.21.15.26:0/2679449242 >> v2:172.21.15.124:3300/0 conn(0x7f5c94071f00 0x7f5c94166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.764+0000 7f5c9aa00640 1 --2- 172.21.15.26:0/2679449242 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 0x7f5c94146770 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.764+0000 7f5c9aa00640 1 -- 172.21.15.26:0/2679449242 >> 172.21.15.26:0/2679449242 conn(0x7f5c9406a800 msgr2=0x7f5c9406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:14.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.765+0000 7f5c9aa00640 1 -- 172.21.15.26:0/2679449242 shutdown_connections 2024-01-15T18:16:14.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.765+0000 7f5c9aa00640 1 -- 172.21.15.26:0/2679449242 wait complete. 2024-01-15T18:16:14.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.767+0000 7f5c9aa00640 1 Processor -- start 2024-01-15T18:16:14.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.767+0000 7f5c9aa00640 1 -- start start 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c9aa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c9406c7c0 0x7f5c940cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c9aa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 0x7f5c9412d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c9aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94071f00 0x7f5c941f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c9aa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94142590 con 0x7f5c94071f00 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c9aa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940cdf90 con 0x7f5c9406c7c0 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c9aa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c94141980 con 0x7f5c94071410 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c9a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94071f00 0x7f5c941f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c9a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94071f00 0x7f5c941f2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60832/0 (socket says 172.21.15.26:60832) 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c9a000640 1 -- 172.21.15.26:0/3306763175 learned_addr learned my addr 172.21.15.26:0/3306763175 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c99600640 1 --2- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c9406c7c0 0x7f5c940cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.768+0000 7f5c98c00640 1 --2- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 0x7f5c9412d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c9a000640 1 -- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 msgr2=0x7f5c9412d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c9a000640 1 --2- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5c94071410 0x7f5c9412d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c9a000640 1 -- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c9406c7c0 msgr2=0x7f5c940cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c9a000640 1 --2- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c9406c7c0 0x7f5c940cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c9a000640 1 -- 172.21.15.26:0/3306763175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c84009a70 con 0x7f5c94071f00 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c99600640 1 --2- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5c9406c7c0 0x7f5c940cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c9a000640 1 --2- 172.21.15.26:0/3306763175 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5c94071f00 0x7f5c941f2eb0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00e7f0 tx=0x7f5c8c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c91600640 1 -- 172.21.15.26:0/3306763175 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c009510 con 0x7f5c94071f00 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c91600640 1 -- 172.21.15.26:0/3306763175 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c00ee40 con 0x7f5c94071f00 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c91600640 1 -- 172.21.15.26:0/3306763175 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c00cb80 con 0x7f5c94071f00 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.769+0000 7f5c9aa00640 1 -- 172.21.15.26:0/3306763175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c94141c60 con 0x7f5c94071f00 2024-01-15T18:16:14.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.770+0000 7f5c9aa00640 1 -- 172.21.15.26:0/3306763175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c94142220 con 0x7f5c94071f00 2024-01-15T18:16:14.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.771+0000 7f5c9aa00640 1 -- 172.21.15.26:0/3306763175 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c94142740 con 0x7f5c94071f00 2024-01-15T18:16:14.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.772+0000 7f5c91600640 1 -- 172.21.15.26:0/3306763175 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f5c8c00cce0 con 0x7f5c94071f00 2024-01-15T18:16:14.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.773+0000 7f5c91600640 1 --2- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c6c06c670 0x7f5c6c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:14.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.773+0000 7f5c91600640 1 -- 172.21.15.26:0/3306763175 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f5c8c088c90 con 0x7f5c94071f00 2024-01-15T18:16:14.775 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.773+0000 7f5c99600640 1 --2- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c6c06c670 0x7f5c6c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:14.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.775+0000 7f5c99600640 1 --2- 172.21.15.26:0/3306763175 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5c6c06c670 0x7f5c6c06eb30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f5c84036000 tx=0x7f5c84030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:14.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.777+0000 7f5c91600640 1 -- 172.21.15.26:0/3306763175 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c8c0551a0 con 0x7f5c94071f00 2024-01-15T18:16:15.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:14.998+0000 7f5c9aa00640 1 -- 172.21.15.26:0/3306763175 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c9406da40 con 0x7f5c6c06c670 2024-01-15T18:16:15.020 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:15.018+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:15.018+0000 7f5c91600640 1 -- 172.21.15.26:0/3306763175 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f5c840346a0 con 0x7f5c6c06c670 2024-01-15T18:16:15.021 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:15.035 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:17.036 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:17.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.267+0000 7fb8df000640 1 Processor -- start 2024-01-15T18:16:17.269 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.267+0000 7fb8df000640 1 -- start start 2024-01-15T18:16:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.269+0000 7fb8df000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fb8d80cc500 0x7fb8d80cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.269+0000 7fb8df000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d8070e50 con 0x7fb8d8143c40 2024-01-15T18:16:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.269+0000 7fb8df000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d8144020 con 0x7fb8d806d160 2024-01-15T18:16:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.269+0000 7fb8df000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d80723c0 con 0x7fb8d80cc500 2024-01-15T18:16:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.269+0000 7fb8ddc00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fb8d8143c40 0x7fb8d8071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38900/0 (socket says 172.21.15.26:38900) 2024-01-15T18:16:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.269+0000 7fb8ddc00640 1 -- 172.21.15.26:0/2415229991 learned_addr learned my addr 172.21.15.26:0/2415229991 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:17.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.269+0000 7fb8dd200640 1 --2- 172.21.15.26:0/2415229991 >> v2:172.21.15.26:3300/0 conn(0x7fb8d80cc500 0x7fb8d80cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.270+0000 7fb8d7e00640 1 -- 172.21.15.26:0/2415229991 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137319606 0 0) 0x7fb8d8070e50 con 0x7fb8d8143c40 2024-01-15T18:16:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.270+0000 7fb8d7e00640 1 -- 172.21.15.26:0/2415229991 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b8003520 con 0x7fb8d8143c40 2024-01-15T18:16:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.270+0000 7fb8dd200640 1 -- 172.21.15.26:0/2415229991 >> v1:172.21.15.124:6789/0 conn(0x7fb8d806d160 legacy=0x7fb8d806d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.270+0000 7fb8dd200640 1 -- 172.21.15.26:0/2415229991 >> v1:172.21.15.26:6789/0 conn(0x7fb8d8143c40 legacy=0x7fb8d8071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.270+0000 7fb8dd200640 1 -- 172.21.15.26:0/2415229991 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d8071e00 con 0x7fb8d80cc500 2024-01-15T18:16:17.273 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.271+0000 7fb8dd200640 1 --2- 172.21.15.26:0/2415229991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d80cc500 0x7fb8d80cc990 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00097a0 tx=0x7fb8c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a574f820aee4750 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:17.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.271+0000 7fb8d7e00640 1 -- 172.21.15.26:0/2415229991 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0031830 con 0x7fb8d80cc500 2024-01-15T18:16:17.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.271+0000 7fb8d7e00640 1 -- 172.21.15.26:0/2415229991 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0031990 con 0x7fb8d80cc500 2024-01-15T18:16:17.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.271+0000 7fb8d7e00640 1 -- 172.21.15.26:0/2415229991 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0031d20 con 0x7fb8d80cc500 2024-01-15T18:16:17.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.272+0000 7fb8df000640 1 -- 172.21.15.26:0/2415229991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d80cc500 msgr2=0x7fb8d80cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.272+0000 7fb8df000640 1 --2- 172.21.15.26:0/2415229991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d80cc500 0x7fb8d80cc990 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb8c00097a0 tx=0x7fb8c002b8d0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.272+0000 7fb8df000640 1 -- 172.21.15.26:0/2415229991 shutdown_connections 2024-01-15T18:16:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.272+0000 7fb8df000640 1 --2- 172.21.15.26:0/2415229991 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d80cc500 0x7fb8d80cc990 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.272+0000 7fb8df000640 1 -- 172.21.15.26:0/2415229991 >> 172.21.15.26:0/2415229991 conn(0x7fb8d806a800 msgr2=0x7fb8d806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.272+0000 7fb8df000640 1 -- 172.21.15.26:0/2415229991 shutdown_connections 2024-01-15T18:16:17.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.272+0000 7fb8df000640 1 -- 172.21.15.26:0/2415229991 wait complete. 2024-01-15T18:16:17.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.274+0000 7fb8df000640 1 Processor -- start 2024-01-15T18:16:17.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.274+0000 7fb8df000640 1 -- start start 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.275+0000 7fb8df000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8d806d160 0x7fb8d80aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.275+0000 7fb8df000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8d80cc500 0x7fb8d80abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.275+0000 7fb8df000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d8143c40 0x7fb8d815a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.275+0000 7fb8df000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d815aec0 con 0x7fb8d8143c40 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.275+0000 7fb8df000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80aad50 con 0x7fb8d80cc500 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.275+0000 7fb8df000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80ab030 con 0x7fb8d806d160 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.275+0000 7fb8de600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d8143c40 0x7fb8d815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8de600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d8143c40 0x7fb8d815a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60860/0 (socket says 172.21.15.26:60860) 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8de600640 1 -- 172.21.15.26:0/457421878 learned_addr learned my addr 172.21.15.26:0/457421878 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8ddc00640 1 --2- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8d806d160 0x7fb8d80aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8dd200640 1 --2- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8d80cc500 0x7fb8d80abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8de600640 1 -- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8d806d160 msgr2=0x7fb8d80aa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8de600640 1 --2- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb8d806d160 0x7fb8d80aa770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8de600640 1 -- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8d80cc500 msgr2=0x7fb8d80abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:17.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8de600640 1 --2- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8d80cc500 0x7fb8d80abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:17.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8de600640 1 -- 172.21.15.26:0/457421878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c0009400 con 0x7fb8d8143c40 2024-01-15T18:16:17.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8dd200640 1 --2- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb8d80cc500 0x7fb8d80abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:17.280 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.276+0000 7fb8de600640 1 --2- 172.21.15.26:0/457421878 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb8d8143c40 0x7fb8d815a980 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000c6b0 tx=0x7fb8d000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:17.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.277+0000 7fb8d5600640 1 -- 172.21.15.26:0/457421878 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003b70 con 0x7fb8d8143c40 2024-01-15T18:16:17.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.277+0000 7fb8d5600640 1 -- 172.21.15.26:0/457421878 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d0003cf0 con 0x7fb8d8143c40 2024-01-15T18:16:17.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.277+0000 7fb8df000640 1 -- 172.21.15.26:0/457421878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d80ab310 con 0x7fb8d8143c40 2024-01-15T18:16:17.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.277+0000 7fb8d5600640 1 -- 172.21.15.26:0/457421878 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003ea0 con 0x7fb8d8143c40 2024-01-15T18:16:17.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.277+0000 7fb8df000640 1 -- 172.21.15.26:0/457421878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d80ab8d0 con 0x7fb8d8143c40 2024-01-15T18:16:17.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.278+0000 7fb8df000640 1 -- 172.21.15.26:0/457421878 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d8140f30 con 0x7fb8d8143c40 2024-01-15T18:16:17.281 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.279+0000 7fb8d5600640 1 -- 172.21.15.26:0/457421878 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fb8d0005310 con 0x7fb8d8143c40 2024-01-15T18:16:17.282 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.280+0000 7fb8d5600640 1 --2- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb8b006c670 0x7fb8b006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:17.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.280+0000 7fb8d5600640 1 -- 172.21.15.26:0/457421878 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fb8d0088620 con 0x7fb8d8143c40 2024-01-15T18:16:17.283 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.281+0000 7fb8ddc00640 1 --2- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb8b006c670 0x7fb8b006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:17.284 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.282+0000 7fb8ddc00640 1 --2- 172.21.15.26:0/457421878 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb8b006c670 0x7fb8b006eb30 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80011e0 tx=0x7fb8c8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:17.287 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.285+0000 7fb8d5600640 1 -- 172.21.15.26:0/457421878 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8d0054ee0 con 0x7fb8d8143c40 2024-01-15T18:16:17.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.509+0000 7fb8df000640 1 -- 172.21.15.26:0/457421878 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8d8142e00 con 0x7fb8b006c670 2024-01-15T18:16:17.532 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:17.529+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:17.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:17.529+0000 7fb8d5600640 1 -- 172.21.15.26:0/457421878 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fb8d8142e00 con 0x7fb8b006c670 2024-01-15T18:16:17.532 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:17.545 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:19.546 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:19.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.777+0000 7f6a14e00640 1 Processor -- start 2024-01-15T18:16:19.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.777+0000 7f6a14e00640 1 -- start start 2024-01-15T18:16:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.779+0000 7f6a14e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100cc500 0x7f6a100cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.779+0000 7f6a14e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6a1006d160 0x7f6a1006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.779+0000 7f6a14e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a10070e50 con 0x7f6a10143c40 2024-01-15T18:16:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.779+0000 7f6a14e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a100723c0 con 0x7f6a1006d160 2024-01-15T18:16:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.779+0000 7f6a14e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10144020 con 0x7f6a100cc500 2024-01-15T18:16:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.779+0000 7f6a0f400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6a10143c40 0x7f6a10071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:38914/0 (socket says 172.21.15.26:38914) 2024-01-15T18:16:19.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.779+0000 7f6a0f400640 1 -- 172.21.15.26:0/1175469476 learned_addr learned my addr 172.21.15.26:0/1175469476 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0fe00640 1 --2- 172.21.15.26:0/1175469476 >> v2:172.21.15.124:3300/0 conn(0x7f6a1006d160 0x7f6a1006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0ea00640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100cc500 0x7f6a100cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413621545 0 0) 0x7f6a10070e50 con 0x7f6a10143c40 2024-01-15T18:16:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69f0003520 con 0x7f6a10143c40 2024-01-15T18:16:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1811680442 0 0) 0x7f69f0003520 con 0x7f6a10143c40 2024-01-15T18:16:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100cc500 msgr2=0x7f6a100cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:19.784 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100cc500 0x7f6a100cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 >> v2:172.21.15.124:3300/0 conn(0x7f6a1006d160 msgr2=0x7f6a1006d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 --2- 172.21.15.26:0/1175469476 >> v2:172.21.15.124:3300/0 conn(0x7f6a1006d160 0x7f6a1006d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a100ccea0 con 0x7f6a10143c40 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f6a000019f0 con 0x7f6a10143c40 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 >> v1:172.21.15.26:6789/0 conn(0x7f6a10143c40 legacy=0x7f6a10071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f0003970 0x7f69f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69f00047c0 0x7f69f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100cc500 0x7f69f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100723c0 con 0x7f69f0003970 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a10144020 con 0x7f69f00047c0 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.780+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f000ba00 con 0x7f6a100cc500 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.781+0000 7f6a0f400640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f0003970 0x7f69f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.781+0000 7f6a0ea00640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69f00047c0 0x7f69f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.781+0000 7f6a0fe00640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100cc500 0x7f69f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.781+0000 7f6a0f400640 1 -- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100cc500 msgr2=0x7f69f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.781+0000 7f6a0f400640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100cc500 0x7f69f0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.781+0000 7f6a0f400640 1 -- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69f00047c0 msgr2=0x7f69f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.781+0000 7f6a0f400640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69f00047c0 0x7f69f0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.781+0000 7f6a0f400640 1 -- 172.21.15.26:0/1175469476 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0003520 con 0x7f69f0003970 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.782+0000 7f6a0f400640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f0003970 0x7f69f0003d90 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000a790 tx=0x7f69f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0a27bee1469aad server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.783+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00034630 con 0x7f69f0003970 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.783+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0002f070 con 0x7f69f0003970 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.783+0000 7f6a0e000640 1 -- 172.21.15.26:0/1175469476 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00034a30 con 0x7f69f0003970 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.783+0000 7f6a14e00640 1 -- 172.21.15.26:0/1175469476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f0003970 msgr2=0x7f69f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.783+0000 7f6a14e00640 1 --2- 172.21.15.26:0/1175469476 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f0003970 0x7f69f0003d90 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000a790 tx=0x7f69f00057f0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.784+0000 7f6a0fe00640 1 -- 172.21.15.26:0/1175469476 reap_dead start 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.784+0000 7f6a14e00640 1 -- 172.21.15.26:0/1175469476 shutdown_connections 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.784+0000 7f6a14e00640 1 -- 172.21.15.26:0/1175469476 >> 172.21.15.26:0/1175469476 conn(0x7f6a1006a800 msgr2=0x7f6a1006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.784+0000 7f6a14e00640 1 -- 172.21.15.26:0/1175469476 shutdown_connections 2024-01-15T18:16:19.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.784+0000 7f6a14e00640 1 -- 172.21.15.26:0/1175469476 wait complete. 2024-01-15T18:16:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.786+0000 7f6a14e00640 1 Processor -- start 2024-01-15T18:16:19.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.786+0000 7f6a14e00640 1 -- start start 2024-01-15T18:16:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a14e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f0003970 0x7f6a101e03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a14e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1013f050 0x7f6a1013e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a14e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100ac8d0 0x7f6a100acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a14e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a100ad290 con 0x7f69f0003970 2024-01-15T18:16:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a14e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1013ee70 con 0x7f6a1013f050 2024-01-15T18:16:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a14e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a101e09c0 con 0x7f6a100ac8d0 2024-01-15T18:16:19.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f0003970 0x7f6a101e03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a0f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f0003970 0x7f6a101e03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:60870/0 (socket says 172.21.15.26:60870) 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a0f400640 1 -- 172.21.15.26:0/832797508 learned_addr learned my addr 172.21.15.26:0/832797508 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a0f400640 1 -- 172.21.15.26:0/832797508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100ac8d0 msgr2=0x7f6a100acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.787+0000 7f6a0fe00640 1 --2- 172.21.15.26:0/832797508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100ac8d0 0x7f6a100acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a0ea00640 1 --2- 172.21.15.26:0/832797508 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1013f050 0x7f6a1013e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a0f400640 1 --2- 172.21.15.26:0/832797508 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6a100ac8d0 0x7f6a100acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a0f400640 1 -- 172.21.15.26:0/832797508 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1013f050 msgr2=0x7f6a1013e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a0f400640 1 --2- 172.21.15.26:0/832797508 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6a1013f050 0x7f6a1013e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a0f400640 1 -- 172.21.15.26:0/832797508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a101e0ca0 con 0x7f69f0003970 2024-01-15T18:16:19.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a0f400640 1 --2- 172.21.15.26:0/832797508 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69f0003970 0x7f6a101e03e0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f6a0002cba0 tx=0x7f6a00001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a07400640 1 -- 172.21.15.26:0/832797508 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00044030 con 0x7f69f0003970 2024-01-15T18:16:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a07400640 1 -- 172.21.15.26:0/832797508 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a00004e30 con 0x7f69f0003970 2024-01-15T18:16:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a07400640 1 -- 172.21.15.26:0/832797508 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a00005250 con 0x7f69f0003970 2024-01-15T18:16:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a14e00640 1 -- 172.21.15.26:0/832797508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a101e0f80 con 0x7f69f0003970 2024-01-15T18:16:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.788+0000 7f6a14e00640 1 -- 172.21.15.26:0/832797508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a101e1540 con 0x7f69f0003970 2024-01-15T18:16:19.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.789+0000 7f6a14e00640 1 -- 172.21.15.26:0/832797508 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a10067450 con 0x7f69f0003970 2024-01-15T18:16:19.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.791+0000 7f6a07400640 1 -- 172.21.15.26:0/832797508 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f6a00047070 con 0x7f69f0003970 2024-01-15T18:16:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.791+0000 7f6a07400640 1 --2- 172.21.15.26:0/832797508 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f69e806c6c0 0x7f69e806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.792+0000 7f6a07400640 1 -- 172.21.15.26:0/832797508 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f6a00045070 con 0x7f69f0003970 2024-01-15T18:16:19.794 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.792+0000 7f6a0ea00640 1 --2- 172.21.15.26:0/832797508 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f69e806c6c0 0x7f69e806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:19.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.794+0000 7f6a0ea00640 1 --2- 172.21.15.26:0/832797508 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f69e806c6c0 0x7f69e806eb80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6a1015b0f0 tx=0x7f69f8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:19.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:19.797+0000 7f6a07400640 1 -- 172.21.15.26:0/832797508 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a00077660 con 0x7f69f0003970 2024-01-15T18:16:20.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:20.017+0000 7f6a14e00640 1 -- 172.21.15.26:0/832797508 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a1006c960 con 0x7f69e806c6c0 2024-01-15T18:16:20.039 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:20.036+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:20.037+0000 7f6a07400640 1 -- 172.21.15.26:0/832797508 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f69f8012950 con 0x7f69e806c6c0 2024-01-15T18:16:20.039 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:20.058 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:22.059 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:22.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.293+0000 7fab2ae00640 1 Processor -- start 2024-01-15T18:16:22.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.293+0000 7fab2ae00640 1 -- start start 2024-01-15T18:16:22.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.295+0000 7fab2ae00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fab24143c40 0x7fab24071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:22.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.295+0000 7fab2ae00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab24070e50 con 0x7fab240717d0 2024-01-15T18:16:22.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.295+0000 7fab2ae00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab24144020 con 0x7fab240ccc10 2024-01-15T18:16:22.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.295+0000 7fab2ae00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab240723c0 con 0x7fab24143c40 2024-01-15T18:16:22.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.295+0000 7fab29000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fab240717d0 0x7fab240cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57562/0 (socket says 172.21.15.26:57562) 2024-01-15T18:16:22.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.295+0000 7fab29000640 1 -- 172.21.15.26:0/1732221740 learned_addr learned my addr 172.21.15.26:0/1732221740 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab29a00640 1 --2- 172.21.15.26:0/1732221740 >> v2:172.21.15.124:3300/0 conn(0x7fab24143c40 0x7fab24071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322647979 0 0) 0x7fab24070e50 con 0x7fab240717d0 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab04003520 con 0x7fab240717d0 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 553885631 0 0) 0x7fab04003520 con 0x7fab240717d0 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 >> v2:172.21.15.124:3300/0 conn(0x7fab24143c40 msgr2=0x7fab24071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 --2- 172.21.15.26:0/1732221740 >> v2:172.21.15.124:3300/0 conn(0x7fab24143c40 0x7fab24071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 >> v1:172.21.15.124:6789/0 conn(0x7fab240ccc10 legacy=0x7fab2406d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24071e00 con 0x7fab240717d0 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fab0c001670 con 0x7fab240717d0 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 >> v1:172.21.15.26:6789/0 conn(0x7fab240717d0 legacy=0x7fab240cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:22.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.296+0000 7fab23e00640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab040039c0 0x7fab04003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab23e00640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040047c0 0x7fab04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab23e00640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040051d0 0x7fab0400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240723c0 con 0x7fab040047c0 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0400c120 con 0x7fab040051d0 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0400c2e0 con 0x7fab040039c0 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab29000640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040047c0 0x7fab04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab29a00640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab040039c0 0x7fab04003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab2a400640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040051d0 0x7fab0400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab29000640 1 -- 172.21.15.26:0/1732221740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab040039c0 msgr2=0x7fab04003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab29000640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab040039c0 0x7fab04003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab29000640 1 -- 172.21.15.26:0/1732221740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040051d0 msgr2=0x7fab0400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab29000640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040051d0 0x7fab0400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab29000640 1 -- 172.21.15.26:0/1732221740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04003520 con 0x7fab040047c0 2024-01-15T18:16:22.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab2a400640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040051d0 0x7fab0400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:22.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.297+0000 7fab2a400640 1 -- 172.21.15.26:0/1732221740 reap_dead start 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.299+0000 7fab29000640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040047c0 0x7fab04004c40 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fab0c001eb0 tx=0x7fab040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a257c03ad4be341f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.299+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c005ad0 con 0x7fab040047c0 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.299+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c005c50 con 0x7fab040047c0 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.299+0000 7fab23e00640 1 -- 172.21.15.26:0/1732221740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c02c7c0 con 0x7fab040047c0 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.300+0000 7fab2ae00640 1 -- 172.21.15.26:0/1732221740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040047c0 msgr2=0x7fab04004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.300+0000 7fab2ae00640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040047c0 0x7fab04004c40 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fab0c001eb0 tx=0x7fab040057f0 comp rx=0 tx=0).stop 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.300+0000 7fab2ae00640 1 -- 172.21.15.26:0/1732221740 shutdown_connections 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.300+0000 7fab2ae00640 1 --2- 172.21.15.26:0/1732221740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab040047c0 0x7fab04004c40 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.300+0000 7fab2ae00640 1 -- 172.21.15.26:0/1732221740 >> 172.21.15.26:0/1732221740 conn(0x7fab2406a800 msgr2=0x7fab2406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.300+0000 7fab2ae00640 1 -- 172.21.15.26:0/1732221740 shutdown_connections 2024-01-15T18:16:22.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.300+0000 7fab2ae00640 1 -- 172.21.15.26:0/1732221740 wait complete. 2024-01-15T18:16:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.302+0000 7fab2ae00640 1 Processor -- start 2024-01-15T18:16:22.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.302+0000 7fab2ae00640 1 -- start start 2024-01-15T18:16:22.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.303+0000 7fab2ae00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040047c0 0x7fab2415cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.303+0000 7fab2ae00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab2415a130 0x7fab240afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.303+0000 7fab2ae00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab240b0130 0x7fab2415eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.303+0000 7fab2ae00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2415f940 con 0x7fab240b0130 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.303+0000 7fab2ae00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240b05b0 con 0x7fab040047c0 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.303+0000 7fab2ae00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2415c1f0 con 0x7fab2415a130 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.303+0000 7fab2a400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab240b0130 0x7fab2415eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab2a400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab240b0130 0x7fab2415eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41150/0 (socket says 172.21.15.26:41150) 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab2a400640 1 -- 172.21.15.26:0/451391898 learned_addr learned my addr 172.21.15.26:0/451391898 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab29a00640 1 --2- 172.21.15.26:0/451391898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040047c0 0x7fab2415cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab29000640 1 --2- 172.21.15.26:0/451391898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab2415a130 0x7fab240afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab2a400640 1 -- 172.21.15.26:0/451391898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab2415a130 msgr2=0x7fab240afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab2a400640 1 --2- 172.21.15.26:0/451391898 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fab2415a130 0x7fab240afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab2a400640 1 -- 172.21.15.26:0/451391898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040047c0 msgr2=0x7fab2415cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:22.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab2a400640 1 --2- 172.21.15.26:0/451391898 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fab040047c0 0x7fab2415cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab2a400640 1 -- 172.21.15.26:0/451391898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2415c4d0 con 0x7fab240b0130 2024-01-15T18:16:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.304+0000 7fab2a400640 1 --2- 172.21.15.26:0/451391898 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fab240b0130 0x7fab2415eb80 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fab240ce170 tx=0x7fab1c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:22.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.305+0000 7fab21600640 1 -- 172.21.15.26:0/451391898 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c003d80 con 0x7fab240b0130 2024-01-15T18:16:22.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.305+0000 7fab21600640 1 -- 172.21.15.26:0/451391898 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c005070 con 0x7fab240b0130 2024-01-15T18:16:22.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.305+0000 7fab21600640 1 -- 172.21.15.26:0/451391898 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c005420 con 0x7fab240b0130 2024-01-15T18:16:22.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.305+0000 7fab2ae00640 1 -- 172.21.15.26:0/451391898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2415c7b0 con 0x7fab240b0130 2024-01-15T18:16:22.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.305+0000 7fab2ae00640 1 -- 172.21.15.26:0/451391898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2415a590 con 0x7fab240b0130 2024-01-15T18:16:22.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.306+0000 7fab2ae00640 1 -- 172.21.15.26:0/451391898 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab2415ccd0 con 0x7fab240b0130 2024-01-15T18:16:22.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.307+0000 7fab21600640 1 -- 172.21.15.26:0/451391898 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fab1c005580 con 0x7fab240b0130 2024-01-15T18:16:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.308+0000 7fab21600640 1 --2- 172.21.15.26:0/451391898 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faafc06c6a0 0x7faafc06eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:22.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.308+0000 7fab21600640 1 -- 172.21.15.26:0/451391898 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fab1c0884c0 con 0x7fab240b0130 2024-01-15T18:16:22.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.309+0000 7fab29a00640 1 --2- 172.21.15.26:0/451391898 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faafc06c6a0 0x7faafc06eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:22.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.310+0000 7fab29a00640 1 --2- 172.21.15.26:0/451391898 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faafc06c6a0 0x7faafc06eb60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fab14009a90 tx=0x7fab14008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:22.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.313+0000 7fab21600640 1 -- 172.21.15.26:0/451391898 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fab2415ccd0 con 0x7fab240b0130 2024-01-15T18:16:22.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.537+0000 7fab2ae00640 1 -- 172.21.15.26:0/451391898 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fab2406cc40 con 0x7faafc06c6a0 2024-01-15T18:16:22.563 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:22.561+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:22.561+0000 7fab21600640 1 -- 172.21.15.26:0/451391898 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fab14013a20 con 0x7faafc06c6a0 2024-01-15T18:16:22.564 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:22.577 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:24.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:24.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.814+0000 7f2c7b200640 1 Processor -- start 2024-01-15T18:16:24.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.815+0000 7f2c7b200640 1 -- start start 2024-01-15T18:16:24.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.816+0000 7f2c7b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0b3f30 0x7f2c7c0d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:24.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.816+0000 7f2c7b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c0d0a10 con 0x7f2c7c160920 2024-01-15T18:16:24.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.816+0000 7f2c7b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c0cf080 con 0x7f2c7c0d0100 2024-01-15T18:16:24.819 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.817+0000 7f2c7b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0c6be0 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.817+0000 7f2c79e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2c7c160920 0x7f2c7c0cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57574/0 (socket says 172.21.15.26:57574) 2024-01-15T18:16:24.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.817+0000 7f2c79e00640 1 -- 172.21.15.26:0/734048857 learned_addr learned my addr 172.21.15.26:0/734048857 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:24.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.817+0000 7f2c7a800640 1 --2- 172.21.15.26:0/734048857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0b3f30 0x7f2c7c0d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:24.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.818+0000 7f2c79400640 1 -- 172.21.15.26:0/734048857 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051722580 0 0) 0x7f2c7c0d0a10 con 0x7f2c7c160920 2024-01-15T18:16:24.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.818+0000 7f2c79400640 1 -- 172.21.15.26:0/734048857 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c5c003520 con 0x7f2c7c160920 2024-01-15T18:16:24.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.818+0000 7f2c79400640 1 -- 172.21.15.26:0/734048857 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806765789 0 0) 0x7f2c7c0cf080 con 0x7f2c7c0d0100 2024-01-15T18:16:24.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.818+0000 7f2c7a800640 1 -- 172.21.15.26:0/734048857 >> v1:172.21.15.124:6789/0 conn(0x7f2c7c0d0100 legacy=0x7f2c7c0d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:24.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.818+0000 7f2c7a800640 1 -- 172.21.15.26:0/734048857 >> v1:172.21.15.26:6789/0 conn(0x7f2c7c160920 legacy=0x7f2c7c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:24.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.818+0000 7f2c7a800640 1 -- 172.21.15.26:0/734048857 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c0ce7e0 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.821 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.819+0000 7f2c7a800640 1 --2- 172.21.15.26:0/734048857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0b3f30 0x7f2c7c0d1650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009dc0 tx=0x7f2c7402bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2727e37d3af16498 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:24.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.820+0000 7f2c79400640 1 -- 172.21.15.26:0/734048857 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7402ccc0 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.820+0000 7f2c79400640 1 -- 172.21.15.26:0/734048857 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7402ce20 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.820+0000 7f2c79400640 1 -- 172.21.15.26:0/734048857 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c74034660 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.820+0000 7f2c7b200640 1 -- 172.21.15.26:0/734048857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0b3f30 msgr2=0x7f2c7c0d1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.820+0000 7f2c7b200640 1 --2- 172.21.15.26:0/734048857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0b3f30 0x7f2c7c0d1650 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009dc0 tx=0x7f2c7402bf10 comp rx=0 tx=0).stop 2024-01-15T18:16:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.820+0000 7f2c7b200640 1 -- 172.21.15.26:0/734048857 shutdown_connections 2024-01-15T18:16:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.820+0000 7f2c7b200640 1 --2- 172.21.15.26:0/734048857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0b3f30 0x7f2c7c0d1650 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.820+0000 7f2c7b200640 1 -- 172.21.15.26:0/734048857 >> 172.21.15.26:0/734048857 conn(0x7f2c7c0d3090 msgr2=0x7f2c7c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.821+0000 7f2c7b200640 1 -- 172.21.15.26:0/734048857 shutdown_connections 2024-01-15T18:16:24.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.821+0000 7f2c7b200640 1 -- 172.21.15.26:0/734048857 wait complete. 2024-01-15T18:16:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.822+0000 7f2c7b200640 1 Processor -- start 2024-01-15T18:16:24.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.823+0000 7f2c7b200640 1 -- start start 2024-01-15T18:16:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c7b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7c0b3f30 0x7f2c7c1492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c7b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0d0100 0x7f2c7c14c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c7b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c7c160920 0x7f2c7c1499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c7b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0b79f0 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c7b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c14c950 con 0x7f2c7c160920 2024-01-15T18:16:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c7b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0b5ff0 con 0x7f2c7c0d0100 2024-01-15T18:16:24.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c79e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7c0b3f30 0x7f2c7c1492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c79e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7c0b3f30 0x7f2c7c1492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41160/0 (socket says 172.21.15.26:41160) 2024-01-15T18:16:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c79e00640 1 -- 172.21.15.26:0/1168934740 learned_addr learned my addr 172.21.15.26:0/1168934740 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c79e00640 1 -- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0d0100 msgr2=0x7f2c7c14c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c73e00640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0d0100 0x7f2c7c14c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c7a800640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c7c160920 0x7f2c7c1499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c79e00640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0d0100 0x7f2c7c14c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c79e00640 1 -- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c7c160920 msgr2=0x7f2c7c1499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c79e00640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c7c160920 0x7f2c7c1499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:24.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c79e00640 1 -- 172.21.15.26:0/1168934740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c74009a70 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c73e00640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7c0d0100 0x7f2c7c14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:24.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.824+0000 7f2c7a800640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c7c160920 0x7f2c7c1499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:24.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.825+0000 7f2c79e00640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7c0b3f30 0x7f2c7c1492d0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00c390 tx=0x7f2c6c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.825+0000 7f2c72000640 1 -- 172.21.15.26:0/1168934740 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c005ce0 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.825+0000 7f2c72000640 1 -- 172.21.15.26:0/1168934740 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c005e40 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.825+0000 7f2c7b200640 1 -- 172.21.15.26:0/1168934740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c0b7cb0 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.825+0000 7f2c72000640 1 -- 172.21.15.26:0/1168934740 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0072f0 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.825+0000 7f2c7b200640 1 -- 172.21.15.26:0/1168934740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c0b5c50 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.826+0000 7f2c7b200640 1 -- 172.21.15.26:0/1168934740 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c0b7850 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.827+0000 7f2c72000640 1 -- 172.21.15.26:0/1168934740 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f2c6c007450 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.828+0000 7f2c72000640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c5406c670 0x7f2c5406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.828+0000 7f2c72000640 1 -- 172.21.15.26:0/1168934740 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f2c6c0882d0 con 0x7f2c7c0b3f30 2024-01-15T18:16:24.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.829+0000 7f2c73e00640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c5406c670 0x7f2c5406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:24.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.830+0000 7f2c73e00640 1 --2- 172.21.15.26:0/1168934740 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c5406c670 0x7f2c5406eb30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0cd360 tx=0x7f2c64009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:24.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:24.833+0000 7f2c72000640 1 -- 172.21.15.26:0/1168934740 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c7c0b7850 con 0x7f2c7c0b3f30 2024-01-15T18:16:25.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:25.055+0000 7f2c7b200640 1 -- 172.21.15.26:0/1168934740 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c7c160d00 con 0x7f2c5406c670 2024-01-15T18:16:25.077 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:25.075+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:25.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:25.075+0000 7f2c72000640 1 -- 172.21.15.26:0/1168934740 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f2c64001d50 con 0x7f2c5406c670 2024-01-15T18:16:25.078 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:25.091 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:27.093 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.319+0000 7fa731c00640 1 Processor -- start 2024-01-15T18:16:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.319+0000 7fa731c00640 1 -- start start 2024-01-15T18:16:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.320+0000 7fa731c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c0cc500 0x7fa72c0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.321+0000 7fa731c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c070e50 con 0x7fa72c06d160 2024-01-15T18:16:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.321+0000 7fa731c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c0ccea0 con 0x7fa72c143c40 2024-01-15T18:16:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.321+0000 7fa731c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0723c0 con 0x7fa72c0cc500 2024-01-15T18:16:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.321+0000 7fa731200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa72c06d160 0x7fa72c06d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57586/0 (socket says 172.21.15.26:57586) 2024-01-15T18:16:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.321+0000 7fa731200640 1 -- 172.21.15.26:0/2743131104 learned_addr learned my addr 172.21.15.26:0/2743131104 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.321+0000 7fa723e00640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c0cc500 0x7fa72c0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.321+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299461041 0 0) 0x7fa72c070e50 con 0x7fa72c06d160 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.321+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa70c003520 con 0x7fa72c06d160 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 245074868 0 0) 0x7fa70c003520 con 0x7fa72c06d160 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c0cc500 msgr2=0x7fa72c0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c0cc500 0x7fa72c0cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 >> v1:172.21.15.124:6789/0 conn(0x7fa72c143c40 legacy=0x7fa72c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c144020 con 0x7fa72c06d160 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa7240019f0 con 0x7fa72c06d160 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 >> v1:172.21.15.26:6789/0 conn(0x7fa72c06d160 legacy=0x7fa72c06d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa70c0039a0 0x7fa70c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa70c0047c0 0x7fa70c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c0cc500 0x7fa70c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0723c0 con 0x7fa70c0047c0 2024-01-15T18:16:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c00bb40 con 0x7fa70c0039a0 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c00bd00 con 0x7fa72c0cc500 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.322+0000 7fa723e00640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa70c0047c0 0x7fa70c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.323+0000 7fa72be00640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa70c0039a0 0x7fa70c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.323+0000 7fa731200640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c0cc500 0x7fa70c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.323+0000 7fa723e00640 1 -- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c0cc500 msgr2=0x7fa70c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.323+0000 7fa723e00640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c0cc500 0x7fa70c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.323+0000 7fa723e00640 1 -- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa70c0039a0 msgr2=0x7fa70c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.323+0000 7fa723e00640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa70c0039a0 0x7fa70c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.323+0000 7fa723e00640 1 -- 172.21.15.26:0/2743131104 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c003520 con 0x7fa70c0047c0 2024-01-15T18:16:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.323+0000 7fa731200640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c0cc500 0x7fa70c005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:16:27.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.324+0000 7fa723e00640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa70c0047c0 0x7fa70c004c40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa714009ac0 tx=0x7fa70c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a89628fb95c024 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:27.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.324+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71402c6b0 con 0x7fa70c0047c0 2024-01-15T18:16:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.325+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71402c810 con 0x7fa70c0047c0 2024-01-15T18:16:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.325+0000 7fa72b400640 1 -- 172.21.15.26:0/2743131104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71402cba0 con 0x7fa70c0047c0 2024-01-15T18:16:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.325+0000 7fa731c00640 1 -- 172.21.15.26:0/2743131104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa70c0047c0 msgr2=0x7fa70c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.325+0000 7fa731c00640 1 --2- 172.21.15.26:0/2743131104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa70c0047c0 0x7fa70c004c40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa714009ac0 tx=0x7fa70c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:16:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.325+0000 7fa731200640 1 -- 172.21.15.26:0/2743131104 reap_dead start 2024-01-15T18:16:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.325+0000 7fa731c00640 1 -- 172.21.15.26:0/2743131104 shutdown_connections 2024-01-15T18:16:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.325+0000 7fa731c00640 1 -- 172.21.15.26:0/2743131104 >> 172.21.15.26:0/2743131104 conn(0x7fa72c06a800 msgr2=0x7fa72c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.325+0000 7fa731c00640 1 -- 172.21.15.26:0/2743131104 shutdown_connections 2024-01-15T18:16:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.325+0000 7fa731c00640 1 -- 172.21.15.26:0/2743131104 wait complete. 2024-01-15T18:16:27.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.327+0000 7fa731c00640 1 Processor -- start 2024-01-15T18:16:27.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.327+0000 7fa731c00640 1 -- start start 2024-01-15T18:16:27.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.328+0000 7fa731c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa70c0047c0 0x7fa72c13e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:27.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.328+0000 7fa731c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa72c13b840 0x7fa72c13cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:27.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.328+0000 7fa731c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c159a90 0x7fa72c139c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:27.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.328+0000 7fa731c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c13ec20 con 0x7fa70c0047c0 2024-01-15T18:16:27.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.328+0000 7fa731c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c13d2b0 con 0x7fa72c13b840 2024-01-15T18:16:27.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.328+0000 7fa731c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c13bcd0 con 0x7fa72c159a90 2024-01-15T18:16:27.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa72be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa70c0047c0 0x7fa72c13e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:27.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa72be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa70c0047c0 0x7fa72c13e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41184/0 (socket says 172.21.15.26:41184) 2024-01-15T18:16:27.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa72be00640 1 -- 172.21.15.26:0/846798402 learned_addr learned my addr 172.21.15.26:0/846798402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:27.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa731200640 1 --2- 172.21.15.26:0/846798402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c159a90 0x7fa72c139c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:27.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa723e00640 1 --2- 172.21.15.26:0/846798402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa72c13b840 0x7fa72c13cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:27.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa72be00640 1 -- 172.21.15.26:0/846798402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c159a90 msgr2=0x7fa72c139c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:27.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa72be00640 1 --2- 172.21.15.26:0/846798402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa72c159a90 0x7fa72c139c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:27.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa72be00640 1 -- 172.21.15.26:0/846798402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa72c13b840 msgr2=0x7fa72c13cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:27.333 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa72be00640 1 --2- 172.21.15.26:0/846798402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa72c13b840 0x7fa72c13cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:27.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.329+0000 7fa72be00640 1 -- 172.21.15.26:0/846798402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c13bfb0 con 0x7fa70c0047c0 2024-01-15T18:16:27.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.330+0000 7fa72be00640 1 --2- 172.21.15.26:0/846798402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa70c0047c0 0x7fa72c13e6e0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00f5f0 tx=0x7fa71c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:27.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.330+0000 7fa728c00640 1 -- 172.21.15.26:0/846798402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0052e0 con 0x7fa70c0047c0 2024-01-15T18:16:27.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.330+0000 7fa728c00640 1 -- 172.21.15.26:0/846798402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c005440 con 0x7fa70c0047c0 2024-01-15T18:16:27.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.330+0000 7fa728c00640 1 -- 172.21.15.26:0/846798402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c003830 con 0x7fa70c0047c0 2024-01-15T18:16:27.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.330+0000 7fa731c00640 1 -- 172.21.15.26:0/846798402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c13c290 con 0x7fa70c0047c0 2024-01-15T18:16:27.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.330+0000 7fa731c00640 1 -- 172.21.15.26:0/846798402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c13c850 con 0x7fa70c0047c0 2024-01-15T18:16:27.334 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.331+0000 7fa731c00640 1 -- 172.21.15.26:0/846798402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c13ca10 con 0x7fa70c0047c0 2024-01-15T18:16:27.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.332+0000 7fa728c00640 1 -- 172.21.15.26:0/846798402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fa71c003990 con 0x7fa70c0047c0 2024-01-15T18:16:27.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.333+0000 7fa728c00640 1 --2- 172.21.15.26:0/846798402 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa70406c670 0x7fa70406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:27.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.333+0000 7fa728c00640 1 -- 172.21.15.26:0/846798402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fa71c088b60 con 0x7fa70c0047c0 2024-01-15T18:16:27.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.334+0000 7fa723e00640 1 --2- 172.21.15.26:0/846798402 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa70406c670 0x7fa70406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:27.337 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.335+0000 7fa723e00640 1 --2- 172.21.15.26:0/846798402 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa70406c670 0x7fa70406eb30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa714036000 tx=0x7fa714030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:27.340 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.338+0000 7fa728c00640 1 -- 172.21.15.26:0/846798402 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa71c055070 con 0x7fa70c0047c0 2024-01-15T18:16:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.562+0000 7fa731c00640 1 -- 172.21.15.26:0/846798402 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fa72c06c960 con 0x7fa70406c670 2024-01-15T18:16:27.585 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:27.582+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:27.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:27.583+0000 7fa728c00640 1 -- 172.21.15.26:0/846798402 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fa7140041c0 con 0x7fa70406c670 2024-01-15T18:16:27.585 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:27.596 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:29.597 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.833+0000 7f9d58e00640 1 Processor -- start 2024-01-15T18:16:29.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.833+0000 7f9d58e00640 1 -- start start 2024-01-15T18:16:29.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.835+0000 7f9d58e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9d540cf9a0 0x7f9d5415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.835+0000 7f9d58e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d5415e1e0 0x7f9d540702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.835+0000 7f9d58e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d540b1ee0 con 0x7f9d54143ae0 2024-01-15T18:16:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.835+0000 7f9d58e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540b2780 con 0x7f9d540cf9a0 2024-01-15T18:16:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.835+0000 7f9d58e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5415e990 con 0x7f9d5415e1e0 2024-01-15T18:16:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.836+0000 7f9d52800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d5415e1e0 0x7f9d540702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.836+0000 7f9d4be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9d540cf9a0 0x7f9d5415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:29.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.836+0000 7f9d51e00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f9d54143ae0 0x7f9d540cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:47920/0 (socket says 172.21.15.26:47920) 2024-01-15T18:16:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.836+0000 7f9d51e00640 1 -- 172.21.15.26:0/784289735 learned_addr learned my addr 172.21.15.26:0/784289735 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.837+0000 7f9d52800640 1 -- 172.21.15.26:0/784289735 >> v2:172.21.15.124:3300/0 conn(0x7f9d540cf9a0 msgr2=0x7f9d5415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.837+0000 7f9d52800640 1 --2- 172.21.15.26:0/784289735 >> v2:172.21.15.124:3300/0 conn(0x7f9d540cf9a0 0x7f9d5415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.837+0000 7f9d52800640 1 -- 172.21.15.26:0/784289735 >> v1:172.21.15.124:6789/0 conn(0x7f9d54143ae0 legacy=0x7f9d540cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:29.839 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.837+0000 7f9d52800640 1 -- 172.21.15.26:0/784289735 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d540b2130 con 0x7f9d5415e1e0 2024-01-15T18:16:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.839+0000 7f9d52800640 1 --2- 172.21.15.26:0/784289735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d5415e1e0 0x7f9d540702c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d90 tx=0x7f9d4c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a6dc56b2fe546c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:29.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.839+0000 7f9d51400640 1 -- 172.21.15.26:0/784289735 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c02cc50 con 0x7f9d5415e1e0 2024-01-15T18:16:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.839+0000 7f9d51400640 1 -- 172.21.15.26:0/784289735 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4c02cdb0 con 0x7f9d5415e1e0 2024-01-15T18:16:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.839+0000 7f9d51400640 1 -- 172.21.15.26:0/784289735 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c034660 con 0x7f9d5415e1e0 2024-01-15T18:16:29.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.840+0000 7f9d58e00640 1 -- 172.21.15.26:0/784289735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d5415e1e0 msgr2=0x7f9d540702c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.840+0000 7f9d58e00640 1 --2- 172.21.15.26:0/784289735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d5415e1e0 0x7f9d540702c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d90 tx=0x7f9d4c02bea0 comp rx=0 tx=0).stop 2024-01-15T18:16:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.840+0000 7f9d58e00640 1 -- 172.21.15.26:0/784289735 shutdown_connections 2024-01-15T18:16:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.840+0000 7f9d58e00640 1 --2- 172.21.15.26:0/784289735 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d5415e1e0 0x7f9d540702c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.840+0000 7f9d58e00640 1 --2- 172.21.15.26:0/784289735 >> v2:172.21.15.124:3300/0 conn(0x7f9d540cf9a0 0x7f9d5415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.840+0000 7f9d58e00640 1 -- 172.21.15.26:0/784289735 >> 172.21.15.26:0/784289735 conn(0x7f9d5406a800 msgr2=0x7f9d5406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.840+0000 7f9d58e00640 1 -- 172.21.15.26:0/784289735 shutdown_connections 2024-01-15T18:16:29.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.840+0000 7f9d58e00640 1 -- 172.21.15.26:0/784289735 wait complete. 2024-01-15T18:16:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.842+0000 7f9d58e00640 1 Processor -- start 2024-01-15T18:16:29.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.843+0000 7f9d58e00640 1 -- start start 2024-01-15T18:16:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.844+0000 7f9d58e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d540cf9a0 0x7f9d541f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.844+0000 7f9d58e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d54143ae0 0x7f9d540aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.844+0000 7f9d58e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d5415e1e0 0x7f9d540ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.844+0000 7f9d58e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540b0360 con 0x7f9d540cf9a0 2024-01-15T18:16:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.844+0000 7f9d58e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540af090 con 0x7f9d5415e1e0 2024-01-15T18:16:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.844+0000 7f9d58e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540af2c0 con 0x7f9d54143ae0 2024-01-15T18:16:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.844+0000 7f9d51e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d540cf9a0 0x7f9d541f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.844+0000 7f9d51e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d540cf9a0 0x7f9d541f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:41188/0 (socket says 172.21.15.26:41188) 2024-01-15T18:16:29.847 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.844+0000 7f9d51e00640 1 -- 172.21.15.26:0/2953434900 learned_addr learned my addr 172.21.15.26:0/2953434900 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.845+0000 7f9d4be00640 1 --2- 172.21.15.26:0/2953434900 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d54143ae0 0x7f9d540aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:29.848 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.845+0000 7f9d52800640 1 --2- 172.21.15.26:0/2953434900 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d5415e1e0 0x7f9d540ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.845+0000 7f9d51e00640 1 -- 172.21.15.26:0/2953434900 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d54143ae0 msgr2=0x7f9d540aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.845+0000 7f9d51e00640 1 --2- 172.21.15.26:0/2953434900 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9d54143ae0 0x7f9d540aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.845+0000 7f9d51e00640 1 -- 172.21.15.26:0/2953434900 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d5415e1e0 msgr2=0x7f9d540ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.845+0000 7f9d51e00640 1 --2- 172.21.15.26:0/2953434900 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9d5415e1e0 0x7f9d540ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.845+0000 7f9d51e00640 1 -- 172.21.15.26:0/2953434900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009a70 con 0x7f9d540cf9a0 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.845+0000 7f9d51e00640 1 --2- 172.21.15.26:0/2953434900 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9d540cf9a0 0x7f9d541f2c90 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f9d4400c400 tx=0x7f9d4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.846+0000 7f9d4a000640 1 -- 172.21.15.26:0/2953434900 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d44005ce0 con 0x7f9d540cf9a0 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.846+0000 7f9d4a000640 1 -- 172.21.15.26:0/2953434900 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d44005e40 con 0x7f9d540cf9a0 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.846+0000 7f9d4a000640 1 -- 172.21.15.26:0/2953434900 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d440072f0 con 0x7f9d540cf9a0 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.846+0000 7f9d58e00640 1 -- 172.21.15.26:0/2953434900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d540af5a0 con 0x7f9d540cf9a0 2024-01-15T18:16:29.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.846+0000 7f9d58e00640 1 -- 172.21.15.26:0/2953434900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d540afb60 con 0x7f9d540cf9a0 2024-01-15T18:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.847+0000 7f9d58e00640 1 -- 172.21.15.26:0/2953434900 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d540675d0 con 0x7f9d540cf9a0 2024-01-15T18:16:29.850 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.848+0000 7f9d4a000640 1 -- 172.21.15.26:0/2953434900 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f9d44007510 con 0x7f9d540cf9a0 2024-01-15T18:16:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.849+0000 7f9d4a000640 1 --2- 172.21.15.26:0/2953434900 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d2c06c670 0x7f9d2c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:29.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.849+0000 7f9d4a000640 1 -- 172.21.15.26:0/2953434900 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f9d44088240 con 0x7f9d540cf9a0 2024-01-15T18:16:29.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.850+0000 7f9d4be00640 1 --2- 172.21.15.26:0/2953434900 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d2c06c670 0x7f9d2c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:29.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.851+0000 7f9d4be00640 1 --2- 172.21.15.26:0/2953434900 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9d2c06c670 0x7f9d2c06eb30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9d541f61e0 tx=0x7f9d3c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:29.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:29.853+0000 7f9d4a000640 1 -- 172.21.15.26:0/2953434900 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d44054750 con 0x7f9d540cf9a0 2024-01-15T18:16:30.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:30.078+0000 7f9d58e00640 1 -- 172.21.15.26:0/2953434900 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d5415e670 con 0x7f9d2c06c670 2024-01-15T18:16:30.100 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:30.097+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:30.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:30.098+0000 7f9d4a000640 1 -- 172.21.15.26:0/2953434900 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f9d3c011970 con 0x7f9d2c06c670 2024-01-15T18:16:30.101 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:30.117 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:32.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:32.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.352+0000 7f2dfb400640 1 Processor -- start 2024-01-15T18:16:32.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.353+0000 7f2dfb400640 1 -- start start 2024-01-15T18:16:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.354+0000 7f2dfb400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2df40d1ad0 0x7f2df40c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.354+0000 7f2dfb400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df40cec40 con 0x7f2df40d1650 2024-01-15T18:16:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.354+0000 7f2dfb400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df40d0420 con 0x7f2df40b34e0 2024-01-15T18:16:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.354+0000 7f2dfb400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df40d0e70 con 0x7f2df40d1ad0 2024-01-15T18:16:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.355+0000 7f2df9600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f2df40d1650 0x7f2df40c6be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55174/0 (socket says 172.21.15.26:55174) 2024-01-15T18:16:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.355+0000 7f2df9600640 1 -- 172.21.15.26:0/2286443330 learned_addr learned my addr 172.21.15.26:0/2286443330 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:32.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.355+0000 7f2dfaa00640 1 --2- 172.21.15.26:0/2286443330 >> v2:172.21.15.26:3300/0 conn(0x7f2df40d1ad0 0x7f2df40c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:32.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.355+0000 7f2df8c00640 1 -- 172.21.15.26:0/2286443330 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423750044 0 0) 0x7f2df40cec40 con 0x7f2df40d1650 2024-01-15T18:16:32.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.355+0000 7f2df8c00640 1 -- 172.21.15.26:0/2286443330 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dd4003520 con 0x7f2df40d1650 2024-01-15T18:16:32.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.355+0000 7f2dfaa00640 1 -- 172.21.15.26:0/2286443330 >> v1:172.21.15.124:6789/0 conn(0x7f2df40b34e0 legacy=0x7f2df40b3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:32.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.355+0000 7f2dfaa00640 1 -- 172.21.15.26:0/2286443330 >> v1:172.21.15.26:6789/0 conn(0x7f2df40d1650 legacy=0x7f2df40c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:32.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.355+0000 7f2dfaa00640 1 -- 172.21.15.26:0/2286443330 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40d1f60 con 0x7f2df40d1ad0 2024-01-15T18:16:32.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.357+0000 7f2dfaa00640 1 --2- 172.21.15.26:0/2286443330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2df40d1ad0 0x7f2df40c72f0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009d90 tx=0x7f2de402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721bac282519f8da server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:32.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.357+0000 7f2df8c00640 1 -- 172.21.15.26:0/2286443330 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de402cd00 con 0x7f2df40d1ad0 2024-01-15T18:16:32.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.357+0000 7f2df8c00640 1 -- 172.21.15.26:0/2286443330 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de402ce60 con 0x7f2df40d1ad0 2024-01-15T18:16:32.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.357+0000 7f2df8c00640 1 -- 172.21.15.26:0/2286443330 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4034660 con 0x7f2df40d1ad0 2024-01-15T18:16:32.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.357+0000 7f2dfb400640 1 -- 172.21.15.26:0/2286443330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2df40d1ad0 msgr2=0x7f2df40c72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:32.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.357+0000 7f2dfb400640 1 --2- 172.21.15.26:0/2286443330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2df40d1ad0 0x7f2df40c72f0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009d90 tx=0x7f2de402bf50 comp rx=0 tx=0).stop 2024-01-15T18:16:32.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.358+0000 7f2dfb400640 1 -- 172.21.15.26:0/2286443330 shutdown_connections 2024-01-15T18:16:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.358+0000 7f2dfb400640 1 --2- 172.21.15.26:0/2286443330 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2df40d1ad0 0x7f2df40c72f0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.358+0000 7f2dfb400640 1 -- 172.21.15.26:0/2286443330 >> 172.21.15.26:0/2286443330 conn(0x7f2df40d3090 msgr2=0x7f2df40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.358+0000 7f2dfb400640 1 -- 172.21.15.26:0/2286443330 shutdown_connections 2024-01-15T18:16:32.361 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.358+0000 7f2dfb400640 1 -- 172.21.15.26:0/2286443330 wait complete. 2024-01-15T18:16:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.360+0000 7f2dfb400640 1 Processor -- start 2024-01-15T18:16:32.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.360+0000 7f2dfb400640 1 -- start start 2024-01-15T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfb400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2df40b34e0 0x7f2df40b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfb400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2df40d1650 0x7f2df40ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfb400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2df40d1ad0 0x7f2df40ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:32.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfb400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40c66e0 con 0x7f2df40b34e0 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfb400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40b6c60 con 0x7f2df40d1650 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfb400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df40b6e70 con 0x7f2df40d1ad0 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2df40b34e0 0x7f2df40b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2df40b34e0 0x7f2df40b7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33890/0 (socket says 172.21.15.26:33890) 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfa000640 1 -- 172.21.15.26:0/3246356934 learned_addr learned my addr 172.21.15.26:0/3246356934 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.361+0000 7f2dfa000640 1 -- 172.21.15.26:0/3246356934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2df40d1ad0 msgr2=0x7f2df40ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2dfa000640 1 --2- 172.21.15.26:0/3246356934 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2df40d1ad0 0x7f2df40ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2dfa000640 1 -- 172.21.15.26:0/3246356934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2df40d1650 msgr2=0x7f2df40ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2dfa000640 1 --2- 172.21.15.26:0/3246356934 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2df40d1650 0x7f2df40ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2dfa000640 1 -- 172.21.15.26:0/3246356934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4009a70 con 0x7f2df40b34e0 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2dfa000640 1 --2- 172.21.15.26:0/3246356934 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2df40b34e0 0x7f2df40b7f10 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00c450 tx=0x7f2dec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2df2000640 1 -- 172.21.15.26:0/3246356934 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec0052d0 con 0x7f2df40b34e0 2024-01-15T18:16:32.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2df2000640 1 -- 172.21.15.26:0/3246356934 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dec005430 con 0x7f2df40b34e0 2024-01-15T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2df2000640 1 -- 172.21.15.26:0/3246356934 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec012870 con 0x7f2df40b34e0 2024-01-15T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2dfb400640 1 -- 172.21.15.26:0/3246356934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df40b7150 con 0x7f2df40b34e0 2024-01-15T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.362+0000 7f2dfb400640 1 -- 172.21.15.26:0/3246356934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df40b7710 con 0x7f2df40b34e0 2024-01-15T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.363+0000 7f2dfb400640 1 -- 172.21.15.26:0/3246356934 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df4067450 con 0x7f2df40b34e0 2024-01-15T18:16:32.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.364+0000 7f2df2000640 1 -- 172.21.15.26:0/3246356934 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f2dec0129d0 con 0x7f2df40b34e0 2024-01-15T18:16:32.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.365+0000 7f2df2000640 1 --2- 172.21.15.26:0/3246356934 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2dcc06c670 0x7f2dcc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:32.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.365+0000 7f2df2000640 1 -- 172.21.15.26:0/3246356934 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f2dec0883b0 con 0x7f2df40b34e0 2024-01-15T18:16:32.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.366+0000 7f2df9600640 1 --2- 172.21.15.26:0/3246356934 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2dcc06c670 0x7f2dcc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:32.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.367+0000 7f2df9600640 1 --2- 172.21.15.26:0/3246356934 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2dcc06c670 0x7f2dcc06eb30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2df41e9260 tx=0x7f2ddc007930 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:32.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.370+0000 7f2df2000640 1 -- 172.21.15.26:0/3246356934 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2dec0bee90 con 0x7f2df40b34e0 2024-01-15T18:16:32.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.595+0000 7f2dfb400640 1 -- 172.21.15.26:0/3246356934 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f2df40cf1c0 con 0x7f2dcc06c670 2024-01-15T18:16:32.618 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:32.614+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:32.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:32.616+0000 7f2df2000640 1 -- 172.21.15.26:0/3246356934 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f2ddc0058b0 con 0x7f2dcc06c670 2024-01-15T18:16:32.619 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:32.638 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:32.935 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:16:32.943 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:16:34.639 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:34.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.876+0000 7f0e5ec00640 1 Processor -- start 2024-01-15T18:16:34.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.876+0000 7f0e5ec00640 1 -- start start 2024-01-15T18:16:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.878+0000 7f0e5ec00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0e5813e360 0x7f0e580cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.878+0000 7f0e5ec00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e5815f9c0 0x7f0e580cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.878+0000 7f0e5ec00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e5815c5f0 con 0x7f0e5813df80 2024-01-15T18:16:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.878+0000 7f0e5ec00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5806c470 con 0x7f0e5815f9c0 2024-01-15T18:16:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.878+0000 7f0e5ec00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e5806c750 con 0x7f0e5813e360 2024-01-15T18:16:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.879+0000 7f0e5cc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e5815f9c0 0x7f0e580cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:34.881 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.879+0000 7f0e5cc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f0e5815f9c0 0x7f0e580cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33898/0 (socket says 172.21.15.26:33898) 2024-01-15T18:16:34.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.879+0000 7f0e5cc00640 1 -- 172.21.15.26:0/654137614 learned_addr learned my addr 172.21.15.26:0/654137614 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:34.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.879+0000 7f0e57400640 1 --2- 172.21.15.26:0/654137614 >> v2:172.21.15.124:3300/0 conn(0x7f0e5813e360 0x7f0e580cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:34.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.879+0000 7f0e5cc00640 1 -- 172.21.15.26:0/654137614 >> v2:172.21.15.124:3300/0 conn(0x7f0e5813e360 msgr2=0x7f0e580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:34.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.879+0000 7f0e5cc00640 1 --2- 172.21.15.26:0/654137614 >> v2:172.21.15.124:3300/0 conn(0x7f0e5813e360 0x7f0e580cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:34.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.879+0000 7f0e5cc00640 1 -- 172.21.15.26:0/654137614 >> v1:172.21.15.124:6789/0 conn(0x7f0e5813df80 legacy=0x7f0e5815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:34.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.879+0000 7f0e5cc00640 1 -- 172.21.15.26:0/654137614 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5806ca30 con 0x7f0e5815f9c0 2024-01-15T18:16:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.879+0000 7f0e57400640 1 --2- 172.21.15.26:0/654137614 >> v2:172.21.15.124:3300/0 conn(0x7f0e5813e360 0x7f0e580cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.880+0000 7f0e5cc00640 1 --2- 172.21.15.26:0/654137614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5815f9c0 0x7f0e580cf7d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f0e580ccae0 tx=0x7f0e5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc80c13e801e39d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.881+0000 7f0e56a00640 1 -- 172.21.15.26:0/654137614 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50031640 con 0x7f0e5815f9c0 2024-01-15T18:16:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.881+0000 7f0e56a00640 1 -- 172.21.15.26:0/654137614 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e500317a0 con 0x7f0e5815f9c0 2024-01-15T18:16:34.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.881+0000 7f0e56a00640 1 -- 172.21.15.26:0/654137614 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50031b30 con 0x7f0e5815f9c0 2024-01-15T18:16:34.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.881+0000 7f0e5ec00640 1 -- 172.21.15.26:0/654137614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5815f9c0 msgr2=0x7f0e580cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:34.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.881+0000 7f0e5ec00640 1 --2- 172.21.15.26:0/654137614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5815f9c0 0x7f0e580cf7d0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f0e580ccae0 tx=0x7f0e5002b8f0 comp rx=0 tx=0).stop 2024-01-15T18:16:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.882+0000 7f0e5ec00640 1 -- 172.21.15.26:0/654137614 shutdown_connections 2024-01-15T18:16:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.882+0000 7f0e5ec00640 1 --2- 172.21.15.26:0/654137614 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5815f9c0 0x7f0e580cf7d0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.882+0000 7f0e5ec00640 1 --2- 172.21.15.26:0/654137614 >> v2:172.21.15.124:3300/0 conn(0x7f0e5813e360 0x7f0e580cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.882+0000 7f0e5ec00640 1 -- 172.21.15.26:0/654137614 >> 172.21.15.26:0/654137614 conn(0x7f0e5806a800 msgr2=0x7f0e5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.882+0000 7f0e5ec00640 1 -- 172.21.15.26:0/654137614 shutdown_connections 2024-01-15T18:16:34.885 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.882+0000 7f0e5ec00640 1 -- 172.21.15.26:0/654137614 wait complete. 2024-01-15T18:16:34.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.883+0000 7f0e5ec00640 1 Processor -- start 2024-01-15T18:16:34.886 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.884+0000 7f0e5ec00640 1 -- start start 2024-01-15T18:16:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.885+0000 7f0e5ec00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5813df80 0x7f0e58150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.885+0000 7f0e5ec00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e5813e360 0x7f0e5813d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.885+0000 7f0e5ec00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e5815f9c0 0x7f0e5814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.885+0000 7f0e5ec00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5814ad60 con 0x7f0e5813df80 2024-01-15T18:16:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.885+0000 7f0e5ec00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5813de30 con 0x7f0e5815f9c0 2024-01-15T18:16:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.885+0000 7f0e5ec00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58150b90 con 0x7f0e5813e360 2024-01-15T18:16:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.885+0000 7f0e57e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5813df80 0x7f0e58150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:34.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.885+0000 7f0e57e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5813df80 0x7f0e58150580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33914/0 (socket says 172.21.15.26:33914) 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.885+0000 7f0e57e00640 1 -- 172.21.15.26:0/113233267 learned_addr learned my addr 172.21.15.26:0/113233267 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e5cc00640 1 --2- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e5815f9c0 0x7f0e5814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e57400640 1 --2- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e5813e360 0x7f0e5813d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e57e00640 1 -- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e5813e360 msgr2=0x7f0e5813d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e57e00640 1 --2- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f0e5813e360 0x7f0e5813d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e57e00640 1 -- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e5815f9c0 msgr2=0x7f0e5814ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e57e00640 1 --2- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e5815f9c0 0x7f0e5814ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e57e00640 1 -- 172.21.15.26:0/113233267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e500094d0 con 0x7f0e5813df80 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e5cc00640 1 --2- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0e5815f9c0 0x7f0e5814ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e57e00640 1 --2- 172.21.15.26:0/113233267 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0e5813df80 0x7f0e58150580 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800b3f0 tx=0x7f0e4800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e4fe00640 1 -- 172.21.15.26:0/113233267 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48011830 con 0x7f0e5813df80 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e4fe00640 1 -- 172.21.15.26:0/113233267 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48011990 con 0x7f0e5813df80 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e4fe00640 1 -- 172.21.15.26:0/113233267 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48011d40 con 0x7f0e5813df80 2024-01-15T18:16:34.890 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.886+0000 7f0e5ec00640 1 -- 172.21.15.26:0/113233267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e58150e70 con 0x7f0e5813df80 2024-01-15T18:16:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.887+0000 7f0e5ec00640 1 -- 172.21.15.26:0/113233267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5814a980 con 0x7f0e5813df80 2024-01-15T18:16:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.888+0000 7f0e5ec00640 1 -- 172.21.15.26:0/113233267 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e580675d0 con 0x7f0e5813df80 2024-01-15T18:16:34.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.889+0000 7f0e4fe00640 1 -- 172.21.15.26:0/113233267 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f0e4801a400 con 0x7f0e5813df80 2024-01-15T18:16:34.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.890+0000 7f0e4fe00640 1 --2- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e3006c670 0x7f0e3006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:34.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.890+0000 7f0e4fe00640 1 -- 172.21.15.26:0/113233267 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f0e48088650 con 0x7f0e5813df80 2024-01-15T18:16:34.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.890+0000 7f0e57400640 1 --2- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e3006c670 0x7f0e3006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:34.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.891+0000 7f0e57400640 1 --2- 172.21.15.26:0/113233267 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f0e3006c670 0x7f0e3006eb30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0e580ce650 tx=0x7f0e40008510 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:34.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:34.894+0000 7f0e4fe00640 1 -- 172.21.15.26:0/113233267 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e48054f10 con 0x7f0e5813df80 2024-01-15T18:16:35.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:35.119+0000 7f0e5ec00640 1 -- 172.21.15.26:0/113233267 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e5806cff0 con 0x7f0e3006c670 2024-01-15T18:16:35.140 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:35.136+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:35.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:35.138+0000 7f0e4fe00640 1 -- 172.21.15.26:0/113233267 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f0e40003990 con 0x7f0e3006c670 2024-01-15T18:16:35.140 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:35.154 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:37.156 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:37.388 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.386+0000 7f3781200640 1 Processor -- start 2024-01-15T18:16:37.389 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.386+0000 7f3781200640 1 -- start start 2024-01-15T18:16:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3781200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f377c0cc500 0x7f377c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3781200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377c070e50 con 0x7f377c143c40 2024-01-15T18:16:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3781200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377c144020 con 0x7f377c06d160 2024-01-15T18:16:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3781200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377c0723c0 con 0x7f377c0cc500 2024-01-15T18:16:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3773e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f377c0cc500 0x7f377c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3773e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f377c0cc500 0x7f377c0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33918/0 (socket says 172.21.15.26:33918) 2024-01-15T18:16:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3773e00640 1 -- 172.21.15.26:0/1386336396 learned_addr learned my addr 172.21.15.26:0/1386336396 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f377b400640 1 --1- 172.21.15.26:0/1386336396 >> v1:172.21.15.26:6789/0 conn(0x7f377c143c40 0x7f377c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55188/0 (socket says 172.21.15.26:55188) 2024-01-15T18:16:37.391 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3773e00640 1 -- 172.21.15.26:0/1386336396 >> v1:172.21.15.124:6789/0 conn(0x7f377c06d160 legacy=0x7f377c06d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3773e00640 1 -- 172.21.15.26:0/1386336396 >> v1:172.21.15.26:6789/0 conn(0x7f377c143c40 legacy=0x7f377c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.388+0000 7f3773e00640 1 -- 172.21.15.26:0/1386336396 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c071e00 con 0x7f377c0cc500 2024-01-15T18:16:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.390+0000 7f3773e00640 1 --2- 172.21.15.26:0/1386336396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c0cc500 0x7f377c0cc990 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f3764009870 tx=0x7f376402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7eebad774a1a65d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.390+0000 7f377aa00640 1 -- 172.21.15.26:0/1386336396 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764031870 con 0x7f377c0cc500 2024-01-15T18:16:37.392 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.390+0000 7f377aa00640 1 -- 172.21.15.26:0/1386336396 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37640319d0 con 0x7f377c0cc500 2024-01-15T18:16:37.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.390+0000 7f377aa00640 1 -- 172.21.15.26:0/1386336396 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764031d60 con 0x7f377c0cc500 2024-01-15T18:16:37.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.390+0000 7f3781200640 1 -- 172.21.15.26:0/1386336396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c0cc500 msgr2=0x7f377c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:37.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.390+0000 7f3781200640 1 --2- 172.21.15.26:0/1386336396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c0cc500 0x7f377c0cc990 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f3764009870 tx=0x7f376402b910 comp rx=0 tx=0).stop 2024-01-15T18:16:37.393 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.391+0000 7f3781200640 1 -- 172.21.15.26:0/1386336396 shutdown_connections 2024-01-15T18:16:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.391+0000 7f3781200640 1 --2- 172.21.15.26:0/1386336396 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c0cc500 0x7f377c0cc990 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.391+0000 7f3781200640 1 -- 172.21.15.26:0/1386336396 >> 172.21.15.26:0/1386336396 conn(0x7f377c06a800 msgr2=0x7f377c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.391+0000 7f3781200640 1 -- 172.21.15.26:0/1386336396 shutdown_connections 2024-01-15T18:16:37.394 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.391+0000 7f3781200640 1 -- 172.21.15.26:0/1386336396 wait complete. 2024-01-15T18:16:37.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.392+0000 7f3781200640 1 Processor -- start 2024-01-15T18:16:37.395 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.392+0000 7f3781200640 1 -- start start 2024-01-15T18:16:37.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.393+0000 7f3781200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c06d160 0x7f377c0a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:37.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.393+0000 7f3781200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f377c0cc500 0x7f377c139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:37.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.393+0000 7f3781200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f377c143c40 0x7f377c139d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:37.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.393+0000 7f3781200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c139460 con 0x7f377c06d160 2024-01-15T18:16:37.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.393+0000 7f3781200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c13a370 con 0x7f377c143c40 2024-01-15T18:16:37.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.393+0000 7f3781200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c13a650 con 0x7f377c0cc500 2024-01-15T18:16:37.396 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c06d160 0x7f377c0a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:37.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377b400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c06d160 0x7f377c0a8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33920/0 (socket says 172.21.15.26:33920) 2024-01-15T18:16:37.397 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377b400640 1 -- 172.21.15.26:0/2893545218 learned_addr learned my addr 172.21.15.26:0/2893545218 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:37.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377be00640 1 --2- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f377c143c40 0x7f377c139d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:37.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f3773e00640 1 --2- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f377c0cc500 0x7f377c139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:37.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377b400640 1 -- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f377c0cc500 msgr2=0x7f377c139800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:37.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377b400640 1 --2- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f377c0cc500 0x7f377c139800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:37.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377b400640 1 -- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f377c143c40 msgr2=0x7f377c139d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:37.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377b400640 1 --2- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f377c143c40 0x7f377c139d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:37.398 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377b400640 1 -- 172.21.15.26:0/2893545218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37640094d0 con 0x7f377c06d160 2024-01-15T18:16:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.394+0000 7f377be00640 1 --2- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f377c143c40 0x7f377c139d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.395+0000 7f377b400640 1 --2- 172.21.15.26:0/2893545218 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f377c06d160 0x7f377c0a8db0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c4d0 tx=0x7f376c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.395+0000 7f3773400640 1 -- 172.21.15.26:0/2893545218 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c007480 con 0x7f377c06d160 2024-01-15T18:16:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.395+0000 7f3773400640 1 -- 172.21.15.26:0/2893545218 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c0075e0 con 0x7f377c06d160 2024-01-15T18:16:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.395+0000 7f3773400640 1 -- 172.21.15.26:0/2893545218 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c0049e0 con 0x7f377c06d160 2024-01-15T18:16:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.395+0000 7f3781200640 1 -- 172.21.15.26:0/2893545218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c13a930 con 0x7f377c06d160 2024-01-15T18:16:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.395+0000 7f3781200640 1 -- 172.21.15.26:0/2893545218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c13aef0 con 0x7f377c06d160 2024-01-15T18:16:37.399 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.396+0000 7f3781200640 1 -- 172.21.15.26:0/2893545218 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c13b0e0 con 0x7f377c06d160 2024-01-15T18:16:37.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.397+0000 7f3773400640 1 -- 172.21.15.26:0/2893545218 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f376c004b40 con 0x7f377c06d160 2024-01-15T18:16:37.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.398+0000 7f3773400640 1 --2- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f375406c670 0x7f375406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:37.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.398+0000 7f3773400640 1 -- 172.21.15.26:0/2893545218 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f376c087d00 con 0x7f377c06d160 2024-01-15T18:16:37.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.399+0000 7f3773e00640 1 --2- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f375406c670 0x7f375406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:37.402 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.400+0000 7f3773e00640 1 --2- 172.21.15.26:0/2893545218 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f375406c670 0x7f375406eb30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f3764036000 tx=0x7f376403c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:37.405 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.403+0000 7f3773400640 1 -- 172.21.15.26:0/2893545218 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f376c0542c0 con 0x7f377c06d160 2024-01-15T18:16:37.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.622+0000 7f3781200640 1 -- 172.21.15.26:0/2893545218 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f377c06cc40 con 0x7f375406c670 2024-01-15T18:16:37.648 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:37.645+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:37.647+0000 7f3773400640 1 -- 172.21.15.26:0/2893545218 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f3764031db0 con 0x7f375406c670 2024-01-15T18:16:37.649 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:37.668 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:39.669 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:39.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.903+0000 7f4e8be00640 1 Processor -- start 2024-01-15T18:16:39.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.904+0000 7f4e8be00640 1 -- start start 2024-01-15T18:16:39.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.905+0000 7f4e8be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e8c143c40 0x7f4e8c071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:39.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.905+0000 7f4e8be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f4e8c0ccc10 0x7f4e8c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:39.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.906+0000 7f4e8be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c070e50 con 0x7f4e8c0717d0 2024-01-15T18:16:39.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.906+0000 7f4e8be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8c0723c0 con 0x7f4e8c143c40 2024-01-15T18:16:39.908 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.906+0000 7f4e8be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8c144020 con 0x7f4e8c0ccc10 2024-01-15T18:16:39.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.906+0000 7f4e8a000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4e8c0717d0 0x7f4e8c0cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:55200/0 (socket says 172.21.15.26:55200) 2024-01-15T18:16:39.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.906+0000 7f4e8a000640 1 -- 172.21.15.26:0/893543067 learned_addr learned my addr 172.21.15.26:0/893543067 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:39.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.906+0000 7f4e8aa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4e8c143c40 0x7f4e8c071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:39.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.906+0000 7f4e8b400640 1 --2- 172.21.15.26:0/893543067 >> v2:172.21.15.124:3300/0 conn(0x7f4e8c0ccc10 0x7f4e8c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:39.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.907+0000 7f4e89600640 1 -- 172.21.15.26:0/893543067 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197527138 0 0) 0x7f4e8c070e50 con 0x7f4e8c0717d0 2024-01-15T18:16:39.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.907+0000 7f4e89600640 1 -- 172.21.15.26:0/893543067 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e6c003520 con 0x7f4e8c0717d0 2024-01-15T18:16:39.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.907+0000 7f4e8aa00640 1 -- 172.21.15.26:0/893543067 >> v2:172.21.15.124:3300/0 conn(0x7f4e8c0ccc10 msgr2=0x7f4e8c06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:39.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.907+0000 7f4e8aa00640 1 --2- 172.21.15.26:0/893543067 >> v2:172.21.15.124:3300/0 conn(0x7f4e8c0ccc10 0x7f4e8c06d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:39.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.907+0000 7f4e8aa00640 1 -- 172.21.15.26:0/893543067 >> v1:172.21.15.26:6789/0 conn(0x7f4e8c0717d0 legacy=0x7f4e8c0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:39.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.907+0000 7f4e8aa00640 1 -- 172.21.15.26:0/893543067 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c071e00 con 0x7f4e8c143c40 2024-01-15T18:16:39.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.908+0000 7f4e8aa00640 1 --2- 172.21.15.26:0/893543067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e8c143c40 0x7f4e8c071290 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c070690 tx=0x7f4e7c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c00714b921c6b69 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:39.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.909+0000 7f4e89600640 1 -- 172.21.15.26:0/893543067 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c02cbf0 con 0x7f4e8c143c40 2024-01-15T18:16:39.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.909+0000 7f4e89600640 1 -- 172.21.15.26:0/893543067 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c02cd50 con 0x7f4e8c143c40 2024-01-15T18:16:39.911 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.909+0000 7f4e89600640 1 -- 172.21.15.26:0/893543067 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c034660 con 0x7f4e8c143c40 2024-01-15T18:16:39.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.909+0000 7f4e8be00640 1 -- 172.21.15.26:0/893543067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e8c143c40 msgr2=0x7f4e8c071290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:39.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.909+0000 7f4e8be00640 1 --2- 172.21.15.26:0/893543067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e8c143c40 0x7f4e8c071290 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c070690 tx=0x7f4e7c02be40 comp rx=0 tx=0).stop 2024-01-15T18:16:39.912 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.910+0000 7f4e8be00640 1 -- 172.21.15.26:0/893543067 shutdown_connections 2024-01-15T18:16:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.910+0000 7f4e8be00640 1 --2- 172.21.15.26:0/893543067 >> v2:172.21.15.124:3300/0 conn(0x7f4e8c0ccc10 0x7f4e8c06d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.910+0000 7f4e8be00640 1 --2- 172.21.15.26:0/893543067 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e8c143c40 0x7f4e8c071290 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.910+0000 7f4e8be00640 1 -- 172.21.15.26:0/893543067 >> 172.21.15.26:0/893543067 conn(0x7f4e8c06a800 msgr2=0x7f4e8c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.910+0000 7f4e8be00640 1 -- 172.21.15.26:0/893543067 shutdown_connections 2024-01-15T18:16:39.913 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.910+0000 7f4e8be00640 1 -- 172.21.15.26:0/893543067 wait complete. 2024-01-15T18:16:39.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.911+0000 7f4e8be00640 1 Processor -- start 2024-01-15T18:16:39.914 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.912+0000 7f4e8be00640 1 -- start start 2024-01-15T18:16:39.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.912+0000 7f4e8be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e8c0717d0 0x7f4e8c0ca810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:39.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e8c0ccc10 0x7f4e8c0ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:39.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e8c143c40 0x7f4e8c0ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:39.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0a9ea0 con 0x7f4e8c0717d0 2024-01-15T18:16:39.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0ab740 con 0x7f4e8c143c40 2024-01-15T18:16:39.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0cad50 con 0x7f4e8c0ccc10 2024-01-15T18:16:39.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e8c0717d0 0x7f4e8c0ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:39.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e8c0717d0 0x7f4e8c0ca810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:33948/0 (socket says 172.21.15.26:33948) 2024-01-15T18:16:39.916 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8aa00640 1 -- 172.21.15.26:0/3177459636 learned_addr learned my addr 172.21.15.26:0/3177459636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:39.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8a000640 1 --2- 172.21.15.26:0/3177459636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e8c0ccc10 0x7f4e8c0ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:39.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8aa00640 1 -- 172.21.15.26:0/3177459636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e8c0ccc10 msgr2=0x7f4e8c0ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:39.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8aa00640 1 --2- 172.21.15.26:0/3177459636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4e8c0ccc10 0x7f4e8c0ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:39.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8aa00640 1 -- 172.21.15.26:0/3177459636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e8c143c40 msgr2=0x7f4e8c0ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:39.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8aa00640 1 --2- 172.21.15.26:0/3177459636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4e8c143c40 0x7f4e8c0ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:39.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.913+0000 7f4e8aa00640 1 -- 172.21.15.26:0/3177459636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c009a70 con 0x7f4e8c0717d0 2024-01-15T18:16:39.917 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.914+0000 7f4e8aa00640 1 --2- 172.21.15.26:0/3177459636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4e8c0717d0 0x7f4e8c0ca810 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c037040 tx=0x7f4e7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:39.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.914+0000 7f4e82a00640 1 -- 172.21.15.26:0/3177459636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c040650 con 0x7f4e8c0717d0 2024-01-15T18:16:39.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.914+0000 7f4e82a00640 1 -- 172.21.15.26:0/3177459636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c0407f0 con 0x7f4e8c0717d0 2024-01-15T18:16:39.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.914+0000 7f4e82a00640 1 -- 172.21.15.26:0/3177459636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c040ba0 con 0x7f4e8c0717d0 2024-01-15T18:16:39.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.914+0000 7f4e8be00640 1 -- 172.21.15.26:0/3177459636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8c0a9bc0 con 0x7f4e8c0717d0 2024-01-15T18:16:39.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.914+0000 7f4e8be00640 1 -- 172.21.15.26:0/3177459636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8c0ab3d0 con 0x7f4e8c0717d0 2024-01-15T18:16:39.918 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.915+0000 7f4e8be00640 1 -- 172.21.15.26:0/3177459636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8c0aaf50 con 0x7f4e8c0717d0 2024-01-15T18:16:39.919 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.917+0000 7f4e82a00640 1 -- 172.21.15.26:0/3177459636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f4e7c040d00 con 0x7f4e8c0717d0 2024-01-15T18:16:39.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.918+0000 7f4e82a00640 1 --2- 172.21.15.26:0/3177459636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e6406c6a0 0x7f4e6406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:39.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.919+0000 7f4e82a00640 1 -- 172.21.15.26:0/3177459636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f4e7c0aba20 con 0x7f4e8c0717d0 2024-01-15T18:16:39.921 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.919+0000 7f4e8a000640 1 --2- 172.21.15.26:0/3177459636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e6406c6a0 0x7f4e6406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:39.922 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.920+0000 7f4e8a000640 1 --2- 172.21.15.26:0/3177459636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f4e6406c6a0 0x7f4e6406eb60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0cbfc0 tx=0x7f4e74009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:39.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:39.923+0000 7f4e82a00640 1 -- 172.21.15.26:0/3177459636 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e8c0aaf50 con 0x7f4e8c0717d0 2024-01-15T18:16:40.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:40.145+0000 7f4e8be00640 1 -- 172.21.15.26:0/3177459636 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e8c0701b0 con 0x7f4e6406c6a0 2024-01-15T18:16:40.172 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:40.168+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:40.172 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:40.169+0000 7f4e82a00640 1 -- 172.21.15.26:0/3177459636 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f4e8c0701b0 con 0x7f4e6406c6a0 2024-01-15T18:16:40.172 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:40.185 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:42.187 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.420+0000 7ff6ec600640 1 Processor -- start 2024-01-15T18:16:42.423 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.420+0000 7ff6ec600640 1 -- start start 2024-01-15T18:16:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.422+0000 7ff6ec600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff6e40cf9a0 0x7ff6e4070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:42.425 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.422+0000 7ff6ec600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e4071690 con 0x7ff6e4070ea0 2024-01-15T18:16:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.422+0000 7ff6ec600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e4071970 con 0x7ff6e414b2c0 2024-01-15T18:16:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.422+0000 7ff6ec600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e414b6a0 con 0x7ff6e40cf9a0 2024-01-15T18:16:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.423+0000 7ff6ea600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff6e4070ea0 0x7ff6e40cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57252/0 (socket says 172.21.15.26:57252) 2024-01-15T18:16:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.423+0000 7ff6ea600640 1 -- 172.21.15.26:0/3128489079 learned_addr learned my addr 172.21.15.26:0/3128489079 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:42.426 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.423+0000 7ff6e9200640 1 --2- 172.21.15.26:0/3128489079 >> v2:172.21.15.26:3300/0 conn(0x7ff6e40cf9a0 0x7ff6e4070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.423+0000 7ff6e9200640 1 -- 172.21.15.26:0/3128489079 >> v1:172.21.15.124:6789/0 conn(0x7ff6e414b2c0 legacy=0x7ff6e40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.423+0000 7ff6e9200640 1 -- 172.21.15.26:0/3128489079 >> v1:172.21.15.26:6789/0 conn(0x7ff6e4070ea0 legacy=0x7ff6e40cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.423+0000 7ff6e9200640 1 -- 172.21.15.26:0/3128489079 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4071c50 con 0x7ff6e40cf9a0 2024-01-15T18:16:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.423+0000 7ff6e3e00640 1 -- 172.21.15.26:0/3128489079 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575812983 0 0) 0x7ff6e4071690 con 0x7ff6e4070ea0 2024-01-15T18:16:42.427 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.424+0000 7ff6e9200640 1 --2- 172.21.15.26:0/3128489079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40cf9a0 0x7ff6e4070960 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0097a0 tx=0x7ff6cc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36781891b34a8b60 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.425+0000 7ff6e3e00640 1 -- 172.21.15.26:0/3128489079 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc030610 con 0x7ff6e40cf9a0 2024-01-15T18:16:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.425+0000 7ff6e3e00640 1 -- 172.21.15.26:0/3128489079 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6cc030770 con 0x7ff6e40cf9a0 2024-01-15T18:16:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.425+0000 7ff6e3e00640 1 -- 172.21.15.26:0/3128489079 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6cc030b00 con 0x7ff6e40cf9a0 2024-01-15T18:16:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.425+0000 7ff6ec600640 1 -- 172.21.15.26:0/3128489079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40cf9a0 msgr2=0x7ff6e4070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:42.428 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.425+0000 7ff6ec600640 1 --2- 172.21.15.26:0/3128489079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40cf9a0 0x7ff6e4070960 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc0097a0 tx=0x7ff6cc02b960 comp rx=0 tx=0).stop 2024-01-15T18:16:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.425+0000 7ff6ec600640 1 -- 172.21.15.26:0/3128489079 shutdown_connections 2024-01-15T18:16:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.425+0000 7ff6ec600640 1 --2- 172.21.15.26:0/3128489079 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e40cf9a0 0x7ff6e4070960 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.426+0000 7ff6ec600640 1 -- 172.21.15.26:0/3128489079 >> 172.21.15.26:0/3128489079 conn(0x7ff6e406a800 msgr2=0x7ff6e406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.426+0000 7ff6ec600640 1 -- 172.21.15.26:0/3128489079 shutdown_connections 2024-01-15T18:16:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.426+0000 7ff6ec600640 1 -- 172.21.15.26:0/3128489079 wait complete. 2024-01-15T18:16:42.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.427+0000 7ff6ec600640 1 Processor -- start 2024-01-15T18:16:42.430 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.427+0000 7ff6ec600640 1 -- start start 2024-01-15T18:16:42.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.428+0000 7ff6ec600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4070ea0 0x7ff6e4158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:42.432 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.428+0000 7ff6ec600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e40cf9a0 0x7ff6e4161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.428+0000 7ff6ec600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e414b2c0 0x7ff6e4141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.428+0000 7ff6ec600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e41627a0 con 0x7ff6e4070ea0 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.428+0000 7ff6ec600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4162a80 con 0x7ff6e40cf9a0 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.428+0000 7ff6ec600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4162d60 con 0x7ff6e414b2c0 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6e9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4070ea0 0x7ff6e4158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6e9c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4070ea0 0x7ff6e4158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55406/0 (socket says 172.21.15.26:55406) 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6e9c00640 1 -- 172.21.15.26:0/1935877201 learned_addr learned my addr 172.21.15.26:0/1935877201 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6e9200640 1 --2- 172.21.15.26:0/1935877201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e40cf9a0 0x7ff6e4161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6ea600640 1 --2- 172.21.15.26:0/1935877201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e414b2c0 0x7ff6e4141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6e9c00640 1 -- 172.21.15.26:0/1935877201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e414b2c0 msgr2=0x7ff6e4141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6e9c00640 1 --2- 172.21.15.26:0/1935877201 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff6e414b2c0 0x7ff6e4141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6e9c00640 1 -- 172.21.15.26:0/1935877201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e40cf9a0 msgr2=0x7ff6e4161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6e9c00640 1 --2- 172.21.15.26:0/1935877201 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff6e40cf9a0 0x7ff6e4161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:42.433 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.429+0000 7ff6e9c00640 1 -- 172.21.15.26:0/1935877201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc009400 con 0x7ff6e4070ea0 2024-01-15T18:16:42.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.430+0000 7ff6e9c00640 1 --2- 172.21.15.26:0/1935877201 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff6e4070ea0 0x7ff6e4158000 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400c400 tx=0x7ff6d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:42.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.430+0000 7ff6e1600640 1 -- 172.21.15.26:0/1935877201 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40052d0 con 0x7ff6e4070ea0 2024-01-15T18:16:42.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.430+0000 7ff6e1600640 1 -- 172.21.15.26:0/1935877201 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d4005430 con 0x7ff6e4070ea0 2024-01-15T18:16:42.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.430+0000 7ff6e1600640 1 -- 172.21.15.26:0/1935877201 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4003830 con 0x7ff6e4070ea0 2024-01-15T18:16:42.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.430+0000 7ff6ec600640 1 -- 172.21.15.26:0/1935877201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e4163040 con 0x7ff6e4070ea0 2024-01-15T18:16:42.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.430+0000 7ff6ec600640 1 -- 172.21.15.26:0/1935877201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e4158540 con 0x7ff6e4070ea0 2024-01-15T18:16:42.434 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.431+0000 7ff6ec600640 1 -- 172.21.15.26:0/1935877201 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e40675d0 con 0x7ff6e4070ea0 2024-01-15T18:16:42.435 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.433+0000 7ff6e1600640 1 -- 172.21.15.26:0/1935877201 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7ff6d4003990 con 0x7ff6e4070ea0 2024-01-15T18:16:42.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.433+0000 7ff6e1600640 1 --2- 172.21.15.26:0/1935877201 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff6bc06c670 0x7ff6bc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:42.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.434+0000 7ff6e1600640 1 -- 172.21.15.26:0/1935877201 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7ff6d4087df0 con 0x7ff6e4070ea0 2024-01-15T18:16:42.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.434+0000 7ff6e9200640 1 --2- 172.21.15.26:0/1935877201 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff6bc06c670 0x7ff6bc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:42.437 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.435+0000 7ff6e9200640 1 --2- 172.21.15.26:0/1935877201 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff6bc06c670 0x7ff6bc06eb30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc036000 tx=0x7ff6cc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:42.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.439+0000 7ff6e1600640 1 -- 172.21.15.26:0/1935877201 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6d40546b0 con 0x7ff6e4070ea0 2024-01-15T18:16:42.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.658+0000 7ff6ec600640 1 -- 172.21.15.26:0/1935877201 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6e40cbc40 con 0x7ff6bc06c670 2024-01-15T18:16:42.678 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:42.675+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:42.676+0000 7ff6e1600640 1 -- 172.21.15.26:0/1935877201 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7ff6cc0309e0 con 0x7ff6bc06c670 2024-01-15T18:16:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:42.690 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:44.691 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:44.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.920+0000 7fd454c00640 1 Processor -- start 2024-01-15T18:16:44.923 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.920+0000 7fd454c00640 1 -- start start 2024-01-15T18:16:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.922+0000 7fd454c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd450143c40 0x7fd450071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.922+0000 7fd454c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd4500ccc10 0x7fd45006d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.922+0000 7fd454c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd450070e50 con 0x7fd4500717d0 2024-01-15T18:16:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.922+0000 7fd454c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4500723c0 con 0x7fd4500ccc10 2024-01-15T18:16:44.925 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.922+0000 7fd454c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd450144020 con 0x7fd450143c40 2024-01-15T18:16:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.923+0000 7fd447e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd4500717d0 0x7fd4500cc500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57266/0 (socket says 172.21.15.26:57266) 2024-01-15T18:16:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.923+0000 7fd447e00640 1 -- 172.21.15.26:0/2260404237 learned_addr learned my addr 172.21.15.26:0/2260404237 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.923+0000 7fd44f400640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd450143c40 0x7fd450071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.923+0000 7fd44fe00640 1 --2- 172.21.15.26:0/2260404237 >> v2:172.21.15.124:3300/0 conn(0x7fd4500ccc10 0x7fd45006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.927 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.923+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324402132 0 0) 0x7fd450070e50 con 0x7fd4500717d0 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.923+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd430003520 con 0x7fd4500717d0 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1584782184 0 0) 0x7fd430003520 con 0x7fd4500717d0 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd450143c40 msgr2=0x7fd450071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd450143c40 0x7fd450071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 >> v2:172.21.15.124:3300/0 conn(0x7fd4500ccc10 msgr2=0x7fd45006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 --2- 172.21.15.26:0/2260404237 >> v2:172.21.15.124:3300/0 conn(0x7fd4500ccc10 0x7fd45006d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd450071e00 con 0x7fd4500717d0 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd438001670 con 0x7fd4500717d0 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 >> v1:172.21.15.26:6789/0 conn(0x7fd4500717d0 legacy=0x7fd4500cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd450143c40 0x7fd430003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430003f40 0x7fd4300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd430004d00 0x7fd430005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4500723c0 con 0x7fd430003f40 2024-01-15T18:16:44.928 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd450144020 con 0x7fd430004d00 2024-01-15T18:16:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43000ba30 con 0x7fd450143c40 2024-01-15T18:16:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.924+0000 7fd447e00640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430003f40 0x7fd4300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.925+0000 7fd44f400640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd450143c40 0x7fd430003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.925+0000 7fd447e00640 1 -- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd450143c40 msgr2=0x7fd430003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.925+0000 7fd447e00640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd450143c40 0x7fd430003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.925+0000 7fd447e00640 1 -- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd430004d00 msgr2=0x7fd430005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.925+0000 7fd447e00640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd430004d00 0x7fd430005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.925+0000 7fd447e00640 1 -- 172.21.15.26:0/2260404237 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd430003520 con 0x7fd430003f40 2024-01-15T18:16:44.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.925+0000 7fd44f400640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd450143c40 0x7fd430003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.926+0000 7fd447e00640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430003f40 0x7fd4300047c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c160 tx=0x7fd4300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34b28f6a9b3b609 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.926+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438005ad0 con 0x7fd430003f40 2024-01-15T18:16:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.926+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd438005c30 con 0x7fd430003f40 2024-01-15T18:16:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.926+0000 7fd44ea00640 1 -- 172.21.15.26:0/2260404237 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43802c810 con 0x7fd430003f40 2024-01-15T18:16:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.927+0000 7fd454c00640 1 -- 172.21.15.26:0/2260404237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430003f40 msgr2=0x7fd4300047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.927+0000 7fd454c00640 1 --2- 172.21.15.26:0/2260404237 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd430003f40 0x7fd4300047c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd43800c160 tx=0x7fd4300057f0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.930 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.927+0000 7fd44fe00640 1 -- 172.21.15.26:0/2260404237 reap_dead start 2024-01-15T18:16:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.927+0000 7fd454c00640 1 -- 172.21.15.26:0/2260404237 shutdown_connections 2024-01-15T18:16:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.927+0000 7fd454c00640 1 -- 172.21.15.26:0/2260404237 >> 172.21.15.26:0/2260404237 conn(0x7fd45006a800 msgr2=0x7fd45006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.927+0000 7fd454c00640 1 -- 172.21.15.26:0/2260404237 shutdown_connections 2024-01-15T18:16:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.927+0000 7fd454c00640 1 -- 172.21.15.26:0/2260404237 wait complete. 2024-01-15T18:16:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.929+0000 7fd454c00640 1 Processor -- start 2024-01-15T18:16:44.931 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.929+0000 7fd454c00640 1 -- start start 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd454c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd430003f40 0x7fd4500afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd454c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd45015a3a0 0x7fd45015a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd454c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501f2ca0 0x7fd4501f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd454c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501f5f20 con 0x7fd4501f2ca0 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd454c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501f6200 con 0x7fd45015a3a0 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd454c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4501f64e0 con 0x7fd430003f40 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd44fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501f2ca0 0x7fd4501f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd44fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501f2ca0 0x7fd4501f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55424/0 (socket says 172.21.15.26:55424) 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd44fe00640 1 -- 172.21.15.26:0/3698266174 learned_addr learned my addr 172.21.15.26:0/3698266174 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd447e00640 1 --2- 172.21.15.26:0/3698266174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd45015a3a0 0x7fd45015a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.933 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd44f400640 1 --2- 172.21.15.26:0/3698266174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd430003f40 0x7fd4500afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd44fe00640 1 -- 172.21.15.26:0/3698266174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd430003f40 msgr2=0x7fd4500afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd44fe00640 1 --2- 172.21.15.26:0/3698266174 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd430003f40 0x7fd4500afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.930+0000 7fd44fe00640 1 -- 172.21.15.26:0/3698266174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd45015a3a0 msgr2=0x7fd45015a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.931+0000 7fd44fe00640 1 --2- 172.21.15.26:0/3698266174 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd45015a3a0 0x7fd45015a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:44.934 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.931+0000 7fd44fe00640 1 -- 172.21.15.26:0/3698266174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4501f67c0 con 0x7fd4501f2ca0 2024-01-15T18:16:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.931+0000 7fd44fe00640 1 --2- 172.21.15.26:0/3698266174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd4501f2ca0 0x7fd4501f3120 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fd4500cbcd0 tx=0x7fd44800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.931+0000 7fd447400640 1 -- 172.21.15.26:0/3698266174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448010040 con 0x7fd4501f2ca0 2024-01-15T18:16:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.931+0000 7fd447400640 1 -- 172.21.15.26:0/3698266174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44800bae0 con 0x7fd4501f2ca0 2024-01-15T18:16:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.931+0000 7fd454c00640 1 -- 172.21.15.26:0/3698266174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4501f6aa0 con 0x7fd4501f2ca0 2024-01-15T18:16:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.931+0000 7fd447400640 1 -- 172.21.15.26:0/3698266174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44800be70 con 0x7fd4501f2ca0 2024-01-15T18:16:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.931+0000 7fd454c00640 1 -- 172.21.15.26:0/3698266174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4501f3660 con 0x7fd4501f2ca0 2024-01-15T18:16:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.932+0000 7fd454c00640 1 -- 172.21.15.26:0/3698266174 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4501f6c90 con 0x7fd4501f2ca0 2024-01-15T18:16:44.935 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.933+0000 7fd447400640 1 -- 172.21.15.26:0/3698266174 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fd44801e410 con 0x7fd4501f2ca0 2024-01-15T18:16:44.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.934+0000 7fd447400640 1 --2- 172.21.15.26:0/3698266174 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd42806c6a0 0x7fd42806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:44.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.934+0000 7fd447400640 1 -- 172.21.15.26:0/3698266174 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fd448088f10 con 0x7fd4501f2ca0 2024-01-15T18:16:44.937 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.935+0000 7fd44f400640 1 --2- 172.21.15.26:0/3698266174 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd42806c6a0 0x7fd42806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:44.938 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.936+0000 7fd44f400640 1 --2- 172.21.15.26:0/3698266174 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd42806c6a0 0x7fd42806eb60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd4400099c0 tx=0x7fd440008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:44.942 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:44.939+0000 7fd447400640 1 -- 172.21.15.26:0/3698266174 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd448055420 con 0x7fd4501f2ca0 2024-01-15T18:16:45.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:45.158+0000 7fd454c00640 1 -- 172.21.15.26:0/3698266174 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4501f6370 con 0x7fd42806c6a0 2024-01-15T18:16:45.182 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:45.179+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:45.183 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:45.180+0000 7fd447400640 1 -- 172.21.15.26:0/3698266174 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fd4501f6370 con 0x7fd42806c6a0 2024-01-15T18:16:45.183 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:45.199 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:47.201 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.433+0000 7f9b5e600640 1 Processor -- start 2024-01-15T18:16:47.436 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.433+0000 7f9b5e600640 1 -- start start 2024-01-15T18:16:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.435+0000 7f9b5e600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5813e360 0x7f9b580cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.435+0000 7f9b5e600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b5815f9c0 0x7f9b580cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.435+0000 7f9b5e600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5815c5f0 con 0x7f9b5813df80 2024-01-15T18:16:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.435+0000 7f9b5e600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5806c470 con 0x7f9b5815f9c0 2024-01-15T18:16:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.435+0000 7f9b5e600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5806c750 con 0x7f9b5813e360 2024-01-15T18:16:47.438 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.435+0000 7f9b57e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b5815f9c0 0x7f9b580cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.435+0000 7f9b57e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9b5815f9c0 0x7f9b580cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55432/0 (socket says 172.21.15.26:55432) 2024-01-15T18:16:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.435+0000 7f9b57e00640 1 -- 172.21.15.26:0/1375456645 learned_addr learned my addr 172.21.15.26:0/1375456645 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.435+0000 7f9b4fe00640 1 --2- 172.21.15.26:0/1375456645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5813e360 0x7f9b580cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.436+0000 7f9b57e00640 1 -- 172.21.15.26:0/1375456645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5813e360 msgr2=0x7f9b580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.436+0000 7f9b57e00640 1 --2- 172.21.15.26:0/1375456645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5813e360 0x7f9b580cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.436+0000 7f9b57e00640 1 -- 172.21.15.26:0/1375456645 >> v1:172.21.15.124:6789/0 conn(0x7f9b5813df80 legacy=0x7f9b5815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:47.439 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.436+0000 7f9b57e00640 1 -- 172.21.15.26:0/1375456645 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5806ca30 con 0x7f9b5815f9c0 2024-01-15T18:16:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.437+0000 7f9b57e00640 1 --2- 172.21.15.26:0/1375456645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5815f9c0 0x7f9b580cf7d0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009e50 tx=0x7f9b5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0e278038509b57 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.437+0000 7f9b56a00640 1 -- 172.21.15.26:0/1375456645 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5002ca70 con 0x7f9b5815f9c0 2024-01-15T18:16:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.437+0000 7f9b56a00640 1 -- 172.21.15.26:0/1375456645 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5002cbd0 con 0x7f9b5815f9c0 2024-01-15T18:16:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.437+0000 7f9b56a00640 1 -- 172.21.15.26:0/1375456645 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b500343f0 con 0x7f9b5815f9c0 2024-01-15T18:16:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.438+0000 7f9b5e600640 1 -- 172.21.15.26:0/1375456645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5815f9c0 msgr2=0x7f9b580cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.440 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.438+0000 7f9b5e600640 1 --2- 172.21.15.26:0/1375456645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5815f9c0 0x7f9b580cf7d0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f9b50009e50 tx=0x7f9b5002bed0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.438+0000 7f9b5e600640 1 -- 172.21.15.26:0/1375456645 shutdown_connections 2024-01-15T18:16:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.438+0000 7f9b5e600640 1 --2- 172.21.15.26:0/1375456645 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5815f9c0 0x7f9b580cf7d0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.441 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.438+0000 7f9b5e600640 1 --2- 172.21.15.26:0/1375456645 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5813e360 0x7f9b580cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.438+0000 7f9b5e600640 1 -- 172.21.15.26:0/1375456645 >> 172.21.15.26:0/1375456645 conn(0x7f9b5806a800 msgr2=0x7f9b5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.438+0000 7f9b5e600640 1 -- 172.21.15.26:0/1375456645 shutdown_connections 2024-01-15T18:16:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.438+0000 7f9b5e600640 1 -- 172.21.15.26:0/1375456645 wait complete. 2024-01-15T18:16:47.442 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.440+0000 7f9b5e600640 1 Processor -- start 2024-01-15T18:16:47.443 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.440+0000 7f9b5e600640 1 -- start start 2024-01-15T18:16:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.441+0000 7f9b5e600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5813df80 0x7f9b5814d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.441+0000 7f9b5e600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5813e360 0x7f9b5813d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.441+0000 7f9b5e600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5815f9c0 0x7f9b5815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.441+0000 7f9b5e600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580703e0 con 0x7f9b5815f9c0 2024-01-15T18:16:47.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.441+0000 7f9b5e600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5814dcb0 con 0x7f9b5813df80 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.441+0000 7f9b5e600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b580705f0 con 0x7f9b5813e360 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.441+0000 7f9b57e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5815f9c0 0x7f9b5815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5815f9c0 0x7f9b5815a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55448/0 (socket says 172.21.15.26:55448) 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57e00640 1 -- 172.21.15.26:0/260462252 learned_addr learned my addr 172.21.15.26:0/260462252 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57400640 1 --2- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5813df80 0x7f9b5814d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b4fe00640 1 --2- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5813e360 0x7f9b5813d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57e00640 1 -- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5813e360 msgr2=0x7f9b5813d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57e00640 1 --2- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9b5813e360 0x7f9b5813d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57e00640 1 -- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5813df80 msgr2=0x7f9b5814d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57e00640 1 --2- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5813df80 0x7f9b5814d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57e00640 1 -- 172.21.15.26:0/260462252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b50009b30 con 0x7f9b5815f9c0 2024-01-15T18:16:47.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57400640 1 --2- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9b5813df80 0x7f9b5814d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.442+0000 7f9b57e00640 1 --2- 172.21.15.26:0/260462252 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9b5815f9c0 0x7f9b5815a140 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f9b50036040 tx=0x7f9b50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.443+0000 7f9b4f400640 1 -- 172.21.15.26:0/260462252 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b50009510 con 0x7f9b5815f9c0 2024-01-15T18:16:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.443+0000 7f9b4f400640 1 -- 172.21.15.26:0/260462252 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5003e450 con 0x7f9b5815f9c0 2024-01-15T18:16:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.443+0000 7f9b4f400640 1 -- 172.21.15.26:0/260462252 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5003e7e0 con 0x7f9b5815f9c0 2024-01-15T18:16:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.443+0000 7f9b5e600640 1 -- 172.21.15.26:0/260462252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b580708d0 con 0x7f9b5815f9c0 2024-01-15T18:16:47.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.443+0000 7f9b5e600640 1 -- 172.21.15.26:0/260462252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5814d8e0 con 0x7f9b5815f9c0 2024-01-15T18:16:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.444+0000 7f9b5e600640 1 -- 172.21.15.26:0/260462252 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5814da70 con 0x7f9b5815f9c0 2024-01-15T18:16:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.445+0000 7f9b4f400640 1 -- 172.21.15.26:0/260462252 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7f9b5003e940 con 0x7f9b5815f9c0 2024-01-15T18:16:47.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.446+0000 7f9b4f400640 1 --2- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b3006c670 0x7f9b3006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.446+0000 7f9b4f400640 1 -- 172.21.15.26:0/260462252 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f9b500aba00 con 0x7f9b5815f9c0 2024-01-15T18:16:47.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.447+0000 7f9b57400640 1 --2- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b3006c670 0x7f9b3006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:47.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.448+0000 7f9b57400640 1 --2- 172.21.15.26:0/260462252 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9b3006c670 0x7f9b3006eb30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9b4800a830 tx=0x7f9b48008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:47.453 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.451+0000 7f9b4f400640 1 -- 172.21.15.26:0/260462252 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b5814da70 con 0x7f9b5815f9c0 2024-01-15T18:16:47.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.672+0000 7f9b5e600640 1 -- 172.21.15.26:0/260462252 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b5806cff0 con 0x7f9b3006c670 2024-01-15T18:16:47.693 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:47.691+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:47.691+0000 7f9b4f400640 1 -- 172.21.15.26:0/260462252 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f9b4800adb0 con 0x7f9b3006c670 2024-01-15T18:16:47.694 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:47.713 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:49.715 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.952+0000 7fcb72e00640 1 Processor -- start 2024-01-15T18:16:49.956 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.953+0000 7fcb72e00640 1 -- start start 2024-01-15T18:16:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.954+0000 7fcb72e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb6c0cd240 0x7fcb6c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.954+0000 7fcb72e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb6c06c6b0 0x7fcb6c099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.954+0000 7fcb72e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6c06cea0 con 0x7fcb6c099fa0 2024-01-15T18:16:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.954+0000 7fcb72e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6c09a430 con 0x7fcb6c0cd240 2024-01-15T18:16:49.958 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.954+0000 7fcb72e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c06cbc0 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.955+0000 7fcb6be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb6c0cd240 0x7fcb6c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.955+0000 7fcb6be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcb6c0cd240 0x7fcb6c0b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55458/0 (socket says 172.21.15.26:55458) 2024-01-15T18:16:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.955+0000 7fcb6be00640 1 -- 172.21.15.26:0/3866697891 learned_addr learned my addr 172.21.15.26:0/3866697891 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.955+0000 7fcb6b400640 1 --2- 172.21.15.26:0/3866697891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb6c06c6b0 0x7fcb6c099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.955+0000 7fcb6be00640 1 -- 172.21.15.26:0/3866697891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb6c06c6b0 msgr2=0x7fcb6c099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.955+0000 7fcb6be00640 1 --2- 172.21.15.26:0/3866697891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb6c06c6b0 0x7fcb6c099a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.955+0000 7fcb6be00640 1 -- 172.21.15.26:0/3866697891 >> v1:172.21.15.124:6789/0 conn(0x7fcb6c099fa0 legacy=0x7fcb6c06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:49.959 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.955+0000 7fcb6be00640 1 -- 172.21.15.26:0/3866697891 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6c06f350 con 0x7fcb6c0cd240 2024-01-15T18:16:49.960 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.957+0000 7fcb6be00640 1 --2- 172.21.15.26:0/3866697891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb6c0cd240 0x7fcb6c0b3820 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009e50 tx=0x7fcb5c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efac6ccf038928d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.957+0000 7fcb6aa00640 1 -- 172.21.15.26:0/3866697891 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c02ca70 con 0x7fcb6c0cd240 2024-01-15T18:16:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.957+0000 7fcb6aa00640 1 -- 172.21.15.26:0/3866697891 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c02cbd0 con 0x7fcb6c0cd240 2024-01-15T18:16:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.957+0000 7fcb6aa00640 1 -- 172.21.15.26:0/3866697891 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c0343f0 con 0x7fcb6c0cd240 2024-01-15T18:16:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.958+0000 7fcb72e00640 1 -- 172.21.15.26:0/3866697891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb6c0cd240 msgr2=0x7fcb6c0b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:49.961 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.958+0000 7fcb72e00640 1 --2- 172.21.15.26:0/3866697891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb6c0cd240 0x7fcb6c0b3820 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009e50 tx=0x7fcb5c02bed0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.958+0000 7fcb72e00640 1 -- 172.21.15.26:0/3866697891 shutdown_connections 2024-01-15T18:16:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.958+0000 7fcb72e00640 1 --2- 172.21.15.26:0/3866697891 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb6c06c6b0 0x7fcb6c099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.958+0000 7fcb72e00640 1 --2- 172.21.15.26:0/3866697891 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb6c0cd240 0x7fcb6c0b3820 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.958+0000 7fcb72e00640 1 -- 172.21.15.26:0/3866697891 >> 172.21.15.26:0/3866697891 conn(0x7fcb6c0b2d00 msgr2=0x7fcb6c0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.958+0000 7fcb72e00640 1 -- 172.21.15.26:0/3866697891 shutdown_connections 2024-01-15T18:16:49.962 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.958+0000 7fcb72e00640 1 -- 172.21.15.26:0/3866697891 wait complete. 2024-01-15T18:16:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.960+0000 7fcb72e00640 1 Processor -- start 2024-01-15T18:16:49.963 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.960+0000 7fcb72e00640 1 -- start start 2024-01-15T18:16:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.961+0000 7fcb72e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb6c06c6b0 0x7fcb6c1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.961+0000 7fcb72e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb6c099fa0 0x7fcb6c0c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.961+0000 7fcb72e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb6c0cd240 0x7fcb6c0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.965 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.961+0000 7fcb72e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c1f4470 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.961+0000 7fcb72e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c157ce0 con 0x7fcb6c099fa0 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.961+0000 7fcb72e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c0c6390 con 0x7fcb6c0cd240 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.961+0000 7fcb6be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb6c06c6b0 0x7fcb6c1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.961+0000 7fcb6be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb6c06c6b0 0x7fcb6c1577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55464/0 (socket says 172.21.15.26:55464) 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.961+0000 7fcb6be00640 1 -- 172.21.15.26:0/2641397477 learned_addr learned my addr 172.21.15.26:0/2641397477 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb6be00640 1 -- 172.21.15.26:0/2641397477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb6c0cd240 msgr2=0x7fcb6c0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb6be00640 1 --2- 172.21.15.26:0/2641397477 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcb6c0cd240 0x7fcb6c0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb6be00640 1 -- 172.21.15.26:0/2641397477 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb6c099fa0 msgr2=0x7fcb6c0c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb6be00640 1 --2- 172.21.15.26:0/2641397477 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcb6c099fa0 0x7fcb6c0c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb6be00640 1 -- 172.21.15.26:0/2641397477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c009b30 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.966 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb6be00640 1 --2- 172.21.15.26:0/2641397477 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcb6c06c6b0 0x7fcb6c1577a0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c000f80 tx=0x7fcb5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb63e00640 1 -- 172.21.15.26:0/2641397477 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c009510 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb63e00640 1 -- 172.21.15.26:0/2641397477 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c03e450 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb63e00640 1 -- 172.21.15.26:0/2641397477 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c03e800 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.962+0000 7fcb72e00640 1 -- 172.21.15.26:0/2641397477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c0c6670 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.963+0000 7fcb72e00640 1 -- 172.21.15.26:0/2641397477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c0b4680 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.964+0000 7fcb72e00640 1 -- 172.21.15.26:0/2641397477 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c0675d0 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.968 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.965+0000 7fcb63e00640 1 -- 172.21.15.26:0/2641397477 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 43) v1 ==== 91324+0+0 (secure 0 0 0) 0x7fcb5c03e960 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.966+0000 7fcb63e00640 1 --2- 172.21.15.26:0/2641397477 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb4406c6a0 0x7fcb4406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.966+0000 7fcb63e00640 1 -- 172.21.15.26:0/2641397477 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fcb5c0aba00 con 0x7fcb6c06c6b0 2024-01-15T18:16:49.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.966+0000 7fcb6b400640 1 --2- 172.21.15.26:0/2641397477 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb4406c6a0 0x7fcb4406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:49.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.968+0000 7fcb6b400640 1 --2- 172.21.15.26:0/2641397477 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcb4406c6a0 0x7fcb4406eb60 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c09aa30 tx=0x7fcb54008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:49.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:49.970+0000 7fcb63e00640 1 -- 172.21.15.26:0/2641397477 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb5c077f10 con 0x7fcb6c06c6b0 2024-01-15T18:16:50.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:50.197+0000 7fcb72e00640 1 -- 172.21.15.26:0/2641397477 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb6c0cd620 con 0x7fcb4406c6a0 2024-01-15T18:16:50.218 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:50.216+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:50.219 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:50.215+0000 7fcb63e00640 1 -- 172.21.15.26:0/2641397477 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fcb54009480 con 0x7fcb4406c6a0 2024-01-15T18:16:50.219 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:50.232 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:52.233 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.468+0000 7fbf2a200640 1 Processor -- start 2024-01-15T18:16:52.472 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.468+0000 7fbf2a200640 1 -- start start 2024-01-15T18:16:52.474 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.470+0000 7fbf2a200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf240b3ae0 0x7fbf2415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.470+0000 7fbf2a200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 0x7fbf240cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.470+0000 7fbf2a200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf24070c40 con 0x7fbf240ae5a0 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.470+0000 7fbf2a200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf24071450 con 0x7fbf240b3ae0 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.470+0000 7fbf2a200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240cfc60 con 0x7fbf240cf290 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.471+0000 7fbf22a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf240b3ae0 0x7fbf2415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.471+0000 7fbf22a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbf240b3ae0 0x7fbf2415e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38022/0 (socket says 172.21.15.26:38022) 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.471+0000 7fbf22a00640 1 -- 172.21.15.26:0/2829883815 learned_addr learned my addr 172.21.15.26:0/2829883815 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.471+0000 7fbf23e00640 1 --2- 172.21.15.26:0/2829883815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 0x7fbf240cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.471+0000 7fbf22a00640 1 -- 172.21.15.26:0/2829883815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 msgr2=0x7fbf240cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.471+0000 7fbf22a00640 1 --2- 172.21.15.26:0/2829883815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 0x7fbf240cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.471+0000 7fbf22a00640 1 -- 172.21.15.26:0/2829883815 >> v1:172.21.15.124:6789/0 conn(0x7fbf240ae5a0 legacy=0x7fbf2406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:52.475 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.471+0000 7fbf22a00640 1 -- 172.21.15.26:0/2829883815 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2406c470 con 0x7fbf240b3ae0 2024-01-15T18:16:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.472+0000 7fbf22a00640 1 --2- 172.21.15.26:0/2829883815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf240b3ae0 0x7fbf2415e430 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009870 tx=0x7fbf0c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c653a52821b4917c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf22000640 1 -- 172.21.15.26:0/2829883815 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c030840 con 0x7fbf240b3ae0 2024-01-15T18:16:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf22000640 1 -- 172.21.15.26:0/2829883815 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c0309a0 con 0x7fbf240b3ae0 2024-01-15T18:16:52.476 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf22000640 1 -- 172.21.15.26:0/2829883815 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c030d30 con 0x7fbf240b3ae0 2024-01-15T18:16:52.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf2a200640 1 -- 172.21.15.26:0/2829883815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf240b3ae0 msgr2=0x7fbf2415e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:52.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf2a200640 1 --2- 172.21.15.26:0/2829883815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf240b3ae0 0x7fbf2415e430 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009870 tx=0x7fbf0c02b980 comp rx=0 tx=0).stop 2024-01-15T18:16:52.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf2a200640 1 -- 172.21.15.26:0/2829883815 shutdown_connections 2024-01-15T18:16:52.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf2a200640 1 --2- 172.21.15.26:0/2829883815 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 0x7fbf240cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf2a200640 1 --2- 172.21.15.26:0/2829883815 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf240b3ae0 0x7fbf2415e430 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.477 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf2a200640 1 -- 172.21.15.26:0/2829883815 >> 172.21.15.26:0/2829883815 conn(0x7fbf2406a800 msgr2=0x7fbf2406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf2a200640 1 -- 172.21.15.26:0/2829883815 shutdown_connections 2024-01-15T18:16:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.473+0000 7fbf2a200640 1 -- 172.21.15.26:0/2829883815 wait complete. 2024-01-15T18:16:52.478 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.475+0000 7fbf2a200640 1 Processor -- start 2024-01-15T18:16:52.479 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.475+0000 7fbf2a200640 1 -- start start 2024-01-15T18:16:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.476+0000 7fbf2a200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf240ae5a0 0x7fbf240cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.476+0000 7fbf2a200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf240b3ae0 0x7fbf2414f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.476+0000 7fbf2a200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 0x7fbf24157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.476+0000 7fbf2a200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241584a0 con 0x7fbf240b3ae0 2024-01-15T18:16:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.476+0000 7fbf2a200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24157a40 con 0x7fbf240ae5a0 2024-01-15T18:16:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.476+0000 7fbf2a200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24156e30 con 0x7fbf240cf290 2024-01-15T18:16:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf22a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf240b3ae0 0x7fbf2414f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf22a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf240b3ae0 0x7fbf2414f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38028/0 (socket says 172.21.15.26:38028) 2024-01-15T18:16:52.481 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf22a00640 1 -- 172.21.15.26:0/3308775896 learned_addr learned my addr 172.21.15.26:0/3308775896 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf23400640 1 --2- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf240ae5a0 0x7fbf240cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf23e00640 1 --2- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 0x7fbf24157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf22a00640 1 -- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 msgr2=0x7fbf24157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf22a00640 1 --2- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 0x7fbf24157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf22a00640 1 -- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf240ae5a0 msgr2=0x7fbf240cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf22a00640 1 --2- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbf240ae5a0 0x7fbf240cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf22a00640 1 -- 172.21.15.26:0/3308775896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c0094d0 con 0x7fbf240b3ae0 2024-01-15T18:16:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.477+0000 7fbf23e00640 1 --2- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbf240cf290 0x7fbf24157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:52.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.478+0000 7fbf22a00640 1 --2- 172.21.15.26:0/3308775896 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbf240b3ae0 0x7fbf2414f980 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009870 tx=0x7fbf0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.478+0000 7fbf1b400640 1 -- 172.21.15.26:0/3308775896 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0307f0 con 0x7fbf240b3ae0 2024-01-15T18:16:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.478+0000 7fbf1b400640 1 -- 172.21.15.26:0/3308775896 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c043400 con 0x7fbf240b3ae0 2024-01-15T18:16:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.478+0000 7fbf2a200640 1 -- 172.21.15.26:0/3308775896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf24157110 con 0x7fbf240b3ae0 2024-01-15T18:16:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.478+0000 7fbf1b400640 1 -- 172.21.15.26:0/3308775896 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0437b0 con 0x7fbf240b3ae0 2024-01-15T18:16:52.483 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.478+0000 7fbf2a200640 1 -- 172.21.15.26:0/3308775896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf241576d0 con 0x7fbf240b3ae0 2024-01-15T18:16:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.479+0000 7fbf2a200640 1 -- 172.21.15.26:0/3308775896 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf24157bf0 con 0x7fbf240b3ae0 2024-01-15T18:16:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.480+0000 7fbf1b400640 1 -- 172.21.15.26:0/3308775896 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 44) v1 ==== 91273+0+0 (secure 0 0 0) 0x7fbf0c043910 con 0x7fbf240b3ae0 2024-01-15T18:16:52.484 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.481+0000 7fbf1b400640 1 --2- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbefc06c5e0 0x7fbefc06eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:52.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.481+0000 7fbf1b400640 1 -- 172.21.15.26:0/3308775896 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fbf0c03e070 con 0x7fbf240b3ae0 2024-01-15T18:16:52.485 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.482+0000 7fbf23400640 1 --2- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbefc06c5e0 0x7fbefc06eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:52.486 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.483+0000 7fbf23400640 1 --2- 172.21.15.26:0/3308775896 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbefc06c5e0 0x7fbefc06eaa0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbf240cb7f0 tx=0x7fbf14009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:52.489 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.486+0000 7fbf1b400640 1 -- 172.21.15.26:0/3308775896 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf24157bf0 con 0x7fbf240b3ae0 2024-01-15T18:16:52.720 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.717+0000 7fbf2a200640 1 -- 172.21.15.26:0/3308775896 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf24071170 con 0x7fbefc06c5e0 2024-01-15T18:16:52.741 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:52.738+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:52.738+0000 7fbf1b400640 1 -- 172.21.15.26:0/3308775896 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fbf1400e900 con 0x7fbefc06c5e0 2024-01-15T18:16:52.741 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:52.759 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:54.760 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.992+0000 7f8d15400640 1 Processor -- start 2024-01-15T18:16:54.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.992+0000 7f8d15400640 1 -- start start 2024-01-15T18:16:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.994+0000 7f8d15400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d100cc500 0x7f8d100cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.994+0000 7f8d15400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d1006d160 0x7f8d1006d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.994+0000 7f8d15400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d10070e50 con 0x7f8d10143c40 2024-01-15T18:16:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.994+0000 7f8d15400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d100723c0 con 0x7f8d1006d160 2024-01-15T18:16:54.998 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.994+0000 7f8d15400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d10144020 con 0x7f8d100cc500 2024-01-15T18:16:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.995+0000 7f8d14a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d1006d160 0x7f8d1006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.995+0000 7f8d14a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f8d1006d160 0x7f8d1006d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38032/0 (socket says 172.21.15.26:38032) 2024-01-15T18:16:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.995+0000 7f8d14a00640 1 -- 172.21.15.26:0/785103873 learned_addr learned my addr 172.21.15.26:0/785103873 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.995+0000 7f8d14a00640 1 -- 172.21.15.26:0/785103873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d100cc500 msgr2=0x7f8d100cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.995+0000 7f8d07200640 1 --2- 172.21.15.26:0/785103873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d100cc500 0x7f8d100cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.995+0000 7f8d14a00640 1 --2- 172.21.15.26:0/785103873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d100cc500 0x7f8d100cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.995+0000 7f8d14a00640 1 -- 172.21.15.26:0/785103873 >> v1:172.21.15.124:6789/0 conn(0x7f8d10143c40 legacy=0x7f8d10071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.995+0000 7f8d14a00640 1 -- 172.21.15.26:0/785103873 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d100ccea0 con 0x7f8d1006d160 2024-01-15T18:16:54.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.995+0000 7f8d07200640 1 --2- 172.21.15.26:0/785103873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d100cc500 0x7f8d100cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:55.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.996+0000 7f8d14a00640 1 --2- 172.21.15.26:0/785103873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d1006d160 0x7f8d1006d590 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009e50 tx=0x7f8d0802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564d515082f711e4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:55.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.997+0000 7f8d0f400640 1 -- 172.21.15.26:0/785103873 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0802c9e0 con 0x7f8d1006d160 2024-01-15T18:16:55.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.997+0000 7f8d0f400640 1 -- 172.21.15.26:0/785103873 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0802cb40 con 0x7f8d1006d160 2024-01-15T18:16:55.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.997+0000 7f8d0f400640 1 -- 172.21.15.26:0/785103873 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d080343f0 con 0x7f8d1006d160 2024-01-15T18:16:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.997+0000 7f8d15400640 1 -- 172.21.15.26:0/785103873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d1006d160 msgr2=0x7f8d1006d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.997+0000 7f8d15400640 1 --2- 172.21.15.26:0/785103873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d1006d160 0x7f8d1006d590 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009e50 tx=0x7f8d0802be40 comp rx=0 tx=0).stop 2024-01-15T18:16:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.997+0000 7f8d15400640 1 -- 172.21.15.26:0/785103873 shutdown_connections 2024-01-15T18:16:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.997+0000 7f8d15400640 1 --2- 172.21.15.26:0/785103873 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d1006d160 0x7f8d1006d590 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.997+0000 7f8d15400640 1 --2- 172.21.15.26:0/785103873 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d100cc500 0x7f8d100cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.997+0000 7f8d15400640 1 -- 172.21.15.26:0/785103873 >> 172.21.15.26:0/785103873 conn(0x7f8d1006a800 msgr2=0x7f8d1006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:55.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.998+0000 7f8d15400640 1 -- 172.21.15.26:0/785103873 shutdown_connections 2024-01-15T18:16:55.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.998+0000 7f8d15400640 1 -- 172.21.15.26:0/785103873 wait complete. 2024-01-15T18:16:55.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.999+0000 7f8d15400640 1 Processor -- start 2024-01-15T18:16:55.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:54.999+0000 7f8d15400640 1 -- start start 2024-01-15T18:16:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.000+0000 7f8d15400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d1006d160 0x7f8d101f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.000+0000 7f8d15400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d100cc500 0x7f8d10142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.000+0000 7f8d15400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d10143c40 0x7f8d100ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.000+0000 7f8d15400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1013b740 con 0x7f8d100cc500 2024-01-15T18:16:55.004 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.000+0000 7f8d15400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d100c3e10 con 0x7f8d1006d160 2024-01-15T18:16:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.000+0000 7f8d15400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d100c40f0 con 0x7f8d10143c40 2024-01-15T18:16:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d07200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d100cc500 0x7f8d10142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d07200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d100cc500 0x7f8d10142e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38038/0 (socket says 172.21.15.26:38038) 2024-01-15T18:16:55.005 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d07200640 1 -- 172.21.15.26:0/2441059352 learned_addr learned my addr 172.21.15.26:0/2441059352 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d14a00640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d10143c40 0x7f8d100ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d0fe00640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d1006d160 0x7f8d101f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d07200640 1 -- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d10143c40 msgr2=0x7f8d100ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d07200640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d10143c40 0x7f8d100ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d07200640 1 -- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d1006d160 msgr2=0x7f8d101f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d07200640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d1006d160 0x7f8d101f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d07200640 1 -- 172.21.15.26:0/2441059352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d08009b30 con 0x7f8d100cc500 2024-01-15T18:16:55.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d14a00640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f8d10143c40 0x7f8d100ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.001+0000 7f8d0fe00640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f8d1006d160 0x7f8d101f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.002+0000 7f8d07200640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8d100cc500 0x7f8d10142e40 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c7f0 tx=0x7f8cf800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.002+0000 7f8d0cc00640 1 -- 172.21.15.26:0/2441059352 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8007a30 con 0x7f8d100cc500 2024-01-15T18:16:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.002+0000 7f8d0cc00640 1 -- 172.21.15.26:0/2441059352 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf8007b90 con 0x7f8d100cc500 2024-01-15T18:16:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.002+0000 7f8d0cc00640 1 -- 172.21.15.26:0/2441059352 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf800ce40 con 0x7f8d100cc500 2024-01-15T18:16:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.002+0000 7f8d15400640 1 -- 172.21.15.26:0/2441059352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d100c43d0 con 0x7f8d100cc500 2024-01-15T18:16:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.002+0000 7f8d15400640 1 -- 172.21.15.26:0/2441059352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d10143380 con 0x7f8d100cc500 2024-01-15T18:16:55.007 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.003+0000 7f8d15400640 1 -- 172.21.15.26:0/2441059352 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d100c45c0 con 0x7f8d100cc500 2024-01-15T18:16:55.008 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.004+0000 7f8d0cc00640 1 -- 172.21.15.26:0/2441059352 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f8cf80113f0 con 0x7f8d100cc500 2024-01-15T18:16:55.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.006+0000 7f8d0cc00640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8ce806c550 0x7f8ce806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.006+0000 7f8d0cc00640 1 -- 172.21.15.26:0/2441059352 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f8cf80889b0 con 0x7f8d100cc500 2024-01-15T18:16:55.011 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.006+0000 7f8d0fe00640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8ce806c550 0x7f8ce806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:55.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.007+0000 7f8d0fe00640 1 --2- 172.21.15.26:0/2441059352 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f8ce806c550 0x7f8ce806ea10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8d100b08a0 tx=0x7f8d00009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:55.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.010+0000 7f8d0cc00640 1 -- 172.21.15.26:0/2441059352 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d100c45c0 con 0x7f8d100cc500 2024-01-15T18:16:55.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.230+0000 7f8d15400640 1 -- 172.21.15.26:0/2441059352 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d1006c960 con 0x7f8ce806c550 2024-01-15T18:16:55.253 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:55.250+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:55.253 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:55.250+0000 7f8d0cc00640 1 -- 172.21.15.26:0/2441059352 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f8d00003a80 con 0x7f8ce806c550 2024-01-15T18:16:55.253 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:55.272 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:57.274 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:16:57.504 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.500+0000 7f78fc800640 1 Processor -- start 2024-01-15T18:16:57.505 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.501+0000 7f78fc800640 1 -- start start 2024-01-15T18:16:57.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.502+0000 7f78fc800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f78f40cf9a0 0x7f78f4070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:57.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.502+0000 7f78fc800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f4071690 con 0x7f78f4070ea0 2024-01-15T18:16:57.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.502+0000 7f78fc800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f4071970 con 0x7f78f414b2c0 2024-01-15T18:16:57.508 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.502+0000 7f78fc800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f414b6a0 con 0x7f78f40cf9a0 2024-01-15T18:16:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.502+0000 7f78fa800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f78f4070ea0 0x7f78f40cc160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:45218/0 (socket says 172.21.15.26:45218) 2024-01-15T18:16:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78fa800640 1 -- 172.21.15.26:0/4136999010 learned_addr learned my addr 172.21.15.26:0/4136999010 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:16:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f9400640 1 --2- 172.21.15.26:0/4136999010 >> v2:172.21.15.124:3300/0 conn(0x7f78f40cf9a0 0x7f78f4070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616141197 0 0) 0x7f78f4071690 con 0x7f78f4070ea0 2024-01-15T18:16:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78d4003520 con 0x7f78f4070ea0 2024-01-15T18:16:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3286310188 0 0) 0x7f78d4003520 con 0x7f78f4070ea0 2024-01-15T18:16:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 >> v2:172.21.15.124:3300/0 conn(0x7f78f40cf9a0 msgr2=0x7f78f4070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f8a00640 1 --2- 172.21.15.26:0/4136999010 >> v2:172.21.15.124:3300/0 conn(0x7f78f40cf9a0 0x7f78f4070960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 >> v1:172.21.15.124:6789/0 conn(0x7f78f414b2c0 legacy=0x7f78f40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4071c50 con 0x7f78f4070ea0 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f78ec0019f0 con 0x7f78f4070ea0 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.503+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 >> v1:172.21.15.26:6789/0 conn(0x7f78f4070ea0 legacy=0x7f78f40cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78f8a00640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d40039c0 0x7f78d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78f8a00640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d40047c0 0x7f78d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78f8a00640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 0x7f78d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f414b6a0 con 0x7f78d4005180 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d400c080 con 0x7f78d40047c0 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d400c240 con 0x7f78d40039c0 2024-01-15T18:16:57.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78fa800640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 0x7f78d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78f9e00640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d40039c0 0x7f78d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78fa800640 1 -- 172.21.15.26:0/4136999010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d40039c0 msgr2=0x7f78d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78f9400640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d40047c0 0x7f78d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78fa800640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78d40039c0 0x7f78d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78fa800640 1 -- 172.21.15.26:0/4136999010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d40047c0 msgr2=0x7f78d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:16:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78fa800640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78d40047c0 0x7f78d4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.504+0000 7f78fa800640 1 -- 172.21.15.26:0/4136999010 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d4003520 con 0x7f78d4005180 2024-01-15T18:16:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.505+0000 7f78fa800640 1 -- 172.21.15.26:0/4136999010 reap_dead start 2024-01-15T18:16:57.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.505+0000 7f78fa800640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 0x7f78d400b980 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00a760 tx=0x7f78d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272b10ba5c02af3d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.506+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec034630 con 0x7f78d4005180 2024-01-15T18:16:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.506+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec02f070 con 0x7f78d4005180 2024-01-15T18:16:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.506+0000 7f78f8a00640 1 -- 172.21.15.26:0/4136999010 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec034a10 con 0x7f78d4005180 2024-01-15T18:16:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.506+0000 7f78fc800640 1 -- 172.21.15.26:0/4136999010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 msgr2=0x7f78d400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.506+0000 7f78fc800640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 0x7f78d400b980 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00a760 tx=0x7f78d40057f0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.507+0000 7f78fc800640 1 -- 172.21.15.26:0/4136999010 shutdown_connections 2024-01-15T18:16:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.507+0000 7f78fc800640 1 --2- 172.21.15.26:0/4136999010 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 0x7f78d400b980 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.507+0000 7f78fc800640 1 -- 172.21.15.26:0/4136999010 >> 172.21.15.26:0/4136999010 conn(0x7f78f406a800 msgr2=0x7f78f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:16:57.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.507+0000 7f78fc800640 1 -- 172.21.15.26:0/4136999010 shutdown_connections 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.507+0000 7f78fc800640 1 -- 172.21.15.26:0/4136999010 wait complete. 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.509+0000 7f78fc800640 1 Processor -- start 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.509+0000 7f78fc800640 1 -- start start 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78fc800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 0x7f78f4144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78fc800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78f415ee40 0x7f78f415f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78fc800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78f4163ff0 0x7f78f4164470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78fc800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f41fb2a0 con 0x7f78d4005180 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78fc800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f41fa6b0 con 0x7f78f4163ff0 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78fc800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f41fa900 con 0x7f78f415ee40 2024-01-15T18:16:57.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 0x7f78f4144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 0x7f78f4144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38068/0 (socket says 172.21.15.26:38068) 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9e00640 1 -- 172.21.15.26:0/3874746893 learned_addr learned my addr 172.21.15.26:0/3874746893 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9400640 1 --2- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78f415ee40 0x7f78f415f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78fa800640 1 --2- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78f4163ff0 0x7f78f4164470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9e00640 1 -- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78f415ee40 msgr2=0x7f78f415f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9e00640 1 --2- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78f415ee40 0x7f78f415f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9e00640 1 -- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78f4163ff0 msgr2=0x7f78f4164470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9e00640 1 --2- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f78f4163ff0 0x7f78f4164470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9e00640 1 -- 172.21.15.26:0/3874746893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f41fabe0 con 0x7f78d4005180 2024-01-15T18:16:57.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.510+0000 7f78f9400640 1 --2- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f78f415ee40 0x7f78f415f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:16:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.511+0000 7f78f9e00640 1 --2- 172.21.15.26:0/3874746893 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f78d4005180 0x7f78f4144830 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f78e400e410 tx=0x7f78e400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.511+0000 7f78f2000640 1 -- 172.21.15.26:0/3874746893 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e40052e0 con 0x7f78d4005180 2024-01-15T18:16:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.511+0000 7f78f2000640 1 -- 172.21.15.26:0/3874746893 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e4005440 con 0x7f78d4005180 2024-01-15T18:16:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.511+0000 7f78f2000640 1 -- 172.21.15.26:0/3874746893 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4003810 con 0x7f78d4005180 2024-01-15T18:16:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.511+0000 7f78fc800640 1 -- 172.21.15.26:0/3874746893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f41faec0 con 0x7f78d4005180 2024-01-15T18:16:57.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.511+0000 7f78fc800640 1 -- 172.21.15.26:0/3874746893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f41649b0 con 0x7f78d4005180 2024-01-15T18:16:57.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.512+0000 7f78fc800640 1 -- 172.21.15.26:0/3874746893 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f40675d0 con 0x7f78d4005180 2024-01-15T18:16:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.513+0000 7f78f2000640 1 -- 172.21.15.26:0/3874746893 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f78e4003970 con 0x7f78d4005180 2024-01-15T18:16:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.515+0000 7f78f2000640 1 --2- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78cc06c550 0x7f78cc06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:16:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.515+0000 7f78f2000640 1 -- 172.21.15.26:0/3874746893 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f78e40882e0 con 0x7f78d4005180 2024-01-15T18:16:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.515+0000 7f78f9400640 1 --2- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78cc06c550 0x7f78cc06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:16:57.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.517+0000 7f78f9400640 1 --2- 172.21.15.26:0/3874746893 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f78cc06c550 0x7f78cc06ea10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f78f4148480 tx=0x7f78dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:16:57.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.519+0000 7f78f2000640 1 -- 172.21.15.26:0/3874746893 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78e40548d0 con 0x7f78d4005180 2024-01-15T18:16:57.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.740+0000 7f78fc800640 1 -- 172.21.15.26:0/3874746893 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f78f40cbc40 con 0x7f78cc06c550 2024-01-15T18:16:57.767 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:16:57.764+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:16:57.763+0000 7f78f2000640 1 -- 172.21.15.26:0/3874746893 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f78dc0118e0 con 0x7f78cc06c550 2024-01-15T18:16:57.767 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:16:57.778 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:16:59.780 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:00.009 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.006+0000 7fd003e00640 1 Processor -- start 2024-01-15T18:17:00.010 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.006+0000 7fd003e00640 1 -- start start 2024-01-15T18:17:00.012 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.008+0000 7fd003e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd004159590 0x7fd004070160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.008+0000 7fd003e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd0040b3c20 0x7fd0040ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.008+0000 7fd003e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd0040cd220 0x7fd00406d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.008+0000 7fd003e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd004071790 con 0x7fd0040b3c20 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.008+0000 7fd003e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd004071100 con 0x7fd0040cd220 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.008+0000 7fd003e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd004070860 con 0x7fd004159590 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.008+0000 7fd002000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd0040b3c20 0x7fd0040ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.008+0000 7fd002000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd0040b3c20 0x7fd0040ccce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38076/0 (socket says 172.21.15.26:38076) 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.008+0000 7fd002000640 1 -- 172.21.15.26:0/625421882 learned_addr learned my addr 172.21.15.26:0/625421882 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.009+0000 7fd002a00640 1 --2- 172.21.15.26:0/625421882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd004159590 0x7fd004070160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.009+0000 7fd003400640 1 --2- 172.21.15.26:0/625421882 >> v2:172.21.15.124:3300/0 conn(0x7fd0040cd220 0x7fd00406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.009+0000 7fd002000640 1 -- 172.21.15.26:0/625421882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd004159590 msgr2=0x7fd004070160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:00.013 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.009+0000 7fd002000640 1 --2- 172.21.15.26:0/625421882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd004159590 0x7fd004070160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.009+0000 7fd002000640 1 -- 172.21.15.26:0/625421882 >> v2:172.21.15.124:3300/0 conn(0x7fd0040cd220 msgr2=0x7fd00406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.009+0000 7fd002000640 1 --2- 172.21.15.26:0/625421882 >> v2:172.21.15.124:3300/0 conn(0x7fd0040cd220 0x7fd00406d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.009+0000 7fd002000640 1 -- 172.21.15.26:0/625421882 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004070b40 con 0x7fd0040b3c20 2024-01-15T18:17:00.014 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.009+0000 7fd003400640 1 --2- 172.21.15.26:0/625421882 >> v2:172.21.15.124:3300/0 conn(0x7fd0040cd220 0x7fd00406d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.010+0000 7fd002000640 1 --2- 172.21.15.26:0/625421882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0040b3c20 0x7fd0040ccce0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fd0040ce820 tx=0x7fcfec02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ff28e66bb38743 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.010+0000 7fd001600640 1 -- 172.21.15.26:0/625421882 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec0317d0 con 0x7fd0040b3c20 2024-01-15T18:17:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.010+0000 7fd001600640 1 -- 172.21.15.26:0/625421882 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec031930 con 0x7fd0040b3c20 2024-01-15T18:17:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.010+0000 7fd001600640 1 -- 172.21.15.26:0/625421882 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec031cc0 con 0x7fd0040b3c20 2024-01-15T18:17:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.011+0000 7fd003e00640 1 -- 172.21.15.26:0/625421882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0040b3c20 msgr2=0x7fd0040ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:00.015 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.011+0000 7fd003e00640 1 --2- 172.21.15.26:0/625421882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0040b3c20 0x7fd0040ccce0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fd0040ce820 tx=0x7fcfec02b870 comp rx=0 tx=0).stop 2024-01-15T18:17:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.011+0000 7fd003e00640 1 -- 172.21.15.26:0/625421882 shutdown_connections 2024-01-15T18:17:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.011+0000 7fd003e00640 1 --2- 172.21.15.26:0/625421882 >> v2:172.21.15.124:3300/0 conn(0x7fd0040cd220 0x7fd00406d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.011+0000 7fd003e00640 1 --2- 172.21.15.26:0/625421882 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0040b3c20 0x7fd0040ccce0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.011+0000 7fd003e00640 1 --2- 172.21.15.26:0/625421882 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd004159590 0x7fd004070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.011+0000 7fd003e00640 1 -- 172.21.15.26:0/625421882 >> 172.21.15.26:0/625421882 conn(0x7fd00406a800 msgr2=0x7fd00406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.011+0000 7fd003e00640 1 -- 172.21.15.26:0/625421882 shutdown_connections 2024-01-15T18:17:00.016 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.011+0000 7fd003e00640 1 -- 172.21.15.26:0/625421882 wait complete. 2024-01-15T18:17:00.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.013+0000 7fd003e00640 1 Processor -- start 2024-01-15T18:17:00.017 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.013+0000 7fd003e00640 1 -- start start 2024-01-15T18:17:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.014+0000 7fd003e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0040b3c20 0x7fd00414eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.014+0000 7fd003e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0040cd220 0x7fd00406c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.014+0000 7fd003e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd004159590 0x7fd00406ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.014+0000 7fd003e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041f6350 con 0x7fd0040cd220 2024-01-15T18:17:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.014+0000 7fd003e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004154330 con 0x7fd004159590 2024-01-15T18:17:00.018 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.014+0000 7fd003e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041f65a0 con 0x7fd0040b3c20 2024-01-15T18:17:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd002000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0040cd220 0x7fd00406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd002000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0040cd220 0x7fd00406c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:38084/0 (socket says 172.21.15.26:38084) 2024-01-15T18:17:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd002000640 1 -- 172.21.15.26:0/4033849278 learned_addr learned my addr 172.21.15.26:0/4033849278 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd002a00640 1 --2- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0040b3c20 0x7fd00414eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:00.019 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd003400640 1 --2- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd004159590 0x7fd00406ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd002000640 1 -- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0040b3c20 msgr2=0x7fd00414eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd002000640 1 --2- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd0040b3c20 0x7fd00414eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd002000640 1 -- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd004159590 msgr2=0x7fd00406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:00.020 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd002000640 1 --2- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd004159590 0x7fd00406ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd002000640 1 -- 172.21.15.26:0/4033849278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0094d0 con 0x7fd0040cd220 2024-01-15T18:17:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.015+0000 7fd003400640 1 --2- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd004159590 0x7fd00406ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.016+0000 7fd002000640 1 --2- 172.21.15.26:0/4033849278 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd0040cd220 0x7fd00406c790 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fcfec000c00 tx=0x7fcfec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.016+0000 7fcffaa00640 1 -- 172.21.15.26:0/4033849278 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec032870 con 0x7fd0040cd220 2024-01-15T18:17:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.016+0000 7fcffaa00640 1 -- 172.21.15.26:0/4033849278 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfec032a10 con 0x7fd0040cd220 2024-01-15T18:17:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.016+0000 7fcffaa00640 1 -- 172.21.15.26:0/4033849278 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec032dc0 con 0x7fd0040cd220 2024-01-15T18:17:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.016+0000 7fd003e00640 1 -- 172.21.15.26:0/4033849278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0041f6880 con 0x7fd0040cd220 2024-01-15T18:17:00.021 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.016+0000 7fd003e00640 1 -- 172.21.15.26:0/4033849278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00414f090 con 0x7fd0040cd220 2024-01-15T18:17:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.017+0000 7fd003e00640 1 -- 172.21.15.26:0/4033849278 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0041f6da0 con 0x7fd0040cd220 2024-01-15T18:17:00.022 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.018+0000 7fcffaa00640 1 -- 172.21.15.26:0/4033849278 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fcfec030070 con 0x7fd0040cd220 2024-01-15T18:17:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.020+0000 7fcffaa00640 1 --2- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcfdc06c550 0x7fcfdc06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:00.023 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.020+0000 7fcffaa00640 1 -- 172.21.15.26:0/4033849278 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fcfec0aa600 con 0x7fd0040cd220 2024-01-15T18:17:00.024 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.020+0000 7fd002a00640 1 --2- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcfdc06c550 0x7fcfdc06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:00.025 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.021+0000 7fd002a00640 1 --2- 172.21.15.26:0/4033849278 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcfdc06c550 0x7fcfdc06ea10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcff4007920 tx=0x7fcff4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:00.028 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.024+0000 7fcffaa00640 1 -- 172.21.15.26:0/4033849278 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0041f6da0 con 0x7fd0040cd220 2024-01-15T18:17:00.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.251+0000 7fd003e00640 1 -- 172.21.15.26:0/4033849278 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fd004159970 con 0x7fcfdc06c550 2024-01-15T18:17:00.274 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:00.271+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:00.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:00.271+0000 7fcffaa00640 1 -- 172.21.15.26:0/4033849278 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fcff4007570 con 0x7fcfdc06c550 2024-01-15T18:17:00.274 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:00.285 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:02.286 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:02.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.523+0000 7f444c600640 1 Processor -- start 2024-01-15T18:17:02.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.523+0000 7f444c600640 1 -- start start 2024-01-15T18:17:02.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444c600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4444143ae0 0x7f44440cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:02.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444c600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f44440cf7d0 0x7f444415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:02.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444c600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f444415e1e0 0x7f444415a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:02.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444c600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44440cfc30 con 0x7f444415e1e0 2024-01-15T18:17:02.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444c600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44440b1ee0 con 0x7f44440cf7d0 2024-01-15T18:17:02.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444c600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44440b2780 con 0x7f4444143ae0 2024-01-15T18:17:02.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444a600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f444415e1e0 0x7f444415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444a600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f444415e1e0 0x7f444415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55158/0 (socket says 172.21.15.26:55158) 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444a600640 1 -- 172.21.15.26:0/173340454 learned_addr learned my addr 172.21.15.26:0/173340454 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f4449200640 1 --2- 172.21.15.26:0/173340454 >> v2:172.21.15.124:3300/0 conn(0x7f44440cf7d0 0x7f444415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f4449c00640 1 --2- 172.21.15.26:0/173340454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4444143ae0 0x7f44440cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444a600640 1 -- 172.21.15.26:0/173340454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4444143ae0 msgr2=0x7f44440cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444a600640 1 --2- 172.21.15.26:0/173340454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4444143ae0 0x7f44440cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444a600640 1 -- 172.21.15.26:0/173340454 >> v2:172.21.15.124:3300/0 conn(0x7f44440cf7d0 msgr2=0x7f444415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.525+0000 7f444a600640 1 --2- 172.21.15.26:0/173340454 >> v2:172.21.15.124:3300/0 conn(0x7f44440cf7d0 0x7f444415dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.526+0000 7f444a600640 1 -- 172.21.15.26:0/173340454 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444415e990 con 0x7f444415e1e0 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.526+0000 7f4449c00640 1 --2- 172.21.15.26:0/173340454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4444143ae0 0x7f44440cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:02.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.526+0000 7f4449200640 1 --2- 172.21.15.26:0/173340454 >> v2:172.21.15.124:3300/0 conn(0x7f44440cf7d0 0x7f444415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:02.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.527+0000 7f444a600640 1 --2- 172.21.15.26:0/173340454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f444415e1e0 0x7f444415a670 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f444406e350 tx=0x7f443c02e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5391d127462f083d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:02.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f4443e00640 1 -- 172.21.15.26:0/173340454 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c02ec90 con 0x7f444415e1e0 2024-01-15T18:17:02.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f4443e00640 1 -- 172.21.15.26:0/173340454 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c02edf0 con 0x7f444415e1e0 2024-01-15T18:17:02.531 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f4443e00640 1 -- 172.21.15.26:0/173340454 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c036610 con 0x7f444415e1e0 2024-01-15T18:17:02.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f444c600640 1 -- 172.21.15.26:0/173340454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f444415e1e0 msgr2=0x7f444415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:02.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f444c600640 1 --2- 172.21.15.26:0/173340454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f444415e1e0 0x7f444415a670 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f444406e350 tx=0x7f443c02e0f0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f444c600640 1 -- 172.21.15.26:0/173340454 shutdown_connections 2024-01-15T18:17:02.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f444c600640 1 --2- 172.21.15.26:0/173340454 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f444415e1e0 0x7f444415a670 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f444c600640 1 --2- 172.21.15.26:0/173340454 >> v2:172.21.15.124:3300/0 conn(0x7f44440cf7d0 0x7f444415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f444c600640 1 --2- 172.21.15.26:0/173340454 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4444143ae0 0x7f44440cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f444c600640 1 -- 172.21.15.26:0/173340454 >> 172.21.15.26:0/173340454 conn(0x7f444406a800 msgr2=0x7f444406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:02.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f444c600640 1 -- 172.21.15.26:0/173340454 shutdown_connections 2024-01-15T18:17:02.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.528+0000 7f444c600640 1 -- 172.21.15.26:0/173340454 wait complete. 2024-01-15T18:17:02.533 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.530+0000 7f444c600640 1 Processor -- start 2024-01-15T18:17:02.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.530+0000 7f444c600640 1 -- start start 2024-01-15T18:17:02.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.531+0000 7f444c600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44440cf7d0 0x7f44440c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.531+0000 7f444c600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4444143ae0 0x7f44441417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.531+0000 7f444c600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f444415e1e0 0x7f44440af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.531+0000 7f444c600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4444134ad0 con 0x7f444415e1e0 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.531+0000 7f444c600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4444134db0 con 0x7f4444143ae0 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.531+0000 7f444c600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4444135090 con 0x7f44440cf7d0 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f444a600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f444415e1e0 0x7f44440af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f444a600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f444415e1e0 0x7f44440af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55172/0 (socket says 172.21.15.26:55172) 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f444a600640 1 -- 172.21.15.26:0/3851845730 learned_addr learned my addr 172.21.15.26:0/3851845730 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f444a600640 1 -- 172.21.15.26:0/3851845730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44440cf7d0 msgr2=0x7f44440c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f4449c00640 1 --2- 172.21.15.26:0/3851845730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44440cf7d0 0x7f44440c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f4449200640 1 --2- 172.21.15.26:0/3851845730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4444143ae0 0x7f44441417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f444a600640 1 --2- 172.21.15.26:0/3851845730 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f44440cf7d0 0x7f44440c3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f444a600640 1 -- 172.21.15.26:0/3851845730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4444143ae0 msgr2=0x7f44441417f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:02.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f444a600640 1 --2- 172.21.15.26:0/3851845730 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4444143ae0 0x7f44441417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f444a600640 1 -- 172.21.15.26:0/3851845730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c009b30 con 0x7f444415e1e0 2024-01-15T18:17:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.532+0000 7f444a600640 1 --2- 172.21.15.26:0/3851845730 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f444415e1e0 0x7f44440af8c0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f443c038040 tx=0x7f443c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:02.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.533+0000 7f4441600640 1 -- 172.21.15.26:0/3851845730 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c02ec90 con 0x7f444415e1e0 2024-01-15T18:17:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.533+0000 7f4441600640 1 -- 172.21.15.26:0/3851845730 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f443c009510 con 0x7f444415e1e0 2024-01-15T18:17:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.533+0000 7f444c600640 1 -- 172.21.15.26:0/3851845730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4444135370 con 0x7f444415e1e0 2024-01-15T18:17:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.533+0000 7f4441600640 1 -- 172.21.15.26:0/3851845730 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f443c040640 con 0x7f444415e1e0 2024-01-15T18:17:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.533+0000 7f444c600640 1 -- 172.21.15.26:0/3851845730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4444141d30 con 0x7f444415e1e0 2024-01-15T18:17:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.534+0000 7f444c600640 1 -- 172.21.15.26:0/3851845730 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4444135890 con 0x7f444415e1e0 2024-01-15T18:17:02.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.535+0000 7f4441600640 1 -- 172.21.15.26:0/3851845730 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f443c0407a0 con 0x7f444415e1e0 2024-01-15T18:17:02.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.536+0000 7f4441600640 1 --2- 172.21.15.26:0/3851845730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f441c06c580 0x7f441c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:02.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.536+0000 7f4441600640 1 -- 172.21.15.26:0/3851845730 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f443c0ab6c0 con 0x7f444415e1e0 2024-01-15T18:17:02.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.537+0000 7f4449c00640 1 --2- 172.21.15.26:0/3851845730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f441c06c580 0x7f441c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:02.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.538+0000 7f4449c00640 1 --2- 172.21.15.26:0/3851845730 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f441c06c580 0x7f441c06ea40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f44440b0b10 tx=0x7f4434008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:02.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.541+0000 7f4441600640 1 -- 172.21.15.26:0/3851845730 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4444135890 con 0x7f444415e1e0 2024-01-15T18:17:02.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.765+0000 7f444c600640 1 -- 172.21.15.26:0/3851845730 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f4444134f20 con 0x7f441c06c580 2024-01-15T18:17:02.792 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:02.790+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:02.789+0000 7f4441600640 1 -- 172.21.15.26:0/3851845730 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f4444134f20 con 0x7f441c06c580 2024-01-15T18:17:02.793 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:02.805 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:03.115 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:17:03.123 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:17:04.806 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:05.033 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.030+0000 7f9cb4000640 1 Processor -- start 2024-01-15T18:17:05.034 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.030+0000 7f9cb4000640 1 -- start start 2024-01-15T18:17:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.032+0000 7f9cb4000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0ae5a0 0x7f9cac15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.032+0000 7f9cb4000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac070c40 con 0x7f9cac0b3ae0 2024-01-15T18:17:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.032+0000 7f9cb4000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cac06c470 con 0x7f9cac0cf290 2024-01-15T18:17:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.032+0000 7f9cb4000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac071450 con 0x7f9cac0ae5a0 2024-01-15T18:17:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.033+0000 7f9cb0c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9cac0b3ae0 0x7f9cac06cbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49724/0 (socket says 172.21.15.26:49724) 2024-01-15T18:17:05.036 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.033+0000 7f9cb0c00640 1 -- 172.21.15.26:0/2640863754 learned_addr learned my addr 172.21.15.26:0/2640863754 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:05.037 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.033+0000 7f9cb1600640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0ae5a0 0x7f9cac15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.033+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3013624043 0 0) 0x7f9cac070c40 con 0x7f9cac0b3ae0 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.033+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c8c003520 con 0x7f9cac0b3ae0 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2771293777 0 0) 0x7f9c8c003520 con 0x7f9cac0b3ae0 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0ae5a0 msgr2=0x7f9cac15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0ae5a0 0x7f9cac15e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 >> v1:172.21.15.124:6789/0 conn(0x7f9cac0cf290 legacy=0x7f9cac0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac06c750 con 0x7f9cac0b3ae0 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9c94001670 con 0x7f9cac0b3ae0 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 >> v1:172.21.15.26:6789/0 conn(0x7f9cac0b3ae0 legacy=0x7f9cac06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c8c0039a0 0x7f9c8c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8c0047c0 0x7f9c8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:05.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0ae5a0 0x7f9c8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac071450 con 0x7f9c8c0039a0 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c00ba30 con 0x7f9c8c0047c0 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c00bba0 con 0x7f9cac0ae5a0 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cb1600640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c8c0039a0 0x7f9c8c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.034+0000 7f9cb0c00640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8c0047c0 0x7f9c8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.035+0000 7f9cb1600640 1 -- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0ae5a0 msgr2=0x7f9c8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.035+0000 7f9cb2000640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0ae5a0 0x7f9c8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.035+0000 7f9cb1600640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0ae5a0 0x7f9c8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.035+0000 7f9cb1600640 1 -- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8c0047c0 msgr2=0x7f9c8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.035+0000 7f9cb1600640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8c0047c0 0x7f9c8c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.035+0000 7f9cb1600640 1 -- 172.21.15.26:0/2640863754 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c003520 con 0x7f9c8c0039a0 2024-01-15T18:17:05.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.035+0000 7f9cb0c00640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8c0047c0 0x7f9c8c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.035+0000 7f9cb2000640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0ae5a0 0x7f9c8c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.036+0000 7f9cb1600640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c8c0039a0 0x7f9c8c003d80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009a90 tx=0x7f9c8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f98b255056611c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.036+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c02c650 con 0x7f9c8c0039a0 2024-01-15T18:17:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.036+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9c02c7b0 con 0x7f9c8c0039a0 2024-01-15T18:17:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.036+0000 7f9cabe00640 1 -- 172.21.15.26:0/2640863754 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c02cb40 con 0x7f9c8c0039a0 2024-01-15T18:17:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.037+0000 7f9cb4000640 1 -- 172.21.15.26:0/2640863754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c8c0039a0 msgr2=0x7f9c8c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.037+0000 7f9cb4000640 1 --2- 172.21.15.26:0/2640863754 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9c8c0039a0 0x7f9c8c003d80 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009a90 tx=0x7f9c8c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.037+0000 7f9cb2000640 1 -- 172.21.15.26:0/2640863754 reap_dead start 2024-01-15T18:17:05.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.037+0000 7f9cb4000640 1 -- 172.21.15.26:0/2640863754 shutdown_connections 2024-01-15T18:17:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.037+0000 7f9cb4000640 1 -- 172.21.15.26:0/2640863754 >> 172.21.15.26:0/2640863754 conn(0x7f9cac06a800 msgr2=0x7f9cac06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.037+0000 7f9cb4000640 1 -- 172.21.15.26:0/2640863754 shutdown_connections 2024-01-15T18:17:05.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.037+0000 7f9cb4000640 1 -- 172.21.15.26:0/2640863754 wait complete. 2024-01-15T18:17:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.039+0000 7f9cb4000640 1 Processor -- start 2024-01-15T18:17:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.039+0000 7f9cb4000640 1 -- start start 2024-01-15T18:17:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb4000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8c0039a0 0x7f9cac0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb4000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cac0cb8a0 0x7f9cac0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb4000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0acd40 0x7f9cac0ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb4000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0ad730 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb4000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac1f5fd0 con 0x7f9c8c0039a0 2024-01-15T18:17:05.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb4000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cac0cbda0 con 0x7f9cac0acd40 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cac0cb8a0 0x7f9cac0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb0c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cac0cb8a0 0x7f9cac0ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55190/0 (socket says 172.21.15.26:55190) 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb0c00640 1 -- 172.21.15.26:0/3012267637 learned_addr learned my addr 172.21.15.26:0/3012267637 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb1600640 1 --2- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8c0039a0 0x7f9cac0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb2000640 1 --2- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0acd40 0x7f9cac0ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb0c00640 1 -- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0acd40 msgr2=0x7f9cac0ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb0c00640 1 --2- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0acd40 0x7f9cac0ad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb0c00640 1 -- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8c0039a0 msgr2=0x7f9cac0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.040+0000 7f9cb0c00640 1 --2- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9c8c0039a0 0x7f9cac0cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:05.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.041+0000 7f9cb0c00640 1 -- 172.21.15.26:0/3012267637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cac1f6220 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.041+0000 7f9cb2000640 1 --2- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cac0acd40 0x7f9cac0ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.041+0000 7f9cb0c00640 1 --2- 172.21.15.26:0/3012267637 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cac0cb8a0 0x7f9cac0ce540 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400d860 tx=0x7f9c9400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.041+0000 7f9ca9600640 1 -- 172.21.15.26:0/3012267637 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940074e0 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.041+0000 7f9ca9600640 1 -- 172.21.15.26:0/3012267637 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c94007640 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.041+0000 7f9ca9600640 1 -- 172.21.15.26:0/3012267637 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940049e0 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.041+0000 7f9cb4000640 1 -- 172.21.15.26:0/3012267637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cac1f6500 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.041+0000 7f9cb4000640 1 -- 172.21.15.26:0/3012267637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cac0cc840 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.042+0000 7f9cb4000640 1 -- 172.21.15.26:0/3012267637 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cac0675d0 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.043+0000 7f9ca9600640 1 -- 172.21.15.26:0/3012267637 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f9c94004b40 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.046+0000 7f9ca9600640 1 --2- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c8406c5d0 0x7f9c8406ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.046+0000 7f9ca9600640 1 -- 172.21.15.26:0/3012267637 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f9c94088730 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.046+0000 7f9cb1600640 1 --2- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c8406c5d0 0x7f9c8406ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:05.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.046+0000 7f9cb1600640 1 --2- 172.21.15.26:0/3012267637 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9c8406c5d0 0x7f9c8406ea90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c036000 tx=0x7f9c9c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:05.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.051+0000 7f9ca9600640 1 -- 172.21.15.26:0/3012267637 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c94054d20 con 0x7f9cac0cb8a0 2024-01-15T18:17:05.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.271+0000 7f9cb4000640 1 -- 172.21.15.26:0/3012267637 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cac070550 con 0x7f9c8406c5d0 2024-01-15T18:17:05.295 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:05.293+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:05.292+0000 7f9ca9600640 1 -- 172.21.15.26:0/3012267637 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f9c9c004020 con 0x7f9c8406c5d0 2024-01-15T18:17:05.296 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:05.311 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:07.313 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:07.550 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.548+0000 7f121e400640 1 Processor -- start 2024-01-15T18:17:07.551 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.548+0000 7f121e400640 1 -- start start 2024-01-15T18:17:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.550+0000 7f121e400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f121814b2c0 0x7f12180cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.550+0000 7f121e400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f12180cf7d0 0x7f1218070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.550+0000 7f121e400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12180cfbf0 con 0x7f1218070ea0 2024-01-15T18:17:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.550+0000 7f121e400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121814b6a0 con 0x7f12180cf7d0 2024-01-15T18:17:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.550+0000 7f121e400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1218071690 con 0x7f121814b2c0 2024-01-15T18:17:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.550+0000 7f1217e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f1218070ea0 0x7f12180b1ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49738/0 (socket says 172.21.15.26:49738) 2024-01-15T18:17:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.550+0000 7f1217e00640 1 -- 172.21.15.26:0/1082271349 learned_addr learned my addr 172.21.15.26:0/1082271349 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:07.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.550+0000 7f120e800640 1 --2- 172.21.15.26:0/1082271349 >> v2:172.21.15.26:3300/0 conn(0x7f12180cf7d0 0x7f1218070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.551+0000 7f1217400640 1 --2- 172.21.15.26:0/1082271349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f121814b2c0 0x7f12180cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.551+0000 7f1216a00640 1 -- 172.21.15.26:0/1082271349 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050611785 0 0) 0x7f12180cfbf0 con 0x7f1218070ea0 2024-01-15T18:17:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.551+0000 7f120e800640 1 -- 172.21.15.26:0/1082271349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f121814b2c0 msgr2=0x7f12180cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.551+0000 7f120e800640 1 --2- 172.21.15.26:0/1082271349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f121814b2c0 0x7f12180cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.551+0000 7f120e800640 1 -- 172.21.15.26:0/1082271349 >> v1:172.21.15.26:6789/0 conn(0x7f1218070ea0 legacy=0x7f12180b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.554 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.551+0000 7f120e800640 1 -- 172.21.15.26:0/1082271349 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1218071970 con 0x7f12180cf7d0 2024-01-15T18:17:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.552+0000 7f120e800640 1 --2- 172.21.15.26:0/1082271349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12180cf7d0 0x7f1218070960 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f12000097a0 tx=0x7f120002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5ac5943b5a3344 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.552+0000 7f1216a00640 1 -- 172.21.15.26:0/1082271349 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120002d040 con 0x7f12180cf7d0 2024-01-15T18:17:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.552+0000 7f1216a00640 1 -- 172.21.15.26:0/1082271349 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1200031810 con 0x7f12180cf7d0 2024-01-15T18:17:07.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.552+0000 7f1216a00640 1 -- 172.21.15.26:0/1082271349 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200031ba0 con 0x7f12180cf7d0 2024-01-15T18:17:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.553+0000 7f121e400640 1 -- 172.21.15.26:0/1082271349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12180cf7d0 msgr2=0x7f1218070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.553+0000 7f121e400640 1 --2- 172.21.15.26:0/1082271349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12180cf7d0 0x7f1218070960 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f12000097a0 tx=0x7f120002b8b0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.553+0000 7f121e400640 1 -- 172.21.15.26:0/1082271349 shutdown_connections 2024-01-15T18:17:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.553+0000 7f121e400640 1 --2- 172.21.15.26:0/1082271349 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f12180cf7d0 0x7f1218070960 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.553+0000 7f121e400640 1 --2- 172.21.15.26:0/1082271349 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f121814b2c0 0x7f12180cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.553+0000 7f121e400640 1 -- 172.21.15.26:0/1082271349 >> 172.21.15.26:0/1082271349 conn(0x7f121806a800 msgr2=0x7f121806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.553+0000 7f121e400640 1 -- 172.21.15.26:0/1082271349 shutdown_connections 2024-01-15T18:17:07.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.553+0000 7f121e400640 1 -- 172.21.15.26:0/1082271349 wait complete. 2024-01-15T18:17:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.555+0000 7f121e400640 1 Processor -- start 2024-01-15T18:17:07.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.555+0000 7f121e400640 1 -- start start 2024-01-15T18:17:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.556+0000 7f121e400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1218070ea0 0x7f12181420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.556+0000 7f121e400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12180cf7d0 0x7f12180caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.556+0000 7f121e400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f121814b2c0 0x7f1218163a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.556+0000 7f121e400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12181425e0 con 0x7f1218070ea0 2024-01-15T18:17:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.556+0000 7f121e400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218143e10 con 0x7f12180cf7d0 2024-01-15T18:17:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.556+0000 7f121e400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1218162bd0 con 0x7f121814b2c0 2024-01-15T18:17:07.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f1217400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1218070ea0 0x7f12181420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f1217400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1218070ea0 0x7f12181420a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55210/0 (socket says 172.21.15.26:55210) 2024-01-15T18:17:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f1217400640 1 -- 172.21.15.26:0/2591142081 learned_addr learned my addr 172.21.15.26:0/2591142081 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f1217e00640 1 --2- 172.21.15.26:0/2591142081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f121814b2c0 0x7f1218163a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f120e800640 1 --2- 172.21.15.26:0/2591142081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12180cf7d0 0x7f12180caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f1217400640 1 -- 172.21.15.26:0/2591142081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f121814b2c0 msgr2=0x7f1218163a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f1217400640 1 --2- 172.21.15.26:0/2591142081 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f121814b2c0 0x7f1218163a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f1217400640 1 -- 172.21.15.26:0/2591142081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12180cf7d0 msgr2=0x7f12180caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f1217400640 1 --2- 172.21.15.26:0/2591142081 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f12180cf7d0 0x7f12180caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.557+0000 7f1217400640 1 -- 172.21.15.26:0/2591142081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200009400 con 0x7f1218070ea0 2024-01-15T18:17:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.558+0000 7f1217400640 1 --2- 172.21.15.26:0/2591142081 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f1218070ea0 0x7f12181420a0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f120800d5f0 tx=0x7f120800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.558+0000 7f120fe00640 1 -- 172.21.15.26:0/2591142081 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12080125f0 con 0x7f1218070ea0 2024-01-15T18:17:07.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.558+0000 7f120fe00640 1 -- 172.21.15.26:0/2591142081 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1208012750 con 0x7f1218070ea0 2024-01-15T18:17:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.558+0000 7f120fe00640 1 -- 172.21.15.26:0/2591142081 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208012b00 con 0x7f1218070ea0 2024-01-15T18:17:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.558+0000 7f121e400640 1 -- 172.21.15.26:0/2591142081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1218162eb0 con 0x7f1218070ea0 2024-01-15T18:17:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.558+0000 7f121e400640 1 -- 172.21.15.26:0/2591142081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12181440b0 con 0x7f1218070ea0 2024-01-15T18:17:07.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.559+0000 7f121e400640 1 -- 172.21.15.26:0/2591142081 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12180675d0 con 0x7f1218070ea0 2024-01-15T18:17:07.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.561+0000 7f120fe00640 1 -- 172.21.15.26:0/2591142081 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f1208012c60 con 0x7f1218070ea0 2024-01-15T18:17:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.562+0000 7f120fe00640 1 --2- 172.21.15.26:0/2591142081 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11f006c580 0x7f11f006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:07.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.562+0000 7f120fe00640 1 -- 172.21.15.26:0/2591142081 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f1208089020 con 0x7f1218070ea0 2024-01-15T18:17:07.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.562+0000 7f120e800640 1 --2- 172.21.15.26:0/2591142081 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11f006c580 0x7f11f006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:07.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.563+0000 7f120e800640 1 --2- 172.21.15.26:0/2591142081 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f11f006c580 0x7f11f006ea40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1200036000 tx=0x7f120003c040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:07.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.567+0000 7f120fe00640 1 -- 172.21.15.26:0/2591142081 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1208055560 con 0x7f1218070ea0 2024-01-15T18:17:07.790 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.788+0000 7f121e400640 1 -- 172.21.15.26:0/2591142081 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f121806abe0 con 0x7f11f006c580 2024-01-15T18:17:07.813 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:07.811+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:07.811+0000 7f120fe00640 1 -- 172.21.15.26:0/2591142081 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f121806abe0 con 0x7f11f006c580 2024-01-15T18:17:07.814 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:07.826 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:09.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:10.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.061+0000 7f9710e00640 1 Processor -- start 2024-01-15T18:17:10.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.061+0000 7f9710e00640 1 -- start start 2024-01-15T18:17:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.063+0000 7f9710e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f970c06d8d0 0x7f970c16c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.063+0000 7f9710e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970c0702e0 con 0x7f970c162810 2024-01-15T18:17:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.063+0000 7f9710e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970c0705c0 con 0x7f970c162430 2024-01-15T18:17:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.063+0000 7f9710e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970c0708a0 con 0x7f970c06d8d0 2024-01-15T18:17:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.063+0000 7f970aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f970c162810 0x7f970c071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49740/0 (socket says 172.21.15.26:49740) 2024-01-15T18:17:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.064+0000 7f970aa00640 1 -- 172.21.15.26:0/3228642835 learned_addr learned my addr 172.21.15.26:0/3228642835 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:10.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.064+0000 7f970be00640 1 --2- 172.21.15.26:0/3228642835 >> v2:172.21.15.26:3300/0 conn(0x7f970c06d8d0 0x7f970c16c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.064+0000 7f970a000640 1 -- 172.21.15.26:0/3228642835 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254280464 0 0) 0x7f970c0702e0 con 0x7f970c162810 2024-01-15T18:17:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.064+0000 7f970a000640 1 -- 172.21.15.26:0/3228642835 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96ec003520 con 0x7f970c162810 2024-01-15T18:17:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.064+0000 7f970be00640 1 -- 172.21.15.26:0/3228642835 >> v1:172.21.15.124:6789/0 conn(0x7f970c162430 legacy=0x7f970c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.064+0000 7f970be00640 1 -- 172.21.15.26:0/3228642835 >> v1:172.21.15.26:6789/0 conn(0x7f970c162810 legacy=0x7f970c071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:10.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.064+0000 7f970be00640 1 -- 172.21.15.26:0/3228642835 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c070b80 con 0x7f970c06d8d0 2024-01-15T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.066+0000 7f970be00640 1 --2- 172.21.15.26:0/3228642835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f970c06d8d0 0x7f970c16c450 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f97040097a0 tx=0x7f970402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4796485a9b4650 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:10.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.066+0000 7f970a000640 1 -- 172.21.15.26:0/3228642835 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704030610 con 0x7f970c06d8d0 2024-01-15T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.066+0000 7f970a000640 1 -- 172.21.15.26:0/3228642835 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9704030770 con 0x7f970c06d8d0 2024-01-15T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.066+0000 7f970a000640 1 -- 172.21.15.26:0/3228642835 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704030ae0 con 0x7f970c06d8d0 2024-01-15T18:17:10.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.067+0000 7f9710e00640 1 -- 172.21.15.26:0/3228642835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f970c06d8d0 msgr2=0x7f970c16c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.067+0000 7f9710e00640 1 --2- 172.21.15.26:0/3228642835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f970c06d8d0 0x7f970c16c450 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f97040097a0 tx=0x7f970402b960 comp rx=0 tx=0).stop 2024-01-15T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.067+0000 7f9710e00640 1 -- 172.21.15.26:0/3228642835 shutdown_connections 2024-01-15T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.067+0000 7f9710e00640 1 --2- 172.21.15.26:0/3228642835 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f970c06d8d0 0x7f970c16c450 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.067+0000 7f9710e00640 1 -- 172.21.15.26:0/3228642835 >> 172.21.15.26:0/3228642835 conn(0x7f970c06a800 msgr2=0x7f970c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.067+0000 7f9710e00640 1 -- 172.21.15.26:0/3228642835 shutdown_connections 2024-01-15T18:17:10.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.067+0000 7f9710e00640 1 -- 172.21.15.26:0/3228642835 wait complete. 2024-01-15T18:17:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.069+0000 7f9710e00640 1 Processor -- start 2024-01-15T18:17:10.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.069+0000 7f9710e00640 1 -- start start 2024-01-15T18:17:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.070+0000 7f9710e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f970c06d8d0 0x7f970c1ed320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:10.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.070+0000 7f9710e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f970c162430 0x7f970c1469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.070+0000 7f9710e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f970c154290 0x7f970c146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.070+0000 7f9710e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c147420 con 0x7f970c154290 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.070+0000 7f9710e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0b0fd0 con 0x7f970c162430 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.070+0000 7f9710e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0b1500 con 0x7f970c06d8d0 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.070+0000 7f970be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f970c154290 0x7f970c146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.070+0000 7f970be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f970c154290 0x7f970c146ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:55236/0 (socket says 172.21.15.26:55236) 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.070+0000 7f970be00640 1 -- 172.21.15.26:0/1176859541 learned_addr learned my addr 172.21.15.26:0/1176859541 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f970aa00640 1 --2- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f970c162430 0x7f970c1469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f970b400640 1 --2- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f970c06d8d0 0x7f970c1ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f970be00640 1 -- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f970c06d8d0 msgr2=0x7f970c1ed320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f970be00640 1 --2- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f970c06d8d0 0x7f970c1ed320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:10.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f970be00640 1 -- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f970c162430 msgr2=0x7f970c1469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f970be00640 1 --2- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f970c162430 0x7f970c1469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f970be00640 1 -- 172.21.15.26:0/1176859541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9704009400 con 0x7f970c154290 2024-01-15T18:17:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f970aa00640 1 --2- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f970c162430 0x7f970c1469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:10.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f970be00640 1 --2- 172.21.15.26:0/1176859541 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f970c154290 0x7f970c146ee0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f970402bf70 tx=0x7f97040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.071+0000 7f9703400640 1 -- 172.21.15.26:0/1176859541 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970403f070 con 0x7f970c154290 2024-01-15T18:17:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.072+0000 7f9703400640 1 -- 172.21.15.26:0/1176859541 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9704006e10 con 0x7f970c154290 2024-01-15T18:17:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.072+0000 7f9703400640 1 -- 172.21.15.26:0/1176859541 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704030ea0 con 0x7f970c154290 2024-01-15T18:17:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.072+0000 7f9710e00640 1 -- 172.21.15.26:0/1176859541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970c162d30 con 0x7f970c154290 2024-01-15T18:17:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.072+0000 7f9710e00640 1 -- 172.21.15.26:0/1176859541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970c162fd0 con 0x7f970c154290 2024-01-15T18:17:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.073+0000 7f9710e00640 1 -- 172.21.15.26:0/1176859541 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970c0b37a0 con 0x7f970c154290 2024-01-15T18:17:10.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.074+0000 7f9703400640 1 -- 172.21.15.26:0/1176859541 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f9704003830 con 0x7f970c154290 2024-01-15T18:17:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.075+0000 7f9703400640 1 --2- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96e406c550 0x7f96e406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.075+0000 7f9703400640 1 -- 172.21.15.26:0/1176859541 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f97040aa8b0 con 0x7f970c154290 2024-01-15T18:17:10.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.075+0000 7f970b400640 1 --2- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96e406c550 0x7f96e406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:10.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.077+0000 7f970b400640 1 --2- 172.21.15.26:0/1176859541 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f96e406c550 0x7f96e406ea10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f96fc007fd0 tx=0x7f96fc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:10.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.079+0000 7f9703400640 1 -- 172.21.15.26:0/1176859541 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9704076df0 con 0x7f970c154290 2024-01-15T18:17:10.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.310+0000 7f9710e00640 1 -- 172.21.15.26:0/1176859541 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f970c067450 con 0x7f96e406c550 2024-01-15T18:17:10.334 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:10.332+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:10.332+0000 7f9703400640 1 -- 172.21.15.26:0/1176859541 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f970c067450 con 0x7f96e406c550 2024-01-15T18:17:10.335 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:10.347 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:12.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:12.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.586+0000 7f4041c00640 1 Processor -- start 2024-01-15T18:17:12.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.586+0000 7f4041c00640 1 -- start start 2024-01-15T18:17:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.588+0000 7f4041c00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f403c071f00 0x7f403c146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.588+0000 7f4041c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f403c06c7c0 0x7f403c166cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.588+0000 7f4041c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0b2e10 con 0x7f403c071410 2024-01-15T18:17:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.588+0000 7f4041c00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c06cc10 con 0x7f403c071f00 2024-01-15T18:17:12.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.588+0000 7f4041c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c06cec0 con 0x7f403c06c7c0 2024-01-15T18:17:12.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.589+0000 7f403b400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f403c071f00 0x7f403c146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.589+0000 7f4041200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f403c06c7c0 0x7f403c166cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.589+0000 7f403b400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f403c071f00 0x7f403c146770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:35250/0 (socket says 172.21.15.26:35250) 2024-01-15T18:17:12.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.589+0000 7f403b400640 1 -- 172.21.15.26:0/53226699 learned_addr learned my addr 172.21.15.26:0/53226699 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:12.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.589+0000 7f403b400640 1 -- 172.21.15.26:0/53226699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f403c06c7c0 msgr2=0x7f403c166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:12.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.589+0000 7f403b400640 1 --2- 172.21.15.26:0/53226699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f403c06c7c0 0x7f403c166cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.589+0000 7f403b400640 1 -- 172.21.15.26:0/53226699 >> v1:172.21.15.124:6789/0 conn(0x7f403c071410 legacy=0x7f403c0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.589+0000 7f403b400640 1 -- 172.21.15.26:0/53226699 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c06d1a0 con 0x7f403c071f00 2024-01-15T18:17:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.591+0000 7f403b400640 1 --2- 172.21.15.26:0/53226699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f403c071f00 0x7f403c146770 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4024009870 tx=0x7f402402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1139a4027997038b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.591+0000 7f403aa00640 1 -- 172.21.15.26:0/53226699 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024031640 con 0x7f403c071f00 2024-01-15T18:17:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.591+0000 7f403aa00640 1 -- 172.21.15.26:0/53226699 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40240317a0 con 0x7f403c071f00 2024-01-15T18:17:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.591+0000 7f403aa00640 1 -- 172.21.15.26:0/53226699 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024031b30 con 0x7f403c071f00 2024-01-15T18:17:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.592+0000 7f4041c00640 1 -- 172.21.15.26:0/53226699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f403c071f00 msgr2=0x7f403c146770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.592+0000 7f4041c00640 1 --2- 172.21.15.26:0/53226699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f403c071f00 0x7f403c146770 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4024009870 tx=0x7f402402b8f0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.592+0000 7f4041c00640 1 -- 172.21.15.26:0/53226699 shutdown_connections 2024-01-15T18:17:12.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.592+0000 7f4041c00640 1 --2- 172.21.15.26:0/53226699 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f403c06c7c0 0x7f403c166cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.592+0000 7f4041c00640 1 --2- 172.21.15.26:0/53226699 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f403c071f00 0x7f403c146770 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.592+0000 7f4041c00640 1 -- 172.21.15.26:0/53226699 >> 172.21.15.26:0/53226699 conn(0x7f403c06a800 msgr2=0x7f403c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.592+0000 7f4041c00640 1 -- 172.21.15.26:0/53226699 shutdown_connections 2024-01-15T18:17:12.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.592+0000 7f4041c00640 1 -- 172.21.15.26:0/53226699 wait complete. 2024-01-15T18:17:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.594+0000 7f4041c00640 1 Processor -- start 2024-01-15T18:17:12.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.594+0000 7f4041c00640 1 -- start start 2024-01-15T18:17:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.595+0000 7f4041c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f403c06c7c0 0x7f403c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.595+0000 7f4041c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f403c071410 0x7f403c0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.595+0000 7f4041c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c071f00 0x7f403c140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.595+0000 7f4041c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0c4c60 con 0x7f403c071f00 2024-01-15T18:17:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.595+0000 7f4041c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c15e9a0 con 0x7f403c06c7c0 2024-01-15T18:17:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.595+0000 7f4041c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c15ec80 con 0x7f403c071410 2024-01-15T18:17:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.595+0000 7f4041200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c071f00 0x7f403c140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f4041200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c071f00 0x7f403c140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50432/0 (socket says 172.21.15.26:50432) 2024-01-15T18:17:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f4041200640 1 -- 172.21.15.26:0/2773835331 learned_addr learned my addr 172.21.15.26:0/2773835331 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f403b400640 1 --2- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f403c071410 0x7f403c0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f403be00640 1 --2- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f403c06c7c0 0x7f403c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f4041200640 1 -- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f403c071410 msgr2=0x7f403c0c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f4041200640 1 --2- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f403c071410 0x7f403c0c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f4041200640 1 -- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f403c06c7c0 msgr2=0x7f403c15e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f4041200640 1 --2- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f403c06c7c0 0x7f403c15e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f4041200640 1 -- 172.21.15.26:0/2773835331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240094d0 con 0x7f403c071f00 2024-01-15T18:17:12.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f403be00640 1 --2- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f403c06c7c0 0x7f403c15e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.596+0000 7f4041200640 1 --2- 172.21.15.26:0/2773835331 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f403c071f00 0x7f403c140500 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f403400efc0 tx=0x7f403400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.597+0000 7f4033e00640 1 -- 172.21.15.26:0/2773835331 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034014480 con 0x7f403c071f00 2024-01-15T18:17:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.597+0000 7f4033e00640 1 -- 172.21.15.26:0/2773835331 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403400f040 con 0x7f403c071f00 2024-01-15T18:17:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.597+0000 7f4033e00640 1 -- 172.21.15.26:0/2773835331 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034014820 con 0x7f403c071f00 2024-01-15T18:17:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.597+0000 7f4041c00640 1 -- 172.21.15.26:0/2773835331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c15ef60 con 0x7f403c071f00 2024-01-15T18:17:12.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.597+0000 7f4041c00640 1 -- 172.21.15.26:0/2773835331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c0c4f00 con 0x7f403c071f00 2024-01-15T18:17:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.598+0000 7f4041c00640 1 -- 172.21.15.26:0/2773835331 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403c15f480 con 0x7f403c071f00 2024-01-15T18:17:12.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.599+0000 7f4033e00640 1 -- 172.21.15.26:0/2773835331 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f4034014980 con 0x7f403c071f00 2024-01-15T18:17:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.600+0000 7f4033e00640 1 --2- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f401406c550 0x7f401406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.600+0000 7f4033e00640 1 -- 172.21.15.26:0/2773835331 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f4034089760 con 0x7f403c071f00 2024-01-15T18:17:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.601+0000 7f403be00640 1 --2- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f401406c550 0x7f401406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:12.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.602+0000 7f403be00640 1 --2- 172.21.15.26:0/2773835331 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f401406c550 0x7f401406ea10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f403c15e820 tx=0x7f402c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:12.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.605+0000 7f4033e00640 1 -- 172.21.15.26:0/2773835331 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f403c15f480 con 0x7f403c071f00 2024-01-15T18:17:12.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.824+0000 7f4041c00640 1 -- 172.21.15.26:0/2773835331 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f403c06da40 con 0x7f401406c550 2024-01-15T18:17:12.845 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:12.843+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:12.843+0000 7f4033e00640 1 -- 172.21.15.26:0/2773835331 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f402c001cd0 con 0x7f401406c550 2024-01-15T18:17:12.846 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:12.862 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:14.863 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.104+0000 7fcd67200640 1 Processor -- start 2024-01-15T18:17:15.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.105+0000 7fcd67200640 1 -- start start 2024-01-15T18:17:15.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.106+0000 7fcd67200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fcd60099a60 0x7fcd60099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:15.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.106+0000 7fcd67200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcd6006e7c0 0x7fcd6006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:15.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.106+0000 7fcd67200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd600cfa70 con 0x7fcd6015e880 2024-01-15T18:17:15.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.106+0000 7fcd67200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6009a3e0 con 0x7fcd60099a60 2024-01-15T18:17:15.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.106+0000 7fcd67200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd600cff00 con 0x7fcd6006e7c0 2024-01-15T18:17:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.107+0000 7fcd5fe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcd6015e880 0x7fcd6006ff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49968/0 (socket says 172.21.15.26:49968) 2024-01-15T18:17:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.107+0000 7fcd5fe00640 1 -- 172.21.15.26:0/11412797 learned_addr learned my addr 172.21.15.26:0/11412797 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.107+0000 7fcd5f400640 1 --2- 172.21.15.26:0/11412797 >> v2:172.21.15.26:3300/0 conn(0x7fcd60099a60 0x7fcd60099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.107+0000 7fcd65200640 1 --2- 172.21.15.26:0/11412797 >> v2:172.21.15.124:3300/0 conn(0x7fcd6006e7c0 0x7fcd6006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.107+0000 7fcd5ea00640 1 -- 172.21.15.26:0/11412797 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712139267 0 0) 0x7fcd600cfa70 con 0x7fcd6015e880 2024-01-15T18:17:15.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.107+0000 7fcd5ea00640 1 -- 172.21.15.26:0/11412797 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd40003520 con 0x7fcd6015e880 2024-01-15T18:17:15.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.108+0000 7fcd5f400640 1 -- 172.21.15.26:0/11412797 >> v2:172.21.15.124:3300/0 conn(0x7fcd6006e7c0 msgr2=0x7fcd6006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:15.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.108+0000 7fcd5f400640 1 --2- 172.21.15.26:0/11412797 >> v2:172.21.15.124:3300/0 conn(0x7fcd6006e7c0 0x7fcd6006ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:15.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.108+0000 7fcd5f400640 1 -- 172.21.15.26:0/11412797 >> v1:172.21.15.26:6789/0 conn(0x7fcd6015e880 legacy=0x7fcd6006ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:15.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.108+0000 7fcd5f400640 1 -- 172.21.15.26:0/11412797 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6006fc70 con 0x7fcd60099a60 2024-01-15T18:17:15.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.109+0000 7fcd5f400640 1 --2- 172.21.15.26:0/11412797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60099a60 0x7fcd60099ea0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fcd600b72c0 tx=0x7fcd4802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76240d00994998b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:15.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.109+0000 7fcd5ea00640 1 -- 172.21.15.26:0/11412797 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48030610 con 0x7fcd60099a60 2024-01-15T18:17:15.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.109+0000 7fcd5ea00640 1 -- 172.21.15.26:0/11412797 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48030770 con 0x7fcd60099a60 2024-01-15T18:17:15.112 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.109+0000 7fcd5ea00640 1 -- 172.21.15.26:0/11412797 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48030b00 con 0x7fcd60099a60 2024-01-15T18:17:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.110+0000 7fcd67200640 1 -- 172.21.15.26:0/11412797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60099a60 msgr2=0x7fcd60099ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.110+0000 7fcd67200640 1 --2- 172.21.15.26:0/11412797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60099a60 0x7fcd60099ea0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fcd600b72c0 tx=0x7fcd4802b960 comp rx=0 tx=0).stop 2024-01-15T18:17:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.110+0000 7fcd67200640 1 -- 172.21.15.26:0/11412797 shutdown_connections 2024-01-15T18:17:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.110+0000 7fcd67200640 1 --2- 172.21.15.26:0/11412797 >> v2:172.21.15.124:3300/0 conn(0x7fcd6006e7c0 0x7fcd6006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.110+0000 7fcd67200640 1 --2- 172.21.15.26:0/11412797 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60099a60 0x7fcd60099ea0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.110+0000 7fcd67200640 1 -- 172.21.15.26:0/11412797 >> 172.21.15.26:0/11412797 conn(0x7fcd600b2d00 msgr2=0x7fcd600b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.110+0000 7fcd67200640 1 -- 172.21.15.26:0/11412797 shutdown_connections 2024-01-15T18:17:15.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.110+0000 7fcd67200640 1 -- 172.21.15.26:0/11412797 wait complete. 2024-01-15T18:17:15.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.112+0000 7fcd67200640 1 Processor -- start 2024-01-15T18:17:15.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.112+0000 7fcd67200640 1 -- start start 2024-01-15T18:17:15.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.113+0000 7fcd67200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd6006e7c0 0x7fcd60153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:15.116 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.113+0000 7fcd67200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60099a60 0x7fcd601549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.113+0000 7fcd67200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd6015e880 0x7fcd600b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.113+0000 7fcd67200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd600b1ee0 con 0x7fcd60099a60 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.113+0000 7fcd67200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd60153930 con 0x7fcd6006e7c0 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.113+0000 7fcd67200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd60153c10 con 0x7fcd6015e880 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.113+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60099a60 0x7fcd601549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.113+0000 7fcd5f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60099a60 0x7fcd601549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50450/0 (socket says 172.21.15.26:50450) 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.113+0000 7fcd5f400640 1 -- 172.21.15.26:0/759959264 learned_addr learned my addr 172.21.15.26:0/759959264 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.114+0000 7fcd5fe00640 1 --2- 172.21.15.26:0/759959264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd6006e7c0 0x7fcd60153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.114+0000 7fcd65200640 1 --2- 172.21.15.26:0/759959264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd6015e880 0x7fcd600b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.114+0000 7fcd5f400640 1 -- 172.21.15.26:0/759959264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd6015e880 msgr2=0x7fcd600b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.114+0000 7fcd5f400640 1 --2- 172.21.15.26:0/759959264 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcd6015e880 0x7fcd600b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.114+0000 7fcd5f400640 1 -- 172.21.15.26:0/759959264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd6006e7c0 msgr2=0x7fcd60153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:15.117 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.114+0000 7fcd5f400640 1 --2- 172.21.15.26:0/759959264 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcd6006e7c0 0x7fcd60153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.114+0000 7fcd5f400640 1 -- 172.21.15.26:0/759959264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd48009400 con 0x7fcd60099a60 2024-01-15T18:17:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.114+0000 7fcd5f400640 1 --2- 172.21.15.26:0/759959264 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcd60099a60 0x7fcd601549d0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009890 tx=0x7fcd48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:15.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.115+0000 7fcd57e00640 1 -- 172.21.15.26:0/759959264 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4803e040 con 0x7fcd60099a60 2024-01-15T18:17:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.115+0000 7fcd57e00640 1 -- 172.21.15.26:0/759959264 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48039cb0 con 0x7fcd60099a60 2024-01-15T18:17:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.115+0000 7fcd57e00640 1 -- 172.21.15.26:0/759959264 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48044630 con 0x7fcd60099a60 2024-01-15T18:17:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.115+0000 7fcd67200640 1 -- 172.21.15.26:0/759959264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd60153ef0 con 0x7fcd60099a60 2024-01-15T18:17:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.115+0000 7fcd67200640 1 -- 172.21.15.26:0/759959264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd601544b0 con 0x7fcd60099a60 2024-01-15T18:17:15.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.116+0000 7fcd67200640 1 -- 172.21.15.26:0/759959264 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd601546a0 con 0x7fcd60099a60 2024-01-15T18:17:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.117+0000 7fcd57e00640 1 -- 172.21.15.26:0/759959264 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fcd48044790 con 0x7fcd60099a60 2024-01-15T18:17:15.120 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.118+0000 7fcd57e00640 1 --2- 172.21.15.26:0/759959264 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcd3806c550 0x7fcd3806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:15.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.118+0000 7fcd57e00640 1 -- 172.21.15.26:0/759959264 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fcd480aaf40 con 0x7fcd60099a60 2024-01-15T18:17:15.121 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.119+0000 7fcd5fe00640 1 --2- 172.21.15.26:0/759959264 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcd3806c550 0x7fcd3806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:15.122 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.120+0000 7fcd5fe00640 1 --2- 172.21.15.26:0/759959264 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcd3806c550 0x7fcd3806ea10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fcd500011e0 tx=0x7fcd50009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:15.125 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.123+0000 7fcd57e00640 1 -- 172.21.15.26:0/759959264 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd48077480 con 0x7fcd60099a60 2024-01-15T18:17:15.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.344+0000 7fcd67200640 1 -- 172.21.15.26:0/759959264 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd600cf700 con 0x7fcd3806c550 2024-01-15T18:17:15.365 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:15.362+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:15.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:15.363+0000 7fcd57e00640 1 -- 172.21.15.26:0/759959264 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7fcd50001c60 con 0x7fcd3806c550 2024-01-15T18:17:15.366 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:15.388 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:17.389 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.618+0000 7f980d600640 1 Processor -- start 2024-01-15T18:17:17.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.618+0000 7f980d600640 1 -- start start 2024-01-15T18:17:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.619+0000 7f980d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98080b3f30 0x7f98080d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.619+0000 7f980d600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98080d0a10 con 0x7f98080d0100 2024-01-15T18:17:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.619+0000 7f980d600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98080cf080 con 0x7f9808160920 2024-01-15T18:17:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.619+0000 7f980d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98080c6be0 con 0x7f98080b3f30 2024-01-15T18:17:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.619+0000 7f9807400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f98080d0100 0x7f98080d0f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49984/0 (socket says 172.21.15.26:49984) 2024-01-15T18:17:17.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.619+0000 7f9807400640 1 -- 172.21.15.26:0/3865800547 learned_addr learned my addr 172.21.15.26:0/3865800547 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.620+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948379360 0 0) 0x7f98080d0a10 con 0x7f98080d0100 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.620+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e8003520 con 0x7f98080d0100 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.620+0000 7f980cc00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98080b3f30 0x7f98080d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.620+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1753982701 0 0) 0x7f97e8003520 con 0x7f98080d0100 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.620+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98080b3f30 msgr2=0x7f98080d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.620+0000 7f9806a00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98080b3f30 0x7f98080d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.620+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 >> v1:172.21.15.124:6789/0 conn(0x7f9808160920 legacy=0x7f98080cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.620+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080ce7e0 con 0x7f98080d0100 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f97f0001670 con 0x7f98080d0100 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 >> v1:172.21.15.26:6789/0 conn(0x7f98080d0100 legacy=0x7f98080d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9806a00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97e80039a0 0x7f97e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9806a00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98080b3f30 0x7f97e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:17.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9806a00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e8004d00 0x7f97e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98080c6be0 con 0x7f97e8004d00 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e800bb60 con 0x7f97e80039a0 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e800bd20 con 0x7f98080b3f30 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f980cc00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e8004d00 0x7f97e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9807400640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98080b3f30 0x7f97e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9807e00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97e80039a0 0x7f97e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f980cc00640 1 -- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98080b3f30 msgr2=0x7f97e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f980cc00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98080b3f30 0x7f97e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f980cc00640 1 -- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97e80039a0 msgr2=0x7f97e8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f980cc00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97e80039a0 0x7f97e8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f980cc00640 1 -- 172.21.15.26:0/3865800547 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8003520 con 0x7f97e8004d00 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9807400640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f98080b3f30 0x7f97e80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:17.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.621+0000 7f9807e00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f97e80039a0 0x7f97e8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:17.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.623+0000 7f980cc00640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e8004d00 0x7f97e8005180 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f9800007360 tx=0x7f97e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8318bccd8dc60deb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:17.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.623+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980002ccb0 con 0x7f97e8004d00 2024-01-15T18:17:17.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.623+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980002ce10 con 0x7f97e8004d00 2024-01-15T18:17:17.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.623+0000 7f9806a00640 1 -- 172.21.15.26:0/3865800547 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800034660 con 0x7f97e8004d00 2024-01-15T18:17:17.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.623+0000 7f980d600640 1 -- 172.21.15.26:0/3865800547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e8004d00 msgr2=0x7f97e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:17.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.623+0000 7f980d600640 1 --2- 172.21.15.26:0/3865800547 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f97e8004d00 0x7f97e8005180 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f9800007360 tx=0x7f97e80057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:17.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.623+0000 7f980cc00640 1 -- 172.21.15.26:0/3865800547 reap_dead start 2024-01-15T18:17:17.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.623+0000 7f980d600640 1 -- 172.21.15.26:0/3865800547 shutdown_connections 2024-01-15T18:17:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.623+0000 7f980d600640 1 -- 172.21.15.26:0/3865800547 >> 172.21.15.26:0/3865800547 conn(0x7f98080d3090 msgr2=0x7f98080d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.624+0000 7f980d600640 1 -- 172.21.15.26:0/3865800547 shutdown_connections 2024-01-15T18:17:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.624+0000 7f980d600640 1 -- 172.21.15.26:0/3865800547 wait complete. 2024-01-15T18:17:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.625+0000 7f980d600640 1 Processor -- start 2024-01-15T18:17:17.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.626+0000 7f980d600640 1 -- start start 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.626+0000 7f980d600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97e8004d00 0x7f9808151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.626+0000 7f980d600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98080b8ab0 0x7f980815dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f980815e4f0 0x7f98080c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980d600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9808151e70 con 0x7f980815e4f0 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980d600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98080cbf30 con 0x7f98080b8ab0 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980d600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98080ccf60 con 0x7f97e8004d00 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f980815e4f0 0x7f98080c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f980815e4f0 0x7f98080c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50466/0 (socket says 172.21.15.26:50466) 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980cc00640 1 -- 172.21.15.26:0/1080687857 learned_addr learned my addr 172.21.15.26:0/1080687857 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:17.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f9807e00640 1 --2- 172.21.15.26:0/1080687857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97e8004d00 0x7f9808151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f9807400640 1 --2- 172.21.15.26:0/1080687857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98080b8ab0 0x7f980815dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980cc00640 1 -- 172.21.15.26:0/1080687857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97e8004d00 msgr2=0x7f9808151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980cc00640 1 --2- 172.21.15.26:0/1080687857 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f97e8004d00 0x7f9808151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980cc00640 1 -- 172.21.15.26:0/1080687857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98080b8ab0 msgr2=0x7f980815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980cc00640 1 --2- 172.21.15.26:0/1080687857 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f98080b8ab0 0x7f980815dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.627+0000 7f980cc00640 1 -- 172.21.15.26:0/1080687857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080cc1a0 con 0x7f980815e4f0 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.628+0000 7f980cc00640 1 --2- 172.21.15.26:0/1080687857 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f980815e4f0 0x7f98080c8a30 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f9800037040 tx=0x7f9800004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.628+0000 7f97ffe00640 1 -- 172.21.15.26:0/1080687857 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980003e640 con 0x7f980815e4f0 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.628+0000 7f97ffe00640 1 -- 172.21.15.26:0/1080687857 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980003e7a0 con 0x7f980815e4f0 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.628+0000 7f980d600640 1 -- 172.21.15.26:0/1080687857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98080cc480 con 0x7f980815e4f0 2024-01-15T18:17:17.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.628+0000 7f97ffe00640 1 -- 172.21.15.26:0/1080687857 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980003eb50 con 0x7f980815e4f0 2024-01-15T18:17:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.628+0000 7f980d600640 1 -- 172.21.15.26:0/1080687857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98080cca40 con 0x7f980815e4f0 2024-01-15T18:17:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.629+0000 7f980d600640 1 -- 172.21.15.26:0/1080687857 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98080ccc60 con 0x7f980815e4f0 2024-01-15T18:17:17.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.631+0000 7f97ffe00640 1 -- 172.21.15.26:0/1080687857 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f980003ecb0 con 0x7f980815e4f0 2024-01-15T18:17:17.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.631+0000 7f97ffe00640 1 --2- 172.21.15.26:0/1080687857 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f97e006c580 0x7f97e006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:17.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.632+0000 7f97ffe00640 1 -- 172.21.15.26:0/1080687857 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f98000ab0b0 con 0x7f980815e4f0 2024-01-15T18:17:17.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.632+0000 7f9807e00640 1 --2- 172.21.15.26:0/1080687857 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f97e006c580 0x7f97e006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:17.636 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.633+0000 7f9807e00640 1 --2- 172.21.15.26:0/1080687857 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f97e006c580 0x7f97e006ea40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f97f8001750 tx=0x7f97f8009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:17.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.636+0000 7f97ffe00640 1 -- 172.21.15.26:0/1080687857 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9800077710 con 0x7f980815e4f0 2024-01-15T18:17:17.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.867+0000 7f980d600640 1 -- 172.21.15.26:0/1080687857 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f9808160d00 con 0x7f97e006c580 2024-01-15T18:17:17.893 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:17.890+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:17.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:17.891+0000 7f97ffe00640 1 -- 172.21.15.26:0/1080687857 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f97f800d050 con 0x7f97e006c580 2024-01-15T18:17:17.894 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:17.909 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:19.910 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.141+0000 7f2141000640 1 Processor -- start 2024-01-15T18:17:20.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.141+0000 7f2141000640 1 -- start start 2024-01-15T18:17:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.143+0000 7f2141000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f213c0cc500 0x7f213c0cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.143+0000 7f2141000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f213c06d160 0x7f213c06d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.143+0000 7f2141000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213c070e50 con 0x7f213c143c40 2024-01-15T18:17:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.143+0000 7f2141000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213c0723c0 con 0x7f213c06d160 2024-01-15T18:17:20.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.143+0000 7f2141000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c144020 con 0x7f213c0cc500 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.143+0000 7f213b400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f213c143c40 0x7f213c071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:49998/0 (socket says 172.21.15.26:49998) 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.143+0000 7f213b400640 1 -- 172.21.15.26:0/1792919132 learned_addr learned my addr 172.21.15.26:0/1792919132 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.143+0000 7f213be00640 1 --2- 172.21.15.26:0/1792919132 >> v2:172.21.15.26:3300/0 conn(0x7f213c06d160 0x7f213c06d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.144+0000 7f2133e00640 1 --2- 172.21.15.26:0/1792919132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f213c0cc500 0x7f213c0cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.144+0000 7f213aa00640 1 -- 172.21.15.26:0/1792919132 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571935334 0 0) 0x7f213c070e50 con 0x7f213c143c40 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.144+0000 7f213aa00640 1 -- 172.21.15.26:0/1792919132 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f211c003520 con 0x7f213c143c40 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.144+0000 7f213be00640 1 -- 172.21.15.26:0/1792919132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f213c0cc500 msgr2=0x7f213c0cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:20.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.144+0000 7f213be00640 1 --2- 172.21.15.26:0/1792919132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f213c0cc500 0x7f213c0cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.144+0000 7f213be00640 1 -- 172.21.15.26:0/1792919132 >> v1:172.21.15.26:6789/0 conn(0x7f213c143c40 legacy=0x7f213c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.144+0000 7f213be00640 1 -- 172.21.15.26:0/1792919132 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c0ccea0 con 0x7f213c06d160 2024-01-15T18:17:20.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.145+0000 7f213be00640 1 --2- 172.21.15.26:0/1792919132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c06d160 0x7f213c06d590 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f2134009d90 tx=0x7f213402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46539a489c1f0131 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:20.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.145+0000 7f213aa00640 1 -- 172.21.15.26:0/1792919132 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213402cbc0 con 0x7f213c06d160 2024-01-15T18:17:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.145+0000 7f213aa00640 1 -- 172.21.15.26:0/1792919132 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213402cd20 con 0x7f213c06d160 2024-01-15T18:17:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.145+0000 7f213aa00640 1 -- 172.21.15.26:0/1792919132 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134034660 con 0x7f213c06d160 2024-01-15T18:17:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.146+0000 7f2141000640 1 -- 172.21.15.26:0/1792919132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c06d160 msgr2=0x7f213c06d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.146+0000 7f2141000640 1 --2- 172.21.15.26:0/1792919132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c06d160 0x7f213c06d590 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f2134009d90 tx=0x7f213402be10 comp rx=0 tx=0).stop 2024-01-15T18:17:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.146+0000 7f2141000640 1 -- 172.21.15.26:0/1792919132 shutdown_connections 2024-01-15T18:17:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.146+0000 7f2141000640 1 --2- 172.21.15.26:0/1792919132 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c06d160 0x7f213c06d590 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.146+0000 7f2141000640 1 --2- 172.21.15.26:0/1792919132 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f213c0cc500 0x7f213c0cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.146+0000 7f2141000640 1 -- 172.21.15.26:0/1792919132 >> 172.21.15.26:0/1792919132 conn(0x7f213c06a800 msgr2=0x7f213c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:20.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.146+0000 7f2141000640 1 -- 172.21.15.26:0/1792919132 shutdown_connections 2024-01-15T18:17:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.146+0000 7f2141000640 1 -- 172.21.15.26:0/1792919132 wait complete. 2024-01-15T18:17:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.148+0000 7f2141000640 1 Processor -- start 2024-01-15T18:17:20.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.148+0000 7f2141000640 1 -- start start 2024-01-15T18:17:20.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.149+0000 7f2141000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f213c06d160 0x7f213c13c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.149+0000 7f2141000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f213c0cc500 0x7f213c0c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.149+0000 7f2141000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c143c40 0x7f213c142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.149+0000 7f2141000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0c4710 con 0x7f213c143c40 2024-01-15T18:17:20.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.149+0000 7f2141000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0c64f0 con 0x7f213c06d160 2024-01-15T18:17:20.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.149+0000 7f2141000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c0c3d70 con 0x7f213c0cc500 2024-01-15T18:17:20.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.149+0000 7f213be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c143c40 0x7f213c142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.149+0000 7f213be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c143c40 0x7f213c142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:50484/0 (socket says 172.21.15.26:50484) 2024-01-15T18:17:20.155 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.149+0000 7f213be00640 1 -- 172.21.15.26:0/2395007028 learned_addr learned my addr 172.21.15.26:0/2395007028 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.150+0000 7f2133e00640 1 --2- 172.21.15.26:0/2395007028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f213c0cc500 0x7f213c0c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.150+0000 7f213be00640 1 -- 172.21.15.26:0/2395007028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f213c0cc500 msgr2=0x7f213c0c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.150+0000 7f213be00640 1 --2- 172.21.15.26:0/2395007028 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f213c0cc500 0x7f213c0c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.150+0000 7f213b400640 1 --2- 172.21.15.26:0/2395007028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f213c06d160 0x7f213c13c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.150+0000 7f213be00640 1 -- 172.21.15.26:0/2395007028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f213c06d160 msgr2=0x7f213c13c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.150+0000 7f213be00640 1 --2- 172.21.15.26:0/2395007028 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f213c06d160 0x7f213c13c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.150+0000 7f213be00640 1 -- 172.21.15.26:0/2395007028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2134009a70 con 0x7f213c143c40 2024-01-15T18:17:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.150+0000 7f213be00640 1 --2- 172.21.15.26:0/2395007028 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f213c143c40 0x7f213c142280 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f2134009d90 tx=0x7f2134004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:20.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.150+0000 7f2133400640 1 -- 172.21.15.26:0/2395007028 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2134040650 con 0x7f213c143c40 2024-01-15T18:17:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.151+0000 7f2133400640 1 -- 172.21.15.26:0/2395007028 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213402ce90 con 0x7f213c143c40 2024-01-15T18:17:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.151+0000 7f2141000640 1 -- 172.21.15.26:0/2395007028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c0c4050 con 0x7f213c143c40 2024-01-15T18:17:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.151+0000 7f2133400640 1 -- 172.21.15.26:0/2395007028 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21340409f0 con 0x7f213c143c40 2024-01-15T18:17:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.151+0000 7f2141000640 1 -- 172.21.15.26:0/2395007028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c13cbc0 con 0x7f213c143c40 2024-01-15T18:17:20.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.152+0000 7f2141000640 1 -- 172.21.15.26:0/2395007028 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c0c4850 con 0x7f213c143c40 2024-01-15T18:17:20.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.153+0000 7f2133400640 1 -- 172.21.15.26:0/2395007028 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f2134040b50 con 0x7f213c143c40 2024-01-15T18:17:20.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.154+0000 7f2133400640 1 --2- 172.21.15.26:0/2395007028 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f211406c550 0x7f211406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:20.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.154+0000 7f2133400640 1 -- 172.21.15.26:0/2395007028 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f21340ab800 con 0x7f213c143c40 2024-01-15T18:17:20.158 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.155+0000 7f213b400640 1 --2- 172.21.15.26:0/2395007028 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f211406c550 0x7f211406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:20.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.156+0000 7f213b400640 1 --2- 172.21.15.26:0/2395007028 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f211406c550 0x7f211406ea10 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f212c0011e0 tx=0x7f212c009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:20.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.159+0000 7f2133400640 1 -- 172.21.15.26:0/2395007028 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f213c0c4850 con 0x7f213c143c40 2024-01-15T18:17:20.384 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.381+0000 7f2141000640 1 -- 172.21.15.26:0/2395007028 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f213c06c960 con 0x7f211406c550 2024-01-15T18:17:20.404 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:20.401+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:20.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:20.401+0000 7f2133400640 1 -- 172.21.15.26:0/2395007028 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f212c0012d0 con 0x7f211406c550 2024-01-15T18:17:20.405 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:20.426 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:22.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.665+0000 7f9d02600640 1 Processor -- start 2024-01-15T18:17:22.668 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.665+0000 7f9d02600640 1 -- start start 2024-01-15T18:17:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.667+0000 7f9d02600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9cfc1607a0 0x7f9cfc0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.667+0000 7f9d02600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9cfc0b3eb0 0x7f9cfc1610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.667+0000 7f9d02600640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cfc0b2b90 con 0x7f9cfc160b80 2024-01-15T18:17:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.667+0000 7f9d02600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cfc0d21d0 con 0x7f9cfc0b3eb0 2024-01-15T18:17:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.667+0000 7f9d02600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cfc0b2e30 con 0x7f9cfc1607a0 2024-01-15T18:17:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.667+0000 7f9cf3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9cfc0b3eb0 0x7f9cfc1610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.668+0000 7f9cf3e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9cfc0b3eb0 0x7f9cfc1610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34488/0 (socket says 172.21.15.26:34488) 2024-01-15T18:17:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.668+0000 7f9cf3e00640 1 -- 172.21.15.26:0/64762605 learned_addr learned my addr 172.21.15.26:0/64762605 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:22.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.668+0000 7f9d01200640 1 --2- 172.21.15.26:0/64762605 >> v2:172.21.15.124:3300/0 conn(0x7f9cfc1607a0 0x7f9cfc0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.668+0000 7f9cf3e00640 1 -- 172.21.15.26:0/64762605 >> v2:172.21.15.124:3300/0 conn(0x7f9cfc1607a0 msgr2=0x7f9cfc0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.668+0000 7f9cf3e00640 1 --2- 172.21.15.26:0/64762605 >> v2:172.21.15.124:3300/0 conn(0x7f9cfc1607a0 0x7f9cfc0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.668+0000 7f9cf3e00640 1 -- 172.21.15.26:0/64762605 >> v1:172.21.15.124:6789/0 conn(0x7f9cfc160b80 legacy=0x7f9cfc161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:22.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.668+0000 7f9cf3e00640 1 -- 172.21.15.26:0/64762605 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc0b3110 con 0x7f9cfc0b3eb0 2024-01-15T18:17:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.669+0000 7f9cf3e00640 1 --2- 172.21.15.26:0/64762605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc0b3eb0 0x7f9cfc1610c0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc08d7c0 tx=0x7f9ce402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4c854c42550a3e server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9cfbe00640 1 -- 172.21.15.26:0/64762605 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4031870 con 0x7f9cfc0b3eb0 2024-01-15T18:17:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9cfbe00640 1 -- 172.21.15.26:0/64762605 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce40319d0 con 0x7f9cfc0b3eb0 2024-01-15T18:17:22.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9cfbe00640 1 -- 172.21.15.26:0/64762605 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4031d60 con 0x7f9cfc0b3eb0 2024-01-15T18:17:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9d02600640 1 -- 172.21.15.26:0/64762605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc0b3eb0 msgr2=0x7f9cfc1610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9d02600640 1 --2- 172.21.15.26:0/64762605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc0b3eb0 0x7f9cfc1610c0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc08d7c0 tx=0x7f9ce402b910 comp rx=0 tx=0).stop 2024-01-15T18:17:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9d02600640 1 -- 172.21.15.26:0/64762605 shutdown_connections 2024-01-15T18:17:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9d02600640 1 --2- 172.21.15.26:0/64762605 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc0b3eb0 0x7f9cfc1610c0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9d02600640 1 --2- 172.21.15.26:0/64762605 >> v2:172.21.15.124:3300/0 conn(0x7f9cfc1607a0 0x7f9cfc0b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9d02600640 1 -- 172.21.15.26:0/64762605 >> 172.21.15.26:0/64762605 conn(0x7f9cfc09c590 msgr2=0x7f9cfc0d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.670+0000 7f9d02600640 1 -- 172.21.15.26:0/64762605 shutdown_connections 2024-01-15T18:17:22.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.671+0000 7f9d02600640 1 -- 172.21.15.26:0/64762605 wait complete. 2024-01-15T18:17:22.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.672+0000 7f9d02600640 1 Processor -- start 2024-01-15T18:17:22.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.673+0000 7f9d02600640 1 -- start start 2024-01-15T18:17:22.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.673+0000 7f9d02600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9cfc0b3eb0 0x7f9cfc0c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.673+0000 7f9d02600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc1607a0 0x7f9cfc1512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9d02600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cfc160b80 0x7f9cfc151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9d02600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0b6210 con 0x7f9cfc1607a0 2024-01-15T18:17:22.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9d02600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0c88d0 con 0x7f9cfc0b3eb0 2024-01-15T18:17:22.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9d02600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0c8bb0 con 0x7f9cfc160b80 2024-01-15T18:17:22.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9cf3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc1607a0 0x7f9cfc1512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9cf3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc1607a0 0x7f9cfc1512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34498/0 (socket says 172.21.15.26:34498) 2024-01-15T18:17:22.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9cf3e00640 1 -- 172.21.15.26:0/188830524 learned_addr learned my addr 172.21.15.26:0/188830524 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:22.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9d01200640 1 --2- 172.21.15.26:0/188830524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9cfc0b3eb0 0x7f9cfc0c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9d01c00640 1 --2- 172.21.15.26:0/188830524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cfc160b80 0x7f9cfc151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9cf3e00640 1 -- 172.21.15.26:0/188830524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cfc160b80 msgr2=0x7f9cfc151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9cf3e00640 1 --2- 172.21.15.26:0/188830524 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9cfc160b80 0x7f9cfc151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9cf3e00640 1 -- 172.21.15.26:0/188830524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9cfc0b3eb0 msgr2=0x7f9cfc0c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:22.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9cf3e00640 1 --2- 172.21.15.26:0/188830524 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9cfc0b3eb0 0x7f9cfc0c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:22.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.674+0000 7f9cf3e00640 1 -- 172.21.15.26:0/188830524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce40094d0 con 0x7f9cfc1607a0 2024-01-15T18:17:22.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.675+0000 7f9cf3e00640 1 --2- 172.21.15.26:0/188830524 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9cfc1607a0 0x7f9cfc1512c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4000c00 tx=0x7f9ce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:22.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.675+0000 7f9cf9600640 1 -- 172.21.15.26:0/188830524 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce403abe0 con 0x7f9cfc1607a0 2024-01-15T18:17:22.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.675+0000 7f9cf9600640 1 -- 172.21.15.26:0/188830524 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce403ad40 con 0x7f9cfc1607a0 2024-01-15T18:17:22.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.675+0000 7f9cf9600640 1 -- 172.21.15.26:0/188830524 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4044610 con 0x7f9cfc1607a0 2024-01-15T18:17:22.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.675+0000 7f9d02600640 1 -- 172.21.15.26:0/188830524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc0c8e90 con 0x7f9cfc1607a0 2024-01-15T18:17:22.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.675+0000 7f9d02600640 1 -- 172.21.15.26:0/188830524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc0b5e30 con 0x7f9cfc1607a0 2024-01-15T18:17:22.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.676+0000 7f9d02600640 1 -- 172.21.15.26:0/188830524 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cfc0c93b0 con 0x7f9cfc1607a0 2024-01-15T18:17:22.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.677+0000 7f9cf9600640 1 -- 172.21.15.26:0/188830524 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f9ce404d450 con 0x7f9cfc1607a0 2024-01-15T18:17:22.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.678+0000 7f9cf9600640 1 --2- 172.21.15.26:0/188830524 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9cd406c550 0x7f9cd406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:22.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.678+0000 7f9cf9600640 1 -- 172.21.15.26:0/188830524 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f9ce402d040 con 0x7f9cfc1607a0 2024-01-15T18:17:22.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.679+0000 7f9d01200640 1 --2- 172.21.15.26:0/188830524 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9cd406c550 0x7f9cd406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:22.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.680+0000 7f9d01200640 1 --2- 172.21.15.26:0/188830524 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9cd406c550 0x7f9cd406ea10 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006fd0 tx=0x7f9cec008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:22.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.683+0000 7f9cf9600640 1 -- 172.21.15.26:0/188830524 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9cfc0c93b0 con 0x7f9cfc1607a0 2024-01-15T18:17:22.909 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.906+0000 7f9d02600640 1 -- 172.21.15.26:0/188830524 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cfc0c8a40 con 0x7f9cd406c550 2024-01-15T18:17:22.928 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:22.926+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:22.926+0000 7f9cf9600640 1 -- 172.21.15.26:0/188830524 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f9cfc0c8a40 con 0x7f9cd406c550 2024-01-15T18:17:22.929 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:22.946 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:24.947 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:25.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.186+0000 7f9ea6a00640 1 Processor -- start 2024-01-15T18:17:25.189 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.186+0000 7f9ea6a00640 1 -- start start 2024-01-15T18:17:25.193 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.188+0000 7f9ea6a00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f9ea0162810 0x7f9ea0071240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.188+0000 7f9ea6a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea0071c10 con 0x7f9ea0162430 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.188+0000 7f9ea6a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea00702e0 con 0x7f9ea0071780 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.188+0000 7f9ea6a00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00705c0 con 0x7f9ea0162810 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.188+0000 7f9ea5600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9ea0162430 0x7f9ea00b3110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60590/0 (socket says 172.21.15.26:60590) 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.188+0000 7f9ea5600640 1 -- 172.21.15.26:0/3337002909 learned_addr learned my addr 172.21.15.26:0/3337002909 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97170134 0 0) 0x7f9ea0071c10 con 0x7f9ea0162430 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9ea4c00640 1 --2- 172.21.15.26:0/3337002909 >> v2:172.21.15.124:3300/0 conn(0x7f9ea0162810 0x7f9ea0071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e80003520 con 0x7f9ea0162430 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091228340 0 0) 0x7f9e80003520 con 0x7f9ea0162430 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 >> v2:172.21.15.124:3300/0 conn(0x7f9ea0162810 msgr2=0x7f9ea0071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 --2- 172.21.15.26:0/3337002909 >> v2:172.21.15.124:3300/0 conn(0x7f9ea0162810 0x7f9ea0071240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.194 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 >> v1:172.21.15.124:6789/0 conn(0x7f9ea0071780 legacy=0x7f9ea006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea00708a0 con 0x7f9ea0162430 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f9e90001670 con 0x7f9ea0162430 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 >> v1:172.21.15.26:6789/0 conn(0x7f9ea0162430 legacy=0x7f9ea00b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800039c0 0x7f9e80003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e800047c0 0x7f9e80004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.189+0000 7f9e9fe00640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 0x7f9e8000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00705c0 con 0x7f9e800051d0 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8000c100 con 0x7f9e800047c0 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8000c2c0 con 0x7f9e800039c0 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea6000640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 0x7f9e8000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea5600640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800039c0 0x7f9e80003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea4c00640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e800047c0 0x7f9e80004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea6000640 1 -- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800039c0 msgr2=0x7f9e80003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea6000640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800039c0 0x7f9e80003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea6000640 1 -- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e800047c0 msgr2=0x7f9e80004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea6000640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e800047c0 0x7f9e80004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea6000640 1 -- 172.21.15.26:0/3337002909 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80003520 con 0x7f9e800051d0 2024-01-15T18:17:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea5600640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9e800039c0 0x7f9e80003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.190+0000 7f9ea4c00640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9e800047c0 0x7f9e80004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.191+0000 7f9ea6000640 1 -- 172.21.15.26:0/3337002909 reap_dead start 2024-01-15T18:17:25.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.191+0000 7f9ea6000640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 0x7f9e8000b980 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f9e98004480 tx=0x7f9e800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a6e273d21fdef4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.192+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98006b50 con 0x7f9e800051d0 2024-01-15T18:17:25.199 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.192+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98006cd0 con 0x7f9e800051d0 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.192+0000 7f9e9fe00640 1 -- 172.21.15.26:0/3337002909 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98007260 con 0x7f9e800051d0 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.192+0000 7f9ea6a00640 1 -- 172.21.15.26:0/3337002909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 msgr2=0x7f9e8000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.192+0000 7f9ea6a00640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 0x7f9e8000b980 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f9e98004480 tx=0x7f9e800057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.192+0000 7f9ea6a00640 1 -- 172.21.15.26:0/3337002909 shutdown_connections 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.192+0000 7f9ea6a00640 1 --2- 172.21.15.26:0/3337002909 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 0x7f9e8000b980 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.192+0000 7f9ea6a00640 1 -- 172.21.15.26:0/3337002909 >> 172.21.15.26:0/3337002909 conn(0x7f9ea006a800 msgr2=0x7f9ea006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.192+0000 7f9ea6a00640 1 -- 172.21.15.26:0/3337002909 shutdown_connections 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.193+0000 7f9ea6a00640 1 -- 172.21.15.26:0/3337002909 wait complete. 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.194+0000 7f9ea6a00640 1 Processor -- start 2024-01-15T18:17:25.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.194+0000 7f9ea6a00640 1 -- start start 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.195+0000 7f9ea6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 0x7f9ea0158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.195+0000 7f9ea6a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ea014d500 0x7f9ea0142bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.195+0000 7f9ea6a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ea015af50 0x7f9ea015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.195+0000 7f9ea6a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea015b910 con 0x7f9e800051d0 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.195+0000 7f9ea6a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea01593b0 con 0x7f9ea015af50 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.195+0000 7f9ea6a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0159690 con 0x7f9ea014d500 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea5600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 0x7f9ea0158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea5600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 0x7f9ea0158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34508/0 (socket says 172.21.15.26:34508) 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea5600640 1 -- 172.21.15.26:0/281369732 learned_addr learned my addr 172.21.15.26:0/281369732 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea4c00640 1 --2- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ea014d500 0x7f9ea0142bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea6000640 1 --2- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ea015af50 0x7f9ea015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea5600640 1 -- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ea014d500 msgr2=0x7f9ea0142bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea5600640 1 --2- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9ea014d500 0x7f9ea0142bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea5600640 1 -- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ea015af50 msgr2=0x7f9ea015b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea5600640 1 --2- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ea015af50 0x7f9ea015b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea5600640 1 -- 172.21.15.26:0/281369732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea0159970 con 0x7f9e800051d0 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.196+0000 7f9ea6000640 1 --2- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9ea015af50 0x7f9ea015b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.197+0000 7f9ea5600640 1 --2- 172.21.15.26:0/281369732 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9e800051d0 0x7f9ea0158b60 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f9e9000d770 tx=0x7f9e9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.197+0000 7f9e9d600640 1 -- 172.21.15.26:0/281369732 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900074e0 con 0x7f9e800051d0 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.197+0000 7f9e9d600640 1 -- 172.21.15.26:0/281369732 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e90007640 con 0x7f9e800051d0 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.197+0000 7f9e9d600640 1 -- 172.21.15.26:0/281369732 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900049e0 con 0x7f9e800051d0 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.197+0000 7f9ea6a00640 1 -- 172.21.15.26:0/281369732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea0159c50 con 0x7f9e800051d0 2024-01-15T18:17:25.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.197+0000 7f9ea6a00640 1 -- 172.21.15.26:0/281369732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea01590a0 con 0x7f9e800051d0 2024-01-15T18:17:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.198+0000 7f9ea6a00640 1 -- 172.21.15.26:0/281369732 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea0067450 con 0x7f9e800051d0 2024-01-15T18:17:25.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.199+0000 7f9e9d600640 1 -- 172.21.15.26:0/281369732 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f9e90004b40 con 0x7f9e800051d0 2024-01-15T18:17:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.201+0000 7f9e9d600640 1 --2- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9e7806c580 0x7f9e7806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.201+0000 7f9e9d600640 1 -- 172.21.15.26:0/281369732 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f9e900884d0 con 0x7f9e800051d0 2024-01-15T18:17:25.204 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.201+0000 7f9ea4c00640 1 --2- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9e7806c580 0x7f9e7806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:25.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.202+0000 7f9ea4c00640 1 --2- 172.21.15.26:0/281369732 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9e7806c580 0x7f9e7806ea40 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0143580 tx=0x7f9e88008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:25.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.205+0000 7f9e9d600640 1 -- 172.21.15.26:0/281369732 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e90054a10 con 0x7f9e800051d0 2024-01-15T18:17:25.429 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.427+0000 7f9ea6a00640 1 -- 172.21.15.26:0/281369732 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ea00b3bf0 con 0x7f9e7806c580 2024-01-15T18:17:25.448 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:25.446+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:25.446+0000 7f9e9d600640 1 -- 172.21.15.26:0/281369732 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f9ea00b3bf0 con 0x7f9e7806c580 2024-01-15T18:17:25.449 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:25.463 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:27.469 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:27.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.702+0000 7ff95cc00640 1 Processor -- start 2024-01-15T18:17:27.705 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.703+0000 7ff95cc00640 1 -- start start 2024-01-15T18:17:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff95cc00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff9580ae5a0 0x7ff95815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff95cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9580b3ae0 0x7ff95806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff95cc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff958070c40 con 0x7ff95806d120 2024-01-15T18:17:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff95cc00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff958071450 con 0x7ff9580ae5a0 2024-01-15T18:17:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff95cc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95806c470 con 0x7ff9580b3ae0 2024-01-15T18:17:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff956800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff95806d120 0x7ff9580cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60594/0 (socket says 172.21.15.26:60594) 2024-01-15T18:17:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff956800640 1 -- 172.21.15.26:0/2052381985 learned_addr learned my addr 172.21.15.26:0/2052381985 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff955e00640 1 --2- 172.21.15.26:0/2052381985 >> v2:172.21.15.124:3300/0 conn(0x7ff9580ae5a0 0x7ff95815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff94d200640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9580b3ae0 0x7ff95806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.705+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 299235637 0 0) 0x7ff958070c40 con 0x7ff95806d120 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff938003520 con 0x7ff95806d120 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1861780205 0 0) 0x7ff938003520 con 0x7ff95806d120 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9580b3ae0 msgr2=0x7ff95806cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9580b3ae0 0x7ff95806cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 >> v2:172.21.15.124:3300/0 conn(0x7ff9580ae5a0 msgr2=0x7ff95815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 --2- 172.21.15.26:0/2052381985 >> v2:172.21.15.124:3300/0 conn(0x7ff9580ae5a0 0x7ff95815e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95806c750 con 0x7ff95806d120 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7ff9500019f0 con 0x7ff95806d120 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 >> v1:172.21.15.26:6789/0 conn(0x7ff95806d120 legacy=0x7ff9580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff938003970 0x7ff938003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9380047c0 0x7ff938004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff938005180 0x7ff93800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff958071450 con 0x7ff938005180 2024-01-15T18:17:27.710 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95806c470 con 0x7ff938003970 2024-01-15T18:17:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93800bf40 con 0x7ff9380047c0 2024-01-15T18:17:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff94d200640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9580b3ae0 0x7ff95806cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.706+0000 7ff956800640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff938005180 0x7ff93800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.707+0000 7ff955e00640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff938003970 0x7ff938003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.707+0000 7ff956800640 1 -- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9380047c0 msgr2=0x7ff938004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.707+0000 7ff956800640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9380047c0 0x7ff938004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.707+0000 7ff956800640 1 -- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff938003970 msgr2=0x7ff938003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.707+0000 7ff956800640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff938003970 0x7ff938003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.707+0000 7ff956800640 1 -- 172.21.15.26:0/2052381985 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938003520 con 0x7ff938005180 2024-01-15T18:17:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.708+0000 7ff956800640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff938005180 0x7ff93800b980 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff95000a770 tx=0x7ff9380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882f0085eadd0ff8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.708+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950034630 con 0x7ff938005180 2024-01-15T18:17:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.708+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95002f070 con 0x7ff938005180 2024-01-15T18:17:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.708+0000 7ff955400640 1 -- 172.21.15.26:0/2052381985 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9500349f0 con 0x7ff938005180 2024-01-15T18:17:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.709+0000 7ff95cc00640 1 -- 172.21.15.26:0/2052381985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff938005180 msgr2=0x7ff93800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.709+0000 7ff95cc00640 1 --2- 172.21.15.26:0/2052381985 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff938005180 0x7ff93800b980 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff95000a770 tx=0x7ff9380057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.709+0000 7ff956800640 1 -- 172.21.15.26:0/2052381985 reap_dead start 2024-01-15T18:17:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.709+0000 7ff95cc00640 1 -- 172.21.15.26:0/2052381985 shutdown_connections 2024-01-15T18:17:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.709+0000 7ff95cc00640 1 -- 172.21.15.26:0/2052381985 >> 172.21.15.26:0/2052381985 conn(0x7ff95806a800 msgr2=0x7ff95806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.709+0000 7ff95cc00640 1 -- 172.21.15.26:0/2052381985 shutdown_connections 2024-01-15T18:17:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.709+0000 7ff95cc00640 1 -- 172.21.15.26:0/2052381985 wait complete. 2024-01-15T18:17:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.711+0000 7ff95cc00640 1 Processor -- start 2024-01-15T18:17:27.713 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.711+0000 7ff95cc00640 1 -- start start 2024-01-15T18:17:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.712+0000 7ff95cc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff938005180 0x7ff9581f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.712+0000 7ff95cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95815bcf0 0x7ff9580cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.712+0000 7ff95cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9580cdac0 0x7ff9580cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.712+0000 7ff95cc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95815c1f0 con 0x7ff95815bcf0 2024-01-15T18:17:27.715 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.712+0000 7ff95cc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95815c4d0 con 0x7ff938005180 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.712+0000 7ff95cc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95815c7b0 con 0x7ff9580cdac0 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.712+0000 7ff94d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95815bcf0 0x7ff9580cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.712+0000 7ff94d200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95815bcf0 0x7ff9580cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34526/0 (socket says 172.21.15.26:34526) 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.712+0000 7ff94d200640 1 -- 172.21.15.26:0/3814770388 learned_addr learned my addr 172.21.15.26:0/3814770388 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff955e00640 1 --2- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff938005180 0x7ff9581f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff956800640 1 --2- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9580cdac0 0x7ff9580cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff94d200640 1 -- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9580cdac0 msgr2=0x7ff9580cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff94d200640 1 --2- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff9580cdac0 0x7ff9580cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff94d200640 1 -- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff938005180 msgr2=0x7ff9581f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff94d200640 1 --2- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff938005180 0x7ff9581f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff94d200640 1 -- 172.21.15.26:0/3814770388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95815ca90 con 0x7ff95815bcf0 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff955e00640 1 --2- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff938005180 0x7ff9581f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff94d200640 1 --2- 172.21.15.26:0/3814770388 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff95815bcf0 0x7ff9580cbf00 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7ff94000e560 tx=0x7ff94000ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff94ea00640 1 -- 172.21.15.26:0/3814770388 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940004230 con 0x7ff95815bcf0 2024-01-15T18:17:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff94ea00640 1 -- 172.21.15.26:0/3814770388 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9400043b0 con 0x7ff95815bcf0 2024-01-15T18:17:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff95cc00640 1 -- 172.21.15.26:0/3814770388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95815cd70 con 0x7ff95815bcf0 2024-01-15T18:17:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.713+0000 7ff94ea00640 1 -- 172.21.15.26:0/3814770388 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940004520 con 0x7ff95815bcf0 2024-01-15T18:17:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.714+0000 7ff95cc00640 1 -- 172.21.15.26:0/3814770388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9580cc440 con 0x7ff95815bcf0 2024-01-15T18:17:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.715+0000 7ff95cc00640 1 -- 172.21.15.26:0/3814770388 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9580675d0 con 0x7ff95815bcf0 2024-01-15T18:17:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.716+0000 7ff94ea00640 1 -- 172.21.15.26:0/3814770388 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7ff940003850 con 0x7ff95815bcf0 2024-01-15T18:17:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.717+0000 7ff94ea00640 1 --2- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff93006c580 0x7ff93006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.717+0000 7ff94ea00640 1 -- 172.21.15.26:0/3814770388 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7ff940088320 con 0x7ff95815bcf0 2024-01-15T18:17:27.719 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.717+0000 7ff955e00640 1 --2- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff93006c580 0x7ff93006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.718+0000 7ff955e00640 1 --2- 172.21.15.26:0/3814770388 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff93006c580 0x7ff93006ea40 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff948009a60 tx=0x7ff948008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.723+0000 7ff94ea00640 1 -- 172.21.15.26:0/3814770388 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff940054b60 con 0x7ff95815bcf0 2024-01-15T18:17:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.950+0000 7ff95cc00640 1 -- 172.21.15.26:0/3814770388 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7ff958070550 con 0x7ff93006c580 2024-01-15T18:17:27.972 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:27.968+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:27.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:27.970+0000 7ff94ea00640 1 -- 172.21.15.26:0/3814770388 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7ff9480119a0 con 0x7ff93006c580 2024-01-15T18:17:27.972 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:27.985 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:29.986 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.217+0000 7f0829600640 1 Processor -- start 2024-01-15T18:17:30.220 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.218+0000 7f0829600640 1 -- start start 2024-01-15T18:17:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.219+0000 7f0829600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240ae3f0 0x7f0824071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.219+0000 7f0829600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f0824072210 0x7f0824070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.219+0000 7f0829600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0824072660 con 0x7f082415b3f0 2024-01-15T18:17:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.219+0000 7f0829600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08240ae8a0 con 0x7f0824072210 2024-01-15T18:17:30.222 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.219+0000 7f0829600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082415fd40 con 0x7f08240ae3f0 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.220+0000 7f0822a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f082415b3f0 0x7f08240adce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:60596/0 (socket says 172.21.15.26:60596) 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.220+0000 7f0822a00640 1 -- 172.21.15.26:0/3634010126 learned_addr learned my addr 172.21.15.26:0/3634010126 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.220+0000 7f0823400640 1 --2- 172.21.15.26:0/3634010126 >> v2:172.21.15.124:3300/0 conn(0x7f0824072210 0x7f0824070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.220+0000 7f081be00640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240ae3f0 0x7f0824071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121198356 0 0) 0x7f0814001110 con 0x7f082415b3f0 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0804003520 con 0x7f082415b3f0 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 672120992 0 0) 0x7f0804003520 con 0x7f082415b3f0 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240ae3f0 msgr2=0x7f0824071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240ae3f0 0x7f0824071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 >> v2:172.21.15.124:3300/0 conn(0x7f0824072210 msgr2=0x7f0824070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 --2- 172.21.15.26:0/3634010126 >> v2:172.21.15.124:3300/0 conn(0x7f0824072210 0x7f0824070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.225 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082415ef80 con 0x7f082415b3f0 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f0814001800 con 0x7f082415b3f0 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 >> v1:172.21.15.26:6789/0 conn(0x7f082415b3f0 legacy=0x7f08240adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0804003970 0x7f0804003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08040047c0 0x7f0804004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240ae3f0 0x7f0804005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08240ae8a0 con 0x7f0804003970 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082415fd40 con 0x7f08040047c0 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.221+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080400b980 con 0x7f08240ae3f0 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.222+0000 7f0822a00640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0804003970 0x7f0804003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.222+0000 7f081be00640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08040047c0 0x7f0804004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.222+0000 7f0823400640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240ae3f0 0x7f0804005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.222+0000 7f0822a00640 1 -- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240ae3f0 msgr2=0x7f0804005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.222+0000 7f0822a00640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f08240ae3f0 0x7f0804005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.222+0000 7f0822a00640 1 -- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08040047c0 msgr2=0x7f0804004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.222+0000 7f0822a00640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f08040047c0 0x7f0804004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.226 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.222+0000 7f0822a00640 1 -- 172.21.15.26:0/3634010126 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0804003520 con 0x7f0804003970 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.223+0000 7f0822a00640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0804003970 0x7f0804003d90 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f081400c0e0 tx=0x7f08040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9a64ac44279d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.224+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814005ad0 con 0x7f0804003970 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.224+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0814005c30 con 0x7f0804003970 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.224+0000 7f0822000640 1 -- 172.21.15.26:0/3634010126 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081402c7f0 con 0x7f0804003970 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.224+0000 7f0829600640 1 -- 172.21.15.26:0/3634010126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0804003970 msgr2=0x7f0804003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.224+0000 7f0829600640 1 --2- 172.21.15.26:0/3634010126 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f0804003970 0x7f0804003d90 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f081400c0e0 tx=0x7f08040057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.224+0000 7f0823400640 1 -- 172.21.15.26:0/3634010126 reap_dead start 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.225+0000 7f0829600640 1 -- 172.21.15.26:0/3634010126 shutdown_connections 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.225+0000 7f0829600640 1 -- 172.21.15.26:0/3634010126 >> 172.21.15.26:0/3634010126 conn(0x7f08240b0c00 msgr2=0x7f08240b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.225+0000 7f0829600640 1 -- 172.21.15.26:0/3634010126 shutdown_connections 2024-01-15T18:17:30.228 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.225+0000 7f0829600640 1 -- 172.21.15.26:0/3634010126 wait complete. 2024-01-15T18:17:30.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.227+0000 7f0829600640 1 Processor -- start 2024-01-15T18:17:30.229 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.227+0000 7f0829600640 1 -- start start 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0829600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0804003970 0x7f08241457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0829600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f082414f340 0x7f08241f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0829600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08241f17f0 0x7f0824157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0829600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082415b970 con 0x7f08241f17f0 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0829600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082409b3c0 con 0x7f0804003970 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0829600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082409b6a0 con 0x7f082414f340 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0823400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08241f17f0 0x7f0824157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0823400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08241f17f0 0x7f0824157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:34548/0 (socket says 172.21.15.26:34548) 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0823400640 1 -- 172.21.15.26:0/3361585915 learned_addr learned my addr 172.21.15.26:0/3361585915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:30.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f0822a00640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0804003970 0x7f08241457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.228+0000 7f081be00640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f082414f340 0x7f08241f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f0823400640 1 -- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f082414f340 msgr2=0x7f08241f12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f0823400640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f082414f340 0x7f08241f12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f0823400640 1 -- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0804003970 msgr2=0x7f08241457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f0823400640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0804003970 0x7f08241457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f0823400640 1 -- 172.21.15.26:0/3361585915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082409b980 con 0x7f08241f17f0 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f081be00640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f082414f340 0x7f08241f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f0822a00640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0804003970 0x7f08241457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f0823400640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f08241f17f0 0x7f0824157b00 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f08240743d0 tx=0x7f081c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f081aa00640 1 -- 172.21.15.26:0/3361585915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c017400 con 0x7f08241f17f0 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f081aa00640 1 -- 172.21.15.26:0/3361585915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c011040 con 0x7f08241f17f0 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f0829600640 1 -- 172.21.15.26:0/3361585915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082409bc60 con 0x7f08241f17f0 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.229+0000 7f081aa00640 1 -- 172.21.15.26:0/3361585915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c0177c0 con 0x7f08241f17f0 2024-01-15T18:17:30.233 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.230+0000 7f0829600640 1 -- 172.21.15.26:0/3361585915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0824145d20 con 0x7f08241f17f0 2024-01-15T18:17:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.231+0000 7f0829600640 1 -- 172.21.15.26:0/3361585915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082415bab0 con 0x7f08241f17f0 2024-01-15T18:17:30.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.232+0000 7f081aa00640 1 -- 172.21.15.26:0/3361585915 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f081c017920 con 0x7f08241f17f0 2024-01-15T18:17:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.233+0000 7f081aa00640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07fc06c550 0x7f07fc06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.233+0000 7f081aa00640 1 -- 172.21.15.26:0/3361585915 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f081c088fd0 con 0x7f08241f17f0 2024-01-15T18:17:30.235 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.233+0000 7f0822a00640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07fc06c550 0x7f07fc06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:30.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.235+0000 7f0822a00640 1 --2- 172.21.15.26:0/3361585915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f07fc06c550 0x7f07fc06ea10 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0814034000 tx=0x7f0814007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:30.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.237+0000 7f081aa00640 1 -- 172.21.15.26:0/3361585915 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f082415bab0 con 0x7f08241f17f0 2024-01-15T18:17:30.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.460+0000 7f0829600640 1 -- 172.21.15.26:0/3361585915 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f082415f820 con 0x7f07fc06c550 2024-01-15T18:17:30.482 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:30.478+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:30.480+0000 7f081aa00640 1 -- 172.21.15.26:0/3361585915 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f081403a070 con 0x7f07fc06c550 2024-01-15T18:17:30.482 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:30.494 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:32.495 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:32.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.730+0000 7f6b5ca00640 1 Processor -- start 2024-01-15T18:17:32.732 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.730+0000 7f6b5ca00640 1 -- start start 2024-01-15T18:17:32.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.731+0000 7f6b5ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b5815afb0 0x7f6b5813f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:32.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.732+0000 7f6b5ca00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b5813ff00 con 0x7f6b5813fae0 2024-01-15T18:17:32.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.732+0000 7f6b5ca00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b580cdc40 con 0x7f6b5813cfa0 2024-01-15T18:17:32.734 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.732+0000 7f6b5ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580cdec0 con 0x7f6b5815afb0 2024-01-15T18:17:32.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.732+0000 7f6b57e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6b5813fae0 0x7f6b580b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41582/0 (socket says 172.21.15.26:41582) 2024-01-15T18:17:32.735 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.732+0000 7f6b57e00640 1 -- 172.21.15.26:0/609440679 learned_addr learned my addr 172.21.15.26:0/609440679 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.732+0000 7f6b4fe00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b5815afb0 0x7f6b5813f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.732+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323175621 0 0) 0x7f6b5813ff00 con 0x7f6b5813fae0 2024-01-15T18:17:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.732+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b38003520 con 0x7f6b5813fae0 2024-01-15T18:17:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1725431026 0 0) 0x7f6b38003520 con 0x7f6b5813fae0 2024-01-15T18:17:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b5815afb0 msgr2=0x7f6b5813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b5815afb0 0x7f6b5813f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.737 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 >> v1:172.21.15.124:6789/0 conn(0x7f6b5813cfa0 legacy=0x7f6b580cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b580ce1a0 con 0x7f6b5813fae0 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f6b500019f0 con 0x7f6b5813fae0 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 >> v1:172.21.15.26:6789/0 conn(0x7f6b5813fae0 legacy=0x7f6b580b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b380039a0 0x7f6b38003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b5815afb0 0x7f6b380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b38004d00 0x7f6b38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b580cdec0 con 0x7f6b38004d00 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3800dd50 con 0x7f6b380039a0 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3800df10 con 0x7f6b5815afb0 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.733+0000 7f6b57e00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b38004d00 0x7f6b38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.734+0000 7f6b57400640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b380039a0 0x7f6b38003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.734+0000 7f6b4fe00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b5815afb0 0x7f6b380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.734+0000 7f6b57e00640 1 -- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b5815afb0 msgr2=0x7f6b380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:32.738 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.734+0000 7f6b57e00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b5815afb0 0x7f6b380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.734+0000 7f6b57e00640 1 -- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b380039a0 msgr2=0x7f6b38003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.734+0000 7f6b57e00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b380039a0 0x7f6b38003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.734+0000 7f6b57e00640 1 -- 172.21.15.26:0/609440679 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b38003520 con 0x7f6b38004d00 2024-01-15T18:17:32.739 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.735+0000 7f6b57e00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b38004d00 0x7f6b38005180 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000a750 tx=0x7f6b380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7cb54d6ad958d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:32.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.735+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50034630 con 0x7f6b38004d00 2024-01-15T18:17:32.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.735+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5002f070 con 0x7f6b38004d00 2024-01-15T18:17:32.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.735+0000 7f6b56a00640 1 -- 172.21.15.26:0/609440679 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b500349f0 con 0x7f6b38004d00 2024-01-15T18:17:32.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.736+0000 7f6b5ca00640 1 -- 172.21.15.26:0/609440679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b38004d00 msgr2=0x7f6b38005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:32.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.736+0000 7f6b5ca00640 1 --2- 172.21.15.26:0/609440679 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b38004d00 0x7f6b38005180 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000a750 tx=0x7f6b380057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.736+0000 7f6b57e00640 1 -- 172.21.15.26:0/609440679 reap_dead start 2024-01-15T18:17:32.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.736+0000 7f6b5ca00640 1 -- 172.21.15.26:0/609440679 shutdown_connections 2024-01-15T18:17:32.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.736+0000 7f6b5ca00640 1 -- 172.21.15.26:0/609440679 >> 172.21.15.26:0/609440679 conn(0x7f6b5806a800 msgr2=0x7f6b5806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:32.740 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.736+0000 7f6b5ca00640 1 -- 172.21.15.26:0/609440679 shutdown_connections 2024-01-15T18:17:32.741 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.736+0000 7f6b5ca00640 1 -- 172.21.15.26:0/609440679 wait complete. 2024-01-15T18:17:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.738+0000 7f6b5ca00640 1 Processor -- start 2024-01-15T18:17:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.738+0000 7f6b5ca00640 1 -- start start 2024-01-15T18:17:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.739+0000 7f6b5ca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b38004d00 0x7f6b580afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.739+0000 7f6b5ca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b581f7520 0x7f6b581f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.739+0000 7f6b5ca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b58153810 0x7f6b58153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.739+0000 7f6b5ca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581541d0 con 0x7f6b581f7520 2024-01-15T18:17:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.739+0000 7f6b5ca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581e3a60 con 0x7f6b58153810 2024-01-15T18:17:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.739+0000 7f6b5ca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b581f7fe0 con 0x7f6b38004d00 2024-01-15T18:17:32.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.739+0000 7f6b4fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b581f7520 0x7f6b581f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.739+0000 7f6b4fe00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b581f7520 0x7f6b581f79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48816/0 (socket says 172.21.15.26:48816) 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.739+0000 7f6b4fe00640 1 -- 172.21.15.26:0/125631228 learned_addr learned my addr 172.21.15.26:0/125631228 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b57e00640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b58153810 0x7f6b58153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b57400640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b38004d00 0x7f6b580afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b4fe00640 1 -- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b38004d00 msgr2=0x7f6b580afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b4fe00640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b38004d00 0x7f6b580afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b4fe00640 1 -- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b58153810 msgr2=0x7f6b58153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b4fe00640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b58153810 0x7f6b58153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b4fe00640 1 -- 172.21.15.26:0/125631228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b581f82c0 con 0x7f6b581f7520 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b57e00640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6b58153810 0x7f6b58153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b57400640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6b38004d00 0x7f6b580afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:32.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b4fe00640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6b581f7520 0x7f6b581f79a0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f6b40011600 tx=0x7f6b40011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.740+0000 7f6b4f400640 1 -- 172.21.15.26:0/125631228 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4000f920 con 0x7f6b581f7520 2024-01-15T18:17:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.741+0000 7f6b4f400640 1 -- 172.21.15.26:0/125631228 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4000fa80 con 0x7f6b581f7520 2024-01-15T18:17:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.741+0000 7f6b4f400640 1 -- 172.21.15.26:0/125631228 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4000fdf0 con 0x7f6b581f7520 2024-01-15T18:17:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.741+0000 7f6b5ca00640 1 -- 172.21.15.26:0/125631228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b581e3d20 con 0x7f6b581f7520 2024-01-15T18:17:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.741+0000 7f6b5ca00640 1 -- 172.21.15.26:0/125631228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b580b00a0 con 0x7f6b581f7520 2024-01-15T18:17:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.742+0000 7f6b5ca00640 1 -- 172.21.15.26:0/125631228 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b580b1a00 con 0x7f6b581f7520 2024-01-15T18:17:32.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.743+0000 7f6b4f400640 1 -- 172.21.15.26:0/125631228 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f6b40015460 con 0x7f6b581f7520 2024-01-15T18:17:32.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.744+0000 7f6b4f400640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b3006c550 0x7f6b3006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:32.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.744+0000 7f6b4f400640 1 -- 172.21.15.26:0/125631228 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f6b40089010 con 0x7f6b581f7520 2024-01-15T18:17:32.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.745+0000 7f6b57400640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b3006c550 0x7f6b3006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:32.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.746+0000 7f6b57400640 1 --2- 172.21.15.26:0/125631228 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6b3006c550 0x7f6b3006ea10 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6b480016b0 tx=0x7f6b48009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:32.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.750+0000 7f6b4f400640 1 -- 172.21.15.26:0/125631228 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b400555b0 con 0x7f6b581f7520 2024-01-15T18:17:32.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.970+0000 7f6b5ca00640 1 -- 172.21.15.26:0/125631228 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "force": true, "vol_name": "cephfs", "sub_name": "subvol_clone1113", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b581e38e0 con 0x7f6b3006c550 2024-01-15T18:17:32.992 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:17:32.989+0000 7f8f54600640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:32.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:32.990+0000 7f6b4f400640 1 -- 172.21.15.26:0/125631228 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: -11 subvol_clone1113 clone in-progress -- please cancel the clone and retry) v1 ==== 79+0+0 (secure 0 0 0) 0x7f6b4800d050 con 0x7f6b3006c550 2024-01-15T18:17:32.993 INFO:teuthology.orchestra.run.smithi026.stderr:Error EAGAIN: subvol_clone1113 clone in-progress -- please cancel the clone and retry 2024-01-15T18:17:33.007 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-01-15T18:17:33.011 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the ... ERROR 2024-01-15T18:17:33.011 INFO:tasks.cephfs_test_runner:ERROR 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner: 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner:ERROR: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 8163, in test_when_3_clones_are_cancelled 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner: self._cancel_clones_and_assert(c) 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7984, in _cancel_clones_and_assert 2024-01-15T18:17:33.012 INFO:tasks.cephfs_test_runner: self._cancel_clones(clones, check_status=True) 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7965, in _cancel_clones 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: self.run_ceph_cmd(f'fs clone cancel {v} {i}', 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/remote.py", line 523, in run 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 455, in run 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: r.wait() 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi026 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone1112' 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner: 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-15T18:17:33.013 INFO:tasks.cephfs_test_runner:ERROR: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7588, in tearDown 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner: self._loop_cmd(f'fs subvolume rm --force {v} {sv}') 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7566, in _loop_cmd 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner: raise RuntimeError(f'This cmd failed 50 times: {cmdargs}') 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner:RuntimeError: This cmd failed 50 times: fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner: 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner:Ran 7 tests in 1306.890s 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner: 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner: 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-15T18:17:33.014 INFO:tasks.cephfs_test_runner:ERROR: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 8163, in test_when_3_clones_are_cancelled 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: self._cancel_clones_and_assert(c) 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7984, in _cancel_clones_and_assert 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: self._cancel_clones(clones, check_status=True) 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7965, in _cancel_clones 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: self.run_ceph_cmd(f'fs clone cancel {v} {i}', 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/ceph_manager.py", line 1602, in run_cluster_cmd 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/remote.py", line 523, in run 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-01-15T18:17:33.015 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 455, in run 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: r.wait() 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi026 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone cancel cephfs subvol_clone1112' 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner:====================================================================== 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner:ERROR: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner:Test that the progress bars for ongoing clones is removed from the 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7588, in tearDown 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: self._loop_cmd(f'fs subvolume rm --force {v} {sv}') 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs/test_volumes.py", line 7566, in _loop_cmd 2024-01-15T18:17:33.016 INFO:tasks.cephfs_test_runner: raise RuntimeError(f'This cmd failed 50 times: {cmdargs}') 2024-01-15T18:17:33.017 INFO:tasks.cephfs_test_runner:RuntimeError: This cmd failed 50 times: fs subvolume rm --force cephfs subvol_clone1113 2024-01-15T18:17:33.017 INFO:tasks.cephfs_test_runner: 2024-01-15T18:17:33.017 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress), test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:17:33.211 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=51c3b4f9295244b5a5dad8c6e91fda27 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_95c90f413c4bb65e3e56909855f2308fd9bda05b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress), test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress) 2024-01-15T18:17:33.212 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-01-15T18:17:33.223 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-01-15T18:17:33.231 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-01-15T18:17:33.231 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:17:33.231 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:33.256 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:17:33.299 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:17:33.300 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:17:33.301 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:33.388 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.675+0000 7fd926000640 1 Processor -- start 2024-01-15T18:17:33.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.675+0000 7fd926000640 1 -- start start 2024-01-15T18:17:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.676+0000 7fd926000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fd92013cfa0 0x7fd92015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.677+0000 7fd926000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fd9200cd530 0x7fd92013f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.677+0000 7fd926000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9200cdec0 con 0x7fd9200cd9e0 2024-01-15T18:17:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.677+0000 7fd926000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9200ce1a0 con 0x7fd92013cfa0 2024-01-15T18:17:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.677+0000 7fd926000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9200ce480 con 0x7fd9200cd530 2024-01-15T18:17:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.677+0000 7fd925600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd9200cd9e0 0x7fd92015d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41584/0 (socket says 172.21.15.26:41584) 2024-01-15T18:17:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.677+0000 7fd925600640 1 -- 172.21.15.26:0/1252942174 learned_addr learned my addr 172.21.15.26:0/1252942174 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:33.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.677+0000 7fd924c00640 1 --2- 172.21.15.26:0/1252942174 >> v2:172.21.15.26:3300/0 conn(0x7fd92013cfa0 0x7fd92015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.677+0000 7fd917e00640 1 --2- 172.21.15.26:0/1252942174 >> v2:172.21.15.124:3300/0 conn(0x7fd9200cd530 0x7fd92013f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.677+0000 7fd91fe00640 1 -- 172.21.15.26:0/1252942174 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685719399 0 0) 0x7fd9200cdec0 con 0x7fd9200cd9e0 2024-01-15T18:17:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.678+0000 7fd91fe00640 1 -- 172.21.15.26:0/1252942174 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd900003520 con 0x7fd9200cd9e0 2024-01-15T18:17:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.678+0000 7fd924c00640 1 -- 172.21.15.26:0/1252942174 >> v2:172.21.15.124:3300/0 conn(0x7fd9200cd530 msgr2=0x7fd92013f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.678+0000 7fd924c00640 1 --2- 172.21.15.26:0/1252942174 >> v2:172.21.15.124:3300/0 conn(0x7fd9200cd530 0x7fd92013f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.678+0000 7fd924c00640 1 -- 172.21.15.26:0/1252942174 >> v1:172.21.15.26:6789/0 conn(0x7fd9200cd9e0 legacy=0x7fd92015d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.678+0000 7fd924c00640 1 -- 172.21.15.26:0/1252942174 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200ce760 con 0x7fd92013cfa0 2024-01-15T18:17:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.679+0000 7fd924c00640 1 --2- 172.21.15.26:0/1252942174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92013cfa0 0x7fd92015afb0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fd92015b4f0 tx=0x7fd91002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b1fc103b700592 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:33.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.679+0000 7fd91fe00640 1 -- 172.21.15.26:0/1252942174 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91002e070 con 0x7fd92013cfa0 2024-01-15T18:17:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.679+0000 7fd91fe00640 1 -- 172.21.15.26:0/1252942174 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100315a0 con 0x7fd92013cfa0 2024-01-15T18:17:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.679+0000 7fd91fe00640 1 -- 172.21.15.26:0/1252942174 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910031930 con 0x7fd92013cfa0 2024-01-15T18:17:33.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.680+0000 7fd926000640 1 -- 172.21.15.26:0/1252942174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92013cfa0 msgr2=0x7fd92015afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.680+0000 7fd926000640 1 --2- 172.21.15.26:0/1252942174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92013cfa0 0x7fd92015afb0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7fd92015b4f0 tx=0x7fd91002b840 comp rx=0 tx=0).stop 2024-01-15T18:17:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.680+0000 7fd926000640 1 -- 172.21.15.26:0/1252942174 shutdown_connections 2024-01-15T18:17:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.680+0000 7fd926000640 1 --2- 172.21.15.26:0/1252942174 >> v2:172.21.15.124:3300/0 conn(0x7fd9200cd530 0x7fd92013f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.680+0000 7fd926000640 1 --2- 172.21.15.26:0/1252942174 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd92013cfa0 0x7fd92015afb0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.680+0000 7fd926000640 1 -- 172.21.15.26:0/1252942174 >> 172.21.15.26:0/1252942174 conn(0x7fd92006a800 msgr2=0x7fd92006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.680+0000 7fd926000640 1 -- 172.21.15.26:0/1252942174 shutdown_connections 2024-01-15T18:17:33.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.680+0000 7fd926000640 1 -- 172.21.15.26:0/1252942174 wait complete. 2024-01-15T18:17:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.682+0000 7fd926000640 1 Processor -- start 2024-01-15T18:17:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.682+0000 7fd926000640 1 -- start start 2024-01-15T18:17:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.683+0000 7fd926000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9200cd530 0x7fd92013c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.683+0000 7fd926000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9200cd9e0 0x7fd9201513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.683+0000 7fd926000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd92013cfa0 0x7fd92015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.683+0000 7fd926000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd920152940 con 0x7fd9200cd530 2024-01-15T18:17:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.683+0000 7fd926000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92013cb50 con 0x7fd9200cd9e0 2024-01-15T18:17:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.683+0000 7fd926000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd920151cc0 con 0x7fd92013cfa0 2024-01-15T18:17:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.683+0000 7fd924c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9200cd530 0x7fd92013c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.683+0000 7fd924c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9200cd530 0x7fd92013c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48834/0 (socket says 172.21.15.26:48834) 2024-01-15T18:17:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.683+0000 7fd924c00640 1 -- 172.21.15.26:0/1595441953 learned_addr learned my addr 172.21.15.26:0/1595441953 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:33.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd924c00640 1 -- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd92013cfa0 msgr2=0x7fd92015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd924c00640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd92013cfa0 0x7fd92015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd924c00640 1 -- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9200cd9e0 msgr2=0x7fd9201513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd924c00640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9200cd9e0 0x7fd9201513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd924c00640 1 -- 172.21.15.26:0/1595441953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910009400 con 0x7fd9200cd530 2024-01-15T18:17:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd924c00640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9200cd530 0x7fd92013c610 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fd910009890 tx=0x7fd9100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd91d600640 1 -- 172.21.15.26:0/1595441953 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910032840 con 0x7fd9200cd530 2024-01-15T18:17:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd91d600640 1 -- 172.21.15.26:0/1595441953 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd910031f00 con 0x7fd9200cd530 2024-01-15T18:17:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd91d600640 1 -- 172.21.15.26:0/1595441953 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910032be0 con 0x7fd9200cd530 2024-01-15T18:17:33.689 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.684+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd920151fa0 con 0x7fd9200cd530 2024-01-15T18:17:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.685+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd920152560 con 0x7fd9200cd530 2024-01-15T18:17:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.686+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd920152a80 con 0x7fd9200cd530 2024-01-15T18:17:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.687+0000 7fd91d600640 1 -- 172.21.15.26:0/1595441953 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fd910032d40 con 0x7fd9200cd530 2024-01-15T18:17:33.690 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.687+0000 7fd91d600640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd8f806c550 0x7fd8f806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.688+0000 7fd91d600640 1 -- 172.21.15.26:0/1595441953 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fd9100aa320 con 0x7fd9200cd530 2024-01-15T18:17:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.688+0000 7fd917e00640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd8f806c550 0x7fd8f806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:33.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.689+0000 7fd917e00640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd8f806c550 0x7fd8f806ea10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd9200b0150 tx=0x7fd908008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:33.694 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.691+0000 7fd91d600640 1 -- 172.21.15.26:0/1595441953 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd920152a80 con 0x7fd9200cd530 2024-01-15T18:17:33.889 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.887+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd920152a80 con 0x7fd9200cd530 2024-01-15T18:17:33.892 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.890+0000 7fd91d600640 1 -- 172.21.15.26:0/1595441953 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16729 (secure 0 0 0) 0x7fd910076c40 con 0x7fd9200cd530 2024-01-15T18:17:33.893 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:17:33.893 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":117,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:12:53.309154+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:12:08.696115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:12:09.497434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.30","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:33.895 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd8f806c550 msgr2=0x7fd8f806ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd8f806c550 0x7fd8f806ea10 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd9200b0150 tx=0x7fd908008040 comp rx=0 tx=0).stop 2024-01-15T18:17:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9200cd530 msgr2=0x7fd92013c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9200cd530 0x7fd92013c610 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fd910009890 tx=0x7fd9100046c0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 shutdown_connections 2024-01-15T18:17:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd92013cfa0 0x7fd92015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd8f806c550 0x7fd8f806ea10 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd9200cd9e0 0x7fd9201513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 --2- 172.21.15.26:0/1595441953 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd9200cd530 0x7fd92013c610 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 >> 172.21.15.26:0/1595441953 conn(0x7fd92006a800 msgr2=0x7fd9200b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 shutdown_connections 2024-01-15T18:17:33.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:33.893+0000 7fd926000640 1 -- 172.21.15.26:0/1595441953 wait complete. 2024-01-15T18:17:33.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:12:08.696115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:12:09.497434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:17:33.915 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.131+0000 7f1906400640 1 Processor -- start 2024-01-15T18:17:34.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.132+0000 7f1906400640 1 -- start start 2024-01-15T18:17:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.133+0000 7f1906400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f190013df80 0x7f190015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.133+0000 7f1906400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f190013e360 0x7f19000cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.133+0000 7f1906400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190006c470 con 0x7f190015f9c0 2024-01-15T18:17:34.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.134+0000 7f1906400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190006c750 con 0x7f190013df80 2024-01-15T18:17:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.134+0000 7f1906400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190006ca30 con 0x7f190013e360 2024-01-15T18:17:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.134+0000 7f18ffe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f190015f9c0 0x7f19001408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41588/0 (socket says 172.21.15.26:41588) 2024-01-15T18:17:34.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.134+0000 7f18ffe00640 1 -- 172.21.15.26:0/1892422704 learned_addr learned my addr 172.21.15.26:0/1892422704 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.134+0000 7f18ff400640 1 --2- 172.21.15.26:0/1892422704 >> v2:172.21.15.124:3300/0 conn(0x7f190013df80 0x7f190015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.134+0000 7f18fea00640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f190013e360 0x7f19000cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.134+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445738667 0 0) 0x7f190006c470 con 0x7f190015f9c0 2024-01-15T18:17:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18e0003520 con 0x7f190015f9c0 2024-01-15T18:17:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 853362740 0 0) 0x7f18e0003520 con 0x7f190015f9c0 2024-01-15T18:17:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f190013e360 msgr2=0x7f19000cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f190013e360 0x7f19000cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 >> v2:172.21.15.124:3300/0 conn(0x7f190013df80 msgr2=0x7f190015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 --2- 172.21.15.26:0/1892422704 >> v2:172.21.15.124:3300/0 conn(0x7f190013df80 0x7f190015bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190006cd10 con 0x7f190015f9c0 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f18f8001670 con 0x7f190015f9c0 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 >> v1:172.21.15.26:6789/0 conn(0x7f190015f9c0 legacy=0x7f19001408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18e0003970 0x7f18e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e00047c0 0x7f18e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f190013e360 0x7f18e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190006c750 con 0x7f18e0003970 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190006ca30 con 0x7f18e00047c0 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e000b980 con 0x7f190013e360 2024-01-15T18:17:34.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.135+0000 7f18ff400640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18e0003970 0x7f18e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.136+0000 7f18fea00640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e00047c0 0x7f18e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.136+0000 7f18ffe00640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f190013e360 0x7f18e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.136+0000 7f18ff400640 1 -- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f190013e360 msgr2=0x7f18e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.136+0000 7f18ff400640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f190013e360 0x7f18e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.136+0000 7f18ff400640 1 -- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e00047c0 msgr2=0x7f18e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.136+0000 7f18ff400640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e00047c0 0x7f18e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.136+0000 7f18ff400640 1 -- 172.21.15.26:0/1892422704 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e0003520 con 0x7f18e0003970 2024-01-15T18:17:34.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.136+0000 7f18ffe00640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f190013e360 0x7f18e0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.136+0000 7f18fea00640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e00047c0 0x7f18e0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:34.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.137+0000 7f18ff400640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18e0003970 0x7f18e0003d90 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009fc0 tx=0x7f18e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a8ba411c1664c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.137+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f002cc60 con 0x7f18e0003970 2024-01-15T18:17:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.138+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f000f040 con 0x7f18e0003970 2024-01-15T18:17:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.138+0000 7f18fe000640 1 -- 172.21.15.26:0/1892422704 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0034440 con 0x7f18e0003970 2024-01-15T18:17:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.138+0000 7f1906400640 1 -- 172.21.15.26:0/1892422704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18e0003970 msgr2=0x7f18e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.138+0000 7f1906400640 1 --2- 172.21.15.26:0/1892422704 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18e0003970 0x7f18e0003d90 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009fc0 tx=0x7f18e00057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.138+0000 7f18ffe00640 1 -- 172.21.15.26:0/1892422704 reap_dead start 2024-01-15T18:17:34.146 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.138+0000 7f1906400640 1 -- 172.21.15.26:0/1892422704 shutdown_connections 2024-01-15T18:17:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.138+0000 7f1906400640 1 -- 172.21.15.26:0/1892422704 >> 172.21.15.26:0/1892422704 conn(0x7f190006a800 msgr2=0x7f190006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.138+0000 7f1906400640 1 -- 172.21.15.26:0/1892422704 shutdown_connections 2024-01-15T18:17:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.138+0000 7f1906400640 1 -- 172.21.15.26:0/1892422704 wait complete. 2024-01-15T18:17:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.140+0000 7f1906400640 1 Processor -- start 2024-01-15T18:17:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.140+0000 7f1906400640 1 -- start start 2024-01-15T18:17:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.141+0000 7f1906400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e0003970 0x7f19001f0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.141+0000 7f1906400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f190014baf0 0x7f19000b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.141+0000 7f1906400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b36f0 0x7f19001f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.141+0000 7f1906400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1900150020 con 0x7f190014baf0 2024-01-15T18:17:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.141+0000 7f1906400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190013a300 con 0x7f18e0003970 2024-01-15T18:17:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.141+0000 7f1906400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19000b3b70 con 0x7f19000b36f0 2024-01-15T18:17:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18fea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f190014baf0 0x7f19000b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18fea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f190014baf0 0x7f19000b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48854/0 (socket says 172.21.15.26:48854) 2024-01-15T18:17:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18fea00640 1 -- 172.21.15.26:0/3928915431 learned_addr learned my addr 172.21.15.26:0/3928915431 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18ff400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e0003970 0x7f19001f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18ffe00640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b36f0 0x7f19001f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18fea00640 1 -- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b36f0 msgr2=0x7f19001f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18fea00640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b36f0 0x7f19001f3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18fea00640 1 -- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e0003970 msgr2=0x7f19001f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18fea00640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e0003970 0x7f19001f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18fea00640 1 -- 172.21.15.26:0/3928915431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190014eca0 con 0x7f190014baf0 2024-01-15T18:17:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.142+0000 7f18fea00640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f190014baf0 0x7f19000b31b0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f18e80108b0 tx=0x7f18e8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.143+0000 7f18f7400640 1 -- 172.21.15.26:0/3928915431 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e800eb60 con 0x7f190014baf0 2024-01-15T18:17:34.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.143+0000 7f18f7400640 1 -- 172.21.15.26:0/3928915431 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e800ecc0 con 0x7f190014baf0 2024-01-15T18:17:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.143+0000 7f18f7400640 1 -- 172.21.15.26:0/3928915431 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e801c400 con 0x7f190014baf0 2024-01-15T18:17:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.143+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190014ef80 con 0x7f190014baf0 2024-01-15T18:17:34.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.143+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190014f540 con 0x7f190014baf0 2024-01-15T18:17:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.144+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19000675d0 con 0x7f190014baf0 2024-01-15T18:17:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.145+0000 7f18f7400640 1 -- 172.21.15.26:0/3928915431 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f18e801c560 con 0x7f190014baf0 2024-01-15T18:17:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.147+0000 7f18f7400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18d806c580 0x7f18d806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.147+0000 7f18f7400640 1 -- 172.21.15.26:0/3928915431 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f18e8088db0 con 0x7f190014baf0 2024-01-15T18:17:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.147+0000 7f18ff400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18d806c580 0x7f18d806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.151 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.149+0000 7f18ff400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18d806c580 0x7f18d806ea40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f18f0036000 tx=0x7f18f002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.152+0000 7f18f7400640 1 -- 172.21.15.26:0/3928915431 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18e8055780 con 0x7f190014baf0 2024-01-15T18:17:34.351 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.349+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f190006df00 con 0x7f190014baf0 2024-01-15T18:17:34.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.349+0000 7f18f7400640 1 -- 172.21.15.26:0/3928915431 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18e8021020 con 0x7f190014baf0 2024-01-15T18:17:34.352 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:17:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18d806c580 msgr2=0x7f18d806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18d806c580 0x7f18d806ea40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f18f0036000 tx=0x7f18f002e000 comp rx=0 tx=0).stop 2024-01-15T18:17:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f190014baf0 msgr2=0x7f19000b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f190014baf0 0x7f19000b31b0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f18e80108b0 tx=0x7f18e8010d70 comp rx=0 tx=0).stop 2024-01-15T18:17:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 shutdown_connections 2024-01-15T18:17:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f19000b36f0 0x7f19001f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f18d806c580 0x7f18d806ea40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f190014baf0 0x7f19000b31b0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 --2- 172.21.15.26:0/3928915431 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18e0003970 0x7f19001f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 >> 172.21.15.26:0/3928915431 conn(0x7f190006a800 msgr2=0x7f190015e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 shutdown_connections 2024-01-15T18:17:34.356 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.352+0000 7f1906400640 1 -- 172.21.15.26:0/3928915431 wait complete. 2024-01-15T18:17:34.374 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:17:34.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.596+0000 7ff05b200640 1 Processor -- start 2024-01-15T18:17:34.599 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.596+0000 7ff05b200640 1 -- start start 2024-01-15T18:17:34.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.598+0000 7ff05b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7ff05c1505b0 0x7ff05c0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.598+0000 7ff05b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff05c06d6d0 0x7ff05c0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.598+0000 7ff05b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05c0b1e30 con 0x7ff05c15a690 2024-01-15T18:17:34.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.598+0000 7ff05b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05c15ab20 con 0x7ff05c1505b0 2024-01-15T18:17:34.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.598+0000 7ff05b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0b20a0 con 0x7ff05c06d6d0 2024-01-15T18:17:34.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.598+0000 7ff053e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff05c15a690 0x7ff05c06cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41600/0 (socket says 172.21.15.26:41600) 2024-01-15T18:17:34.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.598+0000 7ff053e00640 1 -- 172.21.15.26:0/281375698 learned_addr learned my addr 172.21.15.26:0/281375698 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:34.601 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.598+0000 7ff059e00640 1 --2- 172.21.15.26:0/281375698 >> v2:172.21.15.26:3300/0 conn(0x7ff05c1505b0 0x7ff05c0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.599+0000 7ff05a800640 1 --2- 172.21.15.26:0/281375698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff05c06d6d0 0x7ff05c0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.599+0000 7ff059e00640 1 -- 172.21.15.26:0/281375698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff05c06d6d0 msgr2=0x7ff05c0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:34.602 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.599+0000 7ff059e00640 1 --2- 172.21.15.26:0/281375698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff05c06d6d0 0x7ff05c0ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.599+0000 7ff059e00640 1 -- 172.21.15.26:0/281375698 >> v1:172.21.15.26:6789/0 conn(0x7ff05c15a690 legacy=0x7ff05c06cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.599+0000 7ff059e00640 1 -- 172.21.15.26:0/281375698 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c0b2380 con 0x7ff05c1505b0 2024-01-15T18:17:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.599+0000 7ff059400640 1 -- 172.21.15.26:0/281375698 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696201683 0 0) 0x7ff05c0b1e30 con 0x7ff05c15a690 2024-01-15T18:17:34.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.600+0000 7ff059e00640 1 --2- 172.21.15.26:0/281375698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c1505b0 0x7ff05c0b3b20 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0097a0 tx=0x7ff04c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7156a08f173552d6 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.600+0000 7ff059400640 1 -- 172.21.15.26:0/281375698 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c02d040 con 0x7ff05c1505b0 2024-01-15T18:17:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.600+0000 7ff059400640 1 -- 172.21.15.26:0/281375698 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c031810 con 0x7ff05c1505b0 2024-01-15T18:17:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.600+0000 7ff059400640 1 -- 172.21.15.26:0/281375698 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c031ba0 con 0x7ff05c1505b0 2024-01-15T18:17:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.601+0000 7ff05b200640 1 -- 172.21.15.26:0/281375698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c1505b0 msgr2=0x7ff05c0b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.604 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.601+0000 7ff05b200640 1 --2- 172.21.15.26:0/281375698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c1505b0 0x7ff05c0b3b20 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7ff04c0097a0 tx=0x7ff04c02b8b0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.601+0000 7ff05b200640 1 -- 172.21.15.26:0/281375698 shutdown_connections 2024-01-15T18:17:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.601+0000 7ff05b200640 1 --2- 172.21.15.26:0/281375698 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff05c06d6d0 0x7ff05c0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.601+0000 7ff05b200640 1 --2- 172.21.15.26:0/281375698 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c1505b0 0x7ff05c0b3b20 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.601+0000 7ff05b200640 1 -- 172.21.15.26:0/281375698 >> 172.21.15.26:0/281375698 conn(0x7ff05c06a7d0 msgr2=0x7ff05c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.601+0000 7ff05b200640 1 -- 172.21.15.26:0/281375698 shutdown_connections 2024-01-15T18:17:34.605 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.601+0000 7ff05b200640 1 -- 172.21.15.26:0/281375698 wait complete. 2024-01-15T18:17:34.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.603+0000 7ff05b200640 1 Processor -- start 2024-01-15T18:17:34.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.603+0000 7ff05b200640 1 -- start start 2024-01-15T18:17:34.606 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.604+0000 7ff05b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c06d6d0 0x7ff05c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.604+0000 7ff05b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff05c1505b0 0x7ff05c15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.604+0000 7ff05b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff05c15a690 0x7ff05c1f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.604+0000 7ff05b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c1f3340 con 0x7ff05c06d6d0 2024-01-15T18:17:34.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.604+0000 7ff05b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c1f7430 con 0x7ff05c1505b0 2024-01-15T18:17:34.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.604+0000 7ff05b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c1f7150 con 0x7ff05c15a690 2024-01-15T18:17:34.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.604+0000 7ff059e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c06d6d0 0x7ff05c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.604+0000 7ff059e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c06d6d0 0x7ff05c153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48870/0 (socket says 172.21.15.26:48870) 2024-01-15T18:17:34.607 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.604+0000 7ff059e00640 1 -- 172.21.15.26:0/262579652 learned_addr learned my addr 172.21.15.26:0/262579652 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:34.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.605+0000 7ff053e00640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff05c1505b0 0x7ff05c15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.605+0000 7ff059e00640 1 -- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff05c15a690 msgr2=0x7ff05c1f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.605+0000 7ff059e00640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff05c15a690 0x7ff05c1f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.605+0000 7ff059e00640 1 -- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff05c1505b0 msgr2=0x7ff05c15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.605+0000 7ff059e00640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff05c1505b0 0x7ff05c15d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.605+0000 7ff059e00640 1 -- 172.21.15.26:0/262579652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c009400 con 0x7ff05c06d6d0 2024-01-15T18:17:34.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.605+0000 7ff059e00640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c06d6d0 0x7ff05c153750 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff04c000c00 tx=0x7ff04c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.609 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.605+0000 7ff052000640 1 -- 172.21.15.26:0/262579652 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c03ba60 con 0x7ff05c06d6d0 2024-01-15T18:17:34.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.605+0000 7ff052000640 1 -- 172.21.15.26:0/262579652 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04c03bbc0 con 0x7ff05c06d6d0 2024-01-15T18:17:34.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.606+0000 7ff052000640 1 -- 172.21.15.26:0/262579652 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04c004030 con 0x7ff05c06d6d0 2024-01-15T18:17:34.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.606+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c152c70 con 0x7ff05c06d6d0 2024-01-15T18:17:34.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.606+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c153230 con 0x7ff05c06d6d0 2024-01-15T18:17:34.610 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.607+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c067450 con 0x7ff05c06d6d0 2024-01-15T18:17:34.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.608+0000 7ff052000640 1 -- 172.21.15.26:0/262579652 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7ff04c043070 con 0x7ff05c06d6d0 2024-01-15T18:17:34.611 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.609+0000 7ff052000640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff03406c550 0x7ff03406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:34.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.609+0000 7ff052000640 1 -- 172.21.15.26:0/262579652 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7ff04c0aa5a0 con 0x7ff05c06d6d0 2024-01-15T18:17:34.612 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.610+0000 7ff053e00640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff03406c550 0x7ff03406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:34.613 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.611+0000 7ff053e00640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff03406c550 0x7ff03406ea10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff05c14fed0 tx=0x7ff044009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:34.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.614+0000 7ff052000640 1 -- 172.21.15.26:0/262579652 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff04c076ec0 con 0x7ff05c06d6d0 2024-01-15T18:17:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.809+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff05c0b3660 con 0x7ff05c06d6d0 2024-01-15T18:17:34.812 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.810+0000 7ff052000640 1 -- 172.21.15.26:0/262579652 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff04c0350a0 con 0x7ff05c06d6d0 2024-01-15T18:17:34.813 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T18:17:34.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff03406c550 msgr2=0x7ff03406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff03406c550 0x7ff03406ea10 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff05c14fed0 tx=0x7ff044009040 comp rx=0 tx=0).stop 2024-01-15T18:17:34.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c06d6d0 msgr2=0x7ff05c153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:34.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c06d6d0 0x7ff05c153750 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7ff04c000c00 tx=0x7ff04c0046c0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 shutdown_connections 2024-01-15T18:17:34.816 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff05c15a690 0x7ff05c1f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff03406c550 0x7ff03406ea10 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff05c1505b0 0x7ff05c15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 --2- 172.21.15.26:0/262579652 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff05c06d6d0 0x7ff05c153750 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 >> 172.21.15.26:0/262579652 conn(0x7ff05c06a7d0 msgr2=0x7ff05c14f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.813+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 shutdown_connections 2024-01-15T18:17:34.817 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:34.814+0000 7ff05b200640 1 -- 172.21.15.26:0/262579652 wait complete. 2024-01-15T18:17:34.836 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:17:35.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.060+0000 7f2fa9800640 1 Processor -- start 2024-01-15T18:17:35.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.060+0000 7f2fa9800640 1 -- start start 2024-01-15T18:17:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.061+0000 7f2fa9800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa4070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.061+0000 7f2fa9800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2fa415bf10 0x7f2fa40cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.061+0000 7f2fa9800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f2fa40cc980 0x7f2fa4143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.061+0000 7f2fa9800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa415f970 con 0x7f2fa415bf10 2024-01-15T18:17:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.061+0000 7f2fa9800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa415f4e0 con 0x7f2fa40cc980 2024-01-15T18:17:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.061+0000 7f2fa9800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa40cce10 con 0x7f2fa40adf20 2024-01-15T18:17:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2fa415bf10 0x7f2fa40cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2fa415bf10 0x7f2fa40cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48884/0 (socket says 172.21.15.26:48884) 2024-01-15T18:17:35.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3400640 1 -- 172.21.15.26:0/1320897486 learned_addr learned my addr 172.21.15.26:0/1320897486 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa8e00640 1 --2- 172.21.15.26:0/1320897486 >> v2:172.21.15.124:3300/0 conn(0x7f2fa40cc980 0x7f2fa4143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3400640 1 -- 172.21.15.26:0/1320897486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 msgr2=0x7f2fa4070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3e00640 1 --2- 172.21.15.26:0/1320897486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa4070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3400640 1 --2- 172.21.15.26:0/1320897486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa4070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3400640 1 -- 172.21.15.26:0/1320897486 >> v2:172.21.15.124:3300/0 conn(0x7f2fa40cc980 msgr2=0x7f2fa4143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3400640 1 --2- 172.21.15.26:0/1320897486 >> v2:172.21.15.124:3300/0 conn(0x7f2fa40cc980 0x7f2fa4143d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3400640 1 -- 172.21.15.26:0/1320897486 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa4071130 con 0x7f2fa415bf10 2024-01-15T18:17:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa8e00640 1 --2- 172.21.15.26:0/1320897486 >> v2:172.21.15.124:3300/0 conn(0x7f2fa40cc980 0x7f2fa4143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.062+0000 7f2fa3e00640 1 --2- 172.21.15.26:0/1320897486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa4070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.064+0000 7f2fa3400640 1 --2- 172.21.15.26:0/1320897486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 0x7f2fa40cc440 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f2fa406e490 tx=0x7f2f8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946ae3861955ba84 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.064+0000 7f2fa2a00640 1 -- 172.21.15.26:0/1320897486 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c0317e0 con 0x7f2fa415bf10 2024-01-15T18:17:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.064+0000 7f2fa2a00640 1 -- 172.21.15.26:0/1320897486 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8c031940 con 0x7f2fa415bf10 2024-01-15T18:17:35.068 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.064+0000 7f2fa2a00640 1 -- 172.21.15.26:0/1320897486 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c031cd0 con 0x7f2fa415bf10 2024-01-15T18:17:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.065+0000 7f2fa9800640 1 -- 172.21.15.26:0/1320897486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 msgr2=0x7f2fa40cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.065+0000 7f2fa9800640 1 --2- 172.21.15.26:0/1320897486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 0x7f2fa40cc440 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f2fa406e490 tx=0x7f2f8c02b880 comp rx=0 tx=0).stop 2024-01-15T18:17:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.065+0000 7f2fa9800640 1 -- 172.21.15.26:0/1320897486 shutdown_connections 2024-01-15T18:17:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.065+0000 7f2fa9800640 1 --2- 172.21.15.26:0/1320897486 >> v2:172.21.15.124:3300/0 conn(0x7f2fa40cc980 0x7f2fa4143d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.065+0000 7f2fa9800640 1 --2- 172.21.15.26:0/1320897486 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 0x7f2fa40cc440 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.069 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.065+0000 7f2fa9800640 1 --2- 172.21.15.26:0/1320897486 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa4070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.065+0000 7f2fa9800640 1 -- 172.21.15.26:0/1320897486 >> 172.21.15.26:0/1320897486 conn(0x7f2fa406a7d0 msgr2=0x7f2fa406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.065+0000 7f2fa9800640 1 -- 172.21.15.26:0/1320897486 shutdown_connections 2024-01-15T18:17:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.065+0000 7f2fa9800640 1 -- 172.21.15.26:0/1320897486 wait complete. 2024-01-15T18:17:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.067+0000 7f2fa9800640 1 Processor -- start 2024-01-15T18:17:35.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.067+0000 7f2fa9800640 1 -- start start 2024-01-15T18:17:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa9800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa40cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa9800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2fa40cc980 0x7f2fa4142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa9800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 0x7f2fa415cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa9800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa40b0100 con 0x7f2fa415bf10 2024-01-15T18:17:35.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa9800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa41f3870 con 0x7f2fa40cc980 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa9800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa415e050 con 0x7f2fa40adf20 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 0x7f2fa415cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 0x7f2fa415cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48900/0 (socket says 172.21.15.26:48900) 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa8e00640 1 -- 172.21.15.26:0/443245222 learned_addr learned my addr 172.21.15.26:0/443245222 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa3400640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2fa40cc980 0x7f2fa4142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.068+0000 7f2fa3e00640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa40cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2fa8e00640 1 -- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 msgr2=0x7f2fa40cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2fa8e00640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa40cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2fa8e00640 1 -- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2fa40cc980 msgr2=0x7f2fa4142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2fa8e00640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2fa40cc980 0x7f2fa4142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2fa8e00640 1 -- 172.21.15.26:0/443245222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c0094d0 con 0x7f2fa415bf10 2024-01-15T18:17:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2fa3e00640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa40cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2fa3400640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2fa40cc980 0x7f2fa4142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2fa8e00640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 0x7f2fa415cd60 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00cf20 tx=0x7f2f9c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2f9be00640 1 -- 172.21.15.26:0/443245222 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00d840 con 0x7f2fa415bf10 2024-01-15T18:17:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2f9be00640 1 -- 172.21.15.26:0/443245222 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c00d9a0 con 0x7f2fa415bf10 2024-01-15T18:17:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2f9be00640 1 -- 172.21.15.26:0/443245222 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00dd50 con 0x7f2fa415bf10 2024-01-15T18:17:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.069+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa415e580 con 0x7f2fa415bf10 2024-01-15T18:17:35.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.070+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa41f3260 con 0x7f2fa415bf10 2024-01-15T18:17:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.071+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa40c8e80 con 0x7f2fa415bf10 2024-01-15T18:17:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.072+0000 7f2f9be00640 1 -- 172.21.15.26:0/443245222 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f2f9c01e3b0 con 0x7f2fa415bf10 2024-01-15T18:17:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.073+0000 7f2f9be00640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f7c06c550 0x7f2f7c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.073+0000 7f2f9be00640 1 -- 172.21.15.26:0/443245222 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f2f9c0895f0 con 0x7f2fa415bf10 2024-01-15T18:17:35.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.073+0000 7f2fa3e00640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f7c06c550 0x7f2f7c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.075+0000 7f2fa3e00640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f7c06c550 0x7f2f7c06ea10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40cc740 tx=0x7f2f94008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.077+0000 7f2f9be00640 1 -- 172.21.15.26:0/443245222 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f9c055b90 con 0x7f2fa415bf10 2024-01-15T18:17:35.274 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.272+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2fa4067450 con 0x7f2fa415bf10 2024-01-15T18:17:35.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.272+0000 7f2f9be00640 1 -- 172.21.15.26:0/443245222 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f9c01a070 con 0x7f2fa415bf10 2024-01-15T18:17:35.275 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T18:17:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f7c06c550 msgr2=0x7f2f7c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f7c06c550 0x7f2f7c06ea10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40cc740 tx=0x7f2f94008040 comp rx=0 tx=0).stop 2024-01-15T18:17:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 msgr2=0x7f2fa415cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 0x7f2fa415cd60 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00cf20 tx=0x7f2f9c00a7d0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 shutdown_connections 2024-01-15T18:17:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2f7c06c550 0x7f2f7c06ea10 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2fa415bf10 0x7f2fa415cd60 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.278 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2fa40cc980 0x7f2fa4142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 --2- 172.21.15.26:0/443245222 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2fa40adf20 0x7f2fa40cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 >> 172.21.15.26:0/443245222 conn(0x7f2fa406a7d0 msgr2=0x7f2fa406d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 shutdown_connections 2024-01-15T18:17:35.279 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.275+0000 7f2fa9800640 1 -- 172.21.15.26:0/443245222 wait complete. 2024-01-15T18:17:35.296 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:17:35.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.509+0000 7f59c2e00640 1 Processor -- start 2024-01-15T18:17:35.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.510+0000 7f59c2e00640 1 -- start start 2024-01-15T18:17:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.511+0000 7f59c2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59bc071410 0x7f59bc146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.511+0000 7f59c2e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc0b2e10 con 0x7f59bc06c7c0 2024-01-15T18:17:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.511+0000 7f59c2e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59bc06cc20 con 0x7f59bc071f00 2024-01-15T18:17:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.511+0000 7f59c2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc06cec0 con 0x7f59bc071410 2024-01-15T18:17:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.512+0000 7f59c1000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f59bc06c7c0 0x7f59bc0717f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41608/0 (socket says 172.21.15.26:41608) 2024-01-15T18:17:35.514 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.512+0000 7f59c1000640 1 -- 172.21.15.26:0/897143722 learned_addr learned my addr 172.21.15.26:0/897143722 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:35.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.512+0000 7f59c1a00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59bc071410 0x7f59bc146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.512+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538509088 0 0) 0x7f59bc0b2e10 con 0x7f59bc06c7c0 2024-01-15T18:17:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f599c003520 con 0x7f59bc06c7c0 2024-01-15T18:17:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1893910430 0 0) 0x7f599c003520 con 0x7f59bc06c7c0 2024-01-15T18:17:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59bc071410 msgr2=0x7f59bc146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59bc071410 0x7f59bc146770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 >> v1:172.21.15.124:6789/0 conn(0x7f59bc071f00 legacy=0x7f59bc166cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc06d1a0 con 0x7f59bc06c7c0 2024-01-15T18:17:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f59a4001670 con 0x7f59bc06c7c0 2024-01-15T18:17:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 >> v1:172.21.15.26:6789/0 conn(0x7f59bc06c7c0 legacy=0x7f59bc0717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c0039a0 0x7f599c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f599c0047c0 0x7f599c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59bc071410 0x7f599c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc06cec0 con 0x7f599c0039a0 2024-01-15T18:17:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c00ba60 con 0x7f599c0047c0 2024-01-15T18:17:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c00bc20 con 0x7f59bc071410 2024-01-15T18:17:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.513+0000 7f59c1a00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c0039a0 0x7f599c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.514+0000 7f59c1000640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f599c0047c0 0x7f599c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.514+0000 7f59c1a00640 1 -- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59bc071410 msgr2=0x7f599c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.514+0000 7f59c1a00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f59bc071410 0x7f599c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.514+0000 7f59c1a00640 1 -- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f599c0047c0 msgr2=0x7f599c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.514+0000 7f59c1a00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f599c0047c0 0x7f599c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.514+0000 7f59c1a00640 1 -- 172.21.15.26:0/897143722 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c003520 con 0x7f599c0039a0 2024-01-15T18:17:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.515+0000 7f59c1a00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c0039a0 0x7f599c003dc0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009a90 tx=0x7f599c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d37db2e474fefa1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.517+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac02c6b0 con 0x7f599c0039a0 2024-01-15T18:17:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.517+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac02c810 con 0x7f599c0039a0 2024-01-15T18:17:35.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.517+0000 7f59bbe00640 1 -- 172.21.15.26:0/897143722 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac02cba0 con 0x7f599c0039a0 2024-01-15T18:17:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.517+0000 7f59c2e00640 1 -- 172.21.15.26:0/897143722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c0039a0 msgr2=0x7f599c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.517+0000 7f59c2e00640 1 --2- 172.21.15.26:0/897143722 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f599c0039a0 0x7f599c003dc0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009a90 tx=0x7f599c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.517+0000 7f59c2400640 1 -- 172.21.15.26:0/897143722 reap_dead start 2024-01-15T18:17:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.518+0000 7f59c2e00640 1 -- 172.21.15.26:0/897143722 shutdown_connections 2024-01-15T18:17:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.518+0000 7f59c2e00640 1 -- 172.21.15.26:0/897143722 >> 172.21.15.26:0/897143722 conn(0x7f59bc06a800 msgr2=0x7f59bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:35.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.518+0000 7f59c2e00640 1 -- 172.21.15.26:0/897143722 shutdown_connections 2024-01-15T18:17:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.518+0000 7f59c2e00640 1 -- 172.21.15.26:0/897143722 wait complete. 2024-01-15T18:17:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.520+0000 7f59c2e00640 1 Processor -- start 2024-01-15T18:17:35.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.520+0000 7f59c2e00640 1 -- start start 2024-01-15T18:17:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c2e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f599c0039a0 0x7f59bc0ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c2e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59bc0cd150 0x7f59bc140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c2e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59bc141300 0x7f59bc0cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c2e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc141780 con 0x7f59bc0cd150 2024-01-15T18:17:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c2e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0cc0b0 con 0x7f59bc141300 2024-01-15T18:17:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c2e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc0cc390 con 0x7f599c0039a0 2024-01-15T18:17:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59bc0cd150 0x7f59bc140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c1000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59bc0cd150 0x7f59bc140dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48928/0 (socket says 172.21.15.26:48928) 2024-01-15T18:17:35.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c1000640 1 -- 172.21.15.26:0/2923704492 learned_addr learned my addr 172.21.15.26:0/2923704492 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:35.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c1a00640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f599c0039a0 0x7f59bc0ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.521+0000 7f59c2400640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59bc141300 0x7f59bc0cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59c1000640 1 -- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f599c0039a0 msgr2=0x7f59bc0ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59c1000640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f599c0039a0 0x7f59bc0ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59c1000640 1 -- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59bc141300 msgr2=0x7f59bc0cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59c1000640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59bc141300 0x7f59bc0cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59c1000640 1 -- 172.21.15.26:0/2923704492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0cc670 con 0x7f59bc0cd150 2024-01-15T18:17:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59c1000640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59bc0cd150 0x7f59bc140dc0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f59a400d810 tx=0x7f59a400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59b9600640 1 -- 172.21.15.26:0/2923704492 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a40074e0 con 0x7f59bc0cd150 2024-01-15T18:17:35.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59b9600640 1 -- 172.21.15.26:0/2923704492 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a4007640 con 0x7f59bc0cd150 2024-01-15T18:17:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59bc0cc950 con 0x7f59bc0cd150 2024-01-15T18:17:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59b9600640 1 -- 172.21.15.26:0/2923704492 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4004a00 con 0x7f59bc0cd150 2024-01-15T18:17:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.522+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc0cd5d0 con 0x7f59bc0cd150 2024-01-15T18:17:35.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.524+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59bc0cce70 con 0x7f59bc0cd150 2024-01-15T18:17:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.525+0000 7f59b9600640 1 -- 172.21.15.26:0/2923704492 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f59a4004b60 con 0x7f59bc0cd150 2024-01-15T18:17:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.526+0000 7f59b9600640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f599406c580 0x7f599406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:35.528 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.526+0000 7f59b9600640 1 -- 172.21.15.26:0/2923704492 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f59a40884e0 con 0x7f59bc0cd150 2024-01-15T18:17:35.529 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.527+0000 7f59c1a00640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f599406c580 0x7f599406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:35.530 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.528+0000 7f59c1a00640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f599406c580 0x7f599406ea40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f59ac036000 tx=0x7f59ac030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:35.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.531+0000 7f59b9600640 1 -- 172.21.15.26:0/2923704492 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59bc0cce70 con 0x7f59bc0cd150 2024-01-15T18:17:35.743 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.740+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f59bc067450 con 0x7f59bc0cd150 2024-01-15T18:17:35.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.741+0000 7f59b9600640 1 -- 172.21.15.26:0/2923704492 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v117) v1 ==== 80+0+0 (secure 0 0 0) 0x7f59a4019070 con 0x7f59bc0cd150 2024-01-15T18:17:35.744 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T18:17:35.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f599406c580 msgr2=0x7f599406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f599406c580 0x7f599406ea40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f59ac036000 tx=0x7f59ac030000 comp rx=0 tx=0).stop 2024-01-15T18:17:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59bc0cd150 msgr2=0x7f59bc140dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59bc0cd150 0x7f59bc140dc0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f59a400d810 tx=0x7f59a400dcd0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 shutdown_connections 2024-01-15T18:17:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f599406c580 0x7f599406ea40 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f59bc141300 0x7f59bc0cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f59bc0cd150 0x7f59bc140dc0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.747 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 --2- 172.21.15.26:0/2923704492 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f599c0039a0 0x7f59bc0ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:35.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 >> 172.21.15.26:0/2923704492 conn(0x7f59bc06a800 msgr2=0x7f59bc142140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:35.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 shutdown_connections 2024-01-15T18:17:35.748 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:35.744+0000 7f59c2e00640 1 -- 172.21.15.26:0/2923704492 wait complete. 2024-01-15T18:17:35.765 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:17:35.765 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:35.814 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:36.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.075+0000 7f86baa00640 1 Processor -- start 2024-01-15T18:17:36.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.075+0000 7f86baa00640 1 -- start start 2024-01-15T18:17:36.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.077+0000 7f86baa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f86b40cc500 0x7f86b40cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.077+0000 7f86baa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b4070e50 con 0x7f86b4143c40 2024-01-15T18:17:36.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.077+0000 7f86baa00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b4144020 con 0x7f86b406d160 2024-01-15T18:17:36.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.077+0000 7f86baa00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b40723c0 con 0x7f86b40cc500 2024-01-15T18:17:36.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.077+0000 7f86b9600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f86b4143c40 0x7f86b4071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41622/0 (socket says 172.21.15.26:41622) 2024-01-15T18:17:36.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.077+0000 7f86b9600640 1 -- 172.21.15.26:0/63648178 learned_addr learned my addr 172.21.15.26:0/63648178 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:36.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996176632 0 0) 0x7f86b4070e50 con 0x7f86b4143c40 2024-01-15T18:17:36.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b8c00640 1 --2- 172.21.15.26:0/63648178 >> v2:172.21.15.124:3300/0 conn(0x7f86b40cc500 0x7f86b40cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8694003520 con 0x7f86b4143c40 2024-01-15T18:17:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 448339049 0 0) 0x7f8694003520 con 0x7f86b4143c40 2024-01-15T18:17:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 >> v2:172.21.15.124:3300/0 conn(0x7f86b40cc500 msgr2=0x7f86b40cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 --2- 172.21.15.26:0/63648178 >> v2:172.21.15.124:3300/0 conn(0x7f86b40cc500 0x7f86b40cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 >> v1:172.21.15.124:6789/0 conn(0x7f86b406d160 legacy=0x7f86b406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4071e00 con 0x7f86b4143c40 2024-01-15T18:17:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f86a40019f0 con 0x7f86b4143c40 2024-01-15T18:17:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 >> v1:172.21.15.26:6789/0 conn(0x7f86b4143c40 legacy=0x7f86b4071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86940039c0 0x7f8694003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86940047c0 0x7f8694004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f869400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40723c0 con 0x7f8694005250 2024-01-15T18:17:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869400c1a0 con 0x7f86940047c0 2024-01-15T18:17:36.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.078+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869400c360 con 0x7f86940039c0 2024-01-15T18:17:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.079+0000 7f86ba000640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f869400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.079+0000 7f86b8c00640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86940047c0 0x7f8694004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.079+0000 7f86b9600640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86940039c0 0x7f8694003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.079+0000 7f86ba000640 1 -- 172.21.15.26:0/63648178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86940039c0 msgr2=0x7f8694003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.079+0000 7f86ba000640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86940039c0 0x7f8694003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.079+0000 7f86ba000640 1 -- 172.21.15.26:0/63648178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86940047c0 msgr2=0x7f8694004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.079+0000 7f86ba000640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86940047c0 0x7f8694004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.079+0000 7f86ba000640 1 -- 172.21.15.26:0/63648178 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694003520 con 0x7f8694005250 2024-01-15T18:17:36.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.079+0000 7f86b9600640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86940039c0 0x7f8694003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.080+0000 7f86ba000640 1 -- 172.21.15.26:0/63648178 reap_dead start 2024-01-15T18:17:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.080+0000 7f86ba000640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f869400b980 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f86ac003db0 tx=0x7f86940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8e3a1e06dce7f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac007220 con 0x7f8694005250 2024-01-15T18:17:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac02e050 con 0x7f8694005250 2024-01-15T18:17:36.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86b3e00640 1 -- 172.21.15.26:0/63648178 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0075e0 con 0x7f8694005250 2024-01-15T18:17:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86baa00640 1 -- 172.21.15.26:0/63648178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 msgr2=0x7f869400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86baa00640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f869400b980 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f86ac003db0 tx=0x7f86940057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86baa00640 1 -- 172.21.15.26:0/63648178 shutdown_connections 2024-01-15T18:17:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86baa00640 1 --2- 172.21.15.26:0/63648178 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f869400b980 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86baa00640 1 -- 172.21.15.26:0/63648178 >> 172.21.15.26:0/63648178 conn(0x7f86b406a800 msgr2=0x7f86b406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86baa00640 1 -- 172.21.15.26:0/63648178 shutdown_connections 2024-01-15T18:17:36.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.081+0000 7f86baa00640 1 -- 172.21.15.26:0/63648178 wait complete. 2024-01-15T18:17:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.083+0000 7f86baa00640 1 Processor -- start 2024-01-15T18:17:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.083+0000 7f86baa00640 1 -- start start 2024-01-15T18:17:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.084+0000 7f86baa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f86b415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.084+0000 7f86baa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b40ab0c0 0x7f86b40ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.084+0000 7f86baa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86b40addf0 0x7f86b415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.084+0000 7f86baa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b415d210 con 0x7f8694005250 2024-01-15T18:17:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.084+0000 7f86baa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b40ae280 con 0x7f86b40addf0 2024-01-15T18:17:36.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.084+0000 7f86baa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b415e4f0 con 0x7f86b40ab0c0 2024-01-15T18:17:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.084+0000 7f86b9600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f86b415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86b9600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f86b415a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48936/0 (socket says 172.21.15.26:48936) 2024-01-15T18:17:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86b9600640 1 -- 172.21.15.26:0/524778287 learned_addr learned my addr 172.21.15.26:0/524778287 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86b8c00640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b40ab0c0 0x7f86b40ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86ba000640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86b40addf0 0x7f86b415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86b9600640 1 -- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b40ab0c0 msgr2=0x7f86b40ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86b9600640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b40ab0c0 0x7f86b40ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86b9600640 1 -- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86b40addf0 msgr2=0x7f86b415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86b9600640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86b40addf0 0x7f86b415c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86b9600640 1 -- 172.21.15.26:0/524778287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b415cf30 con 0x7f8694005250 2024-01-15T18:17:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.085+0000 7f86b9600640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f86b415a0e0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f86a4002130 tx=0x7f86a400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.086+0000 7f86b1600640 1 -- 172.21.15.26:0/524778287 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4004d60 con 0x7f8694005250 2024-01-15T18:17:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.086+0000 7f86b1600640 1 -- 172.21.15.26:0/524778287 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a40070d0 con 0x7f8694005250 2024-01-15T18:17:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.086+0000 7f86b1600640 1 -- 172.21.15.26:0/524778287 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a4007480 con 0x7f8694005250 2024-01-15T18:17:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.086+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b415dea0 con 0x7f8694005250 2024-01-15T18:17:36.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.086+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b415c7b0 con 0x7f8694005250 2024-01-15T18:17:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.087+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b415dd00 con 0x7f8694005250 2024-01-15T18:17:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.088+0000 7f86b1600640 1 -- 172.21.15.26:0/524778287 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f86a40075e0 con 0x7f8694005250 2024-01-15T18:17:36.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.090+0000 7f86b1600640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f868c06c580 0x7f868c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.090+0000 7f86b1600640 1 -- 172.21.15.26:0/524778287 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f86a4088bc0 con 0x7f8694005250 2024-01-15T18:17:36.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.090+0000 7f86b8c00640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f868c06c580 0x7f868c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.091+0000 7f86b8c00640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f868c06c580 0x7f868c06ea40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f86b40ab4a0 tx=0x7f869c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.094+0000 7f86b1600640 1 -- 172.21.15.26:0/524778287 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86b415dd00 con 0x7f8694005250 2024-01-15T18:17:36.292 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.289+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86b415dd00 con 0x7f8694005250 2024-01-15T18:17:36.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.292+0000 7f86b1600640 1 -- 172.21.15.26:0/524778287 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16729 (secure 0 0 0) 0x7f86b415dd00 con 0x7f8694005250 2024-01-15T18:17:36.295 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:17:36.296 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":117,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:12:53.309154+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:12:08.696115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:12:09.497434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.30","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:36.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.295+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f868c06c580 msgr2=0x7f868c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.295+0000 7f86baa00640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f868c06c580 0x7f868c06ea40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f86b40ab4a0 tx=0x7f869c008040 comp rx=0 tx=0).stop 2024-01-15T18:17:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.295+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 msgr2=0x7f86b415a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.295+0000 7f86baa00640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f86b415a0e0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f86a4002130 tx=0x7f86a400b4a0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.296+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 shutdown_connections 2024-01-15T18:17:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.296+0000 7f86baa00640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f86b40ab0c0 0x7f86b40ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.296+0000 7f86baa00640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f868c06c580 0x7f868c06ea40 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.296+0000 7f86baa00640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f86b40addf0 0x7f86b415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.296+0000 7f86baa00640 1 --2- 172.21.15.26:0/524778287 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f8694005250 0x7f86b415a0e0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.296+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 >> 172.21.15.26:0/524778287 conn(0x7f86b406a800 msgr2=0x7f86b40afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.296+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 shutdown_connections 2024-01-15T18:17:36.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.296+0000 7f86baa00640 1 -- 172.21.15.26:0/524778287 wait complete. 2024-01-15T18:17:36.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:12:08.696115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:12:09.497434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:17:36.316 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:17:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.529+0000 7fec7be00640 1 Processor -- start 2024-01-15T18:17:36.532 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.530+0000 7fec7be00640 1 -- start start 2024-01-15T18:17:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.531+0000 7fec7be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fec7c15b060 0x7fec7c0ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.531+0000 7fec7be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c0cdbe0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.531+0000 7fec7be00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c0cde80 con 0x7fec7c0714a0 2024-01-15T18:17:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.531+0000 7fec7be00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c0cf010 con 0x7fec7c15b060 2024-01-15T18:17:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.532+0000 7fec7a000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fec7c0ceb80 0x7fec7c070d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41628/0 (socket says 172.21.15.26:41628) 2024-01-15T18:17:36.534 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.532+0000 7fec7a000640 1 -- 172.21.15.26:0/2766893181 learned_addr learned my addr 172.21.15.26:0/2766893181 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:36.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.532+0000 7fec7aa00640 1 --2- 172.21.15.26:0/2766893181 >> v2:172.21.15.26:3300/0 conn(0x7fec7c15b060 0x7fec7c0ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.535 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.532+0000 7fec79600640 1 -- 172.21.15.26:0/2766893181 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666734955 0 0) 0x7fec7c0cdbe0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.532+0000 7fec79600640 1 -- 172.21.15.26:0/2766893181 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec5c003520 con 0x7fec7c0ceb80 2024-01-15T18:17:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.532+0000 7fec7aa00640 1 -- 172.21.15.26:0/2766893181 >> v1:172.21.15.124:6789/0 conn(0x7fec7c0714a0 legacy=0x7fec7c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.532+0000 7fec7aa00640 1 -- 172.21.15.26:0/2766893181 >> v1:172.21.15.26:6789/0 conn(0x7fec7c0ceb80 legacy=0x7fec7c070d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.536 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.532+0000 7fec7aa00640 1 -- 172.21.15.26:0/2766893181 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c06b030 con 0x7fec7c15b060 2024-01-15T18:17:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.534+0000 7fec7aa00640 1 --2- 172.21.15.26:0/2766893181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c15b060 0x7fec7c0ce640 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0097a0 tx=0x7fec6c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3c9e7d38a1d17c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.537 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.534+0000 7fec79600640 1 -- 172.21.15.26:0/2766893181 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c030610 con 0x7fec7c15b060 2024-01-15T18:17:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.534+0000 7fec79600640 1 -- 172.21.15.26:0/2766893181 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c030770 con 0x7fec7c15b060 2024-01-15T18:17:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.534+0000 7fec79600640 1 -- 172.21.15.26:0/2766893181 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c030b00 con 0x7fec7c15b060 2024-01-15T18:17:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.534+0000 7fec7be00640 1 -- 172.21.15.26:0/2766893181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c15b060 msgr2=0x7fec7c0ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.534+0000 7fec7be00640 1 --2- 172.21.15.26:0/2766893181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c15b060 0x7fec7c0ce640 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0097a0 tx=0x7fec6c02b960 comp rx=0 tx=0).stop 2024-01-15T18:17:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.535+0000 7fec7be00640 1 -- 172.21.15.26:0/2766893181 shutdown_connections 2024-01-15T18:17:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.535+0000 7fec7be00640 1 --2- 172.21.15.26:0/2766893181 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c15b060 0x7fec7c0ce640 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.535+0000 7fec7be00640 1 -- 172.21.15.26:0/2766893181 >> 172.21.15.26:0/2766893181 conn(0x7fec7c0b15b0 msgr2=0x7fec7c0b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.538 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.535+0000 7fec7be00640 1 -- 172.21.15.26:0/2766893181 shutdown_connections 2024-01-15T18:17:36.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.535+0000 7fec7be00640 1 -- 172.21.15.26:0/2766893181 wait complete. 2024-01-15T18:17:36.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.536+0000 7fec7be00640 1 Processor -- start 2024-01-15T18:17:36.539 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.537+0000 7fec7be00640 1 -- start start 2024-01-15T18:17:36.540 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.537+0000 7fec7be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec7c0714a0 0x7fec7c14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0ceb80 0x7fec7c0c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec7c15b060 0x7fec7c1f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0c7ff0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0c7520 con 0x7fec7c15b060 2024-01-15T18:17:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c147de0 con 0x7fec7c0714a0 2024-01-15T18:17:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0ceb80 0x7fec7c0c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7a000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0ceb80 0x7fec7c0c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48958/0 (socket says 172.21.15.26:48958) 2024-01-15T18:17:36.541 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7a000640 1 -- 172.21.15.26:0/1412265549 learned_addr learned my addr 172.21.15.26:0/1412265549 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7a000640 1 -- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec7c0714a0 msgr2=0x7fec7c14a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7a000640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec7c0714a0 0x7fec7c14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7a000640 1 -- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec7c15b060 msgr2=0x7fec7c1f00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7a000640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec7c15b060 0x7fec7c1f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.542 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.538+0000 7fec7a000640 1 -- 172.21.15.26:0/1412265549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c009400 con 0x7fec7c0ceb80 2024-01-15T18:17:36.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.539+0000 7fec7a000640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0ceb80 0x7fec7c0c6fe0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fec6400c500 tx=0x7fec6400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.539+0000 7fec72a00640 1 -- 172.21.15.26:0/1412265549 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64005ce0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.539+0000 7fec72a00640 1 -- 172.21.15.26:0/1412265549 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec64005e40 con 0x7fec7c0ceb80 2024-01-15T18:17:36.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.539+0000 7fec72a00640 1 -- 172.21.15.26:0/1412265549 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64007330 con 0x7fec7c0ceb80 2024-01-15T18:17:36.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.539+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c1f05f0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.543 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.539+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c0c7a30 con 0x7fec7c0ceb80 2024-01-15T18:17:36.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.541+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c147bb0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.541+0000 7fec72a00640 1 -- 172.21.15.26:0/1412265549 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fec64007490 con 0x7fec7c0ceb80 2024-01-15T18:17:36.544 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.542+0000 7fec72a00640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec5406c550 0x7fec5406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.542+0000 7fec72a00640 1 -- 172.21.15.26:0/1412265549 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fec640881a0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.545 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.543+0000 7fec7aa00640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec5406c550 0x7fec5406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.546 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.544+0000 7fec7aa00640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec5406c550 0x7fec5406ea10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fec6c036000 tx=0x7fec6c030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.549 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.547+0000 7fec72a00640 1 -- 172.21.15.26:0/1412265549 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec7c147bb0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.745 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.743+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec7c147bb0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.746 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.744+0000 7fec72a00640 1 -- 172.21.15.26:0/1412265549 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec7c147bb0 con 0x7fec7c0ceb80 2024-01-15T18:17:36.747 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec5406c550 msgr2=0x7fec5406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec5406c550 0x7fec5406ea10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fec6c036000 tx=0x7fec6c030000 comp rx=0 tx=0).stop 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0ceb80 msgr2=0x7fec7c0c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0ceb80 0x7fec7c0c6fe0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fec6400c500 tx=0x7fec6400c9c0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.749 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 shutdown_connections 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fec5406c550 0x7fec5406ea10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fec7c15b060 0x7fec7c1f00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fec7c0ceb80 0x7fec7c0c6fe0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 --2- 172.21.15.26:0/1412265549 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fec7c0714a0 0x7fec7c14a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.746+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 >> 172.21.15.26:0/1412265549 conn(0x7fec7c0b15b0 msgr2=0x7fec7c15e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.747+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 shutdown_connections 2024-01-15T18:17:36.750 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.747+0000 7fec7be00640 1 -- 172.21.15.26:0/1412265549 wait complete. 2024-01-15T18:17:36.767 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:17:36.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.984+0000 7f2903e00640 1 Processor -- start 2024-01-15T18:17:36.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.984+0000 7f2903e00640 1 -- start start 2024-01-15T18:17:36.988 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.986+0000 7f2903e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040ccb50 0x7f2904070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.986+0000 7f2903e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f290415bf10 0x7f2904143d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.986+0000 7f2903e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290415f4e0 con 0x7f29040adf20 2024-01-15T18:17:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.986+0000 7f2903e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290415f970 con 0x7f290415bf10 2024-01-15T18:17:36.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.986+0000 7f2903e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290415c3c0 con 0x7f29040ccb50 2024-01-15T18:17:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.987+0000 7f2903400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f290415bf10 0x7f2904143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.987+0000 7f28fbe00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040ccb50 0x7f2904070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.987+0000 7f2902a00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f29040adf20 0x7f29040cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:53616/0 (socket says 172.21.15.26:53616) 2024-01-15T18:17:36.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.987+0000 7f2902a00640 1 -- 172.21.15.26:0/3405155625 learned_addr learned my addr 172.21.15.26:0/3405155625 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.988+0000 7f2903400640 1 -- 172.21.15.26:0/3405155625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040ccb50 msgr2=0x7f2904070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.988+0000 7f2903400640 1 --2- 172.21.15.26:0/3405155625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040ccb50 0x7f2904070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.988+0000 7f2903400640 1 -- 172.21.15.26:0/3405155625 >> v1:172.21.15.124:6789/0 conn(0x7f29040adf20 legacy=0x7f29040cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.988+0000 7f2903400640 1 -- 172.21.15.26:0/3405155625 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904071130 con 0x7f290415bf10 2024-01-15T18:17:36.991 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.988+0000 7f2902000640 1 -- 172.21.15.26:0/3405155625 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264730689 0 0) 0x7f290415f4e0 con 0x7f29040adf20 2024-01-15T18:17:36.992 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.990+0000 7f2903400640 1 --2- 172.21.15.26:0/3405155625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f290415bf10 0x7f2904143d30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009d90 tx=0x7f28fc02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b42e1391ac0f9b server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.990+0000 7f2902000640 1 -- 172.21.15.26:0/3405155625 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc02cbc0 con 0x7f290415bf10 2024-01-15T18:17:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2902000640 1 -- 172.21.15.26:0/3405155625 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc02cd20 con 0x7f290415bf10 2024-01-15T18:17:36.993 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2902000640 1 -- 172.21.15.26:0/3405155625 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc034660 con 0x7f290415bf10 2024-01-15T18:17:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2903e00640 1 -- 172.21.15.26:0/3405155625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f290415bf10 msgr2=0x7f2904143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2903e00640 1 --2- 172.21.15.26:0/3405155625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f290415bf10 0x7f2904143d30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009d90 tx=0x7f28fc02be10 comp rx=0 tx=0).stop 2024-01-15T18:17:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2903e00640 1 -- 172.21.15.26:0/3405155625 shutdown_connections 2024-01-15T18:17:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2903e00640 1 --2- 172.21.15.26:0/3405155625 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040ccb50 0x7f2904070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2903e00640 1 --2- 172.21.15.26:0/3405155625 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f290415bf10 0x7f2904143d30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2903e00640 1 -- 172.21.15.26:0/3405155625 >> 172.21.15.26:0/3405155625 conn(0x7f290406a7d0 msgr2=0x7f290406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:36.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2903e00640 1 -- 172.21.15.26:0/3405155625 shutdown_connections 2024-01-15T18:17:36.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.991+0000 7f2903e00640 1 -- 172.21.15.26:0/3405155625 wait complete. 2024-01-15T18:17:36.995 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.993+0000 7f2903e00640 1 Processor -- start 2024-01-15T18:17:36.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.993+0000 7f2903e00640 1 -- start start 2024-01-15T18:17:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.994+0000 7f2903e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040adf20 0x7f2904139450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.994+0000 7f2903e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29040ccb50 0x7f29041f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.994+0000 7f2903e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f290415bf10 0x7f29040b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.994+0000 7f2903e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29040c78b0 con 0x7f290415bf10 2024-01-15T18:17:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.994+0000 7f2903e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2904139990 con 0x7f29040ccb50 2024-01-15T18:17:36.997 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.994+0000 7f2903e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29041f3c20 con 0x7f29040adf20 2024-01-15T18:17:36.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f2903400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f290415bf10 0x7f29040b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f2903400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f290415bf10 0x7f29040b2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48974/0 (socket says 172.21.15.26:48974) 2024-01-15T18:17:36.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f2903400640 1 -- 172.21.15.26:0/3904715636 learned_addr learned my addr 172.21.15.26:0/3904715636 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:36.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f28fbe00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29040ccb50 0x7f29041f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f2902a00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040adf20 0x7f2904139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:36.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f2903400640 1 -- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040adf20 msgr2=0x7f2904139450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:36.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f2903400640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040adf20 0x7f2904139450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:36.999 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f2903400640 1 -- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29040ccb50 msgr2=0x7f29041f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f2903400640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29040ccb50 0x7f29041f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.995+0000 7f2903400640 1 -- 172.21.15.26:0/3904715636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc009a70 con 0x7f290415bf10 2024-01-15T18:17:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.996+0000 7f2903400640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f290415bf10 0x7f29040b2ee0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009d90 tx=0x7f28fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.996+0000 7f28fb400640 1 -- 172.21.15.26:0/3904715636 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc040650 con 0x7f290415bf10 2024-01-15T18:17:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.996+0000 7f28fb400640 1 -- 172.21.15.26:0/3904715636 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc02ce90 con 0x7f290415bf10 2024-01-15T18:17:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.996+0000 7f28fb400640 1 -- 172.21.15.26:0/3904715636 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0409f0 con 0x7f290415bf10 2024-01-15T18:17:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.996+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29041f3320 con 0x7f290415bf10 2024-01-15T18:17:37.000 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.996+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29041f38e0 con 0x7f290415bf10 2024-01-15T18:17:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.997+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2904067450 con 0x7f290415bf10 2024-01-15T18:17:37.001 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.998+0000 7f28fb400640 1 -- 172.21.15.26:0/3904715636 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f28fc040b50 con 0x7f290415bf10 2024-01-15T18:17:37.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.999+0000 7f28fb400640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28dc06c550 0x7f28dc06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:36.999+0000 7f28fb400640 1 -- 172.21.15.26:0/3904715636 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f28fc0ab810 con 0x7f290415bf10 2024-01-15T18:17:37.002 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.000+0000 7f2902a00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28dc06c550 0x7f28dc06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:37.003 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.001+0000 7f2902a00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28dc06c550 0x7f28dc06ea10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000f60 tx=0x7f28f4007b50 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:37.006 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.004+0000 7f28fb400640 1 -- 172.21.15.26:0/3904715636 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28fc077d50 con 0x7f290415bf10 2024-01-15T18:17:37.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.199+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2904144600 con 0x7f290415bf10 2024-01-15T18:17:37.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.200+0000 7f28fb400640 1 -- 172.21.15.26:0/3904715636 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28fc044020 con 0x7f290415bf10 2024-01-15T18:17:37.202 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T18:17:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.202+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28dc06c550 msgr2=0x7f28dc06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.202+0000 7f2903e00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28dc06c550 0x7f28dc06ea10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000f60 tx=0x7f28f4007b50 comp rx=0 tx=0).stop 2024-01-15T18:17:37.205 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.202+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f290415bf10 msgr2=0x7f29040b2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.202+0000 7f2903e00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f290415bf10 0x7f29040b2ee0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009d90 tx=0x7f28fc004a40 comp rx=0 tx=0).stop 2024-01-15T18:17:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.203+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 shutdown_connections 2024-01-15T18:17:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.203+0000 7f2903e00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f28dc06c550 0x7f28dc06ea10 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.203+0000 7f2903e00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f290415bf10 0x7f29040b2ee0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.203+0000 7f2903e00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29040ccb50 0x7f29041f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.203+0000 7f2903e00640 1 --2- 172.21.15.26:0/3904715636 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29040adf20 0x7f2904139450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.203+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 >> 172.21.15.26:0/3904715636 conn(0x7f290406a7d0 msgr2=0x7f290406abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.203+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 shutdown_connections 2024-01-15T18:17:37.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.203+0000 7f2903e00640 1 -- 172.21.15.26:0/3904715636 wait complete. 2024-01-15T18:17:37.225 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:17:37.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.441+0000 7f38f8e00640 1 Processor -- start 2024-01-15T18:17:37.444 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.442+0000 7f38f8e00640 1 -- start start 2024-01-15T18:17:37.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.443+0000 7f38f8e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f38f413e360 0x7f38f40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.443+0000 7f38f8e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f415c5f0 con 0x7f38f413df80 2024-01-15T18:17:37.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.443+0000 7f38f8e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f406c470 con 0x7f38f415f9c0 2024-01-15T18:17:37.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.443+0000 7f38f8e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f406c750 con 0x7f38f413e360 2024-01-15T18:17:37.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.444+0000 7f38f1e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f38f413df80 0x7f38f415bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41636/0 (socket says 172.21.15.26:41636) 2024-01-15T18:17:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.444+0000 7f38f1e00640 1 -- 172.21.15.26:0/2669824215 learned_addr learned my addr 172.21.15.26:0/2669824215 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:37.447 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.444+0000 7f38f1400640 1 --2- 172.21.15.26:0/2669824215 >> v2:172.21.15.124:3300/0 conn(0x7f38f413e360 0x7f38f40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:37.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.445+0000 7f38f0a00640 1 -- 172.21.15.26:0/2669824215 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403749366 0 0) 0x7f38f415c5f0 con 0x7f38f413df80 2024-01-15T18:17:37.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.445+0000 7f38f0a00640 1 -- 172.21.15.26:0/2669824215 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d4003520 con 0x7f38f413df80 2024-01-15T18:17:37.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.445+0000 7f38f1400640 1 -- 172.21.15.26:0/2669824215 >> v1:172.21.15.124:6789/0 conn(0x7f38f415f9c0 legacy=0x7f38f40cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.445+0000 7f38f1400640 1 -- 172.21.15.26:0/2669824215 >> v1:172.21.15.26:6789/0 conn(0x7f38f413df80 legacy=0x7f38f415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.445+0000 7f38f1400640 1 -- 172.21.15.26:0/2669824215 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f406ca30 con 0x7f38f413e360 2024-01-15T18:17:37.448 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.446+0000 7f38f1400640 1 --2- 172.21.15.26:0/2669824215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 0x7f38f40cf290 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0097a0 tx=0x7f38dc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87a02d15ab615f60 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:37.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.447+0000 7f38f0a00640 1 -- 172.21.15.26:0/2669824215 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc030610 con 0x7f38f413e360 2024-01-15T18:17:37.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.447+0000 7f38f0a00640 1 -- 172.21.15.26:0/2669824215 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc030770 con 0x7f38f413e360 2024-01-15T18:17:37.449 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.447+0000 7f38f0a00640 1 -- 172.21.15.26:0/2669824215 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc030b00 con 0x7f38f413e360 2024-01-15T18:17:37.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.448+0000 7f38f8e00640 1 -- 172.21.15.26:0/2669824215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 msgr2=0x7f38f40cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.448+0000 7f38f8e00640 1 --2- 172.21.15.26:0/2669824215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 0x7f38f40cf290 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0097a0 tx=0x7f38dc02b960 comp rx=0 tx=0).stop 2024-01-15T18:17:37.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.448+0000 7f38f8e00640 1 -- 172.21.15.26:0/2669824215 shutdown_connections 2024-01-15T18:17:37.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.448+0000 7f38f8e00640 1 --2- 172.21.15.26:0/2669824215 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 0x7f38f40cf290 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.448+0000 7f38f8e00640 1 -- 172.21.15.26:0/2669824215 >> 172.21.15.26:0/2669824215 conn(0x7f38f406a800 msgr2=0x7f38f406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:37.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.448+0000 7f38f8e00640 1 -- 172.21.15.26:0/2669824215 shutdown_connections 2024-01-15T18:17:37.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.448+0000 7f38f8e00640 1 -- 172.21.15.26:0/2669824215 wait complete. 2024-01-15T18:17:37.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.449+0000 7f38f8e00640 1 Processor -- start 2024-01-15T18:17:37.452 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.450+0000 7f38f8e00640 1 -- start start 2024-01-15T18:17:37.454 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f8e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f413df80 0x7f38f414a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f8e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 0x7f38f40af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f8e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f415f9c0 0x7f38f41f10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f8e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f41f09d0 con 0x7f38f415f9c0 2024-01-15T18:17:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f8e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f414bbb0 con 0x7f38f413e360 2024-01-15T18:17:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f8e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f414b310 con 0x7f38f413df80 2024-01-15T18:17:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f415f9c0 0x7f38f41f10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f415f9c0 0x7f38f41f10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48988/0 (socket says 172.21.15.26:48988) 2024-01-15T18:17:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f2800640 1 -- 172.21.15.26:0/1338147402 learned_addr learned my addr 172.21.15.26:0/1338147402 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:37.455 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f1e00640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f413df80 0x7f38f414a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f1400640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 0x7f38f40af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f2800640 1 -- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f413df80 msgr2=0x7f38f414a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f2800640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f413df80 0x7f38f414a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f2800640 1 -- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 msgr2=0x7f38f40af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f2800640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 0x7f38f40af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f2800640 1 -- 172.21.15.26:0/1338147402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc009400 con 0x7f38f415f9c0 2024-01-15T18:17:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.451+0000 7f38f1400640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 0x7f38f40af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.452+0000 7f38f2800640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f415f9c0 0x7f38f41f10c0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c730 tx=0x7f38ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:37.456 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.452+0000 7f38ea000640 1 -- 172.21.15.26:0/1338147402 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec003b70 con 0x7f38f415f9c0 2024-01-15T18:17:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.452+0000 7f38ea000640 1 -- 172.21.15.26:0/1338147402 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec003cd0 con 0x7f38f415f9c0 2024-01-15T18:17:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.452+0000 7f38ea000640 1 -- 172.21.15.26:0/1338147402 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec00aa90 con 0x7f38f415f9c0 2024-01-15T18:17:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.452+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f414b5f0 con 0x7f38f415f9c0 2024-01-15T18:17:37.457 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.452+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f41f0bc0 con 0x7f38f415f9c0 2024-01-15T18:17:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.453+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f40675d0 con 0x7f38f415f9c0 2024-01-15T18:17:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.454+0000 7f38ea000640 1 -- 172.21.15.26:0/1338147402 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f38ec00abf0 con 0x7f38f415f9c0 2024-01-15T18:17:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.455+0000 7f38ea000640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38cc06c550 0x7f38cc06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.455+0000 7f38ea000640 1 -- 172.21.15.26:0/1338147402 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f38ec088950 con 0x7f38f415f9c0 2024-01-15T18:17:37.458 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.456+0000 7f38f1e00640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38cc06c550 0x7f38cc06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:37.459 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.457+0000 7f38f1e00640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38cc06c550 0x7f38cc06ea10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f38e4005880 tx=0x7f38e4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:37.462 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.460+0000 7f38ea000640 1 -- 172.21.15.26:0/1338147402 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38ec054ef0 con 0x7f38f415f9c0 2024-01-15T18:17:37.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.668+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38f406df00 con 0x7f38f415f9c0 2024-01-15T18:17:37.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.669+0000 7f38ea000640 1 -- 172.21.15.26:0/1338147402 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38ec058510 con 0x7f38f415f9c0 2024-01-15T18:17:37.671 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T18:17:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38cc06c550 msgr2=0x7f38cc06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38cc06c550 0x7f38cc06ea10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f38e4005880 tx=0x7f38e4009040 comp rx=0 tx=0).stop 2024-01-15T18:17:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f415f9c0 msgr2=0x7f38f41f10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f415f9c0 0x7f38f41f10c0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c730 tx=0x7f38ec00cbf0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 shutdown_connections 2024-01-15T18:17:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f38cc06c550 0x7f38cc06ea10 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f38f415f9c0 0x7f38f41f10c0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f38f413e360 0x7f38f40af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 --2- 172.21.15.26:0/1338147402 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f38f413df80 0x7f38f414a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 >> 172.21.15.26:0/1338147402 conn(0x7f38f406a800 msgr2=0x7f38f415cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 shutdown_connections 2024-01-15T18:17:37.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.672+0000 7f38f8e00640 1 -- 172.21.15.26:0/1338147402 wait complete. 2024-01-15T18:17:37.696 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:17:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.964+0000 7f18c0200640 1 Processor -- start 2024-01-15T18:17:37.967 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.964+0000 7f18c0200640 1 -- start start 2024-01-15T18:17:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.966+0000 7f18c0200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f18b813df80 0x7f18b815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.966+0000 7f18c0200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b813e360 0x7f18b80cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.966+0000 7f18c0200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b806c470 con 0x7f18b815f9c0 2024-01-15T18:17:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.967+0000 7f18c0200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b806c750 con 0x7f18b813df80 2024-01-15T18:17:37.969 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.967+0000 7f18c0200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b806ca30 con 0x7f18b813e360 2024-01-15T18:17:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.967+0000 7f18bd800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f18b813df80 0x7f18b815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.967+0000 7f18bd800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f18b813df80 0x7f18b815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:48992/0 (socket says 172.21.15.26:48992) 2024-01-15T18:17:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.967+0000 7f18bd800640 1 -- 172.21.15.26:0/3907338279 learned_addr learned my addr 172.21.15.26:0/3907338279 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.967+0000 7f18bd800640 1 -- 172.21.15.26:0/3907338279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b813e360 msgr2=0x7f18b80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:37.970 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.967+0000 7f18bd800640 1 --2- 172.21.15.26:0/3907338279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b813e360 0x7f18b80cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.968+0000 7f18bd800640 1 -- 172.21.15.26:0/3907338279 >> v1:172.21.15.124:6789/0 conn(0x7f18b815f9c0 legacy=0x7f18b81408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.968+0000 7f18bd800640 1 -- 172.21.15.26:0/3907338279 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b806cd10 con 0x7f18b813df80 2024-01-15T18:17:37.971 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.969+0000 7f18bd800640 1 --2- 172.21.15.26:0/3907338279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813df80 0x7f18b815bee0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009e50 tx=0x7f18b002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207a05b7d4333227 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:37.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.970+0000 7f18b7e00640 1 -- 172.21.15.26:0/3907338279 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b002ca80 con 0x7f18b813df80 2024-01-15T18:17:37.972 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.970+0000 7f18b7e00640 1 -- 172.21.15.26:0/3907338279 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b002cbe0 con 0x7f18b813df80 2024-01-15T18:17:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.970+0000 7f18b7e00640 1 -- 172.21.15.26:0/3907338279 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b00343f0 con 0x7f18b813df80 2024-01-15T18:17:37.974 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.971+0000 7f18c0200640 1 -- 172.21.15.26:0/3907338279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813df80 msgr2=0x7f18b815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.971+0000 7f18c0200640 1 --2- 172.21.15.26:0/3907338279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813df80 0x7f18b815bee0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009e50 tx=0x7f18b002bee0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.972+0000 7f18c0200640 1 -- 172.21.15.26:0/3907338279 shutdown_connections 2024-01-15T18:17:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.972+0000 7f18c0200640 1 --2- 172.21.15.26:0/3907338279 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b813e360 0x7f18b80cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.972+0000 7f18c0200640 1 --2- 172.21.15.26:0/3907338279 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813df80 0x7f18b815bee0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.975 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.972+0000 7f18c0200640 1 -- 172.21.15.26:0/3907338279 >> 172.21.15.26:0/3907338279 conn(0x7f18b806a800 msgr2=0x7f18b806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:37.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.974+0000 7f18c0200640 1 -- 172.21.15.26:0/3907338279 shutdown_connections 2024-01-15T18:17:37.976 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.974+0000 7f18c0200640 1 -- 172.21.15.26:0/3907338279 wait complete. 2024-01-15T18:17:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.977+0000 7f18c0200640 1 Processor -- start 2024-01-15T18:17:37.980 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.977+0000 7f18c0200640 1 -- start start 2024-01-15T18:17:37.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.979+0000 7f18c0200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b813df80 0x7f18b813c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.979+0000 7f18c0200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813e360 0x7f18b80af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.979+0000 7f18c0200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b815f9c0 0x7f18b80af590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.979+0000 7f18c0200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b814a4a0 con 0x7f18b813e360 2024-01-15T18:17:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.979+0000 7f18c0200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b814ae40 con 0x7f18b815f9c0 2024-01-15T18:17:37.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.979+0000 7f18c0200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b813ddb0 con 0x7f18b813df80 2024-01-15T18:17:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.979+0000 7f18bce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813e360 0x7f18b80af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.980+0000 7f18bce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813e360 0x7f18b80af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49000/0 (socket says 172.21.15.26:49000) 2024-01-15T18:17:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.980+0000 7f18bce00640 1 -- 172.21.15.26:0/3592156915 learned_addr learned my addr 172.21.15.26:0/3592156915 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:37.983 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.980+0000 7f18bce00640 1 -- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b813df80 msgr2=0x7f18b813c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.980+0000 7f18bce00640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b813df80 0x7f18b813c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.980+0000 7f18bce00640 1 -- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b815f9c0 msgr2=0x7f18b80af590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.980+0000 7f18bce00640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b815f9c0 0x7f18b80af590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.980+0000 7f18bce00640 1 -- 172.21.15.26:0/3592156915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b0009b30 con 0x7f18b813e360 2024-01-15T18:17:37.984 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.981+0000 7f18bce00640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813e360 0x7f18b80af050 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f18a800b4a0 tx=0x7f18a800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.981+0000 7f18b5600640 1 -- 172.21.15.26:0/3592156915 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8010610 con 0x7f18b813e360 2024-01-15T18:17:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.981+0000 7f18b5600640 1 -- 172.21.15.26:0/3592156915 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a8010770 con 0x7f18b813e360 2024-01-15T18:17:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.981+0000 7f18b5600640 1 -- 172.21.15.26:0/3592156915 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a8010b20 con 0x7f18b813e360 2024-01-15T18:17:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.981+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b813dad0 con 0x7f18b813e360 2024-01-15T18:17:37.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.981+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b814aa60 con 0x7f18b813e360 2024-01-15T18:17:37.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.984+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b8067000 con 0x7f18b813e360 2024-01-15T18:17:37.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.985+0000 7f18b5600640 1 -- 172.21.15.26:0/3592156915 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f18a8010c80 con 0x7f18b813e360 2024-01-15T18:17:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.986+0000 7f18b5600640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f189006c550 0x7f189006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:37.989 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.987+0000 7f18b5600640 1 -- 172.21.15.26:0/3592156915 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f18a80883b0 con 0x7f18b813e360 2024-01-15T18:17:37.990 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.988+0000 7f18bd800640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f189006c550 0x7f189006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:37.994 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.991+0000 7f18bd800640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f189006c550 0x7f189006ea10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f18b0035000 tx=0x7f18b002e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:37.996 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:37.994+0000 7f18b5600640 1 -- 172.21.15.26:0/3592156915 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18a8054cd0 con 0x7f18b813e360 2024-01-15T18:17:38.325 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.323+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f18b80675d0 con 0x7f18b813e360 2024-01-15T18:17:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.323+0000 7f18b5600640 1 -- 172.21.15.26:0/3592156915 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v117) v1 ==== 80+0+0 (secure 0 0 0) 0x7f18a801e020 con 0x7f18b813e360 2024-01-15T18:17:38.326 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T18:17:38.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f189006c550 msgr2=0x7f189006ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:38.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f189006c550 0x7f189006ea10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f18b0035000 tx=0x7f18b002e000 comp rx=0 tx=0).stop 2024-01-15T18:17:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813e360 msgr2=0x7f18b80af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:38.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813e360 0x7f18b80af050 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f18a800b4a0 tx=0x7f18a800b960 comp rx=0 tx=0).stop 2024-01-15T18:17:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 shutdown_connections 2024-01-15T18:17:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f189006c550 0x7f189006ea10 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18b815f9c0 0x7f18b80af590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f18b813e360 0x7f18b80af050 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 --2- 172.21.15.26:0/3592156915 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18b813df80 0x7f18b813c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 >> 172.21.15.26:0/3592156915 conn(0x7f18b806a800 msgr2=0x7f18b815bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.326+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 shutdown_connections 2024-01-15T18:17:38.330 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:38.327+0000 7f18c0200640 1 -- 172.21.15.26:0/3592156915 wait complete. 2024-01-15T18:17:38.358 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi026.front.sepia.ceph.com... 2024-01-15T18:17:38.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:38.358 DEBUG:teuthology.orchestra.run.smithi026:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-01-15T18:17:38.463 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi026.stderr:ceph-fuse[76436]: fuse finished with error 0 and tester_r 0 2024-01-15T18:17:39.391 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:17:39.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:17:39.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:39.392 DEBUG:teuthology.orchestra.run.smithi026:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-15T18:17:39.455 DEBUG:teuthology.orchestra.run.smithi026:> set -ex 2024-01-15T18:17:39.455 DEBUG:teuthology.orchestra.run.smithi026:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:39.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054633,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250024k,nr_inodes=812506,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500052k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-01-15T18:17:39.519 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:17:39.519 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:39.560 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:17:39.561 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:39.641 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:39.853 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.851+0000 7fef9f400640 1 Processor -- start 2024-01-15T18:17:39.854 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.851+0000 7fef9f400640 1 -- start start 2024-01-15T18:17:39.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.853+0000 7fef9f400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fef9813fcb0 0x7fef98164d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.853+0000 7fef9f400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef980cdc40 con 0x7fef9813cfa0 2024-01-15T18:17:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.853+0000 7fef9f400640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef980cdec0 con 0x7fef9815afb0 2024-01-15T18:17:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.853+0000 7fef9f400640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef980ce1a0 con 0x7fef9813fcb0 2024-01-15T18:17:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.853+0000 7fef9e000640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fef9813cfa0 0x7fef980cd530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41638/0 (socket says 172.21.15.26:41638) 2024-01-15T18:17:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.853+0000 7fef9e000640 1 -- 172.21.15.26:0/1430601924 learned_addr learned my addr 172.21.15.26:0/1430601924 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:39.856 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.853+0000 7fef9ea00640 1 --2- 172.21.15.26:0/1430601924 >> v2:172.21.15.26:3300/0 conn(0x7fef9813fcb0 0x7fef98164d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.854+0000 7fef9ea00640 1 -- 172.21.15.26:0/1430601924 >> v1:172.21.15.124:6789/0 conn(0x7fef9815afb0 legacy=0x7fef9813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.854+0000 7fef9ea00640 1 -- 172.21.15.26:0/1430601924 >> v1:172.21.15.26:6789/0 conn(0x7fef9813cfa0 legacy=0x7fef980cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.854+0000 7fef9ea00640 1 -- 172.21.15.26:0/1430601924 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef980ce480 con 0x7fef9813fcb0 2024-01-15T18:17:39.857 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.855+0000 7fef9ea00640 1 --2- 172.21.15.26:0/1430601924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 0x7fef98164d80 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fef90009d90 tx=0x7fef9002e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8c50156d8d8905 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:39.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.855+0000 7fef9cc00640 1 -- 172.21.15.26:0/1430601924 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9002ede0 con 0x7fef9813fcb0 2024-01-15T18:17:39.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.855+0000 7fef9cc00640 1 -- 172.21.15.26:0/1430601924 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef90036440 con 0x7fef9813fcb0 2024-01-15T18:17:39.858 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.855+0000 7fef9cc00640 1 -- 172.21.15.26:0/1430601924 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef900367d0 con 0x7fef9813fcb0 2024-01-15T18:17:39.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.855+0000 7fef9f400640 1 -- 172.21.15.26:0/1430601924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 msgr2=0x7fef98164d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:39.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.855+0000 7fef9f400640 1 --2- 172.21.15.26:0/1430601924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 0x7fef98164d80 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fef90009d90 tx=0x7fef9002e030 comp rx=0 tx=0).stop 2024-01-15T18:17:39.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.856+0000 7fef9f400640 1 -- 172.21.15.26:0/1430601924 shutdown_connections 2024-01-15T18:17:39.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.856+0000 7fef9f400640 1 --2- 172.21.15.26:0/1430601924 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 0x7fef98164d80 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.856+0000 7fef9f400640 1 -- 172.21.15.26:0/1430601924 >> 172.21.15.26:0/1430601924 conn(0x7fef9806a800 msgr2=0x7fef9806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:39.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.856+0000 7fef9f400640 1 -- 172.21.15.26:0/1430601924 shutdown_connections 2024-01-15T18:17:39.859 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.856+0000 7fef9f400640 1 -- 172.21.15.26:0/1430601924 wait complete. 2024-01-15T18:17:39.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.857+0000 7fef9f400640 1 Processor -- start 2024-01-15T18:17:39.860 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.858+0000 7fef9f400640 1 -- start start 2024-01-15T18:17:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.858+0000 7fef9f400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef9813cfa0 0x7fef9813f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.858+0000 7fef9f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 0x7fef98153c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.858+0000 7fef9f400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef9815afb0 0x7fef981e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.858+0000 7fef9f400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef98154160 con 0x7fef9813fcb0 2024-01-15T18:17:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.858+0000 7fef9f400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef981e6460 con 0x7fef9813cfa0 2024-01-15T18:17:39.861 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.858+0000 7fef9f400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef981e6740 con 0x7fef9815afb0 2024-01-15T18:17:39.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 0x7fef98153c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9d600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 0x7fef98153c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49028/0 (socket says 172.21.15.26:49028) 2024-01-15T18:17:39.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9d600640 1 -- 172.21.15.26:0/393467339 learned_addr learned my addr 172.21.15.26:0/393467339 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9ea00640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef9815afb0 0x7fef981e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9d600640 1 -- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef9815afb0 msgr2=0x7fef981e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9d600640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef9815afb0 0x7fef981e53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9d600640 1 -- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef9813cfa0 msgr2=0x7fef9813f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9d600640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef9813cfa0 0x7fef9813f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:39.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9d600640 1 -- 172.21.15.26:0/393467339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef90009a70 con 0x7fef9813fcb0 2024-01-15T18:17:39.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.859+0000 7fef9d600640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 0x7fef98153c20 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fef8000b3f0 tx=0x7fef8000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:39.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.860+0000 7fef96000640 1 -- 172.21.15.26:0/393467339 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80011830 con 0x7fef9813fcb0 2024-01-15T18:17:39.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.860+0000 7fef96000640 1 -- 172.21.15.26:0/393467339 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef80011990 con 0x7fef9813fcb0 2024-01-15T18:17:39.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.860+0000 7fef96000640 1 -- 172.21.15.26:0/393467339 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80011d40 con 0x7fef9813fcb0 2024-01-15T18:17:39.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.860+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef981e6a20 con 0x7fef9813fcb0 2024-01-15T18:17:39.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.860+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9813abc0 con 0x7fef9813fcb0 2024-01-15T18:17:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.861+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef98067450 con 0x7fef9813fcb0 2024-01-15T18:17:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.862+0000 7fef96000640 1 -- 172.21.15.26:0/393467339 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fef8001a400 con 0x7fef9813fcb0 2024-01-15T18:17:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.863+0000 7fef96000640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef7006c580 0x7fef7006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:39.865 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.863+0000 7fef96000640 1 -- 172.21.15.26:0/393467339 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fef80088990 con 0x7fef9813fcb0 2024-01-15T18:17:39.866 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.864+0000 7fef9e000640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef7006c580 0x7fef7006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:39.867 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.865+0000 7fef9e000640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef7006c580 0x7fef7006ea40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fef880015f0 tx=0x7fef88007a40 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:39.870 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:39.867+0000 7fef96000640 1 -- 172.21.15.26:0/393467339 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef80054ed0 con 0x7fef9813fcb0 2024-01-15T18:17:40.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.054+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef980ae380 con 0x7fef9813fcb0 2024-01-15T18:17:40.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.058+0000 7fef96000640 1 -- 172.21.15.26:0/393467339 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16729 (secure 0 0 0) 0x7fef80015020 con 0x7fef9813fcb0 2024-01-15T18:17:40.061 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:17:40.061 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":117,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:12:53.309154+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:12:08.696115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:12:09.497434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.30","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef7006c580 msgr2=0x7fef7006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef7006c580 0x7fef7006ea40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fef880015f0 tx=0x7fef88007a40 comp rx=0 tx=0).stop 2024-01-15T18:17:40.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 msgr2=0x7fef98153c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 0x7fef98153c20 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fef8000b3f0 tx=0x7fef8000b8b0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 shutdown_connections 2024-01-15T18:17:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fef9815afb0 0x7fef981e53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fef7006c580 0x7fef7006ea40 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fef9813fcb0 0x7fef98153c20 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 --2- 172.21.15.26:0/393467339 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fef9813cfa0 0x7fef9813f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 >> 172.21.15.26:0/393467339 conn(0x7fef9806a800 msgr2=0x7fef9813c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:40.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.060+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 shutdown_connections 2024-01-15T18:17:40.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.061+0000 7fef9f400640 1 -- 172.21.15.26:0/393467339 wait complete. 2024-01-15T18:17:40.082 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:12:08.696115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:12:09.497434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:17:40.083 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:17:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.300+0000 7f2c76000640 1 Processor -- start 2024-01-15T18:17:40.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.300+0000 7f2c76000640 1 -- start start 2024-01-15T18:17:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.302+0000 7f2c76000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 0x7f2c700b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.302+0000 7f2c76000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2c7006cb10 0x7f2c7006cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.302+0000 7f2c76000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c700708d0 con 0x7f2c70070160 2024-01-15T18:17:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.302+0000 7f2c76000640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c70070bb0 con 0x7f2c7006cb10 2024-01-15T18:17:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.302+0000 7f2c76000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70070e90 con 0x7f2c7006c730 2024-01-15T18:17:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.302+0000 7f2c75600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2c7006cb10 0x7f2c7006cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.302+0000 7f2c75600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f2c7006cb10 0x7f2c7006cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49042/0 (socket says 172.21.15.26:49042) 2024-01-15T18:17:40.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.302+0000 7f2c75600640 1 -- 172.21.15.26:0/2872947520 learned_addr learned my addr 172.21.15.26:0/2872947520 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:40.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.302+0000 7f2c74c00640 1 --2- 172.21.15.26:0/2872947520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 0x7f2c700b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.303+0000 7f2c75600640 1 -- 172.21.15.26:0/2872947520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 msgr2=0x7f2c700b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.303+0000 7f2c75600640 1 --2- 172.21.15.26:0/2872947520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 0x7f2c700b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.303+0000 7f2c75600640 1 -- 172.21.15.26:0/2872947520 >> v1:172.21.15.124:6789/0 conn(0x7f2c70070160 legacy=0x7f2c70071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.303+0000 7f2c75600640 1 -- 172.21.15.26:0/2872947520 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c70071170 con 0x7f2c7006cb10 2024-01-15T18:17:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.304+0000 7f2c75600640 1 --2- 172.21.15.26:0/2872947520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 0x7f2c7006cf80 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009e50 tx=0x7f2c6802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb2c3033b7afdd4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.304+0000 7f2c6fe00640 1 -- 172.21.15.26:0/2872947520 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6802ca80 con 0x7f2c7006cb10 2024-01-15T18:17:40.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.304+0000 7f2c6fe00640 1 -- 172.21.15.26:0/2872947520 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6802cbe0 con 0x7f2c7006cb10 2024-01-15T18:17:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.304+0000 7f2c6fe00640 1 -- 172.21.15.26:0/2872947520 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c680343f0 con 0x7f2c7006cb10 2024-01-15T18:17:40.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.305+0000 7f2c76000640 1 -- 172.21.15.26:0/2872947520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 msgr2=0x7f2c7006cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.305+0000 7f2c76000640 1 --2- 172.21.15.26:0/2872947520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 0x7f2c7006cf80 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009e50 tx=0x7f2c6802bee0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.305+0000 7f2c76000640 1 -- 172.21.15.26:0/2872947520 shutdown_connections 2024-01-15T18:17:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.305+0000 7f2c76000640 1 --2- 172.21.15.26:0/2872947520 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 0x7f2c7006cf80 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.305+0000 7f2c76000640 1 --2- 172.21.15.26:0/2872947520 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 0x7f2c700b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.305+0000 7f2c76000640 1 -- 172.21.15.26:0/2872947520 >> 172.21.15.26:0/2872947520 conn(0x7f2c7006a7d0 msgr2=0x7f2c7006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.305+0000 7f2c76000640 1 -- 172.21.15.26:0/2872947520 shutdown_connections 2024-01-15T18:17:40.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.305+0000 7f2c76000640 1 -- 172.21.15.26:0/2872947520 wait complete. 2024-01-15T18:17:40.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.307+0000 7f2c76000640 1 Processor -- start 2024-01-15T18:17:40.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.307+0000 7f2c76000640 1 -- start start 2024-01-15T18:17:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.308+0000 7f2c76000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 0x7f2c700a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.308+0000 7f2c76000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 0x7f2c70144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.308+0000 7f2c76000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c70070160 0x7f2c700ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.308+0000 7f2c76000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70146650 con 0x7f2c7006cb10 2024-01-15T18:17:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.308+0000 7f2c76000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70146930 con 0x7f2c70070160 2024-01-15T18:17:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.308+0000 7f2c76000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70146c10 con 0x7f2c7006c730 2024-01-15T18:17:40.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 0x7f2c70144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c67e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 0x7f2c70144130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49056/0 (socket says 172.21.15.26:49056) 2024-01-15T18:17:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c67e00640 1 -- 172.21.15.26:0/2646141601 learned_addr learned my addr 172.21.15.26:0/2646141601 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c75600640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c70070160 0x7f2c700ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c74c00640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 0x7f2c700a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c67e00640 1 -- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 msgr2=0x7f2c700a8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c67e00640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 0x7f2c700a8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c67e00640 1 -- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c70070160 msgr2=0x7f2c700ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c67e00640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c70070160 0x7f2c700ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.309+0000 7f2c67e00640 1 -- 172.21.15.26:0/2646141601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c68009b30 con 0x7f2c7006cb10 2024-01-15T18:17:40.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.310+0000 7f2c67e00640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 0x7f2c70144130 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800b3f0 tx=0x7f2c5800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.310+0000 7f2c6d600640 1 -- 172.21.15.26:0/2646141601 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58011830 con 0x7f2c7006cb10 2024-01-15T18:17:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.310+0000 7f2c6d600640 1 -- 172.21.15.26:0/2646141601 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58011990 con 0x7f2c7006cb10 2024-01-15T18:17:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.310+0000 7f2c6d600640 1 -- 172.21.15.26:0/2646141601 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58011d40 con 0x7f2c7006cb10 2024-01-15T18:17:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.310+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c70146ef0 con 0x7f2c7006cb10 2024-01-15T18:17:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.310+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c700a93c0 con 0x7f2c7006cb10 2024-01-15T18:17:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.311+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c70147110 con 0x7f2c7006cb10 2024-01-15T18:17:40.315 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.312+0000 7f2c6d600640 1 -- 172.21.15.26:0/2646141601 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f2c5801a400 con 0x7f2c7006cb10 2024-01-15T18:17:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.313+0000 7f2c6d600640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c4806c550 0x7f2c4806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.313+0000 7f2c6d600640 1 -- 172.21.15.26:0/2646141601 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f2c580888a0 con 0x7f2c7006cb10 2024-01-15T18:17:40.316 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.314+0000 7f2c74c00640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c4806c550 0x7f2c4806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.315+0000 7f2c74c00640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c4806c550 0x7f2c4806ea10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2c60006fd0 tx=0x7f2c60008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:40.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.317+0000 7f2c6d600640 1 -- 172.21.15.26:0/2646141601 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c58054e90 con 0x7f2c7006cb10 2024-01-15T18:17:40.522 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.520+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c70146790 con 0x7f2c7006cb10 2024-01-15T18:17:40.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.521+0000 7f2c6d600640 1 -- 172.21.15.26:0/2646141601 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c70146790 con 0x7f2c7006cb10 2024-01-15T18:17:40.523 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:17:40.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c4806c550 msgr2=0x7f2c4806ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c4806c550 0x7f2c4806ea10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f2c60006fd0 tx=0x7f2c60008040 comp rx=0 tx=0).stop 2024-01-15T18:17:40.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 msgr2=0x7f2c70144130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 0x7f2c70144130 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800b3f0 tx=0x7f2c5800b8b0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.526 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 shutdown_connections 2024-01-15T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f2c4806c550 0x7f2c4806ea10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f2c70070160 0x7f2c700ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f2c7006cb10 0x7f2c70144130 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 --2- 172.21.15.26:0/2646141601 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f2c7006c730 0x7f2c700a8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 >> 172.21.15.26:0/2646141601 conn(0x7f2c7006a7d0 msgr2=0x7f2c70144aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.523+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 shutdown_connections 2024-01-15T18:17:40.527 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.524+0000 7f2c76000640 1 -- 172.21.15.26:0/2646141601 wait complete. 2024-01-15T18:17:40.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:17:40.765 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.763+0000 7f1843e00640 1 Processor -- start 2024-01-15T18:17:40.766 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.763+0000 7f1843e00640 1 -- start start 2024-01-15T18:17:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.765+0000 7f1843e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f184415d960 0x7f18440cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.765+0000 7f1843e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18440b20d0 0x7f18440b0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.765+0000 7f1843e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18440ce760 con 0x7f184415d580 2024-01-15T18:17:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.765+0000 7f1843e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18440d0a00 con 0x7f184415d960 2024-01-15T18:17:40.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.765+0000 7f1843e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18440cf250 con 0x7f18440b20d0 2024-01-15T18:17:40.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.765+0000 7f1839e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f184415d960 0x7f18440cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.765+0000 7f1843400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18440b20d0 0x7f18440b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.765+0000 7f1842a00640 1 --1- >> v1:172.21.15.124:6789/0 conn(0x7f184415d580 0x7f18440cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.124:6789/0 says I am v1:172.21.15.26:53662/0 (socket says 172.21.15.26:53662) 2024-01-15T18:17:40.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.765+0000 7f1842a00640 1 -- 172.21.15.26:0/1373334230 learned_addr learned my addr 172.21.15.26:0/1373334230 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:40.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.766+0000 7f1843400640 1 -- 172.21.15.26:0/1373334230 >> v2:172.21.15.124:3300/0 conn(0x7f184415d960 msgr2=0x7f18440cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:40.769 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.766+0000 7f1843400640 1 --2- 172.21.15.26:0/1373334230 >> v2:172.21.15.124:3300/0 conn(0x7f184415d960 0x7f18440cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.766+0000 7f1842000640 1 -- 172.21.15.26:0/1373334230 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359240686 0 0) 0x7f18440ce760 con 0x7f184415d580 2024-01-15T18:17:40.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.766+0000 7f1843400640 1 -- 172.21.15.26:0/1373334230 >> v1:172.21.15.124:6789/0 conn(0x7f184415d580 legacy=0x7f18440cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.766+0000 7f1843400640 1 -- 172.21.15.26:0/1373334230 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18440d1f50 con 0x7f18440b20d0 2024-01-15T18:17:40.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.768+0000 7f1843400640 1 --2- 172.21.15.26:0/1373334230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18440b20d0 0x7f18440b0000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f183c009d90 tx=0x7f183c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f25d627fd871339 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:40.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.768+0000 7f1842000640 1 -- 172.21.15.26:0/1373334230 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c02cbc0 con 0x7f18440b20d0 2024-01-15T18:17:40.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.768+0000 7f1842000640 1 -- 172.21.15.26:0/1373334230 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183c02cd20 con 0x7f18440b20d0 2024-01-15T18:17:40.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.768+0000 7f1842000640 1 -- 172.21.15.26:0/1373334230 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c034660 con 0x7f18440b20d0 2024-01-15T18:17:40.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.769+0000 7f1843e00640 1 -- 172.21.15.26:0/1373334230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18440b20d0 msgr2=0x7f18440b0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.769+0000 7f1843e00640 1 --2- 172.21.15.26:0/1373334230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18440b20d0 0x7f18440b0000 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f183c009d90 tx=0x7f183c02be10 comp rx=0 tx=0).stop 2024-01-15T18:17:40.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.769+0000 7f1843e00640 1 -- 172.21.15.26:0/1373334230 shutdown_connections 2024-01-15T18:17:40.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.769+0000 7f1843e00640 1 --2- 172.21.15.26:0/1373334230 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f18440b20d0 0x7f18440b0000 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.769+0000 7f1843e00640 1 --2- 172.21.15.26:0/1373334230 >> v2:172.21.15.124:3300/0 conn(0x7f184415d960 0x7f18440cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.769+0000 7f1843e00640 1 -- 172.21.15.26:0/1373334230 >> 172.21.15.26:0/1373334230 conn(0x7f18440d3040 msgr2=0x7f18440d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.769+0000 7f1843e00640 1 -- 172.21.15.26:0/1373334230 shutdown_connections 2024-01-15T18:17:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.769+0000 7f1843e00640 1 -- 172.21.15.26:0/1373334230 wait complete. 2024-01-15T18:17:40.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.771+0000 7f1843e00640 1 Processor -- start 2024-01-15T18:17:40.774 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.771+0000 7f1843e00640 1 -- start start 2024-01-15T18:17:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.772+0000 7f1843e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18440b20d0 0x7f18440cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.772+0000 7f1843e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184415d580 0x7f18441f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.772+0000 7f1843e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f184415d960 0x7f1844143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.772+0000 7f1843e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440b6020 con 0x7f184415d580 2024-01-15T18:17:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.772+0000 7f1843e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18440b7420 con 0x7f18440b20d0 2024-01-15T18:17:40.776 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.772+0000 7f1843e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1844143b50 con 0x7f184415d960 2024-01-15T18:17:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1839e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184415d580 0x7f18441f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1839e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184415d580 0x7f18441f2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49062/0 (socket says 172.21.15.26:49062) 2024-01-15T18:17:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1839e00640 1 -- 172.21.15.26:0/1781138458 learned_addr learned my addr 172.21.15.26:0/1781138458 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1843400640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f184415d960 0x7f1844143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1842a00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18440b20d0 0x7f18440cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1839e00640 1 -- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f184415d960 msgr2=0x7f1844143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1839e00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f184415d960 0x7f1844143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1839e00640 1 -- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18440b20d0 msgr2=0x7f18440cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1839e00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18440b20d0 0x7f18440cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.773+0000 7f1839e00640 1 -- 172.21.15.26:0/1781138458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183c009a70 con 0x7f184415d580 2024-01-15T18:17:40.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.774+0000 7f1839e00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184415d580 0x7f18441f2610 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f182c00e870 tx=0x7f182c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:40.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.774+0000 7f183be00640 1 -- 172.21.15.26:0/1781138458 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c0094f0 con 0x7f184415d580 2024-01-15T18:17:40.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.774+0000 7f183be00640 1 -- 172.21.15.26:0/1781138458 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182c00c940 con 0x7f184415d580 2024-01-15T18:17:40.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.774+0000 7f183be00640 1 -- 172.21.15.26:0/1781138458 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00ccf0 con 0x7f184415d580 2024-01-15T18:17:40.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.774+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18440b76c0 con 0x7f184415d580 2024-01-15T18:17:40.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.774+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18440b7c80 con 0x7f184415d580 2024-01-15T18:17:40.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.776+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1844067450 con 0x7f184415d580 2024-01-15T18:17:40.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.777+0000 7f183be00640 1 -- 172.21.15.26:0/1781138458 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f182c013470 con 0x7f184415d580 2024-01-15T18:17:40.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.778+0000 7f183be00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f181c06c550 0x7f181c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:40.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.778+0000 7f183be00640 1 -- 172.21.15.26:0/1781138458 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f182c088810 con 0x7f184415d580 2024-01-15T18:17:40.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.779+0000 7f1842a00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f181c06c550 0x7f181c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:40.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.780+0000 7f1842a00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f181c06c550 0x7f181c06ea10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1834007f70 tx=0x7f1834007f00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:40.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.783+0000 7f183be00640 1 -- 172.21.15.26:0/1781138458 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f182c055180 con 0x7f184415d580 2024-01-15T18:17:40.981 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.979+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18440cdcb0 con 0x7f184415d580 2024-01-15T18:17:40.982 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.980+0000 7f183be00640 1 -- 172.21.15.26:0/1781138458 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f182c017020 con 0x7f184415d580 2024-01-15T18:17:40.983 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T18:17:40.985 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f181c06c550 msgr2=0x7f181c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f181c06c550 0x7f181c06ea10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1834007f70 tx=0x7f1834007f00 comp rx=0 tx=0).stop 2024-01-15T18:17:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184415d580 msgr2=0x7f18441f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184415d580 0x7f18441f2610 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f182c00e870 tx=0x7f182c00ed30 comp rx=0 tx=0).stop 2024-01-15T18:17:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 shutdown_connections 2024-01-15T18:17:40.986 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f184415d960 0x7f1844143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f181c06c550 0x7f181c06ea10 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f184415d580 0x7f18441f2610 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 --2- 172.21.15.26:0/1781138458 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f18440b20d0 0x7f18440cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 >> 172.21.15.26:0/1781138458 conn(0x7f18440d3040 msgr2=0x7f1844152510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 shutdown_connections 2024-01-15T18:17:40.987 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:40.983+0000 7f1843e00640 1 -- 172.21.15.26:0/1781138458 wait complete. 2024-01-15T18:17:41.004 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:17:41.231 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.229+0000 7f5e05200640 1 Processor -- start 2024-01-15T18:17:41.232 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.229+0000 7f5e05200640 1 -- start start 2024-01-15T18:17:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.231+0000 7f5e05200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000ccb50 0x7f5e00070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.231+0000 7f5e05200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0015f4e0 con 0x7f5e0015bf10 2024-01-15T18:17:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.231+0000 7f5e05200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0015c3c0 con 0x7f5e000adf20 2024-01-15T18:17:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.231+0000 7f5e05200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0015f970 con 0x7f5e000ccb50 2024-01-15T18:17:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.231+0000 7f5dffe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5e0015bf10 0x7f5e00143d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41652/0 (socket says 172.21.15.26:41652) 2024-01-15T18:17:41.234 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.231+0000 7f5dffe00640 1 -- 172.21.15.26:0/3064123571 learned_addr learned my addr 172.21.15.26:0/3064123571 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfea00640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000ccb50 0x7f5e00070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.236 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541784589 0 0) 0x7f5e0015f4e0 con 0x7f5e0015bf10 2024-01-15T18:17:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de0003520 con 0x7f5e0015bf10 2024-01-15T18:17:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44119869 0 0) 0x7f5de0003520 con 0x7f5e0015bf10 2024-01-15T18:17:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000ccb50 msgr2=0x7f5e00070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000ccb50 0x7f5e00070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 >> v1:172.21.15.124:6789/0 conn(0x7f5e000adf20 legacy=0x7f5e000cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e00071130 con 0x7f5e0015bf10 2024-01-15T18:17:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f5df80019f0 con 0x7f5e0015bf10 2024-01-15T18:17:41.237 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 >> v1:172.21.15.26:6789/0 conn(0x7f5e0015bf10 legacy=0x7f5e00143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5de00039a0 0x7f5de0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000ccb50 0x7f5de00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5de0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0015f970 con 0x7f5de0004d00 2024-01-15T18:17:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de000bb10 con 0x7f5de00039a0 2024-01-15T18:17:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.232+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de000bcd0 con 0x7f5e000ccb50 2024-01-15T18:17:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.233+0000 7f5dffe00640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5de0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.233+0000 7f5dff400640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5de00039a0 0x7f5de0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.238 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.233+0000 7f5dfea00640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000ccb50 0x7f5de00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.233+0000 7f5dffe00640 1 -- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000ccb50 msgr2=0x7f5de00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.233+0000 7f5dffe00640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000ccb50 0x7f5de00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.233+0000 7f5dffe00640 1 -- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5de00039a0 msgr2=0x7f5de0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.233+0000 7f5dffe00640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5de00039a0 0x7f5de0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.233+0000 7f5dffe00640 1 -- 172.21.15.26:0/3064123571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de0003520 con 0x7f5de0004d00 2024-01-15T18:17:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.233+0000 7f5dfea00640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000ccb50 0x7f5de00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:41.239 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.234+0000 7f5dffe00640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5de0005180 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f5df800a730 tx=0x7f5de00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263f5b69fa18552c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8034630 con 0x7f5de0004d00 2024-01-15T18:17:41.242 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df802f070 con 0x7f5de0004d00 2024-01-15T18:17:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5dfe000640 1 -- 172.21.15.26:0/3064123571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df8034a10 con 0x7f5de0004d00 2024-01-15T18:17:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5e05200640 1 -- 172.21.15.26:0/3064123571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 msgr2=0x7f5de0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5e05200640 1 --2- 172.21.15.26:0/3064123571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5de0005180 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f5df800a730 tx=0x7f5de00057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5dffe00640 1 -- 172.21.15.26:0/3064123571 reap_dead start 2024-01-15T18:17:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5e05200640 1 -- 172.21.15.26:0/3064123571 shutdown_connections 2024-01-15T18:17:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5e05200640 1 -- 172.21.15.26:0/3064123571 >> 172.21.15.26:0/3064123571 conn(0x7f5e0006a7d0 msgr2=0x7f5e0006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5e05200640 1 -- 172.21.15.26:0/3064123571 shutdown_connections 2024-01-15T18:17:41.243 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.235+0000 7f5e05200640 1 -- 172.21.15.26:0/3064123571 wait complete. 2024-01-15T18:17:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.237+0000 7f5e05200640 1 Processor -- start 2024-01-15T18:17:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.237+0000 7f5e05200640 1 -- start start 2024-01-15T18:17:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.238+0000 7f5e05200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5e001454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.238+0000 7f5e05200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e000af3a0 0x7f5e000b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.238+0000 7f5e05200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000af820 0x7f5e000b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.238+0000 7f5e05200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000b1010 con 0x7f5de0004d00 2024-01-15T18:17:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.238+0000 7f5e05200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000b1240 con 0x7f5e000af3a0 2024-01-15T18:17:41.244 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.238+0000 7f5e05200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000b1520 con 0x7f5e000af820 2024-01-15T18:17:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.238+0000 7f5dff400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5e001454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dff400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5e001454e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49070/0 (socket says 172.21.15.26:49070) 2024-01-15T18:17:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dff400640 1 -- 172.21.15.26:0/1764401590 learned_addr learned my addr 172.21.15.26:0/1764401590 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dff400640 1 -- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000af820 msgr2=0x7f5e000b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dff400640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000af820 0x7f5e000b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dff400640 1 -- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e000af3a0 msgr2=0x7f5e000b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dfea00640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e000af3a0 0x7f5e000b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.245 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dff400640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e000af3a0 0x7f5e000b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dff400640 1 -- 172.21.15.26:0/1764401590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e00152d80 con 0x7f5de0004d00 2024-01-15T18:17:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dfea00640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e000af3a0 0x7f5e000b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.239+0000 7f5dff400640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5e001454e0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e4d0 tx=0x7f5df000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.240+0000 7f5df7400640 1 -- 172.21.15.26:0/1764401590 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df00052e0 con 0x7f5de0004d00 2024-01-15T18:17:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.240+0000 7f5df7400640 1 -- 172.21.15.26:0/1764401590 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df0005440 con 0x7f5de0004d00 2024-01-15T18:17:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.240+0000 7f5df7400640 1 -- 172.21.15.26:0/1764401590 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df0003810 con 0x7f5de0004d00 2024-01-15T18:17:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.240+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e00153060 con 0x7f5de0004d00 2024-01-15T18:17:41.246 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.240+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e00151a40 con 0x7f5de0004d00 2024-01-15T18:17:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.241+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e00067450 con 0x7f5de0004d00 2024-01-15T18:17:41.247 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.242+0000 7f5df7400640 1 -- 172.21.15.26:0/1764401590 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f5df0003970 con 0x7f5de0004d00 2024-01-15T18:17:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.243+0000 7f5df7400640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5dd806c550 0x7f5dd806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.243+0000 7f5df7400640 1 -- 172.21.15.26:0/1764401590 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f5df0088830 con 0x7f5de0004d00 2024-01-15T18:17:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.244+0000 7f5dfea00640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5dd806c550 0x7f5dd806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.248 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.245+0000 7f5dfea00640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5dd806c550 0x7f5dd806ea10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5e001574b0 tx=0x7f5de8008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.249 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.247+0000 7f5df7400640 1 -- 172.21.15.26:0/1764401590 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5df0054d70 con 0x7f5de0004d00 2024-01-15T18:17:41.445 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.443+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5e00157200 con 0x7f5de0004d00 2024-01-15T18:17:41.446 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.444+0000 7f5df7400640 1 -- 172.21.15.26:0/1764401590 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5df0058390 con 0x7f5de0004d00 2024-01-15T18:17:41.447 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T18:17:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5dd806c550 msgr2=0x7f5dd806ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5dd806c550 0x7f5dd806ea10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5e001574b0 tx=0x7f5de8008040 comp rx=0 tx=0).stop 2024-01-15T18:17:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 msgr2=0x7f5e001454e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.450 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5e001454e0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f5df000e4d0 tx=0x7f5df000e990 comp rx=0 tx=0).stop 2024-01-15T18:17:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 shutdown_connections 2024-01-15T18:17:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5e000af820 0x7f5e000b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5dd806c550 0x7f5dd806ea10 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5e000af3a0 0x7f5e000b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 --2- 172.21.15.26:0/1764401590 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5de0004d00 0x7f5e001454e0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 >> 172.21.15.26:0/1764401590 conn(0x7f5e0006a7d0 msgr2=0x7f5e000ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 shutdown_connections 2024-01-15T18:17:41.451 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.447+0000 7f5e05200640 1 -- 172.21.15.26:0/1764401590 wait complete. 2024-01-15T18:17:41.468 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:17:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.670+0000 7f29aca00640 1 Processor -- start 2024-01-15T18:17:41.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.670+0000 7f29aca00640 1 -- start start 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.672+0000 7f29aca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a409a170 0x7f29a40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.672+0000 7f29aca00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29a406c6b0 0x7f29a40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.672+0000 7f29aca00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a406cea0 con 0x7f29a40cd240 2024-01-15T18:17:41.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.672+0000 7f29aca00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a406cbc0 con 0x7f29a406c6b0 2024-01-15T18:17:41.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.672+0000 7f29aca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a406f350 con 0x7f29a409a170 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.673+0000 7f29aaa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29a406c6b0 0x7f29a40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.673+0000 7f29a9600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a409a170 0x7f29a40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.673+0000 7f29aaa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f29a406c6b0 0x7f29a40cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3300/0 says I am v2:172.21.15.26:50716/0 (socket says 172.21.15.26:50716) 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.673+0000 7f29aaa00640 1 -- 172.21.15.26:0/513336129 learned_addr learned my addr 172.21.15.26:0/513336129 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.674+0000 7f29aaa00640 1 -- 172.21.15.26:0/513336129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a409a170 msgr2=0x7f29a40b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:41.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.674+0000 7f29aaa00640 1 --2- 172.21.15.26:0/513336129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a409a170 0x7f29a40b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.674+0000 7f29aaa00640 1 -- 172.21.15.26:0/513336129 >> v1:172.21.15.124:6789/0 conn(0x7f29a40cd240 legacy=0x7f29a4099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.674+0000 7f29aaa00640 1 -- 172.21.15.26:0/513336129 --> v2:172.21.15.124:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a406f630 con 0x7f29a406c6b0 2024-01-15T18:17:41.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.675+0000 7f29aaa00640 1 --2- 172.21.15.26:0/513336129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a406c6b0 0x7f29a40cf640 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f299c009870 tx=0x7f299c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca84b65073e3f15 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.675+0000 7f29a8c00640 1 -- 172.21.15.26:0/513336129 <== mon.1 v2:172.21.15.124:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c030840 con 0x7f29a406c6b0 2024-01-15T18:17:41.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.675+0000 7f29a8c00640 1 -- 172.21.15.26:0/513336129 <== mon.1 v2:172.21.15.124:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299c0309a0 con 0x7f29a406c6b0 2024-01-15T18:17:41.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.675+0000 7f29a8c00640 1 -- 172.21.15.26:0/513336129 <== mon.1 v2:172.21.15.124:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c030d30 con 0x7f29a406c6b0 2024-01-15T18:17:41.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.676+0000 7f29aca00640 1 -- 172.21.15.26:0/513336129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a406c6b0 msgr2=0x7f29a40cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.676+0000 7f29aca00640 1 --2- 172.21.15.26:0/513336129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a406c6b0 0x7f29a40cf640 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f299c009870 tx=0x7f299c02b980 comp rx=0 tx=0).stop 2024-01-15T18:17:41.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.676+0000 7f29aca00640 1 -- 172.21.15.26:0/513336129 shutdown_connections 2024-01-15T18:17:41.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.676+0000 7f29aca00640 1 --2- 172.21.15.26:0/513336129 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a409a170 0x7f29a40b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.676+0000 7f29aca00640 1 --2- 172.21.15.26:0/513336129 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a406c6b0 0x7f29a40cf640 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.676+0000 7f29aca00640 1 -- 172.21.15.26:0/513336129 >> 172.21.15.26:0/513336129 conn(0x7f29a40b2d00 msgr2=0x7f29a40b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:41.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.676+0000 7f29aca00640 1 -- 172.21.15.26:0/513336129 shutdown_connections 2024-01-15T18:17:41.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.676+0000 7f29aca00640 1 -- 172.21.15.26:0/513336129 wait complete. 2024-01-15T18:17:41.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.678+0000 7f29aca00640 1 Processor -- start 2024-01-15T18:17:41.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.678+0000 7f29aca00640 1 -- start start 2024-01-15T18:17:41.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.679+0000 7f29aca00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a406c6b0 0x7f29a4159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.682 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.679+0000 7f29aca00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a409a170 0x7f29a40c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.679+0000 7f29aca00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a40cd240 0x7f29a40c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.679+0000 7f29aca00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40c9ea0 con 0x7f29a406c6b0 2024-01-15T18:17:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.679+0000 7f29aca00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40ca180 con 0x7f29a409a170 2024-01-15T18:17:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.679+0000 7f29aca00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40ca460 con 0x7f29a40cd240 2024-01-15T18:17:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.679+0000 7f29aa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a406c6b0 0x7f29a4159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.679+0000 7f29aa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a406c6b0 0x7f29a4159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49074/0 (socket says 172.21.15.26:49074) 2024-01-15T18:17:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.679+0000 7f29aa000640 1 -- 172.21.15.26:0/3513681597 learned_addr learned my addr 172.21.15.26:0/3513681597 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:41.683 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29aaa00640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a40cd240 0x7f29a40c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29a9600640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a409a170 0x7f29a40c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29aa000640 1 -- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a40cd240 msgr2=0x7f29a40c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29aa000640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a40cd240 0x7f29a40c9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29aa000640 1 -- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a409a170 msgr2=0x7f29a40c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29aa000640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a409a170 0x7f29a40c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29aa000640 1 -- 172.21.15.26:0/3513681597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c0094d0 con 0x7f29a406c6b0 2024-01-15T18:17:41.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29aaa00640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a40cd240 0x7f29a40c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:41.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29a9600640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a409a170 0x7f29a40c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:41.685 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29aa000640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a406c6b0 0x7f29a4159b50 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f298c00c400 tx=0x7f298c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29a2000640 1 -- 172.21.15.26:0/3513681597 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c0052d0 con 0x7f29a406c6b0 2024-01-15T18:17:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29a2000640 1 -- 172.21.15.26:0/3513681597 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c005450 con 0x7f29a406c6b0 2024-01-15T18:17:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.680+0000 7f29a2000640 1 -- 172.21.15.26:0/3513681597 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c007260 con 0x7f29a406c6b0 2024-01-15T18:17:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.681+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a40ca740 con 0x7f29a406c6b0 2024-01-15T18:17:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.681+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a40b6540 con 0x7f29a406c6b0 2024-01-15T18:17:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.682+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a40b3820 con 0x7f29a406c6b0 2024-01-15T18:17:41.686 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.683+0000 7f29a2000640 1 -- 172.21.15.26:0/3513681597 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f298c0073c0 con 0x7f29a406c6b0 2024-01-15T18:17:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.684+0000 7f29a2000640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f297c06c550 0x7f297c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.684+0000 7f29a2000640 1 -- 172.21.15.26:0/3513681597 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f298c087f90 con 0x7f29a406c6b0 2024-01-15T18:17:41.687 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.685+0000 7f29a9600640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f297c06c550 0x7f297c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:41.688 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.686+0000 7f29a9600640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f297c06c550 0x7f297c06ea10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2994007520 tx=0x7f299400b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:41.691 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.688+0000 7f29a2000640 1 -- 172.21.15.26:0/3513681597 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f298c0544d0 con 0x7f29a406c6b0 2024-01-15T18:17:41.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.898+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f29a40cac60 con 0x7f29a406c6b0 2024-01-15T18:17:41.902 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.899+0000 7f29a2000640 1 -- 172.21.15.26:0/3513681597 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v117) v1 ==== 80+0+0 (secure 0 0 0) 0x7f298c057af0 con 0x7f29a406c6b0 2024-01-15T18:17:41.902 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T18:17:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f297c06c550 msgr2=0x7f297c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f297c06c550 0x7f297c06ea10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f2994007520 tx=0x7f299400b040 comp rx=0 tx=0).stop 2024-01-15T18:17:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a406c6b0 msgr2=0x7f29a4159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a406c6b0 0x7f29a4159b50 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f298c00c400 tx=0x7f298c00c8c0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 shutdown_connections 2024-01-15T18:17:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f29a40cd240 0x7f29a40c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f297c06c550 0x7f297c06ea10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f29a409a170 0x7f29a40c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 --2- 172.21.15.26:0/3513681597 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f29a406c6b0 0x7f29a4159b50 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 >> 172.21.15.26:0/3513681597 conn(0x7f29a40b2d00 msgr2=0x7f29a4099a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 shutdown_connections 2024-01-15T18:17:41.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:41.902+0000 7f29aca00640 1 -- 172.21.15.26:0/3513681597 wait complete. 2024-01-15T18:17:41.923 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:17:41.923 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:41.980 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:42.195 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.193+0000 7fcc73e00640 1 Processor -- start 2024-01-15T18:17:42.196 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.193+0000 7fcc73e00640 1 -- start start 2024-01-15T18:17:42.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.194+0000 7fcc73e00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fcc74143c40 0x7fcc74071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.194+0000 7fcc73e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc740717d0 0x7fcc740cc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.197 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.194+0000 7fcc73e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc74070e50 con 0x7fcc740cca40 2024-01-15T18:17:42.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.194+0000 7fcc73e00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc740723c0 con 0x7fcc74143c40 2024-01-15T18:17:42.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.194+0000 7fcc73e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc740cce70 con 0x7fcc740717d0 2024-01-15T18:17:42.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.195+0000 7fcc73400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fcc740cca40 0x7fcc7415eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57668/0 (socket says 172.21.15.26:57668) 2024-01-15T18:17:42.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.195+0000 7fcc73400640 1 -- 172.21.15.26:0/1979989475 learned_addr learned my addr 172.21.15.26:0/1979989475 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:42.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.195+0000 7fcc69e00640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc740717d0 0x7fcc740cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.198 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.195+0000 7fcc72a00640 1 --2- 172.21.15.26:0/1979989475 >> v2:172.21.15.124:3300/0 conn(0x7fcc74143c40 0x7fcc74071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.195+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052078397 0 0) 0x7fcc74070e50 con 0x7fcc740cca40 2024-01-15T18:17:42.200 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc54003520 con 0x7fcc740cca40 2024-01-15T18:17:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1400713934 0 0) 0x7fcc54003520 con 0x7fcc740cca40 2024-01-15T18:17:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc740717d0 msgr2=0x7fcc740cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc740717d0 0x7fcc740cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 >> v2:172.21.15.124:3300/0 conn(0x7fcc74143c40 msgr2=0x7fcc74071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 --2- 172.21.15.26:0/1979989475 >> v2:172.21.15.124:3300/0 conn(0x7fcc74143c40 0x7fcc74071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7406e170 con 0x7fcc740cca40 2024-01-15T18:17:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fcc6c001670 con 0x7fcc740cca40 2024-01-15T18:17:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 >> v1:172.21.15.26:6789/0 conn(0x7fcc740cca40 legacy=0x7fcc7415eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.201 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc740717d0 0x7fcc540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc54004d00 0x7fcc54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc740723c0 con 0x7fcc54003970 2024-01-15T18:17:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc740cce70 con 0x7fcc54004d00 2024-01-15T18:17:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5400b9b0 con 0x7fcc740717d0 2024-01-15T18:17:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.196+0000 7fcc72a00640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.197+0000 7fcc72a00640 1 -- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc740717d0 msgr2=0x7fcc540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.197+0000 7fcc69e00640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc740717d0 0x7fcc540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.202 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.197+0000 7fcc72a00640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc740717d0 0x7fcc540047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.197+0000 7fcc72a00640 1 -- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc54004d00 msgr2=0x7fcc54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.197+0000 7fcc72a00640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc54004d00 0x7fcc54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.197+0000 7fcc72a00640 1 -- 172.21.15.26:0/1979989475 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54003520 con 0x7fcc54003970 2024-01-15T18:17:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.198+0000 7fcc72a00640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc54003d90 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009ac0 tx=0x7fcc540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d398d742633a71 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.198+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6402c710 con 0x7fcc54003970 2024-01-15T18:17:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.198+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6402c870 con 0x7fcc54003970 2024-01-15T18:17:42.203 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.198+0000 7fcc72000640 1 -- 172.21.15.26:0/1979989475 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6402cc00 con 0x7fcc54003970 2024-01-15T18:17:42.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.199+0000 7fcc73e00640 1 -- 172.21.15.26:0/1979989475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 msgr2=0x7fcc54003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.199+0000 7fcc73e00640 1 --2- 172.21.15.26:0/1979989475 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc54003d90 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009ac0 tx=0x7fcc540057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.199+0000 7fcc73400640 1 -- 172.21.15.26:0/1979989475 reap_dead start 2024-01-15T18:17:42.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.199+0000 7fcc73e00640 1 -- 172.21.15.26:0/1979989475 shutdown_connections 2024-01-15T18:17:42.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.199+0000 7fcc73e00640 1 -- 172.21.15.26:0/1979989475 >> 172.21.15.26:0/1979989475 conn(0x7fcc7406a800 msgr2=0x7fcc7406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.199+0000 7fcc73e00640 1 -- 172.21.15.26:0/1979989475 shutdown_connections 2024-01-15T18:17:42.206 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.199+0000 7fcc73e00640 1 -- 172.21.15.26:0/1979989475 wait complete. 2024-01-15T18:17:42.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.201+0000 7fcc73e00640 1 Processor -- start 2024-01-15T18:17:42.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.201+0000 7fcc73e00640 1 -- start start 2024-01-15T18:17:42.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc73e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc741424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc73e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc7413ee90 0x7fcc74140070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc73e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc7406d060 0x7fcc7406d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc73e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7406da50 con 0x7fcc54003970 2024-01-15T18:17:42.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc73e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc741429f0 con 0x7fcc7406d060 2024-01-15T18:17:42.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc73e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc741409e0 con 0x7fcc7413ee90 2024-01-15T18:17:42.207 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc72a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc741424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc72a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc741424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56674/0 (socket says 172.21.15.26:56674) 2024-01-15T18:17:42.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc72a00640 1 -- 172.21.15.26:0/2004318908 learned_addr learned my addr 172.21.15.26:0/2004318908 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:42.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc72a00640 1 -- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc7413ee90 msgr2=0x7fcc74140070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc72a00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc7413ee90 0x7fcc74140070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc72a00640 1 -- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc7406d060 msgr2=0x7fcc7406d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc72a00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc7406d060 0x7fcc7406d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.202+0000 7fcc72a00640 1 -- 172.21.15.26:0/2004318908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc74140cc0 con 0x7fcc54003970 2024-01-15T18:17:42.208 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.203+0000 7fcc72a00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc741424b0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fcc64037040 tx=0x7fcc64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.203+0000 7fcc6b400640 1 -- 172.21.15.26:0/2004318908 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64003810 con 0x7fcc54003970 2024-01-15T18:17:42.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.203+0000 7fcc6b400640 1 -- 172.21.15.26:0/2004318908 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6402de90 con 0x7fcc54003970 2024-01-15T18:17:42.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.203+0000 7fcc6b400640 1 -- 172.21.15.26:0/2004318908 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64003b70 con 0x7fcc54003970 2024-01-15T18:17:42.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.203+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc74140fa0 con 0x7fcc54003970 2024-01-15T18:17:42.209 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.203+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc74141560 con 0x7fcc54003970 2024-01-15T18:17:42.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.204+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc74141720 con 0x7fcc54003970 2024-01-15T18:17:42.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.205+0000 7fcc6b400640 1 -- 172.21.15.26:0/2004318908 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fcc64003cd0 con 0x7fcc54003970 2024-01-15T18:17:42.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.206+0000 7fcc6b400640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc4c06c580 0x7fcc4c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.210 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.206+0000 7fcc6b400640 1 -- 172.21.15.26:0/2004318908 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fcc640aa680 con 0x7fcc54003970 2024-01-15T18:17:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.207+0000 7fcc69e00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc4c06c580 0x7fcc4c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.211 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.208+0000 7fcc69e00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc4c06c580 0x7fcc4c06ea40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fcc74141d60 tx=0x7fcc5c009750 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.212 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.210+0000 7fcc6b400640 1 -- 172.21.15.26:0/2004318908 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc64077110 con 0x7fcc54003970 2024-01-15T18:17:42.413 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.411+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc74067450 con 0x7fcc54003970 2024-01-15T18:17:42.415 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.412+0000 7fcc6b400640 1 -- 172.21.15.26:0/2004318908 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16729 (secure 0 0 0) 0x7fcc6404c090 con 0x7fcc54003970 2024-01-15T18:17:42.415 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:17:42.416 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":117,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:12:53.309154+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:12:08.696115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:12:09.497434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.30","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.415+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc4c06c580 msgr2=0x7fcc4c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.415+0000 7fcc73e00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc4c06c580 0x7fcc4c06ea40 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fcc74141d60 tx=0x7fcc5c009750 comp rx=0 tx=0).stop 2024-01-15T18:17:42.418 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.415+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 msgr2=0x7fcc741424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.415+0000 7fcc73e00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc741424b0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fcc64037040 tx=0x7fcc64004730 comp rx=0 tx=0).stop 2024-01-15T18:17:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.416+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 shutdown_connections 2024-01-15T18:17:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.416+0000 7fcc73e00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fcc7413ee90 0x7fcc74140070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.416+0000 7fcc73e00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fcc4c06c580 0x7fcc4c06ea40 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.416+0000 7fcc73e00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fcc7406d060 0x7fcc7406d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.416+0000 7fcc73e00640 1 --2- 172.21.15.26:0/2004318908 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fcc54003970 0x7fcc741424b0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.416+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 >> 172.21.15.26:0/2004318908 conn(0x7fcc7406a800 msgr2=0x7fcc7415a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.416+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 shutdown_connections 2024-01-15T18:17:42.419 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.416+0000 7fcc73e00640 1 -- 172.21.15.26:0/2004318908 wait complete. 2024-01-15T18:17:42.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-15T17:54:57.981100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-15T18:12:08.696115+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-01-15T18:12:09.497434+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '115', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-15T18:17:42.437 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-15T18:17:42.664 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.662+0000 7fa2f3000640 1 Processor -- start 2024-01-15T18:17:42.665 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.662+0000 7fa2f3000640 1 -- start start 2024-01-15T18:17:42.666 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.664+0000 7fa2f3000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2ec0adf20 0x7fa2ec070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.664+0000 7fa2f3000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec15f4e0 con 0x7fa2ec15bf10 2024-01-15T18:17:42.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.664+0000 7fa2f3000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec071130 con 0x7fa2ec0ccb50 2024-01-15T18:17:42.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.664+0000 7fa2f3000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec15f970 con 0x7fa2ec0adf20 2024-01-15T18:17:42.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.664+0000 7fa2f1200640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fa2ec15bf10 0x7fa2ec0cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57682/0 (socket says 172.21.15.26:57682) 2024-01-15T18:17:42.667 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.664+0000 7fa2f1200640 1 -- 172.21.15.26:0/975161902 learned_addr learned my addr 172.21.15.26:0/975161902 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2f1c00640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2ec0adf20 0x7fa2ec070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728519726 0 0) 0x7fa2ec15f4e0 con 0x7fa2ec15bf10 2024-01-15T18:17:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2cc003520 con 0x7fa2ec15bf10 2024-01-15T18:17:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2714450361 0 0) 0x7fa2cc003520 con 0x7fa2ec15bf10 2024-01-15T18:17:42.669 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2ec0adf20 msgr2=0x7fa2ec070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2ec0adf20 0x7fa2ec070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 >> v1:172.21.15.124:6789/0 conn(0x7fa2ec0ccb50 legacy=0x7fa2ec143d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec15c390 con 0x7fa2ec15bf10 2024-01-15T18:17:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fa2d4001670 con 0x7fa2ec15bf10 2024-01-15T18:17:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 >> v1:172.21.15.26:6789/0 conn(0x7fa2ec15bf10 legacy=0x7fa2ec0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.665+0000 7fa2ebe00640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2cc0039a0 0x7fa2cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2ebe00640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2cc0047c0 0x7fa2cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.670 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2ebe00640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2ec0adf20 0x7fa2cc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec15f970 con 0x7fa2cc0047c0 2024-01-15T18:17:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc00bb10 con 0x7fa2cc0039a0 2024-01-15T18:17:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc00bcd0 con 0x7fa2ec0adf20 2024-01-15T18:17:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2f1200640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2cc0047c0 0x7fa2cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2f1c00640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2cc0039a0 0x7fa2cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2f2600640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2ec0adf20 0x7fa2cc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2f1200640 1 -- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2ec0adf20 msgr2=0x7fa2cc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2f1200640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2ec0adf20 0x7fa2cc005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.671 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2f1200640 1 -- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2cc0039a0 msgr2=0x7fa2cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:42.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2f1200640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2cc0039a0 0x7fa2cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.672 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.666+0000 7fa2f1200640 1 -- 172.21.15.26:0/975161902 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc003520 con 0x7fa2cc0047c0 2024-01-15T18:17:42.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.668+0000 7fa2f1200640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2cc0047c0 0x7fa2cc004c40 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400c0e0 tx=0x7fa2cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a04928c7c62a25 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.668+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4005ad0 con 0x7fa2cc0047c0 2024-01-15T18:17:42.673 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.668+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d4005c30 con 0x7fa2cc0047c0 2024-01-15T18:17:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.668+0000 7fa2ebe00640 1 -- 172.21.15.26:0/975161902 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d402c7d0 con 0x7fa2cc0047c0 2024-01-15T18:17:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.669+0000 7fa2f3000640 1 -- 172.21.15.26:0/975161902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2cc0047c0 msgr2=0x7fa2cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.669+0000 7fa2f3000640 1 --2- 172.21.15.26:0/975161902 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2cc0047c0 0x7fa2cc004c40 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400c0e0 tx=0x7fa2cc0057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.669+0000 7fa2f2600640 1 -- 172.21.15.26:0/975161902 reap_dead start 2024-01-15T18:17:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.669+0000 7fa2f3000640 1 -- 172.21.15.26:0/975161902 shutdown_connections 2024-01-15T18:17:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.669+0000 7fa2f3000640 1 -- 172.21.15.26:0/975161902 >> 172.21.15.26:0/975161902 conn(0x7fa2ec06a7d0 msgr2=0x7fa2ec06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.669+0000 7fa2f3000640 1 -- 172.21.15.26:0/975161902 shutdown_connections 2024-01-15T18:17:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.669+0000 7fa2f3000640 1 -- 172.21.15.26:0/975161902 wait complete. 2024-01-15T18:17:42.674 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.671+0000 7fa2f3000640 1 Processor -- start 2024-01-15T18:17:42.675 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.671+0000 7fa2f3000640 1 -- start start 2024-01-15T18:17:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.672+0000 7fa2f3000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2cc0047c0 0x7fa2ec13ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.672+0000 7fa2f3000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ec0ad0a0 0x7fa2ec0a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.676 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.672+0000 7fa2f3000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2ec0ca450 0x7fa2ec0aa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.672+0000 7fa2f3000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0acc10 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.672+0000 7fa2f3000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0ac630 con 0x7fa2ec0ca450 2024-01-15T18:17:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.672+0000 7fa2f3000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0caca0 con 0x7fa2cc0047c0 2024-01-15T18:17:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.672+0000 7fa2f1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ec0ad0a0 0x7fa2ec0a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.672+0000 7fa2f1200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ec0ad0a0 0x7fa2ec0a95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56682/0 (socket says 172.21.15.26:56682) 2024-01-15T18:17:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.673+0000 7fa2f1200640 1 -- 172.21.15.26:0/3429484569 learned_addr learned my addr 172.21.15.26:0/3429484569 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.673+0000 7fa2f1200640 1 -- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2cc0047c0 msgr2=0x7fa2ec13ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.673+0000 7fa2f1200640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2cc0047c0 0x7fa2ec13ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.677 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.673+0000 7fa2f1200640 1 -- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2ec0ca450 msgr2=0x7fa2ec0aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.673+0000 7fa2f1200640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2ec0ca450 0x7fa2ec0aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.673+0000 7fa2f1200640 1 -- 172.21.15.26:0/3429484569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0ac8a0 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.673+0000 7fa2f1200640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ec0ad0a0 0x7fa2ec0a95d0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400c200 tx=0x7fa2ec15b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.673+0000 7fa2e9600640 1 -- 172.21.15.26:0/3429484569 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4005a40 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.674+0000 7fa2e9600640 1 -- 172.21.15.26:0/3429484569 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d402dda0 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.674+0000 7fa2e9600640 1 -- 172.21.15.26:0/3429484569 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4007270 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.674+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec0caf80 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.678 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.674+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec0aa9c0 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.675+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec0ac490 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.676+0000 7fa2e9600640 1 -- 172.21.15.26:0/3429484569 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fa2d4032070 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.679 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.677+0000 7fa2e9600640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa2c406c550 0x7fa2c406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.677+0000 7fa2e9600640 1 -- 172.21.15.26:0/3429484569 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fa2d40a9f30 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.680 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.678+0000 7fa2f1c00640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa2c406c550 0x7fa2c406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:42.681 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.679+0000 7fa2f1c00640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa2c406c550 0x7fa2c406ea10 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009910 tx=0x7fa2dc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:42.684 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.682+0000 7fa2e9600640 1 -- 172.21.15.26:0/3429484569 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2ec0ac490 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.878 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.875+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2ec0ac490 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.879 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.877+0000 7fa2e9600640 1 -- 172.21.15.26:0/3429484569 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2ec0ac490 con 0x7fa2ec0ad0a0 2024-01-15T18:17:42.879 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 1 2024-01-15T18:17:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.879+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa2c406c550 msgr2=0x7fa2c406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.879+0000 7fa2f3000640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa2c406c550 0x7fa2c406ea10 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009910 tx=0x7fa2dc008040 comp rx=0 tx=0).stop 2024-01-15T18:17:42.882 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.879+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ec0ad0a0 msgr2=0x7fa2ec0a95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.879+0000 7fa2f3000640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ec0ad0a0 0x7fa2ec0a95d0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400c200 tx=0x7fa2ec15b490 comp rx=0 tx=0).stop 2024-01-15T18:17:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.880+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 shutdown_connections 2024-01-15T18:17:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.880+0000 7fa2f3000640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fa2c406c550 0x7fa2c406ea10 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.880+0000 7fa2f3000640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fa2ec0ca450 0x7fa2ec0aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.880+0000 7fa2f3000640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fa2ec0ad0a0 0x7fa2ec0a95d0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.880+0000 7fa2f3000640 1 --2- 172.21.15.26:0/3429484569 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fa2cc0047c0 0x7fa2ec13ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.880+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 >> 172.21.15.26:0/3429484569 conn(0x7fa2ec06a7d0 msgr2=0x7fa2ec071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:42.883 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.880+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 shutdown_connections 2024-01-15T18:17:42.884 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:42.880+0000 7fa2f3000640 1 -- 172.21.15.26:0/3429484569 wait complete. 2024-01-15T18:17:42.902 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-15T18:17:43.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.126+0000 7f87e2200640 1 Processor -- start 2024-01-15T18:17:43.129 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.126+0000 7f87e2200640 1 -- start start 2024-01-15T18:17:43.130 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.128+0000 7f87e2200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc143810 0x7f87dc071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.128+0000 7f87e2200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f87dc0ae170 0x7f87dc15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.128+0000 7f87e2200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc06c730 con 0x7f87dc15d770 2024-01-15T18:17:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.128+0000 7f87e2200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87dc143bf0 con 0x7f87dc0ae170 2024-01-15T18:17:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.128+0000 7f87e2200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0ae5f0 con 0x7f87dc143810 2024-01-15T18:17:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.128+0000 7f87e1800640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f87dc15d770 0x7f87dc15dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57690/0 (socket says 172.21.15.26:57690) 2024-01-15T18:17:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.128+0000 7f87e1800640 1 -- 172.21.15.26:0/3789890056 learned_addr learned my addr 172.21.15.26:0/3789890056 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:43.131 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.128+0000 7f87d3c00640 1 --2- 172.21.15.26:0/3789890056 >> v2:172.21.15.124:3300/0 conn(0x7f87dc0ae170 0x7f87dc15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.128+0000 7f87e0e00640 1 --2- 172.21.15.26:0/3789890056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc143810 0x7f87dc071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.129+0000 7f87dbe00640 1 -- 172.21.15.26:0/3789890056 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955618914 0 0) 0x7f87dc06c730 con 0x7f87dc15d770 2024-01-15T18:17:43.132 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.129+0000 7f87dbe00640 1 -- 172.21.15.26:0/3789890056 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87bc003520 con 0x7f87dc15d770 2024-01-15T18:17:43.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.129+0000 7f87e0e00640 1 -- 172.21.15.26:0/3789890056 >> v2:172.21.15.124:3300/0 conn(0x7f87dc0ae170 msgr2=0x7f87dc15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:43.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.129+0000 7f87e0e00640 1 --2- 172.21.15.26:0/3789890056 >> v2:172.21.15.124:3300/0 conn(0x7f87dc0ae170 0x7f87dc15e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.129+0000 7f87e0e00640 1 -- 172.21.15.26:0/3789890056 >> v1:172.21.15.26:6789/0 conn(0x7f87dc15d770 legacy=0x7f87dc15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.133 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.129+0000 7f87e0e00640 1 -- 172.21.15.26:0/3789890056 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc0724d0 con 0x7f87dc143810 2024-01-15T18:17:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.131+0000 7f87e0e00640 1 --2- 172.21.15.26:0/3789890056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc143810 0x7f87dc071e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097a0 tx=0x7f87cc02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acda0874895ccdab server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.131+0000 7f87dbe00640 1 -- 172.21.15.26:0/3789890056 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc031780 con 0x7f87dc143810 2024-01-15T18:17:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.131+0000 7f87dbe00640 1 -- 172.21.15.26:0/3789890056 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc0318e0 con 0x7f87dc143810 2024-01-15T18:17:43.134 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.131+0000 7f87dbe00640 1 -- 172.21.15.26:0/3789890056 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc031c70 con 0x7f87dc143810 2024-01-15T18:17:43.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.132+0000 7f87e2200640 1 -- 172.21.15.26:0/3789890056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc143810 msgr2=0x7f87dc071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.132+0000 7f87e2200640 1 --2- 172.21.15.26:0/3789890056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc143810 0x7f87dc071e40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f87cc0097a0 tx=0x7f87cc02b820 comp rx=0 tx=0).stop 2024-01-15T18:17:43.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.132+0000 7f87e2200640 1 -- 172.21.15.26:0/3789890056 shutdown_connections 2024-01-15T18:17:43.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.132+0000 7f87e2200640 1 --2- 172.21.15.26:0/3789890056 >> v2:172.21.15.124:3300/0 conn(0x7f87dc0ae170 0x7f87dc15e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.135 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.132+0000 7f87e2200640 1 --2- 172.21.15.26:0/3789890056 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc143810 0x7f87dc071e40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.132+0000 7f87e2200640 1 -- 172.21.15.26:0/3789890056 >> 172.21.15.26:0/3789890056 conn(0x7f87dc06a7d0 msgr2=0x7f87dc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:43.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.132+0000 7f87e2200640 1 -- 172.21.15.26:0/3789890056 shutdown_connections 2024-01-15T18:17:43.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.132+0000 7f87e2200640 1 -- 172.21.15.26:0/3789890056 wait complete. 2024-01-15T18:17:43.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.134+0000 7f87e2200640 1 Processor -- start 2024-01-15T18:17:43.136 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.134+0000 7f87e2200640 1 -- start start 2024-01-15T18:17:43.137 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.135+0000 7f87e2200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc0ae170 0x7f87dc1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.135+0000 7f87e2200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc143810 0x7f87dc0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.135+0000 7f87e2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15d770 0x7f87dc0cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.135+0000 7f87e2200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc1f4030 con 0x7f87dc15d770 2024-01-15T18:17:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.135+0000 7f87e2200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0c4a90 con 0x7f87dc143810 2024-01-15T18:17:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.135+0000 7f87e2200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0c3e80 con 0x7f87dc0ae170 2024-01-15T18:17:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.135+0000 7f87e1800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15d770 0x7f87dc0cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.138 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.135+0000 7f87e1800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15d770 0x7f87dc0cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56692/0 (socket says 172.21.15.26:56692) 2024-01-15T18:17:43.139 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.135+0000 7f87e1800640 1 -- 172.21.15.26:0/1247294057 learned_addr learned my addr 172.21.15.26:0/1247294057 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:43.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87d3c00640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc143810 0x7f87dc0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87e0e00640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc0ae170 0x7f87dc1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87e1800640 1 -- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc0ae170 msgr2=0x7f87dc1f3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87e1800640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc0ae170 0x7f87dc1f3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.140 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87e1800640 1 -- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc143810 msgr2=0x7f87dc0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87e1800640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc143810 0x7f87dc0ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87e1800640 1 -- 172.21.15.26:0/1247294057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc009400 con 0x7f87dc15d770 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87e0e00640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc0ae170 0x7f87dc1f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87d3c00640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc143810 0x7f87dc0ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87e1800640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15d770 0x7f87dc0cbff0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f87d400caa0 tx=0x7f87d400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87d9600640 1 -- 172.21.15.26:0/1247294057 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400ac00 con 0x7f87dc15d770 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87d9600640 1 -- 172.21.15.26:0/1247294057 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d400ad60 con 0x7f87dc15d770 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87d9600640 1 -- 172.21.15.26:0/1247294057 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4012620 con 0x7f87dc15d770 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.136+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc0c4160 con 0x7f87dc15d770 2024-01-15T18:17:43.141 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.137+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc0c4720 con 0x7f87dc15d770 2024-01-15T18:17:43.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.138+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc0c4c00 con 0x7f87dc15d770 2024-01-15T18:17:43.142 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.139+0000 7f87d9600640 1 -- 172.21.15.26:0/1247294057 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f87d4007070 con 0x7f87dc15d770 2024-01-15T18:17:43.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.140+0000 7f87d9600640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f87b406c550 0x7f87b406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.140+0000 7f87d9600640 1 -- 172.21.15.26:0/1247294057 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f87d4088b80 con 0x7f87dc15d770 2024-01-15T18:17:43.143 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.141+0000 7f87e0e00640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f87b406c550 0x7f87b406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.144 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.142+0000 7f87e0e00640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f87b406c550 0x7f87b406ea10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f87cc036000 tx=0x7f87cc03d040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:43.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.145+0000 7f87d9600640 1 -- 172.21.15.26:0/1247294057 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f87d40550c0 con 0x7f87dc15d770 2024-01-15T18:17:43.342 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.339+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87dc067450 con 0x7f87dc15d770 2024-01-15T18:17:43.343 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.340+0000 7f87d9600640 1 -- 172.21.15.26:0/1247294057 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v117) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87d40170a0 con 0x7f87dc15d770 2024-01-15T18:17:43.343 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 32 2024-01-15T18:17:43.345 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f87b406c550 msgr2=0x7f87b406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f87b406c550 0x7f87b406ea10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f87cc036000 tx=0x7f87cc03d040 comp rx=0 tx=0).stop 2024-01-15T18:17:43.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15d770 msgr2=0x7f87dc0cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15d770 0x7f87dc0cbff0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f87d400caa0 tx=0x7f87d400cf60 comp rx=0 tx=0).stop 2024-01-15T18:17:43.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 shutdown_connections 2024-01-15T18:17:43.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f87b406c550 0x7f87b406ea10 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f87dc15d770 0x7f87dc0cbff0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f87dc143810 0x7f87dc0ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 --2- 172.21.15.26:0/1247294057 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f87dc0ae170 0x7f87dc1f3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 >> 172.21.15.26:0/1247294057 conn(0x7f87dc06a7d0 msgr2=0x7f87dc0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:43.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 shutdown_connections 2024-01-15T18:17:43.347 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.343+0000 7f87e2200640 1 -- 172.21.15.26:0/1247294057 wait complete. 2024-01-15T18:17:43.364 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-15T18:17:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.576+0000 7f699aa00640 1 Processor -- start 2024-01-15T18:17:43.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.576+0000 7f699aa00640 1 -- start start 2024-01-15T18:17:43.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.578+0000 7f699aa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699415d240 0x7f69940b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.578+0000 7f699aa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69940b1d20 con 0x7f699406d7d0 2024-01-15T18:17:43.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.578+0000 7f699aa00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69940b38d0 con 0x7f69940af4c0 2024-01-15T18:17:43.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.578+0000 7f699aa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69940b1fc0 con 0x7f699415d240 2024-01-15T18:17:43.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.579+0000 7f6993400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f699406d7d0 0x7f69940aedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57700/0 (socket says 172.21.15.26:57700) 2024-01-15T18:17:43.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.579+0000 7f6993400640 1 -- 172.21.15.26:0/830114593 learned_addr learned my addr 172.21.15.26:0/830114593 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:43.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.579+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433473000 0 0) 0x7f69940b1d20 con 0x7f699406d7d0 2024-01-15T18:17:43.583 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.579+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6974003520 con 0x7f699406d7d0 2024-01-15T18:17:43.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.579+0000 7f6993e00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699415d240 0x7f69940b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.579+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3809586046 0 0) 0x7f6974003520 con 0x7f699406d7d0 2024-01-15T18:17:43.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.579+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699415d240 msgr2=0x7f69940b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:43.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.579+0000 7f6992a00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699415d240 0x7f69940b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.579+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 >> v1:172.21.15.124:6789/0 conn(0x7f69940af4c0 legacy=0x7f699415d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69940b22a0 con 0x7f699406d7d0 2024-01-15T18:17:43.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f697c001670 con 0x7f699406d7d0 2024-01-15T18:17:43.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6993e00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699415d240 0x7f69940b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:43.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 >> v1:172.21.15.26:6789/0 conn(0x7f699406d7d0 legacy=0x7f69940aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6992a00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69740039a0 0x7f6974003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6992a00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699415d240 0x7f69740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6992a00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974004d00 0x7f6974005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940b1fc0 con 0x7f6974004d00 2024-01-15T18:17:43.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697400bb10 con 0x7f69740039a0 2024-01-15T18:17:43.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697400bcd0 con 0x7f699415d240 2024-01-15T18:17:43.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6998a00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974004d00 0x7f6974005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6993e00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69740039a0 0x7f6974003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6993400640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699415d240 0x7f69740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6998a00640 1 -- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699415d240 msgr2=0x7f69740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:43.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.580+0000 7f6998a00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699415d240 0x7f69740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.581+0000 7f6998a00640 1 -- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69740039a0 msgr2=0x7f6974003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:43.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.581+0000 7f6998a00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f69740039a0 0x7f6974003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.581+0000 7f6998a00640 1 -- 172.21.15.26:0/830114593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974003520 con 0x7f6974004d00 2024-01-15T18:17:43.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.582+0000 7f6998a00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974004d00 0x7f6974005180 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f698c001e50 tx=0x7f69740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff8dd4ff73141c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:43.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.582+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c003b70 con 0x7f6974004d00 2024-01-15T18:17:43.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.582+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c003cd0 con 0x7f6974004d00 2024-01-15T18:17:43.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.582+0000 7f6992a00640 1 -- 172.21.15.26:0/830114593 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c005310 con 0x7f6974004d00 2024-01-15T18:17:43.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.582+0000 7f699aa00640 1 -- 172.21.15.26:0/830114593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974004d00 msgr2=0x7f6974005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.583+0000 7f699aa00640 1 --2- 172.21.15.26:0/830114593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6974004d00 0x7f6974005180 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f698c001e50 tx=0x7f69740057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.583+0000 7f6998a00640 1 -- 172.21.15.26:0/830114593 reap_dead start 2024-01-15T18:17:43.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.583+0000 7f699aa00640 1 -- 172.21.15.26:0/830114593 shutdown_connections 2024-01-15T18:17:43.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.583+0000 7f699aa00640 1 -- 172.21.15.26:0/830114593 >> 172.21.15.26:0/830114593 conn(0x7f699406b4f0 msgr2=0x7f699406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.583+0000 7f699aa00640 1 -- 172.21.15.26:0/830114593 shutdown_connections 2024-01-15T18:17:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.583+0000 7f699aa00640 1 -- 172.21.15.26:0/830114593 wait complete. 2024-01-15T18:17:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.584+0000 7f699aa00640 1 Processor -- start 2024-01-15T18:17:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.585+0000 7f699aa00640 1 -- start start 2024-01-15T18:17:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.585+0000 7f699aa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6974004d00 0x7f69940c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f699aa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699414d9c0 0x7f699415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f699aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69940b0b20 0x7f69940b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f699aa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940b14e0 con 0x7f69940b0b20 2024-01-15T18:17:43.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f699aa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69940c4ed0 con 0x7f6974004d00 2024-01-15T18:17:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f699aa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699415c310 con 0x7f699414d9c0 2024-01-15T18:17:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6998a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69940b0b20 0x7f69940b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6998a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69940b0b20 0x7f69940b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56706/0 (socket says 172.21.15.26:56706) 2024-01-15T18:17:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6998a00640 1 -- 172.21.15.26:0/4276398504 learned_addr learned my addr 172.21.15.26:0/4276398504 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6993e00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6974004d00 0x7f69940c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6993400640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699414d9c0 0x7f699415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6998a00640 1 -- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699414d9c0 msgr2=0x7f699415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6998a00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699414d9c0 0x7f699415bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6998a00640 1 -- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6974004d00 msgr2=0x7f69940c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6998a00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6974004d00 0x7f69940c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.586+0000 7f6998a00640 1 -- 172.21.15.26:0/4276398504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69940c5120 con 0x7f69940b0b20 2024-01-15T18:17:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.587+0000 7f6998a00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69940b0b20 0x7f69940b0fa0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f698c001f40 tx=0x7f698c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.587+0000 7f698be00640 1 -- 172.21.15.26:0/4276398504 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c03e070 con 0x7f69940b0b20 2024-01-15T18:17:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.587+0000 7f698be00640 1 -- 172.21.15.26:0/4276398504 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c003e40 con 0x7f69940b0b20 2024-01-15T18:17:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.587+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69940c5400 con 0x7f69940b0b20 2024-01-15T18:17:43.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.587+0000 7f698be00640 1 -- 172.21.15.26:0/4276398504 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c006d70 con 0x7f69940b0b20 2024-01-15T18:17:43.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.588+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69940c59c0 con 0x7f69940b0b20 2024-01-15T18:17:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.589+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69940675d0 con 0x7f69940b0b20 2024-01-15T18:17:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.590+0000 7f698be00640 1 -- 172.21.15.26:0/4276398504 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f698c041070 con 0x7f69940b0b20 2024-01-15T18:17:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.591+0000 7f698be00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696c06c550 0x7f696c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.592+0000 7f698be00640 1 -- 172.21.15.26:0/4276398504 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f698c0a9780 con 0x7f69940b0b20 2024-01-15T18:17:43.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.592+0000 7f6993e00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696c06c550 0x7f696c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:43.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.593+0000 7f6993e00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696c06c550 0x7f696c06ea10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6984006fd0 tx=0x7f6984008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:43.600 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.598+0000 7f698be00640 1 -- 172.21.15.26:0/4276398504 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f698c075fb0 con 0x7f69940b0b20 2024-01-15T18:17:43.796 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.793+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69940725d0 con 0x7f69940b0b20 2024-01-15T18:17:43.797 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.795+0000 7f698be00640 1 -- 172.21.15.26:0/4276398504 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v117) v1 ==== 100+0+11 (secure 0 0 0) 0x7f698c048070 con 0x7f69940b0b20 2024-01-15T18:17:43.798 INFO:teuthology.orchestra.run.smithi026.stdout:pg_num: 64 2024-01-15T18:17:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696c06c550 msgr2=0x7f696c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696c06c550 0x7f696c06ea10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6984006fd0 tx=0x7f6984008040 comp rx=0 tx=0).stop 2024-01-15T18:17:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69940b0b20 msgr2=0x7f69940b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69940b0b20 0x7f69940b0fa0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f698c001f40 tx=0x7f698c0014a0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 shutdown_connections 2024-01-15T18:17:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f699414d9c0 0x7f699415bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f696c06c550 0x7f696c06ea10 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f69940b0b20 0x7f69940b0fa0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 --2- 172.21.15.26:0/4276398504 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6974004d00 0x7f69940c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.797+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 >> 172.21.15.26:0/4276398504 conn(0x7f699406b4f0 msgr2=0x7f699415c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.798+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 shutdown_connections 2024-01-15T18:17:43.801 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:43.798+0000 7f699aa00640 1 -- 172.21.15.26:0/4276398504 wait complete. 2024-01-15T18:17:43.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-15T18:17:44.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.042+0000 7f6bdaa00640 1 Processor -- start 2024-01-15T18:17:44.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.042+0000 7f6bdaa00640 1 -- start start 2024-01-15T18:17:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.044+0000 7f6bdaa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd413df80 0x7f6bd415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.044+0000 7f6bdaa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6bd413e360 0x7f6bd40cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.044+0000 7f6bdaa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd406c470 con 0x7f6bd415f9c0 2024-01-15T18:17:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.044+0000 7f6bdaa00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd406c750 con 0x7f6bd413e360 2024-01-15T18:17:44.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.044+0000 7f6bdaa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd406ca30 con 0x7f6bd413df80 2024-01-15T18:17:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.044+0000 7f6bd8a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6bd415f9c0 0x7f6bd41408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57702/0 (socket says 172.21.15.26:57702) 2024-01-15T18:17:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.044+0000 7f6bd8a00640 1 -- 172.21.15.26:0/3747679456 learned_addr learned my addr 172.21.15.26:0/3747679456 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.044+0000 7f6bd3400640 1 --2- 172.21.15.26:0/3747679456 >> v2:172.21.15.124:3300/0 conn(0x7f6bd413e360 0x7f6bd40cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:44.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.044+0000 7f6bd3e00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd413df80 0x7f6bd415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248304145 0 0) 0x7f6bd406c470 con 0x7f6bd415f9c0 2024-01-15T18:17:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb4003520 con 0x7f6bd415f9c0 2024-01-15T18:17:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203787083 0 0) 0x7f6bb4003520 con 0x7f6bd415f9c0 2024-01-15T18:17:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd413df80 msgr2=0x7f6bd415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:44.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd413df80 0x7f6bd415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 >> v2:172.21.15.124:3300/0 conn(0x7f6bd413e360 msgr2=0x7f6bd40cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 --2- 172.21.15.26:0/3747679456 >> v2:172.21.15.124:3300/0 conn(0x7f6bd413e360 0x7f6bd40cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd406cd10 con 0x7f6bd415f9c0 2024-01-15T18:17:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f6bcc0019f0 con 0x7f6bd415f9c0 2024-01-15T18:17:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 >> v1:172.21.15.26:6789/0 conn(0x7f6bd415f9c0 legacy=0x7f6bd41408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:44.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb4003970 0x7f6bb4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd413df80 0x7f6bb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.045+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd406c750 con 0x7f6bb4004d00 2024-01-15T18:17:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.046+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd406ca30 con 0x7f6bb4003970 2024-01-15T18:17:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.046+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb400b980 con 0x7f6bd413df80 2024-01-15T18:17:44.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.046+0000 7f6bd8a00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.046+0000 7f6bd3400640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd413df80 0x7f6bb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.046+0000 7f6bd8a00640 1 -- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd413df80 msgr2=0x7f6bb40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.046+0000 7f6bd8a00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd413df80 0x7f6bb40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.046+0000 7f6bd8a00640 1 -- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb4003970 msgr2=0x7f6bb4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.046+0000 7f6bd8a00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bb4003970 0x7f6bb4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.046+0000 7f6bd8a00640 1 -- 172.21.15.26:0/3747679456 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb4003520 con 0x7f6bb4004d00 2024-01-15T18:17:44.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.047+0000 7f6bd8a00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bb4005180 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00a780 tx=0x7f6bb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bace11eb732d81 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.048+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc034630 con 0x7f6bb4004d00 2024-01-15T18:17:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.048+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc02f070 con 0x7f6bb4004d00 2024-01-15T18:17:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.048+0000 7f6bd2a00640 1 -- 172.21.15.26:0/3747679456 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0349d0 con 0x7f6bb4004d00 2024-01-15T18:17:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.048+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3747679456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 msgr2=0x7f6bb4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.048+0000 7f6bdaa00640 1 --2- 172.21.15.26:0/3747679456 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bb4005180 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00a780 tx=0x7f6bb40057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.048+0000 7f6bd8a00640 1 -- 172.21.15.26:0/3747679456 reap_dead start 2024-01-15T18:17:44.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.048+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3747679456 shutdown_connections 2024-01-15T18:17:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.048+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3747679456 >> 172.21.15.26:0/3747679456 conn(0x7f6bd406a800 msgr2=0x7f6bd406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.048+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3747679456 shutdown_connections 2024-01-15T18:17:44.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.049+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3747679456 wait complete. 2024-01-15T18:17:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.050+0000 7f6bdaa00640 1 Processor -- start 2024-01-15T18:17:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.050+0000 7f6bdaa00640 1 -- start start 2024-01-15T18:17:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.051+0000 7f6bdaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bd415d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.051+0000 7f6bdaa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd4135af0 0x7f6bd41352d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:44.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.051+0000 7f6bdaa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd41506b0 0x7f6bd4150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.051+0000 7f6bdaa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd415e200 con 0x7f6bb4004d00 2024-01-15T18:17:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.051+0000 7f6bdaa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd4151070 con 0x7f6bd4135af0 2024-01-15T18:17:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.051+0000 7f6bdaa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd41f33f0 con 0x7f6bd41506b0 2024-01-15T18:17:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bd415d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bd415d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56726/0 (socket says 172.21.15.26:56726) 2024-01-15T18:17:44.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3e00640 1 -- 172.21.15.26:0/3222314286 learned_addr learned my addr 172.21.15.26:0/3222314286 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3400640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd4135af0 0x7f6bd41352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd8a00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd41506b0 0x7f6bd4150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3e00640 1 -- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd41506b0 msgr2=0x7f6bd4150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3e00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd41506b0 0x7f6bd4150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3e00640 1 -- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd4135af0 msgr2=0x7f6bd41352d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3e00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd4135af0 0x7f6bd41352d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3e00640 1 -- 172.21.15.26:0/3222314286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd414db10 con 0x7f6bb4004d00 2024-01-15T18:17:44.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.052+0000 7f6bd3400640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd4135af0 0x7f6bd41352d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.053+0000 7f6bd3e00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bd415d260 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400e7c0 tx=0x7f6bc400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.053+0000 7f6bcbe00640 1 -- 172.21.15.26:0/3222314286 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400eed0 con 0x7f6bb4004d00 2024-01-15T18:17:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.053+0000 7f6bcbe00640 1 -- 172.21.15.26:0/3222314286 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc400c8c0 con 0x7f6bb4004d00 2024-01-15T18:17:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.053+0000 7f6bcbe00640 1 -- 172.21.15.26:0/3222314286 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400cc70 con 0x7f6bb4004d00 2024-01-15T18:17:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.053+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd4153a20 con 0x7f6bb4004d00 2024-01-15T18:17:44.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.053+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd415dc40 con 0x7f6bb4004d00 2024-01-15T18:17:44.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.054+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd414d8e0 con 0x7f6bb4004d00 2024-01-15T18:17:44.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.055+0000 7f6bcbe00640 1 -- 172.21.15.26:0/3222314286 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f6bc4013470 con 0x7f6bb4004d00 2024-01-15T18:17:44.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.056+0000 7f6bcbe00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6bac06c550 0x7f6bac06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:44.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.056+0000 7f6bcbe00640 1 -- 172.21.15.26:0/3222314286 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f6bc4088920 con 0x7f6bb4004d00 2024-01-15T18:17:44.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.057+0000 7f6bd3400640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6bac06c550 0x7f6bac06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:44.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.058+0000 7f6bd3400640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6bac06c550 0x7f6bac06ea10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6bd413df50 tx=0x7f6bbc008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:44.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.061+0000 7f6bcbe00640 1 -- 172.21.15.26:0/3222314286 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bd414d8e0 con 0x7f6bb4004d00 2024-01-15T18:17:44.271 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.268+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6bd414d8e0 con 0x7f6bb4004d00 2024-01-15T18:17:44.272 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.269+0000 7f6bcbe00640 1 -- 172.21.15.26:0/3222314286 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v117) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6bd414d8e0 con 0x7f6bb4004d00 2024-01-15T18:17:44.272 INFO:teuthology.orchestra.run.smithi026.stderr:listed 0 entries 2024-01-15T18:17:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6bac06c550 msgr2=0x7f6bac06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6bac06c550 0x7f6bac06ea10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6bd413df50 tx=0x7f6bbc008040 comp rx=0 tx=0).stop 2024-01-15T18:17:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 msgr2=0x7f6bd415d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bd415d260 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400e7c0 tx=0x7f6bc400ec80 comp rx=0 tx=0).stop 2024-01-15T18:17:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 shutdown_connections 2024-01-15T18:17:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6bd41506b0 0x7f6bd4150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.275 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6bac06c550 0x7f6bac06ea10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6bd4135af0 0x7f6bd41352d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 --2- 172.21.15.26:0/3222314286 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6bb4004d00 0x7f6bd415d260 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:44.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 >> 172.21.15.26:0/3222314286 conn(0x7f6bd406a800 msgr2=0x7f6bd40b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:44.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 shutdown_connections 2024-01-15T18:17:44.276 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:44.272+0000 7f6bdaa00640 1 -- 172.21.15.26:0/3222314286 wait complete. 2024-01-15T18:17:44.295 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi124.front.sepia.ceph.com... 2024-01-15T18:17:44.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:44.295 DEBUG:teuthology.orchestra.run.smithi124:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-01-15T18:17:44.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi124.stderr:ceph-fuse[48133]: fuse finished with error 0 and tester_r 0 2024-01-15T18:17:44.395 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:17:50.497 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:17:50.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:50.497 DEBUG:teuthology.orchestra.run.smithi124:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-01-15T18:17:50.556 DEBUG:teuthology.orchestra.run.smithi124:> set -ex 2024-01-15T18:17:50.556 DEBUG:teuthology.orchestra.run.smithi124:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-15T18:17:50.608 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053843,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249392k,nr_inodes=812348,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498788k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-01-15T18:17:50.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:50.609 DEBUG:teuthology.orchestra.run.smithi124:> ip netns list 2024-01-15T18:17:50.661 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-01-15T18:17:50.661 INFO:teuthology.orchestra.run.smithi124.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-01-15T18:17:50.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:50.662 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-01-15T18:17:50.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:50.744 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-01-15T18:17:50.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:50.826 DEBUG:teuthology.orchestra.run.smithi124:> sudo ip link delete ceph-brx 2024-01-15T18:17:50.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:50.958 DEBUG:teuthology.orchestra.run.smithi026:> ip netns list 2024-01-15T18:17:51.018 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-15T18:17:51.018 INFO:teuthology.orchestra.run.smithi026.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-01-15T18:17:51.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:51.020 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-15T18:17:51.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:51.117 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-01-15T18:17:51.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-15T18:17:51.206 DEBUG:teuthology.orchestra.run.smithi026:> sudo ip link delete ceph-brx 2024-01-15T18:17:51.348 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-15T18:17:51.358 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-15T18:17:51.358 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:17:51.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.555+0000 7f548be00640 1 Processor -- start 2024-01-15T18:17:51.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.556+0000 7f548be00640 1 -- start start 2024-01-15T18:17:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.557+0000 7f548be00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f548c071490 0x7f548c15e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.557+0000 7f548be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06b3f0 0x7f548c06b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.557+0000 7f548be00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c15fc10 con 0x7f548c15d1c0 2024-01-15T18:17:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.557+0000 7f548be00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548c06bdc0 con 0x7f548c071490 2024-01-15T18:17:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.557+0000 7f548be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0aeff0 con 0x7f548c06b3f0 2024-01-15T18:17:51.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.558+0000 7f548aa00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f548c15d1c0 0x7f548c0b2570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:57710/0 (socket says 172.21.15.26:57710) 2024-01-15T18:17:51.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.558+0000 7f548aa00640 1 -- 172.21.15.26:0/2083974716 learned_addr learned my addr 172.21.15.26:0/2083974716 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.558+0000 7f548b400640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06b3f0 0x7f548c06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.558+0000 7f548a000640 1 --2- 172.21.15.26:0/2083974716 >> v2:172.21.15.124:3300/0 conn(0x7f548c071490 0x7f548c15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.558+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446002 0 0) 0x7f548c15fc10 con 0x7f548c15d1c0 2024-01-15T18:17:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.558+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f546c003520 con 0x7f548c15d1c0 2024-01-15T18:17:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 11443912 0 0) 0x7f546c003520 con 0x7f548c15d1c0 2024-01-15T18:17:51.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06b3f0 msgr2=0x7f548c06b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:51.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06b3f0 0x7f548c06b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 >> v2:172.21.15.124:3300/0 conn(0x7f548c071490 msgr2=0x7f548c15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:51.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 --2- 172.21.15.26:0/2083974716 >> v2:172.21.15.124:3300/0 conn(0x7f548c071490 0x7f548c15e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c15f000 con 0x7f548c15d1c0 2024-01-15T18:17:51.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f547c001670 con 0x7f548c15d1c0 2024-01-15T18:17:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 >> v1:172.21.15.26:6789/0 conn(0x7f548c15d1c0 legacy=0x7f548c0b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06b3f0 0x7f546c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f546c003ef0 0x7f546c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f546c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c06bdc0 con 0x7f546c004d00 2024-01-15T18:17:51.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0aeff0 con 0x7f546c003ef0 2024-01-15T18:17:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f546c00b980 con 0x7f548c06b3f0 2024-01-15T18:17:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f548b400640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f546c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f548aa00640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06b3f0 0x7f546c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.559+0000 7f548a000640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f546c003ef0 0x7f546c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.560+0000 7f548b400640 1 -- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06b3f0 msgr2=0x7f546c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.560+0000 7f548b400640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06b3f0 0x7f546c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.560+0000 7f548b400640 1 -- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f546c003ef0 msgr2=0x7f546c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:51.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.560+0000 7f548b400640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f546c003ef0 0x7f546c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.560+0000 7f548b400640 1 -- 172.21.15.26:0/2083974716 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f546c003520 con 0x7f546c004d00 2024-01-15T18:17:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.561+0000 7f548b400640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f546c005180 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f5484009ac0 tx=0x7f546c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6427715ca26ac937 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:51.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.561+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548402c6b0 con 0x7f546c004d00 2024-01-15T18:17:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.561+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548402c810 con 0x7f546c004d00 2024-01-15T18:17:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.561+0000 7f5489600640 1 -- 172.21.15.26:0/2083974716 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548402cba0 con 0x7f546c004d00 2024-01-15T18:17:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.562+0000 7f548be00640 1 -- 172.21.15.26:0/2083974716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 msgr2=0x7f546c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.562+0000 7f548be00640 1 --2- 172.21.15.26:0/2083974716 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f546c005180 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f5484009ac0 tx=0x7f546c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.562+0000 7f548b400640 1 -- 172.21.15.26:0/2083974716 reap_dead start 2024-01-15T18:17:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.562+0000 7f548be00640 1 -- 172.21.15.26:0/2083974716 shutdown_connections 2024-01-15T18:17:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.562+0000 7f548be00640 1 -- 172.21.15.26:0/2083974716 >> 172.21.15.26:0/2083974716 conn(0x7f548c070970 msgr2=0x7f548c070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.562+0000 7f548be00640 1 -- 172.21.15.26:0/2083974716 shutdown_connections 2024-01-15T18:17:51.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.562+0000 7f548be00640 1 -- 172.21.15.26:0/2083974716 wait complete. 2024-01-15T18:17:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.564+0000 7f548be00640 1 Processor -- start 2024-01-15T18:17:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.564+0000 7f548be00640 1 -- start start 2024-01-15T18:17:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.565+0000 7f548be00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f548c142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.565+0000 7f548be00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06ec50 0x7f548c141ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.565+0000 7f548be00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f548c06da50 0x7f548c06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.565+0000 7f548be00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c1f6c30 con 0x7f546c004d00 2024-01-15T18:17:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.565+0000 7f548be00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c148350 con 0x7f548c06da50 2024-01-15T18:17:51.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.565+0000 7f548be00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c06e410 con 0x7f548c06ec50 2024-01-15T18:17:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.565+0000 7f548aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f548c142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.565+0000 7f548aa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f548c142430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56754/0 (socket says 172.21.15.26:56754) 2024-01-15T18:17:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.565+0000 7f548aa00640 1 -- 172.21.15.26:0/3044754561 learned_addr learned my addr 172.21.15.26:0/3044754561 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f548aa00640 1 -- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06ec50 msgr2=0x7f548c141ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f548aa00640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06ec50 0x7f548c141ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f548aa00640 1 -- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f548c06da50 msgr2=0x7f548c06ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f548aa00640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f548c06da50 0x7f548c06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f548aa00640 1 -- 172.21.15.26:0/3044754561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c157060 con 0x7f546c004d00 2024-01-15T18:17:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f548aa00640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f548c142430 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f547c00e8f0 tx=0x7f547c00edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f5483400640 1 -- 172.21.15.26:0/3044754561 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c005320 con 0x7f546c004d00 2024-01-15T18:17:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f5483400640 1 -- 172.21.15.26:0/3044754561 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c005480 con 0x7f546c004d00 2024-01-15T18:17:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c1f60b0 con 0x7f546c004d00 2024-01-15T18:17:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f5483400640 1 -- 172.21.15.26:0/3044754561 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0047e0 con 0x7f546c004d00 2024-01-15T18:17:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.566+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c1f6670 con 0x7f546c004d00 2024-01-15T18:17:51.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.567+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c067b60 con 0x7f546c004d00 2024-01-15T18:17:51.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.568+0000 7f5483400640 1 -- 172.21.15.26:0/3044754561 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f547c004940 con 0x7f546c004d00 2024-01-15T18:17:51.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.569+0000 7f5483400640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f546406c550 0x7f546406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:51.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.569+0000 7f5483400640 1 -- 172.21.15.26:0/3044754561 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f547c0886a0 con 0x7f546c004d00 2024-01-15T18:17:51.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.570+0000 7f548a000640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f546406c550 0x7f546406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:51.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.571+0000 7f548a000640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f546406c550 0x7f546406ea10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f548c0b0eb0 tx=0x7f5474009750 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:51.576 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.574+0000 7f5483400640 1 -- 172.21.15.26:0/3044754561 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f547c054be0 con 0x7f546c004d00 2024-01-15T18:17:51.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.765+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f548c15f5c0 con 0x7f546406c550 2024-01-15T18:17:51.795 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.793+0000 7f5483400640 1 -- 172.21.15.26:0/3044754561 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288837 (secure 0 0 0) 0x7f547400f050 con 0x7f546406c550 2024-01-15T18:17:51.797 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:17:51.801 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:17:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f546406c550 msgr2=0x7f546406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f546406c550 0x7f546406ea10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f548c0b0eb0 tx=0x7f5474009750 comp rx=0 tx=0).stop 2024-01-15T18:17:51.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 msgr2=0x7f548c142430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:51.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f548c142430 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f547c00e8f0 tx=0x7f547c00edb0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 shutdown_connections 2024-01-15T18:17:51.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f548c06ec50 0x7f548c141ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f546406c550 0x7f546406ea10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f548c06da50 0x7f548c06ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 --2- 172.21.15.26:0/3044754561 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f546c004d00 0x7f548c142430 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:51.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 >> 172.21.15.26:0/3044754561 conn(0x7f548c070970 msgr2=0x7f548c0b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:51.804 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 shutdown_connections 2024-01-15T18:17:51.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:51.800+0000 7f548be00640 1 -- 172.21.15.26:0/3044754561 wait complete. 2024-01-15T18:17:51.823 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":797,"stamp":"2024-01-15T18:17:50.450846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369934897,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1739,"num_write_kb":5246057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3181,"ondisk_log_size":3181,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11062216,"kb_used_data":10499592,"kb_used_omap":12,"kb_used_meta":562547,"kb_avail":743912504,"statfs":{"total":773094113280,"available":761766404096,"internally_reserved":0,"allocated":10751582208,"data_stored":10744948594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":576048688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1073781794,"num_objects":256,"num_object_clones":0,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":256,"num_read_kb":1048576,"num_write":275,"num_write_kb":1048622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005631"},"pg_stats":[{"pgid":"17.3f","version":"117'53","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.789093+0000","last_change":"2024-01-15T18:12:53.636580+0000","last_active":"2024-01-15T18:17:38.789093+0000","last_peered":"2024-01-15T18:17:38.789093+0000","last_clean":"2024-01-15T18:17:38.789093+0000","last_became_active":"2024-01-15T18:12:53.334688+0000","last_became_peered":"2024-01-15T18:12:53.334688+0000","last_unstale":"2024-01-15T18:17:38.789093+0000","last_undegraded":"2024-01-15T18:17:38.789093+0000","last_fullsized":"2024-01-15T18:17:38.789093+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:39.949271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3e","version":"117'45","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.174108+0000","last_change":"2024-01-15T18:12:11.053712+0000","last_active":"2024-01-15T18:17:39.174108+0000","last_peered":"2024-01-15T18:17:39.174108+0000","last_clean":"2024-01-15T18:17:39.174108+0000","last_became_active":"2024-01-15T18:12:11.053279+0000","last_became_peered":"2024-01-15T18:12:11.053279+0000","last_unstale":"2024-01-15T18:17:39.174108+0000","last_undegraded":"2024-01-15T18:17:39.174108+0000","last_fullsized":"2024-01-15T18:17:39.174108+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"117'44","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.301593+0000","last_change":"2024-01-15T18:12:11.058625+0000","last_active":"2024-01-15T18:17:39.301593+0000","last_peered":"2024-01-15T18:17:39.301593+0000","last_clean":"2024-01-15T18:17:39.301593+0000","last_became_active":"2024-01-15T18:12:11.057543+0000","last_became_peered":"2024-01-15T18:12:11.057543+0000","last_unstale":"2024-01-15T18:17:39.301593+0000","last_undegraded":"2024-01-15T18:17:39.301593+0000","last_fullsized":"2024-01-15T18:17:39.301593+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:17.569977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"117'45","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.237120+0000","last_change":"2024-01-15T18:12:11.047313+0000","last_active":"2024-01-15T18:17:39.237120+0000","last_peered":"2024-01-15T18:17:39.237120+0000","last_clean":"2024-01-15T18:17:39.237120+0000","last_became_active":"2024-01-15T18:12:11.047008+0000","last_became_peered":"2024-01-15T18:12:11.047008+0000","last_unstale":"2024-01-15T18:17:39.237120+0000","last_undegraded":"2024-01-15T18:17:39.237120+0000","last_fullsized":"2024-01-15T18:17:39.237120+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:03:15.070494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"117'27","reported_seq":49,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.653418+0000","last_change":"2024-01-15T18:12:11.035567+0000","last_active":"2024-01-15T18:17:38.653418+0000","last_peered":"2024-01-15T18:17:38.653418+0000","last_clean":"2024-01-15T18:17:38.653418+0000","last_became_active":"2024-01-15T18:12:11.034909+0000","last_became_peered":"2024-01-15T18:12:11.034909+0000","last_unstale":"2024-01-15T18:17:38.653418+0000","last_undegraded":"2024-01-15T18:17:38.653418+0000","last_fullsized":"2024-01-15T18:17:38.653418+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:46.811591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"117'48","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.922505+0000","last_change":"2024-01-15T18:12:11.055133+0000","last_active":"2024-01-15T18:17:38.922505+0000","last_peered":"2024-01-15T18:17:38.922505+0000","last_clean":"2024-01-15T18:17:38.922505+0000","last_became_active":"2024-01-15T18:12:11.054803+0000","last_became_peered":"2024-01-15T18:12:11.054803+0000","last_unstale":"2024-01-15T18:17:38.922505+0000","last_undegraded":"2024-01-15T18:17:38.922505+0000","last_fullsized":"2024-01-15T18:17:38.922505+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:53.540866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"117'43","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.895097+0000","last_change":"2024-01-15T18:12:11.045438+0000","last_active":"2024-01-15T18:17:38.895097+0000","last_peered":"2024-01-15T18:17:38.895097+0000","last_clean":"2024-01-15T18:17:38.895097+0000","last_became_active":"2024-01-15T18:12:11.044607+0000","last_became_peered":"2024-01-15T18:12:11.044607+0000","last_unstale":"2024-01-15T18:17:38.895097+0000","last_undegraded":"2024-01-15T18:17:38.895097+0000","last_fullsized":"2024-01-15T18:17:38.895097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:44.315673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"117'44","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.859692+0000","last_change":"2024-01-15T18:12:11.051056+0000","last_active":"2024-01-15T18:17:38.859692+0000","last_peered":"2024-01-15T18:17:38.859692+0000","last_clean":"2024-01-15T18:17:38.859692+0000","last_became_active":"2024-01-15T18:12:11.050697+0000","last_became_peered":"2024-01-15T18:12:11.050697+0000","last_unstale":"2024-01-15T18:17:38.859692+0000","last_undegraded":"2024-01-15T18:17:38.859692+0000","last_fullsized":"2024-01-15T18:17:38.859692+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:56.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.359403+0000","last_change":"2024-01-15T18:12:11.048227+0000","last_active":"2024-01-15T18:17:39.359403+0000","last_peered":"2024-01-15T18:17:39.359403+0000","last_clean":"2024-01-15T18:17:39.359403+0000","last_became_active":"2024-01-15T18:12:11.047693+0000","last_became_peered":"2024-01-15T18:12:11.047693+0000","last_unstale":"2024-01-15T18:17:39.359403+0000","last_undegraded":"2024-01-15T18:17:39.359403+0000","last_fullsized":"2024-01-15T18:17:39.359403+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:43.904890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"117'47","reported_seq":72,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.466111+0000","last_change":"2024-01-15T18:12:11.054486+0000","last_active":"2024-01-15T18:17:39.466111+0000","last_peered":"2024-01-15T18:17:39.466111+0000","last_clean":"2024-01-15T18:17:39.466111+0000","last_became_active":"2024-01-15T18:12:11.052429+0000","last_became_peered":"2024-01-15T18:12:11.052429+0000","last_unstale":"2024-01-15T18:17:39.466111+0000","last_undegraded":"2024-01-15T18:17:39.466111+0000","last_fullsized":"2024-01-15T18:17:39.466111+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.440558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"117'47","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.167604+0000","last_change":"2024-01-15T18:12:11.043835+0000","last_active":"2024-01-15T18:17:39.167604+0000","last_peered":"2024-01-15T18:17:39.167604+0000","last_clean":"2024-01-15T18:17:39.167604+0000","last_became_active":"2024-01-15T18:12:11.042720+0000","last_became_peered":"2024-01-15T18:12:11.042720+0000","last_unstale":"2024-01-15T18:17:39.167604+0000","last_undegraded":"2024-01-15T18:17:39.167604+0000","last_fullsized":"2024-01-15T18:17:39.167604+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"117'48","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.287663+0000","last_change":"2024-01-15T18:12:11.044468+0000","last_active":"2024-01-15T18:17:39.287663+0000","last_peered":"2024-01-15T18:17:39.287663+0000","last_clean":"2024-01-15T18:17:39.287663+0000","last_became_active":"2024-01-15T18:12:11.044155+0000","last_became_peered":"2024-01-15T18:12:11.044155+0000","last_unstale":"2024-01-15T18:17:39.287663+0000","last_undegraded":"2024-01-15T18:17:39.287663+0000","last_fullsized":"2024-01-15T18:17:39.287663+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:57.310458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"117'38","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.737859+0000","last_change":"2024-01-15T18:12:11.052950+0000","last_active":"2024-01-15T18:17:38.737859+0000","last_peered":"2024-01-15T18:17:38.737859+0000","last_clean":"2024-01-15T18:17:38.737859+0000","last_became_active":"2024-01-15T18:12:11.051875+0000","last_became_peered":"2024-01-15T18:12:11.051875+0000","last_unstale":"2024-01-15T18:17:38.737859+0000","last_undegraded":"2024-01-15T18:17:38.737859+0000","last_fullsized":"2024-01-15T18:17:38.737859+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:22.849560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"117'51","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.417564+0000","last_change":"2024-01-15T18:12:11.056103+0000","last_active":"2024-01-15T18:17:39.417564+0000","last_peered":"2024-01-15T18:17:39.417564+0000","last_clean":"2024-01-15T18:17:39.417564+0000","last_became_active":"2024-01-15T18:12:11.055813+0000","last_became_peered":"2024-01-15T18:12:11.055813+0000","last_unstale":"2024-01-15T18:17:39.417564+0000","last_undegraded":"2024-01-15T18:17:39.417564+0000","last_fullsized":"2024-01-15T18:17:39.417564+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.025824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"117'34","reported_seq":53,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.330592+0000","last_change":"2024-01-15T18:12:11.056863+0000","last_active":"2024-01-15T18:17:39.330592+0000","last_peered":"2024-01-15T18:17:39.330592+0000","last_clean":"2024-01-15T18:17:39.330592+0000","last_became_active":"2024-01-15T18:12:11.055217+0000","last_became_peered":"2024-01-15T18:12:11.055217+0000","last_unstale":"2024-01-15T18:17:39.330592+0000","last_undegraded":"2024-01-15T18:17:39.330592+0000","last_fullsized":"2024-01-15T18:17:39.330592+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:33.702581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"117'42","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.869131+0000","last_change":"2024-01-15T18:12:53.618454+0000","last_active":"2024-01-15T18:17:38.869131+0000","last_peered":"2024-01-15T18:17:38.869131+0000","last_clean":"2024-01-15T18:17:38.869131+0000","last_became_active":"2024-01-15T18:12:53.338860+0000","last_became_peered":"2024-01-15T18:12:53.338860+0000","last_unstale":"2024-01-15T18:17:38.869131+0000","last_undegraded":"2024-01-15T18:17:38.869131+0000","last_fullsized":"2024-01-15T18:17:38.869131+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:40.129590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2f","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.108098+0000","last_change":"2024-01-15T18:12:11.043060+0000","last_active":"2024-01-15T18:17:39.108098+0000","last_peered":"2024-01-15T18:17:39.108098+0000","last_clean":"2024-01-15T18:17:39.108098+0000","last_became_active":"2024-01-15T18:12:11.042031+0000","last_became_peered":"2024-01-15T18:12:11.042031+0000","last_unstale":"2024-01-15T18:17:39.108098+0000","last_undegraded":"2024-01-15T18:17:39.108098+0000","last_fullsized":"2024-01-15T18:17:39.108098+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:44.791297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"117'48","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.357897+0000","last_change":"2024-01-15T18:12:11.045284+0000","last_active":"2024-01-15T18:17:39.357897+0000","last_peered":"2024-01-15T18:17:39.357897+0000","last_clean":"2024-01-15T18:17:39.357897+0000","last_became_active":"2024-01-15T18:12:11.042814+0000","last_became_peered":"2024-01-15T18:12:11.042814+0000","last_unstale":"2024-01-15T18:17:39.357897+0000","last_undegraded":"2024-01-15T18:17:39.357897+0000","last_fullsized":"2024-01-15T18:17:39.357897+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.108742+0000","last_change":"2024-01-15T18:12:11.055119+0000","last_active":"2024-01-15T18:17:39.108742+0000","last_peered":"2024-01-15T18:17:39.108742+0000","last_clean":"2024-01-15T18:17:39.108742+0000","last_became_active":"2024-01-15T18:12:11.054790+0000","last_became_peered":"2024-01-15T18:12:11.054790+0000","last_unstale":"2024-01-15T18:17:39.108742+0000","last_undegraded":"2024-01-15T18:17:39.108742+0000","last_fullsized":"2024-01-15T18:17:39.108742+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:17.513018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"117'46","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.362355+0000","last_change":"2024-01-15T18:12:11.052755+0000","last_active":"2024-01-15T18:17:39.362355+0000","last_peered":"2024-01-15T18:17:39.362355+0000","last_clean":"2024-01-15T18:17:39.362355+0000","last_became_active":"2024-01-15T18:12:11.052289+0000","last_became_peered":"2024-01-15T18:12:11.052289+0000","last_unstale":"2024-01-15T18:17:39.362355+0000","last_undegraded":"2024-01-15T18:17:39.362355+0000","last_fullsized":"2024-01-15T18:17:39.362355+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:38:56.937377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"117'56","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.083979+0000","last_change":"2024-01-15T18:12:11.036748+0000","last_active":"2024-01-15T18:17:39.083979+0000","last_peered":"2024-01-15T18:17:39.083979+0000","last_clean":"2024-01-15T18:17:39.083979+0000","last_became_active":"2024-01-15T18:12:11.036003+0000","last_became_peered":"2024-01-15T18:12:11.036003+0000","last_unstale":"2024-01-15T18:17:39.083979+0000","last_undegraded":"2024-01-15T18:17:39.083979+0000","last_fullsized":"2024-01-15T18:17:39.083979+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:20.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.456439+0000","last_change":"2024-01-15T18:12:11.050445+0000","last_active":"2024-01-15T18:17:39.456439+0000","last_peered":"2024-01-15T18:17:39.456439+0000","last_clean":"2024-01-15T18:17:39.456439+0000","last_became_active":"2024-01-15T18:12:11.044093+0000","last_became_peered":"2024-01-15T18:12:11.044093+0000","last_unstale":"2024-01-15T18:17:39.456439+0000","last_undegraded":"2024-01-15T18:17:39.456439+0000","last_fullsized":"2024-01-15T18:17:39.456439+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:38:59.097623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"117'39","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.269675+0000","last_change":"2024-01-15T18:12:11.053397+0000","last_active":"2024-01-15T18:17:39.269675+0000","last_peered":"2024-01-15T18:17:39.269675+0000","last_clean":"2024-01-15T18:17:39.269675+0000","last_became_active":"2024-01-15T18:12:11.052428+0000","last_became_peered":"2024-01-15T18:12:11.052428+0000","last_unstale":"2024-01-15T18:17:39.269675+0000","last_undegraded":"2024-01-15T18:17:39.269675+0000","last_fullsized":"2024-01-15T18:17:39.269675+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:07.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"117'43","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.364184+0000","last_change":"2024-01-15T18:12:11.051820+0000","last_active":"2024-01-15T18:17:39.364184+0000","last_peered":"2024-01-15T18:17:39.364184+0000","last_clean":"2024-01-15T18:17:39.364184+0000","last_became_active":"2024-01-15T18:12:11.051488+0000","last_became_peered":"2024-01-15T18:12:11.051488+0000","last_unstale":"2024-01-15T18:17:39.364184+0000","last_undegraded":"2024-01-15T18:17:39.364184+0000","last_fullsized":"2024-01-15T18:17:39.364184+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:45.480493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"117'38","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.840614+0000","last_change":"2024-01-15T18:12:11.041672+0000","last_active":"2024-01-15T18:17:38.840614+0000","last_peered":"2024-01-15T18:17:38.840614+0000","last_clean":"2024-01-15T18:17:38.840614+0000","last_became_active":"2024-01-15T18:12:11.041266+0000","last_became_peered":"2024-01-15T18:12:11.041266+0000","last_unstale":"2024-01-15T18:17:38.840614+0000","last_undegraded":"2024-01-15T18:17:38.840614+0000","last_fullsized":"2024-01-15T18:17:38.840614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"117'53","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.369799+0000","last_change":"2024-01-15T18:12:11.048567+0000","last_active":"2024-01-15T18:17:39.369799+0000","last_peered":"2024-01-15T18:17:39.369799+0000","last_clean":"2024-01-15T18:17:39.369799+0000","last_became_active":"2024-01-15T18:12:11.048234+0000","last_became_peered":"2024-01-15T18:12:11.048234+0000","last_unstale":"2024-01-15T18:17:39.369799+0000","last_undegraded":"2024-01-15T18:17:39.369799+0000","last_fullsized":"2024-01-15T18:17:39.369799+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:12.290504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"117'37","reported_seq":59,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.691915+0000","last_change":"2024-01-15T18:12:11.062039+0000","last_active":"2024-01-15T18:17:38.691915+0000","last_peered":"2024-01-15T18:17:38.691915+0000","last_clean":"2024-01-15T18:17:38.691915+0000","last_became_active":"2024-01-15T18:12:11.061724+0000","last_became_peered":"2024-01-15T18:12:11.061724+0000","last_unstale":"2024-01-15T18:17:38.691915+0000","last_undegraded":"2024-01-15T18:17:38.691915+0000","last_fullsized":"2024-01-15T18:17:38.691915+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:03.329007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"117'38","reported_seq":56,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.276302+0000","last_change":"2024-01-15T18:12:11.044940+0000","last_active":"2024-01-15T18:17:39.276302+0000","last_peered":"2024-01-15T18:17:39.276302+0000","last_clean":"2024-01-15T18:17:39.276302+0000","last_became_active":"2024-01-15T18:12:11.043801+0000","last_became_peered":"2024-01-15T18:12:11.043801+0000","last_unstale":"2024-01-15T18:17:39.276302+0000","last_undegraded":"2024-01-15T18:17:39.276302+0000","last_fullsized":"2024-01-15T18:17:39.276302+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:43.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"117'46","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.347960+0000","last_change":"2024-01-15T18:12:11.048119+0000","last_active":"2024-01-15T18:17:39.347960+0000","last_peered":"2024-01-15T18:17:39.347960+0000","last_clean":"2024-01-15T18:17:39.347960+0000","last_became_active":"2024-01-15T18:12:11.047739+0000","last_became_peered":"2024-01-15T18:12:11.047739+0000","last_unstale":"2024-01-15T18:17:39.347960+0000","last_undegraded":"2024-01-15T18:17:39.347960+0000","last_fullsized":"2024-01-15T18:17:39.347960+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:49:08.557895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"117'41","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.292578+0000","last_change":"2024-01-15T18:12:11.040703+0000","last_active":"2024-01-15T18:17:39.292578+0000","last_peered":"2024-01-15T18:17:39.292578+0000","last_clean":"2024-01-15T18:17:39.292578+0000","last_became_active":"2024-01-15T18:12:11.039393+0000","last_became_peered":"2024-01-15T18:12:11.039393+0000","last_unstale":"2024-01-15T18:17:39.292578+0000","last_undegraded":"2024-01-15T18:17:39.292578+0000","last_fullsized":"2024-01-15T18:17:39.292578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:40:41.329686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"117'62","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.906525+0000","last_change":"2024-01-15T18:12:11.059603+0000","last_active":"2024-01-15T18:17:38.906525+0000","last_peered":"2024-01-15T18:17:38.906525+0000","last_clean":"2024-01-15T18:17:38.906525+0000","last_became_active":"2024-01-15T18:12:11.059276+0000","last_became_peered":"2024-01-15T18:12:11.059276+0000","last_unstale":"2024-01-15T18:17:38.906525+0000","last_undegraded":"2024-01-15T18:17:38.906525+0000","last_fullsized":"2024-01-15T18:17:38.906525+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:43:52.890489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"117'54","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.208710+0000","last_change":"2024-01-15T18:12:11.053438+0000","last_active":"2024-01-15T18:17:39.208710+0000","last_peered":"2024-01-15T18:17:39.208710+0000","last_clean":"2024-01-15T18:17:39.208710+0000","last_became_active":"2024-01-15T18:12:11.052655+0000","last_became_peered":"2024-01-15T18:12:11.052655+0000","last_unstale":"2024-01-15T18:17:39.208710+0000","last_undegraded":"2024-01-15T18:17:39.208710+0000","last_fullsized":"2024-01-15T18:17:39.208710+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:45.306312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"117'34","reported_seq":55,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.008114+0000","last_change":"2024-01-15T18:12:11.053689+0000","last_active":"2024-01-15T18:17:39.008114+0000","last_peered":"2024-01-15T18:17:39.008114+0000","last_clean":"2024-01-15T18:17:39.008114+0000","last_became_active":"2024-01-15T18:12:11.053389+0000","last_became_peered":"2024-01-15T18:12:11.053389+0000","last_unstale":"2024-01-15T18:17:39.008114+0000","last_undegraded":"2024-01-15T18:17:39.008114+0000","last_fullsized":"2024-01-15T18:17:39.008114+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:39.171657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"114'4","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.409090+0000","last_change":"2024-01-15T18:12:10.021097+0000","last_active":"2024-01-15T18:14:17.409090+0000","last_peered":"2024-01-15T18:14:17.409090+0000","last_clean":"2024-01-15T18:14:17.409090+0000","last_became_active":"2024-01-15T18:12:10.020810+0000","last_became_peered":"2024-01-15T18:12:10.020810+0000","last_unstale":"2024-01-15T18:14:17.409090+0000","last_undegraded":"2024-01-15T18:14:17.409090+0000","last_fullsized":"2024-01-15T18:14:17.409090+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:09:27.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.721958+0000","last_change":"2024-01-15T18:12:11.043347+0000","last_active":"2024-01-15T18:17:38.721958+0000","last_peered":"2024-01-15T18:17:38.721958+0000","last_clean":"2024-01-15T18:17:38.721958+0000","last_became_active":"2024-01-15T18:12:11.042427+0000","last_became_peered":"2024-01-15T18:12:11.042427+0000","last_unstale":"2024-01-15T18:17:38.721958+0000","last_undegraded":"2024-01-15T18:17:38.721958+0000","last_fullsized":"2024-01-15T18:17:38.721958+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:12.407664+0000","last_change":"2024-01-15T18:12:10.023175+0000","last_active":"2024-01-15T18:14:12.407664+0000","last_peered":"2024-01-15T18:14:12.407664+0000","last_clean":"2024-01-15T18:14:12.407664+0000","last_became_active":"2024-01-15T18:12:10.022654+0000","last_became_peered":"2024-01-15T18:12:10.022654+0000","last_unstale":"2024-01-15T18:14:12.407664+0000","last_undegraded":"2024-01-15T18:14:12.407664+0000","last_fullsized":"2024-01-15T18:14:12.407664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:12.136270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'43","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.190370+0000","last_change":"2024-01-15T18:12:11.053138+0000","last_active":"2024-01-15T18:17:39.190370+0000","last_peered":"2024-01-15T18:17:39.190370+0000","last_clean":"2024-01-15T18:17:39.190370+0000","last_became_active":"2024-01-15T18:12:11.052812+0000","last_became_peered":"2024-01-15T18:12:11.052812+0000","last_unstale":"2024-01-15T18:17:39.190370+0000","last_undegraded":"2024-01-15T18:17:39.190370+0000","last_fullsized":"2024-01-15T18:17:39.190370+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:40.787531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"117'3","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.287481+0000","last_change":"2024-01-15T18:12:10.018995+0000","last_active":"2024-01-15T18:14:42.287481+0000","last_peered":"2024-01-15T18:14:42.287481+0000","last_clean":"2024-01-15T18:14:42.287481+0000","last_became_active":"2024-01-15T18:12:10.018670+0000","last_became_peered":"2024-01-15T18:12:10.018670+0000","last_unstale":"2024-01-15T18:14:42.287481+0000","last_undegraded":"2024-01-15T18:14:42.287481+0000","last_fullsized":"2024-01-15T18:14:42.287481+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:39.773731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"117'56","reported_seq":82,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.844198+0000","last_change":"2024-01-15T18:12:11.059355+0000","last_active":"2024-01-15T18:17:38.844198+0000","last_peered":"2024-01-15T18:17:38.844198+0000","last_clean":"2024-01-15T18:17:38.844198+0000","last_became_active":"2024-01-15T18:12:11.057592+0000","last_became_peered":"2024-01-15T18:12:11.057592+0000","last_unstale":"2024-01-15T18:17:38.844198+0000","last_undegraded":"2024-01-15T18:17:38.844198+0000","last_fullsized":"2024-01-15T18:17:38.844198+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:14.826977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.287454+0000","last_change":"2024-01-15T18:12:10.020565+0000","last_active":"2024-01-15T18:14:42.287454+0000","last_peered":"2024-01-15T18:14:42.287454+0000","last_clean":"2024-01-15T18:14:42.287454+0000","last_became_active":"2024-01-15T18:12:10.020143+0000","last_became_peered":"2024-01-15T18:12:10.020143+0000","last_unstale":"2024-01-15T18:14:42.287454+0000","last_undegraded":"2024-01-15T18:14:42.287454+0000","last_fullsized":"2024-01-15T18:14:42.287454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:09.129175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"117'39","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.444992+0000","last_change":"2024-01-15T18:12:11.057876+0000","last_active":"2024-01-15T18:17:39.444992+0000","last_peered":"2024-01-15T18:17:39.444992+0000","last_clean":"2024-01-15T18:17:39.444992+0000","last_became_active":"2024-01-15T18:12:11.057471+0000","last_became_peered":"2024-01-15T18:12:11.057471+0000","last_unstale":"2024-01-15T18:17:39.444992+0000","last_undegraded":"2024-01-15T18:17:39.444992+0000","last_fullsized":"2024-01-15T18:17:39.444992+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:22:19.460852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.414286+0000","last_change":"2024-01-15T18:12:10.020217+0000","last_active":"2024-01-15T18:14:42.414286+0000","last_peered":"2024-01-15T18:14:42.414286+0000","last_clean":"2024-01-15T18:14:42.414286+0000","last_became_active":"2024-01-15T18:12:10.019818+0000","last_became_peered":"2024-01-15T18:12:10.019818+0000","last_unstale":"2024-01-15T18:14:42.414286+0000","last_undegraded":"2024-01-15T18:14:42.414286+0000","last_fullsized":"2024-01-15T18:14:42.414286+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"117'49","reported_seq":77,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.264806+0000","last_change":"2024-01-15T18:12:11.050383+0000","last_active":"2024-01-15T18:17:39.264806+0000","last_peered":"2024-01-15T18:17:39.264806+0000","last_clean":"2024-01-15T18:17:39.264806+0000","last_became_active":"2024-01-15T18:12:11.050002+0000","last_became_peered":"2024-01-15T18:12:11.050002+0000","last_unstale":"2024-01-15T18:17:39.264806+0000","last_undegraded":"2024-01-15T18:17:39.264806+0000","last_fullsized":"2024-01-15T18:17:39.264806+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.309045+0000","last_change":"2024-01-15T18:12:10.013940+0000","last_active":"2024-01-15T18:14:42.309045+0000","last_peered":"2024-01-15T18:14:42.309045+0000","last_clean":"2024-01-15T18:14:42.309045+0000","last_became_active":"2024-01-15T18:12:10.009676+0000","last_became_peered":"2024-01-15T18:12:10.009676+0000","last_unstale":"2024-01-15T18:14:42.309045+0000","last_undegraded":"2024-01-15T18:14:42.309045+0000","last_fullsized":"2024-01-15T18:14:42.309045+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:00.059047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"117'53","reported_seq":78,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.258378+0000","last_change":"2024-01-15T18:12:11.057984+0000","last_active":"2024-01-15T18:17:39.258378+0000","last_peered":"2024-01-15T18:17:39.258378+0000","last_clean":"2024-01-15T18:17:39.258378+0000","last_became_active":"2024-01-15T18:12:11.057640+0000","last_became_peered":"2024-01-15T18:12:11.057640+0000","last_unstale":"2024-01-15T18:17:39.258378+0000","last_undegraded":"2024-01-15T18:17:39.258378+0000","last_fullsized":"2024-01-15T18:17:39.258378+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:57:42.482689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857791,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":36865,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.331147+0000","last_change":"2024-01-15T18:12:10.002836+0000","last_active":"2024-01-15T18:14:37.331147+0000","last_peered":"2024-01-15T18:14:37.331147+0000","last_clean":"2024-01-15T18:14:37.331147+0000","last_became_active":"2024-01-15T18:12:10.002190+0000","last_became_peered":"2024-01-15T18:12:10.002190+0000","last_unstale":"2024-01-15T18:14:37.331147+0000","last_undegraded":"2024-01-15T18:14:37.331147+0000","last_fullsized":"2024-01-15T18:14:37.331147+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:39.115790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"117'43","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.016147+0000","last_change":"2024-01-15T18:12:11.048560+0000","last_active":"2024-01-15T18:17:39.016147+0000","last_peered":"2024-01-15T18:17:39.016147+0000","last_clean":"2024-01-15T18:17:39.016147+0000","last_became_active":"2024-01-15T18:12:11.046934+0000","last_became_peered":"2024-01-15T18:12:11.046934+0000","last_unstale":"2024-01-15T18:17:39.016147+0000","last_undegraded":"2024-01-15T18:17:39.016147+0000","last_fullsized":"2024-01-15T18:17:39.016147+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:44:29.629308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.426824+0000","last_change":"2024-01-15T18:12:10.020864+0000","last_active":"2024-01-15T18:14:37.426824+0000","last_peered":"2024-01-15T18:14:37.426824+0000","last_clean":"2024-01-15T18:14:37.426824+0000","last_became_active":"2024-01-15T18:12:10.020577+0000","last_became_peered":"2024-01-15T18:12:10.020577+0000","last_unstale":"2024-01-15T18:14:37.426824+0000","last_undegraded":"2024-01-15T18:14:37.426824+0000","last_fullsized":"2024-01-15T18:14:37.426824+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:55.380272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"117'42","reported_seq":61,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.550415+0000","last_change":"2024-01-15T18:12:11.048003+0000","last_active":"2024-01-15T18:17:38.550415+0000","last_peered":"2024-01-15T18:17:38.550415+0000","last_clean":"2024-01-15T18:17:38.550415+0000","last_became_active":"2024-01-15T18:12:11.047704+0000","last_became_peered":"2024-01-15T18:12:11.047704+0000","last_unstale":"2024-01-15T18:17:38.550415+0000","last_undegraded":"2024-01-15T18:17:38.550415+0000","last_fullsized":"2024-01-15T18:17:38.550415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:02.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"117'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.286410+0000","last_change":"2024-01-15T18:12:10.020498+0000","last_active":"2024-01-15T18:14:37.286410+0000","last_peered":"2024-01-15T18:14:37.286410+0000","last_clean":"2024-01-15T18:14:37.286410+0000","last_became_active":"2024-01-15T18:12:10.020028+0000","last_became_peered":"2024-01-15T18:12:10.020028+0000","last_unstale":"2024-01-15T18:14:37.286410+0000","last_undegraded":"2024-01-15T18:14:37.286410+0000","last_fullsized":"2024-01-15T18:14:37.286410+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"117'40","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.283436+0000","last_change":"2024-01-15T18:12:11.060166+0000","last_active":"2024-01-15T18:17:39.283436+0000","last_peered":"2024-01-15T18:17:39.283436+0000","last_clean":"2024-01-15T18:17:39.283436+0000","last_became_active":"2024-01-15T18:12:11.059393+0000","last_became_peered":"2024-01-15T18:12:11.059393+0000","last_unstale":"2024-01-15T18:17:39.283436+0000","last_undegraded":"2024-01-15T18:17:39.283436+0000","last_fullsized":"2024-01-15T18:17:39.283436+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.308163+0000","last_change":"2024-01-15T18:12:10.007879+0000","last_active":"2024-01-15T18:14:37.308163+0000","last_peered":"2024-01-15T18:14:37.308163+0000","last_clean":"2024-01-15T18:14:37.308163+0000","last_became_active":"2024-01-15T18:12:10.007549+0000","last_became_peered":"2024-01-15T18:12:10.007549+0000","last_unstale":"2024-01-15T18:14:37.308163+0000","last_undegraded":"2024-01-15T18:14:37.308163+0000","last_fullsized":"2024-01-15T18:14:37.308163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:33.832426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"117'43","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.394428+0000","last_change":"2024-01-15T18:12:11.031666+0000","last_active":"2024-01-15T18:17:39.394428+0000","last_peered":"2024-01-15T18:17:39.394428+0000","last_clean":"2024-01-15T18:17:39.394428+0000","last_became_active":"2024-01-15T18:12:11.030958+0000","last_became_peered":"2024-01-15T18:12:11.030958+0000","last_unstale":"2024-01-15T18:17:39.394428+0000","last_undegraded":"2024-01-15T18:17:39.394428+0000","last_fullsized":"2024-01-15T18:17:39.394428+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:58.112590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"117'12","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:33.625051+0000","last_change":"2024-01-15T18:12:10.024809+0000","last_active":"2024-01-15T18:15:33.625051+0000","last_peered":"2024-01-15T18:15:33.625051+0000","last_clean":"2024-01-15T18:15:33.625051+0000","last_became_active":"2024-01-15T18:12:10.024120+0000","last_became_peered":"2024-01-15T18:12:10.024120+0000","last_unstale":"2024-01-15T18:15:33.625051+0000","last_undegraded":"2024-01-15T18:15:33.625051+0000","last_fullsized":"2024-01-15T18:15:33.625051+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:17:59.130803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"117'43","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.025487+0000","last_change":"2024-01-15T18:12:11.055694+0000","last_active":"2024-01-15T18:17:39.025487+0000","last_peered":"2024-01-15T18:17:39.025487+0000","last_clean":"2024-01-15T18:17:39.025487+0000","last_became_active":"2024-01-15T18:12:11.055290+0000","last_became_peered":"2024-01-15T18:12:11.055290+0000","last_unstale":"2024-01-15T18:17:39.025487+0000","last_undegraded":"2024-01-15T18:17:39.025487+0000","last_fullsized":"2024-01-15T18:17:39.025487+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:44.436928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"117'183","reported_seq":199,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:44.393465+0000","last_change":"2024-01-15T18:12:10.022950+0000","last_active":"2024-01-15T18:17:44.393465+0000","last_peered":"2024-01-15T18:17:44.393465+0000","last_clean":"2024-01-15T18:17:44.393465+0000","last_became_active":"2024-01-15T18:12:10.021497+0000","last_became_peered":"2024-01-15T18:12:10.021497+0000","last_unstale":"2024-01-15T18:17:44.393465+0000","last_undegraded":"2024-01-15T18:17:44.393465+0000","last_fullsized":"2024-01-15T18:17:44.393465+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:16.556177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":632277,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"117'44","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.050496+0000","last_change":"2024-01-15T18:12:11.046101+0000","last_active":"2024-01-15T18:17:39.050496+0000","last_peered":"2024-01-15T18:17:39.050496+0000","last_clean":"2024-01-15T18:17:39.050496+0000","last_became_active":"2024-01-15T18:12:11.045774+0000","last_became_peered":"2024-01-15T18:12:11.045774+0000","last_unstale":"2024-01-15T18:17:39.050496+0000","last_undegraded":"2024-01-15T18:17:39.050496+0000","last_fullsized":"2024-01-15T18:17:39.050496+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:41:11.078868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.285076+0000","last_change":"2024-01-15T18:12:10.016912+0000","last_active":"2024-01-15T18:14:32.285076+0000","last_peered":"2024-01-15T18:14:32.285076+0000","last_clean":"2024-01-15T18:14:32.285076+0000","last_became_active":"2024-01-15T18:12:10.016618+0000","last_became_peered":"2024-01-15T18:12:10.016618+0000","last_unstale":"2024-01-15T18:14:32.285076+0000","last_undegraded":"2024-01-15T18:14:32.285076+0000","last_fullsized":"2024-01-15T18:14:32.285076+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:40:48.957828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"117'52","reported_seq":72,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.238652+0000","last_change":"2024-01-15T18:12:11.051223+0000","last_active":"2024-01-15T18:17:39.238652+0000","last_peered":"2024-01-15T18:17:39.238652+0000","last_clean":"2024-01-15T18:17:39.238652+0000","last_became_active":"2024-01-15T18:12:11.050928+0000","last_became_peered":"2024-01-15T18:12:11.050928+0000","last_unstale":"2024-01-15T18:17:39.238652+0000","last_undegraded":"2024-01-15T18:17:39.238652+0000","last_fullsized":"2024-01-15T18:17:39.238652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:11.318479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.543703+0000","last_change":"2024-01-15T18:12:10.015868+0000","last_active":"2024-01-15T18:14:32.543703+0000","last_peered":"2024-01-15T18:14:32.543703+0000","last_clean":"2024-01-15T18:14:32.543703+0000","last_became_active":"2024-01-15T18:12:10.015578+0000","last_became_peered":"2024-01-15T18:12:10.015578+0000","last_unstale":"2024-01-15T18:14:32.543703+0000","last_undegraded":"2024-01-15T18:14:32.543703+0000","last_fullsized":"2024-01-15T18:14:32.543703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"117'46","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.210181+0000","last_change":"2024-01-15T18:12:11.029654+0000","last_active":"2024-01-15T18:17:39.210181+0000","last_peered":"2024-01-15T18:17:39.210181+0000","last_clean":"2024-01-15T18:17:39.210181+0000","last_became_active":"2024-01-15T18:12:11.028865+0000","last_became_peered":"2024-01-15T18:12:11.028865+0000","last_unstale":"2024-01-15T18:17:39.210181+0000","last_undegraded":"2024-01-15T18:17:39.210181+0000","last_fullsized":"2024-01-15T18:17:39.210181+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:29.536846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663556,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"114'8","reported_seq":24,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.285243+0000","last_change":"2024-01-15T18:12:10.022690+0000","last_active":"2024-01-15T18:14:32.285243+0000","last_peered":"2024-01-15T18:14:32.285243+0000","last_clean":"2024-01-15T18:14:32.285243+0000","last_became_active":"2024-01-15T18:12:10.022398+0000","last_became_peered":"2024-01-15T18:12:10.022398+0000","last_unstale":"2024-01-15T18:14:32.285243+0000","last_undegraded":"2024-01-15T18:14:32.285243+0000","last_fullsized":"2024-01-15T18:14:32.285243+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:52.527066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"117'24","reported_seq":43,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.192858+0000","last_change":"2024-01-15T18:12:11.045910+0000","last_active":"2024-01-15T18:17:39.192858+0000","last_peered":"2024-01-15T18:17:39.192858+0000","last_clean":"2024-01-15T18:17:39.192858+0000","last_became_active":"2024-01-15T18:12:11.043735+0000","last_became_peered":"2024-01-15T18:12:11.043735+0000","last_unstale":"2024-01-15T18:17:39.192858+0000","last_undegraded":"2024-01-15T18:17:39.192858+0000","last_fullsized":"2024-01-15T18:17:39.192858+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.728315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.424881+0000","last_change":"2024-01-15T18:12:10.027277+0000","last_active":"2024-01-15T18:14:27.424881+0000","last_peered":"2024-01-15T18:14:27.424881+0000","last_clean":"2024-01-15T18:14:27.424881+0000","last_became_active":"2024-01-15T18:12:10.026730+0000","last_became_peered":"2024-01-15T18:12:10.026730+0000","last_unstale":"2024-01-15T18:14:27.424881+0000","last_undegraded":"2024-01-15T18:14:27.424881+0000","last_fullsized":"2024-01-15T18:14:27.424881+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:52.737012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"117'36","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.289041+0000","last_change":"2024-01-15T18:12:11.061270+0000","last_active":"2024-01-15T18:17:39.289041+0000","last_peered":"2024-01-15T18:17:39.289041+0000","last_clean":"2024-01-15T18:17:39.289041+0000","last_became_active":"2024-01-15T18:12:11.061002+0000","last_became_peered":"2024-01-15T18:12:11.061002+0000","last_unstale":"2024-01-15T18:17:39.289041+0000","last_undegraded":"2024-01-15T18:17:39.289041+0000","last_fullsized":"2024-01-15T18:17:39.289041+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:10.142975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.284329+0000","last_change":"2024-01-15T18:12:10.020526+0000","last_active":"2024-01-15T18:14:27.284329+0000","last_peered":"2024-01-15T18:14:27.284329+0000","last_clean":"2024-01-15T18:14:27.284329+0000","last_became_active":"2024-01-15T18:12:10.019706+0000","last_became_peered":"2024-01-15T18:12:10.019706+0000","last_unstale":"2024-01-15T18:14:27.284329+0000","last_undegraded":"2024-01-15T18:14:27.284329+0000","last_fullsized":"2024-01-15T18:14:27.284329+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:49.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"117'44","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.211608+0000","last_change":"2024-01-15T18:12:11.060480+0000","last_active":"2024-01-15T18:17:39.211608+0000","last_peered":"2024-01-15T18:17:39.211608+0000","last_clean":"2024-01-15T18:17:39.211608+0000","last_became_active":"2024-01-15T18:12:11.060154+0000","last_became_peered":"2024-01-15T18:12:11.060154+0000","last_unstale":"2024-01-15T18:17:39.211608+0000","last_undegraded":"2024-01-15T18:17:39.211608+0000","last_fullsized":"2024-01-15T18:17:39.211608+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:52.364460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.542920+0000","last_change":"2024-01-15T18:12:10.003308+0000","last_active":"2024-01-15T18:14:27.542920+0000","last_peered":"2024-01-15T18:14:27.542920+0000","last_clean":"2024-01-15T18:14:27.542920+0000","last_became_active":"2024-01-15T18:12:10.002957+0000","last_became_peered":"2024-01-15T18:12:10.002957+0000","last_unstale":"2024-01-15T18:14:27.542920+0000","last_undegraded":"2024-01-15T18:14:27.542920+0000","last_fullsized":"2024-01-15T18:14:27.542920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:43.777497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"117'41","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.319156+0000","last_change":"2024-01-15T18:12:11.059234+0000","last_active":"2024-01-15T18:17:39.319156+0000","last_peered":"2024-01-15T18:17:39.319156+0000","last_clean":"2024-01-15T18:17:39.319156+0000","last_became_active":"2024-01-15T18:12:11.058945+0000","last_became_peered":"2024-01-15T18:12:11.058945+0000","last_unstale":"2024-01-15T18:17:39.319156+0000","last_undegraded":"2024-01-15T18:17:39.319156+0000","last_fullsized":"2024-01-15T18:17:39.319156+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:37.113206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.411059+0000","last_change":"2024-01-15T18:12:10.028589+0000","last_active":"2024-01-15T18:14:27.411059+0000","last_peered":"2024-01-15T18:14:27.411059+0000","last_clean":"2024-01-15T18:14:27.411059+0000","last_became_active":"2024-01-15T18:12:10.028284+0000","last_became_peered":"2024-01-15T18:12:10.028284+0000","last_unstale":"2024-01-15T18:14:27.411059+0000","last_undegraded":"2024-01-15T18:14:27.411059+0000","last_fullsized":"2024-01-15T18:14:27.411059+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:46:23.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"117'45","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.543742+0000","last_change":"2024-01-15T18:12:11.051787+0000","last_active":"2024-01-15T18:17:39.543742+0000","last_peered":"2024-01-15T18:17:39.543742+0000","last_clean":"2024-01-15T18:17:39.543742+0000","last_became_active":"2024-01-15T18:12:11.051403+0000","last_became_peered":"2024-01-15T18:12:11.051403+0000","last_unstale":"2024-01-15T18:17:39.543742+0000","last_undegraded":"2024-01-15T18:17:39.543742+0000","last_fullsized":"2024-01-15T18:17:39.543742+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:05:35.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"117'3","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.284165+0000","last_change":"2024-01-15T18:12:10.021913+0000","last_active":"2024-01-15T18:14:27.284165+0000","last_peered":"2024-01-15T18:14:27.284165+0000","last_clean":"2024-01-15T18:14:27.284165+0000","last_became_active":"2024-01-15T18:12:10.021344+0000","last_became_peered":"2024-01-15T18:12:10.021344+0000","last_unstale":"2024-01-15T18:14:27.284165+0000","last_undegraded":"2024-01-15T18:14:27.284165+0000","last_fullsized":"2024-01-15T18:14:27.284165+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:00.107213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"117'43","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.906847+0000","last_change":"2024-01-15T18:12:11.059848+0000","last_active":"2024-01-15T18:17:38.906847+0000","last_peered":"2024-01-15T18:17:38.906847+0000","last_clean":"2024-01-15T18:17:38.906847+0000","last_became_active":"2024-01-15T18:12:11.059311+0000","last_became_peered":"2024-01-15T18:12:11.059311+0000","last_unstale":"2024-01-15T18:17:38.906847+0000","last_undegraded":"2024-01-15T18:17:38.906847+0000","last_fullsized":"2024-01-15T18:17:38.906847+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:13.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.283095+0000","last_change":"2024-01-15T18:12:10.018301+0000","last_active":"2024-01-15T18:14:22.283095+0000","last_peered":"2024-01-15T18:14:22.283095+0000","last_clean":"2024-01-15T18:14:22.283095+0000","last_became_active":"2024-01-15T18:12:10.017411+0000","last_became_peered":"2024-01-15T18:12:10.017411+0000","last_unstale":"2024-01-15T18:14:22.283095+0000","last_undegraded":"2024-01-15T18:14:22.283095+0000","last_fullsized":"2024-01-15T18:14:22.283095+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:32.688781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.231615+0000","last_change":"2024-01-15T18:12:11.056765+0000","last_active":"2024-01-15T18:17:39.231615+0000","last_peered":"2024-01-15T18:17:39.231615+0000","last_clean":"2024-01-15T18:17:39.231615+0000","last_became_active":"2024-01-15T18:12:11.056311+0000","last_became_peered":"2024-01-15T18:12:11.056311+0000","last_unstale":"2024-01-15T18:17:39.231615+0000","last_undegraded":"2024-01-15T18:17:39.231615+0000","last_fullsized":"2024-01-15T18:17:39.231615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:53.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.282960+0000","last_change":"2024-01-15T18:12:10.023638+0000","last_active":"2024-01-15T18:14:22.282960+0000","last_peered":"2024-01-15T18:14:22.282960+0000","last_clean":"2024-01-15T18:14:22.282960+0000","last_became_active":"2024-01-15T18:12:10.022687+0000","last_became_peered":"2024-01-15T18:12:10.022687+0000","last_unstale":"2024-01-15T18:14:22.282960+0000","last_undegraded":"2024-01-15T18:14:22.282960+0000","last_fullsized":"2024-01-15T18:14:22.282960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:54.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"117'42","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.425793+0000","last_change":"2024-01-15T18:12:11.062171+0000","last_active":"2024-01-15T18:17:39.425793+0000","last_peered":"2024-01-15T18:17:39.425793+0000","last_clean":"2024-01-15T18:17:39.425793+0000","last_became_active":"2024-01-15T18:12:11.061830+0000","last_became_peered":"2024-01-15T18:12:11.061830+0000","last_unstale":"2024-01-15T18:17:39.425793+0000","last_undegraded":"2024-01-15T18:17:39.425793+0000","last_fullsized":"2024-01-15T18:17:39.425793+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:50.541819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109124,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.423571+0000","last_change":"2024-01-15T18:12:10.024893+0000","last_active":"2024-01-15T18:14:22.423571+0000","last_peered":"2024-01-15T18:14:22.423571+0000","last_clean":"2024-01-15T18:14:22.423571+0000","last_became_active":"2024-01-15T18:12:10.024554+0000","last_became_peered":"2024-01-15T18:12:10.024554+0000","last_unstale":"2024-01-15T18:14:22.423571+0000","last_undegraded":"2024-01-15T18:14:22.423571+0000","last_fullsized":"2024-01-15T18:14:22.423571+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:51.350660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"117'36","reported_seq":55,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.210156+0000","last_change":"2024-01-15T18:12:11.051179+0000","last_active":"2024-01-15T18:17:39.210156+0000","last_peered":"2024-01-15T18:17:39.210156+0000","last_clean":"2024-01-15T18:17:39.210156+0000","last_became_active":"2024-01-15T18:12:11.050747+0000","last_became_peered":"2024-01-15T18:12:11.050747+0000","last_unstale":"2024-01-15T18:17:39.210156+0000","last_undegraded":"2024-01-15T18:17:39.210156+0000","last_fullsized":"2024-01-15T18:17:39.210156+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.305559+0000","last_change":"2024-01-15T18:12:10.025674+0000","last_active":"2024-01-15T18:14:22.305559+0000","last_peered":"2024-01-15T18:14:22.305559+0000","last_clean":"2024-01-15T18:14:22.305559+0000","last_became_active":"2024-01-15T18:12:10.025356+0000","last_became_peered":"2024-01-15T18:12:10.025356+0000","last_unstale":"2024-01-15T18:14:22.305559+0000","last_undegraded":"2024-01-15T18:14:22.305559+0000","last_fullsized":"2024-01-15T18:14:22.305559+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:06:50.285646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"117'41","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.010642+0000","last_change":"2024-01-15T18:12:11.050465+0000","last_active":"2024-01-15T18:17:39.010642+0000","last_peered":"2024-01-15T18:17:39.010642+0000","last_clean":"2024-01-15T18:17:39.010642+0000","last_became_active":"2024-01-15T18:12:11.050116+0000","last_became_peered":"2024-01-15T18:12:11.050116+0000","last_unstale":"2024-01-15T18:17:39.010642+0000","last_undegraded":"2024-01-15T18:17:39.010642+0000","last_fullsized":"2024-01-15T18:17:39.010642+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:45:04.891353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"117'17","reported_seq":33,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:55.208163+0000","last_change":"2024-01-15T18:12:10.020848+0000","last_active":"2024-01-15T18:16:55.208163+0000","last_peered":"2024-01-15T18:16:55.208163+0000","last_clean":"2024-01-15T18:16:55.208163+0000","last_became_active":"2024-01-15T18:12:10.020497+0000","last_became_peered":"2024-01-15T18:12:10.020497+0000","last_unstale":"2024-01-15T18:16:55.208163+0000","last_undegraded":"2024-01-15T18:16:55.208163+0000","last_fullsized":"2024-01-15T18:16:55.208163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:37.798936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"117'46","reported_seq":75,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.193190+0000","last_change":"2024-01-15T18:12:11.051721+0000","last_active":"2024-01-15T18:17:39.193190+0000","last_peered":"2024-01-15T18:17:39.193190+0000","last_clean":"2024-01-15T18:17:39.193190+0000","last_became_active":"2024-01-15T18:12:11.050810+0000","last_became_peered":"2024-01-15T18:12:11.050810+0000","last_unstale":"2024-01-15T18:17:39.193190+0000","last_undegraded":"2024-01-15T18:17:39.193190+0000","last_fullsized":"2024-01-15T18:17:39.193190+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:34:44.264660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.422714+0000","last_change":"2024-01-15T18:12:10.025006+0000","last_active":"2024-01-15T18:14:17.422714+0000","last_peered":"2024-01-15T18:14:17.422714+0000","last_clean":"2024-01-15T18:14:17.422714+0000","last_became_active":"2024-01-15T18:12:10.024524+0000","last_became_peered":"2024-01-15T18:12:10.024524+0000","last_unstale":"2024-01-15T18:14:17.422714+0000","last_undegraded":"2024-01-15T18:14:17.422714+0000","last_fullsized":"2024-01-15T18:14:17.422714+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:16.499218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"117'39","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.572497+0000","last_change":"2024-01-15T18:12:11.061751+0000","last_active":"2024-01-15T18:17:38.572497+0000","last_peered":"2024-01-15T18:17:38.572497+0000","last_clean":"2024-01-15T18:17:38.572497+0000","last_became_active":"2024-01-15T18:12:11.061454+0000","last_became_peered":"2024-01-15T18:12:11.061454+0000","last_unstale":"2024-01-15T18:17:38.572497+0000","last_undegraded":"2024-01-15T18:17:38.572497+0000","last_fullsized":"2024-01-15T18:17:38.572497+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.304508+0000","last_change":"2024-01-15T18:12:10.018428+0000","last_active":"2024-01-15T18:14:17.304508+0000","last_peered":"2024-01-15T18:14:17.304508+0000","last_clean":"2024-01-15T18:14:17.304508+0000","last_became_active":"2024-01-15T18:12:10.018057+0000","last_became_peered":"2024-01-15T18:12:10.018057+0000","last_unstale":"2024-01-15T18:14:17.304508+0000","last_undegraded":"2024-01-15T18:14:17.304508+0000","last_fullsized":"2024-01-15T18:14:17.304508+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:52:09.544387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"117'44","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.352556+0000","last_change":"2024-01-15T18:12:11.039995+0000","last_active":"2024-01-15T18:17:39.352556+0000","last_peered":"2024-01-15T18:17:39.352556+0000","last_clean":"2024-01-15T18:17:39.352556+0000","last_became_active":"2024-01-15T18:12:11.039586+0000","last_became_peered":"2024-01-15T18:12:11.039586+0000","last_unstale":"2024-01-15T18:17:39.352556+0000","last_undegraded":"2024-01-15T18:17:39.352556+0000","last_fullsized":"2024-01-15T18:17:39.352556+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:38.812736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.408970+0000","last_change":"2024-01-15T18:12:10.019758+0000","last_active":"2024-01-15T18:14:17.408970+0000","last_peered":"2024-01-15T18:14:17.408970+0000","last_clean":"2024-01-15T18:14:17.408970+0000","last_became_active":"2024-01-15T18:12:10.019424+0000","last_became_peered":"2024-01-15T18:12:10.019424+0000","last_unstale":"2024-01-15T18:14:17.408970+0000","last_undegraded":"2024-01-15T18:14:17.408970+0000","last_fullsized":"2024-01-15T18:14:17.408970+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:45.797791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"117'56","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.420858+0000","last_change":"2024-01-15T18:12:11.035857+0000","last_active":"2024-01-15T18:17:39.420858+0000","last_peered":"2024-01-15T18:17:39.420858+0000","last_clean":"2024-01-15T18:17:39.420858+0000","last_became_active":"2024-01-15T18:12:11.035238+0000","last_became_peered":"2024-01-15T18:12:11.035238+0000","last_unstale":"2024-01-15T18:17:39.420858+0000","last_undegraded":"2024-01-15T18:17:39.420858+0000","last_fullsized":"2024-01-15T18:17:39.420858+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.326548+0000","last_change":"2024-01-15T18:12:10.017511+0000","last_active":"2024-01-15T18:14:17.326548+0000","last_peered":"2024-01-15T18:14:17.326548+0000","last_clean":"2024-01-15T18:14:17.326548+0000","last_became_active":"2024-01-15T18:12:10.017190+0000","last_became_peered":"2024-01-15T18:12:10.017190+0000","last_unstale":"2024-01-15T18:14:17.326548+0000","last_undegraded":"2024-01-15T18:14:17.326548+0000","last_fullsized":"2024-01-15T18:14:17.326548+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"117'46","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.384006+0000","last_change":"2024-01-15T18:12:11.056667+0000","last_active":"2024-01-15T18:17:39.384006+0000","last_peered":"2024-01-15T18:17:39.384006+0000","last_clean":"2024-01-15T18:17:39.384006+0000","last_became_active":"2024-01-15T18:12:11.056111+0000","last_became_peered":"2024-01-15T18:12:11.056111+0000","last_unstale":"2024-01-15T18:17:39.384006+0000","last_undegraded":"2024-01-15T18:17:39.384006+0000","last_fullsized":"2024-01-15T18:17:39.384006+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:18:00.144603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"117'10","reported_seq":35,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:33.616962+0000","last_change":"2024-01-15T18:12:10.021210+0000","last_active":"2024-01-15T18:15:33.616962+0000","last_peered":"2024-01-15T18:15:33.616962+0000","last_clean":"2024-01-15T18:15:33.616962+0000","last_became_active":"2024-01-15T18:12:10.020890+0000","last_became_peered":"2024-01-15T18:12:10.020890+0000","last_unstale":"2024-01-15T18:15:33.616962+0000","last_undegraded":"2024-01-15T18:15:33.616962+0000","last_fullsized":"2024-01-15T18:15:33.616962+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:44.292512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"117'39","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.353330+0000","last_change":"2024-01-15T18:12:11.059132+0000","last_active":"2024-01-15T18:17:39.353330+0000","last_peered":"2024-01-15T18:17:39.353330+0000","last_clean":"2024-01-15T18:17:39.353330+0000","last_became_active":"2024-01-15T18:12:11.057943+0000","last_became_peered":"2024-01-15T18:12:11.057943+0000","last_unstale":"2024-01-15T18:17:39.353330+0000","last_undegraded":"2024-01-15T18:17:39.353330+0000","last_fullsized":"2024-01-15T18:17:39.353330+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:52.248678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:12.421404+0000","last_change":"2024-01-15T18:12:10.010681+0000","last_active":"2024-01-15T18:14:12.421404+0000","last_peered":"2024-01-15T18:14:12.421404+0000","last_clean":"2024-01-15T18:14:12.421404+0000","last_became_active":"2024-01-15T18:12:10.009870+0000","last_became_peered":"2024-01-15T18:12:10.009870+0000","last_unstale":"2024-01-15T18:14:12.421404+0000","last_undegraded":"2024-01-15T18:14:12.421404+0000","last_fullsized":"2024-01-15T18:14:12.421404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:21.835760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"117'31","reported_seq":54,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.239562+0000","last_change":"2024-01-15T18:12:11.051496+0000","last_active":"2024-01-15T18:17:39.239562+0000","last_peered":"2024-01-15T18:17:39.239562+0000","last_clean":"2024-01-15T18:17:39.239562+0000","last_became_active":"2024-01-15T18:12:11.051092+0000","last_became_peered":"2024-01-15T18:12:11.051092+0000","last_unstale":"2024-01-15T18:17:39.239562+0000","last_undegraded":"2024-01-15T18:17:39.239562+0000","last_fullsized":"2024-01-15T18:17:39.239562+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:38:09.219058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.412208+0000","last_change":"2024-01-15T18:12:10.026923+0000","last_active":"2024-01-15T18:14:32.412208+0000","last_peered":"2024-01-15T18:14:32.412208+0000","last_clean":"2024-01-15T18:14:32.412208+0000","last_became_active":"2024-01-15T18:12:10.026651+0000","last_became_peered":"2024-01-15T18:12:10.026651+0000","last_unstale":"2024-01-15T18:14:32.412208+0000","last_undegraded":"2024-01-15T18:14:32.412208+0000","last_fullsized":"2024-01-15T18:14:32.412208+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:14.981958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":251,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:02.417603+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:15:02.417603+0000","last_peered":"2024-01-15T18:15:02.417603+0000","last_clean":"2024-01-15T18:15:02.417603+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:15:02.417603+0000","last_undegraded":"2024-01-15T18:15:02.417603+0000","last_fullsized":"2024-01-15T18:15:02.417603+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710133,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2810,"ondisk_log_size":2810,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":634396,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":274,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1335296,"data_stored":1268792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":254,"ondisk_log_size":254,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542421,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834588,"kb_used_data":1767380,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537252,"statfs":{"total":96636764160,"available":94758146048,"internally_reserved":0,"allocated":1809797120,"data_stored":1808970858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.254,"15min":1.383},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.40400000000000003},"max":{"1min":1.599,"5min":4.0229999999999997,"15min":19.843},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.49,"15min":1.5609999999999999},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.53400000000000003},"max":{"1min":1.6319999999999999,"5min":16.678999999999998,"15min":30.498000000000001},"last":1.552}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2649999999999999,"15min":1.397},"min":{"1min":0.91400000000000003,"5min":0.56599999999999995,"15min":0.45600000000000002},"max":{"1min":1.5700000000000001,"5min":3.9430000000000001,"15min":19.917000000000002},"last":1.25},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.54,"15min":1.5669999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.71199999999999997},"max":{"1min":1.6319999999999999,"5min":16.626999999999999,"15min":28.613},"last":1.196}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2689999999999999,"15min":1.391},"min":{"1min":0.98399999999999999,"5min":0.75600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6479999999999999,"5min":1.8140000000000001,"15min":12.061},"last":1.752},{"interface":"front","average":{"1min":1.355,"5min":1.528,"15min":1.5800000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.48999999999999999},"max":{"1min":1.77,"5min":16.792000000000002,"15min":16.792000000000002},"last":1.4139999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.306,"15min":1.5229999999999999},"min":{"1min":1.0029999999999999,"5min":0.80700000000000005,"15min":0.56000000000000005},"max":{"1min":1.6180000000000001,"5min":1.734,"15min":16.632000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.619,"15min":1.639},"min":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.56699999999999995},"max":{"1min":1.6930000000000001,"5min":16.129000000000001,"15min":16.129000000000001},"last":1.6100000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.571,"15min":1.627},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.54200000000000004},"max":{"1min":1.585,"5min":16.454000000000001,"15min":23.042000000000002},"last":1.657},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.357,"15min":1.677},"min":{"1min":0.873,"5min":0.80000000000000004,"15min":0.496},"max":{"1min":1.7410000000000001,"5min":1.885,"15min":38.682000000000002},"last":1.8460000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4259999999999999,"15min":1.5860000000000001},"min":{"1min":0.96799999999999997,"5min":0.76800000000000002,"15min":0.52700000000000002},"max":{"1min":1.867,"5min":4.5359999999999996,"15min":13.379},"last":1.679},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.71,"15min":1.7430000000000001},"min":{"1min":1.0649999999999999,"5min":0.98499999999999999,"15min":0.624},"max":{"1min":1.843,"5min":16.553000000000001,"15min":22.933},"last":1.706}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.7969999999999999,"15min":1.8600000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":0.57399999999999995},"max":{"1min":1.8140000000000001,"5min":16.725999999999999,"15min":23.004000000000001},"last":1.804},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4850000000000001,"15min":1.659},"min":{"1min":1.1220000000000001,"5min":1.008,"15min":0.58299999999999996},"max":{"1min":1.897,"5min":1.952,"15min":9.2599999999999998},"last":1.9550000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542420,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1256408,"kb_used_data":1193360,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93115432,"statfs":{"total":96636764160,"available":95350202368,"internally_reserved":0,"allocated":1222000640,"data_stored":1221177506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2629999999999999,"15min":1.464},"min":{"1min":0.79600000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":31.805},"last":1.361},{"interface":"front","average":{"1min":1.244,"5min":1.3,"15min":1.4870000000000001},"min":{"1min":0.749,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8340000000000001,"5min":3.5409999999999999,"15min":31.702999999999999},"last":1.411}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1339999999999999,"15min":1.2609999999999999},"min":{"1min":0.69699999999999995,"5min":0.59999999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.778,"15min":15.064},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2050000000000001,"15min":1.4299999999999999},"min":{"1min":0.86499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6279999999999999,"5min":1.9650000000000001,"15min":23.495999999999999},"last":0.92400000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2849999999999999,"15min":1.5349999999999999},"min":{"1min":0.81699999999999995,"5min":0.79100000000000004,"15min":0.59399999999999997},"max":{"1min":1.502,"5min":1.8899999999999999,"15min":23.087},"last":1.081},{"interface":"front","average":{"1min":1.339,"5min":1.331,"15min":1.397},"min":{"1min":1.0089999999999999,"5min":0.77000000000000002,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.9990000000000001,"15min":14.94},"last":1.5309999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4359999999999999,"15min":1.6040000000000001},"min":{"1min":1.0620000000000001,"5min":0.65200000000000002,"15min":0.59899999999999998},"max":{"1min":1.7,"5min":1.8160000000000001,"15min":15.151999999999999},"last":1.698},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3779999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6080000000000001,"5min":3.8500000000000001,"15min":13.795},"last":1.573}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2729999999999999,"15min":1.585},"min":{"1min":0.90900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.53,"5min":3.3719999999999999,"15min":23.327000000000002},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.3560000000000001,"15min":1.548},"min":{"1min":0.80900000000000005,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.6839999999999999,"5min":1.8839999999999999,"15min":15.239000000000001},"last":1.1950000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.331,"15min":1.502},"min":{"1min":0.88700000000000001,"5min":0.58099999999999996,"15min":0.52500000000000002},"max":{"1min":1.7210000000000001,"5min":3.6480000000000001,"15min":13.593},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.55,"15min":1.8049999999999999},"min":{"1min":1.1930000000000001,"5min":0.66200000000000003,"15min":0.621},"max":{"1min":1.702,"5min":2.306,"15min":23.417999999999999},"last":1.595}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.47,"15min":1.716},"min":{"1min":0.98799999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8220000000000001,"5min":3.105,"15min":21.265000000000001},"last":1.252},{"interface":"front","average":{"1min":1.486,"5min":1.542,"15min":1.899},"min":{"1min":1.1200000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.7789999999999999,"5min":2.2229999999999999,"15min":23.738},"last":1.4790000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542420,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908440,"kb_used_data":832912,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93463400,"statfs":{"total":96636764160,"available":95706521600,"internally_reserved":0,"allocated":852901888,"data_stored":852079166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.202,"15min":1.2669999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.47799999999999998},"max":{"1min":1.595,"5min":2.7919999999999998,"15min":19.266999999999999},"last":1.054},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.752,"5min":0.748,"15min":0.51400000000000001},"max":{"1min":1.671,"5min":1.7529999999999999,"15min":3.1760000000000002},"last":0.99099999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.359,"15min":1.3149999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.60699999999999998},"max":{"1min":1.653,"5min":15.554,"15min":15.554},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.302,"15min":1.3540000000000001},"min":{"1min":0.80100000000000005,"5min":0.69599999999999995,"15min":0.59099999999999997},"max":{"1min":1.643,"5min":6.1449999999999996,"15min":9.2379999999999995},"last":1.1519999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.621},"max":{"1min":1.694,"5min":2.3500000000000001,"15min":4.0369999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.512,"15min":1.4299999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.56000000000000005},"max":{"1min":1.617,"5min":15.468999999999999,"15min":15.468999999999999},"last":1.4199999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.327,"15min":1.409},"min":{"1min":0.78200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.718,"5min":1.984,"15min":9.1929999999999996},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.357,"5min":1.4379999999999999,"15min":1.4650000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.64600000000000002},"max":{"1min":1.774,"5min":2.2949999999999999,"15min":7.0110000000000001},"last":1.0920000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.4319999999999999,"15min":1.47},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.41399999999999998},"max":{"1min":1.6040000000000001,"5min":15.254,"15min":15.254},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.4019999999999999,"15min":1.4710000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.8080000000000001,"5min":1.923,"15min":9.1519999999999992},"last":1.228}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.456},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.39600000000000002},"max":{"1min":1.8480000000000001,"5min":2.2250000000000001,"15min":3.7559999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.593,"15min":1.5680000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53000000000000003},"max":{"1min":1.7330000000000001,"5min":15.366,"15min":15.366},"last":1.625}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6100000000000001},"min":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.45800000000000002},"max":{"1min":1.873,"5min":2.0609999999999999,"15min":9.0600000000000005},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.522,"5min":1.5840000000000001,"15min":1.6890000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.57299999999999995},"max":{"1min":1.899,"5min":3.1659999999999999,"15min":19.356999999999999},"last":1.504}]}]},{"osd":4,"up_from":14,"seq":60129542420,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1711124,"kb_used_data":1643916,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92660716,"statfs":{"total":96636764160,"available":94884573184,"internally_reserved":0,"allocated":1683369984,"data_stored":1682551014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.264,"15min":1.3360000000000001},"min":{"1min":0.94799999999999995,"5min":0.70199999999999996,"15min":0.39200000000000002},"max":{"1min":1.601,"5min":2.004,"15min":17.346},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.3500000000000001,"15min":1.361},"min":{"1min":1.0900000000000001,"5min":0.97499999999999998,"15min":0.40100000000000002},"max":{"1min":1.744,"5min":1.804,"15min":7.9160000000000004},"last":1.6120000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.1950000000000001,"15min":1.24},"min":{"1min":0.95199999999999996,"5min":0.76600000000000001,"15min":0.439},"max":{"1min":1.6699999999999999,"5min":1.861,"15min":11.128},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.23,"15min":1.234},"min":{"1min":0.876,"5min":0.82999999999999996,"15min":0.60799999999999998},"max":{"1min":1.619,"5min":1.7470000000000001,"15min":5.6109999999999998},"last":1.4059999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.294,"15min":1.343},"min":{"1min":0.92300000000000004,"5min":0.83399999999999996,"15min":0.56899999999999995},"max":{"1min":1.544,"5min":1.8560000000000001,"15min":18.442},"last":1.266},{"interface":"front","average":{"1min":1.341,"5min":1.3720000000000001,"15min":1.399},"min":{"1min":1.1459999999999999,"5min":0.94799999999999995,"15min":0.81899999999999995},"max":{"1min":1.669,"5min":1.861,"15min":5.7050000000000001},"last":1.5649999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.375,"15min":1.462},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.56699999999999995},"max":{"1min":1.6719999999999999,"5min":2.1859999999999999,"15min":17.364999999999998},"last":1.351},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3380000000000001,"15min":1.3899999999999999},"min":{"1min":1.022,"5min":0.85099999999999998,"15min":0.45300000000000001},"max":{"1min":1.601,"5min":1.7050000000000001,"15min":9.3320000000000007},"last":1.298}]},{"osd":5,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.29,"15min":1.4590000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.49099999999999999},"max":{"1min":1.6519999999999999,"5min":2.4089999999999998,"15min":29.422000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3420000000000001,"15min":1.4379999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.53700000000000003},"max":{"1min":1.7549999999999999,"5min":1.9450000000000001,"15min":12.332000000000001},"last":1.4990000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2629999999999999,"15min":1.3899999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.54200000000000004},"max":{"1min":1.752,"5min":2.117,"15min":11.09},"last":1.212},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.482,"15min":1.6120000000000001},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":0.51100000000000001},"max":{"1min":1.718,"5min":2.2639999999999998,"15min":17.224},"last":1.004}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.504,"15min":1.593},"min":{"1min":1.0169999999999999,"5min":0.93300000000000005,"15min":0.60499999999999998},"max":{"1min":1.8400000000000001,"5min":2.052,"15min":12.247},"last":1.101},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6439999999999999,"15min":1.73},"min":{"1min":1.294,"5min":1.159,"15min":0.61399999999999999},"max":{"1min":1.9039999999999999,"5min":2.3570000000000002,"15min":17.324999999999999},"last":1.538}]}]},{"osd":3,"up_from":14,"seq":60129542422,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462032,"kb_used_data":1386504,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":92909808,"statfs":{"total":96636764160,"available":95139643392,"internally_reserved":0,"allocated":1419780096,"data_stored":1418942469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0349999999999999,"15min":1.1639999999999999},"min":{"1min":0.66400000000000003,"5min":0.52800000000000002,"15min":0.33300000000000002},"max":{"1min":1.7989999999999999,"5min":2.915,"15min":18.399999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2509999999999999,"15min":1.2849999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.44400000000000001},"max":{"1min":1.8560000000000001,"5min":3.028,"15min":18.795999999999999},"last":1.3140000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.05,"15min":1.1299999999999999},"min":{"1min":0.65800000000000003,"5min":0.51200000000000001,"15min":0.32600000000000001},"max":{"1min":1.4870000000000001,"5min":5.3520000000000003,"15min":12.372},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.1180000000000001,"15min":1.2589999999999999},"min":{"1min":0.66000000000000003,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.736,"5min":4.125,"15min":11.228999999999999},"last":1.2589999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.218,"15min":1.2529999999999999},"min":{"1min":0.74199999999999999,"5min":0.59999999999999998,"15min":0.375},"max":{"1min":1.8680000000000001,"5min":2.3929999999999998,"15min":10.964},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.45,"15min":1.534},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7569999999999999,"5min":3.0659999999999998,"15min":20.495000000000001},"last":1.361}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.389,"15min":1.51},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.67200000000000004},"max":{"1min":1.865,"5min":2.1139999999999999,"15min":13.536},"last":1.708},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4530000000000001,"15min":1.5529999999999999},"min":{"1min":0.98999999999999999,"5min":0.83099999999999996,"15min":0.63700000000000001},"max":{"1min":1.905,"5min":6.2439999999999998,"15min":16.398},"last":1.5720000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.478,"15min":1.6519999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.67200000000000004},"max":{"1min":2.0670000000000002,"5min":2.7269999999999999,"15min":20.606000000000002},"last":1.675},{"interface":"front","average":{"1min":1.587,"5min":1.6060000000000001,"15min":1.631},"min":{"1min":1.0269999999999999,"5min":0.94299999999999995,"15min":0.66800000000000004},"max":{"1min":2.0489999999999999,"5min":2.617,"15min":11.603},"last":1.625}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.554,"15min":1.675},"min":{"1min":0.93100000000000005,"5min":0.85499999999999998,"15min":0.53500000000000003},"max":{"1min":2.1230000000000002,"5min":6.3289999999999997,"15min":13.648999999999999},"last":1.917},{"interface":"front","average":{"1min":1.554,"5min":1.6100000000000001,"15min":1.7529999999999999},"min":{"1min":1.2190000000000001,"5min":1.0249999999999999,"15min":0.65800000000000003},"max":{"1min":1.849,"5min":6.367,"15min":20.556000000000001},"last":1.6080000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.641,"15min":1.76},"min":{"1min":1.159,"5min":0.97899999999999998,"15min":0.60499999999999998},"max":{"1min":2.1269999999999998,"5min":4.8630000000000004,"15min":28.843},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6559999999999999,"15min":1.738},"min":{"1min":1.1830000000000001,"5min":0.80100000000000005,"15min":0.67200000000000004},"max":{"1min":2.0680000000000001,"5min":6.9569999999999999,"15min":15.143000000000001},"last":1.7509999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542422,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":982100,"kb_used_data":910732,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93389740,"statfs":{"total":96636764160,"available":95631093760,"internally_reserved":0,"allocated":932589568,"data_stored":931771659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.161,"15min":1.083},"min":{"1min":0.67600000000000005,"5min":0.51000000000000001,"15min":0.26600000000000001},"max":{"1min":1.419,"5min":22.707000000000001,"15min":22.707000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1970000000000001,"15min":1.357},"min":{"1min":0.76300000000000001,"5min":0.61099999999999999,"15min":0.39300000000000002},"max":{"1min":1.6879999999999999,"5min":2.5710000000000002,"15min":15.314},"last":1.2010000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1759999999999999,"15min":1.3240000000000001},"min":{"1min":0.73299999999999998,"5min":0.58599999999999997,"15min":0.38300000000000001},"max":{"1min":1.546,"5min":2.8679999999999999,"15min":34.463999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.327,"15min":1.504},"min":{"1min":0.90700000000000003,"5min":0.73899999999999999,"15min":0.373},"max":{"1min":1.675,"5min":2.794,"15min":25.859999999999999},"last":1.3200000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1870000000000001,"15min":1.302},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.34399999999999997},"max":{"1min":1.464,"5min":2.7080000000000002,"15min":12.93},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.6919999999999999,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.46400000000000002},"max":{"1min":1.903,"5min":37.021999999999998,"15min":37.021999999999998},"last":1.601}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6879999999999999},"min":{"1min":1.0289999999999999,"5min":0.96999999999999997,"15min":0.55700000000000005},"max":{"1min":1.78,"5min":3.419,"15min":29.986999999999998},"last":1.496},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.518,"15min":1.7150000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.56799999999999995},"max":{"1min":1.883,"5min":2.976,"15min":21.199999999999999},"last":1.4590000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.456,"15min":1.6499999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.51500000000000001},"max":{"1min":2.0880000000000001,"5min":2.218,"15min":14.066000000000001},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.643,"15min":1.7849999999999999},"min":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":0.60899999999999999},"max":{"1min":1.8180000000000001,"5min":3.4790000000000001,"15min":14.025},"last":1.76}]},{"osd":6,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.6160000000000001,"15min":1.8069999999999999},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":0.68000000000000005},"max":{"1min":1.9910000000000001,"5min":3.0800000000000001,"15min":15.683999999999999},"last":1.659},{"interface":"front","average":{"1min":1.409,"5min":1.585,"15min":1.784},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.58899999999999997},"max":{"1min":2.2930000000000001,"5min":10.371,"15min":12.629},"last":1.546}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.671,"15min":2.1419999999999999},"min":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":0.63100000000000001},"max":{"1min":2.2210000000000001,"5min":3.177,"15min":57.006999999999998},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.595,"5min":1.6299999999999999,"15min":1.9630000000000001},"min":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":0.56100000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":22.853999999999999},"last":1.8149999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542421,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1137764,"kb_used_data":1066396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93234076,"statfs":{"total":96636764160,"available":95471693824,"internally_reserved":0,"allocated":1091989504,"data_stored":1091155706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.96499999999999997,"15min":1.177},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.34399999999999997},"max":{"1min":1.1419999999999999,"5min":2.7389999999999999,"15min":30.460999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.115,"15min":1.26},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.314},"max":{"1min":1.6539999999999999,"5min":5.4779999999999998,"15min":18.658000000000001},"last":1.093}]},{"osd":2,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.075,"15min":1.1579999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.5589999999999999,"5min":1.7809999999999999,"15min":7.117},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.325,"15min":1.5129999999999999},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.728,"5min":1.98,"15min":30.593},"last":1.03}]},{"osd":3,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2529999999999999,"15min":1.45},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.48599999999999999},"max":{"1min":1.6779999999999999,"5min":1.889,"15min":30.547000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.1619999999999999,"15min":1.3200000000000001},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":2.0459999999999998,"15min":17.263000000000002},"last":1.268}]},{"osd":4,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.6639999999999999,"15min":1.6899999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.72699999999999998},"max":{"1min":1.6319999999999999,"5min":14.103,"15min":17.420000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.653,"15min":1.6040000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7849999999999999,"5min":14.140000000000001,"15min":14.140000000000001},"last":1.371}]},{"osd":5,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3859999999999999,"15min":1.5329999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.59599999999999997},"max":{"1min":1.843,"5min":2.077,"15min":9.0440000000000005},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.508,"15min":1.7989999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.68300000000000005},"max":{"1min":1.891,"5min":2.3599999999999999,"15min":34.042999999999999},"last":1.9850000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.494,"15min":1.6830000000000001},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":0.64100000000000001},"max":{"1min":1.9790000000000001,"5min":2.3780000000000001,"15min":34.158000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.4850000000000001,"15min":1.637},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.62},"max":{"1min":1.8260000000000001,"5min":2.5579999999999998,"15min":17.510000000000002},"last":1.9399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.603,"15min":1.8700000000000001},"min":{"1min":0.98299999999999998,"5min":0.82999999999999996,"15min":0.77300000000000002},"max":{"1min":1.998,"5min":2.4329999999999998,"15min":33.659999999999997},"last":2.129},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.6439999999999999,"15min":1.742},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.70299999999999996},"max":{"1min":1.9370000000000001,"5min":2.6669999999999998,"15min":11.882999999999999},"last":1.5569999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542422,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1769760,"kb_used_data":1698392,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":92602080,"statfs":{"total":96636764160,"available":94824529920,"internally_reserved":0,"allocated":1739153408,"data_stored":1738300216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.1080000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4630000000000001,"5min":2.5209999999999999,"15min":12.050000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3100000000000001,"15min":1.357},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.36399999999999999},"max":{"1min":1.609,"5min":6.6890000000000001,"15min":11.962},"last":0.98299999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.248,"15min":1.2230000000000001},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.45500000000000002},"max":{"1min":1.7050000000000001,"5min":6.8099999999999996,"15min":11.59},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.995,"5min":1.0620000000000001,"15min":1.1970000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.6870000000000001,"5min":1.825,"15min":14.512},"last":1.1899999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.385,"15min":1.425},"min":{"1min":0.58899999999999997,"5min":0.51300000000000001,"15min":0.50800000000000001},"max":{"1min":1.7130000000000001,"5min":6.8719999999999999,"15min":11.904},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1899999999999999,"15min":1.1830000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.35199999999999998},"max":{"1min":1.696,"5min":2.3919999999999999,"15min":6.1539999999999999},"last":1.1439999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.29,"15min":1.542},"min":{"1min":0.68000000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.5860000000000001,"5min":2.1829999999999998,"15min":32.115000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.579,"15min":1.6259999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.67700000000000005},"max":{"1min":1.6890000000000001,"5min":6.9329999999999998,"15min":11.992000000000001},"last":1.4970000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3879999999999999,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.60899999999999999,"15min":0.47499999999999998},"max":{"1min":1.514,"5min":2.593,"15min":32.174999999999997},"last":1.091},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.6020000000000001,"15min":1.6160000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.442},"max":{"1min":2.157,"5min":15.907,"15min":17.225000000000001},"last":1.464}]},{"osd":6,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.528,"15min":1.754},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.66600000000000004},"max":{"1min":2.3290000000000002,"5min":2.5590000000000002,"15min":24.039999999999999},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.49,"15min":1.7789999999999999},"min":{"1min":0.99199999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.758,"5min":2.5449999999999999,"15min":32.216999999999999},"last":1.2989999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.708,"15min":2.0800000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.69099999999999995},"max":{"1min":1.8300000000000001,"5min":7.0979999999999999,"15min":34.131},"last":1.395},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.641,"15min":1.984},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.61799999999999999},"max":{"1min":2.266,"5min":3.3759999999999999,"15min":44.694000000000003},"last":1.6559999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":632389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":647168,"data_stored":632347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1736458240,"data_stored":1736442609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":931135488,"data_stored":931135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":851443712,"data_stored":851443712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:17:51.829 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:17:52.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.087+0000 7f10c2200640 1 Processor -- start 2024-01-15T18:17:52.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.087+0000 7f10c2200640 1 -- start start 2024-01-15T18:17:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.089+0000 7f10c2200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f10bc143c40 0x7f10bc071290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.089+0000 7f10c2200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f10bc0ccc10 0x7f10bc06d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.092 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.089+0000 7f10c2200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10bc070e50 con 0x7f10bc0717d0 2024-01-15T18:17:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.089+0000 7f10c2200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc0723c0 con 0x7f10bc0ccc10 2024-01-15T18:17:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.089+0000 7f10c2200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10bc144020 con 0x7f10bc143c40 2024-01-15T18:17:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.089+0000 7f10c1800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f10bc0ccc10 0x7f10bc06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.090+0000 7f10c1800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f10bc0ccc10 0x7f10bc06d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56422/0 (socket says 172.21.15.26:56422) 2024-01-15T18:17:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.090+0000 7f10c1800640 1 -- 172.21.15.26:0/28346329 learned_addr learned my addr 172.21.15.26:0/28346329 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.090+0000 7f10c1800640 1 -- 172.21.15.26:0/28346329 >> v2:172.21.15.124:3300/0 conn(0x7f10bc143c40 msgr2=0x7f10bc071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.090+0000 7f10c0e00640 1 --2- 172.21.15.26:0/28346329 >> v2:172.21.15.124:3300/0 conn(0x7f10bc143c40 0x7f10bc071290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.093 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.090+0000 7f10c1800640 1 --2- 172.21.15.26:0/28346329 >> v2:172.21.15.124:3300/0 conn(0x7f10bc143c40 0x7f10bc071290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.090+0000 7f10c1800640 1 -- 172.21.15.26:0/28346329 >> v1:172.21.15.124:6789/0 conn(0x7f10bc0717d0 legacy=0x7f10bc0cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.090+0000 7f10c1800640 1 -- 172.21.15.26:0/28346329 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10bc071e00 con 0x7f10bc0ccc10 2024-01-15T18:17:52.094 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.090+0000 7f10c0e00640 1 --2- 172.21.15.26:0/28346329 >> v2:172.21.15.124:3300/0 conn(0x7f10bc143c40 0x7f10bc071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.091+0000 7f10c1800640 1 --2- 172.21.15.26:0/28346329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 0x7f10bc06d160 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f10bc06e410 tx=0x7f10b402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574bdfd48f79d111 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.092+0000 7f10bbe00640 1 -- 172.21.15.26:0/28346329 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b402c9f0 con 0x7f10bc0ccc10 2024-01-15T18:17:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.092+0000 7f10bbe00640 1 -- 172.21.15.26:0/28346329 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b402cb50 con 0x7f10bc0ccc10 2024-01-15T18:17:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.092+0000 7f10bbe00640 1 -- 172.21.15.26:0/28346329 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b40343f0 con 0x7f10bc0ccc10 2024-01-15T18:17:52.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.092+0000 7f10c2200640 1 -- 172.21.15.26:0/28346329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 msgr2=0x7f10bc06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.092+0000 7f10c2200640 1 --2- 172.21.15.26:0/28346329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 0x7f10bc06d160 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f10bc06e410 tx=0x7f10b402be50 comp rx=0 tx=0).stop 2024-01-15T18:17:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.092+0000 7f10c2200640 1 -- 172.21.15.26:0/28346329 shutdown_connections 2024-01-15T18:17:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.092+0000 7f10c2200640 1 --2- 172.21.15.26:0/28346329 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 0x7f10bc06d160 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.092+0000 7f10c2200640 1 --2- 172.21.15.26:0/28346329 >> v2:172.21.15.124:3300/0 conn(0x7f10bc143c40 0x7f10bc071290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.092+0000 7f10c2200640 1 -- 172.21.15.26:0/28346329 >> 172.21.15.26:0/28346329 conn(0x7f10bc06a800 msgr2=0x7f10bc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.093+0000 7f10c2200640 1 -- 172.21.15.26:0/28346329 shutdown_connections 2024-01-15T18:17:52.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.093+0000 7f10c2200640 1 -- 172.21.15.26:0/28346329 wait complete. 2024-01-15T18:17:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.095+0000 7f10c2200640 1 Processor -- start 2024-01-15T18:17:52.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.095+0000 7f10c2200640 1 -- start start 2024-01-15T18:17:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.096+0000 7f10c2200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10bc0717d0 0x7f10bc15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.096+0000 7f10c2200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 0x7f10bc0ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.096+0000 7f10c2200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10bc143c40 0x7f10bc0ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.096+0000 7f10c2200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc0ca890 con 0x7f10bc0ccc10 2024-01-15T18:17:52.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.096+0000 7f10c2200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc0cab70 con 0x7f10bc0717d0 2024-01-15T18:17:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.096+0000 7f10c2200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10bc0c3e10 con 0x7f10bc143c40 2024-01-15T18:17:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 0x7f10bc0ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10b3e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 0x7f10bc0ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56432/0 (socket says 172.21.15.26:56432) 2024-01-15T18:17:52.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10b3e00640 1 -- 172.21.15.26:0/3271604338 learned_addr learned my addr 172.21.15.26:0/3271604338 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10c1800640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10bc143c40 0x7f10bc0ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10c0e00640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10bc0717d0 0x7f10bc15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10b3e00640 1 -- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10bc143c40 msgr2=0x7f10bc0ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10b3e00640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10bc143c40 0x7f10bc0ade60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.101 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10b3e00640 1 -- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10bc0717d0 msgr2=0x7f10bc15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10b3e00640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10bc0717d0 0x7f10bc15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.097+0000 7f10b3e00640 1 -- 172.21.15.26:0/3271604338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b4009b30 con 0x7f10bc0ccc10 2024-01-15T18:17:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.098+0000 7f10b3e00640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 0x7f10bc0ad920 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f10a400b3f0 tx=0x7f10a400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.098+0000 7f10b9600640 1 -- 172.21.15.26:0/3271604338 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4011830 con 0x7f10bc0ccc10 2024-01-15T18:17:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.098+0000 7f10b9600640 1 -- 172.21.15.26:0/3271604338 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a4011990 con 0x7f10bc0ccc10 2024-01-15T18:17:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.098+0000 7f10b9600640 1 -- 172.21.15.26:0/3271604338 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4011d40 con 0x7f10bc0ccc10 2024-01-15T18:17:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.098+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10bc0c40f0 con 0x7f10bc0ccc10 2024-01-15T18:17:52.102 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.098+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10bc15acc0 con 0x7f10bc0ccc10 2024-01-15T18:17:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.099+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10bc067450 con 0x7f10bc0ccc10 2024-01-15T18:17:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.100+0000 7f10b9600640 1 -- 172.21.15.26:0/3271604338 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f10a401a400 con 0x7f10bc0ccc10 2024-01-15T18:17:52.103 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.101+0000 7f10b9600640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f109406c550 0x7f109406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.101+0000 7f10b9600640 1 -- 172.21.15.26:0/3271604338 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f10a4088990 con 0x7f10bc0ccc10 2024-01-15T18:17:52.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.102+0000 7f10c0e00640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f109406c550 0x7f109406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.103+0000 7f10c0e00640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f109406c550 0x7f109406ea10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f10bc1f61a0 tx=0x7f10ac008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:52.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.106+0000 7f10b9600640 1 -- 172.21.15.26:0/3271604338 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10a4054ed0 con 0x7f10bc0ccc10 2024-01-15T18:17:52.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.296+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10bc06cc40 con 0x7f109406c550 2024-01-15T18:17:52.327 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.325+0000 7f10b9600640 1 -- 172.21.15.26:0/3271604338 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288837 (secure 0 0 0) 0x7f10ac00a660 con 0x7f109406c550 2024-01-15T18:17:52.332 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:17:52.333 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:17:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f109406c550 msgr2=0x7f109406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f109406c550 0x7f109406ea10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f10bc1f61a0 tx=0x7f10ac008040 comp rx=0 tx=0).stop 2024-01-15T18:17:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 msgr2=0x7f10bc0ad920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 0x7f10bc0ad920 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f10a400b3f0 tx=0x7f10a400b8b0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 shutdown_connections 2024-01-15T18:17:52.335 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f10bc143c40 0x7f10bc0ade60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f109406c550 0x7f109406ea10 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f10bc0ccc10 0x7f10bc0ad920 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 --2- 172.21.15.26:0/3271604338 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f10bc0717d0 0x7f10bc15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 >> 172.21.15.26:0/3271604338 conn(0x7f10bc06a800 msgr2=0x7f10bc0aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 shutdown_connections 2024-01-15T18:17:52.336 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.331+0000 7f10c2200640 1 -- 172.21.15.26:0/3271604338 wait complete. 2024-01-15T18:17:52.352 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":797,"stamp":"2024-01-15T18:17:50.450846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369934897,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1739,"num_write_kb":5246057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3181,"ondisk_log_size":3181,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11062216,"kb_used_data":10499592,"kb_used_omap":12,"kb_used_meta":562547,"kb_avail":743912504,"statfs":{"total":773094113280,"available":761766404096,"internally_reserved":0,"allocated":10751582208,"data_stored":10744948594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":576048688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1073781794,"num_objects":256,"num_object_clones":0,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":256,"num_read_kb":1048576,"num_write":275,"num_write_kb":1048622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005631"},"pg_stats":[{"pgid":"17.3f","version":"117'53","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.789093+0000","last_change":"2024-01-15T18:12:53.636580+0000","last_active":"2024-01-15T18:17:38.789093+0000","last_peered":"2024-01-15T18:17:38.789093+0000","last_clean":"2024-01-15T18:17:38.789093+0000","last_became_active":"2024-01-15T18:12:53.334688+0000","last_became_peered":"2024-01-15T18:12:53.334688+0000","last_unstale":"2024-01-15T18:17:38.789093+0000","last_undegraded":"2024-01-15T18:17:38.789093+0000","last_fullsized":"2024-01-15T18:17:38.789093+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:39.949271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3e","version":"117'45","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.174108+0000","last_change":"2024-01-15T18:12:11.053712+0000","last_active":"2024-01-15T18:17:39.174108+0000","last_peered":"2024-01-15T18:17:39.174108+0000","last_clean":"2024-01-15T18:17:39.174108+0000","last_became_active":"2024-01-15T18:12:11.053279+0000","last_became_peered":"2024-01-15T18:12:11.053279+0000","last_unstale":"2024-01-15T18:17:39.174108+0000","last_undegraded":"2024-01-15T18:17:39.174108+0000","last_fullsized":"2024-01-15T18:17:39.174108+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"117'44","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.301593+0000","last_change":"2024-01-15T18:12:11.058625+0000","last_active":"2024-01-15T18:17:39.301593+0000","last_peered":"2024-01-15T18:17:39.301593+0000","last_clean":"2024-01-15T18:17:39.301593+0000","last_became_active":"2024-01-15T18:12:11.057543+0000","last_became_peered":"2024-01-15T18:12:11.057543+0000","last_unstale":"2024-01-15T18:17:39.301593+0000","last_undegraded":"2024-01-15T18:17:39.301593+0000","last_fullsized":"2024-01-15T18:17:39.301593+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:17.569977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"117'45","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.237120+0000","last_change":"2024-01-15T18:12:11.047313+0000","last_active":"2024-01-15T18:17:39.237120+0000","last_peered":"2024-01-15T18:17:39.237120+0000","last_clean":"2024-01-15T18:17:39.237120+0000","last_became_active":"2024-01-15T18:12:11.047008+0000","last_became_peered":"2024-01-15T18:12:11.047008+0000","last_unstale":"2024-01-15T18:17:39.237120+0000","last_undegraded":"2024-01-15T18:17:39.237120+0000","last_fullsized":"2024-01-15T18:17:39.237120+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:03:15.070494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"117'27","reported_seq":49,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.653418+0000","last_change":"2024-01-15T18:12:11.035567+0000","last_active":"2024-01-15T18:17:38.653418+0000","last_peered":"2024-01-15T18:17:38.653418+0000","last_clean":"2024-01-15T18:17:38.653418+0000","last_became_active":"2024-01-15T18:12:11.034909+0000","last_became_peered":"2024-01-15T18:12:11.034909+0000","last_unstale":"2024-01-15T18:17:38.653418+0000","last_undegraded":"2024-01-15T18:17:38.653418+0000","last_fullsized":"2024-01-15T18:17:38.653418+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:46.811591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"117'48","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.922505+0000","last_change":"2024-01-15T18:12:11.055133+0000","last_active":"2024-01-15T18:17:38.922505+0000","last_peered":"2024-01-15T18:17:38.922505+0000","last_clean":"2024-01-15T18:17:38.922505+0000","last_became_active":"2024-01-15T18:12:11.054803+0000","last_became_peered":"2024-01-15T18:12:11.054803+0000","last_unstale":"2024-01-15T18:17:38.922505+0000","last_undegraded":"2024-01-15T18:17:38.922505+0000","last_fullsized":"2024-01-15T18:17:38.922505+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:53.540866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"117'43","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.895097+0000","last_change":"2024-01-15T18:12:11.045438+0000","last_active":"2024-01-15T18:17:38.895097+0000","last_peered":"2024-01-15T18:17:38.895097+0000","last_clean":"2024-01-15T18:17:38.895097+0000","last_became_active":"2024-01-15T18:12:11.044607+0000","last_became_peered":"2024-01-15T18:12:11.044607+0000","last_unstale":"2024-01-15T18:17:38.895097+0000","last_undegraded":"2024-01-15T18:17:38.895097+0000","last_fullsized":"2024-01-15T18:17:38.895097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:44.315673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"117'44","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.859692+0000","last_change":"2024-01-15T18:12:11.051056+0000","last_active":"2024-01-15T18:17:38.859692+0000","last_peered":"2024-01-15T18:17:38.859692+0000","last_clean":"2024-01-15T18:17:38.859692+0000","last_became_active":"2024-01-15T18:12:11.050697+0000","last_became_peered":"2024-01-15T18:12:11.050697+0000","last_unstale":"2024-01-15T18:17:38.859692+0000","last_undegraded":"2024-01-15T18:17:38.859692+0000","last_fullsized":"2024-01-15T18:17:38.859692+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:56.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.359403+0000","last_change":"2024-01-15T18:12:11.048227+0000","last_active":"2024-01-15T18:17:39.359403+0000","last_peered":"2024-01-15T18:17:39.359403+0000","last_clean":"2024-01-15T18:17:39.359403+0000","last_became_active":"2024-01-15T18:12:11.047693+0000","last_became_peered":"2024-01-15T18:12:11.047693+0000","last_unstale":"2024-01-15T18:17:39.359403+0000","last_undegraded":"2024-01-15T18:17:39.359403+0000","last_fullsized":"2024-01-15T18:17:39.359403+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:43.904890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"117'47","reported_seq":72,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.466111+0000","last_change":"2024-01-15T18:12:11.054486+0000","last_active":"2024-01-15T18:17:39.466111+0000","last_peered":"2024-01-15T18:17:39.466111+0000","last_clean":"2024-01-15T18:17:39.466111+0000","last_became_active":"2024-01-15T18:12:11.052429+0000","last_became_peered":"2024-01-15T18:12:11.052429+0000","last_unstale":"2024-01-15T18:17:39.466111+0000","last_undegraded":"2024-01-15T18:17:39.466111+0000","last_fullsized":"2024-01-15T18:17:39.466111+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.440558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"117'47","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.167604+0000","last_change":"2024-01-15T18:12:11.043835+0000","last_active":"2024-01-15T18:17:39.167604+0000","last_peered":"2024-01-15T18:17:39.167604+0000","last_clean":"2024-01-15T18:17:39.167604+0000","last_became_active":"2024-01-15T18:12:11.042720+0000","last_became_peered":"2024-01-15T18:12:11.042720+0000","last_unstale":"2024-01-15T18:17:39.167604+0000","last_undegraded":"2024-01-15T18:17:39.167604+0000","last_fullsized":"2024-01-15T18:17:39.167604+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"117'48","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.287663+0000","last_change":"2024-01-15T18:12:11.044468+0000","last_active":"2024-01-15T18:17:39.287663+0000","last_peered":"2024-01-15T18:17:39.287663+0000","last_clean":"2024-01-15T18:17:39.287663+0000","last_became_active":"2024-01-15T18:12:11.044155+0000","last_became_peered":"2024-01-15T18:12:11.044155+0000","last_unstale":"2024-01-15T18:17:39.287663+0000","last_undegraded":"2024-01-15T18:17:39.287663+0000","last_fullsized":"2024-01-15T18:17:39.287663+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:57.310458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"117'38","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.737859+0000","last_change":"2024-01-15T18:12:11.052950+0000","last_active":"2024-01-15T18:17:38.737859+0000","last_peered":"2024-01-15T18:17:38.737859+0000","last_clean":"2024-01-15T18:17:38.737859+0000","last_became_active":"2024-01-15T18:12:11.051875+0000","last_became_peered":"2024-01-15T18:12:11.051875+0000","last_unstale":"2024-01-15T18:17:38.737859+0000","last_undegraded":"2024-01-15T18:17:38.737859+0000","last_fullsized":"2024-01-15T18:17:38.737859+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:22.849560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"117'51","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.417564+0000","last_change":"2024-01-15T18:12:11.056103+0000","last_active":"2024-01-15T18:17:39.417564+0000","last_peered":"2024-01-15T18:17:39.417564+0000","last_clean":"2024-01-15T18:17:39.417564+0000","last_became_active":"2024-01-15T18:12:11.055813+0000","last_became_peered":"2024-01-15T18:12:11.055813+0000","last_unstale":"2024-01-15T18:17:39.417564+0000","last_undegraded":"2024-01-15T18:17:39.417564+0000","last_fullsized":"2024-01-15T18:17:39.417564+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.025824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"117'34","reported_seq":53,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.330592+0000","last_change":"2024-01-15T18:12:11.056863+0000","last_active":"2024-01-15T18:17:39.330592+0000","last_peered":"2024-01-15T18:17:39.330592+0000","last_clean":"2024-01-15T18:17:39.330592+0000","last_became_active":"2024-01-15T18:12:11.055217+0000","last_became_peered":"2024-01-15T18:12:11.055217+0000","last_unstale":"2024-01-15T18:17:39.330592+0000","last_undegraded":"2024-01-15T18:17:39.330592+0000","last_fullsized":"2024-01-15T18:17:39.330592+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:33.702581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"117'42","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.869131+0000","last_change":"2024-01-15T18:12:53.618454+0000","last_active":"2024-01-15T18:17:38.869131+0000","last_peered":"2024-01-15T18:17:38.869131+0000","last_clean":"2024-01-15T18:17:38.869131+0000","last_became_active":"2024-01-15T18:12:53.338860+0000","last_became_peered":"2024-01-15T18:12:53.338860+0000","last_unstale":"2024-01-15T18:17:38.869131+0000","last_undegraded":"2024-01-15T18:17:38.869131+0000","last_fullsized":"2024-01-15T18:17:38.869131+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:40.129590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2f","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.108098+0000","last_change":"2024-01-15T18:12:11.043060+0000","last_active":"2024-01-15T18:17:39.108098+0000","last_peered":"2024-01-15T18:17:39.108098+0000","last_clean":"2024-01-15T18:17:39.108098+0000","last_became_active":"2024-01-15T18:12:11.042031+0000","last_became_peered":"2024-01-15T18:12:11.042031+0000","last_unstale":"2024-01-15T18:17:39.108098+0000","last_undegraded":"2024-01-15T18:17:39.108098+0000","last_fullsized":"2024-01-15T18:17:39.108098+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:44.791297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"117'48","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.357897+0000","last_change":"2024-01-15T18:12:11.045284+0000","last_active":"2024-01-15T18:17:39.357897+0000","last_peered":"2024-01-15T18:17:39.357897+0000","last_clean":"2024-01-15T18:17:39.357897+0000","last_became_active":"2024-01-15T18:12:11.042814+0000","last_became_peered":"2024-01-15T18:12:11.042814+0000","last_unstale":"2024-01-15T18:17:39.357897+0000","last_undegraded":"2024-01-15T18:17:39.357897+0000","last_fullsized":"2024-01-15T18:17:39.357897+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.108742+0000","last_change":"2024-01-15T18:12:11.055119+0000","last_active":"2024-01-15T18:17:39.108742+0000","last_peered":"2024-01-15T18:17:39.108742+0000","last_clean":"2024-01-15T18:17:39.108742+0000","last_became_active":"2024-01-15T18:12:11.054790+0000","last_became_peered":"2024-01-15T18:12:11.054790+0000","last_unstale":"2024-01-15T18:17:39.108742+0000","last_undegraded":"2024-01-15T18:17:39.108742+0000","last_fullsized":"2024-01-15T18:17:39.108742+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:17.513018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"117'46","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.362355+0000","last_change":"2024-01-15T18:12:11.052755+0000","last_active":"2024-01-15T18:17:39.362355+0000","last_peered":"2024-01-15T18:17:39.362355+0000","last_clean":"2024-01-15T18:17:39.362355+0000","last_became_active":"2024-01-15T18:12:11.052289+0000","last_became_peered":"2024-01-15T18:12:11.052289+0000","last_unstale":"2024-01-15T18:17:39.362355+0000","last_undegraded":"2024-01-15T18:17:39.362355+0000","last_fullsized":"2024-01-15T18:17:39.362355+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:38:56.937377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"117'56","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.083979+0000","last_change":"2024-01-15T18:12:11.036748+0000","last_active":"2024-01-15T18:17:39.083979+0000","last_peered":"2024-01-15T18:17:39.083979+0000","last_clean":"2024-01-15T18:17:39.083979+0000","last_became_active":"2024-01-15T18:12:11.036003+0000","last_became_peered":"2024-01-15T18:12:11.036003+0000","last_unstale":"2024-01-15T18:17:39.083979+0000","last_undegraded":"2024-01-15T18:17:39.083979+0000","last_fullsized":"2024-01-15T18:17:39.083979+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:20.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.456439+0000","last_change":"2024-01-15T18:12:11.050445+0000","last_active":"2024-01-15T18:17:39.456439+0000","last_peered":"2024-01-15T18:17:39.456439+0000","last_clean":"2024-01-15T18:17:39.456439+0000","last_became_active":"2024-01-15T18:12:11.044093+0000","last_became_peered":"2024-01-15T18:12:11.044093+0000","last_unstale":"2024-01-15T18:17:39.456439+0000","last_undegraded":"2024-01-15T18:17:39.456439+0000","last_fullsized":"2024-01-15T18:17:39.456439+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:38:59.097623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"117'39","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.269675+0000","last_change":"2024-01-15T18:12:11.053397+0000","last_active":"2024-01-15T18:17:39.269675+0000","last_peered":"2024-01-15T18:17:39.269675+0000","last_clean":"2024-01-15T18:17:39.269675+0000","last_became_active":"2024-01-15T18:12:11.052428+0000","last_became_peered":"2024-01-15T18:12:11.052428+0000","last_unstale":"2024-01-15T18:17:39.269675+0000","last_undegraded":"2024-01-15T18:17:39.269675+0000","last_fullsized":"2024-01-15T18:17:39.269675+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:07.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"117'43","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.364184+0000","last_change":"2024-01-15T18:12:11.051820+0000","last_active":"2024-01-15T18:17:39.364184+0000","last_peered":"2024-01-15T18:17:39.364184+0000","last_clean":"2024-01-15T18:17:39.364184+0000","last_became_active":"2024-01-15T18:12:11.051488+0000","last_became_peered":"2024-01-15T18:12:11.051488+0000","last_unstale":"2024-01-15T18:17:39.364184+0000","last_undegraded":"2024-01-15T18:17:39.364184+0000","last_fullsized":"2024-01-15T18:17:39.364184+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:45.480493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"117'38","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.840614+0000","last_change":"2024-01-15T18:12:11.041672+0000","last_active":"2024-01-15T18:17:38.840614+0000","last_peered":"2024-01-15T18:17:38.840614+0000","last_clean":"2024-01-15T18:17:38.840614+0000","last_became_active":"2024-01-15T18:12:11.041266+0000","last_became_peered":"2024-01-15T18:12:11.041266+0000","last_unstale":"2024-01-15T18:17:38.840614+0000","last_undegraded":"2024-01-15T18:17:38.840614+0000","last_fullsized":"2024-01-15T18:17:38.840614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"117'53","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.369799+0000","last_change":"2024-01-15T18:12:11.048567+0000","last_active":"2024-01-15T18:17:39.369799+0000","last_peered":"2024-01-15T18:17:39.369799+0000","last_clean":"2024-01-15T18:17:39.369799+0000","last_became_active":"2024-01-15T18:12:11.048234+0000","last_became_peered":"2024-01-15T18:12:11.048234+0000","last_unstale":"2024-01-15T18:17:39.369799+0000","last_undegraded":"2024-01-15T18:17:39.369799+0000","last_fullsized":"2024-01-15T18:17:39.369799+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:12.290504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"117'37","reported_seq":59,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.691915+0000","last_change":"2024-01-15T18:12:11.062039+0000","last_active":"2024-01-15T18:17:38.691915+0000","last_peered":"2024-01-15T18:17:38.691915+0000","last_clean":"2024-01-15T18:17:38.691915+0000","last_became_active":"2024-01-15T18:12:11.061724+0000","last_became_peered":"2024-01-15T18:12:11.061724+0000","last_unstale":"2024-01-15T18:17:38.691915+0000","last_undegraded":"2024-01-15T18:17:38.691915+0000","last_fullsized":"2024-01-15T18:17:38.691915+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:03.329007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"117'38","reported_seq":56,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.276302+0000","last_change":"2024-01-15T18:12:11.044940+0000","last_active":"2024-01-15T18:17:39.276302+0000","last_peered":"2024-01-15T18:17:39.276302+0000","last_clean":"2024-01-15T18:17:39.276302+0000","last_became_active":"2024-01-15T18:12:11.043801+0000","last_became_peered":"2024-01-15T18:12:11.043801+0000","last_unstale":"2024-01-15T18:17:39.276302+0000","last_undegraded":"2024-01-15T18:17:39.276302+0000","last_fullsized":"2024-01-15T18:17:39.276302+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:43.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"117'46","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.347960+0000","last_change":"2024-01-15T18:12:11.048119+0000","last_active":"2024-01-15T18:17:39.347960+0000","last_peered":"2024-01-15T18:17:39.347960+0000","last_clean":"2024-01-15T18:17:39.347960+0000","last_became_active":"2024-01-15T18:12:11.047739+0000","last_became_peered":"2024-01-15T18:12:11.047739+0000","last_unstale":"2024-01-15T18:17:39.347960+0000","last_undegraded":"2024-01-15T18:17:39.347960+0000","last_fullsized":"2024-01-15T18:17:39.347960+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:49:08.557895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"117'41","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.292578+0000","last_change":"2024-01-15T18:12:11.040703+0000","last_active":"2024-01-15T18:17:39.292578+0000","last_peered":"2024-01-15T18:17:39.292578+0000","last_clean":"2024-01-15T18:17:39.292578+0000","last_became_active":"2024-01-15T18:12:11.039393+0000","last_became_peered":"2024-01-15T18:12:11.039393+0000","last_unstale":"2024-01-15T18:17:39.292578+0000","last_undegraded":"2024-01-15T18:17:39.292578+0000","last_fullsized":"2024-01-15T18:17:39.292578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:40:41.329686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"117'62","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.906525+0000","last_change":"2024-01-15T18:12:11.059603+0000","last_active":"2024-01-15T18:17:38.906525+0000","last_peered":"2024-01-15T18:17:38.906525+0000","last_clean":"2024-01-15T18:17:38.906525+0000","last_became_active":"2024-01-15T18:12:11.059276+0000","last_became_peered":"2024-01-15T18:12:11.059276+0000","last_unstale":"2024-01-15T18:17:38.906525+0000","last_undegraded":"2024-01-15T18:17:38.906525+0000","last_fullsized":"2024-01-15T18:17:38.906525+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:43:52.890489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"117'54","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.208710+0000","last_change":"2024-01-15T18:12:11.053438+0000","last_active":"2024-01-15T18:17:39.208710+0000","last_peered":"2024-01-15T18:17:39.208710+0000","last_clean":"2024-01-15T18:17:39.208710+0000","last_became_active":"2024-01-15T18:12:11.052655+0000","last_became_peered":"2024-01-15T18:12:11.052655+0000","last_unstale":"2024-01-15T18:17:39.208710+0000","last_undegraded":"2024-01-15T18:17:39.208710+0000","last_fullsized":"2024-01-15T18:17:39.208710+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:45.306312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"117'34","reported_seq":55,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.008114+0000","last_change":"2024-01-15T18:12:11.053689+0000","last_active":"2024-01-15T18:17:39.008114+0000","last_peered":"2024-01-15T18:17:39.008114+0000","last_clean":"2024-01-15T18:17:39.008114+0000","last_became_active":"2024-01-15T18:12:11.053389+0000","last_became_peered":"2024-01-15T18:12:11.053389+0000","last_unstale":"2024-01-15T18:17:39.008114+0000","last_undegraded":"2024-01-15T18:17:39.008114+0000","last_fullsized":"2024-01-15T18:17:39.008114+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:39.171657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"114'4","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.409090+0000","last_change":"2024-01-15T18:12:10.021097+0000","last_active":"2024-01-15T18:14:17.409090+0000","last_peered":"2024-01-15T18:14:17.409090+0000","last_clean":"2024-01-15T18:14:17.409090+0000","last_became_active":"2024-01-15T18:12:10.020810+0000","last_became_peered":"2024-01-15T18:12:10.020810+0000","last_unstale":"2024-01-15T18:14:17.409090+0000","last_undegraded":"2024-01-15T18:14:17.409090+0000","last_fullsized":"2024-01-15T18:14:17.409090+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:09:27.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.721958+0000","last_change":"2024-01-15T18:12:11.043347+0000","last_active":"2024-01-15T18:17:38.721958+0000","last_peered":"2024-01-15T18:17:38.721958+0000","last_clean":"2024-01-15T18:17:38.721958+0000","last_became_active":"2024-01-15T18:12:11.042427+0000","last_became_peered":"2024-01-15T18:12:11.042427+0000","last_unstale":"2024-01-15T18:17:38.721958+0000","last_undegraded":"2024-01-15T18:17:38.721958+0000","last_fullsized":"2024-01-15T18:17:38.721958+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:12.407664+0000","last_change":"2024-01-15T18:12:10.023175+0000","last_active":"2024-01-15T18:14:12.407664+0000","last_peered":"2024-01-15T18:14:12.407664+0000","last_clean":"2024-01-15T18:14:12.407664+0000","last_became_active":"2024-01-15T18:12:10.022654+0000","last_became_peered":"2024-01-15T18:12:10.022654+0000","last_unstale":"2024-01-15T18:14:12.407664+0000","last_undegraded":"2024-01-15T18:14:12.407664+0000","last_fullsized":"2024-01-15T18:14:12.407664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:12.136270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'43","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.190370+0000","last_change":"2024-01-15T18:12:11.053138+0000","last_active":"2024-01-15T18:17:39.190370+0000","last_peered":"2024-01-15T18:17:39.190370+0000","last_clean":"2024-01-15T18:17:39.190370+0000","last_became_active":"2024-01-15T18:12:11.052812+0000","last_became_peered":"2024-01-15T18:12:11.052812+0000","last_unstale":"2024-01-15T18:17:39.190370+0000","last_undegraded":"2024-01-15T18:17:39.190370+0000","last_fullsized":"2024-01-15T18:17:39.190370+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:40.787531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"117'3","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.287481+0000","last_change":"2024-01-15T18:12:10.018995+0000","last_active":"2024-01-15T18:14:42.287481+0000","last_peered":"2024-01-15T18:14:42.287481+0000","last_clean":"2024-01-15T18:14:42.287481+0000","last_became_active":"2024-01-15T18:12:10.018670+0000","last_became_peered":"2024-01-15T18:12:10.018670+0000","last_unstale":"2024-01-15T18:14:42.287481+0000","last_undegraded":"2024-01-15T18:14:42.287481+0000","last_fullsized":"2024-01-15T18:14:42.287481+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:39.773731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"117'56","reported_seq":82,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.844198+0000","last_change":"2024-01-15T18:12:11.059355+0000","last_active":"2024-01-15T18:17:38.844198+0000","last_peered":"2024-01-15T18:17:38.844198+0000","last_clean":"2024-01-15T18:17:38.844198+0000","last_became_active":"2024-01-15T18:12:11.057592+0000","last_became_peered":"2024-01-15T18:12:11.057592+0000","last_unstale":"2024-01-15T18:17:38.844198+0000","last_undegraded":"2024-01-15T18:17:38.844198+0000","last_fullsized":"2024-01-15T18:17:38.844198+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:14.826977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.287454+0000","last_change":"2024-01-15T18:12:10.020565+0000","last_active":"2024-01-15T18:14:42.287454+0000","last_peered":"2024-01-15T18:14:42.287454+0000","last_clean":"2024-01-15T18:14:42.287454+0000","last_became_active":"2024-01-15T18:12:10.020143+0000","last_became_peered":"2024-01-15T18:12:10.020143+0000","last_unstale":"2024-01-15T18:14:42.287454+0000","last_undegraded":"2024-01-15T18:14:42.287454+0000","last_fullsized":"2024-01-15T18:14:42.287454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:09.129175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"117'39","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.444992+0000","last_change":"2024-01-15T18:12:11.057876+0000","last_active":"2024-01-15T18:17:39.444992+0000","last_peered":"2024-01-15T18:17:39.444992+0000","last_clean":"2024-01-15T18:17:39.444992+0000","last_became_active":"2024-01-15T18:12:11.057471+0000","last_became_peered":"2024-01-15T18:12:11.057471+0000","last_unstale":"2024-01-15T18:17:39.444992+0000","last_undegraded":"2024-01-15T18:17:39.444992+0000","last_fullsized":"2024-01-15T18:17:39.444992+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:22:19.460852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.414286+0000","last_change":"2024-01-15T18:12:10.020217+0000","last_active":"2024-01-15T18:14:42.414286+0000","last_peered":"2024-01-15T18:14:42.414286+0000","last_clean":"2024-01-15T18:14:42.414286+0000","last_became_active":"2024-01-15T18:12:10.019818+0000","last_became_peered":"2024-01-15T18:12:10.019818+0000","last_unstale":"2024-01-15T18:14:42.414286+0000","last_undegraded":"2024-01-15T18:14:42.414286+0000","last_fullsized":"2024-01-15T18:14:42.414286+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"117'49","reported_seq":77,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.264806+0000","last_change":"2024-01-15T18:12:11.050383+0000","last_active":"2024-01-15T18:17:39.264806+0000","last_peered":"2024-01-15T18:17:39.264806+0000","last_clean":"2024-01-15T18:17:39.264806+0000","last_became_active":"2024-01-15T18:12:11.050002+0000","last_became_peered":"2024-01-15T18:12:11.050002+0000","last_unstale":"2024-01-15T18:17:39.264806+0000","last_undegraded":"2024-01-15T18:17:39.264806+0000","last_fullsized":"2024-01-15T18:17:39.264806+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.309045+0000","last_change":"2024-01-15T18:12:10.013940+0000","last_active":"2024-01-15T18:14:42.309045+0000","last_peered":"2024-01-15T18:14:42.309045+0000","last_clean":"2024-01-15T18:14:42.309045+0000","last_became_active":"2024-01-15T18:12:10.009676+0000","last_became_peered":"2024-01-15T18:12:10.009676+0000","last_unstale":"2024-01-15T18:14:42.309045+0000","last_undegraded":"2024-01-15T18:14:42.309045+0000","last_fullsized":"2024-01-15T18:14:42.309045+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:00.059047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"117'53","reported_seq":78,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.258378+0000","last_change":"2024-01-15T18:12:11.057984+0000","last_active":"2024-01-15T18:17:39.258378+0000","last_peered":"2024-01-15T18:17:39.258378+0000","last_clean":"2024-01-15T18:17:39.258378+0000","last_became_active":"2024-01-15T18:12:11.057640+0000","last_became_peered":"2024-01-15T18:12:11.057640+0000","last_unstale":"2024-01-15T18:17:39.258378+0000","last_undegraded":"2024-01-15T18:17:39.258378+0000","last_fullsized":"2024-01-15T18:17:39.258378+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:57:42.482689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857791,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":36865,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.331147+0000","last_change":"2024-01-15T18:12:10.002836+0000","last_active":"2024-01-15T18:14:37.331147+0000","last_peered":"2024-01-15T18:14:37.331147+0000","last_clean":"2024-01-15T18:14:37.331147+0000","last_became_active":"2024-01-15T18:12:10.002190+0000","last_became_peered":"2024-01-15T18:12:10.002190+0000","last_unstale":"2024-01-15T18:14:37.331147+0000","last_undegraded":"2024-01-15T18:14:37.331147+0000","last_fullsized":"2024-01-15T18:14:37.331147+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:39.115790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"117'43","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.016147+0000","last_change":"2024-01-15T18:12:11.048560+0000","last_active":"2024-01-15T18:17:39.016147+0000","last_peered":"2024-01-15T18:17:39.016147+0000","last_clean":"2024-01-15T18:17:39.016147+0000","last_became_active":"2024-01-15T18:12:11.046934+0000","last_became_peered":"2024-01-15T18:12:11.046934+0000","last_unstale":"2024-01-15T18:17:39.016147+0000","last_undegraded":"2024-01-15T18:17:39.016147+0000","last_fullsized":"2024-01-15T18:17:39.016147+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:44:29.629308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.426824+0000","last_change":"2024-01-15T18:12:10.020864+0000","last_active":"2024-01-15T18:14:37.426824+0000","last_peered":"2024-01-15T18:14:37.426824+0000","last_clean":"2024-01-15T18:14:37.426824+0000","last_became_active":"2024-01-15T18:12:10.020577+0000","last_became_peered":"2024-01-15T18:12:10.020577+0000","last_unstale":"2024-01-15T18:14:37.426824+0000","last_undegraded":"2024-01-15T18:14:37.426824+0000","last_fullsized":"2024-01-15T18:14:37.426824+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:55.380272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"117'42","reported_seq":61,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.550415+0000","last_change":"2024-01-15T18:12:11.048003+0000","last_active":"2024-01-15T18:17:38.550415+0000","last_peered":"2024-01-15T18:17:38.550415+0000","last_clean":"2024-01-15T18:17:38.550415+0000","last_became_active":"2024-01-15T18:12:11.047704+0000","last_became_peered":"2024-01-15T18:12:11.047704+0000","last_unstale":"2024-01-15T18:17:38.550415+0000","last_undegraded":"2024-01-15T18:17:38.550415+0000","last_fullsized":"2024-01-15T18:17:38.550415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:02.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"117'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.286410+0000","last_change":"2024-01-15T18:12:10.020498+0000","last_active":"2024-01-15T18:14:37.286410+0000","last_peered":"2024-01-15T18:14:37.286410+0000","last_clean":"2024-01-15T18:14:37.286410+0000","last_became_active":"2024-01-15T18:12:10.020028+0000","last_became_peered":"2024-01-15T18:12:10.020028+0000","last_unstale":"2024-01-15T18:14:37.286410+0000","last_undegraded":"2024-01-15T18:14:37.286410+0000","last_fullsized":"2024-01-15T18:14:37.286410+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"117'40","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.283436+0000","last_change":"2024-01-15T18:12:11.060166+0000","last_active":"2024-01-15T18:17:39.283436+0000","last_peered":"2024-01-15T18:17:39.283436+0000","last_clean":"2024-01-15T18:17:39.283436+0000","last_became_active":"2024-01-15T18:12:11.059393+0000","last_became_peered":"2024-01-15T18:12:11.059393+0000","last_unstale":"2024-01-15T18:17:39.283436+0000","last_undegraded":"2024-01-15T18:17:39.283436+0000","last_fullsized":"2024-01-15T18:17:39.283436+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.308163+0000","last_change":"2024-01-15T18:12:10.007879+0000","last_active":"2024-01-15T18:14:37.308163+0000","last_peered":"2024-01-15T18:14:37.308163+0000","last_clean":"2024-01-15T18:14:37.308163+0000","last_became_active":"2024-01-15T18:12:10.007549+0000","last_became_peered":"2024-01-15T18:12:10.007549+0000","last_unstale":"2024-01-15T18:14:37.308163+0000","last_undegraded":"2024-01-15T18:14:37.308163+0000","last_fullsized":"2024-01-15T18:14:37.308163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:33.832426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"117'43","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.394428+0000","last_change":"2024-01-15T18:12:11.031666+0000","last_active":"2024-01-15T18:17:39.394428+0000","last_peered":"2024-01-15T18:17:39.394428+0000","last_clean":"2024-01-15T18:17:39.394428+0000","last_became_active":"2024-01-15T18:12:11.030958+0000","last_became_peered":"2024-01-15T18:12:11.030958+0000","last_unstale":"2024-01-15T18:17:39.394428+0000","last_undegraded":"2024-01-15T18:17:39.394428+0000","last_fullsized":"2024-01-15T18:17:39.394428+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:58.112590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"117'12","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:33.625051+0000","last_change":"2024-01-15T18:12:10.024809+0000","last_active":"2024-01-15T18:15:33.625051+0000","last_peered":"2024-01-15T18:15:33.625051+0000","last_clean":"2024-01-15T18:15:33.625051+0000","last_became_active":"2024-01-15T18:12:10.024120+0000","last_became_peered":"2024-01-15T18:12:10.024120+0000","last_unstale":"2024-01-15T18:15:33.625051+0000","last_undegraded":"2024-01-15T18:15:33.625051+0000","last_fullsized":"2024-01-15T18:15:33.625051+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:17:59.130803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"117'43","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.025487+0000","last_change":"2024-01-15T18:12:11.055694+0000","last_active":"2024-01-15T18:17:39.025487+0000","last_peered":"2024-01-15T18:17:39.025487+0000","last_clean":"2024-01-15T18:17:39.025487+0000","last_became_active":"2024-01-15T18:12:11.055290+0000","last_became_peered":"2024-01-15T18:12:11.055290+0000","last_unstale":"2024-01-15T18:17:39.025487+0000","last_undegraded":"2024-01-15T18:17:39.025487+0000","last_fullsized":"2024-01-15T18:17:39.025487+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:44.436928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"117'183","reported_seq":199,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:44.393465+0000","last_change":"2024-01-15T18:12:10.022950+0000","last_active":"2024-01-15T18:17:44.393465+0000","last_peered":"2024-01-15T18:17:44.393465+0000","last_clean":"2024-01-15T18:17:44.393465+0000","last_became_active":"2024-01-15T18:12:10.021497+0000","last_became_peered":"2024-01-15T18:12:10.021497+0000","last_unstale":"2024-01-15T18:17:44.393465+0000","last_undegraded":"2024-01-15T18:17:44.393465+0000","last_fullsized":"2024-01-15T18:17:44.393465+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:16.556177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":632277,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"117'44","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.050496+0000","last_change":"2024-01-15T18:12:11.046101+0000","last_active":"2024-01-15T18:17:39.050496+0000","last_peered":"2024-01-15T18:17:39.050496+0000","last_clean":"2024-01-15T18:17:39.050496+0000","last_became_active":"2024-01-15T18:12:11.045774+0000","last_became_peered":"2024-01-15T18:12:11.045774+0000","last_unstale":"2024-01-15T18:17:39.050496+0000","last_undegraded":"2024-01-15T18:17:39.050496+0000","last_fullsized":"2024-01-15T18:17:39.050496+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:41:11.078868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.285076+0000","last_change":"2024-01-15T18:12:10.016912+0000","last_active":"2024-01-15T18:14:32.285076+0000","last_peered":"2024-01-15T18:14:32.285076+0000","last_clean":"2024-01-15T18:14:32.285076+0000","last_became_active":"2024-01-15T18:12:10.016618+0000","last_became_peered":"2024-01-15T18:12:10.016618+0000","last_unstale":"2024-01-15T18:14:32.285076+0000","last_undegraded":"2024-01-15T18:14:32.285076+0000","last_fullsized":"2024-01-15T18:14:32.285076+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:40:48.957828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"117'52","reported_seq":72,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.238652+0000","last_change":"2024-01-15T18:12:11.051223+0000","last_active":"2024-01-15T18:17:39.238652+0000","last_peered":"2024-01-15T18:17:39.238652+0000","last_clean":"2024-01-15T18:17:39.238652+0000","last_became_active":"2024-01-15T18:12:11.050928+0000","last_became_peered":"2024-01-15T18:12:11.050928+0000","last_unstale":"2024-01-15T18:17:39.238652+0000","last_undegraded":"2024-01-15T18:17:39.238652+0000","last_fullsized":"2024-01-15T18:17:39.238652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:11.318479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.543703+0000","last_change":"2024-01-15T18:12:10.015868+0000","last_active":"2024-01-15T18:14:32.543703+0000","last_peered":"2024-01-15T18:14:32.543703+0000","last_clean":"2024-01-15T18:14:32.543703+0000","last_became_active":"2024-01-15T18:12:10.015578+0000","last_became_peered":"2024-01-15T18:12:10.015578+0000","last_unstale":"2024-01-15T18:14:32.543703+0000","last_undegraded":"2024-01-15T18:14:32.543703+0000","last_fullsized":"2024-01-15T18:14:32.543703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"117'46","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.210181+0000","last_change":"2024-01-15T18:12:11.029654+0000","last_active":"2024-01-15T18:17:39.210181+0000","last_peered":"2024-01-15T18:17:39.210181+0000","last_clean":"2024-01-15T18:17:39.210181+0000","last_became_active":"2024-01-15T18:12:11.028865+0000","last_became_peered":"2024-01-15T18:12:11.028865+0000","last_unstale":"2024-01-15T18:17:39.210181+0000","last_undegraded":"2024-01-15T18:17:39.210181+0000","last_fullsized":"2024-01-15T18:17:39.210181+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:29.536846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663556,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"114'8","reported_seq":24,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.285243+0000","last_change":"2024-01-15T18:12:10.022690+0000","last_active":"2024-01-15T18:14:32.285243+0000","last_peered":"2024-01-15T18:14:32.285243+0000","last_clean":"2024-01-15T18:14:32.285243+0000","last_became_active":"2024-01-15T18:12:10.022398+0000","last_became_peered":"2024-01-15T18:12:10.022398+0000","last_unstale":"2024-01-15T18:14:32.285243+0000","last_undegraded":"2024-01-15T18:14:32.285243+0000","last_fullsized":"2024-01-15T18:14:32.285243+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:52.527066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"117'24","reported_seq":43,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.192858+0000","last_change":"2024-01-15T18:12:11.045910+0000","last_active":"2024-01-15T18:17:39.192858+0000","last_peered":"2024-01-15T18:17:39.192858+0000","last_clean":"2024-01-15T18:17:39.192858+0000","last_became_active":"2024-01-15T18:12:11.043735+0000","last_became_peered":"2024-01-15T18:12:11.043735+0000","last_unstale":"2024-01-15T18:17:39.192858+0000","last_undegraded":"2024-01-15T18:17:39.192858+0000","last_fullsized":"2024-01-15T18:17:39.192858+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.728315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.424881+0000","last_change":"2024-01-15T18:12:10.027277+0000","last_active":"2024-01-15T18:14:27.424881+0000","last_peered":"2024-01-15T18:14:27.424881+0000","last_clean":"2024-01-15T18:14:27.424881+0000","last_became_active":"2024-01-15T18:12:10.026730+0000","last_became_peered":"2024-01-15T18:12:10.026730+0000","last_unstale":"2024-01-15T18:14:27.424881+0000","last_undegraded":"2024-01-15T18:14:27.424881+0000","last_fullsized":"2024-01-15T18:14:27.424881+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:52.737012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"117'36","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.289041+0000","last_change":"2024-01-15T18:12:11.061270+0000","last_active":"2024-01-15T18:17:39.289041+0000","last_peered":"2024-01-15T18:17:39.289041+0000","last_clean":"2024-01-15T18:17:39.289041+0000","last_became_active":"2024-01-15T18:12:11.061002+0000","last_became_peered":"2024-01-15T18:12:11.061002+0000","last_unstale":"2024-01-15T18:17:39.289041+0000","last_undegraded":"2024-01-15T18:17:39.289041+0000","last_fullsized":"2024-01-15T18:17:39.289041+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:10.142975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.284329+0000","last_change":"2024-01-15T18:12:10.020526+0000","last_active":"2024-01-15T18:14:27.284329+0000","last_peered":"2024-01-15T18:14:27.284329+0000","last_clean":"2024-01-15T18:14:27.284329+0000","last_became_active":"2024-01-15T18:12:10.019706+0000","last_became_peered":"2024-01-15T18:12:10.019706+0000","last_unstale":"2024-01-15T18:14:27.284329+0000","last_undegraded":"2024-01-15T18:14:27.284329+0000","last_fullsized":"2024-01-15T18:14:27.284329+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:49.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"117'44","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.211608+0000","last_change":"2024-01-15T18:12:11.060480+0000","last_active":"2024-01-15T18:17:39.211608+0000","last_peered":"2024-01-15T18:17:39.211608+0000","last_clean":"2024-01-15T18:17:39.211608+0000","last_became_active":"2024-01-15T18:12:11.060154+0000","last_became_peered":"2024-01-15T18:12:11.060154+0000","last_unstale":"2024-01-15T18:17:39.211608+0000","last_undegraded":"2024-01-15T18:17:39.211608+0000","last_fullsized":"2024-01-15T18:17:39.211608+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:52.364460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.542920+0000","last_change":"2024-01-15T18:12:10.003308+0000","last_active":"2024-01-15T18:14:27.542920+0000","last_peered":"2024-01-15T18:14:27.542920+0000","last_clean":"2024-01-15T18:14:27.542920+0000","last_became_active":"2024-01-15T18:12:10.002957+0000","last_became_peered":"2024-01-15T18:12:10.002957+0000","last_unstale":"2024-01-15T18:14:27.542920+0000","last_undegraded":"2024-01-15T18:14:27.542920+0000","last_fullsized":"2024-01-15T18:14:27.542920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:43.777497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"117'41","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.319156+0000","last_change":"2024-01-15T18:12:11.059234+0000","last_active":"2024-01-15T18:17:39.319156+0000","last_peered":"2024-01-15T18:17:39.319156+0000","last_clean":"2024-01-15T18:17:39.319156+0000","last_became_active":"2024-01-15T18:12:11.058945+0000","last_became_peered":"2024-01-15T18:12:11.058945+0000","last_unstale":"2024-01-15T18:17:39.319156+0000","last_undegraded":"2024-01-15T18:17:39.319156+0000","last_fullsized":"2024-01-15T18:17:39.319156+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:37.113206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.411059+0000","last_change":"2024-01-15T18:12:10.028589+0000","last_active":"2024-01-15T18:14:27.411059+0000","last_peered":"2024-01-15T18:14:27.411059+0000","last_clean":"2024-01-15T18:14:27.411059+0000","last_became_active":"2024-01-15T18:12:10.028284+0000","last_became_peered":"2024-01-15T18:12:10.028284+0000","last_unstale":"2024-01-15T18:14:27.411059+0000","last_undegraded":"2024-01-15T18:14:27.411059+0000","last_fullsized":"2024-01-15T18:14:27.411059+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:46:23.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"117'45","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.543742+0000","last_change":"2024-01-15T18:12:11.051787+0000","last_active":"2024-01-15T18:17:39.543742+0000","last_peered":"2024-01-15T18:17:39.543742+0000","last_clean":"2024-01-15T18:17:39.543742+0000","last_became_active":"2024-01-15T18:12:11.051403+0000","last_became_peered":"2024-01-15T18:12:11.051403+0000","last_unstale":"2024-01-15T18:17:39.543742+0000","last_undegraded":"2024-01-15T18:17:39.543742+0000","last_fullsized":"2024-01-15T18:17:39.543742+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:05:35.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"117'3","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.284165+0000","last_change":"2024-01-15T18:12:10.021913+0000","last_active":"2024-01-15T18:14:27.284165+0000","last_peered":"2024-01-15T18:14:27.284165+0000","last_clean":"2024-01-15T18:14:27.284165+0000","last_became_active":"2024-01-15T18:12:10.021344+0000","last_became_peered":"2024-01-15T18:12:10.021344+0000","last_unstale":"2024-01-15T18:14:27.284165+0000","last_undegraded":"2024-01-15T18:14:27.284165+0000","last_fullsized":"2024-01-15T18:14:27.284165+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:00.107213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"117'43","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.906847+0000","last_change":"2024-01-15T18:12:11.059848+0000","last_active":"2024-01-15T18:17:38.906847+0000","last_peered":"2024-01-15T18:17:38.906847+0000","last_clean":"2024-01-15T18:17:38.906847+0000","last_became_active":"2024-01-15T18:12:11.059311+0000","last_became_peered":"2024-01-15T18:12:11.059311+0000","last_unstale":"2024-01-15T18:17:38.906847+0000","last_undegraded":"2024-01-15T18:17:38.906847+0000","last_fullsized":"2024-01-15T18:17:38.906847+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:13.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.283095+0000","last_change":"2024-01-15T18:12:10.018301+0000","last_active":"2024-01-15T18:14:22.283095+0000","last_peered":"2024-01-15T18:14:22.283095+0000","last_clean":"2024-01-15T18:14:22.283095+0000","last_became_active":"2024-01-15T18:12:10.017411+0000","last_became_peered":"2024-01-15T18:12:10.017411+0000","last_unstale":"2024-01-15T18:14:22.283095+0000","last_undegraded":"2024-01-15T18:14:22.283095+0000","last_fullsized":"2024-01-15T18:14:22.283095+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:32.688781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.231615+0000","last_change":"2024-01-15T18:12:11.056765+0000","last_active":"2024-01-15T18:17:39.231615+0000","last_peered":"2024-01-15T18:17:39.231615+0000","last_clean":"2024-01-15T18:17:39.231615+0000","last_became_active":"2024-01-15T18:12:11.056311+0000","last_became_peered":"2024-01-15T18:12:11.056311+0000","last_unstale":"2024-01-15T18:17:39.231615+0000","last_undegraded":"2024-01-15T18:17:39.231615+0000","last_fullsized":"2024-01-15T18:17:39.231615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:53.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.282960+0000","last_change":"2024-01-15T18:12:10.023638+0000","last_active":"2024-01-15T18:14:22.282960+0000","last_peered":"2024-01-15T18:14:22.282960+0000","last_clean":"2024-01-15T18:14:22.282960+0000","last_became_active":"2024-01-15T18:12:10.022687+0000","last_became_peered":"2024-01-15T18:12:10.022687+0000","last_unstale":"2024-01-15T18:14:22.282960+0000","last_undegraded":"2024-01-15T18:14:22.282960+0000","last_fullsized":"2024-01-15T18:14:22.282960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:54.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"117'42","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.425793+0000","last_change":"2024-01-15T18:12:11.062171+0000","last_active":"2024-01-15T18:17:39.425793+0000","last_peered":"2024-01-15T18:17:39.425793+0000","last_clean":"2024-01-15T18:17:39.425793+0000","last_became_active":"2024-01-15T18:12:11.061830+0000","last_became_peered":"2024-01-15T18:12:11.061830+0000","last_unstale":"2024-01-15T18:17:39.425793+0000","last_undegraded":"2024-01-15T18:17:39.425793+0000","last_fullsized":"2024-01-15T18:17:39.425793+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:50.541819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109124,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.423571+0000","last_change":"2024-01-15T18:12:10.024893+0000","last_active":"2024-01-15T18:14:22.423571+0000","last_peered":"2024-01-15T18:14:22.423571+0000","last_clean":"2024-01-15T18:14:22.423571+0000","last_became_active":"2024-01-15T18:12:10.024554+0000","last_became_peered":"2024-01-15T18:12:10.024554+0000","last_unstale":"2024-01-15T18:14:22.423571+0000","last_undegraded":"2024-01-15T18:14:22.423571+0000","last_fullsized":"2024-01-15T18:14:22.423571+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:51.350660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"117'36","reported_seq":55,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.210156+0000","last_change":"2024-01-15T18:12:11.051179+0000","last_active":"2024-01-15T18:17:39.210156+0000","last_peered":"2024-01-15T18:17:39.210156+0000","last_clean":"2024-01-15T18:17:39.210156+0000","last_became_active":"2024-01-15T18:12:11.050747+0000","last_became_peered":"2024-01-15T18:12:11.050747+0000","last_unstale":"2024-01-15T18:17:39.210156+0000","last_undegraded":"2024-01-15T18:17:39.210156+0000","last_fullsized":"2024-01-15T18:17:39.210156+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.305559+0000","last_change":"2024-01-15T18:12:10.025674+0000","last_active":"2024-01-15T18:14:22.305559+0000","last_peered":"2024-01-15T18:14:22.305559+0000","last_clean":"2024-01-15T18:14:22.305559+0000","last_became_active":"2024-01-15T18:12:10.025356+0000","last_became_peered":"2024-01-15T18:12:10.025356+0000","last_unstale":"2024-01-15T18:14:22.305559+0000","last_undegraded":"2024-01-15T18:14:22.305559+0000","last_fullsized":"2024-01-15T18:14:22.305559+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:06:50.285646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"117'41","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.010642+0000","last_change":"2024-01-15T18:12:11.050465+0000","last_active":"2024-01-15T18:17:39.010642+0000","last_peered":"2024-01-15T18:17:39.010642+0000","last_clean":"2024-01-15T18:17:39.010642+0000","last_became_active":"2024-01-15T18:12:11.050116+0000","last_became_peered":"2024-01-15T18:12:11.050116+0000","last_unstale":"2024-01-15T18:17:39.010642+0000","last_undegraded":"2024-01-15T18:17:39.010642+0000","last_fullsized":"2024-01-15T18:17:39.010642+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:45:04.891353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"117'17","reported_seq":33,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:55.208163+0000","last_change":"2024-01-15T18:12:10.020848+0000","last_active":"2024-01-15T18:16:55.208163+0000","last_peered":"2024-01-15T18:16:55.208163+0000","last_clean":"2024-01-15T18:16:55.208163+0000","last_became_active":"2024-01-15T18:12:10.020497+0000","last_became_peered":"2024-01-15T18:12:10.020497+0000","last_unstale":"2024-01-15T18:16:55.208163+0000","last_undegraded":"2024-01-15T18:16:55.208163+0000","last_fullsized":"2024-01-15T18:16:55.208163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:37.798936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"117'46","reported_seq":75,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.193190+0000","last_change":"2024-01-15T18:12:11.051721+0000","last_active":"2024-01-15T18:17:39.193190+0000","last_peered":"2024-01-15T18:17:39.193190+0000","last_clean":"2024-01-15T18:17:39.193190+0000","last_became_active":"2024-01-15T18:12:11.050810+0000","last_became_peered":"2024-01-15T18:12:11.050810+0000","last_unstale":"2024-01-15T18:17:39.193190+0000","last_undegraded":"2024-01-15T18:17:39.193190+0000","last_fullsized":"2024-01-15T18:17:39.193190+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:34:44.264660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.422714+0000","last_change":"2024-01-15T18:12:10.025006+0000","last_active":"2024-01-15T18:14:17.422714+0000","last_peered":"2024-01-15T18:14:17.422714+0000","last_clean":"2024-01-15T18:14:17.422714+0000","last_became_active":"2024-01-15T18:12:10.024524+0000","last_became_peered":"2024-01-15T18:12:10.024524+0000","last_unstale":"2024-01-15T18:14:17.422714+0000","last_undegraded":"2024-01-15T18:14:17.422714+0000","last_fullsized":"2024-01-15T18:14:17.422714+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:16.499218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"117'39","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.572497+0000","last_change":"2024-01-15T18:12:11.061751+0000","last_active":"2024-01-15T18:17:38.572497+0000","last_peered":"2024-01-15T18:17:38.572497+0000","last_clean":"2024-01-15T18:17:38.572497+0000","last_became_active":"2024-01-15T18:12:11.061454+0000","last_became_peered":"2024-01-15T18:12:11.061454+0000","last_unstale":"2024-01-15T18:17:38.572497+0000","last_undegraded":"2024-01-15T18:17:38.572497+0000","last_fullsized":"2024-01-15T18:17:38.572497+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.304508+0000","last_change":"2024-01-15T18:12:10.018428+0000","last_active":"2024-01-15T18:14:17.304508+0000","last_peered":"2024-01-15T18:14:17.304508+0000","last_clean":"2024-01-15T18:14:17.304508+0000","last_became_active":"2024-01-15T18:12:10.018057+0000","last_became_peered":"2024-01-15T18:12:10.018057+0000","last_unstale":"2024-01-15T18:14:17.304508+0000","last_undegraded":"2024-01-15T18:14:17.304508+0000","last_fullsized":"2024-01-15T18:14:17.304508+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:52:09.544387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"117'44","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.352556+0000","last_change":"2024-01-15T18:12:11.039995+0000","last_active":"2024-01-15T18:17:39.352556+0000","last_peered":"2024-01-15T18:17:39.352556+0000","last_clean":"2024-01-15T18:17:39.352556+0000","last_became_active":"2024-01-15T18:12:11.039586+0000","last_became_peered":"2024-01-15T18:12:11.039586+0000","last_unstale":"2024-01-15T18:17:39.352556+0000","last_undegraded":"2024-01-15T18:17:39.352556+0000","last_fullsized":"2024-01-15T18:17:39.352556+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:38.812736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.408970+0000","last_change":"2024-01-15T18:12:10.019758+0000","last_active":"2024-01-15T18:14:17.408970+0000","last_peered":"2024-01-15T18:14:17.408970+0000","last_clean":"2024-01-15T18:14:17.408970+0000","last_became_active":"2024-01-15T18:12:10.019424+0000","last_became_peered":"2024-01-15T18:12:10.019424+0000","last_unstale":"2024-01-15T18:14:17.408970+0000","last_undegraded":"2024-01-15T18:14:17.408970+0000","last_fullsized":"2024-01-15T18:14:17.408970+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:45.797791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"117'56","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.420858+0000","last_change":"2024-01-15T18:12:11.035857+0000","last_active":"2024-01-15T18:17:39.420858+0000","last_peered":"2024-01-15T18:17:39.420858+0000","last_clean":"2024-01-15T18:17:39.420858+0000","last_became_active":"2024-01-15T18:12:11.035238+0000","last_became_peered":"2024-01-15T18:12:11.035238+0000","last_unstale":"2024-01-15T18:17:39.420858+0000","last_undegraded":"2024-01-15T18:17:39.420858+0000","last_fullsized":"2024-01-15T18:17:39.420858+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.326548+0000","last_change":"2024-01-15T18:12:10.017511+0000","last_active":"2024-01-15T18:14:17.326548+0000","last_peered":"2024-01-15T18:14:17.326548+0000","last_clean":"2024-01-15T18:14:17.326548+0000","last_became_active":"2024-01-15T18:12:10.017190+0000","last_became_peered":"2024-01-15T18:12:10.017190+0000","last_unstale":"2024-01-15T18:14:17.326548+0000","last_undegraded":"2024-01-15T18:14:17.326548+0000","last_fullsized":"2024-01-15T18:14:17.326548+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"117'46","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.384006+0000","last_change":"2024-01-15T18:12:11.056667+0000","last_active":"2024-01-15T18:17:39.384006+0000","last_peered":"2024-01-15T18:17:39.384006+0000","last_clean":"2024-01-15T18:17:39.384006+0000","last_became_active":"2024-01-15T18:12:11.056111+0000","last_became_peered":"2024-01-15T18:12:11.056111+0000","last_unstale":"2024-01-15T18:17:39.384006+0000","last_undegraded":"2024-01-15T18:17:39.384006+0000","last_fullsized":"2024-01-15T18:17:39.384006+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:18:00.144603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"117'10","reported_seq":35,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:33.616962+0000","last_change":"2024-01-15T18:12:10.021210+0000","last_active":"2024-01-15T18:15:33.616962+0000","last_peered":"2024-01-15T18:15:33.616962+0000","last_clean":"2024-01-15T18:15:33.616962+0000","last_became_active":"2024-01-15T18:12:10.020890+0000","last_became_peered":"2024-01-15T18:12:10.020890+0000","last_unstale":"2024-01-15T18:15:33.616962+0000","last_undegraded":"2024-01-15T18:15:33.616962+0000","last_fullsized":"2024-01-15T18:15:33.616962+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:44.292512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"117'39","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.353330+0000","last_change":"2024-01-15T18:12:11.059132+0000","last_active":"2024-01-15T18:17:39.353330+0000","last_peered":"2024-01-15T18:17:39.353330+0000","last_clean":"2024-01-15T18:17:39.353330+0000","last_became_active":"2024-01-15T18:12:11.057943+0000","last_became_peered":"2024-01-15T18:12:11.057943+0000","last_unstale":"2024-01-15T18:17:39.353330+0000","last_undegraded":"2024-01-15T18:17:39.353330+0000","last_fullsized":"2024-01-15T18:17:39.353330+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:52.248678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:12.421404+0000","last_change":"2024-01-15T18:12:10.010681+0000","last_active":"2024-01-15T18:14:12.421404+0000","last_peered":"2024-01-15T18:14:12.421404+0000","last_clean":"2024-01-15T18:14:12.421404+0000","last_became_active":"2024-01-15T18:12:10.009870+0000","last_became_peered":"2024-01-15T18:12:10.009870+0000","last_unstale":"2024-01-15T18:14:12.421404+0000","last_undegraded":"2024-01-15T18:14:12.421404+0000","last_fullsized":"2024-01-15T18:14:12.421404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:21.835760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"117'31","reported_seq":54,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.239562+0000","last_change":"2024-01-15T18:12:11.051496+0000","last_active":"2024-01-15T18:17:39.239562+0000","last_peered":"2024-01-15T18:17:39.239562+0000","last_clean":"2024-01-15T18:17:39.239562+0000","last_became_active":"2024-01-15T18:12:11.051092+0000","last_became_peered":"2024-01-15T18:12:11.051092+0000","last_unstale":"2024-01-15T18:17:39.239562+0000","last_undegraded":"2024-01-15T18:17:39.239562+0000","last_fullsized":"2024-01-15T18:17:39.239562+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:38:09.219058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.412208+0000","last_change":"2024-01-15T18:12:10.026923+0000","last_active":"2024-01-15T18:14:32.412208+0000","last_peered":"2024-01-15T18:14:32.412208+0000","last_clean":"2024-01-15T18:14:32.412208+0000","last_became_active":"2024-01-15T18:12:10.026651+0000","last_became_peered":"2024-01-15T18:12:10.026651+0000","last_unstale":"2024-01-15T18:14:32.412208+0000","last_undegraded":"2024-01-15T18:14:32.412208+0000","last_fullsized":"2024-01-15T18:14:32.412208+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:14.981958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":251,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:02.417603+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:15:02.417603+0000","last_peered":"2024-01-15T18:15:02.417603+0000","last_clean":"2024-01-15T18:15:02.417603+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:15:02.417603+0000","last_undegraded":"2024-01-15T18:15:02.417603+0000","last_fullsized":"2024-01-15T18:15:02.417603+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710133,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2810,"ondisk_log_size":2810,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":634396,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":274,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1335296,"data_stored":1268792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":254,"ondisk_log_size":254,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542421,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834588,"kb_used_data":1767380,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537252,"statfs":{"total":96636764160,"available":94758146048,"internally_reserved":0,"allocated":1809797120,"data_stored":1808970858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.254,"15min":1.383},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.40400000000000003},"max":{"1min":1.599,"5min":4.0229999999999997,"15min":19.843},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.49,"15min":1.5609999999999999},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.53400000000000003},"max":{"1min":1.6319999999999999,"5min":16.678999999999998,"15min":30.498000000000001},"last":1.552}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2649999999999999,"15min":1.397},"min":{"1min":0.91400000000000003,"5min":0.56599999999999995,"15min":0.45600000000000002},"max":{"1min":1.5700000000000001,"5min":3.9430000000000001,"15min":19.917000000000002},"last":1.25},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.54,"15min":1.5669999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.71199999999999997},"max":{"1min":1.6319999999999999,"5min":16.626999999999999,"15min":28.613},"last":1.196}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2689999999999999,"15min":1.391},"min":{"1min":0.98399999999999999,"5min":0.75600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6479999999999999,"5min":1.8140000000000001,"15min":12.061},"last":1.752},{"interface":"front","average":{"1min":1.355,"5min":1.528,"15min":1.5800000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.48999999999999999},"max":{"1min":1.77,"5min":16.792000000000002,"15min":16.792000000000002},"last":1.4139999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.306,"15min":1.5229999999999999},"min":{"1min":1.0029999999999999,"5min":0.80700000000000005,"15min":0.56000000000000005},"max":{"1min":1.6180000000000001,"5min":1.734,"15min":16.632000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.619,"15min":1.639},"min":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.56699999999999995},"max":{"1min":1.6930000000000001,"5min":16.129000000000001,"15min":16.129000000000001},"last":1.6100000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.571,"15min":1.627},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.54200000000000004},"max":{"1min":1.585,"5min":16.454000000000001,"15min":23.042000000000002},"last":1.657},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.357,"15min":1.677},"min":{"1min":0.873,"5min":0.80000000000000004,"15min":0.496},"max":{"1min":1.7410000000000001,"5min":1.885,"15min":38.682000000000002},"last":1.8460000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4259999999999999,"15min":1.5860000000000001},"min":{"1min":0.96799999999999997,"5min":0.76800000000000002,"15min":0.52700000000000002},"max":{"1min":1.867,"5min":4.5359999999999996,"15min":13.379},"last":1.679},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.71,"15min":1.7430000000000001},"min":{"1min":1.0649999999999999,"5min":0.98499999999999999,"15min":0.624},"max":{"1min":1.843,"5min":16.553000000000001,"15min":22.933},"last":1.706}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.7969999999999999,"15min":1.8600000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":0.57399999999999995},"max":{"1min":1.8140000000000001,"5min":16.725999999999999,"15min":23.004000000000001},"last":1.804},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4850000000000001,"15min":1.659},"min":{"1min":1.1220000000000001,"5min":1.008,"15min":0.58299999999999996},"max":{"1min":1.897,"5min":1.952,"15min":9.2599999999999998},"last":1.9550000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542420,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1256408,"kb_used_data":1193360,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93115432,"statfs":{"total":96636764160,"available":95350202368,"internally_reserved":0,"allocated":1222000640,"data_stored":1221177506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2629999999999999,"15min":1.464},"min":{"1min":0.79600000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":31.805},"last":1.361},{"interface":"front","average":{"1min":1.244,"5min":1.3,"15min":1.4870000000000001},"min":{"1min":0.749,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8340000000000001,"5min":3.5409999999999999,"15min":31.702999999999999},"last":1.411}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1339999999999999,"15min":1.2609999999999999},"min":{"1min":0.69699999999999995,"5min":0.59999999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.778,"15min":15.064},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2050000000000001,"15min":1.4299999999999999},"min":{"1min":0.86499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6279999999999999,"5min":1.9650000000000001,"15min":23.495999999999999},"last":0.92400000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2849999999999999,"15min":1.5349999999999999},"min":{"1min":0.81699999999999995,"5min":0.79100000000000004,"15min":0.59399999999999997},"max":{"1min":1.502,"5min":1.8899999999999999,"15min":23.087},"last":1.081},{"interface":"front","average":{"1min":1.339,"5min":1.331,"15min":1.397},"min":{"1min":1.0089999999999999,"5min":0.77000000000000002,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.9990000000000001,"15min":14.94},"last":1.5309999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4359999999999999,"15min":1.6040000000000001},"min":{"1min":1.0620000000000001,"5min":0.65200000000000002,"15min":0.59899999999999998},"max":{"1min":1.7,"5min":1.8160000000000001,"15min":15.151999999999999},"last":1.698},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3779999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6080000000000001,"5min":3.8500000000000001,"15min":13.795},"last":1.573}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2729999999999999,"15min":1.585},"min":{"1min":0.90900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.53,"5min":3.3719999999999999,"15min":23.327000000000002},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.3560000000000001,"15min":1.548},"min":{"1min":0.80900000000000005,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.6839999999999999,"5min":1.8839999999999999,"15min":15.239000000000001},"last":1.1950000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.331,"15min":1.502},"min":{"1min":0.88700000000000001,"5min":0.58099999999999996,"15min":0.52500000000000002},"max":{"1min":1.7210000000000001,"5min":3.6480000000000001,"15min":13.593},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.55,"15min":1.8049999999999999},"min":{"1min":1.1930000000000001,"5min":0.66200000000000003,"15min":0.621},"max":{"1min":1.702,"5min":2.306,"15min":23.417999999999999},"last":1.595}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.47,"15min":1.716},"min":{"1min":0.98799999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8220000000000001,"5min":3.105,"15min":21.265000000000001},"last":1.252},{"interface":"front","average":{"1min":1.486,"5min":1.542,"15min":1.899},"min":{"1min":1.1200000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.7789999999999999,"5min":2.2229999999999999,"15min":23.738},"last":1.4790000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542420,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908440,"kb_used_data":832912,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93463400,"statfs":{"total":96636764160,"available":95706521600,"internally_reserved":0,"allocated":852901888,"data_stored":852079166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.202,"15min":1.2669999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.47799999999999998},"max":{"1min":1.595,"5min":2.7919999999999998,"15min":19.266999999999999},"last":1.054},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.752,"5min":0.748,"15min":0.51400000000000001},"max":{"1min":1.671,"5min":1.7529999999999999,"15min":3.1760000000000002},"last":0.99099999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.359,"15min":1.3149999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.60699999999999998},"max":{"1min":1.653,"5min":15.554,"15min":15.554},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.302,"15min":1.3540000000000001},"min":{"1min":0.80100000000000005,"5min":0.69599999999999995,"15min":0.59099999999999997},"max":{"1min":1.643,"5min":6.1449999999999996,"15min":9.2379999999999995},"last":1.1519999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.621},"max":{"1min":1.694,"5min":2.3500000000000001,"15min":4.0369999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.512,"15min":1.4299999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.56000000000000005},"max":{"1min":1.617,"5min":15.468999999999999,"15min":15.468999999999999},"last":1.4199999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.327,"15min":1.409},"min":{"1min":0.78200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.718,"5min":1.984,"15min":9.1929999999999996},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.357,"5min":1.4379999999999999,"15min":1.4650000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.64600000000000002},"max":{"1min":1.774,"5min":2.2949999999999999,"15min":7.0110000000000001},"last":1.0920000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.4319999999999999,"15min":1.47},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.41399999999999998},"max":{"1min":1.6040000000000001,"5min":15.254,"15min":15.254},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.4019999999999999,"15min":1.4710000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.8080000000000001,"5min":1.923,"15min":9.1519999999999992},"last":1.228}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.456},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.39600000000000002},"max":{"1min":1.8480000000000001,"5min":2.2250000000000001,"15min":3.7559999999999998},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.593,"15min":1.5680000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53000000000000003},"max":{"1min":1.7330000000000001,"5min":15.366,"15min":15.366},"last":1.625}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6100000000000001},"min":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.45800000000000002},"max":{"1min":1.873,"5min":2.0609999999999999,"15min":9.0600000000000005},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.522,"5min":1.5840000000000001,"15min":1.6890000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.57299999999999995},"max":{"1min":1.899,"5min":3.1659999999999999,"15min":19.356999999999999},"last":1.504}]}]},{"osd":4,"up_from":14,"seq":60129542420,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1711124,"kb_used_data":1643916,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92660716,"statfs":{"total":96636764160,"available":94884573184,"internally_reserved":0,"allocated":1683369984,"data_stored":1682551014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.264,"15min":1.3360000000000001},"min":{"1min":0.94799999999999995,"5min":0.70199999999999996,"15min":0.39200000000000002},"max":{"1min":1.601,"5min":2.004,"15min":17.346},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.3500000000000001,"15min":1.361},"min":{"1min":1.0900000000000001,"5min":0.97499999999999998,"15min":0.40100000000000002},"max":{"1min":1.744,"5min":1.804,"15min":7.9160000000000004},"last":1.6120000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.1950000000000001,"15min":1.24},"min":{"1min":0.95199999999999996,"5min":0.76600000000000001,"15min":0.439},"max":{"1min":1.6699999999999999,"5min":1.861,"15min":11.128},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.23,"15min":1.234},"min":{"1min":0.876,"5min":0.82999999999999996,"15min":0.60799999999999998},"max":{"1min":1.619,"5min":1.7470000000000001,"15min":5.6109999999999998},"last":1.4059999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.294,"15min":1.343},"min":{"1min":0.92300000000000004,"5min":0.83399999999999996,"15min":0.56899999999999995},"max":{"1min":1.544,"5min":1.8560000000000001,"15min":18.442},"last":1.266},{"interface":"front","average":{"1min":1.341,"5min":1.3720000000000001,"15min":1.399},"min":{"1min":1.1459999999999999,"5min":0.94799999999999995,"15min":0.81899999999999995},"max":{"1min":1.669,"5min":1.861,"15min":5.7050000000000001},"last":1.5649999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.375,"15min":1.462},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.56699999999999995},"max":{"1min":1.6719999999999999,"5min":2.1859999999999999,"15min":17.364999999999998},"last":1.351},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3380000000000001,"15min":1.3899999999999999},"min":{"1min":1.022,"5min":0.85099999999999998,"15min":0.45300000000000001},"max":{"1min":1.601,"5min":1.7050000000000001,"15min":9.3320000000000007},"last":1.298}]},{"osd":5,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.29,"15min":1.4590000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.49099999999999999},"max":{"1min":1.6519999999999999,"5min":2.4089999999999998,"15min":29.422000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3420000000000001,"15min":1.4379999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.53700000000000003},"max":{"1min":1.7549999999999999,"5min":1.9450000000000001,"15min":12.332000000000001},"last":1.4990000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2629999999999999,"15min":1.3899999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.54200000000000004},"max":{"1min":1.752,"5min":2.117,"15min":11.09},"last":1.212},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.482,"15min":1.6120000000000001},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":0.51100000000000001},"max":{"1min":1.718,"5min":2.2639999999999998,"15min":17.224},"last":1.004}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.504,"15min":1.593},"min":{"1min":1.0169999999999999,"5min":0.93300000000000005,"15min":0.60499999999999998},"max":{"1min":1.8400000000000001,"5min":2.052,"15min":12.247},"last":1.101},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6439999999999999,"15min":1.73},"min":{"1min":1.294,"5min":1.159,"15min":0.61399999999999999},"max":{"1min":1.9039999999999999,"5min":2.3570000000000002,"15min":17.324999999999999},"last":1.538}]}]},{"osd":3,"up_from":14,"seq":60129542422,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462032,"kb_used_data":1386504,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":92909808,"statfs":{"total":96636764160,"available":95139643392,"internally_reserved":0,"allocated":1419780096,"data_stored":1418942469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0349999999999999,"15min":1.1639999999999999},"min":{"1min":0.66400000000000003,"5min":0.52800000000000002,"15min":0.33300000000000002},"max":{"1min":1.7989999999999999,"5min":2.915,"15min":18.399999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2509999999999999,"15min":1.2849999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.44400000000000001},"max":{"1min":1.8560000000000001,"5min":3.028,"15min":18.795999999999999},"last":1.3140000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.05,"15min":1.1299999999999999},"min":{"1min":0.65800000000000003,"5min":0.51200000000000001,"15min":0.32600000000000001},"max":{"1min":1.4870000000000001,"5min":5.3520000000000003,"15min":12.372},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.1180000000000001,"15min":1.2589999999999999},"min":{"1min":0.66000000000000003,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.736,"5min":4.125,"15min":11.228999999999999},"last":1.2589999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.218,"15min":1.2529999999999999},"min":{"1min":0.74199999999999999,"5min":0.59999999999999998,"15min":0.375},"max":{"1min":1.8680000000000001,"5min":2.3929999999999998,"15min":10.964},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.45,"15min":1.534},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7569999999999999,"5min":3.0659999999999998,"15min":20.495000000000001},"last":1.361}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.389,"15min":1.51},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.67200000000000004},"max":{"1min":1.865,"5min":2.1139999999999999,"15min":13.536},"last":1.708},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4530000000000001,"15min":1.5529999999999999},"min":{"1min":0.98999999999999999,"5min":0.83099999999999996,"15min":0.63700000000000001},"max":{"1min":1.905,"5min":6.2439999999999998,"15min":16.398},"last":1.5720000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.478,"15min":1.6519999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.67200000000000004},"max":{"1min":2.0670000000000002,"5min":2.7269999999999999,"15min":20.606000000000002},"last":1.675},{"interface":"front","average":{"1min":1.587,"5min":1.6060000000000001,"15min":1.631},"min":{"1min":1.0269999999999999,"5min":0.94299999999999995,"15min":0.66800000000000004},"max":{"1min":2.0489999999999999,"5min":2.617,"15min":11.603},"last":1.625}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.554,"15min":1.675},"min":{"1min":0.93100000000000005,"5min":0.85499999999999998,"15min":0.53500000000000003},"max":{"1min":2.1230000000000002,"5min":6.3289999999999997,"15min":13.648999999999999},"last":1.917},{"interface":"front","average":{"1min":1.554,"5min":1.6100000000000001,"15min":1.7529999999999999},"min":{"1min":1.2190000000000001,"5min":1.0249999999999999,"15min":0.65800000000000003},"max":{"1min":1.849,"5min":6.367,"15min":20.556000000000001},"last":1.6080000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.641,"15min":1.76},"min":{"1min":1.159,"5min":0.97899999999999998,"15min":0.60499999999999998},"max":{"1min":2.1269999999999998,"5min":4.8630000000000004,"15min":28.843},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6559999999999999,"15min":1.738},"min":{"1min":1.1830000000000001,"5min":0.80100000000000005,"15min":0.67200000000000004},"max":{"1min":2.0680000000000001,"5min":6.9569999999999999,"15min":15.143000000000001},"last":1.7509999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542422,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":982100,"kb_used_data":910732,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93389740,"statfs":{"total":96636764160,"available":95631093760,"internally_reserved":0,"allocated":932589568,"data_stored":931771659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.161,"15min":1.083},"min":{"1min":0.67600000000000005,"5min":0.51000000000000001,"15min":0.26600000000000001},"max":{"1min":1.419,"5min":22.707000000000001,"15min":22.707000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1970000000000001,"15min":1.357},"min":{"1min":0.76300000000000001,"5min":0.61099999999999999,"15min":0.39300000000000002},"max":{"1min":1.6879999999999999,"5min":2.5710000000000002,"15min":15.314},"last":1.2010000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1759999999999999,"15min":1.3240000000000001},"min":{"1min":0.73299999999999998,"5min":0.58599999999999997,"15min":0.38300000000000001},"max":{"1min":1.546,"5min":2.8679999999999999,"15min":34.463999999999999},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.327,"15min":1.504},"min":{"1min":0.90700000000000003,"5min":0.73899999999999999,"15min":0.373},"max":{"1min":1.675,"5min":2.794,"15min":25.859999999999999},"last":1.3200000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1870000000000001,"15min":1.302},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.34399999999999997},"max":{"1min":1.464,"5min":2.7080000000000002,"15min":12.93},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.6919999999999999,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.46400000000000002},"max":{"1min":1.903,"5min":37.021999999999998,"15min":37.021999999999998},"last":1.601}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6879999999999999},"min":{"1min":1.0289999999999999,"5min":0.96999999999999997,"15min":0.55700000000000005},"max":{"1min":1.78,"5min":3.419,"15min":29.986999999999998},"last":1.496},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.518,"15min":1.7150000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.56799999999999995},"max":{"1min":1.883,"5min":2.976,"15min":21.199999999999999},"last":1.4590000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.456,"15min":1.6499999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.51500000000000001},"max":{"1min":2.0880000000000001,"5min":2.218,"15min":14.066000000000001},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.643,"15min":1.7849999999999999},"min":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":0.60899999999999999},"max":{"1min":1.8180000000000001,"5min":3.4790000000000001,"15min":14.025},"last":1.76}]},{"osd":6,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.6160000000000001,"15min":1.8069999999999999},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":0.68000000000000005},"max":{"1min":1.9910000000000001,"5min":3.0800000000000001,"15min":15.683999999999999},"last":1.659},{"interface":"front","average":{"1min":1.409,"5min":1.585,"15min":1.784},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.58899999999999997},"max":{"1min":2.2930000000000001,"5min":10.371,"15min":12.629},"last":1.546}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.671,"15min":2.1419999999999999},"min":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":0.63100000000000001},"max":{"1min":2.2210000000000001,"5min":3.177,"15min":57.006999999999998},"last":1.8580000000000001},{"interface":"front","average":{"1min":1.595,"5min":1.6299999999999999,"15min":1.9630000000000001},"min":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":0.56100000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":22.853999999999999},"last":1.8149999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542421,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1137764,"kb_used_data":1066396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93234076,"statfs":{"total":96636764160,"available":95471693824,"internally_reserved":0,"allocated":1091989504,"data_stored":1091155706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.96499999999999997,"15min":1.177},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.34399999999999997},"max":{"1min":1.1419999999999999,"5min":2.7389999999999999,"15min":30.460999999999999},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.115,"15min":1.26},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.314},"max":{"1min":1.6539999999999999,"5min":5.4779999999999998,"15min":18.658000000000001},"last":1.093}]},{"osd":2,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.075,"15min":1.1579999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.5589999999999999,"5min":1.7809999999999999,"15min":7.117},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.325,"15min":1.5129999999999999},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.728,"5min":1.98,"15min":30.593},"last":1.03}]},{"osd":3,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2529999999999999,"15min":1.45},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.48599999999999999},"max":{"1min":1.6779999999999999,"5min":1.889,"15min":30.547000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.1619999999999999,"15min":1.3200000000000001},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":2.0459999999999998,"15min":17.263000000000002},"last":1.268}]},{"osd":4,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.6639999999999999,"15min":1.6899999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.72699999999999998},"max":{"1min":1.6319999999999999,"5min":14.103,"15min":17.420000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.653,"15min":1.6040000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7849999999999999,"5min":14.140000000000001,"15min":14.140000000000001},"last":1.371}]},{"osd":5,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3859999999999999,"15min":1.5329999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.59599999999999997},"max":{"1min":1.843,"5min":2.077,"15min":9.0440000000000005},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.508,"15min":1.7989999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.68300000000000005},"max":{"1min":1.891,"5min":2.3599999999999999,"15min":34.042999999999999},"last":1.9850000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.494,"15min":1.6830000000000001},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":0.64100000000000001},"max":{"1min":1.9790000000000001,"5min":2.3780000000000001,"15min":34.158000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.4850000000000001,"15min":1.637},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.62},"max":{"1min":1.8260000000000001,"5min":2.5579999999999998,"15min":17.510000000000002},"last":1.9399999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.603,"15min":1.8700000000000001},"min":{"1min":0.98299999999999998,"5min":0.82999999999999996,"15min":0.77300000000000002},"max":{"1min":1.998,"5min":2.4329999999999998,"15min":33.659999999999997},"last":2.129},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.6439999999999999,"15min":1.742},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.70299999999999996},"max":{"1min":1.9370000000000001,"5min":2.6669999999999998,"15min":11.882999999999999},"last":1.5569999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542422,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1769760,"kb_used_data":1698392,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":92602080,"statfs":{"total":96636764160,"available":94824529920,"internally_reserved":0,"allocated":1739153408,"data_stored":1738300216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.1080000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4630000000000001,"5min":2.5209999999999999,"15min":12.050000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3100000000000001,"15min":1.357},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.36399999999999999},"max":{"1min":1.609,"5min":6.6890000000000001,"15min":11.962},"last":0.98299999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.248,"15min":1.2230000000000001},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.45500000000000002},"max":{"1min":1.7050000000000001,"5min":6.8099999999999996,"15min":11.59},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.995,"5min":1.0620000000000001,"15min":1.1970000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.6870000000000001,"5min":1.825,"15min":14.512},"last":1.1899999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.385,"15min":1.425},"min":{"1min":0.58899999999999997,"5min":0.51300000000000001,"15min":0.50800000000000001},"max":{"1min":1.7130000000000001,"5min":6.8719999999999999,"15min":11.904},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1899999999999999,"15min":1.1830000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.35199999999999998},"max":{"1min":1.696,"5min":2.3919999999999999,"15min":6.1539999999999999},"last":1.1439999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.29,"15min":1.542},"min":{"1min":0.68000000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.5860000000000001,"5min":2.1829999999999998,"15min":32.115000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.579,"15min":1.6259999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.67700000000000005},"max":{"1min":1.6890000000000001,"5min":6.9329999999999998,"15min":11.992000000000001},"last":1.4970000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3879999999999999,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.60899999999999999,"15min":0.47499999999999998},"max":{"1min":1.514,"5min":2.593,"15min":32.174999999999997},"last":1.091},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.6020000000000001,"15min":1.6160000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.442},"max":{"1min":2.157,"5min":15.907,"15min":17.225000000000001},"last":1.464}]},{"osd":6,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.528,"15min":1.754},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.66600000000000004},"max":{"1min":2.3290000000000002,"5min":2.5590000000000002,"15min":24.039999999999999},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.49,"15min":1.7789999999999999},"min":{"1min":0.99199999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.758,"5min":2.5449999999999999,"15min":32.216999999999999},"last":1.2989999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.708,"15min":2.0800000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.69099999999999995},"max":{"1min":1.8300000000000001,"5min":7.0979999999999999,"15min":34.131},"last":1.395},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.641,"15min":1.984},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.61799999999999999},"max":{"1min":2.266,"5min":3.3759999999999999,"15min":44.694000000000003},"last":1.6559999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":632389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":647168,"data_stored":632347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1736458240,"data_stored":1736442609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":931135488,"data_stored":931135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":851443712,"data_stored":851443712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:17:52.357 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-15T18:17:52.357 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:17:52.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.615+0000 7f5b44600640 1 Processor -- start 2024-01-15T18:17:52.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.616+0000 7f5b44600640 1 -- start start 2024-01-15T18:17:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.617+0000 7f5b44600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b3c13df80 0x7f5b3c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.617+0000 7f5b44600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5b3c13e360 0x7f5b3c0cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.617+0000 7f5b44600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c15f9c0 0x7f5b3c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.617+0000 7f5b44600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c06c470 con 0x7f5b3c13df80 2024-01-15T18:17:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.617+0000 7f5b44600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3c06c750 con 0x7f5b3c13e360 2024-01-15T18:17:52.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.617+0000 7f5b44600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c06ca30 con 0x7f5b3c15f9c0 2024-01-15T18:17:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b41c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b3c13df80 0x7f5b3c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b41c00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f5b3c13df80 0x7f5b3c15bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56440/0 (socket says 172.21.15.26:56440) 2024-01-15T18:17:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b41c00640 1 -- 172.21.15.26:0/434430314 learned_addr learned my addr 172.21.15.26:0/434430314 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b41200640 1 --2- 172.21.15.26:0/434430314 >> v2:172.21.15.124:3300/0 conn(0x7f5b3c13e360 0x7f5b3c0cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b42600640 1 --2- 172.21.15.26:0/434430314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c15f9c0 0x7f5b3c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b41c00640 1 -- 172.21.15.26:0/434430314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c15f9c0 msgr2=0x7f5b3c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b41c00640 1 --2- 172.21.15.26:0/434430314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c15f9c0 0x7f5b3c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b41c00640 1 -- 172.21.15.26:0/434430314 >> v2:172.21.15.124:3300/0 conn(0x7f5b3c13e360 msgr2=0x7f5b3c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b41c00640 1 --2- 172.21.15.26:0/434430314 >> v2:172.21.15.124:3300/0 conn(0x7f5b3c13e360 0x7f5b3c0cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.618+0000 7f5b41c00640 1 -- 172.21.15.26:0/434430314 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c06cd10 con 0x7f5b3c13df80 2024-01-15T18:17:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.620+0000 7f5b41c00640 1 --2- 172.21.15.26:0/434430314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 0x7f5b3c15bee0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c06ded0 tx=0x7f5b2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3afda882ddef36 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.620+0000 7f5b3be00640 1 -- 172.21.15.26:0/434430314 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c031870 con 0x7f5b3c13df80 2024-01-15T18:17:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.620+0000 7f5b3be00640 1 -- 172.21.15.26:0/434430314 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c0319d0 con 0x7f5b3c13df80 2024-01-15T18:17:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.620+0000 7f5b3be00640 1 -- 172.21.15.26:0/434430314 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c031d60 con 0x7f5b3c13df80 2024-01-15T18:17:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.621+0000 7f5b44600640 1 -- 172.21.15.26:0/434430314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 msgr2=0x7f5b3c15bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.624 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.621+0000 7f5b44600640 1 --2- 172.21.15.26:0/434430314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 0x7f5b3c15bee0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c06ded0 tx=0x7f5b2c02b910 comp rx=0 tx=0).stop 2024-01-15T18:17:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.621+0000 7f5b44600640 1 -- 172.21.15.26:0/434430314 shutdown_connections 2024-01-15T18:17:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.621+0000 7f5b44600640 1 --2- 172.21.15.26:0/434430314 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c15f9c0 0x7f5b3c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.621+0000 7f5b44600640 1 --2- 172.21.15.26:0/434430314 >> v2:172.21.15.124:3300/0 conn(0x7f5b3c13e360 0x7f5b3c0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.621+0000 7f5b44600640 1 --2- 172.21.15.26:0/434430314 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 0x7f5b3c15bee0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.621+0000 7f5b44600640 1 -- 172.21.15.26:0/434430314 >> 172.21.15.26:0/434430314 conn(0x7f5b3c06a800 msgr2=0x7f5b3c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.621+0000 7f5b44600640 1 -- 172.21.15.26:0/434430314 shutdown_connections 2024-01-15T18:17:52.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.621+0000 7f5b44600640 1 -- 172.21.15.26:0/434430314 wait complete. 2024-01-15T18:17:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.623+0000 7f5b44600640 1 Processor -- start 2024-01-15T18:17:52.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.623+0000 7f5b44600640 1 -- start start 2024-01-15T18:17:52.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.624+0000 7f5b44600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 0x7f5b3c1f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.624+0000 7f5b44600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c13e360 0x7f5b3c1e4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.624+0000 7f5b44600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b3c15f9c0 0x7f5b3c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.624+0000 7f5b44600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1f3520 con 0x7f5b3c13df80 2024-01-15T18:17:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.624+0000 7f5b44600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1e56d0 con 0x7f5b3c15f9c0 2024-01-15T18:17:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.624+0000 7f5b44600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3c1e59b0 con 0x7f5b3c13e360 2024-01-15T18:17:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b41c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 0x7f5b3c1f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b41c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 0x7f5b3c1f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56454/0 (socket says 172.21.15.26:56454) 2024-01-15T18:17:52.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b41c00640 1 -- 172.21.15.26:0/1295432680 learned_addr learned my addr 172.21.15.26:0/1295432680 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b42600640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b3c15f9c0 0x7f5b3c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b41200640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c13e360 0x7f5b3c1e4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b41c00640 1 -- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c13e360 msgr2=0x7f5b3c1e4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b41c00640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c13e360 0x7f5b3c1e4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b41c00640 1 -- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b3c15f9c0 msgr2=0x7f5b3c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b41c00640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b3c15f9c0 0x7f5b3c0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b41c00640 1 -- 172.21.15.26:0/1295432680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0094d0 con 0x7f5b3c13df80 2024-01-15T18:17:52.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.625+0000 7f5b42600640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b3c15f9c0 0x7f5b3c0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.626+0000 7f5b41c00640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 0x7f5b3c1f2fe0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0090b0 tx=0x7f5b2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:52.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.626+0000 7f5b39600640 1 -- 172.21.15.26:0/1295432680 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c03aca0 con 0x7f5b3c13df80 2024-01-15T18:17:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.626+0000 7f5b39600640 1 -- 172.21.15.26:0/1295432680 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c03ae00 con 0x7f5b3c13df80 2024-01-15T18:17:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.626+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3c1e5c90 con 0x7f5b3c13df80 2024-01-15T18:17:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.626+0000 7f5b39600640 1 -- 172.21.15.26:0/1295432680 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c044630 con 0x7f5b3c13df80 2024-01-15T18:17:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.626+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3c1e6250 con 0x7f5b3c13df80 2024-01-15T18:17:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.627+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3c140890 con 0x7f5b3c13df80 2024-01-15T18:17:52.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.629+0000 7f5b39600640 1 -- 172.21.15.26:0/1295432680 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f5b2c04d450 con 0x7f5b3c13df80 2024-01-15T18:17:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.630+0000 7f5b39600640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b1406c550 0x7f5b1406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.630+0000 7f5b39600640 1 -- 172.21.15.26:0/1295432680 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f5b2c02d040 con 0x7f5b3c13df80 2024-01-15T18:17:52.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.631+0000 7f5b41200640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b1406c550 0x7f5b1406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:52.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.632+0000 7f5b41200640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b1406c550 0x7f5b1406ea10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c1407e0 tx=0x7f5b24008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:52.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.635+0000 7f5b39600640 1 -- 172.21.15.26:0/1295432680 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b2c077600 con 0x7f5b3c13df80 2024-01-15T18:17:52.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.826+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b3c0675d0 con 0x7f5b1406c550 2024-01-15T18:17:52.855 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.853+0000 7f5b39600640 1 -- 172.21.15.26:0/1295432680 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288822 (secure 0 0 0) 0x7f5b3c0675d0 con 0x7f5b1406c550 2024-01-15T18:17:52.859 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:17:52.861 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:17:52.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.859+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b1406c550 msgr2=0x7f5b1406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.862 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.859+0000 7f5b44600640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b1406c550 0x7f5b1406ea10 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c1407e0 tx=0x7f5b24008040 comp rx=0 tx=0).stop 2024-01-15T18:17:52.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.859+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 msgr2=0x7f5b3c1f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:52.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.859+0000 7f5b44600640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 0x7f5b3c1f2fe0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0090b0 tx=0x7f5b2c0046c0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.860+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 shutdown_connections 2024-01-15T18:17:52.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.860+0000 7f5b44600640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b3c13e360 0x7f5b3c1e4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.860+0000 7f5b44600640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b1406c550 0x7f5b1406ea10 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.860+0000 7f5b44600640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b3c15f9c0 0x7f5b3c0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.860+0000 7f5b44600640 1 --2- 172.21.15.26:0/1295432680 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b3c13df80 0x7f5b3c1f2fe0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:52.863 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.860+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 >> 172.21.15.26:0/1295432680 conn(0x7f5b3c06a800 msgr2=0x7f5b3c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:52.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.860+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 shutdown_connections 2024-01-15T18:17:52.864 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:52.860+0000 7f5b44600640 1 -- 172.21.15.26:0/1295432680 wait complete. 2024-01-15T18:17:52.882 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":798,"stamp":"2024-01-15T18:17:52.451446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369934897,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1739,"num_write_kb":5246057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3181,"ondisk_log_size":3181,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11062216,"kb_used_data":10499592,"kb_used_omap":12,"kb_used_meta":562547,"kb_avail":743912504,"statfs":{"total":773094113280,"available":761766404096,"internally_reserved":0,"allocated":10751582208,"data_stored":10744951498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":576048688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1073781794,"num_objects":256,"num_object_clones":0,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":256,"num_read_kb":1048576,"num_write":275,"num_write_kb":1048622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004813"},"pg_stats":[{"pgid":"17.3f","version":"117'53","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.789093+0000","last_change":"2024-01-15T18:12:53.636580+0000","last_active":"2024-01-15T18:17:38.789093+0000","last_peered":"2024-01-15T18:17:38.789093+0000","last_clean":"2024-01-15T18:17:38.789093+0000","last_became_active":"2024-01-15T18:12:53.334688+0000","last_became_peered":"2024-01-15T18:12:53.334688+0000","last_unstale":"2024-01-15T18:17:38.789093+0000","last_undegraded":"2024-01-15T18:17:38.789093+0000","last_fullsized":"2024-01-15T18:17:38.789093+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:39.949271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3e","version":"117'45","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.174108+0000","last_change":"2024-01-15T18:12:11.053712+0000","last_active":"2024-01-15T18:17:39.174108+0000","last_peered":"2024-01-15T18:17:39.174108+0000","last_clean":"2024-01-15T18:17:39.174108+0000","last_became_active":"2024-01-15T18:12:11.053279+0000","last_became_peered":"2024-01-15T18:12:11.053279+0000","last_unstale":"2024-01-15T18:17:39.174108+0000","last_undegraded":"2024-01-15T18:17:39.174108+0000","last_fullsized":"2024-01-15T18:17:39.174108+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"117'44","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.301593+0000","last_change":"2024-01-15T18:12:11.058625+0000","last_active":"2024-01-15T18:17:39.301593+0000","last_peered":"2024-01-15T18:17:39.301593+0000","last_clean":"2024-01-15T18:17:39.301593+0000","last_became_active":"2024-01-15T18:12:11.057543+0000","last_became_peered":"2024-01-15T18:12:11.057543+0000","last_unstale":"2024-01-15T18:17:39.301593+0000","last_undegraded":"2024-01-15T18:17:39.301593+0000","last_fullsized":"2024-01-15T18:17:39.301593+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:17.569977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"117'45","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.237120+0000","last_change":"2024-01-15T18:12:11.047313+0000","last_active":"2024-01-15T18:17:39.237120+0000","last_peered":"2024-01-15T18:17:39.237120+0000","last_clean":"2024-01-15T18:17:39.237120+0000","last_became_active":"2024-01-15T18:12:11.047008+0000","last_became_peered":"2024-01-15T18:12:11.047008+0000","last_unstale":"2024-01-15T18:17:39.237120+0000","last_undegraded":"2024-01-15T18:17:39.237120+0000","last_fullsized":"2024-01-15T18:17:39.237120+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:03:15.070494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"117'27","reported_seq":49,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.653418+0000","last_change":"2024-01-15T18:12:11.035567+0000","last_active":"2024-01-15T18:17:38.653418+0000","last_peered":"2024-01-15T18:17:38.653418+0000","last_clean":"2024-01-15T18:17:38.653418+0000","last_became_active":"2024-01-15T18:12:11.034909+0000","last_became_peered":"2024-01-15T18:12:11.034909+0000","last_unstale":"2024-01-15T18:17:38.653418+0000","last_undegraded":"2024-01-15T18:17:38.653418+0000","last_fullsized":"2024-01-15T18:17:38.653418+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:46.811591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"117'48","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.922505+0000","last_change":"2024-01-15T18:12:11.055133+0000","last_active":"2024-01-15T18:17:38.922505+0000","last_peered":"2024-01-15T18:17:38.922505+0000","last_clean":"2024-01-15T18:17:38.922505+0000","last_became_active":"2024-01-15T18:12:11.054803+0000","last_became_peered":"2024-01-15T18:12:11.054803+0000","last_unstale":"2024-01-15T18:17:38.922505+0000","last_undegraded":"2024-01-15T18:17:38.922505+0000","last_fullsized":"2024-01-15T18:17:38.922505+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:53.540866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"117'43","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.895097+0000","last_change":"2024-01-15T18:12:11.045438+0000","last_active":"2024-01-15T18:17:38.895097+0000","last_peered":"2024-01-15T18:17:38.895097+0000","last_clean":"2024-01-15T18:17:38.895097+0000","last_became_active":"2024-01-15T18:12:11.044607+0000","last_became_peered":"2024-01-15T18:12:11.044607+0000","last_unstale":"2024-01-15T18:17:38.895097+0000","last_undegraded":"2024-01-15T18:17:38.895097+0000","last_fullsized":"2024-01-15T18:17:38.895097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:44.315673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"117'44","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.859692+0000","last_change":"2024-01-15T18:12:11.051056+0000","last_active":"2024-01-15T18:17:38.859692+0000","last_peered":"2024-01-15T18:17:38.859692+0000","last_clean":"2024-01-15T18:17:38.859692+0000","last_became_active":"2024-01-15T18:12:11.050697+0000","last_became_peered":"2024-01-15T18:12:11.050697+0000","last_unstale":"2024-01-15T18:17:38.859692+0000","last_undegraded":"2024-01-15T18:17:38.859692+0000","last_fullsized":"2024-01-15T18:17:38.859692+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:56.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.359403+0000","last_change":"2024-01-15T18:12:11.048227+0000","last_active":"2024-01-15T18:17:39.359403+0000","last_peered":"2024-01-15T18:17:39.359403+0000","last_clean":"2024-01-15T18:17:39.359403+0000","last_became_active":"2024-01-15T18:12:11.047693+0000","last_became_peered":"2024-01-15T18:12:11.047693+0000","last_unstale":"2024-01-15T18:17:39.359403+0000","last_undegraded":"2024-01-15T18:17:39.359403+0000","last_fullsized":"2024-01-15T18:17:39.359403+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:23:43.904890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"117'47","reported_seq":72,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.466111+0000","last_change":"2024-01-15T18:12:11.054486+0000","last_active":"2024-01-15T18:17:39.466111+0000","last_peered":"2024-01-15T18:17:39.466111+0000","last_clean":"2024-01-15T18:17:39.466111+0000","last_became_active":"2024-01-15T18:12:11.052429+0000","last_became_peered":"2024-01-15T18:12:11.052429+0000","last_unstale":"2024-01-15T18:17:39.466111+0000","last_undegraded":"2024-01-15T18:17:39.466111+0000","last_fullsized":"2024-01-15T18:17:39.466111+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:42:17.440558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"117'47","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.167604+0000","last_change":"2024-01-15T18:12:11.043835+0000","last_active":"2024-01-15T18:17:39.167604+0000","last_peered":"2024-01-15T18:17:39.167604+0000","last_clean":"2024-01-15T18:17:39.167604+0000","last_became_active":"2024-01-15T18:12:11.042720+0000","last_became_peered":"2024-01-15T18:12:11.042720+0000","last_unstale":"2024-01-15T18:17:39.167604+0000","last_undegraded":"2024-01-15T18:17:39.167604+0000","last_fullsized":"2024-01-15T18:17:39.167604+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"117'48","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.287663+0000","last_change":"2024-01-15T18:12:11.044468+0000","last_active":"2024-01-15T18:17:39.287663+0000","last_peered":"2024-01-15T18:17:39.287663+0000","last_clean":"2024-01-15T18:17:39.287663+0000","last_became_active":"2024-01-15T18:12:11.044155+0000","last_became_peered":"2024-01-15T18:12:11.044155+0000","last_unstale":"2024-01-15T18:17:39.287663+0000","last_undegraded":"2024-01-15T18:17:39.287663+0000","last_fullsized":"2024-01-15T18:17:39.287663+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:07:57.310458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"117'38","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.737859+0000","last_change":"2024-01-15T18:12:11.052950+0000","last_active":"2024-01-15T18:17:38.737859+0000","last_peered":"2024-01-15T18:17:38.737859+0000","last_clean":"2024-01-15T18:17:38.737859+0000","last_became_active":"2024-01-15T18:12:11.051875+0000","last_became_peered":"2024-01-15T18:12:11.051875+0000","last_unstale":"2024-01-15T18:17:38.737859+0000","last_undegraded":"2024-01-15T18:17:38.737859+0000","last_fullsized":"2024-01-15T18:17:38.737859+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:22.849560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"117'51","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.417564+0000","last_change":"2024-01-15T18:12:11.056103+0000","last_active":"2024-01-15T18:17:39.417564+0000","last_peered":"2024-01-15T18:17:39.417564+0000","last_clean":"2024-01-15T18:17:39.417564+0000","last_became_active":"2024-01-15T18:12:11.055813+0000","last_became_peered":"2024-01-15T18:12:11.055813+0000","last_unstale":"2024-01-15T18:17:39.417564+0000","last_undegraded":"2024-01-15T18:17:39.417564+0000","last_fullsized":"2024-01-15T18:17:39.417564+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.025824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"117'34","reported_seq":53,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.330592+0000","last_change":"2024-01-15T18:12:11.056863+0000","last_active":"2024-01-15T18:17:39.330592+0000","last_peered":"2024-01-15T18:17:39.330592+0000","last_clean":"2024-01-15T18:17:39.330592+0000","last_became_active":"2024-01-15T18:12:11.055217+0000","last_became_peered":"2024-01-15T18:12:11.055217+0000","last_unstale":"2024-01-15T18:17:39.330592+0000","last_undegraded":"2024-01-15T18:17:39.330592+0000","last_fullsized":"2024-01-15T18:17:39.330592+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:33.702581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"117'42","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.869131+0000","last_change":"2024-01-15T18:12:53.618454+0000","last_active":"2024-01-15T18:17:38.869131+0000","last_peered":"2024-01-15T18:17:38.869131+0000","last_clean":"2024-01-15T18:17:38.869131+0000","last_became_active":"2024-01-15T18:12:53.338860+0000","last_became_peered":"2024-01-15T18:12:53.338860+0000","last_unstale":"2024-01-15T18:17:38.869131+0000","last_undegraded":"2024-01-15T18:17:38.869131+0000","last_fullsized":"2024-01-15T18:17:38.869131+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:40.129590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2f","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.108098+0000","last_change":"2024-01-15T18:12:11.043060+0000","last_active":"2024-01-15T18:17:39.108098+0000","last_peered":"2024-01-15T18:17:39.108098+0000","last_clean":"2024-01-15T18:17:39.108098+0000","last_became_active":"2024-01-15T18:12:11.042031+0000","last_became_peered":"2024-01-15T18:12:11.042031+0000","last_unstale":"2024-01-15T18:17:39.108098+0000","last_undegraded":"2024-01-15T18:17:39.108098+0000","last_fullsized":"2024-01-15T18:17:39.108098+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:44.791297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"117'48","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.357897+0000","last_change":"2024-01-15T18:12:11.045284+0000","last_active":"2024-01-15T18:17:39.357897+0000","last_peered":"2024-01-15T18:17:39.357897+0000","last_clean":"2024-01-15T18:17:39.357897+0000","last_became_active":"2024-01-15T18:12:11.042814+0000","last_became_peered":"2024-01-15T18:12:11.042814+0000","last_unstale":"2024-01-15T18:17:39.357897+0000","last_undegraded":"2024-01-15T18:17:39.357897+0000","last_fullsized":"2024-01-15T18:17:39.357897+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.108742+0000","last_change":"2024-01-15T18:12:11.055119+0000","last_active":"2024-01-15T18:17:39.108742+0000","last_peered":"2024-01-15T18:17:39.108742+0000","last_clean":"2024-01-15T18:17:39.108742+0000","last_became_active":"2024-01-15T18:12:11.054790+0000","last_became_peered":"2024-01-15T18:12:11.054790+0000","last_unstale":"2024-01-15T18:17:39.108742+0000","last_undegraded":"2024-01-15T18:17:39.108742+0000","last_fullsized":"2024-01-15T18:17:39.108742+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:17.513018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"117'46","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.362355+0000","last_change":"2024-01-15T18:12:11.052755+0000","last_active":"2024-01-15T18:17:39.362355+0000","last_peered":"2024-01-15T18:17:39.362355+0000","last_clean":"2024-01-15T18:17:39.362355+0000","last_became_active":"2024-01-15T18:12:11.052289+0000","last_became_peered":"2024-01-15T18:12:11.052289+0000","last_unstale":"2024-01-15T18:17:39.362355+0000","last_undegraded":"2024-01-15T18:17:39.362355+0000","last_fullsized":"2024-01-15T18:17:39.362355+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:38:56.937377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"117'56","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.083979+0000","last_change":"2024-01-15T18:12:11.036748+0000","last_active":"2024-01-15T18:17:39.083979+0000","last_peered":"2024-01-15T18:17:39.083979+0000","last_clean":"2024-01-15T18:17:39.083979+0000","last_became_active":"2024-01-15T18:12:11.036003+0000","last_became_peered":"2024-01-15T18:12:11.036003+0000","last_unstale":"2024-01-15T18:17:39.083979+0000","last_undegraded":"2024-01-15T18:17:39.083979+0000","last_fullsized":"2024-01-15T18:17:39.083979+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:20.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.456439+0000","last_change":"2024-01-15T18:12:11.050445+0000","last_active":"2024-01-15T18:17:39.456439+0000","last_peered":"2024-01-15T18:17:39.456439+0000","last_clean":"2024-01-15T18:17:39.456439+0000","last_became_active":"2024-01-15T18:12:11.044093+0000","last_became_peered":"2024-01-15T18:12:11.044093+0000","last_unstale":"2024-01-15T18:17:39.456439+0000","last_undegraded":"2024-01-15T18:17:39.456439+0000","last_fullsized":"2024-01-15T18:17:39.456439+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:38:59.097623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"117'39","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.269675+0000","last_change":"2024-01-15T18:12:11.053397+0000","last_active":"2024-01-15T18:17:39.269675+0000","last_peered":"2024-01-15T18:17:39.269675+0000","last_clean":"2024-01-15T18:17:39.269675+0000","last_became_active":"2024-01-15T18:12:11.052428+0000","last_became_peered":"2024-01-15T18:12:11.052428+0000","last_unstale":"2024-01-15T18:17:39.269675+0000","last_undegraded":"2024-01-15T18:17:39.269675+0000","last_fullsized":"2024-01-15T18:17:39.269675+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:07.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"117'43","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.364184+0000","last_change":"2024-01-15T18:12:11.051820+0000","last_active":"2024-01-15T18:17:39.364184+0000","last_peered":"2024-01-15T18:17:39.364184+0000","last_clean":"2024-01-15T18:17:39.364184+0000","last_became_active":"2024-01-15T18:12:11.051488+0000","last_became_peered":"2024-01-15T18:12:11.051488+0000","last_unstale":"2024-01-15T18:17:39.364184+0000","last_undegraded":"2024-01-15T18:17:39.364184+0000","last_fullsized":"2024-01-15T18:17:39.364184+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:42:45.480493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"117'38","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.840614+0000","last_change":"2024-01-15T18:12:11.041672+0000","last_active":"2024-01-15T18:17:38.840614+0000","last_peered":"2024-01-15T18:17:38.840614+0000","last_clean":"2024-01-15T18:17:38.840614+0000","last_became_active":"2024-01-15T18:12:11.041266+0000","last_became_peered":"2024-01-15T18:12:11.041266+0000","last_unstale":"2024-01-15T18:17:38.840614+0000","last_undegraded":"2024-01-15T18:17:38.840614+0000","last_fullsized":"2024-01-15T18:17:38.840614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"117'53","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.369799+0000","last_change":"2024-01-15T18:12:11.048567+0000","last_active":"2024-01-15T18:17:39.369799+0000","last_peered":"2024-01-15T18:17:39.369799+0000","last_clean":"2024-01-15T18:17:39.369799+0000","last_became_active":"2024-01-15T18:12:11.048234+0000","last_became_peered":"2024-01-15T18:12:11.048234+0000","last_unstale":"2024-01-15T18:17:39.369799+0000","last_undegraded":"2024-01-15T18:17:39.369799+0000","last_fullsized":"2024-01-15T18:17:39.369799+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:12.290504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"117'37","reported_seq":59,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.691915+0000","last_change":"2024-01-15T18:12:11.062039+0000","last_active":"2024-01-15T18:17:38.691915+0000","last_peered":"2024-01-15T18:17:38.691915+0000","last_clean":"2024-01-15T18:17:38.691915+0000","last_became_active":"2024-01-15T18:12:11.061724+0000","last_became_peered":"2024-01-15T18:12:11.061724+0000","last_unstale":"2024-01-15T18:17:38.691915+0000","last_undegraded":"2024-01-15T18:17:38.691915+0000","last_fullsized":"2024-01-15T18:17:38.691915+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:25:03.329007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"117'38","reported_seq":56,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.276302+0000","last_change":"2024-01-15T18:12:11.044940+0000","last_active":"2024-01-15T18:17:39.276302+0000","last_peered":"2024-01-15T18:17:39.276302+0000","last_clean":"2024-01-15T18:17:39.276302+0000","last_became_active":"2024-01-15T18:12:11.043801+0000","last_became_peered":"2024-01-15T18:12:11.043801+0000","last_unstale":"2024-01-15T18:17:39.276302+0000","last_undegraded":"2024-01-15T18:17:39.276302+0000","last_fullsized":"2024-01-15T18:17:39.276302+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:53:43.341609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"117'46","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.347960+0000","last_change":"2024-01-15T18:12:11.048119+0000","last_active":"2024-01-15T18:17:39.347960+0000","last_peered":"2024-01-15T18:17:39.347960+0000","last_clean":"2024-01-15T18:17:39.347960+0000","last_became_active":"2024-01-15T18:12:11.047739+0000","last_became_peered":"2024-01-15T18:12:11.047739+0000","last_unstale":"2024-01-15T18:17:39.347960+0000","last_undegraded":"2024-01-15T18:17:39.347960+0000","last_fullsized":"2024-01-15T18:17:39.347960+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:49:08.557895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"117'41","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.292578+0000","last_change":"2024-01-15T18:12:11.040703+0000","last_active":"2024-01-15T18:17:39.292578+0000","last_peered":"2024-01-15T18:17:39.292578+0000","last_clean":"2024-01-15T18:17:39.292578+0000","last_became_active":"2024-01-15T18:12:11.039393+0000","last_became_peered":"2024-01-15T18:12:11.039393+0000","last_unstale":"2024-01-15T18:17:39.292578+0000","last_undegraded":"2024-01-15T18:17:39.292578+0000","last_fullsized":"2024-01-15T18:17:39.292578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:40:41.329686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"117'62","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.906525+0000","last_change":"2024-01-15T18:12:11.059603+0000","last_active":"2024-01-15T18:17:38.906525+0000","last_peered":"2024-01-15T18:17:38.906525+0000","last_clean":"2024-01-15T18:17:38.906525+0000","last_became_active":"2024-01-15T18:12:11.059276+0000","last_became_peered":"2024-01-15T18:12:11.059276+0000","last_unstale":"2024-01-15T18:17:38.906525+0000","last_undegraded":"2024-01-15T18:17:38.906525+0000","last_fullsized":"2024-01-15T18:17:38.906525+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:43:52.890489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"117'54","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.208710+0000","last_change":"2024-01-15T18:12:11.053438+0000","last_active":"2024-01-15T18:17:39.208710+0000","last_peered":"2024-01-15T18:17:39.208710+0000","last_clean":"2024-01-15T18:17:39.208710+0000","last_became_active":"2024-01-15T18:12:11.052655+0000","last_became_peered":"2024-01-15T18:12:11.052655+0000","last_unstale":"2024-01-15T18:17:39.208710+0000","last_undegraded":"2024-01-15T18:17:39.208710+0000","last_fullsized":"2024-01-15T18:17:39.208710+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:45.306312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"117'34","reported_seq":55,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.008114+0000","last_change":"2024-01-15T18:12:11.053689+0000","last_active":"2024-01-15T18:17:39.008114+0000","last_peered":"2024-01-15T18:17:39.008114+0000","last_clean":"2024-01-15T18:17:39.008114+0000","last_became_active":"2024-01-15T18:12:11.053389+0000","last_became_peered":"2024-01-15T18:12:11.053389+0000","last_unstale":"2024-01-15T18:17:39.008114+0000","last_undegraded":"2024-01-15T18:17:39.008114+0000","last_fullsized":"2024-01-15T18:17:39.008114+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:53:39.171657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"114'4","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.409090+0000","last_change":"2024-01-15T18:12:10.021097+0000","last_active":"2024-01-15T18:14:17.409090+0000","last_peered":"2024-01-15T18:14:17.409090+0000","last_clean":"2024-01-15T18:14:17.409090+0000","last_became_active":"2024-01-15T18:12:10.020810+0000","last_became_peered":"2024-01-15T18:12:10.020810+0000","last_unstale":"2024-01-15T18:14:17.409090+0000","last_undegraded":"2024-01-15T18:14:17.409090+0000","last_fullsized":"2024-01-15T18:14:17.409090+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:09:27.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.721958+0000","last_change":"2024-01-15T18:12:11.043347+0000","last_active":"2024-01-15T18:17:38.721958+0000","last_peered":"2024-01-15T18:17:38.721958+0000","last_clean":"2024-01-15T18:17:38.721958+0000","last_became_active":"2024-01-15T18:12:11.042427+0000","last_became_peered":"2024-01-15T18:12:11.042427+0000","last_unstale":"2024-01-15T18:17:38.721958+0000","last_undegraded":"2024-01-15T18:17:38.721958+0000","last_fullsized":"2024-01-15T18:17:38.721958+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:12.407664+0000","last_change":"2024-01-15T18:12:10.023175+0000","last_active":"2024-01-15T18:14:12.407664+0000","last_peered":"2024-01-15T18:14:12.407664+0000","last_clean":"2024-01-15T18:14:12.407664+0000","last_became_active":"2024-01-15T18:12:10.022654+0000","last_became_peered":"2024-01-15T18:12:10.022654+0000","last_unstale":"2024-01-15T18:14:12.407664+0000","last_undegraded":"2024-01-15T18:14:12.407664+0000","last_fullsized":"2024-01-15T18:14:12.407664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:12.136270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'43","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.190370+0000","last_change":"2024-01-15T18:12:11.053138+0000","last_active":"2024-01-15T18:17:39.190370+0000","last_peered":"2024-01-15T18:17:39.190370+0000","last_clean":"2024-01-15T18:17:39.190370+0000","last_became_active":"2024-01-15T18:12:11.052812+0000","last_became_peered":"2024-01-15T18:12:11.052812+0000","last_unstale":"2024-01-15T18:17:39.190370+0000","last_undegraded":"2024-01-15T18:17:39.190370+0000","last_fullsized":"2024-01-15T18:17:39.190370+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:40.787531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"117'3","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.287481+0000","last_change":"2024-01-15T18:12:10.018995+0000","last_active":"2024-01-15T18:14:42.287481+0000","last_peered":"2024-01-15T18:14:42.287481+0000","last_clean":"2024-01-15T18:14:42.287481+0000","last_became_active":"2024-01-15T18:12:10.018670+0000","last_became_peered":"2024-01-15T18:12:10.018670+0000","last_unstale":"2024-01-15T18:14:42.287481+0000","last_undegraded":"2024-01-15T18:14:42.287481+0000","last_fullsized":"2024-01-15T18:14:42.287481+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:41:39.773731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"117'56","reported_seq":82,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.844198+0000","last_change":"2024-01-15T18:12:11.059355+0000","last_active":"2024-01-15T18:17:38.844198+0000","last_peered":"2024-01-15T18:17:38.844198+0000","last_clean":"2024-01-15T18:17:38.844198+0000","last_became_active":"2024-01-15T18:12:11.057592+0000","last_became_peered":"2024-01-15T18:12:11.057592+0000","last_unstale":"2024-01-15T18:17:38.844198+0000","last_undegraded":"2024-01-15T18:17:38.844198+0000","last_fullsized":"2024-01-15T18:17:38.844198+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:14.826977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.287454+0000","last_change":"2024-01-15T18:12:10.020565+0000","last_active":"2024-01-15T18:14:42.287454+0000","last_peered":"2024-01-15T18:14:42.287454+0000","last_clean":"2024-01-15T18:14:42.287454+0000","last_became_active":"2024-01-15T18:12:10.020143+0000","last_became_peered":"2024-01-15T18:12:10.020143+0000","last_unstale":"2024-01-15T18:14:42.287454+0000","last_undegraded":"2024-01-15T18:14:42.287454+0000","last_fullsized":"2024-01-15T18:14:42.287454+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:09.129175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"117'39","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.444992+0000","last_change":"2024-01-15T18:12:11.057876+0000","last_active":"2024-01-15T18:17:39.444992+0000","last_peered":"2024-01-15T18:17:39.444992+0000","last_clean":"2024-01-15T18:17:39.444992+0000","last_became_active":"2024-01-15T18:12:11.057471+0000","last_became_peered":"2024-01-15T18:12:11.057471+0000","last_unstale":"2024-01-15T18:17:39.444992+0000","last_undegraded":"2024-01-15T18:17:39.444992+0000","last_fullsized":"2024-01-15T18:17:39.444992+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:22:19.460852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.414286+0000","last_change":"2024-01-15T18:12:10.020217+0000","last_active":"2024-01-15T18:14:42.414286+0000","last_peered":"2024-01-15T18:14:42.414286+0000","last_clean":"2024-01-15T18:14:42.414286+0000","last_became_active":"2024-01-15T18:12:10.019818+0000","last_became_peered":"2024-01-15T18:12:10.019818+0000","last_unstale":"2024-01-15T18:14:42.414286+0000","last_undegraded":"2024-01-15T18:14:42.414286+0000","last_fullsized":"2024-01-15T18:14:42.414286+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"117'49","reported_seq":77,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.264806+0000","last_change":"2024-01-15T18:12:11.050383+0000","last_active":"2024-01-15T18:17:39.264806+0000","last_peered":"2024-01-15T18:17:39.264806+0000","last_clean":"2024-01-15T18:17:39.264806+0000","last_became_active":"2024-01-15T18:12:11.050002+0000","last_became_peered":"2024-01-15T18:12:11.050002+0000","last_unstale":"2024-01-15T18:17:39.264806+0000","last_undegraded":"2024-01-15T18:17:39.264806+0000","last_fullsized":"2024-01-15T18:17:39.264806+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.309045+0000","last_change":"2024-01-15T18:12:10.013940+0000","last_active":"2024-01-15T18:14:42.309045+0000","last_peered":"2024-01-15T18:14:42.309045+0000","last_clean":"2024-01-15T18:14:42.309045+0000","last_became_active":"2024-01-15T18:12:10.009676+0000","last_became_peered":"2024-01-15T18:12:10.009676+0000","last_unstale":"2024-01-15T18:14:42.309045+0000","last_undegraded":"2024-01-15T18:14:42.309045+0000","last_fullsized":"2024-01-15T18:14:42.309045+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:08:00.059047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"117'53","reported_seq":78,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.258378+0000","last_change":"2024-01-15T18:12:11.057984+0000","last_active":"2024-01-15T18:17:39.258378+0000","last_peered":"2024-01-15T18:17:39.258378+0000","last_clean":"2024-01-15T18:17:39.258378+0000","last_became_active":"2024-01-15T18:12:11.057640+0000","last_became_peered":"2024-01-15T18:12:11.057640+0000","last_unstale":"2024-01-15T18:17:39.258378+0000","last_undegraded":"2024-01-15T18:17:39.258378+0000","last_fullsized":"2024-01-15T18:17:39.258378+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:57:42.482689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857791,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":36865,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.331147+0000","last_change":"2024-01-15T18:12:10.002836+0000","last_active":"2024-01-15T18:14:37.331147+0000","last_peered":"2024-01-15T18:14:37.331147+0000","last_clean":"2024-01-15T18:14:37.331147+0000","last_became_active":"2024-01-15T18:12:10.002190+0000","last_became_peered":"2024-01-15T18:12:10.002190+0000","last_unstale":"2024-01-15T18:14:37.331147+0000","last_undegraded":"2024-01-15T18:14:37.331147+0000","last_fullsized":"2024-01-15T18:14:37.331147+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:39.115790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"117'43","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.016147+0000","last_change":"2024-01-15T18:12:11.048560+0000","last_active":"2024-01-15T18:17:39.016147+0000","last_peered":"2024-01-15T18:17:39.016147+0000","last_clean":"2024-01-15T18:17:39.016147+0000","last_became_active":"2024-01-15T18:12:11.046934+0000","last_became_peered":"2024-01-15T18:12:11.046934+0000","last_unstale":"2024-01-15T18:17:39.016147+0000","last_undegraded":"2024-01-15T18:17:39.016147+0000","last_fullsized":"2024-01-15T18:17:39.016147+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:44:29.629308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.426824+0000","last_change":"2024-01-15T18:12:10.020864+0000","last_active":"2024-01-15T18:14:37.426824+0000","last_peered":"2024-01-15T18:14:37.426824+0000","last_clean":"2024-01-15T18:14:37.426824+0000","last_became_active":"2024-01-15T18:12:10.020577+0000","last_became_peered":"2024-01-15T18:12:10.020577+0000","last_unstale":"2024-01-15T18:14:37.426824+0000","last_undegraded":"2024-01-15T18:14:37.426824+0000","last_fullsized":"2024-01-15T18:14:37.426824+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:55.380272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"117'42","reported_seq":61,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.550415+0000","last_change":"2024-01-15T18:12:11.048003+0000","last_active":"2024-01-15T18:17:38.550415+0000","last_peered":"2024-01-15T18:17:38.550415+0000","last_clean":"2024-01-15T18:17:38.550415+0000","last_became_active":"2024-01-15T18:12:11.047704+0000","last_became_peered":"2024-01-15T18:12:11.047704+0000","last_unstale":"2024-01-15T18:17:38.550415+0000","last_undegraded":"2024-01-15T18:17:38.550415+0000","last_fullsized":"2024-01-15T18:17:38.550415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:02.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"117'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.286410+0000","last_change":"2024-01-15T18:12:10.020498+0000","last_active":"2024-01-15T18:14:37.286410+0000","last_peered":"2024-01-15T18:14:37.286410+0000","last_clean":"2024-01-15T18:14:37.286410+0000","last_became_active":"2024-01-15T18:12:10.020028+0000","last_became_peered":"2024-01-15T18:12:10.020028+0000","last_unstale":"2024-01-15T18:14:37.286410+0000","last_undegraded":"2024-01-15T18:14:37.286410+0000","last_fullsized":"2024-01-15T18:14:37.286410+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"117'40","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.283436+0000","last_change":"2024-01-15T18:12:11.060166+0000","last_active":"2024-01-15T18:17:39.283436+0000","last_peered":"2024-01-15T18:17:39.283436+0000","last_clean":"2024-01-15T18:17:39.283436+0000","last_became_active":"2024-01-15T18:12:11.059393+0000","last_became_peered":"2024-01-15T18:12:11.059393+0000","last_unstale":"2024-01-15T18:17:39.283436+0000","last_undegraded":"2024-01-15T18:17:39.283436+0000","last_fullsized":"2024-01-15T18:17:39.283436+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.308163+0000","last_change":"2024-01-15T18:12:10.007879+0000","last_active":"2024-01-15T18:14:37.308163+0000","last_peered":"2024-01-15T18:14:37.308163+0000","last_clean":"2024-01-15T18:14:37.308163+0000","last_became_active":"2024-01-15T18:12:10.007549+0000","last_became_peered":"2024-01-15T18:12:10.007549+0000","last_unstale":"2024-01-15T18:14:37.308163+0000","last_undegraded":"2024-01-15T18:14:37.308163+0000","last_fullsized":"2024-01-15T18:14:37.308163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:12:33.832426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"117'43","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.394428+0000","last_change":"2024-01-15T18:12:11.031666+0000","last_active":"2024-01-15T18:17:39.394428+0000","last_peered":"2024-01-15T18:17:39.394428+0000","last_clean":"2024-01-15T18:17:39.394428+0000","last_became_active":"2024-01-15T18:12:11.030958+0000","last_became_peered":"2024-01-15T18:12:11.030958+0000","last_unstale":"2024-01-15T18:17:39.394428+0000","last_undegraded":"2024-01-15T18:17:39.394428+0000","last_fullsized":"2024-01-15T18:17:39.394428+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:04:58.112590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"117'12","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:33.625051+0000","last_change":"2024-01-15T18:12:10.024809+0000","last_active":"2024-01-15T18:15:33.625051+0000","last_peered":"2024-01-15T18:15:33.625051+0000","last_clean":"2024-01-15T18:15:33.625051+0000","last_became_active":"2024-01-15T18:12:10.024120+0000","last_became_peered":"2024-01-15T18:12:10.024120+0000","last_unstale":"2024-01-15T18:15:33.625051+0000","last_undegraded":"2024-01-15T18:15:33.625051+0000","last_fullsized":"2024-01-15T18:15:33.625051+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:17:59.130803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"117'43","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.025487+0000","last_change":"2024-01-15T18:12:11.055694+0000","last_active":"2024-01-15T18:17:39.025487+0000","last_peered":"2024-01-15T18:17:39.025487+0000","last_clean":"2024-01-15T18:17:39.025487+0000","last_became_active":"2024-01-15T18:12:11.055290+0000","last_became_peered":"2024-01-15T18:12:11.055290+0000","last_unstale":"2024-01-15T18:17:39.025487+0000","last_undegraded":"2024-01-15T18:17:39.025487+0000","last_fullsized":"2024-01-15T18:17:39.025487+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:44.436928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"117'183","reported_seq":199,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:44.393465+0000","last_change":"2024-01-15T18:12:10.022950+0000","last_active":"2024-01-15T18:17:44.393465+0000","last_peered":"2024-01-15T18:17:44.393465+0000","last_clean":"2024-01-15T18:17:44.393465+0000","last_became_active":"2024-01-15T18:12:10.021497+0000","last_became_peered":"2024-01-15T18:12:10.021497+0000","last_unstale":"2024-01-15T18:17:44.393465+0000","last_undegraded":"2024-01-15T18:17:44.393465+0000","last_fullsized":"2024-01-15T18:17:44.393465+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:16.556177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":632277,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":182,"num_write_kb":708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"117'44","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.050496+0000","last_change":"2024-01-15T18:12:11.046101+0000","last_active":"2024-01-15T18:17:39.050496+0000","last_peered":"2024-01-15T18:17:39.050496+0000","last_clean":"2024-01-15T18:17:39.050496+0000","last_became_active":"2024-01-15T18:12:11.045774+0000","last_became_peered":"2024-01-15T18:12:11.045774+0000","last_unstale":"2024-01-15T18:17:39.050496+0000","last_undegraded":"2024-01-15T18:17:39.050496+0000","last_fullsized":"2024-01-15T18:17:39.050496+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:41:11.078868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.285076+0000","last_change":"2024-01-15T18:12:10.016912+0000","last_active":"2024-01-15T18:14:32.285076+0000","last_peered":"2024-01-15T18:14:32.285076+0000","last_clean":"2024-01-15T18:14:32.285076+0000","last_became_active":"2024-01-15T18:12:10.016618+0000","last_became_peered":"2024-01-15T18:12:10.016618+0000","last_unstale":"2024-01-15T18:14:32.285076+0000","last_undegraded":"2024-01-15T18:14:32.285076+0000","last_fullsized":"2024-01-15T18:14:32.285076+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:40:48.957828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"117'52","reported_seq":72,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.238652+0000","last_change":"2024-01-15T18:12:11.051223+0000","last_active":"2024-01-15T18:17:39.238652+0000","last_peered":"2024-01-15T18:17:39.238652+0000","last_clean":"2024-01-15T18:17:39.238652+0000","last_became_active":"2024-01-15T18:12:11.050928+0000","last_became_peered":"2024-01-15T18:12:11.050928+0000","last_unstale":"2024-01-15T18:17:39.238652+0000","last_undegraded":"2024-01-15T18:17:39.238652+0000","last_fullsized":"2024-01-15T18:17:39.238652+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:11.318479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.543703+0000","last_change":"2024-01-15T18:12:10.015868+0000","last_active":"2024-01-15T18:14:32.543703+0000","last_peered":"2024-01-15T18:14:32.543703+0000","last_clean":"2024-01-15T18:14:32.543703+0000","last_became_active":"2024-01-15T18:12:10.015578+0000","last_became_peered":"2024-01-15T18:12:10.015578+0000","last_unstale":"2024-01-15T18:14:32.543703+0000","last_undegraded":"2024-01-15T18:14:32.543703+0000","last_fullsized":"2024-01-15T18:14:32.543703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"117'46","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.210181+0000","last_change":"2024-01-15T18:12:11.029654+0000","last_active":"2024-01-15T18:17:39.210181+0000","last_peered":"2024-01-15T18:17:39.210181+0000","last_clean":"2024-01-15T18:17:39.210181+0000","last_became_active":"2024-01-15T18:12:11.028865+0000","last_became_peered":"2024-01-15T18:12:11.028865+0000","last_unstale":"2024-01-15T18:17:39.210181+0000","last_undegraded":"2024-01-15T18:17:39.210181+0000","last_fullsized":"2024-01-15T18:17:39.210181+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:36:29.536846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663556,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"114'8","reported_seq":24,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.285243+0000","last_change":"2024-01-15T18:12:10.022690+0000","last_active":"2024-01-15T18:14:32.285243+0000","last_peered":"2024-01-15T18:14:32.285243+0000","last_clean":"2024-01-15T18:14:32.285243+0000","last_became_active":"2024-01-15T18:12:10.022398+0000","last_became_peered":"2024-01-15T18:12:10.022398+0000","last_unstale":"2024-01-15T18:14:32.285243+0000","last_undegraded":"2024-01-15T18:14:32.285243+0000","last_fullsized":"2024-01-15T18:14:32.285243+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:03:52.527066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"117'24","reported_seq":43,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.192858+0000","last_change":"2024-01-15T18:12:11.045910+0000","last_active":"2024-01-15T18:17:39.192858+0000","last_peered":"2024-01-15T18:17:39.192858+0000","last_clean":"2024-01-15T18:17:39.192858+0000","last_became_active":"2024-01-15T18:12:11.043735+0000","last_became_peered":"2024-01-15T18:12:11.043735+0000","last_unstale":"2024-01-15T18:17:39.192858+0000","last_undegraded":"2024-01-15T18:17:39.192858+0000","last_fullsized":"2024-01-15T18:17:39.192858+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.728315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.424881+0000","last_change":"2024-01-15T18:12:10.027277+0000","last_active":"2024-01-15T18:14:27.424881+0000","last_peered":"2024-01-15T18:14:27.424881+0000","last_clean":"2024-01-15T18:14:27.424881+0000","last_became_active":"2024-01-15T18:12:10.026730+0000","last_became_peered":"2024-01-15T18:12:10.026730+0000","last_unstale":"2024-01-15T18:14:27.424881+0000","last_undegraded":"2024-01-15T18:14:27.424881+0000","last_fullsized":"2024-01-15T18:14:27.424881+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:52.737012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"117'36","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.289041+0000","last_change":"2024-01-15T18:12:11.061270+0000","last_active":"2024-01-15T18:17:39.289041+0000","last_peered":"2024-01-15T18:17:39.289041+0000","last_clean":"2024-01-15T18:17:39.289041+0000","last_became_active":"2024-01-15T18:12:11.061002+0000","last_became_peered":"2024-01-15T18:12:11.061002+0000","last_unstale":"2024-01-15T18:17:39.289041+0000","last_undegraded":"2024-01-15T18:17:39.289041+0000","last_fullsized":"2024-01-15T18:17:39.289041+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:10.142975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.284329+0000","last_change":"2024-01-15T18:12:10.020526+0000","last_active":"2024-01-15T18:14:27.284329+0000","last_peered":"2024-01-15T18:14:27.284329+0000","last_clean":"2024-01-15T18:14:27.284329+0000","last_became_active":"2024-01-15T18:12:10.019706+0000","last_became_peered":"2024-01-15T18:12:10.019706+0000","last_unstale":"2024-01-15T18:14:27.284329+0000","last_undegraded":"2024-01-15T18:14:27.284329+0000","last_fullsized":"2024-01-15T18:14:27.284329+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:24:49.230722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"117'44","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.211608+0000","last_change":"2024-01-15T18:12:11.060480+0000","last_active":"2024-01-15T18:17:39.211608+0000","last_peered":"2024-01-15T18:17:39.211608+0000","last_clean":"2024-01-15T18:17:39.211608+0000","last_became_active":"2024-01-15T18:12:11.060154+0000","last_became_peered":"2024-01-15T18:12:11.060154+0000","last_unstale":"2024-01-15T18:17:39.211608+0000","last_undegraded":"2024-01-15T18:17:39.211608+0000","last_fullsized":"2024-01-15T18:17:39.211608+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:52.364460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.542920+0000","last_change":"2024-01-15T18:12:10.003308+0000","last_active":"2024-01-15T18:14:27.542920+0000","last_peered":"2024-01-15T18:14:27.542920+0000","last_clean":"2024-01-15T18:14:27.542920+0000","last_became_active":"2024-01-15T18:12:10.002957+0000","last_became_peered":"2024-01-15T18:12:10.002957+0000","last_unstale":"2024-01-15T18:14:27.542920+0000","last_undegraded":"2024-01-15T18:14:27.542920+0000","last_fullsized":"2024-01-15T18:14:27.542920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:43.777497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"117'41","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.319156+0000","last_change":"2024-01-15T18:12:11.059234+0000","last_active":"2024-01-15T18:17:39.319156+0000","last_peered":"2024-01-15T18:17:39.319156+0000","last_clean":"2024-01-15T18:17:39.319156+0000","last_became_active":"2024-01-15T18:12:11.058945+0000","last_became_peered":"2024-01-15T18:12:11.058945+0000","last_unstale":"2024-01-15T18:17:39.319156+0000","last_undegraded":"2024-01-15T18:17:39.319156+0000","last_fullsized":"2024-01-15T18:17:39.319156+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:37.113206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.411059+0000","last_change":"2024-01-15T18:12:10.028589+0000","last_active":"2024-01-15T18:14:27.411059+0000","last_peered":"2024-01-15T18:14:27.411059+0000","last_clean":"2024-01-15T18:14:27.411059+0000","last_became_active":"2024-01-15T18:12:10.028284+0000","last_became_peered":"2024-01-15T18:12:10.028284+0000","last_unstale":"2024-01-15T18:14:27.411059+0000","last_undegraded":"2024-01-15T18:14:27.411059+0000","last_fullsized":"2024-01-15T18:14:27.411059+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:46:23.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"117'45","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.543742+0000","last_change":"2024-01-15T18:12:11.051787+0000","last_active":"2024-01-15T18:17:39.543742+0000","last_peered":"2024-01-15T18:17:39.543742+0000","last_clean":"2024-01-15T18:17:39.543742+0000","last_became_active":"2024-01-15T18:12:11.051403+0000","last_became_peered":"2024-01-15T18:12:11.051403+0000","last_unstale":"2024-01-15T18:17:39.543742+0000","last_undegraded":"2024-01-15T18:17:39.543742+0000","last_fullsized":"2024-01-15T18:17:39.543742+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:05:35.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"117'3","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.284165+0000","last_change":"2024-01-15T18:12:10.021913+0000","last_active":"2024-01-15T18:14:27.284165+0000","last_peered":"2024-01-15T18:14:27.284165+0000","last_clean":"2024-01-15T18:14:27.284165+0000","last_became_active":"2024-01-15T18:12:10.021344+0000","last_became_peered":"2024-01-15T18:12:10.021344+0000","last_unstale":"2024-01-15T18:14:27.284165+0000","last_undegraded":"2024-01-15T18:14:27.284165+0000","last_fullsized":"2024-01-15T18:14:27.284165+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:00.107213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"117'43","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.906847+0000","last_change":"2024-01-15T18:12:11.059848+0000","last_active":"2024-01-15T18:17:38.906847+0000","last_peered":"2024-01-15T18:17:38.906847+0000","last_clean":"2024-01-15T18:17:38.906847+0000","last_became_active":"2024-01-15T18:12:11.059311+0000","last_became_peered":"2024-01-15T18:12:11.059311+0000","last_unstale":"2024-01-15T18:17:38.906847+0000","last_undegraded":"2024-01-15T18:17:38.906847+0000","last_fullsized":"2024-01-15T18:17:38.906847+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:47:13.715785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.283095+0000","last_change":"2024-01-15T18:12:10.018301+0000","last_active":"2024-01-15T18:14:22.283095+0000","last_peered":"2024-01-15T18:14:22.283095+0000","last_clean":"2024-01-15T18:14:22.283095+0000","last_became_active":"2024-01-15T18:12:10.017411+0000","last_became_peered":"2024-01-15T18:12:10.017411+0000","last_unstale":"2024-01-15T18:14:22.283095+0000","last_undegraded":"2024-01-15T18:14:22.283095+0000","last_fullsized":"2024-01-15T18:14:22.283095+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:26:32.688781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.231615+0000","last_change":"2024-01-15T18:12:11.056765+0000","last_active":"2024-01-15T18:17:39.231615+0000","last_peered":"2024-01-15T18:17:39.231615+0000","last_clean":"2024-01-15T18:17:39.231615+0000","last_became_active":"2024-01-15T18:12:11.056311+0000","last_became_peered":"2024-01-15T18:12:11.056311+0000","last_unstale":"2024-01-15T18:17:39.231615+0000","last_undegraded":"2024-01-15T18:17:39.231615+0000","last_fullsized":"2024-01-15T18:17:39.231615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:53.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.282960+0000","last_change":"2024-01-15T18:12:10.023638+0000","last_active":"2024-01-15T18:14:22.282960+0000","last_peered":"2024-01-15T18:14:22.282960+0000","last_clean":"2024-01-15T18:14:22.282960+0000","last_became_active":"2024-01-15T18:12:10.022687+0000","last_became_peered":"2024-01-15T18:12:10.022687+0000","last_unstale":"2024-01-15T18:14:22.282960+0000","last_undegraded":"2024-01-15T18:14:22.282960+0000","last_fullsized":"2024-01-15T18:14:22.282960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:16:54.875991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"117'42","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.425793+0000","last_change":"2024-01-15T18:12:11.062171+0000","last_active":"2024-01-15T18:17:39.425793+0000","last_peered":"2024-01-15T18:17:39.425793+0000","last_clean":"2024-01-15T18:17:39.425793+0000","last_became_active":"2024-01-15T18:12:11.061830+0000","last_became_peered":"2024-01-15T18:12:11.061830+0000","last_unstale":"2024-01-15T18:17:39.425793+0000","last_undegraded":"2024-01-15T18:17:39.425793+0000","last_fullsized":"2024-01-15T18:17:39.425793+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:07:50.541819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109124,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.423571+0000","last_change":"2024-01-15T18:12:10.024893+0000","last_active":"2024-01-15T18:14:22.423571+0000","last_peered":"2024-01-15T18:14:22.423571+0000","last_clean":"2024-01-15T18:14:22.423571+0000","last_became_active":"2024-01-15T18:12:10.024554+0000","last_became_peered":"2024-01-15T18:12:10.024554+0000","last_unstale":"2024-01-15T18:14:22.423571+0000","last_undegraded":"2024-01-15T18:14:22.423571+0000","last_fullsized":"2024-01-15T18:14:22.423571+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:51.350660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"117'36","reported_seq":55,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.210156+0000","last_change":"2024-01-15T18:12:11.051179+0000","last_active":"2024-01-15T18:17:39.210156+0000","last_peered":"2024-01-15T18:17:39.210156+0000","last_clean":"2024-01-15T18:17:39.210156+0000","last_became_active":"2024-01-15T18:12:11.050747+0000","last_became_peered":"2024-01-15T18:12:11.050747+0000","last_unstale":"2024-01-15T18:17:39.210156+0000","last_undegraded":"2024-01-15T18:17:39.210156+0000","last_fullsized":"2024-01-15T18:17:39.210156+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"114'2","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:22.305559+0000","last_change":"2024-01-15T18:12:10.025674+0000","last_active":"2024-01-15T18:14:22.305559+0000","last_peered":"2024-01-15T18:14:22.305559+0000","last_clean":"2024-01-15T18:14:22.305559+0000","last_became_active":"2024-01-15T18:12:10.025356+0000","last_became_peered":"2024-01-15T18:12:10.025356+0000","last_unstale":"2024-01-15T18:14:22.305559+0000","last_undegraded":"2024-01-15T18:14:22.305559+0000","last_fullsized":"2024-01-15T18:14:22.305559+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:06:50.285646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"117'41","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.010642+0000","last_change":"2024-01-15T18:12:11.050465+0000","last_active":"2024-01-15T18:17:39.010642+0000","last_peered":"2024-01-15T18:17:39.010642+0000","last_clean":"2024-01-15T18:17:39.010642+0000","last_became_active":"2024-01-15T18:12:11.050116+0000","last_became_peered":"2024-01-15T18:12:11.050116+0000","last_unstale":"2024-01-15T18:17:39.010642+0000","last_undegraded":"2024-01-15T18:17:39.010642+0000","last_fullsized":"2024-01-15T18:17:39.010642+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:45:04.891353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"117'17","reported_seq":33,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:55.208163+0000","last_change":"2024-01-15T18:12:10.020848+0000","last_active":"2024-01-15T18:16:55.208163+0000","last_peered":"2024-01-15T18:16:55.208163+0000","last_clean":"2024-01-15T18:16:55.208163+0000","last_became_active":"2024-01-15T18:12:10.020497+0000","last_became_peered":"2024-01-15T18:12:10.020497+0000","last_unstale":"2024-01-15T18:16:55.208163+0000","last_undegraded":"2024-01-15T18:16:55.208163+0000","last_fullsized":"2024-01-15T18:16:55.208163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:37.798936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"117'46","reported_seq":75,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.193190+0000","last_change":"2024-01-15T18:12:11.051721+0000","last_active":"2024-01-15T18:17:39.193190+0000","last_peered":"2024-01-15T18:17:39.193190+0000","last_clean":"2024-01-15T18:17:39.193190+0000","last_became_active":"2024-01-15T18:12:11.050810+0000","last_became_peered":"2024-01-15T18:12:11.050810+0000","last_unstale":"2024-01-15T18:17:39.193190+0000","last_undegraded":"2024-01-15T18:17:39.193190+0000","last_fullsized":"2024-01-15T18:17:39.193190+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:34:44.264660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.422714+0000","last_change":"2024-01-15T18:12:10.025006+0000","last_active":"2024-01-15T18:14:17.422714+0000","last_peered":"2024-01-15T18:14:17.422714+0000","last_clean":"2024-01-15T18:14:17.422714+0000","last_became_active":"2024-01-15T18:12:10.024524+0000","last_became_peered":"2024-01-15T18:12:10.024524+0000","last_unstale":"2024-01-15T18:14:17.422714+0000","last_undegraded":"2024-01-15T18:14:17.422714+0000","last_fullsized":"2024-01-15T18:14:17.422714+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:16.499218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"117'39","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.572497+0000","last_change":"2024-01-15T18:12:11.061751+0000","last_active":"2024-01-15T18:17:38.572497+0000","last_peered":"2024-01-15T18:17:38.572497+0000","last_clean":"2024-01-15T18:17:38.572497+0000","last_became_active":"2024-01-15T18:12:11.061454+0000","last_became_peered":"2024-01-15T18:12:11.061454+0000","last_unstale":"2024-01-15T18:17:38.572497+0000","last_undegraded":"2024-01-15T18:17:38.572497+0000","last_fullsized":"2024-01-15T18:17:38.572497+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.304508+0000","last_change":"2024-01-15T18:12:10.018428+0000","last_active":"2024-01-15T18:14:17.304508+0000","last_peered":"2024-01-15T18:14:17.304508+0000","last_clean":"2024-01-15T18:14:17.304508+0000","last_became_active":"2024-01-15T18:12:10.018057+0000","last_became_peered":"2024-01-15T18:12:10.018057+0000","last_unstale":"2024-01-15T18:14:17.304508+0000","last_undegraded":"2024-01-15T18:14:17.304508+0000","last_fullsized":"2024-01-15T18:14:17.304508+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:52:09.544387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"117'44","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.352556+0000","last_change":"2024-01-15T18:12:11.039995+0000","last_active":"2024-01-15T18:17:39.352556+0000","last_peered":"2024-01-15T18:17:39.352556+0000","last_clean":"2024-01-15T18:17:39.352556+0000","last_became_active":"2024-01-15T18:12:11.039586+0000","last_became_peered":"2024-01-15T18:12:11.039586+0000","last_unstale":"2024-01-15T18:17:39.352556+0000","last_undegraded":"2024-01-15T18:17:39.352556+0000","last_fullsized":"2024-01-15T18:17:39.352556+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:38.812736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.408970+0000","last_change":"2024-01-15T18:12:10.019758+0000","last_active":"2024-01-15T18:14:17.408970+0000","last_peered":"2024-01-15T18:14:17.408970+0000","last_clean":"2024-01-15T18:14:17.408970+0000","last_became_active":"2024-01-15T18:12:10.019424+0000","last_became_peered":"2024-01-15T18:12:10.019424+0000","last_unstale":"2024-01-15T18:14:17.408970+0000","last_undegraded":"2024-01-15T18:14:17.408970+0000","last_fullsized":"2024-01-15T18:14:17.408970+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:45.797791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"117'56","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.420858+0000","last_change":"2024-01-15T18:12:11.035857+0000","last_active":"2024-01-15T18:17:39.420858+0000","last_peered":"2024-01-15T18:17:39.420858+0000","last_clean":"2024-01-15T18:17:39.420858+0000","last_became_active":"2024-01-15T18:12:11.035238+0000","last_became_peered":"2024-01-15T18:12:11.035238+0000","last_unstale":"2024-01-15T18:17:39.420858+0000","last_undegraded":"2024-01-15T18:17:39.420858+0000","last_fullsized":"2024-01-15T18:17:39.420858+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.326548+0000","last_change":"2024-01-15T18:12:10.017511+0000","last_active":"2024-01-15T18:14:17.326548+0000","last_peered":"2024-01-15T18:14:17.326548+0000","last_clean":"2024-01-15T18:14:17.326548+0000","last_became_active":"2024-01-15T18:12:10.017190+0000","last_became_peered":"2024-01-15T18:12:10.017190+0000","last_unstale":"2024-01-15T18:14:17.326548+0000","last_undegraded":"2024-01-15T18:14:17.326548+0000","last_fullsized":"2024-01-15T18:14:17.326548+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"117'46","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.384006+0000","last_change":"2024-01-15T18:12:11.056667+0000","last_active":"2024-01-15T18:17:39.384006+0000","last_peered":"2024-01-15T18:17:39.384006+0000","last_clean":"2024-01-15T18:17:39.384006+0000","last_became_active":"2024-01-15T18:12:11.056111+0000","last_became_peered":"2024-01-15T18:12:11.056111+0000","last_unstale":"2024-01-15T18:17:39.384006+0000","last_undegraded":"2024-01-15T18:17:39.384006+0000","last_fullsized":"2024-01-15T18:17:39.384006+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:18:00.144603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"117'10","reported_seq":35,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:33.616962+0000","last_change":"2024-01-15T18:12:10.021210+0000","last_active":"2024-01-15T18:15:33.616962+0000","last_peered":"2024-01-15T18:15:33.616962+0000","last_clean":"2024-01-15T18:15:33.616962+0000","last_became_active":"2024-01-15T18:12:10.020890+0000","last_became_peered":"2024-01-15T18:12:10.020890+0000","last_unstale":"2024-01-15T18:15:33.616962+0000","last_undegraded":"2024-01-15T18:15:33.616962+0000","last_fullsized":"2024-01-15T18:15:33.616962+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:44.292512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"117'39","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.353330+0000","last_change":"2024-01-15T18:12:11.059132+0000","last_active":"2024-01-15T18:17:39.353330+0000","last_peered":"2024-01-15T18:17:39.353330+0000","last_clean":"2024-01-15T18:17:39.353330+0000","last_became_active":"2024-01-15T18:12:11.057943+0000","last_became_peered":"2024-01-15T18:12:11.057943+0000","last_unstale":"2024-01-15T18:17:39.353330+0000","last_undegraded":"2024-01-15T18:17:39.353330+0000","last_fullsized":"2024-01-15T18:17:39.353330+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:54:52.248678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:12.421404+0000","last_change":"2024-01-15T18:12:10.010681+0000","last_active":"2024-01-15T18:14:12.421404+0000","last_peered":"2024-01-15T18:14:12.421404+0000","last_clean":"2024-01-15T18:14:12.421404+0000","last_became_active":"2024-01-15T18:12:10.009870+0000","last_became_peered":"2024-01-15T18:12:10.009870+0000","last_unstale":"2024-01-15T18:14:12.421404+0000","last_undegraded":"2024-01-15T18:14:12.421404+0000","last_fullsized":"2024-01-15T18:14:12.421404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:21.835760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"117'31","reported_seq":54,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.239562+0000","last_change":"2024-01-15T18:12:11.051496+0000","last_active":"2024-01-15T18:17:39.239562+0000","last_peered":"2024-01-15T18:17:39.239562+0000","last_clean":"2024-01-15T18:17:39.239562+0000","last_became_active":"2024-01-15T18:12:11.051092+0000","last_became_peered":"2024-01-15T18:12:11.051092+0000","last_unstale":"2024-01-15T18:17:39.239562+0000","last_undegraded":"2024-01-15T18:17:39.239562+0000","last_fullsized":"2024-01-15T18:17:39.239562+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:38:09.219058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.412208+0000","last_change":"2024-01-15T18:12:10.026923+0000","last_active":"2024-01-15T18:14:32.412208+0000","last_peered":"2024-01-15T18:14:32.412208+0000","last_clean":"2024-01-15T18:14:32.412208+0000","last_became_active":"2024-01-15T18:12:10.026651+0000","last_became_peered":"2024-01-15T18:12:10.026651+0000","last_unstale":"2024-01-15T18:14:32.412208+0000","last_undegraded":"2024-01-15T18:14:32.412208+0000","last_fullsized":"2024-01-15T18:14:32.412208+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:14.981958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":251,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:02.417603+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:15:02.417603+0000","last_peered":"2024-01-15T18:15:02.417603+0000","last_clean":"2024-01-15T18:15:02.417603+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:15:02.417603+0000","last_undegraded":"2024-01-15T18:15:02.417603+0000","last_fullsized":"2024-01-15T18:15:02.417603+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710133,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2810,"ondisk_log_size":2810,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":634396,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":274,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1335296,"data_stored":1271696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":254,"ondisk_log_size":254,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542421,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834588,"kb_used_data":1767380,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537252,"statfs":{"total":96636764160,"available":94758146048,"internally_reserved":0,"allocated":1809797120,"data_stored":1808970858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.254,"15min":1.383},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.40400000000000003},"max":{"1min":1.599,"5min":4.0229999999999997,"15min":19.843},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.49,"15min":1.5609999999999999},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.53400000000000003},"max":{"1min":1.6319999999999999,"5min":16.678999999999998,"15min":30.498000000000001},"last":1.552}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2649999999999999,"15min":1.397},"min":{"1min":0.91400000000000003,"5min":0.56599999999999995,"15min":0.45600000000000002},"max":{"1min":1.5700000000000001,"5min":3.9430000000000001,"15min":19.917000000000002},"last":1.25},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.54,"15min":1.5669999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.71199999999999997},"max":{"1min":1.6319999999999999,"5min":16.626999999999999,"15min":28.613},"last":1.196}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2689999999999999,"15min":1.391},"min":{"1min":0.98399999999999999,"5min":0.75600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6479999999999999,"5min":1.8140000000000001,"15min":12.061},"last":1.752},{"interface":"front","average":{"1min":1.355,"5min":1.528,"15min":1.5800000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.48999999999999999},"max":{"1min":1.77,"5min":16.792000000000002,"15min":16.792000000000002},"last":1.4139999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.306,"15min":1.5229999999999999},"min":{"1min":1.0029999999999999,"5min":0.80700000000000005,"15min":0.56000000000000005},"max":{"1min":1.6180000000000001,"5min":1.734,"15min":16.632000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.619,"15min":1.639},"min":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.56699999999999995},"max":{"1min":1.6930000000000001,"5min":16.129000000000001,"15min":16.129000000000001},"last":1.6100000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.571,"15min":1.627},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.54200000000000004},"max":{"1min":1.585,"5min":16.454000000000001,"15min":23.042000000000002},"last":1.657},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.357,"15min":1.677},"min":{"1min":0.873,"5min":0.80000000000000004,"15min":0.496},"max":{"1min":1.7410000000000001,"5min":1.885,"15min":38.682000000000002},"last":1.8460000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4259999999999999,"15min":1.5860000000000001},"min":{"1min":0.96799999999999997,"5min":0.76800000000000002,"15min":0.52700000000000002},"max":{"1min":1.867,"5min":4.5359999999999996,"15min":13.379},"last":1.679},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.71,"15min":1.7430000000000001},"min":{"1min":1.0649999999999999,"5min":0.98499999999999999,"15min":0.624},"max":{"1min":1.843,"5min":16.553000000000001,"15min":22.933},"last":1.706}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.7969999999999999,"15min":1.8600000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":0.57399999999999995},"max":{"1min":1.8140000000000001,"5min":16.725999999999999,"15min":23.004000000000001},"last":1.804},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4850000000000001,"15min":1.659},"min":{"1min":1.1220000000000001,"5min":1.008,"15min":0.58299999999999996},"max":{"1min":1.897,"5min":1.952,"15min":9.2599999999999998},"last":1.9550000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542420,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1256408,"kb_used_data":1193360,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93115432,"statfs":{"total":96636764160,"available":95350202368,"internally_reserved":0,"allocated":1222000640,"data_stored":1221177506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2629999999999999,"15min":1.464},"min":{"1min":0.79600000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":31.805},"last":1.361},{"interface":"front","average":{"1min":1.244,"5min":1.3,"15min":1.4870000000000001},"min":{"1min":0.749,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8340000000000001,"5min":3.5409999999999999,"15min":31.702999999999999},"last":1.411}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1339999999999999,"15min":1.2609999999999999},"min":{"1min":0.69699999999999995,"5min":0.59999999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.778,"15min":15.064},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2050000000000001,"15min":1.4299999999999999},"min":{"1min":0.86499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6279999999999999,"5min":1.9650000000000001,"15min":23.495999999999999},"last":0.92400000000000004}]},{"osd":2,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2849999999999999,"15min":1.5349999999999999},"min":{"1min":0.81699999999999995,"5min":0.79100000000000004,"15min":0.59399999999999997},"max":{"1min":1.502,"5min":1.8899999999999999,"15min":23.087},"last":1.081},{"interface":"front","average":{"1min":1.339,"5min":1.331,"15min":1.397},"min":{"1min":1.0089999999999999,"5min":0.77000000000000002,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.9990000000000001,"15min":14.94},"last":1.5309999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4359999999999999,"15min":1.6040000000000001},"min":{"1min":1.0620000000000001,"5min":0.65200000000000002,"15min":0.59899999999999998},"max":{"1min":1.7,"5min":1.8160000000000001,"15min":15.151999999999999},"last":1.698},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3779999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6080000000000001,"5min":3.8500000000000001,"15min":13.795},"last":1.573}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2729999999999999,"15min":1.585},"min":{"1min":0.90900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.53,"5min":3.3719999999999999,"15min":23.327000000000002},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.3560000000000001,"15min":1.548},"min":{"1min":0.80900000000000005,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.6839999999999999,"5min":1.8839999999999999,"15min":15.239000000000001},"last":1.1950000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.331,"15min":1.502},"min":{"1min":0.88700000000000001,"5min":0.58099999999999996,"15min":0.52500000000000002},"max":{"1min":1.7210000000000001,"5min":3.6480000000000001,"15min":13.593},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.55,"15min":1.8049999999999999},"min":{"1min":1.1930000000000001,"5min":0.66200000000000003,"15min":0.621},"max":{"1min":1.702,"5min":2.306,"15min":23.417999999999999},"last":1.595}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.47,"15min":1.716},"min":{"1min":0.98799999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8220000000000001,"5min":3.105,"15min":21.265000000000001},"last":1.252},{"interface":"front","average":{"1min":1.486,"5min":1.542,"15min":1.899},"min":{"1min":1.1200000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.7789999999999999,"5min":2.2229999999999999,"15min":23.738},"last":1.4790000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542421,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908440,"kb_used_data":832912,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93463400,"statfs":{"total":96636764160,"available":95706521600,"internally_reserved":0,"allocated":852901888,"data_stored":852079166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.202,"15min":1.2669999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.47799999999999998},"max":{"1min":1.595,"5min":2.7919999999999998,"15min":19.266999999999999},"last":1.087},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.752,"5min":0.748,"15min":0.51400000000000001},"max":{"1min":1.671,"5min":1.7529999999999999,"15min":3.1760000000000002},"last":0.997}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.359,"15min":1.3149999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.60699999999999998},"max":{"1min":1.653,"5min":15.554,"15min":15.554},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.242,"5min":1.302,"15min":1.3540000000000001},"min":{"1min":0.80100000000000005,"5min":0.69599999999999995,"15min":0.59099999999999997},"max":{"1min":1.643,"5min":6.1449999999999996,"15min":9.2379999999999995},"last":1.464}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.621},"max":{"1min":1.694,"5min":2.3500000000000001,"15min":4.0369999999999999},"last":1.323},{"interface":"front","average":{"1min":1.349,"5min":1.512,"15min":1.4299999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.56000000000000005},"max":{"1min":1.617,"5min":15.468999999999999,"15min":15.468999999999999},"last":1.0369999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.327,"15min":1.409},"min":{"1min":0.78200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.718,"5min":1.984,"15min":9.1929999999999996},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.357,"5min":1.4379999999999999,"15min":1.4650000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.64600000000000002},"max":{"1min":1.774,"5min":2.2949999999999999,"15min":7.0110000000000001},"last":1.4359999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.4319999999999999,"15min":1.47},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.41399999999999998},"max":{"1min":1.6040000000000001,"5min":15.254,"15min":15.254},"last":1.7529999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.4019999999999999,"15min":1.4710000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.8080000000000001,"5min":1.923,"15min":9.1519999999999992},"last":1.7110000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.456},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.39600000000000002},"max":{"1min":1.8480000000000001,"5min":2.2250000000000001,"15min":3.7559999999999998},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.593,"15min":1.5680000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53000000000000003},"max":{"1min":1.7330000000000001,"5min":15.366,"15min":15.366},"last":1.9470000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6100000000000001},"min":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.45800000000000002},"max":{"1min":1.873,"5min":2.0609999999999999,"15min":9.0600000000000005},"last":1.9059999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.5840000000000001,"15min":1.6890000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.57299999999999995},"max":{"1min":1.899,"5min":3.1659999999999999,"15min":19.356999999999999},"last":1.6870000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1711124,"kb_used_data":1643916,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92660716,"statfs":{"total":96636764160,"available":94884573184,"internally_reserved":0,"allocated":1683369984,"data_stored":1682551014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.264,"15min":1.3360000000000001},"min":{"1min":0.94799999999999995,"5min":0.70199999999999996,"15min":0.39200000000000002},"max":{"1min":1.601,"5min":2.004,"15min":17.346},"last":1.141},{"interface":"front","average":{"1min":1.409,"5min":1.3500000000000001,"15min":1.361},"min":{"1min":1.0900000000000001,"5min":0.97499999999999998,"15min":0.40100000000000002},"max":{"1min":1.744,"5min":1.804,"15min":7.9160000000000004},"last":1.333}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.1950000000000001,"15min":1.24},"min":{"1min":0.95199999999999996,"5min":0.76600000000000001,"15min":0.439},"max":{"1min":1.6699999999999999,"5min":1.861,"15min":11.128},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.23,"15min":1.234},"min":{"1min":0.876,"5min":0.82999999999999996,"15min":0.60799999999999998},"max":{"1min":1.619,"5min":1.7470000000000001,"15min":5.6109999999999998},"last":1.0920000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.294,"15min":1.343},"min":{"1min":0.92300000000000004,"5min":0.83399999999999996,"15min":0.56899999999999995},"max":{"1min":1.544,"5min":1.8560000000000001,"15min":18.442},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.341,"5min":1.3720000000000001,"15min":1.399},"min":{"1min":1.1459999999999999,"5min":0.94799999999999995,"15min":0.81899999999999995},"max":{"1min":1.669,"5min":1.861,"15min":5.7050000000000001},"last":0.96199999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.375,"15min":1.462},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.56699999999999995},"max":{"1min":1.6719999999999999,"5min":2.1859999999999999,"15min":17.364999999999998},"last":1.012},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3380000000000001,"15min":1.3899999999999999},"min":{"1min":1.022,"5min":0.85099999999999998,"15min":0.45300000000000001},"max":{"1min":1.601,"5min":1.7050000000000001,"15min":9.3320000000000007},"last":1.04}]},{"osd":5,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.29,"15min":1.4590000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.49099999999999999},"max":{"1min":1.6519999999999999,"5min":2.4089999999999998,"15min":29.422000000000001},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3420000000000001,"15min":1.4379999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.53700000000000003},"max":{"1min":1.7549999999999999,"5min":1.9450000000000001,"15min":12.332000000000001},"last":1.462}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2629999999999999,"15min":1.3899999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.54200000000000004},"max":{"1min":1.752,"5min":2.117,"15min":11.09},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.482,"15min":1.6120000000000001},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":0.51100000000000001},"max":{"1min":1.718,"5min":2.2639999999999998,"15min":17.224},"last":1.3879999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.504,"15min":1.593},"min":{"1min":1.0169999999999999,"5min":0.93300000000000005,"15min":0.60499999999999998},"max":{"1min":1.8400000000000001,"5min":2.052,"15min":12.247},"last":1.585},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6439999999999999,"15min":1.73},"min":{"1min":1.294,"5min":1.159,"15min":0.61399999999999999},"max":{"1min":1.9039999999999999,"5min":2.3570000000000002,"15min":17.324999999999999},"last":1.508}]}]},{"osd":3,"up_from":14,"seq":60129542423,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462032,"kb_used_data":1386504,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":92909808,"statfs":{"total":96636764160,"available":95139643392,"internally_reserved":0,"allocated":1419780096,"data_stored":1418945373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0349999999999999,"15min":1.1639999999999999},"min":{"1min":0.66400000000000003,"5min":0.52800000000000002,"15min":0.33300000000000002},"max":{"1min":1.7989999999999999,"5min":2.915,"15min":18.399999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.258,"5min":1.2509999999999999,"15min":1.2849999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.44400000000000001},"max":{"1min":1.8560000000000001,"5min":3.028,"15min":18.795999999999999},"last":1.395}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.05,"15min":1.1299999999999999},"min":{"1min":0.65800000000000003,"5min":0.51200000000000001,"15min":0.32600000000000001},"max":{"1min":1.4870000000000001,"5min":5.3520000000000003,"15min":12.372},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.085,"5min":1.1180000000000001,"15min":1.2589999999999999},"min":{"1min":0.66000000000000003,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.736,"5min":4.125,"15min":11.228999999999999},"last":1}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.218,"15min":1.2529999999999999},"min":{"1min":0.74199999999999999,"5min":0.59999999999999998,"15min":0.375},"max":{"1min":1.8680000000000001,"5min":2.3929999999999998,"15min":10.964},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.45,"15min":1.534},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7569999999999999,"5min":3.0659999999999998,"15min":20.495000000000001},"last":1.476}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.389,"15min":1.51},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.67200000000000004},"max":{"1min":1.865,"5min":2.1139999999999999,"15min":13.536},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4530000000000001,"15min":1.5529999999999999},"min":{"1min":0.98999999999999999,"5min":0.83099999999999996,"15min":0.63700000000000001},"max":{"1min":1.905,"5min":6.2439999999999998,"15min":16.398},"last":1.5149999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.478,"15min":1.6519999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.67200000000000004},"max":{"1min":2.0670000000000002,"5min":2.7269999999999999,"15min":20.606000000000002},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.6060000000000001,"15min":1.631},"min":{"1min":1.0269999999999999,"5min":0.94299999999999995,"15min":0.66800000000000004},"max":{"1min":2.0489999999999999,"5min":2.617,"15min":11.603},"last":1.6739999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.554,"15min":1.675},"min":{"1min":0.93100000000000005,"5min":0.85499999999999998,"15min":0.53500000000000003},"max":{"1min":2.1230000000000002,"5min":6.3289999999999997,"15min":13.648999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.554,"5min":1.6100000000000001,"15min":1.7529999999999999},"min":{"1min":1.2190000000000001,"5min":1.0249999999999999,"15min":0.65800000000000003},"max":{"1min":1.849,"5min":6.367,"15min":20.556000000000001},"last":1.3660000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.641,"15min":1.76},"min":{"1min":1.159,"5min":0.97899999999999998,"15min":0.60499999999999998},"max":{"1min":2.1269999999999998,"5min":4.8630000000000004,"15min":28.843},"last":1.823},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6559999999999999,"15min":1.738},"min":{"1min":1.1830000000000001,"5min":0.80100000000000005,"15min":0.67200000000000004},"max":{"1min":2.0680000000000001,"5min":6.9569999999999999,"15min":15.143000000000001},"last":1.3260000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542423,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":982100,"kb_used_data":910732,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93389740,"statfs":{"total":96636764160,"available":95631093760,"internally_reserved":0,"allocated":932589568,"data_stored":931771659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.161,"15min":1.083},"min":{"1min":0.67600000000000005,"5min":0.51000000000000001,"15min":0.26600000000000001},"max":{"1min":1.419,"5min":22.707000000000001,"15min":22.707000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1970000000000001,"15min":1.357},"min":{"1min":0.76300000000000001,"5min":0.61099999999999999,"15min":0.39300000000000002},"max":{"1min":1.6879999999999999,"5min":2.5710000000000002,"15min":15.314},"last":0.80500000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1759999999999999,"15min":1.3240000000000001},"min":{"1min":0.73299999999999998,"5min":0.58599999999999997,"15min":0.38300000000000001},"max":{"1min":1.546,"5min":2.8679999999999999,"15min":34.463999999999999},"last":1.111},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.327,"15min":1.504},"min":{"1min":0.90700000000000003,"5min":0.73899999999999999,"15min":0.373},"max":{"1min":1.675,"5min":2.794,"15min":25.859999999999999},"last":0.96999999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1870000000000001,"15min":1.302},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.34399999999999997},"max":{"1min":1.464,"5min":2.7080000000000002,"15min":12.93},"last":1.026},{"interface":"front","average":{"1min":1.284,"5min":1.6919999999999999,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.46400000000000002},"max":{"1min":1.903,"5min":37.021999999999998,"15min":37.021999999999998},"last":1.548}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6879999999999999},"min":{"1min":1.0289999999999999,"5min":0.96999999999999997,"15min":0.55700000000000005},"max":{"1min":1.78,"5min":3.419,"15min":29.986999999999998},"last":1.395},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.518,"15min":1.7150000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.56799999999999995},"max":{"1min":1.883,"5min":2.976,"15min":21.199999999999999},"last":1.3660000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.456,"15min":1.6499999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.51500000000000001},"max":{"1min":2.0880000000000001,"5min":2.218,"15min":14.066000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.643,"15min":1.7849999999999999},"min":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":0.60899999999999999},"max":{"1min":1.8180000000000001,"5min":3.4790000000000001,"15min":14.025},"last":1.621}]},{"osd":6,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.6160000000000001,"15min":1.8069999999999999},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":0.68000000000000005},"max":{"1min":1.9910000000000001,"5min":3.0800000000000001,"15min":15.683999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.409,"5min":1.585,"15min":1.784},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.58899999999999997},"max":{"1min":2.2930000000000001,"5min":10.371,"15min":12.629},"last":1.3200000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.671,"15min":2.1419999999999999},"min":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":0.63100000000000001},"max":{"1min":2.2210000000000001,"5min":3.177,"15min":57.006999999999998},"last":1.492},{"interface":"front","average":{"1min":1.595,"5min":1.6299999999999999,"15min":1.9630000000000001},"min":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":0.56100000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":22.853999999999999},"last":1.5129999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542422,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1137764,"kb_used_data":1066396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93234076,"statfs":{"total":96636764160,"available":95471693824,"internally_reserved":0,"allocated":1091989504,"data_stored":1091155706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.96499999999999997,"15min":1.177},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.34399999999999997},"max":{"1min":1.1419999999999999,"5min":2.7389999999999999,"15min":30.460999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.115,"15min":1.26},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.314},"max":{"1min":1.6539999999999999,"5min":5.4779999999999998,"15min":18.658000000000001},"last":0.85099999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.075,"15min":1.1579999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.5589999999999999,"5min":1.7809999999999999,"15min":7.117},"last":1.46},{"interface":"front","average":{"1min":1.099,"5min":1.325,"15min":1.5129999999999999},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.728,"5min":1.98,"15min":30.593},"last":1.3919999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2529999999999999,"15min":1.45},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.48599999999999999},"max":{"1min":1.6779999999999999,"5min":1.889,"15min":30.547000000000001},"last":1.056},{"interface":"front","average":{"1min":1.018,"5min":1.1619999999999999,"15min":1.3200000000000001},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":2.0459999999999998,"15min":17.263000000000002},"last":1.298}]},{"osd":4,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.6639999999999999,"15min":1.6899999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.72699999999999998},"max":{"1min":1.6319999999999999,"5min":14.103,"15min":17.420000000000002},"last":1.423},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.653,"15min":1.6040000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7849999999999999,"5min":14.140000000000001,"15min":14.140000000000001},"last":1.6040000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3859999999999999,"15min":1.5329999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.59599999999999997},"max":{"1min":1.843,"5min":2.077,"15min":9.0440000000000005},"last":1.153},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.508,"15min":1.7989999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.68300000000000005},"max":{"1min":1.891,"5min":2.3599999999999999,"15min":34.042999999999999},"last":1.6359999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.494,"15min":1.6830000000000001},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":0.64100000000000001},"max":{"1min":1.9790000000000001,"5min":2.3780000000000001,"15min":34.158000000000001},"last":1.351},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.4850000000000001,"15min":1.637},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.62},"max":{"1min":1.8260000000000001,"5min":2.5579999999999998,"15min":17.510000000000002},"last":1.5600000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.603,"15min":1.8700000000000001},"min":{"1min":0.98299999999999998,"5min":0.82999999999999996,"15min":0.77300000000000002},"max":{"1min":1.998,"5min":2.4329999999999998,"15min":33.659999999999997},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.6439999999999999,"15min":1.742},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.70299999999999996},"max":{"1min":1.9370000000000001,"5min":2.6669999999999998,"15min":11.882999999999999},"last":1.54}]}]},{"osd":0,"up_from":14,"seq":60129542422,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1769760,"kb_used_data":1698392,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":92602080,"statfs":{"total":96636764160,"available":94824529920,"internally_reserved":0,"allocated":1739153408,"data_stored":1738300216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.1080000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4630000000000001,"5min":2.5209999999999999,"15min":12.050000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3100000000000001,"15min":1.357},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.36399999999999999},"max":{"1min":1.609,"5min":6.6890000000000001,"15min":11.962},"last":0.98299999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.248,"15min":1.2230000000000001},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.45500000000000002},"max":{"1min":1.7050000000000001,"5min":6.8099999999999996,"15min":11.59},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.995,"5min":1.0620000000000001,"15min":1.1970000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.6870000000000001,"5min":1.825,"15min":14.512},"last":1.1899999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.385,"15min":1.425},"min":{"1min":0.58899999999999997,"5min":0.51300000000000001,"15min":0.50800000000000001},"max":{"1min":1.7130000000000001,"5min":6.8719999999999999,"15min":11.904},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1899999999999999,"15min":1.1830000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.35199999999999998},"max":{"1min":1.696,"5min":2.3919999999999999,"15min":6.1539999999999999},"last":1.1439999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.29,"15min":1.542},"min":{"1min":0.68000000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.5860000000000001,"5min":2.1829999999999998,"15min":32.115000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.579,"15min":1.6259999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.67700000000000005},"max":{"1min":1.6890000000000001,"5min":6.9329999999999998,"15min":11.992000000000001},"last":1.4970000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3879999999999999,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.60899999999999999,"15min":0.47499999999999998},"max":{"1min":1.514,"5min":2.593,"15min":32.174999999999997},"last":1.091},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.6020000000000001,"15min":1.6160000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.442},"max":{"1min":2.157,"5min":15.907,"15min":17.225000000000001},"last":1.464}]},{"osd":6,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.528,"15min":1.754},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.66600000000000004},"max":{"1min":2.3290000000000002,"5min":2.5590000000000002,"15min":24.039999999999999},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.49,"15min":1.7789999999999999},"min":{"1min":0.99199999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.758,"5min":2.5449999999999999,"15min":32.216999999999999},"last":1.2989999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.708,"15min":2.0800000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.69099999999999995},"max":{"1min":1.8300000000000001,"5min":7.0979999999999999,"15min":34.131},"last":1.395},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.641,"15min":1.984},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.61799999999999999},"max":{"1min":2.266,"5min":3.3759999999999999,"15min":44.694000000000003},"last":1.6559999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":632389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":647168,"data_stored":635251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1736458240,"data_stored":1736442609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":931135488,"data_stored":931135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":851443712,"data_stored":851443712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:17:52.886 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-15T18:17:53.147 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.145+0000 7fb9ff200640 1 Processor -- start 2024-01-15T18:17:53.148 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.145+0000 7fb9ff200640 1 -- start start 2024-01-15T18:17:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.147+0000 7fb9ff200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fba000cc990 0x7fba0006d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.147+0000 7fb9ff200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba00070e50 con 0x7fba00143c40 2024-01-15T18:17:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.147+0000 7fb9ff200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba00144020 con 0x7fba000cc500 2024-01-15T18:17:53.149 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.147+0000 7fb9ff200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba000723c0 con 0x7fba000cc990 2024-01-15T18:17:53.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.147+0000 7fb9fde00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fba00143c40 0x7fba00071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56846/0 (socket says 172.21.15.26:56846) 2024-01-15T18:17:53.150 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.147+0000 7fb9fde00640 1 -- 172.21.15.26:0/3025927644 learned_addr learned my addr 172.21.15.26:0/3025927644 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.147+0000 7fb9fe800640 1 --2- 172.21.15.26:0/3025927644 >> v2:172.21.15.124:3300/0 conn(0x7fba000cc990 0x7fba0006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984259840 0 0) 0x7fba00070e50 con 0x7fba00143c40 2024-01-15T18:17:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e0003520 con 0x7fba00143c40 2024-01-15T18:17:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677395753 0 0) 0x7fb9e0003520 con 0x7fba00143c40 2024-01-15T18:17:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 >> v2:172.21.15.124:3300/0 conn(0x7fba000cc990 msgr2=0x7fba0006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 --2- 172.21.15.26:0/3025927644 >> v2:172.21.15.124:3300/0 conn(0x7fba000cc990 0x7fba0006d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 >> v1:172.21.15.124:6789/0 conn(0x7fba000cc500 legacy=0x7fba0006d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba00071e00 con 0x7fba00143c40 2024-01-15T18:17:53.152 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fb9f0001670 con 0x7fba00143c40 2024-01-15T18:17:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 >> v1:172.21.15.26:6789/0 conn(0x7fba00143c40 legacy=0x7fba00071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e0003970 0x7fb9e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e00047c0 0x7fb9e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e0005180 0x7fb9e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba000723c0 con 0x7fb9e0003970 2024-01-15T18:17:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e000c120 con 0x7fb9e0005180 2024-01-15T18:17:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e000c2e0 con 0x7fb9e00047c0 2024-01-15T18:17:53.153 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.148+0000 7fb9fde00640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e0003970 0x7fb9e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9fe800640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e0005180 0x7fb9e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9f7e00640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e00047c0 0x7fb9e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9fde00640 1 -- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e00047c0 msgr2=0x7fb9e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9fde00640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e00047c0 0x7fb9e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9fde00640 1 -- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e0005180 msgr2=0x7fb9e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9fde00640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e0005180 0x7fb9e000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9fde00640 1 -- 172.21.15.26:0/3025927644 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0003520 con 0x7fb9e0003970 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9fe800640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fb9e0005180 0x7fb9e000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9fe800640 1 -- 172.21.15.26:0/3025927644 reap_dead start 2024-01-15T18:17:53.154 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.149+0000 7fb9f7e00640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e00047c0 0x7fb9e0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.150+0000 7fb9fde00640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e0003970 0x7fb9e0003d90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000c240 tx=0x7fb9e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334dd1c4a6d6e839 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.150+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0005ad0 con 0x7fb9e0003970 2024-01-15T18:17:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.150+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f0005c30 con 0x7fb9e0003970 2024-01-15T18:17:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.150+0000 7fb9fd400640 1 -- 172.21.15.26:0/3025927644 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f002ca70 con 0x7fb9e0003970 2024-01-15T18:17:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.151+0000 7fb9ff200640 1 -- 172.21.15.26:0/3025927644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e0003970 msgr2=0x7fb9e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.151+0000 7fb9ff200640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e0003970 0x7fb9e0003d90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000c240 tx=0x7fb9e00057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.151+0000 7fb9ff200640 1 -- 172.21.15.26:0/3025927644 shutdown_connections 2024-01-15T18:17:53.156 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.151+0000 7fb9ff200640 1 --2- 172.21.15.26:0/3025927644 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fb9e0003970 0x7fb9e0003d90 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.151+0000 7fb9ff200640 1 -- 172.21.15.26:0/3025927644 >> 172.21.15.26:0/3025927644 conn(0x7fba0006a800 msgr2=0x7fba0006ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.151+0000 7fb9ff200640 1 -- 172.21.15.26:0/3025927644 shutdown_connections 2024-01-15T18:17:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.151+0000 7fb9ff200640 1 -- 172.21.15.26:0/3025927644 wait complete. 2024-01-15T18:17:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.153+0000 7fb9ff200640 1 Processor -- start 2024-01-15T18:17:53.157 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.153+0000 7fb9ff200640 1 -- start start 2024-01-15T18:17:53.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.154+0000 7fb9ff200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e0003970 0x7fba0013aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:53.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.154+0000 7fb9ff200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0013d6f0 0x7fba0013db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:53.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.154+0000 7fb9ff200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba00159070 0x7fba00159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:53.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.154+0000 7fb9ff200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0013e0b0 con 0x7fba0013d6f0 2024-01-15T18:17:53.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.154+0000 7fb9ff200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba00140c40 con 0x7fba00159070 2024-01-15T18:17:53.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.154+0000 7fb9ff200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba00140ee0 con 0x7fb9e0003970 2024-01-15T18:17:53.159 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.154+0000 7fb9f7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0013d6f0 0x7fba0013db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.154+0000 7fb9f7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0013d6f0 0x7fba0013db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56462/0 (socket says 172.21.15.26:56462) 2024-01-15T18:17:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.154+0000 7fb9f7e00640 1 -- 172.21.15.26:0/1873179466 learned_addr learned my addr 172.21.15.26:0/1873179466 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.155+0000 7fb9fde00640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e0003970 0x7fba0013aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.155+0000 7fb9fe800640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba00159070 0x7fba00159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.155+0000 7fb9f7e00640 1 -- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e0003970 msgr2=0x7fba0013aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.155+0000 7fb9f7e00640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e0003970 0x7fba0013aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.155+0000 7fb9f7e00640 1 -- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba00159070 msgr2=0x7fba00159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.155+0000 7fb9f7e00640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba00159070 0x7fba00159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.160 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.155+0000 7fb9f7e00640 1 -- 172.21.15.26:0/1873179466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba001411c0 con 0x7fba0013d6f0 2024-01-15T18:17:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.155+0000 7fb9f7e00640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0013d6f0 0x7fba0013db70 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800d4a0 tx=0x7fb9e800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.155+0000 7fb9f6000640 1 -- 172.21.15.26:0/1873179466 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8005480 con 0x7fba0013d6f0 2024-01-15T18:17:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.156+0000 7fb9f6000640 1 -- 172.21.15.26:0/1873179466 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e8007030 con 0x7fba0013d6f0 2024-01-15T18:17:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.156+0000 7fb9f6000640 1 -- 172.21.15.26:0/1873179466 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8007400 con 0x7fba0013d6f0 2024-01-15T18:17:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.156+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba001414a0 con 0x7fba0013d6f0 2024-01-15T18:17:53.161 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.156+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba00159a60 con 0x7fba0013d6f0 2024-01-15T18:17:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.157+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba00067450 con 0x7fba0013d6f0 2024-01-15T18:17:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.158+0000 7fb9f6000640 1 -- 172.21.15.26:0/1873179466 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fb9e8007560 con 0x7fba0013d6f0 2024-01-15T18:17:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.159+0000 7fb9f6000640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9d806c580 0x7fb9d806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.159+0000 7fb9f6000640 1 -- 172.21.15.26:0/1873179466 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fb9e8087f90 con 0x7fba0013d6f0 2024-01-15T18:17:53.162 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.160+0000 7fb9fde00640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9d806c580 0x7fb9d806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:53.163 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.161+0000 7fb9fde00640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9d806c580 0x7fb9d806ea40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0034000 tx=0x7fb9f0009030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:53.166 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.163+0000 7fb9f6000640 1 -- 172.21.15.26:0/1873179466 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9e80545c0 con 0x7fba0013d6f0 2024-01-15T18:17:53.362 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.360+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba000ceb30 con 0x7fba0013d6f0 2024-01-15T18:17:53.365 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.363+0000 7fb9f6000640 1 -- 172.21.15.26:0/1873179466 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16729 (secure 0 0 0) 0x7fb9e8019070 con 0x7fba0013d6f0 2024-01-15T18:17:53.366 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:17:53.366 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":117,"fsid":"64ff4128-2fc8-4c27-b6c0-fe9f8ca7d9d1","created":"2024-01-15T17:54:44.821019+0000","modified":"2024-01-15T18:12:53.309154+0000","last_up_change":"2024-01-15T17:54:56.455609+0000","last_in_change":"2024-01-15T17:54:48.372160+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-15T17:54:57.981100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-01-15T18:12:08.696115+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-01-15T18:12:09.497434+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c8f57d9b-0259-4478-80d0-1ed0a378a557","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6800","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6801","nonce":3842439308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6802","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6803","nonce":3842439308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6806","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6807","nonce":3842439308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6804","nonce":3842439308},{"type":"v1","addr":"172.21.15.26:6805","nonce":3842439308}]},"public_addr":"172.21.15.26:6801/3842439308","cluster_addr":"172.21.15.26:6803/3842439308","heartbeat_back_addr":"172.21.15.26:6807/3842439308","heartbeat_front_addr":"172.21.15.26:6805/3842439308","state":["exists","up"]},{"osd":1,"uuid":"b107ddc6-f573-40ca-8c09-62c5e8f4bfdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6824","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6825","nonce":639007827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6826","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6827","nonce":639007827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6830","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6831","nonce":639007827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6828","nonce":639007827},{"type":"v1","addr":"172.21.15.26:6829","nonce":639007827}]},"public_addr":"172.21.15.26:6825/639007827","cluster_addr":"172.21.15.26:6827/639007827","heartbeat_back_addr":"172.21.15.26:6831/639007827","heartbeat_front_addr":"172.21.15.26:6829/639007827","state":["exists","up"]},{"osd":2,"uuid":"2c4d3745-c37d-438b-8065-3bb12e54e54a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6816","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6817","nonce":1591707592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6818","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6819","nonce":1591707592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6822","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6823","nonce":1591707592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6820","nonce":1591707592},{"type":"v1","addr":"172.21.15.26:6821","nonce":1591707592}]},"public_addr":"172.21.15.26:6817/1591707592","cluster_addr":"172.21.15.26:6819/1591707592","heartbeat_back_addr":"172.21.15.26:6823/1591707592","heartbeat_front_addr":"172.21.15.26:6821/1591707592","state":["exists","up"]},{"osd":3,"uuid":"e0b8df60-d19e-43a8-94bd-3607d215133f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6808","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6809","nonce":696680319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6810","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6811","nonce":696680319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6814","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6815","nonce":696680319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6812","nonce":696680319},{"type":"v1","addr":"172.21.15.26:6813","nonce":696680319}]},"public_addr":"172.21.15.26:6809/696680319","cluster_addr":"172.21.15.26:6811/696680319","heartbeat_back_addr":"172.21.15.26:6815/696680319","heartbeat_front_addr":"172.21.15.26:6813/696680319","state":["exists","up"]},{"osd":4,"uuid":"592f3f51-ada0-4adb-bfc0-86535d57b38e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6800","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6801","nonce":126047302}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6802","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6803","nonce":126047302}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6806","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6807","nonce":126047302}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6804","nonce":126047302},{"type":"v1","addr":"172.21.15.124:6805","nonce":126047302}]},"public_addr":"172.21.15.124:6801/126047302","cluster_addr":"172.21.15.124:6803/126047302","heartbeat_back_addr":"172.21.15.124:6807/126047302","heartbeat_front_addr":"172.21.15.124:6805/126047302","state":["exists","up"]},{"osd":5,"uuid":"9b649ed7-e5e5-4689-9746-d801e518f19b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6824","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6825","nonce":4019293128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6826","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6827","nonce":4019293128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6830","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6831","nonce":4019293128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6828","nonce":4019293128},{"type":"v1","addr":"172.21.15.124:6829","nonce":4019293128}]},"public_addr":"172.21.15.124:6825/4019293128","cluster_addr":"172.21.15.124:6827/4019293128","heartbeat_back_addr":"172.21.15.124:6831/4019293128","heartbeat_front_addr":"172.21.15.124:6829/4019293128","state":["exists","up"]},{"osd":6,"uuid":"2bf4bcd1-9a42-426c-b294-46b7b6f9b1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6816","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6817","nonce":4103149233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6818","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6819","nonce":4103149233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6822","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6823","nonce":4103149233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6820","nonce":4103149233},{"type":"v1","addr":"172.21.15.124:6821","nonce":4103149233}]},"public_addr":"172.21.15.124:6817/4103149233","cluster_addr":"172.21.15.124:6819/4103149233","heartbeat_back_addr":"172.21.15.124:6823/4103149233","heartbeat_front_addr":"172.21.15.124:6821/4103149233","state":["exists","up"]},{"osd":7,"uuid":"e21d396c-465f-43a6-b770-ac1f831974b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6808","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6809","nonce":1393332266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6810","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6811","nonce":1393332266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6814","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6815","nonce":1393332266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6812","nonce":1393332266},{"type":"v1","addr":"172.21.15.124:6813","nonce":1393332266}]},"public_addr":"172.21.15.124:6809/1393332266","cluster_addr":"172.21.15.124:6811/1393332266","heartbeat_back_addr":"172.21.15.124:6815/1393332266","heartbeat_front_addr":"172.21.15.124:6813/1393332266","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.858368+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676259+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.680937+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.949090+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.916389+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.765955+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.688503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-01-15T17:54:53.676388+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.30","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":2}]},{"pool":9,"snaps":[{"begin":2,"length":2}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-15T18:17:53.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.365+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9d806c580 msgr2=0x7fb9d806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:53.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.365+0000 7fb9ff200640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9d806c580 0x7fb9d806ea40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0034000 tx=0x7fb9f0009030 comp rx=0 tx=0).stop 2024-01-15T18:17:53.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.365+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0013d6f0 msgr2=0x7fba0013db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.365+0000 7fb9ff200640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0013d6f0 0x7fba0013db70 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800d4a0 tx=0x7fb9e800d960 comp rx=0 tx=0).stop 2024-01-15T18:17:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.366+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 shutdown_connections 2024-01-15T18:17:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.366+0000 7fb9ff200640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fb9d806c580 0x7fb9d806ea40 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.366+0000 7fb9ff200640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fba00159070 0x7fba00159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.366+0000 7fb9ff200640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fba0013d6f0 0x7fba0013db70 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.366+0000 7fb9ff200640 1 --2- 172.21.15.26:0/1873179466 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fb9e0003970 0x7fba0013aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.366+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 >> 172.21.15.26:0/1873179466 conn(0x7fba0006a800 msgr2=0x7fba00140060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.366+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 shutdown_connections 2024-01-15T18:17:53.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:53.366+0000 7fb9ff200640 1 -- 172.21.15.26:0/1873179466 wait complete. 2024-01-15T18:17:54.389 INFO:tasks.ceph:Scrubbing osd.0 2024-01-15T18:17:54.390 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:17:54.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.625+0000 7fbad7200640 1 Processor -- start 2024-01-15T18:17:54.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.625+0000 7fbad7200640 1 -- start start 2024-01-15T18:17:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.627+0000 7fbad7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbad80cc500 0x7fbad80cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.627+0000 7fbad7200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbad806d160 0x7fbad806d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.627+0000 7fbad7200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad8070e50 con 0x7fbad8143c40 2024-01-15T18:17:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.627+0000 7fbad7200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad80723c0 con 0x7fbad806d160 2024-01-15T18:17:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.627+0000 7fbad7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad8144020 con 0x7fbad80cc500 2024-01-15T18:17:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.627+0000 7fbad6800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbad806d160 0x7fbad806d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.627+0000 7fbad5e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fbad8143c40 0x7fbad8071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56852/0 (socket says 172.21.15.26:56852) 2024-01-15T18:17:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.627+0000 7fbad5e00640 1 -- 172.21.15.26:0/488797290 learned_addr learned my addr 172.21.15.26:0/488797290 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.628+0000 7fbacfe00640 1 --2- 172.21.15.26:0/488797290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbad80cc500 0x7fbad80cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.631 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.628+0000 7fbad6800640 1 -- 172.21.15.26:0/488797290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbad80cc500 msgr2=0x7fbad80cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.628+0000 7fbad6800640 1 --2- 172.21.15.26:0/488797290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbad80cc500 0x7fbad80cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.628+0000 7fbad6800640 1 -- 172.21.15.26:0/488797290 >> v1:172.21.15.26:6789/0 conn(0x7fbad8143c40 legacy=0x7fbad8071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.628+0000 7fbad6800640 1 -- 172.21.15.26:0/488797290 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80ccea0 con 0x7fbad806d160 2024-01-15T18:17:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.628+0000 7fbad5400640 1 -- 172.21.15.26:0/488797290 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820380881 0 0) 0x7fbad8070e50 con 0x7fbad8143c40 2024-01-15T18:17:54.632 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.629+0000 7fbad6800640 1 --2- 172.21.15.26:0/488797290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 0x7fbad806d590 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009d90 tx=0x7fbad002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcf99155959a78a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:54.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.630+0000 7fbad5400640 1 -- 172.21.15.26:0/488797290 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad002cbd0 con 0x7fbad806d160 2024-01-15T18:17:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.630+0000 7fbad5400640 1 -- 172.21.15.26:0/488797290 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad002cd30 con 0x7fbad806d160 2024-01-15T18:17:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.630+0000 7fbad5400640 1 -- 172.21.15.26:0/488797290 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad002cea0 con 0x7fbad806d160 2024-01-15T18:17:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.630+0000 7fbad7200640 1 -- 172.21.15.26:0/488797290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 msgr2=0x7fbad806d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.630+0000 7fbad7200640 1 --2- 172.21.15.26:0/488797290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 0x7fbad806d590 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009d90 tx=0x7fbad002be20 comp rx=0 tx=0).stop 2024-01-15T18:17:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.630+0000 7fbad7200640 1 -- 172.21.15.26:0/488797290 shutdown_connections 2024-01-15T18:17:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.630+0000 7fbad7200640 1 --2- 172.21.15.26:0/488797290 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 0x7fbad806d590 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.630+0000 7fbad7200640 1 --2- 172.21.15.26:0/488797290 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbad80cc500 0x7fbad80cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.634 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.630+0000 7fbad7200640 1 -- 172.21.15.26:0/488797290 >> 172.21.15.26:0/488797290 conn(0x7fbad806a800 msgr2=0x7fbad806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:54.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.631+0000 7fbad7200640 1 -- 172.21.15.26:0/488797290 shutdown_connections 2024-01-15T18:17:54.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.631+0000 7fbad7200640 1 -- 172.21.15.26:0/488797290 wait complete. 2024-01-15T18:17:54.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.632+0000 7fbad7200640 1 Processor -- start 2024-01-15T18:17:54.635 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.633+0000 7fbad7200640 1 -- start start 2024-01-15T18:17:54.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad7200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 0x7fbad813e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad7200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbad80cc500 0x7fbad81399e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad7200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbad8143c40 0x7fbad8139f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad7200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad813b120 con 0x7fbad806d160 2024-01-15T18:17:54.637 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad7200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad813a5a0 con 0x7fbad8143c40 2024-01-15T18:17:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad7200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad813a880 con 0x7fbad80cc500 2024-01-15T18:17:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 0x7fbad813e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad5e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 0x7fbad813e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56484/0 (socket says 172.21.15.26:56484) 2024-01-15T18:17:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad5e00640 1 -- 172.21.15.26:0/2095170750 learned_addr learned my addr 172.21.15.26:0/2095170750 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbacfe00640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbad80cc500 0x7fbad81399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.634+0000 7fbad6800640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbad8143c40 0x7fbad8139f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbad5e00640 1 -- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbad80cc500 msgr2=0x7fbad81399e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.638 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbad5e00640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbad80cc500 0x7fbad81399e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbad5e00640 1 -- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbad8143c40 msgr2=0x7fbad8139f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbad5e00640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbad8143c40 0x7fbad8139f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbad5e00640 1 -- 172.21.15.26:0/2095170750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad0009a70 con 0x7fbad806d160 2024-01-15T18:17:54.639 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbad5e00640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 0x7fbad813e310 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c430 tx=0x7fbac800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:54.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbace000640 1 -- 172.21.15.26:0/2095170750 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac80074e0 con 0x7fbad806d160 2024-01-15T18:17:54.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbace000640 1 -- 172.21.15.26:0/2095170750 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac8007660 con 0x7fbad806d160 2024-01-15T18:17:54.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad813ab60 con 0x7fbad806d160 2024-01-15T18:17:54.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbace000640 1 -- 172.21.15.26:0/2095170750 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac8004a20 con 0x7fbad806d160 2024-01-15T18:17:54.640 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.635+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad80b3d70 con 0x7fbad806d160 2024-01-15T18:17:54.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.636+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbad80ce280 con 0x7fbad806d160 2024-01-15T18:17:54.641 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.637+0000 7fbace000640 1 -- 172.21.15.26:0/2095170750 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fbac8004b80 con 0x7fbad806d160 2024-01-15T18:17:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.638+0000 7fbace000640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbab006c580 0x7fbab006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.639+0000 7fbace000640 1 -- 172.21.15.26:0/2095170750 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fbac8088a40 con 0x7fbad806d160 2024-01-15T18:17:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.639+0000 7fbace000640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7fbab0074900 0x7fbab0076da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.639+0000 7fbace000640 1 -- 172.21.15.26:0/2095170750 --> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbab0077470 con 0x7fbab0074900 2024-01-15T18:17:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.639+0000 7fbace000640 1 -- 172.21.15.26:0/2095170750 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac8089660 con 0x7fbad806d160 2024-01-15T18:17:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.639+0000 7fbad6800640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7fbab0074900 0x7fbab0076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.642 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.639+0000 7fbacfe00640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbab006c580 0x7fbab006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.640+0000 7fbad6800640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7fbab0074900 0x7fbab0076da0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:54.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.640+0000 7fbacfe00640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbab006c580 0x7fbab006ea40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbad80afc80 tx=0x7fbac0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:54.643 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.641+0000 7fbace000640 1 -- 172.21.15.26:0/2095170750 <== osd.0 v2:172.21.15.26:6800/3842439308 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fbab0077470 con 0x7fbab0074900 2024-01-15T18:17:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.655+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 --> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbad806c960 con 0x7fbab0074900 2024-01-15T18:17:54.658 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.656+0000 7fbace000640 1 -- 172.21.15.26:0/2095170750 <== osd.0 v2:172.21.15.26:6800/3842439308 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbad0009450 con 0x7fbab0074900 2024-01-15T18:17:54.658 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:17:54.658 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:17:54.659 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:17:54.659 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.656+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7fbab0074900 msgr2=0x7fbab0076da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.656+0000 7fbad7200640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.26:6800/3842439308,v1:172.21.15.26:6801/3842439308] conn(0x7fbab0074900 0x7fbab0076da0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.657+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbab006c580 msgr2=0x7fbab006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.657+0000 7fbad7200640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbab006c580 0x7fbab006ea40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbad80afc80 tx=0x7fbac0008040 comp rx=0 tx=0).stop 2024-01-15T18:17:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.657+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 msgr2=0x7fbad813e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.657+0000 7fbad7200640 1 --2- 172.21.15.26:0/2095170750 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbad806d160 0x7fbad813e310 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c430 tx=0x7fbac800c8f0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.657+0000 7fbad6800640 1 -- 172.21.15.26:0/2095170750 reap_dead start 2024-01-15T18:17:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.657+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 shutdown_connections 2024-01-15T18:17:54.660 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.657+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 >> 172.21.15.26:0/2095170750 conn(0x7fbad806a800 msgr2=0x7fbad806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:54.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.657+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 shutdown_connections 2024-01-15T18:17:54.661 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.657+0000 7fbad7200640 1 -- 172.21.15.26:0/2095170750 wait complete. 2024-01-15T18:17:54.677 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-01-15T18:17:54.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.888+0000 7f03adc00640 1 Processor -- start 2024-01-15T18:17:54.891 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.889+0000 7f03adc00640 1 -- start start 2024-01-15T18:17:54.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.891+0000 7f03adc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a80cccf0 0x7f03a815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.891+0000 7f03adc00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a806ca10 con 0x7f03a81447f0 2024-01-15T18:17:54.893 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.891+0000 7f03adc00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a806ccf0 con 0x7f03a806c470 2024-01-15T18:17:54.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.891+0000 7f03adc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a806d690 con 0x7f03a80cccf0 2024-01-15T18:17:54.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.891+0000 7f03a7e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f03a81447f0 0x7f03a815c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56854/0 (socket says 172.21.15.26:56854) 2024-01-15T18:17:54.894 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.891+0000 7f03a7e00640 1 -- 172.21.15.26:0/4190317423 learned_addr learned my addr 172.21.15.26:0/4190317423 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.891+0000 7f03ad200640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a80cccf0 0x7f03a815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2527507447 0 0) 0x7f03a806ca10 con 0x7f03a81447f0 2024-01-15T18:17:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0388003520 con 0x7f03a81447f0 2024-01-15T18:17:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214427761 0 0) 0x7f03a806ccf0 con 0x7f03a806c470 2024-01-15T18:17:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a806ca10 con 0x7f03a806c470 2024-01-15T18:17:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 397282277 0 0) 0x7f0388003520 con 0x7f03a81447f0 2024-01-15T18:17:54.896 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a80cccf0 msgr2=0x7f03a815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a80cccf0 0x7f03a815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 >> v1:172.21.15.124:6789/0 conn(0x7f03a806c470 legacy=0x7f03a80cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a806cfd0 con 0x7f03a81447f0 2024-01-15T18:17:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f0398001670 con 0x7f03a81447f0 2024-01-15T18:17:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 >> v1:172.21.15.26:6789/0 conn(0x7f03a81447f0 legacy=0x7f03a815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a80cccf0 0x7f0388003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03880040e0 0x7f03880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0388004d00 0x7f0388005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.897 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a806d690 con 0x7f03880040e0 2024-01-15T18:17:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038800bb40 con 0x7f0388004d00 2024-01-15T18:17:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.892+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038800bd00 con 0x7f03a80cccf0 2024-01-15T18:17:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.893+0000 7f03a7400640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03880040e0 0x7f03880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.893+0000 7f03a7e00640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a80cccf0 0x7f0388003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.893+0000 7f03ad200640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0388004d00 0x7f0388005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.893+0000 7f03a7400640 1 -- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a80cccf0 msgr2=0x7f0388003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.893+0000 7f03a7400640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03a80cccf0 0x7f0388003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.893+0000 7f03a7400640 1 -- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0388004d00 msgr2=0x7f0388005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:54.898 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.893+0000 7f03a7400640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f0388004d00 0x7f0388005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.899 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.893+0000 7f03a7400640 1 -- 172.21.15.26:0/4190317423 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0388003520 con 0x7f03880040e0 2024-01-15T18:17:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.894+0000 7f03a7400640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03880040e0 0x7f03880047c0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f03900018d0 tx=0x7f03880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aae1e1e6fbeff65 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03900052d0 con 0x7f03880040e0 2024-01-15T18:17:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390005430 con 0x7f03880040e0 2024-01-15T18:17:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03a6a00640 1 -- 172.21.15.26:0/4190317423 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390003810 con 0x7f03880040e0 2024-01-15T18:17:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03adc00640 1 -- 172.21.15.26:0/4190317423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03880040e0 msgr2=0x7f03880047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.900 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03adc00640 1 --2- 172.21.15.26:0/4190317423 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03880040e0 0x7f03880047c0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f03900018d0 tx=0x7f03880057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03ad200640 1 -- 172.21.15.26:0/4190317423 reap_dead start 2024-01-15T18:17:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03adc00640 1 -- 172.21.15.26:0/4190317423 shutdown_connections 2024-01-15T18:17:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03adc00640 1 -- 172.21.15.26:0/4190317423 >> 172.21.15.26:0/4190317423 conn(0x7f03a806a3e0 msgr2=0x7f03a806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03adc00640 1 -- 172.21.15.26:0/4190317423 shutdown_connections 2024-01-15T18:17:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.895+0000 7f03adc00640 1 -- 172.21.15.26:0/4190317423 wait complete. 2024-01-15T18:17:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.897+0000 7f03adc00640 1 Processor -- start 2024-01-15T18:17:54.901 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.897+0000 7f03adc00640 1 -- start start 2024-01-15T18:17:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.898+0000 7f03adc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03880040e0 0x7f03a815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.898+0000 7f03adc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a8070280 0x7f03a813ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.898+0000 7f03adc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a8070700 0x7f03a813b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.898+0000 7f03adc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8071680 con 0x7f03a8070280 2024-01-15T18:17:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.898+0000 7f03adc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8071960 con 0x7f03a8070700 2024-01-15T18:17:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.898+0000 7f03adc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a8071c40 con 0x7f03880040e0 2024-01-15T18:17:54.903 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03a7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a8070280 0x7f03a813ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03a7400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a8070280 0x7f03a813ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56506/0 (socket says 172.21.15.26:56506) 2024-01-15T18:17:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03a7400640 1 -- 172.21.15.26:0/3040173671 learned_addr learned my addr 172.21.15.26:0/3040173671 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03a7e00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03880040e0 0x7f03a815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03ad200640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a8070700 0x7f03a813b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03a7400640 1 -- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03880040e0 msgr2=0x7f03a815a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03a7400640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03880040e0 0x7f03a815a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03a7400640 1 -- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a8070700 msgr2=0x7f03a813b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03a7400640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a8070700 0x7f03a813b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:54.904 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03a7400640 1 -- 172.21.15.26:0/3040173671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8071f20 con 0x7f03a8070280 2024-01-15T18:17:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.899+0000 7f03ad200640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a8070700 0x7f03a813b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.900+0000 7f03a7400640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a8070280 0x7f03a813ab80 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f0390035040 tx=0x7f03900012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.900+0000 7f039fe00640 1 -- 172.21.15.26:0/3040173671 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390006ea0 con 0x7f03a8070280 2024-01-15T18:17:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.900+0000 7f039fe00640 1 -- 172.21.15.26:0/3040173671 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0390007450 con 0x7f03a8070280 2024-01-15T18:17:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.900+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a813a2c0 con 0x7f03a8070280 2024-01-15T18:17:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.900+0000 7f039fe00640 1 -- 172.21.15.26:0/3040173671 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03900035f0 con 0x7f03a8070280 2024-01-15T18:17:54.905 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.900+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a81f5ec0 con 0x7f03a8070280 2024-01-15T18:17:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.901+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a8067450 con 0x7f03a8070280 2024-01-15T18:17:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.902+0000 7f039fe00640 1 -- 172.21.15.26:0/3040173671 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f0390003750 con 0x7f03a8070280 2024-01-15T18:17:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.903+0000 7f039fe00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038006c580 0x7f038006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:54.906 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.904+0000 7f039fe00640 1 -- 172.21.15.26:0/3040173671 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f03900aa650 con 0x7f03a8070280 2024-01-15T18:17:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.904+0000 7f03a7e00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038006c580 0x7f038006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:54.907 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.905+0000 7f03a7e00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038006c580 0x7f038006ea40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0398001b60 tx=0x7f0398009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:54.910 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:54.908+0000 7f039fe00640 1 -- 172.21.15.26:0/3040173671 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0390076fc0 con 0x7f03a8070280 2024-01-15T18:17:55.113 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.111+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f03a8071ad0 con 0x7f038006c580 2024-01-15T18:17:55.114 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.112+0000 7f039fe00640 1 -- 172.21.15.26:0/3040173671 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f03a8071ad0 con 0x7f038006c580 2024-01-15T18:17:55.115 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 0 to deep-scrub 2024-01-15T18:17:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038006c580 msgr2=0x7f038006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038006c580 0x7f038006ea40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0398001b60 tx=0x7f0398009040 comp rx=0 tx=0).stop 2024-01-15T18:17:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a8070280 msgr2=0x7f03a813ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.118 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a8070280 0x7f03a813ab80 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f0390035040 tx=0x7f03900012d0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 shutdown_connections 2024-01-15T18:17:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f038006c580 0x7f038006ea40 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f03a8070700 0x7f03a813b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f03a8070280 0x7f03a813ab80 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 --2- 172.21.15.26:0/3040173671 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f03880040e0 0x7f03a815a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 >> 172.21.15.26:0/3040173671 conn(0x7f03a806a3e0 msgr2=0x7f03a815e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 shutdown_connections 2024-01-15T18:17:55.119 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.115+0000 7f03adc00640 1 -- 172.21.15.26:0/3040173671 wait complete. 2024-01-15T18:17:55.137 INFO:tasks.ceph:Scrubbing osd.1 2024-01-15T18:17:55.138 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:17:55.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.361+0000 7f6754a00640 1 Processor -- start 2024-01-15T18:17:55.364 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.361+0000 7f6754a00640 1 -- start start 2024-01-15T18:17:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.363+0000 7f6754a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675006f430 0x7f675015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.363+0000 7f6754a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67500ccd60 con 0x7f67500ce9e0 2024-01-15T18:17:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.363+0000 7f6754a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67500cef50 con 0x7f675015eb80 2024-01-15T18:17:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.363+0000 7f6754a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67500cf230 con 0x7f675006f430 2024-01-15T18:17:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.363+0000 7f674ea00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f67500ce9e0 0x7f67500cc650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56870/0 (socket says 172.21.15.26:56870) 2024-01-15T18:17:55.366 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.363+0000 7f674ea00640 1 -- 172.21.15.26:0/3900537157 learned_addr learned my addr 172.21.15.26:0/3900537157 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:55.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.364+0000 7f674fe00640 1 --2- 172.21.15.26:0/3900537157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675006f430 0x7f675015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.364+0000 7f674e000640 1 -- 172.21.15.26:0/3900537157 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595834609 0 0) 0x7f67500ccd60 con 0x7f67500ce9e0 2024-01-15T18:17:55.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.364+0000 7f674e000640 1 -- 172.21.15.26:0/3900537157 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6730003520 con 0x7f67500ce9e0 2024-01-15T18:17:55.367 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.364+0000 7f674fe00640 1 -- 172.21.15.26:0/3900537157 >> v1:172.21.15.124:6789/0 conn(0x7f675015eb80 legacy=0x7f675015ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.364+0000 7f674fe00640 1 -- 172.21.15.26:0/3900537157 >> v1:172.21.15.26:6789/0 conn(0x7f67500ce9e0 legacy=0x7f67500cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.365+0000 7f674fe00640 1 -- 172.21.15.26:0/3900537157 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67500cf510 con 0x7f675006f430 2024-01-15T18:17:55.368 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.366+0000 7f674fe00640 1 --2- 172.21.15.26:0/3900537157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675006f430 0x7f675015aa60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6748009dc0 tx=0x7f674802be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be152f33c94a1133 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:55.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.367+0000 7f674e000640 1 -- 172.21.15.26:0/3900537157 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674802cc30 con 0x7f675006f430 2024-01-15T18:17:55.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.367+0000 7f674e000640 1 -- 172.21.15.26:0/3900537157 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674802cd90 con 0x7f675006f430 2024-01-15T18:17:55.369 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.367+0000 7f674e000640 1 -- 172.21.15.26:0/3900537157 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6748034660 con 0x7f675006f430 2024-01-15T18:17:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.367+0000 7f6754a00640 1 -- 172.21.15.26:0/3900537157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675006f430 msgr2=0x7f675015aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.367+0000 7f6754a00640 1 --2- 172.21.15.26:0/3900537157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675006f430 0x7f675015aa60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6748009dc0 tx=0x7f674802be80 comp rx=0 tx=0).stop 2024-01-15T18:17:55.370 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.367+0000 7f6754a00640 1 -- 172.21.15.26:0/3900537157 shutdown_connections 2024-01-15T18:17:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.367+0000 7f6754a00640 1 --2- 172.21.15.26:0/3900537157 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f675006f430 0x7f675015aa60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.368+0000 7f6754a00640 1 -- 172.21.15.26:0/3900537157 >> 172.21.15.26:0/3900537157 conn(0x7f67500d0e10 msgr2=0x7f67500d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.368+0000 7f6754a00640 1 -- 172.21.15.26:0/3900537157 shutdown_connections 2024-01-15T18:17:55.371 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.368+0000 7f6754a00640 1 -- 172.21.15.26:0/3900537157 wait complete. 2024-01-15T18:17:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.370+0000 7f6754a00640 1 Processor -- start 2024-01-15T18:17:55.372 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.370+0000 7f6754a00640 1 -- start start 2024-01-15T18:17:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.371+0000 7f6754a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675006f430 0x7f6750159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.371+0000 7f6754a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67500ce9e0 0x7f67500c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.371+0000 7f6754a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f675015eb80 0x7f67500c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.371+0000 7f6754a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67501f29d0 con 0x7f675006f430 2024-01-15T18:17:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.371+0000 7f6754a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675015cb60 con 0x7f675015eb80 2024-01-15T18:17:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.371+0000 7f6754a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67500c5d80 con 0x7f67500ce9e0 2024-01-15T18:17:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675006f430 0x7f6750159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.375 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674f400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675006f430 0x7f6750159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56520/0 (socket says 172.21.15.26:56520) 2024-01-15T18:17:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674f400640 1 -- 172.21.15.26:0/2060395994 learned_addr learned my addr 172.21.15.26:0/2060395994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674ea00640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67500ce9e0 0x7f67500c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674fe00640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f675015eb80 0x7f67500c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674f400640 1 -- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67500ce9e0 msgr2=0x7f67500c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674f400640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f67500ce9e0 0x7f67500c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674f400640 1 -- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f675015eb80 msgr2=0x7f67500c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674f400640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f675015eb80 0x7f67500c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.376 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.372+0000 7f674f400640 1 -- 172.21.15.26:0/2060395994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6748009a70 con 0x7f675006f430 2024-01-15T18:17:55.377 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.373+0000 7f674f400640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675006f430 0x7f6750159ee0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f674000c3c0 tx=0x7f674000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.373+0000 7f6747400640 1 -- 172.21.15.26:0/2060395994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67400052d0 con 0x7f675006f430 2024-01-15T18:17:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.373+0000 7f6747400640 1 -- 172.21.15.26:0/2060395994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6740005450 con 0x7f675006f430 2024-01-15T18:17:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.373+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67500c57c0 con 0x7f675006f430 2024-01-15T18:17:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.373+0000 7f6747400640 1 -- 172.21.15.26:0/2060395994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740007260 con 0x7f675006f430 2024-01-15T18:17:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.373+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675015c7f0 con 0x7f675006f430 2024-01-15T18:17:55.378 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.374+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f67500cfad0 con 0x7f675006f430 2024-01-15T18:17:55.379 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.377+0000 7f6747400640 1 -- 172.21.15.26:0/2060395994 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f67400073c0 con 0x7f675006f430 2024-01-15T18:17:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.378+0000 7f6747400640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f672806c550 0x7f672806ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.380 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.378+0000 7f6747400640 1 -- 172.21.15.26:0/2060395994 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f6740087f10 con 0x7f675006f430 2024-01-15T18:17:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.378+0000 7f6747400640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7f67280748d0 0x7f6728076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.378+0000 7f6747400640 1 -- 172.21.15.26:0/2060395994 --> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6728077440 con 0x7f67280748d0 2024-01-15T18:17:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.378+0000 7f6747400640 1 -- 172.21.15.26:0/2060395994 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6740088b30 con 0x7f675006f430 2024-01-15T18:17:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.379+0000 7f674ea00640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f672806c550 0x7f672806ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.381 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.379+0000 7f674fe00640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7f67280748d0 0x7f6728076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.379+0000 7f674fe00640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7f67280748d0 0x7f6728076d70 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:55.382 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.380+0000 7f674ea00640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f672806c550 0x7f672806ea10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f675015e680 tx=0x7f6738009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:55.383 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.381+0000 7f6747400640 1 -- 172.21.15.26:0/2060395994 <== osd.1 v2:172.21.15.26:6824/639007827 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6728077440 con 0x7f67280748d0 2024-01-15T18:17:55.400 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.398+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 --> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f675015c650 con 0x7f67280748d0 2024-01-15T18:17:55.401 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.399+0000 7f6747400640 1 -- 172.21.15.26:0/2060395994 <== osd.1 v2:172.21.15.26:6824/639007827 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f675015c650 con 0x7f67280748d0 2024-01-15T18:17:55.401 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:17:55.401 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:17:55.402 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:17:55.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.399+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7f67280748d0 msgr2=0x7f6728076d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.399+0000 7f6754a00640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.26:6824/639007827,v1:172.21.15.26:6825/639007827] conn(0x7f67280748d0 0x7f6728076d70 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.399+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f672806c550 msgr2=0x7f672806ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.399+0000 7f6754a00640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f672806c550 0x7f672806ea10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f675015e680 tx=0x7f6738009040 comp rx=0 tx=0).stop 2024-01-15T18:17:55.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.399+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675006f430 msgr2=0x7f6750159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.399+0000 7f6754a00640 1 --2- 172.21.15.26:0/2060395994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f675006f430 0x7f6750159ee0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f674000c3c0 tx=0x7f674000c880 comp rx=0 tx=0).stop 2024-01-15T18:17:55.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.399+0000 7f674fe00640 1 -- 172.21.15.26:0/2060395994 reap_dead start 2024-01-15T18:17:55.403 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.400+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 shutdown_connections 2024-01-15T18:17:55.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.400+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 >> 172.21.15.26:0/2060395994 conn(0x7f67500d0e10 msgr2=0x7f67500d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:55.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.400+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 shutdown_connections 2024-01-15T18:17:55.404 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.400+0000 7f6754a00640 1 -- 172.21.15.26:0/2060395994 wait complete. 2024-01-15T18:17:55.422 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-01-15T18:17:55.615 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.613+0000 7fbd8cc00640 1 Processor -- start 2024-01-15T18:17:55.616 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.613+0000 7fbd8cc00640 1 -- start start 2024-01-15T18:17:55.618 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.615+0000 7fbd8cc00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fbd880ae5a0 0x7fbd8815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.615+0000 7fbd8cc00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd880b3ae0 0x7fbd8806cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.615+0000 7fbd8cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd8806d120 0x7fbd880cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.615+0000 7fbd8cc00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd88071450 con 0x7fbd880b3ae0 2024-01-15T18:17:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.615+0000 7fbd8cc00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd88070c40 con 0x7fbd880ae5a0 2024-01-15T18:17:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.615+0000 7fbd8cc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8806c470 con 0x7fbd8806d120 2024-01-15T18:17:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.616+0000 7fbd7d200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd880b3ae0 0x7fbd8806cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.619 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.616+0000 7fbd7d200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fbd880b3ae0 0x7fbd8806cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56532/0 (socket says 172.21.15.26:56532) 2024-01-15T18:17:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.616+0000 7fbd7d200640 1 -- 172.21.15.26:0/2286561994 learned_addr learned my addr 172.21.15.26:0/2286561994 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.616+0000 7fbd7d200640 1 -- 172.21.15.26:0/2286561994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd8806d120 msgr2=0x7fbd880cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.616+0000 7fbd7d200640 1 --2- 172.21.15.26:0/2286561994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd8806d120 0x7fbd880cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.616+0000 7fbd85e00640 1 --2- 172.21.15.26:0/2286561994 >> v2:172.21.15.124:3300/0 conn(0x7fbd880ae5a0 0x7fbd8815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.616+0000 7fbd7d200640 1 -- 172.21.15.26:0/2286561994 >> v2:172.21.15.124:3300/0 conn(0x7fbd880ae5a0 msgr2=0x7fbd8815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.616+0000 7fbd7d200640 1 --2- 172.21.15.26:0/2286561994 >> v2:172.21.15.124:3300/0 conn(0x7fbd880ae5a0 0x7fbd8815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.620 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.616+0000 7fbd7d200640 1 -- 172.21.15.26:0/2286561994 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8806c750 con 0x7fbd880b3ae0 2024-01-15T18:17:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.617+0000 7fbd7d200640 1 --2- 172.21.15.26:0/2286561994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 0x7fbd8806cbe0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fbd880cfe20 tx=0x7fbd7002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e30302202c4d62 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:55.621 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd85400640 1 -- 172.21.15.26:0/2286561994 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70031860 con 0x7fbd880b3ae0 2024-01-15T18:17:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd85400640 1 -- 172.21.15.26:0/2286561994 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd700319c0 con 0x7fbd880b3ae0 2024-01-15T18:17:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd85400640 1 -- 172.21.15.26:0/2286561994 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70031d50 con 0x7fbd880b3ae0 2024-01-15T18:17:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2286561994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 msgr2=0x7fbd8806cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2286561994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 0x7fbd8806cbe0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fbd880cfe20 tx=0x7fbd7002b900 comp rx=0 tx=0).stop 2024-01-15T18:17:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2286561994 shutdown_connections 2024-01-15T18:17:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2286561994 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd8806d120 0x7fbd880cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2286561994 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 0x7fbd8806cbe0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.622 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2286561994 >> v2:172.21.15.124:3300/0 conn(0x7fbd880ae5a0 0x7fbd8815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.618+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2286561994 >> 172.21.15.26:0/2286561994 conn(0x7fbd8806a800 msgr2=0x7fbd8806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.619+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2286561994 shutdown_connections 2024-01-15T18:17:55.623 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.619+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2286561994 wait complete. 2024-01-15T18:17:55.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.620+0000 7fbd8cc00640 1 Processor -- start 2024-01-15T18:17:55.625 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.620+0000 7fbd8cc00640 1 -- start start 2024-01-15T18:17:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.621+0000 7fbd8cc00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd8806d120 0x7fbd88155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.621+0000 7fbd8cc00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd880ae5a0 0x7fbd880cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.621+0000 7fbd8cc00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 0x7fbd88160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.621+0000 7fbd8cc00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd88156440 con 0x7fbd880b3ae0 2024-01-15T18:17:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.621+0000 7fbd8cc00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd881f68f0 con 0x7fbd8806d120 2024-01-15T18:17:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.621+0000 7fbd8cc00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd881f6bd0 con 0x7fbd880ae5a0 2024-01-15T18:17:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd86800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 0x7fbd88160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd86800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 0x7fbd88160f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56536/0 (socket says 172.21.15.26:56536) 2024-01-15T18:17:55.626 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd86800640 1 -- 172.21.15.26:0/2121529089 learned_addr learned my addr 172.21.15.26:0/2121529089 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd7d200640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd880ae5a0 0x7fbd880cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd85e00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd8806d120 0x7fbd88155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd86800640 1 -- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd880ae5a0 msgr2=0x7fbd880cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd86800640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd880ae5a0 0x7fbd880cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd86800640 1 -- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd8806d120 msgr2=0x7fbd88155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd86800640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd8806d120 0x7fbd88155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd86800640 1 -- 172.21.15.26:0/2121529089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700094d0 con 0x7fbd880b3ae0 2024-01-15T18:17:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.622+0000 7fbd7d200640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd880ae5a0 0x7fbd880cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:55.627 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.623+0000 7fbd86800640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 0x7fbd88160f30 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000b970 tx=0x7fbd8000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.623+0000 7fbd7ea00640 1 -- 172.21.15.26:0/2121529089 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8000cb40 con 0x7fbd880b3ae0 2024-01-15T18:17:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.623+0000 7fbd7ea00640 1 -- 172.21.15.26:0/2121529089 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8000ccc0 con 0x7fbd880b3ae0 2024-01-15T18:17:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.623+0000 7fbd7ea00640 1 -- 172.21.15.26:0/2121529089 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80014680 con 0x7fbd880b3ae0 2024-01-15T18:17:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.623+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd881f6eb0 con 0x7fbd880b3ae0 2024-01-15T18:17:55.628 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.623+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd88155fb0 con 0x7fbd880b3ae0 2024-01-15T18:17:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.624+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd881f73d0 con 0x7fbd880b3ae0 2024-01-15T18:17:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.625+0000 7fbd7ea00640 1 -- 172.21.15.26:0/2121529089 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fbd8001d3b0 con 0x7fbd880b3ae0 2024-01-15T18:17:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.626+0000 7fbd7ea00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd6006c550 0x7fbd6006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.626+0000 7fbd7ea00640 1 -- 172.21.15.26:0/2121529089 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fbd80088b40 con 0x7fbd880b3ae0 2024-01-15T18:17:55.629 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.627+0000 7fbd85e00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd6006c550 0x7fbd6006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:55.630 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.628+0000 7fbd85e00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd6006c550 0x7fbd6006ea10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800a970 tx=0x7fbd78008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:55.633 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.631+0000 7fbd7ea00640 1 -- 172.21.15.26:0/2121529089 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd881f73d0 con 0x7fbd880b3ae0 2024-01-15T18:17:55.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.824+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd88071170 con 0x7fbd6006c550 2024-01-15T18:17:55.828 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.826+0000 7fbd7ea00640 1 -- 172.21.15.26:0/2121529089 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbd7800c050 con 0x7fbd6006c550 2024-01-15T18:17:55.828 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 1 to deep-scrub 2024-01-15T18:17:55.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.828+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd6006c550 msgr2=0x7fbd6006ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.828+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd6006c550 0x7fbd6006ea10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800a970 tx=0x7fbd78008040 comp rx=0 tx=0).stop 2024-01-15T18:17:55.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.828+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 msgr2=0x7fbd88160f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:55.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.828+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 0x7fbd88160f30 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000b970 tx=0x7fbd8000be30 comp rx=0 tx=0).stop 2024-01-15T18:17:55.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.828+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 shutdown_connections 2024-01-15T18:17:55.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.828+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fbd880ae5a0 0x7fbd880cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.828+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fbd6006c550 0x7fbd6006ea10 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.829+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fbd880b3ae0 0x7fbd88160f30 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.829+0000 7fbd8cc00640 1 --2- 172.21.15.26:0/2121529089 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fbd8806d120 0x7fbd88155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:55.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.829+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 >> 172.21.15.26:0/2121529089 conn(0x7fbd8806a800 msgr2=0x7fbd8806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:55.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.829+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 shutdown_connections 2024-01-15T18:17:55.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:55.829+0000 7fbd8cc00640 1 -- 172.21.15.26:0/2121529089 wait complete. 2024-01-15T18:17:55.851 INFO:tasks.ceph:Scrubbing osd.2 2024-01-15T18:17:55.851 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:17:56.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.073+0000 7f4609600640 1 Processor -- start 2024-01-15T18:17:56.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.073+0000 7f4609600640 1 -- start start 2024-01-15T18:17:56.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.075+0000 7f4609600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4604162430 0x7f46040b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.075+0000 7f4609600640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f46040b3650 0x7f460406c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.075+0000 7f4609600640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46040702e0 con 0x7f4604162810 2024-01-15T18:17:56.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.075+0000 7f4609600640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46040705c0 con 0x7f4604162430 2024-01-15T18:17:56.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.075+0000 7f4609600640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46040708a0 con 0x7f46040b3650 2024-01-15T18:17:56.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.075+0000 7f4603e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f4604162430 0x7f46040b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.075+0000 7f45fbe00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f4604162810 0x7f4604071240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56878/0 (socket says 172.21.15.26:56878) 2024-01-15T18:17:56.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.075+0000 7f45fbe00640 1 -- 172.21.15.26:0/3370834260 learned_addr learned my addr 172.21.15.26:0/3370834260 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:56.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.076+0000 7f4608c00640 1 --2- 172.21.15.26:0/3370834260 >> v2:172.21.15.124:3300/0 conn(0x7f46040b3650 0x7f460406c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.076+0000 7f4603e00640 1 -- 172.21.15.26:0/3370834260 >> v2:172.21.15.124:3300/0 conn(0x7f46040b3650 msgr2=0x7f460406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:56.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.076+0000 7f4603e00640 1 --2- 172.21.15.26:0/3370834260 >> v2:172.21.15.124:3300/0 conn(0x7f46040b3650 0x7f460406c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.076+0000 7f4603e00640 1 -- 172.21.15.26:0/3370834260 >> v1:172.21.15.26:6789/0 conn(0x7f4604162810 legacy=0x7f4604071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.076+0000 7f4603e00640 1 -- 172.21.15.26:0/3370834260 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604070b80 con 0x7f4604162430 2024-01-15T18:17:56.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.076+0000 7f4603400640 1 -- 172.21.15.26:0/3370834260 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341418527 0 0) 0x7f46040702e0 con 0x7f4604162810 2024-01-15T18:17:56.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.076+0000 7f4608c00640 1 --2- 172.21.15.26:0/3370834260 >> v2:172.21.15.124:3300/0 conn(0x7f46040b3650 0x7f460406c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:56.080 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.077+0000 7f4603e00640 1 --2- 172.21.15.26:0/3370834260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4604162430 0x7f46040b3110 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f4604161ad0 tx=0x7f45f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2991854c7b475b8d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.077+0000 7f4603400640 1 -- 172.21.15.26:0/3370834260 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4031830 con 0x7f4604162430 2024-01-15T18:17:56.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4603400640 1 -- 172.21.15.26:0/3370834260 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f4031990 con 0x7f4604162430 2024-01-15T18:17:56.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4603400640 1 -- 172.21.15.26:0/3370834260 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4031d20 con 0x7f4604162430 2024-01-15T18:17:56.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4609600640 1 -- 172.21.15.26:0/3370834260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4604162430 msgr2=0x7f46040b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4609600640 1 --2- 172.21.15.26:0/3370834260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4604162430 0x7f46040b3110 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f4604161ad0 tx=0x7f45f402b8d0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4609600640 1 -- 172.21.15.26:0/3370834260 shutdown_connections 2024-01-15T18:17:56.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4609600640 1 --2- 172.21.15.26:0/3370834260 >> v2:172.21.15.124:3300/0 conn(0x7f46040b3650 0x7f460406c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4609600640 1 --2- 172.21.15.26:0/3370834260 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f4604162430 0x7f46040b3110 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4609600640 1 -- 172.21.15.26:0/3370834260 >> 172.21.15.26:0/3370834260 conn(0x7f460406a800 msgr2=0x7f460406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:56.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4609600640 1 -- 172.21.15.26:0/3370834260 shutdown_connections 2024-01-15T18:17:56.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.078+0000 7f4609600640 1 -- 172.21.15.26:0/3370834260 wait complete. 2024-01-15T18:17:56.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.080+0000 7f4609600640 1 Processor -- start 2024-01-15T18:17:56.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.080+0000 7f4609600640 1 -- start start 2024-01-15T18:17:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.081+0000 7f4609600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040b3650 0x7f4604164130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.081+0000 7f4609600640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4604162430 0x7f4604153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.081+0000 7f4609600640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4604162c00 0x7f46041654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.081+0000 7f4609600640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4604153dd0 con 0x7f46040b3650 2024-01-15T18:17:56.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.081+0000 7f4609600640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4604163090 con 0x7f4604162430 2024-01-15T18:17:56.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.081+0000 7f4609600640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4604163370 con 0x7f4604162c00 2024-01-15T18:17:56.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.081+0000 7f4603e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040b3650 0x7f4604164130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.082+0000 7f4603e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040b3650 0x7f4604164130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56560/0 (socket says 172.21.15.26:56560) 2024-01-15T18:17:56.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.082+0000 7f4603e00640 1 -- 172.21.15.26:0/3479296724 learned_addr learned my addr 172.21.15.26:0/3479296724 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:56.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.082+0000 7f4603e00640 1 -- 172.21.15.26:0/3479296724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4604162c00 msgr2=0x7f46041654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.082+0000 7f4603e00640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f4604162c00 0x7f46041654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.082+0000 7f4603e00640 1 -- 172.21.15.26:0/3479296724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4604162430 msgr2=0x7f4604153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.082+0000 7f4603e00640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f4604162430 0x7f4604153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.086 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.082+0000 7f4603e00640 1 -- 172.21.15.26:0/3479296724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4009400 con 0x7f46040b3650 2024-01-15T18:17:56.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.082+0000 7f4603e00640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040b3650 0x7f4604164130 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009890 tx=0x7f45f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.083+0000 7f4600c00640 1 -- 172.21.15.26:0/3479296724 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f403ba70 con 0x7f46040b3650 2024-01-15T18:17:56.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.083+0000 7f4600c00640 1 -- 172.21.15.26:0/3479296724 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f403bbd0 con 0x7f46040b3650 2024-01-15T18:17:56.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.083+0000 7f4600c00640 1 -- 172.21.15.26:0/3479296724 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f40453f0 con 0x7f46040b3650 2024-01-15T18:17:56.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.083+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4604163650 con 0x7f46040b3650 2024-01-15T18:17:56.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.083+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4604163c10 con 0x7f46040b3650 2024-01-15T18:17:56.087 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.084+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4604067450 con 0x7f46040b3650 2024-01-15T18:17:56.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.085+0000 7f4600c00640 1 -- 172.21.15.26:0/3479296724 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f45f4045550 con 0x7f46040b3650 2024-01-15T18:17:56.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.086+0000 7f4600c00640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45dc06c550 0x7f45dc06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.086+0000 7f4600c00640 1 -- 172.21.15.26:0/3479296724 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f45f40ab720 con 0x7f46040b3650 2024-01-15T18:17:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.086+0000 7f4600c00640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f45dc0748d0 0x7f45dc076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.086+0000 7f4600c00640 1 -- 172.21.15.26:0/3479296724 --> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45dc077440 con 0x7f45dc0748d0 2024-01-15T18:17:56.089 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.086+0000 7f4600c00640 1 -- 172.21.15.26:0/3479296724 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45f40aba90 con 0x7f46040b3650 2024-01-15T18:17:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.087+0000 7f45fbe00640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45dc06c550 0x7f45dc06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.087+0000 7f4608c00640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f45dc0748d0 0x7f45dc076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.087+0000 7f4608c00640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f45dc0748d0 0x7f45dc076d70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.090 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.088+0000 7f45fbe00640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45dc06c550 0x7f45dc06ea10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f45ec007fd0 tx=0x7f45ec007960 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.089+0000 7f4600c00640 1 -- 172.21.15.26:0/3479296724 <== osd.2 v2:172.21.15.26:6816/1591707592 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f45dc077440 con 0x7f45dc0748d0 2024-01-15T18:17:56.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.104+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 --> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f46040cb5f0 con 0x7f45dc0748d0 2024-01-15T18:17:56.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.104+0000 7f4600c00640 1 -- 172.21.15.26:0/3479296724 <== osd.2 v2:172.21.15.26:6816/1591707592 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f46040cb5f0 con 0x7f45dc0748d0 2024-01-15T18:17:56.107 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:17:56.107 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:17:56.107 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:17:56.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f45dc0748d0 msgr2=0x7f45dc076d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.26:6816/1591707592,v1:172.21.15.26:6817/1591707592] conn(0x7f45dc0748d0 0x7f45dc076d70 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45dc06c550 msgr2=0x7f45dc06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f45dc06c550 0x7f45dc06ea10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f45ec007fd0 tx=0x7f45ec007960 comp rx=0 tx=0).stop 2024-01-15T18:17:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040b3650 msgr2=0x7f4604164130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 --2- 172.21.15.26:0/3479296724 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f46040b3650 0x7f4604164130 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009890 tx=0x7f45f40046c0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4608c00640 1 -- 172.21.15.26:0/3479296724 reap_dead start 2024-01-15T18:17:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 shutdown_connections 2024-01-15T18:17:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 >> 172.21.15.26:0/3479296724 conn(0x7f460406a800 msgr2=0x7f460406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 shutdown_connections 2024-01-15T18:17:56.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.105+0000 7f4609600640 1 -- 172.21.15.26:0/3479296724 wait complete. 2024-01-15T18:17:56.126 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-01-15T18:17:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.344+0000 7faefaa00640 1 Processor -- start 2024-01-15T18:17:56.346 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.344+0000 7faefaa00640 1 -- start start 2024-01-15T18:17:56.348 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.345+0000 7faefaa00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7faef41447f0 0x7faef415c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.345+0000 7faefaa00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faef40cccf0 0x7faef415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.345+0000 7faefaa00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef415c5c0 con 0x7faef406c470 2024-01-15T18:17:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.345+0000 7faefaa00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef406d690 con 0x7faef40cccf0 2024-01-15T18:17:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.345+0000 7faefaa00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef40cebb0 con 0x7faef41447f0 2024-01-15T18:17:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.346+0000 7faefa000640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faef40cccf0 0x7faef415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.349 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.346+0000 7faef8c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7faef406c470 0x7faef40cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56890/0 (socket says 172.21.15.26:56890) 2024-01-15T18:17:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.346+0000 7faef8c00640 1 -- 172.21.15.26:0/3762210726 learned_addr learned my addr 172.21.15.26:0/3762210726 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.346+0000 7faef9600640 1 --2- 172.21.15.26:0/3762210726 >> v2:172.21.15.124:3300/0 conn(0x7faef41447f0 0x7faef415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.346+0000 7faefa000640 1 -- 172.21.15.26:0/3762210726 >> v2:172.21.15.124:3300/0 conn(0x7faef41447f0 msgr2=0x7faef415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.346+0000 7faefa000640 1 --2- 172.21.15.26:0/3762210726 >> v2:172.21.15.124:3300/0 conn(0x7faef41447f0 0x7faef415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.346+0000 7faefa000640 1 -- 172.21.15.26:0/3762210726 >> v1:172.21.15.26:6789/0 conn(0x7faef406c470 legacy=0x7faef40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.346+0000 7faefa000640 1 -- 172.21.15.26:0/3762210726 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef406ca10 con 0x7faef40cccf0 2024-01-15T18:17:56.350 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.346+0000 7faef3e00640 1 -- 172.21.15.26:0/3762210726 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733863595 0 0) 0x7faef415c5c0 con 0x7faef406c470 2024-01-15T18:17:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.348+0000 7faefa000640 1 --2- 172.21.15.26:0/3762210726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef40cccf0 0x7faef415c7c0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7faef4072520 tx=0x7faeec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80da40b2ee617cf server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.348+0000 7faef3e00640 1 -- 172.21.15.26:0/3762210726 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec02cbf0 con 0x7faef40cccf0 2024-01-15T18:17:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.348+0000 7faef3e00640 1 -- 172.21.15.26:0/3762210726 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec02cd50 con 0x7faef40cccf0 2024-01-15T18:17:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.348+0000 7faef3e00640 1 -- 172.21.15.26:0/3762210726 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec034660 con 0x7faef40cccf0 2024-01-15T18:17:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.349+0000 7faefaa00640 1 -- 172.21.15.26:0/3762210726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef40cccf0 msgr2=0x7faef415c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.352 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.349+0000 7faefaa00640 1 --2- 172.21.15.26:0/3762210726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef40cccf0 0x7faef415c7c0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7faef4072520 tx=0x7faeec02be40 comp rx=0 tx=0).stop 2024-01-15T18:17:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.349+0000 7faefaa00640 1 -- 172.21.15.26:0/3762210726 shutdown_connections 2024-01-15T18:17:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.349+0000 7faefaa00640 1 --2- 172.21.15.26:0/3762210726 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef40cccf0 0x7faef415c7c0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.349+0000 7faefaa00640 1 --2- 172.21.15.26:0/3762210726 >> v2:172.21.15.124:3300/0 conn(0x7faef41447f0 0x7faef415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.349+0000 7faefaa00640 1 -- 172.21.15.26:0/3762210726 >> 172.21.15.26:0/3762210726 conn(0x7faef406a3e0 msgr2=0x7faef406a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.349+0000 7faefaa00640 1 -- 172.21.15.26:0/3762210726 shutdown_connections 2024-01-15T18:17:56.353 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.349+0000 7faefaa00640 1 -- 172.21.15.26:0/3762210726 wait complete. 2024-01-15T18:17:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.351+0000 7faefaa00640 1 Processor -- start 2024-01-15T18:17:56.354 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.351+0000 7faefaa00640 1 -- start start 2024-01-15T18:17:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faefaa00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faef406c470 0x7faef41f0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faefaa00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faef40cccf0 0x7faef41f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faefaa00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef41447f0 0x7faef41f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faefaa00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef40af010 con 0x7faef41447f0 2024-01-15T18:17:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faefaa00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef413eb40 con 0x7faef406c470 2024-01-15T18:17:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faefaa00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef413ee20 con 0x7faef40cccf0 2024-01-15T18:17:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faefa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef41447f0 0x7faef41f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faefa000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef41447f0 0x7faef41f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56580/0 (socket says 172.21.15.26:56580) 2024-01-15T18:17:56.355 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faefa000640 1 -- 172.21.15.26:0/1112447866 learned_addr learned my addr 172.21.15.26:0/1112447866 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:56.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faef8c00640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faef40cccf0 0x7faef41f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.352+0000 7faef9600640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faef406c470 0x7faef41f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faefa000640 1 -- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faef40cccf0 msgr2=0x7faef41f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faefa000640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faef40cccf0 0x7faef41f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.357 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faefa000640 1 -- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faef406c470 msgr2=0x7faef41f0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faefa000640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faef406c470 0x7faef41f0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faefa000640 1 -- 172.21.15.26:0/1112447866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec009a70 con 0x7faef41447f0 2024-01-15T18:17:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faefa000640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef41447f0 0x7faef41f42f0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7faeec037040 tx=0x7faeec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faef1600640 1 -- 172.21.15.26:0/1112447866 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec040650 con 0x7faef41447f0 2024-01-15T18:17:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faef1600640 1 -- 172.21.15.26:0/1112447866 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec0407b0 con 0x7faef41447f0 2024-01-15T18:17:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faef1600640 1 -- 172.21.15.26:0/1112447866 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec040b40 con 0x7faef41447f0 2024-01-15T18:17:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.353+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef413f100 con 0x7faef41447f0 2024-01-15T18:17:56.358 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.354+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef414e7d0 con 0x7faef41447f0 2024-01-15T18:17:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.355+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef40cc340 con 0x7faef41447f0 2024-01-15T18:17:56.359 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.356+0000 7faef1600640 1 -- 172.21.15.26:0/1112447866 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7faeec040ca0 con 0x7faef41447f0 2024-01-15T18:17:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.356+0000 7faef1600640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faecc06c550 0x7faecc06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.357+0000 7faef1600640 1 -- 172.21.15.26:0/1112447866 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7faeec0ab980 con 0x7faef41447f0 2024-01-15T18:17:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.357+0000 7faef9600640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faecc06c550 0x7faecc06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.360 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.358+0000 7faef9600640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faecc06c550 0x7faecc06ea10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7faee4006fd0 tx=0x7faee4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.363 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.361+0000 7faef1600640 1 -- 172.21.15.26:0/1112447866 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faeec078040 con 0x7faef41447f0 2024-01-15T18:17:56.565 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.562+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7faef40cea40 con 0x7faecc06c550 2024-01-15T18:17:56.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.564+0000 7faef1600640 1 -- 172.21.15.26:0/1112447866 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7faef40cea40 con 0x7faecc06c550 2024-01-15T18:17:56.567 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 2 to deep-scrub 2024-01-15T18:17:56.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faecc06c550 msgr2=0x7faecc06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faecc06c550 0x7faecc06ea10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7faee4006fd0 tx=0x7faee4008040 comp rx=0 tx=0).stop 2024-01-15T18:17:56.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef41447f0 msgr2=0x7faef41f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef41447f0 0x7faef41f42f0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7faeec037040 tx=0x7faeec004a40 comp rx=0 tx=0).stop 2024-01-15T18:17:56.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 shutdown_connections 2024-01-15T18:17:56.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faef40cccf0 0x7faef41f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faecc06c550 0x7faecc06ea10 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faef41447f0 0x7faef41f42f0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 --2- 172.21.15.26:0/1112447866 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faef406c470 0x7faef41f0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 >> 172.21.15.26:0/1112447866 conn(0x7faef406a3e0 msgr2=0x7faef4142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:56.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 shutdown_connections 2024-01-15T18:17:56.571 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.567+0000 7faefaa00640 1 -- 172.21.15.26:0/1112447866 wait complete. 2024-01-15T18:17:56.589 INFO:tasks.ceph:Scrubbing osd.3 2024-01-15T18:17:56.590 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:17:56.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.817+0000 7f9139a00640 1 Processor -- start 2024-01-15T18:17:56.820 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.818+0000 7f9139a00640 1 -- start start 2024-01-15T18:17:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.819+0000 7f9139a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91340d0100 0x7f91340b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.819+0000 7f9139a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f91340d0f40 0x7f91340d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.819+0000 7f9139a00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91340d0a10 con 0x7f9134160920 2024-01-15T18:17:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.819+0000 7f9139a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91340cf080 con 0x7f91340d0f40 2024-01-15T18:17:56.822 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.819+0000 7f9139a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340c6be0 con 0x7f91340d0100 2024-01-15T18:17:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.819+0000 7f9133e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9134160920 0x7f91340cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56892/0 (socket says 172.21.15.26:56892) 2024-01-15T18:17:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.819+0000 7f9133e00640 1 -- 172.21.15.26:0/153237653 learned_addr learned my addr 172.21.15.26:0/153237653 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.819+0000 7f9139000640 1 --2- 172.21.15.26:0/153237653 >> v2:172.21.15.26:3300/0 conn(0x7f91340d0f40 0x7f91340d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.819+0000 7f912be00640 1 --2- 172.21.15.26:0/153237653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91340d0100 0x7f91340b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.820+0000 7f9139000640 1 -- 172.21.15.26:0/153237653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91340d0100 msgr2=0x7f91340b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:56.823 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.820+0000 7f9139000640 1 --2- 172.21.15.26:0/153237653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91340d0100 0x7f91340b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.820+0000 7f9133400640 1 -- 172.21.15.26:0/153237653 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018264489 0 0) 0x7f91340d0a10 con 0x7f9134160920 2024-01-15T18:17:56.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.820+0000 7f9139000640 1 -- 172.21.15.26:0/153237653 >> v1:172.21.15.26:6789/0 conn(0x7f9134160920 legacy=0x7f91340cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.824 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.820+0000 7f9139000640 1 -- 172.21.15.26:0/153237653 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91340ce7e0 con 0x7f91340d0f40 2024-01-15T18:17:56.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.821+0000 7f9139000640 1 --2- 172.21.15.26:0/153237653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0f40 0x7f91340d13f0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f912c009d90 tx=0x7f912c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7110d9764e5e163 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.821+0000 7f9133400640 1 -- 172.21.15.26:0/153237653 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c02cc50 con 0x7f91340d0f40 2024-01-15T18:17:56.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.821+0000 7f9133400640 1 -- 172.21.15.26:0/153237653 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c02cdb0 con 0x7f91340d0f40 2024-01-15T18:17:56.825 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.821+0000 7f9133400640 1 -- 172.21.15.26:0/153237653 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c034660 con 0x7f91340d0f40 2024-01-15T18:17:56.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.822+0000 7f9139a00640 1 -- 172.21.15.26:0/153237653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0f40 msgr2=0x7f91340d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.822+0000 7f9139a00640 1 --2- 172.21.15.26:0/153237653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0f40 0x7f91340d13f0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f912c009d90 tx=0x7f912c02bea0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.822+0000 7f9139a00640 1 -- 172.21.15.26:0/153237653 shutdown_connections 2024-01-15T18:17:56.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.822+0000 7f9139a00640 1 --2- 172.21.15.26:0/153237653 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0f40 0x7f91340d13f0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.822+0000 7f9139a00640 1 --2- 172.21.15.26:0/153237653 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91340d0100 0x7f91340b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.822+0000 7f9139a00640 1 -- 172.21.15.26:0/153237653 >> 172.21.15.26:0/153237653 conn(0x7f91340d3090 msgr2=0x7f91340d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:56.826 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.822+0000 7f9139a00640 1 -- 172.21.15.26:0/153237653 shutdown_connections 2024-01-15T18:17:56.827 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.822+0000 7f9139a00640 1 -- 172.21.15.26:0/153237653 wait complete. 2024-01-15T18:17:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.824+0000 7f9139a00640 1 Processor -- start 2024-01-15T18:17:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.824+0000 7f9139a00640 1 -- start start 2024-01-15T18:17:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9139a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0100 0x7f913415a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9139a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91340d0f40 0x7f91340afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9139a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9134160920 0x7f91340b8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9139a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913415abc0 con 0x7f91340d0100 2024-01-15T18:17:56.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9139a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340cc210 con 0x7f9134160920 2024-01-15T18:17:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9139a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340cb600 con 0x7f91340d0f40 2024-01-15T18:17:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9133e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0100 0x7f913415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9133e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0100 0x7f913415a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56600/0 (socket says 172.21.15.26:56600) 2024-01-15T18:17:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9133e00640 1 -- 172.21.15.26:0/1264870433 learned_addr learned my addr 172.21.15.26:0/1264870433 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f912be00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91340d0f40 0x7f91340afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9133e00640 1 -- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91340d0f40 msgr2=0x7f91340afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9133e00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f91340d0f40 0x7f91340afc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9133e00640 1 -- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9134160920 msgr2=0x7f91340b8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9133e00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9134160920 0x7f91340b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.825+0000 7f9133e00640 1 -- 172.21.15.26:0/1264870433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c009a70 con 0x7f91340d0100 2024-01-15T18:17:56.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.826+0000 7f9133e00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0100 0x7f913415a680 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f912400c4d0 tx=0x7f912400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.826+0000 7f9130c00640 1 -- 172.21.15.26:0/1264870433 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91240074e0 con 0x7f91340d0100 2024-01-15T18:17:56.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.826+0000 7f9130c00640 1 -- 172.21.15.26:0/1264870433 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9124007660 con 0x7f91340d0100 2024-01-15T18:17:56.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.826+0000 7f9130c00640 1 -- 172.21.15.26:0/1264870433 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124004a20 con 0x7f91340d0100 2024-01-15T18:17:56.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.826+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91340cb8e0 con 0x7f91340d0100 2024-01-15T18:17:56.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.826+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91340cbea0 con 0x7f91340d0100 2024-01-15T18:17:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.827+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f91340ceda0 con 0x7f91340d0100 2024-01-15T18:17:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.829+0000 7f9130c00640 1 -- 172.21.15.26:0/1264870433 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f9124004b80 con 0x7f91340d0100 2024-01-15T18:17:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.829+0000 7f9130c00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06c5d0 0x7f910c06ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.830+0000 7f9130c00640 1 -- 172.21.15.26:0/1264870433 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f91240881e0 con 0x7f91340d0100 2024-01-15T18:17:56.833 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.830+0000 7f9130c00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f910c0749d0 0x7f910c076e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.830+0000 7f9130c00640 1 -- 172.21.15.26:0/1264870433 --> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f910c077540 con 0x7f910c0749d0 2024-01-15T18:17:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.830+0000 7f9130c00640 1 -- 172.21.15.26:0/1264870433 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9124088cf0 con 0x7f91340d0100 2024-01-15T18:17:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.830+0000 7f912be00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06c5d0 0x7f910c06ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.830+0000 7f9139000640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f910c0749d0 0x7f910c076e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.831+0000 7f9139000640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f910c0749d0 0x7f910c076e70 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.831+0000 7f912be00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06c5d0 0x7f910c06ea90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f911c006fd0 tx=0x7f911c008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:56.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.832+0000 7f9130c00640 1 -- 172.21.15.26:0/1264870433 <== osd.3 v2:172.21.15.26:6808/696680319 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f910c077540 con 0x7f910c0749d0 2024-01-15T18:17:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.847+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 --> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f91340d3bb0 con 0x7f910c0749d0 2024-01-15T18:17:56.849 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.847+0000 7f9130c00640 1 -- 172.21.15.26:0/1264870433 <== osd.3 v2:172.21.15.26:6808/696680319 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f912c02ca50 con 0x7f910c0749d0 2024-01-15T18:17:56.850 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:17:56.850 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:17:56.850 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:17:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.847+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f910c0749d0 msgr2=0x7f910c076e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.847+0000 7f9139a00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.26:6808/696680319,v1:172.21.15.26:6809/696680319] conn(0x7f910c0749d0 0x7f910c076e70 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.848+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06c5d0 msgr2=0x7f910c06ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.851 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.848+0000 7f9139a00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f910c06c5d0 0x7f910c06ea90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f911c006fd0 tx=0x7f911c008040 comp rx=0 tx=0).stop 2024-01-15T18:17:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.848+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0100 msgr2=0x7f913415a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.848+0000 7f9139a00640 1 --2- 172.21.15.26:0/1264870433 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f91340d0100 0x7f913415a680 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f912400c4d0 tx=0x7f912400c990 comp rx=0 tx=0).stop 2024-01-15T18:17:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.848+0000 7f9139000640 1 -- 172.21.15.26:0/1264870433 reap_dead start 2024-01-15T18:17:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.848+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 shutdown_connections 2024-01-15T18:17:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.848+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 >> 172.21.15.26:0/1264870433 conn(0x7f91340d3090 msgr2=0x7f913415ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.848+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 shutdown_connections 2024-01-15T18:17:56.852 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:56.848+0000 7f9139a00640 1 -- 172.21.15.26:0/1264870433 wait complete. 2024-01-15T18:17:56.869 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-01-15T18:17:57.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.088+0000 7f5bc3400640 1 Processor -- start 2024-01-15T18:17:57.091 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.088+0000 7f5bc3400640 1 -- start start 2024-01-15T18:17:57.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.089+0000 7f5bc3400640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5bbc1607a0 0x7f5bbc0b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.089+0000 7f5bc3400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bbc160b80 0x7f5bbc1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.089+0000 7f5bc3400640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc0b2b90 con 0x7f5bbc0b3eb0 2024-01-15T18:17:57.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.089+0000 7f5bc3400640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc0d21d0 con 0x7f5bbc1607a0 2024-01-15T18:17:57.095 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.090+0000 7f5bc3400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0b2e30 con 0x7f5bbc160b80 2024-01-15T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.090+0000 7f5bc1600640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5bbc0b3eb0 0x7f5bbc1610c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56898/0 (socket says 172.21.15.26:56898) 2024-01-15T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.090+0000 7f5bc1600640 1 -- 172.21.15.26:0/48497426 learned_addr learned my addr 172.21.15.26:0/48497426 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.090+0000 7f5bc2a00640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bbc160b80 0x7f5bbc1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.090+0000 7f5bc2000640 1 --2- 172.21.15.26:0/48497426 >> v2:172.21.15.124:3300/0 conn(0x7f5bbc1607a0 0x7f5bbc0b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.090+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285395495 0 0) 0x7f5bbc0b2b90 con 0x7f5bbc0b3eb0 2024-01-15T18:17:57.096 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b9c003520 con 0x7f5bbc0b3eb0 2024-01-15T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2936598719 0 0) 0x7f5b9c003520 con 0x7f5bbc0b3eb0 2024-01-15T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bbc160b80 msgr2=0x7f5bbc1617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bbc160b80 0x7f5bbc1617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 >> v2:172.21.15.124:3300/0 conn(0x7f5bbc1607a0 msgr2=0x7f5bbc0b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 --2- 172.21.15.26:0/48497426 >> v2:172.21.15.124:3300/0 conn(0x7f5bbc1607a0 0x7f5bbc0b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.097 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0b3110 con 0x7f5bbc0b3eb0 2024-01-15T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f5bac0019f0 con 0x7f5bbc0b3eb0 2024-01-15T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 >> v1:172.21.15.26:6789/0 conn(0x7f5bbc0b3eb0 legacy=0x7f5bbc1610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bbc160b80 0x7f5b9c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b9c003f40 0x7f5b9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9c004d00 0x7f5b9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.098 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0d21d0 con 0x7f5b9c004d00 2024-01-15T18:17:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0b2e30 con 0x7f5b9c003f40 2024-01-15T18:17:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.091+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c00b980 con 0x7f5bbc160b80 2024-01-15T18:17:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.092+0000 7f5bc2a00640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9c004d00 0x7f5b9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.092+0000 7f5bc2000640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bbc160b80 0x7f5b9c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.092+0000 7f5bc2a00640 1 -- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bbc160b80 msgr2=0x7f5b9c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:57.099 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.092+0000 7f5bc2a00640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5bbc160b80 0x7f5b9c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.092+0000 7f5bc2a00640 1 -- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b9c003f40 msgr2=0x7f5b9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.092+0000 7f5bc2a00640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b9c003f40 0x7f5b9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.100 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.092+0000 7f5bc2a00640 1 -- 172.21.15.26:0/48497426 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c003520 con 0x7f5b9c004d00 2024-01-15T18:17:57.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.093+0000 7f5bc2a00640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9c004d00 0x7f5b9c005180 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009af0 tx=0x7f5b9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667685424474fc6c server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.093+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb402c6b0 con 0x7f5b9c004d00 2024-01-15T18:17:57.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.093+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb402c810 con 0x7f5b9c004d00 2024-01-15T18:17:57.104 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.093+0000 7f5bc0c00640 1 -- 172.21.15.26:0/48497426 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb402cba0 con 0x7f5b9c004d00 2024-01-15T18:17:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.094+0000 7f5bc3400640 1 -- 172.21.15.26:0/48497426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9c004d00 msgr2=0x7f5b9c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.094+0000 7f5bc3400640 1 --2- 172.21.15.26:0/48497426 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b9c004d00 0x7f5b9c005180 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009af0 tx=0x7f5b9c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.094+0000 7f5bc2a00640 1 -- 172.21.15.26:0/48497426 reap_dead start 2024-01-15T18:17:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.094+0000 7f5bc3400640 1 -- 172.21.15.26:0/48497426 shutdown_connections 2024-01-15T18:17:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.094+0000 7f5bc3400640 1 -- 172.21.15.26:0/48497426 >> 172.21.15.26:0/48497426 conn(0x7f5bbc09c590 msgr2=0x7f5bbc0d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:57.105 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.094+0000 7f5bc3400640 1 -- 172.21.15.26:0/48497426 shutdown_connections 2024-01-15T18:17:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.094+0000 7f5bc3400640 1 -- 172.21.15.26:0/48497426 wait complete. 2024-01-15T18:17:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.096+0000 7f5bc3400640 1 Processor -- start 2024-01-15T18:17:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.096+0000 7f5bc3400640 1 -- start start 2024-01-15T18:17:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.097+0000 7f5bc3400640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b9c004d00 0x7f5bbc0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.097+0000 7f5bc3400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc15a3a0 0x7f5bbc0b8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.106 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.097+0000 7f5bc3400640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bbc1e69c0 0x7f5bbc0b8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.097+0000 7f5bc3400640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc1e8c70 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.097+0000 7f5bc3400640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0b6370 con 0x7f5bbc1e69c0 2024-01-15T18:17:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.097+0000 7f5bc3400640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc1e8e80 con 0x7f5b9c004d00 2024-01-15T18:17:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.097+0000 7f5bc1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc15a3a0 0x7f5bbc0b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.097+0000 7f5bc1600640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc15a3a0 0x7f5bbc0b8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56610/0 (socket says 172.21.15.26:56610) 2024-01-15T18:17:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.097+0000 7f5bc1600640 1 -- 172.21.15.26:0/1427492191 learned_addr learned my addr 172.21.15.26:0/1427492191 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:57.107 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bc1600640 1 -- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b9c004d00 msgr2=0x7f5bbc0b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bc1600640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b9c004d00 0x7f5bbc0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bc1600640 1 -- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bbc1e69c0 msgr2=0x7f5bbc0b8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bc1600640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bbc1e69c0 0x7f5bbc0b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bc1600640 1 -- 172.21.15.26:0/1427492191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc1e9160 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bc1600640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc15a3a0 0x7f5bbc0b8010 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00caa0 tx=0x7f5bac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.108 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bba000640 1 -- 172.21.15.26:0/1427492191 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00a7f0 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bba000640 1 -- 172.21.15.26:0/1427492191 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac00a970 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bba000640 1 -- 172.21.15.26:0/1427492191 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac00ad20 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.098+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc1e9440 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.099+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc0b6740 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.100+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc0b3bf0 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.109 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.101+0000 7f5bba000640 1 -- 172.21.15.26:0/1427492191 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f5bac0070d0 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.110 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.102+0000 7f5bba000640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b9406c580 0x7f5b9406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.102+0000 7f5bba000640 1 -- 172.21.15.26:0/1427492191 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f5bac088270 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.103+0000 7f5bc2000640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b9406c580 0x7f5b9406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.104+0000 7f5bc2000640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b9406c580 0x7f5b9406ea40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009950 tx=0x7f5ba4008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.111 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.106+0000 7f5bba000640 1 -- 172.21.15.26:0/1427492191 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5bac0547b0 con 0x7f5bbc15a3a0 2024-01-15T18:17:57.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.299+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bbc0b39b0 con 0x7f5b9406c580 2024-01-15T18:17:57.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.301+0000 7f5bba000640 1 -- 172.21.15.26:0/1427492191 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5ba40095c0 con 0x7f5b9406c580 2024-01-15T18:17:57.305 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 3 to deep-scrub 2024-01-15T18:17:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.303+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b9406c580 msgr2=0x7f5b9406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.303+0000 7f5bc3400640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b9406c580 0x7f5b9406ea40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009950 tx=0x7f5ba4008040 comp rx=0 tx=0).stop 2024-01-15T18:17:57.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.303+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc15a3a0 msgr2=0x7f5bbc0b8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.303+0000 7f5bc3400640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc15a3a0 0x7f5bbc0b8010 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00caa0 tx=0x7f5bac00cf60 comp rx=0 tx=0).stop 2024-01-15T18:17:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.304+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 shutdown_connections 2024-01-15T18:17:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.304+0000 7f5bc3400640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b9406c580 0x7f5b9406ea40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.304+0000 7f5bc3400640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5bbc1e69c0 0x7f5bbc0b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.304+0000 7f5bc3400640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5bbc15a3a0 0x7f5bbc0b8010 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.304+0000 7f5bc3400640 1 --2- 172.21.15.26:0/1427492191 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b9c004d00 0x7f5bbc0b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.304+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 >> 172.21.15.26:0/1427492191 conn(0x7f5bbc09c590 msgr2=0x7f5bbc06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.304+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 shutdown_connections 2024-01-15T18:17:57.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.304+0000 7f5bc3400640 1 -- 172.21.15.26:0/1427492191 wait complete. 2024-01-15T18:17:57.328 INFO:tasks.ceph:Scrubbing osd.4 2024-01-15T18:17:57.328 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:17:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.551+0000 7f5783200640 1 Processor -- start 2024-01-15T18:17:57.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.551+0000 7f5783200640 1 -- start start 2024-01-15T18:17:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f5783200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5784071410 0x7f5784146770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f5783200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f578406c7c0 0x7f57840717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f5783200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f578406cc50 con 0x7f5784071d30 2024-01-15T18:17:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f5783200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578406cec0 con 0x7f578406c7c0 2024-01-15T18:17:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f5783200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f578406d1a0 con 0x7f5784071410 2024-01-15T18:17:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f577be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f578406c7c0 0x7f57840717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f577be00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f578406c7c0 0x7f57840717f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56620/0 (socket says 172.21.15.26:56620) 2024-01-15T18:17:57.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f577be00640 1 -- 172.21.15.26:0/4041171577 learned_addr learned my addr 172.21.15.26:0/4041171577 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f577be00640 1 -- 172.21.15.26:0/4041171577 >> v2:172.21.15.124:3300/0 conn(0x7f5784071410 msgr2=0x7f5784146770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f577be00640 1 --2- 172.21.15.26:0/4041171577 >> v2:172.21.15.124:3300/0 conn(0x7f5784071410 0x7f5784146770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f577be00640 1 -- 172.21.15.26:0/4041171577 >> v1:172.21.15.124:6789/0 conn(0x7f5784071d30 legacy=0x7f57840b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:57.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.553+0000 7f577be00640 1 -- 172.21.15.26:0/4041171577 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578406d480 con 0x7f578406c7c0 2024-01-15T18:17:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.555+0000 7f577be00640 1 --2- 172.21.15.26:0/4041171577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f578406c7c0 0x7f57840717f0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f57840ad5f0 tx=0x7f576c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a6c2d279e6bf9a server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.555+0000 7f5781400640 1 -- 172.21.15.26:0/4041171577 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c031870 con 0x7f578406c7c0 2024-01-15T18:17:57.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.555+0000 7f5781400640 1 -- 172.21.15.26:0/4041171577 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c0319d0 con 0x7f578406c7c0 2024-01-15T18:17:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.555+0000 7f5781400640 1 -- 172.21.15.26:0/4041171577 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c031d60 con 0x7f578406c7c0 2024-01-15T18:17:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.556+0000 7f5783200640 1 -- 172.21.15.26:0/4041171577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f578406c7c0 msgr2=0x7f57840717f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.556+0000 7f5783200640 1 --2- 172.21.15.26:0/4041171577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f578406c7c0 0x7f57840717f0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f57840ad5f0 tx=0x7f576c02b910 comp rx=0 tx=0).stop 2024-01-15T18:17:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.556+0000 7f5783200640 1 -- 172.21.15.26:0/4041171577 shutdown_connections 2024-01-15T18:17:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.556+0000 7f5783200640 1 --2- 172.21.15.26:0/4041171577 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f578406c7c0 0x7f57840717f0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.556+0000 7f5783200640 1 --2- 172.21.15.26:0/4041171577 >> v2:172.21.15.124:3300/0 conn(0x7f5784071410 0x7f5784146770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.556+0000 7f5783200640 1 -- 172.21.15.26:0/4041171577 >> 172.21.15.26:0/4041171577 conn(0x7f578406a800 msgr2=0x7f578406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.556+0000 7f5783200640 1 -- 172.21.15.26:0/4041171577 shutdown_connections 2024-01-15T18:17:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.556+0000 7f5783200640 1 -- 172.21.15.26:0/4041171577 wait complete. 2024-01-15T18:17:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.558+0000 7f5783200640 1 Processor -- start 2024-01-15T18:17:57.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.558+0000 7f5783200640 1 -- start start 2024-01-15T18:17:57.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5783200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f578406c7c0 0x7f578415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5783200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5784071410 0x7f578415d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5783200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5784071d30 0x7f57840c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5783200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57840c46a0 con 0x7f5784071d30 2024-01-15T18:17:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5783200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57841f40d0 con 0x7f578406c7c0 2024-01-15T18:17:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5783200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57840c3d70 con 0x7f5784071410 2024-01-15T18:17:57.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5782800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5784071d30 0x7f57840c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5782800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5784071d30 0x7f57840c4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56622/0 (socket says 172.21.15.26:56622) 2024-01-15T18:17:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5782800640 1 -- 172.21.15.26:0/2470498907 learned_addr learned my addr 172.21.15.26:0/2470498907 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f5781e00640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f578406c7c0 0x7f578415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.559+0000 7f577be00640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5784071410 0x7f578415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f5782800640 1 -- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5784071410 msgr2=0x7f578415d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f5782800640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5784071410 0x7f578415d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f5782800640 1 -- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f578406c7c0 msgr2=0x7f578415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f5782800640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f578406c7c0 0x7f578415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f5782800640 1 -- 172.21.15.26:0/2470498907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0094d0 con 0x7f5784071d30 2024-01-15T18:17:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f577be00640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5784071410 0x7f578415d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f5782800640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5784071d30 0x7f57840c4b30 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f577c00b990 tx=0x7f577c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f577a000640 1 -- 172.21.15.26:0/2470498907 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c00cba0 con 0x7f5784071d30 2024-01-15T18:17:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f577a000640 1 -- 172.21.15.26:0/2470498907 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c00cd00 con 0x7f5784071d30 2024-01-15T18:17:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57840c4050 con 0x7f5784071d30 2024-01-15T18:17:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f577a000640 1 -- 172.21.15.26:0/2470498907 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c014680 con 0x7f5784071d30 2024-01-15T18:17:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.560+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578415cde0 con 0x7f5784071d30 2024-01-15T18:17:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.561+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5784072180 con 0x7f5784071d30 2024-01-15T18:17:57.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.562+0000 7f577a000640 1 -- 172.21.15.26:0/2470498907 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f577c01d3b0 con 0x7f5784071d30 2024-01-15T18:17:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.563+0000 7f577a000640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f575c06c550 0x7f575c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.572 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.563+0000 7f577a000640 1 -- 172.21.15.26:0/2470498907 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f577c088b40 con 0x7f5784071d30 2024-01-15T18:17:57.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.564+0000 7f577a000640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7f575c0748d0 0x7f575c076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.564+0000 7f577a000640 1 -- 172.21.15.26:0/2470498907 --> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f575c077440 con 0x7f575c0748d0 2024-01-15T18:17:57.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.564+0000 7f577a000640 1 -- 172.21.15.26:0/2470498907 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7f577c089760 con 0x7f5784071d30 2024-01-15T18:17:57.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.564+0000 7f5781e00640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f575c06c550 0x7f575c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.573 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.564+0000 7f577be00640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7f575c0748d0 0x7f575c076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.565+0000 7f5781e00640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f575c06c550 0x7f575c06ea10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5774009890 tx=0x7f5774008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.566+0000 7f577be00640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7f575c0748d0 0x7f575c076d70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.568+0000 7f577a000640 1 -- 172.21.15.26:0/2470498907 <== osd.4 v2:172.21.15.124:6800/126047302 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f575c077440 con 0x7f575c0748d0 2024-01-15T18:17:57.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.583+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 --> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f578406da40 con 0x7f575c0748d0 2024-01-15T18:17:57.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.584+0000 7f577a000640 1 -- 172.21.15.26:0/2470498907 <== osd.4 v2:172.21.15.124:6800/126047302 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f576c031bc0 con 0x7f575c0748d0 2024-01-15T18:17:57.588 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:17:57.588 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:17:57.588 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:17:57.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.584+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7f575c0748d0 msgr2=0x7f575c076d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.584+0000 7f5783200640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6800/126047302,v1:172.21.15.124:6801/126047302] conn(0x7f575c0748d0 0x7f575c076d70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.585+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f575c06c550 msgr2=0x7f575c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.585+0000 7f5783200640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f575c06c550 0x7f575c06ea10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5774009890 tx=0x7f5774008040 comp rx=0 tx=0).stop 2024-01-15T18:17:57.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.585+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5784071d30 msgr2=0x7f57840c4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.585+0000 7f5783200640 1 --2- 172.21.15.26:0/2470498907 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5784071d30 0x7f57840c4b30 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f577c00b990 tx=0x7f577c00be50 comp rx=0 tx=0).stop 2024-01-15T18:17:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.585+0000 7f5782800640 1 -- 172.21.15.26:0/2470498907 reap_dead start 2024-01-15T18:17:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.585+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 shutdown_connections 2024-01-15T18:17:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.585+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 >> 172.21.15.26:0/2470498907 conn(0x7f578406a800 msgr2=0x7f578406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.585+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 shutdown_connections 2024-01-15T18:17:57.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.585+0000 7f5783200640 1 -- 172.21.15.26:0/2470498907 wait complete. 2024-01-15T18:17:57.606 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-01-15T18:17:57.829 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.825+0000 7fc67b200640 1 Processor -- start 2024-01-15T18:17:57.830 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.826+0000 7fc67b200640 1 -- start start 2024-01-15T18:17:57.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.827+0000 7fc67b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc674160920 0x7fc6740b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.827+0000 7fc67b200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fc6740cfe70 0x7fc674161720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.831 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.827+0000 7fc67b200640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6740d0a10 con 0x7fc6740cf9f0 2024-01-15T18:17:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.827+0000 7fc67b200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6740d1f50 con 0x7fc6740cfe70 2024-01-15T18:17:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.827+0000 7fc67b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740cf080 con 0x7fc674160920 2024-01-15T18:17:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.828+0000 7fc679e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc674160920 0x7fc6740b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.828+0000 7fc679e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc674160920 0x7fc6740b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.124:3301/0 says I am v2:172.21.15.26:43438/0 (socket says 172.21.15.26:43438) 2024-01-15T18:17:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.828+0000 7fc679e00640 1 -- 172.21.15.26:0/306605670 learned_addr learned my addr 172.21.15.26:0/306605670 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:57.832 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.828+0000 7fc67a800640 1 --2- 172.21.15.26:0/306605670 >> v2:172.21.15.124:3300/0 conn(0x7fc6740cfe70 0x7fc674161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.828+0000 7fc679e00640 1 -- 172.21.15.26:0/306605670 >> v2:172.21.15.124:3300/0 conn(0x7fc6740cfe70 msgr2=0x7fc674161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.828+0000 7fc679e00640 1 --2- 172.21.15.26:0/306605670 >> v2:172.21.15.124:3300/0 conn(0x7fc6740cfe70 0x7fc674161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.828+0000 7fc679e00640 1 -- 172.21.15.26:0/306605670 >> v1:172.21.15.124:6789/0 conn(0x7fc6740cf9f0 legacy=0x7fc674161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.829+0000 7fc679e00640 1 -- 172.21.15.26:0/306605670 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6740c6be0 con 0x7fc674160920 2024-01-15T18:17:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.829+0000 7fc678a00640 1 -- 172.21.15.26:0/306605670 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780251103 0 0) 0x7fc6740d0a10 con 0x7fc6740cf9f0 2024-01-15T18:17:57.834 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.830+0000 7fc679e00640 1 --2- 172.21.15.26:0/306605670 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc674160920 0x7fc6740b3f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc664009870 tx=0x7fc66402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a81c8d9934b8f2db server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.830+0000 7fc678a00640 1 -- 172.21.15.26:0/306605670 <== mon.2 v2:172.21.15.124:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664030840 con 0x7fc674160920 2024-01-15T18:17:57.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.830+0000 7fc678a00640 1 -- 172.21.15.26:0/306605670 <== mon.2 v2:172.21.15.124:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6640309a0 con 0x7fc674160920 2024-01-15T18:17:57.835 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.830+0000 7fc678a00640 1 -- 172.21.15.26:0/306605670 <== mon.2 v2:172.21.15.124:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664030d30 con 0x7fc674160920 2024-01-15T18:17:57.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.831+0000 7fc67b200640 1 -- 172.21.15.26:0/306605670 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc674160920 msgr2=0x7fc6740b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.836 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.831+0000 7fc67b200640 1 --2- 172.21.15.26:0/306605670 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc674160920 0x7fc6740b3f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc664009870 tx=0x7fc66402b980 comp rx=0 tx=0).stop 2024-01-15T18:17:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.831+0000 7fc67b200640 1 -- 172.21.15.26:0/306605670 shutdown_connections 2024-01-15T18:17:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.831+0000 7fc67b200640 1 --2- 172.21.15.26:0/306605670 >> v2:172.21.15.124:3300/0 conn(0x7fc6740cfe70 0x7fc674161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.831+0000 7fc67b200640 1 --2- 172.21.15.26:0/306605670 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc674160920 0x7fc6740b3f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.831+0000 7fc67b200640 1 -- 172.21.15.26:0/306605670 >> 172.21.15.26:0/306605670 conn(0x7fc6740d3090 msgr2=0x7fc6740d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.831+0000 7fc67b200640 1 -- 172.21.15.26:0/306605670 shutdown_connections 2024-01-15T18:17:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.831+0000 7fc67b200640 1 -- 172.21.15.26:0/306605670 wait complete. 2024-01-15T18:17:57.837 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.833+0000 7fc67b200640 1 Processor -- start 2024-01-15T18:17:57.838 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.833+0000 7fc67b200640 1 -- start start 2024-01-15T18:17:57.840 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.834+0000 7fc67b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6740cf9f0 0x7fc674150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.834+0000 7fc67b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6740cfe70 0x7fc6740cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.834+0000 7fc67b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674160920 0x7fc6740c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.834+0000 7fc67b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740b64b0 con 0x7fc674160920 2024-01-15T18:17:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.834+0000 7fc67b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc674150bd0 con 0x7fc6740cf9f0 2024-01-15T18:17:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.834+0000 7fc67b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6740cd060 con 0x7fc6740cfe70 2024-01-15T18:17:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.834+0000 7fc67a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674160920 0x7fc6740c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.841 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.834+0000 7fc67a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674160920 0x7fc6740c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56638/0 (socket says 172.21.15.26:56638) 2024-01-15T18:17:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.834+0000 7fc67a800640 1 -- 172.21.15.26:0/1049474239 learned_addr learned my addr 172.21.15.26:0/1049474239 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.835+0000 7fc679e00640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6740cf9f0 0x7fc674150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.835+0000 7fc679400640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6740cfe70 0x7fc6740cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.835+0000 7fc67a800640 1 -- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6740cfe70 msgr2=0x7fc6740cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.835+0000 7fc67a800640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6740cfe70 0x7fc6740cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.842 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.835+0000 7fc67a800640 1 -- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6740cf9f0 msgr2=0x7fc674150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.835+0000 7fc67a800640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6740cf9f0 0x7fc674150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.835+0000 7fc67a800640 1 -- 172.21.15.26:0/1049474239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6640094d0 con 0x7fc674160920 2024-01-15T18:17:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.835+0000 7fc679e00640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6740cf9f0 0x7fc674150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:17:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.835+0000 7fc67a800640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674160920 0x7fc6740c88b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00efc0 tx=0x7fc66c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.836+0000 7fc672000640 1 -- 172.21.15.26:0/1049474239 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c00ced0 con 0x7fc674160920 2024-01-15T18:17:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.836+0000 7fc672000640 1 -- 172.21.15.26:0/1049474239 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c00f040 con 0x7fc674160920 2024-01-15T18:17:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.836+0000 7fc672000640 1 -- 172.21.15.26:0/1049474239 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0146c0 con 0x7fc674160920 2024-01-15T18:17:57.843 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.836+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6740ccd80 con 0x7fc674160920 2024-01-15T18:17:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.836+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6740b60d0 con 0x7fc674160920 2024-01-15T18:17:57.844 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.837+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6740b65f0 con 0x7fc674160920 2024-01-15T18:17:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.838+0000 7fc672000640 1 -- 172.21.15.26:0/1049474239 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fc66c014820 con 0x7fc674160920 2024-01-15T18:17:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.839+0000 7fc672000640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc64c06c550 0x7fc64c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.839+0000 7fc672000640 1 -- 172.21.15.26:0/1049474239 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fc66c089a70 con 0x7fc674160920 2024-01-15T18:17:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.840+0000 7fc679e00640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc64c06c550 0x7fc64c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:57.845 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.841+0000 7fc679e00640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc64c06c550 0x7fc64c06ea10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc664036000 tx=0x7fc664008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:57.846 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:57.843+0000 7fc672000640 1 -- 172.21.15.26:0/1049474239 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6740b65f0 con 0x7fc674160920 2024-01-15T18:17:58.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.035+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6740ceac0 con 0x7fc64c06c550 2024-01-15T18:17:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.037+0000 7fc672000640 1 -- 172.21.15.26:0/1049474239 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc664030c10 con 0x7fc64c06c550 2024-01-15T18:17:58.040 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 4 to deep-scrub 2024-01-15T18:17:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.039+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc64c06c550 msgr2=0x7fc64c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.039+0000 7fc67b200640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc64c06c550 0x7fc64c06ea10 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fc664036000 tx=0x7fc664008030 comp rx=0 tx=0).stop 2024-01-15T18:17:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.039+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674160920 msgr2=0x7fc6740c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.039+0000 7fc67b200640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674160920 0x7fc6740c88b0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fc66c00efc0 tx=0x7fc66c00c490 comp rx=0 tx=0).stop 2024-01-15T18:17:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.040+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 shutdown_connections 2024-01-15T18:17:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.040+0000 7fc67b200640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fc6740cfe70 0x7fc6740cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.040+0000 7fc67b200640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fc64c06c550 0x7fc64c06ea10 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.040+0000 7fc67b200640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fc674160920 0x7fc6740c88b0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.040+0000 7fc67b200640 1 --2- 172.21.15.26:0/1049474239 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fc6740cf9f0 0x7fc674150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.040+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 >> 172.21.15.26:0/1049474239 conn(0x7fc6740d3090 msgr2=0x7fc6740cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.040+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 shutdown_connections 2024-01-15T18:17:58.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.040+0000 7fc67b200640 1 -- 172.21.15.26:0/1049474239 wait complete. 2024-01-15T18:17:58.063 INFO:tasks.ceph:Scrubbing osd.5 2024-01-15T18:17:58.063 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:17:58.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.292+0000 7fea4b200640 1 Processor -- start 2024-01-15T18:17:58.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.292+0000 7fea4b200640 1 -- start start 2024-01-15T18:17:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea4b200640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea4c0d0100 0x7fea4c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.298 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea4b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea4c0d0f40 0x7fea4c0d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea4b200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c0d0a10 con 0x7fea4c160920 2024-01-15T18:17:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea4b200640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c0cf080 con 0x7fea4c0d0100 2024-01-15T18:17:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea4b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0c6be0 con 0x7fea4c0d0f40 2024-01-15T18:17:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea49400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea4c0d0100 0x7fea4c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea49400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fea4c0d0100 0x7fea4c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56644/0 (socket says 172.21.15.26:56644) 2024-01-15T18:17:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea49400640 1 -- 172.21.15.26:0/4113535104 learned_addr learned my addr 172.21.15.26:0/4113535104 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea49e00640 1 --1- 172.21.15.26:0/4113535104 >> v1:172.21.15.26:6789/0 conn(0x7fea4c160920 0x7fea4c0cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56910/0 (socket says 172.21.15.26:56910) 2024-01-15T18:17:58.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea4a800640 1 --2- 172.21.15.26:0/4113535104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea4c0d0f40 0x7fea4c0d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea49400640 1 -- 172.21.15.26:0/4113535104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea4c0d0f40 msgr2=0x7fea4c0d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea49400640 1 --2- 172.21.15.26:0/4113535104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea4c0d0f40 0x7fea4c0d13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea49400640 1 -- 172.21.15.26:0/4113535104 >> v1:172.21.15.26:6789/0 conn(0x7fea4c160920 legacy=0x7fea4c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.294+0000 7fea49400640 1 -- 172.21.15.26:0/4113535104 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c0ce7e0 con 0x7fea4c0d0100 2024-01-15T18:17:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.296+0000 7fea49400640 1 --2- 172.21.15.26:0/4113535104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 0x7fea4c0b3f30 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fea34009e50 tx=0x7fea3402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c724398199e940b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:58.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.296+0000 7fea48a00640 1 -- 172.21.15.26:0/4113535104 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3402cb00 con 0x7fea4c0d0100 2024-01-15T18:17:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.296+0000 7fea48a00640 1 -- 172.21.15.26:0/4113535104 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3402cc60 con 0x7fea4c0d0100 2024-01-15T18:17:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.296+0000 7fea48a00640 1 -- 172.21.15.26:0/4113535104 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340343f0 con 0x7fea4c0d0100 2024-01-15T18:17:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.297+0000 7fea4b200640 1 -- 172.21.15.26:0/4113535104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 msgr2=0x7fea4c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.297+0000 7fea4b200640 1 --2- 172.21.15.26:0/4113535104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 0x7fea4c0b3f30 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7fea34009e50 tx=0x7fea3402bf60 comp rx=0 tx=0).stop 2024-01-15T18:17:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.297+0000 7fea4b200640 1 -- 172.21.15.26:0/4113535104 shutdown_connections 2024-01-15T18:17:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.297+0000 7fea4b200640 1 --2- 172.21.15.26:0/4113535104 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea4c0d0f40 0x7fea4c0d13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.297+0000 7fea4b200640 1 --2- 172.21.15.26:0/4113535104 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 0x7fea4c0b3f30 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.297+0000 7fea4b200640 1 -- 172.21.15.26:0/4113535104 >> 172.21.15.26:0/4113535104 conn(0x7fea4c0d3090 msgr2=0x7fea4c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:58.302 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.297+0000 7fea4b200640 1 -- 172.21.15.26:0/4113535104 shutdown_connections 2024-01-15T18:17:58.303 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.297+0000 7fea4b200640 1 -- 172.21.15.26:0/4113535104 wait complete. 2024-01-15T18:17:58.304 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.299+0000 7fea4b200640 1 Processor -- start 2024-01-15T18:17:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.299+0000 7fea4b200640 1 -- start start 2024-01-15T18:17:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea4b200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 0x7fea4c149520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea4b200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea4c0d0f40 0x7fea4c0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea4b200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4c160920 0x7fea4c1612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea4b200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c149a60 con 0x7fea4c0d0100 2024-01-15T18:17:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea4b200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0b8960 con 0x7fea4c160920 2024-01-15T18:17:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea4b200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c0b7d50 con 0x7fea4c0d0f40 2024-01-15T18:17:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea49e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 0x7fea4c149520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea49e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 0x7fea4c149520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56660/0 (socket says 172.21.15.26:56660) 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea49e00640 1 -- 172.21.15.26:0/1627273974 learned_addr learned my addr 172.21.15.26:0/1627273974 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea49400640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea4c0d0f40 0x7fea4c0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.300+0000 7fea4a800640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4c160920 0x7fea4c1612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea49e00640 1 -- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea4c0d0f40 msgr2=0x7fea4c0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea49e00640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fea4c0d0f40 0x7fea4c0b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea49e00640 1 -- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4c160920 msgr2=0x7fea4c1612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea49e00640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fea4c160920 0x7fea4c1612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea49e00640 1 -- 172.21.15.26:0/1627273974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea34009b30 con 0x7fea4c0d0100 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea49e00640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 0x7fea4c149520 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c4d0 tx=0x7fea3c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:58.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea42000640 1 -- 172.21.15.26:0/1627273974 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c007480 con 0x7fea4c0d0100 2024-01-15T18:17:58.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea42000640 1 -- 172.21.15.26:0/1627273974 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c0075e0 con 0x7fea4c0d0100 2024-01-15T18:17:58.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea42000640 1 -- 172.21.15.26:0/1627273974 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0049e0 con 0x7fea4c0d0100 2024-01-15T18:17:58.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c0b8030 con 0x7fea4c0d0100 2024-01-15T18:17:58.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.301+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c0b85f0 con 0x7fea4c0d0100 2024-01-15T18:17:58.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.302+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea4c160d00 con 0x7fea4c0d0100 2024-01-15T18:17:58.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.304+0000 7fea42000640 1 -- 172.21.15.26:0/1627273974 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fea3c004b40 con 0x7fea4c0d0100 2024-01-15T18:17:58.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.305+0000 7fea42000640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea2406c550 0x7fea2406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.305+0000 7fea42000640 1 -- 172.21.15.26:0/1627273974 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fea3c087c70 con 0x7fea4c0d0100 2024-01-15T18:17:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.305+0000 7fea42000640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fea240748d0 0x7fea24076d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.305+0000 7fea42000640 1 -- 172.21.15.26:0/1627273974 --> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea24077440 con 0x7fea240748d0 2024-01-15T18:17:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.305+0000 7fea42000640 1 -- 172.21.15.26:0/1627273974 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea3c088890 con 0x7fea4c0d0100 2024-01-15T18:17:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.305+0000 7fea49400640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea2406c550 0x7fea2406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.306+0000 7fea4a800640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fea240748d0 0x7fea24076d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.306+0000 7fea49400640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea2406c550 0x7fea2406ea10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fea34035000 tx=0x7fea34030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:58.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.307+0000 7fea4a800640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fea240748d0 0x7fea24076d70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:58.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.309+0000 7fea42000640 1 -- 172.21.15.26:0/1627273974 <== osd.5 v2:172.21.15.124:6824/4019293128 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fea24077440 con 0x7fea240748d0 2024-01-15T18:17:58.328 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.325+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 --> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fea4c0ceda0 con 0x7fea240748d0 2024-01-15T18:17:58.329 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea42000640 1 -- 172.21.15.26:0/1627273974 <== osd.5 v2:172.21.15.124:6824/4019293128 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fea4400baa0 con 0x7fea240748d0 2024-01-15T18:17:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:17:58.330 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:17:58.330 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:17:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fea240748d0 msgr2=0x7fea24076d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6824/4019293128,v1:172.21.15.124:6825/4019293128] conn(0x7fea240748d0 0x7fea24076d70 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea2406c550 msgr2=0x7fea2406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fea2406c550 0x7fea2406ea10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fea34035000 tx=0x7fea34030000 comp rx=0 tx=0).stop 2024-01-15T18:17:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 msgr2=0x7fea4c149520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 --2- 172.21.15.26:0/1627273974 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fea4c0d0100 0x7fea4c149520 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c4d0 tx=0x7fea3c00c990 comp rx=0 tx=0).stop 2024-01-15T18:17:58.331 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4a800640 1 -- 172.21.15.26:0/1627273974 reap_dead start 2024-01-15T18:17:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 shutdown_connections 2024-01-15T18:17:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 >> 172.21.15.26:0/1627273974 conn(0x7fea4c0d3090 msgr2=0x7fea4c0d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 shutdown_connections 2024-01-15T18:17:58.332 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.327+0000 7fea4b200640 1 -- 172.21.15.26:0/1627273974 wait complete. 2024-01-15T18:17:58.349 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-01-15T18:17:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.572+0000 7f5b5e200640 1 Processor -- start 2024-01-15T18:17:58.575 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.572+0000 7f5b5e200640 1 -- start start 2024-01-15T18:17:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.573+0000 7f5b5e200640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f5b5815d590 0x7f5b580cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.573+0000 7f5b5e200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b580ce8c0 0x7f5b580b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.573+0000 7f5b5e200640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b580b6750 con 0x7f5b580d00b0 2024-01-15T18:17:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.573+0000 7f5b5e200640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b580d0530 con 0x7f5b5815d590 2024-01-15T18:17:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.573+0000 7f5b5e200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580cf1f0 con 0x7f5b580ce8c0 2024-01-15T18:17:58.577 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.574+0000 7f5b56a00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f5b580d00b0 0x7f5b580ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56924/0 (socket says 172.21.15.26:56924) 2024-01-15T18:17:58.578 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.574+0000 7f5b56a00640 1 -- 172.21.15.26:0/2661567885 learned_addr learned my addr 172.21.15.26:0/2661567885 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.574+0000 7f5b57400640 1 --2- 172.21.15.26:0/2661567885 >> v2:172.21.15.124:3300/0 conn(0x7f5b5815d590 0x7f5b580cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.579 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.574+0000 7f5b57e00640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b580ce8c0 0x7f5b580b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.574+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3899808055 0 0) 0x7f5b580b6750 con 0x7f5b580d00b0 2024-01-15T18:17:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.574+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b38003520 con 0x7f5b580d00b0 2024-01-15T18:17:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 595411857 0 0) 0x7f5b38003520 con 0x7f5b580d00b0 2024-01-15T18:17:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b580ce8c0 msgr2=0x7f5b580b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b580ce8c0 0x7f5b580b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 >> v2:172.21.15.124:3300/0 conn(0x7f5b5815d590 msgr2=0x7f5b580cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 --2- 172.21.15.26:0/2661567885 >> v2:172.21.15.124:3300/0 conn(0x7f5b5815d590 0x7f5b580cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.580 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580cef10 con 0x7f5b580d00b0 2024-01-15T18:17:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f5b40001670 con 0x7f5b580d00b0 2024-01-15T18:17:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 >> v1:172.21.15.26:6789/0 conn(0x7f5b580d00b0 legacy=0x7f5b580ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b580ce8c0 0x7f5b38003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b38003f40 0x7f5b380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580d0530 con 0x7f5b38004d00 2024-01-15T18:17:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580cf1f0 con 0x7f5b38003f40 2024-01-15T18:17:58.581 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3800b980 con 0x7f5b580ce8c0 2024-01-15T18:17:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b57400640 1 --2- 172.21.15.26:0/2661567885 >> v2:172.21.15.124:3300/0 conn(0x7f5b5815d590 0x7f5b580cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-15T18:17:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b57e00640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b56a00640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b38003f40 0x7f5b380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.575+0000 7f5b57400640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b580ce8c0 0x7f5b38003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.576+0000 7f5b57e00640 1 -- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b580ce8c0 msgr2=0x7f5b38003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.576+0000 7f5b57e00640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b580ce8c0 0x7f5b38003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.576+0000 7f5b57e00640 1 -- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b38003f40 msgr2=0x7f5b380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.576+0000 7f5b57e00640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b38003f40 0x7f5b380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.576+0000 7f5b57e00640 1 -- 172.21.15.26:0/2661567885 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b38003520 con 0x7f5b38004d00 2024-01-15T18:17:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.577+0000 7f5b57e00640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b38005180 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009af0 tx=0x7f5b380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9527e8aa8c3290 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.577+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5002c6b0 con 0x7f5b38004d00 2024-01-15T18:17:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.577+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5002c810 con 0x7f5b38004d00 2024-01-15T18:17:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.577+0000 7f5b56000640 1 -- 172.21.15.26:0/2661567885 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5002cba0 con 0x7f5b38004d00 2024-01-15T18:17:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.578+0000 7f5b5e200640 1 -- 172.21.15.26:0/2661567885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 msgr2=0x7f5b38005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.578+0000 7f5b5e200640 1 --2- 172.21.15.26:0/2661567885 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b38005180 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009af0 tx=0x7f5b380057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.578+0000 7f5b57e00640 1 -- 172.21.15.26:0/2661567885 reap_dead start 2024-01-15T18:17:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.578+0000 7f5b5e200640 1 -- 172.21.15.26:0/2661567885 shutdown_connections 2024-01-15T18:17:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.578+0000 7f5b5e200640 1 -- 172.21.15.26:0/2661567885 >> 172.21.15.26:0/2661567885 conn(0x7f5b5809c540 msgr2=0x7f5b580d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.578+0000 7f5b5e200640 1 -- 172.21.15.26:0/2661567885 shutdown_connections 2024-01-15T18:17:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.578+0000 7f5b5e200640 1 -- 172.21.15.26:0/2661567885 wait complete. 2024-01-15T18:17:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.580+0000 7f5b5e200640 1 Processor -- start 2024-01-15T18:17:58.585 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.580+0000 7f5b5e200640 1 -- start start 2024-01-15T18:17:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.581+0000 7f5b5e200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b581f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.581+0000 7f5b5e200640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b5809e280 0x7f5b5809e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.581+0000 7f5b5e200640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b5806c020 0x7f5b5806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.581+0000 7f5b5e200640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5809fa70 con 0x7f5b38004d00 2024-01-15T18:17:58.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.581+0000 7f5b5e200640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5806c9e0 con 0x7f5b5809e280 2024-01-15T18:17:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.581+0000 7f5b5e200640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b580a0320 con 0x7f5b5806c020 2024-01-15T18:17:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.581+0000 7f5b57400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b581f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.581+0000 7f5b57400640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b581f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56684/0 (socket says 172.21.15.26:56684) 2024-01-15T18:17:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.581+0000 7f5b57400640 1 -- 172.21.15.26:0/2051487515 learned_addr learned my addr 172.21.15.26:0/2051487515 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b57e00640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b5806c020 0x7f5b5806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b56a00640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b5809e280 0x7f5b5809e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b57400640 1 -- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b5806c020 msgr2=0x7f5b5806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b57400640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b5806c020 0x7f5b5806c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b57400640 1 -- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b5809e280 msgr2=0x7f5b5809e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b57400640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b5809e280 0x7f5b5809e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b57400640 1 -- 172.21.15.26:0/2051487515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5809edf0 con 0x7f5b38004d00 2024-01-15T18:17:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b57400640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b581f21c0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f5b48011480 tx=0x7f5b48011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b4f400640 1 -- 172.21.15.26:0/2051487515 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4800e030 con 0x7f5b38004d00 2024-01-15T18:17:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b4f400640 1 -- 172.21.15.26:0/2051487515 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b48016740 con 0x7f5b38004d00 2024-01-15T18:17:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b4f400640 1 -- 172.21.15.26:0/2051487515 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48016ad0 con 0x7f5b38004d00 2024-01-15T18:17:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.582+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5809f0d0 con 0x7f5b38004d00 2024-01-15T18:17:58.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.583+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5809f690 con 0x7f5b38004d00 2024-01-15T18:17:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.584+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5809fce0 con 0x7f5b38004d00 2024-01-15T18:17:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.585+0000 7f5b4f400640 1 -- 172.21.15.26:0/2051487515 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f5b48016c30 con 0x7f5b38004d00 2024-01-15T18:17:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.586+0000 7f5b4f400640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b3006c550 0x7f5b3006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:58.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.586+0000 7f5b4f400640 1 -- 172.21.15.26:0/2051487515 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f5b4800f070 con 0x7f5b38004d00 2024-01-15T18:17:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.586+0000 7f5b56a00640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b3006c550 0x7f5b3006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:58.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.587+0000 7f5b56a00640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b3006c550 0x7f5b3006ea10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5b5815cb90 tx=0x7f5b40000ff0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:58.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.590+0000 7f5b4f400640 1 -- 172.21.15.26:0/2051487515 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b48056380 con 0x7f5b38004d00 2024-01-15T18:17:58.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.783+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b5815d970 con 0x7f5b3006c550 2024-01-15T18:17:58.788 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.785+0000 7f5b4f400640 1 -- 172.21.15.26:0/2051487515 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5b40005e60 con 0x7f5b3006c550 2024-01-15T18:17:58.788 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 5 to deep-scrub 2024-01-15T18:17:58.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.787+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b3006c550 msgr2=0x7f5b3006ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.787+0000 7f5b5e200640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b3006c550 0x7f5b3006ea10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5b5815cb90 tx=0x7f5b40000ff0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.791 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.787+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 msgr2=0x7f5b581f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.787+0000 7f5b5e200640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b581f21c0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f5b48011480 tx=0x7f5b48011940 comp rx=0 tx=0).stop 2024-01-15T18:17:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.788+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 shutdown_connections 2024-01-15T18:17:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.788+0000 7f5b5e200640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f5b5806c020 0x7f5b5806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.788+0000 7f5b5e200640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f5b3006c550 0x7f5b3006ea10 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.788+0000 7f5b5e200640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5b5809e280 0x7f5b5809e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.788+0000 7f5b5e200640 1 --2- 172.21.15.26:0/2051487515 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f5b38004d00 0x7f5b581f21c0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.788+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 >> 172.21.15.26:0/2051487515 conn(0x7f5b5809c540 msgr2=0x7f5b580b6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:58.792 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.788+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 shutdown_connections 2024-01-15T18:17:58.793 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:58.788+0000 7f5b5e200640 1 -- 172.21.15.26:0/2051487515 wait complete. 2024-01-15T18:17:58.811 INFO:tasks.ceph:Scrubbing osd.6 2024-01-15T18:17:58.812 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:17:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.035+0000 7f6fea800640 1 Processor -- start 2024-01-15T18:17:59.039 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.035+0000 7f6fea800640 1 -- start start 2024-01-15T18:17:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.037+0000 7f6fea800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f6fe40b3f30 0x7f6fe40d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.037+0000 7f6fea800640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe40d0a10 con 0x7f6fe4160920 2024-01-15T18:17:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.037+0000 7f6fea800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe40cf080 con 0x7f6fe40d0100 2024-01-15T18:17:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.037+0000 7f6fea800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe40c6be0 con 0x7f6fe40b3f30 2024-01-15T18:17:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.037+0000 7f6fe9400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f6fe4160920 0x7f6fe40cf9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56926/0 (socket says 172.21.15.26:56926) 2024-01-15T18:17:59.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.037+0000 7f6fe9400640 1 -- 172.21.15.26:0/3858989326 learned_addr learned my addr 172.21.15.26:0/3858989326 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:59.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe9e00640 1 --2- 172.21.15.26:0/3858989326 >> v2:172.21.15.124:3300/0 conn(0x7f6fe40b3f30 0x7f6fe40d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137906097 0 0) 0x7f6fe40d0a10 con 0x7f6fe4160920 2024-01-15T18:17:59.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc4003520 con 0x7f6fe4160920 2024-01-15T18:17:59.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2524906426 0 0) 0x7f6fc4003520 con 0x7f6fe4160920 2024-01-15T18:17:59.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 >> v2:172.21.15.124:3300/0 conn(0x7f6fe40b3f30 msgr2=0x7f6fe40d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:59.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 --2- 172.21.15.26:0/3858989326 >> v2:172.21.15.124:3300/0 conn(0x7f6fe40b3f30 0x7f6fe40d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 >> v1:172.21.15.124:6789/0 conn(0x7f6fe40d0100 legacy=0x7f6fe40d0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40ce7e0 con 0x7f6fe4160920 2024-01-15T18:17:59.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7f6fd40019f0 con 0x7f6fe4160920 2024-01-15T18:17:59.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 >> v1:172.21.15.26:6789/0 conn(0x7f6fe4160920 legacy=0x7f6fe40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc4003970 0x7f6fc4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.038+0000 7f6fe3e00640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fc40047c0 0x7f6fc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe3e00640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc4005180 0x7f6fc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40c6be0 con 0x7f6fc4005180 2024-01-15T18:17:59.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc400c020 con 0x7f6fc40047c0 2024-01-15T18:17:59.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc400c1e0 con 0x7f6fc4003970 2024-01-15T18:17:59.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe9e00640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc4005180 0x7f6fc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe8a00640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fc40047c0 0x7f6fc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe9400640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc4003970 0x7f6fc4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe9e00640 1 -- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc4003970 msgr2=0x7f6fc4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe9e00640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc4003970 0x7f6fc4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe9e00640 1 -- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fc40047c0 msgr2=0x7f6fc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe9e00640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fc40047c0 0x7f6fc4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe9e00640 1 -- 172.21.15.26:0/3858989326 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc4003520 con 0x7f6fc4005180 2024-01-15T18:17:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe9400640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc4003970 0x7f6fc4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.039+0000 7f6fe8a00640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fc40047c0 0x7f6fc4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.040+0000 7f6fe9e00640 1 -- 172.21.15.26:0/3858989326 reap_dead start 2024-01-15T18:17:59.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.040+0000 7f6fe9e00640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc4005180 0x7f6fc400b980 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009ac0 tx=0x7f6fc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2de7f742fc36cb server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc004020 con 0x7f6fc4005180 2024-01-15T18:17:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc004180 con 0x7f6fc4005180 2024-01-15T18:17:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fe3e00640 1 -- 172.21.15.26:0/3858989326 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc02c710 con 0x7f6fc4005180 2024-01-15T18:17:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fea800640 1 -- 172.21.15.26:0/3858989326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc4005180 msgr2=0x7f6fc400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fea800640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc4005180 0x7f6fc400b980 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc009ac0 tx=0x7f6fc40057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fea800640 1 -- 172.21.15.26:0/3858989326 shutdown_connections 2024-01-15T18:17:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fea800640 1 --2- 172.21.15.26:0/3858989326 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fc4005180 0x7f6fc400b980 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fea800640 1 -- 172.21.15.26:0/3858989326 >> 172.21.15.26:0/3858989326 conn(0x7f6fe40d3090 msgr2=0x7f6fe40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fea800640 1 -- 172.21.15.26:0/3858989326 shutdown_connections 2024-01-15T18:17:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.041+0000 7f6fea800640 1 -- 172.21.15.26:0/3858989326 wait complete. 2024-01-15T18:17:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.043+0000 7f6fea800640 1 Processor -- start 2024-01-15T18:17:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.043+0000 7f6fea800640 1 -- start start 2024-01-15T18:17:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.044+0000 7f6fea800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc4005180 0x7f6fe40cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.044+0000 7f6fea800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fe40b7940 0x7f6fe40b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.044+0000 7f6fea800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fe40b72d0 0x7f6fe40cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.044+0000 7f6fea800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b6900 con 0x7f6fe40b7940 2024-01-15T18:17:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.044+0000 7f6fea800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b5cf0 con 0x7f6fe40b72d0 2024-01-15T18:17:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.044+0000 7f6fea800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b5fd0 con 0x7f6fc4005180 2024-01-15T18:17:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.044+0000 7f6fe8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fe40b7940 0x7f6fe40b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.044+0000 7f6fe8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fe40b7940 0x7f6fe40b6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56698/0 (socket says 172.21.15.26:56698) 2024-01-15T18:17:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe8a00640 1 -- 172.21.15.26:0/4144082865 learned_addr learned my addr 172.21.15.26:0/4144082865 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe9400640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc4005180 0x7f6fe40cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe9e00640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fe40b72d0 0x7f6fe40cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe8a00640 1 -- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc4005180 msgr2=0x7f6fe40cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe8a00640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f6fc4005180 0x7f6fe40cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe8a00640 1 -- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fe40b72d0 msgr2=0x7f6fe40cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe8a00640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fe40b72d0 0x7f6fe40cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe8a00640 1 -- 172.21.15.26:0/4144082865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40b62b0 con 0x7f6fe40b7940 2024-01-15T18:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe9e00640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f6fe40b72d0 0x7f6fe40cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.045+0000 7f6fe8a00640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fe40b7940 0x7f6fe40b6d90 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00d770 tx=0x7f6fcc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.046+0000 7f6fe1600640 1 -- 172.21.15.26:0/4144082865 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc005270 con 0x7f6fe40b7940 2024-01-15T18:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.046+0000 7f6fe1600640 1 -- 172.21.15.26:0/4144082865 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc0053d0 con 0x7f6fe40b7940 2024-01-15T18:17:59.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.046+0000 7f6fe1600640 1 -- 172.21.15.26:0/4144082865 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc00b8f0 con 0x7f6fe40b7940 2024-01-15T18:17:59.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.046+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe40b6590 con 0x7f6fe40b7940 2024-01-15T18:17:59.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.046+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe40cb260 con 0x7f6fe40b7940 2024-01-15T18:17:59.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.047+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6fe4160d00 con 0x7f6fe40b7940 2024-01-15T18:17:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.048+0000 7f6fe1600640 1 -- 172.21.15.26:0/4144082865 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f6fcc00ba50 con 0x7f6fe40b7940 2024-01-15T18:17:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.049+0000 7f6fe1600640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fbc06c580 0x7f6fbc06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.049+0000 7f6fe1600640 1 -- 172.21.15.26:0/4144082865 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f6fcc088a90 con 0x7f6fe40b7940 2024-01-15T18:17:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.049+0000 7f6fe1600640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7f6fbc074900 0x7f6fbc076da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.049+0000 7f6fe1600640 1 -- 172.21.15.26:0/4144082865 --> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6fbc077470 con 0x7f6fbc074900 2024-01-15T18:17:59.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.049+0000 7f6fe1600640 1 -- 172.21.15.26:0/4144082865 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fcc089730 con 0x7f6fe40b7940 2024-01-15T18:17:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.050+0000 7f6fe9400640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fbc06c580 0x7f6fbc06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.050+0000 7f6fe9e00640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7f6fbc074900 0x7f6fbc076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.051+0000 7f6fe9400640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fbc06c580 0x7f6fbc06ea40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40012e0 tx=0x7f6fd4009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.051+0000 7f6fe9e00640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7f6fbc074900 0x7f6fbc076da0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.053+0000 7f6fe1600640 1 -- 172.21.15.26:0/4144082865 <== osd.6 v2:172.21.15.124:6816/4103149233 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6fbc077470 con 0x7f6fbc074900 2024-01-15T18:17:59.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.067+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 --> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6fe40ceda0 con 0x7f6fbc074900 2024-01-15T18:17:59.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.068+0000 7f6fe1600640 1 -- 172.21.15.26:0/4144082865 <== osd.6 v2:172.21.15.124:6816/4103149233 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6fdc03a040 con 0x7f6fbc074900 2024-01-15T18:17:59.072 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:17:59.072 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:17:59.073 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:17:59.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.069+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7f6fbc074900 msgr2=0x7f6fbc076da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.069+0000 7f6fea800640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6816/4103149233,v1:172.21.15.124:6817/4103149233] conn(0x7f6fbc074900 0x7f6fbc076da0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.069+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fbc06c580 msgr2=0x7f6fbc06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.069+0000 7f6fea800640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f6fbc06c580 0x7f6fbc06ea40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40012e0 tx=0x7f6fd4009040 comp rx=0 tx=0).stop 2024-01-15T18:17:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.069+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fe40b7940 msgr2=0x7f6fe40b6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.069+0000 7f6fea800640 1 --2- 172.21.15.26:0/4144082865 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f6fe40b7940 0x7f6fe40b6d90 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00d770 tx=0x7f6fcc00dc30 comp rx=0 tx=0).stop 2024-01-15T18:17:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.069+0000 7f6fe9e00640 1 -- 172.21.15.26:0/4144082865 reap_dead start 2024-01-15T18:17:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.069+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 shutdown_connections 2024-01-15T18:17:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.069+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 >> 172.21.15.26:0/4144082865 conn(0x7f6fe40d3090 msgr2=0x7f6fe40d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:59.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.070+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 shutdown_connections 2024-01-15T18:17:59.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.070+0000 7f6fea800640 1 -- 172.21.15.26:0/4144082865 wait complete. 2024-01-15T18:17:59.090 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-01-15T18:17:59.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.302+0000 7fd383c00640 1 Processor -- start 2024-01-15T18:17:59.305 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.302+0000 7fd383c00640 1 -- start start 2024-01-15T18:17:59.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.304+0000 7fd383c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd37c13df80 0x7fd37c15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.304+0000 7fd383c00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c06c470 con 0x7fd37c13e360 2024-01-15T18:17:59.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.304+0000 7fd383c00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c06c750 con 0x7fd37c0cf9a0 2024-01-15T18:17:59.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.304+0000 7fd383c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c06ca30 con 0x7fd37c13df80 2024-01-15T18:17:59.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.304+0000 7fd373e00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7fd37c13e360 0x7fd37c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56942/0 (socket says 172.21.15.26:56942) 2024-01-15T18:17:59.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.304+0000 7fd373e00640 1 -- 172.21.15.26:0/1980828450 learned_addr learned my addr 172.21.15.26:0/1980828450 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:17:59.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.304+0000 7fd381200640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd37c13df80 0x7fd37c15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851746895 0 0) 0x7fd37c06c470 con 0x7fd37c13e360 2024-01-15T18:17:59.310 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd35c003520 con 0x7fd37c13e360 2024-01-15T18:17:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1531159196 0 0) 0x7fd35c003520 con 0x7fd37c13e360 2024-01-15T18:17:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd37c13df80 msgr2=0x7fd37c15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd37c13df80 0x7fd37c15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 >> v1:172.21.15.124:6789/0 conn(0x7fd37c0cf9a0 legacy=0x7fd37c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c06cd10 con 0x7fd37c13e360 2024-01-15T18:17:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7fd364001670 con 0x7fd37c13e360 2024-01-15T18:17:59.311 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 >> v1:172.21.15.26:6789/0 conn(0x7fd37c13e360 legacy=0x7fd37c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd35c0039a0 0x7fd35c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd35c0047c0 0x7fd35c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd37c13df80 0x7fd35c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c06ca30 con 0x7fd35c0047c0 2024-01-15T18:17:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c00ba30 con 0x7fd35c0039a0 2024-01-15T18:17:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c00bbf0 con 0x7fd37c13df80 2024-01-15T18:17:59.312 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.305+0000 7fd373e00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd35c0047c0 0x7fd35c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.306+0000 7fd381200640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd35c0039a0 0x7fd35c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.306+0000 7fd381c00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd37c13df80 0x7fd35c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.306+0000 7fd373e00640 1 -- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd37c13df80 msgr2=0x7fd35c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:59.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.306+0000 7fd373e00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd37c13df80 0x7fd35c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.306+0000 7fd373e00640 1 -- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd35c0039a0 msgr2=0x7fd35c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:59.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.306+0000 7fd373e00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd35c0039a0 0x7fd35c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.313 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.306+0000 7fd373e00640 1 -- 172.21.15.26:0/1980828450 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c003520 con 0x7fd35c0047c0 2024-01-15T18:17:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.306+0000 7fd381c00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd37c13df80 0x7fd35c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:59.314 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.307+0000 7fd373e00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd35c0047c0 0x7fd35c004c40 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fd36400c210 tx=0x7fd35c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef657db1a2abf181 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.307+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364005ad0 con 0x7fd35c0047c0 2024-01-15T18:17:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.307+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd364005c30 con 0x7fd35c0047c0 2024-01-15T18:17:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.307+0000 7fd37be00640 1 -- 172.21.15.26:0/1980828450 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36402c870 con 0x7fd35c0047c0 2024-01-15T18:17:59.317 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.308+0000 7fd383c00640 1 -- 172.21.15.26:0/1980828450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd35c0047c0 msgr2=0x7fd35c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.308+0000 7fd383c00640 1 --2- 172.21.15.26:0/1980828450 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd35c0047c0 0x7fd35c004c40 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fd36400c210 tx=0x7fd35c0057f0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.308+0000 7fd381c00640 1 -- 172.21.15.26:0/1980828450 reap_dead start 2024-01-15T18:17:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.308+0000 7fd383c00640 1 -- 172.21.15.26:0/1980828450 shutdown_connections 2024-01-15T18:17:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.308+0000 7fd383c00640 1 -- 172.21.15.26:0/1980828450 >> 172.21.15.26:0/1980828450 conn(0x7fd37c06a800 msgr2=0x7fd37c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.308+0000 7fd383c00640 1 -- 172.21.15.26:0/1980828450 shutdown_connections 2024-01-15T18:17:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.308+0000 7fd383c00640 1 -- 172.21.15.26:0/1980828450 wait complete. 2024-01-15T18:17:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.310+0000 7fd383c00640 1 Processor -- start 2024-01-15T18:17:59.318 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.310+0000 7fd383c00640 1 -- start start 2024-01-15T18:17:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.311+0000 7fd383c00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0047c0 0x7fd37c152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.311+0000 7fd383c00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd37c0b32b0 0x7fd37c156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.311+0000 7fd383c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c0ad1c0 0x7fd37c13c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.311+0000 7fd383c00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c153ab0 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.311+0000 7fd383c00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0aca00 con 0x7fd37c0b32b0 2024-01-15T18:17:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.311+0000 7fd383c00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c13cfa0 con 0x7fd35c0047c0 2024-01-15T18:17:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.311+0000 7fd381c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c0ad1c0 0x7fd37c13c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.319 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.311+0000 7fd381c00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c0ad1c0 0x7fd37c13c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56722/0 (socket says 172.21.15.26:56722) 2024-01-15T18:17:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.311+0000 7fd381c00640 1 -- 172.21.15.26:0/219946593 learned_addr learned my addr 172.21.15.26:0/219946593 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd381200640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0047c0 0x7fd37c152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd373e00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd37c0b32b0 0x7fd37c156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd381c00640 1 -- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0047c0 msgr2=0x7fd37c152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd381c00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0047c0 0x7fd37c152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd381c00640 1 -- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd37c0b32b0 msgr2=0x7fd37c156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd381c00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd37c0b32b0 0x7fd37c156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.320 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd381c00640 1 -- 172.21.15.26:0/219946593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c1534f0 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd381c00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c0ad1c0 0x7fd37c13c5d0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0cfea0 tx=0x7fd37400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd379600640 1 -- 172.21.15.26:0/219946593 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400b070 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd379600640 1 -- 172.21.15.26:0/219946593 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd374007030 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd379600640 1 -- 172.21.15.26:0/219946593 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3740073e0 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.312+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c1537d0 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.313+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c156fc0 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.321 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.314+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c0675d0 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.315+0000 7fd379600640 1 -- 172.21.15.26:0/219946593 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fd374007540 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.322 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.316+0000 7fd379600640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd35406c550 0x7fd35406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.316+0000 7fd379600640 1 -- 172.21.15.26:0/219946593 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fd374088670 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.317+0000 7fd381200640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd35406c550 0x7fd35406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.323 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.318+0000 7fd381200640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd35406c550 0x7fd35406ea10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd36c007520 tx=0x7fd36c00b040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.324 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.321+0000 7fd379600640 1 -- 172.21.15.26:0/219946593 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd374054bb0 con 0x7fd37c0ad1c0 2024-01-15T18:17:59.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.515+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fd37c15bee0 con 0x7fd35406c550 2024-01-15T18:17:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.516+0000 7fd379600640 1 -- 172.21.15.26:0/219946593 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd36c003990 con 0x7fd35406c550 2024-01-15T18:17:59.520 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 6 to deep-scrub 2024-01-15T18:17:59.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd35406c550 msgr2=0x7fd35406ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.523 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd35406c550 0x7fd35406ea10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd36c007520 tx=0x7fd36c00b040 comp rx=0 tx=0).stop 2024-01-15T18:17:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c0ad1c0 msgr2=0x7fd37c13c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c0ad1c0 0x7fd37c13c5d0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0cfea0 tx=0x7fd37400d890 comp rx=0 tx=0).stop 2024-01-15T18:17:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 shutdown_connections 2024-01-15T18:17:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fd35406c550 0x7fd35406ea10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fd37c0ad1c0 0x7fd37c13c5d0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fd37c0b32b0 0x7fd37c156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.524 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 --2- 172.21.15.26:0/219946593 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fd35c0047c0 0x7fd37c152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.519+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 >> 172.21.15.26:0/219946593 conn(0x7fd37c06a800 msgr2=0x7fd37c06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.520+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 shutdown_connections 2024-01-15T18:17:59.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.520+0000 7fd383c00640 1 -- 172.21.15.26:0/219946593 wait complete. 2024-01-15T18:17:59.543 INFO:tasks.ceph:Scrubbing osd.7 2024-01-15T18:17:59.543 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-01-15T18:17:59.767 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.764+0000 7fdb6ce00640 1 Processor -- start 2024-01-15T18:17:59.768 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.764+0000 7fdb6ce00640 1 -- start start 2024-01-15T18:17:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.765+0000 7fdb6ce00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb6815d540 0x7fdb680d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb6ce00640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7fdb680b2000 0x7fdb681610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.770 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb6ce00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb68161610 0x7fdb680b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb6ce00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6806b2a0 con 0x7fdb68161610 2024-01-15T18:17:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb6ce00640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb680b1ad0 con 0x7fdb680b2000 2024-01-15T18:17:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb6ce00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680b1d20 con 0x7fdb6815d540 2024-01-15T18:17:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb67e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb68161610 0x7fdb680b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.771 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb67e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7fdb68161610 0x7fdb680b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56724/0 (socket says 172.21.15.26:56724) 2024-01-15T18:17:59.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb67e00640 1 -- 172.21.15.26:0/1663670852 learned_addr learned my addr 172.21.15.26:0/1663670852 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:59.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb67400640 1 --2- 172.21.15.26:0/1663670852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb6815d540 0x7fdb680d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb66a00640 1 --2- 172.21.15.26:0/1663670852 >> v2:172.21.15.124:3300/0 conn(0x7fdb680b2000 0x7fdb681610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb67e00640 1 -- 172.21.15.26:0/1663670852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb6815d540 msgr2=0x7fdb680d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:59.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb67e00640 1 --2- 172.21.15.26:0/1663670852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb6815d540 0x7fdb680d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.772 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.766+0000 7fdb67e00640 1 -- 172.21.15.26:0/1663670852 >> v2:172.21.15.124:3300/0 conn(0x7fdb680b2000 msgr2=0x7fdb681610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:17:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.767+0000 7fdb67e00640 1 --2- 172.21.15.26:0/1663670852 >> v2:172.21.15.124:3300/0 conn(0x7fdb680b2000 0x7fdb681610d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.767+0000 7fdb67e00640 1 -- 172.21.15.26:0/1663670852 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb68161ee0 con 0x7fdb68161610 2024-01-15T18:17:59.773 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.767+0000 7fdb66a00640 1 --2- 172.21.15.26:0/1663670852 >> v2:172.21.15.124:3300/0 conn(0x7fdb680b2000 0x7fdb681610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.768+0000 7fdb67e00640 1 --2- 172.21.15.26:0/1663670852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68161610 0x7fdb680b3d30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fdb68071d50 tx=0x7fdb6002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af0dc2aca4b6bc8 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.777 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.768+0000 7fdb66000640 1 -- 172.21.15.26:0/1663670852 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6002ca70 con 0x7fdb68161610 2024-01-15T18:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.768+0000 7fdb66000640 1 -- 172.21.15.26:0/1663670852 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6002cbd0 con 0x7fdb68161610 2024-01-15T18:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.768+0000 7fdb66000640 1 -- 172.21.15.26:0/1663670852 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb600343f0 con 0x7fdb68161610 2024-01-15T18:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.768+0000 7fdb6ce00640 1 -- 172.21.15.26:0/1663670852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68161610 msgr2=0x7fdb680b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.768+0000 7fdb6ce00640 1 --2- 172.21.15.26:0/1663670852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68161610 0x7fdb680b3d30 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fdb68071d50 tx=0x7fdb6002bed0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.769+0000 7fdb6ce00640 1 -- 172.21.15.26:0/1663670852 shutdown_connections 2024-01-15T18:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.769+0000 7fdb6ce00640 1 --2- 172.21.15.26:0/1663670852 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb68161610 0x7fdb680b3d30 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.778 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.769+0000 7fdb6ce00640 1 --2- 172.21.15.26:0/1663670852 >> v2:172.21.15.124:3300/0 conn(0x7fdb680b2000 0x7fdb681610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.769+0000 7fdb6ce00640 1 --2- 172.21.15.26:0/1663670852 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb6815d540 0x7fdb680d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.769+0000 7fdb6ce00640 1 -- 172.21.15.26:0/1663670852 >> 172.21.15.26:0/1663670852 conn(0x7fdb680b2a30 msgr2=0x7fdb680b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.769+0000 7fdb6ce00640 1 -- 172.21.15.26:0/1663670852 shutdown_connections 2024-01-15T18:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.769+0000 7fdb6ce00640 1 -- 172.21.15.26:0/1663670852 wait complete. 2024-01-15T18:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.771+0000 7fdb6ce00640 1 Processor -- start 2024-01-15T18:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.771+0000 7fdb6ce00640 1 -- start start 2024-01-15T18:17:59.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb6ce00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb6815d540 0x7fdb681555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb6ce00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb68161610 0x7fdb68159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb6ce00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb6815e690 0x7fdb6815a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb6ce00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb680c71c0 con 0x7fdb6815e690 2024-01-15T18:17:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb6ce00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb681f5fd0 con 0x7fdb68161610 2024-01-15T18:17:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb6ce00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb681f62b0 con 0x7fdb6815d540 2024-01-15T18:17:59.780 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb67e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb6815e690 0x7fdb6815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb67e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb6815e690 0x7fdb6815a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56734/0 (socket says 172.21.15.26:56734) 2024-01-15T18:17:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb67e00640 1 -- 172.21.15.26:0/978216315 learned_addr learned my addr 172.21.15.26:0/978216315 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:17:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.772+0000 7fdb67400640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb6815d540 0x7fdb681555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb66a00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb68161610 0x7fdb68159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb67e00640 1 -- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb6815d540 msgr2=0x7fdb681555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.781 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb67e00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7fdb6815d540 0x7fdb681555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb67e00640 1 -- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb68161610 msgr2=0x7fdb68159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb67e00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb68161610 0x7fdb68159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb67e00640 1 -- 172.21.15.26:0/978216315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb60009b30 con 0x7fdb6815e690 2024-01-15T18:17:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb66a00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7fdb68161610 0x7fdb68159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:17:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb67e00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb6815e690 0x7fdb6815a350 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fdb60036040 tx=0x7fdb60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.782 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb5f400640 1 -- 172.21.15.26:0/978216315 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60009510 con 0x7fdb6815e690 2024-01-15T18:17:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb5f400640 1 -- 172.21.15.26:0/978216315 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6003e450 con 0x7fdb6815e690 2024-01-15T18:17:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb681f6590 con 0x7fdb6815e690 2024-01-15T18:17:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.773+0000 7fdb5f400640 1 -- 172.21.15.26:0/978216315 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6003e800 con 0x7fdb6815e690 2024-01-15T18:17:59.783 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.774+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb681f6b50 con 0x7fdb6815e690 2024-01-15T18:17:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.774+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb6815d920 con 0x7fdb6815e690 2024-01-15T18:17:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.776+0000 7fdb5f400640 1 -- 172.21.15.26:0/978216315 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7fdb6003e960 con 0x7fdb6815e690 2024-01-15T18:17:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.777+0000 7fdb5f400640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb4006c580 0x7fdb4006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.785 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.777+0000 7fdb5f400640 1 -- 172.21.15.26:0/978216315 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7fdb600abb60 con 0x7fdb6815e690 2024-01-15T18:17:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.777+0000 7fdb5f400640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fdb40074900 0x7fdb40076da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:17:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.777+0000 7fdb5f400640 1 -- 172.21.15.26:0/978216315 --> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb40077470 con 0x7fdb40074900 2024-01-15T18:17:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.777+0000 7fdb5f400640 1 -- 172.21.15.26:0/978216315 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_get_version_reply(handle=1 version=117) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb600abed0 con 0x7fdb6815e690 2024-01-15T18:17:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.778+0000 7fdb67400640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb4006c580 0x7fdb4006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.778+0000 7fdb66a00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fdb40074900 0x7fdb40076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:17:59.786 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.779+0000 7fdb67400640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb4006c580 0x7fdb4006ea40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fdb50006fd0 tx=0x7fdb50008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.779+0000 7fdb66a00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fdb40074900 0x7fdb40076da0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:17:59.787 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.781+0000 7fdb5f400640 1 -- 172.21.15.26:0/978216315 <== osd.7 v2:172.21.15.124:6808/1393332266 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fdb40077470 con 0x7fdb40074900 2024-01-15T18:17:59.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.796+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 --> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdb681f6d40 con 0x7fdb40074900 2024-01-15T18:17:59.800 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb5f400640 1 -- 172.21.15.26:0/978216315 <== osd.7 v2:172.21.15.124:6808/1393332266 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdb681f6d40 con 0x7fdb40074900 2024-01-15T18:17:59.800 INFO:teuthology.orchestra.run.smithi026.stdout:{ 2024-01-15T18:17:59.801 INFO:teuthology.orchestra.run.smithi026.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-15T18:17:59.801 INFO:teuthology.orchestra.run.smithi026.stdout:} 2024-01-15T18:17:59.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fdb40074900 msgr2=0x7fdb40076da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb6ce00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6808/1393332266,v1:172.21.15.124:6809/1393332266] conn(0x7fdb40074900 0x7fdb40076da0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:17:59.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb4006c580 msgr2=0x7fdb4006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.802 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb6ce00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7fdb4006c580 0x7fdb4006ea40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fdb50006fd0 tx=0x7fdb50008040 comp rx=0 tx=0).stop 2024-01-15T18:17:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb6815e690 msgr2=0x7fdb6815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:17:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb6ce00640 1 --2- 172.21.15.26:0/978216315 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7fdb6815e690 0x7fdb6815a350 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7fdb60036040 tx=0x7fdb60004a40 comp rx=0 tx=0).stop 2024-01-15T18:17:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb67e00640 1 -- 172.21.15.26:0/978216315 reap_dead start 2024-01-15T18:17:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 shutdown_connections 2024-01-15T18:17:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.797+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 >> 172.21.15.26:0/978216315 conn(0x7fdb680b2a30 msgr2=0x7fdb680d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:17:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.798+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 shutdown_connections 2024-01-15T18:17:59.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:17:59.798+0000 7fdb6ce00640 1 -- 172.21.15.26:0/978216315 wait complete. 2024-01-15T18:17:59.819 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-01-15T18:18:00.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.037+0000 7f9dd7e00640 1 Processor -- start 2024-01-15T18:18:00.041 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.038+0000 7f9dd7e00640 1 -- start start 2024-01-15T18:18:00.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.039+0000 7f9dd7e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9dd815efa0 0x7f9dd8071180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.040+0000 7f9dd7e00640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd80b2db0 con 0x7f9dd80b3740 2024-01-15T18:18:00.043 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.040+0000 7f9dd7e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd80b1ff0 con 0x7f9dd815f380 2024-01-15T18:18:00.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.040+0000 7f9dd7e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd80b22d0 con 0x7f9dd815efa0 2024-01-15T18:18:00.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.040+0000 7f9dd6a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f9dd815efa0 0x7f9dd8071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.040+0000 7f9dd7400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7f9dd80b3740 0x7f9dd806c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:56948/0 (socket says 172.21.15.26:56948) 2024-01-15T18:18:00.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.040+0000 7f9dd7400640 1 -- 172.21.15.26:0/2495673685 learned_addr learned my addr 172.21.15.26:0/2495673685 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:18:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.040+0000 7f9dd6a00640 1 -- 172.21.15.26:0/2495673685 >> v1:172.21.15.124:6789/0 conn(0x7f9dd815f380 legacy=0x7f9dd80b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.041+0000 7f9dd6a00640 1 -- 172.21.15.26:0/2495673685 >> v1:172.21.15.26:6789/0 conn(0x7f9dd80b3740 legacy=0x7f9dd806c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.041+0000 7f9dd6a00640 1 -- 172.21.15.26:0/2495673685 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd80b25b0 con 0x7f9dd815efa0 2024-01-15T18:18:00.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.041+0000 7f9dd5600640 1 -- 172.21.15.26:0/2495673685 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464441361 0 0) 0x7f9dd80b2db0 con 0x7f9dd80b3740 2024-01-15T18:18:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.042+0000 7f9dd6a00640 1 --2- 172.21.15.26:0/2495673685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd815efa0 0x7f9dd8071180 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80097d0 tx=0x7f9dc802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0db456d3426c78 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.042+0000 7f9dd5600640 1 -- 172.21.15.26:0/2495673685 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8031830 con 0x7f9dd815efa0 2024-01-15T18:18:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.042+0000 7f9dd5600640 1 -- 172.21.15.26:0/2495673685 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc8031990 con 0x7f9dd815efa0 2024-01-15T18:18:00.046 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.042+0000 7f9dd5600640 1 -- 172.21.15.26:0/2495673685 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8031d20 con 0x7f9dd815efa0 2024-01-15T18:18:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.043+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2495673685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd815efa0 msgr2=0x7f9dd8071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.043+0000 7f9dd7e00640 1 --2- 172.21.15.26:0/2495673685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd815efa0 0x7f9dd8071180 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80097d0 tx=0x7f9dc802b8d0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.043+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2495673685 shutdown_connections 2024-01-15T18:18:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.043+0000 7f9dd7e00640 1 --2- 172.21.15.26:0/2495673685 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd815efa0 0x7f9dd8071180 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.043+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2495673685 >> 172.21.15.26:0/2495673685 conn(0x7f9dd806a3e0 msgr2=0x7f9dd806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.043+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2495673685 shutdown_connections 2024-01-15T18:18:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.043+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2495673685 wait complete. 2024-01-15T18:18:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.045+0000 7f9dd7e00640 1 Processor -- start 2024-01-15T18:18:00.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.045+0000 7f9dd7e00640 1 -- start start 2024-01-15T18:18:00.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.046+0000 7f9dd7e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd80b3740 0x7f9dd8148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.046+0000 7f9dd7e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd815efa0 0x7f9dd815b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.051 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.046+0000 7f9dd7e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd815f380 0x7f9dd815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.046+0000 7f9dd7e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd80ae430 con 0x7f9dd80b3740 2024-01-15T18:18:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.046+0000 7f9dd7e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd8149990 con 0x7f9dd815f380 2024-01-15T18:18:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.046+0000 7f9dd7e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd80cdbf0 con 0x7f9dd815efa0 2024-01-15T18:18:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd80b3740 0x7f9dd8148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd80b3740 0x7f9dd8148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56748/0 (socket says 172.21.15.26:56748) 2024-01-15T18:18:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6a00640 1 -- 172.21.15.26:0/2726480360 learned_addr learned my addr 172.21.15.26:0/2726480360 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:18:00.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6000640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd815efa0 0x7f9dd815b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd7400640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd815f380 0x7f9dd815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6a00640 1 -- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd815efa0 msgr2=0x7f9dd815b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6a00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd815efa0 0x7f9dd815b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6a00640 1 -- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd815f380 msgr2=0x7f9dd815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6a00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd815f380 0x7f9dd815b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6a00640 1 -- 172.21.15.26:0/2726480360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc8009400 con 0x7f9dd80b3740 2024-01-15T18:18:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6000640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd815efa0 0x7f9dd815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd7400640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd815f380 0x7f9dd815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:00.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.047+0000 7f9dd6a00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd80b3740 0x7f9dd8148a80 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f9dc802bee0 tx=0x7f9dc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.048+0000 7f9dcea00640 1 -- 172.21.15.26:0/2726480360 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc803b9f0 con 0x7f9dd80b3740 2024-01-15T18:18:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.048+0000 7f9dcea00640 1 -- 172.21.15.26:0/2726480360 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc803bb50 con 0x7f9dd80b3740 2024-01-15T18:18:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.048+0000 7f9dcea00640 1 -- 172.21.15.26:0/2726480360 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80453f0 con 0x7f9dd80b3740 2024-01-15T18:18:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.048+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd8149340 con 0x7f9dd80b3740 2024-01-15T18:18:00.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.048+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd80cce70 con 0x7f9dd80b3740 2024-01-15T18:18:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.049+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd80cba90 con 0x7f9dd80b3740 2024-01-15T18:18:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.050+0000 7f9dcea00640 1 -- 172.21.15.26:0/2726480360 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f9dc8045550 con 0x7f9dd80b3740 2024-01-15T18:18:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.051+0000 7f9dcea00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9db006c550 0x7f9db006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.051+0000 7f9dcea00640 1 -- 172.21.15.26:0/2726480360 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f9dc80ab130 con 0x7f9dd80b3740 2024-01-15T18:18:00.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.052+0000 7f9dd6000640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9db006c550 0x7f9db006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.053+0000 7f9dd6000640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9db006c550 0x7f9db006ea10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80ce240 tx=0x7f9dc0008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:00.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.056+0000 7f9dcea00640 1 -- 172.21.15.26:0/2726480360 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9dc80b5080 con 0x7f9dd80b3740 2024-01-15T18:18:00.255 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.252+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f9dd81491c0 con 0x7f9db006c550 2024-01-15T18:18:00.256 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.253+0000 7f9dcea00640 1 -- 172.21.15.26:0/2726480360 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9dc000ad80 con 0x7f9db006c550 2024-01-15T18:18:00.256 INFO:teuthology.orchestra.run.smithi026.stderr:instructed osd(s) 7 to deep-scrub 2024-01-15T18:18:00.259 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.255+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9db006c550 msgr2=0x7f9db006ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.255+0000 7f9dd7e00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9db006c550 0x7f9db006ea10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f9dd80ce240 tx=0x7f9dc0008040 comp rx=0 tx=0).stop 2024-01-15T18:18:00.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd80b3740 msgr2=0x7f9dd8148a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd80b3740 0x7f9dd8148a80 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f9dc802bee0 tx=0x7f9dc80046c0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 shutdown_connections 2024-01-15T18:18:00.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f9dd815efa0 0x7f9dd815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f9db006c550 0x7f9db006ea10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f9dd815f380 0x7f9dd815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.260 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 --2- 172.21.15.26:0/2726480360 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f9dd80b3740 0x7f9dd8148a80 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 >> 172.21.15.26:0/2726480360 conn(0x7f9dd806a3e0 msgr2=0x7f9dd815e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:00.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 shutdown_connections 2024-01-15T18:18:00.261 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.256+0000 7f9dd7e00640 1 -- 172.21.15.26:0/2726480360 wait complete. 2024-01-15T18:18:00.280 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:18:00.506 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.503+0000 7faae8a00640 1 Processor -- start 2024-01-15T18:18:00.507 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.503+0000 7faae8a00640 1 -- start start 2024-01-15T18:18:00.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.504+0000 7faae8a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faae4160890 0x7faae40d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.504+0000 7faae8a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae40c6be0 0x7faae40cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.504+0000 7faae8a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae40cf6a0 con 0x7faae4160c70 2024-01-15T18:18:00.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.504+0000 7faae8a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae40d1430 con 0x7faae4160890 2024-01-15T18:18:00.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.504+0000 7faae8a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae40b4370 con 0x7faae40c6be0 2024-01-15T18:18:00.509 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.505+0000 7faae2800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faae4160890 0x7faae40d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.505+0000 7faae2800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7faae4160890 0x7faae40d3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56752/0 (socket says 172.21.15.26:56752) 2024-01-15T18:18:00.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.505+0000 7faae2800640 1 -- 172.21.15.26:0/2483813612 learned_addr learned my addr 172.21.15.26:0/2483813612 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:18:00.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.505+0000 7faae3200640 1 --2- 172.21.15.26:0/2483813612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae40c6be0 0x7faae40cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.505+0000 7faae2800640 1 -- 172.21.15.26:0/2483813612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae40c6be0 msgr2=0x7faae40cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:00.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.505+0000 7faae2800640 1 --2- 172.21.15.26:0/2483813612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae40c6be0 0x7faae40cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.505+0000 7faae2800640 1 -- 172.21.15.26:0/2483813612 >> v1:172.21.15.124:6789/0 conn(0x7faae4160c70 legacy=0x7faae40d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:00.510 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.505+0000 7faae2800640 1 -- 172.21.15.26:0/2483813612 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae40ce7c0 con 0x7faae4160890 2024-01-15T18:18:00.511 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.505+0000 7faae3200640 1 --2- 172.21.15.26:0/2483813612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae40c6be0 0x7faae40cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:18:00.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.506+0000 7faae2800640 1 --2- 172.21.15.26:0/2483813612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae4160890 0x7faae40d3bb0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7faad4009870 tx=0x7faad402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba6f06e8b4cc8b4 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:00.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.507+0000 7faae1400640 1 -- 172.21.15.26:0/2483813612 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4031740 con 0x7faae4160890 2024-01-15T18:18:00.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.507+0000 7faae1400640 1 -- 172.21.15.26:0/2483813612 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad40318a0 con 0x7faae4160890 2024-01-15T18:18:00.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.507+0000 7faae1400640 1 -- 172.21.15.26:0/2483813612 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4031c30 con 0x7faae4160890 2024-01-15T18:18:00.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.507+0000 7faae8a00640 1 -- 172.21.15.26:0/2483813612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae4160890 msgr2=0x7faae40d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.512 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.507+0000 7faae8a00640 1 --2- 172.21.15.26:0/2483813612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae4160890 0x7faae40d3bb0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7faad4009870 tx=0x7faad402b7e0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.508+0000 7faae8a00640 1 -- 172.21.15.26:0/2483813612 shutdown_connections 2024-01-15T18:18:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.508+0000 7faae8a00640 1 --2- 172.21.15.26:0/2483813612 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae40c6be0 0x7faae40cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.508+0000 7faae8a00640 1 --2- 172.21.15.26:0/2483813612 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae4160890 0x7faae40d3bb0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.508+0000 7faae8a00640 1 -- 172.21.15.26:0/2483813612 >> 172.21.15.26:0/2483813612 conn(0x7faae40d3090 msgr2=0x7faae40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.508+0000 7faae8a00640 1 -- 172.21.15.26:0/2483813612 shutdown_connections 2024-01-15T18:18:00.513 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.508+0000 7faae8a00640 1 -- 172.21.15.26:0/2483813612 wait complete. 2024-01-15T18:18:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.510+0000 7faae8a00640 1 Processor -- start 2024-01-15T18:18:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.510+0000 7faae8a00640 1 -- start start 2024-01-15T18:18:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae8a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40c6be0 0x7faae40b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae8a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae4160890 0x7faae41523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae8a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faae4160c70 0x7faae40cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae8a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae40b5fe0 con 0x7faae40c6be0 2024-01-15T18:18:00.515 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae8a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae40b7740 con 0x7faae4160c70 2024-01-15T18:18:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae8a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae40b7a20 con 0x7faae4160890 2024-01-15T18:18:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40c6be0 0x7faae40b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae2800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40c6be0 0x7faae40b87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:56764/0 (socket says 172.21.15.26:56764) 2024-01-15T18:18:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae2800640 1 -- 172.21.15.26:0/549778807 learned_addr learned my addr 172.21.15.26:0/549778807 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:18:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae3200640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faae4160c70 0x7faae40cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae1e00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae4160890 0x7faae41523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.516 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae2800640 1 -- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae4160890 msgr2=0x7faae41523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae2800640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae4160890 0x7faae41523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae2800640 1 -- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faae4160c70 msgr2=0x7faae40cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.511+0000 7faae2800640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faae4160c70 0x7faae40cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.517 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.512+0000 7faae2800640 1 -- 172.21.15.26:0/549778807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad40094d0 con 0x7faae40c6be0 2024-01-15T18:18:00.518 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.512+0000 7faae2800640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40c6be0 0x7faae40b87e0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7faad4009870 tx=0x7faad40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.512+0000 7faadaa00640 1 -- 172.21.15.26:0/549778807 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4033d60 con 0x7faae40c6be0 2024-01-15T18:18:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.512+0000 7faadaa00640 1 -- 172.21.15.26:0/549778807 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad403c400 con 0x7faae40c6be0 2024-01-15T18:18:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.512+0000 7faadaa00640 1 -- 172.21.15.26:0/549778807 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad403c7b0 con 0x7faae40c6be0 2024-01-15T18:18:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.512+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae40b7d00 con 0x7faae40c6be0 2024-01-15T18:18:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.512+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae40b82c0 con 0x7faae40c6be0 2024-01-15T18:18:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.514+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae4067450 con 0x7faae40c6be0 2024-01-15T18:18:00.519 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.515+0000 7faadaa00640 1 -- 172.21.15.26:0/549778807 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7faad403a040 con 0x7faae40c6be0 2024-01-15T18:18:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.516+0000 7faadaa00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faabc06c5a0 0x7faabc06ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.516+0000 7faadaa00640 1 -- 172.21.15.26:0/549778807 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7faad40aa410 con 0x7faae40c6be0 2024-01-15T18:18:00.520 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.517+0000 7faae1e00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faabc06c5a0 0x7faabc06ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:00.521 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.518+0000 7faae1e00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faabc06c5a0 0x7faabc06ea60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7faae40c7530 tx=0x7faacc009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:00.525 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.522+0000 7faadaa00640 1 -- 172.21.15.26:0/549778807 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faad4076e30 con 0x7faae40c6be0 2024-01-15T18:18:00.717 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.714+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faae40ced80 con 0x7faabc06c5a0 2024-01-15T18:18:00.744 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.741+0000 7faadaa00640 1 -- 172.21.15.26:0/549778807 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287082 (secure 0 0 0) 0x7faacc0053f0 con 0x7faabc06c5a0 2024-01-15T18:18:00.749 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:18:00.750 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:18:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faabc06c5a0 msgr2=0x7faabc06ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.752 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faabc06c5a0 0x7faabc06ea60 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7faae40c7530 tx=0x7faacc009040 comp rx=0 tx=0).stop 2024-01-15T18:18:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40c6be0 msgr2=0x7faae40b87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40c6be0 0x7faae40b87e0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7faad4009870 tx=0x7faad40046c0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 shutdown_connections 2024-01-15T18:18:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7faae4160890 0x7faae41523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7faabc06c5a0 0x7faabc06ea60 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7faae4160c70 0x7faae40cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.753 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 --2- 172.21.15.26:0/549778807 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7faae40c6be0 0x7faae40b87e0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.746+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 >> 172.21.15.26:0/549778807 conn(0x7faae40d3090 msgr2=0x7faae40b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.747+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 shutdown_connections 2024-01-15T18:18:00.754 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:00.747+0000 7faae8a00640 1 -- 172.21.15.26:0/549778807 wait complete. 2024-01-15T18:18:00.769 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":802,"stamp":"2024-01-15T18:18:00.454831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369942071,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1744,"num_write_kb":5246066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3200,"ondisk_log_size":3200,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11062224,"kb_used_data":10499600,"kb_used_omap":12,"kb_used_meta":562547,"kb_avail":743912496,"statfs":{"total":773094113280,"available":761766395904,"internally_reserved":0,"allocated":10751590400,"data_stored":10744962942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":576048688},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7174,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005096"},"pg_stats":[{"pgid":"17.3f","version":"117'53","reported_seq":85,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.128966+0000","last_change":"2024-01-15T18:12:53.636580+0000","last_active":"2024-01-15T18:17:39.128966+0000","last_peered":"2024-01-15T18:17:39.128966+0000","last_clean":"2024-01-15T18:17:39.128966+0000","last_became_active":"2024-01-15T18:12:53.334688+0000","last_became_peered":"2024-01-15T18:12:53.334688+0000","last_unstale":"2024-01-15T18:17:39.128966+0000","last_undegraded":"2024-01-15T18:17:39.128966+0000","last_fullsized":"2024-01-15T18:17:39.128966+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3e","version":"117'45","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.174108+0000","last_change":"2024-01-15T18:12:11.053712+0000","last_active":"2024-01-15T18:17:39.174108+0000","last_peered":"2024-01-15T18:17:39.174108+0000","last_clean":"2024-01-15T18:17:39.174108+0000","last_became_active":"2024-01-15T18:12:11.053279+0000","last_became_peered":"2024-01-15T18:12:11.053279+0000","last_unstale":"2024-01-15T18:17:39.174108+0000","last_undegraded":"2024-01-15T18:17:39.174108+0000","last_fullsized":"2024-01-15T18:17:39.174108+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"117'44","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.301593+0000","last_change":"2024-01-15T18:12:11.058625+0000","last_active":"2024-01-15T18:17:39.301593+0000","last_peered":"2024-01-15T18:17:39.301593+0000","last_clean":"2024-01-15T18:17:39.301593+0000","last_became_active":"2024-01-15T18:12:11.057543+0000","last_became_peered":"2024-01-15T18:12:11.057543+0000","last_unstale":"2024-01-15T18:17:39.301593+0000","last_undegraded":"2024-01-15T18:17:39.301593+0000","last_fullsized":"2024-01-15T18:17:39.301593+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:29:17.569977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"117'45","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.270438+0000","last_change":"2024-01-15T18:12:11.047313+0000","last_active":"2024-01-15T18:17:39.270438+0000","last_peered":"2024-01-15T18:17:39.270438+0000","last_clean":"2024-01-15T18:17:39.270438+0000","last_became_active":"2024-01-15T18:12:11.047008+0000","last_became_peered":"2024-01-15T18:12:11.047008+0000","last_unstale":"2024-01-15T18:17:39.270438+0000","last_undegraded":"2024-01-15T18:17:39.270438+0000","last_fullsized":"2024-01-15T18:17:39.270438+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"117'27","reported_seq":50,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.811618+0000","last_change":"2024-01-15T18:12:11.035567+0000","last_active":"2024-01-15T18:17:38.811618+0000","last_peered":"2024-01-15T18:17:38.811618+0000","last_clean":"2024-01-15T18:17:38.811618+0000","last_became_active":"2024-01-15T18:12:11.034909+0000","last_became_peered":"2024-01-15T18:12:11.034909+0000","last_unstale":"2024-01-15T18:17:38.811618+0000","last_undegraded":"2024-01-15T18:17:38.811618+0000","last_fullsized":"2024-01-15T18:17:38.811618+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"117'48","reported_seq":75,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.305662+0000","last_change":"2024-01-15T18:12:11.055133+0000","last_active":"2024-01-15T18:17:39.305662+0000","last_peered":"2024-01-15T18:17:39.305662+0000","last_clean":"2024-01-15T18:17:39.305662+0000","last_became_active":"2024-01-15T18:12:11.054803+0000","last_became_peered":"2024-01-15T18:12:11.054803+0000","last_unstale":"2024-01-15T18:17:39.305662+0000","last_undegraded":"2024-01-15T18:17:39.305662+0000","last_fullsized":"2024-01-15T18:17:39.305662+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"117'43","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.895097+0000","last_change":"2024-01-15T18:12:11.045438+0000","last_active":"2024-01-15T18:17:38.895097+0000","last_peered":"2024-01-15T18:17:38.895097+0000","last_clean":"2024-01-15T18:17:38.895097+0000","last_became_active":"2024-01-15T18:12:11.044607+0000","last_became_peered":"2024-01-15T18:12:11.044607+0000","last_unstale":"2024-01-15T18:17:38.895097+0000","last_undegraded":"2024-01-15T18:17:38.895097+0000","last_fullsized":"2024-01-15T18:17:38.895097+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:55:44.315673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"117'44","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.859692+0000","last_change":"2024-01-15T18:12:11.051056+0000","last_active":"2024-01-15T18:17:38.859692+0000","last_peered":"2024-01-15T18:17:38.859692+0000","last_clean":"2024-01-15T18:17:38.859692+0000","last_became_active":"2024-01-15T18:12:11.050697+0000","last_became_peered":"2024-01-15T18:12:11.050697+0000","last_unstale":"2024-01-15T18:17:38.859692+0000","last_undegraded":"2024-01-15T18:17:38.859692+0000","last_fullsized":"2024-01-15T18:17:38.859692+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:58:56.394072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"117'47","reported_seq":71,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.479691+0000","last_change":"2024-01-15T18:12:11.048227+0000","last_active":"2024-01-15T18:17:39.479691+0000","last_peered":"2024-01-15T18:17:39.479691+0000","last_clean":"2024-01-15T18:17:39.479691+0000","last_became_active":"2024-01-15T18:12:11.047693+0000","last_became_peered":"2024-01-15T18:12:11.047693+0000","last_unstale":"2024-01-15T18:17:39.479691+0000","last_undegraded":"2024-01-15T18:17:39.479691+0000","last_fullsized":"2024-01-15T18:17:39.479691+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"117'58","reported_seq":92,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:55.436550+0000","last_change":"2024-01-15T18:17:55.436587+0000","last_active":"2024-01-15T18:17:55.436550+0000","last_peered":"2024-01-15T18:17:55.436550+0000","last_clean":"2024-01-15T18:17:55.436550+0000","last_became_active":"2024-01-15T18:12:11.052429+0000","last_became_peered":"2024-01-15T18:12:11.052429+0000","last_unstale":"2024-01-15T18:17:55.436550+0000","last_undegraded":"2024-01-15T18:17:55.436550+0000","last_fullsized":"2024-01-15T18:17:55.436550+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'58","last_scrub_stamp":"2024-01-15T18:17:55.436465+0000","last_deep_scrub":"117'58","last_deep_scrub_stamp":"2024-01-15T18:17:55.436465+0000","last_clean_scrub_stamp":"2024-01-15T18:17:55.436465+0000","objects_scrubbed":17,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:11:39.403468+0000","scrub_duration":0.096421670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"117'47","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.167604+0000","last_change":"2024-01-15T18:12:11.043835+0000","last_active":"2024-01-15T18:17:39.167604+0000","last_peered":"2024-01-15T18:17:39.167604+0000","last_clean":"2024-01-15T18:17:39.167604+0000","last_became_active":"2024-01-15T18:12:11.042720+0000","last_became_peered":"2024-01-15T18:12:11.042720+0000","last_unstale":"2024-01-15T18:17:39.167604+0000","last_undegraded":"2024-01-15T18:17:39.167604+0000","last_fullsized":"2024-01-15T18:17:39.167604+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"117'48","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.437359+0000","last_change":"2024-01-15T18:12:11.044468+0000","last_active":"2024-01-15T18:17:39.437359+0000","last_peered":"2024-01-15T18:17:39.437359+0000","last_clean":"2024-01-15T18:17:39.437359+0000","last_became_active":"2024-01-15T18:12:11.044155+0000","last_became_peered":"2024-01-15T18:12:11.044155+0000","last_unstale":"2024-01-15T18:17:39.437359+0000","last_undegraded":"2024-01-15T18:17:39.437359+0000","last_fullsized":"2024-01-15T18:17:39.437359+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"117'38","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.737859+0000","last_change":"2024-01-15T18:12:11.052950+0000","last_active":"2024-01-15T18:17:38.737859+0000","last_peered":"2024-01-15T18:17:38.737859+0000","last_clean":"2024-01-15T18:17:38.737859+0000","last_became_active":"2024-01-15T18:12:11.051875+0000","last_became_peered":"2024-01-15T18:12:11.051875+0000","last_unstale":"2024-01-15T18:17:38.737859+0000","last_undegraded":"2024-01-15T18:17:38.737859+0000","last_fullsized":"2024-01-15T18:17:38.737859+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:55:22.849560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"117'51","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.417564+0000","last_change":"2024-01-15T18:12:11.056103+0000","last_active":"2024-01-15T18:17:39.417564+0000","last_peered":"2024-01-15T18:17:39.417564+0000","last_clean":"2024-01-15T18:17:39.417564+0000","last_became_active":"2024-01-15T18:12:11.055813+0000","last_became_peered":"2024-01-15T18:12:11.055813+0000","last_unstale":"2024-01-15T18:17:39.417564+0000","last_undegraded":"2024-01-15T18:17:39.417564+0000","last_fullsized":"2024-01-15T18:17:39.417564+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:26:39.025824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"117'34","reported_seq":54,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.451682+0000","last_change":"2024-01-15T18:12:11.056863+0000","last_active":"2024-01-15T18:17:39.451682+0000","last_peered":"2024-01-15T18:17:39.451682+0000","last_clean":"2024-01-15T18:17:39.451682+0000","last_became_active":"2024-01-15T18:12:11.055217+0000","last_became_peered":"2024-01-15T18:12:11.055217+0000","last_unstale":"2024-01-15T18:17:39.451682+0000","last_undegraded":"2024-01-15T18:17:39.451682+0000","last_fullsized":"2024-01-15T18:17:39.451682+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"117'42","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.869131+0000","last_change":"2024-01-15T18:12:53.618454+0000","last_active":"2024-01-15T18:17:38.869131+0000","last_peered":"2024-01-15T18:17:38.869131+0000","last_clean":"2024-01-15T18:17:38.869131+0000","last_became_active":"2024-01-15T18:12:53.338860+0000","last_became_peered":"2024-01-15T18:12:53.338860+0000","last_unstale":"2024-01-15T18:17:38.869131+0000","last_undegraded":"2024-01-15T18:17:38.869131+0000","last_fullsized":"2024-01-15T18:17:38.869131+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:40.129590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2f","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.108098+0000","last_change":"2024-01-15T18:12:11.043060+0000","last_active":"2024-01-15T18:17:39.108098+0000","last_peered":"2024-01-15T18:17:39.108098+0000","last_clean":"2024-01-15T18:17:39.108098+0000","last_became_active":"2024-01-15T18:12:11.042031+0000","last_became_peered":"2024-01-15T18:12:11.042031+0000","last_unstale":"2024-01-15T18:17:39.108098+0000","last_undegraded":"2024-01-15T18:17:39.108098+0000","last_fullsized":"2024-01-15T18:17:39.108098+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:44.791297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"117'48","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.357897+0000","last_change":"2024-01-15T18:12:11.045284+0000","last_active":"2024-01-15T18:17:39.357897+0000","last_peered":"2024-01-15T18:17:39.357897+0000","last_clean":"2024-01-15T18:17:39.357897+0000","last_became_active":"2024-01-15T18:12:11.042814+0000","last_became_peered":"2024-01-15T18:12:11.042814+0000","last_unstale":"2024-01-15T18:17:39.357897+0000","last_undegraded":"2024-01-15T18:17:39.357897+0000","last_fullsized":"2024-01-15T18:17:39.357897+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:55:38.645347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.108742+0000","last_change":"2024-01-15T18:12:11.055119+0000","last_active":"2024-01-15T18:17:39.108742+0000","last_peered":"2024-01-15T18:17:39.108742+0000","last_clean":"2024-01-15T18:17:39.108742+0000","last_became_active":"2024-01-15T18:12:11.054790+0000","last_became_peered":"2024-01-15T18:12:11.054790+0000","last_unstale":"2024-01-15T18:17:39.108742+0000","last_undegraded":"2024-01-15T18:17:39.108742+0000","last_fullsized":"2024-01-15T18:17:39.108742+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:17.513018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"117'46","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.362355+0000","last_change":"2024-01-15T18:12:11.052755+0000","last_active":"2024-01-15T18:17:39.362355+0000","last_peered":"2024-01-15T18:17:39.362355+0000","last_clean":"2024-01-15T18:17:39.362355+0000","last_became_active":"2024-01-15T18:12:11.052289+0000","last_became_peered":"2024-01-15T18:12:11.052289+0000","last_unstale":"2024-01-15T18:17:39.362355+0000","last_undegraded":"2024-01-15T18:17:39.362355+0000","last_fullsized":"2024-01-15T18:17:39.362355+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:38:56.937377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"117'56","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.083979+0000","last_change":"2024-01-15T18:12:11.036748+0000","last_active":"2024-01-15T18:17:39.083979+0000","last_peered":"2024-01-15T18:17:39.083979+0000","last_clean":"2024-01-15T18:17:39.083979+0000","last_became_active":"2024-01-15T18:12:11.036003+0000","last_became_peered":"2024-01-15T18:12:11.036003+0000","last_unstale":"2024-01-15T18:17:39.083979+0000","last_undegraded":"2024-01-15T18:17:39.083979+0000","last_fullsized":"2024-01-15T18:17:39.083979+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:58:20.390936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"117'47","reported_seq":71,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.544145+0000","last_change":"2024-01-15T18:12:11.050445+0000","last_active":"2024-01-15T18:17:39.544145+0000","last_peered":"2024-01-15T18:17:39.544145+0000","last_clean":"2024-01-15T18:17:39.544145+0000","last_became_active":"2024-01-15T18:12:11.044093+0000","last_became_peered":"2024-01-15T18:12:11.044093+0000","last_unstale":"2024-01-15T18:17:39.544145+0000","last_undegraded":"2024-01-15T18:17:39.544145+0000","last_fullsized":"2024-01-15T18:17:39.544145+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"117'39","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.269675+0000","last_change":"2024-01-15T18:12:11.053397+0000","last_active":"2024-01-15T18:17:39.269675+0000","last_peered":"2024-01-15T18:17:39.269675+0000","last_clean":"2024-01-15T18:17:39.269675+0000","last_became_active":"2024-01-15T18:12:11.052428+0000","last_became_peered":"2024-01-15T18:12:11.052428+0000","last_unstale":"2024-01-15T18:17:39.269675+0000","last_undegraded":"2024-01-15T18:17:39.269675+0000","last_fullsized":"2024-01-15T18:17:39.269675+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:07.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"117'43","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.484082+0000","last_change":"2024-01-15T18:12:11.051820+0000","last_active":"2024-01-15T18:17:39.484082+0000","last_peered":"2024-01-15T18:17:39.484082+0000","last_clean":"2024-01-15T18:17:39.484082+0000","last_became_active":"2024-01-15T18:12:11.051488+0000","last_became_peered":"2024-01-15T18:12:11.051488+0000","last_unstale":"2024-01-15T18:17:39.484082+0000","last_undegraded":"2024-01-15T18:17:39.484082+0000","last_fullsized":"2024-01-15T18:17:39.484082+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"117'38","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.840614+0000","last_change":"2024-01-15T18:12:11.041672+0000","last_active":"2024-01-15T18:17:38.840614+0000","last_peered":"2024-01-15T18:17:38.840614+0000","last_clean":"2024-01-15T18:17:38.840614+0000","last_became_active":"2024-01-15T18:12:11.041266+0000","last_became_peered":"2024-01-15T18:12:11.041266+0000","last_unstale":"2024-01-15T18:17:38.840614+0000","last_undegraded":"2024-01-15T18:17:38.840614+0000","last_fullsized":"2024-01-15T18:17:38.840614+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"117'53","reported_seq":77,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.486490+0000","last_change":"2024-01-15T18:12:11.048567+0000","last_active":"2024-01-15T18:17:39.486490+0000","last_peered":"2024-01-15T18:17:39.486490+0000","last_clean":"2024-01-15T18:17:39.486490+0000","last_became_active":"2024-01-15T18:12:11.048234+0000","last_became_peered":"2024-01-15T18:12:11.048234+0000","last_unstale":"2024-01-15T18:17:39.486490+0000","last_undegraded":"2024-01-15T18:17:39.486490+0000","last_fullsized":"2024-01-15T18:17:39.486490+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"117'37","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.870834+0000","last_change":"2024-01-15T18:12:11.062039+0000","last_active":"2024-01-15T18:17:38.870834+0000","last_peered":"2024-01-15T18:17:38.870834+0000","last_clean":"2024-01-15T18:17:38.870834+0000","last_became_active":"2024-01-15T18:12:11.061724+0000","last_became_peered":"2024-01-15T18:12:11.061724+0000","last_unstale":"2024-01-15T18:17:38.870834+0000","last_undegraded":"2024-01-15T18:17:38.870834+0000","last_fullsized":"2024-01-15T18:17:38.870834+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"117'38","reported_seq":57,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.487271+0000","last_change":"2024-01-15T18:12:11.044940+0000","last_active":"2024-01-15T18:17:39.487271+0000","last_peered":"2024-01-15T18:17:39.487271+0000","last_clean":"2024-01-15T18:17:39.487271+0000","last_became_active":"2024-01-15T18:12:11.043801+0000","last_became_peered":"2024-01-15T18:12:11.043801+0000","last_unstale":"2024-01-15T18:17:39.487271+0000","last_undegraded":"2024-01-15T18:17:39.487271+0000","last_fullsized":"2024-01-15T18:17:39.487271+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"117'46","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.347960+0000","last_change":"2024-01-15T18:12:11.048119+0000","last_active":"2024-01-15T18:17:39.347960+0000","last_peered":"2024-01-15T18:17:39.347960+0000","last_clean":"2024-01-15T18:17:39.347960+0000","last_became_active":"2024-01-15T18:12:11.047739+0000","last_became_peered":"2024-01-15T18:12:11.047739+0000","last_unstale":"2024-01-15T18:17:39.347960+0000","last_undegraded":"2024-01-15T18:17:39.347960+0000","last_fullsized":"2024-01-15T18:17:39.347960+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:49:08.557895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"117'41","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.292578+0000","last_change":"2024-01-15T18:12:11.040703+0000","last_active":"2024-01-15T18:17:39.292578+0000","last_peered":"2024-01-15T18:17:39.292578+0000","last_clean":"2024-01-15T18:17:39.292578+0000","last_became_active":"2024-01-15T18:12:11.039393+0000","last_became_peered":"2024-01-15T18:12:11.039393+0000","last_unstale":"2024-01-15T18:17:39.292578+0000","last_undegraded":"2024-01-15T18:17:39.292578+0000","last_fullsized":"2024-01-15T18:17:39.292578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:40:41.329686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"117'62","reported_seq":90,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.405837+0000","last_change":"2024-01-15T18:12:11.059603+0000","last_active":"2024-01-15T18:17:39.405837+0000","last_peered":"2024-01-15T18:17:39.405837+0000","last_clean":"2024-01-15T18:17:39.405837+0000","last_became_active":"2024-01-15T18:12:11.059276+0000","last_became_peered":"2024-01-15T18:12:11.059276+0000","last_unstale":"2024-01-15T18:17:39.405837+0000","last_undegraded":"2024-01-15T18:17:39.405837+0000","last_fullsized":"2024-01-15T18:17:39.405837+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"117'54","reported_seq":77,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.529194+0000","last_change":"2024-01-15T18:12:11.053438+0000","last_active":"2024-01-15T18:17:39.529194+0000","last_peered":"2024-01-15T18:17:39.529194+0000","last_clean":"2024-01-15T18:17:39.529194+0000","last_became_active":"2024-01-15T18:12:11.052655+0000","last_became_peered":"2024-01-15T18:12:11.052655+0000","last_unstale":"2024-01-15T18:17:39.529194+0000","last_undegraded":"2024-01-15T18:17:39.529194+0000","last_fullsized":"2024-01-15T18:17:39.529194+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"117'34","reported_seq":56,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.198777+0000","last_change":"2024-01-15T18:12:11.053689+0000","last_active":"2024-01-15T18:17:39.198777+0000","last_peered":"2024-01-15T18:17:39.198777+0000","last_clean":"2024-01-15T18:17:39.198777+0000","last_became_active":"2024-01-15T18:12:11.053389+0000","last_became_peered":"2024-01-15T18:12:11.053389+0000","last_unstale":"2024-01-15T18:17:39.198777+0000","last_undegraded":"2024-01-15T18:17:39.198777+0000","last_fullsized":"2024-01-15T18:17:39.198777+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"114'4","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.409090+0000","last_change":"2024-01-15T18:12:10.021097+0000","last_active":"2024-01-15T18:14:17.409090+0000","last_peered":"2024-01-15T18:14:17.409090+0000","last_clean":"2024-01-15T18:14:17.409090+0000","last_became_active":"2024-01-15T18:12:10.020810+0000","last_became_peered":"2024-01-15T18:12:10.020810+0000","last_unstale":"2024-01-15T18:14:17.409090+0000","last_undegraded":"2024-01-15T18:14:17.409090+0000","last_fullsized":"2024-01-15T18:14:17.409090+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:09:27.223787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"117'47","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.721958+0000","last_change":"2024-01-15T18:12:11.043347+0000","last_active":"2024-01-15T18:17:38.721958+0000","last_peered":"2024-01-15T18:17:38.721958+0000","last_clean":"2024-01-15T18:17:38.721958+0000","last_became_active":"2024-01-15T18:12:11.042427+0000","last_became_peered":"2024-01-15T18:12:11.042427+0000","last_unstale":"2024-01-15T18:17:38.721958+0000","last_undegraded":"2024-01-15T18:17:38.721958+0000","last_fullsized":"2024-01-15T18:17:38.721958+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:12.407664+0000","last_change":"2024-01-15T18:12:10.023175+0000","last_active":"2024-01-15T18:14:12.407664+0000","last_peered":"2024-01-15T18:14:12.407664+0000","last_clean":"2024-01-15T18:14:12.407664+0000","last_became_active":"2024-01-15T18:12:10.022654+0000","last_became_peered":"2024-01-15T18:12:10.022654+0000","last_unstale":"2024-01-15T18:14:12.407664+0000","last_undegraded":"2024-01-15T18:14:12.407664+0000","last_fullsized":"2024-01-15T18:14:12.407664+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:16:12.136270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'43","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.493120+0000","last_change":"2024-01-15T18:12:11.053138+0000","last_active":"2024-01-15T18:17:39.493120+0000","last_peered":"2024-01-15T18:17:39.493120+0000","last_clean":"2024-01-15T18:17:39.493120+0000","last_became_active":"2024-01-15T18:12:11.052812+0000","last_became_peered":"2024-01-15T18:12:11.052812+0000","last_unstale":"2024-01-15T18:17:39.493120+0000","last_undegraded":"2024-01-15T18:17:39.493120+0000","last_fullsized":"2024-01-15T18:17:39.493120+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"117'3","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:12.312702+0000","last_change":"2024-01-15T18:12:10.018995+0000","last_active":"2024-01-15T18:17:12.312702+0000","last_peered":"2024-01-15T18:17:12.312702+0000","last_clean":"2024-01-15T18:17:12.312702+0000","last_became_active":"2024-01-15T18:12:10.018670+0000","last_became_peered":"2024-01-15T18:12:10.018670+0000","last_unstale":"2024-01-15T18:17:12.312702+0000","last_undegraded":"2024-01-15T18:17:12.312702+0000","last_fullsized":"2024-01-15T18:17:12.312702+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"117'56","reported_seq":82,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.844198+0000","last_change":"2024-01-15T18:12:11.059355+0000","last_active":"2024-01-15T18:17:38.844198+0000","last_peered":"2024-01-15T18:17:38.844198+0000","last_clean":"2024-01-15T18:17:38.844198+0000","last_became_active":"2024-01-15T18:12:11.057592+0000","last_became_peered":"2024-01-15T18:12:11.057592+0000","last_unstale":"2024-01-15T18:17:38.844198+0000","last_undegraded":"2024-01-15T18:17:38.844198+0000","last_fullsized":"2024-01-15T18:17:38.844198+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:15:14.826977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"114'2","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:12.312674+0000","last_change":"2024-01-15T18:12:10.020565+0000","last_active":"2024-01-15T18:17:12.312674+0000","last_peered":"2024-01-15T18:17:12.312674+0000","last_clean":"2024-01-15T18:17:12.312674+0000","last_became_active":"2024-01-15T18:12:10.020143+0000","last_became_peered":"2024-01-15T18:12:10.020143+0000","last_unstale":"2024-01-15T18:17:12.312674+0000","last_undegraded":"2024-01-15T18:17:12.312674+0000","last_fullsized":"2024-01-15T18:17:12.312674+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"117'39","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.465226+0000","last_change":"2024-01-15T18:12:11.057876+0000","last_active":"2024-01-15T18:17:39.465226+0000","last_peered":"2024-01-15T18:17:39.465226+0000","last_clean":"2024-01-15T18:17:39.465226+0000","last_became_active":"2024-01-15T18:12:11.057471+0000","last_became_peered":"2024-01-15T18:12:11.057471+0000","last_unstale":"2024-01-15T18:17:39.465226+0000","last_undegraded":"2024-01-15T18:17:39.465226+0000","last_fullsized":"2024-01-15T18:17:39.465226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:42.414286+0000","last_change":"2024-01-15T18:12:10.020217+0000","last_active":"2024-01-15T18:14:42.414286+0000","last_peered":"2024-01-15T18:14:42.414286+0000","last_clean":"2024-01-15T18:14:42.414286+0000","last_became_active":"2024-01-15T18:12:10.019818+0000","last_became_peered":"2024-01-15T18:12:10.019818+0000","last_unstale":"2024-01-15T18:14:42.414286+0000","last_undegraded":"2024-01-15T18:14:42.414286+0000","last_fullsized":"2024-01-15T18:14:42.414286+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:46:30.025957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"117'49","reported_seq":77,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.264806+0000","last_change":"2024-01-15T18:12:11.050383+0000","last_active":"2024-01-15T18:17:39.264806+0000","last_peered":"2024-01-15T18:17:39.264806+0000","last_clean":"2024-01-15T18:17:39.264806+0000","last_became_active":"2024-01-15T18:12:11.050002+0000","last_became_peered":"2024-01-15T18:12:11.050002+0000","last_unstale":"2024-01-15T18:17:39.264806+0000","last_undegraded":"2024-01-15T18:17:39.264806+0000","last_fullsized":"2024-01-15T18:17:39.264806+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:12.339171+0000","last_change":"2024-01-15T18:12:10.013940+0000","last_active":"2024-01-15T18:17:12.339171+0000","last_peered":"2024-01-15T18:17:12.339171+0000","last_clean":"2024-01-15T18:17:12.339171+0000","last_became_active":"2024-01-15T18:12:10.009676+0000","last_became_peered":"2024-01-15T18:12:10.009676+0000","last_unstale":"2024-01-15T18:17:12.339171+0000","last_undegraded":"2024-01-15T18:17:12.339171+0000","last_fullsized":"2024-01-15T18:17:12.339171+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"117'53","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.421022+0000","last_change":"2024-01-15T18:12:11.057984+0000","last_active":"2024-01-15T18:17:39.421022+0000","last_peered":"2024-01-15T18:17:39.421022+0000","last_clean":"2024-01-15T18:17:39.421022+0000","last_became_active":"2024-01-15T18:12:11.057640+0000","last_became_peered":"2024-01-15T18:12:11.057640+0000","last_unstale":"2024-01-15T18:17:39.421022+0000","last_undegraded":"2024-01-15T18:17:39.421022+0000","last_fullsized":"2024-01-15T18:17:39.421022+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857791,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":36865,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:37.331147+0000","last_change":"2024-01-15T18:12:10.002836+0000","last_active":"2024-01-15T18:14:37.331147+0000","last_peered":"2024-01-15T18:14:37.331147+0000","last_clean":"2024-01-15T18:14:37.331147+0000","last_became_active":"2024-01-15T18:12:10.002190+0000","last_became_peered":"2024-01-15T18:12:10.002190+0000","last_unstale":"2024-01-15T18:14:37.331147+0000","last_undegraded":"2024-01-15T18:14:37.331147+0000","last_fullsized":"2024-01-15T18:14:37.331147+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:45:39.115790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"117'43","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.016147+0000","last_change":"2024-01-15T18:12:11.048560+0000","last_active":"2024-01-15T18:17:39.016147+0000","last_peered":"2024-01-15T18:17:39.016147+0000","last_clean":"2024-01-15T18:17:39.016147+0000","last_became_active":"2024-01-15T18:12:11.046934+0000","last_became_peered":"2024-01-15T18:12:11.046934+0000","last_unstale":"2024-01-15T18:17:39.016147+0000","last_undegraded":"2024-01-15T18:17:39.016147+0000","last_fullsized":"2024-01-15T18:17:39.016147+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:44:29.629308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:02.456235+0000","last_change":"2024-01-15T18:12:10.020864+0000","last_active":"2024-01-15T18:17:02.456235+0000","last_peered":"2024-01-15T18:17:02.456235+0000","last_clean":"2024-01-15T18:17:02.456235+0000","last_became_active":"2024-01-15T18:12:10.020577+0000","last_became_peered":"2024-01-15T18:12:10.020577+0000","last_unstale":"2024-01-15T18:17:02.456235+0000","last_undegraded":"2024-01-15T18:17:02.456235+0000","last_fullsized":"2024-01-15T18:17:02.456235+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"117'42","reported_seq":61,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.550415+0000","last_change":"2024-01-15T18:12:11.048003+0000","last_active":"2024-01-15T18:17:38.550415+0000","last_peered":"2024-01-15T18:17:38.550415+0000","last_clean":"2024-01-15T18:17:38.550415+0000","last_became_active":"2024-01-15T18:12:11.047704+0000","last_became_peered":"2024-01-15T18:12:11.047704+0000","last_unstale":"2024-01-15T18:17:38.550415+0000","last_undegraded":"2024-01-15T18:17:38.550415+0000","last_fullsized":"2024-01-15T18:17:38.550415+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:02.001761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"117'1","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:02.310842+0000","last_change":"2024-01-15T18:12:10.020498+0000","last_active":"2024-01-15T18:17:02.310842+0000","last_peered":"2024-01-15T18:17:02.310842+0000","last_clean":"2024-01-15T18:17:02.310842+0000","last_became_active":"2024-01-15T18:12:10.020028+0000","last_became_peered":"2024-01-15T18:12:10.020028+0000","last_unstale":"2024-01-15T18:17:02.310842+0000","last_undegraded":"2024-01-15T18:17:02.310842+0000","last_fullsized":"2024-01-15T18:17:02.310842+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"117'40","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.283436+0000","last_change":"2024-01-15T18:12:11.060166+0000","last_active":"2024-01-15T18:17:39.283436+0000","last_peered":"2024-01-15T18:17:39.283436+0000","last_clean":"2024-01-15T18:17:39.283436+0000","last_became_active":"2024-01-15T18:12:11.059393+0000","last_became_peered":"2024-01-15T18:12:11.059393+0000","last_unstale":"2024-01-15T18:17:39.283436+0000","last_undegraded":"2024-01-15T18:17:39.283436+0000","last_fullsized":"2024-01-15T18:17:39.283436+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:32:38.817573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:02.337284+0000","last_change":"2024-01-15T18:12:10.007879+0000","last_active":"2024-01-15T18:17:02.337284+0000","last_peered":"2024-01-15T18:17:02.337284+0000","last_clean":"2024-01-15T18:17:02.337284+0000","last_became_active":"2024-01-15T18:12:10.007549+0000","last_became_peered":"2024-01-15T18:12:10.007549+0000","last_unstale":"2024-01-15T18:17:02.337284+0000","last_undegraded":"2024-01-15T18:17:02.337284+0000","last_fullsized":"2024-01-15T18:17:02.337284+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"117'43","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.538313+0000","last_change":"2024-01-15T18:12:11.031666+0000","last_active":"2024-01-15T18:17:39.538313+0000","last_peered":"2024-01-15T18:17:39.538313+0000","last_clean":"2024-01-15T18:17:39.538313+0000","last_became_active":"2024-01-15T18:12:11.030958+0000","last_became_peered":"2024-01-15T18:12:11.030958+0000","last_unstale":"2024-01-15T18:17:39.538313+0000","last_undegraded":"2024-01-15T18:17:39.538313+0000","last_fullsized":"2024-01-15T18:17:39.538313+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"117'12","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:02.456412+0000","last_change":"2024-01-15T18:12:10.024809+0000","last_active":"2024-01-15T18:17:02.456412+0000","last_peered":"2024-01-15T18:17:02.456412+0000","last_clean":"2024-01-15T18:17:02.456412+0000","last_became_active":"2024-01-15T18:12:10.024120+0000","last_became_peered":"2024-01-15T18:12:10.024120+0000","last_unstale":"2024-01-15T18:17:02.456412+0000","last_undegraded":"2024-01-15T18:17:02.456412+0000","last_fullsized":"2024-01-15T18:17:02.456412+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"117'43","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.025487+0000","last_change":"2024-01-15T18:12:11.055694+0000","last_active":"2024-01-15T18:17:39.025487+0000","last_peered":"2024-01-15T18:17:39.025487+0000","last_clean":"2024-01-15T18:17:39.025487+0000","last_became_active":"2024-01-15T18:12:11.055290+0000","last_became_peered":"2024-01-15T18:12:11.055290+0000","last_unstale":"2024-01-15T18:17:39.025487+0000","last_undegraded":"2024-01-15T18:17:39.025487+0000","last_fullsized":"2024-01-15T18:17:39.025487+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:15:44.436928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"117'186","reported_seq":203,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:55.210787+0000","last_change":"2024-01-15T18:12:10.022950+0000","last_active":"2024-01-15T18:17:55.210787+0000","last_peered":"2024-01-15T18:17:55.210787+0000","last_clean":"2024-01-15T18:17:55.210787+0000","last_became_active":"2024-01-15T18:12:10.021497+0000","last_became_peered":"2024-01-15T18:12:10.021497+0000","last_unstale":"2024-01-15T18:17:55.210787+0000","last_undegraded":"2024-01-15T18:17:55.210787+0000","last_fullsized":"2024-01-15T18:17:55.210787+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639451,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"117'44","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.543606+0000","last_change":"2024-01-15T18:12:11.046101+0000","last_active":"2024-01-15T18:17:39.543606+0000","last_peered":"2024-01-15T18:17:39.543606+0000","last_clean":"2024-01-15T18:17:39.543606+0000","last_became_active":"2024-01-15T18:12:11.045774+0000","last_became_peered":"2024-01-15T18:12:11.045774+0000","last_unstale":"2024-01-15T18:17:39.543606+0000","last_undegraded":"2024-01-15T18:17:39.543606+0000","last_fullsized":"2024-01-15T18:17:39.543606+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"114'2","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:52.308533+0000","last_change":"2024-01-15T18:12:10.016912+0000","last_active":"2024-01-15T18:16:52.308533+0000","last_peered":"2024-01-15T18:16:52.308533+0000","last_clean":"2024-01-15T18:16:52.308533+0000","last_became_active":"2024-01-15T18:12:10.016618+0000","last_became_peered":"2024-01-15T18:12:10.016618+0000","last_unstale":"2024-01-15T18:16:52.308533+0000","last_undegraded":"2024-01-15T18:16:52.308533+0000","last_fullsized":"2024-01-15T18:16:52.308533+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"117'52","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.370924+0000","last_change":"2024-01-15T18:12:11.051223+0000","last_active":"2024-01-15T18:17:39.370924+0000","last_peered":"2024-01-15T18:17:39.370924+0000","last_clean":"2024-01-15T18:17:39.370924+0000","last_became_active":"2024-01-15T18:12:11.050928+0000","last_became_peered":"2024-01-15T18:12:11.050928+0000","last_unstale":"2024-01-15T18:17:39.370924+0000","last_undegraded":"2024-01-15T18:17:39.370924+0000","last_fullsized":"2024-01-15T18:17:39.370924+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.543703+0000","last_change":"2024-01-15T18:12:10.015868+0000","last_active":"2024-01-15T18:14:32.543703+0000","last_peered":"2024-01-15T18:14:32.543703+0000","last_clean":"2024-01-15T18:14:32.543703+0000","last_became_active":"2024-01-15T18:12:10.015578+0000","last_became_peered":"2024-01-15T18:12:10.015578+0000","last_unstale":"2024-01-15T18:14:32.543703+0000","last_undegraded":"2024-01-15T18:14:32.543703+0000","last_fullsized":"2024-01-15T18:14:32.543703+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"117'46","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.308117+0000","last_change":"2024-01-15T18:12:11.029654+0000","last_active":"2024-01-15T18:17:39.308117+0000","last_peered":"2024-01-15T18:17:39.308117+0000","last_clean":"2024-01-15T18:17:39.308117+0000","last_became_active":"2024-01-15T18:12:11.028865+0000","last_became_peered":"2024-01-15T18:12:11.028865+0000","last_unstale":"2024-01-15T18:17:39.308117+0000","last_undegraded":"2024-01-15T18:17:39.308117+0000","last_fullsized":"2024-01-15T18:17:39.308117+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663556,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"117'12","reported_seq":37,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:56.807714+0000","last_change":"2024-01-15T18:17:56.807742+0000","last_active":"2024-01-15T18:17:56.807714+0000","last_peered":"2024-01-15T18:17:56.807714+0000","last_clean":"2024-01-15T18:17:56.807714+0000","last_became_active":"2024-01-15T18:12:10.022398+0000","last_became_peered":"2024-01-15T18:12:10.022398+0000","last_unstale":"2024-01-15T18:17:56.807714+0000","last_undegraded":"2024-01-15T18:17:56.807714+0000","last_fullsized":"2024-01-15T18:17:56.807714+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'12","last_scrub_stamp":"2024-01-15T18:17:56.807649+0000","last_deep_scrub":"117'12","last_deep_scrub_stamp":"2024-01-15T18:17:56.807649+0000","last_clean_scrub_stamp":"2024-01-15T18:17:56.807649+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:43.916577+0000","scrub_duration":0.008646964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"117'24","reported_seq":44,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.459308+0000","last_change":"2024-01-15T18:12:11.045910+0000","last_active":"2024-01-15T18:17:39.459308+0000","last_peered":"2024-01-15T18:17:39.459308+0000","last_clean":"2024-01-15T18:17:39.459308+0000","last_became_active":"2024-01-15T18:12:11.043735+0000","last_became_peered":"2024-01-15T18:12:11.043735+0000","last_unstale":"2024-01-15T18:17:39.459308+0000","last_undegraded":"2024-01-15T18:17:39.459308+0000","last_fullsized":"2024-01-15T18:17:39.459308+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"114'1","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:42.453388+0000","last_change":"2024-01-15T18:12:10.027277+0000","last_active":"2024-01-15T18:16:42.453388+0000","last_peered":"2024-01-15T18:16:42.453388+0000","last_clean":"2024-01-15T18:16:42.453388+0000","last_became_active":"2024-01-15T18:12:10.026730+0000","last_became_peered":"2024-01-15T18:12:10.026730+0000","last_unstale":"2024-01-15T18:16:42.453388+0000","last_undegraded":"2024-01-15T18:16:42.453388+0000","last_fullsized":"2024-01-15T18:16:42.453388+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"117'36","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.414999+0000","last_change":"2024-01-15T18:12:11.061270+0000","last_active":"2024-01-15T18:17:39.414999+0000","last_peered":"2024-01-15T18:17:39.414999+0000","last_clean":"2024-01-15T18:17:39.414999+0000","last_became_active":"2024-01-15T18:12:11.061002+0000","last_became_peered":"2024-01-15T18:12:11.061002+0000","last_unstale":"2024-01-15T18:17:39.414999+0000","last_undegraded":"2024-01-15T18:17:39.414999+0000","last_fullsized":"2024-01-15T18:17:39.414999+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"114'2","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:42.308185+0000","last_change":"2024-01-15T18:12:10.020526+0000","last_active":"2024-01-15T18:16:42.308185+0000","last_peered":"2024-01-15T18:16:42.308185+0000","last_clean":"2024-01-15T18:16:42.308185+0000","last_became_active":"2024-01-15T18:12:10.019706+0000","last_became_peered":"2024-01-15T18:12:10.019706+0000","last_unstale":"2024-01-15T18:16:42.308185+0000","last_undegraded":"2024-01-15T18:16:42.308185+0000","last_fullsized":"2024-01-15T18:16:42.308185+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"117'44","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.211608+0000","last_change":"2024-01-15T18:12:11.060480+0000","last_active":"2024-01-15T18:17:39.211608+0000","last_peered":"2024-01-15T18:17:39.211608+0000","last_clean":"2024-01-15T18:17:39.211608+0000","last_became_active":"2024-01-15T18:12:11.060154+0000","last_became_peered":"2024-01-15T18:12:11.060154+0000","last_unstale":"2024-01-15T18:17:39.211608+0000","last_undegraded":"2024-01-15T18:17:39.211608+0000","last_fullsized":"2024-01-15T18:17:39.211608+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:40:52.364460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"114'1","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.542920+0000","last_change":"2024-01-15T18:12:10.003308+0000","last_active":"2024-01-15T18:14:27.542920+0000","last_peered":"2024-01-15T18:14:27.542920+0000","last_clean":"2024-01-15T18:14:27.542920+0000","last_became_active":"2024-01-15T18:12:10.002957+0000","last_became_peered":"2024-01-15T18:12:10.002957+0000","last_unstale":"2024-01-15T18:14:27.542920+0000","last_undegraded":"2024-01-15T18:14:27.542920+0000","last_fullsized":"2024-01-15T18:14:27.542920+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:43.777497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"117'41","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.319156+0000","last_change":"2024-01-15T18:12:11.059234+0000","last_active":"2024-01-15T18:17:39.319156+0000","last_peered":"2024-01-15T18:17:39.319156+0000","last_clean":"2024-01-15T18:17:39.319156+0000","last_became_active":"2024-01-15T18:12:11.058945+0000","last_became_peered":"2024-01-15T18:12:11.058945+0000","last_unstale":"2024-01-15T18:17:39.319156+0000","last_undegraded":"2024-01-15T18:17:39.319156+0000","last_fullsized":"2024-01-15T18:17:39.319156+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:36:37.113206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:27.411059+0000","last_change":"2024-01-15T18:12:10.028589+0000","last_active":"2024-01-15T18:14:27.411059+0000","last_peered":"2024-01-15T18:14:27.411059+0000","last_clean":"2024-01-15T18:14:27.411059+0000","last_became_active":"2024-01-15T18:12:10.028284+0000","last_became_peered":"2024-01-15T18:12:10.028284+0000","last_unstale":"2024-01-15T18:14:27.411059+0000","last_undegraded":"2024-01-15T18:14:27.411059+0000","last_fullsized":"2024-01-15T18:14:27.411059+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:46:23.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"117'45","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.543742+0000","last_change":"2024-01-15T18:12:11.051787+0000","last_active":"2024-01-15T18:17:39.543742+0000","last_peered":"2024-01-15T18:17:39.543742+0000","last_clean":"2024-01-15T18:17:39.543742+0000","last_became_active":"2024-01-15T18:12:11.051403+0000","last_became_peered":"2024-01-15T18:12:11.051403+0000","last_unstale":"2024-01-15T18:17:39.543742+0000","last_undegraded":"2024-01-15T18:17:39.543742+0000","last_fullsized":"2024-01-15T18:17:39.543742+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:05:35.722215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"117'3","reported_seq":20,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:42.307166+0000","last_change":"2024-01-15T18:12:10.021913+0000","last_active":"2024-01-15T18:16:42.307166+0000","last_peered":"2024-01-15T18:16:42.307166+0000","last_clean":"2024-01-15T18:16:42.307166+0000","last_became_active":"2024-01-15T18:12:10.021344+0000","last_became_peered":"2024-01-15T18:12:10.021344+0000","last_unstale":"2024-01-15T18:16:42.307166+0000","last_undegraded":"2024-01-15T18:16:42.307166+0000","last_fullsized":"2024-01-15T18:16:42.307166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"117'43","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.951562+0000","last_change":"2024-01-15T18:12:11.059848+0000","last_active":"2024-01-15T18:17:38.951562+0000","last_peered":"2024-01-15T18:17:38.951562+0000","last_clean":"2024-01-15T18:17:38.951562+0000","last_became_active":"2024-01-15T18:12:11.059311+0000","last_became_peered":"2024-01-15T18:12:11.059311+0000","last_unstale":"2024-01-15T18:17:38.951562+0000","last_undegraded":"2024-01-15T18:17:38.951562+0000","last_fullsized":"2024-01-15T18:17:38.951562+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:32.306579+0000","last_change":"2024-01-15T18:12:10.018301+0000","last_active":"2024-01-15T18:16:32.306579+0000","last_peered":"2024-01-15T18:16:32.306579+0000","last_clean":"2024-01-15T18:16:32.306579+0000","last_became_active":"2024-01-15T18:12:10.017411+0000","last_became_peered":"2024-01-15T18:12:10.017411+0000","last_unstale":"2024-01-15T18:16:32.306579+0000","last_undegraded":"2024-01-15T18:16:32.306579+0000","last_fullsized":"2024-01-15T18:16:32.306579+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"117'44","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.231615+0000","last_change":"2024-01-15T18:12:11.056765+0000","last_active":"2024-01-15T18:17:39.231615+0000","last_peered":"2024-01-15T18:17:39.231615+0000","last_clean":"2024-01-15T18:17:39.231615+0000","last_became_active":"2024-01-15T18:12:11.056311+0000","last_became_peered":"2024-01-15T18:12:11.056311+0000","last_unstale":"2024-01-15T18:17:39.231615+0000","last_undegraded":"2024-01-15T18:17:39.231615+0000","last_fullsized":"2024-01-15T18:17:39.231615+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:59:53.750812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:32.305367+0000","last_change":"2024-01-15T18:12:10.023638+0000","last_active":"2024-01-15T18:16:32.305367+0000","last_peered":"2024-01-15T18:16:32.305367+0000","last_clean":"2024-01-15T18:16:32.305367+0000","last_became_active":"2024-01-15T18:12:10.022687+0000","last_became_peered":"2024-01-15T18:12:10.022687+0000","last_unstale":"2024-01-15T18:16:32.305367+0000","last_undegraded":"2024-01-15T18:16:32.305367+0000","last_fullsized":"2024-01-15T18:16:32.305367+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"117'42","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.467336+0000","last_change":"2024-01-15T18:12:11.062171+0000","last_active":"2024-01-15T18:17:39.467336+0000","last_peered":"2024-01-15T18:17:39.467336+0000","last_clean":"2024-01-15T18:17:39.467336+0000","last_became_active":"2024-01-15T18:12:11.061830+0000","last_became_peered":"2024-01-15T18:12:11.061830+0000","last_unstale":"2024-01-15T18:17:39.467336+0000","last_undegraded":"2024-01-15T18:17:39.467336+0000","last_fullsized":"2024-01-15T18:17:39.467336+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109124,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:32.451352+0000","last_change":"2024-01-15T18:12:10.024893+0000","last_active":"2024-01-15T18:16:32.451352+0000","last_peered":"2024-01-15T18:16:32.451352+0000","last_clean":"2024-01-15T18:16:32.451352+0000","last_became_active":"2024-01-15T18:12:10.024554+0000","last_became_peered":"2024-01-15T18:12:10.024554+0000","last_unstale":"2024-01-15T18:16:32.451352+0000","last_undegraded":"2024-01-15T18:16:32.451352+0000","last_fullsized":"2024-01-15T18:16:32.451352+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"117'36","reported_seq":55,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.210156+0000","last_change":"2024-01-15T18:12:11.051179+0000","last_active":"2024-01-15T18:17:39.210156+0000","last_peered":"2024-01-15T18:17:39.210156+0000","last_clean":"2024-01-15T18:17:39.210156+0000","last_became_active":"2024-01-15T18:12:11.050747+0000","last_became_peered":"2024-01-15T18:12:11.050747+0000","last_unstale":"2024-01-15T18:17:39.210156+0000","last_undegraded":"2024-01-15T18:17:39.210156+0000","last_fullsized":"2024-01-15T18:17:39.210156+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:55:55.003111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"114'2","reported_seq":19,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:32.332461+0000","last_change":"2024-01-15T18:12:10.025674+0000","last_active":"2024-01-15T18:16:32.332461+0000","last_peered":"2024-01-15T18:16:32.332461+0000","last_clean":"2024-01-15T18:16:32.332461+0000","last_became_active":"2024-01-15T18:12:10.025356+0000","last_became_peered":"2024-01-15T18:12:10.025356+0000","last_unstale":"2024-01-15T18:16:32.332461+0000","last_undegraded":"2024-01-15T18:16:32.332461+0000","last_fullsized":"2024-01-15T18:16:32.332461+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"117'41","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.072114+0000","last_change":"2024-01-15T18:12:11.050465+0000","last_active":"2024-01-15T18:17:39.072114+0000","last_peered":"2024-01-15T18:17:39.072114+0000","last_clean":"2024-01-15T18:17:39.072114+0000","last_became_active":"2024-01-15T18:12:11.050116+0000","last_became_peered":"2024-01-15T18:12:11.050116+0000","last_unstale":"2024-01-15T18:17:39.072114+0000","last_undegraded":"2024-01-15T18:17:39.072114+0000","last_fullsized":"2024-01-15T18:17:39.072114+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"117'18","reported_seq":34,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:40.209200+0000","last_change":"2024-01-15T18:12:10.020848+0000","last_active":"2024-01-15T18:17:40.209200+0000","last_peered":"2024-01-15T18:17:40.209200+0000","last_clean":"2024-01-15T18:17:40.209200+0000","last_became_active":"2024-01-15T18:12:10.020497+0000","last_became_peered":"2024-01-15T18:12:10.020497+0000","last_unstale":"2024-01-15T18:17:40.209200+0000","last_undegraded":"2024-01-15T18:17:40.209200+0000","last_fullsized":"2024-01-15T18:17:40.209200+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:37.798936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"117'46","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.396693+0000","last_change":"2024-01-15T18:12:11.051721+0000","last_active":"2024-01-15T18:17:39.396693+0000","last_peered":"2024-01-15T18:17:39.396693+0000","last_clean":"2024-01-15T18:17:39.396693+0000","last_became_active":"2024-01-15T18:12:11.050810+0000","last_became_peered":"2024-01-15T18:12:11.050810+0000","last_unstale":"2024-01-15T18:17:39.396693+0000","last_undegraded":"2024-01-15T18:17:39.396693+0000","last_fullsized":"2024-01-15T18:17:39.396693+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:22.449328+0000","last_change":"2024-01-15T18:12:10.025006+0000","last_active":"2024-01-15T18:16:22.449328+0000","last_peered":"2024-01-15T18:16:22.449328+0000","last_clean":"2024-01-15T18:16:22.449328+0000","last_became_active":"2024-01-15T18:12:10.024524+0000","last_became_peered":"2024-01-15T18:12:10.024524+0000","last_unstale":"2024-01-15T18:16:22.449328+0000","last_undegraded":"2024-01-15T18:16:22.449328+0000","last_fullsized":"2024-01-15T18:16:22.449328+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"117'39","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.572497+0000","last_change":"2024-01-15T18:12:11.061751+0000","last_active":"2024-01-15T18:17:38.572497+0000","last_peered":"2024-01-15T18:17:38.572497+0000","last_clean":"2024-01-15T18:17:38.572497+0000","last_became_active":"2024-01-15T18:12:11.061454+0000","last_became_peered":"2024-01-15T18:12:11.061454+0000","last_unstale":"2024-01-15T18:17:38.572497+0000","last_undegraded":"2024-01-15T18:17:38.572497+0000","last_fullsized":"2024-01-15T18:17:38.572497+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:20:32.804971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:22.331134+0000","last_change":"2024-01-15T18:12:10.018428+0000","last_active":"2024-01-15T18:16:22.331134+0000","last_peered":"2024-01-15T18:16:22.331134+0000","last_clean":"2024-01-15T18:16:22.331134+0000","last_became_active":"2024-01-15T18:12:10.018057+0000","last_became_peered":"2024-01-15T18:12:10.018057+0000","last_unstale":"2024-01-15T18:16:22.331134+0000","last_undegraded":"2024-01-15T18:16:22.331134+0000","last_fullsized":"2024-01-15T18:16:22.331134+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"117'44","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.352556+0000","last_change":"2024-01-15T18:12:11.039995+0000","last_active":"2024-01-15T18:17:39.352556+0000","last_peered":"2024-01-15T18:17:39.352556+0000","last_clean":"2024-01-15T18:17:39.352556+0000","last_became_active":"2024-01-15T18:12:11.039586+0000","last_became_peered":"2024-01-15T18:12:11.039586+0000","last_unstale":"2024-01-15T18:17:39.352556+0000","last_undegraded":"2024-01-15T18:17:39.352556+0000","last_fullsized":"2024-01-15T18:17:39.352556+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:34:38.812736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.408970+0000","last_change":"2024-01-15T18:12:10.019758+0000","last_active":"2024-01-15T18:14:17.408970+0000","last_peered":"2024-01-15T18:14:17.408970+0000","last_clean":"2024-01-15T18:14:17.408970+0000","last_became_active":"2024-01-15T18:12:10.019424+0000","last_became_peered":"2024-01-15T18:12:10.019424+0000","last_unstale":"2024-01-15T18:14:17.408970+0000","last_undegraded":"2024-01-15T18:14:17.408970+0000","last_fullsized":"2024-01-15T18:14:17.408970+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:04:45.797791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"117'56","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.420858+0000","last_change":"2024-01-15T18:12:11.035857+0000","last_active":"2024-01-15T18:17:39.420858+0000","last_peered":"2024-01-15T18:17:39.420858+0000","last_clean":"2024-01-15T18:17:39.420858+0000","last_became_active":"2024-01-15T18:12:11.035238+0000","last_became_peered":"2024-01-15T18:12:11.035238+0000","last_unstale":"2024-01-15T18:17:39.420858+0000","last_undegraded":"2024-01-15T18:17:39.420858+0000","last_fullsized":"2024-01-15T18:17:39.420858+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:29:53.321848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:17.326548+0000","last_change":"2024-01-15T18:12:10.017511+0000","last_active":"2024-01-15T18:14:17.326548+0000","last_peered":"2024-01-15T18:14:17.326548+0000","last_clean":"2024-01-15T18:14:17.326548+0000","last_became_active":"2024-01-15T18:12:10.017190+0000","last_became_peered":"2024-01-15T18:12:10.017190+0000","last_unstale":"2024-01-15T18:14:17.326548+0000","last_undegraded":"2024-01-15T18:14:17.326548+0000","last_fullsized":"2024-01-15T18:14:17.326548+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:00:16.742355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"117'46","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.384006+0000","last_change":"2024-01-15T18:12:11.056667+0000","last_active":"2024-01-15T18:17:39.384006+0000","last_peered":"2024-01-15T18:17:39.384006+0000","last_clean":"2024-01-15T18:17:39.384006+0000","last_became_active":"2024-01-15T18:12:11.056111+0000","last_became_peered":"2024-01-15T18:12:11.056111+0000","last_unstale":"2024-01-15T18:17:39.384006+0000","last_undegraded":"2024-01-15T18:17:39.384006+0000","last_fullsized":"2024-01-15T18:17:39.384006+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:18:00.144603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"117'10","reported_seq":36,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:22.305011+0000","last_change":"2024-01-15T18:12:10.021210+0000","last_active":"2024-01-15T18:16:22.305011+0000","last_peered":"2024-01-15T18:16:22.305011+0000","last_clean":"2024-01-15T18:16:22.305011+0000","last_became_active":"2024-01-15T18:12:10.020890+0000","last_became_peered":"2024-01-15T18:12:10.020890+0000","last_unstale":"2024-01-15T18:16:22.305011+0000","last_undegraded":"2024-01-15T18:16:22.305011+0000","last_fullsized":"2024-01-15T18:16:22.305011+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"117'39","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.477235+0000","last_change":"2024-01-15T18:12:11.059132+0000","last_active":"2024-01-15T18:17:39.477235+0000","last_peered":"2024-01-15T18:17:39.477235+0000","last_clean":"2024-01-15T18:17:39.477235+0000","last_became_active":"2024-01-15T18:12:11.057943+0000","last_became_peered":"2024-01-15T18:12:11.057943+0000","last_unstale":"2024-01-15T18:17:39.477235+0000","last_undegraded":"2024-01-15T18:17:39.477235+0000","last_fullsized":"2024-01-15T18:17:39.477235+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:12.447139+0000","last_change":"2024-01-15T18:12:10.010681+0000","last_active":"2024-01-15T18:16:12.447139+0000","last_peered":"2024-01-15T18:16:12.447139+0000","last_clean":"2024-01-15T18:16:12.447139+0000","last_became_active":"2024-01-15T18:12:10.009870+0000","last_became_peered":"2024-01-15T18:12:10.009870+0000","last_unstale":"2024-01-15T18:16:12.447139+0000","last_undegraded":"2024-01-15T18:16:12.447139+0000","last_fullsized":"2024-01-15T18:16:12.447139+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"117'31","reported_seq":55,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.499297+0000","last_change":"2024-01-15T18:12:11.051496+0000","last_active":"2024-01-15T18:17:39.499297+0000","last_peered":"2024-01-15T18:17:39.499297+0000","last_clean":"2024-01-15T18:17:39.499297+0000","last_became_active":"2024-01-15T18:12:11.051092+0000","last_became_peered":"2024-01-15T18:12:11.051092+0000","last_unstale":"2024-01-15T18:17:39.499297+0000","last_undegraded":"2024-01-15T18:17:39.499297+0000","last_fullsized":"2024-01-15T18:17:39.499297+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":16,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:14:32.412208+0000","last_change":"2024-01-15T18:12:10.026923+0000","last_active":"2024-01-15T18:14:32.412208+0000","last_peered":"2024-01-15T18:14:32.412208+0000","last_clean":"2024-01-15T18:14:32.412208+0000","last_became_active":"2024-01-15T18:12:10.026651+0000","last_became_peered":"2024-01-15T18:12:10.026651+0000","last_unstale":"2024-01-15T18:14:32.412208+0000","last_undegraded":"2024-01-15T18:14:32.412208+0000","last_fullsized":"2024-01-15T18:14:32.412208+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:24:14.981958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":251,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:15:02.417603+0000","last_change":"2024-01-15T17:54:59.626595+0000","last_active":"2024-01-15T18:15:02.417603+0000","last_peered":"2024-01-15T18:15:02.417603+0000","last_clean":"2024-01-15T18:15:02.417603+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:15:02.417603+0000","last_undegraded":"2024-01-15T18:15:02.417603+0000","last_fullsized":"2024-01-15T18:15:02.417603+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T17:54:58.463479+0000","last_clean_scrub_stamp":"2024-01-15T17:54:58.463479+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:54:54.764393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710133,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2821,"ondisk_log_size":2821,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":641570,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279,"num_write_kb":755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1343488,"data_stored":1283140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542423,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834588,"kb_used_data":1767380,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537252,"statfs":{"total":96636764160,"available":94758146048,"internally_reserved":0,"allocated":1809797120,"data_stored":1808970858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.254,"15min":1.383},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.40400000000000003},"max":{"1min":1.599,"5min":4.0229999999999997,"15min":19.843},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.49,"15min":1.5609999999999999},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.53400000000000003},"max":{"1min":1.6319999999999999,"5min":16.678999999999998,"15min":30.498000000000001},"last":0.94499999999999995}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2649999999999999,"15min":1.397},"min":{"1min":0.91400000000000003,"5min":0.56599999999999995,"15min":0.45600000000000002},"max":{"1min":1.5700000000000001,"5min":3.9430000000000001,"15min":19.917000000000002},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.54,"15min":1.5669999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.71199999999999997},"max":{"1min":1.6319999999999999,"5min":16.626999999999999,"15min":28.613},"last":1.292}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2689999999999999,"15min":1.391},"min":{"1min":0.98399999999999999,"5min":0.75600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6479999999999999,"5min":1.8140000000000001,"15min":12.061},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.528,"15min":1.5800000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.48999999999999999},"max":{"1min":1.77,"5min":16.792000000000002,"15min":16.792000000000002},"last":1.5720000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.306,"15min":1.5229999999999999},"min":{"1min":1.0029999999999999,"5min":0.80700000000000005,"15min":0.56000000000000005},"max":{"1min":1.6180000000000001,"5min":1.734,"15min":16.632000000000001},"last":1.327},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.619,"15min":1.639},"min":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.56699999999999995},"max":{"1min":1.6930000000000001,"5min":16.129000000000001,"15min":16.129000000000001},"last":1.7390000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.571,"15min":1.627},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.54200000000000004},"max":{"1min":1.585,"5min":16.454000000000001,"15min":23.042000000000002},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.357,"15min":1.677},"min":{"1min":0.873,"5min":0.80000000000000004,"15min":0.496},"max":{"1min":1.7410000000000001,"5min":1.885,"15min":38.682000000000002},"last":1.4710000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4259999999999999,"15min":1.5860000000000001},"min":{"1min":0.96799999999999997,"5min":0.76800000000000002,"15min":0.52700000000000002},"max":{"1min":1.867,"5min":4.5359999999999996,"15min":13.379},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.71,"15min":1.7430000000000001},"min":{"1min":1.0649999999999999,"5min":0.98499999999999999,"15min":0.624},"max":{"1min":1.843,"5min":16.553000000000001,"15min":22.933},"last":1.524}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.7969999999999999,"15min":1.8600000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":0.57399999999999995},"max":{"1min":1.8140000000000001,"5min":16.725999999999999,"15min":23.004000000000001},"last":1.637},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4850000000000001,"15min":1.659},"min":{"1min":1.1220000000000001,"5min":1.008,"15min":0.58299999999999996},"max":{"1min":1.897,"5min":1.952,"15min":9.2599999999999998},"last":1.71}]}]},{"osd":6,"up_from":14,"seq":60129542422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1256408,"kb_used_data":1193360,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":93115432,"statfs":{"total":96636764160,"available":95350202368,"internally_reserved":0,"allocated":1222000640,"data_stored":1221177506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2629999999999999,"15min":1.464},"min":{"1min":0.79600000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":31.805},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.244,"5min":1.3,"15min":1.4870000000000001},"min":{"1min":0.749,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8340000000000001,"5min":3.5409999999999999,"15min":31.702999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1339999999999999,"15min":1.2609999999999999},"min":{"1min":0.69699999999999995,"5min":0.59999999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.778,"15min":15.064},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2050000000000001,"15min":1.4299999999999999},"min":{"1min":0.86499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6279999999999999,"5min":1.9650000000000001,"15min":23.495999999999999},"last":0.90400000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2849999999999999,"15min":1.5349999999999999},"min":{"1min":0.81699999999999995,"5min":0.79100000000000004,"15min":0.59399999999999997},"max":{"1min":1.502,"5min":1.8899999999999999,"15min":23.087},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.331,"15min":1.397},"min":{"1min":1.0089999999999999,"5min":0.77000000000000002,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.9990000000000001,"15min":14.94},"last":1.135}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4359999999999999,"15min":1.6040000000000001},"min":{"1min":1.0620000000000001,"5min":0.65200000000000002,"15min":0.59899999999999998},"max":{"1min":1.7,"5min":1.8160000000000001,"15min":15.151999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3779999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6080000000000001,"5min":3.8500000000000001,"15min":13.795},"last":0.96299999999999997}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2729999999999999,"15min":1.585},"min":{"1min":0.90900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.53,"5min":3.3719999999999999,"15min":23.327000000000002},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.3560000000000001,"15min":1.548},"min":{"1min":0.80900000000000005,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.6839999999999999,"5min":1.8839999999999999,"15min":15.239000000000001},"last":1.361}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.331,"15min":1.502},"min":{"1min":0.88700000000000001,"5min":0.58099999999999996,"15min":0.52500000000000002},"max":{"1min":1.7210000000000001,"5min":3.6480000000000001,"15min":13.593},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.55,"15min":1.8049999999999999},"min":{"1min":1.1930000000000001,"5min":0.66200000000000003,"15min":0.621},"max":{"1min":1.702,"5min":2.306,"15min":23.417999999999999},"last":1.47}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.47,"15min":1.716},"min":{"1min":0.98799999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8220000000000001,"5min":3.105,"15min":21.265000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.542,"15min":1.899},"min":{"1min":1.1200000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.7789999999999999,"5min":2.2229999999999999,"15min":23.738},"last":1.5569999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542422,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908440,"kb_used_data":832912,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93463400,"statfs":{"total":96636764160,"available":95706521600,"internally_reserved":0,"allocated":852901888,"data_stored":852079166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.202,"15min":1.2669999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.47799999999999998},"max":{"1min":1.595,"5min":2.7919999999999998,"15min":19.266999999999999},"last":1.522},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.752,"5min":0.748,"15min":0.51400000000000001},"max":{"1min":1.671,"5min":1.7529999999999999,"15min":3.1760000000000002},"last":1.2669999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.359,"15min":1.3149999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.60699999999999998},"max":{"1min":1.653,"5min":15.554,"15min":15.554},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.242,"5min":1.302,"15min":1.3540000000000001},"min":{"1min":0.80100000000000005,"5min":0.69599999999999995,"15min":0.59099999999999997},"max":{"1min":1.643,"5min":6.1449999999999996,"15min":9.2379999999999995},"last":1.1990000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.621},"max":{"1min":1.694,"5min":2.3500000000000001,"15min":4.0369999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.512,"15min":1.4299999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.56000000000000005},"max":{"1min":1.617,"5min":15.468999999999999,"15min":15.468999999999999},"last":1.0489999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.327,"15min":1.409},"min":{"1min":0.78200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.718,"5min":1.984,"15min":9.1929999999999996},"last":1.629},{"interface":"front","average":{"1min":1.357,"5min":1.4379999999999999,"15min":1.4650000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.64600000000000002},"max":{"1min":1.774,"5min":2.2949999999999999,"15min":7.0110000000000001},"last":1.6739999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.4319999999999999,"15min":1.47},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.41399999999999998},"max":{"1min":1.6040000000000001,"5min":15.254,"15min":15.254},"last":1.397},{"interface":"front","average":{"1min":1.331,"5min":1.4019999999999999,"15min":1.4710000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.8080000000000001,"5min":1.923,"15min":9.1519999999999992},"last":1.7210000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.456},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.39600000000000002},"max":{"1min":1.8480000000000001,"5min":2.2250000000000001,"15min":3.7559999999999998},"last":1.569},{"interface":"front","average":{"1min":1.415,"5min":1.593,"15min":1.5680000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53000000000000003},"max":{"1min":1.7330000000000001,"5min":15.366,"15min":15.366},"last":1.323}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6100000000000001},"min":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.45800000000000002},"max":{"1min":1.873,"5min":2.0609999999999999,"15min":9.0600000000000005},"last":1.788},{"interface":"front","average":{"1min":1.522,"5min":1.5840000000000001,"15min":1.6890000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.57299999999999995},"max":{"1min":1.899,"5min":3.1659999999999999,"15min":19.356999999999999},"last":1.8999999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1711124,"kb_used_data":1643916,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92660716,"statfs":{"total":96636764160,"available":94884573184,"internally_reserved":0,"allocated":1683369984,"data_stored":1682551014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.264,"15min":1.3360000000000001},"min":{"1min":0.94799999999999995,"5min":0.70199999999999996,"15min":0.39200000000000002},"max":{"1min":1.601,"5min":2.004,"15min":17.346},"last":1.413},{"interface":"front","average":{"1min":1.409,"5min":1.3500000000000001,"15min":1.361},"min":{"1min":1.0900000000000001,"5min":0.97499999999999998,"15min":0.40100000000000002},"max":{"1min":1.744,"5min":1.804,"15min":7.9160000000000004},"last":1.4330000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.1950000000000001,"15min":1.24},"min":{"1min":0.95199999999999996,"5min":0.76600000000000001,"15min":0.439},"max":{"1min":1.6699999999999999,"5min":1.861,"15min":11.128},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.23,"15min":1.234},"min":{"1min":0.876,"5min":0.82999999999999996,"15min":0.60799999999999998},"max":{"1min":1.619,"5min":1.7470000000000001,"15min":5.6109999999999998},"last":0.81799999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.294,"15min":1.343},"min":{"1min":0.92300000000000004,"5min":0.83399999999999996,"15min":0.56899999999999995},"max":{"1min":1.544,"5min":1.8560000000000001,"15min":18.442},"last":1.355},{"interface":"front","average":{"1min":1.341,"5min":1.3720000000000001,"15min":1.399},"min":{"1min":1.1459999999999999,"5min":0.94799999999999995,"15min":0.81899999999999995},"max":{"1min":1.669,"5min":1.861,"15min":5.7050000000000001},"last":1.0880000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.375,"15min":1.462},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.56699999999999995},"max":{"1min":1.6719999999999999,"5min":2.1859999999999999,"15min":17.364999999999998},"last":1.026},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3380000000000001,"15min":1.3899999999999999},"min":{"1min":1.022,"5min":0.85099999999999998,"15min":0.45300000000000001},"max":{"1min":1.601,"5min":1.7050000000000001,"15min":9.3320000000000007},"last":1.1679999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.29,"15min":1.4590000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.49099999999999999},"max":{"1min":1.6519999999999999,"5min":2.4089999999999998,"15min":29.422000000000001},"last":1.125},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3420000000000001,"15min":1.4379999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.53700000000000003},"max":{"1min":1.7549999999999999,"5min":1.9450000000000001,"15min":12.332000000000001},"last":1.4810000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2629999999999999,"15min":1.3899999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.54200000000000004},"max":{"1min":1.752,"5min":2.117,"15min":11.09},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.482,"15min":1.6120000000000001},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":0.51100000000000001},"max":{"1min":1.718,"5min":2.2639999999999998,"15min":17.224},"last":1.2529999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.504,"15min":1.593},"min":{"1min":1.0169999999999999,"5min":0.93300000000000005,"15min":0.60499999999999998},"max":{"1min":1.8400000000000001,"5min":2.052,"15min":12.247},"last":1.383},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6439999999999999,"15min":1.73},"min":{"1min":1.294,"5min":1.159,"15min":0.61399999999999999},"max":{"1min":1.9039999999999999,"5min":2.3570000000000002,"15min":17.324999999999999},"last":1.4570000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542424,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462036,"kb_used_data":1386508,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":92909804,"statfs":{"total":96636764160,"available":95139639296,"internally_reserved":0,"allocated":1419784192,"data_stored":1418949643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0349999999999999,"15min":1.1639999999999999},"min":{"1min":0.66400000000000003,"5min":0.52800000000000002,"15min":0.33300000000000002},"max":{"1min":1.7989999999999999,"5min":2.915,"15min":18.399999999999999},"last":0.63},{"interface":"front","average":{"1min":1.258,"5min":1.2509999999999999,"15min":1.2849999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.44400000000000001},"max":{"1min":1.8560000000000001,"5min":3.028,"15min":18.795999999999999},"last":1.1719999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.05,"15min":1.1299999999999999},"min":{"1min":0.65800000000000003,"5min":0.51200000000000001,"15min":0.32600000000000001},"max":{"1min":1.4870000000000001,"5min":5.3520000000000003,"15min":12.372},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.085,"5min":1.1180000000000001,"15min":1.2589999999999999},"min":{"1min":0.66000000000000003,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.736,"5min":4.125,"15min":11.228999999999999},"last":0.69699999999999995}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.218,"15min":1.2529999999999999},"min":{"1min":0.74199999999999999,"5min":0.59999999999999998,"15min":0.375},"max":{"1min":1.8680000000000001,"5min":2.3929999999999998,"15min":10.964},"last":1.28},{"interface":"front","average":{"1min":1.401,"5min":1.45,"15min":1.534},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7569999999999999,"5min":3.0659999999999998,"15min":20.495000000000001},"last":0.90000000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.389,"15min":1.51},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.67200000000000004},"max":{"1min":1.865,"5min":2.1139999999999999,"15min":13.536},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4530000000000001,"15min":1.5529999999999999},"min":{"1min":0.98999999999999999,"5min":0.83099999999999996,"15min":0.63700000000000001},"max":{"1min":1.905,"5min":6.2439999999999998,"15min":16.398},"last":1.3240000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.478,"15min":1.6519999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.67200000000000004},"max":{"1min":2.0670000000000002,"5min":2.7269999999999999,"15min":20.606000000000002},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.6060000000000001,"15min":1.631},"min":{"1min":1.0269999999999999,"5min":0.94299999999999995,"15min":0.66800000000000004},"max":{"1min":2.0489999999999999,"5min":2.617,"15min":11.603},"last":1.9910000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.554,"15min":1.675},"min":{"1min":0.93100000000000005,"5min":0.85499999999999998,"15min":0.53500000000000003},"max":{"1min":2.1230000000000002,"5min":6.3289999999999997,"15min":13.648999999999999},"last":1.589},{"interface":"front","average":{"1min":1.554,"5min":1.6100000000000001,"15min":1.7529999999999999},"min":{"1min":1.2190000000000001,"5min":1.0249999999999999,"15min":0.65800000000000003},"max":{"1min":1.849,"5min":6.367,"15min":20.556000000000001},"last":1.365}]},{"osd":7,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.641,"15min":1.76},"min":{"1min":1.159,"5min":0.97899999999999998,"15min":0.60499999999999998},"max":{"1min":2.1269999999999998,"5min":4.8630000000000004,"15min":28.843},"last":2.2050000000000001},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6559999999999999,"15min":1.738},"min":{"1min":1.1830000000000001,"5min":0.80100000000000005,"15min":0.67200000000000004},"max":{"1min":2.0680000000000001,"5min":6.9569999999999999,"15min":15.143000000000001},"last":1.528}]}]},{"osd":2,"up_from":14,"seq":60129542424,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":982100,"kb_used_data":910732,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93389740,"statfs":{"total":96636764160,"available":95631093760,"internally_reserved":0,"allocated":932589568,"data_stored":931771659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.161,"15min":1.083},"min":{"1min":0.67600000000000005,"5min":0.51000000000000001,"15min":0.26600000000000001},"max":{"1min":1.419,"5min":22.707000000000001,"15min":22.707000000000001},"last":1.274},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1970000000000001,"15min":1.357},"min":{"1min":0.76300000000000001,"5min":0.61099999999999999,"15min":0.39300000000000002},"max":{"1min":1.6879999999999999,"5min":2.5710000000000002,"15min":15.314},"last":1.1140000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1759999999999999,"15min":1.3240000000000001},"min":{"1min":0.73299999999999998,"5min":0.58599999999999997,"15min":0.38300000000000001},"max":{"1min":1.546,"5min":2.8679999999999999,"15min":34.463999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.327,"15min":1.504},"min":{"1min":0.90700000000000003,"5min":0.73899999999999999,"15min":0.373},"max":{"1min":1.675,"5min":2.794,"15min":25.859999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1870000000000001,"15min":1.302},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.34399999999999997},"max":{"1min":1.464,"5min":2.7080000000000002,"15min":12.93},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.284,"5min":1.6919999999999999,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.46400000000000002},"max":{"1min":1.903,"5min":37.021999999999998,"15min":37.021999999999998},"last":1.3460000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6879999999999999},"min":{"1min":1.0289999999999999,"5min":0.96999999999999997,"15min":0.55700000000000005},"max":{"1min":1.78,"5min":3.419,"15min":29.986999999999998},"last":1.577},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.518,"15min":1.7150000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.56799999999999995},"max":{"1min":1.883,"5min":2.976,"15min":21.199999999999999},"last":1.256}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.456,"15min":1.6499999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.51500000000000001},"max":{"1min":2.0880000000000001,"5min":2.218,"15min":14.066000000000001},"last":1.665},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.643,"15min":1.7849999999999999},"min":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":0.60899999999999999},"max":{"1min":1.8180000000000001,"5min":3.4790000000000001,"15min":14.025},"last":1.867}]},{"osd":6,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.6160000000000001,"15min":1.8069999999999999},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":0.68000000000000005},"max":{"1min":1.9910000000000001,"5min":3.0800000000000001,"15min":15.683999999999999},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.585,"15min":1.784},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.58899999999999997},"max":{"1min":2.2930000000000001,"5min":10.371,"15min":12.629},"last":1.4950000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.671,"15min":2.1419999999999999},"min":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":0.63100000000000001},"max":{"1min":2.2210000000000001,"5min":3.177,"15min":57.006999999999998},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.6299999999999999,"15min":1.9630000000000001},"min":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":0.56100000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":22.853999999999999},"last":1.806}]}]},{"osd":1,"up_from":14,"seq":60129542423,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1137764,"kb_used_data":1066396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93234076,"statfs":{"total":96636764160,"available":95471693824,"internally_reserved":0,"allocated":1091989504,"data_stored":1091155706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.96499999999999997,"15min":1.177},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.34399999999999997},"max":{"1min":1.1419999999999999,"5min":2.7389999999999999,"15min":30.460999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.115,"15min":1.26},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.314},"max":{"1min":1.6539999999999999,"5min":5.4779999999999998,"15min":18.658000000000001},"last":0.86599999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.075,"15min":1.1579999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.5589999999999999,"5min":1.7809999999999999,"15min":7.117},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.099,"5min":1.325,"15min":1.5129999999999999},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.728,"5min":1.98,"15min":30.593},"last":0.79800000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2529999999999999,"15min":1.45},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.48599999999999999},"max":{"1min":1.6779999999999999,"5min":1.889,"15min":30.547000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.1619999999999999,"15min":1.3200000000000001},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":2.0459999999999998,"15min":17.263000000000002},"last":0.73799999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.6639999999999999,"15min":1.6899999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.72699999999999998},"max":{"1min":1.6319999999999999,"5min":14.103,"15min":17.420000000000002},"last":1.149},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.653,"15min":1.6040000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7849999999999999,"5min":14.140000000000001,"15min":14.140000000000001},"last":1.2849999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3859999999999999,"15min":1.5329999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.59599999999999997},"max":{"1min":1.843,"5min":2.077,"15min":9.0440000000000005},"last":1.111},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.508,"15min":1.7989999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.68300000000000005},"max":{"1min":1.891,"5min":2.3599999999999999,"15min":34.042999999999999},"last":1.2589999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.494,"15min":1.6830000000000001},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":0.64100000000000001},"max":{"1min":1.9790000000000001,"5min":2.3780000000000001,"15min":34.158000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.4850000000000001,"15min":1.637},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.62},"max":{"1min":1.8260000000000001,"5min":2.5579999999999998,"15min":17.510000000000002},"last":1.085}]},{"osd":7,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.603,"15min":1.8700000000000001},"min":{"1min":0.98299999999999998,"5min":0.82999999999999996,"15min":0.77300000000000002},"max":{"1min":1.998,"5min":2.4329999999999998,"15min":33.659999999999997},"last":1.179},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.6439999999999999,"15min":1.742},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.70299999999999996},"max":{"1min":1.9370000000000001,"5min":2.6669999999999998,"15min":11.882999999999999},"last":1.3420000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542424,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1769764,"kb_used_data":1698396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":92602076,"statfs":{"total":96636764160,"available":94824525824,"internally_reserved":0,"allocated":1739157504,"data_stored":1738307390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.1080000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4630000000000001,"5min":2.5209999999999999,"15min":12.050000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3100000000000001,"15min":1.357},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.36399999999999999},"max":{"1min":1.609,"5min":6.6890000000000001,"15min":11.962},"last":1.0369999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.248,"15min":1.2230000000000001},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.45500000000000002},"max":{"1min":1.7050000000000001,"5min":6.8099999999999996,"15min":11.59},"last":1.387},{"interface":"front","average":{"1min":0.995,"5min":1.0620000000000001,"15min":1.1970000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.6870000000000001,"5min":1.825,"15min":14.512},"last":0.92100000000000004}]},{"osd":3,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.385,"15min":1.425},"min":{"1min":0.58899999999999997,"5min":0.51300000000000001,"15min":0.50800000000000001},"max":{"1min":1.7130000000000001,"5min":6.8719999999999999,"15min":11.904},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1899999999999999,"15min":1.1830000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.35199999999999998},"max":{"1min":1.696,"5min":2.3919999999999999,"15min":6.1539999999999999},"last":1.226}]},{"osd":4,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.29,"15min":1.542},"min":{"1min":0.68000000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.5860000000000001,"5min":2.1829999999999998,"15min":32.115000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.579,"15min":1.6259999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.67700000000000005},"max":{"1min":1.6890000000000001,"5min":6.9329999999999998,"15min":11.992000000000001},"last":1.3149999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3879999999999999,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.60899999999999999,"15min":0.47499999999999998},"max":{"1min":1.514,"5min":2.593,"15min":32.174999999999997},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.6020000000000001,"15min":1.6160000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.442},"max":{"1min":2.157,"5min":15.907,"15min":17.225000000000001},"last":1.0740000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.528,"15min":1.754},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.66600000000000004},"max":{"1min":2.3290000000000002,"5min":2.5590000000000002,"15min":24.039999999999999},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.49,"15min":1.7789999999999999},"min":{"1min":0.99199999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.758,"5min":2.5449999999999999,"15min":32.216999999999999},"last":1.3380000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.708,"15min":2.0800000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.69099999999999995},"max":{"1min":1.8300000000000001,"5min":7.0979999999999999,"15min":34.131},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.641,"15min":1.984},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.61799999999999999},"max":{"1min":2.266,"5min":3.3759999999999999,"15min":44.694000000000003},"last":1.2889999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":655360,"data_stored":639563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":639521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1736458240,"data_stored":1736442609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":931135488,"data_stored":931135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":851443712,"data_stored":851443712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:18:00.777 INFO:tasks.ceph:pgid 17.3f last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.777 INFO:tasks.ceph:pgid 17.3e last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.777 INFO:tasks.ceph:pgid 17.3d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.777 INFO:tasks.ceph:pgid 17.3c last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.778 INFO:tasks.ceph:pgid 17.3b last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.778 INFO:tasks.ceph:pgid 17.3a last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.778 INFO:tasks.ceph:pgid 17.39 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.778 INFO:tasks.ceph:pgid 17.38 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.778 INFO:tasks.ceph:pgid 17.37 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.778 INFO:tasks.ceph:pgid 17.35 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.779 INFO:tasks.ceph:pgid 17.34 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.779 INFO:tasks.ceph:pgid 17.33 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.779 INFO:tasks.ceph:pgid 17.32 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.779 INFO:tasks.ceph:pgid 17.31 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.779 INFO:tasks.ceph:pgid 17.30 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.780 INFO:tasks.ceph:pgid 17.2f last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.780 INFO:tasks.ceph:pgid 17.2e last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.780 INFO:tasks.ceph:pgid 17.2d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.780 INFO:tasks.ceph:pgid 17.2c last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.781 INFO:tasks.ceph:pgid 17.2b last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.781 INFO:tasks.ceph:pgid 17.2a last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.781 INFO:tasks.ceph:pgid 17.29 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.781 INFO:tasks.ceph:pgid 17.28 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.781 INFO:tasks.ceph:pgid 17.27 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.782 INFO:tasks.ceph:pgid 17.26 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.782 INFO:tasks.ceph:pgid 17.25 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.782 INFO:tasks.ceph:pgid 17.24 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.782 INFO:tasks.ceph:pgid 17.23 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.782 INFO:tasks.ceph:pgid 17.22 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.783 INFO:tasks.ceph:pgid 17.21 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.783 INFO:tasks.ceph:pgid 17.20 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.783 INFO:tasks.ceph:pgid 17.1e last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.783 INFO:tasks.ceph:pgid 16.1f last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.783 INFO:tasks.ceph:pgid 17.1f last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.783 INFO:tasks.ceph:pgid 16.1e last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.784 INFO:tasks.ceph:pgid 17.1c last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.784 INFO:tasks.ceph:pgid 16.1d last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.784 INFO:tasks.ceph:pgid 17.1d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.784 INFO:tasks.ceph:pgid 16.1c last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.784 INFO:tasks.ceph:pgid 17.1a last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.785 INFO:tasks.ceph:pgid 16.1b last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.785 INFO:tasks.ceph:pgid 17.1b last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.785 INFO:tasks.ceph:pgid 16.1a last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.785 INFO:tasks.ceph:pgid 17.18 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.785 INFO:tasks.ceph:pgid 16.19 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.785 INFO:tasks.ceph:pgid 17.19 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.786 INFO:tasks.ceph:pgid 16.18 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.786 INFO:tasks.ceph:pgid 17.16 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.786 INFO:tasks.ceph:pgid 16.17 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.786 INFO:tasks.ceph:pgid 17.17 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.786 INFO:tasks.ceph:pgid 16.16 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.787 INFO:tasks.ceph:pgid 17.14 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.787 INFO:tasks.ceph:pgid 16.15 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.787 INFO:tasks.ceph:pgid 17.15 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.787 INFO:tasks.ceph:pgid 16.14 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.787 INFO:tasks.ceph:pgid 17.12 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.787 INFO:tasks.ceph:pgid 16.13 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.788 INFO:tasks.ceph:pgid 17.13 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.788 INFO:tasks.ceph:pgid 16.12 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.788 INFO:tasks.ceph:pgid 17.11 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.788 INFO:tasks.ceph:pgid 17.e last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.788 INFO:tasks.ceph:pgid 16.f last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.789 INFO:tasks.ceph:pgid 17.f last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.789 INFO:tasks.ceph:pgid 16.e last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.789 INFO:tasks.ceph:pgid 17.c last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.789 INFO:tasks.ceph:pgid 16.d last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.789 INFO:tasks.ceph:pgid 17.d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.790 INFO:tasks.ceph:pgid 16.c last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.790 INFO:tasks.ceph:pgid 17.a last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.790 INFO:tasks.ceph:pgid 16.b last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.790 INFO:tasks.ceph:pgid 17.b last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.790 INFO:tasks.ceph:pgid 16.a last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.791 INFO:tasks.ceph:pgid 17.8 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.791 INFO:tasks.ceph:pgid 16.9 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.791 INFO:tasks.ceph:pgid 17.9 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.791 INFO:tasks.ceph:pgid 16.8 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.791 INFO:tasks.ceph:pgid 17.6 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.792 INFO:tasks.ceph:pgid 16.7 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.792 INFO:tasks.ceph:pgid 17.7 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.792 INFO:tasks.ceph:pgid 16.6 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.792 INFO:tasks.ceph:pgid 17.4 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.792 INFO:tasks.ceph:pgid 16.5 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.793 INFO:tasks.ceph:pgid 17.5 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.793 INFO:tasks.ceph:pgid 16.4 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.793 INFO:tasks.ceph:pgid 17.2 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.793 INFO:tasks.ceph:pgid 16.3 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.793 INFO:tasks.ceph:pgid 17.3 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.794 INFO:tasks.ceph:pgid 16.2 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.794 INFO:tasks.ceph:pgid 17.0 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.794 INFO:tasks.ceph:pgid 16.1 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.794 INFO:tasks.ceph:pgid 17.1 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.794 INFO:tasks.ceph:pgid 16.0 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.794 INFO:tasks.ceph:pgid 17.10 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.795 INFO:tasks.ceph:pgid 16.11 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.795 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-01-15T17:54:58.463479+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=17, tm_min=54, tm_sec=58, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:00.795 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-15T18:18:03.428 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:18:03.437 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:18:20.797 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:18:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.047+0000 7f5030a00640 1 Processor -- start 2024-01-15T18:18:21.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.047+0000 7f5030a00640 1 -- start start 2024-01-15T18:18:21.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.049+0000 7f5030a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50280cf9a0 0x7f502815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.052 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.049+0000 7f5030a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f502815e1e0 0x7f50280702c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.049+0000 7f5030a00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50280b1ee0 con 0x7f5028143ae0 2024-01-15T18:18:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.049+0000 7f5030a00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50280b2780 con 0x7f50280cf9a0 2024-01-15T18:18:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.049+0000 7f5030a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502815e990 con 0x7f502815e1e0 2024-01-15T18:18:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.049+0000 7f502d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50280cf9a0 0x7f502815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.050+0000 7f502d600640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f50280cf9a0 0x7f502815dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52530/0 (socket says 172.21.15.26:52530) 2024-01-15T18:18:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.050+0000 7f502d600640 1 -- 172.21.15.26:0/283978752 learned_addr learned my addr 172.21.15.26:0/283978752 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:18:21.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.050+0000 7f502ea00640 1 --2- 172.21.15.26:0/283978752 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f502815e1e0 0x7f50280702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.050+0000 7f502d600640 1 -- 172.21.15.26:0/283978752 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f502815e1e0 msgr2=0x7f50280702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.050+0000 7f502d600640 1 --2- 172.21.15.26:0/283978752 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f502815e1e0 0x7f50280702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.050+0000 7f502d600640 1 -- 172.21.15.26:0/283978752 >> v1:172.21.15.124:6789/0 conn(0x7f5028143ae0 legacy=0x7f50280cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.050+0000 7f502d600640 1 -- 172.21.15.26:0/283978752 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50280b2130 con 0x7f50280cf9a0 2024-01-15T18:18:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.052+0000 7f502d600640 1 --2- 172.21.15.26:0/283978752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50280cf9a0 0x7f502815dca0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f5010009870 tx=0x7f501002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a5c8a5999c087d server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.052+0000 7f502cc00640 1 -- 172.21.15.26:0/283978752 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010030840 con 0x7f50280cf9a0 2024-01-15T18:18:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.052+0000 7f502cc00640 1 -- 172.21.15.26:0/283978752 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50100309a0 con 0x7f50280cf9a0 2024-01-15T18:18:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.052+0000 7f502cc00640 1 -- 172.21.15.26:0/283978752 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010030d30 con 0x7f50280cf9a0 2024-01-15T18:18:21.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.053+0000 7f5030a00640 1 -- 172.21.15.26:0/283978752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50280cf9a0 msgr2=0x7f502815dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.053+0000 7f5030a00640 1 --2- 172.21.15.26:0/283978752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50280cf9a0 0x7f502815dca0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f5010009870 tx=0x7f501002b980 comp rx=0 tx=0).stop 2024-01-15T18:18:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.053+0000 7f5030a00640 1 -- 172.21.15.26:0/283978752 shutdown_connections 2024-01-15T18:18:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.053+0000 7f5030a00640 1 --2- 172.21.15.26:0/283978752 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f502815e1e0 0x7f50280702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.053+0000 7f5030a00640 1 --2- 172.21.15.26:0/283978752 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f50280cf9a0 0x7f502815dca0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.053+0000 7f5030a00640 1 -- 172.21.15.26:0/283978752 >> 172.21.15.26:0/283978752 conn(0x7f502806a800 msgr2=0x7f502806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:21.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.053+0000 7f5030a00640 1 -- 172.21.15.26:0/283978752 shutdown_connections 2024-01-15T18:18:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.053+0000 7f5030a00640 1 -- 172.21.15.26:0/283978752 wait complete. 2024-01-15T18:18:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.055+0000 7f5030a00640 1 Processor -- start 2024-01-15T18:18:21.059 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.055+0000 7f5030a00640 1 -- start start 2024-01-15T18:18:21.061 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f5030a00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50280cf9a0 0x7f502813ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f5030a00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5028143ae0 0x7f502813ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f5030a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502815e1e0 0x7f5028159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f5030a00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502813e490 con 0x7f502815e1e0 2024-01-15T18:18:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f5030a00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502813de90 con 0x7f5028143ae0 2024-01-15T18:18:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f5030a00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502813d370 con 0x7f50280cf9a0 2024-01-15T18:18:21.062 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f502ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502815e1e0 0x7f5028159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f502ea00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502815e1e0 0x7f5028159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:52546/0 (socket says 172.21.15.26:52546) 2024-01-15T18:18:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f502ea00640 1 -- 172.21.15.26:0/560109780 learned_addr learned my addr 172.21.15.26:0/560109780 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:18:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f502e000640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50280cf9a0 0x7f502813ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.057+0000 7f502d600640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5028143ae0 0x7f502813ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.058+0000 7f502ea00640 1 -- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50280cf9a0 msgr2=0x7f502813ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.058+0000 7f502ea00640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50280cf9a0 0x7f502813ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.063 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.058+0000 7f502ea00640 1 -- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5028143ae0 msgr2=0x7f502813ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.058+0000 7f502ea00640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5028143ae0 0x7f502813ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.058+0000 7f502ea00640 1 -- 172.21.15.26:0/560109780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50100094d0 con 0x7f502815e1e0 2024-01-15T18:18:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.058+0000 7f502d600640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5028143ae0 0x7f502813ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-15T18:18:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.058+0000 7f502ea00640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502815e1e0 0x7f5028159620 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f502000efc0 tx=0x7f502000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.059+0000 7f5026000640 1 -- 172.21.15.26:0/560109780 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020009ba0 con 0x7f502815e1e0 2024-01-15T18:18:21.064 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.059+0000 7f5026000640 1 -- 172.21.15.26:0/560109780 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502000f040 con 0x7f502815e1e0 2024-01-15T18:18:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.059+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502813d5f0 con 0x7f502815e1e0 2024-01-15T18:18:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.059+0000 7f5026000640 1 -- 172.21.15.26:0/560109780 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50200146c0 con 0x7f502815e1e0 2024-01-15T18:18:21.065 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.059+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502813dbb0 con 0x7f502815e1e0 2024-01-15T18:18:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.060+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502813e1f0 con 0x7f502815e1e0 2024-01-15T18:18:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.061+0000 7f5026000640 1 -- 172.21.15.26:0/560109780 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f5020014820 con 0x7f502815e1e0 2024-01-15T18:18:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.062+0000 7f5026000640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f500006c550 0x7f500006ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:21.066 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.062+0000 7f5026000640 1 -- 172.21.15.26:0/560109780 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f50200896f0 con 0x7f502815e1e0 2024-01-15T18:18:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.063+0000 7f502e000640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f500006c550 0x7f500006ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:21.067 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.064+0000 7f502e000640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f500006c550 0x7f500006ea10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5018000f60 tx=0x7f5018009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:21.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.067+0000 7f5026000640 1 -- 172.21.15.26:0/560109780 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f502813e1f0 con 0x7f502815e1e0 2024-01-15T18:18:21.268 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.265+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f502815e670 con 0x7f500006c550 2024-01-15T18:18:21.299 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.296+0000 7f5026000640 1 -- 172.21.15.26:0/560109780 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288821 (secure 0 0 0) 0x7f5018001c70 con 0x7f500006c550 2024-01-15T18:18:21.303 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:18:21.304 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:18:21.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f500006c550 msgr2=0x7f500006ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f500006c550 0x7f500006ea10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5018000f60 tx=0x7f5018009040 comp rx=0 tx=0).stop 2024-01-15T18:18:21.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502815e1e0 msgr2=0x7f5028159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:21.306 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502815e1e0 0x7f5028159620 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f502000efc0 tx=0x7f502000c490 comp rx=0 tx=0).stop 2024-01-15T18:18:21.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 shutdown_connections 2024-01-15T18:18:21.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f500006c550 0x7f500006ea10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f502815e1e0 0x7f5028159620 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f5028143ae0 0x7f502813ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 --2- 172.21.15.26:0/560109780 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f50280cf9a0 0x7f502813ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:21.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 >> 172.21.15.26:0/560109780 conn(0x7f502806a800 msgr2=0x7f502815cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:21.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 shutdown_connections 2024-01-15T18:18:21.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:21.303+0000 7f5030a00640 1 -- 172.21.15.26:0/560109780 wait complete. 2024-01-15T18:18:21.324 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":812,"stamp":"2024-01-15T18:18:20.463122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369942071,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1744,"num_write_kb":5246066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5503,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3772,"ondisk_log_size":3772,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11070544,"kb_used_data":10499600,"kb_used_omap":12,"kb_used_meta":570867,"kb_avail":743904176,"statfs":{"total":773094113280,"available":761757876224,"internally_reserved":0,"allocated":10751590400,"data_stored":10744962942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":584568368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005403"},"pg_stats":[{"pgid":"17.3f","version":"117'78","reported_seq":119,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.182705+0000","last_change":"2024-01-15T18:18:15.182793+0000","last_active":"2024-01-15T18:18:15.182705+0000","last_peered":"2024-01-15T18:18:15.182705+0000","last_clean":"2024-01-15T18:18:15.182705+0000","last_became_active":"2024-01-15T18:12:53.334688+0000","last_became_peered":"2024-01-15T18:12:53.334688+0000","last_unstale":"2024-01-15T18:18:15.182705+0000","last_undegraded":"2024-01-15T18:18:15.182705+0000","last_fullsized":"2024-01-15T18:18:15.182705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'78","last_scrub_stamp":"2024-01-15T18:18:15.182527+0000","last_deep_scrub":"117'78","last_deep_scrub_stamp":"2024-01-15T18:18:15.182527+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.182527+0000","objects_scrubbed":25,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:21:28.406331+0000","scrub_duration":0.27837603599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3e","version":"117'64","reported_seq":96,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:16.135841+0000","last_change":"2024-01-15T18:18:16.135979+0000","last_active":"2024-01-15T18:18:16.135841+0000","last_peered":"2024-01-15T18:18:16.135841+0000","last_clean":"2024-01-15T18:18:16.135841+0000","last_became_active":"2024-01-15T18:12:11.053279+0000","last_became_peered":"2024-01-15T18:12:11.053279+0000","last_unstale":"2024-01-15T18:18:16.135841+0000","last_undegraded":"2024-01-15T18:18:16.135841+0000","last_fullsized":"2024-01-15T18:18:16.135841+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'64","last_scrub_stamp":"2024-01-15T18:18:16.135588+0000","last_deep_scrub":"117'64","last_deep_scrub_stamp":"2024-01-15T18:18:16.135588+0000","last_clean_scrub_stamp":"2024-01-15T18:18:16.135588+0000","objects_scrubbed":19,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:16:49.227522+0000","scrub_duration":0.232285464,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"117'44","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.351022+0000","last_change":"2024-01-15T18:12:11.058625+0000","last_active":"2024-01-15T18:17:39.351022+0000","last_peered":"2024-01-15T18:17:39.351022+0000","last_clean":"2024-01-15T18:17:39.351022+0000","last_became_active":"2024-01-15T18:12:11.057543+0000","last_became_peered":"2024-01-15T18:12:11.057543+0000","last_unstale":"2024-01-15T18:17:39.351022+0000","last_undegraded":"2024-01-15T18:17:39.351022+0000","last_fullsized":"2024-01-15T18:17:39.351022+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"117'45","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:55.114267+0000","last_change":"2024-01-15T18:12:11.047313+0000","last_active":"2024-01-15T18:17:55.114267+0000","last_peered":"2024-01-15T18:17:55.114267+0000","last_clean":"2024-01-15T18:17:55.114267+0000","last_became_active":"2024-01-15T18:12:11.047008+0000","last_became_peered":"2024-01-15T18:12:11.047008+0000","last_unstale":"2024-01-15T18:17:55.114267+0000","last_undegraded":"2024-01-15T18:17:55.114267+0000","last_fullsized":"2024-01-15T18:17:55.114267+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"117'37","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.943087+0000","last_change":"2024-01-15T18:18:09.943156+0000","last_active":"2024-01-15T18:18:09.943087+0000","last_peered":"2024-01-15T18:18:09.943087+0000","last_clean":"2024-01-15T18:18:09.943087+0000","last_became_active":"2024-01-15T18:12:11.034909+0000","last_became_peered":"2024-01-15T18:12:11.034909+0000","last_unstale":"2024-01-15T18:18:09.943087+0000","last_undegraded":"2024-01-15T18:18:09.943087+0000","last_fullsized":"2024-01-15T18:18:09.943087+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'37","last_scrub_stamp":"2024-01-15T18:18:09.942945+0000","last_deep_scrub":"117'37","last_deep_scrub_stamp":"2024-01-15T18:18:09.942945+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.942945+0000","objects_scrubbed":10,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:42:24.509082+0000","scrub_duration":0.101423941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"117'69","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.095936+0000","last_change":"2024-01-15T18:18:15.096025+0000","last_active":"2024-01-15T18:18:15.095936+0000","last_peered":"2024-01-15T18:18:15.095936+0000","last_clean":"2024-01-15T18:18:15.095936+0000","last_became_active":"2024-01-15T18:12:11.054803+0000","last_became_peered":"2024-01-15T18:12:11.054803+0000","last_unstale":"2024-01-15T18:18:15.095936+0000","last_undegraded":"2024-01-15T18:18:15.095936+0000","last_fullsized":"2024-01-15T18:18:15.095936+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'69","last_scrub_stamp":"2024-01-15T18:18:15.095819+0000","last_deep_scrub":"117'69","last_deep_scrub_stamp":"2024-01-15T18:18:15.095819+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.095819+0000","objects_scrubbed":21,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:46.439597+0000","scrub_duration":0.18604361999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"117'60","reported_seq":88,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.442469+0000","last_change":"2024-01-15T18:18:09.442535+0000","last_active":"2024-01-15T18:18:09.442469+0000","last_peered":"2024-01-15T18:18:09.442469+0000","last_clean":"2024-01-15T18:18:09.442469+0000","last_became_active":"2024-01-15T18:12:11.044607+0000","last_became_peered":"2024-01-15T18:12:11.044607+0000","last_unstale":"2024-01-15T18:18:09.442469+0000","last_undegraded":"2024-01-15T18:18:09.442469+0000","last_fullsized":"2024-01-15T18:18:09.442469+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'60","last_scrub_stamp":"2024-01-15T18:18:09.442344+0000","last_deep_scrub":"117'60","last_deep_scrub_stamp":"2024-01-15T18:18:09.442344+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.442344+0000","objects_scrubbed":17,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:43:41.911933+0000","scrub_duration":0.21794185399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"117'44","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.995547+0000","last_change":"2024-01-15T18:12:11.051056+0000","last_active":"2024-01-15T18:17:38.995547+0000","last_peered":"2024-01-15T18:17:38.995547+0000","last_clean":"2024-01-15T18:17:38.995547+0000","last_became_active":"2024-01-15T18:12:11.050697+0000","last_became_peered":"2024-01-15T18:12:11.050697+0000","last_unstale":"2024-01-15T18:17:38.995547+0000","last_undegraded":"2024-01-15T18:17:38.995547+0000","last_fullsized":"2024-01-15T18:17:38.995547+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"117'47","reported_seq":71,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.479691+0000","last_change":"2024-01-15T18:12:11.048227+0000","last_active":"2024-01-15T18:17:39.479691+0000","last_peered":"2024-01-15T18:17:39.479691+0000","last_clean":"2024-01-15T18:17:39.479691+0000","last_became_active":"2024-01-15T18:12:11.047693+0000","last_became_peered":"2024-01-15T18:12:11.047693+0000","last_unstale":"2024-01-15T18:17:39.479691+0000","last_undegraded":"2024-01-15T18:17:39.479691+0000","last_fullsized":"2024-01-15T18:17:39.479691+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"117'58","reported_seq":92,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:55.436550+0000","last_change":"2024-01-15T18:17:55.436587+0000","last_active":"2024-01-15T18:17:55.436550+0000","last_peered":"2024-01-15T18:17:55.436550+0000","last_clean":"2024-01-15T18:17:55.436550+0000","last_became_active":"2024-01-15T18:12:11.052429+0000","last_became_peered":"2024-01-15T18:12:11.052429+0000","last_unstale":"2024-01-15T18:17:55.436550+0000","last_undegraded":"2024-01-15T18:17:55.436550+0000","last_fullsized":"2024-01-15T18:17:55.436550+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'58","last_scrub_stamp":"2024-01-15T18:17:55.436465+0000","last_deep_scrub":"117'58","last_deep_scrub_stamp":"2024-01-15T18:17:55.436465+0000","last_clean_scrub_stamp":"2024-01-15T18:17:55.436465+0000","objects_scrubbed":17,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:11:39.403468+0000","scrub_duration":0.096421670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"117'69","reported_seq":106,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.262993+0000","last_change":"2024-01-15T18:18:07.263053+0000","last_active":"2024-01-15T18:18:07.262993+0000","last_peered":"2024-01-15T18:18:07.262993+0000","last_clean":"2024-01-15T18:18:07.262993+0000","last_became_active":"2024-01-15T18:12:11.042720+0000","last_became_peered":"2024-01-15T18:12:11.042720+0000","last_unstale":"2024-01-15T18:18:07.262993+0000","last_undegraded":"2024-01-15T18:18:07.262993+0000","last_fullsized":"2024-01-15T18:18:07.262993+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'69","last_scrub_stamp":"2024-01-15T18:18:07.262832+0000","last_deep_scrub":"117'69","last_deep_scrub_stamp":"2024-01-15T18:18:07.262832+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.262832+0000","objects_scrubbed":28,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:08:52.313757+0000","scrub_duration":0.35287618199999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"117'73","reported_seq":114,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:08.692443+0000","last_change":"2024-01-15T18:18:08.692487+0000","last_active":"2024-01-15T18:18:08.692443+0000","last_peered":"2024-01-15T18:18:08.692443+0000","last_clean":"2024-01-15T18:18:08.692443+0000","last_became_active":"2024-01-15T18:12:11.044155+0000","last_became_peered":"2024-01-15T18:12:11.044155+0000","last_unstale":"2024-01-15T18:18:08.692443+0000","last_undegraded":"2024-01-15T18:18:08.692443+0000","last_fullsized":"2024-01-15T18:18:08.692443+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'73","last_scrub_stamp":"2024-01-15T18:18:08.692341+0000","last_deep_scrub":"117'73","last_deep_scrub_stamp":"2024-01-15T18:18:08.692341+0000","last_clean_scrub_stamp":"2024-01-15T18:18:08.692341+0000","objects_scrubbed":28,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:17:19.915429+0000","scrub_duration":0.266379529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"117'38","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.020687+0000","last_change":"2024-01-15T18:12:11.052950+0000","last_active":"2024-01-15T18:17:39.020687+0000","last_peered":"2024-01-15T18:17:39.020687+0000","last_clean":"2024-01-15T18:17:39.020687+0000","last_became_active":"2024-01-15T18:12:11.051875+0000","last_became_peered":"2024-01-15T18:12:11.051875+0000","last_unstale":"2024-01-15T18:17:39.020687+0000","last_undegraded":"2024-01-15T18:17:39.020687+0000","last_fullsized":"2024-01-15T18:17:39.020687+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"117'74","reported_seq":106,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:08.869762+0000","last_change":"2024-01-15T18:18:08.869841+0000","last_active":"2024-01-15T18:18:08.869762+0000","last_peered":"2024-01-15T18:18:08.869762+0000","last_clean":"2024-01-15T18:18:08.869762+0000","last_became_active":"2024-01-15T18:12:11.055813+0000","last_became_peered":"2024-01-15T18:12:11.055813+0000","last_unstale":"2024-01-15T18:18:08.869762+0000","last_undegraded":"2024-01-15T18:18:08.869762+0000","last_fullsized":"2024-01-15T18:18:08.869762+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'74","last_scrub_stamp":"2024-01-15T18:18:08.869620+0000","last_deep_scrub":"117'74","last_deep_scrub_stamp":"2024-01-15T18:18:08.869620+0000","last_clean_scrub_stamp":"2024-01-15T18:18:08.869620+0000","objects_scrubbed":27,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:52.461222+0000","scrub_duration":0.34224997800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"117'46","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:16.051211+0000","last_change":"2024-01-15T18:18:16.051305+0000","last_active":"2024-01-15T18:18:16.051211+0000","last_peered":"2024-01-15T18:18:16.051211+0000","last_clean":"2024-01-15T18:18:16.051211+0000","last_became_active":"2024-01-15T18:12:11.055217+0000","last_became_peered":"2024-01-15T18:12:11.055217+0000","last_unstale":"2024-01-15T18:18:16.051211+0000","last_undegraded":"2024-01-15T18:18:16.051211+0000","last_fullsized":"2024-01-15T18:18:16.051211+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'46","last_scrub_stamp":"2024-01-15T18:18:16.051013+0000","last_deep_scrub":"117'46","last_deep_scrub_stamp":"2024-01-15T18:18:16.051013+0000","last_clean_scrub_stamp":"2024-01-15T18:18:16.051013+0000","objects_scrubbed":12,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:03.749881+0000","scrub_duration":0.141849805,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"117'42","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.147921+0000","last_change":"2024-01-15T18:12:53.618454+0000","last_active":"2024-01-15T18:17:39.147921+0000","last_peered":"2024-01-15T18:17:39.147921+0000","last_clean":"2024-01-15T18:17:39.147921+0000","last_became_active":"2024-01-15T18:12:53.338860+0000","last_became_peered":"2024-01-15T18:12:53.338860+0000","last_unstale":"2024-01-15T18:17:39.147921+0000","last_undegraded":"2024-01-15T18:17:39.147921+0000","last_fullsized":"2024-01-15T18:17:39.147921+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2f","version":"117'44","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.394701+0000","last_change":"2024-01-15T18:12:11.043060+0000","last_active":"2024-01-15T18:17:39.394701+0000","last_peered":"2024-01-15T18:17:39.394701+0000","last_clean":"2024-01-15T18:17:39.394701+0000","last_became_active":"2024-01-15T18:12:11.042031+0000","last_became_peered":"2024-01-15T18:12:11.042031+0000","last_unstale":"2024-01-15T18:17:39.394701+0000","last_undegraded":"2024-01-15T18:17:39.394701+0000","last_fullsized":"2024-01-15T18:17:39.394701+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"117'67","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:06.389526+0000","last_change":"2024-01-15T18:18:06.389619+0000","last_active":"2024-01-15T18:18:06.389526+0000","last_peered":"2024-01-15T18:18:06.389526+0000","last_clean":"2024-01-15T18:18:06.389526+0000","last_became_active":"2024-01-15T18:12:11.042814+0000","last_became_peered":"2024-01-15T18:12:11.042814+0000","last_unstale":"2024-01-15T18:18:06.389526+0000","last_undegraded":"2024-01-15T18:18:06.389526+0000","last_fullsized":"2024-01-15T18:18:06.389526+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'67","last_scrub_stamp":"2024-01-15T18:18:06.389332+0000","last_deep_scrub":"117'67","last_deep_scrub_stamp":"2024-01-15T18:18:06.389332+0000","last_clean_scrub_stamp":"2024-01-15T18:18:06.389332+0000","objects_scrubbed":21,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:57:01.960290+0000","scrub_duration":0.25979249500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"117'44","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.193054+0000","last_change":"2024-01-15T18:12:11.055119+0000","last_active":"2024-01-15T18:17:39.193054+0000","last_peered":"2024-01-15T18:17:39.193054+0000","last_clean":"2024-01-15T18:17:39.193054+0000","last_became_active":"2024-01-15T18:12:11.054790+0000","last_became_peered":"2024-01-15T18:12:11.054790+0000","last_unstale":"2024-01-15T18:17:39.193054+0000","last_undegraded":"2024-01-15T18:17:39.193054+0000","last_fullsized":"2024-01-15T18:17:39.193054+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"117'63","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:11.200840+0000","last_change":"2024-01-15T18:18:11.200952+0000","last_active":"2024-01-15T18:18:11.200840+0000","last_peered":"2024-01-15T18:18:11.200840+0000","last_clean":"2024-01-15T18:18:11.200840+0000","last_became_active":"2024-01-15T18:12:11.052289+0000","last_became_peered":"2024-01-15T18:12:11.052289+0000","last_unstale":"2024-01-15T18:18:11.200840+0000","last_undegraded":"2024-01-15T18:18:11.200840+0000","last_fullsized":"2024-01-15T18:18:11.200840+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:11.200610+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:11.200610+0000","last_clean_scrub_stamp":"2024-01-15T18:18:11.200610+0000","objects_scrubbed":17,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:42.709925+0000","scrub_duration":0.222891758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"117'56","reported_seq":85,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.484520+0000","last_change":"2024-01-15T18:12:11.036748+0000","last_active":"2024-01-15T18:17:39.484520+0000","last_peered":"2024-01-15T18:17:39.484520+0000","last_clean":"2024-01-15T18:17:39.484520+0000","last_became_active":"2024-01-15T18:12:11.036003+0000","last_became_peered":"2024-01-15T18:12:11.036003+0000","last_unstale":"2024-01-15T18:17:39.484520+0000","last_undegraded":"2024-01-15T18:17:39.484520+0000","last_fullsized":"2024-01-15T18:17:39.484520+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"117'47","reported_seq":71,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.544145+0000","last_change":"2024-01-15T18:12:11.050445+0000","last_active":"2024-01-15T18:17:39.544145+0000","last_peered":"2024-01-15T18:17:39.544145+0000","last_clean":"2024-01-15T18:17:39.544145+0000","last_became_active":"2024-01-15T18:12:11.044093+0000","last_became_peered":"2024-01-15T18:12:11.044093+0000","last_unstale":"2024-01-15T18:17:39.544145+0000","last_undegraded":"2024-01-15T18:17:39.544145+0000","last_fullsized":"2024-01-15T18:17:39.544145+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"117'54","reported_seq":87,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:14.102196+0000","last_change":"2024-01-15T18:18:14.102285+0000","last_active":"2024-01-15T18:18:14.102196+0000","last_peered":"2024-01-15T18:18:14.102196+0000","last_clean":"2024-01-15T18:18:14.102196+0000","last_became_active":"2024-01-15T18:12:11.052428+0000","last_became_peered":"2024-01-15T18:12:11.052428+0000","last_unstale":"2024-01-15T18:18:14.102196+0000","last_undegraded":"2024-01-15T18:18:14.102196+0000","last_fullsized":"2024-01-15T18:18:14.102196+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'54","last_scrub_stamp":"2024-01-15T18:18:14.102006+0000","last_deep_scrub":"117'54","last_deep_scrub_stamp":"2024-01-15T18:18:14.102006+0000","last_clean_scrub_stamp":"2024-01-15T18:18:14.102006+0000","objects_scrubbed":15,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:05:29.630951+0000","scrub_duration":0.188245571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"117'43","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.484082+0000","last_change":"2024-01-15T18:12:11.051820+0000","last_active":"2024-01-15T18:17:39.484082+0000","last_peered":"2024-01-15T18:17:39.484082+0000","last_clean":"2024-01-15T18:17:39.484082+0000","last_became_active":"2024-01-15T18:12:11.051488+0000","last_became_peered":"2024-01-15T18:12:11.051488+0000","last_unstale":"2024-01-15T18:17:39.484082+0000","last_undegraded":"2024-01-15T18:17:39.484082+0000","last_fullsized":"2024-01-15T18:17:39.484082+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"117'56","reported_seq":90,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.157818+0000","last_change":"2024-01-15T18:18:17.157892+0000","last_active":"2024-01-15T18:18:17.157818+0000","last_peered":"2024-01-15T18:18:17.157818+0000","last_clean":"2024-01-15T18:18:17.157818+0000","last_became_active":"2024-01-15T18:12:11.041266+0000","last_became_peered":"2024-01-15T18:12:11.041266+0000","last_unstale":"2024-01-15T18:18:17.157818+0000","last_undegraded":"2024-01-15T18:18:17.157818+0000","last_fullsized":"2024-01-15T18:18:17.157818+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'56","last_scrub_stamp":"2024-01-15T18:18:17.157660+0000","last_deep_scrub":"117'56","last_deep_scrub_stamp":"2024-01-15T18:18:17.157660+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.157660+0000","objects_scrubbed":18,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:31:02.204776+0000","scrub_duration":0.223683413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"117'53","reported_seq":77,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.486490+0000","last_change":"2024-01-15T18:12:11.048567+0000","last_active":"2024-01-15T18:17:39.486490+0000","last_peered":"2024-01-15T18:17:39.486490+0000","last_clean":"2024-01-15T18:17:39.486490+0000","last_became_active":"2024-01-15T18:12:11.048234+0000","last_became_peered":"2024-01-15T18:12:11.048234+0000","last_unstale":"2024-01-15T18:17:39.486490+0000","last_undegraded":"2024-01-15T18:17:39.486490+0000","last_fullsized":"2024-01-15T18:17:39.486490+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"117'37","reported_seq":60,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.870834+0000","last_change":"2024-01-15T18:12:11.062039+0000","last_active":"2024-01-15T18:17:38.870834+0000","last_peered":"2024-01-15T18:17:38.870834+0000","last_clean":"2024-01-15T18:17:38.870834+0000","last_became_active":"2024-01-15T18:12:11.061724+0000","last_became_peered":"2024-01-15T18:12:11.061724+0000","last_unstale":"2024-01-15T18:17:38.870834+0000","last_undegraded":"2024-01-15T18:17:38.870834+0000","last_fullsized":"2024-01-15T18:17:38.870834+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"117'38","reported_seq":57,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.487271+0000","last_change":"2024-01-15T18:12:11.044940+0000","last_active":"2024-01-15T18:17:39.487271+0000","last_peered":"2024-01-15T18:17:39.487271+0000","last_clean":"2024-01-15T18:17:39.487271+0000","last_became_active":"2024-01-15T18:12:11.043801+0000","last_became_peered":"2024-01-15T18:12:11.043801+0000","last_unstale":"2024-01-15T18:17:39.487271+0000","last_undegraded":"2024-01-15T18:17:39.487271+0000","last_fullsized":"2024-01-15T18:17:39.487271+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"117'46","reported_seq":71,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.445255+0000","last_change":"2024-01-15T18:12:11.048119+0000","last_active":"2024-01-15T18:17:39.445255+0000","last_peered":"2024-01-15T18:17:39.445255+0000","last_clean":"2024-01-15T18:17:39.445255+0000","last_became_active":"2024-01-15T18:12:11.047739+0000","last_became_peered":"2024-01-15T18:12:11.047739+0000","last_unstale":"2024-01-15T18:17:39.445255+0000","last_undegraded":"2024-01-15T18:17:39.445255+0000","last_fullsized":"2024-01-15T18:17:39.445255+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"117'57","reported_seq":90,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.431341+0000","last_change":"2024-01-15T18:18:10.431430+0000","last_active":"2024-01-15T18:18:10.431341+0000","last_peered":"2024-01-15T18:18:10.431341+0000","last_clean":"2024-01-15T18:18:10.431341+0000","last_became_active":"2024-01-15T18:12:11.039393+0000","last_became_peered":"2024-01-15T18:12:11.039393+0000","last_unstale":"2024-01-15T18:18:10.431341+0000","last_undegraded":"2024-01-15T18:18:10.431341+0000","last_fullsized":"2024-01-15T18:18:10.431341+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'57","last_scrub_stamp":"2024-01-15T18:18:10.431168+0000","last_deep_scrub":"117'57","last_deep_scrub_stamp":"2024-01-15T18:18:10.431168+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.431168+0000","objects_scrubbed":16,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:49:04.771755+0000","scrub_duration":0.21320547200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"117'62","reported_seq":90,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.405837+0000","last_change":"2024-01-15T18:12:11.059603+0000","last_active":"2024-01-15T18:17:39.405837+0000","last_peered":"2024-01-15T18:17:39.405837+0000","last_clean":"2024-01-15T18:17:39.405837+0000","last_became_active":"2024-01-15T18:12:11.059276+0000","last_became_peered":"2024-01-15T18:12:11.059276+0000","last_unstale":"2024-01-15T18:17:39.405837+0000","last_undegraded":"2024-01-15T18:17:39.405837+0000","last_fullsized":"2024-01-15T18:17:39.405837+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"117'76","reported_seq":107,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:13.141602+0000","last_change":"2024-01-15T18:18:13.141663+0000","last_active":"2024-01-15T18:18:13.141602+0000","last_peered":"2024-01-15T18:18:13.141602+0000","last_clean":"2024-01-15T18:18:13.141602+0000","last_became_active":"2024-01-15T18:12:11.052655+0000","last_became_peered":"2024-01-15T18:12:11.052655+0000","last_unstale":"2024-01-15T18:18:13.141602+0000","last_undegraded":"2024-01-15T18:18:13.141602+0000","last_fullsized":"2024-01-15T18:18:13.141602+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'76","last_scrub_stamp":"2024-01-15T18:18:13.141496+0000","last_deep_scrub":"117'76","last_deep_scrub_stamp":"2024-01-15T18:18:13.141496+0000","last_clean_scrub_stamp":"2024-01-15T18:18:13.141496+0000","objects_scrubbed":22,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:46.765713+0000","scrub_duration":0.21211069099999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"117'49","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.216437+0000","last_change":"2024-01-15T18:18:10.216437+0000","last_active":"2024-01-15T18:18:10.216437+0000","last_peered":"2024-01-15T18:18:10.216437+0000","last_clean":"2024-01-15T18:18:10.216437+0000","last_became_active":"2024-01-15T18:12:11.053389+0000","last_became_peered":"2024-01-15T18:12:11.053389+0000","last_unstale":"2024-01-15T18:18:10.216437+0000","last_undegraded":"2024-01-15T18:18:10.216437+0000","last_fullsized":"2024-01-15T18:18:10.216437+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'49","last_scrub_stamp":"2024-01-15T18:18:10.216244+0000","last_deep_scrub":"117'49","last_deep_scrub_stamp":"2024-01-15T18:18:10.216244+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.216244+0000","objects_scrubbed":15,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:33.117273+0000","scrub_duration":0.16396555099999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"117'6","reported_seq":31,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:03.244704+0000","last_change":"2024-01-15T18:18:03.244761+0000","last_active":"2024-01-15T18:18:03.244704+0000","last_peered":"2024-01-15T18:18:03.244704+0000","last_clean":"2024-01-15T18:18:03.244704+0000","last_became_active":"2024-01-15T18:12:10.020810+0000","last_became_peered":"2024-01-15T18:12:10.020810+0000","last_unstale":"2024-01-15T18:18:03.244704+0000","last_undegraded":"2024-01-15T18:18:03.244704+0000","last_fullsized":"2024-01-15T18:18:03.244704+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'6","last_scrub_stamp":"2024-01-15T18:18:03.244524+0000","last_deep_scrub":"117'6","last_deep_scrub_stamp":"2024-01-15T18:18:03.244524+0000","last_clean_scrub_stamp":"2024-01-15T18:18:03.244524+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:14:16.865481+0000","scrub_duration":0.0076147139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"117'66","reported_seq":98,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.420340+0000","last_change":"2024-01-15T18:18:07.420458+0000","last_active":"2024-01-15T18:18:07.420340+0000","last_peered":"2024-01-15T18:18:07.420340+0000","last_clean":"2024-01-15T18:18:07.420340+0000","last_became_active":"2024-01-15T18:12:11.042427+0000","last_became_peered":"2024-01-15T18:12:11.042427+0000","last_unstale":"2024-01-15T18:18:07.420340+0000","last_undegraded":"2024-01-15T18:18:07.420340+0000","last_fullsized":"2024-01-15T18:18:07.420340+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'66","last_scrub_stamp":"2024-01-15T18:18:07.420124+0000","last_deep_scrub":"117'66","last_deep_scrub_stamp":"2024-01-15T18:18:07.420124+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.420124+0000","objects_scrubbed":22,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:28.299453+0000","scrub_duration":0.26970153499999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:06.548413+0000","last_change":"2024-01-15T18:18:06.548413+0000","last_active":"2024-01-15T18:18:06.548413+0000","last_peered":"2024-01-15T18:18:06.548413+0000","last_clean":"2024-01-15T18:18:06.548413+0000","last_became_active":"2024-01-15T18:12:10.022654+0000","last_became_peered":"2024-01-15T18:12:10.022654+0000","last_unstale":"2024-01-15T18:18:06.548413+0000","last_undegraded":"2024-01-15T18:18:06.548413+0000","last_fullsized":"2024-01-15T18:18:06.548413+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'1","last_scrub_stamp":"2024-01-15T18:18:06.548172+0000","last_deep_scrub":"114'1","last_deep_scrub_stamp":"2024-01-15T18:18:06.548172+0000","last_clean_scrub_stamp":"2024-01-15T18:18:06.548172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:40.711970+0000","scrub_duration":0.003727925,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'62","reported_seq":97,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.077838+0000","last_change":"2024-01-15T18:18:17.077904+0000","last_active":"2024-01-15T18:18:17.077838+0000","last_peered":"2024-01-15T18:18:17.077838+0000","last_clean":"2024-01-15T18:18:17.077838+0000","last_became_active":"2024-01-15T18:12:11.052812+0000","last_became_peered":"2024-01-15T18:12:11.052812+0000","last_unstale":"2024-01-15T18:18:17.077838+0000","last_undegraded":"2024-01-15T18:18:17.077838+0000","last_fullsized":"2024-01-15T18:18:17.077838+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'62","last_scrub_stamp":"2024-01-15T18:18:17.077727+0000","last_deep_scrub":"117'62","last_deep_scrub_stamp":"2024-01-15T18:18:17.077727+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.077727+0000","objects_scrubbed":19,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:22.148520+0000","scrub_duration":0.19122593199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"117'4","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.759072+0000","last_change":"2024-01-15T18:17:58.759128+0000","last_active":"2024-01-15T18:17:58.759072+0000","last_peered":"2024-01-15T18:17:58.759072+0000","last_clean":"2024-01-15T18:17:58.759072+0000","last_became_active":"2024-01-15T18:12:10.018670+0000","last_became_peered":"2024-01-15T18:12:10.018670+0000","last_unstale":"2024-01-15T18:17:58.759072+0000","last_undegraded":"2024-01-15T18:17:58.759072+0000","last_fullsized":"2024-01-15T18:17:58.759072+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'4","last_scrub_stamp":"2024-01-15T18:17:58.758892+0000","last_deep_scrub":"117'4","last_deep_scrub_stamp":"2024-01-15T18:17:58.758892+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.758892+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:09:01.602528+0000","scrub_duration":0.0068205260000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"117'56","reported_seq":83,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.364043+0000","last_change":"2024-01-15T18:12:11.059355+0000","last_active":"2024-01-15T18:17:39.364043+0000","last_peered":"2024-01-15T18:17:39.364043+0000","last_clean":"2024-01-15T18:17:39.364043+0000","last_became_active":"2024-01-15T18:12:11.057592+0000","last_became_peered":"2024-01-15T18:12:11.057592+0000","last_unstale":"2024-01-15T18:17:39.364043+0000","last_undegraded":"2024-01-15T18:17:39.364043+0000","last_fullsized":"2024-01-15T18:17:39.364043+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"117'3","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:57.800153+0000","last_change":"2024-01-15T18:17:57.800193+0000","last_active":"2024-01-15T18:17:57.800153+0000","last_peered":"2024-01-15T18:17:57.800153+0000","last_clean":"2024-01-15T18:17:57.800153+0000","last_became_active":"2024-01-15T18:12:10.020143+0000","last_became_peered":"2024-01-15T18:12:10.020143+0000","last_unstale":"2024-01-15T18:17:57.800153+0000","last_undegraded":"2024-01-15T18:17:57.800153+0000","last_fullsized":"2024-01-15T18:17:57.800153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'3","last_scrub_stamp":"2024-01-15T18:17:57.800057+0000","last_deep_scrub":"117'3","last_deep_scrub_stamp":"2024-01-15T18:17:57.800057+0000","last_clean_scrub_stamp":"2024-01-15T18:17:57.800057+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:23:19.671055+0000","scrub_duration":0.0055516410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"117'39","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.465226+0000","last_change":"2024-01-15T18:12:11.057876+0000","last_active":"2024-01-15T18:17:39.465226+0000","last_peered":"2024-01-15T18:17:39.465226+0000","last_clean":"2024-01-15T18:17:39.465226+0000","last_became_active":"2024-01-15T18:12:11.057471+0000","last_became_peered":"2024-01-15T18:12:11.057471+0000","last_unstale":"2024-01-15T18:17:39.465226+0000","last_undegraded":"2024-01-15T18:17:39.465226+0000","last_fullsized":"2024-01-15T18:17:39.465226+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:05.257137+0000","last_change":"2024-01-15T18:18:05.257194+0000","last_active":"2024-01-15T18:18:05.257137+0000","last_peered":"2024-01-15T18:18:05.257137+0000","last_clean":"2024-01-15T18:18:05.257137+0000","last_became_active":"2024-01-15T18:12:10.019818+0000","last_became_peered":"2024-01-15T18:12:10.019818+0000","last_unstale":"2024-01-15T18:18:05.257137+0000","last_undegraded":"2024-01-15T18:18:05.257137+0000","last_fullsized":"2024-01-15T18:18:05.257137+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'1","last_scrub_stamp":"2024-01-15T18:18:05.256946+0000","last_deep_scrub":"114'1","last_deep_scrub_stamp":"2024-01-15T18:18:05.256946+0000","last_clean_scrub_stamp":"2024-01-15T18:18:05.256946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:47:07.033483+0000","scrub_duration":0.0043509610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"117'73","reported_seq":110,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:13.209425+0000","last_change":"2024-01-15T18:18:13.209530+0000","last_active":"2024-01-15T18:18:13.209425+0000","last_peered":"2024-01-15T18:18:13.209425+0000","last_clean":"2024-01-15T18:18:13.209425+0000","last_became_active":"2024-01-15T18:12:11.050002+0000","last_became_peered":"2024-01-15T18:12:11.050002+0000","last_unstale":"2024-01-15T18:18:13.209425+0000","last_undegraded":"2024-01-15T18:18:13.209425+0000","last_fullsized":"2024-01-15T18:18:13.209425+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'73","last_scrub_stamp":"2024-01-15T18:18:13.209243+0000","last_deep_scrub":"117'73","last_deep_scrub_stamp":"2024-01-15T18:18:13.209243+0000","last_clean_scrub_stamp":"2024-01-15T18:18:13.209243+0000","objects_scrubbed":24,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:23:57.942383+0000","scrub_duration":0.27580295799999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.489503+0000","last_change":"2024-01-15T18:18:00.489523+0000","last_active":"2024-01-15T18:18:00.489503+0000","last_peered":"2024-01-15T18:18:00.489503+0000","last_clean":"2024-01-15T18:18:00.489503+0000","last_became_active":"2024-01-15T18:12:10.009676+0000","last_became_peered":"2024-01-15T18:12:10.009676+0000","last_unstale":"2024-01-15T18:18:00.489503+0000","last_undegraded":"2024-01-15T18:18:00.489503+0000","last_fullsized":"2024-01-15T18:18:00.489503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'1","last_scrub_stamp":"2024-01-15T18:18:00.489431+0000","last_deep_scrub":"114'1","last_deep_scrub_stamp":"2024-01-15T18:18:00.489431+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.489431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:33:48.954551+0000","scrub_duration":0.00316505,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"117'53","reported_seq":79,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.421022+0000","last_change":"2024-01-15T18:12:11.057984+0000","last_active":"2024-01-15T18:17:39.421022+0000","last_peered":"2024-01-15T18:17:39.421022+0000","last_clean":"2024-01-15T18:17:39.421022+0000","last_became_active":"2024-01-15T18:12:11.057640+0000","last_became_peered":"2024-01-15T18:12:11.057640+0000","last_unstale":"2024-01-15T18:17:39.421022+0000","last_undegraded":"2024-01-15T18:17:39.421022+0000","last_fullsized":"2024-01-15T18:17:39.421022+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857791,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":36865,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.801159+0000","last_change":"2024-01-15T18:17:58.801207+0000","last_active":"2024-01-15T18:17:58.801159+0000","last_peered":"2024-01-15T18:17:58.801159+0000","last_clean":"2024-01-15T18:17:58.801159+0000","last_became_active":"2024-01-15T18:12:10.002190+0000","last_became_peered":"2024-01-15T18:12:10.002190+0000","last_unstale":"2024-01-15T18:17:58.801159+0000","last_undegraded":"2024-01-15T18:17:58.801159+0000","last_fullsized":"2024-01-15T18:17:58.801159+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:58.800969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:58.800969+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.800969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:55.566839+0000","scrub_duration":0.0032888890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"117'59","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.379605+0000","last_change":"2024-01-15T18:18:17.379679+0000","last_active":"2024-01-15T18:18:17.379605+0000","last_peered":"2024-01-15T18:18:17.379605+0000","last_clean":"2024-01-15T18:18:17.379605+0000","last_became_active":"2024-01-15T18:12:11.046934+0000","last_became_peered":"2024-01-15T18:12:11.046934+0000","last_unstale":"2024-01-15T18:18:17.379605+0000","last_undegraded":"2024-01-15T18:18:17.379605+0000","last_fullsized":"2024-01-15T18:18:17.379605+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'59","last_scrub_stamp":"2024-01-15T18:18:17.379455+0000","last_deep_scrub":"117'59","last_deep_scrub_stamp":"2024-01-15T18:18:17.379455+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.379455+0000","objects_scrubbed":16,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:14.145325+0000","scrub_duration":0.19649055300000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":17,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:02.456235+0000","last_change":"2024-01-15T18:12:10.020864+0000","last_active":"2024-01-15T18:17:02.456235+0000","last_peered":"2024-01-15T18:17:02.456235+0000","last_clean":"2024-01-15T18:17:02.456235+0000","last_became_active":"2024-01-15T18:12:10.020577+0000","last_became_peered":"2024-01-15T18:12:10.020577+0000","last_unstale":"2024-01-15T18:17:02.456235+0000","last_undegraded":"2024-01-15T18:17:02.456235+0000","last_fullsized":"2024-01-15T18:17:02.456235+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"117'50","reported_seq":78,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.308905+0000","last_change":"2024-01-15T18:18:15.308980+0000","last_active":"2024-01-15T18:18:15.308905+0000","last_peered":"2024-01-15T18:18:15.308905+0000","last_clean":"2024-01-15T18:18:15.308905+0000","last_became_active":"2024-01-15T18:12:11.047704+0000","last_became_peered":"2024-01-15T18:12:11.047704+0000","last_unstale":"2024-01-15T18:18:15.308905+0000","last_undegraded":"2024-01-15T18:18:15.308905+0000","last_fullsized":"2024-01-15T18:18:15.308905+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'50","last_scrub_stamp":"2024-01-15T18:18:15.308693+0000","last_deep_scrub":"117'50","last_deep_scrub_stamp":"2024-01-15T18:18:15.308693+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.308693+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:38.450367+0000","scrub_duration":0.093927007000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"117'1","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.742613+0000","last_change":"2024-01-15T18:18:01.742668+0000","last_active":"2024-01-15T18:18:01.742613+0000","last_peered":"2024-01-15T18:18:01.742613+0000","last_clean":"2024-01-15T18:18:01.742613+0000","last_became_active":"2024-01-15T18:12:10.020028+0000","last_became_peered":"2024-01-15T18:12:10.020028+0000","last_unstale":"2024-01-15T18:18:01.742613+0000","last_undegraded":"2024-01-15T18:18:01.742613+0000","last_fullsized":"2024-01-15T18:18:01.742613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'1","last_scrub_stamp":"2024-01-15T18:18:01.742421+0000","last_deep_scrub":"117'1","last_deep_scrub_stamp":"2024-01-15T18:18:01.742421+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.742421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:19:38.888476+0000","scrub_duration":0.0037945140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"117'55","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.374401+0000","last_change":"2024-01-15T18:18:09.374484+0000","last_active":"2024-01-15T18:18:09.374401+0000","last_peered":"2024-01-15T18:18:09.374401+0000","last_clean":"2024-01-15T18:18:09.374401+0000","last_became_active":"2024-01-15T18:12:11.059393+0000","last_became_peered":"2024-01-15T18:12:11.059393+0000","last_unstale":"2024-01-15T18:18:09.374401+0000","last_undegraded":"2024-01-15T18:18:09.374401+0000","last_fullsized":"2024-01-15T18:18:09.374401+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'55","last_scrub_stamp":"2024-01-15T18:18:09.374248+0000","last_deep_scrub":"117'55","last_deep_scrub_stamp":"2024-01-15T18:18:09.374248+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.374248+0000","objects_scrubbed":18,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:44:26.424477+0000","scrub_duration":0.23458093399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.527582+0000","last_change":"2024-01-15T18:17:58.527616+0000","last_active":"2024-01-15T18:17:58.527582+0000","last_peered":"2024-01-15T18:17:58.527582+0000","last_clean":"2024-01-15T18:17:58.527582+0000","last_became_active":"2024-01-15T18:12:10.007549+0000","last_became_peered":"2024-01-15T18:12:10.007549+0000","last_unstale":"2024-01-15T18:17:58.527582+0000","last_undegraded":"2024-01-15T18:17:58.527582+0000","last_fullsized":"2024-01-15T18:17:58.527582+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:58.527475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:58.527475+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.527475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:32.463562+0000","scrub_duration":0.003638007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"117'43","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.538313+0000","last_change":"2024-01-15T18:12:11.031666+0000","last_active":"2024-01-15T18:17:39.538313+0000","last_peered":"2024-01-15T18:17:39.538313+0000","last_clean":"2024-01-15T18:17:39.538313+0000","last_became_active":"2024-01-15T18:12:11.030958+0000","last_became_peered":"2024-01-15T18:12:11.030958+0000","last_unstale":"2024-01-15T18:17:39.538313+0000","last_undegraded":"2024-01-15T18:17:39.538313+0000","last_fullsized":"2024-01-15T18:17:39.538313+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"117'14","reported_seq":39,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:14.392225+0000","last_change":"2024-01-15T18:18:14.392289+0000","last_active":"2024-01-15T18:18:14.392225+0000","last_peered":"2024-01-15T18:18:14.392225+0000","last_clean":"2024-01-15T18:18:14.392225+0000","last_became_active":"2024-01-15T18:12:10.024120+0000","last_became_peered":"2024-01-15T18:12:10.024120+0000","last_unstale":"2024-01-15T18:18:14.392225+0000","last_undegraded":"2024-01-15T18:18:14.392225+0000","last_fullsized":"2024-01-15T18:18:14.392225+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'14","last_scrub_stamp":"2024-01-15T18:18:14.391949+0000","last_deep_scrub":"117'14","last_deep_scrub_stamp":"2024-01-15T18:18:14.391949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:14.391949+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:31:42.989864+0000","scrub_duration":0.0085194199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"117'43","reported_seq":67,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.041055+0000","last_change":"2024-01-15T18:12:11.055694+0000","last_active":"2024-01-15T18:17:39.041055+0000","last_peered":"2024-01-15T18:17:39.041055+0000","last_clean":"2024-01-15T18:17:39.041055+0000","last_became_active":"2024-01-15T18:12:11.055290+0000","last_became_peered":"2024-01-15T18:12:11.055290+0000","last_unstale":"2024-01-15T18:17:39.041055+0000","last_undegraded":"2024-01-15T18:17:39.041055+0000","last_fullsized":"2024-01-15T18:17:39.041055+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"117'188","reported_seq":213,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:16.445379+0000","last_change":"2024-01-15T18:18:16.445450+0000","last_active":"2024-01-15T18:18:16.445379+0000","last_peered":"2024-01-15T18:18:16.445379+0000","last_clean":"2024-01-15T18:18:16.445379+0000","last_became_active":"2024-01-15T18:12:10.021497+0000","last_became_peered":"2024-01-15T18:12:10.021497+0000","last_unstale":"2024-01-15T18:18:16.445379+0000","last_undegraded":"2024-01-15T18:18:16.445379+0000","last_fullsized":"2024-01-15T18:18:16.445379+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'188","last_scrub_stamp":"2024-01-15T18:18:16.445190+0000","last_deep_scrub":"117'188","last_deep_scrub_stamp":"2024-01-15T18:18:16.445190+0000","last_clean_scrub_stamp":"2024-01-15T18:18:16.445190+0000","objects_scrubbed":4,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:09:41.548591+0000","scrub_duration":0.015015664,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639451,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"117'44","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.543606+0000","last_change":"2024-01-15T18:12:11.046101+0000","last_active":"2024-01-15T18:17:39.543606+0000","last_peered":"2024-01-15T18:17:39.543606+0000","last_clean":"2024-01-15T18:17:39.543606+0000","last_became_active":"2024-01-15T18:12:11.045774+0000","last_became_peered":"2024-01-15T18:12:11.045774+0000","last_unstale":"2024-01-15T18:17:39.543606+0000","last_undegraded":"2024-01-15T18:17:39.543606+0000","last_fullsized":"2024-01-15T18:17:39.543606+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"117'4","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.064488+0000","last_change":"2024-01-15T18:18:07.064553+0000","last_active":"2024-01-15T18:18:07.064488+0000","last_peered":"2024-01-15T18:18:07.064488+0000","last_clean":"2024-01-15T18:18:07.064488+0000","last_became_active":"2024-01-15T18:12:10.016618+0000","last_became_peered":"2024-01-15T18:12:10.016618+0000","last_unstale":"2024-01-15T18:18:07.064488+0000","last_undegraded":"2024-01-15T18:18:07.064488+0000","last_fullsized":"2024-01-15T18:18:07.064488+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'4","last_scrub_stamp":"2024-01-15T18:18:07.064304+0000","last_deep_scrub":"117'4","last_deep_scrub_stamp":"2024-01-15T18:18:07.064304+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.064304+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:55.108425+0000","scrub_duration":0.0089004340000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"117'52","reported_seq":73,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.370924+0000","last_change":"2024-01-15T18:12:11.051223+0000","last_active":"2024-01-15T18:17:39.370924+0000","last_peered":"2024-01-15T18:17:39.370924+0000","last_clean":"2024-01-15T18:17:39.370924+0000","last_became_active":"2024-01-15T18:12:11.050928+0000","last_became_peered":"2024-01-15T18:12:11.050928+0000","last_unstale":"2024-01-15T18:17:39.370924+0000","last_undegraded":"2024-01-15T18:17:39.370924+0000","last_fullsized":"2024-01-15T18:17:39.370924+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:04.129497+0000","last_change":"2024-01-15T18:18:04.129593+0000","last_active":"2024-01-15T18:18:04.129497+0000","last_peered":"2024-01-15T18:18:04.129497+0000","last_clean":"2024-01-15T18:18:04.129497+0000","last_became_active":"2024-01-15T18:12:10.015578+0000","last_became_peered":"2024-01-15T18:12:10.015578+0000","last_unstale":"2024-01-15T18:18:04.129497+0000","last_undegraded":"2024-01-15T18:18:04.129497+0000","last_fullsized":"2024-01-15T18:18:04.129497+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:04.129272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:04.129272+0000","last_clean_scrub_stamp":"2024-01-15T18:18:04.129272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:09.247715+0000","scrub_duration":0.003855626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"117'68","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:03.667643+0000","last_change":"2024-01-15T18:18:03.667703+0000","last_active":"2024-01-15T18:18:03.667643+0000","last_peered":"2024-01-15T18:18:03.667643+0000","last_clean":"2024-01-15T18:18:03.667643+0000","last_became_active":"2024-01-15T18:12:11.028865+0000","last_became_peered":"2024-01-15T18:12:11.028865+0000","last_unstale":"2024-01-15T18:18:03.667643+0000","last_undegraded":"2024-01-15T18:18:03.667643+0000","last_fullsized":"2024-01-15T18:18:03.667643+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'68","last_scrub_stamp":"2024-01-15T18:18:03.667447+0000","last_deep_scrub":"117'68","last_deep_scrub_stamp":"2024-01-15T18:18:03.667447+0000","last_clean_scrub_stamp":"2024-01-15T18:18:03.667447+0000","objects_scrubbed":25,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:28:27.833994+0000","scrub_duration":0.20722174700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663556,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"117'12","reported_seq":37,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:56.807714+0000","last_change":"2024-01-15T18:17:56.807742+0000","last_active":"2024-01-15T18:17:56.807714+0000","last_peered":"2024-01-15T18:17:56.807714+0000","last_clean":"2024-01-15T18:17:56.807714+0000","last_became_active":"2024-01-15T18:12:10.022398+0000","last_became_peered":"2024-01-15T18:12:10.022398+0000","last_unstale":"2024-01-15T18:17:56.807714+0000","last_undegraded":"2024-01-15T18:17:56.807714+0000","last_fullsized":"2024-01-15T18:17:56.807714+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'12","last_scrub_stamp":"2024-01-15T18:17:56.807649+0000","last_deep_scrub":"117'12","last_deep_scrub_stamp":"2024-01-15T18:17:56.807649+0000","last_clean_scrub_stamp":"2024-01-15T18:17:56.807649+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:43.916577+0000","scrub_duration":0.008646964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"117'34","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.960812+0000","last_change":"2024-01-15T18:18:10.960886+0000","last_active":"2024-01-15T18:18:10.960812+0000","last_peered":"2024-01-15T18:18:10.960812+0000","last_clean":"2024-01-15T18:18:10.960812+0000","last_became_active":"2024-01-15T18:12:11.043735+0000","last_became_peered":"2024-01-15T18:12:11.043735+0000","last_unstale":"2024-01-15T18:18:10.960812+0000","last_undegraded":"2024-01-15T18:18:10.960812+0000","last_fullsized":"2024-01-15T18:18:10.960812+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'34","last_scrub_stamp":"2024-01-15T18:18:10.960638+0000","last_deep_scrub":"117'34","last_deep_scrub_stamp":"2024-01-15T18:18:10.960638+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.960638+0000","objects_scrubbed":10,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:14:24.581595+0000","scrub_duration":0.105112926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"114'1","reported_seq":18,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:16:42.453388+0000","last_change":"2024-01-15T18:12:10.027277+0000","last_active":"2024-01-15T18:16:42.453388+0000","last_peered":"2024-01-15T18:16:42.453388+0000","last_clean":"2024-01-15T18:16:42.453388+0000","last_became_active":"2024-01-15T18:12:10.026730+0000","last_became_peered":"2024-01-15T18:12:10.026730+0000","last_unstale":"2024-01-15T18:16:42.453388+0000","last_undegraded":"2024-01-15T18:16:42.453388+0000","last_fullsized":"2024-01-15T18:16:42.453388+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:08.969356+0000","last_clean_scrub_stamp":"2024-01-15T18:12:08.969356+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"117'36","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.414999+0000","last_change":"2024-01-15T18:12:11.061270+0000","last_active":"2024-01-15T18:17:39.414999+0000","last_peered":"2024-01-15T18:17:39.414999+0000","last_clean":"2024-01-15T18:17:39.414999+0000","last_became_active":"2024-01-15T18:12:11.061002+0000","last_became_peered":"2024-01-15T18:12:11.061002+0000","last_unstale":"2024-01-15T18:17:39.414999+0000","last_undegraded":"2024-01-15T18:17:39.414999+0000","last_fullsized":"2024-01-15T18:17:39.414999+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"117'3","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.859053+0000","last_change":"2024-01-15T18:18:07.859110+0000","last_active":"2024-01-15T18:18:07.859053+0000","last_peered":"2024-01-15T18:18:07.859053+0000","last_clean":"2024-01-15T18:18:07.859053+0000","last_became_active":"2024-01-15T18:12:10.019706+0000","last_became_peered":"2024-01-15T18:12:10.019706+0000","last_unstale":"2024-01-15T18:18:07.859053+0000","last_undegraded":"2024-01-15T18:18:07.859053+0000","last_fullsized":"2024-01-15T18:18:07.859053+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'3","last_scrub_stamp":"2024-01-15T18:18:07.858872+0000","last_deep_scrub":"117'3","last_deep_scrub_stamp":"2024-01-15T18:18:07.858872+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.858872+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:12:03.175146+0000","scrub_duration":0.0077351470000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"117'44","reported_seq":71,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.218402+0000","last_change":"2024-01-15T18:12:11.060480+0000","last_active":"2024-01-15T18:17:39.218402+0000","last_peered":"2024-01-15T18:17:39.218402+0000","last_clean":"2024-01-15T18:17:39.218402+0000","last_became_active":"2024-01-15T18:12:11.060154+0000","last_became_peered":"2024-01-15T18:12:11.060154+0000","last_unstale":"2024-01-15T18:17:39.218402+0000","last_undegraded":"2024-01-15T18:17:39.218402+0000","last_fullsized":"2024-01-15T18:17:39.218402+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"117'2","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.161684+0000","last_change":"2024-01-15T18:18:01.161747+0000","last_active":"2024-01-15T18:18:01.161684+0000","last_peered":"2024-01-15T18:18:01.161684+0000","last_clean":"2024-01-15T18:18:01.161684+0000","last_became_active":"2024-01-15T18:12:10.002957+0000","last_became_peered":"2024-01-15T18:12:10.002957+0000","last_unstale":"2024-01-15T18:18:01.161684+0000","last_undegraded":"2024-01-15T18:18:01.161684+0000","last_fullsized":"2024-01-15T18:18:01.161684+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'2","last_scrub_stamp":"2024-01-15T18:18:01.161505+0000","last_deep_scrub":"117'2","last_deep_scrub_stamp":"2024-01-15T18:18:01.161505+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.161505+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:01.536885+0000","scrub_duration":0.0071468110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"117'41","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.364689+0000","last_change":"2024-01-15T18:12:11.059234+0000","last_active":"2024-01-15T18:17:39.364689+0000","last_peered":"2024-01-15T18:17:39.364689+0000","last_clean":"2024-01-15T18:17:39.364689+0000","last_became_active":"2024-01-15T18:12:11.058945+0000","last_became_peered":"2024-01-15T18:12:11.058945+0000","last_unstale":"2024-01-15T18:17:39.364689+0000","last_undegraded":"2024-01-15T18:17:39.364689+0000","last_fullsized":"2024-01-15T18:17:39.364689+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:06.288831+0000","last_change":"2024-01-15T18:18:06.288945+0000","last_active":"2024-01-15T18:18:06.288831+0000","last_peered":"2024-01-15T18:18:06.288831+0000","last_clean":"2024-01-15T18:18:06.288831+0000","last_became_active":"2024-01-15T18:12:10.028284+0000","last_became_peered":"2024-01-15T18:12:10.028284+0000","last_unstale":"2024-01-15T18:18:06.288831+0000","last_undegraded":"2024-01-15T18:18:06.288831+0000","last_fullsized":"2024-01-15T18:18:06.288831+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:06.288498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:06.288498+0000","last_clean_scrub_stamp":"2024-01-15T18:18:06.288498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:21:19.512303+0000","scrub_duration":0.0041979610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"117'75","reported_seq":108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.815411+0000","last_change":"2024-01-15T18:18:15.815534+0000","last_active":"2024-01-15T18:18:15.815411+0000","last_peered":"2024-01-15T18:18:15.815411+0000","last_clean":"2024-01-15T18:18:15.815411+0000","last_became_active":"2024-01-15T18:12:11.051403+0000","last_became_peered":"2024-01-15T18:12:11.051403+0000","last_unstale":"2024-01-15T18:18:15.815411+0000","last_undegraded":"2024-01-15T18:18:15.815411+0000","last_fullsized":"2024-01-15T18:18:15.815411+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'75","last_scrub_stamp":"2024-01-15T18:18:15.815153+0000","last_deep_scrub":"117'75","last_deep_scrub_stamp":"2024-01-15T18:18:15.815153+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.815153+0000","objects_scrubbed":30,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:09.886495+0000","scrub_duration":0.40068222799999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"117'4","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:04.093090+0000","last_change":"2024-01-15T18:18:04.093163+0000","last_active":"2024-01-15T18:18:04.093090+0000","last_peered":"2024-01-15T18:18:04.093090+0000","last_clean":"2024-01-15T18:18:04.093090+0000","last_became_active":"2024-01-15T18:12:10.021344+0000","last_became_peered":"2024-01-15T18:12:10.021344+0000","last_unstale":"2024-01-15T18:18:04.093090+0000","last_undegraded":"2024-01-15T18:18:04.093090+0000","last_fullsized":"2024-01-15T18:18:04.093090+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'4","last_scrub_stamp":"2024-01-15T18:18:04.092822+0000","last_deep_scrub":"117'4","last_deep_scrub_stamp":"2024-01-15T18:18:04.092822+0000","last_clean_scrub_stamp":"2024-01-15T18:18:04.092822+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:51.201750+0000","scrub_duration":0.0068652299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"117'43","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.951562+0000","last_change":"2024-01-15T18:12:11.059848+0000","last_active":"2024-01-15T18:17:38.951562+0000","last_peered":"2024-01-15T18:17:38.951562+0000","last_clean":"2024-01-15T18:17:38.951562+0000","last_became_active":"2024-01-15T18:12:11.059311+0000","last_became_peered":"2024-01-15T18:12:11.059311+0000","last_unstale":"2024-01-15T18:17:38.951562+0000","last_undegraded":"2024-01-15T18:17:38.951562+0000","last_fullsized":"2024-01-15T18:17:38.951562+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.698869+0000","last_change":"2024-01-15T18:18:00.698919+0000","last_active":"2024-01-15T18:18:00.698869+0000","last_peered":"2024-01-15T18:18:00.698869+0000","last_clean":"2024-01-15T18:18:00.698869+0000","last_became_active":"2024-01-15T18:12:10.017411+0000","last_became_peered":"2024-01-15T18:12:10.017411+0000","last_unstale":"2024-01-15T18:18:00.698869+0000","last_undegraded":"2024-01-15T18:18:00.698869+0000","last_fullsized":"2024-01-15T18:18:00.698869+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:00.698524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:00.698524+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.698524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:05:24.402242+0000","scrub_duration":0.0031317810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"117'44","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.439791+0000","last_change":"2024-01-15T18:12:11.056765+0000","last_active":"2024-01-15T18:17:39.439791+0000","last_peered":"2024-01-15T18:17:39.439791+0000","last_clean":"2024-01-15T18:17:39.439791+0000","last_became_active":"2024-01-15T18:12:11.056311+0000","last_became_peered":"2024-01-15T18:12:11.056311+0000","last_unstale":"2024-01-15T18:17:39.439791+0000","last_undegraded":"2024-01-15T18:17:39.439791+0000","last_fullsized":"2024-01-15T18:17:39.439791+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.185102+0000","last_change":"2024-01-15T18:17:58.185126+0000","last_active":"2024-01-15T18:17:58.185102+0000","last_peered":"2024-01-15T18:17:58.185102+0000","last_clean":"2024-01-15T18:17:58.185102+0000","last_became_active":"2024-01-15T18:12:10.022687+0000","last_became_peered":"2024-01-15T18:12:10.022687+0000","last_unstale":"2024-01-15T18:17:58.185102+0000","last_undegraded":"2024-01-15T18:17:58.185102+0000","last_fullsized":"2024-01-15T18:17:58.185102+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:58.185008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:58.185008+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.185008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:00:35.863748+0000","scrub_duration":0.0025740260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"117'42","reported_seq":65,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.467336+0000","last_change":"2024-01-15T18:12:11.062171+0000","last_active":"2024-01-15T18:17:39.467336+0000","last_peered":"2024-01-15T18:17:39.467336+0000","last_clean":"2024-01-15T18:17:39.467336+0000","last_became_active":"2024-01-15T18:12:11.061830+0000","last_became_peered":"2024-01-15T18:12:11.061830+0000","last_unstale":"2024-01-15T18:17:39.467336+0000","last_undegraded":"2024-01-15T18:17:39.467336+0000","last_fullsized":"2024-01-15T18:17:39.467336+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109124,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.447947+0000","last_change":"2024-01-15T18:18:17.448002+0000","last_active":"2024-01-15T18:18:17.447947+0000","last_peered":"2024-01-15T18:18:17.447947+0000","last_clean":"2024-01-15T18:18:17.447947+0000","last_became_active":"2024-01-15T18:12:10.024554+0000","last_became_peered":"2024-01-15T18:12:10.024554+0000","last_unstale":"2024-01-15T18:18:17.447947+0000","last_undegraded":"2024-01-15T18:18:17.447947+0000","last_fullsized":"2024-01-15T18:18:17.447947+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:17.447735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:17.447735+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.447735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:29:15.174266+0000","scrub_duration":0.0035608710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"117'47","reported_seq":75,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:08.292548+0000","last_change":"2024-01-15T18:18:08.292673+0000","last_active":"2024-01-15T18:18:08.292548+0000","last_peered":"2024-01-15T18:18:08.292548+0000","last_clean":"2024-01-15T18:18:08.292548+0000","last_became_active":"2024-01-15T18:12:11.050747+0000","last_became_peered":"2024-01-15T18:12:11.050747+0000","last_unstale":"2024-01-15T18:18:08.292548+0000","last_undegraded":"2024-01-15T18:18:08.292548+0000","last_fullsized":"2024-01-15T18:18:08.292548+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'47","last_scrub_stamp":"2024-01-15T18:18:08.292356+0000","last_deep_scrub":"117'47","last_deep_scrub_stamp":"2024-01-15T18:18:08.292356+0000","last_clean_scrub_stamp":"2024-01-15T18:18:08.292356+0000","objects_scrubbed":13,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:42.792622+0000","scrub_duration":0.148150529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"117'3","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.477391+0000","last_change":"2024-01-15T18:18:01.477457+0000","last_active":"2024-01-15T18:18:01.477391+0000","last_peered":"2024-01-15T18:18:01.477391+0000","last_clean":"2024-01-15T18:18:01.477391+0000","last_became_active":"2024-01-15T18:12:10.025356+0000","last_became_peered":"2024-01-15T18:12:10.025356+0000","last_unstale":"2024-01-15T18:18:01.477391+0000","last_undegraded":"2024-01-15T18:18:01.477391+0000","last_fullsized":"2024-01-15T18:18:01.477391+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'3","last_scrub_stamp":"2024-01-15T18:18:01.477202+0000","last_deep_scrub":"117'3","last_deep_scrub_stamp":"2024-01-15T18:18:01.477202+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.477202+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:59.723407+0000","scrub_duration":0.0075668130000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"117'41","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.072114+0000","last_change":"2024-01-15T18:12:11.050465+0000","last_active":"2024-01-15T18:17:39.072114+0000","last_peered":"2024-01-15T18:17:39.072114+0000","last_clean":"2024-01-15T18:17:39.072114+0000","last_became_active":"2024-01-15T18:12:11.050116+0000","last_became_peered":"2024-01-15T18:12:11.050116+0000","last_unstale":"2024-01-15T18:17:39.072114+0000","last_undegraded":"2024-01-15T18:17:39.072114+0000","last_fullsized":"2024-01-15T18:17:39.072114+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"117'18","reported_seq":43,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:02.206701+0000","last_change":"2024-01-15T18:18:02.206754+0000","last_active":"2024-01-15T18:18:02.206701+0000","last_peered":"2024-01-15T18:18:02.206701+0000","last_clean":"2024-01-15T18:18:02.206701+0000","last_became_active":"2024-01-15T18:12:10.020497+0000","last_became_peered":"2024-01-15T18:12:10.020497+0000","last_unstale":"2024-01-15T18:18:02.206701+0000","last_undegraded":"2024-01-15T18:18:02.206701+0000","last_fullsized":"2024-01-15T18:18:02.206701+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'18","last_scrub_stamp":"2024-01-15T18:18:02.206517+0000","last_deep_scrub":"117'18","last_deep_scrub_stamp":"2024-01-15T18:18:02.206517+0000","last_clean_scrub_stamp":"2024-01-15T18:18:02.206517+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:40:05.823508+0000","scrub_duration":0.0043064160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":503,"num_omap_keys":10,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"117'70","reported_seq":108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.709629+0000","last_change":"2024-01-15T18:18:09.709777+0000","last_active":"2024-01-15T18:18:09.709629+0000","last_peered":"2024-01-15T18:18:09.709629+0000","last_clean":"2024-01-15T18:18:09.709629+0000","last_became_active":"2024-01-15T18:12:11.050810+0000","last_became_peered":"2024-01-15T18:12:11.050810+0000","last_unstale":"2024-01-15T18:18:09.709629+0000","last_undegraded":"2024-01-15T18:18:09.709629+0000","last_fullsized":"2024-01-15T18:18:09.709629+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'70","last_scrub_stamp":"2024-01-15T18:18:09.709451+0000","last_deep_scrub":"117'70","last_deep_scrub_stamp":"2024-01-15T18:18:09.709451+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.709451+0000","objects_scrubbed":24,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:33.702809+0000","scrub_duration":0.24709842800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:12.368071+0000","last_change":"2024-01-15T18:18:12.368166+0000","last_active":"2024-01-15T18:18:12.368071+0000","last_peered":"2024-01-15T18:18:12.368071+0000","last_clean":"2024-01-15T18:18:12.368071+0000","last_became_active":"2024-01-15T18:12:10.024524+0000","last_became_peered":"2024-01-15T18:12:10.024524+0000","last_unstale":"2024-01-15T18:18:12.368071+0000","last_undegraded":"2024-01-15T18:18:12.368071+0000","last_fullsized":"2024-01-15T18:18:12.368071+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:12.367669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:12.367669+0000","last_clean_scrub_stamp":"2024-01-15T18:18:12.367669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:30:45.736811+0000","scrub_duration":0.0043195339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"117'58","reported_seq":88,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.345583+0000","last_change":"2024-01-15T18:18:10.345686+0000","last_active":"2024-01-15T18:18:10.345583+0000","last_peered":"2024-01-15T18:18:10.345583+0000","last_clean":"2024-01-15T18:18:10.345583+0000","last_became_active":"2024-01-15T18:12:11.061454+0000","last_became_peered":"2024-01-15T18:12:11.061454+0000","last_unstale":"2024-01-15T18:18:10.345583+0000","last_undegraded":"2024-01-15T18:18:10.345583+0000","last_fullsized":"2024-01-15T18:18:10.345583+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'58","last_scrub_stamp":"2024-01-15T18:18:10.345394+0000","last_deep_scrub":"117'58","last_deep_scrub_stamp":"2024-01-15T18:18:10.345394+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.345394+0000","objects_scrubbed":19,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:23:55.078534+0000","scrub_duration":0.209955119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:59.502083+0000","last_change":"2024-01-15T18:17:59.502129+0000","last_active":"2024-01-15T18:17:59.502083+0000","last_peered":"2024-01-15T18:17:59.502083+0000","last_clean":"2024-01-15T18:17:59.502083+0000","last_became_active":"2024-01-15T18:12:10.018057+0000","last_became_peered":"2024-01-15T18:12:10.018057+0000","last_unstale":"2024-01-15T18:17:59.502083+0000","last_undegraded":"2024-01-15T18:17:59.502083+0000","last_fullsized":"2024-01-15T18:17:59.502083+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:59.501895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:59.501895+0000","last_clean_scrub_stamp":"2024-01-15T18:17:59.501895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:07.865255+0000","scrub_duration":0.0038048180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"117'60","reported_seq":91,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:03.094203+0000","last_change":"2024-01-15T18:18:03.094301+0000","last_active":"2024-01-15T18:18:03.094203+0000","last_peered":"2024-01-15T18:18:03.094203+0000","last_clean":"2024-01-15T18:18:03.094203+0000","last_became_active":"2024-01-15T18:12:11.039586+0000","last_became_peered":"2024-01-15T18:12:11.039586+0000","last_unstale":"2024-01-15T18:18:03.094203+0000","last_undegraded":"2024-01-15T18:18:03.094203+0000","last_fullsized":"2024-01-15T18:18:03.094203+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'60","last_scrub_stamp":"2024-01-15T18:18:03.093993+0000","last_deep_scrub":"117'60","last_deep_scrub_stamp":"2024-01-15T18:18:03.093993+0000","last_clean_scrub_stamp":"2024-01-15T18:18:03.093993+0000","objects_scrubbed":16,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:14:52.828664+0000","scrub_duration":0.201843208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.271057+0000","last_change":"2024-01-15T18:18:00.271104+0000","last_active":"2024-01-15T18:18:00.271057+0000","last_peered":"2024-01-15T18:18:00.271057+0000","last_clean":"2024-01-15T18:18:00.271057+0000","last_became_active":"2024-01-15T18:12:10.019424+0000","last_became_peered":"2024-01-15T18:12:10.019424+0000","last_unstale":"2024-01-15T18:18:00.271057+0000","last_undegraded":"2024-01-15T18:18:00.271057+0000","last_fullsized":"2024-01-15T18:18:00.271057+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:00.270864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:00.270864+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.270864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:50.723379+0000","scrub_duration":0.0035368880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"117'82","reported_seq":115,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.250993+0000","last_change":"2024-01-15T18:18:09.251086+0000","last_active":"2024-01-15T18:18:09.250993+0000","last_peered":"2024-01-15T18:18:09.250993+0000","last_clean":"2024-01-15T18:18:09.250993+0000","last_became_active":"2024-01-15T18:12:11.035238+0000","last_became_peered":"2024-01-15T18:12:11.035238+0000","last_unstale":"2024-01-15T18:18:09.250993+0000","last_undegraded":"2024-01-15T18:18:09.250993+0000","last_fullsized":"2024-01-15T18:18:09.250993+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'82","last_scrub_stamp":"2024-01-15T18:18:09.250811+0000","last_deep_scrub":"117'82","last_deep_scrub_stamp":"2024-01-15T18:18:09.250811+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.250811+0000","objects_scrubbed":29,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:19.387926+0000","scrub_duration":0.33453240099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.884459+0000","last_change":"2024-01-15T18:18:00.884506+0000","last_active":"2024-01-15T18:18:00.884459+0000","last_peered":"2024-01-15T18:18:00.884459+0000","last_clean":"2024-01-15T18:18:00.884459+0000","last_became_active":"2024-01-15T18:12:10.017190+0000","last_became_peered":"2024-01-15T18:12:10.017190+0000","last_unstale":"2024-01-15T18:18:00.884459+0000","last_undegraded":"2024-01-15T18:18:00.884459+0000","last_fullsized":"2024-01-15T18:18:00.884459+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:00.884268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:00.884268+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.884268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:38.405683+0000","scrub_duration":0.003782021,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"117'46","reported_seq":71,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.255267+0000","last_change":"2024-01-15T18:12:11.056667+0000","last_active":"2024-01-15T18:18:00.255267+0000","last_peered":"2024-01-15T18:18:00.255267+0000","last_clean":"2024-01-15T18:18:00.255267+0000","last_became_active":"2024-01-15T18:12:11.056111+0000","last_became_peered":"2024-01-15T18:12:11.056111+0000","last_unstale":"2024-01-15T18:18:00.255267+0000","last_undegraded":"2024-01-15T18:18:00.255267+0000","last_fullsized":"2024-01-15T18:18:00.255267+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"117'11","reported_seq":45,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:02.775900+0000","last_change":"2024-01-15T18:18:02.775960+0000","last_active":"2024-01-15T18:18:02.775900+0000","last_peered":"2024-01-15T18:18:02.775900+0000","last_clean":"2024-01-15T18:18:02.775900+0000","last_became_active":"2024-01-15T18:12:10.020890+0000","last_became_peered":"2024-01-15T18:12:10.020890+0000","last_unstale":"2024-01-15T18:18:02.775900+0000","last_undegraded":"2024-01-15T18:18:02.775900+0000","last_fullsized":"2024-01-15T18:18:02.775900+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'11","last_scrub_stamp":"2024-01-15T18:18:02.775685+0000","last_deep_scrub":"117'11","last_deep_scrub_stamp":"2024-01-15T18:18:02.775685+0000","last_clean_scrub_stamp":"2024-01-15T18:18:02.775685+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:04.277157+0000","scrub_duration":0.0073569229999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"117'39","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.477235+0000","last_change":"2024-01-15T18:12:11.059132+0000","last_active":"2024-01-15T18:17:39.477235+0000","last_peered":"2024-01-15T18:17:39.477235+0000","last_clean":"2024-01-15T18:17:39.477235+0000","last_became_active":"2024-01-15T18:12:11.057943+0000","last_became_peered":"2024-01-15T18:12:11.057943+0000","last_unstale":"2024-01-15T18:17:39.477235+0000","last_undegraded":"2024-01-15T18:17:39.477235+0000","last_fullsized":"2024-01-15T18:17:39.477235+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:11.404503+0000","last_change":"2024-01-15T18:18:11.404503+0000","last_active":"2024-01-15T18:18:11.404503+0000","last_peered":"2024-01-15T18:18:11.404503+0000","last_clean":"2024-01-15T18:18:11.404503+0000","last_became_active":"2024-01-15T18:12:10.009870+0000","last_became_peered":"2024-01-15T18:12:10.009870+0000","last_unstale":"2024-01-15T18:18:11.404503+0000","last_undegraded":"2024-01-15T18:18:11.404503+0000","last_fullsized":"2024-01-15T18:18:11.404503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:11.404131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:11.404131+0000","last_clean_scrub_stamp":"2024-01-15T18:18:11.404131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:34:10.272987+0000","scrub_duration":0.0037587990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"117'31","reported_seq":55,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.499297+0000","last_change":"2024-01-15T18:12:11.051496+0000","last_active":"2024-01-15T18:17:39.499297+0000","last_peered":"2024-01-15T18:17:39.499297+0000","last_clean":"2024-01-15T18:17:39.499297+0000","last_became_active":"2024-01-15T18:12:11.051092+0000","last_became_peered":"2024-01-15T18:12:11.051092+0000","last_unstale":"2024-01-15T18:17:39.499297+0000","last_undegraded":"2024-01-15T18:17:39.499297+0000","last_fullsized":"2024-01-15T18:17:39.499297+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:02.284897+0000","last_change":"2024-01-15T18:18:02.284945+0000","last_active":"2024-01-15T18:18:02.284897+0000","last_peered":"2024-01-15T18:18:02.284897+0000","last_clean":"2024-01-15T18:18:02.284897+0000","last_became_active":"2024-01-15T18:12:10.026651+0000","last_became_peered":"2024-01-15T18:12:10.026651+0000","last_unstale":"2024-01-15T18:18:02.284897+0000","last_undegraded":"2024-01-15T18:18:02.284897+0000","last_fullsized":"2024-01-15T18:18:02.284897+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:02.284683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:02.284683+0000","last_clean_scrub_stamp":"2024-01-15T18:18:02.284683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:07.355476+0000","scrub_duration":0.0041976940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"117'119","reported_seq":262,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.554023+0000","last_change":"2024-01-15T18:18:01.554131+0000","last_active":"2024-01-15T18:18:01.554023+0000","last_peered":"2024-01-15T18:18:01.554023+0000","last_clean":"2024-01-15T18:18:01.554023+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:18:01.554023+0000","last_undegraded":"2024-01-15T18:18:01.554023+0000","last_fullsized":"2024-01-15T18:18:01.554023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'119","last_scrub_stamp":"2024-01-15T18:18:01.553859+0000","last_deep_scrub":"117'119","last_deep_scrub_stamp":"2024-01-15T18:18:01.553859+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.553859+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:28.775696+0000","scrub_duration":0.056845899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710133,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3376,"ondisk_log_size":3376,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":641570,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279,"num_write_kb":755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5503,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1343488,"data_stored":1283140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834588,"kb_used_data":1767380,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537252,"statfs":{"total":96636764160,"available":94758146048,"internally_reserved":0,"allocated":1809797120,"data_stored":1808970858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.254,"15min":1.383},"min":{"1min":0.80500000000000005,"5min":0.66900000000000004,"15min":0.40400000000000003},"max":{"1min":1.599,"5min":4.0229999999999997,"15min":19.843},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.49,"15min":1.5609999999999999},"min":{"1min":0.872,"5min":0.74399999999999999,"15min":0.53400000000000003},"max":{"1min":1.6319999999999999,"5min":16.678999999999998,"15min":30.498000000000001},"last":1.0840000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2649999999999999,"15min":1.397},"min":{"1min":0.91400000000000003,"5min":0.56599999999999995,"15min":0.45600000000000002},"max":{"1min":1.5700000000000001,"5min":3.9430000000000001,"15min":19.917000000000002},"last":1.764},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.54,"15min":1.5669999999999999},"min":{"1min":0.753,"5min":0.753,"15min":0.71199999999999997},"max":{"1min":1.6319999999999999,"5min":16.626999999999999,"15min":28.613},"last":1.2070000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2689999999999999,"15min":1.391},"min":{"1min":0.98399999999999999,"5min":0.75600000000000001,"15min":0.50600000000000001},"max":{"1min":1.6479999999999999,"5min":1.8140000000000001,"15min":12.061},"last":1.123},{"interface":"front","average":{"1min":1.355,"5min":1.528,"15min":1.5800000000000001},"min":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.48999999999999999},"max":{"1min":1.77,"5min":16.792000000000002,"15min":16.792000000000002},"last":1.667}]},{"osd":3,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.306,"15min":1.5229999999999999},"min":{"1min":1.0029999999999999,"5min":0.80700000000000005,"15min":0.56000000000000005},"max":{"1min":1.6180000000000001,"5min":1.734,"15min":16.632000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.619,"15min":1.639},"min":{"1min":0.94699999999999995,"5min":0.94399999999999995,"15min":0.56699999999999995},"max":{"1min":1.6930000000000001,"5min":16.129000000000001,"15min":16.129000000000001},"last":1.716}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.571,"15min":1.627},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.54200000000000004},"max":{"1min":1.585,"5min":16.454000000000001,"15min":23.042000000000002},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.357,"15min":1.677},"min":{"1min":0.873,"5min":0.80000000000000004,"15min":0.496},"max":{"1min":1.7410000000000001,"5min":1.885,"15min":38.682000000000002},"last":1.256}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.391,"5min":1.4259999999999999,"15min":1.5860000000000001},"min":{"1min":0.96799999999999997,"5min":0.76800000000000002,"15min":0.52700000000000002},"max":{"1min":1.867,"5min":4.5359999999999996,"15min":13.379},"last":1.48},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.71,"15min":1.7430000000000001},"min":{"1min":1.0649999999999999,"5min":0.98499999999999999,"15min":0.624},"max":{"1min":1.843,"5min":16.553000000000001,"15min":22.933},"last":1.556}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.7969999999999999,"15min":1.8600000000000001},"min":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":0.57399999999999995},"max":{"1min":1.8140000000000001,"5min":16.725999999999999,"15min":23.004000000000001},"last":1.333},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.4850000000000001,"15min":1.659},"min":{"1min":1.1220000000000001,"5min":1.008,"15min":0.58299999999999996},"max":{"1min":1.897,"5min":1.952,"15min":9.2599999999999998},"last":1.377}]}]},{"osd":6,"up_from":14,"seq":60129542426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1260568,"kb_used_data":1193360,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93111272,"statfs":{"total":96636764160,"available":95345942528,"internally_reserved":0,"allocated":1222000640,"data_stored":1221177506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2629999999999999,"15min":1.464},"min":{"1min":0.79600000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":31.805},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.244,"5min":1.3,"15min":1.4870000000000001},"min":{"1min":0.749,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8340000000000001,"5min":3.5409999999999999,"15min":31.702999999999999},"last":1.5800000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1339999999999999,"15min":1.2609999999999999},"min":{"1min":0.69699999999999995,"5min":0.59999999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.778,"15min":15.064},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2050000000000001,"15min":1.4299999999999999},"min":{"1min":0.86499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6279999999999999,"5min":1.9650000000000001,"15min":23.495999999999999},"last":0.88400000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2849999999999999,"15min":1.5349999999999999},"min":{"1min":0.81699999999999995,"5min":0.79100000000000004,"15min":0.59399999999999997},"max":{"1min":1.502,"5min":1.8899999999999999,"15min":23.087},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.331,"15min":1.397},"min":{"1min":1.0089999999999999,"5min":0.77000000000000002,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.9990000000000001,"15min":14.94},"last":1.538}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4359999999999999,"15min":1.6040000000000001},"min":{"1min":1.0620000000000001,"5min":0.65200000000000002,"15min":0.59899999999999998},"max":{"1min":1.7,"5min":1.8160000000000001,"15min":15.151999999999999},"last":1.728},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3779999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6080000000000001,"5min":3.8500000000000001,"15min":13.795},"last":1.381}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2729999999999999,"15min":1.585},"min":{"1min":0.90900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.53,"5min":3.3719999999999999,"15min":23.327000000000002},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.3560000000000001,"15min":1.548},"min":{"1min":0.80900000000000005,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.6839999999999999,"5min":1.8839999999999999,"15min":15.239000000000001},"last":1.7909999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.331,"15min":1.502},"min":{"1min":0.88700000000000001,"5min":0.58099999999999996,"15min":0.52500000000000002},"max":{"1min":1.7210000000000001,"5min":3.6480000000000001,"15min":13.593},"last":1.482},{"interface":"front","average":{"1min":1.49,"5min":1.55,"15min":1.8049999999999999},"min":{"1min":1.1930000000000001,"5min":0.66200000000000003,"15min":0.621},"max":{"1min":1.702,"5min":2.306,"15min":23.417999999999999},"last":1.23}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.47,"15min":1.716},"min":{"1min":0.98799999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8220000000000001,"5min":3.105,"15min":21.265000000000001},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.486,"5min":1.542,"15min":1.899},"min":{"1min":1.1200000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.7789999999999999,"5min":2.2229999999999999,"15min":23.738},"last":1.421}]}]},{"osd":5,"up_from":14,"seq":60129542426,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908440,"kb_used_data":832912,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93463400,"statfs":{"total":96636764160,"available":95706521600,"internally_reserved":0,"allocated":852901888,"data_stored":852079166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.202,"15min":1.2669999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.47799999999999998},"max":{"1min":1.595,"5min":2.7919999999999998,"15min":19.266999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.752,"5min":0.748,"15min":0.51400000000000001},"max":{"1min":1.671,"5min":1.7529999999999999,"15min":3.1760000000000002},"last":1.1120000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.359,"15min":1.3149999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.60699999999999998},"max":{"1min":1.653,"5min":15.554,"15min":15.554},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.242,"5min":1.302,"15min":1.3540000000000001},"min":{"1min":0.80100000000000005,"5min":0.69599999999999995,"15min":0.59099999999999997},"max":{"1min":1.643,"5min":6.1449999999999996,"15min":9.2379999999999995},"last":0.81100000000000005}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.621},"max":{"1min":1.694,"5min":2.3500000000000001,"15min":4.0369999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.349,"5min":1.512,"15min":1.4299999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.56000000000000005},"max":{"1min":1.617,"5min":15.468999999999999,"15min":15.468999999999999},"last":1.3280000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.327,"15min":1.409},"min":{"1min":0.78200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.718,"5min":1.984,"15min":9.1929999999999996},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4379999999999999,"15min":1.4650000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.64600000000000002},"max":{"1min":1.774,"5min":2.2949999999999999,"15min":7.0110000000000001},"last":1.232}]},{"osd":4,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.4319999999999999,"15min":1.47},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.41399999999999998},"max":{"1min":1.6040000000000001,"5min":15.254,"15min":15.254},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.4019999999999999,"15min":1.4710000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.8080000000000001,"5min":1.923,"15min":9.1519999999999992},"last":1.0349999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.456},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.39600000000000002},"max":{"1min":1.8480000000000001,"5min":2.2250000000000001,"15min":3.7559999999999998},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.593,"15min":1.5680000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53000000000000003},"max":{"1min":1.7330000000000001,"5min":15.366,"15min":15.366},"last":1.4039999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6100000000000001},"min":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.45800000000000002},"max":{"1min":1.873,"5min":2.0609999999999999,"15min":9.0600000000000005},"last":1.639},{"interface":"front","average":{"1min":1.522,"5min":1.5840000000000001,"15min":1.6890000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.57299999999999995},"max":{"1min":1.899,"5min":3.1659999999999999,"15min":19.356999999999999},"last":1.6699999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1711124,"kb_used_data":1643916,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92660716,"statfs":{"total":96636764160,"available":94884573184,"internally_reserved":0,"allocated":1683369984,"data_stored":1682551014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.264,"15min":1.3360000000000001},"min":{"1min":0.94799999999999995,"5min":0.70199999999999996,"15min":0.39200000000000002},"max":{"1min":1.601,"5min":2.004,"15min":17.346},"last":1.323},{"interface":"front","average":{"1min":1.409,"5min":1.3500000000000001,"15min":1.361},"min":{"1min":1.0900000000000001,"5min":0.97499999999999998,"15min":0.40100000000000002},"max":{"1min":1.744,"5min":1.804,"15min":7.9160000000000004},"last":1.413}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.1950000000000001,"15min":1.24},"min":{"1min":0.95199999999999996,"5min":0.76600000000000001,"15min":0.439},"max":{"1min":1.6699999999999999,"5min":1.861,"15min":11.128},"last":2.0819999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.23,"15min":1.234},"min":{"1min":0.876,"5min":0.82999999999999996,"15min":0.60799999999999998},"max":{"1min":1.619,"5min":1.7470000000000001,"15min":5.6109999999999998},"last":1.528}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.294,"15min":1.343},"min":{"1min":0.92300000000000004,"5min":0.83399999999999996,"15min":0.56899999999999995},"max":{"1min":1.544,"5min":1.8560000000000001,"15min":18.442},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.3720000000000001,"15min":1.399},"min":{"1min":1.1459999999999999,"5min":0.94799999999999995,"15min":0.81899999999999995},"max":{"1min":1.669,"5min":1.861,"15min":5.7050000000000001},"last":1.669}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.375,"15min":1.462},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.56699999999999995},"max":{"1min":1.6719999999999999,"5min":2.1859999999999999,"15min":17.364999999999998},"last":1.923},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3380000000000001,"15min":1.3899999999999999},"min":{"1min":1.022,"5min":0.85099999999999998,"15min":0.45300000000000001},"max":{"1min":1.601,"5min":1.7050000000000001,"15min":9.3320000000000007},"last":2.2389999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.29,"15min":1.4590000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.49099999999999999},"max":{"1min":1.6519999999999999,"5min":2.4089999999999998,"15min":29.422000000000001},"last":2.028},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3420000000000001,"15min":1.4379999999999999},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.53700000000000003},"max":{"1min":1.7549999999999999,"5min":1.9450000000000001,"15min":12.332000000000001},"last":2.3420000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2629999999999999,"15min":1.3899999999999999},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.54200000000000004},"max":{"1min":1.752,"5min":2.117,"15min":11.09},"last":1.867},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.482,"15min":1.6120000000000001},"min":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":0.51100000000000001},"max":{"1min":1.718,"5min":2.2639999999999998,"15min":17.224},"last":2.1619999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.504,"15min":1.593},"min":{"1min":1.0169999999999999,"5min":0.93300000000000005,"15min":0.60499999999999998},"max":{"1min":1.8400000000000001,"5min":2.052,"15min":12.247},"last":2.4319999999999999},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6439999999999999,"15min":1.73},"min":{"1min":1.294,"5min":1.159,"15min":0.61399999999999999},"max":{"1min":1.9039999999999999,"5min":2.3570000000000002,"15min":17.324999999999999},"last":2.2879999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542428,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462036,"kb_used_data":1386508,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":92909804,"statfs":{"total":96636764160,"available":95139639296,"internally_reserved":0,"allocated":1419784192,"data_stored":1418949643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.0349999999999999,"15min":1.1639999999999999},"min":{"1min":0.66400000000000003,"5min":0.52800000000000002,"15min":0.33300000000000002},"max":{"1min":1.7989999999999999,"5min":2.915,"15min":18.399999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.2509999999999999,"15min":1.2849999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.44400000000000001},"max":{"1min":1.8560000000000001,"5min":3.028,"15min":18.795999999999999},"last":1.292}]},{"osd":1,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.05,"15min":1.1299999999999999},"min":{"1min":0.65800000000000003,"5min":0.51200000000000001,"15min":0.32600000000000001},"max":{"1min":1.4870000000000001,"5min":5.3520000000000003,"15min":12.372},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.1180000000000001,"15min":1.2589999999999999},"min":{"1min":0.66000000000000003,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.736,"5min":4.125,"15min":11.228999999999999},"last":1.0249999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.218,"15min":1.2529999999999999},"min":{"1min":0.74199999999999999,"5min":0.59999999999999998,"15min":0.375},"max":{"1min":1.8680000000000001,"5min":2.3929999999999998,"15min":10.964},"last":1.5389999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.45,"15min":1.534},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.56100000000000005},"max":{"1min":1.7569999999999999,"5min":3.0659999999999998,"15min":20.495000000000001},"last":2.077}]},{"osd":4,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.389,"15min":1.51},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.67200000000000004},"max":{"1min":1.865,"5min":2.1139999999999999,"15min":13.536},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4530000000000001,"15min":1.5529999999999999},"min":{"1min":0.98999999999999999,"5min":0.83099999999999996,"15min":0.63700000000000001},"max":{"1min":1.905,"5min":6.2439999999999998,"15min":16.398},"last":1.726}]},{"osd":5,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.478,"15min":1.6519999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.67200000000000004},"max":{"1min":2.0670000000000002,"5min":2.7269999999999999,"15min":20.606000000000002},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.587,"5min":1.6060000000000001,"15min":1.631},"min":{"1min":1.0269999999999999,"5min":0.94299999999999995,"15min":0.66800000000000004},"max":{"1min":2.0489999999999999,"5min":2.617,"15min":11.603},"last":1.659}]},{"osd":6,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.554,"15min":1.675},"min":{"1min":0.93100000000000005,"5min":0.85499999999999998,"15min":0.53500000000000003},"max":{"1min":2.1230000000000002,"5min":6.3289999999999997,"15min":13.648999999999999},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.554,"5min":1.6100000000000001,"15min":1.7529999999999999},"min":{"1min":1.2190000000000001,"5min":1.0249999999999999,"15min":0.65800000000000003},"max":{"1min":1.849,"5min":6.367,"15min":20.556000000000001},"last":1.915}]},{"osd":7,"last update":"Mon Jan 15 18:17:35 2024","interfaces":[{"interface":"back","average":{"1min":1.6020000000000001,"5min":1.641,"15min":1.76},"min":{"1min":1.159,"5min":0.97899999999999998,"15min":0.60499999999999998},"max":{"1min":2.1269999999999998,"5min":4.8630000000000004,"15min":28.843},"last":1.8},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.6559999999999999,"15min":1.738},"min":{"1min":1.1830000000000001,"5min":0.80100000000000005,"15min":0.67200000000000004},"max":{"1min":2.0680000000000001,"5min":6.9569999999999999,"15min":15.143000000000001},"last":2.141}]}]},{"osd":2,"up_from":14,"seq":60129542428,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":982100,"kb_used_data":910732,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93389740,"statfs":{"total":96636764160,"available":95631093760,"internally_reserved":0,"allocated":932589568,"data_stored":931771659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.161,"15min":1.083},"min":{"1min":0.67600000000000005,"5min":0.51000000000000001,"15min":0.26600000000000001},"max":{"1min":1.419,"5min":22.707000000000001,"15min":22.707000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1970000000000001,"15min":1.357},"min":{"1min":0.76300000000000001,"5min":0.61099999999999999,"15min":0.39300000000000002},"max":{"1min":1.6879999999999999,"5min":2.5710000000000002,"15min":15.314},"last":0.72099999999999997}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1759999999999999,"15min":1.3240000000000001},"min":{"1min":0.73299999999999998,"5min":0.58599999999999997,"15min":0.38300000000000001},"max":{"1min":1.546,"5min":2.8679999999999999,"15min":34.463999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.327,"15min":1.504},"min":{"1min":0.90700000000000003,"5min":0.73899999999999999,"15min":0.373},"max":{"1min":1.675,"5min":2.794,"15min":25.859999999999999},"last":1.353}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1870000000000001,"15min":1.302},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.34399999999999997},"max":{"1min":1.464,"5min":2.7080000000000002,"15min":12.93},"last":1.397},{"interface":"front","average":{"1min":1.284,"5min":1.6919999999999999,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.46400000000000002},"max":{"1min":1.903,"5min":37.021999999999998,"15min":37.021999999999998},"last":1.4390000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6879999999999999},"min":{"1min":1.0289999999999999,"5min":0.96999999999999997,"15min":0.55700000000000005},"max":{"1min":1.78,"5min":3.419,"15min":29.986999999999998},"last":1.202},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.518,"15min":1.7150000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.56799999999999995},"max":{"1min":1.883,"5min":2.976,"15min":21.199999999999999},"last":1.0429999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.456,"15min":1.6499999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.51500000000000001},"max":{"1min":2.0880000000000001,"5min":2.218,"15min":14.066000000000001},"last":1.617},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.643,"15min":1.7849999999999999},"min":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":0.60899999999999999},"max":{"1min":1.8180000000000001,"5min":3.4790000000000001,"15min":14.025},"last":1.091}]},{"osd":6,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.6160000000000001,"15min":1.8069999999999999},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":0.68000000000000005},"max":{"1min":1.9910000000000001,"5min":3.0800000000000001,"15min":15.683999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.585,"15min":1.784},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.58899999999999997},"max":{"1min":2.2930000000000001,"5min":10.371,"15min":12.629},"last":1.669}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.671,"15min":2.1419999999999999},"min":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":0.63100000000000001},"max":{"1min":2.2210000000000001,"5min":3.177,"15min":57.006999999999998},"last":1.264},{"interface":"front","average":{"1min":1.595,"5min":1.6299999999999999,"15min":1.9630000000000001},"min":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":0.56100000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":22.853999999999999},"last":1.8440000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1141924,"kb_used_data":1066396,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93229916,"statfs":{"total":96636764160,"available":95467433984,"internally_reserved":0,"allocated":1091989504,"data_stored":1091155706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.96499999999999997,"15min":1.177},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.34399999999999997},"max":{"1min":1.1419999999999999,"5min":2.7389999999999999,"15min":30.460999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.115,"15min":1.26},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.314},"max":{"1min":1.6539999999999999,"5min":5.4779999999999998,"15min":18.658000000000001},"last":1.008}]},{"osd":2,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.075,"15min":1.1579999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.5589999999999999,"5min":1.7809999999999999,"15min":7.117},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.099,"5min":1.325,"15min":1.5129999999999999},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.728,"5min":1.98,"15min":30.593},"last":1.4630000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2529999999999999,"15min":1.45},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.48599999999999999},"max":{"1min":1.6779999999999999,"5min":1.889,"15min":30.547000000000001},"last":1.28},{"interface":"front","average":{"1min":1.018,"5min":1.1619999999999999,"15min":1.3200000000000001},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":2.0459999999999998,"15min":17.263000000000002},"last":1.4910000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.6639999999999999,"15min":1.6899999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.72699999999999998},"max":{"1min":1.6319999999999999,"5min":14.103,"15min":17.420000000000002},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.653,"15min":1.6040000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7849999999999999,"5min":14.140000000000001,"15min":14.140000000000001},"last":1.115}]},{"osd":5,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3859999999999999,"15min":1.5329999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.59599999999999997},"max":{"1min":1.843,"5min":2.077,"15min":9.0440000000000005},"last":1.335},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.508,"15min":1.7989999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.68300000000000005},"max":{"1min":1.891,"5min":2.3599999999999999,"15min":34.042999999999999},"last":1.9530000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.494,"15min":1.6830000000000001},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":0.64100000000000001},"max":{"1min":1.9790000000000001,"5min":2.3780000000000001,"15min":34.158000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.4850000000000001,"15min":1.637},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.62},"max":{"1min":1.8260000000000001,"5min":2.5579999999999998,"15min":17.510000000000002},"last":1.389}]},{"osd":7,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.603,"15min":1.8700000000000001},"min":{"1min":0.98299999999999998,"5min":0.82999999999999996,"15min":0.77300000000000002},"max":{"1min":1.998,"5min":2.4329999999999998,"15min":33.659999999999997},"last":1.879},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.6439999999999999,"15min":1.742},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.70299999999999996},"max":{"1min":1.9370000000000001,"5min":2.6669999999999998,"15min":11.882999999999999},"last":1.4370000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542428,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1769764,"kb_used_data":1698396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":92602076,"statfs":{"total":96636764160,"available":94824525824,"internally_reserved":0,"allocated":1739157504,"data_stored":1738307390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.1080000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4630000000000001,"5min":2.5209999999999999,"15min":12.050000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3100000000000001,"15min":1.357},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.36399999999999999},"max":{"1min":1.609,"5min":6.6890000000000001,"15min":11.962},"last":1.1459999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.248,"15min":1.2230000000000001},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.45500000000000002},"max":{"1min":1.7050000000000001,"5min":6.8099999999999996,"15min":11.59},"last":0.87},{"interface":"front","average":{"1min":0.995,"5min":1.0620000000000001,"15min":1.1970000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.6870000000000001,"5min":1.825,"15min":14.512},"last":1.4259999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.385,"15min":1.425},"min":{"1min":0.58899999999999997,"5min":0.51300000000000001,"15min":0.50800000000000001},"max":{"1min":1.7130000000000001,"5min":6.8719999999999999,"15min":11.904},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1899999999999999,"15min":1.1830000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.35199999999999998},"max":{"1min":1.696,"5min":2.3919999999999999,"15min":6.1539999999999999},"last":1.482}]},{"osd":4,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.29,"15min":1.542},"min":{"1min":0.68000000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.5860000000000001,"5min":2.1829999999999998,"15min":32.115000000000002},"last":1.212},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.579,"15min":1.6259999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.67700000000000005},"max":{"1min":1.6890000000000001,"5min":6.9329999999999998,"15min":11.992000000000001},"last":1.389}]},{"osd":5,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3879999999999999,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.60899999999999999,"15min":0.47499999999999998},"max":{"1min":1.514,"5min":2.593,"15min":32.174999999999997},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.6020000000000001,"15min":1.6160000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.442},"max":{"1min":2.157,"5min":15.907,"15min":17.225000000000001},"last":1.6799999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.528,"15min":1.754},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.66600000000000004},"max":{"1min":2.3290000000000002,"5min":2.5590000000000002,"15min":24.039999999999999},"last":1.806},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.49,"15min":1.7789999999999999},"min":{"1min":0.99199999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.758,"5min":2.5449999999999999,"15min":32.216999999999999},"last":1.53}]},{"osd":7,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.708,"15min":2.0800000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.69099999999999995},"max":{"1min":1.8300000000000001,"5min":7.0979999999999999,"15min":34.131},"last":1.556},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.641,"15min":1.984},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.61799999999999999},"max":{"1min":2.266,"5min":3.3759999999999999,"15min":44.694000000000003},"last":1.913}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":655360,"data_stored":639563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":639521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1736458240,"data_stored":1736442609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":931135488,"data_stored":931135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":851443712,"data_stored":851443712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:18:21.330 INFO:tasks.ceph:pgid 17.3d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.331 INFO:tasks.ceph:pgid 17.3c last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.331 INFO:tasks.ceph:pgid 17.38 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.331 INFO:tasks.ceph:pgid 17.37 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.331 INFO:tasks.ceph:pgid 17.33 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.332 INFO:tasks.ceph:pgid 17.30 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.332 INFO:tasks.ceph:pgid 17.2f last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.332 INFO:tasks.ceph:pgid 17.2d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.332 INFO:tasks.ceph:pgid 17.2b last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.332 INFO:tasks.ceph:pgid 17.2a last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.332 INFO:tasks.ceph:pgid 17.28 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.333 INFO:tasks.ceph:pgid 17.26 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.333 INFO:tasks.ceph:pgid 17.25 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.333 INFO:tasks.ceph:pgid 17.24 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.333 INFO:tasks.ceph:pgid 17.23 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.333 INFO:tasks.ceph:pgid 17.21 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.334 INFO:tasks.ceph:pgid 17.1d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.334 INFO:tasks.ceph:pgid 17.1a last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.334 INFO:tasks.ceph:pgid 17.18 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.334 INFO:tasks.ceph:pgid 16.18 last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.334 INFO:tasks.ceph:pgid 17.14 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.335 INFO:tasks.ceph:pgid 17.15 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.335 INFO:tasks.ceph:pgid 17.12 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.335 INFO:tasks.ceph:pgid 17.13 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.335 INFO:tasks.ceph:pgid 16.f last_scrub_stamp 2024-01-15T18:12:08.969356+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=8, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.335 INFO:tasks.ceph:pgid 17.f last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.335 INFO:tasks.ceph:pgid 17.c last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.336 INFO:tasks.ceph:pgid 17.d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.336 INFO:tasks.ceph:pgid 17.b last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.336 INFO:tasks.ceph:pgid 17.8 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.336 INFO:tasks.ceph:pgid 17.9 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.336 INFO:tasks.ceph:pgid 17.7 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.337 INFO:tasks.ceph:pgid 17.0 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.337 INFO:tasks.ceph:pgid 17.1 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.337 INFO:tasks.ceph:pgid 17.10 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:21.337 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-15T18:18:33.578 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:18:33.587 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:18:41.338 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:18:41.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.550+0000 7f7b4a800640 1 Processor -- start 2024-01-15T18:18:41.553 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.551+0000 7f7b4a800640 1 -- start start 2024-01-15T18:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.552+0000 7f7b4a800640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b440cc500 0x7f7b440cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.552+0000 7f7b4a800640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7f7b4406d160 0x7f7b4406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.555 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.552+0000 7f7b4a800640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b44070e50 con 0x7f7b44143c40 2024-01-15T18:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.552+0000 7f7b4a800640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b440723c0 con 0x7f7b440cc500 2024-01-15T18:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.552+0000 7f7b4a800640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b440cced0 con 0x7f7b4406d160 2024-01-15T18:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.553+0000 7f7b48a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b440cc500 0x7f7b440cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.553+0000 7f7b48a00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7f7b440cc500 0x7f7b440cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49986/0 (socket says 172.21.15.26:49986) 2024-01-15T18:18:41.556 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.553+0000 7f7b48a00640 1 -- 172.21.15.26:0/2403400829 learned_addr learned my addr 172.21.15.26:0/2403400829 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:18:41.557 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.553+0000 7f7b49e00640 1 --2- 172.21.15.26:0/2403400829 >> v2:172.21.15.124:3300/0 conn(0x7f7b4406d160 0x7f7b4406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.553+0000 7f7b48a00640 1 -- 172.21.15.26:0/2403400829 >> v2:172.21.15.124:3300/0 conn(0x7f7b4406d160 msgr2=0x7f7b4406d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:41.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.553+0000 7f7b48a00640 1 --2- 172.21.15.26:0/2403400829 >> v2:172.21.15.124:3300/0 conn(0x7f7b4406d160 0x7f7b4406d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.553+0000 7f7b48a00640 1 -- 172.21.15.26:0/2403400829 >> v1:172.21.15.124:6789/0 conn(0x7f7b44143c40 legacy=0x7f7b44071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:18:41.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.553+0000 7f7b48a00640 1 -- 172.21.15.26:0/2403400829 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b44144020 con 0x7f7b440cc500 2024-01-15T18:18:41.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.555+0000 7f7b48a00640 1 --2- 172.21.15.26:0/2403400829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b440cc500 0x7f7b440cc990 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f7b44070690 tx=0x7f7b2c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c09599583adae3 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:41.558 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.555+0000 7f7b43e00640 1 -- 172.21.15.26:0/2403400829 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c0317e0 con 0x7f7b440cc500 2024-01-15T18:18:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.555+0000 7f7b43e00640 1 -- 172.21.15.26:0/2403400829 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c031940 con 0x7f7b440cc500 2024-01-15T18:18:41.559 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.555+0000 7f7b43e00640 1 -- 172.21.15.26:0/2403400829 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c031cd0 con 0x7f7b440cc500 2024-01-15T18:18:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.556+0000 7f7b4a800640 1 -- 172.21.15.26:0/2403400829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b440cc500 msgr2=0x7f7b440cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.556+0000 7f7b4a800640 1 --2- 172.21.15.26:0/2403400829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b440cc500 0x7f7b440cc990 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f7b44070690 tx=0x7f7b2c02b880 comp rx=0 tx=0).stop 2024-01-15T18:18:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.556+0000 7f7b4a800640 1 -- 172.21.15.26:0/2403400829 shutdown_connections 2024-01-15T18:18:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.556+0000 7f7b4a800640 1 --2- 172.21.15.26:0/2403400829 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b440cc500 0x7f7b440cc990 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.560 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.556+0000 7f7b4a800640 1 --2- 172.21.15.26:0/2403400829 >> v2:172.21.15.124:3300/0 conn(0x7f7b4406d160 0x7f7b4406d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.556+0000 7f7b4a800640 1 -- 172.21.15.26:0/2403400829 >> 172.21.15.26:0/2403400829 conn(0x7f7b4406a800 msgr2=0x7f7b4406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.556+0000 7f7b4a800640 1 -- 172.21.15.26:0/2403400829 shutdown_connections 2024-01-15T18:18:41.561 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.556+0000 7f7b4a800640 1 -- 172.21.15.26:0/2403400829 wait complete. 2024-01-15T18:18:41.562 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.558+0000 7f7b4a800640 1 Processor -- start 2024-01-15T18:18:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.559+0000 7f7b4a800640 1 -- start start 2024-01-15T18:18:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.559+0000 7f7b4a800640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b4406d160 0x7f7b440b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b4a800640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b440cc500 0x7f7b440c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b4a800640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44143c40 0x7f7b4415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b4a800640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4413d800 con 0x7f7b44143c40 2024-01-15T18:18:41.563 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b4a800640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440b1b90 con 0x7f7b4406d160 2024-01-15T18:18:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b4a800640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b440c4b60 con 0x7f7b440cc500 2024-01-15T18:18:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b49e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44143c40 0x7f7b4415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b49e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44143c40 0x7f7b4415a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:49992/0 (socket says 172.21.15.26:49992) 2024-01-15T18:18:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b49e00640 1 -- 172.21.15.26:0/1011717571 learned_addr learned my addr 172.21.15.26:0/1011717571 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:18:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b48a00640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b440cc500 0x7f7b440c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.564 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.560+0000 7f7b49400640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b4406d160 0x7f7b440b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b49e00640 1 -- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b440cc500 msgr2=0x7f7b440c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b49e00640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b440cc500 0x7f7b440c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b49e00640 1 -- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b4406d160 msgr2=0x7f7b440b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b49e00640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b4406d160 0x7f7b440b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.566 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b49e00640 1 -- 172.21.15.26:0/1011717571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c0094d0 con 0x7f7b44143c40 2024-01-15T18:18:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b49e00640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44143c40 0x7f7b4415a580 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c007e40 tx=0x7f7b3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b41600640 1 -- 172.21.15.26:0/1011717571 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c014620 con 0x7f7b44143c40 2024-01-15T18:18:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b41600640 1 -- 172.21.15.26:0/1011717571 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c00ee90 con 0x7f7b44143c40 2024-01-15T18:18:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b41600640 1 -- 172.21.15.26:0/1011717571 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0149e0 con 0x7f7b44143c40 2024-01-15T18:18:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.561+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b440c4e40 con 0x7f7b44143c40 2024-01-15T18:18:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.562+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b440c5400 con 0x7f7b44143c40 2024-01-15T18:18:41.567 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.563+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b440b1db0 con 0x7f7b44143c40 2024-01-15T18:18:41.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.565+0000 7f7b41600640 1 -- 172.21.15.26:0/1011717571 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 45) v1 ==== 91222+0+0 (secure 0 0 0) 0x7f7b3c014b40 con 0x7f7b44143c40 2024-01-15T18:18:41.568 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.566+0000 7f7b41600640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b1c06c550 0x7f7b1c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:18:41.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.566+0000 7f7b41600640 1 -- 172.21.15.26:0/1011717571 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7f7b3c089590 con 0x7f7b44143c40 2024-01-15T18:18:41.569 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.566+0000 7f7b49400640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b1c06c550 0x7f7b1c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:18:41.570 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.568+0000 7f7b49400640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b1c06c550 0x7f7b1c06ea10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7b34006fd0 tx=0x7f7b34008040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:18:41.574 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.571+0000 7f7b41600640 1 -- 172.21.15.26:0/1011717571 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b440b1db0 con 0x7f7b44143c40 2024-01-15T18:18:41.779 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.777+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b44066e70 con 0x7f7b1c06c550 2024-01-15T18:18:41.805 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.803+0000 7f7b41600640 1 -- 172.21.15.26:0/1011717571 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290632 (secure 0 0 0) 0x7f7b44066e70 con 0x7f7b1c06c550 2024-01-15T18:18:41.810 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:18:41.811 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:18:41.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.808+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b1c06c550 msgr2=0x7f7b1c06ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.813 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.808+0000 7f7b4a800640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b1c06c550 0x7f7b1c06ea10 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7b34006fd0 tx=0x7f7b34008040 comp rx=0 tx=0).stop 2024-01-15T18:18:41.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.808+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44143c40 msgr2=0x7f7b4415a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:18:41.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.808+0000 7f7b4a800640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44143c40 0x7f7b4415a580 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c007e40 tx=0x7f7b3c00e4b0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.809+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 shutdown_connections 2024-01-15T18:18:41.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.809+0000 7f7b4a800640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7f7b440cc500 0x7f7b440c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.809+0000 7f7b4a800640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7f7b1c06c550 0x7f7b1c06ea10 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.809+0000 7f7b4a800640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7f7b44143c40 0x7f7b4415a580 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.814 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.809+0000 7f7b4a800640 1 --2- 172.21.15.26:0/1011717571 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7f7b4406d160 0x7f7b440b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:18:41.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.809+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 >> 172.21.15.26:0/1011717571 conn(0x7f7b4406a800 msgr2=0x7f7b4406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:18:41.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.809+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 shutdown_connections 2024-01-15T18:18:41.815 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:18:41.809+0000 7f7b4a800640 1 -- 172.21.15.26:0/1011717571 wait complete. 2024-01-15T18:18:41.830 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":822,"stamp":"2024-01-15T18:18:40.471462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369942071,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1744,"num_write_kb":5246066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5503,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4241,"ondisk_log_size":4241,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11070544,"kb_used_data":10499600,"kb_used_omap":12,"kb_used_meta":570867,"kb_avail":743904176,"statfs":{"total":773094113280,"available":761757876224,"internally_reserved":0,"allocated":10751590400,"data_stored":10744962942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":584568368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005321"},"pg_stats":[{"pgid":"17.3f","version":"117'78","reported_seq":119,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.182705+0000","last_change":"2024-01-15T18:18:15.182793+0000","last_active":"2024-01-15T18:18:15.182705+0000","last_peered":"2024-01-15T18:18:15.182705+0000","last_clean":"2024-01-15T18:18:15.182705+0000","last_became_active":"2024-01-15T18:12:53.334688+0000","last_became_peered":"2024-01-15T18:12:53.334688+0000","last_unstale":"2024-01-15T18:18:15.182705+0000","last_undegraded":"2024-01-15T18:18:15.182705+0000","last_fullsized":"2024-01-15T18:18:15.182705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'78","last_scrub_stamp":"2024-01-15T18:18:15.182527+0000","last_deep_scrub":"117'78","last_deep_scrub_stamp":"2024-01-15T18:18:15.182527+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.182527+0000","objects_scrubbed":25,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:21:28.406331+0000","scrub_duration":0.27837603599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3e","version":"117'64","reported_seq":97,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:16.135979+0000","last_change":"2024-01-15T18:18:16.135979+0000","last_active":"2024-01-15T18:18:16.135979+0000","last_peered":"2024-01-15T18:18:16.135979+0000","last_clean":"2024-01-15T18:18:16.135979+0000","last_became_active":"2024-01-15T18:12:11.053279+0000","last_became_peered":"2024-01-15T18:12:11.053279+0000","last_unstale":"2024-01-15T18:18:16.135979+0000","last_undegraded":"2024-01-15T18:18:16.135979+0000","last_fullsized":"2024-01-15T18:18:16.135979+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'64","last_scrub_stamp":"2024-01-15T18:18:16.135588+0000","last_deep_scrub":"117'64","last_deep_scrub_stamp":"2024-01-15T18:18:16.135588+0000","last_clean_scrub_stamp":"2024-01-15T18:18:16.135588+0000","objects_scrubbed":19,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:16:49.227522+0000","scrub_duration":0.232285464,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"117'63","reported_seq":96,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:35.501988+0000","last_change":"2024-01-15T18:18:35.502092+0000","last_active":"2024-01-15T18:18:35.501988+0000","last_peered":"2024-01-15T18:18:35.501988+0000","last_clean":"2024-01-15T18:18:35.501988+0000","last_became_active":"2024-01-15T18:12:11.057543+0000","last_became_peered":"2024-01-15T18:12:11.057543+0000","last_unstale":"2024-01-15T18:18:35.501988+0000","last_undegraded":"2024-01-15T18:18:35.501988+0000","last_fullsized":"2024-01-15T18:18:35.501988+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:35.501850+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:35.501850+0000","last_clean_scrub_stamp":"2024-01-15T18:18:35.501850+0000","objects_scrubbed":19,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:25.874738+0000","scrub_duration":0.25578943700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"117'68","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:32.563179+0000","last_change":"2024-01-15T18:18:32.563337+0000","last_active":"2024-01-15T18:18:32.563179+0000","last_peered":"2024-01-15T18:18:32.563179+0000","last_clean":"2024-01-15T18:18:32.563179+0000","last_became_active":"2024-01-15T18:12:11.047008+0000","last_became_peered":"2024-01-15T18:12:11.047008+0000","last_unstale":"2024-01-15T18:18:32.563179+0000","last_undegraded":"2024-01-15T18:18:32.563179+0000","last_fullsized":"2024-01-15T18:18:32.563179+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'68","last_scrub_stamp":"2024-01-15T18:18:32.562960+0000","last_deep_scrub":"117'68","last_deep_scrub_stamp":"2024-01-15T18:18:32.562960+0000","last_clean_scrub_stamp":"2024-01-15T18:18:32.562960+0000","objects_scrubbed":23,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:05.499582+0000","scrub_duration":0.25917564100000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"117'37","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.943087+0000","last_change":"2024-01-15T18:18:09.943156+0000","last_active":"2024-01-15T18:18:09.943087+0000","last_peered":"2024-01-15T18:18:09.943087+0000","last_clean":"2024-01-15T18:18:09.943087+0000","last_became_active":"2024-01-15T18:12:11.034909+0000","last_became_peered":"2024-01-15T18:12:11.034909+0000","last_unstale":"2024-01-15T18:18:09.943087+0000","last_undegraded":"2024-01-15T18:18:09.943087+0000","last_fullsized":"2024-01-15T18:18:09.943087+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'37","last_scrub_stamp":"2024-01-15T18:18:09.942945+0000","last_deep_scrub":"117'37","last_deep_scrub_stamp":"2024-01-15T18:18:09.942945+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.942945+0000","objects_scrubbed":10,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:42:24.509082+0000","scrub_duration":0.101423941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"117'69","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.095936+0000","last_change":"2024-01-15T18:18:15.096025+0000","last_active":"2024-01-15T18:18:15.095936+0000","last_peered":"2024-01-15T18:18:15.095936+0000","last_clean":"2024-01-15T18:18:15.095936+0000","last_became_active":"2024-01-15T18:12:11.054803+0000","last_became_peered":"2024-01-15T18:12:11.054803+0000","last_unstale":"2024-01-15T18:18:15.095936+0000","last_undegraded":"2024-01-15T18:18:15.095936+0000","last_fullsized":"2024-01-15T18:18:15.095936+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'69","last_scrub_stamp":"2024-01-15T18:18:15.095819+0000","last_deep_scrub":"117'69","last_deep_scrub_stamp":"2024-01-15T18:18:15.095819+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.095819+0000","objects_scrubbed":21,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:46.439597+0000","scrub_duration":0.18604361999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"117'60","reported_seq":88,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.442469+0000","last_change":"2024-01-15T18:18:09.442535+0000","last_active":"2024-01-15T18:18:09.442469+0000","last_peered":"2024-01-15T18:18:09.442469+0000","last_clean":"2024-01-15T18:18:09.442469+0000","last_became_active":"2024-01-15T18:12:11.044607+0000","last_became_peered":"2024-01-15T18:12:11.044607+0000","last_unstale":"2024-01-15T18:18:09.442469+0000","last_undegraded":"2024-01-15T18:18:09.442469+0000","last_fullsized":"2024-01-15T18:18:09.442469+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'60","last_scrub_stamp":"2024-01-15T18:18:09.442344+0000","last_deep_scrub":"117'60","last_deep_scrub_stamp":"2024-01-15T18:18:09.442344+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.442344+0000","objects_scrubbed":17,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:43:41.911933+0000","scrub_duration":0.21794185399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"117'63","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:26.581150+0000","last_change":"2024-01-15T18:18:26.581229+0000","last_active":"2024-01-15T18:18:26.581150+0000","last_peered":"2024-01-15T18:18:26.581150+0000","last_clean":"2024-01-15T18:18:26.581150+0000","last_became_active":"2024-01-15T18:12:11.050697+0000","last_became_peered":"2024-01-15T18:12:11.050697+0000","last_unstale":"2024-01-15T18:18:26.581150+0000","last_undegraded":"2024-01-15T18:18:26.581150+0000","last_fullsized":"2024-01-15T18:18:26.581150+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:26.580974+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:26.580974+0000","last_clean_scrub_stamp":"2024-01-15T18:18:26.580974+0000","objects_scrubbed":19,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:04.570500+0000","scrub_duration":0.20942063299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"117'68","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:36.525482+0000","last_change":"2024-01-15T18:18:36.525588+0000","last_active":"2024-01-15T18:18:36.525482+0000","last_peered":"2024-01-15T18:18:36.525482+0000","last_clean":"2024-01-15T18:18:36.525482+0000","last_became_active":"2024-01-15T18:12:11.047693+0000","last_became_peered":"2024-01-15T18:12:11.047693+0000","last_unstale":"2024-01-15T18:18:36.525482+0000","last_undegraded":"2024-01-15T18:18:36.525482+0000","last_fullsized":"2024-01-15T18:18:36.525482+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'68","last_scrub_stamp":"2024-01-15T18:18:36.525358+0000","last_deep_scrub":"117'68","last_deep_scrub_stamp":"2024-01-15T18:18:36.525358+0000","last_clean_scrub_stamp":"2024-01-15T18:18:36.525358+0000","objects_scrubbed":21,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:39:38.655063+0000","scrub_duration":0.201605748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"117'58","reported_seq":92,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:55.436550+0000","last_change":"2024-01-15T18:17:55.436587+0000","last_active":"2024-01-15T18:17:55.436550+0000","last_peered":"2024-01-15T18:17:55.436550+0000","last_clean":"2024-01-15T18:17:55.436550+0000","last_became_active":"2024-01-15T18:12:11.052429+0000","last_became_peered":"2024-01-15T18:12:11.052429+0000","last_unstale":"2024-01-15T18:17:55.436550+0000","last_undegraded":"2024-01-15T18:17:55.436550+0000","last_fullsized":"2024-01-15T18:17:55.436550+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'58","last_scrub_stamp":"2024-01-15T18:17:55.436465+0000","last_deep_scrub":"117'58","last_deep_scrub_stamp":"2024-01-15T18:17:55.436465+0000","last_clean_scrub_stamp":"2024-01-15T18:17:55.436465+0000","objects_scrubbed":17,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:11:39.403468+0000","scrub_duration":0.096421670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"117'69","reported_seq":106,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.262993+0000","last_change":"2024-01-15T18:18:07.263053+0000","last_active":"2024-01-15T18:18:07.262993+0000","last_peered":"2024-01-15T18:18:07.262993+0000","last_clean":"2024-01-15T18:18:07.262993+0000","last_became_active":"2024-01-15T18:12:11.042720+0000","last_became_peered":"2024-01-15T18:12:11.042720+0000","last_unstale":"2024-01-15T18:18:07.262993+0000","last_undegraded":"2024-01-15T18:18:07.262993+0000","last_fullsized":"2024-01-15T18:18:07.262993+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'69","last_scrub_stamp":"2024-01-15T18:18:07.262832+0000","last_deep_scrub":"117'69","last_deep_scrub_stamp":"2024-01-15T18:18:07.262832+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.262832+0000","objects_scrubbed":28,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:08:52.313757+0000","scrub_duration":0.35287618199999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"117'73","reported_seq":114,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:08.692443+0000","last_change":"2024-01-15T18:18:08.692487+0000","last_active":"2024-01-15T18:18:08.692443+0000","last_peered":"2024-01-15T18:18:08.692443+0000","last_clean":"2024-01-15T18:18:08.692443+0000","last_became_active":"2024-01-15T18:12:11.044155+0000","last_became_peered":"2024-01-15T18:12:11.044155+0000","last_unstale":"2024-01-15T18:18:08.692443+0000","last_undegraded":"2024-01-15T18:18:08.692443+0000","last_fullsized":"2024-01-15T18:18:08.692443+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'73","last_scrub_stamp":"2024-01-15T18:18:08.692341+0000","last_deep_scrub":"117'73","last_deep_scrub_stamp":"2024-01-15T18:18:08.692341+0000","last_clean_scrub_stamp":"2024-01-15T18:18:08.692341+0000","objects_scrubbed":28,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:17:19.915429+0000","scrub_duration":0.266379529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"117'57","reported_seq":93,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:27.628796+0000","last_change":"2024-01-15T18:18:27.628885+0000","last_active":"2024-01-15T18:18:27.628796+0000","last_peered":"2024-01-15T18:18:27.628796+0000","last_clean":"2024-01-15T18:18:27.628796+0000","last_became_active":"2024-01-15T18:12:11.051875+0000","last_became_peered":"2024-01-15T18:12:11.051875+0000","last_unstale":"2024-01-15T18:18:27.628796+0000","last_undegraded":"2024-01-15T18:18:27.628796+0000","last_fullsized":"2024-01-15T18:18:27.628796+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'57","last_scrub_stamp":"2024-01-15T18:18:27.628608+0000","last_deep_scrub":"117'57","last_deep_scrub_stamp":"2024-01-15T18:18:27.628608+0000","last_clean_scrub_stamp":"2024-01-15T18:18:27.628608+0000","objects_scrubbed":19,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:34:26.497463+0000","scrub_duration":0.22558270899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"117'74","reported_seq":106,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:08.869762+0000","last_change":"2024-01-15T18:18:08.869841+0000","last_active":"2024-01-15T18:18:08.869762+0000","last_peered":"2024-01-15T18:18:08.869762+0000","last_clean":"2024-01-15T18:18:08.869762+0000","last_became_active":"2024-01-15T18:12:11.055813+0000","last_became_peered":"2024-01-15T18:12:11.055813+0000","last_unstale":"2024-01-15T18:18:08.869762+0000","last_undegraded":"2024-01-15T18:18:08.869762+0000","last_fullsized":"2024-01-15T18:18:08.869762+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'74","last_scrub_stamp":"2024-01-15T18:18:08.869620+0000","last_deep_scrub":"117'74","last_deep_scrub_stamp":"2024-01-15T18:18:08.869620+0000","last_clean_scrub_stamp":"2024-01-15T18:18:08.869620+0000","objects_scrubbed":27,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:52.461222+0000","scrub_duration":0.34224997800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"117'46","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:16.051211+0000","last_change":"2024-01-15T18:18:16.051305+0000","last_active":"2024-01-15T18:18:16.051211+0000","last_peered":"2024-01-15T18:18:16.051211+0000","last_clean":"2024-01-15T18:18:16.051211+0000","last_became_active":"2024-01-15T18:12:11.055217+0000","last_became_peered":"2024-01-15T18:12:11.055217+0000","last_unstale":"2024-01-15T18:18:16.051211+0000","last_undegraded":"2024-01-15T18:18:16.051211+0000","last_fullsized":"2024-01-15T18:18:16.051211+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'46","last_scrub_stamp":"2024-01-15T18:18:16.051013+0000","last_deep_scrub":"117'46","last_deep_scrub_stamp":"2024-01-15T18:18:16.051013+0000","last_clean_scrub_stamp":"2024-01-15T18:18:16.051013+0000","objects_scrubbed":12,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:03.749881+0000","scrub_duration":0.141849805,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"117'60","reported_seq":94,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:27.470382+0000","last_change":"2024-01-15T18:18:27.470514+0000","last_active":"2024-01-15T18:18:27.470382+0000","last_peered":"2024-01-15T18:18:27.470382+0000","last_clean":"2024-01-15T18:18:27.470382+0000","last_became_active":"2024-01-15T18:12:53.338860+0000","last_became_peered":"2024-01-15T18:12:53.338860+0000","last_unstale":"2024-01-15T18:18:27.470382+0000","last_undegraded":"2024-01-15T18:18:27.470382+0000","last_fullsized":"2024-01-15T18:18:27.470382+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'60","last_scrub_stamp":"2024-01-15T18:18:27.470106+0000","last_deep_scrub":"117'60","last_deep_scrub_stamp":"2024-01-15T18:18:27.470106+0000","last_clean_scrub_stamp":"2024-01-15T18:18:27.470106+0000","objects_scrubbed":18,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:57:23.041064+0000","scrub_duration":0.228570462,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2f","version":"117'66","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:23.332457+0000","last_change":"2024-01-15T18:18:23.332546+0000","last_active":"2024-01-15T18:18:23.332457+0000","last_peered":"2024-01-15T18:18:23.332457+0000","last_clean":"2024-01-15T18:18:23.332457+0000","last_became_active":"2024-01-15T18:12:11.042031+0000","last_became_peered":"2024-01-15T18:12:11.042031+0000","last_unstale":"2024-01-15T18:18:23.332457+0000","last_undegraded":"2024-01-15T18:18:23.332457+0000","last_fullsized":"2024-01-15T18:18:23.332457+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'66","last_scrub_stamp":"2024-01-15T18:18:23.332267+0000","last_deep_scrub":"117'66","last_deep_scrub_stamp":"2024-01-15T18:18:23.332267+0000","last_clean_scrub_stamp":"2024-01-15T18:18:23.332267+0000","objects_scrubbed":22,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:46.636204+0000","scrub_duration":0.27732246900000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"117'67","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:06.389526+0000","last_change":"2024-01-15T18:18:06.389619+0000","last_active":"2024-01-15T18:18:06.389526+0000","last_peered":"2024-01-15T18:18:06.389526+0000","last_clean":"2024-01-15T18:18:06.389526+0000","last_became_active":"2024-01-15T18:12:11.042814+0000","last_became_peered":"2024-01-15T18:12:11.042814+0000","last_unstale":"2024-01-15T18:18:06.389526+0000","last_undegraded":"2024-01-15T18:18:06.389526+0000","last_fullsized":"2024-01-15T18:18:06.389526+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'67","last_scrub_stamp":"2024-01-15T18:18:06.389332+0000","last_deep_scrub":"117'67","last_deep_scrub_stamp":"2024-01-15T18:18:06.389332+0000","last_clean_scrub_stamp":"2024-01-15T18:18:06.389332+0000","objects_scrubbed":21,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:57:01.960290+0000","scrub_duration":0.25979249500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"117'44","reported_seq":69,"reported_epoch":117,"state":"active+clean+scrubbing+deep","last_fresh":"2024-01-15T18:18:37.259019+0000","last_change":"2024-01-15T18:18:37.257309+0000","last_active":"2024-01-15T18:18:37.259019+0000","last_peered":"2024-01-15T18:18:37.259019+0000","last_clean":"2024-01-15T18:18:37.259019+0000","last_became_active":"2024-01-15T18:12:11.054790+0000","last_became_peered":"2024-01-15T18:12:11.054790+0000","last_unstale":"2024-01-15T18:18:37.259019+0000","last_undegraded":"2024-01-15T18:18:37.259019+0000","last_fullsized":"2024-01-15T18:18:37.259019+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"117'63","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:11.200840+0000","last_change":"2024-01-15T18:18:11.200952+0000","last_active":"2024-01-15T18:18:11.200840+0000","last_peered":"2024-01-15T18:18:11.200840+0000","last_clean":"2024-01-15T18:18:11.200840+0000","last_became_active":"2024-01-15T18:12:11.052289+0000","last_became_peered":"2024-01-15T18:12:11.052289+0000","last_unstale":"2024-01-15T18:18:11.200840+0000","last_undegraded":"2024-01-15T18:18:11.200840+0000","last_fullsized":"2024-01-15T18:18:11.200840+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:11.200610+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:11.200610+0000","last_clean_scrub_stamp":"2024-01-15T18:18:11.200610+0000","objects_scrubbed":17,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:42.709925+0000","scrub_duration":0.222891758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"117'82","reported_seq":120,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:20.266976+0000","last_change":"2024-01-15T18:18:20.267040+0000","last_active":"2024-01-15T18:18:20.266976+0000","last_peered":"2024-01-15T18:18:20.266976+0000","last_clean":"2024-01-15T18:18:20.266976+0000","last_became_active":"2024-01-15T18:12:11.036003+0000","last_became_peered":"2024-01-15T18:12:11.036003+0000","last_unstale":"2024-01-15T18:18:20.266976+0000","last_undegraded":"2024-01-15T18:18:20.266976+0000","last_fullsized":"2024-01-15T18:18:20.266976+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'82","last_scrub_stamp":"2024-01-15T18:18:20.266728+0000","last_deep_scrub":"117'82","last_deep_scrub_stamp":"2024-01-15T18:18:20.266728+0000","last_clean_scrub_stamp":"2024-01-15T18:18:20.266728+0000","objects_scrubbed":26,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:16:39.953673+0000","scrub_duration":0.28888784099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"117'68","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:35.796115+0000","last_change":"2024-01-15T18:18:35.796351+0000","last_active":"2024-01-15T18:18:35.796115+0000","last_peered":"2024-01-15T18:18:35.796115+0000","last_clean":"2024-01-15T18:18:35.796115+0000","last_became_active":"2024-01-15T18:12:11.044093+0000","last_became_peered":"2024-01-15T18:12:11.044093+0000","last_unstale":"2024-01-15T18:18:35.796115+0000","last_undegraded":"2024-01-15T18:18:35.796115+0000","last_fullsized":"2024-01-15T18:18:35.796115+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'68","last_scrub_stamp":"2024-01-15T18:18:35.795912+0000","last_deep_scrub":"117'68","last_deep_scrub_stamp":"2024-01-15T18:18:35.795912+0000","last_clean_scrub_stamp":"2024-01-15T18:18:35.795912+0000","objects_scrubbed":21,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:56:50.382706+0000","scrub_duration":0.24977105099999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"117'54","reported_seq":87,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:14.102196+0000","last_change":"2024-01-15T18:18:14.102285+0000","last_active":"2024-01-15T18:18:14.102196+0000","last_peered":"2024-01-15T18:18:14.102196+0000","last_clean":"2024-01-15T18:18:14.102196+0000","last_became_active":"2024-01-15T18:12:11.052428+0000","last_became_peered":"2024-01-15T18:12:11.052428+0000","last_unstale":"2024-01-15T18:18:14.102196+0000","last_undegraded":"2024-01-15T18:18:14.102196+0000","last_fullsized":"2024-01-15T18:18:14.102196+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'54","last_scrub_stamp":"2024-01-15T18:18:14.102006+0000","last_deep_scrub":"117'54","last_deep_scrub_stamp":"2024-01-15T18:18:14.102006+0000","last_clean_scrub_stamp":"2024-01-15T18:18:14.102006+0000","objects_scrubbed":15,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:05:29.630951+0000","scrub_duration":0.188245571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"117'61","reported_seq":92,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:19.782445+0000","last_change":"2024-01-15T18:18:19.782542+0000","last_active":"2024-01-15T18:18:19.782445+0000","last_peered":"2024-01-15T18:18:19.782445+0000","last_clean":"2024-01-15T18:18:19.782445+0000","last_became_active":"2024-01-15T18:12:11.051488+0000","last_became_peered":"2024-01-15T18:12:11.051488+0000","last_unstale":"2024-01-15T18:18:19.782445+0000","last_undegraded":"2024-01-15T18:18:19.782445+0000","last_fullsized":"2024-01-15T18:18:19.782445+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'61","last_scrub_stamp":"2024-01-15T18:18:19.782263+0000","last_deep_scrub":"117'61","last_deep_scrub_stamp":"2024-01-15T18:18:19.782263+0000","last_clean_scrub_stamp":"2024-01-15T18:18:19.782263+0000","objects_scrubbed":18,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:21.371708+0000","scrub_duration":0.21495783399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"117'56","reported_seq":90,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.157818+0000","last_change":"2024-01-15T18:18:17.157892+0000","last_active":"2024-01-15T18:18:17.157818+0000","last_peered":"2024-01-15T18:18:17.157818+0000","last_clean":"2024-01-15T18:18:17.157818+0000","last_became_active":"2024-01-15T18:12:11.041266+0000","last_became_peered":"2024-01-15T18:12:11.041266+0000","last_unstale":"2024-01-15T18:18:17.157818+0000","last_undegraded":"2024-01-15T18:18:17.157818+0000","last_fullsized":"2024-01-15T18:18:17.157818+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'56","last_scrub_stamp":"2024-01-15T18:18:17.157660+0000","last_deep_scrub":"117'56","last_deep_scrub_stamp":"2024-01-15T18:18:17.157660+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.157660+0000","objects_scrubbed":18,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:31:02.204776+0000","scrub_duration":0.223683413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"117'53","reported_seq":81,"reported_epoch":117,"state":"active+clean+scrubbing+deep","last_fresh":"2024-01-15T18:18:37.350387+0000","last_change":"2024-01-15T18:18:37.348453+0000","last_active":"2024-01-15T18:18:37.350387+0000","last_peered":"2024-01-15T18:18:37.350387+0000","last_clean":"2024-01-15T18:18:37.350387+0000","last_became_active":"2024-01-15T18:12:11.048234+0000","last_became_peered":"2024-01-15T18:12:11.048234+0000","last_unstale":"2024-01-15T18:18:37.350387+0000","last_undegraded":"2024-01-15T18:18:37.350387+0000","last_fullsized":"2024-01-15T18:18:37.350387+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"117'58","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:23.161077+0000","last_change":"2024-01-15T18:18:23.161141+0000","last_active":"2024-01-15T18:18:23.161077+0000","last_peered":"2024-01-15T18:18:23.161077+0000","last_clean":"2024-01-15T18:18:23.161077+0000","last_became_active":"2024-01-15T18:12:11.061724+0000","last_became_peered":"2024-01-15T18:12:11.061724+0000","last_unstale":"2024-01-15T18:18:23.161077+0000","last_undegraded":"2024-01-15T18:18:23.161077+0000","last_fullsized":"2024-01-15T18:18:23.161077+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'58","last_scrub_stamp":"2024-01-15T18:18:23.160963+0000","last_deep_scrub":"117'58","last_deep_scrub_stamp":"2024-01-15T18:18:23.160963+0000","last_clean_scrub_stamp":"2024-01-15T18:18:23.160963+0000","objects_scrubbed":21,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:47:24.937500+0000","scrub_duration":0.21027944000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"117'38","reported_seq":57,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.487271+0000","last_change":"2024-01-15T18:12:11.044940+0000","last_active":"2024-01-15T18:17:39.487271+0000","last_peered":"2024-01-15T18:17:39.487271+0000","last_clean":"2024-01-15T18:17:39.487271+0000","last_became_active":"2024-01-15T18:12:11.043801+0000","last_became_peered":"2024-01-15T18:12:11.043801+0000","last_unstale":"2024-01-15T18:17:39.487271+0000","last_undegraded":"2024-01-15T18:17:39.487271+0000","last_fullsized":"2024-01-15T18:17:39.487271+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"117'72","reported_seq":107,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:19.260856+0000","last_change":"2024-01-15T18:18:19.260856+0000","last_active":"2024-01-15T18:18:19.260856+0000","last_peered":"2024-01-15T18:18:19.260856+0000","last_clean":"2024-01-15T18:18:19.260856+0000","last_became_active":"2024-01-15T18:12:11.047739+0000","last_became_peered":"2024-01-15T18:12:11.047739+0000","last_unstale":"2024-01-15T18:18:19.260856+0000","last_undegraded":"2024-01-15T18:18:19.260856+0000","last_fullsized":"2024-01-15T18:18:19.260856+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'72","last_scrub_stamp":"2024-01-15T18:18:19.260675+0000","last_deep_scrub":"117'72","last_deep_scrub_stamp":"2024-01-15T18:18:19.260675+0000","last_clean_scrub_stamp":"2024-01-15T18:18:19.260675+0000","objects_scrubbed":26,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:05:02.852277+0000","scrub_duration":0.32170053799999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"117'57","reported_seq":91,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.431430+0000","last_change":"2024-01-15T18:18:10.431430+0000","last_active":"2024-01-15T18:18:10.431430+0000","last_peered":"2024-01-15T18:18:10.431430+0000","last_clean":"2024-01-15T18:18:10.431430+0000","last_became_active":"2024-01-15T18:12:11.039393+0000","last_became_peered":"2024-01-15T18:12:11.039393+0000","last_unstale":"2024-01-15T18:18:10.431430+0000","last_undegraded":"2024-01-15T18:18:10.431430+0000","last_fullsized":"2024-01-15T18:18:10.431430+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'57","last_scrub_stamp":"2024-01-15T18:18:10.431168+0000","last_deep_scrub":"117'57","last_deep_scrub_stamp":"2024-01-15T18:18:10.431168+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.431168+0000","objects_scrubbed":16,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:49:04.771755+0000","scrub_duration":0.21320547200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"117'62","reported_seq":91,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:55.114156+0000","last_change":"2024-01-15T18:12:11.059603+0000","last_active":"2024-01-15T18:17:55.114156+0000","last_peered":"2024-01-15T18:17:55.114156+0000","last_clean":"2024-01-15T18:17:55.114156+0000","last_became_active":"2024-01-15T18:12:11.059276+0000","last_became_peered":"2024-01-15T18:12:11.059276+0000","last_unstale":"2024-01-15T18:17:55.114156+0000","last_undegraded":"2024-01-15T18:17:55.114156+0000","last_fullsized":"2024-01-15T18:17:55.114156+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"117'76","reported_seq":108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:13.141663+0000","last_change":"2024-01-15T18:18:13.141663+0000","last_active":"2024-01-15T18:18:13.141663+0000","last_peered":"2024-01-15T18:18:13.141663+0000","last_clean":"2024-01-15T18:18:13.141663+0000","last_became_active":"2024-01-15T18:12:11.052655+0000","last_became_peered":"2024-01-15T18:12:11.052655+0000","last_unstale":"2024-01-15T18:18:13.141663+0000","last_undegraded":"2024-01-15T18:18:13.141663+0000","last_fullsized":"2024-01-15T18:18:13.141663+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'76","last_scrub_stamp":"2024-01-15T18:18:13.141496+0000","last_deep_scrub":"117'76","last_deep_scrub_stamp":"2024-01-15T18:18:13.141496+0000","last_clean_scrub_stamp":"2024-01-15T18:18:13.141496+0000","objects_scrubbed":22,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:46.765713+0000","scrub_duration":0.21211069099999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"117'49","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.216437+0000","last_change":"2024-01-15T18:18:10.216437+0000","last_active":"2024-01-15T18:18:10.216437+0000","last_peered":"2024-01-15T18:18:10.216437+0000","last_clean":"2024-01-15T18:18:10.216437+0000","last_became_active":"2024-01-15T18:12:11.053389+0000","last_became_peered":"2024-01-15T18:12:11.053389+0000","last_unstale":"2024-01-15T18:18:10.216437+0000","last_undegraded":"2024-01-15T18:18:10.216437+0000","last_fullsized":"2024-01-15T18:18:10.216437+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'49","last_scrub_stamp":"2024-01-15T18:18:10.216244+0000","last_deep_scrub":"117'49","last_deep_scrub_stamp":"2024-01-15T18:18:10.216244+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.216244+0000","objects_scrubbed":15,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:33.117273+0000","scrub_duration":0.16396555099999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"117'6","reported_seq":32,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:03.244761+0000","last_change":"2024-01-15T18:18:03.244761+0000","last_active":"2024-01-15T18:18:03.244761+0000","last_peered":"2024-01-15T18:18:03.244761+0000","last_clean":"2024-01-15T18:18:03.244761+0000","last_became_active":"2024-01-15T18:12:10.020810+0000","last_became_peered":"2024-01-15T18:12:10.020810+0000","last_unstale":"2024-01-15T18:18:03.244761+0000","last_undegraded":"2024-01-15T18:18:03.244761+0000","last_fullsized":"2024-01-15T18:18:03.244761+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'6","last_scrub_stamp":"2024-01-15T18:18:03.244524+0000","last_deep_scrub":"117'6","last_deep_scrub_stamp":"2024-01-15T18:18:03.244524+0000","last_clean_scrub_stamp":"2024-01-15T18:18:03.244524+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:14:16.865481+0000","scrub_duration":0.0076147139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"117'66","reported_seq":99,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.420458+0000","last_change":"2024-01-15T18:18:07.420458+0000","last_active":"2024-01-15T18:18:07.420458+0000","last_peered":"2024-01-15T18:18:07.420458+0000","last_clean":"2024-01-15T18:18:07.420458+0000","last_became_active":"2024-01-15T18:12:11.042427+0000","last_became_peered":"2024-01-15T18:12:11.042427+0000","last_unstale":"2024-01-15T18:18:07.420458+0000","last_undegraded":"2024-01-15T18:18:07.420458+0000","last_fullsized":"2024-01-15T18:18:07.420458+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'66","last_scrub_stamp":"2024-01-15T18:18:07.420124+0000","last_deep_scrub":"117'66","last_deep_scrub_stamp":"2024-01-15T18:18:07.420124+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.420124+0000","objects_scrubbed":22,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:28.299453+0000","scrub_duration":0.26970153499999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:06.548413+0000","last_change":"2024-01-15T18:18:06.548413+0000","last_active":"2024-01-15T18:18:06.548413+0000","last_peered":"2024-01-15T18:18:06.548413+0000","last_clean":"2024-01-15T18:18:06.548413+0000","last_became_active":"2024-01-15T18:12:10.022654+0000","last_became_peered":"2024-01-15T18:12:10.022654+0000","last_unstale":"2024-01-15T18:18:06.548413+0000","last_undegraded":"2024-01-15T18:18:06.548413+0000","last_fullsized":"2024-01-15T18:18:06.548413+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'1","last_scrub_stamp":"2024-01-15T18:18:06.548172+0000","last_deep_scrub":"114'1","last_deep_scrub_stamp":"2024-01-15T18:18:06.548172+0000","last_clean_scrub_stamp":"2024-01-15T18:18:06.548172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:40.711970+0000","scrub_duration":0.003727925,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'62","reported_seq":97,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.077838+0000","last_change":"2024-01-15T18:18:17.077904+0000","last_active":"2024-01-15T18:18:17.077838+0000","last_peered":"2024-01-15T18:18:17.077838+0000","last_clean":"2024-01-15T18:18:17.077838+0000","last_became_active":"2024-01-15T18:12:11.052812+0000","last_became_peered":"2024-01-15T18:12:11.052812+0000","last_unstale":"2024-01-15T18:18:17.077838+0000","last_undegraded":"2024-01-15T18:18:17.077838+0000","last_fullsized":"2024-01-15T18:18:17.077838+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'62","last_scrub_stamp":"2024-01-15T18:18:17.077727+0000","last_deep_scrub":"117'62","last_deep_scrub_stamp":"2024-01-15T18:18:17.077727+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.077727+0000","objects_scrubbed":19,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:22.148520+0000","scrub_duration":0.19122593199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"117'4","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.759072+0000","last_change":"2024-01-15T18:17:58.759128+0000","last_active":"2024-01-15T18:17:58.759072+0000","last_peered":"2024-01-15T18:17:58.759072+0000","last_clean":"2024-01-15T18:17:58.759072+0000","last_became_active":"2024-01-15T18:12:10.018670+0000","last_became_peered":"2024-01-15T18:12:10.018670+0000","last_unstale":"2024-01-15T18:17:58.759072+0000","last_undegraded":"2024-01-15T18:17:58.759072+0000","last_fullsized":"2024-01-15T18:17:58.759072+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'4","last_scrub_stamp":"2024-01-15T18:17:58.758892+0000","last_deep_scrub":"117'4","last_deep_scrub_stamp":"2024-01-15T18:17:58.758892+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.758892+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:09:01.602528+0000","scrub_duration":0.0068205260000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"117'56","reported_seq":83,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.364043+0000","last_change":"2024-01-15T18:12:11.059355+0000","last_active":"2024-01-15T18:17:39.364043+0000","last_peered":"2024-01-15T18:17:39.364043+0000","last_clean":"2024-01-15T18:17:39.364043+0000","last_became_active":"2024-01-15T18:12:11.057592+0000","last_became_peered":"2024-01-15T18:12:11.057592+0000","last_unstale":"2024-01-15T18:17:39.364043+0000","last_undegraded":"2024-01-15T18:17:39.364043+0000","last_fullsized":"2024-01-15T18:17:39.364043+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"117'3","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:57.800153+0000","last_change":"2024-01-15T18:17:57.800193+0000","last_active":"2024-01-15T18:17:57.800153+0000","last_peered":"2024-01-15T18:17:57.800153+0000","last_clean":"2024-01-15T18:17:57.800153+0000","last_became_active":"2024-01-15T18:12:10.020143+0000","last_became_peered":"2024-01-15T18:12:10.020143+0000","last_unstale":"2024-01-15T18:17:57.800153+0000","last_undegraded":"2024-01-15T18:17:57.800153+0000","last_fullsized":"2024-01-15T18:17:57.800153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'3","last_scrub_stamp":"2024-01-15T18:17:57.800057+0000","last_deep_scrub":"117'3","last_deep_scrub_stamp":"2024-01-15T18:17:57.800057+0000","last_clean_scrub_stamp":"2024-01-15T18:17:57.800057+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:23:19.671055+0000","scrub_duration":0.0055516410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"117'56","reported_seq":88,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:33.493481+0000","last_change":"2024-01-15T18:18:33.493543+0000","last_active":"2024-01-15T18:18:33.493481+0000","last_peered":"2024-01-15T18:18:33.493481+0000","last_clean":"2024-01-15T18:18:33.493481+0000","last_became_active":"2024-01-15T18:12:11.057471+0000","last_became_peered":"2024-01-15T18:12:11.057471+0000","last_unstale":"2024-01-15T18:18:33.493481+0000","last_undegraded":"2024-01-15T18:18:33.493481+0000","last_fullsized":"2024-01-15T18:18:33.493481+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'56","last_scrub_stamp":"2024-01-15T18:18:33.493371+0000","last_deep_scrub":"117'56","last_deep_scrub_stamp":"2024-01-15T18:18:33.493371+0000","last_clean_scrub_stamp":"2024-01-15T18:18:33.493371+0000","objects_scrubbed":17,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:13.195463+0000","scrub_duration":0.16716834899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:05.257137+0000","last_change":"2024-01-15T18:18:05.257194+0000","last_active":"2024-01-15T18:18:05.257137+0000","last_peered":"2024-01-15T18:18:05.257137+0000","last_clean":"2024-01-15T18:18:05.257137+0000","last_became_active":"2024-01-15T18:12:10.019818+0000","last_became_peered":"2024-01-15T18:12:10.019818+0000","last_unstale":"2024-01-15T18:18:05.257137+0000","last_undegraded":"2024-01-15T18:18:05.257137+0000","last_fullsized":"2024-01-15T18:18:05.257137+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'1","last_scrub_stamp":"2024-01-15T18:18:05.256946+0000","last_deep_scrub":"114'1","last_deep_scrub_stamp":"2024-01-15T18:18:05.256946+0000","last_clean_scrub_stamp":"2024-01-15T18:18:05.256946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:47:07.033483+0000","scrub_duration":0.0043509610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"117'73","reported_seq":110,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:13.209425+0000","last_change":"2024-01-15T18:18:13.209530+0000","last_active":"2024-01-15T18:18:13.209425+0000","last_peered":"2024-01-15T18:18:13.209425+0000","last_clean":"2024-01-15T18:18:13.209425+0000","last_became_active":"2024-01-15T18:12:11.050002+0000","last_became_peered":"2024-01-15T18:12:11.050002+0000","last_unstale":"2024-01-15T18:18:13.209425+0000","last_undegraded":"2024-01-15T18:18:13.209425+0000","last_fullsized":"2024-01-15T18:18:13.209425+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'73","last_scrub_stamp":"2024-01-15T18:18:13.209243+0000","last_deep_scrub":"117'73","last_deep_scrub_stamp":"2024-01-15T18:18:13.209243+0000","last_clean_scrub_stamp":"2024-01-15T18:18:13.209243+0000","objects_scrubbed":24,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:23:57.942383+0000","scrub_duration":0.27580295799999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.489503+0000","last_change":"2024-01-15T18:18:00.489523+0000","last_active":"2024-01-15T18:18:00.489503+0000","last_peered":"2024-01-15T18:18:00.489503+0000","last_clean":"2024-01-15T18:18:00.489503+0000","last_became_active":"2024-01-15T18:12:10.009676+0000","last_became_peered":"2024-01-15T18:12:10.009676+0000","last_unstale":"2024-01-15T18:18:00.489503+0000","last_undegraded":"2024-01-15T18:18:00.489503+0000","last_fullsized":"2024-01-15T18:18:00.489503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'1","last_scrub_stamp":"2024-01-15T18:18:00.489431+0000","last_deep_scrub":"114'1","last_deep_scrub_stamp":"2024-01-15T18:18:00.489431+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.489431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:33:48.954551+0000","scrub_duration":0.00316505,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"117'79","reported_seq":114,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:28.702871+0000","last_change":"2024-01-15T18:18:28.702924+0000","last_active":"2024-01-15T18:18:28.702871+0000","last_peered":"2024-01-15T18:18:28.702871+0000","last_clean":"2024-01-15T18:18:28.702871+0000","last_became_active":"2024-01-15T18:12:11.057640+0000","last_became_peered":"2024-01-15T18:12:11.057640+0000","last_unstale":"2024-01-15T18:18:28.702871+0000","last_undegraded":"2024-01-15T18:18:28.702871+0000","last_fullsized":"2024-01-15T18:18:28.702871+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'79","last_scrub_stamp":"2024-01-15T18:18:28.702706+0000","last_deep_scrub":"117'79","last_deep_scrub_stamp":"2024-01-15T18:18:28.702706+0000","last_clean_scrub_stamp":"2024-01-15T18:18:28.702706+0000","objects_scrubbed":26,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:39:40.150910+0000","scrub_duration":0.27044170200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857791,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":36865,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.801159+0000","last_change":"2024-01-15T18:17:58.801207+0000","last_active":"2024-01-15T18:17:58.801159+0000","last_peered":"2024-01-15T18:17:58.801159+0000","last_clean":"2024-01-15T18:17:58.801159+0000","last_became_active":"2024-01-15T18:12:10.002190+0000","last_became_peered":"2024-01-15T18:12:10.002190+0000","last_unstale":"2024-01-15T18:17:58.801159+0000","last_undegraded":"2024-01-15T18:17:58.801159+0000","last_fullsized":"2024-01-15T18:17:58.801159+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:58.800969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:58.800969+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.800969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:55.566839+0000","scrub_duration":0.0032888890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"117'59","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.379605+0000","last_change":"2024-01-15T18:18:17.379679+0000","last_active":"2024-01-15T18:18:17.379605+0000","last_peered":"2024-01-15T18:18:17.379605+0000","last_clean":"2024-01-15T18:18:17.379605+0000","last_became_active":"2024-01-15T18:12:11.046934+0000","last_became_peered":"2024-01-15T18:12:11.046934+0000","last_unstale":"2024-01-15T18:18:17.379605+0000","last_undegraded":"2024-01-15T18:18:17.379605+0000","last_fullsized":"2024-01-15T18:18:17.379605+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'59","last_scrub_stamp":"2024-01-15T18:18:17.379455+0000","last_deep_scrub":"117'59","last_deep_scrub_stamp":"2024-01-15T18:18:17.379455+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.379455+0000","objects_scrubbed":16,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:14.145325+0000","scrub_duration":0.19649055300000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:19.441143+0000","last_change":"2024-01-15T18:18:19.441198+0000","last_active":"2024-01-15T18:18:19.441143+0000","last_peered":"2024-01-15T18:18:19.441143+0000","last_clean":"2024-01-15T18:18:19.441143+0000","last_became_active":"2024-01-15T18:12:10.020577+0000","last_became_peered":"2024-01-15T18:12:10.020577+0000","last_unstale":"2024-01-15T18:18:19.441143+0000","last_undegraded":"2024-01-15T18:18:19.441143+0000","last_fullsized":"2024-01-15T18:18:19.441143+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:19.440940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:19.440940+0000","last_clean_scrub_stamp":"2024-01-15T18:18:19.440940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:44:56.813275+0000","scrub_duration":0.0035696320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"117'50","reported_seq":78,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.308905+0000","last_change":"2024-01-15T18:18:15.308980+0000","last_active":"2024-01-15T18:18:15.308905+0000","last_peered":"2024-01-15T18:18:15.308905+0000","last_clean":"2024-01-15T18:18:15.308905+0000","last_became_active":"2024-01-15T18:12:11.047704+0000","last_became_peered":"2024-01-15T18:12:11.047704+0000","last_unstale":"2024-01-15T18:18:15.308905+0000","last_undegraded":"2024-01-15T18:18:15.308905+0000","last_fullsized":"2024-01-15T18:18:15.308905+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'50","last_scrub_stamp":"2024-01-15T18:18:15.308693+0000","last_deep_scrub":"117'50","last_deep_scrub_stamp":"2024-01-15T18:18:15.308693+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.308693+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:38.450367+0000","scrub_duration":0.093927007000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"117'1","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.742613+0000","last_change":"2024-01-15T18:18:01.742668+0000","last_active":"2024-01-15T18:18:01.742613+0000","last_peered":"2024-01-15T18:18:01.742613+0000","last_clean":"2024-01-15T18:18:01.742613+0000","last_became_active":"2024-01-15T18:12:10.020028+0000","last_became_peered":"2024-01-15T18:12:10.020028+0000","last_unstale":"2024-01-15T18:18:01.742613+0000","last_undegraded":"2024-01-15T18:18:01.742613+0000","last_fullsized":"2024-01-15T18:18:01.742613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'1","last_scrub_stamp":"2024-01-15T18:18:01.742421+0000","last_deep_scrub":"117'1","last_deep_scrub_stamp":"2024-01-15T18:18:01.742421+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.742421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:19:38.888476+0000","scrub_duration":0.0037945140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"117'55","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.374401+0000","last_change":"2024-01-15T18:18:09.374484+0000","last_active":"2024-01-15T18:18:09.374401+0000","last_peered":"2024-01-15T18:18:09.374401+0000","last_clean":"2024-01-15T18:18:09.374401+0000","last_became_active":"2024-01-15T18:12:11.059393+0000","last_became_peered":"2024-01-15T18:12:11.059393+0000","last_unstale":"2024-01-15T18:18:09.374401+0000","last_undegraded":"2024-01-15T18:18:09.374401+0000","last_fullsized":"2024-01-15T18:18:09.374401+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'55","last_scrub_stamp":"2024-01-15T18:18:09.374248+0000","last_deep_scrub":"117'55","last_deep_scrub_stamp":"2024-01-15T18:18:09.374248+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.374248+0000","objects_scrubbed":18,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:44:26.424477+0000","scrub_duration":0.23458093399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.527582+0000","last_change":"2024-01-15T18:17:58.527616+0000","last_active":"2024-01-15T18:17:58.527582+0000","last_peered":"2024-01-15T18:17:58.527582+0000","last_clean":"2024-01-15T18:17:58.527582+0000","last_became_active":"2024-01-15T18:12:10.007549+0000","last_became_peered":"2024-01-15T18:12:10.007549+0000","last_unstale":"2024-01-15T18:17:58.527582+0000","last_undegraded":"2024-01-15T18:17:58.527582+0000","last_fullsized":"2024-01-15T18:17:58.527582+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:58.527475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:58.527475+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.527475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:32.463562+0000","scrub_duration":0.003638007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"117'43","reported_seq":70,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.538313+0000","last_change":"2024-01-15T18:12:11.031666+0000","last_active":"2024-01-15T18:17:39.538313+0000","last_peered":"2024-01-15T18:17:39.538313+0000","last_clean":"2024-01-15T18:17:39.538313+0000","last_became_active":"2024-01-15T18:12:11.030958+0000","last_became_peered":"2024-01-15T18:12:11.030958+0000","last_unstale":"2024-01-15T18:17:39.538313+0000","last_undegraded":"2024-01-15T18:17:39.538313+0000","last_fullsized":"2024-01-15T18:17:39.538313+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"117'14","reported_seq":39,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:14.392225+0000","last_change":"2024-01-15T18:18:14.392289+0000","last_active":"2024-01-15T18:18:14.392225+0000","last_peered":"2024-01-15T18:18:14.392225+0000","last_clean":"2024-01-15T18:18:14.392225+0000","last_became_active":"2024-01-15T18:12:10.024120+0000","last_became_peered":"2024-01-15T18:12:10.024120+0000","last_unstale":"2024-01-15T18:18:14.392225+0000","last_undegraded":"2024-01-15T18:18:14.392225+0000","last_fullsized":"2024-01-15T18:18:14.392225+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'14","last_scrub_stamp":"2024-01-15T18:18:14.391949+0000","last_deep_scrub":"117'14","last_deep_scrub_stamp":"2024-01-15T18:18:14.391949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:14.391949+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:31:42.989864+0000","scrub_duration":0.0085194199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"117'63","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:21.473343+0000","last_change":"2024-01-15T18:18:21.473422+0000","last_active":"2024-01-15T18:18:21.473343+0000","last_peered":"2024-01-15T18:18:21.473343+0000","last_clean":"2024-01-15T18:18:21.473343+0000","last_became_active":"2024-01-15T18:12:11.055290+0000","last_became_peered":"2024-01-15T18:12:11.055290+0000","last_unstale":"2024-01-15T18:18:21.473343+0000","last_undegraded":"2024-01-15T18:18:21.473343+0000","last_fullsized":"2024-01-15T18:18:21.473343+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:21.473208+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:21.473208+0000","last_clean_scrub_stamp":"2024-01-15T18:18:21.473208+0000","objects_scrubbed":20,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:26.591651+0000","scrub_duration":0.266702363,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"117'188","reported_seq":213,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:16.445379+0000","last_change":"2024-01-15T18:18:16.445450+0000","last_active":"2024-01-15T18:18:16.445379+0000","last_peered":"2024-01-15T18:18:16.445379+0000","last_clean":"2024-01-15T18:18:16.445379+0000","last_became_active":"2024-01-15T18:12:10.021497+0000","last_became_peered":"2024-01-15T18:12:10.021497+0000","last_unstale":"2024-01-15T18:18:16.445379+0000","last_undegraded":"2024-01-15T18:18:16.445379+0000","last_fullsized":"2024-01-15T18:18:16.445379+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'188","last_scrub_stamp":"2024-01-15T18:18:16.445190+0000","last_deep_scrub":"117'188","last_deep_scrub_stamp":"2024-01-15T18:18:16.445190+0000","last_clean_scrub_stamp":"2024-01-15T18:18:16.445190+0000","objects_scrubbed":4,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:09:41.548591+0000","scrub_duration":0.015015664,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639451,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":185,"num_write_kb":717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"117'44","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.543606+0000","last_change":"2024-01-15T18:12:11.046101+0000","last_active":"2024-01-15T18:17:39.543606+0000","last_peered":"2024-01-15T18:17:39.543606+0000","last_clean":"2024-01-15T18:17:39.543606+0000","last_became_active":"2024-01-15T18:12:11.045774+0000","last_became_peered":"2024-01-15T18:12:11.045774+0000","last_unstale":"2024-01-15T18:17:39.543606+0000","last_undegraded":"2024-01-15T18:17:39.543606+0000","last_fullsized":"2024-01-15T18:17:39.543606+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"117'4","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.064488+0000","last_change":"2024-01-15T18:18:07.064553+0000","last_active":"2024-01-15T18:18:07.064488+0000","last_peered":"2024-01-15T18:18:07.064488+0000","last_clean":"2024-01-15T18:18:07.064488+0000","last_became_active":"2024-01-15T18:12:10.016618+0000","last_became_peered":"2024-01-15T18:12:10.016618+0000","last_unstale":"2024-01-15T18:18:07.064488+0000","last_undegraded":"2024-01-15T18:18:07.064488+0000","last_fullsized":"2024-01-15T18:18:07.064488+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'4","last_scrub_stamp":"2024-01-15T18:18:07.064304+0000","last_deep_scrub":"117'4","last_deep_scrub_stamp":"2024-01-15T18:18:07.064304+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.064304+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:55.108425+0000","scrub_duration":0.0089004340000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"117'73","reported_seq":102,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:36.738982+0000","last_change":"2024-01-15T18:18:36.739074+0000","last_active":"2024-01-15T18:18:36.738982+0000","last_peered":"2024-01-15T18:18:36.738982+0000","last_clean":"2024-01-15T18:18:36.738982+0000","last_became_active":"2024-01-15T18:12:11.050928+0000","last_became_peered":"2024-01-15T18:12:11.050928+0000","last_unstale":"2024-01-15T18:18:36.738982+0000","last_undegraded":"2024-01-15T18:18:36.738982+0000","last_fullsized":"2024-01-15T18:18:36.738982+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'73","last_scrub_stamp":"2024-01-15T18:18:36.738814+0000","last_deep_scrub":"117'73","last_deep_scrub_stamp":"2024-01-15T18:18:36.738814+0000","last_clean_scrub_stamp":"2024-01-15T18:18:36.738814+0000","objects_scrubbed":21,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:42.767283+0000","scrub_duration":0.23757112899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:04.129497+0000","last_change":"2024-01-15T18:18:04.129593+0000","last_active":"2024-01-15T18:18:04.129497+0000","last_peered":"2024-01-15T18:18:04.129497+0000","last_clean":"2024-01-15T18:18:04.129497+0000","last_became_active":"2024-01-15T18:12:10.015578+0000","last_became_peered":"2024-01-15T18:12:10.015578+0000","last_unstale":"2024-01-15T18:18:04.129497+0000","last_undegraded":"2024-01-15T18:18:04.129497+0000","last_fullsized":"2024-01-15T18:18:04.129497+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:04.129272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:04.129272+0000","last_clean_scrub_stamp":"2024-01-15T18:18:04.129272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:09.247715+0000","scrub_duration":0.003855626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"117'68","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:03.667643+0000","last_change":"2024-01-15T18:18:03.667703+0000","last_active":"2024-01-15T18:18:03.667643+0000","last_peered":"2024-01-15T18:18:03.667643+0000","last_clean":"2024-01-15T18:18:03.667643+0000","last_became_active":"2024-01-15T18:12:11.028865+0000","last_became_peered":"2024-01-15T18:12:11.028865+0000","last_unstale":"2024-01-15T18:18:03.667643+0000","last_undegraded":"2024-01-15T18:18:03.667643+0000","last_fullsized":"2024-01-15T18:18:03.667643+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'68","last_scrub_stamp":"2024-01-15T18:18:03.667447+0000","last_deep_scrub":"117'68","last_deep_scrub_stamp":"2024-01-15T18:18:03.667447+0000","last_clean_scrub_stamp":"2024-01-15T18:18:03.667447+0000","objects_scrubbed":25,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:28:27.833994+0000","scrub_duration":0.20722174700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663556,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"117'12","reported_seq":37,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:56.807714+0000","last_change":"2024-01-15T18:17:56.807742+0000","last_active":"2024-01-15T18:17:56.807714+0000","last_peered":"2024-01-15T18:17:56.807714+0000","last_clean":"2024-01-15T18:17:56.807714+0000","last_became_active":"2024-01-15T18:12:10.022398+0000","last_became_peered":"2024-01-15T18:12:10.022398+0000","last_unstale":"2024-01-15T18:17:56.807714+0000","last_undegraded":"2024-01-15T18:17:56.807714+0000","last_fullsized":"2024-01-15T18:17:56.807714+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'12","last_scrub_stamp":"2024-01-15T18:17:56.807649+0000","last_deep_scrub":"117'12","last_deep_scrub_stamp":"2024-01-15T18:17:56.807649+0000","last_clean_scrub_stamp":"2024-01-15T18:17:56.807649+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:43.916577+0000","scrub_duration":0.008646964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"117'34","reported_seq":62,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.960812+0000","last_change":"2024-01-15T18:18:10.960886+0000","last_active":"2024-01-15T18:18:10.960812+0000","last_peered":"2024-01-15T18:18:10.960812+0000","last_clean":"2024-01-15T18:18:10.960812+0000","last_became_active":"2024-01-15T18:12:11.043735+0000","last_became_peered":"2024-01-15T18:12:11.043735+0000","last_unstale":"2024-01-15T18:18:10.960812+0000","last_undegraded":"2024-01-15T18:18:10.960812+0000","last_fullsized":"2024-01-15T18:18:10.960812+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'34","last_scrub_stamp":"2024-01-15T18:18:10.960638+0000","last_deep_scrub":"117'34","last_deep_scrub_stamp":"2024-01-15T18:18:10.960638+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.960638+0000","objects_scrubbed":10,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:14:24.581595+0000","scrub_duration":0.105112926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"117'2","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:20.431912+0000","last_change":"2024-01-15T18:18:20.431973+0000","last_active":"2024-01-15T18:18:20.431912+0000","last_peered":"2024-01-15T18:18:20.431912+0000","last_clean":"2024-01-15T18:18:20.431912+0000","last_became_active":"2024-01-15T18:12:10.026730+0000","last_became_peered":"2024-01-15T18:12:10.026730+0000","last_unstale":"2024-01-15T18:18:20.431912+0000","last_undegraded":"2024-01-15T18:18:20.431912+0000","last_fullsized":"2024-01-15T18:18:20.431912+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'2","last_scrub_stamp":"2024-01-15T18:18:20.431712+0000","last_deep_scrub":"117'2","last_deep_scrub_stamp":"2024-01-15T18:18:20.431712+0000","last_clean_scrub_stamp":"2024-01-15T18:18:20.431712+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:45:32.145232+0000","scrub_duration":0.0084741669999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"117'53","reported_seq":88,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:21.086410+0000","last_change":"2024-01-15T18:18:21.086448+0000","last_active":"2024-01-15T18:18:21.086410+0000","last_peered":"2024-01-15T18:18:21.086410+0000","last_clean":"2024-01-15T18:18:21.086410+0000","last_became_active":"2024-01-15T18:12:11.061002+0000","last_became_peered":"2024-01-15T18:12:11.061002+0000","last_unstale":"2024-01-15T18:18:21.086410+0000","last_undegraded":"2024-01-15T18:18:21.086410+0000","last_fullsized":"2024-01-15T18:18:21.086410+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'53","last_scrub_stamp":"2024-01-15T18:18:21.086323+0000","last_deep_scrub":"117'53","last_deep_scrub_stamp":"2024-01-15T18:18:21.086323+0000","last_clean_scrub_stamp":"2024-01-15T18:18:21.086323+0000","objects_scrubbed":17,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:51.309071+0000","scrub_duration":0.12936230400000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"117'3","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.859053+0000","last_change":"2024-01-15T18:18:07.859110+0000","last_active":"2024-01-15T18:18:07.859053+0000","last_peered":"2024-01-15T18:18:07.859053+0000","last_clean":"2024-01-15T18:18:07.859053+0000","last_became_active":"2024-01-15T18:12:10.019706+0000","last_became_peered":"2024-01-15T18:12:10.019706+0000","last_unstale":"2024-01-15T18:18:07.859053+0000","last_undegraded":"2024-01-15T18:18:07.859053+0000","last_fullsized":"2024-01-15T18:18:07.859053+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'3","last_scrub_stamp":"2024-01-15T18:18:07.858872+0000","last_deep_scrub":"117'3","last_deep_scrub_stamp":"2024-01-15T18:18:07.858872+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.858872+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:12:03.175146+0000","scrub_duration":0.0077351470000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"117'62","reported_seq":97,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:23.538400+0000","last_change":"2024-01-15T18:18:23.538485+0000","last_active":"2024-01-15T18:18:23.538400+0000","last_peered":"2024-01-15T18:18:23.538400+0000","last_clean":"2024-01-15T18:18:23.538400+0000","last_became_active":"2024-01-15T18:12:11.060154+0000","last_became_peered":"2024-01-15T18:12:11.060154+0000","last_unstale":"2024-01-15T18:18:23.538400+0000","last_undegraded":"2024-01-15T18:18:23.538400+0000","last_fullsized":"2024-01-15T18:18:23.538400+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'62","last_scrub_stamp":"2024-01-15T18:18:23.538243+0000","last_deep_scrub":"117'62","last_deep_scrub_stamp":"2024-01-15T18:18:23.538243+0000","last_clean_scrub_stamp":"2024-01-15T18:18:23.538243+0000","objects_scrubbed":18,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:11:51.960622+0000","scrub_duration":0.202197823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"117'2","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.161684+0000","last_change":"2024-01-15T18:18:01.161747+0000","last_active":"2024-01-15T18:18:01.161684+0000","last_peered":"2024-01-15T18:18:01.161684+0000","last_clean":"2024-01-15T18:18:01.161684+0000","last_became_active":"2024-01-15T18:12:10.002957+0000","last_became_peered":"2024-01-15T18:12:10.002957+0000","last_unstale":"2024-01-15T18:18:01.161684+0000","last_undegraded":"2024-01-15T18:18:01.161684+0000","last_fullsized":"2024-01-15T18:18:01.161684+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'2","last_scrub_stamp":"2024-01-15T18:18:01.161505+0000","last_deep_scrub":"117'2","last_deep_scrub_stamp":"2024-01-15T18:18:01.161505+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.161505+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:01.536885+0000","scrub_duration":0.0071468110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"117'41","reported_seq":69,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:39.364689+0000","last_change":"2024-01-15T18:12:11.059234+0000","last_active":"2024-01-15T18:17:39.364689+0000","last_peered":"2024-01-15T18:17:39.364689+0000","last_clean":"2024-01-15T18:17:39.364689+0000","last_became_active":"2024-01-15T18:12:11.058945+0000","last_became_peered":"2024-01-15T18:12:11.058945+0000","last_unstale":"2024-01-15T18:17:39.364689+0000","last_undegraded":"2024-01-15T18:17:39.364689+0000","last_fullsized":"2024-01-15T18:17:39.364689+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:06.288831+0000","last_change":"2024-01-15T18:18:06.288945+0000","last_active":"2024-01-15T18:18:06.288831+0000","last_peered":"2024-01-15T18:18:06.288831+0000","last_clean":"2024-01-15T18:18:06.288831+0000","last_became_active":"2024-01-15T18:12:10.028284+0000","last_became_peered":"2024-01-15T18:12:10.028284+0000","last_unstale":"2024-01-15T18:18:06.288831+0000","last_undegraded":"2024-01-15T18:18:06.288831+0000","last_fullsized":"2024-01-15T18:18:06.288831+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:06.288498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:06.288498+0000","last_clean_scrub_stamp":"2024-01-15T18:18:06.288498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:21:19.512303+0000","scrub_duration":0.0041979610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"117'75","reported_seq":108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.815411+0000","last_change":"2024-01-15T18:18:15.815534+0000","last_active":"2024-01-15T18:18:15.815411+0000","last_peered":"2024-01-15T18:18:15.815411+0000","last_clean":"2024-01-15T18:18:15.815411+0000","last_became_active":"2024-01-15T18:12:11.051403+0000","last_became_peered":"2024-01-15T18:12:11.051403+0000","last_unstale":"2024-01-15T18:18:15.815411+0000","last_undegraded":"2024-01-15T18:18:15.815411+0000","last_fullsized":"2024-01-15T18:18:15.815411+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'75","last_scrub_stamp":"2024-01-15T18:18:15.815153+0000","last_deep_scrub":"117'75","last_deep_scrub_stamp":"2024-01-15T18:18:15.815153+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.815153+0000","objects_scrubbed":30,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:09.886495+0000","scrub_duration":0.40068222799999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"117'4","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:04.093090+0000","last_change":"2024-01-15T18:18:04.093163+0000","last_active":"2024-01-15T18:18:04.093090+0000","last_peered":"2024-01-15T18:18:04.093090+0000","last_clean":"2024-01-15T18:18:04.093090+0000","last_became_active":"2024-01-15T18:12:10.021344+0000","last_became_peered":"2024-01-15T18:12:10.021344+0000","last_unstale":"2024-01-15T18:18:04.093090+0000","last_undegraded":"2024-01-15T18:18:04.093090+0000","last_fullsized":"2024-01-15T18:18:04.093090+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'4","last_scrub_stamp":"2024-01-15T18:18:04.092822+0000","last_deep_scrub":"117'4","last_deep_scrub_stamp":"2024-01-15T18:18:04.092822+0000","last_clean_scrub_stamp":"2024-01-15T18:18:04.092822+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:51.201750+0000","scrub_duration":0.0068652299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"117'43","reported_seq":66,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:38.951562+0000","last_change":"2024-01-15T18:12:11.059848+0000","last_active":"2024-01-15T18:17:38.951562+0000","last_peered":"2024-01-15T18:17:38.951562+0000","last_clean":"2024-01-15T18:17:38.951562+0000","last_became_active":"2024-01-15T18:12:11.059311+0000","last_became_peered":"2024-01-15T18:12:11.059311+0000","last_unstale":"2024-01-15T18:17:38.951562+0000","last_undegraded":"2024-01-15T18:17:38.951562+0000","last_fullsized":"2024-01-15T18:17:38.951562+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.698869+0000","last_change":"2024-01-15T18:18:00.698919+0000","last_active":"2024-01-15T18:18:00.698869+0000","last_peered":"2024-01-15T18:18:00.698869+0000","last_clean":"2024-01-15T18:18:00.698869+0000","last_became_active":"2024-01-15T18:12:10.017411+0000","last_became_peered":"2024-01-15T18:12:10.017411+0000","last_unstale":"2024-01-15T18:18:00.698869+0000","last_undegraded":"2024-01-15T18:18:00.698869+0000","last_fullsized":"2024-01-15T18:18:00.698869+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:00.698524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:00.698524+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.698524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:05:24.402242+0000","scrub_duration":0.0031317810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"117'61","reported_seq":90,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:29.517193+0000","last_change":"2024-01-15T18:18:29.517272+0000","last_active":"2024-01-15T18:18:29.517193+0000","last_peered":"2024-01-15T18:18:29.517193+0000","last_clean":"2024-01-15T18:18:29.517193+0000","last_became_active":"2024-01-15T18:12:11.056311+0000","last_became_peered":"2024-01-15T18:12:11.056311+0000","last_unstale":"2024-01-15T18:18:29.517193+0000","last_undegraded":"2024-01-15T18:18:29.517193+0000","last_fullsized":"2024-01-15T18:18:29.517193+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'61","last_scrub_stamp":"2024-01-15T18:18:29.517021+0000","last_deep_scrub":"117'61","last_deep_scrub_stamp":"2024-01-15T18:18:29.517021+0000","last_clean_scrub_stamp":"2024-01-15T18:18:29.517021+0000","objects_scrubbed":17,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:41.064706+0000","scrub_duration":0.192502231,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.185102+0000","last_change":"2024-01-15T18:17:58.185126+0000","last_active":"2024-01-15T18:17:58.185102+0000","last_peered":"2024-01-15T18:17:58.185102+0000","last_clean":"2024-01-15T18:17:58.185102+0000","last_became_active":"2024-01-15T18:12:10.022687+0000","last_became_peered":"2024-01-15T18:12:10.022687+0000","last_unstale":"2024-01-15T18:17:58.185102+0000","last_undegraded":"2024-01-15T18:17:58.185102+0000","last_fullsized":"2024-01-15T18:17:58.185102+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:58.185008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:58.185008+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.185008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:00:35.863748+0000","scrub_duration":0.0025740260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"117'59","reported_seq":90,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:27.774704+0000","last_change":"2024-01-15T18:18:27.774812+0000","last_active":"2024-01-15T18:18:27.774704+0000","last_peered":"2024-01-15T18:18:27.774704+0000","last_clean":"2024-01-15T18:18:27.774704+0000","last_became_active":"2024-01-15T18:12:11.061830+0000","last_became_peered":"2024-01-15T18:12:11.061830+0000","last_unstale":"2024-01-15T18:18:27.774704+0000","last_undegraded":"2024-01-15T18:18:27.774704+0000","last_fullsized":"2024-01-15T18:18:27.774704+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'59","last_scrub_stamp":"2024-01-15T18:18:27.774550+0000","last_deep_scrub":"117'59","last_deep_scrub_stamp":"2024-01-15T18:18:27.774550+0000","last_clean_scrub_stamp":"2024-01-15T18:18:27.774550+0000","objects_scrubbed":17,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:32.866216+0000","scrub_duration":0.17066413899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109124,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.447947+0000","last_change":"2024-01-15T18:18:17.448002+0000","last_active":"2024-01-15T18:18:17.447947+0000","last_peered":"2024-01-15T18:18:17.447947+0000","last_clean":"2024-01-15T18:18:17.447947+0000","last_became_active":"2024-01-15T18:12:10.024554+0000","last_became_peered":"2024-01-15T18:12:10.024554+0000","last_unstale":"2024-01-15T18:18:17.447947+0000","last_undegraded":"2024-01-15T18:18:17.447947+0000","last_fullsized":"2024-01-15T18:18:17.447947+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:17.447735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:17.447735+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.447735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:29:15.174266+0000","scrub_duration":0.0035608710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"117'47","reported_seq":75,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:08.292548+0000","last_change":"2024-01-15T18:18:08.292673+0000","last_active":"2024-01-15T18:18:08.292548+0000","last_peered":"2024-01-15T18:18:08.292548+0000","last_clean":"2024-01-15T18:18:08.292548+0000","last_became_active":"2024-01-15T18:12:11.050747+0000","last_became_peered":"2024-01-15T18:12:11.050747+0000","last_unstale":"2024-01-15T18:18:08.292548+0000","last_undegraded":"2024-01-15T18:18:08.292548+0000","last_fullsized":"2024-01-15T18:18:08.292548+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'47","last_scrub_stamp":"2024-01-15T18:18:08.292356+0000","last_deep_scrub":"117'47","last_deep_scrub_stamp":"2024-01-15T18:18:08.292356+0000","last_clean_scrub_stamp":"2024-01-15T18:18:08.292356+0000","objects_scrubbed":13,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:42.792622+0000","scrub_duration":0.148150529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"117'3","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.477391+0000","last_change":"2024-01-15T18:18:01.477457+0000","last_active":"2024-01-15T18:18:01.477391+0000","last_peered":"2024-01-15T18:18:01.477391+0000","last_clean":"2024-01-15T18:18:01.477391+0000","last_became_active":"2024-01-15T18:12:10.025356+0000","last_became_peered":"2024-01-15T18:12:10.025356+0000","last_unstale":"2024-01-15T18:18:01.477391+0000","last_undegraded":"2024-01-15T18:18:01.477391+0000","last_fullsized":"2024-01-15T18:18:01.477391+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'3","last_scrub_stamp":"2024-01-15T18:18:01.477202+0000","last_deep_scrub":"117'3","last_deep_scrub_stamp":"2024-01-15T18:18:01.477202+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.477202+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:59.723407+0000","scrub_duration":0.0075668130000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"117'63","reported_seq":99,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:35.601157+0000","last_change":"2024-01-15T18:18:35.601329+0000","last_active":"2024-01-15T18:18:35.601157+0000","last_peered":"2024-01-15T18:18:35.601157+0000","last_clean":"2024-01-15T18:18:35.601157+0000","last_became_active":"2024-01-15T18:12:11.050116+0000","last_became_peered":"2024-01-15T18:12:11.050116+0000","last_unstale":"2024-01-15T18:18:35.601157+0000","last_undegraded":"2024-01-15T18:18:35.601157+0000","last_fullsized":"2024-01-15T18:18:35.601157+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:35.600822+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:35.600822+0000","last_clean_scrub_stamp":"2024-01-15T18:18:35.600822+0000","objects_scrubbed":22,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:39.296272+0000","scrub_duration":0.31021504900000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"117'18","reported_seq":43,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:02.206701+0000","last_change":"2024-01-15T18:18:02.206754+0000","last_active":"2024-01-15T18:18:02.206701+0000","last_peered":"2024-01-15T18:18:02.206701+0000","last_clean":"2024-01-15T18:18:02.206701+0000","last_became_active":"2024-01-15T18:12:10.020497+0000","last_became_peered":"2024-01-15T18:12:10.020497+0000","last_unstale":"2024-01-15T18:18:02.206701+0000","last_undegraded":"2024-01-15T18:18:02.206701+0000","last_fullsized":"2024-01-15T18:18:02.206701+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'18","last_scrub_stamp":"2024-01-15T18:18:02.206517+0000","last_deep_scrub":"117'18","last_deep_scrub_stamp":"2024-01-15T18:18:02.206517+0000","last_clean_scrub_stamp":"2024-01-15T18:18:02.206517+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:40:05.823508+0000","scrub_duration":0.0043064160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":503,"num_omap_keys":10,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"117'70","reported_seq":109,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.709777+0000","last_change":"2024-01-15T18:18:09.709777+0000","last_active":"2024-01-15T18:18:09.709777+0000","last_peered":"2024-01-15T18:18:09.709777+0000","last_clean":"2024-01-15T18:18:09.709777+0000","last_became_active":"2024-01-15T18:12:11.050810+0000","last_became_peered":"2024-01-15T18:12:11.050810+0000","last_unstale":"2024-01-15T18:18:09.709777+0000","last_undegraded":"2024-01-15T18:18:09.709777+0000","last_fullsized":"2024-01-15T18:18:09.709777+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'70","last_scrub_stamp":"2024-01-15T18:18:09.709451+0000","last_deep_scrub":"117'70","last_deep_scrub_stamp":"2024-01-15T18:18:09.709451+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.709451+0000","objects_scrubbed":24,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:33.702809+0000","scrub_duration":0.24709842800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:12.368166+0000","last_change":"2024-01-15T18:18:12.368166+0000","last_active":"2024-01-15T18:18:12.368166+0000","last_peered":"2024-01-15T18:18:12.368166+0000","last_clean":"2024-01-15T18:18:12.368166+0000","last_became_active":"2024-01-15T18:12:10.024524+0000","last_became_peered":"2024-01-15T18:12:10.024524+0000","last_unstale":"2024-01-15T18:18:12.368166+0000","last_undegraded":"2024-01-15T18:18:12.368166+0000","last_fullsized":"2024-01-15T18:18:12.368166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:12.367669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:12.367669+0000","last_clean_scrub_stamp":"2024-01-15T18:18:12.367669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:30:45.736811+0000","scrub_duration":0.0043195339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"117'58","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.345686+0000","last_change":"2024-01-15T18:18:10.345686+0000","last_active":"2024-01-15T18:18:10.345686+0000","last_peered":"2024-01-15T18:18:10.345686+0000","last_clean":"2024-01-15T18:18:10.345686+0000","last_became_active":"2024-01-15T18:12:11.061454+0000","last_became_peered":"2024-01-15T18:12:11.061454+0000","last_unstale":"2024-01-15T18:18:10.345686+0000","last_undegraded":"2024-01-15T18:18:10.345686+0000","last_fullsized":"2024-01-15T18:18:10.345686+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'58","last_scrub_stamp":"2024-01-15T18:18:10.345394+0000","last_deep_scrub":"117'58","last_deep_scrub_stamp":"2024-01-15T18:18:10.345394+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.345394+0000","objects_scrubbed":19,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:23:55.078534+0000","scrub_duration":0.209955119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:59.502129+0000","last_change":"2024-01-15T18:17:59.502129+0000","last_active":"2024-01-15T18:17:59.502129+0000","last_peered":"2024-01-15T18:17:59.502129+0000","last_clean":"2024-01-15T18:17:59.502129+0000","last_became_active":"2024-01-15T18:12:10.018057+0000","last_became_peered":"2024-01-15T18:12:10.018057+0000","last_unstale":"2024-01-15T18:17:59.502129+0000","last_undegraded":"2024-01-15T18:17:59.502129+0000","last_fullsized":"2024-01-15T18:17:59.502129+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:59.501895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:59.501895+0000","last_clean_scrub_stamp":"2024-01-15T18:17:59.501895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:07.865255+0000","scrub_duration":0.0038048180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"117'60","reported_seq":92,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:03.094301+0000","last_change":"2024-01-15T18:18:03.094301+0000","last_active":"2024-01-15T18:18:03.094301+0000","last_peered":"2024-01-15T18:18:03.094301+0000","last_clean":"2024-01-15T18:18:03.094301+0000","last_became_active":"2024-01-15T18:12:11.039586+0000","last_became_peered":"2024-01-15T18:12:11.039586+0000","last_unstale":"2024-01-15T18:18:03.094301+0000","last_undegraded":"2024-01-15T18:18:03.094301+0000","last_fullsized":"2024-01-15T18:18:03.094301+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'60","last_scrub_stamp":"2024-01-15T18:18:03.093993+0000","last_deep_scrub":"117'60","last_deep_scrub_stamp":"2024-01-15T18:18:03.093993+0000","last_clean_scrub_stamp":"2024-01-15T18:18:03.093993+0000","objects_scrubbed":16,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:14:52.828664+0000","scrub_duration":0.201843208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.271104+0000","last_change":"2024-01-15T18:18:00.271104+0000","last_active":"2024-01-15T18:18:00.271104+0000","last_peered":"2024-01-15T18:18:00.271104+0000","last_clean":"2024-01-15T18:18:00.271104+0000","last_became_active":"2024-01-15T18:12:10.019424+0000","last_became_peered":"2024-01-15T18:12:10.019424+0000","last_unstale":"2024-01-15T18:18:00.271104+0000","last_undegraded":"2024-01-15T18:18:00.271104+0000","last_fullsized":"2024-01-15T18:18:00.271104+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:00.270864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:00.270864+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.270864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:50.723379+0000","scrub_duration":0.0035368880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"117'82","reported_seq":116,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.251086+0000","last_change":"2024-01-15T18:18:09.251086+0000","last_active":"2024-01-15T18:18:09.251086+0000","last_peered":"2024-01-15T18:18:09.251086+0000","last_clean":"2024-01-15T18:18:09.251086+0000","last_became_active":"2024-01-15T18:12:11.035238+0000","last_became_peered":"2024-01-15T18:12:11.035238+0000","last_unstale":"2024-01-15T18:18:09.251086+0000","last_undegraded":"2024-01-15T18:18:09.251086+0000","last_fullsized":"2024-01-15T18:18:09.251086+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'82","last_scrub_stamp":"2024-01-15T18:18:09.250811+0000","last_deep_scrub":"117'82","last_deep_scrub_stamp":"2024-01-15T18:18:09.250811+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.250811+0000","objects_scrubbed":29,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:19.387926+0000","scrub_duration":0.33453240099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.884506+0000","last_change":"2024-01-15T18:18:00.884506+0000","last_active":"2024-01-15T18:18:00.884506+0000","last_peered":"2024-01-15T18:18:00.884506+0000","last_clean":"2024-01-15T18:18:00.884506+0000","last_became_active":"2024-01-15T18:12:10.017190+0000","last_became_peered":"2024-01-15T18:12:10.017190+0000","last_unstale":"2024-01-15T18:18:00.884506+0000","last_undegraded":"2024-01-15T18:18:00.884506+0000","last_fullsized":"2024-01-15T18:18:00.884506+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:00.884268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:00.884268+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.884268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:38.405683+0000","scrub_duration":0.003782021,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"117'72","reported_seq":106,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:31.659694+0000","last_change":"2024-01-15T18:18:31.659742+0000","last_active":"2024-01-15T18:18:31.659694+0000","last_peered":"2024-01-15T18:18:31.659694+0000","last_clean":"2024-01-15T18:18:31.659694+0000","last_became_active":"2024-01-15T18:12:11.056111+0000","last_became_peered":"2024-01-15T18:12:11.056111+0000","last_unstale":"2024-01-15T18:18:31.659694+0000","last_undegraded":"2024-01-15T18:18:31.659694+0000","last_fullsized":"2024-01-15T18:18:31.659694+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'72","last_scrub_stamp":"2024-01-15T18:18:31.659564+0000","last_deep_scrub":"117'72","last_deep_scrub_stamp":"2024-01-15T18:18:31.659564+0000","last_clean_scrub_stamp":"2024-01-15T18:18:31.659564+0000","objects_scrubbed":26,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:37.299332+0000","scrub_duration":0.32621962500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"117'11","reported_seq":46,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:02.775960+0000","last_change":"2024-01-15T18:18:02.775960+0000","last_active":"2024-01-15T18:18:02.775960+0000","last_peered":"2024-01-15T18:18:02.775960+0000","last_clean":"2024-01-15T18:18:02.775960+0000","last_became_active":"2024-01-15T18:12:10.020890+0000","last_became_peered":"2024-01-15T18:12:10.020890+0000","last_unstale":"2024-01-15T18:18:02.775960+0000","last_undegraded":"2024-01-15T18:18:02.775960+0000","last_fullsized":"2024-01-15T18:18:02.775960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'11","last_scrub_stamp":"2024-01-15T18:18:02.775685+0000","last_deep_scrub":"117'11","last_deep_scrub_stamp":"2024-01-15T18:18:02.775685+0000","last_clean_scrub_stamp":"2024-01-15T18:18:02.775685+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:04.277157+0000","scrub_duration":0.0073569229999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"117'39","reported_seq":64,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:55.113795+0000","last_change":"2024-01-15T18:12:11.059132+0000","last_active":"2024-01-15T18:17:55.113795+0000","last_peered":"2024-01-15T18:17:55.113795+0000","last_clean":"2024-01-15T18:17:55.113795+0000","last_became_active":"2024-01-15T18:12:11.057943+0000","last_became_peered":"2024-01-15T18:12:11.057943+0000","last_unstale":"2024-01-15T18:17:55.113795+0000","last_undegraded":"2024-01-15T18:17:55.113795+0000","last_fullsized":"2024-01-15T18:17:55.113795+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:12:09.983156+0000","last_clean_scrub_stamp":"2024-01-15T18:12:09.983156+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:11.404503+0000","last_change":"2024-01-15T18:18:11.404503+0000","last_active":"2024-01-15T18:18:11.404503+0000","last_peered":"2024-01-15T18:18:11.404503+0000","last_clean":"2024-01-15T18:18:11.404503+0000","last_became_active":"2024-01-15T18:12:10.009870+0000","last_became_peered":"2024-01-15T18:12:10.009870+0000","last_unstale":"2024-01-15T18:18:11.404503+0000","last_undegraded":"2024-01-15T18:18:11.404503+0000","last_fullsized":"2024-01-15T18:18:11.404503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:11.404131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:11.404131+0000","last_clean_scrub_stamp":"2024-01-15T18:18:11.404131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:34:10.272987+0000","scrub_duration":0.0037587990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"117'45","reported_seq":77,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:31.743706+0000","last_change":"2024-01-15T18:18:31.743814+0000","last_active":"2024-01-15T18:18:31.743706+0000","last_peered":"2024-01-15T18:18:31.743706+0000","last_clean":"2024-01-15T18:18:31.743706+0000","last_became_active":"2024-01-15T18:12:11.051092+0000","last_became_peered":"2024-01-15T18:12:11.051092+0000","last_unstale":"2024-01-15T18:18:31.743706+0000","last_undegraded":"2024-01-15T18:18:31.743706+0000","last_fullsized":"2024-01-15T18:18:31.743706+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'45","last_scrub_stamp":"2024-01-15T18:18:31.743522+0000","last_deep_scrub":"117'45","last_deep_scrub_stamp":"2024-01-15T18:18:31.743522+0000","last_clean_scrub_stamp":"2024-01-15T18:18:31.743522+0000","objects_scrubbed":14,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:14:34.678849+0000","scrub_duration":0.17999523000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:02.284897+0000","last_change":"2024-01-15T18:18:02.284945+0000","last_active":"2024-01-15T18:18:02.284897+0000","last_peered":"2024-01-15T18:18:02.284897+0000","last_clean":"2024-01-15T18:18:02.284897+0000","last_became_active":"2024-01-15T18:12:10.026651+0000","last_became_peered":"2024-01-15T18:12:10.026651+0000","last_unstale":"2024-01-15T18:18:02.284897+0000","last_undegraded":"2024-01-15T18:18:02.284897+0000","last_fullsized":"2024-01-15T18:18:02.284897+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:02.284683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:02.284683+0000","last_clean_scrub_stamp":"2024-01-15T18:18:02.284683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:07.355476+0000","scrub_duration":0.0041976940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"117'119","reported_seq":262,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.554023+0000","last_change":"2024-01-15T18:18:01.554131+0000","last_active":"2024-01-15T18:18:01.554023+0000","last_peered":"2024-01-15T18:18:01.554023+0000","last_clean":"2024-01-15T18:18:01.554023+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:18:01.554023+0000","last_undegraded":"2024-01-15T18:18:01.554023+0000","last_fullsized":"2024-01-15T18:18:01.554023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'119","last_scrub_stamp":"2024-01-15T18:18:01.553859+0000","last_deep_scrub":"117'119","last_deep_scrub_stamp":"2024-01-15T18:18:01.553859+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.553859+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:28.775696+0000","scrub_duration":0.056845899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710133,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3844,"ondisk_log_size":3844,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":641570,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":279,"num_write_kb":755,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5503,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1343488,"data_stored":1283140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":278,"ondisk_log_size":278,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542431,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834588,"kb_used_data":1767380,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537252,"statfs":{"total":96636764160,"available":94758146048,"internally_reserved":0,"allocated":1809797120,"data_stored":1808970858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2629999999999999,"15min":1.2669999999999999},"min":{"1min":0.92600000000000005,"5min":0.66900000000000004,"15min":0.64200000000000002},"max":{"1min":1.72,"5min":1.8080000000000001,"15min":11.449999999999999},"last":1.413},{"interface":"front","average":{"1min":1.206,"5min":1.2370000000000001,"15min":1.3600000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.53400000000000003},"max":{"1min":1.6819999999999999,"5min":1.7090000000000001,"15min":16.678999999999998},"last":1.298}]},{"osd":1,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.2549999999999999,"15min":1.2809999999999999},"min":{"1min":1.022,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":11.51},"last":2.1840000000000002},{"interface":"front","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.508},"min":{"1min":1.0409999999999999,"5min":0.753,"15min":0.753},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":28.613},"last":1.9259999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.264,"15min":1.3180000000000001},"min":{"1min":0.93400000000000005,"5min":0.75600000000000001,"15min":0.67200000000000004},"max":{"1min":1.752,"5min":1.8140000000000001,"15min":12.061},"last":1.24},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.3620000000000001,"15min":1.47},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.67300000000000004},"max":{"1min":1.827,"5min":1.9299999999999999,"15min":16.792000000000002},"last":1.827}]},{"osd":3,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3049999999999999,"15min":1.391},"min":{"1min":1.0940000000000001,"5min":0.80700000000000005,"15min":0.77700000000000002},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":9.8800000000000008},"last":1.573},{"interface":"front","average":{"1min":1.518,"5min":1.429,"15min":1.528},"min":{"1min":1.2390000000000001,"5min":0.94399999999999995,"15min":0.72499999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":16.129000000000001},"last":2.3370000000000002}]},{"osd":4,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.288,"15min":1.5489999999999999},"min":{"1min":0.88800000000000001,"5min":0.78900000000000003,"15min":0.56299999999999994},"max":{"1min":2.077,"5min":2.077,"15min":23.042000000000002},"last":2.0630000000000002},{"interface":"front","average":{"1min":3.073,"5min":1.694,"15min":1.583},"min":{"1min":0.91200000000000003,"5min":0.80000000000000004,"15min":0.496},"max":{"1min":34.511000000000003,"5min":34.511000000000003,"15min":34.511000000000003},"last":1.8799999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.4019999999999999,"15min":1.502},"min":{"1min":1.0249999999999999,"5min":0.76800000000000002,"15min":0.746},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":11.696999999999999},"last":2.6339999999999999},{"interface":"front","average":{"1min":1.637,"5min":1.48,"15min":1.657},"min":{"1min":1.097,"5min":0.98499999999999999,"15min":0.624},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":22.933},"last":2.2850000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.694,"5min":1.548,"15min":1.8080000000000001},"min":{"1min":1.1930000000000001,"5min":1.0660000000000001,"15min":0.57399999999999995},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":23.004000000000001},"last":2.4209999999999998},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.4990000000000001,"15min":1.5780000000000001},"min":{"1min":1.119,"5min":1.008,"15min":0.64900000000000002},"max":{"1min":2.548,"5min":2.548,"15min":7.5220000000000002},"last":2.548}]}]},{"osd":6,"up_from":14,"seq":60129542430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1260568,"kb_used_data":1193360,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93111272,"statfs":{"total":96636764160,"available":95345942528,"internally_reserved":0,"allocated":1222000640,"data_stored":1221177506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2629999999999999,"15min":1.464},"min":{"1min":0.79600000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.6419999999999999,"5min":1.7470000000000001,"15min":31.805},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.3,"15min":1.4870000000000001},"min":{"1min":0.749,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8340000000000001,"5min":3.5409999999999999,"15min":31.702999999999999},"last":1.0580000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1339999999999999,"15min":1.2609999999999999},"min":{"1min":0.69699999999999995,"5min":0.59999999999999998,"15min":0.59899999999999998},"max":{"1min":1.6359999999999999,"5min":1.778,"15min":15.064},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.2050000000000001,"15min":1.4299999999999999},"min":{"1min":0.86499999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6279999999999999,"5min":1.9650000000000001,"15min":23.495999999999999},"last":1.2050000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2849999999999999,"15min":1.5349999999999999},"min":{"1min":0.81699999999999995,"5min":0.79100000000000004,"15min":0.59399999999999997},"max":{"1min":1.502,"5min":1.8899999999999999,"15min":23.087},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.331,"15min":1.397},"min":{"1min":1.0089999999999999,"5min":0.77000000000000002,"15min":0.51400000000000001},"max":{"1min":1.6319999999999999,"5min":1.9990000000000001,"15min":14.94},"last":1.181}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4359999999999999,"15min":1.6040000000000001},"min":{"1min":1.0620000000000001,"5min":0.65200000000000002,"15min":0.59899999999999998},"max":{"1min":1.7,"5min":1.8160000000000001,"15min":15.151999999999999},"last":1.407},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3779999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6080000000000001,"5min":3.8500000000000001,"15min":13.795},"last":0.94599999999999995}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.2729999999999999,"15min":1.585},"min":{"1min":0.90900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.53,"5min":3.3719999999999999,"15min":23.327000000000002},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.3560000000000001,"15min":1.548},"min":{"1min":0.80900000000000005,"5min":0.56299999999999994,"15min":0.52000000000000002},"max":{"1min":1.6839999999999999,"5min":1.8839999999999999,"15min":15.239000000000001},"last":0.91100000000000003}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.244,"5min":1.331,"15min":1.502},"min":{"1min":0.88700000000000001,"5min":0.58099999999999996,"15min":0.52500000000000002},"max":{"1min":1.7210000000000001,"5min":3.6480000000000001,"15min":13.593},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.49,"5min":1.55,"15min":1.8049999999999999},"min":{"1min":1.1930000000000001,"5min":0.66200000000000003,"15min":0.621},"max":{"1min":1.702,"5min":2.306,"15min":23.417999999999999},"last":1.4670000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.47,"15min":1.716},"min":{"1min":0.98799999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8220000000000001,"5min":3.105,"15min":21.265000000000001},"last":1.147},{"interface":"front","average":{"1min":1.486,"5min":1.542,"15min":1.899},"min":{"1min":1.1200000000000001,"5min":0.62,"15min":0.62},"max":{"1min":1.7789999999999999,"5min":2.2229999999999999,"15min":23.738},"last":1.357}]}]},{"osd":5,"up_from":14,"seq":60129542430,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908440,"kb_used_data":832912,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93463400,"statfs":{"total":96636764160,"available":95706521600,"internally_reserved":0,"allocated":852901888,"data_stored":852079166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.202,"15min":1.2669999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.47799999999999998},"max":{"1min":1.595,"5min":2.7919999999999998,"15min":19.266999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.196,"15min":1.1970000000000001},"min":{"1min":0.752,"5min":0.748,"15min":0.51400000000000001},"max":{"1min":1.671,"5min":1.7529999999999999,"15min":3.1760000000000002},"last":1.708}]},{"osd":1,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.359,"15min":1.3149999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.60699999999999998},"max":{"1min":1.653,"5min":15.554,"15min":15.554},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.242,"5min":1.302,"15min":1.3540000000000001},"min":{"1min":0.80100000000000005,"5min":0.69599999999999995,"15min":0.59099999999999997},"max":{"1min":1.643,"5min":6.1449999999999996,"15min":9.2379999999999995},"last":1.6579999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.26,"15min":1.2629999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.621},"max":{"1min":1.694,"5min":2.3500000000000001,"15min":4.0369999999999999},"last":1.615},{"interface":"front","average":{"1min":1.349,"5min":1.512,"15min":1.4299999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.56000000000000005},"max":{"1min":1.617,"5min":15.468999999999999,"15min":15.468999999999999},"last":1.006}]},{"osd":3,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.327,"15min":1.409},"min":{"1min":0.78200000000000003,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.718,"5min":1.984,"15min":9.1929999999999996},"last":1.351},{"interface":"front","average":{"1min":1.357,"5min":1.4379999999999999,"15min":1.4650000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.64600000000000002},"max":{"1min":1.774,"5min":2.2949999999999999,"15min":7.0110000000000001},"last":1.4990000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.4319999999999999,"15min":1.47},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.41399999999999998},"max":{"1min":1.6040000000000001,"5min":15.254,"15min":15.254},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.4019999999999999,"15min":1.4710000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.8080000000000001,"5min":1.923,"15min":9.1519999999999992},"last":1.0629999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.4039999999999999,"15min":1.456},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.39600000000000002},"max":{"1min":1.8480000000000001,"5min":2.2250000000000001,"15min":3.7559999999999998},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.593,"15min":1.5680000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53000000000000003},"max":{"1min":1.7330000000000001,"5min":15.366,"15min":15.366},"last":1.103}]},{"osd":7,"last update":"Mon Jan 15 18:17:32 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6100000000000001},"min":{"1min":0.66400000000000003,"5min":0.65300000000000002,"15min":0.45800000000000002},"max":{"1min":1.873,"5min":2.0609999999999999,"15min":9.0600000000000005},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.5840000000000001,"15min":1.6890000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.57299999999999995},"max":{"1min":1.899,"5min":3.1659999999999999,"15min":19.356999999999999},"last":1.7989999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1711124,"kb_used_data":1643916,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92660716,"statfs":{"total":96636764160,"available":94884573184,"internally_reserved":0,"allocated":1683369984,"data_stored":1682551014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.25,"15min":1.3029999999999999},"min":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.39200000000000002},"max":{"1min":1.6519999999999999,"5min":1.8420000000000001,"15min":17.346},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3520000000000001,"15min":1.3260000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.40100000000000002},"max":{"1min":1.623,"5min":1.804,"15min":3.726},"last":1.409}]},{"osd":1,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.224,"15min":1.2370000000000001},"min":{"1min":0.86799999999999999,"5min":0.76600000000000001,"15min":0.439},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":11.128},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.226,"15min":1.216},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.67500000000000004},"max":{"1min":1.528,"5min":1.7470000000000001,"15min":3.7370000000000001},"last":0.95999999999999996}]},{"osd":2,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3029999999999999,"15min":1.333},"min":{"1min":0.873,"5min":0.83399999999999996,"15min":0.56899999999999995},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":18.442},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.365,"15min":1.371},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.81899999999999995},"max":{"1min":1.8029999999999999,"5min":1.861,"15min":5.1340000000000003},"last":1.103}]},{"osd":3,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.371,"15min":1.4259999999999999},"min":{"1min":0.83499999999999996,"5min":0.73499999999999999,"15min":0.60199999999999998},"max":{"1min":1.923,"5min":1.923,"15min":17.364999999999998},"last":1.038},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.337,"15min":1.3700000000000001},"min":{"1min":1.04,"5min":0.85099999999999998,"15min":0.45300000000000001},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":9.3320000000000007},"last":1.5620000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3029999999999999,"15min":1.3500000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.49099999999999999},"max":{"1min":2.028,"5min":2.028,"15min":17.292999999999999},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.3520000000000001,"15min":1.4199999999999999},"min":{"1min":1.0109999999999999,"5min":0.88300000000000001,"15min":0.53700000000000003},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":12.332000000000001},"last":1.3280000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2589999999999999,"15min":1.367},"min":{"1min":0.94199999999999995,"5min":0.80100000000000005,"15min":0.54200000000000004},"max":{"1min":1.867,"5min":1.867,"15min":11.09},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.4670000000000001,"15min":1.603},"min":{"1min":1.004,"5min":1.004,"15min":0.51100000000000001},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":17.224},"last":1.365}]},{"osd":7,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.518,"15min":1.5660000000000001},"min":{"1min":1.101,"5min":0.93300000000000005,"15min":0.625},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":12.247},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.6519999999999999,"15min":1.718},"min":{"1min":1.3080000000000001,"5min":1.159,"15min":0.61799999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":17.324999999999999},"last":1.5229999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542432,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462036,"kb_used_data":1386508,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":92909804,"statfs":{"total":96636764160,"available":95139639296,"internally_reserved":0,"allocated":1419784192,"data_stored":1418949643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.006,"15min":1.0620000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5660000000000001,"5min":2.843,"15min":9.2899999999999991},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.234,"5min":1.242,"15min":1.1830000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.44800000000000001},"max":{"1min":1.6859999999999999,"5min":2.5590000000000002,"15min":3.028},"last":1.0980000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.022,"15min":1.0249999999999999},"min":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.32600000000000001},"max":{"1min":1.99,"5min":5.3520000000000003,"15min":12.172000000000001},"last":1.024},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.228},"min":{"1min":0.623,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.5900000000000001,"5min":2.3370000000000002,"15min":11.228999999999999},"last":0.98199999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.214,"15min":1.2230000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.375},"max":{"1min":1.885,"5min":1.885,"15min":2.7519999999999998},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4410000000000001,"15min":1.478},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.56100000000000005},"max":{"1min":2.2120000000000002,"5min":3.0659999999999998,"15min":9.4920000000000009},"last":1.2909999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.359,"15min":1.4319999999999999},"min":{"1min":0.81000000000000005,"5min":0.78200000000000003,"15min":0.67200000000000004},"max":{"1min":1.7490000000000001,"5min":1.865,"15min":13.536},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.4339999999999999,"15min":1.4330000000000001},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.63700000000000001},"max":{"1min":1.774,"5min":6.2439999999999998,"15min":16.398},"last":1.6200000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.472,"15min":1.583},"min":{"1min":1.175,"5min":0.83599999999999997,"15min":0.68899999999999995},"max":{"1min":2.1000000000000001,"5min":2.7269999999999999,"15min":9.5690000000000008},"last":1.175},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5820000000000001,"15min":1.5860000000000001},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.66800000000000004},"max":{"1min":1.9910000000000001,"5min":2.617,"15min":5.6980000000000004},"last":1.573}]},{"osd":6,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4730000000000001,"15min":1.5669999999999999},"min":{"1min":0.97299999999999998,"5min":0.85499999999999998,"15min":0.61399999999999999},"max":{"1min":2.0059999999999998,"5min":6.3109999999999999,"15min":13.648999999999999},"last":1.496},{"interface":"front","average":{"1min":1.51,"5min":1.54,"15min":1.694},"min":{"1min":1.0609999999999999,"5min":1.0249999999999999,"15min":0.77800000000000002},"max":{"1min":1.915,"5min":2.5910000000000002,"15min":13.679},"last":1.5169999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.6240000000000001,"15min":1.6080000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.60499999999999998},"max":{"1min":2.3679999999999999,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.679},{"interface":"front","average":{"1min":1.486,"5min":1.609,"15min":1.6040000000000001},"min":{"1min":1.125,"5min":0.995,"15min":0.70199999999999996},"max":{"1min":2.141,"5min":6.9569999999999999,"15min":9.4100000000000001},"last":1.3959999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542432,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":982100,"kb_used_data":910732,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93389740,"statfs":{"total":96636764160,"available":95631093760,"internally_reserved":0,"allocated":932589568,"data_stored":931771659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":1.161,"15min":1.083},"min":{"1min":0.67600000000000005,"5min":0.51000000000000001,"15min":0.26600000000000001},"max":{"1min":1.419,"5min":22.707000000000001,"15min":22.707000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1970000000000001,"15min":1.357},"min":{"1min":0.76300000000000001,"5min":0.61099999999999999,"15min":0.39300000000000002},"max":{"1min":1.6879999999999999,"5min":2.5710000000000002,"15min":15.314},"last":1.085}]},{"osd":1,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1759999999999999,"15min":1.3240000000000001},"min":{"1min":0.73299999999999998,"5min":0.58599999999999997,"15min":0.38300000000000001},"max":{"1min":1.546,"5min":2.8679999999999999,"15min":34.463999999999999},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.327,"15min":1.504},"min":{"1min":0.90700000000000003,"5min":0.73899999999999999,"15min":0.373},"max":{"1min":1.675,"5min":2.794,"15min":25.859999999999999},"last":0.96699999999999997}]},{"osd":3,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.1870000000000001,"15min":1.302},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.34399999999999997},"max":{"1min":1.464,"5min":2.7080000000000002,"15min":12.93},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.6919999999999999,"15min":1.6379999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.46400000000000002},"max":{"1min":1.903,"5min":37.021999999999998,"15min":37.021999999999998},"last":1.1399999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.52,"15min":1.6879999999999999},"min":{"1min":1.0289999999999999,"5min":0.96999999999999997,"15min":0.55700000000000005},"max":{"1min":1.78,"5min":3.419,"15min":29.986999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.518,"15min":1.7150000000000001},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.56799999999999995},"max":{"1min":1.883,"5min":2.976,"15min":21.199999999999999},"last":1.337}]},{"osd":5,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.456,"15min":1.6499999999999999},"min":{"1min":0.88,"5min":0.88,"15min":0.51500000000000001},"max":{"1min":2.0880000000000001,"5min":2.218,"15min":14.066000000000001},"last":1.423},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.643,"15min":1.7849999999999999},"min":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":0.60899999999999999},"max":{"1min":1.8180000000000001,"5min":3.4790000000000001,"15min":14.025},"last":1.7170000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.5189999999999999,"5min":1.6160000000000001,"15min":1.8069999999999999},"min":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":0.68000000000000005},"max":{"1min":1.9910000000000001,"5min":3.0800000000000001,"15min":15.683999999999999},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.585,"15min":1.784},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.58899999999999997},"max":{"1min":2.2930000000000001,"5min":10.371,"15min":12.629},"last":1.8720000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:37 2024","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.671,"15min":2.1419999999999999},"min":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":0.63100000000000001},"max":{"1min":2.2210000000000001,"5min":3.177,"15min":57.006999999999998},"last":1.772},{"interface":"front","average":{"1min":1.595,"5min":1.6299999999999999,"15min":1.9630000000000001},"min":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":0.56100000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":22.853999999999999},"last":1.9390000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542431,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1141924,"kb_used_data":1066396,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93229916,"statfs":{"total":96636764160,"available":95467433984,"internally_reserved":0,"allocated":1091989504,"data_stored":1091155706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.96499999999999997,"15min":1.177},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.34399999999999997},"max":{"1min":1.1419999999999999,"5min":2.7389999999999999,"15min":30.460999999999999},"last":1.337},{"interface":"front","average":{"1min":0.97899999999999998,"5min":1.115,"15min":1.26},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.314},"max":{"1min":1.6539999999999999,"5min":5.4779999999999998,"15min":18.658000000000001},"last":1.3899999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.075,"15min":1.1579999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47299999999999998},"max":{"1min":1.5589999999999999,"5min":1.7809999999999999,"15min":7.117},"last":1.244},{"interface":"front","average":{"1min":1.099,"5min":1.325,"15min":1.5129999999999999},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.496},"max":{"1min":1.728,"5min":1.98,"15min":30.593},"last":1.0940000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2529999999999999,"15min":1.45},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.48599999999999999},"max":{"1min":1.6779999999999999,"5min":1.889,"15min":30.547000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.018,"5min":1.1619999999999999,"15min":1.3200000000000001},"min":{"1min":0.55000000000000004,"5min":0.42299999999999999,"15min":0.40200000000000002},"max":{"1min":1.474,"5min":2.0459999999999998,"15min":17.263000000000002},"last":0.93100000000000005}]},{"osd":4,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.6639999999999999,"15min":1.6899999999999999},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.72699999999999998},"max":{"1min":1.6319999999999999,"5min":14.103,"15min":17.420000000000002},"last":1.28},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.653,"15min":1.6040000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7849999999999999,"5min":14.140000000000001,"15min":14.140000000000001},"last":1.5469999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3859999999999999,"15min":1.5329999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.59599999999999997},"max":{"1min":1.843,"5min":2.077,"15min":9.0440000000000005},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.508,"15min":1.7989999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.68300000000000005},"max":{"1min":1.891,"5min":2.3599999999999999,"15min":34.042999999999999},"last":1.4950000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.494,"15min":1.6830000000000001},"min":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":0.64100000000000001},"max":{"1min":1.9790000000000001,"5min":2.3780000000000001,"15min":34.158000000000001},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.4850000000000001,"15min":1.637},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.62},"max":{"1min":1.8260000000000001,"5min":2.5579999999999998,"15min":17.510000000000002},"last":1.7390000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:17:42 2024","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.603,"15min":1.8700000000000001},"min":{"1min":0.98299999999999998,"5min":0.82999999999999996,"15min":0.77300000000000002},"max":{"1min":1.998,"5min":2.4329999999999998,"15min":33.659999999999997},"last":1.593},{"interface":"front","average":{"1min":1.5229999999999999,"5min":1.6439999999999999,"15min":1.742},"min":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.70299999999999996},"max":{"1min":1.9370000000000001,"5min":2.6669999999999998,"15min":11.882999999999999},"last":1.7030000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542432,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1769764,"kb_used_data":1698396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":92602076,"statfs":{"total":96636764160,"available":94824525824,"internally_reserved":0,"allocated":1739157504,"data_stored":1738307390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.1080000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.32400000000000001},"max":{"1min":1.4630000000000001,"5min":2.5209999999999999,"15min":12.050000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.3100000000000001,"15min":1.357},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.36399999999999999},"max":{"1min":1.609,"5min":6.6890000000000001,"15min":11.962},"last":1.3080000000000001}]},{"osd":2,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.248,"15min":1.2230000000000001},"min":{"1min":0.48599999999999999,"5min":0.45600000000000002,"15min":0.45500000000000002},"max":{"1min":1.7050000000000001,"5min":6.8099999999999996,"15min":11.59},"last":1.147},{"interface":"front","average":{"1min":0.995,"5min":1.0620000000000001,"15min":1.1970000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.6870000000000001,"5min":1.825,"15min":14.512},"last":1.212}]},{"osd":3,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.385,"15min":1.425},"min":{"1min":0.58899999999999997,"5min":0.51300000000000001,"15min":0.50800000000000001},"max":{"1min":1.7130000000000001,"5min":6.8719999999999999,"15min":11.904},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1899999999999999,"15min":1.1830000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.35199999999999998},"max":{"1min":1.696,"5min":2.3919999999999999,"15min":6.1539999999999999},"last":1.0940000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.29,"15min":1.542},"min":{"1min":0.68000000000000005,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.5860000000000001,"5min":2.1829999999999998,"15min":32.115000000000002},"last":1.331},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.579,"15min":1.6259999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.67700000000000005},"max":{"1min":1.6890000000000001,"5min":6.9329999999999998,"15min":11.992000000000001},"last":1.4750000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3879999999999999,"15min":1.655},"min":{"1min":0.96299999999999997,"5min":0.60899999999999999,"15min":0.47499999999999998},"max":{"1min":1.514,"5min":2.593,"15min":32.174999999999997},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.6020000000000001,"15min":1.6160000000000001},"min":{"1min":0.75,"5min":0.75,"15min":0.442},"max":{"1min":2.157,"5min":15.907,"15min":17.225000000000001},"last":1.3600000000000001}]},{"osd":6,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.528,"15min":1.754},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.66600000000000004},"max":{"1min":2.3290000000000002,"5min":2.5590000000000002,"15min":24.039999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.49,"15min":1.7789999999999999},"min":{"1min":0.99199999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.758,"5min":2.5449999999999999,"15min":32.216999999999999},"last":1.8029999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:17:33 2024","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.708,"15min":2.0800000000000001},"min":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.69099999999999995},"max":{"1min":1.8300000000000001,"5min":7.0979999999999999,"15min":34.131},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.641,"15min":1.984},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.61799999999999999},"max":{"1min":2.266,"5min":3.3759999999999999,"15min":44.694000000000003},"last":1.5149999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":655360,"data_stored":639563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":639521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1736458240,"data_stored":1736442609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":931135488,"data_stored":931135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":851443712,"data_stored":851443712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:18:41.836 INFO:tasks.ceph:pgid 17.2d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.836 INFO:tasks.ceph:pgid 17.26 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.836 INFO:tasks.ceph:pgid 17.24 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.836 INFO:tasks.ceph:pgid 17.21 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.837 INFO:tasks.ceph:pgid 17.1d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.837 INFO:tasks.ceph:pgid 17.14 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.837 INFO:tasks.ceph:pgid 17.12 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.838 INFO:tasks.ceph:pgid 17.d last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.838 INFO:tasks.ceph:pgid 17.b last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.838 INFO:tasks.ceph:pgid 17.1 last_scrub_stamp 2024-01-15T18:12:09.983156+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=12, tm_sec=9, tm_wday=0, tm_yday=15, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=15, tm_hour=18, tm_min=17, tm_sec=53, tm_wday=0, tm_yday=15, tm_isdst=0) 2024-01-15T18:18:41.838 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-15T18:19:01.839 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-15T18:19:02.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.067+0000 7fef03e00640 1 Processor -- start 2024-01-15T18:19:02.070 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.068+0000 7fef03e00640 1 -- start start 2024-01-15T18:19:02.071 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.069+0000 7fef03e00640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feefc0ae5a0 0x7feefc15e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.069+0000 7fef03e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 0x7feefc06cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.069+0000 7fef03e00640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc070c40 con 0x7feefc06d120 2024-01-15T18:19:02.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.069+0000 7fef03e00640 1 -- --> v2:172.21.15.26:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc071450 con 0x7feefc0ae5a0 2024-01-15T18:19:02.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.069+0000 7fef03e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc06c470 con 0x7feefc0b3ae0 2024-01-15T18:19:02.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.069+0000 7fef01400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feefc0ae5a0 0x7feefc15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.069+0000 7fef01400640 1 --2- >> v2:172.21.15.26:3300/0 conn(0x7feefc0ae5a0 0x7feefc15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53170/0 (socket says 172.21.15.26:53170) 2024-01-15T18:19:02.072 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.069+0000 7fef01400640 1 -- 172.21.15.26:0/410055904 learned_addr learned my addr 172.21.15.26:0/410055904 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:19:02.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.070+0000 7fef00a00640 1 --2- 172.21.15.26:0/410055904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 0x7feefc06cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.070+0000 7fef01400640 1 -- 172.21.15.26:0/410055904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 msgr2=0x7feefc06cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:02.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.070+0000 7fef01400640 1 --2- 172.21.15.26:0/410055904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 0x7feefc06cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.073 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.070+0000 7fef01400640 1 -- 172.21.15.26:0/410055904 >> v1:172.21.15.124:6789/0 conn(0x7feefc06d120 legacy=0x7feefc0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.070+0000 7fef01400640 1 -- 172.21.15.26:0/410055904 --> v2:172.21.15.26:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc06c750 con 0x7feefc0ae5a0 2024-01-15T18:19:02.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.072+0000 7fef01400640 1 --2- 172.21.15.26:0/410055904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 0x7feefc15e430 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7feeec009870 tx=0x7feeec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aa6f5c86433b4f server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:02.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.072+0000 7feefbe00640 1 -- 172.21.15.26:0/410055904 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec030840 con 0x7feefc0ae5a0 2024-01-15T18:19:02.074 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.072+0000 7feefbe00640 1 -- 172.21.15.26:0/410055904 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec0309a0 con 0x7feefc0ae5a0 2024-01-15T18:19:02.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.072+0000 7feefbe00640 1 -- 172.21.15.26:0/410055904 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec030d30 con 0x7feefc0ae5a0 2024-01-15T18:19:02.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.073+0000 7fef03e00640 1 -- 172.21.15.26:0/410055904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 msgr2=0x7feefc15e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.075 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.073+0000 7fef03e00640 1 --2- 172.21.15.26:0/410055904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 0x7feefc15e430 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7feeec009870 tx=0x7feeec02b980 comp rx=0 tx=0).stop 2024-01-15T18:19:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.073+0000 7fef03e00640 1 -- 172.21.15.26:0/410055904 shutdown_connections 2024-01-15T18:19:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.073+0000 7fef03e00640 1 --2- 172.21.15.26:0/410055904 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 0x7feefc06cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.073+0000 7fef03e00640 1 --2- 172.21.15.26:0/410055904 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 0x7feefc15e430 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.073+0000 7fef03e00640 1 -- 172.21.15.26:0/410055904 >> 172.21.15.26:0/410055904 conn(0x7feefc06a800 msgr2=0x7feefc06ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.073+0000 7fef03e00640 1 -- 172.21.15.26:0/410055904 shutdown_connections 2024-01-15T18:19:02.076 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.073+0000 7fef03e00640 1 -- 172.21.15.26:0/410055904 wait complete. 2024-01-15T18:19:02.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.075+0000 7fef03e00640 1 Processor -- start 2024-01-15T18:19:02.077 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.075+0000 7fef03e00640 1 -- start start 2024-01-15T18:19:02.078 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.076+0000 7fef03e00640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc06d120 0x7feefc0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.076+0000 7fef03e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 0x7feefc0ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.076+0000 7fef03e00640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 0x7feefc157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.076+0000 7fef03e00640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0cc7b0 con 0x7feefc0ae5a0 2024-01-15T18:19:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.076+0000 7fef03e00640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc156f80 con 0x7feefc06d120 2024-01-15T18:19:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.076+0000 7fef03e00640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc160350 con 0x7feefc0b3ae0 2024-01-15T18:19:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.076+0000 7fef00a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 0x7feefc0ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.076+0000 7fef00a00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 0x7feefc0ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53184/0 (socket says 172.21.15.26:53184) 2024-01-15T18:19:02.079 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.076+0000 7fef00a00640 1 -- 172.21.15.26:0/2037285474 learned_addr learned my addr 172.21.15.26:0/2037285474 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:19:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7fef01400640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc06d120 0x7feefc0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7fef01e00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 0x7feefc157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7fef00a00640 1 -- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 msgr2=0x7feefc157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7fef00a00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 0x7feefc157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7fef00a00640 1 -- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc06d120 msgr2=0x7feefc0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7fef00a00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc06d120 0x7feefc0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7fef00a00640 1 -- 172.21.15.26:0/2037285474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0094d0 con 0x7feefc0ae5a0 2024-01-15T18:19:02.081 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7fef00a00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 0x7feefc0ce1e0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7feee400d800 tx=0x7feee400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7feef9600640 1 -- 172.21.15.26:0/2037285474 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400b6c0 con 0x7feefc0ae5a0 2024-01-15T18:19:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7feef9600640 1 -- 172.21.15.26:0/2037285474 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee400b840 con 0x7feefc0ae5a0 2024-01-15T18:19:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7feef9600640 1 -- 172.21.15.26:0/2037285474 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400bbf0 con 0x7feefc0ae5a0 2024-01-15T18:19:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.077+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc156930 con 0x7feefc0ae5a0 2024-01-15T18:19:02.082 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.078+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc0c3e50 con 0x7feefc0ae5a0 2024-01-15T18:19:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.079+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc156790 con 0x7feefc0ae5a0 2024-01-15T18:19:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.080+0000 7feef9600640 1 -- 172.21.15.26:0/2037285474 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 91171+0+0 (secure 0 0 0) 0x7feee400bd50 con 0x7feefc0ae5a0 2024-01-15T18:19:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.081+0000 7feef9600640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406c4c0 0x7feed406e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.083 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.081+0000 7feef9600640 1 -- 172.21.15.26:0/2037285474 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7feee4088db0 con 0x7feefc0ae5a0 2024-01-15T18:19:02.084 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.082+0000 7fef01400640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406c4c0 0x7feed406e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.085 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.083+0000 7fef01400640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406c4c0 0x7feed406e980 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7feeec036000 tx=0x7feeec008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:02.088 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.086+0000 7feef9600640 1 -- 172.21.15.26:0/2037285474 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feefc156790 con 0x7feefc0ae5a0 2024-01-15T18:19:02.277 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.275+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 --> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feefc070550 con 0x7feed406c4c0 2024-01-15T18:19:02.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.298+0000 7feef9600640 1 -- 172.21.15.26:0/2037285474 <== mgr.4102 v2:172.21.15.124:6832/2029932521 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291068 (secure 0 0 0) 0x7feeec039450 con 0x7feed406c4c0 2024-01-15T18:19:02.303 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:19:02.306 INFO:teuthology.orchestra.run.smithi026.stderr:dumped all 2024-01-15T18:19:02.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406c4c0 msgr2=0x7feed406e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.307 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406c4c0 0x7feed406e980 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7feeec036000 tx=0x7feeec008030 comp rx=0 tx=0).stop 2024-01-15T18:19:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 msgr2=0x7feefc0ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 0x7feefc0ce1e0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7feee400d800 tx=0x7feee400dcc0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 shutdown_connections 2024-01-15T18:19:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7feefc0b3ae0 0x7feefc157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7feed406c4c0 0x7feed406e980 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7feefc0ae5a0 0x7feefc0ce1e0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 --2- 172.21.15.26:0/2037285474 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7feefc06d120 0x7feefc0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.308 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 >> 172.21.15.26:0/2037285474 conn(0x7feefc06a800 msgr2=0x7feefc15a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 shutdown_connections 2024-01-15T18:19:02.309 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.305+0000 7fef03e00640 1 -- 172.21.15.26:0/2037285474 wait complete. 2024-01-15T18:19:02.329 INFO:teuthology.orchestra.run.smithi026.stdout:{"pg_ready":true,"pg_map":{"version":832,"stamp":"2024-01-15T18:19:00.479488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5369942212,"num_objects":1311,"num_object_clones":1,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1311,"num_whiteouts":1,"num_read":650,"num_read_kb":2097274,"num_write":1746,"num_write_kb":5246068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5503,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4460,"ondisk_log_size":4460,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":11070544,"kb_used_data":10499600,"kb_used_omap":12,"kb_used_meta":570867,"kb_avail":743904176,"statfs":{"total":773094113280,"available":761757876224,"internally_reserved":0,"allocated":10751590400,"data_stored":10744963224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":584568368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004882"},"pg_stats":[{"pgid":"17.3f","version":"117'78","reported_seq":119,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.182705+0000","last_change":"2024-01-15T18:18:15.182793+0000","last_active":"2024-01-15T18:18:15.182705+0000","last_peered":"2024-01-15T18:18:15.182705+0000","last_clean":"2024-01-15T18:18:15.182705+0000","last_became_active":"2024-01-15T18:12:53.334688+0000","last_became_peered":"2024-01-15T18:12:53.334688+0000","last_unstale":"2024-01-15T18:18:15.182705+0000","last_undegraded":"2024-01-15T18:18:15.182705+0000","last_fullsized":"2024-01-15T18:18:15.182705+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'78","last_scrub_stamp":"2024-01-15T18:18:15.182527+0000","last_deep_scrub":"117'78","last_deep_scrub_stamp":"2024-01-15T18:18:15.182527+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.182527+0000","objects_scrubbed":25,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:21:28.406331+0000","scrub_duration":0.27837603599999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":25,"num_write_kb":102400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":33554432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3e","version":"117'64","reported_seq":97,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:16.135979+0000","last_change":"2024-01-15T18:18:16.135979+0000","last_active":"2024-01-15T18:18:16.135979+0000","last_peered":"2024-01-15T18:18:16.135979+0000","last_clean":"2024-01-15T18:18:16.135979+0000","last_became_active":"2024-01-15T18:12:11.053279+0000","last_became_peered":"2024-01-15T18:12:11.053279+0000","last_unstale":"2024-01-15T18:18:16.135979+0000","last_undegraded":"2024-01-15T18:18:16.135979+0000","last_fullsized":"2024-01-15T18:18:16.135979+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'64","last_scrub_stamp":"2024-01-15T18:18:16.135588+0000","last_deep_scrub":"117'64","last_deep_scrub_stamp":"2024-01-15T18:18:16.135588+0000","last_clean_scrub_stamp":"2024-01-15T18:18:16.135588+0000","objects_scrubbed":19,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:16:49.227522+0000","scrub_duration":0.232285464,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"117'63","reported_seq":96,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:35.501988+0000","last_change":"2024-01-15T18:18:35.502092+0000","last_active":"2024-01-15T18:18:35.501988+0000","last_peered":"2024-01-15T18:18:35.501988+0000","last_clean":"2024-01-15T18:18:35.501988+0000","last_became_active":"2024-01-15T18:12:11.057543+0000","last_became_peered":"2024-01-15T18:12:11.057543+0000","last_unstale":"2024-01-15T18:18:35.501988+0000","last_undegraded":"2024-01-15T18:18:35.501988+0000","last_fullsized":"2024-01-15T18:18:35.501988+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:35.501850+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:35.501850+0000","last_clean_scrub_stamp":"2024-01-15T18:18:35.501850+0000","objects_scrubbed":19,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:02:25.874738+0000","scrub_duration":0.25578943700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"117'68","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:32.563179+0000","last_change":"2024-01-15T18:18:32.563337+0000","last_active":"2024-01-15T18:18:32.563179+0000","last_peered":"2024-01-15T18:18:32.563179+0000","last_clean":"2024-01-15T18:18:32.563179+0000","last_became_active":"2024-01-15T18:12:11.047008+0000","last_became_peered":"2024-01-15T18:12:11.047008+0000","last_unstale":"2024-01-15T18:18:32.563179+0000","last_undegraded":"2024-01-15T18:18:32.563179+0000","last_fullsized":"2024-01-15T18:18:32.563179+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'68","last_scrub_stamp":"2024-01-15T18:18:32.562960+0000","last_deep_scrub":"117'68","last_deep_scrub_stamp":"2024-01-15T18:18:32.562960+0000","last_clean_scrub_stamp":"2024-01-15T18:18:32.562960+0000","objects_scrubbed":23,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:42:05.499582+0000","scrub_duration":0.25917564100000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"117'37","reported_seq":68,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.943087+0000","last_change":"2024-01-15T18:18:09.943156+0000","last_active":"2024-01-15T18:18:09.943087+0000","last_peered":"2024-01-15T18:18:09.943087+0000","last_clean":"2024-01-15T18:18:09.943087+0000","last_became_active":"2024-01-15T18:12:11.034909+0000","last_became_peered":"2024-01-15T18:12:11.034909+0000","last_unstale":"2024-01-15T18:18:09.943087+0000","last_undegraded":"2024-01-15T18:18:09.943087+0000","last_fullsized":"2024-01-15T18:18:09.943087+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'37","last_scrub_stamp":"2024-01-15T18:18:09.942945+0000","last_deep_scrub":"117'37","last_deep_scrub_stamp":"2024-01-15T18:18:09.942945+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.942945+0000","objects_scrubbed":10,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:42:24.509082+0000","scrub_duration":0.101423941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"117'69","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.095936+0000","last_change":"2024-01-15T18:18:15.096025+0000","last_active":"2024-01-15T18:18:15.095936+0000","last_peered":"2024-01-15T18:18:15.095936+0000","last_clean":"2024-01-15T18:18:15.095936+0000","last_became_active":"2024-01-15T18:12:11.054803+0000","last_became_peered":"2024-01-15T18:12:11.054803+0000","last_unstale":"2024-01-15T18:18:15.095936+0000","last_undegraded":"2024-01-15T18:18:15.095936+0000","last_fullsized":"2024-01-15T18:18:15.095936+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'69","last_scrub_stamp":"2024-01-15T18:18:15.095819+0000","last_deep_scrub":"117'69","last_deep_scrub_stamp":"2024-01-15T18:18:15.095819+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.095819+0000","objects_scrubbed":21,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:37:46.439597+0000","scrub_duration":0.18604361999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"117'60","reported_seq":88,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.442469+0000","last_change":"2024-01-15T18:18:09.442535+0000","last_active":"2024-01-15T18:18:09.442469+0000","last_peered":"2024-01-15T18:18:09.442469+0000","last_clean":"2024-01-15T18:18:09.442469+0000","last_became_active":"2024-01-15T18:12:11.044607+0000","last_became_peered":"2024-01-15T18:12:11.044607+0000","last_unstale":"2024-01-15T18:18:09.442469+0000","last_undegraded":"2024-01-15T18:18:09.442469+0000","last_fullsized":"2024-01-15T18:18:09.442469+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'60","last_scrub_stamp":"2024-01-15T18:18:09.442344+0000","last_deep_scrub":"117'60","last_deep_scrub_stamp":"2024-01-15T18:18:09.442344+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.442344+0000","objects_scrubbed":17,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:43:41.911933+0000","scrub_duration":0.21794185399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"117'63","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:26.581150+0000","last_change":"2024-01-15T18:18:26.581229+0000","last_active":"2024-01-15T18:18:26.581150+0000","last_peered":"2024-01-15T18:18:26.581150+0000","last_clean":"2024-01-15T18:18:26.581150+0000","last_became_active":"2024-01-15T18:12:11.050697+0000","last_became_peered":"2024-01-15T18:12:11.050697+0000","last_unstale":"2024-01-15T18:18:26.581150+0000","last_undegraded":"2024-01-15T18:18:26.581150+0000","last_fullsized":"2024-01-15T18:18:26.581150+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:26.580974+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:26.580974+0000","last_clean_scrub_stamp":"2024-01-15T18:18:26.580974+0000","objects_scrubbed":19,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:34:04.570500+0000","scrub_duration":0.20942063299999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":77825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"117'68","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:36.525482+0000","last_change":"2024-01-15T18:18:36.525588+0000","last_active":"2024-01-15T18:18:36.525482+0000","last_peered":"2024-01-15T18:18:36.525482+0000","last_clean":"2024-01-15T18:18:36.525482+0000","last_became_active":"2024-01-15T18:12:11.047693+0000","last_became_peered":"2024-01-15T18:12:11.047693+0000","last_unstale":"2024-01-15T18:18:36.525482+0000","last_undegraded":"2024-01-15T18:18:36.525482+0000","last_fullsized":"2024-01-15T18:18:36.525482+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'68","last_scrub_stamp":"2024-01-15T18:18:36.525358+0000","last_deep_scrub":"117'68","last_deep_scrub_stamp":"2024-01-15T18:18:36.525358+0000","last_clean_scrub_stamp":"2024-01-15T18:18:36.525358+0000","objects_scrubbed":21,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:39:38.655063+0000","scrub_duration":0.201605748,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"117'58","reported_seq":92,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:55.436550+0000","last_change":"2024-01-15T18:17:55.436587+0000","last_active":"2024-01-15T18:17:55.436550+0000","last_peered":"2024-01-15T18:17:55.436550+0000","last_clean":"2024-01-15T18:17:55.436550+0000","last_became_active":"2024-01-15T18:12:11.052429+0000","last_became_peered":"2024-01-15T18:12:11.052429+0000","last_unstale":"2024-01-15T18:17:55.436550+0000","last_undegraded":"2024-01-15T18:17:55.436550+0000","last_fullsized":"2024-01-15T18:17:55.436550+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'58","last_scrub_stamp":"2024-01-15T18:17:55.436465+0000","last_deep_scrub":"117'58","last_deep_scrub_stamp":"2024-01-15T18:17:55.436465+0000","last_clean_scrub_stamp":"2024-01-15T18:17:55.436465+0000","objects_scrubbed":17,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:11:39.403468+0000","scrub_duration":0.096421670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":10,"num_read_kb":40960,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"117'69","reported_seq":106,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.262993+0000","last_change":"2024-01-15T18:18:07.263053+0000","last_active":"2024-01-15T18:18:07.262993+0000","last_peered":"2024-01-15T18:18:07.262993+0000","last_clean":"2024-01-15T18:18:07.262993+0000","last_became_active":"2024-01-15T18:12:11.042720+0000","last_became_peered":"2024-01-15T18:12:11.042720+0000","last_unstale":"2024-01-15T18:18:07.262993+0000","last_undegraded":"2024-01-15T18:18:07.262993+0000","last_fullsized":"2024-01-15T18:18:07.262993+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'69","last_scrub_stamp":"2024-01-15T18:18:07.262832+0000","last_deep_scrub":"117'69","last_deep_scrub_stamp":"2024-01-15T18:18:07.262832+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.262832+0000","objects_scrubbed":28,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:08:52.313757+0000","scrub_duration":0.35287618199999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":28,"num_write_kb":114688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"117'73","reported_seq":114,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:08.692443+0000","last_change":"2024-01-15T18:18:08.692487+0000","last_active":"2024-01-15T18:18:08.692443+0000","last_peered":"2024-01-15T18:18:08.692443+0000","last_clean":"2024-01-15T18:18:08.692443+0000","last_became_active":"2024-01-15T18:12:11.044155+0000","last_became_peered":"2024-01-15T18:12:11.044155+0000","last_unstale":"2024-01-15T18:18:08.692443+0000","last_undegraded":"2024-01-15T18:18:08.692443+0000","last_fullsized":"2024-01-15T18:18:08.692443+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'73","last_scrub_stamp":"2024-01-15T18:18:08.692341+0000","last_deep_scrub":"117'73","last_deep_scrub_stamp":"2024-01-15T18:18:08.692341+0000","last_clean_scrub_stamp":"2024-01-15T18:18:08.692341+0000","objects_scrubbed":28,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:17:19.915429+0000","scrub_duration":0.266379529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":16,"num_read_kb":61441,"num_write":30,"num_write_kb":114689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"117'57","reported_seq":93,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:27.628796+0000","last_change":"2024-01-15T18:18:27.628885+0000","last_active":"2024-01-15T18:18:27.628796+0000","last_peered":"2024-01-15T18:18:27.628796+0000","last_clean":"2024-01-15T18:18:27.628796+0000","last_became_active":"2024-01-15T18:12:11.051875+0000","last_became_peered":"2024-01-15T18:12:11.051875+0000","last_unstale":"2024-01-15T18:18:27.628796+0000","last_undegraded":"2024-01-15T18:18:27.628796+0000","last_fullsized":"2024-01-15T18:18:27.628796+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'57","last_scrub_stamp":"2024-01-15T18:18:27.628608+0000","last_deep_scrub":"117'57","last_deep_scrub_stamp":"2024-01-15T18:18:27.628608+0000","last_clean_scrub_stamp":"2024-01-15T18:18:27.628608+0000","objects_scrubbed":19,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:34:26.497463+0000","scrub_duration":0.22558270899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"117'74","reported_seq":106,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:08.869762+0000","last_change":"2024-01-15T18:18:08.869841+0000","last_active":"2024-01-15T18:18:08.869762+0000","last_peered":"2024-01-15T18:18:08.869762+0000","last_clean":"2024-01-15T18:18:08.869762+0000","last_became_active":"2024-01-15T18:12:11.055813+0000","last_became_peered":"2024-01-15T18:12:11.055813+0000","last_unstale":"2024-01-15T18:18:08.869762+0000","last_undegraded":"2024-01-15T18:18:08.869762+0000","last_fullsized":"2024-01-15T18:18:08.869762+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'74","last_scrub_stamp":"2024-01-15T18:18:08.869620+0000","last_deep_scrub":"117'74","last_deep_scrub_stamp":"2024-01-15T18:18:08.869620+0000","last_clean_scrub_stamp":"2024-01-15T18:18:08.869620+0000","objects_scrubbed":27,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:04:52.461222+0000","scrub_duration":0.34224997800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":27,"num_write_kb":110592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"117'46","reported_seq":74,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:16.051211+0000","last_change":"2024-01-15T18:18:16.051305+0000","last_active":"2024-01-15T18:18:16.051211+0000","last_peered":"2024-01-15T18:18:16.051211+0000","last_clean":"2024-01-15T18:18:16.051211+0000","last_became_active":"2024-01-15T18:12:11.055217+0000","last_became_peered":"2024-01-15T18:12:11.055217+0000","last_unstale":"2024-01-15T18:18:16.051211+0000","last_undegraded":"2024-01-15T18:18:16.051211+0000","last_fullsized":"2024-01-15T18:18:16.051211+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'46","last_scrub_stamp":"2024-01-15T18:18:16.051013+0000","last_deep_scrub":"117'46","last_deep_scrub_stamp":"2024-01-15T18:18:16.051013+0000","last_clean_scrub_stamp":"2024-01-15T18:18:16.051013+0000","objects_scrubbed":12,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:00:03.749881+0000","scrub_duration":0.141849805,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50331648,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":12,"num_write_kb":49152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"117'60","reported_seq":94,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:27.470382+0000","last_change":"2024-01-15T18:18:27.470514+0000","last_active":"2024-01-15T18:18:27.470382+0000","last_peered":"2024-01-15T18:18:27.470382+0000","last_clean":"2024-01-15T18:18:27.470382+0000","last_became_active":"2024-01-15T18:12:53.338860+0000","last_became_peered":"2024-01-15T18:12:53.338860+0000","last_unstale":"2024-01-15T18:18:27.470382+0000","last_undegraded":"2024-01-15T18:18:27.470382+0000","last_fullsized":"2024-01-15T18:18:27.470382+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'60","last_scrub_stamp":"2024-01-15T18:18:27.470106+0000","last_deep_scrub":"117'60","last_deep_scrub_stamp":"2024-01-15T18:18:27.470106+0000","last_clean_scrub_stamp":"2024-01-15T18:18:27.470106+0000","objects_scrubbed":18,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:57:23.041064+0000","scrub_duration":0.228570462,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":29360128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2f","version":"117'66","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:23.332457+0000","last_change":"2024-01-15T18:18:23.332546+0000","last_active":"2024-01-15T18:18:23.332457+0000","last_peered":"2024-01-15T18:18:23.332457+0000","last_clean":"2024-01-15T18:18:23.332457+0000","last_became_active":"2024-01-15T18:12:11.042031+0000","last_became_peered":"2024-01-15T18:12:11.042031+0000","last_unstale":"2024-01-15T18:18:23.332457+0000","last_undegraded":"2024-01-15T18:18:23.332457+0000","last_fullsized":"2024-01-15T18:18:23.332457+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'66","last_scrub_stamp":"2024-01-15T18:18:23.332267+0000","last_deep_scrub":"117'66","last_deep_scrub_stamp":"2024-01-15T18:18:23.332267+0000","last_clean_scrub_stamp":"2024-01-15T18:18:23.332267+0000","objects_scrubbed":22,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:38:46.636204+0000","scrub_duration":0.27732246900000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"117'67","reported_seq":104,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:06.389526+0000","last_change":"2024-01-15T18:18:06.389619+0000","last_active":"2024-01-15T18:18:06.389526+0000","last_peered":"2024-01-15T18:18:06.389526+0000","last_clean":"2024-01-15T18:18:06.389526+0000","last_became_active":"2024-01-15T18:12:11.042814+0000","last_became_peered":"2024-01-15T18:12:11.042814+0000","last_unstale":"2024-01-15T18:18:06.389526+0000","last_undegraded":"2024-01-15T18:18:06.389526+0000","last_fullsized":"2024-01-15T18:18:06.389526+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'67","last_scrub_stamp":"2024-01-15T18:18:06.389332+0000","last_deep_scrub":"117'67","last_deep_scrub_stamp":"2024-01-15T18:18:06.389332+0000","last_clean_scrub_stamp":"2024-01-15T18:18:06.389332+0000","objects_scrubbed":21,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:57:01.960290+0000","scrub_duration":0.25979249500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"117'65","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:37.525401+0000","last_change":"2024-01-15T18:18:37.525555+0000","last_active":"2024-01-15T18:18:37.525401+0000","last_peered":"2024-01-15T18:18:37.525401+0000","last_clean":"2024-01-15T18:18:37.525401+0000","last_became_active":"2024-01-15T18:12:11.054790+0000","last_became_peered":"2024-01-15T18:12:11.054790+0000","last_unstale":"2024-01-15T18:18:37.525401+0000","last_undegraded":"2024-01-15T18:18:37.525401+0000","last_fullsized":"2024-01-15T18:18:37.525401+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'65","last_scrub_stamp":"2024-01-15T18:18:37.525171+0000","last_deep_scrub":"117'65","last_deep_scrub_stamp":"2024-01-15T18:18:37.525171+0000","last_clean_scrub_stamp":"2024-01-15T18:18:37.525171+0000","objects_scrubbed":21,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:29:35.251702+0000","scrub_duration":0.267533778,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"117'63","reported_seq":96,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:11.200952+0000","last_change":"2024-01-15T18:18:11.200952+0000","last_active":"2024-01-15T18:18:11.200952+0000","last_peered":"2024-01-15T18:18:11.200952+0000","last_clean":"2024-01-15T18:18:11.200952+0000","last_became_active":"2024-01-15T18:12:11.052289+0000","last_became_peered":"2024-01-15T18:12:11.052289+0000","last_unstale":"2024-01-15T18:18:11.200952+0000","last_undegraded":"2024-01-15T18:18:11.200952+0000","last_fullsized":"2024-01-15T18:18:11.200952+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:11.200610+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:11.200610+0000","last_clean_scrub_stamp":"2024-01-15T18:18:11.200610+0000","objects_scrubbed":17,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:20:42.709925+0000","scrub_duration":0.222891758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"117'82","reported_seq":121,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:20.267040+0000","last_change":"2024-01-15T18:18:20.267040+0000","last_active":"2024-01-15T18:18:20.267040+0000","last_peered":"2024-01-15T18:18:20.267040+0000","last_clean":"2024-01-15T18:18:20.267040+0000","last_became_active":"2024-01-15T18:12:11.036003+0000","last_became_peered":"2024-01-15T18:12:11.036003+0000","last_unstale":"2024-01-15T18:18:20.267040+0000","last_undegraded":"2024-01-15T18:18:20.267040+0000","last_fullsized":"2024-01-15T18:18:20.267040+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'82","last_scrub_stamp":"2024-01-15T18:18:20.266728+0000","last_deep_scrub":"117'82","last_deep_scrub_stamp":"2024-01-15T18:18:20.266728+0000","last_clean_scrub_stamp":"2024-01-15T18:18:20.266728+0000","objects_scrubbed":26,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:16:39.953673+0000","scrub_duration":0.28888784099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13,"num_read_kb":45058,"num_write":28,"num_write_kb":106497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"117'68","reported_seq":101,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:35.796351+0000","last_change":"2024-01-15T18:18:35.796351+0000","last_active":"2024-01-15T18:18:35.796351+0000","last_peered":"2024-01-15T18:18:35.796351+0000","last_clean":"2024-01-15T18:18:35.796351+0000","last_became_active":"2024-01-15T18:12:11.044093+0000","last_became_peered":"2024-01-15T18:12:11.044093+0000","last_unstale":"2024-01-15T18:18:35.796351+0000","last_undegraded":"2024-01-15T18:18:35.796351+0000","last_fullsized":"2024-01-15T18:18:35.796351+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'68","last_scrub_stamp":"2024-01-15T18:18:35.795912+0000","last_deep_scrub":"117'68","last_deep_scrub_stamp":"2024-01-15T18:18:35.795912+0000","last_clean_scrub_stamp":"2024-01-15T18:18:35.795912+0000","objects_scrubbed":21,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:56:50.382706+0000","scrub_duration":0.24977105099999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":8,"num_read_kb":24578,"num_write":23,"num_write_kb":86017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"117'54","reported_seq":88,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:14.102285+0000","last_change":"2024-01-15T18:18:14.102285+0000","last_active":"2024-01-15T18:18:14.102285+0000","last_peered":"2024-01-15T18:18:14.102285+0000","last_clean":"2024-01-15T18:18:14.102285+0000","last_became_active":"2024-01-15T18:12:11.052428+0000","last_became_peered":"2024-01-15T18:12:11.052428+0000","last_unstale":"2024-01-15T18:18:14.102285+0000","last_undegraded":"2024-01-15T18:18:14.102285+0000","last_fullsized":"2024-01-15T18:18:14.102285+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'54","last_scrub_stamp":"2024-01-15T18:18:14.102006+0000","last_deep_scrub":"117'54","last_deep_scrub_stamp":"2024-01-15T18:18:14.102006+0000","last_clean_scrub_stamp":"2024-01-15T18:18:14.102006+0000","objects_scrubbed":15,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:05:29.630951+0000","scrub_duration":0.188245571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"117'61","reported_seq":93,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:19.782542+0000","last_change":"2024-01-15T18:18:19.782542+0000","last_active":"2024-01-15T18:18:19.782542+0000","last_peered":"2024-01-15T18:18:19.782542+0000","last_clean":"2024-01-15T18:18:19.782542+0000","last_became_active":"2024-01-15T18:12:11.051488+0000","last_became_peered":"2024-01-15T18:12:11.051488+0000","last_unstale":"2024-01-15T18:18:19.782542+0000","last_undegraded":"2024-01-15T18:18:19.782542+0000","last_fullsized":"2024-01-15T18:18:19.782542+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'61","last_scrub_stamp":"2024-01-15T18:18:19.782263+0000","last_deep_scrub":"117'61","last_deep_scrub_stamp":"2024-01-15T18:18:19.782263+0000","last_clean_scrub_stamp":"2024-01-15T18:18:19.782263+0000","objects_scrubbed":18,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:45:21.371708+0000","scrub_duration":0.21495783399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"117'56","reported_seq":91,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.157892+0000","last_change":"2024-01-15T18:18:17.157892+0000","last_active":"2024-01-15T18:18:17.157892+0000","last_peered":"2024-01-15T18:18:17.157892+0000","last_clean":"2024-01-15T18:18:17.157892+0000","last_became_active":"2024-01-15T18:12:11.041266+0000","last_became_peered":"2024-01-15T18:12:11.041266+0000","last_unstale":"2024-01-15T18:18:17.157892+0000","last_undegraded":"2024-01-15T18:18:17.157892+0000","last_fullsized":"2024-01-15T18:18:17.157892+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'56","last_scrub_stamp":"2024-01-15T18:18:17.157660+0000","last_deep_scrub":"117'56","last_deep_scrub_stamp":"2024-01-15T18:18:17.157660+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.157660+0000","objects_scrubbed":18,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:31:02.204776+0000","scrub_duration":0.223683413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":10,"num_read_kb":32770,"num_write":20,"num_write_kb":73729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"117'76","reported_seq":109,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:37.637062+0000","last_change":"2024-01-15T18:18:37.637152+0000","last_active":"2024-01-15T18:18:37.637062+0000","last_peered":"2024-01-15T18:18:37.637062+0000","last_clean":"2024-01-15T18:18:37.637062+0000","last_became_active":"2024-01-15T18:12:11.048234+0000","last_became_peered":"2024-01-15T18:12:11.048234+0000","last_unstale":"2024-01-15T18:18:37.637062+0000","last_undegraded":"2024-01-15T18:18:37.637062+0000","last_fullsized":"2024-01-15T18:18:37.637062+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'76","last_scrub_stamp":"2024-01-15T18:18:37.636893+0000","last_deep_scrub":"117'76","last_deep_scrub_stamp":"2024-01-15T18:18:37.636893+0000","last_clean_scrub_stamp":"2024-01-15T18:18:37.636893+0000","objects_scrubbed":23,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:18:29.618282+0000","scrub_duration":0.28808103499999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"117'58","reported_seq":90,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:23.161141+0000","last_change":"2024-01-15T18:18:23.161141+0000","last_active":"2024-01-15T18:18:23.161141+0000","last_peered":"2024-01-15T18:18:23.161141+0000","last_clean":"2024-01-15T18:18:23.161141+0000","last_became_active":"2024-01-15T18:12:11.061724+0000","last_became_peered":"2024-01-15T18:12:11.061724+0000","last_unstale":"2024-01-15T18:18:23.161141+0000","last_undegraded":"2024-01-15T18:18:23.161141+0000","last_fullsized":"2024-01-15T18:18:23.161141+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'58","last_scrub_stamp":"2024-01-15T18:18:23.160963+0000","last_deep_scrub":"117'58","last_deep_scrub_stamp":"2024-01-15T18:18:23.160963+0000","last_clean_scrub_stamp":"2024-01-15T18:18:23.160963+0000","objects_scrubbed":21,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:47:24.937500+0000","scrub_duration":0.21027944000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886260,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":21,"num_write_kb":81921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"117'55","reported_seq":83,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.621391+0000","last_change":"2024-01-15T18:18:41.621391+0000","last_active":"2024-01-15T18:18:41.621391+0000","last_peered":"2024-01-15T18:18:41.621391+0000","last_clean":"2024-01-15T18:18:41.621391+0000","last_became_active":"2024-01-15T18:12:11.043801+0000","last_became_peered":"2024-01-15T18:12:11.043801+0000","last_unstale":"2024-01-15T18:18:41.621391+0000","last_undegraded":"2024-01-15T18:18:41.621391+0000","last_fullsized":"2024-01-15T18:18:41.621391+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'55","last_scrub_stamp":"2024-01-15T18:18:41.621228+0000","last_deep_scrub":"117'55","last_deep_scrub_stamp":"2024-01-15T18:18:41.621228+0000","last_clean_scrub_stamp":"2024-01-15T18:18:41.621228+0000","objects_scrubbed":17,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:12:44.993562+0000","scrub_duration":0.115137798,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":3,"num_read_kb":12288,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"117'72","reported_seq":107,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:19.260856+0000","last_change":"2024-01-15T18:18:19.260856+0000","last_active":"2024-01-15T18:18:19.260856+0000","last_peered":"2024-01-15T18:18:19.260856+0000","last_clean":"2024-01-15T18:18:19.260856+0000","last_became_active":"2024-01-15T18:12:11.047739+0000","last_became_peered":"2024-01-15T18:12:11.047739+0000","last_unstale":"2024-01-15T18:18:19.260856+0000","last_undegraded":"2024-01-15T18:18:19.260856+0000","last_fullsized":"2024-01-15T18:18:19.260856+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'72","last_scrub_stamp":"2024-01-15T18:18:19.260675+0000","last_deep_scrub":"117'72","last_deep_scrub_stamp":"2024-01-15T18:18:19.260675+0000","last_clean_scrub_stamp":"2024-01-15T18:18:19.260675+0000","objects_scrubbed":26,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:05:02.852277+0000","scrub_duration":0.32170053799999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":9,"num_read_kb":36864,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"117'57","reported_seq":91,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.431430+0000","last_change":"2024-01-15T18:18:10.431430+0000","last_active":"2024-01-15T18:18:10.431430+0000","last_peered":"2024-01-15T18:18:10.431430+0000","last_clean":"2024-01-15T18:18:10.431430+0000","last_became_active":"2024-01-15T18:12:11.039393+0000","last_became_peered":"2024-01-15T18:12:11.039393+0000","last_unstale":"2024-01-15T18:18:10.431430+0000","last_undegraded":"2024-01-15T18:18:10.431430+0000","last_fullsized":"2024-01-15T18:18:10.431430+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'57","last_scrub_stamp":"2024-01-15T18:18:10.431168+0000","last_deep_scrub":"117'57","last_deep_scrub_stamp":"2024-01-15T18:18:10.431168+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.431168+0000","objects_scrubbed":16,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:49:04.771755+0000","scrub_duration":0.21320547200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":9,"num_read_kb":32769,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"117'88","reported_seq":126,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:44.553866+0000","last_change":"2024-01-15T18:18:44.553929+0000","last_active":"2024-01-15T18:18:44.553866+0000","last_peered":"2024-01-15T18:18:44.553866+0000","last_clean":"2024-01-15T18:18:44.553866+0000","last_became_active":"2024-01-15T18:12:11.059276+0000","last_became_peered":"2024-01-15T18:12:11.059276+0000","last_unstale":"2024-01-15T18:18:44.553866+0000","last_undegraded":"2024-01-15T18:18:44.553866+0000","last_fullsized":"2024-01-15T18:18:44.553866+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'88","last_scrub_stamp":"2024-01-15T18:18:44.553660+0000","last_deep_scrub":"117'88","last_deep_scrub_stamp":"2024-01-15T18:18:44.553660+0000","last_clean_scrub_stamp":"2024-01-15T18:18:44.553660+0000","objects_scrubbed":26,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:11:16.118748+0000","scrub_duration":0.31460638899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"117'76","reported_seq":108,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:13.141663+0000","last_change":"2024-01-15T18:18:13.141663+0000","last_active":"2024-01-15T18:18:13.141663+0000","last_peered":"2024-01-15T18:18:13.141663+0000","last_clean":"2024-01-15T18:18:13.141663+0000","last_became_active":"2024-01-15T18:12:11.052655+0000","last_became_peered":"2024-01-15T18:12:11.052655+0000","last_unstale":"2024-01-15T18:18:13.141663+0000","last_undegraded":"2024-01-15T18:18:13.141663+0000","last_fullsized":"2024-01-15T18:18:13.141663+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'76","last_scrub_stamp":"2024-01-15T18:18:13.141496+0000","last_deep_scrub":"117'76","last_deep_scrub_stamp":"2024-01-15T18:18:13.141496+0000","last_clean_scrub_stamp":"2024-01-15T18:18:13.141496+0000","objects_scrubbed":22,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:54:46.765713+0000","scrub_duration":0.21211069099999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"117'49","reported_seq":80,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.216437+0000","last_change":"2024-01-15T18:18:10.216437+0000","last_active":"2024-01-15T18:18:10.216437+0000","last_peered":"2024-01-15T18:18:10.216437+0000","last_clean":"2024-01-15T18:18:10.216437+0000","last_became_active":"2024-01-15T18:12:11.053389+0000","last_became_peered":"2024-01-15T18:12:11.053389+0000","last_unstale":"2024-01-15T18:18:10.216437+0000","last_undegraded":"2024-01-15T18:18:10.216437+0000","last_fullsized":"2024-01-15T18:18:10.216437+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'49","last_scrub_stamp":"2024-01-15T18:18:10.216244+0000","last_deep_scrub":"117'49","last_deep_scrub_stamp":"2024-01-15T18:18:10.216244+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.216244+0000","objects_scrubbed":15,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:23:33.117273+0000","scrub_duration":0.16396555099999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62914560,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":15,"num_write_kb":61440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"117'6","reported_seq":32,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:03.244761+0000","last_change":"2024-01-15T18:18:03.244761+0000","last_active":"2024-01-15T18:18:03.244761+0000","last_peered":"2024-01-15T18:18:03.244761+0000","last_clean":"2024-01-15T18:18:03.244761+0000","last_became_active":"2024-01-15T18:12:10.020810+0000","last_became_peered":"2024-01-15T18:12:10.020810+0000","last_unstale":"2024-01-15T18:18:03.244761+0000","last_undegraded":"2024-01-15T18:18:03.244761+0000","last_fullsized":"2024-01-15T18:18:03.244761+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'6","last_scrub_stamp":"2024-01-15T18:18:03.244524+0000","last_deep_scrub":"117'6","last_deep_scrub_stamp":"2024-01-15T18:18:03.244524+0000","last_clean_scrub_stamp":"2024-01-15T18:18:03.244524+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:14:16.865481+0000","scrub_duration":0.0076147139999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"117'66","reported_seq":99,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.420458+0000","last_change":"2024-01-15T18:18:07.420458+0000","last_active":"2024-01-15T18:18:07.420458+0000","last_peered":"2024-01-15T18:18:07.420458+0000","last_clean":"2024-01-15T18:18:07.420458+0000","last_became_active":"2024-01-15T18:12:11.042427+0000","last_became_peered":"2024-01-15T18:12:11.042427+0000","last_unstale":"2024-01-15T18:18:07.420458+0000","last_undegraded":"2024-01-15T18:18:07.420458+0000","last_fullsized":"2024-01-15T18:18:07.420458+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'66","last_scrub_stamp":"2024-01-15T18:18:07.420124+0000","last_deep_scrub":"117'66","last_deep_scrub_stamp":"2024-01-15T18:18:07.420124+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.420124+0000","objects_scrubbed":22,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:18:28.299453+0000","scrub_duration":0.26970153499999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"114'1","reported_seq":27,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:06.548413+0000","last_change":"2024-01-15T18:18:06.548413+0000","last_active":"2024-01-15T18:18:06.548413+0000","last_peered":"2024-01-15T18:18:06.548413+0000","last_clean":"2024-01-15T18:18:06.548413+0000","last_became_active":"2024-01-15T18:12:10.022654+0000","last_became_peered":"2024-01-15T18:12:10.022654+0000","last_unstale":"2024-01-15T18:18:06.548413+0000","last_undegraded":"2024-01-15T18:18:06.548413+0000","last_fullsized":"2024-01-15T18:18:06.548413+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'1","last_scrub_stamp":"2024-01-15T18:18:06.548172+0000","last_deep_scrub":"114'1","last_deep_scrub_stamp":"2024-01-15T18:18:06.548172+0000","last_clean_scrub_stamp":"2024-01-15T18:18:06.548172+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:41:40.711970+0000","scrub_duration":0.003727925,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"117'62","reported_seq":97,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.077838+0000","last_change":"2024-01-15T18:18:17.077904+0000","last_active":"2024-01-15T18:18:17.077838+0000","last_peered":"2024-01-15T18:18:17.077838+0000","last_clean":"2024-01-15T18:18:17.077838+0000","last_became_active":"2024-01-15T18:12:11.052812+0000","last_became_peered":"2024-01-15T18:12:11.052812+0000","last_unstale":"2024-01-15T18:18:17.077838+0000","last_undegraded":"2024-01-15T18:18:17.077838+0000","last_fullsized":"2024-01-15T18:18:17.077838+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'62","last_scrub_stamp":"2024-01-15T18:18:17.077727+0000","last_deep_scrub":"117'62","last_deep_scrub_stamp":"2024-01-15T18:18:17.077727+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.077727+0000","objects_scrubbed":19,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:22.148520+0000","scrub_duration":0.19122593199999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"117'4","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.759072+0000","last_change":"2024-01-15T18:17:58.759128+0000","last_active":"2024-01-15T18:17:58.759072+0000","last_peered":"2024-01-15T18:17:58.759072+0000","last_clean":"2024-01-15T18:17:58.759072+0000","last_became_active":"2024-01-15T18:12:10.018670+0000","last_became_peered":"2024-01-15T18:12:10.018670+0000","last_unstale":"2024-01-15T18:17:58.759072+0000","last_undegraded":"2024-01-15T18:17:58.759072+0000","last_fullsized":"2024-01-15T18:17:58.759072+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'4","last_scrub_stamp":"2024-01-15T18:17:58.758892+0000","last_deep_scrub":"117'4","last_deep_scrub_stamp":"2024-01-15T18:17:58.758892+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.758892+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:09:01.602528+0000","scrub_duration":0.0068205260000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"117'79","reported_seq":114,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:41.627853+0000","last_change":"2024-01-15T18:18:41.627938+0000","last_active":"2024-01-15T18:18:41.627853+0000","last_peered":"2024-01-15T18:18:41.627853+0000","last_clean":"2024-01-15T18:18:41.627853+0000","last_became_active":"2024-01-15T18:12:11.057592+0000","last_became_peered":"2024-01-15T18:12:11.057592+0000","last_unstale":"2024-01-15T18:18:41.627853+0000","last_undegraded":"2024-01-15T18:18:41.627853+0000","last_fullsized":"2024-01-15T18:18:41.627853+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'79","last_scrub_stamp":"2024-01-15T18:18:41.627660+0000","last_deep_scrub":"117'79","last_deep_scrub_stamp":"2024-01-15T18:18:41.627660+0000","last_clean_scrub_stamp":"2024-01-15T18:18:41.627660+0000","objects_scrubbed":23,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:18:09.916511+0000","scrub_duration":0.26311400099999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"117'3","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:57.800153+0000","last_change":"2024-01-15T18:17:57.800193+0000","last_active":"2024-01-15T18:17:57.800153+0000","last_peered":"2024-01-15T18:17:57.800153+0000","last_clean":"2024-01-15T18:17:57.800153+0000","last_became_active":"2024-01-15T18:12:10.020143+0000","last_became_peered":"2024-01-15T18:12:10.020143+0000","last_unstale":"2024-01-15T18:17:57.800153+0000","last_undegraded":"2024-01-15T18:17:57.800153+0000","last_fullsized":"2024-01-15T18:17:57.800153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'3","last_scrub_stamp":"2024-01-15T18:17:57.800057+0000","last_deep_scrub":"117'3","last_deep_scrub_stamp":"2024-01-15T18:17:57.800057+0000","last_clean_scrub_stamp":"2024-01-15T18:17:57.800057+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:23:19.671055+0000","scrub_duration":0.0055516410000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"117'56","reported_seq":88,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:33.493481+0000","last_change":"2024-01-15T18:18:33.493543+0000","last_active":"2024-01-15T18:18:33.493481+0000","last_peered":"2024-01-15T18:18:33.493481+0000","last_clean":"2024-01-15T18:18:33.493481+0000","last_became_active":"2024-01-15T18:12:11.057471+0000","last_became_peered":"2024-01-15T18:12:11.057471+0000","last_unstale":"2024-01-15T18:18:33.493481+0000","last_undegraded":"2024-01-15T18:18:33.493481+0000","last_fullsized":"2024-01-15T18:18:33.493481+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'56","last_scrub_stamp":"2024-01-15T18:18:33.493371+0000","last_deep_scrub":"117'56","last_deep_scrub_stamp":"2024-01-15T18:18:33.493371+0000","last_clean_scrub_stamp":"2024-01-15T18:18:33.493371+0000","objects_scrubbed":17,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:30:13.195463+0000","scrub_duration":0.16716834899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"114'1","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:05.257137+0000","last_change":"2024-01-15T18:18:05.257194+0000","last_active":"2024-01-15T18:18:05.257137+0000","last_peered":"2024-01-15T18:18:05.257137+0000","last_clean":"2024-01-15T18:18:05.257137+0000","last_became_active":"2024-01-15T18:12:10.019818+0000","last_became_peered":"2024-01-15T18:12:10.019818+0000","last_unstale":"2024-01-15T18:18:05.257137+0000","last_undegraded":"2024-01-15T18:18:05.257137+0000","last_fullsized":"2024-01-15T18:18:05.257137+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'1","last_scrub_stamp":"2024-01-15T18:18:05.256946+0000","last_deep_scrub":"114'1","last_deep_scrub_stamp":"2024-01-15T18:18:05.256946+0000","last_clean_scrub_stamp":"2024-01-15T18:18:05.256946+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:47:07.033483+0000","scrub_duration":0.0043509610000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"117'73","reported_seq":110,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:13.209425+0000","last_change":"2024-01-15T18:18:13.209530+0000","last_active":"2024-01-15T18:18:13.209425+0000","last_peered":"2024-01-15T18:18:13.209425+0000","last_clean":"2024-01-15T18:18:13.209425+0000","last_became_active":"2024-01-15T18:12:11.050002+0000","last_became_peered":"2024-01-15T18:12:11.050002+0000","last_unstale":"2024-01-15T18:18:13.209425+0000","last_undegraded":"2024-01-15T18:18:13.209425+0000","last_fullsized":"2024-01-15T18:18:13.209425+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'73","last_scrub_stamp":"2024-01-15T18:18:13.209243+0000","last_deep_scrub":"117'73","last_deep_scrub_stamp":"2024-01-15T18:18:13.209243+0000","last_clean_scrub_stamp":"2024-01-15T18:18:13.209243+0000","objects_scrubbed":24,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:23:57.942383+0000","scrub_duration":0.27580295799999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":13,"num_read_kb":53248,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"114'1","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.489503+0000","last_change":"2024-01-15T18:18:00.489523+0000","last_active":"2024-01-15T18:18:00.489503+0000","last_peered":"2024-01-15T18:18:00.489503+0000","last_clean":"2024-01-15T18:18:00.489503+0000","last_became_active":"2024-01-15T18:12:10.009676+0000","last_became_peered":"2024-01-15T18:12:10.009676+0000","last_unstale":"2024-01-15T18:18:00.489503+0000","last_undegraded":"2024-01-15T18:18:00.489503+0000","last_fullsized":"2024-01-15T18:18:00.489503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"114'1","last_scrub_stamp":"2024-01-15T18:18:00.489431+0000","last_deep_scrub":"114'1","last_deep_scrub_stamp":"2024-01-15T18:18:00.489431+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.489431+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:33:48.954551+0000","scrub_duration":0.00316505,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"117'79","reported_seq":114,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:28.702871+0000","last_change":"2024-01-15T18:18:28.702924+0000","last_active":"2024-01-15T18:18:28.702871+0000","last_peered":"2024-01-15T18:18:28.702871+0000","last_clean":"2024-01-15T18:18:28.702871+0000","last_became_active":"2024-01-15T18:12:11.057640+0000","last_became_peered":"2024-01-15T18:12:11.057640+0000","last_unstale":"2024-01-15T18:18:28.702871+0000","last_undegraded":"2024-01-15T18:18:28.702871+0000","last_fullsized":"2024-01-15T18:18:28.702871+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'79","last_scrub_stamp":"2024-01-15T18:18:28.702706+0000","last_deep_scrub":"117'79","last_deep_scrub_stamp":"2024-01-15T18:18:28.702706+0000","last_clean_scrub_stamp":"2024-01-15T18:18:28.702706+0000","objects_scrubbed":26,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:39:40.150910+0000","scrub_duration":0.27044170200000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857791,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":10,"num_read_kb":36865,"num_write":26,"num_write_kb":102401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.801159+0000","last_change":"2024-01-15T18:17:58.801207+0000","last_active":"2024-01-15T18:17:58.801159+0000","last_peered":"2024-01-15T18:17:58.801159+0000","last_clean":"2024-01-15T18:17:58.801159+0000","last_became_active":"2024-01-15T18:12:10.002190+0000","last_became_peered":"2024-01-15T18:12:10.002190+0000","last_unstale":"2024-01-15T18:17:58.801159+0000","last_undegraded":"2024-01-15T18:17:58.801159+0000","last_fullsized":"2024-01-15T18:17:58.801159+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:58.800969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:58.800969+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.800969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:04:55.566839+0000","scrub_duration":0.0032888890000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"117'59","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.379605+0000","last_change":"2024-01-15T18:18:17.379679+0000","last_active":"2024-01-15T18:18:17.379605+0000","last_peered":"2024-01-15T18:18:17.379605+0000","last_clean":"2024-01-15T18:18:17.379605+0000","last_became_active":"2024-01-15T18:12:11.046934+0000","last_became_peered":"2024-01-15T18:12:11.046934+0000","last_unstale":"2024-01-15T18:18:17.379605+0000","last_undegraded":"2024-01-15T18:18:17.379605+0000","last_fullsized":"2024-01-15T18:18:17.379605+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'59","last_scrub_stamp":"2024-01-15T18:18:17.379455+0000","last_deep_scrub":"117'59","last_deep_scrub_stamp":"2024-01-15T18:18:17.379455+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.379455+0000","objects_scrubbed":16,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:14.145325+0000","scrub_duration":0.19649055300000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":18,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:19.441143+0000","last_change":"2024-01-15T18:18:19.441198+0000","last_active":"2024-01-15T18:18:19.441143+0000","last_peered":"2024-01-15T18:18:19.441143+0000","last_clean":"2024-01-15T18:18:19.441143+0000","last_became_active":"2024-01-15T18:12:10.020577+0000","last_became_peered":"2024-01-15T18:12:10.020577+0000","last_unstale":"2024-01-15T18:18:19.441143+0000","last_undegraded":"2024-01-15T18:18:19.441143+0000","last_fullsized":"2024-01-15T18:18:19.441143+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:19.440940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:19.440940+0000","last_clean_scrub_stamp":"2024-01-15T18:18:19.440940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:44:56.813275+0000","scrub_duration":0.0035696320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"117'50","reported_seq":78,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.308905+0000","last_change":"2024-01-15T18:18:15.308980+0000","last_active":"2024-01-15T18:18:15.308905+0000","last_peered":"2024-01-15T18:18:15.308905+0000","last_clean":"2024-01-15T18:18:15.308905+0000","last_became_active":"2024-01-15T18:12:11.047704+0000","last_became_peered":"2024-01-15T18:12:11.047704+0000","last_unstale":"2024-01-15T18:18:15.308905+0000","last_undegraded":"2024-01-15T18:18:15.308905+0000","last_fullsized":"2024-01-15T18:18:15.308905+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'50","last_scrub_stamp":"2024-01-15T18:18:15.308693+0000","last_deep_scrub":"117'50","last_deep_scrub_stamp":"2024-01-15T18:18:15.308693+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.308693+0000","objects_scrubbed":8,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:28:38.450367+0000","scrub_duration":0.093927007000000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":8,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"117'1","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.742613+0000","last_change":"2024-01-15T18:18:01.742668+0000","last_active":"2024-01-15T18:18:01.742613+0000","last_peered":"2024-01-15T18:18:01.742613+0000","last_clean":"2024-01-15T18:18:01.742613+0000","last_became_active":"2024-01-15T18:12:10.020028+0000","last_became_peered":"2024-01-15T18:12:10.020028+0000","last_unstale":"2024-01-15T18:18:01.742613+0000","last_undegraded":"2024-01-15T18:18:01.742613+0000","last_fullsized":"2024-01-15T18:18:01.742613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'1","last_scrub_stamp":"2024-01-15T18:18:01.742421+0000","last_deep_scrub":"117'1","last_deep_scrub_stamp":"2024-01-15T18:18:01.742421+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.742421+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:19:38.888476+0000","scrub_duration":0.0037945140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"117'55","reported_seq":84,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.374401+0000","last_change":"2024-01-15T18:18:09.374484+0000","last_active":"2024-01-15T18:18:09.374401+0000","last_peered":"2024-01-15T18:18:09.374401+0000","last_clean":"2024-01-15T18:18:09.374401+0000","last_became_active":"2024-01-15T18:12:11.059393+0000","last_became_peered":"2024-01-15T18:12:11.059393+0000","last_unstale":"2024-01-15T18:18:09.374401+0000","last_undegraded":"2024-01-15T18:18:09.374401+0000","last_fullsized":"2024-01-15T18:18:09.374401+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'55","last_scrub_stamp":"2024-01-15T18:18:09.374248+0000","last_deep_scrub":"117'55","last_deep_scrub_stamp":"2024-01-15T18:18:09.374248+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.374248+0000","objects_scrubbed":18,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:44:26.424477+0000","scrub_duration":0.23458093399999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":18,"num_write_kb":73728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.527582+0000","last_change":"2024-01-15T18:17:58.527616+0000","last_active":"2024-01-15T18:17:58.527582+0000","last_peered":"2024-01-15T18:17:58.527582+0000","last_clean":"2024-01-15T18:17:58.527582+0000","last_became_active":"2024-01-15T18:12:10.007549+0000","last_became_peered":"2024-01-15T18:12:10.007549+0000","last_unstale":"2024-01-15T18:17:58.527582+0000","last_undegraded":"2024-01-15T18:17:58.527582+0000","last_fullsized":"2024-01-15T18:17:58.527582+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:58.527475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:58.527475+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.527475+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:17:32.463562+0000","scrub_duration":0.003638007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"117'65","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:46.819309+0000","last_change":"2024-01-15T18:18:46.819410+0000","last_active":"2024-01-15T18:18:46.819309+0000","last_peered":"2024-01-15T18:18:46.819309+0000","last_clean":"2024-01-15T18:18:46.819309+0000","last_became_active":"2024-01-15T18:12:11.030958+0000","last_became_peered":"2024-01-15T18:12:11.030958+0000","last_unstale":"2024-01-15T18:18:46.819309+0000","last_undegraded":"2024-01-15T18:18:46.819309+0000","last_fullsized":"2024-01-15T18:18:46.819309+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'65","last_scrub_stamp":"2024-01-15T18:18:46.819118+0000","last_deep_scrub":"117'65","last_deep_scrub_stamp":"2024-01-15T18:18:46.819118+0000","last_clean_scrub_stamp":"2024-01-15T18:18:46.819118+0000","objects_scrubbed":22,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:05.674854+0000","scrub_duration":0.24515778999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"117'14","reported_seq":39,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:14.392225+0000","last_change":"2024-01-15T18:18:14.392289+0000","last_active":"2024-01-15T18:18:14.392225+0000","last_peered":"2024-01-15T18:18:14.392225+0000","last_clean":"2024-01-15T18:18:14.392225+0000","last_became_active":"2024-01-15T18:12:10.024120+0000","last_became_peered":"2024-01-15T18:12:10.024120+0000","last_unstale":"2024-01-15T18:18:14.392225+0000","last_undegraded":"2024-01-15T18:18:14.392225+0000","last_fullsized":"2024-01-15T18:18:14.392225+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'14","last_scrub_stamp":"2024-01-15T18:18:14.391949+0000","last_deep_scrub":"117'14","last_deep_scrub_stamp":"2024-01-15T18:18:14.391949+0000","last_clean_scrub_stamp":"2024-01-15T18:18:14.391949+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:31:42.989864+0000","scrub_duration":0.0085194199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"117'63","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:21.473343+0000","last_change":"2024-01-15T18:18:21.473422+0000","last_active":"2024-01-15T18:18:21.473343+0000","last_peered":"2024-01-15T18:18:21.473343+0000","last_clean":"2024-01-15T18:18:21.473343+0000","last_became_active":"2024-01-15T18:12:11.055290+0000","last_became_peered":"2024-01-15T18:12:11.055290+0000","last_unstale":"2024-01-15T18:18:21.473343+0000","last_undegraded":"2024-01-15T18:18:21.473343+0000","last_fullsized":"2024-01-15T18:18:21.473343+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:21.473208+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:21.473208+0000","last_clean_scrub_stamp":"2024-01-15T18:18:21.473208+0000","objects_scrubbed":20,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:26.591651+0000","scrub_duration":0.266702363,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":20,"num_write_kb":81920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"117'190","reported_seq":215,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:51.878290+0000","last_change":"2024-01-15T18:18:16.445450+0000","last_active":"2024-01-15T18:18:51.878290+0000","last_peered":"2024-01-15T18:18:51.878290+0000","last_clean":"2024-01-15T18:18:51.878290+0000","last_became_active":"2024-01-15T18:12:10.021497+0000","last_became_peered":"2024-01-15T18:12:10.021497+0000","last_unstale":"2024-01-15T18:18:51.878290+0000","last_undegraded":"2024-01-15T18:18:51.878290+0000","last_fullsized":"2024-01-15T18:18:51.878290+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'188","last_scrub_stamp":"2024-01-15T18:18:16.445190+0000","last_deep_scrub":"117'188","last_deep_scrub_stamp":"2024-01-15T18:18:16.445190+0000","last_clean_scrub_stamp":"2024-01-15T18:18:16.445190+0000","objects_scrubbed":4,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:09:41.548591+0000","scrub_duration":0.015015664,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639592,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":187,"num_write_kb":719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"117'65","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:42.772520+0000","last_change":"2024-01-15T18:18:42.772614+0000","last_active":"2024-01-15T18:18:42.772520+0000","last_peered":"2024-01-15T18:18:42.772520+0000","last_clean":"2024-01-15T18:18:42.772520+0000","last_became_active":"2024-01-15T18:12:11.045774+0000","last_became_peered":"2024-01-15T18:12:11.045774+0000","last_unstale":"2024-01-15T18:18:42.772520+0000","last_undegraded":"2024-01-15T18:18:42.772520+0000","last_fullsized":"2024-01-15T18:18:42.772520+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'65","last_scrub_stamp":"2024-01-15T18:18:42.772338+0000","last_deep_scrub":"117'65","last_deep_scrub_stamp":"2024-01-15T18:18:42.772338+0000","last_clean_scrub_stamp":"2024-01-15T18:18:42.772338+0000","objects_scrubbed":21,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:18:37.461547+0000","scrub_duration":0.231393029,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":24576,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"117'4","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.064488+0000","last_change":"2024-01-15T18:18:07.064553+0000","last_active":"2024-01-15T18:18:07.064488+0000","last_peered":"2024-01-15T18:18:07.064488+0000","last_clean":"2024-01-15T18:18:07.064488+0000","last_became_active":"2024-01-15T18:12:10.016618+0000","last_became_peered":"2024-01-15T18:12:10.016618+0000","last_unstale":"2024-01-15T18:18:07.064488+0000","last_undegraded":"2024-01-15T18:18:07.064488+0000","last_fullsized":"2024-01-15T18:18:07.064488+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'4","last_scrub_stamp":"2024-01-15T18:18:07.064304+0000","last_deep_scrub":"117'4","last_deep_scrub_stamp":"2024-01-15T18:18:07.064304+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.064304+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:47:55.108425+0000","scrub_duration":0.0089004340000000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"117'73","reported_seq":102,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:36.738982+0000","last_change":"2024-01-15T18:18:36.739074+0000","last_active":"2024-01-15T18:18:36.738982+0000","last_peered":"2024-01-15T18:18:36.738982+0000","last_clean":"2024-01-15T18:18:36.738982+0000","last_became_active":"2024-01-15T18:12:11.050928+0000","last_became_peered":"2024-01-15T18:12:11.050928+0000","last_unstale":"2024-01-15T18:18:36.738982+0000","last_undegraded":"2024-01-15T18:18:36.738982+0000","last_fullsized":"2024-01-15T18:18:36.738982+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'73","last_scrub_stamp":"2024-01-15T18:18:36.738814+0000","last_deep_scrub":"117'73","last_deep_scrub_stamp":"2024-01-15T18:18:36.738814+0000","last_clean_scrub_stamp":"2024-01-15T18:18:36.738814+0000","objects_scrubbed":21,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:42.767283+0000","scrub_duration":0.23757112899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":21,"num_write_kb":86016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:04.129497+0000","last_change":"2024-01-15T18:18:04.129593+0000","last_active":"2024-01-15T18:18:04.129497+0000","last_peered":"2024-01-15T18:18:04.129497+0000","last_clean":"2024-01-15T18:18:04.129497+0000","last_became_active":"2024-01-15T18:12:10.015578+0000","last_became_peered":"2024-01-15T18:12:10.015578+0000","last_unstale":"2024-01-15T18:18:04.129497+0000","last_undegraded":"2024-01-15T18:18:04.129497+0000","last_fullsized":"2024-01-15T18:18:04.129497+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:04.129272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:04.129272+0000","last_clean_scrub_stamp":"2024-01-15T18:18:04.129272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:51:09.247715+0000","scrub_duration":0.003855626,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"117'68","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:03.667643+0000","last_change":"2024-01-15T18:18:03.667703+0000","last_active":"2024-01-15T18:18:03.667643+0000","last_peered":"2024-01-15T18:18:03.667643+0000","last_clean":"2024-01-15T18:18:03.667643+0000","last_became_active":"2024-01-15T18:12:11.028865+0000","last_became_peered":"2024-01-15T18:12:11.028865+0000","last_unstale":"2024-01-15T18:18:03.667643+0000","last_undegraded":"2024-01-15T18:18:03.667643+0000","last_fullsized":"2024-01-15T18:18:03.667643+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'68","last_scrub_stamp":"2024-01-15T18:18:03.667447+0000","last_deep_scrub":"117'68","last_deep_scrub_stamp":"2024-01-15T18:18:03.667447+0000","last_clean_scrub_stamp":"2024-01-15T18:18:03.667447+0000","objects_scrubbed":25,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:28:27.833994+0000","scrub_duration":0.20722174700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663556,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":25,"num_write_kb":98305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"117'12","reported_seq":37,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:56.807714+0000","last_change":"2024-01-15T18:17:56.807742+0000","last_active":"2024-01-15T18:17:56.807714+0000","last_peered":"2024-01-15T18:17:56.807714+0000","last_clean":"2024-01-15T18:17:56.807714+0000","last_became_active":"2024-01-15T18:12:10.022398+0000","last_became_peered":"2024-01-15T18:12:10.022398+0000","last_unstale":"2024-01-15T18:17:56.807714+0000","last_undegraded":"2024-01-15T18:17:56.807714+0000","last_fullsized":"2024-01-15T18:17:56.807714+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'12","last_scrub_stamp":"2024-01-15T18:17:56.807649+0000","last_deep_scrub":"117'12","last_deep_scrub_stamp":"2024-01-15T18:17:56.807649+0000","last_clean_scrub_stamp":"2024-01-15T18:17:56.807649+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:43.916577+0000","scrub_duration":0.008646964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"117'34","reported_seq":63,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.960886+0000","last_change":"2024-01-15T18:18:10.960886+0000","last_active":"2024-01-15T18:18:10.960886+0000","last_peered":"2024-01-15T18:18:10.960886+0000","last_clean":"2024-01-15T18:18:10.960886+0000","last_became_active":"2024-01-15T18:12:11.043735+0000","last_became_peered":"2024-01-15T18:12:11.043735+0000","last_unstale":"2024-01-15T18:18:10.960886+0000","last_undegraded":"2024-01-15T18:18:10.960886+0000","last_fullsized":"2024-01-15T18:18:10.960886+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'34","last_scrub_stamp":"2024-01-15T18:18:10.960638+0000","last_deep_scrub":"117'34","last_deep_scrub_stamp":"2024-01-15T18:18:10.960638+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.960638+0000","objects_scrubbed":10,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:14:24.581595+0000","scrub_duration":0.105112926,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41943040,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":10,"num_write_kb":40960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"117'2","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:20.431973+0000","last_change":"2024-01-15T18:18:20.431973+0000","last_active":"2024-01-15T18:18:20.431973+0000","last_peered":"2024-01-15T18:18:20.431973+0000","last_clean":"2024-01-15T18:18:20.431973+0000","last_became_active":"2024-01-15T18:12:10.026730+0000","last_became_peered":"2024-01-15T18:12:10.026730+0000","last_unstale":"2024-01-15T18:18:20.431973+0000","last_undegraded":"2024-01-15T18:18:20.431973+0000","last_fullsized":"2024-01-15T18:18:20.431973+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'2","last_scrub_stamp":"2024-01-15T18:18:20.431712+0000","last_deep_scrub":"117'2","last_deep_scrub_stamp":"2024-01-15T18:18:20.431712+0000","last_clean_scrub_stamp":"2024-01-15T18:18:20.431712+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:45:32.145232+0000","scrub_duration":0.0084741669999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"117'53","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:21.086448+0000","last_change":"2024-01-15T18:18:21.086448+0000","last_active":"2024-01-15T18:18:21.086448+0000","last_peered":"2024-01-15T18:18:21.086448+0000","last_clean":"2024-01-15T18:18:21.086448+0000","last_became_active":"2024-01-15T18:12:11.061002+0000","last_became_peered":"2024-01-15T18:12:11.061002+0000","last_unstale":"2024-01-15T18:18:21.086448+0000","last_undegraded":"2024-01-15T18:18:21.086448+0000","last_fullsized":"2024-01-15T18:18:21.086448+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'53","last_scrub_stamp":"2024-01-15T18:18:21.086323+0000","last_deep_scrub":"117'53","last_deep_scrub_stamp":"2024-01-15T18:18:21.086323+0000","last_clean_scrub_stamp":"2024-01-15T18:18:21.086323+0000","objects_scrubbed":17,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:24:51.309071+0000","scrub_duration":0.12936230400000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":11,"num_read_kb":45056,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"117'3","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:07.859110+0000","last_change":"2024-01-15T18:18:07.859110+0000","last_active":"2024-01-15T18:18:07.859110+0000","last_peered":"2024-01-15T18:18:07.859110+0000","last_clean":"2024-01-15T18:18:07.859110+0000","last_became_active":"2024-01-15T18:12:10.019706+0000","last_became_peered":"2024-01-15T18:12:10.019706+0000","last_unstale":"2024-01-15T18:18:07.859110+0000","last_undegraded":"2024-01-15T18:18:07.859110+0000","last_fullsized":"2024-01-15T18:18:07.859110+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'3","last_scrub_stamp":"2024-01-15T18:18:07.858872+0000","last_deep_scrub":"117'3","last_deep_scrub_stamp":"2024-01-15T18:18:07.858872+0000","last_clean_scrub_stamp":"2024-01-15T18:18:07.858872+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:12:03.175146+0000","scrub_duration":0.0077351470000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"117'62","reported_seq":98,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:23.538485+0000","last_change":"2024-01-15T18:18:23.538485+0000","last_active":"2024-01-15T18:18:23.538485+0000","last_peered":"2024-01-15T18:18:23.538485+0000","last_clean":"2024-01-15T18:18:23.538485+0000","last_became_active":"2024-01-15T18:12:11.060154+0000","last_became_peered":"2024-01-15T18:12:11.060154+0000","last_unstale":"2024-01-15T18:18:23.538485+0000","last_undegraded":"2024-01-15T18:18:23.538485+0000","last_fullsized":"2024-01-15T18:18:23.538485+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'62","last_scrub_stamp":"2024-01-15T18:18:23.538243+0000","last_deep_scrub":"117'62","last_deep_scrub_stamp":"2024-01-15T18:18:23.538243+0000","last_clean_scrub_stamp":"2024-01-15T18:18:23.538243+0000","objects_scrubbed":18,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:11:51.960622+0000","scrub_duration":0.202197823,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75497472,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":11,"num_read_kb":28676,"num_write":22,"num_write_kb":73730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"117'2","reported_seq":28,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.161747+0000","last_change":"2024-01-15T18:18:01.161747+0000","last_active":"2024-01-15T18:18:01.161747+0000","last_peered":"2024-01-15T18:18:01.161747+0000","last_clean":"2024-01-15T18:18:01.161747+0000","last_became_active":"2024-01-15T18:12:10.002957+0000","last_became_peered":"2024-01-15T18:12:10.002957+0000","last_unstale":"2024-01-15T18:18:01.161747+0000","last_undegraded":"2024-01-15T18:18:01.161747+0000","last_fullsized":"2024-01-15T18:18:01.161747+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'2","last_scrub_stamp":"2024-01-15T18:18:01.161505+0000","last_deep_scrub":"117'2","last_deep_scrub_stamp":"2024-01-15T18:18:01.161505+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.161505+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:25:01.536885+0000","scrub_duration":0.0071468110000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"117'63","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:39.569719+0000","last_change":"2024-01-15T18:18:39.569719+0000","last_active":"2024-01-15T18:18:39.569719+0000","last_peered":"2024-01-15T18:18:39.569719+0000","last_clean":"2024-01-15T18:18:39.569719+0000","last_became_active":"2024-01-15T18:12:11.058945+0000","last_became_peered":"2024-01-15T18:12:11.058945+0000","last_unstale":"2024-01-15T18:18:39.569719+0000","last_undegraded":"2024-01-15T18:18:39.569719+0000","last_fullsized":"2024-01-15T18:18:39.569719+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:39.569491+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:39.569491+0000","last_clean_scrub_stamp":"2024-01-15T18:18:39.569491+0000","objects_scrubbed":22,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:45:16.941826+0000","scrub_duration":0.27134443600000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:06.288945+0000","last_change":"2024-01-15T18:18:06.288945+0000","last_active":"2024-01-15T18:18:06.288945+0000","last_peered":"2024-01-15T18:18:06.288945+0000","last_clean":"2024-01-15T18:18:06.288945+0000","last_became_active":"2024-01-15T18:12:10.028284+0000","last_became_peered":"2024-01-15T18:12:10.028284+0000","last_unstale":"2024-01-15T18:18:06.288945+0000","last_undegraded":"2024-01-15T18:18:06.288945+0000","last_fullsized":"2024-01-15T18:18:06.288945+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:06.288498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:06.288498+0000","last_clean_scrub_stamp":"2024-01-15T18:18:06.288498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:21:19.512303+0000","scrub_duration":0.0041979610000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"117'75","reported_seq":109,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:15.815534+0000","last_change":"2024-01-15T18:18:15.815534+0000","last_active":"2024-01-15T18:18:15.815534+0000","last_peered":"2024-01-15T18:18:15.815534+0000","last_clean":"2024-01-15T18:18:15.815534+0000","last_became_active":"2024-01-15T18:12:11.051403+0000","last_became_peered":"2024-01-15T18:12:11.051403+0000","last_unstale":"2024-01-15T18:18:15.815534+0000","last_undegraded":"2024-01-15T18:18:15.815534+0000","last_fullsized":"2024-01-15T18:18:15.815534+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'75","last_scrub_stamp":"2024-01-15T18:18:15.815153+0000","last_deep_scrub":"117'75","last_deep_scrub_stamp":"2024-01-15T18:18:15.815153+0000","last_clean_scrub_stamp":"2024-01-15T18:18:15.815153+0000","objects_scrubbed":30,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:39:09.886495+0000","scrub_duration":0.40068222799999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":30,"num_write_kb":122880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"117'4","reported_seq":30,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:04.093163+0000","last_change":"2024-01-15T18:18:04.093163+0000","last_active":"2024-01-15T18:18:04.093163+0000","last_peered":"2024-01-15T18:18:04.093163+0000","last_clean":"2024-01-15T18:18:04.093163+0000","last_became_active":"2024-01-15T18:12:10.021344+0000","last_became_peered":"2024-01-15T18:12:10.021344+0000","last_unstale":"2024-01-15T18:18:04.093163+0000","last_undegraded":"2024-01-15T18:18:04.093163+0000","last_fullsized":"2024-01-15T18:18:04.093163+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'4","last_scrub_stamp":"2024-01-15T18:18:04.092822+0000","last_deep_scrub":"117'4","last_deep_scrub_stamp":"2024-01-15T18:18:04.092822+0000","last_clean_scrub_stamp":"2024-01-15T18:18:04.092822+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:23:51.201750+0000","scrub_duration":0.0068652299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"117'62","reported_seq":94,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:50.388144+0000","last_change":"2024-01-15T18:18:50.388144+0000","last_active":"2024-01-15T18:18:50.388144+0000","last_peered":"2024-01-15T18:18:50.388144+0000","last_clean":"2024-01-15T18:18:50.388144+0000","last_became_active":"2024-01-15T18:12:11.059311+0000","last_became_peered":"2024-01-15T18:12:11.059311+0000","last_unstale":"2024-01-15T18:18:50.388144+0000","last_undegraded":"2024-01-15T18:18:50.388144+0000","last_fullsized":"2024-01-15T18:18:50.388144+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'62","last_scrub_stamp":"2024-01-15T18:18:50.387862+0000","last_deep_scrub":"117'62","last_deep_scrub_stamp":"2024-01-15T18:18:50.387862+0000","last_clean_scrub_stamp":"2024-01-15T18:18:50.387862+0000","objects_scrubbed":19,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:14:59.502693+0000","scrub_duration":0.215974153,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79691776,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":77824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.698919+0000","last_change":"2024-01-15T18:18:00.698919+0000","last_active":"2024-01-15T18:18:00.698919+0000","last_peered":"2024-01-15T18:18:00.698919+0000","last_clean":"2024-01-15T18:18:00.698919+0000","last_became_active":"2024-01-15T18:12:10.017411+0000","last_became_peered":"2024-01-15T18:12:10.017411+0000","last_unstale":"2024-01-15T18:18:00.698919+0000","last_undegraded":"2024-01-15T18:18:00.698919+0000","last_fullsized":"2024-01-15T18:18:00.698919+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:00.698524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:00.698524+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.698524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:05:24.402242+0000","scrub_duration":0.0031317810000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"117'61","reported_seq":91,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:29.517272+0000","last_change":"2024-01-15T18:18:29.517272+0000","last_active":"2024-01-15T18:18:29.517272+0000","last_peered":"2024-01-15T18:18:29.517272+0000","last_clean":"2024-01-15T18:18:29.517272+0000","last_became_active":"2024-01-15T18:12:11.056311+0000","last_became_peered":"2024-01-15T18:12:11.056311+0000","last_unstale":"2024-01-15T18:18:29.517272+0000","last_undegraded":"2024-01-15T18:18:29.517272+0000","last_fullsized":"2024-01-15T18:18:29.517272+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'61","last_scrub_stamp":"2024-01-15T18:18:29.517021+0000","last_deep_scrub":"117'61","last_deep_scrub_stamp":"2024-01-15T18:18:29.517021+0000","last_clean_scrub_stamp":"2024-01-15T18:18:29.517021+0000","objects_scrubbed":17,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:05:41.064706+0000","scrub_duration":0.192502231,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5,"num_read_kb":20480,"num_write":17,"num_write_kb":69632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:58.185126+0000","last_change":"2024-01-15T18:17:58.185126+0000","last_active":"2024-01-15T18:17:58.185126+0000","last_peered":"2024-01-15T18:17:58.185126+0000","last_clean":"2024-01-15T18:17:58.185126+0000","last_became_active":"2024-01-15T18:12:10.022687+0000","last_became_peered":"2024-01-15T18:12:10.022687+0000","last_unstale":"2024-01-15T18:17:58.185126+0000","last_undegraded":"2024-01-15T18:17:58.185126+0000","last_fullsized":"2024-01-15T18:17:58.185126+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:58.185008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:58.185008+0000","last_clean_scrub_stamp":"2024-01-15T18:17:58.185008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:00:35.863748+0000","scrub_duration":0.0025740260000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"117'59","reported_seq":91,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:27.774812+0000","last_change":"2024-01-15T18:18:27.774812+0000","last_active":"2024-01-15T18:18:27.774812+0000","last_peered":"2024-01-15T18:18:27.774812+0000","last_clean":"2024-01-15T18:18:27.774812+0000","last_became_active":"2024-01-15T18:12:11.061830+0000","last_became_peered":"2024-01-15T18:12:11.061830+0000","last_unstale":"2024-01-15T18:18:27.774812+0000","last_undegraded":"2024-01-15T18:18:27.774812+0000","last_fullsized":"2024-01-15T18:18:27.774812+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'59","last_scrub_stamp":"2024-01-15T18:18:27.774550+0000","last_deep_scrub":"117'59","last_deep_scrub_stamp":"2024-01-15T18:18:27.774550+0000","last_clean_scrub_stamp":"2024-01-15T18:18:27.774550+0000","objects_scrubbed":17,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T22:10:32.866216+0000","scrub_duration":0.17066413899999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67109124,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":17,"num_write_kb":65537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:17.448002+0000","last_change":"2024-01-15T18:18:17.448002+0000","last_active":"2024-01-15T18:18:17.448002+0000","last_peered":"2024-01-15T18:18:17.448002+0000","last_clean":"2024-01-15T18:18:17.448002+0000","last_became_active":"2024-01-15T18:12:10.024554+0000","last_became_peered":"2024-01-15T18:12:10.024554+0000","last_unstale":"2024-01-15T18:18:17.448002+0000","last_undegraded":"2024-01-15T18:18:17.448002+0000","last_fullsized":"2024-01-15T18:18:17.448002+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:17.447735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:17.447735+0000","last_clean_scrub_stamp":"2024-01-15T18:18:17.447735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:29:15.174266+0000","scrub_duration":0.0035608710000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"117'47","reported_seq":76,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:08.292673+0000","last_change":"2024-01-15T18:18:08.292673+0000","last_active":"2024-01-15T18:18:08.292673+0000","last_peered":"2024-01-15T18:18:08.292673+0000","last_clean":"2024-01-15T18:18:08.292673+0000","last_became_active":"2024-01-15T18:12:11.050747+0000","last_became_peered":"2024-01-15T18:12:11.050747+0000","last_unstale":"2024-01-15T18:18:08.292673+0000","last_undegraded":"2024-01-15T18:18:08.292673+0000","last_fullsized":"2024-01-15T18:18:08.292673+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'47","last_scrub_stamp":"2024-01-15T18:18:08.292356+0000","last_deep_scrub":"117'47","last_deep_scrub_stamp":"2024-01-15T18:18:08.292356+0000","last_clean_scrub_stamp":"2024-01-15T18:18:08.292356+0000","objects_scrubbed":13,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T20:14:42.792622+0000","scrub_duration":0.148150529,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54525952,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":4,"num_read_kb":16384,"num_write":13,"num_write_kb":53248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"117'3","reported_seq":29,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.477457+0000","last_change":"2024-01-15T18:18:01.477457+0000","last_active":"2024-01-15T18:18:01.477457+0000","last_peered":"2024-01-15T18:18:01.477457+0000","last_clean":"2024-01-15T18:18:01.477457+0000","last_became_active":"2024-01-15T18:12:10.025356+0000","last_became_peered":"2024-01-15T18:12:10.025356+0000","last_unstale":"2024-01-15T18:18:01.477457+0000","last_undegraded":"2024-01-15T18:18:01.477457+0000","last_fullsized":"2024-01-15T18:18:01.477457+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'3","last_scrub_stamp":"2024-01-15T18:18:01.477202+0000","last_deep_scrub":"117'3","last_deep_scrub_stamp":"2024-01-15T18:18:01.477202+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.477202+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T19:57:59.723407+0000","scrub_duration":0.0075668130000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"117'63","reported_seq":100,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:35.601329+0000","last_change":"2024-01-15T18:18:35.601329+0000","last_active":"2024-01-15T18:18:35.601329+0000","last_peered":"2024-01-15T18:18:35.601329+0000","last_clean":"2024-01-15T18:18:35.601329+0000","last_became_active":"2024-01-15T18:12:11.050116+0000","last_became_peered":"2024-01-15T18:12:11.050116+0000","last_unstale":"2024-01-15T18:18:35.601329+0000","last_undegraded":"2024-01-15T18:18:35.601329+0000","last_fullsized":"2024-01-15T18:18:35.601329+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'63","last_scrub_stamp":"2024-01-15T18:18:35.600822+0000","last_deep_scrub":"117'63","last_deep_scrub_stamp":"2024-01-15T18:18:35.600822+0000","last_clean_scrub_stamp":"2024-01-15T18:18:35.600822+0000","objects_scrubbed":22,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T06:02:39.296272+0000","scrub_duration":0.31021504900000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":12,"num_read_kb":49152,"num_write":22,"num_write_kb":90112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"117'18","reported_seq":44,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:02.206754+0000","last_change":"2024-01-15T18:18:02.206754+0000","last_active":"2024-01-15T18:18:02.206754+0000","last_peered":"2024-01-15T18:18:02.206754+0000","last_clean":"2024-01-15T18:18:02.206754+0000","last_became_active":"2024-01-15T18:12:10.020497+0000","last_became_peered":"2024-01-15T18:12:10.020497+0000","last_unstale":"2024-01-15T18:18:02.206754+0000","last_undegraded":"2024-01-15T18:18:02.206754+0000","last_fullsized":"2024-01-15T18:18:02.206754+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'18","last_scrub_stamp":"2024-01-15T18:18:02.206517+0000","last_deep_scrub":"117'18","last_deep_scrub_stamp":"2024-01-15T18:18:02.206517+0000","last_clean_scrub_stamp":"2024-01-15T18:18:02.206517+0000","objects_scrubbed":1,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:40:05.823508+0000","scrub_duration":0.0043064160000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":503,"num_omap_keys":10,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"117'70","reported_seq":109,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.709777+0000","last_change":"2024-01-15T18:18:09.709777+0000","last_active":"2024-01-15T18:18:09.709777+0000","last_peered":"2024-01-15T18:18:09.709777+0000","last_clean":"2024-01-15T18:18:09.709777+0000","last_became_active":"2024-01-15T18:12:11.050810+0000","last_became_peered":"2024-01-15T18:12:11.050810+0000","last_unstale":"2024-01-15T18:18:09.709777+0000","last_undegraded":"2024-01-15T18:18:09.709777+0000","last_fullsized":"2024-01-15T18:18:09.709777+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'70","last_scrub_stamp":"2024-01-15T18:18:09.709451+0000","last_deep_scrub":"117'70","last_deep_scrub_stamp":"2024-01-15T18:18:09.709451+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.709451+0000","objects_scrubbed":24,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T01:50:33.702809+0000","scrub_duration":0.24709842800000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":14,"num_read_kb":57344,"num_write":24,"num_write_kb":98304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:12.368166+0000","last_change":"2024-01-15T18:18:12.368166+0000","last_active":"2024-01-15T18:18:12.368166+0000","last_peered":"2024-01-15T18:18:12.368166+0000","last_clean":"2024-01-15T18:18:12.368166+0000","last_became_active":"2024-01-15T18:12:10.024524+0000","last_became_peered":"2024-01-15T18:12:10.024524+0000","last_unstale":"2024-01-15T18:18:12.368166+0000","last_undegraded":"2024-01-15T18:18:12.368166+0000","last_fullsized":"2024-01-15T18:18:12.368166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:12.367669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:12.367669+0000","last_clean_scrub_stamp":"2024-01-15T18:18:12.367669+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:30:45.736811+0000","scrub_duration":0.0043195339999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"117'58","reported_seq":89,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:10.345686+0000","last_change":"2024-01-15T18:18:10.345686+0000","last_active":"2024-01-15T18:18:10.345686+0000","last_peered":"2024-01-15T18:18:10.345686+0000","last_clean":"2024-01-15T18:18:10.345686+0000","last_became_active":"2024-01-15T18:12:11.061454+0000","last_became_peered":"2024-01-15T18:12:11.061454+0000","last_unstale":"2024-01-15T18:18:10.345686+0000","last_undegraded":"2024-01-15T18:18:10.345686+0000","last_fullsized":"2024-01-15T18:18:10.345686+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'58","last_scrub_stamp":"2024-01-15T18:18:10.345394+0000","last_deep_scrub":"117'58","last_deep_scrub_stamp":"2024-01-15T18:18:10.345394+0000","last_clean_scrub_stamp":"2024-01-15T18:18:10.345394+0000","objects_scrubbed":19,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:23:55.078534+0000","scrub_duration":0.209955119,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303290,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":1,"num_read":7,"num_read_kb":28672,"num_write":19,"num_write_kb":69633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:17:59.502129+0000","last_change":"2024-01-15T18:17:59.502129+0000","last_active":"2024-01-15T18:17:59.502129+0000","last_peered":"2024-01-15T18:17:59.502129+0000","last_clean":"2024-01-15T18:17:59.502129+0000","last_became_active":"2024-01-15T18:12:10.018057+0000","last_became_peered":"2024-01-15T18:12:10.018057+0000","last_unstale":"2024-01-15T18:17:59.502129+0000","last_undegraded":"2024-01-15T18:17:59.502129+0000","last_fullsized":"2024-01-15T18:17:59.502129+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:17:59.501895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:17:59.501895+0000","last_clean_scrub_stamp":"2024-01-15T18:17:59.501895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:50:07.865255+0000","scrub_duration":0.0038048180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"117'60","reported_seq":92,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:03.094301+0000","last_change":"2024-01-15T18:18:03.094301+0000","last_active":"2024-01-15T18:18:03.094301+0000","last_peered":"2024-01-15T18:18:03.094301+0000","last_clean":"2024-01-15T18:18:03.094301+0000","last_became_active":"2024-01-15T18:12:11.039586+0000","last_became_peered":"2024-01-15T18:12:11.039586+0000","last_unstale":"2024-01-15T18:18:03.094301+0000","last_undegraded":"2024-01-15T18:18:03.094301+0000","last_fullsized":"2024-01-15T18:18:03.094301+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'60","last_scrub_stamp":"2024-01-15T18:18:03.093993+0000","last_deep_scrub":"117'60","last_deep_scrub_stamp":"2024-01-15T18:18:03.093993+0000","last_clean_scrub_stamp":"2024-01-15T18:18:03.093993+0000","objects_scrubbed":16,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:14:52.828664+0000","scrub_duration":0.201843208,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67108864,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":7,"num_read_kb":28672,"num_write":16,"num_write_kb":65536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.271104+0000","last_change":"2024-01-15T18:18:00.271104+0000","last_active":"2024-01-15T18:18:00.271104+0000","last_peered":"2024-01-15T18:18:00.271104+0000","last_clean":"2024-01-15T18:18:00.271104+0000","last_became_active":"2024-01-15T18:12:10.019424+0000","last_became_peered":"2024-01-15T18:12:10.019424+0000","last_unstale":"2024-01-15T18:18:00.271104+0000","last_undegraded":"2024-01-15T18:18:00.271104+0000","last_fullsized":"2024-01-15T18:18:00.271104+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:00.270864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:00.270864+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.270864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:23:50.723379+0000","scrub_duration":0.0035368880000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"117'82","reported_seq":116,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:09.251086+0000","last_change":"2024-01-15T18:18:09.251086+0000","last_active":"2024-01-15T18:18:09.251086+0000","last_peered":"2024-01-15T18:18:09.251086+0000","last_clean":"2024-01-15T18:18:09.251086+0000","last_became_active":"2024-01-15T18:12:11.035238+0000","last_became_peered":"2024-01-15T18:12:11.035238+0000","last_unstale":"2024-01-15T18:18:09.251086+0000","last_undegraded":"2024-01-15T18:18:09.251086+0000","last_fullsized":"2024-01-15T18:18:09.251086+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'82","last_scrub_stamp":"2024-01-15T18:18:09.250811+0000","last_deep_scrub":"117'82","last_deep_scrub_stamp":"2024-01-15T18:18:09.250811+0000","last_clean_scrub_stamp":"2024-01-15T18:18:09.250811+0000","objects_scrubbed":29,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T21:21:19.387926+0000","scrub_duration":0.33453240099999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121634816,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":29,"num_write_kb":118784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:00.884506+0000","last_change":"2024-01-15T18:18:00.884506+0000","last_active":"2024-01-15T18:18:00.884506+0000","last_peered":"2024-01-15T18:18:00.884506+0000","last_clean":"2024-01-15T18:18:00.884506+0000","last_became_active":"2024-01-15T18:12:10.017190+0000","last_became_peered":"2024-01-15T18:12:10.017190+0000","last_unstale":"2024-01-15T18:18:00.884506+0000","last_undegraded":"2024-01-15T18:18:00.884506+0000","last_fullsized":"2024-01-15T18:18:00.884506+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:00.884268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:00.884268+0000","last_clean_scrub_stamp":"2024-01-15T18:18:00.884268+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:26:38.405683+0000","scrub_duration":0.003782021,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"117'72","reported_seq":106,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:31.659694+0000","last_change":"2024-01-15T18:18:31.659742+0000","last_active":"2024-01-15T18:18:31.659694+0000","last_peered":"2024-01-15T18:18:31.659694+0000","last_clean":"2024-01-15T18:18:31.659694+0000","last_became_active":"2024-01-15T18:12:11.056111+0000","last_became_peered":"2024-01-15T18:12:11.056111+0000","last_unstale":"2024-01-15T18:18:31.659694+0000","last_undegraded":"2024-01-15T18:18:31.659694+0000","last_fullsized":"2024-01-15T18:18:31.659694+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'72","last_scrub_stamp":"2024-01-15T18:18:31.659564+0000","last_deep_scrub":"117'72","last_deep_scrub_stamp":"2024-01-15T18:18:31.659564+0000","last_clean_scrub_stamp":"2024-01-15T18:18:31.659564+0000","objects_scrubbed":26,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T23:56:37.299332+0000","scrub_duration":0.32621962500000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":26,"num_write_kb":106496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"117'11","reported_seq":46,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:02.775960+0000","last_change":"2024-01-15T18:18:02.775960+0000","last_active":"2024-01-15T18:18:02.775960+0000","last_peered":"2024-01-15T18:18:02.775960+0000","last_clean":"2024-01-15T18:18:02.775960+0000","last_became_active":"2024-01-15T18:12:10.020890+0000","last_became_peered":"2024-01-15T18:12:10.020890+0000","last_unstale":"2024-01-15T18:18:02.775960+0000","last_undegraded":"2024-01-15T18:18:02.775960+0000","last_fullsized":"2024-01-15T18:18:02.775960+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'11","last_scrub_stamp":"2024-01-15T18:18:02.775685+0000","last_deep_scrub":"117'11","last_deep_scrub_stamp":"2024-01-15T18:18:02.775685+0000","last_clean_scrub_stamp":"2024-01-15T18:18:02.775685+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T00:29:04.277157+0000","scrub_duration":0.0073569229999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1321,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":9,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"117'62","reported_seq":95,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:38.539909+0000","last_change":"2024-01-15T18:18:38.540000+0000","last_active":"2024-01-15T18:18:38.539909+0000","last_peered":"2024-01-15T18:18:38.539909+0000","last_clean":"2024-01-15T18:18:38.539909+0000","last_became_active":"2024-01-15T18:12:11.057943+0000","last_became_peered":"2024-01-15T18:12:11.057943+0000","last_unstale":"2024-01-15T18:18:38.539909+0000","last_undegraded":"2024-01-15T18:18:38.539909+0000","last_fullsized":"2024-01-15T18:18:38.539909+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'62","last_scrub_stamp":"2024-01-15T18:18:38.539738+0000","last_deep_scrub":"117'62","last_deep_scrub_stamp":"2024-01-15T18:18:38.539738+0000","last_clean_scrub_stamp":"2024-01-15T18:18:38.539738+0000","objects_scrubbed":23,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-16T18:31:59.119021+0000","scrub_duration":0.23195958999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":23,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":26,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:11.404503+0000","last_change":"2024-01-15T18:18:11.404503+0000","last_active":"2024-01-15T18:18:11.404503+0000","last_peered":"2024-01-15T18:18:11.404503+0000","last_clean":"2024-01-15T18:18:11.404503+0000","last_became_active":"2024-01-15T18:12:10.009870+0000","last_became_peered":"2024-01-15T18:12:10.009870+0000","last_unstale":"2024-01-15T18:18:11.404503+0000","last_undegraded":"2024-01-15T18:18:11.404503+0000","last_fullsized":"2024-01-15T18:18:11.404503+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:11.404131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:11.404131+0000","last_clean_scrub_stamp":"2024-01-15T18:18:11.404131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T02:34:10.272987+0000","scrub_duration":0.0037587990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"117'45","reported_seq":77,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:31.743706+0000","last_change":"2024-01-15T18:18:31.743814+0000","last_active":"2024-01-15T18:18:31.743706+0000","last_peered":"2024-01-15T18:18:31.743706+0000","last_clean":"2024-01-15T18:18:31.743706+0000","last_became_active":"2024-01-15T18:12:11.051092+0000","last_became_peered":"2024-01-15T18:12:11.051092+0000","last_unstale":"2024-01-15T18:18:31.743706+0000","last_undegraded":"2024-01-15T18:18:31.743706+0000","last_fullsized":"2024-01-15T18:18:31.743706+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'45","last_scrub_stamp":"2024-01-15T18:18:31.743522+0000","last_deep_scrub":"117'45","last_deep_scrub_stamp":"2024-01-15T18:18:31.743522+0000","last_clean_scrub_stamp":"2024-01-15T18:18:31.743522+0000","objects_scrubbed":14,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T04:14:34.678849+0000","scrub_duration":0.17999523000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58720256,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":8,"num_read_kb":32768,"num_write":14,"num_write_kb":57344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":25,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:02.284897+0000","last_change":"2024-01-15T18:18:02.284945+0000","last_active":"2024-01-15T18:18:02.284897+0000","last_peered":"2024-01-15T18:18:02.284897+0000","last_clean":"2024-01-15T18:18:02.284897+0000","last_became_active":"2024-01-15T18:12:10.026651+0000","last_became_peered":"2024-01-15T18:12:10.026651+0000","last_unstale":"2024-01-15T18:18:02.284897+0000","last_undegraded":"2024-01-15T18:18:02.284897+0000","last_fullsized":"2024-01-15T18:18:02.284897+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-15T18:18:02.284683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-15T18:18:02.284683+0000","last_clean_scrub_stamp":"2024-01-15T18:18:02.284683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T05:30:07.355476+0000","scrub_duration":0.0041976940000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"117'119","reported_seq":262,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-01-15T18:18:01.554023+0000","last_change":"2024-01-15T18:18:01.554131+0000","last_active":"2024-01-15T18:18:01.554023+0000","last_peered":"2024-01-15T18:18:01.554023+0000","last_clean":"2024-01-15T18:18:01.554023+0000","last_became_active":"2024-01-15T17:54:59.623548+0000","last_became_peered":"2024-01-15T17:54:59.623548+0000","last_unstale":"2024-01-15T18:18:01.554023+0000","last_undegraded":"2024-01-15T18:18:01.554023+0000","last_fullsized":"2024-01-15T18:18:01.554023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"117'119","last_scrub_stamp":"2024-01-15T18:18:01.553859+0000","last_deep_scrub":"117'119","last_deep_scrub_stamp":"2024-01-15T18:18:01.553859+0000","last_clean_scrub_stamp":"2024-01-15T18:18:01.553859+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-17T03:40:28.775696+0000","scrub_duration":0.056845899999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":5368710133,"num_objects":1286,"num_object_clones":1,"num_object_copies":2572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":1,"num_read":525,"num_read_kb":2097165,"num_write":1306,"num_write_kb":5242895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":62914560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":10737459200,"data_stored":10737420266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4061,"ondisk_log_size":4061,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":641711,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":281,"num_write_kb":757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5503,"num_omap_keys":20,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1343488,"data_stored":1283422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1834588,"kb_used_data":1767380,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92537252,"statfs":{"total":96636764160,"available":94758146048,"internally_reserved":0,"allocated":1809797120,"data_stored":1808970858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68811210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2629999999999999,"15min":1.2669999999999999},"min":{"1min":0.92600000000000005,"5min":0.66900000000000004,"15min":0.64200000000000002},"max":{"1min":1.72,"5min":1.8080000000000001,"15min":11.449999999999999},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.2370000000000001,"15min":1.3600000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.53400000000000003},"max":{"1min":1.6819999999999999,"5min":1.7090000000000001,"15min":16.678999999999998},"last":1.1060000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.2549999999999999,"15min":1.2809999999999999},"min":{"1min":1.022,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":11.51},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.393,"5min":1.3149999999999999,"15min":1.508},"min":{"1min":1.0409999999999999,"5min":0.753,"15min":0.753},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":28.613},"last":1.361}]},{"osd":2,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.264,"15min":1.3180000000000001},"min":{"1min":0.93400000000000005,"5min":0.75600000000000001,"15min":0.67200000000000004},"max":{"1min":1.752,"5min":1.8140000000000001,"15min":12.061},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.3620000000000001,"15min":1.47},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.67300000000000004},"max":{"1min":1.827,"5min":1.9299999999999999,"15min":16.792000000000002},"last":1.2589999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.3049999999999999,"15min":1.391},"min":{"1min":1.0940000000000001,"5min":0.80700000000000005,"15min":0.77700000000000002},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":9.8800000000000008},"last":1.325},{"interface":"front","average":{"1min":1.518,"5min":1.429,"15min":1.528},"min":{"1min":1.2390000000000001,"5min":0.94399999999999995,"15min":0.72499999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":16.129000000000001},"last":1.0880000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.288,"15min":1.5489999999999999},"min":{"1min":0.88800000000000001,"5min":0.78900000000000003,"15min":0.56299999999999994},"max":{"1min":2.077,"5min":2.077,"15min":23.042000000000002},"last":0.93999999999999995},{"interface":"front","average":{"1min":3.073,"5min":1.694,"15min":1.583},"min":{"1min":0.91200000000000003,"5min":0.80000000000000004,"15min":0.496},"max":{"1min":34.511000000000003,"5min":34.511000000000003,"15min":34.511000000000003},"last":1.1679999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.4019999999999999,"15min":1.502},"min":{"1min":1.0249999999999999,"5min":0.76800000000000002,"15min":0.746},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":11.696999999999999},"last":1.393},{"interface":"front","average":{"1min":1.637,"5min":1.48,"15min":1.657},"min":{"1min":1.097,"5min":0.98499999999999999,"15min":0.624},"max":{"1min":2.2850000000000001,"5min":2.2850000000000001,"15min":22.933},"last":1.296}]},{"osd":6,"last update":"Mon Jan 15 18:18:35 2024","interfaces":[{"interface":"back","average":{"1min":1.694,"5min":1.548,"15min":1.8080000000000001},"min":{"1min":1.1930000000000001,"5min":1.0660000000000001,"15min":0.57399999999999995},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":23.004000000000001},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.4990000000000001,"15min":1.5780000000000001},"min":{"1min":1.119,"5min":1.008,"15min":0.64900000000000002},"max":{"1min":2.548,"5min":2.548,"15min":7.5220000000000002},"last":1.444}]}]},{"osd":6,"up_from":14,"seq":60129542434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1260568,"kb_used_data":1193360,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":93111272,"statfs":{"total":96636764160,"available":95345942528,"internally_reserved":0,"allocated":1222000640,"data_stored":1221177506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:39 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.294,"15min":1.4419999999999999},"min":{"1min":0.71899999999999997,"5min":0.499,"15min":0.499},"max":{"1min":3.0870000000000002,"5min":3.0870000000000002,"15min":31.805},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.274,"15min":1.4339999999999999},"min":{"1min":0.67700000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":2.879,"5min":2.879,"15min":31.702999999999999},"last":1.3080000000000001}]},{"osd":1,"last update":"Mon Jan 15 18:18:39 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.119,"15min":1.2130000000000001},"min":{"1min":0.66800000000000004,"5min":0.59999999999999998,"15min":0.59899999999999998},"max":{"1min":1.6379999999999999,"5min":1.778,"15min":15.064},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.171,"15min":1.3069999999999999},"min":{"1min":0.84599999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6040000000000001,"5min":1.9119999999999999,"15min":23.495999999999999},"last":1.22}]},{"osd":2,"last update":"Mon Jan 15 18:18:39 2024","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.2709999999999999,"15min":1.419},"min":{"1min":1.0269999999999999,"5min":0.79100000000000004,"15min":0.73799999999999999},"max":{"1min":1.589,"5min":1.8120000000000001,"15min":23.087},"last":1.349},{"interface":"front","average":{"1min":1.329,"5min":1.325,"15min":1.3660000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.51400000000000001},"max":{"1min":1.774,"5min":1.976,"15min":14.94},"last":1.2809999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:39 2024","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4399999999999999,"15min":1.5589999999999999},"min":{"1min":0.94999999999999996,"5min":0.65200000000000002,"15min":0.59899999999999998},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":15.151999999999999},"last":1.375},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.327,"15min":1.4790000000000001},"min":{"1min":0.90500000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.639,"5min":1.7649999999999999,"15min":13.708},"last":1.252}]},{"osd":4,"last update":"Mon Jan 15 18:18:39 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2470000000000001,"15min":1.399},"min":{"1min":0.77300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.71,"5min":1.766,"15min":23.327000000000002},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.347,"15min":1.4870000000000001},"min":{"1min":0.91100000000000003,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.982,"5min":1.982,"15min":15.239000000000001},"last":0.92300000000000004}]},{"osd":5,"last update":"Mon Jan 15 18:18:39 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2789999999999999,"15min":1.389},"min":{"1min":0.80600000000000005,"5min":0.58099999999999996,"15min":0.52500000000000002},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":7.8780000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.472,"5min":1.516,"15min":1.6699999999999999},"min":{"1min":0.80900000000000005,"5min":0.66200000000000003,"15min":0.64400000000000002},"max":{"1min":1.712,"5min":1.913,"15min":23.417999999999999},"last":1.4610000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:39 2024","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.419,"15min":1.5800000000000001},"min":{"1min":1.0509999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.125,"5min":2.125,"15min":16.454000000000001},"last":1.107},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.526,"15min":1.708},"min":{"1min":1.228,"5min":0.62,"15min":0.62},"max":{"1min":1.921,"5min":2.2229999999999999,"15min":23.738},"last":1.407}]}]},{"osd":5,"up_from":14,"seq":60129542434,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908440,"kb_used_data":832912,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93463400,"statfs":{"total":96636764160,"available":95706521600,"internally_reserved":0,"allocated":852901888,"data_stored":852079166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1779999999999999,"15min":1.2090000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.54200000000000004},"max":{"1min":1.6259999999999999,"5min":2.7919999999999998,"15min":7.6580000000000004},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.121,"5min":1.1899999999999999,"15min":1.1850000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.51400000000000001},"max":{"1min":1.708,"5min":1.7529999999999999,"15min":2.1869999999999998},"last":1.266}]},{"osd":1,"last update":"Mon Jan 15 18:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1890000000000001,"15min":1.274},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.60699999999999998},"max":{"1min":1.649,"5min":1.6879999999999999,"15min":15.554},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2569999999999999,"15min":1.3120000000000001},"min":{"1min":0.81100000000000005,"5min":0.69599999999999995,"15min":0.59099999999999997},"max":{"1min":1.7949999999999999,"5min":1.7949999999999999,"15min":7.7649999999999997},"last":0.98099999999999998}]},{"osd":2,"last update":"Mon Jan 15 18:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.244,"15min":1.2450000000000001},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.647,"5min":1.694,"15min":3.516},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.306,"5min":1.361,"15min":1.3959999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.56000000000000005},"max":{"1min":1.74,"5min":1.7769999999999999,"15min":15.468999999999999},"last":1.1599999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3380000000000001,"15min":1.3700000000000001},"min":{"1min":0.88800000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.843,"5min":1.843,"15min":7.8609999999999998},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4339999999999999,"15min":1.4279999999999999},"min":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.64600000000000002},"max":{"1min":1.8320000000000001,"5min":1.8380000000000001,"15min":2.2949999999999999},"last":1.3779999999999999}]},{"osd":4,"last update":"Mon Jan 15 18:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.306,"15min":1.4279999999999999},"min":{"1min":0.76500000000000001,"5min":0.48299999999999998,"15min":0.41399999999999998},"max":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":15.254},"last":1.046},{"interface":"front","average":{"1min":1.45,"5min":1.423,"15min":1.452},"min":{"1min":1.0009999999999999,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.99,"5min":1.99,"15min":7.5359999999999996},"last":1.409}]},{"osd":6,"last update":"Mon Jan 15 18:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.409,"15min":1.4419999999999999},"min":{"1min":0.85199999999999998,"5min":0.51300000000000001,"15min":0.39600000000000002},"max":{"1min":1.948,"5min":1.948,"15min":3.7559999999999998},"last":1.49},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.4770000000000001,"15min":1.5349999999999999},"min":{"1min":0.91600000000000004,"5min":0.55300000000000005,"15min":0.53000000000000003},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":15.366},"last":1.5269999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:18:38 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.5489999999999999,"15min":1.5900000000000001},"min":{"1min":0.94199999999999995,"5min":0.65300000000000002,"15min":0.45800000000000002},"max":{"1min":2.113,"5min":2.113,"15min":7.96},"last":1.714},{"interface":"front","average":{"1min":1.6779999999999999,"5min":1.593,"15min":1.597},"min":{"1min":0.95899999999999996,"5min":0.63600000000000001,"15min":0.57299999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":3.1659999999999999},"last":1.589}]}]},{"osd":4,"up_from":14,"seq":60129542434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1711124,"kb_used_data":1643916,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":92660716,"statfs":{"total":96636764160,"available":94884573184,"internally_reserved":0,"allocated":1683369984,"data_stored":1682551014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.25,"15min":1.3029999999999999},"min":{"1min":0.73199999999999998,"5min":0.70199999999999996,"15min":0.39200000000000002},"max":{"1min":1.6519999999999999,"5min":1.8420000000000001,"15min":17.346},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3520000000000001,"15min":1.3260000000000001},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.40100000000000002},"max":{"1min":1.623,"5min":1.804,"15min":3.726},"last":1.0389999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.224,"15min":1.2370000000000001},"min":{"1min":0.86799999999999999,"5min":0.76600000000000001,"15min":0.439},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":11.128},"last":1.325},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.226,"15min":1.216},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.67500000000000004},"max":{"1min":1.528,"5min":1.7470000000000001,"15min":3.7370000000000001},"last":1.258}]},{"osd":2,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3029999999999999,"15min":1.333},"min":{"1min":0.873,"5min":0.83399999999999996,"15min":0.56899999999999995},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":18.442},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.365,"15min":1.371},"min":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.81899999999999995},"max":{"1min":1.8029999999999999,"5min":1.861,"15min":5.1340000000000003},"last":1.488}]},{"osd":3,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.371,"15min":1.4259999999999999},"min":{"1min":0.83499999999999996,"5min":0.73499999999999999,"15min":0.60199999999999998},"max":{"1min":1.923,"5min":1.923,"15min":17.364999999999998},"last":1.198},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.337,"15min":1.3700000000000001},"min":{"1min":1.04,"5min":0.85099999999999998,"15min":0.45300000000000001},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":9.3320000000000007},"last":1.1639999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3029999999999999,"15min":1.3500000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.49099999999999999},"max":{"1min":2.028,"5min":2.028,"15min":17.292999999999999},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.3520000000000001,"15min":1.4199999999999999},"min":{"1min":1.0109999999999999,"5min":0.88300000000000001,"15min":0.53700000000000003},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":12.332000000000001},"last":1.4359999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2589999999999999,"15min":1.367},"min":{"1min":0.94199999999999995,"5min":0.80100000000000005,"15min":0.54200000000000004},"max":{"1min":1.867,"5min":1.867,"15min":11.09},"last":1.385},{"interface":"front","average":{"1min":1.462,"5min":1.4670000000000001,"15min":1.603},"min":{"1min":1.004,"5min":1.004,"15min":0.51100000000000001},"max":{"1min":2.1619999999999999,"5min":2.1619999999999999,"15min":17.224},"last":1.1100000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:33 2024","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.518,"15min":1.5660000000000001},"min":{"1min":1.101,"5min":0.93300000000000005,"15min":0.625},"max":{"1min":2.4319999999999999,"5min":2.4319999999999999,"15min":12.247},"last":1.512},{"interface":"front","average":{"1min":1.6799999999999999,"5min":1.6519999999999999,"15min":1.718},"min":{"1min":1.3080000000000001,"5min":1.159,"15min":0.61799999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":17.324999999999999},"last":1.4710000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542436,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1462036,"kb_used_data":1386508,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":92909804,"statfs":{"total":96636764160,"available":95139639296,"internally_reserved":0,"allocated":1419784192,"data_stored":1418949784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77330887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":1.006,"15min":1.0620000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.5660000000000001,"5min":2.843,"15min":9.2899999999999991},"last":0.746},{"interface":"front","average":{"1min":1.234,"5min":1.242,"15min":1.1830000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.44800000000000001},"max":{"1min":1.6859999999999999,"5min":2.5590000000000002,"15min":3.028},"last":1.095}]},{"osd":1,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":1.022,"15min":1.0249999999999999},"min":{"1min":0.56699999999999995,"5min":0.56200000000000006,"15min":0.32600000000000001},"max":{"1min":1.99,"5min":5.3520000000000003,"15min":12.172000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.075,"15min":1.228},"min":{"1min":0.623,"5min":0.59699999999999998,"15min":0.46300000000000002},"max":{"1min":1.5900000000000001,"5min":2.3370000000000002,"15min":11.228999999999999},"last":0.876}]},{"osd":2,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.214,"15min":1.2230000000000001},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.375},"max":{"1min":1.885,"5min":1.885,"15min":2.7519999999999998},"last":1.52},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4410000000000001,"15min":1.478},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.56100000000000005},"max":{"1min":2.2120000000000002,"5min":3.0659999999999998,"15min":9.4920000000000009},"last":1.4790000000000001}]},{"osd":4,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.359,"15min":1.4319999999999999},"min":{"1min":0.81000000000000005,"5min":0.78200000000000003,"15min":0.67200000000000004},"max":{"1min":1.7490000000000001,"5min":1.865,"15min":13.536},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.4339999999999999,"15min":1.4330000000000001},"min":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.63700000000000001},"max":{"1min":1.774,"5min":6.2439999999999998,"15min":16.398},"last":1.4319999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.472,"15min":1.583},"min":{"1min":1.175,"5min":0.83599999999999997,"15min":0.68899999999999995},"max":{"1min":2.1000000000000001,"5min":2.7269999999999999,"15min":9.5690000000000008},"last":1.349},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.5820000000000001,"15min":1.5860000000000001},"min":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.66800000000000004},"max":{"1min":1.9910000000000001,"5min":2.617,"15min":5.6980000000000004},"last":1.383}]},{"osd":6,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4730000000000001,"15min":1.5669999999999999},"min":{"1min":0.97299999999999998,"5min":0.85499999999999998,"15min":0.61399999999999999},"max":{"1min":2.0059999999999998,"5min":6.3109999999999999,"15min":13.648999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.54,"15min":1.694},"min":{"1min":1.0609999999999999,"5min":1.0249999999999999,"15min":0.77800000000000002},"max":{"1min":1.915,"5min":2.5910000000000002,"15min":13.679},"last":1.1830000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:36 2024","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.6240000000000001,"15min":1.6080000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.60499999999999998},"max":{"1min":2.3679999999999999,"5min":4.8630000000000004,"15min":4.8630000000000004},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.486,"5min":1.609,"15min":1.6040000000000001},"min":{"1min":1.125,"5min":0.995,"15min":0.70199999999999996},"max":{"1min":2.141,"5min":6.9569999999999999,"15min":9.4100000000000001},"last":1.7310000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542436,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":982100,"kb_used_data":910732,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":93389740,"statfs":{"total":96636764160,"available":95631093760,"internally_reserved":0,"allocated":932589568,"data_stored":931771659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:41 2024","interfaces":[{"interface":"back","average":{"1min":0.98999999999999999,"5min":1.1439999999999999,"15min":1.0860000000000001},"min":{"1min":0.501,"5min":0.501,"15min":0.32100000000000001},"max":{"1min":1.663,"5min":22.707000000000001,"15min":22.707000000000001},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.038,"5min":1.1379999999999999,"15min":1.3},"min":{"1min":0.69399999999999995,"5min":0.61099999999999999,"15min":0.39300000000000002},"max":{"1min":1.5329999999999999,"5min":1.831,"15min":15.314},"last":1.2649999999999999}]},{"osd":1,"last update":"Mon Jan 15 18:18:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1399999999999999,"15min":1.3109999999999999},"min":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.38300000000000001},"max":{"1min":1.5700000000000001,"5min":1.708,"15min":34.463999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2889999999999999,"15min":1.4950000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.46000000000000002},"max":{"1min":1.794,"5min":2.6139999999999999,"15min":25.859999999999999},"last":1.135}]},{"osd":3,"last update":"Mon Jan 15 18:18:41 2024","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.155,"15min":1.2030000000000001},"min":{"1min":0.66700000000000004,"5min":0.53100000000000003,"15min":0.34399999999999997},"max":{"1min":1.865,"5min":1.865,"15min":12.449999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.6459999999999999,"15min":1.6160000000000001},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.46999999999999997},"max":{"1min":1.796,"5min":37.021999999999998,"15min":37.021999999999998},"last":1.554}]},{"osd":4,"last update":"Mon Jan 15 18:18:41 2024","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.494,"15min":1.5429999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.69499999999999995},"max":{"1min":1.893,"5min":2.016,"15min":12.728},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.4470000000000001,"15min":1.611},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.71299999999999997},"max":{"1min":1.7649999999999999,"5min":1.9710000000000001,"15min":15.553000000000001},"last":1.732}]},{"osd":5,"last update":"Mon Jan 15 18:18:41 2024","interfaces":[{"interface":"back","average":{"1min":1.401,"5min":1.4379999999999999,"15min":1.597},"min":{"1min":1.081,"5min":0.88,"15min":0.63700000000000001},"max":{"1min":2.1200000000000001,"5min":2.218,"15min":14.066000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.603,"5min":1.6080000000000001,"15min":1.7749999999999999},"min":{"1min":1.091,"5min":1.091,"15min":0.84799999999999998},"max":{"1min":2.0350000000000001,"5min":2.3100000000000001,"15min":14.025},"last":1.629}]},{"osd":6,"last update":"Mon Jan 15 18:18:41 2024","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5620000000000001,"15min":1.794},"min":{"1min":1.1279999999999999,"5min":1.0640000000000001,"15min":0.74099999999999999},"max":{"1min":1.796,"5min":2.149,"15min":15.683999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.496,"15min":1.752},"min":{"1min":1.194,"5min":0.95399999999999996,"15min":0.69099999999999995},"max":{"1min":2.363,"5min":2.363,"15min":12.629},"last":1.3180000000000001}]},{"osd":7,"last update":"Mon Jan 15 18:18:41 2024","interfaces":[{"interface":"back","average":{"1min":1.6299999999999999,"5min":1.6319999999999999,"15min":1.8759999999999999},"min":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.77800000000000002},"max":{"1min":2.173,"5min":2.2210000000000001,"15min":17.099},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.659,"5min":1.619,"15min":1.8149999999999999},"min":{"1min":1.3029999999999999,"5min":1.0860000000000001,"15min":0.72599999999999998},"max":{"1min":2.2959999999999998,"5min":2.4140000000000001,"15min":22.853999999999999},"last":1.496}]}]},{"osd":1,"up_from":14,"seq":60129542435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1141924,"kb_used_data":1066396,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":93229916,"statfs":{"total":96636764160,"available":95467433984,"internally_reserved":0,"allocated":1091989504,"data_stored":1091155706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77330885},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Jan 15 18:18:42 2024","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.94699999999999995,"15min":1.1080000000000001},"min":{"1min":0.55800000000000005,"5min":0.40999999999999998,"15min":0.34399999999999997},"max":{"1min":1.593,"5min":2.7389999999999999,"15min":30.460999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.1020000000000001,"15min":1.212},"min":{"1min":0.55700000000000005,"5min":0.52200000000000002,"15min":0.34999999999999998},"max":{"1min":1.532,"5min":5.4779999999999998,"15min":17.157},"last":0.90500000000000003}]},{"osd":2,"last update":"Mon Jan 15 18:18:42 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0469999999999999,"15min":1.1659999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":7.117},"last":0.998},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.296,"15min":1.4970000000000001},"min":{"1min":0.79800000000000004,"5min":0.496,"15min":0.496},"max":{"1min":1.651,"5min":1.728,"15min":30.593},"last":1.4319999999999999}]},{"osd":3,"last update":"Mon Jan 15 18:18:42 2024","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.26,"15min":1.4259999999999999},"min":{"1min":0.72899999999999998,"5min":0.53600000000000003,"15min":0.52300000000000002},"max":{"1min":1.9610000000000001,"5min":1.9610000000000001,"15min":30.547000000000001},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1399999999999999,"15min":1.3080000000000001},"min":{"1min":0.68799999999999994,"5min":0.42299999999999999,"15min":0.40200000000000002},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":17.263000000000002},"last":1.24}]},{"osd":4,"last update":"Mon Jan 15 18:18:42 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.431,"15min":1.6950000000000001},"min":{"1min":0.96899999999999997,"5min":0.78100000000000003,"15min":0.75900000000000001},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":17.420000000000002},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.421,"15min":1.601},"min":{"1min":0.70799999999999996,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":14.140000000000001},"last":1.5309999999999999}]},{"osd":5,"last update":"Mon Jan 15 18:18:42 2024","interfaces":[{"interface":"back","average":{"1min":1.3839999999999999,"5min":1.359,"15min":1.4990000000000001},"min":{"1min":0.90700000000000003,"5min":0.81100000000000005,"15min":0.76200000000000001},"max":{"1min":2.0760000000000001,"5min":2.0760000000000001,"15min":9.0440000000000005},"last":1.466},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.508,"15min":1.6950000000000001},"min":{"1min":1.006,"5min":0.79100000000000004,"15min":0.68300000000000005},"max":{"1min":2.0150000000000001,"5min":2.1019999999999999,"15min":34.042999999999999},"last":1.2969999999999999}]},{"osd":6,"last update":"Mon Jan 15 18:18:42 2024","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4670000000000001,"15min":1.577},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":10.250999999999999},"last":1.393},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.47,"15min":1.649},"min":{"1min":1.085,"5min":0.71499999999999997,"15min":0.67900000000000005},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":17.510000000000002},"last":1.722}]},{"osd":7,"last update":"Mon Jan 15 18:18:42 2024","interfaces":[{"interface":"back","average":{"1min":1.694,"5min":1.5980000000000001,"15min":1.804},"min":{"1min":0.93799999999999994,"5min":0.82999999999999996,"15min":0.77300000000000002},"max":{"1min":2.2829999999999999,"5min":2.4329999999999998,"15min":33.659999999999997},"last":1.637},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.627,"15min":1.7410000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":11.882999999999999},"last":1.7589999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542436,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1769764,"kb_used_data":1698396,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":92602076,"statfs":{"total":96636764160,"available":94824525824,"internally_reserved":0,"allocated":1739157504,"data_stored":1738307531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Jan 15 18:18:37 2024","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.98599999999999999,"15min":1.0880000000000001},"min":{"1min":0.69999999999999996,"5min":0.44400000000000001,"15min":0.42799999999999999},"max":{"1min":1.4019999999999999,"5min":1.55,"15min":12.050000000000001},"last":1.262},{"interface":"front","average":{"1min":1.171,"5min":1.218,"15min":1.3140000000000001},"min":{"1min":0.56899999999999995,"5min":0.54600000000000004,"15min":0.36399999999999999},"max":{"1min":1.6259999999999999,"5min":3.9529999999999998,"15min":11.962},"last":1.1379999999999999}]},{"osd":2,"last update":"Mon Jan 15 18:18:37 2024","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.2050000000000001,"15min":1.181},"min":{"1min":0.70199999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6859999999999999,"5min":1.9770000000000001,"15min":7.4560000000000004},"last":1.5920000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":1.026,"15min":1.0820000000000001},"min":{"1min":0.76300000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.4259999999999999,"5min":1.825,"15min":7.3239999999999998},"last":1.2090000000000001}]},{"osd":3,"last update":"Mon Jan 15 18:18:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2929999999999999,"15min":1.355},"min":{"1min":0.80900000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.6950000000000001,"5min":1.879,"15min":11.904},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.1499999999999999,"15min":1.163},"min":{"1min":0.81100000000000005,"5min":0.39000000000000001,"15min":0.35199999999999998},"max":{"1min":1.8919999999999999,"5min":2.0030000000000001,"15min":3.202},"last":1.075}]},{"osd":4,"last update":"Mon Jan 15 18:18:37 2024","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.27,"15min":1.331},"min":{"1min":0.91800000000000004,"5min":0.55400000000000005,"15min":0.54100000000000004},"max":{"1min":1.5580000000000001,"5min":1.9330000000000001,"15min":7.4000000000000004},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.498,"5min":1.5109999999999999,"15min":1.577},"min":{"1min":1.046,"5min":0.80900000000000005,"15min":0.67700000000000005},"max":{"1min":1.8180000000000001,"5min":3.7029999999999998,"15min":11.209},"last":1.3100000000000001}]},{"osd":5,"last update":"Mon Jan 15 18:18:37 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.3260000000000001,"15min":1.4390000000000001},"min":{"1min":0.876,"5min":0.60899999999999999,"15min":0.47499999999999998},"max":{"1min":1.679,"5min":2.593,"15min":12.006},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.5920000000000001,"15min":1.522},"min":{"1min":1.0740000000000001,"5min":0.75,"15min":0.442},"max":{"1min":2.0409999999999999,"5min":15.907,"15min":15.907},"last":1.488}]},{"osd":6,"last update":"Mon Jan 15 18:18:37 2024","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.52,"15min":1.72},"min":{"1min":1.1739999999999999,"5min":0.82499999999999996,"15min":0.66600000000000004},"max":{"1min":2.1680000000000001,"5min":2.5590000000000002,"15min":24.039999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.466,"15min":1.591},"min":{"1min":0.95499999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8029999999999999,"5min":2.3159999999999998,"15min":18.143999999999998},"last":1.5469999999999999}]},{"osd":7,"last update":"Mon Jan 15 18:18:37 2024","interfaces":[{"interface":"back","average":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.8200000000000001},"min":{"1min":1.2050000000000001,"5min":0.89400000000000002,"15min":0.69099999999999995},"max":{"1min":2.0539999999999998,"5min":3.8839999999999999,"15min":26.427},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.603,"15min":1.7849999999999999},"min":{"1min":1.2689999999999999,"5min":0.78100000000000003,"15min":0.61799999999999999},"max":{"1min":2.2250000000000001,"5min":2.3500000000000001,"15min":44.694000000000003},"last":1.4319999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":655360,"data_stored":639704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":1321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":639662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1736458240,"data_stored":1736442609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1090527232,"data_stored":1090519411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":931135488,"data_stored":931135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1417682944,"data_stored":1417675272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1681920000,"data_stored":1681916164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":851443712,"data_stored":851443712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1220546560,"data_stored":1220542586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1807745024,"data_stored":1807745024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-15T18:19:02.335 DEBUG:teuthology.orchestra.run.smithi026:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-01-15T18:19:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.579+0000 7ff12d000640 1 Processor -- start 2024-01-15T18:19:02.582 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.580+0000 7ff12d000640 1 -- start start 2024-01-15T18:19:02.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.581+0000 7ff12d000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff1280b35a0 0x7ff1280cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.581+0000 7ff12d000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280ce440 con 0x7ff12813cf10 2024-01-15T18:19:02.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.581+0000 7ff12d000640 1 -- --> v1:172.21.15.124:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280ce720 con 0x7ff12813fcb0 2024-01-15T18:19:02.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.581+0000 7ff12d000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1280cea00 con 0x7ff1280b35a0 2024-01-15T18:19:02.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.582+0000 7ff127400640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff12813cf10 0x7ff12813f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41560/0 (socket says 172.21.15.26:41560) 2024-01-15T18:19:02.584 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.582+0000 7ff127400640 1 -- 172.21.15.26:0/2620713292 learned_addr learned my addr 172.21.15.26:0/2620713292 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:19:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.582+0000 7ff127e00640 1 --2- 172.21.15.26:0/2620713292 >> v2:172.21.15.124:3300/0 conn(0x7ff1280b35a0 0x7ff1280cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.582+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278830890 0 0) 0x7ff1280ce440 con 0x7ff12813cf10 2024-01-15T18:19:02.586 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.582+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff108003520 con 0x7ff12813cf10 2024-01-15T18:19:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 <== mon.1 v1:172.21.15.124:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714828086 0 0) 0x7ff1280ce720 con 0x7ff12813fcb0 2024-01-15T18:19:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 --> v1:172.21.15.124:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1280ce440 con 0x7ff12813fcb0 2024-01-15T18:19:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3853293883 0 0) 0x7ff108003520 con 0x7ff12813cf10 2024-01-15T18:19:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 >> v2:172.21.15.124:3300/0 conn(0x7ff1280b35a0 msgr2=0x7ff1280cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 --2- 172.21.15.26:0/2620713292 >> v2:172.21.15.124:3300/0 conn(0x7ff1280b35a0 0x7ff1280cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 >> v1:172.21.15.124:6789/0 conn(0x7ff12813fcb0 legacy=0x7ff1280cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280b1d90 con 0x7ff12813cf10 2024-01-15T18:19:02.587 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7ff118001670 con 0x7ff12813cf10 2024-01-15T18:19:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 >> v1:172.21.15.26:6789/0 conn(0x7ff12813cf10 legacy=0x7ff12813f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff108003b00 0x7ff108003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1080047c0 0x7ff108004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff108005180 0x7ff10800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280cea00 con 0x7ff108003b00 2024-01-15T18:19:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10800c000 con 0x7ff1080047c0 2024-01-15T18:19:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10800c1c0 con 0x7ff108005180 2024-01-15T18:19:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.583+0000 7ff127400640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff108003b00 0x7ff108003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.588 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.584+0000 7ff11fe00640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1080047c0 0x7ff108004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.584+0000 7ff127e00640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff108005180 0x7ff10800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.584+0000 7ff127400640 1 -- 172.21.15.26:0/2620713292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff108005180 msgr2=0x7ff10800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.584+0000 7ff127400640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff108005180 0x7ff10800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.584+0000 7ff127400640 1 -- 172.21.15.26:0/2620713292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1080047c0 msgr2=0x7ff108004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.584+0000 7ff127400640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff1080047c0 0x7ff108004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.584+0000 7ff127400640 1 -- 172.21.15.26:0/2620713292 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff108003520 con 0x7ff108003b00 2024-01-15T18:19:02.589 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.584+0000 7ff127e00640 1 -- 172.21.15.26:0/2620713292 reap_dead start 2024-01-15T18:19:02.590 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.585+0000 7ff127400640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff108003b00 0x7ff108003f00 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7ff118007fd0 tx=0x7ff1080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1019f0006d4ff3d0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.585+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118005ad0 con 0x7ff108003b00 2024-01-15T18:19:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.585+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118005c50 con 0x7ff108003b00 2024-01-15T18:19:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.585+0000 7ff126a00640 1 -- 172.21.15.26:0/2620713292 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11802c7c0 con 0x7ff108003b00 2024-01-15T18:19:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.586+0000 7ff12d000640 1 -- 172.21.15.26:0/2620713292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff108003b00 msgr2=0x7ff108003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.586+0000 7ff12d000640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff108003b00 0x7ff108003f00 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7ff118007fd0 tx=0x7ff1080057f0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.586+0000 7ff12d000640 1 -- 172.21.15.26:0/2620713292 shutdown_connections 2024-01-15T18:19:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.586+0000 7ff12d000640 1 --2- 172.21.15.26:0/2620713292 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff108003b00 0x7ff108003f00 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.586+0000 7ff12d000640 1 -- 172.21.15.26:0/2620713292 >> 172.21.15.26:0/2620713292 conn(0x7ff12806a7d0 msgr2=0x7ff12806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:02.591 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.586+0000 7ff12d000640 1 -- 172.21.15.26:0/2620713292 shutdown_connections 2024-01-15T18:19:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.586+0000 7ff12d000640 1 -- 172.21.15.26:0/2620713292 wait complete. 2024-01-15T18:19:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.588+0000 7ff12d000640 1 Processor -- start 2024-01-15T18:19:02.592 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.588+0000 7ff12d000640 1 -- start start 2024-01-15T18:19:02.593 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.589+0000 7ff12d000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff108003b00 0x7ff1281529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.589+0000 7ff12d000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12813abc0 0x7ff12815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.589+0000 7ff12d000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12815c460 0x7ff12815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.589+0000 7ff12d000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff128159f70 con 0x7ff12815c460 2024-01-15T18:19:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.589+0000 7ff12d000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12815a7d0 con 0x7ff12813abc0 2024-01-15T18:19:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.589+0000 7ff12d000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12815cd60 con 0x7ff108003b00 2024-01-15T18:19:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.589+0000 7ff127e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12815c460 0x7ff12815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.589+0000 7ff127e00640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12815c460 0x7ff12815b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53202/0 (socket says 172.21.15.26:53202) 2024-01-15T18:19:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.589+0000 7ff127e00640 1 -- 172.21.15.26:0/2299541929 learned_addr learned my addr 172.21.15.26:0/2299541929 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:19:02.594 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff127400640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff108003b00 0x7ff1281529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff11fe00640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12813abc0 0x7ff12815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff127e00640 1 -- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff108003b00 msgr2=0x7ff1281529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff127e00640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff108003b00 0x7ff1281529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff127e00640 1 -- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12813abc0 msgr2=0x7ff12815ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff127e00640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12813abc0 0x7ff12815ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff127e00640 1 -- 172.21.15.26:0/2299541929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12815a180 con 0x7ff12815c460 2024-01-15T18:19:02.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff127e00640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12815c460 0x7ff12815b6a0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7ff108004740 tx=0x7ff120011520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:02.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff11f400640 1 -- 172.21.15.26:0/2299541929 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12001b070 con 0x7ff12815c460 2024-01-15T18:19:02.595 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.590+0000 7ff11f400640 1 -- 172.21.15.26:0/2299541929 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff120011dd0 con 0x7ff12815c460 2024-01-15T18:19:02.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.591+0000 7ff11f400640 1 -- 172.21.15.26:0/2299541929 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1200176b0 con 0x7ff12815c460 2024-01-15T18:19:02.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.591+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12815a460 con 0x7ff12815c460 2024-01-15T18:19:02.596 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.591+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12815c8f0 con 0x7ff12815c460 2024-01-15T18:19:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.592+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12815cb10 con 0x7ff12815c460 2024-01-15T18:19:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.593+0000 7ff11f400640 1 -- 172.21.15.26:0/2299541929 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 91171+0+0 (secure 0 0 0) 0x7ff120017810 con 0x7ff12815c460 2024-01-15T18:19:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.594+0000 7ff11f400640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10006c4c0 0x7ff10006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.594+0000 7ff11f400640 1 -- 172.21.15.26:0/2299541929 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7ff12008b030 con 0x7ff12815c460 2024-01-15T18:19:02.597 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.595+0000 7ff127400640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10006c4c0 0x7ff10006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:02.598 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.596+0000 7ff127400640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10006c4c0 0x7ff10006e980 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff118034000 tx=0x7ff118007000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:02.603 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.601+0000 7ff11f400640 1 -- 172.21.15.26:0/2299541929 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff120090e60 con 0x7ff12815c460 2024-01-15T18:19:02.799 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.797+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7ff12815a960 con 0x7ff12815c460 2024-01-15T18:19:02.803 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.801+0000 7ff11f400640 1 -- 172.21.15.26:0/2299541929 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v43)=0 v43) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff120057580 con 0x7ff12815c460 2024-01-15T18:19:02.806 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.804+0000 7ff11f400640 1 -- 172.21.15.26:0/2299541929 <== mon.0 v2:172.21.15.26:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff120057580 con 0x7ff12815c460 2024-01-15T18:19:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10006c4c0 msgr2=0x7ff10006e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10006c4c0 0x7ff10006e980 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff118034000 tx=0x7ff118007000 comp rx=0 tx=0).stop 2024-01-15T18:19:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12815c460 msgr2=0x7ff12815b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12815c460 0x7ff12815b6a0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7ff108004740 tx=0x7ff120011520 comp rx=0 tx=0).stop 2024-01-15T18:19:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 shutdown_connections 2024-01-15T18:19:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff10006c4c0 0x7ff10006e980 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.809 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff12815c460 0x7ff12815b6a0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff12813abc0 0x7ff12815ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 --2- 172.21.15.26:0/2299541929 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff108003b00 0x7ff1281529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:02.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 >> 172.21.15.26:0/2299541929 conn(0x7ff12806a7d0 msgr2=0x7ff1280cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:02.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 shutdown_connections 2024-01-15T18:19:02.810 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:02.806+0000 7ff12d000640 1 -- 172.21.15.26:0/2299541929 wait complete. 2024-01-15T18:19:02.827 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-15T18:19:02.827 DEBUG:teuthology.orchestra.run.smithi026:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-15T18:19:02.843 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:19:02.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:02.843 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:19:02.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:02.843 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:19:02.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:02.843 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:19:02.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:02.843 INFO:tasks.ceph.mgr.x.smithi124.stderr:2024-01-15T18:19:02.840+0000 7f8f5be00640 -1 client.0 error registering admin socket command: (17) File exists 2024-01-15T18:19:03.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.039+0000 7ff53b000640 1 Processor -- start 2024-01-15T18:19:03.042 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.040+0000 7ff53b000640 1 -- start start 2024-01-15T18:19:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.041+0000 7ff53b000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5340cc500 0x7ff5340cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.041+0000 7ff53b000640 1 --2- >> v2:172.21.15.124:3300/0 conn(0x7ff53406d160 0x7ff53406d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.041+0000 7ff53b000640 1 -- --> v1:172.21.15.26:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff534070e50 con 0x7ff534143c40 2024-01-15T18:19:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.041+0000 7ff53b000640 1 -- --> v2:172.21.15.124:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5340723c0 con 0x7ff53406d160 2024-01-15T18:19:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.041+0000 7ff53b000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff534144020 con 0x7ff5340cc500 2024-01-15T18:19:03.044 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff539c00640 1 --1- >> v1:172.21.15.26:6789/0 conn(0x7ff534143c40 0x7ff534071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.26:6789/0 says I am v1:172.21.15.26:41568/0 (socket says 172.21.15.26:41568) 2024-01-15T18:19:03.045 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff539c00640 1 -- 172.21.15.26:0/2216266004 learned_addr learned my addr 172.21.15.26:0/2216266004 (peer_addr_for_me v1:172.21.15.26:0/0) 2024-01-15T18:19:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff539200640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5340cc500 0x7ff5340cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff53a600640 1 --2- 172.21.15.26:0/2216266004 >> v2:172.21.15.124:3300/0 conn(0x7ff53406d160 0x7ff53406d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 <== mon.0 v1:172.21.15.26:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807897374 0 0) 0x7ff534070e50 con 0x7ff534143c40 2024-01-15T18:19:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 --> v1:172.21.15.26:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff514003520 con 0x7ff534143c40 2024-01-15T18:19:03.047 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 <== mon.0 v1:172.21.15.26:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1384525497 0 0) 0x7ff514003520 con 0x7ff534143c40 2024-01-15T18:19:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5340cc500 msgr2=0x7ff5340cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff533e00640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5340cc500 0x7ff5340cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 >> v2:172.21.15.124:3300/0 conn(0x7ff53406d160 msgr2=0x7ff53406d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff533e00640 1 --2- 172.21.15.26:0/2216266004 >> v2:172.21.15.124:3300/0 conn(0x7ff53406d160 0x7ff53406d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 --> v1:172.21.15.26:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340ccea0 con 0x7ff534143c40 2024-01-15T18:19:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.042+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 <== mon.0 v1:172.21.15.26:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2694160502 0 0) 0x7ff524001670 con 0x7ff534143c40 2024-01-15T18:19:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 >> v1:172.21.15.26:6789/0 conn(0x7ff534143c40 legacy=0x7ff534071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:03.048 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff533e00640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff514003970 0x7ff514003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff533e00640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5140047c0 0x7ff514004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff533e00640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5340cc500 0x7ff514005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340723c0 con 0x7ff5140047c0 2024-01-15T18:19:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff534144020 con 0x7ff514003970 2024-01-15T18:19:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51400ba00 con 0x7ff5340cc500 2024-01-15T18:19:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff539200640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5140047c0 0x7ff514004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff539c00640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff514003970 0x7ff514003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff53a600640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5340cc500 0x7ff514005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:03.049 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff539200640 1 -- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5340cc500 msgr2=0x7ff514005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff539200640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5340cc500 0x7ff514005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff539200640 1 -- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff514003970 msgr2=0x7ff514003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff539200640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff514003970 0x7ff514003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff539200640 1 -- 172.21.15.26:0/2216266004 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514003520 con 0x7ff5140047c0 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff539c00640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff514003970 0x7ff514003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.043+0000 7ff53a600640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff5340cc500 0x7ff514005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.045+0000 7ff539200640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5140047c0 0x7ff514004c40 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009ac0 tx=0x7ff5140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92919f6962ad72e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.045+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c02c6b0 con 0x7ff5140047c0 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.045+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff51c02c810 con 0x7ff5140047c0 2024-01-15T18:19:03.050 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.045+0000 7ff533e00640 1 -- 172.21.15.26:0/2216266004 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c02cbc0 con 0x7ff5140047c0 2024-01-15T18:19:03.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.045+0000 7ff53b000640 1 -- 172.21.15.26:0/2216266004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5140047c0 msgr2=0x7ff514004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:03.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.045+0000 7ff53b000640 1 --2- 172.21.15.26:0/2216266004 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff5140047c0 0x7ff514004c40 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7ff51c009ac0 tx=0x7ff5140057f0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.045+0000 7ff53a600640 1 -- 172.21.15.26:0/2216266004 reap_dead start 2024-01-15T18:19:03.053 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.046+0000 7ff53b000640 1 -- 172.21.15.26:0/2216266004 shutdown_connections 2024-01-15T18:19:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.046+0000 7ff53b000640 1 -- 172.21.15.26:0/2216266004 >> 172.21.15.26:0/2216266004 conn(0x7ff53406a800 msgr2=0x7ff53406ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.046+0000 7ff53b000640 1 -- 172.21.15.26:0/2216266004 shutdown_connections 2024-01-15T18:19:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.046+0000 7ff53b000640 1 -- 172.21.15.26:0/2216266004 wait complete. 2024-01-15T18:19:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.047+0000 7ff53b000640 1 Processor -- start 2024-01-15T18:19:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.048+0000 7ff53b000640 1 -- start start 2024-01-15T18:19:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.048+0000 7ff53b000640 1 --2- >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5140047c0 0x7ff534159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.048+0000 7ff53b000640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff534138660 0x7ff534138ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.048+0000 7ff53b000640 1 --2- >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff534133f00 0x7ff534134360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:03.054 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff53b000640 1 -- --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff534159bd0 con 0x7ff534138660 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff53b000640 1 -- --> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff534139020 con 0x7ff5140047c0 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff53b000640 1 -- --> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff534132c60 con 0x7ff534133f00 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff534138660 0x7ff534138ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539200640 1 --2- >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff534138660 0x7ff534138ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.26:3300/0 says I am v2:172.21.15.26:53226/0 (socket says 172.21.15.26:53226) 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539200640 1 -- 172.21.15.26:0/2856322949 learned_addr learned my addr 172.21.15.26:0/2856322949 (peer_addr_for_me v2:172.21.15.26:0/0) 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff53a600640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff534133f00 0x7ff534134360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539c00640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5140047c0 0x7ff534159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539200640 1 -- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff534133f00 msgr2=0x7ff534134360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539200640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff534133f00 0x7ff534134360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.055 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539200640 1 -- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5140047c0 msgr2=0x7ff534159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539200640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5140047c0 0x7ff534159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539200640 1 -- 172.21.15.26:0/2856322949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff534158da0 con 0x7ff534138660 2024-01-15T18:19:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff53a600640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff534133f00 0x7ff534134360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:19:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.049+0000 7ff539c00640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5140047c0 0x7ff534159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-15T18:19:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.050+0000 7ff539200640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff534138660 0x7ff534138ae0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7ff51c037040 tx=0x7ff51c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.050+0000 7ff531600640 1 -- 172.21.15.26:0/2856322949 <== mon.0 v2:172.21.15.26:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c003810 con 0x7ff534138660 2024-01-15T18:19:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.050+0000 7ff531600640 1 -- 172.21.15.26:0/2856322949 <== mon.0 v2:172.21.15.26:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff51c003970 con 0x7ff534138660 2024-01-15T18:19:03.056 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.050+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff534132050 con 0x7ff534138660 2024-01-15T18:19:03.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.050+0000 7ff531600640 1 -- 172.21.15.26:0/2856322949 <== mon.0 v2:172.21.15.26:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c003d20 con 0x7ff534138660 2024-01-15T18:19:03.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.050+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff534132610 con 0x7ff534138660 2024-01-15T18:19:03.057 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.052+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5341539c0 con 0x7ff534138660 2024-01-15T18:19:03.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.052+0000 7ff531600640 1 -- 172.21.15.26:0/2856322949 <== mon.0 v2:172.21.15.26:3300/0 4 ==== mgrmap(e 46) v1 ==== 91171+0+0 (secure 0 0 0) 0x7ff51c034440 con 0x7ff534138660 2024-01-15T18:19:03.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.053+0000 7ff531600640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff50c06c4f0 0x7ff50c06e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-15T18:19:03.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.053+0000 7ff531600640 1 -- 172.21.15.26:0/2856322949 <== mon.0 v2:172.21.15.26:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5851+0+0 (secure 0 0 0) 0x7ff51c0aa6f0 con 0x7ff534138660 2024-01-15T18:19:03.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.054+0000 7ff539c00640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff50c06c4f0 0x7ff50c06e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-15T18:19:03.058 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.055+0000 7ff539c00640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff50c06c4f0 0x7ff50c06e9b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff524001eb0 tx=0x7ff524009040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-15T18:19:03.060 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.058+0000 7ff531600640 1 -- 172.21.15.26:0/2856322949 <== mon.0 v2:172.21.15.26:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5341539c0 con 0x7ff534138660 2024-01-15T18:19:03.295 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.293+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 --> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5341539a0 con 0x7ff534138660 2024-01-15T18:19:03.296 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.294+0000 7ff531600640 1 -- 172.21.15.26:0/2856322949 <== mon.0 v2:172.21.15.26:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4915 (secure 0 0 0) 0x7ff51c0771e0 con 0x7ff534138660 2024-01-15T18:19:03.297 INFO:teuthology.orchestra.run.smithi026.stdout: 2024-01-15T18:19:03.297 INFO:teuthology.orchestra.run.smithi026.stdout:{"epoch":73,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7312,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6835/3480872940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6834","nonce":3480872940},{"type":"v1","addr":"172.21.15.124:6835","nonce":3480872940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7315,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.124:6837/662436785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.124:6836","nonce":662436785},{"type":"v1","addr":"172.21.15.124:6837","nonce":662436785}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":9222,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.26:6833/2088544410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6832","nonce":2088544410},{"type":"v1","addr":"172.21.15.26:6833","nonce":2088544410}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-15T18:12:10.468680+0000","modified":"2024-01-15T18:12:13.801724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.26:6835/483783473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.26:6834","nonce":483783473},{"type":"v1","addr":"172.21.15.26:6835","nonce":483783473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-01-15T18:19:03.297 INFO:teuthology.orchestra.run.smithi026.stderr:dumped fsmap epoch 73 2024-01-15T18:19:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.297+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff50c06c4f0 msgr2=0x7ff50c06e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.297+0000 7ff53b000640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff50c06c4f0 0x7ff50c06e9b0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff524001eb0 tx=0x7ff524009040 comp rx=0 tx=0).stop 2024-01-15T18:19:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.297+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff534138660 msgr2=0x7ff534138ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-15T18:19:03.300 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.297+0000 7ff53b000640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff534138660 0x7ff534138ae0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7ff51c037040 tx=0x7ff51c004730 comp rx=0 tx=0).stop 2024-01-15T18:19:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.298+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 shutdown_connections 2024-01-15T18:19:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.298+0000 7ff53b000640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3301/0,v1:172.21.15.124:6790/0] conn(0x7ff534133f00 0x7ff534134360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.298+0000 7ff53b000640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:6832/2029932521,v1:172.21.15.124:6833/2029932521] conn(0x7ff50c06c4f0 0x7ff50c06e9b0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.298+0000 7ff53b000640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.26:3300/0,v1:172.21.15.26:6789/0] conn(0x7ff534138660 0x7ff534138ae0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.298+0000 7ff53b000640 1 --2- 172.21.15.26:0/2856322949 >> [v2:172.21.15.124:3300/0,v1:172.21.15.124:6789/0] conn(0x7ff5140047c0 0x7ff534159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-15T18:19:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.298+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 >> 172.21.15.26:0/2856322949 conn(0x7ff53406a800 msgr2=0x7ff5340cbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-15T18:19:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.298+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 shutdown_connections 2024-01-15T18:19:03.301 INFO:teuthology.orchestra.run.smithi026.stderr:2024-01-15T18:19:03.298+0000 7ff53b000640 1 -- 172.21.15.26:0/2856322949 wait complete. 2024-01-15T18:19:03.319 DEBUG:tasks.cephfs.filesystem:already dead... 2024-01-15T18:19:03.320 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-15T18:19:03.320 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-15T18:19:03.320 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:03.345 INFO:tasks.ceph.mds.a:Stopped 2024-01-15T18:19:03.345 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-15T18:19:03.345 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:03.412 INFO:tasks.ceph.mds.c:Stopped 2024-01-15T18:19:03.412 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-15T18:19:03.413 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:03.439 INFO:tasks.ceph.mds.b:Stopped 2024-01-15T18:19:03.440 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-15T18:19:03.440 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:03.514 INFO:tasks.ceph.mds.d:Stopped 2024-01-15T18:19:03.514 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-15T18:19:03.515 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-15T18:19:03.515 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:03.728 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:19:03.736 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:19:06.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-01-15T18:19:09.617 INFO:tasks.ceph.osd.0:Stopped 2024-01-15T18:19:09.617 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-15T18:19:09.617 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:09.690 INFO:tasks.ceph.osd.1:Stopped 2024-01-15T18:19:09.690 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-15T18:19:09.690 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:09.789 INFO:tasks.ceph.osd.2:Stopped 2024-01-15T18:19:09.789 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-15T18:19:09.789 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:09.889 INFO:tasks.ceph.osd.3:Stopped 2024-01-15T18:19:09.890 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-15T18:19:09.890 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:09.977 INFO:tasks.ceph.osd.4:Stopped 2024-01-15T18:19:09.977 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-15T18:19:09.977 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:10.055 INFO:tasks.ceph.osd.5:Stopped 2024-01-15T18:19:10.056 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-01-15T18:19:10.056 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:10.126 INFO:tasks.ceph.osd.6:Stopped 2024-01-15T18:19:10.127 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-01-15T18:19:10.127 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:12.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-01-15T18:19:12.431 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-01-15T18:19:16.229 INFO:tasks.ceph.osd.7:Stopped 2024-01-15T18:19:16.229 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-15T18:19:16.229 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-01-15T18:19:16.229 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:16.326 INFO:tasks.ceph.mgr.y:Stopped 2024-01-15T18:19:16.326 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-15T18:19:16.326 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:17.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-01-15T18:19:17.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-01-15T18:19:22.428 INFO:tasks.ceph.mgr.x:Stopped 2024-01-15T18:19:22.428 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-15T18:19:22.429 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-15T18:19:22.429 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:22.487 INFO:tasks.ceph.mon.a:Stopped 2024-01-15T18:19:22.487 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-15T18:19:22.487 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:22.534 INFO:tasks.ceph.mon.b:Stopped 2024-01-15T18:19:22.534 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-15T18:19:22.534 INFO:teuthology.orchestra.run:waiting for 300 2024-01-15T18:19:22.564 INFO:tasks.ceph.mon.c:Stopped 2024-01-15T18:19:22.565 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-15T18:19:22.565 DEBUG:teuthology.orchestra.run.smithi026:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-01-15T18:19:22.682 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:19:22.682 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-15T18:19:22.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-01-15T18:19:22.885 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:19:22.885 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-15T18:19:23.060 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:19:23.060 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-15T18:19:23.249 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:19:23.249 DEBUG:teuthology.orchestra.run.smithi026:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-15T18:19:23.420 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:19:23.420 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-15T18:19:23.693 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:19:23.693 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-15T18:19:23.858 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:19:23.858 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-01-15T18:19:24.026 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:19:24.027 DEBUG:teuthology.orchestra.run.smithi124:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-01-15T18:19:24.206 INFO:tasks.ceph:Archiving mon data... 2024-01-15T18:19:24.207 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/data/mon.a.tgz 2024-01-15T18:19:24.208 DEBUG:teuthology.orchestra.run.smithi026:> mktemp 2024-01-15T18:19:24.268 INFO:teuthology.orchestra.run.smithi026.stdout:/tmp/tmp.FwN9a5MYDK 2024-01-15T18:19:24.268 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.FwN9a5MYDK 2024-01-15T18:19:24.606 DEBUG:teuthology.orchestra.run.smithi026:> sudo chmod 0666 /tmp/tmp.FwN9a5MYDK 2024-01-15T18:19:24.775 DEBUG:teuthology.orchestra.remote:smithi026:/tmp/tmp.FwN9a5MYDK is 1MB 2024-01-15T18:19:24.901 DEBUG:teuthology.orchestra.run.smithi026:> rm -fr /tmp/tmp.FwN9a5MYDK 2024-01-15T18:19:24.960 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/data/mon.b.tgz 2024-01-15T18:19:24.960 DEBUG:teuthology.orchestra.run.smithi124:> mktemp 2024-01-15T18:19:25.018 INFO:teuthology.orchestra.run.smithi124.stdout:/tmp/tmp.S309rLG4Bv 2024-01-15T18:19:25.018 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.S309rLG4Bv 2024-01-15T18:19:25.324 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0666 /tmp/tmp.S309rLG4Bv 2024-01-15T18:19:25.470 DEBUG:teuthology.orchestra.remote:smithi124:/tmp/tmp.S309rLG4Bv is 1MB 2024-01-15T18:19:25.590 DEBUG:teuthology.orchestra.run.smithi124:> rm -fr /tmp/tmp.S309rLG4Bv 2024-01-15T18:19:25.647 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/data/mon.c.tgz 2024-01-15T18:19:25.648 DEBUG:teuthology.orchestra.run.smithi124:> mktemp 2024-01-15T18:19:25.699 INFO:teuthology.orchestra.run.smithi124.stdout:/tmp/tmp.8hQYOEFH8X 2024-01-15T18:19:25.699 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.8hQYOEFH8X 2024-01-15T18:19:26.005 DEBUG:teuthology.orchestra.run.smithi124:> sudo chmod 0666 /tmp/tmp.8hQYOEFH8X 2024-01-15T18:19:26.153 DEBUG:teuthology.orchestra.remote:smithi124:/tmp/tmp.8hQYOEFH8X is 1MB 2024-01-15T18:19:26.267 DEBUG:teuthology.orchestra.run.smithi124:> rm -fr /tmp/tmp.8hQYOEFH8X 2024-01-15T18:19:26.324 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-15T18:19:26.325 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-15T18:19:26.332 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-15T18:19:26.488 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-15T18:19:26.544 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-01-15T18:19:26.544 INFO:tasks.ceph:Archiving crash dumps... 2024-01-15T18:19:26.546 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/remote/smithi026/crash 2024-01-15T18:19:26.547 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-15T18:19:26.649 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/remote/smithi124/crash 2024-01-15T18:19:26.650 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-15T18:19:26.745 INFO:tasks.ceph:Shutting down logrotate 2024-01-15T18:19:26.745 DEBUG:teuthology.orchestra.run.smithi026:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:19:26.753 DEBUG:teuthology.orchestra.run.smithi124:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-15T18:19:26.889 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-15T18:19:26.990 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-15T18:19:27.090 INFO:tasks.ceph:Compressing logs... 2024-01-15T18:19:27.090 DEBUG:teuthology.orchestra.run.smithi026:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-15T18:19:27.098 DEBUG:teuthology.orchestra.run.smithi124:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-15T18:19:27.188 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58064.log 2024-01-15T18:19:27.190 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53011.log 2024-01-15T18:19:27.191 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42366.log 2024-01-15T18:19:27.191 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47374.log 2024-01-15T18:19:27.192 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58064.log.gz 2024-01-15T18:19:27.192 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42450.log 2024-01-15T18:19:27.193 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53011.log.gz 2024-01-15T18:19:27.194 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-01-15T18:19:27.194 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.1.47374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36202.log 2024-01-15T18:19:27.195 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- 93.9% /var/log/ceph/ceph-client.1.43154.log -- replaced with /var/log/ceph/ceph-client.1.47374.log.gz 2024-01-15T18:19:27.195 INFO:teuthology.orchestra.run.smithi124.stderr: 2024-01-15T18:19:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56174.log 2024-01-15T18:19:27.196 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42366.log.gz 2024-01-15T18:19:27.196 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66545.log 2024-01-15T18:19:27.196 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42450.log.gz 2024-01-15T18:19:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56174.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56174.log.gz 2024-01-15T18:19:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41196.log 2024-01-15T18:19:27.198 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64251.log 2024-01-15T18:19:27.198 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66545.log.gz 2024-01-15T18:19:27.199 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.36202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36242.log 2024-01-15T18:19:27.199 INFO:teuthology.orchestra.run.smithi124.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36202.log.gz 2024-01-15T18:19:27.199 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.1.43154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42402.log 2024-01-15T18:19:27.200 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.36242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43918.log 2024-01-15T18:19:27.200 INFO:teuthology.orchestra.run.smithi124.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36242.log.gz 2024-01-15T18:19:27.200 INFO:teuthology.orchestra.run.smithi124.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.1.43154.log.gz 2024-01-15T18:19:27.200 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.3.42402.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-15T18:19:27.202 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.1.43918.log: 91.6% -- replaced with /var/log/ceph/ceph-client.3.42402.log.gz 2024-01-15T18:19:27.202 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-01-15T18:19:27.202 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45691.log 2024-01-15T18:19:27.203 INFO:teuthology.orchestra.run.smithi124.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.1.43918.log.gz 2024-01-15T18:19:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81780.log 2024-01-15T18:19:27.204 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41196.log.gz 2024-01-15T18:19:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64251.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30114.log 2024-01-15T18:19:27.204 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64251.log.gz 2024-01-15T18:19:27.204 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63687.log 2024-01-15T18:19:27.204 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81780.log.gz 2024-01-15T18:19:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.tmp-client.admin.30114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75203.log 2024-01-15T18:19:27.205 INFO:teuthology.orchestra.run.smithi026.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30114.log.gz 2024-01-15T18:19:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63687.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63687.log.gz 2024-01-15T18:19:27.205 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54535.log 2024-01-15T18:19:27.207 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36122.log 2024-01-15T18:19:27.207 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.1.45691.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-15T18:19:27.208 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.36122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36282.log 2024-01-15T18:19:27.208 INFO:teuthology.orchestra.run.smithi124.stderr: 88.6%/var/log/ceph/ceph-osd.4.log: -- replaced with /var/log/ceph/ceph-client.admin.36122.log.gz 2024-01-15T18:19:27.208 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67471.log 2024-01-15T18:19:27.209 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75203.log.gz 2024-01-15T18:19:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54535.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54535.log.gz 2024-01-15T18:19:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58106.log 2024-01-15T18:19:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2024-01-15T18:19:27.209 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67471.log.gz 2024-01-15T18:19:27.209 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81158.log 2024-01-15T18:19:27.209 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58106.log.gz 2024-01-15T18:19:27.212 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36322.log 2024-01-15T18:19:27.212 INFO:teuthology.orchestra.run.smithi124.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.1.45691.log.gz 2024-01-15T18:19:27.213 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.36282.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-15T18:19:27.213 INFO:teuthology.orchestra.run.smithi124.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36282.log.gz 2024-01-15T18:19:27.213 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36162.log 2024-01-15T18:19:27.213 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.36322.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36322.log.gz 2024-01-15T18:19:27.213 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-01-15T18:19:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50232.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2024-01-15T18:19:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-15T18:19:27.214 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81158.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.81158.log.gz 2024-01-15T18:19:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76869.log 2024-01-15T18:19:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64997.log 2024-01-15T18:19:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76869.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76869.log.gz 2024-01-15T18:19:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69475.log 2024-01-15T18:19:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64997.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64997.log.gz 2024-01-15T18:19:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81698.log 2024-01-15T18:19:27.215 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69475.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69475.log.gz 2024-01-15T18:19:27.218 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.36162.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-15T18:19:27.218 INFO:teuthology.orchestra.run.smithi124.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36162.log.gz 2024-01-15T18:19:27.218 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41950.log 2024-01-15T18:19:27.218 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46583.log 2024-01-15T18:19:27.219 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49503.log 2024-01-15T18:19:27.219 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81698.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81698.log.gz 2024-01-15T18:19:27.219 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-01-15T18:19:27.219 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49503.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49503.log.gz 2024-01-15T18:19:27.219 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68647.log 2024-01-15T18:19:27.220 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42198.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-01-15T18:19:27.220 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41861.log 2024-01-15T18:19:27.221 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.1.41950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44855.log 2024-01-15T18:19:27.221 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.1.46583.log: 90.8% -- replaced with /var/log/ceph/ceph-client.1.41950.log.gz 2024-01-15T18:19:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68647.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68647.log.gz 2024-01-15T18:19:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58317.log 2024-01-15T18:19:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41861.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41861.log.gz 2024-01-15T18:19:27.222 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73881.log 2024-01-15T18:19:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58317.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58317.log.gz 2024-01-15T18:19:27.223 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64685.log 2024-01-15T18:19:27.224 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.1.44855.log: 94.2% -- replaced with /var/log/ceph/ceph-client.1.46583.log.gz 2024-01-15T18:19:27.225 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36362.log 2024-01-15T18:19:27.225 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-15T18:19:27.225 INFO:teuthology.orchestra.run.smithi124.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.1.44855.log.gz 2024-01-15T18:19:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73881.log: gzip -5 --verbose -- 88.4% /var/log/ceph/ceph-client.admin.68857.log -- replaced with /var/log/ceph/ceph-client.admin.73881.log.gz 2024-01-15T18:19:27.225 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-01-15T18:19:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64685.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64685.log.gz 2024-01-15T18:19:27.225 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59222.log 2024-01-15T18:19:27.226 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.36362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.48133.log 2024-01-15T18:19:27.226 INFO:teuthology.orchestra.run.smithi124.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36362.log.gz 2024-01-15T18:19:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79411.log 2024-01-15T18:19:27.228 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.68857.log.gz 2024-01-15T18:19:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59222.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59222.log.gz 2024-01-15T18:19:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46139.log 2024-01-15T18:19:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79411.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79411.log.gz 2024-01-15T18:19:27.228 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58190.log 2024-01-15T18:19:27.230 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-15T18:19:27.231 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.1.48133.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-15T18:19:27.231 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46139.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46139.log.gz 2024-01-15T18:19:27.231 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46181.log 2024-01-15T18:19:27.231 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58190.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58190.log.gz 2024-01-15T18:19:27.231 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41022.log 2024-01-15T18:19:27.232 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46181.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46181.log.gz 2024-01-15T18:19:27.232 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54703.log 2024-01-15T18:19:27.233 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41022.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41022.log.gz 2024-01-15T18:19:27.233 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56468.log 2024-01-15T18:19:27.233 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54703.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54703.log.gz 2024-01-15T18:19:27.233 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64065.log 2024-01-15T18:19:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56468.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56468.log.gz 2024-01-15T18:19:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45770.log 2024-01-15T18:19:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64065.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64065.log.gz 2024-01-15T18:19:27.236 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48433.log 2024-01-15T18:19:27.236 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-01-15T18:19:27.236 INFO:teuthology.orchestra.run.smithi124.stderr: 96.3% -- replaced with /var/log/ceph/ceph-client.1.48133.log.gz 2024-01-15T18:19:27.237 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45770.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45770.log.gz 2024-01-15T18:19:27.238 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61714.log 2024-01-15T18:19:27.238 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48433.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48433.log.gz 2024-01-15T18:19:27.238 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44334.log 2024-01-15T18:19:27.239 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61714.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61714.log.gz 2024-01-15T18:19:27.239 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52217.log 2024-01-15T18:19:27.240 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44334.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44334.log.gz 2024-01-15T18:19:27.241 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79329.log 2024-01-15T18:19:27.241 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-15T18:19:27.241 INFO:teuthology.orchestra.run.smithi124.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36082.log 2024-01-15T18:19:27.242 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62870.log 2024-01-15T18:19:27.242 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52217.log.gz 2024-01-15T18:19:27.242 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79329.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.79329.log.gz 2024-01-15T18:19:27.243 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74517.log 2024-01-15T18:19:27.244 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62870.log: gzip 88.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.62870.log.gz 2024-01-15T18:19:27.244 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42408.log 2024-01-15T18:19:27.245 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74517.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74517.log.gz 2024-01-15T18:19:27.245 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57659.log 2024-01-15T18:19:27.246 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-mon.c.log: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-15T18:19:27.247 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42408.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42408.log.gz 2024-01-15T18:19:27.247 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45573.log 2024-01-15T18:19:27.248 INFO:teuthology.orchestra.run.smithi124.stderr:/var/log/ceph/ceph-client.admin.36082.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36082.log.gz 2024-01-15T18:19:27.248 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57659.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57659.log.gz 2024-01-15T18:19:27.248 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46323.log 2024-01-15T18:19:27.249 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45573.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45573.log.gz 2024-01-15T18:19:27.249 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50785.log 2024-01-15T18:19:27.250 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46323.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46323.log.gz 2024-01-15T18:19:27.250 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56828.log 2024-01-15T18:19:27.252 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50785.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50785.log.gz 2024-01-15T18:19:27.252 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80614.log 2024-01-15T18:19:27.252 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56828.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56828.log.gz 2024-01-15T18:19:27.253 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70671.log 2024-01-15T18:19:27.254 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56342.log 2024-01-15T18:19:27.254 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.80614.log.gz 2024-01-15T18:19:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70671.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70671.log.gz 2024-01-15T18:19:27.255 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72309.log 2024-01-15T18:19:27.257 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59436.log 2024-01-15T18:19:27.257 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56342.log.gz 2024-01-15T18:19:27.257 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72309.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72309.log.gz 2024-01-15T18:19:27.258 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41529.log 2024-01-15T18:19:27.259 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70755.log 2024-01-15T18:19:27.259 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59436.log.gz 2024-01-15T18:19:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41529.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41529.log.gz 2024-01-15T18:19:27.260 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67977.log 2024-01-15T18:19:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70755.log.gz 2024-01-15T18:19:27.262 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65477.log 2024-01-15T18:19:27.263 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67977.log: gzip -5 --verbose 88.2% -- -- replaced with /var/log/ceph/ceph-client.admin.67977.log.gz 2024-01-15T18:19:27.263 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.77376.log 2024-01-15T18:19:27.264 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61822.log 88.2% 2024-01-15T18:19:27.264 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65477.log.gz 2024-01-15T18:19:27.265 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77418.log 2024-01-15T18:19:27.265 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77376.log.gz 2024-01-15T18:19:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61822.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61822.log.gz 2024-01-15T18:19:27.267 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47493.log 2024-01-15T18:19:27.268 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44014.log 2024-01-15T18:19:27.268 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77418.log.gz 2024-01-15T18:19:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47493.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47493.log.gz 2024-01-15T18:19:27.269 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55325.log 2024-01-15T18:19:27.270 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56636.log 2024-01-15T18:19:27.270 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44014.log.gz 2024-01-15T18:19:27.271 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55325.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55325.log.gz 2024-01-15T18:19:27.272 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53465.log 2024-01-15T18:19:27.273 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77531.log 2024-01-15T18:19:27.273 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56636.log.gz 2024-01-15T18:19:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53465.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53465.log.gz 2024-01-15T18:19:27.274 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71885.log 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79974.log 2024-01-15T18:19:27.275 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77531.log.gz 2024-01-15T18:19:27.276 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71885.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71885.log.gz 2024-01-15T18:19:27.277 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68138.log 2024-01-15T18:19:27.278 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65309.log 2024-01-15T18:19:27.278 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.79974.log.gz 2024-01-15T18:19:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68138.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68138.log.gz 2024-01-15T18:19:27.279 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53576.log 2024-01-15T18:19:27.281 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71927.log 2024-01-15T18:19:27.281 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65309.log.gz 2024-01-15T18:19:27.282 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58588.log 2024-01-15T18:19:27.282 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53576.log.gz 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46569.log 2024-01-15T18:19:27.283 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71927.log.gz 2024-01-15T18:19:27.284 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80822.log 2024-01-15T18:19:27.285 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58588.log.gz 2024-01-15T18:19:27.286 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75533.log 2024-01-15T18:19:27.286 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46569.log.gz 2024-01-15T18:19:27.287 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60291.log 2024-01-15T18:19:27.287 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80822.log.gz 2024-01-15T18:19:27.288 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53786.log 2024-01-15T18:19:27.288 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75533.log.gz 2024-01-15T18:19:27.289 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60291.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60291.log.gz 2024-01-15T18:19:27.293 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63645.log 2024-01-15T18:19:27.293 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54986.log 2024-01-15T18:19:27.293 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53786.log.gz 2024-01-15T18:19:27.293 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-01-15T18:19:27.293 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63645.log.gz 2024-01-15T18:19:27.294 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78636.log 2024-01-15T18:19:27.294 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54986.log.gz 2024-01-15T18:19:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49892.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 2024-01-15T18:19:27.295 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54080.log 2024-01-15T18:19:27.296 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75407.log 2024-01-15T18:19:27.296 INFO:teuthology.orchestra.run.smithi026.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.78636.log.gz 2024-01-15T18:19:27.297 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46223.log 2024-01-15T18:19:27.297 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54080.log.gz 2024-01-15T18:19:27.298 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73409.log 2024-01-15T18:19:27.299 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75407.log.gz 2024-01-15T18:19:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46223.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46223.log.gz 2024-01-15T18:19:27.300 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47781.log 2024-01-15T18:19:27.301 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46905.log 2024-01-15T18:19:27.302 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73409.log.gz 2024-01-15T18:19:27.302 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57920.log 2024-01-15T18:19:27.302 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47781.log.gz 2024-01-15T18:19:27.304 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40219.log 2024-01-15T18:19:27.304 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46905.log.gz 2024-01-15T18:19:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57920.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57920.log.gz 2024-01-15T18:19:27.305 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60968.log 2024-01-15T18:19:27.306 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49335.log 2024-01-15T18:19:27.307 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40219.log.gz 2024-01-15T18:19:27.308 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56552.log 2024-01-15T18:19:27.308 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60968.log.gz 2024-01-15T18:19:27.309 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70797.log 2024-01-15T18:19:27.309 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49335.log.gz 2024-01-15T18:19:27.310 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56552.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56552.log.gz 2024-01-15T18:19:27.311 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51148.log 2024-01-15T18:19:27.312 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53828.log 2024-01-15T18:19:27.312 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70797.log.gz 2024-01-15T18:19:27.313 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51274.log 2024-01-15T18:19:27.313 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51148.log.gz 2024-01-15T18:19:27.314 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79616.log 2024-01-15T18:19:27.314 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53828.log.gz 2024-01-15T18:19:27.316 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71334.log 2024-01-15T18:19:27.316 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51274.log.gz 2024-01-15T18:19:27.317 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64376.log 2024-01-15T18:19:27.317 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79616.log.gz 2024-01-15T18:19:27.318 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55714.log 2024-01-15T18:19:27.318 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71334.log.gz 2024-01-15T18:19:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64376.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.64376.log.gz 2024-01-15T18:19:27.320 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61098.log 2024-01-15T18:19:27.321 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47949.log 2024-01-15T18:19:27.321 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55714.log.gz 2024-01-15T18:19:27.322 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74353.log 2024-01-15T18:19:27.322 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61098.log.gz 2024-01-15T18:19:27.323 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62238.log 2024-01-15T18:19:27.324 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47949.log.gz 2024-01-15T18:19:27.325 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67588.log 2024-01-15T18:19:27.325 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74353.log.gz 2024-01-15T18:19:27.326 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41874.log 2024-01-15T18:19:27.326 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.62238.log.gz 2024-01-15T18:19:27.327 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41202.log 2024-01-15T18:19:27.328 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67588.log.gz 2024-01-15T18:19:27.329 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50276.log 2024-01-15T18:19:27.329 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41874.log.gz 2024-01-15T18:19:27.330 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54325.log 2024-01-15T18:19:27.330 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41202.log.gz 2024-01-15T18:19:27.331 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59053.log 2024-01-15T18:19:27.332 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50276.log.gz 2024-01-15T18:19:27.333 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71502.log 2024-01-15T18:19:27.333 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54325.log.gz 2024-01-15T18:19:27.334 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71460.log 2024-01-15T18:19:27.334 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59053.log.gz 2024-01-15T18:19:27.335 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62710.log 2024-01-15T18:19:27.335 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71502.log.gz 2024-01-15T18:19:27.337 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59394.log 2024-01-15T18:19:27.337 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.71460.log.gz 2024-01-15T18:19:27.338 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46695.log 2024-01-15T18:19:27.338 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62710.log.gz 2024-01-15T18:19:27.339 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57962.log 2024-01-15T18:19:27.339 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59394.log.gz 2024-01-15T18:19:27.341 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46695.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46695.log.gz 2024-01-15T18:19:27.341 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68521.log 2024-01-15T18:19:27.342 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42534.log 2024-01-15T18:19:27.342 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57962.log.gz 2024-01-15T18:19:27.343 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41189.log 2024-01-15T18:19:27.343 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.68521.log.gz 2024-01-15T18:19:27.345 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42534.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42534.log.gz 2024-01-15T18:19:27.345 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42492.log 2024-01-15T18:19:27.346 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74796.log 2024-01-15T18:19:27.346 INFO:teuthology.orchestra.run.smithi026.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41189.log.gz 2024-01-15T18:19:27.347 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64499.log 2024-01-15T18:19:27.347 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42492.log.gz 2024-01-15T18:19:27.348 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69208.log 2024-01-15T18:19:27.348 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74796.log.gz 2024-01-15T18:19:27.349 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52007.log 2024-01-15T18:19:27.350 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64499.log.gz 2024-01-15T18:19:27.351 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.69208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76009.log 2024-01-15T18:19:27.352 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48033.log 2024-01-15T18:19:27.352 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52007.log.gz 2024-01-15T18:19:27.354 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52091.log 2024-01-15T18:19:27.354 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76009.log.gz 2024-01-15T18:19:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48033.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48033.log.gz 2024-01-15T18:19:27.355 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50622.log 2024-01-15T18:19:27.357 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48601.log 2024-01-15T18:19:27.357 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52091.log.gz 2024-01-15T18:19:27.358 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51316.log 2024-01-15T18:19:27.358 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50622.log.gz 2024-01-15T18:19:27.359 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58967.log 2024-01-15T18:19:27.360 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48601.log.gz 2024-01-15T18:19:27.361 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40042.log 2024-01-15T18:19:27.361 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51316.log.gz 2024-01-15T18:19:27.362 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48119.log 2024-01-15T18:19:27.363 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58967.log.gz 2024-01-15T18:19:27.364 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76701.log 2024-01-15T18:19:27.364 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40042.log.gz 2024-01-15T18:19:27.365 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80948.log 2024-01-15T18:19:27.365 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48119.log.gz 2024-01-15T18:19:27.366 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47991.log 2024-01-15T18:19:27.367 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76701.log.gz 2024-01-15T18:19:27.368 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56384.log 2024-01-15T18:19:27.368 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80948.log.gz 2024-01-15T18:19:27.369 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50947.log 2024-01-15T18:19:27.369 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47991.log.gz 2024-01-15T18:19:27.371 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63603.log 2024-01-15T18:19:27.371 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.56384.log.gz 2024-01-15T18:19:27.372 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44208.log 2024-01-15T18:19:27.373 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50947.log.gz 2024-01-15T18:19:27.373 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74025.log 2024-01-15T18:19:27.374 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63603.log.gz 2024-01-15T18:19:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44208.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44208.log.gz 2024-01-15T18:19:27.375 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76575.log 2024-01-15T18:19:27.376 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51064.log 2024-01-15T18:19:27.376 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74025.log.gz 2024-01-15T18:19:27.377 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74982.log 2024-01-15T18:19:27.377 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76575.log.gz 2024-01-15T18:19:27.379 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81578.log 2024-01-15T18:19:27.379 INFO:teuthology.orchestra.run.smithi026.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51064.log.gz 2024-01-15T18:19:27.380 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47573.log 2024-01-15T18:19:27.380 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74982.log.gz 2024-01-15T18:19:27.381 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81578.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81578.log.gz 2024-01-15T18:19:27.382 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79748.log 2024-01-15T18:19:27.383 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56720.log 2024-01-15T18:19:27.383 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47573.log.gz 2024-01-15T18:19:27.384 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80488.log 2024-01-15T18:19:27.384 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79748.log.gz 2024-01-15T18:19:27.385 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56720.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56720.log.gz 2024-01-15T18:19:27.385 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78859.log 2024-01-15T18:19:27.386 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74252.log 2024-01-15T18:19:27.386 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80488.log.gz 2024-01-15T18:19:27.387 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78002.log 2024-01-15T18:19:27.388 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78859.log.gz 2024-01-15T18:19:27.388 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74252.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74252.log.gz 2024-01-15T18:19:27.389 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60760.log 2024-01-15T18:19:27.390 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66419.log 2024-01-15T18:19:27.390 INFO:teuthology.orchestra.run.smithi026.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.78002.log.gz 2024-01-15T18:19:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60760.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60760.log.gz 2024-01-15T18:19:27.391 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51657.log 2024-01-15T18:19:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66419.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66419.log.gz 2024-01-15T18:19:27.392 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71376.log 2024-01-15T18:19:27.393 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51657.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51657.log.gz 2024-01-15T18:19:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78941.log 2024-01-15T18:19:27.394 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50061.log 2024-01-15T18:19:27.395 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.71376.log.gz 2024-01-15T18:19:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78941.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.78941.log.gz 2024-01-15T18:19:27.396 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47325.log 2024-01-15T18:19:27.397 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66377.log 2024-01-15T18:19:27.397 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50061.log.gz 2024-01-15T18:19:27.398 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78513.log 2024-01-15T18:19:27.398 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47325.log.gz 2024-01-15T18:19:27.399 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64955.log 2024-01-15T18:19:27.399 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66377.log.gz 2024-01-15T18:19:27.400 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78513.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.78513.log.gz 2024-01-15T18:19:27.400 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59268.log 2024-01-15T18:19:27.401 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70107.log 2024-01-15T18:19:27.401 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64955.log.gz 2024-01-15T18:19:27.403 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59268.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59268.log.gz 2024-01-15T18:19:27.403 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52969.log 2024-01-15T18:19:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70107.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70107.log.gz 2024-01-15T18:19:27.404 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53870.log 2024-01-15T18:19:27.405 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52969.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52969.log.gz 2024-01-15T18:19:27.405 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54409.log 2024-01-15T18:19:27.406 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65729.log 2024-01-15T18:19:27.406 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53870.log.gz 2024-01-15T18:19:27.407 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54409.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54409.log.gz 2024-01-15T18:19:27.407 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60884.log 2024-01-15T18:19:27.408 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81536.log 2024-01-15T18:19:27.409 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65729.log.gz 2024-01-15T18:19:27.409 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60884.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60884.log.gz 2024-01-15T18:19:27.410 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80446.log 2024-01-15T18:19:27.411 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46055.log 2024-01-15T18:19:27.411 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81536.log.gz 2024-01-15T18:19:27.412 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70419.log 2024-01-15T18:19:27.412 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80446.log.gz 2024-01-15T18:19:27.413 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65519.log 2024-01-15T18:19:27.413 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46055.log.gz 2024-01-15T18:19:27.414 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70419.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70419.log.gz -5 2024-01-15T18:19:27.414 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44460.log 2024-01-15T18:19:27.415 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63855.log 2024-01-15T18:19:27.415 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65519.log.gz 2024-01-15T18:19:27.416 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44460.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44460.log.gz 2024-01-15T18:19:27.417 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81740.log 2024-01-15T18:19:27.418 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67150.log 2024-01-15T18:19:27.418 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63855.log.gz 2024-01-15T18:19:27.419 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81740.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81740.log.gz 2024-01-15T18:19:27.419 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40422.log 2024-01-15T18:19:27.420 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48349.log 2024-01-15T18:19:27.420 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67150.log.gz 2024-01-15T18:19:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40422.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40422.log.gz 2024-01-15T18:19:27.421 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81032.log 2024-01-15T18:19:27.422 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74168.log 2024-01-15T18:19:27.422 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48349.log.gz 2024-01-15T18:19:27.423 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81032.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81032.log.gz 2024-01-15T18:19:27.423 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44796.log 2024-01-15T18:19:27.424 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55577.log 2024-01-15T18:19:27.424 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74168.log.gz 2024-01-15T18:19:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44796.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44796.log.gz 2024-01-15T18:19:27.426 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79370.log 2024-01-15T18:19:27.427 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60132.log 2024-01-15T18:19:27.427 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55577.log.gz 2024-01-15T18:19:27.428 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79370.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.79370.log.gzgzip 2024-01-15T18:19:27.428 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45453.log 2024-01-15T18:19:27.429 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54661.log 2024-01-15T18:19:27.429 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60132.log.gz 2024-01-15T18:19:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45453.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45453.log.gz 2024-01-15T18:19:27.430 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65393.log 2024-01-15T18:19:27.431 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56088.log 2024-01-15T18:19:27.432 INFO:teuthology.orchestra.run.smithi026.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54661.log.gz 2024-01-15T18:19:27.432 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65393.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.65393.log.gz 2024-01-15T18:19:27.433 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60174.log 2024-01-15T18:19:27.434 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45939.log 2024-01-15T18:19:27.434 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56088.log.gz 2024-01-15T18:19:27.435 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53913.log 2024-01-15T18:19:27.435 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60174.log.gz 2024-01-15T18:19:27.436 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61504.log 2024-01-15T18:19:27.436 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45939.log.gz 2024-01-15T18:19:27.437 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53913.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53913.log.gz 2024-01-15T18:19:27.437 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74679.log 2024-01-15T18:19:27.438 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63308.log 2024-01-15T18:19:27.438 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61504.log.gz 2024-01-15T18:19:27.439 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74679.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74679.log.gz 2024-01-15T18:19:27.439 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62828.log 2024-01-15T18:19:27.440 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73451.log 2024-01-15T18:19:27.440 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63308.log.gz 2024-01-15T18:19:27.441 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62828.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62828.log.gz 2024-01-15T18:19:27.442 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78185.log 2024-01-15T18:19:27.442 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59180.log 2024-01-15T18:19:27.443 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73451.log.gz 2024-01-15T18:19:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78185.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.78185.log.gz 2024-01-15T18:19:27.444 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50318.log 2024-01-15T18:19:27.445 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62668.log 2024-01-15T18:19:27.445 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59180.log.gz 2024-01-15T18:19:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50318.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50318.log.gz 2024-01-15T18:19:27.446 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59970.log 2024-01-15T18:19:27.447 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46449.log 2024-01-15T18:19:27.447 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62668.log.gz 2024-01-15T18:19:27.448 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59970.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59970.log.gz 2024-01-15T18:19:27.448 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54194.log 2024-01-15T18:19:27.449 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79106.log 2024-01-15T18:19:27.449 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46449.log.gz 2024-01-15T18:19:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54194.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54194.log.gz 2024-01-15T18:19:27.450 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64811.log 2024-01-15T18:19:27.451 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65141.log 2024-01-15T18:19:27.452 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79106.log.gz 2024-01-15T18:19:27.453 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64811.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64811.log.gz 2024-01-15T18:19:27.453 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70503.log 2024-01-15T18:19:27.454 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52049.log 2024-01-15T18:19:27.454 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65141.log.gz 2024-01-15T18:19:27.455 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70503.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70503.log.gz -5 2024-01-15T18:19:27.455 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47409.log 2024-01-15T18:19:27.456 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68689.log 2024-01-15T18:19:27.456 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52049.log.gz 2024-01-15T18:19:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47409.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47409.log.gz 2024-01-15T18:19:27.457 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48265.log 2024-01-15T18:19:27.458 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69559.log 2024-01-15T18:19:27.458 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68689.log.gz 2024-01-15T18:19:27.459 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48265.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48265.log.gz 2024-01-15T18:19:27.459 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51965.log 2024-01-15T18:19:27.460 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69559.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69559.log.gz 2024-01-15T18:19:27.460 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72055.log 2024-01-15T18:19:27.461 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51965.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51965.log.gz 2024-01-15T18:19:27.462 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55960.log 2024-01-15T18:19:27.462 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80142.log 2024-01-15T18:19:27.462 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72055.log.gz 2024-01-15T18:19:27.463 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55960.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55960.log.gz 2024-01-15T18:19:27.464 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64727.log 2024-01-15T18:19:27.464 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72717.log 2024-01-15T18:19:27.465 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80142.log.gz 2024-01-15T18:19:27.465 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64727.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64727.log.gz 2024-01-15T18:19:27.466 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36546.log 2024-01-15T18:19:27.467 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54846.log 2024-01-15T18:19:27.467 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.72717.log.gz 2024-01-15T18:19:27.468 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.36546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47241.log 2024-01-15T18:19:27.468 INFO:teuthology.orchestra.run.smithi026.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36546.log.gz 2024-01-15T18:19:27.469 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45981.log 2024-01-15T18:19:27.469 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54846.log.gz 2024-01-15T18:19:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47241.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47241.log.gz 2024-01-15T18:19:27.470 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55917.log 2024-01-15T18:19:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45981.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45981.log.gz 2024-01-15T18:19:27.471 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42324.log 2024-01-15T18:19:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55917.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55917.log.gz 2024-01-15T18:19:27.472 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73102.log 2024-01-15T18:19:27.473 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42324.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.42324.log.gz /var/log/ceph/ceph-client.admin.74433.log 2024-01-15T18:19:27.473 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-01-15T18:19:27.474 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.73102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81452.log 2024-01-15T18:19:27.476 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52403.log 2024-01-15T18:19:27.476 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74433.log.gz 2024-01-15T18:19:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81452.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81452.log.gz 2024-01-15T18:19:27.477 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68605.log 2024-01-15T18:19:27.478 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51741.log 2024-01-15T18:19:27.478 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52403.log.gz 2024-01-15T18:19:27.480 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55535.log 2024-01-15T18:19:27.480 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68605.log.gz 2024-01-15T18:19:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51741.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51741.log.gz 2024-01-15T18:19:27.481 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76159.log 2024-01-15T18:19:27.482 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70873.log 2024-01-15T18:19:27.483 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55535.log.gz 2024-01-15T18:19:27.484 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48075.log 2024-01-15T18:19:27.484 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76159.log.gz 2024-01-15T18:19:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70873.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70873.log.gz 2024-01-15T18:19:27.485 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65561.log 2024-01-15T18:19:27.486 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67714.log 2024-01-15T18:19:27.487 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48075.log.gz 2024-01-15T18:19:27.488 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75491.log 2024-01-15T18:19:27.488 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65561.log.gz 2024-01-15T18:19:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67714.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67714.log.gz 2024-01-15T18:19:27.489 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53339.log 2024-01-15T18:19:27.490 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78818.log 2024-01-15T18:19:27.490 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75491.log.gz 2024-01-15T18:19:27.491 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53339.log: gzip 88.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.53339.log.gz -- 2024-01-15T18:19:27.492 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.59138.log 2024-01-15T18:19:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78818.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.78818.log.gz 2024-01-15T18:19:27.493 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75841.log 2024-01-15T18:19:27.494 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72435.log 2024-01-15T18:19:27.494 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59138.log.gz 2024-01-15T18:19:27.496 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41192.log 2024-01-15T18:19:27.496 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75841.log.gz 2024-01-15T18:19:27.497 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67291.log 2024-01-15T18:19:27.497 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72435.log.gz 2024-01-15T18:19:27.499 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63477.log 2024-01-15T18:19:27.499 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41192.log.gz 2024-01-15T18:19:27.499 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67630.log 2024-01-15T18:19:27.500 INFO:teuthology.orchestra.run.smithi026.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.67291.log.gz 2024-01-15T18:19:27.501 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68223.log 2024-01-15T18:19:27.501 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63477.log.gz 2024-01-15T18:19:27.502 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67630.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67630.log.gz 2024-01-15T18:19:27.502 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47739.log 2024-01-15T18:19:27.503 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77573.log 2024-01-15T18:19:27.504 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68223.log.gz 2024-01-15T18:19:27.505 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61336.log 2024-01-15T18:19:27.505 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47739.log.gz 2024-01-15T18:19:27.506 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77573.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77573.log.gz 2024-01-15T18:19:27.506 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50018.log 2024-01-15T18:19:27.507 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45369.log 2024-01-15T18:19:27.508 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61336.log.gz 2024-01-15T18:19:27.509 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66461.log 2024-01-15T18:19:27.509 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50018.log.gz 2024-01-15T18:19:27.510 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45369.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45369.log.gz 2024-01-15T18:19:27.510 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56510.log 2024-01-15T18:19:27.511 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45006.log 2024-01-15T18:19:27.511 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66461.log.gz 2024-01-15T18:19:27.513 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76051.log 2024-01-15T18:19:27.513 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56510.log.gz 2024-01-15T18:19:27.514 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45006.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45006.log.gz 2024-01-15T18:19:27.514 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79790.log 2024-01-15T18:19:27.515 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61420.log 2024-01-15T18:19:27.516 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76051.log.gz 2024-01-15T18:19:27.517 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66755.log 2024-01-15T18:19:27.517 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.79790.log.gz 2024-01-15T18:19:27.519 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74595.log 2024-01-15T18:19:27.519 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61420.log.gz 2024-01-15T18:19:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66755.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66755.log.gz 2024-01-15T18:19:27.520 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59706.log 2024-01-15T18:19:27.521 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59790.log 2024-01-15T18:19:27.521 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74595.log.gz 2024-01-15T18:19:27.522 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77615.log 2024-01-15T18:19:27.523 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59706.log.gz 2024-01-15T18:19:27.524 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70545.log 2024-01-15T18:19:27.524 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59790.log.gz 2024-01-15T18:19:27.525 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48391.log 2024-01-15T18:19:27.525 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77615.log.gz 2024-01-15T18:19:27.526 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64643.log 2024-01-15T18:19:27.527 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70545.log.gz 2024-01-15T18:19:27.528 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72099.log 2024-01-15T18:19:27.528 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48391.log.gz 2024-01-15T18:19:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64643.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64643.log.gz 2024-01-15T18:19:27.529 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41531.log 2024-01-15T18:19:27.530 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74084.log 2024-01-15T18:19:27.531 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72099.log.gz 2024-01-15T18:19:27.532 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49774.log 2024-01-15T18:19:27.532 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41531.log.gz 2024-01-15T18:19:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74084.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.74084.log.gz 2024-01-15T18:19:27.533 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58672.log 2024-01-15T18:19:27.534 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73493.log 2024-01-15T18:19:27.535 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49774.log.gz 2024-01-15T18:19:27.536 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44166.log 2024-01-15T18:19:27.536 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58672.log.gz 2024-01-15T18:19:27.537 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48223.log 2024-01-15T18:19:27.538 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73493.log.gz 2024-01-15T18:19:27.539 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45661.log 2024-01-15T18:19:27.539 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44166.log.gz 2024-01-15T18:19:27.540 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62996.log 2024-01-15T18:19:27.540 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48223.log.gz 2024-01-15T18:19:27.541 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67756.log 2024-01-15T18:19:27.542 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45661.log.gz 2024-01-15T18:19:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62996.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62996.log.gz 2024-01-15T18:19:27.543 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81638.log 2024-01-15T18:19:27.544 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60543.log 2024-01-15T18:19:27.544 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67756.log.gz 2024-01-15T18:19:27.545 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66797.log 2024-01-15T18:19:27.545 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81638.log.gz 2024-01-15T18:19:27.547 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49293.log 2024-01-15T18:19:27.547 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60543.log.gz 2024-01-15T18:19:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66797.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66797.log.gz 2024-01-15T18:19:27.548 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63435.log 2024-01-15T18:19:27.549 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49293.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-15T18:19:27.550 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49293.log.gz 2024-01-15T18:19:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63435.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63435.log.gz 2024-01-15T18:19:27.551 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75967.log 2024-01-15T18:19:27.552 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75671.log 2024-01-15T18:19:27.553 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64853.log 2024-01-15T18:19:27.554 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75967.log.gz 2024-01-15T18:19:27.555 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40938.log 2024-01-15T18:19:27.555 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75671.log.gz 2024-01-15T18:19:27.556 INFO:teuthology.orchestra.run.smithi026.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-15T18:19:27.556 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40602.log 2024-01-15T18:19:27.557 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64853.log.gz 2024-01-15T18:19:27.557 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66839.log 2024-01-15T18:19:27.558 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40938.log.gz 2024-01-15T18:19:27.558 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40602.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40602.log.gz 2024-01-15T18:19:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44964.log 2024-01-15T18:19:27.559 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75365.log 2024-01-15T18:19:27.560 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66839.log.gz 2024-01-15T18:19:27.561 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44964.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44964.log.gz -5 2024-01-15T18:19:27.561 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.78267.log 2024-01-15T18:19:27.562 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75365.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75365.log.gz 2024-01-15T18:19:27.563 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72674.log 2024-01-15T18:19:27.563 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70233.log 2024-01-15T18:19:27.564 INFO:teuthology.orchestra.run.smithi026.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.78267.log.gz 2024-01-15T18:19:27.565 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53618.log 2024-01-15T18:19:27.565 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.72674.log.gz 2024-01-15T18:19:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70233.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70233.log.gz 2024-01-15T18:19:27.566 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78472.log 2024-01-15T18:19:27.567 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80100.log 2024-01-15T18:19:27.568 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53618.log.gz 2024-01-15T18:19:27.569 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67513.log 2024-01-15T18:19:27.569 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78472.log.gz 2024-01-15T18:19:27.570 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47199.log 2024-01-15T18:19:27.570 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80100.log.gz 2024-01-15T18:19:27.572 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67513.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.67513.log.gz 2024-01-15T18:19:27.572 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45855.log 2024-01-15T18:19:27.572 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81822.log 2024-01-15T18:19:27.573 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47199.log.gz 2024-01-15T18:19:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45855.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45855.log.gz 2024-01-15T18:19:27.574 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70629.log 2024-01-15T18:19:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81822.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81822.log.gz 2024-01-15T18:19:27.575 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48307.log 2024-01-15T18:19:27.576 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65645.log 2024-01-15T18:19:27.577 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70629.log.gz 2024-01-15T18:19:27.578 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69939.log 2024-01-15T18:19:27.578 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48307.log.gz 2024-01-15T18:19:27.579 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55367.log 2024-01-15T18:19:27.580 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65645.log.gz 2024-01-15T18:19:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69939.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69939.log.gz 2024-01-15T18:19:27.581 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55124.log 2024-01-15T18:19:27.582 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48811.log 2024-01-15T18:19:27.582 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55367.log.gz 2024-01-15T18:19:27.583 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45185.log 88.9% 2024-01-15T18:19:27.583 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55124.log.gz 2024-01-15T18:19:27.585 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54619.log 2024-01-15T18:19:27.585 INFO:teuthology.orchestra.run.smithi026.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.48811.log.gz 2024-01-15T18:19:27.586 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67798.log 2024-01-15T18:19:27.586 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45185.log.gz 2024-01-15T18:19:27.587 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54619.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54619.log.gz 2024-01-15T18:19:27.587 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45813.log 2024-01-15T18:19:27.588 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67798.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67798.log.gz 2024-01-15T18:19:27.589 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81242.log 2024-01-15T18:19:27.589 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41204.log 2024-01-15T18:19:27.590 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45813.log.gz 2024-01-15T18:19:27.591 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42804.log 2024-01-15T18:19:27.591 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.81242.log.gz 2024-01-15T18:19:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41204.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41204.log.gz 2024-01-15T18:19:27.592 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58756.log 2024-01-15T18:19:27.593 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36173.log 2024-01-15T18:19:27.594 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42804.log.gz 2024-01-15T18:19:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58756.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58756.log.gz 2024-01-15T18:19:27.595 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69821.log 2024-01-15T18:19:27.596 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.36173.log: 92.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.36173.log.gz 2024-01-15T18:19:27.596 INFO:teuthology.orchestra.run.smithi026.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50701.log 2024-01-15T18:19:27.597 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47451.log 2024-01-15T18:19:27.598 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69821.log.gz 2024-01-15T18:19:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50701.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50701.log.gz 2024-01-15T18:19:27.599 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40380.log 2024-01-15T18:19:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47451.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47451.log.gz 2024-01-15T18:19:27.600 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52301.log 2024-01-15T18:19:27.601 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68563.log 2024-01-15T18:19:27.602 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40380.log.gz 2024-01-15T18:19:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52301.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52301.log.gz 2024-01-15T18:19:27.603 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55166.log 2024-01-15T18:19:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68563.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68563.log.gz 2024-01-15T18:19:27.604 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67091.log 2024-01-15T18:19:27.605 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69779.log 2024-01-15T18:19:27.605 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55166.log.gz 2024-01-15T18:19:27.606 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70149.log 2024-01-15T18:19:27.607 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67091.log.gz 2024-01-15T18:19:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69779.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69779.log.gz 2024-01-15T18:19:27.608 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62322.log 2024-01-15T18:19:27.609 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78982.log 2024-01-15T18:19:27.609 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70149.log.gz 2024-01-15T18:19:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62322.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62322.log.gz 2024-01-15T18:19:27.610 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52830.log 2024-01-15T18:19:27.612 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80320.log 2024-01-15T18:19:27.612 INFO:teuthology.orchestra.run.smithi026.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.78982.log.gz 2024-01-15T18:19:27.613 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.52830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40770.log 2024-01-15T18:19:27.615 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47907.log 2024-01-15T18:19:27.615 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80320.log.gz 2024-01-15T18:19:27.616 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59849.log 2024-01-15T18:19:27.616 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40770.log.gz 2024-01-15T18:19:27.618 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81494.log 2024-01-15T18:19:27.618 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47907.log.gz 2024-01-15T18:19:27.619 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48877.log 2024-01-15T18:19:27.619 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59849.log.gz 2024-01-15T18:19:27.620 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58882.log 2024-01-15T18:19:27.620 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.81494.log.gz 2024-01-15T18:19:27.621 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62448.log 2024-01-15T18:19:27.622 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48877.log.gz 2024-01-15T18:19:27.623 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66023.log 2024-01-15T18:19:27.623 INFO:teuthology.orchestra.run.smithi026.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.58882.log.gz 2024-01-15T18:19:27.624 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53137.log 2024-01-15T18:19:27.624 INFO:teuthology.orchestra.run.smithi026.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62448.log.gz 2024-01-15T18:19:27.626 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72561.log 2024-01-15T18:19:27.626 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66023.log.gz 2024-01-15T18:19:27.627 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57358.log 2024-01-15T18:19:27.627 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53137.log.gz 2024-01-15T18:19:27.629 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81116.log 2024-01-15T18:19:27.629 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.72561.log.gz 2024-01-15T18:19:27.630 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71418.log 2024-01-15T18:19:27.630 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57358.log.gz 2024-01-15T18:19:27.632 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61148.log 2024-01-15T18:19:27.632 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.81116.log.gz 2024-01-15T18:19:27.633 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56002.log 2024-01-15T18:19:27.633 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.71418.log.gz 2024-01-15T18:19:27.634 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71292.log 2024-01-15T18:19:27.634 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61148.log.gz 2024-01-15T18:19:27.635 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52445.log 2024-01-15T18:19:27.635 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56002.log.gz 2024-01-15T18:19:27.637 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65435.log 2024-01-15T18:19:27.637 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71292.log.gz 2024-01-15T18:19:27.638 INFO:teuthology.orchestra.run.smithi124.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-01-15T18:19:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48475.log 2024-01-15T18:19:27.639 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52445.log.gz 2024-01-15T18:19:27.639 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43236.log 2024-01-15T18:19:27.640 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65435.log.gz 2024-01-15T18:19:27.641 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40138.log 2024-01-15T18:19:27.641 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48475.log.gz 2024-01-15T18:19:27.642 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.43236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65603.log 2024-01-15T18:19:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40138.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.43236.log.gz 2024-01-15T18:19:27.644 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66629.log 2024-01-15T18:19:27.644 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40138.log.gz 2024-01-15T18:19:27.645 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81284.log 2024-01-15T18:19:27.645 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65603.log.gz 2024-01-15T18:19:27.647 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81074.log 2024-01-15T18:19:27.647 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66629.log.gz 2024-01-15T18:19:27.648 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41186.log 2024-01-15T18:19:27.648 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.81284.log.gz 2024-01-15T18:19:27.649 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40516.log 2024-01-15T18:19:27.649 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81074.log.gz 2024-01-15T18:19:27.651 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51699.log 2024-01-15T18:19:27.651 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41186.log.gz 2024-01-15T18:19:27.652 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76785.log 2024-01-15T18:19:27.652 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40516.log.gz 2024-01-15T18:19:27.653 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56258.log 2024-01-15T18:19:27.654 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51699.log.gz 2024-01-15T18:19:27.655 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42072.log 2024-01-15T18:19:27.655 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76785.log.gz 2024-01-15T18:19:27.656 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77090.log 2024-01-15T18:19:27.656 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56258.log.gz 2024-01-15T18:19:27.657 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45143.log 2024-01-15T18:19:27.658 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42072.log.gz 2024-01-15T18:19:27.659 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65813.log 2024-01-15T18:19:27.659 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77090.log.gz 2024-01-15T18:19:27.660 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70953.log 2024-01-15T18:19:27.660 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45143.log.gz 2024-01-15T18:19:27.662 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-01-15T18:19:27.662 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65813.log.gz 2024-01-15T18:19:27.663 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75577.log 2024-01-15T18:19:27.663 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70953.log.gz 2024-01-15T18:19:27.664 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48769.log 2024-01-15T18:19:27.664 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-01-15T18:19:27.666 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46779.log 2024-01-15T18:19:27.666 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75577.log.gz 2024-01-15T18:19:27.667 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74922.log 2024-01-15T18:19:27.668 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48769.log.gz 2024-01-15T18:19:27.668 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56426.log 2024-01-15T18:19:27.669 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46779.log.gz 2024-01-15T18:19:27.670 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78349.log 2024-01-15T18:19:27.670 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74922.log.gz 2024-01-15T18:19:27.672 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69863.log 2024-01-15T18:19:27.672 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56426.log.gz 2024-01-15T18:19:27.673 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52175.log 2024-01-15T18:19:27.673 INFO:teuthology.orchestra.run.smithi026.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.78349.log.gz 2024-01-15T18:19:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69863.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69863.log.gz 2024-01-15T18:19:27.674 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80864.log 2024-01-15T18:19:27.676 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71175.log 2024-01-15T18:19:27.676 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52175.log.gz 2024-01-15T18:19:27.677 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79534.log 2024-01-15T18:19:27.678 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80864.log.gz 2024-01-15T18:19:27.679 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58840.log 2024-01-15T18:19:27.679 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71175.log.gz 2024-01-15T18:19:27.680 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49377.log 2024-01-15T18:19:27.680 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79534.log.gz 2024-01-15T18:19:27.681 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62912.log 2024-01-15T18:19:27.681 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58840.log.gz 2024-01-15T18:19:27.683 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57105.log 2024-01-15T18:19:27.683 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49377.log.gz 2024-01-15T18:19:27.684 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63729.log 2024-01-15T18:19:27.685 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62912.log.gz 2024-01-15T18:19:27.686 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.57105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61588.log 2024-01-15T18:19:27.687 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47157.log 2024-01-15T18:19:27.687 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63729.log.gz 2024-01-15T18:19:27.689 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68180.log 2024-01-15T18:19:27.689 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61588.log.gz 2024-01-15T18:19:27.690 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65225.log 2024-01-15T18:19:27.690 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47157.log.gz 2024-01-15T18:19:27.692 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68180.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68180.log.gz 2024-01-15T18:19:27.692 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70335.log 2024-01-15T18:19:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65225.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.65225.log.gz 2024-01-15T18:19:27.693 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62786.log 2024-01-15T18:19:27.695 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70335.log: gzip 88.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.70335.log.gz -- 2024-01-15T18:19:27.695 INFO:teuthology.orchestra.run.smithi026.stderr: /var/log/ceph/ceph-client.admin.51829.log 2024-01-15T18:19:27.696 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65771.log 2024-01-15T18:19:27.697 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62786.log.gz 2024-01-15T18:19:27.698 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51829.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-01-15T18:19:27.698 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51829.log.gz 2024-01-15T18:19:27.699 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68351.log 2024-01-15T18:19:27.699 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65771.log.gz 2024-01-15T18:19:27.701 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40686.log 2024-01-15T18:19:27.703 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68351.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68351.log.gz 2024-01-15T18:19:27.704 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68057.log 2024-01-15T18:19:27.704 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40686.log.gz 2024-01-15T18:19:27.705 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78390.log 2024-01-15T18:19:27.707 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68057.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68057.log.gz 2024-01-15T18:19:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51190.log 2024-01-15T18:19:27.708 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78390.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.78390.log.gz 2024-01-15T18:19:27.709 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81410.log 2024-01-15T18:19:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51190.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51190.log.gz 2024-01-15T18:19:27.711 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80016.log 2024-01-15T18:19:27.712 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81410.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81410.log.gz 2024-01-15T18:19:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78126.log 2024-01-15T18:19:27.714 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80016.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80016.log.gz 2024-01-15T18:19:27.716 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78126.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.78126.log.gz 2024-01-15T18:19:27.717 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80738.log 2024-01-15T18:19:27.718 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50743.log 2024-01-15T18:19:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80738.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80738.log.gz 2024-01-15T18:19:27.720 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75449.log 2024-01-15T18:19:27.721 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50743.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50743.log.gz 2024-01-15T18:19:27.722 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73283.log 2024-01-15T18:19:27.724 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75449.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75449.log.gz 2024-01-15T18:19:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64769.log 2024-01-15T18:19:27.725 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73283.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73283.log.gz 2024-01-15T18:19:27.726 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56216.log 2024-01-15T18:19:27.728 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64769.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64769.log.gz 2024-01-15T18:19:27.729 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79493.log/var/log/ceph/ceph-client.admin.56216.log: 2024-01-15T18:19:27.729 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56216.log.gz 2024-01-15T18:19:27.730 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65183.log 2024-01-15T18:19:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79493.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.79493.log.gz 2024-01-15T18:19:27.732 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50444.log 2024-01-15T18:19:27.733 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65183.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65183.log.gz 2024-01-15T18:19:27.734 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63350.log 2024-01-15T18:19:27.736 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50444.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50444.log.gz 2024-01-15T18:19:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69433.log 2024-01-15T18:19:27.737 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63350.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63350.log.gz 2024-01-15T18:19:27.739 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58233.log 2024-01-15T18:19:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69433.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69433.log.gz 2024-01-15T18:19:27.740 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77132.log 2024-01-15T18:19:27.741 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58233.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58233.log.gz 2024-01-15T18:19:27.742 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81200.log 2024-01-15T18:19:27.744 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77132.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77132.log.gz 2024-01-15T18:19:27.745 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65939.log 2024-01-15T18:19:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81200.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81200.log.gz 2024-01-15T18:19:27.746 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70461.log 2024-01-15T18:19:27.748 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65939.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65939.log.gz 2024-01-15T18:19:27.749 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79206.log 2024-01-15T18:19:27.750 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70461.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70461.log.gz 2024-01-15T18:19:27.751 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72351.log 2024-01-15T18:19:27.752 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79206.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79206.log.gz 2024-01-15T18:19:27.753 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61054.log 2024-01-15T18:19:27.754 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72351.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.72351.log.gz 2024-01-15T18:19:27.754 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41876.log 2024-01-15T18:19:27.756 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61054.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.61054.log.gz 2024-01-15T18:19:27.757 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63080.log 2024-01-15T18:19:27.757 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41876.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41876.log.gz 2024-01-15T18:19:27.758 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63038.log 2024-01-15T18:19:27.760 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63080.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63080.log.gz 2024-01-15T18:19:27.761 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66881.log 2024-01-15T18:19:27.761 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63038.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.63038.log.gz 2024-01-15T18:19:27.763 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52259.log 2024-01-15T18:19:27.764 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66881.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66881.log.gz 2024-01-15T18:19:27.765 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44754.log 2024-01-15T18:19:27.766 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52259.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52259.log.gz 2024-01-15T18:19:27.767 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44754.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44754.log.gz 2024-01-15T18:19:27.768 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77292.log 2024-01-15T18:19:27.769 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41868.log 2024-01-15T18:19:27.771 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77292.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77292.log.gz 2024-01-15T18:19:27.772 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41868.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41868.log.gz 2024-01-15T18:19:27.773 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63981.log 2024-01-15T18:19:27.774 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44838.log 2024-01-15T18:19:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63981.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63981.log.gz 2024-01-15T18:19:27.776 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51785.log 2024-01-15T18:19:27.778 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44838.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44838.log.gz 2024-01-15T18:19:27.779 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61294.log 2024-01-15T18:19:27.779 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51785.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51785.log.gz 2024-01-15T18:19:27.780 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61378.log 2024-01-15T18:19:27.782 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61294.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61294.log.gz 2024-01-15T18:19:27.783 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75713.log 2024-01-15T18:19:27.783 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61378.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61378.log.gz 2024-01-15T18:19:27.784 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63164.log 2024-01-15T18:19:27.786 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75713.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75713.log.gz 2024-01-15T18:19:27.787 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78044.log 2024-01-15T18:19:27.787 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63164.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63164.log.gz 2024-01-15T18:19:27.789 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78044.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78044.log.gz 2024-01-15T18:19:27.790 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45227.log 2024-01-15T18:19:27.791 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65351.log 2024-01-15T18:19:27.793 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45227.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45227.log.gz 2024-01-15T18:19:27.794 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50905.log 2024-01-15T18:19:27.795 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65351.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65351.log.gz 2024-01-15T18:19:27.795 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77797.log 2024-01-15T18:19:27.797 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50905.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50905.log.gz 2024-01-15T18:19:27.798 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5/var/log/ceph/ceph-client.admin.77797.log: --verbose -- /var/log/ceph/ceph-client.admin.64209.log 2024-01-15T18:19:27.798 INFO:teuthology.orchestra.run.smithi026.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.77797.log.gz 2024-01-15T18:19:27.799 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61234.log 2024-01-15T18:19:27.801 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64209.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64209.log.gz 2024-01-15T18:19:27.802 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77879.log 2024-01-15T18:19:27.802 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61234.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61234.log.gz 2024-01-15T18:19:27.803 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49419.log 2024-01-15T18:19:27.805 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77879.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.77879.log.gz 2024-01-15T18:19:27.806 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64601.log 2024-01-15T18:19:27.807 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49419.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49419.log.gz 2024-01-15T18:19:27.808 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64601.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64601.log.gz 2024-01-15T18:19:27.809 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58401.log 2024-01-15T18:19:27.811 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66293.log 2024-01-15T18:19:27.812 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58401.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58401.log.gz 2024-01-15T18:19:27.813 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61462.log 2024-01-15T18:19:27.814 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66293.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66293.log.gz 2024-01-15T18:19:27.815 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.61462.log: -- /var/log/ceph/ceph-client.admin.81326.log 2024-01-15T18:19:27.815 INFO:teuthology.orchestra.run.smithi026.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61462.log.gz 2024-01-15T18:19:27.817 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67249.log 2024-01-15T18:19:27.818 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.81326.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.81326.log.gz 2024-01-15T18:19:27.819 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44376.log 2024-01-15T18:19:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67249.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67249.log.gz 2024-01-15T18:19:27.820 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80990.log 2024-01-15T18:19:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44376.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44376.log.gz 2024-01-15T18:19:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79247.log 2024-01-15T18:19:27.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80990.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80990.log.gz 2024-01-15T18:19:27.825 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56132.log 2024-01-15T18:19:27.826 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79247.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.79247.log.gz 2024-01-15T18:19:27.827 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61190.log 2024-01-15T18:19:27.827 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56132.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56132.log.gz 2024-01-15T18:19:27.829 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78085.log 2024-01-15T18:19:27.830 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61190.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61190.log.gz 2024-01-15T18:19:27.831 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.78085.log: /var/log/ceph/ceph-client.admin.55283.log 2024-01-15T18:19:27.832 INFO:teuthology.orchestra.run.smithi026.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.78085.log.gz 2024-01-15T18:19:27.832 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79832.log 2024-01-15T18:19:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55283.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55283.log.gz 2024-01-15T18:19:27.834 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60926.log 2024-01-15T18:19:27.835 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79832.log.gz 2024-01-15T18:19:27.836 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56044.log 2024-01-15T18:19:27.838 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60926.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.60926.log.gz 2024-01-15T18:19:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57701.log 2024-01-15T18:19:27.839 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56044.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56044.log.gz 2024-01-15T18:19:27.840 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73941.log 2024-01-15T18:19:27.842 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57701.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57701.log.gz 2024-01-15T18:19:27.843 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.73941.log: /var/log/ceph/ceph-client.admin.62954.log 2024-01-15T18:19:27.843 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73941.log.gz 2024-01-15T18:19:27.844 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49732.log 2024-01-15T18:19:27.846 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62954.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62954.log.gz 2024-01-15T18:19:27.847 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68265.log 2024-01-15T18:19:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49732.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49732.log.gz 2024-01-15T18:19:27.849 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41201.log 2024-01-15T18:19:27.851 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68265.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68265.log.gz 2024-01-15T18:19:27.851 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55875.log 2024-01-15T18:19:27.852 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41201.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41201.log.gz 2024-01-15T18:19:27.853 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55875.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55875.log.gz 2024-01-15T18:19:27.854 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64107.log 2024-01-15T18:19:27.856 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55619.log 2024-01-15T18:19:27.857 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64107.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64107.log.gz 2024-01-15T18:19:27.858 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53423.log 2024-01-15T18:19:27.859 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55619.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55619.log.gz 2024-01-15T18:19:27.860 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58148.log 2024-01-15T18:19:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53423.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53423.log.gz 2024-01-15T18:19:27.861 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68923.log 2024-01-15T18:19:27.863 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58148.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58148.log.gz 2024-01-15T18:19:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68923.log.gz 2024-01-15T18:19:27.864 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78695.log 2024-01-15T18:19:27.866 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40326.log 2024-01-15T18:19:27.867 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78695.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.78695.log.gz 2024-01-15T18:19:27.868 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62626.log 2024-01-15T18:19:27.870 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40326.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40326.log.gz 2024-01-15T18:19:27.870 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59538.log 2024-01-15T18:19:27.871 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62626.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62626.log.gz 2024-01-15T18:19:27.872 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40896.log 2024-01-15T18:19:27.873 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59538.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59538.log.gz 2024-01-15T18:19:27.874 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63939.log 2024-01-15T18:19:27.875 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40896.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40896.log.gz 2024-01-15T18:19:27.876 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46281.log 2024-01-15T18:19:27.877 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63939.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63939.log.gz 2024-01-15T18:19:27.878 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40644.log 2024-01-15T18:19:27.879 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46281.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46281.log.gz 2024-01-15T18:19:27.880 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51106.log 2024-01-15T18:19:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40644.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40644.log.gz 2024-01-15T18:19:27.881 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50402.log 2024-01-15T18:19:27.883 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51106.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51106.log.gz 2024-01-15T18:19:27.884 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.50402.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.63771.log 2024-01-15T18:19:27.884 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50402.log.gz 2024-01-15T18:19:27.885 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80236.log 2024-01-15T18:19:27.887 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63771.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63771.log.gz 2024-01-15T18:19:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71681.log 2024-01-15T18:19:27.888 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80236.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80236.log.gz 2024-01-15T18:19:27.889 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60375.log 2024-01-15T18:19:27.890 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71681.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.71681.log.gz 2024-01-15T18:19:27.891 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53958.log 2024-01-15T18:19:27.892 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60375.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60375.log.gz 2024-01-15T18:19:27.893 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68731.log 2024-01-15T18:19:27.894 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53958.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53958.log.gz 2024-01-15T18:19:27.895 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51572.log 2024-01-15T18:19:27.896 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68731.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68731.log.gz 2024-01-15T18:19:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51572.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51572.log.gz 2024-01-15T18:19:27.898 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53381.log 2024-01-15T18:19:27.900 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68815.log 2024-01-15T18:19:27.901 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53381.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53381.log.gz 2024-01-15T18:19:27.902 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55794.log 2024-01-15T18:19:27.903 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68815.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68815.log.gz 2024-01-15T18:19:27.904 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70191.log 2024-01-15T18:19:27.905 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55794.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55794.log.gz 2024-01-15T18:19:27.905 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60585.log 2024-01-15T18:19:27.907 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70191.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.70191.log.gz 2024-01-15T18:19:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66107.log 2024-01-15T18:19:27.908 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60585.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60585.log.gz 2024-01-15T18:19:27.909 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79452.log 2024-01-15T18:19:27.910 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66107.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66107.log.gz 2024-01-15T18:19:27.911 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67049.log 2024-01-15T18:19:27.912 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79452.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.79452.log.gz 2024-01-15T18:19:27.913 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80278.log 2024-01-15T18:19:27.914 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67049.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.67049.log.gz 2024-01-15T18:19:27.915 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57315.log 2024-01-15T18:19:27.916 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80278.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80278.log.gz 2024-01-15T18:19:27.917 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57442.log 2024-01-15T18:19:27.918 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57315.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57315.log.gz 2024-01-15T18:19:27.920 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.57442.log: /var/log/ceph/ceph-client.admin.42576.log 2024-01-15T18:19:27.920 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57442.log.gz 2024-01-15T18:19:27.921 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58004.log 2024-01-15T18:19:27.922 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42576.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42576.log.gz 2024-01-15T18:19:27.923 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log/var/log/ceph/ceph-client.admin.58004.log: 2024-01-15T18:19:27.924 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58004.log.gz 2024-01-15T18:19:27.925 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65897.log 2024-01-15T18:19:27.926 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51232.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2024-01-15T18:19:27.927 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44502.log 2024-01-15T18:19:27.928 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65897.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65897.log.gz 2024-01-15T18:19:27.929 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71217.log 2024-01-15T18:19:27.930 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44502.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44502.log.gz 2024-01-15T18:19:27.931 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44124.log 2024-01-15T18:19:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71217.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.71217.log.gz 2024-01-15T18:19:27.932 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46653.log 2024-01-15T18:19:27.934 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44124.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44124.log.gz 2024-01-15T18:19:27.934 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67429.log 2024-01-15T18:19:27.935 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46653.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46653.log.gz 2024-01-15T18:19:27.936 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74838.log 2024-01-15T18:19:27.937 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67429.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67429.log.gz 2024-01-15T18:19:27.938 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41863.log/var/log/ceph/ceph-client.admin.74838.log: 2024-01-15T18:19:27.939 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74838.log.gz 2024-01-15T18:19:27.940 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-15T18:19:27.941 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41863.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41863.log.gz 2024-01-15T18:19:27.942 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70023.log 2024-01-15T18:19:27.944 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.70023.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70023.log.gz 2024-01-15T18:19:27.945 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48559.log 2024-01-15T18:19:27.948 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48727.log 2024-01-15T18:19:27.949 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48559.log.gz 2024-01-15T18:19:27.950 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71544.log 2024-01-15T18:19:27.952 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48727.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48727.log.gz 2024-01-15T18:19:27.953 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60459.log 2024-01-15T18:19:27.954 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71544.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.71544.log.gz 2024-01-15T18:19:27.956 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75799.log 2024-01-15T18:19:27.957 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60459.log.gz 2024-01-15T18:19:27.959 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65855.log 2024-01-15T18:19:27.964 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75799.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75799.log.gz 2024-01-15T18:19:27.964 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44712.log 2024-01-15T18:19:27.964 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65855.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65855.log.gz 2024-01-15T18:19:27.967 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44712.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44712.log.gz 2024-01-15T18:19:27.967 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48517.log 2024-01-15T18:19:27.969 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41875.log 2024-01-15T18:19:27.970 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48517.log.gz 2024-01-15T18:19:27.972 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72477.log 2024-01-15T18:19:27.977 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41875.log: /var/log/ceph/ceph-client.admin.72477.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54132.log 2024-01-15T18:19:27.977 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41875.log.gz 2024-01-15T18:19:27.978 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72477.log.gz 2024-01-15T18:19:27.979 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81368.log 2024-01-15T18:19:27.984 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54132.log: /var/log/ceph/ceph-client.admin.81368.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46097.log 2024-01-15T18:19:27.984 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54132.log.gz 2024-01-15T18:19:27.986 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81368.log.gz 2024-01-15T18:19:27.986 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-01-15T18:19:27.987 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46097.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46097.log.gz 2024-01-15T18:19:27.990 INFO:teuthology.orchestra.run.smithi124.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-15T18:19:27.990 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43972.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-01-15T18:19:27.991 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75621.log 2024-01-15T18:19:27.993 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75024.log 2024-01-15T18:19:27.994 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75621.log.gz 2024-01-15T18:19:27.996 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51491.log 2024-01-15T18:19:28.000 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75024.log: /var/log/ceph/ceph-client.admin.51491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36432.log 2024-01-15T18:19:28.001 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75024.log.gz 2024-01-15T18:19:28.002 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51491.log.gz 2024-01-15T18:19:28.003 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.36432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64335.log 2024-01-15T18:19:28.005 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71133.log 2024-01-15T18:19:28.006 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3%/var/log/ceph/ceph-client.admin.64335.log: -- replaced with /var/log/ceph/ceph-client.admin.36432.log.gz 2024-01-15T18:19:28.008 INFO:teuthology.orchestra.run.smithi026.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.64335.log.gz 2024-01-15T18:19:28.009 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59748.log 2024-01-15T18:19:28.011 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59748.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71133.log.gz 2024-01-15T18:19:28.012 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41846.log 2024-01-15T18:19:28.013 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59748.log.gz 2024-01-15T18:19:28.015 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76436.log 2024-01-15T18:19:28.018 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.76436.log: /var/log/ceph/ceph-client.admin.41846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80530.log 2024-01-15T18:19:28.022 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41846.log.gz 2024-01-15T18:19:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80530.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80530.log.gz 2024-01-15T18:19:28.025 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72011.log 2024-01-15T18:19:28.030 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57878.log 2024-01-15T18:19:28.031 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72011.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72011.log.gz 2024-01-15T18:19:28.034 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67333.log 2024-01-15T18:19:28.036 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57878.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57878.log.gz 2024-01-15T18:19:28.037 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48685.log 2024-01-15T18:19:28.038 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67333.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67333.log.gz 2024-01-15T18:19:28.041 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73839.log/var/log/ceph/ceph-client.admin.48685.log: 2024-01-15T18:19:28.042 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48685.log.gz 2024-01-15T18:19:28.043 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72759.log 2024-01-15T18:19:28.045 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73839.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73839.log.gz 2024-01-15T18:19:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72759.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72759.log.gz 2024-01-15T18:19:28.049 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76093.log 2024-01-15T18:19:28.053 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44544.log 2024-01-15T18:19:28.054 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76093.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76093.log.gz 2024-01-15T18:19:28.057 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54451.log 2024-01-15T18:19:28.059 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44544.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44544.log.gz 2024-01-15T18:19:28.060 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69391.log 2024-01-15T18:19:28.061 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54451.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54451.log.gz 2024-01-15T18:19:28.064 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77838.log 2024-01-15T18:19:28.065 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69391.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69391.log.gz 2024-01-15T18:19:28.069 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72267.log 2024-01-15T18:19:28.069 INFO:teuthology.orchestra.run.smithi026.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77838.log.gz 2024-01-15T18:19:28.071 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78431.log 2024-01-15T18:19:28.073 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72267.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72267.log.gz 2024-01-15T18:19:28.075 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78431.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78431.log.gz 2024-01-15T18:19:28.077 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45616.log 2024-01-15T18:19:28.079 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76911.log 2024-01-15T18:19:28.081 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45616.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45616.log.gz 2024-01-15T18:19:28.084 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41873.log 2024-01-15T18:19:28.086 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76911.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76911.log.gz 2024-01-15T18:19:28.090 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48181.log 2024-01-15T18:19:28.091 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41873.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41873.log.gz 2024-01-15T18:19:28.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41534.log 2024-01-15T18:19:28.095 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48181.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48181.log.gz 2024-01-15T18:19:28.096 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77920.log 2024-01-15T18:19:28.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41534.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41534.log.gz 2024-01-15T18:19:28.100 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50528.log 2024-01-15T18:19:28.101 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77920.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.77920.log.gz 2024-01-15T18:19:28.105 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59352.log 2024-01-15T18:19:28.108 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50528.log.gz 2024-01-15T18:19:28.110 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76743.log 2024-01-15T18:19:28.111 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59352.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59352.log.gz 2024-01-15T18:19:28.112 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77961.log 2024-01-15T18:19:28.114 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76743.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76743.log.gz 2024-01-15T18:19:28.116 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53095.log 2024-01-15T18:19:28.116 INFO:teuthology.orchestra.run.smithi124.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-15T18:19:28.117 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77961.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.77961.log.gz 2024-01-15T18:19:28.120 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53095.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53095.log.gz 2024-01-15T18:19:28.121 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54787.log 2024-01-15T18:19:28.124 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71055.log 2024-01-15T18:19:28.125 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54787.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54787.log.gz 2024-01-15T18:19:28.129 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46527.log 2024-01-15T18:19:28.130 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71055.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71055.log.gz 2024-01-15T18:19:28.132 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71013.log 2024-01-15T18:19:28.133 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46527.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46527.log.gz 2024-01-15T18:19:28.138 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40464.log 2024-01-15T18:19:28.140 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45101.log 2024-01-15T18:19:28.141 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71013.log.gz 2024-01-15T18:19:28.142 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40464.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40464.log.gz 2024-01-15T18:19:28.146 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79916.log 2024-01-15T18:19:28.147 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45101.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45101.log.gz 2024-01-15T18:19:28.148 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47367.log 2024-01-15T18:19:28.152 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40854.log 2024-01-15T18:19:28.155 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79916.log.gz 2024-01-15T18:19:28.156 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47367.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47367.log.gz 2024-01-15T18:19:28.156 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45728.log 2024-01-15T18:19:28.157 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40854.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40854.log.gz 2024-01-15T18:19:28.158 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44250.log 2024-01-15T18:19:28.160 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45728.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45728.log.gz 2024-01-15T18:19:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60842.log 2024-01-15T18:19:28.162 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44250.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44250.log.gz 2024-01-15T18:19:28.165 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68773.log 2024-01-15T18:19:28.167 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60842.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60842.log.gz 2024-01-15T18:19:28.169 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68773.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68773.log.gz 2024-01-15T18:19:28.170 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72630.log 2024-01-15T18:19:28.175 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44418.log 2024-01-15T18:19:28.176 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72630.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72630.log.gz 2024-01-15T18:19:28.177 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64417.log 2024-01-15T18:19:28.179 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64417.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.64417.log.gz 2024-01-15T18:19:28.181 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44418.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44418.log.gz 2024-01-15T18:19:28.182 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62406.log 2024-01-15T18:19:28.186 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64541.log 2024-01-15T18:19:28.188 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72225.log 2024-01-15T18:19:28.189 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64541.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64541.log.gz 2024-01-15T18:19:28.189 INFO:teuthology.orchestra.run.smithi026.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62406.log.gz 2024-01-15T18:19:28.193 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75108.log 2024-01-15T18:19:28.194 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72225.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.72225.log.gz 2024-01-15T18:19:28.199 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78554.log 2024-01-15T18:19:28.199 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75108.log.gz 2024-01-15T18:19:28.202 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51921.log 2024-01-15T18:19:28.204 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78554.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78554.log.gz 2024-01-15T18:19:28.206 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46407.log 2024-01-15T18:19:28.207 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51921.log.gz 2024-01-15T18:19:28.209 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60090.log 2024-01-15T18:19:28.213 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46407.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46407.log.gz 2024-01-15T18:19:28.215 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54577.log 2024-01-15T18:19:28.215 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60090.log.gz 2024-01-15T18:19:28.219 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78226.log 2024-01-15T18:19:28.221 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54577.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54577.log.gz 2024-01-15T18:19:28.221 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72141.log 2024-01-15T18:19:28.223 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78226.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.78226.log.gz 2024-01-15T18:19:28.227 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56594.log 2024-01-15T18:19:28.227 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72141.log.gz 2024-01-15T18:19:28.230 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75757.log 2024-01-15T18:19:28.232 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56594.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56594.log.gz 2024-01-15T18:19:28.236 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75757.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75757.log.gz 2024-01-15T18:19:28.237 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59096.log 2024-01-15T18:19:28.241 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59622.log 2024-01-15T18:19:28.242 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59096.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.59096.log.gz 2024-01-15T18:19:28.246 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60417.log 2024-01-15T18:19:28.247 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59622.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59622.log.gz 2024-01-15T18:19:28.249 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41536.log 2024-01-15T18:19:28.250 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60417.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60417.log.gz 2024-01-15T18:19:28.254 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53744.log 2024-01-15T18:19:28.255 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41536.log.gz 2024-01-15T18:19:28.258 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61672.log 2024-01-15T18:19:28.259 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53744.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53744.log.gz 2024-01-15T18:19:28.261 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44922.log 2024-01-15T18:19:28.262 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61672.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61672.log.gz 2024-01-15T18:19:28.265 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77715.log 2024-01-15T18:19:28.266 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44922.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44922.log.gz 2024-01-15T18:19:28.268 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58443.log 2024-01-15T18:19:28.269 INFO:teuthology.orchestra.run.smithi026.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.77715.log.gz 2024-01-15T18:19:28.273 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45495.log 2024-01-15T18:19:28.274 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58443.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58443.log.gz 2024-01-15T18:19:28.276 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66713.log 2024-01-15T18:19:28.276 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45495.log.gz 2024-01-15T18:19:28.281 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70713.log 2024-01-15T18:19:28.282 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66713.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66713.log.gz 2024-01-15T18:19:28.284 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40812.log 2024-01-15T18:19:28.284 INFO:teuthology.orchestra.run.smithi026.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70713.log.gz 2024-01-15T18:19:28.290 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40812.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40812.log.gz 2024-01-15T18:19:28.291 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-01-15T18:19:28.296 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63393.log 2024-01-15T18:19:28.300 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77334.log 2024-01-15T18:19:28.301 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63393.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63393.log.gz 2024-01-15T18:19:28.303 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70065.log 2024-01-15T18:19:28.304 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77334.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77334.log.gz 2024-01-15T18:19:28.305 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60501.log 2024-01-15T18:19:28.306 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70065.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.70065.log.gz 2024-01-15T18:19:28.309 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46989.log 2024-01-15T18:19:28.309 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60501.log.gz 2024-01-15T18:19:28.313 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66965.log 2024-01-15T18:19:28.315 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46989.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46989.log.gz 2024-01-15T18:19:28.318 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66965.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66965.log.gz 2024-01-15T18:19:28.320 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80362.log 2024-01-15T18:19:28.324 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80906.log 2024-01-15T18:19:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80362.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80362.log.gz 2024-01-15T18:19:28.325 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66587.log 2024-01-15T18:19:28.326 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80906.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80906.log.gz 2024-01-15T18:19:28.328 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76617.log 2024-01-15T18:19:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66587.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66587.log.gz 2024-01-15T18:19:28.330 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54745.log 2024-01-15T18:19:28.332 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76617.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76617.log.gz 2024-01-15T18:19:28.334 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47696.log 2024-01-15T18:19:28.336 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54745.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54745.log.gz 2024-01-15T18:19:28.336 INFO:teuthology.orchestra.run.smithi026.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-01-15T18:19:28.340 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65981.log 2024-01-15T18:19:28.340 INFO:teuthology.orchestra.run.smithi026.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47696.log.gz 2024-01-15T18:19:28.344 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55409.log 2024-01-15T18:19:28.346 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65981.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65981.log.gz 2024-01-15T18:19:28.347 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66065.log 2024-01-15T18:19:28.348 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55409.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55409.log.gz 2024-01-15T18:19:28.352 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66065.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66065.log.gz 2024-01-15T18:19:28.353 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68395.log 2024-01-15T18:19:28.357 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64023.log 2024-01-15T18:19:28.358 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68395.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68395.log.gz 2024-01-15T18:19:28.361 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44880.log 2024-01-15T18:19:28.362 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64023.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64023.log.gz 2024-01-15T18:19:28.363 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55451.log 2024-01-15T18:19:28.368 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55451.log: /var/log/ceph/ceph-client.admin.44880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58485.log 2024-01-15T18:19:28.368 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55451.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44880.log.gz 2024-01-15T18:19:28.368 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-01-15T18:19:28.373 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42114.log 2024-01-15T18:19:28.374 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58485.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58485.log.gz 2024-01-15T18:19:28.376 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72825.log 2024-01-15T18:19:28.376 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42114.log.gz 2024-01-15T18:19:28.379 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80058.log 2024-01-15T18:19:28.380 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72825.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72825.log.gz 2024-01-15T18:19:28.383 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80058.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80058.log.gz 2024-01-15T18:19:28.384 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73535.log 2024-01-15T18:19:28.387 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61010.log 2024-01-15T18:19:28.390 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73535.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73535.log.gz 2024-01-15T18:19:28.391 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63897.log 2024-01-15T18:19:28.393 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61010.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61010.log.gz 2024-01-15T18:19:28.394 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54006.log 2024-01-15T18:19:28.396 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63897.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63897.log.gz 2024-01-15T18:19:28.398 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49154.log 2024-01-15T18:19:28.399 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54006.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54006.log.gz 2024-01-15T18:19:28.404 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.0.49154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52133.log 2024-01-15T18:19:28.408 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75066.log 2024-01-15T18:19:28.409 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52133.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52133.log.gz 2024-01-15T18:19:28.413 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60249.log 2024-01-15T18:19:28.414 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75066.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.75066.log.gz 2024-01-15T18:19:28.417 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57743.log 2024-01-15T18:19:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60249.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60249.log.gz 2024-01-15T18:19:28.418 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63519.log 2024-01-15T18:19:28.420 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57743.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57743.log.gz 2024-01-15T18:19:28.422 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59496.log 2024-01-15T18:19:28.424 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63519.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63519.log.gz 2024-01-15T18:19:28.426 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59496.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59496.log.gz 2024-01-15T18:19:28.428 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50147.log 2024-01-15T18:19:28.430 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69981.log 2024-01-15T18:19:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50147.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50147.log.gz 2024-01-15T18:19:28.432 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54367.log 2024-01-15T18:19:28.434 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69981.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.69981.log.gz 2024-01-15T18:19:28.435 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58630.log 2024-01-15T18:19:28.436 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54367.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54367.log.gz 2024-01-15T18:19:28.438 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44670.log 2024-01-15T18:19:28.440 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58630.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58630.log.gz 2024-01-15T18:19:28.442 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66335.log 2024-01-15T18:19:28.444 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44670.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44670.log.gz 2024-01-15T18:19:28.446 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66335.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66335.log.gz 2024-01-15T18:19:28.448 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59010.log 2024-01-15T18:19:28.450 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77674.log 2024-01-15T18:19:28.453 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59010.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59010.log.gz 2024-01-15T18:19:28.454 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71586.log 2024-01-15T18:19:28.455 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77674.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.77674.log.gz 2024-01-15T18:19:28.458 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70377.log 2024-01-15T18:19:28.459 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71586.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.71586.log.gz 2024-01-15T18:19:28.460 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57244.log 2024-01-15T18:19:28.461 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70377.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.70377.log.gz 2024-01-15T18:19:28.464 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65687.log 2024-01-15T18:19:28.465 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57244.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57244.log.gz 2024-01-15T18:19:28.467 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65687.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65687.log.gz 2024-01-15T18:19:28.468 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78736.log 2024-01-15T18:19:28.470 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79023.log 2024-01-15T18:19:28.471 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78736.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.78736.log.gz 2024-01-15T18:19:28.472 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46611.log 2024-01-15T18:19:28.476 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47823.log 2024-01-15T18:19:28.477 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46611.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79023.log.gz 2024-01-15T18:19:28.477 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46611.log.gz 2024-01-15T18:19:28.481 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46947.log 2024-01-15T18:19:28.482 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47823.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47823.log.gz 2024-01-15T18:19:28.485 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69601.log 2024-01-15T18:19:28.485 INFO:teuthology.orchestra.run.smithi026.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46947.log.gz 2024-01-15T18:19:28.490 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65099.log 2024-01-15T18:19:28.491 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69601.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69601.log.gz 2024-01-15T18:19:28.495 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63561.log 2024-01-15T18:19:28.497 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66503.log 2024-01-15T18:19:28.497 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65099.log.gz 2024-01-15T18:19:28.502 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63561.log: /var/log/ceph/ceph-client.admin.66503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57400.log 2024-01-15T18:19:28.502 INFO:teuthology.orchestra.run.smithi026.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66503.log.gz 2024-01-15T18:19:28.504 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63561.log.gz 2024-01-15T18:19:28.504 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67882.log 2024-01-15T18:19:28.507 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41517.log 2024-01-15T18:19:28.508 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57400.log: /var/log/ceph/ceph-client.admin.67882.log: 88.9% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57400.log.gz 2024-01-15T18:19:28.509 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67882.log.gz 2024-01-15T18:19:28.512 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45327.log 2024-01-15T18:19:28.513 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41517.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41517.log.gz 2024-01-15T18:19:28.515 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-15T18:19:28.517 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45327.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45327.log.gz 2024-01-15T18:19:28.520 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78595.log 2024-01-15T18:19:28.527 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.78595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55028.log 2024-01-15T18:19:28.528 INFO:teuthology.orchestra.run.smithi026.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.78595.log.gz 2024-01-15T18:19:28.528 INFO:teuthology.orchestra.run.smithi026.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.69208.log.gz 2024-01-15T18:19:28.530 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73983.log 2024-01-15T18:19:28.531 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55028.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55028.log.gz 2024-01-15T18:19:28.533 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62280.log 2024-01-15T18:19:28.537 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58714.log 2024-01-15T18:19:28.537 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73983.log.gz 2024-01-15T18:19:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.62280.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57785.log 2024-01-15T18:19:28.541 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62280.log.gz 2024-01-15T18:19:28.541 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58714.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58714.log.gz 2024-01-15T18:19:28.547 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57785.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57785.log.gz 2024-01-15T18:19:28.548 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41530.log 2024-01-15T18:19:28.550 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61546.log 2024-01-15T18:19:28.551 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41530.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41530.log.gz 2024-01-15T18:19:28.555 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66149.log 2024-01-15T18:19:28.557 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61546.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61546.log.gz 2024-01-15T18:19:28.558 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77756.log 2024-01-15T18:19:28.559 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66149.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.66149.log.gz 2024-01-15T18:19:28.562 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48643.log 2024-01-15T18:19:28.567 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77756.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.77756.log.gz 2024-01-15T18:19:28.568 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74210.log 2024-01-15T18:19:28.569 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.48643.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48643.log.gz 2024-01-15T18:19:28.572 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74210.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74210.log.gz 2024-01-15T18:19:28.575 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66251.log 2024-01-15T18:19:28.576 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73241.log 2024-01-15T18:19:28.578 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66251.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66251.log.gz 2024-01-15T18:19:28.582 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41533.log 2024-01-15T18:19:28.584 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73241.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73241.log.gz 2024-01-15T18:19:28.585 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63122.log 2024-01-15T18:19:28.590 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41533.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41533.log.gz 2024-01-15T18:19:28.591 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74754.log 2024-01-15T18:19:28.593 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63122.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63122.log.gz 2024-01-15T18:19:28.595 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51411.log 2024-01-15T18:19:28.597 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74754.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74754.log.gz 2024-01-15T18:19:28.598 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79147.log 2024-01-15T18:19:28.601 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51411.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51411.log.gz 2024-01-15T18:19:28.601 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.79147.log: /var/log/ceph/ceph-client.admin.42282.log 2024-01-15T18:19:28.602 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79147.log.gz 2024-01-15T18:19:28.605 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42240.log 2024-01-15T18:19:28.606 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42282.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42282.log.gz 2024-01-15T18:19:28.609 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44628.log 2024-01-15T18:19:28.612 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42240.log.gz 2024-01-15T18:19:28.613 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60333.log 2024-01-15T18:19:28.614 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44628.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44628.log.gz 2024-01-15T18:19:28.617 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74475.log 2024-01-15T18:19:28.619 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60333.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60333.log.gz 2024-01-15T18:19:28.620 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60680.log 2024-01-15T18:19:28.622 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74475.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74475.log.gz 2024-01-15T18:19:28.625 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49850.log 2024-01-15T18:19:28.625 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60680.log.gz 2024-01-15T18:19:28.626 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50486.log 2024-01-15T18:19:28.631 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50486.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50486.log.gz 2024-01-15T18:19:28.632 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47654.log 2024-01-15T18:19:28.632 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49850.log.gz 2024-01-15T18:19:28.633 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80572.log 2024-01-15T18:19:28.635 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47654.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47654.log.gz 2024-01-15T18:19:28.636 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55208.log 2024-01-15T18:19:28.637 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80572.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80572.log.gz 2024-01-15T18:19:28.638 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64293.log 2024-01-15T18:19:28.640 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55208.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55208.log.gz 2024-01-15T18:19:28.642 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53053.log 2024-01-15T18:19:28.644 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64293.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64293.log.gz 2024-01-15T18:19:28.646 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53053.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53053.log.gz 2024-01-15T18:19:28.648 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68307.log 2024-01-15T18:19:28.650 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51614.log 2024-01-15T18:19:28.652 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68307.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68307.log.gz 2024-01-15T18:19:28.653 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-15T18:19:28.654 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51614.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51614.log.gz 2024-01-15T18:19:28.655 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53702.log 2024-01-15T18:19:28.657 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46821.log 2024-01-15T18:19:28.659 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53702.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53702.log.gz 2024-01-15T18:19:28.660 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63266.log 2024-01-15T18:19:28.662 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46821.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46821.log.gz 2024-01-15T18:19:28.664 INFO:teuthology.orchestra.run.smithi026.stderr: 86.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-15T18:19:28.668 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57484.log 2024-01-15T18:19:28.668 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63266.log.gz 2024-01-15T18:19:28.670 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69517.log 2024-01-15T18:19:28.671 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.57484.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57484.log.gz 2024-01-15T18:19:28.672 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56678.log 2024-01-15T18:19:28.673 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.69517.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69517.log.gz 2024-01-15T18:19:28.677 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71969.log 2024-01-15T18:19:28.677 INFO:teuthology.orchestra.run.smithi026.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56678.log.gz 2024-01-15T18:19:28.678 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-15T18:19:28.680 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71969.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.71969.log.gz 2024-01-15T18:19:28.684 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42618.log 2024-01-15T18:19:28.686 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68479.log 2024-01-15T18:19:28.688 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42618.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42618.log.gz 2024-01-15T18:19:28.690 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43410.log 2024-01-15T18:19:28.692 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68479.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68479.log.gz 2024-01-15T18:19:28.694 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41203.log 2024-01-15T18:19:28.696 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.43410.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43410.log.gz 2024-01-15T18:19:28.700 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41203.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41203.log.gz 2024-01-15T18:19:28.701 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74637.log 2024-01-15T18:19:28.706 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-01-15T18:19:28.707 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74637.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.74637.log.gz 2024-01-15T18:19:28.709 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71781.log 2024-01-15T18:19:28.713 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49461.log 2024-01-15T18:19:28.715 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71781.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71781.log.gz 2024-01-15T18:19:28.718 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49461.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49461.log.gz 2024-01-15T18:19:28.720 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72519.log 2024-01-15T18:19:28.721 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50190.log 2024-01-15T18:19:28.723 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73577.log 2024-01-15T18:19:28.725 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50190.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50190.log.gz 2024-01-15T18:19:28.727 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72519.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.72519.log.gz 2024-01-15T18:19:28.728 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76659.log 2024-01-15T18:19:28.729 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73577.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73577.log.gz 2024-01-15T18:19:28.729 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67672.log 2024-01-15T18:19:28.731 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76659.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.76659.log.gz 2024-01-15T18:19:28.733 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56762.log 2024-01-15T18:19:28.735 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67672.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67672.log.gz 2024-01-15T18:19:28.739 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66191.log 2024-01-15T18:19:28.739 INFO:teuthology.orchestra.run.smithi026.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56762.log.gz 2024-01-15T18:19:28.741 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62099.log 2024-01-15T18:19:28.745 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66191.log: /var/log/ceph/ceph-client.0.62099.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66191.log.gz 2024-01-15T18:19:28.747 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64149.log 2024-01-15T18:19:28.748 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74294.log 2024-01-15T18:19:28.753 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64149.log: /var/log/ceph/ceph-client.admin.74294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44586.log 2024-01-15T18:19:28.753 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64149.log.gz 2024-01-15T18:19:28.758 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4%/var/log/ceph/ceph-client.admin.44586.log: -- replaced with /var/log/ceph/ceph-client.admin.74294.log.gz 2024-01-15T18:19:28.758 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52487.log 2024-01-15T18:19:28.759 INFO:teuthology.orchestra.run.smithi026.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44586.log.gz 2024-01-15T18:19:28.763 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47115.log 2024-01-15T18:19:28.764 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52487.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52487.log.gz 2024-01-15T18:19:28.767 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54238.log 2024-01-15T18:19:28.768 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47115.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47115.log.gz 2024-01-15T18:19:28.769 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72393.log 2024-01-15T18:19:28.771 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54238.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54238.log.gz 2024-01-15T18:19:28.774 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72393.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.72393.log.gz 2024-01-15T18:19:28.775 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47865.log 2024-01-15T18:19:28.778 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72183.log 2024-01-15T18:19:28.779 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47865.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47865.log.gz 2024-01-15T18:19:28.781 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.72183.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72183.log.gz 2024-01-15T18:19:28.782 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66671.log 2024-01-15T18:19:28.787 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66671.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66671.log.gz 2024-01-15T18:19:28.789 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46863.log 2024-01-15T18:19:28.791 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63813.log 2024-01-15T18:19:28.793 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49934.log 2024-01-15T18:19:28.793 INFO:teuthology.orchestra.run.smithi026.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46863.log.gz 2024-01-15T18:19:28.795 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63813.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63813.log.gz 2024-01-15T18:19:28.795 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47073.log 2024-01-15T18:19:28.796 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49934.log.gz 2024-01-15T18:19:28.798 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44292.log 2024-01-15T18:19:28.799 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47073.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47073.log.gz 2024-01-15T18:19:28.803 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.44292.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44292.log.gz 2024-01-15T18:19:28.804 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70587.log 2024-01-15T18:19:28.806 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70587.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.70587.log.gz 2024-01-15T18:19:28.810 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79065.log 2024-01-15T18:19:28.811 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75925.log 2024-01-15T18:19:28.815 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58275.log 2024-01-15T18:19:28.816 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77250.log 2024-01-15T18:19:28.816 INFO:teuthology.orchestra.run.smithi026.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75925.log.gz 2024-01-15T18:19:28.817 INFO:teuthology.orchestra.run.smithi026.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79065.log.gz 2024-01-15T18:19:28.819 INFO:teuthology.orchestra.run.smithi124.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-01-15T18:19:28.821 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58275.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58275.log.gz 2024-01-15T18:19:28.822 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66923.log 2024-01-15T18:19:28.823 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77250.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77250.log.gz 2024-01-15T18:19:28.824 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40271.log 2024-01-15T18:19:28.825 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.66923.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66923.log.gz 2024-01-15T18:19:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40271.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40271.log.gz 2024-01-15T18:19:28.829 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59580.log 2024-01-15T18:19:28.832 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59580.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59580.log.gz 2024-01-15T18:19:28.836 INFO:teuthology.orchestra.run.smithi026.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-15T18:19:28.836 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67007.log 2024-01-15T18:19:28.837 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53660.log 2024-01-15T18:19:28.839 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67007.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67007.log.gz 2024-01-15T18:19:28.840 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60012.log 2024-01-15T18:19:28.842 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.53660.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53660.log.gz 2024-01-15T18:19:28.843 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42156.log 2024-01-15T18:19:28.845 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.60012.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60012.log.gz 2024-01-15T18:19:28.848 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40560.log 2024-01-15T18:19:28.850 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42156.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42156.log.gz 2024-01-15T18:19:28.852 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71843.log 2024-01-15T18:19:28.854 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40560.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40560.log.gz 2024-01-15T18:19:28.858 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.71843.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71843.log.gz 2024-01-15T18:19:28.860 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80780.log 2024-01-15T18:19:28.862 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76827.log 2024-01-15T18:19:28.864 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80780.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80780.log.gz 2024-01-15T18:19:28.868 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47031.log 2024-01-15T18:19:28.869 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.76827.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.76827.log.gz 2024-01-15T18:19:28.872 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45269.log 2024-01-15T18:19:28.873 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.47031.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47031.log.gz 2024-01-15T18:19:28.876 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78308.log 2024-01-15T18:19:28.877 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45269.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45269.log.gz 2024-01-15T18:19:28.880 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73737.log 2024-01-15T18:19:28.881 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78308.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78308.log.gz 2024-01-15T18:19:28.885 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58527.log 2024-01-15T18:19:28.885 INFO:teuthology.orchestra.run.smithi026.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73737.log.gz 2024-01-15T18:19:28.888 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42660.log 2024-01-15T18:19:28.889 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58527.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58527.log.gz 2024-01-15T18:19:28.893 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42660.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42660.log.gz 2024-01-15T18:19:28.894 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68437.log 2024-01-15T18:19:28.896 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52361.log 2024-01-15T18:19:28.898 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75883.log 2024-01-15T18:19:28.900 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52361.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52361.log.gz 2024-01-15T18:19:28.901 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.68437.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68437.log.gz 2024-01-15T18:19:28.901 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79874.log 2024-01-15T18:19:28.903 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75883.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75883.log.gz 2024-01-15T18:19:28.906 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79288.log 2024-01-15T18:19:28.908 INFO:teuthology.orchestra.run.smithi026.stderr: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53179.log 2024-01-15T18:19:28.908 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79874.log.gz 2024-01-15T18:19:28.912 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79288.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.79288.log.gz/var/log/ceph/ceph-client.admin.53179.log: 2024-01-15T18:19:28.914 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53179.log.gz 2024-01-15T18:19:28.914 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59664.log 2024-01-15T18:19:28.920 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50863.log 2024-01-15T18:19:28.921 INFO:teuthology.orchestra.run.smithi026.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59664.log.gz 2024-01-15T18:19:28.923 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64458.log 2024-01-15T18:19:28.924 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50863.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50863.log.gz 2024-01-15T18:19:28.927 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73325.log 2024-01-15T18:19:28.928 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64458.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64458.log.gz 2024-01-15T18:19:28.931 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40980.log 2024-01-15T18:19:28.932 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73325.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73325.log.gz 2024-01-15T18:19:28.935 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74880.log 2024-01-15T18:19:28.936 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40980.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40980.log.gz 2024-01-15T18:19:28.940 INFO:teuthology.orchestra.run.smithi026.stderr:gzip/var/log/ceph/ceph-client.admin.74880.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64913.log 2024-01-15T18:19:28.943 INFO:teuthology.orchestra.run.smithi026.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46365.log 2024-01-15T18:19:28.943 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74880.log.gz 2024-01-15T18:19:28.945 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.64913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70293.log 2024-01-15T18:19:28.946 INFO:teuthology.orchestra.run.smithi026.stderr: 88.4%/var/log/ceph/ceph-client.admin.46365.log: 96.2% -- replaced with /var/log/ceph/ceph-client.0.73102.log.gz 2024-01-15T18:19:28.948 INFO:teuthology.orchestra.run.smithi026.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64913.log.gz 2024-01-15T18:19:28.949 INFO:teuthology.orchestra.run.smithi026.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46365.log.gz 2024-01-15T18:19:28.950 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.70293.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.70293.log.gz 2024-01-15T18:19:28.952 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40728.log 2024-01-15T18:19:28.957 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45411.log 2024-01-15T18:19:28.958 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.40728.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40728.log.gz 2024-01-15T18:19:28.962 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58359.log 2024-01-15T18:19:28.963 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45411.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45411.log.gz 2024-01-15T18:19:28.966 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51879.log 2024-01-15T18:19:28.967 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58359.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58359.log.gz 2024-01-15T18:19:28.970 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79575.log 2024-01-15T18:19:28.971 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51879.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51879.log.gz 2024-01-15T18:19:28.973 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74126.log 2024-01-15T18:19:28.975 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79575.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79575.log.gz 2024-01-15T18:19:28.979 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.74126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45897.log 2024-01-15T18:19:28.981 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74126.log.gz 2024-01-15T18:19:28.982 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59928.log 2024-01-15T18:19:28.984 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.45897.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45897.log.gz 2024-01-15T18:19:28.986 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61630.log 2024-01-15T18:19:28.989 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59928.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79657.log 2024-01-15T18:19:28.990 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61630.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61630.log.gz 2024-01-15T18:19:28.991 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.59928.log.gz 2024-01-15T18:19:28.991 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47283.log 2024-01-15T18:19:28.997 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.79657.log: gzip 87.5%/var/log/ceph/ceph-client.admin.47283.log: 88.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.49690.log 2024-01-15T18:19:28.999 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.79657.log.gz 2024-01-15T18:19:28.999 INFO:teuthology.orchestra.run.smithi026.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56300.log 2024-01-15T18:19:29.000 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.49690.log: -- replaced with /var/log/ceph/ceph-client.admin.47283.log.gz 2024-01-15T18:19:29.000 INFO:teuthology.orchestra.run.smithi026.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49690.log.gz 2024-01-15T18:19:29.003 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59310.log 2024-01-15T18:19:29.007 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.56300.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56300.log.gz 2024-01-15T18:19:29.008 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52553.log 2024-01-15T18:19:29.009 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.59310.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59310.log.gz 2024-01-15T18:19:29.013 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.52553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65267.log 2024-01-15T18:19:29.013 INFO:teuthology.orchestra.run.smithi026.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52553.log.gz 2024-01-15T18:19:29.016 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49976.log 2024-01-15T18:19:29.021 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54944.log 2024-01-15T18:19:29.021 INFO:teuthology.orchestra.run.smithi026.stderr: 89.3%/var/log/ceph/ceph-client.admin.49976.log: -- replaced with /var/log/ceph/ceph-client.admin.65267.log.gz 2024-01-15T18:19:29.021 INFO:teuthology.orchestra.run.smithi026.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49976.log.gz 2024-01-15T18:19:29.025 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-01-15T18:19:29.026 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54944.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54944.log.gz 2024-01-15T18:19:29.027 INFO:teuthology.orchestra.run.smithi026.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.52830.log.gz 2024-01-15T18:19:29.032 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78777.log 2024-01-15T18:19:29.037 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43823.log 2024-01-15T18:19:29.038 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78777.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.78777.log.gz 2024-01-15T18:19:29.040 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55493.log 2024-01-15T18:19:29.043 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.2.43823.log: 91.8% -- replaced with /var/log/ceph/ceph-client.2.43823.log.gz 2024-01-15T18:19:29.044 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54493.log 2024-01-15T18:19:29.046 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.55493.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55493.log.gz 2024-01-15T18:19:29.048 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77208.log 2024-01-15T18:19:29.048 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54493.log.gz 2024-01-15T18:19:29.052 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54282.log 2024-01-15T18:19:29.053 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.77208.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77208.log.gz 2024-01-15T18:19:29.058 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.54282.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54282.log.gz 2024-01-15T18:19:29.059 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73779.log 2024-01-15T18:19:29.061 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73367.log 2024-01-15T18:19:29.064 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67840.log 2024-01-15T18:19:29.066 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.73367.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73367.log.gz 2024-01-15T18:19:29.067 INFO:teuthology.orchestra.run.smithi026.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73779.log.gz 2024-01-15T18:19:29.068 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58924.log 2024-01-15T18:19:29.069 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.67840.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67840.log.gz 2024-01-15T18:19:29.070 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80404.log 2024-01-15T18:19:29.071 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58924.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58924.log.gz 2024-01-15T18:19:29.076 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.80404.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80404.log.gz 2024-01-15T18:19:29.079 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51022.log 2024-01-15T18:19:29.082 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42702.log 2024-01-15T18:19:29.083 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.51022.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51022.log.gz 2024-01-15T18:19:29.084 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58798.log 2024-01-15T18:19:29.088 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.42702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41540.log 2024-01-15T18:19:29.089 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.58798.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58798.log.gz 2024-01-15T18:19:29.090 INFO:teuthology.orchestra.run.smithi026.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42702.log.gz 2024-01-15T18:19:29.090 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78900.log 2024-01-15T18:19:29.092 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.41540.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41540.log.gz 2024-01-15T18:19:29.094 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50360.log 2024-01-15T18:19:29.096 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.78900.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.78900.log.gz 2024-01-15T18:19:29.097 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63224.log 2024-01-15T18:19:29.098 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.50360.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50360.log.gz 2024-01-15T18:19:29.100 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75283.log 2024-01-15T18:19:29.102 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.63224.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63224.log.gz 2024-01-15T18:19:29.105 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.75283.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75283.log.gz 2024-01-15T18:19:29.107 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61756.log 2024-01-15T18:19:29.111 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62364.log 2024-01-15T18:19:29.114 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.61756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46737.log 2024-01-15T18:19:29.114 INFO:teuthology.orchestra.run.smithi026.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61756.log.gz 2024-01-15T18:19:29.117 INFO:teuthology.orchestra.run.smithi026.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65039.log 2024-01-15T18:19:29.119 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.46737.log: /var/log/ceph/ceph-client.admin.62364.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62364.log.gz 2024-01-15T18:19:29.120 INFO:teuthology.orchestra.run.smithi026.stderr:/var/log/ceph/ceph-client.admin.65039.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65039.log.gz 2024-01-15T18:19:29.121 INFO:teuthology.orchestra.run.smithi026.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46737.log.gz 2024-01-15T18:19:29.138 INFO:teuthology.orchestra.run.smithi026.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.57105.log.gz 2024-01-15T18:19:29.435 INFO:teuthology.orchestra.run.smithi026.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.76436.log.gz 2024-01-15T18:19:29.688 INFO:teuthology.orchestra.run.smithi026.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-01-15T18:19:29.908 INFO:teuthology.orchestra.run.smithi124.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-01-15T18:19:30.112 INFO:teuthology.orchestra.run.smithi026.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.49154.log.gz 2024-01-15T18:19:30.157 INFO:teuthology.orchestra.run.smithi124.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-15T18:19:30.191 INFO:teuthology.orchestra.run.smithi026.stderr: 96.2% -- replaced with /var/log/ceph/ceph-client.0.62099.log.gz 2024-01-15T18:19:30.211 INFO:teuthology.orchestra.run.smithi124.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-01-15T18:19:30.516 INFO:teuthology.orchestra.run.smithi124.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-15T18:19:30.713 INFO:teuthology.orchestra.run.smithi124.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-15T18:19:30.717 INFO:teuthology.orchestra.run.smithi124.stderr: 2024-01-15T18:19:30.717 INFO:teuthology.orchestra.run.smithi124.stderr:real 0m3.567s 2024-01-15T18:19:30.718 INFO:teuthology.orchestra.run.smithi124.stderr:user 0m17.429s 2024-01-15T18:19:30.718 INFO:teuthology.orchestra.run.smithi124.stderr:sys 0m1.462s 2024-01-15T18:19:31.152 INFO:teuthology.orchestra.run.smithi026.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-15T18:19:31.436 INFO:teuthology.orchestra.run.smithi026.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-15T18:19:31.578 INFO:teuthology.orchestra.run.smithi026.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-15T18:19:31.834 INFO:teuthology.orchestra.run.smithi026.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-15T18:19:32.111 INFO:teuthology.orchestra.run.smithi026.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-01-15T18:19:32.115 INFO:teuthology.orchestra.run.smithi026.stderr: 2024-01-15T18:19:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:real 0m4.969s 2024-01-15T18:19:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:user 0m24.508s 2024-01-15T18:19:32.115 INFO:teuthology.orchestra.run.smithi026.stderr:sys 0m4.254s 2024-01-15T18:19:32.116 INFO:tasks.ceph:Archiving logs... 2024-01-15T18:19:32.117 DEBUG:teuthology.misc:Transferring archived files from smithi026:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/remote/smithi026/log 2024-01-15T18:19:32.118 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-15T18:19:35.455 DEBUG:teuthology.misc:Transferring archived files from smithi124:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/remote/smithi124/log 2024-01-15T18:19:35.456 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-15T18:19:37.717 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-15T18:19:37.727 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-15T18:19:37.727 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-15T18:19:37.736 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-15T18:19:37.882 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-15T18:19:37.891 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-15T18:19:37.892 DEBUG:teuthology.orchestra.run.smithi026:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-15T18:19:37.900 DEBUG:teuthology.orchestra.run.smithi124:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-15T18:19:37.949 INFO:teuthology.orchestra.run.smithi026.stderr:bash: line 1: ntpq: command not found 2024-01-15T18:19:37.954 INFO:teuthology.orchestra.run.smithi124.stderr:bash: line 1: ntpq: command not found 2024-01-15T18:19:37.959 INFO:teuthology.orchestra.run.smithi026.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-15T18:19:37.959 INFO:teuthology.orchestra.run.smithi026.stdout:=============================================================================== 2024-01-15T18:19:37.959 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 113 +1737us[+1753us] +/- 79ms 2024-01-15T18:19:37.960 INFO:teuthology.orchestra.run.smithi026.stdout:^* hv02.front.sepia.ceph.com 3 7 377 42 -1673us[-1654us] +/- 75ms 2024-01-15T18:19:37.960 INFO:teuthology.orchestra.run.smithi026.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +824us[ +842us] +/- 90ms 2024-01-15T18:19:37.960 INFO:teuthology.orchestra.run.smithi026.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T18:19:37.964 INFO:teuthology.orchestra.run.smithi124.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-15T18:19:37.964 INFO:teuthology.orchestra.run.smithi124.stdout:=============================================================================== 2024-01-15T18:19:37.964 INFO:teuthology.orchestra.run.smithi124.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 +1527us[+1527us] +/- 80ms 2024-01-15T18:19:37.964 INFO:teuthology.orchestra.run.smithi124.stdout:^* hv02.front.sepia.ceph.com 3 7 377 108 -1824us[-1836us] +/- 74ms 2024-01-15T18:19:37.965 INFO:teuthology.orchestra.run.smithi124.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 +661us[ +661us] +/- 90ms 2024-01-15T18:19:37.965 INFO:teuthology.orchestra.run.smithi124.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-15T18:19:37.966 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-15T18:19:37.975 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-15T18:19:37.976 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-15T18:19:37.984 DEBUG:teuthology.orchestra.run.smithi026:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-15T18:19:38.256 DEBUG:teuthology.orchestra.run.smithi124:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-15T18:19:38.466 DEBUG:teuthology.orchestra.run.smithi026:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-15T18:19:38.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:19:38.578 DEBUG:teuthology.orchestra.run.smithi124:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-15T18:19:38.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:19:38.681 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-15T18:19:38.691 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-15T18:19:38.700 INFO:teuthology.task.internal:Duration was 2018.007768 seconds 2024-01-15T18:19:38.701 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-15T18:19:38.710 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-15T18:19:38.710 DEBUG:teuthology.orchestra.run.smithi026:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-15T18:19:38.718 DEBUG:teuthology.orchestra.run.smithi124:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-15T18:19:38.883 INFO:teuthology.orchestra.run.smithi124.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-15T18:19:38.890 INFO:teuthology.orchestra.run.smithi026.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-15T18:19:39.185 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-15T18:19:39.186 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi026.front.sepia.ceph.com 2024-01-15T18:19:39.186 DEBUG:teuthology.orchestra.run.smithi026:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-15T18:19:39.253 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi124.front.sepia.ceph.com 2024-01-15T18:19:39.253 DEBUG:teuthology.orchestra.run.smithi124:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-15T18:19:39.330 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-15T18:19:39.330 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-15T18:19:39.338 DEBUG:teuthology.orchestra.run.smithi124:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-15T18:19:39.435 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-15T18:19:39.435 DEBUG:teuthology.orchestra.run.smithi026:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-15T18:19:39.441 DEBUG:teuthology.orchestra.run.smithi124:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-15T18:19:39.663 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-15T18:19:39.674 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-15T18:19:39.674 DEBUG:teuthology.orchestra.run.smithi026:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-15T18:19:39.779 DEBUG:teuthology.orchestra.run.smithi124:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-15T18:19:39.873 INFO:teuthology.orchestra.run.smithi124.stderr:mv: cannot stat '/etc/sudoers.orig.teuthology': No such file or directory 2024-01-15T18:19:39.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:19:39.878 ERROR:teuthology.run_tasks:Manager failed: internal.sudo Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/task/internal/__init__.py", line 455, in sudo ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi124 with status 1: 'sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers' 2024-01-15T18:19:39.889 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-15T18:19:39.900 DEBUG:teuthology.orchestra.run.smithi026:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-15T18:19:39.916 DEBUG:teuthology.orchestra.run.smithi124:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-15T18:19:40.019 INFO:teuthology.orchestra.run.smithi026.stdout:kernel.core_pattern = core 2024-01-15T18:19:40.023 INFO:teuthology.orchestra.run.smithi124.stdout:kernel.core_pattern = core 2024-01-15T18:19:40.066 INFO:teuthology.orchestra.run.smithi124.stderr:find: ‘/home/ubuntu/cephtest/archive/coredump’: No such file or directory 2024-01-15T18:19:40.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-15T18:19:40.074 INFO:teuthology.orchestra.run.smithi124.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/archive/coredump': No such file or directory 2024-01-15T18:19:40.075 ERROR:teuthology.run_tasks:Manager failed: internal.coredump Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/task/internal/__init__.py", line 488, in coredump yield File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/task/internal/__init__.py", line 455, in sudo ctx.cluster.run( File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/cluster.py", line 85, in run procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/cluster.py", line 85, in procs = [remote.run(**kwargs, wait=_wait) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi124 with status 1: 'sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/task/internal/__init__.py", line 491, in coredump run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi124 with status 1: "sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump" 2024-01-15T18:19:40.087 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-15T18:19:40.097 INFO:teuthology.task.internal:Transferring archived files... 2024-01-15T18:19:40.098 DEBUG:teuthology.misc:Transferring archived files from smithi026:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/remote/smithi026 2024-01-15T18:19:40.098 DEBUG:teuthology.orchestra.run.smithi026:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-15T18:19:40.208 DEBUG:teuthology.misc:Transferring archived files from smithi124:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100/remote/smithi124 2024-01-15T18:19:40.208 DEBUG:teuthology.orchestra.run.smithi124:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-15T18:19:40.315 INFO:teuthology.task.internal:Removing archive directory... 2024-01-15T18:19:40.315 DEBUG:teuthology.orchestra.run.smithi026:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-15T18:19:40.323 DEBUG:teuthology.orchestra.run.smithi124:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-15T18:19:40.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-15T18:19:40.391 INFO:teuthology.task.internal:Not uploading archives. 2024-01-15T18:19:40.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-15T18:19:40.434 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-15T18:19:40.434 DEBUG:teuthology.orchestra.run.smithi026:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-15T18:19:40.442 DEBUG:teuthology.orchestra.run.smithi124:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-15T18:19:40.492 INFO:teuthology.orchestra.run.smithi026.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 15 18:19 /home/ubuntu/cephtest 2024-01-15T18:19:40.499 INFO:teuthology.orchestra.run.smithi124.stdout: 394453 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 15 18:19 /home/ubuntu/cephtest 2024-01-15T18:19:40.502 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-15T18:19:40.514 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-15T18:19:40.525 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi124 with status 1: 'sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers' 2024-01-15T18:19:40.525 INFO:teuthology.nuke:Checking targets against current locks 2024-01-15T18:19:40.553 DEBUG:teuthology.nuke:shortname: smithi026 2024-01-15T18:19:40.554 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-15T18:19:40.578 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi026.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-15 17:35:49.483362', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKC4M7/gUHS0qgi/R2ONHIyBfj0g7m7E7n2NR4YtUuMPa7fYJ+XHdPpjCv5mpg7wZvsKv3pKjLthZdgU6MmnXW4='} 2024-01-15T18:19:40.582 DEBUG:teuthology.nuke:shortname: smithi124 2024-01-15T18:19:40.582 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-15T18:19:40.602 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi124.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-01-15_17:35:13-fs:volumes-rishabh-clone-stats-testing-default-smithi/7517100', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-15 17:35:49.481638', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMnT11QoYwAOugPZL1E5SnA/BOcm9WyIhGutN3SBvKT0jssJcwBm3ezEJRkCO7YICWbWBHF3rH5E87M7LpTyhsQ='} 2024-01-15T18:19:40.635 INFO:teuthology.orchestra.console.smithi026:Power off 2024-01-15T18:19:40.635 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-15T18:19:40.659 DEBUG:teuthology.orchestra.console.smithi026:power off output: Chassis Power Control: Down/Off 2024-01-15T18:19:40.660 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:19:40.676 INFO:teuthology.orchestra.console.smithi124:Power off 2024-01-15T18:19:40.676 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-15T18:19:40.686 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-01-15T18:19:40.697 DEBUG:teuthology.orchestra.console.smithi124:power off output: Chassis Power Control: Down/Off 2024-01-15T18:19:40.697 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:19:40.719 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2024-01-15T18:19:44.687 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:19:44.720 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:19:44.799 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-01-15T18:19:44.830 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is on 2024-01-15T18:19:48.801 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:19:48.831 DEBUG:teuthology.orchestra.console.smithi124:pexpect command: ipmitool -H smithi124.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:19:48.912 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is on 2024-01-15T18:19:48.940 DEBUG:teuthology.orchestra.console.smithi124:check power output: Chassis Power is off 2024-01-15T18:19:49.041 INFO:teuthology.orchestra.console.smithi124:Power off completed 2024-01-15T18:19:52.914 DEBUG:teuthology.orchestra.console.smithi026:pexpect command: ipmitool -H smithi026.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-15T18:19:53.025 DEBUG:teuthology.orchestra.console.smithi026:check power output: Chassis Power is off 2024-01-15T18:19:53.125 INFO:teuthology.orchestra.console.smithi026:Power off completed 2024-01-15T18:19:53.230 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone-progress}} duration: 2018.0077679157257 failure_reason: 'Test failure: test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress), test_when_3_clones_are_cancelled (tasks.cephfs.test_volumes.TestCloneProgress)' flavor: default mon.a-kernel-sha1: 2668949e0971849893a3fb7f177ce9032fc65780 mon.b-kernel-sha1: 2668949e0971849893a3fb7f177ce9032fc65780 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=51c3b4f9295244b5a5dad8c6e91fda27 status: fail success: false 2024-01-15T18:19:53.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-15T18:19:53.310 INFO:teuthology.run:FAIL